From greg@cosc.canterbury.ac.nz Wed May 1 03:13:13 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Wed, 01 May 2002 14:13:13 +1200 (NZST) Subject: [Python-Dev] Making None a keyword In-Reply-To: <200204301250.g3UCoGW16679@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200205010213.OAA02719@s454.cosc.canterbury.ac.nz> Guido: > I don't think I *want* people to use None as an attribute name (or in > any other context). Okay, everything's fine, then. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tim.one@comcast.net Wed May 1 03:28:36 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 30 Apr 2002 22:28:36 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: Message-ID: [Martin v. Loewis] > ... > However, the visit function could record this result. For example, for > move_root_reachable: I'm afraid it needs more knowledge than this, although it may be surmountable. A problem is that, e.g., tuples get tracked by GC immediately after they're initialized in PyTuple_New(), and they're initialized to all NULL slots. So if gc happens to occur before a tuple T is filled in "for real", traversing T isn't going to call visit_move (because T's slots are still NULL). This would cause gc to untrack T, even though T's creator may eventually get around to making T part of a cycle. > struct m_r_r{ > PyGC_Head *reachable; > int immutable_nonnested; > }; > > static int > visit_move(PyObject *op, struct m_r_r *tolist) > { > if(PyObject_IS_GC(op)){ > if(GC_is_tracked(op)) > m_r_r->tolist.immutable_nonnested = 0; I assume this was meant to be tolist->immutable_nonnested = 0. > ... > } > return 0; > } > > static void > move_root_reachable(PyGC_Head *reachable) > { > struct m_r_r my_m_r_r; > my_m_r_r.reachable = reachable; m_r_r.reachable isn't referenced elsewhere. > for (; gc != reachable; gc=gc->gc.gc_next) { > /* careful, reachable list is growing here */ > PyObject *op = FROM_GC(gc); > traverse = op->ob_type->tp_traverse; > my_m_r_r.immutable_nonnested = 1; > (void) traverse(op, > (visitproc)visit_move, > (void *)reachable); > if(my_m_r_r.immutable_nonnested > && op->ob_type->tp_immutable(op)){ What the heck is tp_immutable()? Are you implicitly assuming the existence of a new typeslot function? I'd be happy to special-case the snot out of tuples, and leave it at that. For that purpose, it may be enough just to see whether a thing *is* a tuple, and all its slots are non-NULL and filled with objects of the popular scalar immutable types (strings and numbers). Hmm! I bet we could go a long away leaving gc out of this entirely: BUILD_TUPLE in ceval.c could check the types of the objects it stuffs into a new tuple, and untrack it immediately then if it were safe to do so. And tupleconcat() could mindlessly inherit trackedness from the logical "or" of its inputs' trackedness. Would that get 80% of the potential benefit with 5% of the potential work? From tim.one@comcast.net Wed May 1 03:44:11 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 30 Apr 2002 22:44:11 -0400 Subject: [Python-Dev] millions of tuples In-Reply-To: <01bb01c1f082$63f50020$cc058dd8@avatartech.com> Message-ID: [Ian Kjos] > ... > Why not we change the semantics of visit() in this case to provide the > required information? There is no function named "visit", and the semantics of tp_traverse are etched in stone because extension modules we don't control already implement its long-advertised interface. No offense intended, but if you want to help with this you should really learn what the Python implementation does first; random ideas (which may or may not apply in some other implementation) are a distraction on Python-Dev (this list pretends <0.9 wink> to be more focused than c.l.py). From tim.one@comcast.net Wed May 1 04:05:50 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 30 Apr 2002 23:05:50 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: Message-ID: [Andrew MacIntyre] > OS/2 EMX (2.2.1 without pymalloc) > justpush 0.59 > justzip 8.85 > > OS/2 EMX (recent CVS with pymalloc) > justpush 0.54 > justzip 8.81 > > FreeBSD 4.4 (2.1.1 w/o pymalloc) > justpush 89.72 > justzip 110.41 > > FreeBSD 4.4 (recent CVS with pymalloc) > justpush 19.21 > justzip 46.32 > > The FreeBSD box is more mature hardware (P5-166). By "more mature" in this context I assume you mean more "obsolete" than "better developed". > I'm surprised at the difference in the 2 sets of results on it. AFAIK, > the compiler version and switches are identical for the two interpreters > (the 2.1.1 is from the binary package on the FreeBSD 4.4 CDs). justpush() primarily tests realloc() speed, and pymalloc isn't (yet) involved in managing list-gut memory. So I expect this has much more to do with the libc(s) they're using than with the compiler(s). From tim.one@comcast.net Wed May 1 06:19:09 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 01 May 2002 01:19:09 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: Message-ID: [Tim] > ... > Hmm! I bet we could go a long away leaving gc out of this entirely: > BUILD_TUPLE in ceval.c could check the types of the objects it > stuffs into a new tuple, and untrack it immediately then if it were safe > to do so. And tupleconcat() could mindlessly inherit trackedness from > the logical "or" of its inputs' trackedness. Would that get 80% of the > potential benefit with 5% of the potential work? This looks promising, but needs more work. First off, BUILD_TUPLE gets surprised by the empty tuple: after untracking it once, it gets back the same empty tuple later, and blows up when trying to untrack it again. So I had to break into the GC abstraction to avoid untracking when something was already untracked. That test is a heartbreaker, since it's only the empty tuple that's a potential problem there. Probably better to have tupleobject.c do something special. After fiddling builtin_zip to untrack the tuples it creates when that's safe (untrack is safe == for each slot s in the tuple, either s is not of a gcable type, or s is of a gcable type but s is already untracked), the justzip() test showed a wonderful surprise: it took *forever*! This is cute: _PyObject_GC_UNTRACK() doesn't decrease gc's "excess allocation" count, but tuples keep getting created. After "the list" makes it into gen2, the excess allocation count just keeps growing, and collect_generations() keeps getting called. But there's nothing *in* gen0 or gen1 anymore (the tuples get untracked), and collect_generations() neither calls collect(), nor sets allocations back to 0 itself, when a gen0 or gen1 collection occurs and the gen0 & gen1 lists are empty. The upshoot is that "allocated" is (almost) always greater than threshold0, so collect_generations() gets called *every* time a tuple gets allocated. Every 10th tuple allocation a useless null gen1 collection gets done then, and about about every 100th a gen2 collection gets done. The tuples aren't getting tracked anymore, but the list containing them is, and it crawls over all the list slots each time a gen2 collection gets done (and that happens about 700x more frequently than it's supposed to). So that turned out to be quadratic-time with extreme prejudice . Anyway, setting 'allocated' back to 0 at the end of collect_generations cured that, and untracking the tuples did do some good: the runtime of justzip() fell by more than a factor of 2. The gen2 traversals of "the list" still slowed it down considerably, though (over what happens when gc is disabled). So, if we want to pursue this, perhaps what gc really wants to count is the excess of gc tracks over gc untracks. Right now they're strongly correlated with gc allocations and deallocations, respectively. But if we started untracking "safe" tuples, that connection would be broken. If we did exempt "safe tuples" from gc, and counted the excess of tracks over untracks, gc would never trigger in justzip(). So that's the attraction. I think that much would be principled, since I view the "excess" statistic as trying to guess how many objects are sitting in gen0, triggering a round of cycle gc when gen0 gets big enough; track-untrack is a more direct measure of gen0's size than allocated-deallocated. From skip@pobox.com Wed May 1 07:10:00 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 1 May 2002 01:10:00 -0500 Subject: [Python-Dev] Unicode howto in the works - feedback appreciated Message-ID: <15567.34488.180301.776312@12-248-41-177.client.attbi.com> I began working on a Unicode HOWTO a few weeks ago, got a little ways o= n it, then ignored it until this morning. I added a little bit more to it th= en decided I should get some feedback. You can view it at http://www.musi-cal.com/~skip/unicode/ Pick your favorite format. This document is written from the perspective of a dumb American progra= mmer who has ignored Unicode for too long, not a savvy international program= mer who's had to consider multiple character sets for awhile. Consequently= , the content is going to be a lot different than if Fredrik, Martin or Marc-= Andr=E9 wrote it. I'm mostly concerned with recovering from previous bad decis= ions or dealing with implicit encoding information. Consider it a start. Feedback welcome. Skip From niemeyer@conectiva.com Wed May 1 08:19:49 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Wed, 1 May 2002 04:19:49 -0300 Subject: [Python-Dev] Unicode howto in the works - feedback appreciated In-Reply-To: <15567.34488.180301.776312@12-248-41-177.client.attbi.com> References: <15567.34488.180301.776312@12-248-41-177.client.attbi.com> Message-ID: <20020501041949.B10944@ibook.distro.conectiva> Hi Skip! > I began working on a Unicode HOWTO a few weeks ago, got a little ways on it, > then ignored it until this morning. I added a little bit more to it then > decided I should get some feedback. You can view it at [...] > Consider it a start. Feedback welcome. Thanks for the initiative! Something which should probably be added is about this issue: >>> s = "Marc-André".decode('iso-8859-1') >>> print s Traceback (most recent call last): File "", line 1, in ? UnicodeError: ASCII encoding error: ordinal not in range(128) I'm not sure if this is a common issue with other languages. But at least here (brazilian portuguese), this is a little bit inconvenient. Fortunately, fixing this is easy, since site.py was nicely prepared for this (thanks ;-): if 0: # Enable to support locale aware default string encodings. import locale loc = locale.getdefaultlocale() if loc[1]: encoding = loc[1] Enabling it should be enough: >>> s = "Marc-André".decode('iso-8859-1') >>> print s Marc-André (Marc-André, sorry for taking your name as an example ;-) -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From stephen@xemacs.org Wed May 1 08:52:13 2002 From: stephen@xemacs.org (Stephen J. Turnbull) Date: 01 May 2002 16:52:13 +0900 Subject: [Python-Dev] Unicode howto in the works - feedback appreciated In-Reply-To: <15567.34488.180301.776312@12-248-41-177.client.attbi.com> References: <15567.34488.180301.776312@12-248-41-177.client.attbi.com> Message-ID: <87r8kwxq36.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Skip" == Skip Montanaro writes: Skip> I began working on a Unicode HOWTO a few weeks ago, got a Skip> little ways on it, then ignored it until this morning. I Skip> added a little bit more to it then decided I should get some Skip> feedback. You can view it at Skip> http://www.musi-cal.com/~skip/unicode/ Thanks! A few comments. Overall, I like this intro. Technically it's horrible but I think it will hit your target audience where they live. [What Is Unicode?] 1. Characters are "atomic units of text" that have properties. Since they're atoms, we represent them by integers in computer programs. Among the properties are their glyphs (graphical representation), classes (alpha, num, whitespace, etc), and so on. It is a bad idea to identify characters with their glyphs. 2. Alphabets are abstract sets of characters. Coded character sets map characters to integer representations. "Encoding" is a reasonable synonym for "coded character set". Avoid "charset" except when talking about the charset parameter of Content-Type. 3. Typo in last sentence "I will suggest that YOU should use UTF-8." [Why UTF-8?] 1. Most programming languages are restricted to ASCII, except perhaps for user-defined identifiers. This means that programming tools need only be 8-bit clean to handle UTF without corruption.[1] 2. Space efficiency is _not_ an advantage of UTF-8 vs. UTF-16. ASCII and most Western European languages, yes. Greek, Hebrew, Arabic or Russian will be nearly a wash (whitespace, punctuation, and numerals give you what savings you're gonna get), and everybody east of Eden takes a 50% hit. The real tradeoff is "string == array of fixed-width object" semantics[2] vs upward compatibility from ASCII for languages where most tokens contain only ASCII. [Email] 1. If you don't get a Content-Type charset parameter, you _must_ assume US-ASCII. [Mildly Corrupt Data] 1. You can expect people to develop libraries for this kind of thing, but they are unlikely to be distributed. Suggest that newbies ask around. Footnotes: [1] This isn't quite true; consider the Lisp ?A notation for character literals. A naive byte-oriented parser will pick up only the leading byte of a non-ASCII UTF-8 character, and probably choke fatally on the trailing bytes. But Python, C, Java, et al don't have such literals---tokens with delimiters that are ASCII characters are safe, both strings and identifiers. You can ignore this issue. [2] Which UTF-16 actually doesn't give you! Grrr. -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN My nostalgia for Icon makes me forget about any of the bad things. I don't have much nostalgia for Perl, so its faults I remember. Scott Gilbert c.l.py From fredrik@pythonware.com Wed May 1 09:14:43 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Wed, 1 May 2002 10:14:43 +0200 Subject: [Python-Dev] Unicode howto in the works - feedback appreciated References: <15567.34488.180301.776312@12-248-41-177.client.attbi.com> <20020501041949.B10944@ibook.distro.conectiva> Message-ID: <002b01c1f0e8$426e0530$ced241d5@hagrid> Gustavo Niemeyer wrote: > I'm not sure if this is a common issue with other languages. But at > least here (brazilian portuguese), this is a little bit inconvenient. > Fortunately, fixing this is easy, since site.py was nicely prepared > for this (thanks ;-): > > if 0: > # Enable to support locale aware default string encodings. > import locale > loc = locale.getdefaultlocale() > if loc[1]: > encoding = loc[1] > > Enabling it should be enough that's a really great way to write non-portable code, of course. please keep your encodings explicit. (if you're printing lots of stuff, use an encoded stream) From simon@netthink.co.uk Wed May 1 09:23:51 2002 From: simon@netthink.co.uk (Simon Cozens) Date: Wed, 1 May 2002 09:23:51 +0100 Subject: [Python-Dev] Unicode howto in the works - feedback appreciated In-Reply-To: <15567.34488.180301.776312@12-248-41-177.client.attbi.com> References: <15567.34488.180301.776312@12-248-41-177.client.attbi.com> Message-ID: <20020501082351.GA6903@netthink.co.uk> Skip Montanaro: > http://www.musi-cal.com/~skip/unicode/ For comparison, http://www.netthink.co.uk/downloads/unicode.pdf Feel free to steal ideas if you find it useful. -- It's 106 miles from Birmingham, we've got an eighth of a tank of gas, half a pack of Dorritos, it's dusk, and we're wearing contacts. - Malcolm Ray From mal@lemburg.com Wed May 1 10:06:50 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 01 May 2002 11:06:50 +0200 Subject: [Python-Dev] Unicode howto in the works - feedback appreciated References: <15567.34488.180301.776312@12-248-41-177.client.attbi.com> <87r8kwxq36.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <3CCFB02A.95EDA2E3@lemburg.com> "Stephen J. Turnbull" wrote: > > [What Is Unicode?] > > 1. Characters are "atomic units of text" that have properties. Since > they're atoms, we represent them by integers in computer programs. > Among the properties are their glyphs (graphical representation), > classes (alpha, num, whitespace, etc), and so on. It is a bad > idea to identify characters with their glyphs. > > 2. Alphabets are abstract sets of characters. Coded character sets > map characters to integer representations. "Encoding" is a > reasonable synonym for "coded character set". Avoid "charset" > except when talking about the charset parameter of Content-Type. > > 3. Typo in last sentence "I will suggest that YOU should use UTF-8." You might also want to grab some ideas from my "Python and Unicode" presentation I gave at Bordeaux last year: http://www.egenix.com/files/python/Unicode-Talk.pdf This also explains the various terms used in Unicode space and how they relate to Python. > [Email] > > 1. If you don't get a Content-Type charset parameter, you _must_ assume > US-ASCII. [WWW] 1. If you don't get a Content-Type charset parameter in an HTTP request, you _must_ assume Latin-1. [Console Input] I'd suggest to change the order of encodings (e.g. putting Latin-1 near the end isn't a good idea). Also, the fact that decoding works doesn't necessarily mean that the input did in fact use that encoding. A more appropriate way would be to try to reencode the decoded data in the given encoding since that is likely to fail for e.g. CP-1252 vs. Latin-1 if people use accented characters. If you're more into guessing an encoding, you should probably use an entropy approach: http://www.familie-holtwick.de/python/ > [Mildly Corrupt Data] Same comment here: you have to test round-trips, not just whether decoding fails. (Please note that not all codecs are round-trip safe -- see test_unicode.py for a list of ones that are) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From niemeyer@conectiva.com Wed May 1 10:09:00 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Wed, 1 May 2002 06:09:00 -0300 Subject: [Python-Dev] Unicode howto in the works - feedback appreciated In-Reply-To: <002b01c1f0e8$426e0530$ced241d5@hagrid> References: <15567.34488.180301.776312@12-248-41-177.client.attbi.com> <20020501041949.B10944@ibook.distro.conectiva> <002b01c1f0e8$426e0530$ced241d5@hagrid> Message-ID: <20020501060900.A918@ibook.distro.conectiva> Hi Fred! > that's a really great way to write non-portable code, of > course. > > please keep your encodings explicit. Oops.. sorry then. I must confess I'm a unicode newbie. Could you please help me understanding some issues about this? Will "print s.encode('iso-8859-1')" work for everybody? If not, how is it better than doing it implicitly in site.py? Why isn't "iso-8859-1" used as the default encoding instead of "ascii", since it would help a broader number of langugages, and is the default in operating systems like Linux. Maybe it would cause harm for other encodings in some way? Thanks a lot! -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From paul@prescod.net Wed May 1 10:46:13 2002 From: paul@prescod.net (Paul Prescod) Date: Wed, 01 May 2002 02:46:13 -0700 Subject: [Python-Dev] Unicode howto in the works - feedback appreciated References: <15567.34488.180301.776312@12-248-41-177.client.attbi.com> <20020501041949.B10944@ibook.distro.conectiva> <002b01c1f0e8$426e0530$ced241d5@hagrid> <20020501060900.A918@ibook.distro.conectiva> Message-ID: <3CCFB965.56D651F4@prescod.net> Gustavo Niemeyer wrote: > >... > > Why isn't "iso-8859-1" used as the default encoding instead of "ascii", > since it would help a broader number of langugages, and is the default > in operating systems like Linux. Maybe it would cause harm for other > encodings in some way? Latin-1 is just one encoding and it would likely not be the "right guess" for a Russian programmer for example. There is some basis for treating Latin-1 as the "default encoding" in the Unicode specification but everytime that flamewar erupts, Guido decides that it is better to force everyone to be explicit about what they mean, even if it does result in a much, much higher incidence of that annoying ASCII error message. Paul Prescod From guido@python.org Wed May 1 13:06:37 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 01 May 2002 08:06:37 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: Your message of "Tue, 30 Apr 2002 22:28:36 EDT." References: Message-ID: <200205011206.g41C6bx20263@pcp742651pcs.reston01.va.comcast.net> > Hmm! I bet we could go a long away leaving gc out of this entirely: > BUILD_TUPLE in ceval.c could check the types of the objects it > stuffs into a new tuple, and untrack it immediately then if it were > safe to do so. But this still takes time, and most tuples never make it into GC, so it's wasted time, isn't it? I'd put such code in tuplevisit (assuming it's safe to call GC_UnTrack there). > And tupleconcat() could mindlessly inherit trackedness from the > logical "or" of its inputs' trackedness. Would that get 80% of the > potential benefit with 5% of the potential work? Too sophisticated already. ;-) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed May 1 14:16:08 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 01 May 2002 09:16:08 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: Your message of "Tue, 30 Apr 2002 15:40:10 +1200." <200204300340.PAA02565@s454.cosc.canterbury.ac.nz> References: <200204300340.PAA02565@s454.cosc.canterbury.ac.nz> Message-ID: <200205011316.g41DG8x20411@pcp742651pcs.reston01.va.comcast.net> [Barry] > > I'd much rather see a patch that just changed zip() to be an iterator > > instead of adding an iterzip(). [Greg E] > Wouldn't it be better as a lazy sequence a la xrange() > that produces an iterator, rather than being an > iterator itself? No, lazy sequences are a thing from the past, trying to do what iterators are for. (And yes, I know that a lazy sequence has different semantics.) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed May 1 14:32:08 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 01 May 2002 09:32:08 -0400 Subject: [Python-Dev] Unicode howto in the works - feedback appreciated In-Reply-To: Your message of "Wed, 01 May 2002 01:10:00 CDT." <15567.34488.180301.776312@12-248-41-177.client.attbi.com> References: <15567.34488.180301.776312@12-248-41-177.client.attbi.com> Message-ID: <200205011332.g41DWDb20500@pcp742651pcs.reston01.va.comcast.net> > http://www.musi-cal.com/~skip/unicode/ Cool. Maybe this should be added to the standard docs, or at least to Andrew Kuchling's how-to collection? --Guido van Rossum (home page: http://www.python.org/~guido/) From nas@python.ca Wed May 1 15:31:00 2002 From: nas@python.ca (Neil Schemenauer) Date: Wed, 1 May 2002 07:31:00 -0700 Subject: [Python-Dev] iterzip() In-Reply-To: <200205011206.g41C6bx20263@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Wed, May 01, 2002 at 08:06:37AM -0400 References: <200205011206.g41C6bx20263@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020501073100.B27024@glacier.arctrix.com> Guido van Rossum wrote: > But this still takes time, and most tuples never make it into GC, so > it's wasted time, isn't it? I agree. The idea I though of last night was to have the GC special case certain objects. It could know about tuples. After collecting the first generation any surviving tuples that don't refer to objects with the GC flag could be untracked. We might even generalize it by untracking objects that don't have a tp_clear method and don't refer to GC objects. BTW, I also looked into the train algorithm last night. It's cool because it never examines the entire set of objects (unless they all happen to be a big cyclic garbage struture). Unfortunately I don't think it can be adapted for Python. > I'd put such code in tuplevisit (assuming it's safe to call GC_UnTrack > there). That would be slick. I don't know if it's safe without looking at the code but it could probably be made so. Neil From akuchlin@mems-exchange.org Wed May 1 15:33:51 2002 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Wed, 1 May 2002 10:33:51 -0400 Subject: [Python-Dev] Unicode howto in the works In-Reply-To: <200205011332.g41DWDb20500@pcp742651pcs.reston01.va.comcast.net> References: <15567.34488.180301.776312@12-248-41-177.client.attbi.com> <200205011332.g41DWDb20500@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020501143351.GB31688@ute.mems-exchange.org> On Wed, May 01, 2002 at 09:32:08AM -0400, Guido van Rossum wrote: >Cool. Maybe this should be added to the standard docs, or at least to >Andrew Kuchling's how-to collection? I'd like to make the py-howto collection evaporate, eventually; the documents would get more proofreading and visibility if they were in the main CVS tree, though perhaps in nondist/, but I haven't yet talked with Fred about this. --amk From guido@python.org Wed May 1 15:52:45 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 01 May 2002 10:52:45 -0400 Subject: [Python-Dev] Unicode howto in the works In-Reply-To: Your message of "Wed, 01 May 2002 10:33:51 EDT." <20020501143351.GB31688@ute.mems-exchange.org> References: <15567.34488.180301.776312@12-248-41-177.client.attbi.com> <200205011332.g41DWDb20500@pcp742651pcs.reston01.va.comcast.net> <20020501143351.GB31688@ute.mems-exchange.org> Message-ID: <200205011452.g41Eqjd08379@odiug.zope.com> > I'd like to make the py-howto collection evaporate, eventually; the > documents would get more proofreading and visibility if they were in > the main CVS tree, though perhaps in nondist/, but I haven't yet > talked with Fred about this. Sounds good to me. Put everything on python.org, make it one happy family of docs. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Wed May 1 16:44:10 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 01 May 2002 11:44:10 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: <200205011206.g41C6bx20263@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Tim] > Hmm! I bet we could go a long away leaving gc out of this entirely: > ... [Guido] > But this still takes time, and most tuples never make it into GC, so > it's wasted time, isn't it? Absolutely. I'm trying to understand the true causes here by fiddling various things. It turned out that "useless" tracing of tuples accounted for more than half the gc burden in this test, but not much more than half (the rest was due to "useless" repeated tracing of the list containing all the tuples). > I'd put such code in tuplevisit (assuming it's safe to call GC_UnTrack > there). I'm pretty sure that would lead to instant segfaults, but the gc loops calling tp_traverse could be fiddled to save a local pointer to "the next guy" before invoking tp_traverse. I'm not sure that's the best place to put it. From tim.one@comcast.net Wed May 1 16:54:33 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 01 May 2002 11:54:33 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: <20020501073100.B27024@glacier.arctrix.com> Message-ID: [Neil Schemenauer] > ... > The idea I though of last night was to have the GC special case certain > objects. It could know about tuples. After collecting the first > generation any surviving tuples that don't refer to objects with the GC > flag could be untracked. Just butting in to be obnoxious about that a NULL slot is a disqualifier for untracking too. > We might even generalize it by untracking objects that don't have a > tp_clear method and don't refer to GC objects. How do we know which other objects they refer to? How do we know that the set of objects they refer to will never change? And do we have a reason to believe that any type other than tuples makes a lick of difference ? > BTW, I also looked into the train algorithm last night. It's cool > because it never examines the entire set of objects (unless they all > happen to be a big cyclic garbage struture). Unfortunately I don't > think it can be adapted for Python. Try adapting Python to it . [Guido] >> I'd put such code in tuplevisit (assuming it's safe to call GC_UnTrack >> there). > That would be slick. I don't know if it's safe without looking at the > code but it could probably be made so. I believe it would be easy, requiring minor rewrites of all the gc loops that invoke tp_traverse (the gc->gc.gc_next field they rely on to get "the next guy" in the list could go NULL on them across tp_traverse calls). From barry@zope.com Wed May 1 18:59:12 2002 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 1 May 2002 13:59:12 -0400 Subject: [Python-Dev] Unicode howto in the works - feedback appreciated References: <15567.34488.180301.776312@12-248-41-177.client.attbi.com> Message-ID: <15568.11504.911716.797669@anthem.wooz.org> [Email] A Stephen Turnbull says, if there's no charset, you must assume it's us-ascii. Here's an (untested) example using the email package: import email msg = email.message_from_string(...) if msg.get_main_type() == 'text': charset = msg.get_param('charset', 'us-ascii') else: charset = NotAppropriate -Barry From lkcl@samba-tng.org Wed May 1 20:14:14 2002 From: lkcl@samba-tng.org (Luke Kenneth Casson Leighton) Date: Wed, 1 May 2002 19:14:14 +0000 Subject: [Python-Dev] [bug?] UserLong.py - pow(5, UserLong(3), 26) fails Message-ID: <20020501191414.A29030@samba-tng.org> hiya people, been a while, i'm back onto python exploration for a serious monster exciting parallelisation project. as part of the evaluation, i have created UserLong.py, and am working my way through the list of all operations. i may have some mistakes and/or missing operators, and am trying to work this one out. at the bottom of the file are some inline debug tests. one of these - pow(int(5), UserLong(3), 26) - is failing: Script started on Wed May 1 19:06:30 2002 highfield:/usr/local/lib/python2.2# python UserLong.py int 3 long 3 5**int(y) 125 5**y rpow: (5,) 3 x^y%26 (3L, 26) 21L 5^y%26 Traceback (most recent call last): File "UserLong.py", line 153, in ? print "5^y%26", pow(5, y, 26) TypeError: unsupported operand type(s) for pow(): 'int', 'instance', 'int' highfield:/usr/local/lib/python2.2# Script done on Wed May 1 19:06:37 2002 now, if i read this correctly, what is happening is that in the [optimised] code, Objects/intobject.c, support for types other than int, float and long - via the macro CONVERT_TO_LONG - are simply not supported. this i believe can be demonstrated to be so because if you do pow(5, [], 26) you get the same error except replace 'instance' above with 'list'. now, could someone with a little more knowledge than i kindly evaluate, if they have the time, whether: - _is_ this happening because i have missed out a function in UserLong.py that i do not know the name of? or: - _is_ this due to a bug in intobject.c's int_pow() or: - _is_ this due to a bug in the implementation of the pow() "spam" function? - other? many thanks, luke -- ---------------------------------------------------------- this message is private, confidential, and is intented for the specified recipients only. if you received in error, altered, deleted, modified, destroyed or interfered with the contents of this message, in whole or in part, please inform the sender (that's me), immediately. if you, the recipient, reply to this message, and do not then receive a response, please consider your reply to have been lost or deliberately destroyed: i *always* acknowledge personal email received. please therefore take appropriate action and use appropriate protocols to ensure effective communication. thank you. From lkcl@samba-tng.org Wed May 1 20:16:11 2002 From: lkcl@samba-tng.org (Luke Kenneth Casson Leighton) Date: Wed, 1 May 2002 19:16:11 +0000 Subject: [Python-Dev] Re: [bug?] UserLong.py - pow(5, UserLong(3), 26) fails In-Reply-To: <20020501191414.A29030@samba-tng.org>; from lkcl@samba-tng.org on Wed, May 01, 2002 at 07:14:14PM +0000 References: <20020501191414.A29030@samba-tng.org> Message-ID: <20020501191611.B29030@samba-tng.org> """A more or less complete user-defined wrapper around long objects.""" class UserLong: def __init__(self, initlong=None): self.data = 0L if initlong is not None: # XXX should this accept an arbitrary sequence? if type(initlong) == type(self.data): self.data = initlong elif isinstance(initlong, UserLong): self.data = initlong.data else: self.data = long(initlong) def __repr__(self): return repr(self.data) def __lt__(self, other): return self.data < self.__cast(other) def __le__(self, other): return self.data <= self.__cast(other) def __eq__(self, other): return self.data == self.__cast(other) def __ne__(self, other): return self.data != self.__cast(other) def __gt__(self, other): return self.data > self.__cast(other) def __ge__(self, other): return self.data >= self.__cast(other) def __cast(self, other): if isinstance(other, UserLong): return other.data else: return other def __long__(self): return self.data def __int__(self): return int(self.data) def __cmp__(self, other): return cmp(self.data, self.__cast(other)) def __sub__(self, other): if isinstance(other, UserLong): return self.__class__(self.data - other.data) elif isinstance(other, type(self.data)): return self.__class__(self.data - other) else: return self.__class__(self.data - long(other)) def __add__(self, other): if isinstance(other, UserLong): return self.__class__(self.data + other.data) elif isinstance(other, type(self.data)): return self.__class__(self.data + other) else: return self.__class__(self.data + long(other)) def __rsub__(self, other): if isinstance(other, UserLong): return self.__class__(other.data - self.data) elif isinstance(other, type(self.data)): return self.__class__(other - self.data) else: return self.__class__(long(other) - self.data) def __radd__(self, other): if isinstance(other, UserLong): return self.__class__(other.data + self.data) elif isinstance(other, type(self.data)): return self.__class__(other + self.data) else: return self.__class__(long(other) + self.data) def __iadd__(self, other): if isinstance(other, UserLong): self.data += other.data elif isinstance(other, type(self.data)): self.data += other else: self.data += long(other) return self def __isub__(self, other): if isinstance(other, UserLong): self.data -= other.data elif isinstance(other, type(self.data)): self.data -= other else: self.data -= long(other) return self def __pow__(self, *args): print args if len(args) == 1: (other,) = args if isinstance(other, UserLong): return self.__class__(pow(self.data, other.data)) elif isinstance(other, type(self.data)): return self.__class__(pow(self.data, other)) else: return self.__class__(pow(self.data, long(other))) elif len(args) == 2: (other, other2) = args if isinstance(other, UserLong): x = other.data elif isinstance(other, type(self.data)): x = other else: x2 = long(other) if isinstance(other2, UserLong): x2 = other2.data elif isinstance(other2, type(self.data)): x2 = other2 else: x2 = long(other2) return self.__class__(pow(self.data, x, x2)) def __rpow__(self, *args): print "rpow:", args return self.data def __mod__(self, other): if isinstance(other, UserLong): return self.__class__(self.data % other.data) elif isinstance(other, type(self.data)): return self.__class__(self.data % other) else: return self.__class__(self.data % long(other)) def __rmod__(self, other): if isinstance(other, UserLong): return self.__class__(other.data % self.data) elif isinstance(other, type(self.data)): return self.__class__(other % self.data) else: return self.__class__(long(other) % self.data) def __divmod__(self, m): return self.__class__(divmod(self.data, m)) def __div__(self, n): return self.__class__(self.data/n) def __rdiv__(self, n): return self.__class__(n/self.data) def __mul__(self, n): return self.__class__(self.data*n) __rmul__ = __mul__ def __imul__(self, n): self.data *= n return self def __and__(self, n): return self.__class__(self.data&n) __rand__ = __and__ def __iand__(self, n): self.data &= n return self x = UserLong(5L) y = UserLong(3L) print "int", int(y) print "long", long(y) print "5**int(y)", 5 ** int(y) print "5**y", 5 ** y print "x^y%26", pow(x, y, 26) print "5^y%26", pow(5, y, 26) print "x^3%26", pow(x, 3, 26) print x % y print 5 % y print x % 3 print "x-y", x - y print "5-y", 5 - y print "x-3", x - 3 print "x/y", x / y print "5/y", 5 / y print "x/3", x / 3 From mwh@python.net Wed May 1 19:13:30 2002 From: mwh@python.net (Michael Hudson) Date: 01 May 2002 19:13:30 +0100 Subject: [Python-Dev] Built in objects supporting slices In-Reply-To: Paul Moore's message of "Sun, 28 Apr 2002 13:40:20 +0100" References: <3uqncusdqo1gs2tvvs7v878ki8ftsqbmt9@4ax.com> Message-ID: <2md6wfn3cl.fsf@starship.python.net> Paul Moore writes: > At the moment, built in objects such as lists don't support the slice > notaton l[a:b:c]. Would there be support for including this in Python > 2.3? If so, what would be the necessary procedure? Would this require a > PEP? Well, I wrote a patch for this ages and ages ago: http://www.python.org/sf/400998 It got rejected, mostly for lack of interest. > Alex Martelli has offered to do the code, if someone will champion the > work. I'm happy to do this, but I don't have commit privileges, so it > would require someone else to actually commit any changes. I presume > that the correct approach would be to submit the changes as a patch to > SourceForge? I'd have thought so. Cheers, M. -- ... with these conditions cam the realisation that ... nothing turned a perfectly normal healthy individual into a great political or military leader better than irreversible brain damage. -- The Hitch-Hikers Guide to the Galaxy, Episode 11 From nas@python.ca Wed May 1 19:31:05 2002 From: nas@python.ca (Neil Schemenauer) Date: Wed, 1 May 2002 11:31:05 -0700 Subject: [Python-Dev] iterzip() In-Reply-To: ; from tim.one@comcast.net on Wed, May 01, 2002 at 11:54:33AM -0400 References: <20020501073100.B27024@glacier.arctrix.com> Message-ID: <20020501113105.A27699@glacier.arctrix.com> Tim Peters wrote: > How do we know which other objects they refer to? We call tp_traverse on them. > How do we know that the > set of objects they refer to will never change? The assumption is that if the object doesn't have tp_clear then it's immutable. That's a little risky I guess. > And do we have a reason to believe that any type other than tuples > makes a lick of difference ? Probably not. [train algorithm] > Try adapting Python to it . Can't do it without dropping the reference counting and breaking every extension module (I least I can't figure out a way). Neil From mwh@python.net Wed May 1 19:33:51 2002 From: mwh@python.net (Michael Hudson) Date: 01 May 2002 19:33:51 +0100 Subject: [Python-Dev] [bug?] UserLong.py - pow(5, UserLong(3), 26) fails In-Reply-To: Luke Kenneth Casson Leighton's message of "Wed, 1 May 2002 19:14:14 +0000" References: <20020501191414.A29030@samba-tng.org> Message-ID: <2my9f3wwds.fsf@starship.python.net> Luke Kenneth Casson Leighton writes: > one of these - pow(int(5), UserLong(3), 26) - is failing: What's the point of UserLong now we have 2.2? > Script started on Wed May 1 19:06:30 2002 > highfield:/usr/local/lib/python2.2# python UserLong.py > int 3 > long 3 > 5**int(y) 125 > 5**y rpow: (5,) > 3 > x^y%26 (3L, 26) > 21L > 5^y%26 > Traceback (most recent call last): > File "UserLong.py", line 153, in ? > print "5^y%26", pow(5, y, 26) > TypeError: unsupported operand type(s) for pow(): 'int', 'instance', 'int' > highfield:/usr/local/lib/python2.2# > Script done on Wed May 1 19:06:37 2002 > > now, if i read this correctly, what is happening is that in > the [optimised] code, Objects/intobject.c, support for types > other than int, float and long - via the macro CONVERT_TO_LONG - > are simply not supported. > > this i believe can be demonstrated to be so because if you > do pow(5, [], 26) you get the same error except replace > 'instance' above with 'list'. > > now, could someone with a little more knowledge than i kindly > evaluate, if they have the time, whether: > > - _is_ this happening because i have missed out a function in > UserLong.py that i do not know the name of? > > or: > > - _is_ this due to a bug in intobject.c's int_pow() > > or: > > - _is_ this due to a bug in the implementation of the pow() > "spam" function? > > - other? Are you aware of this text in section 3.3.6 of the lang ref: Note that ternary pow() will not try calling __rpow__() (the coercion rules would become too complicated). so I'd say this is, if anything, a "documented limitation", though that might something of a stretch... Cheers, M. -- The meaning of "brunch" is as yet undefined. -- Simon Booth, ucam.chat From lkcl@samba-tng.org Wed May 1 20:42:26 2002 From: lkcl@samba-tng.org (Luke Kenneth Casson Leighton) Date: Wed, 1 May 2002 19:42:26 +0000 Subject: [Python-Dev] Re: [bug?] UserLong.py - pow(5, UserLong(3), 26) fails In-Reply-To: <20020501191414.A29030@samba-tng.org>; from lkcl@samba-tng.org on Wed, May 01, 2002 at 07:14:14PM +0000 References: <20020501191414.A29030@samba-tng.org> Message-ID: <20020501194226.A29206@samba-tng.org> On Wed, May 01, 2002 at 07:14:14PM +0000, Luke Kenneth Casson Leighton wrote: > as part of the evaluation, i have created UserLong.py, > and am working my way through the list of all operations. > one of these - pow(int(5), UserLong(3), 26) - is failing further preliminary analysis shows that the error message is being reported from Objects/abstract.c:ternary_op.c. without doing any debugging, i presume that PyNumber_Coerce and the macro NEW_STYLE_NUMBER are both failing to identify "class UserLong" as supporting numerical operations. now, there _may_ be some way to add an operator to class UserLong which identifies it as a "number" - i don't know what it is. anyone know? PyNumber_Coerce requires that the HOF table (PyObject*)w->ob_type->tp_as_number be != NULL in order for it to call tp_as_number->nb_coerce() and _that_ apparently will give number coercion. however, that means that _even_ if i define "__coerce__" in class UserLong, it won't get called because tp_as_number is an optimisation/speedup HOF table of functions, yes? so, the check in PyNumber_Coerce needs to also have "if (tp_as_number == NULL) lookup_function("__coerce__") and call it" am i right? l. -- ---------------------------------------------------------- this message is private, confidential, and is intented for the specified recipients only. if you received in error, altered, deleted, modified, destroyed or interfered with the contents of this message, in whole or in part, please inform the sender (that's me), immediately. if you, the recipient, reply to this message, and do not then receive a response, please consider your reply to have been lost or deliberately destroyed: i *always* acknowledge personal email received. please therefore take appropriate action and use appropriate protocols to ensure effective communication. thank you. From lkcl@samba-tng.org Wed May 1 20:49:59 2002 From: lkcl@samba-tng.org (Luke Kenneth Casson Leighton) Date: Wed, 1 May 2002 19:49:59 +0000 Subject: [Python-Dev] Re: [bug?] UserLong.py - pow(5, UserLong(3), 26) fails In-Reply-To: <20020501194226.A29206@samba-tng.org>; from lkcl@samba-tng.org on Wed, May 01, 2002 at 07:42:26PM +0000 References: <20020501191414.A29030@samba-tng.org> <20020501194226.A29206@samba-tng.org> Message-ID: <20020501194959.A29296@samba-tng.org> michael, the point of doing a UserLong.py is that it is an easy way to add in profiling and debug info and to evaluate alternative implementations of Long numbers. the ultimate aim is to find out if it is worth adding in support for Aspex Technology's massively parallel signal processor chip as a python co-processor. l. From mwh@python.net Wed May 1 19:56:15 2002 From: mwh@python.net (Michael Hudson) Date: 01 May 2002 19:56:15 +0100 Subject: [Python-Dev] Re: [bug?] UserLong.py - pow(5, UserLong(3), 26) fails In-Reply-To: Luke Kenneth Casson Leighton's message of "Wed, 1 May 2002 19:49:59 +0000" References: <20020501191414.A29030@samba-tng.org> <20020501194226.A29206@samba-tng.org> <20020501194959.A29296@samba-tng.org> Message-ID: <2msn5bwvcg.fsf@starship.python.net> Luke Kenneth Casson Leighton writes: > michael, > > the point of doing a UserLong.py is that it is an easy way > to add in profiling and debug info and to evaluate alternative > implementations of Long numbers. > > the ultimate aim is to find out if it is worth adding in > support for Aspex Technology's massively parallel signal > processor chip as a python co-processor. I see. Another idea: make UserLong a "new-style class". I'm not sure what this changes, but it changes "stuff" in this area... Cheers, M. -- C is not clean -- the language has _many_ gotchas and traps, and although its semantics are _simple_ in some sense, it is not any cleaner than the assembly-language design it is based on. -- Erik Naggum, comp.lang.lisp From tim.one@comcast.net Wed May 1 20:03:29 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 01 May 2002 15:03:29 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: <20020501113105.A27699@glacier.arctrix.com> Message-ID: [Tim] >> How do we know which other objects they refer to? [Neil Schemenauer] > We call tp_traverse on them. We just went thru this for tuples, right? PyTuple_New returns a tracked tuple T all of whose data slots are NULL. If tupletraverse(T, visit, arg) is called when T is in this state, tupletraverse will never call visit. And, AFAICT, whether or not the visit() function is called is the only information you can get out of calling tp_traverse. In which case, calling tp_traverse will tell you if a tuple *does* point to a gcable object, but tells you nothing about the tuple if visit doesn't get called, only that the tuple doesn't currently happen to point to anything interesting (but well may if you try again later). IOW, "immutable" isn't a truthful description of tuples at this level; tuples are all but guaranteed to mutate after they become tracked, and knowing when a tuple has undergone its final under-the-covers mutation requires exact knowledge of how tuples are layed out and used. s/tuple/xyzobject/g. >> How do we know that the set of objects they refer to will never change? > The assumption is that if the object doesn't have tp_clear then it's > immutable. That's a little risky I guess. Seems likely, yes. From martin@v.loewis.de Wed May 1 20:31:18 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 01 May 2002 21:31:18 +0200 Subject: [Python-Dev] iterzip() In-Reply-To: References: Message-ID: Tim Peters writes: > How do we know which other objects they refer to? How do we know that the > set of objects they refer to will never change? And do we have a reason to > believe that any type other than tuples makes a lick of difference ? I'd suggest a tp_is_immutable predicate. If a container is immutable and non-cyclic, it can be untracked. A container is non-cyclic if all contained objects are either non-gc, or untracked. Regards, Martin From martin@v.loewis.de Wed May 1 20:21:47 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 01 May 2002 21:21:47 +0200 Subject: [Python-Dev] Unicode howto in the works - feedback appreciated In-Reply-To: <20020501060900.A918@ibook.distro.conectiva> References: <15567.34488.180301.776312@12-248-41-177.client.attbi.com> <20020501041949.B10944@ibook.distro.conectiva> <002b01c1f0e8$426e0530$ced241d5@hagrid> <20020501060900.A918@ibook.distro.conectiva> Message-ID: Gustavo Niemeyer writes: > Will "print s.encode('iso-8859-1')" work for everybody? If not, how is > it better than doing it implicitly in site.py? It won't work for everybody: it will only work if the terminal's (*) encoding is latin-1. For printing to the terminal, using the locale's encoding is a good guess (**). That does not mean the default encoding should be changed - if you change it, it would apply also to other Unicode/byte string conversions, in places that doesn't have to do with the locale's encoding. That's why it is a bad idea to change the default encoding. Instead, there should be a convenient way to obtain the terminal's encoding. Unfortunately, this is impossible to implement in the general case. Regards, Martin (*) this assumes that print goes to the terminal. If it goes to some other stream, the definition of "will work" changes. (**) there are still cases where it won't work. E.g. in a Windows cmd.exe window, the OEM code page should be used, whereas the locale's encoding would be the ANSI code page. From tim.one@comcast.net Wed May 1 20:56:10 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 01 May 2002 15:56:10 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: Message-ID: [martin@v.loewis.de] > I'd suggest a tp_is_immutable predicate. If a container is immutable > and non-cyclic, it can be untracked. A container is non-cyclic if all > contained objects are either non-gc, or untracked. And none of the contained objects are NULL (else the scheme doesn't even work for tuples). How do you know which other objects it contains? Calling tp_traverse and seeing whether that ever calls the passed-in visit function doesn't work (see the already-repeated discussion about this for tuples). From aahz@pythoncraft.com Thu May 2 04:52:08 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 1 May 2002 23:52:08 -0400 Subject: [Python-Dev] testing... (ignore this) Message-ID: <20020502035208.GA17687@panix.com> Is python-dev dead or quiet? Does exim need a boot? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "I used to have a .sig but I found it impossible to please everyone..." --SFJ From andymac@bullseye.apana.org.au Wed May 1 23:49:06 2002 From: andymac@bullseye.apana.org.au (Andrew MacIntyre) Date: Thu, 2 May 2002 09:49:06 +1100 (edt) Subject: [Python-Dev] iterzip() In-Reply-To: Message-ID: On Tue, 30 Apr 2002, Tim Peters wrote: > [Andrew MacIntyre] {...} > > FreeBSD 4.4 (2.1.1 w/o pymalloc) > > justpush 89.72 > > justzip 110.41 > > > > FreeBSD 4.4 (recent CVS with pymalloc) > > justpush 19.21 > > justzip 46.32 > > > > The FreeBSD box is more mature hardware (P5-166). > > By "more mature" in this context I assume you mean more "obsolete" than > "better developed". Elderly but still reliable. > > I'm surprised at the difference in the 2 sets of results on it. AFAIK, > > the compiler version and switches are identical for the two interpreters > > (the 2.1.1 is from the binary package on the FreeBSD 4.4 CDs). > > justpush() primarily tests realloc() speed, and pymalloc isn't (yet) > involved in managing list-gut memory. So I expect this has much more to do > with the libc(s) they're using than with the compiler(s). I was only referring to the two FreeBSD runs, where the same libc is involved. -- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au | Snail: PO Box 370 andymac@pcug.org.au | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From skip@pobox.com Thu May 2 05:15:35 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 1 May 2002 23:15:35 -0500 Subject: [Python-Dev] Unicode objects more space efficient than plain strings? can that be? Message-ID: <15568.48487.339982.314930@12-248-41-177.client.attbi.com> I'm busy absorbing all the great feedback I got on the Unicode how-to. Thanks to all who've responded. As Aahz has said, the best way to lear= n something isn't to ask questions, it's to post an incorrect program (or= in this case, text). After reading Simon's Perl/Unicode course notes and Marc-Andr=E9's Python/Unicode EuroPython slides, I formed a simple, seemingly obvious,= hypothesis: When considering just ASCII data, plain Python strings should be mo= re space efficient than Unicode strings. I compared ps output for two interactive sessions. In the first, I exe= cuted this statement at the interpreter prompt: l =3D [u"abc%d"%i for i in xrange(1000000)] In the second I executed this similar statement: l =3D ["abc%d"%i for i in xrange(1000000)] Ps showed that the interpreter consumed 57MB or so of virtual memory fo= r the list of Unicode strings case, and a whopping 152MB for the list of plai= n strings case. Just to be sure I wasn't dreaming, I repeated the crude experiment. Same result. I then looked at the typedefs for Unicode an= d string objects. The sizes of the two structs are approximately the sam= e. There's certainly not a factor of three difference in the per-object overhead. I expect the raw Unicode buffer to refer to a chunk of memor= y that is roughly two times the size of the plain string version of the b= ytes because the internal representation is (I seem to recall from MAL's not= es) UCS2. It seemed the only thing that might be a problem was string interning, so based on the comment in stringobject.h about interning st= rings that "look like" Python identifiers, I tried one more time with strings= that didn't look like identifers (the automatic string interning would only happen for string literals anyway, right?): l =3D ["-bc%d"%i for i in xrange(1000000)] Same result. I ran these tests with a two-week old build from CVS. I just tried it = with a build from today using xrange(100000) and got similar, though obvious= ly smaller virtual memory sizes. I must be missing something obvious, but what is it? Something about pymalloc? Skip From tim.one@comcast.net Thu May 2 05:53:33 2002 From: tim.one@comcast.net (Tim Peters) Date: Thu, 02 May 2002 00:53:33 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: Message-ID: [Andrew MacIntyre] >>> FreeBSD 4.4 (2.1.1 w/o pymalloc) >>> justpush 89.72 >>> justzip 110.41 >>> >>> FreeBSD 4.4 (recent CVS with pymalloc) >>> justpush 19.21 >>> justzip 46.32 >>> ... >>> I'm surprised at the difference in the 2 sets of results on >>> it. AFAIK, the compiler version and switches are identical for >>> the two interpreters (the 2.1.1 is from the binary package on the >>> FreeBSD 4.4 CDs). [Tim] >> justpush() primarily tests realloc() speed, and pymalloc isn't (yet) >> involved in managing list-gut memory. So I expect this has much more >> to do with the libc(s) they're using than with the compiler(s). [Andrew] > I was only referring to the two FreeBSD runs, I know. > where the same libc is involved. That I didn't know (only the compiler was mentioned above). So you've got your own mystery. What are you going to do about it ? Calling the platform realloc is the only potentially expensive thing justpush() does, pymalloc isn't involved with list realloc, and your 2.1.1 spent 70 extra seconds doing *something*. From tim_one@email.msn.com Thu May 2 06:49:32 2002 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 2 May 2002 01:49:32 -0400 Subject: [Python-Dev] Re: iterzip() Message-ID: >>> FreeBSD 4.4 (2.1.1 w/o pymalloc) >>> justpush 89.72 >>> justzip 110.41 >>> >>> FreeBSD 4.4 (recent CVS with pymalloc) >>> justpush 19.21 >>> justzip 46.32 >>> [Tim] > ... > That I didn't know (only the compiler was mentioned above). So you've > got your own mystery. What are you going to do about it ? The good news: This isn't a mystery after all. Between 2.1 and 2.2, I changed list.append() to do mildly exponential overallocation instead of fixed-increment overallocation. "In theory" that makes the worst cases amortized linear time instead of quadratic time. Looks like FreeBSD's realloc is more sensitive to this change than most. the-bad-news-is-i-barely-remember-changing-it-ly y'rs - tim From Damien.Morton@acm.org Thu May 2 07:04:08 2002 From: Damien.Morton@acm.org (Damien Morton) Date: Thu, 2 May 2002 02:04:08 -0400 Subject: [Python-Dev] slice notation and RDF Message-ID: <007101c1f19f$2c44b940$56296c42@damien> This is a multi-part message in MIME format. ------=_NextPart_000_0072_01C1F17D.A5331940 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit I was reading TBLs presentation of webizing Python, and specifically his ideas about incorporating RDF into the syntax of Python. The recent thread on slice notation, and some of my experiences playing around implementing syntax for TBLs RDF-in-Python proposal got me thinking. Could the notation a:b:c:d:... be generalised, with slices becoming a kind of tuple. For backwards compatability, the first three elements of the tuple could be accessed using the start, stop, step attributes. Could the dictionary notation be related to slice notation, such that a dictionary is defined as being an object initialised by a list of slices. { 1:2, 3:4 } <--> dict([slice(1,2), slice(3,4)]) For RDF notation, you could then happily write: {1:2:3, 4:5:6, 7:8:9} --> dict([slice(1,2,3), slice(4,5,6), slice(7,8,9)]) For sets, you would write: {1, 2, 3} -> dict([slice(1), slice(2), slice(3)]} A slice on its own would be writen: (1:2:3) -> slice(1,2,3) A 1-element slice might be written similarily to a 1-element tuple: (1:) -> slice(1) ------=_NextPart_000_0072_01C1F17D.A5331940 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Message
I was = reading TBLs=20 presentation of webizing Python, and specifically his ideas about = incorporating=20 RDF into the syntax of Python.
 
The = recent thread on=20 slice notation, and some of my experiences playing around implementing = syntax=20 for TBLs RDF-in-Python proposal got me thinking.
 
Could = the=20 notation   a:b:c:d:... be generalised, with slices becoming a = kind of=20 tuple. For backwards compatability, the first three elements of the = tuple could=20 be accessed using the start, stop, step attributes.
 
Could = the dictionary=20 notation be related to slice notation, such that a dictionary is defined = as=20 being an object initialised by a list of = slices.
 
{ 1:2, = 3:4 }=20 <--> dict([slice(1,2), slice(3,4)])
 
For = RDF notation,=20 you could then happily write:
 
{1:2:3, 4:5:6,=20 7:8:9} --> dict([slice(1,2,3), slice(4,5,6),=20 slice(7,8,9)])
 
For = sets, you would=20 write:
 
{1, 2, = 3} ->=20 dict([slice(1), slice(2), slice(3)]}
 
A = slice on its own=20 would be writen:
 
(1:2:3)  ->=20 slice(1,2,3)
 
A = 1-element slice=20 might be written similarily to a 1-element tuple:
(1:) = ->=20 slice(1)
------=_NextPart_000_0072_01C1F17D.A5331940-- From aleax@aleax.it Thu May 2 07:39:26 2002 From: aleax@aleax.it (Alex Martelli) Date: Thu, 2 May 2002 08:39:26 +0200 Subject: [Python-Dev] iterable slices In-Reply-To: <007101c1f19f$2c44b940$56296c42@damien> References: <007101c1f19f$2c44b940$56296c42@damien> Message-ID: <02050208392603.28570@arthur> On Thursday 02 May 2002 08:04, Damien Morton wrote: ... > Could the notation a:b:c:d:... be generalised, with slices becoming a > kind of tuple. For backwards compatability, the first three elements of > the tuple could be accessed using the start, stop, step attributes. ... > {1, 2, 3} -> dict([slice(1), slice(2), slice(3)]} > > A slice on its own would be writen: > > (1:2:3) -> slice(1,2,3) > > A 1-element slice might be written similarily to a 1-element tuple: > (1:) -> slice(1) And presumably None could be omitted, e.g. (:1) as slice(None,1). Related but somewhat orthogonal to this idea -- I'm starting to think that slices should be iterable, with iter(slice(a,b,c)) yielding exactly the same numbers as iter(range(a,b,c)) or iter(xrange(a,b,c)). If any such abbreviated notation existed for slices, then "for i in (:6):" might work. Risks: perhaps error-prone ("for i in {:6}:", "for i in (6:):", etc, might be likely typos yielding unexpected behavior); no idea of what behavior would be expected of "for i in (a:b:c:d:e):". Alex From Damien.Morton@acm.org Thu May 2 08:05:01 2002 From: Damien.Morton@acm.org (Damien Morton) Date: Thu, 2 May 2002 03:05:01 -0400 Subject: [Python-Dev] RE: iterable slices In-Reply-To: <02050208392603.28570@arthur> Message-ID: <007601c1f1a7$ae197160$56296c42@damien> I hadnt thought of that, but its kind of elegant. for i in (a:b:c:d:e): Would probably be an error. Trying to make an iterator out of anything but a 2 or 3 element slice should fail, unless you supply your own iterator factory. Omitted slice elements being None is a nice idea, and would work with the RDF notation (subj::obj) -> slice(subj, None, obj) It gets a little hairy when you start doing things like this, however (::) -> slice(None, None, None) And what then of the 1-element slice? (if its necessary at all) (1:) -> slice(1, None) or slice(1)?? > -----Original Message----- > From: Alex Martelli [mailto:aleax@aleax.it] > Sent: Thursday, 2 May 2002 02:39 > To: Damien Morton; python-dev@python.org > Subject: iterable slices > > > On Thursday 02 May 2002 08:04, Damien Morton wrote: > ... > > Could the notation a:b:c:d:... be generalised, with > slices becoming a > > kind of tuple. For backwards compatability, the first three > elements > > of the tuple could be accessed using the start, stop, step > attributes. > ... > > {1, 2, 3} -> dict([slice(1), slice(2), slice(3)]} > > > > A slice on its own would be writen: > > > > (1:2:3) -> slice(1,2,3) > > > > A 1-element slice might be written similarily to a 1-element tuple: > > (1:) -> slice(1) > > And presumably None could be omitted, e.g. (:1) as slice(None,1). > > Related but somewhat orthogonal to this idea -- I'm starting > to think that slices should be iterable, with > iter(slice(a,b,c)) yielding exactly the same numbers as > iter(range(a,b,c)) or iter(xrange(a,b,c)). If any such > abbreviated notation existed for slices, then "for i in > (:6):" might work. Risks: perhaps error-prone ("for i in > {:6}:", "for i in (6:):", etc, might be likely typos yielding > unexpected behavior); no idea of what behavior would be > expected of "for i in (a:b:c:d:e):". > > > Alex > From aleax@aleax.it Thu May 2 08:20:52 2002 From: aleax@aleax.it (Alex Martelli) Date: Thu, 2 May 2002 09:20:52 +0200 Subject: [Python-Dev] RE: iterable slices In-Reply-To: <007601c1f1a7$ae197160$56296c42@damien> References: <007601c1f1a7$ae197160$56296c42@damien> Message-ID: <02050209205206.28570@arthur> On Thursday 02 May 2002 09:05, Damien Morton wrote: ... > I hadnt thought of that, but its kind of elegant. > > for i in (a:b:c:d:e): > > Would probably be an error. Trying to make an iterator out of anything > but a 2 or 3 element slice should fail, unless you supply your own > iterator factory. Currently you can't "supply your own factory" between iter() and a given object type -- iter() looks for the object's __iter__ (&c), not in any registry of adapter-factories. PEP 246 might be tweaked to change that, but it would be a doubtful change IMHO. > Omitted slice elements being None is a nice idea, and would work with > the RDF notation > > (subj::obj) -> slice(subj, None, obj) > > It gets a little hairy when you start doing things like this, however > > (::) -> slice(None, None, None) You could forbid it: >>> slice() Traceback (most recent call last): File "", line 1, in ? TypeError: slice() takes at least 1 argument (0 given) > And what then of the 1-element slice? (if its necessary at all) It definitely is (to indicate "all items from i" or "all items up to i", depending on which 'element' you mean, start or stop). > (1:) -> slice(1, None) or slice(1)?? >>> slice(3) slice(None, 3, None) >>> i.e., like (:3). (1:) must clearly mean "all from 1 upwards" by analogy with sequence slicing such as somelist[1:]. Alex From Damien.Morton@acm.org Thu May 2 08:49:20 2002 From: Damien.Morton@acm.org (Damien Morton) Date: Thu, 2 May 2002 03:49:20 -0400 Subject: [Python-Dev] RE: iterable slices In-Reply-To: <02050209205206.28570@arthur> Message-ID: <007701c1f1ad$de473790$56296c42@damien> > >>> slice(3) > slice(None, 3, None) > >>> This is where it breaks down. You'd have to do some buggering around with the start/stop/step attributes to make it work. There wouldn't be a direct correspondance between the slice/tuple element positions and the start and stop attributes. a = slice(3) a[0]==3, a.start==None, a.stop==3, a.step==None b = slice(1,3) b[0]==1, b[1]==3, b.start=1, b.stop=3, b.step=None By supplying your own factory I was thinking of something like the range() function. for i in weirditer(a:b:c:d:e): Probably not very usefull, given that you can already do that without having to resort to slices. for i in weirditer(a,b,c,d,e): Since lists happily deal with empty slices [1,2,3,4,5][:], I don't see any reason why youd forbid them at higher (or lower) dimensions. The issue with something like: (::) Is the line-noise quality of it. Its not too bad though. You can push this idea into the realms of the extremely strange by imagining nested slices and slices of tuples and such. (1:(2:3:(1,2,3)):("hello",f(y))) > -----Original Message----- > From: Alex Martelli [mailto:aleax@aleax.it] > Sent: Thursday, 2 May 2002 03:21 > To: Damien Morton; python-dev@python.org > Subject: Re: [Python-Dev] RE: iterable slices > > > On Thursday 02 May 2002 09:05, Damien Morton wrote: > ... > > I hadnt thought of that, but its kind of elegant. > > > > for i in (a:b:c:d:e): > > > > Would probably be an error. Trying to make an iterator out > of anything > > but a 2 or 3 element slice should fail, unless you supply your own > > iterator factory. > > Currently you can't "supply your own factory" between iter() > and a given object type -- iter() looks for the object's > __iter__ (&c), not in any > registry of adapter-factories. PEP 246 might be tweaked to > change that, but it would be a doubtful change IMHO. > > > > Omitted slice elements being None is a nice idea, and would > work with > > the RDF notation > > > > (subj::obj) -> slice(subj, None, obj) > > > > It gets a little hairy when you start doing things like > this, however > > > > (::) -> slice(None, None, None) > > You could forbid it: > > >>> slice() > Traceback (most recent call last): > File "", line 1, in ? > TypeError: slice() takes at least 1 argument (0 given) > > > And what then of the 1-element slice? (if its necessary at all) > > It definitely is (to indicate "all items from i" or "all > items up to i", depending on which 'element' you mean, start or stop). > > > (1:) -> slice(1, None) or slice(1)?? > > >>> slice(3) > slice(None, 3, None) > >>> > > i.e., like (:3). (1:) must clearly mean "all from 1 upwards" > by analogy with sequence slicing such as somelist[1:]. > > > Alex > From martin@v.loewis.de Thu May 2 08:10:01 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 02 May 2002 09:10:01 +0200 Subject: [Python-Dev] iterzip() In-Reply-To: References: Message-ID: Tim Peters writes: > [martin@v.loewis.de] > > I'd suggest a tp_is_immutable predicate. If a container is immutable > > and non-cyclic, it can be untracked. A container is non-cyclic if all > > contained objects are either non-gc, or untracked. > > And none of the contained objects are NULL (else the scheme doesn't even > work for tuples). If a tuple contains a NULL, its tp_is_immutable will return false, hence it won't be untracked, henced it isn't cyclic. > How do you know which other objects it contains? Calling tp_traverse and > seeing whether that ever calls the passed-in visit function doesn't work > (see the already-repeated discussion about this for tuples). That's why I propose the tp_is_immutable predicate. Regards, Martin From lkcl@samba-tng.org Thu May 2 11:26:41 2002 From: lkcl@samba-tng.org (Luke Kenneth Casson Leighton) Date: Thu, 02 May 2002 10:26:41 +0000 Subject: [Python-Dev] Re: [bug?] UserLong.py - pow(5, UserLong(3), 26) fails Message-ID: <3CD11461.EEC57A7@samba-tng.org> dear michael, i tried this: class UserLong(object): ... which i presume is the means by which new-style classes are specified (search google "new style python class" jumped into a tutorial: couldn't find an explicit place straight away). i get a segmentation fault at line 1217 of Objects/typeobject.c, because mro - type->tp_mro - is NULL. uhm... :) -- This email and any files transmitted with it, including replies and forwarded copies subsequently transmitted from Aspex Technology Limited, are confidential and solely for the use of the intended recipient. Any opinions expressed in this email are those of the individual and not necessarily those of Aspex Technology Limited. If you are not the intended recipient, be advised that you have received this email in error and that any use is strictly prohibited. If you have received this email in error, please notify us immediately by e-mail and delete the original message without keeping any copies. Aspex Technology Limited is registered in England and Wales No.3469577. The registered office for Aspex Technology Limited is York House, Cottingley Business Park, Bradford, BD16 1PF. From xscottg@yahoo.com Thu May 2 12:02:23 2002 From: xscottg@yahoo.com (Scott Gilbert) Date: Thu, 2 May 2002 04:02:23 -0700 (PDT) Subject: [Python-Dev] buffer objects Message-ID: <20020502110223.28981.qmail@web12903.mail.yahoo.com> I recently submitted a (really tiny) patch to allow the buffer() builtin to return a read/write PyBufferObject when possible. Mostly this was just to get my feet wet on the patch submission process, but I also think it is the correct behavior (am I wrong about this?). Being new at this, I don't know: Am I supposed to find a committer to champion my patch? Also... Several weeks ago, I brought up the suggestion that arraymodule.c arrays should be able to pickle directly and efficiently. I've since found out that the topic of arrays is a complicated one, and now I'm wondering if anyone else thinks that buffer objects should pickle. I look at buffers as mutable byte-strings. Having buffers pickle/unpickle (without a temporary copy) would avoid most of the questions about data types/sizes, endian-ness, ..., while allowing things which built on top of buffers (array modules for instance) to pickle efficiently. Just curious if a patch along these lines would be received well (assuming it's a good patch). Cheers, -Scott __________________________________________________ Do You Yahoo!? Yahoo! Health - your guide to health and wellness http://health.yahoo.com From guido@python.org Thu May 2 13:37:06 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 02 May 2002 08:37:06 -0400 Subject: [Python-Dev] Unicode objects more space efficient than plain strings? can that be? In-Reply-To: Your message of "Wed, 01 May 2002 23:15:35 CDT." <15568.48487.339982.314930@12-248-41-177.client.attbi.com> References: <15568.48487.339982.314930@12-248-41-177.client.attbi.com> Message-ID: <200205021237.g42Cb6P00871@pcp742651pcs.reston01.va.comcast.net> [Skip notices that the second below eats much more memory than the first:] > l = [u"abc%d"%i for i in xrange(1000000)] > l = ["abc%d"%i for i in xrange(1000000)] Yup, I can confirm this on Linux with a recent 2.3 build. A 2.2 build works as expected though (plain strings take up less space). One for the PyMalloc/GC team, I'm afraid... (Using a debug build and PYTHONDUMPREFS=1, I see no leaked objects.) --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik@pythonware.com Thu May 2 13:37:46 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Thu, 2 May 2002 14:37:46 +0200 Subject: [Python-Dev] buffer objects References: <20020502110223.28981.qmail@web12903.mail.yahoo.com> Message-ID: <091d01c1f1d6$2a7635d0$0900a8c0@spiff> scott wrote: > I look at buffers as mutable byte-strings. Having buffers = pickle/unpickle > (without a temporary copy) would avoid most of the questions about = data > types/sizes, endian-ness, ..., while allowing things which built on = top of > buffers (array modules for instance) to pickle efficiently. umm. pickles are supposed to be machine-independent, so how can you pickle stuff built on buffers *without* taking types/sizes/endianess = into account? From guido@python.org Thu May 2 13:55:11 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 02 May 2002 08:55:11 -0400 Subject: [Python-Dev] buffer objects In-Reply-To: Your message of "Thu, 02 May 2002 04:02:23 PDT." <20020502110223.28981.qmail@web12903.mail.yahoo.com> References: <20020502110223.28981.qmail@web12903.mail.yahoo.com> Message-ID: <200205021255.g42CtBO01138@pcp742651pcs.reston01.va.comcast.net> > I recently submitted a (really tiny) patch to allow the buffer() builtin to > return a read/write PyBufferObject when possible. Mostly this was just to > get my feet wet on the patch submission process, but I also think it is the > correct behavior (am I wrong about this?). Being new at this, I don't > know: Am I supposed to find a committer to champion my patch? If you feel that your patch isn't paid enough attention, please do! :-) > Also... Several weeks ago, I brought up the suggestion that arraymodule.c > arrays should be able to pickle directly and efficiently. I've since found > out that the topic of arrays is a complicated one, and now I'm wondering if > anyone else thinks that buffer objects should pickle. > > I look at buffers as mutable byte-strings. Having buffers pickle/unpickle > (without a temporary copy) would avoid most of the questions about data > types/sizes, endian-ness, ..., while allowing things which built on top of > buffers (array modules for instance) to pickle efficiently. Are you referring to the buffer object or the buffer interface? The buffer interface doesn't define an object type, it defines a particular way to look at an object (just like the numeric, sequence and mapping interfaces). There's no point in prescribing a pickle format for it. The buffer object was a mistake. If you want an efficient way of reading/writing memory buffers, look at the support for the buffer interface of the file readinto and write methods. You already can read and write arrays without copying. --Guido van Rossum (home page: http://www.python.org/~guido/) From mwh@python.net Thu May 2 14:01:17 2002 From: mwh@python.net (Michael Hudson) Date: 02 May 2002 14:01:17 +0100 Subject: [Python-Dev] Re: [bug?] UserLong.py - pow(5, UserLong(3), 26) fails In-Reply-To: Luke Kenneth Casson Leighton's message of "Thu, 02 May 2002 10:26:41 +0000" References: <3CD11461.EEC57A7@samba-tng.org> Message-ID: <2my9f2g0v6.fsf@starship.python.net> Luke Kenneth Casson Leighton writes: > class UserLong(object): > ... > > which i presume is the means by which new-style classes are specified > (search google "new style python class" jumped into a tutorial: > couldn't find an explicit place straight away). > > i get a segmentation fault at line 1217 of Objects/typeobject.c, > because mro - type->tp_mro - is NULL. You can make the segfault go away by putting "int.__mro__" at the top of the file. mro's are lazily calculated and here we manage to go through the code in an order that gets this wrong. Shouldn't be too hard to fix, but it's probably easier if you're Guido. Unfortunately, the situation overall is not improved: $ ./python lkcl.py Adding parser accelerators ... Done. int 3 long 3 5**int(y) 125 5**y rpow: (5,) 3 x^y%26 (3L, 26) 21L 5^y%26 Traceback (most recent call last): File "lkcl.py", line 172, in ? test() File "lkcl.py", line 156, in test print "5^y%26", pow(5, y, 26) TypeError: unsupported operand type(s) for pow(): 'int', 'UserLong', 'int' [6613 refs] I'm out of my depth in this area... Cheers, M. -- ARTHUR: Why are there three of you? LINTILLAS: Why is there only one of you? ARTHUR: Er... Could I have notice of that question? -- The Hitch-Hikers Guide to the Galaxy, Episode 11 From lsloan@umich.edu Thu May 2 16:36:00 2002 From: lsloan@umich.edu (Lance Sloan) Date: Thu, 02 May 2002 11:36:00 -0400 Subject: [Python-Dev] Why was Misc/Makefile.pre.in removed? Message-ID: <490479.1020339360@blue-four.us.itd.umich.edu> I'm wondering why Makefile.pre.in was removed from Python 2.2. I understand that is was due to a "BDFL pronouncement" (whatever that is), but unlike other such pronouncements, I cannot find an explanation of why this was done. Thanks! -- Lance E Sloan Web Services, Univ. of Michigan: Full-service Web and database design, development, and hosting. Specializing in Python & Perl CGIs. http://websvcs.itd.umich.edu/ - "Putting U on the Web" From guido@python.org Thu May 2 16:53:06 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 02 May 2002 11:53:06 -0400 Subject: [Python-Dev] Why was Misc/Makefile.pre.in removed? In-Reply-To: Your message of "Thu, 02 May 2002 11:36:00 EDT." <490479.1020339360@blue-four.us.itd.umich.edu> References: <490479.1020339360@blue-four.us.itd.umich.edu> Message-ID: <200205021553.g42Fr6D28071@odiug.zope.com> > I'm wondering why Makefile.pre.in was removed from Python 2.2. It was an error-prone mechanism that was impossible to maintain; distutils works much better. --Guido van Rossum (home page: http://www.python.org/~guido/) From lkcl@samba-tng.org Thu May 2 17:48:09 2002 From: lkcl@samba-tng.org (Luke Kenneth Casson Leighton) Date: Thu, 02 May 2002 16:48:09 +0000 Subject: [Python-Dev] Re: [bug?] UserLong.py - pow(5, UserLong(3), 26) fails Message-ID: <3CD16DC9.73216E9A@samba-tng.org> michael, thanks for the int.__mru tip. i confirm: pow(5, UserLong(3), 26) - gets the same error that you report. i also confirmed: pow(5, UserLong(3)) - calls UserLong.__rpow__. it works! errrr :) :) so it looks like it's _almost_ there, but not quite. i _believe_ i can get away with this, for DSA.py, at this time, if i correctly implement UserLong.__rpow__ (which was previously a stub). later tonight i'll raise a proper bug report. thanks for the tips, michael. l. From xscottg@yahoo.com Thu May 2 16:49:01 2002 From: xscottg@yahoo.com (Scott Gilbert) Date: Thu, 2 May 2002 08:49:01 -0700 (PDT) Subject: [Python-Dev] buffer objects In-Reply-To: <091d01c1f1d6$2a7635d0$0900a8c0@spiff> Message-ID: <20020502154901.25118.qmail@web12904.mail.yahoo.com> --- Fredrik Lundh wrote: > > umm. pickles are supposed to be machine-independent, so how can you > pickle stuff built on buffers *without* taking types/sizes/endianess into > account? > :-) It's not that I, personally, wouldn't have to take type/size/endianness into account if I built something on top of a buffer in which type/size/endianness matters. It's that I wouldn't have to get a concensus from anyone else on how to do so. I'd define my __getstate__/__setstate__ for a class to work with a tuple of (type, endian, size, buffer, ...) if needed, and someone else could define theirs however fit their needs. The buffer itself would just pickle from mutable-bytes to mutable-bytes (hopefully with the same alignment that malloc provides for the native machine). Cheers, -Scott __________________________________________________ Do You Yahoo!? Yahoo! Health - your guide to health and wellness http://health.yahoo.com From xscottg@yahoo.com Thu May 2 17:46:18 2002 From: xscottg@yahoo.com (Scott Gilbert) Date: Thu, 2 May 2002 09:46:18 -0700 (PDT) Subject: [Python-Dev] buffer objects In-Reply-To: <200205021255.g42CtBO01138@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020502164618.46216.qmail@web12908.mail.yahoo.com> --- Guido van Rossum wrote: > > If you feel that your patch isn't paid enough attention, please do! :-) > I just wanted to know how the process worked. I read the FAQ, but I wasn't sure if I was supposed to assign it to someone or just wait patiently. Here I am talking about it though, so clearly I'm not good at that patience thing! :-) > > Are you referring to the buffer object or the buffer interface? > PyBufferObject* as returned by the builtin buffer() function. > > The buffer interface doesn't define an object type, it defines a > particular way to look at an object (just like the numeric, sequence > and mapping interfaces). There's no point in prescribing a pickle > format for it. > Agreed completely. > > The buffer object was a mistake. > Bummer that you feel this way. :-( It looked to me like PyBufferProcs made a small mistake by including bf_getsegcount() which is mostly a YAGNI in your terms. I'm sure someone is actually using the segcount, but they probably wouldn't have missed it if it didn't exist. A lot of code seems to assume the segcount is always 1. I thought the buffer object was on the right track (with a tiny read/write patch :-). I guess I'm not getting the Python philosophy of things then. We have "string"s which are doubling as readonly-byte-arrays and text-strings, and I thought the concensus around here was that this was an unfortunate duality. Then we have "unicode"s which are clearly just for text-strings. Both of these are immutable as per the philosophy that strings are a lot like numbers. Then we have buffer objects. If the buffer object is a mistake, then there is no endorsed way to get at a (possibly mutable) array of bytes from Python. One can use arrays of typecode 'B', but you can't point those at your own memory, and they don't pickle. So if someone would only charge up the time machine, I thought it would be preferrable to only have unicode objects, and buffer objects. (Possibly with unicode objects being renamed as strings instead...) I think you're saying that the only use for PyBufferProcs is from C/C++ extensions. Because without a PyBufferObject type of thing, you can't manipulate PyBufferProcs gotten memory from pure Python. > > If you want an efficient way of reading/writing memory buffers, look > at the support for the buffer interface of the file readinto and write > methods. You already can read and write arrays without copying. > Yep, but you can't pickle them without copying to a (potentially large) string first. We want to use cPickle to pass objects between processes/machines (across sockets, or through shared memory). We know that sometimes (frequently) those objects will have array data mixed in with dictionaries/lists/.... If arrays/bytes/buffers don't pickle nicely, then do we need to use two channels? One for picklable objects, and one for arrays/bytes? The algorithm for separating the picklable parts from the non-picklable parts of a nested data structure would have to be nearly as complicated as pickle itself. Cheers, -Scott __________________________________________________ Do You Yahoo!? Yahoo! Health - your guide to health and wellness http://health.yahoo.com From xscottg@yahoo.com Thu May 2 18:38:46 2002 From: xscottg@yahoo.com (Scott Gilbert) Date: Thu, 2 May 2002 10:38:46 -0700 (PDT) Subject: [Python-Dev] buffer objects In-Reply-To: <200205021255.g42CtBO01138@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020502173846.86413.qmail@web12903.mail.yahoo.com> --- Guido van Rossum wrote: > > [...] look > at the support for the buffer interface of the file readinto and write > methods. > >>> file.readinto.__doc__ "readinto() -> Undocumented. Don't use this; it may go away." I'll submit a doc patch if this is no longer the case... __________________________________________________ Do You Yahoo!? Yahoo! Health - your guide to health and wellness http://health.yahoo.com From thomas.heller@ion-tof.com Thu May 2 19:04:12 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Thu, 2 May 2002 20:04:12 +0200 Subject: [Python-Dev] __doc__ string of builtin types Message-ID: <032801c1f203$c4209d60$e000a8c0@thomasnotebook> (I've filed a slightly confused bug report 550290.) "spam".__doc__ is the same as str.__doc__, it describes what the str(...) would do. Same for 42 . __doc__, and certainly many other objects. While normally I don't care about the doc-string of these objects, the output of pydoc looks strange: >>> class O(object): ... "some text" ... >>> import pydoc >>> pydoc.help(O) Help on class O in module __main__: class O(__builtin__.object) | some text | | Data and non-method functions defined here: | | __dict__ = | | __doc__ = 'some text' | str(object) -> string | | Return a nice string representation of the object. | If the argument is a string, the return value is the same object. | Thomas From martin@v.loewis.de Thu May 2 19:16:07 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 02 May 2002 20:16:07 +0200 Subject: [Python-Dev] Why was Misc/Makefile.pre.in removed? In-Reply-To: <200205021553.g42Fr6D28071@odiug.zope.com> References: <490479.1020339360@blue-four.us.itd.umich.edu> <200205021553.g42Fr6D28071@odiug.zope.com> Message-ID: Guido van Rossum writes: > > I'm wondering why Makefile.pre.in was removed from Python 2.2. > > It was an error-prone mechanism that was impossible to maintain; > distutils works much better. More specifically, it was already broken when it was removed; nobody volunteered to fix it. Regards, Martin From pobrien@orbtech.com Thu May 2 19:32:30 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Thu, 2 May 2002 13:32:30 -0500 Subject: [Python-Dev] __doc__ string of builtin types In-Reply-To: <032801c1f203$c4209d60$e000a8c0@thomasnotebook> Message-ID: [Thomas Heller] > > (I've filed a slightly confused bug report 550290.) > > "spam".__doc__ is the same as str.__doc__, > it describes what the str(...) would do. Same for > 42 . __doc__, and certainly many other objects. > While normally I don't care about the doc-string of > these objects, the output of pydoc looks strange: I agree. I see the same thing with the namespace viewer in PyCrust under Python 2.2 and it is less than ideal, imho. --- Patrick K. O'Brien Orbtech From tim.one@comcast.net Thu May 2 19:36:36 2002 From: tim.one@comcast.net (Tim Peters) Date: Thu, 02 May 2002 14:36:36 -0400 Subject: [Python-Dev] Unicode objects more space efficient than plain strings? can that be? In-Reply-To: <200205021237.g42Cb6P00871@pcp742651pcs.reston01.va.comcast.net> Message-ID: It's a cute one. Setting the envar PYTHONMALLOCSTATS in a debug build zeroes in on the cause: PyString_Format() allocates its result space via reslen = rescnt = fmtcnt + 100; result = PyString_FromStringAndSize((char *)NULL, reslen); and that's way more space (about 100 bytes more) than is actually needed for the result of "abc%d" % i But it's still small enough for pymalloc to handle on its own, and the pymalloc realloc currently never shrinks a block (it's in the nature of this kind of allocator that shrinking requires copying the whold block -- it's "a speed thing"). So each result string object contains more than 100 bytes of string space, mostly unused. We could worm around this in lots of ways. I'm inclined to change the pymalloc realloc to copy a shrinking block if at least 25% of the input block would go away, else leave it alone. In this specific case, something like 90% of the input block could be reclaimed. From tim.one@comcast.net Thu May 2 21:23:23 2002 From: tim.one@comcast.net (Tim Peters) Date: Thu, 02 May 2002 16:23:23 -0400 Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Modules arraymodule.c,2.70,2.71 In-Reply-To: Message-ID: > Update of /cvsroot/python/python/dist/src/Modules > In directory usw-pr-cvs1:/tmp/cvs-serv16518 > > Modified Files: > arraymodule.c > Log Message: > Patch #551009: Initialize array type dynamically. I like these a lot better if a do-nothing macro is plugged into the "0" slots, like #define DELAYED(X) DELAYED(PyObject_Del), /* tp_free */ instead of 0, /* tp_free */ It's confusing and error-prone to need to search all over the source code to find out what's really in a type object. From tim.one@comcast.net Thu May 2 21:29:33 2002 From: tim.one@comcast.net (Tim Peters) Date: Thu, 02 May 2002 16:29:33 -0400 Subject: [Python-Dev] Unicode objects more space efficient than plain strings? can that be? In-Reply-To: Message-ID: [Tim] > ... > I'm inclined to change the pymalloc realloc to copy a shrinking block > if at least 25% of the input block would go away, else leave it alone. > In this specific case, something like 90% of the input block could be > reclaimed. So I did that. In the test program """ if 1: L = [u"abc%d"%i for i in xrange(1000000)] else: L = ["abc%d"%i for i in xrange(1000000)] raw_input('L is built') del L raw_input('L is deleted') """ the process size grew to 46MB using regular strings, and to 69MB using Unicode strings. The second time the test program pauses shows another difference: pymalloc never gives small-block memory back to the system free now, so the process size only shrunk by 4MB (for the list guts) after deleting L when using regular strings. The process size shrunk by 42MB when using Unicode strings. From Jack.Jansen@oratrix.com Thu May 2 21:45:52 2002 From: Jack.Jansen@oratrix.com (Jack Jansen) Date: Thu, 2 May 2002 22:45:52 +0200 Subject: [Python-Dev] buffer objects In-Reply-To: <20020502164618.46216.qmail@web12908.mail.yahoo.com> Message-ID: <97CEE8FC-5E0D-11D6-8F13-003065517236@oratrix.com> On donderdag, mei 2, 2002, at 06:46 , Scott Gilbert wrote: > It looked to me like PyBufferProcs made a small mistake by including > bf_getsegcount() which is mostly a YAGNI in your terms. I'm > sure someone > is actually using the segcount, but they probably wouldn't have > missed it > if it didn't exist. A lot of code seems to assume the segcount > is always > 1. This was put in on the specific request of the people who wanted the buffer interface in the first place: the Numeric folks. They have all sorts of funny sparse arrays and arrays with strides and arrays that are really projections on other data structures (and lots more stuff I don't understand as should by now be clear:-). They were the driving force behind readinto() (and hence behind the whole buffer interface in the first place) because that's the only reasonable way they could read a gigabyte array of floats into memory without having two gigabytes of memory available (or wasting oodles of cycles by doing small read()s in a loop). But, I think I agree on the YAGNI, because I don't think they ever got around to using the segmented buffer stuff. -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From Jack.Jansen@oratrix.com Thu May 2 21:51:01 2002 From: Jack.Jansen@oratrix.com (Jack Jansen) Date: Thu, 2 May 2002 22:51:01 +0200 Subject: [Python-Dev] buffer objects In-Reply-To: <20020502164618.46216.qmail@web12908.mail.yahoo.com> Message-ID: <4FA9414C-5E0E-11D6-8F13-003065517236@oratrix.com> On donderdag, mei 2, 2002, at 06:46 , Scott Gilbert wrote: > I guess I'm not getting the Python philosophy of things then. We have > "string"s which are doubling as readonly-byte-arrays and > text-strings, and > I thought the concensus around here was that this was an unfortunate > duality. Then we have "unicode"s which are clearly just for > text-strings. > Both of these are immutable as per the philosophy that strings > are a lot > like numbers. > > Then we have buffer objects. If the buffer object is a > mistake, then there > is no endorsed way to get at a (possibly mutable) array of bytes from > Python. One can use arrays of typecode 'B', but you can't > point those at > your own memory, and they don't pickle. > > So if someone would only charge up the time machine, I thought > it would be > preferrable to only have unicode objects, and buffer objects. > (Possibly > with unicode objects being renamed as strings instead...) I think we need immutable binary byte arrays as well, these are used for code objects and such. But I agree that a mutable binary array type such as the bufferobject is needed for efficient Python implementation of lots of things. -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From skip@pobox.com Thu May 2 22:41:13 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 2 May 2002 16:41:13 -0500 Subject: [Python-Dev] Unicode objects more space efficient than plain strings? can that be? In-Reply-To: References: Message-ID: <15569.45689.171793.299099@beluga.mojam.com> >> I'm inclined to change the pymalloc realloc to copy a shrinking block >> if at least 25% of the input block would go away, else leave it >> alone. In this specific case, something like 90% of the input block >> could be reclaimed. Tim> So I did that. Much better, thanks. Strange that such a naive "benchmark" would lead to such odd behavior. I wonder if this is potentially a bugfix candidate, at least for 2.2.x? Skip From tim.one@comcast.net Thu May 2 23:02:42 2002 From: tim.one@comcast.net (Tim Peters) Date: Thu, 02 May 2002 18:02:42 -0400 Subject: [Python-Dev] Unicode objects more space efficient than plain strings? can that be? In-Reply-To: <15569.45689.171793.299099@beluga.mojam.com> Message-ID: [Skip Montanaro] > Much better, thanks. Strange that such a naive "benchmark" would lead to > such odd behavior. Na, the interesting ones are always one-liners . > I wonder if this is potentially a bugfix candidate, at least for 2.2.x? pymalloc was an experimental use-at-your-own-risk thing in 2.2 and before, and Neil and I have both spent major time now making it fully usable for 2.3. IOW, backpatching pymalloc improvements is over my limit for what "a bugfix" means; 2.3's pymalloc is more of a new feature. From mhammond@skippinet.com.au Fri May 3 00:40:15 2002 From: mhammond@skippinet.com.au (Mark Hammond) Date: Fri, 3 May 2002 09:40:15 +1000 Subject: [Python-Dev] buffer objects In-Reply-To: <20020502164618.46216.qmail@web12908.mail.yahoo.com> Message-ID: [Scott, quoting Guido] > > The buffer object was a mistake. > > > > Bummer that you feel this way. :-( > > It looked to me like PyBufferProcs made a small mistake by including > bf_getsegcount() which is mostly a YAGNI in your terms. I'm sure someone > is actually using the segcount, but they probably wouldn't have missed it > if it didn't exist. A lot of code seems to assume the segcount is always > 1. > > I thought the buffer object was on the right track (with a tiny read/write > patch :-). Jack chimes in: > But I agree that a mutable > binary array type such as the bufferobject is needed for > efficient Python implementation of lots of things. [Back to Scott] > I guess I'm not getting the Python philosophy of things then. You are not alone in this specific case. I too thought the bufffer object was a Good Thing(tm), and indeed the win32 extensions have good use for them (supporting asynchronous IO on Windows for one). It turns out that the buffer *interface* has a design flaw, best described by using the array object as an example. The real possibility exists that code can use the buffer interfaces to get a pointer to writable memory for the array. The array may then be resized, thereby moving the buffer but leaving the buffer pointer dangling. So from my side of the fence, it seems the buffer interface was a mistake, and at least 2 significant extension authors believe a r/w buffer *object* is useful ;) I am not too sure what to do here. I can never remember the conversation moving past "it was a mistake" into the realm of "and here is what we should do instead." As far as I can tell, this same stalemate has also prevented any progress on the buffer interface flaw described above - no one really knows what the "right thing" is. Mark. From greg@cosc.canterbury.ac.nz Fri May 3 01:00:52 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Fri, 03 May 2002 12:00:52 +1200 (NZST) Subject: [Python-Dev] buffer objects In-Reply-To: Message-ID: <200205030000.MAA03068@s454.cosc.canterbury.ac.nz> Mark Hammond : > It turns out that the buffer *interface* has a design flaw, best described > by using the array object as an example. The real possibility exists that > code can use the buffer interfaces to get a pointer to writable memory for > the array. The array may then be resized, thereby moving the buffer but > leaving the buffer pointer dangling. If you do that using the (C-level) buffer interface, you're misusing it. You shouldn't be keeping a pointer to the internals of an object across any operation which could cause it to move, however that pointer is obtained. The design flaw is in the Python-level buffer *object*, which misuses the buffer interface in exactly that way. This is so insane that I can't understand why the buffer object hasn't either been fixed or ripped out by now. The fix seems obvious to me -- don't cache the pointer in the buffer object, but use the buffer interface to re-fetch it on each Python-level call which needs it. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From timo@alum.mit.edu Fri May 3 07:45:08 2002 From: timo@alum.mit.edu (Timothy O'Malley) Date: Fri, 03 May 2002 02:45:08 -0400 Subject: [Python-Dev] timeoutsocket.py In-Reply-To: <3CBB3A20.35C8132F@3captus.com> Message-ID: hola. I've taken a whack at option #1 (grand unification of socket and timeoutsocket). It's larger than I'd prefer to send in email. If you are interested: http://www.timo-tasi.org/python/socket.py -- TimO "Create like a God, Command like a King, and Work like a slave" -- Brancusi > From: Bernard Yue > Date: Mon, 15 Apr 2002 14:37:53 -0600 > To: skip@pobox.com > Cc: Guido van Rossum , python-dev@python.org, > timo@alum.mit.edu > Subject: Re: [Python-Dev] timeoutsocket.py > > Skip Montanaro wrote: > >>>> What is the status of integrating timeoutsocket.py into standard >>>> library? Are there any body responsible for it currently? If not I >>>> would like to give it a shot. >> >> Guido> Isn't one of the problems that the timeout is global? >> >> That's the easiest way to use it, but you can specify timeouts on a >> per-socket basis. From the module docstring: >> >> import timeoutsocket >> import httplib >> H = httplib.HTTP("www.python.org") >> H.sock.set_timeout(30) >> >> The author, Tim O'Malley, is probably the best person to address issues with >> this package, so I've added him to the cc list in case he's not a python-dev >> subscriber. >> >> Skip > > Just have a look at socket.py and timeoutsocket.py (version 1.21, assuming the > lastest). Looks like that class _socketobject and class Timeoutsocket can be > merged with no conflict. > > Now it comes the question: > > 1. Shall we merge socket.py and timeoutsocket.py? > 2. If yes to Q1, how are we going to due with the copyright issue? > 3. If no to Q1, are we going to just add timeoutsocket.py to standard library > (Though there the copyright issue stays)? > > > Bernie > > -- > In Scotland, a new game was invented. It was entitled > Gentlemen Only Ladies Forbidden.... and > thus the word GOLF entered into the English language. > From bernie@3captus.com Fri May 3 08:49:02 2002 From: bernie@3captus.com (Bernard Yue) Date: Fri, 03 May 2002 01:49:02 -0600 Subject: [Python-Dev] timeoutsocket.py References: Message-ID: <3CD240EE.7ABCCCE2@3captus.com> Timothy O'Malley wrote: > > hola. > > I've taken a whack at option #1 (grand unification of socket and > timeoutsocket). > > It's larger than I'd prefer to send in email. If you are interested: > > http://www.timo-tasi.org/python/socket.py > > -- > TimO > Tim, this is a complete rewrite of timeoutsocket.py. It is quite an effort :) > 1. Shall we merge socket.py and timeoutsocket.py? > > It certainly is possible to merge TimeoutSocket and _socketobject. > If the merge was done in a straightforward manner, then every TCP > socket would use Python code (instead of C code) for recv() and > send() methods. Some might consider this an unacceptable performance > hit. I agree on the performance issue. Over the past two weeks, Michael Gilfix and I moved ahead with the implementing of timeout socket in C level. Michael translates what you've done in timeoutsocket.py and put it into socket.c. So far we did nothing to TimeoutFile. The proposed patch is basically done (except for the test case). You can find it on http://www.3captus.com/Downloads/. We are now working on the test case for connect() and sendfoo(). Currently, test for connect() is done by trying a timeout connect to an external site (same to what you've done). However, we wanted to remove the external dependency on standard python test suit. So far no luck of make it working, and I cannot make sendfoo() to raise timeout error at all. Would you like to help us with the test? Bernie -- There are three schools of magic. One: State a tautology, then ring the changes on its corollaries; that's philosophy. Two: Record many facts. Try to find a pattern. Then make a wrong guess at the next fact; that's science. Three: Be aware that you live in a malevolent Universe controlled by Murphy's Law, sometimes offset by Brewster's Factor; that's engineering. -- Robert A. Heinlein From xscottg@yahoo.com Fri May 3 12:08:07 2002 From: xscottg@yahoo.com (Scott Gilbert) Date: Fri, 3 May 2002 04:08:07 -0700 (PDT) Subject: [Python-Dev] buffer objects In-Reply-To: <200205030000.MAA03068@s454.cosc.canterbury.ac.nz> Message-ID: <20020503110807.94863.qmail@web12903.mail.yahoo.com> Mark Hammond : > > It turns out that the buffer *interface* has a design flaw, best > described by using the array object as an example. The real > possibility exists that code can use the buffer interfaces to get a > pointer to writable memory for the array. The array may then be > resized, thereby moving the buffer but leaving the buffer pointer > dangling. > Thank you for the very clear explanation. Now I see why there is a problem. Actually there are a number of implementation problems in PyBufferObject. This isn't a problem with the buffer() builtin though, and it's not a really a problem with making the buffer() builtin return a read-write object. Even without my change to the buffer() builtin, the C-API to PyBufferObjects can currently still cause real damage. I'm working on a patch to fix this if anyone wants to champion it. If no one wants to champion this new patch, then I need to withdraw my older patch as it is working on incorrect assumptions. I'll also submit bug reports about the list of other issues in bufferobject.c. (Hashing is broken for buffer objects too.) Greg Ewing : > > If you do that using the (C-level) buffer interface, > you're misusing it. You shouldn't be keeping a pointer > to the internals of an object across any operation which > could cause it to move, however that pointer is obtained. > > The design flaw is in the Python-level buffer *object*, > which misuses the buffer interface in exactly that way. > This is so insane that I can't understand why the > buffer object hasn't either been fixed or ripped > out by now. > You're right. Bummer too. That means an extension module can't release the GIL while working with the pointer to memory. (For instance to perform intensive calculations with the pointer on one processor while letting another thread run concurrently.) It also means that other asynchronous operations can't safely work on pointers returned from PyBufferProcs. (Unless the GIL is held the whole time, and that would probably make doing anything asynchronous less interesting...) So PyBufferProcs is less useful than I thought it was a day ago. I naively assumed the pointer was supposed to be constant for the lifetime of the object. > > The fix seems obvious to me -- don't cache the pointer > in the buffer object, but use the buffer interface to > re-fetch it on each Python-level call which needs it. > Right you are, and that's what I'm doing. Patch coming soon... Cheers, -Scott __________________________________________________ Do You Yahoo!? Yahoo! Health - your guide to health and wellness http://health.yahoo.com From magnus@hetland.org Sat May 4 01:19:08 2002 From: magnus@hetland.org (Magnus Lie Hetland) Date: Sat, 4 May 2002 02:19:08 +0200 Subject: [Python-Dev] slice notation and RDF In-Reply-To: <007101c1f19f$2c44b940$56296c42@damien>; from Damien.Morton@acm.org on Thu, May 02, 2002 at 02:04:08AM -0400 References: <007101c1f19f$2c44b940$56296c42@damien> Message-ID: <20020504021908.A3162@idi.ntnu.no> Damien Morton : [snip] > { 1:2, 3:4 } <--> dict([slice(1,2), slice(3,4)]) > > For RDF notation, you could then happily write: > > {1:2:3, 4:5:6, 7:8:9} --> dict([slice(1,2,3), slice(4,5,6), > slice(7,8,9)]) Hm. Is it really necessary to invent a new notation for this? The link between these tuple-like structures (e.g. 1:2:3) and slices seems tenuous and very confusing to me. If we're not going with the W3C XML notation, why not use a more Pythonic one? Assuming that we'll eventually get sets, should the Python-RDF-notation simply be a set of 3-tuples? E.g: {(1, 2, 3), (4, 5, 6), (7, 8, 9)} -- Magnus Lie Hetland The Anygui Project http://hetland.org http://anygui.org From David Abrahams" Hi, I'm hoping I can raise some interest in resolving this problem: By default, Python does not use the RTLD_GLOBAL flag when opening extension modules. Unfortunately, this breaks many C++ features when used across modules(http://gcc.gnu.org/faq.html#dso). It also causes these features to fail across the boundary between modules and any shared library they might be linked to. This is a key arrangement for Boost.Python: its extension modules all make use of a common shared library. I realize that we can change how modules are loaded using sys.setdlopenflags(), imputils, etc., but all that puts knowledge in the wrong place: the importer needs to know the special way to import a given module. It seems to me that extension modules themselves should have a way to report to Python that they need to be loaded with RTLD_GLOBAL. It appears that Boost.Python is not the only project that is having a problem with the way this works (http://aspn.activestate.com/ASPN/Mail/Message/xml-sig/1040230), so perhaps there's a good reason think about alternatives? I am not by any means an expert in GNU dynamic loading, so I only have what are probably crackpot ideas about how to address this (if, indeed Python is using the correct default). Making a show of constructive suggestion: Have Python look for a special symbol, say init_dlopenflags(). If it's found, it's called. If the result doesn't match the current dlopenflags the module is dlclose()d and re-opened with the requested flags. Thoughts? -Dave +---------------------------------------------------------------+ David Abrahams C++ Booster (http://www.boost.org) O__ == Pythonista (http://www.python.org) c/ /'_ == resume: http://users.rcn.com/abrahams/resume.html (*) \(*) == email: david.abrahams@rcn.com +---------------------------------------------------------------+ From tim.one@comcast.net Sat May 4 03:53:21 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 03 May 2002 22:53:21 -0400 Subject: [Python-Dev] buffer objects In-Reply-To: <20020503110807.94863.qmail@web12903.mail.yahoo.com> Message-ID: Note that "the buffer object problem" is an annual pointless discussion. The last big round on Python-Dev was near the start of June 2001, under the unlikely Subject "strop vs. string". Nobody volunteered to do anything then either, although one person made a big show of agreeing to plug the holes, and then didn't. That's how it usually ends. If you want to "do something" here, I suggest reading the old threads to get coverage of the outstanding issues. Nothing new has been said or suggested in years. From martin@v.loewis.de Sat May 4 07:37:56 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 04 May 2002 08:37:56 +0200 Subject: [Python-Dev] Problems with Python's default dlopen flags In-Reply-To: <092501c1f30f$337b9280$6401a8c0@boostconsulting.com> References: <092501c1f30f$337b9280$6401a8c0@boostconsulting.com> Message-ID: "David Abrahams" writes: > It seems to me that extension modules themselves should have a way > to report to Python that they need to be loaded with RTLD_GLOBAL. No way; to change Python in this way would be extremely foolish. Python was using RTLD_GLOBAL until 1.5.1, then this was changed in 1.5.2 due to bug reports by users. Redhat decided to revert this change, and consequently people run into problems with the Redhat Python 1.5.2 installation. Here is the original problem: A Python application was using both Oracle and sockets, so it had the Oracle and socket modules loaded. Unfortunately, both provided an initsocket function (at that time; today the socket module provides init_socket). It so happened that the dynamic linker chose the initsocket definition from the socket module. When Oracle called its own initsocket function, the call ended up in the Python module, and the application crashed; this is painful to analyse. Now, people apparently want to share symbols across modules. Let me say that I find this desire misguided: Python extension modules are *not* shared libraries, they *only* interface with the Python interpreter. If you want to share symbols, use shared libraries: If modules A.so and B.so have symbols in common, create a shared library C.so that provides those symbols, and link both A.so and B.so with this shared library. Now, people still want to share symbols across modules. For that, you can use CObjects: Export a CObject with an array of function pointers in module A (e.g. as A.API), and import that C object in module B's initialization code. See cStringIO and Numeric for examples. Now, people still want to share symbols across modules. For that, they can use sys.setdlopenflags. It seems that this is a lose-lose situation: you can't please everybody. In the current state, people that want to share symbols can, if they really want to. With your proposed change, symbols that accidentally clash between unrelated extensions cause problems, and users of those modules can do nothing about it. Hence, the current state is preferable. HTH, Martin From evilzr@yahoo.com Sat May 4 10:25:54 2002 From: evilzr@yahoo.com (Daniel Dunbar) Date: Sat, 4 May 2002 02:25:54 -0700 (PDT) Subject: [Python-Dev] Lazily GC tracking tuples Message-ID: <20020504092554.37178.qmail@web14706.mail.yahoo.com> A few days ago the idea of untracking, or somehow removing, immutable and provably uncyclic tuples from garbage collection was bounced about - at the time the idea was to untrack a tuple once it was noticed that it contained no cyclic objects... this proves annoying to implement because of unfilled tuples, and C-api calls that mutate the tuple (highly unusual it seems). Lazily tracking the tuples seems to be a much simpler proposition: a tuple can only be a member of a cycle once an element of ob_item is set to an object that could contain a cycle, presumably items are set in a tuple far less frequently than they are accessed (by normal code or the collector), so I imagine this method is also more efficient. The changes required are fairly small, PyObject_GC_IS_TRACKED is added to determine if an object is tracked, the gc track and untrack operations in tupleobject.c are modified to account for the tuple not always being tracked, and PyTuple_SET_ITEM and PyTuple_SetItem are modified to begin tracking the tuple if a) it is not already tracked, and b) the item being set _could_ contain a cycle. At the moment, I use PyObject_IS_GC as a conserative approximation to '_could_ contain a cycle', which works well except for nested tuples. In theory if v is a tuple, _SetItem(o,i,v) should only be called when v has been completely filled, so if v is untracked then we know it also cannot be member of a cycle, which solves the nested tuple problem. In practice making the nested tuple change did not cause any change in behavior on the test suite, but I have left it out for simplicity. Large trees of atomic data represented as tuples seems like a case where it could possibly make a difference. A 2.23 patch for the changes outlined above are at: http://www.geocities.com/evilzr/lazytuplegc/ (I wasn't sure if such things are appropriate for the SF-patch manager - yes/no?) If you want to actually test the changes you need a recent checkout, there was a minor collector bug that the lazy tracking exploited heavily. Some stats - cvs | lazytuplegc --------------------------------------- iterzip_test, N=500000 juststore: 1.46 | 1.43 gc size 1143 | 788 justtups: 1.40 | 1.30 gc size 1143 | 788 storetups: 8.48 | 4.11 gc size 501143 | 788 base gc set size python: 1081 | 742 idle: 7754 | 3818 pybench 1.0 avg time (n=10,warp=20) 69813.40 | 69708.04 The iterzip_test is only slightly munged from the one Tim Peters posted on the iterzip() thread. The pybench averages are almost identical, however there are large fluctuations on the individual tests, see http://www.geocities.com/evilzr/lazytuplegc/cmp.txt The base gc set numbers are obtained by starting python (or idle) and immediatly running a collection with gc.DEBUG_STATS on. All gc size numbers are size(gen0)+size(gen1)+size(gen2). this patch causes one regrtest to fail (test_gc, in test_staticmethod, for reasons I do not yet understand (help!)). ... make of it what you will :) Oh and, not to make this post longer, but heres the first-pydev-list-posting-info: I'm Daniel Dunbar, my introduction to Python was embedding it into the Blender 3D package (www.blender3d.com, but we went out of business - oops). Since then I have used it for lots of esoteric little tasks, but only recently after wrapping some of my C-based libraries have I started writing complex (ie. applications not utilities) programs with it. My main interest related to python development is making it go faster (so I can optimize my code less)... but thats probably not rare. ===== daniel dunbar evilzr@yahoo.com __________________________________________________ Do You Yahoo!? Yahoo! Health - your guide to health and wellness http://health.yahoo.com From David Abrahams" Message-ID: <09be01c1f36e$1fe46b20$6401a8c0@boostconsulting.com> ----- Original Message ----- From: "Martin v. Loewis" > "David Abrahams" writes: > > > It seems to me that extension modules themselves should have a way > > to report to Python that they need to be loaded with RTLD_GLOBAL. > > No way; to change Python in this way would be extremely > foolish. Python was using RTLD_GLOBAL until 1.5.1, then this was > changed in 1.5.2 due to bug reports by users. Redhat decided to revert > this change, and consequently people run into problems with the Redhat > Python 1.5.2 installation. Did you misread my suggestion? I didn't say that RTLD_GLOBAL should be the default way to load an extension module, only that there should be a way for the module itself to determine how it's loaded. > Here is the original problem: A Python application was using both > Oracle and sockets, so it had the Oracle and socket modules > loaded. Unfortunately, both provided an initsocket function (at that > time; today the socket module provides init_socket). It so happened > that the dynamic linker chose the initsocket definition from the > socket module. When Oracle called its own initsocket function, the > call ended up in the Python module, and the application crashed; this > is painful to analyse. Yes, that must have been. I can also imagine that it causes problems for identically-named (sub)modules in packages (though my lack of expertise should be apparent here again: maybe dlsym() will always grab the symbol from the newly opened library). > Now, people apparently want to share symbols across modules. Let me > say that I find this desire misguided: Python extension modules are > *not* shared libraries, they *only* interface with the Python > interpreter. It surprised me as well when I started developing Boost.Python, but it turns out that people really think it's important to be able to do component-based development on their Python extensions and occasionally they need to be able to register things like exception translators from one module which will be used by another module. However, as you can see below, nothing that fancy is in play in this case... > If you want to share symbols, use shared libraries: If > modules A.so and B.so have symbols in common, create a shared library > C.so that provides those symbols, and link both A.so and B.so with > this shared library. Guess what? That's what Boost.Python does! In fact, in the cases we're seeing that are fixed by using RTLD_GLOBAL **there's no need for sharing of symbols across across A.so and B.so**! The arrangement looks like this: python / \ (dlopen) (dlopen) / \ | | V V ext1.so ext2.so \ / (ld) (ld) \ / \ / | | V V libboost_python.so And in fact, I expect to ask users to do something special, like explicitly linking between extension modules, if they want to share exception/RTTI information between ext1 and ext2 directly. However, this is what I didn't expect: the lack of RTLD_GLOBAL flags interferes with the ability for ext1.so to catch C++ exceptions thrown by libboost_python.so! Are you suggesting that in order to do this, my users need to add yet another .so, a thin layer between Python and the guts of their extension modules? > Now, people still want to share symbols across modules. For that, you > can use CObjects: Export a CObject with an array of function pointers > in module A (e.g. as A.API), and import that C object in module B's > initialization code. See cStringIO and Numeric for examples. Of course you realize that won't help with C++ exception tables... > Now, people still want to share symbols across modules. For that, they > can use sys.setdlopenflags. ...which leads us back to the fact that the smarts are in the wrong place. The extension module writer knows that this particular extension needs to share symbols, and once the module is loaded it's too late. > It seems that this is a lose-lose situation: you can't please > everybody. In the current state, people that want to share symbols > can, if they really want to. With your proposed change, symbols that > accidentally clash between unrelated extensions cause problems, and > users of those modules can do nothing about it. Hence, the current > state is preferable. So give setdlopenflags a "force" option which overrides the setting designated by the extension module. I realize it's messy (probably too messy). If I could think of some non-messy advice for my users that avoids a language change, I'd like that just as well. -Dave From David Abrahams" <09be01c1f36e$1fe46b20$6401a8c0@boostconsulting.com> Message-ID: <09fe01c1f373$207aec80$6401a8c0@boostconsulting.com> [remembering to delete the ----- Original Message ----- line for Guido this time <1E-15 wink>] I Wrote: > MvL wrote: > > It seems that this is a lose-lose situation: you can't please > > everybody. In the current state, people that want to share symbols > > can, if they really want to. With your proposed change, symbols that > > accidentally clash between unrelated extensions cause problems, and > > users of those modules can do nothing about it. Hence, the current > > state is preferable. > > So give setdlopenflags a "force" option which overrides the setting > designated by the extension module. I realize it's messy (probably too > messy). If I could think of some non-messy advice for my users that > avoids a language change, I'd like that just as well. Come to think of it, no override is needed. If the module won't work without sharing symbols, there's no point in overriding its desire for RTLD_GLOBAL, because it still won't work. So I'm back to suggesting that the module ought to tell python how to load it, period. -Dave From gisle@ActiveState.com Sat May 4 16:32:11 2002 From: gisle@ActiveState.com (Gisle Aas) Date: 04 May 2002 08:32:11 -0700 Subject: [Python-Dev] Problems with Python's default dlopen flags In-Reply-To: <092501c1f30f$337b9280$6401a8c0@boostconsulting.com> References: <092501c1f30f$337b9280$6401a8c0@boostconsulting.com> Message-ID: You should take a look at how pyperl does it. It uses a stub loader that python itself end up loading on 'import perl' and then this stub loader loads the real module with RTLD_GLOBAL. The stub loader is 'dlhack.c'. Get the source from http://downloads.activestate.com/Zope-Perl/pyperl-1.0.1.tar.gz -- Gisle Aas, ActiveState From David Abrahams" Message-ID: <0a6501c1f383$ec4d2a70$6401a8c0@boostconsulting.com> ----- Original Message ----- From: "Gisle Aas" > You should take a look at how pyperl does it. It uses a stub loader > that python itself end up loading on 'import perl' and then this stub > loader loads the real module with RTLD_GLOBAL. > > The stub loader is 'dlhack.c'. Get the source from > http://downloads.activestate.com/Zope-Perl/pyperl-1.0.1.tar.gz AFAICT, this is what I meant when I wrote: "Are you suggesting that in order to do this, my users need to add yet another .so, a thin layer between Python and the guts of their extension modules?" There biggest problem with this arrangement is that my users are creating lots of extension modules. Each one would require a seperate stub loader, thereby doubling the number of shared objects they need to create, complicating distribution and the build process. I'm not familiar with pyperl, but from the source it looks like there's just one Python extension module in this case (perl.so), so the same problems may not apply. -Dave From mal@lemburg.com Sat May 4 18:16:25 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Sat, 04 May 2002 19:16:25 +0200 Subject: [Python-Dev] timeoutsocket.py References: Message-ID: <3CD41769.689FD5D0@lemburg.com> Timothy O'Malley wrote: > > hola. > > I've taken a whack at option #1 (grand unification of socket and > timeoutsocket). > > It's larger than I'd prefer to send in email. If you are interested: > > http://www.timo-tasi.org/python/socket.py Looks nice. One quirk: the 32k constant in the implementation should probably be turned into a class attribute. Depending on the network being used, this would be a parameter to tweak to increase throughput. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From timo@alum.mit.edu Sat May 4 21:15:01 2002 From: timo@alum.mit.edu (Timothy O'Malley) Date: Sat, 04 May 2002 16:15:01 -0400 Subject: [Python-Dev] timeoutsocket.py In-Reply-To: <3CD240EE.7ABCCCE2@3captus.com> Message-ID: hola. That's awesome that you put the timeout functionality into the C-level socket module. I'd vote that it is "The Right"(tm) way to include this feature in the standard library. You guys rock! I couldn't get to the web site you indicated, so pardon this question if I could have found the answer on-line somewhere. How did you handle the _fileobject class in the standard socket.py? The version in Python-2.2 will lose data if the underlying socket raises an exception during a read() or readlines(). Basically, these two routines use a local variable to store the incoming data -- and that local variable will go out of scope after the exception during recv(). I believe that the TimeoutFile class in the socket.py version I thew together fixes this issue. Of course, you may have already noticed and fixed this, too. Again, very awesome! -- TimO "Everything I did in life that was worthwhile I caught hell for." - former U.S. Supreme Court Chief Justice Earl Warren > From: Bernard Yue > Date: Fri, 03 May 2002 01:49:02 -0600 > To: "Timothy O'Malley" > Cc: skip@pobox.com, python-dev@python.org, mgilfix@eecs.tufts.edu > Subject: Re: [Python-Dev] timeoutsocket.py > > Timothy O'Malley wrote: >> >> hola. >> >> I've taken a whack at option #1 (grand unification of socket and >> timeoutsocket). >> >> It's larger than I'd prefer to send in email. If you are interested: >> >> http://www.timo-tasi.org/python/socket.py >> >> -- >> TimO >> > > Tim, this is a complete rewrite of timeoutsocket.py. It is quite an > effort :) > >> 1. Shall we merge socket.py and timeoutsocket.py? >> >> It certainly is possible to merge TimeoutSocket and _socketobject. >> If the merge was done in a straightforward manner, then every TCP >> socket would use Python code (instead of C code) for recv() and >> send() methods. Some might consider this an unacceptable performance >> hit. > > I agree on the performance issue. Over the past two weeks, Michael > Gilfix and I moved ahead with the implementing of timeout socket in C > level. Michael translates what you've done in timeoutsocket.py and put > it into socket.c. So far we did nothing to TimeoutFile. The proposed > patch is basically done (except for the test case). You can find it on > http://www.3captus.com/Downloads/. > > We are now working on the test case for connect() and sendfoo(). > Currently, test for connect() is done by trying a timeout connect to an > external site (same to what you've done). However, we wanted to remove > the external dependency on standard python test suit. So far no luck of > make it working, and I cannot make sendfoo() to raise timeout error at > all. > > Would you like to help us with the test? > > > Bernie > > -- > There are three schools of magic. One: State a tautology, then ring > the changes on its corollaries; that's philosophy. Two: Record many > facts. Try to find a pattern. Then make a wrong guess at the next > fact; that's science. Three: Be aware that you live in a malevolent > Universe controlled by Murphy's Law, sometimes offset by Brewster's > Factor; that's engineering. > -- Robert A. Heinlein From Jack.Jansen@oratrix.com Sat May 4 21:21:28 2002 From: Jack.Jansen@oratrix.com (Jack Jansen) Date: Sat, 4 May 2002 22:21:28 +0200 Subject: [Python-Dev] No "check syntax" command line option Message-ID: <83B832E3-5F9C-11D6-AC62-003065517236@oratrix.com> I just realized (after being prompted by the makers of BBEdit, a very fine programmers editor for the Mac) that Python doesn't appear to have a "check syntax only" switch, like Perl and other languages do have. They'd like to have this, for (you guessed it) their "check syntax" command. Is there a roundabout way to do this? And, if there isn't, is it okay if I have a look at adding one? Suggestions for a switch letter? -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From Oleg Broytmann Sat May 4 21:37:52 2002 From: Oleg Broytmann (Oleg Broytmann) Date: Sun, 5 May 2002 00:37:52 +0400 Subject: [Python-Dev] No "check syntax" command line option In-Reply-To: <83B832E3-5F9C-11D6-AC62-003065517236@oratrix.com>; from Jack.Jansen@oratrix.com on Sat, May 04, 2002 at 10:21:28PM +0200 References: <83B832E3-5F9C-11D6-AC62-003065517236@oratrix.com> Message-ID: <20020505003752.B31748@phd.pp.ru> On Sat, May 04, 2002 at 10:21:28PM +0200, Jack Jansen wrote: > I just realized (after being prompted by the makers of BBEdit, a > very fine programmers editor for the Mac) that Python doesn't > appear to have a "check syntax only" switch, like Perl and other > languages do have. They'd like to have this, for (you guessed > it) their "check syntax" command. > > Is there a roundabout way to do this? pychecker.sf.net Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From fredrik@pythonware.com Sat May 4 22:19:37 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Sat, 4 May 2002 23:19:37 +0200 Subject: [Python-Dev] No "check syntax" command line option References: <83B832E3-5F9C-11D6-AC62-003065517236@oratrix.com> Message-ID: <002701c1f3b1$66b84d80$ced241d5@hagrid> jack wrote: > Is there a roundabout way to do this? python -c "compile(open('$filename').read(), '$filename', 'exec')" works with all versions. From guido@python.org Sat May 4 22:48:10 2002 From: guido@python.org (Guido van Rossum) Date: Sat, 04 May 2002 17:48:10 -0400 Subject: [Python-Dev] No "check syntax" command line option In-Reply-To: Your message of "Sat, 04 May 2002 22:21:28 +0200." <83B832E3-5F9C-11D6-AC62-003065517236@oratrix.com> References: <83B832E3-5F9C-11D6-AC62-003065517236@oratrix.com> Message-ID: <200205042148.g44LmAq05642@pcp742651pcs.reston01.va.comcast.net> > I just realized (after being prompted by the makers of BBEdit, a > very fine programmers editor for the Mac) that Python doesn't > appear to have a "check syntax only" switch, like Perl and other > languages do have. They'd like to have this, for (you guessed > it) their "check syntax" command. > > Is there a roundabout way to do this? It should be trivial to write a script that uses the built-in function compile() on the text of a given module. No need to add another command line switch (that's not "the Python way" :-). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Sat May 4 22:49:19 2002 From: guido@python.org (Guido van Rossum) Date: Sat, 04 May 2002 17:49:19 -0400 Subject: [Python-Dev] No "check syntax" command line option In-Reply-To: Your message of "Sun, 05 May 2002 00:37:52 +0400." <20020505003752.B31748@phd.pp.ru> References: <83B832E3-5F9C-11D6-AC62-003065517236@oratrix.com> <20020505003752.B31748@phd.pp.ru> Message-ID: <200205042149.g44LnJF05723@pcp742651pcs.reston01.va.comcast.net> > > They'd like to have this, for (you guessed > > it) their "check syntax" command. > > > > Is there a roundabout way to do this? > > pychecker.sf.net Um, that does much more than syntax checking. Maybe nice to have but probably not what BBEdit wants. --Guido van Rossum (home page: http://www.python.org/~guido/) From mhammond@skippinet.com.au Sun May 5 05:08:44 2002 From: mhammond@skippinet.com.au (Mark Hammond) Date: Sun, 5 May 2002 14:08:44 +1000 Subject: [Python-Dev] Problems with Python's default dlopen flags In-Reply-To: <092501c1f30f$337b9280$6401a8c0@boostconsulting.com> Message-ID: > I'm hoping I can raise some interest in resolving this problem: Or at least documenting it well :) I don't understand the issues at all, other than my belief that Linux makes it harder than it needs to be. Eg, in the recent PyXPCOM changes, I found that I needed to add the following line of code: // *sob* - seems necessary to open the .so as RTLD_GLOBAL dlopen(PYTHON_SO,RTLD_NOW | RTLD_GLOBAL); Where PYTHON_SO is the base-name of the Python shared library. I wont pretend to understand the issues, but without this flag *my* code worked fine, and I could call Python fine. However, as soon as Python attempted to import a module in a dynamic library, it would fail with "_PyNone_Struct" as unresolved. This includes standard Python extension modules (cStringIO, new, etc) which obviously import fine normally. Now, in this case, "my code" above is actually itself implemented in a .so. (and has had the Python .so linked against it as I can call and use Python itself fine without the extra dlopen()). This .so has been previously dynamically loaded by Mozilla. Presumably this impacts the situation as otherwise my code is doing a bog-standard Python initialization that works perfectly well in its own executable. As I said, I wont even pretend to understand the issues, but I do know that: * The line above made my code work. * Windows works perfectly in this situation, and always has. * It shouldn't be this hard. In the interests of moving this forward, I would be happy to build some test or example code that implements some "interesting but reasonable" solutions using dynamic linking. Eg, Martin's mail in this thread documents how symbol sharing should be implemented using shims etc - however, in my particular situation, I have no special symbol sharing requirements - all I have is a dynamically loaded .so that itself needs to use Python in a .so, but I am having problems with Python itself using other .so modules. Again, note that my .so works fine, and can use Python itself (linked as a .so) just fine, but Python itself is having problems loading .so modules. All this is ActivePython, as Python doesn't (didn't?) support .so creation by default. Mark. From andymac@bullseye.apana.org.au Sun May 5 01:40:09 2002 From: andymac@bullseye.apana.org.au (Andrew MacIntyre) Date: Sun, 5 May 2002 11:40:09 +1100 (edt) Subject: [Python-Dev] Problems with Python's default dlopen flags In-Reply-To: <09be01c1f36e$1fe46b20$6401a8c0@boostconsulting.com> Message-ID: On Sat, 4 May 2002, David Abrahams wrote: > Did you misread my suggestion? I didn't say that RTLD_GLOBAL should be > the default way to load an extension module, only that there should be a > way for the module itself to determine how it's loaded. I don't want to rain on your parade, but some reasearch into dynamic linking loaders proves distressing:- - some always link globally (Windows, OS/2 as I understand); - the others require explicit specification of the method (global/local). The ones that give you the choice require you to specify the method when the first reference to the shared object is made. If you want to change the mode, you have to dereference all entry points and unload the SO before having another go. This turns out to be nightmarish. In addition to the shim module elsewhere referred to, I think that you might also be able to leverage a pure Python wrapper to import the other modules (in effect a specialised version of the import hook wrappers available. Another apprach is to use a "core" module which links all the necessary bits and provides a vector of entry points which can be retrieved via a Python API call (I forget which relatively common module I've seen which does this). -- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au | Snail: PO Box 370 andymac@pcug.org.au | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From David Abrahams" Message-ID: <003201c1f3ed$70dec190$6501a8c0@boostconsulting.com> ----- Original Message ----- From: "Andrew MacIntyre" > On Sat, 4 May 2002, David Abrahams wrote: > > > Did you misread my suggestion? I didn't say that RTLD_GLOBAL should be > > the default way to load an extension module, only that there should be a > > way for the module itself to determine how it's loaded. > > I don't want to rain on your parade, but some reasearch into dynamic > linking loaders proves distressing:- > > - some always link globally (Windows, OS/2 as I understand); > - the others require explicit specification of the method (global/local). This information is not new to me; I'm aware of the differences. I'm quite familiar with the Windows model, and it's not quite global linking in the same sense as on Unix: you explicitly specify which symbols are designated for sharing. > The ones that give you the choice require you to specify the method when > the first reference to the shared object is made. If you want to change > the mode, you have to dereference all entry points and unload the SO > before having another go. This turns out to be nightmarish. AFAICT that would not be a nightmare in the scenario I'm suggesting. The initial dlopen wouldn't use RTLD_GLOBAL, so there would only be one or two entry points to dereference before unloading the module and trying again. > In addition to the shim module elsewhere referred to, I think that you > might also be able to leverage a pure Python wrapper to import the other > modules (in effect a specialised version of the import hook wrappers > available. Yes, (as I'll repeat once again) this approach puts the burden on users of the extension module to know that it needs to be imported in a special way. That's just wrong. > Another apprach is to use a "core" module which links all the necessary > bits and provides a vector of entry points which can be retrieved via a > Python API call (I forget which relatively common module I've seen which > does this). That won't work for C++, and even if it could, it doesn't conform to my users' needs. The symbols (not just entry points) which the compiler generates to support RTTI and exception-handling aren't normally available to users. -Dave From xscottg@yahoo.com Sun May 5 05:32:19 2002 From: xscottg@yahoo.com (Scott Gilbert) Date: Sat, 4 May 2002 21:32:19 -0700 (PDT) Subject: [Python-Dev] buffer objects In-Reply-To: Message-ID: <20020505043219.366.qmail@web12907.mail.yahoo.com> --- Tim Peters wrote: > > Note that "the buffer object problem" is an annual pointless > discussion. The last big round on Python-Dev was near the start > of June 2001, under the unlikely Subject "strop vs. string". > Nobody volunteered to do anything then either, although one > person made a big show of agreeing to plug the holes, and then > didn't. That's how it usually ends. > > If you want to "do something" here, I suggest reading the old > threads to get coverage of the outstanding issues. Nothing new > has been said or suggested in years. > Lest I become associated with other do nothing whiners, I've submitted a (bug fix only) patch that addresses the following issues: 1) Dangling pointer problem 2) buffer allocated by PyBuffer_New not aligned It doesn't fix these: 3) The hash is cached, and can be made invalid 4) An int is not big enough for 64 bit pointers 5) Can't handle multiple segments It introduces: 6) Working with the buffer object that refers to a separate "base" object might be slower. Issue #2 only aligns it to the alignment of doubles. It's probably not worth the effort to promise more, but I'll revisit that (and do it the way you suggest in bug 472568) if you think it is. Issue #3 is easy to fix IMNSHO. My first choice would be to just delete the hash function in there. My second would be to have the hash delegate to the contained "base" object. Any of these could probably break code somewhere though, and that sounds like a Guido call to me... Issue #4 is too big to worry about here. PyBufferProcs don't return a LONG_LONG, and changing that WOULD break a lot of code. Issue #5 is confusing for me, and probably a YAGNI for most everyone. I don't know what the correct behavior is. Are you supposed to consider the discontiguous segments as one contiguous buffer? If so, then it could be fixed, but it probably double the code length... Issue #6 is probably a non-issue. I haven't profiled it, but calling PyBufferProcs to get the pointers each time is probably not that expensive. Since this object is really only useful from Python script, loop overhead would probably dominate any true to life problem. Are there any other problems in PyBufferObject that I'm missing? I searched SourceForge, but searching for "buffer" returns a lot of false hits, and searching for PyBufferObject only returns 2. I hope I'm not out of line, but since you seem to be aware of the issues, I've assigned this patch to you. I was hoping to test it on Unix Friday at work, but a long lunch led to seeing Spiderman led to an early happy hour led to Cinco de Mayo led to a hangover today. Since my reputation as a non-whiner is on the line here, I think I'll submit it now rather than wait til Monday to test it on Unix. It passes the regression tests on Win32. I'm also assigning my feature patch to you. If you don't find any problems with my bug fixes above, then there shouldn't be any reason why the buffer builtin can't return a read-write PyBufferObject. BTW: It's pretty easy to understand why this thing never gets fixed. As soon as you discover what the problems are, you realize that even if they are fixed, this thing isn't very applicable to what you were hoping to use it for. I'll submit a PEP for what I think would be more usable... Cheers, -Scott __________________________________________________ Do You Yahoo!? Yahoo! Health - your guide to health and wellness http://health.yahoo.com From martin@v.loewis.de Sun May 5 08:26:58 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 05 May 2002 09:26:58 +0200 Subject: [Python-Dev] Problems with Python's default dlopen flags In-Reply-To: <09be01c1f36e$1fe46b20$6401a8c0@boostconsulting.com> References: <092501c1f30f$337b9280$6401a8c0@boostconsulting.com> <09be01c1f36e$1fe46b20$6401a8c0@boostconsulting.com> Message-ID: "David Abrahams" writes: > Did you misread my suggestion? I didn't say that RTLD_GLOBAL should be > the default way to load an extension module, only that there should be a > way for the module itself to determine how it's loaded. I dismissed your suggestion as being too complex. There are a number of questions involved which I cannot answer that may effect usability of this approach; most of them have to do with dlclosing the library: 1. If the extension module is C++ code, dlopening the module will run constructors for global objects. dlclosing it will run destructors. So the dlopen/dlclose/dlopen cycle might have side effects; that might be confusing. 2. Again, with C++ code, on Linux, with gcc 2.95.x, a block-local static object will register its destructor with atexit(3). When the module is dlclosed, the code to be called at exit goes away; then the program crashes atexit. This is undesirable. 3. If the module is also used as a library that some other module links against, I'm not sure what the semantics of dlclose is. I'd feel uncomfortable with such a feature if I don't know precisely how it acts in boundary cases. > And in fact, I expect to ask users to do something special, like > explicitly linking between extension modules Indeed, that should also work fine - if users explicitly link extension modules against each other, they should be able to share symbols. The need for RTLD_GLOBAL only occurs when they want to share symbols, but don't want to link the modules against each other. > However, this is what I didn't expect: the lack of RTLD_GLOBAL flags > interferes with the ability for ext1.so to catch C++ exceptions > thrown by libboost_python.so! That is surprising indeed, and hard to believe. Can you demonstrate that in a small example? > Are you suggesting that in order to do this, my users need to add > yet another .so, a thin layer between Python and the guts of their > extension modules? Originally, that's what I suggested. I now think that, for symbol sharing, linking the modules against each other should be sufficient. > > Now, people still want to share symbols across modules. For that, you > > can use CObjects: Export a CObject with an array of function pointers > > in module A (e.g. as A.API), and import that C object in module B's > > initialization code. See cStringIO and Numeric for examples. > > Of course you realize that won't help with C++ exception tables... Actually, I don't: I can't see what C++ exception tables have to do with it - the exception regions are local in any case. > ...which leads us back to the fact that the smarts are in the wrong > place. The extension module writer knows that this particular > extension needs to share symbols, and once the module is loaded it's > too late. The extension module writer can't possibly have this knowledge - to know whether it is _safe_ to share symbols, you have to know the complete set of extension modules in the application. If a single module uses your proposed feature, it would export its symbols to all other extensions - whether they want those symbols or not. Hence you might still end up with a situation where you can't use two extensions in a single application because of module clashes. > So give setdlopenflags a "force" option which overrides the setting > designated by the extension module. I realize it's messy (probably too > messy). If I could think of some non-messy advice for my users that > avoids a language change, I'd like that just as well. For that, I'd need to understand the problem of your users first. I'm unhappy to introduce work-arounds for incompletely-understood problems. Regards, Martin From martin@v.loewis.de Sun May 5 08:39:39 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 05 May 2002 09:39:39 +0200 Subject: [Python-Dev] Problems with Python's default dlopen flags In-Reply-To: References: Message-ID: "Mark Hammond" writes: > I don't understand the issues at all, other than my belief that Linux makes > it harder than it needs to be. Eg, in the recent PyXPCOM changes, I found > that I needed to add the following line of code: > > // *sob* - seems necessary to open the .so as RTLD_GLOBAL > dlopen(PYTHON_SO,RTLD_NOW | RTLD_GLOBAL); > > Where PYTHON_SO is the base-name of the Python shared library. I have a number of problems following your description, so let me try to guess. In no released version of Python, libpython is build as a shared library. However, I'll assume that you have build libpythonxy.so as a shared library, and that this library provides the symbols that normally the Python executable provides. I further assume that the dlopen call is not in the code of the Python interpreter itself, but somewhere inside Mozilla (I don't know what PyXPCOM is). In that case, using RTLD_GLOBAL is one option. It is quite safe to use here, since it clutters the global namespace primarily with symbols that all start with Py; those are unlikely to clash with other symbols. Using RTLD_GLOBAL is needed since extension modules search the Py symbols in the global namespace; they normally come from the executable. The other option is to explicitly link all extension modules against your libpythonxy.so. If that is done, you can drop the RTLD_GLOBAL. Of course, the standard build process won't link extension modules with libpythonxy.so, since that library does not even exist in the standard build process. > As I said, I wont even pretend to understand the issues, but I do know that: > * The line above made my code work. Yes, see above. > * Windows works perfectly in this situation, and always has. This is because on Windows, every extension module is linked against pythonxy.dll, to find the extension symbols. > * It shouldn't be this hard. I don't think it is hard. I feel that the situation on Windows is much worse - to build an extension module, you need the import library, and you need to link against it, and you thus hard-code the version of the interpreter into the extension module, and you thus cannot use extension modules across Python releases. There are trade-offs on both sides. Regards, Martin From martin@v.loewis.de Sun May 5 08:49:20 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 05 May 2002 09:49:20 +0200 Subject: [Python-Dev] Lazily GC tracking tuples In-Reply-To: <20020504092554.37178.qmail@web14706.mail.yahoo.com> References: <20020504092554.37178.qmail@web14706.mail.yahoo.com> Message-ID: Daniel Dunbar writes: > Lazily tracking the tuples seems to be a much simpler > proposition This proposal sounds good, but there are a few issues. > and PyTuple_SET_ITEM and PyTuple_SetItem are modified This is a problem: PyTuple_SET_ITEM is compiled into the extension module, so you need to bump the API version (or else old modules would be used that do not have your change). > In theory if v is a tuple, _SetItem(o,i,v) should only > be called when v has been completely filled, so if v > is untracked then we know it also cannot be member of a > cycle, which solves the nested tuple problem. I think it is safe to make this assumption. The worst case is that you get cyclic garbage that won't be collected. Given that it is a bug to refer to an "incomplete" tuple, I think this consequence of this bug is acceptable. > A 2.23 patch for the changes outlined above are at: > http://www.geocities.com/evilzr/lazytuplegc/ > (I wasn't sure if such things are appropriate for > the SF-patch manager - yes/no?) If you think this patch is reasonably complete, and you wish that it is included in Python, then yes, put it onto SF. Regards, Martin From mhammond@skippinet.com.au Sun May 5 10:05:24 2002 From: mhammond@skippinet.com.au (Mark Hammond) Date: Sun, 5 May 2002 19:05:24 +1000 Subject: [Python-Dev] Problems with Python's default dlopen flags In-Reply-To: Message-ID: [Martin] > I have a number of problems following your description, so let me try > to guess. In no released version of Python, libpython is build as a > shared library. However, I'll assume that you have build > libpythonxy.so as a shared library, and that this library provides the > symbols that normally the Python executable provides. Thanks for the note! I now realize I left critical information deep in the body - I was using ActivePython as it is the only released version of Python that comes with a version built as a shared library (along with a traditional static one) > In that case, using RTLD_GLOBAL is one option. It is quite safe to use > here, since it clutters the global namespace primarily with symbols > that all start with Py; those are unlikely to clash with other > symbols. Using RTLD_GLOBAL is needed since extension modules search > the Py symbols in the global namespace; they normally come from the > executable. OK - cool. So you are saying that what I have done is reasonable (whereas I assumed it was a nasty hack :) > The other option is to explicitly link all extension modules against > your libpythonxy.so. If that is done, you can drop the RTLD_GLOBAL. > Of course, the standard build process won't link extension modules > with libpythonxy.so, since that library does not even exist in the > standard build process. Right. Notwithstanding that the distribution I used *does* have a libpython, I see that the same extension modules are used by the static and dynamic versions of the core, so therefore can't have that reference. Thanks for the info - strangely enough, it appears it is not a "problem" at all ;) Mark. From David Abrahams" <09be01c1f36e$1fe46b20$6401a8c0@boostconsulting.com> Message-ID: <008b01c1f439$7538e260$6501a8c0@boostconsulting.com> From: "Martin v. Loewis" > "David Abrahams" writes: > > > Did you misread my suggestion? I didn't say that RTLD_GLOBAL should be > > the default way to load an extension module, only that there should be a > > way for the module itself to determine how it's loaded. > > I dismissed your suggestion as being too complex. Fair enough [explicit dismissal helps to reduce confusion]. > There are a number > of questions involved which I cannot answer that may effect usability > of this approach; most of them have to do with dlclosing the library: > > 1. If the extension module is C++ code, dlopening the module will run > constructors for global objects. dlclosing it will run destructors. > So the dlopen/dlclose/dlopen cycle might have side effects; that > might be confusing. and quite possibly unacceptable to users; granted. I hadn't thought of that. > 2. Again, with C++ code, on Linux, with gcc 2.95.x, a block-local > static object will register its destructor with atexit(3). When > the module is dlclosed, the code to be called at exit goes away; > then the program crashes atexit. This is undesirable. and absolutely unacceptable to me, so I guess that approach is out. > 3. If the module is also used as a library that some other module > links against, I'm not sure what the semantics of dlclose is. I'd > feel uncomfortable with such a feature if I don't know precisely > how it acts in boundary cases. I would hope that it would be nicely reference-counted, but if *you* don't know that answer I'm guessing it's undocumented and I wouldn't want to count on that either. > > And in fact, I expect to ask users to do something special, like > > explicitly linking between extension modules > > Indeed, that should also work fine - if users explicitly link > extension modules against each other, they should be able to share > symbols. The need for RTLD_GLOBAL only occurs when they want to share > symbols, but don't want to link the modules against each other. Heh, that's what I'd have thought, too. > > However, this is what I didn't expect: the lack of RTLD_GLOBAL flags > > interferes with the ability for ext1.so to catch C++ exceptions > > thrown by libboost_python.so! > > That is surprising indeed, and hard to believe. Can you demonstrate > that in a small example? Unfortunately, the only reproducible case we have is not exactly small. However, we can give anyone interested full access to the machine and test case where it's occurring [details appended at the bottom of this message]. > > Are you suggesting that in order to do this, my users need to add > > yet another .so, a thin layer between Python and the guts of their > > extension modules? > > Originally, that's what I suggested. I now think that, for symbol > sharing, linking the modules against each other should be sufficient. > > > > Now, people still want to share symbols across modules. For that, you > > > can use CObjects: Export a CObject with an array of function pointers > > > in module A (e.g. as A.API), and import that C object in module B's > > > initialization code. See cStringIO and Numeric for examples. > > > > Of course you realize that won't help with C++ exception tables... > > Actually, I don't: I can't see what C++ exception tables have to do > with it - the exception regions are local in any case. You're right, I mis-spoke: it has nothing to do with exception tables. It's an RTTI problem: the type-specific catch clause exception-handler doesn't catch the thrown exception. > > ...which leads us back to the fact that the smarts are in the wrong > > place. The extension module writer knows that this particular > > extension needs to share symbols, and once the module is loaded it's > > too late. > > The extension module writer can't possibly have this knowledge - to > know whether it is _safe_ to share symbols, you have to know the > complete set of extension modules in the application. Yes, but if the module /requires/ symbol sharing it sort of doesn't matter whether it's safe. If you don't share symbols, the module won't work (and will probably crash), just as when you share symbols and there's a clash you'll probably get a crash. > If a single > module uses your proposed feature, it would export its symbols to all > other extensions - whether they want those symbols or not. Hence you > might still end up with a situation where you can't use two extensions > in a single application because of module clashes. Yep. The existence of namespaces in the C++ case may mitigate the situation somewhat, but I understand the problem. > > So give setdlopenflags a "force" option which overrides the setting > > designated by the extension module. I realize it's messy (probably too > > messy). If I could think of some non-messy advice for my users that > > avoids a language change, I'd like that just as well. > > For that, I'd need to understand the problem of your users first. I'm > unhappy to introduce work-arounds for incompletely-understood > problems. I agree with that attitude. And after all, my suggestion was really just a straw-man. --------- Environment: RedHat 7.1 on Intel gcc 3.0.4 used for everything, incl. Python compilation. All our code (but not Python) is compiled with -g. Example compile and link lines: g++ -fPIC -ftemplate-depth-50 -DNDEBUG -w -g -I"/net/cci/rwgk/phenix/in clude" -I"/net/cci/rwgk/cctbx/include" -I"/net/cci/rwgk/boost" -I/usr/l ocal_cci/Python-2.2.1_gcc304/include/python2.2 -c refinementtbxmodule.cpp g++ -shared -Wl,-E -o refinementtbx.so refinementtbxmodule.o error.o -L/net/taipan/scratch1/rwgk/py221gcc304/lib -lboost_python -lm Here is the core of where things go wrong: (part of the Boost.Python library) try { PyObject* const result = f->do_call(args, keywords); if (result != 0) return result; } catch(const argument_error&) { // SHOULD COME BACK HERE } catch(...) // this catch clause inserted for debugging { // BUT COMES BACK HERE throw; } The exact some source works with Compaq cxx on Alpha (and also Visual C++ 6 & 7, Win32 CodeWarrior 7.0). I put in print statements and compared the output from the Linux machine with the output from the Alpha. This shows that under Linux the identity of the exception is somehow lost: Alpha comes back where it says "SHOULD COME BACK HERE", Linux comes back where it says "BUT COMES BACK HERE". This is after many successful passes through "SHOULD COME BACK HERE". The problem is very elusive. Most of the time the exception handling works correctly. For example, if we change small details about the order in which exceptions are thrown gcc works just fine (execution flow still passes repeatedly through "SHOULD COME BACK HERE"). Also, changing the dlopen flags used when loading extension modules to include RTLD_GLOBAL makes the problem disappear. It is very difficult to generate a simple test case (our application requires both Python and Boost). I have spent days trying to isolate what is going wrong, unfortunately to no avail. Suggestion: I (Ralf) could set up an account for you on our machines. Using this account you would have direct access to all the source code files and compiled binaries that are needed to reproduce our problem. You could directly enter gdb to investigate. From tim.one@comcast.net Sun May 5 18:07:21 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 05 May 2002 13:07:21 -0400 Subject: [Python-Dev] Problems with Python's default dlopen flags In-Reply-To: <008b01c1f439$7538e260$6501a8c0@boostconsulting.com> Message-ID: [David Abrahams, presumably quoting Ralf W. Grosse-Kunstleve] > ... > The exact some source works with Compaq cxx on Alpha (and also Visual > C++ 6 & 7, Win32 CodeWarrior 7.0). I put in print statements and > compared the output from the Linux machine with the output from the > Alpha. This shows that under Linux the identity of the exception is > somehow lost: Alpha comes back where it says "SHOULD COME BACK HERE", > Linux comes back where it says "BUT COMES BACK HERE". This is after > many successful passes through "SHOULD COME BACK HERE". > > The problem is very elusive. Most of the time the exception handling > works correctly. For example, if we change small details about the order > in which exceptions are thrown gcc works just fine > (execution flow still passes repeatedly through "SHOULD COME BACK > HERE"). Also, changing the dlopen flags used when loading extension > modules to include RTLD_GLOBAL makes the problem disappear. Whoa -- this shows all the signs of a wild store and consequent memory corruption. If it were truly a problem with resolving symbols, it would fail every time. Instead it sometimes works, sometimes doesn't, and all depending on "stuff that shouldn't matter". Try efence? From gmcm@hypernet.com Sun May 5 20:33:16 2002 From: gmcm@hypernet.com (Gordon McMillan) Date: Sun, 5 May 2002 15:33:16 -0400 Subject: [Python-Dev] Problems with Python's default dlopen flags In-Reply-To: References: <008b01c1f439$7538e260$6501a8c0@boostconsulting.com> Message-ID: <3CD550BC.13523.3DD3A761@localhost> On 5 May 2002 at 13:07, Tim Peters wrote: > [David Abrahams, presumably quoting Ralf W. > Grosse-Kunstleve] > [stuff about exceptions on Linux...] > Whoa -- this shows all the signs of a wild store > and consequent memory corruption. If it were truly > a problem with resolving symbols, it would fail > every time. Instead it sometimes works, sometimes > doesn't, and all depending on "stuff that shouldn't > matter". There is some strangeness to exceptions, Linux, gcc and linking. In scxx (my minimalist C++ / Python interface), there's no separate .so involved - the scxx code is compiled in with the extension. There are no statics involved, so C linkage works (you don't need a relinked Python). At a certain gcc release, exceptions thrown and caught at the top level stopped working (abort). "eric" of scipy fame had a similar (but not identical) experience. I think scipy's fix was to require Python be built and linked by g++. Mine was to stop doing that (throwing and catching at the same level). So we all have gcc and C++ exceptions and linkage in common. Leg 4 of the elephant is out there someplace. -- Gordon http://www.mcmillan-inc.com/ From rwgk@cci.lbl.gov Sun May 5 20:45:13 2002 From: rwgk@cci.lbl.gov (Ralf W. Grosse-Kunstleve) Date: Sun, 5 May 2002 12:45:13 -0700 (PDT) Subject: [Python-Dev] Problems with Python's default dlopen flags Message-ID: <200205051945.g45JjDv454621@boa.lbl.gov> > > Unfortunately, the only reproducible case we have is not exactly small. > > However, we can give anyone interested full access to the machine and > > test case where it's occurring [details appended at the bottom of this > > message]. That message is referring to a slightly different situation: I am using a statically linked libboost_python.a. When I wrote the message that David attached I only had a very complex example. Now I have a significantly simpler one, and this is what I think is happening: python -> dlopen ext1.so with statically linked libboost_python.a python -> dlopen ext2.so with statically linked libboost_python.a obj1 = ext1.a_type() obj1 has a member function which will dispatch to one of two alternative C++ function. In the course of resolving which C++ function to use an exception is raised in ext2.so that is meant to be caught in ext1.so. If both extension modules are imported with RTLD_LOCAL the exception is not caught correctly. However, when using RTLD_GLOBAL this works properly. Considering the discussion in this thread it is now pretty clear that this behavior is expected. So... I have just done the step that I should apparently have done months ago (this is for how long the problem is bugging me): Compiling our extension modules against a libboost_python.so. With that my example works even without changing setdlopenflags. I am obviously learning the hard way here. At least things are starting to make sense now. And it seems as if RTLD_LOCAL will work for a shared Boost.Python library. Thanks for sharing your valuable knowledge about dynamic linking. Ralf From Jack.Jansen@oratrix.com Sun May 5 21:10:32 2002 From: Jack.Jansen@oratrix.com (Jack Jansen) Date: Sun, 5 May 2002 22:10:32 +0200 Subject: [Python-Dev] Problems with Python's default dlopen flags In-Reply-To: Message-ID: <27A5C4F3-6064-11D6-A113-003065517236@oratrix.com> On zondag, mei 5, 2002, at 06:08 , Mark Hammond wrote: >> I'm hoping I can raise some interest in resolving this problem: > > Or at least documenting it well :) > > I don't understand the issues at all, other than my belief that > Linux makes > it harder than it needs to be. If I understand correctly it isn't Linux, it's the Python way of dynamic loading which makes it more difficult. The way Python does dynamic loading is modeled after how old Unix dynamic loaders worked (which was mainly a dirty hack without much design): you load a .so and at load time it will invoke a special version of the "ld" link editor which will at that time do symbol resolving, etc. As this "magic resolving symbols from what happens to be the loading application" is so ubiquitous in Unix most current unix variants support it, one way or another. This is a whole different situation from Windows and MacOS, which are in this respect more modern, and have a conceptually more simple model: you always link against a stub library, and at runtime the symbols are always taken from the corresponding dynamic library. This is a lot safer (no more symbols that are accidentally satisfied from a completely unexpected library, version checking, etc) but it can make a few things more difficult, such as dynamic modules that refer to each other, or loading extension modules into a statically linked program. -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From Jack.Jansen@oratrix.com Sun May 5 21:20:21 2002 From: Jack.Jansen@oratrix.com (Jack Jansen) Date: Sun, 5 May 2002 22:20:21 +0200 Subject: [Python-Dev] Problems with Python's default dlopen flags In-Reply-To: Message-ID: <861BB3F0-6065-11D6-A113-003065517236@oratrix.com> On zondag, mei 5, 2002, at 09:26 , Martin v. Loewis wrote: >> And in fact, I expect to ask users to do something special, like >> explicitly linking between extension modules > > Indeed, that should also work fine - if users explicitly link > extension modules against each other, they should be able to share > symbols. From experience I can say this is not a good idea. MacOS extension modules used to do this until Python 1.5.2, and it can have undesired side effects (similar to the C++ initializer problems you noted). If you need to communicate symbols between module A and B it's better to make a normal shared library ABglue.so which contains the glue code, and link both A and B against it. The only question is where to put the ABglue.so file, on Mac and some (most?) unixen it's probably okay to drop it in lib-dynload, but it could be you need some fiddling of the flags that control the shared library search path for A and B. I think on Windows you're stuck with putting it in the system directory (at least, I assume that that's why PyWinTypes is there). -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From David Abrahams" Message-ID: <005901c1f477$ee178950$6501a8c0@boostconsulting.com> ----- Original Message ----- From: "Ralf W. Grosse-Kunstleve" > Thanks for sharing your valuable knowledge about dynamic linking. ...and my apologies for wasting bandwidth here on python-dev. I had asked Ralf specific questions that would have led to my understanding what he was doing wrong, but there was apparently a miscommunication somewhere. I guess the lesson is not to bring bugs forward unless I can reproduce them myself. re-lurking-ly y'rs, dave From David Abrahams" <3CD550BC.13523.3DD3A761@localhost> Message-ID: <008501c1f478$a1cbdeb0$6501a8c0@boostconsulting.com> From: "Gordon McMillan" > There is some strangeness to exceptions, Linux, gcc > and linking. In scxx (my minimalist C++ / Python > interface), there's no separate .so involved - the > scxx code is compiled in with the extension. There > are no statics involved, so C linkage works (you don't > need a relinked Python). At a certain gcc release, > exceptions thrown and caught at the top level What does "at the top level" mean? > stopped > working (abort). "eric" of scipy fame had a > similar (but not identical) experience. > > I think scipy's fix was to require Python be built and > linked by g++. Mine was to stop doing that (throwing > and catching at the same level). Same question, I guess: what is a "level"? -Dave From David Abrahams" Message-ID: <009b01c1f479$558934c0$6501a8c0@boostconsulting.com> From: "Jack Jansen" > On zondag, mei 5, 2002, at 09:26 , Martin v. Loewis wrote: > >> And in fact, I expect to ask users to do something special, like > >> explicitly linking between extension modules > > > > Indeed, that should also work fine - if users explicitly link > > extension modules against each other, they should be able to share > > symbols. > > From experience I can say this is not a good idea. MacOS > extension modules used to do this until Python 1.5.2, and it can > have undesired side effects (similar to the C++ initializer > problems you noted). If you need to communicate symbols between > module A and B it's better to make a normal shared library > ABglue.so which contains the glue code, and link both A and B > against it. The only question is where to put the ABglue.so > file, That's hardly the only question: If A needs to throw an exception which is caught in B, what is "the glue code" that you put in ABglue.so? If a class with inlined virtual functions is used by A with subclasses defined in B, what is the "glue code"? What about templates? ;-) (vague, but relevant) -Dave From martin@v.loewis.de Sun May 5 22:45:39 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 05 May 2002 23:45:39 +0200 Subject: [Python-Dev] Problems with Python's default dlopen flags In-Reply-To: <3CD550BC.13523.3DD3A761@localhost> References: <008b01c1f439$7538e260$6501a8c0@boostconsulting.com> <3CD550BC.13523.3DD3A761@localhost> Message-ID: "Gordon McMillan" writes: > There is some strangeness to exceptions, Linux, gcc > and linking. In scxx (my minimalist C++ / Python > interface), there's no separate .so involved - the > scxx code is compiled in with the extension. There > are no statics involved, so C linkage works (you don't > need a relinked Python). At a certain gcc release, > exceptions thrown and caught at the top level stopped > working (abort). I think Tim's analysis is right: If it fails every time, there is some conceptual problem somewhere - this one sounds like a bug in gcc, which might fail to emit exception regions correctly or some such (I think I recall one such bug in g++). If it sometimes fails, sometimes "succeed", it rather sounds like memory corruption. I can well imagine how memory corruption could affect RTTI: either the vtable pointer in an object gets overwritten, or the RTTI objects themselves get overwritten. Regards, Martin From martin@v.loewis.de Sun May 5 22:50:21 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 05 May 2002 23:50:21 +0200 Subject: [Python-Dev] Problems with Python's default dlopen flags In-Reply-To: <861BB3F0-6065-11D6-A113-003065517236@oratrix.com> References: <861BB3F0-6065-11D6-A113-003065517236@oratrix.com> Message-ID: Jack Jansen writes: > From experience I can say this is not a good idea. MacOS extension > modules used to do this until Python 1.5.2, and it can have undesired > side effects (similar to the C++ initializer problems you noted). If > you need to communicate symbols between module A and B it's better to > make a normal shared library ABglue.so which contains the glue code, > and link both A and B against it. I completely agree that this is the best thing to do, as it is most portable. On Linux (and probably all other SysV ELF systems), you can get away with linking against an extension module. > The only question is where to put the ABglue.so file, on Mac and > some (most?) unixen it's probably okay to drop it in lib-dynload, > but it could be you need some fiddling of the flags that control the > shared library search path for A and B. On Unix, you'll need to set LD_LIBRARY_PATH, or put the library into a directory where ld.so automatically searches for libraries; that is a deployment problem. > I think on Windows you're stuck with putting it in the system > directory (at least, I assume that that's why PyWinTypes is there). Not necessarily: Any directory on PATH will do, plus the directory where the executable is (i.e. c:\python22). For some reason, c:\python22\DLLs is also searched, although I'm not sure what magic arranges that. Regards, Martin From David Abrahams" <3CD550BC.13523.3DD3A761@localhost> Message-ID: <011501c1f480$491ca5d0$6501a8c0@boostconsulting.com> ----- Original Message ----- From: "Martin v. Loewis" > I think Tim's analysis is right: If it fails every time, there is some > conceptual problem somewhere - this one sounds like a bug in gcc, > which might fail to emit exception regions correctly or some such (I > think I recall one such bug in g++). Yes, 2.95.x definitely has EH bugs. 3.0.x is much better, AFAICT. -Dave From Jack.Jansen@oratrix.com Sun May 5 23:09:33 2002 From: Jack.Jansen@oratrix.com (Jack Jansen) Date: Mon, 6 May 2002 00:09:33 +0200 Subject: [Python-Dev] Problems with Python's default dlopen flags In-Reply-To: <009b01c1f479$558934c0$6501a8c0@boostconsulting.com> Message-ID: On zondag, mei 5, 2002, at 11:08 , David Abrahams wrote: > From: "Jack Jansen" > >> On zondag, mei 5, 2002, at 09:26 , Martin v. Loewis wrote: >>>> And in fact, I expect to ask users to do something special, like >>>> explicitly linking between extension modules >>> >>> Indeed, that should also work fine - if users explicitly link >>> extension modules against each other, they should be able to share >>> symbols. >> >> From experience I can say this is not a good idea. MacOS >> extension modules used to do this until Python 1.5.2, and it can >> have undesired side effects (similar to the C++ initializer >> problems you noted). If you need to communicate symbols between >> module A and B it's better to make a normal shared library >> ABglue.so which contains the glue code, and link both A and B >> against it. The only question is where to put the ABglue.so >> file, > > That's hardly the only question: > > If A needs to throw an exception which is caught in B, what is > "the glue > code" that you put in ABglue.so? > > If a class with inlined virtual functions is used by A with subclasses > defined in B, what is the "glue code"? > > What about templates? ;-) (vague, but relevant) Well, you're now going into territory where I'm not really familiar, C++, and more precisely it's runtime system on various platforms (especially the stuff with zero-overhead runtimes and such tends to make my head explode), but I would guess that if you take the code from A that declares symbols used by B and vice versa and move that to a common ancestor you should be done. If all else fails you could even take the whole A module except the init routine and stuff it in Acore.so, similarly for B, link Acore.so against Bcore.so and vice versa and have the A and B modules be skeletons with just the init routine. That way you know that your Acore and Bcore will be loaded via the normal shared library loading method, which probably handles initializers and cross-segment exceptions correctly, -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From martin@v.loewis.de Sun May 5 23:20:12 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 06 May 2002 00:20:12 +0200 Subject: [Python-Dev] Problems with Python's default dlopen flags In-Reply-To: <200205051945.g45JjDv454621@boa.lbl.gov> References: <200205051945.g45JjDv454621@boa.lbl.gov> Message-ID: "Ralf W. Grosse-Kunstleve" writes: > python -> dlopen ext1.so with statically linked libboost_python.a > python -> dlopen ext2.so with statically linked libboost_python.a That explains a lot of things indeed. It doesn't explain why the exception handling on Linux fails (that should still work fine even with two separate copy of each typeinfo object, IMO), but it gives a clue as to what things may have broken: you get two copies of each global object, and gives you access to both copies. Depending on the exact interaction sequence, it is well possible that all kinds of corruption occur. Regards, Martin From David Abrahams" Message-ID: <013e01c1f482$c9fab0a0$6501a8c0@boostconsulting.com> From: "Jack Jansen" > > On zondag, mei 5, 2002, at 11:08 , David Abrahams wrote: > > > That's hardly the only question: > > > > If A needs to throw an exception which is caught in B, what is > > "the glue > > code" that you put in ABglue.so? > > > > If a class with inlined virtual functions is used by A with subclasses > > defined in B, what is the "glue code"? > > > > What about templates? ;-) (vague, but relevant) > > Well, you're now going into territory where I'm not really > familiar, C++, and more precisely it's runtime system on various > platforms (especially the stuff with zero-overhead runtimes and > such tends to make my head explode), but I would guess that if > you take the code from A that declares symbols used by B and > vice versa and move that to a common ancestor you should be done. Speculation aside, the RTTI info which allows the compiler to identify exceptions is never explicitly declared by the user. In some implementations it's generated automatically wherever the first declared virtual function is implemented. I don't know what the rules are when there are no virtual functions, or the virtual functions are inlinied, or the thrown type is a tempalte which is instantiated implicitly... of course this is OT for Python-dev but I'm hoping now that I've got the attention of MvL (a gcc developer) he'll answer these questions or direct me to someone who can ;-) > If all else fails you could even take the whole A module except > the init routine and stuff it in Acore.so, similarly for B, link > Acore.so against Bcore.so and vice versa and have the A and B > modules be skeletons with just the init routine. That way you > know that your Acore and Bcore will be loaded via the normal > shared library loading method, which probably handles > initializers and cross-segment exceptions correctly, Yes, now we're back to MvL's suggestion to use shim libraries, which provides very certain behavior but also complicates building and deployment. -Dave From David Abrahams" Message-ID: <017b01c1f484$e4692140$6501a8c0@boostconsulting.com> From: "Martin v. Loewis" > "Ralf W. Grosse-Kunstleve" writes: > > > python -> dlopen ext1.so with statically linked libboost_python.a > > python -> dlopen ext2.so with statically linked libboost_python.a > > That explains a lot of things indeed. It doesn't explain why the > exception handling on Linux fails (that should still work fine even > with two separate copy of each typeinfo object, IMO), Not the way I read http://gcc.gnu.org/faq.html#dso. Am I missing something? If an exception is thrown from ext1 -> ext2 and they're not sharing symbols, there will be distinct copies of all typeinfo objects used in the two modules, and the address comparisons used to determine whether a catch clause matches ought to fail, no? > but it gives a > clue as to what things may have broken: you get two copies of each > global object, and gives you access to both copies. Depending on the > exact interaction sequence, it is well possible that all kinds of > corruption occur. I don't think Ralf is explicitly using any non-const global objects or explicitly relying on on object identity across extension modules, so it's hard to imagine that this is at play. -Dave From gmcm@hypernet.com Mon May 6 00:02:58 2002 From: gmcm@hypernet.com (Gordon McMillan) Date: Sun, 5 May 2002 19:02:58 -0400 Subject: [Python-Dev] Problems with Python's default dlopen flags In-Reply-To: <008501c1f478$a1cbdeb0$6501a8c0@boostconsulting.com> Message-ID: <3CD581E2.11068.3E93A595@localhost> On 5 May 2002 at 16:01, David Abrahams wrote: > From: "Gordon McMillan" > > > There is some strangeness to exceptions, Linux, gcc > > and linking. In scxx (my minimalist C++ / Python > > interface), there's no separate .so involved - the > > scxx code is compiled in with the extension. There > > are no statics involved, so C linkage works (you > > don't need a relinked Python). At a certain gcc > > release, exceptions thrown and caught at the top > > level > > What does "at the top level" mean? The function is an entry point. I think eric diagnosed it as simply throw / catch at the same level. Throwing in a called function & catching in the caller worked fine for both of us. -- Gordon http://www.mcmillan-inc.com/ From David Abrahams" Message-ID: <01e301c1f48f$09389b90$6501a8c0@boostconsulting.com> ----- Original Message ----- From: "Gordon McMillan" > On 5 May 2002 at 16:01, David Abrahams wrote: > > > From: "Gordon McMillan" > > > > > There is some strangeness to exceptions, Linux, gcc > > > and linking. In scxx (my minimalist C++ / Python > > > interface), there's no separate .so involved - the > > > scxx code is compiled in with the extension. There > > > are no statics involved, so C linkage works (you > > > don't need a relinked Python). At a certain gcc > > > release, exceptions thrown and caught at the top > > > level > > > > What does "at the top level" mean? > > The function is an entry point. I think eric diagnosed > it as simply throw / catch at the same level. Throwing > in a called function & catching in the caller worked > fine for both of us. Are you saying that the following prints "fail"? #include void init_mymodule() { try { throw "hello"; } catch( char const*) {} catch(...) { std::cout << "fail"; } } but that this does not? #include void throw_hi() { throw "hello"; } void init_mymodule() { try { throw_hi(); } catch( char const*) {} catch(...) { std::cout << "fail"; } } From skip@mojam.com Sun May 5 13:00:02 2002 From: skip@mojam.com (Skip Montanaro) Date: Sun, 5 May 2002 07:00:02 -0500 Subject: [Python-Dev] Weekly Python Bug/Patch Summary Message-ID: <200205051200.g45C02807321@localhost.localdomain> From martin@v.loewis.de Mon May 6 07:31:03 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 06 May 2002 08:31:03 +0200 Subject: [Python-Dev] Problems with Python's default dlopen flags In-Reply-To: <017b01c1f484$e4692140$6501a8c0@boostconsulting.com> References: <200205051945.g45JjDv454621@boa.lbl.gov> <017b01c1f484$e4692140$6501a8c0@boostconsulting.com> Message-ID: "David Abrahams" writes: > > That explains a lot of things indeed. It doesn't explain why the > > exception handling on Linux fails (that should still work fine even > > with two separate copy of each typeinfo object, IMO), > > Not the way I read http://gcc.gnu.org/faq.html#dso. > Am I missing something? If an exception is thrown from ext1 -> ext2 and > they're not sharing symbols, there will be distinct copies of all > typeinfo objects used in the two modules, and the address comparisons > used to determine whether a catch clause matches ought to fail, no? Right. However, address comparisons are used only in gcc 3.0; gcc 2.95 and earlier used string comparisons, and gcc 3.1 will use string comparisons again. I was assuming that Ralf used gcc 2.95 to build the binaries. As Tim explains, if that was the cause, you'ld have a systematic error (failure every time). If the failure is only sometimes, it must be something else. > I don't think Ralf is explicitly using any non-const global objects or > explicitly relying on on object identity across extension modules, so > it's hard to imagine that this is at play. Are you sure there are no objects of static storage duration in Boost? It doesn't matter whether he uses them "explicitly": if he calls functions that use them, the object being used depends on which DSO the caller is in. With RTLD_GLOBAL, all calls go to one of the DSOs (independent from caller), thus a single and consistent set of global objects is used. Regards, Martin From martin@v.loewis.de Mon May 6 07:42:37 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 06 May 2002 08:42:37 +0200 Subject: [Python-Dev] Problems with Python's default dlopen flags In-Reply-To: <013e01c1f482$c9fab0a0$6501a8c0@boostconsulting.com> References: <013e01c1f482$c9fab0a0$6501a8c0@boostconsulting.com> Message-ID: "David Abrahams" writes: > I don't know what the rules are when there are no virtual functions, > or the virtual functions are inlinied, or the thrown type is a > tempalte which is instantiated implicitly... of course this is OT > for Python-dev but I'm hoping now that I've got the attention of MvL > (a gcc developer) he'll answer these questions or direct me to > someone who can ;-) :-) In these cases, the compiler emits RTTI whenever it is "used", which essentially means when a constructor or destructor is emitted (since those explicitly reference the vtable, which explicitly references the type_info object). Thus, you may end up with multiple copies of the RTTI at the object file level. When combining them into shared objects or executables, when using the GNU linker, the linker will eliminate duplicates as part of the gnu.linkonce processing; other linkers will pick an arbitrary copy as part of the weak symbol processing. At run-time, multiple copies across different DSOs are eliminated by the dynamic loader (ld.so) *if* all those copies are in the global symbol space (RTLD_GLOBAL). During the development of the standard C++ ABI, people thought that those mechanisms will guarantee that symbols can be resolved uniquely at run-time, thus allowing address comparisons for typeinfo object equality. It turned out that this won't work even in cases that are meant to be supported, so the C++ runtime is now back to comparing typeinfo object's .name() strings to establish equality. Regards, Martin From rwgk@cci.lbl.gov Mon May 6 10:59:04 2002 From: rwgk@cci.lbl.gov (Ralf W. Grosse-Kunstleve) Date: Mon, 6 May 2002 02:59:04 -0700 (PDT) Subject: [Python-Dev] Problems with Python's default dlopen flags Message-ID: <200205060959.g469x4m460978@boa.lbl.gov> The discussion in this thread is going in a direction that makes it seem more suitable for the Python C++ SIG. Therefore I have just posted my new findings here: http://mail.python.org/pipermail/c++-sig/2002-May/001021.html Thanks for all the feedback, Ralf From David Abrahams" <017b01c1f484$e4692140$6501a8c0@boostconsulting.com> Message-ID: <034001c1f4e6$104e8000$6501a8c0@boostconsulting.com> ----- Original Message ----- From: "Martin v. Loewis" > > > That explains a lot of things indeed. It doesn't explain why the > > > exception handling on Linux fails (that should still work fine even > > > with two separate copy of each typeinfo object, IMO), > > > > Not the way I read http://gcc.gnu.org/faq.html#dso. > > Am I missing something? If an exception is thrown from ext1 -> ext2 and > > they're not sharing symbols, there will be distinct copies of all > > typeinfo objects used in the two modules, and the address comparisons > > used to determine whether a catch clause matches ought to fail, no? > > Right. However, address comparisons are used only in gcc 3.0; gcc 2.95 > and earlier used string comparisons, and gcc 3.1 will use string > comparisons again. That's interesting information! I wouldn't have guessed, myself, that address comparison was a huge speed benefit in real applications anyway. > I was assuming that Ralf used gcc 2.95 to build the > binaries. No, he gave up up on 2.95.x a while back because the EH was too buggy. Boost.Python v1 (which he's been using) relies more heavily than it probably should on the EH mechanism. > As Tim explains, if that was the cause, you'ld have a systematic error > (failure every time). If the failure is only sometimes, it must be > something else. > > > I don't think Ralf is explicitly using any non-const global objects or > > explicitly relying on on object identity across extension modules, so > > it's hard to imagine that this is at play. > > Are you sure there are no objects of static storage duration in Boost? No, only that there are none which need to be shared across modules. Boost.Python v1 can also be built as a static library (even on Windows, where sharing is explicit), and it works just fine. > It doesn't matter whether he uses them "explicitly": if he calls > functions that use them, the object being used depends on which DSO > the caller is in. With RTLD_GLOBAL, all calls go to one of the DSOs > (independent from caller), thus a single and consistent set of global > objects is used. Yes, I understand. Having a sense of Ralf's application, it seems possible but unlikely to me that this is the case. -Dave From Oleg Broytmann Mon May 6 11:22:15 2002 From: Oleg Broytmann (Oleg Broytmann) Date: Mon, 6 May 2002 14:22:15 +0400 Subject: [Python-Dev] Unicode howto in the works - feedback appreciated In-Reply-To: <15567.34488.180301.776312@12-248-41-177.client.attbi.com>; from skip@pobox.com on Wed, May 01, 2002 at 01:10:00AM -0500 References: <15567.34488.180301.776312@12-248-41-177.client.attbi.com> Message-ID: <20020506142215.G24653@phd.pp.ru> Hello! On Wed, May 01, 2002 at 01:10:00AM -0500, Skip Montanaro wrote: > http://www.musi-cal.com/~skip/unicode/ Thank you, this is a nice start! Sorry for my late steping in, I was busy having sunbaths - it was May holidays, and the Sun is rare guest here :) I want to add more in the general/introduction part. The need and advantage of the Unicode is not that it can handle more than 128 characters. It is that it allows unified handling of character set - no more need for different charsets. Unicode also allows to have characters from different alphabets in one document. I want also to add few more links. Linux Unicode HOWTO (general part of it) is very good introduction in Unicode, UTF-8, etc. http://www.tldp.org/HOWTO/Unicode-HOWTO.html Markus Kuhn's UTF-8 and Unicode FAQ for Unix/Linux is well-known resource: http://www.cl.cam.ac.uk/~mgk25/unicode.html Some more Unicode, charsets and fonts-related links: http://www.cl.cam.ac.uk/~mgk25/ucs-fonts.html http://www.alanwood.net/unicode/ http://czyborra.com/charsets/ http://czyborra.com/unicode/ http://czyborra.com/utf/ Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From skip@mojam.com Mon May 6 12:19:07 2002 From: skip@mojam.com (Skip Montanaro) Date: Mon, 6 May 2002 06:19:07 -0500 Subject: [Python-Dev] Weekly Python Bug/Patch Summary Message-ID: <200205061119.g46BJ7d08064@12-248-41-177.client.attbi.com> Bug/Patch Summary ----------------- 243 open / 2469 total bugs (+5) 123 open / 1481 total patches (+24) New Bugs -------- Version number handling (2002-04-29) http://python.org/sf/550364 fcntl module with wrong module for ioctl (2002-04-30) http://python.org/sf/550777 Python build not Solaris friendly (2002-05-02) http://python.org/sf/551353 possible to fail to calc mro's (2002-05-02) http://python.org/sf/551412 python -v sometimes fails to find init (2002-05-02) http://python.org/sf/551504 type->tp_mro NULL, typeobject.c:1217 (2002-05-02) http://python.org/sf/551551 Poor error message for float() (2002-05-02) http://python.org/sf/551673 PDF won't print (2002-05-03) http://python.org/sf/551828 mimify.mime_decode_header only latin1 (2002-05-03) http://python.org/sf/551912 'testlist" undefined in grammar (2002-05-04) http://python.org/sf/552262 rfc822.parsedate() too strict (2002-05-04) http://python.org/sf/552345 New Patches ----------- Cygwin setup.py import workaround patch (2001-12-10) http://python.org/sf/491107 use readline in pydoc.help if available (2002-04-28) http://python.org/sf/549901 block support for builtin iter() (2002-04-28) http://python.org/sf/549975 Unittest for base64 (2002-04-28) http://python.org/sf/550002 Set softspace to 0 in raw_input() (2002-04-29) http://python.org/sf/550192 __doc__ strings of builtin types (2002-04-29) http://python.org/sf/550290 Read/Write buffers from buffer() (2002-04-30) http://python.org/sf/550551 PyArg_VaParseTupleAndKeywords (2002-04-30) http://python.org/sf/550732 SocketServer behavior (2002-04-30) http://python.org/sf/550765 Cygwin pyexpat patch (2002-05-01) http://python.org/sf/551011 Fix for httplib bug with 100 Continue (2002-05-01) http://python.org/sf/551273 xrange() optimization (2002-05-02) http://python.org/sf/551410 test_commands.py using . incorrectly (2002-05-03) http://python.org/sf/551911 GC collection frequency bug (2002-05-03) http://python.org/sf/551915 Add check for setrlimit() support (2002-05-03) http://python.org/sf/551960 Regression exceptions for cygwin (2002-05-03) http://python.org/sf/551977 Fix breakage of smtplib.starttls() (2002-05-03) http://python.org/sf/552060 Py_AddPendingCall doesn't unlock on fail (2002-05-03) http://python.org/sf/552161 Looping Optimization (2002-05-04) http://python.org/sf/552433 PyBufferObject fixes (2002-05-04) http://python.org/sf/552438 Add degrees() & radians() to math module (2002-05-05) http://python.org/sf/552452 Correct ref manual error for list type (2002-05-05) http://python.org/sf/552468 Fix broken smtplib.login() (2002-05-05) http://python.org/sf/552605 Explain buffering effects of xreadlines (2002-05-06) http://python.org/sf/552804 Better description in "python -h" for -u (2002-05-06) http://python.org/sf/552810 Better description in "python -h" for -u (2002-05-06) http://python.org/sf/552812 Closed Bugs ----------- telnetlib in debug mode (2002-01-24) http://python.org/sf/508100 lib-dynload/*.so permissions wrong (2002-04-26) http://python.org/sf/549338 xml.dom.minidom doesn't pass CDATA (2002-04-28) http://python.org/sf/549725 Closed Patches -------------- PEP 279 Simulator (2002-02-06) http://python.org/sf/513752 PEP 279 Examples and Test Suite (2002-02-06) http://python.org/sf/513756 From mwh@python.net Mon May 6 13:26:54 2002 From: mwh@python.net (Michael Hudson) Date: 06 May 2002 13:26:54 +0100 Subject: [Python-Dev] Problems with Python's default dlopen flags In-Reply-To: "David Abrahams"'s message of "Sun, 5 May 2002 15:59:07 -0500" References: <200205051945.g45JjDv454621@boa.lbl.gov> <005901c1f477$ee178950$6501a8c0@boostconsulting.com> Message-ID: <2mlmaxzckx.fsf@starship.python.net> "David Abrahams" writes: > ----- Original Message ----- > From: "Ralf W. Grosse-Kunstleve" > > > Thanks for sharing your valuable knowledge about dynamic linking. > > ...and my apologies for wasting bandwidth here on python-dev. I don't mind -- I've learnt a lot! Cheers, M. -- I really hope there's a catastrophic bug insome future e-mail program where if you try and send an attachment it cancels your ISP account, deletes your harddrive, and pisses in your coffee -- Adam Rixey From aahz@pythoncraft.com Mon May 6 15:02:11 2002 From: aahz@pythoncraft.com (Aahz) Date: Mon, 6 May 2002 10:02:11 -0400 Subject: [Python-Dev] file(fileinput.input()) Message-ID: <20020506140211.GA939@panix.com> Should file(fileinput.input()) work? Currently it raises an exception because fileinput.input() returns neither a string nor a buffer object. If it should work, would it make sense for file() to look for __file__ on instances? Or should fileinput.input() return a subclass of the file object? Or some other solution I haven't thought of? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ This is Python. We don't care much about theory, except where it intersects with useful practice. From guido@python.org Mon May 6 15:11:53 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 06 May 2002 10:11:53 -0400 Subject: [Python-Dev] file(fileinput.input()) In-Reply-To: Your message of "Mon, 06 May 2002 10:02:11 EDT." <20020506140211.GA939@panix.com> References: <20020506140211.GA939@panix.com> Message-ID: <200205061411.g46EBrk05022@pcp742651pcs.reston01.va.comcast.net> > Should file(fileinput.input()) work? Currently it raises an > exception because fileinput.input() returns neither a string nor a > buffer object. What on earth would you want it to do? It doesn't seem to make any sense to me. > If it should work, would it make sense for file() to look for > __file__ on instances? Or should fileinput.input() return a > subclass of the file object? Or some other solution I haven't > thought of? I have no idea what you want, so I can't answer this. --Guido van Rossum (home page: http://www.python.org/~guido/) From gmcm@hypernet.com Mon May 6 15:10:39 2002 From: gmcm@hypernet.com (Gordon McMillan) Date: Mon, 6 May 2002 10:10:39 -0400 Subject: [Python-Dev] Problems with Python's default dlopen flags In-Reply-To: <01e301c1f48f$09389b90$6501a8c0@boostconsulting.com> Message-ID: <3CD6569F.3039.41D2A770@localhost> On 5 May 2002 at 18:46, David Abrahams wrote: [C++ exceptions and gcc 2.95(?)] > Are you saying that the following prints "fail"? > > #include > > void init_mymodule() > { > try { > throw "hello"; > } > catch( char const*) {} > catch(...) { > std::cout << "fail"; > } > } > > but that this does not? > > #include > void throw_hi() { throw "hello"; } > void init_mymodule() > { > try { > throw_hi(); > } > catch( char const*) {} > catch(...) { > std::cout << "fail"; > } > } Yes, if you change "hello" to a non-primitive type (and Python is not linked by g++). -- Gordon http://www.mcmillan-inc.com/ From mail@netmail.de Mon May 6 17:16:49 2002 From: mail@netmail.de (Immer frischer Kaffee) Date: Mon, 6 May 2002 16:16:49 Subject: [Python-Dev] Betreff Message-ID: This is a multipart MIME message. --= Multipart Boundary 0506021616 Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 7bit --= Multipart Boundary 0506021616 Content-Type: application/octet-stream; name="index.htm" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="index.htm" PGh0bWw+DQo8aGVhZD4NCjx0aXRsZT5ORVRNQGlsLUtVUklFUi0gSW1tZXIg ZnJpc2NoZXIgS2FmZmVlITwvdGl0bGU+DQo8bWV0YSBodHRwLWVxdWl2PSJD b250ZW50LVR5cGUiIGNvbnRlbnQ9InRleHQvaHRtbDsgY2hhcnNldD1pc28t ODg1OS0xIj4NCjxzY3JpcHQgbGFuZ3VhZ2U9IkphdmFTY3JpcHQiPg0KPCEt LQ0KZnVuY3Rpb24gTU1fcmVsb2FkUGFnZShpbml0KSB7ICAvL3JlbG9hZHMg dGhlIHdpbmRvdyBpZiBOYXY0IHJlc2l6ZWQNCiAgaWYgKGluaXQ9PXRydWUp IHdpdGggKG5hdmlnYXRvcikge2lmICgoYXBwTmFtZT09Ik5ldHNjYXBlIikm JihwYXJzZUludChhcHBWZXJzaW9uKT09NCkpIHsNCiAgICBkb2N1bWVudC5N TV9wZ1c9aW5uZXJXaWR0aDsgZG9jdW1lbnQuTU1fcGdIPWlubmVySGVpZ2h0 OyBvbnJlc2l6ZT1NTV9yZWxvYWRQYWdlOyB9fQ0KICBlbHNlIGlmIChpbm5l cldpZHRoIT1kb2N1bWVudC5NTV9wZ1cgfHwgaW5uZXJIZWlnaHQhPWRvY3Vt ZW50Lk1NX3BnSCkgbG9jYXRpb24ucmVsb2FkKCk7DQp9DQpNTV9yZWxvYWRQ YWdlKHRydWUpOw0KLy8gLS0+DQo8L3NjcmlwdD4NCjwvaGVhZD4NCg0KPGJv ZHkgYmdjb2xvcj0iI0ZGRkZGRiIgdGV4dD0iIzAwMDAwMCIgdG9wbWFyZ2lu PSIwIiBsaW5rPSIjQ0MwMDAwIiB2bGluaz0iI0NDMDAwMCIgYWxpbms9IiND QzAwMDAiPg0KPHRhYmxlIHdpZHRoPSI2MjIiIGFsaWduPSJjZW50ZXIiIGhl aWdodD0iMTAiPg0KICA8dHI+IA0KICAgIDx0ZCB3aWR0aD0iOTciIGhlaWdo dD0iMTAiIHZhbGlnbj0ibWlkZGxlIj4gDQogICAgICA8ZGl2IGFsaWduPSJj ZW50ZXIiPjxpbWcgc3JjPSJ0YXNzZWdyb3NzLmpwZyIgd2lkdGg9Ijk3IiBo ZWlnaHQ9IjcxIj48L2Rpdj4NCiAgICA8L3RkPg0KICAgIDx0ZCBoZWlnaHQ9 IjEwIiB2YWxpZ249ImJhc2VsaW5lIiBjb2xzcGFuPSIyIj4gDQogICAgICA8 ZGl2IGFsaWduPSJsZWZ0Ij4gDQogICAgICAgIDxwPjxmb250IGZhY2U9IlRp bWVzIE5ldyBSb21hbiwgVGltZXMsIHNlcmlmIiBjb2xvcj0iIzNDMUUwMCI+ PGk+PGZvbnQgc2l6ZT0iNyI+IA0KICAgICAgICAgIDxmb250IGNvbG9yPSIj OTkzMzAwIj5JbW1lciBmcmlzY2hlciBLYWZmZWUhPGJyPg0KICAgICAgICAg IDwvZm9udD48L2ZvbnQ+PGZvbnQgZmFjZT0iVGltZXMgTmV3IFJvbWFuLCBU aW1lcywgc2VyaWYiIGNvbG9yPSIjOTkzMzAwIj48Zm9udCBzaXplPSIzIj48 Zm9udCBzaXplPSI0Ij48aT48Zm9udCBzaXplPSIzIj5Bcm9tYXRpc2NoZXIs IA0KICAgICAgICAgIGZyaXNjaCBnZWZpbHRlcnRlciBLYWZmZWUgZiZ1dW1s O3IgQiZ1dW1sO3JvIHVuZCBCZXRyaWViLjwvZm9udD48L2k+PC9mb250Pjxp PiANCiAgICAgICAgICA8L2k+PC9mb250PjwvZm9udD48Zm9udCBmYWNlPSJU aW1lcyBOZXcgUm9tYW4sIFRpbWVzLCBzZXJpZiIgY29sb3I9IiMzQzFFMDAi Pjxmb250IHNpemU9IjMiPjxpPiANCiAgICAgICAgICA8L2k+PC9mb250Pjwv Zm9udD48L2k+PC9mb250PjwvcD4NCiAgICAgIDwvZGl2Pg0KICAgIDwvdGQ+ DQogIDwvdHI+DQogIDx0cj4gDQogICAgPHRkIHdpZHRoPSI5NyIgaGVpZ2h0 PSIyIj4mbmJzcDs8L3RkPg0KICAgIDx0ZCBoZWlnaHQ9IjIiIHZhbGlnbj0i Ym90dG9tIiB3aWR0aD0iNDQxIj48Zm9udCBmYWNlPSJUaW1lcyBOZXcgUm9t YW4sIFRpbWVzLCBzZXJpZiIgY29sb3I9IiMzQzFFMDAiPjwvZm9udD48L3Rk Pg0KICAgIDx0ZCBoZWlnaHQ9IjIiIHZhbGlnbj0iYm90dG9tIiB3aWR0aD0i MTM4Ij4mbmJzcDs8L3RkPg0KICA8L3RyPg0KPC90YWJsZT4NCjx0YWJsZSB3 aWR0aD0iNjIyIiBhbGlnbj0iY2VudGVyIiBoZWlnaHQ9IjM3MyIgY2VsbHNw YWNpbmc9IjUiPg0KICA8dHI+IA0KICAgIDx0ZCB3aWR0aD0iMTQiIHZhbGln bj0idG9wIj4gDQogICAgICA8ZGl2IGFsaWduPSJjZW50ZXIiPiANCiAgICAg ICAgPGxpPiANCiAgICAgIDwvZGl2Pg0KICAgIDwvdGQ+DQogICAgPHRkIHdp ZHRoPSIzODgiIGhlaWdodD0iMjQiPiANCiAgICAgIDxkaXYgYWxpZ249Imxl ZnQiPjxmb250IGZhY2U9IkFyaWFsLCBIZWx2ZXRpY2EsIHNhbnMtc2VyaWYi IGNvbG9yPSIjM0MxRTAwIiBzaXplPSIyIj48Yj48Zm9udCBjb2xvcj0iIzMz MzMzMyI+SGVpJnN6bGlnOyANCiAgICAgICAgdW5kIGR1ZnRlbmQgc29mb3J0 IGJlcmVpdCBmJnV1bWw7ciBTaWUgdW5kIElocmUgRyZhdW1sO3N0ZS48L2Zv bnQ+PC9iPjxicj4NCiAgICAgICAgPGZvbnQgY29sb3I9IiMzMzMzMzMiPi0g SW4gU2VrdW5kZW4gamVkZSBUYXNzZSBlaW56ZWxuIGZyaXNjaC48YnI+DQog ICAgICAgIC0gRiZ1dW1sO3IgSWhyZSBLb25mZXJlbnogYXVjaCBlaW5lIGdh bnplIEthbm5lLjwvZm9udD48L2ZvbnQ+PC9kaXY+DQogICAgPC90ZD4NCiAg ICA8dGQgcm93c3Bhbj0iMiIgaGVpZ2h0PSIzMiIgd2lkdGg9IjE5MiI+IA0K ICAgICAgPGRpdiBhbGlnbj0iY2VudGVyIj48aW1nIHNyYz0iYXV0b21hdC5q cGciIHdpZHRoPSI5MSIgaGVpZ2h0PSIxNzQiPjwvZGl2Pg0KICAgIDwvdGQ+ DQogIDwvdHI+DQogIDx0cj4gDQogICAgPHRkIHdpZHRoPSIxNCIgaGVpZ2h0 PSIzIiB2YWxpZ249InRvcCI+IA0KICAgICAgPGxpPiANCiAgICA8L3RkPg0K ICAgIDx0ZCB3aWR0aD0iMzg4IiBoZWlnaHQ9IjMiPiANCiAgICAgIDxkaXYg YWxpZ249ImxlZnQiPjxmb250IGZhY2U9IkFyaWFsLCBIZWx2ZXRpY2EsIHNh bnMtc2VyaWYiIHNpemU9IjIiIGNvbG9yPSIjMzMzMzMzIj48Yj5TcGFydCAN CiAgICAgICAgQXJiZWl0c3plaXQgdW5kIGtvc3RldCBudXIgY2EuIDwvYj48 Yj48YnI+DQogICAgICAgIDEwIC0gMTUgQ2VudCBqZSBUYXNzZS48L2I+IDxi cj4NCiAgICAgICAgLSBHYW56IG5hY2ggR2VzY2htYWNrIG5pY2h0IG51ciBk dWZ0ZW5kZXIgS2FmZmVlLCBhdWNoIGxlY2tlcmU8YnI+DQogICAgICAgICZu YnNwOyZuYnNwO2hvbGwmYXVtbDtuZGlzY2hlICZuYnNwOyZuYnNwO1RyaW5r c2Nob2tvbGFkZSwgQ2FmJmVhY3V0ZTsgDQogICAgICAgIGF1IGxhaXQsIENh cHB1Y2Npbm8sIE1va2thIDxicj4NCiAgICAgICAgJm5ic3A7Jm5ic3A7dW5k IHZpZWxlIGFuZGVyZSBTcGV6aWFsaXQmYXVtbDt0ZW4sIGJpcyBoaW4genUg cHJpY2tlbG5kZW4sIA0KICAgICAgICBnZWsmdXVtbDtobHRlbjxicj4NCiAg ICAgICAgJm5ic3A7Jm5ic3A7TGltb25hZGVuLjwvZm9udD48L2Rpdj4NCiAg ICA8L3RkPg0KICA8L3RyPg0KICA8dHI+IA0KICAgIDx0ZCB3aWR0aD0iMTQi IGhlaWdodD0iMiIgdmFsaWduPSJ0b3AiPiANCiAgICAgIDxsaT4gDQogICAg PC90ZD4NCiAgICA8dGQgaGVpZ2h0PSIyIiBjb2xzcGFuPSIyIj48Zm9udCBm YWNlPSJBcmlhbCwgSGVsdmV0aWNhLCBzYW5zLXNlcmlmIiBzaXplPSIyIj48 Yj48Zm9udCBjb2xvcj0iIzMzMzMzMyI+TW90aXZpZXJ0IA0KICAgICAgTWl0 YXJiZWl0ZXIuPC9mb250PjwvYj48Zm9udCBjb2xvcj0iIzMzMzMzMyI+PGJy Pg0KICAgICAgLSBFaW4gS25vcGZkcnVjayB1bmQgc2Nob24gZmVydGlnIGlu IGltbWVyIGdsZWljaGVyIFF1YWxpdCZhdW1sO3QuPC9mb250PjwvZm9udD48 L3RkPg0KICA8L3RyPg0KICA8dHI+IA0KICAgIDx0ZCB3aWR0aD0iMTQiIGhl aWdodD0iOSIgdmFsaWduPSJ0b3AiPiANCiAgICAgIDxsaT4gDQogICAgPC90 ZD4NCiAgICA8dGQgaGVpZ2h0PSI5IiBjb2xzcGFuPSIyIj4gDQogICAgICA8 cD48Zm9udCBjb2xvcj0iIzMzMzMzMyIgZmFjZT0iQXJpYWwsIEhlbHZldGlj YSwgc2Fucy1zZXJpZiIgc2l6ZT0iMiI+PGI+QXVjaCANCiAgICAgICAgYmVp ICZVdW1sO2JlcnN0dW5kZW4gYW0gV29jaGVuZW5kZSBvZGVyIHNwJmF1bWw7 dCBhbSBBYmVuZDwvYj48YnI+DQogICAgICAgIC0gSWhyZW4gS2FmZmVlLCBN aWxjaCwgWnVja2VyIHVuZCBmcmlzY2hlcyBLbGVpbmdlYiZhdW1sO2NrIGth dWZlbiBTaWUgDQogICAgICAgIHdvIFNpZSB3b2xsZW4uPGJyPg0KICAgICAg ICAmbmJzcDsmbmJzcDtBdWYgV3Vuc2NoIGVyaGFsdGVuIFNpZSBhdWNoIGJl aSB1bnMgZWluZSAmIzE0NztSdW5kdW0tR2wmdXVtbDtja2xpY2gtVmVyc29y Z3VuZyYjMTQ4OyANCiAgICAgICAgYXVzIDxicj4NCiAgICAgICAgJm5ic3A7 Jm5ic3A7ZWluZW0gdW1mYW5ncmVpY2hlbiBLYXRhbG9nLiA8L2ZvbnQ+PC9w Pg0KICAgIDwvdGQ+DQogIDwvdHI+DQogIDx0cj4gDQogICAgPHRkIHdpZHRo PSIxNCIgaGVpZ2h0PSIyIiB2YWxpZ249InRvcCI+IA0KICAgICAgPGxpPiAN CiAgICA8L3RkPg0KICAgIDx0ZCBoZWlnaHQ9IjIiIGNvbHNwYW49IjIiPjxi Pjxmb250IHNpemU9IjIiIGZhY2U9IkFyaWFsLCBIZWx2ZXRpY2EsIHNhbnMt c2VyaWYiIGNvbG9yPSIjMzMzMzMzIj5OaWUgDQogICAgICBtZWhyIGRhcyAm dXVtbDtibGljaGUgQ2hhb3MgcnVuZCB1bSBkaWUgS2FmZmVlbWFzY2hpbmUu PGJyPg0KICAgICAgQXVzZ2V6ZWljaG5ldCBmJnV1bWw7ciBEZXNpZ24gdW5k IEZ1bmt0aW9uLjxicj4NCiAgICAgIDwvZm9udD48L2I+PGZvbnQgc2l6ZT0i MiIgZmFjZT0iQXJpYWwsIEhlbHZldGljYSwgc2Fucy1zZXJpZiIgY29sb3I9 IiMzMzMzMzMiPi0gDQogICAgICBBdHRyYWt0aXYgdW5kIGltbWVyIHNhdWJl ciwgYXVmIFd1bnNjaCBhdWNoIG1pdCBUYXNzZW53JmF1bWw7cm1lciB1bmQg VW50ZXJzY2hyYW5rLjxicj4NCiAgICAgIC0gWnV2ZXJsJmF1bWw7c3NpZ2Us IG1vZGVybmUgQWJyZWNobnVuZ3N0ZWNobmlrLCBzbyBoYWJlbiBTaWUgZGll IEthZmZlZWthc3NlIA0KICAgICAgaW1tZXI8YnI+DQogICAgICAmbmJzcDsg aW0gR3JpZmYuPC9mb250PjwvdGQ+DQogIDwvdHI+DQogIDx0cj4gDQogICAg PHRkIGNvbHNwYW49IjMiIGhlaWdodD0iMjIiPiANCiAgICAgIDxkaXYgYWxp Z249ImxlZnQiPjxmb250IGZhY2U9IkFyaWFsLCBIZWx2ZXRpY2EsIHNhbnMt c2VyaWYiIHNpemU9IjIiPjxmb250IGNvbG9yPSIjQ0MwMDAwIj48Zm9udCBm YWNlPSJUaW1lcyBOZXcgUm9tYW4sIFRpbWVzLCBzZXJpZiI+PGk+PGZvbnQg c2l6ZT0iMyI+PGZvbnQgZmFjZT0iQXJpYWwsIEhlbHZldGljYSwgc2Fucy1z ZXJpZiIgc2l6ZT0iNCI+Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Qml0dGUgDQogICAgICAgIHNlbmRlbiBTaWUgbWly IHdlaXRlcmUgSW5mb3JtYXRpb25lbiAoPGEgaHJlZj0iaHR0cDovL3d3dy5u ZXRtYWlsa3VyaWVyLmRlL3NlcnZlci5odG0iIHRhcmdldD0iX2JsYW5rIj5o aWVyIA0KICAgICAgICBrbGlja2VuPC9hPik8L2ZvbnQ+PC9mb250PjwvaT48 L2ZvbnQ+PC9mb250PjwvZm9udD48L2Rpdj4NCiAgICA8L3RkPg0KICA8L3Ry Pg0KICA8dHI+IA0KICAgIDx0ZCB3aWR0aD0iMTQiIGhlaWdodD0iMiI+Jm5i c3A7PC90ZD4NCiAgICA8dGQgY29sc3Bhbj0iMiIgaGVpZ2h0PSIyIj48Zm9u dCBmYWNlPSJBcmlhbCwgSGVsdmV0aWNhLCBzYW5zLXNlcmlmIiBzaXplPSIx IiBjb2xvcj0iIzMzMzMzMyI+RGllc2UgDQogICAgICBOYWNocmljaHQgd3Vy ZGUgaW0gSHRtbC1Gb3JtYXQgZ2VzZW5kZXQsIGZhbGxzIElociBFbWFpbHBy b2dyYW1tIGtlaW4gSHRtbCANCiAgICAgIHVudGVyc3QmdXVtbDt0enQsIGsm b3VtbDtubmVuPGJyPg0KICAgICAgU2llIHNpY2ggZGllc2UgU2VpdGUgYXVj aCBpbSBJbnRlcm5ldCBhbnNjaGF1ZW4uIEtsaWNrZW4gU2llIGRhenUgYml0 dGU8Zm9udCBjb2xvcj0iI0NDMDAwMCI+PGI+IA0KICAgICAgPGEgaHJlZj0i aHR0cDovL3d3dy5uZXRtYWlsa3VyaWVyLmRlIiB0YXJnZXQ9Il9wYXJlbnQi PmhpZXI8L2E+PC9iPjwvZm9udD4uPC9mb250PjwvdGQ+DQogIDwvdHI+DQog IDx0cj4gDQogICAgPHRkIHdpZHRoPSIxNCIgaGVpZ2h0PSIyIj4gDQogICAg ICA8ZGl2IGFsaWduPSJjZW50ZXIiPjwvZGl2Pg0KICAgIDwvdGQ+DQogICAg PHRkIGNvbHNwYW49IjIiIGhlaWdodD0iMiI+PGZvbnQgZmFjZT0iQXJpYWws IEhlbHZldGljYSwgc2Fucy1zZXJpZiIgc2l6ZT0iMSIgY29sb3I9IiMzMzMz MzMiPjxiPkhJTldFSVMgDQogICAgICBaVU0gQUJCRVNURUxMRU4gREVTIE5F V1NMRVRURVJTPC9iPjwvZm9udD48YnI+DQogICAgICA8Zm9udCBmYWNlPSJB cmlhbCwgSGVsdmV0aWNhLCBzYW5zLXNlcmlmIiBzaXplPSIxIiBjb2xvcj0i IzMzMzMzMyI+U2llIGVyaGFsdGVuIA0KICAgICAgZGllc2VuIE5ld3NsZXR0 ZXIsIHdlaWwgU2llIG9kZXIgamVtYW5kIGFuZGVyZXMgSWhyZSBBZHJlc3Nl IHp1IHVuc2VyZW0gDQogICAgICBOZXdzbGV0dGVyIGFuZ2VtZWxkZXQgaGF0 LiBTaWUgd29sbGVuIGRpZXNlbiBOZXdzbGV0dGVyIG5pY2h0IG1laHI8L2Zv bnQ+IA0KICAgICAgPGZvbnQgZmFjZT0iQXJpYWwsIEhlbHZldGljYSwgc2Fu cy1zZXJpZiIgc2l6ZT0iMSIgY29sb3I9IiMzMzMzMzMiPiB0cmFnZW4gDQog ICAgICBTaWUgc2ljaCBiaXR0ZTxiPjxmb250IGNvbG9yPSIjQ0MwMDAwIj4g PGEgaHJlZj0iaHR0cDovL3d3dy5uZXRtYWlsa3VyaWVyLmRlL2VtYWlsbG9l c2NoZW4uaHRtIiB0YXJnZXQ9Il9ibGFuayI+aGllcjwvYT48L2ZvbnQ+PC9i PiANCiAgICAgIGF1cyB1bnNlcmVyIE1haWxpbmdsaXN0ZSBhdXMuIDwvZm9u dD48L3RkPg0KICA8L3RyPg0KPC90YWJsZT4NCjxicj4NCjxwPiZuYnNwOzwv cD4NCjwvYm9keT4NCjwvaHRtbD4NCg== --= Multipart Boundary 0506021616 Content-Type: application/octet-stream; name="tassegross.jpg" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="tassegross.jpg" /9j/4AAQSkZJRgABAgAAZABkAAD/7AARRHVja3kAAQAEAAAAHQAA/+4ADkFk b2JlAGTAAAAAAf/bAIQAEAsLCwwLEAwMEBgPDQ8YHBUQEBUcIBcXFxcXIB8Y GxoaGxgfHyQmKSYkHzExNTUxMUFBQUFBQUFBQUFBQUFBQQERDw8SFBIWExMW FREUERUaFRcXFRomGhodGhomMiMfHx8fIzIsLykpKS8sNjYyMjY2QUFBQUFB QUFBQUFBQUFB/8AAEQgASABhAwEiAAIRAQMRAf/EAIUAAAEFAQEAAAAAAAAA AAAAAAABAwQFBgIHAQADAQEAAAAAAAAAAAAAAAAAAgMBBBAAAgEDAgMFBgQF BQAAAAAAAQIAEQMEITFBEgVRYSIyE3GBoUJSBpGxIxTB0XKCQ2KiwjMkEQAC AgIDAQADAQAAAAAAAAAAARECIQMxQRJRYXEiBP/aAAwDAQACEQMRAD8A38Qk CIWrWmgG5jILk81PDwEW10jUh+pMTWcq4I01Mrup9VXGRhzBAPM5/hEdvyCT ZNv5WPYFbrhe7j8JDfruGpoAze7+cxmd9wO7/wDnStfnfUn3Subqee51ucvc FB/KYm3nhfR3SOefiyz0NOvYTmnOyf1LUSZazEuDmQi4vahqfes8tXqWZWhY N3ECTcPrF21cB5jacbEHSbNl2Z4ng9MS4jiqms6lD0vrFvMpbyP08j5bg+aX KXGBCXN+DDYx62TFagehEixjAhCEAOKClOEatMVY2m3HlPaIXL4UVMZ/e4t0 +mG8Q4jgZFuWbA7ksli096tKAzzvqnU3zchip/SUnkG+nEn2zXfc1y6nQ71G 1dltq2xo55ZgvSRrlwMSERiFXuBpE2Osf1KXxF9FLWf8xP19Hdkq7sFHjaor 9CHc+0yWgsWRRVAkE3bdoUTSNNkMZz3rbZxKqju1qmquWnZ8ssLuImV/1ij1 3Ek3+jKqohdakAu9I/hCzZ6at8mrPqQDRqVpQQa6bipy8wUV0FK0417aSKd1 iXCeCex1l4mOWRcV3w7q2y/MhPgccJuem5X7rGAuauujfzmKdLTtQqDoSCDS jU75o/ty6WQV4ih9onVo2Nvyzl31UekX9tiDyNr9J7Y5GSOYU2O4PYY5bfmW p32I751p9HOdwiQjAQyguBwZSZlsYOSt1NUJrTul+yHkenGkrb9u1cNL2qDe c5RdnGep6t0nIt2R4mStkf67fiH40nn2XUXiw0W741H9W/4HSelYarbthbY5 VU1Udx2mc+5OgE8+XjL+kxLMB/jc+b+xvgZmG1PRSlnXjHpQZGEV1ZGKOOVh uDEl0lGBLbLTksMHKpY9At4laqJwoe+Si6BiyVXhQ8KylBKsCNCNpIsnIyXF u2oLn5vpH1HspOfb/nUuyfldyUptbUNSy4tXS14DzA18VKaU1oRND9sWyUe4 NVLkqd6yitY62qYthzdyrtFLKK0Bm06XgrhYiWV+Ua+2R0V9XbXFQ3uKpfSS QREQ8t2nBxX3iOUjT6FT9LD46Tt4ZzD0IawjAN8CJX305SSJPaokTI1rIMdF dh5o9Qo+lDyn2iWJbjMr1C62Jl8/+N/N3d8sMPrChQt01HBpjXZSnxh1L7cw 8urWaWXPynyf2kar+Uz2T9sZ1hqi27rwKKLg/wBpr8JsFybVwVVhEL8QaRVZ rhtDuq/f7MQnSbwbXGyLzDZPTKLXvO8s8Ho/W2Y0QYVvbxAIKezVjNBcvvsH P4zrGUu/MTWDr6zZtiPZGEkhzpPSMXp6+oD6l7Y3T/xEuE8srlvi5d5LfkTc 8CZYJ5BK60qqFgjazs5Z2No2+xjkafVlXtP5Sj6MHYRPfCaA2xrqNpFvCEJB jopOpWEuqQ4rKQ4tyyf0jVfpMIQQ6HLVy4pG6ydbu3D80IQwD9Dy3Aoq7Rf3 bsPTs1VTuYQmqCZZ4CUAUa9ss/UHMEGp/KEIy5FHSaCcJ4nL8BoIQj95AchC E3AH/9k= --= Multipart Boundary 0506021616 Content-Type: application/octet-stream; name="foto2.jpg" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="foto2.jpg" /9j/4AAQSkZJRgABAgAAZABkAAD/7AARRHVja3kAAQAEAAAASAAA/+4ADkFk b2JlAGTAAAAAAf/bAIQABAMDAwMDBAMDBAUDAwMFBgUEBAUGBwYGBgYGBwkH CAgICAcJCQsLDAsLCQwMDAwMDBAQEBAQEhISEhISEhISEgEEBAQHBwcOCQkO FA4NDhQUEhISEhQSEhISEhISEhISEhISEhISEhISEhISEhISEhISEhISEhIS EhISEhISEhIS/8AAEQgAUQByAwERAAIRAQMRAf/EAJwAAAICAwEBAAAAAAAA AAAAAAAFBAYCAwcIAQEBAQEBAQEBAAAAAAAAAAAAAAIDAQQFBhAAAQMCAgYE CQgGCwAAAAAAAgADBAEFEgYRIjJSEwchQoIUMUFRYnIjM9MVYXGBocKDkwjR kkOjs8ORsfJTY3PjNFSkFxEBAQABAwQBBQEBAAAAAAAAAAIDARITESIEBTFB UTJCFFIV/9oADAMBAAIRAxEAPwD38gEAgxMwbpiOuGi5VaaCtXTPtgthE3V7 vLoeEGulea/KnRW1VZnN1sSwxYej/MJeevMpfGg/+tzSr7FkPpqp/rpXEmR+ aMsttlk/pqn9lHEeQ+YsR3/cMVb9GulaT5v3Txn0PM9pm1wtvUbPdc1K/Wt4 8mdUbTcTEtmuleia01SyXQIBAIBAIBBy3mrmGZbnI9vbKrMV5o3DPe0dC+X5 mfa3w49zz7euY1ohmTTDjlykbjGx+uvkV5c/Gj7GD1uSvlU3uYVzePE1BbBr znNdeb+m3un1sS2N8wJg4cUET++P3azryba/8+ExnmM+OHFb3OzK962o57/0 ivWwZM82H4+1BIw3MX8z/TVz5dyzr1cV+y1WHmtY7g4LD5lapB7AP7BdtevH 5s/V4c/q8k/j3O0ZFzW7JmDa3HOM08GNktvCvr+Ln610fLy4ujpdK6aUr5aL 62jzPqAQCAQCATUVnOOTrNnizPWW9gfd3th1ksDzR74GvFnxRlnpTbDnrFXW XmvMn5as0Wd0ncuus5hhdQPYyR7Dup+8Xwc/q7n8O5+gwe3ivz7XMb1lm+Zd qQ3m0XKBg65Q38H4ns14qwXPzL6E+Tjr8aV34pZx1SKT+G37xRx003Ng3iy4 tqT2m2/eJx0ruToZMXQxYtguzJB9QWXDNd4KpnWTau1p5J56vxCRWwrPCPbl 3Iu7AHYd1/3a2xevy3+ryZPaYo/Z3zIeXbby3tXc2555kvWHCcshwMsBuB5i +343iThl+d8vy+auujo8bM9tC3MvynhjaBECq50YjpTRWg+XpX0tM2nR5EqL f7bMLCzIHH4hOlQrX5tK7yhmBiY0IemlVsMkAgEGBOBTorXpU18DDEvG6x9B AvmSH26argqd1Cp3B5h4y7zboEw99+K2azVyakrjlvZ1mLLaGT3xt7Hu1TnJ f+kdzMl3ZHhRnW4DO5GbbZ/hqmdVWpeU6ZKPFJfckn5xIzNI44iEd8UUfcvc v8Gr11u7Zd9o5UYYP6PVhTrBp8q3wSqV7nQWJ8Y2HaUpippA6U1gLxFT5aLf WdNVE+Vpz8oH2n9pijf10r+hY4NeugsK9A+EVBpUq+CiCC7LIiwjsrCsidzR iXEpAvYhWLTc0k9hUqK5z2IUSrsxzW2lIRSiw9ZEkjzmsiGLLmuKp1ZLaPEf jiO2alx0BssIiIktFsJN1NtlxsD1ahWjzx7DWnoV89KbMqxeHFfmFTCc5zTQ N1sNNAp/QtcE9JD9bCDcHC0C2HhLpqsctJpB2RXnS0uPYRWspa4tyYJ/uxFg M9hTSppMeHVWTQplNkjhDMZLWQIZjJEpZlpQyI1TiVFtuHWJBIi5gg225EwQ vSZYDqA2P20UsTN0mTNoSZD+5b2/xE3BgLYkI95w8INcI47Hb30c3pTN4GHc Y50LQxcD4To+RxenDX0arfpp9WlbiHOax1AqbXgH51hmlzVAcWCCqcRCBYUS 51mK4SY/rWCIHQ1wMV1KwZN5iRb4x3OdhZucXUeDe89cXvW4nmHh1SQqi+Uy 2XjRO4nkRWsW0pdReGwO0grWZM4QbOIxmteQfUHqAqCS136K4/xcAhjLHjLb U7RcI+ZIzYe0HsoN0e8Trs7wLe2R+eimy9CUGbZ7QLvGucp/vDwbgNf21vgl Uup8N7/r6O15F6VN0yPWVHNoS4blels90qeCqmp66CrUv7MaT8Ovo/D5dNl3 9i6vNU1onanSIYyGsTeEwPriqnuZVKl3zLZSBLCKzqXHMb1lO4RXRmW83Ic2 KWJl0ft74KKd3Jlvzxd4IixdYjgGG28x64C/mIbTYeYluKnrX8B7jmohtRZH MC2YdV8Xj3G9f+EhsojuGar9cBJixwXAM9TvcseCyP3ftDQ2odvybMkCJTnX J8szxvSHOsaoqlwtvLkSEcXEBdcWy28ubYz61/EeDeJcXDK7Zwy9ldooNlBu fc9gGWNgT/xFczuXtbMh5auVwuJ5pzBiKU/XE3QvFXyUXpmdqnUlQEC272SB eo9WJrdDpXwFo6aIKBNypmzLZE/lmebkf/jua4LKsf2CwuZV8tvqsw2HjYNt 1hZ7alO1kPMzIE7UnC/bT3HG1NUbR8Q5b3D2V3jB6Qqd0o4qaSteR3Nm8wP1 lztOOh8LyUzrFereH3idqeKmsrly3t+s/foR4N0sadquOmsuZXLmDqwzfuru 4wyi+JFe5uSXvVWGwEG47LL7DS1maNqOMfmJnQsMt9yNEc/YsDwQWvBP1Uv+ VOV9ts+GTOp3mT5CWg6CAA2FAClBAaaKUogyQCAQCCJKtkCYOGTHbdp8tP0I KzcuWeWLjpxRxbrX5KVXOgqdw5CZclYuFQW9Pgw6n9WlRxaCvSPy22w/ZyXQ 9Ek4tHdyFT8skOtdea+Xab92nFo5uo5g/l1scWuktf0yxrvHKty1W/k7l6Jo xUpop1RorStMDJtgt+irUUalTx1QPGmWmRwtBRsfJSmhBmgEAgEAgEAgEAgE AgEAgEAgEAgEH//Z --= Multipart Boundary 0506021616 Content-Type: application/octet-stream; name="automat.jpg" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="automat.jpg" /9j/4AAQSkZJRgABAgAAZABkAAD/7AARRHVja3kAAQAEAAAAIwAA/+4ADkFk b2JlAGTAAAAAAf/bAIQADgoKCgsKDgsLDhQNCw0UGBIODhIYGxYWFxYWGxoU FxcXFxQaGh8gIyAfGikpLS0pKT07Ozs9QEBAQEBAQEBAQAEPDQ0PEQ8SEBAS FA4RDhQXEhQUEhchFxcZFxchKh4aGhoaHiomKSMjIykmLy8qKi8vOjo4OjpA QEBAQEBAQEBA/8AAEQgAwgBkAwEiAAIRAQMRAf/EAJgAAAEFAQEBAAAAAAAA AAAAAAADBAUGBwIBCAEBAQEBAQAAAAAAAAAAAAAAAAECAwQQAAEDAgMCCQgG CAYDAAAAAAEAAgMRBCESBTEGQVFxsSITcxQ1YYGRMnKy0jShwUJSM0PRYiNT g3QVFoKSosIkB/DhkxEBAQACAQMEAgMAAAAAAAAAAAERAjEhEgNBUZETYVKB IjL/2gAMAwEAAhEDEQA/ANJVPP8A2Po4kdH3a5qxxaTSOmBp+8VwWBz16+4I ND1hxHtOQapHv3pkgqLeccoZ8aV/vTTv3E/oZ8az/TdLurm0Zcm7MbZK5WAV PRNMaUonf9FmO28f/q+JMxMrp/eum/uZ/Q340f3tpv7mb0N+JUgaG6Rzm96d VvG4ivpeFBXzTa3MluayZKdISHGorwFydDq1F2/Wlt2wT+hnxrgb/aUTQQXH oZ8ayN9zj6pH+Ny8bdZT6p/zuV6HVsbN9dOfsgn9DPjS7d6rJ2yGb0N+JZho wdeNe7M6Pq3sZTM5wOfZ9pqu+m7rR3lo2fvcja1BAaaYfxE6HVLS73WEQq6G Y8gb8SktK1OHVLQXcDXMYXFtH0Bq3kJWfa1Yf03UBatldKBiXOJFQ6NzqZS4 7CrfuZ4K3tZOdLgzcp9CEKKFhAi66+khrl6yfLXiq5wW7rDbbxU/zI98olSV pFLaRFr7uSCPOQxraAOAwzCp405Li+KR8N9K98bS4gu4Bx0KbytkktainQnl biaYGjuIpW3AFvP950by7h4OBDDmGMSsa6S9dGXirjmwB4jwqH1AmK4pC9t4 0k1eGPDhQ06ReOZTVpm6plLQTdE4GnS/W+8oPVfmelF3F1T0QJRmx29M0w8i pEjHpmkSaLcXst41moRupFa0YMwFKDKemc1TiMAoljGZgMoxI4ArLa95/tS+ DdNZJCXkuvCWhzaZekGu6ZyeQ0xVbZtVguMWjaCw0Zdt8z4v0L2ZxtZOps7l 7oKAgtkwqdvqUCbHSJ7e0sJpLYs7wCTJmBz16batqcvRSty2NszzEzqoXGsc dcxa3iqp+GTS8ke+WNz3Oe7MRmc7Mfw3faV43M8Fb2r+dUS49aL2z7j1e9zP BW9rJzpVnKfQhCjQWG23ip/mB75W5LDbbxU/zA98olTNwC2B7GCp64uoMTjg k7Z2aOUcBik+hpKXvZI3xsMMbm5MzJjStZQ5zq+dpCZWDJ2C4MmLXslc0H7I yO/8opvr/bM9+F9Di27v3dhffGF4+wAOjyYVUTfRW7y9/eTcTNJLGOikIPkz l1BU+SimrHvBtmhlmycU2k48uCg9QZJ1crpLEW7A4ZpxG8FpcQW0c5/2uRVI lbV9kN271j9SkhuC6rLEEBj/AFaAspmdn4SDwYqDj2hWOwOof2pqDIrOGS1L iXzucA+gawucGUJdkFCDUbVXIfWxWoLJp971rYY765uHQwNpFG0D9nUYhpc9 2HmTu5NsXg2z5HsLekZaZgfNgubfR76CCxkfBC8X9GQZiSczyHsL8RTo/Qlt QtJbO7kglaxj20OWKuShGGWuKdERlxti9s+49XvczwUdq/nCodz+V7Z9x6ve 5fgje1fzhKTlYEIQstBYZb+Kn+YHvrc1g75HRXk0rMHslLm8ocShVk1Bw6vG gPWOpQ7RThHGkICeqlHB1UnuOT3d6TUtQikl62EEH8yMOP0OaAnd1YalLG+M yQgO6Li1gaeIjGRLzlFejt9VdCHQNk6k+rTZt4POo2a2nzOMlvKSekXGN5HH WtKKdmtdSs2iMXZYweq1tCB/lcoUxajIZWMccraFwM+QEPGYdFzgNnArkha2 s9al0q6ntes/pjDW5a14DXFoDiclelQUJoEwiJBqNqdQP1plrLZQv6u1lc4T RddGAS2jXbTXlptTCSSS2kMcjBnbicrw4Y+VlQmRa9Fnt5BAzVZrjq4QcjIz VrOEdX0iR6FJai6xfOH2T5pGubWR85JcXcpx2KH0S2N1IGyua1rXZT1TmyP2 AgtbhUYq0/0O0jEmaS5d1T2xnLE3Eu4RtTMTqrFz+X7Z9x6vW5fgje1fzhUz VbdttcmFpLmskcATgT0ZArnuV4I3tZOcK0nKwIQhZaCwWUVurhuysjsf8RW9 LBZPnJu0d7xQSen96gaRBcviDvWDaivoKdOkvT611IfT8Sb2nqhOCqwa1llG bvMlMeDi86ZyWNsSS5xLjiTlbt9Kcwn9lj953OkpCrhcmhsrUHa4/wCFqUis rao2+hv6EEpaFMLeD/T2COZwt3mN8VBnaMpqR+rRTbRdPb0rqQ+c/W4qC00n vt2OJzPpYrDH6isZRl1BlOZ0jn5SSAabaEY4eVXTcrwRvayc4VQvTWqt+5Xg g7WTnClWcrChCFloLBpPnJu0d7xW8rBZPnJu0d7xRKlbT1U4cU3tfVS7lWTG I/sj7TklIUpF+GfackZCqsJE4paA4psTil4DikWpHTR/zLryuj9xWFnqKvWH zc/lye6rBGasVjJhecKuO5Xgg7WTnCp17QVVx3J8EHayc4U2WcrChCFloLBZ PnJu0d7xW9LBZPm5u0dzlEqUtTgnDtia2mxOH7FayYR/hn23JGUpWP8ADPtu SEpVWEScUvb7U34U4g2pFqTsfm5uRnuqeiPQUBYn/lzckfuqdjPRVjJjfO2q 57keBt7WTnCpd5wq6bj+BN7WTnClWcrEhCFloLBJPm5u0d7xW9rBZPm5u0d7 xQqRtTQJw44JvbbEu84KuZgw/s3e25ISFKsqGur976aCqReq3CXCl4dqQO1L QnEJCpSy+bl8oZzKbjPRUFZhwuXk8IB5QfV9Cmoz0cVWTW82FXTcfwIdtJzh Ui9mjDTV4HKQrtuNX+hCu3rpOdSrOVjQhCy0FgV7UCdwNCJziPKSt9WBX5o2 ccc55ygRmmlYyAxyOaTGC6hOJqUn368H5zvOapNz84aPujKFwhgoby5GHWH6 F4Lm5eaAlx8gB+pIu2rUtKt7eCxtpNMit2mSNrg+VhJdmGOZzQTtVnVnbbtx 05ZoXXm0tePLl/8AS47zOPtkLVm2e8Erjnu7IRHa1sT3GnEK0UHvTu1by92k juLa1kcSx8sp6oSONMrQGgjzphmeTr1nwpztRmfbRRAZJIy7NO1zg94Oxrsd jeBIGaZ3rSPdyuJ+tFzazWdzLa3Dcs0Lix4rXEeVchR0egft2+01bVuT4Ke3 l51i5+Zb7TfqWz7j+CHt5edX0FjQhCgFgGpbZO2ct/WA6ltl8kzvrQMG8a8X WWgafvCv0p6zTJHtDhICDxCqlsnLWnj23uNZnCPyPd6rSeRXvdC+kmsO6SVE tocAdpidsPmOCq3dpbZpY1pe5+OamwbNg2qQ0e7h0+4NyHPEpaWO61ji1wPB Ruxa1s5yx5fHtM63W5jRo+sc3i8gVT3m1Pd2SW3fPK69msi7LZRfhOeT+bJw YjEBOHb2sMRbE+KIkUz0eXDgqA4UVPfBZQukIm66riQQ12I5KJ8OWmlz1m3w ZXl5LfXUt1NTrpnl76Cgx4AOIJEKSZaRXZOUlgZ5Mu3l2pC8tI7UhucmQ0OU 02HGuCz3TOHp+rfs78Y1NWn9s0+ULatx/BD28vOsVqOtafKFte5Hgh7eXnWv RhYkIQoBYDqX53bHnK35fP8AqD8z5gMQJTX0lAzc6rWjiFF62edjcjJHNZ90 EgLyNnWODK5SdleEpY2MnGFLj1a1m3Ouf4IGWU7XuPnK5qeMpx3N/C5ddxP3 kzDt3vua1PGvKlP2acHEAuI5EpLpcbBUPcTyBO6NfVv7IypRVPDYHgf9C4ls zEwvc7AbMNpTMZum3sbg9IHyhbbuR4J/Gk51iQBqFtW4crJdAD2GreukFfOF WVlQhCAWC3NtE+6mqD+I7YfKVvSzm9/671MSyS2tzDM1znODX5o3UOP6w+lB Sm6bC7Y97fQf0J02ItaAXhxH2i3Hz0cpt+5+8UJA7mZMK1Y+M/7gmz9C1pta 2FxhtpG4j6AlkvJrvvr/AJuEU5jhskYOVhP+9cEyD85g/hH4k+k07UW+taTj ljf8KRdpmpbe5z0PD1T6enKnbr7Nfb5P2psJJmnCdv8A8v0lddbO/bOD/Cb8 SVGl6i40FpOT2T/hTiHQ9ZeehYXDv4T/ANCduvtF+3yftt8kYYHP9aX0RgfW nB0q1mIMzpJMuwAtYP8AS1SNpu5rrsRYygD7wDPfLVKQbq628AugEdcOm9uH LlJVxrOJGL5PJel2titO06wi9S3FeNxLveK0XchoboTQAGjrZMAKDaFGR7j3 cn49zHHtqGNL+TbkVm0jS49KsxaRyOlaHF2ZwANXciWsyXJ+hCFGghCEAhCE AhCEAhCEAhCEAhCEAhCEH//Z --= Multipart Boundary 0506021616-- From aahz@pythoncraft.com Mon May 6 17:07:56 2002 From: aahz@pythoncraft.com (Aahz) Date: Mon, 6 May 2002 12:07:56 -0400 Subject: [Python-Dev] file(fileinput.input()) In-Reply-To: <200205061411.g46EBrk05022@pcp742651pcs.reston01.va.comcast.net> References: <20020506140211.GA939@panix.com> <200205061411.g46EBrk05022@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020506160756.GA5592@panix.com> On Mon, May 06, 2002, Guido van Rossum wrote: >Aahz: >> >> Should file(fileinput.input()) work? Currently it raises an >> exception because fileinput.input() returns neither a string nor a >> buffer object. > > What on earth would you want it to do? It doesn't seem to make any > sense to me. Well, if it should work, it should return the FileInput instance (i.e. return self). On thinking further, I'm going to use a slightly different question divorced from my current problem: What should list(UserList()) return? What about class myList(list): pass l = list(myList()) I suppose this question is related to PEPs 245/246. The broader question is, what should a constructor return when called on a related object or subclass? Getting back to my specific example, I've got this function: def grep(f, regex): f = file(f) regex = re.compile(regex) for line in f: if regex.search(line): yield line I was originally passing in file handles or filenames, demonstrating to the students that calling a constructor on an existing object returns that object if it doesn't need to do any real work. Alex suggested that I use fileinput.input(), whereupon my function blew up. The meta question is, what kind of programming style do we want to push? More smarts on top or bottom? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ See me at OSCON! I'm teaching Python for [Perl] Programmers, a fast intro for all experienced programmers (not just Perl). Early bird ends June 10. http://conferences.oreillynet.com/os2002/ From guido@python.org Mon May 6 17:21:13 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 06 May 2002 12:21:13 -0400 Subject: [Python-Dev] file(fileinput.input()) In-Reply-To: Your message of "Mon, 06 May 2002 12:07:56 EDT." <20020506160756.GA5592@panix.com> References: <20020506140211.GA939@panix.com> <200205061411.g46EBrk05022@pcp742651pcs.reston01.va.comcast.net> <20020506160756.GA5592@panix.com> Message-ID: <200205061621.g46GLDC05555@pcp742651pcs.reston01.va.comcast.net> > >Aahz: > >> Should file(fileinput.input()) work? Currently it raises an > >> exception because fileinput.input() returns neither a string nor a > >> buffer object. Guido: > > What on earth would you want it to do? It doesn't seem to make any > > sense to me. Aahz: > Well, if it should work, it should return the FileInput instance (i.e. > return self). Hm, what analogy would lead you to that thinking? 'file' is the concrete file type. > On thinking further, I'm going to use a slightly > different question divorced from my current problem: > > What should list(UserList()) return? That's perfectly well-defined already: a list consisting of the items of the UserList. > What about > > class myList(list): > pass > l = list(myList()) > > I suppose this question is related to PEPs 245/246. The broader > question is, what should a constructor return when called on a related > object or subclass? I don't know where you want to go with this. If Foo is a class, Foo(...) should return a foo instance constructed from the arguments. > Getting back to my specific example, I've got this function: > > def grep(f, regex): > f = file(f) > regex = re.compile(regex) > for line in f: > if regex.search(line): > yield line > > I was originally passing in file handles or filenames, demonstrating to > the students that calling a constructor on an existing object returns > that object if it doesn't need to do any real work. But that's not even a rule! str() and tuple() can return the original, but only if has exactly the required type. list() always returns a new list -- that's its *purpose*. > Alex suggested that > I use fileinput.input(), whereupon my function blew up. > > The meta question is, what kind of programming style do we want to push? > More smarts on top or bottom? Teach them Python, please. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com Mon May 6 17:32:23 2002 From: skip@pobox.com (Skip Montanaro) Date: Mon, 6 May 2002 11:32:23 -0500 Subject: [Python-Dev] Weekly Python Bug/Patch Summary In-Reply-To: <200205051200.g45C02807321@localhost.localdomain> References: <200205051200.g45C02807321@localhost.localdomain> Message-ID: <15574.45079.119219.587594@12-248-41-177.client.attbi.com> Sorry about the empty email from Sunday morning. I was off-net and my SF checker simply blindly piped its null results into a mail command. I'll twiddle things to avoid that in the future. Skip From sjmachin@lexicon.net Mon May 6 22:28:31 2002 From: sjmachin@lexicon.net (John Machin) Date: Tue, 07 May 2002 07:28:31 +1000 Subject: [Python-Dev] file(fileinput.input()) In-Reply-To: <20020506160756.GA5592@panix.com> Message-ID: 07/05/2002 2:07:56 AM, Aahz wrote: [snip] >Getting back to my specific example, I've got this function: > > def grep(f, regex): > f = file(f) > regex = re.compile(regex) > for line in f: > if regex.search(line): > yield line > >I was originally passing in file handles or filenames, demonstrating to >the students that calling a constructor on an existing object returns >that object if it doesn't need to do any real work. Since when? Where is this documented? The following simple example concords with the documentation -- "The first two arguments are the same as for stdio's fopen(): filename is the file name to be opened," -- and seems to flatly contradict you. Python 2.2.1 (#34, Apr 9 2002, 19:34:33) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> foo = file(sys.stdout) Traceback (most recent call last): File "", line 1, in ? TypeError: coercing to Unicode: need string or buffer, file found >>> bar = file(sys.stdin) Traceback (most recent call last): File "", line 1, in ? TypeError: coercing to Unicode: need string or buffer, file found >>> The fact that str('foo') -> 'foo' is due to the historical fact that str() started out as a builtin str(42) -> '42' and very much later also became a constructor, NOT due to some general rule that ctor(ctor(some_args)) == ctor(some_args). >Alex suggested that >I use fileinput.input(), whereupon my function blew up. Was Alex expecting this to work or to fail? >The meta question is, what kind of programming style do we want to push? >More smarts on top or bottom? A general 'rule' that a constructor should accept an instance of the class and silently do nothing with it would be IMO a totally unnecessary burden on the author of the constructor for no perceived gain. From aahz@pythoncraft.com Tue May 7 05:23:23 2002 From: aahz@pythoncraft.com (Aahz) Date: Tue, 7 May 2002 00:23:23 -0400 Subject: [Python-Dev] file(fileinput.input()) In-Reply-To: References: <20020506160756.GA5592@panix.com> Message-ID: <20020507042323.GA18689@panix.com> On Tue, May 07, 2002, John Machin wrote: > > Since when? Where is this documented? The following simple example concords > with the documentation -- "The first two arguments are the same as for > stdio's fopen(): filename is the file name to be opened," -- and seems to > flatly contradict you. > > Python 2.2.1 (#34, Apr 9 2002, 19:34:33) [MSC 32 bit (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> import sys > >>> foo = file(sys.stdout) > Traceback (most recent call last): > File "", line 1, in ? > TypeError: coercing to Unicode: need string or buffer, file found > >>> bar = file(sys.stdin) > Traceback (most recent call last): > File "", line 1, in ? > TypeError: coercing to Unicode: need string or buffer, file found Oops. "I know I tested it. Really, I did." -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ See me at OSCON! I'm teaching Python for [Perl] Programmers, a fast intro for all experienced programmers (not just Perl). Early bird ends June 10. http://conferences.oreillynet.com/os2002/ From niemeyer@conectiva.com Mon May 6 14:45:44 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Mon, 6 May 2002 10:45:44 -0300 Subject: [Python-Dev] Problems with Python's default dlopen flags In-Reply-To: References: <861BB3F0-6065-11D6-A113-003065517236@oratrix.com> Message-ID: <20020506104544.A2798@ibook.distro.conectiva> Hello Martin! > On Unix, you'll need to set LD_LIBRARY_PATH, or put the library into a > directory where ld.so automatically searches for libraries; that is > a deployment problem. That's solvable. He could create a custom distutils class to include a parameter like -Wl,-rpath, at compile time. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From akuchlin@mems-exchange.org Tue May 7 22:08:04 2002 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Tue, 07 May 2002 17:08:04 -0400 Subject: [Python-Dev] Incorrect Misc/NEWS entry Message-ID: One entry in Misc/NEWS reads: - Objects allocated using the new PyMalloc_New and PyMalloc_NewVar functions will be allocated using pymalloc if it is enabled. These objects should be deallocated using PyMalloc_Del. The PyObject_{New, NewVar,NEW_VAR,Del,DEL} APIs have been changed to always use PyMem_MALLOC and PyMem_FREE, even if pymalloc is enabled. The PyCore_* family of APIs have been removed. I can't find PyMalloc_New anywhere in the source code. Is this perhaps a typo for PyObject_New()? Also, PLAN.txt is still in the top-level directory. Is it still relevant for 2.3, or was everything in it finished for 2.2? --amk From barry@barrys-emacs.org Tue May 7 23:41:56 2002 From: barry@barrys-emacs.org (Barry Scott) Date: Tue, 7 May 2002 23:41:56 +0100 Subject: [Python-Dev] No "check syntax" command line option In-Reply-To: <002701c1f3b1$66b84d80$ced241d5@hagrid> Message-ID: <001001c1f618$6403e770$070210ac@LAPDANCE> Oh. I've been using python -c "import py_compile,sys;sys.stderr = sys.stdout;py_compile.compile('$file')" Is this better or worse? BArry -----Original Message----- From: python-dev-admin@python.org [mailto:python-dev-admin@python.org]On Behalf Of Fredrik Lundh Sent: 04 May 2002 22:20 To: Jack Jansen; python-dev@python.org Subject: Re: [Python-Dev] No "check syntax" command line option jack wrote: > Is there a roundabout way to do this? python -c "compile(open('$filename').read(), '$filename', 'exec')" works with all versions. _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev From guido@python.org Wed May 8 00:42:23 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 07 May 2002 19:42:23 -0400 Subject: [Python-Dev] Incorrect Misc/NEWS entry In-Reply-To: Your message of "Tue, 07 May 2002 17:08:04 EDT." References: Message-ID: <200205072342.g47NgNr09508@pcp742651pcs.reston01.va.comcast.net> > I can't find PyMalloc_New anywhere in the source code. Is this perhaps > a typo for PyObject_New()? I'll let Tim answer that. > Also, PLAN.txt is still in the top-level directory. Is it still > relevant for 2.3, or was everything in it finished for 2.2? Some of what it says is still relevant, alas. My aim is to be able to remove it before 2.3 final ships, but right now I am busy on Zope Corp customer projects. --Guido van Rossum (home page: http://www.python.org/~guido/) From nas@python.ca Wed May 8 01:46:54 2002 From: nas@python.ca (Neil Schemenauer) Date: Tue, 7 May 2002 17:46:54 -0700 Subject: [Python-Dev] Lazily GC tracking tuples In-Reply-To: <20020504092554.37178.qmail@web14706.mail.yahoo.com>; from evilzr@yahoo.com on Sat, May 04, 2002 at 02:25:54AM -0700 References: <20020504092554.37178.qmail@web14706.mail.yahoo.com> Message-ID: <20020507174654.A21398@glacier.arctrix.com> I'm not sure I like this approach. Slowing down PyTuple_SET_ITEM to optimize a micro-benchmark seems dubious. However, dropping the number of tracked objects by 50% _does_ seem worthwhile. Couldn't we get the same effect by checking and untracking the appropriate tuples after finishing a gen1 collection? Each tuple would be checked once and short lived tuples will probably not be checked at all. Neil From nas@python.ca Wed May 8 01:50:23 2002 From: nas@python.ca (Neil Schemenauer) Date: Tue, 7 May 2002 17:50:23 -0700 Subject: [Python-Dev] Incorrect Misc/NEWS entry In-Reply-To: ; from akuchlin@mems-exchange.org on Tue, May 07, 2002 at 05:08:04PM -0400 References: Message-ID: <20020507175023.B21398@glacier.arctrix.com> Andrew Kuchling wrote: > One entry in Misc/NEWS reads: > > - Objects allocated using the new PyMalloc_New and PyMalloc_NewVar > functions will be allocated using pymalloc if it is enabled. These > objects should be deallocated using PyMalloc_Del. The PyObject_{New, > NewVar,NEW_VAR,Del,DEL} APIs have been changed to always use > PyMem_MALLOC and PyMem_FREE, even if pymalloc is enabled. The > PyCore_* family of APIs have been removed. > > I can't find PyMalloc_New anywhere in the source code. Is this perhaps > a typo for PyObject_New()? Oops, that's written by me and is now totally wrong. PyObject_{New,...} will allocate using pymalloc if it's enabled. Forget about PyMalloc_{New,NewVar}. Neil From tommy@ilm.com Wed May 8 02:08:46 2002 From: tommy@ilm.com (Mr. Bawkbugawk) Date: Tue, 7 May 2002 18:08:46 -0700 (PDT) Subject: [Python-Dev] import Numeric fails with user-supplied __import__? Message-ID: <15576.31242.916416.115906@mace.lucasdigital.com> Hi All, I posted this to the numpy discussion list yesterday and haven't heard back, so I thought I'd swing it through here as well... ------------%< snip %<----------------------%< snip %<------------ Hey Folks, using the recently released python 2.1.3 with version 20.3 of the Numeric package, we are getting this behavior: (tommy@mace)/u0/tommy$ python2 Python 2.1.3 (#1, Apr 22 2002, 16:14:11) [C] on irix6 Type "copyright", "credits" or "license" for more information. >>> import ihooks >>> ihooks.install() >>> import Numeric Segmentation fault this is happening on both irix and linux. I checked the release notes for Numeric 20.1 but didn't see any mention of import bugs being fixed. a little bit of investigation shows that it isn't even ihooks that causes this, it is any user-supplied __import__: blah.py: import __builtin__ class blah: def __init__( self ): self.real_importer = __builtin__.__import__ __builtin__.__import__ = self.my_import def my_import( self, name, globs={}, locs={}, fromlist=[] ): return self.real_importer( name, globs, locs, fromlist ) _blah = blah() 859:tpelle@parro > python2 Python 2.1.3 (#1, Apr 19 2002, 21:22:42) [GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-81)] on linux2 Type "copyright", "credits" or "license" for more information. >>> import blah >>> import Numeric Segmentation fault (core dumped) anyone out there seen this before? any guess if its Numeric's problem or perhaps a bug deeper down in python's import architecture? any and all clues appreciated, thanks! From David Abrahams" FYI, Ralf Grosse-Kunstleve has reduced the exception-handling problem mentioned here http://mail.python.org/pipermail/c++-sig/2002-May/001021.html to a minimal example: http://cci.lbl.gov/~rwgk/tmp/gcc_dl_eh.tar.gz gunzip -c gcc_dl_eh.tar.gz | tar xvf - cd gcc_dl_eh more 0README The problem here is clearly a GCC/Linux interaction problem, *not* a Python bug. However, it does have an impact on anyone writing Python extension modules with g++ on Linux. -Dave From tim_one@email.msn.com Wed May 8 05:53:38 2002 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 8 May 2002 00:53:38 -0400 Subject: [Python-Dev] Lazily GC tracking tuples In-Reply-To: <20020507174654.A21398@glacier.arctrix.com> Message-ID: [Neil Schemenauer, on Daniel Dunbar's interesting tuple-untrack scheme] > I'm not sure I like this approach. Slowing down PyTuple_SET_ITEM to > optimize a micro-benchmark seems dubious. However, dropping the number > of tracked objects by 50% _does_ seem worthwhile. Couldn't we get the > same effect by checking and untracking the appropriate tuples after > finishing a gen1 collection? Each tuple would be checked once and short > lived tuples will probably not be checked at all. I agree with Neil here. Do you really mean after a gen1 collection? Or after a gen0 collection? Or before a gen2 collection ? The micro-optimizer in me would like to combine testing for "safe to untrack" objects with the whole-list crawl being done anyway in update_refs() at the start of a collection. Offhand I guess I'd do that at the start of a gen0 collection. If that untracks a tuple that was going to die soon anyway, it's not really a waste of time, because untracking at the very start saves the rest of the collection from crawling over it twice (subtract_refs won't see it if it gets untracked, and ditto move_roots). So that seems a net win. OTOH, it that decides a tuple isn't safe to untrack, and the tuple was going to die soon anyway, it's an "extra" peek at the tuple guts. That's a net loss. But I expect an overwhelming majority of tuples are safe to untrack, so playing a frequent net win against an infrequent net loss seems a net win. From martin@v.loewis.de Wed May 8 06:04:02 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 08 May 2002 07:04:02 +0200 Subject: [Python-Dev] Lazily GC tracking tuples In-Reply-To: References: Message-ID: "Tim Peters" writes: > The micro-optimizer in me would like to combine testing for "safe to > untrack" objects with the whole-list crawl being done anyway in > update_refs() at the start of a collection. But that brings us back to determining whether an object is "safe to untrack". Are you suggesting that update_refs should know what tuples are? Regards, Martin From martin@v.loewis.de Wed May 8 06:09:16 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 08 May 2002 07:09:16 +0200 Subject: [Python-Dev] import Numeric fails with user-supplied __import__? In-Reply-To: <15576.31242.916416.115906@mace.lucasdigital.com> References: <15576.31242.916416.115906@mace.lucasdigital.com> Message-ID: "Mr. Bawkbugawk" writes: > Segmentation fault (core dumped) > > > anyone out there seen this before? Yes, I've seen it before. It wasn't when importing Numeric, though, but when reading a file, and it wasn't with Python, but with Mozilla, and it wasn't on Linux, but on Solaris :-) Seriously, the question "has anybody seen segfaults when importing Numeric" is not appropriate for this list. To post to this list, *at a minimum* you should provide a gdb backtrace. Otherwise, use your usual Numeric or Python support channels. Regards, Martin From tim_one@email.msn.com Wed May 8 06:20:13 2002 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 8 May 2002 01:20:13 -0400 Subject: [Python-Dev] Lazily GC tracking tuples In-Reply-To: Message-ID: [martin@v.loewis.de] > But that brings us back to determining whether an object is "safe to > untrack". Are you suggesting that update_refs should know what tuples > are? To start with, sure, and maybe to end with. Tuples are the only type where we have actual evidence that it can make a lick of difference. I don't want to delay getting an instant win there. If somebody cares enough to invent a Thoroughly General Protocol, and can convince Guido it's not just YAGNI, fine. The 6 lines of tuple-checking code can be reworked easily to use the protocol when it exists. Believe me : we already spent more time typing about this than it would have taken to write the tuple code ten times over, and tuples are still the only specific type anyone has mentioned as being a potential winner for this trick. From rwgk@cci.lbl.gov Wed May 8 08:13:59 2002 From: rwgk@cci.lbl.gov (Ralf W. Grosse-Kunstleve) Date: Wed, 8 May 2002 00:13:59 -0700 (PDT) Subject: [Python-Dev] MacOSX dlopen equivalent? Message-ID: <200205080713.g487DxU49132@boa.lbl.gov> I am interested in investigating the feasibility of porting the Boost.Python library to Mac OSX. A very good first step would be if we could port the example that I just made for tracking down gcc exception handling bugs (http://cci.lbl.gov/~rwgk/tmp/gcc_dl_eh.tar.gz) because it is both a small and an accurate model of the mechanism used by Boost.Python. Unfortunately I did not get very far: # make macosx make CC=cc CPP="c++" LDDLL=-shared LIBDL="" cc -g -c master.c master.c:8: header file 'dlfcn.h' not found Looking at the Python source to figure out what is used in place of dlopen was not immediately inspiring. Then I found dlcompat. It is not clear to me what is the best route to choose for my feasibility test. Are there resources that could give me some guidance? Thanks, Ralf From bernie@3captus.com Mon May 6 21:26:03 2002 From: bernie@3captus.com (Bernard Yue) Date: Mon, 06 May 2002 14:26:03 -0600 Subject: [Python-Dev] timeoutsocket.py References: Message-ID: <3CD6E6DB.7875130A@3captus.com> Timothy O'Malley wrote: > > hola. > > That's awesome that you put the timeout functionality into the C-level > socket module. I'd vote that it is "The Right"(tm) way to include this > feature in the standard library. You guys rock! > [snip] > I couldn't get to the web site you indicated, so pardon this question > if I could have found the answer on-line somewhere. > The site was down . IS is now back up. > How did you handle the _fileobject class in the standard socket.py? We didn't. > The version in Python-2.2 will lose data if the underlying socket raises > an exception during a read() or readlines(). Basically, these two > routines use a local variable to store the incoming data -- and that > local variable will go out of scope after the exception during recv(). > I'll modify the test case to see what happens. > I believe that the TimeoutFile class in the socket.py version I thew > together fixes this issue. Of course, you may have already noticed > and fixed this, too. > So it is why TimeoutFile was there in the first place. Bernie -- There are three schools of magic. One: State a tautology, then ring the changes on its corollaries; that's philosophy. Two: Record many facts. Try to find a pattern. Then make a wrong guess at the next fact; that's science. Three: Be aware that you live in a malevolent Universe controlled by Murphy's Law, sometimes offset by Brewster's Factor; that's engineering. -- Robert A. Heinlein From Jack.Jansen@cwi.nl Wed May 8 10:06:25 2002 From: Jack.Jansen@cwi.nl (Jack Jansen) Date: Wed, 8 May 2002 11:06:25 +0200 Subject: [Python-Dev] MacOSX dlopen equivalent? In-Reply-To: <200205080713.g487DxU49132@boa.lbl.gov> Message-ID: On Wednesday, May 8, 2002, at 09:13 , Ralf W. Grosse-Kunstleve wrote: > I am interested in investigating the feasibility of porting the > Boost.Python library to Mac OSX. [...] > Looking at the Python source to figure out what is used in place of > dlopen was not immediately inspiring. Then I found dlcompat. I haven't used dlcompat but people have reported success using it with Python. But I don't known whether mixing dlcompat and dyld is a good idea, so you may have to build a special Python that uses dynload_dl with dlcompat in stead of dynload_next. Another but: if I look at the readme for your test code I get the impression that the gcc that OSX has (2.92.2) may be a problem. gcc 3 is going to be supported in the next major release (late summer, according to apple). -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From simon@netthink.co.uk Wed May 8 11:02:46 2002 From: simon@netthink.co.uk (Simon Cozens) Date: Wed, 8 May 2002 11:02:46 +0100 Subject: [Python-Dev] MacOSX dlopen equivalent? In-Reply-To: <200205080713.g487DxU49132@boa.lbl.gov> References: <200205080713.g487DxU49132@boa.lbl.gov> Message-ID: <20020508100246.GA12815@netthink.co.uk> Ralf W. Grosse-Kunstleve: > Unfortunately I did not get very far: > > # make macosx > make CC=cc CPP="c++" LDDLL=-shared LIBDL="" > cc -g -c master.c > master.c:8: header file 'dlfcn.h' not found A decent workaround is to use fink (http://fink.sourceforge.net/) which includes a working dlfcn.h and dl library. You'll want the following flags: CC: -I/sw/include LD: -L/sw/lib -flat_namespace libs: -ldl -- Putting heated bricks close to the news.admin.net-abuse.* groups. -- Megahal (trained on asr), 1998-11-06 From guido@python.org Wed May 8 14:18:53 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 08 May 2002 09:18:53 -0400 Subject: [Python-Dev] Patch #552433 In-Reply-To: Your message of "Wed, 08 May 2002 01:44:23 PDT." References: Message-ID: <200205081318.g48DIrY28337@pcp742651pcs.reston01.va.comcast.net> > Patch #552433: Special-case tuples. Avoid sub-type checking for lists. > Avoid checks for negative indices and duplicate checks for support of > the sequence protocol. I've missed the review of this patch. Why bother special-casing tuples? Who's ever going to iterate over a tuple long enough that it makes a difference? --Guido van Rossum (home page: http://www.python.org/~guido/) From loewis@informatik.hu-berlin.de Wed May 8 14:41:51 2002 From: loewis@informatik.hu-berlin.de (Martin v. =?iso-8859-1?q?L=F6wis?=) Date: Wed, 8 May 2002 15:41:51 +0200 (CEST) Subject: [Python-Dev] AtheOS port Message-ID: <200205081341.g48Dfpgs006314@paros.informatik.hu-berlin.de> IMO, the AtheOS port (http://python.org/sf/488073) is ready for integration. Would anybody object if I commit this change? Regards, Martin From aahz@pythoncraft.com Wed May 8 14:49:34 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 8 May 2002 09:49:34 -0400 Subject: [Python-Dev] Patch #552433 In-Reply-To: <200205081318.g48DIrY28337@pcp742651pcs.reston01.va.comcast.net> References: <200205081318.g48DIrY28337@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020508134933.GA24723@panix.com> On Wed, May 08, 2002, Guido van Rossum wrote: > > > Patch #552433: Special-case tuples. Avoid sub-type checking for lists. > > Avoid checks for negative indices and duplicate checks for support of > > the sequence protocol. > > I've missed the review of this patch. Why bother special-casing > tuples? Who's ever going to iterate over a tuple long enough that it > makes a difference? Define "long enough"; I'm expecting that my BCD module will generate tuples at least a hundred elements. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ See me at OSCON! I'm teaching Python for [Perl] Programmers, a fast intro for all experienced programmers (not just Perl). Early bird ends June 10. http://conferences.oreillynet.com/os2002/ From guido@python.org Wed May 8 15:01:19 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 08 May 2002 10:01:19 -0400 Subject: [Python-Dev] Patch #552433 In-Reply-To: Your message of "Wed, 08 May 2002 09:49:34 EDT." <20020508134933.GA24723@panix.com> References: <200205081318.g48DIrY28337@pcp742651pcs.reston01.va.comcast.net> <20020508134933.GA24723@panix.com> Message-ID: <200205081401.g48E1JY28709@pcp742651pcs.reston01.va.comcast.net> > Define "long enough"; I'm expecting that my BCD module will generate > tuples at least a hundred elements. Hm. Creating such a long tuple in Python code will be the bottleneck in your program, not iterating over it. Have you benchmarked this particular fix? I find it hard to believe that one can even notice the difference. --Guido van Rossum (home page: http://www.python.org/~guido/) From jason@tishler.net Wed May 8 15:06:27 2002 From: jason@tishler.net (Jason Tishler) Date: Wed, 08 May 2002 10:06:27 -0400 Subject: [Python-Dev] Python 2.3 patch cut-off date Message-ID: <20020508140627.GH1700@tishler.net> I reviewed the schedule for Python 2.3: http://www.python.org/peps/pep-0283.html What is the last date that patches will be accepted for 2.3? Is the answer to the previous question dependent of the nature of the patch? I'm mainly interested in submitted two more Cygwin Python build related patches and I was wondering whether or not I was running out of time. Thanks, Jason From nas@python.ca Wed May 8 15:13:35 2002 From: nas@python.ca (Neil Schemenauer) Date: Wed, 8 May 2002 07:13:35 -0700 Subject: [Python-Dev] Lazily GC tracking tuples In-Reply-To: ; from tim_one@email.msn.com on Wed, May 08, 2002 at 12:53:38AM -0400 References: <20020507174654.A21398@glacier.arctrix.com> Message-ID: <20020508071335.A23115@glacier.arctrix.com> Tim Peters wrote: > Do you really mean after a gen1 collection? Yes. By that time the tuple was probably around for a while. Also, the chances that it will be mutated will be smaller. Neil From akuchlin@mems-exchange.org Wed May 8 15:19:20 2002 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Wed, 8 May 2002 10:19:20 -0400 Subject: [Python-Dev] Python 2.3 patch cut-off date In-Reply-To: <20020508140627.GH1700@tishler.net> References: <20020508140627.GH1700@tishler.net> Message-ID: <20020508141920.GB10180@ute.mems-exchange.org> On Wed, May 08, 2002 at 10:06:27AM -0400, Jason Tishler wrote: >What is the last date that patches will be accepted for 2.3? Is the >answer to the previous question dependent of the nature of the patch? The answer to your second question is "yes". Channelling the Pythoneers: bugfixes, especially small ones, can go in at any time, but the sooner the better so they get more testing. Build problems can be complicated, so I'd suggest getting your Cygwin patches in as soon as possible, but if the patches are small and "obviously correct" or only affect Cygwin builds, there shouldn't be much controversy over them. The real deadline applies to PEPs. I'd expect that no significant new features would be added after 2.3b1, so a PEP written after that date would have to be deferred until Python 2.4. --amk (www.amk.ca) "It doesn't work." " You astound me." -- The Doctor and the Brigadier, in "The Time Monster" From guido@python.org Wed May 8 15:42:01 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 08 May 2002 10:42:01 -0400 Subject: [Python-Dev] Python 2.3 patch cut-off date In-Reply-To: Your message of "Wed, 08 May 2002 10:06:27 EDT." <20020508140627.GH1700@tishler.net> References: <20020508140627.GH1700@tishler.net> Message-ID: <200205081442.g48Eg1a01105@odiug.zope.com> > I reviewed the schedule for Python 2.3: > > http://www.python.org/peps/pep-0283.html > > What is the last date that patches will be accepted for 2.3? Is the > answer to the previous question dependent of the nature of the patch? > > I'm mainly interested in submitted two more Cygwin Python build related > patches and I was wondering whether or not I was running out of time. The schedule is slipping; we're likely to switch to a more goal-oriented (as opposed to deadline-oriented) approach. That doesn't mean we're necessarily going to wait for Cywin patches! :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed May 8 16:21:00 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 08 May 2002 11:21:00 -0400 Subject: [Python-Dev] Re: Python insecure? Message-ID: <200205081521.g48FL0101329@odiug.zope.com> http://www.ornl.gov/its/archives/mailing-lists/qmail/2002/05/msg00309.html http://www.ornl.gov/its/archives/mailing-lists/qmail/2002/05/msg00415.html --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed May 8 16:44:39 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 08 May 2002 11:44:39 -0400 Subject: [Python-Dev] AtheOS port In-Reply-To: Your message of "Wed, 08 May 2002 15:41:51 +0200." <200205081341.g48Dfpgs006314@paros.informatik.hu-berlin.de> References: <200205081341.g48Dfpgs006314@paros.informatik.hu-berlin.de> Message-ID: <200205081544.g48FidD01393@odiug.zope.com> > IMO, the AtheOS port (http://python.org/sf/488073) is ready for > integration. Would anybody object if I commit this change? I'd like to state it differently. What's the compelling reason for this? Does anybody even know what AtheOS is? --Guido van Rossum (home page: http://www.python.org/~guido/) From evilzr@yahoo.com Wed May 8 16:32:52 2002 From: evilzr@yahoo.com (Daniel Dunbar) Date: Wed, 8 May 2002 08:32:52 -0700 (PDT) Subject: [Python-Dev] Lazily GC tracking tuples In-Reply-To: Message-ID: <20020508153252.344.qmail@web14704.mail.yahoo.com> (replying to Neil & Tim) --- Tim Peters wrote: > [Neil Schemenauer, on Daniel Dunbar's interesting tuple-untrack scheme] > > I'm not sure I like this approach. Slowing down PyTuple_SET_ITEM to > > optimize a micro-benchmark seems dubious. However, dropping the number > > of tracked objects by 50% _does_ seem worthwhile. Couldn't we get the That was basically the opinion I reached as well - changing PyTuple_SET_ITEM seems nasty in principle, and the performance results were dubious at best... the 50% drop in tracked objects also seemed like the most interesting thing to me, however, its not clear that that percentage would remain under the dynamic conditions of a real application. > > same effect by checking and untracking the appropriate tuples after > > finishing a gen1 collection? Each tuple would be checked once and short > > lived tuples will probably not be checked at all. The other thing to note is that untracking such tuples is a very small win regardless - if they were not untracked the tuple visit routine will quickly scan down the tuple (which I guess is likely to be small to start with), and the GC visitor will quickly reject adding any of the contained objects because they are not PyObject_IS_GC(). That suggests that the only case it would make a difference is a) when there are soooo many untrackable tuples that simply traversing them takes a large amount of time (ie. the iterzip case), or b) when there is a massive nesting of untrackable tuples - an large AST or binary-tree might benefit from this, but there are perhaps better solutions to that problem (see below) < snip - Tim talks about issues with removing tuples in the collector > I still don't understand how you 'check the tuple' and then untrack it - even if it has no NULL slots it still isn't safe to remove. I don't think I agree that special casing tuples in the collector is a good idea - this whole issue just came up because of the iterzip test, where it was noticed that the collector really didn't have to check all the objects, which is just a special case solution to the problem that many of the objects in the gen* lists are not going to be participating in a cycle. A solution to that general problem would be much more beneficial, although presumably much more work ;) If the collector could somehow prioritize objects, it seems like various heuristics could be used to gauge the likelyhood of an object participating in a cycle vs. the 'good' of collecting that cycle. As an aside: are there some nice established tests for estimating the benefit of modifications to the collector? Something that needs the collector (so getting cycles early keeps the memory consumption down, improving performance) and also has dynamic object allocation properties that mimic some ideal of real-applications? ===== daniel dunbar evilzr@yahoo.com __________________________________________________ Do You Yahoo!? Yahoo! Health - your guide to health and wellness http://health.yahoo.com From jeremy@zope.com Wed May 8 16:50:23 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Wed, 8 May 2002 11:50:23 -0400 Subject: [Python-Dev] Re: Python insecure? In-Reply-To: <200205081521.g48FL0101329@odiug.zope.com> References: <200205081521.g48FL0101329@odiug.zope.com> Message-ID: <15577.18751.32382.473985@slothrop.zope.com> It sure looks like FUD. Anyone know this Michael Sierchio guy? I've never heard of him, and Google / citeseer searches don't reveal much. I suppose the paper he mentioned was written by someone else. Jeremy From mgilfix@eecs.tufts.edu Wed May 8 17:28:04 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Wed, 8 May 2002 12:28:04 -0400 Subject: [Python-Dev] AtheOS port In-Reply-To: <200205081544.g48FidD01393@odiug.zope.com>; from guido@python.org on Wed, May 08, 2002 at 11:44:39AM -0400 References: <200205081341.g48Dfpgs006314@paros.informatik.hu-berlin.de> <200205081544.g48FidD01393@odiug.zope.com> Message-ID: <20020508122803.B23913@eecs.tufts.edu> Know yes. Use it myself, no. I'd also be interested in knowing who is AtheOS' niche market... On Wed, May 08 @ 11:44, Guido van Rossum wrote: > > IMO, the AtheOS port (http://python.org/sf/488073) is ready for > > integration. Would anybody object if I commit this change? > > I'd like to state it differently. What's the compelling reason for > this? Does anybody even know what AtheOS is? > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev `-> (guido) -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From gerhard@bigfoot.de Wed May 8 17:35:20 2002 From: gerhard@bigfoot.de (Gerhard =?iso-8859-15?Q?H=E4ring?=) Date: Wed, 8 May 2002 18:35:20 +0200 Subject: [Python-Dev] AtheOS port In-Reply-To: <20020508122803.B23913@eecs.tufts.edu> References: <200205081341.g48Dfpgs006314@paros.informatik.hu-berlin.de> <200205081544.g48FidD01393@odiug.zope.com> <20020508122803.B23913@eecs.tufts.edu> Message-ID: <20020508163520.GA902@lilith.my-fqdn.de> * Michael Gilfix [2002-05-08 12:28 -0400]: > Know yes. Use it myself, no. Same for me. > I'd also be interested in knowing who is AtheOS' niche market... Probably people who like to try out new OSes and those who did like BeOS and look for something similar (AtheOS is modeled after BeOS, IIRC). Gerhard -- mail: gerhard bigfoot de registered Linux user #64239 web: http://www.cs.fhm.edu/~ifw00065/ OpenPGP public key id AD24C930 public key fingerprint: 3FCC 8700 3012 0A9E B0C9 3667 814B 9CAA AD24 C930 reduce(lambda x,y:x+y,map(lambda x:chr(ord(x)^42),tuple('zS^BED\nX_FOY\x0b'))) From guido@python.org Wed May 8 17:56:53 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 08 May 2002 12:56:53 -0400 Subject: [Python-Dev] AtheOS port In-Reply-To: Your message of "Wed, 08 May 2002 12:28:04 EDT." <20020508122803.B23913@eecs.tufts.edu> References: <200205081341.g48Dfpgs006314@paros.informatik.hu-berlin.de> <200205081544.g48FidD01393@odiug.zope.com> <20020508122803.B23913@eecs.tufts.edu> Message-ID: <200205081656.g48Gurb01823@odiug.zope.com> > Know yes. Can you tell us anything about AtheOS? --Guido van Rossum (home page: http://www.python.org/~guido/) From loewis@informatik.hu-berlin.de Wed May 8 17:52:10 2002 From: loewis@informatik.hu-berlin.de (Martin v. =?iso-8859-1?q?L=F6wis?=) Date: 08 May 2002 18:52:10 +0200 Subject: [Python-Dev] AtheOS port In-Reply-To: <200205081544.g48FidD01393@odiug.zope.com> References: <200205081341.g48Dfpgs006314@paros.informatik.hu-berlin.de> <200205081544.g48FidD01393@odiug.zope.com> Message-ID: Guido van Rossum writes: > > IMO, the AtheOS port (http://python.org/sf/488073) is ready for > > integration. Would anybody object if I commit this change? > > I'd like to state it differently. What's the compelling reason for > this? Does anybody even know what AtheOS is? >From the Web page (www.atheos.cx, which appears to be down at the moment - ask Google for a cached copy): # AtheOS is a free desktop operating system under the GPL # license. AtheOS currently run on Intel, AMD and other compatible # processors and support the Intel Multi Processor architecture. I # have seen quite a few anouncements of "promising" OSes with "great # potential" during the development of AtheOS. The problem is that # when I follow the links I normally find a description of the # concept, a floppy-bootloader written in assembly, and not much # else. AtheOS is a bit more mature, and is already running quite a # lot of software. This server for example is running AtheOS. The HTTP # server is a AtheOS port of Apache, and most of the content is # generated by the AtheOS port of PHP3 and perl.The native AtheOS file # system is 64-bit and journaled. The rationale for the patch as given by its author is that AtheOS apparently comes with a Python 1.5.2 binary, and that he wanted to see whether he could update it to the current Python. I'm copying Octavian Cerna to see whether he can come up with a good reason why his patch should be included into Python - I was suggesting to include because I saw no reason to reject it. Regards, Martin From aahz@pythoncraft.com Wed May 8 18:01:00 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 8 May 2002 13:01:00 -0400 Subject: [Python-Dev] Patch #552433 In-Reply-To: <200205081401.g48E1JY28709@pcp742651pcs.reston01.va.comcast.net> References: <200205081318.g48DIrY28337@pcp742651pcs.reston01.va.comcast.net> <20020508134933.GA24723@panix.com> <200205081401.g48E1JY28709@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020508170100.GA581@panix.com> On Wed, May 08, 2002, Guido van Rossum wrote: > > > Define "long enough"; I'm expecting that my BCD module will generate > > tuples at least a hundred elements. > > Hm. Creating such a long tuple in Python code will be the bottleneck > in your program, not iterating over it. Have you benchmarked this > particular fix? I find it hard to believe that one can even notice > the difference. You're probably right, though for long-lived Decimal() instances, I'd expect many rounds of iteration over the tuple. I was just questioning what you meant by "long enough". (No benchmark, I'm still not set up for CVS yet.) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ See me at OSCON! I'm teaching Python for [Perl] Programmers, a fast intro for all experienced programmers (not just Perl). Early bird ends June 10. http://conferences.oreillynet.com/os2002/ From aahz@pythoncraft.com Wed May 8 18:40:55 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 8 May 2002 13:40:55 -0400 Subject: [Python-Dev] Re: Python insecure? In-Reply-To: <200205081521.g48FL0101329@odiug.zope.com> References: <200205081521.g48FL0101329@odiug.zope.com> Message-ID: <20020508174054.GA11317@panix.com> On Wed, May 08, 2002, Guido van Rossum wrote: > > http://www.ornl.gov/its/archives/mailing-lists/qmail/2002/05/msg00309.html > http://www.ornl.gov/its/archives/mailing-lists/qmail/2002/05/msg00415.html Oh, I didn't realize this the first time -- it's a *qmail* list. That simply means we running across the DJB fanatics. Ignore it. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ See me at OSCON! I'm teaching Python for [Perl] Programmers, a fast intro for all experienced programmers (not just Perl). Early bird ends June 10. http://conferences.oreillynet.com/os2002/ From jeremy@zope.com Wed May 8 18:48:56 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Wed, 8 May 2002 13:48:56 -0400 Subject: [Python-Dev] Re: Python insecure? In-Reply-To: <20020508174054.GA11317@panix.com> References: <200205081521.g48FL0101329@odiug.zope.com> <20020508174054.GA11317@panix.com> Message-ID: <15577.25864.849305.645324@slothrop.zope.com> What's a "DJB fanatic"? It seems possible to expand that as "_D_r. _J_. R. _B_ob Dobbs" but that seems a bit unlikely? but-maybe-python-is-pink-ly y'rs, Jeremy From aahz@pythoncraft.com Wed May 8 18:38:40 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 8 May 2002 13:38:40 -0400 Subject: [Python-Dev] Re: Python insecure? In-Reply-To: <15577.18751.32382.473985@slothrop.zope.com> References: <200205081521.g48FL0101329@odiug.zope.com> <15577.18751.32382.473985@slothrop.zope.com> Message-ID: <20020508173840.GA9194@panix.com> On Wed, May 08, 2002, Jeremy Hylton wrote: > > It sure looks like FUD. Anyone know this Michael Sierchio guy? I've > never heard of him, and Google / citeseer searches don't reveal much. > I suppose the paper he mentioned was written by someone else. I know him from ba.food, and a look in Gooja shows that his posting history has been mostly on food, BSD, and sci.crypt. He hasn't been a particularly unpleasant person, if a bit of a snob. Dunno what his hobby horse is here. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ See me at OSCON! I'm teaching Python for [Perl] Programmers, a fast intro for all experienced programmers (not just Perl). Early bird ends June 10. http://conferences.oreillynet.com/os2002/ From aahz@pythoncraft.com Wed May 8 19:03:11 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 8 May 2002 14:03:11 -0400 Subject: [Python-Dev] Re: Python insecure? In-Reply-To: <15577.25864.849305.645324@slothrop.zope.com> References: <200205081521.g48FL0101329@odiug.zope.com> <20020508174054.GA11317@panix.com> <15577.25864.849305.645324@slothrop.zope.com> Message-ID: <20020508180311.GA14934@panix.com> On Wed, May 08, 2002, Jeremy Hylton wrote: > > What's a "DJB fanatic"? Daniel J. Bernstein, author of qmail. He tends to, er, incite rather strong feelings in the Open Source community. For a somewhat over-the-top anti-DJB rant, see http://www.linuxmafia.com/~rick/faq/#djb (Rick -- and his wife -- are rather strong Python bigots, so I'm not surprised that the DJB forces are anti-Python. ) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ See me at OSCON! I'm teaching Python for [Perl] Programmers, a fast intro for all experienced programmers (not just Perl). Early bird ends June 10. http://conferences.oreillynet.com/os2002/ From python@discworld.dyndns.org Wed May 8 19:19:28 2002 From: python@discworld.dyndns.org (Charles Cazabon) Date: Wed, 8 May 2002 12:19:28 -0600 Subject: [Python-Dev] Re: Python insecure? In-Reply-To: <15577.25864.849305.645324@slothrop.zope.com>; from jeremy@zope.com on Wed, May 08, 2002 at 01:48:56PM -0400 References: <200205081521.g48FL0101329@odiug.zope.com> <20020508174054.GA11317@panix.com> <15577.25864.849305.645324@slothrop.zope.com> Message-ID: <20020508121928.B17767@twoflower.internal.do> Jeremy Hylton wrote: > What's a "DJB fanatic"? > > It seems possible to expand that as "_D_r. _J_. R. _B_ob Dobbs" but > that seems a bit unlikely? D. J. Bernstein, professor of mathematics at the University of Illinois at Chicago. He's most famous for being the person who took the U.S. government to court for forbidding the export of "strong" cryptographic information. His second claim to fame is writing extremely reliable, secure software, like qmail and djbdns. As for "fanatic", that's just slander. Some idiot posted a troll to the qmail list claiming Python was unreliable and insecure. A couple of people corrected him. Someone then posted his message to this list. Charles -- ----------------------------------------------------------------------- Charles Cazabon GPL'ed software available at: http://www.qcc.ca/~charlesc/software/ ----------------------------------------------------------------------- From python@discworld.dyndns.org Wed May 8 19:21:03 2002 From: python@discworld.dyndns.org (Charles Cazabon) Date: Wed, 8 May 2002 12:21:03 -0600 Subject: [Python-Dev] Re: Python insecure? In-Reply-To: <20020508180311.GA14934@panix.com>; from aahz@pythoncraft.com on Wed, May 08, 2002 at 02:03:11PM -0400 References: <200205081521.g48FL0101329@odiug.zope.com> <20020508174054.GA11317@panix.com> <15577.25864.849305.645324@slothrop.zope.com> <20020508180311.GA14934@panix.com> Message-ID: <20020508122103.C17767@twoflower.internal.do> Aahz wrote: > > (Rick -- and his wife -- are rather strong Python bigots, so I'm not > surprised that the DJB forces are anti-Python. ) The djb "forces" are not anti-Python. I'm the author of several tools that work well with qmail and are written in Python, and TMDA (a great spam-prevention tool) originally required qmail to work, and was written in Python. Charles -- ----------------------------------------------------------------------- Charles Cazabon GPL'ed software available at: http://www.qcc.ca/~charlesc/software/ ----------------------------------------------------------------------- From mgilfix@eecs.tufts.edu Wed May 8 19:20:55 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Wed, 8 May 2002 14:20:55 -0400 Subject: [Python-Dev] AtheOS port In-Reply-To: ; from loewis@informatik.hu-berlin.de on Wed, May 08, 2002 at 06:52:10PM +0200 References: <200205081341.g48Dfpgs006314@paros.informatik.hu-berlin.de> <200205081544.g48FidD01393@odiug.zope.com> Message-ID: <20020508142055.A27673@eecs.tufts.edu> This is the best description I can manage myself. Haven't had a chance to try out the OS. It does seem much more mature though and I have looked through the code base briefly when I was interested to see how another OS implemented certain functionality. It's definitely very runnable though... On Wed, May 08 @ 18:52, Martin v. Löwis wrote: > Guido van Rossum writes: > > > > IMO, the AtheOS port (http://python.org/sf/488073) is ready for > > > integration. Would anybody object if I commit this change? > > > > I'd like to state it differently. What's the compelling reason for > > this? Does anybody even know what AtheOS is? > > >From the Web page (www.atheos.cx, which appears to be down at the > moment - ask Google for a cached copy): > > # AtheOS is a free desktop operating system under the GPL > # license. AtheOS currently run on Intel, AMD and other compatible > # processors and support the Intel Multi Processor architecture. I > # have seen quite a few anouncements of "promising" OSes with "great > # potential" during the development of AtheOS. The problem is that > # when I follow the links I normally find a description of the > # concept, a floppy-bootloader written in assembly, and not much > # else. AtheOS is a bit more mature, and is already running quite a > # lot of software. This server for example is running AtheOS. The HTTP > # server is a AtheOS port of Apache, and most of the content is > # generated by the AtheOS port of PHP3 and perl.The native AtheOS file > # system is 64-bit and journaled. > > The rationale for the patch as given by its author is that AtheOS > apparently comes with a Python 1.5.2 binary, and that he wanted to see > whether he could update it to the current Python. > > I'm copying Octavian Cerna to see whether he can come up with a good > reason why his patch should be included into Python - I was suggesting > to include because I saw no reason to reject it. > > Regards, > Martin > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev `-> (loewis) -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From aahz@pythoncraft.com Wed May 8 19:23:58 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 8 May 2002 14:23:58 -0400 Subject: [Python-Dev] Re: Python insecure? In-Reply-To: <20020508122103.C17767@twoflower.internal.do> References: <200205081521.g48FL0101329@odiug.zope.com> <20020508174054.GA11317@panix.com> <15577.25864.849305.645324@slothrop.zope.com> <20020508180311.GA14934@panix.com> <20020508122103.C17767@twoflower.internal.do> Message-ID: <20020508182358.GA20821@panix.com> On Wed, May 08, 2002, Charles Cazabon wrote: > Aahz wrote: >> >> (Rick -- and his wife -- are rather strong Python bigots, so I'm not >> surprised that the DJB forces are anti-Python. ) > > The djb "forces" are not anti-Python. I'm the author of several tools > that work well with qmail and are written in Python, and TMDA (a great > spam-prevention tool) originally required qmail to work, and was > written in Python. Ah. Okay, do *you* know anything about Sierchio? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ See me at OSCON! I'm teaching Python for [Perl] Programmers, a fast intro for all experienced programmers (not just Perl). Early bird ends June 10. http://conferences.oreillynet.com/os2002/ From aahz@pythoncraft.com Wed May 8 19:32:01 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 8 May 2002 14:32:01 -0400 Subject: [Python-Dev] Re: Python insecure? In-Reply-To: <20020508121928.B17767@twoflower.internal.do> References: <200205081521.g48FL0101329@odiug.zope.com> <20020508174054.GA11317@panix.com> <15577.25864.849305.645324@slothrop.zope.com> <20020508121928.B17767@twoflower.internal.do> Message-ID: <20020508183201.GA21375@panix.com> On Wed, May 08, 2002, Charles Cazabon wrote: > > As for "fanatic", that's just slander. Some idiot posted a troll to > the qmail list claiming Python was unreliable and insecure. A couple > of people corrected him. Someone then posted his message to this > list. "Slander" is as much hyperbole as my original use of "fanatic". My impression is that DJB generates more emotion and choler than RMS, particularly relative to his contributions. I don't really care, though; I'm not involved in those debates. I do have my own personal reason to dislike DJB that colors my responses to such debates, but I mostly consider myself a neutral party. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ See me at OSCON! I'm teaching Python for [Perl] Programmers, a fast intro for all experienced programmers (not just Perl). Early bird ends June 10. http://conferences.oreillynet.com/os2002/ From python@discworld.dyndns.org Wed May 8 19:33:29 2002 From: python@discworld.dyndns.org (Charles Cazabon) Date: Wed, 8 May 2002 12:33:29 -0600 Subject: [Python-Dev] Re: Python insecure? In-Reply-To: <20020508182358.GA20821@panix.com>; from aahz@pythoncraft.com on Wed, May 08, 2002 at 02:23:58PM -0400 References: <200205081521.g48FL0101329@odiug.zope.com> <20020508174054.GA11317@panix.com> <15577.25864.849305.645324@slothrop.zope.com> <20020508180311.GA14934@panix.com> <20020508122103.C17767@twoflower.internal.do> <20020508182358.GA20821@panix.com> Message-ID: <20020508123329.A18004@twoflower.internal.do> Aahz wrote: > > > > The djb "forces" are not anti-Python. I'm the author of several tools > > that work well with qmail and are written in Python, and TMDA (a great > > spam-prevention tool) originally required qmail to work, and was > > written in Python. > > Ah. Okay, do *you* know anything about Sierchio? Only from his postings to the qmail list. He posts trivial FAQs, ignores the answers, suggests "solutions" to problems that have been debunked in the list archives dozens of times, and then whines that no ones listens to him. Basically, he's a troll. Killfile him if you care about your time. Charles -- ----------------------------------------------------------------------- Charles Cazabon GPL'ed software available at: http://www.qcc.ca/~charlesc/software/ ----------------------------------------------------------------------- From tavyc@users.sourceforge.net Wed May 8 20:24:42 2002 From: tavyc@users.sourceforge.net (Octavian Cerna) Date: Wed, 8 May 2002 22:24:42 +0300 (EEST) Subject: [Python-Dev] AtheOS port In-Reply-To: Message-ID: On 8 May 2002, Martin v. [iso-8859-1] L=F6wis wrote: > From the Web page (www.atheos.cx, which appears to be down at the > moment - ask Google for a cached copy): > > # AtheOS is a free desktop operating system under the GPL > # license. AtheOS currently run on Intel, AMD and other compatible > # processors and support the Intel Multi Processor architecture. I > # have seen quite a few anouncements of "promising" OSes with "great > # potential" during the development of AtheOS. The problem is that > # when I follow the links I normally find a description of the > # concept, a floppy-bootloader written in assembly, and not much > # else. AtheOS is a bit more mature, and is already running quite a > # lot of software. This server for example is running AtheOS. The HTTP > # server is a AtheOS port of Apache, and most of the content is > # generated by the AtheOS port of PHP3 and perl.The native AtheOS file > # system is 64-bit and journaled. > > The rationale for the patch as given by its author is that AtheOS > apparently comes with a Python 1.5.2 binary, and that he wanted to see > whether he could update it to the current Python. > > I'm copying Octavian Cerna to see whether he can come up with a good > reason why his patch should be included into Python - I was suggesting > to include because I saw no reason to reject it. Hi all, One of Python's (and many other programming languages) goal is to be portab= le on many systems. Quoting from http://www.python.org/doc/Summary.html: "The Python implementation is portable: it runs on many brands of UNIX, on Windows, DOS, OS/2, Mac, Amiga... If your favorite system isn't listed here, it may still be supported, if there's a C compiler for it. Ask around on comp.lang.python -- or just try compiling Python yourself." And that's what I did. I tried compiling Python by myself on AtheOS which was an unsupported platform, and it didn't compile. So I just thought it would be nice to have Python working on this system too. AtheOS is a free OS, similar to BeOS, mostly POSIX-compliant, and mature enough to run lots of Unix/GNU software with minimal/no changes. Examples are GCC, Emacs, Perl, Ruby. It's not as popular as other OSes like Linux, *BSD or BeOS. If Python runs on many POSIX-like systems, then why should AtheOS be an exc= eption? Porting Python to AtheOS was almost straightforward, I only had to add nati= ve dynamic loading and threading in order to take full advantage of Python. I had a fully working Python on AtheOS (passing all tests), and I wanted to share the changes, so I submitted the patch to the SourceForge tracker. Thank you Martin for your support. I hope that integrating this port will attract Python developers to AtheOS, and AtheOS developers to Python. It will help increase both Python and AtheOS developer communities. Best Regards, Octavian Cerna Please excuse my bad English. From sdm7g@Virginia.EDU Wed May 8 20:32:11 2002 From: sdm7g@Virginia.EDU (Steven Majewski) Date: Wed, 8 May 2002 15:32:11 -0400 (EDT) Subject: [Python-Dev] MacOSX dlopen equivalent? In-Reply-To: Message-ID: On Wed, 8 May 2002, Jack Jansen wrote: > Another but: if I look at the readme for your test code I get the > impression that the gcc that OSX has (2.92.2) may be a problem. gcc 3 is > going to be supported in the next major release (late summer, according > to apple). gcc 3 is in the latest online (beta) developer tools, but it's 'experimental' -- ProjectBuilder defaults to using the earlier version which is also included. -- Steve From martin@v.loewis.de Wed May 8 20:35:05 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 08 May 2002 21:35:05 +0200 Subject: [Python-Dev] Patch #552433 In-Reply-To: <200205081318.g48DIrY28337@pcp742651pcs.reston01.va.comcast.net> References: <200205081318.g48DIrY28337@pcp742651pcs.reston01.va.comcast.net> Message-ID: Guido van Rossum writes: > I've missed the review of this patch. Why bother special-casing > tuples? Who's ever going to iterate over a tuple long enough that it > makes a difference? Notice that the length of the tuple is not an issue - or rather, the longer the tuple, the less interesting this change. When profiling, I found that Python, with iterators, spends a significant amount of time in allocating IndexError exceptions, so I thought that patch makes sense. There is also speed to be gained from accessing the tuple directly (thus bypassing the function call), but that may not be so relevant. I have now tried to measure code usage of this change, using two metrics: listcount,tuplecount,othercount: Number of times the list, tuple, and "other" case is taken listlast,tuplelast,otherlast: Number of times that the return NULL is taken for each case I made the following runs: print "Hello": 1744 149 70 69 60 13 make sharedmods: 14217 2142 447 641 949 60 regrtest.py: 3184596 152384 1170921 134457 49271 15765 PyXML testsuite: 713585 110278 7192 21306 2357 2964 As you can see, the percentage of cases in which iternext is called for a tuple varies between 3% (regrtest) and 13% (PyXML). However, the percentage of exhausted tuples compared to other exhausted containers varies between 8% (PyXML) and 61% (make sharedmods). So it appears that a significant fraction of the exhausted containers are indeed tuples, justifying the special-casing. Regards, Martin From guido@python.org Wed May 8 20:53:28 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 08 May 2002 15:53:28 -0400 Subject: [Python-Dev] Patch #552433 In-Reply-To: Your message of "08 May 2002 21:35:05 +0200." References: <200205081318.g48DIrY28337@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200205081953.g48JrSM02748@odiug.zope.com> > When profiling, I found that Python, with iterators, spends a > significant amount of time in allocating IndexError exceptions, so I > thought that patch makes sense. There is also speed to be gained from > accessing the tuple directly (thus bypassing the function call), but > that may not be so relevant. Ah, that helps. [...] > So it appears that a significant fraction of the exhausted containers are > indeed tuples, justifying the special-casing. OK, I'm convinced. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Wed May 8 20:55:43 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 08 May 2002 15:55:43 -0400 Subject: [Python-Dev] Re: Python insecure? In-Reply-To: <15577.18751.32382.473985@slothrop.zope.com> Message-ID: [Jeremy Hylton] > It sure looks like FUD. Anyone know this Michael Sierchio guy? I've > never heard of him, and Google / citeseer searches don't reveal much. > I suppose the paper he mentioned was written by someone else. The plain "Look for a forthcoming paper on Python vulnerabilities" is a sure sign that no such paper exists, and is unlikely ever to come from the person making the claim. However, I bet *you* could write a good and scary paper on the topic! I would, except I'm too busy rebooting Win98 . From tim_one@email.msn.com Thu May 9 03:39:49 2002 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 8 May 2002 22:39:49 -0400 Subject: [Python-Dev] Lazily GC tracking tuples In-Reply-To: <20020508153252.344.qmail@web14704.mail.yahoo.com> Message-ID: [Daniel Dunbar] > ... > The other thing to note is that untracking such tuples is a > very small win regardless - if they were not untracked the > tuple visit routine will quickly scan down the tuple (which > I guess is likely to be small to start with), and the GC > visitor will quickly reject adding any of the contained > objects because they are not PyObject_IS_GC(). I don't know that it's small. It's a function call per tuple scan, and apps slinging millions of tuples aren't rare: just touching the memory to scan all of them is expensive. > That suggests that the only case it would make a difference > is a) when there are soooo many untrackable tuples that > simply traversing them takes a large amount of time (ie. > the iterzip case), or b) when there is a massive nesting > of untrackable tuples - an large AST or binary-tree might > benefit from this, but there are perhaps better solutions > to that problem (see below) Mabye. > ... > I still don't understand how you 'check the tuple' and then > untrack it - even if it has no NULL slots it still isn't safe > to remove. It's very likely safe, and if we do this I can easily guarantee it's 100% safe for the core distribution. It's not possible to create a tuple with NULL slots from Python code, so it's just a matter of checking the core C code that creates tuples. Most of it is utterly and obviously safe. A little bit isn't. For example, there's a very small hole in PySequence_Tuple() then, when the allocated tuple turns out not to be big enough: then invoking the input sequence iterator can trigger gc, and that might erroneously untrack the tuple being constructed. But that hole is easily plugged, and I expect that all such holes are just as easily plugged. Extension modules aren't supposed to call _PyTuple_Resize() at all (it's part of the private API), and it's very unlikely that anything not calling that routine is vulnerable. It's only called from 5 places in the core, and 2 of them are in PySequence_Tuple. Still, it would be a new restriction, and may fool something *somewhere*. If it does, and they're tuples that turn out to be in a cycle (itself rare, especially for tuples that grow), they may get a leak. BFD . > I don't think I agree that special casing tuples in the > collector is a good idea - this whole issue just came > up because of the iterzip test, where it was noticed > that the collector really didn't have to check all > the objects, which is just a special case solution to > the problem that many of the objects in the gen* > lists are not going to be participating in a cycle. As above, apps slinging millions of tuples aren't rare, and it's not doing anyone any good in cases where they survive an unbounded number of collections. They may be important not because it's pretty clear how to exempt them, but because they may be important <0.9 wink>. > A solution to that general problem would be much more > beneficial, although presumably much more work ;) > > If the collector could somehow prioritize objects, > it seems like various heuristics could be used to > gauge the likelyhood of an object participating in > a cycle vs. the 'good' of collecting that cycle. Neil has since generalized the gc module to make it easy to add more generations. That's a general and effective approach to reducing useless scans. We're never going to have anything like, say, write barriers in Python, and OS-specific and platform-specific tricks are right out. GC in Python lives within many severe constraints. > As an aside: are there some nice established tests > for estimating the benefit of modifications to > the collector? Something that needs the collector > (so getting cycles early keeps the memory consumption > down, improving performance) and also has dynamic > object allocation properties that mimic some ideal > of real-applications? Not that I know of. The cycle gc has performed (IMO), overall, very well since its inception. Improvements are driven more by the rare pathological cases that pop up than by systematic experimentation. Neil seems to use one of his large apps to "try things out on", and that's as close to an ideal real-life app as I expect anyone is going to hand Neil. Glitches tend to show up as timing anomalies (iterzip was typical in that respect). From tim.one@comcast.net Thu May 9 03:47:17 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 08 May 2002 22:47:17 -0400 Subject: [Python-Dev] Lazily GC tracking tuples In-Reply-To: <20020508071335.A23115@glacier.arctrix.com> Message-ID: [Tim] > Do you really mean after a gen1 collection? [Neil] > Yes. By that time the tuple was probably around for a while. Also, the > chances that it will be mutated will be smaller. I won't bother arguing , but then why not at the start of a gen1 collection? Nothing that transpires within a gen1 collection is going to change a given tuple's trackable-vs-untrackable nature, and if untrackable tuples are indeed much more likely, there seems nothing to gain (but something to lose) by making gen1 scan untrackable tuples multiple times. From skip@pobox.com Thu May 9 04:11:22 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 8 May 2002 22:11:22 -0500 Subject: [Python-Dev] bsddb module and unicode - is it worth it? Message-ID: <15577.59610.406503.190279@12-248-41-177.client.attbi.com> I just noticed the bsddb module barfs on Unicode objects. Is it worth updating it to handle them transparently? Or does the "explicit is better than implicit" rubric hold here? Skip From martin@v.loewis.de Thu May 9 08:48:12 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 09 May 2002 09:48:12 +0200 Subject: [Python-Dev] bsddb module and unicode - is it worth it? In-Reply-To: <15577.59610.406503.190279@12-248-41-177.client.attbi.com> References: <15577.59610.406503.190279@12-248-41-177.client.attbi.com> Message-ID: Skip Montanaro writes: > I just noticed the bsddb module barfs on Unicode objects. Is it worth > updating it to handle them transparently? Or does the "explicit is better > than implicit" rubric hold here? bsddb implements a byte-string keys, byte-string values mapping; see http://www.sleepycat.com/docs/api_cxx/dbt_class.html If you want anything else (such as text), you'll need to use shelve, or define your own mapping from bytes to your data types. Regards, Martin From jacobs@penguin.theopalgroup.com Thu May 9 12:02:19 2002 From: jacobs@penguin.theopalgroup.com (Kevin Jacobs) Date: Thu, 9 May 2002 07:02:19 -0400 (EDT) Subject: [Python-Dev] Lazily GC tracking tuples In-Reply-To: Message-ID: On Wed, 8 May 2002, Tim Peters wrote: > [...] apps slinging millions of tuples aren't rare, and it's not doing > anyone any good in cases where they survive an unbounded number of > collections. They may be important not because it's pretty clear how to > exempt them, but because they may be important <0.9 wink>. Well, I've found one case where the garbage collector _does_ cause serious performance problems, and coincidentally enough it is due to the creation of zillions of cycle-less tuples. Our current solution has been to disable GC in some parts of our code, and then manually trigger collection at the correct points. When I can find some free time, I will definitly test drive any available tuple untracking patches to see if they make any substantial difference in our code. Regards, -Kevin -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com From jason@tishler.net Thu May 9 13:01:55 2002 From: jason@tishler.net (Jason Tishler) Date: Thu, 09 May 2002 08:01:55 -0400 Subject: [Python-Dev] Python 2.3 patch cut-off date In-Reply-To: <200205081442.g48Eg1a01105@odiug.zope.com> References: <20020508140627.GH1700@tishler.net> <200205081442.g48Eg1a01105@odiug.zope.com> Message-ID: <20020509120154.GB1236@tishler.net> Guido, On Wed, May 08, 2002 at 10:42:01AM -0400, Guido van Rossum wrote: > > I'm mainly interested in submitted two more Cygwin Python build related > > patches and I was wondering whether or not I was running out of time. > > The schedule is slipping; we're likely to switch to a more > goal-oriented (as opposed to deadline-oriented) approach. OK, but that makes planning by patch submitters more challenging. > That doesn't mean we're necessarily going to wait for Cywin patches! :-) Understood, I have already submitted the critical ones. The two that I'm considering are along the lines of clean-up or "that would be nice." Jason From guido@python.org Thu May 9 13:43:49 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 09 May 2002 08:43:49 -0400 Subject: [Python-Dev] Python 2.3 patch cut-off date In-Reply-To: Your message of "Thu, 09 May 2002 08:01:55 EDT." <20020509120154.GB1236@tishler.net> References: <20020508140627.GH1700@tishler.net> <200205081442.g48Eg1a01105@odiug.zope.com> <20020509120154.GB1236@tishler.net> Message-ID: <200205091243.g49Chn202210@pcp742651pcs.reston01.va.comcast.net> > > The schedule is slipping; we're likely to switch to a more > > goal-oriented (as opposed to deadline-oriented) approach. > > OK, but that makes planning by patch submitters more challenging. That's life. We're all doing this part-time: even PythonLabs has to do customer work to keep Zope Corp pushing forward. I realize that it's helpful for some developers to have a hard deadline, and as we get closer to folfilling our goals we should set ourselves some, but I'd rather slip a schedule than release shoddy work. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu May 9 14:05:51 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 09 May 2002 09:05:51 -0400 Subject: [Python-Dev] AtheOS port In-Reply-To: Your message of "Wed, 08 May 2002 22:24:42 +0300." References: Message-ID: <200205091305.g49D5pP02421@pcp742651pcs.reston01.va.comcast.net> Well, since we accepted changes for the GNU Hurd, I think AtheOS is okay. --Guido van Rossum (home page: http://www.python.org/~guido/) From jason@tishler.net Thu May 9 14:19:44 2002 From: jason@tishler.net (Jason Tishler) Date: Thu, 09 May 2002 09:19:44 -0400 Subject: [Python-Dev] Python 2.3 patch cut-off date In-Reply-To: <200205091243.g49Chn202210@pcp742651pcs.reston01.va.comcast.net> References: <20020508140627.GH1700@tishler.net> <200205081442.g48Eg1a01105@odiug.zope.com> <20020509120154.GB1236@tishler.net> <200205091243.g49Chn202210@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020509131944.GF1236@tishler.net> Guido, On Thu, May 09, 2002 at 08:43:49AM -0400, Guido van Rossum wrote: > > OK, but that makes planning by patch submitters more challenging. > > [snip] > I'd rather slip a schedule than release shoddy work. Sorry for being unclear, but you misunderstood me. I completely agree with the above. My point was not having a (proposed) deadline makes me feel that I have to drop everything in order a guarantee submitting a patch "on time." You have already indicated that you would not necessarily wait for Cygwin patches. I would rather have clearly stated deadlines that are subject to change (i.e., slippage). In this way, volunteers can better plan their time. Can I play with my daughter now or should I submit a patch? Can I go to sleep now or should I submit a patch? Etc. Jason From mgilfix@eecs.tufts.edu Thu May 9 14:37:19 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Thu, 9 May 2002 09:37:19 -0400 Subject: [Python-Dev] Python 2.3 patch cut-off date In-Reply-To: <200205091243.g49Chn202210@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Thu, May 09, 2002 at 08:43:49AM -0400 References: <20020508140627.GH1700@tishler.net> <200205081442.g48Eg1a01105@odiug.zope.com> <20020509120154.GB1236@tishler.net> <200205091243.g49Chn202210@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020509093719.A22324@eecs.tufts.edu> Sorry if this is redundant or a stupid question - but where can we find out about the patch deadline? Or did I miss a thread? Sadly, I need to finish exams before I can finish mine. -- Mike On Thu, May 09 @ 08:43, Guido van Rossum wrote: > > > The schedule is slipping; we're likely to switch to a more > > > goal-oriented (as opposed to deadline-oriented) approach. > > > > OK, but that makes planning by patch submitters more challenging. > > That's life. We're all doing this part-time: even PythonLabs has to > do customer work to keep Zope Corp pushing forward. I realize that > it's helpful for some developers to have a hard deadline, and as we > get closer to folfilling our goals we should set ourselves some, but > I'd rather slip a schedule than release shoddy work. > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev `-> (guido) -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From martin@v.loewis.de Thu May 9 14:50:33 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 09 May 2002 15:50:33 +0200 Subject: [Python-Dev] Python 2.3 patch cut-off date In-Reply-To: <20020509131944.GF1236@tishler.net> References: <20020508140627.GH1700@tishler.net> <200205081442.g48Eg1a01105@odiug.zope.com> <20020509120154.GB1236@tishler.net> <200205091243.g49Chn202210@pcp742651pcs.reston01.va.comcast.net> <20020509131944.GF1236@tishler.net> Message-ID: Jason Tishler writes: > I would rather have clearly stated deadlines that are subject to change > (i.e., slippage). In this way, volunteers can better plan their time. > Can I play with my daughter now or should I submit a patch? Can I go > to sleep now or should I submit a patch? Etc. In any schedule, there likely would be a month between a1 and a2, so the release of Python 2.3a1 would simultaneously be a last call for great new features. Between a1 and c1, there would be four months, giving plenty of time (IMO) for bug fixes. Anything that takes more time then four months is complex enough to wait for 2.4 to mature. There will be always a next release. Regards, Martin From aahz@pythoncraft.com Thu May 9 15:10:25 2002 From: aahz@pythoncraft.com (Aahz) Date: Thu, 9 May 2002 10:10:25 -0400 Subject: [Python-Dev] Python 2.3 patch cut-off date In-Reply-To: <20020509093719.A22324@eecs.tufts.edu> References: <20020508140627.GH1700@tishler.net> <200205081442.g48Eg1a01105@odiug.zope.com> <20020509120154.GB1236@tishler.net> <200205091243.g49Chn202210@pcp742651pcs.reston01.va.comcast.net> <20020509093719.A22324@eecs.tufts.edu> Message-ID: <20020509141025.GA24412@panix.com> On Thu, May 09, 2002, Michael Gilfix wrote: > > Sorry if this is redundant or a stupid question - but where can we > find out about the patch deadline? Or did I miss a thread? Sadly, I > need to finish exams before I can finish mine. The schedule is always maintained as a PEP; the schedule for 2.3 is PEP 283. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ See me at OSCON! I'm teaching Python for [Perl] Programmers, a fast intro for all experienced programmers (not just Perl). Early bird ends June 10. http://conferences.oreillynet.com/os2002/ From nas@python.ca Thu May 9 15:17:52 2002 From: nas@python.ca (Neil Schemenauer) Date: Thu, 9 May 2002 07:17:52 -0700 Subject: [Python-Dev] Lazily GC tracking tuples In-Reply-To: ; from tim.one@comcast.net on Wed, May 08, 2002 at 10:47:17PM -0400 References: <20020508071335.A23115@glacier.arctrix.com> Message-ID: <20020509071751.A27177@glacier.arctrix.com> Tim Peters wrote: > I won't bother arguing , but then why not at the start of a gen1 > collection? Nothing that transpires within a gen1 collection is going to > change a given tuple's trackable-vs-untrackable nature You're right. two-and-a-half-beautiful-words-ly y'rs Neil From martin@v.loewis.de Thu May 9 14:51:05 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 09 May 2002 15:51:05 +0200 Subject: [Python-Dev] Python 2.3 patch cut-off date In-Reply-To: <20020509093719.A22324@eecs.tufts.edu> References: <20020508140627.GH1700@tishler.net> <200205081442.g48Eg1a01105@odiug.zope.com> <20020509120154.GB1236@tishler.net> <200205091243.g49Chn202210@pcp742651pcs.reston01.va.comcast.net> <20020509093719.A22324@eecs.tufts.edu> Message-ID: Michael Gilfix writes: > Sorry if this is redundant or a stupid question - but where can we > find out about the patch deadline? Or did I miss a thread? Sadly, I > need to finish exams before I can finish mine. For Python 2.3, it's in PEP 283. Regards, Martin From fredrik@pythonware.com Thu May 9 17:43:01 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Thu, 9 May 2002 18:43:01 +0200 Subject: [Python-Dev] Python 2.3 patch cut-off date References: <20020508140627.GH1700@tishler.net> <200205081442.g48Eg1a01105@odiug.zope.com> <20020509120154.GB1236@tishler.net> <200205091243.g49Chn202210@pcp742651pcs.reston01.va.comcast.net> <20020509131944.GF1236@tishler.net> Message-ID: <007701c1f778$96e2b2a0$ced241d5@hagrid> jason wrote: > Can I play with my daughter now or should I submit a patch? the answer to that should always be "if she wants to play, go play with her" From tim.one@comcast.net Thu May 9 20:10:34 2002 From: tim.one@comcast.net (Tim Peters) Date: Thu, 09 May 2002 15:10:34 -0400 Subject: [Python-Dev] Python 2.3 patch cut-off date In-Reply-To: <007701c1f778$96e2b2a0$ced241d5@hagrid> Message-ID: [Jason Tishler] > Can I play with my daughter now or should I submit a patch? [Fredrik Lundh] > the answer to that should always be "if she wants to > play, go play with her" Fiddlesticks. If she *really* wants to play, she can damn well write a Playtime Enhancement Proposal first, get community input, summarize the arguments pro and con, present them to Guido, and then he'll decide whether playtime is appropriate, and if so when. Be sure she includes full documentation for the intended modes of play, a test suite verifying compliance to the Playtime Plan, and-- if at all possible --checking for potentially unfortunate interactions with cyclic gc and the trashcan mechanism. deeply-nested-tuples-are-playtime-for-all-ly y'rs - tim From barry@zope.com Thu May 9 20:23:23 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 9 May 2002 15:23:23 -0400 Subject: [Python-Dev] Python 2.3 patch cut-off date References: <007701c1f778$96e2b2a0$ced241d5@hagrid> Message-ID: <15578.52395.451800.578483@anthem.wooz.org> >>>>> "TP" == Tim Peters writes: TP> Fiddlesticks. If she *really* wants to play, she can damn TP> well write a Playtime Enhancement Proposal first, get TP> community input, summarize the arguments pro and con, present TP> them to Guido, and then he'll decide whether playtime is TP> appropriate, and if so when. Be sure she includes full TP> documentation for the intended modes of play, a test suite TP> verifying compliance to the Playtime Plan, and-- if at all TP> possible --checking for potentially unfortunate interactions TP> with cyclic gc and the trashcan mechanism. Just be sure she doesn't dump core. -Barry From greg@cosc.canterbury.ac.nz Fri May 10 02:32:04 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Fri, 10 May 2002 13:32:04 +1200 (NZST) Subject: [Python-Dev] Python 2.3 patch cut-off date In-Reply-To: <007701c1f778$96e2b2a0$ced241d5@hagrid> Message-ID: <200205100132.NAA03991@s454.cosc.canterbury.ac.nz> Fredrik Lundh : > jason wrote: > > > Can I play with my daughter now or should I submit a patch? > > the answer to that should always be "if she wants to > play, go play with her" If you teach her to play the exciting game of "Help Daddy Submit a Python Patch", you can have the best of both. :-) Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From jason@tishler.net Fri May 10 13:33:05 2002 From: jason@tishler.net (Jason Tishler) Date: Fri, 10 May 2002 08:33:05 -0400 Subject: [Python-Dev] Python 2.3 patch cut-off date In-Reply-To: <007701c1f778$96e2b2a0$ced241d5@hagrid> References: <20020508140627.GH1700@tishler.net> <200205081442.g48Eg1a01105@odiug.zope.com> <20020509120154.GB1236@tishler.net> <200205091243.g49Chn202210@pcp742651pcs.reston01.va.comcast.net> <20020509131944.GF1236@tishler.net> <007701c1f778$96e2b2a0$ced241d5@hagrid> Message-ID: <20020510123305.GD2124@tishler.net> Fredrik, On Thu, May 09, 2002 at 06:43:01PM +0200, Fredrik Lundh wrote: > jason wrote: > > > Can I play with my daughter now or should I submit a patch? > > the answer to that should always be "if she wants to > play, go play with her" Of course you are correct. I guess that it will be at least a few more years before I can submit patches again... :,) Jason From jason@tishler.net Fri May 10 13:38:07 2002 From: jason@tishler.net (Jason Tishler) Date: Fri, 10 May 2002 08:38:07 -0400 Subject: [Python-Dev] Python 2.3 patch cut-off date In-Reply-To: References: <20020508140627.GH1700@tishler.net> <200205081442.g48Eg1a01105@odiug.zope.com> <20020509120154.GB1236@tishler.net> <200205091243.g49Chn202210@pcp742651pcs.reston01.va.comcast.net> <20020509131944.GF1236@tishler.net> Message-ID: <20020510123807.GE2124@tishler.net> Martin, On Thu, May 09, 2002 at 03:50:33PM +0200, Martin v. Loewis wrote: > Jason Tishler writes: > > > I would rather have clearly stated deadlines that are subject to change > > (i.e., slippage). In this way, volunteers can better plan their time. > > Can I play with my daughter now or should I submit a patch? Can I go > > to sleep now or should I submit a patch? Etc. > > In any schedule, there likely would be a month between a1 and a2, so > the release of Python 2.3a1 would simultaneously be a last call for > great new features. Between a1 and c1, there would be four months, > giving plenty of time (IMO) for bug fixes. If I interpret the above correctly, then for my typical patches it seems like I would have to at least b1 (and possibly c1) to submit them for consideration. Is my interpretation correct? Thanks, Jason From guido@python.org Fri May 10 14:43:42 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 10 May 2002 09:43:42 -0400 Subject: [Python-Dev] Python 2.3 patch cut-off date In-Reply-To: Your message of "Fri, 10 May 2002 08:33:05 EDT." <20020510123305.GD2124@tishler.net> References: <20020508140627.GH1700@tishler.net> <200205081442.g48Eg1a01105@odiug.zope.com> <20020509120154.GB1236@tishler.net> <200205091243.g49Chn202210@pcp742651pcs.reston01.va.comcast.net> <20020509131944.GF1236@tishler.net> <007701c1f778$96e2b2a0$ced241d5@hagrid> <20020510123305.GD2124@tishler.net> Message-ID: <200205101343.g4ADhht04661@pcp742651pcs.reston01.va.comcast.net> > > the answer to that should always be "if she wants to > > play, go play with her" > > Of course you are correct. I guess that it will be at least a few more > years before I can submit patches again... :,) Does that mean you're declining the offer of checkin privileges? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri May 10 14:45:06 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 10 May 2002 09:45:06 -0400 Subject: [Python-Dev] Python 2.3 patch cut-off date In-Reply-To: Your message of "Fri, 10 May 2002 08:38:07 EDT." <20020510123807.GE2124@tishler.net> References: <20020508140627.GH1700@tishler.net> <200205081442.g48Eg1a01105@odiug.zope.com> <20020509120154.GB1236@tishler.net> <200205091243.g49Chn202210@pcp742651pcs.reston01.va.comcast.net> <20020509131944.GF1236@tishler.net> <20020510123807.GE2124@tishler.net> Message-ID: <200205101345.g4ADj6D04683@pcp742651pcs.reston01.va.comcast.net> > > In any schedule, there likely would be a month between a1 and a2, so > > the release of Python 2.3a1 would simultaneously be a last call for > > great new features. Between a1 and c1, there would be four months, > > giving plenty of time (IMO) for bug fixes. > > If I interpret the above correctly, then for my typical patches it seems > like I would have to at least b1 (and possibly c1) to submit them for > consideration. Is my interpretation correct? I don't know what your "typical patches" are. Why do you expect they need to be submitted after b1? After c1 would suggest they are too late! --Guido van Rossum (home page: http://www.python.org/~guido/) From jason@tishler.net Fri May 10 15:10:48 2002 From: jason@tishler.net (Jason Tishler) Date: Fri, 10 May 2002 10:10:48 -0400 Subject: [Python-Dev] Python 2.3 patch cut-off date In-Reply-To: <200205101343.g4ADhht04661@pcp742651pcs.reston01.va.comcast.net> References: <20020508140627.GH1700@tishler.net> <200205081442.g48Eg1a01105@odiug.zope.com> <20020509120154.GB1236@tishler.net> <200205091243.g49Chn202210@pcp742651pcs.reston01.va.comcast.net> <20020509131944.GF1236@tishler.net> <007701c1f778$96e2b2a0$ced241d5@hagrid> <20020510123305.GD2124@tishler.net> <200205101343.g4ADhht04661@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020510141048.GG2124@tishler.net> Guido, On Fri, May 10, 2002 at 09:43:42AM -0400, Guido van Rossum wrote: > > > the answer to that should always be "if she wants to > > > play, go play with her" > > > > Of course you are correct. I guess that it will be at least a few more > > years before I can submit patches again... :,) ^^^ *** I should have quit when I was behind. I was trying to be funny, but it appears that I was unsuccessful. > Does that mean you're declining the offer of checkin privileges? No. Although I saw mention of this in the patch collector, I wasn't sure what was the final decision. Jason From guido@python.org Fri May 10 15:17:11 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 10 May 2002 10:17:11 -0400 Subject: [Python-Dev] Python 2.3 patch cut-off date In-Reply-To: Your message of "Fri, 10 May 2002 10:10:48 EDT." <20020510141048.GG2124@tishler.net> References: <20020508140627.GH1700@tishler.net> <200205081442.g48Eg1a01105@odiug.zope.com> <20020509120154.GB1236@tishler.net> <200205091243.g49Chn202210@pcp742651pcs.reston01.va.comcast.net> <20020509131944.GF1236@tishler.net> <007701c1f778$96e2b2a0$ced241d5@hagrid> <20020510123305.GD2124@tishler.net> <200205101343.g4ADhht04661@pcp742651pcs.reston01.va.comcast.net> <20020510141048.GG2124@tishler.net> Message-ID: <200205101417.g4AEHCh04990@pcp742651pcs.reston01.va.comcast.net> > Guido, > > On Fri, May 10, 2002 at 09:43:42AM -0400, Guido van Rossum wrote: > > > > the answer to that should always be "if she wants to > > > > play, go play with her" > > > > > > Of course you are correct. I guess that it will be at least a few more > > > years before I can submit patches again... :,) > ^^^ > *** > > I should have quit when I was behind. I was trying to be funny, but it > appears that I was unsuccessful. Aargh! I was trying to be funny too, but my :) key was stuck. :) > > Does that mean you're declining the offer of checkin privileges? > > No. Although I saw mention of this in the patch collector, I wasn't > sure what was the final decision. You're in. If you're in a hurry, mail Barry. --Guido van Rossum (home page: http://www.python.org/~guido/) From jason@tishler.net Fri May 10 15:49:38 2002 From: jason@tishler.net (Jason Tishler) Date: Fri, 10 May 2002 10:49:38 -0400 Subject: [Python-Dev] Python 2.3 patch cut-off date In-Reply-To: <200205101345.g4ADj6D04683@pcp742651pcs.reston01.va.comcast.net> References: <20020508140627.GH1700@tishler.net> <200205081442.g48Eg1a01105@odiug.zope.com> <20020509120154.GB1236@tishler.net> <200205091243.g49Chn202210@pcp742651pcs.reston01.va.comcast.net> <20020509131944.GF1236@tishler.net> <20020510123807.GE2124@tishler.net> <200205101345.g4ADj6D04683@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020510144938.GH2124@tishler.net> Guido, On Fri, May 10, 2002 at 09:45:06AM -0400, Guido van Rossum wrote: > > > In any schedule, there likely would be a month between a1 and a2, so > > > the release of Python 2.3a1 would simultaneously be a last call for > > > great new features. Between a1 and c1, there would be four months, > > > giving plenty of time (IMO) for bug fixes. > > > > If I interpret the above correctly, then for my typical patches it seems > > like I would have to at least b1 (and possibly c1) to submit them for > > consideration. Is my interpretation correct? > > I don't know what your "typical patches" are. Sorry for being unclear. My patches are usually Cygwin related but sometimes more generally build related (such as my readline 4.2 patch). > Why do you expect they need to be submitted after b1? I don't. I'm just trying to determine whether I have until a1, a2, or b1. I have a *big* Cygwin rebase patch to get into Cygwin's setup.exe that is sorely needed for Python, Apache, and other apps that load many DLLs and fork(). So, I'm just trying to ascertain priorities. > After c1 would suggest they are too late! Understood. Jason From jason@tishler.net Fri May 10 15:53:09 2002 From: jason@tishler.net (Jason Tishler) Date: Fri, 10 May 2002 10:53:09 -0400 Subject: [Python-Dev] Python 2.3 patch cut-off date In-Reply-To: <200205101417.g4AEHCh04990@pcp742651pcs.reston01.va.comcast.net> References: <20020508140627.GH1700@tishler.net> <200205081442.g48Eg1a01105@odiug.zope.com> <20020509120154.GB1236@tishler.net> <200205091243.g49Chn202210@pcp742651pcs.reston01.va.comcast.net> <20020509131944.GF1236@tishler.net> <007701c1f778$96e2b2a0$ced241d5@hagrid> <20020510123305.GD2124@tishler.net> <200205101343.g4ADhht04661@pcp742651pcs.reston01.va.comcast.net> <20020510141048.GG2124@tishler.net> <200205101417.g4AEHCh04990@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020510145309.GI2124@tishler.net> Guido, On Fri, May 10, 2002 at 10:17:11AM -0400, Guido van Rossum wrote: > > I should have quit when I was behind. I was trying to be funny, but it > > appears that I was unsuccessful. > > Aargh! I was trying to be funny too, but my :) key was stuck. :) Phew! > > > Does that mean you're declining the offer of checkin privileges? > > > > No. Although I saw mention of this in the patch collector, I wasn't > > sure what was the final decision. > > You're in. Thanks! > If you're in a hurry, mail Barry. Am I? See my previous post. :,) Jason From martin@v.loewis.de Fri May 10 17:14:25 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 10 May 2002 18:14:25 +0200 Subject: [Python-Dev] Python 2.3 patch cut-off date In-Reply-To: <20020510123807.GE2124@tishler.net> References: <20020508140627.GH1700@tishler.net> <200205081442.g48Eg1a01105@odiug.zope.com> <20020509120154.GB1236@tishler.net> <200205091243.g49Chn202210@pcp742651pcs.reston01.va.comcast.net> <20020509131944.GF1236@tishler.net> <20020510123807.GE2124@tishler.net> Message-ID: Jason Tishler writes: > If I interpret the above correctly, then for my typical patches it seems > like I would have to at least b1 (and possibly c1) to submit them for > consideration. Is my interpretation correct? If they are bug fixes, they ought to be submitted by b1, yes. If they are really critical (like: there is no system where ceval.c can compile), they might be accepted after c1. What constitutes a bug fix is sometimes difficult to tell: If it fixes a regression over a previous release, it certainly is a bug fix. If it makes a feature available on a platform which previously did not have that feature, it might be a new feature - even if other platforms previously offered that feature. The rule is (or, IMO, should be): no new features after b1 is released. So if, by above definition, you are planning new features for your port, you should submit them by a2, or atleast sufficiently before b1. HTH, Martin From python@rcn.com Fri May 10 19:23:09 2002 From: python@rcn.com (Raymond Hettinger) Date: Fri, 10 May 2002 14:23:09 -0400 Subject: [Python-Dev] Oberver Pattern Message-ID: <009e01c1f84f$bd75bfc0$89b53bd0@othello> I would like to know what you all think about formalizing an observer protocol for python objects. A possible implementation would add Py_TPFLAGS_OBSERVABLE and a slot, tp_observer, with a function, PyObject_GenericAttach which registers a callable to be notified when the object updates. Another approach would be to keep a single registry list with access functions: attach(subject,observer), detach(subject,observer), and notifyall(subject). PyObject_HEAD would have a field, int being_observed=0. The attach() function sets being_observed. Any code affecting object state is obliged to call PyObject_NOTIFY, a macro that checks being_observed and, if set, calls notifyall(subject). Alex's smart shelves provide an immediate application -- it needs to know which of its cache objects have changed state. Another application would be to enable computation caching in my matrix package which needs to invalidate the cache whenever a matrix element gets updated. Raymond Hettinger From pyth@devel.trillke.net Fri May 10 19:39:48 2002 From: pyth@devel.trillke.net (holger krekel) Date: Fri, 10 May 2002 20:39:48 +0200 Subject: [Python-Dev] Oberver Pattern In-Reply-To: <009e01c1f84f$bd75bfc0$89b53bd0@othello>; from python@rcn.com on Fri, May 10, 2002 at 02:23:09PM -0400 References: <009e01c1f84f$bd75bfc0$89b53bd0@othello> Message-ID: <20020510203948.G25906@prim.han.de> Raymond Hettinger wrote: > I would like to know what you all think about formalizing an observer > protocol for python objects. > > A possible implementation would add Py_TPFLAGS_OBSERVABLE and a slot, > tp_observer, with a function, PyObject_GenericAttach which registers a > callable to be notified when the object updates. > > Another approach would be to keep a single registry list with access > functions: attach(subject,observer), detach(subject,observer), and > notifyall(subject). PyObject_HEAD would have a field, int being_observed=0. > The attach() function sets being_observed. Any code affecting object state > is obliged to call PyObject_NOTIFY, a macro that checks being_observed and, > if set, calls notifyall(subject). doesn't "Any code affecting object state" include an awful lot of places? holger From pobrien@orbtech.com Fri May 10 19:56:33 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Fri, 10 May 2002 13:56:33 -0500 Subject: [Python-Dev] Oberver Pattern In-Reply-To: <009e01c1f84f$bd75bfc0$89b53bd0@othello> Message-ID: [Raymond Hettinger] > > I would like to know what you all think about formalizing an observer > protocol for python objects. Interesting question. I'm curious to see the responses this will get. For discussion sake I'll throw in my dispatcher recipe as an example of a very loosely coupled alternative to the traditional Observer pattern: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/87056 I'm not suggesting that it does exactly what you want. I just want to round out the discussion. Zope3 is doing something similar with its EventHub, I believe. There is something about the Observer pattern that I've never fully liked. It seems to introduce a level of coupling that, to my way of thinking, "corrupts" the design of the object being observed. In real life, things can be observed without their knowledge. So why should an observable object have to keep track of all of its listeners and notify them regularly? That doesn't seem like the right distribution of responsibility to me. And that line of thinking lead me to create dispatcher. But I'm willing to hear about the benefits of your idea. A standardized implementation might overcome my initial reluctance. So what are the benefits of formalizing this? --- Patrick K. O'Brien Orbtech From python@rcn.com Fri May 10 21:29:07 2002 From: python@rcn.com (Raymond Hettinger) Date: Fri, 10 May 2002 16:29:07 -0400 Subject: [Python-Dev] Oberver Pattern References: <009e01c1f84f$bd75bfc0$89b53bd0@othello> <02fa01c1f853$ac7e31d0$6300000a@holdenweb.com> Message-ID: <002601c1f861$568ecd80$29e97ad1@othello> I'm thinking that this can be implemented in a way that has an almost zero speed penalty unless observation is actually taking place for an object and, even then, having only the cost of a function call. Changed to existing code are expected to be minimal. For a simple container type, implementation could be as small as changing the flag line and inserting one line, PyNotify(self). The addition of the int being_observed field would take place implicitly through PyObject_HEAD. From: "Steve Holden" > The idea, presumably, being to make the observer protocol implicit at the C > level? Implemented at the C level and accessible at the Python level. > I'd only thought of doing this explicitly in Python. It seems like > this testing could have a huge performance penalty. Taking dictionaryobject for example, anywhere the code assigns a value to dp->me_key or dp->me_value, add a macro line: PyNotify(self); which expands to: if (self->being_observed) PyObject_Notify_All(self); When not being observed, the performance penalty is nearly zero. When being observed, the cost is only a for loop over the function calls: for( i=0 ; iAlex's smart > shelved, being implemented in Python, would clearly need some way (yet more > magic methods?) to hook into this mechanism. No magic methods required, you can notify any callable: tgt = ['the', 'quick' , 'brown'] m = MyClass() Observer.attach(tgt, m.invalidatecache) tgt[2] = 'red' # updates tgt and triggers call: m.invalidatecache(tgt) From: "Patrick K. O'Brien" > There is something about the Observer pattern that I've never fully liked. > It seems to introduce a level of coupling that, to my way of thinking, > "corrupts" the design of the object being observed. There is no coupling. The observed never knows anything about the observers. When a state change is made, it runs PyNotify(self). No outside knowledge is required. The GoF book specifically lists a need for loose coupling as a force leading to the Observer Pattern. > In real life, things can > be observed without their knowledge. When moving to a new house, you send out a change of address to all of your observers. When the phone number changes, you put on a message with the new number. This keeps the observers from having to constantly check-up on you. > So why should an observable object have > to keep track of all of its listeners and notify them regularly? A standard block of registry code will do the work. The only responsibility of the observed is to say, "hey, i've changed". > But I'm willing to hear about the benefits of your idea. A standardized > implementation might overcome my initial reluctance. So what are the > benefits of formalizing this? Thanks. Alex's code is a prime example. Not knowing which shelve values have changed, it is forced to do a 100% rewrite of all data in the shelve whether or not any changes have occurred. What a bummer. From: "holger krekel" > doesn't "Any code affecting object state" include an awful lot of places? Immutables don't need to be observable. So that leaves jcontainers like dictionaries and lists. The code for these objects each have about a dozen places with some action like, ob_item[x] = val. Outside the object's code, the direct accesses have been factored-out into just a couple of of direct access macros in abstract.h. If we want to exclude some object class from participating, we just leave it alone. The players identify themselves with: tp_flags |= Py_TPFLAGS_OBSERVABLE Raymond Hettinger From sholden@holdenweb.com Fri May 10 21:45:22 2002 From: sholden@holdenweb.com (Steve Holden) Date: Fri, 10 May 2002 16:45:22 -0400 Subject: [Python-Dev] Oberver Pattern References: <009e01c1f84f$bd75bfc0$89b53bd0@othello> Message-ID: <037d01c1f863$9b865f50$6300000a@holdenweb.com> ----- Original Message ----- From: "Raymond Hettinger" To: Sent: Friday, May 10, 2002 2:23 PM Subject: [Python-Dev] Oberver Pattern > I would like to know what you all think about formalizing an observer > protocol for python objects. > > A possible implementation would add Py_TPFLAGS_OBSERVABLE and a slot, > tp_observer, with a function, PyObject_GenericAttach which registers a > callable to be notified when the object updates. > "The object" seems a bit general. Do you mean to attach notification requests separately to each modifiable piece of state, or have the observer analyze which particular piece of object state had changed when the notification call was made? > Another approach would be to keep a single registry list with access > functions: attach(subject,observer), detach(subject,observer), and > notifyall(subject). PyObject_HEAD would have a field, int being_observed=0. > The attach() function sets being_observed. Any code affecting object state > is obliged to call PyObject_NOTIFY, a macro that checks being_observed and, > if set, calls notifyall(subject). > Correct me if I'm wrong, that seems pretty much like the Java event notification model. I could never make my mind up whether that sucked because Java's such a pain (I have never liked explicit declarations) or whether it was simply because I didn't like the model. Java's verbosity certainly makes it a bind having to deal with low-level details like GUI events, although the flexibility is worthwhile. > Alex's smart shelves provide an immediate application -- it needs to know > which of its cache objects have changed state. Another application would be > to enable computation caching in my matrix package which needs to invalidate > the cache whenever a matrix element gets updated. > The idea, presumably, being to make the observer protocol implicit at the C level? I'd only thought of doing this explicitly in Python. It seems like this testing could have a huge performance penalty. There's also the issue that the feature might need to interact with pure-Python objects, which in turn means that notifyall() might end up calling back into the Python environment. Seems like this might place more stress on the GIL? Or was your intention to restrict this feature solely to the C portions of the implementation (new-style classes)? Alex's smart shelved, being implemented in Python, would clearly need some way (yet more magic methods?) to hook into this mechanism. random-thoughts-from-a-random-brain-ly y'rs - steve -- ----------------------------------------------------------------------- Steve Holden http://www.holdenweb.com/ Python Web Programming http://pydish.holdenweb.com/pwp/ ----------------------------------------------------------------------- From David Abrahams" 1. I realize that some objects have no dict. Must every /type/ have a dict, or is it possible to have a type with only slots? 2. Suppose I want to generate a new type object dynamically? How do I go about choosing an appropriate allocator? TIA, Dave +---------------------------------------------------------------+ David Abrahams C++ Booster (http://www.boost.org) O__ == Pythonista (http://www.python.org) c/ /'_ == resume: http://users.rcn.com/abrahams/resume.html (*) \(*) == email: david.abrahams@rcn.com +---------------------------------------------------------------+ From guido@python.org Fri May 10 23:16:59 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 10 May 2002 18:16:59 -0400 Subject: [Python-Dev] Oberver Pattern In-Reply-To: Your message of "Fri, 10 May 2002 16:29:07 EDT." <002601c1f861$568ecd80$29e97ad1@othello> References: <009e01c1f84f$bd75bfc0$89b53bd0@othello> <02fa01c1f853$ac7e31d0$6300000a@holdenweb.com> <002601c1f861$568ecd80$29e97ad1@othello> Message-ID: <200205102216.g4AMGxo22077@pcp742651pcs.reston01.va.comcast.net> [Raymond Hettinger] > Changed to existing code are expected to be minimal. > For a simple container type, implementation could be as small as > changing the flag line and inserting one line, PyNotify(self). [and later] > Taking dictionaryobject for example, anywhere the code assigns > a value to dp->me_key or dp->me_value, add a macro line: > > PyNotify(self); > > which expands to: > > if (self->being_observed) PyObject_Notify_All(self); I think these two quoted sections contadict each other. If you have to insert a call to PyNotify(self) everywhere the dict is modified, you end up having to change a lot of places: every API that can change the dict. Note that you don't want to put the PyNotify() call in the lowest-level insertdict() call, because then operations like update() would notify the observer for each updated element, rather than once for all updated elements. It's worse for lists: there's a macro PyList_SET_ITEM(). You can't change the macro to automatically call PyNotify(self), because (again) that would notify the observer at too fine a granularity. But that means you have to update all code that *uses* PyList_SET_ITEM(). And that's a lot. > When not being observed, the performance penalty is nearly zero. Again, I think you are being just a dat naive here -- all those jumps (that are nearly always taken!) add up, and you're proposing to add this to every mutable object. > When being observed, the cost is only a for loop over the function > calls: > > for( i=0 ; i observerlist[i](self); > > The cost is only setting up the loop and making the call. If you *have* to do this, I would implement (at the C level) just a single observer object. If multiple observers are needed, this can easily be done using an adapter in Python. --Guido van Rossum (home page: http://www.python.org/~guido/) From bsder@allcaps.org Fri May 10 23:30:25 2002 From: bsder@allcaps.org (Andrew P. Lentvorski) Date: Fri, 10 May 2002 15:30:25 -0700 (PDT) Subject: [Python-Dev] Oberver Pattern In-Reply-To: <009e01c1f84f$bd75bfc0$89b53bd0@othello> Message-ID: <20020510145215.E20394-100000@mail.allcaps.org> On Fri, 10 May 2002, Raymond Hettinger wrote: > I would like to know what you all think about formalizing an observer > protocol for python objects. How do you reconcile the range of subject/observer coupling and the complexity of the notify object? At one end, the subject sends out all the necessary data for the observer to update its view without querying the subject at all. At the other end, the subject merely sends out a minimal notify and the observer has to figure out what changed. Do you intend to allow this flexibility or do you intend to impose one particular implementation? While I would love a formalized, reviewed set of python modules implementing patterns, I don't understand why you would want to do this at the C level. The normal advantage to going to C is performance. As a rule of thumb, the observer pattern and performance are mutually exclusive. -a From python@rcn.com Sat May 11 00:15:42 2002 From: python@rcn.com (Raymond Hettinger) Date: Fri, 10 May 2002 19:15:42 -0400 Subject: [Python-Dev] Oberver Pattern References: <009e01c1f84f$bd75bfc0$89b53bd0@othello> <02fa01c1f853$ac7e31d0$6300000a@holdenweb.com> <002601c1f861$568ecd80$29e97ad1@othello> <200205102216.g4AMGxo22077@pcp742651pcs.reston01.va.comcast.net> Message-ID: <002a01c1f878$9bfb82c0$77ec7ad1@othello> From: "Guido van Rossum" > Again, I think you are being just a dat naive here -- all those jumps > (that are nearly always taken!) add up, and you're proposing to add > this to every mutable object. How about a simpler alternative? Have just a single field in PyObject_HEAD, int cache_valid. Update routines wouldn't even need a jump. Everywhere the object assigns a value to ob_item[i], it would also run: PyObject_INVALIDATE_CACHE which would expand to: self->cache_valid=0. The only services would be macros for PyObject_IS_CACHE_VALID(obj) and PyObject_VALIDATE_CACHE(obj). > If you *have* to do this, I would implement (at the C level) just a > single observer object. If multiple observers are needed, this can > easily be done using an adapter in Python. Will do. Since it's not that hard to implement (meaning, less time than I spent on iterzip), I may as well try it and time it. Raymond Hettinge From python@rcn.com Sat May 11 00:19:38 2002 From: python@rcn.com (Raymond Hettinger) Date: Fri, 10 May 2002 19:19:38 -0400 Subject: [Python-Dev] Oberver Pattern References: <20020510145215.E20394-100000@mail.allcaps.org> Message-ID: <002b01c1f879$28f412a0$77ec7ad1@othello> From: "Andrew P. Lentvorski" > How do you reconcile the range of subject/observer coupling and the > complexity of the notify object? At one end, the subject sends out all > the necessary data for the observer to update its view without querying > the subject at all. At the other end, the subject merely sends out a > minimal notify and the observer has to figure out what changed. Like the GoF pattern, I had intended a minimal notify with self as a parameter; however, there is certainly merit in PyNotify(self, keyOrIndex). That would help the observer know what part of the container changed. > Do you > intend to allow this flexibility or do you intend to impose one particular > implementation? Hmm. Flexibility would be great; however, I should really start with the simplest possible implementation as a proof-of-concept and to see how it is received. > While I would love a formalized, reviewed set of python modules > implementing patterns, I don't understand why you would want to do this at > the C level. The normal advantage to going to C is performance. The other advantage of going to C is to make something possible that cannot be done at the pure Python level. In this case, there is currently no way to tell whether a mutuable container has changed without reviewing its entire content. > As a > rule of thumb, the observer pattern and performance are mutually > exclusive. Agreed :( I may have to implement the simplified version (GvR's idea for having only a single observer) in order to find-out for certain whether observation is cheap or expensive. Raymond Hettinger From guido@python.org Sat May 11 00:53:10 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 10 May 2002 19:53:10 -0400 Subject: [Python-Dev] A couple of quick type system questions In-Reply-To: Your message of "Fri, 10 May 2002 17:05:06 CDT." <09ce01c1f86f$333e6530$8f01a8c0@boostconsulting.com> References: <09ce01c1f86f$333e6530$8f01a8c0@boostconsulting.com> Message-ID: <200205102353.g4ANrAc22270@pcp742651pcs.reston01.va.comcast.net> > 1. I realize that some objects have no dict. Must every /type/ have > a dict, or is it possible to have a type with only slots? ob_type has the type PyType_Object *, and a PyType_Object has a tp_dict pointer, so I think the answer is that every /type/ has a dict. > 2. Suppose I want to generate a new type object dynamically? How do > I go about choosing an appropriate allocator? You could use the one used by type_new(). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Sat May 11 01:21:33 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 10 May 2002 20:21:33 -0400 Subject: [Python-Dev] Oberver Pattern In-Reply-To: Your message of "Fri, 10 May 2002 19:15:42 EDT." <002a01c1f878$9bfb82c0$77ec7ad1@othello> References: <009e01c1f84f$bd75bfc0$89b53bd0@othello> <02fa01c1f853$ac7e31d0$6300000a@holdenweb.com> <002601c1f861$568ecd80$29e97ad1@othello> <200205102216.g4AMGxo22077@pcp742651pcs.reston01.va.comcast.net> <002a01c1f878$9bfb82c0$77ec7ad1@othello> Message-ID: <200205110021.g4B0LX922370@pcp742651pcs.reston01.va.comcast.net> > How about a simpler alternative? > > Have just a single field in PyObject_HEAD, int cache_valid. This would add 4 bytes to all objects, not just to mutable containers. E.g. ints would grow from 12 to 16 bytes!!! Also, it would break binary compatibility (which we try to maintain for 3rd party extensions, despite appearances). > Update routines wouldn't even need a jump. > Everywhere the object assigns a value to ob_item[i], it would also run: > PyObject_INVALIDATE_CACHE which would expand to: > self->cache_valid=0. If this field could somehow exist only for mutable containers it *might* work. The type object could contain the offset of the field in the instances (like we use for weakrefs and for the dict offset). It's not a full observer pattern because it doesn't call anything, so algorithms that require notification aren't going to work (you'd have to check the cache flag in each object to see if you need to redisplay it, for example). > > If you *have* to do this, I would implement (at the C level) just a > > single observer object. If multiple observers are needed, this can > > easily be done using an adapter in Python. > > Will do. Since it's not that hard to implement (meaning, less time > than I spent on iterzip), I may as well try it and time it. But the impact is much bigger, because it potentially affects many files. I'm skeptical. OTOH, Jim Fulton gave this a +100 on an internal list. But he also wants notification upon access. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From thomas.heller@ion-tof.com Sat May 11 13:24:09 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Sat, 11 May 2002 14:24:09 +0200 Subject: [Python-Dev] A couple of quick type system questions References: <09ce01c1f86f$333e6530$8f01a8c0@boostconsulting.com> <200205102353.g4ANrAc22270@pcp742651pcs.reston01.va.comcast.net> Message-ID: <071401c1f8e6$c0be2d60$e000a8c0@thomasnotebook> > > 1. I realize that some objects have no dict. Must every /type/ have > > a dict, or is it possible to have a type with only slots? > > ob_type has the type PyType_Object *, and a PyType_Object has a > tp_dict pointer, so I think the answer is that every /type/ has a dict. > Huh? tp_dict of int (_PyInt_Type) is NULL. Thomas From martin@v.loewis.de Sat May 11 14:08:32 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 11 May 2002 15:08:32 +0200 Subject: [Python-Dev] A couple of quick type system questions In-Reply-To: <071401c1f8e6$c0be2d60$e000a8c0@thomasnotebook> References: <09ce01c1f86f$333e6530$8f01a8c0@boostconsulting.com> <200205102353.g4ANrAc22270@pcp742651pcs.reston01.va.comcast.net> <071401c1f8e6$c0be2d60$e000a8c0@thomasnotebook> Message-ID: "Thomas Heller" writes: > tp_dict of int (_PyInt_Type) is NULL. Yet >>> print len(int.__dict__) 46 Regards, Martin From stephen@xemacs.org Sat May 11 14:21:03 2002 From: stephen@xemacs.org (Stephen J. Turnbull) Date: 11 May 2002 22:21:03 +0900 Subject: [Python-Dev] Oberver Pattern In-Reply-To: <200205110021.g4B0LX922370@pcp742651pcs.reston01.va.comcast.net> References: <009e01c1f84f$bd75bfc0$89b53bd0@othello> <02fa01c1f853$ac7e31d0$6300000a@holdenweb.com> <002601c1f861$568ecd80$29e97ad1@othello> <200205102216.g4AMGxo22077@pcp742651pcs.reston01.va.comcast.net> <002a01c1f878$9bfb82c0$77ec7ad1@othello> <200205110021.g4B0LX922370@pcp742651pcs.reston01.va.comcast.net> Message-ID: <87y9eqes74.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Guido" == Guido van Rossum writes: Guido> Also, it would break binary compatibility (which we try to Guido> maintain for 3rd party extensions, despite appearances). The missing is a typo, right? -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN From guido@python.org Sat May 11 15:40:15 2002 From: guido@python.org (Guido van Rossum) Date: Sat, 11 May 2002 10:40:15 -0400 Subject: [Python-Dev] A couple of quick type system questions In-Reply-To: Your message of "Sat, 11 May 2002 14:24:09 +0200." <071401c1f8e6$c0be2d60$e000a8c0@thomasnotebook> References: <09ce01c1f86f$333e6530$8f01a8c0@boostconsulting.com> <200205102353.g4ANrAc22270@pcp742651pcs.reston01.va.comcast.net> <071401c1f8e6$c0be2d60$e000a8c0@thomasnotebook> Message-ID: <200205111440.g4BEeF923244@pcp742651pcs.reston01.va.comcast.net> > > ob_type has the type PyType_Object *, and a PyType_Object has a > > tp_dict pointer, so I think the answer is that every /type/ has a dict. > > > Huh? > > tp_dict of int (_PyInt_Type) is NULL. Huh? Not for me: [guido@pcp742651pcs guido]$ python2.2 Python 2.2.1 (#19, Apr 29 2002, 16:14:02) [GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-98)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> print int.__dict__.keys() ['__int__', '__ror__', '__rtruediv__', '__add__', '__str__', '__getattribute__', '__radd__', '__rmul__', '__truediv__', '__rrshift__', '__rsub__', '__rdiv__', '__and__', '__rmod__', '__abs__', '__float__', '__rshift__', '__rand__', '__new__', '__rlshift__', '__cmp__', '__pos__', '__rfloordiv__', '__doc__', '__mul__', '__rpow__', '__rdivmod__', '__invert__', '__coerce__', '__pow__', '__divmod__', '__lshift__', '__hex__', '__oct__', '__rxor__', '__nonzero__', '__mod__', '__neg__', '__xor__', '__div__', '__repr__', '__floordiv__', '__hash__', '__sub__', '__long__', '__or__'] >>> --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Sat May 11 19:04:37 2002 From: tim.one@comcast.net (Tim Peters) Date: Sat, 11 May 2002 14:04:37 -0400 Subject: [Python-Dev] Oberver Pattern In-Reply-To: <87y9eqes74.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: [Guido] > Also, it would break binary compatibility (which we try to > maintain for 3rd party extensions, despite appearances). [Stephen J. Turnbull] > The missing is a typo, right? Nope! Even if you get a warning about binary API mismatch across releases, it's *just* a warning -- the extension is very likely to run correctly despite it. For example, the binary API got bumped for 2.1 because new parameters were added to PyCode_New() and PyFrame_New(), but few extensions have a reason to use those functions. From stephen@xemacs.org Sat May 11 19:26:19 2002 From: stephen@xemacs.org (Stephen J. Turnbull) Date: 12 May 2002 03:26:19 +0900 Subject: [Python-Dev] Oberver Pattern In-Reply-To: References: Message-ID: <87vg9uczhw.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Tim" == Tim Peters writes: Tim> [Guido] >> Also, it would break binary compatibility (which we try to >> maintain for 3rd party extensions, despite appearances). Tim> [Stephen J. Turnbull] >> The missing is a typo, right? Tim> Nope! Even if you get a warning about binary API mismatch Tim> across releases, it's *just* a warning -- the extension is Tim> very likely to run correctly despite it. Er, I meant a on the "despite appearances," not on the "try to maintain." -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN From jason@redhat.com Sun May 12 11:09:38 2002 From: jason@redhat.com (Jason Merrill) Date: Sun, 12 May 2002 11:09:38 +0100 Subject: [Python-Dev] Re: Minimal GCC/Linux shared lib + EH bug example In-Reply-To: <09b501c1f634$04747d80$6501a8c0@boostconsulting.com> ("David Abrahams"'s message of "Tue, 7 May 2002 20:59:40 -0500") References: <09b501c1f634$04747d80$6501a8c0@boostconsulting.com> Message-ID: >>>>> "David" == David Abrahams writes: > FYI, Ralf Grosse-Kunstleve has reduced the exception-handling problem > mentioned here > http://mail.python.org/pipermail/c++-sig/2002-May/001021.html to a minimal > example: > http://cci.lbl.gov/~rwgk/tmp/gcc_dl_eh.tar.gz > gunzip -c gcc_dl_eh.tar.gz | tar xvf - > cd gcc_dl_eh > more 0README > The problem here is clearly a GCC/Linux interaction problem, *not* a Python > bug. However, it does have an impact on anyone writing Python extension > modules with g++ on Linux. IMO, it is unreasonable to expect C++ to work with RTLD_LOCAL unless the object so loaded is indeed self-contained (which precludes linking against a common shared library, as in this case). Too many aspects of the language depend on being able to merge duplicates coming from different sources. In this case, the problem comes from std::type_info; the runtime library expects to be able to compare type_info nodes by pointer equivalence. Templates and static variables in inline functions would also have trouble. Jason From David Abrahams" Message-ID: <0c2501c1f9ab$32ea8a40$8f01a8c0@boostconsulting.com> ----- Original Message ----- From: "Jason Merrill" > IMO, it is unreasonable to expect C++ to work with RTLD_LOCAL unless the > object so loaded is indeed self-contained (which precludes linking against > a common shared library, as in this case). Too many aspects of the > language depend on being able to merge duplicates coming from different > sources. I think there's an implicit assumption in your statement which should be brought into the open: that there's an agreed-upon idea of what it means for C++ to "work" with shared libraries. As you know, the language standard doesn't define how share libs are supposed to work, and people have different mental models. For example, on Windows, imports and exports are explicitly declared. Nobody expects to share static variables in inline functions across DLLs unless the function is explicitly exported. However, exception-handling and RTTI /are/ expected to work. > In this case, the problem comes from std::type_info; the runtime > library expects to be able to compare type_info nodes by pointer > equivalence. Templates and static variables in inline functions would also > have trouble. As I understand (guess) it, what happens is this: 1. lib1.so is loaded with RTLD_LOCAL. All of its symbols go into a new "symbol space" 2. the loader notices the dependency on X.so, and loads any /new/ symbols from the shared lib X.so into the same space, eliminating duplicates. 3. Now that all dependent libs are loaded, any unresolved symbols in lib1.so and X.so are resolved now; if any fail to resolve or there are duplicates, runtime error. ----- 4. lib2.so is loaded with RTLD_LOCAL. Because it's RTLD_LOCAL, the loader again creates a new "symbol space"; no duplicates are shared with X.so. 5. The loader notices the dependency on X.so, but X.so is already loaded 6. Any unresolved symbols in lib2.so (and X.so, though there are none) are resolved now ----- What I'd prefer to happen is that in step 4, the loader would use the existing definition for any loaded symbol which is defined in or used by lib2's immediate dependencies. That would nicely model the concept that lib2.so is sharing globally with X.so but not with lib1.so, and it seems like the "right" solution. However, for my application I'd be content if EH was just comparing the type_info::name() strings, as Martin von Loewis stated was the case in 2.95.x and again in 3.1: http://aspn.activestate.com/ASPN/Mail/Message/1191899 [This statement appears to be contradicted empirically, though: Ralf reports similar problems with GCC 3.1 - GNATS id 6629]. This would bring GCC sufficiently close to the model of Windows compilers (and of compilers on the other *NIX OSes he's tested on) to allow practical cross-platform authoring of plugins in C++. -Dave From skip@mojam.com Sun May 12 13:00:22 2002 From: skip@mojam.com (Skip Montanaro) Date: Sun, 12 May 2002 07:00:22 -0500 Subject: [Python-Dev] Weekly Python Bug/Patch Summary Message-ID: <200205121200.g4CC0M806255@12-248-41-177.client.attbi.com> Bug/Patch Summary ----------------- 252 open / 2487 total bugs (+6) 128 open / 1497 total patches (+5) New Bugs -------- email.Utils.encode doesn't obey rfc2047 (2002-05-06) http://python.org/sf/552957 "./configure" crashes (2002-05-06) http://python.org/sf/553000 [RefMan] Special status of "as" (2002-05-07) http://python.org/sf/553262 cPickle dies on short reads (2002-05-07) http://python.org/sf/553512 thread_pthread.h on FreeBSD (2002-05-08) http://python.org/sf/553736 bug in telnetlib- 'opt' instead of 'c' (2002-05-09) http://python.org/sf/554073 test_fcntl fails on OpenBSD 3.0 (2002-05-10) http://python.org/sf/554663 unknown locale de_DE@euro on Suse 8.0 Linux (2002-05-10) http://python.org/sf/554676 Circular reference in Index for frame (2002-05-10) http://python.org/sf/554750 --disable-unicode builds horked (2002-05-11) http://python.org/sf/554912 test_unicode fails in wide unicode build (2002-05-11) http://python.org/sf/554916 rfc822.Message.getaddrlist broken (2002-05-11) http://python.org/sf/555035 New Patches ----------- error about string formatting rewording? (2002-04-26) http://python.org/sf/549187 add support for HtmlHelp output (2002-05-06) http://python.org/sf/552835 texi2html.py - add support for HTML Help (2002-05-06) http://python.org/sf/552837 Deprecate bsddb (2002-05-06) http://python.org/sf/553108 optionally make shelve less surprising (2002-05-07) http://python.org/sf/553171 Cygwin Makefile.pre.in vestige patch (2002-05-08) http://python.org/sf/553678 Cygwin make install patch (2002-05-08) http://python.org/sf/553702 mimetypes: all extensions for a type (2002-05-09) http://python.org/sf/554192 __va_copy patches (2002-05-10) http://python.org/sf/554716 OpenBSD fixes for Python 2.1 (2002-05-10) http://python.org/sf/554718 OpenBSD fixes for Python 2.2 (2002-05-10) http://python.org/sf/554719 Add _winreg support for Cygwin (2002-05-11) http://python.org/sf/554807 THREAD_STACK_SIZE for 2.1 (2002-05-11) http://python.org/sf/554841 Closed Bugs ----------- Lists do have __getitem__ method (2002-04-25) http://python.org/sf/548874 Python build not Solaris friendly (2002-05-02) http://python.org/sf/551353 type->tp_mro NULL, typeobject.c:1217 (2002-05-02) http://python.org/sf/551551 Closed Patches -------------- iterzip() implementation (2002-04-27) http://python.org/sf/549662 block support for builtin iter() (2002-04-28) http://python.org/sf/549975 Cygwin pyexpat patch (2002-05-01) http://python.org/sf/551011 xrange() optimization (2002-05-02) http://python.org/sf/551410 Looping Optimization (2002-05-04) http://python.org/sf/552433 Correct ref manual error for list type (2002-05-05) http://python.org/sf/552468 Explain buffering effects of xreadlines (2002-05-06) http://python.org/sf/552804 Better description in "python -h" for -u (2002-05-06) http://python.org/sf/552810 From mgilfix@eecs.tufts.edu Sun May 12 13:27:40 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Sun, 12 May 2002 08:27:40 -0400 Subject: [Python-Dev] Socket timeout patch Message-ID: <20020512082740.C10230@eecs.tufts.edu> The socket timeout patch is finally available with doc updates n' unit test as patch #555085 in the SF tracker. This implementation provides timeout functionality at the C level. A patch to socket.py also fixes the problem of losing data while an exception is thrown to the underlying socket. -- Mike -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From jason@redhat.com Sun May 12 14:42:06 2002 From: jason@redhat.com (Jason Merrill) Date: Sun, 12 May 2002 14:42:06 +0100 Subject: [Python-Dev] Re: Minimal GCC/Linux shared lib + EH bug example In-Reply-To: <0c2501c1f9ab$32ea8a40$8f01a8c0@boostconsulting.com> ("David Abrahams"'s message of "Sun, 12 May 2002 06:50:21 -0500") References: <09b501c1f634$04747d80$6501a8c0@boostconsulting.com> <0c2501c1f9ab$32ea8a40$8f01a8c0@boostconsulting.com> Message-ID: >>>>> "David" == David Abrahams writes: > I think there's an implicit assumption in your statement which should be > brought into the open: that there's an agreed-upon idea of what it means > for C++ to "work" with shared libraries. As you know, the language standard > doesn't define how share libs are supposed to work, and people have > different mental models. For example, on Windows, imports and exports are > explicitly declared. Nobody expects to share static variables in inline > functions across DLLs unless the function is explicitly exported. However, > exception-handling and RTTI /are/ expected to work. And on Windows, we don't rely on address equivalence. > What I'd prefer to happen is that in step 4, the loader would use the > existing definition for any loaded symbol which is defined in or used by > lib2's immediate dependencies. That would nicely model the concept that > lib2.so is sharing globally with X.so but not with lib1.so, and it seems > like the "right" solution. I noticed that the readme says that the test passes on Solaris. Does it provide these semantics? How about SCO? Anyone? > However, for my application I'd be content if EH was just comparing the > type_info::name() strings, as Martin von Loewis stated was the case in > 2.95.x and again in 3.1: > http://aspn.activestate.com/ASPN/Mail/Message/1191899 [This statement > appears to be contradicted empirically, though: Ralf reports similar > problems with GCC 3.1 - GNATS id 6629]. Yes, 3.1 still relies on pointer comparison. I find this testcase somewhat persuasive, as the offending dlopen call is not in the C++ code. What do others think? Jason From David Abrahams" <0c2501c1f9ab$32ea8a40$8f01a8c0@boostconsulting.com> Message-ID: <0c4501c1f9bf$583bd330$8f01a8c0@boostconsulting.com> For the c++std-ext recipients, just added: previous messages in this thread, which concerns the semantics of shared libraries w.r.t. exception-handling and other C++ features which require "vague linkage", can be found here: http://gcc.gnu.org/ml/gcc/2002-05/msg00866.html http://gcc.gnu.org/ml/gcc/2002-05/msg00869.html http://gcc.gnu.org/ml/gcc/2002-05/msg00873.html ----- Original Message ----- From: "Jason Merrill" > > What I'd prefer to happen is that in step 4, the loader would use the > > existing definition for any loaded symbol which is defined in or used by > > lib2's immediate dependencies. That would nicely model the concept that > > lib2.so is sharing globally with X.so but not with lib1.so, and it seems > > like the "right" solution. > > I noticed that the readme says that the test passes on Solaris. Does it > provide these semantics? How about SCO? Anyone? The test as written doesn't really tell us the answer since it uses EH and any implementation can make it a non-issue by comparing type_info::name() strings instead of addresses. The test could easily be modified the so it looks at the address of a class template's static data member, of course. > > However, for my application I'd be content if EH was just comparing the > > type_info::name() strings, as Martin von Loewis stated was the case in > > 2.95.x and again in 3.1: > > http://aspn.activestate.com/ASPN/Mail/Message/1191899 [This statement > > appears to be contradicted empirically, though: Ralf reports similar > > problems with GCC 3.1 - GNATS id 6629]. > > Yes, 3.1 still relies on pointer comparison. > > I find this testcase somewhat persuasive, as the offending dlopen call is > not in the C++ code. What do others think? I guess you /know/ what I think: I just want it to work ;-) I'd also like to point out what I think is a fundamental difference in the Windows and Linux models for shared libraries: in the Windows model, sharing has a "direction" but in the Linux model the direction is determined by who got there first. So, for example, in the Windows model when lib.dll and lib2.dll are loaded there are certain symbols which come in from their dependency, X.dll, because they're explicitly imported. It's the fact that the libraries explicitly "pull" symbols from their dependencies that makes the model work. In the Linux model, it appears that lib1.so and lib2.so essentially load everything they've got and "push" their symbols onto X.so. In the case we're looking at, the dependency may already be loaded so the push might leave us with two copies of some symbols in the lib.so/X.so relationship. -Dave From jason@redhat.com Sun May 12 15:30:48 2002 From: jason@redhat.com (Jason Merrill) Date: Sun, 12 May 2002 15:30:48 +0100 Subject: [Python-Dev] Re: Minimal GCC/Linux shared lib + EH bug example In-Reply-To: <0c4501c1f9bf$583bd330$8f01a8c0@boostconsulting.com> ("David Abrahams"'s message of "Sun, 12 May 2002 09:14:32 -0500") References: <09b501c1f634$04747d80$6501a8c0@boostconsulting.com> <0c2501c1f9ab$32ea8a40$8f01a8c0@boostconsulting.com> <0c4501c1f9bf$583bd330$8f01a8c0@boostconsulting.com> Message-ID: >>>>> "David" == David Abrahams writes: >> I noticed that the readme says that the test passes on Solaris. Does it >> provide these semantics? How about SCO? Anyone? > The test as written doesn't really tell us the answer since it uses EH and > any implementation can make it a non-issue by comparing type_info::name() > strings instead of addresses. I meant using gcc 3.0.4 on Solaris. Jason From David Abrahams" <0c2501c1f9ab$32ea8a40$8f01a8c0@boostconsulting.com><0c4501c1f9bf$583bd330$8f01a8c0@boostconsulting.com> Message-ID: <0c8601c1f9c4$3f9414a0$8f01a8c0@boostconsulting.com> Jason, I had to write all the following exposition to understand your reply, but then it dawned on me what you meant ;-) I wrote, describing existing Linux/GCC semantics: > >>> 4. lib2.so is loaded with RTLD_LOCAL. Because it's RTLD_LOCAL, the loader > >>> again creates a new "symbol space"; no duplicates are shared with X.so. And then, describing my preferred semantics: > >>> What I'd prefer to happen is that in step 4, the loader would use the > >>> existing definition for any loaded symbol which is defined in or used by > >>> lib2's immediate dependencies. That would nicely model the concept that > >>> lib2.so is sharing globally with X.so but not with lib1.so, and it seems > >>> like the "right" solution. Jason replied: > >> I noticed that the readme says that the test passes on Solaris. Does it > >> provide these semantics? How about SCO? Anyone? Assuming by "these semantics", Jason meant my preferred semantics: > > The test as written doesn't really tell us the answer since it uses EH and > > any implementation can make it a non-issue by comparing type_info::name() > > strings instead of addresses. > > I meant using gcc 3.0.4 on Solaris. Ah yes, GCC 3.0.4 would tell us something, since it is using address comparison. If it worked on Solaris, that would be just as good as using a different test with that looked at addresses of template static data members. Good question. -Dave From loewis@informatik.hu-berlin.de Sun May 12 18:01:39 2002 From: loewis@informatik.hu-berlin.de (Martin v. =?iso-8859-1?q?L=F6wis?=) Date: 12 May 2002 19:01:39 +0200 Subject: [Python-Dev] Deprecating ports Message-ID: The AtheOS patch, and patch 554718 trigger, for me, the question that we need a way to stop supporting certain platforms. I would recommend a two-phase approach: In release X, announce that support for certain platforms is "dead"; in release X+1, actively remove the code introduced to support them. In the specific case of patch 554718, I'd recommend to stop supporting "old" OpenBSD releases already with Python 2.3 (or perhaps even with 2.2.x); some OpenBSD expert should give us a conservative definition of "old" (requiring 1.x sounds conservative to me). I'm sure many patches to support more recent releases have silently broken older releases. For many of the older releases, we will never be able to find out, since nobody is using them anymore. With the announcement of removing code for a certain platform, we will give users of that platform a chance to indicate that they are still interested in using new Python releases on those platforms. If nobody contributes patches, or atleast indicates that the ports are still working, we should remove the dead code. High on my list would be to remove the support for the various pthreads drafts. Regards, Martin From mark@codesourcery.com Sun May 12 19:29:44 2002 From: mark@codesourcery.com (Mark Mitchell) Date: Sun, 12 May 2002 11:29:44 -0700 Subject: [Python-Dev] Re: Minimal GCC/Linux shared lib + EH bug example In-Reply-To: Message-ID: <50980000.1021228184@warlock.codesourcery.com> > I find this testcase somewhat persuasive, as the offending dlopen call is > not in the C++ code. What do others think? I agree with your other statement: RTLD_LOCAL and C++ don't really make sense. I think we're running down a slippery slope; once EH works, people *will* wonder why things involving inlines and templates don't. If, for example, you have *two* Python modules in C++, each of which uses a nice package for managing global resources, and you can load either module just fine, but loading both causes subtle runtime problems, ... We will have given people a bigger bazooka, but it will be aimed at their own feet. -- Mark Mitchell mark@codesourcery.com CodeSourcery, LLC http://www.codesourcery.com From guido@python.org Sun May 12 21:49:57 2002 From: guido@python.org (Guido van Rossum) Date: Sun, 12 May 2002 16:49:57 -0400 Subject: [Python-Dev] Deprecating ports In-Reply-To: Your message of "12 May 2002 19:01:39 +0200." References: Message-ID: <200205122049.g4CKnvU00451@pcp742651pcs.reston01.va.comcast.net> > The AtheOS patch, and patch 554718 trigger, for me, the question that > we need a way to stop supporting certain platforms. I would recommend > a two-phase approach: In release X, announce that support for certain > platforms is "dead"; in release X+1, actively remove the code > introduced to support them. > > In the specific case of patch 554718, I'd recommend to stop supporting > "old" OpenBSD releases already with Python 2.3 (or perhaps even with > 2.2.x); some OpenBSD expert should give us a conservative definition > of "old" (requiring 1.x sounds conservative to me). > > I'm sure many patches to support more recent releases have silently > broken older releases. For many of the older releases, we will never > be able to find out, since nobody is using them anymore. > > With the announcement of removing code for a certain platform, we will > give users of that platform a chance to indicate that they are still > interested in using new Python releases on those platforms. If nobody > contributes patches, or atleast indicates that the ports are still > working, we should remove the dead code. > > High on my list would be to remove the support for the various > pthreads drafts. +1. --Guido van Rossum (home page: http://www.python.org/~guido/) From greg@cosc.canterbury.ac.nz Mon May 13 00:55:44 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Mon, 13 May 2002 11:55:44 +1200 (NZST) Subject: [Python-Dev] Oberver Pattern In-Reply-To: <009e01c1f84f$bd75bfc0$89b53bd0@othello> Message-ID: <200205122355.LAA04466@s454.cosc.canterbury.ac.nz> Raymond Hettinger : > I would like to know what you all think about formalizing an observer > protocol for python objects. > > Alex's smart shelves provide an immediate application -- it needs to know > which of its cache objects have changed state. Another application would be > to enable computation caching in my matrix package which needs to invalidate > the cache whenever a matrix element gets updated. This could be very useful for GUIs, too, for implementing model-view relationships. The anygui people would be very interested, I'm sure. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From guido@python.org Mon May 13 01:20:04 2002 From: guido@python.org (Guido van Rossum) Date: Sun, 12 May 2002 20:20:04 -0400 Subject: [Python-Dev] Oberver Pattern In-Reply-To: Your message of "Mon, 13 May 2002 11:55:44 +1200." <200205122355.LAA04466@s454.cosc.canterbury.ac.nz> References: <200205122355.LAA04466@s454.cosc.canterbury.ac.nz> Message-ID: <200205130020.g4D0K4208556@pcp742651pcs.reston01.va.comcast.net> > This could be very useful for GUIs, too, for implementing > model-view relationships. The anygui people would be very > interested, I'm sure. Obviously it's useful for GUIs, it was invented for that use. But does every built-in mutable container type need to implement it to be useful there? --Guido van Rossum (home page: http://www.python.org/~guido/) From python@rcn.com Mon May 13 01:41:26 2002 From: python@rcn.com (Raymond Hettinger) Date: Sun, 12 May 2002 20:41:26 -0400 Subject: [Python-Dev] Mutable object change flag References: <009e01c1f84f$bd75bfc0$89b53bd0@othello> <02fa01c1f853$ac7e31d0$6300000a@holdenweb.com> <002601c1f861$568ecd80$29e97ad1@othello> <200205102216.g4AMGxo22077@pcp742651pcs.reston01.va.comcast.net> <002a01c1f878$9bfb82c0$77ec7ad1@othello> <200205110021.g4B0LX922370@pcp742651pcs.reston01.va.comcast.net> Message-ID: <002701c1fa16$eb0851a0$56f7a4d8@othello> As an early step toward implementing an observer pattern, I built a simpler protocol that sets a mutable object's attribute to zero whenever the object changes state. If you guys would like to try it out, see the patch at www.python.org/sf/555251. I tried to time the cost of frequent attribute resets but they were so fast I couldn't produce a measurable difference. Raymond Hettinger Here's a sample session: >>> a = list('abcd') >>> a.cachevalid 0 >>> a.cachevalid=1 >>> a.cachevalid 1 >>> a[2] = 'k' >>> a.cachevalid 0 From fredrik@pythonware.com Mon May 13 07:45:35 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Mon, 13 May 2002 08:45:35 +0200 Subject: [Python-Dev] Oberver Pattern References: <200205122355.LAA04466@s454.cosc.canterbury.ac.nz> <200205130020.g4D0K4208556@pcp742651pcs.reston01.va.comcast.net> Message-ID: <00dd01c1fa49$cd8078f0$0900a8c0@spiff> guido wrote: > > This could be very useful for GUIs, too, for implementing > > model-view relationships. The anygui people would be very > > interested, I'm sure. >=20 > Obviously it's useful for GUIs, it was invented for that use. But > does every built-in mutable container type need to implement it to be > useful there? no. -1 from here. there are already ways to do everything that C-level observers can do, and the existing ways are far more flexible. From thomas.heller@ion-tof.com Mon May 13 08:28:33 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Mon, 13 May 2002 09:28:33 +0200 Subject: [Python-Dev] A couple of quick type system questions References: <09ce01c1f86f$333e6530$8f01a8c0@boostconsulting.com> <200205102353.g4ANrAc22270@pcp742651pcs.reston01.va.comcast.net> <071401c1f8e6$c0be2d60$e000a8c0@thomasnotebook> <200205111440.g4BEeF923244@pcp742651pcs.reston01.va.comcast.net> Message-ID: <0aa101c1fa4f$c9acccf0$e000a8c0@thomasnotebook> > > > ob_type has the type PyType_Object *, and a PyType_Object has a > > > tp_dict pointer, so I think the answer is that every /type/ has a dict. > > > > > Huh? > > > > tp_dict of int (_PyInt_Type) is NULL. > > Huh? Not for me: > > [guido@pcp742651pcs guido]$ python2.2 > Python 2.2.1 (#19, Apr 29 2002, 16:14:02) > [GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-98)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> print int.__dict__.keys() > ['__int__', '__ror__', '__rtruediv__', '__add__', It seems tp_dict is lazily initialized: Python 2.2.1 (#34, Apr 9 2002, 19:34:33) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> from get_dict import get_dict >>> get_dict(int) Traceback (most recent call last): File "", line 1, in ? SystemError: error return without exception set >>> len(int.__dict__) 46 >>> get_dict(int) {'__int__': , '__ror__': tp_dict); return tp->tp_dict; } static PyMethodDef module_methods[] = { {"get_dict", get_dict, METH_O}, {NULL, NULL} /* Sentinel */ }; DL_EXPORT(void) initget_dict(void) { Py_InitModule3("get_dict", module_methods, NULL); } ---EOF--- From guido@python.org Mon May 13 13:47:58 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 13 May 2002 08:47:58 -0400 Subject: [Python-Dev] A couple of quick type system questions In-Reply-To: Your message of "Mon, 13 May 2002 09:28:33 +0200." <0aa101c1fa4f$c9acccf0$e000a8c0@thomasnotebook> References: <09ce01c1f86f$333e6530$8f01a8c0@boostconsulting.com> <200205102353.g4ANrAc22270@pcp742651pcs.reston01.va.comcast.net> <071401c1f8e6$c0be2d60$e000a8c0@thomasnotebook> <200205111440.g4BEeF923244@pcp742651pcs.reston01.va.comcast.net> <0aa101c1fa4f$c9acccf0$e000a8c0@thomasnotebook> Message-ID: <200205131247.g4DClwZ09375@pcp742651pcs.reston01.va.comcast.net> > It seems tp_dict is lazily initialized: Correct. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon May 13 18:41:13 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 13 May 2002 13:41:13 -0400 Subject: [Python-Dev] pthread backport to 2.1.x Message-ID: <200205131741.g4DHfDN21021@pcp742651pcs.reston01.va.comcast.net> Reference: python.org/sf/553736 There's been a request to backport the pthread changes made in 2.2 to the 2.1.x branch. Is any of the new (or old?) developers interested in helping out with this? It *may* be a simple backport, but I'm not sure -- the code is too full of #ifdefs to tell. --Guido van Rossum (home page: http://www.python.org/~guido/) From magnus@hetland.org Tue May 14 16:44:33 2002 From: magnus@hetland.org (Magnus Lie Hetland) Date: Tue, 14 May 2002 17:44:33 +0200 Subject: [Python-Dev] Oberver Pattern In-Reply-To: <200205122355.LAA04466@s454.cosc.canterbury.ac.nz>; from greg@cosc.canterbury.ac.nz on Mon, May 13, 2002 at 11:55:44AM +1200 References: <009e01c1f84f$bd75bfc0$89b53bd0@othello> <200205122355.LAA04466@s454.cosc.canterbury.ac.nz> Message-ID: <20020514174433.A4625@idi.ntnu.no> Greg Ewing : [snip] > This could be very useful for GUIs, too, for implementing > model-view relationships. The anygui people would be very > interested, I'm sure. Indeed! -- Magnus Lie Hetland The Anygui Project http://hetland.org http://anygui.org From fredrik@pythonware.com Tue May 14 17:21:22 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Tue, 14 May 2002 18:21:22 +0200 Subject: [Python-Dev] Oberver Pattern References: <009e01c1f84f$bd75bfc0$89b53bd0@othello> <200205122355.LAA04466@s454.cosc.canterbury.ac.nz> <20020514174433.A4625@idi.ntnu.no> Message-ID: <05a501c1fb63$63fb9250$0900a8c0@spiff> Magnus wrote: > Greg Ewing : > [snip] > > This could be very useful for GUIs, too, for implementing > > model-view relationships. The anygui people would be very > > interested, I'm sure. >=20 > Indeed! have you actually built large-scale UI applications using a one-size-fits-all observer model, or are you just guessing? in real life, "something happened" is almost entirely useless, and the other extreme "this and this and this and this and this happened" isn't much better. to get performance, you need to think in data model terms... From casey@zope.com Tue May 14 17:30:36 2002 From: casey@zope.com (Casey Duncan) Date: Tue, 14 May 2002 12:30:36 -0400 Subject: [Python-Dev] Python Thread Stack Size Message-ID: <200205141230.36659.casey@zope.com> Hi All: For those of you who don't know me, I am Casey Duncan, a recent addition = to=20 the engineering team at Zope corp.=20 Jim Fulton and I were talking about a problem that manifests itself on BS= D in=20 threaded Python applications that use a lot of stack space (notably the C= MF=20 application in Zope). It seems that FreeBSD has a default thread stack size of 64K, whereas mo= st=20 unices have somewhere more like 1Mb. A patch exists to have Python bump u= p=20 the stack size, but this involves patching the Python source. Jim suggested that Python should set the thread stack size itself to some= thing=20 like 256 * the pointer size, which would work out to 1Mb on 32 bit platfo= rms=20 and 2Mb on 64 bit platforms. This would hopefully lead to more consistent= =20 behavior across platforms. Alternately (or additionally) it could be a configure option like=20 --thread-stack=3D1024 or somesuch. A third alternative (which is less desireable IMO) would be to change the= =20 Python BSD port so that it includes a patch to do this. Obviously that=20 doesn't help ppl building directly from source very much tho. Thoughts? -Casey=20 From geoff-pdev@gerrietts.net Tue May 14 21:25:22 2002 From: geoff-pdev@gerrietts.net (Geoff Gerrietts) Date: Tue, 14 May 2002 13:25:22 -0700 Subject: [Python-Dev] Python Thread Stack Size In-Reply-To: <200205141230.36659.casey@zope.com> References: <200205141230.36659.casey@zope.com> Message-ID: <20020514202522.GB31960@isis.gerrietts.net> Quoting Casey Duncan (casey@zope.com): > Hi All: > > For those of you who don't know me, I am Casey Duncan, a recent addition to > the engineering team at Zope corp. > > Jim Fulton and I were talking about a problem that manifests itself on BSD in > threaded Python applications that use a lot of stack space (notably the CMF > application in Zope). > > It seems that FreeBSD has a default thread stack size of 64K, whereas most > unices have somewhere more like 1Mb. A patch exists to have Python bump up > the stack size, but this involves patching the Python source. > > Jim suggested that Python should set the thread stack size itself to something > like 256 * the pointer size, which would work out to 1Mb on 32 bit platforms > and 2Mb on 64 bit platforms. This would hopefully lead to more consistent > behavior across platforms. > > Alternately (or additionally) it could be a configure option like > --thread-stack=1024 or somesuch. > > A third alternative (which is less desireable IMO) would be to change the > Python BSD port so that it includes a patch to do this. Obviously that > doesn't help ppl building directly from source very much tho. > > Thoughts? Having spent a lot of time looking at this problem (it was a red herring, but thoroughly investigated) on Linux, I'd offer a slight warning for that platform: Linux uses pthreads just like the other platforms, has a default thread stack size of 2MB, and under the RedHat builds of glibc I've played with, that stack size is not mutable due to compile-time settings on glibc. When you try to call pthread_attr_setstacksize for anything but 2 * 1024 * 1024, it returns EINVAL. This may be an aberration of the RedHat setup -- I haven't tested other Linux distributions, and I never got to the point of recompiling glibc to see if I could make "dynamic stacks" work. Because of the limitation -- and because of the large default stack size on Linux -- I'd suggest that if you're going to build stack size readjustment into Python, you go with something like: int rc = 0, sz; pthread_attr_t pta; rc = phtread_attr_init(&pta); rc = pthread_attr_getstacksize(&pta, &sz); if (sz < YOUR_STACK_SIZE_HERE) { /* now try setting the stacksize */ } When I was thinking about this problem -- when I still thought it might be a problem -- I gave some serious thought to making the stack size a parameter on the thread module, so that threaded applications could specify the amount of stack space they expected to require. There are probably a billion reasons that's not a good idea, but it seemed to me like it offered the flexibility at the place where you were likely to need it, rather than at the place where it was most convenient to provide it. Thanks, --G. -- Geoff Gerrietts "I am always doing that which I can not do, in order that I may learn how to do it." http://www.gerrietts.net --Pablo Picasso From Jack.Jansen@oratrix.com Tue May 14 23:01:38 2002 From: Jack.Jansen@oratrix.com (Jack Jansen) Date: Wed, 15 May 2002 00:01:38 +0200 Subject: [Python-Dev] Python Thread Stack Size In-Reply-To: <200205141230.36659.casey@zope.com> Message-ID: <2A5B2660-6786-11D6-A7EA-003065517236@oratrix.com> On dinsdag, mei 14, 2002, at 06:30 , Casey Duncan wrote: > Jim suggested that Python should set the thread stack size > itself to something > like 256 * the pointer size, which would work out to 1Mb on 32 > bit platforms > and 2Mb on 64 bit platforms. This would hopefully lead to more > consistent > behavior across platforms. > > Alternately (or additionally) it could be a configure option like > --thread-stack=1024 or somesuch. At the very least it would need to be configurable. On some machines (such as MacOS9 and earlier) the space is actually allocated, so by setting it to 2MB you would quickly run out of memory. An even better option might be to let the Python program influence the thread stack size (thread.setstacksize(), maybe? An optional parameter to the thread start method? A more abstract hint not mentioning actual numbers but just small/normal/big?), after all it will know (to some extent) whether it wants many threads with small stacks or a few threads with big stacks. -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From martin@v.loewis.de Tue May 14 19:54:52 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 14 May 2002 20:54:52 +0200 Subject: [Python-Dev] Python Thread Stack Size In-Reply-To: <200205141230.36659.casey@zope.com> References: <200205141230.36659.casey@zope.com> Message-ID: Casey Duncan writes: > It seems that FreeBSD has a default thread stack size of 64K Can you point to any documentation that says this is the case? > Jim suggested that Python should set the thread stack size itself to > something like 256 * the pointer size, which would work out to 1Mb > on 32 bit platforms and 2Mb on 64 bit platforms. This would > hopefully lead to more consistent behavior across platforms. I think the stack size should accommodate for sys.getrecursionlimit, with a generous estimate of how much one recursion (in ceval) consumes. There not much point of setting the default to a higher value, since you will get an exception anyway if you exhaust the stack. In addition, I think on platforms supporting setstacksize, the thread stack size should be runtime variable, so that sys.setrecursionlimit can expand the stack reserve estimate if necessary. Regards, Martin From greg@cosc.canterbury.ac.nz Wed May 15 01:09:14 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Wed, 15 May 2002 12:09:14 +1200 (NZST) Subject: [Python-Dev] Oberver Pattern Message-ID: <200205150009.MAA04723@s454.cosc.canterbury.ac.nz> Fredrik Lundh : > have you actually built large-scale UI applications using a > one-size-fits-all observer model, or are you just guessing? > > in real life, "something happened" is almost entirely useless, > and the other extreme "this and this and this and this and > this happened" isn't much better. Having thought it through a bit more, you're probably right. I just thought it might appeal to Magnus, since he seems to be fond of lots of layers of magic, without much if any regard for performance... Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From python@rcn.com Wed May 15 01:27:17 2002 From: python@rcn.com (Raymond Hettinger) Date: Tue, 14 May 2002 20:27:17 -0400 Subject: [Python-Dev] Oberver Pattern References: <200205150009.MAA04723@s454.cosc.canterbury.ac.nz> Message-ID: <001501c1fba7$457e4d60$e861accf@othello> From: "Greg Ewing" > Having thought it through a bit more, you're probably > right. I just thought it might appeal to Magnus, since > he seems to be fond of lots of layers of magic, without > much if any regard for performance... FWIW, the early results from my test implementation is that there is no measurable performance impact if an observer function has not been attached or if the observer function is a short C call (perhaps setting a flag or logging the object id). The only way to slow it down enough for the timer to notice seems to be attaching a pure python function to be notified. IOW, the observer pattern doesn't slow you down until you need it. And, if you need it, then you're probably doing it to save much more time being spent on repeated checks to see if a change has taken place and to save programmer effort by de-coupling the objects. BTW, Magnus rules. i-haven't-forgotten-+1-votes-on-generator-enhancements-ly yours , Raymond Hettinger _ ~ @ @ \_/ From andymac@bullseye.apana.org.au Tue May 14 22:09:12 2002 From: andymac@bullseye.apana.org.au (Andrew MacIntyre) Date: Wed, 15 May 2002 08:09:12 +1100 (edt) Subject: [Python-Dev] Python Thread Stack Size In-Reply-To: <200205141230.36659.casey@zope.com> Message-ID: On Tue, 14 May 2002, Casey Duncan wrote: > A third alternative (which is less desireable IMO) would be to change the > Python BSD port so that it includes a patch to do this. Obviously that > doesn't help ppl building directly from source very much tho. Including a suitable patch in the FreeBSD port doesn't have to be considered as a permanent solution. It is not unusual for patches to appear in ports then make their way upstream, so this approach can get a work around into circulation while a more permanent solution is evolved. -- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au | Snail: PO Box 370 andymac@pcug.org.au | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From just@letterror.com Wed May 15 08:10:38 2002 From: just@letterror.com (Just van Rossum) Date: Wed, 15 May 2002 09:10:38 +0200 Subject: [Python-Dev] Oberver Pattern In-Reply-To: <05a501c1fb63$63fb9250$0900a8c0@spiff> Message-ID: Fredrik Lundh wrote: > in real life, "something happened" is almost entirely useless, > and the other extreme "this and this and this and this and > this happened" isn't much better. to get performance, you > need to think in data model terms... There's one application for which I've occasionally wished there was a deeply integrated observer API: object browsers in an IDE. But other than that, I think you're absolutely right. Just From pyth@devel.trillke.net Wed May 15 10:34:45 2002 From: pyth@devel.trillke.net (holger krekel) Date: Wed, 15 May 2002 11:34:45 +0200 Subject: [Python-Dev] Oberver Pattern In-Reply-To: <001501c1fba7$457e4d60$e861accf@othello>; from python@rcn.com on Tue, May 14, 2002 at 08:27:17PM -0400 References: <200205150009.MAA04723@s454.cosc.canterbury.ac.nz> <001501c1fba7$457e4d60$e861accf@othello> Message-ID: <20020515113445.P28033@prim.han.de> Raymond Hettinger wrote: > From: "Greg Ewing" > > Having thought it through a bit more, you're probably > > right. I just thought it might appeal to Magnus, since > > he seems to be fond of lots of layers of magic, without > > much if any regard for performance... > > FWIW, the early results from my test implementation is > that there is no measurable performance impact if > an observer function has not been attached or if > the observer function is a short C call (perhaps > setting a flag or logging the object id). The only way to > slow it down enough for the timer to notice seems to be > attaching a pure python function to be notified. > > IOW, the observer pattern doesn't slow you down until > you need it. And, if you need it, then you're probably > doing it to save much more time being spent on repeated > checks to see if a change has taken place and to save > programmer effort by de-coupling the objects. Is the functionality complete enough to justify such statements? Not everything is setattr(self, ...) but you have e.g. somelist.append(...) where 'somelist' might be bound to an object which in turn is also bound as an attribute of an observed object. In my understanding objects don't know which names bind it, right? Additionally this is a recursive problem: A has a B-instance has a C instance has a D instance attribute if you observe A and change D, what happens? without this functionality speed/space measurements are obsolete IMO. holger From magnus@hetland.org Wed May 15 13:13:20 2002 From: magnus@hetland.org (Magnus Lie Hetland) Date: Wed, 15 May 2002 14:13:20 +0200 Subject: [Python-Dev] Oberver Pattern In-Reply-To: <05a501c1fb63$63fb9250$0900a8c0@spiff>; from fredrik@pythonware.com on Tue, May 14, 2002 at 06:21:22PM +0200 References: <009e01c1f84f$bd75bfc0$89b53bd0@othello> <200205122355.LAA04466@s454.cosc.canterbury.ac.nz> <20020514174433.A4625@idi.ntnu.no> <05a501c1fb63$63fb9250$0900a8c0@spiff> Message-ID: <20020515141320.H7799@idi.ntnu.no> Fredrik Lundh : > > Magnus wrote: > > > Greg Ewing : > > [snip] > > > This could be very useful for GUIs, too, for implementing > > > model-view relationships. The anygui people would be very > > > interested, I'm sure. > > > > Indeed! > > have you actually built large-scale UI applications using a > one-size-fits-all observer model, or are you just guessing? Guessing about what? Greg sayd that the anygui people would be very interested, and since I'm the project leader and I'm very interested, I think my reply was quite authoritative . I'm not saying that we'd need it -- but I find it interesting, and would like to try it out as an alternative to the current synchronisation mechanism (where, unless you either rebind an attribute or use a model class you have to manually notify an object that one of its attributes have been modified). The current model seems to work well (although it's still under construction), but built-in language support could make it simpler. Maybe. (And I'm only guessing about that -- you're right. But I would be interested in trying it out.) > in real life, "something happened" is almost entirely useless, > and the other extreme "this and this and this and this and > this happened" isn't much better. to get performance, you > need to think in data model terms... Performance wasn't really my first concern. Here is a (admittedly somewhat contrived) Anygui example: >>> rect = [0, 0, 10, 10] >>> win.geometry = rect # Window automatically resized >>> rect[1] += 10 # Window not resized >>> win.sync('geometry') # Window resized Being able to "listen in" on lists would make the last line unnecessary... But this is all mainly motivated by curiosity on my part. > -- Magnus Lie Hetland The Anygui Project http://hetland.org http://anygui.org From magnus@hetland.org Wed May 15 13:20:31 2002 From: magnus@hetland.org (Magnus Lie Hetland) Date: Wed, 15 May 2002 14:20:31 +0200 Subject: [Python-Dev] Oberver Pattern In-Reply-To: <200205150009.MAA04723@s454.cosc.canterbury.ac.nz>; from greg@cosc.canterbury.ac.nz on Wed, May 15, 2002 at 12:09:14PM +1200 References: <200205150009.MAA04723@s454.cosc.canterbury.ac.nz> Message-ID: <20020515142031.B10116@idi.ntnu.no> Greg Ewing : > > Fredrik Lundh : > > > have you actually built large-scale UI applications using a > > one-size-fits-all observer model, or are you just guessing? > > > > in real life, "something happened" is almost entirely useless, > > and the other extreme "this and this and this and this and > > this happened" isn't much better. > > Having thought it through a bit more, you're probably > right. I just thought it might appeal to Magnus, since > he seems to be fond of lots of layers of magic, without > much if any regard for performance... Heh... Dead on ;) Node that both in this case (and others) I'm mainly interested in trying things out without brusing them off immediately. Several of the more "magical things" we've played with in Anygui have been scrapped. (It's not really that magical at the moment, IMO.) Yes, I'm easily charmed by shiny objects, but it's usually not too hard to talk me out of it :) I guess I'll just have a look at the prototype (of the C level observable) to satisfy my curiosity. -- Magnus Lie Hetland The Anygui Project http://hetland.org http://anygui.org From walter@livinglogic.de Wed May 15 13:25:16 2002 From: walter@livinglogic.de (=?ISO-8859-15?Q?Walter_D=F6rwald?=) Date: Wed, 15 May 2002 14:25:16 +0200 Subject: [Python-Dev] PEP282 and the warnings framework Message-ID: <3CE253AC.8070207@livinglogic.de> I've reread PEP 282 (the logging PEP) and much of what I read reminded me of the warning framework. Both systems are responsible for reporting (or ignoring) messages. Both associate messages with locations in the source where the messages originated and both have a way to configure what to report and what to ignore. So I guess it would make sense to somehow merge the two APIs. And looking at the warning framework suggests a different way of handling log levels/types: Instead of specifying types with integer constants and configuring what to log with a threshold value, we could have a class hierarchy of message types similar to the hierarchy for warning types: class Message(Exeption): pass class InfoMessage(Message): pass class ErrorMessage(InfoMessage): pass class FatalMessage(ErrorMessage): pass This hierarchy is customizable with user defined classes: class ResourceMessage(FatalMessage): pass class DiskFullMessage(ResourceMessage): pass class OutOfMemoryMessage(ResourceMessage): pass Configuration would be similar to the configuration for the warnings framework, i.e. it would be possible to specify which Message classes should be logged and which shouldn't: The filter simply calls isinstance() to check if the message should be logged. Calling the logging function would be similar to calling warnings.warn: You can either pass a message string and a message class or a message instance, i.e. logger.log("disk full", log.DiskFullMessage) or logger.log(log.DiskFullMessage("disk full")) Logging exceptions with this API is then simple and natural: logger.log(ExceptionClass("message")) Does this make sense and if yes, do we still have time to change the PEP/implementation until 2.3? Bye, Walter Dörwald From guido@python.org Wed May 15 15:19:13 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 15 May 2002 10:19:13 -0400 Subject: [Python-Dev] PEP282 and the warnings framework In-Reply-To: Your message of "Wed, 15 May 2002 14:25:16 +0200." <3CE253AC.8070207@livinglogic.de> References: <3CE253AC.8070207@livinglogic.de> Message-ID: <200205151419.g4FEJDl26291@pcp742651pcs.reston01.va.comcast.net> > I've reread PEP 282 (the logging PEP) and much of what I read > reminded me of the warning framework. > > Both systems are responsible for reporting (or ignoring) messages. > Both associate messages with locations in the source where the > messages originated and both have a way to configure what to report > and what to ignore. > > So I guess it would make sense to somehow merge the two APIs. What follows doesn't really seem to *merge* the two APIs, it just picks a clever trick used by the warnings framework and applies it to the logging API. I won't comment on whether this fits the logging API; if the folks who wrote the PEP and the implementation agree that it is a good idea, there will be time to integrate this into Python 2.3. But I want to warn against any attempts to go further and actually *unify* the two APIs (in the sense of designing a single API used for both purposes). IMO the goals of logging and the warnings framework are quite different: the warnings framework is for the benefit of the *programmer*, while the logging framework is for the benefit of the *end user*. While both act at run-time, warnings are typically about code that is doing something that may not be right (such as the import of a deprecated module), while logging is about situations in the "real" world that may need to be brought to the attention of the user (for example a disk filling up). While similar approaches to filtering may apply, in general the destination of warnings and logging messages should not be mixed up. If there were any warnings left in a program by the time it was deployed in the field, as a user I'd want to have a hearty word with the programmer. The program probably should have been tested with -Werror, turning all warnings into errors, or (if it's impossible to silence all warnings for some reason) with -Wignore -- but the programmer better have a darn good excuse in the latter case. --Guido van Rossum (home page: http://www.python.org/~guido/) From pyth@devel.trillke.net Wed May 15 16:43:18 2002 From: pyth@devel.trillke.net (holger krekel) Date: Wed, 15 May 2002 17:43:18 +0200 Subject: [Python-Dev] PEP282 and the warnings framework In-Reply-To: <200205151419.g4FEJDl26291@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Wed, May 15, 2002 at 10:19:13AM -0400 References: <3CE253AC.8070207@livinglogic.de> <200205151419.g4FEJDl26291@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020515174318.S28033@prim.han.de> Guido van Rossum wrote: > > I've reread PEP 282 (the logging PEP) and much of what I read > > reminded me of the warning framework. > > > > Both systems are responsible for reporting (or ignoring) messages. > > Both associate messages with locations in the source where the > > messages originated and both have a way to configure what to report > > and what to ignore. > > > > So I guess it would make sense to somehow merge the two APIs. > > What follows doesn't really seem to *merge* the two APIs, it just > picks a clever trick used by the warnings framework and applies it to > the logging API. > ... > IMO the goals of logging and the warnings framework are quite > different: the warnings framework is for the benefit of the > *programmer*, while the logging framework is for the benefit of the > *end user*. Is this true? e.g. Debug/Warning messages are probably meant for developers while Errors/FatalErrors *may* be directed to the end-user. To whom (programmer/'enduser'/other program) and how messages are routed is a matter of configuration IMO. while merging the two APIs is probably not right the warning-module might well be a 'user' of the logging-module. I like Gerhard's basic suggestion of filtering on types of classes rather than some fixed integers. But this should be commented by the PEP282-authors. regards, holger From guido@python.org Wed May 15 16:52:05 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 15 May 2002 11:52:05 -0400 Subject: [Python-Dev] PEP282 and the warnings framework In-Reply-To: Your message of "Wed, 15 May 2002 17:43:18 +0200." <20020515174318.S28033@prim.han.de> References: <3CE253AC.8070207@livinglogic.de> <200205151419.g4FEJDl26291@pcp742651pcs.reston01.va.comcast.net> <20020515174318.S28033@prim.han.de> Message-ID: <200205151552.g4FFq6626814@pcp742651pcs.reston01.va.comcast.net> [me] > > IMO the goals of logging and the warnings framework are quite > > different: the warnings framework is for the benefit of the > > *programmer*, while the logging framework is for the benefit of the > > *end user*. [Holger] > Is this true? e.g. Debug/Warning messages are probably meant > for developers while Errors/FatalErrors *may* be directed > to the end-user. To whom (programmer/'enduser'/other program) > and how messages are routed is a matter of configuration IMO. You have a point there. I still think they are relevant at different stages of development though. Having just gone through the pain of debugging a largeish distributed application, language warnings are something you deal with immediately during unit testing (I treat them the same as SyntaxErrors), while the logging framework comes into play during functional testing. > while merging the two APIs is probably not right > the warning-module might well be a 'user' of the > logging-module. Good point. You can easily do this: the function warnings.showwarning() is a hook that you are allowed to replace. > I like Gerhard's basic suggestion of filtering on types of > classes rather than some fixed integers. But this should > be commented by the PEP282-authors. --Guido van Rossum (home page: http://www.python.org/~guido/) From kbutler@campuspipeline.com Wed May 15 16:51:26 2002 From: kbutler@campuspipeline.com (Kevin Butler) Date: Wed, 15 May 2002 09:51:26 -0600 Subject: [Python-Dev] PEP282 and the warnings framework References: <20020515141648.9926.76949.Mailman@mail.python.org> Message-ID: <3CE283FE.8000403@campuspipeline.com> wrote: > > And looking at the warning framework suggests a different way > of handling log levels/types: Instead of specifying types > with integer constants and configuring what to log with a > threshold value, we could have a class hierarchy of message > types similar to the hierarchy for warning types: > > class Message(Exeption): > pass > class InfoMessage(Message): > pass > class ErrorMessage(InfoMessage): > pass > class FatalMessage(ErrorMessage): > pass > > This hierarchy is customizable with user defined classes: > > class ResourceMessage(FatalMessage): > pass > class DiskFullMessage(ResourceMessage): > pass > class OutOfMemoryMessage(ResourceMessage): > pass We could, but it would be a mistake. :-) Subclasses should differ in behavior - if they differ only in data, that data should go into attributes instead. (Yes, this is disputable, but it does tend to simplify designs.) The Info/Error/Fatal classes differ only in name (with the name mapping to a 'severity'), making it more appropriate for them to be distinguished by some sort of 'severity' attribute. Similarly for the message subclasses (with a name mapping to 'message type'). Having struggled with a message-hierarchy-based system, I found that: - Programmers tend to avoid creating a new subclass just to log a new message. This results in lots of inappropriate reuse of the existing message classes, or hesitancy to log a message at all, reducing your ability to configure individual sets of messages you're interested in. This is similar to the hesitancy to create a new exception subclass, but seems to be stronger - probably because you never 'catch' the logged messages, so the presence/absence of the class doesn't simplify/complicate other code. In contrast, creating a new Logger (or "Category") instance meets little resistance, and a shift to that model was welcomed by all...except the developer of the message hierarchy logging system. - Some messages are awfully hard to categorize statically: the severity of a FileNotFound message could be FATAL, WARNING, or DEBUG, depending on what file is not found and the error-handling code around the message, so you end up needing multiple FileNotFound message classes. Thus, it is better for the calling code to determine the severity of a message, rather than some static property of the message itself. - Configuring logging by message type is usually less helpful than enabling messages from a specific area of code, so configuring by package or class tends to be more useful. I think the ideal would be to enable logging per thread, based on execution path: 'log all messages from any thread that has Class1.method2 in its stack', but I haven't worked with that yet. kb From fredrik@pythonware.com Wed May 15 17:30:12 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Wed, 15 May 2002 18:30:12 +0200 Subject: [Python-Dev] PEP282 and the warnings framework References: <20020515141648.9926.76949.Mailman@mail.python.org> <3CE283FE.8000403@campuspipeline.com> Message-ID: <008f01c1fc2d$cb3ceca0$ced241d5@hagrid> Kevin Butler wrote: > Subclasses should differ in behavior - if they differ only in data, that data > should go into attributes instead. (Yes, this is disputable, but it does tend > to simplify designs.) have you ever used exceptions in python? From walter@livinglogic.de Wed May 15 17:32:11 2002 From: walter@livinglogic.de (=?ISO-8859-15?Q?Walter_D=F6rwald?=) Date: Wed, 15 May 2002 18:32:11 +0200 Subject: [Python-Dev] PEP282 and the warnings framework References: <3CE253AC.8070207@livinglogic.de> <200205151419.g4FEJDl26291@pcp742651pcs.reston01.va.comcast.net> <20020515174318.S28033@prim.han.de> <200205151552.g4FFq6626814@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CE28D8B.4080109@livinglogic.de> Guido van Rossum wrote: > [...] >>while merging the two APIs is probably not right >>the warning-module might well be a 'user' of the >>logging-module. > > > Good point. You can easily do this: the function > warnings.showwarning() is a hook that you are allowed to replace. Yes, but to configure the filter for warnings you have to use warnings.filterwarnings() with its API and for filtering logging you're using a different API. Conceptually they are doing the same, so they should have similar APIs. Bye, Walter Dörwald From fredrik@pythonware.com Wed May 15 17:42:42 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Wed, 15 May 2002 18:42:42 +0200 Subject: [Python-Dev] Oberver Pattern References: <009e01c1f84f$bd75bfc0$89b53bd0@othello> <200205122355.LAA04466@s454.cosc.canterbury.ac.nz> <20020514174433.A4625@idi.ntnu.no> <05a501c1fb63$63fb9250$0900a8c0@spiff> <20020515141320.H7799@idi.ntnu.no> Message-ID: <009b01c1fc2f$920b2170$ced241d5@hagrid> magnus wrote: > > > This could be very useful for GUIs, too, for implementing > > > model-view relationships. The anygui people would be very > > > interested, I'm sure. > > > > Indeed! > > > > have you actually built large-scale UI applications using a > > one-size-fits-all observer model, or are you just guessing? > > Guessing about what? the "very useful for implementing model-view relationships" part, of course. since you didn't reply to the "large-scale" part, and gave an example that doesn't really have much to do with model-view design (unless you're writing a GUI builder, perhaps) I assume you're still mostly guessing. if you want something closer to real-life, consider how well a "standard observer" would deal with the following example: # load the model dom = xml.dom.minidom.parse(filename) # create a view mywidget = MyDisplayView() mywidget.show(dom) # modify the model for element in dom.getElementsByTagName("sometag"): ... From tim@zope.com Wed May 15 18:05:16 2002 From: tim@zope.com (Tim Peters) Date: Wed, 15 May 2002 13:05:16 -0400 Subject: [Python-Dev] Hundreds of warnings Message-ID: Some recent patch to Python.h moved its include of pyport.h above its include of limits.h. As a result, pyport.h defines LONG_MIN and LONG_MAX itself, and then limit.h's attempt to define them correctly leads to legit warnings about macro redefinitions. This happens for every file in the project, of course, since everyone includes Python.h. Don't know more and don't want to -- whatever this change was trying to solve needs to be done in a different way. From guido@python.org Wed May 15 18:21:06 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 15 May 2002 13:21:06 -0400 Subject: [Python-Dev] Hundreds of warnings In-Reply-To: Your message of "Wed, 15 May 2002 13:05:16 EDT." References: Message-ID: <200205151721.g4FHL8C29290@pcp742651pcs.reston01.va.comcast.net> > Some recent patch to Python.h moved its include of pyport.h above its > include of limits.h. As a result, pyport.h defines LONG_MIN and LONG_MAX > itself, and then limit.h's attempt to define them correctly leads to legit > warnings about macro redefinitions. This happens for every file in the > project, of course, since everyone includes Python.h. > > Don't know more and don't want to -- whatever this change was trying > to solve needs to be done in a different way. Looks like Jason Tishler was a bit rash. I specifically told him: | Please use the patch manager to request review until we start telling | you to check things in directly. :-) when he asked about our commit rules. --Guido van Rossum (home page: http://www.python.org/~guido/) From nas@python.ca Wed May 15 18:29:44 2002 From: nas@python.ca (Neil Schemenauer) Date: Wed, 15 May 2002 10:29:44 -0700 Subject: [Python-Dev] Hundreds of warnings In-Reply-To: ; from tim@zope.com on Wed, May 15, 2002 at 01:05:16PM -0400 References: Message-ID: <20020515102944.A21134@glacier.arctrix.com> Tim Peters wrote: > Some recent patch to Python.h moved its include of pyport.h above its > include of limits.h. The change was 2.47 by Jason Tishler. The log message is: Patch #555929: Cygwin AH_BOTTOM cleanup patch This patch complies with the following request found near the top of configure.in: # This is for stuff that absolutely must end up in pyconfig.h. # Please use pyport.h instead, if possible. I tested this patch under Cygwin, Win32, and Red Hat Linux. Python built and ran successfully on each of these platforms. Another reason for me to work on a tinderbox-like system for Python. Neil From martin@v.loewis.de Wed May 15 19:11:52 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 15 May 2002 20:11:52 +0200 Subject: [Python-Dev] Hundreds of warnings In-Reply-To: <200205151721.g4FHL8C29290@pcp742651pcs.reston01.va.comcast.net> References: <200205151721.g4FHL8C29290@pcp742651pcs.reston01.va.comcast.net> Message-ID: Guido van Rossum writes: > Looks like Jason Tishler was a bit rash. I specifically told him: > > | Please use the patch manager to request review until we start telling > | you to check things in directly. :-) > > when he asked about our commit rules. And I'm guilty of approving the patch. Regards, Martin From pyth@devel.trillke.net Wed May 15 19:13:42 2002 From: pyth@devel.trillke.net (holger krekel) Date: Wed, 15 May 2002 20:13:42 +0200 Subject: [Python-Dev] PEP282 and the warnings framework In-Reply-To: <3CE283FE.8000403@campuspipeline.com>; from kbutler@campuspipeline.com on Wed, May 15, 2002 at 09:51:26AM -0600 References: <20020515141648.9926.76949.Mailman@mail.python.org> <3CE283FE.8000403@campuspipeline.com> Message-ID: <20020515201342.T28033@prim.han.de> Kevin Butler wrote: > - Programmers tend to avoid creating a new subclass just to log a new message. true if you are using java or c++ but not neccesarily for python. subclassing is done quickly and without error-prone noise. But anyway, who says that you should create a new subclass for a new message? > This results in lots of inappropriate reuse of the existing message classes, > or hesitancy to log a message at all, reducing your ability to configure > individual sets of messages you're interested in. This is similar to the > hesitancy to create a new exception subclass, but seems to be stronger - > probably because you never 'catch' the logged messages, so the > presence/absence of the class doesn't simplify/complicate other code. Subclassing of exceptions is often not neccessary for simple applications because python provides a nice hierarchic preset of Exceptions. For larger applications you *must* define categories of Errors. Or you decide to pay later :-) IMO the situation with logging-messages might be very similar. > In contrast, creating a new Logger (or "Category") instance meets little > resistance, and a shift to that model was welcomed by all...except the > developer of the message hierarchy logging system. having a Logger-instance which processes messages is quite orthogonal to how the messages are categorized. > - Some messages are awfully hard to categorize statically: the severity of a > FileNotFound message could be FATAL, WARNING, or DEBUG, depending on what file > is not found and the error-handling code around the message, yes. > so you end up needing multiple FileNotFound message classes. Why? First of all, you should catch e.g. an IOError close to where it occurs because the higher up the request-stack the less you know what to do (usually). The more specific the error the higher up you can catch it (usually) and still know what to do. With 'typed messages' you can achieve the same convenience for the logging api e.g. log.debug(x) maps to log.log(DebugMessage(x)) log.error(x) maps to log.log(ErrorMessage(x)) ... log.log(x) maps to log.log(log.DebugMessage(x)) if not isinstance(x,Message) (The base Message class should accept Exceptions and Strings IMO.) > Thus, it is better for the > calling code to determine the severity of a message, rather than some static > property of the message itself. With the above definitions you can interprete and dispatch error conditions like so: except IOError,e: if e.errno in (a,b,c): log.debug(e) elif e.errno in (x,y,z): log.error(e) raise Where is the advantage if you use integer log levels? Am i missing something? > - Configuring logging by message type is usually less helpful than enabling > messages from a specific area of code, so configuring by package or class > tends to be more useful. much like with the warning module there can be more than one parameter for filtering. This doesn't touch the question how messages should be categorized (by integer or by class-type). regards, holger From nas@python.ca Wed May 15 19:26:01 2002 From: nas@python.ca (Neil Schemenauer) Date: Wed, 15 May 2002 11:26:01 -0700 Subject: [Python-Dev] automated testing system [was: Hundreds of warnings] In-Reply-To: <3CE2A31D.1080005@ActiveState.com>; from DavidA@ActiveState.com on Wed, May 15, 2002 at 11:04:13AM -0700 References: <20020515102944.A21134@glacier.arctrix.com> <3CE2A31D.1080005@ActiveState.com> Message-ID: <20020515112601.A21436@glacier.arctrix.com> David Ascher wrote: > Are you really doing that? Still thinking about it. I haven't written a line of code yet. :-( I was originally was thinking of trying to get a hold of lots of weird hardware and operating systems and setting up a little test farm. Emutators like bochs and vmware would be useful too. I've decided that's not the right solution though. Now I want to build a distributed system. There would be some cross-platform software that people could install on machines they have laying around. The more and weirder the better. The system would receive signals to start a build and regression test and send the results to some collector. The signal could be based on the CVS checkin list. The collector could generate summaries and post them to python-dev. Neil From martin@v.loewis.de Wed May 15 19:25:32 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 15 May 2002 20:25:32 +0200 Subject: [Python-Dev] Hundreds of warnings In-Reply-To: References: Message-ID: "Tim Peters" writes: > Don't know more and don't want to -- whatever this change was trying > to solve needs to be done in a different way. Ok, I've backed the patch out. Martin From kbutler@campuspipeline.com Wed May 15 19:32:33 2002 From: kbutler@campuspipeline.com (Kevin Butler) Date: Wed, 15 May 2002 12:32:33 -0600 Subject: [Python-Dev] PEP282 and the warnings framework References: <20020515141648.9926.76949.Mailman@mail.python.org> <3CE283FE.8000403@campuspipeline.com> <008f01c1fc2d$cb3ceca0$ced241d5@hagrid> Message-ID: <3CE2A9C1.3050003@campuspipeline.com> Fredrik Lundh wrote: > Kevin Butler wrote: > > >>Subclasses should differ in behavior - if they differ only in data, that data >>should go into attributes instead. (Yes, this is disputable, but it does tend >>to simplify designs.) > > have you ever used exceptions in python? Yes. Have you ever noticed how developers tend to re-use an existing exception class even when they should define a new subclass? :-) Have you ever had to map a 'return code' into an exception hierarchy? Ever just wrapped the data value as an attribute instead, pushing the switching behavior off to the exception handlers? Exception subclasses differing only in class name exist because the language provides specific support for switching behavior based on the class of a caught exception. (Python's string-based exceptions separate that behavioral switching from the exception class hierarchy, but their flat namespace is too constraining.) This specific language support pushes the design tradeoff away from the simplicity of attributes toward the more complex subclass hierarchy, but often not enough to justify a full mapping of all return codes into classes. In logging, where there is no language support motivation for additional class creation, it makes more sense to avoid the subclass creation altogether. Creating a logger instance with a category value and then sending it lots of different types messages is easier than creating multiple Message subclasses for different types of messages and sending them to a single logger instance. Since you usually want to turn on all message types in a specific area of code anyway, the single category instance works nicely. Besides, if you want to build a message-based system on top of a category-based system, it is easy - mostly just the work you'd have to do anyway (not tested, yada-yada): class Message: logger = Logger( "Message" ) priority = INFO msg = "Message" def __init__( self, *args ): self.args = args def log( self ): self.logger.log( self.priority, self.msg, self.args ) Usage: class MyMessage( Message ): logger = Logger( "Message.MyMessage" ) priority = WARNING msg = "MyMessage: Your %s has %s" MyMessage( "dog", "fleas" ).log() kb From kbutler@campuspipeline.com Wed May 15 19:53:09 2002 From: kbutler@campuspipeline.com (Kevin Butler) Date: Wed, 15 May 2002 12:53:09 -0600 Subject: [Python-Dev] PEP282 and the warnings framework References: <20020515141648.9926.76949.Mailman@mail.python.org> <3CE283FE.8000403@campuspipeline.com> <20020515201342.T28033@prim.han.de> Message-ID: <3CE2AE95.4030900@campuspipeline.com> holger krekel wrote: > Kevin Butler wrote: > >>In contrast, creating a new Logger (or "Category") instance meets little >>resistance, and a shift to that model was welcomed by all...except the >>developer of the message hierarchy logging system. > > > having a Logger-instance which processes messages is quite orthogonal > to how the messages are categorized. ... > Where is the advantage if you use integer log levels? > Am i missing something? I think you're missing something. The proposed API (PEP 282) provides integer priorities (log/error/debug/etc.), and hierarchical categorization by the "logger" instance. The logger instance contains the category (I forget if we call Logger constructors or call a factory method): logger = Logger( "pil.image.jpeg" ) logger2 = Logger( "pil.image.gif" ) logger.debug( "My JPEG Image PIL Debug message" ) logger2.error( "My GIF image PIL error message" ) The logging of theses message is configurable based on the "pil" category, the "pil.image" category, the "pil.image.jpeg"/"pil.image.gif" category, and their priority. This means if I am debugging something in PIL images, I can enable all log messages in the "image" category. Or I could enable just everything having to do with JPEG images, and ignore all that proprietary GIF stuff. To get similar functionality with a message-based hierarchy, I'd have to define separate "PilImageJpegMessage" and "PilImageGifMessage" classes, and have them both extend a PilImageMessage class. > (The base Message class should accept Exceptions and Strings IMO.) FWIW, once you allow logging 'string'-type messages, most logged messages will be a string (especially debug messages), because it is much easier than creating an instance of some other clsas. Thus, if your categorization is based on the class of the logged message, the "string" category gets very large... > This doesn't touch the question how messages > should be categorized (by integer or by class-type). The question, as I understood it, was whether to categorize by message-class-name, or by logger-instance-category. Although the 'message-class' hierarchy appears attractive, logger-instance categorization has been more useful to me. kb From python@rcn.com Wed May 15 20:32:07 2002 From: python@rcn.com (Raymond Hettinger) Date: Wed, 15 May 2002 15:32:07 -0400 Subject: [Python-Dev] automated testing system [was: Hundreds of warnings] References: <20020515102944.A21134@glacier.arctrix.com> <3CE2A31D.1080005@ActiveState.com> <20020515112601.A21436@glacier.arctrix.com> Message-ID: <001701c1fc47$343ee820$57b63bd0@othello> From: "Neil Schemenauer" > Still thinking about it. I haven't written a line of code yet. :-( I > was originally was thinking of trying to get a hold of lots of weird > hardware and operating systems and setting up a little test farm. Laura Creighton is also working on a test farm (though for different purposes). Maybe some of the effort can be shared. > Now I want to build a distributed system. There would be some > cross-platform software that people could install on machines they have > laying around. The more and weirder the better. Windows is pretty weird. But I assume that's not what you meant . Raymond Hettinger From magnus@hetland.org Wed May 15 21:40:52 2002 From: magnus@hetland.org (Magnus Lie Hetland) Date: Wed, 15 May 2002 22:40:52 +0200 Subject: [Python-Dev] Oberver Pattern In-Reply-To: <009b01c1fc2f$920b2170$ced241d5@hagrid>; from fredrik@pythonware.com on Wed, May 15, 2002 at 06:42:42PM +0200 References: <009e01c1f84f$bd75bfc0$89b53bd0@othello> <200205122355.LAA04466@s454.cosc.canterbury.ac.nz> <20020514174433.A4625@idi.ntnu.no> <05a501c1fb63$63fb9250$0900a8c0@spiff> <20020515141320.H7799@idi.ntnu.no> <009b01c1fc2f$920b2170$ced241d5@hagrid> Message-ID: <20020515224052.A9272@idi.ntnu.no> Fredrik Lundh : > > magnus wrote: > > > > > This could be very useful for GUIs, too, for implementing > > > > model-view relationships. The anygui people would be very > > > > interested, I'm sure. > > > > > > Indeed! > > > > > > have you actually built large-scale UI applications using a > > > one-size-fits-all observer model, or are you just guessing? > > > > Guessing about what? > > the "very useful for implementing model-view relationships" > part, of course. > > since you didn't reply to the "large-scale" part, and gave an > example that doesn't really have much to do with model-view > design (unless you're writing a GUI builder, perhaps) I assume > you're still mostly guessing. Hey - lighten up... If you read my answer, I was replying to the "The anygui people would be very interested, I'm sure" part. I was interested. The reason the Anygui MVC architecture is the way it is is mainly to avoid the need for notification in every little modification; with observable built-ins this sort of thing would have to be handled in another manner, of course. I'm not sure exactly how this should be done, but, as I said, I am interested. > if you want something closer to real-life, consider how well a > "standard observer" would deal with the following example: > > # load the model > dom = xml.dom.minidom.parse(filename) > > # create a view > mywidget = MyDisplayView() > mywidget.show(dom) > > # modify the model > for element in dom.getElementsByTagName("sometag"): > ... I'm not sure I see your point here (although I'm sure it's entirely my fault). Are you saying that there would be a problem because you would get too many notifications? If so, I don't see how that's inherent in being able to observe values belonging to built-in types (which is what I'm interested in). Oh, well. I guess it's not going to happen anyway, so this discussion is probably going nowhere. > -- Magnus Lie Hetland The Anygui Project http://hetland.org http://anygui.org From tim.one@comcast.net Wed May 15 22:48:34 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 15 May 2002 17:48:34 -0400 Subject: [Python-Dev] Hundreds of warnings In-Reply-To: Message-ID: I suppose there's no need for pyport.h to define LONG_MIN or LONG_MAX anymore, since C89 requires that limits.h define those. LONG_BIT is an extension to C89, but Python only uses LONG_BIT in two places now (it used to use it more; I nuke these when I can), and those could easily enough be rewritten; e.g., instead of if (b >= LONG_BIT) we could do if (b >= 8 * SIZEOF_LONG) Quite a while ago I added this to pyport.h, hoping to do that someday: #if LONG_BIT != 8 * SIZEOF_LONG /* 04-Oct-2000 LONG_BIT is apparently (mis)defined as 64 on some recent * 32-bit platforms using gcc. We try to catch that here at compile-time * rather than waiting for integer multiplication to trigger bogus * overflows. */ #error "LONG_BIT definition appears wrong for platform (bad gcc/glibc config?)." #endif [Note that integer multiplication no longer uses LONG_BIT.] So if there's any platform on which LONG_BIT isn't redundant, nobody has tried to compile Python on it since October of 2000. From pyth@devel.trillke.net Wed May 15 23:33:21 2002 From: pyth@devel.trillke.net (holger krekel) Date: Thu, 16 May 2002 00:33:21 +0200 Subject: [Python-Dev] PEP282 and the warnings framework In-Reply-To: <3CE2AE95.4030900@campuspipeline.com>; from kbutler@campuspipeline.com on Wed, May 15, 2002 at 12:53:09PM -0600 References: <20020515141648.9926.76949.Mailman@mail.python.org> <3CE283FE.8000403@campuspipeline.com> <20020515201342.T28033@prim.han.de> <3CE2AE95.4030900@campuspipeline.com> Message-ID: <20020516003321.U28033@prim.han.de> Kevin Butler wrote: > The logger instance contains the category (I forget if we call Logger > constructors or call a factory method): > > logger = Logger( "pil.image.jpeg" ) > logger2 = Logger( "pil.image.gif" ) > > logger.debug( "My JPEG Image PIL Debug message" ) > logger2.error( "My GIF image PIL error message" ) > The logging of theses message is configurable based on the "pil" category, the > "pil.image" category, the "pil.image.jpeg"/"pil.image.gif" category, and their > priority. i don't think you read my last posting. This logging-instance behaviour is *fine* but it is ortoghonal to whether an integer or a type characterizes a message. You example would work *without* change. > To get similar functionality with a message-based hierarchy, I'd have to > define separate "PilImageJpegMessage" and "PilImageGifMessage" classes, and > have them both extend a PilImageMessage class. No, far from it. > > (The base Message class should accept Exceptions and Strings IMO.) > > FWIW, once you allow logging 'string'-type messages, most logged messages will > be a string (especially debug messages), because it is much easier than > creating an instance of some other clsas. Thus, if your categorization is > based on the class of the logged message, the "string" category gets very large... NO. Only because a typed message accepts a string in its constructor does not make it a string. Or since when are all python exceptions in the 'string' category? > > This doesn't touch the question how messages > > should be categorized (by integer or by class-type). > > The question, as I understood it, was whether to categorize by > message-class-name, or by logger-instance-category. no. it is whether Messages are characterized either by string/integer or Message-instance/type. > Although the 'message-class' hierarchy appears attractive, logger-instance > categorization has been more useful to me. logger-instance filtering/categorization DOES NOT suffer. But you gain more control e.g. how expensive your logging gets. If you want to e.g. trace the state of certain objects you define a TraceStateMessage and defer the actual (expensive) introspection until __str__ is called on the message [*]. regards, holger --- [*] In basic code words: class TraceStateMessage(DebugMessage): def __init__(self, obj): self.obj=obj def __str__(self): """ this is called only if someone is interested in the message """ names = filter(lambda x: not x.startswith('_'), dir(self.obj)) d = map(lambda x: (x,getattr(self.obj, x)), names) return "%s, attrs: %s" %(type(obj),d) log.traceobj = lambda self,obj: self.log(TraceStateMessage(obj)) Now you can can do log.traceobj(myobj) easy & straight forward. Doesn't touch the categorization configuration of 'log' at all: if it wants to see DebugMessages then introspection is done, otherwise overhead is minimal. From David Abrahams" Message-ID: <075a01c1fc65$13e00730$6601a8c0@boostconsulting.com> ----- Original Message ----- From: "Tim Peters" > Quite a while ago I added this to pyport.h, hoping to do that someday: > > #if LONG_BIT != 8 * SIZEOF_LONG > /* 04-Oct-2000 LONG_BIT is apparently (mis)defined as 64 on some recent > * 32-bit platforms using gcc. We try to catch that here at compile-time > * rather than waiting for integer multiplication to trigger bogus > * overflows. > */ > #error "LONG_BIT definition appears wrong for platform (bad gcc/glibc > config?)." > #endif ...which explains why I have to # define SIZEOF_LONG 4 explicitly to build any Cygwin extension modules. From vinay_sajip@red-dove.com Thu May 16 00:35:49 2002 From: vinay_sajip@red-dove.com (Vinay Sajip) Date: Thu, 16 May 2002 00:35:49 +0100 Subject: [Python-Dev] PEP282 and the warnings framework Message-ID: <012301c1fc69$3fe7b540$652b6992@alpha> I've got some comments on Walter's suggestion and the various posts responding to it. Most recently, both Holger Krekel and Kevin Butler have made what I think of as reasonable comments. FWIW here are my thoughts: A call to the logging API is essentially a statement by the developer about an "event of interest". The difference between Walter's idea and the current state of PEP282 (and my logging module) focuses on two points of interest: "what happened?" and "how important is it?". In both cases, the "what happened?" part is codified by a message (e.g. "disk full"). The "how important is it?" part is codified in the case of PEP282/logging.py by an integer, and in the case of Walter's idea by a class. This, as I see it, is the core difference. Given that an implementation already exists and is in use by quite a few people, I don't think it's right to make changes of a fundamental nature which would require users to rewrite their code, unless the reason is a *really* good one. (the module is still in an "alpha" stage so I am not dead against making changes of this type; I just need to know what the justification is). So far, I am not sure what benefit switching from the status quo of integer-based levels to class-based levels will give. Certainly, the idea of relative levels of importance of events must be preserved, since severity-based filtering is a key function of logging. This could be in theory be done using the inheritance hierarchy of message classes, but I don't think this is particularly elegant here (though it's fine in "except" clauses). And why bother, when integers do the job quite well? And if you use some level attribute in message classes, what has really been gained by using classes? There is also sometimes a possibility that a user-defined handler or filter might, based on the application context and the configuration selected by a developer, choose to change the severity of an event dynamically. This is fine with integers, less so with classes. (I know you can change the class of an instance dynamically, but I'm sure that instances of assignments to __class__ should not be multiplied beyond necessity. Other approaches such as wrapping exceptions are also possible, but in many cases would just obscure the developer's intention, and seeming like trying to "get around the system".) It's not clear to me that class-based levels are a benefit over integer-based levels. The existing implementation, using integer-based levels, seems to offer all the flexibility mentioned by Holger in his post. For example, expensive formatting operations are deferred until (and only if) needed. Factory methods are in place where necessary to facilitate creating user-defined Loggers, Handlers, Filters and LogRecords (the analogue of the Message class). The present design also makes it much easier for users to define their own levels, which can completely slot in and replace the default levels provided (these are based on log4j's experience and strongly recommended, but sometimes users just have to go their own way - I don't like to be too prescriptive in this kind of area). One more limitation of the class-based level idea surfaces from considering how Python applications play in the wider world. Integer levels map quite easily to Unix syslog, NT event log levels, etc. etc. It's easier to pickle logging events and send them to a remote audience if they're integer based, so you don't need a guarantee that the receiving system has the same classes loaded as the sending system. You can send logging events to non-Python SOAP and HTTP receivers which may not be able to deal with severities in terms of classes. Why, you could even interoperate with log4j! Certainly in the heterogeneous corporate world, interoperability is very important beyond the language level. We don't want to put roadblocks to interoperability which force developers to roll their own solutions. In log4j, levels *are* classes, but they are not used in the same way as Walter/Holger have advocated - they are not part of the exception hierarchy, nor are they treated analogously. So, while not wishing to sound abrupt, and respectful of the sentiments which led Walter to make his suggestion, I say -1 invoking KISS, YAGNI and inconvenience to existing users. I discussed with Trent Mick the gist of my argument, and he is in broad agreement with the views expressed above. Regards Vinay From jeremy@zope.com Thu May 16 00:40:31 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Wed, 15 May 2002 19:40:31 -0400 Subject: [Python-Dev] PEP282 and the warnings framework In-Reply-To: <012301c1fc69$3fe7b540$652b6992@alpha> References: <012301c1fc69$3fe7b540$652b6992@alpha> Message-ID: <15586.61935.395177.224459@slothrop.zope.com> Integer levels make a lot of sense to me. Since the levels are primarily used to order the different classes of log messages so that they can be filtered, it seems spot on to use an object, like integers, that have a natural and intuitive order. Jeremy From skip@pobox.com Thu May 16 00:45:10 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 15 May 2002 18:45:10 -0500 Subject: [Python-Dev] automated testing system [was: Hundreds of warnings] In-Reply-To: <20020515112601.A21436@glacier.arctrix.com> References: <20020515102944.A21134@glacier.arctrix.com> <3CE2A31D.1080005@ActiveState.com> <20020515112601.A21436@glacier.arctrix.com> Message-ID: <15586.62214.689426.622713@12-248-41-177.client.attbi.com> Neil> Now I want to build a distributed system. There would be some Neil> cross-platform software that people could install on machines they Neil> have laying around. The more and weirder the better. The system Neil> would receive signals to start a build and regression test and Neil> send the results to some collector. The signal could be based on Neil> the CVS checkin list. The collector could generate summaries and Neil> post them to python-dev. I created a little XML-RPC server sometime ago to catch instruction set frequency information (-DDXPAIRs in ceval.c, etc) and store them in a database. I think this would be the ideal sort of interface for a regression test reporting system. The only change I had to make to my application to make it "aware" was def send_instruction_counts(appname, email): if not hasattr(sys, "getdxp"): return dxpserver = xmlrpclib.ServerProxy("http://manatee.mojam.com:7304") dxpserver.add_dx_info(appname, email, sys.version_info[:3], rle(sys.getdxp())) atexit.register(send_instruction_counts, "musi-cal", "skip@pobox.com") As for posting to python-dev I think the results would just get lost or would be so voluminous after awhile that people would quickly get desensitized to them. I'd rather see a tabular summary on a website somewhere with some color coding (read/yellow/green) based the number or percentage of tests that passed without problem. If necessary, a post to python-dev could remind people about the results page with perhaps a simple two- or three-line summary identifying the overall change from the previous day's or week's results, something like: 3 new configurations -13 green (no fails) +13 yellow (< 3 fails) + 3 red (>= 3 fails) I'd be willing to implement the recording and reporting stuff if someone wants to help specify the API. Skip From greg@cosc.canterbury.ac.nz Thu May 16 00:48:20 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Thu, 16 May 2002 11:48:20 +1200 (NZST) Subject: [Python-Dev] Oberver Pattern In-Reply-To: Message-ID: <200205152348.LAA04898@s454.cosc.canterbury.ac.nz> Just van Rossum : > There's one application for which I've occasionally wished there was a > deeply integrated observer API: object browsers in an IDE. And setting watchpoints in a debugger, etc. There's a hook for code tracing, so why not a hook for data tracing too? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From guido@python.org Thu May 16 00:59:23 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 15 May 2002 19:59:23 -0400 Subject: [Python-Dev] PEP282 and the warnings framework In-Reply-To: Your message of "Thu, 16 May 2002 00:35:49 BST." <012301c1fc69$3fe7b540$652b6992@alpha> References: <012301c1fc69$3fe7b540$652b6992@alpha> Message-ID: <200205152359.g4FNxNO30642@pcp742651pcs.reston01.va.comcast.net> > Given that an implementation already exists and is in use by quite a > few people, I don't think it's right to make changes of a > fundamental nature which would require users to rewrite their code, > unless the reason is a *really* good one. (the module is still in an > "alpha" stage so I am not dead against making changes of this type; > I just need to know what the justification is). That's not the reason I'd like to hear not to consider this change. :-) There's an anecdote (not sure if it's true, but I heard it told a long time ago as a true story) that the author of the original Unix v7 "Make" utility was asked to change the dreaded dependence on tabs, and refused because there were already ten users. There's also the argument used occasionally by Bjarne Stroustrup that the number of programs that will be written is much larger than the number of programs already written, and that you should not sacrifice the convenience of those many future programmers for that of the (relatively) few current users. That said, I'm pretty neutral on this particular issue, and trust you to make the right decision. Just don't be biased by what you've already implemented. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu May 16 01:00:03 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 15 May 2002 20:00:03 -0400 Subject: [Python-Dev] PEP282 and the warnings framework In-Reply-To: Your message of "Wed, 15 May 2002 19:40:31 EDT." <15586.61935.395177.224459@slothrop.zope.com> References: <012301c1fc69$3fe7b540$652b6992@alpha> <15586.61935.395177.224459@slothrop.zope.com> Message-ID: <200205160000.g4G003430655@pcp742651pcs.reston01.va.comcast.net> > Integer levels make a lot of sense to me. Since the levels are > primarily used to order the different classes of log messages so that > they can be filtered, it seems spot on to use an object, like > integers, that have a natural and intuitive order. Especially when the particular levels are -300, -200, -100, 0, 100, 200, and 300. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From kbutler@campuspipeline.com Thu May 16 01:29:05 2002 From: kbutler@campuspipeline.com (Kevin Butler) Date: Wed, 15 May 2002 18:29:05 -0600 Subject: [Python-Dev] PEP282 and the warnings framework References: <20020515141648.9926.76949.Mailman@mail.python.org> <3CE283FE.8000403@campuspipeline.com> <20020515201342.T28033@prim.han.de> <3CE2AE95.4030900@campuspipeline.com> <20020516003321.U28033@prim.han.de> Message-ID: <3CE2FD51.2060301@campuspipeline.com> holger krekel wrote: > Kevin Butler wrote: > i don't think you read my last posting. This logging-instance behaviour > is *fine* but it is ortoghonal to whether an integer or a type characterizes > a message. You example would work *without* change. I read it, but I obviously didn't understand it as you meant it. :-) *deep thought* I don't see how it would work without drastically increasing the configuration complexity. log = logging.getLogger( "pil.image.gif" ) log.log( IOError( reason )) Currently, I can configure the hierachy pil...image...gif, and I can specify the integer priority level below which I won't log a message in that category. If you add the hierarchy IOError...EnvironmentError...StandardError...Exception, how do you decide whether to log the message or not? - Do you have two separate hierarchies, making each message enabled/disabled for all loggers? This doesn't seem to give you much flexibility (IOError enabled everywhere). - Or does each logger configuration have a hierarchy of message types, and each message type can be enabled/disabled for each logger? (This seems to require too much configuration.) Some other approach I'm not seeing? >>FWIW, once you allow logging 'string'-type messages, most logged messages will >>be a string (especially debug messages), because it is much easier than >>creating an instance of some other clsas. Thus, if your categorization is >>based on the class of the logged message, the "string" category gets very large... > > NO. Only because a typed message accepts a string in its constructor > does not make it a string. Or since when are all python exceptions > in the 'string' category? If I have a choice between writing: log.debug( "HOME environment variable set to non-existent directory %s", homedir ) and: log.debug( ConfigurationException( "variable set to non-existent directory", "HOME", homedir )) I'm most likely to just use the string option - and if I don't have a 'ConfigurationException' class & constructor that exactly matches what I need, I'm even more likely to just log the string. > But you gain more control e.g. how expensive your logging gets. > If you want to e.g. trace the state of certain objects > you define a TraceStateMessage and defer the actual > (expensive) introspection until __str__ is called on the > message [*]. The API discussions already allow deferred rendering using *args, specifically to avoid expensive rendering that isn't needed. http://www.red-dove.com/python_logging.html You can get approximately the TraceStateMessage example you provided by just doing the following (formatting differs, yada-yada-yada): trace = logging.getLogger( "trace" ) trace.debug( "%s, attrs: %s", type( obj ), obj.__dict__ ) And you can get /exactly/ the TraceStateMessage example, including formatting, by: # # TraceStateMessage class you defined # trace = logging.getLogger( "trace" ) trace.debug( "%s", TraceStateMessage( obj )) Configuring on both MessageType and Logger (category) seems like unneeded bloat to me. kb From pyth@devel.trillke.net Thu May 16 01:54:08 2002 From: pyth@devel.trillke.net (holger krekel) Date: Thu, 16 May 2002 02:54:08 +0200 Subject: [Python-Dev] PEP282 and the warnings framework In-Reply-To: <012301c1fc69$3fe7b540$652b6992@alpha>; from vinay_sajip@red-dove.com on Thu, May 16, 2002 at 12:35:49AM +0100 References: <012301c1fc69$3fe7b540$652b6992@alpha> Message-ID: <20020516025408.V28033@prim.han.de> Vinay Sajip wrote: > ... > So far, I am not sure what benefit switching from the status quo of > integer-based levels to class-based levels will give. Certainly, the idea of > relative levels of importance of events must be preserved, since > severity-based filtering is a key function of logging. first off, the DebugMessage/InfoMessage/... could have a class-level severity property (integer) if one really wants to have it for compatibility etc. this leaves the question how a Message-instance/type is more powerful than a tuple(string/integer). Some thoughts regarding Message-instances/types: - the convenient logging API and categorization features need *not* be changed largely (but the signature of the log method should/might be changed) - beeing forced to totally order all types of messages can be difficult. You may want e.g. to log 'TimeoutMessages' no matter what the current integer threshold might be. why not something like: loginstance.accept(TimeoutMessage) - Message instances can defer the *construction* of a string until it is really needed. (see my other posting for a code example). This is more effective if you choose to ignore certain messages. This has *nothing* to do with formatting. - customized messages are powerful and allow more advanced filtering than with strings, especially for large apps. a connection-logger could choose to ignore a TimeoutMessage(socket) based on the target address of the socket. doing this with strings is hard (and perlish:-) - a SocketMessage-class may hold a weakref-list of sockets allowing to log 'sock<1>', 'sock<2>' instead of addresses which are hard to read. stuff like this is *very* convenient. Again there is hardly any performance penalty if SocketMessages are ignored. Maybe the categorization features of the logging API make up for some of the (relative) inflexibilities of string/integer messages. But what do you really gain by restricting a message to a (string,integer) taking my arguments into account? regards, holger From pyth@devel.trillke.net Thu May 16 02:48:24 2002 From: pyth@devel.trillke.net (holger krekel) Date: Thu, 16 May 2002 03:48:24 +0200 Subject: [Python-Dev] PEP282 and the warnings framework In-Reply-To: <3CE2FD51.2060301@campuspipeline.com>; from kbutler@campuspipeline.com on Wed, May 15, 2002 at 06:29:05PM -0600 References: <20020515141648.9926.76949.Mailman@mail.python.org> <3CE283FE.8000403@campuspipeline.com> <20020515201342.T28033@prim.han.de> <3CE2AE95.4030900@campuspipeline.com> <20020516003321.U28033@prim.han.de> <3CE2FD51.2060301@campuspipeline.com> Message-ID: <20020516034824.W28033@prim.han.de> Kevin Butler wrote: > I don't see how it would work without drastically increasing the configuration > complexity. > > log = logging.getLogger( "pil.image.gif" ) > log.log( IOError( reason )) > Currently, I can configure the hierachy pil...image...gif, and I can specify > the integer priority level below which I won't log a message in that category. > > If you add the hierarchy > IOError...EnvironmentError...StandardError...Exception, how do you decide > whether to log the message or not? e.g. log.warn(IOError(...)) log.err(IOError(...)) note that Exceptions would *not* be a Message but the constructor of a DebugMessage/WarnMessage should accept an exception. i still don't think that the functionality of the logger-hierarchy would be compromised or more difficult to configure. I do think though that giving 'type-based' filtering precedence over integer-comparisons makes sense. > If I have a choice between writing: > > log.debug( > "HOME environment variable set to non-existent directory %s", > homedir > ) with the mapping in my previous posting you would write log.debug( "HOME environment variable set to non-existent directory " + homedir ) but if you app grows you might prefer log.debug(EnvMessage('HOME','points to non-existent directory')) and have EnvMessage handle any environment problems report systematically and well encapsulated. > I'm most likely to just use the string option - and > if I don't have a 'ConfigurationException' class & constructor that exactly > matches what I need, I'm even more likely to just log the string. i use strings often for ad-hoc coding. heck, i even like debugging with 'print' if possible. But there comes a point where you you want to log more and typed information and more intelligently and conveniently. Having to write '%s' in a logging call is not convenient IMO. > And you can get /exactly/ the TraceStateMessage example, including formatting, by: > # > # TraceStateMessage class you defined > # > > trace = logging.getLogger( "trace" ) > trace.debug( "%s", TraceStateMessage( obj )) nice :-) But does this beat somelog.traceobj(obj) People seem to apply their (logging) experience from the java/c++ world to python not realizing how much easier it is with python to work with objects. Were we to speak in java terms i wouldn't bother to argue :-) just my 2c, holger From nas@python.ca Thu May 16 03:32:53 2002 From: nas@python.ca (Neil Schemenauer) Date: Wed, 15 May 2002 19:32:53 -0700 Subject: [Python-Dev] automated testing system [was: Hundreds of warnings] In-Reply-To: <001701c1fc47$343ee820$57b63bd0@othello>; from python@rcn.com on Wed, May 15, 2002 at 03:32:07PM -0400 References: <20020515102944.A21134@glacier.arctrix.com> <3CE2A31D.1080005@ActiveState.com> <20020515112601.A21436@glacier.arctrix.com> <001701c1fc47$343ee820$57b63bd0@othello> Message-ID: <20020515193253.A22972@glacier.arctrix.com> Raymond Hettinger wrote: > Laura Creighton is also working on a test farm (though for different > purposes). Maybe some of the effort can be shared. I just saw the posting on c.l.p.a announcing the Python Business Forum. Great news for Python. Neil From tim.one@comcast.net Thu May 16 04:41:05 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 15 May 2002 23:41:05 -0400 Subject: [Python-Dev] Hundreds of warnings In-Reply-To: <075a01c1fc65$13e00730$6601a8c0@boostconsulting.com> Message-ID: [David Abrahams] > ...which explains why I have to > # define SIZEOF_LONG 4 > explicitly to build any Cygwin extension modules. I don't believe anything said here explained that. All Python ports have to define SIZEOF_LONG, and if Cygwin's isn't then that's a bug unique to Cygwin's config. On Unixoid systems, I believe that's supposed to be done by the autoconf hair. From martin@v.loewis.de Thu May 16 07:10:46 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 16 May 2002 08:10:46 +0200 Subject: [Python-Dev] Hundreds of warnings In-Reply-To: References: Message-ID: Tim Peters writes: > I don't believe anything said here explained that. All Python ports have to > define SIZEOF_LONG, and if Cygwin's isn't then that's a bug unique to > Cygwin's config. On Unixoid systems, I believe that's supposed to be done > by the autoconf hair. Correct. Martin From vinay_sajip@yahoo.co.uk Thu May 16 09:19:27 2002 From: vinay_sajip@yahoo.co.uk (Vinay Sajip) Date: Thu, 16 May 2002 09:19:27 +0100 Subject: [Python-Dev] PEP282 and the warnings framework References: <012301c1fc69$3fe7b540$652b6992@alpha> <20020516025408.V28033@prim.han.de> Message-ID: <001101c1fcb2$6827c6a0$652b6992@alpha> [Vinay] > > ... > > So far, I am not sure what benefit switching from the status quo of > > integer-based levels to class-based levels will give. Certainly, the idea of > > relative levels of importance of events must be preserved, since > > severity-based filtering is a key function of logging. [Holger] > first off, the DebugMessage/InfoMessage/... could > have a class-level severity property (integer) if one > really wants to have it for compatibility etc. Yes, I am aware of this - as I said in my last post, "...if you use some level attribute in message classes, what has really been gained by using classes?" What I mean is, that an integer level is simpler than a class with an integer level attribute. > this leaves the question how a Message-instance/type is more > powerful than a tuple(string/integer). > > Some thoughts regarding Message-instances/types: > > - the convenient logging API and categorization features > need *not* be changed largely (but the signature of the > log method should/might be changed) > > - beeing forced to totally order all types of messages can > be difficult. You may want e.g. to log 'TimeoutMessages' > no matter what the current integer threshold might be. > why not something like: loginstance.accept(TimeoutMessage) If timeouts are of concern in a design, you could just log to a channel called "timeout" or a channel such as "timeout.tcp.connect". If it's agreed that the idea of a hierarchical namespace of loggers is a good idea (it's one of log4j's Big Ideas), then it can be used for categorizing log events either according to functional areas of the application, or implementation layers of the application, or both. You get finer grained control than with classes, unless you want to create *lots* of classes. > - Message instances can defer the *construction* of a string > until it is really needed. (see my other posting for a code > example). This is more effective if you choose to ignore > certain messages. This has *nothing* to do with formatting. "Construction of a string"? If you are not talking about formatting, then I presume you mean a string literal or value bound to a string. If efficiency is the driver here, then I would have thought initialization of a Python class instance would be more expensive than a string. > - customized messages are powerful and allow more advanced > filtering than with strings, especially for large apps. > a connection-logger could choose to ignore a > TimeoutMessage(socket) based on the target address of the > socket. doing this with strings is hard (and perlish:-) Nobody is saying you have to "just use strings". For example, you could use a customized Filter for this. To take the timeout example further, it might be that other context is important when trying to log the message - e.g. not just the socket address, but perhaps also the number of timeouts seen recently. (After all, the odd timeout is expected - otherwise one wouldn't design for it.) It might be that such context is more readily available in a higher-level class (such as an HTTPServer). By using the Filter interface (any class implementing the filter() method can be a filter), you get more flexibility even in the scenario you mentioned. The decision is left up to the application designer, not imposed by the logging system. > - a SocketMessage-class may hold a weakref-list of sockets allowing > to log 'sock<1>', 'sock<2>' instead of addresses which are > hard to read. stuff like this is *very* convenient. Again > there is hardly any performance penalty if SocketMessages > are ignored. A Filter class can do this just as effectively. If messages are ignored because of level configuration, the filter wouldn't even get called, so no performance penalty would apply. > Maybe the categorization features of the logging > API make up for some of the (relative) inflexibilities of > string/integer messages. > But what do you really gain by restricting a message > to a (string,integer) taking my arguments into account? I'm not saying that a message must be restricted to only strings and integers. I'm only saying that the existing API provides the flexibility you need, and more besides. I also feel that the proposal you're endorsing holds some disbenefits. For example, you have not really responded to my arguments about interoperability, pickling, or the ability to change levels dynamically if the developer wants to for a particular application. And now we're talking about syntactic sugar, rather than using exception classes as a dispatch mechanism. Design choices are personal things. For many APIs we could say "but I'd rather do it like this...". Walter's original post seemed to be motivated by musing on the similarities between the two frameworks, rather than by a specific problem with PEP282/logging.py which needed to be addressed. I don't want to seem negative, or hidebound; a look at logging.py's change history shows that many changes have been added in response to user feedback. Much of this feedback has come from people actually using the system to solve their problems. But I really feel that all of the flexibility is available already. Kevin Butler's post showed how you could easily get what Holger wanted from the existing API: [Kevin] > trace = logging.getLogger( "trace" ) > trace.debug( "%s", TraceStateMessage( obj )) [Holger] >Having to write '%s' in a logging call is not convenient IMO. ... >nice :-) But does this beat > somelog.traceobj(obj) [Holger] >Having to write '%s' in a logging call is not convenient IMO. ... Now, it's getting to be a matter of personal taste rather than some fundamental problem with functionality. Some might prefer to say log.debug(EnvMessage('HOME','points to non-existent directory')) but I prefer log.debug("HOME is a non-existent directory: %s", the_home_value) in most cases. In the cases where I *do* want more flexibility, the existing API allows me to do this without *forcing* me to use classes. For example, in 0.4.5 (not yet released) I've made a minor change in Formatter.format(), replacing record.msg with str(record.msg). This allows code like this: #-code-------------------------------------- import logging class MySpecialClass: def __init__(self, param): self.param = param def __str__(self): return "%s, %%s" % self.param class MyOtherSpecialClass(MySpecialClass): def __str__(self): return "%s" % self.param logging.basicConfig() root = logging.getLogger("") root.warn(MySpecialClass("Hello"), "world!") root.warn(MyOtherSpecialClass("Goodbye!")) #-output------------------------------------- WARN:root:Hello, world! WARN:root:Goodbye! #-------------------------------------------- 'Nuff said? [Holger, replying to Kevin's post] >People seem to apply their (logging) experience from the >java/c++ world to python not realizing how much easier it >is with python to work with objects. Were we to speak >in java terms i wouldn't bother to argue :-) Er...not really, at least in this case. Java style seems to proliferate classes beyond necessity; the use of the Level class in log4j is, to me, an example of this. If you want an example of a "calqué" (literal) translation of log4j to Python, see the log4p project on Sourceforge (unmaintained since 2000). I didn't want logging.py to look like log4j, beyond using their best ideas; it doesn't feel Java-like to me, nor does it have any particular C++ idioms. Regards Vinay Sajip _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From thomas.heller@ion-tof.com Thu May 16 09:54:31 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Thu, 16 May 2002 10:54:31 +0200 Subject: [Python-Dev] __doc__ string of builtin types References: Message-ID: <023a01c1fcb7$4b663a10$e000a8c0@thomasnotebook> From: "Patrick K. O'Brien" > [Thomas Heller] > > > > (I've filed a slightly confused bug report 550290.) > > > > "spam".__doc__ is the same as str.__doc__, > > it describes what the str(...) would do. Same for > > 42 . __doc__, and certainly many other objects. > > While normally I don't care about the doc-string of > > these objects, the output of pydoc looks strange: > > I agree. I see the same thing with the namespace viewer in PyCrust under > Python 2.2 and it is less than ideal, imho. > I've uploaded a patch to http://www.python.org/sf/550290, currently for stringobject.c only, which keeps the current doc-string for the 'str' type/function, but returns None as the doc-string of str instances. The same could (and should) be done for int, float, and maybe other builtin types as well. The patch can easily be changed to return an actual doc-string. Thomas From mwh@python.net Thu May 16 10:12:44 2002 From: mwh@python.net (Michael Hudson) Date: 16 May 2002 10:12:44 +0100 Subject: [Python-Dev] automated testing system [was: Hundreds of warnings] In-Reply-To: Neil Schemenauer's message of "Wed, 15 May 2002 11:26:01 -0700" References: <20020515102944.A21134@glacier.arctrix.com> <3CE2A31D.1080005@ActiveState.com> <20020515112601.A21436@glacier.arctrix.com> Message-ID: <2mk7q4mp6b.fsf@starship.python.net> Neil Schemenauer writes: > David Ascher wrote: > > Are you really doing that? > > Still thinking about it. I haven't written a line of code yet. :-( Well, a few days ago I set up a cronjob that updates and builds the 3 unicode variants of Python every night, albeit only on linux/x86. So you won't get away with this sort of thing for more than 24 hours... Cheers, M. -- Monte Carlo sampling is no way to understand code. -- Gordon McMillan, comp.lang.python From pyth@devel.trillke.net Thu May 16 11:25:48 2002 From: pyth@devel.trillke.net (holger krekel) Date: Thu, 16 May 2002 12:25:48 +0200 Subject: [Python-Dev] PEP282 and the warnings framework In-Reply-To: <001101c1fcb2$6827c6a0$652b6992@alpha>; from vinay_sajip@yahoo.co.uk on Thu, May 16, 2002 at 09:19:27AM +0100 References: <012301c1fc69$3fe7b540$652b6992@alpha> <20020516025408.V28033@prim.han.de> <001101c1fcb2$6827c6a0$652b6992@alpha> Message-ID: <20020516122548.X28033@prim.han.de> Hello Vinay, for the records: i appreciate your logger functionality. I guess i should find time to modify logging.py to really get a better comparison of concepts. Shouldn't be a big thing as i would probably aim at replacing your LogRecord class with a Message class (hierarchy) while providing much the same Logger-interface as it is now. Hope to find some time for this little experiment... regards, holger From pyth@devel.trillke.net Thu May 16 11:35:59 2002 From: pyth@devel.trillke.net (holger krekel) Date: Thu, 16 May 2002 12:35:59 +0200 Subject: [Python-Dev] __doc__ string of builtin types In-Reply-To: <023a01c1fcb7$4b663a10$e000a8c0@thomasnotebook>; from thomas.heller@ion-tof.com on Thu, May 16, 2002 at 10:54:31AM +0200 References: <023a01c1fcb7$4b663a10$e000a8c0@thomasnotebook> Message-ID: <20020516123559.Y28033@prim.han.de> Thomas Heller wrote: > From: "Patrick K. O'Brien" > > [Thomas Heller] > > > > > > (I've filed a slightly confused bug report 550290.) > > > > > > "spam".__doc__ is the same as str.__doc__, > > > it describes what the str(...) would do. Same for > > > 42 . __doc__, and certainly many other objects. > > > While normally I don't care about the doc-string of > > > these objects, the output of pydoc looks strange: > > > > I agree. I see the same thing with the namespace viewer in PyCrust under > > Python 2.2 and it is less than ideal, imho. > > > I've uploaded a patch to http://www.python.org/sf/550290, currently > for stringobject.c only, which keeps the current doc-string > for the 'str' type/function, but returns None as the doc-string > of str instances. The same could (and should) be done for > int, float, and maybe other builtin types as well. The patch > can easily be changed to return an actual doc-string. as a workaround i have in the past used the check getattr(type(obj),'__doc__','')==getattr(obj,'__doc__','') to determine if an object has 'real' documentation. only doesn't work for the type-object because type(type)==type. holger From s_lott@yahoo.com Thu May 16 12:29:29 2002 From: s_lott@yahoo.com (Steven Lott) Date: Thu, 16 May 2002 04:29:29 -0700 (PDT) Subject: [Python-Dev] PEP282 and the warnings framework Message-ID: <20020516112929.40768.qmail@web9607.mail.yahoo.com> After having implement several real-time debuggers for military systems (and lurking for months on this list) let me weigh in. First, the "Severity" of an event is one of many attributes. The attributes include an event name (including a subject), time, the host, the environment, a descriptive string, an instance of an object, etc. Second, the Severity is a tricky business to get right. As has been observed, one person's "Fatal" is another person's "Warning". Depends on context. I can remember putting in lists of overrides that would treat ordinarily fatal OS exceptions as warnings so our application could continue to run in spite of lossed I/O interrupts. Third, all programming boils down to comparison of integers. If we start from comparison of integers, we've lost any possibility of putting more meaning into the program. If we start with class definitions, then the final implementation may be some kind of RTTI that is implemented as an integer comparison. Fourth, all logging systems need filters. Often those filters are subject-area based, and have little to do with severity or with the ultimate exception/error class to which the event belongs. For "normal production" we only want to see severe errors or fatal errors that are the final outcome of the application. For "QA testing" we want to see all errors from all component modules. For "Integration testing" we want to see all warnings from all classes. etc. Simple numeric codes won't provide much useful filtering. Programmers will work around it by dynamically assigning a severity to permit logging under some circumstances. People will build applications that use an environment variable to set message severity so that it will be very severe during testing (and get logged) but not severe in production (to reduce the log blather). A subject area string (or tuple) is a cool way to do business. It allows one to label the event with information like module, class, method, event. If someone wants to include severity as part of their subject string, that's good. Then you can filter on subject areas. The idea is to use a simple fnmatch-style filter. log.watch( 'error.*' ) or log.watch( 'warn.*', 'error.*' ) or log.watch( 'error.pil.gif.*' ). I'm not the only one who's found this effective. If you dig into the TIBCO commercial product, they devote some real energy to making useful subject names for the message traffic in their middleware product. My vote is: A) existing exception/error hierarchy B) LogEvent class with message string, subject string, delayed introspection, automatic timestamping. LogEventObject and LogEventEnviron subclasses would drag around additional details. C) The logger would filter LogEvents based on subject string. Since each event is an object, the logger could simply pickle the events, or write their __str__ to stderr. ===== -- S. Lott, CCP :-{) S_LOTT@YAHOO.COM http://www.mindspring.com/~slott1 Buccaneer #468: KaDiMa Macintosh user: drinking upstream from the herd. __________________________________________________ Do You Yahoo!? LAUNCH - Your Yahoo! Music Experience http://launch.yahoo.com From walter@livinglogic.de Thu May 16 12:44:03 2002 From: walter@livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Thu, 16 May 2002 13:44:03 +0200 Subject: [Python-Dev] PEP282 and the warnings framework References: <20020515141648.9926.76949.Mailman@mail.python.org> <3CE283FE.8000403@campuspipeline.com> <20020515201342.T28033@prim.han.de> <3CE2AE95.4030900@campuspipeline.com> <20020516003321.U28033@prim.han.de> <3CE2FD51.2060301@campuspipeline.com> Message-ID: <3CE39B83.40006@livinglogic.de> Kevin Butler wrote: > [...] > I don't see how it would work without drastically increasing the > configuration complexity. > > log = logging.getLogger( "pil.image.gif" ) > log.log( IOError( reason )) > > Currently, I can configure the hierachy pil...image...gif, and I can > specify the integer priority level below which I won't log a message in > that category. > > If you add the hierarchy > IOError...EnvironmentError...StandardError...Exception, how do you > decide whether to log the message or not? The standard module could have the following: class ClassFilter(Filter): def __init__(self, classes, name="") super(ClassFilter, self).__init__(name) self.classes = classes def filter(self, record): return isinstance(record, self.classes) then you could use this filter. It should be possible to derive from your class LogRecord, formatting of the message should not be done by Fomatter.format() which does a record.message = record.msg % record.args, but should be the responsibility of LogRecord.__str__(), the default implementation could simply have def __str__(self): return self.msg % self.args The constructor for LogRecord should be lightweight enough, so that it is convenient to create LogRecord instances and pass them to Logger.log etc. i.e. determining file name and line number should not be the responsiblity of the function that creates the LogRecord instance, but the LogRecord constructor. > - Do you have two separate hierarchies, making each message > enabled/disabled for all loggers? This doesn't seem to give you much > flexibility (IOError enabled everywhere). > > - Or does each logger configuration have a hierarchy of message types, > and each message type can be enabled/disabled for each logger? (This > seems to require too much configuration.) > > Some other approach I'm not seeing? The type of the message/log record is simply another property I can use to filter. And I can extend my filter criteria, because I'm not tied to the LogRecord instance that gets implicitely created by Logger.makeRecord. Yes, Logger.makeRecord is documented as a "factory method", but I couldn't find any possibility to replace the factory. >>> FWIW, once you allow logging 'string'-type messages, most logged >>> messages will be a string (especially debug messages), because it is >>> much easier than creating an instance of some other clsas. Thus, if >>> your categorization is based on the class of the logged message, the >>> "string" category gets very large... So what is different from the current implementation where all messages are LogRecord instances? >> NO. Only because a typed message accepts a string in its constructor >> does not make it a string. Or since when are all python exceptions in >> the 'string' category? > > > If I have a choice between writing: > > log.debug( > "HOME environment variable set to non-existent directory %s", > homedir > ) > > and: > > log.debug( ConfigurationException( > "variable set to non-existent directory", > "HOME", > homedir > )) > > I'm most likely to just use the string option - and > if I don't have a 'ConfigurationException' class & constructor that exactly > matches what I need, I'm even more likely to just log the string. I think the distinction is between log.log( logging.DEBUG, "HOME variable set to non-existent directory %s", homedir ) and log.log( ConfigurationException( "variable set to non-existent directory", "HOME", homedir ) ) And it would be possible to extend the custom classes with e.g. localized messages. > [...] Bye, Walter Dörwald From anthony@interlink.com.au Thu May 16 13:36:44 2002 From: anthony@interlink.com.au (Anthony Baxter) Date: Thu, 16 May 2002 12:36:44 +0000 Subject: [Python-Dev] SF -checkins list. Message-ID: <20020516123644.6313F13243@devhost1.off.ekorp.com> [resend - I'm pretty sure the previous version of this got eaten] I'm finding the sourceforge -checkins list to be incredibly unreliable of late - is it just me that's having this problem? Messages arrive delayed by 12 - 24 hours, or don't turn up at all. Since I'm trying to use this for the backporting notification, this is double-plus bad. From guido@python.org Thu May 16 13:48:23 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 16 May 2002 08:48:23 -0400 Subject: [Python-Dev] SF -checkins list. In-Reply-To: Your message of "Thu, 16 May 2002 12:36:44 -0000." <20020516123644.6313F13243@devhost1.off.ekorp.com> References: <20020516123644.6313F13243@devhost1.off.ekorp.com> Message-ID: <200205161248.g4GCmN031788@pcp742651pcs.reston01.va.comcast.net> > I'm finding the sourceforge -checkins list to be incredibly unreliable of > late - is it just me that's having this problem? Messages arrive delayed > by 12 - 24 hours, or don't turn up at all. Since I'm trying to use this > for the backporting notification, this is double-plus bad. They've been having this problem on and off for weeks. :-( Time to set up our own CVS repository? --Guido van Rossum (home page: http://www.python.org/~guido/) From Anthony Baxter Thu May 16 13:47:53 2002 From: Anthony Baxter (Anthony Baxter) Date: Thu, 16 May 2002 22:47:53 +1000 Subject: [Python-Dev] SF -checkins list. In-Reply-To: Message from Guido van Rossum of "Thu, 16 May 2002 08:48:23 -0400." <200205161248.g4GCmN031788@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200205161247.g4GClre01658@localhost.localdomain> > They've been having this problem on and off for weeks. :-( > Time to set up our own CVS repository? I'm tempted to write something that does up -c or something on a regular basis, finds the changes, and sends it's own email to me. But this is something of a desperation move. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From pobrien@orbtech.com Thu May 16 14:40:38 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Thu, 16 May 2002 08:40:38 -0500 Subject: [Python-Dev] __doc__ string of builtin types In-Reply-To: <20020516123559.Y28033@prim.han.de> Message-ID: [holger krekel] > as a workaround i have in the past used the check > > getattr(type(obj),'__doc__','')==getattr(obj,'__doc__','') > > to determine if an object has 'real' documentation. > only doesn't work for the type-object because type(type)==type. Thanks for the tip. In my particular case I'm actually using inspect.getdoc(). I suppose I could apply the same check to what it returns. Or someone might want to patch inspect.getdoc() if that would be easier in the short term. Just thought I'd mention that in case Ka-Ping Yee is following this thread. --- Patrick K. O'Brien Orbtech From pyth@devel.trillke.net Thu May 16 15:27:32 2002 From: pyth@devel.trillke.net (holger krekel) Date: Thu, 16 May 2002 16:27:32 +0200 Subject: [Python-Dev] __doc__ string of builtin types In-Reply-To: ; from pobrien@orbtech.com on Thu, May 16, 2002 at 08:40:38AM -0500 References: <20020516123559.Y28033@prim.han.de> Message-ID: <20020516162732.Z28033@prim.han.de> Patrick K. O'Brien wrote: > [holger krekel] > > as a workaround i have in the past used the check > > > > getattr(type(obj),'__doc__','')==getattr(obj,'__doc__','') > > > > to determine if an object has 'real' documentation. > > only doesn't work for the type-object because type(type)==type. > > Thanks for the tip. In my particular case I'm actually using > inspect.getdoc(). I suppose I could apply the same check to what it returns. > Or someone might want to patch inspect.getdoc() if that would be easier in > the short term. Just thought I'd mention that in case Ka-Ping Yee is > following this thread. i have just patched my inspect.py and pydoc.py to include the above check. works fine! next time i update my cvs tree i submit a patch. regards, holger From vinay_sajip@red-dove.com Thu May 16 16:32:59 2002 From: vinay_sajip@red-dove.com (Vinay Sajip) Date: Thu, 16 May 2002 16:32:59 +0100 Subject: [Python-Dev] PEP282 and the warnings framework References: <20020516112929.40768.qmail@web9607.mail.yahoo.com> Message-ID: <007101c1fcee$f6becd00$652b6992@alpha> > After having implement several real-time debuggers for military > systems (and lurking for months on this list) let me weigh in. > > First, the "Severity" of an event is one of many attributes. > The attributes include an event name (including a subject), > time, the host, the environment, a descriptive string, an > instance of an object, etc. Yes. You can boil all these attributes down to "what happened?", "when did it happen?", "where did it happen?" (i.e. which area of the application) and "how important is it?", all of which are more or less orthogonal. > Second, the Severity is a tricky business to get right. As has > been observed, one person's "Fatal" is another person's > "Warning". Depends on context. I can remember putting in lists > of overrides that would treat ordinarily fatal OS exceptions as > warnings so our application could continue to run in spite of > lossed I/O interrupts. This is why a logging system should not pre-empt designer decisions about severity, nor take unilateral actions such as shutdown. It's there for only one task - notifying interested audiences of interesting events which occur during an application's execution. The events themselves are usually defined by the logging calls defined by the developer; the definitions of "interesting events" and "interested audience" are mutable by changing the logging configuration. > Third, all programming boils down to comparison of integers. If > we start from comparison of integers, we've lost any possibility > of putting more meaning into the program. If we start with > class definitions, then the final implementation may be some > kind of RTTI that is implemented as an integer comparison. I don't think you lose much flexibility if you use integers to define levels, as ultimately levels are only one component of log filtering anyway. Their main advantage is that they are cheap to construct, quick to compare with each other, easy to understand and readily available :-) > Fourth, all logging systems need filters. Often those filters > are subject-area based, and have little to do with severity or > with the ultimate exception/error class to which the event > belongs. For "normal production" we only want to see severe > errors or fatal errors that are the final outcome of the > application. For "QA testing" we want to see all errors from > all component modules. For "Integration testing" we want to see > all warnings from all classes. etc. Yes. PEP282/logging.py offers these mechanisms. Logger channels offer subject-area filtering, levels offer severity filtering, and filters offer pretty much anything you want in terms of flexibility. > Simple numeric codes won't provide much useful filtering. > Programmers will work around it by dynamically assigning a > severity to permit logging under some circumstances. People > will build applications that use an environment variable to set > message severity so that it will be very severe during testing > (and get logged) but not severe in production (to reduce the log > blather). Earlier, you said 'For "normal production" we only want to see severe errors or fatal errors that are the final outcome of the application'. To me, that maps most naturally to a numeric level. > A subject area string (or tuple) is a cool way to do business. You get this by using logging channels with subject-based names. > It allows one to label the event with information like module, > class, method, event. If someone wants to include severity as > part of their subject string, that's good. Then you can filter > on subject areas. Not only, but also. Why constrain needlessly to just one way of filtering? > The idea is to use a simple fnmatch-style filter. > log.watch( 'error.*' ) or log.watch( 'warn.*', 'error.*' ) > or log.watch( 'error.pil.gif.*' ). The logging.py distribution includes a simple example of a MatchFilter (in a test script) which works something like this. It's relatively easy to adapt to more specialized needs. > I'm not the only one who's found this effective. If you dig > into the TIBCO commercial product, they devote some real energy > to making useful subject names for the message traffic in their > middleware product. Yes, and developers who use logging should, for large applications, devote the same energy to ensuring that their logging design is in keeping with the complexity of their application. After all, logging is a kind of instrumentation. > My vote is: > A) existing exception/error hierarchy > B) LogEvent class with message string, subject string, delayed > introspection, automatic timestamping. LogEventObject and > LogEventEnviron subclasses would drag around additional details. > C) The logger would filter LogEvents based on subject string. > Since each event is an object, the logger could simply pickle > the events, or write their __str__ to stderr. Not sure what you mean by (A). As far as I can see, logging.py provides all of (B)'s first sentence. You can use derived classes where needed, too. (C)'s first sentence I find a little restrictive, why does it *need* to work like that? Regards Vinay Sajip From vinay_sajip@red-dove.com Thu May 16 16:33:06 2002 From: vinay_sajip@red-dove.com (Vinay Sajip) Date: Thu, 16 May 2002 16:33:06 +0100 Subject: [Python-Dev] PEP282 and the warnings framework References: <20020515141648.9926.76949.Mailman@mail.python.org> <3CE283FE.8000403@campuspipeline.com> <20020515201342.T28033@prim.han.de> <3CE2AE95.4030900@campuspipeline.com> <20020516003321.U28033@prim.han.de> <3CE2FD51.2060301@campuspipeline.com> <3CE39B83.40006@livinglogic.de> Message-ID: <007501c1fcee$fa693760$652b6992@alpha> [Walter] > The standard module could have the following: > > class ClassFilter(Filter): > def __init__(self, classes, name="") > super(ClassFilter, self).__init__(name) > self.classes = classes > def filter(self, record): > return isinstance(record, self.classes) > then you could use this filter. > > It should be possible to derive from your class LogRecord, > formatting of the message should not be done by Fomatter.format() > which does a record.message = record.msg % record.args, but should > be the responsibility of LogRecord.__str__(), the default implementation > could simply have > def __str__(self): > return self.msg % self.args The last part (about __str__) is reasonable. But I disagree with the statement that the LogRecord should do formatting. Remember - logging is used by lots of people in lots of scenarios. For example, I have used it in such a way that messages to stderr/stdout do not have timestamps (e.g. for warning/error messages, timestamps appear a little loud), but information sent to file *does* have timestamps. Likewise, I'm sometimes interested in threads, sometimes not. I'd like to change the formatting behaviour of a long-running program, while it's running. Ideally, I might choose to only change certain handlers, e.g. the FileHandlers. The point here is that you should be able to change these sorts of things *just* by changing the logging configuration. You can't do this effectively if the basic decision making is devolved to lots of user-defined classes in an arbitrary way - you would need to either change source code, or have configurability of these user-defined classes. To me, this is an undesirable outcome. > The constructor for LogRecord should be lightweight enough, so that it > is convenient to create LogRecord instances and pass them to Logger.log > etc. i.e. determining file name and line number should not be the > responsiblity of the function that creates the LogRecord instance, > but the LogRecord constructor. As it happens, it's done in the Logger in logging.py. LogRecord is kept fairly minimal; it's just a handy place to keep attributes of the event. > > The type of the message/log record is simply another property I > can use to filter. And I can extend my filter criteria, because > I'm not tied to the LogRecord instance that gets implicitely > created by Logger.makeRecord. Yes, Logger.makeRecord is documented > as a "factory method", but I couldn't find any possibility to replace > the factory. You can subclass Logger and redefine makeRecord. Then use logging.setLoggerClass to ask the logging system to instantiate your own logger class. Or, if you really want to take a shortcut, you can bind Logger.makeRecord to a function of your own devising by a simple assignment. Quick'n'dirty: import logging class MyLogRecord(logging.LogRecord): def __init__(self, name, lvl, fn, lno, msg, args, exc_info): logging.LogRecord.__init__(self, name, lvl, fn, lno, msg, args, exc_info) self.msg += "(MyLogRecord)" def myMakeRecord(self, name, lvl, fn, lno, msg, args, exc_info): return MyLogRecord(name, lvl, fn, lno, msg, args, exc_info) logging.Logger.makeRecord = myMakeRecord logging.basicConfig() root = logging.getLogger("") root.warn("Hello") > >>> FWIW, once you allow logging 'string'-type messages, most logged > >>> messages will be a string (especially debug messages), because it is > >>> much easier than creating an instance of some other clsas. Thus, if > >>> your categorization is based on the class of the logged message, the > >>> "string" category gets very large... > > So what is different from the current implementation where all messages > are LogRecord instances? It's a stylistic difference. In one case you pass strings into the logging API; in the other, you pass instances. The question is, what's most convenient for most users most of the time? To my way of thinking, strings serve in nearly all cases and are simpler. If you *want* classes, you can utilize them using the existing functionality; there's no need to force *everyone* to use classes. Regards Vinay Sajip From aahz@pythoncraft.com Thu May 16 16:37:30 2002 From: aahz@pythoncraft.com (Aahz) Date: Thu, 16 May 2002 11:37:30 -0400 Subject: [Python-Dev] PEP282 and the warnings framework In-Reply-To: <20020516034824.W28033@prim.han.de> References: <20020515141648.9926.76949.Mailman@mail.python.org> <3CE283FE.8000403@campuspipeline.com> <20020515201342.T28033@prim.han.de> <3CE2AE95.4030900@campuspipeline.com> <20020516003321.U28033@prim.han.de> <3CE2FD51.2060301@campuspipeline.com> <20020516034824.W28033@prim.han.de> Message-ID: <20020516153729.GA29986@panix.com> On Thu, May 16, 2002, holger krekel wrote: > Kevin Butler wrote: >> >> If I have a choice between writing: >> >> log.debug( >> "HOME environment variable set to non-existent directory %s", >> homedir >> ) > > with the mapping in my previous posting you would write > > log.debug( > "HOME environment variable set to non-existent directory " + homedir > ) There is no difference between these two. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "In the end, outside of spy agencies, people are far too trusting and willing to help." --Ira Winkler From pyth@devel.trillke.net Thu May 16 16:40:48 2002 From: pyth@devel.trillke.net (holger krekel) Date: Thu, 16 May 2002 17:40:48 +0200 Subject: [Python-Dev] PEP282 and the warnings framework In-Reply-To: <20020516153729.GA29986@panix.com>; from aahz@pythoncraft.com on Thu, May 16, 2002 at 11:37:30AM -0400 References: <20020515141648.9926.76949.Mailman@mail.python.org> <3CE283FE.8000403@campuspipeline.com> <20020515201342.T28033@prim.han.de> <3CE2AE95.4030900@campuspipeline.com> <20020516003321.U28033@prim.han.de> <3CE2FD51.2060301@campuspipeline.com> <20020516034824.W28033@prim.han.de> <20020516153729.GA29986@panix.com> Message-ID: <20020516174048.E28033@prim.han.de> Aahz wrote: > On Thu, May 16, 2002, holger krekel wrote: > > Kevin Butler wrote: > >> > >> If I have a choice between writing: > >> > >> log.debug( > >> "HOME environment variable set to non-existent directory %s", > >> homedir > >> ) > > > > with the mapping in my previous posting you would write > > > > log.debug( > > "HOME environment variable set to non-existent directory " + homedir > > ) > > There is no difference between these two. that was my implicit argument (that you don't loose convenience). thanks for making it explicit :-) holger From fredrik@pythonware.com Thu May 16 16:47:31 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Thu, 16 May 2002 17:47:31 +0200 Subject: [Python-Dev] PEP282 and the warnings framework References: <20020515141648.9926.76949.Mailman@mail.python.org> <3CE283FE.8000403@campuspipeline.com> <20020515201342.T28033@prim.han.de> <3CE2AE95.4030900@campuspipeline.com> <20020516003321.U28033@prim.han.de> <3CE2FD51.2060301@campuspipeline.com> <20020516034824.W28033@prim.han.de> <20020516153729.GA29986@panix.com> Message-ID: <02fa01c1fcf1$00a58550$ced241d5@hagrid> "Aahz" wrote: > >> If I have a choice between writing: > >> > >> log.debug( > >> "HOME environment variable set to non-existent directory %s", > >> homedir > >> ) > > > > with the mapping in my previous posting you would write > > > > log.debug( > > "HOME environment variable set to non-existent directory " + homedir > > ) > > There is no difference between these two. so why does one of them give me a TypeError? From aahz@pythoncraft.com Thu May 16 16:52:22 2002 From: aahz@pythoncraft.com (Aahz) Date: Thu, 16 May 2002 11:52:22 -0400 Subject: [Python-Dev] PEP282 and the warnings framework In-Reply-To: <20020516174048.E28033@prim.han.de> References: <20020515141648.9926.76949.Mailman@mail.python.org> <3CE283FE.8000403@campuspipeline.com> <20020515201342.T28033@prim.han.de> <3CE2AE95.4030900@campuspipeline.com> <20020516003321.U28033@prim.han.de> <3CE2FD51.2060301@campuspipeline.com> <20020516034824.W28033@prim.han.de> <20020516153729.GA29986@panix.com> <20020516174048.E28033@prim.han.de> Message-ID: <20020516155221.GA3466@panix.com> On Thu, May 16, 2002, holger krekel wrote: > Aahz wrote: >> On Thu, May 16, 2002, holger krekel wrote: >>> Kevin Butler wrote: >>>> >>>> If I have a choice between writing: >>>> >>>> log.debug( >>>> "HOME environment variable set to non-existent directory %s", >>>> homedir >>>> ) >>> >>> with the mapping in my previous posting you would write >>> >>> log.debug( >>> "HOME environment variable set to non-existent directory " + homedir >>> ) >> >> There is no difference between these two. > > that was my implicit argument (that you don't loose convenience). > thanks for making it explicit :-) Whoops. I just realized that I misread the "," in the first example for a "%". Never mind, I should just stay out of this thread, but before I leave, I have to say that my esthetic sense prefers the class-based logging levels. I doubt it will make much practical difference, though. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "In the end, outside of spy agencies, people are far too trusting and willing to help." --Ira Winkler From pyth@devel.trillke.net Thu May 16 17:17:53 2002 From: pyth@devel.trillke.net (holger krekel) Date: Thu, 16 May 2002 18:17:53 +0200 Subject: [Python-Dev] PEP282 and the warnings framework In-Reply-To: <15587.54313.847988.80055@anthem.wooz.org>; from barry@zope.com on Thu, May 16, 2002 at 11:45:45AM -0400 References: <20020515141648.9926.76949.Mailman@mail.python.org> <3CE283FE.8000403@campuspipeline.com> <20020515201342.T28033@prim.han.de> <3CE2AE95.4030900@campuspipeline.com> <20020516003321.U28033@prim.han.de> <3CE2FD51.2060301@campuspipeline.com> <20020516034824.W28033@prim.han.de> <20020516153729.GA29986@panix.com> <15587.54313.847988.80055@anthem.wooz.org> Message-ID: <20020516181753.G28033@prim.han.de> Barry A. Warsaw wrote: > > >> If I have a choice between writing: > >> > >> log.debug( > >> "HOME environment variable set to non-existent directory %s", > >> homedir > >> ) > > > > with the mapping in my previous posting you would write > > > > log.debug( > > "HOME environment variable set to non-existent directory " + homedir > > ) > > >>>>> "A" == Aahz writes: > > A> There is no difference between these two. > > I'm coming in in the middle of this thread, but there is a very > important difference between these two. true. This citation is misleading. Originally this was a question of convenience, not of 'what is better' really. The better suggestion was, IMO, that for a growing application you (might) prefer to write log.log(EnvMessage('HOME','non-existant directory')) where EnvMessage will only acess os.environ if somebody is interested in the Message (e.g. __str__ is called). This way you can systematically handle any problems with Environment variables and avoid typos etc... holger From martin@v.loewis.de Thu May 16 17:50:18 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 16 May 2002 18:50:18 +0200 Subject: [Python-Dev] SF -checkins list. In-Reply-To: <20020516123644.6313F13243@devhost1.off.ekorp.com> References: <20020516123644.6313F13243@devhost1.off.ekorp.com> Message-ID: Anthony Baxter writes: > I'm finding the sourceforge -checkins list to be incredibly unreliable of > late - is it just me that's having this problem? Messages arrive delayed > by 12 - 24 hours, or don't turn up at all. Since I'm trying to use this > for the backporting notification, this is double-plus bad. Can you point to specific checkin messages that are actually lost? I can believe that SF mail builds up a huge backlog. I can't really believe that mail is actually getting lost. Regards, Martin From David Abrahams" Message-ID: <01d701c1fcfa$359b7a90$6601a8c0@boostconsulting.com> From: "Martin v. Loewis" > Anthony Baxter writes: > > > I'm finding the sourceforge -checkins list to be incredibly unreliable of > > late - is it just me that's having this problem? Messages arrive delayed > > by 12 - 24 hours, or don't turn up at all. Since I'm trying to use this > > for the backporting notification, this is double-plus bad. > > Can you point to specific checkin messages that are actually lost? > > I can believe that SF mail builds up a huge backlog. I can't really > believe that mail is actually getting lost. If this is a syncmail list, FYI, we recently set this up on boost and were getting no mails at all. Eventually we rolled back to an earlier version of the syncmail script and it started working for us. HTH, Dave From barry@zope.com Thu May 16 16:45:45 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 16 May 2002 11:45:45 -0400 Subject: [Python-Dev] PEP282 and the warnings framework References: <20020515141648.9926.76949.Mailman@mail.python.org> <3CE283FE.8000403@campuspipeline.com> <20020515201342.T28033@prim.han.de> <3CE2AE95.4030900@campuspipeline.com> <20020516003321.U28033@prim.han.de> <3CE2FD51.2060301@campuspipeline.com> <20020516034824.W28033@prim.han.de> <20020516153729.GA29986@panix.com> Message-ID: <15587.54313.847988.80055@anthem.wooz.org> >> If I have a choice between writing: >> >> log.debug( >> "HOME environment variable set to non-existent directory %s", >> homedir >> ) > > with the mapping in my previous posting you would write > > log.debug( > "HOME environment variable set to non-existent directory " + homedir > ) >>>>> "A" == Aahz writes: A> There is no difference between these two. I'm coming in in the middle of this thread, but there is a very important difference between these two. The latter does more work because it does a string concat before log.debug() is called. Which means if the logging level is such that the message is suppressed, you've just wasted some time and memory. The former can avoid all that by just not doing the string interpolation if it's just going to throw away the log message. I use the first form in Mailman's own logging system and I think it's a win. I don't know which form is being argued for here, but the first has my +1, the second a -1. -Barry From rwgk@cci.lbl.gov Thu May 16 19:33:18 2002 From: rwgk@cci.lbl.gov (Ralf W. Grosse-Kunstleve) Date: Thu, 16 May 2002 11:33:18 -0700 (PDT) Subject: [Python-Dev] Re: Minimal GCC/Linux shared lib + EH bug example Message-ID: <200205161833.g4GIXIq48566@boa.lbl.gov> Mark Mitchell writes: > That might change if the ELF/C++/etc. semantics get better in the ways > that are being discussed, but at present, I will simply say that if > I were going to build a Python extension module, I would do it in C, > even though I generally find that I am more cost-effective when working > in C++. The traditional idea of "writing an extension" does not apply if you work with Boost.Python. Effectively, Boost.Python is turning Python into a scripting front-end for C++. We find this to be extremely powerful and productive. For us there is no alternative to this approach: we need an object oriented language in both the compiled and the scripted layer. Martin Loewis writes: > Mark Mitchell writes: > > If the rule was that, for example, all externally visible names in the > > loaded modules had to be within namespaces that were assigned by some > > naming authority, or otherwise consistent, then you could load modules > > with RTLD_GLOBAL. > > > > Yes, I know this is non-trivial, but if you want to use C++, it's the > > practical solution over the medium term. > > That indeed is an option: with sys.setdlopenflags, the application can > override Python's default of RTLD_LOCAL. If this reliably solves the problem it will work for us. However, it is not ideal that sys.setdlopenflags is a one-size-fits-all global setting. It would be better if the user could have more fine-grained control, e.g. package-level control to be used in the __init__.py files. Ralf From walter@livinglogic.de Thu May 16 19:36:08 2002 From: walter@livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Thu, 16 May 2002 20:36:08 +0200 Subject: [Python-Dev] PEP282 and the warnings framework References: <20020515141648.9926.76949.Mailman@mail.python.org> <3CE283FE.8000403@campuspipeline.com> <20020515201342.T28033@prim.han.de> <3CE2AE95.4030900@campuspipeline.com> <20020516003321.U28033@prim.han.de> <3CE2FD51.2060301@campuspipeline.com> <3CE39B83.40006@livinglogic.de> <007501c1fcee$fa693760$652b6992@alpha> Message-ID: <3CE3FC18.4080509@livinglogic.de> Vinay Sajip wrote: > [...] >>It should be possible to derive from your class LogRecord, >>formatting of the message should not be done by Fomatter.format() >>which does a record.message = record.msg % record.args, but should >>be the responsibility of LogRecord.__str__(), the default implementation >>could simply have >> def __str__(self): >> return self.msg % self.args > > > The last part (about __str__) is reasonable. But I disagree with the > statement that the LogRecord should do formatting. I meant only the formatting of the message itself, i.e. the "self.msg % self.args" part. Timestamps, file/line info etc. should be the responsiblity of the Logger/Formatter. > The point here is that you > should be able to change these sorts of things *just* by changing the > logging configuration. You can't do this effectively if the basic decision > making is devolved to lots of user-defined classes in an arbitrary way - you > would need to either change source code, or have configurability of these > user-defined classes. To me, this is an undesirable outcome. You're right, configuration should be changable at runtime. Doing this when configuration involves custom classes is hard, but this shouldn't be your job, but the job of those that implement these custom classes. >>The constructor for LogRecord should be lightweight enough, so that it >>is convenient to create LogRecord instances and pass them to Logger.log >>etc. i.e. determining file name and line number should not be the >>responsiblity of the function that creates the LogRecord instance, >>but the LogRecord constructor. > > As it happens, it's done in the Logger in logging.py. LogRecord is kept > fairly minimal; it's just a handy place to keep attributes of the event. I'd say it *is* the event. >>The type of the message/log record is simply another property I >>can use to filter. And I can extend my filter criteria, because >>I'm not tied to the LogRecord instance that gets implicitely >>created by Logger.makeRecord. Yes, Logger.makeRecord is documented >>as a "factory method", but I couldn't find any possibility to replace >>the factory. > > > You can subclass Logger and redefine makeRecord. Yes, that's the purpose of the factory method, but then I have to replace the factory method that generates the factory ... > Then use > logging.setLoggerClass to ask the logging system to instantiate your own > logger class. ... which is done here. But this would change the LogRecord classes that are used on a global scale. I'd like to decide which class should be used when I'm creating the event. BTW, in setLoggerClass() you should probably use if not issubclass(klass, Logger): instead of if not (Logger in klass.__bases__): > Or, if you really want to take a shortcut, you can bind > Logger.makeRecord to a function of your own devising by a simple assignment. > Quick'n'dirty: Exactly. This work more out of accident, then out of design. > [...] >>>>>FWIW, once you allow logging 'string'-type messages, most logged >>>>>messages will be a string (especially debug messages), because it is >>>>>much easier than creating an instance of some other clsas. Thus, if >>>>>your categorization is based on the class of the logged message, the >>>>>"string" category gets very large... >>>> >>So what is different from the current implementation where all messages >>are LogRecord instances? > > > It's a stylistic difference. In one case you pass strings into the logging > API; in the other, you pass instances. The question is, what's most > convenient for most users most of the time? To my way of thinking, strings > serve in nearly all cases and are simpler. If you *want* classes, you can > utilize them using the existing functionality; there's no need to force > *everyone* to use classes. What I want is something like this (similar to what the warning framework does): def log(event, eventtype=LogRecord): if not isinstance(event, LogRecord): event = eventtype(event) ... the rest of the code This way you can use simple strings: log("something wonderful has happened") you can use string/type combinations: log("something wonderful has happened", logging.CriticalLogRecord) and you can use instances: log(logging.CriticalLogRecord("something wonderful has happened")) I you really want the keep the severity out of the class, we could do this: def log(event, level=INFO, eventtype=LogRecord): if not isinstance(event, LogRecord): event = eventtype(event, level) ... the rest of the code Bye, Walter Dörwald From s_lott@yahoo.com Thu May 16 19:52:52 2002 From: s_lott@yahoo.com (Steven Lott) Date: Thu, 16 May 2002 11:52:52 -0700 (PDT) Subject: [Python-Dev] PEP282 and the warnings framework In-Reply-To: <007101c1fcee$f6becd00$652b6992@alpha> Message-ID: <20020516185252.13889.qmail@web9601.mail.yahoo.com> [snip] > > Yes. PEP282/logging.py offers these mechanisms. Logger > channels offer > subject-area filtering, levels offer severity filtering, and > filters offer > pretty much anything you want in terms of flexibility. > How does a "channel" offer subject area filtering? How do I get a single log with some subjects enabled and some subjects disabled? [snip] > > Yes, and developers who use logging should, for large > applications, devote > the same energy to ensuring that their logging design is in > keeping with the > complexity of their application. After all, logging is a kind > of > instrumentation. Actually it is more than this. Much more. The deal is that most applications are driven by events. For trivial "read/filter/write" applications, the events are hardly worth describing as unique objects. For almost everything else, however, the events are the core of the application's work. If the logger uses an extensible "Event" class as the root of its work, then an event-driven application can simply log events; a filter and formatter can selectively display them. The filter can be initialized with runtime parameters to allow flexible debugging. Since each Event instance can carry a severity code, the filter can use this information instead (or in addition to) the subject. Since Python is so flexible, a well-designed logger can tolerate Strings instead of Event instances and work well with those, also. Strings, however, would require additional parameters (or method names) for severity, subject, timestamp, etc. For example, a commonly-used subclass of the logger would log Exception and Error instances. The class information would indicate the severity in an obvious way. > > > My vote is: > > A) existing exception/error hierarchy > > B) LogEvent class with message string, subject string, > delayed > > introspection, automatic timestamping. LogEventObject and > > LogEventEnviron subclasses would drag around additional > details. > > C) The logger would filter LogEvents based on subject > string. > > Since each event is an object, the logger could simply > pickle > > the events, or write their __str__ to stderr. > [snip] A) Keep separate log events from the exception/error hierarchy. However, the a subclass of the standard logger should gracefully accept instances of Error or Exception and log them B) I'm lobbying for an Event or LogEvent class to allow an an application to work with events as first-class objects, especially if this becomes core to the Python library. ===== -- S. Lott, CCP :-{) S_LOTT@YAHOO.COM http://www.mindspring.com/~slott1 Buccaneer #468: KaDiMa Macintosh user: drinking upstream from the herd. __________________________________________________ Do You Yahoo!? LAUNCH - Your Yahoo! Music Experience http://launch.yahoo.com From martin@v.loewis.de Thu May 16 20:49:10 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 16 May 2002 21:49:10 +0200 Subject: [Python-Dev] Re: Minimal GCC/Linux shared lib + EH bug example In-Reply-To: <200205161833.g4GIXIq48566@boa.lbl.gov> References: <200205161833.g4GIXIq48566@boa.lbl.gov> Message-ID: "Ralf W. Grosse-Kunstleve" writes: > If this reliably solves the problem it will work for us. However, it is > not ideal that sys.setdlopenflags is a one-size-fits-all global > setting. It would be better if the user could have more fine-grained > control, e.g. package-level control to be used in the __init__.py > files. It's not reliable. It will break if you have independent extensions that use the same symbol to denote different things. Regards, Martin From vinay_sajip@red-dove.com Thu May 16 22:43:38 2002 From: vinay_sajip@red-dove.com (Vinay Sajip) Date: Thu, 16 May 2002 22:43:38 +0100 Subject: [Python-Dev] PEP282 and the warnings framework References: <20020515141648.9926.76949.Mailman@mail.python.org> <3CE283FE.8000403@campuspipeline.com> <20020515201342.T28033@prim.han.de> <3CE2AE95.4030900@campuspipeline.com> <20020516003321.U28033@prim.han.de> <3CE2FD51.2060301@campuspipeline.com> <3CE39B83.40006@livinglogic.de> <007501c1fcee$fa693760$652b6992@alpha> <3CE3FC18.4080509@livinglogic.de> Message-ID: <012f01c1fd22$bdd5b7e0$652b6992@alpha> > > The last part (about __str__) is reasonable. But I disagree with the > > statement that the LogRecord should do formatting. > > I meant only the formatting of the message itself, i.e. the > "self.msg % self.args" part. Timestamps, file/line info etc. should > be the responsiblity of the Logger/Formatter. Oh, sorry I misunderstood. I thought you meant that the self.msg % self.args was just the *default* implementation, and that subclasses might change this. > You're right, configuration should be changable at runtime. Doing this > when configuration involves custom classes is hard, but this shouldn't > be your job, but the job of those that implement these custom classes. I agree it's not my job. I only point out that it involves extra work, whoever ends up doing it. I don't want people who want to use logging in a simple way (for less complex applications) to have to do the extra work. > > You can subclass Logger and redefine makeRecord. > > Yes, that's the purpose of the factory method, but then I have to > replace the factory method that generates the factory ... Perhaps I used the wrong term? According to my reading of "Design Patterns" (Gamma, Helms et al) the term "factory method" is intended to refer to a method which encapsulates creation of an object. It's intended to be redefined in subclasses, so that they can manage their own object creation. But I get what you mean. > > ... which is done here. But this would change the LogRecord classes > that are used on a global scale. I'd like to decide which class should > be used when I'm creating the event. BTW, in setLoggerClass() you should > probably use > if not issubclass(klass, Logger): > instead of > if not (Logger in klass.__bases__): Thanks for the tip. I've updated the source to use issubclass for 0.4.5! > What I want is something like this (similar to what the warning > framework does): > > def log(event, eventtype=LogRecord): > if not isinstance(event, LogRecord): > event = eventtype(event) > ... the rest of the code I think a better design is to pass in a callable which constructs the LogRecord or a derived class thereof. This way, you can use a class, a function, a bound method which has access to other context, or whatever. def log(..., record=LogRecord): ...check to see if logging will happen... record = record(...args) self.handle(record) The callable would be called with the same arguments as Logger.makeRecord currently is - it can do what it wants with them. So you could do... def myRecordMaker(**kwargs): record = MySpecialRecord(...) record.__dict__.update(kwargs) return record and then: logger.warn("Hello, %s", "world!", record=myRecordMaker) I think this is neater and more Pythonic than the factory method :-) Let me mull this one over a bit to see if there are any other repercussions. Comments, please? Regards Vinay From vinay_sajip@red-dove.com Thu May 16 22:58:32 2002 From: vinay_sajip@red-dove.com (Vinay Sajip) Date: Thu, 16 May 2002 22:58:32 +0100 Subject: [Python-Dev] PEP282 and the warnings framework References: <20020516185252.13889.qmail@web9601.mail.yahoo.com> Message-ID: <016201c1fd24$d3011cc0$652b6992@alpha> [Steven Lott] > How does a "channel" offer subject area filtering? I'm using the term "logger" and "channel" interchangeably. You name your loggers after your subject areas. For example - "search.indexing", "search.query", "search.stats" for the search portion of an application. idxlog = logging.getLogger("search.indexing") qrylog = logging.getLogger("search.query") statlog = logging.getLogger("search.stats") > How do I get a single log with some subjects enabled > and some subjects disabled? For example, idxlog.setLevel(logging.DEBUG) qrylog.setLevel(logging.CRITICAL) statlog.setLevel(logging.ERROR) ... idxlog.info("Hello") #gets logged qrylog.error("My bad!") #gets dropped statlog.warn("Watch it!") #gets dropped statlog.error("Too late!") #gets logged and so on. > > Actually it is more than this. Much more. [Ideas on event-driven apps snipped] Some of your ideas echo those suggested by Richard Jones a little while ago - see http://mechanicalcat.net/tech/pylogging for his ideas. He took the idea as far as creating an experimental logging hub; as far as I know he had relatively few problems building on top of logging.py. Regards Vinay From mgilfix@eecs.tufts.edu Thu May 16 23:01:36 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Thu, 16 May 2002 18:01:36 -0400 Subject: [Python-Dev] PEP282 and the warnings framework In-Reply-To: <20020516025408.V28033@prim.han.de>; from pyth@devel.trillke.net on Thu, May 16, 2002 at 02:54:08AM +0200 References: <012301c1fc69$3fe7b540$652b6992@alpha> <20020516025408.V28033@prim.han.de> Message-ID: <20020516180136.A19159@eecs.tufts.edu> Sorry for joining this discussion a little late... Also, do you have a URL to the module? Perhaps I missed that post :) I don't see why we can't support both class-based logging levels and integer logging levels. The logging module can simply contain a table that maps classes to levels. Then, the application can implement filtering in the manner most convenient for it. For classes, a class set can be used for filter (with the implication that if a class is within the set, filter it and its sub classes out). Or numerically, the app can combine logging levels like flags. Sometimes it's nice to structure logging levels as such (I've used this scheme in my own apps): level1 = 1 << 0 level2 = 1 << 1 level3 = 1 << 2 debug_level = level1 & level2 logger = Logger () # Print everything less than level3 logger.set_debug_level (level3) ... and debug_level can be used as a set. Classes could work similarly by using a list. Adding a class could filter out all its subclasses, and the hierarchy can be structured from more general, harmless errors to more specific and fatal errors. ... I'm not sure I'm a big fan of having methods describe severity (as I believe I either saw in a post or read in the PEP). Some apps might want to bypass the built-in scheme entirely. I liked the design otherwise though :) -- Mike On Thu, May 16 @ 02:54, holger krekel wrote: > first off, the DebugMessage/InfoMessage/... could > have a class-level severity property (integer) if one > really wants to have it for compatibility etc. > > this leaves the question how a Message-instance/type is more > powerful than a tuple(string/integer). > > Some thoughts regarding Message-instances/types: > > - the convenient logging API and categorization features > need *not* be changed largely (but the signature of the > log method should/might be changed) > > - beeing forced to totally order all types of messages can > be difficult. You may want e.g. to log 'TimeoutMessages' > no matter what the current integer threshold might be. > why not something like: loginstance.accept(TimeoutMessage) > > - Message instances can defer the *construction* of a string > until it is really needed. (see my other posting for a code > example). This is more effective if you choose to ignore > certain messages. This has *nothing* to do with formatting. > > - customized messages are powerful and allow more advanced > filtering than with strings, especially for large apps. > a connection-logger could choose to ignore a > TimeoutMessage(socket) based on the target address of the > socket. doing this with strings is hard (and perlish:-) > > - a SocketMessage-class may hold a weakref-list of sockets allowing > to log 'sock<1>', 'sock<2>' instead of addresses which are > hard to read. stuff like this is *very* convenient. Again > there is hardly any performance penalty if SocketMessages > are ignored. > > Maybe the categorization features of the logging > API make up for some of the (relative) inflexibilities of > string/integer messages. > > But what do you really gain by restricting a message > to a (string,integer) taking my arguments into account? > > regards, > > holger > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev `-> (pyth) -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From mgilfix@eecs.tufts.edu Thu May 16 23:13:38 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Thu, 16 May 2002 18:13:38 -0400 Subject: [Python-Dev] PEP282 and the warnings framework In-Reply-To: <3CE3FC18.4080509@livinglogic.de>; from walter@livinglogic.de on Thu, May 16, 2002 at 08:36:08PM +0200 References: <20020515141648.9926.76949.Mailman@mail.python.org> <3CE283FE.8000403@campuspipeline.com> <20020515201342.T28033@prim.han.de> <3CE2AE95.4030900@campuspipeline.com> <20020516003321.U28033@prim.han.de> <3CE2FD51.2060301@campuspipeline.com> <3CE39B83.40006@livinglogic.de> <007501c1fcee$fa693760$652b6992@alpha> <3CE3FC18.4080509@livinglogic.de> Message-ID: <20020516181338.C19159@eecs.tufts.edu> On Thu, May 16 @ 20:36, Walter Dörwald wrote: > > Then use > > logging.setLoggerClass to ask the logging system to instantiate your own > > logger class. > > ... which is done here. But this would change the LogRecord classes > that are used on a global scale. I'd like to decide which class should > be used when I'm creating the event. BTW, in setLoggerClass() you should > probably use > if not issubclass(klass, Logger): > instead of > if not (Logger in klass.__bases__): Wish I had stumbled across this post before mine. +1 on this, and I'm glad to hear it'll be included. Perhaps you might find my set idea useful.. -- Mike -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From s_lott@yahoo.com Fri May 17 11:24:39 2002 From: s_lott@yahoo.com (Steven Lott) Date: Fri, 17 May 2002 03:24:39 -0700 (PDT) Subject: [Python-Dev] PEP282 and the warnings framework In-Reply-To: <016201c1fd24$d3011cc0$652b6992@alpha> Message-ID: <20020517102439.30064.qmail@web9601.mail.yahoo.com> > [Steven Lott] > > How does a "channel" offer subject area filtering? > > I'm using the term "logger" and "channel" interchangeably. You > name your > loggers after your subject areas. For example - > "search.indexing", > "search.query", "search.stats" for the search portion of an > application. > > idxlog = logging.getLogger("search.indexing") > qrylog = logging.getLogger("search.query") > statlog = logging.getLogger("search.stats") > [snip] ouch. That is too cumbersome to be an essential feature. When debugging a fairly sophisticated class, some methods deserve their own subjects. This would be awfully complex to create a channel/logger instance for each interesting method. It would be much more useful to have a A) truly Singleton logger instance. B) the ability to specify subject, severity, etc. as part of the object being logged; via a simple log.log() method that accepts an Event instance. A default subject would be used for people who didn't provide one in the constructor. C) the ability to use method names or arguments to fill in part of an Event object; log.warn(), for example would fill in the module-specified warning severity; log.log(...,severity=logging.WARN) for those who prefer fewer methods with more options. D) the ability to create an Event object out of a standard Exception instance (filling in subject and severity automatically) E) the ability to construct a "this class" or "this module" accessor as shown above that would fill in the subject for people who found the "channel" model helpful. Items A-D would be the core logger, maximum flexibility, minimal implementation. Item E would add a feature to the above model that some people find useful. ===== -- S. Lott, CCP :-{) S_LOTT@YAHOO.COM http://www.mindspring.com/~slott1 Buccaneer #468: KaDiMa Macintosh user: drinking upstream from the herd. __________________________________________________ Do You Yahoo!? LAUNCH - Your Yahoo! Music Experience http://launch.yahoo.com From vinay_sajip@yahoo.co.uk Fri May 17 12:34:54 2002 From: vinay_sajip@yahoo.co.uk (Vinay Sajip) Date: Fri, 17 May 2002 12:34:54 +0100 Subject: [Python-Dev] PEP282 and the warnings framework References: <20020517102439.30064.qmail@web9601.mail.yahoo.com> Message-ID: <000501c1fd96$df9218a0$652b6992@alpha> [Steven Lott] > ouch. That is too cumbersome to be an essential feature. When > debugging a fairly sophisticated class, some methods deserve > their own subjects. This would be awfully complex to create a > channel/logger instance for each interesting method. I'm not suggesting that you need to create a logger for every method. Remember, you can already see which module/file and the line number where the logging call was made. If you wanted more information, you could easily include it in the logging message - this could either be the whole of the subject, or include the subject. > It would be much more useful to have a > A) truly Singleton logger instance. Different people have different needs. You can just use the root logger. Others can create more loggers if they want. > B) the ability to specify subject, severity, etc. as part of > the object being logged; via a simple log.log() method that > accepts an Event instance. A default subject would be used for > people who didn't provide one in the constructor. I understand why some people might like this, but I don't think it's everyone's preferred way of working. The successful logging systems which inspired PEP282 do not force one to instantiate a class for the simple case. My other posts indicate how some support for your needs might be provided, without forcing everyone down the same path. > C) the ability to use method names or arguments to fill in part > of an Event object; log.warn(), for example would fill in the > module-specified warning severity; > log.log(...,severity=logging.WARN) for those who prefer fewer > methods with more options. logging.py already uses convenience methods to fill in the level. Have you looked at the module yet, or am I misunderstanding? > D) the ability to create an Event object out of a standard > Exception instance (filling in subject and severity > automatically) What do you mean by "create out of"? If you mean "inherit from", I disagree, as not all events are logically exceptions (e.g. informational events). If you mean "have as an attribute", then that's supported by existing mechanisms. "Filling in subject and severity automatically" - I can just see a plethora of disagreeing views over what severity should be used for this exception or that. Of course, all of this is possible by building on the core framework. I'd hope I've tried to make "simple things easy, and hard things possible". > E) the ability to construct a "this class" or "this module" > accessor as shown above that would fill in the subject for > people who found the "channel" model helpful. > > Items A-D would be the core logger, maximum flexibility, minimal > implementation. Item E would add a feature to the above model > that some people find useful. If you ask me, logging.py is fairly minimal when compared to (say) log4j. log4j - 24 source files, 7000 lines, over 200Kbytes - core logging + console and file handlers logging.py - 1 source file, under 2000 lines, 7Kbytes, core logging + console, file, socket, datagram, and other handlers. I have considered moving some of the less used loggers out of the core, but even with them in it's still pretty small. No, don't all disagree at once!! Just trying to show off :-) Regards Vinay _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From walter@livinglogic.de Fri May 17 12:39:29 2002 From: walter@livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Fri, 17 May 2002 13:39:29 +0200 Subject: [Python-Dev] PEP282 and the warnings framework References: <20020515141648.9926.76949.Mailman@mail.python.org> <3CE283FE.8000403@campuspipeline.com> <20020515201342.T28033@prim.han.de> <3CE2AE95.4030900@campuspipeline.com> <20020516003321.U28033@prim.han.de> <3CE2FD51.2060301@campuspipeline.com> <3CE39B83.40006@livinglogic.de> <007501c1fcee$fa693760$652b6992@alpha> <3CE3FC18.4080509@livinglogic.de> <012f01c1fd22$bdd5b7e0$652b6992@alpha> Message-ID: <3CE4EBF1.1030303@livinglogic.de> Vinay Sajip wrote: >>>The last part (about __str__) is reasonable. But I disagree with the >>>statement that the LogRecord should do formatting. >> >>I meant only the formatting of the message itself, i.e. the >>"self.msg % self.args" part. Timestamps, file/line info etc. should >>be the responsiblity of the Logger/Formatter. > > Oh, sorry I misunderstood. I thought you meant that the self.msg % self.args > was just the *default* implementation, and that subclasses might change > this. > >>You're right, configuration should be changable at runtime. Doing this >>when configuration involves custom classes is hard, but this shouldn't >>be your job, but the job of those that implement these custom classes. > > I agree it's not my job. I only point out that it involves extra work, > whoever ends up doing it. I don't want people who want to use logging in a > simple way (for less complex applications) to have to do the extra work. They don't have to as long as they don't use custom LogRecord. > [...] > Perhaps I used the wrong term? According to my reading of "Design Patterns" > (Gamma, Helms et al) the term "factory method" is intended to refer to a > method which encapsulates creation of an object. It's intended to be > redefined in subclasses, so that they can manage their own object creation. Exactly, but then have have to same problem one level up. > But I get what you mean. I just did't find setLoggerClass(). >>... which is done here. But this would change the LogRecord classes >>that are used on a global scale. I'd like to decide which class should >>be used when I'm creating the event. BTW, in setLoggerClass() you should >>probably use >> if not issubclass(klass, Logger): >>instead of >> if not (Logger in klass.__bases__): > > > Thanks for the tip. I've updated the source to use issubclass for 0.4.5! > >>What I want is something like this (similar to what the warning >>framework does): >> >>def log(event, eventtype=LogRecord): >> if not isinstance(event, LogRecord): >> event = eventtype(event) >> ... the rest of the code > > > I think a better design is to pass in a callable which constructs the > LogRecord or a derived class thereof. This way, you can use a class, a > function, a bound method which has access to other context, or whatever. This will work in my version without a problem, because a function is not an instance of LogRecord. > def log(..., record=LogRecord): > ...check to see if logging will happen... > record = record(...args) > self.handle(record) OK, but when you pass in something that is already a LogRecord instance you have a problem with the "record = record(...args)" call. And maybe the LogRecord instance would like to decide about whether logging should happen or not. > The callable would be called with the same arguments as Logger.makeRecord > currently is - it can do what it wants with them. But why do you want to move the construction of the LogRecord instance into the logger code. Instead of passing the constructor arguments of the LogRecord and the type/callable to the logger and let the logger create the LogRecord instance, why shouldn't it be possible to pass in a LogRecord instance? > So you could do... > > def myRecordMaker(**kwargs): > record = MySpecialRecord(...) > record.__dict__.update(kwargs) > return record > > and then: > > logger.warn("Hello, %s", "world!", record=myRecordMaker) So go one step further and allow: logger.warn(myRecordMaker("Hello, %s", "world!")) (I assume that MySpecialRecord is derived from LogRecord). Either the LogRecord constructor should be responsible for creating the timestamp and finding file/line info. etc. or this could be done by the logger (via calling a method on the LogRecord instance). > I think this is neater and more Pythonic than the factory method :-) Let me > mull this one over a bit to see if there are any other repercussions. > Comments, please? Bye, Walter Dörwald From vinay_sajip@yahoo.co.uk Fri May 17 13:24:32 2002 From: vinay_sajip@yahoo.co.uk (Vinay Sajip) Date: Fri, 17 May 2002 13:24:32 +0100 Subject: [Python-Dev] PEP282 and the warnings framework References: <20020515141648.9926.76949.Mailman@mail.python.org> <3CE283FE.8000403@campuspipeline.com> <20020515201342.T28033@prim.han.de> <3CE2AE95.4030900@campuspipeline.com> <20020516003321.U28033@prim.han.de> <3CE2FD51.2060301@campuspipeline.com> <3CE39B83.40006@livinglogic.de> <007501c1fcee$fa693760$652b6992@alpha> <3CE3FC18.4080509@livinglogic.de> <012f01c1fd22$bdd5b7e0$652b6992@alpha> <3CE4EBF1.1030303@livinglogic.de> Message-ID: <001101c1fd9d$cea1fd60$652b6992@alpha> > They don't have to as long as they don't use custom LogRecord. Yes, but by offering a class-based interface (user-created LogRecords) I am more or less encouraging it, since people would not expect adverse effects just by deriving a class. > >>def log(event, eventtype=LogRecord): > >> if not isinstance(event, LogRecord): > >> event = eventtype(event) > >> ... the rest of the code > > > > > > I think a better design is to pass in a callable which constructs the > > LogRecord or a derived class thereof. This way, you can use a class, a > > function, a bound method which has access to other context, or whatever. > > This will work in my version without a problem, because > a function is not an instance of LogRecord. True enough, though it's more by accident than design. > > def log(..., record=LogRecord): > > ...check to see if logging will happen... > > record = record(...args) > > self.handle(record) > > OK, but when you pass in something that is already a LogRecord instance > you have a problem with the "record = record(...args)" call. And The semantics of my suggestion is - pass in a callable which returns an instance. It wasn't intended to be used for passing in an instance. A class is just a convenient callable. > maybe the LogRecord instance would like to decide about whether > logging should happen or not. There's already three places where decisions about "whether to process an event" take place - loggers, handlers and filters. I'm already concerned about having three, and I'm not sure its a good idea to have a fourth! Some argument can be made for having the three: Logger level filtering to control verbosity based on application area - offers simplicity for best performance Handler level filtering to control verbosity as shown to a given audience Filter level filtering to support more flexible filtering, at some cost to performance > > The callable would be called with the same arguments as Logger.makeRecord > > currently is - it can do what it wants with them. My idea is still incomplete - I still haven't considered what the callable should be called with (my last suggestion - the same as Logger.makeRecord is, I think, too clunky). > But why do you want to move the construction of the LogRecord > instance into the logger code. Instead of passing the constructor > arguments of the LogRecord and the type/callable to the logger > and let the logger create the LogRecord instance, why shouldn't > it be possible to pass in a LogRecord instance? I'm not yet sure what's best. The main reason why I wanted to control LogRecord creation was that some aspects of this creation need to be thread-safe: particularly the calls which walk the stack to find the filename/lineno of the caller. But I could make this a method of LogRecord which is called when the Logger actually starts to process the LogRecord. The reason I didn't do this initially was that I viewed a LogRecord as just a home for the event attributes, little more than a dictionary. > > logger.warn("Hello, %s", "world!", record=myRecordMaker) > > So go one step further and allow: > logger.warn(myRecordMaker("Hello, %s", "world!")) It's more than just a step further. In my case, it's an optional argument. In your case, everyone has to do it that way. I respect your needs, but also the needs of those who just want to pass a format string and some optional arguments. After all, most instances of use will probably follow the simpler idiom. > (I assume that MySpecialRecord is derived from LogRecord). > Either the LogRecord constructor should be responsible > for creating the timestamp and finding file/line info. etc. > or this could be done by the logger (via calling a method > on the LogRecord instance). Yes, I had the same idea, mentioned earlier in this post. Still thinking on it :-) Regards Vinay _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From s_lott@yahoo.com Fri May 17 15:21:56 2002 From: s_lott@yahoo.com (Steven Lott) Date: Fri, 17 May 2002 07:21:56 -0700 (PDT) Subject: [Python-Dev] PEP282 and the warnings framework In-Reply-To: <000501c1fd96$df9218a0$652b6992@alpha> Message-ID: <20020517142156.36404.qmail@web9602.mail.yahoo.com> > [Steven Lott] > > ouch. That is too cumbersome to be an essential feature. > When > > debugging a fairly sophisticated class, some methods deserve > > their own subjects. This would be awfully complex to create > a > > channel/logger instance for each interesting method. > [Vijay] > I'm not suggesting that you need to create a logger for every > method. > Remember, you can already see which module/file and the line > number where > the logging call was made. If you wanted more information, you > could easily > include it in the logging message - this could either be the > whole of the > subject, or include the subject. I think your previous example showed that subject-level filtering required a separate logger/channel per subject. Is there another mechanism for subject tracking? The point here is that subjects can (and often are) as fine-grained as individual methods of a class. Sometimes they are even more fine-grained when dealing with fairly complex algorithms, although this is rare. When I need fine-grained subjects management, I don't want to have many logger/channel instances. I want to have one channel with a subject attached to each loggable Event. [Steve] > > B) the ability to specify subject, severity, etc. as part of > > the object being logged; via a simple log.log() method that > > accepts an Event instance. A default subject would be used > for > > people who didn't provide one in the constructor. [Vijay] > I understand why some people might like this, but I don't > think it's > everyone's preferred way of working. The successful logging > systems which > inspired PEP282 do not force one to instantiate a class for > the simple case. > My other posts indicate how some support for your needs might > be provided, > without forcing everyone down the same path. Agreed. My position is that an Event instance should be created, even when someone uses the convenience methods available under PEP282. The core function is to track Events; even when done through convenience functions that make it appear as though logging is simply tracking a string. [Steve] > > > D) the ability to create an Event object out of a standard > > Exception instance (filling in subject and severity > > automatically) [Vijay] > What do you mean by "create out of"? If you mean "inherit > from", I disagree, [snip] Agreed, loggable Events are not Exceptions, but the information in a loggable Event would be derived from the attributes of an Exception The constructor for a loggable Event would accept an Exception as an argument and create a new loggable Event from the Exception's attributes. [Vijay] > "Filling in subject and severity automatically" - > I can just see > a plethora of disagreeing views over what severity should be > used for this > exception or that. [snip] Precisely the point - severity is a difficult concept, not core to the way logging works. It is only one of the many attibutes of a loggable Event; the primary purpose of logging is to accept Events, filtering them and pickling them for future analysis. The pickling can be by formatting for printing or by storage to some other file system or RDBMS (or broadcasting through a network or whatever). [snip] ===== -- S. Lott, CCP :-{) S_LOTT@YAHOO.COM http://www.mindspring.com/~slott1 Buccaneer #468: KaDiMa Macintosh user: drinking upstream from the herd. __________________________________________________ Do You Yahoo!? LAUNCH - Your Yahoo! Music Experience http://launch.yahoo.com From vinay_sajip@yahoo.co.uk Fri May 17 16:04:34 2002 From: vinay_sajip@yahoo.co.uk (Vinay Sajip) Date: Fri, 17 May 2002 16:04:34 +0100 Subject: [Python-Dev] PEP282 and the warnings framework References: <20020517142156.36404.qmail@web9602.mail.yahoo.com> Message-ID: <001a01c1fdb4$2a038960$652b6992@alpha> [Steve] > I think your previous example showed that subject-level > filtering required a separate logger/channel per subject. > Is there another mechanism for subject tracking? Yes, you can use Filters. Attach one or more Filter instances to the root logger and they can allow selective processing of certain events, based on any criteria you choose. The Filter implements a method which is called with a LogRecord. In my system, the LogRecord is analogous to what you term an Event. > The point here is that subjects can (and often are) as > fine-grained as individual methods of a class. Sometimes > they are even more fine-grained when dealing with fairly > complex algorithms, although this is rare. > > When I need fine-grained subjects management, I don't want > to have many logger/channel instances. I want to have one > channel with a subject attached to each loggable Event. Then you can do... logger = logging.getLogger("") #root logger logger.log(logging.INFO, "subject 1", ...) logger.log(logging.WARN, "subject 2", ...) etc. A filter can process these events: class SubjectFilter(Filter): def filter(self, record): if record.msg == "subject 1": ... elif record.msg == "subject 2": ... My example's a bit crude, but you get the picture. > [Vinay] > > I understand why some people might like this, but I don't > > think it's > > everyone's preferred way of working. The successful logging > > systems which > > inspired PEP282 do not force one to instantiate a class for > > the simple case. > > My other posts indicate how some support for your needs might > > be provided, > > without forcing everyone down the same path. [Steve] > Agreed. My position is that an Event instance should be > created, even when someone uses the convenience methods > available under PEP282. The core function is to track > Events; even when done through convenience functions that > make it appear as though logging is simply tracking a string. This is what happens now, except that the class is called LogRecord rather than Event. > Agreed, loggable Events are not Exceptions, but the information > in a loggable Event would be derived from the attributes > of an Exception This is what happens now, except that in the default case, the logging system calls sys.exc_info() automatically when it knows it is called in the context of an exception. The web page has more details: http://www.red-dove.com/python_logging.html The distribution also has numerous examples of logging exceptions. > The constructor for a loggable Event would accept an > Exception as an argument and create a new loggable Event > from the Exception's attributes. This is not exactly how it happens, but the effect is analogous. Currently, the LogRecord constructor is not exposed to users (since it was meant as an internal class); this may change in the future. > [Vinay] > > "Filling in subject and severity automatically" - > > I can just see > > a plethora of disagreeing views over what severity should be > > used for this > > exception or that. [Steve] > Precisely the point - severity is a difficult concept, not > core to the way logging works. It is only one of the many My point wasn't that severity is a difficult concept, just that answers to the question "How important?" are very often subjective. It may be that in a given scenario no two developers agree, and neither is wrong. The difficulty is in the subjectivity of the severity :-) > attibutes of a loggable Event; the primary purpose of logging > is to accept Events, filtering them and pickling them for > future analysis. > The pickling can be by formatting for printing or by storage > to some other file system or RDBMS (or broadcasting through > a network or whatever). The logging.py distribution has many examples of this. Regards Vinay _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From s_lott@yahoo.com Fri May 17 16:27:34 2002 From: s_lott@yahoo.com (Steven Lott) Date: Fri, 17 May 2002 08:27:34 -0700 (PDT) Subject: [Python-Dev] PEP282 and the warnings framework In-Reply-To: <001a01c1fdb4$2a038960$652b6992@alpha> Message-ID: <20020517152734.30051.qmail@web9606.mail.yahoo.com> > [Steve] > > I think your previous example showed that subject-level > > filtering required a separate logger/channel per subject. [snip] > > > The point here is that subjects can (and often are) as > > fine-grained as individual methods of a class. Sometimes > > they are even more fine-grained when dealing with fairly > > complex algorithms, although this is rare. > > > > When I need fine-grained subjects management, I don't want > > to have many logger/channel instances. I want to have one > > channel with a subject attached to each loggable Event. > > Then you can do... > > logger = logging.getLogger("") #root logger > > logger.log(logging.INFO, "subject 1", ...) > logger.log(logging.WARN, "subject 2", ...) > > etc. > > A filter can process these events: > > class SubjectFilter(Filter): > def filter(self, record): > if record.msg == "subject 1": > ... > elif record.msg == "subject 2": > ... > > My example's a bit crude, but you get the picture. Yes, very clearly. This is far too cumbersome for fine-grained control of subjects. An individual logger/channel per subject is too difficult to use for large or complex applications. [snip] > > The constructor for a loggable Event would accept an > > Exception as an argument and create a new loggable Event > > from the Exception's attributes. > > This is not exactly how it happens, but the effect is > analogous. Currently, > the LogRecord constructor is not exposed to users (since it > was meant as an > internal class); this may change in the future. Excellent. Exposing the LogRecord (maybe renamed to a LogEvent in v2.x) is a very nice solution, permitting flexible creation of LogEvent instances from Exceptions. This also allows adding Subject information to the LogEvent, and doing proper fine-grained subject filtering without creating innumerable channels. [snip] ===== -- S. Lott, CCP :-{) S_LOTT@YAHOO.COM http://www.mindspring.com/~slott1 Buccaneer #468: KaDiMa Macintosh user: drinking upstream from the herd. __________________________________________________ Do You Yahoo!? LAUNCH - Your Yahoo! Music Experience http://launch.yahoo.com From pyth@devel.trillke.net Fri May 17 17:37:02 2002 From: pyth@devel.trillke.net (holger krekel) Date: Fri, 17 May 2002 18:37:02 +0200 Subject: [Python-Dev] PEP282 and the warnings framework In-Reply-To: <001a01c1fdb4$2a038960$652b6992@alpha>; from vinay_sajip@yahoo.co.uk on Fri, May 17, 2002 at 04:04:34PM +0100 References: <20020517142156.36404.qmail@web9602.mail.yahoo.com> <001a01c1fdb4$2a038960$652b6992@alpha> Message-ID: <20020517183702.E23086@prim.han.de> Vinay Sajip wrote: > [Steve] > > I think your previous example showed that subject-level > > filtering required a separate logger/channel per subject. > > Is there another mechanism for subject tracking? > > Yes, you can use Filters. Attach one or more Filter instances to the root > logger and they can allow selective processing of certain events, based on > any criteria you choose. The Filter implements a method which is called with > a LogRecord. In my system, the LogRecord is analogous to what you term an > Event. this is an interesting point. LogRecord is the basic unit of the current logging. But you can not pass customized (subclassed) LogRecords without them beeing wrapped (read: contained) in another LogRecord instance. Thus you have to e.g. access 'record.msg' in filters. Why two instances where one is enough, more flexible and pretty straightforward? In addition you cannot currently do *early* filtering based on anything else than 'severity'. I find that strange considering that the logging-api would hardly loose its convenience (warn/error/...) if you made LogRecord or better logging.Event the basic working unit. Early severity checking could still be done in these methods. The log-method though should work with Events & subjects (and a default factory if the first object isn't already an Event). This is by no means a complete concept but i think it's worthwhile to consider. have fun, holger From faassen@vet.uu.nl Fri May 17 19:38:08 2002 From: faassen@vet.uu.nl (Martijn Faassen) Date: Fri, 17 May 2002 20:38:08 +0200 Subject: [Python-Dev] EuroPython speakers? Message-ID: <20020517183808.GA22347@vet.uu.nl> Hi there, As hopefully you all know, we're organizing the EuroPython conference, to be held June 26-28, 2002 in Charleroi Belgium. For a lot more information, see here: http://www.europython.org There still are some empty slots for speakers on the Python Language track (and Python Applications track speakers are also still being considered though that'll be more tight). This track focuses on the Python Language itself. We already have scheduled talks about Psyco, Stackless and Jython by their respective developers, and Guido will be giving a keynote. If you'd like to be a speaker, please mail me ASAP with a short abstract on what your talk will be about as well as a short bio of yourself. Your talk should be 30 to 40 minutes. Speakers get free entrance, but of course you all don't care about that and just want to discuss some spiffy technical issue very badly. ;) Hope to see you there! Regards, Martijn From vinay_sajip@yahoo.co.uk Fri May 17 20:00:00 2002 From: vinay_sajip@yahoo.co.uk (Vinay Sajip) Date: Fri, 17 May 2002 20:00:00 +0100 Subject: [Python-Dev] PEP282 and the warnings framework References: <20020517142156.36404.qmail@web9602.mail.yahoo.com> <001a01c1fdb4$2a038960$652b6992@alpha> <20020517183702.E23086@prim.han.de> Message-ID: <000901c1fdd5$0d9c6820$652b6992@alpha> > this is an interesting point. LogRecord is the basic unit of the current logging. > But you can not pass customized (subclassed) LogRecords without them > beeing wrapped (read: contained) in another LogRecord instance. Thus > you have to e.g. access 'record.msg' in filters. Why two instances > where one is enough, more flexible and pretty straightforward? There aren't two instances. At the moment, you pass in a level, a string and some optional arguments. These get parcelled up into a LogRecord which is passed between loggers, handlers and filters. At present the LogRecord is an internal class only; this debate is (at least partly) about whether the class should be available for public use. > In addition you cannot currently do *early* filtering based > on anything else than 'severity'. I find that strange considering > that the logging-api would hardly loose its convenience (warn/error/...) You've got it the wrong way round; severity is *defined* as a measure of importance the developer assigns to the event. It's not imposed by the logging system, merely used by it when dispatching the event. The logging system merely constrains these measures to be integers, and provides some commonly used integer severity values with friendly names, and with convenience methods and builtin filtering mechanisms for them. Integer values, besides being easy to use, offer less processing overhead. The prevalent use and popularity of log4j and similar systems underlines the popularity of integer based severities; many man-years of trial and error and practical experience have gone into their current design, and many good brains have worked on them. (But please feel free to point to other systems in widespread use which use your kind of approach, so that I can look into what practical experience has shown *their* implementors.) If you want to use criteria other than numeric severities, logging.py makes this possible for you. It does not make this as convenient as possible for you at the expense of a simpler interface for everyone else. If you are arguing for the *default* filtering mechanism to be something other than integer levels, I am not sure I have the inclination to rewrite the PEP and the implementation to accommodate what seems to be a minority view. > if you made LogRecord or better logging.Event the basic working unit. > Early severity checking could still be done in these methods. The > log-method though should work with Events & subjects (and a default > factory if the first object isn't already an Event). I don't see how the checking could get any earlier.The first thing checked by a logger is the severity of the event against the severity threshold configured for that logger. Any earlier, and you're in your own code deciding whether to call the logging API or not. If you need to find what the severity of a logger is before you call a logging method on it, there's always getEffectiveLevel(). Thinking about this thread, there are two strands: the first talks about changing the interface - the method signatures - to the logging API. The second comes from wanting to be able to pass other, arbitrary state into an event, which can perhaps be used by user-defined loggers, handlers and filters. To me, the second strand is valid; the first is not, as it relates not to functional requirements, but esthetics and personal preferences. Looking at the second strand, there are two approaches which spring to mind: allow users to inherit from LogRecord, and pass that in to the logging API, or allow them to pass some other arbitrary instance into a logging call, which they can query in their own subclasses of Logger/Handler/Filter. With the second approach, the passed in instance becomes an attribute of the LogRecord for that event. It might appear that allowing subclasses of LogRecord is the more natural thing to do. But this has the potential to stop me from extending LogRecord's functionality in the future, as some new attribute I introduce might conflict with the same-named attribute in some user's LogRecord-derived class. Since I want to preserve this freedom for myself and other maintainers of logging, the best option seems to be: allow another instance of an arbitrary class to hold whatever additional information the caller wants. Next question: how to pass in this information? two choices spring to mind: 1. Use a keyword argument, extra_info=None. You can use this on any call such as warn(), debug() etc. just as exc_info is currently used. A passed in value would become the extra_info attribute of the LogRecord. 2. Use the "msg" parameter, which all logging calls already have, to have the semantics such that it is either a format string, or an instance such that str(msg) returns the desired format string. For the simple case, just pass in a string. For the more complex case, pass an instance - the caller can put any state in there that they want. It will not be used by the core logging module, except to call __str__() on it for the format string. But it can be used by user-derived classes, which will find it in the msg attribute of the LogRecord. This might seem "hackish" to some, but it's not really - it's just a question of packaging. Don't want multiple logging channels? Easy - just use the root logger. Don't want to use integer-based severities? Easy. Just never set any severity on any logger, and implement your own dispatch logic in a handler or filter, and call the logger with any severity. After a quick level check, which will succeed, any filter you configure for a logger will be called. There, you can do your own thing. You have access to both the LogRecord and your own additional instance which implements whatever behaviour you want. Slightly OT: There's a lot of people who think that the logging system should be a completely generalized system for event generation and processing. I've shied away from using a word like "Event", preferring "LogRecord" which tries to indicate that this is a focused class for a specific job. Some of what you want seems to point in the same direction, though that may not be your intention. I disagree with this - the logging system should be a simple system which allows developers, both novice and experienced, to flag events of interest in an application, and for different audiences for those events (whether developers themselves, support staff or end users) to decide which of the events they want to see. Developers get to say "what happened" (message/args), "where it happened" (logger name), "when it happened" (whenever the logging call is made) and "how important" (as a numeric level). The audiences (or their mediator who configures the logging system) get to decide which of of the audiences gets to see events for particular values of "where" and "how important". The wider ranging event generation system has its place, but I don't think that place is in the core Python distribution. Maybe in Python 3000 :-) Regards Vinay _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From s_lott@yahoo.com Fri May 17 20:28:33 2002 From: s_lott@yahoo.com (Steven Lott) Date: Fri, 17 May 2002 12:28:33 -0700 (PDT) Subject: [Python-Dev] PEP282 and the warnings framework In-Reply-To: <000901c1fdd5$0d9c6820$652b6992@alpha> Message-ID: <20020517192834.96368.qmail@web9605.mail.yahoo.com> [Vinay] > The prevalent use and popularity of log4j and similar systems > underlines the > popularity of integer based severities; many man-years of > trial and error > and practical experience have gone into their current design, > and many good > brains have worked on them. This doesn't make it the best design. Most of the numeric coding method was used because old computers were slow and small and simple integers (often only single bytes) were all that were practical. For example, Unix process termination status was only 7 of the 8 bits of a single bytes. This wealth of experience doesn't make this an optimal design; merely convenient for the available languages and processors. I recall vividly the arguments surrounding the 32-bit status codes in the VAX architecture. (Back when DEC was a powerhouse, not the forsaken child of the forsaken child of HP) Too long, too complex, too much overhead processing 4 bytes when only the 3 status code bits were really relevant. I respect your passion for a complete implementation of the PEP. I submit that the next generation should get past this "old/small" system technique and broaden the approach to use the full power of the Python class capabilities. Please look closely at the TIBCO subject-based messaging architecture. ===== -- S. Lott, CCP :-{) S_LOTT@YAHOO.COM http://www.mindspring.com/~slott1 Buccaneer #468: KaDiMa Macintosh user: drinking upstream from the herd. __________________________________________________ Do You Yahoo!? LAUNCH - Your Yahoo! Music Experience http://launch.yahoo.com From pyth@devel.trillke.net Fri May 17 21:06:20 2002 From: pyth@devel.trillke.net (holger krekel) Date: Fri, 17 May 2002 22:06:20 +0200 Subject: [Python-Dev] PEP282 and the warnings framework In-Reply-To: <000901c1fdd5$0d9c6820$652b6992@alpha>; from vinay_sajip@yahoo.co.uk on Fri, May 17, 2002 at 08:00:00PM +0100 References: <20020517142156.36404.qmail@web9602.mail.yahoo.com> <001a01c1fdb4$2a038960$652b6992@alpha> <20020517183702.E23086@prim.han.de> <000901c1fdd5$0d9c6820$652b6992@alpha> Message-ID: <20020517220620.F23086@prim.han.de> [trying to strip the discussion down] Vinay Sajip wrote: > > this is an interesting point. LogRecord is the basic unit of the current > logging. > > But you can not pass customized (subclassed) LogRecords without them > > beeing wrapped (read: contained) in another LogRecord instance. Thus > > you have to e.g. access 'record.msg' in filters. Why two instances > > where one is enough, more flexible and pretty straightforward? > > There aren't two instances. and > ... > Next question: how to pass in this information? two choices spring to mind: > > 1. Use a keyword argument, extra_info=None. (...) A passed in > value would become the extra_info attribute of the LogRecord. > > 2. Use the "msg" parameter, which all logging calls already have (...) > it can be used by user-derived classes, which will find it in the msg attribute > of the LogRecord. are not very consistent statements . > Slightly OT: There's a lot of people who think that the logging system > should be a completely generalized system for event generation and > processing. I've shied away from using a word like "Event", preferring > "LogRecord" which tries to indicate that this is a focused class for a > specific job. You will find in almost every major software system that events are generated, filtered and dispatched. I don't see the a problem if a logging module would live to these realities. Just because unix/sysloging or other software has evolved with string/severity-logging doesn't make it todays choice. IMO python's ease of working with classes/types *makes* a difference. In distributed transaction systems logging even provides some ACID-guarantees as it is used for driving the 2PC-protocol. Think about it a minute. I don't say that python's logging must do this in the standard lib. But arguing with 'syslog' or 'log4j' seems to be a quite narrow view. I appreciate your efforts but still disagree with your design decisions. This should not prevent us to make a break discussing these issues. Everybody (including us) has probably heard enough arguments. regards & thanks, holger From vinay_sajip@yahoo.co.uk Fri May 17 23:15:31 2002 From: vinay_sajip@yahoo.co.uk (Vinay Sajip) Date: Fri, 17 May 2002 23:15:31 +0100 Subject: [Python-Dev] PEP282 and the warnings framework References: <20020517192834.96368.qmail@web9605.mail.yahoo.com> Message-ID: <001001c1fdf0$5dc2a420$652b6992@alpha> > [Vinay] > > The prevalent use and popularity of log4j and similar systems > > underlines the > > popularity of integer based severities; many man-years of > > trial and error > > and practical experience have gone into their current design, > > and many good > > brains have worked on them. [Steve] > This doesn't make it the best design. Most of the numeric > coding method was used because old computers were slow and > small and simple integers (often only single bytes) were > all that were practical. I agree it doesn't make it the best design. But I don't think numeric coding is used *just* because its space and time efficient - it's also more intuitive. I don't think log4j is like it is because of platform limitations. > For example, Unix process termination status was only > 7 of the 8 bits of a single bytes. > > This wealth of experience doesn't make this an optimal design; > merely convenient for the available languages and processors. I'm not sure "optimal" can be applied in this kind of context, in the true sense. Depending on the kinds of applications you and I work on, "optimal" for you might be different to "optimal" for me. > I respect your passion for a complete implementation of the > PEP. I submit that the next generation should get past this > "old/small" system technique and broaden the approach to use > the full power of the Python class capabilities. Please don't get me wrong. I'm not averse to using a new approach just because it's new, nor is the present approach chosen because of "old/small". The posts in this thread have got me thinking, but I don't know how long it'll take me to "get it". I *still* can't see what it is you'd like to achieve, but which you can't with the current functionality. A specific example from a real application would sure help :-) > Please look closely at the TIBCO subject-based messaging > architecture. Thanks. I will, time permitting - I'm not intimately familiar with the TIBCO system but IIRC their messaging *is* the product - it's a full-blown middleware layer. Probably much bigger than I can develop in my spare time :-) Regards, and thanks for your patience... Vinay _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From vinay_sajip@yahoo.co.uk Fri May 17 23:32:48 2002 From: vinay_sajip@yahoo.co.uk (Vinay Sajip) Date: Fri, 17 May 2002 23:32:48 +0100 Subject: [Python-Dev] PEP282 and the warnings framework References: <20020517142156.36404.qmail@web9602.mail.yahoo.com> <001a01c1fdb4$2a038960$652b6992@alpha> <20020517183702.E23086@prim.han.de> <000901c1fdd5$0d9c6820$652b6992@alpha> <20020517220620.F23086@prim.han.de> Message-ID: <002001c1fdf2$c8164460$652b6992@alpha> > [trying to strip the discussion down] > Vinay Sajip wrote: > > > this is an interesting point. LogRecord is the basic unit of the current > > logging. > > > But you can not pass customized (subclassed) LogRecords without them > > > beeing wrapped (read: contained) in another LogRecord instance. Thus > > > you have to e.g. access 'record.msg' in filters. Why two instances > > > where one is enough, more flexible and pretty straightforward? > > > > There aren't two instances. > > and > > > ... > > Next question: how to pass in this information? two choices spring to mind: > > > > 1. Use a keyword argument, extra_info=None. (...) A passed in > > value would become the extra_info attribute of the LogRecord. > > > > 2. Use the "msg" parameter, which all logging calls already have (...) > > it can be used by user-derived classes, which will find it in the msg attribute > > of the LogRecord. > > are not very consistent statements . No, because the first is saying how it *is*, the second is saying how it *could be*. You see how hard I'm trying to accommodate you? > > Slightly OT: There's a lot of people who think that the logging system > > should be a completely generalized system for event generation and > > processing. I've shied away from using a word like "Event", preferring > > "LogRecord" which tries to indicate that this is a focused class for a > > specific job. > > You will find in almost every major software system that events are > generated, filtered and dispatched. I don't see the a problem if a > logging module would live to these realities. Just because unix/sysloging If it's a beneficial side effect you get for free, fine. If it imposes additional work for the simplest case, not so fine. > or other software has evolved with string/severity-logging doesn't > make it todays choice. IMO python's ease of working with classes/types > *makes* a difference. Yes, but you are missing the point that you *can* have flexibility of classe s/types etc., and in my last post I just gave one example of how it could be done. I'd prefer if you gave a specific critique of the approach I suggested, indicating why it wouldn't work. For example, a particular problem you are trying to solve which you just can't, with the existing functionality or that proposed in my last post. > In distributed transaction systems logging even provides some > ACID-guarantees as it is used for driving the 2PC-protocol. > Think about it a minute. I don't say that python's logging must > do this in the standard lib. Okay, we've got some common ground here. I am not for a minute arguing that the functionality you want is somehow "wrong". It's all a question of what goes in the standard lib - at least for now, with the 2.3 release date approaching. That's my focus, it probably makes me look a bit slow-witted. > I appreciate your efforts but still disagree with your design decisions. > This should not prevent us to make a break discussing these issues. Holger, I'm sorry if you're losing patience with me. As my wife is often telling me, sometimes I just don't get it ;-( Thanks for your patience so far, Vinay _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From pyth@devel.trillke.net Sat May 18 00:01:41 2002 From: pyth@devel.trillke.net (holger krekel) Date: Sat, 18 May 2002 01:01:41 +0200 Subject: [Python-Dev] PEP282 and the warnings framework In-Reply-To: <002001c1fdf2$c8164460$652b6992@alpha>; from vinay_sajip@yahoo.co.uk on Fri, May 17, 2002 at 11:32:48PM +0100 References: <20020517142156.36404.qmail@web9602.mail.yahoo.com> <001a01c1fdb4$2a038960$652b6992@alpha> <20020517183702.E23086@prim.han.de> <000901c1fdd5$0d9c6820$652b6992@alpha> <20020517220620.F23086@prim.han.de> <002001c1fdf2$c8164460$652b6992@alpha> Message-ID: <20020518010141.G23086@prim.han.de> Vinay Sajip wrote: > > I appreciate your efforts but still disagree with your design decisions. > > This should not prevent us to make a break discussing these issues. > > Holger, I'm sorry if you're losing patience with me. i am not. it just seems better that we revive the discussion after there is some alternative example code. Will you be at EuroPython by chance? regards, holger From tismer@tismer.com Sat May 18 02:30:41 2002 From: tismer@tismer.com (Christian Tismer) Date: Sat, 18 May 2002 03:30:41 +0200 Subject: [Python-Dev] Ann: Stackless Limbo Dancing Works Fine! Message-ID: <3CE5AEC1.9000304@tismer.com> Announcement: Stackless Python At It's Best, Ever. 18-May-02: Limbo Dancing works fine! Yet some more thinking for weeks, and a few days of implementation. Now, the first implementation of channels is available. It isn't complete, but works just fine. The stackless module now supports to create channels by function channel(). A channel has the methods send() and receive(). Try it yourself! You might either want to build your own Stackless, checking out the module stackless from :pserver:anonymous@tismer.com:/home/cvs , or, for Windows users, just take the pre-built binary, extract it into some directory, and start Python or PythonWin fom there. I will for sure build an installer again, after I have got some feedback. This implementation tries to follow the concepts of the OCCAM/ALEF/LIMBO languages by supporting the channel concept as the central communication/blocking/control feature. Tasks do not communicate directly, but through channels. You might have guessed it: Stackless will try to implement/support Hoare's CSP, finally. But before, I have to understand more of it. This will happen in the next few weeks. This release is an alpha release. There are most probably a lot of omissions. Some are obvious: - There isn't yet a scheduler, like there was one for the uthread module. This is trivial to implement and will be added in almost no time. - There isn't yet an ALT/PRI ALT construct like in OCCAM/ALEF/LIMBO. This is most sophisticated and very hard to implement correctly! I will try to find out how to do this the best way, during my stay at IronPort, from May 18th to May 29. If somebody from the US would like to contact me by phone, please use the opportunity and call me via IronPort. I am employed by this company. - You shouldn't use channels from the interpreter prompt, but in scripts, only. The machinery doesn't check for changed interactive sessions yet and will most likely crash, soon. Despite of that, it seems to work nicely with PythonWin and extensions. -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From andreas@andreas-jung.com Sun May 19 13:52:27 2002 From: andreas@andreas-jung.com (Andreas Jung) Date: Sun, 19 May 2002 08:52:27 -0400 Subject: [Python-Dev] getting the UCS-2 representation of a unicode object Message-ID: <05e001c1ff34$1cf86e30$02010a0a@suxlap> Py_BuildValue() allows the usage of "u" or "u#" to convert UCS-2 data into a Python unicode object, however Py_ParseTuple() converts a unicode object to UTF-16. Is this an error in the documentation or why is there a asymmmetry in the API? Andreas --------------------------------------------------------------------- - Andreas Jung Zope Corporation - - EMail: andreas@zope.com http://www.zope.com - - "Python Powered" http://www.python.org - - "Makers of Zope" http://www.zope.org - - "Life is too short to (re)write parsers" - --------------------------------------------------------------------- From mal@lemburg.com Sun May 19 14:25:26 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Sun, 19 May 2002 15:25:26 +0200 Subject: [Python-Dev] getting the UCS-2 representation of a unicode object References: <05e001c1ff34$1cf86e30$02010a0a@suxlap> Message-ID: <3CE7A7C6.1010909@lemburg.com> Andreas Jung wrote: > Py_BuildValue() allows the usage of "u" or "u#" to convert UCS-2 > data into a Python unicode object, however Py_ParseTuple() converts > a unicode object to UTF-16. Is this an error in the documentation > or why is there a asymmmetry in the API? Sounds like a documentation bug: it should be UCS-2 since that's what's used as internal data represenation in narrow Unicode builds (the default). -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From andreas@andreas-jung.com Sun May 19 15:35:19 2002 From: andreas@andreas-jung.com (Andreas Jung) Date: Sun, 19 May 2002 10:35:19 -0400 Subject: [Python-Dev] getting the UCS-2 representation of a unicode object References: <05e001c1ff34$1cf86e30$02010a0a@suxlap> <3CE7A7C6.1010909@lemburg.com> Message-ID: <060d01c1ff42$677ec450$02010a0a@suxlap> Sounds reasonable..but since Py_ParseTuple() only applies to function arguments it can not be used to convert a unicode object to UCS-2. So what is the easiest way to get the UCS-2 representation? PyUnicode_AS_DATA() returns for u'computer' a char * with strlen()==1, however PyUnicode_GET_DATA_SIZE() on the same string returns 16 (looks fine for the two byes encoding of UCS-2). Am I missing something? Andreas ----- Original Message ----- From: "M.-A. Lemburg" To: "Andreas Jung" Cc: Sent: Sunday, May 19, 2002 09:25 Subject: Re: [Python-Dev] getting the UCS-2 representation of a unicode object > Andreas Jung wrote: > > Py_BuildValue() allows the usage of "u" or "u#" to convert UCS-2 > > data into a Python unicode object, however Py_ParseTuple() converts > > a unicode object to UTF-16. Is this an error in the documentation > > or why is there a asymmmetry in the API? > > Sounds like a documentation bug: it should be UCS-2 since that's > what's used as internal data represenation in narrow Unicode > builds (the default). > > -- > Marc-Andre Lemburg > CEO eGenix.com Software GmbH > ______________________________________________________________________ > Company & Consulting: http://www.egenix.com/ > Python Software: http://www.egenix.com/files/python/ > From fredrik@pythonware.com Sun May 19 16:23:28 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Sun, 19 May 2002 17:23:28 +0200 Subject: [Python-Dev] getting the UCS-2 representation of a unicode object References: <05e001c1ff34$1cf86e30$02010a0a@suxlap> <3CE7A7C6.1010909@lemburg.com> <060d01c1ff42$677ec450$02010a0a@suxlap> Message-ID: <003a01c1ff49$225a18f0$ced241d5@hagrid> andreas wrote: > Sounds reasonable..but since Py_ParseTuple() only applies to function > arguments it can not be used to convert a unicode object to UCS-2. > So what is the easiest way to get the UCS-2 representation? > PyUnicode_AS_DATA() returns for u'computer' a char * with strlen()==1, > however PyUnicode_GET_DATA_SIZE() on the same string returns 16 > (looks fine for the two byes encoding of UCS-2). strlen() looks for the first null byte. in a UCS-2 string containing ASCII data, every second byte will be a null byte. > Am I missing something? trust the macros, and don't use 8-bit functions on 16-bit strings. From martin@v.loewis.de Sun May 19 20:03:12 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 19 May 2002 21:03:12 +0200 Subject: [Python-Dev] getting the UCS-2 representation of a unicode object In-Reply-To: <060d01c1ff42$677ec450$02010a0a@suxlap> References: <05e001c1ff34$1cf86e30$02010a0a@suxlap> <3CE7A7C6.1010909@lemburg.com> <060d01c1ff42$677ec450$02010a0a@suxlap> Message-ID: "Andreas Jung" writes: > Sounds reasonable..but since Py_ParseTuple() only applies to > function arguments it can not be used to convert a unicode object to > UCS-2. So what is the easiest way to get the UCS-2 representation? > PyUnicode_AS_DATA() returns for u'computer' a char * with > strlen()==1, however PyUnicode_GET_DATA_SIZE() on the same string > returns 16 (looks fine for the two byes encoding of UCS-2). Am I > missing something? As Fredrik explains, you are getting what I believe you mean by "UCS-2" - you get the internal representation, which, in your build, most likely uses unsigned short as Py_UNICODE. If you are really interested in UCS-2 data, you need to use PyUnicode_EncodeUTF16, since the internal representation, when interpreted as a byte sequence, may or may not be UCS-2. Regards, Martin From daveh@cadlink.com Sun May 19 22:24:17 2002 From: daveh@cadlink.com (Dave Hawkes) Date: Sun, 19 May 2002 17:24:17 -0400 Subject: [Python-Dev] imputil modifications Message-ID: (Repost with correct attachment!) Hi, Recently I tried to use the imputil and module and noticed that it had a few shortcomings, particularly in an environment like pythonwin. I implemented a few changes/additions (applicable to python 2.2 and up) to imputil (attached in newimputil.zip) to fix these problems, without breaking its basic structure. Issues addressed: 1) Objects in sys.path can break some other modules. A function has been created to make these objects appear like strings so we can do this for example: sys.path.insert(0, StringedImporter(BuiltinImporter())) The StringedImporter function will take a class instance and return another instance derived from str that also contains the Importer functionality. 2) Chaining to previous (foreign) import handlers supported if desired. A previously installed import handler can be used as shown by this example: im = ImportManager() im.install(chain=1) chain must be true to enable chaining as it is disabled by default so current assumptions should not be broken. 3) reload is implemented. An initial crude implementation of reload is provided as a starting point. An override in the Importer class can be coded to improve it in specific cases. Here is an example of code that will work for pythonwin: #====================================== import sys, new from newimputil import * class TestImporter(Importer): def get_code(self, parent, modname, fqname): if modname == 'xyzzy': def test(): return 'magic!' m = new.module('modname') vars(m)['test'] = test return (0, m, {}) return None im = ImportManager() im.install(chain=1) sys.path.insert(0, StringedImporter(TestImporter())) import xyzzy print xyzzy.test() #======================================= Any comments as to whether these changes are worth persuing? Thanks Dave Hawkes begin 666 newimputil.zip M4$L#!!0````(`+N*LRQ2K'E@B2(``$AW```-````;F5W:6UP=71I;"YP>;T] M:W/CQI'?7>7_@%"5(^A0V%TG57>E1.NRX]U$=UZO:W?SN%NK5" QE&"!&!@` MI:53]]^OG_,`0$KK.,>4LR(PT]/3W=.OZ1G.9K-//[G8-K;MDUU?5F5?FN[3 M3S[]Y,4'?&:*9%WE76>ZLT\_2>##35_E=7YM6OC.?R7]C4E*AM*T=FTZ@N$[ M4%OZ?)5WAF$F&]LFK6FJ?%W6UTG7YW61MX7"V>SJ=5_:NF,X7^W*JB]K!^[% M=E?E?33RUJQO\KKLM@1YQ1T2@)IL6ON3J9.M+7:5<:A]O:^_L7GQ=K?9E!\4 M\*>?S) DGWYRDM2V-V=)?F=+10KQK&U]JK =0,$`_DDF/R55=765G"?O9Q$'9\N9@H$_!S2&)V_[%N9L"O?HDL#QXW?[Q@#( M'OY)Y_,%/']%] B?PY06$14*:[IZWB?WMKT-B)!E&4)FP8A0%,&;'92T;*8, M++GDR0N?$[MK$WM?9`'7M>57J@CWW&+R?7\GSJZOYY:!GB*?[ MV[HJ6RE#\0&2Q7R:]<&7@OX-6TI+P MGGB%GR8$,REGTR\^"IH3A:G'8TBFZLP!=(\QZB-Q'8(:BM^#:*Y:D]_&CU5" M6&3#-\&+P]0.I3POBJN.;*DH7_ZR%'%^"?8\% KT*D#(U[ LT""F0:O!`MW0 M>%D(?PQ9,3GY13X"2_Y)OGMS\=H M<%B"Z*UX*UGX_$&U\O+*V!W\C;]*!]7B'1A/-'@SC:-'?Y&#Q3;(#WP_] MF.LV7ZW8Z0-.HGNVQ0ZQBE**9LE%S]0P70+$`J\+R(-SMT4B8G.%(*Z*LET0 M?%1M,-9X1DSSP7R ?[J$IED)$*_&CV,@#!F-==6ETQ(,+_F/F&(7P.4RK\J? MV ITID_L1A:/Z1SMP.NSUS6V0H^6IYW%E">AR7<5M"_!-(L9*?:@,\KU*7%? M/%8@;]OU((.VJNP]B,%J'X+*FCTS($EA[KG[#H*TSM*(ZQ7,Q97.(ITP!]CV_>>@O M+L"`#QBSF8#/[)O M]P-J-GG;H[PSSUT%A378W=E>!/&M4< `A="8(QK2C M<9-WY-+H.,X\FQ8,]%!H]<.DAEERMRSL!).V\M5!)9)[7B_&4(\X$&@/=+:K M?'V[:[RSQM2"J2MOX$]4<[NZ,+"^K9UP`O#C)C!MU=.!X*K,'IT"?DY4``B/ M>\(%#(U=M&YYQ7X 05>S"E MIE9I*KZ(.XX7#3VUS94GW+[+1)>^)_:",KJ,^Y@/:]/TR7^9_8NVM1C'CF&> M`&5A26ML#T-L+1AJG<^P>8Q"&._X-ZFB,\$CL8*^\4&9&V.3W.<4N&\L2%<" M-@G44C;=O3J4S-TUV@%7DVO<;)[HO<'A'.E^( MZVC;+2C5==Z9I=>_M05D]LX#69,.FM*R'EZW6YVJCT']['J]:YV'*FQ(TM6. M0>9(!9:7O#X$U* ,P8I=Y^A EIC:VK-'NV)Q`O[K6P!:6 1ZBPKHQMX?E' + MU,\K<)5 SB"@N@4[XKRMUD3DMTB2A-V_[F>2&-SM7DB$#O#S$@P?7L7,?SH^NX\ % M&.D&?VK9=4J"7-,:B)IR) MOV>[#!7D;-CC2V %24_#20"C+C]2'>6]9X$G<4N__MN?!Q/<1I2)WZ&5*G$I M?VCS^MJDSY9>-J94GBA,];&V.NOR\J!C== 1P,]$^D([:6HJ'&/$N>TTCH^S M#=N#[#XL;2>8"/8):Q [S'X344B,Q:^%A7L#+A(%^'E99? M?.WZ>K `/];B:S_QD8*.XS@1R>73X1-<\D'#G&P6N\:Y1+9$8J3K! ,1Z\%$ MF*?_A%_,\"@B/!C^<'BH05 <];TA#+HHY"IK49T:YKF P#,\"Z7O7=!YK2WS M1#1"*N9NX9SBN@CE5'6&AT?Y! "!WBHFC4 3H!HALPXR*/"RB7'1-F!/"G<] M0!1@2FH%OH9V/#T-A#>8:3 XN$AY0#58@J>5N3/53,/@B*2#K1,4**$].D_X M-?4"AH_D[90]&*JN$P?*K>W6;##G92E9!8L&LU%N';7.UT!JY?T0&CJ\)3DB MGC"4JROOW,:7$BH%,FQRH@:L;HBDAL"@X=JT8&07K / =1)RDI?0E.M;?&8W MO++=%BR:EFR$F6QJ"=M(;]4&5QBKBU!@D4>[&A6+-X?=$"#,;,M.T?V($+(. MPQ0J=T()4XH[0KMEXTCBR_Y&""O$6)1;;>XUMY;\22;%L)\H8$F< M>$ Y%QLE:W-(:I;%.`LG8]5U9D%AXFM]0J$4N0[G>5;"4 MXA46$:]$__=T&&0>%%1BKOI>]4^YW96K"@"%G!X*9/3]_5EY M.6SZ2!8*^QS/6+*1+%$/WV&2@6&R-$A;L$W$_O%N2(=!#6)'"0&T`4@V,@*< M!RUM[>)+EZ1GNX1L!)F.EDGH%R&0VF]3V-4/9MU+UUPU5] C6/3D#.- I<=N M[+L$NZ?8>.G&FO* 15C."6[F*40[]0-WUE13T(/"C*,#A-MJ,HYNH BDR2$G M=2=2A)&Z$ M,3"T(VWXR#G$#-&(9'HR'[U]SNA-5Y <)NE1;];MZDXLQ@E:GS@ZR J;'O:X M`SV .Z$\23L$=12B%QBOR%4^O#TBN_=^3R36S/P:M8@-=#3FE&!-XS_HQ(&_ MV=@.E2LHW<(TIN8>(\VLYOO,TD^ M[KK FF$EH8B)"SZWZ'$6IC+72(VR)W*-1,$)Y\3 <7F8KPS[^)+"0L9Y:>;UMZ!-U6(S^(*"V;Q-M^0'ED\$CP30%,=B+62+E,4 >(Z!+J*I<9\HBU!0&@/7% M?*-6Q+K"-!AU>BX:P\AB3\2 4W!.J8A=0V,Y/A/(,2PM%U+B#--9O^>,N>:L M%/4^2C(HL$!::7)4CEDHD=RDEKP'U&&6)3$?&H@33#$Q3;6W0> ZBEC'>_ A!'2,8J2< M(6.(6.\$*H-T)"G$$&-.Q814[+#T2DP65C]AW_00Q2*AD+[ ._06IBG+2>+ M-DC.S%%622.+PO%`W1IONYWN2JY M+Y(TU+,Y#2RY@9!KG2_%".(^QC*LPX\IK4$4#^Y2[21B:U@"6-9%64A,'@,+ MS >S9@R0(Q$;' 8#3CBTL%NP%L;QNY_ MLE@4 W_G(@&F)]@X$0G`]QF>9ZL MP-@5Y09L&?"@VG\1+%A9]6'0(K(ZU49WPQ/Z*R18:W[ M6 (S%$-:P3\.%N#$"LB+HD37%:2!)]!CV#6DB'E\1O MNP9CP%2$=^P,ZJ)#SXUJ;MAP9!G$H[=4\G!74BSJ.T[H/1"(W+2<]^W66_[F;)KY-TZRC% MC:Q4^W+``)F#=TT]9'$Z9\*I>?J M'4F%CU\,+)S89T;4SN8C)V_^V9RV#\2_''GR/0?C047A7([^S4-(=WE;TB%$ MT&QT%"C0>IC^[1))NW0+;,*SC+U8PJ0^%#4[#_Z<9I(Z0@!AOQ_S1#[45+>2 MAZDOGL4R>7^YB/0G3A3H> 09YQZ3Y)&*95>ST>(I(ANGL,'-]%-&O9N6HRW: M;7E]TVN24/.'OPJC8OQ@HAGP^M4YT0I#B[ 8QLD(M)G*]<+C\4+QM-'E@KTG M.P-24\LF'':I@TQ T+(+`C2!'GZF5A)$\#X5R03'Q20#16O'ETG'D<61-!1H MFB]!&6V;/N!18#[=/GEO=9,J%)5WG'&@J%.M`;K^"R;7A:.-%.C&+P>[==YZ_":99U@[HQ#B;MXB MC7G-!-5^RV"8<1+V[9:\/5:E_0:=_A3;9NC)Y&Q_B(B;(%LRQ M$#8([<)1FUU/!Q5H9(CF.+$6Y'$F*,8AB(VWDX9$&T<'8TYBF6 N87 32A1MH6N\%5AE$ M+Y=[$VCU2IU%"(C6&4CA^B9"3*8<819EE)(450_[IXLPJ)G0_;)J0K6.!R:K M:G_Z(X3^?)0_< (S=Y QJ#.+T\3B&!-R.3\56O!WG`DL:TK^] M" Y!U=8)`?DITOK\W-4A^7ZAA9-"KT#?#8XY,,/V M*$!@JGO.RC@T>>,`J^#H4$J8V4^XLF\"N\Z \!52O*%UEB[#XR129:SL-9,< M8R91=1$*'N9MN%\842]4F'!$W!,>U\N3(\GB&%=M"U*IR6"9R2O.[@(5\Y8" ML]5B8I: 4^LGF9,)0 \?_%O9%7A"V0B@5]GJ%LS4/#DQ0BEQW*UP,TPD#S*D M'XRFI.%Y#\#YS4,-T -R2\)5,NSP!8;&D@?)Q:3=;GT3P\/:1UHJ`^DEB0#* M_TT=O ';U6^/H?$<1.#A_SI9O$\$`=U3<$3Y+-^ S?HL&#T&B%EN2FDS[YZ4 M[K!O<Z(@O6CFUK;9AVC&_@U^G.6/$W XAGL8>*83N='C MZ2UMA=-YR MXW ?@X#4_;W'K"C<]]WHX7O31A?A''37$IG@&^W1MMTV)6ZQ8 M^R:GQK&XH\W7Y$SIF65\!$R"&9C5#ITII.-\/4<].;?S!R Z. ""CD-#/!)" MQN[D?TOG%*NG).@`*08.;QME)XC2'[D5;^32B?(%+$C9#^SLKEUS+9_3:VQ5 MJO#>G=4^^8..\AQ7M,.;"H)#E02C,)B1S?H.=QI<'6;>@&IJVA*3\/#/'4## M-+<_J^&.O7>QKNSR.U.(T\LV/:A$1 ;RU0SB,G5<)Q_,B0CK)CWLX7&>8_])("6(1! M>UY=^O7]Z;-+RF!\7\^#Y1'!#[[\AAIZ'$C>W(U+FO![N#/*N(#6)M<(FNV_!?*3S M[Y_R_\(B+'W)EW!EZ(6D\S]V^76Y3A=1BW5E.^-#//$(F8BPL.62#V06_$FUWC_8$N^= MXB"=UG5\Y8>NY+<0)>\:T!_=7%4Q5U^1=?+U5GP>`*Q:L2-!D62+@^CWALG: M2GY=)8 @@Z'8-GN\X^#!1T.P=KFZTE8U ]>COOA M!P4"Z9_FRV24O"RQI/(<$)D/7N!'"PWB-U38#2P?`9J?S=G?`9=Y`EJ.\SU# M4Y2/<0#5=<:ZZVP*$^R;H\(ZF\N)9C.8<_OL%"(N[.6+WF1XCB+X%K0YT!L)'@,;AP0(23;Q! M<:3WW(*/?+@;A?AAH'KXX:BMTT:^0ZB@^(U74?2N[+",74V TU7?X%&]6#W! M;.4<;<:=%EK/&=N53G0C(ND!BW2P&7G]=FA&HD1G\"#%U&_R;\G39__^%#YT MJOKIT]_AWVXJSF2,I_'&5R"20S1R&;QS\/\PF0J&@0G]QR59EU_8%?ZGKO@\ MT6KAP:66J?ZA%/UY2;=#53 G$C4*'N$M7H4E9<87(DC2P*.1F3WKP:W MA@SCSI)*FD&DU?=*92)#E=+S19TEW;[TU5\NOGEW\:UOP4=,&!!3]Q%POOOO MJY=O7O_/BPC.*+8Z\(^04POJ6M5]U4H)VI*,7J$XP^ ML AD?.)VLDDYCVZ@"[E(L;N[9]2BEY#68+5VF)\[H:B75*X-<@YR)(FC MUN@1#D,9U!#&D$4+S4@'>[:#>I1.#]?&Y&Z\&W2F>H5X- MX$6G%T/M$K'PN)"XO=GF]AKFAEOPJA,?<14=?09YL+".Q VJ_-?8>-+^/.#P MZ.>C%D$0CJNNF#]Z9OPY5) SN7KPXQ*(W.+]YY>'VN#MJ4IXNO94T3W:(:?# MRY>DU5QB8&(,$8-G2T7DV:5F-Q]EI+@29:B^,'B)5-V `JC/"2,_&RJ8X*Q; MV'+Z5K(-[FF&+IT"''#@H&NG'Y_V<%VF#WRJU753=$,N&9N'%=Y#EO%Q'\V8 M_N7ERXN_GW[UY=L77R<7K[Y[_>;=BS=O)4N*RVM\<>-!C&6ZY%F?.]Z\/SO] M[:7/)7$;6$-[:$,>, %A+UA?K9$GWH^7*R;YO>3&HN0%]Y&#([351L=HZ>GS M<_HCX(7+="%<3.JM!@<"-YSC^]U"K]<,DD-#[XW"+\I:[>H@;^5 +*):%<47 M`0^PTH_,3W-<5)$19[=<2DJ@<9Y]%!D/^.!?:'914XI,!D1GF.9ZJOL4_T"5 MANU &21G#)K<.7:A)J\)/>0Z@<>!]X<.'2BZ_?26]N3\(RC:^ M*O%IPEEC:9F4)G%'[S/>D3=]F<>-YJ"HF1#LB M@7Y(:.@-RUG?#F&TIFDG(HL`%6R1>D C+ #5XP"Z_FA_.7)?KG:]<;/)1WDT MF0039K+/G-I,W/VFU9OT?@(V?@17/?_OFL8MC]O6([@I(&U9IDJ,7VXA2A8* M@H&Z=S;3W^',)<)1#7Q&SYQ"IVL2.^BF3] W8NVC&\LUMPIH4!ZXS('0B'H? M.QI!A?Z8J^&792?KRW>NO7\N?E$UX6=9U\I5=XP\MX/E%_WLM[OBJ9Y;^3 M= MU821'-W$R0#/(^'80$QCQBFY*L2HN?X((LE[6/10P(PC^=,OKE@Z'C_ZLMO MA)Y2BS7\>25:`A2[Z,T[P+4;H-WI?2GG8R/!%QF@^\ TYM(?Z&&3C6N-KSU" M\\D/MO;.!)>*X;Y]MUNQW7M__4-WQA=F4 $-%K,6E]RO,/KS,[Z43&_KX!8_ M[DR[#]+%B)@ISKCRQVW4N.D*.*.RW.R?@.,-_]FD*1M$#N\6Z?C 1 4]GLC< M)-U+63GN>E]V-XD<0V#I+. +W6W'VTQ>\:7037(@)=O3U=YYP30]1[<(^>>WQK3>#9T-I&[2JA8 M#<]#)76YEL-5?!6NR"$1)92K/]FVL+7,O-D!)4#2@(."(960J43A-@CM5? . M?TK2A;ZZ7/'7-'A2+H"]*PM[%LA93U<]@B8ED)\E=*8>DUIZ#PA6\)CZKFSI M5\1[HUI[O&:9:^S7MSO:>"R7?E%AKEJ\Y6>(A.WPFS60$P'4'CQ19(2\[K#UT9R'8MH/)BZ60=7 M83G>,@*B,^14(YJWCFZ[:=*#=Z;38X_OB:7!^'NX9X+I/4FZ:A2`J<)*DD4/QWY0M4#;1T* MV-D.DOU1-A.^O=W7??Z!$C=?!,2AFE"LWZ\3^5& 'O'',B1EOQ8*)K/";F%] MSQ(^H-??G-*FC59HY%79[^4.'SRQ0 +#%&9))9G?*1UO\*;C&7?8_:ILTVTQ*D&=(M8T_E+)Z2H)@"_,VG[-[V0 M7 N$2CI$%U:ZGHV3V*32Z+;='(%P7^Q!*HJF+UFXSN>KW=5P'=]FE./N=HYZ M`&%(@AH)"W@C"."&IMX=`/9FW+2IXRS] M]%%S_H0V`TOU^-)/][RL^3HNVL+7,\4D(72K/(HYU@R#HF^P`/)D]!-^', O MG*%K3=[2C=8B M8!AP3LJ.?]X`[QNC-*)LVV0BX*_V3Z;6@U= \;Q0,-6?T37R//FSONQ9I&%/U%)H>UH`H=5U&1>GW/79PNO/VAAH [?CSP_W%)R8@XX 2&T*%#\BBGH!E6 MB#=%;+?TBW^;P! '%+T7'QJ!H$*O:)KW>=U[A%#H578VQK#M0LAX"Z<[?ZZ& MMG<_X4,W:U1\IQZ>WM_!O(6I=%U>\EEM*$CH]Y_A:'SP#J 35_!70G$DUIZL M'JI*N(>_*N.J"_&=_VT]\BG.],@(860U98(M;^C>`>B5EQ7?`ABO!PQHX'4___`%!+`0(4"Q0````(`+N*LRQ2K'E@B2(``$AW M```-``````````$`( ````````!N97=I;7!U=&EL+G!Y4$L%!@`````!``$` *.P```+0B```````` ` end From sjmachin@lexicon.net Sun May 19 23:10:22 2002 From: sjmachin@lexicon.net (John Machin) Date: Mon, 20 May 2002 08:10:22 +1000 Subject: [Python-Dev] getting the UCS-2 representation of a unicode object In-Reply-To: <05e001c1ff34$1cf86e30$02010a0a@suxlap> Message-ID: 19/05/2002 10:52:27 PM, "Andreas Jung" wrote: > >Py_BuildValue() allows the usage of "u" or "u#" to convert UCS-2 >data into a Python unicode object, however Py_ParseTuple() converts >a unicode object to UTF-16. Is this an error in the documentation >or why is there a asymmmetry in the API? > I hope it is a doc problem. s/UCS-2/UTF-16/ UCS-2 is old hat. It is 16-bit-only, UTF-16 without the "surrogates" that allow representation of the whole 2**21 (possible) character set. BTW, isn't this a c.l.py question? From sjmachin@lexicon.net Sun May 19 23:14:25 2002 From: sjmachin@lexicon.net (John Machin) Date: Mon, 20 May 2002 08:14:25 +1000 Subject: [Python-Dev] getting the UCS-2 representation of a unicode object In-Reply-To: <3CE7A7C6.1010909@lemburg.com> Message-ID: <83A8QMDBMHC7YXWVB62YZWQPB7W1U6Z.3ce823c1@Egil> 19/05/2002 11:25:26 PM, "M.-A. Lemburg" wrote: >Andreas Jung wrote: >> Py_BuildValue() allows the usage of "u" or "u#" to convert UCS-2 >> data into a Python unicode object, however Py_ParseTuple() converts >> a unicode object to UTF-16. Is this an error in the documentation >> or why is there a asymmmetry in the API? > >Sounds like a documentation bug: it should be UCS-2 since that's >what's used as internal data represenation in narrow Unicode >builds (the default). > I sincerely hope that you are mistaken in your belief that UCS-2 is so used. From sjmachin@lexicon.net Mon May 20 01:22:39 2002 From: sjmachin@lexicon.net (John Machin) Date: Mon, 20 May 2002 10:22:39 +1000 Subject: [Python-Dev] getting the UCS-2 representation of a unicode object In-Reply-To: <060d01c1ff42$677ec450$02010a0a@suxlap> Message-ID: 20/05/2002 12:35:19 AM, "Andreas Jung" wrote: >Sounds reasonable..but since Py_ParseTuple() only applies to function >arguments >it can not be used to convert a unicode object to UCS-2. So what is the >easiest >way to get the UCS-2 representation? PyUnicode_AS_DATA() returns for >u'computer' >a char * with strlen()==1, however PyUnicode_GET_DATA_SIZE() on the >same string returns 16 (looks fine for the two byes encoding of UCS-2). Am I >missing >something? > Andreas, If you don't care about surrogates or weird things like the Hong Kong extended character set that are outside the 2**16 range, pretend UCS-2 == UTF-16. Then on a narrow Python build, the unicode object is in effect in UCS-2; no conversion required. You are indeed missing something about PyUnicode_AS_DATA -- the doc says it returns a char * pointer to the internal buffer. I can't imagine what relevance strlen(such_a_pointer) has. The buffer will contain "c\0o\0m\0 etc etc" when viewed as a series of bytes (on a little-endian box) so yes strlen -> 1 but so what? What is there about the PyUnicode_AS_UNICODE() function that you don't like? Perhaps you might like to (a) say what you are trying to achieve (b) move the discussion to c.l.py Regards, John From andreas@andreas-jung.com Mon May 20 01:31:04 2002 From: andreas@andreas-jung.com (Andreas Jung) Date: Sun, 19 May 2002 20:31:04 -0400 Subject: [Python-Dev] getting the UCS-2 representation of a unicode object References: Message-ID: <083601c1ff95$a0cd3ea0$02010a0a@suxlap> I was just confused that a part of documentation talks about UTF-16 vs. UCS-2 since Python uses UCS-2(4) as internal representation. I also did not know that UCS-2 is a subset of UTF-16...I think my problems are now solved...at least from the Python side. Andreas ----- Original Message ----- From: "John Machin" To: "Andreas Jung" Cc: Sent: Sunday, May 19, 2002 20:22 Subject: Re: [Python-Dev] getting the UCS-2 representation of a unicode object > 20/05/2002 12:35:19 AM, "Andreas Jung" wrote: > > >Sounds reasonable..but since Py_ParseTuple() only applies to function > >arguments > >it can not be used to convert a unicode object to UCS-2. So what is the > >easiest > >way to get the UCS-2 representation? PyUnicode_AS_DATA() returns for > >u'computer' > >a char * with strlen()==1, however PyUnicode_GET_DATA_SIZE() on the > >same string returns 16 (looks fine for the two byes encoding of UCS-2). Am I > >missing > >something? > > > > Andreas, > > If you don't care about surrogates or weird things like the Hong Kong extended character set that are outside the 2**16 range, pretend UCS-2 == UTF-16. Then on a narrow Python build, the > unicode object is in effect in UCS-2; no conversion required. > > You are indeed missing something about PyUnicode_AS_DATA -- the doc says it returns a char * pointer to the internal buffer. I can't imagine what relevance strlen(such_a_pointer) has. The > buffer will contain "c\0o\0m\0 etc etc" when viewed as a series of bytes (on a little-endian box) so yes strlen -> 1 but so what? > > What is there about the PyUnicode_AS_UNICODE() function that you don't like? > > Perhaps you might like to (a) say what you are trying to achieve (b) move the discussion to c.l.py > > Regards, > > John > > From mal@lemburg.com Mon May 20 13:57:25 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 20 May 2002 14:57:25 +0200 Subject: [Python-Dev] getting the UCS-2 representation of a unicode object References: <83A8QMDBMHC7YXWVB62YZWQPB7W1U6Z.3ce823c1@Egil> Message-ID: <3CE8F2B5.8080004@lemburg.com> John Machin wrote: > 19/05/2002 11:25:26 PM, "M.-A. Lemburg" wrote: > > >>Andreas Jung wrote: >> >>>Py_BuildValue() allows the usage of "u" or "u#" to convert UCS-2 >>>data into a Python unicode object, however Py_ParseTuple() converts >>>a unicode object to UTF-16. Is this an error in the documentation >>>or why is there a asymmmetry in the API? >> >>Sounds like a documentation bug: it should be UCS-2 since that's >>what's used as internal data represenation in narrow Unicode >>builds (the default). >> > > > > I sincerely hope that you are mistaken in your belief that > UCS-2 is so used. I should know... -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From gmcm@hypernet.com Mon May 20 15:03:20 2002 From: gmcm@hypernet.com (Gordon McMillan) Date: Mon, 20 May 2002 10:03:20 -0400 Subject: [Python-Dev] imputil modifications In-Reply-To: Message-ID: <3CE8C9E8.32382.DFB8918@localhost> On 19 May 2002 at 17:24, Dave Hawkes wrote: > Recently I tried to use the imputil and module and > noticed that it had a few shortcomings, particularly > in an environment like pythonwin. > > I implemented a few changes/additions (applicable > to python 2.2 and up) to imputil (attached in > newimputil.zip) to fix these problems, without > breaking its basic structure. I'd suggest you check out iu.py (see http://www.mcmillan-inc.com/iu.html). > Issues addressed: > > 1) Objects in sys.path can break some other modules. In iu, sys.path contains strings. > 2) Chaining to previous (foreign) import handlers > supported if desired. No chaining, but iu is a complete import replacement. There are a couple levels at which you can override or extend it. > 3) reload is implemented. iu has reload. Also has an import lock. Also puts None is sys.modules on failure (as builtin import does, but not imputil). Also faster. -- Gordon http://www.mcmillan-inc.com/ From martin@v.loewis.de Mon May 20 18:19:11 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 20 May 2002 19:19:11 +0200 Subject: [Python-Dev] getting the UCS-2 representation of a unicode object In-Reply-To: References: Message-ID: John Machin writes: > BTW, isn't this a c.l.py question? Certainly - I was somehow assuming I read python-i18n when I responded. Regards, Martin From martin@v.loewis.de Mon May 20 18:21:31 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 20 May 2002 19:21:31 +0200 Subject: [Python-Dev] getting the UCS-2 representation of a unicode object In-Reply-To: <3CE8F2B5.8080004@lemburg.com> References: <83A8QMDBMHC7YXWVB62YZWQPB7W1U6Z.3ce823c1@Egil> <3CE8F2B5.8080004@lemburg.com> Message-ID: "M.-A. Lemburg" writes: > > I sincerely hope that you are mistaken in your belief that > > UCS-2 is so used. > > I should know... But, as John explains - you don't, right? Python uses UTF-16 internally, see PEP 261. Regards, Martin From mal@lemburg.com Mon May 20 22:22:45 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 20 May 2002 23:22:45 +0200 Subject: [Python-Dev] getting the UCS-2 representation of a unicode object References: <83A8QMDBMHC7YXWVB62YZWQPB7W1U6Z.3ce823c1@Egil> <3CE8F2B5.8080004@lemburg.com> Message-ID: <3CE96925.3070405@lemburg.com> Martin v. Loewis wrote: > "M.-A. Lemburg" writes: > > >>>I sincerely hope that you are mistaken in your belief that >> >> > UCS-2 is so used. >> >>I should know... > > > But, as John explains - you don't, right? Python uses UTF-16 > internally, see PEP 261. This is really only academic fuzz: Python uses two bytes to store Unicode code points -- it doesn't pay special attention to UTF-16 things like surrogates in the internals; only a few codecs do which provide interfaces to the outside world. BTW, PEP 261 uses the same terminology (UCS-2 instead of UTF-16) and for the same reason: --enable-unicode=ucs2 configures a narrow Py_UNICODE, and uses wchar_t if it fits --enable-unicode=ucs4 configures a wide Py_UNICODE, and uses wchar_t if it fits --enable-unicode same as "=ucs2" If you're interested in more details, you should come to the Unicode talk I'll give at EuroPython. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From walter@livinglogic.de Tue May 21 11:30:26 2002 From: walter@livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Tue, 21 May 2002 12:30:26 +0200 Subject: [Python-Dev] PEP282 and the warnings framework References: <20020517142156.36404.qmail@web9602.mail.yahoo.com> <001a01c1fdb4$2a038960$652b6992@alpha> <20020517183702.E23086@prim.han.de> <000901c1fdd5$0d9c6820$652b6992@alpha> Message-ID: <3CEA21C2.8080904@livinglogic.de> Vinay Sajip wrote: > [...] > > It might appear that allowing subclasses of LogRecord is the more natural > thing to do. But this has the potential to stop me from extending > LogRecord's functionality in the future, as some new attribute I introduce > might conflict with the same-named attribute in some user's > LogRecord-derived class. Since I want to preserve this freedom for myself > and other maintainers of logging, But you have the same problem with Logger, which is made to be subclassed. And Logger probably changes more often than LogRecord. > the best option seems to be: allow another > instance of an arbitrary class to hold whatever additional information the > caller wants. > > Next question: how to pass in this information? two choices spring to mind: > > 1. Use a keyword argument, extra_info=None. You can use this on any call > such as warn(), debug() etc. just as exc_info is currently used. A passed in > value would become the extra_info attribute of the LogRecord. This smells to much like extensibility to per-OO way, i.e. simple add a void *user_data to every struct you have and you're done. > 2. Use the "msg" parameter, which all logging calls already have, to have > the semantics such that it is either a format string, or an instance such > that str(msg) returns the desired format string. For the simple case, just > pass in a string. For the more complex case, pass an instance - the caller > can put any state in there that they want. It will not be used by the core > logging module, except to call __str__() on it for the format string. But it > can be used by user-derived classes, which will find it in the msg attribute > of the LogRecord. This might seem "hackish" to some, but it's not really - > it's just a question of packaging. This is more or less the same as 1.: use an (additional) argument, that will become an attribute of the LogRecord. > [...] > Slightly OT: There's a lot of people who think that the logging system > should be a completely generalized system for event generation and > processing. I've shied away from using a word like "Event", preferring > "LogRecord" which tries to indicate that this is a focused class for a > specific job. Even if LogRecord was renamed Event, it would be a "focused class for a specific job", because it would be logging.Event. But the name is not so important as the functionality. > [...] Bye, Walter Dörwald From walter@livinglogic.de Tue May 21 11:58:15 2002 From: walter@livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Tue, 21 May 2002 12:58:15 +0200 Subject: [Python-Dev] PEP282 and the warnings framework References: <20020515141648.9926.76949.Mailman@mail.python.org> <3CE283FE.8000403@campuspipeline.com> <20020515201342.T28033@prim.han.de> <3CE2AE95.4030900@campuspipeline.com> <20020516003321.U28033@prim.han.de> <3CE2FD51.2060301@campuspipeline.com> <3CE39B83.40006@livinglogic.de> <007501c1fcee$fa693760$652b6992@alpha> <3CE3FC18.4080509@livinglogic.de> <012f01c1fd22$bdd5b7e0$652b6992@alpha> <3CE4EBF1.1030303@livinglogic.de> <001101c1fd9d$cea1fd60$652b6992@alpha> Message-ID: <3CEA2847.6050507@livinglogic.de> Vinay Sajip wrote: > [...] >>>I think a better design is to pass in a callable which constructs the >>>LogRecord or a derived class thereof. This way, you can use a class, a >>>function, a bound method which has access to other context, or whatever. >> >>This will work in my version without a problem, because >>a function is not an instance of LogRecord. > > > True enough, though it's more by accident than design. Ouch ;), you're right, I didn't think about other callables. > [...] >>>logger.warn("Hello, %s", "world!", record=myRecordMaker) >> >>So go one step further and allow: >> logger.warn(myRecordMaker("Hello, %s", "world!")) > > > It's more than just a step further. In my case, it's an optional argument. > In your case, everyone has to do it that way. Not with something like this: def log(self, lvl, msg, args, exc_info, eventtype=LogRecord): if not isinstance(msg, LogRecord): event = eventtype(msg, args) with this solution you can still use a string for msg. > [...] >>(I assume that MySpecialRecord is derived from LogRecord). >>Either the LogRecord constructor should be responsible >>for creating the timestamp and finding file/line info. etc. >>or this could be done by the logger (via calling a method >>on the LogRecord instance). > > > Yes, I had the same idea, mentioned earlier in this post. Still thinking on > it :-) BTW, is there a public CVS repository for the logging module? Bye, Walter Dörwald From guido@python.org Tue May 21 12:54:16 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 21 May 2002 07:54:16 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib unittest.py,1.14,1.15 In-Reply-To: Your message of "Mon, 20 May 2002 20:49:03 PDT." References: Message-ID: <200205211154.g4LBsG930560@pcp742651pcs.reston01.va.comcast.net> [Fred] > Munge the RCS keywords to avoid updates, so the version number matches that > of the PyUNIT version of the same file. This helps people understand that > this version is the same as the version from the independent PyUNIT > release (confusion was indicated on the PyUNIT mailing list). But now it's gonna confuse Python developers. And what if Python's unittest.py gets a life of its own? What is the meaning of the version number then? I propose to remove the __author__, __email__ and __version__ strings, and instead include that information in the docstring or in a comment. > ! __version__ = "#Revision: 1.43 $"[11:-2] Why not simply __version__ = "1.43" ??? --Guido van Rossum (home page: http://www.python.org/~guido/) From daveh@cadlink.com Sun May 19 21:52:44 2002 From: daveh@cadlink.com (Dave Hawkes) Date: Sun, 19 May 2002 16:52:44 -0400 Subject: [Python-Dev] imputil modifications Message-ID: Hi, Recently I tried to use the imputil and module and noticed that it had a few shortcomings, particularly in an environment like pythonwin. I implemented a few changes/additions (applicable to python 2.2 and up) to imputil (attached as newimputil.py) to fix these problems, without breaking its basic structure. Issues addressed: 1) Objects in sys.path can break some other modules. A function has been created to make these objects appear like strings so we can do this for example: sys.path.insert(0, StringedImporter(BuiltinImporter())) The StringedImporter function will take a class instance and return another instance derived from str that also contains the Importer functionality. 2) Chaining to previous (foreign) import handlers supported if desired. A previously installed import handler can be used as shown by this example: im = ImportManager() im.install(chain=1) chain must be true to enable chaining as it is disabled by default so current assumptions should not be broken. 3) reload is implemented. An initial crude implementation of reload is provided as a starting point. An override in the Importer class can be coded to improve it in specific cases. Here is an example of code that will work for pythonwin: #====================================== import sys, new from newimputil import * class TestImporter(Importer): def get_code(self, parent, modname, fqname): if modname == 'xyzzy': def test(): return 'magic!' m = new.module('modname') vars(m)['test'] = test return (0, m, {}) return None im = ImportManager() im.install(chain=1) sys.path.insert(0, StringedImporter(TestImporter())) import xyzzy print xyzzy.test() #======================================= Any comments as to whether these changes are worth persuing? Thanks Dave Hawkes begin 666 newimputil.py M(B(B#0I);7!O$EM<&]R M=&5R#0HB(B(-"@T*(R!N;W1E.B!A=F]I9"!I;7!O7!E(#T@='EP92@G M)RD-"E]-;V1U;&54>7!E(#T@='EP92AS>7,I(" @(" @(" @(R,C(&1O97-N M)W0@=V]R:R!I;B!*4'ET:&]N+BXN#0H-"F-L87-S($EM<&]R=$UA;F%G97(Z M#0H@(" @(DUA;F%G92!T:&4@:6UP;W)T('!R;V-E"P@:6UP;W)T1G5N8RDZ M#0H@(" @(" @(&%S"P@:6UP;W)T1G5N8RD- M"@T*(" @(",C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C M(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,-"B @(" C#0H@(" @ M(R!04DE6051%($U%5$A/1%,-"B @(" C#0H-"B @("!C;'-&:6QE7-T96TN($ET(&1E9FEN97,@;VYE(&UO7-T96U);7!OF4@=&AE('-E="!O9B!S M=69F:7AE0T*(" @(" @(" C("YP>2!F:6QE71E8V]D92D-"B @(" @(" @9F]R M(&1E5]S M=69F:7A?:6UP;W)T97(I#0H-"B @("!D968@7VEM<&]R=%]H;V]K*'-E;&8L M(&9Q;F%M92P@9VQO8F%L'0@:7,@ M;F]T('5N9&5R45R M2!T:&%T('=E(&-O=6QD(&EM<&]R M= T*(" @(" @(" @(" @(" @(" @(" C('-U8BUM;V1U;&5S('1H870@;V-C M=7(@:6X@=&AE(&9R;VUL:7-T("AB=70@=V4@8V%N)W0@2!J=7-T M(&)E(&YA;65S*2!B96-A=7-E('=E(&1O;B=T(&MN;W<@:&]W#0H@(" @(" @ M(" @(" @(" @(" @(",@=&\@9&5A;"!W:71H('!A8VMA9V5S('1H870@=V5R M92!I;7!O2!O=&AE7-T96US+@T*(" @(" @(" @(" @(" @ M(" @(" C#0H@(" @(" @(" @(" @(" @(" @(",@26X@=&AE(&QA='1E2!S=6(M M#0H@(" @(" @(" @(" @(" @(" @(",@;6]D=6QE'0@9F]R('5N:VYO=VX@:6UP;W)T M97)S#0H@(" @(" @(" @(" C(&YO(&UA='1E7,N<&%T:#H-"B @(" @ M(" @(" @(&EF(&ES:6YS=&%N8V4H:71E;2P@26UP;W)T97(I.@T*(" @(" @ M(" @(" @(" @(&UO9'5L92 ](&ET96TN:6UP;W)T7W1O<"AN86UE*0T*(" @ M(" @(" @(" @96QI9B!I7!E*3H- M"B @(" @(" @(" @(" @("!M;V1U;&4@/2!S96QF+F9S7VEM<"YI;7!O2!O2!S96QF("AJ=7-T M#0H@(" @(" @(",@;F]W+"!O71H;VX@=VEL;"!P:6-K('5P(&%L;"!F M3H-"B @ M(" @(" @(" @(')E='5R;B!S>7,N;6]D=6QE7,N;6]D=6QE&5C=71E('1H92!C;V1E('=I=&AI;B!T:&4@;6]D=6QE)W,@;F%M97-P M86-E#0H@(" @(" @(&EF(&YO="!I&5C(&-O9&4@:6X@;6]D=6QE+E]?9&EC=%]?#0H-"B @(" @(" @(R!F971C M:"!F2X-"B @(" @(" @(R!A;'-O(&UA:V4@;6]D=6QE)W,@7U]N M86UE7U\@86=R964@=VET:"!F<6YA;64L(&EN(&-A2!S=6(M;6]D=6QE2D-"B @(" @(" @(" @(')E='5R;B!.;VYE#0H- M"B @(" @(" @2!B92!.;VYE M+"!I;F1I8V%T:6YG(&YO('!A2D@9&]T M=&5D(&YA;64@9G)O;2!T:&4@(G)O;W0B(&]F('1H92!M;V1U;&4@;F%M97-P M86-E#0H@(" @(" @(&1O=VX@=&\@=&AE(&UO9&YA;64N#0H@(" @(" @($EF M('1H97)E(&ES(&YO('!A6EN9R!W:&5T:&5R('1H92!M;V1U;&4@=&AA="!W87,@9F]U;F0@:7,@82!P M86-K86=E(&]R(&YO="X-"@T*(" @(" @(" J(%1H92!S96-O;F0@:71E;2!I M2UL;V%D960@;6]D=6QE(&]B:F5C=" H92YG+B!L;V%D960@ M9G)O;2!A('-H87)E9"!L:6(I+@T*#0H@(" @(" @("H@5&AE('1H:7)D(&ET M96T@:7,@82!D:6-T:6]N87)Y(&]F(&YA;64O=F%L=64@<&%I'!E8W1S(&-E7!E*3H-"B @(" @(" @(" @(",@=')Y M(&%N9"!C;W!Y('1H92!M;V1U;&4@9&EC= T*(" @(" @(" @(" @;6]D=6QE M+E]?9&EC=%]?+G5P9&%T92AC;V1E+E]?9&EC=%]?*0T*(" @(" @("!E;'-E M.@T*(" @(" @(" @(" @97AE8R!C;V1E(&EN(&UO9'5L92Y?7V1I8W1?7PT* M(" @(" @(" -"@T*(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C M(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(PT*(PT*(R!3 M;VUE(&AA;F1Y('-T=69F(&9O"!C:&%R86-T97(-"E]S=69F:7A?8VAA M2 \<&%T:&YA;64^(&ES(&-O M;7!I;&5D('1O(&$@8V]D92!O8FIE8W0@86YD#0H@(" @71E8V]D97,@=VEL;"!B90T*(" @('-A=F5D(&)A8VL@=&\@ M=&AE(&9I;&5S>7-T96T@9F]R(&9U='5R92!I;7!O&5C M)RD-"@T*(" @(",@=')Y('1O(&-A8VAE('1H92!C;VUP:6QE9"!C;V1E#0H@ M(" @=')Y.@T*(" @(" @("!F(#T@;W!E;BAP871H;F%M92 K(%]S=69F:7A? M8VAA&-E<'0@24]%7,N8G5I;'1I;E]M;V1U;&5? M;F%M97,-"@T*(" @(&IO:6X@/2!.;VYE#0H@(" @:68@)W!O3H-"B @(" @(" @&-E<'0@3U-%F5N(&UO M9'5L97,-"B,-"F-L87-S($)U:6QT:6Y);7!O7!E(#T@:6UP+D-?0E5) M3%1)3@T*(" @(" @("!E;&EF(&EM<"YI7!E(#T@:6UP+E!97T923UI%3@T*(" @(" @("!E;'-E M.@T*(" @(" @(" @(" @(R!N;W0@9F]U;F0-"B @(" @(" @(" @(')E='5R M;B!.;VYE#0H-"B @(" @(" @(R!G;W0@:70N(&YO=R!L;V%D(&%N9"!R971U M7!E*2D-"B @(" @(" @ MR!]#0H-"@T*(R,C(R,C(R,C(R,C(R,C(R,C M(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C M(R,C(R,C(PT*(PT*(R!);G1E7-T M96U);7!O&5S(#T@6R!=#0H-"B @("!D968@861D M7W-U9F9I>"AS96QF+"!S=69F:7@L(&EM<&]R=$9U;F,I.@T*(" @(" @("!A M"P@:6UP;W)T1G5N8RDI#0H-"B @("!D968@ M:6UP;W)T7V9R;VU?9&ER*'-E;&8L(&1I"P@:6UP;W)T1G5N8R!I;B!S96QF+G-U9F9I M>&5S.@T*(" @(" @(" @(" @9FEL96YA;64@/2!P871H;F%M92 K('-U9F9I M> T*(" @(" @(" @(" @=')Y.@T*(" @(" @(" @(" @(" @(&9I;F9O(#T@ M7V]S7W-T870H9FEL96YA;64I#0H@(" @(" @(" @("!E>&-E<'0@3U-%%]I;7!O2 ](&QO;F6,@/2!? M=&EM97-T86UP*&9I;&4I#0H-"B @("!C;V1E(#T@3F]N90T*(" @(&EF('1? M<'EC(&ES(&YO="!.;VYE(&%N9"!T7W!Y8R ^/2!T7W!Y.@T*(" @(" @("!F M(#T@;W!E;BAF:6QE+" GR G7U]F:6QE M7U\G(#H@9FEL92!]#0H-"F-L87-S($1Y;DQO8613=69F:7A);7!OR!]#0H-"B,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C M(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C M(R,-"B,-"B,@36%K92!A;B!I;7!O7,N<&%T:"YI;G-E7,N<&%T:"YI;G-E7,N<&%T:"YI;G-E71H;VXN('-A;64@9F]R(&=E=%]S=69F:7AE M2!O9CH-"B,@ M(" @("!I;7!O6\@<&EP M:6YG(&AO;VMS('1O(&9I;'1E0T*(R @('=A M=&-H(&]U="!F;W(@'0@86YD#0HC(" @(" @(&-H96-K:6YG(&9O7,N<&%T:" @*'-E92!Q=6]T92!B96QO=RD-"B,@("!N;W1E M.B!R96QO860@9&]E2!T;R!D96%L('=I=&@@7V]S7V)O;W1S=')A M<"!P2!U65T#0HC(&EN('1H92!C86-H92X@(%1H92!M971H;V0@ M7,N<&%T:"!D;R!T:&4-"B,@2!N;R!L;VYG97(@=7-E9"X- M"B,-"B,@37DO1W5I9&\G&5C('-I='5A=&EO;G,-"B,@/B R*2!I;7!L96UE;G1I;F<@<&]L M:6-Y(&EN($EM<&]R=&5R(&%S0T*(R ^(" @(&-H86YG97,@9F]R(&YO6)E('1H92!);7!O0T*(R ^(&1E;&5G871I M;F<@=&\@26UP;W)T36%N86=E2!B=70@82 J;F5C97-S:71Y*B!T;R!G M970@=&AI;F=S#0HC('=O Message-ID: <02e301c200db$fcdf84e0$6601a8c0@boostconsulting.com> ----- Original Message ----- From: "Fred L. Drake" To: ; ; Sent: Tuesday, May 21, 2002 11:05 AM Subject: [Python-Dev] [development doc updates] > The development version of the documentation has been updated: > > http://www.python.org/dev/doc/devel/ > > Experimental change: Add some really heavy markup in function signatures > so that when the signature line is wrapped because the browser window is > too narrow, the argument list will wrap to the left parenthesis opening the > argument list. Cool idea! I'm going to have to steal it... after you fix it ;-) On IE6, the doc for excepthook at http://www.python.org/dev/doc/devel/lib/module-sys.html looks like this when wrapped: excepthook type, value, ( traceback) -Dave From fredrik@pythonware.com Tue May 21 17:09:22 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Tue, 21 May 2002 18:09:22 +0200 Subject: [Python-Dev] [development doc updates] References: <20020521150555.2B93C18EB28@grendel.zope.com> <02e301c200db$fcdf84e0$6601a8c0@boostconsulting.com> Message-ID: <002401c200e1$e0662f70$ced241d5@hagrid> david wrote: > > Experimental change: Add some really heavy markup in function signatures > > so that when the signature line is wrapped because the browser window is > > too narrow, the argument list will wrap to the left parenthesis opening > > the argument list. > > Cool idea! I'm going to have to steal it... after you fix it ;-) > > On IE6, the doc for excepthook at > http://www.python.org/dev/doc/devel/lib/module-sys.html looks like this > when wrapped: > > excepthook type, value, > ( traceback) to fix this, just add a "nowrap" attribute to the first TD in each function table. From guido@python.org Tue May 21 17:38:30 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 21 May 2002 12:38:30 -0400 Subject: [Python-Dev] New bugtracker project Message-ID: <200205211638.g4LGcVw19247@odiug.zope.com> The Python Business Forum (http://pbf.nuxeo.org/) and McMillan Enterprises, Inc. (http://www.mcmillan-inc.com/) are joining forces to help the Python developers with their tasks. (IOW, the PBF is funding Gordon McMillan to do some hacking for us. :-) The goal is that through these efforts, the Python developers will be able to produce and maintain stable Python releases that are usable by its business members, while simultaneously continuing to develop and release bleeding-edge releases for the hacker community. Of course, today's bleeding-edge release will become tomorrow's business release, but it takes much water under the bridge for a release to mature enough to that status. I believe that one of the areas where we could use help is a bug tracker. While the SourceForge tracker that we currently use is reasonable, it has many problems, and at least one meta-problem: we don't control it, so the problems don't get fixed. Some folks have suggested to switch to Bugzilla, but that got a loud booh from people who have tried it. A much better option appears to be RoundUp: Ka-Ping Yee's winning entry in the Software Carpentry competition, re-implemented by Richard Jones and Anthony Baxter, with four co-developers, now in beta (release 0.4.1 at http://sourceforge.net/projects/roundup). It's all Python, My proposal is for Gordon to join forces with Richard and Anthony and the other RoundUp developers to make RoundUp usable for the needs of the Python development team. That's not just PythonLabs, but includes all developers with checkin privileges, and to some extent all readers of python-dev. (Of course, it must be usable for the end users reporting bugs too. :-) The point of this message is to start gathering requirements. Gordon will gather requirements, and has asked me to send out this email to announce the project to the Python developer community. Some of my own requirements: - A way to migrate the existing SF tracker contents. There are references to 6-digit SF tracker IDs all over the Python source code and CVS history, and it would be nice if these IDs were usable in the new tracker. - Features that simplify the tracking of how bugs and patches relate to different release branches. This could ease the work for release managers tremendously compared to the status quo. It should be possible to indicate exactly which releases are affected by a bug, and also in which releases it is (or will be) fixed. (Integration with CVS would be nice but seems out of scope.) --Guido van Rossum (home page: http://www.python.org/~guido/) From David Abrahams" Hi, this just showed up on the C++-sig. Somehow I can't believe it's the first we've heard of it, but I thought I'd post a heads-up just in case. -Dave ----- Original Message ----- From: "Achim Domma" > > -----Original Message----- > > From: c++-sig-admin@python.org [mailto:c++-sig-admin@python.org]On > > Behalf Of David Abrahams > > > > Achim wrote: > > > I have a very strang problem with boost.python V1 and V2: On Win2K I > > build > > > two extensions with boost.python, one with V1 and one with V2. Both > > compile > > > without problems, but fail to load. The problem is a missing dll called > > > 'apphelp.dll' which is only available on WinXP. According to Microsofts > > > 'depends' utility the dll is neede by python22.dll, but my python works > > > without problems. > > > Has anybody an idea ? I compiled working extensions some days befor, all > > on > > > this computer!? > > > > very strange. I'm developing on XP without any trouble. My apphelp.dll > > lives in c:\Windows\System32. Perhaps you deleted yours, or you > > removed the > > directory from your PATH? > > No, I develop on Win2K and don't have an XP installation. But apphelp.dll is > XP specific, so I don't know how the dependency comes to my computer. > Searching on groups.google.com I found lots of posts from php-people having > the same problem. > I just used 'depends' on python22.dll (from ActiveState and from > python.org). Both version depend on apphelp.dll. Don't know why the problem > only arises when I try to load my extensions. Later I will try to compile a > version on my own. > > Achim From guido@python.org Tue May 21 17:49:27 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 21 May 2002 12:49:27 -0400 Subject: [Python-Dev] Windows XP dependency in Python release? In-Reply-To: Your message of "Tue, 21 May 2002 12:40:39 EDT." <037401c200e6$3e158130$6601a8c0@boostconsulting.com> References: <037401c200e6$3e158130$6601a8c0@boostconsulting.com> Message-ID: <200205211649.g4LGnRY19317@odiug.zope.com> > Hi, this just showed up on the C++-sig. Somehow I can't believe it's the > first we've heard of it, but I thought I'd post a heads-up just in case. I find it difficult to extract the issue from the series of quoted and folded messages. What's your summary of the problem? --Guido van Rossum (home page: http://www.python.org/~guido/) From David Abrahams" <200205211649.g4LGnRY19317@odiug.zope.com> Message-ID: <03a501c200e8$59d74cd0$6601a8c0@boostconsulting.com> From: "Guido van Rossum" > > Hi, this just showed up on the C++-sig. Somehow I can't believe it's the > > first we've heard of it, but I thought I'd post a heads-up just in case. > > I find it difficult to extract the issue from the series of quoted and > folded messages. What's your summary of the problem? Sorry, I have trouble predicting how my messages will get wrapped; time to switch to emacs for mail. The summary is that python22.dll seems to have been built so that it depends on a DLL which only comes with Windows XP, though it strangely doesn't cause a problem until loading an extension module. From: "Achim Domma" > > From: c++-sig-admin@python.org [mailto:c++-sig-admin@python.org]On > > Behalf Of David Abrahams > > > > Achim wrote: > > > > > I have a very strang problem with boost.python V1 and V2: On > > > Win2K I build two extensions with boost.python, one with V1 and > > > one with V2. Both compile without problems, but fail to > > > load. The problem is a missing dll called 'apphelp.dll' which is > > > only available on WinXP. According to Microsofts 'depends' > > > utility the dll is neede by python22.dll, but my python works > > > without problems. Has anybody an idea ? I compiled working > > > extensions some days befor, all on this computer!? > > > > very strange. I'm developing on XP without any trouble. My > > apphelp.dll lives in c:\Windows\System32. Perhaps you deleted > > yours, or you removed the directory from your PATH? > > No, I develop on Win2K and don't have an XP installation. But > apphelp.dll is XP specific, so I don't know how the dependency comes > to my computer. Searching on groups.google.com I found lots of > posts from php-people having the same problem. I just used > 'depends' on python22.dll (from ActiveState and from > python.org). Both version depend on apphelp.dll. Don't know why the > problem only arises when I try to load my extensions. Later I will > try to compile a version on my own. > > Achim From nas@python.ca Tue May 21 18:41:26 2002 From: nas@python.ca (Neil Schemenauer) Date: Tue, 21 May 2002 10:41:26 -0700 Subject: [Python-Dev] New bugtracker project In-Reply-To: <200205211638.g4LGcVw19247@odiug.zope.com>; from guido@python.org on Tue, May 21, 2002 at 12:38:30PM -0400 References: <200205211638.g4LGcVw19247@odiug.zope.com> Message-ID: <20020521104126.A12155@glacier.arctrix.com> Guido van Rossum wrote: > Some of my own requirements: [snip] Here's mine: - Usable via email. I hate having the log into SF to add a comment, post a new patch, report or close a bug. Roundup works this way already so this should be an easy one. :-) - It should be possible to close bugs by adding some magic to the CVS checkin message. Debian uses the Perl regex: /closes:\s*(?:bug)?\#\s*\d+(?:,\s*(?:bug)?\#\s*\d+)*/ig I would prefer something simpler like: re.compile("closes:\s*#\d+(,\s*#\d+)*", re.I) That's all I can think of for now. Neil From tim.one@comcast.net Tue May 21 18:57:05 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 21 May 2002 13:57:05 -0400 Subject: [Python-Dev] Windows XP dependency in Python release? In-Reply-To: <03a501c200e8$59d74cd0$6601a8c0@boostconsulting.com> Message-ID: [avid Abrahams] > The summary is that python22.dll seems to have been built so that it > depends on a DLL which only comes with Windows XP, though it strangely > doesn't cause a problem until loading an extension module. The python22.dll distributed by PythonLabs was built on a Win98SE box. It doesn't depend on apphelp.dll. I've never been on an XP box, and there is no apphelp.dll on any machine I do use. Here are the system DLLs it does depend on: ADVAPI32.DLL KERNEL32.DLL USER32.DLL MSVCRT.DLL SHELL32.DLL Those in turn depend on (transitive closure): COMCTL32.DLL GDI32.DLL NTDLL.DLL RPCRT4.DLL SHLWAPI.DLL That's it. From David Abrahams" Message-ID: <03e301c200f4$423428d0$6601a8c0@boostconsulting.com> ----- Original Message ----- From: "Tim Peters" To: "David Abrahams" Cc: Sent: Tuesday, May 21, 2002 1:57 PM Subject: RE: [Python-Dev] Windows XP dependency in Python release? > [avid Abrahams] > > The summary is that python22.dll seems to have been built so that it > > depends on a DLL which only comes with Windows XP, though it strangely > > doesn't cause a problem until loading an extension module. > > The python22.dll distributed by PythonLabs was built on a Win98SE box. It > doesn't depend on apphelp.dll. I've never been on an XP box, and there is > no apphelp.dll on any machine I do use. Here are the system DLLs it does > depend on: > > ADVAPI32.DLL > KERNEL32.DLL > USER32.DLL > MSVCRT.DLL > SHELL32.DLL > > Those in turn depend on (transitive closure): > > COMCTL32.DLL > GDI32.DLL > NTDLL.DLL > RPCRT4.DLL > SHLWAPI.DLL > > That's it. Hey, it seemed legit since he reports a firestorm of similar issues on some php list (no, I'm not shipping anyone any binaries). I'm just passing this along. Do you want to talk to the guy who's having the problem, or do you want to assume there's a luser error somewhere? the-choice-is-y'rs-ly-y'rs, dave From tim.one@comcast.net Tue May 21 20:39:09 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 21 May 2002 15:39:09 -0400 Subject: [Python-Dev] Windows XP dependency in Python release? In-Reply-To: <03e301c200f4$423428d0$6601a8c0@boostconsulting.com> Message-ID: > Do you want to talk to the guy who's having the problem, or do you > want to assume there's a luser error somewhere? If he thinks there's a bug, he should open a bug report on SourceForge. From mats@laplaza.org Tue May 21 20:53:41 2002 From: mats@laplaza.org (Mats Wichmann) Date: Tue, 21 May 2002 13:53:41 -0600 Subject: [Python-Dev] Re: [development doc updates] In-Reply-To: <20020521160007.29170.77697.Mailman@mail.python.org> Message-ID: <5.1.0.14.1.20020521135256.00af5ea8@204.151.72.2> >On IE6, the doc for excepthook at >http://www.python.org/dev/doc/devel/lib/module-sys.html looks like this >when wrapped: > >excepthook type, value, >( traceback) Same for IE 5.5. From aahz@pythoncraft.com Tue May 21 22:25:46 2002 From: aahz@pythoncraft.com (Aahz) Date: Tue, 21 May 2002 17:25:46 -0400 Subject: [Python-Dev] New bugtracker project In-Reply-To: <200205211638.g4LGcVw19247@odiug.zope.com> References: <200205211638.g4LGcVw19247@odiug.zope.com> Message-ID: <20020521212546.GB19070@panix.com> On Tue, May 21, 2002, Guido van Rossum wrote: > > The point of this message is to start gathering requirements. Gordon > will gather requirements, and has asked me to send out this email to > announce the project to the Python developer community. I've got two requirements that are basically identical (assuming the system is properly coded): * The system should work with Lynx and other non-standard browsers (this is my biggest problem with SF) * The system should be remotely scriptable from Python (that is, I should be able to have a Python script that automatically creates a bug report and uploads the files for me) It's not a requirement, but I strongly suggest that all linkages be capable of many-to-many relationships. (This comes from a bad experience with a tech support package that only permitted one bug report to be associated with a tech support incident.) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "In the end, outside of spy agencies, people are far too trusting and willing to help." --Ira Winkler From martin@v.loewis.de Tue May 21 22:56:21 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 21 May 2002 23:56:21 +0200 Subject: [Python-Dev] New bugtracker project In-Reply-To: <20020521212546.GB19070@panix.com> References: <200205211638.g4LGcVw19247@odiug.zope.com> <20020521212546.GB19070@panix.com> Message-ID: Aahz writes: > * The system should be remotely scriptable from Python (that is, I > should be able to have a Python script that automatically creates a bug > report and uploads the files for me) I agree. As an extension, it should be possibly to synchronize with other bugtrackers, in particular the Debian BTS. Regards, Martin From gmcm@hypernet.com Tue May 21 23:09:02 2002 From: gmcm@hypernet.com (Gordon McMillan) Date: Tue, 21 May 2002 18:09:02 -0400 Subject: [Python-Dev] New bugtracker project In-Reply-To: References: <20020521212546.GB19070@panix.com> Message-ID: <3CEA8D3E.28934.14DE8F96@localhost> On 21 May 2002 at 23:56, Martin v. Loewis wrote: > ... As an extension, it should be possibly to > synchronize with other bugtrackers, in particular > the Debian BTS. What does it mean to "synchronize bugtrackers"? -- Gordon http://www.mcmillan-inc.com/ From skip@pobox.com Wed May 22 00:08:08 2002 From: skip@pobox.com (Skip Montanaro) Date: Tue, 21 May 2002 18:08:08 -0500 Subject: [Python-Dev] New bugtracker project In-Reply-To: <20020521212546.GB19070@panix.com> References: <200205211638.g4LGcVw19247@odiug.zope.com> <20020521212546.GB19070@panix.com> Message-ID: <15594.54104.523273.180886@12-248-41-177.client.attbi.com> aahz> * The system should be remotely scriptable from Python (that is, I aahz> should be able to have a Python script that automatically aahz> creates a bug report and uploads the files for me) This would be easy with XML-RPC (much easier than using urllib or httplib to submit to a plain old web server). Skip From aahz@pythoncraft.com Wed May 22 00:34:40 2002 From: aahz@pythoncraft.com (Aahz) Date: Tue, 21 May 2002 19:34:40 -0400 Subject: [Python-Dev] New bugtracker project In-Reply-To: <15594.54104.523273.180886@12-248-41-177.client.attbi.com> References: <200205211638.g4LGcVw19247@odiug.zope.com> <20020521212546.GB19070@panix.com> <15594.54104.523273.180886@12-248-41-177.client.attbi.com> Message-ID: <20020521233440.GA13843@panix.com> On Tue, May 21, 2002, Skip Montanaro wrote: > > aahz> * The system should be remotely scriptable from Python (that is, I > aahz> should be able to have a Python script that automatically > aahz> creates a bug report and uploads the files for me) > > This would be easy with XML-RPC (much easier than using urllib or httplib to > submit to a plain old web server). "Would be" or "is"? I.e., does Roundup currently support this? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "In the end, outside of spy agencies, people are far too trusting and willing to help." --Ira Winkler From DavidA@ActiveState.com Wed May 22 00:55:10 2002 From: DavidA@ActiveState.com (David Ascher) Date: Tue, 21 May 2002 16:55:10 -0700 Subject: [Python-Dev] New bugtracker project References: <200205211638.g4LGcVw19247@odiug.zope.com> <20020521212546.GB19070@panix.com> <15594.54104.523273.180886@12-248-41-177.client.attbi.com> <20020521233440.GA13843@panix.com> Message-ID: <3CEADE5E.5090607@ActiveState.com> My list of requirements isn't so much a list of reqs as a list of features from bugzilla that we've found exceedingly useful. 0) keywords -- massively useful for targeting, filtering, etc. 1) Attachments on bugs (somewhat of a merge of the patch tracker and bug tracker, but also useful for attaching test cases, screenshots, etc.) 2) Being able to "watch" people - be notified when their bugs have events happen to them, e.g.. when they're on vacation. 3) Saving and recalling searches 4) non-web, non-email access to the database (e.g. take all the bugs assigned to so and so, and take those submitted by people in england, and set the keyword "never" to them). 5) scalability (in # of developers, # of users, # of bugs, # of 'versions', # of 'products', etc.) 6) Being able to migrate a bug from one product to another (not all that relevant for a Python-only bug tracker, except if the versions are handled as different products). 7) Bug linkages (depends on/blocks, duplicate handling, etc.) Useful things we've added to bugzilla include: 1) Ability to designate bugs as 'for internal use only' (not especially important for python-dev =) 2) an 'efficient' UI. The bugzilla default search results pages are very inefficient from a UI point of view. Andy McKay's version is much better, I think. Compare: http://bugzilla.mozilla.org/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&email1=&emailtype1=substring&emailassigned_to1=1&email2=&emailtype2=substring&emailreporter2=1&bugidtype=include&bug_id=&changedin=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&component=Address+Book&short_desc=&short_desc_type=allwordssubstr&long_desc=&long_desc_type=allwordssubstr&bug_file_loc=&bug_file_loc_type=allwordssubstr&status_whiteboard=&status_whiteboard_type=allwordssubstr&keywords=&keywords_type=anywords&field0-0-0=noop&type0-0-0=noop&value0-0-0=&cmdtype=doit&order=Reuse+same+sort+as+last+time and: http://bugs.activestate.com/buglist.cgi?product=&querytype=simple&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&email1=&emailtype1=substring&emailassigned_to1=1&bugidtype=include&bug_id=&changedin=&chfieldfrom=&chfieldto=Now&chfieldvalue=&short_desc=&short_desc_type=substring&long_desc=&long_desc_type=substring&bug_file_loc=&bug_file_loc_type=substring&status_whiteboard=&status_whiteboard_type=substring&keywords_type=anywords&field0-0-0=noop&type0-0-0=noop&value0-0-0=&order=bugs.priority%20ASC%2C%20bugs.bug_status%20ASC%2C%20map_assigned_to.login_name%20ASC%2C%20bugs.bug_id%20DESC As I've mentioned in the past, we used an earlier version of roundup and found it didn't scale well with thousands of users. I haven't look at the new roundup to know if it's dealt with the problems we had (way too much mail generation by default, much too brittle, and poorly architected =). It's also important to have flexibility in handling the bug cycle (who closes, who verifies, etc.). Both roundup and bugzilla did that pretty well. --david From mhammond@skippinet.com.au Wed May 22 01:28:12 2002 From: mhammond@skippinet.com.au (Mark Hammond) Date: Wed, 22 May 2002 10:28:12 +1000 Subject: [Python-Dev] New bugtracker project In-Reply-To: <200205211638.g4LGcVw19247@odiug.zope.com> Message-ID: [Guido] > Some folks have > suggested to switch to Bugzilla, but that got a loud booh from people > who have tried it. A much better option appears to be RoundUp: > Ka-Ping Yee's winning entry in the Software Carpentry competition, > re-implemented by Richard Jones and Anthony Baxter, with four > co-developers, now in beta (release 0.4.1 at > http://sourceforge.net/projects/roundup). It's all Python, I must have missed this, as I have been seeing people who *use* bugzilla rave about it, and people who haven't assume it must suck. Of particular note, the opinions of *developers* using the system are, IMO, more important than the opinions of the *reporters*. Having a system that encourage users to report bugs, but discourage developers from fixing them would not be productive. People can not "walk in off the street" and start filing meaningful bugs. If the barrier to entry is that these users must invest a little time in learning how to search bugzilla then that is a good thing. IMO, roundup would be *excellent* as an "short-lived interest list" - eg, for discussing a single PEP over its life time. I would really like to hear some experiences from people who have actually *used* roundup in this kind of environment (as opposed to people who have said "it is 100% Python, so it must be cool" ;) I do know that ActiveState *did* use roundup (for basically that exact reason), but it *quickly* became apparent that this wasn't reasonable. It was an earlier version of roundup, but still, it was a square peg in a round hole - roundup was *not* designed as a general purpose bug tracking tool. bugzilla is. Please - if we are going to go through this pain, let us make sure that the tool we choose is appropriate for the task. bugzilla has a number of reference sites we could look at, but I am not aware of any such bugtracking systems hosted by roundup. Let us not make the same mistake AvtiveState did - go through the pain of moving to roundup, just to find that the requirement was really for a "real" bug tracking system, and doing it all again. FWIW, I have absolutely *no* interest in bugzilla other than as a very happy user. Mark. From skip@pobox.com Wed May 22 01:32:05 2002 From: skip@pobox.com (Skip Montanaro) Date: Tue, 21 May 2002 19:32:05 -0500 Subject: [Python-Dev] New bugtracker project In-Reply-To: <20020521233440.GA13843@panix.com> References: <200205211638.g4LGcVw19247@odiug.zope.com> <20020521212546.GB19070@panix.com> <15594.54104.523273.180886@12-248-41-177.client.attbi.com> <20020521233440.GA13843@panix.com> Message-ID: <15594.59141.118808.951628@12-248-41-177.client.attbi.com> aahz> * The system should be remotely scriptable from Python skip> This would be easy with XML-RPC aahz> "Would be" or "is"? Dunno. I've never looked at RoundUp, however having written screen scrapers using urllib/httplib and XML-RPC servers, I can tell you unequivocally that the XML-RPC route is the better one to take. Skip From gherron@islandtraining.com Wed May 22 01:24:16 2002 From: gherron@islandtraining.com (Gary Herron) Date: Tue, 21 May 2002 17:24:16 -0700 Subject: [Python-Dev] New bugtracker project In-Reply-To: <20020521233440.GA13843@panix.com> References: <200205211638.g4LGcVw19247@odiug.zope.com> <15594.54104.523273.180886@12-248-41-177.client.attbi.com> <20020521233440.GA13843@panix.com> Message-ID: <200205211724.16876.gherron@islandtraining.com> On Tuesday 21 May 2002 04:34 pm, Aahz wrote: > On Tue, May 21, 2002, Skip Montanaro wrote: > > aahz> * The system should be remotely scriptable from Python (that > > is, I aahz> should be able to have a Python script that automatically > > aahz> creates a bug report and uploads the files for me) > > > > This would be easy with XML-RPC (much easier than using urllib or httplib > > to submit to a plain old web server). > > "Would be" or "is"? I.e., does Roundup currently support this? It does not. However, it does supply an email interface for the creation and modification of issues. One could write a local python script which creates and sends an email to the tracking session. By creating an eamil with the proper keywords in the subject line and one or more mime multipart attachements one can: * Create an issue with specific fields, a message, and attached files. * Add to the discussion (message list) or list of attached files * Modify any of the fields (priority, assigned-to, status, platform ...) All these actions (of course), and more can be performed via the web interface. Further, all admin operations can be performed from a command line interface, and presumably with python scripts run on the host machine. I would guess that it would not be hard to provide an XML-RPC interface to this, but such a thing may not be necessary given that the most common operations can be conducted via e-mail. Gary Herron From skip@pobox.com Wed May 22 01:38:29 2002 From: skip@pobox.com (Skip Montanaro) Date: Tue, 21 May 2002 19:38:29 -0500 Subject: [Python-Dev] New bugtracker project In-Reply-To: <3CEADE5E.5090607@ActiveState.com> References: <200205211638.g4LGcVw19247@odiug.zope.com> <20020521212546.GB19070@panix.com> <15594.54104.523273.180886@12-248-41-177.client.attbi.com> <20020521233440.GA13843@panix.com> <3CEADE5E.5090607@ActiveState.com> Message-ID: <15594.59525.30466.309094@12-248-41-177.client.attbi.com> David> Compare: David> http://bugzilla.mozilla.org/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&email1=&emailtype1=substring&emailassigned_to1=1&email2=&emailtype2=substring&emailreporter2=1&bugidtype=include&bug_id=&changedin=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&component=Address+Book&short_desc=&short_desc_type=allwordssubstr&long_desc=&long_desc_type=allwordssubstr&bug_file_loc=&bug_file_loc_type=allwordssubstr&status_whiteboard=&status_whiteboard_type=allwordssubstr&keywords=&keywords_type=anywords&field0-0-0=noop&type0-0-0=noop&value0-0-0=&cmdtype=doit&order=Reuse+same+sort+as+last+time David> and: David> http://bugs.activestate.com/buglist.cgi?product=&querytype=simple&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&email1=&emailtype1=substring&emailassigned_to1=1&bugidtype=include&bug_id=&changedin=&chfieldfrom=&chfieldto=Now&chfieldvalue=&short_desc=&short_desc_type=substring&long_desc=&long_desc_type=substring&bug_file_loc=&bug_file_loc_type=substring&status_whiteboard=&status_whiteboard_type=substring&keywords_type=anywords&field0-0-0=noop&type0-0-0=noop&value0-0-0=&order=bugs.priority%20ASC%2C%20bugs.bug_status%20ASC%2C%20map_assigned_to.login_name%20ASC%2C%20bugs.bug_id%20DESC Can you elaborate what the differences are? Other than smaller fonts and pink backgrounds I didn't notice any functional difference between the two pages. In particular, they both seemed to have several columns of information which could be used to sort the display in ascending or descending order. (I use Opera. Maybe some other tidbits are available to people using Netscape or MSIE.) My biggest gripe with bugzilla has always been the damn search form. Talk about an inefficient UI! If I need to submit a bug report to a bugzilla-based system I refuse to even try to see if the bug has been reported before. Skip From Anthony Baxter Wed May 22 01:42:46 2002 From: Anthony Baxter (Anthony Baxter) Date: Wed, 22 May 2002 10:42:46 +1000 Subject: [Python-Dev] New bugtracker project In-Reply-To: Message from Guido van Rossum of "Tue, 21 May 2002 12:38:30 -0400." <200205211638.g4LGcVw19247@odiug.zope.com> Message-ID: <200205220042.g4M0gkL07064@localhost.localdomain> >>> Guido van Rossum wrote > A much better option appears to be RoundUp: > Ka-Ping Yee's winning entry in the Software Carpentry competition, > re-implemented by Richard Jones and Anthony Baxter, with four > co-developers, now in beta (release 0.4.1 at > http://sourceforge.net/projects/roundup). It's all Python, First bit - Richard's the roundup guy. I just do little helper bits. In pretty much every way, the above is giving me far too much credit. > (Of course, it must be usable for the end users > reporting bugs too. :-) What "usable" means here is possibly open to debate. Making the system as easy as possible to submit bugs means we get bugs like "From: Anonymous. Subject: python gives KeyError. Body: my homework gives a keyerror." > The point of this message is to start gathering requirements. Should this discussion continue on python-dev? > - A way to migrate the existing SF tracker contents. There are > references to 6-digit SF tracker IDs all over the Python source code > and CVS history, and it would be nice if these IDs were usable in > the new tracker. This should be doable. There's already an import facility. > - Features that simplify the tracking of how bugs and patches relate > to different release branches. This could ease the work for release > managers tremendously compared to the status quo. It should be > possible to indicate exactly which releases are affected by a bug, Doable. > and also in which releases it is (or will be) fixed. (Integration > with CVS would be nice but seems out of scope.) Integration with CVS in the sense of NeilS' message (look for magic keywords) would be possible with a variant on the standard mailgw code. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From Anthony Baxter Wed May 22 01:45:23 2002 From: Anthony Baxter (Anthony Baxter) Date: Wed, 22 May 2002 10:45:23 +1000 Subject: [Python-Dev] New bugtracker project In-Reply-To: Message from Aahz of "Tue, 21 May 2002 19:34:40 -0400." <20020521233440.GA13843@panix.com> Message-ID: <200205220045.g4M0jN807120@localhost.localdomain> >>> Aahz wrote > > This would be easy with XML-RPC (much easier than using urllib or httplib to > > submit to a plain old web server). > "Would be" or "is"? I.e., does Roundup currently support this? Nope. Never been asked for. Can't see it being that hard to add, tho. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From skip@pobox.com Wed May 22 01:45:40 2002 From: skip@pobox.com (Skip Montanaro) Date: Tue, 21 May 2002 19:45:40 -0500 Subject: [Python-Dev] New bugtracker project In-Reply-To: <200205211724.16876.gherron@islandtraining.com> References: <200205211638.g4LGcVw19247@odiug.zope.com> <15594.54104.523273.180886@12-248-41-177.client.attbi.com> <20020521233440.GA13843@panix.com> <200205211724.16876.gherron@islandtraining.com> Message-ID: <15594.59956.44876.258953@12-248-41-177.client.attbi.com> >> "Would be" or "is"? I.e., does Roundup currently support this? Gary> It does not. However, it does supply an email interface for the Gary> creation and modification of issues. One could write a local Gary> python script which creates and sends an email to the tracking Gary> session. This would also be suboptimal. The issues generating an email-based submission are no different than an http-based submission. The API is not really well-defined by the server author. This is left up to clients who need access to the server. If something changes in the layout of the email input or http form parameters, it's quite likely that clients will break. In addition, each client winds up having to write all their own specialized interface code, causing pottentially substantial duplication of effort. Use of XML-RPC or SOAP as remote scripting interface completely avoids these problems and open the system up to cross-language scripting. From Anthony Baxter Wed May 22 01:46:22 2002 From: Anthony Baxter (Anthony Baxter) Date: Wed, 22 May 2002 10:46:22 +1000 Subject: [Python-Dev] New bugtracker project In-Reply-To: Message from Aahz of "Tue, 21 May 2002 17:25:46 -0400." <20020521212546.GB19070@panix.com> Message-ID: <200205220046.g4M0kMI07135@localhost.localdomain> >>> Aahz wrote > * The system should work with Lynx and other non-standard browsers (this > is my biggest problem with SF) In "lynx" it's usable but ugly. In "links" or "w3m" it's fine. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From Anthony Baxter Wed May 22 01:59:02 2002 From: Anthony Baxter (Anthony Baxter) Date: Wed, 22 May 2002 10:59:02 +1000 Subject: [Python-Dev] New bugtracker project In-Reply-To: Message from David Ascher of "Tue, 21 May 2002 16:55:10 MST." <3CEADE5E.5090607@ActiveState.com> Message-ID: <200205220059.g4M0x2S07210@localhost.localdomain> >>> David Ascher wrote > 0) keywords -- massively useful for targeting, filtering, etc. Yes, but not in either of the standard templates. > 1) Attachments on bugs (somewhat of a merge of the patch tracker and > bug tracker, but also useful for attaching test cases, screenshots, etc.) Yes. It's also possible to add "state" to an attachment (ala bugzilla.mozilla.org). Can submit attachements via email. > 2) Being able to "watch" people - be notified when their bugs have > events happen to them, e.g.. when they're on vacation. You could install a simple reactor to say "when a nosy event happens for this user, also add that user to the nosy list." > 3) Saving and recalling searches This has been my #1 requested feature to Richard for some time. I'd ask him again, but I think he's sick of the nagging. :) > 4) non-web, non-email access to the database (e.g. take all the bugs > assigned to so and so, and take those submitted by people in england, > and set the keyword "never" to them). There's the roundupdb interface, where you can directly script the database. This requires local access (unless there's some sort of remote RPC for it). > 5) scalability (in # of developers, # of users, # of bugs, # of > 'versions', # of 'products', etc.) Unknown. > 6) Being able to migrate a bug from one product to another (not all that > relevant for a Python-only bug tracker, except if the versions are > handled as different products). Assuming we're not just talking about tweaking the metadata, this is an export/import task. Done. > 7) Bug linkages (depends on/blocks, duplicate handling, etc.) It's on the "to be done" feature list, looking for someone to do the work. > Useful things we've added to bugzilla include: > 1) Ability to designate bugs as 'for internal use only' (not especially > important for python-dev =) This is something that's being worked on at the moment. > 2) an 'efficient' UI. The bugzilla default search results pages are > very inefficient from a UI point of view. Andy McKay's version is much > better, I think. I don't see the difference in the two, aside from a smaller font? The UI for roundup's pretty much completely flexible. > As I've mentioned in the past, we used an earlier version of roundup and > found it didn't scale well with thousands of users. This was based on ping's initial prototype? The current roundup.sf.net project is a completely different thing, developed from scratch, and initially on the SC spec. > I haven't look at > the new roundup to know if it's dealt with the problems we had > (way too much mail generation by default, Ok. The way we have roundup configured is that a message is sent to a set of users when an issue is opened, and then to users that are "interested" subsequently - they've either commented on the bug, or added themselves to the nosy list. What did you find meant too much email? The "initial email out" was something we added. > much too brittle, Can't say I've seen that with the current version. > and poorly > architected =). Would need more details for this. > It's also important to have flexibility in handling the bug cycle (who > closes, who verifies, etc.). Both roundup and bugzilla did that pretty > well. Yep. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From tim_one@email.msn.com Wed May 22 03:45:40 2002 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 21 May 2002 22:45:40 -0400 Subject: [Python-Dev] New bugtracker project In-Reply-To: <200205220059.g4M0x2S07210@localhost.localdomain> Message-ID: If Gordon is working on this, I want to make it a requirement that the system not just track bugs, but fix them too. The man needs a challenge. and-we-need-the-help-ly y'rs - tim From Anthony Baxter Wed May 22 03:55:19 2002 From: Anthony Baxter (Anthony Baxter) Date: Wed, 22 May 2002 12:55:19 +1000 Subject: [Python-Dev] New bugtracker project In-Reply-To: Message from "Tim Peters" of "Tue, 21 May 2002 22:45:40 -0400." Message-ID: <200205220255.g4M2tJL08405@localhost.localdomain> >>> "Tim Peters" wrote > If Gordon is working on this, I want to make it a requirement that the > system not just track bugs, but fix them too. The man needs a challenge. Nah. It should just triage the bugs and assign them to the correct developer. Actually, one thing that Would Be Good is a different 'new issue' page for the end-user that prompts for all the info that we typically don't get. -- Anthony Baxter It's never too late to have a happy childhood. From martin@v.loewis.de Wed May 22 07:33:20 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 22 May 2002 08:33:20 +0200 Subject: [Python-Dev] New bugtracker project In-Reply-To: <3CEA8D3E.28934.14DE8F96@localhost> References: <20020521212546.GB19070@panix.com> <3CEA8D3E.28934.14DE8F96@localhost> Message-ID: "Gordon McMillan" writes: > > ... As an extension, it should be possibly to > > synchronize with other bugtrackers, in particular > > the Debian BTS. > > What does it mean to "synchronize bugtrackers"? Suppose somebody invokes 'debbug python', thus creating a bug report in the Debian bugtracker. Then I want to instruct the Python tracker to fetch the report from Debian, and create a report in the Python tracker. Later, all changes in one bug report should be reflected in the other - potentially by setting up an email account somewhere that receives email notifications from both trackers, analyses what changed, then programmatically performs the same changes in the other tracker. The exact procedure for synchronization will vary from tracker to tracker, so it is not a requirement that the tracker does that - the requirement is that the tracker can be fully controlled programmatically, and sends email notifications that allow easy automatic processing, preferably indicating what the change is. Regards, Martin From DavidA@ActiveState.com Wed May 22 07:40:13 2002 From: DavidA@ActiveState.com (David Ascher) Date: Tue, 21 May 2002 23:40:13 -0700 Subject: [Python-Dev] New bugtracker project References: <200205211638.g4LGcVw19247@odiug.zope.com> <20020521212546.GB19070@panix.com> <15594.54104.523273.180886@12-248-41-177.client.attbi.com> <20020521233440.GA13843@panix.com> <3CEADE5E.5090607@ActiveState.com> <15594.59525.30466.309094@12-248-41-177.client.attbi.com> Message-ID: <3CEB3D4D.6040609@ActiveState.com> Skip Montanaro wrote: >Can you elaborate what the differences are? Other than smaller fonts and >pink backgrounds I didn't notice any functional difference between the two >pages. In particular, they both seemed to have several columns of >information which could be used to sort the display in ascending or >descending order. (I use Opera. Maybe some other tidbits are available to >people using Netscape or MSIE.) > The choice of default columns and the font choices and the coloring make a big difference in how much information fits on a screen. It's minor stuff, but it makes big usability differences. >My biggest gripe with bugzilla has always been the damn search form. Talk >about an inefficient UI! If I need to submit a bug report to a >bugzilla-based system I refuse to even try to see if the bug has been >reported before. > Try ActiveState's -- we have a "simple search" form which does the right thing in 95% of bugs for any bug database with less than 50,000 bugs in it =). http://bugs.activestate.com/ActivePerl/query.cgi --da From DavidA@ActiveState.com Wed May 22 07:52:01 2002 From: DavidA@ActiveState.com (David Ascher) Date: Tue, 21 May 2002 23:52:01 -0700 Subject: [Python-Dev] New bugtracker project References: <200205220059.g4M0x2S07210@localhost.localdomain> Message-ID: <3CEB4011.4060304@ActiveState.com> Re: Anthony Baxter's comments: I am aware that NewRoundup is a brand new beast, and that's a very good thing. =) As Ping himself noted in his release, roundup wasn't really architected. We pushed it too hard, and paid the price. I'm quite confident that the new roundup is much better. I'm also confident that both the new roundup and bugzilla would be better than the SF tracker =). The fact that the new roundup is in Python is a huge win IMO compared to bugzilla -- I can't tweak bugzilla, since my Perl sucks. >>4) non-web, non-email access to the database (e.g. take all the bugs >>assigned to so and so, and take those submitted by people in england, >>and set the keyword "never" to them). >> >There's the roundupdb interface, where you can directly script the database. >This requires local access (unless there's some sort of remote RPC for it). > That's fine by me. >>5) scalability (in # of developers, # of users, # of bugs, # of >>'versions', # of 'products', etc.) >> > >Unknown. > The biggest issue I see with roundup there is that the original model at least tended to generate wayyy too much mail, and the nosy list concept didn't really work well for long-lived bugs (I didn't really analyze the deeper 'why', sorry -- maybe Mark's memory is better than mine =). For one thing, it was basically impossible for people with only a peripheral view of the database (managers, QA folks) to get accurate pictures of the database. Nothing that can't be fixed if one has a real DB backend. >>6) Being able to migrate a bug from one product to another (not all that >>relevant for a Python-only bug tracker, except if the versions are >>handled as different products). >> >Assuming we're not just talking about tweaking the metadata, this is an >export/import task. Done. > Well, it's an export task done on a bug -- and all the metadata goes along with it, if you see what I mean. It just shows up in a different set of queries. >>2) an 'efficient' UI. The bugzilla default search results pages are >>very inefficient from a UI point of view. Andy McKay's version is much >>better, I think. >> >I don't see the difference in the two, aside from a smaller font? The UI >for roundup's pretty much completely flexible. > My only point is that flexibility is fine, but getting the details right is what matters in the end. Bugzilla's UI is flexible (it's all Perl, after all), but the defaults suck. >Ok. The way we have roundup configured is that a message is sent to >a set of users when an issue is opened, and then to users that are >"interested" subsequently - they've either commented on the bug, or >added themselves to the nosy list. What did you find meant too much >email? The "initial email out" was something we added. > Yeah, we added the same thing. As I said, I forget the details of the failure. >Would need more details for this. > No you don't. =) Roundup used the filesystem as a database, and lots of things didnt' work right in high load situations. I'm confident that the new one has a much better design. --david PS: bugzilla _does_ have a learning curve, not too bad a one for users w/ some UI cleanup (as in the 'simple search'), but pretty steep if people want to become power-users. Bugzilla also has a _ton_ of features, and someone else is worrying about it. I hear that bugzilla is also not very 'open' as a project -- getting changes back into the core is hard, and some of the changes that are needed (refactoring to make the UI more easily configurable, for example, and the email interface) are hard to get 'in'. This is mostly hearsay though. From Anthony Baxter Wed May 22 08:15:27 2002 From: Anthony Baxter (Anthony Baxter) Date: Wed, 22 May 2002 17:15:27 +1000 Subject: [Python-Dev] New bugtracker project In-Reply-To: Message from David Ascher of "Tue, 21 May 2002 23:52:01 MST." <3CEB4011.4060304@ActiveState.com> Message-ID: <200205220715.g4M7FRU10099@localhost.localdomain> >>> David Ascher wrote > I'm also confident that > both the new roundup and bugzilla would be better than the SF tracker =). Aside from the multi-user nature, I think a bunch of postit notes beats the SF tracker in some ways. > The biggest issue I see with roundup there is that the original model at > least tended to generate wayyy too much mail, and the nosy list concept > didn't really work well for long-lived bugs (I didn't really analyze the > deeper 'why', sorry -- maybe Mark's memory is better than mine =). I'm not sure how and why the nosy list concept differs from the way bugzilla does it. If it's a matter of also having, say, a QA person cc'd on things that are in their area, well then that's all achievable without a lot of fuss. > For one thing, it was basically impossible for people with only a > peripheral view of the database (managers, QA folks) to get accurate > pictures of the database. Nothing that can't be fixed if one has a real > DB backend. Yep. The default roundupdb is various sorts of files, but there's not a lot of work to put a real SQL database behind it - it's abstracted out already. We were talking today about making a network-client-backend that talked over a network connection to a remote persistent roundupdb (it's something that we're starting to need here). > Well, it's an export task done on a bug -- and all the metadata goes > along with it, if you see what I mean. It just shows up in a different > set of queries. That should be pretty easy, assuming the existing export/import isn't what you want. > My only point is that flexibility is fine, but getting the details right > is what matters in the end. Bugzilla's UI is flexible (it's all Perl, > after all), but the defaults suck. Richard's comment was that the default that's there now "looks like the original roundup - it seemed like such a good idea at the time. *shudder*". Web usability and graphic design people are encouraged to volunteer. :) > No you don't. =) Roundup used the filesystem as a database, and lots > of things didnt' work right in high load situations. I'm confident that > the new one has a much better design. The newer one has abstracted out that stuff so that we can make it Do What We Want. Putting these two comments out of order and at the end: > I hear that bugzilla > is also not very 'open' as a project I've heard someone else comment similarly. > The fact that the new roundup is in Python is a huge win IMO compared to > bugzilla -- I can't tweak bugzilla, since my Perl sucks. I know I looked at bugzilla for work and gave up on it. It's just too ugly inside, and I really couldn't be bothered trying to figure out the right magic for changing it to do what I wanted. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From vinay_sajip@yahoo.co.uk Wed May 22 09:03:07 2002 From: vinay_sajip@yahoo.co.uk (Vinay Sajip) Date: Wed, 22 May 2002 09:03:07 +0100 Subject: [Python-Dev] PEP282 and the warnings framework References: <20020517142156.36404.qmail@web9602.mail.yahoo.com> <001a01c1fdb4$2a038960$652b6992@alpha> <20020517183702.E23086@prim.han.de> <000901c1fdd5$0d9c6820$652b6992@alpha> <3CEA21C2.8080904@livinglogic.de> Message-ID: <002101c20167$1e6b4140$652b6992@alpha> [Vinay] > > It might appear that allowing subclasses of LogRecord is the more natural > > thing to do. But this has the potential to stop me from extending > > LogRecord's functionality in the future, as some new attribute I introduce > > might conflict with the same-named attribute in some user's > > LogRecord-derived class. Since I want to preserve this freedom for myself > > and other maintainers of logging, [Walter] > But you have the same problem with Logger, which is made to be > subclassed. And Logger probably changes more often than LogRecord. Not necessarily. LogRecord is a grab-bag for attributes, I expect more volatility in LogRecord attributes (in the sense that even if I only make a similar number of attribute changes to LogRecord and to Logger, I would expect may more changes by users to LogRecord than to Logger - or at least, I wouldn't be surprised if that were the case). [Vinay] > > > > 1. Use a keyword argument, extra_info=None. You can use this on any call > > such as warn(), debug() etc. just as exc_info is currently used. A passed in > > value would become the extra_info attribute of the LogRecord. [Walter] > This smells to much like extensibility to per-OO way, i.e. simple add a > void *user_data to every struct you have and you're done. It's not necessarily a bad smell. It's analogous to the "inheritance or composition?" question. There are many cases where designers wrongly use inheritance where composition will suffice. > > 2. Use the "msg" parameter, which all logging calls already have, to have > > the semantics such that it is either a format string, or an instance such > > that str(msg) returns the desired format string. For the simple case, just > > pass in a string. For the more complex case, pass an instance - the caller > > can put any state in there that they want. It will not be used by the core > > logging module, except to call __str__() on it for the format string. But it > > can be used by user-derived classes, which will find it in the msg attribute > > of the LogRecord. This might seem "hackish" to some, but it's not really - > > it's just a question of packaging. > > This is more or less the same as 1.: use an (additional) argument, that > will become an attribute of the LogRecord. Yes - a minor variation. [Vinay] > > Slightly OT: There's a lot of people who think that the logging system > > should be a completely generalized system for event generation and > > processing. I've shied away from using a word like "Event", preferring > > "LogRecord" which tries to indicate that this is a focused class for a > > specific job. [Walter] > Even if LogRecord was renamed Event, it would be a "focused class for a > specific job", because it would be logging.Event. But the name is not so > important as the functionality. Yes, but focused for what job? Names *can* be important. I've committed my share of naming boo-boos in the past, and no doubt will in the future. But Event, even when qualified with "logging.", signifies to me a more general construct (e.g. used for more general messaging applications) than a "LogRecord" (used for holding information about a specific error/diagnostic logging event). I suppose LogEvent would be OK, too - I think I just picked LogRecord to echo the PEP. Regards, Vinay _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From vinay_sajip@yahoo.co.uk Wed May 22 09:03:14 2002 From: vinay_sajip@yahoo.co.uk (Vinay Sajip) Date: Wed, 22 May 2002 09:03:14 +0100 Subject: [Python-Dev] PEP282 and the warnings framework References: <20020515141648.9926.76949.Mailman@mail.python.org> <3CE283FE.8000403@campuspipeline.com> <20020515201342.T28033@prim.han.de> <3CE2AE95.4030900@campuspipeline.com> <20020516003321.U28033@prim.han.de> <3CE2FD51.2060301@campuspipeline.com> <3CE39B83.40006@livinglogic.de> <007501c1fcee$fa693760$652b6992@alpha> <3CE3FC18.4080509@livinglogic.de> <012f01c1fd22$bdd5b7e0$652b6992@alpha> <3CE4EBF1.1030303@livinglogic.de> <001101c1fd9d$cea1fd60$652b6992@alpha> <3CEA2847.6050507@livinglogic.de> Message-ID: <002801c20167$21fd41a0$652b6992@alpha> [Walter] > Not with something like this: > def log(self, lvl, msg, args, exc_info, eventtype=LogRecord): > if not isinstance(msg, LogRecord): > event = eventtype(msg, args) > with this solution you can still use a string for msg. This is equivalent to my suggestion: allow msg to be a class instance whose __str__() returns the format string. I know it's slightly different, but you can use it in the same sort of way, and we're discussing minor variations on a theme. [Walter] > BTW, is there a public CVS repository for the logging module? No - I don't think it warrants one, yet. I use version control so I can get past versions, but it's not on CVS anywhere - it's not really been a shared development as such, though a few people have been kind enough to submit small patches. If, as I hope, logging.py makes it into Python 2.3, it will be automatically in Python's CVS and I won't need to trouble to create one just for logging... Regards Vinay _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From mwh@python.net Wed May 22 10:14:35 2002 From: mwh@python.net (Michael Hudson) Date: 22 May 2002 10:14:35 +0100 Subject: [Python-Dev] New bugtracker project In-Reply-To: Guido van Rossum's message of "Tue, 21 May 2002 12:38:30 -0400" References: <200205211638.g4LGcVw19247@odiug.zope.com> Message-ID: <2mr8k4o878.fsf@starship.python.net> Guido van Rossum writes: > I believe that one of the areas where we could use help is a bug > tracker. If I may so bold, I suggest the distutils could use some (paid) tender loving care, too... [...] > The point of this message is to start gathering requirements. Gordon > will gather requirements, and has asked me to send out this email to > announce the project to the Python developer community. 1) An email interface (OK, so this has been said, and already exists, but...) 2) Not having this lunatic distinction between patches and bugs or restrictions on who can add files to issues, and let people delete at least their own attachments. Cheers, M. -- SPIDER: 'Scuse me. [scuttles off] ZAPHOD: One huge spider. FORD: Polite though. -- The Hitch-Hikers Guide to the Galaxy, Episode 11 From mwh@python.net Wed May 22 10:40:42 2002 From: mwh@python.net (Michael Hudson) Date: 22 May 2002 10:40:42 +0100 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: montanaro@sourceforge.net's message of "Tue, 21 May 2002 16:17:15 -0700" References: Message-ID: <2moff8o6zp.fsf@starship.python.net> montanaro@sourceforge.net writes: > Update of /cvsroot/python/python/dist/src/Lib > In directory usw-pr-cvs1:/tmp/cvs-serv23491 > > Modified Files: > types.py > Log Message: > add BooleanType I thought we weren't doing this anymore...? Cheers, M. -- [3] Modem speeds being what they are, large .avi files were generally downloaded to the shell server instead[4]. [4] Where they were usually found by the technical staff, and burned to CD. -- Carlfish, asr From David Abrahams" Message-ID: <05a901c2017e$869b0a90$6601a8c0@boostconsulting.com> Here's a final (?) development on the issue from the C++-sig... From: "Achim Domma" > > From: c++-sig-admin@python.org [mailto:c++-sig-admin@python.org]On > > Behalf Of David Abrahams > > Those in turn depend on (transitive closure): > > > ... > > SHLWAPI.DLL > ... > > Shlwapi.dll is the problem. The newest version of this dll introduces the > dependency to apphelp.dll, which should be no problem due to delayed > loading. I still don't know why an extension 'activates' the dependency to > apphelp.dll, but it's definitively a configuration problem of my system. So > it is a microsoft problem and I will try it in their newsgroups. > > thanks for you help > Achim ------ From: "Tim Peters" > [avid Abrahams] > > The summary is that python22.dll seems to have been built so that it > > depends on a DLL which only comes with Windows XP, though it strangely > > doesn't cause a problem until loading an extension module. > > The python22.dll distributed by PythonLabs was built on a Win98SE box. It > doesn't depend on apphelp.dll. I've never been on an XP box, and there is > no apphelp.dll on any machine I do use. Here are the system DLLs it does > depend on: > > ADVAPI32.DLL > KERNEL32.DLL > USER32.DLL > MSVCRT.DLL > SHELL32.DLL > > Those in turn depend on (transitive closure): > > COMCTL32.DLL > GDI32.DLL > NTDLL.DLL > RPCRT4.DLL > SHLWAPI.DLL > > That's it. > > From guido@python.org Wed May 22 12:31:30 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 22 May 2002 07:31:30 -0400 Subject: [Python-Dev] New bugtracker project In-Reply-To: Your message of "Tue, 21 May 2002 23:52:01 PDT." <3CEB4011.4060304@ActiveState.com> References: <200205220059.g4M0x2S07210@localhost.localdomain> <3CEB4011.4060304@ActiveState.com> Message-ID: <200205221131.g4MBVUx09765@pcp742651pcs.reston01.va.comcast.net> [David] > For one thing, it was basically impossible for people with only a > peripheral view of the database (managers, QA folks) to get accurate > pictures of the database. Nothing that can't be fixed if one has a > real DB backend. Speaking of which, how real is RoundUp's DB backend? Does it use MySQL? *Could* it use MySQL? *Should* it use MySQL? When this idea first came up, Andy Robinson asserted that it should, basically for the reason that David states above; I'm neutral but I certainly see the advantage of being able to run SQL(-ish) queries against the database (this helps for the SF tracker too). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed May 22 12:36:09 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 22 May 2002 07:36:09 -0400 Subject: [Python-Dev] New bugtracker project In-Reply-To: Your message of "22 May 2002 10:14:35 BST." <2mr8k4o878.fsf@starship.python.net> References: <200205211638.g4LGcVw19247@odiug.zope.com> <2mr8k4o878.fsf@starship.python.net> Message-ID: <200205221136.g4MBa9E09783@pcp742651pcs.reston01.va.comcast.net> [Michael] > If I may so bold, I suggest the distutils could use some (paid) tender > loving care, too... I know of a lot of other areas (my pet peeve: IDLEFORK). However, the scope of the current proposal needs to be limited to something that can be sold to the PBF as something that will significantly improve the quality of "conservative" releases (Python-in-a-tie, business release, name it what you want). Are there are specific platforms where distutils doesn't do the right thing? One of the PBF goals is also to have binary "sumo" distributions for many platforms (especially the platforms that commercial users love like Solaris, AIX and HP-UX). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed May 22 12:39:17 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 22 May 2002 07:39:17 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: Your message of "22 May 2002 10:40:42 BST." <2moff8o6zp.fsf@starship.python.net> References: <2moff8o6zp.fsf@starship.python.net> Message-ID: <200205221139.g4MBdHL09826@pcp742651pcs.reston01.va.comcast.net> > > Modified Files: > > types.py > > Log Message: > > add BooleanType > > I thought we weren't doing this anymore...? I have mixed feelings. On the one hand this really isn't necessary (you can just is isinstance(x, bool)). On the other hand for existing code that imports a bunch of types from types.py it may be more consistent to use BooleanType. --Guido van Rossum (home page: http://www.python.org/~guido/) From jacobs@penguin.theopalgroup.com Wed May 22 13:38:47 2002 From: jacobs@penguin.theopalgroup.com (Kevin Jacobs) Date: Wed, 22 May 2002 08:38:47 -0400 (EDT) Subject: [Python-Dev] New bugtracker project - Relational backend? In-Reply-To: <200205221131.g4MBVUx09765@pcp742651pcs.reston01.va.comcast.net> Message-ID: On Wed, 22 May 2002, Guido van Rossum wrote: > [David] > > For one thing, it was basically impossible for people with only a > > peripheral view of the database (managers, QA folks) to get accurate > > pictures of the database. Nothing that can't be fixed if one has a > > real DB backend. > > Speaking of which, how real is RoundUp's DB backend? Does it use > MySQL? *Could* it use MySQL? *Should* it use MySQL? When this idea > first came up, Andy Robinson asserted that it should, basically for > the reason that David states above; I'm neutral but I certainly see > the advantage of being able to run SQL(-ish) queries against the > database (this helps for the SF tracker too). My 2e-10 cents: NOTE: This is NOT a troll. YMMV, and all that. Relational backends are nice, especially when creating applications that benefit from having extensible reporting. However, from my experience, I would _not_ recommend MySQL for the following reasons: 1) Immature and flakey support for ACID transactions. 2) Relatively non-standards complaint SQL, though it does have lots of nifty extentions. However, for me, the advantages of being able to write fairly portable SQL92 queries far out-weighs what is gained by all of the syntactic sugar. 3) Does not scale well for many simultaneous readers and writers. Here are my recommendations for suitable open source relational backends with strong Python support (in no particular order, I use all three regularly): 1) PostgreSQL 2) Firebird 3) SAPdb The nice thing about all three is that they are have very good support for transactions, work hard to be SQL92 compliant, and typically scale well under demanding workloads. Porting between them is also fairly easy. Also looking forward to having a better bug-tracker than Bugzilla, -Kevin PS: One more thing to add to the Roundup wish list (if it isn't already there): Pervasive use of stylesheets for display-formatting content. The days of embedding such information inline are long since over. -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com From mal@lemburg.com Wed May 22 13:57:17 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 22 May 2002 14:57:17 +0200 Subject: [Python-Dev] New bugtracker project - Relational backend? References: Message-ID: <3CEB95AD.3010504@lemburg.com> Kevin Jacobs wrote: > On Wed, 22 May 2002, Guido van Rossum wrote: > >>[David] >> >>>For one thing, it was basically impossible for people with only a >>>peripheral view of the database (managers, QA folks) to get accurate >>>pictures of the database. Nothing that can't be fixed if one has a >>>real DB backend. >> >>Speaking of which, how real is RoundUp's DB backend? Does it use >>MySQL? *Could* it use MySQL? *Should* it use MySQL? When this idea >>first came up, Andy Robinson asserted that it should, basically for >>the reason that David states above; I'm neutral but I certainly see >>the advantage of being able to run SQL(-ish) queries against the >>database (this helps for the SF tracker too). > > > My 2e-10 cents: > > NOTE: This is NOT a troll. YMMV, and all that. > > Relational backends are nice, especially when creating applications that > benefit from having extensible reporting. However, from my experience, I > would _not_ recommend MySQL for the following reasons: > > 1) Immature and flakey support for ACID transactions. > 2) Relatively non-standards complaint SQL, though it does have lots > of nifty extentions. However, for me, the advantages of being able to > write fairly portable SQL92 queries far out-weighs what is gained by > all of the syntactic sugar. > 3) Does not scale well for many simultaneous readers and writers. 4) Recent versions only come with GPLed client libs, ie. the application using them gets infected by the GPL. The whole license issue is a general mess: see http://www.mysql.com/downloads/mysql-max-4.0.html > Here are my recommendations for suitable open source relational backends > with strong Python support (in no particular order, I use all three > regularly): > > 1) PostgreSQL > 2) Firebird > 3) SAPdb 4) Generic ODBC > The nice thing about all three is that they are have very good support for > transactions, work hard to be SQL92 compliant, and typically scale well > under demanding workloads. Porting between them is also fairly easy. > > Also looking forward to having a better bug-tracker than Bugzilla, > -Kevin > > PS: One more thing to add to the Roundup wish list (if it isn't already > there): Pervasive use of stylesheets for display-formatting content. > The days of embedding such information inline are long since over. +1 -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From guido@python.org Wed May 22 14:12:50 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 22 May 2002 09:12:50 -0400 Subject: [Python-Dev] New bugtracker project - Relational backend? In-Reply-To: Your message of "Wed, 22 May 2002 08:38:47 EDT." References: Message-ID: <200205221312.g4MDCoX09999@pcp742651pcs.reston01.va.comcast.net> > PS: One more thing to add to the Roundup wish list (if it isn't already > there): Pervasive use of stylesheets for display-formatting content. > The days of embedding such information inline are long since over. Semi-unrelated: a thing to consider for flexible presentation might be Zope Page Templates. The core TAL (Template Attribute Language) code is independent from the Zope infrastructure. http://cvs.zope.org/Packages/TAL/ (Read the README.txt file.) (Disclaimer: I was involved in the design and implementation.) --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com Wed May 22 14:30:25 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 22 May 2002 08:30:25 -0500 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: <200205221139.g4MBdHL09826@pcp742651pcs.reston01.va.comcast.net> References: <2moff8o6zp.fsf@starship.python.net> <200205221139.g4MBdHL09826@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15595.40305.965276.823314@12-248-41-177.client.attbi.com> skip> add BooleanType mwh> I thought we weren't doing this anymore...? Guido> On the other hand for existing code that imports a bunch of types Guido> from types.py it may be more consistent to use BooleanType. You find this in a lot of code: from types import * Given that every other type is exposed through the types module it just seems kind of odd to force people to use isinstance() for bools, but have the old way of doing things continue to work for every other type. I think you can stop adding new types to the types module if and when you deprecate it, but not before. Skip From Anthony Baxter Wed May 22 14:54:59 2002 From: Anthony Baxter (Anthony Baxter) Date: Wed, 22 May 2002 23:54:59 +1000 Subject: [Python-Dev] New bugtracker project In-Reply-To: Message from Guido van Rossum of "Wed, 22 May 2002 07:31:30 -0400." <200205221131.g4MBVUx09765@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200205221355.g4MDsxP12701@localhost.localdomain> >>> Guido van Rossum wrote > Speaking of which, how real is RoundUp's DB backend? Have a look at the backends. the basic one is http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/roundup/roundup/roundup/backends/back_anydbm.py (sorry bout the URL). It's been on my round-tuit list sometime to look at the best way to do an Oracle (or db-sig, to be more precise) backend, but it's never bubbled to the top. > Does it use > MySQL? *Could* it use MySQL? *Should* it use MySQL? Not currently, probably, no. in that order. If we're going to use an RDBMS, it should be postgres, not mysql. > When this idea > first came up, Andy Robinson asserted that it should, basically for > the reason that David states above; I'm neutral but I certainly see > the advantage of being able to run SQL(-ish) queries against the > database (this helps for the SF tracker too). I suspect that knowing Richard, and seeing where he's been spending his train time lately, that there might be a gadfly-bolted-onto-roundupdb in some form soon. But I could be wrong... :) Anthony -- Anthony Baxter It's never too late to have a happy childhood. From paul-python@svensson.org Wed May 22 15:59:11 2002 From: paul-python@svensson.org (Paul Svensson) Date: Wed, 22 May 2002 10:59:11 -0400 (EDT) Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: <15595.40305.965276.823314@12-248-41-177.client.attbi.com> Message-ID: On Wed, 22 May 2002, Skip Montanaro wrote: >You find this in a lot of code: > > from types import * > >Given that every other type is exposed through the types module it just >seems kind of odd to force people to use isinstance() for bools, but have >the old way of doing things continue to work for every other type. I think >you can stop adding new types to the types module if and when you deprecate >it, but not before. Assuming there's no doubt that it will eventually go away, wouldn't it be better to depreceate it sooner rather than later, at least in the documentation ? /Paul From guido@python.org Wed May 22 16:06:05 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 22 May 2002 11:06:05 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: Your message of "Wed, 22 May 2002 08:30:25 CDT." <15595.40305.965276.823314@12-248-41-177.client.attbi.com> References: <2moff8o6zp.fsf@starship.python.net> <200205221139.g4MBdHL09826@pcp742651pcs.reston01.va.comcast.net> <15595.40305.965276.823314@12-248-41-177.client.attbi.com> Message-ID: <200205221506.g4MF65T10184@pcp742651pcs.reston01.va.comcast.net> > skip> add BooleanType > > mwh> I thought we weren't doing this anymore...? > > Guido> On the other hand for existing code that imports a bunch of types > Guido> from types.py it may be more consistent to use BooleanType. > > You find this in a lot of code: > > from types import * > > Given that every other type is exposed through the types module it > just seems kind of odd to force people to use isinstance() for > bools, but have the old way of doing things continue to work for > every other type. I think you can stop adding new types to the > types module if and when you deprecate it, but not before. I don't see where I'm forcing people to use isinstance(). BooleanType is exactly the same object as bool -- you can use 'type(x) is bool' or 'isinstance(x, BooleanType)' if you want to. I think the plan is to stop recommending the types module -- maybe deprecating it too. --Guido van Rossum (home page: http://www.python.org/~guido/) From gmcm@hypernet.com Wed May 22 17:03:50 2002 From: gmcm@hypernet.com (Gordon McMillan) Date: Wed, 22 May 2002 12:03:50 -0400 Subject: [Python-Dev] New bugtracker project In-Reply-To: References: <3CEA8D3E.28934.14DE8F96@localhost> Message-ID: <3CEB8926.11768.18B69207@localhost> On 22 May 2002 at 8:33, Martin v. Loewis wrote: > "Gordon McMillan" writes: [snip] > > What does it mean to "synchronize bugtrackers"? > > Suppose somebody invokes 'debbug python', thus > creating a bug report in the Debian bugtracker. Then > I want to instruct the Python tracker to fetch the > report from Debian, and create a report in the > Python tracker. Later, all changes in one bug > report should be reflected in the other - > potentially by setting up an email account somewhere > that receives email notifications from both > trackers, analyses what changed, then > programmatically performs the same changes in the > other tracker. But "fixing" it in Debian doesn't necessarily mean that it's "fixed" in Python. > The exact procedure for synchronization will vary > from tracker to tracker, so it is not a requirement > that the tracker does that - the requirement is that > the tracker can be fully controlled > programmatically, and sends email notifications that > allow easy automatic processing, preferably > indicating what the change is. That seems perfectly sensible. -- Gordon http://www.mcmillan-inc.com/ From gmcm@hypernet.com Wed May 22 17:03:50 2002 From: gmcm@hypernet.com (Gordon McMillan) Date: Wed, 22 May 2002 12:03:50 -0400 Subject: [Python-Dev] New bugtracker project In-Reply-To: <200205220715.g4M7FRU10099@localhost.localdomain> References: Message from David Ascher of "Tue, 21 May 2002 23:52:01 MST." <3CEB4011.4060304@ActiveState.com> Message-ID: <3CEB8926.10338.18B6926B@localhost> On 22 May 2002 at 17:15, Anthony Baxter wrote: [snip] > Yep. The default roundupdb is various sorts of > files, but there's not a lot of work to put a real > SQL database behind it - it's abstracted out > already. Hmm. From a quick look, it appears that the abstraction tends to hide the roundup schema from the DB schema. I suspect that what Andy wants is for the DB schema to reflect the roundup schema so he can query directly on those "arbitrary" properties. -- Gordon http://www.mcmillan-inc.com/ From gmcm@hypernet.com Wed May 22 17:03:50 2002 From: gmcm@hypernet.com (Gordon McMillan) Date: Wed, 22 May 2002 12:03:50 -0400 Subject: [Python-Dev] New bugtracker project In-Reply-To: <2mr8k4o878.fsf@starship.python.net> References: Guido van Rossum's message of "Tue, 21 May 2002 12:38:30 -0400" Message-ID: <3CEB8926.16289.18B692CF@localhost> On 22 May 2002 at 10:14, Michael Hudson wrote: > 2) Not having this lunatic distinction between > patches and bugs ... They seem distinct to me, but maybe I'm a bit old-fashioned :-). What artificial distinction do you find irksome? -- Gordon http://www.mcmillan-inc.com/ From aahz@pythoncraft.com Wed May 22 17:16:55 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 22 May 2002 12:16:55 -0400 Subject: [Python-Dev] New bugtracker project In-Reply-To: <3CEB8926.16289.18B692CF@localhost> References: <2mr8k4o878.fsf@starship.python.net> <3CEB8926.16289.18B692CF@localhost> Message-ID: <20020522161655.GB4869@panix.com> On Wed, May 22, 2002, Gordon McMillan wrote: > On 22 May 2002 at 10:14, Michael Hudson wrote: >> >> 2) Not having this lunatic distinction between >> patches and bugs ... > > They seem distinct to me, but maybe I'm a bit old-fashioned :-). What > artificial distinction do you find irksome? He's talking about SourceForge. If a bug has a patch, it should be directly part of the bug, not a separate PatchTracker submission. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "In the end, outside of spy agencies, people are far too trusting and willing to help." --Ira Winkler From mwh@python.net Wed May 22 17:26:53 2002 From: mwh@python.net (Michael Hudson) Date: 22 May 2002 17:26:53 +0100 Subject: [Python-Dev] New bugtracker project In-Reply-To: Guido van Rossum's message of "Tue, 21 May 2002 12:38:30 -0400" References: <200205211638.g4LGcVw19247@odiug.zope.com> Message-ID: <2madqsdu7m.fsf@starship.python.net> Guido van Rossum writes: > The point of this message is to start gathering requirements. Gordon > will gather requirements, and has asked me to send out this email to > announce the project to the Python developer community. Some of my > own requirements: Ooh! I just stumbled into another one: serving up text attachments as Content-Type: text/plain! I'm not the only person who gets pissed off with having to save files to disk for no good reason, am I? Cheers, M. -- ... but I guess there are some things that are so gross you just have to forget, or it'll destroy something within you. perl is the first such thing I have known. -- Erik Naggum, comp.lang.lisp From mwh@python.net Wed May 22 17:06:51 2002 From: mwh@python.net (Michael Hudson) Date: Wed, 22 May 2002 17:06:51 +0100 (BST) Subject: [Python-Dev] New bugtracker project In-Reply-To: <3CEB8926.16289.18B692CF@localhost> Message-ID: On Wed, 22 May 2002, Gordon McMillan wrote: > On 22 May 2002 at 10:14, Michael Hudson wrote: > > > 2) Not having this lunatic distinction between > > patches and bugs ... > > They seem distinct to me, but maybe I'm > a bit old-fashioned :-). What artificial > distinction do you find irksome? Well, I think it's the way bugs sometimes have patches attached to them, or sometimes a patch gets put in the patch tracker and a link to it posted in the bug tracker. I guess there is a distinction, but it's too high up on sf. You can't get a list of all issues assigned to one developer, for instance, or the fact that the "groups" field for the various trackers has diverged over much. I really could have done with a "2.2.1 issues" view -- instead I had to ping back and forth between bugs and patches. Etc. Does that explain my grievance? I can try harder if you want. Cheers, M. From pyth@devel.trillke.net Wed May 22 17:33:34 2002 From: pyth@devel.trillke.net (holger krekel) Date: Wed, 22 May 2002 18:33:34 +0200 Subject: [Python-Dev] New bugtracker project In-Reply-To: <3CEB8926.16289.18B692CF@localhost>; from gmcm@hypernet.com on Wed, May 22, 2002 at 12:03:50PM -0400 References: <2mr8k4o878.fsf@starship.python.net> <3CEB8926.16289.18B692CF@localhost> Message-ID: <20020522183334.B26513@prim.han.de> Gordon McMillan wrote: > On 22 May 2002 at 10:14, Michael Hudson wrote: > > > 2) Not having this lunatic distinction between > > patches and bugs ... > > They seem distinct to me, but maybe I'm > a bit old-fashioned :-). What artificial > distinction do you find irksome? i somewhat agree to michael. It becomes obvious when you want to submit a *patch for a bug* that you discovered. Even if reporting a bug and providing a patch is not done by the same person it's still one "issue" where the bug-submitter can help by testing the patch etc.. if-it's-not-a-bug-don't-fix-it-ly yours, Holger regards, holger From barry@zope.com Wed May 22 17:45:32 2002 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 22 May 2002 12:45:32 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 References: <2moff8o6zp.fsf@starship.python.net> <200205221139.g4MBdHL09826@pcp742651pcs.reston01.va.comcast.net> <15595.40305.965276.823314@12-248-41-177.client.attbi.com> <200205221506.g4MF65T10184@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15595.52012.527627.751636@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> I think the plan is to stop recommending the types module -- GvR> maybe deprecating it too. Why? Isn't this still a useful idiom: from types import * ... if isinstance(x, StringTypes): # blah Yah, I know many of the fundamental types are available as builtins, but not all of them. Or are we saying that we'd now rather see the above written as: if isinstance(x, (str, unicode)): # blah ? -Barry From skip@pobox.com Wed May 22 17:58:09 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 22 May 2002 11:58:09 -0500 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: <200205221506.g4MF65T10184@pcp742651pcs.reston01.va.comcast.net> References: <2moff8o6zp.fsf@starship.python.net> <200205221139.g4MBdHL09826@pcp742651pcs.reston01.va.comcast.net> <15595.40305.965276.823314@12-248-41-177.client.attbi.com> <200205221506.g4MF65T10184@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15595.52769.255064.888318@12-248-41-177.client.attbi.com> Guido> I don't see where I'm forcing people to use isinstance(). Assume you've got a module that uses elements of the types module. Now you need to add some code to it that manipulates boolean objects. Without BooleanType you have two choices: * convert all your uses of types.* objects to use isinstance() or "type(x) is something". Your code is uniform. * leave your use of types.* objects alone and just use isinstance() or "type(x) is bool" for boolean objects. Your code works, but treats booleans different than other objects. All I'm doing by adding BooleanType to types is providing a third option: * do nothing. Given that types is not yet deprecated, I think that's a viable option. I'd be happy to see the types module deprecated, but it isn't at this point in time, so I believe it should support BooleanType. Guido> I think the plan is to stop recommending the types module -- Guido> maybe deprecating it too. deprecate-away-ly, y'rs, Skip From tim.one@comcast.net Wed May 22 18:07:47 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 22 May 2002 13:07:47 -0400 Subject: [Python-Dev] New bugtracker project In-Reply-To: <3CEB8926.16289.18B692CF@localhost> Message-ID: [Michael Hudson] > > 2) Not having this lunatic distinction between > > patches and bugs ... [Gordon McMillan} > They seem distinct to me, but maybe I'm > a bit old-fashioned :-). What artificial > distinction do you find irksome? That they're in entirely distinct trackers. Would be much better if everything were in one tracker, with a bug-vs-patch-vs-feature tag for filtering displays. It's just extra work for everyone when, e.g., someone opens an SF bug in "the bug tracker" and then someone submits a patch to fix that bug in "the patch tracker". Now you've got two disconnected blobs (distinct URLs, distinct "id"s, distinct comments and descriptions, etc). From trentm@ActiveState.com Wed May 22 18:08:46 2002 From: trentm@ActiveState.com (Trent Mick) Date: Wed, 22 May 2002 10:08:46 -0700 Subject: [Python-Dev] New bugtracker project In-Reply-To: <3CEADE5E.5090607@ActiveState.com>; from DavidA@ActiveState.com on Tue, May 21, 2002 at 04:55:10PM -0700 References: <200205211638.g4LGcVw19247@odiug.zope.com> <20020521212546.GB19070@panix.com> <15594.54104.523273.180886@12-248-41-177.client.attbi.com> <20020521233440.GA13843@panix.com> <3CEADE5E.5090607@ActiveState.com> Message-ID: <20020522100846.C24109@ActiveState.com> [David Ascher wrote] > My list of requirements isn't so much a list of reqs as a list of > features from bugzilla that we've found exceedingly useful. I'll add another one to this: - After you submit changes to a bug in bugzilla via the Web interface you are told to whom an email update was sent and to whom it was NOT sent. This implies that the bug tracking system allows users to select under what conditions they would like to receive email (e.g. "I would like to receive an email for any changes if I own a bug, but only for status changes if I am the QA contact on a bug.) Trent -- Trent Mick TrentM@ActiveState.com From guido@python.org Wed May 22 18:16:59 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 22 May 2002 13:16:59 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: Your message of "Wed, 22 May 2002 12:45:32 EDT." <15595.52012.527627.751636@anthem.wooz.org> References: <2moff8o6zp.fsf@starship.python.net> <200205221139.g4MBdHL09826@pcp742651pcs.reston01.va.comcast.net> <15595.40305.965276.823314@12-248-41-177.client.attbi.com> <200205221506.g4MF65T10184@pcp742651pcs.reston01.va.comcast.net> <15595.52012.527627.751636@anthem.wooz.org> Message-ID: <200205221716.g4MHGxX10625@pcp742651pcs.reston01.va.comcast.net> > GvR> I think the plan is to stop recommending the types module -- > GvR> maybe deprecating it too. [BAW] > Why? Isn't this still a useful idiom: > > from types import * > ... > if isinstance(x, StringTypes): > # blah > > Yah, I know many of the fundamental types are available as builtins, > but not all of them. Or are we saying that we'd now rather see the > above written as: > > if isinstance(x, (str, unicode)): > # blah > > ? You've got a point there. (str, unicode) is no good in the standard library since Python might be compiled without Unicode support. --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Wed May 22 18:14:36 2002 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 22 May 2002 13:14:36 -0400 Subject: [Python-Dev] New bugtracker project References: <3CEB8926.16289.18B692CF@localhost> Message-ID: <15595.53756.795411.555473@anthem.wooz.org> >>>>> "MH" == Michael Hudson writes: MH> Well, I think it's the way bugs sometimes have patches MH> attached to them, or sometimes a patch gets put in the patch MH> tracker and a link to it posted in the bug tracker. In fact, it is really hard to link issues in SF, especially if they span trackers. Add to it the fact that you have to know in advance whether an issue number is a bug or a patch (or a feature request) before you can search for it in the SF search box. www.python.org/sf helps with that, but it's still very inconvenient. -Barry From barry@zope.com Wed May 22 18:15:20 2002 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 22 May 2002 13:15:20 -0400 Subject: [Python-Dev] New bugtracker project - Relational backend? References: <200205221312.g4MDCoX09999@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15595.53800.540281.154909@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> Semi-unrelated: a thing to consider for flexible presentation GvR> might be Zope Page Templates. The core TAL (Template GvR> Attribute Language) code is independent from the Zope GvR> infrastructure. While you're at it, why not ZODB as the back end "database"? :) -Barry From DavidA@ActiveState.com Wed May 22 18:15:12 2002 From: DavidA@ActiveState.com (David Ascher) Date: Wed, 22 May 2002 10:15:12 -0700 Subject: [Python-Dev] New bugtracker project References: Guido van Rossum's message of "Tue, 21 May 2002 12:38:30 -0400" <3CEB8926.16289.18B692CF@localhost> Message-ID: <3CEBD220.8070706@ActiveState.com> Gordon McMillan wrote: >>2) Not having this lunatic distinction between >>patches and bugs ... >> >> > >They seem distinct to me, but maybe I'm >a bit old-fashioned :-). What artificial >distinction do you find irksome? > > As someone said, he's referring to the SF concepts. In Mozilla land, bugzilla is used for everything from bug management, patch management (it is the repository on which people indicate who'se reviewed what patch, when it was checked in, etc.), mailing-list-proxy and random banter (e.g. the "give hyatt $50" bug), user feedback mechanism (via bug voting, which AFAICT doesn't "work") as well as general project management. I think it's a lousy tool for project management, but then again I think all the tools I've seen are lousy tools for project management =). It is a feature that bugzilla is corruptible that way, one which would be good to have in the new system. The key low-level feature which enables this is keywords and a peer-enforced policy rather than rule-based policy (as in "_please_ don't change milestones if you're not part of the PDT team" or such). --david From guido@python.org Wed May 22 18:20:38 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 22 May 2002 13:20:38 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: Your message of "Wed, 22 May 2002 11:58:09 CDT." <15595.52769.255064.888318@12-248-41-177.client.attbi.com> References: <2moff8o6zp.fsf@starship.python.net> <200205221139.g4MBdHL09826@pcp742651pcs.reston01.va.comcast.net> <15595.40305.965276.823314@12-248-41-177.client.attbi.com> <200205221506.g4MF65T10184@pcp742651pcs.reston01.va.comcast.net> <15595.52769.255064.888318@12-248-41-177.client.attbi.com> Message-ID: <200205221720.g4MHKcs10667@pcp742651pcs.reston01.va.comcast.net> > Guido> I don't see where I'm forcing people to use isinstance(). > > Assume you've got a module that uses elements of the types module. Now you > need to add some code to it that manipulates boolean objects. Without > BooleanType you have two choices: > > * convert all your uses of types.* objects to use isinstance() > or "type(x) is something". Your code is uniform. > > * leave your use of types.* objects alone and just use > isinstance() or "type(x) is bool" for boolean objects. Your > code works, but treats booleans different than other objects. > > All I'm doing by adding BooleanType to types is providing a third option: > > * do nothing. I'm still confused. How would you then *use* BooleanType? The only uses for the types.* names I see are either with isinstance() or with "type(x) is T". Apparently there's another use that I'm missing, otherwise your first bullet would already be the "do nothing" case. Can you give a code example using BooleanType? --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com Wed May 22 19:12:04 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 22 May 2002 13:12:04 -0500 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: <200205221720.g4MHKcs10667@pcp742651pcs.reston01.va.comcast.net> References: <2moff8o6zp.fsf@starship.python.net> <200205221139.g4MBdHL09826@pcp742651pcs.reston01.va.comcast.net> <15595.40305.965276.823314@12-248-41-177.client.attbi.com> <200205221506.g4MF65T10184@pcp742651pcs.reston01.va.comcast.net> <15595.52769.255064.888318@12-248-41-177.client.attbi.com> <200205221720.g4MHKcs10667@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15595.57204.624940.28865@12-248-41-177.client.attbi.com> Guido> Can you give a code example using BooleanType? http://sourceforge.net/tracker/index.php?func=detail&aid=559288&group_id=5470&atid=305470 S From aahz@pythoncraft.com Wed May 22 19:21:14 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 22 May 2002 14:21:14 -0400 Subject: [Python-Dev] New bugtracker project In-Reply-To: <3CEBD220.8070706@ActiveState.com> References: <3CEB8926.16289.18B692CF@localhost> <3CEBD220.8070706@ActiveState.com> Message-ID: <20020522182114.GA3002@panix.com> On Wed, May 22, 2002, David Ascher wrote: > > As someone said, he's referring to the SF concepts. In Mozilla land, > bugzilla is used for everything from bug management, patch management > (it is the repository on which people indicate who'se reviewed what > patch, when it was checked in, etc.), mailing-list-proxy and random > banter (e.g. the "give hyatt $50" bug), user feedback mechanism (via bug > voting, which AFAICT doesn't "work") as well as general project > management. I think it's a lousy tool for project management, but then > again I think all the tools I've seen are lousy tools for project > management =). > > It is a feature that bugzilla is corruptible that way, one which would > be good to have in the new system. The key low-level feature which > enables this is keywords and a peer-enforced policy rather than > rule-based policy (as in "_please_ don't change milestones if you're not > part of the PDT team" or such). Well, no, I don't agree with that last part. I'm all in favor of rolling all possible project tracking features into a single tool, but that doesn't mean one wants all features available to every user, and it especially doesn't mean that one wants all features available to every user for any specific subset of system records. For example, I think it's highly appropriate that if we use this system for actual project management that project goals assigned to a specific release get "locked" to everyone but Guido after Guido has approved assignment of that goal to that release. I think preventing stupid mistakes is an eminently reasonable use of the system. Which reminds me of another feature request: * CVS-like system rollbacks to defined states (minimum) or archiving of every change (preferred). -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "In the end, outside of spy agencies, people are far too trusting and willing to help." --Ira Winkler From guido@python.org Wed May 22 19:37:19 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 22 May 2002 14:37:19 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: Your message of "Wed, 22 May 2002 13:12:04 CDT." <15595.57204.624940.28865@12-248-41-177.client.attbi.com> References: <2moff8o6zp.fsf@starship.python.net> <200205221139.g4MBdHL09826@pcp742651pcs.reston01.va.comcast.net> <15595.40305.965276.823314@12-248-41-177.client.attbi.com> <200205221506.g4MF65T10184@pcp742651pcs.reston01.va.comcast.net> <15595.52769.255064.888318@12-248-41-177.client.attbi.com> <200205221720.g4MHKcs10667@pcp742651pcs.reston01.va.comcast.net> <15595.57204.624940.28865@12-248-41-177.client.attbi.com> Message-ID: <200205221837.g4MIbJL10931@pcp742651pcs.reston01.va.comcast.net> > Guido> Can you give a code example using BooleanType? > > http://sourceforge.net/tracker/index.php?func=detail&aid=559288&group_id=5470&atid=305470 You mean the use of dispatch[BooleanType]? You can use dispatch[bool] just as well -- 'bool' is the same object as 'BooleanType'. So I'm still confused why you'd have to do anything. --Guido van Rossum (home page: http://www.python.org/~guido/) From martin@v.loewis.de Wed May 22 19:47:47 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 22 May 2002 20:47:47 +0200 Subject: [Python-Dev] New bugtracker project In-Reply-To: <20020522161655.GB4869@panix.com> References: <2mr8k4o878.fsf@starship.python.net> <3CEB8926.16289.18B692CF@localhost> <20020522161655.GB4869@panix.com> Message-ID: Aahz writes: > >> 2) Not having this lunatic distinction between > >> patches and bugs ... > > > > They seem distinct to me, but maybe I'm a bit old-fashioned :-). What > > artificial distinction do you find irksome? > > He's talking about SourceForge. If a bug has a patch, it should be > directly part of the bug, not a separate PatchTracker submission. And indeed, on SourceForge, it can be directly part of the bug report. Regards, Martin From martin@v.loewis.de Wed May 22 19:49:45 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 22 May 2002 20:49:45 +0200 Subject: [Python-Dev] New bugtracker project In-Reply-To: <20020522183334.B26513@prim.han.de> References: <2mr8k4o878.fsf@starship.python.net> <3CEB8926.16289.18B692CF@localhost> <20020522183334.B26513@prim.han.de> Message-ID: holger krekel writes: > if-it's-not-a-bug-don't-fix-it-ly yours, Holger Tell that the submitters of the 50 or so patches for Python which don't fix bugs, but add features. Perhaps you can talk some of them into withdrawing their patch... Regards, Martin From martin@v.loewis.de Wed May 22 19:56:16 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 22 May 2002 20:56:16 +0200 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: <15595.40305.965276.823314@12-248-41-177.client.attbi.com> References: <2moff8o6zp.fsf@starship.python.net> <200205221139.g4MBdHL09826@pcp742651pcs.reston01.va.comcast.net> <15595.40305.965276.823314@12-248-41-177.client.attbi.com> Message-ID: Skip Montanaro writes: > You find this in a lot of code: > > from types import * > > Given that every other type is exposed through the types module it just > seems kind of odd to force people to use isinstance() for bools, but have > the old way of doing things continue to work for every other type. What old way? You mean, you want to write type(x) is BooleanType You can write just as fine type(x) is bool > I think you can stop adding new types to the types module if and > when you deprecate it, but not before. I feel adding BooleanType was a bad thing. It is not needed; there is only one way to do it. Regards, Martin From martin@v.loewis.de Wed May 22 19:59:00 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 22 May 2002 20:59:00 +0200 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: <200205221837.g4MIbJL10931@pcp742651pcs.reston01.va.comcast.net> References: <2moff8o6zp.fsf@starship.python.net> <200205221139.g4MBdHL09826@pcp742651pcs.reston01.va.comcast.net> <15595.40305.965276.823314@12-248-41-177.client.attbi.com> <200205221506.g4MF65T10184@pcp742651pcs.reston01.va.comcast.net> <15595.52769.255064.888318@12-248-41-177.client.attbi.com> <200205221720.g4MHKcs10667@pcp742651pcs.reston01.va.comcast.net> <15595.57204.624940.28865@12-248-41-177.client.attbi.com> <200205221837.g4MIbJL10931@pcp742651pcs.reston01.va.comcast.net> Message-ID: Guido van Rossum writes: > You mean the use of dispatch[BooleanType]? You can use dispatch[bool] > just as well -- 'bool' is the same object as 'BooleanType'. So I'm > still confused why you'd have to do anything. I think Skip still assumes that bool is a function. Regards, Martin From gmcm@hypernet.com Wed May 22 20:16:04 2002 From: gmcm@hypernet.com (Gordon McMillan) Date: Wed, 22 May 2002 15:16:04 -0400 Subject: [Python-Dev] New bugtracker project In-Reply-To: <20020522100846.C24109@ActiveState.com> References: <3CEADE5E.5090607@ActiveState.com>; from DavidA@ActiveState.com on Tue, May 21, 2002 at 04:55:10PM -0700 Message-ID: <3CEBB634.11681.1966903A@localhost> On 22 May 2002 at 10:08, Trent Mick wrote: > I'll add another one to this: > > - After you submit changes to a bug in bugzilla via > the Web interface you are told to whom an email > update was sent and to whom it was NOT sent. This > implies that the bug tracking system allows users > to select under what conditions they would like to > receive email (e.g. "I would like to receive an > email for any changes if I own a bug, but only for > status changes if I am the QA contact on a bug.) Which part of this are you deeming "useful"? Seeing the list of to whom it was NOT sent[1]? Or the fact that there seems to be a fancy filtering / triggering mechanism behind it? -- Gordon http://www.mcmillan-inc.com/ [1] scalability alert From pyth@devel.trillke.net Wed May 22 20:24:21 2002 From: pyth@devel.trillke.net (holger krekel) Date: Wed, 22 May 2002 21:24:21 +0200 Subject: [Python-Dev] New bugtracker project In-Reply-To: ; from martin@v.loewis.de on Wed, May 22, 2002 at 08:49:45PM +0200 References: <2mr8k4o878.fsf@starship.python.net> <3CEB8926.16289.18B692CF@localhost> <20020522183334.B26513@prim.han.de> Message-ID: <20020522212421.E26513@prim.han.de> Martin v. Loewis wrote: > holger krekel writes: > > > if-it's-not-a-bug-don't-fix-it-ly yours, Holger > > Tell that the submitters of the 50 or so patches for Python which > don't fix bugs, but add features. Perhaps you can talk some of them > into withdrawing their patch... For an appropriate definition of 'bug' they fix a bug. Seriously, even implementations of PEPs fix a bug in that the current behaviour is not how it *should* be. I guess the problem is that what some people think what *should* be in python is not what the core developers (or GvR at last) thinks. That's fine! The issue/feature/bug/patch should be rejected or at least deferred then. Why are the 50 or so patches still in the queue? holger From martin@v.loewis.de Wed May 22 20:33:14 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 22 May 2002 21:33:14 +0200 Subject: [Python-Dev] New bugtracker project In-Reply-To: <20020522212421.E26513@prim.han.de> References: <2mr8k4o878.fsf@starship.python.net> <3CEB8926.16289.18B692CF@localhost> <20020522183334.B26513@prim.han.de> <20020522212421.E26513@prim.han.de> Message-ID: holger krekel writes: > Why are the 50 or so patches still in the queue? There are 130 bugs in the back log (sf.net/projects/python), because nobody reviews them. This, in turn, is because there are no volunteers to try them out, and recommend patches for acceptal or rejection (or suggest improvements to the submitter). Some of the do fix bugs, so the 50 was an estimate. Regards, Martin From martin@v.loewis.de Wed May 22 19:52:33 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 22 May 2002 20:52:33 +0200 Subject: [Python-Dev] New bugtracker project In-Reply-To: <2madqsdu7m.fsf@starship.python.net> References: <200205211638.g4LGcVw19247@odiug.zope.com> <2madqsdu7m.fsf@starship.python.net> Message-ID: Michael Hudson writes: > Ooh! I just stumbled into another one: serving up text attachments as > Content-Type: text/plain! I'm not the only person who gets pissed off > with having to save files to disk for no good reason, am I? I think SF works correctly, here: it records the MIME type of each attachment if one was provided during upload (or somehow guesses if there was not), then hands out the patch with the same MIME type. I'm more annoyed about the Mozilla bug that it insists to call the file download.php, even though SF properly reports the original file name. Regards, Martin From skip@pobox.com Wed May 22 20:47:53 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 22 May 2002 14:47:53 -0500 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: <200205221837.g4MIbJL10931@pcp742651pcs.reston01.va.comcast.net> References: <2moff8o6zp.fsf@starship.python.net> <200205221139.g4MBdHL09826@pcp742651pcs.reston01.va.comcast.net> <15595.40305.965276.823314@12-248-41-177.client.attbi.com> <200205221506.g4MF65T10184@pcp742651pcs.reston01.va.comcast.net> <15595.52769.255064.888318@12-248-41-177.client.attbi.com> <200205221720.g4MHKcs10667@pcp742651pcs.reston01.va.comcast.net> <15595.57204.624940.28865@12-248-41-177.client.attbi.com> <200205221837.g4MIbJL10931@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15595.62953.588961.245809@12-248-41-177.client.attbi.com> Guido> You mean the use of dispatch[BooleanType]? You can use Guido> dispatch[bool] just as well -- 'bool' is the same object as Guido> 'BooleanType'. So I'm still confused why you'd have to do Guido> anything. It's just for consistency. While bool == types.BooleanType, all the other dispatch[] keys are symbols taken from the types module. I actually don't really care. The only reason I even noticed its absence was that I went to patch xmlrpclib to use Python booleans instead of /F's Boolean class (when available). I think consistency counts for something here. I know xmlrpclib is one of that small number of modules that is actively maintained for older versions of Python. It makes sense to me to make as few changes to xmlrpclib to make that happen. If adding BooleanType to the types module helps, I think it's a good thing. Skip From skip@pobox.com Wed May 22 20:50:51 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 22 May 2002 14:50:51 -0500 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: References: <2moff8o6zp.fsf@starship.python.net> <200205221139.g4MBdHL09826@pcp742651pcs.reston01.va.comcast.net> <15595.40305.965276.823314@12-248-41-177.client.attbi.com> Message-ID: <15595.63131.828231.739313@12-248-41-177.client.attbi.com> Martin> I feel adding BooleanType was a bad thing. It is not needed; Martin> there is only one way to do it. Fine. Back the change out. I'm amazed at how much pushback I'm getting for such a trivial change. All I was trying to do was keep types consistent with the current set of types built into Python. Can we please argue about something more productive please? Skip From skip@pobox.com Wed May 22 20:51:49 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 22 May 2002 14:51:49 -0500 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: References: <2moff8o6zp.fsf@starship.python.net> <200205221139.g4MBdHL09826@pcp742651pcs.reston01.va.comcast.net> <15595.40305.965276.823314@12-248-41-177.client.attbi.com> <200205221506.g4MF65T10184@pcp742651pcs.reston01.va.comcast.net> <15595.52769.255064.888318@12-248-41-177.client.attbi.com> <200205221720.g4MHKcs10667@pcp742651pcs.reston01.va.comcast.net> <15595.57204.624940.28865@12-248-41-177.client.attbi.com> <200205221837.g4MIbJL10931@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15595.63189.422183.227598@12-248-41-177.client.attbi.com> Martin> I think Skip still assumes that bool is a function. Just for the record, no, I don't. What I assumed was that a symbol named "BooleanType" was missing from types. Skip From barry@zope.com Wed May 22 21:24:13 2002 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 22 May 2002 16:24:13 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 References: <2moff8o6zp.fsf@starship.python.net> <200205221139.g4MBdHL09826@pcp742651pcs.reston01.va.comcast.net> <15595.40305.965276.823314@12-248-41-177.client.attbi.com> <200205221506.g4MF65T10184@pcp742651pcs.reston01.va.comcast.net> <15595.52769.255064.888318@12-248-41-177.client.attbi.com> <200205221720.g4MHKcs10667@pcp742651pcs.reston01.va.comcast.net> <15595.57204.624940.28865@12-248-41-177.client.attbi.com> <200205221837.g4MIbJL10931@pcp742651pcs.reston01.va.comcast.net> <15595.63189.422183.227598@12-248-41-177.client.attbi.com> Message-ID: <15595.65133.821518.129971@anthem.wooz.org> >>>>> "SM" == Skip Montanaro writes: SM> Just for the record, no, I don't. What I assumed was that a SM> symbol named "BooleanType" was missing from types. FWIW, I'm +1 on adding BooleanType for consistency. this-statement-is-false-ly y'rs, -Barry From mats@laplaza.org Wed May 22 18:04:50 2002 From: mats@laplaza.org (Mats Wichmann) Date: Wed, 22 May 2002 11:04:50 -0600 Subject: [Python-Dev] Re: New bugtracker project In-Reply-To: <20020522133224.12354.96906.Mailman@mail.python.org> Message-ID: <5.1.0.14.1.20020522110148.00a9bc00@204.151.72.2> MySQL licensing: > 4) Recent versions only come with GPLed client libs, ie. the > application using them gets infected by the GPL. The whole > license issue is a general mess: see > http://www.mysql.com/downloads/mysql-max-4.0.html This isn't quite true, mysql-max includes some extra stuff so it itself got "infected", if you want to use that term. That's why it's packaged as a separate "product". The general statement is this: A license is not required if: =B7 You include the MySQL client code in a commercial program. The=20 client part of MySQL is licensed under the LGPL GNU Library General Public= =20 License. The mysql command-line client does include code from the readline= =20 library that is under the GPL, however. (see http://www.mysql.com/support/arrangements.html) From DavidA@ActiveState.com Wed May 22 22:06:24 2002 From: DavidA@ActiveState.com (David Ascher) Date: Wed, 22 May 2002 14:06:24 -0700 Subject: [Python-Dev] New bugtracker project References: <3CEADE5E.5090607@ActiveState.com>; from DavidA@ActiveState.com on Tue, May 21, 2002 at 04:55:10PM -0700 <3CEBB634.11681.1966903A@localhost> Message-ID: <3CEC0850.4020803@ActiveState.com> Gordon McMillan wrote: >On 22 May 2002 at 10:08, Trent Mick wrote: > > > >>I'll add another one to this: >> >>- After you submit changes to a bug in bugzilla via >>the Web interface you are told to whom an email >>update was sent and to whom it was NOT sent. This >>implies that the bug tracking system allows users >>to select under what conditions they would like to >>receive email (e.g. "I would like to receive an >>email for any changes if I own a bug, but only for >>status changes if I am the QA contact on a bug.) >> >> > >Which part of this are you deeming "useful"? >Seeing the list of to whom it was NOT sent[1]? >Or the fact that there seems to be a fancy >filtering / triggering mechanism behind it? > > > Both. The set of people mentioned (either as being sent an email or _not_ sent an email) is the set of people who are either: reporter assignee QA contact on the CC: list. Note that another feature we've added to bugzilla is a way for us to "force a reply to reporter", so that we can communicate with customers as part of the bug mgmt process, but they don't need to be sent every comment on a bug (the defaults for our bugzilla is that users only get comments when the bug is closed). Saying that there are scalability issues with this process is somewhat silly =) since Mozilla's bugzilla does this with >100,000 bugs and tens of thousands of registered bugzilla users, w/ no problems. --david From tismer@tismer.com Thu May 23 01:43:21 2002 From: tismer@tismer.com (Christian Tismer) Date: Wed, 22 May 2002 17:43:21 -0700 Subject: [Python-Dev] tstate->curexc_type etc Message-ID: <3CEC3B29.2060101@tismer.com> Hi friends, in new Stackless, I saved tstate->exc_type/value/traceback and tstate->curexc_type/value/traceback before switching to adifferent tasklet, and restored that one's. Under PythonWin, I saw crashes due to dangling references through that, which vanished when I increfed the exc_/... things. Problably they were multiply reset. But anyway, I even don't planto allow for multiple exceptions in parallel at all, so maybe I can drop this code completely. What I again and again do not understand, please help me out: Why is there this double set of variables, and which one should be preserved if at all between tasklet switches? many thanks - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From greg@cosc.canterbury.ac.nz Thu May 23 02:08:51 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Thu, 23 May 2002 13:08:51 +1200 (NZST) Subject: [Python-Dev] tstate->curexc_type etc In-Reply-To: <3CEC3B29.2060101@tismer.com> Message-ID: <200205230108.NAA05775@s454.cosc.canterbury.ac.nz> Christian Tismer : > in new Stackless, I saved > tstate->exc_type/value/traceback and > tstate->curexc_type/value/traceback > Why is there this double set of variables, I've just been wrestling with this stuff recently for Pyrex, and as far as I can tell, this is how it works: When an exception is set, it is put into curexc_{type,value,traceback} and unwinding of the stack begins, searching for a handler. When a handler is found, the exception is transferred to exc_{type,value,traceback} and execution of the handler begins. In other words, curexc_* holds an exception which has been set but not yet caught, and exc_* holds an exception which has been caught and is now being handled. > which one should be preserved if at all > between tasklet switches? I think you'll have to save exc_*, since a task switch could occur during execution of an except-block. You may get away without saving curexc_*, if it's impossible for a task switch to occur while in the middle of unwinding the stack looking for a matching except-clause. > Under PythonWin, I saw crashes due to dangling > references through that, which vanished when > I increfed the exc_/... things. Problably they > were multiply reset. I wouldn't fix that by putting in random increfs. You'd better find out exactly what's going on and make it do the right thing! Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tismer@tismer.com Thu May 23 02:51:51 2002 From: tismer@tismer.com (Christian Tismer) Date: Wed, 22 May 2002 18:51:51 -0700 Subject: [Python-Dev] tstate->curexc_type etc References: <200205230108.NAA05775@s454.cosc.canterbury.ac.nz> Message-ID: <3CEC4B37.30902@tismer.com> Greg Ewing wrote: [thanks for explaining!] >>Under PythonWin, I saw crashes due to dangling >>references through that, which vanished when >>I increfed the exc_/... things. Problably they >>were multiply reset. > > > I wouldn't fix that by putting in random increfs. > You'd better find out exactly what's going on and > make it do the right thing! Well, I followed your proposal. I didn't still completely get what's exactly going on, but I see that these are global variables. The increfs are not random, but I incref when I store them in a tasklet to be stopped and I decref when the tasklet is restarted and the vars restored. Seems to be ok. thanks - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From neal@metaslash.com Thu May 23 03:24:34 2002 From: neal@metaslash.com (Neal Norwitz) Date: Wed, 22 May 2002 22:24:34 -0400 Subject: [Python-Dev] Out of date FAQ entries Message-ID: <3CEC52E2.962B1CB6@metaslash.com> I was looking over the FAQ and noticed a out-of-date entries. I fixed a few: 6.6. Why can't I derive a class from built-in types (e.g. lists or files)? ** Added note that this doesn't apply to 2.2+ 6.21. How do you specify and enforce an interface spec in Python? ** Added note that pychecker can find these problems. But there are some others, which I can't do a good job updating. The one's I did update could use some work too. :-) 6.5. Why does Python use methods for some functionality (e.g. list.index()) but functions for other (e.g. len(list))? Perhaps update comment about strings, numbers not having methods. 6.12. Why is there no more efficient way of iterating over a dictionary than first constructing the list of keys()? for k in d: # now works 6.13. Can Python be compiled to machine code, C or some other language? Should mention pysco? 6.24. Why no class methods or mutable class variables? Mention classmethod(), staticmethod()? I'm sure there's others that could use fixing. Neal From rjones@ekit-inc.com Thu May 23 04:51:32 2002 From: rjones@ekit-inc.com (Richard Jones) Date: Thu, 23 May 2002 13:51:32 +1000 Subject: [Python-Dev] Re: New bugtracker project Message-ID: <200205231351.32303.rjones@ekit-inc.com> [note: I'm not subscribed to this list, just posting to let Anthony have a breather] In order to help people understand roundup's capabilities, I've written a features page, which is accessible at http://roundup.sf.net/doc/features.html ... it's not complete, since I've only just hacked it together. I'll flesh it out as I remember stuff that Roundup does :) Gordon McMillan wrote: > Hmm. From a quick look, it appears that the > abstraction tends to hide the roundup schema > from the DB schema. I suspect that what Andy > wants is for the DB schema to reflect the > roundup schema so he can query directly > on those "arbitrary" properties. That's correct. Roundup's hyperdb is a relational layer over anydbm-alikes. The intent was to make it available to people who don't/can't want/have an SQL database. Putting a real SQL database in the backend doesn't actually gain much - the journals will be stored much more intelligently, but for the most part, it'll be a lot of effort for little gain. Most of the table joining that you need to do is already done automatically by the hyperdb. Having db-level support for locking would be nice though (roundup implements its own locking). I'd hope that we can turn this on in the bsddb[3] backend Real Soon - but I'll need to do a bit of reading before I can do that ;) Michael Hudson wrote: > Ooh! I just stumbled into another one: serving up text attachments as > Content-Type: text/plain! I'm not the only person who gets pissed off > with having to save files to disk for no good reason, am I? Roundup gets this right - it uses the content-type specified in the attachment if submitted by email, or uses python's mime guessing if submitted through the web. It also serves the file up with the correct name. > 2) Not having this lunatic distinction between patches and bugs or > restrictions on who can add files to issues, and let people delete > at least their own attachments. Files may be attached to just about anything in roundup - the schema's completely flexible (so yes, you can attach files to people, people to issue states, messages to files, ...) You could also set your schema up so files come in different classes (roundup hyperdb classes that is) ... so files used in different contexts can have different attributes. Those attached to an issue (bug, feature, whatever) can have a state (pending, accepted, rejected, deleted) and those attached to people can have a type ('favourite car', 'mugshot', ...) and so on. Apologies in advance if I missed any outstanding questions, Richard From guido@python.org Thu May 23 05:01:01 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 23 May 2002 00:01:01 -0400 Subject: [Python-Dev] Re: New bugtracker project In-Reply-To: Your message of "Thu, 23 May 2002 13:51:32 +1000." <200205231351.32303.rjones@ekit-inc.com> References: <200205231351.32303.rjones@ekit-inc.com> Message-ID: <200205230401.g4N411J19949@pcp742651pcs.reston01.va.comcast.net> > > Ooh! I just stumbled into another one: serving up text attachments as > > Content-Type: text/plain! I'm not the only person who gets pissed off > > with having to save files to disk for no good reason, am I? > > Roundup gets this right - it uses the content-type specified in the > attachment if submitted by email, or uses python's mime guessing if > submitted through the web. I think SF's problem is that far too often it guesses that the type is not text/, and then most browsers won't show it. This is annoying for source code and diffs. If you could force the guess to be text/plain for .c, .h, .py, .diff, and .patch, that would be solved. --Guido van Rossum (home page: http://www.python.org/~guido/) From rjones@ekit-inc.com Thu May 23 05:13:42 2002 From: rjones@ekit-inc.com (Richard Jones) Date: Thu, 23 May 2002 14:13:42 +1000 Subject: [Python-Dev] Re: New bugtracker project In-Reply-To: <200205230401.g4N411J19949@pcp742651pcs.reston01.va.comcast.net> References: <200205231351.32303.rjones@ekit-inc.com> <200205230401.g4N411J19949@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200205231413.42145.rjones@ekit-inc.com> On Thu, 23 May 2002 14:01, Guido van Rossum wrote: > I think SF's problem is that far too often it guesses that the type is > not text/, and then most browsers won't show it. This is > annoying for source code and diffs. If you could force the guess to > be text/plain for .c, .h, .py, .diff, and .patch, that would be solved. If submitted via the web, Roundup is able to guess the MIME type and will serve up "source_file.py" as text/x-python, but "source_file" as application/octet-stream. If it's submitted by email, it'll keep the content-type it was submitted with. I guess a pass could be made to see if it's all printable characters before defaulting to a/o-s. An alternative is to have overrides for certain file extensions, and default to text/plain. Not pretty. Richard From smurf@noris.de Thu May 23 06:35:02 2002 From: smurf@noris.de (Matthias Urlichs) Date: Thu, 23 May 2002 07:35:02 +0200 Subject: [Python-Dev] New bugtracker project Message-ID: Guido van Rossum: >[David] > > For one thing, it was basically impossible for people with only a > > peripheral view of the database (managers, QA folks) to get accurate > > pictures of the database. Nothing that can't be fixed if one has a > > real DB backend. Currently it uses bsddb or anydbm. Backends for SQL or ZODB should certainly be writeable, the question is whether they'll be useful. Looking through the roundup code, a SQL backend looks like a good idea. The code for finding elements needs to be rewritten (currently it's not database specific and just scans everything -- that doesn't scale), but other than that it should be relatively straightforward. A ZODB backend should also be writebale, though I don't have enough experience with ZODB to say whether making all classes inherit from Persist and removing the marshalling code is enough. ;-) Both backends are limited to read-only access from outside Roundup, though. If not, triggers and journalling won't work. >Speaking of which, how real is RoundUp's DB backend? Does it use >MySQL? *Could* it use MySQL? *Should* it use MySQL? Umm, please s/MySQL/SQL/, as there are more databases out there -- besides, recommending MySQL is sure to provoke a flame war. (ACID may or may not work 100%, no views, no sub-selects, ...) -- Matthias Urlichs From aahz@pythoncraft.com Thu May 23 06:54:03 2002 From: aahz@pythoncraft.com (Aahz) Date: Thu, 23 May 2002 01:54:03 -0400 Subject: [Python-Dev] Re: New bugtracker project In-Reply-To: <200205231351.32303.rjones@ekit-inc.com> References: <200205231351.32303.rjones@ekit-inc.com> Message-ID: <20020523055402.GA856@panix.com> On Thu, May 23, 2002, Richard Jones wrote: > > That's correct. Roundup's hyperdb is a relational layer over > anydbm-alikes. The intent was to make it available to people who > don't/can't want/have an SQL database. Putting a real SQL database in > the backend doesn't actually gain much - the journals will be stored > much more intelligently, but for the most part, it'll be a lot of > effort for little gain. Most of the table joining that you need to do > is already done automatically by the hyperdb. So you're claiming that any reports I'd want to create already exist in Roundup? That's the main reason for wanting a SQL backend. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "In the end, outside of spy agencies, people are far too trusting and willing to help." --Ira Winkler From rjones@ekit-inc.com Thu May 23 07:15:30 2002 From: rjones@ekit-inc.com (Richard Jones) Date: Thu, 23 May 2002 16:15:30 +1000 Subject: [Python-Dev] Re: New bugtracker project In-Reply-To: <20020523055402.GA856@panix.com> References: <200205231351.32303.rjones@ekit-inc.com> <20020523055402.GA856@panix.com> Message-ID: <200205231615.30804.rjones@ekit-inc.com> On Thu, 23 May 2002 15:54, Aahz wrote: > On Thu, May 23, 2002, Richard Jones wrote: > > That's correct. Roundup's hyperdb is a relational layer over > > anydbm-alikes. The intent was to make it available to people who > > don't/can't want/have an SQL database. Putting a real SQL database in > > the backend doesn't actually gain much - the journals will be stored > > much more intelligently, but for the most part, it'll be a lot of > > effort for little gain. Most of the table joining that you need to do > > is already done automatically by the hyperdb. > > So you're claiming that any reports I'd want to create already exist in > Roundup? That's the main reason for wanting a SQL backend. Actually, the joining claim was a bit bold - it's been a while since I've used the admin tool, and forgot how it worked :) Having said that though, does your SQL database already come with all the reports you'd want to create? ;) You will need to write reports, yes. Since roundup doesn't have a query language, they will be more verbose than SQL. See the roundup-reminder example script: http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/roundup/roundup/scripts/roundup-reminder?rev=1.2&content-type=text/vnd.viewcvs-markup [sorry about the apallingly long url] To clarify on the whole SQL deal (I hope): without a fair bit of effort customising a backend for layering over an SQL database, SQL queries on an SQL backend are going to look fairly hideous. One of the biggies is the Multilink property available in the hyperdb, which would need a separate table if implemented half-decently in SQL. If you want to do lots of SQL, you'll probably want to either invest the time in developing a hyperdb-over-SQL, or you're going to want to look elsewhere. Richard From Anthony Baxter Thu May 23 07:32:02 2002 From: Anthony Baxter (Anthony Baxter) Date: Thu, 23 May 2002 16:32:02 +1000 Subject: [Python-Dev] Re: New bugtracker project In-Reply-To: Message from Aahz of "Thu, 23 May 2002 01:54:03 -0400." <20020523055402.GA856@panix.com> Message-ID: <200205230632.g4N6W2l21582@localhost.localdomain> >>> Aahz wrote > So you're claiming that any reports I'd want to create already exist in > Roundup? That's the main reason for wanting a SQL backend. Is the requirement that we can do arbitrary[*] reports, or that we can do arbitrary reports in SQL? Anthony [*] arbitrary, of course, meaning "arbitrary within reasonable limits" From martin@v.loewis.de Thu May 23 07:35:34 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 23 May 2002 08:35:34 +0200 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: <15595.63131.828231.739313@12-248-41-177.client.attbi.com> References: <2moff8o6zp.fsf@starship.python.net> <200205221139.g4MBdHL09826@pcp742651pcs.reston01.va.comcast.net> <15595.40305.965276.823314@12-248-41-177.client.attbi.com> <15595.63131.828231.739313@12-248-41-177.client.attbi.com> Message-ID: Skip Montanaro writes: > Fine. Back the change out. I'm amazed at how much pushback I'm getting for > such a trivial change. You shouldn't :-) People always like to argue about syntax and naming. > Can we please argue about something more productive please? Sure. I was just surprised that, even though this specific change had been rejected before atleast three times, it suddenly went in. Regards, Martin From martin@v.loewis.de Thu May 23 07:31:49 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 23 May 2002 08:31:49 +0200 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: <15595.62953.588961.245809@12-248-41-177.client.attbi.com> References: <2moff8o6zp.fsf@starship.python.net> <200205221139.g4MBdHL09826@pcp742651pcs.reston01.va.comcast.net> <15595.40305.965276.823314@12-248-41-177.client.attbi.com> <200205221506.g4MF65T10184@pcp742651pcs.reston01.va.comcast.net> <15595.52769.255064.888318@12-248-41-177.client.attbi.com> <200205221720.g4MHKcs10667@pcp742651pcs.reston01.va.comcast.net> <15595.57204.624940.28865@12-248-41-177.client.attbi.com> <200205221837.g4MIbJL10931@pcp742651pcs.reston01.va.comcast.net> <15595.62953.588961.245809@12-248-41-177.client.attbi.com> Message-ID: Skip Montanaro writes: > I actually don't really care. The only reason I even noticed its absence > was that I went to patch xmlrpclib to use Python booleans instead of /F's > Boolean class (when available). I think consistency counts for something > here. I know xmlrpclib is one of that small number of modules that is > actively maintained for older versions of Python. It makes sense to me to > make as few changes to xmlrpclib to make that happen. If adding BooleanType > to the types module helps, I think it's a good thing. Since that needs to be conditional (with catching ImportError and all), I think you can achieve internal consistency and minimal changes with introducing a name BooleanType into xmlrpclib. BTW, why is it "BooleanType" and not "BoolType"? It's also IntType and not IntegerType. Regards, Martin From skip@pobox.com Thu May 23 07:59:44 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 23 May 2002 01:59:44 -0500 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: References: <2moff8o6zp.fsf@starship.python.net> <200205221139.g4MBdHL09826@pcp742651pcs.reston01.va.comcast.net> <15595.40305.965276.823314@12-248-41-177.client.attbi.com> <15595.63131.828231.739313@12-248-41-177.client.attbi.com> Message-ID: <15596.37728.628389.593304@12-248-41-177.client.attbi.com> Martin> Sure. I was just surprised that, even though this specific Martin> change had been rejected before atleast three times, it suddenly Martin> went in. I must of missed them. It seemed like a trivial change to me. Just for the heck of it, I just checked the libref docs for the types module. (I freely admit I didn't consult the docs before making the change.) It begins: This module defines names for all object types that are used by the standard Python interpreter,... Given that types is not yet deprecated, it makes sense to me that the above statement should remain true. It sounds like you are headed in another direction for testing the typeness of objects that relies on isinstance(). If that's the case, I suggest you deprecate the types module and change its documentation to reflect the new preferred way of doing business. Skip From skip@pobox.com Thu May 23 08:05:50 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 23 May 2002 02:05:50 -0500 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: References: <2moff8o6zp.fsf@starship.python.net> <200205221139.g4MBdHL09826@pcp742651pcs.reston01.va.comcast.net> <15595.40305.965276.823314@12-248-41-177.client.attbi.com> <200205221506.g4MF65T10184@pcp742651pcs.reston01.va.comcast.net> <15595.52769.255064.888318@12-248-41-177.client.attbi.com> <200205221720.g4MHKcs10667@pcp742651pcs.reston01.va.comcast.net> <15595.57204.624940.28865@12-248-41-177.client.attbi.com> <200205221837.g4MIbJL10931@pcp742651pcs.reston01.va.comcast.net> <15595.62953.588961.245809@12-248-41-177.client.attbi.com> Message-ID: <15596.38094.167530.39165@12-248-41-177.client.attbi.com> Martin> Since that needs to be conditional (with catching ImportError Martin> and all), I think you can achieve internal consistency and Martin> minimal changes with introducing a name BooleanType into Martin> xmlrpclib. That's fine. It's /F's call how he wants to do it. I assigned that patch to him. Martin> BTW, why is it "BooleanType" and not "BoolType"? It's also Martin> IntType and not IntegerType. I like to read compound words where the elements are themselves words. StringType doesn't match "str". Either way is fine with me. I suppose if I really had my druthers I'd have preferred the function was called "boole()" and the object in the types module "BooleType". After all, that's how George's name was spelled... ;-) Skip From mal@lemburg.com Thu May 23 09:22:51 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 23 May 2002 10:22:51 +0200 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 References: <2moff8o6zp.fsf@starship.python.net> <200205221139.g4MBdHL09826@pcp742651pcs.reston01.va.comcast.net> <15595.40305.965276.823314@12-248-41-177.client.attbi.com> <200205221506.g4MF65T10184@pcp742651pcs.reston01.va.comcast.net> <15595.52769.255064.888318@12-248-41-177.client.attbi.com> <200205221720.g4MHKcs10667@pcp742651pcs.reston01.va.comcast.net> <15595.57204.624940.28865@12-248-41-177.client.attbi.com> <200205221837.g4MIbJL10931@pcp742651pcs.reston01.va.comcast.net> <15595.62953.588961.245809@12-248-41-177.client.attbi.com> <15596.38094.167530.39165@12-248-41-177.client.attbi.com> Message-ID: <3CECA6DB.5040208@lemburg.com> Skip Montanaro wrote: > Martin> Since that needs to be conditional (with catching ImportError > Martin> and all), I think you can achieve internal consistency and > Martin> minimal changes with introducing a name BooleanType into > Martin> xmlrpclib. > > That's fine. It's /F's call how he wants to do it. I assigned that patch > to him. Replacing Boolean in xmlrpclib with the new type should be done with care, though, since esp. on the encoding side things may go wrong due to the fact that boolean values are already being returned by comparisons and other builtin functions. You may end up encoding booleans where you really wanted to encode integers. > Martin> BTW, why is it "BooleanType" and not "BoolType"? It's also > Martin> IntType and not IntegerType. > > I like to read compound words where the elements are themselves words. > StringType doesn't match "str". Either way is fine with me. > > I suppose if I really had my druthers I'd have preferred the function was > called "boole()" and the object in the types module "BooleType". After all, > that's how George's name was spelled... ;-) Good catch :-) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From jafo@tummy.com Thu May 23 09:36:28 2002 From: jafo@tummy.com (Sean Reifschneider) Date: Thu, 23 May 2002 02:36:28 -0600 Subject: [Python-Dev] New bugtracker project In-Reply-To: <2mr8k4o878.fsf@starship.python.net>; from mwh@python.net on Wed, May 22, 2002 at 10:14:35AM +0100 References: <200205211638.g4LGcVw19247@odiug.zope.com> <2mr8k4o878.fsf@starship.python.net> Message-ID: <20020523023628.D24384@tummy.com> On Wed, May 22, 2002 at 10:14:35AM +0100, Michael Hudson wrote: >1) An email interface (OK, so this has been said, and already exists, > but...) This is one of the things that really impressed me about the Debian bugs system. While it includes a web interface as well, it's primary mechanism of control is via e-mail. I haven't actually used it, but our LUG got a presentation on it a couple of months ago and I was very impressed. A few things I seem to recall: Submitting a bug/patch can be as simple as sending a plain text message without any special markup. Special name:value information can be provided if known (releases it's related to, etc)... All attributes can be changed, bugs/patches can be merged or split. Mailing "12345-submitter" (or the like) will send e-mail to the person who submitted but 12345. There are other aliases like that as well (mail people assigned to work on it, the automated system, etc). The Debian bug system is available at: http://www.debian.org/Bugs/ While I'm (not suprisingly) interested in roundup because it's in Python, I've been impressed with the demo of the Debian system. Sean -- "The big bad wolf, he learned the rule. You gotta get hot to play real cool." Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From Oleg Broytmann Thu May 23 09:41:10 2002 From: Oleg Broytmann (Oleg Broytmann) Date: Thu, 23 May 2002 12:41:10 +0400 Subject: [Python-Dev] Out of date FAQ entries In-Reply-To: <3CEC52E2.962B1CB6@metaslash.com>; from neal@metaslash.com on Wed, May 22, 2002 at 10:24:34PM -0400 References: <3CEC52E2.962B1CB6@metaslash.com> Message-ID: <20020523124110.C23951@phd.pp.ru> Hi! Thank you for the work! On Wed, May 22, 2002 at 10:24:34PM -0400, Neal Norwitz wrote: > 6.13. Can Python be compiled to machine code, C or some other language? > Should mention pysco? Psyco, Pyrex, PyInline, Py2Cmod. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From mwh@python.net Thu May 23 10:38:30 2002 From: mwh@python.net (Michael Hudson) Date: 23 May 2002 10:38:30 +0100 Subject: [Python-Dev] New bugtracker project In-Reply-To: martin@v.loewis.de's message of "22 May 2002 20:47:47 +0200" References: <2mr8k4o878.fsf@starship.python.net> <3CEB8926.16289.18B692CF@localhost> <20020522161655.GB4869@panix.com> Message-ID: <2m8z6bi4q1.fsf@starship.python.net> martin@v.loewis.de (Martin v. Loewis) writes: > Aahz writes: > > > He's talking about SourceForge. If a bug has a patch, it should be > > directly part of the bug, not a separate PatchTracker submission. > > And indeed, on SourceForge, it can be directly part of the bug report. But only, I think you'll find, if you are the submitter of the bug or a developer (w/ appropriate tracker rights) on the project. CHeers, M. -- There's an aura of unholy black magic about CLISP. It works, but I have no idea how it does it. I suspect there's a goat involved somewhere. -- Johann Hibschman, comp.lang.scheme From mwh@python.net Thu May 23 10:41:50 2002 From: mwh@python.net (Michael Hudson) Date: 23 May 2002 10:41:50 +0100 Subject: [Python-Dev] New bugtracker project In-Reply-To: martin@v.loewis.de's message of "22 May 2002 20:52:33 +0200" References: <200205211638.g4LGcVw19247@odiug.zope.com> <2madqsdu7m.fsf@starship.python.net> Message-ID: <2m661fi4kh.fsf@starship.python.net> martin@v.loewis.de (Martin v. Loewis) writes: > Michael Hudson writes: > > > Ooh! I just stumbled into another one: serving up text attachments as > > Content-Type: text/plain! I'm not the only person who gets pissed off > > with having to save files to disk for no good reason, am I? > > I think SF works correctly, here: it records the MIME type of each > attachment if one was provided during upload (or somehow guesses if > there was not), then hands out the patch with the same MIME type. So my browser needs to be taught .diff means text... > I'm more annoyed about the Mozilla bug that it insists to call the > file download.php, even though SF properly reports the original file > name. This one seems to have been fixed at last in 1.0rc2. Cheers, M. -- It *is*, however, rather juvenile, in contravention of the Trades Descriptions Act, and will lead eventually to the Dragon of Unhappiness flying up your bottom. -- Peter Ellis, ucam.chat From sholden@holdenweb.com Wed May 22 15:53:06 2002 From: sholden@holdenweb.com (Steve Holden) Date: Wed, 22 May 2002 10:53:06 -0400 Subject: [Python-Dev] New bugtracker project References: Message-ID: <001301c20244$a87421d0$6300000a@holdenweb.com> If you're looking to save a further amount of work, you might consider something that automatically cateogrizes my occasional submissions as "not a bug" ;-) regards ----------------------------------------------------------------------- Steve Holden http://www.holdenweb.com/ Python Web Programming http://pydish.holdenweb.com/pwp/ ----------------------------------------------------------------------- ----- Original Message ----- From: "Tim Peters" To: Sent: Tuesday, May 21, 2002 10:45 PM Subject: RE: [Python-Dev] New bugtracker project > If Gordon is working on this, I want to make it a requirement that the > system not just track bugs, but fix them too. The man needs a challenge. > > and-we-need-the-help-ly y'rs - tim > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > From marklists@mceahern.com Thu May 23 13:15:20 2002 From: marklists@mceahern.com (Mark McEahern) Date: Thu, 23 May 2002 07:15:20 -0500 Subject: [Python-Dev] Re: New bugtracker project In-Reply-To: <200205231351.32303.rjones@ekit-inc.com> Message-ID: [Richard Jones] > http://roundup.sf.net/doc/features.html This page renders oddly in IE 6.0 on Windows XP. Looks fine in Konqueror 2.2.1 on Red Hat Linux 7.2. A simple fix would seem to be removing the (unnecessary?)

tags enclosing each

  • element's contents. Cheers, // mark From guido@python.org Thu May 23 14:08:42 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 23 May 2002 09:08:42 -0400 Subject: [Python-Dev] Re: New bugtracker project In-Reply-To: Your message of "Thu, 23 May 2002 16:32:02 +1000." <200205230632.g4N6W2l21582@localhost.localdomain> References: <200205230632.g4N6W2l21582@localhost.localdomain> Message-ID: <200205231308.g4ND8gH21965@pcp742651pcs.reston01.va.comcast.net> > Is the requirement that we can do arbitrary[*] reports, or that we can > do arbitrary reports in SQL? Personally, I don't care about the SQL bit (since I would have to learn it :-). I liked the code of the reminder script for which Richard posted the URL. just fine. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu May 23 14:10:50 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 23 May 2002 09:10:50 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: Your message of "Thu, 23 May 2002 01:59:44 CDT." <15596.37728.628389.593304@12-248-41-177.client.attbi.com> References: <2moff8o6zp.fsf@starship.python.net> <200205221139.g4MBdHL09826@pcp742651pcs.reston01.va.comcast.net> <15595.40305.965276.823314@12-248-41-177.client.attbi.com> <15595.63131.828231.739313@12-248-41-177.client.attbi.com> <15596.37728.628389.593304@12-248-41-177.client.attbi.com> Message-ID: <200205231310.g4NDAol21983@pcp742651pcs.reston01.va.comcast.net> > If that's the case, I suggest you deprecate the types module and change its > documentation to reflect the new preferred way of doing business. Quit harping about the fact that types isn't deprecated already! We haven't even done a release yet. --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz@pythoncraft.com Thu May 23 14:17:03 2002 From: aahz@pythoncraft.com (Aahz) Date: Thu, 23 May 2002 09:17:03 -0400 Subject: [Python-Dev] Out of date FAQ entries In-Reply-To: <20020523124110.C23951@phd.pp.ru> References: <3CEC52E2.962B1CB6@metaslash.com> <20020523124110.C23951@phd.pp.ru> Message-ID: <20020523131702.GA21548@panix.com> On Thu, May 23, 2002, Oleg Broytmann wrote: > On Wed, May 22, 2002 at 10:24:34PM -0400, Neal Norwitz wrote: >> >> 6.13. Can Python be compiled to machine code, C or some other language? >> Should mention pysco? > > Psyco, Pyrex, PyInline, Py2Cmod. Weave. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "In the end, outside of spy agencies, people are far too trusting and willing to help." --Ira Winkler From skip@pobox.com Thu May 23 14:18:12 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 23 May 2002 08:18:12 -0500 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: <200205231310.g4NDAol21983@pcp742651pcs.reston01.va.comcast.net> References: <2moff8o6zp.fsf@starship.python.net> <200205221139.g4MBdHL09826@pcp742651pcs.reston01.va.comcast.net> <15595.40305.965276.823314@12-248-41-177.client.attbi.com> <15595.63131.828231.739313@12-248-41-177.client.attbi.com> <15596.37728.628389.593304@12-248-41-177.client.attbi.com> <200205231310.g4NDAol21983@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15596.60436.897694.940315@12-248-41-177.client.attbi.com> >> If that's the case, I suggest you deprecate the types module and >> change its documentation to reflect the new preferred way of doing >> business. Guido> Quit harping about the fact that types isn't deprecated already! Guido> We haven't even done a release yet. I'm not. You were the one who brought up maybe deprecating it. If it's not going to be deprecated I see no reason it should not provide names for all the builtin types. Skip From guido@python.org Thu May 23 14:28:51 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 23 May 2002 09:28:51 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: Your message of "Thu, 23 May 2002 08:18:12 CDT." <15596.60436.897694.940315@12-248-41-177.client.attbi.com> References: <2moff8o6zp.fsf@starship.python.net> <200205221139.g4MBdHL09826@pcp742651pcs.reston01.va.comcast.net> <15595.40305.965276.823314@12-248-41-177.client.attbi.com> <15595.63131.828231.739313@12-248-41-177.client.attbi.com> <15596.37728.628389.593304@12-248-41-177.client.attbi.com> <200205231310.g4NDAol21983@pcp742651pcs.reston01.va.comcast.net> <15596.60436.897694.940315@12-248-41-177.client.attbi.com> Message-ID: <200205231328.g4NDSpG22143@pcp742651pcs.reston01.va.comcast.net> > Guido> Quit harping about the fact that types isn't deprecated already! > Guido> We haven't even done a release yet. [Skip] > I'm not. You were the one who brought up maybe deprecating it. If > it's not going to be deprecated I see no reason it should not > provide names for all the builtin types. I think everybody else sort of expects that it will be phased out -- a gentle form of deprecation -- because it's mostly redundant and having two official ways to refer to exactly the same object is somewhat unpythonic. We'll have to figure out what to do about names that are defined in types.py but are not globals; are there any of those that are actually used? I know of one useful one: StringTypes. I *think* I'd like to see that become a common base class for str and unicode. This may require a PEP. But in the mean time, while types.py is in limbo, let's not add to it. --Guido van Rossum (home page: http://www.python.org/~guido/) From jacobs@penguin.theopalgroup.com Thu May 23 14:44:04 2002 From: jacobs@penguin.theopalgroup.com (Kevin Jacobs) Date: Thu, 23 May 2002 09:44:04 -0400 (EDT) Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: <15596.60436.897694.940315@12-248-41-177.client.attbi.com> Message-ID: On Thu, 23 May 2002, Skip Montanaro wrote: > I'm not. You were the one who brought up maybe deprecating it. If it's > not going to be deprecated I see no reason it should not provide names for > all the builtin types. Pointless semantic arguments aside, I agree with Skip. I don't care how many other ways we provide to spell types: until the types module is depricated, I do not see why it should be intentionally broken by not covering all builtin types (unless thus breaking the module is a slimy way of encouraging its deprication, in which case I will object on procedural grounds). The current state of afairs is clearly un-Pythonic -- there IS more than one way to do many things. However, the types module is by far one of the more benign places where this is happening. e.g., concurrently having two distinct class implementations isn't exactly Pythonic either. So everyone should realize that the 2.2 release have left a great number of things in flux, especially in the type system. Once the dust settles, we need to take a hard look at where we are, where we need to go, and how we plan to get there. These steps MUST include wisdom from both theoretical AND empirical study of the problems, or else we risk losing the essential characteristics that made us want to write our programs in Python in the first place. Please, lets keep our urges for microscopic language-lawyering in check for now. Everyone is making very clever arguments, and we're all really impressed, but this argument has turned into a pointless wank-fest. There are bigger conceptual fish to fry and real bugs to fix. So leave the silly BooleanType declaration in types -- it hurts nobody and keeps our house in good order until we decide if/when we want to tear that part down. Best wishes and intravenous valium, -Kevin -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com From aahz@pythoncraft.com Thu May 23 14:50:16 2002 From: aahz@pythoncraft.com (Aahz) Date: Thu, 23 May 2002 09:50:16 -0400 Subject: [Python-Dev] Re: New bugtracker project In-Reply-To: <200205230632.g4N6W2l21582@localhost.localdomain> References: <20020523055402.GA856@panix.com> <200205230632.g4N6W2l21582@localhost.localdomain> Message-ID: <20020523135016.GA26560@panix.com> On Thu, May 23, 2002, Anthony Baxter wrote: >> Aahz wrote >> So you're claiming that any reports I'd want to create already exist in >> Roundup? That's the main reason for wanting a SQL backend. > > Is the requirement that we can do arbitrary[*] reports, or that we can > do arbitrary reports in SQL? Unlike Guido, I do already know SQL, so my preference is for SQL, but if there's a clear/simple API for getting reports that has all the power of SQL, that's fine with me. Note specifically that I'm not a fan of mixing and matching languages for data manipulation. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "In the end, outside of spy agencies, people are far too trusting and willing to help." --Ira Winkler From skip@pobox.com Thu May 23 14:58:00 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 23 May 2002 08:58:00 -0500 Subject: [Python-Dev] the nature of the symbols in types Message-ID: <15596.62824.489800.189706@12-248-41-177.client.attbi.com> There seem to be two classes of symbols in the types module, those with a one-to-one mapping to objects in builtins (IntType, StringType, etc) and those you have to work a bit more diligently to get at from Python code (TracebackType, UnboundMethodType, etc). It's clear the uses of the common types can be replaced, perhaps pretty mechanically. They also probably account for 99.5% of all uses of the module. Furthermore, I would hazard a guess that the tougher-to-get-at types are probably used predominantly in introspective (and thus specialized) code. A quick scan through the Python source bears this out: Bastion.py, bdb.py, calendar.py, copy.py, copy_reg.py, dis.py, imputil.py, inspect.py, pickle.py, posixfile.py, pydoc.py, traceback.py, unittest.py, warnings.py, distutils/dist.py, distutils/unixccompiler.py, cgitb.py, xmlrpclib.py The above files contain references to one or more of the following names: BufferType, BuiltinFunctionType, BuiltinMethodType, ClassType, CodeType, DictProxyType, EllipsisType, FrameType, FunctionType, GeneratorType, InstanceType, LambdaType, MethodType, ModuleType, NoneType, SliceType, TracebackType, UnboundMethodType, XRangeType I suspect most of these uses could be replaced fairly easily by the authors of code that uses them, though it wouldn't be quite as automatic as for the common symbols. Some could probably just be deleted. anything that can just be gotten by calling type() might be a candidate. For some, it should be fairly straightforward to add appropriate symbols to builtins. 'buffer', 'slice' and 'xrange' could be replaced by a callable type. Others could be just non-callable objects that expose the various type objects. Adding the obvious symbol for 'class' to builtins obviously wouldn't work. Skip From mal@lemburg.com Thu May 23 14:58:00 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 23 May 2002 15:58:00 +0200 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 References: Message-ID: <3CECF568.50202@lemburg.com> Kevin Jacobs wrote: > On Thu, 23 May 2002, Skip Montanaro wrote: > >>I'm not. You were the one who brought up maybe deprecating it. If it's >>not going to be deprecated I see no reason it should not provide names for >>all the builtin types. > > > Pointless semantic arguments aside, I agree with Skip. I don't care how > many other ways we provide to spell types: until the types module is > depricated, I do not see why it should be intentionally broken by not > covering all builtin types (unless thus breaking the module is a slimy way > of encouraging its deprication, in which case I will object on procedural > grounds). > > The current state of afairs is clearly un-Pythonic -- there IS more than one > way to do many things. However, the types module is by far one of the more > benign places where this is happening. e.g., concurrently having two > distinct class implementations isn't exactly Pythonic either. So everyone > should realize that the 2.2 release have left a great number of things in > flux, especially in the type system. Once the dust settles, we need to take > a hard look at where we are, where we need to go, and how we plan to get > there. These steps MUST include wisdom from both theoretical AND empirical > study of the problems, or else we risk losing the essential characteristics > that made us want to write our programs in Python in the first place. > > Please, lets keep our urges for microscopic language-lawyering in check for > now. Everyone is making very clever arguments, and we're all really > impressed, but this argument has turned into a pointless wank-fest. There > are bigger conceptual fish to fry and real bugs to fix. So leave the silly > BooleanType declaration in types -- it hurts nobody and keeps our house in > good order until we decide if/when we want to tear that part down. +1 -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From Anthony Baxter Thu May 23 15:10:32 2002 From: Anthony Baxter (Anthony Baxter) Date: Fri, 24 May 2002 00:10:32 +1000 Subject: [Python-Dev] Re: New bugtracker project In-Reply-To: Message from Aahz of "Thu, 23 May 2002 09:50:16 -0400." <20020523135016.GA26560@panix.com> Message-ID: <200205231410.g4NEAW925454@localhost.localdomain> >>> Aahz wrote > Unlike Guido, I do already know SQL, so my preference is for SQL, but if > there's a clear/simple API for getting reports that has all the power of > SQL, that's fine with me. Note specifically that I'm not a fan of > mixing and matching languages for data manipulation. I do a _lot_ of SQL (ask Richard, he gets to hear me swear about it at work), but I'm not hung up on it. I don't know that it's a showstopper. Note that initially, at least, it would be far easier to work with and modify a hyperdb-based schema til it's "right", then, if performance/ reporting/whatever needed it, move to a relational backend. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From pyth@devel.trillke.net Thu May 23 15:22:38 2002 From: pyth@devel.trillke.net (holger krekel) Date: Thu, 23 May 2002 16:22:38 +0200 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: ; from jacobs@penguin.theopalgroup.com on Thu, May 23, 2002 at 09:44:04AM -0400 References: <15596.60436.897694.940315@12-248-41-177.client.attbi.com> Message-ID: <20020523162238.C11253@prim.han.de> Kevin Jacobs wrote: > On Thu, 23 May 2002, Skip Montanaro wrote: > > I'm not. You were the one who brought up maybe deprecating it. If it's > > not going to be deprecated I see no reason it should not provide names for > > all the builtin types. > > Pointless semantic arguments aside, I agree with Skip. I don't care how > many other ways we provide to spell types: until the types module is > depricated, I do not see why it should be intentionally broken by not > covering all builtin types (unless thus breaking the module is a slimy way > of encouraging its deprication, in which case I will object on procedural > grounds). +1 holger From guido@python.org Thu May 23 15:18:30 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 23 May 2002 10:18:30 -0400 Subject: [Python-Dev] the nature of the symbols in types In-Reply-To: Your message of "Thu, 23 May 2002 08:58:00 CDT." <15596.62824.489800.189706@12-248-41-177.client.attbi.com> References: <15596.62824.489800.189706@12-248-41-177.client.attbi.com> Message-ID: <200205231418.g4NEIUC31549@odiug.zope.com> > I suspect most of these uses could be replaced fairly easily by the > authors of code that uses them, though it wouldn't be quite as > automatic as for the common symbols. Some could probably just be > deleted. anything that can just be gotten by calling type() might > be a candidate. For some, it should be fairly straightforward to > add appropriate symbols to builtins. 'buffer', 'slice' and 'xrange' > could be replaced by a callable type. Others could be just > non-callable objects that expose the various type objects. Adding > the obvious symbol for 'class' to builtins obviously wouldn't work. Now these are things for which I'd like to see patches! (And the 'string' common basetype.) --Guido van Rossum (home page: http://www.python.org/~guido/) From paul@pfdubois.com Thu May 23 16:48:18 2002 From: paul@pfdubois.com (Paul F Dubois) Date: Thu, 23 May 2002 08:48:18 -0700 Subject: [Python-Dev] Solaris difficulty Message-ID: <000001c20271$43a93d80$0a01a8c0@NICKLEBY> Is there another member of this list who uses Solaris? If so, could you contact me privately? We have intermittently succeeded in building Python on Solaris 5.6, but it is more off than on. We haven't succeeded on Solaris 8. We set CC="cc -mt" but we die building _socket. We do have /usr/local/ssl. Having tried to search the bug tracker lately due to this problem, I endorse any attempt to do better. From Oleg Broytmann Thu May 23 17:07:34 2002 From: Oleg Broytmann (Oleg Broytmann) Date: Thu, 23 May 2002 20:07:34 +0400 Subject: [Python-Dev] Solaris difficulty In-Reply-To: <000001c20271$43a93d80$0a01a8c0@NICKLEBY>; from paul@pfdubois.com on Thu, May 23, 2002 at 08:48:18AM -0700 References: <000001c20271$43a93d80$0a01a8c0@NICKLEBY> Message-ID: <20020523200734.D29201@phd.pp.ru> On Thu, May 23, 2002 at 08:48:18AM -0700, Paul F Dubois wrote: > Is there another member of this list who uses Solaris? If so, could you > contact me privately? Contacted... Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From amckay@merlintechnologies.com Thu May 23 17:40:15 2002 From: amckay@merlintechnologies.com (Andy McKay) Date: Thu, 23 May 2002 09:40:15 -0700 Subject: [Python-Dev] New bugtracker project In-Reply-To: <3CEB4011.4060304@ActiveState.com> References: <200205220059.g4M0x2S07210@localhost.localdomain> <3CEB4011.4060304@ActiveState.com> Message-ID: <200205231635.g4NGYup07275@mail.merlinsoftech.com> > >>2) an 'efficient' UI. The bugzilla default search results pages are > >>very inefficient from a UI point of view. Andy McKay's version is much > >>better, I think. > > > >I don't see the difference in the two, aside from a smaller font? The UI > >for roundup's pretty much completely flexible. > > My only point is that flexibility is fine, but getting the details right > is what matters in the end. Bugzilla's UI is flexible (it's all Perl, > after all), but the defaults suck. Bugzilla 2.16 now features (at last) seperately templated HTML, I haven't looked into it in any detail though. Most of the changes I made to the Bugzilla interface for ActiveState were cleaning up: making things clearer and more useable, in a few cases that meant removing little bits of functionality. Many of the other changes are internal to ActiveState: email integration (similar to Roundup), bug privacy, workflow changes. Apart from the code which in some places in terrible (and classic argument against Perl), my only real complaint about Bugzilla is the UI. > >Ok. The way we have roundup configured is that a message is sent to > >a set of users when an issue is opened, and then to users that are > >"interested" subsequently - they've either commented on the bug, or > >added themselves to the nosy list. What did you find meant too much > >email? The "initial email out" was something we added. > > Yeah, we added the same thing. As I said, I forget the details of the > failure. There were many, one was there was no real list of users (see point below), so you had no idea who should be getting email and it was easy to enter addresses incorrectly. The idea of roles on a bug (owner, reporter, QA etc) in Bugzilla and configuring how many emails you get is a good thing. My other major problem with the old Roundup (not the new one) and Jitterbug was a lack of clear data definitions. Since Bugzilla has a RDB back end, it has a well defined set of data, Roundup just stored stuff in text files and there was little data integrity. I remember there being about 10 different spellings of one category... Whatever you guys decide I would recommend caution before starting to move bug databases. Maintaining the integrity of the data on old bugs as you move from the SF bug tracker to another can be a pain. If you find you dont like that new bug tracker and move to another... well that wont be fun. been-through-the-jitterbug-roundup-bugzilla-pains'ly yours -- Andy McKay From martin@v.loewis.de Thu May 23 17:35:55 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 23 May 2002 18:35:55 +0200 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: References: Message-ID: Kevin Jacobs writes: > Pointless semantic arguments aside, I agree with Skip. I don't care how > many other ways we provide to spell types: until the types module is > depricated, I do not see why it should be intentionally broken by not > covering all builtin types (unless thus breaking the module is a slimy way > of encouraging its deprication, in which case I will object on procedural > grounds). Nobody suggested to break the types module; not adding BooleanType would not break it. Regards, Martin From jacobs@penguin.theopalgroup.com Thu May 23 18:14:36 2002 From: jacobs@penguin.theopalgroup.com (Kevin Jacobs) Date: Thu, 23 May 2002 13:14:36 -0400 (EDT) Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: Message-ID: On 23 May 2002, Martin v. Loewis wrote: > Kevin Jacobs writes: > > Pointless semantic arguments aside, I agree with Skip. I don't care how > > many other ways we provide to spell types: until the types module is > > depricated, I do not see why it should be intentionally broken by not > > covering all builtin types (unless thus breaking the module is a slimy way > > of encouraging its deprication, in which case I will object on procedural > > grounds). > > Nobody suggested to break the types module; not adding BooleanType > would not break it. http://www.python.org/doc/current/lib/module-types.html: 3.6 types -- Names for ALL built-in types This module defines names for ALL object types that are used by the standard Python interpreter, but not for the types defined by various extension modules. How much clearer does it need to be? Further, the expectation of real-live Python users is that it will contain the boolean type. This is not an academic argument for me, and likely for many others. We have large code bases that use the types module, and expect the module to rigorously track new types until something better is fully implemented, stable, and the old module is properly depricated. I still can't believe we have nothing better to do than argue about this spectacularly piddly issue. If you don't like Skip's patch, then stick your fingers in your ears and humm real loud -- just pretend it isn't there -- it costs absolutely nothing to those who won't be using it. Ate two bowls of cranky flakes this morning, -Kevin -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com From guido@python.org Thu May 23 18:16:15 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 23 May 2002 13:16:15 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: Your message of "Thu, 23 May 2002 13:14:36 EDT." References: Message-ID: <200205231716.g4NHGFN32731@odiug.zope.com> > Further, the expectation of real-live Python users is that it will > contain the boolean type. This is not an academic argument for me, > and likely for many others. We have large code bases that use the > types module, and expect the module to rigorously track new types > until something better is fully implemented, stable, and the old > module is properly depricated. Why don't you wait and see what shows up in Python 2.3 when it is released. It's good that we're arguing about this now -- we should offer something to replace all features of the the types module in 2.3. --Guido van Rossum (home page: http://www.python.org/~guido/) From jacobs@penguin.theopalgroup.com Thu May 23 19:05:46 2002 From: jacobs@penguin.theopalgroup.com (Kevin Jacobs) Date: Thu, 23 May 2002 14:05:46 -0400 (EDT) Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: <200205231716.g4NHGFN32731@odiug.zope.com> Message-ID: On Thu, 23 May 2002, Guido van Rossum wrote: > > Further, the expectation of real-live Python users is that it will > > contain the boolean type. This is not an academic argument for me, > > and likely for many others. We have large code bases that use the > > types module, and expect the module to rigorously track new types > > until something better is fully implemented, stable, and the old > > module is properly depricated. > > Why don't you wait and see what shows up in Python 2.3 when it is > released. No, I won't just wait and see. Do we want a development model where everyone is satisfied to wait to see what gifts magically arrive in the next release? No thank you, Mr. Gates. > It's good that we're arguing about this now -- we should offer > something to replace all features of the the types module in 2.3. I have a hard time seeing how the types module can be replaced by anything substantively different. Its sole purpose is not complex: to be a container of type names and type objects. Clearly, it is not the best API for many tasks, but it is a standard one that is widely used. So if there is something better to be done, we should create a new module and be clever there. Moving right along, -Kevin -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com From guido@python.org Thu May 23 19:34:46 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 23 May 2002 14:34:46 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: Your message of "Thu, 23 May 2002 14:05:46 EDT." References: Message-ID: <200205231834.g4NIYkk00673@odiug.zope.com> [me] > > Why don't you wait and see what shows up in Python 2.3 when it is > > released. [Kevin Jacobs] > No, I won't just wait and see. Do we want a development model where > everyone is satisfied to wait to see what gifts magically arrive in > the next release? No thank you, Mr. Gates. Did you get up on the wrong side of the bed today? :-) My point was that the argument based on "types.py is not deprecated" is flawed because it may well become deprecated. (And I base that expectation upon the fact that most commonly used types now have built-in names, so the need to have types.py is largely going away.) > > It's good that we're arguing about this now -- we should offer > > something to replace all features of the the types module in 2.3. > > I have a hard time seeing how the types module can be replaced by > anything substantively different. Its sole purpose is not complex: > to be a container of type names and type objects. I don't think we specifically need a *container* for those names though; all we need is names. Most people use "from types import *" and then use 1 or 2 of the imported names. In the distant past there was no way to spell those names except by using type(0) etc.; adding types.py was an improvement because it added names for the types, and at the time this was easier done that making them all builtins. But now we have a better way: use int, str, bool etc. directly. > Clearly, it is not the best API for many tasks, but it is a standard > one that is widely used. So if there is something better to be > done, we should create a new module and be clever there. IMO the idea of having a single module that exposes names for "all" types is flawed. In general, the name for a type should be exposed by the module or package that defines it. For example, types related to weak references are exposed by the weakref module. The types for built-in data types are of course exposed as built-in names. What remains in types.py is various types related to how Python represents programs and executes them: for example modules, classes, functions, methods, instances, code, frames, tracebacks. I'm not sure there's much of a need to name those types, and if there is, they should probably be imported from a more specialized module. Perhaps they should go into sys? But I'd like to see a demonstrated need for naming these types first. Note that 2.2 was inconsistent in its addition of names for new types to types.py: GeneratorType and DictProxyType were added, but classmethod, staticmethod, property, and the various internal descriptor types were not. I don't expect that "import types" will issue a warning in 2.3, but the documentation will point out its bleak destiny and in 2.4 a warning might indeed be issued. I don't expect types.py to go away before 3.0 (whenever that is :-), but I don't see why we should keep adding to it. If you want naming consistency, use 'int', 'bool' etc. If you don't want to touch old code, then don't touch it. If you want to be compatible with Python-before-2.2, you have to live with some messiness. That's life. We're building a new intersection to improve traffic flow -- in the mean time, be prepared for backups. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu May 23 19:41:40 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 23 May 2002 14:41:40 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: Your message of "Thu, 23 May 2002 14:05:46 EDT." References: Message-ID: <200205231841.g4NIfek00689@odiug.zope.com> Specific tasks, by the way: - Add a 'string' built-in type that's the common base class for str and unicode. - Make buffer, xrange, and slice into new-stype types. - Figure out which type names are commonly tested for that aren't yet built-ins, and propose a way to expose their names. - The 'new' module may also become deprecated; its functions (instance, instancemethod, function, code, module, and classobj) should be implementable by calling the appropriate types (not all of these are their own factories yet, so that's another task). Then for compatibility, a module "new.py" may be provided that implements the interfaces in a backward-compatible way. --Guido van Rossum (home page: http://www.python.org/~guido/) From amckay@merlintechnologies.com Thu May 23 20:05:33 2002 From: amckay@merlintechnologies.com (Andy McKay) Date: Thu, 23 May 2002 12:05:33 -0700 Subject: [Python-Dev] New bugtracker project In-Reply-To: <20020523023628.D24384@tummy.com> References: <200205211638.g4LGcVw19247@odiug.zope.com> <2mr8k4o878.fsf@starship.python.net> <20020523023628.D24384@tummy.com> Message-ID: <200205231900.g4NJ0Cp13845@mail.merlinsoftech.com> > Submitting a bug/patch can be as simple as sending a plain text message > without any special markup. Special name:value information can be > provided if known (releases it's related to, etc)... I altered ActiveState's bugzilla to do this, although I dont think its such a good idea. By submitting through the web interface you can force people to give you certain data for example os, python version etc. You either end up getting inconsistent data or forcing emails to be written in a certain way. Once a bug is entered, sure send emails all you like. -- Andy McKay From skip@pobox.com Thu May 23 20:09:46 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 23 May 2002 14:09:46 -0500 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: <200205231841.g4NIfek00689@odiug.zope.com> References: <200205231841.g4NIfek00689@odiug.zope.com> Message-ID: <15597.15994.863417.366844@12-248-41-177.client.attbi.com> Guido> Specific tasks, by the way: Guido> - Add a 'string' built-in type that's the common base class for str Guido> and unicode. Guido> - Make buffer, xrange, and slice into new-stype types. I'm working on xrange at the moment. Guido> - Figure out which type names are commonly tested for that aren't Guido> yet built-ins, and propose a way to expose their names. Guido> - The 'new' module may also become deprecated; its functions Guido> (instance, instancemethod, function, code, module, and Guido> classobj) should be implementable by calling the appropriate Guido> types (not all of these are their own factories yet, so that's Guido> another task). Then for compatibility, a module "new.py" may Guido> be provided that implements the interfaces in a Guido> backward-compatible way. If new is important enough to retain for backward compatibility, my guess is types would be as well. I am working on a PEP. If I can get a patch generated for the xrange type builtin in the next little while I will release it for comments today, skeletal though it is. Skip From guido@python.org Thu May 23 20:11:53 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 23 May 2002 15:11:53 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: Your message of "Thu, 23 May 2002 14:09:46 CDT." <15597.15994.863417.366844@12-248-41-177.client.attbi.com> References: <200205231841.g4NIfek00689@odiug.zope.com> <15597.15994.863417.366844@12-248-41-177.client.attbi.com> Message-ID: <200205231911.g4NJBrL07321@odiug.zope.com> > If new is important enough to retain for backward compatibility, my > guess is types would be as well. Is there a misunderstanding of the meaning of deprecation? *Of course* it will be retained for backward compatibility -- for a number of releases. But eventually it will disappear, and several releases before that moment all uses of it should be flagged by warnings. Thanks for working on the tasks; I'm looking forward to the PEP! --Guido van Rossum (home page: http://www.python.org/~guido/) From gmcm@hypernet.com Thu May 23 20:33:18 2002 From: gmcm@hypernet.com (Gordon McMillan) Date: Thu, 23 May 2002 15:33:18 -0400 Subject: [Python-Dev] New bugtracker project In-Reply-To: Message-ID: <3CED0BBE.17241.1E9CB415@localhost> On 23 May 2002 at 7:35, Matthias Urlichs wrote: > Looking through the roundup code, a SQL backend > looks like a good idea. The code for finding > elements needs to be rewritten (currently it's not > database specific and just scans everything -- that > doesn't scale), but other than that it should be > relatively straightforward. The *straightforward* way of doing an SQL backend would be to have (almost) everything in one table, defined as (int key, binary stuff). That's utterly useless for querying, since even the DB doesn't know what it contains. Contrast with Bugzilla's data model: http://www.bugzilla.org/docs/html/dbschema.html [That's not to say the problem is unsolvable - just not straightforward.] -- Gordon http://www.mcmillan-inc.com/ From pyth@devel.trillke.net Thu May 23 20:33:00 2002 From: pyth@devel.trillke.net (holger krekel) Date: Thu, 23 May 2002 21:33:00 +0200 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: <200205231834.g4NIYkk00673@odiug.zope.com>; from guido@python.org on Thu, May 23, 2002 at 02:34:46PM -0400 References: <200205231834.g4NIYkk00673@odiug.zope.com> Message-ID: <20020523213300.D11253@prim.han.de> Guido van Rossum wrote: > [me] > > > Why don't you wait and see what shows up in Python 2.3 when it is > > > released. > > [Kevin Jacobs] > > No, I won't just wait and see. Do we want a development model where > > everyone is satisfied to wait to see what gifts magically arrive in > > the next release? No thank you, Mr. Gates. > > Did you get up on the wrong side of the bed today? :-) > > My point was that the argument based on "types.py is not deprecated" > is flawed because it may well become deprecated. It follows: *every* argument is flawed because the basis of the argument might change or be deprecated :-) I agree to Kevin that deprecating the 'cold way' is not nice. Moreover, I don't think that 'use the builtins' is a brilliant idea. Even in the standard lib more than *80* modules use the name 'list' for their own purposes. People just don't associate 'list', 'dict', 'str' etc. with types but use it in every day live as a convenient name. Much like the infamous 'int i;' they want to have a canonical 'working-horse' name which binds an object of a certain type (but not the type itself!). ('int' is not used like this because 'i' is world-wide-known to an int or unsigned int anyway :-) To me 'types.*' and even easier type(..) are still the consistent and explicit ways to reference type objects. the builtins substitute the functionality but force into conflicting, un-intuitive and implicit naming. many-builtins-are-the-path-to-all-evil-ly yours, holger From guido@python.org Thu May 23 20:35:56 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 23 May 2002 15:35:56 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: Your message of "Thu, 23 May 2002 21:33:00 +0200." <20020523213300.D11253@prim.han.de> References: <200205231834.g4NIYkk00673@odiug.zope.com> <20020523213300.D11253@prim.han.de> Message-ID: <200205231935.g4NJZuU07409@odiug.zope.com> > It follows: *every* argument is flawed because the basis of the > argument might change or be deprecated :-) I agree to > Kevin that deprecating the 'cold way' is not nice. Nobody suggested the module would be removed instantly (which I presume you mean by "deprecating the 'cold way'" -- otherwise I don't understand what you mean by that). Please read the documentation on the deprecation process (PEP 4 and 5, and to some extent PEP 230) before making assumptions. > Moreover, I don't think that 'use the builtins' is > a brilliant idea. Even in the standard lib more than *80* modules use > the name 'list' for their own purposes. People just don't > associate 'list', 'dict', 'str' etc. with types but use it in every > day live as a convenient name. Most of these have been built-in functions for a very long time, so the naming conflict has always been an issue. --Guido van Rossum (home page: http://www.python.org/~guido/) From DavidA@ActiveState.com Thu May 23 20:45:23 2002 From: DavidA@ActiveState.com (David Ascher) Date: Thu, 23 May 2002 12:45:23 -0700 Subject: [Python-Dev] New bugtracker projec References: <3CED0BBE.17241.1E9CB415@localhost> Message-ID: <3CED46D3.4000107@ActiveState.com> As a followup to a previous message, I remembered one problem with the original roundup nosy-list mechanism, and that's the "adding to cc list by replying to a email" behavior. It's the key behind the nosy-list concept, and I must say that it worked great when we had a small team of developers (4-5) and beta users. When we scaled up to over 10 people and thousands of users, then that behavior was much too generous, and you hesitated to make a comment on a bug for fear of being attached to that bug forever. Bugzilla is quite explicit about who gets added to the "candidate cc list" by default (the "component owner", the "QA contact" and the "reporter"), and one has to explicitely add oneself to the bug (although the UI could be smarter, since bugzilla knows you're logged in if you are, it should be a one-click operation). Again, I'm not arguing for Bugzilla as much as for the features and control offered by bugzilla. As an aside -- Gordon, or anyone else looking at these issues, if you want to file fake bugs, post attachments, etc. on our bugzilla installation (e.g. Komodo or ActivePython) to see how the thing "feels", feel free. If you need privileged access (e.g. to be a component owner), we can do that too. see e.g. bugs.activestate.com/ActivePython. --david From guido@python.org Thu May 23 20:50:08 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 23 May 2002 15:50:08 -0400 Subject: [Python-Dev] New bugtracker projec In-Reply-To: Your message of "Thu, 23 May 2002 12:45:23 PDT." <3CED46D3.4000107@ActiveState.com> References: <3CED0BBE.17241.1E9CB415@localhost> <3CED46D3.4000107@ActiveState.com> Message-ID: <200205231950.g4NJo8m07502@odiug.zope.com> [David] > As a followup to a previous message, I remembered one problem with the > original roundup nosy-list mechanism, and that's the "adding to cc list > by replying to a email" behavior. It's the key behind the nosy-list > concept, and I must say that it worked great when we had a small team of > developers (4-5) and beta users. When we scaled up to over 10 people > and thousands of users, then that behavior was much too generous, and > you hesitated to make a comment on a bug for fear of being attached to > that bug forever. There should be a way to remove yourself. Possibly by sending another email (but there should also be an explicit web control that shows whether you're on the list or not and lets you change that). And there should be a way to send a reply without being attached. But the default of being attached seems fine. --Guido van Rossum (home page: http://www.python.org/~guido/) From jacobs@penguin.theopalgroup.com Thu May 23 20:57:28 2002 From: jacobs@penguin.theopalgroup.com (Kevin Jacobs) Date: Thu, 23 May 2002 15:57:28 -0400 (EDT) Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: <200205231834.g4NIYkk00673@odiug.zope.com> Message-ID: > [Kevin Jacobs] > > No, I won't just wait and see. Do we want a development model where > > everyone is satisfied to wait to see what gifts magically arrive in > > the next release? No thank you, Mr. Gates. > > [Guido van Rossum] > Did you get up on the wrong side of the bed today? :-) Yeah. I'm pretty crabby today -- the air conditioning for our offices is not working today and its about 10,000 degrees here. I really should add more smileys, or else people will take me too seriously. ;) > [Guido van Rossum] > My point was that the argument based on "types.py is not deprecated" > is flawed because it may well become deprecated. (And I base that > expectation upon the fact that most commonly used types now have > built-in names, so the need to have types.py is largely going away.) I accept that argument completely. The question is how quickly can we make types.py go away, and how do we treat it until then. I would be disappointed, but not extremely inconvenienced if it went away totally in 2.3. In spite of that, I feel strongly that the right thing to do is to maintain types.py consciously. > > [Kevin] > > I have a hard time seeing how the types module can be replaced by > > anything substantively different. Its sole purpose is not complex: > > to be a container of type names and type objects. > > [Guido van Rossum] > I don't think we specifically need a *container* for those names > though; all we need is names. > [...] > IMO the idea of having a single module that exposes names for "all" > types is flawed. > [...] > I don't expect types.py to go away before 3.0 (whenever that is :-), but I > don't see why we should keep adding to it. I think we should keep adding to it, because users expect core types to be in there. So far there has been no counter-argument to this point *at all*. All of the proposed counter-arguments have really been addressing the entirely different (and valid) reasons why types.py should be depricated. > Note that 2.2 was inconsistent in its addition of names for new types > to types.py: GeneratorType and DictProxyType were added, but > classmethod, staticmethod, property, and the various internal > descriptor types were not. You may remember that I wanted to add some of those types into types.py for 2.2 (or 2.2.1?) and was told that they were implementation details and not appropriate for inclusion. The new boolean type is definitly not just an implementation detail. > If you don't want to touch old code, then don't touch it. If you want > to be compatible with Python-before-2.2, you have to live with some > messiness. That's life. We're building a new intersection to improve > traffic flow -- in the mean time, be prepared for backups. :-) In the costs of maintaining types.py weren't so insignificant, then I'd be all for letting it rot. We even have volunteers who are willing to do the work! So why are we still raining on the parade? Hot and bothered, -Kevin -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com From guido@python.org Thu May 23 21:04:14 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 23 May 2002 16:04:14 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: Your message of "Thu, 23 May 2002 15:57:28 EDT." References: Message-ID: <200205232004.g4NK4Ek07553@odiug.zope.com> > I accept that argument completely. The question is how quickly can > we make types.py go away, and how do we treat it until then. I > would be disappointed, but not extremely inconvenienced if it went > away totally in 2.3. In spite of that, I feel strongly that the > right thing to do is to maintain types.py consciously. There's no way it can go away completely in 2.3. The PEP rules about deprecation are clear: they require at least a year (i.e. two releases or more) from the announcement of the deprecation; in cases of heavily used modules like types.py I think we should go slower. But a general rule is that no new features should be added to deprecated modules, since they are only kept around so that old code using them doesn't break; new code should not use deprecated modules. > I think we should keep adding to it, because users expect core types > to be in there. So far there has been no counter-argument to this > point *at all*. All of the proposed counter-arguments have really > been addressing the entirely different (and valid) reasons why > types.py should be depricated. OK, so let me explain why I'm against adding new stuff to types.py. I'm against adding new features to anything that's (about to) being deprecated. The addition of new features sends the wrong message: it suggests that the module is in active use, recommended for use, supported, maintained, etc. When someone finds that types.py has no name for the Boolean type, maybe this will cause them to realize that it's being deprecated and that in the long run (with emphasis on long) it's going away. > In the costs of maintaining types.py weren't so insignificant, then > I'd be all for letting it rot. We even have volunteers who are > willing to do the work! So why are we still raining on the parade? See my note explaining that having all types bundled together was not an optimal idea. I don't want to break old code, but I do want to nudge people into the right direction, away from types.py. --Guido van Rossum (home page: http://www.python.org/~guido/) From mal@lemburg.com Thu May 23 21:13:20 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 23 May 2002 22:13:20 +0200 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 References: <200205231841.g4NIfek00689@odiug.zope.com> Message-ID: <3CED4D60.1070303@lemburg.com> Guido van Rossum wrote: > Specific tasks, by the way: > > - Add a 'string' built-in type that's the common base class for str > and unicode. This class would have to be an abstract type. strings and unicode are too different in their internals to work together from a common base. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From skip@pobox.com Thu May 23 21:14:14 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 23 May 2002 15:14:14 -0500 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: <200205231911.g4NJBrL07321@odiug.zope.com> References: <200205231841.g4NIfek00689@odiug.zope.com> <15597.15994.863417.366844@12-248-41-177.client.attbi.com> <200205231911.g4NJBrL07321@odiug.zope.com> Message-ID: <15597.19862.110808.915776@12-248-41-177.client.attbi.com> >> If new is important enough to retain for backward compatibility, my >> guess is types would be as well. Guido> Is there a misunderstanding of the meaning of deprecation? No, I missed the "deprecated" part of your discussion of the new module but saw the "new.py for backwards compatibility". My apologies. -- Skip Montanaro (skip@pobox.com - http://www.mojam.com/) "Excellant Written and Communications Skills required" - seen on chi.jobs From mal@lemburg.com Thu May 23 21:17:38 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 23 May 2002 22:17:38 +0200 Subject: [Python-Dev] New bugtracker project References: <3CED0BBE.17241.1E9CB415@localhost> Message-ID: <3CED4E62.4030502@lemburg.com> Could you move this dicussion to a SIG ? Thanks, -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From guido@python.org Thu May 23 21:13:06 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 23 May 2002 16:13:06 -0400 Subject: [Python-Dev] Socket timeout patch In-Reply-To: Your message of "Sun, 12 May 2002 08:27:40 EDT." <20020512082740.C10230@eecs.tufts.edu> References: <20020512082740.C10230@eecs.tufts.edu> Message-ID: <200205232013.g4NKD6X07596@odiug.zope.com> > The socket timeout patch is finally available with doc > updates n' unit test as patch #555085 in the SF tracker. This > implementation provides timeout functionality at the C > level. A patch to socket.py also fixes the problem of losing > data while an exception is thrown to the underlying socket. Hi Michael, Sorry for taking such a long time to review this. (Apparently nobody else had time either.) And thanks for doing this! I think this is a great start. Rather than typing them in the SF bug manager issue I am mailing them (I'll add a link to the archived version of this message to the SF bug manager to track the activity). General style nits: - You submitted a reverse diff! Customary is diff(old, new). - Please don't put a space between the function name and the open parenthesis. (You do this both in C and in Python code.) - Also please don't put a space between the open parenthesis and the first argument (you do this almost consistently in test_timeout.py). - Please don't introduce lines longer than 78 columns. Feedback on the patch to socket.py: - I think you're changing the semantics of unbuffered and line-buffered reads/writes on Windows. For one thing, you no longer implement line-buffered writes correctly. The idea is that if the buffer size is set to 1, data is flushed at \n only, so that if the code builds up the line using many small writes, this doesn't result in many small sends. There was code for this in write() -- why did you delete it? - It also looks like you've broken the semantics of size<0 in read(). - Maybe changing the write buffer to a list makes sense too? - Since this code appears independent from the timeoutsocket code, maybe we can discuss this separately? Feedback on the documentation: - I would document that the argument to settimeout() should be int or float (hm, can it be a long? that should be acceptable even if it's strange), and that the return value of gettimeout() is None or a float. - You may want to document the interaction with blocking mode. Feedback on the C socket module changes: - Why import the select module? It's much more efficient to use the select system call directly. You don't need all the overhead and generality that the select module adds on top of it, and it costs a lot (select allocates lots of objects and lots of memory and hence is very expensive). - is already included by Python.h. - Please don't introduce more static functions with a 'Py' name prefix. - You should raise TypeError if the type of the argument is wrong, and ValueError if the value is wrong (out of range). Not SocketError. - I believe that you can't reliably maintain a "sock_blocking" flag; there are setsockopt() or ioctl() calls that can make a socket blocking or non-blocking. Also, there's no way to know whether a socket passed to fromfd() is in blocking mode or not. - There are refcount bugs. I didn't do a detailed review of these, but I note that the return value from PyFloat_FromDouble() in PySocketSock_settimeout() is leaked. (There's an INCREF that's only needed for the None case.) - The function internal_select() is *always* used like this: count = internal_select (s, 1); if (count < 0) return NULL; else if (count == 0) /* Timeout elapsed */ return PyTimeout_Err (); If internal_select() called PyTimeout_Err() itself, all call sites could be simplified to this: count = internal_select (s, 1); if (count < 0) return NULL; or even (now that the count variable is no longer needed) to this: if(internal_select (s, 1) < 0) return NULL; - The accept() wrapper contains this bit of code (only showing the Unix variant): if (newfd < 0) if (!s->sock_blocking || (errno != EAGAIN && errno != EWOULDBLOCK)) return s->errorhandler (); Isn't the sense of testing s->sock_blocking wrong? I would think that if we're in blocking mode we'd want to return immediately without even checking the errno. I recommend writing this out more clearly, e.g. like this: if (s->sock_blocking) return s->errorhandler(); /* A non-blocking accept() failed */ if (errno != EAGAIN && errno != EWOULDBLOCK) return s->errorhandler(); - What is s->errorhandler() for? AFAICT, this is always equal to PySocket_Err! - The whole interaction between non-blocking mode and timeout mode is confusing to me. Are you sure that this always does the right thing? Have you even thought about what "the right thing" is in all 4 combinations? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu May 23 21:15:27 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 23 May 2002 16:15:27 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: Your message of "Thu, 23 May 2002 22:13:20 +0200." <3CED4D60.1070303@lemburg.com> References: <200205231841.g4NIfek00689@odiug.zope.com> <3CED4D60.1070303@lemburg.com> Message-ID: <200205232015.g4NKFRC07656@odiug.zope.com> > > - Add a 'string' built-in type that's the common base class for str > > and unicode. > > This class would have to be an abstract type. strings and unicode > are too different in their internals to work together from a > common base. Of course. I never intended otherwise. I'll add it today. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com Thu May 23 21:24:11 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 23 May 2002 15:24:11 -0500 Subject: [Python-Dev] New bugtracker project In-Reply-To: <3CED4E62.4030502@lemburg.com> References: <3CED0BBE.17241.1E9CB415@localhost> <3CED4E62.4030502@lemburg.com> Message-ID: <15597.20459.950209.436515@12-248-41-177.client.attbi.com> mal> Could you move this dicussion to a SIG ? Yeah, we need to room to discuss types.py... ;-) S From barry@zope.com Thu May 23 21:28:01 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 23 May 2002 16:28:01 -0400 Subject: [Python-Dev] New bugtracker project References: <3CED0BBE.17241.1E9CB415@localhost> <3CED4E62.4030502@lemburg.com> <15597.20459.950209.436515@12-248-41-177.client.attbi.com> Message-ID: <15597.20689.410200.360489@anthem.wooz.org> >>>>> "SM" == Skip Montanaro writes: mal> Could you move this dicussion to a SIG ? SM> Yeah, we need to room to discuss types.py... ;-) Well, use the types-sig of course! :) -Barry From pyth@devel.trillke.net Thu May 23 21:30:45 2002 From: pyth@devel.trillke.net (holger krekel) Date: Thu, 23 May 2002 22:30:45 +0200 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: <200205231935.g4NJZuU07409@odiug.zope.com>; from guido@python.org on Thu, May 23, 2002 at 03:35:56PM -0400 References: <200205231834.g4NIYkk00673@odiug.zope.com> <20020523213300.D11253@prim.han.de> <200205231935.g4NJZuU07409@odiug.zope.com> Message-ID: <20020523223045.E11253@prim.han.de> Guido van Rossum wrote: > > It follows: *every* argument is flawed because the basis of the > > argument might change or be deprecated :-) I agree to > > Kevin that deprecating the 'cold way' is not nice. > > Nobody suggested the module would be removed instantly (which I > presume you mean by "deprecating the 'cold way'" -- otherwise I don't > understand what you mean by that). no, of course it is not removed instantly. 'Cold' was meant to mean that something is beeing treated as deprecated while it is not. > > Moreover, I don't think that 'use the builtins' is > > a brilliant idea. Even in the standard lib more than *80* modules use > > the name 'list' for their own purposes. People just don't > > associate 'list', 'dict', 'str' etc. with types but use it in every > > day live as a convenient name. > > Most of these have been built-in functions for a very long time, so > the naming conflict has always been an issue. True, but this doesn't really weaken my argument. But let's just drop this issue together with the types module if you think it does too much harm, holger From lolita86@libero.it Thu May 23 17:21:04 2002 From: lolita86@libero.it (lolita) Date: Thu, 23 May 2002 23.19.24 +0200 Subject: [Python-Dev] Eros e soldi:guadagna con internet 0,08 euro a clic Message-ID: sono lolita=2C voglio presentarti il mio nuovo sito affiliazione gratuita con guadagni immediati=3A erotismo=2C chat=2Cloghi e sonerie etc=2C etc=2C l'unico sito che paga cos=EC tanto 0=2C08 euro a clic =2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2Eguarda bene la pg di affiliazione=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2Ee buon divertimento=2E visita il sito=3A http=3A=2F=2Fmembers=2Exoom=2Eit=2Fmarym1976 http=3A=2F=2Fmembers=2Exoom=2Eit=2Fmarym1976 http=3A=2F=2Fmembers=2Exoom=2Eit=2Fmarym1976 From skip@pobox.com Thu May 23 22:21:00 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 23 May 2002 16:21:00 -0500 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: <200205231716.g4NHGFN32731@odiug.zope.com> References: <200205231716.g4NHGFN32731@odiug.zope.com> Message-ID: <15597.23868.169344.420730@12-248-41-177.client.attbi.com> --aH8BSLKya8 Content-Type: text/plain; charset=us-ascii Content-Description: message body text Content-Transfer-Encoding: 7bit Guido> It's good that we're arguing about this now -- we should offer Guido> something to replace all features of the the types module in 2.3. Attached is the start of a PEP (regarding which I welcome all inputs, even cranky ones by people suffering without air conditioning). I will be going out of town tomorrow morning for the Memorial Day weekend, so if you send me feedback be prepared to wait a few days for a reply. Skip --aH8BSLKya8 Content-Type: text/plain Content-Description: skeletal types module deprecation PEP Content-Transfer-Encoding: 7bit PEP: NNN Title: Deprecating the types module Version: $Revision:$ Last-Modified: $Date:$ Author: skip@pobox.com (Skip Montanaro) Status: Draft Type: Standards Track Created: 23-May-2002 Post-History: Python-Version: 2.3 Abstract This PEP describes the steps necessary to deprecate the types module and eventually remove it from the distribution. Introduction The types module has long been available as a convenient source of type objects for those situations where programmers need to perform type-specific operations. It has never truly been necessary, since whatever it does could be done by calling the type() builtin function with object of the desired type as an argument. For many types it is certainly a convenience though. Some objects can only be generated from Python code by indirect means. For example, traceback objects are generated as a side effect of raising exceptions. As type/class unification progresses, the justification for the presence of the types becomes a bit less clear. Since many C types can now be subclassed, the relationship between an object and its type is no longer one-to-one, as this simple example demonstrates: >>> class myint(int): ... pass ... >>> myint(1) 1 >>> type(myint(1)) >>> type(myint(1)) == int False >>> isinstance(myint(1), int) True Still, the types module has been around for a long time. Replacing it will not be easy. This PEP considers each of the objects the types module exports. Easy Types The most commonly used objects exported by the types module are already present in builtins as constructor objects. They are: Symbol in types module Symbol in builtins ---------------------- ------------------ ComplexType complex DictType, DictionaryType dictionary FileType file, open FloatType float IntType int ListType list LongType long ObjectType object StringType str TupleType tuple TypeType type UnicodeType unicode When comparisons against those types are required, programmers can simply use the name present in builtins, e.g.: if type(o) is int: ... or if isinstance(o, int): ... instead of if type(o) is IntType: ... Testing for inclusion in a set of types is a little less straightforward if you are concerned about possible subclassing. Currently, to see if an object is a number you would write if type(o) in (IntType, FloatType, ComplexType): ... That would be converted to if type(o) in (int, float, complex): ... or if (isinstance(o, int) or isinstance(o, float) or isinstance(o, complex)): ... The last case is decidedly cumbersome. In short, converting code to use the easy types would be fairly trivial. Harder Types The types that programmers manipulate most often are the "easy" types -- those they create and manipulate directly. The types module exposes a number of less commonly used types as well. Some of them are created by calling a builtin function, while others are generated by the Python runtime as a side effect of another operation (e.g., an exception handler has access to a traceback object generated by the runtime). The table below suggests how these symbols in the types module might be replaced: Symbol in types module Suggested replacement ---------------------- --------------------- BufferType replace buffer builtin function with a callable type object BuiltinFunctionType ??? BuiltinMethodType ??? ClassType ??? CodeType ??? DictProxyType ??? EllipsisType call type(Ellipsis) FrameType ??? FunctionType, LambdaType ??? GeneratorType ??? InstanceType ??? MethodType, UnboundMethodType ??? ModuleType ??? NoneType call type(None) SliceType replace slice builtin function with a callable type object TracebackType ??? XRangeType replace xrange builtin function with a callable type object Most of these types are only used in fairly introspective code. For the most part, programmers using them can just call type() with an object of the desired type. On the other hand, generating some of them is cumbersome. For example, to get an object to assign to TracebackType, in the types module an exception is raised and then calls type(sys.get_exc_info()). Similar machinations are necessary to get a frame object. Since these types reflect implementation details to a certain degree (are all of these available in Jython?), one possible destination for those marked with "???" is the sys module. Placing them in the builtins module seems like it would just add bloat to that module (the same could probably be said for placing them in sys though). Miscellaneous With the arrival of Unicode objects, it became necessary to see if objects were plain strings or Unicode objects. To make it a little easier to perform this check the StringTypes tuple was added to the types module. Code can thus check for either string type using if type(o) in StringTypes: ... A common (abstract) base class for strings is needed. If implemented, it would replace this functionality. Perhaps a similar approach (create an abstract number type) would improve the numeric test shown earlier. There are currently no symbols in the types module which represent the type of boolean or iterator objects. References PEP 254 - "Making Classes Look More Like Types" PEP 260 - "Simplify xrange()" PEP 285 - "Adding a bool type" Implementation xrange type in builtins - http://www.python.org/sf/559833 others tbd. Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil fill-column: 70 End: --aH8BSLKya8-- From rjones@ekit-inc.com Thu May 23 23:14:44 2002 From: rjones@ekit-inc.com (Richard Jones) Date: Fri, 24 May 2002 08:14:44 +1000 Subject: [Python-Dev] Re: New bugtracker project Message-ID: <200205240814.44171.rjones@ekit-inc.com> Sean Reifschneider wrote: > This is one of the things that really impressed me about the Debian bugs > system. While it includes a web interface as well, it's primary mechanism > of control is via e-mail. Have a look at Roundup's email gateway - I think you'll find it does the same stuff as the debian collector (or at least similar stuff). Though the "email the submitter" part was expanded to the concept of a mini mailing list for all interested users. Richard From DavidA@ActiveState.com Thu May 23 23:15:25 2002 From: DavidA@ActiveState.com (David Ascher) Date: Thu, 23 May 2002 15:15:25 -0700 Subject: [Python-Dev] New bugtracker project References: <3CED0BBE.17241.1E9CB415@localhost> <3CED4E62.4030502@lemburg.com> Message-ID: <3CED69FD.8080608@ActiveState.com> M.-A. Lemburg wrote: > Could you move this dicussion to a SIG ? as soon as someone tells me where to do it, I'll gladly subscribe. Or we could do it on a bug =) Sorry, MAL. Cheers, --da From rjones@ekit-inc.com Thu May 23 23:22:28 2002 From: rjones@ekit-inc.com (Richard Jones) Date: Fri, 24 May 2002 08:22:28 +1000 Subject: [Python-Dev] Re: New bugtracker project Message-ID: <200205240822.28908.rjones@ekit-inc.com> David Ascher wrote: > As a followup to a previous message, I remembered one problem with the > original roundup nosy-list mechanism, and that's the "adding to cc list > by replying to a email" behavior. This is configurable in the new roundup. The authors and recipients (configured separately) may be: 1. always added to the nosy list, 2. only added on the creation of a new issue, or 3. never added. Richard From rjones@ekit-inc.com Thu May 23 23:27:22 2002 From: rjones@ekit-inc.com (Richard Jones) Date: Fri, 24 May 2002 08:27:22 +1000 Subject: [Python-Dev] Re: New bugtracker project Message-ID: <200205240827.22565.rjones@ekit-inc.com> Guido van Rossum wrote: > There should be a way to remove yourself. Possibly by sending another > email (but there should also be an explicit web control that shows > whether you're on the list or not and lets you change that). I didn't really have much experience with the old roundup, but the new definitely tells you who is on the nosy list (through the web). Making add/remove modifications to multilink properties through the web via [nosy=-guido] has been in the wishlist for a while. It's never bubbled up to the top of anyone's annoyance list though. Please note that we do have users of this software - don't be thinking that no-one's actually using this in real life, they are! Richard From rjones@ekit-inc.com Thu May 23 23:29:47 2002 From: rjones@ekit-inc.com (Richard Jones) Date: Fri, 24 May 2002 08:29:47 +1000 Subject: [Python-Dev] Re: New bugtracker project Message-ID: <200205240829.47062.rjones@ekit-inc.com> M.-A. Lemburg wrote: > Could you move this dicussion to a SIG ? I'd be happy for the discussion to move to the roundup-users list - it's very low volume. It exists for users of roundup to get help. Kinda assumes that you're going to go with roundup though, so might be a bit premature. Richard From tismer@tismer.com Fri May 24 01:38:00 2002 From: tismer@tismer.com (Christian Tismer) Date: Thu, 23 May 2002 17:38:00 -0700 Subject: [Python-Dev] small bdb bug resolved. Message-ID: <3CED8B68.9030303@tismer.com> I think I solved bug item #210682. Should I check it in? The story is here: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=210682&group_id=5470 ciao - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From guido@python.org Fri May 24 01:54:06 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 23 May 2002 20:54:06 -0400 Subject: [Python-Dev] small bdb bug resolved. In-Reply-To: Your message of "Thu, 23 May 2002 17:38:00 PDT." <3CED8B68.9030303@tismer.com> References: <3CED8B68.9030303@tismer.com> Message-ID: <200205240054.g4O0s6S22900@pcp742651pcs.reston01.va.comcast.net> > I think I solved bug item #210682. > Should I check it in? > The story is here: > http://sourceforge.net/tracker/?func=detail&atid=105470&aid=210682&group_id=5470 I saw this and started drooling! I haven't had the time to verify that it doesn't have any other side effects. If you're confident that it works, go ahead and check it in! (I'm curious whether this fixes a similar problem in the IDLE debugger.) --Guido van Rossum (home page: http://www.python.org/~guido/) From martin@v.loewis.de Fri May 24 08:26:48 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 24 May 2002 09:26:48 +0200 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: References: Message-ID: Kevin Jacobs writes: > http://www.python.org/doc/current/lib/module-types.html: > > 3.6 types -- Names for ALL built-in types > > This module defines names for ALL object types that are used by the > standard Python interpreter, but not for the types defined by various > extension modules. > > How much clearer does it need to be? So I guess the documentation is wrong; it should read "This module defines names for various object types that are used by the standard interpreter". That documentation is already incorrect, it does not list CellType CObjectType MethodDescrType MemberDescrType GetSetDescrType WrapperDescrType method-wrapperType DictIterType EnumType StaticMethodType SeqIterType CallIterType ImmutableListType NotImplementedType SuperType Regards, Martin From martin@v.loewis.de Fri May 24 08:35:03 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 24 May 2002 09:35:03 +0200 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: <200205231841.g4NIfek00689@odiug.zope.com> References: <200205231841.g4NIfek00689@odiug.zope.com> Message-ID: Guido van Rossum writes: > - The 'new' module may also become deprecated; its functions > (instance, instancemethod, function, code, module, and classobj) > should be implementable by calling the appropriate types (not all of > these are their own factories yet, so that's another task). Then > for compatibility, a module "new.py" may be provided that implements > the interfaces in a backward-compatible way. I think that "new" might be the place where the type objects are exposed; it's not just the matter of making the types callable, but also of exposing their names somewhere. Then, in types.py class _C: def _m(self): pass ClassType = type(_C) UnboundMethodType = type(_C._m) # Same as MethodType _x = _C() InstanceType = type(_x) MethodType = type(_x._m) would become ClassType = new.classobj UnboundMethodType = MethodType = new.instancemethod InstanceType = new.instance Regards, Martin From mwh@python.net Fri May 24 10:33:54 2002 From: mwh@python.net (Michael Hudson) Date: 24 May 2002 10:33:54 +0100 Subject: [Python-Dev] New bugtracker projec In-Reply-To: Guido van Rossum's message of "Thu, 23 May 2002 15:50:08 -0400" References: <3CED0BBE.17241.1E9CB415@localhost> <3CED46D3.4000107@ActiveState.com> <200205231950.g4NJo8m07502@odiug.zope.com> Message-ID: <2m1yc1yjnh.fsf@starship.python.net> Guido van Rossum writes: > [David] > > As a followup to a previous message, I remembered one problem with the > > original roundup nosy-list mechanism, and that's the "adding to cc list > > by replying to a email" behavior. It's the key behind the nosy-list > > concept, and I must say that it worked great when we had a small team of > > developers (4-5) and beta users. When we scaled up to over 10 people > > and thousands of users, then that behavior was much too generous, and > > you hesitated to make a comment on a bug for fear of being attached to > > that bug forever. > > There should be a way to remove yourself. But of course sf doesn't have one of these, and while it's a tad annoying to still get all the comments on bugs that I moved out of the 2.2.1 category, it hasn't been a major problem yet. Though I guess if you can comment via email, you get more comments... Cheers, M. -- > Look I don't know. Thankyou everyone for arguing me round in > circles. No need for thanks, ma'am; that's what we're here for. -- LNR & Michael M Mason, cam.misc From guido@python.org Fri May 24 14:40:26 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 24 May 2002 09:40:26 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: Your message of "24 May 2002 09:35:03 +0200." References: <200205231841.g4NIfek00689@odiug.zope.com> Message-ID: <200205241340.g4ODeRD25255@pcp742651pcs.reston01.va.comcast.net> > I think that "new" might be the place where the type objects are > exposed; Except its name is wrong. --Guido van Rossum (home page: http://www.python.org/~guido/) From pyth@devel.trillke.net Fri May 24 14:44:28 2002 From: pyth@devel.trillke.net (holger krekel) Date: Fri, 24 May 2002 15:44:28 +0200 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: <200205241340.g4ODeRD25255@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Fri, May 24, 2002 at 09:40:26AM -0400 References: <200205231841.g4NIfek00689@odiug.zope.com> <200205241340.g4ODeRD25255@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020524154428.J11253@prim.han.de> Guido van Rossum wrote: > > I think that "new" might be the place where the type objects are > > exposed; > > Except its name is wrong. this may be the road to all evil, but ... couldn't it be 'type.int' or 'type.UnboundMethod' ? it looks so nicely explicit and short and of course 'type.type==type' holds :-) holger From barry@zope.com Fri May 24 14:53:31 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 24 May 2002 09:53:31 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 References: <200205231841.g4NIfek00689@odiug.zope.com> <200205241340.g4ODeRD25255@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15598.17883.167496.493785@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: >> I think that "new" might be the place where the type objects >> are exposed; GvR> Except its name is wrong. Wait, I know! Let's call it `types'. :) smart-aleck-ly y'rs, -Barry From stephen@xemacs.org Fri May 24 15:15:57 2002 From: stephen@xemacs.org (Stephen J. Turnbull) Date: 24 May 2002 23:15:57 +0900 Subject: [Python-Dev] New bugtracker project In-Reply-To: <200205231900.g4NJ0Cp13845@mail.merlinsoftech.com> References: <200205211638.g4LGcVw19247@odiug.zope.com> <2mr8k4o878.fsf@starship.python.net> <20020523023628.D24384@tummy.com> <200205231900.g4NJ0Cp13845@mail.merlinsoftech.com> Message-ID: <87znyp1vj6.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Andy" == Andy McKay writes: >> Submitting a bug/patch can be as simple as sending a plain text >> message without any special markup. Special name:value >> information can be provided if known (releases it's related to, >> etc)... When I last sent an unformatted bug to Debian, I got an autoreply saying "we have a 100GB bit bucket for this and it's about half full. If you don't want to wait your turn, you can really speed things up (and help the maintainer, too!) by formatting in the following way. Information at the following URLs [more detailed instructions, package search, the BTS URL] may be useful." More polite than that, but you get the idea. Worked for me. Andy> I altered ActiveState's bugzilla to do this, although I dont Andy> think its such a good idea. By submitting through the web Andy> interface you can force people to give you certain data for Andy> example os, python version etc. You either end up getting Andy> inconsistent data or forcing emails to be written in a Andy> certain way. True. I think the typical Debian package gets only a few entries a week. But I think a scoring mechanism that files the bug in a low-priority slot and bounces instructions back to the reporter is a good compromise. One thing I don't like about the Debian BTS is that it presents all unclosed bugs ever. Maintainers should have that in their face, I suppose, but from the user's point of view anything whose last entry is a year or more old is irrelevant. -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN My nostalgia for Icon makes me forget about any of the bad things. I don't have much nostalgia for Perl, so its faults I remember. Scott Gilbert c.l.py From akuchlin@mems-exchange.org Fri May 24 15:57:53 2002 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Fri, 24 May 2002 10:57:53 -0400 Subject: [Python-Dev] Overwriting objects before deallocating them Message-ID: In the cryptography toolkit, the deallocators in the C extensions all overwrite the object's contents before deallocating it. Is this dodgy practice? Is it likely or possible that some day PyObject_Del will need to look at the contents of a non-GC-supporting object before freeing it? (Encryption or hashing objects are not containers, and it's highly unlikely they ever will be, so they'll never need to support GC traversal.) --amk Example code: static void ALGdealloc(PyObject *ptr) { int i; char *obj=(char *)ptr; /* Overwrite the contents of the object, just in case... */ for (i = 0; i < sizeof(ALGobject); i++) *(obj + i) = '\0'; PyObject_DEL(ptr); } (Note to self: use memset().) From guido@python.org Fri May 24 16:07:05 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 24 May 2002 11:07:05 -0400 Subject: [Python-Dev] Overwriting objects before deallocating them In-Reply-To: Your message of "Fri, 24 May 2002 10:57:53 EDT." References: Message-ID: <200205241507.g4OF75V25826@pcp742651pcs.reston01.va.comcast.net> > In the cryptography toolkit, the deallocators in the C extensions all > overwrite the object's contents before deallocating it. Is this dodgy > practice? Is it likely or possible that some day PyObject_Del will > need to look at the contents of a non-GC-supporting object before > freeing it? (Encryption or hashing objects are not containers, and > it's highly unlikely they ever will be, so they'll never need to > support GC traversal.) You're safe. PyObject_Del cannot make any assumptions about the contents of the structure -- not even the type pointer or the reference count. Of course you should stay out of the GC header. --Guido van Rossum (home page: http://www.python.org/~guido/) From neal@metaslash.com Fri May 24 16:18:06 2002 From: neal@metaslash.com (Neal Norwitz) Date: Fri, 24 May 2002 11:18:06 -0400 Subject: [Python-Dev] New bugtracker projec References: <3CED0BBE.17241.1E9CB415@localhost> <3CED46D3.4000107@ActiveState.com> <200205231950.g4NJo8m07502@odiug.zope.com> <2m1yc1yjnh.fsf@starship.python.net> Message-ID: <3CEE59AE.7AF6270D@metaslash.com> Michael Hudson wrote: > > Guido van Rossum writes: > > > [David] > > > As a followup to a previous message, I remembered one problem with the > > > original roundup nosy-list mechanism, and that's the "adding to cc list > > > by replying to a email" behavior. It's the key behind the nosy-list > > > concept, and I must say that it worked great when we had a small team of > > > developers (4-5) and beta users. When we scaled up to over 10 people > > > and thousands of users, then that behavior was much too generous, and > > > you hesitated to make a comment on a bug for fear of being attached to > > > that bug forever. > > > > There should be a way to remove yourself. > > But of course sf doesn't have one of these, and while it's a tad > annoying to still get all the comments on bugs that I moved out of the > 2.2.1 category, it hasn't been a major problem yet. Though I guess if > you can comment via email, you get more comments... If you click the Monitor button when viewing the bug, this will toggle getting mail. So if you are already monitoring the bug, it will unmonitor you. http://sourceforge.net/help/tracker.php?helpname=monitor Neal From guido@python.org Fri May 24 16:26:39 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 24 May 2002 11:26:39 -0400 Subject: [Python-Dev] New bugtracker projec In-Reply-To: Your message of "Fri, 24 May 2002 11:18:06 EDT." <3CEE59AE.7AF6270D@metaslash.com> References: <3CED0BBE.17241.1E9CB415@localhost> <3CED46D3.4000107@ActiveState.com> <200205231950.g4NJo8m07502@odiug.zope.com> <2m1yc1yjnh.fsf@starship.python.net> <3CEE59AE.7AF6270D@metaslash.com> Message-ID: <200205241526.g4OFQd406104@pcp742651pcs.reston01.va.comcast.net> > If you click the Monitor button when viewing the bug, > this will toggle getting mail. So if you are already monitoring > the bug, it will unmonitor you. I know, and this is another pet peeve. The Monitor button doesn't tell you whether you're currently monitoring or not, so if you've forgotten that, you may end up having to click the button twice (if you're already in the desired state). Also, I believe there's another way to become subscribed to a bug (maybe by being the author, maybe by adding a comment to it) that is not revertible. These are all evil hidden states! --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Fri May 24 16:46:22 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 24 May 2002 11:46:22 -0400 Subject: [Python-Dev] Overwriting objects before deallocating them In-Reply-To: Message-ID: [Andrew Kuchling] > In the cryptography toolkit, the deallocators in the C extensions all > overwrite the object's contents before deallocating it. Is this dodgy > practice? Is it likely or possible that some day PyObject_Del will > need to look at the contents of a non-GC-supporting object before > freeing it? Nope. I prefer to write PyObject_Free, because it's exactly the same as PyObject_Del, and spelling it "free" doesn't conjure up misleading images of this having something to do with destructors or refcounts, or even with *objects*. They don't -- all they do is release the raw memory. I'm not sure why, but there's a widespread intransigent belief that it's somehow purer to say "del" for freeing memory that happened to hold an object. The memory API is full of now-institutionalized TMTOWTDI. > (Encryption or hashing objects are not containers, and it's highly > unlikely they ever will be, so they'll never need to support GC > traversal.) You would have to use PyObject_GC_Del if they did, and so that's a different question entirely . From guido@python.org Fri May 24 17:01:46 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 24 May 2002 12:01:46 -0400 Subject: [Python-Dev] Overwriting objects before deallocating them In-Reply-To: Your message of "Fri, 24 May 2002 11:46:22 EDT." References: Message-ID: <200205241601.g4OG1k508220@pcp742651pcs.reston01.va.comcast.net> [Tim] > I'm not sure why, but there's a widespread intransigent belief that > it's somehow purer to say "del" for freeing memory that happened to > hold an object. I'm the origin of this convention; it was part of Python 0.0. Switching from [m]alloc/free to new/del was *not* to differentiate between mere blobs of memory and objects; it was to indicate that these macros and functions were a layer on top of malloc/free. (Originally, they merely changed the signature around a bit.) The new/del names are partly borrowed from C++, where new and delete are the memory (de)allocation operators as well as the object (de)allocation operators. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Fri May 24 17:11:45 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 24 May 2002 12:11:45 -0400 Subject: [Python-Dev] Overwriting objects before deallocating them In-Reply-To: <200205241601.g4OG1k508220@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > ... > The new/del names are partly borrowed from C++, where new and delete > are the memory (de)allocation operators as well as the object > (de)allocation operators. I know, but as a former C++ programmer, that's what I keep tripping over: "del" in C++ does a lot more than just release memory (just as "new" in C++ does a lot more than just allocate memory -- and it may not allocate memory at all). OTOH, "del" in Python doesn't release memory. So "del" in Python's C API doesn't match what Python or C++ mean by it. As far as I'm concerned, the instant you don't type "malloc" literally you should already be well aware that you're not necessarily calling malloc literally . From guido@python.org Fri May 24 17:22:33 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 24 May 2002 12:22:33 -0400 Subject: [Python-Dev] Overwriting objects before deallocating them In-Reply-To: Your message of "Fri, 24 May 2002 12:11:45 EDT." References: Message-ID: <200205241622.g4OGMX508964@pcp742651pcs.reston01.va.comcast.net> > [Guido] > > ... > > The new/del names are partly borrowed from C++, where new and delete > > are the memory (de)allocation operators as well as the object > > (de)allocation operators. > > I know, but as a former C++ programmer, that's what I keep tripping over: > "del" in C++ does a lot more than just release memory (just as "new" in C++ > does a lot more than just allocate memory -- and it may not allocate memory > at all). OTOH, "del" in Python doesn't release memory. So "del" in > Python's C API doesn't match what Python or C++ mean by it. As far as I'm > concerned, the instant you don't type "malloc" literally you should already > be well aware that you're not necessarily calling malloc literally . Yup, it's a mess, forever. --Guido van Rossum (home page: http://www.python.org/~guido/) From David Abrahams" <200205241601.g4OG1k508220@pcp742651pcs.reston01.va.comcast.net> Message-ID: <017001c2033f$24abd860$6501a8c0@boostconsulting.com> From: "Guido van Rossum" > [Tim] > > I'm not sure why, but there's a widespread intransigent belief that > > it's somehow purer to say "del" for freeing memory that happened to > > hold an object. > > I'm the origin of this convention; it was part of Python 0.0. > Switching from [m]alloc/free to new/del was *not* to differentiate > between mere blobs of memory and objects; it was to indicate that > these macros and functions were a layer on top of malloc/free. > (Originally, they merely changed the signature around a bit.) > > The new/del names are partly borrowed from C++, where new and delete > are the memory (de)allocation operators as well as the object > (de)allocation operators. Yes, but in C++, when they're used as operators, they *always* do object construction/destruction as well. In fact, in some forms (placement new) its possible to get just the construction/destruction behavior without any allocation/deallocation. I think Tim is on the right track. -Dave From akuchlin@mems-exchange.org Fri May 24 17:30:57 2002 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Fri, 24 May 2002 12:30:57 -0400 Subject: [Python-Dev] Overwriting objects before deallocating them In-Reply-To: <200205241622.g4OGMX508964@pcp742651pcs.reston01.va.comcast.net> References: <200205241622.g4OGMX508964@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020524163057.GA6422@ute.mems-exchange.org> On Fri, May 24, 2002 at 12:22:33PM -0400, Guido van Rossum wrote: >Yup, it's a mess, forever. Is it worth trying to slowly fix it? Picking a preferred name and making the interpreter's code use it consistently would be a start. Anyway, thanks for the answers, Guido and Tim. Off to finish PCT 1.9alpha2... --amk (www.amk.ca) Sometimes, my brilliance astonishes even me. -- The Doctor, in "The Invisible Enemy" From guido@python.org Fri May 24 17:38:14 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 24 May 2002 12:38:14 -0400 Subject: [Python-Dev] Overwriting objects before deallocating them In-Reply-To: Your message of "Fri, 24 May 2002 12:30:57 EDT." <20020524163057.GA6422@ute.mems-exchange.org> References: <200205241622.g4OGMX508964@pcp742651pcs.reston01.va.comcast.net> <20020524163057.GA6422@ute.mems-exchange.org> Message-ID: <200205241638.g4OGcEG09921@pcp742651pcs.reston01.va.comcast.net> > Is it worth trying to slowly fix it? Picking a preferred name and > making the interpreter's code use it consistently would be a start. We're doing our best: new/del (and case variants) are now recommended everywhere. It's still a mess because 'del' just means 'free', and 'new' mostly just means 'alloc', but because of the large amount of existing code using these conventions I don't want to try to migrate to something new. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Fri May 24 17:35:55 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 24 May 2002 12:35:55 -0400 Subject: [Python-Dev] Overwriting objects before deallocating them In-Reply-To: <200205241622.g4OGMX508964@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > Yup, it's a mess, forever. Yup X 3! I was just venting, I didn't actually want to improve anything here . From mwh@python.net Fri May 24 16:59:29 2002 From: mwh@python.net (Michael Hudson) Date: 24 May 2002 16:59:29 +0100 Subject: [Python-Dev] New bugtracker projec In-Reply-To: Neal Norwitz's message of "Fri, 24 May 2002 11:18:06 -0400" References: <3CED0BBE.17241.1E9CB415@localhost> <3CED46D3.4000107@ActiveState.com> <200205231950.g4NJo8m07502@odiug.zope.com> <2m1yc1yjnh.fsf@starship.python.net> <3CEE59AE.7AF6270D@metaslash.com> Message-ID: <2mvg9dwn8e.fsf@starship.python.net> Neal Norwitz writes: > Michael Hudson wrote: > > > > But of course sf doesn't have one of these, and while it's a tad > > annoying to still get all the comments on bugs that I moved out of the > > 2.2.1 category, it hasn't been a major problem yet. Though I guess if > > you can comment via email, you get more comments... > > If you click the Monitor button when viewing the bug, > this will toggle getting mail. So if you are already monitoring > the bug, it will unmonitor you. > > http://sourceforge.net/help/tracker.php?helpname=monitor Learn something every day -- I didn't know that. Still, the UI leaves something to be desired! Cheers, M. -- I'd certainly be shocked to discover a consensus. ;-) -- Aahz Maruch, comp.lang.python From tim.one@comcast.net Fri May 24 17:48:49 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 24 May 2002 12:48:49 -0400 Subject: [Python-Dev] Overwriting objects before deallocating them In-Reply-To: <20020524163057.GA6422@ute.mems-exchange.org> Message-ID: [Andrew] > Is it worth trying to slowly fix it? Picking a preferred name and > making the interpreter's code use it consistently would be a start. The pymalloc-for-2.3 changes are trying to make it possible for new programs to use a much smaller set of memory API names. Unfortunately, the "recommended" set didn't include PyObject_Del (it recommended PyObject_Free instead), because I wrote the recommendations. That's unfortunate because it only lasted until the first time Guido wrote new code . So for 2.3 the recommended set grew again, to /* raw memory */ PyMem_{Malloc, Realloc, Free} /* raw memory possibly using a different allocator than PyMem_ uses; biased towards lots of "small" allocations */ PyObject_{Malloc, Realloc, Free} /* object memory; the allocators do some initialization as well as allocation; PyObject_Del is identical to PyObject_Free */ PyObject_{New, NewVar, Del} That's it. There's really no reason in 2.3 to use anything else for non-gc memory (including macro spellings! they're almost always identical to the function spellings in 2.3). Under the covers, PyMem_Free in 2.3 is also identical to PyObject_{Free, Del}, but that's a horrible backward compatibility hack I hope we can drop some day (it costs runtime cycles). From guido@python.org Fri May 24 17:59:50 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 24 May 2002 12:59:50 -0400 Subject: [Python-Dev] Style guide (PEP 8) updated Message-ID: <200205241659.g4OGxoS10044@pcp742651pcs.reston01.va.comcast.net> Barry has added (with my consent :-) some additional style guidelines to PEP 8. I welcome suggestions for more guidelines. I discourage arguing over what's already in the PEP (it's a BDFL pronouncement, not a democratic document). --Guido van Rossum (home page: http://www.python.org/~guido/) From neal@metaslash.com Fri May 24 18:50:58 2002 From: neal@metaslash.com (Neal Norwitz) Date: Fri, 24 May 2002 13:50:58 -0400 Subject: [Python-Dev] Style guide (PEP 8) updated References: <200205241659.g4OGxoS10044@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CEE7D82.4D0A7E41@metaslash.com> Guido van Rossum wrote: > > Barry has added (with my consent :-) some additional style guidelines > to PEP 8. I welcome suggestions for more guidelines. I discourage > arguing over what's already in the PEP (it's a BDFL pronouncement, not > a democratic document). Is it worth putting a prohibition against using bool() like this: bool(x == 'blah') or boolean-value == True/False? Of course, other comparisons apply too. Neal From thomas.heller@ion-tof.com Fri May 24 19:04:43 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri, 24 May 2002 20:04:43 +0200 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 References: <200205231716.g4NHGFN32731@odiug.zope.com> <15597.23868.169344.420730@12-248-41-177.client.attbi.com> Message-ID: <167b01c2034d$7b862830$e000a8c0@thomasnotebook> > Testing for inclusion in a set of types is a little less > straightforward if you are concerned about possible subclassing. > Currently, to see if an object is a number you would write > > if type(o) in (IntType, FloatType, ComplexType): > ... > > That would be converted to > > if type(o) in (int, float, complex): > ... > > or > > if (isinstance(o, int) or isinstance(o, float) or > isinstance(o, complex)): > ... > > The last case is decidedly cumbersome. > or if isinstance(o, (int, float, complex)): Thomas From guido@python.org Fri May 24 19:04:51 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 24 May 2002 14:04:51 -0400 Subject: [Python-Dev] Style guide (PEP 8) updated In-Reply-To: Your message of "Fri, 24 May 2002 13:50:58 EDT." <3CEE7D82.4D0A7E41@metaslash.com> References: <200205241659.g4OGxoS10044@pcp742651pcs.reston01.va.comcast.net> <3CEE7D82.4D0A7E41@metaslash.com> Message-ID: <200205241804.g4OI4qA10388@pcp742651pcs.reston01.va.comcast.net> > Is it worth putting a prohibition against using bool() like this: > > bool(x == 'blah') Apart from being redundant, I see nothing wrong with it. :-) > or boolean-value == True/False? Of course, other comparisons apply too. Now *that* is something to add, yes! --Guido van Rossum (home page: http://www.python.org/~guido/) From haering_python@gmx.de Fri May 24 19:18:03 2002 From: haering_python@gmx.de (Gerhard =?iso-8859-15?Q?H=E4ring?=) Date: Fri, 24 May 2002 20:18:03 +0200 Subject: [Python-Dev] Style guide (PEP 8) updated In-Reply-To: <200205241659.g4OGxoS10044@pcp742651pcs.reston01.va.comcast.net> References: <200205241659.g4OGxoS10044@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020524181802.GA2310@lilith.my-fqdn.de> * Guido van Rossum [2002-05-24 12:59 -0400]: > Barry has added (with my consent :-) some additional style guidelines > to PEP 8. I welcome suggestions for more guidelines. I discourage > arguing over what's already in the PEP (it's a BDFL pronouncement, not > a democratic document). I'd add that checks for None should always be done using 'is (not) None'. Failing to do so has once bitten me. Proposed text (added to the 'Optimization and Other Programming Recommendations' section): Comparisions to singletons like the None builtin should always be done with 'is' or 'is not'. Rationale: This is slightly faster, and you don't have to care wether the left-hand object supports comparison to the singleton. Gerhard PS: IIRC the case were I was bitten was involving a __coerce__ special method that didn't support None, but I'm not sure any more. -- This sig powered by Python! Außentemperatur in München: 15.0 °C Wind: 2.9 m/s From guido@python.org Fri May 24 19:25:02 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 24 May 2002 14:25:02 -0400 Subject: [Python-Dev] Allowing Unicode literals even without Unicode support? Message-ID: <200205241825.g4OIP2b10779@pcp742651pcs.reston01.va.comcast.net> Many test modules fail because they use Unicode literals. It's easy enough to skip Unicode-related tests (just test for the existence of 'unicode'), but it's hard to avoid having Unicode literals in the text at all. Should we perhaps silently interpret Unicode literals as regular string literals when compiling without Unicode support? It's easy to do: *** compile.c 26 Apr 2002 01:57:19 -0000 2.242 --- compile.c 24 May 2002 18:21:29 -0000 *************** *** 1135,1147 **** #endif if (isalpha(quote) || quote == '_') { if (quote == 'u' || quote == 'U') { - #ifdef Py_USING_UNICODE quote = *++s; unicode = 1; - #else - com_error(com, PyExc_SyntaxError, - "Unicode literals not supported in this Python"); - return NULL; #endif } if (quote == 'r' || quote == 'R') { --- 1135,1143 ---- #endif if (isalpha(quote) || quote == '_') { if (quote == 'u' || quote == 'U') { quote = *++s; + #ifdef Py_USING_UNICODE unicode = 1; #endif } if (quote == 'r' || quote == 'R') { --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri May 24 19:28:22 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 24 May 2002 14:28:22 -0400 Subject: [Python-Dev] Style guide (PEP 8) updated In-Reply-To: Your message of "Fri, 24 May 2002 20:18:03 +0200." <20020524181802.GA2310@lilith.my-fqdn.de> References: <200205241659.g4OGxoS10044@pcp742651pcs.reston01.va.comcast.net> <20020524181802.GA2310@lilith.my-fqdn.de> Message-ID: <200205241828.g4OISM110796@pcp742651pcs.reston01.va.comcast.net> > I'd add that checks for None should always be done using 'is (not) > None'. Yes. Also, more subtle: beware of writing "if x" when you really mean "if x is not None" -- e.g. when testing whether a variable or argument that defaults to None was set to some other value. This bit us recently in a Zope project when x was a database object that happened to respond to len(), unbeknownst to the author. During testing the database was never empty, so we didn't catch it until much later. --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Fri May 24 19:35:04 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 24 May 2002 14:35:04 -0400 Subject: [Python-Dev] Allowing Unicode literals even without Unicode support? References: <200205241825.g4OIP2b10779@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15598.34776.191101.94638@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> Should we perhaps silently interpret Unicode literals as GvR> regular string literals when compiling without Unicode GvR> support? +1, perhaps with a Warning? -Barry From guido@python.org Fri May 24 19:40:37 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 24 May 2002 14:40:37 -0400 Subject: [Python-Dev] Allowing Unicode literals even without Unicode support? In-Reply-To: Your message of "Fri, 24 May 2002 14:35:04 EDT." <15598.34776.191101.94638@anthem.wooz.org> References: <200205241825.g4OIP2b10779@pcp742651pcs.reston01.va.comcast.net> <15598.34776.191101.94638@anthem.wooz.org> Message-ID: <200205241840.g4OIebS11212@pcp742651pcs.reston01.va.comcast.net> > GvR> Should we perhaps silently interpret Unicode literals as > GvR> regular string literals when compiling without Unicode > GvR> support? > > +1, perhaps with a Warning? > > -Barry Adding the warning causes more code, while the only point of compiling without it is to have *less* code. Also, it would just cause tons of warnings when running the tests with Unicode disabled. --Guido van Rossum (home page: http://www.python.org/~guido/) From sholden@holdenweb.com Fri May 24 14:03:05 2002 From: sholden@holdenweb.com (Steve Holden) Date: Fri, 24 May 2002 09:03:05 -0400 Subject: [Python-Dev] New bugtracker project - Relational backend? References: <200205221312.g4MDCoX09999@pcp742651pcs.reston01.va.comcast.net> <15595.53800.540281.154909@anthem.wooz.org> Message-ID: <001901c20351$f39b49f0$6300000a@holdenweb.com> Because it doesn't allow you to query the database relationally? regards ----------------------------------------------------------------------- Steve Holden http://www.holdenweb.com/ Python Web Programming http://pydish.holdenweb.com/pwp/ ----------------------------------------------------------------------- ----- Original Message ----- From: "Barry A. Warsaw" To: "Guido van Rossum" Cc: Sent: Wednesday, May 22, 2002 1:15 PM Subject: Re: [Python-Dev] New bugtracker project - Relational backend? > > >>>>> "GvR" == Guido van Rossum writes: > > GvR> Semi-unrelated: a thing to consider for flexible presentation > GvR> might be Zope Page Templates. The core TAL (Template > GvR> Attribute Language) code is independent from the Zope > GvR> infrastructure. > > While you're at it, why not ZODB as the back end "database"? :) > > -Barry > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > From guido@python.org Fri May 24 20:20:28 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 24 May 2002 15:20:28 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: Your message of "Thu, 23 May 2002 16:21:00 CDT." <15597.23868.169344.420730@12-248-41-177.client.attbi.com> References: <200205231716.g4NHGFN32731@odiug.zope.com> <15597.23868.169344.420730@12-248-41-177.client.attbi.com> Message-ID: <200205241920.g4OJKSk11551@pcp742651pcs.reston01.va.comcast.net> Looks like a good starting point. Note the convenience of isinstance(x, (t1, t2, t3)). > BuiltinFunctionType ??? > BuiltinMethodType ??? > ClassType ??? > CodeType ??? > DictProxyType ??? > FrameType ??? > FunctionType, LambdaType ??? > GeneratorType ??? > InstanceType ??? > MethodType, UnboundMethodType ??? > ModuleType ??? > TracebackType ??? None of these will be greatly missed except by introspective code. Hm, I wonder if inspect would be the right module to export these types? Or maybe one should simply be encouraged to use inspect.ismodule() etc.? (The type names are still somewhat useful because they can be used as keys for dispatch tables.) --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik@pythonware.com Fri May 24 20:32:50 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Fri, 24 May 2002 21:32:50 +0200 Subject: [Python-Dev] Allowing Unicode literals even without Unicode support? References: <200205241825.g4OIP2b10779@pcp742651pcs.reston01.va.comcast.net> Message-ID: <011701c20359$cbc6c7d0$0900a8c0@spiff> guido wrote: > Many test modules fail because they use Unicode literals. It's easy > enough to skip Unicode-related tests (just test for the existence of > 'unicode'), but it's hard to avoid having Unicode literals in the text > at all. I'm not sure having to write unicode("...") instead of u"..." qualifies as "hard", but life would be a bit easier if we didn't have to... > Should we perhaps silently interpret Unicode literals as regular = string > literals when compiling without Unicode support? +1.0 on silently accepting ASCII-only u-literals also in non- unicode builds. -0.2 on silently accepting non-ASCII u-literals (your patch didn't deal with that, right?). From barry@zope.com Fri May 24 20:36:37 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 24 May 2002 15:36:37 -0400 Subject: [Python-Dev] Style guide (PEP 8) updated References: <200205241659.g4OGxoS10044@pcp742651pcs.reston01.va.comcast.net> <3CEE7D82.4D0A7E41@metaslash.com> <200205241804.g4OI4qA10388@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15598.38469.482752.870353@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: >> or boolean-value == True/False? Of course, other comparisons >> apply too. GvR> Now *that* is something to add, yes! How's this: - Don't compare boolean values to True or False using == (bool types are new in Python 2.3). -Barry From barry@zope.com Fri May 24 20:39:13 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 24 May 2002 15:39:13 -0400 Subject: [Python-Dev] Style guide (PEP 8) updated References: <200205241659.g4OGxoS10044@pcp742651pcs.reston01.va.comcast.net> <20020524181802.GA2310@lilith.my-fqdn.de> <200205241828.g4OISM110796@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15598.38625.603775.874961@anthem.wooz.org> Thanks, I'll add these to PEP 8. From guido@python.org Fri May 24 21:08:43 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 24 May 2002 16:08:43 -0400 Subject: [Python-Dev] Out of date FAQ entries In-Reply-To: Your message of "Wed, 22 May 2002 22:24:34 EDT." <3CEC52E2.962B1CB6@metaslash.com> References: <3CEC52E2.962B1CB6@metaslash.com> Message-ID: <200205242008.g4OK8hi11791@pcp742651pcs.reston01.va.comcast.net> > I was looking over the FAQ and noticed a out-of-date entries. > > I fixed a few: Thanks! > But there are some others, which I can't do a good job updating. > The one's I did update could use some work too. :-) I updated the ones you mentione, except: > 6.24. Why no class methods or mutable class variables? > Mention classmethod(), staticmethod()? This already mentions those (at the bottom). > I'm sure there's others that could use fixing. That's what the FAQ roulette (4th bullet on the FAQ homepage) is for! Make this your daily routine, for a better FAQ: click on FAQ roulette; read the entry it shows; fix it. --Guido van Rossum (home page: http://www.python.org/~guido/) From neal@metaslash.com Fri May 24 21:50:46 2002 From: neal@metaslash.com (Neal Norwitz) Date: Fri, 24 May 2002 16:50:46 -0400 Subject: [Python-Dev] Out of date FAQ entries References: <3CEC52E2.962B1CB6@metaslash.com> <200205242008.g4OK8hi11791@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CEEA7A6.BFB21C87@metaslash.com> Guido van Rossum wrote: > I updated the ones you mentione, except: > > > 6.24. Why no class methods or mutable class variables? > > Mention classmethod(), staticmethod()? > > This already mentions those (at the bottom). Oops, sorry about that. There are a few more: 4.28 & 4.93 seem like they should be merged. Both talk about compiling Python applications to a stand-alone program. Also, I updated 4.2, which should be reviewed to make sure it's accurate? The title is: Can I create an object class with some methods implemented in C and others in Python (e.g. through inheritance)? (Also phrased as: Can I use a built-in type as base class?) Neal From DavidA@ActiveState.com Fri May 24 22:10:37 2002 From: DavidA@ActiveState.com (David Ascher) Date: Fri, 24 May 2002 14:10:37 -0700 Subject: [Python-Dev] to string or not to string... Message-ID: <3CEEAC4D.4090901@ActiveState.com> Recent checkins mean that: E:\CVS\python\dist\src\PCbuild>python Python 2.3a0 (#29, May 24 2002, 13:47:58) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> isinstance('foo', string) True >>> import string >>> isinstance('foo', string) Traceback (most recent call last): File "", line 1, in ? TypeError: isinstance() arg 2 must be a class or type >>> I realize that there isn't a backwards compatibility problem per se, but there is something bothersome about the above, I think. I also am not to keen on having to keep "str" and "string" separate, but I don't have any better suggestion for the interface name (for that's what "string" is). Do we want string to be both the module and the interface masquerading as a type? --da [Trent gets the credit for asking the question -- I'm just the tester =)] From guido@python.org Fri May 24 22:20:03 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 24 May 2002 17:20:03 -0400 Subject: [Python-Dev] to string or not to string... In-Reply-To: Your message of "Fri, 24 May 2002 14:10:37 PDT." <3CEEAC4D.4090901@ActiveState.com> References: <3CEEAC4D.4090901@ActiveState.com> Message-ID: <200205242120.g4OLK3W17093@pcp742651pcs.reston01.va.comcast.net> > Recent checkins mean that: > > E:\CVS\python\dist\src\PCbuild>python > Python 2.3a0 (#29, May 24 2002, 13:47:58) [MSC 32 bit (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> isinstance('foo', string) > True > >>> import string > >>> isinstance('foo', string) > Traceback (most recent call last): > File "", line 1, in ? > TypeError: isinstance() arg 2 must be a class or type > >>> > > I realize that there isn't a backwards compatibility problem per se, but > there is something bothersome about the above, I think. I also am not > to keen on having to keep "str" and "string" separate, but I don't have > any better suggestion for the interface name (for that's what "string" is). 'str' and 'string' have to be separate because the whole point is that isinstance(u"", string) returns True. We could rename string to basestring? > Do we want string to be both the module and the interface > masquerading as a type? That would *almost* work (string.lower could be an unbound method that calls the corresponding real class method) except string.join() would have the arguments reversed. :-( > [Trent gets the credit for asking the question -- I'm just the tester =)] Trent needn't be so shy. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From DavidA@ActiveState.com Fri May 24 22:22:54 2002 From: DavidA@ActiveState.com (David Ascher) Date: Fri, 24 May 2002 14:22:54 -0700 Subject: [Python-Dev] to string or not to string... References: <3CEEAC4D.4090901@ActiveState.com> <200205242120.g4OLK3W17093@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CEEAF2E.8010606@ActiveState.com> Guido van Rossum wrote: >'str' and 'string' have to be separate because the whole point is that >isinstance(u"", string) returns True. > I understand that. I was just expressing that the names are awfully semantically close for the newbie. >We could rename string to basestring? > That'd be better, IMO -- or even "strings", "anystring" or "string_interface" . >>Do we want string to be both the module and the interface >>masquerading as a type? >> >> >That would *almost* work (string.lower could be an unbound method that >calls the corresponding real class method) except string.join() would >have the arguments reversed. :-( > Cute. >>[Trent gets the credit for asking the question -- I'm just the tester =)] >> >> > >Trent needn't be so shy. :-) > > Trent wasn't shy, he was just too lazy to update and rebuild to test, while I have paperwork to do so anything is more interesting. --da From barry@zope.com Fri May 24 22:26:15 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 24 May 2002 17:26:15 -0400 Subject: [Python-Dev] to string or not to string... References: <3CEEAC4D.4090901@ActiveState.com> <200205242120.g4OLK3W17093@pcp742651pcs.reston01.va.comcast.net> <3CEEAF2E.8010606@ActiveState.com> Message-ID: <15598.45047.525645.571623@anthem.wooz.org> >>>>> "DA" == David Ascher writes: DA> >We could rename string to basestring? > DA> That'd be better, IMO -- or even "strings", "anystring" or DA> "string_interface" . +1 for strangthang :) From guido@python.org Fri May 24 23:17:37 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 24 May 2002 18:17:37 -0400 Subject: [Python-Dev] Python 2.3 release schedule Message-ID: <200205242217.g4OMHbu25323@pcp742651pcs.reston01.va.comcast.net> The Python 2.3 release schedule as laid out in PEP 283 is unrealistic. It has May 22 (yesterday) as the release date for 2.3a1, and then promises successive releases at approximately 4-week intervals. I think we're nowhere near doing the first alpha release. PythonLabs has been consumed by high priority Zope projects. There's a little less pressure now, and we can go back to thinking about Python releases, but we've lost a lot of time. We haven't even appointed a release manager! (Or the appointment was never recorded in the PEP.) In the discussion about release stability, it was brought up that the release schedule should be driven by feature completeness, not by the calendar. I basically agree, although I believe that when it takes the responsible party for a particular feature forever to implement it, it's better to move the feature to future release than to hold up the release forever. So let's discuss features for 2.3. PEP 283 mentions these: > PEP 266 Optimizing Global Variable/Attribute Access Montanaro > PEP 267 Optimized Access to Module Namespaces Hylton > PEP 280 Optimizing access to globals van Rossum (These are basically three friendly competing proposals.) Jeremy has made a little progress with a new compiler, but it's going slow and the compiler is only the first step. Maybe we'll be able to refactor the compiler in this release. I'm tempted to say we won't hold our breath for this one. > PEP 269 Pgen Module for Python Riehl I haven't heard from Jon Riehl, so I consider dropping this idea. > PEP 273 Import Modules from Zip Archives Ahlstrom I think this is close -- maybe it's already checked in and I don't know about it! > PEP 282 A Logging System Mick Vinay Sajip has been making steady progress on an implementation, and despite a recent near-flamewar (which I haven't followed) I expect that his code will be incorporated in the near future. That's it as far as what PEP 283 mentions (there's also a bunch of things already implemented, like bool, PyMalloc, universal newlines). Here are some ideas of my own for things I expect to see in 2.3a1: - Provide alternatives for common uses of the types module; essentially Skip's proto-PEP. - Extended slice notation for all built-in sequences. Wasn't Raymond Hettinger working on this? - Fix the buffer object??? http://mail.python.org/pipermail/python-dev/2002-May/023896.html - Lazily tracking tuples? Neil, how's that coming? http://mail.python.org/pipermail/python-dev/2002-May/023926.html - Timeoutsocket. Work in progress. http://mail.python.org/pipermail/python-dev/2002-May/024077.html - Making None a keyword. Can't be done right away, but a warning would be a first step. http://mail.python.org/pipermail/python-dev/2002-April/023600.html - Stage 2 of the int/long integration (PEP 237). This mostly means warning about situations where hex, oct or shift of an int returns a different value than for the same value as a long. (I think the PEP misses this step, but it's necessary -- we can't just change the semantics silently without warning first.) - I think Andrew Kuchling has a bunch of distutils features planned; how's that coming? - PEP 286??? (MvL: Enhanced argument tuples.) I haven't had the time to review this thoroughly. It seems a deep optimization hack (also makes better correctness guarantees though). - A standard datetime type. An implementation effort is under way: http://www.zope.org/Members/fdrake/DateTimeWiki/FrontPage Effbot and MAL have a proposal for a basic interface that all datetime types should implement, but there are some problems with UTC. A decision needs to be made. Anything else? --Guido van Rossum (home page: http://www.python.org/~guido/) From ping@zesty.ca Fri May 24 23:24:33 2002 From: ping@zesty.ca (Ka-Ping Yee) Date: Fri, 24 May 2002 17:24:33 -0500 (CDT) Subject: [Python-Dev] Optimizing variable access In-Reply-To: <200205242217.g4OMHbu25323@pcp742651pcs.reston01.va.comcast.net> Message-ID: On Fri, 24 May 2002, Guido van Rossum wrote: > So let's discuss features for 2.3. PEP 283 mentions these: > > > PEP 266 Optimizing Global Variable/Attribute Access Montanaro > > PEP 267 Optimized Access to Module Namespaces Hylton > > PEP 280 Optimizing access to globals van Rossum > > (These are basically three friendly competing proposals.) The last thing i remember from this discussion was spending a lot of time trying to understand everyone's implementation schemes and drawing little pictures (http://web.lfw.org/python/globals.html) to describe them. I never heard anything back about them, and the discussion on this topic went suddenly silent after that. Was there a correlation (i hope not) or did everyone just get busy with other things? I was hoping that the figures would help us compare and evaluate the various schemes. -- ?!ng From barry@zope.com Sat May 25 00:05:29 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 24 May 2002 19:05:29 -0400 Subject: [Python-Dev] to string or not to string... References: <3CEEAC4D.4090901@ActiveState.com> <200205242120.g4OLK3W17093@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15598.51001.363649.407830@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> We could rename string to basestring? Or (more seriously than my last suggestion): strings ? -Barry From nas@python.ca Sat May 25 00:20:34 2002 From: nas@python.ca (Neil Schemenauer) Date: Fri, 24 May 2002 16:20:34 -0700 Subject: [Python-Dev] Python 2.3 release schedule In-Reply-To: <200205242217.g4OMHbu25323@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Fri, May 24, 2002 at 06:17:37PM -0400 References: <200205242217.g4OMHbu25323@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020524162034.A26915@glacier.arctrix.com> Guido van Rossum wrote: > - Lazily tracking tuples? Neil, how's that coming? > http://mail.python.org/pipermail/python-dev/2002-May/023926.html http://www.python.org/sf/558745 Neil From jepler@unpythonic.net Sat May 25 00:21:02 2002 From: jepler@unpythonic.net (jepler@unpythonic.net) Date: Fri, 24 May 2002 18:21:02 -0500 Subject: PEP269 -- Pgen module for Python (was Re: [Python-Dev] Python 2.3 release schedule) In-Reply-To: <200205242217.g4OMHbu25323@pcp742651pcs.reston01.va.comcast.net> References: <200205242217.g4OMHbu25323@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020524182102.A11431@unpythonic.net> On Fri, May 24, 2002 at 06:17:37PM -0400, Guido van Rossum wrote: > > PEP 269 Pgen Module for Python Riehl > > I haven't heard from Jon Riehl, so I consider dropping this idea. I've also done some similar work. I ran into problems when trying to deallocate parser objects, and my interface is nothing like the one in the PEP. If there's interest, I might look at this again in the next few weeks. Jeff From tismer@tismer.com Sat May 25 01:01:13 2002 From: tismer@tismer.com (Christian Tismer) Date: Fri, 24 May 2002 17:01:13 -0700 Subject: [Python-Dev] _tkinter problem with Stackless Message-ID: <3CEED449.5050604@tismer.com> Hi friends, I was trying to test my bdb.py patch with Idle and saw weird behavior under Stackless: When I close a window, Python crashes. After a while I recognized that this is due to my moving-the-stack-away at every 8th recursion level. As a quick hack, I changed the threshold to 16 and everything works just fine, but anyway: I believe that there must be a global data structure for Tcl/Tk living on the C stack, between a couple of Python interpreter incarnations, that is used somehow. This is unsane IMHO. Nothing on the C stack should be used as a globally accessible variable. Instead I'd like to stuff this into a structure on the heap, which is safe. Can anybody please give me a hint where in _tkinter or tkappinit this might be? I really don't understand the code, but I'd like to provide a patch. many thanks in advance - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From tim.one@comcast.net Sat May 25 01:15:03 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 24 May 2002 20:15:03 -0400 Subject: [Python-Dev] Lazily GC tracking tuples In-Reply-To: Message-ID: [Kevin Jacobs] > Well, I've found one case where the garbage collector _does_ cause > serious performance problems, and coincidentally enough it is due to > the creation of zillions of cycle-less tuples. Our current solution has > been to disable GC in some parts of our code, and then manually trigger > collection at the correct points. When I can find some free time, I will > definitly test drive any available tuple untracking patches to see if > they make any substantial difference in our code. Well, I'm told it's a 3-day weekend , and Neil has a short patch you can try: http://www.python.org/sf/558745 If you can, please do. I view this as a diasaster-insurance change, not as a generally applicable speedup (it's almost certainly not, and especially not when using a distinct pass to weed out tuples -- I'd combine it with the copy-refcounts pass), so this lives or dies on whether it helps people suffering pathologies in real life. That's you. yes-you-really-are-the-center-of-the-universe-ly y'rs - tim From guido@python.org Sat May 25 01:24:07 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 24 May 2002 20:24:07 -0400 Subject: [Python-Dev] Allowing Unicode literals even without Unicode support? In-Reply-To: Your message of "Fri, 24 May 2002 21:32:50 +0200." <011701c20359$cbc6c7d0$0900a8c0@spiff> References: <200205241825.g4OIP2b10779@pcp742651pcs.reston01.va.comcast.net> <011701c20359$cbc6c7d0$0900a8c0@spiff> Message-ID: <200205250024.g4P0O7h25457@pcp742651pcs.reston01.va.comcast.net> > I'm not sure having to write unicode("...") instead of u"..." > qualifies as "hard", but life would be a bit easier if we didn't > have to... Life's hard enough without artificial discomfort. > > Should we perhaps silently interpret Unicode literals as regular string > > literals when compiling without Unicode support? > > +1.0 on silently accepting ASCII-only u-literals also in non- > unicode builds. > > -0.2 on silently accepting non-ASCII u-literals (your patch > didn't deal with that, right?). No, all I did was skip the 'u'. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Sat May 25 01:27:17 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 24 May 2002 20:27:17 -0400 Subject: PEP269 -- Pgen module for Python (was Re: [Python-Dev] Python 2.3 release schedule) In-Reply-To: Your message of "Fri, 24 May 2002 18:21:02 CDT." <20020524182102.A11431@unpythonic.net> References: <200205242217.g4OMHbu25323@pcp742651pcs.reston01.va.comcast.net> <20020524182102.A11431@unpythonic.net> Message-ID: <200205250027.g4P0RHv25484@pcp742651pcs.reston01.va.comcast.net> > I've also done some similar work. I ran into problems when trying > to deallocate parser objects, and my interface is nothing like the > one in the PEP. If there's interest, I might look at this again in > the next few weeks. I don't know if there's interest. You might ask on c.l.py. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Sat May 25 01:41:23 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 24 May 2002 20:41:23 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib types.py,1.26,1.27 In-Reply-To: <15597.23868.169344.420730@12-248-41-177.client.attbi.com> Message-ID: [Skip] > Attached is the start of a PEP (regarding which I welcome all inputs, > even cranky ones by people suffering without air conditioning). That's the spirit! A correction: Symbol in types module Symbol in builtins ---------------------- ------------------ ComplexType complex DictType, DictionaryType dictionary The last is actually "dict", not "dictionary". I gotta say I don't see much point to deperecating types.py. Doing so will just annoy people, and fail to improve my life . I see even less point to extending it with additional TMTOWTDI, though. From tim.one@comcast.net Sat May 25 02:25:26 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 24 May 2002 21:25:26 -0400 Subject: [Python-Dev] Optimizing variable access In-Reply-To: Message-ID: [Ka-Ping Yee, on various vrbl-access speedup schemes] > The last thing i remember from this discussion was spending a lot > of time trying to understand everyone's implementation schemes and > drawing little pictures (http://web.lfw.org/python/globals.html) > to describe them. > > I never heard anything back about them, Do you read Python-Dev? Lots of people commented appreciatively on them. > and the discussion on this topic went suddenly silent after that. Was > there a correlation (i hope not) or did everyone just get busy with > other things? The Python conference happened then, and that took everyone (including you) away for a week. Perhaps during the conference people realized that implementing one of these things would actually require work. > I was hoping that the figures would help us compare and evaluate the > various schemes. I'm not sure the figures can measure runtime , but they certainly helped people *understand* the proposals. From tim.one@comcast.net Sat May 25 02:25:26 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 24 May 2002 21:25:26 -0400 Subject: [Python-Dev] to string or not to string... In-Reply-To: <15598.51001.363649.407830@anthem.wooz.org> Message-ID: >> We could rename string to basestring? > Or (more seriously than my last suggestion): strings Less than half joking, but the amount by which it's less may be negative: the new "string" is an abstract base class, which is pretty much the same as an interface. So call it istring now, and people will be subtly conditioned to accept interfaces when they're introduced for real <0.5+-epsilon wink>. From martin@v.loewis.de Sat May 25 09:39:07 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 25 May 2002 10:39:07 +0200 Subject: [Python-Dev] _tkinter problem with Stackless In-Reply-To: <3CEED449.5050604@tismer.com> References: <3CEED449.5050604@tismer.com> Message-ID: Christian Tismer writes: > I believe that there must be a global data structure > for Tcl/Tk living on the C stack, between a couple of > Python interpreter incarnations, that is used somehow. The following functions in Tcl use non-trivial "objects" on the stack which are passed to other functions: - Sleep (struct timeval) - Merge (argvStore, fvStore) - Tkapp_Call (objStore / argvStore,fvStore) Of those, only the Tkapp_Call one has a chance of still being in use when the Python interpreter proper is invoked. However, and more importantly, there is a good chance that Tcl itself also allocates memory on the stack for use in called functions. In a Python -> Tcl -> Python -> Tcl scenario, this may cause problems for stackless Python; one would have to inspect the entire Tcl source base, or analyse the specific problem in more detail to be sure. Regards, Martin From oren-py-d@hishome.net Sat May 25 11:43:58 2002 From: oren-py-d@hishome.net (Oren Tirosh) Date: Sat, 25 May 2002 06:43:58 -0400 Subject: [Python-Dev] Python 2.3 release schedule In-Reply-To: <200205242217.g4OMHbu25323@pcp742651pcs.reston01.va.comcast.net> References: <200205242217.g4OMHbu25323@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020525104358.GA73807@hishome.net> On Fri, May 24, 2002 at 06:17:37PM -0400, Guido van Rossum wrote: > > PEP 266 Optimizing Global Variable/Attribute Access Montanaro > > PEP 267 Optimized Access to Module Namespaces Hylton > > PEP 280 Optimizing access to globals van Rossum My fastnames patch achieves much of what these PEPs are aiming for with minimal changes to the interpreter: http://tothink.com/python/fastnames Oren From guido@python.org Sat May 25 13:17:47 2002 From: guido@python.org (Guido van Rossum) Date: Sat, 25 May 2002 08:17:47 -0400 Subject: [Python-Dev] Python 2.3 release schedule In-Reply-To: Your message of "Sat, 25 May 2002 06:43:58 EDT." <20020525104358.GA73807@hishome.net> References: <200205242217.g4OMHbu25323@pcp742651pcs.reston01.va.comcast.net> <20020525104358.GA73807@hishome.net> Message-ID: <200205251217.g4PCHmh26695@pcp742651pcs.reston01.va.comcast.net> [Oren Tirosh] > My fastnames patch achieves much of what these PEPs are aiming for with > minimal changes to the interpreter: http://tothink.com/python/fastnames Mm, interesting. I must've missed this post in the busy days after the conference. :-( Two observations: - Your benchmarks use an almost empty dict (for locals and globals at least). I'd like to see how they perform with a realistic number of other names in the dict - I'm worried that negative dict entries could fill the dict beyond its capacity. --Guido van Rossum (home page: http://www.python.org/~guido/) From oren-py-d@hishome.net Sat May 25 14:34:39 2002 From: oren-py-d@hishome.net (Oren Tirosh) Date: Sat, 25 May 2002 16:34:39 +0300 Subject: [Python-Dev] Python 2.3 release schedule In-Reply-To: <200205251217.g4PCHmh26695@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Sat, May 25, 2002 at 08:17:47AM -0400 References: <200205242217.g4OMHbu25323@pcp742651pcs.reston01.va.comcast.net> <20020525104358.GA73807@hishome.net> <200205251217.g4PCHmh26695@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020525163439.A11908@hishome.net> On Sat, May 25, 2002 at 08:17:47AM -0400, Guido van Rossum wrote: > Two observations: > > - Your benchmarks use an almost empty dict (for locals and globals at > least). I'd like to see how they perform with a realistic number of > other names in the dict True, the fastest code path (the macro) only works as long as the entry is in the first hash position. For the tests in my posting this is 100% of the cases. For real code the results are around 75%. To enable the display of dictionary statistics on exit compile with -dSHOW_DICTIONARY_STATS. One problem is that 75% is still not good enough - the fallback code path is significantly slower (although still faster than PyDict_GetItem). Another problem is that if you get a hash collision for a global symbol used inside a tight loop the hit rate can drop closer to 0%. One trick that may help is to shuffle the hash entries - for every 100th time the macro fails the entry will be moved up to the first hash position and the entry which previously occupied that position will be moved to the first empty hash position for its own hash chain. Statistically, this will ensure that the most commonly referenced names will tend to stay at the first hash position. I think it may improve the hit rate from 75% to 85% or higher and eliminate the worst-case scenario. > - I'm worried that negative dict entries could fill the dict beyond > its capacity. For each dictionary, the number of negative entries is bound by the number of global and builtin names in the co_names of all code objects that get executed with that dictionary as their namespace. For general-purpose dictionaries, of course, there is no such upper bound and therefore this optimization cannot be used in PyDict_GetItem. Oren From guido@python.org Sat May 25 15:11:26 2002 From: guido@python.org (Guido van Rossum) Date: Sat, 25 May 2002 10:11:26 -0400 Subject: [Python-Dev] Python 2.3 release schedule In-Reply-To: Your message of "Sat, 25 May 2002 16:34:39 +0300." <20020525163439.A11908@hishome.net> References: <200205242217.g4OMHbu25323@pcp742651pcs.reston01.va.comcast.net> <20020525104358.GA73807@hishome.net> <200205251217.g4PCHmh26695@pcp742651pcs.reston01.va.comcast.net> <20020525163439.A11908@hishome.net> Message-ID: <200205251411.g4PEBQO27590@pcp742651pcs.reston01.va.comcast.net> > > - Your benchmarks use an almost empty dict (for locals and globals at > > least). I'd like to see how they perform with a realistic number of > > other names in the dict > > True, the fastest code path (the macro) only works as long as the > entry is in the first hash position. For the tests in my posting > this is 100% of the cases. For real code the results are around > 75%. To enable the display of dictionary statistics on exit compile > with -dSHOW_DICTIONARY_STATS. > > One problem is that 75% is still not good enough - the fallback code > path is significantly slower (although still faster than > PyDict_GetItem). Another problem is that if you get a hash > collision for a global symbol used inside a tight loop the hit rate > can drop closer to 0%. > > One trick that may help is to shuffle the hash entries - for every > 100th time the macro fails the entry will be moved up to the first > hash position and the entry which previously occupied that position > will be moved to the first empty hash position for its own hash > chain. Statistically, this will ensure that the most commonly > referenced names will tend to stay at the first hash position. I > think it may improve the hit rate from 75% to 85% or higher and > eliminate the worst-case scenario. Piling more complexity on is likely to slow down the common case though. > > - I'm worried that negative dict entries could fill the dict beyond > > its capacity. > > For each dictionary, the number of negative entries is bound by the > number of global and builtin names in the co_names of all code > objects that get executed with that dictionary as their namespace. And that's my worry. Suppose a program has only one global but touches every builtin. Will the dictionary properly get resized to accommodate all those negative entries? --Guido van Rossum (home page: http://www.python.org/~guido/) From oren-py-d@hishome.net Sat May 25 16:20:21 2002 From: oren-py-d@hishome.net (Oren Tirosh) Date: Sat, 25 May 2002 18:20:21 +0300 Subject: [Python-Dev] Python 2.3 release schedule In-Reply-To: <200205251411.g4PEBQO27590@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Sat, May 25, 2002 at 10:11:26AM -0400 References: <200205242217.g4OMHbu25323@pcp742651pcs.reston01.va.comcast.net> <20020525104358.GA73807@hishome.net> <200205251217.g4PCHmh26695@pcp742651pcs.reston01.va.comcast.net> <20020525163439.A11908@hishome.net> <200205251411.g4PEBQO27590@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020525182021.A13005@hishome.net> On Sat, May 25, 2002 at 10:11:26AM -0400, Guido van Rossum wrote: > > One trick that may help is to shuffle the hash entries - for every > > 100th time the macro fails the entry will be moved up to the first > > hash position and the entry which previously occupied that position > > will be moved to the first empty hash position for its own hash > > chain. Statistically, this will ensure that the most commonly > > referenced names will tend to stay at the first hash position. I > > think it may improve the hit rate from 75% to 85% or higher and > > eliminate the worst-case scenario. > > Piling more complexity on is likely to slow down the common case though. Not at all. The common case (the macro) will not have a single instruction added. The rare case (fallback function) will be slowed down by two instructions: decrement a global variable, jump if zero. The reshuffle will be triggered every 100th occurence of the rare case (which may be every 10000th occurence of the common case). It's the same approach used when designing a memory cache for a CPU: it's OK if a cache miss is made slower by loading an entire cache row as long as it improves the cache hit rate enough to justify it. > > For each dictionary, the number of negative entries is bound by the > > number of global and builtin names in the co_names of all code > > objects that get executed with that dictionary as their namespace. > > And that's my worry. Suppose a program has only one global but > touches every builtin. Will the dictionary properly get resized to > accommodate all those negative entries? >>>len(dir(__builtins__))*24 # sizeof PyDictEntry on i386 2736 Caching of string hashes also costs memory. Speed optimizations often do. These costs need to be weighed against complexity, performance, and even the code size - other alternatives may use up as much code space as this one uses data in this worst-case scenario. Oren From akuchlin@mems-exchange.org Sat May 25 17:14:47 2002 From: akuchlin@mems-exchange.org (akuchlin@mems-exchange.org) Date: Sat, 25 May 2002 12:14:47 -0400 Subject: [Python-Dev] Python 2.3 release schedule In-Reply-To: <200205242217.g4OMHbu25323@pcp742651pcs.reston01.va.comcast.net> References: <200205242217.g4OMHbu25323@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020525161447.GA23907@mems-exchange.org> On Fri, May 24, 2002 at 06:17:37PM -0400, Guido van Rossum wrote: >- I think Andrew Kuchling has a bunch of distutils features planned; > how's that coming? That would be PEP 262, a database of installed Python packages. This May I've been mostly writing, not programming, but will try to get to it ASAP. --amk From guido@python.org Sat May 25 17:41:23 2002 From: guido@python.org (Guido van Rossum) Date: Sat, 25 May 2002 12:41:23 -0400 Subject: [Python-Dev] Python 2.3 release schedule In-Reply-To: Your message of "Sat, 25 May 2002 18:20:21 +0300." <20020525182021.A13005@hishome.net> References: <200205242217.g4OMHbu25323@pcp742651pcs.reston01.va.comcast.net> <20020525104358.GA73807@hishome.net> <200205251217.g4PCHmh26695@pcp742651pcs.reston01.va.comcast.net> <20020525163439.A11908@hishome.net> <200205251411.g4PEBQO27590@pcp742651pcs.reston01.va.comcast.net> <20020525182021.A13005@hishome.net> Message-ID: <200205251641.g4PGfO827801@pcp742651pcs.reston01.va.comcast.net> > > Piling more complexity on is likely to slow down the common case though. > > Not at all. The common case (the macro) will not have a single > instruction added. The rare case (fallback function) will be slowed > down by two instructions: decrement a global variable, jump if zero. > The reshuffle will be triggered every 100th occurence of the rare > case (which may be every 10000th occurence of the common case). > > It's the same approach used when designing a memory cache for a CPU: > it's OK if a cache miss is made slower by loading an entire cache > row as long as it improves the cache hit rate enough to justify it. OK, OK. I timed pystone, and it's about 1.5% faster. If that's the best we can do (and I think that any of the PEPs trying to optimize global and builtin access probably have about the same improvement rate as your code), I'm not sure we should worry about this. I'd rather try to make method calls and instance attribute (both methods and instance variables) access faster... > > > For each dictionary, the number of negative entries is bound by the > > > number of global and builtin names in the co_names of all code > > > objects that get executed with that dictionary as their namespace. > > > > And that's my worry. Suppose a program has only one global but > > touches every builtin. Will the dictionary properly get resized to > > accommodate all those negative entries? > > >>>len(dir(__builtins__))*24 # sizeof PyDictEntry on i386 > 2736 > > Caching of string hashes also costs memory. Speed optimizations > often do. These costs need to be weighed against complexity, > performance, and even the code size - other alternatives may use up > as much code space as this one uses data in this worst-case > scenario. I wasn't worried about size; I was worried about correctness. But it appears that your code is fine. BTW, here's a benchmarking tip. Instead of timing this: for i in xrange(10000000): hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; you should time the for loop in this code: x = [0]*10000000 for i in x: hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; The xrange iterator allocates (and implicitly deallocates) an int per iteration, and you don't use that int at all. (10 million pointers is 40 MB -- if that's too much, it's OK to test with 1 million iterations. Quick outcome: CVS python 2.3: builtin: 8.480 global: 6.780 local: 5.080 fast: 2.890 With your patch: builtin: 4.660 global: 4.160 local: 4.020 fast: 3.000 Hm, this reports: 0 inline dictionary lookups 1568 fast dictionary lookups 121 slow dictionary lookups 0.00% inline 7.16% slow created 308 string dicts converted 32 to normal dicts 10.39% conversion rate Somehow the counts seem to be off by several orders of magnitude. What *do* these numbers report? Also note that your patch slows down fast access (by 3%)! How can it? Adding more code to the interpreter's inner loop changes the cache behavior, etc. Tim Peters can tell you more about this. --Guido van Rossum (home page: http://www.python.org/~guido/) From oren-py-d@hishome.net Sat May 25 18:21:19 2002 From: oren-py-d@hishome.net (Oren Tirosh) Date: Sat, 25 May 2002 13:21:19 -0400 Subject: [Python-Dev] Python 2.3 release schedule In-Reply-To: <200205251641.g4PGfO827801@pcp742651pcs.reston01.va.comcast.net> References: <200205242217.g4OMHbu25323@pcp742651pcs.reston01.va.comcast.net> <20020525104358.GA73807@hishome.net> <200205251217.g4PCHmh26695@pcp742651pcs.reston01.va.comcast.net> <20020525163439.A11908@hishome.net> <200205251411.g4PEBQO27590@pcp742651pcs.reston01.va.comcast.net> <20020525182021.A13005@hishome.net> <200205251641.g4PGfO827801@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020525172119.GA10543@hishome.net> On Sat, May 25, 2002 at 12:41:23PM -0400, Guido van Rossum wrote: > OK, OK. > > I timed pystone, and it's about 1.5% faster. If that's the best we > can do (and I think that any of the PEPs trying to optimize global and > builtin access probably have about the same improvement rate as your > code), I'm not sure we should worry about this. I'd rather try to > make method calls and instance attribute (both methods and instance > variables) access faster... pystone and other benchmarks spend their time in loops using fastlocals. I used to have a version where SHOW_DICTIONARY_STATS also counted fastlocals accesses for reference - they were 97% for pybench. You're right that we need to find if this is really a problem for real-life code. Are there any measurements for the speedup achieved in Zope by the argument/local tricks? With a little more work this dictionary optimization should be applicable to attributes, too. > I wasn't worried about size; I was worried about correctness. But it > appears that your code is fine. It isn't :-) I haven't implemented resizing of the hash table yet. If there is not enough room the negative entry isn't inserted. This bug is documented in my original posting. > Hm, this reports: > 0 inline dictionary lookups It's a rebuild dependency problem. Make sure that ceval.c also gets recompiled after setting SHOW_DICTIONARY_STATS. When collecting stats the inline is not really inline. Sorry about that. > Also note that your patch slows down fast access (by 3%)! How can it? > Adding more code to the interpreter's inner loop changes the cache > behavior, etc. Tim Peters can tell you more about this. In my measurements there was no such slowdown. Is this consistent? Your cache behavior theory reminds me of an algorithm that was sped up by almost 10% when a certain array type was changed from 64 to 67 entries. For modern CPUs the common practice of using powers of two can cause cache thrashing. Oren From mark@freelance-developer.com Sun May 26 00:11:00 2002 From: mark@freelance-developer.com (Mark J. Nenadov) Date: Sat, 25 May 2002 19:11:00 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: References: <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200205251911.01072.mark@freelance-developer.com> On Sunday 07 April 2002 09:38 pm, François Pinard wrote: > I use Python on a few systems and flavours, and Python versions are not the > same on all machines. At places, people ask me to limit myself to 1.5.2. > At home, I have 2.2, but 2.0 is currently the common denominator for the > set of machines on which I do my own projects (I physically travel between > sites). For one, I have no real problem aiming 1.5.2 or 2.0 instead of > 2.2, when projects or teams require it. Despite I much enjoy most of the > new features, I still have to be careful about various development > contexts. I also use 2.0 as the lowest common denominator. Its hard to avoid 1.5.2 (because Red Hat has been shipping with 1.5.x). I can't complain about the differences in Python versions. For the most part my experience has been hassle free. PHP version issues have been more troublesome to me than Python version issues. > The current pace of change does not look unreasonable to me. I wonder if > the perception would not improve, if there was a bit more documentation > or publicity about what users should do about version inter-operability. > If users understand what they choose, they will less likely feel changes > as imposed to them. Maybe! :-) There are some who think preserving familiarity between versions is way more important than incremental improvement. In my opinion, sticking to that very idea is a good way to kill and stagnate a language. I believe that maintaing and improving the language *usually* is more important than maintaining consistency between versions. That being said, going overboard with changes isn't good either. I think documenting these changes would be a step in the right direction. My two cents. Good day, ~Mark From python@rcn.com Sun May 26 06:19:15 2002 From: python@rcn.com (Raymond Hettinger) Date: Sun, 26 May 2002 01:19:15 -0400 Subject: [Python-Dev] Python 2.3 release schedule References: <200205242217.g4OMHbu25323@pcp742651pcs.reston01.va.comcast.net> Message-ID: <001c01c20474$e1a50140$f9d8accf@othello> > - Extended slice notation for all built-in sequences. Wasn't Raymond > Hettinger working on this? Yes! > Anything else? Perhaps an iterator tools module featuring goodies from SML and Haskell: import itertools ia = itertools.iter(open('readme.txt')) ia[4] # nth / index ia[0] # first ia[-1] # last ia[:10] # tail / drop ia[5:] # head / take ia[0,100,2] # slicing ia.enumerate(countfrom=0) ia.filter(pred) # takewhile ia.invfilter(pred) # dropwhile ia.map(func) ia.starmap(func) # [yield func(*args) for args in ia] ia.cycle() # repeats the seqn infinitely (requires aux mem) ia.unzip() # creates multiple iters from one ia.tabulate(func, countFrom=0) # sml: f[0], f[1], ... = ia.map(func, xrange(countFrom,sys.maxint)) itertools.repeat(obj) # while 1: yield obj itertools.zip(i1, i2, i3, ...) Raymond Hettinger From tismer@tismer.com Sun May 26 10:49:40 2002 From: tismer@tismer.com (Christian Tismer) Date: Sun, 26 May 2002 02:49:40 -0700 Subject: [Python-Dev] _tkinter problem with Stackless References: <3CEED449.5050604@tismer.com> Message-ID: <3CF0AFB4.60709@tismer.com> Martin v. Loewis wrote: > Christian Tismer writes: > > >>I believe that there must be a global data structure >>for Tcl/Tk living on the C stack, between a couple of >>Python interpreter incarnations, that is used somehow. > > > The following functions in Tcl use non-trivial "objects" on the stack > which are passed to other functions: > > - Sleep (struct timeval) > - Merge (argvStore, fvStore) > - Tkapp_Call (objStore / argvStore,fvStore) > > Of those, only the Tkapp_Call one has a chance of still being in use > when the Python interpreter proper is invoked. Yes, I saw this and tried some quick hacks to see if I can get around it, but the following is much worse and really hurts: > However, and more importantly, there is a good chance that Tcl itself > also allocates memory on the stack for use in called functions. In a > Python -> Tcl -> Python -> Tcl scenario, this may cause problems for > stackless Python; one would have to inspect the entire Tcl source > base, or analyse the specific problem in more detail to be sure. Incredibly bad! Bad style, even worse, bad for me. I change my question to: What patterns beyond the Python -> Tcl -> Python -> Tcl scenario are thinkable? I already spent 14 hours on this. I'm happy to work around it if there is a finite number of cases. But if this leads to major changes to Tcl, I'd prefer to declare Stackless incompatible with Tcl. It works great with PythonWin and wxPython, which is a plus for design. ciao - chris p.s.: This would be a great chance to submit a PowerPC patch, since I'm very near to a production release. -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From pinard@iro.umontreal.ca Sun May 26 14:25:58 2002 From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard) Date: 26 May 2002 09:25:58 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200205251911.01072.mark@freelance-developer.com> References: <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> <200205251911.01072.mark@freelance-developer.com> Message-ID: [Mark J. Nenadov] > I also use 2.0 as the lowest common denominator. Linguistic problem :-). Should we say "greatest" instead of "lowest"? Granted that the greatest common denominator is not "greatest" in any other way, but it is lower or equal than any of the things we consider. The real "lowest" common denominator might be very close to nothing, might it not? > I believe that maintaing and improving the language *usually* is more > important than maintaining consistency between versions. That being said, > going overboard with changes isn't good either. I think documenting > these changes would be a step in the right direction. Exactly. The changes _are_ documented in detail, indeed, but in special documents which only serious users read when they about to migrate from one version to another. I'm thinking about users who use the language occasionally or even regularly, but not fanatic about following everything about versions -- they mainly rely on the Python Library Reference, or even the Python Language Reference. These references describe some Python, but not necessarily the Python which happens to run on a given machine, and I guess (without having really experienced this myself) it might be frustrating to read and study, for discovering soon after that the feature is unavailable in this version. Or even, for someone, to have handy information about how to write for the common denominator, without having to compare many printings of the references at various Python levels. I guess that notes or footnotes, about Python levels in which described features have been implemented, might help users having to cope with release lags between Linux releases. Such lags are unavoidable whenever Python evolves. I understand the effort it would require to add and maintain such notes in the references (especially for the Library Reference which has myriad of details, the Language Reference is more clearly cut), my hope is that this might remove some of the irritation people seem to show. -- François Pinard http://www.iro.umontreal.ca/~pinard From mwh@python.net Sun May 26 18:04:09 2002 From: mwh@python.net (Michael Hudson) Date: 26 May 2002 18:04:09 +0100 Subject: [Python-Dev] Python 2.3 release schedule In-Reply-To: Guido van Rossum's message of "Fri, 24 May 2002 18:17:37 -0400" References: <200205242217.g4OMHbu25323@pcp742651pcs.reston01.va.comcast.net> Message-ID: <2mr8jyx2ly.fsf@starship.python.net> Guido van Rossum writes: > Anything else? Writing a pymemcompat.h that people can bundle with their extensions and then use the 2.3 memory interface with all Pythons in the range 1.5.2 to 2.3. I said I'd do this once the interface is settled, so I'll try to get to it in the next week or so. It shouldn't be *that* hard... Cheers, M. -- I never disputed the Perl hacking skill of the Slashdot creators. My objections are to the editors' taste, the site's ugly visual design, and the Slashdot community's raging stupidity. -- http://www.cs.washington.edu/homes/klee/misc/slashdot.html#faq From tim.one@comcast.net Sun May 26 18:17:29 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 26 May 2002 13:17:29 -0400 Subject: [Python-Dev] Python 2.3 release schedule In-Reply-To: <200205242217.g4OMHbu25323@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > ... > - Stage 2 of the int/long integration (PEP 237). This mostly means > warning about situations where hex, oct or shift of an int returns a > different value than for the same value as a long. (I think the PEP > misses this step, but it's necessary -- we can't just change the > semantics silently without warning first.) It's unclear there whether you intend that a warning in 2.3 be coupled with (a) delivering the 2.2 result, or (b) delivering the new (auto-overflow) result. If a _future statement is introduced for this (incompatible changes are what __future__ is there for -- alas), #a is the obvious answer. > ... > Anything else? Nuking SET_LINENO remains the conceptually clearest way to get a >5% boost in pystone. As other high-frequency paths in the interpreter have gotten leaner, the speed benefit of -O has gotten larger. From mwh@python.net Sun May 26 19:11:21 2002 From: mwh@python.net (Michael Hudson) Date: 26 May 2002 19:11:21 +0100 Subject: [Python-Dev] Allowing Unicode literals even without Unicode support? In-Reply-To: Guido van Rossum's message of "Fri, 24 May 2002 14:25:02 -0400" References: <200205241825.g4OIP2b10779@pcp742651pcs.reston01.va.comcast.net> Message-ID: <2mptziwzhy.fsf@starship.python.net> Guido van Rossum writes: > Many test modules fail because they use Unicode literals. It's easy > enough to skip Unicode-related tests (just test for the existence of > 'unicode'), but it's hard to avoid having Unicode literals in the text > at all. Should we perhaps silently interpret Unicode literals as > regular string literals when compiling without Unicode support? Hmm. It would be nice to have less tests fail in --disable-unicode builds (there were real bugs hiding in there!). But I'm not sure that accepting u'' literals is wise, generally speaking, just because of the confusion it might cause. Perhaps a commandline flag? Anyway, I've been meaning to post here about --disable-unicode builds for a while, so here's some issues that currently exist. 1) In --disable-unicode builds, it seems that much of the codec machinery just doesn't exist. While this was fine once, we now have, e.g. >>> "sheesh".encode("rot-13") 'furrfu' But in a disable unicode build: >>> "sheesh".encode("rot-13") Traceback (most recent call last): File "", line 1, in ? File "/home/mwh/src/python/dist/src/Lib/encodings/rot_13.py", line 18, in encode return codecs.charmap_encode(input,errors,encoding_map) AttributeError: 'module' object has no attribute 'charmap_encode' This is why test_string still fails. I'm not sure what to do about this. 2) Some library modules just don't seem to function without unicode. Current bad boys seem to include gettext, urllib (well, this just got fixed I think), minidom, pyexpat and the email package. I could imagine the xml modules and the email package just plain not being supported w/o unicode. gettext, too, I guess. In which case: 3) regrtest should probably be taught not to run test_unicode + close friends in --disable-unicode builds. Anyone feel like digging into any of the above? Cheers, M. -- You're posting to a Scheme group. Around here, arguing that Java is better than C++ is like arguing that grasshoppers taste better than tree bark. -- Thant Tessman, comp.lang.scheme From tim.one@comcast.net Sun May 26 20:01:00 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 26 May 2002 15:01:00 -0400 Subject: [Python-Dev] Python 2.3 release schedule In-Reply-To: <20020525104358.GA73807@hishome.net> Message-ID: Oren, I want to make sure we're measuring something useful here. Your timing programs for "global" and "builtin" access all go through LOAD_NAME, which is rarely interesting -- except in "who cares?" (class bodies) and pathological (some kinds of exec/import* contexts) cases, global and builtin access go through LOAD_GLOBAL instead. On my box, that makes a huge difference. For example, accessing a global via LOAD_GLOBAL is measurably faster than accessing a local via LOAD_NAME here; LOAD_NAME isn't written for speed (it does a test+taken-branch first thing in every non-system-error case; LOAD_GLOBAL doesn't). Attached is a self-contained timing program that times all these ways, and addresses Guido's concerns about timing methodology. A typical run on my box displays: fastlocals 2.40 globals_via_load_global 3.27 (*) locals_via_load_name 3.46 globals_via_load_name 4.56 builtins_via_load_global 5.99 (*) builtins_via_load_name 7.12 (*) The code at doesn't time these common cases. Cross-run timing variance on a quiet machine here is +/-2 in the last digit (< 1%). Timings were done under a current CVS release build, not using Python's -O switch. from time import clock as now indices = [None] * 1000000 def display(name, time): print "%-25s %5.2f" % (name, time) def fastlocals(): hex = 5 for i in indices: hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; def locals_via_load_name(): class f: hex = 5 for i in indices: hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; heh = 5 def globals_via_load_name(): class f: for i in indices: heh ; heh ; heh ; heh ; heh ; heh ; heh ; heh ; heh ; heh ; heh ; heh ; heh ; heh ; heh ; heh ; heh ; heh ; heh ; heh ; def globals_via_load_global(): for i in indices: heh ; heh ; heh ; heh ; heh ; heh ; heh ; heh ; heh ; heh ; heh ; heh ; heh ; heh ; heh ; heh ; heh ; heh ; heh ; heh ; def builtins_via_load_name(): class f: for i in indices: hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; def builtins_via_load_global(): for i in indices: hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; hex ; for f in (fastlocals, globals_via_load_global, locals_via_load_name, globals_via_load_name, builtins_via_load_global, builtins_via_load_name): start = now() f() elapsed = now() - start display(f.__name__, elapsed) From tim.one@comcast.net Sun May 26 20:04:53 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 26 May 2002 15:04:53 -0400 Subject: [Python-Dev] Allowing Unicode literals even without Unicode support? In-Reply-To: <2mptziwzhy.fsf@starship.python.net> Message-ID: [Michael Hudson, on the joys of --disable-unicode] > ... > Anyone feel like digging into any of the above? Not me. I'm not even clear on why we want to support --disable-unicode. We believe Unicode is the future or we don't. talk-about-loaded-statements-ly y'rs - tim From tim.one@comcast.net Sun May 26 20:27:29 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 26 May 2002 15:27:29 -0400 Subject: [Python-Dev] Python 2.3 release schedule In-Reply-To: <200205251641.g4PGfO827801@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido, to Oren Tirosh] > ... > Also note that your patch slows down fast access (by 3%)! How can it? > Adding more code to the interpreter's inner loop changes the cache > behavior, etc. Tim Peters can tell you more about this. Not unless someone pays me to . Here's a cute one: I changed the "break" at the end of LOAD_GLOBAL to "continue". It didn't change the speed of global-lookup tests at all, but did give a small boost to fastlocal lookups. On several occasions we've seen evidence that ceval is supremely sensitive to I-cache accidents, although Marc-Andre is a lot better at provoking them than I am . give-me-two-weeks-of-uninterrupted-time-with-a-hw-simulator-and-you'll- get-a-real-answer-ly y'rs - tim From tim.one@comcast.net Sun May 26 20:38:38 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 26 May 2002 15:38:38 -0400 Subject: [Python-Dev] Python 2.3 release schedule In-Reply-To: <2mr8jyx2ly.fsf@starship.python.net> Message-ID: [Michael Hudson] > Writing a pymemcompat.h that people can bundle with their extensions > and then use the 2.3 memory interface with all Pythons in the range > 1.5.2 to 2.3. I said I'd do this once the interface is settled, so > I'll try to get to it in the next week or so. It shouldn't be *that* > hard... I think that's a great idea! Feel free to commandeer as much time from MAL and /F as you want, since they'll whine if it isn't done . From Jack.Jansen@oratrix.com Sun May 26 23:31:56 2002 From: Jack.Jansen@oratrix.com (Jack Jansen) Date: Mon, 27 May 2002 00:31:56 +0200 Subject: [Python-Dev] _tkinter problem with Stackless In-Reply-To: <3CF0AFB4.60709@tismer.com> Message-ID: <62B17C40-70F8-11D6-8525-003065517236@oratrix.com> On zondag, mei 26, 2002, at 11:49 , Christian Tismer wrote: > >> However, and more importantly, there is a good chance that Tcl itself >> also allocates memory on the stack for use in called functions. In a >> Python -> Tcl -> Python -> Tcl scenario, this may cause problems for >> stackless Python; one would have to inspect the entire Tcl source >> base, or analyse the specific problem in more detail to be sure. > > Incredibly bad! Bad style, even worse, bad for me. > I change my question to: > What patterns beyond the > Python -> Tcl -> Python -> Tcl > scenario are thinkable? > I already spent 14 hours on this. I'm happy > to work around it if there is a finite > number of cases. But if this leads to major > changes to Tcl, I'd prefer to declare Stackless > incompatible with Tcl. It works great with PythonWin > and wxPython, which is a plus for design. Does your assertion "it works great with PythonWin and wxPython" mean that neither of those can have the situation where you go from Python -> Windows -> Python -> Windows (or replace Windows with wx) while you have a reference to the first Python stack passed to the first Windows call, which is then subsequently used by the second Windows call? Or does it mean something "lighter"? Because if you indeed mean what I guess here then I can't be sure that the Mac toolbox modules are safe either, at least not without serious inspection. A lot of structures are allocated on the stack in the glue routines, and a lot of toolbox modules may do callbacks. Or does your "works great" merely mean that you haven't run into any problems yet? -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From tismer@tismer.com Mon May 27 00:42:59 2002 From: tismer@tismer.com (Christian Tismer) Date: Sun, 26 May 2002 16:42:59 -0700 Subject: [Python-Dev] _tkinter problem with Stackless References: <62B17C40-70F8-11D6-8525-003065517236@oratrix.com> Message-ID: <3CF17303.6030107@tismer.com> Jack Jansen wrote: > > On zondag, mei 26, 2002, at 11:49 , Christian Tismer wrote: [Tcl w. STackless probs] > Does your assertion "it works great with PythonWin and wxPython" mean > that neither of those can have the situation where you go from Python -> > Windows -> Python -> Windows (or replace Windows with wx) while you have > a reference to the first Python stack passed to the first Windows call, > which is then subsequently used by the second Windows call? Or does it > mean something "lighter"? AFAICT, neither wx, nor PyWin do put anything on the stack between Python calls. For PythonWin, I'm pretty sure since otherwise the olde stackless would have blocked multitasking, which it didn't. It blocked on every real interpreter recursion which was not avoidable. There was so such thing. For that reason, I was happily implementing the current scheme, assuming this to be a general rule. Well, I remember that in Idle, I had to start scheduling *after* all GUI stuff had been loaded, so I should have been warned. With wx, I'm not yet completely sure. What I used was the interactive shell of the Boa constructor environment. This implementation appears to be super-clean. My tasklets run in the GUI shell, and if some are left, they will run after destruction of the GUI, in the DOS window, as if nothing happened. But I didn't try my own wxPython apps, yet, and for sure I will need to do some locking stuff, but hopefully not tricking thes stack-lockups. > Because if you indeed mean what I guess here then I can't be sure that > the Mac toolbox modules are safe either, at least not without serious > inspection. A lot of structures are allocated on the stack in the glue > routines, and a lot of toolbox modules may do callbacks. Bad news, makes me nervous. I see myself building kind of registry of contexts which are forbidden to switch. Structure which are kept on the stack as globals during the lifetime of some calls, even through a sequence of multiple python calls, really give me a headache. On the other hand: The old stackess worked fine with Just's Mac IDE. If there were callbacks through C code, this stackless again would have blocked. That means, although such situations do exist as you say, they are atomic, and blocking is the right way. I think I should think my design over and provide restrictive blocking by default, together with a table of function calls which are known to be safely switched. That means to me that I have to do a full stack analysis for every platform. I started to do this anyway, since I'm quite near to pickleable threads, but I wasn't aware that I have to go this path so early... Au Backe! > Or does your "works great" merely mean that you haven't run into any > problems yet? As said, it works well, but I didn't do all necessary exhaustive testing yet. I'd love to test it on a Mac, but still the PowerPC implementation is incomplete and I can't get at it before I have access to such a machine (maybe in two weeks). ciao - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From greg@cosc.canterbury.ac.nz Mon May 27 02:41:43 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Mon, 27 May 2002 13:41:43 +1200 (NZST) Subject: [Python-Dev] _tkinter problem with Stackless In-Reply-To: <3CF0AFB4.60709@tismer.com> Message-ID: <200205270141.NAA06269@s454.cosc.canterbury.ac.nz> Christian Tismer : > Incredibly bad! Bad style, even worse, bad for me. I wouldn't call it "bad style" in general -- it's perfectly legal to pass the address of a local variable to another function, as long as the other function uses it before returning. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tismer@tismer.com Mon May 27 04:48:53 2002 From: tismer@tismer.com (Christian Tismer) Date: Sun, 26 May 2002 20:48:53 -0700 Subject: [Python-Dev] _tkinter problem with Stackless References: <200205270141.NAA06269@s454.cosc.canterbury.ac.nz> Message-ID: <3CF1ACA5.1050206@tismer.com> Greg Ewing wrote: > Christian Tismer : > > >>Incredibly bad! Bad style, even worse, bad for me. > > > I wouldn't call it "bad style" in general -- it's > perfectly legal to pass the address of a local variable > to another function, as long as the other function uses > it before returning. Maybe, but I'm really fucked up... Saying "bad style" might help to keep people from producing more problems :-) Full stack analysis is really hard at this stage. On the other hand, I think I have a work-around that might work in most cases: Right now, I install kinda "stack root", whenever I see an initial frame, that is one with no f_back. Maybe that's just bad. I saw myself moving 8kb of stack away in the context of Tk. this is obviously a large "installation" on the stack. I might instead enforce the user to start a new "root", manually. This would then no more start automagically, but "by hand", on top of any stack stuff that I better should not touch. what would you suggest? -- ciao - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From tim.one@comcast.net Mon May 27 05:32:31 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 27 May 2002 00:32:31 -0400 Subject: [Python-Dev] Python 2.3 release schedule In-Reply-To: Message-ID: I finally got around to applying Oren's patch. Nice work, BTW! Here are before-and-after timings for the test program I posted before. This is current CVS, release build, Win98SE, MSVC 6, 866MHz: seconds function before after % speedup -------- ------ ----- --------- fastlocals 2.40 2.39 0 globals_via_load_global 3.27 2.53 29 locals_via_load_name 3.46 2.66 30 globals_via_load_name 4.56 2.82 62 builtins_via_load_global 5.99 4.04 48 builtins_via_load_name 7.12 4.84 47 I only care about the XYZ_load_global times (speeding LOAD_NAME is like speeding multiplication by 1531 ), and these numbers are nice. From greg@cosc.canterbury.ac.nz Mon May 27 06:37:23 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Mon, 27 May 2002 17:37:23 +1200 (NZST) Subject: [Python-Dev] _tkinter problem with Stackless In-Reply-To: <3CF1ACA5.1050206@tismer.com> Message-ID: <200205270537.RAA06295@s454.cosc.canterbury.ac.nz> > I might instead enforce the user to start a new > "root", manually. This would then no more start > automagically, but "by hand", on top of any stack > stuff that I better should not touch. > > what would you suggest? -- ciao - chris I don't understand enough about how your system works or what exactly Tk is doing that stuffs it up to be able to recommend anything. Am I right in thinking that a problem can only occur if a tasklet switch occurs between the time the local variable is created and the time it's used? If the same tasklet is running both times, the stack should be back in it's right place. So maybe some sort of restriction on when a switch can occur would help? Not sure what sort, though. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From martin@v.loewis.de Mon May 27 07:19:39 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 27 May 2002 08:19:39 +0200 Subject: [Python-Dev] _tkinter problem with Stackless In-Reply-To: <200205270537.RAA06295@s454.cosc.canterbury.ac.nz> References: <200205270537.RAA06295@s454.cosc.canterbury.ac.nz> Message-ID: Greg Ewing writes: > Am I right in thinking that a problem can only > occur if a tasklet switch occurs between the time > the local variable is created and the time it's > used? If the same tasklet is running both times, > the stack should be back in it's right place. Not necessarily. Christian segments the stack into slices, and only keeps the top-of-stack slice on the stack. So even within a single tasklet, you cannot refer to stack variables of your callers, if those callers are far away. Regards, Martin From martin@v.loewis.de Mon May 27 07:21:59 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 27 May 2002 08:21:59 +0200 Subject: [Python-Dev] Re: Stability and change In-Reply-To: References: <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> <200205251911.01072.mark@freelance-developer.com> Message-ID: pinard@iro.umontreal.ca (Fran=E7ois Pinard) writes: > > I also use 2.0 as the lowest common denominator. >=20 > Linguistic problem :-). Should we say "greatest" instead of "lowest"? > Granted that the greatest common denominator is not "greatest" in any oth= er > way, but it is lower or equal than any of the things we consider. The re= al > "lowest" common denominator might be very close to nothing, might it not? For any two natural numbers, the lowest common denominator is 1. Finding the greatest (largest?) common denominator is indeed what involves an algorithm. Regards, Martin From martin@v.loewis.de Mon May 27 07:28:15 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 27 May 2002 08:28:15 +0200 Subject: [Python-Dev] Allowing Unicode literals even without Unicode support? In-Reply-To: <2mptziwzhy.fsf@starship.python.net> References: <200205241825.g4OIP2b10779@pcp742651pcs.reston01.va.comcast.net> <2mptziwzhy.fsf@starship.python.net> Message-ID: Michael Hudson writes: > I'm not sure what to do about this. I think adding charmap_encode in disable-unicode builds would be the acceptable. > 2) Some library modules just don't seem to function without unicode. > Current bad boys seem to include gettext, urllib (well, this just got > fixed I think), minidom, pyexpat and the email package. I could > imagine the xml modules and the email package just plain not being > supported w/o unicode. gettext, too, I guess. True. Regards, Martin From martin@v.loewis.de Mon May 27 07:34:58 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 27 May 2002 08:34:58 +0200 Subject: [Python-Dev] Allowing Unicode literals even without Unicode support? In-Reply-To: References: Message-ID: Tim Peters writes: > [Michael Hudson, on the joys of --disable-unicode] > > ... > > Anyone feel like digging into any of the above? > > Not me. I'm not even clear on why we want to support --disable-unicode. We > believe Unicode is the future or we don't. People have requested this because they were shocked of the increase in size of the Python interpreter, which primarily originates from the Unicode character database. While I think people would get over their shock as time passes, I think creating a "small" interpreter for special-purpose applications is desirable, and just deleting Unicode support in such a build was (and still is) a low-hanging fruit. That said, I see no reason why the test suite should pass in such a build. I would suggest to let this code rot (as long as you can still build the interpreter without Unicode), and wait for users to complain that features don't work in disable-unicode builds that they think should work. Maybe we can take this feature out in P3k when we find that nobody uses it. Until then, I would propose to leave it roughly at the state of WITHOUT_COMPLEX. Does anybody know whether the test suite passes if you build WITHOUT_COMPLEX? Regards, Martin From mal@lemburg.com Mon May 27 09:39:01 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 27 May 2002 10:39:01 +0200 Subject: [Python-Dev] Allowing Unicode literals even without Unicode support? References: <200205241825.g4OIP2b10779@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CF1F0A5.4040108@lemburg.com> Guido van Rossum wrote: > Many test modules fail because they use Unicode literals. It's easy > enough to skip Unicode-related tests (just test for the existence of > 'unicode'), but it's hard to avoid having Unicode literals in the text > at all. Should we perhaps silently interpret Unicode literals as > regular string literals when compiling without Unicode support? That would be like silently truncating floats to integers. It works as long as x == int(x), but doesn't for fractional values. -0 if you make sure that the literal is plain ASCII. -1 if you don't. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From tismer@tismer.com Mon May 27 09:52:54 2002 From: tismer@tismer.com (Christian Tismer) Date: Mon, 27 May 2002 01:52:54 -0700 Subject: [Python-Dev] _tkinter problem with Stackless References: <200205270537.RAA06295@s454.cosc.canterbury.ac.nz> Message-ID: <3CF1F3E6.7090400@tismer.com> Martin v. Loewis wrote: > Greg Ewing writes: > > >>Am I right in thinking that a problem can only >>occur if a tasklet switch occurs between the time >>the local variable is created and the time it's >>used? If the same tasklet is running both times, >>the stack should be back in it's right place. > > > Not necessarily. Christian segments the stack into slices, and only > keeps the top-of-stack slice on the stack. So even within a single > tasklet, you cannot refer to stack variables of your callers, if those > callers are far away. Right! At the moment it appears to be the only problem. It isn't the tasklet switching (that behaves as Greg suspects), but the slicing code. By setting the slicing recursion interval to above 20, the Tcl stuff works. Maybe I'm just fine by patching _tkinter a little bit, so that it influences stack slicing in a healthy way? cheers - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From mal@lemburg.com Mon May 27 09:58:00 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 27 May 2002 10:58:00 +0200 Subject: [Python-Dev] Allowing Unicode literals even without Unicode support? References: <200205241825.g4OIP2b10779@pcp742651pcs.reston01.va.comcast.net> <2mptziwzhy.fsf@starship.python.net> Message-ID: <3CF1F518.2010808@lemburg.com> Michael Hudson wrote: > Anyway, I've been meaning to post here about --disable-unicode builds > for a while, so here's some issues that currently exist. > > 1) In --disable-unicode builds, it seems that much of the codec > machinery just doesn't exist. While this was fine once, we now have, > e.g. I'm not sure I understand: the charmap codec is built around Unicode so it could never have worked with --disable-unicode. >>>>"sheesh".encode("rot-13") >>> > 'furrfu' > > But in a disable unicode build: > > >>>>"sheesh".encode("rot-13") >>> > Traceback (most recent call last): > File "", line 1, in ? > File "/home/mwh/src/python/dist/src/Lib/encodings/rot_13.py", line 18, in encode > return codecs.charmap_encode(input,errors,encoding_map) > AttributeError: 'module' object has no attribute 'charmap_encode' > > This is why test_string still fails. > > I'm not sure what to do about this. Make the test conditional in test_string ?! -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From mal@lemburg.com Mon May 27 10:01:38 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 27 May 2002 11:01:38 +0200 Subject: [Python-Dev] Allowing Unicode literals even without Unicode support? References: Message-ID: <3CF1F5F2.1080301@lemburg.com> Tim Peters wrote: > [Michael Hudson, on the joys of --disable-unicode] > >>... >>Anyone feel like digging into any of the above? > > > Not me. I'm not even clear on why we want to support --disable-unicode. We > believe Unicode is the future or we don't. I think --disable-unicode is needed by ports to platforms with low memory, e.g. Palms or embedded devices. The latter is a huge market. I don't think that *we* should put much effort into this, though -- let the porters decide which parts they need and then have them rip out the code. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From oren-py-d@hishome.net Mon May 27 10:45:40 2002 From: oren-py-d@hishome.net (Oren Tirosh) Date: Mon, 27 May 2002 12:45:40 +0300 Subject: [Python-Dev] fastnames patch In-Reply-To: ; from tim.one@comcast.net on Mon, May 27, 2002 at 12:32:31AM -0400 References: Message-ID: <20020527124540.A16035@hishome.net> On Mon, May 27, 2002 at 12:32:31AM -0400, Tim Peters wrote: > I finally got around to applying Oren's patch. Nice work, BTW! Here are > before-and-after timings for the test program I posted before. This is > current CVS, release build, Win98SE, MSVC 6, 866MHz: > > seconds > function before after % speedup > -------- ------ ----- --------- > fastlocals 2.40 2.39 0 > globals_via_load_global 3.27 2.53 29 > locals_via_load_name 3.46 2.66 30 > globals_via_load_name 4.56 2.82 62 > builtins_via_load_global 5.99 4.04 48 > builtins_via_load_name 7.12 4.84 47 > Here are before-and-after timings for your test program. Python 2.2.1, Linux, gcc 2.96, 866MHz: function before after % speedup -------- ------ ----- --------- fastlocals 2.63 2.58 2 (?!?) globals_via_load_global 4.04 3.43 18 builtins_via_load_global 7.87 5.64 40 Again, without INLINE_DICT_GETITEM_INTERNED: function before after % speedup -------- ------ ----- --------- fastlocals 2.63 2.59 2 globals_via_load_global 4.04 3.55 14 builtins_via_load_global 7.87 5.66 39 It looks like the inline macro doesn't really account for much the improvement. Oren From guido@python.org Mon May 27 14:35:39 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 27 May 2002 09:35:39 -0400 Subject: [Python-Dev] _tkinter problem with Stackless In-Reply-To: Your message of "Sun, 26 May 2002 20:48:53 PDT." <3CF1ACA5.1050206@tismer.com> References: <200205270141.NAA06269@s454.cosc.canterbury.ac.nz> <3CF1ACA5.1050206@tismer.com> Message-ID: <200205271335.g4RDZdS01966@pcp742651pcs.reston01.va.comcast.net> > > I wouldn't call it "bad style" in general -- it's > > perfectly legal to pass the address of a local variable > > to another function, as long as the other function uses > > it before returning. > > Maybe, but I'm really fucked up... > Saying "bad style" might help to keep people from > producing more problems :-) Cut it out, Christian. Now! This is *your* problem to solve -- not something to complain to others to. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon May 27 15:00:07 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 27 May 2002 10:00:07 -0400 Subject: [Python-Dev] Allowing Unicode literals even without Unicode support? In-Reply-To: Your message of "27 May 2002 08:34:58 +0200." References: Message-ID: <200205271400.g4RE07N02102@pcp742651pcs.reston01.va.comcast.net> > People have requested this because they were shocked of the increase > in size of the Python interpreter, which primarily originates from the > Unicode character database. While I think people would get over their > shock as time passes, I think creating a "small" interpreter for > special-purpose applications is desirable, and just deleting Unicode > support in such a build was (and still is) a low-hanging fruit. I wanted to make a gesture towards the developers of the PalmOS port (Pippy), who have struggled mightily to keep memory usage small. But nothing is happening there (I've asked them to make it a SF project but apparently Endeavors Technology refuses). I think the iPAQ has enough memory to support Unicode, and that's the only small platform I'm aware of that uses Python. Of course, there are probably other embedded platforms -- but they have to speak up. BTW, the savings aren't that big. I built Python with all the configure options disabled that I could find (hm, there's no way to disable complex from the command line so that's still in) and here are the sizes: text data bss dec hex filename (file size) 531629 92336 9804 633769 9aba9 small_python (1904k) 613718 117588 11612 742918 b5606 python (2168k) > That said, I see no reason why the test suite should pass in such a > build. I would suggest to let this code rot (as long as you can still > build the interpreter without Unicode), and wait for users to complain > that features don't work in disable-unicode builds that they think > should work. I don't think the tests have to pass. But I do think that importing a module and using its basic functionality should work, unless perhaps it is designed to work only with Unicode (like the XML modules). The email package violates this: Charset.py starts with "from types import UnicodeType" and it is in turn imported by the basic Message module. > Maybe we can take this feature out in P3k when we find that nobody > uses it. Until then, I would propose to leave it roughly at the state > of WITHOUT_COMPLEX. Does anybody know whether the test suite passes if > you build WITHOUT_COMPLEX? Dunno, but there's much less code affected. (Although, like unicode, I know of some unittests that use complex as an example of a "weird" type.) --Guido van Rossum (home page: http://www.python.org/~guido/) From aleax@aleax.it Mon May 27 15:07:53 2002 From: aleax@aleax.it (Alex Martelli) Date: Mon, 27 May 2002 16:07:53 +0200 Subject: [Python-Dev] Allowing Unicode literals even without Unicode support? In-Reply-To: <200205271400.g4RE07N02102@pcp742651pcs.reston01.va.comcast.net> References: <200205271400.g4RE07N02102@pcp742651pcs.reston01.va.comcast.net> Message-ID: On Monday 27 May 2002 04:00 pm, Guido van Rossum wrote: ... > I think the iPAQ has enough memory to support Unicode, and that's the > only small platform I'm aware of that uses Python. Of course, there > are probably other embedded platforms -- but they have to speak up. Sharp Zaurus is the one I'm aware of (having just ordered one specifically to put Python on it...), but I'm not sure of its Unicode status/needs (it uses an embedded version of Qt, and normally Qt uses Unicode, but I'm not certain if that holds for the embedded version as well). It has 64 MB (just like the iPAQ, I believe?), so the 200k or 300k worth of difference that you mention would appear to be a marginal issue there, too. There IS a half-price, reduced-functionality sort-of-clone of Zaurus just announced by an India manufacturer (no builtin keyboard, only 32 MB, slower CPU clock -- but, same QTopia, and same embedded-Linux, as the Zaurus), but AFAIK that's still vaporware (I wouldn't know where to go to buy one today). Alex From mal@lemburg.com Mon May 27 15:25:02 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 27 May 2002 16:25:02 +0200 Subject: [Python-Dev] Allowing Unicode literals even without Unicode support? References: <200205271400.g4RE07N02102@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CF241BE.3070601@lemburg.com> Guido van Rossum wrote: >>People have requested this because they were shocked of the increase >>in size of the Python interpreter, which primarily originates from the >>Unicode character database. While I think people would get over their >>shock as time passes, I think creating a "small" interpreter for >>special-purpose applications is desirable, and just deleting Unicode >>support in such a build was (and still is) a low-hanging fruit. > > > I wanted to make a gesture towards the developers of the PalmOS port > (Pippy), who have struggled mightily to keep memory usage small. But > nothing is happening there (I've asked them to make it a SF project > but apparently Endeavors Technology refuses). > > I think the iPAQ has enough memory to support Unicode, and that's the > only small platform I'm aware of that uses Python. Of course, there > are probably other embedded platforms -- but they have to speak up. > > BTW, the savings aren't that big. I built Python with all the > configure options disabled that I could find (hm, there's no way to > disable complex from the command line so that's still in) and here are > the sizes: > > text data bss dec hex filename (file size) > 531629 92336 9804 633769 9aba9 small_python (1904k) > 613718 117588 11612 742918 b5606 python (2168k) Note that the Unicode database is not included in the standard Python interpreter build. It is separated out as module which nowadays is only built as shared module. Now, I don't know whether e.g. Palm OS can support shared modules, but I suppose that's where you get the savings. Don't know whether it's worth our trouble though. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From mwh@python.net Mon May 27 15:51:59 2002 From: mwh@python.net (Michael Hudson) Date: 27 May 2002 15:51:59 +0100 Subject: [Python-Dev] Python 2.3 release schedule In-Reply-To: Tim Peters's message of "Sun, 26 May 2002 15:38:38 -0400" References: Message-ID: <2mwutpve28.fsf@starship.python.net> Tim Peters writes: > [Michael Hudson] > > Writing a pymemcompat.h that people can bundle with their extensions > > and then use the 2.3 memory interface with all Pythons in the range > > 1.5.2 to 2.3. I said I'd do this once the interface is settled, so > > I'll try to get to it in the next week or so. It shouldn't be *that* > > hard... > > I think that's a great idea! Feel free to commandeer as much time from MAL > and /F as you want, since they'll whine if it isn't done . Surprise of the day: this: #if PY_VERSION_HEX < 0x01060000 /* raw memory interface already present */ /* there is no object memory interface in 1.5.2 */ #define PyObject_Malloc(size) PyMem_Malloc((size)) #define PyObject_Realloc(p, size) PyMem_Realloc((p), (size)) #define PyObject_Free(p) PyMem_Free((p)) /* the object interface is there, but the names have changed */ #define PyObject_New(type, typeobj) \ PyObject_NEW((type), (typeobj)) #define PyObject_NewVar(type, typeobj, n) \ PyObject_NEW_VAR((type), (typeobj), (n)) #define PyObject_Del(ob) \ PyMem_Del((ob)) #endif is all that's required! Here's the full header I propose checking in to Misc/pymemcompat.h (although the verbiage seems a bit excessive now). /* this idea of this file is that you bundle it with your extension, #include it, program to Python 2.3's memory API and have your extension build with any version of Python from 1.5.2 through to 2.3 (and hopefully beyond) */ #ifndef Py_PYMEMCOMPAT_H #define Py_PYMEMCOMPAT_H #include "Python.h" /* There are three "families" of memory API: the "raw memory", "object memory" and "object" families. Raw Memory: PyMem_Malloc, PyMem_Realloc, PyMem_Free Object Memory: PyObject_Malloc, PyObject_Realloc, PyObject_Free Object: PyObject_New, PyObject_NewVar, PyObject_Del The raw memory and object memory allocators both mimic the malloc/realloc/free interface from ANSI C, but the object memory allocator can (and, since 2.3, does by default) use a different allocation strategy biased towards lots of lots of "small" allocations. The object family is used for allocating Python objects, and the initializers take care of some basic initialization (setting the refcount to 1 and filling out the ob_type field) as well as having a somewhat different interface. Do not mix the families! I.e. do not allocate memory with PyMem_Malloc and free it with PyObject_Free. You may get away with it quite a lot of the time, but there *are* scenarios where this will break. You Have Been Warned. Also, in many versions of Python there are an insane amount of memory interfaces to choose from. Use the ones described above.*/ #if PY_VERSION_HEX < 0x01060000 /* raw memory interface already present */ /* there is no object memory interface in 1.5.2 */ #define PyObject_Malloc(size) PyMem_Malloc((size)) #define PyObject_Realloc(p, size) PyMem_Realloc((p), (size)) #define PyObject_Free(p) PyMem_Free((p)) /* the object interface is there, but the names have changed */ #define PyObject_New(type, typeobj) \ PyObject_NEW((type), (typeobj)) #define PyObject_NewVar(type, typeobj, n) \ PyObject_NEW_VAR((type), (typeobj), (n)) #define PyObject_Del(ob) \ PyMem_Del((ob)) #endif #endif /* !Py_PYMEMCOMPAT_H */ Comments, flames, etc appreciated. Cheers, M. -- But since I'm not trying to impress anybody in The Software Big Top, I'd rather walk the wire using a big pole, a safety harness, a net, and with the wire not more than 3 feet off the ground. -- Grant Griffin, comp.lang.python From bh@intevation.de Mon May 27 16:17:48 2002 From: bh@intevation.de (Bernhard Herzog) Date: 27 May 2002 17:17:48 +0200 Subject: [Python-Dev] Python 2.3 release schedule In-Reply-To: <2mwutpve28.fsf@starship.python.net> References: <2mwutpve28.fsf@starship.python.net> Message-ID: <6qr8jxsjqb.fsf@abnoba.intevation.de> Michael Hudson writes: > > [Michael Hudson] > > > Writing a pymemcompat.h that people can bundle with their extensions > > > and then use the 2.3 memory interface with all Pythons in the range > > > 1.5.2 to 2.3. [...] > Surprise of the day: this: > > #if PY_VERSION_HEX < 0x01060000 [...] > #define PyObject_Del(ob) \ > PyMem_Del((ob)) > > #endif Have you actually tried it? There is no PyMem_Del in Python 1.5.2. Bernhard -- Intevation GmbH http://intevation.de/ Sketch http://sketch.sourceforge.net/ MapIt! http://www.mapit.de/ From mwh@python.net Mon May 27 16:18:10 2002 From: mwh@python.net (Michael Hudson) Date: 27 May 2002 16:18:10 +0100 Subject: [Python-Dev] Python 2.3 release schedule In-Reply-To: Bernhard Herzog's message of "27 May 2002 17:17:48 +0200" References: <2mwutpve28.fsf@starship.python.net> <6qr8jxsjqb.fsf@abnoba.intevation.de> Message-ID: <2mu1otvcul.fsf@starship.python.net> Bernhard Herzog writes: > Michael Hudson writes: > > > > [Michael Hudson] > > > > Writing a pymemcompat.h that people can bundle with their extensions > > > > and then use the 2.3 memory interface with all Pythons in the range > > > > 1.5.2 to 2.3. > [...] > > Surprise of the day: this: > > > > #if PY_VERSION_HEX < 0x01060000 > [...] > > #define PyObject_Del(ob) \ > > PyMem_Del((ob)) > > > > #endif > > Have you actually tried it? Well, obviously not: > There is no PyMem_Del in Python 1.5.2. You're right of course. Thanks for the heads up. I've changed it to PyMem_Free now. See any other problems? Cheers, M. -- Reading Slashdot can [...] often be worse than useless, especially to young and budding programmers: it can give you exactly the wrong idea about the technical issues it raises. -- http://www.cs.washington.edu/homes/klee/misc/slashdot.html#reasons From tismer@tismer.com Mon May 27 16:23:39 2002 From: tismer@tismer.com (Christian Tismer) Date: Mon, 27 May 2002 08:23:39 -0700 Subject: [Python-Dev] _tkinter problem with Stackless References: <200205270141.NAA06269@s454.cosc.canterbury.ac.nz> <3CF1ACA5.1050206@tismer.com> <200205271335.g4RDZdS01966@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CF24F7B.1000705@tismer.com> Guido van Rossum wrote: >>>I wouldn't call it "bad style" in general -- it's >>>perfectly legal to pass the address of a local variable >>>to another function, as long as the other function uses >>>it before returning. >> >>Maybe, but I'm really fucked up... >>Saying "bad style" might help to keep people from >>producing more problems :-) > > > Cut it out, Christian. Now! This is *your* problem to solve -- not > something to complain to others to. Yes, god father. :-) Cetero censeo Carthaginem esse delendam. Remember? Meanwhile, I've implemented a really cheap solution. By a small 3-line patch, _tkinter is able to tell Stackless that it should better not split stacks for the next N recursions. Works great! Please don't forget about the bdb thingy, it isn't closed yet! kind regards - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From martin@v.loewis.de Mon May 27 18:02:37 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 27 May 2002 19:02:37 +0200 Subject: [Python-Dev] _tkinter problem with Stackless In-Reply-To: <3CF1F3E6.7090400@tismer.com> References: <200205270537.RAA06295@s454.cosc.canterbury.ac.nz> <3CF1F3E6.7090400@tismer.com> Message-ID: Christian Tismer writes: > Maybe I'm just fine by patching _tkinter a little bit, > so that it influences stack slicing in a healthy way? I would still encourage you to investigate the source of the crash. Is a debugger capable of displaying a back trace when it crashes? With a bug trace, we could try to investigate to see whether this is a Tcl problem or a _tkinter problem. Regards, Martin From paul@prescod.net Mon May 27 18:41:23 2002 From: paul@prescod.net (Paul Prescod) Date: Mon, 27 May 2002 10:41:23 -0700 Subject: [Python-Dev] Re: Stability and change References: <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> <200205251911.01072.mark@freelance-developer.com> Message-ID: <3CF26FC3.E2835E09@prescod.net> "Martin v. Loewis" wrote: >=20 > pinard@iro.umontreal.ca (Fran=E7ois Pinard) writes: >=20 > > > I also use 2.0 as the lowest common denominator. > > > > Linguistic problem :-). Should we say "greatest" instead of "lowest"= ? > > Granted that the greatest common denominator is not "greatest" in any= other > > way, but it is lower or equal than any of the things we consider. Th= e real > > "lowest" common denominator might be very close to nothing, might it = not? >=20 > For any two natural numbers, the lowest common denominator is > 1. Finding the greatest (largest?) common denominator is indeed what > involves an algorithm. GCD =3D greatest common divisor LCD =3D lowest common denominator http://www.dictionary.com/cgi-bin/dict.pl?term=3Dgcd&r=3D67 http://www.dictionary.com/cgi-bin/dict.pl?term=3Dlcd&r=3D67 Paul Prescod From mgilfix@eecs.tufts.edu Mon May 27 20:50:37 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Mon, 27 May 2002 15:50:37 -0400 Subject: [Python-Dev] Socket timeout patch In-Reply-To: <200205232013.g4NKD6X07596@odiug.zope.com>; from guido@python.org on Thu, May 23, 2002 at 04:13:06PM -0400 References: <20020512082740.C10230@eecs.tufts.edu> <200205232013.g4NKD6X07596@odiug.zope.com> Message-ID: <20020527155037.C17710@eecs.tufts.edu> Whew. Lots of stuff. Give me a while to address all these issues, rework the patch, and then submit a comprehensive answer. Should have an answer by the end of the week latest. -- Mike On Thu, May 23 @ 16:13, Guido van Rossum wrote: > Sorry for taking such a long time to review this. (Apparently nobody > else had time either.) And thanks for doing this! I think this is a > great start. Rather than typing them in the SF bug manager issue I am > mailing them (I'll add a link to the archived version of this message > to the SF bug manager to track the activity). > > > General style nits: > > - You submitted a reverse diff! Customary is diff(old, new). > > - Please don't put a space between the function name and the open > parenthesis. (You do this both in C and in Python code.) > > - Also please don't put a space between the open parenthesis and the > first argument (you do this almost consistently in test_timeout.py). > > - Please don't introduce lines longer than 78 columns. > > > Feedback on the patch to socket.py: > > - I think you're changing the semantics of unbuffered and > line-buffered reads/writes on Windows. For one thing, you no longer > implement line-buffered writes correctly. The idea is that if the > buffer size is set to 1, data is flushed at \n only, so that if > the code builds up the line using many small writes, this doesn't > result in many small sends. There was code for this in write() -- > why did you delete it? > > - It also looks like you've broken the semantics of size<0 in read(). > > - Maybe changing the write buffer to a list makes sense too? > > - Since this code appears independent from the timeoutsocket code, > maybe we can discuss this separately? > > > Feedback on the documentation: > > - I would document that the argument to settimeout() should be int or > float (hm, can it be a long? that should be acceptable even if it's > strange), and that the return value of gettimeout() is None or a > float. > > - You may want to document the interaction with blocking mode. > > > Feedback on the C socket module changes: > > - Why import the select module? It's much more efficient to use the > select system call directly. You don't need all the overhead and > generality that the select module adds on top of it, and it costs a > lot (select allocates lots of objects and lots of memory and hence > is very expensive). > > - is already included by Python.h. > > - Please don't introduce more static functions with a 'Py' name > prefix. > > - You should raise TypeError if the type of the argument is wrong, and > ValueError if the value is wrong (out of range). Not SocketError. > > - I believe that you can't reliably maintain a "sock_blocking" flag; > there are setsockopt() or ioctl() calls that can make a socket > blocking or non-blocking. Also, there's no way to know whether a > socket passed to fromfd() is in blocking mode or not. > > - There are refcount bugs. I didn't do a detailed review of these, > but I note that the return value from PyFloat_FromDouble() in > PySocketSock_settimeout() is leaked. (There's an INCREF that's only > needed for the None case.) > > - The function internal_select() is *always* used like this: > > count = internal_select (s, 1); > if (count < 0) > return NULL; > else if (count == 0) /* Timeout elapsed */ > return PyTimeout_Err (); > > If internal_select() called PyTimeout_Err() itself, all call sites > could be simplified to this: > > count = internal_select (s, 1); > if (count < 0) > return NULL; > > or even (now that the count variable is no longer needed) to this: > > if(internal_select (s, 1) < 0) > return NULL; > > - The accept() wrapper contains this bit of code (only showing the > Unix variant): > > if (newfd < 0) > if (!s->sock_blocking || (errno != EAGAIN && errno != EWOULDBLOCK)) > return s->errorhandler (); > > Isn't the sense of testing s->sock_blocking wrong? I would think > that if we're in blocking mode we'd want to return immediately > without even checking the errno. I recommend writing this out more > clearly, e.g. like this: > > if (s->sock_blocking) > return s->errorhandler(); > /* A non-blocking accept() failed */ > if (errno != EAGAIN && errno != EWOULDBLOCK) > return s->errorhandler(); > > - What is s->errorhandler() for? AFAICT, this is always equal to > PySocket_Err! > > - The whole interaction between non-blocking mode and timeout mode is > confusing to me. Are you sure that this always does the right > thing? Have you even thought about what "the right thing" is in all > 4 combinations? > > --Guido van Rossum (home page: http://www.python.org/~guido/) `-> (guido) -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From niemeyer@conectiva.com Mon May 27 21:54:00 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Mon, 27 May 2002 17:54:00 -0300 Subject: [Python-Dev] Python 2.3 release schedule In-Reply-To: <200205242217.g4OMHbu25323@pcp742651pcs.reston01.va.comcast.net> References: <200205242217.g4OMHbu25323@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020527175400.A8393@ibook.distro.conectiva> Guido, > Anything else? Are you still pondering the inclusion of tar/bz2 support? Lars (tarfile's author) asked for some time to finish tarfile development and write a PEP about tarfile inclusion. Now that my vacation is over, I must get back to him and check what's going on. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From tim.one@comcast.net Mon May 27 22:01:23 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 27 May 2002 17:01:23 -0400 Subject: [Python-Dev] Python 2.3 release schedule In-Reply-To: <2mwutpve28.fsf@starship.python.net> Message-ID: [Michael Hudson] > Surprise of the day: this [is all that's required!]: > > #if PY_VERSION_HEX < 0x01060000 > /* raw memory interface already present */ > > /* there is no object memory interface in 1.5.2 */ > #define PyObject_Malloc(size) PyMem_Malloc((size)) > #define PyObject_Realloc(p, size) PyMem_Realloc((p), (size)) > ... How come not the simpler #define PyObject_Malloc PyMem_Malloc #define PyObject_Realloc PyMem_Realloc etc instead? Filling in an argument list prevents use of a name as a function designator. > ... > /* There are three "families" of memory API: the "raw memory", "object > memory" and "object" families. > > Raw Memory: > > PyMem_Malloc, PyMem_Realloc, PyMem_Free > > Object Memory: > > PyObject_Malloc, PyObject_Realloc, PyObject_Free > > Object: > > PyObject_New, PyObject_NewVar, PyObject_Del I rarely mention the PyObject_GC_XYZ family because it's already in good shape. But its API changed between 2.1 and 2.2, and it would be good to supply a compatibility layer for it too. Pick Neil's brain . > ... > Comments, flames, etc appreciated. I especially liked the (snipped) to-the-point comment blocks; PyObject_Del problems were already mentioned; thank you! From David Abrahams" <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> <200205251911.01072.mark@freelance-developer.com> <3CF26FC3.E2835E09@prescod.net> Message-ID: <006301c205d8$6558d2f0$6401a8c0@boostconsulting.com> From: "Paul Prescod" > GCD = greatest common divisor > LCD = lowest common denominator ^^^^^^ Should be "least". I still remember when my 3rd grade teacher derided the imprecise use of terms other than "greatest" and "least" for describing numbers. > http://www.dictionary.com/cgi-bin/dict.pl?term=lcd&r=67 See your own link. nitpicking-ly yr's dave From tismer@tismer.com Tue May 28 00:54:08 2002 From: tismer@tismer.com (Christian Tismer) Date: Mon, 27 May 2002 16:54:08 -0700 Subject: [Python-Dev] _tkinter problem with Stackless References: <200205270537.RAA06295@s454.cosc.canterbury.ac.nz> <3CF1F3E6.7090400@tismer.com> Message-ID: <3CF2C720.1020005@tismer.com> Martin v. Loewis wrote: > Christian Tismer writes: > > >>Maybe I'm just fine by patching _tkinter a little bit, >>so that it influences stack slicing in a healthy way? > > > I would still encourage you to investigate the source of the crash. Is > a debugger capable of displaying a back trace when it crashes? Not really. What I see is some five nested calls from tcl83.dll . > With a bug trace, we could try to investigate to see whether this is a > Tcl problem or a _tkinter problem. Then I have to build Tcl to check this out. Will do that when I'm back in Berlin. ciao - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From guido@python.org Tue May 28 01:33:41 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 27 May 2002 20:33:41 -0400 Subject: [Python-Dev] Python 2.3 release schedule In-Reply-To: Your message of "Mon, 27 May 2002 17:54:00 -0300." <20020527175400.A8393@ibook.distro.conectiva> References: <200205242217.g4OMHbu25323@pcp742651pcs.reston01.va.comcast.net> <20020527175400.A8393@ibook.distro.conectiva> Message-ID: <200205280033.g4S0XgU02751@pcp742651pcs.reston01.va.comcast.net> > Are you still pondering the inclusion of tar/bz2 support? That's a library issue; I'm all for adding it if the code is robust enough, but ot for holding up a release for it. > Lars (tarfile's author) asked for some time to finish tarfile > development and write a PEP about tarfile inclusion. Now that my > vacation is over, I must get back to him and check what's going on. Please do! --Guido van Rossum (home page: http://www.python.org/~guido/) From pinard@iro.umontreal.ca Tue May 28 01:36:17 2002 From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard) Date: 27 May 2002 20:36:17 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: References: <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> <200205251911.01072.mark@freelance-developer.com> Message-ID: [Martin v. Loewis] > pinard@iro.umontreal.ca (François Pinard) writes: > > > I also use 2.0 as the lowest common denominator. > > > > Linguistic problem :-). Should we say "greatest" instead of "lowest"? > > Granted that the greatest common denominator is not "greatest" in any other > > way, but it is lower or equal than any of the things we consider. The real > > "lowest" common denominator might be very close to nothing, might it not? > For any two natural numbers, the lowest common denominator is > 1. Finding the greatest (largest?) common denominator is indeed what > involves an algorithm. So, besides arithmetic, what would be the correct expression to replace: "I also use 2.0 as the lowest common denominator." ? P.S. - It's amusing how fast off-topic matters slide even more off topic! :-) The real thread is about how documentation could be amended to (maybe) help many people at being comfortable with Python, despite the rate of change. It is probably worth staying concentrated on the real thread... -- François Pinard http://www.iro.umontreal.ca/~pinard From greg@cosc.canterbury.ac.nz Tue May 28 01:53:34 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Tue, 28 May 2002 12:53:34 +1200 (NZST) Subject: [Python-Dev] Re: Stability and change In-Reply-To: Message-ID: <200205280053.MAA06392@s454.cosc.canterbury.ac.nz> > So, besides arithmetic, what would be the correct expression to replace: > "I also use 2.0 as the lowest common denominator." ? If you want a more accurate mathematical analogy, you should be saying "greatest common factor". Better yet, avoid the cliche altogether and think of a more direct way of saying what you mean. e.g. "I use 2.0 because it supports the largest common feature set." Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From guido@python.org Tue May 28 01:59:46 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 27 May 2002 20:59:46 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 27 May 2002 19:39:31 EDT." <006301c205d8$6558d2f0$6401a8c0@boostconsulting.com> References: <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> <200205251911.01072.mark@freelance-developer.com> <3CF26FC3.E2835E09@prescod.net> <006301c205d8$6558d2f0$6401a8c0@boostconsulting.com> Message-ID: <200205280059.g4S0xkZ02827@pcp742651pcs.reston01.va.comcast.net> > I still remember when my 3rd grade teacher derided the > imprecise use of terms other than "greatest" and "least" for describing > numbers. And you believed him? What bullshit. --Guido van Rossum (home page: http://www.python.org/~guido/) From greg@cosc.canterbury.ac.nz Tue May 28 02:18:39 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Tue, 28 May 2002 13:18:39 +1200 (NZST) Subject: [Python-Dev] _tkinter problem with Stackless In-Reply-To: Message-ID: <200205280118.NAA06397@s454.cosc.canterbury.ac.nz> martin@v.loewis.de (Martin v. Loewis): > Christian segments the stack into slices, and only > keeps the top-of-stack slice on the stack. Cripes! Christian, you're just *asking* for a crash if you do that. Is this really necessary? Why can't you keep the whole stack in one piece? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Tue May 28 02:23:53 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Tue, 28 May 2002 13:23:53 +1200 (NZST) Subject: [Python-Dev] _tkinter problem with Stackless In-Reply-To: <3CF1F3E6.7090400@tismer.com> Message-ID: <200205280123.NAA06400@s454.cosc.canterbury.ac.nz> Christian Tismer : > By setting the slicing recursion interval to above 20, > the Tcl stuff works. > Maybe I'm just fine by patching _tkinter a little bit, > so that it influences stack slicing in a healthy way? No, I don't think you're fine at all. You might just happen to avoid the problem in this particular case, but the basic flaw is still there, waiting to trip up someone else running some other C extension. Have you run any of this on a Sparc? I can see the register window spillage mechanism getting royally screwed by having the stack chopped up. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From David Abrahams" <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> <200205251911.01072.mark@freelance-developer.com> <3CF26FC3.E2835E09@prescod.net> <006301c205d8$6558d2f0$6401a8c0@boostconsulting.com> <200205280059.g4S0xkZ02827@pcp742651pcs.reston01.va.comcast.net> Message-ID: <020501c205fd$aef1eb70$6401a8c0@boostconsulting.com> From: "Guido van Rossum" > > I still remember when my 3rd grade teacher derided the > > imprecise use of terms other than "greatest" and "least" for describing > > numbers. > > And you believed him? What bullshit. Her, actually. She was trying to say that precision in speaking counts, and it made an impression with me. From tismer@tismer.com Tue May 28 06:57:53 2002 From: tismer@tismer.com (Christian Tismer) Date: Mon, 27 May 2002 22:57:53 -0700 Subject: [Python-Dev] _tkinter problem with Stackless References: <200205280118.NAA06397@s454.cosc.canterbury.ac.nz> Message-ID: <3CF31C61.6040606@tismer.com> Greg Ewing wrote: > martin@v.loewis.de (Martin v. Loewis): > > >>Christian segments the stack into slices, and only >>keeps the top-of-stack slice on the stack. > > > Cripes! Christian, you're just *asking* for a crash > if you do that. :-)) > Is this really necessary? Why can't you keep the > whole stack in one piece? Since this is Stackless Python(R)(TM)(Pat.Pend.) If I wouldn't slice the stacks, it wouldn't be Stackless. But now I have a critical section bracked built in, that prevends stacks from slicing to a given recursion level. Works just great! sincerely - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From martin@v.loewis.de Tue May 28 06:59:34 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 28 May 2002 07:59:34 +0200 Subject: [Python-Dev] _tkinter problem with Stackless In-Reply-To: <200205280118.NAA06397@s454.cosc.canterbury.ac.nz> References: <200205280118.NAA06397@s454.cosc.canterbury.ac.nz> Message-ID: Greg Ewing writes: > Is this really necessary? Why can't you keep the > whole stack in one piece? Without that, you won't get - fast tasklet switching (it now needs to copy only the topmost slice), and - deep recursions (there is no stacklimit in stackless python) IOW, without that approach, it won't be stackless. Regards, Martin From tismer@tismer.com Tue May 28 07:03:38 2002 From: tismer@tismer.com (Christian Tismer) Date: Mon, 27 May 2002 23:03:38 -0700 Subject: [Python-Dev] _tkinter problem with Stackless References: <200205280118.NAA06397@s454.cosc.canterbury.ac.nz> Message-ID: <3CF31DBA.6060409@tismer.com> Martin v. Loewis wrote: > Greg Ewing writes: > > >>Is this really necessary? Why can't you keep the >>whole stack in one piece? > > > Without that, you won't get > - fast tasklet switching (it now needs to copy only the topmost > slice), and > - deep recursions (there is no stacklimit in stackless python) > IOW, without that approach, it won't be stackless. Well spoken, Tiger! -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From fredrik@pythonware.com Tue May 28 07:30:23 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Tue, 28 May 2002 08:30:23 +0200 Subject: [Python-Dev] Re: Stability and change References: <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> <200205251911.01072.mark@freelance-developer.com> <3CF26FC3.E2835E09@prescod.net> <006301c205d8$6558d2f0$6401a8c0@boostconsulting.com> Message-ID: <007801c20611$2e47bfe0$ced241d5@hagrid> david wrote: > > LCD = lowest common denominator > ^^^^^^ > > Should be "least". I still remember when my 3rd grade teacher derided the > imprecise use of terms other than "greatest" and "least" for describing > numbers. > > > http://www.dictionary.com/cgi-bin/dict.pl?term=lcd&r=67 > > See your own link. did you check the link? the first entry only expands the abbreviation, the third entry provides the full explanation: least common denominator n. Abbr. lcd The least common multiple of the denominators of a set of fractions: The least common denominator of 1/3 and 1/4 is 12. Also called lowest common denominator. > nitpicking-ly yr's if you're gonna nitpick, you could at least do it right ;-) From Jack.Jansen@cwi.nl Tue May 28 11:09:23 2002 From: Jack.Jansen@cwi.nl (Jack Jansen) Date: Tue, 28 May 2002 12:09:23 +0200 Subject: [Python-Dev] _tkinter problem with Stackless In-Reply-To: <3CF24F7B.1000705@tismer.com> Message-ID: On Monday, May 27, 2002, at 05:23 , Christian Tismer wrote: > Meanwhile, I've implemented a really cheap solution. > By a small 3-line patch, _tkinter is able to tell > Stackless that it should better not split stacks > for the next N recursions. Works great! For that one case (tkinter), but not in general. Would it be possible to somehow restructure this the other way around, i.e. not slice stacks unless you know that all frames on the stack so far don't mind? I'm thinking of something similar to BEGIN_ALLOW_THREADS: any naive extension module whose author didn't know about Python threading will still work fine in a threaded Python, because it'll just hold on to the interpreter lock. If this is impossible (which I can well imagine) then at the very least we should have a macro KEEP_YOUR_DIRTY_FINGERS_OFF_MY_STACK() to tell stackless that you've given away stack addresses to some external package and you don't want your stack to be moved. -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From tismer@tismer.com Tue May 28 12:04:40 2002 From: tismer@tismer.com (Christian Tismer) Date: Tue, 28 May 2002 04:04:40 -0700 Subject: [Python-Dev] _tkinter problem with Stackless References: Message-ID: <3CF36448.8000709@tismer.com> Jack Jansen wrote: ... > For that one case (tkinter), but not in general. Would it be possible to > somehow restructure this the other way around, i.e. not slice stacks > unless you know that all frames on the stack so far don't mind? I'm > thinking of something similar to BEGIN_ALLOW_THREADS: any naive > extension module whose author didn't know about Python threading will > still work fine in a threaded Python, because it'll just hold on to the > interpreter lock. Yes I could do this and also considered it already: This gets me into preparing a registry of all known compatible extensions. I don't like to take on such a role. Instead, I try to get the most popular stuff running, wile notifying everybody else that her stuff won't work without a bit of fiddling. At the time being, Stackless can afford this slightly Etonian aproach... > If this is impossible (which I can well imagine) then at the very least > we should have a macro KEEP_YOUR_DIRTY_FINGERS_OFF_MY_STACK() to tell > stackless that you've given away stack addresses to some external > package and you don't want your stack to be moved. That's nearly exactly what I implemented for _tkinter: macro INSANE_TO_SLICE_STACKS_FOR_N_RECURSIONS(N) does the good job. Will check this in, tomorrow. ciao - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From mwh@python.net Tue May 28 12:12:33 2002 From: mwh@python.net (Michael Hudson) Date: 28 May 2002 12:12:33 +0100 Subject: [Python-Dev] Python 2.3 release schedule In-Reply-To: Tim Peters's message of "Mon, 27 May 2002 17:01:23 -0400" References: Message-ID: <2mr8jwv84e.fsf@starship.python.net> Tim Peters writes: > [Michael Hudson] > > Surprise of the day: this [is all that's required!]: > > > > #if PY_VERSION_HEX < 0x01060000 > > /* raw memory interface already present */ > > > > /* there is no object memory interface in 1.5.2 */ > > #define PyObject_Malloc(size) PyMem_Malloc((size)) > > #define PyObject_Realloc(p, size) PyMem_Realloc((p), (size)) > > ... > > How come not the simpler > > #define PyObject_Malloc PyMem_Malloc > #define PyObject_Realloc PyMem_Realloc > > etc instead? Filling in an argument list prevents use of a name as a > function designator. I thought filling out the arguments was better style, for some reason. Easy enough to change. > > ... > > /* There are three "families" of memory API: the "raw memory", "object > > memory" and "object" families. > > > > Raw Memory: > > > > PyMem_Malloc, PyMem_Realloc, PyMem_Free > > > > Object Memory: > > > > PyObject_Malloc, PyObject_Realloc, PyObject_Free > > > > Object: > > > > PyObject_New, PyObject_NewVar, PyObject_Del > > I rarely mention the PyObject_GC_XYZ family because it's already in good > shape. But its API changed between 2.1 and 2.2, and it would be good to > supply a compatibility layer for it too. Pick Neil's brain . I was under the impression that the 2.1 and 2.2 interfaces differed in ways that couldn't easily be papered over with macros. I'll check. > > ... > > Comments, flames, etc appreciated. > > I especially liked the (snipped) to-the-point comment blocks; Thanks; I think it is a good idea to describe intended usage in no uncertain terms *somewhere* at least. Probably lots of places. Any book authors reading python-dev? > PyObject_Del problems were already mentioned; thank you! I hope it helps! Cheers, M. -- Famous remarks are very seldom quoted correctly. -- Simeon Strunsky From pinard@iro.umontreal.ca Tue May 28 12:20:25 2002 From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard) Date: 28 May 2002 07:20:25 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200205280053.MAA06392@s454.cosc.canterbury.ac.nz> References: <200205280053.MAA06392@s454.cosc.canterbury.ac.nz> Message-ID: [Greg Ewing] > "I use 2.0 because it supports the largest common feature set." Nicely put! :-) -- François Pinard http://www.iro.umontreal.ca/~pinard From jacobs@penguin.theopalgroup.com Tue May 28 12:34:58 2002 From: jacobs@penguin.theopalgroup.com (Kevin Jacobs) Date: Tue, 28 May 2002 07:34:58 -0400 (EDT) Subject: [Python-Dev] Lazily GC tracking tuples In-Reply-To: Message-ID: On Fri, 24 May 2002, Tim Peters wrote: > [Kevin Jacobs] > > Well, I've found one case where the garbage collector _does_ cause > > serious performance problems, and coincidentally enough it is due to > > the creation of zillions of cycle-less tuples. Our current solution has > > been to disable GC in some parts of our code, and then manually trigger > > collection at the correct points. When I can find some free time, I will > > definitly test drive any available tuple untracking patches to see if > > they make any substantial difference in our code. > > Well, I'm told it's a 3-day weekend , and Neil has a short patch you > can try: > > http://www.python.org/sf/558745 > > If you can, please do. I view this as a diasaster-insurance change, not as > a generally applicable speedup (it's almost certainly not, and especially > not when using a distinct pass to weed out tuples -- I'd combine it with the > copy-refcounts pass), so this lives or dies on whether it helps people > suffering pathologies in real life. That's you. It doesn't seem to make much difference in our app. There seems to be some speedup, but nothing dramatic. Turning GC on and off at the right times is still slightly faster. The good news is that another (unrelated) part of our code just became about 20-40% faster with this patch, though I need to do some fairly major surgery to isolate why this is so. So more sleuthing required, -Kevin -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com From sholden@holdenweb.com Tue May 28 13:54:24 2002 From: sholden@holdenweb.com (Steve Holden) Date: Tue, 28 May 2002 08:54:24 -0400 Subject: [Python-Dev] Re: Stability and change References: <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net><200205251911.01072.mark@freelance-developer.com> Message-ID: <03fb01c20646$cdd8a580$6300000a@holdenweb.com> ----- Original Message ----- From: "Martin v. Loewis" To: "François Pinard" Cc: "Mark J. Nenadov" ; Sent: Monday, May 27, 2002 2:21 AM Subject: Re: [Python-Dev] Re: Stability and change pinard@iro.umontreal.ca (François Pinard) writes: > > I also use 2.0 as the lowest common denominator. > > Linguistic problem :-). Should we say "greatest" instead of "lowest"? > Granted that the greatest common denominator is not "greatest" in any other > way, but it is lower or equal than any of the things we consider. The real > "lowest" common denominator might be very close to nothing, might it not? For any two natural numbers, the lowest common denominator is 1. Finding the greatest (largest?) common denominator is indeed what involves an algorithm. This terminology came from fractions. The canonical form of a fraction uses the lowest possible number as the denominator. When performing additive operations on fractions, the easiest way to proceed is to convert them to a form with the same number as a denominator. The usual choice is the lowest common denominator, which is the smallest number that can be used as the denominator in both fractions. This is the LCM (least common multiple) of the two denominators. e.g.: 2/3 + 3/8 = 16/24 + 9/24 [24 being the LCD] = 25/24 = 1 1/24 It doesn't therefore (IMO) make much sense to talk about the lowest common denominator of any two natural numbers. The LCM, yes. The LCD, no. It makes even less sense to talk about the lowest common denominator of Python implementations, except in the metaphorical sense Mark (I believe) intended. but-since-we're-being-literal-please-read-this-with-a-straight-face-ly 'rs - steve ----------------------------------------------------------------------- Steve Holden http://www.holdenweb.com/ Python Web Programming http://pydish.holdenweb.com/pwp/ ----------------------------------------------------------------------- From guido@python.org Tue May 28 14:40:35 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 28 May 2002 09:40:35 -0400 Subject: [Python-Dev] Fatal Python error: GC object already in linked list Message-ID: <200205281340.g4SDea815971@odiug.zope.com> I'm forwarding this report to python-dev since it's unlikely that anyone on the zope3 list will be able to provide insight. (I'm not *much* more optimistic about python-dev, but it is a possibility. :-) Please keep Steve cc'ed as he's not a python-dev subscriber. --Guido van Rossum (home page: http://www.python.org/~guido/) ------- Forwarded Message Date: Tue, 28 May 2002 14:14:10 +0100 From: Steve Alexander To: "'Zope 3 Mailing List'" Subject: [Zope3-dev] Fatal Python error: GC object already in linked list While working on Annotations their interaction with ContextWrappers / Security Proxies, I got this error: Fatal Python error: GC object already in linked list Aborted No other details, no traceback. I was left with the Data.fs locked by three other Python processes that required a kill -9 to get rid of. I haven't been able to reproduce the error since. Python 2.2.1, compiled from source on a Pentuim 4, RedHat 7.3 box. - -- Steve Alexander _______________________________________________ Zope3-dev mailing list Zope3-dev@zope.org http://lists.zope.org/mailman/listinfo/zope3-dev ------- End of Forwarded Message From steve@cat-box.net Tue May 28 14:47:16 2002 From: steve@cat-box.net (Steve Alexander) Date: Tue, 28 May 2002 14:47:16 +0100 Subject: [Python-Dev] Re: Fatal Python error: GC object already in linked list References: <200205281340.g4SDea815971@odiug.zope.com> Message-ID: <3CF38A64.2010903@cat-box.net> Thanks Guido. I have the problem reproducable at will now, although I haven't narrowed it down to something reasonable to package up as a test case. (It involves many parts of Zope 3.) I have no idea how to go about looking into this further, but I'm willing to try any reasonable suggestions to see where things are tripping up. Guido van Rossum wrote: > I'm forwarding this report to python-dev since it's unlikely that > anyone on the zope3 list will be able to provide insight. (I'm not > *much* more optimistic about python-dev, but it is a possibility. :-) > Please keep Steve cc'ed as he's not a python-dev subscriber. > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > ------- Forwarded Message > > Date: Tue, 28 May 2002 14:14:10 +0100 > From: Steve Alexander > To: "'Zope 3 Mailing List'" > Subject: [Zope3-dev] Fatal Python error: GC object already in linked list > > While working on Annotations their interaction with ContextWrappers / > Security Proxies, I got this error: > > Fatal Python error: GC object already in linked list > Aborted > > No other details, no traceback. > > I was left with the Data.fs locked by three other Python processes that > required a kill -9 to get rid of. > > I haven't been able to reproduce the error since. > > Python 2.2.1, compiled from source on a Pentuim 4, RedHat 7.3 box. > > - -- > Steve Alexander > > > > _______________________________________________ > Zope3-dev mailing list > Zope3-dev@zope.org > http://lists.zope.org/mailman/listinfo/zope3-dev > > ------- End of Forwarded Message > From guido@python.org Tue May 28 15:01:35 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 28 May 2002 10:01:35 -0400 Subject: [Python-Dev] Re: Fatal Python error: GC object already in linked list In-Reply-To: Your message of "Tue, 28 May 2002 14:47:16 BST." <3CF38A64.2010903@cat-box.net> References: <200205281340.g4SDea815971@odiug.zope.com> <3CF38A64.2010903@cat-box.net> Message-ID: <200205281401.g4SE1Zt16105@odiug.zope.com> > I have the problem reproducable at will now, although I haven't narrowed > it down to something reasonable to package up as a test case. (It > involves many parts of Zope 3.) I have a current Zope3 checkout, so that shouldn't stop me. > I have no idea how to go about looking into this further, but I'm > willing to try any reasonable suggestions to see where things are > tripping up. Do you know how to use gdb? The next thing to do is to get a C traceback from gdb. --Guido van Rossum (home page: http://www.python.org/~guido/) From nas@python.ca Tue May 28 16:01:49 2002 From: nas@python.ca (Neil Schemenauer) Date: Tue, 28 May 2002 08:01:49 -0700 Subject: [Python-Dev] Python 2.3 release schedule In-Reply-To: <2mr8jwv84e.fsf@starship.python.net>; from mwh@python.net on Tue, May 28, 2002 at 12:12:33PM +0100 References: <2mr8jwv84e.fsf@starship.python.net> Message-ID: <20020528080149.A7799@glacier.arctrix.com> Michael Hudson wrote: > I was under the impression that the 2.1 and 2.2 interfaces differed in > ways that couldn't easily be papered over with macros. I'll check. It's not pretty. Look at pyexpat.c for an example. Perhaps something like this would be good enough (untested): #if PY_VERSION_HEX < 0x020200B1 #define PyObject_GC_New PyObject_New #define PyObject_GC_NewVar PyObject_NewVar #define PyObject_GC_Del PyObject_Del #define PyObject_GC_Track(op) #define PyObject_GC_UnTrack(op) #endif People could then always use the 2.2 API but the objects would only be collected in versions >= 2.2. Using the 2.1 API is a fair bit trickier and you can't hide those differences using macros (although you could make it easier for people who want to support 2.1 and >=2.2). Neil From guido@python.org Tue May 28 16:25:59 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 28 May 2002 11:25:59 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "26 May 2002 09:25:58 EDT." References: <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> <200205251911.01072.mark@freelance-developer.com> Message-ID: <200205281525.g4SFPxe17659@odiug.zope.com> > Exactly. The changes _are_ documented in detail, indeed, but in > special documents which only serious users read when they about to > migrate from one version to another. I'm thinking about users who > use the language occasionally or even regularly, but not fanatic > about following everything about versions -- they mainly rely on the > Python Library Reference, or even the Python Language Reference. Hm. People who don't read the detailed documents shouldn't expect to rely on details. > These references describe some Python, but not necessarily the > Python which happens to run on a given machine, and I guess (without > having really experienced this myself) it might be frustrating to > read and study, for discovering soon after that the feature is > unavailable in this version. Or even, for someone, to have handy > information about how to write for the common denominator, without > having to compare many printings of the references at various Python > levels. I guess that notes or footnotes, about Python levels in > which described features have been implemented, might help users > having to cope with release lags between Linux releases. Such lags > are unavoidable whenever Python evolves. Have you read the library manual recently? We are very careful in adding notes about which version added a particular feature or even detail. So I think there is no reason to complain about this preemptively unless your *own* experience indicates there's a lack of documentation. --Guido van Rossum (home page: http://www.python.org/~guido/) From mal@lemburg.com Tue May 28 16:50:00 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 28 May 2002 17:50:00 +0200 Subject: [Python-Dev] Re: Stability and change References: <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> <200205251911.01072.mark@freelance-developer.com> <200205281525.g4SFPxe17659@odiug.zope.com> Message-ID: <3CF3A728.7000604@lemburg.com> Guido van Rossum wrote: >>Exactly. The changes _are_ documented in detail, indeed, but in >>special documents which only serious users read when they about to >>migrate from one version to another. I'm thinking about users who >>use the language occasionally or even regularly, but not fanatic >>about following everything about versions -- they mainly rely on the >>Python Library Reference, or even the Python Language Reference. > > > Hm. People who don't read the detailed documents shouldn't expect to > rely on details. People do read the detailed documents, but not again and again every 6-8 months... heck, I work in the Python development team and don't even feel like I am up-to-date with all the changes going on in the core. If not even developers can follow the rate of change, how should a typical Python user feel ? Conclusion: I think that we need a migration guide for Python. This would solve a whole lot of these "rate of change" problems. The guide should complement the additions and changes to the other documentation and provide a single source of knowledge in that area. Currently, the only source we have in this area is the Misc/NEWS file and this doesn't provide any upgrade path hints or porting details. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/ From mal@lemburg.com Tue May 28 17:20:54 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 28 May 2002 18:20:54 +0200 Subject: [Python-Dev] Re: Stability and change References: <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> <200205251911.01072.mark@freelance-developer.com> <200205281525.g4SFPxe17659@odiug.zope.com> <3CF3A728.7000604@lemburg.com> <200205281558.g4SFwd317973@odiug.zope.com> Message-ID: <3CF3AE66.8070000@lemburg.com> Guido van Rossum wrote: >>Conclusion: I think that we need a migration guide for Python. > > > I think Andrew's "What's new in Python x.x" documents are a good > start. What's missing from them? The migration path, including hints on how to port Python applications from one version to the next, how to work around deprecations, tips on quick ways to find problem areas in the code, etc. Andrew usually puts more focus on new features than changes which could cause older applications to fail -- which is what you'd expect in a document titled "What's new in ..." :-) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/ From guido@python.org Tue May 28 16:58:39 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 28 May 2002 11:58:39 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Tue, 28 May 2002 17:50:00 +0200." <3CF3A728.7000604@lemburg.com> References: <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> <200205251911.01072.mark@freelance-developer.com> <200205281525.g4SFPxe17659@odiug.zope.com> <3CF3A728.7000604@lemburg.com> Message-ID: <200205281558.g4SFwd317973@odiug.zope.com> > Conclusion: I think that we need a migration guide for Python. I think Andrew's "What's new in Python x.x" documents are a good start. What's missing from them? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue May 28 17:30:25 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 28 May 2002 12:30:25 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Tue, 28 May 2002 18:20:54 +0200." <3CF3AE66.8070000@lemburg.com> References: <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> <200205251911.01072.mark@freelance-developer.com> <200205281525.g4SFPxe17659@odiug.zope.com> <3CF3A728.7000604@lemburg.com> <200205281558.g4SFwd317973@odiug.zope.com> <3CF3AE66.8070000@lemburg.com> Message-ID: <200205281630.g4SGUPa18148@odiug.zope.com> > > I think Andrew's "What's new in Python x.x" documents are a good > > start. What's missing from them? > > The migration path, including hints on how to port Python applications > from one version to the next, how to work around deprecations, > tips on quick ways to find problem areas in the code, etc. > > Andrew usually puts more focus on new features than changes > which could cause older applications to fail -- which is what > you'd expect in a document titled "What's new in ..." :-) OK. The question is, who's going to write the migration guide? One problem is that often specific migration issues don't surface until after the release is made. --Guido van Rossum (home page: http://www.python.org/~guido/) From pinard@iro.umontreal.ca Tue May 28 17:49:09 2002 From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard) Date: 28 May 2002 12:49:09 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200205281525.g4SFPxe17659@odiug.zope.com> References: <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> <200205251911.01072.mark@freelance-developer.com> <200205281525.g4SFPxe17659@odiug.zope.com> Message-ID: [Guido van Rossum] > Have you read the library manual recently? No. I read manuals a few times quite carefully, but now am only using them as references. As a user, I cannot really afford reading everything at each release. > We are very careful in adding notes about which version added a particular > feature or even detail. So I think there is no reason to complain about > this preemptively unless your *own* experience indicates there's a lack > of documentation. Checking random sections of the library reference for actual needs, I remember having had to cross-check with previous printing of the same to make sure that the code I was writing could be ported to other machines running older Python versions. I would probably not have done this if I stumbled on that information in the first place, so it may lack here or there. It is very nice that you are careful about adding notes as you explain above, thanks for all of us. If I meet other unclear cases, I'll try reporting them precisely in the future (yet I'm surprised by the amount of email which is sometimes needed to get a little point through.) My own experience has no frustration, because I took the time to check. I only guess that other's experience may yield frustration for them, witnessing the reluctance or anger of some of my co-workers to any kind of change. Surely, I do not feel real problems myself. Maybe I should shut up. I only thought useful, at least a bit, that I try to explain what I see around, because I know my co-workers are not going to step in the public discussion arena (they also have to go over the English barrier), and presume most users are just silent alike. > Hm. People who don't read the detailed documents shouldn't expect to > rely on details. You know, everything is a detail of some sort, and at least for newcomers, the language and the library are mere accumulation of details. Users just cannot program without relying on these. One needs a lot of perspective for sorting out the relative importance of various specifications. So, in your argument above (which I find a bit harsh), it might not be very realistic expecting that people read everything in every release. It is reasonable to expect users to read everything once, however -- and users may then have to wave between sites, distributions, and Python releases. If you try putting perspective and historical notes in the documentation, this is very good, and I guess it will be helpful for many people. -- François Pinard http://www.iro.umontreal.ca/~pinard From akuchlin@mems-exchange.org Tue May 28 17:59:33 2002 From: akuchlin@mems-exchange.org (akuchlin@mems-exchange.org) Date: Tue, 28 May 2002 12:59:33 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <3CF3AE66.8070000@lemburg.com> References: <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> <200205251911.01072.mark@freelance-developer.com> <200205281525.g4SFPxe17659@odiug.zope.com> <3CF3A728.7000604@lemburg.com> <200205281558.g4SFwd317973@odiug.zope.com> <3CF3AE66.8070000@lemburg.com> Message-ID: <20020528165933.GD8699@mems-exchange.org> On Tue, May 28, 2002 at 06:20:54PM +0200, M.-A. Lemburg wrote: >The migration path, including hints on how to port Python applications >from one version to the next, how to work around deprecations, >tips on quick ways to find problem areas in the code, etc. Part of the problem is that I work from the CVS logs and from the Misc/NEWS file, and people don't always indicate when a potential incompatibility is introduced. (That may not be clear to the person making the change, of course, if they think they're changing an obscure bit of code.) I'd certainly like to point out required changes. The 2.0 document had a "Porting to 2.0" section, but now I generally try to cover such issues in the section for the relevant PEP. --amk From guido@python.org Tue May 28 18:00:38 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 28 May 2002 13:00:38 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "28 May 2002 12:49:09 EDT." References: <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> <200205251911.01072.mark@freelance-developer.com> <200205281525.g4SFPxe17659@odiug.zope.com> Message-ID: <200205281700.g4SH0do20575@odiug.zope.com> > My own experience has no frustration, because I took the time to > check. I only guess that other's experience may yield frustration > for them, witnessing the reluctance or anger of some of my > co-workers to any kind of change. Surely, I do not feel real > problems myself. Maybe I should shut up. I only thought useful, at > least a bit, that I try to explain what I see around, because I know > my co-workers are not going to step in the public discussion arena > (they also have to go over the English barrier), and presume most > users are just silent alike. Now consider my frustration. We go through a *lot* of efforts to make consecutive releases backwards compatible, to document changes, to introduce warnings about future incompatible changes, etc. And here you are referring to second-hand frustration with the pace of change, which cannot be substantiated. Is it not fair that I ask you to provide more details or shut up? After all it is a fact of life that many people like to complain without reason, will not admit to their own mistakes, and blame them on others. > > Hm. People who don't read the detailed documents shouldn't expect > > to rely on details. > > You know, everything is a detail of some sort, and at least for > newcomers, the language and the library are mere accumulation of > details. Users just cannot program without relying on these. One > needs a lot of perspective for sorting out the relative importance > of various specifications. So, in your argument above (which I find > a bit harsh), it might not be very realistic expecting that people > read everything in every release. It is reasonable to expect users > to read everything once, however -- and users may then have to wave > between sites, distributions, and Python releases. Actually, in my experience, most users don't read the manuals even once (what I know of your personality, you are an exception). Example: the possibility to write list.append(a, b, c, ...) was never documented, yet it caused widespread complaints when we disallowed it. My expectation is that what is really going on here is that people write code that uses undocumented features, because they found experimentally (or accidentally) that certain things work in one version, and do not bother to check the reference manual if the language guarantees the observed behavior. If your code only uses documented features, you won't have a problem upgrading, and if upgrading breaks old promises, this is mentioned very prominently in the release notes. --Guido van Rossum (home page: http://www.python.org/~guido/) From pinard@iro.umontreal.ca Tue May 28 18:58:15 2002 From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard) Date: 28 May 2002 13:58:15 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200205281700.g4SH0do20575@odiug.zope.com> References: <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> <200205251911.01072.mark@freelance-developer.com> <200205281525.g4SFPxe17659@odiug.zope.com> <200205281700.g4SH0do20575@odiug.zope.com> Message-ID: [Guido van Rossum] > Now consider my frustration. We go through a *lot* of efforts to make > consecutive releases backwards compatible, to document changes, to > introduce warnings about future incompatible changes, etc. Yes, I quite understand how frustrating it may be for you, and I do witness the results of all these efforts. The Python documentation is of high quality, rather complete, and very abundant for whoever looks around a tiny bit. In my own experience, migrating has been a breeze all over. > Is it not fair that I ask you to provide more details or shut up? Not only fair, but also easier: providing details is natural for me! I sometimes fear being perceived as a nit-picker. On one hand, I met many maintainers who like detailed reports on little things! :-). On the other hand, I've seen a few maintainers getting furious, and this is no fun. > [...] most users don't read the manuals even once You may be right. But then, they miss something! :-) > Example: the possibility to write list.append(a, b, c, ...) was never > documented, yet it caused widespread complaints when we disallowed it. *This* is frustrating, indeed. Wandering outside specs is a capital sin[1]. (This is why, for example, I find that most current HTML usage is horrible.) I wonder if, in your place, I would be so soft with users :-). -------------------- [1] Don't throw the first stone, they say! For one, I routinely abuse of immediate automatic finalisation in C-Python, but _only_ after Tim told me it will never go away. Still, there is no promise of this in the manual :-). -- François Pinard http://www.iro.umontreal.ca/~pinard From tim.one@comcast.net Tue May 28 19:12:15 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 28 May 2002 14:12:15 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Message-ID: [Fran=E7ois Pinard] > ... > [1] Don't throw the first stone, they say! For one, I routinely ab= use of > immediate automatic finalisation in C-Python, but _only_ after Tim = told > me it will never go away. Still, there is no promise of this in th= e > manual :-). That's because there isn't a CPython manual: *the* reference manual = is telling the truth about Python-the-language. You'll regret your sinf= ul ways if you ever move to JPython or Vyper, but you won't, so sleep easy. = If Guido ever gets rid of refcounts in CPython-the-implementation, the l= anguage will get renamed to Orlijnthon (or something like that ). From guido@python.org Tue May 28 20:04:36 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 28 May 2002 15:04:36 -0400 Subject: [Python-Dev] Allowing Unicode literals even without Unicode support? In-Reply-To: Your message of "26 May 2002 19:11:21 BST." <2mptziwzhy.fsf@starship.python.net> References: <200205241825.g4OIP2b10779@pcp742651pcs.reston01.va.comcast.net> <2mptziwzhy.fsf@starship.python.net> Message-ID: <200205281904.g4SJ4aT09052@odiug.zope.com> I implemented code that accepts u"..." literals when Unicode is disabled, as long as they don't contain \u, \U or \N escapes. I didn't implement a test for regular non-ASCII characters. > 2) Some library modules just don't seem to function without unicode. > Current bad boys seem to include gettext, urllib (well, this just got > fixed I think), minidom, pyexpat and the email package. I could > imagine the xml modules and the email package just plain not being > supported w/o unicode. gettext, too, I guess. I fixed urllib, and made a fix to the email package so that at least it won't fail on import. The test suite still fails though. > In which case: > > 3) regrtest should probably be taught not to run test_unicode + close > friends in --disable-unicode builds. > > Anyone feel like digging into any of the above? As was said before, let whoever cares about this do it. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue May 28 20:41:49 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 28 May 2002 15:41:49 -0400 Subject: [Python-Dev] Out of date FAQ entries In-Reply-To: Your message of "Fri, 24 May 2002 16:50:46 EDT." <3CEEA7A6.BFB21C87@metaslash.com> References: <3CEC52E2.962B1CB6@metaslash.com> <200205242008.g4OK8hi11791@pcp742651pcs.reston01.va.comcast.net> <3CEEA7A6.BFB21C87@metaslash.com> Message-ID: <200205281941.g4SJfnP09122@odiug.zope.com> > There are a few more: 4.28 & 4.93 seem like they should be merged. > Both talk about compiling Python applications to a stand-alone program. I merged these into 4.28 and deleted 4.93. (Remember, the convention for deleting FAQ entries is to set the title to "Deleted" and remove all contents, except perhaps a pointer.) > Also, I updated 4.2, which should be reviewed to make sure it's > accurate? The title is: Can I create an object class with some > methods implemented in C and others in Python (e.g. through > inheritance)? (Also phrased as: Can I use a built-in type as base > class?) Looks OK to me, except that the ExtensionClass paper is no longer online at digicool.com. Google found a copy for me though: http://debian.acm.ndsu.nodak.edu/doc/python-extclass/ExtensionClass.html I've fixed that (adding a plug for Zope and explaining that ExtensionClass will go away in Zope 3). --Guido van Rossum (home page: http://www.python.org/~guido/) From thomas.heller@ion-tof.com Tue May 28 20:54:13 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Tue, 28 May 2002 21:54:13 +0200 Subject: [Python-Dev] Out of date FAQ entries References: <3CEC52E2.962B1CB6@metaslash.com> <200205242008.g4OK8hi11791@pcp742651pcs.reston01.va.comcast.net> <3CEEA7A6.BFB21C87@metaslash.com> <200205281941.g4SJfnP09122@odiug.zope.com> Message-ID: <028e01c20681$70f13900$e000a8c0@thomasnotebook> From: "Guido van Rossum" > > There are a few more: 4.28 & 4.93 seem like they should be merged. > > Both talk about compiling Python applications to a stand-alone program. > > I merged these into 4.28 and deleted 4.93. (Remember, the convention > for deleting FAQ entries is to set the title to "Deleted" and remove > all contents, except perhaps a pointer.) The result doesn't look too good, IMO. - Although Gordon's starship pages are hopelessly outdated, they are mentioned as the first and fourth possibility... - Does sqfreeze actually work (I never tried it)? - Should /F's squeeze be mentioned? Thomas From Jack.Jansen@oratrix.com Tue May 28 21:00:23 2002 From: Jack.Jansen@oratrix.com (Jack Jansen) Date: Tue, 28 May 2002 22:00:23 +0200 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200205281700.g4SH0do20575@odiug.zope.com> Message-ID: <8BB00CF9-7275-11D6-8740-003065517236@oratrix.com> On dinsdag, mei 28, 2002, at 07:00 , Guido van Rossum wrote: > Now consider my frustration. We go through a *lot* of efforts to make > consecutive releases backwards compatible, to document changes, to > introduce warnings about future incompatible changes, etc. As an aside, note that this backward compatibility is actually a mixed blessing, because it means you don't have to update your modules now, but there will come a time when it is going to bite you. As a personal example: the MacPython toolbox modules haven't been updated to make use of the GC stuff yet (and that's been there since 2.0, no?), let alone the new type system. And these are almost all generated, so it would probably only take a few dozen lines of code to fix them. And the new type system would be a real boon for some of the modules (such as the windowing and dialog stuff), but because there's no real push (i.e. everything still works) nothing has happened yet... -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From mal@lemburg.com Tue May 28 21:03:03 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 28 May 2002 22:03:03 +0200 Subject: [Python-Dev] Allowing Unicode literals even without Unicode support? References: <200205241825.g4OIP2b10779@pcp742651pcs.reston01.va.comcast.net> <2mptziwzhy.fsf@starship.python.net> <200205281904.g4SJ4aT09052@odiug.zope.com> Message-ID: <3CF3E277.9090005@lemburg.com> Guido van Rossum wrote: > I implemented code that accepts u"..." literals when Unicode is > disabled, as long as they don't contain \u, \U or \N escapes. I > didn't implement a test for regular non-ASCII characters. Something I forgot last time around: as soon as we have Python source code encodings, disabling Unicode will no longer be "easy". If you still want to proceed with this, then please add the test for ASCII-compatibility, since otherwise code can easily become non-portable between Python Unicode/non-Unicode builds anymore without anyone noticing. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/ From guido@python.org Tue May 28 21:46:42 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 28 May 2002 16:46:42 -0400 Subject: [Python-Dev] Out of date FAQ entries In-Reply-To: Your message of "Tue, 28 May 2002 21:54:13 +0200." <028e01c20681$70f13900$e000a8c0@thomasnotebook> References: <3CEC52E2.962B1CB6@metaslash.com> <200205242008.g4OK8hi11791@pcp742651pcs.reston01.va.comcast.net> <3CEEA7A6.BFB21C87@metaslash.com> <200205281941.g4SJfnP09122@odiug.zope.com> <028e01c20681$70f13900$e000a8c0@thomasnotebook> Message-ID: <200205282046.g4SKkg709405@odiug.zope.com> > > I merged these into 4.28 and deleted 4.93. (Remember, the convention > > for deleting FAQ entries is to set the title to "Deleted" and remove > > all contents, except perhaps a pointer.) > The result doesn't look too good, IMO. > - Although Gordon's starship pages are hopelessly outdated, they are > mentioned as the first and fourth possibility... > - Does sqfreeze actually work (I never tried it)? > - Should /F's squeeze be mentioned? Oops. Can you please update the FAQ entry then? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue May 28 21:49:45 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 28 May 2002 16:49:45 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Tue, 28 May 2002 22:00:23 +0200." <8BB00CF9-7275-11D6-8740-003065517236@oratrix.com> References: <8BB00CF9-7275-11D6-8740-003065517236@oratrix.com> Message-ID: <200205282049.g4SKnj009428@odiug.zope.com> > As an aside, note that this backward compatibility is actually a > mixed blessing, because it means you don't have to update your > modules now, but there will come a time when it is going to bite > you. When new releases take features away, we will issue warnings as a gentle push. When they add features, I don't know why you *should* use the new features, unless you need them -- and then you have your motivation in your needs. > As a personal example: the MacPython toolbox modules haven't > been updated to make use of the GC stuff yet (and that's been > there since 2.0, no?), But the API was totally changed for 2.2, so you're actually lucky that you didn't do it for 2.0. ;-) > let alone the new type system. And these > are almost all generated, so it would probably only take a few > dozen lines of code to fix them. And the new type system would > be a real boon for some of the modules (such as the windowing > and dialog stuff), but because there's no real push (i.e. > everything still works) nothing has happened yet... I don't think *anything* can be done to force you to start using new optional features... Eventually classic classes will go away, but that will be a long time. --Guido van Rossum (home page: http://www.python.org/~guido/) From mal@lemburg.com Tue May 28 21:56:06 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 28 May 2002 22:56:06 +0200 Subject: [Python-Dev] Re: Stability and change References: <8BB00CF9-7275-11D6-8740-003065517236@oratrix.com> Message-ID: <3CF3EEE6.2030100@lemburg.com> Jack Jansen wrote: > > On dinsdag, mei 28, 2002, at 07:00 , Guido van Rossum wrote: > >> Now consider my frustration. We go through a *lot* of efforts to make >> consecutive releases backwards compatible, to document changes, to >> introduce warnings about future incompatible changes, etc. > > > As an aside, note that this backward compatibility is actually a mixed > blessing, because it means you don't have to update your modules now, > but there will come a time when it is going to bite you. > > As a personal example: the MacPython toolbox modules haven't been > updated to make use of the GC stuff yet (and that's been there since > 2.0, no?), let alone the new type system. And these are almost all > generated, so it would probably only take a few dozen lines of code to > fix them. And the new type system would be a real boon for some of the > modules (such as the windowing and dialog stuff), but because there's no > real push (i.e. everything still works) nothing has happened yet... I don't see the argument here ? You don't seriously expect all extensions, add-ons, existing applications, etc. to be rewritten, patched or changed in other ways with every new Python release, or do you ? Python builds a lot of its popularity on the huge number of add-ons you can download from the web. We should be *very* careful not to break these in ways which make them unusable, because otherwise, we'll have frustrated users and these would be seriously bad for continuing the ride on the wave we're currently seeing. The developer side of things is a little different, since there changes cost time which is usually a rare resource. For small companies it also costs money which they usually don't have. (A migration guide would help both.) In summary, rapid change is not a good approach to a stable product, rapid bug fixing and addition of useful enhancements in backward compatible ways after longer trial phases is, at least IMHO. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/ From guido@python.org Tue May 28 21:53:57 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 28 May 2002 16:53:57 -0400 Subject: [Python-Dev] Allowing Unicode literals even without Unicode support? In-Reply-To: Your message of "Tue, 28 May 2002 22:03:03 +0200." <3CF3E277.9090005@lemburg.com> References: <200205241825.g4OIP2b10779@pcp742651pcs.reston01.va.comcast.net> <2mptziwzhy.fsf@starship.python.net> <200205281904.g4SJ4aT09052@odiug.zope.com> <3CF3E277.9090005@lemburg.com> Message-ID: <200205282053.g4SKrvN09445@odiug.zope.com> > Something I forgot last time around: as soon as we have > Python source code encodings, disabling Unicode will no > longer be "easy". > > If you still want to proceed with this, then please add the > test for ASCII-compatibility, since otherwise code can easily > become non-portable between Python Unicode/non-Unicode builds > anymore without anyone noticing. IMO a simpler solution would be to check for en encoding cookie and complain when Unicode is not enabled. I am not interested in making everything work perfectly with Unicode disabled -- that's not possible any more. I *am* interested in being able to run as much of the test suite as makes sense with Unicode disabled. This mostly means that code that contains tests of the form "is this object a Unicode string?" should get an answer "no" rather than an exception. I am not interested in making code that uses unicode() somehow work without it. --Guido van Rossum (home page: http://www.python.org/~guido/) From pinard@iro.umontreal.ca Tue May 28 22:06:41 2002 From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard) Date: 28 May 2002 17:06:41 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: References: Message-ID: [Tim Peters] > You'll regret your sinful ways if you ever move to JPython or Vyper, > but you won't, so sleep easy. I might, who knows. Then, I'll have no regret and gladly assume my choices. However, if I ever move, I will have the impression of writing like another language, very familiar of course, but a bit less concise, less clean. :-) -- François Pinard http://www.iro.umontreal.ca/~pinard From skip@pobox.com Tue May 28 22:34:26 2002 From: skip@pobox.com (Skip Montanaro) Date: Tue, 28 May 2002 16:34:26 -0500 Subject: [Python-Dev] deprecating string module? Message-ID: <15603.63458.608739.454153@beluga.mojam.com> I know this has been discussed before. In light of the recent PEP8 update: Avoid the use of the string module; instead use string methods. These are always much faster and share the same API with unicode strings. I thought I'd raise it again. How about deprecating the string module? In the past, this has presented two major problems. First, its near ubiquity in Python programs before the availability of string methods. Second, the presence of a few useful data objects (digits, uppercase, etc). The first problem can be solved by extending the deprecation time suitably (two years/four releases?) I think the second problem can be solved by adding those data objects to either sys or locale(preferably the latter). string.letters and its subsets are locale-dependent. In relevant locales I presume string.punctuation and string.whitespace would also be different than their common ASCII elements. Finally, I'd amend the above PEP8 admontion something like Avoid the use of the string module unless backward-compatibility with versions earlier than Python 2.0 is important; instead use string methods. These are always much faster and share the same API with unicode strings. Skip From greg@cosc.canterbury.ac.nz Wed May 29 00:32:17 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Wed, 29 May 2002 11:32:17 +1200 (NZST) Subject: [Python-Dev] deprecating string module? In-Reply-To: <15603.63458.608739.454153@beluga.mojam.com> Message-ID: <200205282332.LAA06511@s454.cosc.canterbury.ac.nz> Skip Montanaro : > the presence of a few useful data objects (digits, uppercase, > etc). ... I think the second problem can be solved by adding those > data objects to either sys or locale(preferably the latter). Making them class attributes of str has also been suggested. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Wed May 29 00:33:36 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Wed, 29 May 2002 11:33:36 +1200 (NZST) Subject: [Python-Dev] _tkinter problem with Stackless In-Reply-To: <3CF31C61.6040606@tismer.com> Message-ID: <200205282333.LAA06514@s454.cosc.canterbury.ac.nz> Christian Tismer : > But now I have a critical section bracked built > in, that prevends stacks from slicing to a given > recursion level. Works just great! It still strikes me as terribly hairy and dangerous. I would have great difficulty trusting it. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Wed May 29 00:48:19 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Wed, 29 May 2002 11:48:19 +1200 (NZST) Subject: [Python-Dev] Bug in PyNumber_InPlacePower implementation? Message-ID: <200205282348.LAA06518@s454.cosc.canterbury.ac.nz> I was looking at the implementation of PyNumber_InPlacePower last night, and something about it struck me as odd. It starts off: if (HASINPLACE(v) && v->ob_type->tp_as_number && v->ob_type->tp_as_number->nb_inplace_power != NULL) { return ternary_op(v, w, z, NB_SLOT(nb_inplace_power), "**="); Now, looking at ternary_op, it appears that under some circumstances this could call the nb_inplace_power slot of the second or third argument before trying the first one: if (slotv) { if (slotw && PyType_IsSubtype(w->ob_type, v->ob_type)) { x = slotw(v, w, z); i.e. if the 2nd argument is a subtype of the 1st, and it has an nb_inplace_power method, it will be called first. This looks wrong to me. Surely only the *first* argument should be checked for an inplace method when doing an inplace operation? That's the way it seems to be for all the other inplace operations. Is this a bug? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tim.one@comcast.net Wed May 29 00:55:19 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 28 May 2002 19:55:19 -0400 Subject: [Python-Dev] _tkinter problem with Stackless In-Reply-To: <200205282333.LAA06514@s454.cosc.canterbury.ac.nz> Message-ID: [Greg Ewing] > It still strikes me as terribly hairy and dangerous. > I would have great difficulty trusting it. Christian doesn't call his enterprise "Mission Impossible Software" for nothing. If you can trust a fabulous hack, what's the point ? From niemeyer@conectiva.com Wed May 29 01:25:29 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Tue, 28 May 2002 21:25:29 -0300 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200205281630.g4SGUPa18148@odiug.zope.com> References: <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> <200205251911.01072.mark@freelance-developer.com> <200205281525.g4SFPxe17659@odiug.zope.com> <3CF3A728.7000604@lemburg.com> <200205281558.g4SFwd317973@odiug.zope.com> <3CF3AE66.8070000@lemburg.com> <200205281630.g4SGUPa18148@odiug.zope.com> Message-ID: <20020528212529.B927@ibook.distro.conectiva> > > Andrew usually puts more focus on new features than changes > > which could cause older applications to fail -- which is what > > you'd expect in a document titled "What's new in ..." :-) I have fixed minor incompatibilities many times. Adrew's work was always my reference for that changes. Eventualy some changes weren't there, and it was always a pleasure to mention them to Andrew. > OK. The question is, who's going to write the migration guide? One > problem is that often specific migration issues don't surface until > after the release is made. I'd rather help Andrew appending additive porting information in his already excellent work. I'm sure he will appreciate any construtive comments about the paper. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From tismer@tismer.com Wed May 29 01:42:04 2002 From: tismer@tismer.com (Christian Tismer) Date: Tue, 28 May 2002 17:42:04 -0700 Subject: [Python-Dev] _tkinter problem with Stackless References: Message-ID: <3CF423DC.7070604@tismer.com> > [Greg Ewing] > >>It still strikes me as terribly hairy and dangerous. >>I would have great difficulty trusting it. > [Tim Peters] > Christian doesn't call his enterprise "Mission Impossible Software" for > nothing. If you can trust a fabulous hack, what's the point ? Sourcery goes immediately Wonders take a little longer :-) Honestly, it might appear hairy and dangerous. But it is very few, very well tested code that enables everything. The old Stackless was really hairy and dangerous, but it cheated not to do so. Instead it hided in uncomprehensible code (which I nearly can't understand any longer as well). What I now have breaks down to the correct decision when a stack split is bearable (i.e. true for the whole pyhon dll) and to identify which extension needs a little help. one-can't-have-Stackless-and-Riskless-same-time - ly y'rs - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From guido@python.org Wed May 29 02:00:11 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 28 May 2002 21:00:11 -0400 Subject: [Python-Dev] Bug in PyNumber_InPlacePower implementation? In-Reply-To: Your message of "Wed, 29 May 2002 11:48:19 +1200." <200205282348.LAA06518@s454.cosc.canterbury.ac.nz> References: <200205282348.LAA06518@s454.cosc.canterbury.ac.nz> Message-ID: <200205290100.g4T10Ci04452@pcp742651pcs.reston01.va.comcast.net> > I was looking at the implementation of PyNumber_InPlacePower last > night, and something about it struck me as odd. It starts off: > > if (HASINPLACE(v) && v->ob_type->tp_as_number && > v->ob_type->tp_as_number->nb_inplace_power != NULL) { > return ternary_op(v, w, z, NB_SLOT(nb_inplace_power), "**="); > > Now, looking at ternary_op, it appears that under some > circumstances this could call the nb_inplace_power slot > of the second or third argument before trying the first > one: > > if (slotv) { > if (slotw && PyType_IsSubtype(w->ob_type, v->ob_type)) { > x = slotw(v, w, z); > > i.e. if the 2nd argument is a subtype of the 1st, and it > has an nb_inplace_power method, it will be called first. > > This looks wrong to me. Surely only the *first* argument > should be checked for an inplace method when doing an > inplace operation? That's the way it seems to be for > all the other inplace operations. Is this a bug? This is exactly the same as what binary_op1() does. I added this twist intentionally because of a use case where a subclass of a numeric type wants to override a binary (or ternary) operator defined by the base class. If the subclass wasn't tried first, it would never be able to override the case where a base class instance is the left operand, because the base class implementation is generally happy to accept a subclass instance as the right operand. (I admit that a comment explaining this would have been handy. :-) Do you have a use case where this does the wrong thing, or is this just a theoretical musing? --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Wed May 29 02:52:39 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 28 May 2002 21:52:39 -0400 Subject: [Python-Dev] Lazily GC tracking tuples In-Reply-To: Message-ID: [Kevin Jacobs, on Neil's tuple-untracking patch] > It doesn't seem to make much difference in our app. There seems > to be some speedup, but nothing dramatic. Turning GC on and off at the > right times is still slightly faster. I'm confused. You earlier said: I've found one case where the garbage collector _does_ cause serious performance problems, and coincidentally enough it is due to the creation of zillions of cycle-less tuples. Our current solution has been to disable GC in some parts of our code, and then manually trigger collection at the correct points. Now I'm hearing that turning GC on and off is only "slightly faster", and that the speedup is so small you're not even sure there is one ("turning GC on and off" is "slightly faster" than a trial where there "seems to be some speedup"). This is not "serious performance problems" except to nanosecond counters like me . > The good news is that another (unrelated) part of our code just became > about 20-40% faster with this patch, though I need to do some fairly > major surgery to isolate why this is so. Keep us informed! I suspect you're suffering from an app that's more than 20 lines long; I try to stay away from those. most-days-it's-hard-enough-out-thinking-one-line-ly y'rs - tim From greg@cosc.canterbury.ac.nz Wed May 29 03:38:44 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Wed, 29 May 2002 14:38:44 +1200 (NZST) Subject: [Python-Dev] Bug in PyNumber_InPlacePower implementation? In-Reply-To: <200205290100.g4T10Ci04452@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200205290238.OAA06564@s454.cosc.canterbury.ac.nz> > This is exactly the same as what binary_op1() does. I added this > twist intentionally because of a use case where a subclass of a > numeric type wants to override a binary (or ternary) operator defined > by the base class. I don't think you understand what I mean. I'm talking about *in-place* operations. For in-place binary operations we have, e.g. INPLACE_BINOP(PyNumber_InPlaceSubtract, nb_inplace_subtract, nb_subtract, "-=") ^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^ Both the in-place and non-in-place slots are passed to binary_iop, which uses the in-place slot for the first operand and the non-in-place slot for the second. But ternary_op only gets *one* slot passed to it, which it uses for everything. When doing an in-place ternary op, this is the in-place slot. What this seems to mean is that, if we have isinstance(a, A) and isinstance(b, B) and issubclass(B, A), then a **= b has the potential to in-place-modify b instead of a! It seems to me that there ought to be a ternary_iop routine that does for ternary ops what binary_iop does for binary ones. > Do you have a use case where this does the wrong thing, or is this > just a theoretical musing? It's a theoretical musing that came out of my attempts to figure out whether, when one is implementing an nb_inplace_power slot for a new type, the first argument is guaranteed to "self". I need to know this so that Pyrex can do the right thing. I've done some experiments with Python code, and it seems to do the right thing in terms of calling the correct __pow__, __rpow__ and __ipow__ methods. But I can't tell from that what's really going on at the typeslot level. I've looked at the code for the nb_inplace_power of instance objects (the only example I can find of such a method!), and it seems to assume that the first argument is always "self". But I can't see how this is guaranteed. In short, I'm confused! Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From pf@artcom-gmbh.de Wed May 29 07:26:57 2002 From: pf@artcom-gmbh.de (Peter Funk) Date: Wed, 29 May 2002 08:26:57 +0200 (CEST) Subject: [Python-Dev] deprecating string module? In-Reply-To: <15603.63458.608739.454153@beluga.mojam.com> from Skip Montanaro at "May 28, 2002 04:34:26 pm" Message-ID: Hi, Skip Montanaro wrote: > I know this has been discussed before. In light of the recent PEP8 update: > > Avoid the use of the string module; instead use string methods. > These are always much faster and share the same API with unicode > strings. > > I thought I'd raise it again. > > How about deprecating the string module? I'm absolutely against depreceating the string module! Instead I suggest to change the cited wording from PEP8 into Avoid the use of the string module unless you want to keep compatibility with Python 1.5.2; instead use the string methods introduced in later releases of Python. These are always much faster and share the same API with unicode strings. Python 1.5.2 is still very important and even the latest Red Hat distributions still use it. Since the bytecode (.pyc) has changed inbetween, we decided to ship all our Python Software with 1.5.2 and backport useful new library modules from 2.x to 1.5.2 if we want to use them. Of course the edits needed to turn string method calls back into string module function calls for 1.5.2 are trivial but they are nevertheless tedious. Very often the overall performance win of using string methods will be so negligible, that I tend to believe that the CPU cycles saved due to this change don't sum up to the amount of programmer time wasted by editing Python software from string function calls into string method calls or vice versa. Regards, Peter -- Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260 office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen, Germany) From martin@v.loewis.de Wed May 29 07:34:42 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 29 May 2002 08:34:42 +0200 Subject: [Python-Dev] deprecating string module? In-Reply-To: <15603.63458.608739.454153@beluga.mojam.com> References: <15603.63458.608739.454153@beluga.mojam.com> Message-ID: Skip Montanaro writes: > How about deprecating the string module? In the past, this has presented > two major problems. First, its near ubiquity in Python programs before the > availability of string methods. Second, the presence of a few useful data > objects (digits, uppercase, etc). The first problem can be solved by > extending the deprecation time suitably (two years/four releases?) I think > the second problem can be solved by adding those data objects to either sys > or locale(preferably the latter). I think one needs to offer the alternatives first, and deprecate the module then. So have alternatives for *everything* in string available in 2.3, deprecate it in 2.4, add a DeprecationWarning in 2.5. Regards, Martin From mal@lemburg.com Wed May 29 08:44:52 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 29 May 2002 09:44:52 +0200 Subject: [Python-Dev] deprecating string module? References: Message-ID: <3CF486F4.6070302@lemburg.com> Peter Funk wrote: > Hi, > > Skip Montanaro wrote: > >>I know this has been discussed before. In light of the recent PEP8 update: >> >> Avoid the use of the string module; instead use string methods. >> These are always much faster and share the same API with unicode >> strings. >> >>I thought I'd raise it again. >> >>How about deprecating the string module? > > > I'm absolutely against depreceating the string module! > Instead I suggest to change the cited wording from PEP8 into > > Avoid the use of the string module unless you want to keep > compatibility with Python 1.5.2; instead use the string methods > introduced in later releases of Python. These are always much > faster and share the same API with unicode strings. > > Python 1.5.2 is still very important and even the latest Red Hat > distributions still use it. Since the bytecode (.pyc) has changed > inbetween, we decided to ship all our Python Software with 1.5.2 > and backport useful new library modules from 2.x to 1.5.2 if we want > to use them. > > Of course the edits needed to turn string method calls back into > string module function calls for 1.5.2 are trivial but they are > nevertheless tedious. Very often the overall performance win of > using string methods will be so negligible, that I tend to believe > that the CPU cycles saved due to this change don't sum up to the > amount of programmer time wasted by editing Python software from > string function calls into string method calls or vice versa. There's a good example of what I was talking about in the "Stability and Change" thread: careless use of deprecations would cause serious grief among developers and certainly also users (who are then confronted with tons of warnings which they don't understand). Now for Python modules the situation is a little better, since it is possible to wrap up those deprecated modules in a distutils package for users to install on top of their Python install. However, when touching core features of the language things become completely unmanageable. The reason is that various extensions out there start to rely on features which are only available in later releases while others are not migrated yet to these versions and would result in endless streams of deprecation warnings or even failures. As a result you'd lose the ability to fish in the sea of Python extensions... BTW, I don't see the tradeoff in saving 11k worth of diskspace for string.py compared to the good vibes we lose in the Python world for this. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/ From skip@pobox.com Wed May 29 12:04:43 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 29 May 2002 06:04:43 -0500 Subject: [Python-Dev] deprecating string module? In-Reply-To: References: <15603.63458.608739.454153@beluga.mojam.com> Message-ID: <15604.46539.444906.980616@12-248-41-177.client.attbi.com> Peter> Python 1.5.2 is still very important and even the latest Red Hat Peter> distributions still use it. I think the RedHat/1.5.2 thing is (or should soon be) a red herring. In my message I suggested a two-year/four-release deprecation schedule. That could easily be adjusted depending on what RH decides to do: Research Triangle Park, NC (AP) - RedHat Software announced today that it plans to release RedHat 7.77 during the first week of October 2018. Significant new end user software shipped with this release will include XFree86 17.1.7 and gcc 5.2. Python 1.5.2 has proven amazingly stable however, and too many people rely on it, so it will not be phased out in favor of the current version of Python, 3.15, which was released in August of 2016. Sources close to PythonLabs told the Associated Press that after much discussion on the python-dev mailing list, Guido van Rossum, the creator of the Python programming language, has decided the Python 4.0 interpreter will ship with an automatic 1.5.2 compatibility mode which will be automatically activated when a .pyc file with the proper magic number is loaded. There is no firm word yet when 4.0 is expected to ship. Interested parties are urged to keep their eye on PEP 4327: "Python 4.0 Release Schedule" for information about the planned dates for initial alpha testing. Tests posted to comp.lang.python by Tim Peters suggest that on a large corpus of 20-line floating point and long integer math test scripts rigorously developed over the past 15 years performance was not significantly affected. Sources at Zope Corporation could not be reached for comment in time for this story to confirm any of this information. :-) Skip From skip@pobox.com Wed May 29 12:24:22 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 29 May 2002 06:24:22 -0500 Subject: [Python-Dev] deprecating string module? In-Reply-To: References: <15603.63458.608739.454153@beluga.mojam.com> Message-ID: <15604.47718.334686.681186@12-248-41-177.client.attbi.com> >> How about deprecating the string module? Martin> I think one needs to offer the alternatives first, and deprecate Martin> the module then. So have alternatives for *everything* in string Martin> available in 2.3, deprecate it in 2.4, add a DeprecationWarning Martin> in 2.5. I don't see that as being a major problem. As I mentioned in my reply to Peter Funk, I believe you can probably let the deprecation schedule slip according to how important continued 1.5.2 compliance is determined to be down the road. That is, deprecate it no earlier than 2.4 and add a DeprecationWarning no sooner than one or two releases after that. The following string module functions don't seem to have equivalent string methods: zfill - only because string.zfill accepts non-string arguments, otherwise it is already available capwords - I believe it was explicitly decided to not implement this as a string method maketrans - shouldn't be too hard to rewrite in C as a method of str. Also, the data objects obviously have to be put somewhere. Before doing any of this, we need to decide where this stuff belongs. I doubt the technical effort involved will be too challenging. Skip From skip@pobox.com Wed May 29 12:27:52 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 29 May 2002 06:27:52 -0500 Subject: [Python-Dev] deprecating string module? In-Reply-To: <3CF486F4.6070302@lemburg.com> References: <3CF486F4.6070302@lemburg.com> Message-ID: <15604.47928.443522.504730@12-248-41-177.client.attbi.com> mal> There's a good example of what I was talking about in the mal> "Stability and Change" thread: careless use of deprecations would mal> cause serious grief among developers and certainly also users (who mal> are then confronted with tons of warnings which they don't mal> understand). I don't believe I said "let's carelessly deprecate the string module". mal> Now for Python modules the situation is a little better, since it mal> is possible to wrap up those deprecated modules in a distutils mal> package for users to install on top of their Python install. I don't see why this can't be done for string.py. mal> BTW, I don't see the tradeoff in saving 11k worth of diskspace for mal> string.py compared to the good vibes we lose in the Python world mal> for this. It has nothing to do with 11k worth of disk space and everything to do with "there's one (best) way to do it". Skip From mal@lemburg.com Wed May 29 12:54:02 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 29 May 2002 13:54:02 +0200 Subject: [Python-Dev] deprecating string module? References: <3CF486F4.6070302@lemburg.com> <15604.47928.443522.504730@12-248-41-177.client.attbi.com> Message-ID: <3CF4C15A.7070606@lemburg.com> Skip Montanaro wrote: > mal> There's a good example of what I was talking about in the > mal> "Stability and Change" thread: careless use of deprecations would > mal> cause serious grief among developers and certainly also users (who > mal> are then confronted with tons of warnings which they don't > mal> understand). > > I don't believe I said "let's carelessly deprecate the string module". Looking at the subject line, you're almost there ;-) > mal> Now for Python modules the situation is a little better, since it > mal> is possible to wrap up those deprecated modules in a distutils > mal> package for users to install on top of their Python install. > > I don't see why this can't be done for string.py. Sure it can; don't see what we gain, though. Note that I was just addressing the more general case discussed in the other thread "Stability and change". > mal> BTW, I don't see the tradeoff in saving 11k worth of diskspace for > mal> string.py compared to the good vibes we lose in the Python world > mal> for this. > > It has nothing to do with 11k worth of disk space and everything to do with > "there's one (best) way to do it". That's a silly phrase. I agree that there should always be a straight-forward and intuitive way to do things, but I don't go for such a dogma type argument and I'm sure you don't either :-) The intuitive way to do e.g. string.find has changed from import string string.find(x,y) to x.find(y) and that's worth making clear, but forcing the Python people to change their previously adapted intuition sounds too much like dictatorship to me. If I want to write code which works in all Python versions starting from 1.5.2 onwards, I should be able to do this. Deprecating important modules like string and types and then removing them altogether makes this impossible without extra hackery or providing special bwcompat packages to the users. Just look at the hoops which the email package has to go through to maintain such compatibility. That's not the old Python philosophy I'm used to and that's also why I dislike the idea to rip out code for no apparently needed reason at all. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/ From skip@pobox.com Wed May 29 13:05:15 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 29 May 2002 07:05:15 -0500 Subject: [Python-Dev] deprecating string module? In-Reply-To: <3CF4C15A.7070606@lemburg.com> References: <3CF486F4.6070302@lemburg.com> <15604.47928.443522.504730@12-248-41-177.client.attbi.com> <3CF4C15A.7070606@lemburg.com> Message-ID: <15604.50171.868339.126122@12-248-41-177.client.attbi.com> >> It has nothing to do with 11k worth of disk space and everything to >> do with "there's one (best) way to do it". mal> That's a silly phrase. I agree that there should always be a mal> straight-forward and intuitive way to do things, but I don't go for mal> such a dogma type argument and I'm sure you don't either :-) It's not dogma. It's a catch phrase, I agree, but I think the concept does have merit. mal> If I want to write code which works in all Python versions starting mal> from 1.5.2 onwards, I should be able to do this. This seems like a good project for the Python Business Forum. How about developing some hard numbers that indicate what fraction of Python users use what version of Python? It would have to be sensitive to the type of users and their platform, but would be useful information for all Python programmers to have. I am personally getting rather tired of the "you can't do that! it's not 1.5.2-compatible!" whine without any justification other than "RedHat still ships 1.5.2". -- Skip Montanaro (skip@pobox.com - http://www.mojam.com/) Boycott Netflix - they spam From pf@artcom-gmbh.de Wed May 29 13:10:36 2002 From: pf@artcom-gmbh.de (Peter Funk) Date: Wed, 29 May 2002 14:10:36 +0200 (CEST) Subject: [Python-Dev] deprecating string module? In-Reply-To: <15604.47928.443522.504730@12-248-41-177.client.attbi.com> from Skip Montanaro at "May 29, 2002 06:27:52 am" Message-ID: Hi, Skip Montanaro: > mal> BTW, I don't see the tradeoff in saving 11k worth of diskspace for > mal> string.py compared to the good vibes we lose in the Python world > mal> for this. > > It has nothing to do with 11k worth of disk space and everything to do with > "there's one (best) way to do it". In some situations application of zen-rule number 8 (this is "* Although practicality beats purity.") is stronger than zen-rule number 12 (this is "* There should be one-- and preferably only one --obvious way to do it."), which you cited somewhat sloppy above. In my opponion the string module is one such situation and another one is the '<>' operator. Most of my employees work with Modula-2 a lot and we have a huge code base. So they prefer to use '<>' over '!=' in Python also and they will not stop to do so, although the use of '<>' is discouraged in the Python documentation. Regards, Peter -- Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260 office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen, Germany) From tl_news@nexgo.de Wed May 29 13:17:39 2002 From: tl_news@nexgo.de (Tino Lange) Date: Wed, 29 May 2002 14:17:39 +0200 Subject: [Python-Dev] deprecating string module? Message-ID: <200205291417.40030.tl_news@nexgo.de> Skip Montanaro wrote: > It has nothing to do with 11k worth of disk space and everything to do with > "there's one (best) way to do it". Hi! String method's are really nice! (By the way: I miss a length() method) But in some cases the old-school string module yields the more readable code. So the "one (best) way" is not always the clearest to read and understand. I cannot see/understand why it's the "best" way to write for example fromaddr = " ".join(map(lambda x: x[0] , email.Header.decode_header(fromaddr[1]))) with this (IMHO not intuitive empty string " ") instead of fromaddr = string.join(map(lambda x: x[0] , email.Header.decode_header(fromaddr[1]))) It's much cleaner to read and understand with string instead of " " - at least for beginners, or? Tino From s_lott@yahoo.com Wed May 29 13:27:45 2002 From: s_lott@yahoo.com (Steven Lott) Date: Wed, 29 May 2002 05:27:45 -0700 (PDT) Subject: [Python-Dev] deprecating string module? In-Reply-To: <200205291417.40030.tl_news@nexgo.de> Message-ID: <20020529122745.74975.qmail@web9605.mail.yahoo.com> I find the big reason for deprecating string has to be the evolution toward methods and away from modules. e.g.: > > String method's are really nice! (By the way: I miss a > length() method) Deprecation of string is not confusing for new users of Python, they will start with string methods, having no use for the module. It is not confusing for experienced users, they know what deprecation is, and can probably work through the rationale for moving away from the old module technique and toward the new class/type technique. Somewhere in the middle there might be some people who figured the string module out, but never got a grip on string methods. I don't think this is the majority. The true majority are the "yet to start" users, for whom the String class will be the only thing they ever use; irrespective of the deprecation state of string. ===== -- S. Lott, CCP :-{) S_LOTT@YAHOO.COM http://www.mindspring.com/~slott1 Buccaneer #468: KaDiMa Macintosh user: drinking upstream from the herd. __________________________________________________ Do You Yahoo!? Yahoo! - Official partner of 2002 FIFA World Cup http://fifaworldcup.yahoo.com From fredrik@pythonware.com Wed May 29 13:41:47 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Wed, 29 May 2002 14:41:47 +0200 Subject: [Python-Dev] deprecating string module? References: <3CF486F4.6070302@lemburg.com> <15604.47928.443522.504730@12-248-41-177.client.attbi.com> Message-ID: <062d01c2070e$33945190$0900a8c0@spiff> skip wrote: > It has nothing to do with 11k worth of disk space and everything to do = with > "there's one (best) way to do it". in this case, it sounds more like "purity over practicality" (or maybe "I'm bored, let's get some discussion going") and frankly, it doesn't sound much like you either. didn't you use to be a rather practical programmer? From mal@lemburg.com Wed May 29 13:44:24 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 29 May 2002 14:44:24 +0200 Subject: [Python-Dev] deprecating string module? References: <3CF486F4.6070302@lemburg.com> <15604.47928.443522.504730@12-248-41-177.client.attbi.com> <3CF4C15A.7070606@lemburg.com> <15604.50171.868339.126122@12-248-41-177.client.attbi.com> Message-ID: <3CF4CD28.7050601@lemburg.com> Skip Montanaro wrote: > >> It has nothing to do with 11k worth of disk space and everything to > >> do with "there's one (best) way to do it". > > mal> That's a silly phrase. I agree that there should always be a > mal> straight-forward and intuitive way to do things, but I don't go for > mal> such a dogma type argument and I'm sure you don't either :-) > > It's not dogma. It's a catch phrase, I agree, but I think the concept does > have merit. Sure it does, but not in the sense of "there's only one way to do it: mine" ;-) > mal> If I want to write code which works in all Python versions starting > mal> from 1.5.2 onwards, I should be able to do this. > > This seems like a good project for the Python Business Forum. How about > developing some hard numbers that indicate what fraction of Python users use > what version of Python? It would have to be sensitive to the type of users > and their platform, but would be useful information for all Python > programmers to have. Indeed. > I am personally getting rather tired of the "you can't do that! it's not > 1.5.2-compatible!" whine without any justification other than "RedHat still > ships 1.5.2". eGenix and most other companies providing Python extensions I know of still support Python 1.5.2. Many users still like 1.5.2 for its stability and better performance. For some statistics, I get around 4000 hits on Google for "Python 1.4" 52000 hits on Google for "Python 1.5" 31000 hits on Google for "Python 2.0" 44000 hits on Google for "Python 2.1" 36000 hits on Google for "Python 2.2" 2000 hits on Google for "Python 2.3" From this I take that the world has moved away from 1.4 but we're still far from saying Python 1.5.2 is dead. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/ From fredrik@pythonware.com Wed May 29 13:35:43 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Wed, 29 May 2002 14:35:43 +0200 Subject: [Python-Dev] deprecating string module? References: <20020529122745.74975.qmail@web9605.mail.yahoo.com> Message-ID: <062c01c2070e$338b29d0$0900a8c0@spiff> Steven Lott wrote: > The true majority are the "yet to start" users, for whom the > String class will be the only thing they ever use; irrespective > of the deprecation state of string. you mean we should burn all existing python books, and start over from scratch? From sholden@holdenweb.com Wed May 29 13:57:01 2002 From: sholden@holdenweb.com (Steve Holden) Date: Wed, 29 May 2002 08:57:01 -0400 Subject: [Python-Dev] deprecating string module? References: <3CF486F4.6070302@lemburg.com> <15604.47928.443522.504730@12-248-41-177.client.attbi.com> <3CF4C15A.7070606@lemburg.com> Message-ID: <062701c20710$5679a8c0$6300000a@holdenweb.com> ----- Original Message ----- From: "M.-A. Lemburg" To: Cc: Sent: Wednesday, May 29, 2002 7:54 AM Subject: Re: [Python-Dev] deprecating string module? > Skip Montanaro wrote: [on carelessness] > > > mal> Now for Python modules the situation is a little better, since it > > mal> is possible to wrap up those deprecated modules in a distutils > > mal> package for users to install on top of their Python install. > > > > I don't see why this can't be done for string.py. > > Sure it can; don't see what we gain, though. > Well, one thing we would gain is that new users would be educated by the depracation warnings. If nslookup is allowed to tell me """Note: nslookup is deprecated and may be removed from future releases. Consider using the `dig' or `host' programs instead. Run nslookup with the `-sil[ent]' option to prevent this message from appearing.""" then it seems reasonable for Python to say "If you carry on programming like this you will (albeit eventually) come a cropper". The problem that (for instance) the Python Business Thingy might have with this is that it looks like they don't know their job if their software starts to barf all over the operating environment when a new version of Python is deployed. Their customers (or their ISPs) will report that a Python upgrade has "broken something", and that's not good PR. > Note that I was just addressing the more general case discussed > in the other thread "Stability and change". > > > mal> BTW, I don't see the tradeoff in saving 11k worth of diskspace for > > mal> string.py compared to the good vibes we lose in the Python world > > mal> for this. > > > > It has nothing to do with 11k worth of disk space and everything to do with > > "there's one (best) way to do it". > > That's a silly phrase. I agree that there should always be a > straight-forward and intuitive way to do things, but I don't go > for such a dogma type argument and I'm sure you don't > either :-) > "That's a silly phrase" is a silly phrase <0.36759 wink>. Dogma, shmogma. None of us buy the "there can be only one way to do it" philosophy, which clearly isn't Pythonic. Not when I'm the arbiter, anyway. While there will *always* be "more than one way to do it", I might be persuaded that a "single most obvious" way is desirable, and the more common the use cases the more desirable it is. Unless it's *really, waaay* too slow. The important thing is to cut down the repertoire of "micro design patterns" commonly used by literate Python programmers. This will tend to help the readability of actively maintained code. > The intuitive way to do e.g. string.find has changed from > > import string > string.find(x,y) > Clearly not for everybody . > to > > x.find(y) > > and that's worth making clear, but forcing the Python people > to change their previously adapted intuition sounds too much > like dictatorship to me. > Well, I think you're both right (sickening, isn't it). The problem is the "Python-puke" from production systems. If a script is run frequently (as a popular CGI, or similar) it's going to puke a lot, and (to the sysadmins) probably quite visibly. > If I want to write code which works in all Python versions starting > from 1.5.2 onwards, I should be able to do this. Deprecating > important modules like string and types and then removing them > altogether makes this impossible without extra hackery or > providing special bwcompat packages to the users. > I can understand this, but isn't "1.5.2" a little arbitrary. And when does "1.5.2" become something else, and should it be "2.2.3" or "3.0.7"? There will, inevitably, come a point when Python X.Y's new features offer radically more efficient and/or easily-maintained paradigms for certain tasks to the extent that maintaining 1.5.2 compatibility will be a real loss. > Just look at the hoops which the email package has to go through > to maintain such compatibility. That's not the old Python > philosophy I'm used to and that's also why I dislike the > idea to rip out code for no apparently needed reason at all. > I understand both the words and the feelings behind this statement, and support them both. The warning system seems fine to me, but... Suppose that a feature used in existing [Python] code is depracated as a result of a minor (N.X -> N.Y). This should not affect production systems, but should definitely instigate warnings in the development environment. If the depracation warnings were deactivated *for code sourced before the release date* then 1. a commercial "product" of the Python Business Thingy would have the maximum lifetime consistent with feature support in Python, 2. developers would know for three release cycles how to maintain their code, and 3. if an unsuspecting sysadmin makes a minor edit to the code (aha, this release is being "actively maintained") then that particular module starts barfing. Of course this does have the slight downside that when feature support *is* finally removed, death is immediate and terminal. This might not be an altogether bad thing, and in a web server might be regarded as indicating time to seek the Happy Hunting Grounds. regards ----------------------------------------------------------------------- Steve Holden http://www.holdenweb.com/ Python Web Programming http://pydish.holdenweb.com/pwp/ ----------------------------------------------------------------------- From mal@lemburg.com Wed May 29 14:03:27 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 29 May 2002 15:03:27 +0200 Subject: [Python-Dev] Re: Stability and change References: <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> <200205251911.01072.mark@freelance-developer.com> <200205281525.g4SFPxe17659@odiug.zope.com> <3CF3A728.7000604@lemburg.com> <200205281558.g4SFwd317973@odiug.zope.com> <3CF3AE66.8070000@lemburg.com> <20020528165933.GD8699@mems-exchange.org> Message-ID: <3CF4D19F.50607@lemburg.com> akuchlin@mems-exchange.org wrote: > On Tue, May 28, 2002 at 06:20:54PM +0200, M.-A. Lemburg wrote: > >>The migration path, including hints on how to port Python applications > >>from one version to the next, how to work around deprecations, > >>tips on quick ways to find problem areas in the code, etc. > > > Part of the problem is that I work from the CVS logs and from the > Misc/NEWS file, and people don't always indicate when a potential > incompatibility is introduced. (That may not be clear to the person > making the change, of course, if they think they're changing an > obscure bit of code.) > > I'd certainly like to point out required changes. The 2.0 document > had a "Porting to 2.0" section, but now I generally try to cover such > issues in the section for the relevant PEP. You're doing a great hob here, Andrew ! What should we do to make your work easier ? -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/ From s_lott@yahoo.com Wed May 29 14:05:53 2002 From: s_lott@yahoo.com (Steven Lott) Date: Wed, 29 May 2002 06:05:53 -0700 (PDT) Subject: [Python-Dev] deprecating string module? In-Reply-To: <062c01c2070e$338b29d0$0900a8c0@spiff> Message-ID: <20020529130553.21468.qmail@web9602.mail.yahoo.com> Yes. Seriously. As these tools evolve, there is much that is new and good and much that is old that should be (gracefully) deprecated for a few years and then retired. Some modules are interesting only for backward compatibility. Also, the organization of modules in the LIB manual makes very little sense to new users. Clearly, this organization evolved organically with the language, but perhaps it is time to refactor that module tree. Reaching back a few decades - I read in my history books that the Algol 68 effort fragmented into a flame war after the very successful Algol 60 effort. The flame war lead to three parallel efforts that eventually gave us Algol 68 (any users?), PL/1 and Pascal. Which had the biggest user base? I think it was Pascal, the one that went for the fewest-best features. (Or possibly it was the French mathematician after whom it was named, hard to say for sure.) I do not suggest that Python usage will fragment; rather I suggest that a focus on a few important features is the most beneficial. --- Fredrik Lundh wrote: > Steven Lott wrote: > > > The true majority are the "yet to start" users, for whom the > > String class will be the only thing they ever use; > irrespective > > of the deprecation state of string. > > you mean we should burn all existing python books, and > start over from scratch? > > > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev ===== -- S. Lott, CCP :-{) S_LOTT@YAHOO.COM http://www.mindspring.com/~slott1 Buccaneer #468: KaDiMa Macintosh user: drinking upstream from the herd. __________________________________________________ Do You Yahoo!? Yahoo! - Official partner of 2002 FIFA World Cup http://fifaworldcup.yahoo.com From barry@zope.com Wed May 29 14:11:02 2002 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 29 May 2002 09:11:02 -0400 Subject: [Python-Dev] deprecating string module? References: <3CF486F4.6070302@lemburg.com> <15604.47928.443522.504730@12-248-41-177.client.attbi.com> <3CF4C15A.7070606@lemburg.com> Message-ID: <15604.54118.851422.551535@anthem.wooz.org> >>>>> "MAL" == M writes: MAL> Just look at the hoops which the email package has to go MAL> through to maintain such compatibility. That's not a great analogy though. email's got those compatibility modules largely because generators were such a natural fit for three of the methods. I wrote them in Py2.2 first to learn how to use generators and then had to figure out how to backport them. email only works with Python 2.1 and beyond, with a strong bias against the string module. All in all, I'm still not convinced that the compatibility modules are worth it, but for now, I'm sticking with them. They definitely cause pain with distutils. ;/ There's a price to be paid for living on the bleeding edge (or scabbed edge for Python 2.2 ;). But there's also a price to pay for maintaining backwards compatibility and sticking with that forever, More important (to me) than the feature or perceived stability of Py1.5.2, is that so much has been fixed by Python 2.1.3 that it made no sense to use anything earlier. I think/hope that the PBF's Pytie releases will help ease that pain. -Barry From aahz@pythoncraft.com Wed May 29 14:23:28 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 29 May 2002 09:23:28 -0400 Subject: [Python-Dev] deprecating string module? In-Reply-To: <15604.50171.868339.126122@12-248-41-177.client.attbi.com> References: <3CF486F4.6070302@lemburg.com> <15604.47928.443522.504730@12-248-41-177.client.attbi.com> <3CF4C15A.7070606@lemburg.com> <15604.50171.868339.126122@12-248-41-177.client.attbi.com> Message-ID: <20020529132328.GB10613@panix.com> On Wed, May 29, 2002, Skip Montanaro wrote: > > I am personally getting rather tired of the "you can't do that! it's not > 1.5.2-compatible!" whine without any justification other than "RedHat still > ships 1.5.2". So don't keep proposing changes that break 1.5.2 compatibility. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "In the end, outside of spy agencies, people are far too trusting and willing to help." --Ira Winkler From aleax@aleax.it Wed May 29 14:24:14 2002 From: aleax@aleax.it (Alex Martelli) Date: Wed, 29 May 2002 15:24:14 +0200 Subject: [Python-Dev] deprecating string module? In-Reply-To: <20020529130553.21468.qmail@web9602.mail.yahoo.com> References: <20020529130553.21468.qmail@web9602.mail.yahoo.com> Message-ID: On Wednesday 29 May 2002 03:05 pm, Steven Lott wrote: ... > parallel efforts that eventually gave us Algol 68 (any users?), > PL/1 and Pascal. Which had the biggest user base? I think it > was Pascal, the one that went for the fewest-best features. > > (Or possibly it was the French mathematician after whom it was > named, hard to say for sure.) As the small user bases (compared to their merits) of Haskell and Eiffel show, naming languages after either mathematicians OR Frenchmen is no guarantee of big user bases. Maybe you do need both. I guess we could try inventing a language named "Cauchy" or "Fermat" to double-check. Alex From fredrik@pythonware.com Wed May 29 14:28:08 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Wed, 29 May 2002 15:28:08 +0200 Subject: [Python-Dev] deprecating string module? References: <3CF486F4.6070302@lemburg.com> <15604.47928.443522.504730@12-248-41-177.client.attbi.com> <3CF4C15A.7070606@lemburg.com> <062701c20710$5679a8c0$6300000a@holdenweb.com> Message-ID: <002801c20714$ad139fc0$0900a8c0@spiff> > > If I want to write code which works in all Python versions starting > > from 1.5.2 onwards, I should be able to do this. Deprecating > > important modules like string and types and then removing them > > altogether makes this impossible without extra hackery or > > providing special bwcompat packages to the users. > > > I can understand this, but isn't "1.5.2" a little arbitrary. have you tried migrating a huge python system from 1.5.2 to 2.x? if so, what did you learn? From aahz@pythoncraft.com Wed May 29 14:30:34 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 29 May 2002 09:30:34 -0400 Subject: [Python-Dev] deprecating string module? In-Reply-To: <15604.46539.444906.980616@12-248-41-177.client.attbi.com> References: <15603.63458.608739.454153@beluga.mojam.com> <15604.46539.444906.980616@12-248-41-177.client.attbi.com> Message-ID: <20020529133034.GC10613@panix.com> On Wed, May 29, 2002, Skip Montanaro wrote: > > Peter> Python 1.5.2 is still very important and even the latest Red Hat > Peter> distributions still use it. > > I think the RedHat/1.5.2 thing is (or should soon be) a red herring. In my > message I suggested a two-year/four-release deprecation schedule. That > could easily be adjusted depending on what RH decides to do: > > Research Triangle Park, NC (AP) - RedHat Software announced today that > it plans to release RedHat 7.77 during the first week of October 2018. > Significant new end user software shipped with this release will include > XFree86 17.1.7 and gcc 5.2. Python 1.5.2 has proven amazingly stable > however, and too many people rely on it, so it will not be phased out in > favor of the current version of Python, 3.15, which was released in > August of 2016. This is ridiculous. Think about this: how long after ANSI C was released did Python continue to support K&R? That should give you a feeling for long it should take to continue to support old releases for core features. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "In the end, outside of spy agencies, people are far too trusting and willing to help." --Ira Winkler From aleax@aleax.it Wed May 29 14:52:49 2002 From: aleax@aleax.it (Alex Martelli) Date: Wed, 29 May 2002 15:52:49 +0200 Subject: [Python-Dev] deprecating string module? In-Reply-To: <062c01c2070e$338b29d0$0900a8c0@spiff> References: <20020529122745.74975.qmail@web9605.mail.yahoo.com> <062c01c2070e$338b29d0$0900a8c0@spiff> Message-ID: On Wednesday 29 May 2002 02:35 pm, Fredrik Lundh wrote: > Steven Lott wrote: > > The true majority are the "yet to start" users, for whom the > > String class will be the only thing they ever use; irrespective > > of the deprecation state of string. > > you mean we should burn all existing python books, and > start over from scratch? All author of Python books should heartily second this recommendation -- unless the book is right in the sweet spot of its lifecycle curve, where it's selling well without needing any edits and updates, of course (so, please let's do it AT ONCE, before the Cookbook and Nutshell come out, or better wait a hopefully long while for their sales to get on a downward slope). After all, in terms of editing work, moving any reference to the string module to an appendix on "old deprecated stuff that you only need to know when migrating old code to new Python releases" is pretty modest, and yet it will let us make "new importantly updated editions" and sell a bunch more to suck^H^H^H^H discerning book buyers who _know_ they must always have fully updated tomes. Then when THAT sales boost has died down, we can make another killing by introducing, say, a boolean type with two constants True and False that act just about the same as 1 and 0 but also let us make another crucially updated edition (with a gain in readability for our prose, too). Hmmm, maybe _this_ one is a bit too obvious as a ploy to sell new updated editions. But I'm sure we can come up with something. Seriously for a sec: I'd _love_ not to have to explain why most but not all string methods are ALSO in a string module without making it sound as if Python's so encrusted with "needless but backwards-compatible cruft" to make it a match for C++ -- which it isn't, by a long shot, but that ain't easy to communicate. On the other hand, using False and True in explanations instead of 0 and 1 _does_ enhance prose readability -- interesting side effect I guess:-). I'd just love it if old cruft could be removed, but each time that is mooted, there's an uproar. So I guess we just have to live with a slow accumulation of "more than one way to do it" (at least until the mythical will-never-come backwards-incompatible Python 3000, of course) despite the obvious costs (each time two coders get used to different idioms for the same task, they will have a slightly harder time maintaining each other's code). The costs always look small (for each single issue -- there are many small things...), and are spread over time; the benefits always appear more obvious (don't break MY code, let me keep some backwards compatibility with 1.5.2, let me keep my own habits...) and immediate. So, no matter what the actual value of the cruft-removing proposal might be, the uproar against it is inevitably loud, and cruft stays and keeps piling up. Languages always grow, never shrink. I love it when Python manages to do otherwise once in a while (e.g., xrange's cleanup), but I doubt it can happen often enough to make a difference. Oh well. Alex From barry@zope.com Wed May 29 14:59:06 2002 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 29 May 2002 09:59:06 -0400 Subject: [Python-Dev] deprecating string module? References: <200205291417.40030.tl_news@nexgo.de> <20020529122745.74975.qmail@web9605.mail.yahoo.com> Message-ID: <15604.57002.317041.847978@anthem.wooz.org> >>>>> "SL" == Steven Lott writes: SL> The true majority are the "yet to start" users, for whom the SL> String class will be the only thing they ever use; SL> irrespective of the deprecation state of string. Remember that there's a lot more of them ("yet to start" users) than there are of us ("dinosaurs" :). >>>>> "TL" == Tino Lange writes: TL> I cannot see/understand why it's the "best" way to write for TL> example | fromaddr = " ".join(map(lambda x: x[0] , | email.Header.decode_header(fromaddr[1]))) TL> with this (IMHO not intuitive empty string " ") instead of | fromaddr = string.join(map(lambda x: x[0] , | email.Header.decode_header(fromaddr[1]))) TL> It's much cleaner to read and understand with string instead TL> of " " - at least for beginners, or? We've been down this road so many times it hurts. I kind of suspect that it's secretly string.join() keeping the string module alive more than anything else . Personally, I really like ''.join() -- which I spell EMPTYSTRING.join() -- but then I like Rush, uni, and Uncle Timmy's Farm Report. All are acquired tastes. So let's write that join() builtin and be done with it! >>>>> "PF" == Peter Funk writes: PF> In my opponion the string module is one such situation and PF> another one is the '<>' operator. Most of my employees work PF> with Modula-2 a lot and we have a huge code base. So they PF> prefer to use '<>' over '!=' in Python also and they will not PF> stop to do so, although the use of '<>' is discouraged in the PF> Python documentation. The one difference is that <> is favored by some people close to Guido's heart. He'd never piss off his brother or his sysadmin. :) -Barry From guido@python.org Wed May 29 14:58:05 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 09:58:05 -0400 Subject: [Python-Dev] deprecating string module? In-Reply-To: Your message of "Wed, 29 May 2002 08:26:57 +0200." References: Message-ID: <200205291358.g4TDw5v16789@odiug.zope.com> > I'm absolutely against depreceating the string module! Maybe we need a new concept, "silent deprecation"? We would do we could to discourage the use of modules (like types or string) in the documentation, maybe even removing their documentation altogether, but the interpreter would not issue a warning message. If done well, this could have approximately the same effect as a deprecation warning, but without the negative PR effects. Or maybe we could introduce a new warning category, SilentDeprecationWarning, which is normally ignored (like OverflowWarning already is). This could be turned on explicitly with a -W option so someone wanting to check that their code is future-proof would have an easy way to do so. I do not want to ignore all DeprecationWarning messages by default, as it would defeat the main purpose of the warning. --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz@pythoncraft.com Wed May 29 15:08:14 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 29 May 2002 10:08:14 -0400 Subject: [Python-Dev] deprecating string module? In-Reply-To: <200205291358.g4TDw5v16789@odiug.zope.com> References: <200205291358.g4TDw5v16789@odiug.zope.com> Message-ID: <20020529140814.GA19679@panix.com> On Wed, May 29, 2002, Guido van Rossum wrote: > > Or maybe we could introduce a new warning category, > SilentDeprecationWarning, which is normally ignored (like > OverflowWarning already is). This could be turned on explicitly with > a -W option so someone wanting to check that their code is > future-proof would have an easy way to do so. > > I do not want to ignore all DeprecationWarning messages by default, as > it would defeat the main purpose of the warning. +1 -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "In the end, outside of spy agencies, people are far too trusting and willing to help." --Ira Winkler From guido@python.org Wed May 29 15:04:40 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 10:04:40 -0400 Subject: [Python-Dev] deprecating string module? In-Reply-To: Your message of "Wed, 29 May 2002 09:23:28 EDT." <20020529132328.GB10613@panix.com> References: <3CF486F4.6070302@lemburg.com> <15604.47928.443522.504730@12-248-41-177.client.attbi.com> <3CF4C15A.7070606@lemburg.com> <15604.50171.868339.126122@12-248-41-177.client.attbi.com> <20020529132328.GB10613@panix.com> Message-ID: <200205291404.g4TE4e216837@odiug.zope.com> > So don't keep proposing changes that break 1.5.2 compatibility. -1. That means we could never deprecate a feature. --Guido van Rossum (home page: http://www.python.org/~guido/) From aleax@aleax.it Wed May 29 15:27:38 2002 From: aleax@aleax.it (Alex Martelli) Date: Wed, 29 May 2002 16:27:38 +0200 Subject: [Python-Dev] deprecating string module? In-Reply-To: <200205291358.g4TDw5v16789@odiug.zope.com> References: <200205291358.g4TDw5v16789@odiug.zope.com> Message-ID: On Wednesday 29 May 2002 03:58 pm, Guido van Rossum wrote: > > I'm absolutely against depreceating the string module! ... > Or maybe we could introduce a new warning category, > SilentDeprecationWarning, which is normally ignored (like > OverflowWarning already is). This could be turned on explicitly with > a -W option so someone wanting to check that their code is > future-proof would have an easy way to do so. > > I do not want to ignore all DeprecationWarning messages by default, as > it would defeat the main purpose of the warning. This answers ALL the negatives I had just made in my pessimistic last message and gives me hope in the future again...! *YES!!!* Let's have "silent" deprecation warnings, disabled by default, for features that are _eventually_ going to go away "but not just yet", at the same time as their goner status gets in the docs and for at least (whatever number of releases or duration of time) before moving to "real" (noisy) deprecation. Features might be instantly put in "real" deprecation mode like today, when warranted, but a new and even more gradual option would be allowed for features whose eventual removal needs to be handled ever so slowly and gradually. Super! Why, it might even let <> go away some day... I think it should be ensured that (e.g.) a make test with -W gives no warnings -- basically, that the standard library itself uses no features that are deprecated even silently (though it may of course _provide_ some). Otherwise, users would be discouraged from using -W if that could given them warnings "from the library itself" that are not easily removed by working just on their own code. This needs more thought if we also want the silently-deprecated features to keep being tested by the standard test suite at least optionally. Alex From barry@zope.com Wed May 29 15:42:55 2002 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 29 May 2002 10:42:55 -0400 Subject: [Python-Dev] deprecating string module? References: <200205291358.g4TDw5v16789@odiug.zope.com> Message-ID: <15604.59631.910914.930115@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> Or maybe we could introduce a new warning category, GvR> SilentDeprecationWarning, which is normally ignored (like GvR> OverflowWarning already is). This could be turned on GvR> explicitly with a -W option so someone wanting to check that GvR> their code is future-proof would have an easy way to do so. +1 -Barry From neal@metaslash.com Wed May 29 15:45:09 2002 From: neal@metaslash.com (Neal Norwitz) Date: Wed, 29 May 2002 10:45:09 -0400 Subject: [Python-Dev] deprecating string module? References: <200205291358.g4TDw5v16789@odiug.zope.com> Message-ID: <3CF4E975.20346E34@metaslash.com> Guido van Rossum wrote: > Maybe we need a new concept, "silent deprecation"? We would do we > could to discourage the use of modules (like types or string) in the > documentation, maybe even removing their documentation altogether, but > the interpreter would not issue a warning message. I think explicit is better than implicit. The docs should state that the feature is in the process of being deprecated. First quietly, then loudly, then removed. I would also like to see approximate dates for the schedule. Otherwise, I fear people will say, I saw the message for 2 years, I figured it was safe. The message could be something like: XXX is deprecated and will issue a warning after D/D/D. XXX is deprecated and will be removed after D/D/D. The date could be a version too. Doesn't much matter to me. This approach gives more insight into the general direction of the language also, so I think it's a very good idea (+1). Neal From guido@python.org Wed May 29 15:44:13 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 10:44:13 -0400 Subject: [Python-Dev] deprecating string module? In-Reply-To: Your message of "Wed, 29 May 2002 09:59:06 EDT." <15604.57002.317041.847978@anthem.wooz.org> References: <200205291417.40030.tl_news@nexgo.de> <20020529122745.74975.qmail@web9605.mail.yahoo.com> <15604.57002.317041.847978@anthem.wooz.org> Message-ID: <200205291444.g4TEiDX17618@odiug.zope.com> > The one difference is that <> is favored by some people close to > Guido's heart. He'd never piss off his brother or his sysadmin. :) But in Python 3, only != will be supported. Fortunately, it's a simple mechanical translation to fix this. --Guido van Rossum (home page: http://www.python.org/~guido/) From python@rcn.com Wed May 29 16:11:52 2002 From: python@rcn.com (Raymond Hettinger) Date: Wed, 29 May 2002 11:11:52 -0400 Subject: [Python-Dev] deprecating string module? References: <200205291358.g4TDw5v16789@odiug.zope.com> Message-ID: <006b01c20723$2a58aee0$5d61accf@othello> From: "Guido van Rossum" > Maybe we need a new concept, "silent deprecation"? We would do we > could to discourage the use of modules (like types or string) in the > documentation, maybe even removing their documentation altogether, but > the interpreter would not issue a warning message. +100 Last month, I took the bait and went to comp.lang.py with a proposal to deprecate some builtins. I learned that every builtin (except for input()) has a lot of friends. With each set of flames, I mean comments, I modified my proposal until it evolved to "psuedo-deprecation". Amazingly, this attracted no flames at all. The net result is that psuedo-deprecation seems to be acceptable as way of removing cruft and lowering the sum total pool of working knowledge that a Python programmer has to keep in mind. Instead of de-documenting obsolete tools, I propose moving their documentation to a separate section of the docs set aside for cruft. This makes it possible to understand exactly what old code was doing. It makes sure that the newly dedocumented features don't take on never-documented assumptions. Putting it in a separate place allows us to document the rationale for psuedo-deprecation and to suggest alternatives. Collecting it in one place provides authors with a checklist of things to take out of their book updates. On the warning side of the equation, I have two ideas. While there would not be warnings kicking out automatically, we could add an option or tool that developers could use to temporarily flag use of psuedo-deprecated features. The second idea, is to move responsibility for obsolesence warnings to PyChecker. Raymond Hettinger From mwh@python.net Wed May 29 16:13:58 2002 From: mwh@python.net (Michael Hudson) Date: 29 May 2002 16:13:58 +0100 Subject: [Python-Dev] Python 2.3 release schedule In-Reply-To: Neil Schemenauer's message of "Tue, 28 May 2002 08:01:49 -0700" References: <2mr8jwv84e.fsf@starship.python.net> <20020528080149.A7799@glacier.arctrix.com> Message-ID: <2mit57x9zd.fsf@starship.python.net> Neil Schemenauer writes: > Michael Hudson wrote: > > I was under the impression that the 2.1 and 2.2 interfaces differed in > > ways that couldn't easily be papered over with macros. I'll check. > > It's not pretty. Look at pyexpat.c for an example. Perhaps something > like this would be good enough (untested): > > #if PY_VERSION_HEX < 0x020200B1 > #define PyObject_GC_New PyObject_New > #define PyObject_GC_NewVar PyObject_NewVar > #define PyObject_GC_Del PyObject_Del > #define PyObject_GC_Track(op) > #define PyObject_GC_UnTrack(op) > #endif > > People could then always use the 2.2 API but the objects would only be > collected in versions >= 2.2. Using the 2.1 API is a fair bit trickier > and you can't hide those differences using macros (although you could > make it easier for people who want to support 2.1 and >=2.2). Here's the latest effort: /* this idea of this file is that you bundle it with your extension, #include it, program to Python 2.3's memory API and have your extension build with any version of Python from 1.5.2 through to 2.3 (and hopefully beyond) */ #ifndef Py_PYMEMCOMPAT_H #define Py_PYMEMCOMPAT_H #include "Python.h" /* There are three "families" of memory API: the "raw memory", "object memory" and "object" families. (This is ignoring the matter of the cycle collector, about which more is said below). Raw Memory: PyMem_Malloc, PyMem_Realloc, PyMem_Free Object Memory: PyObject_Malloc, PyObject_Realloc, PyObject_Free Object: PyObject_New, PyObject_NewVar, PyObject_Del The raw memory and object memory allocators both mimic the malloc/realloc/free interface from ANSI C, but the object memory allocator can (and, since 2.3, does by default) use a different allocation strategy biased towards lots of lots of "small" allocations. The object family is used for allocating Python objects, and the initializers take care of some basic initialization (setting the refcount to 1 and filling out the ob_type field) as well as having a somewhat different interface. Do not mix the families! E.g. do not allocate memory with PyMem_Malloc and free it with PyObject_Free. You may get away with it quite a lot of the time, but there *are* scenarios where this will break. You Have Been Warned. Also, in many versions of Python there are an insane amount of memory interfaces to choose from. Use the ones described above. */ #if PY_VERSION_HEX < 0x01060000 /* raw memory interface already present */ /* there is no object memory interface in 1.5.2 */ #define PyObject_Malloc PyMem_Malloc #define PyObject_Realloc PyMem_Realloc #define PyObject_Free PyMem_Free /* the object interface is there, but the names have changed */ #define PyObject_New PyObject_NEW #define PyObject_NewVar PyObject_NEW_VAR #define PyObject_Del PyMem_Free #endif /* If your object is a container you probably want to support the cycle collector, which was new in Python 2.0. Unfortunately, the interface to the collector that was present in Python 2.0 and 2.1 proved to be tricky to use, and so changed in 2.2 -- in a way that can't easily be papered over with macros. This file contains macros that let you program to the 2.2 GC API. Your module will compile against any Python since version 1.5.2, but the type will only participate in the GC in versions 2.2 and up. Some work is still necessary on your part to only fill out the tp_traverse and tp_clear fields when they exist and set tp_flags appropriately. It is possible to support both the 2.0 and 2.2 GC APIs, but it's not pretty and this comment block is too narrow to contain a desciption of what's required... */ #if PY_VERSION_HEX < 0x020200B1 #define PyObject_GC_New PyObject_New #define PyObject_GC_NewVar PyObject_NewVar #define PyObject_GC_Del PyObject_Del #define PyObject_GC_Track(op) #define PyObject_GC_UnTrack(op) #endif #endif /* !Py_PYMEMCOMPAT_H */ Any final comments before I check it in? Particularly, I'd like Neil to read the comments about the cycle collector. Cheers, M. -- I'll write on my monitor fifty times 'I must not post self-indulgent wibble nobody is interested in to ucam.chat just because I'm bored and I can't find the bug I'm supposed to fix'. -- Steve Kitson, ucam.chat From python@rcn.com Wed May 29 16:16:55 2002 From: python@rcn.com (Raymond Hettinger) Date: Wed, 29 May 2002 11:16:55 -0400 Subject: [Python-Dev] deprecating string module? References: <200205291417.40030.tl_news@nexgo.de> <20020529122745.74975.qmail@web9605.mail.yahoo.com> <15604.57002.317041.847978@anthem.wooz.org> <200205291444.g4TEiDX17618@odiug.zope.com> Message-ID: <007b01c20723$debbe1e0$5d61accf@othello> Premise 1: > > The one difference is that <> is favored by some people close to > > Guido's heart. He'd never piss off his brother or his sysadmin. :) Premise 2: > But in Python 3, only != will be supported. Conclusion: Python 3 will never happen. Humph! From mgilfix@eecs.tufts.edu Wed May 29 16:11:13 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Wed, 29 May 2002 11:11:13 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200205282049.g4SKnj009428@odiug.zope.com>; from guido@python.org on Tue, May 28, 2002 at 04:49:45PM -0400 References: <8BB00CF9-7275-11D6-8740-003065517236@oratrix.com> <200205282049.g4SKnj009428@odiug.zope.com> Message-ID: <20020529111113.B15532@eecs.tufts.edu> We seem to see this issue a lot here and perhaps the problem stems from the fact that there's no really experimental branch (or at least I don't know about it :) ) to python where the cutting edge people can mess with the cutting edge and won't feel too bad if they get burnt. Perhaps Python should adopt a versioning number such that major changes occur during odd numbers, etc. Just so users have an idea what's coming. Add a minor version number and release more minor versions. That way we can always expect major changes between them and at least everybody was *always* warned, as opposed to downloading the latest version and praying that something you liked wasn't depreciated, a warning was received or that something might break. Maybe that might help identify how to document the major changes, or rather when users should look and see what has changed/what has broken, etc. I think the backwards compatibility is pretty damn good though. I had no problems writing an app and backporting it to 1.5.x (I basically just had to use the string module). I did use functional stuff over list comprehension though... My next app, however, will require python 2.x and that's an audience decision. But not every one has that luxury. Almost an after thought, the last thing we want is to take the Java route though and have 'classes' disappear and new ones replace them with new interfaces. python's uniformity was one of the biggest sellers to me. Ugh. -- Mike On Tue, May 28 @ 16:49, Guido van Rossum wrote: > > As an aside, note that this backward compatibility is actually a > > mixed blessing, because it means you don't have to update your > > modules now, but there will come a time when it is going to bite > > you. > > When new releases take features away, we will issue warnings as a > gentle push. When they add features, I don't know why you *should* > use the new features, unless you need them -- and then you have your > motivation in your needs. > > > As a personal example: the MacPython toolbox modules haven't > > been updated to make use of the GC stuff yet (and that's been > > there since 2.0, no?), > > But the API was totally changed for 2.2, so you're actually lucky that > you didn't do it for 2.0. ;-) > > > let alone the new type system. And these > > are almost all generated, so it would probably only take a few > > dozen lines of code to fix them. And the new type system would > > be a real boon for some of the modules (such as the windowing > > and dialog stuff), but because there's no real push (i.e. > > everything still works) nothing has happened yet... > > I don't think *anything* can be done to force you to start using new > optional features... Eventually classic classes will go away, but > that will be a long time. -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From aahz@pythoncraft.com Wed May 29 16:24:58 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 29 May 2002 11:24:58 -0400 Subject: [Python-Dev] deprecating string module? In-Reply-To: <200205291404.g4TE4e216837@odiug.zope.com> References: <3CF486F4.6070302@lemburg.com> <15604.47928.443522.504730@12-248-41-177.client.attbi.com> <3CF4C15A.7070606@lemburg.com> <15604.50171.868339.126122@12-248-41-177.client.attbi.com> <20020529132328.GB10613@panix.com> <200205291404.g4TE4e216837@odiug.zope.com> Message-ID: <20020529152458.GA27129@panix.com> On Wed, May 29, 2002, Guido van Rossum wrote: >Aahz: >> >> So don't keep proposing changes that break 1.5.2 compatibility. > > -1. That means we could never deprecate a feature. In case it wasn't clear, that was sarcasm in response to Skip's "tired of whining" comment. I like to think I occupy the middle ground on the backward compatibility issue, and at various times I like to take potshots at both extremes. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "In the end, outside of spy agencies, people are far too trusting and willing to help." --Ira Winkler From Oleg Broytmann Wed May 29 16:25:42 2002 From: Oleg Broytmann (Oleg Broytmann) Date: Wed, 29 May 2002 19:25:42 +0400 Subject: [Python-Dev] deprecating string module? In-Reply-To: <007b01c20723$debbe1e0$5d61accf@othello>; from python@rcn.com on Wed, May 29, 2002 at 11:16:55AM -0400 References: <200205291417.40030.tl_news@nexgo.de> <20020529122745.74975.qmail@web9605.mail.yahoo.com> <15604.57002.317041.847978@anthem.wooz.org> <200205291444.g4TEiDX17618@odiug.zope.com> <007b01c20723$debbe1e0$5d61accf@othello> Message-ID: <20020529192542.A15045@phd.pp.ru> On Wed, May 29, 2002 at 11:16:55AM -0400, Raymond Hettinger wrote: > Premise 1: > > > > The one difference is that <> is favored by some people close to > > > Guido's heart. He'd never piss off his brother or his sysadmin. :) > > Premise 2: > > But in Python 3, only != will be supported. > > Conclusion: > Python 3 will never happen. Agree :) I don't understand what is wrong with <> syntax. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From skip@pobox.com Wed May 29 16:20:58 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 29 May 2002 10:20:58 -0500 Subject: [Python-Dev] deprecating string module? In-Reply-To: <062d01c2070e$33945190$0900a8c0@spiff> References: <3CF486F4.6070302@lemburg.com> <15604.47928.443522.504730@12-248-41-177.client.attbi.com> <062d01c2070e$33945190$0900a8c0@spiff> Message-ID: <15604.61914.651010.829734@12-248-41-177.client.attbi.com> >> It has nothing to do with 11k worth of disk space and everything to >> do with "there's one (best) way to do it". Fredrik> in this case, it sounds more like "purity over practicality" Fredrik> (or maybe "I'm bored, let's get some discussion going") Fredrik> and frankly, it doesn't sound much like you either. didn't you Fredrik> use to be a rather practical programmer? I am a rather practical programmer, however my programming goals are generally different than yours. I don't write software that is widely distributed. I understand there is bound to be pushback on such a topic. I was also not motivated by boredom. I was motivated by reading this recent addition to the Style Guide: - Avoid the use of the string module; instead use string methods. These are always much faster and share the same API with unicode strings. Now, granted, Guido and Barry weren't saying, "the string module is deprecated". In my original message I even suggested this change in wording to the above: Avoid the use of the string module unless backward-compatibility with versions earlier than Python 2.0 is important; instead use string methods. These are always much faster and share the same API with unicode strings. Still, it sounds to me like for most purposes there is ample reason to advocate not using the string module. Then, my natural inclination is to wonder if it should simply be deprecated. Three functions in the string module are already officially deprecated (atof, atoi, atol since 2.0 - shouldn't their continued use be generating warnings by now?). At least one other should be (capwords and possibly zfill). I would argue that if Guido feels strongly enough to make an "avoid the string module" statement in the Style Guide, a similarly worded statement should also be made at the top of the string module docs. Skip From skip@pobox.com Wed May 29 16:26:19 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 29 May 2002 10:26:19 -0500 Subject: [Python-Dev] deprecating string module? In-Reply-To: <20020529133034.GC10613@panix.com> References: <15603.63458.608739.454153@beluga.mojam.com> <15604.46539.444906.980616@12-248-41-177.client.attbi.com> <20020529133034.GC10613@panix.com> Message-ID: <15604.62235.222115.367664@12-248-41-177.client.attbi.com> aahz> This is ridiculous. Perhaps you missed the smiley? Skip From guido@python.org Wed May 29 16:23:44 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 11:23:44 -0400 Subject: [Python-Dev] Silent deprecation In-Reply-To: Your message of "Wed, 29 May 2002 11:11:52 EDT." <006b01c20723$2a58aee0$5d61accf@othello> References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> Message-ID: <200205291523.g4TFNiA18104@odiug.zope.com> [Changed subject] [Raymond Hettinger] > Last month, I took the bait and went to comp.lang.py with a proposal > to deprecate some builtins. I learned that every builtin (except for > input()) has a lot of friends. And input() has one important friend. :-) > With each set of flames, I mean comments, I modified my proposal > until it evolved to "psuedo-deprecation". Amazingly, this attracted > no flames at all. The net result is that psuedo-deprecation seems > to be acceptable as way of removing cruft and lowering the sum total > pool of working knowledge that a Python programmer has to keep in > mind. I guess this means that people don't mind us telling them which features are obsolete, but they hate it with a vengeance when their programs that they have deployed in the field with end users start spewing warning messages. > Instead of de-documenting obsolete tools, I propose moving their > documentation to a separate section of the docs set aside for cruft. > This makes it possible to understand exactly what old code was > doing. Hm, getting the docs for a previous version is easy enough. We never throw anything away. > It makes sure that the newly dedocumented features don't take on > never-documented assumptions. Putting it in a separate place allows > us to document the rationale for psuedo-deprecation and to suggest > alternatives. Collecting it in one place provides authors with a > checklist of things to take out of their book updates. I think we can do two things. Either we can keep the docs for deprecated features but clearly mark the chapter or section with a deprecation note at the start, in bold. Or we can remove the docs for deprecated features, and add a chapter "deprecated features" which mentions each deprecated feature without describing it in detail, and provides information about what to use instead. > On the warning side of the equation, I have two ideas. While there > would not be warnings kicking out automatically, we could add an > option or tool that developers could use to temporarily flag use of > psuedo-deprecated features. This is what I meant by having a SilentDeprecationWarning category -- a special form of the -W option can cause it to spew messages, but by default it won't. > The second idea, is to move responsibility for obsolesence warnings > to PyChecker. This is not always feasible; some things can only be discovered at run-time (e.g. the oct/hex/<< warnings). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed May 29 16:28:02 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 11:28:02 -0400 Subject: [Python-Dev] deprecating string module? In-Reply-To: Your message of "Wed, 29 May 2002 19:25:42 +0400." <20020529192542.A15045@phd.pp.ru> References: <200205291417.40030.tl_news@nexgo.de> <20020529122745.74975.qmail@web9605.mail.yahoo.com> <15604.57002.317041.847978@anthem.wooz.org> <200205291444.g4TEiDX17618@odiug.zope.com> <007b01c20723$debbe1e0$5d61accf@othello> <20020529192542.A15045@phd.pp.ru> Message-ID: <200205291528.g4TFS2n18206@odiug.zope.com> > Agree :) I don't understand what is wrong with <> syntax. It's cumbersome to have to support two different spellings for no good reason. (It's different for "..." and '...' strings, where there's a good reason to have both, namely embedding the other type of quote.) Even people who are used to <> can learn to type !=. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com Wed May 29 16:34:18 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 29 May 2002 10:34:18 -0500 Subject: [Python-Dev] deprecating string module? In-Reply-To: <15604.59631.910914.930115@anthem.wooz.org> References: <200205291358.g4TDw5v16789@odiug.zope.com> <15604.59631.910914.930115@anthem.wooz.org> Message-ID: <15604.62714.420967.273341@12-248-41-177.client.attbi.com> GvR> Or maybe we could introduce a new warning category, GvR> SilentDeprecationWarning, which is normally ignored (like GvR> OverflowWarning already is). This could be turned on GvR> explicitly with a -W option so someone wanting to check that GvR> their code is future-proof would have an easy way to do so. BAW> +1 Thank you. Though it is painful to bring up some of these topics (I think I've caught more than my fair share of arrows recently), I find there's often a new way to look at things that makes the problem more tractable. I suggest a migration path from SilentDeprecationWarning to DeprecationWarning. Skip From Oleg Broytmann Wed May 29 16:35:06 2002 From: Oleg Broytmann (Oleg Broytmann) Date: Wed, 29 May 2002 19:35:06 +0400 Subject: [Python-Dev] deprecating string module? In-Reply-To: <200205291528.g4TFS2n18206@odiug.zope.com>; from guido@python.org on Wed, May 29, 2002 at 11:28:02AM -0400 References: <200205291417.40030.tl_news@nexgo.de> <20020529122745.74975.qmail@web9605.mail.yahoo.com> <15604.57002.317041.847978@anthem.wooz.org> <200205291444.g4TEiDX17618@odiug.zope.com> <007b01c20723$debbe1e0$5d61accf@othello> <20020529192542.A15045@phd.pp.ru> <200205291528.g4TFS2n18206@odiug.zope.com> Message-ID: <20020529193506.B15045@phd.pp.ru> On Wed, May 29, 2002 at 11:28:02AM -0400, Guido van Rossum wrote: > Even people who are used to <> can learn to type !=. Then why not the other way? :) Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From barry@zope.com Wed May 29 16:34:41 2002 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 29 May 2002 11:34:41 -0400 Subject: [Python-Dev] deprecating string module? References: <200205291417.40030.tl_news@nexgo.de> <20020529122745.74975.qmail@web9605.mail.yahoo.com> <15604.57002.317041.847978@anthem.wooz.org> <200205291444.g4TEiDX17618@odiug.zope.com> <007b01c20723$debbe1e0$5d61accf@othello> Message-ID: <15604.62737.381796.485644@anthem.wooz.org> >>>>> "RH" == Raymond Hettinger writes: RH> Python 3 will never happen. Oh, I'm sure it will happen one day, but it'll be cool 'cause I'll be in retirement touring the country in my Mr. Neutron powered DeLorean, and my hack-pr0n-drinkmountaindew clone will get to do the upgrade! -Barry From guido@python.org Wed May 29 16:32:51 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 11:32:51 -0400 Subject: [Python-Dev] Bug in PyNumber_InPlacePower implementation? In-Reply-To: Your message of "Wed, 29 May 2002 14:38:44 +1200." <200205290238.OAA06564@s454.cosc.canterbury.ac.nz> References: <200205290238.OAA06564@s454.cosc.canterbury.ac.nz> Message-ID: <200205291532.g4TFWpX18293@odiug.zope.com> > > This is exactly the same as what binary_op1() does. I added this > > twist intentionally because of a use case where a subclass of a > > numeric type wants to override a binary (or ternary) operator defined > > by the base class. > > I don't think you understand what I mean. I'm talking > about *in-place* operations. You're right, I hadn't realized that this was about inplace. > For in-place binary operations we have, e.g. > > INPLACE_BINOP(PyNumber_InPlaceSubtract, nb_inplace_subtract, nb_subtract, "-=") > ^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^ > > Both the in-place and non-in-place slots are passed to > binary_iop, which uses the in-place slot for the first > operand and the non-in-place slot for the second. > > But ternary_op only gets *one* slot passed to it, which it > uses for everything. When doing an in-place ternary op, > this is the in-place slot. What this seems to mean is > that, if we have isinstance(a, A) and isinstance(b, B) > and issubclass(B, A), then > > a **= b > > has the potential to in-place-modify b instead of a! Not quite -- the arguments won't be reversed, but it may call B's inplace power function with an A instance as the first argument. This can return NotImplemented if it doesn't know what to do in that case. > It seems to me that there ought to be a ternary_iop > routine that does for ternary ops what binary_iop does > for binary ones. Or we could just document the status quo. It's a pretty esoteric case -- numeric types are traditionally immutable so don't have to implement inplace power at all, and non-numeric types aren't very likely to implement ** at all, let alone **=. The correct code would be pretty hairy I think (the non-inplace ternary is hairy enough). > > Do you have a use case where this does the wrong thing, or is this > > just a theoretical musing? > > It's a theoretical musing that came out of my attempts > to figure out whether, when one is implementing an > nb_inplace_power slot for a new type, the first argument > is guaranteed to "self". I need to know this so that Pyrex > can do the right thing. > > I've done some experiments with Python code, and it seems > to do the right thing in terms of calling the correct > __pow__, __rpow__ and __ipow__ methods. But I can't tell > from that what's really going on at the typeslot level. > I've looked at the code for the nb_inplace_power of > instance objects (the only example I can find of such > a method!), and it seems to assume that the first > argument is always "self". But I can't see how this is > guaranteed. > > In short, I'm confused! Are you still confused after my assertion above? --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Wed May 29 16:37:44 2002 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 29 May 2002 11:37:44 -0400 Subject: [Python-Dev] deprecating string module? References: <200205291417.40030.tl_news@nexgo.de> <20020529122745.74975.qmail@web9605.mail.yahoo.com> <15604.57002.317041.847978@anthem.wooz.org> <200205291444.g4TEiDX17618@odiug.zope.com> <007b01c20723$debbe1e0$5d61accf@othello> <20020529192542.A15045@phd.pp.ru> Message-ID: <15604.62920.539452.237215@anthem.wooz.org> >>>>> "OB" == Oleg Broytmann writes: OB> Agree :) I don't understand what is wrong with <> syntax. It's an anti-Timbot measure. He'll be forced to retire when he has to write !wink= :) -Barry From neal@metaslash.com Wed May 29 16:42:00 2002 From: neal@metaslash.com (Neal Norwitz) Date: Wed, 29 May 2002 11:42:00 -0400 Subject: [Python-Dev] Silent deprecation References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> Message-ID: <3CF4F6C8.E57807FF@metaslash.com> Note: there's an patch available at http://python.org/sf/561928 Guido van Rossum wrote: > I think we can do two things. Either we can keep the docs for > deprecated features but clearly mark the chapter or section with a > deprecation note at the start, in bold. > > Or we can remove the docs for deprecated features, and add a chapter > "deprecated features" which mentions each deprecated feature without > describing it in detail, and provides information about what to use > instead. Either of these approaches works for me. > > The second idea, is to move responsibility for obsolesence warnings > > to PyChecker. > > This is not always feasible; some things can only be discovered at > run-time (e.g. the oct/hex/<< warnings). Agreed. PyChecker will probably just be an early warning system. Especially since I'm more aggressive on my definition of 'deprecated.' :-) Neal From barry@zope.com Wed May 29 16:42:33 2002 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 29 May 2002 11:42:33 -0400 Subject: [Python-Dev] deprecating string module? References: <3CF486F4.6070302@lemburg.com> <15604.47928.443522.504730@12-248-41-177.client.attbi.com> <062d01c2070e$33945190$0900a8c0@spiff> <15604.61914.651010.829734@12-248-41-177.client.attbi.com> Message-ID: <15604.63209.151939.448785@anthem.wooz.org> >>>>> "SM" == Skip Montanaro writes: SM> Now, granted, Guido and Barry weren't saying, "the string SM> module is deprecated". In my original message I even SM> suggested this change in wording to the above: How's this: - Use string methods instead of the string module unless backward-compatibility with versions earlier than Python 2.0 is important. String methods are always much faster and share the same API with unicode strings. -Barry From mgilfix@eecs.tufts.edu Wed May 29 16:45:49 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Wed, 29 May 2002 11:45:49 -0400 Subject: [Python-Dev] Silent deprecation In-Reply-To: <3CF4F6C8.E57807FF@metaslash.com>; from neal@metaslash.com on Wed, May 29, 2002 at 11:42:00AM -0400 References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> Message-ID: <20020529114548.C15532@eecs.tufts.edu> > > This is not always feasible; some things can only be discovered at > > run-time (e.g. the oct/hex/<< warnings). > > Agreed. PyChecker will probably just be an early warning system. > Especially since I'm more aggressive on my definition of 'deprecated.' :-) Sometimes I wish pychecker was integrated with the distro... But I'm sure there are good reasons for avoiding this. -- Mike -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From guido@python.org Wed May 29 16:44:37 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 11:44:37 -0400 Subject: [Python-Dev] Silent deprecation In-Reply-To: Your message of "Wed, 29 May 2002 11:42:00 EDT." <3CF4F6C8.E57807FF@metaslash.com> References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> Message-ID: <200205291544.g4TFibS18433@odiug.zope.com> > Note: there's an patch available at http://python.org/sf/561928 Thanks! I agree with the name change to PendingDeprecationWarning. Check it in before I change my mind! :-) > Guido van Rossum wrote: > > > I think we can do two things. Either we can keep the docs for > > deprecated features but clearly mark the chapter or section with a > > deprecation note at the start, in bold. > > > > Or we can remove the docs for deprecated features, and add a chapter > > "deprecated features" which mentions each deprecated feature without > > describing it in detail, and provides information about what to use > > instead. > > Either of these approaches works for me. I personally like the second version better -- by not providing full documentation, it discourages new users more strongly from using deprecated features. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed May 29 16:46:07 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 11:46:07 -0400 Subject: [Python-Dev] Silent deprecation In-Reply-To: Your message of "Wed, 29 May 2002 11:45:49 EDT." <20020529114548.C15532@eecs.tufts.edu> References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <20020529114548.C15532@eecs.tufts.edu> Message-ID: <200205291546.g4TFk7f18459@odiug.zope.com> > Sometimes I wish pychecker was integrated with the distro... But I'm > sure there are good reasons for avoiding this. Mostly that the PyChecker development cycle is several orders of magnitude faster than Python's, so at best you would have an out-of-date version of PyChecker in the distro. Once PyChecker slows down a bit, I'd be happy to incorporate it. --Guido van Rossum (home page: http://www.python.org/~guido/) From pf@artcom-gmbh.de Wed May 29 16:52:34 2002 From: pf@artcom-gmbh.de (Peter Funk) Date: Wed, 29 May 2002 17:52:34 +0200 (CEST) Subject: Depreciation (was: Re: [Python-Dev] deprecating string module?) In-Reply-To: <200205291404.g4TE4e216837@odiug.zope.com> from Guido van Rossum at "May 29, 2002 10:04:40 am" Message-ID: Aahz to Skip: > > So don't keep proposing changes that break 1.5.2 compatibility. Guido van Rossum: > -1. That means we could never deprecate a feature. Than don't do it! Simply never remove features from the language (whether the feature is documented or not). Do it like Niklaus Wirth: Always start with a fresh new name for the language, whenever a major cleanup of unwanted features seems to be absolutely unavoidable to you: Wirth invented Pascal, later Modula-2 and finally Oberon. However ... Orlijthon ... hmmm ... this seems to be somewhat hard to type and spell. ;-) Regards, Peter -- Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260 office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen, Germany) From David Abrahams" On AIX, building a shared library for use by extension modules and which uses the Python 'C' API is way harder than it should be**; the best workaround we could find involves hijacking _PyImport_LoadDynamicModule to load the shared library and patch up the symbol references. _PyImportLoadDynamicModule is undocumented, AFAICT. Am I not supposed to touch it? Is it likely to disappear or change its signature? On a related note, I wonder about all of the _[A-Z] names in Python. I know that at least in C++, these names are reserved to the compiler implementation. I assume the same holds for 'C', and wonder if Python's use of these names is intentional, on the "to be changed someday" list, or something else. Thanks in advance, Dave ** details to follow in later messages; AIX is nasty but I think Python could do a few things to help. +---------------------------------------------------------------+ David Abrahams C++ Booster (http://www.boost.org) O__ == Pythonista (http://www.python.org) c/ /'_ == resume: http://users.rcn.com/abrahams/resume.html (*) \(*) == email: david.abrahams@rcn.com +---------------------------------------------------------------+ From neal@metaslash.com Wed May 29 17:11:59 2002 From: neal@metaslash.com (Neal Norwitz) Date: Wed, 29 May 2002 12:11:59 -0400 Subject: [Python-Dev] PyChecker in Python References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <20020529114548.C15532@eecs.tufts.edu> <200205291546.g4TFk7f18459@odiug.zope.com> Message-ID: <3CF4FDCF.F4644EBB@metaslash.com> Title changed. Guido van Rossum wrote: > > > Sometimes I wish pychecker was integrated with the distro... But I'm > > sure there are good reasons for avoiding this. > > Mostly that the PyChecker development cycle is several orders of > magnitude faster than Python's, so at best you would have an > out-of-date version of PyChecker in the distro. Once PyChecker slows > down a bit, I'd be happy to incorporate it. Pychecker development has slowed down a bit, but there are some other reasons to not include it just yet. There's several issues that are slowly converging. The new compiler which Jeremy did a bunch of work on is an important part. Ideally, pychecker would be implemented on top of the new compiler. This will make integration of pychecker easier and should also allow it to work with jython. There's also been work on using ASTs for pychecker, rather than byte codes. These are 2 important parts which are on-going. I've thought about returning to the compiler, but I'm also keen to clean up a lot of cruft. I think that's more important. Generally, I'd like to finish cleaning up the deprecation issues, which include: using PendingDeprecations, Py_DEPRECATED in C headers, and fixing the docs. I'm still thinking about optimizations too (remove SET_LINENO and others). And finally, the compiler and pychecker. (Also, generally improving test coverage is hrown in somewhere.) I'm hopeful that 2.4 may be a good time to get the compiler and pychecker into Python. But I suspect that pychecker will still be maintained separately for a while. Neal From guido@python.org Wed May 29 17:29:52 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 12:29:52 -0400 Subject: [Python-Dev] Out of date FAQ entries In-Reply-To: Your message of "Tue, 28 May 2002 16:46:42 EDT." <200205282046.g4SKkg709405@odiug.zope.com> References: <3CEC52E2.962B1CB6@metaslash.com> <200205242008.g4OK8hi11791@pcp742651pcs.reston01.va.comcast.net> <3CEEA7A6.BFB21C87@metaslash.com> <200205281941.g4SJfnP09122@odiug.zope.com> <028e01c20681$70f13900$e000a8c0@thomasnotebook> <200205282046.g4SKkg709405@odiug.zope.com> Message-ID: <200205291629.g4TGTqP18671@odiug.zope.com> > > > I merged these into 4.28 and deleted 4.93. (Remember, the convention > > > for deleting FAQ entries is to set the title to "Deleted" and remove > > > all contents, except perhaps a pointer.) > > The result doesn't look too good, IMO. > > - Although Gordon's starship pages are hopelessly outdated, they are > > mentioned as the first and fourth possibility... > > - Does sqfreeze actually work (I never tried it)? > > - Should /F's squeeze be mentioned? I fixed the first and third items. I have desire to test any of these and so I'll refrain from commenting on how well they work. --Guido van Rossum (home page: http://www.python.org/~guido/) From python@rcn.com Wed May 29 17:40:02 2002 From: python@rcn.com (Raymond Hettinger) Date: Wed, 29 May 2002 12:40:02 -0400 Subject: [Python-Dev] PendingDeprecationWarning References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <200205291544.g4TFibS18433@odiug.zope.com> Message-ID: <00e001c2072f$7b8d2460$5d61accf@othello> From: "Guido van Rossum" > Thanks! I agree with the name change to PendingDeprecationWarning. > Check it in before I change my mind! :-) Awesome! Now, down to business. What shall we Silently Deprecate? string module types module (after providing substitutes) apply() oct() hex() Raymond Hettinger From guido@python.org Wed May 29 17:42:36 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 12:42:36 -0400 Subject: [Python-Dev] PendingDeprecationWarning In-Reply-To: Your message of "Wed, 29 May 2002 12:40:02 EDT." <00e001c2072f$7b8d2460$5d61accf@othello> References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <200205291544.g4TFibS18433@odiug.zope.com> <00e001c2072f$7b8d2460$5d61accf@othello> Message-ID: <200205291642.g4TGgaf18754@odiug.zope.com> > What shall we Silently Deprecate? > > string module Yes, after providing substitutes. > types module (after providing substitutes) Yes. > apply() Yes. > oct() > hex() Why? I use these a lot... --Guido van Rossum (home page: http://www.python.org/~guido/) From pf@artcom-gmbh.de Wed May 29 18:03:05 2002 From: pf@artcom-gmbh.de (Peter Funk) Date: Wed, 29 May 2002 19:03:05 +0200 (CEST) Subject: <> vs. != (was Re: [Python-Dev] deprecating string module) In-Reply-To: <200205291528.g4TFS2n18206@odiug.zope.com> from Guido van Rossum at "May 29, 2002 11:28:02 am" Message-ID: Hi, Guido van Rossum: > > Agree :) I don't understand what is wrong with <> syntax. > > It's cumbersome to have to support two different spellings for no good > reason. (It's different for "..." and '...' strings, where there's a > good reason to have both, namely embedding the other type of quote.) People want keep both. I believe this is a very good reason to keep both. Since it was there for over a decade, I can't see anything "cumbersome". > Even people who are used to <> can learn to type !=. Of course. But people don't want to change their habits. On Oct 20th, 1991 you decided to put '!=' in as an alternative to '<>'. I guess you did this to please people coming from C to feel more familar and to attract them to Python. That was fine. Using <> mixed != together didn't hurt readability much during the past decade. After all Python is still a very clean language and what lately was called "piled cruft" here in python-dev is largely overestimated. The complexity introduced by the deprecation warning stuff just to deal with future feature removal stuff can also be seen as "cruft" or "bloat" on its own: nnorwitz just fixed a >>> dir(__builtin__) screen shot in the Python tutorial to add 'PendingDeprecationWarning' there. I always tend to mispell this as depreciation... Is it really good PR for Python to point new users noses in the Tutorial to anything they learn may pend to deprecation? These deprecation warnings seem to fullfill one main purpose: to have some lame excuse, that when some dump user installs a new Python version somewhere and breaks a system deployed 4 years ago, that wasn't updated since than. In the industry people tend to run systems a lot longer than two, three or four of your releases cycles. Regards, Peter -- Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260 office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen, Germany) From s_lott@yahoo.com Wed May 29 18:20:01 2002 From: s_lott@yahoo.com (Steven Lott) Date: Wed, 29 May 2002 10:20:01 -0700 (PDT) Subject: [Python-Dev] Re: Deprecation In-Reply-To: Message-ID: <20020529172001.39760.qmail@web9601.mail.yahoo.com> The numbering scheme should cover this need. After all, Wirth had a series of numbered versions for Modula (, 2, 3). I think the individual modules may need explicit version numbering, also. I find 15 of 158 .py files in my release of the library have __version__ numbers. I have a patch I use for the current release of the SGML module; it may never make it to prime time (since it works around an obscure MS-Word-ism). I may have to update it for future releases, and would like to know which releases of the module use wich version of the patch. --- Peter Funk wrote: > Aahz to Skip: > > > So don't keep proposing changes that break 1.5.2 > compatibility. > > Guido van Rossum: > > -1. That means we could never deprecate a feature. > > Than don't do it! Simply never remove features from > the language (whether the feature is documented or not). > > Do it like Niklaus Wirth: Always start with a fresh new name > for the > language, ... ===== -- S. Lott, CCP :-{) S_LOTT@YAHOO.COM http://www.mindspring.com/~slott1 Buccaneer #468: KaDiMa Macintosh user: drinking upstream from the herd. __________________________________________________ Do You Yahoo!? Yahoo! - Official partner of 2002 FIFA World Cup http://fifaworldcup.yahoo.com From aahz@pythoncraft.com Wed May 29 18:27:56 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 29 May 2002 13:27:56 -0400 Subject: [Python-Dev] Death of 1.5.2 Message-ID: <20020529172756.GA8611@panix.com> On Wed, May 29, 2002, Skip Montanaro wrote: > > aahz> This is ridiculous. > > Perhaps you missed the smiley? No, I didn't, but after your comment about "whining", it appeared to me that regardless of the humor your underlying intent was quite serious. My point about ANSI C vs. K&R C goes to the heart of the discussion, I think: although Python development has overall been excellent at maintaining backward compatibility, the discussion about timescales for deprecation are overly aggressive, particularly in comparison with how long it takes to switch C versions. Here's a thought that just occurred to me: Python 2.0 was somewhat buggy and not GPL compatible. So I think it's fair in some respects to start the clock on the death of 1.5.2 sometime later, probably around the release of 2.1 or 2.0.1 -- that's only a year ago. I think there may be some serious underestimation about just how entrenched 1.5.2 has become. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "In the end, outside of spy agencies, people are far too trusting and willing to help." --Ira Winkler From aleax@aleax.it Wed May 29 18:29:46 2002 From: aleax@aleax.it (Alex Martelli) Date: Wed, 29 May 2002 19:29:46 +0200 Subject: [Python-Dev] PendingDeprecationWarning In-Reply-To: <200205291642.g4TGgaf18754@odiug.zope.com> References: <00e001c2072f$7b8d2460$5d61accf@othello> <200205291642.g4TGgaf18754@odiug.zope.com> Message-ID: On Wednesday 29 May 2002 06:42 pm, Guido van Rossum wrote: ... > > oct() > > hex() > > Why? I use these a lot... I assume the duplication of oct and hex wrt '%o'% and '%x'% was the reason to suggest silently-deprecating the former (trying to have 'just one obvious way' and all that). Personally, what seems weird to me is that we have a way to _parse_ numbers in any base (thanks to the 2-args form of str) but not one to _emit_ them in any base -- an issue of asymmetry more than anything else (binary is probably the only format besides oct and hex that would see significant use -- as I would guess may be the case for 2-arguments str today). The % operator is probably a bit too obscure / unreadable to consider it a general substitute for oct and hex, alas. Just MHO of course. Alex From python@rcn.com Wed May 29 18:28:27 2002 From: python@rcn.com (Raymond Hettinger) Date: Wed, 29 May 2002 13:28:27 -0400 Subject: [Python-Dev] String module References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <200205291544.g4TFibS18433@odiug.zope.com> <00e001c2072f$7b8d2460$5d61accf@othello> <200205291642.g4TGgaf18754@odiug.zope.com> Message-ID: <013501c20736$3f7732c0$5d61accf@othello> From: "Guido van Rossum" > > What shall we Silently Deprecate? > > > > string module > > Yes, after providing substitutes. I commented on the substitutes patch, www.python.org/sf/561832, but thought it would be better to kick the idea around here on Py-dev. Instead of making direct substitutes for the character lists, I propose we take advantage of the opportunity and provide them as mappings rather than strings. That way, we can get O(1) behavior instead of O(n) behavior for code like: if c in str.printable: c='*'. If someone needs to know the contents, they can run str.printable.keys(). Also, because the dictionary is mutable, someone can (at runtime) expand or contract the definitions: str.whitespace.append('_'). Raymond Hettinger From pinard@iro.umontreal.ca Wed May 29 18:39:41 2002 From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard) Date: 29 May 2002 13:39:41 -0400 Subject: [Python-Dev] Re: PendingDeprecationWarning In-Reply-To: References: <00e001c2072f$7b8d2460$5d61accf@othello> <200205291642.g4TGgaf18754@odiug.zope.com> Message-ID: [Alex Martelli] > I assume the duplication of oct and hex wrt '%o'% and '%x'% was the > reason to suggest silently-deprecating the former (trying to have 'just > one obvious way' and all that). The same kind of reason says that `str()' and `repr()' could be faded out in favour of `%s' and `%r' specifiers. This is not likely to happen! :-) -- François Pinard http://www.iro.umontreal.ca/~pinard From skip@pobox.com Wed May 29 18:40:13 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 29 May 2002 12:40:13 -0500 Subject: [Python-Dev] PendingDeprecationWarning In-Reply-To: <00e001c2072f$7b8d2460$5d61accf@othello> References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <200205291544.g4TFibS18433@odiug.zope.com> <00e001c2072f$7b8d2460$5d61accf@othello> Message-ID: <15605.4733.165698.269033@12-248-41-177.client.attbi.com> Raymond> What shall we Silently Deprecate? I see a slogan for the next Python Conference: Nobody expects the Silent Deprecation :-) Skip From python@rcn.com Wed May 29 18:44:01 2002 From: python@rcn.com (Raymond Hettinger) Date: Wed, 29 May 2002 13:44:01 -0400 Subject: [Python-Dev] dir() in docs and tests considered irritating References: Message-ID: <000801c20738$6beaace0$5d61accf@othello> From: "Peter Funk" > nnorwitz just fixed a >>> dir(__builtin__) screen shot > in the Python tutorial to add 'PendingDeprecationWarning' there. I built a patch to fill the tp_iter slot for lists. Right away, the regression test test_descrtut.py failed because __iter__ showed up on the dir(list) example. I think everytime we use a dir() example in the docs or in the testsuite, we're creating unnecessary dependencies. It's not a big deal, but if the example or test can make its point without the dir(), it would lower the PITA factor. Raymond Hettinger From mats@laplaza.org Wed May 29 18:55:21 2002 From: mats@laplaza.org (Mats Wichmann) Date: Wed, 29 May 2002 11:55:21 -0600 Subject: [Python-Dev] deprecating string module? In-Reply-To: <20020529140402.28994.11218.Mailman@mail.python.org> Message-ID: <5.1.0.14.1.20020529114935.022c2510@204.151.72.2> >Or maybe we could introduce a new warning category, >SilentDeprecationWarning, which is normally ignored (like >OverflowWarning already is). This could be turned on explicitly with >a -W option so someone wanting to check that their code is >future-proof would have an easy way to do so. > >I do not want to ignore all DeprecationWarning messages by default, as >it would defeat the main purpose of the warning. +1 I'm quite happy to leave deprecation warnings to an external tool, personally. I'm happy to use external tools for a lot of stuff, like "tell me about the likely version compatibility issues of this code". From nas@python.ca Wed May 29 19:10:52 2002 From: nas@python.ca (Neil Schemenauer) Date: Wed, 29 May 2002 11:10:52 -0700 Subject: [Python-Dev] deprecating string module? In-Reply-To: <15604.61914.651010.829734@12-248-41-177.client.attbi.com>; from skip@pobox.com on Wed, May 29, 2002 at 10:20:58AM -0500 References: <3CF486F4.6070302@lemburg.com> <15604.47928.443522.504730@12-248-41-177.client.attbi.com> <062d01c2070e$33945190$0900a8c0@spiff> <15604.61914.651010.829734@12-248-41-177.client.attbi.com> Message-ID: <20020529111052.B12720@glacier.arctrix.com> Skip Montanaro wrote: > Still, it sounds to me like for most purposes there is ample reason to > advocate not using the string module. Sure. Move the documentation for the module into an appendix and add a note saying people should use string methods instead. > Then, my natural inclination is to wonder if it should simply be > deprecated. Removing it will break lots of code. Leaving it alone until Python 3k costs us almost nothing. Can we please just leave it alone? Neil From guido@python.org Wed May 29 19:13:20 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 14:13:20 -0400 Subject: [Python-Dev] Re: String module In-Reply-To: Your message of "Wed, 29 May 2002 13:28:27 EDT." <013501c20736$3f7732c0$5d61accf@othello> References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <200205291544.g4TFibS18433@odiug.zope.com> <00e001c2072f$7b8d2460$5d61accf@othello> <200205291642.g4TGgaf18754@odiug.zope.com> <013501c20736$3f7732c0$5d61accf@othello> Message-ID: <200205291813.g4TIDKK18906@odiug.zope.com> > Instead of making direct substitutes for the character lists, I > propose we take advantage of the opportunity and provide them as > mappings rather than strings. That way, we can get O(1) behavior > instead of O(n) behavior for code like: if c in str.printable: > c='*'. If someone needs to know the contents, they can run > str.printable.keys(). Also, because the dictionary is mutable, > someone can (at runtime) expand or contract the definitions: > str.whitespace.append('_'). I don't like the idea of making this mutable at all. But perhaps these should be replaced by predicates on strings? Most of the constants already have a predicate companion: whitespace -- isspace() lowercase -- islower() uppercase -- isupper() letters -- isalpha() digits -- isdigit() That leaves: hexdigits -- isxdigit() octdigits -- isodigit() punctuation -- ispunct() printable -- isprint() Perhaps we should add isalnum, iscntrl, is graph, to match ? Or perhaps not? (Maybe I'd also like to add isword(), which would be isalnum() or '_' -- this is the definition of \w in the re module.) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed May 29 19:14:04 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 14:14:04 -0400 Subject: [Python-Dev] dir() in docs and tests considered irritating In-Reply-To: Your message of "Wed, 29 May 2002 13:44:01 EDT." <000801c20738$6beaace0$5d61accf@othello> References: <000801c20738$6beaace0$5d61accf@othello> Message-ID: <200205291814.g4TIE4H18922@odiug.zope.com> > I think everytime we use a dir() example in the docs or in > the testsuite, we're creating unnecessary dependencies. > > It's not a big deal, but if the example or test can make its > point without the dir(), it would lower the PITA factor. +1 --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik@pythonware.com Wed May 29 19:20:08 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Wed, 29 May 2002 20:20:08 +0200 Subject: [Python-Dev] deprecating string module? References: <3CF486F4.6070302@lemburg.com> <15604.47928.443522.504730@12-248-41-177.client.attbi.com> <062d01c2070e$33945190$0900a8c0@spiff> <15604.61914.651010.829734@12-248-41-177.client.attbi.com> <20020529111052.B12720@glacier.arctrix.com> Message-ID: <00e401c2073d$78371330$ced241d5@hagrid> neil wrote: > Removing it will break lots of code. Leaving it alone until Python 3k > costs us almost nothing. Can we please just leave it alone? fwiw, I grepped a portion of the pythonware.com source code repository. a rough estimate says that we'll have to change 25,000 to 30,000 lines of code that use the string module in one way or another. From guido@python.org Wed May 29 19:14:56 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 14:14:56 -0400 Subject: [Python-Dev] deprecating string module? In-Reply-To: Your message of "Wed, 29 May 2002 11:10:52 PDT." <20020529111052.B12720@glacier.arctrix.com> References: <3CF486F4.6070302@lemburg.com> <15604.47928.443522.504730@12-248-41-177.client.attbi.com> <062d01c2070e$33945190$0900a8c0@spiff> <15604.61914.651010.829734@12-248-41-177.client.attbi.com> <20020529111052.B12720@glacier.arctrix.com> Message-ID: <200205291814.g4TIEut18936@odiug.zope.com> > Removing it will break lots of code. Leaving it alone until Python 3k > costs us almost nothing. Can we please just leave it alone? That's exactly what the PendingDeprecationWarning is for. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed May 29 19:23:04 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 14:23:04 -0400 Subject: [Python-Dev] _PyImport_LoadDynamicModule questions In-Reply-To: Your message of "Wed, 29 May 2002 12:04:59 EDT." <01cf01c2072a$b69cdeb0$6601a8c0@boostconsulting.com> References: <01cf01c2072a$b69cdeb0$6601a8c0@boostconsulting.com> Message-ID: <200205291823.g4TIN4A18996@odiug.zope.com> > On AIX, building a shared library for use by extension modules and > which uses the Python 'C' API is way harder than it should be**; the > best workaround we could find involves hijacking > _PyImport_LoadDynamicModule to load the shared library and patch up > the symbol references. > > _PyImportLoadDynamicModule is undocumented, AFAICT. Am I not > supposed to touch it? Is it likely to disappear or change its > signature? That's anybody's guess. I have no plans in this area, but it's only an external symbol because we wanted a logical separation between importdl.c and import.c. If someone has a great idea for refactoring this that involves changing its signature, I don't see why not. Now, maybe you're right that this *should* be documented and guaranteed API -- we've promoted internal APIs to that status before without removing the leading underscore (e.g. _PyString_Resize). > On a related note, I wonder about all of the _[A-Z] names in > Python. I know that at least in C++, these names are reserved to the > compiler implementation. I assume the same holds for 'C', and wonder > if Python's use of these names is intentional, on the "to be changed > someday" list, or something else. They're reserved by standard C too, but I figure we'd be safe by using _Py as the prefix. Same thing really as assuming ASCII or 8-bit bytes -- standard C doesn't guarantee those either. --Guido van Rossum (home page: http://www.python.org/~guido/) From pyth@devel.trillke.net Wed May 29 19:29:59 2002 From: pyth@devel.trillke.net (holger krekel) Date: Wed, 29 May 2002 20:29:59 +0200 Subject: [Python-Dev] Re: Deprecation In-Reply-To: <20020529172001.39760.qmail@web9601.mail.yahoo.com>; from s_lott@yahoo.com on Wed, May 29, 2002 at 10:20:01AM -0700 References: <20020529172001.39760.qmail@web9601.mail.yahoo.com> Message-ID: <20020529202959.H17248@prim.han.de> Steven Lott wrote: > The numbering scheme should cover this need. After all, Wirth > had a series of numbered versions for Modula (, 2, 3). > > I think the individual modules may need explicit version > numbering, also. I find 15 of 158 .py files in my release of > the library have __version__ numbers. IMO Enforcing version numbers in standard libraries is a good idea. Eventually, a good versioning scheme for the standard-lib with automated deprecation may be what is needed. Making the introduction of new (or deprecation of) features a) by hand b) by measures of 'x years', example from from PEP4: "The deprecation warning will be added to the module one year after Python 2.3 is released, and the module will be removed one year after that." seems unreliable. How do you construct an if-statement for 'One year after 2.3 is released' if the current version is 2.2 . Instead deprecating a) (semi-) automatically b) by saying '2.3 issues deprecation warning, 2.4 does not support it'. seems much saner. Even a program can be teached to check this. The more intransparent the versioning/deprecation scheme the harder it is to handle and the more people will scream at every deprecation-proposal. deprecated-4223-seconds-after-7th-reply'ly yours, holger From nas@python.ca Wed May 29 19:43:11 2002 From: nas@python.ca (Neil Schemenauer) Date: Wed, 29 May 2002 11:43:11 -0700 Subject: [Python-Dev] release22-maint vs. release22-branch Message-ID: <20020529114311.C12720@glacier.arctrix.com> What's the difference between these two tags? Do we need them both? Neil From paul@prescod.net Wed May 29 19:41:25 2002 From: paul@prescod.net (Paul Prescod) Date: Wed, 29 May 2002 11:41:25 -0700 Subject: [Python-Dev] Re: String module References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <200205291544.g4TFibS18433@odiug.zope.com> <00e001c2072f$7b8d2460$5d61accf@othello> <200205291642.g4TGgaf18754@odiug.zope.com> <013501c20736$3f7732c0$5d61accf@othello> <200205291813.g4TIDKK18906@odiug.zope.com> Message-ID: <3CF520D5.5E71F740@prescod.net> Guido van Rossum wrote: > >... > octdigits -- isodigit() "International organization for standardization" (ISO) digit? And what is an I S X digit? I strongly prefer "ishexdigit" and "isoctdigit". Paul Prescod From guido@python.org Wed May 29 19:46:53 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 14:46:53 -0400 Subject: [Python-Dev] Re: String module Message-ID: <200205291846.g4TIkrS19201@odiug.zope.com> Didn't want to let this useful suggestion miss the archives. ------- Forwarded Message Date: Wed, 29 May 2002 20:37:40 +0200 From: Finn Bock To: Guido van Rossum Subject: Re: [Python-Dev] Re: String module Guido van Rossum wrote: >... > digits -- isdigit() > > That leaves: > > hexdigits -- isxdigit() > octdigits -- isodigit() Java solves this task by a method that takes a radix argument and returns an int for the numeric value and -1 for characters that are invalid in the radix. http://java.sun.com/j2se/1.4/docs/api/java/lang/Character.html#digit(char,%20in t) regards, finn ------- End of Forwarded Message From David Abrahams" <200205291823.g4TIN4A18996@odiug.zope.com> Message-ID: <028401c20742$238f6030$6601a8c0@boostconsulting.com> From: "Guido van Rossum" > > _PyImportLoadDynamicModule is undocumented, AFAICT. Am I not > > supposed to touch it? Is it likely to disappear or change its > > signature? > > That's anybody's guess. I have no plans in this area, but it's only > an external symbol because we wanted a logical separation between > importdl.c and import.c. If someone has a great idea for refactoring > this that involves changing its signature, I don't see why not. > > Now, maybe you're right that this *should* be documented and > guaranteed API -- we've promoted internal APIs to that status before > without removing the leading underscore (e.g. _PyString_Resize). Actually, I'm not going to try to convince anyone to make this a stable public API until after I've failed to convince you that maybe there ought to be a libpython.so on AIX. On AIX, the dynamic linking model is in some ways a lot like that of Windows, and it's really tough for a shared library to get ahold of symbols from an executable (witness the strange implementation of the title function of this email on AIX). If it didn't break something else, it might be a major simplification to move the bulk of Python into a shared library on this platform, just as it is on Windows... especially for poor unsuspecting souls (like me) who try to use the Python 'C' API from a shared library (not an extension module). Without some major hackery to patch up those symbols, it unceremoniously dumps core when you try to call Python functions. > > On a related note, I wonder about all of the _[A-Z] names in > > Python. I know that at least in C++, these names are reserved to the > > compiler implementation. I assume the same holds for 'C', and wonder > > if Python's use of these names is intentional, on the "to be changed > > someday" list, or something else. > > They're reserved by standard C too, but I figure we'd be safe by using > _Py as the prefix. Same thing really as assuming ASCII or 8-bit > bytes -- standard C doesn't guarantee those either. OK; what's the intended meaning of the prefix? -Dave From guido@python.org Wed May 29 19:51:59 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 14:51:59 -0400 Subject: [Python-Dev] release22-maint vs. release22-branch In-Reply-To: Your message of "Wed, 29 May 2002 11:43:11 PDT." <20020529114311.C12720@glacier.arctrix.com> References: <20020529114311.C12720@glacier.arctrix.com> Message-ID: <200205291852.g4TIq0w19266@odiug.zope.com> > What's the difference between these two tags? Do we need them both? AFAIK: release22-maint is what we use for post-2.2 maintenance releases release22-branch was used briefly just before the 2.2 release Also: release22 was the 2.2 release itself r221 was the 2.2.1 release --Guido van Rossum (home page: http://www.python.org/~guido/) From nas@python.ca Wed May 29 20:05:44 2002 From: nas@python.ca (Neil Schemenauer) Date: Wed, 29 May 2002 12:05:44 -0700 Subject: [Python-Dev] release22-maint vs. release22-branch In-Reply-To: <200205291852.g4TIq0w19266@odiug.zope.com>; from guido@python.org on Wed, May 29, 2002 at 02:51:59PM -0400 References: <20020529114311.C12720@glacier.arctrix.com> <200205291852.g4TIq0w19266@odiug.zope.com> Message-ID: <20020529120544.D12720@glacier.arctrix.com> Guido van Rossum wrote: > release22-maint > is what we use for post-2.2 maintenance releases > > release22-branch > was used briefly just before the 2.2 release So can we do "cvs tag -d release22-branch"? It took me a while to figure out which tag to use for bugfixes. Neil From guido@python.org Wed May 29 19:55:57 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 14:55:57 -0400 Subject: [Python-Dev] _PyImport_LoadDynamicModule questions In-Reply-To: Your message of "Wed, 29 May 2002 14:51:10 EDT." <028401c20742$238f6030$6601a8c0@boostconsulting.com> References: <01cf01c2072a$b69cdeb0$6601a8c0@boostconsulting.com> <200205291823.g4TIN4A18996@odiug.zope.com> <028401c20742$238f6030$6601a8c0@boostconsulting.com> Message-ID: <200205291855.g4TItvo21615@odiug.zope.com> > Actually, I'm not going to try to convince anyone to make this a stable > public API until after I've failed to convince you that maybe there ought > to be a libpython.so on AIX. You're talking to the wrong guy. I have no access to AIX, no experience with it, and no understanding of it. Somebody else will have to judge your recommendation, not me. > On AIX, the dynamic linking model is in some ways a lot like that of > Windows, and it's really tough for a shared library to get ahold of > symbols from an executable (witness the strange implementation of > the title function of this email on AIX). If it didn't break > something else, it might be a major simplification to move the bulk > of Python into a shared library on this platform, just as it is on > Windows... especially for poor unsuspecting souls (like me) who try > to use the Python 'C' API from a shared library (not an extension > module). Without some major hackery to patch up those symbols, it > unceremoniously dumps core when you try to call Python functions. Maybe Anthony Baxter can corrobborate your story. > > > On a related note, I wonder about all of the _[A-Z] names in > > > Python. I know that at least in C++, these names are reserved to the > > > compiler implementation. I assume the same holds for 'C', and wonder > > > if Python's use of these names is intentional, on the "to be changed > > > someday" list, or something else. > > > > They're reserved by standard C too, but I figure we'd be safe by using > > _Py as the prefix. Same thing really as assuming ASCII or 8-bit > > bytes -- standard C doesn't guarantee those either. > > OK; what's the intended meaning of the prefix? They are for names that have to be external symbols because they are used across multiple files, but are not part of the official API that we recommend to extension writers. We reserve the right to remove these APIs or change them incompatibly (although when we find that one of them has accidentally become useful in some way we usually don't do that). --Guido van Rossum (home page: http://www.python.org/~guido/) From mal@lemburg.com Wed May 29 20:07:50 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 29 May 2002 21:07:50 +0200 Subject: [Python-Dev] Re: String module References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <200205291544.g4TFibS18433@odiug.zope.com> <00e001c2072f$7b8d2460$5d61accf@othello> <200205291642.g4TGgaf18754@odiug.zope.com> <013501c20736$3f7732c0$5d61accf@othello> <200205291813.g4TIDKK18906@odiug.zope.com> Message-ID: <3CF52706.10008@lemburg.com> Guido van Rossum wrote: > Most of the constants already have a predicate companion: > > whitespace -- isspace() > lowercase -- islower() > uppercase -- isupper() > letters -- isalpha() > digits -- isdigit() > > That leaves: > > hexdigits -- isxdigit() > octdigits -- isodigit() > punctuation -- ispunct() > printable -- isprint() > > Perhaps we should add isalnum, iscntrl, is graph, to match ? > Or perhaps not? (Maybe I'd also like to add isword(), which would be > isalnum() or '_' -- this is the definition of \w in the re module.) FYI, Unicode has these: {"islower", (PyCFunction) unicode_islower, METH_NOARGS, islower__doc__}, {"isupper", (PyCFunction) unicode_isupper, METH_NOARGS, isupper__doc__}, {"istitle", (PyCFunction) unicode_istitle, METH_NOARGS, istitle__doc__}, {"isspace", (PyCFunction) unicode_isspace, METH_NOARGS, isspace__doc__}, {"isdecimal", (PyCFunction) unicode_isdecimal, METH_NOARGS, isdecimal__doc__}, {"isdigit", (PyCFunction) unicode_isdigit, METH_NOARGS, isdigit__doc__}, {"isnumeric", (PyCFunction) unicode_isnumeric, METH_NOARGS, isnumeric__doc__}, {"isalpha", (PyCFunction) unicode_isalpha, METH_NOARGS, isalpha__doc__}, {"isalnum", (PyCFunction) unicode_isalnum, METH_NOARGS, isalnum__doc__}, Note that unlike for ASCII strings, the characters all have defined categories. Strings miss a few of these: {"islower", (PyCFunction)string_islower, METH_NOARGS, islower__doc__}, {"isupper", (PyCFunction)string_isupper, METH_NOARGS, isupper__doc__}, {"isspace", (PyCFunction)string_isspace, METH_NOARGS, isspace__doc__}, {"isdigit", (PyCFunction)string_isdigit, METH_NOARGS, isdigit__doc__}, {"istitle", (PyCFunction)string_istitle, METH_NOARGS, istitle__doc__}, {"isalpha", (PyCFunction)string_isalpha, METH_NOARGS, isalpha__doc__}, {"isalnum", (PyCFunction)string_isalnum, METH_NOARGS, isalnum__doc__}, -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/ From aleax@aleax.it Wed May 29 20:07:43 2002 From: aleax@aleax.it (Alex Martelli) Date: Wed, 29 May 2002 21:07:43 +0200 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <20020529111113.B15532@eecs.tufts.edu> References: <8BB00CF9-7275-11D6-8740-003065517236@oratrix.com> <200205282049.g4SKnj009428@odiug.zope.com> <20020529111113.B15532@eecs.tufts.edu> Message-ID: On Wednesday 29 May 2002 05:11 pm, Michael Gilfix wrote: ... > get burnt. Perhaps Python should adopt a versioning number such > that major changes occur during odd numbers, etc. Just so users Yep. I proposed this in the very first message of the thread with this subject, about 2 months ago I believe. Linux (and perhaps more relevantly Ruby, a language very close to Python in many respects) use even minor versions for the releases of the stable-track (slow, prudent and backwards-compatible change -- in theory, at least; not sure how good is Ruby at satisfying this promise... Linux is so-so), odd minor versions for the releases of the experimental-track (fast, frequently-released, not necessarily backwards compatible changes). However, after bouncing the idea back and forth a bit, in several variations suggested by various people, Guido ended up rejecting it. It may be satisfactory for Ruby, but he decided that it would not be for Python, no matter how close the two cases appear to others. And he _is_ the BDFL, after all. Some help on this score may hopefully come from the recently formed Python Business Forum. Among other tasks the PBF plans to choose and exhaustively test certain Python releases, proposing to Guido to designate those releases "Python in a tie" (I think GvR came up with this specific monicker), then backporting fixes to them &c to ensure the long period of stability typically desired by commercial end-users (I think the target is 18-24 months vs the 6-8 months target of Python minor releases). I apologize if this summary is in any way imprecise -- http://pbf.nuxeo.org/ has authoritative info. Alex From guido@python.org Wed May 29 20:06:58 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 15:06:58 -0400 Subject: [Python-Dev] release22-maint vs. release22-branch In-Reply-To: Your message of "Wed, 29 May 2002 12:05:44 PDT." <20020529120544.D12720@glacier.arctrix.com> References: <20020529114311.C12720@glacier.arctrix.com> <200205291852.g4TIq0w19266@odiug.zope.com> <20020529120544.D12720@glacier.arctrix.com> Message-ID: <200205291906.g4TJ6wI21715@odiug.zope.com> > So can we do "cvs tag -d release22-branch"? It took me a while to > figure out which tag to use for bugfixes. I'm not sure how wise it is to remove tags, ever. Maybe we should have a pep to document the tags. --Guido van Rossum (home page: http://www.python.org/~guido/) From aleax@aleax.it Wed May 29 20:19:44 2002 From: aleax@aleax.it (Alex Martelli) Date: Wed, 29 May 2002 21:19:44 +0200 Subject: [Python-Dev] Re: PendingDeprecationWarning In-Reply-To: References: Message-ID: On Wednesday 29 May 2002 07:39 pm, François Pinard wrote: > [Alex Martelli] > > > I assume the duplication of oct and hex wrt '%o'% and '%x'% was the > > reason to suggest silently-deprecating the former (trying to have 'just > > one obvious way' and all that). > > The same kind of reason says that `str()' and `repr()' could be faded out > in favour of `%s' and `%r' specifiers. This is not likely to happen! :-) Having both repr AND the peculiar `...` operator is another one of those things which I find _really_ hard to explain without handwaving and without sounding negative about Python, a language about which I'm actually enthusiastic. I'd rather lose the `...` (via pendingsilentdeprecation or whatever). As to repr itself, I'd be +0 on taking it out of the builtins, but that's hardly a major issue, nor of course a realistic prospect. str is obviously quite a different kettle of fish -- it's a TYPE and thus it cannot be substituted by whatever operator. It would also be the natural way to put a format-with-whatever-number-base functionality (2-arg str, just like parsing with arbitrary base is 2-arg int -- perfect!). Alex From aleax@aleax.it Wed May 29 20:21:54 2002 From: aleax@aleax.it (Alex Martelli) Date: Wed, 29 May 2002 21:21:54 +0200 Subject: [Python-Dev] String module In-Reply-To: <013501c20736$3f7732c0$5d61accf@othello> References: <200205291642.g4TGgaf18754@odiug.zope.com> <013501c20736$3f7732c0$5d61accf@othello> Message-ID: On Wednesday 29 May 2002 07:28 pm, Raymond Hettinger wrote: ... > strings. That way, we can get O(1) behavior instead of O(n) behavior for > code like: if c in str.printable: c='*'. If someone needs to know the > contents, they can run str.printable.keys(). Also, because the dictionary > is mutable, someone can (at runtime) expand or contract the definitions: > str.whitespace.append('_'). append would of course not work on a dictionary, but the prospect of allowing easy mutation of fundamental built-ins is quite negative in Python -- goes against the grain of the language. A read-only dictionary might be OK. Alex From akuchlin@mems-exchange.org Wed May 29 20:23:02 2002 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Wed, 29 May 2002 15:23:02 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <3CF4D19F.50607@lemburg.com> References: <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> <200205251911.01072.mark@freelance-developer.com> <200205281525.g4SFPxe17659@odiug.zope.com> <3CF3A728.7000604@lemburg.com> <200205281558.g4SFwd317973@odiug.zope.com> <3CF3AE66.8070000@lemburg.com> <20020528165933.GD8699@mems-exchange.org> <3CF4D19F.50607@lemburg.com> Message-ID: <20020529192302.GB25865@ute.mems-exchange.org> On Wed, May 29, 2002 at 03:03:27PM +0200, M.-A. Lemburg wrote: >What should we do to make your work easier ? Just let me know about incompatibilities that you run into with 2.3, and I'll add brief notes about them. (I'm on python-dev and won't miss issues raised here, so don't bother e-mailing me about them, but might easily miss a python-list thread.) Given all the recent flamage over this issue, probably I should have a separate "Porting to 2.3" section that summarizes everything to watch out for. --amk From guido@python.org Wed May 29 20:32:00 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 15:32:00 -0400 Subject: [Python-Dev] Re: String module In-Reply-To: Your message of "Wed, 29 May 2002 21:07:50 +0200." <3CF52706.10008@lemburg.com> References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <200205291544.g4TFibS18433@odiug.zope.com> <00e001c2072f$7b8d2460$5d61accf@othello> <200205291642.g4TGgaf18754@odiug.zope.com> <013501c20736$3f7732c0$5d61accf@othello> <200205291813.g4TIDKK18906@odiug.zope.com> <3CF52706.10008@lemburg.com> Message-ID: <200205291932.g4TJW0g21814@odiug.zope.com> > FYI, Unicode has these: > > {"islower", (PyCFunction) unicode_islower, METH_NOARGS, islower__doc__}, > {"isupper", (PyCFunction) unicode_isupper, METH_NOARGS, isupper__doc__}, > {"istitle", (PyCFunction) unicode_istitle, METH_NOARGS, istitle__doc__}, > {"isspace", (PyCFunction) unicode_isspace, METH_NOARGS, isspace__doc__}, > {"isdecimal", (PyCFunction) unicode_isdecimal, METH_NOARGS, isdecimal__doc__}, > {"isdigit", (PyCFunction) unicode_isdigit, METH_NOARGS, isdigit__doc__}, > {"isnumeric", (PyCFunction) unicode_isnumeric, METH_NOARGS, isnumeric__doc__}, > {"isalpha", (PyCFunction) unicode_isalpha, METH_NOARGS, isalpha__doc__}, > {"isalnum", (PyCFunction) unicode_isalnum, METH_NOARGS, isalnum__doc__}, > > Note that unlike for ASCII strings, the characters all have > defined categories. > > Strings miss a few of these: > > {"islower", (PyCFunction)string_islower, METH_NOARGS, islower__doc__}, > {"isupper", (PyCFunction)string_isupper, METH_NOARGS, isupper__doc__}, > {"isspace", (PyCFunction)string_isspace, METH_NOARGS, isspace__doc__}, > {"isdigit", (PyCFunction)string_isdigit, METH_NOARGS, isdigit__doc__}, > {"istitle", (PyCFunction)string_istitle, METH_NOARGS, istitle__doc__}, > {"isalpha", (PyCFunction)string_isalpha, METH_NOARGS, isalpha__doc__}, > {"isalnum", (PyCFunction)string_isalnum, METH_NOARGS, isalnum__doc__}, I think all of the unicode methods should be added to 8-bit strings, even if they are just aliases for others (what's title case applied to Latin-1? I suppose same as upper case?). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed May 29 20:32:37 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 15:32:37 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Wed, 29 May 2002 21:07:43 +0200." References: <8BB00CF9-7275-11D6-8740-003065517236@oratrix.com> <200205282049.g4SKnj009428@odiug.zope.com> <20020529111113.B15532@eecs.tufts.edu> Message-ID: <200205291932.g4TJWbx21824@odiug.zope.com> > "Python in a tie" (I think GvR came up with this specific monicker) Actually I think it was Andy Robinson. I think he came up with it in a phone conversation we had about the PBF. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed May 29 20:42:08 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 15:42:08 -0400 Subject: [Python-Dev] Re: PendingDeprecationWarning In-Reply-To: Your message of "Wed, 29 May 2002 21:19:44 +0200." References: Message-ID: <200205291942.g4TJg8w21855@odiug.zope.com> > Having both repr AND the peculiar `...` operator is another one of > those things which I find _really_ hard to explain without > handwaving and without sounding negative about Python, a language > about which I'm actually enthusiastic. Give yourself a bit of credit, Alex. Surely you can talk marketing-speak if you want to! FWIW, I consider `...` a historic wart and a failed experiment (even though I use it frequently myself). It appeared in a different form in ABC, where you could use `...` inside a string literal to do variable substitution (expression substitution actually). I think I found it to hard to implement that in the parser, so I decided that `...` would be a separate construct that you could embed in a string using regular string concatenation. But that means that the equivalent of "The sum of $a and $b is $c" would have to be written as "The sum of " + `a` + " and " + `b` + " is " + `c` which contains way too much interpunction. The other killer is that it uses repr() rather than str(). Python 0.0 only had `...`, it had neither str() not repr(). > I'd rather lose the `...` > (via pendingsilentdeprecation or whatever). +0 > As to repr itself, I'd be +0 on taking it out of the builtins, but > that's hardly a major issue, nor of course a realistic prospect. Why get rid of repr()? It's very useful. In error messages I often want to show an object relevant to the message, but if that object is (or may be) a string, I don't want newlines and other control characters in the string to mess up the formatting of the error message. repr() is perfect for that. > str is obviously quite a different kettle of fish -- it's a TYPE and > thus it cannot be substituted by whatever operator. It would also > be the natural way to put a format-with-whatever-number-base > functionality (2-arg str, just like parsing with arbitrary base is > 2-arg int -- perfect!). How useful is formatting numbers with arbitrary bases? I think decimal and hex cover all current use cases; even octal is mostly historic. (I still read octal more quickly than hex, but that's showing my age more than anything else -- I grew up around a CDC mainframe that dumped in octal.) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed May 29 20:52:02 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 15:52:02 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Wed, 29 May 2002 15:23:02 EDT." <20020529192302.GB25865@ute.mems-exchange.org> References: <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> <200205251911.01072.mark@freelance-developer.com> <200205281525.g4SFPxe17659@odiug.zope.com> <3CF3A728.7000604@lemburg.com> <200205281558.g4SFwd317973@odiug.zope.com> <3CF3AE66.8070000@lemburg.com> <20020528165933.GD8699@mems-exchange.org> <3CF4D19F.50607@lemburg.com> <20020529192302.GB25865@ute.mems-exchange.org> Message-ID: <200205291952.g4TJq3H21963@odiug.zope.com> > Given all the recent flamage over this issue, probably I should have > a separate "Porting to 2.3" section that summarizes everything to > watch out for. That section might start by noting that anything that prints Boolean results (e.g. ``print a==b'') or converts them to strings might be affected by the new bool values that are now often returned as Boolean results. Skimming Misc/NEWS might be the most efficient way to find out other things that could affect old code; e.g. I read that xrange's deprecated features are now removed, assert no longer tests for __debug__, an obscure change to __new__ and __init__, a change to pickling objects with __slots__, and a restriction of sys.exit() to a single argument (sure to bite someone). Library: pwd, grp and resource return enhanced tuples; a change in ftplib.retrlines. At the C API level (do you cover that?) the pymalloc changes probably warrant mention. And more, probably. One question is how spin this. Listing a long series of incompatibilities presents the image of a language that has many gratuitous incompatible changes -- which is not true and the opposite of the message we want to convey. --Guido van Rossum (home page: http://www.python.org/~guido/) From mal@lemburg.com Wed May 29 21:01:10 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 29 May 2002 22:01:10 +0200 Subject: [Python-Dev] Re: String module References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <200205291544.g4TFibS18433@odiug.zope.com> <00e001c2072f$7b8d2460$5d61accf@othello> <200205291642.g4TGgaf18754@odiug.zope.com> <013501c20736$3f7732c0$5d61accf@othello> <200205291813.g4TIDKK18906@odiug.zope.com> <3CF52706.10008@lemburg.com> <200205291932.g4TJW0g21814@odiug.zope.com> Message-ID: <3CF53386.8000009@lemburg.com> Guido van Rossum wrote: >>FYI, Unicode has these: >> >> {"islower", (PyCFunction) unicode_islower, METH_NOARGS, islower__doc__}, >> {"isupper", (PyCFunction) unicode_isupper, METH_NOARGS, isupper__doc__}, >> {"istitle", (PyCFunction) unicode_istitle, METH_NOARGS, istitle__doc__}, >> {"isspace", (PyCFunction) unicode_isspace, METH_NOARGS, isspace__doc__}, >> {"isdecimal", (PyCFunction) unicode_isdecimal, METH_NOARGS, isdecimal__doc__}, >> {"isdigit", (PyCFunction) unicode_isdigit, METH_NOARGS, isdigit__doc__}, >> {"isnumeric", (PyCFunction) unicode_isnumeric, METH_NOARGS, isnumeric__doc__}, >> {"isalpha", (PyCFunction) unicode_isalpha, METH_NOARGS, isalpha__doc__}, >> {"isalnum", (PyCFunction) unicode_isalnum, METH_NOARGS, isalnum__doc__}, >> >>Note that unlike for ASCII strings, the characters all have >>defined categories. >> >>Strings miss a few of these: >> >> {"islower", (PyCFunction)string_islower, METH_NOARGS, islower__doc__}, >> {"isupper", (PyCFunction)string_isupper, METH_NOARGS, isupper__doc__}, >> {"isspace", (PyCFunction)string_isspace, METH_NOARGS, isspace__doc__}, >> {"isdigit", (PyCFunction)string_isdigit, METH_NOARGS, isdigit__doc__}, >> {"istitle", (PyCFunction)string_istitle, METH_NOARGS, istitle__doc__}, >> {"isalpha", (PyCFunction)string_isalpha, METH_NOARGS, isalpha__doc__}, >> {"isalnum", (PyCFunction)string_isalnum, METH_NOARGS, isalnum__doc__}, > > > I think all of the unicode methods should be added to 8-bit strings, > even if they are just aliases for others (what's title case applied to > Latin-1? I suppose same as upper case?). >>> for i in range(256): ... assert unichr(i).istitle() == unichr(i).isupper() ... >>> Seems so :-) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/ From mal@lemburg.com Wed May 29 21:06:47 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 29 May 2002 22:06:47 +0200 Subject: [Python-Dev] Re: Stability and change References: <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> <200205251911.01072.mark@freelance-developer.com> <200205281525.g4SFPxe17659@odiug.zope.com> <3CF3A728.7000604@lemburg.com> <200205281558.g4SFwd317973@odiug.zope.com> <3CF3AE66.8070000@lemburg.com> <20020528165933.GD8699@mems-exchange.org> <3CF4D19F.50607@lemburg.com> <20020529192302.GB25865@ute.mems-exchange.org> <200205291952.g4TJq3H21963@odiug.zope.com> Message-ID: <3CF534D7.9020006@lemburg.com> Guido van Rossum wrote: >>Given all the recent flamage over this issue, probably I should have >>a separate "Porting to 2.3" section that summarizes everything to >>watch out for. > > > That section might start by noting that anything that prints Boolean > results (e.g. ``print a==b'') or converts them to strings might be > affected by the new bool values that are now often returned as Boolean > results. > > Skimming Misc/NEWS might be the most efficient way to find out other > things that could affect old code; e.g. I read that xrange's > deprecated features are now removed, assert no longer tests for > __debug__, an obscure change to __new__ and __init__, a change to > pickling objects with __slots__, and a restriction of sys.exit() to a > single argument (sure to bite someone). Library: pwd, grp and > resource return enhanced tuples; a change in ftplib.retrlines. At the > C API level (do you cover that?) the pymalloc changes probably warrant > mention. Please do (maybe as separate section only for the C API changes). > And more, probably. > > One question is how spin this. Listing a long series of > incompatibilities presents the image of a language that has many > gratuitous incompatible changes -- which is not true and the opposite > of the message we want to convey. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/ From mal@lemburg.com Wed May 29 21:06:41 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 29 May 2002 22:06:41 +0200 Subject: [Python-Dev] Re: Stability and change References: <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> <200205251911.01072.mark@freelance-developer.com> <200205281525.g4SFPxe17659@odiug.zope.com> <3CF3A728.7000604@lemburg.com> <200205281558.g4SFwd317973@odiug.zope.com> <3CF3AE66.8070000@lemburg.com> <20020528165933.GD8699@mems-exchange.org> <3CF4D19F.50607@lemburg.com> <20020529192302.GB25865@ute.mems-exchange.org> Message-ID: <3CF534D1.9020004@lemburg.com> Andrew Kuchling wrote: > On Wed, May 29, 2002 at 03:03:27PM +0200, M.-A. Lemburg wrote: > >>What should we do to make your work easier ? > > > Just let me know about incompatibilities that you run into with 2.3, > and I'll add brief notes about them. (I'm on python-dev and won't > miss issues raised here, so don't bother e-mailing me about them, but > might easily miss a python-list thread.) Ok. > Given all the recent flamage over this issue, probably I should have a > separate "Porting to 2.3" section that summarizes everything to watch > out for. Right. Maybe even as separate document, so that a casual visitor doesn't get blown away by the rate of change ;-) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/ From mats@laplaza.org Wed May 29 21:08:58 2002 From: mats@laplaza.org (Mats Wichmann) Date: Wed, 29 May 2002 14:08:58 -0600 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <20020529190902.27855.76828.Mailman@mail.python.org> Message-ID: <5.1.0.14.1.20020529140345.02223e58@204.151.72.2> >Some help on this score may hopefully come from the recently >formed Python Business Forum. Among other tasks the PBF plans >to choose and exhaustively test certain Python releases, proposing >to Guido to designate those releases "Python in a tie" (I think GvR >came up with this specific monicker), then backporting fixes to them >&c to ensure the long period of stability typically desired by commercial >end-users (I think the target is 18-24 months vs the 6-8 months >target of Python minor releases). I apologize if this summary is in >any way imprecise -- http://pbf.nuxeo.org/ has authoritative info. As of today, there's a "real" website (well, it's the same contents): http://www.python-in-business.org/ From aleax@aleax.it Wed May 29 21:13:48 2002 From: aleax@aleax.it (Alex Martelli) Date: Wed, 29 May 2002 22:13:48 +0200 Subject: [Python-Dev] Re: PendingDeprecationWarning In-Reply-To: <200205291942.g4TJg8w21855@odiug.zope.com> References: <200205291942.g4TJg8w21855@odiug.zope.com> Message-ID: On Wednesday 29 May 2002 09:42 pm, Guido van Rossum wrote: > > Having both repr AND the peculiar `...` operator is another one of > > those things which I find _really_ hard to explain without > > handwaving and without sounding negative about Python, a language > > about which I'm actually enthusiastic. > > Give yourself a bit of credit, Alex. Surely you can talk > marketing-speak if you want to! I'm pretty good at marketing _stricto sensu_, but not at obfuscation (not deliberate one, I mean!-). > > As to repr itself, I'd be +0 on taking it out of the builtins, but > > that's hardly a major issue, nor of course a realistic prospect. > > Why get rid of repr()? It's very useful. In error messages I often > want to show an object relevant to the message, but if that object is > (or may be) a string, I don't want newlines and other control > characters in the string to mess up the formatting of the error > message. repr() is perfect for that. Actually, in error messages I most often want to show objects relevant to the message AND other strings or data too, so I almost invariably end up using the % operator to format the message string. Thus, I use the %r formatting specifier in the format string, not a separate call to repr(), of course. The 'representation' _functionality_ is indeed precious, although often it's better supplied by MODULE repr (if I have a string in error to show, I don't want to show an UNLIMITED amount of characters from it -- module repr helps by limiting the amount of data shown). I don't particularly care to have that very nice functionality made available in three ways -- `...`, repr, and %r -- I'd rather have just one. But hey, I'll settle for two:-). > How useful is formatting numbers with arbitrary bases? I think Roughly as useful as _parsing_ numbers in arbitrary bases, as offered by 2-arguments int() -- i.e., not very. It just DOES feel a little weird to have it trivially easy to parse numbers in strange bases but no corresponding ease in _output_. > decimal and hex cover all current use cases; even octal is mostly Almost all, yes. I've found myself using binary and (once) ternary, but mostly in tricky ways rather than as plain I/O. > historic. (I still read octal more quickly than hex, but that's > showing my age more than anything else -- I grew up around a CDC > mainframe that dumped in octal.) We're roughly the same age, I think, and I grew up with DEC & HP minis, CDC mainframes, and Intel and Zilog micros, which had _strong_ octal bias (reading 8080 machine-code dumps in octal was easy -- the typical 1-byte commands had fields of 2-3-3 bits...). The difference is that then I moved to IBM, where I was thoroughly indoctrinated in the beauty of hex (try reading a _370_ machine-code dump in octal...!-). Even IBM's main scripting language *rhymed* with 'hex'...!-) Alex From BPettersen@NAREX.com Wed May 29 21:15:52 2002 From: BPettersen@NAREX.com (Bjorn Pettersen) Date: Wed, 29 May 2002 14:15:52 -0600 Subject: [Python-Dev] True division in embedded Python Message-ID: <60FB8BB7F0EFC7409B75EEEC13E2019221518C@admin56.narex.com> We're currently in the process of adapting Python as the common scripting plug-in for all of our projects (and it's working great :-). We're standardizing on 2.2.1, and unless there is a pressing business need, we'll probably stay with that version for the next 18-24 months. My question is if it's intended for users to start using true division _now_? The reason I'm inclined to do so is that we're going to have snippets of Python code everywhere, including our libraries, disk files, and databases -- it would be a major undertaking to find and fix this later... My second question is how do you enable true division in embedded Python? I've read the documentation and I couldn't find anything appropriate... Any insight would be greatly appreciated. Sincerely, Bjorn Pettersen NAREX Inc. 303.526.4000 ext. 312 303.526.5130 fax www.narex.com From aleax@aleax.it Wed May 29 21:18:37 2002 From: aleax@aleax.it (Alex Martelli) Date: Wed, 29 May 2002 22:18:37 +0200 Subject: [Python-Dev] Re: String module In-Reply-To: <200205291932.g4TJW0g21814@odiug.zope.com> References: <3CF52706.10008@lemburg.com> <200205291932.g4TJW0g21814@odiug.zope.com> Message-ID: On Wednesday 29 May 2002 09:32 pm, Guido van Rossum wrote: ... > > {"istitle", (PyCFunction)string_istitle, METH_NOARGS, istitle__doc__}, ... > I think all of the unicode methods should be added to 8-bit strings, > even if they are just aliases for others (what's title case applied to > Latin-1? I suppose same as upper case?). Current Python strings DO have .istitle() -- it means "the first letter of each substring made of letters is uppercase, all the other letters in the substring are lowercase". (It also means the string DOES have some letters -- I always find it quite tricky that all the string iswhatever methods return False when called on an empty string). >>> "Pol23Zap And Plee+Kuup!".istitle() True Alex From guido@python.org Wed May 29 21:26:36 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 16:26:36 -0400 Subject: [Python-Dev] True division in embedded Python In-Reply-To: Your message of "Wed, 29 May 2002 14:15:52 MDT." <60FB8BB7F0EFC7409B75EEEC13E2019221518C@admin56.narex.com> References: <60FB8BB7F0EFC7409B75EEEC13E2019221518C@admin56.narex.com> Message-ID: <200205292026.g4TKQac22332@odiug.zope.com> > We're currently in the process of adapting Python as the common > scripting plug-in for all of our projects (and it's working great :-). Great! > We're standardizing on 2.2.1, and unless there is a pressing business > need, we'll probably stay with that version for the next 18-24 months. Would you consider upgrading to 2.2.2 when it comes out, and subsequent bugfix releases? The goal for those releases is to make the upgrade 100% painless. > My question is if it's intended for users to start using true division > _now_? The reason I'm inclined to do so is that we're going to have > snippets of Python code everywhere, including our libraries, disk files, > and databases -- it would be a major undertaking to find and fix this > later... According to PEP 238 (http://www.python.org/peps/pep-0238.html), / will default to floor division for all Python 2.x releases, and will switch to true division only in Python 3.0. It doesn't give a timeline for 3.0, but I expect it to be later rather than sooner -- while early alpha releases of 3.0 might appear in two years, it's likely that 3.0 won't be finalized until much later -- say 3-4 years. Enabling the true division dialect now by default is something you should only do if you are sure that you won't be exporting code to other Python installations, and even then you are on your own to make sure that standard library modules aren't affected by the change. We try to make sure that the standard library uses // whenever it needs truncated division and ensures that at least one operand of / is a float when float division is desired, but I can't guarantee that we've caught all cases. > My second question is how do you enable true division in embedded > Python? I've read the documentation and I couldn't find anything > appropriate... On the command line it's -Qnew. Looking through the source of Modules/main.c, it looks like setting the undocumented global variable -Py_QnewFlag to 1 would do it. --Guido van Rossum (home page: http://www.python.org/~guido/) From aleax@aleax.it Wed May 29 21:32:02 2002 From: aleax@aleax.it (Alex Martelli) Date: Wed, 29 May 2002 22:32:02 +0200 Subject: [Python-Dev] True division in embedded Python In-Reply-To: <60FB8BB7F0EFC7409B75EEEC13E2019221518C@admin56.narex.com> References: <60FB8BB7F0EFC7409B75EEEC13E2019221518C@admin56.narex.com> Message-ID: On Wednesday 29 May 2002 10:15 pm, Bjorn Pettersen wrote: > We're currently in the process of adapting Python as the common > scripting plug-in for all of our projects (and it's working great :-). Great idea. > We're standardizing on 2.2.1, and unless there is a pressing business > need, we'll probably stay with that version for the next 18-24 months. Interestingly enough, the same target-range mentioned by many business users and therefore targeted by the Python Business Forum. See www.python-in-business.org (if it doesn't answer yet -- doesn't to me -- pbf.nuxeo.com should for the moment be equivalent). > My question is if it's intended for users to start using true division > _now_? The reason I'm inclined to do so is that we're going to have > snippets of Python code everywhere, including our libraries, disk files, > and databases -- it would be a major undertaking to find and fix this > later... Since you're looking forwards, adopting true division seems sensible. > My second question is how do you enable true division in embedded > Python? I've read the documentation and I couldn't find anything > appropriate... > > Any insight would be greatly appreciated. See Modules/main.c line 153 ff, where it handles the -Q option. Unfortunately it seems this requires setting a flag whose name starts with an underline, _Py_QnewFlag, therefore an internal interface -- however, hopefully it won't go away as long as you're staying with the 2.2.[1+] "family" (surely it won't change in 2.2.1 itself, of course, but even should you want to upgrade to 2.2.2 or whatever in the future to gain bugfixes or optimizations, it seems to me you should be safe on this point). Alex From akuchlin@mems-exchange.org Wed May 29 21:41:30 2002 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Wed, 29 May 2002 16:41:30 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200205291952.g4TJq3H21963@odiug.zope.com> References: <200205251911.01072.mark@freelance-developer.com> <200205281525.g4SFPxe17659@odiug.zope.com> <3CF3A728.7000604@lemburg.com> <200205281558.g4SFwd317973@odiug.zope.com> <3CF3AE66.8070000@lemburg.com> <20020528165933.GD8699@mems-exchange.org> <3CF4D19F.50607@lemburg.com> <20020529192302.GB25865@ute.mems-exchange.org> <200205291952.g4TJq3H21963@odiug.zope.com> Message-ID: <20020529204130.GA26155@ute.mems-exchange.org> On Wed, May 29, 2002 at 03:52:02PM -0400, Guido van Rossum wrote: >resource return enhanced tuples; a change in ftplib.retrlines. At the >C API level (do you cover that?) the pymalloc changes probably warrant >mention. Yes, C-level changes are covered. >One question is how spin this. Listing a long series of >incompatibilities presents the image of a language that has many >gratuitous incompatible changes -- which is not true and the opposite >of the message we want to convey. This way madness lies. Document nothing and there will certainly be complaints. Why worry about losers who are in essence complaining that the documentation is *too* complete? --amk From guido@python.org Wed May 29 21:44:20 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 16:44:20 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Wed, 29 May 2002 16:41:30 EDT." <20020529204130.GA26155@ute.mems-exchange.org> References: <200205251911.01072.mark@freelance-developer.com> <200205281525.g4SFPxe17659@odiug.zope.com> <3CF3A728.7000604@lemburg.com> <200205281558.g4SFwd317973@odiug.zope.com> <3CF3AE66.8070000@lemburg.com> <20020528165933.GD8699@mems-exchange.org> <3CF4D19F.50607@lemburg.com> <20020529192302.GB25865@ute.mems-exchange.org> <200205291952.g4TJq3H21963@odiug.zope.com> <20020529204130.GA26155@ute.mems-exchange.org> Message-ID: <200205292044.g4TKiKH22752@odiug.zope.com> > >One question is how spin this. Listing a long series of > >incompatibilities presents the image of a language that has many > >gratuitous incompatible changes -- which is not true and the opposite > >of the message we want to convey. > > This way madness lies. Document nothing and there will certainly be > complaints. Why worry about losers who are in essence complaining > that the documentation is *too* complete? All I'm asking is a leading paragraph something like this: """While the Python developers do their best to make every subsequent release backwards compatible with previous releases, in a system of this complexity, sometimes incompatibilities in odd corners of the language are unavoidable. While we expect that these will affect only a small minority of Python users, we document them here for completeness.""" --Guido van Rossum (home page: http://www.python.org/~guido/) From gmcm@hypernet.com Wed May 29 21:58:49 2002 From: gmcm@hypernet.com (Gordon McMillan) Date: Wed, 29 May 2002 16:58:49 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200205291952.g4TJq3H21963@odiug.zope.com> References: Your message of "Wed, 29 May 2002 15:23:02 EDT." <20020529192302.GB25865@ute.mems-exchange.org> Message-ID: <3CF508C9.22341.3DD125D5@localhost> On 29 May 2002 at 15:52, Guido van Rossum wrote: > One question is how spin this. Listing a long > series of incompatibilities presents the image of a > language that has many gratuitous incompatible > changes -- which is not true and the opposite of > the message we want to convey. Hopefully, Python-in-a-tie changes the equilibrium somewhat. You can point the "pace of change" whiners to it (once there's something to point to ); conversely, you needn't worry much about spinning a non-PyTie release. Because of the longer release cycle, a PyTie release *will* involve a lot of language changes. MAL's idea of a migration guide is great - just make sure it's presented as an "aid", not a "warning". -- Gordon http://www.mcmillan-inc.com/ From akuchlin@mems-exchange.org Wed May 29 22:02:08 2002 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Wed, 29 May 2002 17:02:08 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200205292044.g4TKiKH22752@odiug.zope.com> References: <200205281525.g4SFPxe17659@odiug.zope.com> <3CF3A728.7000604@lemburg.com> <200205281558.g4SFwd317973@odiug.zope.com> <3CF3AE66.8070000@lemburg.com> <20020528165933.GD8699@mems-exchange.org> <3CF4D19F.50607@lemburg.com> <20020529192302.GB25865@ute.mems-exchange.org> <200205291952.g4TJq3H21963@odiug.zope.com> <20020529204130.GA26155@ute.mems-exchange.org> <200205292044.g4TKiKH22752@odiug.zope.com> Message-ID: <20020529210207.GA26345@ute.mems-exchange.org> On Wed, May 29, 2002 at 04:44:20PM -0400, Guido van Rossum wrote: >"""While the Python developers do their best to make every subsequent >release backwards compatible with previous releases, in a system of >this complexity, sometimes incompatibilities in odd corners of the >language are unavoidable. While we expect that these will affect only I don't like this text, because the section on porting should also mention changes such as 'yield' becoming a keyword. Changes like that are hardly accidental, and hardly in an obscure corner. I'll try to come up with a different leading paragraph. --amk From guido@python.org Wed May 29 22:04:06 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 17:04:06 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Wed, 29 May 2002 14:08:58 MDT." <5.1.0.14.1.20020529140345.02223e58@204.151.72.2> References: <5.1.0.14.1.20020529140345.02223e58@204.151.72.2> Message-ID: <200205292104.g4TL46P23012@odiug.zope.com> > As of today, there's a "real" website (well, it's the same contents): > > http://www.python-in-business.org/ Thanks -- I've added them to the python.org homepage. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed May 29 22:03:45 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 17:03:45 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Wed, 29 May 2002 17:02:08 EDT." <20020529210207.GA26345@ute.mems-exchange.org> References: <200205281525.g4SFPxe17659@odiug.zope.com> <3CF3A728.7000604@lemburg.com> <200205281558.g4SFwd317973@odiug.zope.com> <3CF3AE66.8070000@lemburg.com> <20020528165933.GD8699@mems-exchange.org> <3CF4D19F.50607@lemburg.com> <20020529192302.GB25865@ute.mems-exchange.org> <200205291952.g4TJq3H21963@odiug.zope.com> <20020529204130.GA26155@ute.mems-exchange.org> <200205292044.g4TKiKH22752@odiug.zope.com> <20020529210207.GA26345@ute.mems-exchange.org> Message-ID: <200205292103.g4TL3jb22999@odiug.zope.com> > On Wed, May 29, 2002 at 04:44:20PM -0400, Guido van Rossum wrote: > >"""While the Python developers do their best to make every subsequent > >release backwards compatible with previous releases, in a system of > >this complexity, sometimes incompatibilities in odd corners of the > >language are unavoidable. While we expect that these will affect only > > I don't like this text, because the section on porting should also > mention changes such as 'yield' becoming a keyword. Changes like that > are hardly accidental, and hardly in an obscure corner. I'll try to > come up with a different leading paragraph. Good point. Though yield has been foreshadowed by a warning and a future statement in 2.2, so it's only news for people porting from 2.1 or before. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com Wed May 29 22:31:36 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 29 May 2002 16:31:36 -0500 Subject: [Python-Dev] True division in embedded Python In-Reply-To: <60FB8BB7F0EFC7409B75EEEC13E2019221518C@admin56.narex.com> References: <60FB8BB7F0EFC7409B75EEEC13E2019221518C@admin56.narex.com> Message-ID: <15605.18616.286870.35490@12-248-41-177.client.attbi.com> Bjorn> We're standardizing on 2.2.1, and unless there is a pressing Bjorn> business need, we'll probably stay with that version for the next Bjorn> 18-24 months. I'd suggest you standardize on 2.2.x to allow yourselves enough wiggle room to switch to later bug fix releases of the same feature set. -- Skip Montanaro (skip@pobox.com - http://www.mojam.com/) Boycott Netflix - they spam From skip@pobox.com Wed May 29 22:40:26 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 29 May 2002 16:40:26 -0500 Subject: [Python-Dev] True division in embedded Python In-Reply-To: <200205292026.g4TKQac22332@odiug.zope.com> References: <60FB8BB7F0EFC7409B75EEEC13E2019221518C@admin56.narex.com> <200205292026.g4TKQac22332@odiug.zope.com> Message-ID: <15605.19146.687116.17742@12-248-41-177.client.attbi.com> >> My second question is how do you enable true division in embedded >> Python? I've read the documentation and I couldn't find anything >> appropriate... Guido> On the command line it's -Qnew. Looking through the source of Guido> Modules/main.c, it looks like setting the undocumented global Guido> variable _Py_QnewFlag to 1 would do it. Sounds like there's a need for some documented access functions. At the very least the docs should mention how programmers must currently set these flags. I'll be happy to send in a patch for the ext/embed docs and/or provide a patch to implement the necessary calls. Skip From gball@cfa.harvard.edu Wed May 29 22:46:17 2002 From: gball@cfa.harvard.edu (Greg Ball) Date: Wed, 29 May 2002 17:46:17 -0400 (EDT) Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200205291952.g4TJq3H21963@odiug.zope.com> Message-ID: On Wed, 29 May 2002, Guido van Rossum wrote: ... > One question is how spin this. Listing a long series of > incompatibilities presents the image of a language that has many > gratuitous incompatible changes -- which is not true and the opposite > of the message we want to convey. Maybe an Anti-News document like the one that accompanies Emacs. This explains all the 'benefits' of downgrading to the previous version. It could be useful for porting between versions while at the same time poking fun in a good-natured way at those living in the past... I have rudely interrupted here before without introducing myself. I'm a graduate student in astronomy. I started using python in 1998 because of the Monty Python jokes. I use Numerical Python a lot when I'm working, and lurk around here when I should be. -- Greg Ball From martin@v.loewis.de Wed May 29 22:56:20 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 29 May 2002 23:56:20 +0200 Subject: [Python-Dev] deprecating string module? In-Reply-To: <002801c20714$ad139fc0$0900a8c0@spiff> References: <3CF486F4.6070302@lemburg.com> <15604.47928.443522.504730@12-248-41-177.client.attbi.com> <3CF4C15A.7070606@lemburg.com> <062701c20710$5679a8c0$6300000a@holdenweb.com> <002801c20714$ad139fc0$0900a8c0@spiff> Message-ID: "Fredrik Lundh" writes: > > I can understand this, but isn't "1.5.2" a little arbitrary. > > have you tried migrating a huge python system from 1.5.2 to 2.x? > if so, what did you learn? That it is easier than porting from 1.4. Regards, Martin From martin@v.loewis.de Wed May 29 22:58:21 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 29 May 2002 23:58:21 +0200 Subject: [Python-Dev] deprecating string module? In-Reply-To: <15604.47718.334686.681186@12-248-41-177.client.attbi.com> References: <15603.63458.608739.454153@beluga.mojam.com> <15604.47718.334686.681186@12-248-41-177.client.attbi.com> Message-ID: Skip Montanaro writes: > Also, the data objects obviously have to be put somewhere. Before doing any > of this, we need to decide where this stuff belongs. I doubt the technical > effort involved will be too challenging. So we don't need to make a decision today, right? We can delay that until those easy problems are solved. Regards, Martin From David Abrahams" <200205291823.g4TIN4A18996@odiug.zope.com> <028401c20742$238f6030$6601a8c0@boostconsulting.com> <200205291855.g4TItvo21615@odiug.zope.com> Message-ID: <03b201c2075b$b9df4230$6601a8c0@boostconsulting.com> From: "Guido van Rossum" > > Actually, I'm not going to try to convince anyone to make this a stable > > public API until after I've failed to convince you that maybe there ought > > to be a libpython.so on AIX. > > You're talking to the wrong guy. I have no access to AIX, no > experience with it, and no understanding of it. Somebody else will > have to judge your recommendation, not me. By "you" I meant "the wider you", not "the royal you", nor you (Guido) specifically. > > On AIX, the dynamic linking model is in some ways a lot like that of > > Windows, and it's really tough for a shared library to get ahold of > > symbols from an executable (witness the strange implementation of > > the title function of this email on AIX). If it didn't break > > something else, it might be a major simplification to move the bulk > > of Python into a shared library on this platform, just as it is on > > Windows... especially for poor unsuspecting souls (like me) who try > > to use the Python 'C' API from a shared library (not an extension > > module). Without some major hackery to patch up those symbols, it > > unceremoniously dumps core when you try to call Python functions. > > Maybe Anthony Baxter can corrobborate your story. I hope "you" find it worth looking into at least. From BPettersen@NAREX.com Wed May 29 23:00:41 2002 From: BPettersen@NAREX.com (Bjorn Pettersen) Date: Wed, 29 May 2002 16:00:41 -0600 Subject: [Python-Dev] True division in embedded Python Message-ID: <60FB8BB7F0EFC7409B75EEEC13E2019221518E@admin56.narex.com> > From: Guido van Rossum [mailto:guido@python.org]=20 [snip] > > We're standardizing on 2.2.1, and unless there is a=20 > > pressing business need, we'll probably stay with=20 > > that version for the next 18-24 months. >=20 > Would you consider upgrading to 2.2.2 when it comes out, and=20 > subsequent bugfix releases? The goal for those releases is=20 > to make the upgrade 100% painless. We would probably consider it. What it comes down to is whether the new release is binary compatible (which I'm assuming it is), and if it included a bugfix that we found prudent to incorporate.=20 Don't misunderstand me, it's not that we don't want to update -- the developers here would probably be quite happy to work on the latest and greatest -- updating just inherently takes time (money) and it's not very exciting work. We'd just rather spend our time on something that has a more direct revenue source :-) Thanks for answering my question about true division. Sincerely, Bjorn Pettersen NAREX Inc. 303.526.4000 ext. 312 303.526.5130 fax www.narex.com From martin@v.loewis.de Wed May 29 23:04:33 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 30 May 2002 00:04:33 +0200 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200205292044.g4TKiKH22752@odiug.zope.com> References: <200205251911.01072.mark@freelance-developer.com> <200205281525.g4SFPxe17659@odiug.zope.com> <3CF3A728.7000604@lemburg.com> <200205281558.g4SFwd317973@odiug.zope.com> <3CF3AE66.8070000@lemburg.com> <20020528165933.GD8699@mems-exchange.org> <3CF4D19F.50607@lemburg.com> <20020529192302.GB25865@ute.mems-exchange.org> <200205291952.g4TJq3H21963@odiug.zope.com> <20020529204130.GA26155@ute.mems-exchange.org> <200205292044.g4TKiKH22752@odiug.zope.com> Message-ID: Guido van Rossum writes: > All I'm asking is a leading paragraph something like this: > > """While the Python developers do their best to make every subsequent > release backwards compatible with previous releases, in a system of > this complexity, sometimes incompatibilities in odd corners of the > language are unavoidable. While we expect that these will affect only > a small minority of Python users, we document them here for > completeness.""" When I read complete change lists, I always think "don't they have more important things to worry about", with the implication that the author of the list did a good job, and unless I spot something that I know will affect me, it gives me the warm feeling that I don't have to worry about unexpected breakages. So I'm all for producing a as-complete-as-possible list, and let readers judge the relevance of this list themselves. Regards, Martin From martin@v.loewis.de Wed May 29 23:08:09 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 30 May 2002 00:08:09 +0200 Subject: [Python-Dev] Re: String module In-Reply-To: <200205291932.g4TJW0g21814@odiug.zope.com> References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <200205291544.g4TFibS18433@odiug.zope.com> <00e001c2072f$7b8d2460$5d61accf@othello> <200205291642.g4TGgaf18754@odiug.zope.com> <013501c20736$3f7732c0$5d61accf@othello> <200205291813.g4TIDKK18906@odiug.zope.com> <3CF52706.10008@lemburg.com> <200205291932.g4TJW0g21814@odiug.zope.com> Message-ID: Guido van Rossum writes: > I think all of the unicode methods should be added to 8-bit strings, > even if they are just aliases for others (what's title case applied to > Latin-1? I suppose same as upper case?). So those would be locale dependent? (the Unicode ones are not). Regards, Martin From martin@v.loewis.de Wed May 29 23:28:20 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 30 May 2002 00:28:20 +0200 Subject: [Python-Dev] release22-maint vs. release22-branch In-Reply-To: <200205291852.g4TIq0w19266@odiug.zope.com> References: <20020529114311.C12720@glacier.arctrix.com> <200205291852.g4TIq0w19266@odiug.zope.com> Message-ID: Guido van Rossum writes: > release22-maint > is what we use for post-2.2 maintenance releases > > release22-branch > was used briefly just before the 2.2 release It is custom in other projects to have only one branch for both, and having the x.y.0 release just be a tag on the x.y maintenance branch. This requires freezing the maintenance just before the release to everybody but the release manager, but since the branch is not use for anything but the immediate upcoming release, this is no problem. I suggest that this is done that way for Python 2.3 also. Regards, Martin From martin@v.loewis.de Wed May 29 23:35:11 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 30 May 2002 00:35:11 +0200 Subject: [Python-Dev] _PyImport_LoadDynamicModule questions In-Reply-To: <01cf01c2072a$b69cdeb0$6601a8c0@boostconsulting.com> References: <01cf01c2072a$b69cdeb0$6601a8c0@boostconsulting.com> Message-ID: "David Abrahams" writes: > On AIX, building a shared library for use by extension modules and which > uses the Python 'C' API is way harder than it should be**; the best > workaround we could find involves hijacking _PyImport_LoadDynamicModule to > load the shared library and patch up the symbol references. > > _PyImportLoadDynamicModule is undocumented, AFAICT. Am I not supposed to > touch it? Is it likely to disappear or change its signature? As Guido explains, you should not use it in an extension module: it may go away without notice, or change its signature. As for dynamic linking on AIX: It would be really good if somebody stepped forward who claims to understand the dynamic loader of AIX, and rewrite Python to make this work more "nicely". Before starting the rewrite, I'd be really curious to hear the full story of why it currently is the way it is, and how this state could be improved. If you find that a "good" redesign requires a shared libpython, then so be it - but I'm still quite fond of the "single executable" appraoch, so preserving that would be even better. Regards, Martin From python@rcn.com Wed May 29 23:51:02 2002 From: python@rcn.com (Raymond Hettinger) Date: Wed, 29 May 2002 18:51:02 -0400 Subject: [Python-Dev] String module References: <200205291642.g4TGgaf18754@odiug.zope.com> <013501c20736$3f7732c0$5d61accf@othello> Message-ID: <00b801c20763$4f3dc480$0fea7ad1@othello> From: "Alex Martelli" > > strings. That way, we can get O(1) behavior instead of O(n) behavior for > > code like: if c in str.printable: c='*'. If someone needs to know the > > contents, they can run str.printable.keys(). Also, because the dictionary > > is mutable, someone can (at runtime) expand or contract the definitions: > > str.whitespace.append('_'). > > append would of course not work on a dictionary, but the prospect of > allowing easy mutation of fundamental built-ins is quite negative in Python -- > goes against the grain of the language. A read-only dictionary might be OK. I like the read-only dictionary better than the boolean test methods. It minimizes the effort in upgrading existing code of the form: for c in string.lowercase: do something if c in string.lowercase: do something I can global search/replace string.lowercase with str.lower in dictionary form and everything will run fine (and faster too). Also, I like the mapping because provides a way to see the membership. With the string form or the mapping form, it's easy to find-out exactly what is defined as whitespace. If there is a C coded boolean test, I have to filter the whole alphabet or look it up in the docs. If we can get some agreement that this is the way to go, I can work with the OP on a revised patch so we get the string module silently deprecated. Raymond Hettinger From pinard@iro.umontreal.ca Thu May 30 00:09:17 2002 From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard) Date: 29 May 2002 19:09:17 -0400 Subject: [Python-Dev] Re: PendingDeprecationWarning In-Reply-To: <200205291942.g4TJg8w21855@odiug.zope.com> References: <200205291942.g4TJg8w21855@odiug.zope.com> Message-ID: [Guido van Rossum] > FWIW, I consider `...` a historic wart and a failed experiment (even > though I use it frequently myself). Hey, hey! Glad to hear you do not like it so much yourself :-). I keep saying to people around me that `print' and backquotes are merely debugging devices, that we should not really keep in production code. Nobody speaks about `input()', of course. :-) > I grew up around a CDC mainframe that dumped in octal.) Hi there! I could probably debug an octal dump even today, the CPU codes are rather easy to remember :-). Many members of the PDP series were also favouring octal. Isn't the Cray which was using a mix of hexadecimal and octal in dumps (or do I mix it with something else)? But this is history. I would prefer decimal everywhere nowadays. Too bad that Unicode pushed so strongly on hexadecimal, this is a bit anachronical. -- François Pinard http://www.iro.umontreal.ca/~pinard From tim.one@comcast.net Thu May 30 00:47:43 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 29 May 2002 19:47:43 -0400 Subject: [Python-Dev] Re: PendingDeprecationWarning In-Reply-To: Message-ID: [Fran=E7ois Pinard] > ... > Hi there! I could probably debug an octal dump even today, the CPU= codes > are rather easy to remember :-). Many members of the PDP series we= re also > favouring octal. Isn't the Cray which was using a mix of hexadecim= al and > octal in dumps (or do I mix it with something else)? The early Cray software used only octal, since everyone there came fr= om CDC, and loved octal from 60-bit words, 18-bit address registers, and 6-bi= t characters (). Oct= al proved surprisingly pleasant for 64-bit words too! It left the sign = bit off by itself in the 22nd octal digit, and it was said that Seymour made = the exponent field in Cray floats 15 bits wide so that it would be easy t= o read off from octal dumps too. Octal was so deeply ingrained in Cray culture that a coworker filled = out her timesheet in octal once, 10 hours per day for her 2-week vacation, su= mming to 120 hours. Our boss signed off on it because it looked fine to hi= m. This is the same boss who loved to tell the story of taking his famil= y out for a drive, and excitedly exclaiming "Hey, kids! Look!! The odomet= er is about to flip over to 40000!". Of course it read 37777 at the time, = and when it flipped to 37778 "they looked at me funny, and my family life= was never the same again". Then Cray hired a bunch of young crybabies (like me), who-- with some justification --pointed out that octal dumps were really hard to scan= for character data, given that Cray had moved to 8-bit characters. > But this is history. It doesn't have to be. Unicode surely has nothing going for it over= CDC Display Code . > I would prefer decimal everywhere nowadays. Too bad that Unicode p= ushed > so strongly on hexadecimal, this is a bit anachronical. I suggested to Guido today that we deprecate decimal literals in Pyth= on, in favor of octal everywhere. A killer advantage is that every binary floating-point number can be printed exactly with a few dozen octal d= igits, and that should squash a lot of newbie complaints about confusing floating-point rounding errors. it's-all-about-doing-what's-best-for-the-children-ly y'rs - tim From pinard@iro.umontreal.ca Thu May 30 01:03:57 2002 From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard) Date: 29 May 2002 20:03:57 -0400 Subject: [Python-Dev] Re: String module In-Reply-To: <200205291813.g4TIDKK18906@odiug.zope.com> References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <200205291544.g4TFibS18433@odiug.zope.com> <00e001c2072f$7b8d2460$5d61accf@othello> <200205291642.g4TGgaf18754@odiug.zope.com> <013501c20736$3f7732c0$5d61accf@othello> <200205291813.g4TIDKK18906@odiug.zope.com> Message-ID: [Guido van Rossum] > Perhaps we should add isalnum, iscntrl, is graph, to match ? > Or perhaps not? (Maybe I'd also like to add isword(), which would be > isalnum() or '_' -- this is the definition of \w in the re module.) This reminds me that I often miss, in the standard `ctype.h' and related, a function that would un-combine a character into its base character and its diacritic, and the complementary re-combining function. Even if this might be easier for Latin-1, it is difficult to design something general enough. Characters may have a more complex structure than a mere base and single diacritic. I do not know what to suggest. -- François Pinard http://www.iro.umontreal.ca/~pinard From guido@python.org Thu May 30 02:03:20 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 21:03:20 -0400 Subject: [Python-Dev] True division in embedded Python In-Reply-To: Your message of "Wed, 29 May 2002 16:40:26 CDT." <15605.19146.687116.17742@12-248-41-177.client.attbi.com> References: <60FB8BB7F0EFC7409B75EEEC13E2019221518C@admin56.narex.com> <200205292026.g4TKQac22332@odiug.zope.com> <15605.19146.687116.17742@12-248-41-177.client.attbi.com> Message-ID: <200205300103.g4U13K806047@pcp742651pcs.reston01.va.comcast.net> > Sounds like there's a need for some documented access functions. At > the very least the docs should mention how programmers must > currently set these flags. I'll be happy to send in a patch for the > ext/embed docs and/or provide a patch to implement the necessary > calls. Yes and no. I'm not very keen on people turning this on for themselves. The only acceptable exception (and the reason why -Qnew exists) is in certain educational settings, where most code is written throw-away, examples using floats abound, and the teacher doesn't want to have to explain integer division. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu May 30 02:06:04 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 21:06:04 -0400 Subject: [Python-Dev] True division in embedded Python In-Reply-To: Your message of "Wed, 29 May 2002 16:00:41 MDT." <60FB8BB7F0EFC7409B75EEEC13E2019221518E@admin56.narex.com> References: <60FB8BB7F0EFC7409B75EEEC13E2019221518E@admin56.narex.com> Message-ID: <200205300106.g4U164c06065@pcp742651pcs.reston01.va.comcast.net> > > Would you consider upgrading to 2.2.2 when it comes out, and > > subsequent bugfix releases? The goal for those releases is > > to make the upgrade 100% painless. > > We would probably consider it. What it comes down to is whether the new > release is binary compatible (which I'm assuming it is), and if it > included a bugfix that we found prudent to incorporate. Yes, it would be binary compatible -- that's one of the characteristics of bugfix releases. And it would come with the promise of better stability. (Which may or may not be an issue for you depending on whether you're experiencing instability -- many people have no problems.) > Don't misunderstand me, it's not that we don't want to update -- the > developers here would probably be quite happy to work on the latest > and greatest -- updating just inherently takes time (money) and it's > not very exciting work. We'd just rather spend our time on something > that has a more direct revenue source :-) You may not be aware of it, but in the recent past this position has been made abundantly clear to me, and I agree. I just want people to be reasonable -- if you're not going to upgrade to 2.2.2, I wonder why we should put the effort in even producing it. ;-) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu May 30 02:07:49 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 21:07:49 -0400 Subject: [Python-Dev] deprecating string module? In-Reply-To: Your message of "29 May 2002 23:58:21 +0200." References: <15603.63458.608739.454153@beluga.mojam.com> <15604.47718.334686.681186@12-248-41-177.client.attbi.com> Message-ID: <200205300107.g4U17n606086@pcp742651pcs.reston01.va.comcast.net> > > Also, the data objects obviously have to be put somewhere. Before > > doing any of this, we need to decide where this stuff belongs. I > > doubt the technical effort involved will be too challenging. > > So we don't need to make a decision today, right? We can delay that > until those easy problems are solved. What about my proposal to get rid of the data objects and add a few isxxx() methods instead? I don't think that the data objects have any other function except for membership testing, and a method can be faster. The method approach will also make locale-awareness easier (since the method can use which is already locale-aware). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu May 30 02:12:15 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 21:12:15 -0400 Subject: [Python-Dev] Re: String module In-Reply-To: Your message of "30 May 2002 00:08:09 +0200." References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <200205291544.g4TFibS18433@odiug.zope.com> <00e001c2072f$7b8d2460$5d61accf@othello> <200205291642.g4TGgaf18754@odiug.zope.com> <013501c20736$3f7732c0$5d61accf@othello> <200205291813.g4TIDKK18906@odiug.zope.com> <3CF52706.10008@lemburg.com> <200205291932.g4TJW0g21814@odiug.zope.com> Message-ID: <200205300112.g4U1CFi06122@pcp742651pcs.reston01.va.comcast.net> > > I think all of the unicode methods should be added to 8-bit strings, > > even if they are just aliases for others (what's title case applied to > > Latin-1? I suppose same as upper case?). > > So those would be locale dependent? (the Unicode ones are not). Yes. isalpha() etc. for 8-bit strings are already locale dependent aren't they? BTW, how about deprecating strop while we're at it? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu May 30 02:16:27 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 21:16:27 -0400 Subject: [Python-Dev] release22-maint vs. release22-branch In-Reply-To: Your message of "30 May 2002 00:28:20 +0200." References: <20020529114311.C12720@glacier.arctrix.com> <200205291852.g4TIq0w19266@odiug.zope.com> Message-ID: <200205300116.g4U1GRc06212@pcp742651pcs.reston01.va.comcast.net> > It is custom in other projects to have only one branch for both, and > having the x.y.0 release just be a tag on the x.y maintenance branch. > This requires freezing the maintenance just before the release to > everybody but the release manager, but since the branch is not use for > anything but the immediate upcoming release, this is no problem. > > I suggest that this is done that way for Python 2.3 also. Good idea. This requires changes to PEPs 101 and 102. Would you mind fixing these or suggesting a fix to Barry? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu May 30 02:17:37 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 21:17:37 -0400 Subject: [Python-Dev] _PyImport_LoadDynamicModule questions In-Reply-To: Your message of "30 May 2002 00:35:11 +0200." References: <01cf01c2072a$b69cdeb0$6601a8c0@boostconsulting.com> Message-ID: <200205300117.g4U1HbN06229@pcp742651pcs.reston01.va.comcast.net> > As for dynamic linking on AIX: It would be really good if somebody > stepped forward who claims to understand the dynamic loader of AIX, > and rewrite Python to make this work more "nicely". Before starting > the rewrite, I'd be really curious to hear the full story of why it > currently is the way it is, and how this state could be improved. +1 > If you find that a "good" redesign requires a shared libpython, then > so be it - but I'm still quite fond of the "single executable" > appraoch, so preserving that would be even better. Why? I thought there was serious work underway to make libpython.so a reality on Linux? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu May 30 02:22:49 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 21:22:49 -0400 Subject: [Python-Dev] String module In-Reply-To: Your message of "Wed, 29 May 2002 18:51:02 EDT." <00b801c20763$4f3dc480$0fea7ad1@othello> References: <200205291642.g4TGgaf18754@odiug.zope.com> <013501c20736$3f7732c0$5d61accf@othello> <00b801c20763$4f3dc480$0fea7ad1@othello> Message-ID: <200205300122.g4U1Mnk06265@pcp742651pcs.reston01.va.comcast.net> > I like the read-only dictionary better than the boolean test > methods. It minimizes the effort in upgrading existing code of the > form: > > for c in string.lowercase: > do something > if c in string.lowercase: > do something > > I can global search/replace string.lowercase with str.lower in > dictionary form and everything will run fine (and faster too). I think this is not enough motivation. We have a whole slew of test methods already. Also the dict approach doesn't scale to Unicode (the dicts would have to be enormous) while the test method approach easily scales to Unicode (it's already implemented that way there). > Also, I like the mapping because provides a way to see the > membership. With the string form or the mapping form, it's easy to > find-out exactly what is defined as whitespace. If there is a C > coded boolean test, I have to filter the whole alphabet or look it > up in the docs. A very minor advantage indeed -- and again one that doesn't scale to Unicode. > If we can get some agreement that this is the way to go, I can work > with the OP on a revised patch so we get the string module silently > deprecated. Maybe you can do a patch for isxxx() methods instead? --Guido van Rossum (home page: http://www.python.org/~guido/) From pinard@iro.umontreal.ca Thu May 30 02:19:29 2002 From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard) Date: 29 May 2002 21:19:29 -0400 Subject: [Python-Dev] Re: PendingDeprecationWarning In-Reply-To: References: Message-ID: [Tim Peters] > Then Cray hired a bunch of young crybabies (like me), who-- with some > justification --pointed out that octal dumps were really hard to scan for > character data, given that Cray had moved to 8-bit characters. Seymour Cray once told that story. He hired many young engineers (hardware and software) and was instructing them about his own designs with passion, sharing his findings and choices with great enthusiasm. But Seymour noticed that they were listening with rather dull eyes, and no special pleasure. What was discovery for him was flatly part of their curriculum, or almost. He said something like: "Through their questions, they were criticising our works or raising suggestions and -- damned! -- often they were _right_!". Maybe he was speaking about you, Tim, who knows? :-) -- François Pinard http://www.iro.umontreal.ca/~pinard From guido@python.org Thu May 30 02:29:25 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 21:29:25 -0400 Subject: [Python-Dev] Re: String module In-Reply-To: Your message of "29 May 2002 20:03:57 EDT." References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <200205291544.g4TFibS18433@odiug.zope.com> <00e001c2072f$7b8d2460$5d61accf@othello> <200205291642.g4TGgaf18754@odiug.zope.com> <013501c20736$3f7732c0$5d61accf@othello> <200205291813.g4TIDKK18906@odiug.zope.com> Message-ID: <200205300129.g4U1TPY06349@pcp742651pcs.reston01.va.comcast.net> > This reminds me that I often miss, in the standard `ctype.h' and related, > a function that would un-combine a character into its base character and > its diacritic, and the complementary re-combining function. > > Even if this might be easier for Latin-1, it is difficult to design > something general enough. Characters may have a more complex structure > than a mere base and single diacritic. I do not know what to suggest. I bet the Unicode standard has a standard way to do this. Maybe we can implement that, and then project the same interface on 8-bit characters? Of course character encoding issues might get in the way if doesn't provide the data -- so you may be better off doing this in Unicode only. (We must never assume that 8-bit strings contain Latin-1.) --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Thu May 30 02:42:01 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 29 May 2002 21:42:01 -0400 Subject: [Python-Dev] True division in embedded Python In-Reply-To: <60FB8BB7F0EFC7409B75EEEC13E2019221518C@admin56.narex.com> Message-ID: [Bjorn Pettersen] > We're currently in the process of adapting Python as the common > scripting plug-in for all of our projects (and it's working great :-). > We're standardizing on 2.2.1, and unless there is a pressing business > need, we'll probably stay with that version for the next 18-24 months. > > My question is if it's intended for users to start using true division > _now_? The reason I'm inclined to do so is that we're going to have > snippets of Python code everywhere, including our libraries, disk files, > and databases -- it would be a major undertaking to find and fix this > later... Nevertheless, I recommend you not try to enable true division now. Part of the joy of Python is sharing the many libraries and extension modules people freely provide, and so far nothing except the core distribution is tested with true division enabled -- and even in the core, true division is tested sporadically and lightly. You don't want to be a pioneer here, unless advancing true division is your business . That said, in 2.2.x, you can start using // *now* when you intend flooring integer division. That's already implemented. 5/3 will also produce 1 for now, but don't do it -- do 5//3 instead. Also read the PEP: http://www.python.org/peps/pep-0238.html Guido has implemented some quite elaborate mechanisms to help with the transition, and if you start using i//j now you *should* have a very easy time when Python 3 changes the rules (indeed, you shouldn't need any division changes later, provided you use // now). The simplest of the mechanisms is -Qwarn: C:\Python22>type div.py print 4/3 C:\Python22>python div.py 1 C:\Python22>python -Qwarn div.py div.py:1: DeprecationWarning: classic int division print 4/3 1 C:\Python22> So, if you like, Python will already warn you about division cases that will act differently someday. You can examine them now to determine what you really intended, and use // now when you intend a truncated result: C:\Python22>type div2.py print 4//3 C:\Python22>python -Qwarn div2.py # no warning produced 1 C:\Python22> Likewise when you intend a floating-point result, you can do something now to ensure that (spelling it float(i)/j is one easy way). From greg@cosc.canterbury.ac.nz Thu May 30 03:26:43 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Thu, 30 May 2002 14:26:43 +1200 (NZST) Subject: [Python-Dev] Bug in PyNumber_InPlacePower implementation? In-Reply-To: <200205291532.g4TFWpX18293@odiug.zope.com> Message-ID: <200205300226.OAA06686@s454.cosc.canterbury.ac.nz> Guido: > Me: > > a **= b > > > > has the potential to in-place-modify b instead of a! > > Not quite -- the arguments won't be reversed, but it may call B's > inplace power function with an A instance as the first argument. Hmmm, I see. I guess it's just a matter of being aware that this can happen and doing the necessary type tests. > This can return NotImplemented if it doesn't know what to do > in that case. Although if you're implementing inplace-power you're probably also implementing non-inplace-power, in which case the right thing is probably to call that instead. I don't think that will happen automatically if you return NotImplemented -- or will it? > It's a pretty esoteric case ... The correct code would > be pretty hairy I think (the non-inplace ternary is hairy enough). I think the existing ternary_op routine could be converted fairly easily into one that could be used for both. Just pass in two slot arguments, and use one for the first arg and the other one for the second and third args. For non-inplace ops, call it with the non-inplace slot for both. It's not a big deal, I suppose, but it would be nice if it could be made consistent with the other in-place ops. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From python@rcn.com Thu May 30 03:25:50 2002 From: python@rcn.com (Raymond Hettinger) Date: Wed, 29 May 2002 22:25:50 -0400 Subject: [Python-Dev] String module References: <200205291642.g4TGgaf18754@odiug.zope.com> <013501c20736$3f7732c0$5d61accf@othello> <00b801c20763$4f3dc480$0fea7ad1@othello> <200205300122.g4U1Mnk06265@pcp742651pcs.reston01.va.comcast.net> Message-ID: <002701c20781$51ac09c0$f3e97ad1@othello> From: "Guido van Rossum" > > I like the read-only dictionary better than the boolean test > > methods. It minimizes the effort in upgrading existing code of the > > form: > > > > for c in string.lowercase: > > do something > > if c in string.lowercase: > > do something > > > Maybe you can do a patch for isxxx() methods instead? Will do. Raymond Hettinger From BPettersen@NAREX.com Thu May 30 03:42:52 2002 From: BPettersen@NAREX.com (Bjorn Pettersen) Date: Wed, 29 May 2002 20:42:52 -0600 Subject: [Python-Dev] True division in embedded Python Message-ID: <60FB8BB7F0EFC7409B75EEEC13E20192F1A03D@admin56.narex.com> > From: Tim Peters [mailto:tim.one@comcast.net]=20 >=20 > [Bjorn Pettersen] [snip: me asking whether I should turn on true division] > Nevertheless, I recommend you not try to enable true division=20 > now. Part of the joy of Python is sharing the many libraries=20 > and extension modules people freely provide, and so far=20 > nothing except the core distribution is tested with true=20 > division enabled -- and even in the core, true division is=20 > tested sporadically and lightly. You don't want to be a=20 > pioneer here, unless advancing true division is your business . Nope, we just want the advantage of the best language around while still using our 900KLoc of C++ libraries . > That said, in 2.2.x, you can start using // *now* when you=20 > intend flooring integer division. That's already=20 > implemented. 5/3 will also produce 1 for now, but don't do=20 > it -- do 5//3 instead. >=20 > Also read the PEP: >=20 > http://www.python.org/peps/pep-0238.html > > Guido has implemented some quite elaborate mechanisms to=20 > help with the transition, and if you start using i//j now=20 > you *should* have a very easy time when Python 3 changes=20 > the rules (indeed, you shouldn't need any division changes=20 > later, provided you use // now). The simplest of the=20 > mechanisms is -Qwarn: And if I'm reading the source correctly all I would have to do when embedding Python would be to set Py_DivisionWarningFlag =3D 1? Thanks for the detailed input. Sincerely, Bjorn Pettersen NAREX Inc. 303.526.4000 ext. 312 303.526.5130 fax www.narex.com From guido@python.org Thu May 30 04:29:16 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 29 May 2002 23:29:16 -0400 Subject: [Python-Dev] Bug in PyNumber_InPlacePower implementation? In-Reply-To: Your message of "Thu, 30 May 2002 14:26:43 +1200." <200205300226.OAA06686@s454.cosc.canterbury.ac.nz> References: <200205300226.OAA06686@s454.cosc.canterbury.ac.nz> Message-ID: <200205300329.g4U3TGB06664@pcp742651pcs.reston01.va.comcast.net> > Guido: > > > GregE: > > > a **= b > > > > > > has the potential to in-place-modify b instead of a! > > > > Not quite -- the arguments won't be reversed, but it may call B's > > inplace power function with an A instance as the first argument. > > Hmmm, I see. I guess it's just a matter of being aware > that this can happen and doing the necessary type tests. > > > This can return NotImplemented if it doesn't know what to do > > in that case. > > Although if you're implementing inplace-power you're > probably also implementing non-inplace-power, in which case > the right thing is probably to call that instead. I > don't think that will happen automatically if you > return NotImplemented -- or will it? For binary ops it will (see binary_iop()). But because the ternary code doesn't have the inplace support, it won't. > > It's a pretty esoteric case ... The correct code would > > be pretty hairy I think (the non-inplace ternary is hairy enough). > > I think the existing ternary_op routine could be converted > fairly easily into one that could be used for both. Just pass > in two slot arguments, and use one for the first arg and > the other one for the second and third args. For non-inplace > ops, call it with the non-inplace slot for both. > > It's not a big deal, I suppose, but it would be nice if > it could be made consistent with the other in-place ops. Do you care enough about this to supply a patch? I would apply it, but I don't care enough to write it. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From greg@cosc.canterbury.ac.nz Thu May 30 04:48:29 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Thu, 30 May 2002 15:48:29 +1200 (NZST) Subject: [Python-Dev] Bug in PyNumber_InPlacePower implementation? In-Reply-To: <200205300329.g4U3TGB06664@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200205300348.PAA06695@s454.cosc.canterbury.ac.nz> Guido: > Do you care enough about this to supply a patch? I would apply it, > but I don't care enough to write it. :-) I'll see what I can do. Thanks for the help, Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tim.one@comcast.net Thu May 30 05:02:37 2002 From: tim.one@comcast.net (Tim Peters) Date: Thu, 30 May 2002 00:02:37 -0400 Subject: [Python-Dev] True division in embedded Python In-Reply-To: <60FB8BB7F0EFC7409B75EEEC13E20192F1A03D@admin56.narex.com> Message-ID: [Bjorn Pettersen] > ... > Nope, we just want the advantage of the best language around while still > using our 900KLoc of C++ libraries . Then rather than play with true division now, you'll have a lot more fun rewriting that in 9 lines of Python; C++/Python ratios of a million are common for experienced Python programmers, but as a beginner don't feel bad if you only get a factor of 100,000 savings. By the way, you should become a PSF sponsor too, and especially if you believed that . > ... > And if I'm reading the source correctly all I would have to do when > embedding Python would be to set Py_DivisionWarningFlag = 1? If you're trying to get the effect of passing -Qwarn to a standalone Python, yes, that's all you need to do. And #include'ing Python.h is all you need to do to get at all names in Python's C API (Py_DivisionWarningFlag among them). Have fun! From martin@v.loewis.de Thu May 30 07:43:48 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 30 May 2002 08:43:48 +0200 Subject: [Python-Dev] Re: String module In-Reply-To: <200205300129.g4U1TPY06349@pcp742651pcs.reston01.va.comcast.net> References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <200205291544.g4TFibS18433@odiug.zope.com> <00e001c2072f$7b8d2460$5d61accf@othello> <200205291642.g4TGgaf18754@odiug.zope.com> <013501c20736$3f7732c0$5d61accf@othello> <200205291813.g4TIDKK18906@odiug.zope.com> <200205300129.g4U1TPY06349@pcp742651pcs.reston01.va.comcast.net> Message-ID: Guido van Rossum writes: > > This reminds me that I often miss, in the standard `ctype.h' and related, > > a function that would un-combine a character into its base character and > > its diacritic, and the complementary re-combining function. [...] > I bet the Unicode standard has a standard way to do this. This is called 'unicode normalization forms'. Each "pre-combined" character can also be represented as a base character, and a "combining diacritic". There are symmetric normalization forms: NFC favours pre-combined characters, NFD favours combining characters. There is also a "compatibility decomposition" (K), where e.g. ANGSTROM SIGN decomposes to LATIN CAPITAL LETTER A WITH RING ABOVE. > Maybe we can implement that, and then project the same interface on > 8-bit characters? Not really. Needing to know the character set is one issue; the other issue is that the stand-alone diacritic characters in ASCII are *not* combining. We could certainly provide a mapping between the Unicode combining diacritics and the stand-alone diacritics, say as a codec, but that would be quite special-purpose. Providing a good normalization library is necessary, though, since many other algorithms (both from W3C and IETF) require Unicode normalization as part of the processing (usually to NFKC). Regards, Martin From martin@v.loewis.de Thu May 30 07:49:25 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 30 May 2002 08:49:25 +0200 Subject: [Python-Dev] Re: String module In-Reply-To: <200205300112.g4U1CFi06122@pcp742651pcs.reston01.va.comcast.net> References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <200205291544.g4TFibS18433@odiug.zope.com> <00e001c2072f$7b8d2460$5d61accf@othello> <200205291642.g4TGgaf18754@odiug.zope.com> <013501c20736$3f7732c0$5d61accf@othello> <200205291813.g4TIDKK18906@odiug.zope.com> <3CF52706.10008@lemburg.com> <200205291932.g4TJW0g21814@odiug.zope.com> <200205300112.g4U1CFi06122@pcp742651pcs.reston01.va.comcast.net> Message-ID: Guido van Rossum writes: > > > I think all of the unicode methods should be added to 8-bit strings, > > > even if they are just aliases for others (what's title case applied to > > > Latin-1? I suppose same as upper case?). > > > > So those would be locale dependent? (the Unicode ones are not). > > Yes. isalpha() etc. for 8-bit strings are already locale dependent > aren't they? Yes, certainly. I'm just confirming, since this *is* a notable difference to Unicode strings. > BTW, how about deprecating strop while we're at it? That seems acceptable to me, once string.py stops importing it. I wonder how much time after deprecating a builtin or extension module we should stop building it automatically in setup.py. Regards, Martin From martin@v.loewis.de Thu May 30 07:59:15 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 30 May 2002 08:59:15 +0200 Subject: [Python-Dev] deprecating string module? In-Reply-To: <200205300107.g4U17n606086@pcp742651pcs.reston01.va.comcast.net> References: <15603.63458.608739.454153@beluga.mojam.com> <15604.47718.334686.681186@12-248-41-177.client.attbi.com> <200205300107.g4U17n606086@pcp742651pcs.reston01.va.comcast.net> Message-ID: Guido van Rossum writes: > > > Also, the data objects obviously have to be put somewhere. Before > > > doing any of this, we need to decide where this stuff belongs. I > > > doubt the technical effort involved will be too challenging. > > > > So we don't need to make a decision today, right? We can delay that > > until those easy problems are solved. > > What about my proposal to get rid of the data objects and add a few > isxxx() methods instead? That's a good thing to do, I agree. > I don't think that the data objects have any other function except > for membership testing, and a method can be faster. That is not true. Asking google for "string.letters" brings, as the first hit, non_letters = string.translate(norm, norm, string.letters) I also found random.choice(string.lowercase[:26] + string.uppercase[:26] + string.digits) This is, of course, nonsense, since it tries to "unlocalize" string.lowercase, where a string literal would have been better. I'm sure people have found other uses for these constants. Regards, Martin From martin@v.loewis.de Thu May 30 08:06:55 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 30 May 2002 09:06:55 +0200 Subject: [Python-Dev] _PyImport_LoadDynamicModule questions In-Reply-To: <200205300117.g4U1HbN06229@pcp742651pcs.reston01.va.comcast.net> References: <01cf01c2072a$b69cdeb0$6601a8c0@boostconsulting.com> <200205300117.g4U1HbN06229@pcp742651pcs.reston01.va.comcast.net> Message-ID: Guido van Rossum writes: > > If you find that a "good" redesign requires a shared libpython, then > > so be it - but I'm still quite fond of the "single executable" > > appraoch, so preserving that would be even better. > > Why? I thought there was serious work underway to make libpython.so a > reality on Linux? It already is, with --enable-shared. However, I still think that people creating --enable-shared installations are misguided: You gain nothing (IMO), and you lose a number of benefits: - starting python will always require the dynamic linker to search for the library, after the system already searched for the executable. This will cause a number of extra stat calls. Together with the need to produce PIC code, this will slow down Python. - If Python is installed into a non-standard location (such as /usr/local on Solaris), you will need additional trickery to find the shared library. Even though the default installation achieves this trickery with a -R option, the resulting binary is not relocatable anymore to a different directory (or, the python binary, but libpython2.3.so isn't) Regards, Martin From martin@v.loewis.de Thu May 30 08:07:16 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 30 May 2002 09:07:16 +0200 Subject: [Python-Dev] release22-maint vs. release22-branch In-Reply-To: <200205300116.g4U1GRc06212@pcp742651pcs.reston01.va.comcast.net> References: <20020529114311.C12720@glacier.arctrix.com> <200205291852.g4TIq0w19266@odiug.zope.com> <200205300116.g4U1GRc06212@pcp742651pcs.reston01.va.comcast.net> Message-ID: Guido van Rossum writes: > Good idea. This requires changes to PEPs 101 and 102. Would you mind > fixing these or suggesting a fix to Barry? Will do. Martin From mwh@python.net Thu May 30 10:03:07 2002 From: mwh@python.net (Michael Hudson) Date: 30 May 2002 10:03:07 +0100 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test test_signal.py,1.9,1.10 In-Reply-To: Andrew MacIntyre's message of "Wed, 29 May 2002 22:10:27 +1100 (edt)" References: Message-ID: <2mit56ros4.fsf@starship.python.net> Andrew MacIntyre writes: > On Mon, 27 May 2002 mwh@users.sourceforge.net wrote: > > > Update of /cvsroot/python/python/dist/src/Lib/test > > In directory usw-pr-cvs1:/tmp/cvs-serv24806/Lib/test > > > > Modified Files: > > test_signal.py > > Log Message: > > This is patch > > > > [ 559250 ] more POSIX signal stuff > > > > Adds support (and docs and tests and autoconfery) for posix signal > > mask handling -- sigpending, sigprocmask and sigsuspend. > > Since this batch of checkins, my FreeBSD autobuilder is failing Oh, um, good. > test_socket: > test test_signal failed -- HUP not pending > > Running the test with -v yields: > test_signal > + sleep 2 > starting pause() loop... > call pause()... > + kill -5 33123 > + sleep 2 > handlerA (5, ) > pause() returned > call pause()... > + kill -2 33123 > + sleep 2 > handlerB (2, ) > HandlerBCalled exception caught > call pause()... > + kill -3 33123 > KeyboardInterrupt (assume the alarm() went off) > blocking SIGHUP > sending SIGHUP > test test_signal failed -- HUP not pending > 1 test failed: > test_signal > > > I found the following (apparently) signal related items in the autoconf > generated pyconfig.h: > /* Define to 1 if you have the header file. */ > #define HAVE_SIGNAL_H 1 > > /* Define to 1 if you have the `sigprocmask' function. */ > #define HAVE_SIGPROCMASK 1 The above tests wouldn't have run without this symbol being defined (it's the new one). > /* Define to 1 if you have the `sigrelse' function. */ > /* #undef HAVE_SIGRELSE */ > > {...} > > /* Define as the return type of signal handlers (`int' or `void'). */ > #define RETSIGTYPE void > > > Any hints about where to start looking? > (I'm not au fait with signal() magic :-() Let me try playing on the sf compile farm first. Cheers, M. From fredrik@pythonware.com Thu May 30 10:23:35 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Thu, 30 May 2002 11:23:35 +0200 Subject: [Python-Dev] Re: String module References: <200205291358.g4TDw5v16789@odiug.zope.com><006b01c20723$2a58aee0$5d61accf@othello><200205291523.g4TFNiA18104@odiug.zope.com><3CF4F6C8.E57807FF@metaslash.com><200205291544.g4TFibS18433@odiug.zope.com><00e001c2072f$7b8d2460$5d61accf@othello><200205291642.g4TGgaf18754@odiug.zope.com><013501c20736$3f7732c0$5d61accf@othello><200205291813.g4TIDKK18906@odiug.zope.com> Message-ID: <017301c207bc$2a36b670$0900a8c0@spiff> Fran=E7ois Pinard wrote: > This reminds me that I often miss, in the standard `ctype.h' and = related, > a function that would un-combine a character into its base character = and > its diacritic, and the complementary re-combining function. import unicodedata def uncombine(char): chars =3D unicodedata.decomposition(unichr(ord(char))).split() if not chars: return [char] return [unichr(int(x, 16)) for x in chars if x[0] !=3D "<"] for char in "Fran=E7ois": print uncombine(char) ['F'] ['r'] ['a'] ['n'] [u'c', u'\u0327'] ['o'] ['i'] ['s'] (to go the other way, store all uncombinations longer than one character in a dictionary) From oren-py-d@hishome.net Thu May 30 12:05:48 2002 From: oren-py-d@hishome.net (Oren Tirosh) Date: Thu, 30 May 2002 14:05:48 +0300 Subject: [Python-Dev] PendingDeprecationWarning In-Reply-To: <00e001c2072f$7b8d2460$5d61accf@othello>; from python@rcn.com on Wed, May 29, 2002 at 12:40:02PM -0400 References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <200205291544.g4TFibS18433@odiug.zope.com> <00e001c2072f$7b8d2460$5d61accf@othello> Message-ID: <20020530140548.A23304@hishome.net> On Wed, May 29, 2002 at 12:40:02PM -0400, Raymond Hettinger wrote: > From: "Guido van Rossum" > > Thanks! I agree with the name change to PendingDeprecationWarning. > > Check it in before I change my mind! :-) > > Awesome! > Now, down to business. > What shall we Silently Deprecate? > > string module > types module (after providing substitutes) It looks like many of the names in the types module already have substitutes in __builtins__: >>> import types >>> for name, t in types.__dict__.items(): ... if type(t) is type and hasattr(__builtins__, t.__name__): ... print 'types.%s -> %s' % (name, t.__name__) ... types.IntType -> int types.TypeType -> type types.StringType -> str types.FloatType -> float types.ObjectType -> object types.DictionaryType -> dict types.FileType -> file types.DictType -> dict types.ListType -> list types.TupleType -> tuple types.LongType -> long types.BufferType -> buffer types.UnicodeType -> unicode types.ComplexType -> complex types.SliceType -> slice types.XRangeType -> xrange Some of these are new in 2.2 (like object, dict and file). Some of them used to be functions before Python 2.2 (like str, int and list). Three of them are still builtin functions in Python 2.2: xrange, buffer and slice. Perhaps they should also be converted to types for consistency. Some more factory functions that could be unified with the type of the objects they create module can be found in the new module. They, too can be used as substitutes for names in the types module. >>> import new >>> for name, t in types.__dict__.items(): ... if type(t) is type and hasattr(new, t.__name__): ... print 'types.%s -> new.%s -> %s' % (name, t.__name__, t.__name__) types.CodeType -> new.code -> code types.ModuleType -> new.module -> module types.LambdaType -> new.function -> function types.InstanceType -> new.instance -> instance types.FunctionType -> new.function -> function There are two that almost made it to this list but the name of the factory function in module new is not exactly the same as the type's __name__: types.MethodType -> new.instancemethod -> instancemethod ('instance method') types.ClassType -> new.classobj -> classobj ('class') For instancemethod it's easy to remove the space from the type's __name__. The word class is a reserved word. The type's __name__ could be changed to 'classobj' to match the factory function's name. Some other alternatives I can think of are 'class_', 'ClassType' or 'classtype'. Instances of the remaining types can't be instanciated from Python code. Most of them can be safely identified by their __name__: types.BuiltinMethodType -> builtin_function_or_method types.BuiltinFunctionType -> builtin_function_or_method types.TracebackType -> traceback types.GeneratorType -> generator types.FrameType -> frame types.NoneType -> NoneType The last two remaining troublemakers: types.DictProxyType -> dict_proxy ('dict-proxy') 'dict-proxy' is not a valid Python identifier. types.EllipsisType -> EllipsisType ('ellipsis') The name 'ellipsis' is easily confused with 'Ellipsis'. The name EllipsisType is consistent with NoneType. Both are the type of a specific singleton object. Should all these names be added to __builtins__? Many of them are not very useful in everyday programming. Perhaps the less important ones can be put away in some module. Now how should that module be named? Ummm... maybe 'types'? :-) Oren From mwh@python.net Thu May 30 12:06:25 2002 From: mwh@python.net (Michael Hudson) Date: 30 May 2002 12:06:25 +0100 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test test_signal.py,1.9,1.10 In-Reply-To: Michael Hudson's message of "30 May 2002 10:03:07 +0100" References: <2mit56ros4.fsf@starship.python.net> Message-ID: <2msn49hp3i.fsf@starship.python.net> Michael Hudson writes: > Andrew MacIntyre writes: > > > [test_signal fails on FreeBSD] > It doesn't fail if you disable threading. Oh /good/. Does anyone want to explain BSD threads and their interaction with signals to me? Cheers, M. -- I'll write on my monitor fifty times 'I must not post self-indulgent wibble nobody is interested in to ucam.chat just because I'm bored and I can't find the bug I'm supposed to fix'. -- Steve Kitson, ucam.chat From neal@metaslash.com Thu May 30 12:53:32 2002 From: neal@metaslash.com (Neal Norwitz) Date: Thu, 30 May 2002 07:53:32 -0400 Subject: [Python-Dev] Re: String module References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <200205291544.g4TFibS18433@odiug.zope.com> <00e001c2072f$7b8d2460$5d61accf@othello> <200205291642.g4TGgaf18754@odiug.zope.com> <013501c20736$3f7732c0$5d61accf@othello> <200205291813.g4TIDKK18906@odiug.zope.com> <3CF52706.10008@lemburg.com> <200205291932.g4TJW0g21814@odiug.zope.com> <200205300112.g4U1CFi06122@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CF612BC.C5D34633@metaslash.com> Guido van Rossum wrote: > BTW, how about deprecating strop while we're at it? Right now in string.py, there is: from strop import maketrans, lowercase, uppercase, whitespace letters = lowercase + uppercase However, lowercase, uppercase, and letters are already defined in string.py. It seems the only function being used from strop is maketrans. Is this correct? Should the others be removed from string.py? Neal From mwh@python.net Thu May 30 12:59:39 2002 From: mwh@python.net (Michael Hudson) Date: 30 May 2002 12:59:39 +0100 Subject: [Python-Dev] Re: String module In-Reply-To: Neal Norwitz's message of "Thu, 30 May 2002 07:53:32 -0400" References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <200205291544.g4TFibS18433@odiug.zope.com> <00e001c2072f$7b8d2460$5d61accf@othello> <200205291642.g4TGgaf18754@odiug.zope.com> <013501c20736$3f7732c0$5d61accf@othello> <200205291813.g4TIDKK18906@odiug.zope.com> <3CF52706.10008@lemburg.com> <200205291932.g4TJW0g21814@odiug.zope.com> <200205300112.g4U1CFi06122@pcp742651pcs.reston01.va.comcast.net> <3CF612BC.C5D34633@metaslash.com> Message-ID: <2mptzdhmms.fsf@starship.python.net> Neal Norwitz writes: > Guido van Rossum wrote: > > > BTW, how about deprecating strop while we're at it? > > Right now in string.py, there is: > > from strop import maketrans, lowercase, uppercase, whitespace > letters = lowercase + uppercase > > However, lowercase, uppercase, and letters are already defined > in string.py. It seems the only function being used from strop > is maketrans. Is this correct? Should the others be removed > from string.py? The strop definitions reflect the locale better than the ones in string.py. Cheers, M. -- And then the character-only displays went away (leading to increasingly silly graphical effects and finally to ads on web pages). -- John W. Baxter, comp.lang.python From David Abrahams" <200205300117.g4U1HbN06229@pcp742651pcs.reston01.va.comcast.net> Message-ID: <049601c207d1$7f818870$6601a8c0@boostconsulting.com> From: "Martin v. Loewis" > Guido van Rossum writes: > > > > If you find that a "good" redesign requires a shared libpython, then > > > so be it - but I'm still quite fond of the "single executable" > > > appraoch, so preserving that would be even better. > > > > Why? I thought there was serious work underway to make libpython.so a > > reality on Linux? > > It already is, with --enable-shared. ...interesting... > However, I still think that people creating --enable-shared > installations are misguided: You gain nothing On AIX, you'd gain at least a little something: there would be no need to "manually" patch up the symbol references the way _PyImport_LoadDynamicModule is currently doing, and shared libraries that were not themselves extension modules wouldn't have to duplicate that functionality (as I understand you are saying they should in lieu of hijacking the Python call). Whether that makes the separate shared library worth it or not, I can't say. Probably not in light of the other benefits you list here. The AIX problem with resolving symbols from an executable is strange enough that it can be a significant obstacle to porting systems built around Python. The only way we figured it out was by crawling through the Python source to figure out how the symbols were getting resolved. It would be really nice if there were a "documented and stable" version of the core functionality which is doing the symbol patching. Since Python is an executable which expects to share its symbols with libraries, it's not unreasonable to think that Python ought to provide the means to do it. Coming up with a code patch wouldn't be too hard, I think, but where would the documentation go? -Dave From guido@python.org Thu May 30 13:10:46 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 08:10:46 -0400 Subject: [Python-Dev] Re: String module In-Reply-To: Your message of "30 May 2002 08:49:25 +0200." References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <200205291544.g4TFibS18433@odiug.zope.com> <00e001c2072f$7b8d2460$5d61accf@othello> <200205291642.g4TGgaf18754@odiug.zope.com> <013501c20736$3f7732c0$5d61accf@othello> <200205291813.g4TIDKK18906@odiug.zope.com> <3CF52706.10008@lemburg.com> <200205291932.g4TJW0g21814@odiug.zope.com> <200205300112.g4U1CFi06122@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200205301210.g4UCAk007433@pcp742651pcs.reston01.va.comcast.net> > > > > I think all of the unicode methods should be added to 8-bit strings, > > > > even if they are just aliases for others (what's title case applied to > > > > Latin-1? I suppose same as upper case?). > > > > > > So those would be locale dependent? (the Unicode ones are not). > > > > Yes. isalpha() etc. for 8-bit strings are already locale dependent > > aren't they? > > Yes, certainly. I'm just confirming, since this *is* a notable > difference to Unicode strings. Yes, but that's because locale-dependency is not a Unicode concept! > > BTW, how about deprecating strop while we're at it? > > That seems acceptable to me, once string.py stops importing it. > > I wonder how much time after deprecating a builtin or extension module > we should stop building it automatically in setup.py. Me too. There are probably still places like Greg Stein's original import hack that import strop because they haven't been converted to string methods. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu May 30 13:15:40 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 08:15:40 -0400 Subject: [Python-Dev] deprecating string module? In-Reply-To: Your message of "30 May 2002 08:59:15 +0200." References: <15603.63458.608739.454153@beluga.mojam.com> <15604.47718.334686.681186@12-248-41-177.client.attbi.com> <200205300107.g4U17n606086@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200205301215.g4UCFeU07478@pcp742651pcs.reston01.va.comcast.net> > > I don't think that the data objects have any other function except > > for membership testing, and a method can be faster. > > That is not true. Asking google for "string.letters" brings, as the > first hit, > > non_letters = string.translate(norm, norm, string.letters) > > I also found > > random.choice(string.lowercase[:26] + string.uppercase[:26] + string.digits) > > This is, of course, nonsense, since it tries to "unlocalize" > string.lowercase, where a string literal would have been > better. > > I'm sure people have found other uses for these constants. That just shows we have to be very slow before we remove the string module. I think "not until 3.0" is slow enough. These "use cases" don't convince me that there's a legitimate use case for string.letters etc. that the methods don't cover. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu May 30 13:18:58 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 08:18:58 -0400 Subject: [Python-Dev] _PyImport_LoadDynamicModule questions In-Reply-To: Your message of "30 May 2002 09:06:55 +0200." References: <01cf01c2072a$b69cdeb0$6601a8c0@boostconsulting.com> <200205300117.g4U1HbN06229@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200205301218.g4UCIw107505@pcp742651pcs.reston01.va.comcast.net> > It already is, with --enable-shared. > > However, I still think that people creating --enable-shared > installations are misguided: You gain nothing (IMO), and you lose a > number of benefits: Do you understand why people have always been asking for this? Are they all misguided? It really is a FAQ (3.30). Why? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu May 30 13:28:09 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 08:28:09 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test test_signal.py,1.9,1.10 In-Reply-To: Your message of "30 May 2002 12:06:25 BST." <2msn49hp3i.fsf@starship.python.net> References: <2mit56ros4.fsf@starship.python.net> <2msn49hp3i.fsf@starship.python.net> Message-ID: <200205301228.g4UCS9S07601@pcp742651pcs.reston01.va.comcast.net> > Does anyone want to explain BSD threads and their interaction with > signals to me? Yes. Threads and signals don't mix. Period. (Only half a smiley. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu May 30 13:35:02 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 08:35:02 -0400 Subject: [Python-Dev] PendingDeprecationWarning In-Reply-To: Your message of "Thu, 30 May 2002 14:05:48 +0300." <20020530140548.A23304@hishome.net> References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <200205291544.g4TFibS18433@odiug.zope.com> <00e001c2072f$7b8d2460$5d61accf@othello> <20020530140548.A23304@hishome.net> Message-ID: <200205301235.g4UCZ2207631@pcp742651pcs.reston01.va.comcast.net> > It looks like many of the names in the types module already have substitutes > in __builtins__: This was all covered a few days ago. :-) > Some of these are new in 2.2 (like object, dict and file). Some of > them used to be functions before Python 2.2 (like str, int and > list). Three of them are still builtin functions in Python 2.2: > xrange, buffer and slice. Perhaps they should also be converted to > types for consistency. You can help by contributing patches for these three. (Let us know if you plan to do this, so others can relax.) > Some more factory functions that could be unified with the type of the > objects they create module can be found in the new module. They, too can > be used as substitutes for names in the types module. > > >>> import new > >>> for name, t in types.__dict__.items(): > ... if type(t) is type and hasattr(new, t.__name__): > ... print 'types.%s -> new.%s -> %s' % (name, t.__name__, t.__name__) > types.CodeType -> new.code -> code > types.ModuleType -> new.module -> module > types.LambdaType -> new.function -> function > types.InstanceType -> new.instance -> instance > types.FunctionType -> new.function -> function Except that the new module has the wrong name. But making all these types proper factories would be a first step that's useful anyway. Patch please? > There are two that almost made it to this list but the name of the > factory function in module new is not exactly the same as the type's > __name__: > > types.MethodType -> new.instancemethod -> instancemethod ('instance method') So change the type's __name__. > types.ClassType -> new.classobj -> classobj ('class') Hopefully the classic class will disappear at the same time as types.py is removed (in 3.0). > For instancemethod it's easy to remove the space from the type's __name__. > The word class is a reserved word. The type's __name__ could be changed > to 'classobj' to match the factory function's name. Some other alternatives > I can think of are 'class_', 'ClassType' or 'classtype'. Or 'classic_class'. > Now how should that module be named? Ummm... maybe 'types'? :-) This may be the best solution after all. --Guido van Rossum (home page: http://www.python.org/~guido/) From David Abrahams" <15604.47718.334686.681186@12-248-41-177.client.attbi.com> <200205300107.g4U17n606086@pcp742651pcs.reston01.va.comcast.net> <200205301215.g4UCFeU07478@pcp742651pcs.reston01.va.comcast.net> Message-ID: <04eb01c207d5$b3eda680$6601a8c0@boostconsulting.com> From: "Guido van Rossum" > These "use cases" > don't convince me that there's a legitimate use case for > string.letters etc. that the methods don't cover. This is funny. In the C++ community there's a nearly unanimous consensus that way too much of the functionality of the standard strings is expressed as member functions. -Dave From guido@python.org Thu May 30 13:49:31 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 08:49:31 -0400 Subject: [Python-Dev] Re: String module In-Reply-To: Your message of "Thu, 30 May 2002 07:53:32 EDT." <3CF612BC.C5D34633@metaslash.com> References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <200205291544.g4TFibS18433@odiug.zope.com> <00e001c2072f$7b8d2460$5d61accf@othello> <200205291642.g4TGgaf18754@odiug.zope.com> <013501c20736$3f7732c0$5d61accf@othello> <200205291813.g4TIDKK18906@odiug.zope.com> <3CF52706.10008@lemburg.com> <200205291932.g4TJW0g21814@odiug.zope.com> <200205300112.g4U1CFi06122@pcp742651pcs.reston01.va.comcast.net> <3CF612BC.C5D34633@metaslash.com> Message-ID: <200205301249.g4UCnVD07714@pcp742651pcs.reston01.va.comcast.net> > Right now in string.py, there is: > > from strop import maketrans, lowercase, uppercase, whitespace > letters = lowercase + uppercase > > However, lowercase, uppercase, and letters are already defined > in string.py. It seems the only function being used from strop > is maketrans. Is this correct? Should the others be removed > from string.py? strop.letters etc. are fixed when locale.setconv(LC_CTYPE, ...) is used. string.letters etc. are *also* fixed, but if string hasn't been imported yet when the locale is set, string gets the properly localized versions from strop (which does the right thing upon initialization to reflect the locale). This mechanism would have to be copied into string.py somehow. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu May 30 13:51:57 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 08:51:57 -0400 Subject: [Python-Dev] _PyImport_LoadDynamicModule questions In-Reply-To: Your message of "Thu, 30 May 2002 07:58:18 EDT." <049601c207d1$7f818870$6601a8c0@boostconsulting.com> References: <01cf01c2072a$b69cdeb0$6601a8c0@boostconsulting.com><200205300117.g4U1HbN06229@pcp742651pcs.reston01.va.comcast.net> <049601c207d1$7f818870$6601a8c0@boostconsulting.com> Message-ID: <200205301251.g4UCpv407739@pcp742651pcs.reston01.va.comcast.net> > Coming up with a code patch wouldn't be too hard, I think, but where > would the documentation go? In the C/API docs. There are enough precendents of platform-specific documentation in the core docs. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu May 30 13:55:26 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 08:55:26 -0400 Subject: [Python-Dev] deprecating string module? In-Reply-To: Your message of "Thu, 30 May 2002 08:25:28 EDT." <04eb01c207d5$b3eda680$6601a8c0@boostconsulting.com> References: <15603.63458.608739.454153@beluga.mojam.com> <15604.47718.334686.681186@12-248-41-177.client.attbi.com> <200205300107.g4U17n606086@pcp742651pcs.reston01.va.comcast.net> <200205301215.g4UCFeU07478@pcp742651pcs.reston01.va.comcast.net> <04eb01c207d5$b3eda680$6601a8c0@boostconsulting.com> Message-ID: <200205301255.g4UCtQX07833@pcp742651pcs.reston01.va.comcast.net> > > These "use cases" > > don't convince me that there's a legitimate use case for > > string.letters etc. that the methods don't cover. > > This is funny. In the C++ community there's a nearly unanimous > consensus that way too much of the functionality of the standard > strings is expressed as member functions. Interesting. Python used to have the same attitude, hence the string module -- but the existence of multiple string types made methods more attractive. What's the alternative proposed for C++? --Guido van Rossum (home page: http://www.python.org/~guido/) From David Abrahams" <15604.47718.334686.681186@12-248-41-177.client.attbi.com> <200205300107.g4U17n606086@pcp742651pcs.reston01.va.comcast.net> <200205301215.g4UCFeU07478@pcp742651pcs.reston01.va.comcast.net> <04eb01c207d5$b3eda680$6601a8c0@boostconsulting.com> <200205301255.g4UCtQX07833@pcp742651pcs.reston01.va.comcast.net> Message-ID: <001b01c207da$4c2bd080$6601a8c0@boostconsulting.com> From: "Guido van Rossum" > > > These "use cases" > > > don't convince me that there's a legitimate use case for > > > string.letters etc. that the methods don't cover. > > > > This is funny. In the C++ community there's a nearly unanimous > > consensus that way too much of the functionality of the standard > > strings is expressed as member functions. > > Interesting. Python used to have the same attitude, hence the string > module -- but the existence of multiple string types made methods more > attractive. > > What's the alternative proposed for C++? Free functions at namespace scope. The analogy would be module-level functions in Python. C++ also has multiple string types, but the availability of overloading makes this approach practical (is it time for Python multimethods yet?) If I were to make arguments against string member functions in Python I'd be talking about the degree of coupling between algorithms and data structures, how it interferes with genericity, and the difficulty that users will have in making "string-like" types... but-i-would-never-make-such-silly-arguments-ly y'rs, dave From neal@metaslash.com Thu May 30 14:06:40 2002 From: neal@metaslash.com (Neal Norwitz) Date: Thu, 30 May 2002 09:06:40 -0400 Subject: [Python-Dev] Re: String module References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <200205291544.g4TFibS18433@odiug.zope.com> <00e001c2072f$7b8d2460$5d61accf@othello> <200205291642.g4TGgaf18754@odiug.zope.com> <013501c20736$3f7732c0$5d61accf@othello> <200205291813.g4TIDKK18906@odiug.zope.com> <3CF52706.10008@lemburg.com> <200205291932.g4TJW0g21814@odiug.zope.com> <200205300112.g4U1CFi06122@pcp742651pcs.reston01.va.comcast.net> <3CF612BC.C5D34633@metaslash.com> <200205301249.g4UCnVD07714@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CF623E0.5CF081F3@metaslash.com> Guido van Rossum wrote: > strop.letters etc. are fixed when locale.setconv(LC_CTYPE, ...) is > used. string.letters etc. are *also* fixed, but if string hasn't been > imported yet when the locale is set, string gets the properly > localized versions from strop (which does the right thing upon > initialization to reflect the locale). This mechanism would have to > be copied into string.py somehow. Michael also schooled me on this. I made a comment on Thomas Heller's patch http://python.org/sf/561832 to indicate this info. His patch adds letters, digits, etc. but not maketrans(). Neal From mwh@python.net Thu May 30 14:10:01 2002 From: mwh@python.net (Michael Hudson) Date: 30 May 2002 14:10:01 +0100 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test test_signal.py,1.9,1.10 In-Reply-To: Guido van Rossum's message of "Thu, 30 May 2002 08:28:09 -0400" References: <2mit56ros4.fsf@starship.python.net> <2msn49hp3i.fsf@starship.python.net> <200205301228.g4UCS9S07601@pcp742651pcs.reston01.va.comcast.net> Message-ID: <2mhekphjdi.fsf@starship.python.net> Guido van Rossum writes: > > Does anyone want to explain BSD threads and their interaction with > > signals to me? > > Yes. Threads and signals don't mix. Period. (Only half a smiley. :-) Well, I'm not mixing threads and signals, really. I've now learnt that when a signal is directed at a process on BSD it is delivered to "a" signal from the set of signals that hasn't blocked it. What I need to know, and can't quite work out, is how many threads are present when you just execute $ ./python and are sitting at the interpreter prompt? Is it just the one (the main thread)? That's what I thought, but I'm unable to explain the behaviour I'm seeing if that is indeed the case. Cheers, M. -- [3] Modem speeds being what they are, large .avi files were generally downloaded to the shell server instead[4]. [4] Where they were usually found by the technical staff, and burned to CD. -- Carlfish, asr From jacobs@penguin.theopalgroup.com Thu May 30 14:20:08 2002 From: jacobs@penguin.theopalgroup.com (Kevin Jacobs) Date: Thu, 30 May 2002 09:20:08 -0400 (EDT) Subject: [Python-Dev] Lazily GC tracking tuples In-Reply-To: Message-ID: On Tue, 28 May 2002, Tim Peters wrote: > [Kevin Jacobs, on Neil's tuple-untracking patch] > > It doesn't seem to make much difference in our app. There seems > > to be some speedup, but nothing dramatic. Turning GC on and off at the > > right times is still slightly faster. > > I'm confused. You earlier said: > > I've found one case where the garbage collector _does_ cause > serious performance problems, and coincidentally enough it is due to > the creation of zillions of cycle-less tuples. Our current solution > has been to disable GC in some parts of our code, and then manually > trigger collection at the correct points. > > Now I'm hearing that turning GC on and off is only "slightly faster", and > that the speedup is so small you're not even sure there is one ("turning GC > on and off" is "slightly faster" than a trial where there "seems to be some > speedup"). This is not "serious performance problems" except to nanosecond > counters like me . Sorry, I wasn't very clear here. The patch _does_ fix the performance problem by untracking cycle-less tuples when we use the naive version of our code (i.e., the one that does not play with the garbage collector). However, the performance of the patched GC when compared to our GC-tuned code is very similar. > > The good news is that another (unrelated) part of our code just became > > about 20-40% faster with this patch, though I need to do some fairly > > major surgery to isolate why this is so. > > Keep us informed! I suspect you're suffering from an app that's more than > 20 lines long; I try to stay away from those. Try 20Kloc for this one. Our unit tests tend to be too fine-grained to detect these kinds of interactions, so it may be a while before I isolate exactly why this is. Thanks, -Kevin -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com From pinard@iro.umontreal.ca Thu May 30 14:24:44 2002 From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard) Date: 30 May 2002 09:24:44 -0400 Subject: [Python-Dev] Re: deprecating string module? In-Reply-To: <200205300107.g4U17n606086@pcp742651pcs.reston01.va.comcast.net> References: <15603.63458.608739.454153@beluga.mojam.com> <15604.47718.334686.681186@12-248-41-177.client.attbi.com> <200205300107.g4U17n606086@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido van Rossum] > [...] I don't think that the data objects have any other function except > for membership testing [...] This is useful being able to quickly enumerate letters, or digits. Just yesterday, I wrote: contexte = {} for nom, valeur in map(None, string.uppercase[:len(valeur)], valeur): assert valeur in '01', valeur contexte[nom] = valeur != '0' -- François Pinard http://www.iro.umontreal.ca/~pinard From fredrik@pythonware.com Thu May 30 14:25:58 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Thu, 30 May 2002 15:25:58 +0200 Subject: [Python-Dev] deprecating string module? References: <15603.63458.608739.454153@beluga.mojam.com> <15604.47718.334686.681186@12-248-41-177.client.attbi.com> <200205300107.g4U17n606086@pcp742651pcs.reston01.va.comcast.net> <200205301215.g4UCFeU07478@pcp742651pcs.reston01.va.comcast.net> <04eb01c207d5$b3eda680$6601a8c0@boostconsulting.com> <200205301255.g4UCtQX07833@pcp742651pcs.reston01.va.comcast.net> <001b01c207da$4c2bd080$6601a8c0@boostconsulting.com> Message-ID: <02b701c207dd$89f0c620$0900a8c0@spiff> david wrote: > > What's the alternative proposed for C++? >=20 > Free functions at namespace scope. The analogy would be module-level > functions in Python. C++ also has multiple string types, but the > availability of overloading makes this approach practical it works pretty well in Python too, of course: the string module delegates to object methods for all functions; e.g. "string.join()" uses the "join" implementation method to do the actual work, just like "len()" uses "__len__" (etc). From oren-py-@hishome.net Thu May 30 14:42:46 2002 From: oren-py-@hishome.net (Oren Tirosh) Date: Thu, 30 May 2002 09:42:46 -0400 Subject: [Python-Dev] PendingDeprecationWarning In-Reply-To: <200205301235.g4UCZ2207631@pcp742651pcs.reston01.va.comcast.net> References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <200205291544.g4TFibS18433@odiug.zope.com> <00e001c2072f$7b8d2460$5d61accf@othello> <20020530140548.A23304@hishome.net> <200205301235.g4UCZ2207631@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020530134246.GA88279@hishome.net> On Thu, May 30, 2002 at 08:35:02AM -0400, Guido van Rossum wrote: > > Some of these are new in 2.2 (like object, dict and file). Some of > > them used to be functions before Python 2.2 (like str, int and > > list). Three of them are still builtin functions in Python 2.2: > > xrange, buffer and slice. Perhaps they should also be converted to > > types for consistency. > > You can help by contributing patches for these three. (Let us know if > you plan to do this, so others can relax.) Okay. It will probably take me 20 time longer than someone who knows his way better around the CPython sources, but I guest I gotta start somewhere. Oren From guido@python.org Thu May 30 14:56:17 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 09:56:17 -0400 Subject: [Python-Dev] Re: String module In-Reply-To: Your message of "Thu, 30 May 2002 09:06:40 EDT." <3CF623E0.5CF081F3@metaslash.com> References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <200205291544.g4TFibS18433@odiug.zope.com> <00e001c2072f$7b8d2460$5d61accf@othello> <200205291642.g4TGgaf18754@odiug.zope.com> <013501c20736$3f7732c0$5d61accf@othello> <200205291813.g4TIDKK18906@odiug.zope.com> <3CF52706.10008@lemburg.com> <200205291932.g4TJW0g21814@odiug.zope.com> <200205300112.g4U1CFi06122@pcp742651pcs.reston01.va.comcast.net> <3CF612BC.C5D34633@metaslash.com> <200205301249.g4UCnVD07714@pcp742651pcs.reston01.va.comcast.net> <3CF623E0.5CF081F3@metaslash.com> Message-ID: <200205301356.g4UDuIm23628@odiug.zope.com> > Michael also schooled me on this. I made a comment on Thomas Heller's > patch http://python.org/sf/561832 to indicate this info. His patch > adds letters, digits, etc. but not maketrans(). I've rejected that patch -- we should use isxxx() methods instead of exposing the sets as variables. --Guido van Rossum (home page: http://www.python.org/~guido/) From gward@python.net Thu May 30 16:43:38 2002 From: gward@python.net (Greg Ward) Date: Thu, 30 May 2002 11:43:38 -0400 Subject: [Python-Dev] Adding Optik to the standard library Message-ID: <20020530154338.GA9215@gerg.ca> I think it's time to declare the work of the getopt-sig finished: several competing proposals were put forward, but Optik appears to be the only really complete, documented, field-tested (by someone other than its author) library. Not everyone on the getopt-sig will agree, but I think that's the broad consensus. Take this with a grain of salt, though -- I'm biased. ;-) Anyways, I think further consensus is needed on how precisely to add Optik to the standard library. The only constraint I've heard from Guido is to give it a less-cutesy name, which is fine by me. First, background: Optik consists of three modules: optik.option, optik.option_parser, and optik.errors, but that detail is hidden from users -- Optik applications normally just do this: from optik import OptionParser although there are a handful of other names that are occasionally useful to import from the 'optik' package: Option, SUPPRESS_HELP, OptionValueError, etc. Optik's __init__.py file is here: http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/*checkout*/optik/optik/lib/__init__.py?rev=1.11&content-type=text/plain It's only about 1100 lines, including docs and comments and blanks, so they could easily be merged into one module if people think that's preferable. So the main issues are 1) where those names should be imported from (interface), and 2) how the standard library should be rearranged to make this interface unobtrusive and efficient (implementation). I'm going to toss out ideas at random until I get bored. Please provide feedback and/or extra ideas on getopt-sig@python.org. IDEA #1: interface: from getopt import OptionParser # and the other Optik names implementation: * turn the getopt module into a package * put the current getopt.py into, say, getopt/classic_getopt.py * make getopt/__init__.py import everything from classic_getopt.py and Optik's three modules, so that either interface is there for the asking pros: * dead simple cons: * applications using just the classic getopt interface suddenly find themselves importing lots more code than they used to IDEA #2: interface: from getopt.option_parser import OptionParser, ... implementation: * as before, getopt.py becomes getopt/classic_getopt.py * getopt/__init__.py consists solely of "from classic_getopt import *" * Optik's three modules are copied into getopt, with the right imports added to getopt/option_parser.py so that applications don't have to worry about where Optik's other names come from pros: * only slightly more burden on apps now using classic getopt cons: * interface is a tad clunkier IDEA #3: interface: from getopt.option_parser import OptionParser, SUPPRESS_HELP, ... from getopt.option import Option from getopt.errors import OptionValueError implementation: * classic getopt handled the same as #2 * just dump Optik's three modules into getopt/ and be done with it pros: * dead simple cons: * clunky interface -- imports expose a lot of implementation detail IDEA #4: interface: same as #1 implementation: * same as #1, except use some funky import-time magic to ensure that the Optik code is only imported if someone actually needs it. Barry Warsaw provided a patch to do this: http://sourceforge.net/tracker/index.php?func=detail&aid=544175&group_id=38019&atid=421099 pros: * more efficient for apps using classic getopt cons: * more complicated; apparently Guido expressed distaste for the import-time magic. I'm a little leery of it myself, although I have not carefully read the code. Preferences? Other ideas? Surely the right solution is out there somewhere, just beyond my reach... Greg -- Greg Ward - Unix geek gward@python.net http://starship.python.net/~gward/ "Passionate hatred can give meaning and purpose to an empty life." -- Eric Hoffer From David Abrahams" When creating new-style classes from within an extension module, the current behavior results in the __module__ attribute being set to the name of the Python module which first imports the extension module. I recall having a similar problem with my own extension classes in Boost.Python v1, but was a little surprised to find that it persists in Boost.Python v2, when creating new-style classes. A trivial inspection reveals this bit of code in typeobject.c: /* Set __module__ in the dict */ if (PyDict_GetItemString(dict, "__module__") == NULL) { tmp = PyEval_GetGlobals(); if (tmp != NULL) { tmp = PyDict_GetItemString(tmp, "__name__"); if (tmp != NULL) { if (PyDict_SetItemString(dict, "__module__", tmp) < 0) return NULL; } } } I can work around this problem, but I wonder if it would be a good idea if Python set __name__ automatically during the import of an extension module? I realize that there are ways to undermine this, e.g. explicitly creating a nested module object. +---------------------------------------------------------------+ David Abrahams C++ Booster (http://www.boost.org) O__ == Pythonista (http://www.python.org) c/ /'_ == resume: http://users.rcn.com/abrahams/resume.html (*) \(*) == email: david.abrahams@rcn.com +---------------------------------------------------------------+ From guido@python.org Thu May 30 16:40:41 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 11:40:41 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test test_signal.py,1.9,1.10 In-Reply-To: Your message of "30 May 2002 14:10:01 BST." <2mhekphjdi.fsf@starship.python.net> References: <2mit56ros4.fsf@starship.python.net> <2msn49hp3i.fsf@starship.python.net> <200205301228.g4UCS9S07601@pcp742651pcs.reston01.va.comcast.net> <2mhekphjdi.fsf@starship.python.net> Message-ID: <200205301540.g4UFefk24115@odiug.zope.com> > Well, I'm not mixing threads and signals, really. I've now learnt > that when a signal is directed at a process on BSD it is delivered to > "a" signal from the set of signals that hasn't blocked it. ^^^^^^ ^^^^^^^ You mean theads, right? > What I need to know, and can't quite work out, is how many threads are > present when you just execute > > $ ./python > > and are sitting at the interpreter prompt? Is it just the one (the > main thread)? That's what I thought, but I'm unable to explain the > behaviour I'm seeing if that is indeed the case. Python doesn't create any threads. On Linux, I know that when you start your first thread, the thread library creates an extra thread for some internal reasons. Who knows what BSD does though. --Guido van Rossum (home page: http://www.python.org/~guido/) From s_lott@yahoo.com Thu May 30 17:01:34 2002 From: s_lott@yahoo.com (Steven Lott) Date: Thu, 30 May 2002 09:01:34 -0700 (PDT) Subject: [Python-Dev] deprecating string module? In-Reply-To: <001b01c207da$4c2bd080$6601a8c0@boostconsulting.com> Message-ID: <20020530160134.10173.qmail@web9603.mail.yahoo.com> I think the "nearly unanimous" is a bit overstated. Some people like the older C-style free functions, but I think this is just old habits fighting on until the bitter end. My (perhaps wacko) preference is to carefully segregate Strings and CharacterSets. After all, iswhitespace() maps a fact about a character set to a specific string instance. I think that the classic ASCII, the various ISO 8859 variants, and the Unicode character sets should be objectified and their attributes include strings of whitespace, uppercase, lowercase, etc., etc. --- David Abrahams wrote: > From: "Guido van Rossum" > > > > > > These "use cases" > > > > don't convince me that there's a legitimate use case for > > > > string.letters etc. that the methods don't cover. > > > > > > This is funny. In the C++ community there's a nearly > unanimous > > > consensus that way too much of the functionality of the > standard > > > strings is expressed as member functions. > > > > Interesting. Python used to have the same attitude, hence > the string > > module -- but the existence of multiple string types made > methods more > > attractive. > > > > What's the alternative proposed for C++? > > Free functions at namespace scope. The analogy would be > module-level > functions in Python. C++ also has multiple string types, but > the > availability of overloading makes this approach practical (is > it time for > Python multimethods yet?) > > If I were to make arguments against string member functions in > Python I'd > be talking about the degree of coupling between algorithms and > data > structures, how it interferes with genericity, and the > difficulty that > users will have in making "string-like" types... > > but-i-would-never-make-such-silly-arguments-ly y'rs, > dave > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev ===== -- S. Lott, CCP :-{) S_LOTT@YAHOO.COM http://www.mindspring.com/~slott1 Buccaneer #468: KaDiMa Macintosh user: drinking upstream from the herd. __________________________________________________ Do You Yahoo!? Yahoo! - Official partner of 2002 FIFA World Cup http://fifaworldcup.yahoo.com From mgilfix@eecs.tufts.edu Thu May 30 17:13:09 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Thu, 30 May 2002 12:13:09 -0400 Subject: [Python-Dev] Re: Deprecation In-Reply-To: <20020529202959.H17248@prim.han.de>; from pyth@devel.trillke.net on Wed, May 29, 2002 at 08:29:59PM +0200 References: <20020529172001.39760.qmail@web9601.mail.yahoo.com> <20020529202959.H17248@prim.han.de> Message-ID: <20020530121309.A8344@eecs.tufts.edu> This would be very cool. Rather than having to go by python version numbers, which seem obscure, an application can declare its dependencies by module. Perhaps even some tool to determine an apps dependencies. These dependencies can then be checked using the current version in a perl-esque regression test style to determine how well the current version meets the applications needs (I say this because some code may not be run normally but require more advanced features - this could also allow for some very interesting approaches to modularity and using available python features in larger applications). It would then be very easy to determine the cause of breakage and/or the need of the application. -- Mike On Wed, May 29 @ 20:29, holger krekel wrote: > IMO Enforcing version numbers in standard libraries is a good idea. > > Eventually, a good versioning scheme for the standard-lib with automated > deprecation may be what is needed. Making the introduction > of new (or deprecation of) features > > a) by hand > > b) by measures of 'x years', example from from PEP4: > "The deprecation warning will be added to the module > one year after Python 2.3 is released, and the > module will be removed one year after that." > > seems unreliable. How do you construct an if-statement > for 'One year after 2.3 is released' if the current version > is 2.2 . > > Instead deprecating > > a) (semi-) automatically > > b) by saying '2.3 issues deprecation warning, 2.4 does not support it'. > > seems much saner. Even a program can be teached to check this. > > The more intransparent the versioning/deprecation scheme > the harder it is to handle and the more people > will scream at every deprecation-proposal. > > deprecated-4223-seconds-after-7th-reply'ly yours, holger -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From mwh@python.net Thu May 30 17:05:04 2002 From: mwh@python.net (Michael Hudson) Date: 30 May 2002 17:05:04 +0100 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test test_signal.py,1.9,1.10 In-Reply-To: Guido van Rossum's message of "Thu, 30 May 2002 11:40:41 -0400" References: <2mit56ros4.fsf@starship.python.net> <2msn49hp3i.fsf@starship.python.net> <200205301228.g4UCS9S07601@pcp742651pcs.reston01.va.comcast.net> <2mhekphjdi.fsf@starship.python.net> <200205301540.g4UFefk24115@odiug.zope.com> Message-ID: <2melfthb9r.fsf@starship.python.net> Guido van Rossum writes: > > Well, I'm not mixing threads and signals, really. I've now learnt > > that when a signal is directed at a process on BSD it is delivered to > > "a" signal from the set of signals that hasn't blocked it. > ^^^^^^ ^^^^^^^ > > You mean theads, right? thReads, yes :) > > What I need to know, and can't quite work out, is how many threads are > > present when you just execute > > > > $ ./python > > > > and are sitting at the interpreter prompt? Is it just the one (the > > main thread)? That's what I thought, but I'm unable to explain the > > behaviour I'm seeing if that is indeed the case. > > Python doesn't create any threads. On Linux, I know that when you > start your first thread, the thread library creates an extra thread > for some internal reasons. Who knows what BSD does though. I'm not sure either, but I have convinced myself that signal mask handling is just buggered on BSD when the program is compiled in a multi-threaded style (as, in simple C programs don't do what you (well, I) would expect). Note this isn't about actually using threads -- just using "cc -pthreads". Now what do I do? Back my patch out? Not expose the functions on BSD? It works on Linux... Cheers, M. -- ARTHUR: Why should he want to know where his towel is? FORD: Everybody should know where his towel is. ARTHUR: I think your head's come undone. -- The Hitch-Hikers Guide to the Galaxy, Episode 7 From pobrien@orbtech.com Thu May 30 17:09:44 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Thu, 30 May 2002 11:09:44 -0500 Subject: [Python-Dev] RE: [getopt-sig] Adding Optik to the standard library In-Reply-To: <20020530154338.GA9215@gerg.ca> Message-ID: [Greg Ward] > > Anyways, I think further consensus is needed on how precisely to add > Optik to the standard library. The only constraint I've heard from > Guido is to give it a less-cutesy name, which is fine by me. Any chance we could wean you off your preference for underscores? --- Patrick K. O'Brien Orbtech From niemeyer@conectiva.com Thu May 30 17:26:02 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Thu, 30 May 2002 13:26:02 -0300 Subject: [Python-Dev] __module__ of newly-created extension classes In-Reply-To: <00b401c207f0$5e92a710$6601a8c0@boostconsulting.com> References: <00b401c207f0$5e92a710$6601a8c0@boostconsulting.com> Message-ID: <20020530132602.A24455@ibook.distro.conectiva> Hi David! > When creating new-style classes from within an extension module, the > current behavior results in the __module__ attribute being set to the name > of the Python module which first imports the extension module. [...] I'm not sure I understood your problem. If I got it correctly, you will get the right behavior by setting tp_name to something like "module.TypeName". -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From guido@python.org Thu May 30 17:27:38 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 12:27:38 -0400 Subject: [Python-Dev] Re: [getopt-sig] Adding Optik to the standard library In-Reply-To: Your message of "Thu, 30 May 2002 11:43:38 EDT." <20020530154338.GA9215@gerg.ca> References: <20020530154338.GA9215@gerg.ca> Message-ID: <200205301627.g4UGRcQ24434@odiug.zope.com> I think it's better to pick a new name and leave the existing getopt module alone. I think keeping it a package is fine. I prefer to have little or no magic in __init__.py though (the email package's __init__.py is borderline :-). I think that "options" is a fine package name. Yes, there are other things that one could consider options. No, I don't think that will cause confusion. After all "getopt" isn't much more specific. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu May 30 17:31:59 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 12:31:59 -0400 Subject: [Python-Dev] Re: Deprecation In-Reply-To: Your message of "Thu, 30 May 2002 12:13:09 EDT." <20020530121309.A8344@eecs.tufts.edu> References: <20020529172001.39760.qmail@web9601.mail.yahoo.com> <20020529202959.H17248@prim.han.de> <20020530121309.A8344@eecs.tufts.edu> Message-ID: <200205301631.g4UGVxZ24485@odiug.zope.com> > This would be very cool. Rather than having to go by python > version numbers, which seem obscure, an application can declare its > dependencies by module. Perhaps even some tool to determine an apps > dependencies. These dependencies can then be checked using the > current version in a perl-esque regression test style to determine > how well the current version meets the applications needs (I say > this because some code may not be run normally but require more > advanced features - this could also allow for some very interesting > approaches to modularity and using available python features in larger > applications). It would then be very easy to determine the cause > of breakage and/or the need of the application. Hm, this sounds like overkill to me. And who's going to write the AI software to do this automatic regression testing? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu May 30 17:35:59 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 12:35:59 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test test_signal.py,1.9,1.10 In-Reply-To: Your message of "30 May 2002 17:05:04 BST." <2melfthb9r.fsf@starship.python.net> References: <2mit56ros4.fsf@starship.python.net> <2msn49hp3i.fsf@starship.python.net> <200205301228.g4UCS9S07601@pcp742651pcs.reston01.va.comcast.net> <2mhekphjdi.fsf@starship.python.net> <200205301540.g4UFefk24115@odiug.zope.com> <2melfthb9r.fsf@starship.python.net> Message-ID: <200205301635.g4UGZxs24547@odiug.zope.com> > I'm not sure either, but I have convinced myself that signal mask > handling is just buggered on BSD when the program is compiled in a > multi-threaded style (as, in simple C programs don't do what you > (well, I) would expect). Note this isn't about actually using threads > -- just using "cc -pthreads". > > Now what do I do? Back my patch out? Not expose the functions on > BSD? It works on Linux... Add a configure check for the desired behavior and enable the changes only when the system behaves correctly. --Guido van Rossum (home page: http://www.python.org/~guido/) From mgilfix@eecs.tufts.edu Thu May 30 17:43:29 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Thu, 30 May 2002 12:43:29 -0400 Subject: [Python-Dev] Re: Deprecation In-Reply-To: <200205301631.g4UGVxZ24485@odiug.zope.com>; from guido@python.org on Thu, May 30, 2002 at 12:31:59PM -0400 References: <20020529172001.39760.qmail@web9601.mail.yahoo.com> <20020529202959.H17248@prim.han.de> <20020530121309.A8344@eecs.tufts.edu> <200205301631.g4UGVxZ24485@odiug.zope.com> Message-ID: <20020530124329.B8344@eecs.tufts.edu> On Thu, May 30 @ 12:31, Guido van Rossum wrote: > > This would be very cool. Rather than having to go by python > > version numbers, which seem obscure, an application can declare its > > dependencies by module. Perhaps even some tool to determine an apps > > dependencies. These dependencies can then be checked using the > > current version in a perl-esque regression test style to determine > > how well the current version meets the applications needs (I say > > this because some code may not be run normally but require more > > advanced features - this could also allow for some very interesting > > approaches to modularity and using available python features in larger > > applications). It would then be very easy to determine the cause > > of breakage and/or the need of the application. > > Hm, this sounds like overkill to me. And who's going to write the AI > software to do this automatic regression testing? Perhaps it came across as more than it should be. I meant just specifying the version numbers of the individual modules in a CPAN-like dependency thing. Then, on start-up, the dependencies are just checked. Mostly matching of version numbers. The stats are just a fancy way of saying how screwed you are :) -- Mike -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From David Abrahams" <20020530132602.A24455@ibook.distro.conectiva> Message-ID: <016601c207fa$f5679560$6601a8c0@boostconsulting.com> From: "Gustavo Niemeyer" > Hi David! > > > When creating new-style classes from within an extension module, the > > current behavior results in the __module__ attribute being set to the name > > of the Python module which first imports the extension module. > [...] > > I'm not sure I understood your problem. If I got it correctly, you will > get the right behavior by setting tp_name to something like > "module.TypeName". Well, that wouldn't fix the __module__ attribute AFAICT. I'm creating new classes by invoking the metaclass with the name, bases, and dictionary. -Dave From python@rcn.com Thu May 30 18:05:04 2002 From: python@rcn.com (Raymond Hettinger) Date: Thu, 30 May 2002 13:05:04 -0400 Subject: [Python-Dev] String module Message-ID: <001501c207fc$2573e320$4d66accf@othello> > > Maybe you can do a patch for isxxx() methods instead? > > Will do. I need to un-volunteer on this one. The string module already has a bunch of isxxx methods and I'm not clear which others are needed or how to make them vary by locale. Raymond Hettinger From guido@python.org Thu May 30 18:04:41 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 13:04:41 -0400 Subject: [Python-Dev] String module In-Reply-To: Your message of "Thu, 30 May 2002 13:05:04 EDT." <001501c207fc$2573e320$4d66accf@othello> References: <001501c207fc$2573e320$4d66accf@othello> Message-ID: <200205301704.g4UH4fo24680@odiug.zope.com> > I need to un-volunteer on this one. > > The string module already has a bunch of isxxx methods > and I'm not clear which others are needed or how to > make them vary by locale. OK. Nothing needs to be done to vary them by locale as long as you use the isxxx macros from . I think all that's needed is to add the missing ones. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu May 30 18:06:05 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 13:06:05 -0400 Subject: [Python-Dev] Re: Deprecation In-Reply-To: Your message of "Thu, 30 May 2002 12:43:29 EDT." <20020530124329.B8344@eecs.tufts.edu> References: <20020529172001.39760.qmail@web9601.mail.yahoo.com> <20020529202959.H17248@prim.han.de> <20020530121309.A8344@eecs.tufts.edu> <200205301631.g4UGVxZ24485@odiug.zope.com> <20020530124329.B8344@eecs.tufts.edu> Message-ID: <200205301706.g4UH65324697@odiug.zope.com> > > > This would be very cool. Rather than having to go by python > > > version numbers, which seem obscure, an application can declare its > > > dependencies by module. Perhaps even some tool to determine an apps > > > dependencies. These dependencies can then be checked using the > > > current version in a perl-esque regression test style to determine > > > how well the current version meets the applications needs (I say > > > this because some code may not be run normally but require more > > > advanced features - this could also allow for some very interesting > > > approaches to modularity and using available python features in larger > > > applications). It would then be very easy to determine the cause > > > of breakage and/or the need of the application. > > > > Hm, this sounds like overkill to me. And who's going to write the AI > > software to do this automatic regression testing? > > Perhaps it came across as more than it should be. I meant just specifying > the version numbers of the individual modules in a CPAN-like > dependency thing. Then, on start-up, the dependencies are just > checked. Mostly matching of version numbers. The stats are just a > fancy way of saying how screwed you are :) This has been proposed before, but I don't think anybody really understands how to do version dependencies correctly. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu May 30 18:42:07 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 13:42:07 -0400 Subject: [Python-Dev] __module__ of newly-created extension classes In-Reply-To: Your message of "Thu, 30 May 2002 11:38:29 EDT." <00b401c207f0$5e92a710$6601a8c0@boostconsulting.com> References: <00b401c207f0$5e92a710$6601a8c0@boostconsulting.com> Message-ID: <200205301742.g4UHg7j26440@odiug.zope.com> > When creating new-style classes from within an extension module, the > current behavior results in the __module__ attribute being set to the name > of the Python module which first imports the extension module. > > I recall having a similar problem with my own extension classes in > Boost.Python v1, but was a little surprised to find that it persists in > Boost.Python v2, when creating new-style classes. A trivial inspection > reveals this bit of code in typeobject.c: > > /* Set __module__ in the dict */ > if (PyDict_GetItemString(dict, "__module__") == NULL) { > tmp = PyEval_GetGlobals(); > if (tmp != NULL) { > tmp = PyDict_GetItemString(tmp, "__name__"); > if (tmp != NULL) { > if (PyDict_SetItemString(dict, "__module__", > tmp) < 0) > return NULL; > } > } > } > > I can work around this problem, but I wonder if it would be a good > idea if Python set __name__ automatically during the import of an > extension module? I realize that there are ways to undermine this, > e.g. explicitly creating a nested module object. The __name__ attribute is already set as soon as a module is created. I think the problem is that the extension's dict is not the "current globals dict" as returned by PyEval_GetGlobals(). There's no convenient way to make this so, since the "current globals dict" is taken from the frame (and I don't want to change that). What was your workaround? I thought that the "correct" way to set __module__ is to use a dotted name as the type name (the initializer for tp_name). Can you do that? --Guido van Rossum (home page: http://www.python.org/~guido/) From mgilfix@eecs.tufts.edu Thu May 30 18:53:15 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Thu, 30 May 2002 13:53:15 -0400 Subject: [Python-Dev] Re: Deprecation In-Reply-To: <200205301706.g4UH65324697@odiug.zope.com>; from guido@python.org on Thu, May 30, 2002 at 01:06:05PM -0400 References: <20020529172001.39760.qmail@web9601.mail.yahoo.com> <20020529202959.H17248@prim.han.de> <20020530121309.A8344@eecs.tufts.edu> <200205301631.g4UGVxZ24485@odiug.zope.com> <20020530124329.B8344@eecs.tufts.edu> <200205301706.g4UH65324697@odiug.zope.com> Message-ID: <20020530135314.C8344@eecs.tufts.edu> Well, I'll have to think about it and get back to you :) On Thu, May 30 @ 13:06, Guido van Rossum wrote: > > > > This would be very cool. Rather than having to go by python > > > > version numbers, which seem obscure, an application can declare its > > > > dependencies by module. Perhaps even some tool to determine an apps > > > > dependencies. These dependencies can then be checked using the > > > > current version in a perl-esque regression test style to determine > > > > how well the current version meets the applications needs (I say > > > > this because some code may not be run normally but require more > > > > advanced features - this could also allow for some very interesting > > > > approaches to modularity and using available python features in larger > > > > applications). It would then be very easy to determine the cause > > > > of breakage and/or the need of the application. > > > > > > Hm, this sounds like overkill to me. And who's going to write the AI > > > software to do this automatic regression testing? > > > > Perhaps it came across as more than it should be. I meant just specifying > > the version numbers of the individual modules in a CPAN-like > > dependency thing. Then, on start-up, the dependencies are just > > checked. Mostly matching of version numbers. The stats are just a > > fancy way of saying how screwed you are :) > > This has been proposed before, but I don't think anybody really > understands how to do version dependencies correctly. `-> (guido) -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From python@rcn.com Thu May 30 19:01:35 2002 From: python@rcn.com (Raymond Hettinger) Date: Thu, 30 May 2002 14:01:35 -0400 Subject: [Python-Dev] String module References: <001501c207fc$2573e320$4d66accf@othello> <200205301704.g4UH4fo24680@odiug.zope.com> Message-ID: <004201c20804$0a7ab320$4d66accf@othello> From: "Guido van Rossum" > > I need to un-volunteer on this one. > > > > The string module already has a bunch of isxxx methods > > and I'm not clear which others are needed or how to > > make them vary by locale. > > OK. Nothing needs to be done to vary them by locale as long as you > use the isxxx macros from . I think all that's needed is to > add the missing ones. > It's not that hard once you know what to do. See www.python.org/sf/562501 Raymond Hettinger From kendall@monkeyfist.com Thu May 30 19:37:03 2002 From: kendall@monkeyfist.com (Kendall Grant Clark) Date: Thu, 30 May 2002 13:37:03 -0500 Subject: [Python-Dev] Re: [getopt-sig] Adding Optik to the standard library In-Reply-To: <200205301627.g4UGRcQ24434@odiug.zope.com> References: <20020530154338.GA9215@gerg.ca> <200205301627.g4UGRcQ24434@odiug.zope.com> Message-ID: <15606.29007.98422.889643@rosa.monkeyfist.com> >>>>> "guido" == Guido van Rossum writes: guido> I think it's better to pick a new name and leave the existing getopt guido> module alone. How about "OptParser" (alternatives: OptionsParser, OptsParser) as an analogue to the existing ConfigParser? They do go together, both conceptually and in practice, after all. That would leave the more general "options" free for something, well, more general. :> Best, Kendall Clark From guido@python.org Thu May 30 19:35:26 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 14:35:26 -0400 Subject: [Python-Dev] String module In-Reply-To: Your message of "Thu, 30 May 2002 14:01:35 EDT." <004201c20804$0a7ab320$4d66accf@othello> References: <001501c207fc$2573e320$4d66accf@othello> <200205301704.g4UH4fo24680@odiug.zope.com> <004201c20804$0a7ab320$4d66accf@othello> Message-ID: <200205301835.g4UIZQv26785@odiug.zope.com> > > OK. Nothing needs to be done to vary them by locale as long as you > > use the isxxx macros from . I think all that's needed is to > > add the missing ones. > > It's not that hard once you know what to do. > > See www.python.org/sf/562501 Thanks! But now we have a diverging set of isxxx methods for 8-bit strings and Unicode. I really don't know what the equivalent of these (ispunct, iscntrl, isgraph, isprint) is in Unicode -- maybe MAL or MvL know? Unicode also has a wider definition of digits; do we want to extend isxdigit() for that? (Probably not, but I'm not sure.) Someone commented that isxdigit is a poor name. OTOH it's what C uses. I'm not sure what to say. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu May 30 19:41:58 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 14:41:58 -0400 Subject: [Python-Dev] Re: [getopt-sig] Adding Optik to the standard library In-Reply-To: Your message of "Thu, 30 May 2002 13:37:03 CDT." <15606.29007.98422.889643@rosa.monkeyfist.com> References: <20020530154338.GA9215@gerg.ca> <200205301627.g4UGRcQ24434@odiug.zope.com> <15606.29007.98422.889643@rosa.monkeyfist.com> Message-ID: <200205301841.g4UIfwl26822@odiug.zope.com> > How about "OptParser" (alternatives: OptionsParser, OptsParser) as > an analogue to the existing ConfigParser? They do go together, both > conceptually and in practice, after all. A decent guideline is to use the dominant class name as the module name. That would be OptionParser. Then, instead of from optik import OptionParser we'd be writing from OptionParser import OptionParser I like this! This works best if it is a single file; making OptionParser a package would just complicate things. So maybe I should take Greg up on his offer to refactor the code into a single .py file. (Barry prefers that there's only one class per file; fortunately I don't have that hangup. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From gward@python.net Thu May 30 20:36:44 2002 From: gward@python.net (Greg Ward) Date: Thu, 30 May 2002 15:36:44 -0400 Subject: [Python-Dev] Re: [getopt-sig] Adding Optik to the standard library In-Reply-To: <200205301841.g4UIfwl26822@odiug.zope.com> References: <20020530154338.GA9215@gerg.ca> <200205301627.g4UGRcQ24434@odiug.zope.com> <15606.29007.98422.889643@rosa.monkeyfist.com> <200205301841.g4UIfwl26822@odiug.zope.com> Message-ID: <20020530193644.GA10197@gerg.ca> On 30 May 2002, Guido van Rossum said: > > How about "OptParser" (alternatives: OptionsParser, OptsParser) as > > an analogue to the existing ConfigParser? They do go together, both > > conceptually and in practice, after all. > > A decent guideline is to use the dominant class name as the module > name. That would be OptionParser. Then, instead of > > from optik import OptionParser > > we'd be writing > > from OptionParser import OptionParser > > I like this! I think the BDFL has spoken. I can live with this, although I prefer lower-case module names. Whatever. Greg -- Greg Ward - geek-at-large gward@python.net http://starship.python.net/~gward/ Paranoia is simply an optimistic outlook on life. From fredrik@pythonware.com Thu May 30 20:39:33 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Thu, 30 May 2002 21:39:33 +0200 Subject: [Python-Dev] deprecating string module? References: <3CF486F4.6070302@lemburg.com><15604.47928.443522.504730@12-248-41-177.client.attbi.com><3CF4C15A.7070606@lemburg.com><062701c20710$5679a8c0$6300000a@holdenweb.com><002801c20714$ad139fc0$0900a8c0@spiff> Message-ID: <018101c20811$bf4d36d0$ced241d5@hagrid> martin wrote: > > > have you tried migrating a huge python system from 1.5.2 to 2.x? > > if so, what did you learn? > > That it is easier than porting from 1.4. this does not match my experiences (neither from 24/7 production systems, nor any of our gui applications). what kind of applications have you ported? From python@rcn.com Thu May 30 20:48:32 2002 From: python@rcn.com (Raymond Hettinger) Date: Thu, 30 May 2002 15:48:32 -0400 Subject: [Python-Dev] String module References: <001501c207fc$2573e320$4d66accf@othello> <200205301704.g4UH4fo24680@odiug.zope.com> <004201c20804$0a7ab320$4d66accf@othello> <200205301835.g4UIZQv26785@odiug.zope.com> Message-ID: <001701c20812$fb818c40$95d8accf@othello> From: "Guido van Rossum" > Thanks! But now we have a diverging set of isxxx methods for 8-bit > strings and Unicode. I really don't know what the equivalent of these > (ispunct, iscntrl, isgraph, isprint) is in Unicode -- maybe MAL or MvL > know? Unicode also has a wider definition of digits; do we want to > extend isxdigit() for that? (Probably not, but I'm not sure.) I'll spend some time with the big Unicode 3.0 book this evening and chat with some Unicode techno-weenies. When I've got an answer will add the unicodeobject.c methods to the patch. > Someone commented that isxdigit is a poor name. OTOH it's what C > uses. I'm not sure what to say. I concur. I had to look it up on google to make sure in meant what I surmised it meant. ishexdigit() is more explicit. Besides, C naming conventions aren't exactly role models for clarity ;) While we're at it: isgraph() --> isvisible() iscntrl() --> iscontrol() isprint() --> isprintable() I'm sure everyone will have an opinion or two. Raymond Hettinger From guido@python.org Thu May 30 20:46:21 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 15:46:21 -0400 Subject: [Python-Dev] Re: [getopt-sig] Adding Optik to the standard library In-Reply-To: Your message of "Thu, 30 May 2002 15:36:44 EDT." <20020530193644.GA10197@gerg.ca> References: <20020530154338.GA9215@gerg.ca> <200205301627.g4UGRcQ24434@odiug.zope.com> <15606.29007.98422.889643@rosa.monkeyfist.com> <200205301841.g4UIfwl26822@odiug.zope.com> <20020530193644.GA10197@gerg.ca> Message-ID: <200205301946.g4UJkLL26987@odiug.zope.com> > > from OptionParser import OptionParser > > > > I like this! > > I think the BDFL has spoken. I can live with this, although I prefer > lower-case module names. Whatever. I prefer lower-case module names for most situations, but I make an exception for modulename == classname. --Guido van Rossum (home page: http://www.python.org/~guido/) From ping@zesty.ca Thu May 30 20:58:50 2002 From: ping@zesty.ca (Ka-Ping Yee) Date: Thu, 30 May 2002 14:58:50 -0500 (CDT) Subject: [Python-Dev] String module In-Reply-To: <001701c20812$fb818c40$95d8accf@othello> Message-ID: On Thu, 30 May 2002, Raymond Hettinger wrote: > > Someone commented that isxdigit is a poor name. OTOH it's what C > > uses. I'm not sure what to say. Obviously, it refers to the ex-digits, once honoured and respected along with the other ten Arabic numerals but now long since neglected. This includes, for example, I, V, and X, the first nine Greek letters (supported if the argument is a Unicode string), and so on. -- ?!ng From martin@v.loewis.de Thu May 30 21:10:51 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 30 May 2002 22:10:51 +0200 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test test_signal.py,1.9,1.10 In-Reply-To: <2msn49hp3i.fsf@starship.python.net> References: <2mit56ros4.fsf@starship.python.net> <2msn49hp3i.fsf@starship.python.net> Message-ID: Michael Hudson writes: > It doesn't fail if you disable threading. Oh /good/. > > Does anyone want to explain BSD threads and their interaction with > signals to me? I believe this is an overgeneralization in question. Thread implementations in BSD vary from "distribution to distribution" (Free, Net, Open), and may also vary from release to release. For example, one of them recently dropped the libc_r library. Try compiling with the -mt option, unless this is already done; it appears that this causes the compiler to define things that are otherwise not defined. Regards, Martin From guido@python.org Thu May 30 21:10:49 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 16:10:49 -0400 Subject: [Python-Dev] String module In-Reply-To: Your message of "Thu, 30 May 2002 15:48:32 EDT." <001701c20812$fb818c40$95d8accf@othello> References: <001501c207fc$2573e320$4d66accf@othello> <200205301704.g4UH4fo24680@odiug.zope.com> <004201c20804$0a7ab320$4d66accf@othello> <200205301835.g4UIZQv26785@odiug.zope.com> <001701c20812$fb818c40$95d8accf@othello> Message-ID: <200205302010.g4UKAn927215@odiug.zope.com> > > Someone commented that isxdigit is a poor name. OTOH it's what C > > uses. I'm not sure what to say. > > I concur. I had to look it up on google to make sure in meant > what I surmised it meant. ishexdigit() is more explicit. > Besides, C naming conventions aren't exactly role models for clarity ;) > > While we're at it: > isgraph() --> isvisible() > iscntrl() --> iscontrol() > isprint() --> isprintable() Sure. But I still can't guess the relationships between these three graph excludes space, print includes it). And then maybe also ispunctuation()? Or is that too long? My ctype.h man page also has these: isblank -- space or tab (GNU extension) isascii -- 7-bit value (BSD/SVID extension) --Guido van Rossum (home page: http://www.python.org/~guido/) From martin@v.loewis.de Thu May 30 21:13:05 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 30 May 2002 22:13:05 +0200 Subject: [Python-Dev] Re: String module In-Reply-To: <2mptzdhmms.fsf@starship.python.net> References: <200205291358.g4TDw5v16789@odiug.zope.com> <006b01c20723$2a58aee0$5d61accf@othello> <200205291523.g4TFNiA18104@odiug.zope.com> <3CF4F6C8.E57807FF@metaslash.com> <200205291544.g4TFibS18433@odiug.zope.com> <00e001c2072f$7b8d2460$5d61accf@othello> <200205291642.g4TGgaf18754@odiug.zope.com> <013501c20736$3f7732c0$5d61accf@othello> <200205291813.g4TIDKK18906@odiug.zope.com> <3CF52706.10008@lemburg.com> <200205291932.g4TJW0g21814@odiug.zope.com> <200205300112.g4U1CFi06122@pcp742651pcs.reston01.va.comcast.net> <3CF612BC.C5D34633@metaslash.com> <2mptzdhmms.fsf@starship.python.net> Message-ID: Michael Hudson writes: > The strop definitions reflect the locale better than the ones in > string.py. Can you elaborate? They are always supposed to be synchronous. Regards, Martin From martin@v.loewis.de Thu May 30 21:16:49 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 30 May 2002 22:16:49 +0200 Subject: [Python-Dev] deprecating string module? In-Reply-To: <018101c20811$bf4d36d0$ced241d5@hagrid> References: <3CF486F4.6070302@lemburg.com> <15604.47928.443522.504730@12-248-41-177.client.attbi.com> <3CF4C15A.7070606@lemburg.com> <062701c20710$5679a8c0$6300000a@holdenweb.com> <002801c20714$ad139fc0$0900a8c0@spiff> <018101c20811$bf4d36d0$ced241d5@hagrid> Message-ID: "Fredrik Lundh" writes: > > > have you tried migrating a huge python system from 1.5.2 to 2.x? > > > if so, what did you learn? > > > > That it is easier than porting from 1.4. > > this does not match my experiences (neither from 24/7 production > systems, nor any of our gui applications). > > what kind of applications have you ported? I found the primary difference in the use of the regex module: Python 1.5 code would use the re module, which was not available in 1.4. This now causes a DeprecationWarning. The most recent instance where I had to change this was YAPPS. Regards, Martin From David Abrahams" <200205301742.g4UHg7j26440@odiug.zope.com> Message-ID: <020601c20816$b973c530$6601a8c0@boostconsulting.com> From: "Guido van Rossum" > > I can work around this problem, but I wonder if it would be a good > > idea if Python set __name__ automatically during the import of an > > extension module? I realize that there are ways to undermine this, > > e.g. explicitly creating a nested module object. > > The __name__ attribute is already set as soon as a module is created. > > I think the problem is that the extension's dict is not the "current > globals dict" as returned by PyEval_GetGlobals(). Ah, thanks. I only did a very quick analysis, and didn't remember the way it turned out when I looked at this in past years for v1. > There's no convenient way to make this so, since the "current globals > dict" is taken from the frame (and I don't want to change that). Fine, I understand, I think. > What was your workaround? If I recall correctly, what I was doing in v1 was to keep track of the module being initialized in some kind of stack under-the-covers, and to grab its __name__ attribute. > I thought that the "correct" way to set __module__ is to use a dotted > name as the type name (the initializer for tp_name). You would know. Didn't you write that code? > Can you do that? Hmm, I see that you have a property which grabs the __module__ from the class name... Since I am invoking the metaclass to create classes, I don't set tp_name directly, but I could manufacture the right name and pass it to the metaclass. That requires pretty much the same sort of under-the-covers collaboration between the code that creates the module and the code that creates the class. That's OK, but (minor point) it means that my library, which is meant to be a framework of re-usable components, isn't really as modular as it should be: you couldn't just use the class generation facilities in the context of someone else's module creation. Now I'm wondering if there isn't a better way for Python to get ahold of the current module's __name__ in this case, if you don't want to change what PyEval_GetGlobals() does. -Dave From pyth@devel.trillke.net Thu May 30 21:28:21 2002 From: pyth@devel.trillke.net (holger krekel) Date: Thu, 30 May 2002 22:28:21 +0200 Subject: [Python-Dev] Re: Deprecation In-Reply-To: <20020530121309.A8344@eecs.tufts.edu>; from mgilfix@eecs.tufts.edu on Thu, May 30, 2002 at 12:13:09PM -0400 References: <20020529172001.39760.qmail@web9601.mail.yahoo.com> <20020529202959.H17248@prim.han.de> <20020530121309.A8344@eecs.tufts.edu> Message-ID: <20020530222821.M17248@prim.han.de> Michael Gilfix wrote: > This would be very cool. Rather than having to go by python > version numbers, which seem obscure, an application can declare its > dependencies by module. Perhaps even some tool to determine an apps > dependencies. > (...) while i agree to this basic idea i wanted to remind of the other point buried in my mail ... Changing the deprecation process to be based on *version numbers* rather than *uncertain future points in time*... > On Wed, May 29 @ 20:29, holger krekel wrote: > > IMO Enforcing version numbers in standard libraries is a good idea. > > > > Eventually, a good versioning scheme for the standard-lib with automated > > deprecation may be what is needed. Making the introduction > > of new (or deprecation of) features > > > > a) by hand > > > > b) by measures of 'x years', example from from PEP4: > > "The deprecation warning will be added to the module > > one year after Python 2.3 is released, and the > > module will be removed one year after that." > > > > seems unreliable. How do you construct an if-statement > > for 'One year after 2.3 is released' if the current version > > is 2.2 . > > > > Instead deprecating > > > > a) (semi-) automatically > > > > b) by saying '2.3 issues deprecation warning, 2.4 does not support it'. > > > > seems much saner. Even a program can be teached to check this. Am i the only one to find this an evident change for the good (tm) ? holger From David Abrahams" Message-ID: <026101c20818$d36e3e50$6601a8c0@boostconsulting.com> From: "Steven Lott" > I think the "nearly unanimous" is a bit overstated. Might be. > Some people like the older C-style free functions, but I think > this is just old habits fighting on until the bitter end. It's not just that. People are generally unhappy with "kitchen sink" classes (admittedly C++ strings have a horrible case of this because of the combination of STL and legacy interfaces). Also, member functions tend to be difficult for generic programs. > My (perhaps wacko) preference is to carefully segregate > Strings and CharacterSets. > > After all, iswhitespace() maps a fact about a character set > to a specific string instance. > > I think that the classic ASCII, the various ISO 8859 variants, > and the Unicode character sets should > be objectified and their attributes include strings of > whitespace, uppercase, lowercase, etc., etc. I think we're way OT here, but do you know about char_traits? -Dave From barry@zope.com Thu May 30 21:37:55 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 30 May 2002 16:37:55 -0400 Subject: [Python-Dev] Re: [getopt-sig] Adding Optik to the standard library References: <20020530154338.GA9215@gerg.ca> <200205301627.g4UGRcQ24434@odiug.zope.com> <15606.29007.98422.889643@rosa.monkeyfist.com> <200205301841.g4UIfwl26822@odiug.zope.com> Message-ID: <15606.36259.211854.324615@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> we'd be writing GvR> from OptionParser import OptionParser GvR> I like this! GvR> This works best if it is a single file; making OptionParser a GvR> package would just complicate things. So maybe I should take GvR> Greg up on his offer to refactor the code into a single .py GvR> file. GvR> (Barry prefers that there's only one class per file; GvR> fortunately I don't have that hangup. :-) If OptionParser were to be a package with lots of public classes then yah, I'd be (more of a) neurotic noodge about it, but as it is... +1 ! :) -Barry From pyth@devel.trillke.net Thu May 30 21:41:13 2002 From: pyth@devel.trillke.net (holger krekel) Date: Thu, 30 May 2002 22:41:13 +0200 Subject: [Python-Dev] Re: Deprecation In-Reply-To: <200205301706.g4UH65324697@odiug.zope.com>; from guido@python.org on Thu, May 30, 2002 at 01:06:05PM -0400 References: <20020529172001.39760.qmail@web9601.mail.yahoo.com> <20020529202959.H17248@prim.han.de> <20020530121309.A8344@eecs.tufts.edu> <200205301631.g4UGVxZ24485@odiug.zope.com> <20020530124329.B8344@eecs.tufts.edu> <200205301706.g4UH65324697@odiug.zope.com> Message-ID: <20020530224113.N17248@prim.han.de> Guido van Rossum wrote: > > > > This would be very cool. Rather than having to go by python > > > > version numbers, which seem obscure, an application can declare its > > > > dependencies by module. Perhaps even some tool to determine an apps > > > > dependencies. These dependencies can then be checked using the > > > > current version in a perl-esque regression test style to determine > > > > how well the current version meets the applications needs (I say > > > > this because some code may not be run normally but require more > > > > advanced features - this could also allow for some very interesting > > > > approaches to modularity and using available python features in larger > > > > applications). It would then be very easy to determine the cause > > > > of breakage and/or the need of the application. > > > > > > Hm, this sounds like overkill to me. And who's going to write the AI > > > software to do this automatic regression testing? > > > > Perhaps it came across as more than it should be. I meant just specifying > > the version numbers of the individual modules in a CPAN-like > > dependency thing. Then, on start-up, the dependencies are just > > checked. Mostly matching of version numbers. The stats are just a > > fancy way of saying how screwed you are :) > > This has been proposed before, but I don't think anybody really > understands how to do version dependencies correctly. But i assume, you do agree that a *working* versioning and dependency scheme for the standard-lib would be more than nice. (IMHO the more modules and versions we have the more important it gets). holger From barry@zope.com Thu May 30 21:45:12 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 30 May 2002 16:45:12 -0400 Subject: [Python-Dev] Re: [getopt-sig] Adding Optik to the standard library References: <20020530154338.GA9215@gerg.ca> <200205301627.g4UGRcQ24434@odiug.zope.com> <15606.29007.98422.889643@rosa.monkeyfist.com> <200205301841.g4UIfwl26822@odiug.zope.com> <20020530193644.GA10197@gerg.ca> <200205301946.g4UJkLL26987@odiug.zope.com> Message-ID: <15606.36696.222173.725350@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> I prefer lower-case module names for most situations, but I GvR> make an exception for modulename == classname. Agreed! That's what the style guide says too. :) -Barry From postmaster@python.org Thu May 30 22:47:58 2002 From: postmaster@python.org (postmaster) Date: Thu, 30 May 2002 22:47:58 +0100 Subject: [Python-Dev] Returned mail--"on Tue 4 Jul 2000 17" Message-ID: <3CEE5C170000964D@webmail.fmcf.fr> (added by postmaster@webmail.fmcf.fr) *** A virus was detected by the security administrator; this message was discarded *** From guido@python.org Thu May 30 21:53:44 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 16:53:44 -0400 Subject: [Python-Dev] deprecating string module? In-Reply-To: Your message of "Thu, 30 May 2002 16:30:20 EDT." <026101c20818$d36e3e50$6601a8c0@boostconsulting.com> References: <20020530160134.10173.qmail@web9603.mail.yahoo.com> <026101c20818$d36e3e50$6601a8c0@boostconsulting.com> Message-ID: <200205302053.g4UKriU27444@odiug.zope.com> > It's not just that. People are generally unhappy with "kitchen sink" > classes (admittedly C++ strings have a horrible case of this because of the > combination of STL and legacy interfaces). Also, member functions tend to > be difficult for generic programs. Is this still relevant to Python? Why are C++ member functions difficult to generic programs? Does the same apply to Python methods? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu May 30 21:56:22 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 16:56:22 -0400 Subject: [Python-Dev] Re: Deprecation In-Reply-To: Your message of "Thu, 30 May 2002 22:41:13 +0200." <20020530224113.N17248@prim.han.de> References: <20020529172001.39760.qmail@web9601.mail.yahoo.com> <20020529202959.H17248@prim.han.de> <20020530121309.A8344@eecs.tufts.edu> <200205301631.g4UGVxZ24485@odiug.zope.com> <20020530124329.B8344@eecs.tufts.edu> <200205301706.g4UH65324697@odiug.zope.com> <20020530224113.N17248@prim.han.de> Message-ID: <200205302056.g4UKuNZ27472@odiug.zope.com> > But i assume, you do agree that a *working* versioning and > dependency scheme for the standard-lib would be more than > nice. (IMHO the more modules and versions we have the more important > it gets). Depends on how you define "working". I'm skeptical that such a thing can exist. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu May 30 21:58:22 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 16:58:22 -0400 Subject: [Python-Dev] Re: Deprecation In-Reply-To: Your message of "Thu, 30 May 2002 22:28:21 +0200." <20020530222821.M17248@prim.han.de> References: <20020529172001.39760.qmail@web9601.mail.yahoo.com> <20020529202959.H17248@prim.han.de> <20020530121309.A8344@eecs.tufts.edu> <20020530222821.M17248@prim.han.de> Message-ID: <200205302058.g4UKwMA27491@odiug.zope.com> > while i agree to this basic idea i wanted to remind of the other > point buried in my mail ... Changing the deprecation process to be based > on *version numbers* rather than *uncertain future points in time*... -1. It's hard to predict how often versions will be released. The PEPs (4 and 5) specify time limits, which are better defined. --Guido van Rossum (home page: http://www.python.org/~guido/) From mgilfix@eecs.tufts.edu Thu May 30 22:23:55 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Thu, 30 May 2002 17:23:55 -0400 Subject: [Python-Dev] Re: Deprecation In-Reply-To: <200205302058.g4UKwMA27491@odiug.zope.com>; from guido@python.org on Thu, May 30, 2002 at 04:58:22PM -0400 References: <20020529172001.39760.qmail@web9601.mail.yahoo.com> <20020529202959.H17248@prim.han.de> <20020530121309.A8344@eecs.tufts.edu> <20020530222821.M17248@prim.han.de> <200205302058.g4UKwMA27491@odiug.zope.com> Message-ID: <20020530172355.G8344@eecs.tufts.edu> On Thu, May 30 @ 16:58, Guido van Rossum wrote: > > while i agree to this basic idea i wanted to remind of the other > > point buried in my mail ... Changing the deprecation process to be based > > on *version numbers* rather than *uncertain future points in time*... > > -1. It's hard to predict how often versions will be released. The > PEPs (4 and 5) specify time limits, which are better defined. The problem with this is the "relative to whom" question... Is the time limit better defined relative to the developers or users? No user should really be using bleeding edge CVS (without getting what he asked for) for any production quality thing. As far as they're concerned, version numbers are the atomic units of change. -- Mike -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From martin@v.loewis.de Thu May 30 22:35:08 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 30 May 2002 23:35:08 +0200 Subject: [Python-Dev] _PyImport_LoadDynamicModule questions In-Reply-To: <200205301218.g4UCIw107505@pcp742651pcs.reston01.va.comcast.net> References: <01cf01c2072a$b69cdeb0$6601a8c0@boostconsulting.com> <200205300117.g4U1HbN06229@pcp742651pcs.reston01.va.comcast.net> <200205301218.g4UCIw107505@pcp742651pcs.reston01.va.comcast.net> Message-ID: Guido van Rossum writes: > > However, I still think that people creating --enable-shared > > installations are misguided: You gain nothing (IMO), and you lose a > > number of benefits: > > Do you understand why people have always been asking for this? Are > they all misguided? It really is a FAQ (3.30). Why? People give various reasons: - (from #400938): "Using a shared library should have an advantage if you're running multiple instances of Python (be it standalone interpreter or embedded applications)." This is nonsense, of course: the interpreter executable is shared just as well. - libraries should be shared (405931). There is often no further rationale given, but I believe "... because that saves disk space" is the common implication. Given that /usr/local/bin/python would be the only application of libpythonxy.so on most installation, this rationale is questionable. - it simplifies embedding (with the variant "embedding is not possible without it"). Some people are simply not aware that a libpython.a is also installed. In 497102, James Henstridge argues that PyXPCOM mandates a shared libpython, as does gnome-vfs. He might have a case here, but I think a special-case shared library that exposes all Python symbols might be more appropriate. - on the same topic, the PostgreSQL documentation claims that you cannot build PL/Python without a shared libpython. They admit that it might work to use the static library, and that it is just the fault of their build system to not support this scenario: http://www.postgresql.org/idocs/index.php?plpython-install.html - For embedded applications, people also bring up "allows sharing at run-time" argument). In that case, it is factually true. However, even without a shared libpython, multiple copies of the same embedded executable (or shared library) will still share code. - The Windows port uses a python DLL. To summarize, it probably does have advantages for people who want to embed Python in applications that are themselves shared libraries. I think those advantages are outweighed by the problems people may get with a shared libpython, and who never want to embed Python. Just my 0.02EUR, Martin From pyth@devel.trillke.net Thu May 30 22:41:22 2002 From: pyth@devel.trillke.net (holger krekel) Date: Thu, 30 May 2002 23:41:22 +0200 Subject: [Python-Dev] Re: Deprecation In-Reply-To: <200205302058.g4UKwMA27491@odiug.zope.com>; from guido@python.org on Thu, May 30, 2002 at 04:58:22PM -0400 References: <20020529172001.39760.qmail@web9601.mail.yahoo.com> <20020529202959.H17248@prim.han.de> <20020530121309.A8344@eecs.tufts.edu> <20020530222821.M17248@prim.han.de> <200205302058.g4UKwMA27491@odiug.zope.com> Message-ID: <20020530234122.Q17248@prim.han.de> Guido van Rossum wrote: > > while i agree to this basic idea i wanted to remind of the other > > point buried in my mail ... Changing the deprecation process to be based > > on *version numbers* rather than *uncertain future points in time*... > > -1. It's hard to predict how often versions will be released. The > PEPs (4 and 5) specify time limits, which are better defined. ok then. how would you translate (from PEP4) "The deprecation warning will be added to the module one year after Python 2.3 is released, and the module will be removed one year after that." to an if-statement-check you want to implement *today*? Additionally, if the above was based on versions you could implement the *deprecation process* today, no need to remember at which date some PEP mentioned something which has been forgotten for three years anyway. Are we really supposed to remember release dates of python versions? I mean of course they are of great historic value, but ... holger From guido@python.org Thu May 30 22:51:00 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 17:51:00 -0400 Subject: [Python-Dev] _PyImport_LoadDynamicModule questions In-Reply-To: Your message of "30 May 2002 23:35:08 +0200." References: <01cf01c2072a$b69cdeb0$6601a8c0@boostconsulting.com> <200205300117.g4U1HbN06229@pcp742651pcs.reston01.va.comcast.net> <200205301218.g4UCIw107505@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200205302151.g4ULp0e27797@odiug.zope.com> > > > However, I still think that people creating --enable-shared > > > installations are misguided: You gain nothing (IMO), and you lose a > > > number of benefits: > > > > Do you understand why people have always been asking for this? Are > > they all misguided? It really is a FAQ (3.30). Why? > > People give various reasons: > > - (from #400938): "Using a shared library should have an advantage if > you're running multiple instances of Python (be it standalone > interpreter or embedded applications)." > This is nonsense, of course: the interpreter executable is shared > just as well. It's not nonsense if you have lots of *different* programs that embed the interpreter. If each has a static copy, those static copies aren't shared between different copies. > - libraries should be shared (405931). There is often no further > rationale given, but I believe "... because that saves disk space" > is the common implication. Given that /usr/local/bin/python would be > the only application of libpythonxy.so on most installation, this > rationale is questionable. Agreed. Disk space is cheap. > - it simplifies embedding (with the variant "embedding is not possible > without it"). Some people are simply not aware that a libpython.a is > also installed. In 497102, James Henstridge argues that PyXPCOM > mandates a shared libpython, as does gnome-vfs. He might have a case > here, but I think a special-case shared library that exposes all > Python symbols might be more appropriate. Possibly, but this could be countered with "if we're going to have to provide a shared library anyway, it might as well be the only one we offer." I don't understand why PyXPCOM needs a shared library, but it may point to something we'll hear more in the future -- shared libraries are more open to inspection. I also wonder if the ability to slip a (compatible) newer version of a shared library in might not make a good argument for using shared libraries. Use case: you have an app that embeds Python. The embedded Python version uses the same standard library location as the installed Python, say, Python 2.1.2. Now you upgrade to Python 2.1.3. The standard library is upgraded. (a) Wouldn't it be nice if the embedding app was automatically upgraded too (rather than having to relink it, which may be a pain if the source and objects were thrown away). (b) Some new feature might be added to the core (in a binary compatible way) that is used by some library module. If the embedding app uses that module, it will fail because the statically linked app still is Python 2.1.2. > - on the same topic, the PostgreSQL documentation claims that you > cannot build PL/Python without a shared libpython. They admit that > it might work to use the static library, and that it is just the > fault of their build system to not support this scenario: > http://www.postgresql.org/idocs/index.php?plpython-install.html I expect we'll see this more and more. After all, every library that comes with Linux is a shared lib (even / especially libc). > - For embedded applications, people also bring up "allows sharing at > run-time" argument). In that case, it is factually true. However, > even without a shared libpython, multiple copies of the same > embedded executable (or shared library) will still share code. See above. > - The Windows port uses a python DLL. > > To summarize, it probably does have advantages for people who want to > embed Python in applications that are themselves shared libraries. I > think those advantages are outweighed by the problems people may get > with a shared libpython, and who never want to embed Python. What problems (apart from the pain of getting this to build right on many platforms)? You mentioned a bit of a slowdown due to PIF code (probably not even measurable in pystone) and a slower startup due to a few stat calls. Note that all extensions are already shared libraries -- so the problems can't be too bad. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu May 30 22:53:43 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 17:53:43 -0400 Subject: [Python-Dev] Re: Deprecation In-Reply-To: Your message of "Thu, 30 May 2002 23:41:22 +0200." <20020530234122.Q17248@prim.han.de> References: <20020529172001.39760.qmail@web9601.mail.yahoo.com> <20020529202959.H17248@prim.han.de> <20020530121309.A8344@eecs.tufts.edu> <20020530222821.M17248@prim.han.de> <200205302058.g4UKwMA27491@odiug.zope.com> <20020530234122.Q17248@prim.han.de> Message-ID: <200205302153.g4ULrhB27812@odiug.zope.com> > ok then. how would you translate (from PEP4) > > "The deprecation warning will be added to the module > one year after Python 2.3 is released, and the > module will be removed one year after that." > > to an if-statement-check you want to implement *today*? I guess I don't see why the error message should contain an expiration date. It's enough to know that the feature will expire -- you can find more info in the docs. > Additionally, if the above was based on versions you > could implement the *deprecation process* today, no need > to remember at which date some PEP mentioned something > which has been forgotten for three years anyway. Define deprecation process. I don't think it's a good idea to try to automate this. I want a human to decide that a deprecated module is really ready for deletion. This involves many other factors besides the originally promised deprecation date. > Are we really supposed to remember release dates of > python versions? I mean of course they are of great > historic value, but ... Why would you need to? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu May 30 22:54:37 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 17:54:37 -0400 Subject: [Python-Dev] __module__ of newly-created extension classes In-Reply-To: Your message of "Thu, 30 May 2002 16:12:37 EDT." <020601c20816$b973c530$6601a8c0@boostconsulting.com> References: <00b401c207f0$5e92a710$6601a8c0@boostconsulting.com> <200205301742.g4UHg7j26440@odiug.zope.com> <020601c20816$b973c530$6601a8c0@boostconsulting.com> Message-ID: <200205302154.g4ULsb327843@odiug.zope.com> > Now I'm wondering if there isn't a better way for Python to get > ahold of the current module's __name__ in this case, if you don't > want to change what PyEval_GetGlobals() does. If you can rigorously define "current module" I can propose an API. --Guido van Rossum (home page: http://www.python.org/~guido/) From sholden@holdenweb.com Thu May 30 22:51:37 2002 From: sholden@holdenweb.com (Steve Holden) Date: Thu, 30 May 2002 17:51:37 -0400 Subject: [Python-Dev] Re: Deprecation References: <20020529172001.39760.qmail@web9601.mail.yahoo.com> <20020529202959.H17248@prim.han.de> <20020530121309.A8344@eecs.tufts.edu> <20020530222821.M17248@prim.han.de> <200205302058.g4UKwMA27491@odiug.zope.com> <20020530234122.Q17248@prim.han.de> Message-ID: <11ab01c20824$2f102fb0$6300000a@holdenweb.com> > Guido van Rossum wrote: > > > while i agree to this basic idea i wanted to remind of the other > > > point buried in my mail ... Changing the deprecation process to be based > > > on *version numbers* rather than *uncertain future points in time*... > > > > -1. It's hard to predict how often versions will be released. The > > PEPs (4 and 5) specify time limits, which are better defined. > [Holger Kregel]: > ok then. how would you translate (from PEP4) > > "The deprecation warning will be added to the module > one year after Python 2.3 is released, and the > module will be removed one year after that." > > to an if-statement-check you want to implement *today*? > > Additionally, if the above was based on versions you > could implement the *deprecation process* today, no need > to remember at which date some PEP mentioned something > which has been forgotten for three years anyway. > > Are we really supposed to remember release dates of > python versions? I mean of course they are of great > historic value, but ... > Well, personally I think it would be great if every release silently suppressed deprecation warnings for code older than the release. So it would only need to know its own release date. But I've made the suggestion before, so apparently nobody thinks this is a sensible way to achieve "silent deprecation". at-least-this-thread-taught-me-how-to-spell-deprecation-ly y'rs - steve ----------------------------------------------------------------------- Steve Holden http://www.holdenweb.com/ Python Web Programming http://pydish.holdenweb.com/pwp/ ----------------------------------------------------------------------- From David Abrahams" <200205301742.g4UHg7j26440@odiug.zope.com> <020601c20816$b973c530$6601a8c0@boostconsulting.com> <200205302154.g4ULsb327843@odiug.zope.com> Message-ID: <02fe01c20825$aa72af10$6601a8c0@boostconsulting.com> From: "Guido van Rossum" > > Now I'm wondering if there isn't a better way for Python to get > > ahold of the current module's __name__ in this case, if you don't > > want to change what PyEval_GetGlobals() does. > > If you can rigorously define "current module" I can propose an API. A rigorous definition is easy; I can't promise it would always meet everyone's expectations: The "current module" is given by the current formula (PyEval_GetGlobals(), etc.) except when loading an extension module, in which case the current module is the one most recently created. I'm not sure, but this might be simplifiable to: the "current module" is the one most recently created. -Dave From martin@v.loewis.de Thu May 30 23:05:44 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 31 May 2002 00:05:44 +0200 Subject: [Python-Dev] String module In-Reply-To: <200205301835.g4UIZQv26785@odiug.zope.com> References: <001501c207fc$2573e320$4d66accf@othello> <200205301704.g4UH4fo24680@odiug.zope.com> <004201c20804$0a7ab320$4d66accf@othello> <200205301835.g4UIZQv26785@odiug.zope.com> Message-ID: Guido van Rossum writes: > Thanks! But now we have a diverging set of isxxx methods for 8-bit > strings and Unicode. I really don't know what the equivalent of these > (ispunct, iscntrl, isgraph, isprint) is in Unicode -- maybe MAL or MvL > know? I don't think there is an "official" mapping between these categories and Unicode character categories. I believe an "intuitive" relationship would be: ispunct: Punctuation (Pc, Pd, Ps, Pe, Pi, Pf, Po) iscntrl: Other, control (Cc); perhaps other Other isprint: Letters (L*), Marks (M*), Numbers (N*), Separators (Z*), perhaps informative categories (Symbol, Punctuation) isgraph: everything isprint, except Separators Another approach is to use the classification found in other libraries, such as Qt, Perl, or Win32 (GetStringTypeW). Marcin Kowalczyk presented his intuition in http://mail.nl.linux.org/linux-utf8/2000-09/msg00076.html but some of his classification was challenged later on; I guess glibc would be just another library to draw classificiations from. > Unicode also has a wider definition of digits; do we want to > extend isxdigit() for that? (Probably not, but I'm not sure.) Certainly not. We have to remember the common use for these, which is in computer stuff. There, hexdigit is 0..9{a..f|A..F}. Regards, Martin From kbutler@campuspipeline.com Thu May 30 23:23:44 2002 From: kbutler@campuspipeline.com (Kevin Butler) Date: Thu, 30 May 2002 16:23:44 -0600 Subject: [Python-Dev] Single .py file contains submodule? References: <20020530203306.5918.33552.Mailman@mail.python.org> Message-ID: <3CF6A670.1000500@campuspipeline.com> Guido wrote: > (Barry prefers that there's only one class per file; fortunately I > don't have that hangup. :-) So how about a single .py file containing a module and one-or-more submodules? (Sounds like Barry will definitely not like it... ;-) ) Specifically, I wanted to allow easy isolated import of the PyUnit assertion functions, something like: from unittest.assertions import * There are a few ways of refactoring unittest.py to support this, but part of the beauty of PyUnit is the single, independent file. Consulting the Python Reference Manual, I couldn't find anything that forbade a single file from containing submodules, so I came up with the following implementation, which works in Jython 2.1 and Python 2.2: import new import sys import types class Assertions: failureException = AssertionError def failUnless(self, expr, msg=None): """Fail the test unless the expression is true.""" if not expr: raise self.failureException, msg #... def __publishAsModule( name, dict ): submodule = new.module( name ) submodule.__dict__.update( dict ) sys.modules[ name ] = submodule return submodule def __getMethods( instance ): methods = {} for name, member in instance.__class__.__dict__.items(): if type( member ) is types.FunctionType: methods[name] = getattr( instance, name ) return methods assertions = __publishAsModule( __name__ + ".assertions", __getMethods( Assertions() ) ) So, is this an egregious abuse of a regrettable oversight in the specification, or a reasonable way to publish multiple modules from a single .py file? Thanks kb From pyth@devel.trillke.net Thu May 30 23:34:21 2002 From: pyth@devel.trillke.net (holger krekel) Date: Fri, 31 May 2002 00:34:21 +0200 Subject: [Python-Dev] Re: Deprecation In-Reply-To: <200205302153.g4ULrhB27812@odiug.zope.com>; from guido@python.org on Thu, May 30, 2002 at 05:53:43PM -0400 References: <20020529172001.39760.qmail@web9601.mail.yahoo.com> <20020529202959.H17248@prim.han.de> <20020530121309.A8344@eecs.tufts.edu> <20020530222821.M17248@prim.han.de> <200205302058.g4UKwMA27491@odiug.zope.com> <20020530234122.Q17248@prim.han.de> <200205302153.g4ULrhB27812@odiug.zope.com> Message-ID: <20020531003421.R17248@prim.han.de> Guido van Rossum wrote: > > ok then. how would you translate (from PEP4) > > > > "The deprecation warning will be added to the module > > one year after Python 2.3 is released, and the > > module will be removed one year after that." > > > > to an if-statement-check you want to implement *today*? > > I guess I don't see why the error message should contain an expiration > date. It's enough to know that the feature will expire -- you can > find more info in the docs. You may want to check that the python-version your program is running on might not have a certain feature or library any more. Just seeing an 'ImportError' leaves the user puzzled if he e.g. might have an installation problem. > > Additionally, if the above was based on versions you > > could implement the *deprecation process* today, no need > > to remember at which date some PEP mentioned something > > which has been forgotten for three years anyway. > > Define deprecation process. the citation above is a typical example for a 'deprecation process'. OK, you are a mathematician. i try harder... A deprecation process may be seen as an algorithm which determines the life time (or die-time?) of features and modules. > I don't think it's a good idea to try to > automate this. I want a human to decide that a deprecated module is > really ready for deletion. This involves many other factors besides > the originally promised deprecation date. automation does not mean there is no human intervention possible. It would be horrible indeed if computers would take over python development, no flaming wars, less and less more general modules ... :-) Anyway, removing a module would certainly still require human intervention but its *intended* deprecation/removal could be specified precisely. If a feature lives longer due to gods mercy than this is pure luck. You can hope for it but should not rely on it. I just happen to think that people and programs can work much better with 'version 2.5 or 3.0' than 'some years after some point in time' . time-less-ly yours, holger P.S: another solution is to found a Deprecation-SIG which would find many many friends, i bet. The only SIG that would have no expiration date :-) From goodger@users.sourceforge.net Thu May 30 23:30:51 2002 From: goodger@users.sourceforge.net (David Goodger) Date: Thu, 30 May 2002 18:30:51 -0400 Subject: [Python-Dev] Re: Adding Optik to the standard library In-Reply-To: <200205301946.g4UJkLL26987@odiug.zope.com> Message-ID: [Guido] >>> from OptionParser import OptionParser >>> >>> I like this! [Greg] >> I think the BDFL has spoken. I can live with this, although I prefer >> lower-case module names. Whatever. [Guido] > I prefer lower-case module names for most situations, but I make an > exception for modulename == classname. But there's more than just the one class (OptionParser) in the module, and the other classes (Option, Values) *are* used. Barry's rule may hold for 1 class per module; that's not the case here. +1 for "options" -1 for "OptionParser" -- David Goodger Open-source projects: - Python Docutils: http://docutils.sourceforge.net/ (includes reStructuredText: http://docutils.sf.net/rst.html) - The Go Tools Project: http://gotools.sourceforge.net/ From martin@v.loewis.de Thu May 30 23:41:25 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 31 May 2002 00:41:25 +0200 Subject: [Python-Dev] _PyImport_LoadDynamicModule questions In-Reply-To: <200205302151.g4ULp0e27797@odiug.zope.com> References: <01cf01c2072a$b69cdeb0$6601a8c0@boostconsulting.com> <200205300117.g4U1HbN06229@pcp742651pcs.reston01.va.comcast.net> <200205301218.g4UCIw107505@pcp742651pcs.reston01.va.comcast.net> <200205302151.g4ULp0e27797@odiug.zope.com> Message-ID: Guido van Rossum writes: > What problems (apart from the pain of getting this to build right on > many platforms)? Building it is not the issue; running it is the problem. /usr/local/lib is searched for shared libraries only on Linux, on all other systems, you either have to add a -R option, or require users to set LD_LIBRARY_PATH. The latter is clear undesirable, so you have to hard-code the path to libpython into the executable. In turn, the resulting binary is not relocatable anymore. > You mentioned a bit of a slowdown due to PIF code > (probably not even measurable in pystone) and a slower startup due to > a few stat calls. Note that all extensions are already shared > libraries -- so the problems can't be too bad. You also get a slow-down from the -R option (if you needed to use it in the first place). This will cause all library searches without a path to look into an additional directory. Regards, Martin From David Abrahams" <026101c20818$d36e3e50$6601a8c0@boostconsulting.com> <200205302053.g4UKriU27444@odiug.zope.com> Message-ID: <02d001c2081e$1fbf5870$6601a8c0@boostconsulting.com> From: "Guido van Rossum" > Is this still relevant to Python? Possibly. > Why are C++ member functions difficult to generic programs? 1. built-in types don't have member functions, so there's no way to get them to work like something that does. For that reason, when a generic function relies on member functions, it's "not very generic". 2. By the same token, if a generic function requires that a type have particular member functions, some class types may be ineligible for use with that function. If the requirement is simply that the appropriate free functions exist, it's usually possible to write them. This becomes a factor when using generic library A with a type from library B. > Does the same apply to Python methods? #1 obviously doesn't, except that of course the built-in types have a fixed set of methods. #2 might apply. The general trend towards decoupling algorithms and data structures in C++ is one of those Good Things (tm). -Dave From martin@v.loewis.de Thu May 30 23:49:30 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 31 May 2002 00:49:30 +0200 Subject: [Python-Dev] deprecating string module? In-Reply-To: <200205302053.g4UKriU27444@odiug.zope.com> References: <20020530160134.10173.qmail@web9603.mail.yahoo.com> <026101c20818$d36e3e50$6601a8c0@boostconsulting.com> <200205302053.g4UKriU27444@odiug.zope.com> Message-ID: Guido van Rossum writes: > Is this still relevant to Python? Why are C++ member functions > difficult to generic programs? Does the same apply to Python methods? In a generic algorithm foo, you can write def foo1(x): bar(x) if you have global functions. With methods, you need to write def foo1(x): x.bar() which means that bar must be a method of x. This might be difficult to achieve if x is of a class that you cannot control. In C++, it is then still possible to define a function def bar(x of-type-X): pass which, by means of overload resolution, will be found from foo1 automatically. In Python, this is not so easy since you don't have overloading. Regards, Martin From greg@cosc.canterbury.ac.nz Fri May 31 01:39:38 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Fri, 31 May 2002 12:39:38 +1200 (NZST) Subject: [Python-Dev] deprecating string module? In-Reply-To: <200205301255.g4UCtQX07833@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200205310039.MAA06825@s454.cosc.canterbury.ac.nz> Guido van Rossum : > > This is funny. In the C++ community there's a nearly unanimous > > consensus that way too much of the functionality of the standard > > strings is expressed as member functions. > > Interesting. Python used to have the same attitude, hence the string > module -- but the existence of multiple string types made methods more > attractive. I don't understand how using methods makes multiple string types easier to deal with. Suppose I want to define my own string-like type. How do I arrange things so that " ".join(x) does the right thing when x contains instances of my new type? How is this made easier by the fact that join is a method of " " rather than a free function? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From guido@python.org Fri May 31 01:51:11 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 20:51:11 -0400 Subject: [Python-Dev] Re: Deprecation In-Reply-To: Your message of "Fri, 31 May 2002 00:34:21 +0200." <20020531003421.R17248@prim.han.de> References: <20020529172001.39760.qmail@web9601.mail.yahoo.com> <20020529202959.H17248@prim.han.de> <20020530121309.A8344@eecs.tufts.edu> <20020530222821.M17248@prim.han.de> <200205302058.g4UKwMA27491@odiug.zope.com> <20020530234122.Q17248@prim.han.de> <200205302153.g4ULrhB27812@odiug.zope.com> <20020531003421.R17248@prim.han.de> Message-ID: <200205310051.g4V0pBF08399@pcp742651pcs.reston01.va.comcast.net> > > I guess I don't see why the error message should contain an > > expiration date. It's enough to know that the feature will expire > > -- you can find more info in the docs. > > You may want to check that the python-version your program is > running on might not have a certain feature or library any > more. Just seeing an 'ImportError' leaves the user puzzled if he > e.g. might have an installation problem. Yes, they hve an installation problem. :-) They are using a Python version that's incompatible with the program. I really don't see what you are proposing to do about this. The author of the program should know that it uses a deprecated module and fix the program, rather than simply continuing to distribute it with out change. If the program is very old, the user who downloads it should realize that it might no longer work. > > > Additionally, if the above was based on versions you > > > could implement the *deprecation process* today, no need > > > to remember at which date some PEP mentioned something > > > which has been forgotten for three years anyway. > > > > Define deprecation process. > > the citation above is a typical example for a 'deprecation process'. > OK, you are a mathematician. i try harder... > > A deprecation process may be seen as an algorithm which determines > the life time (or die-time?) of features and modules. I think this cannot and should not be determined algorithmically. > > I don't think it's a good idea to try to > > automate this. I want a human to decide that a deprecated module is > > really ready for deletion. This involves many other factors besides > > the originally promised deprecation date. > > automation does not mean there is no human intervention possible. > It would be horrible indeed if computers would take over python > development, no flaming wars, less and less more general modules ... :-) > > Anyway, removing a module would certainly still require > human intervention but its *intended* deprecation/removal > could be specified precisely. If a feature lives longer > due to gods mercy than this is pure luck. You can > hope for it but should not rely on it. > > I just happen to think that people and programs can work much > better with 'version 2.5 or 3.0' than 'some years after some > point in time' . I don't see the point. Let's end this discussion. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri May 31 01:53:51 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 20:53:51 -0400 Subject: [Python-Dev] Single .py file contains submodule? In-Reply-To: Your message of "Thu, 30 May 2002 16:23:44 MDT." <3CF6A670.1000500@campuspipeline.com> References: <20020530203306.5918.33552.Mailman@mail.python.org> <3CF6A670.1000500@campuspipeline.com> Message-ID: <200205310053.g4V0rpa08421@pcp742651pcs.reston01.va.comcast.net> > So, is this an egregious abuse of a regrettable oversight in the > specification, or a reasonable way to publish multiple modules from > a single .py file? I think it's much better to split it up in multiple files than to use all those hacks. Those hacks are hard to understand for someone trying to read the code. --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Fri May 31 01:58:33 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 30 May 2002 20:58:33 -0400 Subject: [Python-Dev] Single .py file contains submodule? References: <20020530203306.5918.33552.Mailman@mail.python.org> <3CF6A670.1000500@campuspipeline.com> Message-ID: <15606.51897.246906.165561@anthem.wooz.org> >>>>> "KB" == Kevin Butler writes: KB> So how about a single .py file containing a module and KB> one-or-more submodules? (Sounds like Barry will definitely not KB> like it... ;-) ) Maybe I would, if I understood what that meant ;). [...FAST code omitted...] KB> So, is this an egregious abuse of a regrettable oversight in KB> the specification, or a reasonable way to publish multiple KB> modules from a single .py file? That's just sick. :) -Barry From barry@zope.com Fri May 31 02:02:21 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 30 May 2002 21:02:21 -0400 Subject: [Python-Dev] Re: Adding Optik to the standard library References: <200205301946.g4UJkLL26987@odiug.zope.com> Message-ID: <15606.52125.709788.262269@anthem.wooz.org> >>>>> "DG" == David Goodger writes: DG> But there's more than just the one class (OptionParser) in the DG> module, and the other classes (Option, Values) *are* used. DG> Barry's rule may hold for 1 class per module; that's not the DG> case here. If that's so, then I'd prefer to see each class in its own module inside a parent package. | +1 for "options" | -1 for "OptionParser" I still think getopt-as-package could be made to work, but I'd be fine with `options'. Whatever though; Guido's weighed in and Greg should just decide and go for it! -Barry From goodger@users.sourceforge.net Fri May 31 02:11:06 2002 From: goodger@users.sourceforge.net (David Goodger) Date: Thu, 30 May 2002 21:11:06 -0400 Subject: [Python-Dev] Re: Adding Optik to the standard library In-Reply-To: <15606.52125.709788.262269@anthem.wooz.org> Message-ID: Barry A. Warsaw wrote: > If that's so, then I'd prefer to see each class in its own module > inside a parent package. We all know *your* bias! ;-) From guido@python.org Fri May 31 02:20:17 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 21:20:17 -0400 Subject: [Python-Dev] __module__ of newly-created extension classes In-Reply-To: Your message of "Thu, 30 May 2002 18:00:37 EDT." <02fe01c20825$aa72af10$6601a8c0@boostconsulting.com> References: <00b401c207f0$5e92a710$6601a8c0@boostconsulting.com> <200205301742.g4UHg7j26440@odiug.zope.com> <020601c20816$b973c530$6601a8c0@boostconsulting.com> <200205302154.g4ULsb327843@odiug.zope.com> <02fe01c20825$aa72af10$6601a8c0@boostconsulting.com> Message-ID: <200205310120.g4V1KH208510@pcp742651pcs.reston01.va.comcast.net> > > If you can rigorously define "current module" I can propose an API. > > A rigorous definition is easy; I can't promise it would always meet > everyone's expectations: > > The "current module" is given by the current formula (PyEval_GetGlobals(), > etc.) except when loading an extension module, in which case the current > module is the one most recently created. If the init function of an extension module calls PyImport_Import() to import another extension which has to be loaded freshly, does that mean that after that point the definition current module is left to that other module, or does it revert to the first extension? What if an extension imports a Python module which loads an extension module? > I'm not sure, but this might be simplifiable to: > > the "current module" is the one most recently created. No, because in a Python function defined in a Python module, when that function is executing that module is the current module. A possible implementation could maintain a per-thread global which is NULL when we're not loading an extension. Py_InitModule() sets this global to the module it creates. When the import mechanism calls an extension's initxxx function, it saves the value of this per-thread global; when the function returns, it restores it from the saved value. Then there could be a new function Py_GetGlobals() that looks in this per-thread global, and returns its dict if it is not NULL; if NULL, it falls back on PyEval_GetGlobals(). (You can't promise to return the current *module*, because that isn't kept track of; only the current globals are kept track of on the execution stack.) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri May 31 02:30:56 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 21:30:56 -0400 Subject: [Python-Dev] Re: Deprecation In-Reply-To: Your message of "Thu, 30 May 2002 17:51:37 EDT." <11ab01c20824$2f102fb0$6300000a@holdenweb.com> References: <20020529172001.39760.qmail@web9601.mail.yahoo.com> <20020529202959.H17248@prim.han.de> <20020530121309.A8344@eecs.tufts.edu> <20020530222821.M17248@prim.han.de> <200205302058.g4UKwMA27491@odiug.zope.com> <20020530234122.Q17248@prim.han.de> <11ab01c20824$2f102fb0$6300000a@holdenweb.com> Message-ID: <200205310130.g4V1UuR08583@pcp742651pcs.reston01.va.comcast.net> > Well, personally I think it would be great if every release silently > suppressed deprecation warnings for code older than the release. So > it would only need to know its own release date. But I've made the > suggestion before, so apparently nobody thinks this is a sensible > way to achieve "silent deprecation". I think it would defeat the purpose. Plus, how do you determine the age? --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Fri May 31 02:25:47 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 30 May 2002 21:25:47 -0400 Subject: [Python-Dev] Re: Adding Optik to the standard library References: <15606.52125.709788.262269@anthem.wooz.org> Message-ID: <15606.53531.845239.494430@anthem.wooz.org> From guido@python.org Fri May 31 02:32:01 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 30 May 2002 21:32:01 -0400 Subject: [Python-Dev] Re: Adding Optik to the standard library In-Reply-To: Your message of "Thu, 30 May 2002 21:02:21 EDT." <15606.52125.709788.262269@anthem.wooz.org> References: <200205301946.g4UJkLL26987@odiug.zope.com> <15606.52125.709788.262269@anthem.wooz.org> Message-ID: <200205310132.g4V1W1U08610@pcp742651pcs.reston01.va.comcast.net> > | +1 for "options" > | -1 for "OptionParser" > > I still think getopt-as-package could be made to work, but I'd be fine > with `options'. Whatever though; Guido's weighed in and Greg should > just decide and go for it! getopt-as-package is definitely out. I'll leave it to Greg what to make of the remaining two alternatives (options or OptionParser). --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Fri May 31 02:28:07 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 30 May 2002 21:28:07 -0400 Subject: [Python-Dev] Re: Adding Optik to the standard library References: <15606.52125.709788.262269@anthem.wooz.org> Message-ID: <15606.53671.50475.599022@anthem.wooz.org> >>>>> "DG" == David Goodger writes: DG> We all know *your* bias! ;-) You've sat next to me at a Python conference then? My biological interference with acoustic systems is soooo embarrassing. http://www.acronymfinder.com/af-query.asp?p=dict&String=exact&Acronym=BIAS -Barry From skip@pobox.com Fri May 31 02:12:46 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 30 May 2002 20:12:46 -0500 Subject: [Python-Dev] Distutils confuses Berkeley DB and dbm? Message-ID: <15606.52750.844752.46789@12-248-41-177.client.attbi.com> Would someone on Linux please try the following: import dbm f = dbm.open("foo", "c") f["1"] = "1" f.close then ask the file command what kind of file it is. On my system it tells me the file is a Berkeley DB 1.85 hash file. I figure that distutils is getting ahold of the libdb dbm-compatibility include files and libraries and using them. On my system, the only ndbm.h is in /usr/include/db1. Here's how the dbm module builds on my system: building 'dbm' extension gcc -DNDEBUG -O3 -fPIC -I. -I/home/skip/src/python/head/dist/src/./Include -I/usr/local/include \ -IInclude/ -c /home/skip/src/python/head/dist/src/Modules/dbmmodule.c -o \ build/temp.linux-i686-2.3/dbmmodule.o gcc -shared build/temp.linux-i686-2.3/dbmmodule.o -L/usr/local/lib -ldb1 \ -o build/lib.linux-i686-2.3/dbm.so I know there is a bug report open about something related to building Berkeley DB-based modules, but I'm offline at the moment so I can't check it. Barry, were you going to look at this? How about a little collaboration? I think I'm mostly responsible for what distutils does as far as building bsddb. Skip From pobrien@orbtech.com Fri May 31 03:40:45 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Thu, 30 May 2002 21:40:45 -0500 Subject: [Python-Dev] String module In-Reply-To: <001701c20812$fb818c40$95d8accf@othello> Message-ID: [Raymond Hettinger] > While we're at it: > isgraph() --> isvisible() > iscntrl() --> iscontrol() > isprint() --> isprintable() > > I'm sure everyone will have an opinion or two. +1 on the (only slightly) longer versions of these names. --- Patrick K. O'Brien Orbtech From skip@pobox.com Fri May 31 03:51:20 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 30 May 2002 21:51:20 -0500 Subject: [Python-Dev] Re: Distutils confuses Berkeley DB and dbm? Message-ID: <15606.58664.397184.62014@12-248-41-177.client.attbi.com> Okay, after a little investigation (about 30 seconds staring at setup.py), I see where the problem with dbm is. Setup.py has this code to deal with dbm: # The standard Unix dbm module: if platform not in ['cygwin']: if (self.compiler.find_library_file(lib_dirs, 'ndbm')): exts.append( Extension('dbm', ['dbmmodule.c'], libraries = ['ndbm'] ) ) elif self.compiler.find_library_file(lib_dirs, 'db1'): exts.append( Extension('dbm', ['dbmmodule.c'], libraries = ['db1'] ) ) else: exts.append( Extension('dbm', ['dbmmodule.c']) ) That second branch is just plain wrong. (I realize Neil added it in response to a bug report.) While it's true that Berkeley DB does (or did) provide a dbm API, the underlying file is a db1 hash file, which we all should realize by now is horribly broken, and if you've got a workable bsddb there's no sense in trying to trick programmers into thinking they also have real dbm files. I suggest we discard the elif branch. Any argument? Skip From David Abrahams" <200205301742.g4UHg7j26440@odiug.zope.com> <020601c20816$b973c530$6601a8c0@boostconsulting.com> <200205302154.g4ULsb327843@odiug.zope.com> <02fe01c20825$aa72af10$6601a8c0@boostconsulting.com> <200205310120.g4V1KH208510@pcp742651pcs.reston01.va.comcast.net> Message-ID: <03bc01c2084f$27a6da00$6601a8c0@boostconsulting.com> From: "Guido van Rossum" > > The "current module" is given by the current formula (PyEval_GetGlobals(), > > etc.) except when loading an extension module, in which case the current > > module is the one most recently created. > > If the init function of an extension module calls PyImport_Import() to > import another extension which has to be loaded freshly, does that > mean that after that point the definition current module is left to > that other module, or does it revert to the first extension? It reverts of course (glad you asked)! > What if an extension imports a Python module which loads an extension > module? Of course there's a similar reversion after the extension module finishes loading. > > I'm not sure, but this might be simplifiable to: > > > > the "current module" is the one most recently created. > > No, because in a Python function defined in a Python module, when that > function is executing that module is the current module. > > A possible implementation could maintain a per-thread global which is > NULL when we're not loading an extension. Py_InitModule() sets this > global to the module it creates. When the import mechanism calls an > extension's initxxx function, it saves the value of this per-thread > global; when the function returns, it restores it from the saved > value. Then there could be a new function Py_GetGlobals() that looks > in this per-thread global, and returns its dict if it is not NULL; if > NULL, it falls back on PyEval_GetGlobals(). (You can't promise to > return the current *module*, because that isn't kept track of; only > the current globals are kept track of on the execution stack.) Sounds pretty good to me. -Dave From pobrien@orbtech.com Fri May 31 04:10:46 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Thu, 30 May 2002 22:10:46 -0500 Subject: [Python-Dev] Re: Adding Optik to the standard library In-Reply-To: <15606.52125.709788.262269@anthem.wooz.org> Message-ID: [Barry A. Warsaw] > If that's so, then I'd prefer to see each class in its own module > inside a parent package. Without trying to open a can of worms here, is there any sort of consensus on the use of packages with multiple smaller modules vs. one module containing everything? I'm asking about the Python standard library, specifically. According to the one-class-per-module rule of thumb, there are some Python modules that could be refactored into packages. Weighing against that is the convenience of importing a single module. I'm just wondering if there are any guidelines that should frame one's thinking beyond the fairly obvious ones? For example, is the standard library an exceptional case because it must appeal to new users as well as experts? Does a good part of this issue come down to personal preference? Or are there advantages and disadvantages that should be documented? (Maybe they already have.) Is the current library configuration considered healthy? There are a mix of packages and single modules. Are these implementations pretty optimal, or would they be organized differently if one had the chance to do it all over again? Just curious. --- Patrick K. O'Brien Orbtech From andymac@bullseye.apana.org.au Thu May 30 22:55:57 2002 From: andymac@bullseye.apana.org.au (Andrew MacIntyre) Date: Fri, 31 May 2002 08:55:57 +1100 (edt) Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test test_signal.py,1.9,1.10 In-Reply-To: <2melfthb9r.fsf@starship.python.net> Message-ID: On 30 May 2002, Michael Hudson wrote: > > Python doesn't create any threads. On Linux, I know that when you > > start your first thread, the thread library creates an extra thread > > for some internal reasons. Who knows what BSD does though. > > I'm not sure either, but I have convinced myself that signal mask > handling is just buggered on BSD when the program is compiled in a > multi-threaded style (as, in simple C programs don't do what you > (well, I) would expect). Note this isn't about actually using threads > -- just using "cc -pthreads". > > Now what do I do? Back my patch out? Not expose the functions on > BSD? It works on Linux... > > Cheers, > M. Can you forward me your simple C test case and a description of what you expect and what you get? I can take the issue up on the FreeBSD lists and see if I can get any clarifications. If I don't get anything back, then I guess we'll have to try not exposing the extra stuff on FreeBSD at least. Have you been able to test on Solaris or NetBSD/OpenBSD? (I'm assuming these are available in the SF compile farm). -- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au | Snail: PO Box 370 andymac@pcug.org.au | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From guido@python.org Fri May 31 05:35:22 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 31 May 2002 00:35:22 -0400 Subject: [Python-Dev] deprecating string module? In-Reply-To: Your message of "Fri, 31 May 2002 12:39:38 +1200." <200205310039.MAA06825@s454.cosc.canterbury.ac.nz> References: <200205310039.MAA06825@s454.cosc.canterbury.ac.nz> Message-ID: <200205310435.g4V4ZMr08785@pcp742651pcs.reston01.va.comcast.net> > I don't understand how using methods makes multiple > string types easier to deal with. > > Suppose I want to define my own string-like type. How > do I arrange things so that " ".join(x) does the right > thing when x contains instances of my new type? How > is this made easier by the fact that join is a method > of " " rather than a free function? No solution in current Python deals really well with operations of two or more strings that may be of mixed types (the dispatch of binary operators comes closest). The current code checks for both standard string types to overcome this, and that doesn't work for a third, new string type. The point of methods was that homogeneous use (and all operations on a single string like s.lower() are homogeneous) is supported well. Because Python has no dispatch on argument type, the alternative would have been to rewrite the string module to do argument type tests in *all* functions -- and it would not support a third string type at all, not even when used homogeneous. IOW, if I write a function that calls s.lower() for an argument s, I can write a custom string type (e.g. UserString) that will work with f. If I wrote the same function using string.lower(s), I have no hope (short of modifying the string module). --Guido van Rossum (home page: http://www.python.org/~guido/) From rnd@onego.ru Fri May 31 05:44:45 2002 From: rnd@onego.ru (Roman Suzi) Date: Fri, 31 May 2002 08:44:45 +0400 (MSD) Subject: [Python-Dev] String module In-Reply-To: Message-ID: On Thu, 30 May 2002, Patrick K. O'Brien wrote: >[Raymond Hettinger] >> While we're at it: >> isgraph() --> isvisible() >> iscntrl() --> iscontrol() >> isprint() --> isprintable() >> >> I'm sure everyone will have an opinion or two. > >+1 on the (only slightly) longer versions of these names. Whu not isgraph() --> is_visible() iscntrl() --> is_control() isprint() --> is_printable() so "is" is more... visible? >--- >Patrick K. O'Brien >Orbtech > > >_______________________________________________ >Python-Dev mailing list >Python-Dev@python.org >http://mail.python.org/mailman/listinfo/python-dev > Sincerely yours, Roman Suzi -- \_ Russia \_ Karelia \_ Petrozavodsk \_ rnd@onego.ru \_ \_ Thursday, May 30, 2002 \_ Powered by Linux RedHat 7.2 \_ \_ "How do I love thee? My accumulator overflows." \_ From guido@python.org Fri May 31 05:53:32 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 31 May 2002 00:53:32 -0400 Subject: [Python-Dev] Re: Adding Optik to the standard library In-Reply-To: Your message of "Thu, 30 May 2002 22:10:46 CDT." References: Message-ID: <200205310453.g4V4rWW09126@pcp742651pcs.reston01.va.comcast.net> > [Barry A. Warsaw] > > If that's so, then I'd prefer to see each class in its own module > > inside a parent package. > > Without trying to open a can of worms here, is there any sort of > consensus on the use of packages with multiple smaller modules > vs. one module containing everything? I'm asking about the Python > standard library, specifically. According to the > one-class-per-module rule of thumb, there are some Python modules > that could be refactored into packages. Weighing against that is the > convenience of importing a single module. Barry is the proponent of the one-class-per-module rule. I don't mind having more classes per module at all; even having the module named after the "dominant" class doesn't bother me. A single module containing several classes makes for shorter imports, e.g. you can write from damodule import DaClass rather than from dapackage.DaClass import DaClass I realize that you can put magic in dapackage's __init__.py that lets you write from dapackage import DaClass but it's not pretty, and if DaClass is still defined in a module DaClass.py inside dapackage, there's an unpleasant ambiguity: on the one hand dapackage.DaClass is a module, on the other hand it's a class! Barry's email package avoids the __init__ nastiness but you end up having to write the rather verbose from email.Message import Message > I'm just wondering if there are any guidelines that should frame > one's thinking beyond the fairly obvious ones? For example, is the > standard library an exceptional case because it must appeal to new > users as well as experts? Does a good part of this issue come down > to personal preference? Or are there advantages and disadvantages > that should be documented? (Maybe they already have.) The standard library grew organically and represents many points of view. > Is the current library configuration considered healthy? There are a > mix of packages and single modules. Are these implementations pretty > optimal, or would they be organized differently if one had the > chance to do it all over again? If I had to do it all over again, I'd probably organize it differently, but I don't see a point in attempting a massive reorganization -- it will only upset the users without real benefits (humans are pretty good at dealing with semi-disorganized data). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri May 31 05:55:08 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 31 May 2002 00:55:08 -0400 Subject: [Python-Dev] String module In-Reply-To: Your message of "Fri, 31 May 2002 08:44:45 +0400." References: Message-ID: <200205310455.g4V4t8O09160@pcp742651pcs.reston01.va.comcast.net> > Whu not > > isgraph() --> is_visible() > iscntrl() --> is_control() > isprint() --> is_printable() > > so "is" is more... visible? Please no -- we already have islower() etc. without the _. (And _ is a pain to type.) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri May 31 05:57:54 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 31 May 2002 00:57:54 -0400 Subject: [Python-Dev] Re: Distutils confuses Berkeley DB and dbm? In-Reply-To: Your message of "Thu, 30 May 2002 21:51:20 CDT." <15606.58664.397184.62014@12-248-41-177.client.attbi.com> References: <15606.58664.397184.62014@12-248-41-177.client.attbi.com> Message-ID: <200205310457.g4V4vsG09199@pcp742651pcs.reston01.va.comcast.net> > Okay, after a little investigation (about 30 seconds staring at > setup.py), I see where the problem with dbm is. Setup.py has this > code to deal with dbm: > > # The standard Unix dbm module: > if platform not in ['cygwin']: > if (self.compiler.find_library_file(lib_dirs, 'ndbm')): > exts.append( Extension('dbm', ['dbmmodule.c'], > libraries = ['ndbm'] ) ) > elif self.compiler.find_library_file(lib_dirs, 'db1'): > exts.append( Extension('dbm', ['dbmmodule.c'], > libraries = ['db1'] ) ) > else: > exts.append( Extension('dbm', ['dbmmodule.c']) ) > > That second branch is just plain wrong. (I realize Neil added it in > response to a bug report.) While it's true that Berkeley DB does > (or did) provide a dbm API, the underlying file is a db1 hash file, > which we all should realize by now is horribly broken, and if you've > got a workable bsddb there's no sense in trying to trick programmers > into thinking they also have real dbm files. I suggest we discard > the elif branch. Any argument? Sounds reasonable to me, but I'm no expert in this area. (I still have a pending bug report on why whichdb gets confused sometimes.) --Guido van Rossum (home page: http://www.python.org/~guido/) From rnd@onego.ru Fri May 31 05:54:58 2002 From: rnd@onego.ru (Roman Suzi) Date: Fri, 31 May 2002 08:54:58 +0400 (MSD) Subject: [Python-Dev] Re: Adding Optik to the standard library In-Reply-To: Message-ID: On Thu, 30 May 2002, Patrick K. O'Brien wrote: >Without trying to open a can of worms here, is there any sort of consensus >on the use of packages with multiple smaller modules vs. one module >containing everything? I'm asking about the Python standard library, >specifically. According to the one-class-per-module rule of thumb, there are >some Python modules that could be refactored into packages. Weighing against >that is the convenience of importing a single module. > >I'm just wondering if there are any guidelines that should frame one's >thinking beyond the fairly obvious ones? For example, is the standard >library an exceptional case because it must appeal to new users as well as >experts? Does a good part of this issue come down to personal preference? Or >are there advantages and disadvantages that should be documented? (Maybe >they already have.) > >Is the current library configuration considered healthy? There are a mix of >packages and single modules. Are these implementations pretty optimal, or >would they be organized differently if one had the chance to do it all over >again? Interesting question! Python Style Guide doesn't tell much about this. I think that if there is (more or less) natural hierarchy, then monolith module is better to partition into smaller ones. For example, os module. Even docs are mentioning 4 aspects of it. But only os.path is (kinda) separate. os.fd could have their own, as well as os.proc os.fs ... This way they are remembered better... xml-packages is another good example. Otherwise there is no point in adding hierarchy in standard libs. I think, modularizing is very similar to constructing good object class hierarchy. With the main fear - not to overdesign it. Sincerely yours, Roman Suzi -- \_ Russia \_ Karelia \_ Petrozavodsk \_ rnd@onego.ru \_ \_ Thursday, May 30, 2002 \_ Powered by Linux RedHat 7.2 \_ \_ "How do I love thee? My accumulator overflows." \_ From skip@pobox.com Fri May 31 06:23:42 2002 From: skip@pobox.com (Skip Montanaro) Date: Fri, 31 May 2002 00:23:42 -0500 Subject: [Python-Dev] Re: Distutils confuses Berkeley DB and dbm? In-Reply-To: <200205310457.g4V4vsG09199@pcp742651pcs.reston01.va.comcast.net> References: <15606.58664.397184.62014@12-248-41-177.client.attbi.com> <200205310457.g4V4vsG09199@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15607.2270.943362.835037@12-248-41-177.client.attbi.com> Guido> Sounds reasonable to me, but I'm no expert in this area. (I Guido> still have a pending bug report on why whichdb gets confused Guido> sometimes.) I've been looking at that as well. Whichdb isn't actually very smart and has a schizophrenic relationship with its input "filename". It "knows" that certain types of db libraries build files with certain extensions and it knows the magic numbers of a few others. If you ask it what kind of file "foo" is, it first concatenates certain file extensions to see if it can tell if it's a dbm or dumbdbm file. The original dbm (and I think ndbm) C libraries created "foo.pag" and "foo.dir" when asked to create a "foo" file. When you use Berkeley DB's dbm-compatible API it still creates a Berkeley DB file underneath the covers, so the filename sniff tests that would return "dbm" will fail. So it's on to the next step. It then tries to simply open the filename as given. This will fail if you call whichdb.whichdb("foo") if "foo" was created by Berkeley DB, because the actual file is "foo.db": >>> import whichdb >>> whichdb.whichdb("foo") >>> whichdb.whichdb("foo.db") 'dbhash' So whichdb is broken. It tries to treat its argument as both a file prefix and a file name. I don't think you can have it both ways. Fixing it properly will require one or the other of these interpretations of its input argument to disappear. Skip From barry@zope.com Fri May 31 06:33:41 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 31 May 2002 01:33:41 -0400 Subject: [Python-Dev] Re: Adding Optik to the standard library References: <15606.52125.709788.262269@anthem.wooz.org> Message-ID: <15607.2869.648570.950260@anthem.wooz.org> [Trimming recipients to just python-dev... BAW] >>>>> "PKO" == Patrick K O'Brien writes: PKO> Without trying to open a can of worms here, is there any sort PKO> of consensus on the use of packages with multiple smaller PKO> modules vs. one module containing everything? It's an interesting topic, to be sure, and no doubt will generate some nice cooked (charred?) worms. I'll only describe my own thoughts on the matter. I personally like files in bite-sized chunks, which means when they get to be more than one or a few tall emacs screenfuls, I start to get the urge to split things up. I'm probably somewhat influenced too by my early C++ days when we adopted a one class per .h file (and one class implementation per .cc file). IIRC, Objective-C also encouraged this granularity of organization. Even earlier influences include the FORTH convention of organizing everything into 1024 byte blocks, and that's it! For whatever reason, I definitely prefer to edit more smaller files than fewer large files (but all in moderation!), and the class seems to be a good organizational structure around which to split things. PKO> I'm just wondering if there are any guidelines that should PKO> frame one's thinking beyond the fairly obvious ones? For PKO> example, is the standard library an exceptional case because PKO> it must appeal to new users as well as experts? Does a good PKO> part of this issue come down to personal preference? Or are PKO> there advantages and disadvantages that should be documented? PKO> (Maybe they already have.) I think most of the standard modules are special because they were written before Python had a good (or any!) package system. For legacy modules that export lots of classes, there's probably little benefit to refactoring them into packages. That might change if we want to start separating things out into separately installable distutilized packages though. The package seems to be the smallest convenient unit for distutils. PKO> Is the current library configuration considered healthy? PKO> There are a mix of packages and single modules. Are these PKO> implementations pretty optimal, or would they be organized PKO> differently if one had the chance to do it all over again? Some of the newer packages are designed as packages because they're complex and big: distutils, xml, email, compiler. That makes perfect sense to me. It's possible something like Cookie.py if re-done /might/ make sense as a package, but I'm not sure. I doubt it makes much sense for something like smtplib.py to ever be a package, and besides other than exceptions, it only exports one main class. BTW, exceptions are...exceptions! I don't have much problem lumping all of a package's exception classes in one module. so-yeah-it-probably-is-personal-preference-ly y'rs, -Barry From barry@zope.com Fri May 31 06:50:01 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 31 May 2002 01:50:01 -0400 Subject: [Python-Dev] Distutils confuses Berkeley DB and dbm? References: <15606.52750.844752.46789@12-248-41-177.client.attbi.com> Message-ID: <15607.3849.191739.877131@anthem.wooz.org> >>>>> "SM" == Skip Montanaro writes: SM> Would someone on Linux please try the following: | import dbm | f = dbm.open("foo", "c") | f["1"] = "1" | f.close SM> then ask the file command what kind of file it is. On my SM> system it tells me the file is a Berkeley DB 1.85 hash file. On my RH6.1-ish system, foo.db is: Berkeley DB 2.X Hash/Little Endian (Version 5) On my stock, but loaded Mandrake 8.2 system, foo.db is: Berkeley DB (Hash, version 5, native byte-order) Note that my main problem with the bsddb module is building it. On the RH6.1 system I get: gcc -g -Wall -Wstrict-prototypes -fPIC -DHAVE_DB_185_H=1 -I. -I/home/barry/projects/python/./Include -I/usr/local/include -IInclude/ -c /home/barry/projects/python/Modules/bsddbmodule.c -o build/temp.linux-i686-2.3/bsddbmodule.o gcc -shared build/temp.linux-i686-2.3/bsddbmodule.o -L/usr/local/lib -ldb-3.1 -o build/lib.linux-i686-2.3/bsddb.so *** WARNING: renaming "bsddb" since importing it failed: build/lib.linux-i686-2.3/bsddb.so: undefined symbol: dbopen but on the Mandrake 8.2 system I get: gcc -g -Wall -Wstrict-prototypes -fPIC -DHAVE_DB_185_H=1 -I/usr/include/db3 -I. -I/home/barry/projects/python/./Include -I/usr/local/include -IInclude/ -c /home/barry/projects/python/Modules/bsddbmodule.c -o build/temp.linux-i686-2.3/bsddbmodule.o gcc -shared build/temp.linux-i686-2.3/bsddbmodule.o -L/usr/local/lib -ldb2 -o build/lib.linux-i686-2.3/bsddb.so *** WARNING: renaming "bsddb" since importing it failed: build/lib.linux-i686-2.3/bsddb.so: undefined symbol: __db185_open SM> I know there is a bug report open about something related to SM> building Berkeley DB-based modules, but I'm offline at the SM> moment so I can't check it. Barry, were you going to look at SM> this? How about a little collaboration? I think I'm mostly SM> responsible for what distutils does as far as building bsddb. I'm too tired to think about this more tonight, but if you want to hook up on irc.openprojects.net #python tomorrow, we can try to wade our way through things. -Barry From martin@v.loewis.de Fri May 31 08:06:58 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 31 May 2002 09:06:58 +0200 Subject: [Python-Dev] Distutils confuses Berkeley DB and dbm? In-Reply-To: <15606.52750.844752.46789@12-248-41-177.client.attbi.com> References: <15606.52750.844752.46789@12-248-41-177.client.attbi.com> Message-ID: Skip Montanaro writes: > Would someone on Linux please try the following: > > import dbm > f = dbm.open("foo", "c") > f["1"] = "1" > f.close > > then ask the file command what kind of file it is. On my system it tells me > the file is a Berkeley DB 1.85 hash file. On my SuSE 8.0 installation, using /usr/bin/python, it tells me foo.dir: GNU dbm 1.x or ndbm database, little endian foo.pag: GNU dbm 1.x or ndbm database, little endian Regards, Martin From mal@lemburg.com Fri May 31 08:38:31 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 31 May 2002 09:38:31 +0200 Subject: [Python-Dev] String module References: <001501c207fc$2573e320$4d66accf@othello> <200205301704.g4UH4fo24680@odiug.zope.com> <004201c20804$0a7ab320$4d66accf@othello> <200205301835.g4UIZQv26785@odiug.zope.com> <001701c20812$fb818c40$95d8accf@othello> <200205302010.g4UKAn927215@odiug.zope.com> Message-ID: <3CF72877.5020701@lemburg.com> Guido van Rossum wrote: >>>Someone commented that isxdigit is a poor name. OTOH it's what C >>>uses. I'm not sure what to say. >> >>I concur. I had to look it up on google to make sure in meant >>what I surmised it meant. ishexdigit() is more explicit. >>Besides, C naming conventions aren't exactly role models for clarity ;) >> >>While we're at it: >> isgraph() --> isvisible() >> iscntrl() --> iscontrol() >> isprint() --> isprintable() > > > Sure. But I still can't guess the relationships between these three > graph excludes space, print includes it). > > And then maybe also ispunctuation()? Or is that too long? > > My ctype.h man page also has these: > > isblank -- space or tab (GNU extension) > isascii -- 7-bit value (BSD/SVID extension) Do you really think this proliferation of issomething() methods is a good idea ? The same can be had using re.metch() with *much* more flexibility since you're not stuck with an "intuitive" definition relying on the intuition of some non-standard body. FWIW, I've never used a single one of these single character based classification APIs. The only non-trivial issomething() method I can think of is .istitle() because of it's complicated definition. All others can easily be had with re.match(). I'm -0 on adding more of these .issomething() APIs to strings and Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/ From mal@lemburg.com Fri May 31 08:42:38 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 31 May 2002 09:42:38 +0200 Subject: [Python-Dev] Single .py file contains submodule? References: <20020530203306.5918.33552.Mailman@mail.python.org> <3CF6A670.1000500@campuspipeline.com> Message-ID: <3CF7296E.1050103@lemburg.com> Kevin Butler wrote: > Guido wrote: > > (Barry prefers that there's only one class per file; fortunately I > > don't have that hangup. :-) > > So how about a single .py file containing a module and one-or-more > submodules? (Sounds like Barry will definitely not like it... ;-) ) > > Specifically, I wanted to allow easy isolated import of the PyUnit > assertion functions, something like: > > from unittest.assertions import * That's bad style (at least for modules which don't only include constants). Why would you want to enable this ? -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/ From mal@lemburg.com Fri May 31 08:45:05 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 31 May 2002 09:45:05 +0200 Subject: [Python-Dev] Distutils confuses Berkeley DB and dbm? References: <15606.52750.844752.46789@12-248-41-177.client.attbi.com> Message-ID: <3CF72A01.6030207@lemburg.com> Skip Montanaro wrote: > Would someone on Linux please try the following: > > import dbm > f = dbm.open("foo", "c") > f["1"] = "1" > f.close > > then ask the file command what kind of file it is. On my system it tells me > the file is a Berkeley DB 1.85 hash file. I figure that distutils is > getting ahold of the libdb dbm-compatibility include files and libraries and > using them. On my system, the only ndbm.h is in /usr/include/db1. Here's > how the dbm module builds on my system: > > building 'dbm' extension > gcc -DNDEBUG -O3 -fPIC > -I. -I/home/skip/src/python/head/dist/src/./Include -I/usr/local/include \ > -IInclude/ -c /home/skip/src/python/head/dist/src/Modules/dbmmodule.c -o \ > build/temp.linux-i686-2.3/dbmmodule.o > gcc -shared build/temp.linux-i686-2.3/dbmmodule.o -L/usr/local/lib -ldb1 \ > -o build/lib.linux-i686-2.3/dbm.so > > I know there is a bug report open about something related to building > Berkeley DB-based modules, but I'm offline at the moment so I can't check > it. Barry, were you going to look at this? How about a little > collaboration? I think I'm mostly responsible for what distutils does as > far as building bsddb. Perhaps you ought to have a look at mx.BeeBase ? It's portable and fast, has locks and transactions. (And it builds on all platforms where egenix-mx-base builds.) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/ From sholden@holdenweb.com Fri May 31 09:12:25 2002 From: sholden@holdenweb.com (Steve Holden) Date: Fri, 31 May 2002 04:12:25 -0400 Subject: [Python-Dev] Re: Deprecation References: <20020529172001.39760.qmail@web9601.mail.yahoo.com> <20020529202959.H17248@prim.han.de> <20020530121309.A8344@eecs.tufts.edu> <20020530222821.M17248@prim.han.de> <200205302058.g4UKwMA27491@odiug.zope.com> <20020530234122.Q17248@prim.han.de> <11ab01c20824$2f102fb0$6300000a@holdenweb.com> <200205310130.g4V1UuR08583@pcp742651pcs.reston01.va.comcast.net> Message-ID: <122601c2087a$e6448e00$6300000a@holdenweb.com> ----- Original Message ----- From: "Guido van Rossum" To: "Steve Holden" Cc: Sent: Thursday, May 30, 2002 9:30 PM Subject: Re: [Python-Dev] Re: Deprecation > > Well, personally I think it would be great if every release silently > > suppressed deprecation warnings for code older than the release. So > > it would only need to know its own release date. But I've made the > > suggestion before, so apparently nobody thinks this is a sensible > > way to achieve "silent deprecation". > > I think it would defeat the purpose. Plus, how do you determine the > age? > Clearly you'd have to goby the source file date, since a new release would recompile the .pyc's. The main advantage is that deprecation wornings wouldn't start appearing after the installation of a new release. The only advantage I see for it is the silence. Obviously it would eventually mean that programs died a sudden death due to feature loss. The warnings would appear if the code were maintained, however. regards ----------------------------------------------------------------------- Steve Holden http://www.holdenweb.com/ Python Web Programming http://pydish.holdenweb.com/pwp/ ----------------------------------------------------------------------- From fredrik@pythonware.com Fri May 31 09:19:18 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Fri, 31 May 2002 10:19:18 +0200 Subject: [Python-Dev] deprecating string module? References: <200205310039.MAA06825@s454.cosc.canterbury.ac.nz> <200205310435.g4V4ZMr08785@pcp742651pcs.reston01.va.comcast.net> Message-ID: <011b01c2087b$dd371750$0900a8c0@spiff> Guido van Rossum wrote: > IOW, if I write a function that calls s.lower() for an argument s, I > can write a custom string type (e.g. UserString) that will work with > f. If I wrote the same function using string.lower(s), I have no hope > (short of modifying the string module). unless the functions in the string module delegate everything to the object (just like len(), getattr(), and others), and you make sure that your type behaves well when mixed with basic strings. something like, say: def lower(s): return s.lower() or def capwords(s, sep=3DNone): return join(map(capitalize, s.split(sep)), sep or ' ') Cheers /F From pyth@devel.trillke.net Fri May 31 09:37:03 2002 From: pyth@devel.trillke.net (holger krekel) Date: Fri, 31 May 2002 10:37:03 +0200 Subject: [Python-Dev] Single .py file contains submodule? In-Reply-To: <3CF6A670.1000500@campuspipeline.com>; from kbutler@campuspipeline.com on Thu, May 30, 2002 at 04:23:44PM -0600 References: <20020530203306.5918.33552.Mailman@mail.python.org> <3CF6A670.1000500@campuspipeline.com> Message-ID: <20020531103703.S17248@prim.han.de> Kevin Butler wrote: > Guido wrote: > > (Barry prefers that there's only one class per file; fortunately I > > don't have that hangup. :-) > > So how about a single .py file containing a module and one-or-more submodules? I have been asking this myself, too. I definitely see an advantage beeing able to put modules and submodules in one file. IMO it's a logical structure IMO and it is easier to distribute. If you just want to send someone a module to work/test with (without requiring setup.py etc.pp.) a file is a nicer unit. The next better step involves creating a (distutils-) package. But i don't always want to create packages for a module. Maybe there is an elegant way to achieve this? I don't find Kevin's code too hard to understand, btw... holger From mwh@python.net Fri May 31 09:45:00 2002 From: mwh@python.net (Michael Hudson) Date: 31 May 2002 09:45:00 +0100 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test test_signal.py,1.9,1.10 In-Reply-To: martin@v.loewis.de's message of "30 May 2002 22:10:51 +0200" References: <2mit56ros4.fsf@starship.python.net> <2msn49hp3i.fsf@starship.python.net> Message-ID: <2md6vc3dv7.fsf@starship.python.net> martin@v.loewis.de (Martin v. Loewis) writes: > Michael Hudson writes: > > > It doesn't fail if you disable threading. Oh /good/. > > > > Does anyone want to explain BSD threads and their interaction with > > signals to me? > > I believe this is an overgeneralization in question. Well, yes. I think I've found enough to the relavent bits of FreeBSD source to answer at least some of my own questions now (libc_r/uthread/uthread_sig.c, for the curious). > Thread implementations in BSD vary from "distribution to > distribution" (Free, Net, Open), and may also vary from release to > release. That's what makes this such a fun game <1e-9 wink>. > For example, one of them recently dropped the libc_r library. Didn't know that. Which one? Not Free, by the signs. > Try compiling with the -mt option, unless this is already done; it > appears that this causes the compiler to define things that are > otherwise not defined. $ cc -mt -o sigp sigp.c cc1: Invalid option `t' ? Cheers, M. -- ... when all the programmes on all the channels actually were made by actors with cleft pallettes speaking lines by dyslexic writers filmed by blind cameramen instead of merely seeming like that, it somehow made the whole thing more worthwhile. -- HHGTG, Episode 11 From fredrik@pythonware.com Fri May 31 12:50:51 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Fri, 31 May 2002 13:50:51 +0200 Subject: [Python-Dev] String module References: <001501c207fc$2573e320$4d66accf@othello> <200205301704.g4UH4fo24680@odiug.zope.com> <004201c20804$0a7ab320$4d66accf@othello> <200205301835.g4UIZQv26785@odiug.zope.com> <001701c20812$fb818c40$95d8accf@othello> <200205302010.g4UKAn927215@odiug.zope.com> <3CF72877.5020701@lemburg.com> Message-ID: <021e01c20899$6b2ec180$0900a8c0@spiff> mal wrote: =20 > Do you really think this proliferation of issomething() > methods is a good idea ? >=20 > The same can be had using re.metch() with *much* more flexibility > since you're not stuck with an "intuitive" definition relying > on the intuition of some non-standard body. >=20 > FWIW, I've never used a single one of these single > character based classification APIs. The only non-trivial > issomething() method I can think of is .istitle() because > of it's complicated definition. All others can easily > be had with re.match(). I fully agree. the SRE engine already supports character classes based on ASCII, the current locale (via ctype.h), and the Unicode char- set. we should probably add more classes -- at least the full list of POSIX [:name:] classes, and probably also unicode categories. fwiw, I've played with adding "charset" objects to SRE, which would allow you to plug in custom [:spam:] sets as well (e.g. xml name chars). From guido@python.org Fri May 31 14:11:52 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 31 May 2002 09:11:52 -0400 Subject: [Python-Dev] Re: Adding Optik to the standard library In-Reply-To: Your message of "Fri, 31 May 2002 01:33:41 EDT." <15607.2869.648570.950260@anthem.wooz.org> References: <15606.52125.709788.262269@anthem.wooz.org> <15607.2869.648570.950260@anthem.wooz.org> Message-ID: <200205311311.g4VDBqd09818@pcp742651pcs.reston01.va.comcast.net> > I personally like files in bite-sized chunks, which means when they > get to be more than one or a few tall emacs screenfuls, I start to get > the urge to split things up. Let me just clarify that this is Barry Warsaw's opinion. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com Fri May 31 14:09:46 2002 From: skip@pobox.com (Skip Montanaro) Date: Fri, 31 May 2002 08:09:46 -0500 Subject: [Python-Dev] Distutils confuses Berkeley DB and dbm? In-Reply-To: <3CF72A01.6030207@lemburg.com> References: <15606.52750.844752.46789@12-248-41-177.client.attbi.com> <3CF72A01.6030207@lemburg.com> Message-ID: <15607.30234.768340.836822@12-248-41-177.client.attbi.com> mal> Perhaps you ought to have a look at mx.BeeBase ? It's portable and mal> fast, has locks and transactions. (And it builds on all platforms mal> where egenix-mx-base builds.) Perhaps, but that doesn't solve existing problems with building bsddb and the various dbm-compatibility modes available. The bsddb build problem is essentially that some (many? most? all?) Linux distributions ship with multiple versions of the Berkeley DB library now. To make matters worse, they separate the shared libraries (in base rpms) from the include files (in -devel rpms). On my Mandrake system that gives you six possible rpms to install: dbX and dbX-devel, for X in {1,2,3}. Based on the way distutils checks for libraries and include files (which I believe is mostly my fault), if you have only one of any given pair of such rpms, like Barry, you might wind up compiling with one version of the library and trying to link with a different version. I thought you were nominally against the idea of incorporating bits of mx into the core? Skip From guido@python.org Fri May 31 14:30:03 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 31 May 2002 09:30:03 -0400 Subject: [Python-Dev] String module In-Reply-To: Your message of "Fri, 31 May 2002 09:38:31 +0200." <3CF72877.5020701@lemburg.com> References: <001501c207fc$2573e320$4d66accf@othello> <200205301704.g4UH4fo24680@odiug.zope.com> <004201c20804$0a7ab320$4d66accf@othello> <200205301835.g4UIZQv26785@odiug.zope.com> <001701c20812$fb818c40$95d8accf@othello> <200205302010.g4UKAn927215@odiug.zope.com> <3CF72877.5020701@lemburg.com> Message-ID: <200205311330.g4VDU3B09898@pcp742651pcs.reston01.va.comcast.net> > Do you really think this proliferation of issomething() > methods is a good idea ? To tell you the truth, I'm not sure. Maybe we should stick to supplying replacements for the character set variables in the string module; this would mean adding replacements for hexdigits, octdigits, punctuation, and printable. Also, the string module defines ascii_lowercase, ascii_uppercase, and ascii_letters; maybe an isascii() replacement would be the best approach there. (I've never seen those used, but they exist, so if we want to slowly start discouraging people from using the string module, I feel we're obliged to privide an alternative. > The same can be had using re.metch() with *much* more flexibility > since you're not stuck with an "intuitive" definition relying > on the intuition of some non-standard body. I'm not sure what you're trying to say. The re module's definition of "word" characters (which includes '_') is definitely non-standard. How do you spell "letter according to locale" in a regexp? > FWIW, I've never used a single one of these single > character based classification APIs. The only non-trivial > issomething() method I can think of is .istitle() because > of it's complicated definition. All others can easily > be had with re.match(). > > I'm -0 on adding more of these .issomething() APIs > to strings and Unicode. Noted. But what do you propose we should do in 3.0 about the character set variables in the string module? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri May 31 14:33:05 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 31 May 2002 09:33:05 -0400 Subject: [Python-Dev] Re: Deprecation In-Reply-To: Your message of "Fri, 31 May 2002 04:12:25 EDT." <122601c2087a$e6448e00$6300000a@holdenweb.com> References: <20020529172001.39760.qmail@web9601.mail.yahoo.com> <20020529202959.H17248@prim.han.de> <20020530121309.A8344@eecs.tufts.edu> <20020530222821.M17248@prim.han.de> <200205302058.g4UKwMA27491@odiug.zope.com> <20020530234122.Q17248@prim.han.de> <11ab01c20824$2f102fb0$6300000a@holdenweb.com> <200205310130.g4V1UuR08583@pcp742651pcs.reston01.va.comcast.net> <122601c2087a$e6448e00$6300000a@holdenweb.com> Message-ID: <200205311333.g4VDX5P09920@pcp742651pcs.reston01.va.comcast.net> > Clearly you'd have to goby the source file date, since a new release > would recompile the .pyc's. The main advantage is that deprecation > wornings wouldn't start appearing after the installation of a new > release. The only advantage I see for it is the silence. Obviously > it would eventually mean that programs died a sudden death due to > feature loss. The warnings would appear if the code were maintained, > however. I definitely wouldn't want such a feature. --Guido van Rossum (home page: http://www.python.org/~guido/) From smurf@noris.de Fri May 31 14:28:11 2002 From: smurf@noris.de (Matthias Urlichs) Date: Fri, 31 May 2002 15:28:11 +0200 Subject: [Python-Dev] New bugtracker project In-Reply-To: <3CED0BBE.17241.1E9CB415@localhost>; from gmcm@hypernet.com on Thu, May 23, 2002 at 03:33:18PM -0400 References: <3CED0BBE.17241.1E9CB415@localhost> Message-ID: <20020531152811.W26872@noris.de> Hi, Gordon McMillan: > The *straightforward* way of doing an SQL backend > would be to have (almost) everything in one table, > defined as (int key, binary stuff). That's utterly > useless for querying, since even the DB doesn't > know what it contains. > Right. That wasn't what I had in mind, though. ;-) -- Matthias Urlichs | noris network AG | http://smurf.noris.de/ From guido@python.org Fri May 31 14:35:09 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 31 May 2002 09:35:09 -0400 Subject: [Python-Dev] Single .py file contains submodule? In-Reply-To: Your message of "Fri, 31 May 2002 10:37:03 +0200." <20020531103703.S17248@prim.han.de> References: <20020530203306.5918.33552.Mailman@mail.python.org> <3CF6A670.1000500@campuspipeline.com> <20020531103703.S17248@prim.han.de> Message-ID: <200205311335.g4VDZ9P09950@pcp742651pcs.reston01.va.comcast.net> > I have been asking this myself, too. I definitely see an advantage > beeing able to put modules and submodules in one file. Trust me. Don't go there. --Guido van Rossum (home page: http://www.python.org/~guido/) From mal@lemburg.com Fri May 31 14:32:06 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 31 May 2002 15:32:06 +0200 Subject: [Python-Dev] Distutils confuses Berkeley DB and dbm? References: <15606.52750.844752.46789@12-248-41-177.client.attbi.com> <3CF72A01.6030207@lemburg.com> <15607.30234.768340.836822@12-248-41-177.client.attbi.com> Message-ID: <3CF77B56.6040206@lemburg.com> Skip Montanaro wrote: > mal> Perhaps you ought to have a look at mx.BeeBase ? It's portable and > mal> fast, has locks and transactions. (And it builds on all platforms > mal> where egenix-mx-base builds.) > > Perhaps, but that doesn't solve existing problems with building bsddb and > the various dbm-compatibility modes available. True, just thought I'd drop in an idea how to get around all the dbm problems. > The bsddb build problem is essentially that some (many? most? all?) Linux > distributions ship with multiple versions of the Berkeley DB library now. > To make matters worse, they separate the shared libraries (in base rpms) > from the include files (in -devel rpms). On my Mandrake system that gives > you six possible rpms to install: dbX and dbX-devel, for X in {1,2,3}. > Based on the way distutils checks for libraries and include files (which I > believe is mostly my fault), if you have only one of any given pair of such > rpms, like Barry, you might wind up compiling with one version of the > library and trying to link with a different version. > > I thought you were nominally against the idea of incorporating bits of mx > into the core? Yep; but that doesn't keep you from using them :-) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/ From guido@python.org Fri May 31 14:39:18 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 31 May 2002 09:39:18 -0400 Subject: [Python-Dev] String module In-Reply-To: Your message of "Fri, 31 May 2002 13:50:51 +0200." <021e01c20899$6b2ec180$0900a8c0@spiff> References: <001501c207fc$2573e320$4d66accf@othello> <200205301704.g4UH4fo24680@odiug.zope.com> <004201c20804$0a7ab320$4d66accf@othello> <200205301835.g4UIZQv26785@odiug.zope.com> <001701c20812$fb818c40$95d8accf@othello> <200205302010.g4UKAn927215@odiug.zope.com> <3CF72877.5020701@lemburg.com> <021e01c20899$6b2ec180$0900a8c0@spiff> Message-ID: <200205311339.g4VDdIx10000@pcp742651pcs.reston01.va.comcast.net> > the SRE engine already supports character classes based on > ASCII, the current locale (via ctype.h), and the Unicode char- > set. How do I ask for a locale letter? The re module only defines escapes for "word" and "digit". > we should probably add more classes -- at least the full list of > POSIX [:name:] classes, and probably also unicode categories. +1 > fwiw, I've played with adding "charset" objects to SRE, which > would allow you to plug in custom [:spam:] sets as well (e.g. > xml name chars). +1 --Guido van Rossum (home page: http://www.python.org/~guido/) From pinard@iro.umontreal.ca Fri May 31 14:46:00 2002 From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard) Date: 31 May 2002 09:46:00 -0400 Subject: [Python-Dev] Re: String module In-Reply-To: <200205310455.g4V4t8O09160@pcp742651pcs.reston01.va.comcast.net> References: <200205310455.g4V4t8O09160@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido van Rossum] > [...] (And _ is a pain to type.) By the way, I wonder why the `_' in `sys._getframe()'. Was it to mark that this is a function giving access to internals? If yes, this only increases my wondering, since there are other functions in `sys' which give access to internals, and which do not use such a `_' prefix. I presume this has been debated in the proper time, I was probably elsewhere and missed the discussion. Could the reason be summarised in one sentence? :-) -- François Pinard http://www.iro.umontreal.ca/~pinard From guido@python.org Fri May 31 15:03:44 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 31 May 2002 10:03:44 -0400 Subject: [Python-Dev] Re: String module In-Reply-To: Your message of "31 May 2002 09:46:00 EDT." References: <200205310455.g4V4t8O09160@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200205311403.g4VE3il10348@pcp742651pcs.reston01.va.comcast.net> > By the way, I wonder why the `_' in `sys._getframe()'. Was it to > mark that this is a function giving access to internals? If yes, > this only increases my wondering, since there are other functions in > `sys' which give access to internals, and which do not use such a > `_' prefix. I presume this has been debated in the proper time, I > was probably elsewhere and missed the discussion. Could the reason > be summarised in one sentence? :-) To discourage people from using it. This function enables a programming idiom where a function digs in its caller's namespace. I find that highly undesirable so I wanted to emphasize that this function does not exist for that purpose. (Neither does inspect.currentframe(). :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From neal@metaslash.com Fri May 31 15:08:29 2002 From: neal@metaslash.com (Neal Norwitz) Date: Fri, 31 May 2002 10:08:29 -0400 Subject: [Python-Dev] Removing types module from stdlib Message-ID: <3CF783DD.DADFB674@metaslash.com> Should I start converting/removing uses of the types module where possible? So where we have: assert type(lineno) is types.IntType assert type(lineno) in (types.IntType, types.LongType) would become: assert type(lineno) is int assert type(lineno) in (int, long) or assert isinstance(lineno, int) assert isinstance(lineno, (int, long)) Preferences? Neal From guido@python.org Fri May 31 15:29:42 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 31 May 2002 10:29:42 -0400 Subject: [Python-Dev] Making distutils string-free? Message-ID: <200205311429.g4VETgu23416@pcp742651pcs.reston01.va.comcast.net> I notice distutils still imports the string module instead of using string methods. Is it still necessary that this codebase runs with Python 1.5.2? If not, someone can start working on removing the string module references. (The sre module needs to run under 1.5.2, per agreement with /F, so it has to keep using the string module.) --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Fri May 31 15:25:54 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 31 May 2002 10:25:54 -0400 Subject: [Python-Dev] Distutils confuses Berkeley DB and dbm? References: <15606.52750.844752.46789@12-248-41-177.client.attbi.com> <3CF72A01.6030207@lemburg.com> <15607.30234.768340.836822@12-248-41-177.client.attbi.com> Message-ID: <15607.34802.306759.150994@anthem.wooz.org> >>>>> "SM" == Skip Montanaro writes: SM> The bsddb build problem is essentially that some (many? most? SM> all?) Linux distributions ship with multiple versions of the SM> Berkeley DB library now. To make matters worse, they separate SM> the shared libraries (in base rpms) from the include files (in SM> -devel rpms). On my Mandrake system that gives you six SM> possible rpms to install: dbX and dbX-devel, for X in {1,2,3}. SM> Based on the way distutils checks for libraries and include SM> files (which I believe is mostly my fault), if you have only SM> one of any given pair of such rpms, like Barry, you might wind SM> up compiling with one version of the library and trying to SM> link with a different version. It's worse than that. On my MD8.2 system, I've got at least 4 versions of Berkeley installed: db1-1.85-7mdk + db1-devel-1.85-7mdk db2-2.4.14-5mdk + db2-devel-2.4.14-5mdk libdb3.3-3.3.11-7mdk + libdb3.3-devel-3.3.11-7mdk Those are all installed with a fairly beefy package selection, but nonetheless stock packages. Then I've got Berkeley DB 3.3.11 installed from source sitting in its default installation spot of /usr/local/BerkeleyDB.3.3 and 4.0.14 sitting in /usr/local/BerkeleyDB.4.0 ... sigh! I wouldn't argue if you said it was a mess. ;) So what do we do? I still think that pybsddb is a worthy candidate for inclusion in the standard library and it should link against 3.3.11 out of the box. I believe the cvs snapshot of pybsddb links against 4.0.14 as well. Then there's this bug http://sourceforge.net/tracker/index.php?func=detail&aid=408271&group_id=5470&atid=105470 Maybe we should lock bsddbmodule down to Berkeley 1.85 and then pull pybsddb (which exports as module bsddb3) for Berkeley 3.3.11. If there's a clamor for it, I suppose we could fake a bsddb2 and bsddb4 for those major versions of Berkeley. -Barry From gward@python.net Fri May 31 15:26:54 2002 From: gward@python.net (Greg Ward) Date: Fri, 31 May 2002 10:26:54 -0400 Subject: [Python-Dev] Re: Making distutils string-free? In-Reply-To: <200205311429.g4VETgu23416@pcp742651pcs.reston01.va.comcast.net> References: <200205311429.g4VETgu23416@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020531142654.GA14999@gerg.ca> On 31 May 2002, Guido van Rossum said: > I notice distutils still imports the string module instead of using > string methods. Is it still necessary that this codebase runs with > Python 1.5.2? No, I think that requirement can safely be dropped now. > If not, someone can start working on removing the > string module references. Fine by me! Greg From skip@pobox.com Fri May 31 15:34:22 2002 From: skip@pobox.com (Skip Montanaro) Date: Fri, 31 May 2002 09:34:22 -0500 Subject: [Python-Dev] Distutils confuses Berkeley DB and dbm? In-Reply-To: <15607.34802.306759.150994@anthem.wooz.org> References: <15606.52750.844752.46789@12-248-41-177.client.attbi.com> <3CF72A01.6030207@lemburg.com> <15607.30234.768340.836822@12-248-41-177.client.attbi.com> <15607.34802.306759.150994@anthem.wooz.org> Message-ID: <15607.35310.849927.970756@12-248-41-177.client.attbi.com> BAW> Maybe we should lock bsddbmodule down to Berkeley 1.85 and then Please don't do that. That creates two problems. On the one hand it will break code for people like me who successfully use db2 or db3. On the other hand, you will force all users of dbhash and bsddb and all users of anydbm who have bsddb installed to use the provably broken hash file implementation. If you want to lock bsddb down to the 1.85 API, force it to only build with db2 or db3 and reject attempts to compile/link it with db1. BAW> pull pybsddb (which exports as module bsddb3) for Berkeley 3.3.11. BAW> If there's a clamor for it, I suppose we could fake a bsddb2 and BAW> bsddb4 for those major versions of Berkeley. Why the proliferation? I can see the argument for incorporating pybsddb into the core because it offers greater functionality, but why incorporate version names into module names that track releases? If Sleepycat's behavior in the past is any indication of its behavior in the future, they will change both the API and file formats in every release and give you various, incompatible tools with which to migrate each time. Skip From mal@lemburg.com Fri May 31 15:36:41 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 31 May 2002 16:36:41 +0200 Subject: [Python-Dev] Making distutils string-free? References: <200205311429.g4VETgu23416@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CF78A79.6050807@lemburg.com> Guido van Rossum wrote: > I notice distutils still imports the string module instead of using > string methods. Is it still necessary that this codebase runs with > Python 1.5.2? If not, someone can start working on removing the > string module references. (The sre module needs to run under 1.5.2, > per agreement with /F, so it has to keep using the string module.) distutils has to maintain 1.5.2 compatibility. Please don't change anything which would cause this to break. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/ From neal@metaslash.com Fri May 31 15:37:17 2002 From: neal@metaslash.com (Neal Norwitz) Date: Fri, 31 May 2002 10:37:17 -0400 Subject: [Python-Dev] Making distutils string-free? References: <200205311429.g4VETgu23416@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CF78A9D.E4BFFF29@metaslash.com> Guido van Rossum wrote: > > I notice distutils still imports the string module instead of using > string methods. Is it still necessary that this codebase runs with > Python 1.5.2? If not, someone can start working on removing the > string module references. (The sre module needs to run under 1.5.2, > per agreement with /F, so it has to keep using the string module.) There's also some uses of string in xml/, compiler/, and lib-tk/Tix.py. Can these all be removed? Neal From guido@python.org Fri May 31 15:44:43 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 31 May 2002 10:44:43 -0400 Subject: [Python-Dev] Making distutils string-free? In-Reply-To: Your message of "Fri, 31 May 2002 16:36:41 +0200." <3CF78A79.6050807@lemburg.com> References: <200205311429.g4VETgu23416@pcp742651pcs.reston01.va.comcast.net> <3CF78A79.6050807@lemburg.com> Message-ID: <200205311444.g4VEihR24577@pcp742651pcs.reston01.va.comcast.net> > distutils has to maintain 1.5.2 compatibility. Please don't > change anything which would cause this to break. Really? Why? Isn't the existing standalone distutils distribution good enough? --Guido van Rossum (home page: http://www.python.org/~guido/) From gward@python.net Fri May 31 15:39:56 2002 From: gward@python.net (Greg Ward) Date: Fri, 31 May 2002 10:39:56 -0400 Subject: [Python-Dev] Re: Adding Optik to the standard library In-Reply-To: <200205310132.g4V1W1U08610@pcp742651pcs.reston01.va.comcast.net> References: <200205301946.g4UJkLL26987@odiug.zope.com> <15606.52125.709788.262269@anthem.wooz.org> <200205310132.g4V1W1U08610@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020531143956.GA15236@gerg.ca> On 30 May 2002, Guido van Rossum said: > getopt-as-package is definitely out. I'll leave it to Greg what to > make of the remaining two alternatives (options or OptionParser). I strongly prefer OptionParser, because that's the main class; it's the one that's always used (ie. directly instantiated). There are always instances of Option, OptionValues, and the various exception classes floating around -- but most Optik applications don't have to import those names directly. So in spite of David G.'s -1 on OptionParser, that's what I'm going with... Greg -- Greg Ward - Python bigot gward@python.net http://starship.python.net/~gward/ From guido@python.org Fri May 31 15:46:55 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 31 May 2002 10:46:55 -0400 Subject: [Python-Dev] Making distutils string-free? In-Reply-To: Your message of "Fri, 31 May 2002 10:37:17 EDT." <3CF78A9D.E4BFFF29@metaslash.com> References: <200205311429.g4VETgu23416@pcp742651pcs.reston01.va.comcast.net> <3CF78A9D.E4BFFF29@metaslash.com> Message-ID: <200205311446.g4VEktX24621@pcp742651pcs.reston01.va.comcast.net> > There's also some uses of string in xml/, compiler/, and lib-tk/Tix.py. > Can these all be removed? I don't know if the xml/ codebase has a backwards compat requirement. I can't imagine compiler/ does, so go ahead there. Ditto for Tix.py. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri May 31 15:47:19 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 31 May 2002 10:47:19 -0400 Subject: [Python-Dev] Re: Adding Optik to the standard library In-Reply-To: Your message of "Fri, 31 May 2002 10:39:56 EDT." <20020531143956.GA15236@gerg.ca> References: <200205301946.g4UJkLL26987@odiug.zope.com> <15606.52125.709788.262269@anthem.wooz.org> <200205310132.g4V1W1U08610@pcp742651pcs.reston01.va.comcast.net> <20020531143956.GA15236@gerg.ca> Message-ID: <200205311447.g4VElJZ24638@pcp742651pcs.reston01.va.comcast.net> > I strongly prefer OptionParser, because that's the main class; it's the > one that's always used (ie. directly instantiated). There are always > instances of Option, OptionValues, and the various exception classes > floating around -- but most Optik applications don't have to import > those names directly. > > So in spite of David G.'s -1 on OptionParser, that's what I'm going > with... Great! --Guido van Rossum (home page: http://www.python.org/~guido/) From gward@python.net Fri May 31 15:42:30 2002 From: gward@python.net (Greg Ward) Date: Fri, 31 May 2002 10:42:30 -0400 Subject: [Python-Dev] Making distutils string-free? In-Reply-To: <3CF78A79.6050807@lemburg.com> References: <200205311429.g4VETgu23416@pcp742651pcs.reston01.va.comcast.net> <3CF78A79.6050807@lemburg.com> Message-ID: <20020531144230.GB15236@gerg.ca> On 31 May 2002, M.-A. Lemburg said: > distutils has to maintain 1.5.2 compatibility. Please don't > change anything which would cause this to break. Isn't it enough that Distutils 1.0.1 or 1.0.2 or whatever-it-is is available for Python 1.5.2? At some point, the Distutils distributed with Python should catch up with modern Python coding conventions. Now's as good a time as any. Greg -- Greg Ward - programmer-at-big gward@python.net http://starship.python.net/~gward/ War is Peace; Freedom is Slavery; Ignorance is Knowledge From guido@python.org Fri May 31 15:18:53 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 31 May 2002 10:18:53 -0400 Subject: [Python-Dev] Removing types module from stdlib In-Reply-To: Your message of "Fri, 31 May 2002 10:08:29 EDT." <3CF783DD.DADFB674@metaslash.com> References: <3CF783DD.DADFB674@metaslash.com> Message-ID: <200205311418.g4VEIrD17015@pcp742651pcs.reston01.va.comcast.net> > Should I start converting/removing uses of the types module where possible? Go ahead, but don't be too aggressive. I.e. leave cases where it's unclear what to do alone, and don't strive for completeness. OTOH, I see no need to go through the SF patch manager for this, as long as the test suite runs. > So where we have: > assert type(lineno) is types.IntType > assert type(lineno) in (types.IntType, types.LongType) > > would become: > assert type(lineno) is int > assert type(lineno) in (int, long) > > or > assert isinstance(lineno, int) > assert isinstance(lineno, (int, long)) > > Preferences? I strongly prefer the isinstance() forms, but beware that they have different semantics in the light of subtypes. You may have to think about what's intended. Usually isinstance() will be the right thing to do though. --Guido van Rossum (home page: http://www.python.org/~guido/) From mal@lemburg.com Fri May 31 15:44:15 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 31 May 2002 16:44:15 +0200 Subject: [Python-Dev] Making distutils string-free? References: <200205311429.g4VETgu23416@pcp742651pcs.reston01.va.comcast.net> <3CF78A79.6050807@lemburg.com> <200205311444.g4VEihR24577@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CF78C3F.1080702@lemburg.com> Guido van Rossum wrote: >>distutils has to maintain 1.5.2 compatibility. Please don't >>change anything which would cause this to break. > > > Really? Why? Isn't the existing standalone distutils distribution > good enough? No, it's still missing important support for FreeBSD, Debian and some other packages. Also, bug fixes should be easily had for package writers who wan't to maintain 1.5.2 compatibility. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/ From Oleg Broytmann Fri May 31 15:44:18 2002 From: Oleg Broytmann (Oleg Broytmann) Date: Fri, 31 May 2002 18:44:18 +0400 Subject: [Python-Dev] Distutils confuses Berkeley DB and dbm? In-Reply-To: <15607.35310.849927.970756@12-248-41-177.client.attbi.com>; from skip@pobox.com on Fri, May 31, 2002 at 09:34:22AM -0500 References: <15606.52750.844752.46789@12-248-41-177.client.attbi.com> <3CF72A01.6030207@lemburg.com> <15607.30234.768340.836822@12-248-41-177.client.attbi.com> <15607.34802.306759.150994@anthem.wooz.org> <15607.35310.849927.970756@12-248-41-177.client.attbi.com> Message-ID: <20020531184418.I19177@phd.pp.ru> On Fri, May 31, 2002 at 09:34:22AM -0500, Skip Montanaro wrote: > but why incorporate > version names into module names that track releases? I would very much to see few different modules for BSDDB - I still use BerkeleyDB 1.85 (to read/write DBs for a closed-source program), and use newer versions for other needs. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From akuchlin@mems-exchange.org Fri May 31 15:45:31 2002 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Fri, 31 May 2002 10:45:31 -0400 Subject: [Python-Dev] Making distutils string-free? In-Reply-To: <3CF78A9D.E4BFFF29@metaslash.com> References: <200205311429.g4VETgu23416@pcp742651pcs.reston01.va.comcast.net> <3CF78A9D.E4BFFF29@metaslash.com> Message-ID: <20020531144531.GA29513@ute.mems-exchange.org> On Fri, May 31, 2002 at 10:37:17AM -0400, Neal Norwitz wrote: >There's also some uses of string in xml/, compiler/, and lib-tk/Tix.py. >Can these all be removed? The use of 'string' in xml/ likely stems from the original PyXML code aiming to run with 1.5.2. Adding string methods will cause divergence from the PyXML code base for no good reason; I'd suggest leaving it alone. String methods will appear as the PyXML code begins to use them. As for Distutils, I again don't see the need to use string methods everywhere. (Though admittedly I have no plans to issue a standalone release again, and presumably neither does Greg.) --amk From guido@python.org Fri May 31 15:53:13 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 31 May 2002 10:53:13 -0400 Subject: [Python-Dev] Making distutils string-free? In-Reply-To: Your message of "Fri, 31 May 2002 16:44:15 +0200." <3CF78C3F.1080702@lemburg.com> References: <200205311429.g4VETgu23416@pcp742651pcs.reston01.va.comcast.net> <3CF78A79.6050807@lemburg.com> <200205311444.g4VEihR24577@pcp742651pcs.reston01.va.comcast.net> <3CF78C3F.1080702@lemburg.com> Message-ID: <200205311453.g4VErDR24749@pcp742651pcs.reston01.va.comcast.net> > No, it's still missing important support for FreeBSD, Debian > and some other packages. Also, bug fixes should be easily had > for package writers who wan't to maintain 1.5.2 compatibility. Sigh. OK, let's hold off on "fixing" distutils. --Guido van Rossum (home page: http://www.python.org/~guido/) From mal@lemburg.com Fri May 31 15:53:10 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 31 May 2002 16:53:10 +0200 Subject: [Python-Dev] Making distutils string-free? References: <200205311429.g4VETgu23416@pcp742651pcs.reston01.va.comcast.net> <3CF78A79.6050807@lemburg.com> <200205311444.g4VEihR24577@pcp742651pcs.reston01.va.comcast.net> <3CF78C3F.1080702@lemburg.com> <200205311453.g4VErDR24749@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CF78E56.9000205@lemburg.com> Guido van Rossum wrote: >>No, it's still missing important support for FreeBSD, Debian >>and some other packages. Also, bug fixes should be easily had >>for package writers who wan't to maintain 1.5.2 compatibility. > > > Sigh. OK, let's hold off on "fixing" distutils. Side-note: distutils isn't performance sensitive and string methods don't buy you anything else since Unicode is not used in distutils. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/ From guido@python.org Fri May 31 15:58:44 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 31 May 2002 10:58:44 -0400 Subject: [Python-Dev] Making distutils string-free? In-Reply-To: Your message of "Fri, 31 May 2002 10:45:31 EDT." <20020531144531.GA29513@ute.mems-exchange.org> References: <200205311429.g4VETgu23416@pcp742651pcs.reston01.va.comcast.net> <3CF78A9D.E4BFFF29@metaslash.com> <20020531144531.GA29513@ute.mems-exchange.org> Message-ID: <200205311458.g4VEwiM24795@pcp742651pcs.reston01.va.comcast.net> It looks like a decent guideline is: make individual modules string-module-free, but don't touch large packages. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com Fri May 31 15:53:57 2002 From: skip@pobox.com (Skip Montanaro) Date: Fri, 31 May 2002 09:53:57 -0500 Subject: [Python-Dev] Making distutils string-free? In-Reply-To: <200205311444.g4VEihR24577@pcp742651pcs.reston01.va.comcast.net> References: <200205311429.g4VETgu23416@pcp742651pcs.reston01.va.comcast.net> <3CF78A79.6050807@lemburg.com> <200205311444.g4VEihR24577@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15607.36485.37999.696838@12-248-41-177.client.attbi.com> >> distutils has to maintain 1.5.2 compatibility. Please don't change >> anything which would cause this to break. Guido> Really? Why? Isn't the existing standalone distutils Guido> distribution good enough? Porting changes between the in-core and standalone versions of distutils would be more difficult if the former used string methods and the latter used the string module. Skip From mal@lemburg.com Fri May 31 15:59:57 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 31 May 2002 16:59:57 +0200 Subject: [Python-Dev] Making distutils string-free? References: <200205311429.g4VETgu23416@pcp742651pcs.reston01.va.comcast.net> <3CF78A9D.E4BFFF29@metaslash.com> <20020531144531.GA29513@ute.mems-exchange.org> <200205311458.g4VEwiM24795@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CF78FED.9020101@lemburg.com> Guido van Rossum wrote: > It looks like a decent guideline is: make individual modules > string-module-free, but don't touch large packages. Indeed :-) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/ From pinard@iro.umontreal.ca Fri May 31 16:20:11 2002 From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard) Date: 31 May 2002 11:20:11 -0400 Subject: [Python-Dev] Re: String module In-Reply-To: <200205311403.g4VE3il10348@pcp742651pcs.reston01.va.comcast.net> References: <200205310455.g4V4t8O09160@pcp742651pcs.reston01.va.comcast.net> <200205311403.g4VE3il10348@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido van Rossum] > > By the way, I wonder why the `_' in `sys._getframe()'. > To discourage people from using it. Why was it introduced then, since there was already a way without it? Execution speed, maybe? Writing speed is probably not a consideration, as the previous way was a bot discouraging already, in that respect :-). > This function enables a programming idiom where a function digs in > its caller's namespace. I find that highly undesirable so I wanted > to emphasize that this function does not exist for that purpose. Does it exist for another purpose? Getting the frame, you will say! :-) -- François Pinard http://www.iro.umontreal.ca/~pinard From guido@python.org Fri May 31 16:32:33 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 31 May 2002 11:32:33 -0400 Subject: [Python-Dev] Re: String module In-Reply-To: Your message of "31 May 2002 11:20:11 EDT." References: <200205310455.g4V4t8O09160@pcp742651pcs.reston01.va.comcast.net> <200205311403.g4VE3il10348@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200205311532.g4VFWXO07485@pcp742651pcs.reston01.va.comcast.net> > > > By the way, I wonder why the `_' in `sys._getframe()'. > > > To discourage people from using it. > > Why was it introduced then, since there was already a way without it? Because there are legitimate uses -- mostly in the area of introspection or debugging -- and the existing way (catching an exception) was clumsy. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Fri May 31 16:31:51 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 31 May 2002 11:31:51 -0400 Subject: [Python-Dev] Single .py file contains submodule? In-Reply-To: <200205311335.g4VDZ9P09950@pcp742651pcs.reston01.va.comcast.net> Message-ID: [holger krekel] > I have been asking this myself, too. I definitely see an advantage > beeing able to put modules and submodules in one file. [Guido] > Trust me. Don't go there. In Perl you spell this "package NAME;". All "dynamic" bindings from then until the next package statement (or end of enclosing block, etc -- it's Perl ) then end up in the NAME namespace. This is also how you declare a class in Perl. This is what it leads to : If you have a package called m, s, or y, then you can't use the qualified form of an identifier because it would be instead interpreted as a pattern match, a substitution, or a transliteration. From jacobs@penguin.theopalgroup.com Fri May 31 16:33:53 2002 From: jacobs@penguin.theopalgroup.com (Kevin Jacobs) Date: Fri, 31 May 2002 11:33:53 -0400 (EDT) Subject: [Python-Dev] Re: String module In-Reply-To: <200205311532.g4VFWXO07485@pcp742651pcs.reston01.va.comcast.net> Message-ID: On Fri, 31 May 2002, Guido van Rossum wrote: > > > > By the way, I wonder why the `_' in `sys._getframe()'. > > > To discourage people from using it. > > Why was it introduced then, since there was already a way without it? > > Because there are legitimate uses -- mostly in the area of > introspection or debugging -- and the existing way (catching an > exception) was clumsy. Clumsy and broken -- the old trick destroys the current exception context, so it is not safe to use in places where that side-effect is undesirable (which is all of the places I actually wanted to use it). -Kevin -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com From barry@zope.com Fri May 31 16:34:35 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 31 May 2002 11:34:35 -0400 Subject: [Python-Dev] Making distutils string-free? References: <200205311429.g4VETgu23416@pcp742651pcs.reston01.va.comcast.net> <3CF78A9D.E4BFFF29@metaslash.com> <20020531144531.GA29513@ute.mems-exchange.org> <200205311458.g4VEwiM24795@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15607.38923.211862.130582@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> It looks like a decent guideline is: make individual modules GvR> string-module-free, but don't touch large packages. Also: new packages and modules should be string-module-free. -Barry From barry@zope.com Fri May 31 16:36:59 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 31 May 2002 11:36:59 -0400 Subject: [Python-Dev] Re: String module References: <200205310455.g4V4t8O09160@pcp742651pcs.reston01.va.comcast.net> <200205311403.g4VE3il10348@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15607.39067.413160.388247@anthem.wooz.org> >>>>> "FP" =3D=3D Fran=E7ois Pinard writes: >> By the way, I wonder why the `_' in `sys._getframe()'. >> To discourage people from using it. FP> Why was it introduced then, since there was already a way FP> without it? Execution speed, maybe? Because try: 1/0 except ZeroDivisionError: return sys.exc_info()[2].tb_frame.f_back is slow, hard to remember, and too magical. Also, sys._getframe() gives you a better interface for getting frames farther back than currentframe(). -Barry From python@rcn.com Fri May 31 16:38:27 2002 From: python@rcn.com (Raymond Hettinger) Date: Fri, 31 May 2002 11:38:27 -0400 Subject: [Python-Dev] Removing types module from stdlib References: <3CF783DD.DADFB674@metaslash.com> Message-ID: <001201c208b9$36271c60$d061accf@othello> There were a couple of threads on comp.lang.py with headings like "isinstance considered harmful". The conclusion was that most cases should have been coded some other way. The legitimate uses included implementing multiple dispatch (supporting calls with type differentiated signatures) , factory functions, decorator pattern (which needs to know if the object was previously decorated), and the composite pattern (which needs to know if it is dealing with an atom or a composite). Here are some of the ideas for conversion: if type(x) is int --> if x==int(x) # 3L would be passable if type(x) == types.FileType --> if hasattr(x, 'read') # StringIO would work if type(x) is str: x = x.lower() --> try: x = x.lower() except AttributeError: pass If the type check is meant to verify that an interface is supported, the try/except form maximized substitutability of objects which simulate the interface (like a UserDict or Shelve in place of a dictionary). I'm not sure this kind of conversion should be done everywhere but it could be an opportunity to remove unnecessary type dependencies. Raymond Hettinger ----- Original Message ----- From: "Neal Norwitz" To: Sent: Friday, May 31, 2002 10:08 AM Subject: [Python-Dev] Removing types module from stdlib > Should I start converting/removing uses of the types module where possible? > > So where we have: > assert type(lineno) is types.IntType > assert type(lineno) in (types.IntType, types.LongType) > > would become: > assert type(lineno) is int > assert type(lineno) in (int, long) > > or > assert isinstance(lineno, int) > assert isinstance(lineno, (int, long)) > > Preferences? > > Neal > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > From guido@python.org Fri May 31 17:03:54 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 31 May 2002 12:03:54 -0400 Subject: [Python-Dev] Removing types module from stdlib In-Reply-To: Your message of "Fri, 31 May 2002 11:38:27 EDT." <001201c208b9$36271c60$d061accf@othello> References: <3CF783DD.DADFB674@metaslash.com> <001201c208b9$36271c60$d061accf@othello> Message-ID: <200205311603.g4VG3sG14379@pcp742651pcs.reston01.va.comcast.net> > There were a couple of threads on comp.lang.py with headings like > "isinstance considered harmful". The conclusion was that most cases > should have been coded some other way. The legitimate uses included > implementing multiple dispatch (supporting calls with type > differentiated signatures) , factory functions, decorator pattern > (which needs to know if the object was previously decorated), and > the composite pattern (which needs to know if it is dealing with an > atom or a composite). Missing from the list is the important use case where it is known that something you are going to do only works for a specific built-in type. E.g. marshal.dump() requires a real file (and no, don't bother "fixing" this -- marshal is sufficiently close to the implementation that I don't want it to be fixed). There are also places where a built-in int is required. > Here are some of the ideas for conversion: > > if type(x) is int --> if x==int(x) # 3L would be passable -1. This dies with a TypeError if x can't be converted to int. > if type(x) == types.FileType --> if hasattr(x, 'read') # StringIO would > work If that's what you want. But you should test for the method you're actually going to call, often readline, or write. (Don't test for multiple methods though -- that's too much). > if type(x) is str: x = x.lower() --> > > try: > x = x.lower() > except AttributeError: > pass Hm, I hate to see try/except proliferated like this. Why is this needed in the first place? Can't you write "x = x.lower()" without the try/except? > If the type check is meant to verify that an interface is supported, > the try/except form maximized substitutability of objects which > simulate the interface (like a UserDict or Shelve in place of a > dictionary). In general I think explicit type checks are for wimps. > I'm not sure this kind of conversion should be done everywhere but > it could be an opportunity to remove unnecessary type dependencies. I'd be careful. Knowing what's right requires deep understanding of what the code is doing. If today it says type(x) is types.StringType, I'm comfortable that isinstance(x, str) will do the right thing; not about any of the alternatives. I propose that if you find a place that looks like it is making too strict a type assertion, report it here or in the SF bug tracker before fixing it. --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik@pythonware.com Fri May 31 17:01:55 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Fri, 31 May 2002 18:01:55 +0200 Subject: [Python-Dev] Removing types module from stdlib References: <3CF783DD.DADFB674@metaslash.com> <001201c208b9$36271c60$d061accf@othello> Message-ID: <00a301c208bc$7e058500$0900a8c0@spiff> raymond wrote: > There were a couple of threads on comp.lang.py with headings like > "isinstance considered harmful". The conclusion was that most cases = should > have been coded some other way. The legitimate uses included = implementing > multiple dispatch (supporting calls with type differentiated = signatures) , > factory functions, decorator pattern (which needs to know if the = object was > previously decorated), and the composite pattern (which needs to know = if it > is dealing with an atom or a composite). since when is comp.lang.py is an authority on anything? > Here are some of the ideas for conversion: >=20 > if type(x) is int --> if x=3D=3Dint(x) # 3L would be passable are these supposed to be equivalent, for any kind of x? what Python version are you using? From python@rcn.com Fri May 31 17:07:32 2002 From: python@rcn.com (Raymond Hettinger) Date: Fri, 31 May 2002 12:07:32 -0400 Subject: [Python-Dev] Other library code transformations Message-ID: <001501c208bd$46133420$d061accf@othello> While we're eliminating uses of the string and types modules, how about other code clean-ups and modernization: d.has_key(k) --> k in d if k in d.keys() --> if k in d obj.__dict__ --> vars(obj) class X(UserList) --> class X(list) class X(UserDict) --> class X(dict) # and remove .data references 0L --> 0 1L --> 1 lambda x, y=y: fun(x,y) --> lambda x: fun(x,y) x = y.__class__ --> x is type(y) == None --> is None if item in astring --> if item in adict This is from my own list of code updates, the python library already in better shape. Raymond Hettinger From barry@zope.com Fri May 31 17:16:53 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 31 May 2002 12:16:53 -0400 Subject: [Python-Dev] Distutils confuses Berkeley DB and dbm? References: <15606.52750.844752.46789@12-248-41-177.client.attbi.com> <3CF72A01.6030207@lemburg.com> <15607.30234.768340.836822@12-248-41-177.client.attbi.com> <15607.34802.306759.150994@anthem.wooz.org> <15607.35310.849927.970756@12-248-41-177.client.attbi.com> Message-ID: <15607.41461.151921.316288@anthem.wooz.org> >>>>> "SM" == Skip Montanaro writes: SM> If you want to lock bsddb down to the 1.85 API, force it to SM> only build with db2 or db3 and reject attempts to compile/link SM> it with db1. That's more in line with what I was thinking. BAW> pull pybsddb (which exports as module bsddb3) for Berkeley BAW> 3.3.11. If there's a clamor for it, I suppose we could fake BAW> a bsddb2 and bsddb4 for those major versions of Berkeley. SM> Why the proliferation? I can see the argument for SM> incorporating pybsddb into the core because it offers greater SM> functionality, but why incorporate version names into module SM> names that track releases? If Sleepycat's behavior in the SM> past is any indication of its behavior in the future, they SM> will change both the API and file formats in every release and SM> give you various, incompatible tools with which to migrate SM> each time. The problem seems to be trying to figure out which API is available and which you want to use. You might even want to use more than one in any given Python installation. So it seems reasonable to be explicit about the version of the API you need. I'd still default "bsddb" to the 1.85 API even if that links with a later version of the library/headers (as long as it's consistent, as you suggested). Since pybsddb3 already exports itself as `bsddb3' that seems to make sense too, although how that interacts with Berkeley DB 4.0.14, I'm not sure (maybe that blows my theory). -Barry From guido@python.org Fri May 31 17:24:05 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 31 May 2002 12:24:05 -0400 Subject: [Python-Dev] Other library code transformations In-Reply-To: Your message of "Fri, 31 May 2002 12:07:32 EDT." <001501c208bd$46133420$d061accf@othello> References: <001501c208bd$46133420$d061accf@othello> Message-ID: <200205311624.g4VGO5n14589@pcp742651pcs.reston01.va.comcast.net> > While we're eliminating uses of the string and types modules, how about > other code clean-ups and modernization: In general I'd only do these when you're working on a module anyway. And again stay out of the larger packages altogether. > d.has_key(k) --> k in d > if k in d.keys() --> if k in d OK. > obj.__dict__ --> vars(obj) No. vars() exists in the first place to be called without arguments, to look in the interactive namespace. > class X(UserList) --> class X(list) That would be a huge change in semantics. > class X(UserDict) --> class X(dict) # and remove .data references Ditto. > 0L --> 0 > 1L --> 1 Careful -- there are still semantic differences (e.g. 1<<100 == 0, 1L<<100 == 2**100). > lambda x, y=y: fun(x,y) --> lambda x: fun(x,y) This changes semantics! If the outer y is later assigned to, the nested-scopes form tracks that change, the default arg doesn't. > x = y.__class__ --> x is type(y) These are not the same! A metaclass can override __class__ but not type(); also, for a classic class instance type() and __class__ have different values. > == None --> is None Yes! > if item in astring --> if item in adict Huh? What does this mean? > This is from my own list of code updates, the python library already > in better shape. I'd rather see effort go towards other things. Have you looked at IDLEFORK yet? --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Fri May 31 18:04:08 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 31 May 2002 13:04:08 -0400 Subject: [Python-Dev] Other library code transformations In-Reply-To: <200205311624.g4VGO5n14589@pcp742651pcs.reston01.va.comcast.net> Message-ID: ]Raymond Hettinger] >> if item in astring --> if item in adict [Guido] > Huh? What does this mean? I assume the only point is speed. Provided there are no collisions, "in" on a character-keyed dict acting as a set is faster than "in" applied to a string, even if the character searched for is the first character in the target string. But it's only a little faster then today. It's about 50% faster by the time you have to search ~= 50 characters into a string before finding a hit. However, most uses of "character in string" have very short "string" parts (like "ch in '/\\'"), and I don't judge the minor speed gain there worth the extra bother and obscurity of maintaining a static set (dict). From pinard@iro.umontreal.ca Fri May 31 19:22:34 2002 From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard) Date: 31 May 2002 14:22:34 -0400 Subject: [Python-Dev] Re: String module In-Reply-To: <15607.39067.413160.388247@anthem.wooz.org> References: <200205310455.g4V4t8O09160@pcp742651pcs.reston01.va.comcast.net> <200205311403.g4VE3il10348@pcp742651pcs.reston01.va.comcast.net> <15607.39067.413160.388247@anthem.wooz.org> Message-ID: [François] > >> By the way, I wonder why the `_' in `sys._getframe()'. [Guido] > >> To discourage people from using it. [Barry] > Because [the previous way] is slow, hard to remember, and too magical. That is, in short, discouraging, exactly as per the wish of Guido! :-) A bit more seriously, it is fun writing: try: frame = sys._getframe(1) except AttributeError: frame = sys.exc_info()[2].tb_frame.f_back putting the error to good use whenever `_getframe' is not available! -- François Pinard http://www.iro.umontreal.ca/~pinard From barry@zope.com Fri May 31 19:24:06 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 31 May 2002 14:24:06 -0400 Subject: [Python-Dev] Re: String module References: <200205310455.g4V4t8O09160@pcp742651pcs.reston01.va.comcast.net> <200205311403.g4VE3il10348@pcp742651pcs.reston01.va.comcast.net> <15607.39067.413160.388247@anthem.wooz.org> Message-ID: <15607.49094.912033.444110@anthem.wooz.org> >>>>> "FP" =3D=3D Fran=E7ois Pinard writes: FP> A bit more seriously, it is fun writing: | try: | frame =3D sys._getframe(1) | except AttributeError: | frame =3D sys.exc_info()[2].tb_frame.f_back FP> putting the error to good use whenever `_getframe' is not FP> available! LOL! Very cute! -Barry From martin@v.loewis.de Fri May 31 19:29:13 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 31 May 2002 20:29:13 +0200 Subject: [Python-Dev] Distutils confuses Berkeley DB and dbm? In-Reply-To: <15607.34802.306759.150994@anthem.wooz.org> References: <15606.52750.844752.46789@12-248-41-177.client.attbi.com> <3CF72A01.6030207@lemburg.com> <15607.30234.768340.836822@12-248-41-177.client.attbi.com> <15607.34802.306759.150994@anthem.wooz.org> Message-ID: barry@zope.com (Barry A. Warsaw) writes: > Maybe we should lock bsddbmodule down to Berkeley 1.85 and then pull > pybsddb (which exports as module bsddb3) for Berkeley 3.3.11. If > there's a clamor for it, I suppose we could fake a bsddb2 and bsddb4 > for those major versions of Berkeley. I completely agree that this should be the strategy. Regards, Martin From martin@v.loewis.de Fri May 31 19:37:41 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 31 May 2002 20:37:41 +0200 Subject: [Python-Dev] Removing types module from stdlib In-Reply-To: <3CF783DD.DADFB674@metaslash.com> References: <3CF783DD.DADFB674@metaslash.com> Message-ID: Neal Norwitz writes: > Should I start converting/removing uses of the types module where possible? Would you like to review patch 562373. Walter already has some code for that. Regards, Martin From neal@metaslash.com Fri May 31 19:56:19 2002 From: neal@metaslash.com (Neal Norwitz) Date: Fri, 31 May 2002 14:56:19 -0400 Subject: [Python-Dev] Removing types module from stdlib References: <3CF783DD.DADFB674@metaslash.com> Message-ID: <3CF7C753.3A231F8E@metaslash.com> "Martin v. Loewis" wrote: > > Neal Norwitz writes: > > > Should I start converting/removing uses of the types module where possible? > > Would you like to review patch 562373. Walter already has some code > for that. Reviewing now... So far so good, but it brings up a question. There was talk about changing string (the base class) to something else since it was the same name as the module. Was there ever a decision made? Since I expect confusion between string (the module) and string (the abstract class). The name issue is marginally related to what's described below. Right now, if you derive a class from an instance (yes, this shouldn't work): >>> class x(5): pass ... Traceback (most recent call last): File "", line 1, in ? TypeError: int() takes at most 2 arguments (3 given) This makes sense and is what I would expect. But this is not what I would expect: >>> import string >>> class newstr(string): pass ... # i would have expected this to raise a TypeError >>> x = newstr() Traceback (most recent call last): File "", line 1, in ? TypeError: 'module' object is not callable Perhaps this error should be handled when the class is constructed rather than when instantiating an object? Of course, you can't derive from the string class either: >>> del string >>> print string >>> class x(string): pass ... >>> y = x() Traceback (most recent call last): File "", line 1, in ? TypeError: The string type cannot be instantiated Neal From guido@python.org Fri May 31 20:30:13 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 31 May 2002 15:30:13 -0400 Subject: [Python-Dev] Removing types module from stdlib In-Reply-To: Your message of "Fri, 31 May 2002 14:56:19 EDT." <3CF7C753.3A231F8E@metaslash.com> References: <3CF783DD.DADFB674@metaslash.com> <3CF7C753.3A231F8E@metaslash.com> Message-ID: <200205311930.g4VJUDx15580@pcp742651pcs.reston01.va.comcast.net> > There was talk about changing string (the base class) to something > else since it was the same name as the module. Was there ever a > decision made? Oops, no. Let's call it basestring. Can you do that? > Since I expect confusion between string (the module) and > string (the abstract class). The name issue is marginally > related to what's described below. > > Right now, if you derive a class from an instance (yes, this shouldn't work): > > >>> class x(5): pass > ... > Traceback (most recent call last): > File "", line 1, in ? > TypeError: int() takes at most 2 arguments (3 given) > > This makes sense and is what I would expect. > > But this is not what I would expect: > > >>> import string > >>> class newstr(string): pass > ... > # i would have expected this to raise a TypeError > >>> x = newstr() > Traceback (most recent call last): > File "", line 1, in ? > TypeError: 'module' object is not callable > > Perhaps this error should be handled when the class is constructed > rather than when instantiating an object? Can you submit a SF bug report for this? It has nothing to do with string per se -- the bug is that you can use any module as a base class. :-( > Of course, you can't derive from the string class either: > > >>> del string > >>> print string > > >>> class x(string): pass > ... > >>> y = x() > Traceback (most recent call last): > File "", line 1, in ? > TypeError: The string type cannot be instantiated Thinking a little about this, I think this is how I want it to be. str and unicode have special status and it shouldn't be easy to create something else (not deriving from either) that also gets this special status. (You can still do it in C of course.) --Guido van Rossum (home page: http://www.python.org/~guido/) From neal@metaslash.com Fri May 31 20:30:08 2002 From: neal@metaslash.com (Neal Norwitz) Date: Fri, 31 May 2002 15:30:08 -0400 Subject: [Python-Dev] Removing types module from stdlib References: <3CF783DD.DADFB674@metaslash.com> <3CF7C753.3A231F8E@metaslash.com> <200205311930.g4VJUDx15580@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CF7CF40.B2949885@metaslash.com> Guido van Rossum wrote: > > > There was talk about changing string (the base class) to something > > else since it was the same name as the module. Was there ever a > > decision made? > > Oops, no. Let's call it basestring. Can you do that? Will do. > > But this is not what I would expect: > > > > >>> import string > > >>> class newstr(string): pass > > ... > > # i would have expected this to raise a TypeError > > >>> x = newstr() > > Traceback (most recent call last): > > File "", line 1, in ? > > TypeError: 'module' object is not callable > > > > Perhaps this error should be handled when the class is constructed > > rather than when instantiating an object? > > Can you submit a SF bug report for this? It has nothing to do with > string per se -- the bug is that you can use any module as a base > class. :-( Will do. Neal From jacobs@penguin.theopalgroup.com Fri May 31 20:34:27 2002 From: jacobs@penguin.theopalgroup.com (Kevin Jacobs) Date: Fri, 31 May 2002 15:34:27 -0400 (EDT) Subject: [Python-Dev] Removing types module from stdlib In-Reply-To: <200205311930.g4VJUDx15580@pcp742651pcs.reston01.va.comcast.net> Message-ID: On Fri, 31 May 2002, Guido van Rossum wrote: > str and unicode have special status and it shouldn't be easy to create > something else (not deriving from either) that also gets this special > status. (You can still do it in C of course.) I use this little toy on occasion: class istr(str): __slots__ = () def __hash__(self): return hash(self.lower()) def __eq__(self, other): return self.lower() == other.lower() def __ne__(self, other): return self.lower() != other.lower() def __le__(self, other): return self.lower() <= other.lower() def __ge__(self, other): return self.lower() >= other.lower() def __lt__(self, other): return self.lower() < other.lower() def __gt__(self, other): return self.lower() > other.lower() Are you saying that you want to outlaw it? -Kevin -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com From guido@python.org Fri May 31 20:42:43 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 31 May 2002 15:42:43 -0400 Subject: [Python-Dev] Removing types module from stdlib In-Reply-To: Your message of "Fri, 31 May 2002 15:34:27 EDT." References: Message-ID: <200205311942.g4VJghH16511@pcp742651pcs.reston01.va.comcast.net> > I use this little toy on occasion: > > class istr(str): > __slots__ = () > def __hash__(self): > return hash(self.lower()) > def __eq__(self, other): > return self.lower() == other.lower() > def __ne__(self, other): > return self.lower() != other.lower() > def __le__(self, other): > return self.lower() <= other.lower() > def __ge__(self, other): > return self.lower() >= other.lower() > def __lt__(self, other): > return self.lower() < other.lower() > def __gt__(self, other): > return self.lower() > other.lower() > > Are you saying that you want to outlaw it? No, subclassing str (or unicode) is fine, because then you inherit the implementation properties. I want to outlaw (or at least discourage) creating new subclasses of the basestring type (in current CVS it's called string, but it will be renamed soon). If someone writes isinstance(x, basestring) they should be confident that x is either a str, a unicode, an instance of a subclass of those, or something written in C that works real hard to look like a string. --Guido van Rossum (home page: http://www.python.org/~guido/) From jacobs@penguin.theopalgroup.com Fri May 31 20:46:29 2002 From: jacobs@penguin.theopalgroup.com (Kevin Jacobs) Date: Fri, 31 May 2002 15:46:29 -0400 (EDT) Subject: [Python-Dev] Removing types module from stdlib In-Reply-To: <200205311942.g4VJghH16511@pcp742651pcs.reston01.va.comcast.net> Message-ID: On Fri, 31 May 2002, Guido van Rossum wrote: > I want to outlaw (or at least discourage) creating new subclasses of > the basestring type (in current CVS it's called string, but it will be > renamed soon). Ah -- I knew I was missing something fundamental. Thanks, -Kevin PS: I like stringbase better than basestring. Not that it matters much. -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com From guido@python.org Fri May 31 20:59:46 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 31 May 2002 15:59:46 -0400 Subject: [Python-Dev] Removing types module from stdlib In-Reply-To: Your message of "Fri, 31 May 2002 15:46:29 EDT." References: Message-ID: <200205311959.g4VJxkY17179@pcp742651pcs.reston01.va.comcast.net> > PS: I like stringbase better than basestring. Not that it matters much. Both conventions occur with roughly the same frequency in the standard library. I'm not sure I favor either one. --Guido van Rossum (home page: http://www.python.org/~guido/) From martin@v.loewis.de Fri May 31 21:25:10 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 31 May 2002 22:25:10 +0200 Subject: [Python-Dev] Making distutils string-free? In-Reply-To: <200205311446.g4VEktX24621@pcp742651pcs.reston01.va.comcast.net> References: <200205311429.g4VETgu23416@pcp742651pcs.reston01.va.comcast.net> <3CF78A9D.E4BFFF29@metaslash.com> <200205311446.g4VEktX24621@pcp742651pcs.reston01.va.comcast.net> Message-ID: Guido van Rossum writes: > > There's also some uses of string in xml/, compiler/, and lib-tk/Tix.py. > > Can these all be removed? > > I don't know if the xml/ codebase has a backwards compat requirement. The current requirement is that it ought to be 2.0-compatible (1.5.2 compatibility was just dropped). So using string methods is fine; using type builtins is not. Regards, Martin From martin@v.loewis.de Fri May 31 21:27:09 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 31 May 2002 22:27:09 +0200 Subject: [Python-Dev] Making distutils string-free? In-Reply-To: <3CF78E56.9000205@lemburg.com> References: <200205311429.g4VETgu23416@pcp742651pcs.reston01.va.comcast.net> <3CF78A79.6050807@lemburg.com> <200205311444.g4VEihR24577@pcp742651pcs.reston01.va.comcast.net> <3CF78C3F.1080702@lemburg.com> <200205311453.g4VErDR24749@pcp742651pcs.reston01.va.comcast.net> <3CF78E56.9000205@lemburg.com> Message-ID: "M.-A. Lemburg" writes: > Side-note: distutils isn't performance sensitive and string > methods don't buy you anything else since Unicode is not > used in distutils. That is not true: msvccompiler has to deal with Unicode in the registry. Regards, Martin From martin@v.loewis.de Fri May 31 21:30:01 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 31 May 2002 22:30:01 +0200 Subject: [Python-Dev] Distutils confuses Berkeley DB and dbm? In-Reply-To: <15607.35310.849927.970756@12-248-41-177.client.attbi.com> References: <15606.52750.844752.46789@12-248-41-177.client.attbi.com> <3CF72A01.6030207@lemburg.com> <15607.30234.768340.836822@12-248-41-177.client.attbi.com> <15607.34802.306759.150994@anthem.wooz.org> <15607.35310.849927.970756@12-248-41-177.client.attbi.com> Message-ID: Skip Montanaro writes: > Why the proliferation? I can see the argument for incorporating pybsddb > into the core because it offers greater functionality, but why incorporate > version names into module names that track releases? Because any specific version of the Sleepycat code can only access a few database file format versions. Older databases cannot be accessed with newer library implementations. Hence, you need to link explicitly with older libraries to access older databases. Regards, Martin From guido@python.org Fri May 31 21:36:55 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 31 May 2002 16:36:55 -0400 Subject: [Python-Dev] Making distutils string-free? In-Reply-To: Your message of "31 May 2002 22:25:10 +0200." References: <200205311429.g4VETgu23416@pcp742651pcs.reston01.va.comcast.net> <3CF78A9D.E4BFFF29@metaslash.com> <200205311446.g4VEktX24621@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200205312036.g4VKatE17488@pcp742651pcs.reston01.va.comcast.net> > > I don't know if the xml/ codebase has a backwards compat requirement. > > The current requirement is that it ought to be 2.0-compatible (1.5.2 > compatibility was just dropped). So using string methods is fine; > using type builtins is not. Great! It looks like there are only a few places where the string module is used: 3 in minidom.py, 2 in sax/_init__.py, 3 in sax/expatreader.py. Neal? --Guido van Rossum (home page: http://www.python.org/~guido/) From mal@lemburg.com Fri May 31 21:34:11 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 31 May 2002 22:34:11 +0200 Subject: [Python-Dev] Making distutils string-free? References: <200205311429.g4VETgu23416@pcp742651pcs.reston01.va.comcast.net> <3CF78A79.6050807@lemburg.com> <200205311444.g4VEihR24577@pcp742651pcs.reston01.va.comcast.net> <3CF78C3F.1080702@lemburg.com> <200205311453.g4VErDR24749@pcp742651pcs.reston01.va.comcast.net> <3CF78E56.9000205@lemburg.com> Message-ID: <3CF7DE43.7010209@lemburg.com> Martin v. Loewis wrote: > "M.-A. Lemburg" writes: > > >>Side-note: distutils isn't performance sensitive and string >>methods don't buy you anything else since Unicode is not >>used in distutils. > > > That is not true: msvccompiler has to deal with Unicode in the > registry. "Dealing" with it is not using it :-) msvccompiler only tries to encode the registry entry into a string and then goes on to process the string. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/ From neal@metaslash.com Fri May 31 21:34:13 2002 From: neal@metaslash.com (Neal Norwitz) Date: Fri, 31 May 2002 16:34:13 -0400 Subject: [Python-Dev] Making distutils string-free? References: <200205311429.g4VETgu23416@pcp742651pcs.reston01.va.comcast.net> <3CF78A9D.E4BFFF29@metaslash.com> <200205311446.g4VEktX24621@pcp742651pcs.reston01.va.comcast.net> <200205312036.g4VKatE17488@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CF7DE45.5DB5F246@metaslash.com> Guido van Rossum wrote: > > > > I don't know if the xml/ codebase has a backwards compat requirement. > > > > The current requirement is that it ought to be 2.0-compatible (1.5.2 > > compatibility was just dropped). So using string methods is fine; > > using type builtins is not. > > Great! It looks like there are only a few places where the string > module is used: 3 in minidom.py, 2 in sax/_init__.py, 3 in > sax/expatreader.py. Neal? That's what I found, checkin coming soon. Neal From kbutler@campuspipeline.com Fri May 31 21:35:04 2002 From: kbutler@campuspipeline.com (Kevin Butler) Date: Fri, 31 May 2002 14:35:04 -0600 Subject: [Python-Dev] Re: Python-Dev digest, Vol 1 #2276 - 14 msgs References: <20020531011203.22396.95848.Mailman@mail.python.org> Message-ID: <3CF7DE78.8030706@campuspipeline.com> I asked about publishing a module & submodules in one .py file, and showed an implementation. Guido responded: > I think it's much better to split it up in multiple files than to use > all those hacks. Those hacks are hard to understand for someone > trying to read the code. I agree - I don't like the look of that code for user-level code. However, to me the complexity was that I had to show the implementation (and the dynamic "getMethods" call instead of explicitly listing methods to publish). If the 'imp' module had a 'publish_module' method that created a module (with an optional dictionary?) & registered it in sys.modules, client code becomes very simple, clear, and can be very explicit: assertions = imp.publish_module( __name__ + ".assertions" ) _a = Assertions() assertions.failUnless = _a.failUnless # ... Is that more pleasant, or is the whole idea of organizing portions of a single file into submodules unacceptable? kb From skip@pobox.com Fri May 31 21:40:50 2002 From: skip@pobox.com (Skip Montanaro) Date: Fri, 31 May 2002 15:40:50 -0500 Subject: [Python-Dev] Distutils confuses Berkeley DB and dbm? In-Reply-To: References: <15606.52750.844752.46789@12-248-41-177.client.attbi.com> <3CF72A01.6030207@lemburg.com> <15607.30234.768340.836822@12-248-41-177.client.attbi.com> <15607.34802.306759.150994@anthem.wooz.org> <15607.35310.849927.970756@12-248-41-177.client.attbi.com> Message-ID: <15607.57298.252930.25397@12-248-41-177.client.attbi.com> Skip> Why the proliferation? Martin> Because any specific version of the Sleepycat code can only Martin> access a few database file format versions. Older databases Martin> cannot be accessed with newer library implementations. Hence, Martin> you need to link explicitly with older libraries to access older Martin> databases. I think it's a bad idea to tie into an external vendor's release idiosyncracies that way. Sleepycat does provide utilities with each release which allows users to migrate from older file formats to newer ones. I would rather see some embellishment of the bsddb module that allows programmers to query file format types and possibly convert file formats by calling out to the Sleepycat-provided utilities. Skip From barry@zope.com Fri May 31 21:48:14 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 31 May 2002 16:48:14 -0400 Subject: [Python-Dev] Distutils confuses Berkeley DB and dbm? References: <15606.52750.844752.46789@12-248-41-177.client.attbi.com> <3CF72A01.6030207@lemburg.com> <15607.30234.768340.836822@12-248-41-177.client.attbi.com> <15607.34802.306759.150994@anthem.wooz.org> <15607.35310.849927.970756@12-248-41-177.client.attbi.com> <15607.57298.252930.25397@12-248-41-177.client.attbi.com> Message-ID: <15607.57742.583273.471532@anthem.wooz.org> >>>>> "SM" == Skip Montanaro writes: Skip> Why the proliferation? Martin> Because any specific version of the Sleepycat code can Martin> only access a few database file format versions. Older Martin> databases cannot be accessed with newer library Martin> implementations. Hence, you need to link explicitly with Martin> older libraries to access older databases. SM> I think it's a bad idea to tie into an external vendor's SM> release idiosyncracies that way. Sleepycat does provide SM> utilities with each release which allows users to migrate from SM> older file formats to newer ones. I would rather see some SM> embellishment of the bsddb module that allows programmers to SM> query file format types and possibly convert file formats by SM> calling out to the Sleepycat-provided utilities. There's really two issues, the file format and the API version. It probably makes sense to keep `bsddb' as the 1.85 API. Maybe it's enough for pybsddb (aka bsddb3) to be the 3.x API and not expose a 2.x API. But then, what about the BDB 4.x API? I agree that we probably don't need to support mutiple older file formats, since there are tools to upgrade, as long as we fail gracefully when handed a format we don't understand. By that I mean, get an exception, not a crash. -Barry From kbutler@campuspipeline.com Fri May 31 22:03:37 2002 From: kbutler@campuspipeline.com (Kevin Butler) Date: Fri, 31 May 2002 15:03:37 -0600 Subject: [Python-Dev] Single .py file contains submodule? Message-ID: <3CF7E529.8090907@campuspipeline.com> M.-A. Lemburg wrote: > Kevin Butler wrote: > > from unittest.assertions import * > > That's bad style (at least for modules which don't only include > constants). Why would you want to enable this ? :-) - In general, if you provide a submodule, users can do 'from unittest import assertions' or 'import unittest.assertions as test' or some such. Just having a grundle of top-level symbols in a module is less convenient than appropriate grouping into submodules. - The Style Guide suggests prefixing methods that are intended for use with 'import *', and these functions are 'test*' or 'assert*'... - For test code, I tend to be a bit more liberal with 'import *', because in general, test code is pretty clear about what it is exercising. - The assertion methods are very independent of the other TestCase methods, and can be very useful for non-TestCase methods. This low coupling/high cohesion suggests organizing them together, but currently there's no convenient way to get at them separately. And since the methods really have little to do with an object instance, I find the 'self.assert*' construct distracting, although I like self.* in most cases. kb From guido@python.org Fri May 31 22:27:38 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 31 May 2002 17:27:38 -0400 Subject: [Python-Dev] Re: Python-Dev digest, Vol 1 #2276 - 14 msgs In-Reply-To: Your message of "Fri, 31 May 2002 14:35:04 MDT." <3CF7DE78.8030706@campuspipeline.com> References: <20020531011203.22396.95848.Mailman@mail.python.org> <3CF7DE78.8030706@campuspipeline.com> Message-ID: <200205312127.g4VLRcs18391@pcp742651pcs.reston01.va.comcast.net> > Is that more pleasant, or is the whole idea of organizing portions > of a single file into submodules unacceptable? There are already various ways to create sub-namespaces in a module, e.g. classes or functions. I really don't see the point of having submodules. --Guido van Rossum (home page: http://www.python.org/~guido/) From mal@lemburg.com Fri May 31 22:25:33 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 31 May 2002 23:25:33 +0200 Subject: [Python-Dev] Single .py file contains submodule? References: <3CF7E529.8090907@campuspipeline.com> Message-ID: <3CF7EA4D.4050400@lemburg.com> Kevin Butler wrote: > M.-A. Lemburg wrote: > > Kevin Butler wrote: > > > from unittest.assertions import * > > > >> That's bad style (at least for modules which don't only include >> constants). Why would you want to enable this ? > > > :-) > > - In general, if you provide a submodule, users can do 'from unittest > import assertions' or 'import unittest.assertions as test' or some such. > Just having a grundle of top-level symbols in a module is less > convenient than appropriate grouping into submodules. > > - The Style Guide suggests prefixing methods that are intended for use > with 'import *', and these functions are 'test*' or 'assert*'... > > - For test code, I tend to be a bit more liberal with 'import *', > because in general, test code is pretty clear about what it is exercising. > > - The assertion methods are very independent of the other TestCase > methods, and can be very useful for non-TestCase methods. This low > coupling/high cohesion suggests organizing them together, but currently > there's no convenient way to get at them separately. And since the > methods really have little to do with an object instance, I find the > 'self.assert*' construct distracting, although I like self.* in most cases. I don't understand what you're after here, but if it's about making unittest a package and distributing the various bits in submodules of the package with the __init__.py file importing all of them... that's a way to approach the problem (if there is any ;-). I usually refactor modules that way as soon as they become too large to handle -- however, most of the times this is more an editing problem than a real need for a distributed design :-) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/ From josh@janrain.com Fri May 31 22:26:39 2002 From: josh@janrain.com (Josh Hoyt) Date: 31 May 2002 14:26:39 -0700 Subject: [Python-Dev] bug: mro, subclassing and equality Message-ID: <1022880405.1530.9.camel@rye> Hello, I found a bug in Python 2.2.1, that I think is related to method resolution order and type initialization. I'm pretty new to Python, so bear with me while I try to explain. The following code: class Nanite(type): key = "blargchoo" def __eq__(self, other): return self.key == other.key def crashbang(): """KILL!""" c1 = type("molly", (Nanite,), {}) p1 = type('parent1', (), {}) p2 = type('parent2', (), {}) c1('child', (p1, p2), {}) causes the interpreter to seg fault. GDB tells me that it's crashing in typeobject.c, at line 1217. I made the following change, based on a bit of copy-coding, without actually tracing the full execution path, and it seemed to fix the problem, but hopefully someone here knows enough that they can tell whether I just made the symptom go away or this actually fixes the problem: *************** *** 1213,1224 **** /* Look in tp_dict of types in MRO */ mro = type->tp_mro; ! if (mro == NULL) { ! n = 0; ! } else { ! assert(PyTuple_Check(mro)); ! n = PyTuple_GET_SIZE(mro); ! } for (i = 0; i < n; i++) { base = PyTuple_GET_ITEM(mro, i); if (PyClass_Check(base)) --- 1213,1220 ---- /* Look in tp_dict of types in MRO */ mro = type->tp_mro; ! assert(PyTuple_Check(mro)); ! n = PyTuple_GET_SIZE(mro); for (i = 0; i < n; i++) { base = PyTuple_GET_ITEM(mro, i); if (PyClass_Check(base)) Thanks, Josh Hoyt Software Engineer JanRain, Inc. From guido@python.org Fri May 31 22:44:30 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 31 May 2002 17:44:30 -0400 Subject: [Python-Dev] bug: mro, subclassing and equality In-Reply-To: Your message of "31 May 2002 14:26:39 PDT." <1022880405.1530.9.camel@rye> References: <1022880405.1530.9.camel@rye> Message-ID: <200205312144.g4VLiUa18847@pcp742651pcs.reston01.va.comcast.net> > I found a bug in Python 2.2.1, that I think is related to method > resolution order and type initialization. I'm pretty new to Python, so > bear with me while I try to explain. This has already been fixed in CVS, both for Python 2.2.2 and for Python 2.3. See python.org/sf/551412. --Guido van Rossum (home page: http://www.python.org/~guido/) From David Abrahams" A few questions about customization and the docs, TIA: ------ http://www.python.org/dev/doc/devel/ref/customization.html tosses off this sentence in the section on rich comparisons: "A rich comparison method may return NotImplemented if it does not implement the operation for a given pair of arguments." ...but it says nothing about the effect that will have. Using Thomas Heller's search engine, I found http://www.python.org/dev/doc/devel/ref/types.html#l2h-59, which says "...Numeric methods and rich comparison methods may return this value if they do not implement the operation for the operands provided. (The interpreter will then try the reflected operation, or some other fallback, depending on the operator.) " This seems like the wrong place to bury the description of the mechanism for operating on heterogeneous arguments. Shouldn't a short description go into the Customization section of the docs? ------- It also mentions, "There are no reflected (swapped-argument) versions of these methods (to be used when the left argument does not support the operation but the right argument does)..." since these are the first binary operators described the whole concept of reflected methods hasn't been introduced yet, and the user has no reason to think that reflected versions should exist. [incidentally, "reflected" and "reflection" produce no hits with Thomas' engine] ------- >From what I could find in the docs, it's completely non-obvious how the following works for immutable objects in containers: >>> x = [ 1, 2, 3] >>> x[1] += 3 >>> x [1, 5, 3] Is the sequence of operations described someplace? How does Python decide that sequence elements are immutable? +---------------------------------------------------------------+ David Abrahams C++ Booster (http://www.boost.org) O__ == Pythonista (http://www.python.org) c/ /'_ == resume: http://users.rcn.com/abrahams/resume.html (*) \(*) == email: david.abrahams@rcn.com +---------------------------------------------------------------+ From Ilya Osipov" Dear Colleagues, Are you interested in insects? Then you should check out our site "Russian Butterflies" at "http://osipov.org/insects/" There are more than 3,000 insects listed in our price list: Lepidoptera, Coleoptera, Diptera, Hymenoptera, Hemiptera, Odonata and others from Europe, Russia, China, Pakistan, Taiwan, Thailand, Nepal, etc. There is also a huge online information, database of Russian butterflies. Many specimens photos are available. WEBMASTERS AND SITE OWNERS! I am glad to suggest to you new possibilities for advertising your site or homepage on "Russian butterflies" site. There are several ways to do that: 1. Add your message to "INSECTS FORUM" http://osipov.org/insects/forum (FREE) 2. Add your insect site to "TOP 100 INSECTS SITES" http://osipov.org/insects/top_list (FREE) 3. Add your link to http://osipov.org/insects/links.htm (FREE) To do that add our link to your site and reply to this message with URL of your site. 4. If you want your banner to be placed on each page of our site ("Russian
    butterflies" site has about 900 pages and about 1200 page views per day), please visit "http://osipov.org/insects/form.html" Sincerely yours, Ilya Osipov (Ph. D.) osipov@osipov.org http://www.osipov.org __________________________________________________________________________ Your email address was obtained from an opt-in list, Reference # 10023392. If you do not interested in insects, please resend this letter to your friends whom interesting in it. If you do not want to receive "RUSSIAN BUTTERFLIES NEWS" in future -send empty mail with Subject=UNSUBSCRIBE for i.osipov@mtu-net.ru