From greg.ewing at canterbury.ac.nz Tue Aug 1 02:30:09 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 01 Aug 2006 12:30:09 +1200 Subject: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers) In-Reply-To: <44CE4D32.6040909@ewtllc.com> References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com> <44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com> Message-ID: <44CEA091.1010406@canterbury.ac.nz> Raymond Hettinger wrote: > I think this would harm more than it would help. It more confusing to > have several rounding-thingies to choose from than it is have an > explicit two-step. But is it more confusing enough to be worth forcing everyone to pay two function calls instead of one in the most common case? If I'm right that rounding-to-int is much more commonly needed than rounding-to-float, the least confusing thing would be for the builtin round() to return an int, and have something somewhere else, such as math.fround(), for round-to-float. > BTW, I thought the traditional idiom (for positive numbers) was: > int(x+.5) It's the "for positive numbers" that's the problem there. Most of my uses for round() involve graphics coordinates, which I can't be sure won't be negative. It's not immediately obvious what this will do with negative numbers. It's not even immediately obvious that it's doing round-to-nearest-integer unless you're familiar with the idiom. A single well-named function would be much more explicit. -- Greg From mal at egenix.com Tue Aug 1 11:37:50 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 01 Aug 2006 11:37:50 +0200 Subject: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers) In-Reply-To: <44CEA091.1010406@canterbury.ac.nz> References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com> <44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com> <44CEA091.1010406@canterbury.ac.nz> Message-ID: <44CF20EE.70103@egenix.com> Greg Ewing wrote: > Raymond Hettinger wrote: > >> I think this would harm more than it would help. It more confusing to >> have several rounding-thingies to choose from than it is have an >> explicit two-step. > > But is it more confusing enough to be worth forcing > everyone to pay two function calls instead of one > in the most common case? I suppose you don't know about the optional argument to round that lets you round up to a certain decimal ?! If we were to follow your suggestion, we'd have round() return an integer when called without the argument and a float when called with the argument. Dependency of the return type on arguments to a function should be avoided, if possible. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 01 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Tue Aug 1 11:50:19 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 01 Aug 2006 11:50:19 +0200 Subject: [Python-Dev] Which version of distutils to ship with Python 2.5? In-Reply-To: <44CCB0A5.30706@v.loewis.de> References: <43aa6ff70607261924w65e01325o854f87d4c109026a@mail.gmail.com> <44C85FF2.3030701@v.loewis.de> <200607271701.58223.anthony@interlink.com.au> <44CCB0A5.30706@v.loewis.de> Message-ID: <44CF23DB.90909@egenix.com> Martin v. L?wis wrote: > Anthony Baxter schrieb: >>> In any case, I bumped the version number to 2.5, according to the >>> policy discussed in >>> >> Could this not simply use the Python version number directly, instead? > > See the prior discussion at > > http://mail.python.org/pipermail/distutils-sig/2005-January/004366.html > > Some people still believe (at least, believed in January 2005), that > distutils is developed independently of Python, and thus deserves > its own version number. They still do. Even if distutils is part of the stdlib, it is a software package that's intended to be used for more than just the latest Python release. The fact that were no official separate distutils releases shouldn't be regarded as meaning that there's no interest in using distutils with older Python versions. > Of course, Andrew Kuchling officially declared in r1982 of pep 291 > that there won't be any further stand-alone distutils releases, > and therefore, backwards compatibility with 2.1 is not necessary > anymore. AFAIK, there was no public discussion about this on the distutils SIG list. While I don't think that Python 2.1 compatibility is still a requirement, Python 2.3 compatibility should be maintained in order to make SVN distutils work with all versions of Python since 2.3. distutils should then be added back to PEP 291. > So I changed distutils.__version__ again, to be derived from > sys.version_info. See the discussion on the checkins list for why this will only cause problems instead of fixing any. http://mail.python.org/pipermail/python-checkins/2006-July/055139.html > I left the numerous comments still in distutils that compatibility > with 2.1 is desired. We should remove these after 2.5 is released > (or perhaps even before that). Please don't. Instead we should have a discussion about this whole issue on the distutils list and then decide. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 01 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From ncoghlan at gmail.com Tue Aug 1 13:00:33 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 01 Aug 2006 21:00:33 +1000 Subject: [Python-Dev] Bad interaction of __index__ and sequence repeat In-Reply-To: <44CE4BB9.60903@ieee.org> References: <20060728101133.GA339@code0.codespeak.net> <44CB6B7D.1040001@gmail.com> <44CB92F1.9040304@iinet.net.au> <44CE4BB9.60903@ieee.org> Message-ID: <44CF3451.4060500@gmail.com> Travis Oliphant wrote: >> Probably the most interesting thing now would be for Travis to review >> it, and see whether it makes things easier to handle for the Numeric >> scalar types (given the amount of code the patch deleted from the >> builtin and standard library data types, hopefully the benefits to >> Numeric will be comparable). > > > I noticed most of the checks for PyInt where removed in the patch. If I > remember correctly, I left these in for "optimization." Other than > that, I think the patch is great. You're right - there was a fast path based on PyInt_Check in _PyEval_SliceIndex that got lost, which I'll add back in. I'll also add fast paths for PyInt_Check to the functions in abstract.c, too. The other PyInt_Check's (in slot_nb_index and instance_index) were there to check that __index__ returned the right thing. The check was still there in slot_nb_index, but I'd incorrectly removed it from instance_index. I'll add that one back in, too. Once that's done, I'll update the tracker item and reassign to Tim for a review. Cheers, Nick. > As far as helping with NumPy, I think it will help to be able to remove > special-checks for all the different integer-types. But, this has not > yet been done in the NumPy code. > > -Travis > > > -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From greg.ewing at canterbury.ac.nz Tue Aug 1 13:19:03 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 01 Aug 2006 23:19:03 +1200 Subject: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers) In-Reply-To: <44CF20EE.70103@egenix.com> References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com> <44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com> <44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com> Message-ID: <44CF38A7.8040304@canterbury.ac.nz> M.-A. Lemburg wrote: > I suppose you don't know about the optional argument > to round that lets you round up to a certain decimal ?! Yes, I know about, but I rarely if ever use it. Rounding a binary float to a number of decimal places seems a fundamentally ill-considered thing to do anyway. What are the use cases for it, given that one can easily select a number of decimal places when formatting a number for display? > If we were to follow your suggestion, we'd have round() > return an integer when called without the argument and > a float when called with the argument. No, round() wouldn't have that option at all. If you wanted it, you would use fround() instead, which would have the option and return a float always. This would be a Py3k thing, obviously. If done before then, the new function would have to be given a different name. -- Greg From mal at egenix.com Tue Aug 1 13:40:06 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 01 Aug 2006 13:40:06 +0200 Subject: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers) In-Reply-To: <44CF38A7.8040304@canterbury.ac.nz> References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com> <44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com> <44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com> <44CF38A7.8040304@canterbury.ac.nz> Message-ID: <44CF3D96.7090101@egenix.com> Greg Ewing wrote: > M.-A. Lemburg wrote: > >> I suppose you don't know about the optional argument >> to round that lets you round up to a certain decimal ?! > > Yes, I know about, but I rarely if ever use it. > Rounding a binary float to a number of decimal > places seems a fundamentally ill-considered thing > to do anyway. What are the use cases for it, > given that one can easily select a number of decimal > places when formatting a number for display? You often have a need for controlled rounding when doing financial calculations or in situations where you want to compare two floats with a given accuracy, e.g. to work around rounding problems ;-) The usual approach is to use full float accuracy throughout the calculation and then apply rounding a certain key places. Float formatting is an entirely different issue. >> If we were to follow your suggestion, we'd have round() >> return an integer when called without the argument and >> a float when called with the argument. > > No, round() wouldn't have that option at all. If > you wanted it, you would use fround() instead, > which would have the option and return a float > always. > > This would be a Py3k thing, obviously. If done > before then, the new function would have to be > given a different name. Hmm, looks like a YAGNI to me, but perhaps I'm missing something :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 01 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From nmm1 at cus.cam.ac.uk Tue Aug 1 15:12:38 2006 From: nmm1 at cus.cam.ac.uk (Nick Maclaren) Date: Tue, 01 Aug 2006 14:12:38 +0100 Subject: [Python-Dev] Rounding float to int directly ... Message-ID: "M.-A. Lemburg" wrote: > > You often have a need for controlled rounding when doing > financial calculations or in situations where you want to > compare two floats with a given accuracy, e.g. to work > around rounding problems ;-) The latter is a crude hack, and was traditionally used to save cycles when floating-point division was very slow. There are better ways, and have been for decades. > Float formatting is an entirely different issue. Not really. You need controlled rounding to a fixed precision in the other base. But I agree that controlled rounding in binary does not help with controlled rounding in decimal. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1 at cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679 From aahz at pythoncraft.com Tue Aug 1 16:09:24 2006 From: aahz at pythoncraft.com (Aahz) Date: Tue, 1 Aug 2006 07:09:24 -0700 Subject: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers) In-Reply-To: <44CF3D96.7090101@egenix.com> References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com> <44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com> <44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com> <44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com> Message-ID: <20060801140923.GA19963@panix.com> On Tue, Aug 01, 2006, M.-A. Lemburg wrote: > > You often have a need for controlled rounding when doing financial > calculations or in situations where you want to compare two floats > with a given accuracy, e.g. to work around rounding problems ;-) > > The usual approach is to use full float accuracy throughout the > calculation and then apply rounding a certain key places. That's what Decimal() is for. (Note that I don't care all that much about round(), but I do think we want to canonicalize Decimal() for all purposes in standard Python where people care about accuracy. Additional float features can go into NumPy.) -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." --Brian W. Kernighan From nmm1 at cus.cam.ac.uk Tue Aug 1 16:53:14 2006 From: nmm1 at cus.cam.ac.uk (Nick Maclaren) Date: Tue, 01 Aug 2006 15:53:14 +0100 Subject: [Python-Dev] Rounding float to int directly ... Message-ID: Aahz wrote: > On Tue, Aug 01, 2006, M.-A. Lemburg wrote: > > > > You often have a need for controlled rounding when doing financial > > calculations or in situations where you want to compare two floats > > with a given accuracy, e.g. to work around rounding problems ;-) > > > > The usual approach is to use full float accuracy throughout the > > calculation and then apply rounding a certain key places. > > That's what Decimal() is for. Well, maybe. There are other approaches, too, and Decimal has its problems with that. In particular, when people need precisely defined decimal rounding, they ALWAYS need fixed-point and not floating-point. > (Note that I don't care all that much about round(), but I do think we > want to canonicalize Decimal() for all purposes in standard Python where > people care about accuracy. Additional float features can go into > NumPy.) Really? That sounds like dogma, not science. Decimal doesn't even help people who care about accuracy. At most (and with the reservation mentioned above), it means that you can can map external decimal formats to internal ones without loss of precision. Not a big deal, as there ARE no requirements for doing that for floating-point, and there are plenty of other solutions for fixed-point. People who care about the accuracy of calculations prefer binary, as it is a more accurate model. That isn't a big deal, either. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1 at cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679 From rhettinger at ewtllc.com Tue Aug 1 17:21:15 2006 From: rhettinger at ewtllc.com (Raymond Hettinger) Date: Tue, 01 Aug 2006 08:21:15 -0700 Subject: [Python-Dev] Rounding float to int directly ... In-Reply-To: References: Message-ID: <44CF716B.6000603@ewtllc.com> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060801/0beb65ca/attachment.html From nmm1 at cus.cam.ac.uk Tue Aug 1 17:37:29 2006 From: nmm1 at cus.cam.ac.uk (Nick Maclaren) Date: Tue, 01 Aug 2006 16:37:29 +0100 Subject: [Python-Dev] Rounding float to int directly ... Message-ID: Raymond Hettinger wrote: > > Hogwash.  The only issues with decimal are ease-of-use and speed.
I suggest that you get hold of a good 1960s or 1970s book on computer arithmetic, and read up about "wobbling precision". While it is not a big deal, it was regarded as such, and is important enough to cause significant numerical problems to the unwary - which means 99.99% of modern programmers :-( And, as I am sure that Aahz could point out, there are significant reliability issues concerned with frequent base changes where any loss of precision is unacceptable. Yes, it can always be done, but only a few people are likely to do it correctly in all cases. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1 at cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679 From ncoghlan at gmail.com Tue Aug 1 17:42:46 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 02 Aug 2006 01:42:46 +1000 Subject: [Python-Dev] Bad interaction of __index__ and sequence repeat In-Reply-To: <44CF3451.4060500@gmail.com> References: <20060728101133.GA339@code0.codespeak.net> <44CB6B7D.1040001@gmail.com> <44CB92F1.9040304@iinet.net.au> <44CE4BB9.60903@ieee.org> <44CF3451.4060500@gmail.com> Message-ID: <44CF7676.2010102@gmail.com> Nick Coghlan wrote: > Travis Oliphant wrote: >>> Probably the most interesting thing now would be for Travis to review >>> it, and see whether it makes things easier to handle for the Numeric >>> scalar types (given the amount of code the patch deleted from the >>> builtin and standard library data types, hopefully the benefits to >>> Numeric will be comparable). >> >> I noticed most of the checks for PyInt where removed in the patch. If I >> remember correctly, I left these in for "optimization." Other than >> that, I think the patch is great. > > You're right - there was a fast path based on PyInt_Check in > _PyEval_SliceIndex that got lost, which I'll add back in. I'll also add fast > paths for PyInt_Check to the functions in abstract.c, too. > > The other PyInt_Check's (in slot_nb_index and instance_index) were there to > check that __index__ returned the right thing. The check was still there in > slot_nb_index, but I'd incorrectly removed it from instance_index. I'll add > that one back in, too. > > Once that's done, I'll update the tracker item and reassign to Tim for a review. Aside from the 'assign to Tim' part, this is done now (pep357-fix-v3 on the tracker item). I also tweaked the clipping version of the C API to optionally expose the overflow flag that abstract.c was using internally so the client code can find out whether or not clipping actually occurred. The patch also updates the index unit tests to: - cover the various type errors that Travis picked up - actually run the full set of unit tests under -O - only run the basic sequence independent tests once (instead of once per sequence type) Neal also had a number of questions that I responded to in the tracker comments. The most significant question related to the API signature, which Neal considered potentially confusing: PyNumber_Index(PyObject *item, int *is_index) I added the is_index flag to the function signatures instead of providing a separate PyNumber_IndexCheck function in order to avoid doing the same type check twice in the typical mp_subscript implementation cases. One possibility would be to invert the sense of that flag and call it "typeerror", which probably more accurately reflects what it's intended for - it's a way of telling the function "if this object does not have the correct type, tell me by setting this flag instead of by setting the Python error state". Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From skip at pobox.com Tue Aug 1 20:25:56 2006 From: skip at pobox.com (skip at pobox.com) Date: Tue, 1 Aug 2006 13:25:56 -0500 Subject: [Python-Dev] clock_gettime() vs. gettimeofday()? Message-ID: <17615.40116.461692.83632@montanaro.dyndns.org> This has probably been discussed before, however ... Is there any reason to use clock_gettime() in preference to gettimeofday() if it exists? It pretends to at least return seconds + nanoseconds, where gettimeofday() returns seconds + microseconds. Are there perhaps some platforms where it's possible to actually get more useful bits? On my Solaris 10/Intel desktop box clock_getres shows 0.01s resolution (not obviously better) while on a Linux box here it shows 0.001s resolution. Is that better than what gettimeofday() does? Skip From rrr at ronadam.com Tue Aug 1 23:07:57 2006 From: rrr at ronadam.com (Ron Adam) Date: Tue, 01 Aug 2006 16:07:57 -0500 Subject: [Python-Dev] Rounding float to int directly ... In-Reply-To: References: Message-ID: <44CFC2AD.2070205@ronadam.com> Nick Maclaren wrote: > "M.-A. Lemburg" wrote: >> You often have a need for controlled rounding when doing >> financial calculations or in situations where you want to >> compare two floats with a given accuracy, e.g. to work >> around rounding problems ;-) > > The latter is a crude hack, and was traditionally used to save cycles > when floating-point division was very slow. There are better ways, > and have been for decades. > >> Float formatting is an entirely different issue. > > Not really. You need controlled rounding to a fixed precision in > the other base. But I agree that controlled rounding in binary > does not help with controlled rounding in decimal. I'm -1 on implicitly converting to an int when rounding. One reason is if your rounded (to int type) number is then used in an equation you my end up with a integer result when you wanted a floating point result. >>> 23.5/5.2 4.5192307692307692 >>> round(23.5)/round(5.2) 4.7999999999999998 >>> round(23.5/5.2) 5.0 >>> int(round(23.5))/int(round(5.2)) 4 Which one of these is correct probably depends on what you are using the result for. If it's an intermediate calculation you may not want an integer (type) result just yet. Rounding is used in various differing ways in regard to significant digits and accuracy. While it seems like there should be one easy way of handling rounding and significant digits, that would be too narrow a view. See the following page (for starters) to see various uses and rules of significant digits and rounding and how they change depending on the type of calculation and weather or not its a pure or exact number: http://www.epcc.edu/faculty/victors/sigfig.htm I think it's better to view that the common operation (being discussed) is rounding when converting to an int, and not converting to an int when rounding. So the standard idiom mentioned earlier by Raymond ... int(x+.5) If it could be made faster by having a round argument on int for such operations, it may be worth while. int(x, round=mode) Where mode could be one of 'up','down','standard','toeven','toodd', or whatever are useful methods that might be used. If there is no performance advantage to combining the two, I think it would be better to keep them separate. So I'm -0 on giving the int constructor a rounding argument. Just my 2 cents, Ron From t-bruch at microsoft.com Tue Aug 1 23:52:42 2006 From: t-bruch at microsoft.com (Bruce Christensen) Date: Tue, 1 Aug 2006 14:52:42 -0700 Subject: [Python-Dev] Strange memo behavior from cPickle Message-ID: <3581AA168D87A2479D88EA319BDF7D3201069E15@RED-MSG-80.redmond.corp.microsoft.com> We seem to have stumbled upon some strange behavior in cPickle's memo use when pickling instances. Here's the repro: [mymodule.py] class C: def __getstate__(self): return ('s1', 's2', 's3') [interactive interpreter] Python 2.4.3 (#69, Mar 29 2006, 17:35:34) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import cPickle >>> import mymodule >>> class C: ... def __getstate__(self): return ('s1', 's2', 's3') ... >>> for x in mymodule.C(), C(): cPickle.dumps(x) ... "(imymodule\nC\np1\n(S's1'\nS's2'\np2\nS's3'\np3\ntp4\nb." "(i__main__\nC\np1\n(S's1'\nS's2'\nS's3'\ntp2\nb." >>> Note that the second and third strings in the instance's state are memoized in the first case, but not in the second. Any idea why this occurs (and why the first element is never memoized)? --Bruce From tim.peters at gmail.com Wed Aug 2 02:40:13 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 1 Aug 2006 20:40:13 -0400 Subject: [Python-Dev] Strange memo behavior from cPickle In-Reply-To: <3581AA168D87A2479D88EA319BDF7D3201069E15@RED-MSG-80.redmond.corp.microsoft.com> References: <3581AA168D87A2479D88EA319BDF7D3201069E15@RED-MSG-80.redmond.corp.microsoft.com> Message-ID: <1f7befae0608011740t41dc76cchb86124144375fc00@mail.gmail.com> [Bruce Christensen] > We seem to have stumbled upon some strange behavior in cPickle's memo > use when pickling instances. > > Here's the repro: > > [mymodule.py] > class C: > def __getstate__(self): return ('s1', 's2', 's3') > > [interactive interpreter] > Python 2.4.3 (#69, Mar 29 2006, 17:35:34) [MSC v.1310 32 bit (Intel)] on > win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> import cPickle > >>> import mymodule > >>> class C: > ... def __getstate__(self): return ('s1', 's2', 's3') > ... > >>> for x in mymodule.C(), C(): cPickle.dumps(x) > ... > "(imymodule\nC\np1\n(S's1'\nS's2'\np2\nS's3'\np3\ntp4\nb." > "(i__main__\nC\np1\n(S's1'\nS's2'\nS's3'\ntp2\nb." > >>> > > Note that the second and third strings in the instance's state are > memoized in the first case, but not in the second. Any idea why this > occurs (and why the first element is never memoized)? Ideally, a pickle would never contain a `PUT i` unless i was referenced by a `GET i` later. So, ideally, there would be no PUT opcodes in either of these pickles. cPickle is a little bit smarter than pickle.py here, in that cPickle suppresses a PUT if the reference count on the object is less than 2 (in which case the structure being pickled can't possibly reference the sub-object a second time, so it's impossible that a later GET will want to reference the same sub-object). So all you're seeing here is refcount accidents, complicated by accidents concerning exactly which strings get interned. Use pickle.py instead (which doesn't do this refcount micro-optimization), and you'll see the same number of PUTs in both. They're all correct. What would be incorrect is seeing a `GET i` without a preceding `PUT i` using the same `i`. From greg.ewing at canterbury.ac.nz Wed Aug 2 03:02:49 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 02 Aug 2006 13:02:49 +1200 Subject: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers) In-Reply-To: <44CF3D96.7090101@egenix.com> References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com> <44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com> <44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com> <44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com> Message-ID: <44CFF9B9.3010402@canterbury.ac.nz> M.-A. Lemburg wrote: > You often have a need for controlled rounding when doing > financial calculations You should NOT be using binary floats for money in the first place. > or in situations where you want to > compare two floats with a given accuracy, Pseudo-rounding to decimal places is not the right way to do that. The right way is to compare the difference to a tolerance. > e.g. to work > around rounding problems ;-) As Tim Peters has pointed out many times, you can't fix binary/decimal representation problems by rounding. There are always cases that give a different result than you would want. > Hmm, looks like a YAGNI to me, In all my programming so far I've found numerous uses for round-to-int, and exactly zero uses for round-binary-float-to-decimal- places. So from my point of view, the YAGNI applies to the *current* behavour of round()! -- Greg From simon at arrowtheory.com Wed Aug 2 03:58:06 2006 From: simon at arrowtheory.com (Simon Burton) Date: Wed, 2 Aug 2006 11:58:06 +1000 Subject: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers) In-Reply-To: <44CAD9A1.8080000@canterbury.ac.nz> References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com> <44CAD9A1.8080000@canterbury.ac.nz> Message-ID: <20060802115806.7d568586.simon@arrowtheory.com> On Sat, 29 Jul 2006 15:44:33 +1200 Greg Ewing wrote: > Michael Urman wrote: > > The fact that > > round returns a float rather than an int, while intentional, does not > > feature prominently in one's mine when the first version yielded the > > expected results. > > As an aside, does anyone else think that it would be > useful to have a builtin which rounds and converts to > an int in one go? Whenever I use round(), I almost > always want the result as an int, and making me do > it in two steps seems unnecessarily bothersome. It's not even clear to me that int(round(x)) is always the nearest integer to x. Is it always true that float(some_int)>=some_int ? (for positive values). (ie. I am wondering if truncating the float representation of an int always gives back the original int). Simon. -- Simon Burton, B.Sc. Licensed PO Box 8066 ANU Canberra 2601 Australia Ph. 61 2 6249 6940 http://arrowtheory.com From nnorwitz at gmail.com Wed Aug 2 06:36:25 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 1 Aug 2006 21:36:25 -0700 Subject: [Python-Dev] internal weakref API should be Py_ssize_t? Message-ID: I'm wondering if the following change should be made to Include/weakrefobject.h: -PyAPI_FUNC(long) _PyWeakref_GetWeakrefCount(PyWeakReference *head); +PyAPI_FUNC(Py_ssize_t) _PyWeakref_GetWeakrefCount(PyWeakReference *head); And the 2 other files which use this (weakref obj and module). Should this be changed now, wait for 2.6, or not worry about it? n From tim.peters at gmail.com Wed Aug 2 06:53:59 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 2 Aug 2006 00:53:59 -0400 Subject: [Python-Dev] internal weakref API should be Py_ssize_t? In-Reply-To: References: Message-ID: <1f7befae0608012153y7eacfbceke93fcbe1d1c72e32@mail.gmail.com> [Neal Norwitz] > I'm wondering if the following change should be made to Include/weakrefobject.h: Yes. > -PyAPI_FUNC(long) _PyWeakref_GetWeakrefCount(PyWeakReference *head); > +PyAPI_FUNC(Py_ssize_t) _PyWeakref_GetWeakrefCount(PyWeakReference *head); > > And the 2 other files which use this (weakref obj and module). Should > this be changed now, wait for 2.6, or not worry about it? +1 on biting the bullet for 2.5. From fdrake at acm.org Wed Aug 2 07:24:44 2006 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Wed, 2 Aug 2006 01:24:44 -0400 Subject: [Python-Dev] internal weakref API should be Py_ssize_t? In-Reply-To: <1f7befae0608012153y7eacfbceke93fcbe1d1c72e32@mail.gmail.com> References: <1f7befae0608012153y7eacfbceke93fcbe1d1c72e32@mail.gmail.com> Message-ID: <200608020124.44806.fdrake@acm.org> Neal Norwitz wrote: > I'm wondering if the following change should be made to > Include/weakrefobject.h: On Wednesday 02 August 2006 00:53, Tim Peters wrote: > Yes. ... > +1 on biting the bullet for 2.5. Agreed. This should definately go with the rest of the Py_ssize_t changes. -Fred -- Fred L. Drake, Jr. From kbk at shore.net Wed Aug 2 07:49:06 2006 From: kbk at shore.net (Kurt B. Kaiser) Date: Wed, 2 Aug 2006 01:49:06 -0400 (EDT) Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200608020549.k725n6cP029299@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 396 open ( -5) / 3354 closed (+12) / 3750 total ( +7) Bugs : 864 open (-32) / 6087 closed (+52) / 6951 total (+20) RFE : 226 open ( +2) / 234 closed ( +1) / 460 total ( +3) New / Reopened Patches ______________________ Move firewall warning to "about" menu (2006-07-26) http://python.org/sf/1529018 opened by Tal Einat Allowing multiple instances of IDLE with sub-processes (2006-07-26) http://python.org/sf/1529142 opened by Tal Einat Squeezer - squeeze large output in the interpreter (2006-07-27) http://python.org/sf/1529353 opened by Tal Einat More openbsd targets for ctypes (2006-07-27) CLOSED http://python.org/sf/1529514 opened by Thomas Heller test_defaultdict not integrated into test sute (2006-07-27) CLOSED http://python.org/sf/1529686 opened by ?iga Seilnacht misleading info about tarfile.py's "r|*" mode (2006-07-27) CLOSED http://python.org/sf/1529811 opened by Lars Gust?bel pydoc render_doc (2006-07-28) http://python.org/sf/1530482 opened by ganges master test_email_codecs not integrated into test sute (2006-07-27) CLOSED http://python.org/sf/1529686 reopened by zseil Fix __index__() clipping really big numbers (2006-07-29) http://python.org/sf/1530738 opened by Nick Coghlan "a + (2006-07-29) CLOSED http://python.org/sf/1531113 opened by Christopher Tur Lesniewski-Laas Tracing and profiling functions can cause hangs in threads (2006-07-31) http://python.org/sf/1531859 opened by Matt Fleming Support for case-insensitivity in string.Template (2006-07-25) CLOSED http://python.org/sf/1528167 reopened by whit537 Patches Closed ______________ Expose case-insensitivity of string.Template (2006-07-25) http://python.org/sf/1528167 closed by bwarsaw Fix for #1513611 and #1511497; xml.sax imports (2006-07-10) http://python.org/sf/1519796 closed by zseil More openbsd targets for ctypes (2006-07-27) http://python.org/sf/1529514 closed by theller test_defaultdict not integrated into test sute (2006-07-27) http://python.org/sf/1529686 closed by gbrandl misleading info about tarfile.py's "r|*" mode (2006-07-27) http://python.org/sf/1529811 closed by akuchling Support for PyGetSetDefs in pydoc, inspect, and types (2006-07-10) http://python.org/sf/1520294 closed by bwarsaw Fix socketmodule compile on NetBSD (2006-07-21) http://python.org/sf/1526460 closed by splitscreen test_email_codecs not integrated into test sute (2006-07-27) http://python.org/sf/1529686 closed by gbrandl Describe Py_DEBUG and friends... (2006-05-18) http://python.org/sf/1490989 closed by akuchling os.path.exists returns False if no permission (2004-11-17) http://python.org/sf/1068277 closed by akuchling "a + (2006-07-29) http://python.org/sf/1531113 closed by nnorwitz Fix zip file header format in zipfile.py (2003-07-30) http://python.org/sf/780595 closed by gbrandl Don't produce core file in test_subprocess.py (2006-07-11) http://python.org/sf/1520905 closed by akuchling New / Reopened Bugs ___________________ readline module description should mention raw_input() (2006-07-26) CLOSED http://python.org/sf/1529157 opened by Thom Harp possible bug in mystrtol.c with recent gcc (2006-07-13) CLOSED http://python.org/sf/1521947 reopened by nnorwitz unrelated variable messing up doctests (2006-07-26) CLOSED http://python.org/sf/1529297 reopened by macquigg BROWSER path with capital letters (2006-07-28) http://python.org/sf/1529655 opened by Ken McGaugh doc string of read-only properties (2006-07-27) CLOSED http://python.org/sf/1529717 opened by Michael Amrhein distutils regression related to Distribution.run_command (2006-07-27) CLOSED http://python.org/sf/1529871 opened by Collin Winter bz2 lib missing from source distribution (2006-07-27) CLOSED http://python.org/sf/1529998 opened by james yoo Mac Universal Build of Python confuses distutils (2006-07-28) http://python.org/sf/1530142 opened by Richard Jones ssl object documentation lacks a couple of methods (2006-07-28) CLOSED http://python.org/sf/1530382 opened by Lawrence Oluyede file.seek() influelce write() when opened with a+ mode (2006-07-12) http://python.org/sf/1521491 reopened by rudnik_lior ctypes build fails on Solaris 10 (2006-07-28) http://python.org/sf/1530448 opened by Skip Montanaro struct.pack raises TypeError where it used to convert (2006-07-28) http://python.org/sf/1530559 opened by Joe Wreschnig distutils doesn't notice --with-pydebug (2006-07-29) http://python.org/sf/1530959 opened by Collin Winter __weaklist__ in typeobject.c (2006-07-29) http://python.org/sf/1531003 opened by Fred L. Drake, Jr. Comman not allowed at the end of argument list for **argumen (2006-07-29) http://python.org/sf/1531016 opened by Roman Suzi Reflected (swapped) operations overridden by subclasses (2006-07-31) CLOSED http://python.org/sf/1531349 opened by Andrei Polushin format_exception raises if str(exception) raises (2006-07-30) http://python.org/sf/1531405 opened by Jp Calderone parsetok.c emits warnings by writing to stderr (2006-07-30) http://python.org/sf/1531415 opened by Jp Calderone Build fails on MacOSX with missing symbol (2006-07-31) http://python.org/sf/1531662 opened by gideon may HTTPSConnection request hangs (2006-07-31) http://python.org/sf/1531775 opened by Noam Taich subprocess.Popen(cmd, stdout=sys.stdout) fails (2006-07-31) http://python.org/sf/1531862 opened by John A Meinel SocketServer.TCPServer returns different ports (2006-07-31) http://python.org/sf/1531963 opened by hakker.de the csv module writes files that Excel sees as SYLK files (2006-08-01) http://python.org/sf/1532483 opened by Vincent Povirk incorrect behaviour of PyUnicode_EncodeMBCS? (2006-08-01) http://python.org/sf/1532726 opened by Jan-Willem Bugs Closed ___________ Build fails on OS X with case sensitive fs (2006-07-19) http://python.org/sf/1525447 closed by ronaldoussoren readline module description should mention raw_input() (2006-07-26) http://python.org/sf/1529157 closed by akuchling possible bug in mystrtol.c with recent gcc (2006-07-13) http://python.org/sf/1521947 closed by nnorwitz xml.sax.ParseException weirdness in python 2.5b1 (2006-06-27) http://python.org/sf/1513611 closed by fdrake Python 2.4c1 test_univnewlines.py fails within IDLE IDE (2004-11-19) http://python.org/sf/1069433 closed by kbk unrelated variable messing up doctests (2006-07-26) http://python.org/sf/1529297 closed by nnorwitz unrelated variable messing up doctests (2006-07-26) http://python.org/sf/1529297 closed by tim_one wrong handling of character buffers in stringobject (2006-07-01) http://python.org/sf/1515471 closed by nnorwitz xml.sax.expatreader is missing (2006-06-23) http://python.org/sf/1511497 closed by fdrake FTP modue functions are not re-entrant,give odd exceptions (2006-04-13) http://python.org/sf/1469557 closed by gbrandl turtle.py Docs still incomplete (2006-07-09) http://python.org/sf/1519571 closed by akuchling doc string of read-only properties (2006-07-27) http://python.org/sf/1529717 closed by gbrandl distutils regression related to Distribution.run_command (2006-07-27) http://python.org/sf/1529871 closed by pje bz2 lib missing from source distribution (2006-07-27) http://python.org/sf/1529998 closed by nnorwitz missing module names in email package (2005-12-27) http://python.org/sf/1391608 closed by bwarsaw mimetools message's To field can't be changed (2006-06-28) http://python.org/sf/1513913 closed by bwarsaw Docs on import of email.MIMExxx classes wrong (2006-05-04) http://python.org/sf/1481650 closed by bwarsaw email.Utils.py: UnicodeError in RFC2322 header (2006-01-24) http://python.org/sf/1414018 closed by bwarsaw ssl object documentation lacks a couple of methods (2006-07-28) http://python.org/sf/1530382 closed by akuchling new.function() docs out of date (2003-11-03) http://python.org/sf/835255 closed by gbrandl compiler module loses docstrings (2006-03-02) http://python.org/sf/1441397 closed by gbrandl inconsistency in help(set) (2006-01-25) http://python.org/sf/1414697 closed by akuchling xmlcore needs to be documented (2006-06-11) http://python.org/sf/1504456 closed by fdrake urllib2 data argument (2006-07-25) http://python.org/sf/1528258 closed by akuchling Finish PEP 343 terminology cleanup (2006-04-26) http://python.org/sf/1476845 closed by ncoghlan set documentation deficiencies (2006-02-10) http://python.org/sf/1429053 closed by akuchling Weakref types documentation bugs (2005-05-02) http://python.org/sf/1193966 closed by fdrake suspect documentation of operator.isNumberType (2004-01-17) http://python.org/sf/878908 closed by gbrandl Wrong documentation of __radd__ etc. (2004-08-03) http://python.org/sf/1002453 closed by gbrandl Documented grammar for List displays incorrect (testlist) (2004-11-22) http://python.org/sf/1071248 closed by gbrandl os.path.sameopenfile documentation wrong. (2004-08-03) http://python.org/sf/1002398 closed by gbrandl mode 't' not documented as posssible mode for file built-in (2005-12-15) http://python.org/sf/1381717 closed by gbrandl Strange os.path.exists() results with invalid chars (2005-04-30) http://python.org/sf/1193180 closed by gbrandl popen3 on windows loses environment variables (2005-01-13) http://python.org/sf/1101667 closed by gbrandl Reflected (swapped) operations overridden by subclasses (2006-07-30) http://python.org/sf/1531349 closed by gbrandl String Methods 2.3.6.1 not in TOC (2006-06-29) http://python.org/sf/1514540 closed by akuchling PEP 307 pickle extensions not documented (2004-07-04) http://python.org/sf/984952 closed by akuchling 4.2.6 (re) Examples: float regexp exponential on failure (2003-11-24) http://python.org/sf/848556 closed by akuchling pydoc fails on package in ZIP archive (2006-06-15) http://python.org/sf/1506951 closed by gbrandl Installing 2.3b2 on to non-system disk fails (2003-07-03) http://python.org/sf/764975 closed by ronaldoussoren New / Reopened RFE __________________ distutils 'register' command and windows home directories (2006-07-30) http://python.org/sf/1531505 opened by Josiah Carlson RFE Closed __________ No simple example for pickle (2006-03-01) http://python.org/sf/1441072 closed by gbrandl From theller at python.net Wed Aug 2 10:00:25 2006 From: theller at python.net (Thomas Heller) Date: Wed, 02 Aug 2006 10:00:25 +0200 Subject: [Python-Dev] Releasemanager, please approve #1532975 Message-ID: Approval requested for patch: http://python.org/sf/1532975 Thanks, Thomas From nmm1 at cus.cam.ac.uk Wed Aug 2 11:09:30 2006 From: nmm1 at cus.cam.ac.uk (Nick Maclaren) Date: Wed, 02 Aug 2006 10:09:30 +0100 Subject: [Python-Dev] Rounding float to int directly ... Message-ID: Greg Ewing wrote: > > You should NOT be using binary floats for money > in the first place. Or floating-point at all, actually. But binary floating-point is definitely unsuited for such a use. > Pseudo-rounding to decimal places is not > the right way to do that. The right way is > to compare the difference to a tolerance. Right. Where the tolerance should be a combination of relative and absolute accuracy. 1.0e-300 should usually be 'similar' to 0.0. Simon Burton wrote: > > It's not even clear to me that int(round(x)) is always the > nearest integer to x. There is a sense in which this is either true or overflow occurs. > Is it always true that float(some_int)>=some_int ? (for positive values). > > (ie. I am wondering if truncating the float representation > of an int always gives back the original int). No. Consider 'standard' Python representations on a 64-bit system. There are only 53 bits in the mantissa, but an integer can have up to 63. Very large integers need to be rounded, and can be rounded up or down. Please note that I am not arguing against an int_rounded() function. There is as much reason to want one as an int_truncated() one, but there is no very good reason to to want more than one of the two. int_expanded() [i.e. ceiling] is much less useful. For people interested in historical trivia, the dominance of the truncating form of integer conversion over the rounding form seems to be yet another side-effect of the Fortran / IBM 370 dominance over the Algol / other hardware, despite the fact that most modern languages are rooted in CPL rather than Fortran. I am unaware of any technical grounds to prefer one over the other (i.e. the reasons for wanting each are equally balanced). It all comes down to the simple question "Do we regard a single primitive for int(round()) as important enough to provide?" I abstain :-) Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1 at cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679 From mcherm at mcherm.com Wed Aug 2 12:53:38 2006 From: mcherm at mcherm.com (Michael Chermside) Date: Wed, 02 Aug 2006 03:53:38 -0700 Subject: [Python-Dev] Rounding float to int directly ... Message-ID: <20060802035338.fxzcn0gcvsv4kso4@login.werra.lunarpages.com> Greg Ewing writes: > [I propose for Py3k that] builtin round() > to return an int, and have something somewhere > else, such as math.fround(), for round-to-float. +1. The need to convert float to int is pervasive. The need to round (binary) floats to integer (or other decimal) floats is _much_ less common. Furthermore, the need for "round to nearest int" is much more common than the need for "integer floor" (although as pointed out, each could be defined in terms of the other except for certain rules for rounding exact halves). And most telling of all, if I were to stop 10 random programmers on the street, most of them not familiar with Python, and ask them to guess what the function named "round" did, I'm guessing the majority would think of float-to-int. It's a nice feature of Python that "it just works". Marc-Andre Lemburg writes: > You often have a need for controlled rounding when doing > financial calculations or [other reason snipped] Hmm. Not in the banks _I_ have worked at! We *never* use binary floating point for money. The decimal class is fairly useful in that regard. Nick Maclaren write: > Decimal doesn't even help people who care about accuracy. Not true! The float class is incapable of maintaining 700 digits of precision, but Decimal handles it just fine. (Why you would WANT more than around 53 bits of precision is a different question, but Decimal handles it.) > People who care about the accuracy of calculations prefer binary, > as it is a more accurate model. Huh? That doesn't even make sense! A model is not inherently accurate or inaccurate, it is only an accurate or inaccurate representation of some "real" system. Neither binary nor decimal is a better representation of either rational or real numbers, the first candidates for "real" system I thought of. Financial accounting rules tend to be based on paper-and-pencil calculations for which decimal is usually a far better representation. If you had said that binary floats squeeze out more digits of precision per bit of storage than decimal floats, or that binary floats are faster because they are supported by specialized hardware, then I'd go along, but they're not a "better model". -- Michael Chermside From Sjaak.van.Werkhoven at nl.fortis.com Wed Aug 2 13:01:36 2006 From: Sjaak.van.Werkhoven at nl.fortis.com (Werkhoven J.P. van (Sjaak)) Date: Wed, 2 Aug 2006 13:01:36 +0200 Subject: [Python-Dev] FW: using globals Message-ID: > Hi, > > I have got a problem with importing global variables. For instance I have > got two files: > > # t1.py #t2.py > > counter = 1 > > def counter_adder(filenr): def > show_adder(): > global counter import > t1 > counter+= 1 > print("adder is %d" % t1.counter) > if filenr == 1: > show_adder() > else: > import t2 > t2.show_adder() > > def show_adder(): > print("adder is %d" % counter) > > >>counter_adder(1) > adder is 2 > >>counter_adder(2) > adder is 1 > > When I look at the outcome of two function calls I expected no differences > but much to my surprise it goes wrong when the global variable is > imported. It doesn't give the updated value but the value it get when > instantiated. Who come? What should I do to get the updated value > > greetings, > > Sjaak van Werkhoven > > ****************************DISCLAIMER*********************************** Deze e-mail is uitsluitend bestemd voor de geadresseerde(n). Verstrekking aan en gebruik door anderen is niet toegestaan. Fortis sluit iedere aansprakelijkheid uit die voortvloeit uit electronische verzending. This e-mail is intended exclusively for the addressee(s), and may not be passed on to, or made available for use by any person other than the addressee(s). Fortis rules out any and every liability resulting from any electronic transmission. ****************************************************************************** From nmm1 at cus.cam.ac.uk Wed Aug 2 13:03:53 2006 From: nmm1 at cus.cam.ac.uk (Nick Maclaren) Date: Wed, 02 Aug 2006 12:03:53 +0100 Subject: [Python-Dev] Rounding float to int directly ... Message-ID: Michael Chermside wrote: > > > Decimal doesn't even help people who care about accuracy. > > Not true! The float class is incapable of maintaining 700 digits of > precision, but Decimal handles it just fine. (Why you would WANT > more than around 53 bits of precision is a different question, but > Decimal handles it.) Oh, yes, the CURRENT decimal class is potentially more accurate than the CURRENT floating class, but that has nothing to do with the intrinsic differences in the base. > > People who care about the accuracy of calculations prefer binary, > > as it is a more accurate model. > > Huh? That doesn't even make sense! A model is not inherently accurate > or inaccurate, it is only an accurate or inaccurate representation > of some "real" system. Neither binary nor decimal is a better > representation of either rational or real numbers, the first > candidates for "real" system I thought of. Financial accounting rules > tend to be based on paper-and-pencil calculations for which > decimal is usually a far better representation. > > If you had said that binary floats squeeze out more digits of > precision per bit of storage than decimal floats, or that binary > floats are faster because they are supported by specialized hardware, > then I'd go along, but they're not a "better model". No, that isn't true. The "wobbling precision" effect may be overstated, but is real, and gets worse the larger the base is. To the best of my knowledge, that is almost the only way in which binary is more accurate than decimal, in absolute terms, and it is a marginal difference. Note that I said "prefer", not "require" :-) For example, calculating the relative difference between two close numbers is sensitive to whether you are using the numbers in their normal or inverse forms (by a factor on N in base N), and this is a common cause of incorrect answers. A factor of 2 is better than one of 10. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1 at cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679 From mcherm at mcherm.com Wed Aug 2 13:18:34 2006 From: mcherm at mcherm.com (Michael Chermside) Date: Wed, 02 Aug 2006 04:18:34 -0700 Subject: [Python-Dev] Rounding float to int directly ... Message-ID: <20060802041834.1400idlbu6o0wk08@login.werra.lunarpages.com> Nick Maclaren: > The "wobbling precision" effect may be overstated, > but is real, and gets worse the larger the base is. To the best of my > knowledge, that is almost the only way in which binary is more accurate > than decimal, in absolute terms, and it is a marginal difference. Okay, that makes sense. However, if I understand "wobbling precision" correctly (and it's a BIG if) then it refers to the fact that interpolating from the last digit of precision doesn't work so well (intentionally vague phrase). So I would think (but could be wrong) that it is always trumped by adding another digit of precision to your numbers / lookup tables / whatever. If so, then even more credit needs to be given to the system that supports adjustable precision (the Decimal module). -- Michael Chermside From ronaldoussoren at mac.com Wed Aug 2 13:30:57 2006 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Wed, 02 Aug 2006 13:30:57 +0200 Subject: [Python-Dev] FW: using globals In-Reply-To: References: Message-ID: <13237035.1154518257044.JavaMail.ronaldoussoren@mac.com> On Wednesday, August 02, 2006, at 01:02PM, Werkhoven J.P. van (Sjaak) wrote: >> Hi, >> >> I have got a problem with importing global variables. For instance I have >> got two files: Your message is off-topic for python-dev, this list is for the development OF python, not development WITH python. You're much likier to get an answer on python-list or comp.lang.python (which are the same). BTW. The formatting of the source-code in you're e-mail was such that I have no idea how it is supposed to look :-( Regards, Ronald From ncoghlan at gmail.com Wed Aug 2 14:02:41 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 02 Aug 2006 22:02:41 +1000 Subject: [Python-Dev] Rounding float to int directly ... In-Reply-To: <20060802035338.fxzcn0gcvsv4kso4@login.werra.lunarpages.com> References: <20060802035338.fxzcn0gcvsv4kso4@login.werra.lunarpages.com> Message-ID: <44D09461.7060308@gmail.com> Michael Chermside wrote: > Greg Ewing writes: >> [I propose for Py3k that] builtin round() >> to return an int, and have something somewhere >> else, such as math.fround(), for round-to-float. > > +1. > > The need to convert float to int is pervasive. The need to round (binary) > floats to integer (or other decimal) floats is _much_ less common. > Furthermore, the need for "round to nearest int" is much more common > than the need for "integer floor" (although as pointed out, each could > be defined in terms of the other except for certain rules for rounding > exact halves). And most telling of all, if I were to stop 10 random > programmers on the street, most of them not familiar with Python, and > ask them to guess what the function named "round" did, I'm guessing > the majority would think of float-to-int. It's a nice feature of Python > that "it just works". The potential issue with accidentally triggering truncating division also goes away in Py3k, so that's another point in favour of switching round() to return an integer result. Another option would be to provide this as a method of float objects (similar to decimal). > Nick Maclaren write: >> Decimal doesn't even help people who care about accuracy. > > Not true! The float class is incapable of maintaining 700 digits of > precision, but Decimal handles it just fine. (Why you would WANT > more than around 53 bits of precision is a different question, but > Decimal handles it.) The decimal module's default settings actually provide 93 bits or so of precision (28 decimal digits). Sure, there may still be cases where you have to care about dynamic range, but if two numbers are separated by 28 orders of magnitude the smaller one really is pretty damn close to zero as far as the second one is concerned (in a fixed point system, the second one usually *would* be zero). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From mal at egenix.com Wed Aug 2 14:28:12 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 02 Aug 2006 14:28:12 +0200 Subject: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers) In-Reply-To: <44CFF9B9.3010402@canterbury.ac.nz> References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com> <44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com> <44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com> <44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com> <44CFF9B9.3010402@canterbury.ac.nz> Message-ID: <44D09A5C.4030905@egenix.com> Greg Ewing wrote: > M.-A. Lemburg wrote: > >> You often have a need for controlled rounding when doing >> financial calculations > > You should NOT be using binary floats for money > in the first place. Believe me: you have to if you want to do more advanced calculus related to pricing and risk analysis of derivatives. >> or in situations where you want to >> compare two floats with a given accuracy, > > Pseudo-rounding to decimal places is not > the right way to do that. The right way is > to compare the difference to a tolerance. This is not the same: if you round both value and then compare, you test a different interval than by taking the difference and applying a tolerance value comparison: >>> x = 1.12 >>> y = 1.124999 >>> round(x,2) == round(y,2) True >>> abs(x-y) < 1e-2 True yet: >>> y = 1.1275 >>> round(x,2) == round(y,2) False >>> abs(x-y) < 1e-2 True >> e.g. to work >> around rounding problems ;-) > > As Tim Peters has pointed out many times, > you can't fix binary/decimal representation > problems by rounding. There are always cases > that give a different result than you would > want. Right, but in most real-world cases it works :-) >> Hmm, looks like a YAGNI to me, > > In all my programming so far I've found > numerous uses for round-to-int, and exactly > zero uses for round-binary-float-to-decimal- > places. So from my point of view, the YAGNI > applies to the *current* behavour of round()! Most typical uses of round() don't use the optional argument, true, but I still fail to see what returning an integer instead of a float would buy you. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 02 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From anthony at interlink.com.au Wed Aug 2 14:29:44 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed, 2 Aug 2006 22:29:44 +1000 Subject: [Python-Dev] TRUNK FREEZE 2006-07-03, 00:00 UTC for 2.5b3 Message-ID: <200608022229.46957.anthony@interlink.com.au> The trunk is FROZEN from 00:00 UTC/GMT on 03 July 2006. That's in about 12 hours from now. This is for the third (and final, hopefully!) beta of 2.5. Please help the release team out by not making checkins on the trunk after that time until I send out an email that the release is done. Thanks, Anthony -- Anthony Baxter It's never too late to have a happy childhood. From exarkun at divmod.com Wed Aug 2 14:36:54 2006 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Wed, 2 Aug 2006 08:36:54 -0400 Subject: [Python-Dev] TRUNK FREEZE 2006-07-03, 00:00 UTC for 2.5b3 In-Reply-To: <200608022229.46957.anthony@interlink.com.au> Message-ID: <20060802123654.1717.1171821069.divmod.quotient.12149@ohm> On Wed, 2 Aug 2006 22:29:44 +1000, Anthony Baxter wrote: >The trunk is FROZEN from 00:00 UTC/GMT on 03 July 2006. That's in about 12 >hours from now. This is for the third (and final, hopefully!) beta of 2.5. > >Please help the release team out by not making checkins on the trunk after >that time until I send out an email that the release is done. Does this mean that there is no further opportunity to resolve regressions such as , or will changes be allowed between b3 and final? Jean-Paul From skip at pobox.com Wed Aug 2 14:47:32 2006 From: skip at pobox.com (skip at pobox.com) Date: Wed, 2 Aug 2006 07:47:32 -0500 Subject: [Python-Dev] TRUNK FREEZE 2006-07-03, 00:00 UTC for 2.5b3 In-Reply-To: <20060802123654.1717.1171821069.divmod.quotient.12149@ohm> References: <200608022229.46957.anthony@interlink.com.au> <20060802123654.1717.1171821069.divmod.quotient.12149@ohm> Message-ID: <17616.40676.620565.678809@montanaro.dyndns.org> >> Please help the release team out by not making checkins on the trunk >> after that time until I send out an email that the release is done. Jean-Paul> Does this mean that there is no further opportunity to Jean-Paul> resolve regressions such as Jean-Paul> , Jean-Paul> or will changes be allowed between b3 and final? The latter. One would hope bugfixes only. Skip From mal at egenix.com Wed Aug 2 14:48:13 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 02 Aug 2006 14:48:13 +0200 Subject: [Python-Dev] Rounding float to int directly ... In-Reply-To: <20060802035338.fxzcn0gcvsv4kso4@login.werra.lunarpages.com> References: <20060802035338.fxzcn0gcvsv4kso4@login.werra.lunarpages.com> Message-ID: <44D09F0D.5090802@egenix.com> Michael Chermside wrote: > Marc-Andre Lemburg writes: >> You often have a need for controlled rounding when doing >> financial calculations or [other reason snipped] > > Hmm. Not in the banks _I_ have worked at! We *never* use binary > floating point for money. The decimal class is fairly useful in > that regard. For accounting, I agree - banks would rather not lose too many cents :-) However, banks also tend to do more exotic things, esp. in investment banking, where prices are not determined by taking the sum of a list of values, but rather turn out to be the result of some calculus. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 02 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From bjourne at gmail.com Wed Aug 2 18:06:40 2006 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Wed, 2 Aug 2006 18:06:40 +0200 Subject: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers) In-Reply-To: <44D09A5C.4030905@egenix.com> References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com> <44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com> <44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com> <44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com> <44CFF9B9.3010402@canterbury.ac.nz> <44D09A5C.4030905@egenix.com> Message-ID: <740c3aec0608020906q437927ddp175d6843719d26d0@mail.gmail.com> On 8/2/06, M.-A. Lemburg wrote: > Greg Ewing wrote: > > In all my programming so far I've found > > numerous uses for round-to-int, and exactly > > zero uses for round-binary-float-to-decimal- > > places. So from my point of view, the YAGNI > > applies to the *current* behavour of round()! > > Most typical uses of round() don't use the > optional argument, true, but I still fail > to see what returning an integer instead of > a float would buy you. One hundred and sixty two type conversions! http://www.google.se/search?hl=sv&q=filetype%3Apy+%22int%28round%22&btnG=S%C3%B6k&meta= -- mvh Bj?rn From nnorwitz at gmail.com Wed Aug 2 19:26:21 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 2 Aug 2006 10:26:21 -0700 Subject: [Python-Dev] TRUNK FREEZE 2006-07-03, 00:00 UTC for 2.5b3 In-Reply-To: <20060802123654.1717.1171821069.divmod.quotient.12149@ohm> References: <200608022229.46957.anthony@interlink.com.au> <20060802123654.1717.1171821069.divmod.quotient.12149@ohm> Message-ID: This looks like it needs to be fixed as it's a regression. As Skip said, bug fixes are allowed. There will still be at least one release candidate. It's looking like c1 will be around Aug 18 and final around Sept 12. I'll update the PEP when I get a chance and confirm all the dates. n -- On 8/2/06, Jean-Paul Calderone wrote: > On Wed, 2 Aug 2006 22:29:44 +1000, Anthony Baxter wrote: > >The trunk is FROZEN from 00:00 UTC/GMT on 03 July 2006. That's in about 12 > >hours from now. This is for the third (and final, hopefully!) beta of 2.5. > > > >Please help the release team out by not making checkins on the trunk after > >that time until I send out an email that the release is done. > > Does this mean that there is no further opportunity to resolve regressions > such as , or will changes be allowed between b3 and final? > > Jean-Paul > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com > From guido at python.org Wed Aug 2 19:58:20 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 2 Aug 2006 10:58:20 -0700 Subject: [Python-Dev] Bad interaction of __index__ and sequence repeat In-Reply-To: <44CF7676.2010102@gmail.com> References: <20060728101133.GA339@code0.codespeak.net> <44CB6B7D.1040001@gmail.com> <44CB92F1.9040304@iinet.net.au> <44CE4BB9.60903@ieee.org> <44CF3451.4060500@gmail.com> <44CF7676.2010102@gmail.com> Message-ID: All, please decide without me. On 8/1/06, Nick Coghlan wrote: > Nick Coghlan wrote: > > Travis Oliphant wrote: > >>> Probably the most interesting thing now would be for Travis to review > >>> it, and see whether it makes things easier to handle for the Numeric > >>> scalar types (given the amount of code the patch deleted from the > >>> builtin and standard library data types, hopefully the benefits to > >>> Numeric will be comparable). > >> > >> I noticed most of the checks for PyInt where removed in the patch. If I > >> remember correctly, I left these in for "optimization." Other than > >> that, I think the patch is great. > > > > You're right - there was a fast path based on PyInt_Check in > > _PyEval_SliceIndex that got lost, which I'll add back in. I'll also add > fast > > paths for PyInt_Check to the functions in abstract.c, too. > > > > The other PyInt_Check's (in slot_nb_index and instance_index) were there > to > > check that __index__ returned the right thing. The check was still there > in > > slot_nb_index, but I'd incorrectly removed it from instance_index. I'll > add > > that one back in, too. > > > > Once that's done, I'll update the tracker item and reassign to Tim for a > review. > > Aside from the 'assign to Tim' part, this is done now (pep357-fix-v3 on the > tracker item). I also tweaked the clipping version of the C API to > optionally > expose the overflow flag that abstract.c was using internally so the client > code can find out whether or not clipping actually occurred. > > The patch also updates the index unit tests to: > - cover the various type errors that Travis picked up > - actually run the full set of unit tests under -O > - only run the basic sequence independent tests once (instead of once per > sequence type) > > Neal also had a number of questions that I responded to in the tracker > comments. > > The most significant question related to the API signature, which Neal > considered potentially confusing: > PyNumber_Index(PyObject *item, int *is_index) > > I added the is_index flag to the function signatures instead of providing a > separate PyNumber_IndexCheck function in order to avoid doing the same type > check twice in the typical mp_subscript implementation cases. > > One possibility would be to invert the sense of that flag and call it > "typeerror", which probably more accurately reflects what it's intended for > - > it's a way of telling the function "if this object does not have the correct > type, tell me by setting this flag instead of by setting the Python error > state". > > Regards, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > --------------------------------------------------------------- > http://www.boredomandlaziness.org > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From oliphant.travis at ieee.org Wed Aug 2 21:15:24 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 02 Aug 2006 13:15:24 -0600 Subject: [Python-Dev] Bad interaction of __index__ and sequence repeat In-Reply-To: <44CF7676.2010102@gmail.com> References: <20060728101133.GA339@code0.codespeak.net> <44CB6B7D.1040001@gmail.com> <44CB92F1.9040304@iinet.net.au> <44CE4BB9.60903@ieee.org> <44CF3451.4060500@gmail.com> <44CF7676.2010102@gmail.com> Message-ID: Nick Coghlan wrote: > > > One possibility would be to invert the sense of that flag and call it > "typeerror", which probably more accurately reflects what it's intended for - > it's a way of telling the function "if this object does not have the correct > type, tell me by setting this flag instead of by setting the Python error state". +1 on changing the name (type_error might be a better spelling) -Travis From simon at arrowtheory.com Thu Aug 3 03:39:26 2006 From: simon at arrowtheory.com (Simon Burton) Date: Thu, 3 Aug 2006 11:39:26 +1000 Subject: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers) In-Reply-To: <324634B71B159D469BCEB616678A6B94F94B1A@ingdexs1.ingdirect.com> References: <324634B71B159D469BCEB616678A6B94F94B1A@ingdexs1.ingdirect.com> Message-ID: <20060803113926.39fd374a.simon@arrowtheory.com> On Wed, 2 Aug 2006 07:21:02 -0400 "Chermside, Michael" wrote: > > You wrote: > > (ie. I am wondering if truncating the float representation > > of an int always gives back the original int). > > Yes it does, because all integers that can be expressed as floats > can be expressed EXACTLY as floats. Yes, OK this makes sense. My point is that, regardless of this answer, as a python user I should not be worried by such details. At least: the doc string for round() should be expanded to mention the int(round(x)) use case. Simon. -- Simon Burton, B.Sc. Licensed PO Box 8066 ANU Canberra 2601 Australia Ph. 61 2 6249 6940 http://arrowtheory.com From greg.ewing at canterbury.ac.nz Thu Aug 3 04:03:19 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 03 Aug 2006 14:03:19 +1200 Subject: [Python-Dev] Rounding float to int directly ... In-Reply-To: References: Message-ID: <44D15967.5090603@canterbury.ac.nz> Nick Maclaren wrote: > I am unaware of > any technical grounds to prefer one over the other (i.e. the reasons > for wanting each are equally balanced). If they're equally balanced in the sense of there being equal numbers of use cases for both, that would seem to be a reason to *have* both. What's the feeling about this? If, e.g. int() were changed in Py3k to round instead of truncate, would it cause anyone substantial pain? -- Greg From greg.ewing at canterbury.ac.nz Thu Aug 3 04:14:53 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 03 Aug 2006 14:14:53 +1200 Subject: [Python-Dev] Rounding float to int directly ... In-Reply-To: <20060802035338.fxzcn0gcvsv4kso4@login.werra.lunarpages.com> References: <20060802035338.fxzcn0gcvsv4kso4@login.werra.lunarpages.com> Message-ID: <44D15C1D.5030800@canterbury.ac.nz> Michael Chermside wrote: > (Why you would WANT > more than around 53 bits of precision is a different question, but > Decimal handles it.) You can't have travelled far beyond Alpha Centauri recently. The major interstellar banking corporations need *quite* a lot of bits for dealing with the economy of a medium-sized galaxy. The Decimal module has been a godsend to them -- they were all making do with long integers until that came along. Of course they've had to make some modifications to handle the number bases used by some of the more unusual species out there, such as the 13-fingered Quezorpians, but that's just a simple matter of programming. Always remember where your towel is, Zaphod From greg.ewing at canterbury.ac.nz Thu Aug 3 04:21:05 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 03 Aug 2006 14:21:05 +1200 Subject: [Python-Dev] Rounding float to int directly ... In-Reply-To: <44D09461.7060308@gmail.com> References: <20060802035338.fxzcn0gcvsv4kso4@login.werra.lunarpages.com> <44D09461.7060308@gmail.com> Message-ID: <44D15D91.6000401@canterbury.ac.nz> Nick Coghlan wrote: > Another option would be to provide this as a method of float objects > (similar to decimal). That wouldn't be so good. Either kind of rounding ought to be a function that you can apply without knowing what kind of number you've got -- int, float, Decimal, or something else. -- Greg From greg.ewing at canterbury.ac.nz Thu Aug 3 04:30:12 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 03 Aug 2006 14:30:12 +1200 Subject: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers) In-Reply-To: <44D09A5C.4030905@egenix.com> References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com> <44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com> <44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com> <44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com> <44CFF9B9.3010402@canterbury.ac.nz> <44D09A5C.4030905@egenix.com> Message-ID: <44D15FB4.5020605@canterbury.ac.nz> M.-A. Lemburg wrote: > Believe me: you have to if you want to do more > advanced calculus related to pricing and risk > analysis of derivatives. When you do things like that, you're treating money as though it were a continuous quantity. This is an approximation, so you can tolerate having small inaccuracies in the result. But when adding up actual, real amounts of money, where the result must be exact, using binary fractions is a very bad idea. > This is not the same: if you round both value > and then compare, you test a different interval > than by taking the difference and applying > a tolerance value comparison: That's exactly my point. Chopping decimal places is not a substitute for doing tolerance testing properly. > Most typical uses of round() don't use the > optional argument, true, but I still fail > to see what returning an integer instead of > a float would buy you. It saves you a function call in the vast majority of cases, where an int is what you ultimately want. -- Greg From bob at redivi.com Thu Aug 3 05:20:40 2006 From: bob at redivi.com (Bob Ippolito) Date: Wed, 2 Aug 2006 20:20:40 -0700 Subject: [Python-Dev] struct module and coercing floats to integers In-Reply-To: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com> References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com> Message-ID: On Jul 28, 2006, at 1:35 PM, Bob Ippolito wrote: > It seems that the pre-2.5 struct module has some additional > undocumented behavior[1] that didn't percolate into the new version: > http://python.org/sf/1530559 > > Python 2.4 and previous will coerce floats to integers when necessary > as such without any kind of complaint: > > $ python2.4 -c "import struct; print repr(struct.pack('>H', > 0.9999999999999999))" > '\x00\x00' > > Python 2.5 refuses to coerce float to int: > > $ python2.5 -c "import struct; print repr(struct.pack('>H', > 0.9999999999999999))" > Traceback (most recent call last): > File "", line 1, in > File "/Users/bob/src/python/Lib/struct.py", line 63, in pack > return o.pack(*args) > TypeError: unsupported operand type(s) for &: 'float' and 'long' > > The available options are to: > > 1. Reinstate the pre-2.5 weirdness > 2. Reinstate the pre-2.5 weirdness with a DeprecationWarning > 3. Break existing code that relies on undocumented behavior (seems > more like a bug than lack of specification) There's a patch in the tracker for 2. It should get applied when the trunk freeze is over. -bob From rhettinger at ewtllc.com Thu Aug 3 05:26:55 2006 From: rhettinger at ewtllc.com (Raymond Hettinger) Date: Wed, 02 Aug 2006 20:26:55 -0700 Subject: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers) In-Reply-To: <44D15FB4.5020605@canterbury.ac.nz> References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com> <44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com> <44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com> <44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com> <44CFF9B9.3010402@canterbury.ac.nz> <44D09A5C.4030905@egenix.com> <44D15FB4.5020605@canterbury.ac.nz> Message-ID: <44D16CFF.7090001@ewtllc.com> >> Most typical uses of round() don't use the >> optional argument, true, but I still fail >> to see what returning an integer instead of >> a float would buy you. > > > It saves you a function call in the vast > majority of cases, where an int is what > you ultimately want. > -1 on an extra built-in just to save the time for function call -- this is a false optimization, one that is unlikely to ever be the bottleneck in any real program's running time. If it isn't time you care about but just hate writing it, what's wrong with a simple helper function like: rndint=lambda x: int(round(x)) ? There are no shortage of possible function pairs that could be proposed (perhaps list(set(x)) or int(abs(x)) for example). Of course, it would be silly to make new builtins for them. IMHO, the case is weak for adding a new variant of round(). Also, -10 on changing the semantics of int() to round instead of truncate. The truncating version is found is so many other languages and book examples, that it would be a disaster for us to choose a different meaning. Raymond From foom at fuhm.net Thu Aug 3 05:39:43 2006 From: foom at fuhm.net (James Y Knight) Date: Wed, 2 Aug 2006 23:39:43 -0400 Subject: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers) In-Reply-To: <44D16CFF.7090001@ewtllc.com> References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com> <44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com> <44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com> <44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com> <44CFF9B9.3010402@canterbury.ac.nz> <44D09A5C.4030905@egenix.com> <44D15FB4.5020605@canterbury.ac.nz> <44D16CFF.7090001@ewtllc.com> Message-ID: <9E3C8812-877A-4923-A4AD-884B81AE1CDF@fuhm.net> On Aug 2, 2006, at 11:26 PM, Raymond Hettinger wrote: > Also, -10 on changing the semantics of int() to round instead of > truncate. The truncating version is found is so many other languages > and book examples, that it would be a disaster for us to choose a > different meaning. I'd be happy to see floats lose their __int__ method entirely, replaced by an explicit truncate function. I've always thought it quite a hack that python floats have implicit truncation to ints, and then a random smattering of APIs go to extra lengths to explicitly prevent float.__int__ from being called because people thought "passing a float makes no sense!". That's right, it doesn't, and it _never_ should happen implicitly, not just in those particular few cases. Explicit is better than implicit. James From greg.ewing at canterbury.ac.nz Thu Aug 3 08:34:26 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 03 Aug 2006 18:34:26 +1200 Subject: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers) In-Reply-To: <44D16CFF.7090001@ewtllc.com> References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com> <44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com> <44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com> <44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com> <44CFF9B9.3010402@canterbury.ac.nz> <44D09A5C.4030905@egenix.com> <44D15FB4.5020605@canterbury.ac.nz> <44D16CFF.7090001@ewtllc.com> Message-ID: <44D198F2.9020702@canterbury.ac.nz> Raymond Hettinger wrote: > -1 on an extra built-in just to save the time for function call The time isn't the main issue. The main issue is that almost all the use cases for round() involve doing an int() on it afterwards. At least nobody has put forward an argument to the contrary yet. Sure you can define your own function to do this. But that's still a considerable burden when you add up the effort over all the programs that use int(round()). -- Greg From tanzer at swing.co.at Thu Aug 3 08:34:31 2006 From: tanzer at swing.co.at (Christian Tanzer) Date: Thu, 03 Aug 2006 08:34:31 +0200 Subject: [Python-Dev] Rounding float to int directly ... In-Reply-To: Your message of "Thu, 03 Aug 2006 14:03:19 +1200." <44D15967.5090603@canterbury.ac.nz> Message-ID: Greg Ewing wrote: > What's the feeling about this? If, e.g. int() > were changed in Py3k to round instead of truncate, > would it cause anyone substantial pain? Gratuitous breakage! I shudder at the thought of checking hundreds of int-calls to see if they'd still be correct under such a change. -- Christian Tanzer http://www.c-tanzer.at/ From mal at egenix.com Thu Aug 3 10:32:54 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 03 Aug 2006 10:32:54 +0200 Subject: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers) In-Reply-To: <44D15FB4.5020605@canterbury.ac.nz> References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com> <44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com> <44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com> <44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com> <44CFF9B9.3010402@canterbury.ac.nz> <44D09A5C.4030905@egenix.com> <44D15FB4.5020605@canterbury.ac.nz> Message-ID: <44D1B4B6.2080403@egenix.com> Greg Ewing wrote: > M.-A. Lemburg wrote: > >> Believe me: you have to if you want to do more >> advanced calculus related to pricing and risk >> analysis of derivatives. > > When you do things like that, you're treating > money as though it were a continuous quantity. > This is an approximation, so you can tolerate > having small inaccuracies in the result. > > But when adding up actual, real amounts of > money, where the result must be exact, using > binary fractions is a very bad idea. Agreed. Those are different use cases, though. >> This is not the same: if you round both value >> and then compare, you test a different interval >> than by taking the difference and applying >> a tolerance value comparison: > > That's exactly my point. Chopping decimal > places is not a substitute for doing tolerance > testing properly. As I said: those two are different ways of doing comparisons. If you are eventually rounding to say 2 decimal places in the end of the calculation, you won't want to find yourself presenting the user 1.12 and 1.13 as equal values :-) >> Most typical uses of round() don't use the >> optional argument, true, but I still fail >> to see what returning an integer instead of >> a float would buy you. > > It saves you a function call in the vast > majority of cases, where an int is what > you ultimately want. -1 on that. Just saving a single function call isn't really enough to break all sorts of applications out there that use round() in calculations (with or without argument). Most such calculations do work with floats, so having round() return an int would just add another coercion to a float for those use-cases. Seriously, rounding is a complicated business. round() just implements one way of doing it. Perhaps a module providing different rounding models would be of use ?! The decimal module already has a few of those implemented, so it could benefit from such a module as well. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 03 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From rrr at ronadam.com Thu Aug 3 10:31:58 2006 From: rrr at ronadam.com (Ron Adam) Date: Thu, 03 Aug 2006 03:31:58 -0500 Subject: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers) In-Reply-To: <44D198F2.9020702@canterbury.ac.nz> References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com> <44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com> <44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com> <44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com> <44CFF9B9.3010402@canterbury.ac.nz> <44D09A5C.4030905@egenix.com> <44D15FB4.5020605@canterbury.ac.nz> <44D16CFF.7090001@ewtllc.com> <44D198F2.9020702@canterbury.ac.nz> Message-ID: <44D1B47E.5020506@ronadam.com> Greg Ewing wrote: > Raymond Hettinger wrote: > >> -1 on an extra built-in just to save the time for function call > > The time isn't the main issue. The main issue > is that almost all the use cases for round() > involve doing an int() on it afterwards. At > least nobody has put forward an argument to > the contrary yet. I'll try to give a valid argument of the contrary. I think you are only considering part of the usefulness of round(). In statistics and science math equations, rounding isn't used to get an integer, but instead used to give an answer that is meaningful within the margin of error of the data being used. Consider an example where you are combining data that had different number of significant digits. Keeping all the digits of your answer gives a false since of accuracy. The extra digits are meaningless because the margin of error is greater than any of the digits that exceed the minimum number of significant digits in your data. So you need to remove the extraneous digits by rounding. Then this answer can be further used as data, and sense the number of significant digits is maintained, the margin of error can be calculated accurately. [ NOTE: While floating point may contribute a significant error to value that are very large or very small, in most cases it is a very small amount in relation to the precision of the calculation and so the error can be managed without too much trouble. In cases where it is a significant factor, another internal representation should probably be considered. I only mention it here because someone would point it out (again) anyway. The accuracy of floating point is not the issue being discussed in this thread. ] It makes since for round have an argument to specify the number of digits of precision to round to and also for it to return floating point numbers because rounding results to a float of n digits of precision is a necessary operation. If you have the default round() case of 0 (and negative) digits of precision return integers, you then have a function that may sometimes returns integers, and sometimes returns floats. This can be problematic if the values are further used in division operations. Which will result in many cases of.. float(round(x)) in order to convert integer results back to floats. See pep-0238: Changing the division operator. http://www.python.org/dev/peps/pep-0238/ While it is true we often will use round along with converting to an integer, it is also true that functions that are predictable and return a single type are easier to use and learn. So the question of which is better is a matter of point of view in this respect. And in regard to using round() combined with division operators in floating point math, it is an issue of least surprises. > Sure you can define your own function to do > this. But that's still a considerable burden > when you add up the effort over all the > programs that use int(round()). I think the only time it would be a burden is if a single program uses int(round()) many times, otherwise it is only a very small additional amount to type for each program. In the case of single program using it many times, a helper function is a very practical solution. Cheers, Ron From nmm1 at cus.cam.ac.uk Thu Aug 3 10:35:23 2006 From: nmm1 at cus.cam.ac.uk (Nick Maclaren) Date: Thu, 03 Aug 2006 09:35:23 +0100 Subject: [Python-Dev] Rounding float to int directly ... Message-ID: James Y Knight wrote: > > I'd be happy to see floats lose their __int__ method entirely, > replaced by an explicit truncate function. Come back Algol - all is forgiven :-) Yes, indeed. I have favoured that view for 35 years - anything that can lose information quietly should be explicit. tanzer at swing.co.at (Christian Tanzer) wrote: > Greg Ewing wrote: > > > What's the feeling about this? If, e.g. int() > > were changed in Py3k to round instead of truncate, > > would it cause anyone substantial pain? > > Gratuitous breakage! > > I shudder at the thought of checking hundreds of int-calls to see if > they'd still be correct under such a change. My experience of doing that when compilers sometimes did one and sometimes the other is that such breakages are rarer than the conversions to integer that are broken with both rules! And both are rarer than the code that works with either rule. However, a 5% breakage rate is still enough to be of concern. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1 at cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679 From tanzer at swing.co.at Thu Aug 3 10:44:51 2006 From: tanzer at swing.co.at (Christian Tanzer) Date: Thu, 03 Aug 2006 10:44:51 +0200 Subject: [Python-Dev] Rounding float to int directly ... In-Reply-To: Your message of "Thu, 03 Aug 2006 09:35:23 BST." Message-ID: Nick Maclaren wrote: > tanzer at swing.co.at (Christian Tanzer) wrote: > > Greg Ewing wrote: > > > > > What's the feeling about this? If, e.g. int() > > > were changed in Py3k to round instead of truncate, > > > would it cause anyone substantial pain? > > > > Gratuitous breakage! > > > > I shudder at the thought of checking hundreds of int-calls to see if > > they'd still be correct under such a change. > > My experience of doing that when compilers sometimes did one and > sometimes the other is that such breakages are rarer than the > conversions to integer that are broken with both rules! And both are > rarer than the code that works with either rule. > > However, a 5% breakage rate is still enough to be of concern. I couldn't care less about how many calls would break -- I'd still need to look at each and every one. And I know that quite a number of calls need the truncation semantics (I just don't know which without looking). -- Christian Tanzer http://www.c-tanzer.at/ From ncoghlan at gmail.com Thu Aug 3 13:43:27 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 03 Aug 2006 21:43:27 +1000 Subject: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers) In-Reply-To: <44D1B47E.5020506@ronadam.com> References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com> <44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com> <44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com> <44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com> <44CFF9B9.3010402@canterbury.ac.nz> <44D09A5C.4030905@egenix.com> <44D15FB4.5020605@canterbury.ac.nz> <44D16CFF.7090001@ewtllc.com> <44D198F2.9020702@canterbury.ac.nz> <44D1B47E.5020506@ronadam.com> Message-ID: <44D1E15F.9060908@gmail.com> Ron Adam wrote: > Consider an example where you are combining data that had different > number of significant digits. Keeping all the digits of your answer > gives a false since of accuracy. The extra digits are meaningless > because the margin of error is greater than any of the digits that > exceed the minimum number of significant digits in your data. So you > need to remove the extraneous digits by rounding. Then this answer can > be further used as data, and sense the number of significant digits is > maintained, the margin of error can be calculated accurately. This is a fallacy though - add two numbers together each with an error of +/- 0.05, and the error in the total is +/- 0.1. The approach you propose gives the misleading impression that the error in the total is still only +/- 0.05 (as the answer will contain 1 decimal place). If you really care about error, you need to do it properly (e.g. stddev or adding the error margins). Anyway, it's not proposed to remove the "round to x decimal places" capability entirely - merely remove it from the builtin round() so that the return type can be changed. > It makes since for round have an argument to specify the number of > digits of precision to round to and also for it to return floating point > numbers because rounding results to a float of n digits of precision is > a necessary operation. > > If you have the default round() case of 0 (and negative) digits of > precision return integers, you then have a function that may sometimes > returns integers, and sometimes returns floats. That's not the proposal, as Greg has already said (the fround() mentioned would probably be 'math.fround' rather than a builtin): [Greg Ewing] > No, round() wouldn't have that option at all. If > you wanted it, you would use fround() instead, > which would have the option and return a float > always. > > This would be a Py3k thing, obviously. If done > before then, the new function would have to be > given a different name. [Ron Adam] > This can be problematic > if the values are further used in division operations. Which will > result in many cases of.. float(round(x)) in order to convert integer > results back to floats. Not in Py3k where int/int will return a float as needed. > And in regard to using round() combined with division operators in > floating point math, it is an issue of least surprises. And in Py3k, with a round that always returned an integer there wouldn't be any surprises at all: - division would do the right thing, because true division becomes the only behaviour for '/' - the result of the builtin rounding operation would be usable directly as an index without requiring subsequent coercion to an integer Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ralf at brainbot.com Thu Aug 3 15:02:11 2006 From: ralf at brainbot.com (Ralf Schmitt) Date: Thu, 03 Aug 2006 15:02:11 +0200 Subject: [Python-Dev] segmentation fault in Python 2.5b3 (trunk:51066) Message-ID: Hi all, I've been trying to port our software to python 2.5. unfortunately I'm getting constantly hit by segfaults. I've boiled it down to the following code: rs at schrotti:~/bug$ cat t.py import array class Indexer(object): maximumForwardSize = property(array.array.fromstring) rs at schrotti:~/bug$ python t.py Segmentation fault rs at schrotti:~/bug$ python Python 2.5b3 (trunk:51066, Aug 3 2006, 13:18:16) [GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> [18733 refs] [7182 refs] rs at schrotti:~/bug$ gdb python GNU gdb 6.4-debian Copyright 2005 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i486-linux-gnu"...Using host libthread_db library "/lib/tls/i686/cmov/libthread_db.so.1". (gdb) r t.py Starting program: /home/rs/exp/bin/python t.py [Thread debugging using libthread_db enabled] [New Thread -1210304832 (LWP 21660)] Program received signal SIGSEGV, Segmentation fault. [Switching to Thread -1210304832 (LWP 21660)] 0x0811b68a in visit_decref (op=0xb7db21d8, data=0x0) at Modules/gcmodule.c:270 270 if (PyObject_IS_GC(op)) { (gdb) bt #0 0x0811b68a in visit_decref (op=0xb7db21d8, data=0x0) at Modules/gcmodule.c:270 #1 0x0814043d in property_traverse (self=0xb7d7b5b4, visit=0x811b654 , arg=0x0) at Objects/descrobject.c:1235 #2 0x0811b753 in subtract_refs (containers=0x8197d88) at Modules/gcmodule.c:295 #3 0x0811c453 in collect (generation=2) at Modules/gcmodule.c:790 #4 0x0811cf76 in PyGC_Collect () at Modules/gcmodule.c:1264 #5 0x0810d5da in Py_Finalize () at Python/pythonrun.c:377 #6 0x08057556 in Py_Main (argc=2, argv=0xbfb5a6f4) at Modules/main.c:516 #7 0x080565ba in main (argc=2, argv=0xbfb5a6f4) at ./Modules/python.c:23 (gdb) From theller at python.net Thu Aug 3 15:46:28 2006 From: theller at python.net (Thomas Heller) Date: Thu, 03 Aug 2006 15:46:28 +0200 Subject: [Python-Dev] segmentation fault in Python 2.5b3 (trunk:51066) In-Reply-To: References: Message-ID: Ralf Schmitt schrieb: > Hi all, > > I've been trying to port our software to python 2.5. > unfortunately I'm getting constantly hit by segfaults. > > I've boiled it down to the following code: > > rs at schrotti:~/bug$ cat t.py > import array > > class Indexer(object): > maximumForwardSize = property(array.array.fromstring) > rs at schrotti:~/bug$ python t.py > Segmentation fault Confirmed under Windows. Here's the problem (Objects/descrobject.c, near line 1200, in property_init(...): /* if no docstring given and the getter has one, use that one */ if ((doc == NULL || doc == Py_None) && get != NULL && PyObject_HasAttrString(get, "__doc__")) { if (!(get_doc = PyObject_GetAttrString(get, "__doc__"))) return -1; Py_DECREF(get_doc); /* it is INCREF'd again below */ ^^^^^^^^^^^^^^^^^^ doc = get_doc; } Py_XINCREF(get); Py_XINCREF(set); Py_XINCREF(del); Py_XINCREF(doc); If the refcount of get_doc drops to zero in the Py_DECREF(), the Py_XINCREF() shortly after doesn't help ;-). Thomas From theller at python.net Thu Aug 3 15:56:48 2006 From: theller at python.net (Thomas Heller) Date: Thu, 03 Aug 2006 15:56:48 +0200 Subject: [Python-Dev] segmentation fault in Python 2.5b3 (trunk:51066) In-Reply-To: References: Message-ID: > /* if no docstring given and the getter has one, use that one */ > if ((doc == NULL || doc == Py_None) && get != NULL && > PyObject_HasAttrString(get, "__doc__")) { > if (!(get_doc = PyObject_GetAttrString(get, "__doc__"))) > return -1; > Py_DECREF(get_doc); /* it is INCREF'd again below */ > ^^^^^^^^^^^^^^^^^^ > doc = get_doc; > } > > Py_XINCREF(get); > Py_XINCREF(set); > Py_XINCREF(del); > Py_XINCREF(doc); > A strange programming style, if you ask me, and I wonder why Coverity doesn't complain about it. Thomas From duncan.booth at suttoncourtenay.org.uk Thu Aug 3 16:41:45 2006 From: duncan.booth at suttoncourtenay.org.uk (Duncan Booth) Date: Thu, 3 Aug 2006 15:41:45 +0100 Subject: [Python-Dev] segmentation fault in Python 2.5b3 (trunk:51066) References: Message-ID: Thomas Heller wrote in news:easvak$89h$1 at sea.gmane.org: >> /* if no docstring given and the getter has one, use that one */ >> if ((doc == NULL || doc == Py_None) && get != NULL && >> PyObject_HasAttrString(get, "__doc__")) { >> if (!(get_doc = PyObject_GetAttrString(get, "__doc__"))) >> return -1; >> Py_DECREF(get_doc); /* it is INCREF'd again below */ >> ^^^^^^^^^^^^^^^^^^ >> doc = get_doc; >> } >> >> Py_XINCREF(get); >> Py_XINCREF(set); >> Py_XINCREF(del); >> Py_XINCREF(doc); >> > > A strange programming style, if you ask me, and I wonder why Coverity > doesn't complain about it. > Does Coverity recognise objects on Python's internal pools as deallocated? If not it wouldn't complain because all that the Py_DECREF does is link the block into a pool's freeblock list so any subsequent reference from the Py_XINCREF is still a reference to allocated memory. [Off topic: Microsoft have (or had?) a similarly screwy bit in their ActiveX atl libraries: a newly created ActiveX object has its reference count incremented before calling FinalConstruct and then decremented to 0 (using a method which decrements the reference count but doesn't free it) before being incremented again. If in the meanwhile you increment and decrement the reference count in another thread then it goes bang.] The moral is to regard the reference counting rules as law: no matter how sure you are that you can cheat, don't or you'll regret it. From ralf at brainbot.com Thu Aug 3 16:59:08 2006 From: ralf at brainbot.com (Ralf Schmitt) Date: Thu, 03 Aug 2006 16:59:08 +0200 Subject: [Python-Dev] segmentation fault in Python 2.5b3 (trunk:51066) In-Reply-To: References: Message-ID: <44D20F3C.9040602@brainbot.com> Duncan Booth wrote: > > > The moral is to regard the reference counting rules as law: no matter how > sure you are that you can cheat, don't or you'll regret it. > Or someone else will regret it, just like in this case. :) - Ralf From ralf at brainbot.com Thu Aug 3 17:40:57 2006 From: ralf at brainbot.com (Ralf Schmitt) Date: Thu, 03 Aug 2006 17:40:57 +0200 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys Message-ID: Still trying to port our software. here's another thing I noticed: d = {} d[u'm\xe1s'] = 1 d['m\xe1s'] = 1 print d With python 2.4 I can add those two keys to the dictionary and get: $ python2.4 t2.py {u'm\xe1s': 1, 'm\xe1s': 1} With python 2.5 I get: $ python2.5 t2.py Traceback (most recent call last): File "t2.py", line 3, in d['m\xe1s'] = 1 UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 1: ordinal not in range(128) Is this intended behaviour? I guess this might break lots of programs and the way python 2.4 works looks right to me. I think it should be possible to mix str/unicode keys in dicts and let non-ascii strings compare not-equal to any unicode string. At least it should be documented prominently in the "what's new" document. - Ralf From ralf at brainbot.com Thu Aug 3 17:47:37 2006 From: ralf at brainbot.com (Ralf Schmitt) Date: Thu, 03 Aug 2006 17:47:37 +0200 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: References: Message-ID: <44D21A99.2080801@brainbot.com> Ralf Schmitt wrote: > Still trying to port our software. here's another thing I noticed: > > d = {} > d[u'm\xe1s'] = 1 > d['m\xe1s'] = 1 > print d > > With python 2.4 I can add those two keys to the dictionary and get: > $ python2.4 t2.py > {u'm\xe1s': 1, 'm\xe1s': 1} > > With python 2.5 I get: > > $ python2.5 t2.py > Traceback (most recent call last): > File "t2.py", line 3, in > d['m\xe1s'] = 1 > UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 1: > ordinal not in range(128) > > Is this intended behaviour? I guess this might break lots of programs > and the way python 2.4 works looks right to me. > I think it should be possible to mix str/unicode keys in dicts and let > non-ascii strings compare not-equal to any unicode string. Also this behaviour makes your programs break randomly, that is, it will break when the string you add hashes to the same value that the unicode string has (at least that's what I guess..) - Ralf From mwh at python.net Thu Aug 3 17:51:29 2006 From: mwh at python.net (Michael Hudson) Date: Thu, 03 Aug 2006 16:51:29 +0100 Subject: [Python-Dev] segmentation fault in Python 2.5b3 (trunk:51066) In-Reply-To: (Duncan Booth's message of "Thu, 3 Aug 2006 15:41:45 +0100") References: Message-ID: <2mmzalx1fi.fsf@starship.python.net> Duncan Booth writes: > Does Coverity recognise objects on Python's internal pools as deallocated? Coverity doesn't work on that level; it analyzes source code, and knows about Python's INCREFs and DECREFs. > The moral is to regard the reference counting rules as law: no matter how > sure you are that you can cheat, don't or you'll regret it. This is the truth. Cheers, mwh -- As it seems to me, in Perl you have to be an expert to correctly make a nested data structure like, say, a list of hashes of instances. In Python, you have to be an idiot not to be able to do it, because you just write it down. -- Peter Norvig, comp.lang.functional From mal at egenix.com Thu Aug 3 18:51:21 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 03 Aug 2006 18:51:21 +0200 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <44D21A99.2080801@brainbot.com> References: <44D21A99.2080801@brainbot.com> Message-ID: <44D22989.7000202@egenix.com> Ralf Schmitt wrote: > Ralf Schmitt wrote: >> Still trying to port our software. here's another thing I noticed: >> >> d = {} >> d[u'm\xe1s'] = 1 >> d['m\xe1s'] = 1 >> print d >> >> With python 2.4 I can add those two keys to the dictionary and get: >> $ python2.4 t2.py >> {u'm\xe1s': 1, 'm\xe1s': 1} >> >> With python 2.5 I get: >> >> $ python2.5 t2.py >> Traceback (most recent call last): >> File "t2.py", line 3, in >> d['m\xe1s'] = 1 >> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 1: >> ordinal not in range(128) >> >> Is this intended behaviour? I guess this might break lots of programs >> and the way python 2.4 works looks right to me. >> I think it should be possible to mix str/unicode keys in dicts and let >> non-ascii strings compare not-equal to any unicode string. > > Also this behaviour makes your programs break randomly, that is, it will > break when the string you add hashes to the same value that the unicode > string has (at least that's what I guess..) This is because Unicode and 8-bit string keys only work in the same way if and only if they are plain ASCII. The reason lies in the hash function used by Unicode: it is crafted to make hash(u) == hash(s) for all ASCII s, such that s == u. For non-ASCII strings, there are no guarantees as to the hash value of the strings or whether they match or not. This has been like that since Unicode was introduced, so it's not new in Python 2.5. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 03 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From bob at redivi.com Thu Aug 3 19:03:08 2006 From: bob at redivi.com (Bob Ippolito) Date: Thu, 3 Aug 2006 10:03:08 -0700 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <44D22989.7000202@egenix.com> References: <44D21A99.2080801@brainbot.com> <44D22989.7000202@egenix.com> Message-ID: <1009DE87-F7CD-41F0-8409-BA3599ACB65E@redivi.com> On Aug 3, 2006, at 9:51 AM, M.-A. Lemburg wrote: > Ralf Schmitt wrote: >> Ralf Schmitt wrote: >>> Still trying to port our software. here's another thing I noticed: >>> >>> d = {} >>> d[u'm\xe1s'] = 1 >>> d['m\xe1s'] = 1 >>> print d >>> >>> With python 2.4 I can add those two keys to the dictionary and get: >>> $ python2.4 t2.py >>> {u'm\xe1s': 1, 'm\xe1s': 1} >>> >>> With python 2.5 I get: >>> >>> $ python2.5 t2.py >>> Traceback (most recent call last): >>> File "t2.py", line 3, in >>> d['m\xe1s'] = 1 >>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in >>> position 1: >>> ordinal not in range(128) >>> >>> Is this intended behaviour? I guess this might break lots of >>> programs >>> and the way python 2.4 works looks right to me. >>> I think it should be possible to mix str/unicode keys in dicts >>> and let >>> non-ascii strings compare not-equal to any unicode string. >> >> Also this behaviour makes your programs break randomly, that is, >> it will >> break when the string you add hashes to the same value that the >> unicode >> string has (at least that's what I guess..) > > This is because Unicode and 8-bit string keys only work > in the same way if and only if they are plain ASCII. > > The reason lies in the hash function used by Unicode: it is > crafted to make hash(u) == hash(s) for all ASCII s, such > that s == u. > > For non-ASCII strings, there are no guarantees as to the > hash value of the strings or whether they match or not. > > This has been like that since Unicode was introduced, so it's > not new in Python 2.5. What is new is that the exception raised on "u == s" after hash collision is no longer silently swallowed. -bob From martin at v.loewis.de Thu Aug 3 19:15:59 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 03 Aug 2006 19:15:59 +0200 Subject: [Python-Dev] segmentation fault in Python 2.5b3 (trunk:51066) In-Reply-To: References: Message-ID: <44D22F4F.4060707@v.loewis.de> Ralf Schmitt schrieb: > I've been trying to port our software to python 2.5. > unfortunately I'm getting constantly hit by segfaults. I understand it's unfortunate for you, but it is fortunate for us that you have been trying to port your application before 2.5 was released, and reported the bug when you discovered it. So thanks for reporting it. FWIW, the bug that Thomas found was introduced by patch #1434038, and committed as r42916. Regards, Martin From ralf at brainbot.com Thu Aug 3 19:17:27 2006 From: ralf at brainbot.com (Ralf Schmitt) Date: Thu, 03 Aug 2006 19:17:27 +0200 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <44D22989.7000202@egenix.com> References: <44D21A99.2080801@brainbot.com> <44D22989.7000202@egenix.com> Message-ID: <44D22FA7.7030905@brainbot.com> M.-A. Lemburg wrote: > Ralf Schmitt wrote: >> Ralf Schmitt wrote: >>> Still trying to port our software. here's another thing I noticed: >>> >>> d = {} >>> d[u'm\xe1s'] = 1 >>> d['m\xe1s'] = 1 >>> print d >>> >>> With python 2.4 I can add those two keys to the dictionary and get: >>> $ python2.4 t2.py >>> {u'm\xe1s': 1, 'm\xe1s': 1} >>> >>> With python 2.5 I get: >>> >>> $ python2.5 t2.py >>> Traceback (most recent call last): >>> File "t2.py", line 3, in >>> d['m\xe1s'] = 1 >>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 1: >>> ordinal not in range(128) >>> >>> Is this intended behaviour? I guess this might break lots of programs >>> and the way python 2.4 works looks right to me. >>> I think it should be possible to mix str/unicode keys in dicts and let >>> non-ascii strings compare not-equal to any unicode string. >> Also this behaviour makes your programs break randomly, that is, it will >> break when the string you add hashes to the same value that the unicode >> string has (at least that's what I guess..) > > This is because Unicode and 8-bit string keys only work > in the same way if and only if they are plain ASCII. This is okay. But in the case where one is not ASCII I would prefer to be able to compare them (not equal) instead of getting a UnicodeError. I know it's too late to change this, ... > > The reason lies in the hash function used by Unicode: it is > crafted to make hash(u) == hash(s) for all ASCII s, such > that s == u. > > For non-ASCII strings, there are no guarantees as to the > hash value of the strings or whether they match or not. > > This has been like that since Unicode was introduced, so it's > not new in Python 2.5. > ...but in the case of dictionaries this behaviour has changed and in prior versions of python dictionaries did work as I expected them to. Now they don't. When working with unicode strings and (accidently) mixing with str strings, things might seem to work until the first non-ascii string is given to some code and one gets that UnicodeDecodeError (e.g. when comparing them). If one mixes unicode strings and str strings as keys in a dictionary things might seem to work far longer until he tries to put in some non ASCII string with the "wrong" hash value and suddenly things go boom. I'd rather keep the pre 2.5 behaviour. - Ralf From jimjjewett at gmail.com Thu Aug 3 19:36:30 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu, 3 Aug 2006 13:36:30 -0400 Subject: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers) Message-ID: Greg Ewing wrote: > The time isn't the main issue. The main issue > is that almost all the use cases for round() > involve doing an int() on it afterwards. At > least nobody has put forward an argument to > the contrary yet. Or, more accurately, they *should* involve an int afterwards, but often don't, because people expect round(something) to produce an integer. Looking only at the one-argument round() calls not surrounded by int(), I would bet that most are bugs rather than intentional. The proportion is certainly high enough that I can't just trust it. Ron Adam wrote: > I think you are only considering part of the usefulness of round(). In > statistics and science math equations, rounding isn't used to get an > integer, but instead used to give an answer that is meaningful within > the margin of error of the data being used. Yes, but (1) Only for display (or storage). This rounding should explicitly not be done on intermediate values, as it introduces another source of error. (2) You need to pass the precision, so this isn't the one-argument round(). (3) As a side note, even the two-argument round may not be what you want; it isn't uncommon to have accuracy to a number of significant figures, rather than to an absolute "three places past the decimal point". This gets complicated enough that it is reasonable to use math.fround() or some such instead of a builtin. -jJ From mal at egenix.com Thu Aug 3 19:39:15 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 03 Aug 2006 19:39:15 +0200 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <44D22FA7.7030905@brainbot.com> References: <44D21A99.2080801@brainbot.com> <44D22989.7000202@egenix.com> <44D22FA7.7030905@brainbot.com> Message-ID: <44D234C3.4010404@egenix.com> Ralf Schmitt wrote: >>>> Still trying to port our software. here's another thing I noticed: >>>> >>>> d = {} >>>> d[u'm\xe1s'] = 1 >>>> d['m\xe1s'] = 1 >>>> print d >>>> >>>> With python 2.5 I get: >>>> >>>> $ python2.5 t2.py >>>> Traceback (most recent call last): >>>> File "t2.py", line 3, in >>>> d['m\xe1s'] = 1 >>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 1: >>>> ordinal not in range(128) >>>> >> This is because Unicode and 8-bit string keys only work >> in the same way if and only if they are plain ASCII. > > This is okay. But in the case where one is not ASCII I would prefer to > be able to compare them (not equal) instead of getting a UnicodeError. > I know it's too late to change this, ... It is too late to change this, since it was always like this ;-) Seriously, Unicode is doing the right thing here: you should really always get an exception if you compare apples and oranges, rather than reverting to comparing the ids of apples and oranges as fall-back solution. I believe that Py3k will implement this. >> The reason lies in the hash function used by Unicode: it is >> crafted to make hash(u) == hash(s) for all ASCII s, such >> that s == u. >> >> For non-ASCII strings, there are no guarantees as to the >> hash value of the strings or whether they match or not. >> >> This has been like that since Unicode was introduced, so it's >> not new in Python 2.5. >> > > ...but in the case of dictionaries this behaviour has changed and in > prior versions of python dictionaries did work as I expected them to. > Now they don't. Let's put it this way: Python 2.5 uncovered a bug in your application that has always been there. It's better to fix your application than arguing to cover up the bug again. > When working with unicode strings and (accidently) mixing with str > strings, things might seem to work until the first non-ascii string > is given to some code and one gets that UnicodeDecodeError (e.g. when > comparing them). > > If one mixes unicode strings and str strings as keys in a dictionary > things might seem to work far longer until he tries to put in some non > ASCII string with the "wrong" hash value and suddenly things go boom. > I'd rather keep the pre 2.5 behaviour. It's actually a good preparation for Py3k where 1 == u'abc' will (likely) also raise an exception. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 03 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From jjl at pobox.com Thu Aug 3 19:45:23 2006 From: jjl at pobox.com (John J Lee) Date: Thu, 3 Aug 2006 18:45:23 +0100 (GMT Standard Time) Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <44D234C3.4010404@egenix.com> References: <44D21A99.2080801@brainbot.com> <44D22989.7000202@egenix.com> <44D22FA7.7030905@brainbot.com> <44D234C3.4010404@egenix.com> Message-ID: On Thu, 3 Aug 2006, M.-A. Lemburg wrote: [...] > It's actually a good preparation for Py3k where 1 == u'abc' will > (likely) also raise an exception. I though I'd heard (from Guido here or on the py3k list) that it was only 1 < u'abc' that would raise an exception, and that 1 == u'abc' would still evaluate to False. Did I misunderstand? John From g.brandl at gmx.net Thu Aug 3 19:50:58 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 03 Aug 2006 19:50:58 +0200 Subject: [Python-Dev] segmentation fault in Python 2.5b3 (trunk:51066) In-Reply-To: <44D22F4F.4060707@v.loewis.de> References: <44D22F4F.4060707@v.loewis.de> Message-ID: Martin v. L?wis wrote: > Ralf Schmitt schrieb: >> I've been trying to port our software to python 2.5. >> unfortunately I'm getting constantly hit by segfaults. > > I understand it's unfortunate for you, but it is fortunate > for us that you have been trying to port your application > before 2.5 was released, and reported the bug when you > discovered it. So thanks for reporting it. > > FWIW, the bug that Thomas found was introduced by > patch #1434038, and committed as r42916. I already fixed it locally and will commit as soon as trunk is unfrozen. Georg From mal at egenix.com Thu Aug 3 19:53:11 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 03 Aug 2006 19:53:11 +0200 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: References: <44D21A99.2080801@brainbot.com> <44D22989.7000202@egenix.com> <44D22FA7.7030905@brainbot.com> <44D234C3.4010404@egenix.com> Message-ID: <44D23807.4050604@egenix.com> John J Lee wrote: > On Thu, 3 Aug 2006, M.-A. Lemburg wrote: > [...] >> It's actually a good preparation for Py3k where 1 == u'abc' will >> (likely) also raise an exception. > > I though I'd heard (from Guido here or on the py3k list) that it was only > 1 < u'abc' that would raise an exception, and that 1 == u'abc' would still > evaluate to False. Did I misunderstand? Could be that I'm wrong. All such mixed-type compares are wrong depending on the scale of brokenness you apply ;-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 03 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From python-dev at zesty.ca Thu Aug 3 21:02:45 2006 From: python-dev at zesty.ca (Ka-Ping Yee) Date: Thu, 3 Aug 2006 14:02:45 -0500 (CDT) Subject: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers) In-Reply-To: <44D198F2.9020702@canterbury.ac.nz> References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com> <44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com> <44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com> <44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com> <44CFF9B9.3010402@canterbury.ac.nz> <44D09A5C.4030905@egenix.com> <44D15FB4.5020605@canterbury.ac.nz> <44D16CFF.7090001@ewtllc.com> <44D198F2.9020702@canterbury.ac.nz> Message-ID: On Thu, 3 Aug 2006, Greg Ewing wrote: > The time isn't the main issue. The main issue > is that almost all the use cases for round() > involve doing an int() on it afterwards. That's my experience as well. In my opinion, the purpose of round() is most commonly described as "to make an integer". So it should yield an integer. As other comments in this discussion imply, whether you want round() to return an integer really depends on your intended audience and their pattern of usage. It seems to me that for round() to produce an integer is about as intuitive as for 1/2 to produce 0.5. That is, if you bring basic mathematical intuition to the table instead of a background in how C behaves, then "1/2" suggests one-half and "rounding" suggests "convert to an integer". If you have a background in programming languages and you understand the value of type consistency, then you would see why dividing integers should yield integers and rounding floats should yield floats. Since Python was indeed changed to make 1/2 evaluate to 0.5, that suggests that it is Pythonic to target the first audience rather than the second. And that implies that it would also be Pythonic to make round() return an int. -- ?!ng From nmm1 at cus.cam.ac.uk Thu Aug 3 21:43:07 2006 From: nmm1 at cus.cam.ac.uk (Nick Maclaren) Date: Thu, 03 Aug 2006 20:43:07 +0100 Subject: [Python-Dev] Rounding float to int directly ... Message-ID: Ka-Ping Yee wrote: > > That's my experience as well. In my opinion, the purpose of round() > is most commonly described as "to make an integer". So it should > yield an integer. Grrk. No, that logic is flawed. There are algorithms where the operation of rounding (or truncation) is needed, but where the value may be larger than can be held in an integer, and that is not an error. If the only rounding or truncation primitive converts to an integer, those algorithms are unimplementable. You need at least one primitive that converts a float to an integer, held as a float. Which is independent of whether THIS particular facility should yield an integer or float! Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1 at cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679 From ronaldoussoren at mac.com Thu Aug 3 22:12:11 2006 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Thu, 3 Aug 2006 22:12:11 +0200 Subject: [Python-Dev] Rounding float to int directly ... In-Reply-To: References: Message-ID: <9AD92B78-1723-43D4-B1F5-0D6147E59CEE@mac.com> On Aug 3, 2006, at 9:43 PM, Nick Maclaren wrote: > Ka-Ping Yee wrote: >> >> That's my experience as well. In my opinion, the purpose of round() >> is most commonly described as "to make an integer". So it should >> yield an integer. > > Grrk. No, that logic is flawed. > > There are algorithms where the operation of rounding (or truncation) > is needed, but where the value may be larger than can be held in an > integer, and that is not an error. Is that really true for python? Python integers are unbounded in magnitute, they are not the same as int or long in C, therefore any float except exceptional values like NaN can be converted to an integer value. The converse is not true, python integers can contain values that are larger than any float (aka C's double). >>> v = 1e200 >>> int(v) 999999999999999969733122212510361659474503275455023626482417509503468484 355540755341963384047062518680275124159738824081821357343682784846393850 41047239877871023591066789981811181813306167128854888448L >>> type(v) >>> t = int(v) >>> t ** 2 999999999999999939466244425020724235032897553743712247233978162062705420 868772363027380308001932133054230558394675289323324880702327952854432161 552216024892912466614409626956153314556116473848998339762109232220813863 099472521374735119038509661875525607726747258646821773646868361139842288 412173261267039669530389442594522433115448347796339690544576171593343952 0020822843337114038314499908946523848704L >>> float(t**2) Traceback (most recent call last): File "", line 1, in OverflowError: long int too large to convert to float Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2157 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20060803/5e233fa2/attachment.bin From nmm1 at cus.cam.ac.uk Thu Aug 3 22:49:30 2006 From: nmm1 at cus.cam.ac.uk (Nick Maclaren) Date: Thu, 03 Aug 2006 21:49:30 +0100 Subject: [Python-Dev] Rounding float to int directly ... Message-ID: Ronald Oussoren wrote: > > > There are algorithms where the operation of rounding (or truncation) > > is needed, but where the value may be larger than can be held in an > > integer, and that is not an error. > > Is that really true for python? Python integers are unbounded in > magnitute, they are not the same as int or long in C, therefore any > float except exceptional values like NaN can be converted to an > integer value. The converse is not true, python integers can contain > values that are larger than any float (aka C's double). It depends a great deal on what you mean by a Python integer! Yes, I was assuming the (old) Python model, where it is a C long, but so were many (most?) of the other postings. If you are assuming the (future?) model, where there is a single integer type of unlimited size, then that is true. There is still an efficiency point, in that such algorithms positively don't want a float value like 1.0e300 (or, worse, 1.0e4000) expanded to its full decimal representation as an intermediate step. Whatever. There should still be at least one operation that rounds or truncates a float value, returning a float of the same type, on either functionality or efficiency grounds. I and most programmers of such algorithms don't give a damn which it does, provided that it is clearly documented, at least half-sane and doesn't change with versions of Python. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1 at cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679 From jimjjewett at gmail.com Thu Aug 3 23:10:43 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu, 3 Aug 2006 17:10:43 -0400 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys Message-ID: http://mail.python.org/pipermail/python-dev/2006-August/067934.html M.-A. Lemburg mal at egenix.com > Ralf Schmitt wrote: >> Still trying to port our software. here's another thing I noticed: >> d = {} >> d[u'm\xe1s'] = 1 >> d['m\xe1s'] = 1 >> print d (a 2-element dictionary, because they are not equal) >> With python 2.5 I get: [ a traceback ending in ] >> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 1: >> ordinal not in range(128) > Let's put it this way: Python 2.5 uncovered a bug in your > application that has always been there. No; he application would only have a bug if he expected those two objects to compare equal. Trying to stick something hashable into a dictionary should not raise an Exception just because there is already a similar key, (regardless of whether or not the other key is equal or identical). The only way this error could be the right thing is if you were trying to suggest that he shouldn't mix unicode and bytestrings at all. There is a command line switch for that, but it doesn't get much use. -jJ From mal at egenix.com Thu Aug 3 23:47:10 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 03 Aug 2006 23:47:10 +0200 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: References: Message-ID: <44D26EDE.6030907@egenix.com> Jim Jewett wrote: > http://mail.python.org/pipermail/python-dev/2006-August/067934.html > M.-A. Lemburg mal at egenix.com > >> Ralf Schmitt wrote: >>> Still trying to port our software. here's another thing I noticed: > >>> d = {} >>> d[u'm\xe1s'] = 1 >>> d['m\xe1s'] = 1 >>> print d > > (a 2-element dictionary, because they are not equal) > >>> With python 2.5 I get: [ a traceback ending in ] > >>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 1: >>> ordinal not in range(128) > >> Let's put it this way: Python 2.5 uncovered a bug in your >> application that has always been there. > > No; he application would only have a bug if he expected those two > objects to compare equal. Trying to stick something hashable into a > dictionary should not raise an Exception just because there is already > a similar key, (regardless of whether or not the other key is equal or > identical). Hmm, you have a point there... >>> d = {} # Two different objects >>> x = 'a' >>> y = hash(x) >>> x 'a' >>> y 12416037344 # ... with the same hash value >>> hash(x) 12416037344 >>> hash(y) 12416037344 # Put them in the dictionary, causing a hash collision ... >>> d[x] = 1 >>> d[y] = 2 # ... which is resolved by comparing the two for equality # and assigning them to two different slots: >>> d {'a': 1, 12416037344: 2} Since Python 2.5 propagates the compare exception, you get the exception. Python 2.4 silenced the exception. > The only way this error could be the right thing is if you were trying > to suggest that he shouldn't mix unicode and bytestrings at all. Good question. I wonder whether that's a reasonable approach for Python 2.x (I'd say it is for Py3k). Currently you can't safely mix non-ASCII string with Unicode keys in the same dictionary. Perhaps we ought to add an exception to the dict lookup mechanism and continue to silence UnicodeErrors ?! -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 03 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From rrr at ronadam.com Fri Aug 4 00:00:04 2006 From: rrr at ronadam.com (Ron Adam) Date: Thu, 03 Aug 2006 17:00:04 -0500 Subject: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers) In-Reply-To: <44D1E15F.9060908@gmail.com> References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com> <44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com> <44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com> <44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com> <44CFF9B9.3010402@canterbury.ac.nz> <44D09A5C.4030905@egenix.com> <44D15FB4.5020605@canterbury.ac.nz> <44D16CFF.7090001@ewtllc.com> <44D198F2.9020702@canterbury.ac.nz> <44D1B47E.5020506@ronadam.com> <44D1E15F.9060908@gmail.com> Message-ID: <44D271E4.9000009@ronadam.com> Nick Coghlan wrote: > Ron Adam wrote: >> Consider an example where you are combining data that had different >> number of significant digits. Keeping all the digits of your answer >> gives a false since of accuracy. The extra digits are meaningless >> because the margin of error is greater than any of the digits that >> exceed the minimum number of significant digits in your data. So you >> need to remove the extraneous digits by rounding. Then this answer >> can be further used as data, and sense the number of significant >> digits is maintained, the margin of error can be calculated accurately. > > This is a fallacy though - add two numbers together each with an error > of +/- 0.05, and the error in the total is +/- 0.1. The approach you > propose gives the misleading impression that the error in the total is > still only +/- 0.05 (as the answer will contain 1 decimal place). I did say "the margin of error can be calculated accurately". I did not go into detail on the correct method to do that. You do need to round to the correct number of significant digits AND yes, you do need to also take into account the number of items in your data set. If you don't round, the value of your answer would be shifted up or down by the amount of the non significant digits. And the error range would also be shifted up or down by that amount. In any case, my point is (and my opinion) that rounding to floating point types is very common and not something that is needed rarely as it has been expressed in several messages. I don't want to debate the correct method of calculating errors and maintaining accuracy in statistical equations. I'll let the mathematicians do that. > If you really care about error, you need to do it properly (e.g. stddev > or adding the error margins). Yes > Anyway, it's not proposed to remove the "round to x decimal places" > capability entirely - merely remove it from the builtin round() so that > the return type can be changed. That would work. This is one of those places where I wonder when should a function be included in a types methods and called by either using arguments with __init__, an external function to call a private method, or by having it be a public method to be called with the dot syntax like is common for strings? If you look at the Decimal type, it has quite of few rounding methods, (as well as other non-underscored methods), while int, float, and long, do not have them. Also the Decimal _round() method returns the number unchanged (max precision) if no precision argument is given where round() has a default precision of 0. Will there be more efforts (in Py3k to unifiy Decimal and the other types so they are more alike in both the way they work and in how they are used? Would it make since for the numeric type to have methods (other than the underscored ones) that can be called with the dot syntax like we do in unicode and the Decimal types that do not have the underscores? Or would it be better to remove those in favor of builtin or module functions? The extremes of both might be a foolish consistency, but this may also be a good area to make some basic improvements for python 3k. (I'm sure Guido and others here have though some on this in general.) If these types gain a round method that all work the same way, then round() can then use those and it would return the same type as it is given. So then round(float(x)), round(int(x)), round(long(x)) and round(Decimal(x)) will return their own type respectfully. And to get a specific type, you would call the types method directly. int.round(x) -> integer Which is nicer than int(round(x)). And the round methods could have arguments to designate the type of rounding to use. Where 'f','c', and 'e' are floor, ceiling, and even rounding. int.round(x, 'c') # 'f' is the default rounding mode The functions round(), roundcieling(), and roundeven() could then call the method of the type with the appropriate argument, although I'd be happy without these round functions if there are round methods are consistent across types. Well, this is the direction I would go in. Hopefully by Python 9000 the decimal type or something equivalent will be as fast as all the others and/or computers will be so fast as to make it irrelevant, then we can have just one scaler base type. ;-) Cheers, Ron From tdelaney at avaya.com Fri Aug 4 01:24:12 2006 From: tdelaney at avaya.com (Delaney, Timothy (Tim)) Date: Fri, 4 Aug 2006 09:24:12 +1000 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys Message-ID: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com> M.-A. Lemburg wrote: > Perhaps we ought to add an exception to the dict lookup mechanism > and continue to silence UnicodeErrors ?! I'd definitely consider a UnicodeError to be an indication that two objects are not equal. At the very least, in the context of a dictionary lookup. Tim Delaney From Sjaak.van.Werkhoven at nl.fortis.com Wed Aug 2 10:02:20 2006 From: Sjaak.van.Werkhoven at nl.fortis.com (Werkhoven J.P. van (Sjaak)) Date: Wed, 2 Aug 2006 10:02:20 +0200 Subject: [Python-Dev] using globals Message-ID: Hi, I have got a problem with importing global variables. For instance I have got two files: # t1.py #t2.py counter = 1 def counter_adder(filenr): def show_adder(): global counter import t1 counter+= 1 print("adder is %d" % t1.counter) if filenr == 1: show_adder() else: import t2 t2.show_adder() def show_adder(): print("adder is %d" % counter) >>counter_adder(1) adder is 2 >>counter_adder(2) adder is 1 When I look at the outcome of two function calls I expected no differences but much to my surprise it goes wrong when the global variable is imported. It doesn't give the updated value but the value it get when instantiated. Who come? What should I do to get the updated value greetings, Sjaak van Werkhoven ****************************DISCLAIMER*********************************** Deze e-mail is uitsluitend bestemd voor de geadresseerde(n). Verstrekking aan en gebruik door anderen is niet toegestaan. Fortis sluit iedere aansprakelijkheid uit die voortvloeit uit electronische verzending. This e-mail is intended exclusively for the addressee(s), and may not be passed on to, or made available for use by any person other than the addressee(s). Fortis rules out any and every liability resulting from any electronic transmission. ****************************************************************************** From oren at hishome.net Thu Aug 3 22:11:16 2006 From: oren at hishome.net (Oren Tirosh) Date: Thu, 3 Aug 2006 23:11:16 +0300 Subject: [Python-Dev] uuid module - byte order issue Message-ID: <7168d65a0608031311g3479824ds18a9ede4a33cae7e@mail.gmail.com> The UUID module uses network byte order, regardless of the platform byte order. On little-endian platforms like Windows the ".bytes" property of UUID objects is not compatible with the memory layout of UUIDs: >>> import uuid >>> import pywintypes >>> s = '{00112233-4455-6677-8899-aabbccddeeff}' >>> uuid.UUID(s).bytes.encode('hex') '00112233445566778899aabbccddeeff' >>> str(buffer(pywintypes.IID(s))).encode('hex') '33221100554477668899aabbccddeeff' >>> Ka-Ping Yee writes* that the Windows UUID generation calls are not RFC 4122 compliant and have an illegal version field. If the correct byte order is used the UUIDs generated by Windows XP are valid version 4 UUIDs: >>> parts = struct.unpack('>> parts[2] >> 12 # version 4 >>> ord(parts[3][0]) & 0xC0 # variant 128 The first three fields (32 bit time-low, 16 bit time-mid and time-high-and-version) are stored in the platform byte order while the remainder is stored as a vector of 8 bytes. The bytes property and bytes argument to the constructor should use the platform byte order. It would be nice to have explicit little endian and big endian versions available on platforms of either endianness for compatibility in communication and disk formats. There is another issue with version 1 uuid generation: >>> len(set(uuid.uuid1() for i in range(1000))) 992 The problem is that the random clock_seq field is only 14 bits long. If enough UUIDs are generated within the same system clock tick there will be collisions. Suggested solution: use the high-resolution of the time field (100ns) to generate a monotonically increasing timestamp that advances at least by 1 for each call, when time.time() returns the same value on subsequent calls. Oren [*] http://mail.python.org/pipermail/python-dev/2006-June/065869.html From tjreedy at udel.edu Fri Aug 4 02:50:06 2006 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 3 Aug 2006 20:50:06 -0400 Subject: [Python-Dev] using globals References: Message-ID: "Werkhoven J.P. van (Sjaak)" wrote in news:AB7FC09A43F1EB46B9A11FB45F2B2FD507148EDA at dbacaa45.nl.fortis.bank... > I have got a problem with importing global variables. Questions about using Python belong on comp.lang.python or the general python mailing list. From murman at gmail.com Fri Aug 4 03:37:54 2006 From: murman at gmail.com (Michael Urman) Date: Thu, 3 Aug 2006 20:37:54 -0500 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <44D234C3.4010404@egenix.com> References: <44D21A99.2080801@brainbot.com> <44D22989.7000202@egenix.com> <44D22FA7.7030905@brainbot.com> <44D234C3.4010404@egenix.com> Message-ID: On 8/3/06, M.-A. Lemburg wrote: > > ...but in the case of dictionaries this behaviour has changed and in > > prior versions of python dictionaries did work as I expected them to. > > Now they don't. > > Let's put it this way: Python 2.5 uncovered a bug in your > application that has always been there. It's better to > fix your application than arguing to cover up the bug again. I would understand this assertion if Ralf were expecting dictionaries to consider { u'm\xe1s': 1, 'm\xe1s': 1 } == { u'm\xe1s': 1 } == { 'm\xe1s': 1 } This is clearly a mess waiting to explode. But that's not what he said. He expects, as is the case in python2.4, len({ u'm\xe1s': 1, 'm\xe1s': 1 }) == 2 because u'm\xe1s' clearly does not equal 'm\xe1s'. Because it raises an exception, the dictionary shouldn't consider it equal, so there should be the two keys which happen to be somewhat equivalent. While this is in fact in the NEWS (Patch #1497053 & bug #1275608), I think this should be raised for further discussion. Raising the exception is good for debugging mistakes, but bad for dictionaries holding holding inequal objects that happen to hash to the same value, and correclty raise exceptions on comparison. When we thought it was just a debugging tool, it made sense to put it straight in to 2.5. Since it actually can adversely affect behavior in only slightly edgy cases, perhaps it should go through a warning phase (which ideally could show the exception that was thrown, thus yielding most or all of the intended debugging advantage). Michael -- Michael Urman http://www.tortall.net/mu/blog From greg.ewing at canterbury.ac.nz Fri Aug 4 03:51:14 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 04 Aug 2006 13:51:14 +1200 Subject: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers) In-Reply-To: <44D1B4B6.2080403@egenix.com> References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com> <44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com> <44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com> <44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com> <44CFF9B9.3010402@canterbury.ac.nz> <44D09A5C.4030905@egenix.com> <44D15FB4.5020605@canterbury.ac.nz> <44D1B4B6.2080403@egenix.com> Message-ID: <44D2A812.20906@canterbury.ac.nz> M.-A. Lemburg wrote: > If you are eventually rounding to say 2 decimal > places in the end of the calculation, you won't > want to find yourself presenting the user 1.12 > and 1.13 as equal values :-) Even if, before rounding, they were actually 1.12499999999 and 1.125000000001? And if the difference were only due to the unrepresentability of some decimal fraction exactly in binary? I still maintain that (a) rounding a *binary* float to *decimal* places is wrongheaded, and (b) digit chopping is a bad way to decide whether two inexact numbers should be considered equal. Not just a different way, but a poorer way. > Most > such calculations do work with floats, so having > round() return an int would just add another > coercion to a float for those use-cases. I'm *not* proposing to eliminate round-to-float, despite the fact that I can't see much use for it personally. I'm also *not* advocating changing the existing behaviour of round() or int(). That was just tentative speculation. All I'm asking for is another function that does round-to-int directly. I wouldn't have thought that was such a controversial idea, given the frequency of use for that operation. -- Greg From greg.ewing at canterbury.ac.nz Fri Aug 4 03:51:32 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 04 Aug 2006 13:51:32 +1200 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <44D26EDE.6030907@egenix.com> References: <44D26EDE.6030907@egenix.com> Message-ID: <44D2A824.6060502@canterbury.ac.nz> M.-A. Lemburg wrote: > Perhaps we ought to add an exception to the dict lookup mechanism > and continue to silence UnicodeErrors ?! Seems to be that comparison of unicode and non-unicode strings for equality shouldn't raise exceptions in the first place. -- Greg From foom at fuhm.net Fri Aug 4 04:00:31 2006 From: foom at fuhm.net (James Y Knight) Date: Thu, 3 Aug 2006 22:00:31 -0400 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <44D26EDE.6030907@egenix.com> References: <44D26EDE.6030907@egenix.com> Message-ID: <028CF4BD-EFF8-4D26-9918-C754AB7283D1@fuhm.net> On Aug 3, 2006, at 5:47 PM, M.-A. Lemburg wrote: >> The only way this error could be the right thing is if you were >> trying >> to suggest that he shouldn't mix unicode and bytestrings at all. > > Good question. I wonder whether that's a reasonable approach for > Python 2.x (I'd say it is for Py3k). It's my understanding that in py3k, there will be no implicit conversion, bytestrings and unicodes will never be equal (no matter what the contents), and so this wouldn't be an issue. (as u"1" == "1" would be the same sort of situation as 1 == "1" is now) James From foom at fuhm.net Fri Aug 4 04:02:34 2006 From: foom at fuhm.net (James Y Knight) Date: Thu, 3 Aug 2006 22:02:34 -0400 Subject: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers) In-Reply-To: <44D198F2.9020702@canterbury.ac.nz> References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com> <44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com> <44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com> <44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com> <44CFF9B9.3010402@canterbury.ac.nz> <44D09A5C.4030905@egenix.com> <44D15FB4.5020605@canterbury.ac.nz> <44D16CFF.7090001@ewtllc.com> <44D198F2.9020702@canterbury.ac.nz> Message-ID: <06264EFB-FE9A-4E94-B80D-C621D6BC7AD3@fuhm.net> On Aug 3, 2006, at 2:34 AM, Greg Ewing wrote: > Raymond Hettinger wrote: > >> -1 on an extra built-in just to save the time for function call > > The time isn't the main issue. The main issue > is that almost all the use cases for round() > involve doing an int() on it afterwards. At > least nobody has put forward an argument to > the contrary yet. And I bet the main reason why round() in python returns a float is because it does in C. And it does in C because C doesn't have arbitrary size integers, so if round returned integers, round(1e+308) couldn't work. In python, however, that's no problem, since python does have arbitrarily big integers. There's also round(float("inf")), of course, which wouldn't be defined if the result was an integer, but I don't think rounding infinity is much of a use case. And I do think the extension of round to allow the specification of number of decimal places was a mistake. If you want that, you probably really mean to do something like round(x * 10**y) instead. James From bob at redivi.com Fri Aug 4 04:58:47 2006 From: bob at redivi.com (Bob Ippolito) Date: Thu, 3 Aug 2006 19:58:47 -0700 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <44D2A824.6060502@canterbury.ac.nz> References: <44D26EDE.6030907@egenix.com> <44D2A824.6060502@canterbury.ac.nz> Message-ID: On Aug 3, 2006, at 6:51 PM, Greg Ewing wrote: > M.-A. Lemburg wrote: > >> Perhaps we ought to add an exception to the dict lookup mechanism >> and continue to silence UnicodeErrors ?! > > Seems to be that comparison of unicode and non-unicode > strings for equality shouldn't raise exceptions in the > first place. Seems like a slightly better idea than having dictionaries suppress exceptions. Still not ideal though because sticking non-ASCII strings that are supposed to be text and unicode in the same data structures is *probably* still an error. -bob From aahz at pythoncraft.com Fri Aug 4 05:22:00 2006 From: aahz at pythoncraft.com (Aahz) Date: Thu, 3 Aug 2006 20:22:00 -0700 Subject: [Python-Dev] Rounding float to int directly ... In-Reply-To: <44CFC2AD.2070205@ronadam.com> References: <44CFC2AD.2070205@ronadam.com> Message-ID: <20060804032200.GA6515@panix.com> On Tue, Aug 01, 2006, Ron Adam wrote: > > I'm -1 on implicitly converting to an int when rounding. > > One reason is if your rounded (to int type) number is then used in an > equation you my end up with a integer result when you wanted a floating > point result. > > >>> 23.5/5.2 > 4.5192307692307692 > >>> round(23.5)/round(5.2) > 4.7999999999999998 > >>> round(23.5/5.2) > 5.0 > >>> int(round(23.5))/int(round(5.2)) > 4 That's not relevant to Py3K. Which is where this discussion should proably move because round() ain't changing in 2.x. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." --Brian W. Kernighan From jcarlson at uci.edu Fri Aug 4 06:34:04 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu, 03 Aug 2006 21:34:04 -0700 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: References: <44D2A824.6060502@canterbury.ac.nz> Message-ID: <20060803212410.E6B5.JCARLSON@uci.edu> Bob Ippolito wrote: > On Aug 3, 2006, at 6:51 PM, Greg Ewing wrote: > > > M.-A. Lemburg wrote: > > > >> Perhaps we ought to add an exception to the dict lookup mechanism > >> and continue to silence UnicodeErrors ?! > > > > Seems to be that comparison of unicode and non-unicode > > strings for equality shouldn't raise exceptions in the > > first place. > > Seems like a slightly better idea than having dictionaries suppress > exceptions. Still not ideal though because sticking non-ASCII strings > that are supposed to be text and unicode in the same data structures > is *probably* still an error. If/when 'python -U -c "import test.testall"' runs without unexpected error (I doubt it will happen prior to the "all strings are unicode" conversion), then I think that we can say that there aren't any use-cases for strings and unicode being in the same dictionary. As an alternate idea, rather than attempting to .decode('ascii') when strings and unicode compare, why not .decode('latin-1')? We lose the unicode decoding error, but "the right thing" happens (in my opinion) when u'\xa1' and '\xa1' compare. - Josiah From foom at fuhm.net Fri Aug 4 06:39:29 2006 From: foom at fuhm.net (James Y Knight) Date: Fri, 4 Aug 2006 00:39:29 -0400 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <20060803212410.E6B5.JCARLSON@uci.edu> References: <44D2A824.6060502@canterbury.ac.nz> <20060803212410.E6B5.JCARLSON@uci.edu> Message-ID: <1265F73D-C13D-4329-96C1-C8FD09D4F079@fuhm.net> On Aug 4, 2006, at 12:34 AM, Josiah Carlson wrote: > As an alternate idea, rather than attempting to .decode('ascii') when > strings and unicode compare, why not .decode('latin-1')? We lose the > unicode decoding error, but "the right thing" happens (in my opinion) > when u'\xa1' and '\xa1' compare. Maybe you want those to compare equal, but _I_ want u'\xa1' and '\xc2 \xa1' to compare equal, so it should obviously use .decode('utf-8')! (okay, no, I don't really want that.) James From anthony at python.org Fri Aug 4 06:41:24 2006 From: anthony at python.org (Anthony Baxter) Date: Fri, 4 Aug 2006 14:41:24 +1000 Subject: [Python-Dev] RELEASED Python 2.5 (beta 3) Message-ID: <200608041441.32253.anthony@python.org> On behalf of the Python development team and the Python community, I'm happy to announce the third BETA release of Python 2.5. This is an *beta* release of Python 2.5. As such, it is not suitable for a production environment. It is being released to solicit feedback and hopefully discover bugs, as well as allowing you to determine how changes in 2.5 might impact you. If you find things broken or incorrect, please log a bug on Sourceforge. In particular, note that changes to improve Python's support of 64 bit systems might require authors of C extensions to change their code. More information (as well as source distributions and Windows and Universal Mac OSX installers) are available from the 2.5 website: http://www.python.org/2.5/ There's been over 50 fixes since the second beta. This large number of changes meant we felt more comfortable cutting a third beta release, rather than charging ahead to the release candidate. As of this release, Python 2.5 is now in *feature freeze*. Unless absolutely necessary, no functionality changes will be made between now and the final release of Python 2.5. The plan is that this will be the final beta release (no, really, this time for sure (probably)). We should now move to one or more release candidates, leading to a 2.5 final release early August. PEP 356 includes the schedule and will be updated as the schedule evolves. At this point, any testing you can do would be greatly, greatly appreciated. The new features in Python 2.5 are described in Andrew Kuchling's What's New In Python 2.5. It's available from the 2.5 web page. Amongst the language features added include conditional expressions, the with statement, the merge of try/except and try/finally into try/except/finally, enhancements to generators to produce a coroutine kind of functionality, and a brand new AST-based compiler implementation. New modules added include hashlib, ElementTree, sqlite3, wsgiref and ctypes. In addition, a new profiling module "cProfile" was added. Enjoy this new release, Anthony Anthony Baxter anthony at python.org Python Release Manager (on behalf of the entire python-dev team) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 191 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20060804/5d50035d/attachment.pgp From exarkun at divmod.com Fri Aug 4 06:43:23 2006 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Fri, 4 Aug 2006 00:43:23 -0400 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <20060803212410.E6B5.JCARLSON@uci.edu> Message-ID: <20060804044323.1717.140314887.divmod.quotient.13837@ohm> On Thu, 03 Aug 2006 21:34:04 -0700, Josiah Carlson wrote: > >Bob Ippolito wrote: >> On Aug 3, 2006, at 6:51 PM, Greg Ewing wrote: >> >> > M.-A. Lemburg wrote: >> > >> >> Perhaps we ought to add an exception to the dict lookup mechanism >> >> and continue to silence UnicodeErrors ?! >> > >> > Seems to be that comparison of unicode and non-unicode >> > strings for equality shouldn't raise exceptions in the >> > first place. >> >> Seems like a slightly better idea than having dictionaries suppress >> exceptions. Still not ideal though because sticking non-ASCII strings >> that are supposed to be text and unicode in the same data structures >> is *probably* still an error. > >If/when 'python -U -c "import test.testall"' runs without unexpected >error (I doubt it will happen prior to the "all strings are unicode" >conversion), then I think that we can say that there aren't any >use-cases for strings and unicode being in the same dictionary. > >As an alternate idea, rather than attempting to .decode('ascii') when >strings and unicode compare, why not .decode('latin-1')? We lose the >unicode decoding error, but "the right thing" happens (in my opinion) >when u'\xa1' and '\xa1' compare. It might be right for Latin-1 strings. However, it would be even *more* surprising for the person who has to figure out why his program works when his program gets a string containing '\xc0' from one user but fails when it gets '\xe3\x81\x82' from another user. I like the exception that 2.5 raises. I only wish it raised by default when using 'ascii' and u'ascii' as keys in the same dictionary. ;) Oh, and that str and unicode did not hash like they do. ;) Jean-Paul From anthony at interlink.com.au Fri Aug 4 06:43:39 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri, 4 Aug 2006 14:43:39 +1000 Subject: [Python-Dev] TRUNK is UNFROZEN In-Reply-To: <200608022229.46957.anthony@interlink.com.au> References: <200608022229.46957.anthony@interlink.com.au> Message-ID: <200608041443.40668.anthony@interlink.com.au> The trunk is unfrozen now. Sorry about the delay - combination of a laptop motherboard replacement and a DSL provider with a network that was .. ahem... slightly slow. I'm still planning to branch at 2.5c1, the next release. That then opens the trunk up for all the bad craziness that's planned for 2.6. Anthony From murman at gmail.com Fri Aug 4 06:45:05 2006 From: murman at gmail.com (Michael Urman) Date: Thu, 3 Aug 2006 23:45:05 -0500 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <20060803212410.E6B5.JCARLSON@uci.edu> References: <44D2A824.6060502@canterbury.ac.nz> <20060803212410.E6B5.JCARLSON@uci.edu> Message-ID: On 8/3/06, Josiah Carlson wrote: > As an alternate idea, rather than attempting to .decode('ascii') when > strings and unicode compare, why not .decode('latin-1')? We lose the > unicode decoding error, but "the right thing" happens (in my opinion) > when u'\xa1' and '\xa1' compare. Since I use utf-8 way more than I use latin-1, -1. Since others do not, -1 on any not obviously correct encoding other than ascii, which gets grandfathered. This raises an exception for a good reason. Yes it's annoying at times. We should fix those times, not the (unbroken) exception. Michael -- Michael Urman http://www.tortall.net/mu/blog From nnorwitz at gmail.com Fri Aug 4 07:49:22 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 3 Aug 2006 22:49:22 -0700 Subject: [Python-Dev] 2.5 status Message-ID: Things are getting better, but we still have some really important outstanding issues. PLEASE CONTINUE TESTING AS MUCH AS POSSIBLE. Also, it would be great to use as many tools as possible to find bugs and improve quality. It would be especially nice to run Purify on Windows. I've updated PEP 356 with the current status. The current schedule is: rc 1: August 18, 2006 [planned] final: September 12, 2006 [planned] This somewhat strange schedule is to accommodate availability of people cutting the release. A branch will be made when the release candidate is done. Don't try to rush bug fixes in. Let's try to keep things stable. Keep the doc fixes coming. I believe all 3 outstanding issues (and solutions!) could use some more discussion. All bugs/patches blocking release are set to priority 9. http://python.org/sf/1530559 - struct rejecting floats (patch pending) http://mail.python.org/pipermail/python-dev/2006-July/067774.html Problem with __index__ (patch pending) http://mail.python.org/pipermail/python-dev/2006-August/067926.html str/unicode dict keys can raise an exception All the AST issues that I know about have been addressed. There are various other issues that would be nice to fix for 2.5. These are priority 8. All exist in 2.4 and possibly earlier: http://python.org/sf/1526585 - SystemError concat long strings http://python.org/sf/1523610 - PyArg_ParseTupleAndKeywords potential core dump http://python.org/sf/1517042 - Fix crashers/gc_inspection.py http://python.org/sf/1475523 - gettext.py bug http://python.org/sf/1467929 - %-formatting and dicts I will continue to use priorities. Please review bugs and patches and let me know if anything else should be re-prioritized. I will continue to stress test Python and use various tools to keep quality up. The current suite includes in no particular order: valgrind, Coverity, Klocwork, failmalloc, pychecker, not to mention the buildbot testing on commits and refleak test runs every 12 hours. Cheers, n From mal at egenix.com Fri Aug 4 09:25:07 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 04 Aug 2006 09:25:07 +0200 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com> References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com> Message-ID: <44D2F653.5090305@egenix.com> Delaney, Timothy (Tim) wrote: > M.-A. Lemburg wrote: > >> Perhaps we ought to add an exception to the dict lookup mechanism >> and continue to silence UnicodeErrors ?! > > I'd definitely consider a UnicodeError to be an indication that two > objects are not equal. Not really: Python expects all strings to be ASCII whenever they meet Unicode strings and have to be converted to Unicode. If a string is not ASCII and thus causes the exception, there's not a lot you can say, since you don't know the encoding of the string. All you know is that it's not ASCII. Instead of guessing, Python then raises an exception to let the programmer decide. > At the very least, in the context of a dictionary > lookup. The point here is that a typical user won't expect any comparisons to be made when dealing with dictionaries, simply because the fact that you do need to make comparisons is an implementation detail. So in this particular case silencing the exception might be the more user friendly way of dealing with the problem. That said, the problem still lingers in that dictionary, so it may bite you in some other context, e.g. when iterating over the list of keys. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 04 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From rasky at develer.com Fri Aug 4 10:58:16 2006 From: rasky at develer.com (Giovanni Bajo) Date: Fri, 4 Aug 2006 10:58:16 +0200 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com> <44D2F653.5090305@egenix.com> Message-ID: <141f01c6b7a4$2001b460$8cb22997@bagio> M.-A. Lemburg wrote: >> At the very least, in the context of a dictionary >> lookup. > > The point here is that a typical user won't expect any comparisons > to be made when dealing with dictionaries, simply because the fact > that you do need to make comparisons is an implementation detail. I'd also weight in the fact that this is a change of behaviour since the introduction of unicode in 2.0, and it's break real-world applications (as demonstrated by this thread). I think Python ought to revert to the previous behaviour and, in case there's interest, the discussion moved to the py3k list. Giovanni Bajo From greg.ewing at canterbury.ac.nz Fri Aug 4 11:14:57 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 04 Aug 2006 21:14:57 +1200 Subject: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers) In-Reply-To: <06264EFB-FE9A-4E94-B80D-C621D6BC7AD3@fuhm.net> References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com> <44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com> <44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com> <44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com> <44CFF9B9.3010402@canterbury.ac.nz> <44D09A5C.4030905@egenix.com> <44D15FB4.5020605@canterbury.ac.nz> <44D16CFF.7090001@ewtllc.com> <44D198F2.9020702@canterbury.ac.nz> <06264EFB-FE9A-4E94-B80D-C621D6BC7AD3@fuhm.net> Message-ID: <44D31011.4040607@canterbury.ac.nz> James Y Knight wrote: > And it does in C because C doesn't have arbitrary size integers, so if > round returned integers, round(1e+308) couldn't work. Also, in C you can use the result directly in an int context without complaint. In Python these days, that is usually disallowed. -- Greg From ralf at brainbot.com Fri Aug 4 11:18:28 2006 From: ralf at brainbot.com (Ralf Schmitt) Date: Fri, 04 Aug 2006 11:18:28 +0200 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <20060804044323.1717.140314887.divmod.quotient.13837@ohm> References: <20060803212410.E6B5.JCARLSON@uci.edu> <20060804044323.1717.140314887.divmod.quotient.13837@ohm> Message-ID: <44D310E4.10304@brainbot.com> Jean-Paul Calderone wrote: > > I like the exception that 2.5 raises. I only wish it raised by default > when using 'ascii' and u'ascii' as keys in the same dictionary. ;) Oh, > and that str and unicode did not hash like they do. ;) No problem: >>> import sys >>> reload(sys) >>> sys.setdefaultencoding("base64") >>> "a"==u"a" Traceback (most recent call last): File "", line 1, in File "/exp/lib/python2.5/encodings/base64_codec.py", line 42, in base64_decode output = base64.decodestring(input) File "/exp/lib/python2.5/base64.py", line 321, in decodestring return binascii.a2b_base64(s) binascii.Error: Incorrect padding >>> "a"=="a" True >>> d={u"a":1, "a":1} Traceback (most recent call last): File "", line 1, in File "/exp/lib/python2.5/encodings/base64_codec.py", line 42, in base64_decode output = base64.decodestring(input) File "/exp/lib/python2.5/base64.py", line 321, in decodestring return binascii.a2b_base64(s) binascii.Error: Incorrect padding Maybe this is all just a matter of choosing the right defaultencoding ? :) BTW, python 2.4 also suppresses this exception (when instantiating the dictionary) Does python 2.4 catch any exception when comparing keys (which are not basestrings) in dictionaries? - Ralf From mal at egenix.com Fri Aug 4 11:26:42 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 04 Aug 2006 11:26:42 +0200 Subject: [Python-Dev] 2.5 status In-Reply-To: References: Message-ID: <44D312D2.9020608@egenix.com> Neal Norwitz wrote: > I believe all 3 outstanding issues (and solutions!) could use some > more discussion. All bugs/patches blocking release are set to > priority 9. > > http://python.org/sf/1530559 - struct rejecting floats (patch pending) > > http://mail.python.org/pipermail/python-dev/2006-July/067774.html > Problem with __index__ (patch pending) > > http://mail.python.org/pipermail/python-dev/2006-August/067926.html > str/unicode dict keys can raise an exception I'd like to add this issue: http://mail.python.org/pipermail/python-dev/2006-August/067869.html distutils version string should become static again This is part of a longer discussion we can have after 2.6 final is out, but for now, I would like to request that the change http://mail.python.org/pipermail/python-checkins/2006-July/055139.html be backed out again before c1 gets released. In the above patch, the distutils version was made to dynamically depend on the Python version running the code, effectively making it impossible to query the true distutils version that is being used, bypassing the whole purpose of having static version in the code. I don't really see manually bumping the version number every now and then to be such a great issue and have also volunteered to take care of that in the future. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 04 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Aug 4 11:29:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 04 Aug 2006 11:29:38 +0200 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <44D310E4.10304@brainbot.com> References: <20060803212410.E6B5.JCARLSON@uci.edu> <20060804044323.1717.140314887.divmod.quotient.13837@ohm> <44D310E4.10304@brainbot.com> Message-ID: <44D31382.80605@egenix.com> Ralf Schmitt wrote: > Does python 2.4 catch any exception when comparing keys (which are not > basestrings) in dictionaries? Yes. It does so for all equality compares that need to be done as part of the hash collision algorithm (not only w/r to strings and Unicode, but in general). This was changed in 2.5, which now reports the exception. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 04 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From greg.ewing at canterbury.ac.nz Fri Aug 4 11:31:54 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 04 Aug 2006 21:31:54 +1200 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <44D2F653.5090305@egenix.com> References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com> <44D2F653.5090305@egenix.com> Message-ID: <44D3140A.2040600@canterbury.ac.nz> M.-A. Lemburg wrote: > If a string > is not ASCII and thus causes the exception, there's not a lot you > can say, since you don't know the encoding of the string. That's one way of looking at it. Another is that any string containing chars > 127 is not text at all, but binary data, in which case it's not equal to *any* unicode string -- just like bytes objects will not be equal to strings in Py3k. > All you > know is that it's not ASCII. Instead of guessing, Python then raises > an exception to let the programmer decide. There's no disputing that an exception should be raised if the string *must* be interpretable as characters in order to continue. But that's not true here if you allow for the interpretation that they're simply objects of different (duck) type and therefore unequal. -- Greg From ralf at brainbot.com Fri Aug 4 11:42:11 2006 From: ralf at brainbot.com (Ralf Schmitt) Date: Fri, 04 Aug 2006 11:42:11 +0200 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <44D31382.80605@egenix.com> References: <20060803212410.E6B5.JCARLSON@uci.edu> <20060804044323.1717.140314887.divmod.quotient.13837@ohm> <44D310E4.10304@brainbot.com> <44D31382.80605@egenix.com> Message-ID: <44D31673.9040307@brainbot.com> M.-A. Lemburg wrote: > Ralf Schmitt wrote: >> Does python 2.4 catch any exception when comparing keys (which are not >> basestrings) in dictionaries? > > Yes. It does so for all equality compares that need to be done > as part of the hash collision algorithm (not only w/r to strings > and Unicode, but in general). > > This was changed in 2.5, which now reports the exception. > So, this thread isn't about "unicode hell" at all. I guess this change will break lots of code (or will reveal lots of broken code...as it did in my case actually). - Ralf From g.brandl at gmx.net Fri Aug 4 11:49:24 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 04 Aug 2006 11:49:24 +0200 Subject: [Python-Dev] 2.5 status In-Reply-To: References: Message-ID: Neal Norwitz wrote: > Things are getting better, but we still have some really important > outstanding issues. PLEASE CONTINUE TESTING AS MUCH AS POSSIBLE. > Also, it would be great to use as many tools as possible to find bugs > and improve quality. It would be especially nice to run Purify on > Windows. > > I've updated PEP 356 with the current status. The current schedule is: > > rc 1: August 18, 2006 [planned] > final: September 12, 2006 [planned] > > This somewhat strange schedule is to accommodate availability of > people cutting the release. A branch will be made when the release > candidate is done. Don't try to rush bug fixes in. Let's try to keep > things stable. Keep the doc fixes coming. > http://python.org/sf/1523610 - PyArg_ParseTupleAndKeywords > potential core dump This one's almost fixed if we can decide what to do with "levels". I wrote some time ago: """ With respect to this bug (which is about stack issues in Python/getargs.c involving misuse of the "levels" array), I think that we can drop the "levels" thing completely. It's only there to tell the user which exact item passed as part of a tuple argument cannot be accepted (and only if that function is implemented in C code). As tuple arguments are very rare "argument x" should be enough to tell the user that something's wrong with that tuple. """ Georg From mwh at python.net Fri Aug 4 12:06:11 2006 From: mwh at python.net (Michael Hudson) Date: Fri, 04 Aug 2006 11:06:11 +0100 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <44D2F653.5090305@egenix.com> (M.'s message of "Fri, 04 Aug 2006 09:25:07 +0200") References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com> <44D2F653.5090305@egenix.com> Message-ID: <2mejvwx1bg.fsf@starship.python.net> "M.-A. Lemburg" writes: > The point here is that a typical user won't expect any comparisons > to be made when dealing with dictionaries, simply because the fact > that you do need to make comparisons is an implementation detail. Of course looking things up in a dictionary involves comparisons! How could it not? > So in this particular case silencing the exception might be the > more user friendly way of dealing with the problem. Please, no. > That said, the problem still lingers in that dictionary, so it may > bite you in some other context, e.g. when iterating over the list > of keys. For this reason, and others. Cheers, mwh -- web in my head get it out get it out -- from Twisted.Quotes From mal at egenix.com Fri Aug 4 12:16:04 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 04 Aug 2006 12:16:04 +0200 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <44D3140A.2040600@canterbury.ac.nz> References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com> <44D2F653.5090305@egenix.com> <44D3140A.2040600@canterbury.ac.nz> Message-ID: <44D31E64.3070407@egenix.com> Greg Ewing wrote: > M.-A. Lemburg wrote: > >> If a string >> is not ASCII and thus causes the exception, there's not a lot you >> can say, since you don't know the encoding of the string. > > That's one way of looking at it. > > Another is that any string containing chars > 127 is not > text at all, but binary data, in which case it's not equal > to *any* unicode string -- just like bytes objects will > not be equal to strings in Py3k. > >> All you >> know is that it's not ASCII. Instead of guessing, Python then raises >> an exception to let the programmer decide. > > There's no disputing that an exception should be raised > if the string *must* be interpretable as characters in > order to continue. But that's not true here if you allow > for the interpretation that they're simply objects of > different (duck) type and therefore unequal. Hmm, given that interpretation, 1 == 1.0 would have to be False. Note that you do have to interpret the string as characters if you compare it to Unicode and there's nothing wrong with that. What's making this particular case interesting is that the comparison is hidden in the dictionary implementation and only triggers if you get a hash collision, which makes the whole issue appear to be happening randomly. This whole thread aside: it's never recommended to mix strings and Unicode, unless you really have to. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 04 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Aug 4 12:22:35 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 04 Aug 2006 12:22:35 +0200 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <44D31673.9040307@brainbot.com> References: <20060803212410.E6B5.JCARLSON@uci.edu> <20060804044323.1717.140314887.divmod.quotient.13837@ohm> <44D310E4.10304@brainbot.com> <44D31382.80605@egenix.com> <44D31673.9040307@brainbot.com> Message-ID: <44D31FEB.9000806@egenix.com> Ralf Schmitt wrote: > M.-A. Lemburg wrote: >> Ralf Schmitt wrote: >>> Does python 2.4 catch any exception when comparing keys (which are not >>> basestrings) in dictionaries? >> Yes. It does so for all equality compares that need to be done >> as part of the hash collision algorithm (not only w/r to strings >> and Unicode, but in general). >> >> This was changed in 2.5, which now reports the exception. >> > > So, this thread isn't about "unicode hell" at all. For some reason people always think of hell when dealing with Unicode. Instead, the should think of hell when dealing with strings. > I guess this change will break lots of code (or will reveal lots of > broken code...as it did in my case actually). I don't think it will break a lot of code that wasn't already broken. Let's see how many reports we get for 2.5b3 and then decide. If things turn out bad, we might silence the UnicodeError and instead issue a warning everytime this situation occurs. In 2.6 we'd then revert to raising an exception. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 04 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From bob at redivi.com Fri Aug 4 12:50:30 2006 From: bob at redivi.com (Bob Ippolito) Date: Fri, 4 Aug 2006 03:50:30 -0700 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <20060803212410.E6B5.JCARLSON@uci.edu> References: <44D2A824.6060502@canterbury.ac.nz> <20060803212410.E6B5.JCARLSON@uci.edu> Message-ID: <9A11B598-DEF0-4ED7-9D17-789BB89E05A5@redivi.com> On Aug 3, 2006, at 9:34 PM, Josiah Carlson wrote: > > Bob Ippolito wrote: >> On Aug 3, 2006, at 6:51 PM, Greg Ewing wrote: >> >>> M.-A. Lemburg wrote: >>> >>>> Perhaps we ought to add an exception to the dict lookup mechanism >>>> and continue to silence UnicodeErrors ?! >>> >>> Seems to be that comparison of unicode and non-unicode >>> strings for equality shouldn't raise exceptions in the >>> first place. >> >> Seems like a slightly better idea than having dictionaries suppress >> exceptions. Still not ideal though because sticking non-ASCII strings >> that are supposed to be text and unicode in the same data structures >> is *probably* still an error. > > If/when 'python -U -c "import test.testall"' runs without unexpected > error (I doubt it will happen prior to the "all strings are unicode" > conversion), then I think that we can say that there aren't any > use-cases for strings and unicode being in the same dictionary. > > As an alternate idea, rather than attempting to .decode('ascii') when > strings and unicode compare, why not .decode('latin-1')? We lose the > unicode decoding error, but "the right thing" happens (in my opinion) > when u'\xa1' and '\xa1' compare. Well, in this case it would cause different behavior if u'\xa1' and '\xa1' compared equal. It'd just be an even more subtle error. -bob From kristjan at ccpgames.com Fri Aug 4 13:41:21 2006 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_V=2E_J=F3nsson?=) Date: Fri, 4 Aug 2006 11:41:21 -0000 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionarykeys Message-ID: <129CEF95A523704B9D46959C922A280002FE9868@nemesis.central.ccp.cc> The "string" isn?t necessarily text, so selecting latin-1 doesn?t help (in fact, what happens is that the current default encoding is used, in his case this was ascii). What if it is image data? What if you are using a dict to implement a singleton set for arbitrary objects? The point is that if the comparison operator raises an exception, the two objects are likely to be dissimilar. We could even define that behaviour. Propagating the exception means that you can't have objects as keys in a dictionary that raise an exception when compared. This goes over and beyond any unicode vs. string question. If the propagation of the exception was a concious change for debugging purposes, why not make that somehow optional? A flag on the dict object? Or special lookup mehtods for that? Cheers, Kristj?n > -----Original Message----- > From: python-dev-bounces+kristjan=ccpgames.com at python.org > [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] > On Behalf Of Josiah Carlson > Sent: 4. ?g?st 2006 04:34 > To: Bob Ippolito; python-dev at python.org > Subject: Re: [Python-Dev] unicode hell/mixing str and unicode > as dictionarykeys > > > Bob Ippolito wrote: > > On Aug 3, 2006, at 6:51 PM, Greg Ewing wrote: > > > > > M.-A. Lemburg wrote: > > > > > >> Perhaps we ought to add an exception to the dict lookup > mechanism > > >> and continue to silence UnicodeErrors ?! > > > > > > Seems to be that comparison of unicode and non-unicode > strings for > > > equality shouldn't raise exceptions in the first place. > > > > Seems like a slightly better idea than having dictionaries suppress > > exceptions. Still not ideal though because sticking > non-ASCII strings > > that are supposed to be text and unicode in the same data > structures > > is *probably* still an error. > > If/when 'python -U -c "import test.testall"' runs without > unexpected error (I doubt it will happen prior to the "all > strings are unicode" > conversion), then I think that we can say that there aren't > any use-cases for strings and unicode being in the same dictionary. > > As an alternate idea, rather than attempting to > .decode('ascii') when strings and unicode compare, why not > .decode('latin-1')? We lose the unicode decoding error, but > "the right thing" happens (in my opinion) when u'\xa1' and > '\xa1' compare. > > - Josiah > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/kristjan%40c cpgames.com > From mcherm at mcherm.com Fri Aug 4 14:26:52 2006 From: mcherm at mcherm.com (Michael Chermside) Date: Fri, 04 Aug 2006 05:26:52 -0700 Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys Message-ID: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com> I'm changing the subject line because I want to convince everyone that the problem being discussed in the "unicode hell" thread has nothing to do with unicode and strings. It's all about dicts. I have not observed real breakage in my own code, but I will present a sample of made-up-but-completely-reasonable code that works in 2.4, fails in 2.5, and arguably ought to work fine. I think we should restore the behavior of dicts that when they compare keys for equality they suppress exceptions (treating the objects as unequal), or at LEAST retain the behavior for one more release making it a warning this time. Here is my sample code: ----------- problem_with_dicts.py ---------- # A sample program to demonstrate that the proposed behavior # of dicts in Python 2.5 generates bugs. This is not code from # an actual program, but it is completely reasonable. # First import from # http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/413486 # the only 5-star recipe in the Python Cookbook for implementing # enums. import cookbookenum # Then set up some reasonable enums. We'll say we're writing # a program for dealing with dates. DaysOfWeek = cookbookenum.Enum( 'Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat', 'Sun') Months = cookbookenum.Enum( 'Jan','Feb','Mar','Apr','May','Jun','Jul','Aug', 'Sep','Oct','Nov','Dec') # Let's say we also do some translations. Here is a # useful dictionary: translations = {} # which we populate with values translations[ DaysOfWeek.Mon ] = { 'en': 'Monday', 'es': 'Lunes', 'fr': 'Lundi', } # and assume we do the other days translations[ Months.Jan ] = { 'en': 'January', 'es': 'Enero', 'fr': 'Janvier', } # and assume we do the other months # ...then later in the code we could do things like this: language = 'en' dayOfWeek = DaysOfWeek.Mon month = Months.Jan dayOfMonth = 3 print '%s, %s %s' % ( translations[dayOfWeek][language], translations[month][language], dayOfMonth) # this works in 2.4 but fails in 2.5 --------- end problem_with_dicts.py -------- Please reconsider. -- Michael Chermside From radix at twistedmatrix.com Fri Aug 4 16:01:51 2006 From: radix at twistedmatrix.com (Christopher Armstrong) Date: Fri, 4 Aug 2006 10:01:51 -0400 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <44D310E4.10304@brainbot.com> References: <20060803212410.E6B5.JCARLSON@uci.edu> <20060804044323.1717.140314887.divmod.quotient.13837@ohm> <44D310E4.10304@brainbot.com> Message-ID: <60ed19d40608040701v59474febw94f7f374ad47d92a@mail.gmail.com> On 8/4/06, Ralf Schmitt wrote: > > Jean-Paul Calderone wrote: > > > > I like the exception that 2.5 raises. I only wish it raised by default > > when using 'ascii' and u'ascii' as keys in the same dictionary. ;) Oh, > > and that str and unicode did not hash like they do. ;) > > No problem: > > >>> import sys > >>> reload(sys) > > >>> sys.setdefaultencoding("base64") > >>> "a"==u"a" > Traceback (most recent call last): > ... > binascii.Error: Incorrect padding > > > Maybe this is all just a matter of choosing the right defaultencoding ? :) Doing this is amazingly stupid. I can't believe how often I hear this suggestion. Apparently the fact that you have to "reload(sys)" to do it isn't warning enough? -- Christopher Armstrong International Man of Twistery http://radix.twistedmatrix.com/ http://twistedmatrix.com/ http://canonical.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060804/6f4e00fa/attachment.html From ralf at brainbot.com Fri Aug 4 16:30:24 2006 From: ralf at brainbot.com (Ralf Schmitt) Date: Fri, 04 Aug 2006 16:30:24 +0200 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <60ed19d40608040701v59474febw94f7f374ad47d92a@mail.gmail.com> References: <20060803212410.E6B5.JCARLSON@uci.edu> <20060804044323.1717.140314887.divmod.quotient.13837@ohm> <44D310E4.10304@brainbot.com> <60ed19d40608040701v59474febw94f7f374ad47d92a@mail.gmail.com> Message-ID: <44D35A00.70107@brainbot.com> Christopher Armstrong wrote: > On 8/4/06, *Ralf Schmitt* > > wrote: > > > Maybe this is all just a matter of choosing the right > defaultencoding ? :) > > > > Doing this is amazingly stupid. I can't believe how often I hear this > suggestion. Apparently the fact that you have to "reload(sys)" to do it > isn't warning enough? Ahh, the twisted people, friendly as I know them. Did you actually notice the smiley? - Ralf From mwh at python.net Fri Aug 4 16:38:32 2006 From: mwh at python.net (Michael Hudson) Date: Fri, 04 Aug 2006 15:38:32 +0100 Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys In-Reply-To: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com> (Michael Chermside's message of "Fri, 04 Aug 2006 05:26:52 -0700") References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com> Message-ID: <2m3bccwopj.fsf@starship.python.net> Michael Chermside writes: > I'm changing the subject line because I want to convince everyone that > the problem being discussed in the "unicode hell" thread has nothing > to do with unicode and strings. It's all about dicts. I'd say it's more to do with __eq__. It's a strange __eq__ method that raises an Exception, IMHO. Please do realize that the motivation for this change was hours and hours of tortous debugging caused by a buggy __eq__ method making keys "impossibly" seem to not be in dictionaries. > I have not observed real breakage in my own code, but I will present > a sample of made-up-but-completely-reasonable code that works in > 2.4, fails in 2.5, and arguably ought to work fine. I think we should > restore the behavior of dicts that when they compare keys for > equality they suppress exceptions (treating the objects as unequal), > or at LEAST retain the behavior for one more release making it a > warning this time. Please no. Here's just one piece of evidence that the 2.4 semantics are pretty silly too: >>> d = {u'\xe0':1, '\xe0\:2} File "", line 1 d = {u'\xe0':1, '\xe0\:2} ^ SyntaxError: EOL while scanning single-quoted string >>> d = {u'\xe0':1, '\xe0':2} >>> '\xe0' in d True >>> '\xe0' in d.keys() Traceback (most recent call last): File "", line 1, in ? UnicodeDecodeError: 'ascii' codec can't decode byte 0xe0 in position 0: ordinal not in range(128) Cheers, mwh -- same software, different verbosity settings (this one goes to eleven) -- the effbot on the martellibot From tjreedy at udel.edu Fri Aug 4 17:18:02 2006 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 4 Aug 2006 11:18:02 -0400 Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com> <2m3bccwopj.fsf@starship.python.net> Message-ID: "Michael Hudson" wrote in message news:2m3bccwopj.fsf at starship.python.net... > Michael Chermside writes: > >> I'm changing the subject line because I want to convince everyone that >> the problem being discussed in the "unicode hell" thread has nothing >> to do with unicode and strings. It's all about dicts. > > I'd say it's more to do with __eq__. It's a strange __eq__ method > that raises an Exception, IMHO. I agree; a == b should always work, certainly unless explicitly programmed otherwise in Python for a particular class. So I think the proper solution is fix the buggy __eq__ method to return False instead. If a byte string does not have a default (ascii) text interpretation, then it obviously is not equal to any particular unicode text. The fundamental axiom of sets and hence of dict keys is that any object/value either is or is not a member (at any given time for 'mutable' set collections). This requires that testing an object for possible membership by equality give a clean True or False answer. > Please do realize that the motivation for this change was hours and > hours of tortous debugging caused by a buggy __eq__ method making keys > "impossibly" seem to not be in dictionaries. So why not fix the buggy __eq__ method? >> 2.4, fails in 2.5, and arguably ought to work fine. I think we should >> restore the behavior of dicts that when they compare keys for >> equality they suppress exceptions (treating the objects as unequal), >> or at LEAST retain the behavior for one more release making it a >> warning this time. > > Please no. Here's just one piece of evidence that the 2.4 semantics > are pretty silly too: We mostly agreed half a decode ago that 1/2 should be .5 instead of 0, but to avoid breaking code, have (or Guido has) refrained from yet making the change the default. To me, the same principle applies here at least as strongly. Terry Jan Reedy From mal at egenix.com Fri Aug 4 17:43:50 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 04 Aug 2006 17:43:50 +0200 Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys In-Reply-To: References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com> <2m3bccwopj.fsf@starship.python.net> Message-ID: <44D36B36.8030202@egenix.com> Terry Reedy wrote: > "Michael Hudson" wrote in message > news:2m3bccwopj.fsf at starship.python.net... >> Michael Chermside writes: >> >>> I'm changing the subject line because I want to convince everyone that >>> the problem being discussed in the "unicode hell" thread has nothing >>> to do with unicode and strings. It's all about dicts. >> I'd say it's more to do with __eq__. It's a strange __eq__ method >> that raises an Exception, IMHO. > > I agree; a == b should always work, certainly unless explicitly programmed > otherwise in Python for a particular class. ... which this is. > So I think the proper solution > is fix the buggy __eq__ method to return False instead. If a byte string > does not have a default (ascii) text interpretation, then it obviously is > not equal to any particular unicode text. > > The fundamental axiom of sets and hence of dict keys is that any > object/value either is or is not a member (at any given time for 'mutable' > set collections). This requires that testing an object for possible > membership by equality give a clean True or False answer. > >> Please do realize that the motivation for this change was hours and >> hours of tortous debugging caused by a buggy __eq__ method making keys >> "impossibly" seem to not be in dictionaries. > > So why not fix the buggy __eq__ method? Because it's not buggy. Python just doesn't know the encoding of the 8-bit string, so can't make any assumptions on it. As result, it raises an exception to inform the programmer. It is well possible that the string uses an encoding where the Unicode string is indeed the equal to the string, assuming this encoding, e.g. >>> s = 'tr?r??' >>> u = u'tr?r??' >>> s == u Traceback (most recent call last): File "", line 1, in ? UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 2: ordinal not in range(128) >>> hash(s) 673683206 >>> hash(u) 673683206 Here, the encoding that creates the match is Latin-1. >>> 2.4, fails in 2.5, and arguably ought to work fine. I think we should >>> restore the behavior of dicts that when they compare keys for >>> equality they suppress exceptions (treating the objects as unequal), >>> or at LEAST retain the behavior for one more release making it a >>> warning this time. >> Please no. Here's just one piece of evidence that the 2.4 semantics >> are pretty silly too: > > We mostly agreed half a decode ago that 1/2 should be .5 instead of 0, but > to avoid breaking code, have (or Guido has) refrained from yet making the > change the default. To me, the same principle applies here at least as > strongly. I think that's a different category of semantic change: the integer division change will cause applications to return wrong data (if not fixed properly). The exception will just let the application refuse to continue. How about generating a warning instead and then go for the exception in 2.6 ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 04 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From barry at python.org Fri Aug 4 17:59:43 2006 From: barry at python.org (Barry Warsaw) Date: Fri, 4 Aug 2006 11:59:43 -0400 Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys In-Reply-To: <44D36B36.8030202@egenix.com> References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com> <2m3bccwopj.fsf@starship.python.net> <44D36B36.8030202@egenix.com> Message-ID: <9AD92B30-3CF7-4357-9B64-D732F327C28F@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 4, 2006, at 11:43 AM, M.-A. Lemburg wrote: > How about generating a warning instead and then go for the exception > in 2.6 ? From the perspective of wanting to avoid blog entries in 2007 railing against our gratuitous breakages in Python 2.5 but also wanting to avoid perpetuating broken code, this solution seems the most reasonable. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (Darwin) iQCVAwUBRNNu8HEjvBPtnXfVAQL9MQP/SCuLPFwS0m5vWJ3+i2iVZVCg21eXKQte R8zoTngSx7unxfn5WQ7Bi8l9Ai1SkmZ7z0mOr6UbtRXmxM9+HwSFvpYpWazuaC4R AylYA1ZbfsnzplHZW/TPxZKZJG++qWK2+mIcdHa9MS6OoBb583BQ4oXN8gs6tT2P 9VoUL5OW4Gw= =IPeG -----END PGP SIGNATURE----- From mkent at emdeon.com Fri Aug 4 16:55:45 2006 From: mkent at emdeon.com (Michael Kent) Date: Fri, 4 Aug 2006 14:55:45 +0000 (UTC) Subject: [Python-Dev] Python 2.5b3 and AIX 4.3 - It Works Message-ID: Because of a requirement to remain compatible with AIX 4.3, I have been forced to stay with Python 2.3, because while 2.4 would compile under AIX 4.3, it would segfault immediately when run. I'm happy to report that Python 2.5b3 compiles and runs fine under AIX 4.3, and passes most of its test suite. However, here are a few test failures. I realize AIX 4.3 is long obsolete, so is there any interest on the list for me to report which tests failed and how? From python-dev at zesty.ca Fri Aug 4 19:32:32 2006 From: python-dev at zesty.ca (Ka-Ping Yee) Date: Fri, 4 Aug 2006 12:32:32 -0500 (CDT) Subject: [Python-Dev] uuid module - byte order issue In-Reply-To: <7168d65a0608031311g3479824ds18a9ede4a33cae7e@mail.gmail.com> References: <7168d65a0608031311g3479824ds18a9ede4a33cae7e@mail.gmail.com> Message-ID: On Thu, 3 Aug 2006, Oren Tirosh wrote: > The UUID module uses network byte order, regardless of the platform > byte order. On little-endian platforms like Windows the ".bytes" > property of UUID objects is not compatible with the memory layout RFC 4122 says: In the absence of explicit application or presentation protocol specification to the contrary, a UUID is encoded as a 128-bit object, as follows: The fields are encoded as 16 octets, with the sizes and order of the fields defined above, and with each field encoded with the Most Significant Byte first (known as network byte order). > Ka-Ping Yee writes* that the Windows UUID generation calls are not RFC > 4122 compliant and have an illegal version field. [...] > If the correct byte order is used the UUIDs generated by Windows XP > are valid version 4 UUIDs: I see. I stand corrected, then. My interpretation of RFC 4122 would be that the uuid.py module currently completely implements RFC 4122; while Windows XP can't be said to violate RFC 4122, supporting this alternative byte order would be an optional feature. > The bytes property and bytes argument to the constructor should use > the platform byte order. I disagree; i think the main interface to uuid.py should behave consistently independent of the platform, and should conform to the default encoding in RFC 4122. However, we could certainly consider adding a 'bytes_le' field to support the functionality you describe. I'll ask Anthony whether adding such a field would be permitted at this stage of the release process. > There is another issue with version 1 uuid generation: > >>> len(set(uuid.uuid1() for i in range(1000))) > 992 > > The problem is that the random clock_seq field is only 14 bits long. > If enough UUIDs are generated within the same system clock tick there > will be collisions. The clock field of the UUID has enough resolution to avoid collisions; the problem you're describing is a limitation of the platform's clock, not the UUID module. It doesn't happen on Mac OS X, for example. > Suggested solution: use the high-resolution of the > time field (100ns) to generate a monotonically increasing timestamp > that advances at least by 1 for each call, when time.time() returns > the same value on subsequent calls. That sounds like a fine solution to me. I'll look into it. -- ?!ng From nnorwitz at gmail.com Fri Aug 4 19:36:33 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Fri, 4 Aug 2006 10:36:33 -0700 Subject: [Python-Dev] Python 2.5b3 and AIX 4.3 - It Works In-Reply-To: References: Message-ID: There is at least one outstanding bug report for test_mmap failing on AIX IIRC. Possibly another for test_resource. Please review bug reports and file new ones/update old ones with the current status. Unless if you provide patches, they probably won't be fixed though. No one has access to AIX AFAIK. n -- On 8/4/06, Michael Kent wrote: > Because of a requirement to remain compatible with AIX 4.3, I have been forced > to stay with Python 2.3, because while 2.4 would compile under AIX 4.3, it would > segfault immediately when run. > > I'm happy to report that Python 2.5b3 compiles and runs fine under AIX 4.3, and > passes most of its test suite. However, here are a few test failures. I > realize AIX 4.3 is long obsolete, so is there any interest on the list for me to > report which tests failed and how? > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com > From mcherm at mcherm.com Fri Aug 4 19:49:53 2006 From: mcherm at mcherm.com (Michael Chermside) Date: Fri, 04 Aug 2006 10:49:53 -0700 Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys Message-ID: <20060804104953.fhak73l8ffcw8w4o@login.werra.lunarpages.com> Marc-Andre Lemburg writes: > How about generating a warning instead and then go for the exception > in 2.6 ? Agreed. Michael Hudson's explanation convinced me. -- Michael Chermside From jcarlson at uci.edu Fri Aug 4 20:23:10 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Fri, 04 Aug 2006 11:23:10 -0700 Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys In-Reply-To: <9AD92B30-3CF7-4357-9B64-D732F327C28F@python.org> References: <44D36B36.8030202@egenix.com> <9AD92B30-3CF7-4357-9B64-D732F327C28F@python.org> Message-ID: <20060804112104.E6CD.JCARLSON@uci.edu> There's one problem with generating a warning for 2.5, and that is the same problem as generating a warning for possible packages that lack an __init__.py; users may start to get a bunch of warnings, and be unaware of how to suppress them. All in all though, I'm +0 on the warning, and +1 on it not raising an exception in 2.5 . - Josiah Barry Warsaw wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Aug 4, 2006, at 11:43 AM, M.-A. Lemburg wrote: > > > How about generating a warning instead and then go for the exception > > in 2.6 ? > > From the perspective of wanting to avoid blog entries in 2007 > railing against our gratuitous breakages in Python 2.5 but also > wanting to avoid perpetuating broken code, this solution seems the > most reasonable. > > - -Barry > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.3 (Darwin) > > iQCVAwUBRNNu8HEjvBPtnXfVAQL9MQP/SCuLPFwS0m5vWJ3+i2iVZVCg21eXKQte > R8zoTngSx7unxfn5WQ7Bi8l9Ai1SkmZ7z0mOr6UbtRXmxM9+HwSFvpYpWazuaC4R > AylYA1ZbfsnzplHZW/TPxZKZJG++qWK2+mIcdHa9MS6OoBb583BQ4oXN8gs6tT2P > 9VoUL5OW4Gw= > =IPeG > -----END PGP SIGNATURE----- > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/jcarlson%40uci.edu From exarkun at divmod.com Fri Aug 4 20:24:48 2006 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Fri, 4 Aug 2006 14:24:48 -0400 Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys In-Reply-To: <20060804112104.E6CD.JCARLSON@uci.edu> Message-ID: <20060804182448.1717.1264031474.divmod.quotient.14372@ohm> On Fri, 04 Aug 2006 11:23:10 -0700, Josiah Carlson wrote: > >There's one problem with generating a warning for 2.5, and that is the >same problem as generating a warning for possible packages that lack an >__init__.py; users may start to get a bunch of warnings, and be unaware >of how to suppress them. > >All in all though, I'm +0 on the warning, and +1 on it not raising an >exception in 2.5 . > Um. This warning would indicate a bug in the code which will lead to actual misbehavior in a future version of Python. The __init__.py warning would have indicated a deployment configuration which didn't actually cause any misbehavior. They aren't the same case at all, unless you think that all warnings should be classed this way (a position I do not think is completely unreasonable, but since you singled out the package warning by way of comparison, I assume this is not the argument you are trying to make). Jean-Paul From pc at gafol.net Fri Aug 4 20:31:38 2006 From: pc at gafol.net (Paul Colomiets) Date: Fri, 04 Aug 2006 21:31:38 +0300 Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys In-Reply-To: References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com> <2m3bccwopj.fsf@starship.python.net> Message-ID: <44D3928A.1090700@gafol.net> Hi! Terry Reedy wrote: > The fundamental axiom of sets and hence of dict keys is that any > object/value either is or is not a member (at any given time for 'mutable' > set collections). This requires that testing an object for possible > membership by equality give a clean True or False answer. > Yes this makes sense. But returning to dictionaries for python newbies, it will be strange why this >>> d = { u'abc': 1, u'ab\xe8': 2} >>> d['abc'] >1 works as expected, but this >>> d['ab\xe8'] raises an exception. Another good argument pronounced by M.-A. Lemburg: > What's making this particular case interesting is that > the comparison is hidden in the dictionary implementation > and only triggers if you get a hash collision, which makes > the whole issue appear to be happening randomly. > > This whole thread aside: it's never recommended to mix strings > and Unicode, unless you really have to. ... >How about generating a warning instead and then go for the exception >in 2.6 ? Well it's not recomended to mix strings and unicode in the dictionaries but if we mix for example integer and float we have the same thing. It doesn't raise exception but still it is not expected behavior for me: >>> d = { 1.0: 10, 2.0: 20 } then if i somewhere later do: >>> d[1] = 100 >>> d[2] = 200 to have here all floats in d.keys(). May be this is not a best example. So if you generate a warning, it should be generated every time when there are keys of different types inserted into dict. May be python should check type of the key after collision and before testing for equality? So the 1 and 1.0 is different as u'a' and 'a' also different. It even can give some perfomance overhead I think. -- Regards, Paul. From nmm1 at cus.cam.ac.uk Fri Aug 4 20:59:34 2006 From: nmm1 at cus.cam.ac.uk (Nick Maclaren) Date: Fri, 04 Aug 2006 19:59:34 +0100 Subject: [Python-Dev] Dicts are broken ... Message-ID: Michael Hudson wrote: > > I'd say it's more to do with __eq__. It's a strange __eq__ method > that raises an Exception, IMHO. Not entirely. Any type that supports invalid values (e.g. IEEE 754) and is safe against losing the invalid state by accident needs to raise an exception on A == B. IEEE 754 is not safe. > Please do realize that the motivation for this change was hours and > hours of tortous debugging caused by a buggy __eq__ method making keys > "impossibly" seem to not be in dictionaries. Quite. Been there - been caught by that. It is a catastrophic (but very common) misdesign to conflate failure and the answer "no". There is a fundamental flaw of that nature in card-based banking, that I pointed out was insoluble to the Grid people, and then got caught by just a month later! Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1 at cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679 From rasky at develer.com Fri Aug 4 21:51:09 2006 From: rasky at develer.com (Giovanni Bajo) Date: Fri, 4 Aug 2006 21:51:09 +0200 Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com> <2m3bccwopj.fsf@starship.python.net> <44D3928A.1090700@gafol.net> Message-ID: <176301c6b7ff$554d9890$8cb22997@bagio> Paul Colomiets wrote: > Well it's not recomended to mix strings and unicode in the > dictionaries > but if we mix for example integer and float we have the same thing. It > doesn't raise exception but still it is not expected behavior for me: > >>> d = { 1.0: 10, 2.0: 20 } > then if i somewhere later do: > >>> d[1] = 100 > >>> d[2] = 200 > to have here all floats in d.keys(). May be this is not a best > example. There is a strong difference. Python is moving towards unifying number types in a way (see the true division issue): the idea is that, all in all, user shouldn't really care what type a number is, as long as he knows it's a number. On the other hand, unicode and str are going to diverge more and more. Giovanni Bajo From bob at redivi.com Fri Aug 4 22:23:20 2006 From: bob at redivi.com (Bob Ippolito) Date: Fri, 4 Aug 2006 13:23:20 -0700 Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys In-Reply-To: <176301c6b7ff$554d9890$8cb22997@bagio> References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com> <2m3bccwopj.fsf@starship.python.net> <44D3928A.1090700@gafol.net> <176301c6b7ff$554d9890$8cb22997@bagio> Message-ID: On Aug 4, 2006, at 12:51 PM, Giovanni Bajo wrote: > Paul Colomiets wrote: > >> Well it's not recomended to mix strings and unicode in the >> dictionaries >> but if we mix for example integer and float we have the same >> thing. It >> doesn't raise exception but still it is not expected behavior for me: >>>>> d = { 1.0: 10, 2.0: 20 } >> then if i somewhere later do: >>>>> d[1] = 100 >>>>> d[2] = 200 >> to have here all floats in d.keys(). May be this is not a best >> example. > > There is a strong difference. Python is moving towards unifying > number types in > a way (see the true division issue): the idea is that, all in all, > user > shouldn't really care what type a number is, as long as he knows > it's a number. > On the other hand, unicode and str are going to diverge more and more. Well, not really. True division makes int/int return float instead of an int. You really do have to care if you have an int or a float most of the time, they're very different semantically. Unicode and str are eventually going to be the same thing (str would ideally end up becoming a synonym of unicode). The difference being that there will be some other type to contain bytes. -bob From pc at gafol.net Fri Aug 4 22:32:42 2006 From: pc at gafol.net (Paul Colomiets) Date: Fri, 04 Aug 2006 23:32:42 +0300 Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys In-Reply-To: <176301c6b7ff$554d9890$8cb22997@bagio> References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com> <2m3bccwopj.fsf@starship.python.net> <44D3928A.1090700@gafol.net> <176301c6b7ff$554d9890$8cb22997@bagio> Message-ID: <44D3AEEA.3080201@gafol.net> Giovanni Bajo wrote: > Paul Colomiets wrote: > > >> Well it's not recomended to mix strings and unicode in the >> dictionaries >> but if we mix for example integer and float we have the same thing. It >> doesn't raise exception but still it is not expected behavior for me: >> >>> d = { 1.0: 10, 2.0: 20 } >> then if i somewhere later do: >> >>> d[1] = 100 >> >>> d[2] = 200 >> to have here all floats in d.keys(). May be this is not a best >> example. >> > > There is a strong difference. Python is moving towards unifying number types in > a way (see the true division issue): the idea is that, all in all, user > shouldn't really care what type a number is, as long as he knows it's a number. > On the other hand, unicode and str are going to diverge more and more. > > Giovanni Bajo > > It makes sense, but consider this example: >>> from decimal import Decimal >>> d = {} >>> d[Decimal(0)] = 1 >>> d[0] = 2 >>> d[Decimal("0.5")] = 3 >>> d[0.5] = 4 >>> d.keys() [Decimal("0"), 0.5, Decimal("0.5")] I expect d.keys() to have 2 or 4 keys but don't 3, it's confusing. Don't you think that someday line "d[0.5] = 4" will raise exception? Or at least that it should raise if mixing str and unicode raises? -- Regards, Paul. From david.ascher at gmail.com Fri Aug 4 23:20:14 2006 From: david.ascher at gmail.com (David Ascher) Date: Fri, 4 Aug 2006 14:20:14 -0700 Subject: [Python-Dev] More tracker demos online In-Reply-To: <200607261426.29143.richardjones@optushome.com.au> References: <44C6748C.4060701@v.loewis.de> <200607261426.29143.richardjones@optushome.com.au> Message-ID: > > ""Martin v. L?wis"" wrote in message > > news:44C6748C.4060701 at v.loewis.de... > > > > > Currently, we have two running tracker demos online: > > > > > > Roundup: > > > http://efod.se/python-tracker/ > > > > > > Jira: > > > http://jira.python.atlassian.com/secure/Dashboard.jspa Is anyone looking at the Google Code Hosting tracker, just for fun? =) ( code.google.com/hosting, although performance seems to be an issue for now) --david -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060804/2c15f38c/attachment.html From rrr at ronadam.com Sat Aug 5 00:26:02 2006 From: rrr at ronadam.com (Ron Adam) Date: Fri, 04 Aug 2006 17:26:02 -0500 Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys In-Reply-To: <44D36B36.8030202@egenix.com> References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com> <2m3bccwopj.fsf@starship.python.net> <44D36B36.8030202@egenix.com> Message-ID: <44D3C97A.8000209@ronadam.com> M.-A. Lemburg wrote: > Terry Reedy wrote: >> "Michael Hudson" wrote in message >> news:2m3bccwopj.fsf at starship.python.net... >>> Michael Chermside writes: >>> >>>> I'm changing the subject line because I want to convince everyone that >>>> the problem being discussed in the "unicode hell" thread has nothing >>>> to do with unicode and strings. It's all about dicts. >>> I'd say it's more to do with __eq__. It's a strange __eq__ method >>> that raises an Exception, IMHO. >> I agree; a == b should always work, certainly unless explicitly programmed >> otherwise in Python for a particular class. > > ... which this is. > >> So I think the proper solution >> is fix the buggy __eq__ method to return False instead. If a byte string >> does not have a default (ascii) text interpretation, then it obviously is >> not equal to any particular unicode text. >> >> The fundamental axiom of sets and hence of dict keys is that any >> object/value either is or is not a member (at any given time for 'mutable' >> set collections). This requires that testing an object for possible >> membership by equality give a clean True or False answer. >> >>> Please do realize that the motivation for this change was hours and >>> hours of tortous debugging caused by a buggy __eq__ method making keys >>> "impossibly" seem to not be in dictionaries. >> So why not fix the buggy __eq__ method? > > Because it's not buggy. > > Python just doesn't know the encoding of the 8-bit string, so can't > make any assumptions on it. As result, it raises an exception to inform > the programmer. > > It is well possible that the string uses an encoding where the > Unicode string is indeed the equal to the string, assuming this > encoding, e.g. Isn't this a case where it should be up to the programmer to make sure the comparison makes sense in the context it is being used. That is, if I'm comparing two different forms of data, it's up to me to convert them explicitly to the same form before comparing them? In the case of comparing an 8 bit string and unicode, I would think they are always unequal. But changing that now would probably (?) break way too much. (but it may also uncover quite a few potential or even real bugs as well.) ;-) Cheers, Ron From jcarlson at uci.edu Sat Aug 5 00:46:21 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Fri, 04 Aug 2006 15:46:21 -0700 Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys In-Reply-To: <20060804182448.1717.1264031474.divmod.quotient.14372@ohm> References: <20060804112104.E6CD.JCARLSON@uci.edu> <20060804182448.1717.1264031474.divmod.quotient.14372@ohm> Message-ID: <20060804153937.E6E3.JCARLSON@uci.edu> Jean-Paul Calderone wrote: > On Fri, 04 Aug 2006 11:23:10 -0700, Josiah Carlson wrote: > >There's one problem with generating a warning for 2.5, and that is the > >same problem as generating a warning for possible packages that lack an > >__init__.py; users may start to get a bunch of warnings, and be unaware > >of how to suppress them. > > > >All in all though, I'm +0 on the warning, and +1 on it not raising an > >exception in 2.5 . > > Um. This warning would indicate a bug in the code which will lead to > actual misbehavior in a future version of Python. The __init__.py > warning would have indicated a deployment configuration which didn't > actually cause any misbehavior. > > They aren't the same case at all, unless you think that all warnings > should be classed this way (a position I do not think is completely > unreasonable, but since you singled out the package warning by way of > comparison, I assume this is not the argument you are trying to make). I see both as being a potential cause for a large number of warning messages to people starting to use Python 2.5 (from 2.3 or 2.4) . - Josiah From brett at python.org Sat Aug 5 02:11:29 2006 From: brett at python.org (Brett Cannon) Date: Fri, 4 Aug 2006 17:11:29 -0700 Subject: [Python-Dev] need an SSH key removed Message-ID: My bcannon at google.com SSH key should be removed since my internship is now over. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060804/5d1db915/attachment.html From tim.peters at gmail.com Sat Aug 5 05:07:14 2006 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 4 Aug 2006 23:07:14 -0400 Subject: [Python-Dev] need an SSH key removed In-Reply-To: References: Message-ID: <1f7befae0608042007t3f47c7ecrd96eb76e028cc727@mail.gmail.com> [Brett Cannon] > My bcannon at google.com SSH key should be removed since my internship is now > over. Thank you for being conscientious. While it feared death, your key didn't complain about being deleted, and right before it vanished I saw the most astonishing look of profound peace passing over its bits. You should rest assured tonight that yYv6r9zcrb and ZORyPRLc5Sf and all the other little key segments have gone on to a better repository. From aahz at pythoncraft.com Sat Aug 5 06:32:44 2006 From: aahz at pythoncraft.com (Aahz) Date: Fri, 4 Aug 2006 21:32:44 -0700 Subject: [Python-Dev] need an SSH key removed In-Reply-To: <1f7befae0608042007t3f47c7ecrd96eb76e028cc727@mail.gmail.com> References: <1f7befae0608042007t3f47c7ecrd96eb76e028cc727@mail.gmail.com> Message-ID: <20060805043244.GA23050@panix.com> On Fri, Aug 04, 2006, Tim Peters wrote: > [Brett Cannon] >> >> My bcannon at google.com SSH key should be removed since my internship is now >> over. > > Thank you for being conscientious. While it feared death, your key > didn't complain about being deleted, and right before it vanished I > saw the most astonishing look of profound peace passing over its bits. > You should rest assured tonight that yYv6r9zcrb and ZORyPRLc5Sf and > all the other little key segments have gone on to a better repository. That means it's now an ex-key? -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ C'mon, you knew *someone* was gonna say that From rasky at develer.com Sat Aug 5 13:17:18 2006 From: rasky at develer.com (Giovanni Bajo) Date: Sat, 5 Aug 2006 13:17:18 +0200 Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str andunicode asdictionarykeys References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com> <2m3bccwopj.fsf@starship.python.net><44D3928A.1090700@gafol.net> <176301c6b7ff$554d9890$8cb22997@bagio> Message-ID: <1c0801c6b880$b6f804f0$8cb22997@bagio> Bob Ippolito wrote: >>> Well it's not recomended to mix strings and unicode in the >>> dictionaries >>> but if we mix for example integer and float we have the same >>> thing. It >>> doesn't raise exception but still it is not expected behavior for >>> me: >>>>>> d = { 1.0: 10, 2.0: 20 } >>> then if i somewhere later do: >>>>>> d[1] = 100 >>>>>> d[2] = 200 >>> to have here all floats in d.keys(). May be this is not a best >>> example. >> >> There is a strong difference. Python is moving towards unifying >> number types in >> a way (see the true division issue): the idea is that, all in all, >> user >> shouldn't really care what type a number is, as long as he knows >> it's a number. >> On the other hand, unicode and str are going to diverge more and >> more. > > Well, not really. True division makes int/int return float instead of > an int. You really do have to care if you have an int or a float most > of the time, they're very different semantically. Then I'd ask why Python goes through hoops to make sure that hash(1.0) == hash(1), in the first place. Giovanni Bajo From hfoffani at gmail.com Sat Aug 5 13:52:19 2006 From: hfoffani at gmail.com (Hernan M Foffani) Date: Sat, 5 Aug 2006 13:52:19 +0200 Subject: [Python-Dev] More tracker demos online Message-ID: <11fab4bc0608050452l458a2073n250134484cf303e0@mail.gmail.com> > > > > Currently, we have two running tracker demos online: > > > > > > > > Roundup: > > > > http://efod.se/python-tracker/ > > > > > > > > Jira: > > > > http://jira.python.atlassian.com/secure/Dashboard.jspa > > > Is anyone looking at the Google Code Hosting tracker, just for fun? =) ( > code.google.com/hosting, although performance seems to be an issue for now) It's propietary code, isn't it? http://code.google.com/hosting/faq.html#itselfoss (Is that why you said "just for fun"?) Regards, -H. From bob at redivi.com Sun Aug 6 00:11:52 2006 From: bob at redivi.com (Bob Ippolito) Date: Sat, 5 Aug 2006 15:11:52 -0700 Subject: [Python-Dev] More tracker demos online In-Reply-To: <11fab4bc0608050452l458a2073n250134484cf303e0@mail.gmail.com> References: <11fab4bc0608050452l458a2073n250134484cf303e0@mail.gmail.com> Message-ID: <85FDF5FA-853E-4000-8E95-0F46B56E6910@redivi.com> On Aug 5, 2006, at 4:52 AM, Hernan M Foffani wrote: >>>>> Currently, we have two running tracker demos online: >>>>> >>>>> Roundup: >>>>> http://efod.se/python-tracker/ >>>>> >>>>> Jira: >>>>> http://jira.python.atlassian.com/secure/Dashboard.jspa >> >> >> Is anyone looking at the Google Code Hosting tracker, just for >> fun? =) ( >> code.google.com/hosting, although performance seems to be an issue >> for now) > > It's propietary code, isn't it? > http://code.google.com/hosting/faq.html#itselfoss > (Is that why you said "just for fun"?) So is Jira... -bob From l.oluyede at gmail.com Sun Aug 6 10:41:39 2006 From: l.oluyede at gmail.com (Lawrence Oluyede) Date: Sun, 6 Aug 2006 10:41:39 +0200 Subject: [Python-Dev] BZ2File.writelines should raise more meaningful exceptions Message-ID: <9eebf5740608060141k336e036n978e7a4f9f416ce4@mail.gmail.com> In the BZ2File object of bz2 module the writelines() method does not check its closed state before doing the actual work so its behavior it's different from write()'s behavior. See: >>> from bz2 import BZ2File >>> f = BZ2File("foo", "w") >>> f.close() >>> f.closed 1 >>> f.write("foobar") Traceback (most recent call last): File "", line 1, in ValueError: I/O operation on closed file >>> f.closed 1 >>> f.writelines(["foobar"]) Traceback (most recent call last): File "", line 1, in IOError: unknown IO error It also doesn't check if it can write for real: >>> from bz2 import BZ2File >>> f = BZ2File("foo", "r") >>> f.write("foobar") Traceback (most recent call last): File "", line 1, in IOError: file is not ready for writing >>> f.writelines(['foobar']) Traceback (most recent call last): File "", line 1, in RuntimeError: wrong sequence of bz2 library commands used The patch is attached. If you think it's ok to fix this I'll post it to the bug tracker -- Lawrence http://www.oluyede.org/blog -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: bz2module.diff.txt Url: http://mail.python.org/pipermail/python-dev/attachments/20060806/bd63dc42/attachment.txt From andymac at bullseye.apana.org.au Sun Aug 6 14:09:09 2006 From: andymac at bullseye.apana.org.au (Andrew MacIntyre) Date: Sun, 06 Aug 2006 23:09:09 +1100 Subject: [Python-Dev] test_mailbox on Cygwin Message-ID: <44D5DBE5.4030304@bullseye.apana.org.au> I just had a look at the Cygwin buildbot's test log, and noticed that test_mailbox has failures that look very similar to those I addressed in rev 50789 for the OS/2 EMX platform. Hopefully this might give someone with access to Cygwin a starting point in getting this test working on that platform. -- ------------------------------------------------------------------------- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac at bullseye.apana.org.au (pref) | Snail: PO Box 370 andymac at pcug.org.au (alt) | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From aahz at pythoncraft.com Sun Aug 6 16:20:24 2006 From: aahz at pythoncraft.com (Aahz) Date: Sun, 6 Aug 2006 07:20:24 -0700 Subject: [Python-Dev] BZ2File.writelines should raise more meaningful exceptions In-Reply-To: <9eebf5740608060141k336e036n978e7a4f9f416ce4@mail.gmail.com> References: <9eebf5740608060141k336e036n978e7a4f9f416ce4@mail.gmail.com> Message-ID: <20060806142024.GA25227@panix.com> On Sun, Aug 06, 2006, Lawrence Oluyede wrote: > > The patch is attached. If you think it's ok to fix this I'll post it > to the bug tracker Always post patches -- that way they can't get lost. *THEN* post to python-dev with your analysis and explanation (which you presumably also included with the patch), starting with a link to the patch. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." --Brian W. Kernighan From l.oluyede at gmail.com Sun Aug 6 21:33:12 2006 From: l.oluyede at gmail.com (Lawrence Oluyede) Date: Sun, 6 Aug 2006 21:33:12 +0200 Subject: [Python-Dev] BZ2File.writelines should raise more meaningful exceptions In-Reply-To: <20060806142024.GA25227@panix.com> References: <9eebf5740608060141k336e036n978e7a4f9f416ce4@mail.gmail.com> <20060806142024.GA25227@panix.com> Message-ID: <9eebf5740608061233g79926169m41a94d011eb29db7@mail.gmail.com> > Always post patches -- that way they can't get lost. *THEN* post to > python-dev with your analysis and explanation (which you presumably also > included with the patch), starting with a link to the patch. Thanks for the hint. This is the link: http://sourceforge.net/tracker/index.php?func=detail&aid=1535500&group_id=5470&atid=305470 -- Lawrence http://www.oluyede.org/blog From greg at electricrain.com Sun Aug 6 21:41:35 2006 From: greg at electricrain.com (Gregory P. Smith) Date: Sun, 6 Aug 2006 12:41:35 -0700 Subject: [Python-Dev] windows 2.5 build: use OpenSSL for hashlib [bug 1535502] Message-ID: <20060806194135.GC9899@zot.electricrain.com> Whoever knows how the windows build process works and controls the python 2.5 windows release builds could you please make sure the hashlib module gets built + linked with OpenSSL rather than falling back to its much slower builtin implementations. http://sourceforge.net/tracker/index.php?func=detail&aid=1535502&group_id=5470&atid=105470 thanks, greg From brett at python.org Sun Aug 6 23:39:17 2006 From: brett at python.org (Brett Cannon) Date: Sun, 6 Aug 2006 14:39:17 -0700 Subject: [Python-Dev] More tracker demos online In-Reply-To: <11fab4bc0608050452l458a2073n250134484cf303e0@mail.gmail.com> References: <11fab4bc0608050452l458a2073n250134484cf303e0@mail.gmail.com> Message-ID: On 8/5/06, Hernan M Foffani wrote: > > > > > > Currently, we have two running tracker demos online: > > > > > > > > > > Roundup: > > > > > http://efod.se/python-tracker/ > > > > > > > > > > Jira: > > > > > http://jira.python.atlassian.com/secure/Dashboard.jspa > > > > > > Is anyone looking at the Google Code Hosting tracker, just for fun? > =) ( > > code.google.com/hosting, although performance seems to be an issue for > now) > > It's propietary code, isn't it? > http://code.google.com/hosting/faq.html#itselfoss > (Is that why you said "just for fun"?) Yes, it's proprietary, but so what? As Bob pointed out, JIRA is as well. Read the call for trackers and you will see we explicitly said closed-source was fine. But Google Code Hosting is not being considered since it won't have data imported by the deadline for accepting test trackers (it currently doesn't have any support for mass issue uploading). -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060806/69973255/attachment.htm From tim.peters at gmail.com Mon Aug 7 06:00:37 2006 From: tim.peters at gmail.com (Tim Peters) Date: Mon, 7 Aug 2006 00:00:37 -0400 Subject: [Python-Dev] cgi.FieldStorage DOS (sf bug #1112549) In-Reply-To: References: Message-ID: <1f7befae0608062100y601e18dtaad2490e50228acc@mail.gmail.com> [Chris McDonough, on 7/28/06] > From the initial bugreport > (http://sourceforge.net/tracker/index.php? > func=detail&aid=1112549&group_id=5470&atid=105470) > > """ > Various parts of cgi.FieldStorage call its > "read_lines_to_outerboundary", "read_lines" and > "skip_lines" methods. These methods use the > "readline" method of the file object that represents an > input stream. The input stream is typically data > supplied by an untrusted source (such as a user > uploading a file from a web browser). The input data > is not required by the RFC 822/1521/1522/1867 > specifications to contain any newline characters. For > example, it is within the bounds of the specification > to supply a a multipart/form-data input stream with a > "file-data" part that consists of a 2GB string composed > entirely of "x" characters (which happens to be > something I did that led me to noticing this bug). > """ > > This bug has been around for about a year but I just worked up a > patch yesterday that applies OK against current SVN. It's attached > to the issue. Would someone be so kind as to check it in? Guido has > already reviewed it, I believe. Are either of our 2.5 release managers/coordinators on the internal Python security mailing list? I am, but only the list admin can see who's on that list. Since this bug is thought to be a security hole (the "DOS" in the subject line doesn't refer to your favorite operating system -- it's the Denial-Of-Service flavor of DOS), it's important that someone with sufficient power stare at this one and Pronounce on its fate for 2.5c1. Here's a clicky thing: http://www.python.org/sf/1112549 From robinbryce at gmail.com Mon Aug 7 13:01:31 2006 From: robinbryce at gmail.com (Robin Bryce) Date: Mon, 7 Aug 2006 12:01:31 +0100 Subject: [Python-Dev] 2.5b3, commit r46372 regressed PEP 302 machinery (sf not letting me post) Message-ID: Hi, Appologies for the lack of an sf#. I tried to submit this there but couldn't. (sf is logging me out each time I visit a new page and it is refusing my attempt to post anonymously). Python 2.5b3 (trunk:51136M, Aug 7 2006, 10:48:15) [GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2 The need for speed patch commited in revision r46372 included a change whose intent was to reduce the number of open calls. The `continue` statement at line 1283 in import.c:r51136 has the effect of skipping the builtin import machinery if the find_module method of a custom importer returns None. In Python 2.4.3, if find_module returned None the builtin machinery is allowed to process the path tail. In my particular case I am working on an importer that deals with kid templates that may or may not exist as .py[c] files. The short of it is that in Python 2.4.3 this produces a usable module ``__import__('foo.a/templateuri')`` wheras in 2.5b3 I get import error. The python 2.4.3 implementation *allows* module paths that are not seperated with '.' Python 2.5b3 does not allow this and it does not look like this was an intentional change. I believe this point about 'illeagal' module paths is actualy independent of the regresion I am asserting. Detailed session logs are attatched (following the sf guidance even though I'm posting to py-dev) The 'use case' for the importer is: Robin wants to package a default template file as normal python module and provide a custom importer that allows users of his package to reference both: there own templates and html files on the file system in arbitrary locations AND the stock templates provided as python modules under the same name space. He would like to leave normal imports to the standard machinery. Cheers, Robin -------------- next part -------------- A non-text attachment was scrubbed... Name: bugreport.rst Type: application/octet-stream Size: 2594 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20060807/c1a25e6d/attachment.obj From martin at v.loewis.de Mon Aug 7 14:46:45 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 07 Aug 2006 14:46:45 +0200 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <44D31E64.3070407@egenix.com> References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com> <44D2F653.5090305@egenix.com> <44D3140A.2040600@canterbury.ac.nz> <44D31E64.3070407@egenix.com> Message-ID: <44D73635.4000003@v.loewis.de> M.-A. Lemburg schrieb: >> There's no disputing that an exception should be raised >> if the string *must* be interpretable as characters in >> order to continue. But that's not true here if you allow >> for the interpretation that they're simply objects of >> different (duck) type and therefore unequal. > > Hmm, given that interpretation, 1 == 1.0 would have to be > False. No, but 1 == 1.5 would have to be False (and actually is). In that analogy, int relates to float as ascii-bytes to Unicode: some values are shared between int and float (e.g. 1 and 1.0), other values are not shared (e.g. 1.5 has no equivalent in int). An int equals a float only if both values originate from the shared subset. Now, int is a (nearly) true subset of float, so there are no ints with no float equivalent (actually, there are, but Python ignores that). > Note that you do have to interpret the string as characters > if you compare it to Unicode and there's nothing wrong with > that. Consider this: py> int(3+4j) Traceback (most recent call last): File "", line 1, in ? TypeError: can't convert complex to int; use int(abs(z)) py> 3 == 3+4j False So even though the conversion raises an exception, the values are determined to be not equal. Again, because int is a nearly true subset of complex, the conversion goes the other way, but *if* it would use the complex->int conversion, then the TypeError should be taken as a guarantee that the objects don't compare equal. Expanding this view to Unicode should mean that a unicode string U equals a byte string B if U.encode(system_encode) == B or B.decode(system_encoding) == U, and that they don't equal otherwise (e.g. if the conversion fails with a "not convertible" exception). Which of the two conversions is selected is arbitrary; we should, of course, continue to use the one we always used (for "ascii", there is no difference between the two). Regards, Martin From martin at v.loewis.de Mon Aug 7 15:00:20 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 07 Aug 2006 15:00:20 +0200 Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys In-Reply-To: <44D36B36.8030202@egenix.com> References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com> <2m3bccwopj.fsf@starship.python.net> <44D36B36.8030202@egenix.com> Message-ID: <44D73964.3050402@v.loewis.de> M.-A. Lemburg schrieb: > Python just doesn't know the encoding of the 8-bit string, so can't > make any assumptions on it. As result, it raises an exception to inform > the programmer. Oh, Python does make an assumption what the encoding is: it assumes it is the system encoding (i.e. "ascii"). Then invoking the ascii codec raises an exception, because the string clearly isn't ascii. > It is well possible that the string uses an encoding where the > Unicode string is indeed the equal to the string, assuming this > encoding So what? Python uses the system encoding for this operation. What does it matter that the result would be different if it had used a different encoding. The strings are unequal under the system encoding; it's irrelevant that they might be equal under a different encoding. The same holds for the ASCII part (i.e. where you don't get an exception): py> u"foo" == "sbb" False py> u"foo".encode("rot13") == "sbb" True So the strings compare as unequal, even though they compare equal if treated as rot13. That doesn't stop Python from considering them unequal. Regards, Martin From martin at v.loewis.de Mon Aug 7 15:04:24 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 07 Aug 2006 15:04:24 +0200 Subject: [Python-Dev] Python 2.5b3 and AIX 4.3 - It Works In-Reply-To: References: Message-ID: <44D73A58.2040502@v.loewis.de> Michael Kent schrieb: > Because of a requirement to remain compatible with AIX 4.3, I have been forced > to stay with Python 2.3, because while 2.4 would compile under AIX 4.3, it would > segfault immediately when run. > > I'm happy to report that Python 2.5b3 compiles and runs fine under AIX 4.3, and > passes most of its test suite. However, here are a few test failures. I > realize AIX 4.3 is long obsolete, so is there any interest on the list for me to > report which tests failed and how? As Neal says: There would be interest in receiving patches. Interest in receiving bug reports for systems we don't have access to is low. Regards, Martin From martin at v.loewis.de Mon Aug 7 15:16:22 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 07 Aug 2006 15:16:22 +0200 Subject: [Python-Dev] windows 2.5 build: use OpenSSL for hashlib [bug 1535502] In-Reply-To: <20060806194135.GC9899@zot.electricrain.com> References: <20060806194135.GC9899@zot.electricrain.com> Message-ID: <44D73D26.4050803@v.loewis.de> Gregory P. Smith schrieb: > Whoever knows how the windows build process works and controls the > python 2.5 windows release builds could you please make sure the > hashlib module gets built + linked with OpenSSL rather than falling > back to its much slower builtin implementations. If the project files are changed in that direction, then my build will pick that up automatically. I can't promise to change the files myself. I'm somewhat worried about yet another size increase in pythonxy.dll (actually, I'm personally not worried, but I anticipate complaints about such a change). What should happen to the existing sha* modules? I believe that the performance of the OpenSSL routines depends on the way OpenSSL was built, e.g. whether the assembler implementations are used or not. Somebody would have to check, but I doubt they are. So in short: I'm very doubtful that such a change can still be made, and if it is, that it will be "right". I'm accepting patches regardless. Regards, Martin From fuzzyman at voidspace.org.uk Mon Aug 7 15:22:24 2006 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 07 Aug 2006 14:22:24 +0100 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <44D73635.4000003@v.loewis.de> References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com> <44D2F653.5090305@egenix.com> <44D3140A.2040600@canterbury.ac.nz> <44D31E64.3070407@egenix.com> <44D73635.4000003@v.loewis.de> Message-ID: <44D73E90.6070707@voidspace.org.uk> Martin v. L?wis wrote: > [snip..] > Expanding this view to Unicode should mean that a unicode > string U equals a byte string B if > U.encode(system_encode) == B or B.decode(system_encoding) == U, > and that they don't equal otherwise (e.g. if the conversion > fails with a "not convertible" exception). Which of the > two conversions is selected is arbitrary; we should, of > course, continue to use the one we always used (for > "ascii", there is no difference between the two). > > +1 This seems the most (only ?) logical solution. Michael Foord > Regards, > Martin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > > From tanzer at swing.co.at Mon Aug 7 16:26:03 2006 From: tanzer at swing.co.at (tanzer at swing.co.at) Date: Mon, 07 Aug 2006 16:26:03 +0200 Subject: [Python-Dev] 2.5 status In-Reply-To: Your message of "Thu, 03 Aug 2006 22:49:22 PDT." Message-ID: "Neal Norwitz" wrote: > Things are getting better, but we still have some really important > outstanding issues. PLEASE CONTINUE TESTING AS MUCH AS POSSIBLE. I've run into a problem with a big application that I wasn't able to reproduce with a small example. I've submitted a bug report to Sourceforge (Id 1536021)*. As Sourceforge seems to mangle the code I used to describe the problem, I'll include it here once more: The code (exception handler added to demonstrate and work around the problem): try : h = hash(p) except OverflowError, e: print type(p), p, id(p), e h = id(p) & 0x0FFFFFFF prints the following output: > 3066797028 long int too large to convert to int This happens with Python 2.5b3, but didn't happen with Python 2.4.3. I assume that the hash-function for function/methods returns the `id` of the function. The following code demonstrates the same problem with a Python class whose `__hash__` returns the `id` of the object: $ python2.4 Python 2.4.3 (#1, Jun 30 2006, 10:02:59) [GCC 3.4.6 (Gentoo 3.4.6-r1, ssp-3.4.5-1.0, pie-8.7.9)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> class X(object): ... def __hash__(self): return id(self) ... >>> hash (X()) -1211078036 $ python2.5 Python 2.5b3 (r25b3:51041, Aug 7 2006, 15:35:35) [GCC 3.4.6 (Gentoo 3.4.6-r1, ssp-3.4.5-1.0, pie-8.7.9)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> class X(object): ... def __hash__(self): return id(self) ... >>> hash (X()) Traceback (most recent call last): File "", line 1, in OverflowError: long int too large to convert to int (*) http://sourceforge.net/tracker/index.php?func=detail&aid=1536021&group_id=5470&atid=105470 -- Christian Tanzer http://www.c-tanzer.at/ From martin at v.loewis.de Mon Aug 7 16:55:24 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 07 Aug 2006 16:55:24 +0200 Subject: [Python-Dev] 2.5 status In-Reply-To: References: Message-ID: <44D7545C.80205@v.loewis.de> tanzer at swing.co.at schrieb: > The code (exception handler added to demonstrate and work around > the problem): > > try : > h = hash(p) > except OverflowError, e: > print type(p), p, id(p), e > h = id(p) & 0x0FFFFFFF > > prints the following output: > > > > > 3066797028 long int too large to convert to int > > This happens with Python 2.5b3, but didn't happen with Python 2.4.3. > > I assume that the hash-function for function/methods returns the > `id` of the function. No (not really). Instead, it combines the hash of the target object with the address of the function object. The hash function of the method object, in itself, cannot raise this overflow error. However, it involves hash(p.im_self). So if Script_Category.__hash__ is implemented as you show below, this error might occur. > >>> class X(object): > ... def __hash__(self): return id(self) > ... > >>> hash (X()) > Traceback (most recent call last): > File "", line 1, in > OverflowError: long int too large to convert to int Yes, this comes from id() now always returning positive integers, which might be a long if the object pointer is > MAXINT I think both instance_hash and slot_tp_hash should be changed to just truncate long ints to the range LONG_MIN..LONG_MAX Notice that this error could have occurred already in 2.4, on a 64-bit system where sizeof(void*) > sizeof(long) (i.e. on Win64). Regards, Martin From david.nospam.hopwood at blueyonder.co.uk Mon Aug 7 16:57:15 2006 From: david.nospam.hopwood at blueyonder.co.uk (David Hopwood) Date: Mon, 07 Aug 2006 15:57:15 +0100 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <44D73E90.6070707@voidspace.org.uk> References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com> <44D2F653.5090305@egenix.com> <44D3140A.2040600@canterbury.ac.nz> <44D31E64.3070407@egenix.com> <44D73635.4000003@v.loewis.de> <44D73E90.6070707@voidspace.org.uk> Message-ID: <44D754CB.8070607@blueyonder.co.uk> Michael Foord wrote: > Martin v. L?wis wrote: > >>[snip..] >>Expanding this view to Unicode should mean that a unicode >>string U equals a byte string B if >>U.encode(system_encode) == B or B.decode(system_encoding) == U, >>and that they don't equal otherwise (e.g. if the conversion >>fails with a "not convertible" exception). I disagree. Unicode strings should always be considered distinct from non-ASCII byte strings. Implicitly encoding or decoding in order to perform a comparison is a bad idea; it is expensive and will often do the wrong thing. The programmer should explicitly encode the Unicode string or decode the byte string before comparison (which one of these is correct is application-dependent). >>Which of the two conversions is selected is arbitrary; [...] It would not be arbitrary. In the common case where the byte encoding uses "precomposed" characters, using "U.encode(system_encoding) == B" will tend to succeed in more cases than "B.decode(system_encoding) == U", because alternative representations of the same abstract character in Unicode will be mapped to the same precomposed character. (Whether these are cases in which the comparison *should* succeed is, as I said above, application-dependent.) The special case of considering US-ASCII strings to compare equal to the corresponding Unicode string, is more reasonable than this would be for a general byte encoding, because: - it can be done with no (or only a trivial) conversion, - US-ASCII has no precomposed characters or combining marks, so it does not have multiple encodings for the same abstract character, - Unicode has a US-ASCII subset that uses exactly the same encoding model as US-ASCII (whereas in general, a byte encoding might use an arbitrarily different encoding model to Unicode, as for example is the case for ISCII). >>we should, of course, continue to use the one we always used (for >>"ascii", there is no difference between the two). > > +1 > > This seems the most (only ?) logical solution. No; always considering Unicode and non-ASCII byte strings to be distinct is just as logical. -- David Hopwood From ncoghlan at gmail.com Mon Aug 7 17:05:52 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 08 Aug 2006 01:05:52 +1000 Subject: [Python-Dev] 2.5 status In-Reply-To: <44D7545C.80205@v.loewis.de> References: <44D7545C.80205@v.loewis.de> Message-ID: <44D756D0.3030703@gmail.com> Martin v. L?wis wrote: > tanzer at swing.co.at schrieb: >> >>> class X(object): >> ... def __hash__(self): return id(self) >> ... >> >>> hash (X()) >> Traceback (most recent call last): >> File "", line 1, in >> OverflowError: long int too large to convert to int > > Yes, this comes from id() now always returning positive integers, > which might be a long if the object pointer is > MAXINT > > I think both instance_hash and slot_tp_hash should be changed > to just truncate long ints to the range LONG_MIN..LONG_MAX Couldn't they be changed to invoke long's own hash method when a long object is returned from __hash__? Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From martin at v.loewis.de Mon Aug 7 17:41:39 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 07 Aug 2006 17:41:39 +0200 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <44D754CB.8070607@blueyonder.co.uk> References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com> <44D2F653.5090305@egenix.com> <44D3140A.2040600@canterbury.ac.nz> <44D31E64.3070407@egenix.com> <44D73635.4000003@v.loewis.de> <44D73E90.6070707@voidspace.org.uk> <44D754CB.8070607@blueyonder.co.uk> Message-ID: <44D75F33.2020200@v.loewis.de> David Hopwood schrieb: > I disagree. Unicode strings should always be considered distinct from > non-ASCII byte strings. Implicitly encoding or decoding in order to > perform a comparison is a bad idea; it is expensive and will often do > the wrong thing. That's a pretty irrelevant position at this point; Python has had the notion of a system encoding since Unicode was introduced, and we are not going to remove that just before a release candidate of Python 2.5. The question at hand is not whether certain object should compare unequal, but whether comparing them should raise an exception. >>> Which of the two conversions is selected is arbitrary; [...] > > It would not be arbitrary. In the common case where the byte encoding > uses "precomposed" characters, using "U.encode(system_encoding) == B" > will tend to succeed in more cases than "B.decode(system_encoding) == U", > because alternative representations of the same abstract character in > Unicode will be mapped to the same precomposed character. No, they won't (although they should, perhaps): py> u'o\u0308'.encode("latin-1") Traceback (most recent call last): File "", line 1, in ? UnicodeEncodeError: 'latin-1' codec can't encode character u'\u0308' in position 1: ordinal not in range(256) In addition, it's also possible to find encodings (e.g. iso-2022) where different byte sequences decode to the same Unicode string. Regards, Martin From arigo at tunes.org Mon Aug 7 17:55:00 2006 From: arigo at tunes.org (Armin Rigo) Date: Mon, 7 Aug 2006 17:55:00 +0200 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <44D23807.4050604@egenix.com> References: <44D21A99.2080801@brainbot.com> <44D22989.7000202@egenix.com> <44D22FA7.7030905@brainbot.com> <44D234C3.4010404@egenix.com> <44D23807.4050604@egenix.com> Message-ID: <20060807155500.GA11910@code0.codespeak.net> Hi, On Thu, Aug 03, 2006 at 07:53:11PM +0200, M.-A. Lemburg wrote: > > I though I'd heard (from Guido here or on the py3k list) that it was only > > 1 < u'abc' that would raise an exception, and that 1 == u'abc' would still > > evaluate to False. Did I misunderstand? > > Could be that I'm wrong. I also seem to remember that TypeErrors should only signal ordering non-sense, not equality. In this case, I'm on the opinion that unicode objects and completely-unrelated strings of random bytes should successfully compare as unequal, but I'm not enough of a unicode user to be sure. A bientot, Armin. From greg at electricrain.com Mon Aug 7 18:50:27 2006 From: greg at electricrain.com (Gregory P. Smith) Date: Mon, 7 Aug 2006 09:50:27 -0700 Subject: [Python-Dev] windows 2.5 build: use OpenSSL for hashlib [bug 1535502] In-Reply-To: <44D73D26.4050803@v.loewis.de> References: <20060806194135.GC9899@zot.electricrain.com> <44D73D26.4050803@v.loewis.de> Message-ID: <20060807165027.GD9899@zot.electricrain.com> On Mon, Aug 07, 2006 at 03:16:22PM +0200, "Martin v. L?wis" wrote: > Gregory P. Smith schrieb: > > Whoever knows how the windows build process works and controls the > > python 2.5 windows release builds could you please make sure the > > hashlib module gets built + linked with OpenSSL rather than falling > > back to its much slower builtin implementations. > > If the project files are changed in that direction, then my build will > pick that up automatically. I can't promise to change the files myself. > > I'm somewhat worried about yet another size increase in pythonxy.dll > (actually, I'm personally not worried, but I anticipate complaints about > such a change). > > What should happen to the existing sha* modules? Actually, this change will either: (a) leave pythonxy.dll the same size. OR (b) *reduce* the size of pythonxy.dll. hashlib's OpenSSL implementation on windows comes in the form of a 300k _hashlib.pyd library. Build that and pythonxy.dll won't change. If you want to reduce the pythonxy.dll size you can remove _md5, _sha, _sha256 and _sha512 from the builtins list so long as _hashlib.pyd is built. There is OpenSSL library code duplication between the _hashlib (300k) and _ssl (650k) modules the way things are linked today (static). If you wanted absolute smallest distro size and code reuse we'd need to change things to use an OpenSSL.dll. I'm not proposing that (though it is a good idea). > I believe that the performance of the OpenSSL routines depends on > the way OpenSSL was built, e.g. whether the assembler implementations > are used or not. Somebody would have to check, but I doubt they are. That'd be unfortunate as that negatively impacts the socket _ssl module as well. OpenSSL should always be built with the assembler implementations. From fuzzyman at voidspace.org.uk Mon Aug 7 19:07:29 2006 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 07 Aug 2006 18:07:29 +0100 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <44D754CB.8070607@blueyonder.co.uk> References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com> <44D2F653.5090305@egenix.com> <44D3140A.2040600@canterbury.ac.nz> <44D31E64.3070407@egenix.com> <44D73635.4000003@v.loewis.de> <44D73E90.6070707@voidspace.org.uk> <44D754CB.8070607@blueyonder.co.uk> Message-ID: <44D77351.80603@voidspace.org.uk> David Hopwood wrote:[snip..] > > >>> we should, of course, continue to use the one we always used (for >>> "ascii", there is no difference between the two). >>> >> +1 >> >> This seems the most (only ?) logical solution. >> > > No; always considering Unicode and non-ASCII byte strings to be distinct > is just as logical. > Except there has been an implicit promise in Python for years now that ascii byte-strings will compare equally to the unicode equivalent: lots of code assumes this. Breaking this is fine in principle - but for Py3K not Py 2.x. That means Martin's solution is the best for the current problem. (IMHO of course...) Michael http://www.voidspace.org.uk/python/index.shtml From anthony at interlink.com.au Mon Aug 7 19:25:46 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue, 8 Aug 2006 03:25:46 +1000 Subject: [Python-Dev] windows 2.5 build: use OpenSSL for hashlib [bug 1535502] In-Reply-To: <20060807165027.GD9899@zot.electricrain.com> References: <20060806194135.GC9899@zot.electricrain.com> <44D73D26.4050803@v.loewis.de> <20060807165027.GD9899@zot.electricrain.com> Message-ID: <200608080325.50041.anthony@interlink.com.au> I'm nervous about this change being made at this stage of the release process. It seems to me to have a chance of causing breakages - admittedly a small chance, but one that's higher than I'd like. I'd also like to make sure that the PCBuild8 directory is updated at the same time - with the recent disappearance of the previous free MS compiler version, I think this will become more important over the 18 months or so of Python 2.5's life. Anthony From greg at electricrain.com Mon Aug 7 20:21:23 2006 From: greg at electricrain.com (Gregory P. Smith) Date: Mon, 7 Aug 2006 11:21:23 -0700 Subject: [Python-Dev] windows 2.5 build: use OpenSSL for hashlib [bug 1535502] In-Reply-To: <200608080325.50041.anthony@interlink.com.au> References: <20060806194135.GC9899@zot.electricrain.com> <44D73D26.4050803@v.loewis.de> <20060807165027.GD9899@zot.electricrain.com> <200608080325.50041.anthony@interlink.com.au> Message-ID: <20060807182123.GE9899@zot.electricrain.com> On Tue, Aug 08, 2006 at 03:25:46AM +1000, Anthony Baxter wrote: > I'm nervous about this change being made at this stage of the release process. > It seems to me to have a chance of causing breakages - admittedly a small > chance, but one that's higher than I'd like. Sigh. Half the reason I did the hashlib work was to get much faster optimized versions of the hash algorithms into python. I'll be disappointed if that doesn't happen. hashlib passes its test suite with our without openssl. If I make the windows project file updates to simply build and include _hashlib.pyd in the windows installer what harm is that going to cause? IMHO the windows python 2.5 build as it is is missing a feature by not including this. > I'd also like to make sure that the PCBuild8 directory is updated at the same > time - with the recent disappearance of the previous free MS compiler > version, I think this will become more important over the 18 months or so of > Python 2.5's life. agreed. frustrated.. -greg From david.nospam.hopwood at blueyonder.co.uk Mon Aug 7 23:34:10 2006 From: david.nospam.hopwood at blueyonder.co.uk (David Hopwood) Date: Mon, 07 Aug 2006 22:34:10 +0100 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <44D77351.80603@voidspace.org.uk> References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com> <44D2F653.5090305@egenix.com> <44D3140A.2040600@canterbury.ac.nz> <44D31E64.3070407@egenix.com> <44D73635.4000003@v.loewis.de> <44D73E90.6070707@voidspace.org.uk> <44D754CB.8070607@blueyonder.co.uk> <44D77351.80603@voidspace.org.uk> Message-ID: <44D7B1D2.2000105@blueyonder.co.uk> Michael Foord wrote: > David Hopwood wrote:[snip..] > >>>> we should, of course, continue to use the one we always used (for >>>> "ascii", there is no difference between the two). >>> >>> +1 >>> >>> This seems the most (only ?) logical solution. >> >> No; always considering Unicode and non-ASCII byte strings to be distinct >> is just as logical. > > Except there has been an implicit promise in Python for years now that > ascii byte-strings will compare equally to the unicode equivalent: lots > of code assumes this. I think you must have misread my comment: No; always considering Unicode and *non-ASCII* byte strings to be distinct is just as logical. This says nothing about comparing Unicode and ASCII byte strings. -- David Hopwood From rrr at ronadam.com Mon Aug 7 23:37:42 2006 From: rrr at ronadam.com (Ron Adam) Date: Mon, 07 Aug 2006 16:37:42 -0500 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <44D77351.80603@voidspace.org.uk> References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com> <44D2F653.5090305@egenix.com> <44D3140A.2040600@canterbury.ac.nz> <44D31E64.3070407@egenix.com> <44D73635.4000003@v.loewis.de> <44D73E90.6070707@voidspace.org.uk> <44D754CB.8070607@blueyonder.co.uk> <44D77351.80603@voidspace.org.uk> Message-ID: <44D7B2A6.70108@ronadam.com> Michael Foord wrote: > David Hopwood wrote:[snip..] >> >>>> we should, of course, continue to use the one we always used (for >>>> "ascii", there is no difference between the two). >>>> >>> +1 >>> >>> This seems the most (only ?) logical solution. >>> >> No; always considering Unicode and non-ASCII byte strings to be distinct >> is just as logical. Yes, that's true. (But can't be done prior to P3k of course.) Consider the comparison of ... [3] == (3,) -> False These are not the same thing even though it may be trivial to treat them as being equivalent. So how smart should a equivalence comparison be? I think testing for interchangeability and/or taking into account context is going down a very difficult road. Which is what the string to Unicode comparison does by making an assumption that the string type is in the default encoding, which it may not be. Purity in this would insist that comparing floats and integers always return False, but there is little ambiguity when it comes to whether numerical values are equivalent or not. The rules for their comparisons are fairly well established. So numerical equivalence can be the exception when comparing values of differing types and its the expected behavior as well as the established practice in programming. > Except there has been an implicit promise in Python for years now that > ascii byte-strings will compare equally to the unicode equivalent: lots > of code assumes this. Breaking this is fine in principle - but for Py3K > not Py 2.x. Also True. And I hope that a bytes to Unicode comparison in Py3k will always returns False just like [3] == (3,) always returns False. > That means Martin's solution is the best for the current problem. (IMHO > of course...) I think (IMHO) in this particular case, maintaining "backwards compatibility" should take precedence (until Py3k) and be the stated reason for the continued behavior in the documents as well. And so Unicode to String comparisons should be the second exception to not doing data form conversions when comparing two objects. At least for pre-Py3k. Are there other cases where different types of objects compare equal? (Not including those where the user writes or overrides a method to get that functionality of course.) Cheers, Ron From martin at v.loewis.de Tue Aug 8 01:27:24 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 08 Aug 2006 01:27:24 +0200 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <44D7B1D2.2000105@blueyonder.co.uk> References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com> <44D2F653.5090305@egenix.com> <44D3140A.2040600@canterbury.ac.nz> <44D31E64.3070407@egenix.com> <44D73635.4000003@v.loewis.de> <44D73E90.6070707@voidspace.org.uk> <44D754CB.8070607@blueyonder.co.uk> <44D77351.80603@voidspace.org.uk> <44D7B1D2.2000105@blueyonder.co.uk> Message-ID: <44D7CC5C.5080508@v.loewis.de> David Hopwood schrieb: > Michael Foord wrote: >> David Hopwood wrote:[snip..] >> >>>>> we should, of course, continue to use the one we always used (for >>>>> "ascii", there is no difference between the two). >>>> +1 >>>> >>>> This seems the most (only ?) logical solution. >>> No; always considering Unicode and non-ASCII byte strings to be distinct >>> is just as logical. > > I think you must have misread my comment: Indeed. The misunderstanding originates from your sentence starting with "no", when, in fact, you seem to be supporting the proposal I made. Regards, Martin From martin at v.loewis.de Tue Aug 8 01:46:13 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 08 Aug 2006 01:46:13 +0200 Subject: [Python-Dev] windows 2.5 build: use OpenSSL for hashlib [bug 1535502] In-Reply-To: <20060807165027.GD9899@zot.electricrain.com> References: <20060806194135.GC9899@zot.electricrain.com> <44D73D26.4050803@v.loewis.de> <20060807165027.GD9899@zot.electricrain.com> Message-ID: <44D7D0C5.2050204@v.loewis.de> Gregory P. Smith schrieb: > hashlib's OpenSSL implementation on windows comes in the form of a > 300k _hashlib.pyd library. What do you mean by "comes"? I can't find any _hashlib.vcproj file inside the PCbuild directory. >> I believe that the performance of the OpenSSL routines depends on >> the way OpenSSL was built, e.g. whether the assembler implementations >> are used or not. Somebody would have to check, but I doubt they are. > > That'd be unfortunate as that negatively impacts the socket _ssl > module as well. OpenSSL should always be built with the assembler > implementations. But Visual Studio doesn't ship with an assembler! So how could I build it? Regards, Martin From greg at electricrain.com Tue Aug 8 02:21:07 2006 From: greg at electricrain.com (Gregory P. Smith) Date: Mon, 7 Aug 2006 17:21:07 -0700 Subject: [Python-Dev] windows 2.5 build: use OpenSSL for hashlib [bug 1535502] In-Reply-To: <44D7D0C5.2050204@v.loewis.de> References: <20060806194135.GC9899@zot.electricrain.com> <44D73D26.4050803@v.loewis.de> <20060807165027.GD9899@zot.electricrain.com> <44D7D0C5.2050204@v.loewis.de> Message-ID: <20060808002107.GG9899@zot.electricrain.com> On Tue, Aug 08, 2006 at 01:46:13AM +0200, "Martin v. L?wis" wrote: > Gregory P. Smith schrieb: > > hashlib's OpenSSL implementation on windows comes in the form of a > > 300k _hashlib.pyd library. > > What do you mean by "comes"? I can't find any _hashlib.vcproj file > inside the PCbuild directory. I'll see about creating one later today when I'm reunited with my laptop. > >> I believe that the performance of the OpenSSL routines depends on > >> the way OpenSSL was built, e.g. whether the assembler implementations > >> are used or not. Somebody would have to check, but I doubt they are. > > > > That'd be unfortunate as that negatively impacts the socket _ssl > > module as well. OpenSSL should always be built with the assembler > > implementations. > > But Visual Studio doesn't ship with an assembler! So how could I > build it? yes it does. Visual Studio comes with MASM (ml) and OpenSSL ships with build scripts to use it. See openssl's INSTALL.W32 file. Also, a free assembler (NASM) is available that OpenSSL is also capable of building with if for some reason you don't have masm installed. Looking into how the windows python build process works (honestly something i've not looked at since 2.0) it appears that PCbuild/build_ssl.py handles the compilation of OpenSSL.. I haven't tested this yet (I'll try it tonight) but I believe this patch is all thats needed to enable the openssl assembly build: --- build_ssl.py (revision 51136) +++ build_ssl.py (working copy) @@ -98,6 +98,8 @@ if not cmd: break if cmd.strip()[:5].lower() == "nmake": continue + if re.match("set OPTS=no-asm", cmd): + continue temp_bat.write(cmd) in_bat.close() temp_bat.close() Alternatively just modifying build_ssl.py to run "ms\do_masm.bat" before it runs "nmake -f ms\32.mak" should work. -g From martin at v.loewis.de Tue Aug 8 02:23:02 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 08 Aug 2006 02:23:02 +0200 Subject: [Python-Dev] windows 2.5 build: use OpenSSL for hashlib [bug 1535502] In-Reply-To: <20060807182123.GE9899@zot.electricrain.com> References: <20060806194135.GC9899@zot.electricrain.com> <44D73D26.4050803@v.loewis.de> <20060807165027.GD9899@zot.electricrain.com> <200608080325.50041.anthony@interlink.com.au> <20060807182123.GE9899@zot.electricrain.com> Message-ID: <44D7D966.3000407@v.loewis.de> Gregory P. Smith schrieb: > Sigh. Half the reason I did the hashlib work was to get much faster > optimized versions of the hash algorithms into python. I'll be > disappointed if that doesn't happen. Sad as it sounds, it appears you just did half of the work, then (omitting the Windows build process). > hashlib passes its test suite with our without openssl. If I make the > windows project file updates to simply build and include _hashlib.pyd > in the windows installer what harm is that going to cause? None, if you do it correctly (where correctly includes AMD64 and IA64 builds), add text to PCbuild/readme.txt, and edit msi.py properly. That is, assuming hashlib then still works correctly on Windows (which is hard to tell). > IMHO the > windows python 2.5 build as it is is missing a feature by not > including this. Wrong incantation :-) We are in feature freeze now, so adding a feature is a big no-no. You should have argued this is a bug <0.5 wink>. Regards, Martin From martin at v.loewis.de Tue Aug 8 02:29:48 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 08 Aug 2006 02:29:48 +0200 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <20060807155500.GA11910@code0.codespeak.net> References: <44D21A99.2080801@brainbot.com> <44D22989.7000202@egenix.com> <44D22FA7.7030905@brainbot.com> <44D234C3.4010404@egenix.com> <44D23807.4050604@egenix.com> <20060807155500.GA11910@code0.codespeak.net> Message-ID: <44D7DAFC.6060308@v.loewis.de> Armin Rigo schrieb: > I also seem to remember that TypeErrors should only signal ordering > non-sense, not equality. In this case, I'm on the opinion that unicode > objects and completely-unrelated strings of random bytes should > successfully compare as unequal, but I'm not enough of a unicode user to > be sure. I believe this was the original intent for raising TypeErrors here in the first place: string-vs-unicode comparison predates rich comparisons, and there is no way to implement __cmp__ meaningfully if the strings don't convert successfully under the system encoding (if they are inequal, you wouldn't be able to tell which one is smaller). With rich comparisons available, I see no reason to keep raising that exception. As for unicode users: As others have said, they should avoid mixing unicode and ascii strings. We provide a fallback for a limited case (ascii); beyond that, Python assumes that non-ascii strings represent uninterpreted bytes, not characters. Regards, Martin From greg at electricrain.com Tue Aug 8 02:36:23 2006 From: greg at electricrain.com (Gregory P. Smith) Date: Mon, 7 Aug 2006 17:36:23 -0700 Subject: [Python-Dev] windows 2.5 build: use OpenSSL for hashlib [bug 1535502] In-Reply-To: <44D7D966.3000407@v.loewis.de> References: <20060806194135.GC9899@zot.electricrain.com> <44D73D26.4050803@v.loewis.de> <20060807165027.GD9899@zot.electricrain.com> <200608080325.50041.anthony@interlink.com.au> <20060807182123.GE9899@zot.electricrain.com> <44D7D966.3000407@v.loewis.de> Message-ID: <20060808003623.GH9899@zot.electricrain.com> On Tue, Aug 08, 2006 at 02:23:02AM +0200, "Martin v. L?wis" wrote: > Gregory P. Smith schrieb: > > Sigh. Half the reason I did the hashlib work was to get much faster > > optimized versions of the hash algorithms into python. I'll be > > disappointed if that doesn't happen. > > Sad as it sounds, it appears you just did half of the work, then > (omitting the Windows build process). I had no access to a windows build environment at the time (as many python developers do not). Apparently I neglected to bribe someone who did to do it after I checked the module in. ;) So is it worth my time doing this in a hurry for 2.5 or do other people really just not care if python for windows uses a slow OpenSSL? Widely deployed popular applications use python for both large scale hashing and ssl communications. If no, can this go in 2.5.1? Its not an API change. -g From martin at v.loewis.de Tue Aug 8 08:26:08 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 08 Aug 2006 08:26:08 +0200 Subject: [Python-Dev] windows 2.5 build: use OpenSSL for hashlib [bug 1535502] In-Reply-To: <20060808003623.GH9899@zot.electricrain.com> References: <20060806194135.GC9899@zot.electricrain.com> <44D73D26.4050803@v.loewis.de> <20060807165027.GD9899@zot.electricrain.com> <200608080325.50041.anthony@interlink.com.au> <20060807182123.GE9899@zot.electricrain.com> <44D7D966.3000407@v.loewis.de> <20060808003623.GH9899@zot.electricrain.com> Message-ID: <44D82E80.5070905@v.loewis.de> Gregory P. Smith schrieb: > So is it worth my time doing this in a hurry for 2.5 or do other > people really just not care if python for windows uses a slow OpenSSL? As I said: I would accept patches. If you arrange for a separate _hashlib.pyd file, most of my concerns would go away. So please do produce a patch, so we have something to review. > Widely deployed popular applications use python for both large scale > hashing and ssl communications. Yet, nobody has worried about performance in all these years to notice that the assembler code isn't being used. So it can't be that bad. For SSL specifically, the usage of hashing is minimal, as the actual communication uses symmetric encryption. Regards, Martin From mal at egenix.com Tue Aug 8 09:25:44 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 08 Aug 2006 09:25:44 +0200 Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys In-Reply-To: <44D73964.3050402@v.loewis.de> References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com> <2m3bccwopj.fsf@starship.python.net> <44D36B36.8030202@egenix.com> <44D73964.3050402@v.loewis.de> Message-ID: <44D83C78.4070008@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg schrieb: >> Python just doesn't know the encoding of the 8-bit string, so can't >> make any assumptions on it. As result, it raises an exception to inform >> the programmer. > > Oh, Python does make an assumption what the encoding is: it assumes > it is the system encoding (i.e. "ascii"). Then invoking the ascii > codec raises an exception, because the string clearly isn't ascii. Right, and as consequence, Python raises an exception to let the programmer correct the problem. The subsequent solution to the problem may result in the string being decoded into Unicode and the two resulting Unicode objects being unequal, or it may also result in them being equal. Python doesn't have this knowledge, so always returning false is clearly wrong. Hiding programmer errors is not making life easier in the long run, so I'm -1 on having the equality comparison return False. Instead we should generate a warning in Python 2.5 and introduce the exception in Python 2.6. >> Note that you do have to interpret the string as characters >> > if you compare it to Unicode and there's nothing wrong with >> > that. > > Consider this: > py> int(3+4j) > Traceback (most recent call last): > File "", line 1, in ? > TypeError: can't convert complex to int; use int(abs(z)) > py> 3 == 3+4j > False > > So even though the conversion raises an exception, the > values are determined to be not equal. Again, because int > is a nearly true subset of complex, the conversion goes > the other way, but *if* it would use the complex->int > conversion, then the TypeError should be taken as > a guarantee that the objects don't compare equal. In the above example, you clearly know that the two are unequal due to the relationship between complex numbers having an imaginary part and integers.. The same is true for the overflow case: >>> 2**10000 == 1.23 False >>> float(2**10000) Traceback (most recent call last): File "", line 1, in ? OverflowError: long int too large to convert to float (Note that in Python 2.3 this used to raise an exception as well.) However, this is not the case for 8-bit string vs. Unicode, since you cannot use such extra knowledge if you find that ASCII encoding assumption obviously doesn't match the string in question. > Expanding this view to Unicode should mean that a unicode > string U equals a byte string B if > U.encode(system_encode) == B or B.decode(system_encoding) == U, > and that they don't equal otherwise Agreed. Note that Python always coerces to the "bigger" type. As a result, the second option is what is actually implemented in Python. > (e.g. if the conversion > fails with a "not convertible" exception). I disagree with this part. Failure to decode a string doesn't imply inequality. It implies that the programmer needs to step in and correct the problem by making an explicit and conscious decision. The alternative would be to decide that equal comparisons should never be allowed to raise exceptions and instead have the equal comparison return False. In which case, we'd have the revert the dict patch altogether and instead silence all exceptions that are generated during the equal comparison (not only in the dict implementation), replacing them with a False return value. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 08 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From martin at v.loewis.de Tue Aug 8 09:56:53 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 08 Aug 2006 09:56:53 +0200 Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys In-Reply-To: <44D83C78.4070008@egenix.com> References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com> <2m3bccwopj.fsf@starship.python.net> <44D36B36.8030202@egenix.com> <44D73964.3050402@v.loewis.de> <44D83C78.4070008@egenix.com> Message-ID: <44D843C5.1060907@v.loewis.de> M.-A. Lemburg schrieb: > Hiding programmer errors is not making life easier in the > long run, so I'm -1 on having the equality comparison return > False. There is no error to hide here. The objects are inequal, period. > Instead we should generate a warning in Python 2.5 and introduce > the exception in Python 2.6. A warning about what? That you can't put byte string and Unicode strings into the same dictionary (as keys)? Next we start not allowing to put numbers and strings into the same dictionary, because there is no conversion defined between them? > In the above example, you clearly know that the two are > unequal due to the relationship between complex numbers > having an imaginary part and integers.. Right. And so I do when the byte string does not convert to Unicode. > However, this is not the case for 8-bit string vs. Unicode, > since you cannot use such extra knowledge if you find that ASCII > encoding assumption obviously doesn't match the string > in question. It's not the question "Could there be a conversion under which they are equal?" If you ask that question, then py> "3"==3 False should raise an exception, because there exists a conversion under which these objects are equal: py> int("3")==3 True It's just that, under the conversion Python applies, the byte string and the Unicode string are not equal. > Note that Python always coerces to the "bigger" type. As a result, > the second option is what is actually implemented in Python. [which is decode-to-unicode] It might be debatable which of the types is the "bigger" type. It's not that byte strings are a true subset of Unicode strings, under some conversion, since there are byte strings which have no Unicode equivalent (because they are not characters, and don't convert under the encoding), and there are Unicode strings that have no byte string equivalent. For example, if the system encoding is UTF-8, then byte string is the bigger type (all Unicode strings convert to byte strings, but not all byte strings convert to Unicode strings). However, this is a red herring: Python has, for whatever reason, chosen to convert byte->unicode, and nobody is questioning that choice. > I disagree with this part. > > Failure to decode a string doesn't imply inequality. If the failure is "these bytes don't have a meaningful character interpretation", then the bytes are *clearly* not equal to some character string. > It implies > that the programmer needs to step in and correct the problem by > making an explicit and conscious decision. There is no problem to correct. The strings *are* inequal. > The alternative would be to decide that equal comparisons should never > be allowed to raise exceptions and instead have the equal comparison > return False. There are many reasons why comparison could raise an exception. It could be out of memory, it could be that there is an internal/programming error in the codec being used, it could be that the codec is not found (likewise for other comparisons). However, if the codec is working properly, and clearly determines that the byte string has no character string equivalent, then it can't be equal to some character (unicode) string. Regards, Martin From greg at electricrain.com Tue Aug 8 09:59:33 2006 From: greg at electricrain.com (Gregory P. Smith) Date: Tue, 8 Aug 2006 00:59:33 -0700 Subject: [Python-Dev] windows 2.5 build: use OpenSSL for hashlib [bug 1535502] In-Reply-To: <20060808003623.GH9899@zot.electricrain.com> References: <20060806194135.GC9899@zot.electricrain.com> <44D73D26.4050803@v.loewis.de> <20060807165027.GD9899@zot.electricrain.com> <200608080325.50041.anthony@interlink.com.au> <20060807182123.GE9899@zot.electricrain.com> <44D7D966.3000407@v.loewis.de> <20060808003623.GH9899@zot.electricrain.com> Message-ID: <20060808075933.GI9899@zot.electricrain.com> I have supplied a patch that does everything needed to both make the windows build process build OpenSSL with x86 assembly optimizations on Win32 and to build the _hashlib.pyd module. It works for me. The only thing the patch doesn't do is add _hashlib.pyd to the .msi windows installer because I want to go to bed and haven't looked into how that works. http://sourceforge.net/tracker/index.php?func=detail&aid=1535502&group_id=5470&atid=105470 I know Anthony suggested "no" earlier (which is why I haven't committed it) but I really think that should be reconsidered. This doesn't change python at all, only fixes a build process. Ship with optimized code by default! help delay the heat death of the universe from cpu fans. :) either way, enjoy. > So is it worth my time doing this in a hurry for 2.5 or do other > people really just not care if python for windows uses a slow OpenSSL? > > Widely deployed popular applications use python for both large scale > hashing and ssl communications. > > If no, can this go in 2.5.1? Its not an API change. > > -g From ralf at brainbot.com Tue Aug 8 10:12:59 2006 From: ralf at brainbot.com (Ralf Schmitt) Date: Tue, 08 Aug 2006 10:12:59 +0200 Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys In-Reply-To: <44D843C5.1060907@v.loewis.de> References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com> <2m3bccwopj.fsf@starship.python.net> <44D36B36.8030202@egenix.com> <44D73964.3050402@v.loewis.de> <44D83C78.4070008@egenix.com> <44D843C5.1060907@v.loewis.de> Message-ID: Martin v. L?wis wrote: > M.-A. Lemburg schrieb: >> Hiding programmer errors is not making life easier in the >> long run, so I'm -1 on having the equality comparison return >> False. > > There is no error to hide here. The objects are inequal, period. And in the case of dicts it hides errors randomly... > >> Instead we should generate a warning in Python 2.5 and introduce >> the exception in Python 2.6. > > A warning about what? That you can't put byte string and Unicode > strings into the same dictionary (as keys)? Next we start not allowing > to put numbers and strings into the same dictionary, because there > is no conversion defined between them? A warning that an exception has been ignored while adding a key to a dict, I guess. I'd see keep those dict changes, this is where real programmer errors are hidden. >> I disagree with this part. >> >> Failure to decode a string doesn't imply inequality. > > If the failure is "these bytes don't have a meaningful character > interpretation", then the bytes are *clearly* not equal to > some character string. One could also think of a "magic encoding", which decodes non-ascii strings to None, making them compare unequal to any unicode string. From greg at electricrain.com Tue Aug 8 10:43:42 2006 From: greg at electricrain.com (Gregory P. Smith) Date: Tue, 8 Aug 2006 01:43:42 -0700 Subject: [Python-Dev] windows 2.5 build: use OpenSSL for hashlib [bug 1535502] In-Reply-To: <44D82E80.5070905@v.loewis.de> References: <20060806194135.GC9899@zot.electricrain.com> <44D73D26.4050803@v.loewis.de> <20060807165027.GD9899@zot.electricrain.com> <200608080325.50041.anthony@interlink.com.au> <20060807182123.GE9899@zot.electricrain.com> <44D7D966.3000407@v.loewis.de> <20060808003623.GH9899@zot.electricrain.com> <44D82E80.5070905@v.loewis.de> Message-ID: <20060808084342.GJ9899@zot.electricrain.com> On Tue, Aug 08, 2006 at 08:26:08AM +0200, "Martin v. L?wis" wrote: > Gregory P. Smith schrieb: > > Widely deployed popular applications use python for both large scale > > hashing and ssl communications. > > Yet, nobody has worried about performance in all these years to notice > that the assembler code isn't being used. So it can't be that bad. > For SSL specifically, the usage of hashing is minimal, as the actual > communication uses symmetric encryption. OpenSSL uses x86 asm to speed up other things besides hashes. For SSL sockets each new connection requires an RSA or DSA public key operation for the symmetric key exchange. Huge speedup there: 2ghz pentium-m with C ssl: sign verify sign/s verify/s rsa 1024 bits 0.007576s 0.000368s 132.0 2715.4 sign verify sign/s verify/s dsa 1024 bits 0.003655s 0.004409s 273.6 226.8 2ghz pentium-m with x86 asm ssl: sign verify sign/s verify/s rsa 1024 bits 0.003410s 0.000171s 293.3 5843.5 sign verify sign/s verify/s dsa 1024 bits 0.001632s 0.001987s 612.9 503.4 [data comes from running "out32\openssl speed" in openssl 0.9.8b] From ralf at brainbot.com Tue Aug 8 12:08:11 2006 From: ralf at brainbot.com (Ralf Schmitt) Date: Tue, 08 Aug 2006 12:08:11 +0200 Subject: [Python-Dev] free(): invalid pointer Message-ID: Hi all, I've got another error porting our apps. It's a django app and stops with (when pressing CTRL-C): *** glibc detected *** free(): invalid pointer: 0xb684c650 *** with MALLOC_CHECK=1 and gdb I get the following backtrace: Program received signal SIGINT, Interrupt. [Switching to Thread -1209690432 (LWP 10036)] 0xffffe410 in __kernel_vsyscall () (gdb) *** glibc detected *** free(): invalid pointer: 0xb66a22a8 *** (gdb) bt #0 0xffffe410 in __kernel_vsyscall () #1 0xb7fbd463 in __waitpid_nocancel () from /lib/tls/i686/cmov/libpthread.so.0 #2 0x080ed524 in posix_waitpid (self=0x0, args=0xfffffe00) at ./Modules/posixmodule.c:5502 #3 0x080bec34 in PyEval_EvalFrameEx (f=0x825a3b4, throwflag=0) at Python/ceval.c:3565 #4 0x080be459 in PyEval_EvalFrameEx (f=0x81bd60c, throwflag=0) at Python/ceval.c:3650 #5 0x080be459 in PyEval_EvalFrameEx (f=0x82354f4, throwflag=0) at Python/ceval.c:3650 #6 0x080be459 in PyEval_EvalFrameEx (f=0x81dbb94, throwflag=0) at Python/ceval.c:3650 #7 0x080bff75 in PyEval_EvalCodeEx (co=0xb7c91c38, globals=0xb7a3713c, locals=0x0, args=0x822fcd0, argcount=1, kws=0x822fcd4, kwcount=0, defs=0xb7a33058, defcount=2, closure=0x0) at Python/ceval.c:2832 #8 0x080bd771 in PyEval_EvalFrameEx (f=0x822fb6c, throwflag=0) at Python/ceval.c:3661 #9 0x080bff75 in PyEval_EvalCodeEx (co=0xb7dfeba8, globals=0xb7dfd714, locals=0x0, args=0x820165c, argcount=2, kws=0x8201664, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:2832 #10 0x080bd771 in PyEval_EvalFrameEx (f=0x82014cc, throwflag=0) at Python/ceval.c:3661 #11 0x080bff75 in PyEval_EvalCodeEx (co=0xb7dfede8, globals=0xb7dfd714, locals=0x0, args=0x81ec784, argcount=1, kws=0x81ec788, kwcount=0, defs=0xb7cfcdb8, defcount=1, closure=0x0) at Python/ceval.c:2832 #12 0x080bd771 in PyEval_EvalFrameEx (f=0x81ec634, throwflag=0) at Python/ceval.c:3661 #13 0x080be459 in PyEval_EvalFrameEx (f=0x81ae054, throwflag=0) at Python/ceval.c:3650 #14 0x080bff75 in PyEval_EvalCodeEx (co=0xb7de2770, globals=0xb7e319bc, locals=0xb7e319bc, args=0x0, argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:2832 ---Type to continue, or q to quit--- #15 0x080c00f6 in PyEval_EvalCode (co=0xfffffe00, globals=0xfffffe00, locals=0xfffffe00) at Python/ceval.c:494 #16 0x080de682 in PyRun_FileExFlags (fp=0x815f008, filename=0xbfdf18b1 "manage.py", start=-512, globals=0xfffffe00, locals=0xfffffe00, closeit=1, flags=0xbfdefbb8) at Python/pythonrun.c:1255 #17 0x080de9f3 in PyRun_SimpleFileExFlags (fp=, filename=0xbfdf18b1 "manage.py", closeit=1, flags=0xbfdefbb8) at Python/pythonrun.c:861 #18 0x08056a69 in Py_Main (argc=2, argv=0xbfdefc54) at Modules/main.c:496 #19 0xb7e6fea2 in __libc_start_main () from /lib/tls/i686/cmov/libc.so.6 #20 0x08055fa1 in _start () at ../sysdeps/i386/elf/start.S:119 I'm using Python 2.5b3 (trunk:51066M, Aug 3 2006, 16:55:04). Sorry for not using the bugtracker (sf sucks). Did you guys already settle on a new one? - Ralf From arigo at tunes.org Tue Aug 8 12:26:08 2006 From: arigo at tunes.org (Armin Rigo) Date: Tue, 8 Aug 2006 12:26:08 +0200 Subject: [Python-Dev] returning longs from __hash__() Message-ID: <20060808102608.GA9098@code0.codespeak.net> Hi all, The 2.5 change of id() to return positive ints-or-longs is likely to cause quite some breakage in user programs that erroneously implemented custom __hash__() functions returning a value based on an id(). This was discussed a few times already but it showed up again as a bug report (#1536021). Of course it has always been documented that id() is not directly suitable as the return value of a custom __hash__(), but so far it worked on 32-bit machines so people have been doing it all the time. Maybe the user should just be able to return any integer value from a custom __hash__() without having to worry about not exceeding sys.maxint. After all the returned value has no real meaning. If a long is returned we could take its hash again, and use that number internally. A bientot, Armin From ralf at brainbot.com Tue Aug 8 15:02:07 2006 From: ralf at brainbot.com (Ralf Schmitt) Date: Tue, 08 Aug 2006 15:02:07 +0200 Subject: [Python-Dev] free(): invalid pointer In-Reply-To: References: Message-ID: <44D88B4F.2000200@brainbot.com> Ralf Schmitt wrote: > > > Sorry for not using the bugtracker (sf sucks). Did you guys already > settle on a new one? > And sorry for bothering this list. It seems like this problem is related to the python cdb module. - Ralf From bart at r8.cg.nu Tue Aug 8 15:39:32 2006 From: bart at r8.cg.nu (Bart Thate) Date: Tue, 8 Aug 2006 15:39:32 +0200 (CEST) Subject: [Python-Dev] should i put this on the bug tracker ? Message-ID: <20060808153512.R59309@r8.cg.nu> hello python-dev, the following code hangs on FreeBSD 6.1-STABLE, Python 2.5b3 (r25b3:51041, Aug 5 2006, 20:46:57) $ cat pythontest #!/usr/local/bin/python2.5 import os, thread, time, sys def reboot(): print 'reboot' os.execl(sys.argv[0], sys.argv[0]) thread.start_new_thread(reboot, ()) while 1: time.sleep(1) works fine on python2.4 is this a bug or is it something i "should not do" ? Bart -------------- next part -------------- #!/usr/local/bin/python2.5 import os, thread, time, sys def reboot(): print 'reboot' os.execl(sys.argv[0], sys.argv[0]) thread.start_new_thread(reboot, ()) while 1: time.sleep(1) From hyeshik at gmail.com Tue Aug 8 16:36:52 2006 From: hyeshik at gmail.com (Hye-Shik Chang) Date: Tue, 8 Aug 2006 23:36:52 +0900 Subject: [Python-Dev] should i put this on the bug tracker ? In-Reply-To: <20060808153512.R59309@r8.cg.nu> References: <20060808153512.R59309@r8.cg.nu> Message-ID: <4f0b69dc0608080736t51656bb3s878b0dced9610b2a@mail.gmail.com> On 8/8/06, Bart Thate wrote: > hello python-dev, > > the following code hangs on FreeBSD 6.1-STABLE, > Python 2.5b3 (r25b3:51041, Aug 5 2006, 20:46:57) > Python 2.5 now uses system scope threads in FreeBSD just like in other platforms. So python may behave different for corner cases. [snip] > > works fine on python2.4 > > is this a bug or is it something i "should not do" ? In my machine (FreeBSD 6.1), 2.4 and 2.5 work same. What was the problem on your running? Did you install it from the port? BTW, os.exec() from a sub-thread looks not so safe. Hye-Shik From bart at r8.cg.nu Tue Aug 8 16:43:03 2006 From: bart at r8.cg.nu (Bart Thate) Date: Tue, 8 Aug 2006 16:43:03 +0200 (CEST) Subject: [Python-Dev] should i put this on the bug tracker ? In-Reply-To: <4f0b69dc0608080736t51656bb3s878b0dced9610b2a@mail.gmail.com> References: <20060808153512.R59309@r8.cg.nu> <4f0b69dc0608080736t51656bb3s878b0dced9610b2a@mail.gmail.com> Message-ID: <20060808164128.T90747@r8.cg.nu> On Tue, 8 Aug 2006, Hye-Shik Chang wrote: > In my machine (FreeBSD 6.1), 2.4 and 2.5 work same. > What was the problem on your running? Did you install > it from the port? i installed it from the python-devel port Bart From bart at r8.cg.nu Tue Aug 8 17:00:52 2006 From: bart at r8.cg.nu (Bart Thate) Date: Tue, 8 Aug 2006 17:00:52 +0200 (CEST) Subject: [Python-Dev] should i put this on the bug tracker ? In-Reply-To: <4f0b69dc0608080736t51656bb3s878b0dced9610b2a@mail.gmail.com> References: <20060808153512.R59309@r8.cg.nu> <4f0b69dc0608080736t51656bb3s878b0dced9610b2a@mail.gmail.com> Message-ID: <20060808165712.Y90747@r8.cg.nu> On Tue, 8 Aug 2006, Hye-Shik Chang wrote: sorry i should reply a little better ;] > On 8/8/06, Bart Thate wrote: > > hello python-dev, > > > > the following code hangs on FreeBSD 6.1-STABLE, > > Python 2.5b3 (r25b3:51041, Aug 5 2006, 20:46:57) > > > > Python 2.5 now uses system scope threads in FreeBSD just like > in other platforms. So python may behave different for corner cases. > is there a way i can compile python with the old settings ? > [snip] > > > > works fine on python2.4 > > > > is this a bug or is it something i "should not do" ? > > In my machine (FreeBSD 6.1), 2.4 and 2.5 work same. > What was the problem on your running? Did you install > it from the port? in python2.5 i cannot start threads after the os.execl, python2.4 lets me reboot my app and has no problem with starting threads after that. > BTW, os.exec() from a sub-thread looks not so safe. i know it has problems but since it was working fine i thought it was ok. > > > Hye-Shik Bart From mal at egenix.com Tue Aug 8 17:08:18 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 08 Aug 2006 17:08:18 +0200 Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys In-Reply-To: <44D843C5.1060907@v.loewis.de> References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com> <2m3bccwopj.fsf@starship.python.net> <44D36B36.8030202@egenix.com> <44D73964.3050402@v.loewis.de> <44D83C78.4070008@egenix.com> <44D843C5.1060907@v.loewis.de> Message-ID: <44D8A8E2.5090002@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg schrieb: >> Failure to decode a string doesn't imply inequality. > > If the failure is "these bytes don't have a meaningful character > interpretation", then the bytes are *clearly* not equal to > some character string. > >> It implies >> that the programmer needs to step in and correct the problem by >> making an explicit and conscious decision. > > There is no problem to correct. The strings *are* inequal. If the programmer writes: x = '???' y = u'???' ... if x == y: do_something() then he clearly has had the intention to compare two character strings. Now, if what you were saying were true, then the above would simply continue to work without raising an exception, possibly causing the application to return wrong results. With the exception, the programmer will have a chance to correct the problem (in this case, probably a forgotten u-prefix) and also be safe in not having the application produce wrong data - something that's usually hard to detect, debug and, more importantly, can have effects which are a lot worse than a failing application. Note that we are not discussing changing the behavior of the __eq__ comparison between strings and Unicode, since this has always been to raise exceptions in case the automatic propagation fails. The discussion is about silencing exceptions in the dict lookup mechanism - something which used to happen and now no longer is done. Since this behavior is an implementation detail of the dictionary implementation, users perceive this change as random exceptions occurring in their application. While these exceptions do hint at programming errors (the main reason for no longer silencing them), the particular case in the dict implementation requires some extra thought. I've suggested to go about this in a slightly more user-friendly way, namely by giving a warning instead of raising an exception in Python 2.5 and then going for the exception in Python 2.6. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 08 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From exarkun at divmod.com Tue Aug 8 17:11:43 2006 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Tue, 8 Aug 2006 11:11:43 -0400 Subject: [Python-Dev] Release manager pronouncement needed: PEP 302 Fix In-Reply-To: <5.1.1.6.0.20060728173543.01b7c120@sparrow.telecommunity.com> Message-ID: <20060808151143.1717.322851754.divmod.quotient.17504@ohm> On Fri, 28 Jul 2006 18:00:36 -0400, "Phillip J. Eby" wrote: >At 10:55 PM 7/28/2006 +0200, Martin v. L?wis wrote: >>Phillip J. Eby wrote: >> > The issue is that a proper fix that caches existence requires adding new >> > types to import.c and thus might appear to be more of a feature. I was >> > therefore reluctant to embark upon the work without some assurance that it >> > wouldn't be rejected as adding a last-minute feature. >> >>So do you have a patch, or are going to write one? > >Yes, it's checked in as r50916. > >It ultimately turned out to be simpler than I thought; only one new type >(imp.NullImporter) was required. > Is this going to be the final state of PEP 302 support in Python 2.5? I don't particularly care how this ends up, but I'd like to know what has been decided on (PEP 302 doesn't seem to have been updated yet) so I can fix Twisted's test suite (which cannot even be run with Python 2.5b3 right now). Jean-Paul From mal at egenix.com Tue Aug 8 17:33:46 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 08 Aug 2006 17:33:46 +0200 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <20060807155500.GA11910@code0.codespeak.net> References: <44D21A99.2080801@brainbot.com> <44D22989.7000202@egenix.com> <44D22FA7.7030905@brainbot.com> <44D234C3.4010404@egenix.com> <44D23807.4050604@egenix.com> <20060807155500.GA11910@code0.codespeak.net> Message-ID: <44D8AEDA.70305@egenix.com> Armin Rigo wrote: > Hi, > > On Thu, Aug 03, 2006 at 07:53:11PM +0200, M.-A. Lemburg wrote: >>> I though I'd heard (from Guido here or on the py3k list) that it was only >>> 1 < u'abc' that would raise an exception, and that 1 == u'abc' would still >>> evaluate to False. Did I misunderstand? >> Could be that I'm wrong. > > I also seem to remember that TypeErrors should only signal ordering > non-sense, not equality. In this case, I'm on the opinion that unicode > objects and completely-unrelated strings of random bytes should > successfully compare as unequal, but I'm not enough of a unicode user to > be sure. Agreed - for Py3k where strings no longer exist and Unicode is the only text type. In Python 2.x the situation is a little different, since strings are still very often used as container for text data. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 08 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From pje at telecommunity.com Tue Aug 8 18:04:16 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 08 Aug 2006 12:04:16 -0400 Subject: [Python-Dev] Release manager pronouncement needed: PEP 302 Fix In-Reply-To: <20060808151143.1717.322851754.divmod.quotient.17504@ohm> References: <5.1.1.6.0.20060728173543.01b7c120@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20060808120140.03a0b480@sparrow.telecommunity.com> At 11:11 AM 8/8/2006 -0400, Jean-Paul Calderone wrote: >On Fri, 28 Jul 2006 18:00:36 -0400, "Phillip J. Eby" > wrote: >>At 10:55 PM 7/28/2006 +0200, Martin v. L??wis wrote: >>>Phillip J. Eby wrote: >>> > The issue is that a proper fix that caches existence requires adding new >>> > types to import.c and thus might appear to be more of a feature. I was >>> > therefore reluctant to embark upon the work without some assurance >>> that it >>> > wouldn't be rejected as adding a last-minute feature. >>> >>>So do you have a patch, or are going to write one? >> >>Yes, it's checked in as r50916. >> >>It ultimately turned out to be simpler than I thought; only one new type >>(imp.NullImporter) was required. > >Is this going to be the final state of PEP 302 support in Python 2.5? I >don't particularly care how this ends up, but I'd like to know what has >been decided on (PEP 302 doesn't seem to have been updated yet) so I can >fix Twisted's test suite (which cannot even be run with Python 2.5b3 >right now). PEP 302 doesn't need to be changed, since Python now conforms to it again. That is, every object in sys.path_importer_cache is either an importer or None. It's just that there is an additional type of importer there that didn't occur before. If you have code that expects only certain types of importers to be there, that code was already broken. From exarkun at divmod.com Tue Aug 8 18:18:10 2006 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Tue, 8 Aug 2006 12:18:10 -0400 Subject: [Python-Dev] Release manager pronouncement needed: PEP 302 Fix In-Reply-To: <5.1.1.6.0.20060808120140.03a0b480@sparrow.telecommunity.com> Message-ID: <20060808161810.1717.717581672.divmod.quotient.17543@ohm> On Tue, 08 Aug 2006 12:04:16 -0400, "Phillip J. Eby" wrote: > >PEP 302 doesn't need to be changed, since Python now conforms to it again. >That is, every object in sys.path_importer_cache is either an importer or >None. It's just that there is an additional type of importer there that >didn't occur before. If you have code that expects only certain types of >importers to be there, that code was already broken. > Thanks. Jean-Paul From david.nospam.hopwood at blueyonder.co.uk Tue Aug 8 18:42:14 2006 From: david.nospam.hopwood at blueyonder.co.uk (David Hopwood) Date: Tue, 08 Aug 2006 17:42:14 +0100 Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys In-Reply-To: <44D7CC5C.5080508@v.loewis.de> References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com> <44D2F653.5090305@egenix.com> <44D3140A.2040600@canterbury.ac.nz> <44D31E64.3070407@egenix.com> <44D73635.4000003@v.loewis.de> <44D73E90.6070707@voidspace.org.uk> <44D754CB.8070607@blueyonder.co.uk> <44D77351.80603@voidspace.org.uk> <44D7B1D2.2000105@blueyonder.co.uk> <44D7CC5C.5080508@v.loewis.de> Message-ID: <44D8BEE6.70902@blueyonder.co.uk> Martin v. L?wis wrote: > David Hopwood schrieb: >>Michael Foord wrote: >>>David Hopwood wrote:[snip..] >>> >>>>>>we should, of course, continue to use the one we always used (for >>>>>>"ascii", there is no difference between the two). >>>>> >>>>>+1 >>>>> >>>>>This seems the most (only ?) logical solution. >>>> >>>>No; always considering Unicode and non-ASCII byte strings to be distinct >>>>is just as logical. >> >>I think you must have misread my comment: > > Indeed. The misunderstanding originates from your sentence starting with > "no", when, in fact, you seem to be supporting the proposal I made. I had misunderstood what the existing Python behaviour is. I now think the current behaviour (which uses "B.decode(system_encoding) == U") is definitely a bad idea, especially in cases where the system encoding is not US-ASCII, but I agree that it can't be changed for 2.5. -- David Hopwood From jimjjewett at gmail.com Tue Aug 8 19:09:47 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Tue, 8 Aug 2006 13:09:47 -0400 Subject: [Python-Dev] openSSL and windows binaries - license Message-ID: The OpenSSL library implements some algorithms that are patented. The source code should be fine to (re)distribute, but but there may be a slight legal risk with distributing a binary. Note that http://www.openssl.org/support/faq.html#LEGAL1 says that we can avoid building the problem sections with ./config no-idea no-mdc2 no-rc5 As best I could tell from (search for %disabled in) http://svn.python.org/view/external/openssl-0.9.8a/Configure (or search for OPTIONS in) http://svn.python.org/view/external/openssl-0.9.8a/Makefile python just takes the OpenSSL defaults, which excludes RC5 and MDC2 but does build IDEA. The documentation does not promise any of these three, and it doesn't look like they're used internally or advertised, but they are available if built. It might be safer to explicitly exclude IDEA from the binary distribution. (Well, unless the PSF actually has an appropriate license, which is possible. http://svn.python.org/view/external/openssl-0.9.8a/README says where to get them.) -jJ From martin at v.loewis.de Tue Aug 8 21:00:26 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 08 Aug 2006 21:00:26 +0200 Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys In-Reply-To: <44D8A8E2.5090002@egenix.com> References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com> <2m3bccwopj.fsf@starship.python.net> <44D36B36.8030202@egenix.com> <44D73964.3050402@v.loewis.de> <44D83C78.4070008@egenix.com> <44D843C5.1060907@v.loewis.de> <44D8A8E2.5090002@egenix.com> Message-ID: <44D8DF4A.70408@v.loewis.de> M.-A. Lemburg schrieb: > If the programmer writes: > > x = '???' > y = u'???' > ... > if x == y: > do_something() > > then he clearly has had the intention to compare two character > strings. Programmers make all kinds of mistakes when comparing objects, assuming that things ought to be equal that actually aren't: py> 1.6/math.pi*math.pi == 1.6 False py> if 10*10 is 100: ... print "yes" ... else: ... print "no" ... no > Now, if what you were saying were true, then the above would > simply continue to work without raising an exception, possibly > causing the application to return wrong results. That correct. It is a programming mistake, hence you get a wrong result. However, you cannot assume that every comparison between a string and a Unicode object is always a programming mistake. You must not raise exceptions just because of a *potential* programming mistake; that's what PyChecker is there for. > Note that we are not discussing changing the behavior of the > __eq__ comparison between strings and Unicode, since this has > always been to raise exceptions in case the automatic propagation > fails. Not sure what you are discussing: This is *precisely* what I'm discussing. Making that change would solve this problem. > The discussion is about silencing exceptions in the dict lookup > mechanism - something which used to happen and now no longer > is done. No, that's not what the discussion is about. The discussion is about the backwards incompatibility in Python 2.5 wrt. Python 2.4. There are several ways to solve that; silencing the exception is just one way. I think it is the wrong way, as I think that string-unicode-comparison should have a consistent behaviour no matter where the comparison occurs. > Since this behavior is an implementation detail of the > dictionary implementation, users perceive this change as random > exceptions occurring in their application. That key comparison occurs is *not* an implementation detail. It is a necessary and documented aspect of the dictionary lookup. > I've suggested to go about this in a slightly more user-friendly > way, namely by giving a warning instead of raising an exception > in Python 2.5 and then going for the exception in Python 2.6. Yes, and I have suggested to make it even more user-friendly by defining string-unicode-__eq__ in a sensible manner. It is more user-friendly, because it doesn't show the inconsistency Michael Hudson documented in http://mail.python.org/pipermail/python-dev/2006-August/067981.html Regards, Martin From martin at v.loewis.de Tue Aug 8 21:08:56 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 08 Aug 2006 21:08:56 +0200 Subject: [Python-Dev] returning longs from __hash__() In-Reply-To: <20060808102608.GA9098@code0.codespeak.net> References: <20060808102608.GA9098@code0.codespeak.net> Message-ID: <44D8E148.5050701@v.loewis.de> Armin Rigo schrieb: > Maybe the user should just be able to return any integer value from a > custom __hash__() without having to worry about not exceeding > sys.maxint. > > After all the returned value has no real meaning. If a long is returned > we could take its hash again, and use that number internally. Nick Coghlan already suggested that, when __hash__ returns a long int, the tp_hash of long int should be used to compute the true hash value. Could you see any problems with that approach? If not, and if I don't hear other objections, I'd like to go ahead and fix it that way. Regards, Martin From martin at v.loewis.de Tue Aug 8 21:18:36 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 08 Aug 2006 21:18:36 +0200 Subject: [Python-Dev] openSSL and windows binaries - license In-Reply-To: References: Message-ID: <44D8E38C.8000505@v.loewis.de> Jim Jewett schrieb: > The OpenSSL library implements some algorithms that are patented. The > source code should be fine to (re)distribute, but but there may be a > slight legal risk with distributing a binary. I don't want to change the build process in that way (i.e. dropping a feature) just before a release. I personally don't think there is a risk distributing the code (if there was, distribution of OpenSSL would also be a risk); anybody /using/ a patented algorithm would violate the patent. I am not a lawyer, so if you want to know for sure, ask one. If you think it's necessary, a note could be added to the readme. Would you like to create a patch? Regards, Martin From skip at pobox.com Tue Aug 8 21:33:58 2006 From: skip at pobox.com (skip at pobox.com) Date: Tue, 8 Aug 2006 14:33:58 -0500 Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys In-Reply-To: <44D8DF4A.70408@v.loewis.de> References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com> <2m3bccwopj.fsf@starship.python.net> <44D36B36.8030202@egenix.com> <44D73964.3050402@v.loewis.de> <44D83C78.4070008@egenix.com> <44D843C5.1060907@v.loewis.de> <44D8A8E2.5090002@egenix.com> <44D8DF4A.70408@v.loewis.de> Message-ID: <17624.59174.240858.504775@montanaro.dyndns.org> Martin> Programmers make all kinds of mistakes when comparing objects, Martin> assuming that things ought to be equal that actually aren't: py> 1.6/math.pi*math.pi == 1.6 False By extension, perhaps Computer Science departments should begin offering Unicode Analysis as an advanced undergraduate class. ;-) (Sorry, couldn't resist...) Skip From theller at python.net Tue Aug 8 21:46:24 2006 From: theller at python.net (Thomas Heller) Date: Tue, 08 Aug 2006 21:46:24 +0200 Subject: [Python-Dev] Releasemanager, please approve #1532975 In-Reply-To: References: Message-ID: Thomas Heller schrieb: > Approval requested for patch: > http://python.org/sf/1532975 > What does the silence mean? Should I go ahead and commit this patch? Thanks, Thomas From martin at v.loewis.de Tue Aug 8 21:53:48 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 08 Aug 2006 21:53:48 +0200 Subject: [Python-Dev] Releasemanager, please approve #1532975 In-Reply-To: References: Message-ID: <44D8EBCC.2000306@v.loewis.de> Thomas Heller schrieb: > Thomas Heller schrieb: >> Approval requested for patch: >> http://python.org/sf/1532975 >> > > What does the silence mean? Should I go ahead and commit this patch? If it's not there already, you should add it to the PEP. If you think it is "release-critical" (i.e. it would be really embarrassing for Python 2.5 if that change wasn't made), you should put it into the appropriate section. Regards, Martin From tim.peters at gmail.com Tue Aug 8 22:20:01 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 8 Aug 2006 16:20:01 -0400 Subject: [Python-Dev] returning longs from __hash__() In-Reply-To: <44D8E148.5050701@v.loewis.de> References: <20060808102608.GA9098@code0.codespeak.net> <44D8E148.5050701@v.loewis.de> Message-ID: <1f7befae0608081320j61bacd63l8ea8d3a9f32f0424@mail.gmail.com> [Armin] >> Maybe the user should just be able to return any integer value from a >> custom __hash__() without having to worry about not exceeding >> sys.maxint. >> >> After all the returned value has no real meaning. If a long is returned >> we could take its hash again, and use that number internally. [Martin] > Nick Coghlan already suggested that, when __hash__ returns a long int, > the tp_hash of long int should be used to compute the true hash value. > > Could you see any problems with that approach? If not, and if I don't > hear other objections, I'd like to go ahead and fix it that way. It sounds fine to me, except I'm not immediately clear on which code needs to be changed. The internal _Py_HashPointer() already does exactly this (return the hash of a Python long) when PyObject_Hash() decides to hash an address on a SIZEOF_LONG < SIZEOF_VOID_P box ... but on a SIZEOF_LONG == SIZEOF_VOID_P box, _Py_HashPointer() may still return a negative C long. I /hope/ that a class that decides to add def __hash__(self): return id(self) will end up using the same hash code internally as when that supposedly do-nothing-different definition doesn't exist. Note that a while back I changed several custom __hash__ methods in Python's test suite to stop returning id(self) (as a result of tests failing after the "make id() non-negative" change). That's why we haven't seen such complaints from the buildbots recently. I expect that few Python programmers realize it was never legit for __hash__() to return id(self), and that it's not worth forcing them to learn that now ;-) From jimjjewett at gmail.com Tue Aug 8 22:54:44 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Tue, 8 Aug 2006 16:54:44 -0400 Subject: [Python-Dev] openSSL and windows binaries - license In-Reply-To: <44D8E38C.8000505@v.loewis.de> References: <44D8E38C.8000505@v.loewis.de> Message-ID: On 8/8/06, "Martin v. L?wis" wrote: > Jim Jewett schrieb: > > The OpenSSL library implements some algorithms that are patented. The > > source code should be fine to (re)distribute, but but there may be a > > slight legal risk with distributing a binary. > I don't want to change the build process in that way (i.e. dropping a > feature) just before a release. OK, but this does argue against making the fast version available by default on windows. :{ The 2.5c1 windows binary does not ship with _hashlib, so IDEA is only available if someone else has compiled it. The problem is that if the binary is recompiled as part of the python build (as is typical on unix), then a default build will make IDEA available. > I personally don't think there is a risk > distributing the code (if there was, distribution of OpenSSL would also > be a risk); anybody /using/ a patented algorithm would violate the There is at least some legal precedent for treating source code differently from binaries. (http://jya.com/bernstein-9th.htm) The source code is a communication or description (which would not violate the patent), while the binary is an implementation (which would). Note that openssl.org do not distribute binaries themselves, and at least some vendors (such as Red Hat) have excluded the questionable algorithms from their binaries. > If you think it's necessary, a note could be added to the readme. Would > you like to create a patch? I believe the openSSL readme http://svn.python.org/view/external/openssl-0.9.8a/README is sufficient for a source release. But for a binary release, I think that IDEA should be added to the Configure exclude. http://svn.python.org/view/external/openssl-0.9.8a/Configure # All of the following is disabled by default (RC5 was enabled before 0.9.8): my %disabled = ( # "what" => "comment" "gmp" => "default", + "idea" => "default", "mdc2" => "default", "rc5" => "default", "shared" => "default", "zlib" => "default", "zlib-dynamic" => "default" ); -jJ From martin at v.loewis.de Tue Aug 8 23:14:56 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 08 Aug 2006 23:14:56 +0200 Subject: [Python-Dev] returning longs from __hash__() In-Reply-To: <1f7befae0608081320j61bacd63l8ea8d3a9f32f0424@mail.gmail.com> References: <20060808102608.GA9098@code0.codespeak.net> <44D8E148.5050701@v.loewis.de> <1f7befae0608081320j61bacd63l8ea8d3a9f32f0424@mail.gmail.com> Message-ID: <44D8FED0.9060108@v.loewis.de> Tim Peters schrieb: > It sounds fine to me, except I'm not immediately clear on which code > needs to be changed. My change would essentially be the code below, in instance_hash and slot_tp_hash; I have yet to add test cases and check for documentation changes. Regards, Martin Index: Objects/typeobject.c =================================================================== --- Objects/typeobject.c (Revision 51155) +++ Objects/typeobject.c (Arbeitskopie) @@ -4559,7 +4559,10 @@ Py_DECREF(func); if (res == NULL) return -1; - h = PyInt_AsLong(res); + if (PyLong_Check(res)) + h = res->ob_type->tp_hash(res); + else + h = PyInt_AsLong(res); Py_DECREF(res); } else { Index: Objects/classobject.c =================================================================== --- Objects/classobject.c (Revision 51155) +++ Objects/classobject.c (Arbeitskopie) @@ -934,11 +934,9 @@ Py_DECREF(func); if (res == NULL) return -1; - if (PyInt_Check(res)) { - outcome = PyInt_AsLong(res); - if (outcome == -1) - outcome = -2; - } + if (PyInt_Check(res) || PyLong_Check(res)) + /* This already converts a -1 result to -2. */ + outcome = res->ob_type->tp_hash(res); else { PyErr_SetString(PyExc_TypeError, "__hash__() should return an int"); From greg at electricrain.com Wed Aug 9 00:07:48 2006 From: greg at electricrain.com (Gregory P. Smith) Date: Tue, 8 Aug 2006 15:07:48 -0700 Subject: [Python-Dev] openSSL and windows binaries - license In-Reply-To: References: <44D8E38C.8000505@v.loewis.de> Message-ID: <20060808220748.GN9899@zot.electricrain.com> On Tue, Aug 08, 2006 at 04:54:44PM -0400, Jim Jewett wrote: > On 8/8/06, "Martin v. L?wis" wrote: > > Jim Jewett schrieb: > > > The OpenSSL library implements some algorithms that are patented. The > > > source code should be fine to (re)distribute, but but there may be a > > > slight legal risk with distributing a binary. > > > I don't want to change the build process in that way (i.e. dropping a > > feature) just before a release. > > OK, but this does argue against making the fast version available by > default on windows. :{ disabling/enabling a cipher in openssl that isn't commonly used and isn't even directly exposed via any API to a python user hardly sounds like dropping a feature to me. it'll make your _ssl.pyd smaller if anything at all. (any sane SSL connection will negotiate AES or 3DES as its cipher; IDEA isn't required) If the release manager declares, "absolutely no changes to the windows build process!" Then clearly none of the changes I submitted will make it in and neither would removing any hint of IDEA in 2.5 as they're both too late. > The 2.5c1 windows binary does not ship with _hashlib, so IDEA is only > available if someone else has compiled it. IDEA is a cipher not a hash algorithm. it won't appear in _hashlib. the code is probably already linked and present in _ssl.pyd even if the ssl protocol itself doesn't allow that as a cipher. > But for a binary release, I think that IDEA should be added to the > Configure exclude. > http://svn.python.org/view/external/openssl-0.9.8a/Configure > > # All of the following is disabled by default (RC5 was enabled > before 0.9.8): > > my %disabled = ( # "what" => "comment" > "gmp" => "default", > + "idea" => "default", > "mdc2" => "default", > "rc5" => "default", > "shared" => "default", > "zlib" => "default", > "zlib-dynamic" => "default" > ); yeah i'd just do that if you're worried about the code being in the binary causing a problem. From tim.peters at gmail.com Wed Aug 9 03:09:11 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 8 Aug 2006 21:09:11 -0400 Subject: [Python-Dev] 2.5 status In-Reply-To: References: Message-ID: <1f7befae0608081809v3cd63b20l313b9d6b5cd4eef@mail.gmail.com> [Georg Brandl, on http://python.org/sf/1523610 - PyArg_ParseTupleAndKeywords potential core dump ] > This one's almost fixed if we can decide what to do with "levels". > I wrote some time ago: > > """ > With respect to this bug (which is about stack issues in Python/getargs.c > involving misuse of the "levels" array), I think that we can drop the > "levels" thing completely. It's only there to tell the user which exact item > passed as part of a tuple argument cannot be accepted (and only if that > function is implemented in C code). As tuple arguments > are very rare "argument x" should be enough to tell the user that > something's wrong with that tuple. > """ More, the problem that remains is purely "a head bug": nobody ever bumped into it, and the only way to provoke it is to write C (calling, e.g., PyArg_ParseTupleAndKeywords) nesting tuple codes in an argument descriptor string to an absurd depth. This is far from serious -- heck, it's far from even interesting <0.5 wink>. I suggest closing this bug, getting it out of PEP 356, and opening a new low-priority bug report against it. Fine by me if the `levels` convolutions went away entirely, but at this stage in the 2.5 release process I expect that's a bad idea: there is no /actual/ real-life bug here, and getting rid of `levels` seems far more likely (because of the amount of code touched) to introduce a new error than that someone will nest tuple argument descriptors 32 deep. If someone's actually afraid of that, fine, s/32/320/g in getargs.c instead :-) From greg.ewing at canterbury.ac.nz Wed Aug 9 03:43:32 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 09 Aug 2006 13:43:32 +1200 Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys In-Reply-To: <44D83C78.4070008@egenix.com> References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com> <2m3bccwopj.fsf@starship.python.net> <44D36B36.8030202@egenix.com> <44D73964.3050402@v.loewis.de> <44D83C78.4070008@egenix.com> Message-ID: <44D93DC4.4030403@canterbury.ac.nz> M.-A. Lemburg wrote: > Hiding programmer errors is not making life easier in the > long run, so I'm -1 on having the equality comparison return > False. I don't see how this is greatly different from, e.g. [1, 2] == (1, 2) returning False. Comparing things of different types may or may not indicate a bug in the code as well, but we don't seem to worry that it doesn't raise an exception. -- Greg From greg.ewing at canterbury.ac.nz Wed Aug 9 05:02:21 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 09 Aug 2006 15:02:21 +1200 Subject: [Python-Dev] openSSL and windows binaries - license In-Reply-To: <44D8E38C.8000505@v.loewis.de> References: <44D8E38C.8000505@v.loewis.de> Message-ID: <44D9503D.1040809@canterbury.ac.nz> Martin v. L?wis wrote: > I personally don't think there is a risk > distributing the code (if there was, distribution of OpenSSL would also > be a risk); anybody /using/ a patented algorithm would violate the > patent. If distributing the source doesn't violate the patent, and distributing a binary doesn't violate the patent, then what *would* constitute a violation of a software patent? Writing new code using the algorithm? Compiling something which uses it? -- Greg From kbk at shore.net Wed Aug 9 07:34:21 2006 From: kbk at shore.net (Kurt B. Kaiser) Date: Wed, 9 Aug 2006 01:34:21 -0400 (EDT) Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200608090534.k795YLcP005740@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 402 open ( +6) / 3360 closed ( +6) / 3762 total (+12) Bugs : 861 open ( -3) / 6114 closed (+27) / 6975 total (+24) RFE : 228 open ( +2) / 234 closed ( +0) / 462 total ( +2) New / Reopened Patches ______________________ Replace the ctypes internal '_as_parameter_' mechanism (2006-08-02) http://python.org/sf/1532975 opened by Thomas Heller Remove mentions of "PyUnit" from unittest docs (2006-08-02) CLOSED http://python.org/sf/1533336 opened by Collin Winter Allow thread(ing) tests to pass without setting stack size (2006-08-02) http://python.org/sf/1533520 opened by Matt Fleming Let timeit accept functions (2006-08-03) http://python.org/sf/1533909 opened by Erik Demaine Add notes on locale module changes to whatsnew25.tex (2006-08-03) http://python.org/sf/1534027 opened by Iain Lowe Typo in weakref error message (2006-08-03) CLOSED http://python.org/sf/1534048 opened by Christopher Tur Lesniewski-Laas Fix code generation bug in 'compiler' package (2006-08-03) CLOSED http://python.org/sf/1534084 opened by Neil Schemenauer Cleanup/error-correction for unittest's docs (2006-08-05) CLOSED http://python.org/sf/1534922 opened by Collin Winter writelines() in bz2 module does not raise check for errors (2006-08-06) http://python.org/sf/1535500 opened by Lawrence Oluyede CGIHTTPServer doesn't handle path names with embeded space (2006-08-06) http://python.org/sf/1535504 opened by Hartmut Goebel NNTPS support in nntplib (2006-08-06) http://python.org/sf/1535659 opened by Aurojit Panda trace.py on win32 has problems with lowercase drive names (2006-08-07) http://python.org/sf/1536071 opened by Adam Groszer Build ctypes on OpenBSD x86_64 (2006-08-08) http://python.org/sf/1536908 opened by Thomas Heller Patches Closed ______________ New ver. of 1102879: Fix for 926423: socket timeouts (2006-07-07) http://python.org/sf/1519025 closed by nnorwitz Remove mentions of "PyUnit" from unittest docs (2006-08-02) http://python.org/sf/1533336 deleted by collinwinter Typo in weakref error message (2006-08-03) http://python.org/sf/1534048 closed by fdrake Fix code generation bug in 'compiler' package (2006-08-03) http://python.org/sf/1534084 closed by nascheme Cleanup/error-correction for unittest's docs (2006-08-05) http://python.org/sf/1534922 closed by gbrandl New / Reopened Bugs ___________________ NetBSD build with --with-pydebug causes SIGSEGV (2006-08-02) http://python.org/sf/1533105 opened by Matt Fleming the csv module writes files that Excel sees as SYLK files (2006-08-01) CLOSED http://python.org/sf/1532483 reopened by madewokherd Installed but not listed *.pyo break bdist_rpm (2006-08-02) http://python.org/sf/1533164 opened by Shmyrev Nick CTypes _as_parameter_ not working as documented (2006-08-02) http://python.org/sf/1533481 opened by Shane Holloway long -> Py_ssize_t (C/API 1.2.1) (2006-08-02) http://python.org/sf/1533486 opened by Jim Jewett C/API sec 10 is clipped (2006-08-02) http://python.org/sf/1533491 opened by Jim Jewett Tools/modulator does not exist (ext 1.4) (2006-08-02) http://python.org/sf/1533493 opened by Jim Jewett botched html for index subheadings (2004-05-26) CLOSED http://python.org/sf/960860 reopened by jimjjewett __name__ doesn't show up in dir() of class (2006-08-03) http://python.org/sf/1534014 opened by Tim Chase getlines() in linecache.py raises TypeError (2006-08-04) CLOSED http://python.org/sf/1534517 opened by Stefan Behnel Python 2.5 svn crash in _elementtree.c (2006-08-04) http://python.org/sf/1534630 opened by Barry A. Warsaw Win32 debug version of _msi creates _msi.pyd, not _msi_d.pyd (2006-08-04) http://python.org/sf/1534738 opened by John Ehresman sys.path gets munged with certain directory structures (2006-08-04) http://python.org/sf/1534764 opened by Gustavo Tabares logging's fileConfig causes KeyError on shutdown (2006-08-04) http://python.org/sf/1534765 opened by mdbeachy Identical floats print inconsistently (2006-08-04) CLOSED http://python.org/sf/1534769 opened by Marc W. Abel termios.c in qnx4.25 (2005-09-19) http://python.org/sf/1295179 reopened by gbrandl can't staticaly build modules md5 and sha (2006-08-06) CLOSED http://python.org/sf/1535081 opened by kbob_ru python segfaults when reading from closed stdin (2006-08-05) CLOSED http://python.org/sf/1535165 opened by Patrick Mezard typo in test_bz2.py (2006-08-06) CLOSED http://python.org/sf/1535182 opened by Lawrence Oluyede Python 2.5 windows builds should link hashlib with OpenSSL (2006-08-06) http://python.org/sf/1535502 opened by Gregory P. Smith hash(method) sometimes raises OverflowError (2006-08-07) http://python.org/sf/1536021 opened by Christian Tanzer Missing builtin help for with and as (2006-08-08) http://python.org/sf/1536059 opened by Nick Coghlan "make install" doesn't install to /usr/lib64 on x86_64 boxes (2006-08-07) http://python.org/sf/1536339 opened by Bernhard Rosenkraenzer Micro fix for Reference Manual (2006-08-08) CLOSED http://python.org/sf/1536660 opened by Marcelo de Gomensoro Malheiros buffer comparison emits a RuntimeWarning (2006-08-08) CLOSED http://python.org/sf/1536786 opened by Thomas Heller distutils.sysconfig.get_config_h_filename gets useless file (2006-08-08) http://python.org/sf/1536825 opened by Parzival Herzog Typo, types.TypeType should read types.StringType (2006-08-08) CLOSED http://python.org/sf/1536828 opened by Reillyeon Bugs Closed ___________ socket timeouts + Ctrl-C don't play nice (2004-03-30) http://python.org/sf/926423 closed by nnorwitz mailbox.PortableUnixMailbox fails to parse 'From ' in body (2006-04-13) http://python.org/sf/1470212 closed by akuchling Build fails on MacOSX with missing symbol (2006-07-31) http://python.org/sf/1531662 closed by gdm ctypes build fails on Solaris 10 (2006-07-28) http://python.org/sf/1530448 closed by theller the csv module writes files that Excel sees as SYLK files (2006-08-01) http://python.org/sf/1532483 closed by montanaro incorrect behaviour of PyUnicode_EncodeMBCS? (2006-08-01) http://python.org/sf/1532726 closed by nnorwitz format_exception raises if str(exception) raises (2006-07-30) http://python.org/sf/1531405 closed by nnorwitz botched html for index subheadings (2004-05-26) http://python.org/sf/960860 closed by jimjjewett MACOSX_DEPLOYMENT_TARGET checked incorrectly (2005-04-30) http://python.org/sf/1193190 closed by etrepum [AST] Failing tests (2005-04-27) http://python.org/sf/1191458 closed by nnorwitz Bugs of the new AST compiler (2005-10-21) http://python.org/sf/1333982 closed by nnorwitz test_ossaudiodev timing failure (2003-08-04) http://python.org/sf/783242 closed by akuchling repr.repr not always safe (2003-01-12) http://python.org/sf/666958 closed by akuchling ftplib has incomplete transfer when sending files in Windows (2004-09-22) http://python.org/sf/1032875 closed by akuchling datetime.strftime %s (2005-07-07) http://python.org/sf/1234123 closed by akuchling __weaklist__ in typeobject.c (2006-07-29) http://python.org/sf/1531003 closed by fdrake getlines() in linecache.py raises TypeError (2006-08-04) http://python.org/sf/1534517 closed by scoder Identical floats print inconsistently (2006-08-04) http://python.org/sf/1534769 closed by gbrandl zipfile -- too many files? (2006-07-09) http://python.org/sf/1519452 closed by gbrandl can't staticaly build modules md5 and sha (2006-08-05) http://python.org/sf/1535081 closed by gbrandl python segfaults when reading from closed stdin (2006-08-05) http://python.org/sf/1535165 closed by gbrandl typo in test_bz2.py (2006-08-05) http://python.org/sf/1535182 closed by gbrandl TAB SETTINGS DONT WORK (win) (2005-10-27) http://python.org/sf/1339883 closed by kbk non-uniform behavior in 'startswith' / 'endswith' (2006-07-10) http://python.org/sf/1520176 closed by sf-robot Micro fix for Reference Manual (2006-08-08) http://python.org/sf/1536660 closed by gbrandl buffer comparison emits a RuntimeWarning (2006-08-08) http://python.org/sf/1536786 closed by theller Typo, types.TypeType should read types.StringType (2006-08-08) http://python.org/sf/1536828 closed by gbrandl New / Reopened RFE __________________ IndexError: Add bad index to msg (2006-08-04) http://python.org/sf/1534607 opened by Michael Kleehammer Print identical floats consistently (2006-08-05) http://python.org/sf/1534942 opened by Marc W. Abel RFE Closed __________ feature requests for logging lib (2006-04-22) http://python.org/sf/1474577 closed by sf-robot From martin at v.loewis.de Wed Aug 9 08:42:43 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 09 Aug 2006 08:42:43 +0200 Subject: [Python-Dev] openSSL and windows binaries - license In-Reply-To: <44D9503D.1040809@canterbury.ac.nz> References: <44D8E38C.8000505@v.loewis.de> <44D9503D.1040809@canterbury.ac.nz> Message-ID: <44D983E3.701@v.loewis.de> Greg Ewing schrieb: > If distributing the source doesn't violate the patent, > and distributing a binary doesn't violate the patent, > then what *would* constitute a violation of a software > patent? IANAL, but AFAICT, the rights controlled by patent law are the right to make, to use, to sell, to offer to sell, and to import. In the context of an encryption algorithm, the right to use would be the most prominent one; you wouldn't be allowed to use the algorithm unless you have a patent license. In general, the right to sell and to offer to sell would be relevant for software as well, but not so for free software (I assume). Regards, Martin From g.brandl at gmx.net Wed Aug 9 09:10:30 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 09 Aug 2006 09:10:30 +0200 Subject: [Python-Dev] 2.5 status In-Reply-To: <1f7befae0608081809v3cd63b20l313b9d6b5cd4eef@mail.gmail.com> References: <1f7befae0608081809v3cd63b20l313b9d6b5cd4eef@mail.gmail.com> Message-ID: Tim Peters wrote: > [Georg Brandl, on > http://python.org/sf/1523610 - PyArg_ParseTupleAndKeywords > potential core dump > ] >> This one's almost fixed if we can decide what to do with "levels". >> I wrote some time ago: >> >> """ >> With respect to this bug (which is about stack issues in Python/getargs.c >> involving misuse of the "levels" array), I think that we can drop the >> "levels" thing completely. It's only there to tell the user which exact item >> passed as part of a tuple argument cannot be accepted (and only if that >> function is implemented in C code). As tuple arguments >> are very rare "argument x" should be enough to tell the user that >> something's wrong with that tuple. >> """ > > More, the problem that remains is purely "a head bug": nobody ever > bumped into it, and the only way to provoke it is to write C (calling, > e.g., PyArg_ParseTupleAndKeywords) nesting tuple codes in an argument > descriptor string to an absurd depth. This is far from serious -- > heck, it's far from even interesting <0.5 wink>. I have now found a different solution: as the format code is checked for '(' anyway and the nesting level is tracked, I introduced an upper bound for it (rev. 51158). I believe that fixes the problem. Georg From martin at v.loewis.de Wed Aug 9 09:16:19 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 09 Aug 2006 09:16:19 +0200 Subject: [Python-Dev] openSSL and windows binaries - license In-Reply-To: <20060808220748.GN9899@zot.electricrain.com> References: <44D8E38C.8000505@v.loewis.de> <20060808220748.GN9899@zot.electricrain.com> Message-ID: <44D98BC3.5090802@v.loewis.de> Gregory P. Smith schrieb: > disabling/enabling a cipher in openssl that isn't commonly used and > isn't even directly exposed via any API to a python user hardly sounds > like dropping a feature to me. Strictly speaking, it is dropping a feature: a connection that can get established with 2.5b3 might not get established with 2.5c1, assuming a server that requires some IDEA-based cipher. > (any sane SSL connection will negotiate AES or 3DES > as its cipher; IDEA isn't required) Ok, I'll believe you that there isn't a practical risk associated with dropping that algorithm; I'll see what I can do. > If the release manager declares, "absolutely no changes to the windows > build process!" He didn't declare that: bug fixes are always possible. He declared "absolutely no new features without explicit permission". > Then clearly none of the changes I submitted will > make it in and neither would removing any hint of IDEA in 2.5 as > they're both too late. Those changes still need permission, as they constitute a new feature. > yeah i'd just do that if you're worried about the code being in the > binary causing a problem. See, my own worries come from the "if". Should I risk breaking somebody's application just because somebody else is worried about breaking patents? You haven't indicated whether you also worry: do you? Regards, Martin From greg at electricrain.com Wed Aug 9 09:56:21 2006 From: greg at electricrain.com (Gregory P. Smith) Date: Wed, 9 Aug 2006 00:56:21 -0700 Subject: [Python-Dev] openSSL and windows binaries - license In-Reply-To: <44D98BC3.5090802@v.loewis.de> References: <44D8E38C.8000505@v.loewis.de> <20060808220748.GN9899@zot.electricrain.com> <44D98BC3.5090802@v.loewis.de> Message-ID: <20060809075621.GO9899@zot.electricrain.com> > Strictly speaking, it is dropping a feature: a connection that can get > established with 2.5b3 might not get established with 2.5c1, assuming > a server that requires some IDEA-based cipher. > > > (any sane SSL connection will negotiate AES or 3DES > > as its cipher; IDEA isn't required) > > Ok, I'll believe you that there isn't a practical risk associated with > dropping that algorithm; I'll see what I can do. yep. nothing uses it. Plus the RFC for TLS (SSL) lists the minimum supported cipher that all implementations need as 3DES. > > yeah i'd just do that if you're worried about the code being in the > > binary causing a problem. > > See, my own worries come from the "if". Should I risk breaking > somebody's application just because somebody else is worried about > breaking patents? You haven't indicated whether you also worry: > do you? I'm not worried about breaking applications at all in this case. Many linux distributions ship with their OpenSSL (used by pretty much all software on the system including python) without IDEA support. -greg From robinbryce at gmail.com Wed Aug 9 10:09:50 2006 From: robinbryce at gmail.com (Robin Bryce) Date: Wed, 9 Aug 2006 09:09:50 +0100 Subject: [Python-Dev] 2.5b3, commit r46372 regressed PEP 302 machinery (sf not letting me post) In-Reply-To: References: Message-ID: This has an sf number now #1537167, and hopefully a clearer explanation of what I think the problem is. This is not a duplicate of the earlier "PEP 302 Fix" thread. Thanks, Robin On 07/08/06, Robin Bryce wrote: > Hi, > > Appologies for the lack of an sf#. I tried to submit this there but > couldn't. (sf is logging me out each time I visit a new page and it is > refusing my attempt to post anonymously). > > Python 2.5b3 (trunk:51136M, Aug 7 2006, 10:48:15) > [GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2 > > > The need for speed patch commited in revision r46372 included a change > whose intent was to reduce the number of open calls. The `continue` > statement at line 1283 in import.c:r51136 has the effect of skipping > the builtin import machinery if the find_module method of a custom > importer returns None. > > In Python 2.4.3, if find_module returned None the builtin machinery is > allowed to process the path tail. > > In my particular case I am working on an importer that deals with kid > templates that may or may not exist as .py[c] files. > > The short of it is that in Python 2.4.3 this produces a usable module > ``__import__('foo.a/templateuri')`` wheras in 2.5b3 I get import > error. The python 2.4.3 implementation *allows* module paths that are > not seperated with '.' Python 2.5b3 does not allow this and it does > not look like this was an intentional change. I believe this point > about 'illeagal' module paths is actualy independent of the regresion > I am asserting. Detailed session logs are attatched (following the sf > guidance even though I'm posting to py-dev) > > The 'use case' for the importer is: Robin wants to package a default > template file as normal python module and provide a custom importer > that allows users of his package to reference both: there own > templates and html files on the file system in arbitrary locations AND > the stock templates provided as python modules under the same name > space. He would like to leave normal imports to the standard > machinery. > > Cheers, > > Robin > > > From arigo at tunes.org Wed Aug 9 11:47:25 2006 From: arigo at tunes.org (Armin Rigo) Date: Wed, 9 Aug 2006 11:47:25 +0200 Subject: [Python-Dev] returning longs from __hash__() In-Reply-To: <44D8FED0.9060108@v.loewis.de> References: <20060808102608.GA9098@code0.codespeak.net> <44D8E148.5050701@v.loewis.de> <1f7befae0608081320j61bacd63l8ea8d3a9f32f0424@mail.gmail.com> <44D8FED0.9060108@v.loewis.de> Message-ID: <20060809094725.GA16199@code0.codespeak.net> Hi Martin, On Tue, Aug 08, 2006 at 11:14:56PM +0200, "Martin v. L?wis" wrote: > - h = PyInt_AsLong(res); > + if (PyLong_Check(res)) > + h = res->ob_type->tp_hash(res); This bug will keep showing up forever :-) It's unsafe against a user subclassing 'long' and overriding __hash__ of that subclass to return the object itself -- it would cause an infinite C recursion. Instead, we need either a new API like PyLong_Hash(), or call 'PyLong_Type.tp_hash(res)'. Same in the other change. A bientot, Armin From g.brandl at gmx.net Wed Aug 9 16:40:33 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 09 Aug 2006 16:40:33 +0200 Subject: [Python-Dev] Is this a bug? Message-ID: Is this considered a bug? Sure, deleting modules from sys.modules isn't quite common, but it happened to me on one occasion. Python 2.4.3 (#1, Jul 29 2006, 10:52:20) >>> import logging >>> import sys >>> del logging >>> del sys.modules['logging'] >>> ^D Error in atexit._run_exitfuncs: Traceback (most recent call last): File "/usr/lib/python2.4/atexit.py", line 24, in _run_exitfuncs func(*targs, **kargs) File "/usr/lib/python2.4/logging/__init__.py", line 1328, in shutdown for h in _handlerList[:]: # was _handlers.keys(): TypeError: unsubscriptable object Error in sys.exitfunc: Traceback (most recent call last): File "/usr/lib/python2.4/atexit.py", line 24, in _run_exitfuncs func(*targs, **kargs) File "/usr/lib/python2.4/logging/__init__.py", line 1328, in shutdown for h in _handlerList[:]: # was _handlers.keys(): TypeError: unsubscriptable object Obviously, _handlerList (as a global) is already cleaned up, which is why the subscript fails. Georg From mattjfleming at googlemail.com Wed Aug 9 17:19:33 2006 From: mattjfleming at googlemail.com (Matt Fleming) Date: Wed, 9 Aug 2006 15:19:33 +0000 Subject: [Python-Dev] Is this a bug? In-Reply-To: References: Message-ID: <5ff4a1e50608090819i3dad63dfh3eb1dfd3ebcf2230@mail.gmail.com> On 09/08/06, Georg Brandl wrote: > Is this considered a bug? Sure, deleting modules from sys.modules > isn't quite common, but it happened to me on one occasion. > > Python 2.4.3 (#1, Jul 29 2006, 10:52:20) > >>> import logging > >>> import sys > >>> del logging > >>> del sys.modules['logging'] > >>> ^D > Error in atexit._run_exitfuncs: > Traceback (most recent call last): > File "/usr/lib/python2.4/atexit.py", line 24, in _run_exitfuncs > func(*targs, **kargs) > File "/usr/lib/python2.4/logging/__init__.py", line 1328, in shutdown > for h in _handlerList[:]: # was _handlers.keys(): > TypeError: unsubscriptable object > Error in sys.exitfunc: > Traceback (most recent call last): > File "/usr/lib/python2.4/atexit.py", line 24, in _run_exitfuncs > func(*targs, **kargs) > File "/usr/lib/python2.4/logging/__init__.py", line 1328, in shutdown > for h in _handlerList[:]: # was _handlers.keys(): > TypeError: unsubscriptable object > > Obviously, _handlerList (as a global) is already cleaned up, which is why > the subscript fails. > > Georg > > _______________________________________________ Could it be considered a bug in the atexit module (or is that what you meant)? Seeing as there's no _decent_ way to recover from this error, perhaps it could just slip silently passed? -- Matt http://mattssanctuary.blogspot.com From jcarlson at uci.edu Wed Aug 9 17:59:30 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Wed, 09 Aug 2006 08:59:30 -0700 Subject: [Python-Dev] Is this a bug? In-Reply-To: References: Message-ID: <20060809084110.E71E.JCARLSON@uci.edu> Georg Brandl wrote: > > Is this considered a bug? Sure, deleting modules from sys.modules > isn't quite common, but it happened to me on one occasion. > > Python 2.4.3 (#1, Jul 29 2006, 10:52:20) > >>> import logging > >>> import sys > >>> del logging > >>> del sys.modules['logging'] > >>> ^D > Error in atexit._run_exitfuncs: > Traceback (most recent call last): > File "/usr/lib/python2.4/atexit.py", line 24, in _run_exitfuncs > func(*targs, **kargs) > File "/usr/lib/python2.4/logging/__init__.py", line 1328, in shutdown > for h in _handlerList[:]: # was _handlers.keys(): > TypeError: unsubscriptable object > Error in sys.exitfunc: > Traceback (most recent call last): > File "/usr/lib/python2.4/atexit.py", line 24, in _run_exitfuncs > func(*targs, **kargs) > File "/usr/lib/python2.4/logging/__init__.py", line 1328, in shutdown > for h in _handlerList[:]: # was _handlers.keys(): > TypeError: unsubscriptable object > > Obviously, _handlerList (as a global) is already cleaned up, which is why > the subscript fails. Interestingly enough, I recently ran into a series of errors relating to manipulating sys.modules and using imp.load_module... The imp.load_module and the 3 other simple variants involved importing a module 'x.y' without the package 'x' having been imported first. This resulted in all C-modules being re-initialized (builtins, 3rd party, etc.), causing things like sys.stdout wrappers to be removed, and in the case of wx, segfaults galore (especially during shutdown). Python 2.3 and 2.4 performed these imports silently, while 2.5 complains "SystemError: Parent module 'x' not loaded", which is actually a useful message, and helped me fix it. I have a use-case for replacing sys.modules['__main__'] with a different module, and initial attempts to do: sys.modules['__main__'] = other_module other_module.main() Resulted in: AttributeError: 'NoneType' object has no attribute 'main' This is the case for Python 2.3, 2.4, and 2.5 beta. prefixing the above two operations with: sys.modules['_oldmain'] = sys.modules['__main__'] Is sufficient to prevent Python from tearing down everything after the sys.modules['__main__'] reassignment. Not a big deal, but a sys.modules manipulation that has a gotcha. - Josiah From skip at pobox.com Wed Aug 9 18:06:25 2006 From: skip at pobox.com (skip at pobox.com) Date: Wed, 9 Aug 2006 11:06:25 -0500 Subject: [Python-Dev] Is this a bug? In-Reply-To: <5ff4a1e50608090819i3dad63dfh3eb1dfd3ebcf2230@mail.gmail.com> References: <5ff4a1e50608090819i3dad63dfh3eb1dfd3ebcf2230@mail.gmail.com> Message-ID: <17626.2049.928034.555633@montanaro.dyndns.org> Matt> Could it be considered a bug in the atexit module (or is that what Matt> you meant)? Or a bug in the logging package? Skip From jimjjewett at gmail.com Wed Aug 9 18:30:17 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed, 9 Aug 2006 12:30:17 -0400 Subject: [Python-Dev] openSSL and windows binaries - license In-Reply-To: <44D9503D.1040809@canterbury.ac.nz> References: <44D8E38C.8000505@v.loewis.de> <44D9503D.1040809@canterbury.ac.nz> Message-ID: On 8/8/06, Greg Ewing wrote: > If distributing the source doesn't violate the patent, > and distributing a binary doesn't violate the patent, > then what *would* constitute a violation of a software > patent? > Writing new code using the algorithm? Compiling > something which uses it? Any of the above, depending on the particular judge. The difference is that there is now some precedent saying that source (can be) an expression, rather than an implementation. There is no such precedent suggesting protection for anything executable. This still doesn't mean that (writing or distributing) source code is automatically safe -- but in practice, it means lawsuits would be filed first against those using or distributing a binary. Since the PSF does not have deep pockets, plaintiffs would prefer not to involve them. If the actual defendants had made an affirmative choice to violate (by changing their own build instructions), other BigCorps might not blame python for the risk. "Martin v. L?wis" > IANAL, but AFAICT, the rights controlled by patent law > are the right to make, to use, to sell, to offer to sell, > and to import. ... In general, the right to sell and to offer to > sell would be relevant for software as well, but not > so for free software (I assume). Giving it away interferes with the patent owner's ability to sell it, and is therefore considered interference with the exclusive right to sell. -jJ From ndbecker2 at gmail.com Wed Aug 9 18:35:13 2006 From: ndbecker2 at gmail.com (Neal Becker) Date: Wed, 09 Aug 2006 12:35:13 -0400 Subject: [Python-Dev] SyntaxError: can't assign to function call Message-ID: class X (object): pass X() += 2 > SyntaxError: can't assign to function call Suppose I actually had defined __iadd__ for class X. Python says this syntax is invalid. I wish is wasn't. Here's where I might use it. Suppose I have a container class. Suppose I could make a slice of this container, like so: c = Container() s = Slice (c, 0, 10) where "Slice" is a class. Then: s += 2 Would perform iadd on the slice of the container. A convenient syntax would be: Slice (c, 0, 10) += 2 Does anyone else think this would be a good addition to Python? From jcarlson at uci.edu Wed Aug 9 19:13:53 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Wed, 09 Aug 2006 10:13:53 -0700 Subject: [Python-Dev] SyntaxError: can't assign to function call In-Reply-To: References: Message-ID: <20060809101243.E721.JCARLSON@uci.edu> Neal Becker wrote: > > class X (object): > pass > > X() += 2 > > > SyntaxError: can't assign to function call [snip] > Does anyone else think this would be a good addition to Python? No. += implies assignment. As the syntax error states, "can't assign to function call". -1 - Josiah From tjreedy at udel.edu Wed Aug 9 19:41:32 2006 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 9 Aug 2006 13:41:32 -0400 Subject: [Python-Dev] SyntaxError: can't assign to function call References: Message-ID: "Neal Becker" wrote in message news:ebd2sp$nnv$1 at sea.gmane.org... > class X (object): > pass > > X() += 2 > >> SyntaxError: can't assign to function call > > Suppose I actually had defined __iadd__ for class X. Python says this > syntax is invalid. I wish is wasn't. If you translate to x() = x() +2, with x called just once, it does not make sense. You can bind to names and slots in compound objects, but not to objects. > Here's where I might use it. Suppose I have a container class. Suppose > I > could make a slice of this container, like so: > > c = Container() > s = Slice (c, 0, 10) > > where "Slice" is a class. Then: > > s += 2 > > Would perform iadd on the slice of the container. A convenient syntax > would > be: > > Slice (c, 0, 10) += 2 If this were made to work, the unbound object you are making an 'assignment' to, purely for side effect, would disappear. So if you don't want a separate object to be named and kept around, give your container a sliceadd method: c.sliceadd((0,10), 2) > > Does anyone else think this would be a good addition to Python? I don't. Augmented assigment is confusing enough to many. Terry Jan Reedy From ndbecker2 at gmail.com Wed Aug 9 20:10:15 2006 From: ndbecker2 at gmail.com (Neal Becker) Date: Wed, 09 Aug 2006 14:10:15 -0400 Subject: [Python-Dev] SyntaxError: can't assign to function call References: Message-ID: Terry Reedy wrote: > > "Neal Becker" wrote in message > news:ebd2sp$nnv$1 at sea.gmane.org... >> class X (object): >> pass >> >> X() += 2 >> >>> SyntaxError: can't assign to function call >> >> Suppose I actually had defined __iadd__ for class X. Python says this >> syntax is invalid. I wish is wasn't. > > If you translate to x() = x() +2, with x called just once, it does not > make > sense. You can bind to names and slots in compound objects, but not to > objects. > >> Here's where I might use it. Suppose I have a container class. Suppose >> I >> could make a slice of this container, like so: >> >> c = Container() >> s = Slice (c, 0, 10) >> >> where "Slice" is a class. Then: >> >> s += 2 >> >> Would perform iadd on the slice of the container. A convenient syntax >> would >> be: >> >> Slice (c, 0, 10) += 2 > > If this were made to work, the unbound object you are making an > 'assignment' to, purely for side effect, would disappear. So if you don't > want a separate object to be named and kept around, give your container a > sliceadd method: > c.sliceadd((0,10), 2) > >> >> Does anyone else think this would be a good addition to Python? > > I don't. Augmented assigment is confusing enough to many. > I guess I confused things with the augmented assignment. It's not a question specific to augmented assignment. There are 2 questions: 1) Should assignment to a temporary object be allowed? 2) Should the syntax for creation of a temporary object be a constructor call? From mwh at python.net Wed Aug 9 22:19:40 2006 From: mwh at python.net (Michael Hudson) Date: Wed, 09 Aug 2006 21:19:40 +0100 Subject: [Python-Dev] SyntaxError: can't assign to function call In-Reply-To: (Neal Becker's message of "Wed, 09 Aug 2006 14:10:15 -0400") References: Message-ID: <2m4pwlvezn.fsf@starship.python.net> Neal Becker writes: > 1) Should assignment to a temporary object be allowed? The question doesn't make sense: in Python, you assign to a name, an attribute or a subscript, and that's it. Cheers, mwh -- I think there's a rather large difference between a stale twinkie and a kernel swap daemon -- from Twisted.Quotes From g.brandl at gmx.net Wed Aug 9 22:33:21 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 09 Aug 2006 22:33:21 +0200 Subject: [Python-Dev] SyntaxError: can't assign to function call In-Reply-To: References: Message-ID: Terry Reedy wrote: > "Neal Becker" wrote in message > news:ebd2sp$nnv$1 at sea.gmane.org... >> class X (object): >> pass >> >> X() += 2 >> >>> SyntaxError: can't assign to function call >> >> Suppose I actually had defined __iadd__ for class X. Python says this >> syntax is invalid. I wish is wasn't. > > If you translate to x() = x() +2, with x called just once, it does not make > sense. You can bind to names and slots in compound objects, but not to > objects. This is similar to x = ([1], 2) x[0] += [2] which doesn't currently work either, though it could. Georg From jjl at pobox.com Wed Aug 9 22:40:25 2006 From: jjl at pobox.com (John J Lee) Date: Wed, 9 Aug 2006 20:40:25 +0000 (UTC) Subject: [Python-Dev] httplib and bad response chunking In-Reply-To: <20060731020902.GA2950@cthulhu.gerg.ca> References: <20060726023213.GA3106@cthulhu.gerg.ca> <20060731020902.GA2950@cthulhu.gerg.ca> Message-ID: On Sun, 30 Jul 2006, Greg Ward wrote: [...] > Did you look at the crude attempt at testing for this bug that I hacked > into test_httplib.py? I posted it to bug #1486335 here: > > http://sourceforge.net/tracker/download.php?group_id=5470&atid=105470&file_id=186245&aid=1486335 > > The idea is simple: put various chunked responses into strings and then > feed those strings to HTTPConnection. The trouble is that StringIO does > not behave the same as a real socket: where HTTPResponse fails one way > reading from a real socket (eg. infinite loop), it fails differently (or > not at all) reading from a StringIO. Makes testing with the FakeSocket > class in test_httplib.py problematic. There are always ways around unit testing problems, but... > Maybe the right way to test httplib is to spawn a server process > (thread?) to listen on some random port, feed various HTTP responses at > HTTPConnection/HTTPResponse, and see what happens. I'm not sure how to > do that portably, though. Well, I'll see if I can whip up a Unix-y > solution and see if anyone knows how to make it portable. I think adding this kind of functional test is extremely valuable, given that we don't seem to have any for httplib at present (Lib/test/test_httplib.py does not send any network packets). Maybe you could add a basic smoke test that does a simple successful GET, while you're going about adding your chunked encoding regression test(s)? Oh, wait: there is a functional test that uses the network, but it's at the end of httplib.py rather than being part of the test suite, and it follows the "Guru Watches Output" pattern :-) I tried to add a test for urllib2 recently that relied on a python.org URL being set up, but the python.org guys are pretty overworked already and haven't had time to enable that yet -- so I think that simply from that point of view your run-your-own-server approach is better. Why does it need to be unix-y, though? We have SimpleHTTPServer. We're not trying to test the OS, so I don't see a problem with using loopback and a single process -- and that would keep test run times down. Um, except that MS OSes seem a bit odd re the loopback interface -- ISTR that, at least back with NT4, you just didn't get a loopback i/face unless you had an ethernet driver installed (back then, I was on dialup). More unit tests would also be valuable, of course. John From guido at python.org Wed Aug 9 22:41:07 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 9 Aug 2006 13:41:07 -0700 Subject: [Python-Dev] SyntaxError: can't assign to function call In-Reply-To: References: Message-ID: On 8/9/06, Georg Brandl wrote: > Terry Reedy wrote: > > "Neal Becker" wrote in message > > news:ebd2sp$nnv$1 at sea.gmane.org... > >> class X (object): > >> pass > >> > >> X() += 2 > >> > >>> SyntaxError: can't assign to function call > >> > >> Suppose I actually had defined __iadd__ for class X. Python says this > >> syntax is invalid. I wish is wasn't. > > > > If you translate to x() = x() +2, with x called just once, it does not make > > sense. You can bind to names and slots in compound objects, but not to > > objects. > > This is similar to > > x = ([1], 2) > x[0] += [2] > > which doesn't currently work either, though it could. No it couldn't. You can't assign to x[0]. L += R is defined as L = L.__iadd__(R) so L must be a valid assignment target. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From g.brandl at gmx.net Wed Aug 9 22:55:46 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 09 Aug 2006 22:55:46 +0200 Subject: [Python-Dev] SyntaxError: can't assign to function call In-Reply-To: References: Message-ID: Guido van Rossum wrote: >> This is similar to >> >> x = ([1], 2) >> x[0] += [2] >> >> which doesn't currently work either, though it could. > > No it couldn't. You can't assign to x[0]. L += R is defined as L = > L.__iadd__(R) so L must be a valid assignment target. > Thanks for making that clear. I actually had filed a bug about that behavior (http://python.org/sf/1436226), and from your comment to my proposed patch I inferred that the report was valid, only the patch wasn't. Cheers, Georg From guido at python.org Wed Aug 9 23:30:04 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 9 Aug 2006 14:30:04 -0700 Subject: [Python-Dev] Dict suppressing exceptions Message-ID: I've been happily ignoring python-dev for the last three weeks or so, and Neal just pointed me to some thorny issues that are close to resolution but not quite yet resolved, yet need to be before beta 3 on August 18 (Friday next week). Here's my take on the dict-suppressing-exceptions issue (I'll send out separate messages for each issue where Neal has asked me to weigh in). It wasn't my idea to stop ignoring exceptions in dict lookups; I would gladly have put this off until Py3k, where the main problem (str-unicode __eq__ raising UnicodeError) will go away. But since people are adamant that they want this in sooner, I suggest that to minimize breakage we could make an exception for str-unicode comparisons. I came up with the following code to reproduce the issue; this prints 0 in 2.2, False in 2.3 and 2.4, but raises UnicodeDecodeError in 2.5 (head): a = {u"1": 1} x = hash(u"1") class C(str): def __hash__(s): return x print C("\xff") in a The following patch makes this print False in 2.5 again. Notes about the patch: - this also fixes an out-of-date comment that should be fixed even if the rest of the idea is rejected (lookdict_string() can return NULL when it calls lookdict) - the exception could be narrowed even further by only suppressing the exception when startkey and key are both either str or unicode instances. What do people think? --- Objects/dictobject.c (revision 51180) +++ Objects/dictobject.c (working copy) @@ -230,7 +230,8 @@ lookdict() is general-purpose, and may return NULL if (and only if) a comparison raises an exception (this was new in Python 2.5). lookdict_string() below is specialized to string keys, comparison of which can -never raise an exception; that function can never return NULL. For both, when +never raise an exception; that function can never return NULL (except when it +decides to replace itself with the more general lookdict()). For both, when the key isn't found a dictentry* is returned for which the me_value field is NULL; this is the slot in the dict at which the key would have been found, and the caller can (if it wishes) add the pair to the returned @@ -259,8 +260,13 @@ if (ep->me_hash == hash) { startkey = ep->me_key; cmp = PyObject_RichCompareBool(startkey, key, Py_EQ); - if (cmp < 0) - return NULL; + if (cmp < 0) { + if (PyErr_Occurred()==PyExc_UnicodeDecodeError) { + PyErr_Clear(); + } + else + return NULL; + } if (ep0 == mp->ma_table && ep->me_key == startkey) { if (cmp > 0) return ep; @@ -289,8 +295,13 @@ if (ep->me_hash == hash && ep->me_key != dummy) { startkey = ep->me_key; cmp = PyObject_RichCompareBool(startkey, key, Py_EQ); - if (cmp < 0) - return NULL; + if (cmp < 0) { + if (PyErr_Occurred()==PyExc_UnicodeDecodeError) { + PyErr_Clear(); + } + else + return NULL; + } if (ep0 == mp->ma_table && ep->me_key == startkey) { if (cmp > 0) return ep; -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Aug 9 23:46:09 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 9 Aug 2006 14:46:09 -0700 Subject: [Python-Dev] __index__ clipping Message-ID: Here's another issue where Neal thought it would be useful if I weighed in. I'm not quite sure of the current status, but perhaps the following would work? - Called from Python, (10**10).__index__() should return 10000000000L, not raise an exception or return sys.maxint. - The nb_index slot is changed to return an object; it should return a Python int or long without clipping (same as __index__() called from Python). - There should be three C APIs (perhaps fewer if some of them are never needed): - One to call the nb_index slot, or raise an exception if it's not set, returning the object - One to return the clipped value without an exception (can still return an exception if something else went wrong) - One to return the value if it fits, raise OverflowError if it doesn't I know this is quite the deviation from the current code but it seems the most rational and future-proof approach. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Aug 9 23:58:39 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 9 Aug 2006 14:58:39 -0700 Subject: [Python-Dev] cgi DoS attack Message-ID: This is regarding #1112549. I think this can go in. It should also be backported to 2.4 and to 2.3 (if we ever release another one of that). I reviewed the code and added some minor comments to the SF tracker. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jimjjewett at gmail.com Thu Aug 10 02:46:54 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed, 9 Aug 2006 20:46:54 -0400 Subject: [Python-Dev] Dict suppressing exceptions Message-ID: > It wasn't my idea to stop ignoring exceptions in dict lookups; I would > gladly have put this off until Py3k, where the main problem > (str-unicode __eq__ raising UnicodeError) will go away. > But since people are adamant that they want this in sooner, Is this true for dictionaries specifically? Would there really be strong objections to continuing to swallow any Exception (not BaseException) raised by __eq__ ? Writing an __eq__ method that doesn't handle generic objects is sloppy, but it does happen, and I would hate to wrap all dictionaries of objects. -jJ From greg.ewing at canterbury.ac.nz Thu Aug 10 03:34:59 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 10 Aug 2006 13:34:59 +1200 Subject: [Python-Dev] openSSL and windows binaries - license In-Reply-To: <44D983E3.701@v.loewis.de> References: <44D8E38C.8000505@v.loewis.de> <44D9503D.1040809@canterbury.ac.nz> <44D983E3.701@v.loewis.de> Message-ID: <44DA8D43.4040009@canterbury.ac.nz> Martin v. L?wis wrote: > In the context of an encryption algorithm, the right to > use would be the most prominent one; you wouldn't be > allowed to use the algorithm unless you have a patent > license. But what does "use" *mean* in relation to an algorithm? -- Greg From oliphant.travis at ieee.org Thu Aug 10 04:46:11 2006 From: oliphant.travis at ieee.org (Travis E. Oliphant) Date: Wed, 09 Aug 2006 20:46:11 -0600 Subject: [Python-Dev] __index__ clipping In-Reply-To: References: Message-ID: Guido van Rossum wrote: > Here's another issue where Neal thought it would be useful if I > weighed in. I'm not quite sure of the current status, but perhaps the > following would work? > > - Called from Python, (10**10).__index__() should return 10000000000L, > not raise an exception or return sys.maxint. > > - The nb_index slot is changed to return an object; it should return a > Python int or long without clipping (same as __index__() called from > Python). I don't like this approach because nb_int and nb_long are already available to convert an object into an int or a long, so the only value of nb_index is to make sure that floats aren't able to be converted this way. The value of nb_index is that it provides a "small-overhead" means to quickly convert any object that allows it into a Py_ssize_t value which is needed in many internal situations (mostly indexing related). Rather than assume a Python integer is the only candidate for conversion to a Py_ssize_t value, nb_index allows other objects to be used in that fashion. Having another level of indirection where one object is first converted to a Python int or Python long (whichever is needed) and then to a Py_ssize_t value seems like un-necessary waste to me and will needlessly slow certain indexing operations down. It seems like Nick's recent patches solved the problems that were identified. -Travis > > - There should be three C APIs (perhaps fewer if some of them are never needed): > > - One to call the nb_index slot, or raise an exception if it's not > set, returning the object > > - One to return the clipped value without an exception > (can still return an exception if something else went wrong) > > - One to return the value if it fits, raise OverflowError if it doesn't > > I know this is quite the deviation from the current code but it seems > the most rational and future-proof approach. > From guido at python.org Thu Aug 10 06:08:54 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 9 Aug 2006 21:08:54 -0700 Subject: [Python-Dev] __index__ clipping In-Reply-To: References: Message-ID: On 8/9/06, Travis E. Oliphant wrote: > Guido van Rossum wrote: > > Here's another issue where Neal thought it would be useful if I > > weighed in. I'm not quite sure of the current status, but perhaps the > > following would work? > > > > - Called from Python, (10**10).__index__() should return 10000000000L, > > not raise an exception or return sys.maxint. > > > > - The nb_index slot is changed to return an object; it should return a > > Python int or long without clipping (same as __index__() called from > > Python). > > I don't like this approach because nb_int and nb_long are already > available to convert an object into an int or a long, so the only value > of nb_index is to make sure that floats aren't able to be converted this > way. > > The value of nb_index is that it provides a "small-overhead" means to > quickly convert any object that allows it into a Py_ssize_t value which > is needed in many internal situations (mostly indexing related). > > Rather than assume a Python integer is the only candidate for conversion > to a Py_ssize_t value, nb_index allows other objects to be used in that > fashion. Having another level of indirection where one object is first > converted to a Python int or Python long (whichever is needed) and then > to a Py_ssize_t value seems like un-necessary waste to me and will > needlessly slow certain indexing operations down. Not if the object is implemented in Python, since then the value is necessarily returned as an int or long object anyway. Is performance of __index__ really that important? What do you think (10**10).__index__() should return (when called from Python)? > It seems like Nick's recent patches solved the problems that were > identified. Nick, can you summarize how your patches differ from my proposal? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From nnorwitz at gmail.com Thu Aug 10 06:53:19 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 9 Aug 2006 21:53:19 -0700 Subject: [Python-Dev] Is this a bug? In-Reply-To: <20060809084110.E71E.JCARLSON@uci.edu> References: <20060809084110.E71E.JCARLSON@uci.edu> Message-ID: On 8/9/06, Josiah Carlson wrote: > 2.4 performed these imports silently, while 2.5 complains "SystemError: > Parent module 'x' not loaded", which is actually a useful message, and > helped me fix it. Can you make a small, self-contained test case? The SystemError should be a normal exception and might indicate a deeper problem. Thanks, n From steven.bethard at gmail.com Thu Aug 10 07:21:47 2006 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed, 9 Aug 2006 23:21:47 -0600 Subject: [Python-Dev] DRAFT: python-dev summary for 2006-07-01 to 2006-07-15 Message-ID: Here's the summary for the first half of July. Corrections and comments are greatly apprecieated. Particularly for the section on restricted execution (though my understanding is that the current plans are somewhat different than what was discussed during the first half of July). ============= Announcements ============= ------------------- Python 2.5 schedule ------------------- Python continues to make progress towards Python 2.5 final. See `PEP 356`_ for more details and the full schedule. .. _PEP 356: http://www.python.org/dev/peps/pep-0356/ Contributing threads: - `TRUNK FREEZE for 2.5b2, tomorrow Tuesday 11th, 00:00 UTC `__ - `Subject: RELEASED Python 2.5 (beta 2) `__ - `TRUNK is UNFROZEN. `__ -------------------------- ctypes in the Python trunk -------------------------- Martin v. L?wis successfully imported the ctypes history into the Python trunk, so ctypes now shares its repository with the rest of Python. Thanks everyone for your hard work! Contributing threads: - `Moving the ctypes repository to python.org `__ - `Subversion outage Friday 15:00 GMT `__ - `SVN write access is back `__ - `Extended Subversion outage: Friday 16:40 GMT `__ ========= Summaries ========= ------------------------- Rebinding non-local names ------------------------- A bug in the Python trunk that allowed augmented assignment to rebind names in outer scopes initiated a new discussion about how to allow Python code to rebind non-local names, e.g.:: def counter(n=0): def increment(i=1): n += i # change the n in counter return n return increment Guido definitely didn't like the idea of introducing a new rebinding operator, e.g. ``:=`` or ``.=``. Phillip J. Eby suggested that a dot prefix to the name could indicate rebinding, e.g.:: def counter(n=0): def increment(i=1): .n += i return .n return increment Guido felt uncomfortable with this proposal because it would mean that in referencing a value the only difference between ``.NAME`` and ``NAME`` was whether or not the name was assigned to at another place in the same function. This ambiguity could be avoided by requiring all references to names in outer scopes to be prefixed by the dot, but that seemed like too large of a change for Python. There was also some discussion of trying to reuse the global keyword, but most folks felt that ``outer`` or ``nonlocal`` was a better name:: def counter(n=0): def increment(i=1): nonlocal n n += i return n return increment Guido requested a PEP on the subject, but at the time of this summary, none was available. Contributing threads: - `2.5 and beyond `__ - `Lexical scoping in Python 3k `__ - `Explicit Lexical Scoping (pre-PEP?) `__ -------------------------------------- Python and backwards incompatibilities -------------------------------------- A `complaint by Greg Black`_ about Python introducing backward incompatibilities started a long discussion about how Python developers could stay in better touch with existing user code. Greg Black's code had relied on undocumented behavior in time.strftime() that had allowed zeros to be passed in as default values. When that behavior changed in Python 2.4, Greg's code (along with a number of other people's code) was broken. Python core didn't have any unittests for the behavior, so no tests started failing when the change was made. Glyph suggested adding a mechanism so that user projects could submit buildslaves to run their test suites when changes are made to Python. The goal here would be to identify any de facto coding standards that weren't tested for in Python's test suite. If a change to Python caused major projects like Twisted or Zope to start failing dramatically, that change should be reconsidered even if it didn't break any documented behavior. People generally thought this was a great idea, particularly since it might catch some of these changes earlier than alpha or beta releases. There was also some concern that the Python 2.5 release cycle had been sped up too much and users hadn't had enough time to report errors. However, adding more beta releases (which seemed to be the major suggestion) also adds more work for release managers and requires more schedule coordination so that all the release managers can be available. People also had the mistaken impression that the trunk of Python was not so stable, particularly because they had assumed that non-green buildbots meant an unusable Python. In general, non-green buildbots typically mean that some part of the test suite is failing, not that the interpreter could not be built. A.M. Kuchling suggested adding some of the `checkin policies discussion`_ to the release announcements to make this more obvious. .. _complaint by Greg Black: http://www.gbch.net/gjb/blog/software/discuss/python-sucks.html .. _checkin policies discussion: http://www.python.org/dev/tools/ Contributing threads: - `User's complaints `__ - `changing time.strftime() to accept 0s (was: User's complaints) `__ - `Community buildbots (was Re: User's complaints) `__ - `Community buildbots `__ ------------------------------ Restricted execution in Python ------------------------------ Discussion continued on Brett Cannon's PhD thesis aimed at adding restricted execution to Python. Instead of trying to cripple objects like ``file``, a number of people suggested making ``file()`` and ``open()`` return different objects in restricted mode. This would require separating out some capabilities, so that, for example, a file-like object could be returned that didn't support writing. Michael Chermside suggested implementing a new type in C that stores its data privately, and requires the user to provide access-checking functions in order to make the private data visible. That way you could implement new access restrictions from pure Python code, by simply creating appropriate instances of the new type. Ka-Ping Yee asked about protecting one piece of Python code from another, but Guido and Brett suggested that the virtual machine probably couldn't provide such barriers. Brett's current progress is available in the bcannon-sandboxing branch. Contributing threads: - `For sandboxing: alternative to crippling file() `__ - `doc for new restricted execution design for Python `__ - `branch for sandox work created: bcannon-sandboxing `__ - `In defense of Capabilities [was: doc for new restricted execution design for Python] `__ - `what can we do to hide the 'file' type? `__ - `Restricted execution: what's the threat model? `__ - `Capabilities / Restricted Execution `__ - `second draft of sandboxing design doc `__ --------------------------------- Getting the current thread frames --------------------------------- Tim Peters snuck in a late feature, ``sys._current_frames()`` which returns a dict mapping each thread's id to that thread's current Python frame. Some people wanted to expose ``head_mutex`` instead to avoid introducing a new function, but that may not exist, depending on the build type. Given that Zope had been trying to emulate something like this for a while, it's impossible to do correctly in an extension module because the appropriate internals are not available, and it's essential for debugging deadlocks, the feature was eventually approved and checked in. Contributing thread: - `"Missing" 2.5 feature `__ -------------------- Time-out in URL Open -------------------- Facundo Batista asked about adding a timeout argument to ``urllib2.urlopen()``. People generally liked the idea, but since it would be a new feature, it would have to wait until Python 2.6. There was an `existing patch` modifying httplib, ftplib, telnetllib, poplib and smtplib in this way, but it needed updating for Python 2.6. .. _existing patch: http://bugs.python.org/723312 Contributing thread: - `Time-out in URL Open `__ ------------------------------------------- zlib module build failure on Mac OSX 10.4.7 ------------------------------------------- Skip Montanaro had some trouble building the zlib module from Python trunk on Mac OSX. Turns out he had an old static libz.a on the path, and that was getting found instead of the newer dynamic library. Ronald Oussoren checked in a patch to configure that should correctly set HAVE_ZLIB_COPY even if there is an old static library sitting around. At the same time, he suggested that maybe '-Wl,-search_path_first' should be added to the default LDFLAGS on OSX so that the OSX linker doesn't look for a dylib anywhere on the path before looking for a static library. Contributing thread: - `zlib module build failure on Mac OSX 10.4.7 `__ -------------------------------- Command line args in Python 3000 -------------------------------- Greg Ewing suggested that ``sys.argv`` could be split into the program name (``sys.argv[0]``) and the arguments (``sys.argv[1:]``). People liked this idea, and pointed out that it would be handy when there's no real program name, e.g. when exec is used with an inode number as is possible on some Unices. This was also accompanied by a discussion about splitting sys into more coherent subsets, e.g. immutable and mutable objects. Guido asked for people to hold off on this discussion until Brett had a clearer idea what pieces would need to be split off for restricted execution. Contributing thread: - `Handling of sys.args (Re: User's complaints) `__ -------------------------- Import semantics in Jython -------------------------- Jython had been following the Java convention that sub-packages were imported when their parent package was imported. Guido had suggested that at least Python packages in Jython should import with the standard Python semantics. During this fortnight, Guido checked back in with the Jython folks to see if they'd made their mind up about this. Frank Wierzbicki, the new maintainer for Jython, said that he won't have a chance to fix this until after Jython 2.3 is out, but both he and Samuele Pedroni suggested that it was a change that needed to be made. Contributing thread: - `Import semantics `__ ----------------------------------------------- ImportWarnings for directories without __init__ ----------------------------------------------- After the substantial discussion last fortnight on whether or not warnings should be issued if a directory was not imported because it was missing an __init__.py file, Anthony Baxter suggested that the best course of action was to suppress the ImportWarning by default, and allow users to unsuppress it in their sitecustomize.py file. Contributing threads: - `ImportWarning flood `__ - `ImportWarning decision `__ ----------------------------------- Putting doctest code into footnotes ----------------------------------- Benji York posted a `patch for doctest`_ that teaches doctest about ReST-style footnotes so that you can write documentation like:: After initializing the system [#init]_ it is possible to retrieve status information: >>> system.status() 'good to go' [snip some of the doctest] .. [#init] Initialize the system: >>> system = System() >>> system.init() People liked the idea, and Benji promised to update the patch for the 2.5 version of doctest. .. _patch for doctest: http://tinyurl.com/nekam Contributing thread: - `Doctest and Footnotes `__ ------------------------------- Using urllib.quote with unicode ------------------------------- Stefan Rank pointed out that ``urllib.quote()`` fails with a strange KeyError on unicode stings. He suggested either raising a TypeError or automatically encoding to UTF-8 as suggested in http://www.w3.org/International/O-URL-code.html. John J Lee pointed questioned whether this was generally accepted as the Right Thing, given `RFC 2617`_, `RFC 2616`_ and the recent discussion about a `uriparse module`_. In the end, people seemed to agree that the safest thing would be to raise an exception. .. _RFC 2617: http://www.ietf.org/rfc/rfc2617.txt .. _RFC 2616: http://www.ietf.org/rfc/rfc2616.txt .. _uriparse module: http://www.python.org/dev/summary/2006-06-01_2006-06-15/#rfc-3986-uniform-resource-identifiers-uris Contributing thread: - `urllib.quote and unicode bug resuscitation attempt `__ ----------------------------------- Borderline cases for ints and longs ----------------------------------- Neil Schemenauer asked if -2147483648 (``-sys.maxint - 1``)should be an int or a long. In Python 2.4 it was an int, but in the trunk, it was a long. Tim Peters explained that -2147483648 is not actually an int literal -- it is a long literal with a unary minus. Nonetheless, practicality beat purity, and Neal Norwitz and Neil Schemenauer put together a fix to make it an int again. You can still defeat the patch with something like ``eval('-(%s)' % str(-sys.maxint - 1)[1:])``, but no one seemed too worried about that. Contributing thread: - `Unary minus bug `__ --------------------------------- Adding list.get() and tuple.get() --------------------------------- Russell E. Owen asked about adding a ``.get()`` method to lists and tuples that would return a default value if the index was out of range. Most people seemed to think that wanting such a method was a bad code smell and Raymond Hettinger suggested that it could be replaced in Python 2.5 with a simple ``seq[i] if len(seq) > i else default``. Contributing thread: - `get for lists and tuples? `__ ----------------------------- Adding a __dir__ magic method ----------------------------- Tomer Filiba suggested adding a ``__dir__()`` magic method that would be called by ``dir()``. The default one on ``object`` would do the normal search through ``__dict__`` and the superclasses, and subclasses of ``object`` could override this to add attributes that are harder to find, e.g. pseudo-attributed implemented through ``__getattr__``. Everyone liked the idea and Guido said it could be added for Python 2.6. Contributing thread: - `introducing __dir__? `__ ----------------------------------------------------- pydoc support for attributes defined with PyGetSetDef ----------------------------------------------------- Barry Warsaw pointed out that pydoc couldn't handle attributes defined with PyGetSetDef because there was no corresponding type in the types module. He found a similar problem with "member_descriptor" types like ``datetime.timedelta.days``. The latter was particularly a problem because the datetime module was not importable in the types module because the types module is imported before the platform-specific extension module directory is on sys.path. He suggested a `fix for pydoc`_ that would introduce a _types module coded in C that could make the appropriate types available. .. _fix for pydoc: http://bugs.python.org/1520294 Contributing thread: - `Support for PyGetSetDefs in pydoc `__ ================ Deferred Threads ================ - `Python Style Sheets ? Re: User's complaints `__ ================== Previous Summaries ================== - `Cleanup of test harness for Python `__ - `PEP 328 and PEP 338, redux `__ - `Empty Subscript PEP on Wiki - keep or toss? `__ - `More Switch: Explicit freezing `__ - `Proposal to eliminate PySet_Fini `__ - `Switch and static, redux `__ =============== Skipped Threads =============== - `traceback regression `__ - `sys.settrace() in Python 2.3 vs. 2.4 `__ - `Bug in stringobject? `__ - `weakattr `__ - `how long to wait for expat to incorporate a fix to prevent a crasher? `__ - `LOAD_CONST POP_TOP `__ - `Another 2.5 bug candidate? `__ - `DRAFT: python-dev summary for 2006-06-01 to 2006-06-15 `__ - `Proposed beta 2 changes (Q for Anthony/Neal) `__ - `2.5b1 Windows install `__ - `Patch for commands.py to provide callback `__ - `import screwiness `__ - `zipfile.ZipFile('foo.zip', 'a'): file not found -> create? `__ - `Musings on concurrency and scoping ("replacing" Javascript) `__ - `About a month until PSF call for test trackers closes! `__ - `test_ctypes failure on Mac OS X/PowerPC 10.3.9 (Panther) `__ - `[slighly OT] Native speakers and hurting brains `__ - `exception too expensive? `__ - `Weekly Python Patch/Bug Summary `__ - `xml issue in 2.5 `__ - `Fix for Lib/test/leakers/test_gestalt.py `__ - `Discussion on Lib/test/crashers/ `__ - `Add new PyErr_WarnEx() to 2.5? `__ - `Klocwork analysis of source if we want it `__ - `easy_install `__ - `subprocess.CalledProcessError.errno (#1223937) `__ - `Minor: Unix icons for 2.5? `__ - `PEP 356: python.org/sf/1515343 resolution `__ - `Autoloading? (Making Queue.Queue easier to use) `__ - `Long options support `__ - `Behavior change in subprocess.py `__ - `Proposal: Add Sudoku Solver To The "this" Module `__ - `The buffer() function `__ - `Partial support for dlmodule.c in 64-bits OSes `__ - `IDLE - firewall warning `__ From jcarlson at uci.edu Thu Aug 10 08:41:07 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Wed, 09 Aug 2006 23:41:07 -0700 Subject: [Python-Dev] Is this a bug? In-Reply-To: References: <20060809084110.E71E.JCARLSON@uci.edu> Message-ID: <20060809233637.1911.JCARLSON@uci.edu> "Neal Norwitz" wrote: > > On 8/9/06, Josiah Carlson wrote: > > 2.4 performed these imports silently, while 2.5 complains "SystemError: > > Parent module 'x' not loaded", which is actually a useful message, and > > helped me fix it. > > Can you make a small, self-contained test case? The SystemError > should be a normal exception and might indicate a deeper problem. Create a module z.py whose contents are: #z.py import sys import imp sys.stdout = None try: y = imp.load_source('x.y', 'x/y.py', open('x/y.py')) except: raise else: print "printing should raise an AttributeError, but doesn't" #end of listing for z.py Create a module x/y.py whose contents are: (x/__init__.py not required) #x/y.py import sys #end of listing for x/y.py Running z.py in Python 2.3 and 2.4 produces: printing should raise an AttributeError, but doesn't Running z.py in Python 2.5b2 produces: Traceback (most recent call last): File "test.py", line 7, in y = imp.load_source('x.y', 'x/y.py', open('x/y.py')) File "x/y.py", line 1, in import sys SystemError: Parent module 'x' not loaded - Josiah From martin at v.loewis.de Thu Aug 10 09:37:08 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 10 Aug 2006 09:37:08 +0200 Subject: [Python-Dev] openSSL and windows binaries - license In-Reply-To: <44DA8D43.4040009@canterbury.ac.nz> References: <44D8E38C.8000505@v.loewis.de> <44D9503D.1040809@canterbury.ac.nz> <44D983E3.701@v.loewis.de> <44DA8D43.4040009@canterbury.ac.nz> Message-ID: <44DAE224.50705@v.loewis.de> Greg Ewing schrieb: >> In the context of an encryption algorithm, the right to >> use would be the most prominent one; you wouldn't be >> allowed to use the algorithm unless you have a patent >> license. > > But what does "use" *mean* in relation to an > algorithm? Perform it: do the steps that the algorithm says you should do, or let a machine do it. IOW, run the code. If your point is that software patents are evil: I fully agree. Regards, Martin From mwh at python.net Thu Aug 10 10:23:48 2006 From: mwh at python.net (Michael Hudson) Date: Thu, 10 Aug 2006 09:23:48 +0100 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: (Jim Jewett's message of "Wed, 9 Aug 2006 20:46:54 -0400") References: Message-ID: <2mwt9ht2wb.fsf@starship.python.net> "Jim Jewett" writes: >> It wasn't my idea to stop ignoring exceptions in dict lookups; I would >> gladly have put this off until Py3k, where the main problem >> (str-unicode __eq__ raising UnicodeError) will go away. > >> But since people are adamant that they want this in sooner, > > Is this true for dictionaries specifically? > > Would there really be strong objections to continuing to swallow > any Exception (not BaseException) raised by __eq__ ? Armin's reason for changing dictionaries in this way was that enormous debugging pain was caused by dicts swallowing exceptions raised by __eq__ methods. Having the __eq__ methods swallow the exceptions by themselves would make the situation *worse*, not better. Cheers, mwh -- * vegai wears his reading bra. umm, I mean glasses -- from Twisted.Quotes From martin at v.loewis.de Thu Aug 10 10:35:18 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 10 Aug 2006 10:35:18 +0200 Subject: [Python-Dev] SyntaxError: can't assign to function call In-Reply-To: References: Message-ID: <44DAEFC6.8000807@v.loewis.de> Neal Becker schrieb: > 1) Should assignment to a temporary object be allowed? Unlike Michael, I think the question does make sense, and the answer is "no", for the reason Michael gave: you assign to names, not to objects. > 2) Should the syntax for creation of a temporary object be a constructor > call? Now, *that* question makes no sense. Why do you need special syntax to create a temporary object? Regards, Martin From mal at egenix.com Thu Aug 10 10:45:46 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 10 Aug 2006 10:45:46 +0200 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: References: Message-ID: <44DAF23A.103@egenix.com> Guido van Rossum wrote: > I've been happily ignoring python-dev for the last three weeks or so, > and Neal just pointed me to some thorny issues that are close to > resolution but not quite yet resolved, yet need to be before beta 3 on > August 18 (Friday next week). > > Here's my take on the dict-suppressing-exceptions issue (I'll send out > separate messages for each issue where Neal has asked me to weigh in). > > It wasn't my idea to stop ignoring exceptions in dict lookups; I would > gladly have put this off until Py3k, where the main problem > (str-unicode __eq__ raising UnicodeError) will go away. > > But since people are adamant that they want this in sooner, I suggest > that to minimize breakage we could make an exception for str-unicode > comparisons. > > I came up with the following code to reproduce the issue; this prints > 0 in 2.2, False in 2.3 and 2.4, but raises UnicodeDecodeError in 2.5 > (head): > > a = {u"1": 1} > x = hash(u"1") > class C(str): > def __hash__(s): return x > print C("\xff") in a > > The following patch makes this print False in 2.5 again. > > Notes about the patch: > > - this also fixes an out-of-date comment that should be fixed even if > the rest of the idea is rejected (lookdict_string() can return NULL > when it calls lookdict) > > - the exception could be narrowed even further by only suppressing the > exception when startkey and key are both either str or unicode > instances. > > What do people think? I'd suggest that we still inform the programmers of the problem by issuing a warning (which they can then silence at will), maybe a new PyExc_UnicodeWarning. Note that these exceptions help programmers making their applications Unicode compatible, so silencing them completely would remove the possibility to detect the case of mixing strings and Unicode as keys in a dictionary. BTW, in Py3k, this case would not trigger at all, since all text would be Unicode and bytes wouldn't be comparable to Unicode anyway. However, that's a different discussion which we can have after Python 2.5 is out the door. > --- Objects/dictobject.c (revision 51180) > +++ Objects/dictobject.c (working copy) > @@ -230,7 +230,8 @@ > lookdict() is general-purpose, and may return NULL if (and only if) a > comparison raises an exception (this was new in Python 2.5). > lookdict_string() below is specialized to string keys, comparison of which can > -never raise an exception; that function can never return NULL. For both, when > +never raise an exception; that function can never return NULL (except when it > +decides to replace itself with the more general lookdict()). For both, when > the key isn't found a dictentry* is returned for which the me_value field is > NULL; this is the slot in the dict at which the key would have been found, and > the caller can (if it wishes) add the pair to the returned > @@ -259,8 +260,13 @@ > if (ep->me_hash == hash) { > startkey = ep->me_key; > cmp = PyObject_RichCompareBool(startkey, key, Py_EQ); > - if (cmp < 0) > - return NULL; > + if (cmp < 0) { > + if (PyErr_Occurred()==PyExc_UnicodeDecodeError) { if (PyErr_Warn(PyExc_UnicodeWarning, "mixing unicode and strings " "as dictionary keys") < 0) return NULL; > + PyErr_Clear(); > + } > + else > + return NULL; > + } > if (ep0 == mp->ma_table && ep->me_key == startkey) { > if (cmp > 0) > return ep; > @@ -289,8 +295,13 @@ > if (ep->me_hash == hash && ep->me_key != dummy) { > startkey = ep->me_key; > cmp = PyObject_RichCompareBool(startkey, key, Py_EQ); > - if (cmp < 0) > - return NULL; > + if (cmp < 0) { > + if (PyErr_Occurred()==PyExc_UnicodeDecodeError) { > + PyErr_Clear(); > + } > + else > + return NULL; > + } > if (ep0 == mp->ma_table && ep->me_key == startkey) { > if (cmp > 0) > return ep; > > -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 10 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Thu Aug 10 11:10:42 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 10 Aug 2006 11:10:42 +0200 Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys In-Reply-To: <44D8DF4A.70408@v.loewis.de> References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com> <2m3bccwopj.fsf@starship.python.net> <44D36B36.8030202@egenix.com> <44D73964.3050402@v.loewis.de> <44D83C78.4070008@egenix.com> <44D843C5.1060907@v.loewis.de> <44D8A8E2.5090002@egenix.com> <44D8DF4A.70408@v.loewis.de> Message-ID: <44DAF812.3080603@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg schrieb: >> Note that we are not discussing changing the behavior of the >> __eq__ comparison between strings and Unicode, since this has >> always been to raise exceptions in case the automatic propagation >> fails. > > Not sure what you are discussing: This is *precisely* what I'm > discussing. Making that change would solve this problem. ... and introduce a whole set of new problems, namely finding all the places in your code that are not Unicode compatible. Instead of reverting the patch and continue to silencing exceptions in the dict lookup, you are proposing to extend the behavior to all equal comparisons of strings and Unicode. That would be a major change in the semantics, not just a bug fix as you seem to imply. Since the problem of mixing strings and Unicode will go away in Py3k anyway, I suggest we do everything to help programmers make their applications Unicode compatible instead of trying to hide all problem cases from them. Note that all of these problem go away if you stick to Unicode for all text data in your application. This will also make porting the application to Py3k a lot easier. > >> The discussion is about silencing exceptions in the dict lookup >> mechanism - something which used to happen and now no longer >> is done. Regards, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 10 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From ncoghlan at gmail.com Thu Aug 10 14:13:05 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 10 Aug 2006 22:13:05 +1000 Subject: [Python-Dev] Is this a bug? In-Reply-To: <20060809084110.E71E.JCARLSON@uci.edu> References: <20060809084110.E71E.JCARLSON@uci.edu> Message-ID: <44DB22D1.60002@gmail.com> Josiah Carlson wrote: > This is the case for Python 2.3, 2.4, and 2.5 beta. prefixing the above > two operations with: > sys.modules['_oldmain'] = sys.modules['__main__'] > > Is sufficient to prevent Python from tearing down everything after the > sys.modules['__main__'] reassignment. Not a big deal, but a sys.modules > manipulation that has a gotcha. Early versions of the PEP 338 implementation also ran into the problem of a module's globals getting cleared when the module itself goes away - that's why runpy makes sure to return a copy of the module's global namespace instead of the original. The core of the problem in the atexit case seems to be the fact that holding on to a reference to a function from a module only holds a reference to the module's dict, rather than the module object itself. If you kill the reference to the module from sys.modules, the module object goes away, and the dict gets cleared. However, any function objects being held elsewhere (e.g. in the atexit list, or as a result of a class being subclassed in a different module) still have a reference to the now defunct module namespace, and will misbehave when called later on if they attempt to access any global variables. Are we really sure that explicitly clearing the module's globals when the module's refcount falls to zero solves more problems than it causes? Or is it primarily a legacy of the pre-cyclic GC days when it was absolutely essential to manually break the cycle between the module namespace and any functions it contained? Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ralf at brainbot.com Thu Aug 10 14:53:39 2006 From: ralf at brainbot.com (Ralf Schmitt) Date: Thu, 10 Aug 2006 14:53:39 +0200 Subject: [Python-Dev] SimpleXMLWriter missing from elementtree Message-ID: Hi, any chance to get SimpleXMLWriter (and other modules maybe??) getting included into xml.etree? Otherwise people might have to stick to the original elementtree, which doesn't really make sense, since most of elementtree already is included. - Ralf From ralf at brainbot.com Thu Aug 10 14:59:25 2006 From: ralf at brainbot.com (Ralf Schmitt) Date: Thu, 10 Aug 2006 14:59:25 +0200 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: References: Message-ID: <44DB2DAD.1060906@brainbot.com> Guido van Rossum wrote: > > - the exception could be narrowed even further by only suppressing the > exception when startkey and key are both either str or unicode > instances. > I always assumed dictionaries would work exactly like this. So, at least it would now work as I had always expected (and others possibly too). - Ralf From murman at gmail.com Thu Aug 10 15:48:43 2006 From: murman at gmail.com (Michael Urman) Date: Thu, 10 Aug 2006 08:48:43 -0500 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: <44DAF23A.103@egenix.com> References: <44DAF23A.103@egenix.com> Message-ID: On 8/10/06, M.-A. Lemburg wrote: > Guido van Rossum wrote: > > But since people are adamant that they want this in sooner, I suggest > > that to minimize breakage we could make an exception for str-unicode > > comparisons. > > What do people think? > > I'd suggest that we still inform the programmers of the problem > by issuing a warning (which they can then silence at will), > maybe a new PyExc_UnicodeWarning. > > BTW, in Py3k, this case would not trigger at all, since all text > would be Unicode and bytes wouldn't be comparable to Unicode > anyway. However, that's a different discussion which we can have > after Python 2.5 is out the door. I strongly believe that unicode vs str here is the symptom and not the actual problem. The comparison between two non-us-ascii str/unicode instances is but one of many ways to raise an exception during comparison. It's not even obvious ahead of time when it will occur. Try my example below with (sys.maxint << 1) and -2 instead of 1 and 1. Do you expect a problem? Because str/unicode is the common case, we saw it first. If we address the symptom instead of the problem, someone will be complaining within a years time because they have a class that they mix in with other items for a function handler lookup, or who knows what, that works like the following: >>> class hasher(object): ... def __init__(self, hashval): ... self.hashval = hashval ... def __hash__(self): ... return hash(self.hashval) ... def __eq__(self, o): ... if not isinstance(o, hasher): ... raise TypeError("Cannot compare hashval to non hashval") ... return self.hashval == o.hashval in python2.4: >>> dict.fromkeys([1, hasher(1)]) {1: None, <__main__.hasher object at 0xa7a5326c>: None} in python2.5b2: >>> dict.fromkeys([1, hasher(1)]) Traceback (most recent call last): File "", line 1, in File "", line 8, in __eq__ TypeError: Cannot compare hashval to non hashval Yes this is made up code. But I'm not arguing for a feature; I'm arguing for backwards compatibility. Because we do not know where these legitimate uses are, we cannot evaluate their likelihood to exist nor the level of breakage they will cause. If we make this a warning for any exception, we can satisfy both imagined camps. Those in Armin's position can make that warning raise an exception while debugging, and those using it on purpose can squash it. I understand the utility of being able to see this case happening. I'm not sure it's worth the warning. I'm positive it's not worth the backwards incompatibility of the raised exception. Michael -- Michael Urman http://www.tortall.net/mu/blog From murman at gmail.com Thu Aug 10 15:52:48 2006 From: murman at gmail.com (Michael Urman) Date: Thu, 10 Aug 2006 08:52:48 -0500 Subject: [Python-Dev] SyntaxError: can't assign to function call In-Reply-To: <2m4pwlvezn.fsf@starship.python.net> References: <2m4pwlvezn.fsf@starship.python.net> Message-ID: On 8/9/06, Michael Hudson wrote: > The question doesn't make sense: in Python, you assign to a name, > an attribute or a subscript, and that's it. Just to play devil's advocate here, why not to a function call via a new __setcall__? I'm not saying there's the use case to justify it, but I don't see anything that makes it a clear abomination or impossible with python's syntax. Michael -- Michael Urman http://www.tortall.net/mu/blog From ncoghlan at gmail.com Thu Aug 10 16:18:11 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 11 Aug 2006 00:18:11 +1000 Subject: [Python-Dev] __index__ clipping In-Reply-To: References: Message-ID: <44DB4023.5090904@gmail.com> Guido van Rossum wrote: >> It seems like Nick's recent patches solved the problems that were >> identified. > > Nick, can you summarize how your patches differ from my proposal? nb_index and __index__ are essentially exactly as you propose. To make an object implemented in C usable as an index you would take either the nb_int slot or the nb_long slot and put the same function pointer into the nb_index slot. For a Python object, you would write either '__index__ = __int__' or '__index__ = __long__' as part of the class definition. operator.index is provided to support writing __getitem__, __setitem__ and __delitem__ methods - it raises IndexError on overflow so you don't have to catch and reraise to convert an OverflowError to an IndexError. On the C API side, the 3 functions you suggest are all present (although the version returning a Python object is accessed via PyObject_CallMethod), and there's a 4th variant that raises IndexError instead of OverflowError (this version is convenient when writing mp_subscript and mp_ass_subscript functions). Avoiding Py_ssize_t -> PyInt -> Py_ssize_t conversions for all integer types implemented in C would be nice, but I don't think it's practical (the latest version of the patch does at least avoid it for the builtin integer types). Cheers, Nick. P.S. Here's the detailed rationale for the form the patch has evolved to [1]: In addition to allowing (2**100).__index__() == 2**100, having nb_index return a Python object resulted in a decent reduction in code duplication - previously the coercion logic to get a Python integer or long value down to a Py_ssize_t was present in 3 places (long_index, instance_index, slot_nb_index), and would also have needed to be duplicated by any other C implemented index type whose value could exceed the range of a Py_ssize_t. With the patch, that logic appears only inside abstract.c and extension types can just return a PyLong value and let the interpreter figure out how to handle overflow. The biggest benefit of this approach is that a single slot (nb_index) can be used to implement four different overflow behaviours in the core (return PyLong, raise OverflowError, raise IndexError, clip to Py_ssize_t), as well as providing a hook to allow extension module authors to define their own overflow handling. If the nb_index slot does not return a true Python integer or long, TypeError gets raised. Subclasses are not accepted in order to rule out Armin's favourite set of recursion problems :) The C level API is based on the use cases in the standard library, with one of the functions generalised a bit to allow extension modules to easily handle type errors and overflow differently if they want to. The three different use cases for nb_index in the standard library are: - concrete sequence indices (want IndexError on overflow) - 'true integer' retrieval (want OverflowError on overflow) - slice endpoints (want to clip to Py_ssize_t max/min values) The proposed fix (Travis & Neal provided some useful comments on earlier versions) includes a C API function for each of these different use cases: PyNumber_Index(PyObject *obj, int *type_err) PyNumber_AsSsize_t(PyObject *obj, int *type_err) PyNumber_AsClippedSsize_t(PyObject *obj, int *type_err, int *clipped) type_err is an output variable to say "obj does not provide nb_index" in order to get rid of boilerplate dealing with PyErr_Occurred() in mp_subscript and mp_ass_subscript implementations (those methods generally didn't want a TypeError raised at this point - they wanted to go on and check if the object was a slice object instead). It's also useful if you want to provide a specific error message for TypeErrors (sequence repetition takes advantage of this). You can also leave the pointer as NULL and the functions will raise a fairly generic TypeError for you. PyObject_GetItem and friends, use the functions that way. Avoiding repeated code is also why there are two non-clipping variants, one raising IndexError and one raising OverflowError. Raising OverflowError in PyNumber_Index broke half a dozen unit tests, while raising IndexError for things like sequence repetition turned out to break different unit tests. The clipping variant is for slice indices. The interpreter core doesn't actually care whether or not the result gets clipped in this case (it sets the last parameter to NULL), but I kept the output variable in the signature for the benefit of extension authors. All 3 of the C API methods return Py_ssize_t. The "give me a Python object" case isn't actually needed anywhere in the core, but is available to extension modules via: PyObject_CallMethod(obj, "__index__", NULL) As Travis notes, indexing with something other than a builtin integer will be slightly slower due to the temporary object created by calling the nb_index slot (version 4 of the patch avoids this overhead for ints, version 5 avoids it for longs as well). I don't think this is avoidable - a non-PyObject return value really doesn't provide the necessary flexibility to detect and handle overflow correctly. [1] http://www.python.org/sf/1530738 -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From guido at python.org Thu Aug 10 16:26:54 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 10 Aug 2006 07:26:54 -0700 Subject: [Python-Dev] __index__ clipping In-Reply-To: <44DB4023.5090904@gmail.com> References: <44DB4023.5090904@gmail.com> Message-ID: On 8/10/06, Nick Coghlan wrote: > Guido van Rossum wrote: > >> It seems like Nick's recent patches solved the problems that were > >> identified. > > > > Nick, can you summarize how your patches differ from my proposal? > > nb_index and __index__ are essentially exactly as you propose. Then I don't understand why Travis is objecting against my proposal! I'll review the rest later (right now I'm just doing email triage :-). --Guido > To make an > object implemented in C usable as an index you would take either the nb_int > slot or the nb_long slot and put the same function pointer into the nb_index > slot. For a Python object, you would write either '__index__ = __int__' or > '__index__ = __long__' as part of the class definition. > > operator.index is provided to support writing __getitem__, __setitem__ and > __delitem__ methods - it raises IndexError on overflow so you don't have to > catch and reraise to convert an OverflowError to an IndexError. > > On the C API side, the 3 functions you suggest are all present (although the > version returning a Python object is accessed via PyObject_CallMethod), and > there's a 4th variant that raises IndexError instead of OverflowError (this > version is convenient when writing mp_subscript and mp_ass_subscript functions). > > Avoiding Py_ssize_t -> PyInt -> Py_ssize_t conversions for all integer types > implemented in C would be nice, but I don't think it's practical (the latest > version of the patch does at least avoid it for the builtin integer types). > > Cheers, > Nick. > > > > P.S. Here's the detailed rationale for the form the patch has evolved to [1]: > > In addition to allowing (2**100).__index__() == 2**100, having nb_index return > a Python object resulted in a decent reduction in code duplication - > previously the coercion logic to get a Python integer or long value down to a > Py_ssize_t was present in 3 places (long_index, instance_index, > slot_nb_index), and would also have needed to be duplicated by any other C > implemented index type whose value could exceed the range of a Py_ssize_t. > With the patch, that logic appears only inside abstract.c and extension types > can just return a PyLong value and let the interpreter figure out how to > handle overflow. The biggest benefit of this approach is that a single slot > (nb_index) can be used to implement four different overflow behaviours in the > core (return PyLong, raise OverflowError, raise IndexError, clip to > Py_ssize_t), as well as providing a hook to allow extension module authors to > define their own overflow handling. > > If the nb_index slot does not return a true Python integer or long, TypeError > gets raised. Subclasses are not accepted in order to rule out Armin's > favourite set of recursion problems :) > > The C level API is based on the use cases in the standard library, with one of > the functions generalised a bit to allow extension modules to easily handle > type errors and overflow differently if they want to. > > The three different use cases for nb_index in the standard library are: > - concrete sequence indices (want IndexError on overflow) > - 'true integer' retrieval (want OverflowError on overflow) > - slice endpoints (want to clip to Py_ssize_t max/min values) > > The proposed fix (Travis & Neal provided some useful comments on earlier > versions) includes a C API function for each of these different use cases: > > PyNumber_Index(PyObject *obj, int *type_err) > PyNumber_AsSsize_t(PyObject *obj, int *type_err) > PyNumber_AsClippedSsize_t(PyObject *obj, int *type_err, int *clipped) > > type_err is an output variable to say "obj does not provide nb_index" in order > to get rid of boilerplate dealing with PyErr_Occurred() in mp_subscript and > mp_ass_subscript implementations (those methods generally didn't want a > TypeError raised at this point - they wanted to go on and check if the object > was a slice object instead). It's also useful if you want to provide a > specific error message for TypeErrors (sequence repetition takes advantage of > this). You can also leave the pointer as NULL and the functions will raise a > fairly generic TypeError for you. PyObject_GetItem and friends, use the > functions that way. > > Avoiding repeated code is also why there are two non-clipping variants, one > raising IndexError and one raising OverflowError. Raising OverflowError in > PyNumber_Index broke half a dozen unit tests, while raising IndexError for > things like sequence repetition turned out to break different unit tests. > > The clipping variant is for slice indices. The interpreter core doesn't > actually care whether or not the result gets clipped in this case (it sets the > last parameter to NULL), but I kept the output variable in the signature for > the benefit of extension authors. > > All 3 of the C API methods return Py_ssize_t. The "give me a Python object" > case isn't actually needed anywhere in the core, but is available to extension > modules via: > PyObject_CallMethod(obj, "__index__", NULL) > > As Travis notes, indexing with something other than a builtin integer will be > slightly slower due to the temporary object created by calling the nb_index > slot (version 4 of the patch avoids this overhead for ints, version 5 avoids > it for longs as well). I don't think this is avoidable - a non-PyObject return > value really doesn't provide the necessary flexibility to detect and handle > overflow correctly. > > [1] http://www.python.org/sf/1530738 > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > --------------------------------------------------------------- > http://www.boredomandlaziness.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Aug 10 16:32:29 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 10 Aug 2006 07:32:29 -0700 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: References: <44DAF23A.103@egenix.com> Message-ID: On 8/10/06, Michael Urman wrote: > I strongly believe that unicode vs str here is the symptom and not the > actual problem. No. Everywhere when __eq__ fails, we can safely tell the user that it's a bug in their __eq__ that they should fix (maybe by making it return False when the other object isn't a type they recognize). But we can't fix unicode-vs-str comparison without breaking too much code. So we have to somehow deal with that without changing the behavior of that particular __eq__. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Aug 10 16:36:29 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 10 Aug 2006 07:36:29 -0700 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: <44DAF23A.103@egenix.com> References: <44DAF23A.103@egenix.com> Message-ID: On 8/10/06, M.-A. Lemburg wrote: > I'd suggest that we still inform the programmers of the problem > by issuing a warning (which they can then silence at will), > maybe a new PyExc_UnicodeWarning. Hmm... Here's an idea... How about we change unicode-vs-str __eq__ to issue a warning (and return False) instead of raising UnicodeException? That won't break much code (it's unlikely that people *depend* on this exception since it's generally a symptom of insane mixing of str and unicode). Then no further changes to dictobject.c are necessary (except fixing that one misleading comment). > Note that these exceptions help programmers making their applications > Unicode compatible, so silencing them completely would remove the > possibility to detect the case of mixing strings and Unicode as > keys in a dictionary. A warning would arguably have the same helping effect. (I suspect actually that we would have used warnings all along except we didn't have the warning framework when unicode was first introduced.) > BTW, in Py3k, this case would not trigger at all, since all text > would be Unicode and bytes wouldn't be comparable to Unicode > anyway. However, that's a different discussion which we can have > after Python 2.5 is out the door. Which is why I said I would have gladly waited until Py3k to fix this. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Thu Aug 10 16:39:10 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 11 Aug 2006 00:39:10 +1000 Subject: [Python-Dev] __index__ clipping In-Reply-To: References: <44DB4023.5090904@gmail.com> Message-ID: <44DB450E.7050002@gmail.com> Guido van Rossum wrote: > On 8/10/06, Nick Coghlan wrote: >> Guido van Rossum wrote: >> >> It seems like Nick's recent patches solved the problems that were >> >> identified. >> > >> > Nick, can you summarize how your patches differ from my proposal? >> >> nb_index and __index__ are essentially exactly as you propose. > > Then I don't understand why Travis is objecting against my proposal! I must admit to being confused by that, too. . . Cheers, Nick. P.S. Just to mess up the release schedule some more, I'm going to be away from 13-19 August, so the patch either needs to be approved by Neal or Anthony in the next few days, or someone else will need to commit it before rc1. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From mal at egenix.com Thu Aug 10 17:05:27 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 10 Aug 2006 17:05:27 +0200 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: References: <44DAF23A.103@egenix.com> Message-ID: <44DB4B37.1000401@egenix.com> Guido van Rossum wrote: > On 8/10/06, M.-A. Lemburg wrote: >> I'd suggest that we still inform the programmers of the problem >> by issuing a warning (which they can then silence at will), >> maybe a new PyExc_UnicodeWarning. > > Hmm... Here's an idea... How about we change unicode-vs-str __eq__ to > issue a warning (and return False) instead of raising > UnicodeException? That won't break much code (it's unlikely that > people *depend* on this exception since it's generally a symptom of > insane mixing of str and unicode). Then no further changes to > dictobject.c are necessary (except fixing that one misleading > comment). Good idea. >> Note that these exceptions help programmers making their applications >> Unicode compatible, so silencing them completely would remove the >> possibility to detect the case of mixing strings and Unicode as >> keys in a dictionary. > > A warning would arguably have the same helping effect. (I suspect > actually that we would have used warnings all along except we didn't > have the warning framework when unicode was first introduced.) Probably, yes. These days, I believe that we should have not gone for the string-Unicode integration magic at all and instead done the same of what's planned for Py3k. Sometimes a clean cut is better than trying to muddle along. >> BTW, in Py3k, this case would not trigger at all, since all text >> would be Unicode and bytes wouldn't be comparable to Unicode >> anyway. However, that's a different discussion which we can have >> after Python 2.5 is out the door. > > Which is why I said I would have gladly waited until Py3k to fix this. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 10 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From jcarlson at uci.edu Thu Aug 10 18:01:47 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu, 10 Aug 2006 09:01:47 -0700 Subject: [Python-Dev] SyntaxError: can't assign to function call In-Reply-To: References: <2m4pwlvezn.fsf@starship.python.net> Message-ID: <20060810084217.1914.JCARLSON@uci.edu> "Michael Urman" wrote: > > On 8/9/06, Michael Hudson wrote: > > The question doesn't make sense: in Python, you assign to a name, > > an attribute or a subscript, and that's it. > > Just to play devil's advocate here, why not to a function call via a > new __setcall__? I'm not saying there's the use case to justify it, > but I don't see anything that makes it a clear abomination or > impossible with python's syntax. Describe the syntax and semantics. Every time I try to work them out, I end up with a construct that makes less than no sense, to be used in cases I have never seen. Further, if you want to call a method __setcall__ on an object just created, you can use 'x().__setcall__(y)'. There is no reason to muck up Python's syntax. - Josiah From foom at fuhm.net Thu Aug 10 18:19:22 2006 From: foom at fuhm.net (James Y Knight) Date: Thu, 10 Aug 2006 12:19:22 -0400 Subject: [Python-Dev] SyntaxError: can't assign to function call In-Reply-To: <20060810084217.1914.JCARLSON@uci.edu> References: <2m4pwlvezn.fsf@starship.python.net> <20060810084217.1914.JCARLSON@uci.edu> Message-ID: <64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net> On Aug 10, 2006, at 12:01 PM, Josiah Carlson wrote: > > "Michael Urman" wrote: >> >> On 8/9/06, Michael Hudson wrote: >>> The question doesn't make sense: in Python, you assign to a name, >>> an attribute or a subscript, and that's it. >> >> Just to play devil's advocate here, why not to a function call via a >> new __setcall__? I'm not saying there's the use case to justify it, >> but I don't see anything that makes it a clear abomination or >> impossible with python's syntax. > > Describe the syntax and semantics. Every time I try to work them > out, I > end up with a construct that makes less than no sense, to be used in > cases I have never seen. Further, if you want to call a method > __setcall__ on an object just created, you can use 'x().__setcall__ > (y)'. > There is no reason to muck up Python's syntax. It makes just as much sense as assigning to an array access, and the semantics would be pretty similar. There's similarly "no reason" to allow x[5] = True. You can just spell that x.__setitem__(5, True). x(*args, **kwargs) = val could translate into x.__setcall__(val, *args, **kwargs). x(5) = True could translate into x.__setcall__(True, 5) Please note I'm actually arguing for this proposal. Just agreeing that it is not a completely nonsensical idea. James From guido at python.org Thu Aug 10 18:24:39 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 10 Aug 2006 09:24:39 -0700 Subject: [Python-Dev] SyntaxError: can't assign to function call In-Reply-To: <64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net> References: <2m4pwlvezn.fsf@starship.python.net> <20060810084217.1914.JCARLSON@uci.edu> <64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net> Message-ID: On 8/10/06, James Y Knight wrote: > It makes just as much sense as assigning to an array access, and the > semantics would be pretty similar. No. Array references (x[i]) and attribute references (x.a) represent "locations". Function calls represent values. This is no different than the distinction between lvalues and rvalues in C. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Thu Aug 10 18:31:23 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 10 Aug 2006 12:31:23 -0400 Subject: [Python-Dev] SyntaxError: can't assign to function call In-Reply-To: References: <64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net> <2m4pwlvezn.fsf@starship.python.net> <20060810084217.1914.JCARLSON@uci.edu> <64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net> Message-ID: <5.1.1.6.0.20060810122651.0262dd18@sparrow.telecommunity.com> At 09:24 AM 8/10/2006 -0700, Guido van Rossum wrote: >On 8/10/06, James Y Knight wrote: > > It makes just as much sense as assigning to an array access, and the > > semantics would be pretty similar. > >No. Array references (x[i]) and attribute references (x.a) represent >"locations". Function calls represent values. This is no different >than the distinction between lvalues and rvalues in C. IIRC, in Lisp a function call can be an lvalue. OTOH, that's probably because in Lisp there isn't anything besides function calls. :) I'm having a hard time imagining a use case in Python, though, except to allow things like "somedict.setdefault(key,0) += 1". And I'm not sure that's an improvement, although there have certainly been times that I started to write that, and then grumbled and wrote "somedict[key] = somedict.setdefault(key,0) + 1" instead. Honestly, it might make more sense to get rid of augmented assignment in Py3K rather than to add this. It seems that the need for something like this springs primarily from the existence of augmented assignment. From guido at python.org Thu Aug 10 18:33:50 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 10 Aug 2006 09:33:50 -0700 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: <44DB4B37.1000401@egenix.com> References: <44DAF23A.103@egenix.com> <44DB4B37.1000401@egenix.com> Message-ID: On 8/10/06, M.-A. Lemburg wrote: > Guido van Rossum wrote: > > Hmm... Here's an idea... How about we change unicode-vs-str __eq__ to > > issue a warning (and return False) instead of raising > > UnicodeException? That won't break much code (it's unlikely that > > people *depend* on this exception since it's generally a symptom of > > insane mixing of str and unicode). Then no further changes to > > dictobject.c are necessary (except fixing that one misleading > > comment). > > Good idea. Great! Now we need someone to volunteer to write a patch (which should include doc and NEWS updates) in time for beta 3 (Aug 18). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From foom at fuhm.net Thu Aug 10 18:49:04 2006 From: foom at fuhm.net (James Y Knight) Date: Thu, 10 Aug 2006 12:49:04 -0400 Subject: [Python-Dev] SyntaxError: can't assign to function call In-Reply-To: <64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net> References: <2m4pwlvezn.fsf@starship.python.net> <20060810084217.1914.JCARLSON@uci.edu> <64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net> Message-ID: <14E22ABA-4AE4-4C35-B3B4-D6C7EE8D528E@fuhm.net> On Aug 10, 2006, at 12:19 PM, James Y Knight wrote: > Please note I'm actually arguing for this proposal. Just agreeing > that it is not a completely nonsensical idea. ERK! Big typo there. I meant to say: Please note I'm ****NOT*** actually arguing for this proposal. Sorry for any confusion. James From guido at python.org Thu Aug 10 20:14:14 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 10 Aug 2006 11:14:14 -0700 Subject: [Python-Dev] cgi DoS attack In-Reply-To: References: Message-ID: FWIW, I've checked this in (while stuck in a meeting, Neal will know what I mean :-). svn r51190-r51192/ On 8/9/06, Guido van Rossum wrote: > This is regarding #1112549. I think this can go in. It should also be > backported to 2.4 and to 2.3 (if we ever release another one of that). > I reviewed the code and added some minor comments to the SF tracker. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jcarlson at uci.edu Thu Aug 10 20:22:03 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu, 10 Aug 2006 11:22:03 -0700 Subject: [Python-Dev] SyntaxError: can't assign to function call In-Reply-To: <5.1.1.6.0.20060810122651.0262dd18@sparrow.telecommunity.com> References: <5.1.1.6.0.20060810122651.0262dd18@sparrow.telecommunity.com> Message-ID: <20060810111304.191A.JCARLSON@uci.edu> "Phillip J. Eby" wrote: > At 09:24 AM 8/10/2006 -0700, Guido van Rossum wrote: > >On 8/10/06, James Y Knight wrote: > > > It makes just as much sense as assigning to an array access, and the > > > semantics would be pretty similar. > > > >No. Array references (x[i]) and attribute references (x.a) represent > >"locations". Function calls represent values. This is no different > >than the distinction between lvalues and rvalues in C. > > I'm having a hard time imagining a use case in Python, though, except to > allow things like "somedict.setdefault(key,0) += 1". And I'm not sure > that's an improvement, although there have certainly been times that I > started to write that, and then grumbled and wrote "somedict[key] = > somedict.setdefault(key,0) + 1" instead. Thankfully default dicts in Python 2.5 (they are in, right?) should solve this particular nit of the language. And by using default dicts as an instance __dict__, you get similar behavior for attributes. > Honestly, it might make more sense to get rid of augmented assignment in > Py3K rather than to add this. It seems that the need for something like > this springs primarily from the existence of augmented assignment. I'm not sure that your characterization is entirely accurate. I've used and seen augmented assignments being used for many years now, and this is the first time that I've seen anyone propose that it should work. - Josiah From oliphant.travis at ieee.org Thu Aug 10 20:25:00 2006 From: oliphant.travis at ieee.org (Travis E. Oliphant) Date: Thu, 10 Aug 2006 12:25:00 -0600 Subject: [Python-Dev] __index__ clipping In-Reply-To: References: <44DB4023.5090904@gmail.com> Message-ID: Guido van Rossum wrote: > On 8/10/06, Nick Coghlan wrote: >> Guido van Rossum wrote: >>>> It seems like Nick's recent patches solved the problems that were >>>> identified. >>> Nick, can you summarize how your patches differ from my proposal? >> nb_index and __index__ are essentially exactly as you propose. > > Then I don't understand why Travis is objecting against my proposal! I must have missed his most recent patch that changed the result to return a Python object. I thought earlier versions didn't do that. My objection is not particularly solid. At this point it's largely a wish to avoid the extra overhead (there are some camps that already complain about NumPy having too much indexing overhead --- although they should be using Python lists for there purposes if indexing overhead really is a problem). As it appears that several people feel this is the best way forward, then I'll re-work my NumPy code. I still appreciate the change that allows other Python objects to be integers and eliminates the "only true integers allowed" flavor of several places in the Python code. Thanks for all your hard work. -Travis From foom at fuhm.net Thu Aug 10 20:28:20 2006 From: foom at fuhm.net (James Y Knight) Date: Thu, 10 Aug 2006 14:28:20 -0400 Subject: [Python-Dev] SyntaxError: can't assign to function call In-Reply-To: References: <2m4pwlvezn.fsf@starship.python.net> <20060810084217.1914.JCARLSON@uci.edu> <64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net> Message-ID: <1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net> On Aug 10, 2006, at 12:24 PM, Guido van Rossum wrote: > On 8/10/06, James Y Knight wrote: >> It makes just as much sense as assigning to an array access, and the >> semantics would be pretty similar. > > No. Array references (x[i]) and attribute references (x.a) represent > "locations". Function calls represent values. This is no different > than the distinction between lvalues and rvalues in C. Yes, function calls cannot be lvalues right now. However, there is no reason that a function call _could not_ be an lvalue. That is exactly what the addition of __setcall__ would allow. On Aug 10, 2006, at 12:31 PM, Phillip J. Eby wrote: > Honestly, it might make more sense to get rid of augmented > assignment in Py3K rather than to add this. It seems that the need > for something like this springs primarily from the existence of > augmented assignment. It makes just as much (and just as little) sense to have normal assignment to function calls as it does augmented assignment to function calls. I don't see any reason to single out augmented assignment here. Anyhow, enough time wasted on this. I don't really think python should add this feature, but it _does_ make sense, and would have understandable and consistent semantics if it were added. James From armin.ronacher at active-4.com Thu Aug 10 20:37:17 2006 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Thu, 10 Aug 2006 18:37:17 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?Unicode_Data_in_Python2=2E5_is_missing_a_u?= =?utf-8?b?Y2RfNF8xXzAgb2JqZWN0?= Message-ID: Hi, I discovered that unicodedata in python2.5 implements unicode 4.1. While this is ok it's possible enforce unicode 3.2 by using the ucd_3_2_0 object. But it's not possible to enforce a ucd_4_1_0 standard because that object does not exist by now. In the description of #1031288 (http://www.python.org/sf/1031288) Martin v. L?wis says the following: | Python relies on the unicodedata 3.2.0, as the IDNA RFCs | mandate that Unicode 3.2 is used to implement IDNA. So any | integration of 4.0.1 must | a) still maintain access to the 3.2.0 data And furthermore the docstring claims that this module just implements unicode 3.2.0 whereas unidata_version gives me 4.1.0 Doesn't that mean that there should also be an way to enforce unicode 4.1.0? Regards, Armin From martin at v.loewis.de Thu Aug 10 21:05:46 2006 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Thu, 10 Aug 2006 21:05:46 +0200 Subject: [Python-Dev] Unicode Data in Python2.5 is missing a ucd_4_1_0 object In-Reply-To: References: Message-ID: <44DB838A.10600@v.loewis.de> Armin Ronacher schrieb: > I discovered that unicodedata in python2.5 implements unicode 4.1. While > this is ok it's possible enforce unicode 3.2 by using the ucd_3_2_0 object. > But it's not possible to enforce a ucd_4_1_0 standard because that object > does not exist by now. Not sure what you mean by "enforce". Isn't assert unicodedata.unidata_version == '4.1.0' enough enforcement? > And furthermore the docstring claims that this module just implements unicode > 3.2.0 whereas unidata_version gives me 4.1.0 That's a bug; thanks for pointing it out. I just fixed it in r51194. > Doesn't that mean that there should also be an way to enforce unicode 4.1.0? You mean, that there should be a ucd_4_1_0 object? No, why do you think there should be one? We don't plan to provide a copy of the UCD for each UCD version that was ever released (e.g. we skipped some between 3.2 and 4.1 also). Regards, Martin From martin at v.loewis.de Thu Aug 10 21:06:57 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 10 Aug 2006 21:06:57 +0200 Subject: [Python-Dev] SimpleXMLWriter missing from elementtree In-Reply-To: References: Message-ID: <44DB83D1.6040009@v.loewis.de> Ralf Schmitt schrieb: > any chance to get SimpleXMLWriter (and other modules maybe??) getting > included into xml.etree? Not in Python 2.5, no. Regards, Martin From martin at v.loewis.de Thu Aug 10 21:11:42 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 10 Aug 2006 21:11:42 +0200 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: References: <44DAF23A.103@egenix.com> Message-ID: <44DB84EE.8020606@v.loewis.de> Guido van Rossum schrieb: > Hmm... Here's an idea... How about we change unicode-vs-str __eq__ to > issue a warning (and return False) instead of raising > UnicodeException? I'm in favour of having this __eq__ just return False. I don't think the warning is necessary, and believe that people will complain about getting it, but if that is the only way to achieve consensus, so be it. > A warning would arguably have the same helping effect. (I suspect > actually that we would have used warnings all along except we didn't > have the warning framework when unicode was first introduced.) I believe we would have neither had we rich comparisons available at the time. That unicode-vs-str-__eq__ raises an exception is just a fall-out of not supporting __cmp__ (and it is good that __cmp__ is not supported - there is no meaningful way to order str/unicode if the string is not in the system encoding). Regards, Martin From martin at v.loewis.de Thu Aug 10 21:13:22 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 10 Aug 2006 21:13:22 +0200 Subject: [Python-Dev] returning longs from __hash__() In-Reply-To: <20060809094725.GA16199@code0.codespeak.net> References: <20060808102608.GA9098@code0.codespeak.net> <44D8E148.5050701@v.loewis.de> <1f7befae0608081320j61bacd63l8ea8d3a9f32f0424@mail.gmail.com> <44D8FED0.9060108@v.loewis.de> <20060809094725.GA16199@code0.codespeak.net> Message-ID: <44DB8552.5030505@v.loewis.de> Armin Rigo schrieb: > This bug will keep showing up forever :-) It's unsafe against a user > subclassing 'long' and overriding __hash__ of that subclass to return > the object itself -- it would cause an infinite C recursion. I see you've fixed it - thanks. Martin From guido at python.org Thu Aug 10 21:28:08 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 10 Aug 2006 12:28:08 -0700 Subject: [Python-Dev] SyntaxError: can't assign to function call In-Reply-To: <1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net> References: <2m4pwlvezn.fsf@starship.python.net> <20060810084217.1914.JCARLSON@uci.edu> <64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net> <1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net> Message-ID: On 8/10/06, James Y Knight wrote: > On Aug 10, 2006, at 12:24 PM, Guido van Rossum wrote: > > On 8/10/06, James Y Knight wrote: > >> It makes just as much sense as assigning to an array access, and the > >> semantics would be pretty similar. > > > > No. Array references (x[i]) and attribute references (x.a) represent > > "locations". Function calls represent values. This is no different > > than the distinction between lvalues and rvalues in C. > > Yes, function calls cannot be lvalues right now. However, there is no > reason that a function call _could not_ be an lvalue. That is exactly > what the addition of __setcall__ would allow. I have to admit that I don't find it particularly useful -- I still don't think I like the idea much of using function calls as assignment targets. You wrote x(5) = True would mean x.__setcall__(True, 5) What would x(5) += 1 mean? The best I can come up with is __tmp = x(5) # or x.__call__(5) if hasattr(__tmp, "__iadd__"): __val = __tmp.__iadd__(1) else: __val = __tmp + 1 if __val is NotImplemented: raise TypeError(...) __tmp.__setcall__(__val, 5) since this is essentially how x[5] += 1 works (except that the hasattr() check is hidden inside PyNumber_InPlaceAdd and optimized away to class definition time). I expect that most people interested in having f() += 1 to work would have to implement a dummy __setcall__() because their __iadd__ returns self and there's no additional work to be done for the assignment. I'm not convinced that all this complexity is worth it. For lists, += is syntactic sugar for .extend(). I expect that most use cases you can come up with can similarly be argued away. > On Aug 10, 2006, at 12:31 PM, Phillip J. Eby wrote: > > Honestly, it might make more sense to get rid of augmented > > assignment in Py3K rather than to add this. It seems that the need > > for something like this springs primarily from the existence of > > augmented assignment. I assume this was meant tongue-in-cheek. I see no reason to get rid of +=. The opportunity for hypergeneralization (== ill-advised generalization based on the misunderstanding of some mechanism) does not automatically mean a mechanism should not be added (although it can sometimes be a warning sign). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From armin.ronacher at active-4.com Thu Aug 10 22:06:32 2006 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Thu, 10 Aug 2006 20:06:32 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?Unicode_Data_in_Python2=2E5_is_missing_a_u?= =?utf-8?b?Y2RfNF8xXzAgb2JqZWN0?= References: <44DB838A.10600@v.loewis.de> Message-ID: Hi, Martin v. L?wis v.loewis.de> writes: > > Doesn't that mean that there should also be an way to enforce unicode 4.1.0? > > You mean, that there should be a ucd_4_1_0 object? No, why do you think > there should be one? We don't plan to provide a copy of the UCD for each > UCD version that was ever released (e.g. we skipped some between 3.2 and > 4.1 also). Right, I didn't know that. From that old bug report it sounded like a programmer should be able to select a specific UCD version. Regards, Armin From mcherm at mcherm.com Thu Aug 10 22:35:14 2006 From: mcherm at mcherm.com (Michael Chermside) Date: Thu, 10 Aug 2006 13:35:14 -0700 Subject: [Python-Dev] Dict suppressing exceptions Message-ID: <20060810133514.pk8p9e2di4a0o4wg@login.werra.lunarpages.com> Michael Urman writes: > I strongly believe that unicode vs str here is the symptom and not the > actual problem. The comparison between two non-us-ascii str/unicode > instances is but one of many ways to raise an exception during > comparison. [... example ...] > Yes this is made up code. But I'm not arguing for a feature; I'm > arguing for backwards compatibility. Because we do not know where > these legitimate uses are, we cannot evaluate their likelihood to > exist nor the level of breakage they will cause. If we make this a > warning for any exception, we can satisfy both imagined camps. Those > in Armin's position can make that warning raise an exception while > debugging, and those using it on purpose can squash it. See also my example from the beginning of this thread (http://mail.python.org/pipermail/python-dev/2006-August/067978.html). The example wasn't from real code, but it WAS quite plausible -- straightforward use of a popular Python Cookbook recipe. But we seem to have reached a rough consensus: Later, Guido writes: > How about we change unicode-vs-str __eq__ to > issue a warning (and return False) instead of raising > UnicodeException? [... Marc-Andre Lemburg agrees ...] > Great! Now we need someone to volunteer to write a patch (which should > include doc and NEWS updates) in time for beta 3 (Aug 18). I don't *strongly* object to this consensus, but if you haven't glanced at my original example, take a look - it might convince you. The proposed solution will not help with my example. I'm not sure the motivation for breaking code like that example -- the bug-hunting motivation is satisfied by issuing a warning as Michael Urman proposes, then use an exception after one more release when people have had time to fix their code. -- Michael Chermside From pje at telecommunity.com Thu Aug 10 22:57:44 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 10 Aug 2006 16:57:44 -0400 Subject: [Python-Dev] SyntaxError: can't assign to function call In-Reply-To: References: <1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net> <2m4pwlvezn.fsf@starship.python.net> <20060810084217.1914.JCARLSON@uci.edu> <64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net> <1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net> Message-ID: <5.1.1.6.0.20060810164933.02713da8@sparrow.telecommunity.com> At 12:28 PM 8/10/2006 -0700, Guido van Rossum wrote: >>On Aug 10, 2006, at 12:31 PM, Phillip J. Eby wrote: >> > Honestly, it might make more sense to get rid of augmented >> > assignment in Py3K rather than to add this. It seems that the need >> > for something like this springs primarily from the existence of >> > augmented assignment. > >I assume this was meant tongue-in-cheek. Not entirely, no. > I see no reason to get rid of >+=. The opportunity for hypergeneralization (== ill-advised >generalization based on the misunderstanding of some mechanism) does >not automatically mean a mechanism should not be added (although it >can sometimes be a warning sign). I think it's a warning sign because I *know* what augmented assignment's semantics are supposed to be and I *still* try to use it with setdefault() sometimes -- but only when I'm doing an augmented assignment. I never mistakenly try to assign to a function call in any other circumstance. However, I'm also not clear that trying to assign to a function call *is* ill-advised. One of the things that attracted me to Python in the first place is that it had a lot of features that would be considered "hypergeneralization" in other languages, e.g. the ability to create your own sequences, mappings, and callable objects in the first place. That being said, the benefit of hypergeneralizing assignment seems small compared to its price. So eliminating augmented assignment seems a more attractive way to get rid of the nuisance of the perennially repeated proposals to "fix" or otherwise extend it. Sometimes it seems like half the time I want to use augmented assignment, I can't use it anyway for one reason or another, so it doesn't seem like that big of a loss. From arigo at tunes.org Thu Aug 10 23:12:54 2006 From: arigo at tunes.org (Armin Rigo) Date: Thu, 10 Aug 2006 23:12:54 +0200 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: <44DB84EE.8020606@v.loewis.de> References: <44DAF23A.103@egenix.com> <44DB84EE.8020606@v.loewis.de> Message-ID: <20060810211254.GA24181@code0.codespeak.net> Hi, On Thu, Aug 10, 2006 at 09:11:42PM +0200, "Martin v. L?wis" wrote: > I'm in favour of having this __eq__ just return False. I don't think > the warning is necessary, (...) +1 Armin From steven.bethard at gmail.com Thu Aug 10 23:18:06 2006 From: steven.bethard at gmail.com (Steven Bethard) Date: Thu, 10 Aug 2006 15:18:06 -0600 Subject: [Python-Dev] DRAFT: python-dev summary for 2006-07-16 to 2006-07-31 Message-ID: Here's the summary for the second half of July. Thanks in advance for your comments and corrections! ============= Announcements ============= ------------------- Python 2.5 schedule ------------------- After inserting a third beta release to allow some more time for testing the new features, Python continues to make progress towards the final Python 2.5 release. See `PEP 356`_ for more details and the full schedule. .. _PEP 356: http://www.python.org/dev/peps/pep-0356/ Contributing threads: - `outstanding bugs to fix for 2.5 `__ - `Py2.5 release schedule `__ ------------------------------- How to submit a patch to Python ------------------------------- Just a few reminders for all those still new to python-dev. When submitting a new patch to SourceForge, don't assign it to anyone. Most python developers get email notifications for new patches and will assign it to themselves if appropriate. If you feel like the approach the patch takes might need discussion, it's alright to present it to python-dev and ask for some feedback. If you do, be sure to put the patch number and url (e.g. http://bugs.python.org/) near the top of the message, so that developers can easily find it. And if you don't want to wait for your patch to be looked at (which may take some time as all developers are volunteers), a few of the folks here, including Martin v. L?wis, have offered a five-for-one deal. Simply find five other patches, and check them for things like: * Does the code look okay? * Does Python build with it applied? * Do all unit tests pass? * Does the patch have tests? * Does the patch have documentation? Then post your notes to the five patch trackers and post a final message to python-dev indicating the patches you reviewed and the patch which you'd like to have someone look at for you. Contributing threads: - `new guy `__ - `first draft of bug guidelines for www.python.org/dev/ `__ - `Patch submitted, now what? `__ ---------------------------------------- Demos of trackers to replace SourceForge ---------------------------------------- There are currently three potential trackers that have successfully imported the SourceForge data with demos online: roundup_, jira_ and launchpad_. Try 'em out, and send your discussions and comments to infrastructure at python.org and put your reports and reviews `on the wiki`_. .. _roundup: http://efod.se/python-tracker/ .. _jira: http://jira.python.atlassian.com/secure/Dashboard.jspa .. _launchpad: https://demo.launchpad.net/products/python/+bugs .. _on the wiki: http://wiki.python.org/moin/CallForTrackers Contributing thread: - `More tracker demos online `__ ========= Summaries ========= ------------------------------ Restricted execution in Python ------------------------------ Brett Cannon decided this fortnight to go for an all-out capabilities based restricted execution design, and posted a `new design document`_. As part of this work, Brett planned to rewrite most of the import machinery in pure Python code, hopefully cleaning up some of the idiosyncrasies of the current import.c mechanisms, and allowing him to do things like restrict imports to only .py files, not .pyc files. Armin Rigo pointed out that a good place to start would be the `PyPy import implementation`_. On the restricted execution front, one of the things that is now likely to happen in Brett's sandboxing branch is that ``object.__subclasses__()`` and dangerous constructors like the one for the code object will be completely removed from the Python level. This means a few backwards incompatible changes, but Brett suggested that they should only break pretty advanced and esoteric Python code. Since it's for his Ph.D. dissertation, he didn't want to tie his hands by requiring full backwards compatibility, and he was fine with waiting to merge his branch until Python 3000. .. _new design document: http://svn.python.org/view/python/branches/bcannon-sandboxing/securing_python.txt .. _PyPy import implementation: http://codespeak.net/svn/pypy/dist/pypy/module/__builtin__/importing.py Contributing threads: - `Capabilities / Restricted Execution `__ - `new security doc using object-capabilities `__ -------------------------- Character case and locales -------------------------- Mihai Ibanescu asked about a `bug in the logging module`_ due to the fact that ``'INFO'.lower() != 'info'`` in some locales. Marc-Andre Lemburg and Martin v. L?wis explained that since in Unicode, nearly all case-conversions are only script-dependent, not language-dependent, ``u'INFO'.lower() == u'info'`` should always be true. .. _bug in the logging module: http://bugs.python.org/1524081 Contributing thread: - `logging module broken because of locale `__ ----------------------------------------------- Progress on the C version of the decimal module ----------------------------------------------- After looking at the current progress in converting the decimal module to C, Raymond Hettinger suggested that rather than using the Python implementation as an outline of the C implementation, a separate C implementation should be developed and then later wrapped as necessary to provide the Python APIs. Tim Peters explained their incremental approach: leaving most of the module written in Python, and converting methods to C code one at a time. Raymond had originally supported this approach, but after viewing the current C code, thought that it would result in C code that was too complex and convoluted. There was some extended discussion on the mechanism in the current decimal module for holding flags, which uses a dict mapping error types to the counts of their occurrences. Raymond in particular wanted the C decimal module to be able to change this API if it was too complex to implement. A number of others agreed that the API had been a bad decision, and it looked like there would at least be a note in the documentation for Python 2.5 suggesting that users should not rely on the counting feature. Contributing thread: - `Strategy for converting the decimal module to C `__ --------------------------------------- PEP 357: Integer clipping and __index__ --------------------------------------- Armin Rigo pointed out that the current implementation of ``__index__()`` was incorrectly truncating long integers:: >>> (2**100).__index__() 2147483647 As the original ``__index__()`` method was intended only to allow things other than plain Python ints as slice indices, truncating to the maximum value was fine. However, when ``__index__()`` also became the "can you faithfully act like an integer" check, this truncation was no longer acceptable. Nick Coghlan spent some time reworking the `PEP 357`_ C API so that all the use cases of ``__index__()`` were covered. His `patch for fixing __index__`_ changes the nb_index slot to return a PyInt or PyLong instead of a C int, and introduces the C API functions PyNumber_Index, PyNumber_AsSsize_t and PyNumber_AsClippedSsize_t(), all of which have an output variable signifying whether or not the object they received had an ``__index__`` method. .. _PEP 357: http://www.python.org/dev/peps/pep-0357/ .. _patch for fixing __index__: http://bugs.python.org/1530738 Contributing thread: - `Bad interaction of __index__ and sequence repeat `__ -------------------------------------------------------- PEP 302: Non-importer objects on sys.path_importer_cache -------------------------------------------------------- Phillip J. Eby asked about how to best fix the non-PEP-302 compliant changes to the import machinery made by the Need for Speed Sprint. `PEP 302`_ indicates that everything on sys.path_importer_cache should be either None or a valid importer object, but the Need for Speed changes added True and False values to that as well. After getting approval to make the appropriate changes necessary to stay PEP-302-compliant, Phillip added ``imp.NullImporter`` to replace False values, and kept ``None`` to mean that the builtin import machinery should be used. .. _PEP 302: http://www.python.org/dev/peps/pep-0302/ Contributing threads: - `Undocumented PEP 302 protocol change by need-for-speed sprint `__ - `Release manager pronouncement needed: PEP 302 Fix `__ --------------------------------------------------------------- Running the test suites of user projects when Python is updated --------------------------------------------------------------- Grig Gheorghiu volunteered to do some of the work to get community buildbots running, that is, buildbots running the test suites of Python user projects whenever the Python core repository was updated. People were quite enthusiastic and Martin v. L?wis offered to set up a post-commit hook on the python repository to trigger a build on Grig's buildbots if necessary. Contributing threads: - `Community buildbots `__ - `Community buildbots (was Re: User's complaints) `__ - `Community buildbots -- reprise `__ -------------------------------------- Safe dumper/loader using Python syntax -------------------------------------- Sylvain Fourmanoit presented his miniconf_ module which is a safe and cross-version dumper/loader for simple objects using the Python syntax. People generally liked the module, and Phillip J. Eby helped Sylvain clean up the implementation a bit. There was some discussion of including it in the Python 2.6 stdlib and perhaps Bob Ippolito's simplejson_ module alongside it. .. _miniconf: http://cheeseshop.python.org/pypi?:action=display&name=miniconf .. _simplejson: http://undefined.org/python/#simplejson Contributing threads: - `New miniconf module `__ - `JSON implementation in Python 2.6 `__ ------------------------------- Programmatically sending Ctrl-C ------------------------------- In testing his `patch to make sockets and Ctrl-C play nicely`_, Tony Nelson found that he needed a portable way to send a Ctrl-C-like signal. For Unix, he was using ``signal.alarm``, but was wondering if there was a way to get something similar on Windows. Martin v. L?wis pointed out GenerateConsoleCtrlEvent, but also noted that this would send the Ctrl-C to all processes. In the end, Tony decided to punt on Windows, and just stick with the Unix tests. .. _patch to make sockets and Ctrl-C play nicely: http://bugs.python.org/1519025 Contributing threads: - `Socket Timeouts patch 1519025 `__ - `Testing Socket Timeouts patch 1519025 `__ ------------------------------------------ Documenting performance of container types ------------------------------------------ Neal Becker asked about documentation the performance of the basic Python container types, e.g. that lookup in a dict is O(1) and deletion from the beginning of a list is O(N). A number of people agreed that having such information would be helpful, but there was some concern that Guido should be the one to decide what performances guarantees were made by the language, and not just the CPython implementation. The discussion trailed off before any final decisions on how to update the documentation were made. Contributing thread: - `Document performance requirements? `__ --------------------------- Running the uuid test suite --------------------------- Georg Brandl fixed a bug that was causing the new uuid module's test suite not to run at all. The resulting tests indicated a number of problems in determining a MAC address. Neal Norwitz patched the uuid module so that it should work at least on Linux, Tru64, Solaris, and HP-UX, and Tim Peters patched it so that test_uuid no longer thinks that the uuid module knows multiple ways of getting a well-defined MAC address (which it doesn't on Windows). Contributing threads: - `uuid test suite failing `__ - `how about adding ping's uuid module to the standard lib ? `__ - `Another uuid problem `__ - `test_uuid `__ ------------------------------------ CPython and checking for NULL values ------------------------------------ Neal Norwitz took a look at some of the issues raised by the automatic analysis of Python's source code offered by Klocwork_. There was a fairly long discussion around a Py_XINCREF of a variable that was required by the documentation to be non-NULL (and thus the "X" is unnecessary). There was some suggestion of trying to check for NULL values anyway, but the rest of Python doesn't do such checks. .. _Klocwork: http://www.klocwork.com/ Contributing thread: - `remaining issues from Klocwork static analysis `__ --------------------------------------------- Excluding certain constructs from Python code --------------------------------------------- Boris Borcic suggested that to make changing version of Python easier, style sheets should be introduced such that you could allow or disallow particular constructs that you liked or didn't like. People thought this was generally a very bad idea as it would essentially introduce a bunch of language variants, and coders might not be able to read each others' source code without first applying the appropriate transformation. Contributing threads: - `Python Style Sheets ? Re: User's complaints `__ - `Python Style Sheets ? Re: User's complaints `__ --------------------------------------------------------- Making attributes with leading single underscores private --------------------------------------------------------- David Hopwood proposed enforcing the `PEP 8`_ convention that attributes with a single underscore are private to that object. His approach revolved around allowing only the first argument of a function to access attributes starting with '_', but Armin Rigo and others felt that this was not likely to be enforceable, giving an example where subclassing allowed access to supposedly private attributes. People generally felt that without an implementation to back up the proposal, there wasn't much to discuss. .. _PEP 8: http://www.python.org/dev/peps/pep-0008/ Contributing thread: - `Internal namespace proposal `__ ----------------------------------- Loading module attributes on demand ----------------------------------- In order to reduce the memory consumption of GTK+ applications written in Python, Johan Dahlin was looking into dynamically generating module attributes so that they would only be loaded when the application actually accessed them. He was able to produce this behavior by subclassing ModuleType, overridding __getattribute__, and then putting this object onto sys.path, but he felt like this was kind of a hackish solution. Phillip J. Eby pointed out the importing_ package and said that the __getattribute__ approach was generally okay, though it would cause problems for pydoc and inspect which don't handle subclasses of ModuleType well. Andrew Bennetts pointed out mercurial's demandload_ which allows modules to be imported on demand, but this didn't really solve Johan's problem because all attributes of the modules themselves were still imported at the same time. .. _importing: http://cheeseshop.python.org/pypi/Importing .. _demandload: http://selenic.com/repo/hg?f=cb4715847a81;file=mercurial/demandload.py Contributing thread: - `Dynamic module namspaces `__ ------------------------------- Improving the Python test suite ------------------------------- Matt Fleming has put together `a wiki page`_ indicating the tests that are currently missing from Python's test suite, as well as the tests that are incomplete. He plans on working his way through the list when he gets some time, but help for any of the tests is quite welcome. .. _a wiki page: http://wiki.python.org/moin/ImprovingLibTests Contributing thread: - `Improving unit tests for the standard library `__ ---------------------------------- Improving the Python documentation ---------------------------------- Georg Brandl, referring to `bug 469773`_, suggested that python gain a "Using Python" page containing the man page and how to invoke the interpreter. He also suggested that creating a list of frequently needed documentation sections that are hard to find for newbies could go a long way towards making the Python documentation more user-friendly. .. _bug 469773: http://bugs.python.org/469773 Contributing thread: - `Using Python docs `__ ================ Deferred Threads ================ - `struct module and coercing floats to integers `__ - `Rounding float to int directly (Re: struct module and coercing floats to integers) `__ ================== Previous Summaries ================== - `Support for PyGetSetDefs in pydoc `__ =============== Skipped Threads =============== - `Problem with super() usage `__ - `Pronouncement on SF #1520294 sought `__ - `I have submitted a patch that implement IrDA socket support . `__ - `User's complaints `__ - `Pickling objects that return string from reduce `__ - `[Python-checkins] r50708 - in python/trunk: Lib/test/test_sys.py Misc/NEWS Python/pystate.c `__ - `Python sprint in NY and CA, Aug. 21-24 `__ - `Weekly Python Patch/Bug Summary `__ - `os.utime and os.chmod failures (etc) Python 2.5b2 `__ - `FW: Bug? Certainly a new *behavior* from subprocess in 2.5 on Win32 `__ - `Behavior change in subprocess.py `__ - `segfault when using PyGILState_Ensure/Release in Python2.3.4 `__ - `Ireland PyPy sprint 21th-27th August 2006 `__ - `Python 2.4, VS 2005 & Profile Guided Optmization `__ - `Python sprint in Arlington July 29/30 `__ - `setup.py and cross-compiling `__ - `2.5: uses of sys.exc_type, exc_value `__ - `[Windows, buildbot] kill_python.c mystery `__ - `patch for mbcs codec (again) `__ - `Which version of distutils to ship with Python 2.5? `__ - `Patch for building ctypes on more OpenBSD target platforms `__ - `Release manager: pdb bugfix incompatibility `__ - `patching pydoc? `__ - `Fwd: patching pydoc? `__ - `Patch Against shutil.copytree Bug `__ - `httplib and bad response chunking `__ - `cgi.FieldStorage DOS (sf bug #1112549) `__ - `Eliminating loops `__ From guido at python.org Thu Aug 10 23:25:00 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 10 Aug 2006 14:25:00 -0700 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: <20060810133514.pk8p9e2di4a0o4wg@login.werra.lunarpages.com> References: <20060810133514.pk8p9e2di4a0o4wg@login.werra.lunarpages.com> Message-ID: On 8/10/06, Michael Chermside wrote: > See also my example from the beginning of this thread > (http://mail.python.org/pipermail/python-dev/2006-August/067978.html). > The example wasn't from real code, but it WAS quite plausible -- > straightforward use of a popular Python Cookbook recipe. [...] > I don't *strongly* object to this consensus, but if you haven't > glanced at my original example, take a look - it might convince you. Alas, I have no idea what it does. Can you come up with an example that doesn't require enums and localization? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From mal at egenix.com Thu Aug 10 23:28:21 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 10 Aug 2006 23:28:21 +0200 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: <20060810133514.pk8p9e2di4a0o4wg@login.werra.lunarpages.com> References: <20060810133514.pk8p9e2di4a0o4wg@login.werra.lunarpages.com> Message-ID: <44DBA4F5.4080805@egenix.com> Michael Chermside wrote: >> How about we change unicode-vs-str __eq__ to >> issue a warning (and return False) instead of raising >> UnicodeException? > [... Marc-Andre Lemburg agrees ...] >> Great! Now we need someone to volunteer to write a patch (which should >> include doc and NEWS updates) in time for beta 3 (Aug 18). > > I don't *strongly* object to this consensus, but if you haven't > glanced at my original example, take a look - it might convince you. > The proposed solution will not help with my example. I'm not sure the > motivation for breaking code like that example -- the bug-hunting > motivation is satisfied by issuing a warning as Michael Urman proposes, > then use an exception after one more release when people have had time > to fix their code. The warning framework gives programmers great flexibility in handling these situation: * they can silence the warning, just report it once, always report it * they can have the warning raise an exception * they can log the warning in some way It's very flexible to accommodate for all kinds of ways to handle the situation. By introducing a new category of warning for these Unicode-related warnings, adding specific warning filters will be easy. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 10 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Thu Aug 10 23:31:37 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 10 Aug 2006 23:31:37 +0200 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: References: <44DAF23A.103@egenix.com> <44DB4B37.1000401@egenix.com> Message-ID: <44DBA5B9.5030009@egenix.com> Guido van Rossum wrote: > On 8/10/06, M.-A. Lemburg wrote: >> Guido van Rossum wrote: >>> Hmm... Here's an idea... How about we change unicode-vs-str __eq__ to >>> issue a warning (and return False) instead of raising >>> UnicodeException? That won't break much code (it's unlikely that >>> people *depend* on this exception since it's generally a symptom of >>> insane mixing of str and unicode). Then no further changes to >>> dictobject.c are necessary (except fixing that one misleading >>> comment). >> Good idea. > > Great! Now we need someone to volunteer to write a patch (which should > include doc and NEWS updates) in time for beta 3 (Aug 18). I'll give it a go. Not sure what kinds of docs we'll need for this, though (and more importantly: where to put them, e.g. into the tutorial in the dictionary section, somewhere into the Unicode section ?) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 10 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From guido at python.org Thu Aug 10 23:33:38 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 10 Aug 2006 14:33:38 -0700 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: <44DB84EE.8020606@v.loewis.de> References: <44DAF23A.103@egenix.com> <44DB84EE.8020606@v.loewis.de> Message-ID: On 8/10/06, "Martin v. L?wis" wrote: > Guido van Rossum schrieb: > > Hmm... Here's an idea... How about we change unicode-vs-str __eq__ to > > issue a warning (and return False) instead of raising > > UnicodeException? > > I'm in favour of having this __eq__ just return False. I don't think > the warning is necessary, and believe that people will complain about > getting it, but if that is the only way to achieve consensus, > so be it. Not sure I agree. I think that users would find it disturbing (and it could hide bugs) if u"\xff" == "\xff" returned False without warning while u"\x7f" == "\x7f" returns True. > > A warning would arguably have the same helping effect. (I suspect > > actually that we would have used warnings all along except we didn't > > have the warning framework when unicode was first introduced.) > > I believe we would have neither had we rich comparisons available > at the time. That unicode-vs-str-__eq__ raises an exception is just > a fall-out of not supporting __cmp__ (and it is good that __cmp__ > is not supported - there is no meaningful way to order str/unicode > if the string is not in the system encoding). Maybe. I agree that 3-way compare has no business guessing the ordering of u"\xff" and "\xff". I'm not so sure that if we'd had rich comparison we'd have chosen __eq__ to return False for these. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Aug 10 23:36:16 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 10 Aug 2006 14:36:16 -0700 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: <20060810211254.GA24181@code0.codespeak.net> References: <44DAF23A.103@egenix.com> <44DB84EE.8020606@v.loewis.de> <20060810211254.GA24181@code0.codespeak.net> Message-ID: On 8/10/06, Armin Rigo wrote: > Hi, > > On Thu, Aug 10, 2006 at 09:11:42PM +0200, "Martin v. L?wis" wrote: > > I'm in favour of having this __eq__ just return False. I don't think > > the warning is necessary, (...) > > +1 Can you explain why you believe that no warning is necessary? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Aug 10 23:39:04 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 10 Aug 2006 14:39:04 -0700 Subject: [Python-Dev] SyntaxError: can't assign to function call In-Reply-To: <5.1.1.6.0.20060810164933.02713da8@sparrow.telecommunity.com> References: <2m4pwlvezn.fsf@starship.python.net> <20060810084217.1914.JCARLSON@uci.edu> <64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net> <1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net> <5.1.1.6.0.20060810164933.02713da8@sparrow.telecommunity.com> Message-ID: On 8/10/06, Phillip J. Eby wrote: > That being said, the benefit of hypergeneralizing assignment seems small > compared to its price. So eliminating augmented assignment seems a more > attractive way to get rid of the nuisance of the perennially repeated > proposals to "fix" or otherwise extend it. Sometimes it seems like half > the time I want to use augmented assignment, I can't use it anyway for one > reason or another, so it doesn't seem like that big of a loss. I think you are strongly understimating the benefits of augmented assignments. I held out against it for many years. But the users didn't stop asking for it. They're mostly happy. That some aren't happy until we've hypergeneralized it doesn't mean we shouldn't have done it in the first place. (Similar as with lambda. :) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Aug 10 23:44:22 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 10 Aug 2006 14:44:22 -0700 Subject: [Python-Dev] __index__ clipping In-Reply-To: References: <44DB4023.5090904@gmail.com> Message-ID: Thanks for your understanding. Anyway, Nick's patch differs in at least one significant way from my proposal -- (10**10).__index__() returns sys.maxint in his patch, while I want it to return 10000000000L. So this is still an open issue. --Guido On 8/10/06, Travis E. Oliphant wrote: > Guido van Rossum wrote: > > On 8/10/06, Nick Coghlan wrote: > >> Guido van Rossum wrote: > >>>> It seems like Nick's recent patches solved the problems that were > >>>> identified. > >>> Nick, can you summarize how your patches differ from my proposal? > >> nb_index and __index__ are essentially exactly as you propose. > > > > Then I don't understand why Travis is objecting against my proposal! > > I must have missed his most recent patch that changed the result to > return a Python object. I thought earlier versions didn't do that. > > My objection is not particularly solid. At this point it's largely a > wish to avoid the extra overhead (there are some camps that already > complain about NumPy having too much indexing overhead --- although they > should be using Python lists for there purposes if indexing overhead > really is a problem). > > As it appears that several people feel this is the best way forward, > then I'll re-work my NumPy code. I still appreciate the change that > allows other Python objects to be integers and eliminates the "only true > integers allowed" flavor of several places in the Python code. > > Thanks for all your hard work. > > -Travis > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From rrr at ronadam.com Thu Aug 10 23:50:56 2006 From: rrr at ronadam.com (Ron Adam) Date: Thu, 10 Aug 2006 16:50:56 -0500 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: <44DBA4F5.4080805@egenix.com> References: <20060810133514.pk8p9e2di4a0o4wg@login.werra.lunarpages.com> <44DBA4F5.4080805@egenix.com> Message-ID: <44DBAA40.2020502@ronadam.com> M.-A. Lemburg wrote: > Michael Chermside wrote: >>> How about we change unicode-vs-str __eq__ to >>> issue a warning (and return False) instead of raising >>> UnicodeException? >> [... Marc-Andre Lemburg agrees ...] >>> Great! Now we need someone to volunteer to write a patch (which should >>> include doc and NEWS updates) in time for beta 3 (Aug 18). >> I don't *strongly* object to this consensus, but if you haven't >> glanced at my original example, take a look - it might convince you. >> The proposed solution will not help with my example. I'm not sure the >> motivation for breaking code like that example -- the bug-hunting >> motivation is satisfied by issuing a warning as Michael Urman proposes, >> then use an exception after one more release when people have had time >> to fix their code. > > The warning framework gives programmers great flexibility in handling > these situation: > > * they can silence the warning, just report it once, always report it > * they can have the warning raise an exception > * they can log the warning in some way > > It's very flexible to accommodate for all kinds of ways to handle > the situation. > > By introducing a new category of warning for these Unicode-related > warnings, adding specific warning filters will be easy. This sounds like the correct tool to me. +1 Would it be possible to generate warnings when either Unicode or stings are coerced to the other implicitly but not explicitly? That may also generate a warning in the case of the dictionary problem being discussed, (I think), and may be more useful for checking for non-intentionally mixed Unicode and strings. Ron From greg.ewing at canterbury.ac.nz Fri Aug 11 02:20:14 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 11 Aug 2006 12:20:14 +1200 Subject: [Python-Dev] openSSL and windows binaries - license In-Reply-To: <44DAE224.50705@v.loewis.de> References: <44D8E38C.8000505@v.loewis.de> <44D9503D.1040809@canterbury.ac.nz> <44D983E3.701@v.loewis.de> <44DA8D43.4040009@canterbury.ac.nz> <44DAE224.50705@v.loewis.de> Message-ID: <44DBCD3E.8070304@canterbury.ac.nz> Martin v. L?wis wrote: > Perform it: do the steps that the algorithm says you should > do, or let a machine do it. IOW, run the code. That can't be right, because it would mean that anyone who runs a program that contains a patented algorithm, even one bought or otherwise obtained from someone else, would need to individually negotiate a licence with the patent owner. That clearly doesn't happen. > If your point is that software patents are evil: I fully > agree. My point is that software patents don't even seem to make *sense* -- especially in conjunction with open source software. -- Greg From greg.ewing at canterbury.ac.nz Fri Aug 11 02:35:36 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 11 Aug 2006 12:35:36 +1200 Subject: [Python-Dev] Is module clearing still necessary? [Re: Is this a bug?] In-Reply-To: <44DB22D1.60002@gmail.com> References: <20060809084110.E71E.JCARLSON@uci.edu> <44DB22D1.60002@gmail.com> Message-ID: <44DBD0D8.4000404@canterbury.ac.nz> Nick Coghlan wrote: > Early versions of the PEP 338 implementation also ran into the problem of a > module's globals getting cleared when the module itself goes away A while back it was suggested that the module-clearing thing might be dropped now that we have cyclic GC, but I don't remember there being much discussion about it. Has anyone had any thoughts on that? -- Greg From greg.ewing at canterbury.ac.nz Fri Aug 11 03:21:37 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 11 Aug 2006 13:21:37 +1200 Subject: [Python-Dev] SyntaxError: can't assign to function call In-Reply-To: References: <2m4pwlvezn.fsf@starship.python.net> Message-ID: <44DBDBA1.7090402@canterbury.ac.nz> Michael Urman wrote: > Just to play devil's advocate here, why not to a function call via a > new __setcall__? You still haven't answered the question of what this buys you over just giving your object a suitably named method that does whatever your __setcall__ method would have done. Can you show a realistic use case for this, that demonstrates a clear advantage to having new syntax? -- Greg From greg.ewing at canterbury.ac.nz Fri Aug 11 05:42:27 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 11 Aug 2006 15:42:27 +1200 Subject: [Python-Dev] Split augmented assignment into two operator sets? [Re: SyntaxError: can't assign to function call] In-Reply-To: <5.1.1.6.0.20060810164933.02713da8@sparrow.telecommunity.com> References: <1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net> <2m4pwlvezn.fsf@starship.python.net> <20060810084217.1914.JCARLSON@uci.edu> <64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net> <1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net> <5.1.1.6.0.20060810164933.02713da8@sparrow.telecommunity.com> Message-ID: <44DBFCA3.5040708@canterbury.ac.nz> Phillip J. Eby wrote: > I think it's a warning sign because I *know* what augmented assignment's > semantics are supposed to be and I *still* try to use it with setdefault() > sometimes -- but only when I'm doing an augmented assignment. I never > mistakenly try to assign to a function call in any other circumstance. I think this problem arises because of the dual semantics of augmented assignment. One tends to have two *different* mental models of what it does, depending on whether the object in question is immutable: (1) For immutable objects: x += y <--> x = x + y (2) For mutable objects: x += y <--> x.__iadd__(y) and it's easy to forget that case (2) also happens to have the side effect of assigning the result back to x, which is usually redundant in the case of an immutable object, and thus to slip into the error of thinking that x(z) += y <---> x(z).__iadd__(y) should be a reasonable thing to expect. And in the case of a mutable result from x(z), it's hard to argue convincingly that it's not reasonable. As to what this is a warning of, maybe it's that the dual semantics were a mistake in the first place, and that there really should have been two sets of operators, e.g. x += y <---> x = x + y # literally x +: y <---> x.__iadd__(y) # with *no* assignment The +: family of operators would then just be normal infix operators, not assignments at all, and one would be able to say x(z) +: y without any trouble (provided the result of x(z) were mutable, since immutables would not have an __iadd__ method at all). Another benefit to this is that the +: operators would be usable on objects in an outer scope, since they are not assignments and would therefore not imply the localness of anything. Something to consider for Py3k? PS: Why an extra set of operators, rather than ordinary methods? One of the motivations for the augmented assignment operators was Numeric and the desire to be able to express in-place operations on arrays concisely. The +: family would fill this need. PPS: Yes, I know the use of : might be ill-advised. It's just an example - any other concise notation would do as well. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From guido at python.org Fri Aug 11 05:45:07 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 10 Aug 2006 20:45:07 -0700 Subject: [Python-Dev] Split augmented assignment into two operator sets? [Re: SyntaxError: can't assign to function call] In-Reply-To: <44DBFCA3.5040708@canterbury.ac.nz> References: <2m4pwlvezn.fsf@starship.python.net> <20060810084217.1914.JCARLSON@uci.edu> <64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net> <1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net> <5.1.1.6.0.20060810164933.02713da8@sparrow.telecommunity.com> <44DBFCA3.5040708@canterbury.ac.nz> Message-ID: -1. I just don't think we should add more operators -- it'll just cause more question marks on users' faces... --Guido On 8/10/06, Greg Ewing wrote: > Phillip J. Eby wrote: > > > I think it's a warning sign because I *know* what augmented assignment's > > semantics are supposed to be and I *still* try to use it with setdefault() > > sometimes -- but only when I'm doing an augmented assignment. I never > > mistakenly try to assign to a function call in any other circumstance. > > I think this problem arises because of the dual semantics > of augmented assignment. One tends to have two *different* > mental models of what it does, depending on whether the > object in question is immutable: > > (1) For immutable objects: > > x += y <--> x = x + y > > (2) For mutable objects: > > x += y <--> x.__iadd__(y) > > and it's easy to forget that case (2) also happens > to have the side effect of assigning the result > back to x, which is usually redundant in the case > of an immutable object, and thus to slip into the > error of thinking that > > x(z) += y <---> x(z).__iadd__(y) > > should be a reasonable thing to expect. And in > the case of a mutable result from x(z), it's > hard to argue convincingly that it's not > reasonable. > > As to what this is a warning of, maybe it's that > the dual semantics were a mistake in the first > place, and that there really should have been > two sets of operators, e.g. > > x += y <---> x = x + y # literally > > x +: y <---> x.__iadd__(y) # with *no* assignment > > The +: family of operators would then just be > normal infix operators, not assignments at all, > and one would be able to say > > x(z) +: y > > without any trouble (provided the result of x(z) > were mutable, since immutables would not have > an __iadd__ method at all). > > Another benefit to this is that the +: operators > would be usable on objects in an outer scope, > since they are not assignments and would therefore > not imply the localness of anything. > > Something to consider for Py3k? > > PS: Why an extra set of operators, rather than > ordinary methods? One of the motivations for the > augmented assignment operators was Numeric and the > desire to be able to express in-place operations > on arrays concisely. The +: family would fill this > need. > > PPS: Yes, I know the use of : might be ill-advised. > It's just an example - any other concise notation > would do as well. > > -- > Greg Ewing, Computer Science Dept, +--------------------------------------+ > University of Canterbury, | Carpe post meridiem! | > Christchurch, New Zealand | (I'm not a morning person.) | > greg.ewing at canterbury.ac.nz +--------------------------------------+ > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From oliphant.travis at ieee.org Fri Aug 11 06:07:57 2006 From: oliphant.travis at ieee.org (Travis E. Oliphant) Date: Thu, 10 Aug 2006 22:07:57 -0600 Subject: [Python-Dev] __index__ clipping In-Reply-To: References: <44DB4023.5090904@gmail.com> Message-ID: Guido van Rossum wrote: > Thanks for your understanding. > > Anyway, Nick's patch differs in at least one significant way from my > proposal -- (10**10).__index__() returns sys.maxint in his patch, > while I want it to return 10000000000L. So this is still an open > issue. > I've reviewed Nick's patch and finally see the light for the need of nb_index to return a Python object. It also nicely parallels nb_int and nb_long so the same code can often be used (when it's appropriate for the object to be an integer). I understand the general rationale for the C-API, but I share reservations about using output arguments to convey error information. This seems quite a bit different from what Python normally does. I am going to beef up PEP-357 to address the main problem of nb_index returning a Py_ssize_t and then look at the C-API question more closely. I'm also going out of town next week (and have a bunch to prepare for it) so I'll only have a few hours to look at it. -Travis From oliphant.travis at ieee.org Fri Aug 11 06:10:16 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 10 Aug 2006 22:10:16 -0600 Subject: [Python-Dev] __index__ clipping In-Reply-To: References: Message-ID: <44DC0328.9040209@ieee.org> Guido van Rossum wrote: > > What do you think (10**10).__index__() should return (when called from > Python)? > I'm with Guido on this point. I think (10**10).__index__() should return the full long integer when called from within Python. -Travis From aahz at pythoncraft.com Fri Aug 11 06:23:07 2006 From: aahz at pythoncraft.com (Aahz) Date: Thu, 10 Aug 2006 21:23:07 -0700 Subject: [Python-Dev] Split augmented assignment into two operator sets? [Re: SyntaxError: can't assign to function call] In-Reply-To: <44DBFCA3.5040708@canterbury.ac.nz> References: <1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net> <2m4pwlvezn.fsf@starship.python.net> <20060810084217.1914.JCARLSON@uci.edu> <64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net> <1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net> <5.1.1.6.0.20060810164933.02713da8@sparrow.telecommunity.com> <44DBFCA3.5040708@canterbury.ac.nz> Message-ID: <20060811042307.GA19407@panix.com> On Fri, Aug 11, 2006, Greg Ewing wrote: > Phillip J. Eby wrote: >> >> I think it's a warning sign because I *know* what augmented >> assignment's semantics are supposed to be and I *still* try to use it >> with setdefault() sometimes -- but only when I'm doing an augmented >> assignment. I never mistakenly try to assign to a function call in >> any other circumstance. > > I think this problem arises because of the dual semantics of augmented > assignment. One tends to have two *different* mental models of what it > does, depending on whether the object in question is immutable: > > (1) For immutable objects: > > x += y <--> x = x + y > > (2) For mutable objects: > > x += y <--> x.__iadd__(y) What I try to do is always remember that x += y <--> x = x.__iadd__(y) which mostly solves the problem. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." --Brian W. Kernighan From foom at fuhm.net Fri Aug 11 06:53:41 2006 From: foom at fuhm.net (James Y Knight) Date: Fri, 11 Aug 2006 00:53:41 -0400 Subject: [Python-Dev] SyntaxError: can't assign to function call In-Reply-To: <5.1.1.6.0.20060810164933.02713da8@sparrow.telecommunity.com> References: <1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net> <2m4pwlvezn.fsf@starship.python.net> <20060810084217.1914.JCARLSON@uci.edu> <64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net> <1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net> <5.1.1.6.0.20060810164933.02713da8@sparrow.telecommunity.com> Message-ID: <5676E6A1-1FFE-40FC-9A99-90D82B2FE900@fuhm.net> On Aug 10, 2006, at 4:57 PM, Phillip J. Eby wrote: > However, I'm also not clear that trying to assign to a function > call *is* ill-advised. One of the things that attracted me to > Python in the first place is that it had a lot of features that > would be considered "hypergeneralization" in other languages, e.g. > the ability to create your own sequences, mappings, and callable > objects in the first place. > > That being said, the benefit of hypergeneralizing assignment seems > small compared to its price. Well, it's a mostly obvious extension of an existing idea, so the price doesn't seem all that high. The main problem is that so far, there have been 0 convincing use cases. So no matter how moderate the price, it's definitely bigger than the benefit. But anyhow, speaking of hypergeneralization...since this has 0 use cases anyhow, might as well hyperhypergeneralize it... Well, why should assignment be limited to only local variables, item access, and function calls. Why shouldn't you be able to potentially assign to _any_ expression! Since x + a turns into (very roughly...) x.__add__(a), then, x + a = 5 could turn into x.__add__.__setcall__(5, a). Of course, since normal __add__ functions don't have a __setcall__, doing this will raise an error. But, a user defined __add__ could have one! And what would such a user defined __add__.__setcall__ actually *do*? Well, that would be a use case, and I sure don't have any of those! Ta Da. Who's going to make the patch? ;) James From guido at python.org Fri Aug 11 06:58:51 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 10 Aug 2006 21:58:51 -0700 Subject: [Python-Dev] SyntaxError: can't assign to function call In-Reply-To: <5676E6A1-1FFE-40FC-9A99-90D82B2FE900@fuhm.net> References: <2m4pwlvezn.fsf@starship.python.net> <20060810084217.1914.JCARLSON@uci.edu> <64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net> <1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net> <5.1.1.6.0.20060810164933.02713da8@sparrow.telecommunity.com> <5676E6A1-1FFE-40FC-9A99-90D82B2FE900@fuhm.net> Message-ID: On 8/10/06, James Y Knight wrote: > On Aug 10, 2006, at 4:57 PM, Phillip J. Eby wrote: > > That being said, the benefit of hypergeneralizing assignment seems > > small compared to its price. > > Well, it's a mostly obvious extension of an existing idea, so the > price doesn't seem all that high. If you still think that, you don't realize the problem with hypergeneralization. The extension is *not* obvious, and there are several problems (as I pointed out before). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Aug 11 07:47:50 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 10 Aug 2006 22:47:50 -0700 Subject: [Python-Dev] __index__ clipping In-Reply-To: <44DC0FB6.2040206@ieee.org> References: <44DB4023.5090904@gmail.com> <44DC0FB6.2040206@ieee.org> Message-ID: (adding back python-dev in the CC: list) Right. I guess I didn't recompile after patching. Silly me (as Orlijn would say :-). Neal+Anthony, do you need me to review Nick's patch? If I don't have to I'd rather pay more attention to py3k, which I've sadly neglected in the past month (apart from giving a talk about it on at least 4 different occasions :-). --Guido On 8/10/06, Travis Oliphant wrote: > Guido van Rossum wrote: > > Thanks for your understanding. > > > > Anyway, Nick's patch differs in at least one significant way from my > > proposal -- (10**10).__index__() returns sys.maxint in his patch, > > while I want it to return 10000000000L. So this is still an open > > issue. > > Just to clarify. I installed Nick's recent patch and it seems that > (10**10).__index__() actually returns what Guido wants so I think we are > all on the same page. > > Python 2.5b3 (trunk:51199M, Aug 10 2006, 21:59:31) > [GCC 3.4.1 (Mandrakelinux 10.1 3.4.1-4mdk)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> (10**10).__index__() > 10000000000L > > > -Travis > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From g.brandl at gmx.net Fri Aug 11 10:28:09 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 11 Aug 2006 10:28:09 +0200 Subject: [Python-Dev] Is this a bug? In-Reply-To: References: Message-ID: Georg Brandl wrote: > Is this considered a bug? Sure, deleting modules from sys.modules > isn't quite common, but it happened to me on one occasion. > > Python 2.4.3 (#1, Jul 29 2006, 10:52:20) > >>> import logging > >>> import sys > >>> del logging > >>> del sys.modules['logging'] > >>> ^D > Error in atexit._run_exitfuncs: > Traceback (most recent call last): > File "/usr/lib/python2.4/atexit.py", line 24, in _run_exitfuncs > func(*targs, **kargs) > File "/usr/lib/python2.4/logging/__init__.py", line 1328, in shutdown > for h in _handlerList[:]: # was _handlers.keys(): > TypeError: unsubscriptable object > Error in sys.exitfunc: > Traceback (most recent call last): > File "/usr/lib/python2.4/atexit.py", line 24, in _run_exitfuncs > func(*targs, **kargs) > File "/usr/lib/python2.4/logging/__init__.py", line 1328, in shutdown > for h in _handlerList[:]: # was _handlers.keys(): > TypeError: unsubscriptable object I've now fixed the logging issue, but what bothers me additionally is the duplication of tracebacks here. The problem is in atexit._run_exitfuncs: exc_info = None while _exithandlers: func, targs, kargs = _exithandlers.pop() try: func(*targs, **kargs) except SystemExit: exc_info = sys.exc_info() except: import traceback print >> sys.stderr, "Error in atexit._run_exitfuncs:" traceback.print_exc() exc_info = sys.exc_info() if exc_info is not None: raise exc_info[0], exc_info[1], exc_info[2] So the last exception is always reraised and therefore also printed by call_sys_exitfunc. Is this really wanted behavior? Georg From oliphant.travis at ieee.org Fri Aug 11 09:51:04 2006 From: oliphant.travis at ieee.org (Travis E. Oliphant) Date: Fri, 11 Aug 2006 01:51:04 -0600 Subject: [Python-Dev] __index__ clipping In-Reply-To: References: <44DB4023.5090904@gmail.com> <44DC0FB6.2040206@ieee.org> Message-ID: <44DC36E8.6040507@ieee.org> Here is my C-API proposal 1) PyIndex_Check(obj) Similar to PyIter_Check(obj) as it just checks for whether or not the object can call nb_index. Actually implemented as a macro. 2) PyObject* PyNumber_Index(obj) Simple interface around nb_index that calls it if possible and returns TypeError if not (or if the result does not end up in an exact int-or-long 3) Py_ssize_t PyNumber_AsSsize_t(obj, err) This converts the object to a Py_ssize_t by using the nb_index call if possible. The err argument is a Python error object and specifies what error should be raised should the conversion from Int-or-Long to Py_ssize_t cause Overflow. If err is NULL, then no error will be raised and the result will be clipped. Other-wise the corresponding error will be set. Internally PyExc_IndexError and PyExc_OverflowError will be the errors used. -Travis From tomerfiliba at gmail.com Fri Aug 11 11:24:26 2006 From: tomerfiliba at gmail.com (tomer filiba) Date: Fri, 11 Aug 2006 11:24:26 +0200 Subject: [Python-Dev] PyThreadState_SetAsyncExc bug? Message-ID: <1d85506f0608110224s7a398788g1bbab4c97c7979c9@mail.gmail.com> while working on a library for raising exceptions in the context of another thread, i've come across a bug in PyThreadState_SetAsyncExc. if i raise an instance, sys.exc_info() confuses the exception value for the exception type, and the exception value is set None. if i raise the type itself, the interpreter creates an instance internally, but then i can't pass arguments to the exception. code: ===================================== import threading import ctypes def _async_raise(tid, excobj): res = ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, ctypes.py_object(excobj)) if res == 0: raise ValueError("nonexistent thread id") elif res > 1: # """if it returns a number greater than one, you're in trouble, # and you should call it again with exc=NULL to revert the effect""" ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, None) raise SystemError("PyThreadState_SetAsyncExc failed") class Thread(threading.Thread): def raise_exc(self, excobj): assert self.isAlive(), "thread must be started" for tid, tobj in threading._active.items(): if tobj is self: _async_raise(tid, excobj) break # the thread was alive when we entered the loop, but was not found # in the dict, hence it must have been already terminated. should we raise # an exception here? silently ignore? def terminate(self): self.raise_exc(SystemExit()) if __name__ == "__main__": import time import sys i_am_active = False def f(): global i_am_active i_am_active = True try: try: while True: time.sleep(0.01) except IOError, ex: print "IOError handler" except TypeError, ex: print "TypeError handler" print "ex=", repr(ex) typ, val, tb = sys.exc_info() print "typ=", repr(typ) print "val=", repr(val) print "tb=", tb finally: i_am_active = False t1 = Thread(target = f) t1.start() time.sleep(1) t1.raise_exc(TypeError("blah blah")) while i_am_active: time.sleep(0.01) print "!! thread terminated" output: ===================================== TypeError handler ex= None typ= # should be the type val= None # should be the instance tb= !! thread terminated if i change: t1.raise_exc(TypeError("blah blah")) to: t1.raise_exc(TypeError) i get: ===================================== TypeError handler ex= typ= val= tb= !! thread terminated but then of course i can't pass arguments to the exception -tomer From arigo at tunes.org Fri Aug 11 11:27:57 2006 From: arigo at tunes.org (Armin Rigo) Date: Fri, 11 Aug 2006 11:27:57 +0200 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: References: <44DAF23A.103@egenix.com> <44DB84EE.8020606@v.loewis.de> <20060810211254.GA24181@code0.codespeak.net> Message-ID: <20060811092757.GA27929@code0.codespeak.net> Hi, On Thu, Aug 10, 2006 at 02:36:16PM -0700, Guido van Rossum wrote: > > On Thu, Aug 10, 2006 at 09:11:42PM +0200, "Martin v. L?wis" wrote: > > > I'm in favour of having this __eq__ just return False. I don't think > > > the warning is necessary, (...) > > > > +1 > > Can you explain why you believe that no warning is necessary? Ah... mostly out of ignorance, I fear. I did not realize there were strange cases like u"\xff" == "\xff". I will leave this to more unicode-minded people to decide, but I'm still definitely of the idea that the __eq__ should not raise an exception. A bientot, Armin From tomerfiliba at gmail.com Fri Aug 11 11:27:32 2006 From: tomerfiliba at gmail.com (tomer filiba) Date: Fri, 11 Aug 2006 11:27:32 +0200 Subject: [Python-Dev] PyThreadState_SetAsyncExc bug? In-Reply-To: <1d85506f0608110224s7a398788g1bbab4c97c7979c9@mail.gmail.com> References: <1d85506f0608110224s7a398788g1bbab4c97c7979c9@mail.gmail.com> Message-ID: <1d85506f0608110227l283485bei1815d624efe3fe56@mail.gmail.com> opened a new bug: http://sourceforge.net/tracker/index.php?func=detail&aid=1538556&group_id=5470&atid=105470 On 8/11/06, tomer filiba wrote: > while working on a library for raising exceptions in the context > of another thread, i've come across a bug in PyThreadState_SetAsyncExc. > if i raise an instance, sys.exc_info() confuses the exception value for > the exception type, and the exception value is set None. if i raise the > type itself, the interpreter creates an instance internally, but then i can't > pass arguments to the exception. > > code: > ===================================== > import threading > import ctypes > > > def _async_raise(tid, excobj): > res = ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, > ctypes.py_object(excobj)) > if res == 0: > raise ValueError("nonexistent thread id") > elif res > 1: > # """if it returns a number greater than one, you're in trouble, > # and you should call it again with exc=NULL to revert the effect""" > ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, None) > raise SystemError("PyThreadState_SetAsyncExc failed") > > class Thread(threading.Thread): > def raise_exc(self, excobj): > assert self.isAlive(), "thread must be started" > for tid, tobj in threading._active.items(): > if tobj is self: > _async_raise(tid, excobj) > break > > # the thread was alive when we entered the loop, but was not found > # in the dict, hence it must have been already terminated. > should we raise > # an exception here? silently ignore? > > def terminate(self): > self.raise_exc(SystemExit()) > > if __name__ == "__main__": > import time > import sys > > i_am_active = False > > def f(): > global i_am_active > i_am_active = True > try: > try: > while True: > time.sleep(0.01) > except IOError, ex: > print "IOError handler" > except TypeError, ex: > print "TypeError handler" > print "ex=", repr(ex) > typ, val, tb = sys.exc_info() > print "typ=", repr(typ) > print "val=", repr(val) > print "tb=", tb > finally: > i_am_active = False > > t1 = Thread(target = f) > t1.start() > time.sleep(1) > t1.raise_exc(TypeError("blah blah")) > while i_am_active: > time.sleep(0.01) > print "!! thread terminated" > > output: > ===================================== > TypeError handler > ex= None > typ= # should be the type > val= None # should be the instance > tb= > !! thread terminated > > if i change: > t1.raise_exc(TypeError("blah blah")) > > to: > t1.raise_exc(TypeError) > > i get: > ===================================== > TypeError handler > ex= > typ= > val= > tb= > !! thread terminated > > but then of course i can't pass arguments to the exception > > > > -tomer > From mal at egenix.com Fri Aug 11 11:48:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 11 Aug 2006 11:48:25 +0200 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: <20060811092757.GA27929@code0.codespeak.net> References: <44DAF23A.103@egenix.com> <44DB84EE.8020606@v.loewis.de> <20060810211254.GA24181@code0.codespeak.net> <20060811092757.GA27929@code0.codespeak.net> Message-ID: <44DC5269.9030508@egenix.com> Armin Rigo wrote: > Hi, > > On Thu, Aug 10, 2006 at 02:36:16PM -0700, Guido van Rossum wrote: >>> On Thu, Aug 10, 2006 at 09:11:42PM +0200, "Martin v. L?wis" wrote: >>>> I'm in favour of having this __eq__ just return False. I don't think >>>> the warning is necessary, (...) >>> +1 >> Can you explain why you believe that no warning is necessary? > > Ah... mostly out of ignorance, I fear. I did not realize there were > strange cases like u"\xff" == "\xff". This is not all that strange. Perhaps this example looks closer to life ;-) if u'D\xfcsseldorf' != 'D\xfcsseldorf': move_to_Cologne() Now, we wouldn't want that to go through without a warning, do we... ;-) [Background: there's a bit of rivalry between those two cities; see e.g. see http://en.wikipedia.org/wiki/D%C3%BCsseldorf] -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From ndbecker2 at gmail.com Fri Aug 11 12:31:22 2006 From: ndbecker2 at gmail.com (Neal Becker) Date: Fri, 11 Aug 2006 06:31:22 -0400 Subject: [Python-Dev] SyntaxError: can't assign to function call References: <2m4pwlvezn.fsf@starship.python.net> <20060810084217.1914.JCARLSON@uci.edu> <64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net> Message-ID: Guido van Rossum wrote: > On 8/10/06, James Y Knight wrote: >> It makes just as much sense as assigning to an array access, and the >> semantics would be pretty similar. > > No. Array references (x[i]) and attribute references (x.a) represent > "locations". Function calls represent values. This is no different > than the distinction between lvalues and rvalues in C. > Except this syntax is valid in c++ where X() is a constructor call: X(whatever) += 2; is (or can be) valid c++ From oliphant.travis at ieee.org Fri Aug 11 12:53:48 2006 From: oliphant.travis at ieee.org (Travis E. Oliphant) Date: Fri, 11 Aug 2006 04:53:48 -0600 Subject: [Python-Dev] __index__ clipping In-Reply-To: <44DC36E8.6040507@ieee.org> References: <44DB4023.5090904@gmail.com> <44DC0FB6.2040206@ieee.org> <44DC36E8.6040507@ieee.org> Message-ID: Travis E. Oliphant wrote: > Here is my C-API proposal > > 1) PyIndex_Check(obj) > > Similar to PyIter_Check(obj) as it just checks for whether or not the > object can call nb_index. Actually implemented as a macro. > > 2) PyObject* PyNumber_Index(obj) > > Simple interface around nb_index that calls it if possible and returns > TypeError if not (or if the result does not end up in an exact > int-or-long > > 3) Py_ssize_t PyNumber_AsSsize_t(obj, err) > > This converts the object to a Py_ssize_t by using the nb_index call > if possible. The err argument is a Python error object and specifies > what error should be raised should the conversion from Int-or-Long to > Py_ssize_t cause Overflow. > > If err is NULL, then no error will be raised and the result will be > clipped. Other-wise the corresponding error will be set. Internally > PyExc_IndexError and PyExc_OverflowError will be the errors used. > This proposal is implemented as patch 1538606 http://sourceforge.net/tracker/index.php?func=detail&aid=1538606&group_id=5470&atid=305470 -Travis From ncoghlan at gmail.com Fri Aug 11 13:44:26 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 11 Aug 2006 21:44:26 +1000 Subject: [Python-Dev] __index__ clipping In-Reply-To: References: <44DB4023.5090904@gmail.com> <44DC0FB6.2040206@ieee.org> <44DC36E8.6040507@ieee.org> Message-ID: <44DC6D9A.7060401@gmail.com> Travis E. Oliphant wrote: > Travis E. Oliphant wrote: >> Here is my C-API proposal >> >> 1) PyIndex_Check(obj) >> >> Similar to PyIter_Check(obj) as it just checks for whether or not the >> object can call nb_index. Actually implemented as a macro. >> >> 2) PyObject* PyNumber_Index(obj) >> >> Simple interface around nb_index that calls it if possible and returns >> TypeError if not (or if the result does not end up in an exact >> int-or-long >> >> 3) Py_ssize_t PyNumber_AsSsize_t(obj, err) >> >> This converts the object to a Py_ssize_t by using the nb_index call >> if possible. The err argument is a Python error object and specifies >> what error should be raised should the conversion from Int-or-Long to >> Py_ssize_t cause Overflow. >> >> If err is NULL, then no error will be raised and the result will be >> clipped. Other-wise the corresponding error will be set. Internally >> PyExc_IndexError and PyExc_OverflowError will be the errors used. >> > > This proposal is implemented as patch 1538606 > http://sourceforge.net/tracker/index.php?func=detail&aid=1538606&group_id=5470&atid=305470 This interface is a lot simpler to explain and use than the one in my patch, and more in tune with the rest of the C API. I had a minor documentation suggestion which I put on the tracker item, but other than that +1 on using Travis's patch instead of mine (I've already rejected my own tracker item). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From tim.peters at gmail.com Fri Aug 11 13:49:53 2006 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 11 Aug 2006 07:49:53 -0400 Subject: [Python-Dev] PyThreadState_SetAsyncExc bug? In-Reply-To: <1d85506f0608110224s7a398788g1bbab4c97c7979c9@mail.gmail.com> References: <1d85506f0608110224s7a398788g1bbab4c97c7979c9@mail.gmail.com> Message-ID: <1f7befae0608110449o3ca0f5a6gf5edb59a7490705e@mail.gmail.com> [tomer filiba] > while working on a library for raising exceptions in the context > of another thread, i've come across a bug in PyThreadState_SetAsyncExc. > if i raise an instance, sys.exc_info() confuses the exception value for > the exception type, and the exception value is set None. if i raise the > type itself, the interpreter creates an instance internally, but then i can't > pass arguments to the exception. That appears to be the way it was designed; i.e., AFAICT, it's working as intended. This follows from the code in ceval.c that raises the exception: if (tstate->async_exc != NULL) { x = tstate->async_exc; tstate->async_exc = NULL; PyErr_SetNone(x); Py_DECREF(x); why = WHY_EXCEPTION; goto on_error; } PyErr_SetNone(x) there gives no possibility that setting an /instance/ could work as you hope -- `x` has to be an exception type, and tstate->async_exc is simply the `exc` argument that was passed to PyThreadState_SetAsyncExc(). From tomerfiliba at gmail.com Fri Aug 11 14:08:39 2006 From: tomerfiliba at gmail.com (tomer filiba) Date: Fri, 11 Aug 2006 14:08:39 +0200 Subject: [Python-Dev] PyThreadState_SetAsyncExc bug? In-Reply-To: <1f7befae0608110449o3ca0f5a6gf5edb59a7490705e@mail.gmail.com> References: <1d85506f0608110224s7a398788g1bbab4c97c7979c9@mail.gmail.com> <1f7befae0608110449o3ca0f5a6gf5edb59a7490705e@mail.gmail.com> Message-ID: <1d85506f0608110508n148670aq973e81feab4ad245@mail.gmail.com> so it should be fixed, or at least checked for conformness by the code. -tomer On 8/11/06, Tim Peters wrote: > > [tomer filiba] > > while working on a library for raising exceptions in the context > > of another thread, i've come across a bug in PyThreadState_SetAsyncExc. > > if i raise an instance, sys.exc_info() confuses the exception value for > > the exception type, and the exception value is set None. if i raise the > > type itself, the interpreter creates an instance internally, but then i > can't > > pass arguments to the exception. > > That appears to be the way it was designed; i.e., AFAICT, it's working > as intended. This follows from the code in ceval.c that raises the > exception: > > if (tstate->async_exc != NULL) { > x = tstate->async_exc; > tstate->async_exc = NULL; > PyErr_SetNone(x); > Py_DECREF(x); > why = WHY_EXCEPTION; > goto on_error; > } > > PyErr_SetNone(x) there gives no possibility that setting an /instance/ > could work as you hope -- `x` has to be an exception type, and > tstate->async_exc is simply the `exc` argument that was passed to > PyThreadState_SetAsyncExc(). > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060811/93b5d7e3/attachment.html From mcherm at mcherm.com Fri Aug 11 14:37:06 2006 From: mcherm at mcherm.com (Michael Chermside) Date: Fri, 11 Aug 2006 05:37:06 -0700 Subject: [Python-Dev] Dict suppressing exceptions Message-ID: <20060811053706.ljvf9ge314gsg4g0@login.werra.lunarpages.com> Guido writes: > Alas, I have no idea what it does. Can you come up with an example > that doesn't require enums and localization? Sorry. Here's the short version: Fact 1: Sometimes people create objects that raise exceptions when compared for equality. Maybe it's a bad idea to do this, and objects should never raise exceptions when compared except to signal internal inconsistancy -- but the practice is common enough to show up in a 5-star recipe for enums in the Cookbook. Fact 2: Unrelated objects are often put in dictionarys together. I used a "localization dictionary" as an example. I feel this is a legitimate practice. If dictionarys stop suppressing exceptions raised by equality tests then those two facts aren't compatible. Programs that used to work fine will break with 2.4. I'm OK with your proposed solution (fix the case of str-unicode but let everyone else using objects that raise exceptions suffer), but I think it would be more friendly to use warnings for now and exceptions after another full release cycle. The warnings should solve the underlying issue (hard-to-debug problems). -- Michael Chermside From guido at python.org Fri Aug 11 16:56:51 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 11 Aug 2006 07:56:51 -0700 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: <20060811092757.GA27929@code0.codespeak.net> References: <44DAF23A.103@egenix.com> <44DB84EE.8020606@v.loewis.de> <20060810211254.GA24181@code0.codespeak.net> <20060811092757.GA27929@code0.codespeak.net> Message-ID: On 8/11/06, Armin Rigo wrote: > Hi, > > On Thu, Aug 10, 2006 at 02:36:16PM -0700, Guido van Rossum wrote: > > > On Thu, Aug 10, 2006 at 09:11:42PM +0200, "Martin v. L?wis" wrote: > > > > I'm in favour of having this __eq__ just return False. I don't think > > > > the warning is necessary, (...) > > > > > > +1 > > > > Can you explain why you believe that no warning is necessary? > > Ah... mostly out of ignorance, I fear. I did not realize there were > strange cases like u"\xff" == "\xff". I will leave this to more > unicode-minded people to decide, but I'm still definitely of the idea > that the __eq__ should not raise an exception. Me too, and that's what we'll do in py3k. But in 2.5, we're bound by the decisions we made in 1999-2000 about unicode. (Unless Martin has a convincing reason not to have a warning?) Marc-Andre, how's the patch coming along? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Aug 11 16:57:46 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 11 Aug 2006 07:57:46 -0700 Subject: [Python-Dev] __index__ clipping In-Reply-To: <44DC6D9A.7060401@gmail.com> References: <44DB4023.5090904@gmail.com> <44DC0FB6.2040206@ieee.org> <44DC36E8.6040507@ieee.org> <44DC6D9A.7060401@gmail.com> Message-ID: OK, I'll withdraw from this discussion again. Thanks all! On 8/11/06, Nick Coghlan wrote: > Travis E. Oliphant wrote: > > Travis E. Oliphant wrote: > >> Here is my C-API proposal > >> > >> 1) PyIndex_Check(obj) > >> > >> Similar to PyIter_Check(obj) as it just checks for whether or not the > >> object can call nb_index. Actually implemented as a macro. > >> > >> 2) PyObject* PyNumber_Index(obj) > >> > >> Simple interface around nb_index that calls it if possible and returns > >> TypeError if not (or if the result does not end up in an exact > >> int-or-long > >> > >> 3) Py_ssize_t PyNumber_AsSsize_t(obj, err) > >> > >> This converts the object to a Py_ssize_t by using the nb_index call > >> if possible. The err argument is a Python error object and specifies > >> what error should be raised should the conversion from Int-or-Long to > >> Py_ssize_t cause Overflow. > >> > >> If err is NULL, then no error will be raised and the result will be > >> clipped. Other-wise the corresponding error will be set. Internally > >> PyExc_IndexError and PyExc_OverflowError will be the errors used. > >> > > > > This proposal is implemented as patch 1538606 > > http://sourceforge.net/tracker/index.php?func=detail&aid=1538606&group_id=5470&atid=305470 > > This interface is a lot simpler to explain and use than the one in my patch, > and more in tune with the rest of the C API. > > I had a minor documentation suggestion which I put on the tracker item, but > other than that +1 on using Travis's patch instead of mine (I've already > rejected my own tracker item). > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > --------------------------------------------------------------- > http://www.boredomandlaziness.org > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Aug 11 16:59:34 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 11 Aug 2006 07:59:34 -0700 Subject: [Python-Dev] SyntaxError: can't assign to function call In-Reply-To: References: <2m4pwlvezn.fsf@starship.python.net> <20060810084217.1914.JCARLSON@uci.edu> <64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net> Message-ID: On 8/11/06, Neal Becker wrote: > Guido van Rossum wrote: > > > On 8/10/06, James Y Knight wrote: > >> It makes just as much sense as assigning to an array access, and the > >> semantics would be pretty similar. > > > > No. Array references (x[i]) and attribute references (x.a) represent > > "locations". Function calls represent values. This is no different > > than the distinction between lvalues and rvalues in C. > > > > Except this syntax is valid in c++ where X() is a constructor call: > > X(whatever) += 2; is (or can be) valid c++ As I said before, C++ has a fundamentally different concept of what assignment means; it is of no use for understanding Python's assignment. Actually it is a big hindrance knowing about C++ assignment because it's difficult to explain to C++ users why Python can't and won't allow assignment to be overloaded. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From chrisspen at gmail.com Fri Aug 11 17:15:25 2006 From: chrisspen at gmail.com (Chris S) Date: Fri, 11 Aug 2006 11:15:25 -0400 Subject: [Python-Dev] Elementtree and Namespaces in 2.5 Message-ID: I'm happy to see Elementtree being considered for inclusion with 2.5. However, before committing to this decision, there's an issue regarding it's namespace parsing that should be addressed. Although Elmenttree is in most respects an excellent XML parser, a huge gotcha that often makes Elementtree unsuitable for many applications lies in the way it arbitrarily renames namespaces. For example, given: Book Review XML: A Primer AuthorPrice PagesDate Simon St. Laurent 31.98 352 1998/01 Elementtree would rewrite this as: Book Review XML: A Primer AuthorPrice PagesDate Simon St. Laurent 31.98 352 1998/01 There's been some discussion in comp.lang.python about this functionality (http://groups.google.com/group/comp.lang.python/browse_thread/thread/31b2e9f4a8f7338c/363f46513fb8de04?&rnum=3&hl=en) and while most users and the w3 spec (http://www.w3.org/TR/2001/REC-xml-c14n-20010315#NoNSPrefixRewriting) agree this feature is actually a bug, Fredrik Lundh has refused to fix this problem. Of course, this is his right. Unfortunately, Elementtree's design makes a work-around rather awkward. Therefore, we might want to rethink inclusion of Elementtree in the stdlib, or at least patch the stdlib's version of Elementtree to produce an output more in line with the w3 standard. Sincerely, Chris Spencer From mal at egenix.com Fri Aug 11 18:10:31 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 11 Aug 2006 18:10:31 +0200 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: References: <44DAF23A.103@egenix.com> <44DB84EE.8020606@v.loewis.de> <20060810211254.GA24181@code0.codespeak.net> <20060811092757.GA27929@code0.codespeak.net> Message-ID: <44DCABF7.6040709@egenix.com> Guido van Rossum wrote: > Marc-Andre, how's the patch coming along? I'm working on it. Since we only want equal compares to generate the warning, I have to add a rich compare function to Unicode objects. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mcherm at mcherm.com Fri Aug 11 19:38:25 2006 From: mcherm at mcherm.com (Michael Chermside) Date: Fri, 11 Aug 2006 10:38:25 -0700 Subject: [Python-Dev] Elementtree and Namespaces in 2.5 Message-ID: <20060811103825.2s08k51nxnccgcwg@login.werra.lunarpages.com> Chris Spencer writes: > there's an issue > regarding [ElementTree's] namespace parsing that should be addressed. [... it performs namespace rewriting ...] > while most users and the w3 spec > (http://www.w3.org/TR/2001/REC-xml-c14n-20010315#NoNSPrefixRewriting) > agree this feature is actually a bug, Fredrik Lundh has refused to fix > this problem. Of course, this is his right. Unfortunately, > Elementtree's design makes a work-around rather awkward. Therefore, we > might want to rethink inclusion of Elementtree in the stdlib, or at > least patch the stdlib's version of Elementtree to produce an output > more in line with the w3 standard. The good news for you is that: (1) Including ElementTree in the stdlib does not (immediately) take away from any of the other XML librarys out there. (2) Including ElementTree in the stdlib *increases* the chance that requests from users will lead to a change in the library. (3) Changing ElementTree to preserve namespace prefixes would be backward compatible with a version that didn't preserve it, so nothing happening now forecloses fixing this in future releases. Now, in my opinion, the W3 flubbed badly in the way they designed namespaces, prefixes, and allowing namespace prefixes to appear within element content and attributes. Creating a universal namespace with local aliases to prevent name clashes is nice, making the local aliases significant so that we haven't prevented name clashes after all but have merely introduced vast complexity (and *encouraged* clashes by shortening the names) is... not so nice. But that's beside the point. Even if we presume 100% agreement on the need to change ElementTree, the best thing to do is still to release ElementTree in the stdlib exactly as it is in 2.5 and change it for 2.6. -- Michael Chermside From martin at v.loewis.de Fri Aug 11 19:47:44 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 11 Aug 2006 19:47:44 +0200 Subject: [Python-Dev] openSSL and windows binaries - license In-Reply-To: <44DBCD3E.8070304@canterbury.ac.nz> References: <44D8E38C.8000505@v.loewis.de> <44D9503D.1040809@canterbury.ac.nz> <44D983E3.701@v.loewis.de> <44DA8D43.4040009@canterbury.ac.nz> <44DAE224.50705@v.loewis.de> <44DBCD3E.8070304@canterbury.ac.nz> Message-ID: <44DCC2C0.1060703@v.loewis.de> Greg Ewing schrieb: > That can't be right, because it would mean that > anyone who runs a program that contains a > patented algorithm, even one bought or otherwise > obtained from someone else, would need to > individually negotiate a licence with the > patent owner. That clearly doesn't happen. No, that isn't necessary. If you buy a patented screwdriver, and the maker of the screwdriver isn't the patent owner but has an appropriate license, you don't need to negotiate with the patent owner to use the screwdriver. Same with software patents: whoever sold you the software should have negotiated a transferable license that allows use of the algorithm in this software; transfer of the license would likely be bound to usage within this product. So this doesn't get completely OT: The IDEA algorithm is licensed free of charge for non-commercial use (I believe to anybody, ask the patent owner if uncertain); commercial users need to buy a license (I would expect that transferable licenses are also available for sale). Regards, Martin From martin at v.loewis.de Fri Aug 11 20:35:20 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 11 Aug 2006 20:35:20 +0200 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: References: <44DAF23A.103@egenix.com> <44DB84EE.8020606@v.loewis.de> <20060810211254.GA24181@code0.codespeak.net> <20060811092757.GA27929@code0.codespeak.net> Message-ID: <44DCCDE8.70506@v.loewis.de> Guido van Rossum schrieb: > Me too, and that's what we'll do in py3k. But in 2.5, we're bound by > the decisions we made in 1999-2000 about unicode. (Unless Martin has a > convincing reason not to have a warning?) Only the general anti-warning argument: it's not the developer which will see the warning, but the end user. In this case, they might easily get hundreds of warning in a short time, and these fill their Apache log files. They complain about Python eating their disk space. Regards, Martin From theller at python.net Fri Aug 11 21:01:44 2006 From: theller at python.net (Thomas Heller) Date: Fri, 11 Aug 2006 21:01:44 +0200 Subject: [Python-Dev] Errors after running make test Message-ID: There are errors now after the testsuite has finished. Taken from the very end of the amd64 test log (for example). http://www.python.org/dev/buildbot/trunk/amd64%20gentoo%20trunk/builds/1403/step-test/0 [...] 293 tests OK. 26 tests skipped: test_aepack test_al test_applesingle test_bsddb185 test_cd test_cl test_curses test_dl test_gl test_imageop test_imgfile test_linuxaudiodev test_macfs test_macostools test_nis test_ossaudiodev test_pep277 test_plistlib test_rgbimg test_scriptpackages test_startfile test_sunaudiodev test_unicode_file test_winreg test_winsound test_zipfile64 Those skips are all expected on linux2. Error in atexit._run_exitfuncs: Traceback (most recent call last): File "/home/buildslave/buildslave/python/trunk.norwitz-amd64/build/Lib/atexit.py", line 24, in _run_exitfuncs func(*targs, **kargs) File "/home/buildslave/buildslave/python/trunk.norwitz-amd64/build/Lib/logging/__init__.py", line 1351, in shutdown h.flush() File "/home/buildslave/buildslave/python/trunk.norwitz-amd64/build/Lib/logging/__init__.py", line 731, in flush self.stream.flush() ValueError: I/O operation on closed file Error in sys.exitfunc: Traceback (most recent call last): File "/home/buildslave/buildslave/python/trunk.norwitz-amd64/build/Lib/atexit.py", line 24, in _run_exitfuncs func(*targs, **kargs) File "/home/buildslave/buildslave/python/trunk.norwitz-amd64/build/Lib/logging/__init__.py", line 1351, in shutdown h.flush() File "/home/buildslave/buildslave/python/trunk.norwitz-amd64/build/Lib/logging/__init__.py", line 731, in flush self.stream.flush() ValueError: I/O operation on closed file [449107 refs] program finished with exit code 0 From martin at v.loewis.de Fri Aug 11 21:08:51 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 11 Aug 2006 21:08:51 +0200 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: <20060810133514.pk8p9e2di4a0o4wg@login.werra.lunarpages.com> References: <20060810133514.pk8p9e2di4a0o4wg@login.werra.lunarpages.com> Message-ID: <44DCD5C3.6040907@v.loewis.de> Michael Chermside schrieb: > I don't *strongly* object to this consensus, but if you haven't > glanced at my original example, take a look - it might convince you. > The proposed solution will not help with my example. I ignored your example the first time because it was too complicated to understand. Now I looked at it, and think that the recipe is broken. It should add an __eq__ method which is def __eq__(self, other): return type(self) is type(other) \ and self.EnumType is other.EnumType \ and self.__value==other.__value) Regards, Martin From martin at v.loewis.de Fri Aug 11 21:35:52 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 11 Aug 2006 21:35:52 +0200 Subject: [Python-Dev] Elementtree and Namespaces in 2.5 In-Reply-To: References: Message-ID: <44DCDC18.3000701@v.loewis.de> Chris S schrieb: > I'm happy to see Elementtree being considered for inclusion with 2.5. > However, before committing to this decision, there's an issue > regarding it's namespace parsing that should be addressed. Although > Elmenttree is in most respects an excellent XML parser, a huge gotcha > that often makes Elementtree unsuitable for many applications lies in > the way it arbitrarily renames namespaces. Terminology alert: it doesn't rename namespaces. It renames namespace prefixes. The namespaces stay unmodified. > functionality (http://groups.google.com/group/comp.lang.python/browse_thread/thread/31b2e9f4a8f7338c/363f46513fb8de04?&rnum=3&hl=en) > and while most users and the w3 spec > (http://www.w3.org/TR/2001/REC-xml-c14n-20010315#NoNSPrefixRewriting) > agree this feature is actually a bug, Fredrik Lundh has refused to fix > this problem. You are misunderstanding. The "w3 spec" does not say it is a bug; you are looking at the wrong spec. The right spec to look at is http://www.w3.org/TR/REC-xml-names/ You are looking at a resolution of an issue in the spec of Canonical XML (i.e. not even the actual spec, just the resolution of an issue). Now, C14N specifies that the canonical form of an XML document has the same namespace prefixes as the original document. So ElementTree does not implement C14N for that reason, but I guess there are many more reasons why ElementTree is not a C14N implementation. For example, it puts the attributes before the namespace attributes, when C14N says it should be the other way 'round. > Of course, this is his right. Unfortunately, > Elementtree's design makes a work-around rather awkward. Therefore, we > might want to rethink inclusion of Elementtree in the stdlib That is not enough reason. Yes, it makes certain applications impossible, e.g. when namespace prefixes are inside attribute values. It just means you can't use it for that application, then. XML has many other applications, and so does ElementTree. > or at > least patch the stdlib's version of Elementtree to produce an output > more in line with the w3 standard. That is out of the question. It was agreed that ElementTree is added to the standard library only if Fredrik Lundh gets veto rights on all changes. Regards, Martin From martin at v.loewis.de Fri Aug 11 21:40:50 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 11 Aug 2006 21:40:50 +0200 Subject: [Python-Dev] SyntaxError: can't assign to function call In-Reply-To: References: <2m4pwlvezn.fsf@starship.python.net> Message-ID: <44DCDD42.1010202@v.loewis.de> Michael Urman schrieb: > On 8/9/06, Michael Hudson wrote: >> The question doesn't make sense: in Python, you assign to a name, >> an attribute or a subscript, and that's it. > > Just to play devil's advocate here, why not to a function call via a > new __setcall__? Just try specifying this one day. I'm sure a dozen readers of the list will rip apart the first specification you make as underspecified or unimplementable (hint: what is the precise syntax for the left-hand side? how to parse it? what is the precise parameter list?) Regards, Martin From martin at v.loewis.de Fri Aug 11 21:48:43 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 11 Aug 2006 21:48:43 +0200 Subject: [Python-Dev] SyntaxError: can't assign to function call In-Reply-To: References: <2m4pwlvezn.fsf@starship.python.net> <20060810084217.1914.JCARLSON@uci.edu> <64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net> Message-ID: <44DCDF1B.2010804@v.loewis.de> Neal Becker schrieb: >> No. Array references (x[i]) and attribute references (x.a) represent >> "locations". Function calls represent values. This is no different >> than the distinction between lvalues and rvalues in C. >> > > Except this syntax is valid in c++ where X() is a constructor call: > > X(whatever) += 2; is (or can be) valid c++ That's actually the less-interesting case. You would have to overload += to make it work, right? The more interesting case is when X is a function that returns a reference: int& X(int); void foo(){ X(1) += 2; } int bar, foobar; int& X(int t){ if(t) return bar; return foobar; } Here, which variable gets incremented depends on whether the t argument is true; no overloading of assignment comes into play. The trick is that C++ has functions that *return* lvalues; neither C nor Python has such things. Regards, Martin From martin at v.loewis.de Fri Aug 11 21:52:02 2006 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Fri, 11 Aug 2006 21:52:02 +0200 Subject: [Python-Dev] Unicode Data in Python2.5 is missing a ucd_4_1_0 object In-Reply-To: References: <44DB838A.10600@v.loewis.de> Message-ID: <44DCDFE2.5000705@v.loewis.de> Armin Ronacher schrieb: > Right, I didn't know that. From that old bug report it sounded like a programmer > should be able to select a specific UCD version. Well, my comment was that _IDNA_ needs to access _3.2_. This isn't as general as "any application needs to access any version". We likely drop 3.2 when IDNA stops using it, and we likely drop 4.1 when moving to 5.0. Of course, now the infrastructure is there for keeping old versions efficiently, so if compelling reasons are brought forward to keep an old version, it would be possible. Regards, Martin From pje at telecommunity.com Fri Aug 11 21:53:08 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 11 Aug 2006 15:53:08 -0400 Subject: [Python-Dev] SyntaxError: can't assign to function call In-Reply-To: <44DCDD42.1010202@v.loewis.de> References: <2m4pwlvezn.fsf@starship.python.net> Message-ID: <5.1.1.6.0.20060811154306.03be1b28@sparrow.telecommunity.com> At 09:40 PM 8/11/2006 +0200, Martin v. L?wis wrote: >Michael Urman schrieb: > > On 8/9/06, Michael Hudson wrote: > >> The question doesn't make sense: in Python, you assign to a name, > >> an attribute or a subscript, and that's it. > > > > Just to play devil's advocate here, why not to a function call via a > > new __setcall__? > >Just try specifying this one day. I'm sure a dozen readers of the list >will rip apart the first specification you make as underspecified or >unimplementable (hint: what is the precise syntax for the left-hand >side? how to parse it? what is the precise parameter list?) Actually, this isn't as hard as you're implying. In at least the "compiler.ast" package, such an operation would be represented as a CallFunc node as the child of an Assign node. Wrapping the call node's main child expression in a Getattr for __setcall__ would then effect the relatively simple AST transformation required. This is because Python's grammar *already* allows arbitrary LHS expressions; it's just that the compiler rejects LHS nodes that aren't a Getattr, Subscript, or Name. (The above should not be confused with advocacy for the feature; I'm just pointing out that specifying it -- or even implementing it -- isn't that complicated.) From martin at v.loewis.de Fri Aug 11 22:10:20 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 11 Aug 2006 22:10:20 +0200 Subject: [Python-Dev] SyntaxError: can't assign to function call In-Reply-To: <5.1.1.6.0.20060811154306.03be1b28@sparrow.telecommunity.com> References: <2m4pwlvezn.fsf@starship.python.net> <5.1.1.6.0.20060811154306.03be1b28@sparrow.telecommunity.com> Message-ID: <44DCE42C.6030708@v.loewis.de> Phillip J. Eby schrieb: > Actually, this isn't as hard as you're implying. In at least the > "compiler.ast" package, such an operation would be represented as a > CallFunc node as the child of an Assign node. Wrapping the call node's > main child expression in a Getattr for __setcall__ would then effect the > relatively simple AST transformation required. Well, you are experienced with that stuff, and would be one of the people who could rip apart a specification attempt. There are just many details to it, for example, what is the evaluation order when I write f(g()) = h() Or what about chained assignments a = b() = c() Is the return value of __setcall__ used or not? What about augmented assignments? Any specific question that somebody asks gets an easy (although perhaps arbitrary) answer. Still, I guess many people would be surprised how large the specification of a seemingly simple feature will be. Regards, Martin From jimjjewett at gmail.com Fri Aug 11 22:22:07 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri, 11 Aug 2006 16:22:07 -0400 Subject: [Python-Dev] Dict suppressing exceptions Message-ID: Martin v. L?wis wrote: > Now I looked at it, and think that the recipe is broken. It should > add an __eq__ method which is > def __eq__(self, other): > return type(self) is type(other) \ > and self.EnumType is other.EnumType \ > and self.__value==other.__value) Absolutely. But the point is that there are a lot of these broken objects out there; this one was on a 5-star recipe that has been around for a while, and still no one caught it. That suggests the bug could reasonably be in any 3rd-party library. The existence of a (common) bug in someone else's code shouldn't keep me from using a dictionary of objects. hash was just changed to support the common use of id, even though some people argued it was *really* a bug in the classes themselves. This is a similar situation. -jJ From mcherm at mcherm.com Fri Aug 11 23:02:56 2006 From: mcherm at mcherm.com (Michael Chermside) Date: Fri, 11 Aug 2006 14:02:56 -0700 Subject: [Python-Dev] Dict suppressing exceptions Message-ID: <20060811140256.rg49nou0ojhw8cgs@login.werra.lunarpages.com> Martin v. L?wis writes: > Now I looked at it, and think that the recipe is broken. Not broken, but perhaps wrongheaded. The recipe went out of its way to ensure that it would raise an exception of enum values from different enumerations were compared. There's nothing out there saying that this is a bad thing to do. I propose that we institute a new policy. The policy should state: __eq__ methods should always return True or False. They should only raise an exception if there is some internal error within one of the objects being compared -- they should never raise an exception because the other object is of an unexpected type. On the other hand, __lt__, __gt__ and friends SHOULD raise an exception when the object being compared is of some type for which the ordering does not make sense (e.g.: unicode vs byte-string or complex vs anything). I think we should note this policy someplace official -- perhaps in the Language Reference where __eq__ and __lt__ are defined. But I do not think that these changes should be made until Py3K. What do others think? Is this the "right" approach? -- Michael Chermside From mal at egenix.com Fri Aug 11 23:11:10 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 11 Aug 2006 23:11:10 +0200 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: <44DCABF7.6040709@egenix.com> References: <44DAF23A.103@egenix.com> <44DB84EE.8020606@v.loewis.de> <20060810211254.GA24181@code0.codespeak.net> <20060811092757.GA27929@code0.codespeak.net> <44DCABF7.6040709@egenix.com> Message-ID: <44DCF26E.6090003@egenix.com> M.-A. Lemburg wrote: > Guido van Rossum wrote: >> Marc-Andre, how's the patch coming along? > > I'm working on it. > > Since we only want equal compares to generate the warning, > I have to add a rich compare function to Unicode objects. Here's an initial version: http://sourceforge.net/tracker/index.php?func=detail&aid=1538956&group_id=5470&atid=305470 The title of the patch is slightly incorrect - SF doesn't allow more descriptive titles... :-( -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From martin at v.loewis.de Fri Aug 11 23:54:19 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 11 Aug 2006 23:54:19 +0200 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: <20060811140256.rg49nou0ojhw8cgs@login.werra.lunarpages.com> References: <20060811140256.rg49nou0ojhw8cgs@login.werra.lunarpages.com> Message-ID: <44DCFC8B.3070100@v.loewis.de> Michael Chermside schrieb: > Not broken, but perhaps wrongheaded. The recipe went out of its way > to ensure that it would raise an exception of enum values from different > enumerations were compared. There's nothing out there saying that this > is a bad thing to do. And it's actually fine for three-way compare: different enums don't naturally arrange. > I propose that we institute a new policy. The policy should state: > > __eq__ methods should always return True or False. They should > only raise an exception if there is some internal error within > one of the objects being compared -- they should never raise > an exception because the other object is of an unexpected type. That policy is currently difficult to implement, but reasonable (difficult because it is quite some code to write). > On the other hand, __lt__, __gt__ and friends SHOULD raise an > exception when the object being compared is of some type for > which the ordering does not make sense (e.g.: unicode vs > byte-string or complex vs anything). Right; same for three-way compare. > What do others think? Is this the "right" approach? For the moment, the first section gets augmented by "issue a warning if you think the user is comparing things incorrectly". Regards, Martin From alexander.belopolsky at gmail.com Sat Aug 12 00:07:44 2006 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Fri, 11 Aug 2006 18:07:44 -0400 Subject: [Python-Dev] What is the status of file.readinto? Message-ID: I know that it is a subject of an annual discussion, but since I could not find any mention of it in the last year, let me ask this question again: why is file.readinto "Undocumented"? It is quite useful, particularly with numpy. Would a patch adding "readinto" to StringIO be acceptable? From ncoghlan at iinet.net.au Sat Aug 12 04:47:10 2006 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Sat, 12 Aug 2006 12:47:10 +1000 Subject: [Python-Dev] 2.6 idea: a 'function' builtin to parallel classmethod and staticmethod Message-ID: <44DD412E.9060109@iinet.net.au> It's sometimes useful to be able to use an existing callable as a method of a new class. If the callable is a real function, this is easy. You just including the following line in the class definition: method = some_callable However, callable objects without a function-like __get__ method can't be used that way. So, to avoid a dependency on an implementation detail of some_callable (i.e. whether or not it is a true function object), you have to write: def method(): return some_callable() (and you can lose useful metadata in the process!) However, if you're adding a callable as a class method or static method, there is OOWTDI: method = classmethod(some_callable) method = staticmethod(some_callable) It would be nice if there was a similar mechanism for normal instance methods as well: method = function(some_callable) This came up during the PEP 343 implementation - "context = contextlib.closing" is a tempting thing to write in order to provide a "x.context()" method for use in a with statement, but it doesn't actually work properly (because closing is a class, not a function). Similarly, you can't create a method simply by applying functools.partial to an existing function - the result won't have a __get__ method, so it will be treated like a normal attribute instead of as an instance method. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From guido at python.org Sat Aug 12 06:03:40 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 11 Aug 2006 21:03:40 -0700 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: <20060811140256.rg49nou0ojhw8cgs@login.werra.lunarpages.com> References: <20060811140256.rg49nou0ojhw8cgs@login.werra.lunarpages.com> Message-ID: On 8/11/06, Michael Chermside wrote: > Martin v. L?wis writes: > > Now I looked at it, and think that the recipe is broken. > > Not broken, but perhaps wrongheaded. The recipe went out of its way > to ensure that it would raise an exception of enum values from different > enumerations were compared. There's nothing out there saying that this > is a bad thing to do. Except a little voice in my head. :-) > I propose that we institute a new policy. The policy should state: > > __eq__ methods should always return True or False. They should > only raise an exception if there is some internal error within > one of the objects being compared -- they should never raise > an exception because the other object is of an unexpected type. That's pretty much what I want to do in py3k. There, ordering will raise a TypeError by default, but == compares object identity by default, and it's strongly recommended to return False for any unrecognized type. I want the recommendation to be a *little* more relaxed than your proposal; IMO it's fine to raise an exception when your __eq__ implementation is incomplete, i.e. you haven't thought enough about how your class should compare to other classes. The "don't silence exceptions" rule will then cause loud complaints when your class gets compared to something it didn't expect being compared to, signalling that you have more work to do. (And yes, this can cause sleeper bombs when you use these objects as dict keys together with types that it doesn't understand, and the bomb will go off if there's a hash collision; but I've got fairly controlled situations in mind, not robust libraries or frameworks.) > On the other hand, __lt__, __gt__ and friends SHOULD raise an > exception when the object being compared is of some type for > which the ordering does not make sense (e.g.: unicode vs > byte-string or complex vs anything). Yes, that's the py3k rule. > I think we should note this policy someplace official -- perhaps > in the Language Reference where __eq__ and __lt__ are defined. But > I do not think that these changes should be made until Py3K. Fine to update the docs now. > What do others think? Is this the "right" approach? Yes. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Aug 12 06:06:13 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 11 Aug 2006 21:06:13 -0700 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: <44DCFC8B.3070100@v.loewis.de> References: <20060811140256.rg49nou0ojhw8cgs@login.werra.lunarpages.com> <44DCFC8B.3070100@v.loewis.de> Message-ID: On 8/11/06, "Martin v. L?wis" wrote: > Michael Chermside schrieb: > > I propose that we institute a new policy. The policy should state: > > > > __eq__ methods should always return True or False. They should > > only raise an exception if there is some internal error within > > one of the objects being compared -- they should never raise > > an exception because the other object is of an unexpected type. > > That policy is currently difficult to implement, but reasonable > (difficult because it is quite some code to write). Why? Are you thinking of the standard library, or of an end user's __eq__ method? Returning False from your __eq__ if other's type is unexpected doesn't seem a lot of code. Or am I misunderstanding something? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Sat Aug 12 07:57:00 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 12 Aug 2006 07:57:00 +0200 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: References: <20060811140256.rg49nou0ojhw8cgs@login.werra.lunarpages.com> <44DCFC8B.3070100@v.loewis.de> Message-ID: <44DD6DAC.9060400@v.loewis.de> Guido van Rossum schrieb: >> > __eq__ methods should always return True or False. They should >> > only raise an exception if there is some internal error within >> > one of the objects being compared -- they should never raise >> > an exception because the other object is of an unexpected type. >> >> That policy is currently difficult to implement, but reasonable >> (difficult because it is quite some code to write). > > Why? Are you thinking of the standard library, or of an end user's > __eq__ method? Returning False from your __eq__ if other's type is > unexpected doesn't seem a lot of code. Or am I misunderstanding > something? It seemed like a lot of code to me: In the specific enum example, I first wrote def __eq__(self, other): return self.EnumType is other.EnumType \ and self.__value==other.__value So this was wrong, as it did not take into account 'other' being something completely different, and I wrote def __eq__(self, other): return type(self) is type(other) \ and self.EnumType is other.EnumType \ and self.__value==other.__value Having two additional continuation lines seems quite difficult to me, yet a version that breaks the expression into multiple statements is even longer def __eq__(self, other): if type(self) is not type(other): return False if self.EnumType is not other.EnumType: return False return self.__value==other.__value Compare this to the current two-line __cmp__ implementation: def __cmp__(self, other): assert self.Enumtype is other.EnumType return cmp(self.__value, other.__value) This covers all comparisons just fine in two lines of method body; to implement the __eq__ policy, you need another 6 lines. For consistency, you should also implement __ne__, probably as def __ne__(self, other): return not self.__eq__(other) I expect many people get this wrong, for example http://pyref.infogami.com/__eq__ UserList.py Also, there is confusion as to whether NotImplemented should ever be returned in these. decimal.py believes it does (for different types), sets.py believes it doesn't. Regards, Martin From martin at v.loewis.de Sat Aug 12 08:14:04 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 12 Aug 2006 08:14:04 +0200 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: References: Message-ID: <44DD71AC.5060402@v.loewis.de> Jim Jewett schrieb: > hash was just changed to support the common use of id, even though > some people argued it was *really* a bug in the classes themselves. > This is a similar situation. Similar still different. In the case of __hash__ returning id(), it is very clear what the intention of the program is, and it is implementable to give the program its intended meaning (i.e. use the object address as the source of the hash value). In this case, it is not clear what the intention of the program is. __cmp__ raises an exception, and you say the intention is that this exception should be discarded? Errors should never pass silently, unless explicitly silenced. Some users will find "Oops, I didn't know all these years I had this error there", and correct their code. Some will say "why did they break my program", and correct their code - the corrected code will run just fine on older versions of Python as well. Regards, Martin From martin at v.loewis.de Sat Aug 12 08:20:36 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 12 Aug 2006 08:20:36 +0200 Subject: [Python-Dev] 2.6 idea: a 'function' builtin to parallel classmethod and staticmethod In-Reply-To: <44DD412E.9060109@iinet.net.au> References: <44DD412E.9060109@iinet.net.au> Message-ID: <44DD7334.2070003@v.loewis.de> Nick Coghlan schrieb: > It would be nice if there was a similar mechanism for normal instance methods > as well: > > method = function(some_callable) > If you know you have to put something around it, what's wrong with writing method = lambda *args:some_callable(*args) If that happens often enough, you can write def function(f): return lambda *args:f(*args) Regards, Martin From ncoghlan at gmail.com Sat Aug 12 08:38:49 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 12 Aug 2006 16:38:49 +1000 Subject: [Python-Dev] 2.6 idea: a 'function' builtin to parallel classmethod and staticmethod In-Reply-To: <44DD7334.2070003@v.loewis.de> References: <44DD412E.9060109@iinet.net.au> <44DD7334.2070003@v.loewis.de> Message-ID: <44DD7779.4060404@gmail.com> Martin v. L?wis wrote: > Nick Coghlan schrieb: >> It would be nice if there was a similar mechanism for normal instance methods >> as well: >> >> method = function(some_callable) >> > > If you know you have to put something around it, what's wrong with writing > > method = lambda *args:some_callable(*args) > > If that happens often enough, you can write > > def function(f): > return lambda *args:f(*args) Both of those are inferior to defining a new method with the right name and arguments and a docstring directing readers to the relevant function (which is what I would do for this situation at the moment). The idea of the builtin would be to let introspection tools know that the documentation of the existing callable applies to the method as well, so that things like help(x.method) could retrieve something more useful than: >>> help(C().method) Help on method method in module __main__: method(*args, **kwds) method of __main__.C instance Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From tim.peters at gmail.com Sat Aug 12 09:24:05 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sat, 12 Aug 2006 03:24:05 -0400 Subject: [Python-Dev] Recent logging spew Message-ID: <1f7befae0608120024q24cd5474p5aa0a3d5f13eae0b@mail.gmail.com> We're getting an awful lot of these turds at the ends of test runs now: Error in atexit._run_exitfuncs: Traceback (most recent call last): File "C:\Code\bb_slave\trunk.peters-windows\build\lib\atexit.py", line 24, in _run_exitfuncs func(*targs, **kargs) File "C:\Code\bb_slave\trunk.peters-windows\build\lib\logging\__init__.py", line 1351, in shutdown h.flush() File "C:\Code\bb_slave\trunk.peters-windows\build\lib\logging\__init__.py", line 731, in flush self.stream.flush() ValueError: I/O operation on closed file Error in sys.exitfunc: Traceback (most recent call last): File "C:\Code\bb_slave\trunk.peters-windows\build\lib\atexit.py", line 24, in _run_exitfuncs func(*targs, **kargs) File "C:\Code\bb_slave\trunk.peters-windows\build\lib\logging\__init__.py", line 1351, in shutdown h.flush() File "C:\Code\bb_slave\trunk.peters-windows\build\lib\logging\__init__.py", line 731, in flush self.stream.flush() ValueError: I/O operation on closed file [450168 refs] program finished with exit code 0 Not sure what caused it, but suspect this: """ Author: georg.brandl Date: Fri Aug 11 09:26:10 2006 New Revision: 51206 Modified: python/trunk/Lib/logging/__init__.py python/trunk/Misc/NEWS Log: logging's atexit hook now runs even if the rest of the module has already been cleaned up. """ Anyway, can we stop this? It doesn't look like "a feature" from here ;-) From g.brandl at gmx.net Sat Aug 12 10:33:21 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 12 Aug 2006 10:33:21 +0200 Subject: [Python-Dev] Recent logging spew In-Reply-To: <1f7befae0608120024q24cd5474p5aa0a3d5f13eae0b@mail.gmail.com> References: <1f7befae0608120024q24cd5474p5aa0a3d5f13eae0b@mail.gmail.com> Message-ID: Tim Peters wrote: > We're getting an awful lot of these turds at the ends of test runs now: > > Error in atexit._run_exitfuncs: > Not sure what caused it, but suspect this: > > """ > Author: georg.brandl > Date: Fri Aug 11 09:26:10 2006 > New Revision: 51206 > > Modified: > python/trunk/Lib/logging/__init__.py > python/trunk/Misc/NEWS > Log: > logging's atexit hook now runs even if the rest of the module has > already been cleaned up. > """ > > Anyway, can we stop this? It doesn't look like "a feature" from here ;-) If it was a feature, it couldn't have gone in right now ;-) But yes, I confirmed that that change caused it. I've now found the problem: one of the test_logging tests reassigned _handlerList, causing it to go out of sync with the handlerList stored in shutdown(). Fixed in rev. 51235. Georg From tomerfiliba at gmail.com Sat Aug 12 15:42:17 2006 From: tomerfiliba at gmail.com (tomer filiba) Date: Sat, 12 Aug 2006 15:42:17 +0200 Subject: [Python-Dev] dict containment annoyance Message-ID: <1d85506f0608120642y398e3fb4p9ce56c51e915b6d4@mail.gmail.com> >>> a={1:2, 3:4} >>> [] in a Traceback (most recent call last): File "", line 1, in ? TypeError: list objects are unhashable >>> imo, the expression should just evaluate to False instead of raising an exception. it's a question of semantics -- i asked whether the object (a list, in this case) is contained in the dict. i didn't ask whether it's hashable or not. if the object is unhashable, then surely it's not contained in the dict, therefore the answer is False, rather than an exception. dict.__contains__ should just swallow these exceptions, and return False. -tomer -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060812/6a4c7877/attachment.html From aahz at pythoncraft.com Sat Aug 12 15:42:44 2006 From: aahz at pythoncraft.com (Aahz) Date: Sat, 12 Aug 2006 06:42:44 -0700 Subject: [Python-Dev] [Python-3000] Python 2.5 release schedule (was: threading, part 2) In-Reply-To: <20060812012526.195B.JCARLSON@uci.edu> References: <20060812012526.195B.JCARLSON@uci.edu> Message-ID: <20060812134244.GA29374@panix.com> [added python-dev to make sure everyone sees this] On Sat, Aug 12, 2006, Josiah Carlson wrote: > > All other feature additions are too late in the Beta cycle (Beta 3 is > next week) For some reason, this is the second time I've seen this claim. Beta 3 was released August 3 and next week is rc1. We are right now in complete feature lockdown; even documenting an existing API IMO requires approval from the Release Manager. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." --Brian W. Kernighan From aahz at pythoncraft.com Sat Aug 12 15:48:26 2006 From: aahz at pythoncraft.com (Aahz) Date: Sat, 12 Aug 2006 06:48:26 -0700 Subject: [Python-Dev] dict containment annoyance In-Reply-To: <1d85506f0608120642y398e3fb4p9ce56c51e915b6d4@mail.gmail.com> References: <1d85506f0608120642y398e3fb4p9ce56c51e915b6d4@mail.gmail.com> Message-ID: <20060812134826.GC29374@panix.com> On Sat, Aug 12, 2006, tomer filiba wrote: > > >>>a={1:2, 3:4} > >>>[] in a > Traceback (most recent call last): > File "", line 1, in ? > TypeError: list objects are unhashable > >>> > > imo, the expression should just evaluate to False instead of raising an > exception. -1 This is seriously no different from an attempt to do >>> a = {} >>> a[ [] ] = 1 -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." --Brian W. Kernighan From david.nospam.hopwood at blueyonder.co.uk Sat Aug 12 18:26:33 2006 From: david.nospam.hopwood at blueyonder.co.uk (David Hopwood) Date: Sat, 12 Aug 2006 17:26:33 +0100 Subject: [Python-Dev] dict containment annoyance In-Reply-To: <1d85506f0608120642y398e3fb4p9ce56c51e915b6d4@mail.gmail.com> References: <1d85506f0608120642y398e3fb4p9ce56c51e915b6d4@mail.gmail.com> Message-ID: <44DE0139.5080108@blueyonder.co.uk> tomer filiba wrote: >>>> a={1:2, 3:4} >>>> [] in a > > Traceback (most recent call last): > File "", line 1, in ? > TypeError: list objects are unhashable > > imo, the expression should just evaluate to False instead of raising an > exception. it's a question of semantics -- i asked whether the object > (a list, in this case) is contained in the dict. i didn't ask whether > it's hashable or not. However, if it isn't hashable, asking whether it is in a dict is very likely to be a programming error. -- David Hopwood From tomerfiliba at gmail.com Sat Aug 12 18:57:02 2006 From: tomerfiliba at gmail.com (tomer filiba) Date: Sat, 12 Aug 2006 18:57:02 +0200 Subject: [Python-Dev] dict containment annoyance Message-ID: <1d85506f0608120957r7d856211g5f382a0ab796c74c@mail.gmail.com> [Aahz] > -1 > > This is seriously no different from an attempt to do > > >>> a = {} > >>> a[ [] ] = 1 how so? i'm not trying to modify/store anything in a dict. i'm only testing if a certain object is contained in the dict. that's totally different. imagine this: >>> x = 6 >>> x < 2 am i trying to modify 'x' in any way? i'm only testing if x is less than 2. i fail to see your point [David Hopwood] > However, if it isn't hashable, asking whether it is in a dict is very > likely to be a programming error. not really. my program holds a blacklist of "bad objects", which shouldn't be granted access to. instead of a list, i'm using a set (performance, whatever). the "bad objects" are all hashable, and i want to see if an arbitrary object is "good" or "bad", by looking it up in the blacklist. the arbitrary object can be hashable or not, that's not the question. just like `"hello" != 2` returns True, instead of raising IncompatibleTypes, i want `x in y` to return False when `x` is not contained in `y` (with no regard to whether or not it's possible for `y` to contain `x`). the logic is simple: every `x` is either contained in `y` or not. if `x` *cannot* be contained in `y`, then the answer is a "strong no", but that's still a "no". -tomer From exarkun at divmod.com Sat Aug 12 19:07:52 2006 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Sat, 12 Aug 2006 13:07:52 -0400 Subject: [Python-Dev] dict containment annoyance In-Reply-To: <1d85506f0608120957r7d856211g5f382a0ab796c74c@mail.gmail.com> Message-ID: <20060812170752.1717.1940170045.divmod.quotient.20817@ohm> On Sat, 12 Aug 2006 18:57:02 +0200, tomer filiba wrote: > >the logic is simple: every `x` is either contained in `y` or not. >if `x` *cannot* be contained in `y`, then the answer is a "strong no", >but that's still a "no". > def blacklisted(o): try: # Is the object contained in the blacklist set? return o in _blacklistset except TypeError: # If it *cannot* be contained in the blacklist set, # then it probably isn't. return False Jean-Paul From nnorwitz at gmail.com Sat Aug 12 19:13:03 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sat, 12 Aug 2006 10:13:03 -0700 Subject: [Python-Dev] Fwd: [Python-checkins] r51236 - in python/trunk: Doc/api/abstract.tex Include/abstract.h Include/object.h Lib/test/test_index.py Misc/NEWS Modules/arraymodule.c Modules/mmapmodule.c Modules/operator.c Objects/abstract.c Objects/classobject.c Objects/ Message-ID: I checked in this fix for the __index__ clipping issue that's been discussed. This patch is an improvement, but still needs some work. Please pull out any parts you have an issue with and suggest a patch to address your concern. n ---------- Forwarded message ---------- From: neal.norwitz Date: Aug 12, 2006 10:03 AM Subject: [Python-checkins] r51236 - in python/trunk: Doc/api/abstract.tex Include/abstract.h Include/object.h Lib/test/test_index.py Misc/NEWS Modules/arraymodule.c Modules/mmapmodule.c Modules/operator.c Objects/abstract.c Objects/classobject.c Objects/intobject.c Objects/listobject.c Objects/longobject.c Objects/sliceobject.c Objects/stringobject.c Objects/tupleobject.c Objects/typeobject.c Objects/unicodeobject.c Python/ceval.c To: python-checkins at python.org Author: neal.norwitz Date: Sat Aug 12 19:03:09 2006 New Revision: 51236 Modified: python/trunk/Doc/api/abstract.tex python/trunk/Include/abstract.h python/trunk/Include/object.h python/trunk/Lib/test/test_index.py python/trunk/Misc/NEWS python/trunk/Modules/arraymodule.c python/trunk/Modules/mmapmodule.c python/trunk/Modules/operator.c python/trunk/Objects/abstract.c python/trunk/Objects/classobject.c python/trunk/Objects/intobject.c python/trunk/Objects/listobject.c python/trunk/Objects/longobject.c python/trunk/Objects/sliceobject.c python/trunk/Objects/stringobject.c python/trunk/Objects/tupleobject.c python/trunk/Objects/typeobject.c python/trunk/Objects/unicodeobject.c python/trunk/Python/ceval.c Log: Patch #1538606, Patch to fix __index__() clipping. I modified this patch some by fixing style, some error checking, and adding XXX comments. This patch requires review and some changes are to be expected. I'm checking in now to get the greatest possible review and establish a baseline for moving forward. I don't want this to hold up release if possible. Modified: python/trunk/Doc/api/abstract.tex ============================================================================== --- python/trunk/Doc/api/abstract.tex (original) +++ python/trunk/Doc/api/abstract.tex Sat Aug 12 19:03:09 2006 @@ -693,12 +693,31 @@ \samp{float(\var{o})}.\bifuncindex{float} \end{cfuncdesc} -\begin{cfuncdesc}{Py_ssize_t}{PyNumber_Index}{PyObject *o} - Returns the \var{o} converted to a Py_ssize_t integer on success, or - -1 with an exception raised on failure. +\begin{cfuncdesc}{PyObject*}{PyNumber_Index}{PyObject *o} + Returns the \var{o} converted to a Python int or long on success or \NULL{} + with a TypeError exception raised on failure. \versionadded{2.5} \end{cfuncdesc} +\begin{cfuncdesc}{Py_ssize_t}{PyNumber_AsSsize_t}{PyObject *o, PyObject *exc} + Returns \var{o} converted to a Py_ssize_t value if \var{o} + can be interpreted as an integer. If \var{o} can be converted to a Python + int or long but the attempt to convert to a Py_ssize_t value + would raise an \exception{OverflowError}, then the \var{exc} argument + is the type of exception that will be raised (usually \exception{IndexError} + or \exception{OverflowError}). If \var{exc} is \NULL{}, then the exception + is cleared and the value is clipped to \var{PY_SSIZE_T_MIN} + for a negative integer or \var{PY_SSIZE_T_MAX} for a positive integer. + \versionadded{2.5} +\end{cfuncdesc} + +\begin{cfuncdesc}{int}{PyIndex_Check}{PyObject *o} + Returns True if \var{o} is an index integer (has the nb_index slot of + the tp_as_number structure filled in). + \versionadded{2.5} +\end{cfuncdesc} + + \section{Sequence Protocol \label{sequence}} \begin{cfuncdesc}{int}{PySequence_Check}{PyObject *o} Modified: python/trunk/Include/abstract.h ============================================================================== --- python/trunk/Include/abstract.h (original) +++ python/trunk/Include/abstract.h Sat Aug 12 19:03:09 2006 @@ -758,13 +758,27 @@ */ - PyAPI_FUNC(Py_ssize_t) PyNumber_Index(PyObject *); +#define PyIndex_Check(obj) \ + ((obj)->ob_type->tp_as_number != NULL && \ + PyType_HasFeature((obj)->ob_type, Py_TPFLAGS_HAVE_INDEX) && \ + (obj)->ob_type->tp_as_number->nb_index != NULL) + + PyAPI_FUNC(PyObject *) PyNumber_Index(PyObject *o); /* - Returns the object converted to Py_ssize_t on success - or -1 with an error raised on failure. + Returns the object converted to a Python long or int + or NULL with an error raised on failure. */ + PyAPI_FUNC(Py_ssize_t) PyNumber_AsSsize_t(PyObject *o, PyObject *exc); + + /* + Returns the object converted to Py_ssize_t by going through + PyNumber_Index first. If an overflow error occurs while + converting the int-or-long to Py_ssize_t, then the second argument + is the error-type to return. If it is NULL, then the overflow error + is cleared and the value is clipped. + */ PyAPI_FUNC(PyObject *) PyNumber_Int(PyObject *o); Modified: python/trunk/Include/object.h ============================================================================== --- python/trunk/Include/object.h (original) +++ python/trunk/Include/object.h Sat Aug 12 19:03:09 2006 @@ -208,7 +208,7 @@ binaryfunc nb_inplace_true_divide; /* Added in release 2.5 */ - lenfunc nb_index; + unaryfunc nb_index; } PyNumberMethods; typedef struct { Modified: python/trunk/Lib/test/test_index.py ============================================================================== --- python/trunk/Lib/test/test_index.py (original) +++ python/trunk/Lib/test/test_index.py Sat Aug 12 19:03:09 2006 @@ -1,6 +1,7 @@ import unittest from test import test_support import operator +from sys import maxint class oldstyle: def __index__(self): @@ -10,68 +11,115 @@ def __index__(self): return self.ind +class TrapInt(int): + def __index__(self): + return self + +class TrapLong(long): + def __index__(self): + return self + class BaseTestCase(unittest.TestCase): def setUp(self): self.o = oldstyle() self.n = newstyle() - self.o2 = oldstyle() - self.n2 = newstyle() def test_basic(self): self.o.ind = -2 self.n.ind = 2 - assert(self.seq[self.n] == self.seq[2]) - assert(self.seq[self.o] == self.seq[-2]) - assert(operator.index(self.o) == -2) - assert(operator.index(self.n) == 2) + self.assertEqual(operator.index(self.o), -2) + self.assertEqual(operator.index(self.n), 2) + + def test_slice(self): + self.o.ind = 1 + self.n.ind = 2 + slc = slice(self.o, self.o, self.o) + check_slc = slice(1, 1, 1) + self.assertEqual(slc.indices(self.o), check_slc.indices(1)) + slc = slice(self.n, self.n, self.n) + check_slc = slice(2, 2, 2) + self.assertEqual(slc.indices(self.n), check_slc.indices(2)) + def test_wrappers(self): + self.o.ind = 4 + self.n.ind = 5 + self.assertEqual(6 .__index__(), 6) + self.assertEqual(-7L.__index__(), -7) + self.assertEqual(self.o.__index__(), 4) + self.assertEqual(self.n.__index__(), 5) + + def test_infinite_recursion(self): + self.failUnlessRaises(TypeError, operator.index, TrapInt()) + self.failUnlessRaises(TypeError, operator.index, TrapLong()) + self.failUnless(slice(TrapInt()).indices(0)==(0,0,1)) + self.failUnlessRaises(TypeError, slice(TrapLong()).indices, 0) + def test_error(self): self.o.ind = 'dumb' self.n.ind = 'bad' - myfunc = lambda x, obj: obj.seq[x] self.failUnlessRaises(TypeError, operator.index, self.o) self.failUnlessRaises(TypeError, operator.index, self.n) - self.failUnlessRaises(TypeError, myfunc, self.o, self) - self.failUnlessRaises(TypeError, myfunc, self.n, self) + self.failUnlessRaises(TypeError, slice(self.o).indices, 0) + self.failUnlessRaises(TypeError, slice(self.n).indices, 0) + + +class SeqTestCase(unittest.TestCase): + # This test case isn't run directly. It just defines common tests + # to the different sequence types below + def setUp(self): + self.o = oldstyle() + self.n = newstyle() + self.o2 = oldstyle() + self.n2 = newstyle() + + def test_index(self): + self.o.ind = -2 + self.n.ind = 2 + self.assertEqual(self.seq[self.n], self.seq[2]) + self.assertEqual(self.seq[self.o], self.seq[-2]) def test_slice(self): self.o.ind = 1 self.o2.ind = 3 self.n.ind = 2 self.n2.ind = 4 - assert(self.seq[self.o:self.o2] == self.seq[1:3]) - assert(self.seq[self.n:self.n2] == self.seq[2:4]) + self.assertEqual(self.seq[self.o:self.o2], self.seq[1:3]) + self.assertEqual(self.seq[self.n:self.n2], self.seq[2:4]) def test_repeat(self): self.o.ind = 3 self.n.ind = 2 - assert(self.seq * self.o == self.seq * 3) - assert(self.seq * self.n == self.seq * 2) - assert(self.o * self.seq == self.seq * 3) - assert(self.n * self.seq == self.seq * 2) + self.assertEqual(self.seq * self.o, self.seq * 3) + self.assertEqual(self.seq * self.n, self.seq * 2) + self.assertEqual(self.o * self.seq, self.seq * 3) + self.assertEqual(self.n * self.seq, self.seq * 2) def test_wrappers(self): - n = self.n - n.ind = 5 - assert n.__index__() == 5 - assert 6 .__index__() == 6 - assert -7L.__index__() == -7 - assert self.seq.__getitem__(n) == self.seq[5] - assert self.seq.__mul__(n) == self.seq * 5 - assert self.seq.__rmul__(n) == self.seq * 5 - - def test_infinite_recusion(self): - class Trap1(int): - def __index__(self): - return self - class Trap2(long): - def __index__(self): - return self - self.failUnlessRaises(TypeError, operator.getitem, self.seq, Trap1()) - self.failUnlessRaises(TypeError, operator.getitem, self.seq, Trap2()) + self.o.ind = 4 + self.n.ind = 5 + self.assertEqual(self.seq.__getitem__(self.o), self.seq[4]) + self.assertEqual(self.seq.__mul__(self.o), self.seq * 4) + self.assertEqual(self.seq.__rmul__(self.o), self.seq * 4) + self.assertEqual(self.seq.__getitem__(self.n), self.seq[5]) + self.assertEqual(self.seq.__mul__(self.n), self.seq * 5) + self.assertEqual(self.seq.__rmul__(self.n), self.seq * 5) + + def test_infinite_recursion(self): + self.failUnlessRaises(TypeError, operator.getitem, self.seq, TrapInt()) + self.failUnlessRaises(TypeError, operator.getitem, self.seq, TrapLong()) + + def test_error(self): + self.o.ind = 'dumb' + self.n.ind = 'bad' + indexobj = lambda x, obj: obj.seq[x] + self.failUnlessRaises(TypeError, indexobj, self.o, self) + self.failUnlessRaises(TypeError, indexobj, self.n, self) + sliceobj = lambda x, obj: obj.seq[x:] + self.failUnlessRaises(TypeError, sliceobj, self.o, self) + self.failUnlessRaises(TypeError, sliceobj, self.n, self) -class ListTestCase(BaseTestCase): +class ListTestCase(SeqTestCase): seq = [0,10,20,30,40,50] def test_setdelitem(self): @@ -82,36 +130,36 @@ del lst[self.n] lst[self.o] = 'X' lst[self.n] = 'Y' - assert lst == list('abYdefghXj') + self.assertEqual(lst, list('abYdefghXj')) lst = [5, 6, 7, 8, 9, 10, 11] lst.__setitem__(self.n, "here") - assert lst == [5, 6, "here", 8, 9, 10, 11] + self.assertEqual(lst, [5, 6, "here", 8, 9, 10, 11]) lst.__delitem__(self.n) - assert lst == [5, 6, 8, 9, 10, 11] + self.assertEqual(lst, [5, 6, 8, 9, 10, 11]) def test_inplace_repeat(self): self.o.ind = 2 self.n.ind = 3 lst = [6, 4] lst *= self.o - assert lst == [6, 4, 6, 4] + self.assertEqual(lst, [6, 4, 6, 4]) lst *= self.n - assert lst == [6, 4, 6, 4] * 3 + self.assertEqual(lst, [6, 4, 6, 4] * 3) lst = [5, 6, 7, 8, 9, 11] l2 = lst.__imul__(self.n) - assert l2 is lst - assert lst == [5, 6, 7, 8, 9, 11] * 3 + self.assert_(l2 is lst) + self.assertEqual(lst, [5, 6, 7, 8, 9, 11] * 3) -class TupleTestCase(BaseTestCase): +class TupleTestCase(SeqTestCase): seq = (0,10,20,30,40,50) -class StringTestCase(BaseTestCase): +class StringTestCase(SeqTestCase): seq = "this is a test" -class UnicodeTestCase(BaseTestCase): +class UnicodeTestCase(SeqTestCase): seq = u"this is a test" @@ -120,17 +168,47 @@ def test_xrange(self): n = newstyle() n.ind = 5 - assert xrange(1, 20)[n] == 6 - assert xrange(1, 20).__getitem__(n) == 6 + self.assertEqual(xrange(1, 20)[n], 6) + self.assertEqual(xrange(1, 20).__getitem__(n), 6) + +class OverflowTestCase(unittest.TestCase): + + def setUp(self): + self.pos = 2**100 + self.neg = -self.pos + + def test_large_longs(self): + self.assertEqual(self.pos.__index__(), self.pos) + self.assertEqual(self.neg.__index__(), self.neg) + + def test_getitem(self): + class GetItem(object): + def __len__(self): + return maxint + def __getitem__(self, key): + return key + def __getslice__(self, i, j): + return i, j + x = GetItem() + self.assertEqual(x[self.pos], self.pos) + self.assertEqual(x[self.neg], self.neg) + self.assertEqual(x[self.neg:self.pos], (-1, maxint)) + self.assertEqual(x[self.neg:self.pos:1].indices(maxint), (0, maxint, 1)) + + def test_sequence_repeat(self): + self.failUnlessRaises(OverflowError, lambda: "a" * self.pos) + self.failUnlessRaises(OverflowError, lambda: "a" * self.neg) def test_main(): test_support.run_unittest( + BaseTestCase, ListTestCase, TupleTestCase, StringTestCase, UnicodeTestCase, XRangeTestCase, + OverflowTestCase, ) if __name__ == "__main__": Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Sat Aug 12 19:03:09 2006 @@ -12,6 +12,11 @@ Core and builtins ----------------- +- Patch #1538606, Fix __index__() clipping. There were some problems + discovered with the API and how integers that didn't fit into Py_ssize_t + were handled. This patch attempts to provide enough alternatives + to effectively use __index__. + - Bug #1536021: __hash__ may now return long int; the final hash value is obtained by invoking hash on the long int. Modified: python/trunk/Modules/arraymodule.c ============================================================================== --- python/trunk/Modules/arraymodule.c (original) +++ python/trunk/Modules/arraymodule.c Sat Aug 12 19:03:09 2006 @@ -1572,14 +1572,11 @@ return s; } -#define HASINDEX(o) PyType_HasFeature((o)->ob_type, Py_TPFLAGS_HAVE_INDEX) - static PyObject* array_subscr(arrayobject* self, PyObject* item) { - PyNumberMethods *nb = item->ob_type->tp_as_number; - if (nb != NULL && HASINDEX(item) && nb->nb_index != NULL) { - Py_ssize_t i = nb->nb_index(item); + if (PyIndex_Check(item)) { + Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError); if (i==-1 && PyErr_Occurred()) { return NULL; } @@ -1627,9 +1624,8 @@ static int array_ass_subscr(arrayobject* self, PyObject* item, PyObject* value) { - PyNumberMethods *nb = item->ob_type->tp_as_number; - if (nb != NULL && HASINDEX(item) && nb->nb_index != NULL) { - Py_ssize_t i = nb->nb_index(item); + if (PyIndex_Check(item)) { + Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError); if (i==-1 && PyErr_Occurred()) return -1; if (i < 0) Modified: python/trunk/Modules/mmapmodule.c ============================================================================== --- python/trunk/Modules/mmapmodule.c (original) +++ python/trunk/Modules/mmapmodule.c Sat Aug 12 19:03:09 2006 @@ -808,8 +808,6 @@ }; -#define HASINDEX(o) PyType_HasFeature((o)->ob_type, Py_TPFLAGS_HAVE_INDEX) - /* extract the map size from the given PyObject Returns -1 on error, with an appropriate Python exception raised. On @@ -817,31 +815,19 @@ static Py_ssize_t _GetMapSize(PyObject *o) { - PyNumberMethods *nb = o->ob_type->tp_as_number; - if (nb != NULL && HASINDEX(o) && nb->nb_index != NULL) { - Py_ssize_t i = nb->nb_index(o); + if (PyIndex_Check(o)) { + Py_ssize_t i = PyNumber_AsSsize_t(o, PyExc_OverflowError); if (i==-1 && PyErr_Occurred()) return -1; - if (i < 0) - goto onnegoverflow; - if (i==PY_SSIZE_T_MAX) - goto onposoverflow; + if (i < 0) { + PyErr_SetString(PyExc_OverflowError, + "memory mapped size must be positive"); + return -1; + } return i; } - else { - PyErr_SetString(PyExc_TypeError, - "map size must be an integral value"); - return -1; - } - - onnegoverflow: - PyErr_SetString(PyExc_OverflowError, - "memory mapped size must be positive"); - return -1; - onposoverflow: - PyErr_SetString(PyExc_OverflowError, - "memory mapped size is too large (limited by C int)"); + PyErr_SetString(PyExc_TypeError, "map size must be an integral value"); return -1; } Modified: python/trunk/Modules/operator.c ============================================================================== --- python/trunk/Modules/operator.c (original) +++ python/trunk/Modules/operator.c Sat Aug 12 19:03:09 2006 @@ -139,15 +139,7 @@ static PyObject * op_index(PyObject *s, PyObject *a) { - Py_ssize_t i; - PyObject *a1; - if (!PyArg_UnpackTuple(a,"index", 1, 1, &a1)) - return NULL; - i = PyNumber_Index(a1); - if (i == -1 && PyErr_Occurred()) - return NULL; - else - return PyInt_FromSsize_t(i); + return PyNumber_Index(a); } static PyObject* @@ -249,7 +241,7 @@ spam1(is_, "is_(a, b) -- Same as a is b.") spam1(is_not, "is_not(a, b) -- Same as a is not b.") -spam2(index, __index__, "index(a) -- Same as a.__index__()") +spam2o(index, __index__, "index(a) -- Same as a.__index__()") spam2(add,__add__, "add(a, b) -- Same as a + b.") spam2(sub,__sub__, "sub(a, b) -- Same as a - b.") spam2(mul,__mul__, "mul(a, b) -- Same as a * b.") Modified: python/trunk/Objects/abstract.c ============================================================================== --- python/trunk/Objects/abstract.c (original) +++ python/trunk/Objects/abstract.c Sat Aug 12 19:03:09 2006 @@ -8,8 +8,6 @@ #define NEW_STYLE_NUMBER(o) PyType_HasFeature((o)->ob_type, \ Py_TPFLAGS_CHECKTYPES) -#define HASINDEX(o) PyType_HasFeature((o)->ob_type, Py_TPFLAGS_HAVE_INDEX) - /* Shorthands to return certain errors */ @@ -122,9 +120,9 @@ return m->mp_subscript(o, key); if (o->ob_type->tp_as_sequence) { - PyNumberMethods *nb = key->ob_type->tp_as_number; - if (nb != NULL && HASINDEX(key) && nb->nb_index != NULL) { - Py_ssize_t key_value = nb->nb_index(key); + if (PyIndex_Check(key)) { + Py_ssize_t key_value; + key_value = PyNumber_AsSsize_t(key, PyExc_IndexError); if (key_value == -1 && PyErr_Occurred()) return NULL; return PySequence_GetItem(o, key_value); @@ -151,9 +149,9 @@ return m->mp_ass_subscript(o, key, value); if (o->ob_type->tp_as_sequence) { - PyNumberMethods *nb = key->ob_type->tp_as_number; - if (nb != NULL && HASINDEX(key) && nb->nb_index != NULL) { - Py_ssize_t key_value = nb->nb_index(key); + if (PyIndex_Check(key)) { + Py_ssize_t key_value; + key_value = PyNumber_AsSsize_t(key, PyExc_IndexError); if (key_value == -1 && PyErr_Occurred()) return -1; return PySequence_SetItem(o, key_value, value); @@ -183,9 +181,9 @@ return m->mp_ass_subscript(o, key, (PyObject*)NULL); if (o->ob_type->tp_as_sequence) { - PyNumberMethods *nb = key->ob_type->tp_as_number; - if (nb != NULL && HASINDEX(key) && nb->nb_index != NULL) { - Py_ssize_t key_value = nb->nb_index(key); + if (PyIndex_Check(key)) { + Py_ssize_t key_value; + key_value = PyNumber_AsSsize_t(key, PyExc_IndexError); if (key_value == -1 && PyErr_Occurred()) return -1; return PySequence_DelItem(o, key_value); @@ -653,9 +651,8 @@ sequence_repeat(ssizeargfunc repeatfunc, PyObject *seq, PyObject *n) { Py_ssize_t count; - PyNumberMethods *nb = n->ob_type->tp_as_number; - if (nb != NULL && HASINDEX(n) && nb->nb_index != NULL) { - count = nb->nb_index(n); + if (PyIndex_Check(n)) { + count = PyNumber_AsSsize_t(n, PyExc_OverflowError); if (count == -1 && PyErr_Occurred()) return NULL; } @@ -938,23 +935,89 @@ return x; } -/* Return a Py_ssize_t integer from the object item */ -Py_ssize_t +/* Return a Python Int or Long from the object item + Raise TypeError if the result is not an int-or-long + or if the object cannot be interpreted as an index. +*/ +PyObject * PyNumber_Index(PyObject *item) { - Py_ssize_t value = -1; - PyNumberMethods *nb = item->ob_type->tp_as_number; - if (nb != NULL && HASINDEX(item) && nb->nb_index != NULL) { - value = nb->nb_index(item); + PyObject *result = NULL; + if (item == NULL) + return null_error(); + /* XXX(nnorwitz): should these be CheckExact? Aren't subclasses ok? */ + if (PyInt_CheckExact(item) || PyLong_CheckExact(item)) { + Py_INCREF(item); + return item; + } + if (PyIndex_Check(item)) { + result = item->ob_type->tp_as_number->nb_index(item); + /* XXX(nnorwitz): Aren't subclasses ok here too? */ + if (result && + !PyInt_CheckExact(result) && !PyLong_CheckExact(result)) { + PyErr_Format(PyExc_TypeError, + "__index__ returned non-(int,long) " \ + "(type %.200s)", + result->ob_type->tp_name); + Py_DECREF(result); + return NULL; + } } else { PyErr_Format(PyExc_TypeError, "'%.200s' object cannot be interpreted " "as an index", item->ob_type->tp_name); } - return value; + return result; } +/* Return an error on Overflow only if err is not NULL*/ + +Py_ssize_t +PyNumber_AsSsize_t(PyObject *item, PyObject *err) +{ + Py_ssize_t result; + PyObject *runerr; + PyObject *value = PyNumber_Index(item); + if (value == NULL) + return -1; + + /* We're done if PyInt_AsSsize_t() returns without error. */ + result = PyInt_AsSsize_t(value); + if (result != -1 || !(runerr = PyErr_Occurred())) + goto finish; + + /* Error handling code -- only manage OverflowError differently */ + if (!PyErr_GivenExceptionMatches(runerr, PyExc_OverflowError)) + goto finish; + + PyErr_Clear(); + /* If no error-handling desired then the default clipping + is sufficient. + */ + if (!err) { + assert(PyLong_Check(value)); + /* Whether or not it is less than or equal to + zero is determined by the sign of ob_size + */ + if (_PyLong_Sign(value) < 0) + result = PY_SSIZE_T_MIN; + else + result = PY_SSIZE_T_MAX; + } + else { + /* Otherwise replace the error with caller's error object. */ + PyErr_Format(err, + "cannot fit '%.200s' into an index-sized integer", + item->ob_type->tp_name); + } + + finish: + Py_DECREF(value); + return result; +} + + PyObject * PyNumber_Int(PyObject *o) { Modified: python/trunk/Objects/classobject.c ============================================================================== --- python/trunk/Objects/classobject.c (original) +++ python/trunk/Objects/classobject.c Sat Aug 12 19:03:09 2006 @@ -1670,40 +1670,28 @@ return outcome > 0; } -static Py_ssize_t +static PyObject * instance_index(PyInstanceObject *self) { PyObject *func, *res; - Py_ssize_t outcome; static PyObject *indexstr = NULL; if (indexstr == NULL) { indexstr = PyString_InternFromString("__index__"); if (indexstr == NULL) - return -1; + return NULL; } if ((func = instance_getattr(self, indexstr)) == NULL) { if (!PyErr_ExceptionMatches(PyExc_AttributeError)) - return -1; + return NULL; PyErr_Clear(); PyErr_SetString(PyExc_TypeError, "object cannot be interpreted as an index"); - return -1; + return NULL; } res = PyEval_CallObject(func, (PyObject *)NULL); Py_DECREF(func); - if (res == NULL) - return -1; - if (PyInt_Check(res) || PyLong_Check(res)) { - outcome = res->ob_type->tp_as_number->nb_index(res); - } - else { - PyErr_SetString(PyExc_TypeError, - "__index__ must return an int or a long"); - outcome = -1; - } - Py_DECREF(res); - return outcome; + return res; } @@ -2026,7 +2014,7 @@ instance_truediv, /* nb_true_divide */ instance_ifloordiv, /* nb_inplace_floor_divide */ instance_itruediv, /* nb_inplace_true_divide */ - (lenfunc)instance_index, /* nb_index */ + (unaryfunc)instance_index, /* nb_index */ }; PyTypeObject PyInstance_Type = { Modified: python/trunk/Objects/intobject.c ============================================================================== --- python/trunk/Objects/intobject.c (original) +++ python/trunk/Objects/intobject.c Sat Aug 12 19:03:09 2006 @@ -193,16 +193,21 @@ PyIntObject *io; Py_ssize_t val; #endif - if (op && !PyInt_CheckExact(op) && PyLong_Check(op)) + + if (op == NULL) { + PyErr_SetString(PyExc_TypeError, "an integer is required"); + return -1; + } + + if (PyInt_Check(op)) + return PyInt_AS_LONG((PyIntObject*) op); + if (PyLong_Check(op)) return _PyLong_AsSsize_t(op); #if SIZEOF_SIZE_T == SIZEOF_LONG return PyInt_AsLong(op); #else - if (op && PyInt_Check(op)) - return PyInt_AS_LONG((PyIntObject*) op); - - if (op == NULL || (nb = op->ob_type->tp_as_number) == NULL || + if ((nb = op->ob_type->tp_as_number) == NULL || (nb->nb_int == NULL && nb->nb_long == 0)) { PyErr_SetString(PyExc_TypeError, "an integer is required"); return -1; @@ -1079,7 +1084,7 @@ int_true_divide, /* nb_true_divide */ 0, /* nb_inplace_floor_divide */ 0, /* nb_inplace_true_divide */ - PyInt_AsSsize_t, /* nb_index */ + (unaryfunc)int_int, /* nb_index */ }; PyTypeObject PyInt_Type = { Modified: python/trunk/Objects/listobject.c ============================================================================== --- python/trunk/Objects/listobject.c (original) +++ python/trunk/Objects/listobject.c Sat Aug 12 19:03:09 2006 @@ -2450,14 +2450,13 @@ "list() -> new list\n" "list(sequence) -> new list initialized from sequence's items"); -#define HASINDEX(o) PyType_HasFeature((o)->ob_type, Py_TPFLAGS_HAVE_INDEX) static PyObject * list_subscript(PyListObject* self, PyObject* item) { - PyNumberMethods *nb = item->ob_type->tp_as_number; - if (nb != NULL && HASINDEX(item) && nb->nb_index != NULL) { - Py_ssize_t i = nb->nb_index(item); + if (PyIndex_Check(item)) { + Py_ssize_t i; + i = PyNumber_AsSsize_t(item, PyExc_IndexError); if (i == -1 && PyErr_Occurred()) return NULL; if (i < 0) @@ -2504,9 +2503,8 @@ static int list_ass_subscript(PyListObject* self, PyObject* item, PyObject* value) { - PyNumberMethods *nb = item->ob_type->tp_as_number; - if (nb != NULL && HASINDEX(item) && nb->nb_index != NULL) { - Py_ssize_t i = nb->nb_index(item); + if (PyIndex_Check(item)) { + Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError); if (i == -1 && PyErr_Occurred()) return -1; if (i < 0) Modified: python/trunk/Objects/longobject.c ============================================================================== --- python/trunk/Objects/longobject.c (original) +++ python/trunk/Objects/longobject.c Sat Aug 12 19:03:09 2006 @@ -240,8 +240,11 @@ return -1; } -static Py_ssize_t -_long_as_ssize_t(PyObject *vv) { +/* Get a Py_ssize_t from a long int object. + Returns -1 and sets an error condition if overflow occurs. */ + +Py_ssize_t +_PyLong_AsSsize_t(PyObject *vv) { register PyLongObject *v; size_t x, prev; Py_ssize_t i; @@ -277,45 +280,7 @@ overflow: PyErr_SetString(PyExc_OverflowError, "long int too large to convert to int"); - if (sign > 0) - return PY_SSIZE_T_MAX; - else - return PY_SSIZE_T_MIN; -} - -/* Get a Py_ssize_t from a long int object. - Returns -1 and sets an error condition if overflow occurs. */ - -Py_ssize_t -_PyLong_AsSsize_t(PyObject *vv) -{ - Py_ssize_t x; - - x = _long_as_ssize_t(vv); - if (PyErr_Occurred()) return -1; - return x; -} - - -/* Get a Py_ssize_t from a long int object. - Silently reduce values larger than PY_SSIZE_T_MAX to PY_SSIZE_T_MAX, - and silently boost values less than -PY_SSIZE_T_MAX-1 to -PY_SSIZE_T_MAX-1. - On error, return -1 with an exception set. -*/ - -static Py_ssize_t -long_index(PyObject *vv) -{ - Py_ssize_t x; - - x = _long_as_ssize_t(vv); - if (PyErr_Occurred()) { - /* If overflow error, ignore the error */ - if (x != -1) { - PyErr_Clear(); - } - } - return x; + return -1; } /* Get a C unsigned long int from a long int object. @@ -3405,7 +3370,7 @@ long_true_divide, /* nb_true_divide */ 0, /* nb_inplace_floor_divide */ 0, /* nb_inplace_true_divide */ - long_index, /* nb_index */ + long_long, /* nb_index */ }; PyTypeObject PyLong_Type = { Modified: python/trunk/Objects/sliceobject.c ============================================================================== --- python/trunk/Objects/sliceobject.c (original) +++ python/trunk/Objects/sliceobject.c Sat Aug 12 19:03:09 2006 @@ -252,7 +252,7 @@ { Py_ssize_t ilen, start, stop, step, slicelength; - ilen = PyInt_AsSsize_t(len); + ilen = PyNumber_AsSsize_t(len, PyExc_OverflowError); if (ilen == -1 && PyErr_Occurred()) { return NULL; Modified: python/trunk/Objects/stringobject.c ============================================================================== --- python/trunk/Objects/stringobject.c (original) +++ python/trunk/Objects/stringobject.c Sat Aug 12 19:03:09 2006 @@ -1184,14 +1184,11 @@ return x; } -#define HASINDEX(o) PyType_HasFeature((o)->ob_type, Py_TPFLAGS_HAVE_INDEX) - static PyObject* string_subscript(PyStringObject* self, PyObject* item) { - PyNumberMethods *nb = item->ob_type->tp_as_number; - if (nb != NULL && HASINDEX(item) && nb->nb_index != NULL) { - Py_ssize_t i = nb->nb_index(item); + if (PyIndex_Check(item)) { + Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError); if (i == -1 && PyErr_Occurred()) return NULL; if (i < 0) Modified: python/trunk/Objects/tupleobject.c ============================================================================== --- python/trunk/Objects/tupleobject.c (original) +++ python/trunk/Objects/tupleobject.c Sat Aug 12 19:03:09 2006 @@ -577,14 +577,11 @@ (objobjproc)tuplecontains, /* sq_contains */ }; -#define HASINDEX(o) PyType_HasFeature((o)->ob_type, Py_TPFLAGS_HAVE_INDEX) - static PyObject* tuplesubscript(PyTupleObject* self, PyObject* item) { - PyNumberMethods *nb = item->ob_type->tp_as_number; - if (nb != NULL && HASINDEX(item) && nb->nb_index != NULL) { - Py_ssize_t i = nb->nb_index(item); + if (PyIndex_Check(item)) { + Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError); if (i == -1 && PyErr_Occurred()) return NULL; if (i < 0) Modified: python/trunk/Objects/typeobject.c ============================================================================== --- python/trunk/Objects/typeobject.c (original) +++ python/trunk/Objects/typeobject.c Sat Aug 12 19:03:09 2006 @@ -3527,7 +3527,7 @@ if (!PyArg_UnpackTuple(args, "", 1, 1, &o)) return NULL; - i = PyNumber_Index(o); + i = PyNumber_AsSsize_t(o, PyExc_OverflowError); if (i == -1 && PyErr_Occurred()) return NULL; return (*func)(self, i); @@ -3538,7 +3538,7 @@ { Py_ssize_t i; - i = PyNumber_Index(arg); + i = PyNumber_AsSsize_t(arg, PyExc_OverflowError); if (i == -1 && PyErr_Occurred()) return -1; if (i < 0) { @@ -4344,26 +4344,11 @@ } -static Py_ssize_t +static PyObject * slot_nb_index(PyObject *self) { static PyObject *index_str; - PyObject *temp = call_method(self, "__index__", &index_str, "()"); - Py_ssize_t result; - - if (temp == NULL) - return -1; - if (PyInt_CheckExact(temp) || PyLong_CheckExact(temp)) { - result = temp->ob_type->tp_as_number->nb_index(temp); - } - else { - PyErr_Format(PyExc_TypeError, - "__index__ must return an int or a long, " - "not '%.200s'", temp->ob_type->tp_name); - result = -1; - } - Py_DECREF(temp); - return result; + return call_method(self, "__index__", &index_str, "()"); } @@ -5109,7 +5094,7 @@ "oct(x)"), UNSLOT("__hex__", nb_hex, slot_nb_hex, wrap_unaryfunc, "hex(x)"), - NBSLOT("__index__", nb_index, slot_nb_index, wrap_lenfunc, + NBSLOT("__index__", nb_index, slot_nb_index, wrap_unaryfunc, "x[y:z] <==> x[y.__index__():z.__index__()]"), IBSLOT("__iadd__", nb_inplace_add, slot_nb_inplace_add, wrap_binaryfunc, "+"), Modified: python/trunk/Objects/unicodeobject.c ============================================================================== --- python/trunk/Objects/unicodeobject.c (original) +++ python/trunk/Objects/unicodeobject.c Sat Aug 12 19:03:09 2006 @@ -6985,14 +6985,11 @@ PyUnicode_Contains, /* sq_contains */ }; -#define HASINDEX(o) PyType_HasFeature((o)->ob_type, Py_TPFLAGS_HAVE_INDEX) - static PyObject* unicode_subscript(PyUnicodeObject* self, PyObject* item) { - PyNumberMethods *nb = item->ob_type->tp_as_number; - if (nb != NULL && HASINDEX(item) && nb->nb_index != NULL) { - Py_ssize_t i = nb->nb_index(item); + if (PyIndex_Check(item)) { + Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError); if (i == -1 && PyErr_Occurred()) return NULL; if (i < 0) Modified: python/trunk/Python/ceval.c ============================================================================== --- python/trunk/Python/ceval.c (original) +++ python/trunk/Python/ceval.c Sat Aug 12 19:03:09 2006 @@ -3866,12 +3866,14 @@ if (v != NULL) { Py_ssize_t x; if (PyInt_Check(v)) { - x = PyInt_AsSsize_t(v); + /* XXX(nnorwitz): I think PyInt_AS_LONG is correct, + however, it looks like it should be AsSsize_t. + There should be a comment here explaining why. + */ + x = PyInt_AS_LONG(v); } - else if (v->ob_type->tp_as_number && - PyType_HasFeature(v->ob_type, Py_TPFLAGS_HAVE_INDEX) - && v->ob_type->tp_as_number->nb_index) { - x = v->ob_type->tp_as_number->nb_index(v); + else if (PyIndex_Check(v)) { + x = PyNumber_AsSsize_t(v, NULL); if (x == -1 && PyErr_Occurred()) return 0; } @@ -3887,10 +3889,8 @@ } #undef ISINDEX -#define ISINDEX(x) ((x) == NULL || PyInt_Check(x) || PyLong_Check(x) || \ - ((x)->ob_type->tp_as_number && \ - PyType_HasFeature((x)->ob_type, Py_TPFLAGS_HAVE_INDEX) \ - && (x)->ob_type->tp_as_number->nb_index)) +#define ISINDEX(x) ((x) == NULL || \ + PyInt_Check(x) || PyLong_Check(x) || PyIndex_Check(x)) static PyObject * apply_slice(PyObject *u, PyObject *v, PyObject *w) /* return u[v:w] */ _______________________________________________ Python-checkins mailing list Python-checkins at python.org http://mail.python.org/mailman/listinfo/python-checkins From g.brandl at gmx.net Sat Aug 12 20:10:54 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 12 Aug 2006 20:10:54 +0200 Subject: [Python-Dev] dict containment annoyance In-Reply-To: <1d85506f0608120957r7d856211g5f382a0ab796c74c@mail.gmail.com> References: <1d85506f0608120957r7d856211g5f382a0ab796c74c@mail.gmail.com> Message-ID: tomer filiba wrote: > [Aahz] >> -1 >> >> This is seriously no different from an attempt to do >> >> >>> a = {} >> >>> a[ [] ] = 1 > > how so? i'm not trying to modify/store anything in a dict. > i'm only testing if a certain object is contained in the dict. > that's totally different. > imagine this: >>>> x = 6 >>>> x < 2 > am i trying to modify 'x' in any way? i'm only testing if x is > less than 2. The correct analogon is >>> x = [] >>> x < 2 which currently works in Python, but won't in 3.0. Georg From oliphant.travis at ieee.org Sat Aug 12 21:56:27 2006 From: oliphant.travis at ieee.org (Travis E. Oliphant) Date: Sat, 12 Aug 2006 13:56:27 -0600 Subject: [Python-Dev] Fwd: [Python-checkins] r51236 - in python/trunk: Doc/api/abstract.tex Include/abstract.h Include/object.h Lib/test/test_index.py Misc/NEWS Modules/arraymodule.c Modules/mmapmodule.c Modules/operator.c Objects/abstract.c Objects/classobject.c Objects/ In-Reply-To: References: Message-ID: Neal Norwitz wrote: > I checked in this fix for the __index__ clipping issue that's been > discussed. This patch is an improvement, but still needs some work. > > +/* Return a Python Int or Long from the object item > + Raise TypeError if the result is not an int-or-long > + or if the object cannot be interpreted as an index. > +*/ > +PyObject * > PyNumber_Index(PyObject *item) > { > - Py_ssize_t value = -1; > - PyNumberMethods *nb = item->ob_type->tp_as_number; > - if (nb != NULL && HASINDEX(item) && nb->nb_index != NULL) { > - value = nb->nb_index(item); > + PyObject *result = NULL; > + if (item == NULL) > + return null_error(); > + /* XXX(nnorwitz): should these be CheckExact? Aren't subclasses ok? */ The idea is that the __index__() method should return an exact int or exact long or this call will raise an error. The restriction is present to remove the possibility of infinite recursion (though I'm not sure where that would occur exactly). > Modified: python/trunk/Python/ceval.c > ============================================================================== > --- python/trunk/Python/ceval.c (original) > +++ python/trunk/Python/ceval.c Sat Aug 12 19:03:09 2006 > @@ -3866,12 +3866,14 @@ > if (v != NULL) { > Py_ssize_t x; > if (PyInt_Check(v)) { > - x = PyInt_AsSsize_t(v); > + /* XXX(nnorwitz): I think PyInt_AS_LONG is correct, > + however, it looks like it should be AsSsize_t. > + There should be a comment here explaining why. > + */ > + x = PyInt_AS_LONG(v); Right now throughout the Python code it is assumed that sizeof(Py_ssize_t) <= sizeof(long). Because this code is an optimization for integers (or their sub-classes), it seems prudent to truly make it fast rather than make a function call that will just go through a series of checks to eventually make this very same call. -Travis From oliphant.travis at ieee.org Sat Aug 12 22:16:23 2006 From: oliphant.travis at ieee.org (Travis E. Oliphant) Date: Sat, 12 Aug 2006 14:16:23 -0600 Subject: [Python-Dev] Fwd: [Python-checkins] r51236 - in python/trunk: Doc/api/abstract.tex Include/abstract.h Include/object.h Lib/test/test_index.py Misc/NEWS Modules/arraymodule.c Modules/mmapmodule.c Modules/operator.c Objects/abstract.c Objects/classobject.c Objects/ In-Reply-To: References: Message-ID: Neal Norwitz wrote: > I checked in this fix for the __index__ clipping issue that's been > discussed. This patch is an improvement, but still needs some work. > Please pull out any parts you have an issue with and suggest a patch > to address your concern. > For me the only remaining concern is that quite often in the code we do this if (PyIndex_Check(obj)) { ... key = PyNumber_Index(obj); or key_value = PyNumber_AsSize_t(obj, ...) } else {remaining checks} Internally PyNumber_AsSize_t makes a call to PyNumber_Index, and PyNumber_Index also calls the PyIndex_Check as well . So, basically we end up calling PyIndex_Check(obj) 2 times when only one check should be necessary. This code could be re-written to move any other type checks first and replace the PyIndex_Check(obj) code with PyNumber_Index(obj) and error handling but I'm not sure if that's the right way to go or if it's worth it. -Travis Oliphant From martin at v.loewis.de Sun Aug 13 00:24:31 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 13 Aug 2006 00:24:31 +0200 Subject: [Python-Dev] What is the status of file.readinto? In-Reply-To: References: Message-ID: <44DE551F.8050609@v.loewis.de> Alexander Belopolsky schrieb: > I know that it is a subject of an annual discussion, but since I could > not find any mention of it in the last year, let me ask this question > again: why is file.readinto "Undocumented"? It is quite useful, > particularly with numpy. > > Would a patch adding "readinto" to StringIO be acceptable? You mean, whether it would be accepted? Depends on the patch. Or are you asking whether it would be accepted now? Definitely not; no new features are allowed for Python 2.5. readinto was documented as undocumented here r23251 | tim_one | 2001-09-20 09:55:22 +0200 (Do, 20 Sep 2001) | 3 lines SF bug [#463093] File methods need doc strings. Now they don't. I can only guess why it may go away; my guess it will go away when the buffer interface is removed from Python (then it becomes unimplementable). Regards, Martin From tim.peters at gmail.com Sun Aug 13 00:54:58 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sat, 12 Aug 2006 18:54:58 -0400 Subject: [Python-Dev] Fwd: [Python-checkins] r51236 - in python/trunk: Doc/api/abstract.tex Include/abstract.h Include/object.h Lib/test/test_index.py Misc/NEWS Modules/arraymodule.c Modules/mmapmodule.c Modules/operator.c Objects/abstract.c Objects/class Message-ID: <1f7befae0608121554p48ca2255s5179fd22ef34b978@mail.gmail.com> [Travis E. Oliphant] > Right now throughout the Python code it is assumed that > sizeof(Py_ssize_t) <= sizeof(long). If you find any code like that, it's a bug. Win64 is a platform where it's false. From oliphant.travis at ieee.org Sun Aug 13 02:05:21 2006 From: oliphant.travis at ieee.org (Travis E. Oliphant) Date: Sat, 12 Aug 2006 18:05:21 -0600 Subject: [Python-Dev] Fwd: [Python-checkins] r51236 - in python/trunk: Doc/api/abstract.tex Include/abstract.h Include/object.h Lib/test/test_index.py Misc/NEWS Modules/arraymodule.c Modules/mmapmodule.c Modules/operator.c Objects/abstract.c Objects/class In-Reply-To: <1f7befae0608121554p48ca2255s5179fd22ef34b978@mail.gmail.com> References: <1f7befae0608121554p48ca2255s5179fd22ef34b978@mail.gmail.com> Message-ID: Tim Peters wrote: > [Travis E. Oliphant] >> Right now throughout the Python code it is assumed that >> sizeof(Py_ssize_t) <= sizeof(long). > > If you find any code like that, it's a bug. Win64 is a platform where > it's false. Sorry for my confusion. I meant to say that it is assumed that sizeof(long) <= sizeof(Py_ssize_t) because the assumption is that a Python Int (stored as long) will *always* fit into into a Py_ssize_t. I think this is true on all platforms. -Travis From guido at python.org Sun Aug 13 04:28:44 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 12 Aug 2006 19:28:44 -0700 Subject: [Python-Dev] What is the status of file.readinto? In-Reply-To: <44DE551F.8050609@v.loewis.de> References: <44DE551F.8050609@v.loewis.de> Message-ID: On 8/12/06, "Martin v. L?wis" wrote: > I can only guess why it may go away; my guess it will go away when > the buffer interface is removed from Python (then it becomes > unimplementable). In Py3k, the I/O APIs will be redesigned, especially the binary ones. My current idea is to have read() on a binary file return a bytes object. If readinto() continues to be necessary, please make sure the Py3k list (python-3000 at python.org) knows about your use case. We aren't quite writing up the I/O APIs in PEP-form, but when we do, that would be the right time to speak up. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sun Aug 13 04:33:17 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 12 Aug 2006 19:33:17 -0700 Subject: [Python-Dev] 2.6 idea: a 'function' builtin to parallel classmethod and staticmethod In-Reply-To: <44DD412E.9060109@iinet.net.au> References: <44DD412E.9060109@iinet.net.au> Message-ID: It ought to be called @instancemethod for a better analogy. PS Nick how's the book coming along? :-) --Guido On 8/11/06, Nick Coghlan wrote: > It's sometimes useful to be able to use an existing callable as a method of a > new class. If the callable is a real function, this is easy. You just > including the following line in the class definition: > > method = some_callable > > However, callable objects without a function-like __get__ method can't be used > that way. So, to avoid a dependency on an implementation detail of > some_callable (i.e. whether or not it is a true function object), you have to > write: > > > def method(): > return some_callable() > > (and you can lose useful metadata in the process!) > > However, if you're adding a callable as a class method or static method, there > is OOWTDI: > > method = classmethod(some_callable) > method = staticmethod(some_callable) > > It would be nice if there was a similar mechanism for normal instance methods > as well: > > method = function(some_callable) > > This came up during the PEP 343 implementation - "context = > contextlib.closing" is a tempting thing to write in order to provide a > "x.context()" method for use in a with statement, but it doesn't actually work > properly (because closing is a class, not a function). > > Similarly, you can't create a method simply by applying functools.partial to > an existing function - the result won't have a __get__ method, so it will be > treated like a normal attribute instead of as an instance method. > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > --------------------------------------------------------------- > http://www.boredomandlaziness.org > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From alexander.belopolsky at gmail.com Sun Aug 13 05:19:27 2006 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sat, 12 Aug 2006 23:19:27 -0400 Subject: [Python-Dev] What is the status of file.readinto? In-Reply-To: <44DE551F.8050609@v.loewis.de> References: <44DE551F.8050609@v.loewis.de> Message-ID: On 8/12/06, "Martin v. L?wis" wrote: > Alexander Belopolsky schrieb: > > Would a patch adding "readinto" to StringIO be acceptable? > > You mean, whether it would be accepted? Depends on the patch. Here is the patch: https://sourceforge.net/tracker/index.php?func=detail&aid=1539381&group_id=5470&atid=305470 From ncoghlan at gmail.com Sun Aug 13 08:31:06 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 13 Aug 2006 16:31:06 +1000 Subject: [Python-Dev] Fwd: [Python-checkins] r51236 - in python/trunk: Doc/api/abstract.tex Include/abstract.h Include/object.h Lib/test/test_index.py Misc/NEWS Modules/arraymodule.c Modules/mmapmodule.c Modules/operator.c Objects/abstract.c Objects/classobject.c Objects/ In-Reply-To: References: Message-ID: <44DEC72A.7020007@gmail.com> Travis E. Oliphant wrote: > Internally PyNumber_AsSize_t makes a call to PyNumber_Index, and > PyNumber_Index also calls the PyIndex_Check as well . So, basically we > end up calling PyIndex_Check(obj) 2 times when only one check should be > necessary. > > This code could be re-written to move any other type checks first and > replace the PyIndex_Check(obj) code with PyNumber_Index(obj) and error > handling but I'm not sure if that's the right way to go or if it's worth > it. This concern was what lead me up the garden path with the more complicated C API in my patch. I've since become convinced that compared to everything else going on, the repetition of the 3 not-NULL checks performed by that macro really isn't worth worrying about (particularly at this stage of the release cycle). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From scott+python-dev at scottdial.com Mon Aug 14 00:46:27 2006 From: scott+python-dev at scottdial.com (Scott Dial) Date: Sun, 13 Aug 2006 18:46:27 -0400 Subject: [Python-Dev] dict containment annoyance In-Reply-To: <20060812170752.1717.1940170045.divmod.quotient.20817@ohm> References: <20060812170752.1717.1940170045.divmod.quotient.20817@ohm> Message-ID: <44DFABC3.2010201@scottdial.com> Jean-Paul Calderone wrote: > def blacklisted(o): > try: > # Is the object contained in the blacklist set? > return o in _blacklistset > except TypeError: > # If it *cannot* be contained in the blacklist set, > # then it probably isn't. > return False I feel confident that the parent could've wrote this snippet himself. His point was to discuss the semantic logic of "x in y" throwing and exception when x can't be in y (because of the TypeError you are catching in your snippet). FWIW, I think the logic of swallowing the TypeError is completely reasonable. It is surprising in a way that you are presented with an exception. However, I can't say I have ever ran into this. For my money, it would make the most sense to have your own blacklistset type. class BlacklistSet(dict): def __contains__(self, x): try: hash(x) except TypeError: return False return super(BlacklistSet, self).__contains__(x) It's however unfortunate that you cannot use the handy curly-braces anymore. -- Scott Dial scott at scottdial.com scodial at indiana.edu From nnorwitz at gmail.com Mon Aug 14 02:48:20 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 13 Aug 2006 17:48:20 -0700 Subject: [Python-Dev] test_socketserver failure on cygwin Message-ID: I'm not sure if this is a race condition in the test, the test using SocketServer inappropriately, or if the failure is exposing a problem in SocketServer. The issue is that when the second (UDP) server is setup, there is still a child process from the first (TCP) server. So when the UDP server calls collect_children(), it gets back the pid from the TCP server, which is not in the UDP server's list and an exception is raised. In general, this is a problem, if program is using a ForkingServer and spawns it's own children, this problem can occur. This is because os.waitpid() is called with a pid of 0, rather than the individual pids in the list. I suppose that would fix the problem. Is this worth fixing? (It's a pain to handle properly because there is also use of a blocking wait.) http://www.python.org/dev/buildbot/all/x86%20cygwin%20trunk/builds/1107/step-test/0 Add this patch corrects the problem (see below for more details): Index: Lib/test/test_socketserver.py =================================================================== --- Lib/test/test_socketserver.py (revision 51260) +++ Lib/test/test_socketserver.py (working copy) @@ -183,7 +183,11 @@ def testall(): testloop(socket.AF_INET, tcpservers, MyStreamHandler, teststream) + time.sleep(.1) + reap_children() testloop(socket.AF_INET, udpservers, MyDatagramHandler, testdgram) + time.sleep(.1) + reap_children() if hasattr(socket, 'AF_UNIX'): testloop(socket.AF_UNIX, streamservers, MyStreamHandler, teststream) # Alas, on Linux (at least) recvfrom() doesn't return a meaningful So modifying SocketServer like this: -bash-3.1$ svn diff Lib/SocketServer.py Lib/test/test_socketserver.py Index: Lib/SocketServer.py =================================================================== --- Lib/SocketServer.py (revision 51260) +++ Lib/SocketServer.py (working copy) @@ -420,6 +420,7 @@ except os.error: pid = None if not pid: break + print >> sys.__stdout__, 'pid', pid, 'active', self.active_children self.active_children.remove(pid) def process_request(self, request, client_address): @@ -428,6 +429,7 @@ pid = os.fork() if pid: # Parent process + print >> sys.__stdout__, 'in parent, adding', pid, self.active_children if self.active_children is None: self.active_children = [] self.active_children.append(pid) We see this on error (prior to the test patch above): test_socketserver in parent, adding 3084 None pid 3084 active [3084] in parent, adding 2408 [] pid 2408 active [2408] in parent, adding 3908 [] in parent, adding 532 None pid 3908 active [532] Exception in thread Thread-12: Traceback (most recent call last): ... File "/home/neal/build/python/trunk/Lib/SocketServer.py", line 424, in collect_children self.active_children.remove(pid) ValueError: list.remove(x): x not in list From guido at python.org Mon Aug 14 03:49:29 2006 From: guido at python.org (Guido van Rossum) Date: Sun, 13 Aug 2006 18:49:29 -0700 Subject: [Python-Dev] dict containment annoyance In-Reply-To: <44DFABC3.2010201@scottdial.com> References: <20060812170752.1717.1940170045.divmod.quotient.20817@ohm> <44DFABC3.2010201@scottdial.com> Message-ID: On 8/13/06, Scott Dial wrote: > FWIW, I think the logic of swallowing the TypeError is completely > reasonable. Then you haven't debugged enough Python programs. Swallowing an exception of *any* kind is always a trap waiting to shut when you least expect it, because you have no control over what other operations might cause an exception. E.g. when you write try: hash(x) except TypeError: # apparently x is not hashable then you're also swallowing any type errors in the computation of a legitimate hash function. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Mon Aug 14 04:26:14 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 14 Aug 2006 14:26:14 +1200 Subject: [Python-Dev] NotHashableError? (Re: dict containment annoyance) In-Reply-To: References: <20060812170752.1717.1940170045.divmod.quotient.20817@ohm> <44DFABC3.2010201@scottdial.com> Message-ID: <44DFDF46.8060404@canterbury.ac.nz> Guido van Rossum wrote: > try: > hash(x) > except TypeError: > # apparently x is not hashable > > then you're also swallowing any type errors in the computation of a > legitimate hash function. Maybe it would help if there were a specific exception, such as NotHashableError, that hash functions were expected to raise in this situation instead of a generic TypeError. For backwards compatibility it could be a subclass of TypeError. -- Greg From scott+python-dev at scottdial.com Mon Aug 14 05:49:09 2006 From: scott+python-dev at scottdial.com (Scott Dial) Date: Sun, 13 Aug 2006 23:49:09 -0400 Subject: [Python-Dev] NotHashableError? (Re: dict containment annoyance) In-Reply-To: <44DFDF46.8060404@canterbury.ac.nz> References: <20060812170752.1717.1940170045.divmod.quotient.20817@ohm> <44DFABC3.2010201@scottdial.com> <44DFDF46.8060404@canterbury.ac.nz> Message-ID: <44DFF2B5.4050902@scottdial.com> Greg Ewing wrote: > Guido van Rossum wrote: >> try: >> hash(x) >> except TypeError: >> # apparently x is not hashable >> >> then you're also swallowing any type errors in the computation of a >> legitimate hash function. > > Maybe it would help if there were a specific exception, > such as NotHashableError, that hash functions were > expected to raise in this situation instead of a > generic TypeError. I was going to suggest this, but then I realized that what is really desirable is a check for hashability. But since hashability is a deep property, it is difficult to check for.. seems like an very specific exception type would be a simple solution. FYI, I appreciate that swallowing all TypeErrors is bad form, so it would be nice if it was possible to differentiate ;-) Some brief googling on the subject yield a discussion on py3k about this very subject. http://mail.python.org/pipermail/python-3000/2006-July/002697.html Guido wrote: > Personally, I'm not sure this problem needs solving; I don't recall > ever needing to know whether something is hashable. So perhaps it's of > purely theoretical importance? That would suit me fine given the above > dilemma... > > --Guido -- Scott Dial scott at scottdial.com scodial at indiana.edu From ncoghlan at gmail.com Mon Aug 14 11:10:20 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 14 Aug 2006 19:10:20 +1000 Subject: [Python-Dev] 2.6 idea: a 'function' builtin to parallel classmethod and staticmethod In-Reply-To: References: <44DD412E.9060109@iinet.net.au> Message-ID: <44E03DFC.2010600@gmail.com> Guido van Rossum wrote: > It ought to be called @instancemethod for a better analogy. That's actually what I was originally going to suggest. And then I changed my mind after wondering how I would answer the question "Why don't I need to decorate functions with @instancemethod in order to use them as instance methods, but I need to decorate them with @classmethod and @staticmethod to use them as class and static methods?". However, now that I've thought about it some more, the explanation would simply be that functions behave like instancemethod instances by default, while other callables typically behave like staticmethod instances. So instancemethod(f) would return the passed in object for an actual function, and a wrapper with an appropriate __get__ method for anything else (stealing __doc__ etc from the original callable). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From mal at egenix.com Mon Aug 14 12:58:48 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 14 Aug 2006 12:58:48 +0200 Subject: [Python-Dev] Dict suppressing exceptions In-Reply-To: <44DCF26E.6090003@egenix.com> References: <44DAF23A.103@egenix.com> <44DB84EE.8020606@v.loewis.de> <20060810211254.GA24181@code0.codespeak.net> <20060811092757.GA27929@code0.codespeak.net> <44DCABF7.6040709@egenix.com> <44DCF26E.6090003@egenix.com> Message-ID: <44E05768.2070207@egenix.com> M.-A. Lemburg wrote: > M.-A. Lemburg wrote: >> Guido van Rossum wrote: >>> Marc-Andre, how's the patch coming along? >> I'm working on it. >> >> Since we only want equal compares to generate the warning, >> I have to add a rich compare function to Unicode objects. > > Here's an initial version: > > http://sourceforge.net/tracker/index.php?func=detail&aid=1538956&group_id=5470&atid=305470 > > The title of the patch is slightly incorrect - SF doesn't allow > more descriptive titles... :-( FYI: I've checked in the patch as revision 51276. Please give it some serious testing. All tests pass, but there may still be some corner cases left which the tests don't catch. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 14 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From fredrik at pythonware.com Mon Aug 14 13:51:48 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Mon, 14 Aug 2006 13:51:48 +0200 Subject: [Python-Dev] Elementtree and Namespaces in 2.5 References: Message-ID: "Chris S" wrote: > and while most users and the w3 spec > (http://www.w3.org/TR/2001/REC-xml-c14n-20010315#NoNSPrefixRewriting) > agree this feature is actually a bug ET's not a canonicalization library, and doesn't claim to be one, so that reference isn't very relevant. And "most users" know that ET uses a simplified infoset; it's not exactly a secret. I think you need to work on your FUD skills. > at least patch the stdlib's version of Elementtree to produce an output > more in line with the w3 standard. And that patch would look exactly how ? From fredrik at pythonware.com Mon Aug 14 13:58:40 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Mon, 14 Aug 2006 13:58:40 +0200 Subject: [Python-Dev] Elementtree and Namespaces in 2.5 References: <44DCDC18.3000701@v.loewis.de> Message-ID: Martin v. Löwis wrote: > That is not enough reason. Yes, it makes certain applications > impossible, e.g. when namespace prefixes are inside attribute > values. It just means you can't use it for that application, > then. XML has many other applications, and so does ElementTree. there are ways to deal with this in ET, though; the "iterparse" interface gives you access to namespace prefixes, the semi-official _namespace_map dictionary gives you better control over how the default serializer maps URI:s to prefixes, and the "QName" wrapper allows you to use qualified names for individual attribute values. dealing with namespace abusers such as SOAP and similar XML dialects are per- fectly possible, and usually not very hard. (ET 1.3 will probably improve things further, but that won't be out before 2.5 final.) From fredrik at pythonware.com Mon Aug 14 14:06:08 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Mon, 14 Aug 2006 14:06:08 +0200 Subject: [Python-Dev] SimpleXMLWriter missing from elementtree References: Message-ID: Ralf Schmitt wrote: > any chance to get SimpleXMLWriter (and other modules maybe??) getting > included into xml.etree? Otherwise people might have to stick to the > original elementtree, which doesn't really make sense, since most of > elementtree already is included. the original decision was to start with three fundamental modules, and leave out less popular and potentially less stable modules. leaving out SimpleXMLWriter (and perhaps the HTML stuff) might have been a mistake, but with 2.5 in final beta, that's how things are going to be in 2.5. From noamraph at gmail.com Mon Aug 14 21:18:45 2006 From: noamraph at gmail.com (Noam Raphael) Date: Mon, 14 Aug 2006 22:18:45 +0300 Subject: [Python-Dev] ANN: byteplay - a bytecode assembler/disassembler Message-ID: Hello, I wanted to tell you that I wrote a Python bytecode assembler/disassembler, and would be happy if people tried it and said what they think. I send this message to this list because this module deals with pretty low-level Python, so I thought it might interest the audience here. If I was wrong - please forgive me. The idea is to define an object which is equivalent to Python's code object, but which is easy to work with. To explain what I mean, I'll show a quick example. We can define this stupid function: >>> def f(a, b): ... print (a, b) We can convert it to an equivalent object, and see how it stores the byte code: >>> from byteplay import * >>> c = Code.from_code(f.func_code) >>> from pprint import pprint; pprint(c.code) [(SetLineno, 2), (LOAD_FAST, 'a'), (LOAD_FAST, 'b'), (BUILD_TUPLE, 2), (PRINT_ITEM, None), (PRINT_NEWLINE, None), (LOAD_CONST, None), (RETURN_VALUE, None)] We can change the bytecode easily, and see what happens: >>> c.code[3:3] = [(ROT_TWO, None)] >>> f.func_code = c.to_code() >>> f(3, 5) (5, 3) The idea is basically the same as that of Michael Hudson's bytecodehacks, but this one works with Python 2.4 and 2.5. I also think that it's simpler to use. I borrowed some code from Phillip J. Eby's peak.util.assembler - the main difference between his package and mine is that mine lets you play with existing bytecode, not only create new code objects. I learned a lot about Python's bytecode from writing this, and I think that other may learn from it as well - I think it's much easier to understand how code objects work by understanding equivalent objects which were meant to be as simple as possible, instead of as fast as possible. I think it got pretty good testing - I patched __import__ so that after a module is imported, all function objects (found by the gc module) were disassembled and assembled again. I then managed to get the complete test suite to pass! You can download the module from http://byteplay.googlecode.com/svn/trunk/byteplay.py . I wrote a documentation (which goes into some length in purpose of explaining how bytecode works). It's on the wiki: http://wiki.python.org/moin/ByteplayDoc . I even thought that it might get to the standard library, because it seemed to me to be a pretty good documentation of bytecode details, and because it has a pretty straightforward interface. It also makes meddling with bytecode much less dangerous - for example, you can see Raymond's original recipe for binding constants at compile time (I reposted it at http://python.pastebin.com/768312) and how using byteplay makes it simple (posted at http://python.pastebin.com/768318 so you can view the diff). But, of course - it's up to you. I will be entirely satisfied if people simply find it useful. Have a good day, Noam From amk at amk.ca Mon Aug 14 23:51:06 2006 From: amk at amk.ca (A.M. Kuchling) Date: Mon, 14 Aug 2006 17:51:06 -0400 Subject: [Python-Dev] Arlington VA sprint on Sept. 23 Message-ID: <20060814215106.GA851@Siri.local> The CanDo group continues to have sprints in Arlington, so we may as well continue to piggyback on their space. The next one will be Saturday Sept. 23; sign up at . If the PEP 356 plan holds, Python 2.5 will have been released a week and a half before. I plan to work on the bug/patch backlog (try searching for the Python 2.3 bugs on SF and see how many open bugs there are). --amk From g.brandl at gmx.net Tue Aug 15 00:21:34 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 15 Aug 2006 00:21:34 +0200 Subject: [Python-Dev] Arlington VA sprint on Sept. 23 In-Reply-To: <20060814215106.GA851@Siri.local> References: <20060814215106.GA851@Siri.local> Message-ID: A.M. Kuchling wrote: > The CanDo group continues to have sprints in Arlington, so we may as > well continue to piggyback on their space. The next one will be > Saturday Sept. 23; sign up at > . > > If the PEP 356 plan holds, Python 2.5 will have been released a week > and a half before. I plan to work on the bug/patch backlog (try > searching for the Python 2.3 bugs on SF and see how many open bugs > there are). Once we do have a new tracker system, it would be a Good Thing[tm] to go through all the bugs, patches and RFE, and try to get them properly categorized: we'd have to * merge duplicates (also bug + patch duplicates caused by the current split trackers) * assign a realistic priority * assign a realistic milestone * flag RFE patches as RFE ("patch" shouldn't be a category on its own) * try to reproduce if bug is quite old * close old bugs which can't be reproduced or refer to obscure platforms * close RFEs and patches which have no chance of going in (there are many patches on SF having one or two "-1" comments, is anyone ever going to check them in without a python-dev discussion?) Quite a lot of work, but with a fair amount of volunteers it should be doable and cut the number of open issues to under 1000. Georg From oliphant.travis at ieee.org Tue Aug 15 00:28:45 2006 From: oliphant.travis at ieee.org (Travis E. Oliphant) Date: Mon, 14 Aug 2006 16:28:45 -0600 Subject: [Python-Dev] Fwd: [Python-checkins] r51236 - in python/trunk: Doc/api/abstract.tex Include/abstract.h Include/object.h Lib/test/test_index.py Misc/NEWS Modules/arraymodule.c Modules/mmapmodule.c Modules/operator.c Objects/abstract.c Objects/classobject.c Objects/ In-Reply-To: References: Message-ID: <44E0F91D.1090206@ieee.org> Travis E. Oliphant wrote: > > The idea is that the __index__() method should return an exact int or > exact long or this call will raise an error. The restriction is present > to remove the possibility of infinite recursion (though I'm not sure > where that would occur exactly). > I just realized that returning a sub-class of Int and/or Long would not be a problem here. The recursion problem that I was guarding against only arose when the possibility of returning any object with an .__index__() method was suggested. Therefore, I think these exact int or exact long checks can and should be replaced with PyInt_Check() and PyLong_Check(). -Travis From fdrake at acm.org Tue Aug 15 00:34:08 2006 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Mon, 14 Aug 2006 18:34:08 -0400 Subject: [Python-Dev] Arlington VA sprint on Sept. 23 In-Reply-To: References: <20060814215106.GA851@Siri.local> Message-ID: <200608141834.08899.fdrake@acm.org> On Monday 14 August 2006 18:21, Georg Brandl wrote: > * flag RFE patches as RFE ("patch" shouldn't be a category on its own) This is something Martin and I have disagreed over in the past. Martin has indicated that he'd rather see the patches as separate artifacts rather than as attachments to a bug report, while I'd rather see them attached to the relevant bug report or feature request. My thought is that it's easier to deal with fewer items in the tracker. Keeping the candidate patches with the bug report or feature request makes them readily accessible to a reviewer. It's not the only way. I can guess at Martin's thinking, but I'd rather let him speak for himself, since I'm not a trained channeller. ;-) -Fred -- Fred L. Drake, Jr. From alexander.belopolsky at gmail.com Tue Aug 15 01:08:18 2006 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Mon, 14 Aug 2006 19:08:18 -0400 Subject: [Python-Dev] Type of range object members Message-ID: The range object is currently defined in Objects/rangeobject.c as typedef struct { PyObject_HEAD long start; long step; long len; } rangeobject; Is this consistent with PEP 353, or should Py_ssize_t be used instead of long? It looks like some of the code in rangeobject.c is already Py_ssize_t aware (see range_item and range_length), but it assumes that it is safe to cast long to ssize_t and back. From guido at python.org Tue Aug 15 01:32:26 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 14 Aug 2006 16:32:26 -0700 Subject: [Python-Dev] Type of range object members In-Reply-To: References: Message-ID: Methinks that as long as PyIntObject uses long (see intobject.h) there's no point in changing this to long. --Guido On 8/14/06, Alexander Belopolsky wrote: > The range object is currently defined in Objects/rangeobject.c as > > typedef struct { > PyObject_HEAD > long start; > long step; > long len; > } rangeobject; > > Is this consistent with PEP 353, or should Py_ssize_t be used instead of long? > > It looks like some of the code in rangeobject.c is already Py_ssize_t > aware (see range_item and range_length), but it assumes that it is > safe to cast long to ssize_t and back. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From amk at amk.ca Tue Aug 15 01:48:52 2006 From: amk at amk.ca (A.M. Kuchling) Date: Mon, 14 Aug 2006 19:48:52 -0400 Subject: [Python-Dev] Arlington VA sprint on Sept. 23 In-Reply-To: References: <20060814215106.GA851@Siri.local> Message-ID: <20060814234852.GB17594@Andrew-iBook2.local> On Tue, Aug 15, 2006 at 12:21:34AM +0200, Georg Brandl wrote: > * close RFEs and patches which have no chance of going in (there are many > patches on SF having one or two "-1" comments, is anyone ever going to > check them in without a python-dev discussion?) In the past, someone (I think Raymond) has argued that we shouldn't close old bug reports because even if no one is interested in processing them *now*, someone might come along someday who wants to update that module and fix those bugs or apply those enhancements. Perhaps we need a 'not dead, not actively pursued' state for patches and RFEs. --amk From ncoghlan at gmail.com Tue Aug 15 03:54:03 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 15 Aug 2006 11:54:03 +1000 Subject: [Python-Dev] Arlington VA sprint on Sept. 23 In-Reply-To: <20060814234852.GB17594@Andrew-iBook2.local> References: <20060814215106.GA851@Siri.local> <20060814234852.GB17594@Andrew-iBook2.local> Message-ID: <44E1293B.2000607@gmail.com> A.M. Kuchling wrote: > On Tue, Aug 15, 2006 at 12:21:34AM +0200, Georg Brandl wrote: >> * close RFEs and patches which have no chance of going in (there are many >> patches on SF having one or two "-1" comments, is anyone ever going to >> check them in without a python-dev discussion?) > > In the past, someone (I think Raymond) has argued that we shouldn't > close old bug reports because even if no one is interested in > processing them *now*, someone might come along someday who wants to > update that module and fix those bugs or apply those enhancements. > > Perhaps we need a 'not dead, not actively pursued' state for patches > and RFEs. Open with priority 0? Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Tue Aug 15 03:59:46 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 15 Aug 2006 11:59:46 +1000 Subject: [Python-Dev] Arlington VA sprint on Sept. 23 In-Reply-To: <200608141834.08899.fdrake@acm.org> References: <20060814215106.GA851@Siri.local> <200608141834.08899.fdrake@acm.org> Message-ID: <44E12A92.7090208@gmail.com> Fred L. Drake, Jr. wrote: > On Monday 14 August 2006 18:21, Georg Brandl wrote: > > * flag RFE patches as RFE ("patch" shouldn't be a category on its own) > > This is something Martin and I have disagreed over in the past. Martin has > indicated that he'd rather see the patches as separate artifacts rather than > as attachments to a bug report, while I'd rather see them attached to the > relevant bug report or feature request. > > My thought is that it's easier to deal with fewer items in the tracker. > Keeping the candidate patches with the bug report or feature request makes > them readily accessible to a reviewer. It's not the only way. I'd rather see this as a "has patch" flag, similar to "is RFE", rather than completely separate trackers. A patch will be aimed at addressing either an RFE or a bug fix - so long as people can filter out the tracker items with patches attached to find things to review, it makes sense to me to keep the patch with the tracker item it addresses. Another useful categorization would be "what skills are needed to address this item" - Python coding, C coding or documentation writing (although this is covered to some degree by the existing categories, it may not always be obvious). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Tue Aug 15 04:01:46 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 15 Aug 2006 12:01:46 +1000 Subject: [Python-Dev] Type of range object members In-Reply-To: References: Message-ID: <44E12B0A.9020907@gmail.com> Guido van Rossum wrote: > Methinks that as long as PyIntObject uses long (see intobject.h) > there's no point in changing this to long. Those fields are going to have to change to Py_Object* eventually if xrange() is going to become the range() replacement in Py3k. . . Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From alexander.belopolsky at gmail.com Tue Aug 15 04:08:19 2006 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Mon, 14 Aug 2006 22:08:19 -0400 Subject: [Python-Dev] Type of range object members In-Reply-To: References: Message-ID: <04EA6FD8-CECD-4766-93B0-BE7137316669@local> On Aug 14, 2006, at 7:32 PM, Guido van Rossum wrote: > Methinks that as long as PyIntObject uses long (see intobject.h) > there's no point in changing this to long. I guess you meant "changing this to Py_ssize_t ". I don't understand why the type used by PyIntObject is relevant here. Range object's "start" is logically an index, but int object's "ob_ival" is not. Since PyIntObject's is definition is exposed by Python.h, changing the type of ob_ival will probably break a lot of code. This reasoning does not apply to the range object. Since on most platforms ssize_t is the same as long, the choice between the two is just a matter of self-documenting code. Speaking of which, I find it unfortunate that the name Py_ssize_t was selected for the typedef. I would prefer Py_index_t. The first time I saw Py_ssize_t, I did not notice the double 's' and thought it was an unsigned type. On the second look, I've realized that it is signed and started wondering why not ptrdiff_t. I understand that ssize_t is defined by POSIX as the return type of functions such as "read" that can return either size or -1 for error. I don't think POSIX mandates sizeof(size_t) == sizeof(ssize_t), but I may be wrong. I would agree that ptrdiff_t, although standard C, is not a very intuitive name, but ssize_t is even less clear. From guido at python.org Tue Aug 15 04:56:55 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 14 Aug 2006 19:56:55 -0700 Subject: [Python-Dev] Type of range object members In-Reply-To: <04EA6FD8-CECD-4766-93B0-BE7137316669@local> References: <04EA6FD8-CECD-4766-93B0-BE7137316669@local> Message-ID: On 8/14/06, Alexander Belopolsky wrote: > > On Aug 14, 2006, at 7:32 PM, Guido van Rossum wrote: > > > Methinks that as long as PyIntObject uses long (see intobject.h) > > there's no point in changing this to long. > > I guess you meant "changing this to Py_ssize_t ". Yup, sorry. > I don't understand > why the type used by PyIntObject is relevant here. Because the only way to create one (in 2.5 or before) is by passing it a Python int. > Range object's > "start" is logically an index, but int object's "ob_ival" is not. > Since PyIntObject's is definition is exposed by Python.h, changing > the type of ob_ival will probably break a lot of code. This > reasoning does not apply to the range object. But since the start and end come from a Python int, what advantage would it have to use Py_ssize_t instead? We know sizeof(long) <= sizeof(Py_ssize_t). > Since on most platforms ssize_t is the same as long, the choice > between the two is just a matter of self-documenting code. Speaking > of which, I find it unfortunate that the name Py_ssize_t was selected > for the typedef. I would prefer Py_index_t. The first time I saw > Py_ssize_t, I did not notice the double 's' and thought it was an > unsigned type. Blame the C standards committee -- they introduced ssize_t in C99. > On the second look, I've realized that it is signed > and started wondering why not ptrdiff_t. Because it is not used for the difference of pointers? Frankly, you're about 6 months too late with naming concerns. It's not going to change now. The PEP has been discussed and the code reviewed ad infinitum. We're only looking for bugs now, and so far the issue you've brought up decidedly falls in the category "non-bug". > I understand that ssize_t > is defined by POSIX as the return type of functions such as "read" > that can return either size or -1 for error. I don't think POSIX > mandates sizeof(size_t) == sizeof(ssize_t), but I may be wrong. I > would agree that ptrdiff_t, although standard C, is not a very > intuitive name, but ssize_t is even less clear. Water under the bridge. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From alexander.belopolsky at gmail.com Tue Aug 15 05:58:49 2006 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Mon, 14 Aug 2006 23:58:49 -0400 Subject: [Python-Dev] Type of range object members In-Reply-To: References: <04EA6FD8-CECD-4766-93B0-BE7137316669@local> Message-ID: <80BC7ADD-01E0-41E5-B5B5-021765C6C865@local> On Aug 14, 2006, at 10:56 PM, Guido van Rossum wrote: > > Because the only way to create one (in 2.5 or before) is by passing it > a Python int. > Is that true? Python 2.4.2 (#2, Jan 13 2006, 12:00:38) >>> xrange(long(2)) xrange(2) > > But since the start and end come from a Python int, what advantage > would it have to use Py_ssize_t instead? We know sizeof(long) <= > sizeof(Py_ssize_t). They don't have to come from a python int, they can come from a long. I guess range_new would have to be changed to the 'n' conversion code instead of 'l' to do the proper conversion. Similarly, rangeiter_next will need to use PyInt_FromSsize_t . > > Blame the C standards committee -- they introduced ssize_t in C99. I've checked in the Wiley's 2003 edition of "The C Standard" and it does not have ssize_t . I could not find anything suggesting that it is in the C standard on google either. Are you sure it is in C99? > >> On the second look, I've realized that it is signed >> and started wondering why not ptrdiff_t. > > Because it is not used for the difference of pointers? After being exposed to C++, ptrdiff_t (or more generally difference_type) is what I expect as an argument of a [] operator that allows negative indices (such as operator[] of C++ iterators). > > Frankly, you're about 6 months too late with naming concerns. I know :-( I'll shut up now. From guido at python.org Tue Aug 15 06:14:09 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 14 Aug 2006 21:14:09 -0700 Subject: [Python-Dev] Type of range object members In-Reply-To: <80BC7ADD-01E0-41E5-B5B5-021765C6C865@local> References: <04EA6FD8-CECD-4766-93B0-BE7137316669@local> <80BC7ADD-01E0-41E5-B5B5-021765C6C865@local> Message-ID: On 8/14/06, Alexander Belopolsky wrote: > > On Aug 14, 2006, at 10:56 PM, Guido van Rossum wrote: > > > Because the only way to create one (in 2.5 or before) is by passing it > > a Python int. > > Is that true? > > Python 2.4.2 (#2, Jan 13 2006, 12:00:38) > >>> xrange(long(2)) > xrange(2) But a long with a value larger than sys.maxint is never accepted right? > > But since the start and end come from a Python int, what advantage > > would it have to use Py_ssize_t instead? We know sizeof(long) <= > > sizeof(Py_ssize_t). > > They don't have to come from a python int, they can come from a > long. I guess range_new would have to be changed to the 'n' > conversion code instead of 'l' to do the proper conversion. > Similarly, rangeiter_next will need to use PyInt_FromSsize_t . Feel free to submit a patch for Python 2.6. I expect the 2.5 release managers to reject this on the basis of being a new feature. > > Blame the C standards committee -- they introduced ssize_t in C99. > > I've checked in the Wiley's 2003 edition of "The C Standard" and it > does not have ssize_t . I could not find anything suggesting that it > is in the C standard on google either. Are you sure it is in C99? To be honest I have no idea how/why Martin or Tim picked this name. Perhaps it is in POSIX? I doubt they made it up; ssize_t is used all over the standard headers in the 4-year-old Linux distro (Red Hat 7.3) running on my ancient home desktop. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From nnorwitz at gmail.com Tue Aug 15 06:31:16 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 14 Aug 2006 21:31:16 -0700 Subject: [Python-Dev] no remaining issues blocking 2.5 release Message-ID: I just updated the PEP to remove all references to issues blocking release of 2.5. I don't know of any. I haven't heard of any issues with the fixes that have been checked in. If you have issues, respond ASAP! The release candidate is planned to be cut this Thursday/Friday. There are only a few more days before code freeze. A branch will be made when the release candidate is cut. n From anthony at interlink.com.au Tue Aug 15 07:32:09 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue, 15 Aug 2006 15:32:09 +1000 Subject: [Python-Dev] Type of range object members In-Reply-To: References: <80BC7ADD-01E0-41E5-B5B5-021765C6C865@local> Message-ID: <200608151532.11881.anthony@interlink.com.au> On Tuesday 15 August 2006 14:14, Guido van Rossum wrote: > > They don't have to come from a python int, they can come from a > > long. I guess range_new would have to be changed to the 'n' > > conversion code instead of 'l' to do the proper conversion. > > Similarly, rangeiter_next will need to use PyInt_FromSsize_t . > > Feel free to submit a patch for Python 2.6. I expect the 2.5 release > managers to reject this on the basis of being a new feature. Absolutely correct. At this point in the release process, there's going to be a massive resistance to changes. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From anthony at interlink.com.au Tue Aug 15 07:36:46 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue, 15 Aug 2006 15:36:46 +1000 Subject: [Python-Dev] no remaining issues blocking 2.5 release In-Reply-To: References: Message-ID: <200608151536.48043.anthony@interlink.com.au> On Tuesday 15 August 2006 14:31, Neal Norwitz wrote: > I just updated the PEP to remove all references to issues blocking > release of 2.5. > I don't know of any. I haven't heard of any issues with the fixes > that have been checked in. > > If you have issues, respond ASAP! The release candidate is planned to > be cut this Thursday/Friday. There are only a few more days before > code freeze. A branch will be made when the release candidate is cut. More information here: I plan to branch release25-maint on Thursday morning, as the first step in the release process for 2.5c1. This will be done after the repository is updated to identify itself as '2.5c1'. Once this is done, I plan on doing all further 2.5 releases from the release25-maint branch. I will no longer care about the trunk's stability :-) Anthony -- Anthony Baxter It's never too late to have a happy childhood. From fredrik at pythonware.com Tue Aug 15 08:17:09 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue, 15 Aug 2006 08:17:09 +0200 Subject: [Python-Dev] Type of range object members In-Reply-To: References: <04EA6FD8-CECD-4766-93B0-BE7137316669@local> <80BC7ADD-01E0-41E5-B5B5-021765C6C865@local> Message-ID: Guido van Rossum wrote: > To be honest I have no idea how/why Martin or Tim picked this name. > Perhaps it is in POSIX? it's from sys/types.h, which is a posix thing, afaik: http://www.opengroup.org/onlinepubs/007908799/xsh/systypes.h.html From martin at v.loewis.de Tue Aug 15 09:16:59 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 15 Aug 2006 09:16:59 +0200 Subject: [Python-Dev] Type of range object members In-Reply-To: References: Message-ID: <44E174EB.1080305@v.loewis.de> Alexander Belopolsky wrote: > The range object is currently defined in Objects/rangeobject.c as > > typedef struct { > PyObject_HEAD > long start; > long step; > long len; > } rangeobject; > > Is this consistent with PEP 353, or should Py_ssize_t be used instead of long? As others have said: no. The range object produces ints. People have asked to make it produce arbitrary longs instead, but none of these patches ever got successfully reviewed. > It looks like some of the code in rangeobject.c is already Py_ssize_t > aware (see range_item and range_length), but it assumes that it is > safe to cast long to ssize_t and back. Where does it assume that it is safe to case ssize_t -> long? That would be a bug. Regards, Martin From martin at v.loewis.de Tue Aug 15 09:25:35 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 15 Aug 2006 09:25:35 +0200 Subject: [Python-Dev] Type of range object members In-Reply-To: <04EA6FD8-CECD-4766-93B0-BE7137316669@local> References: <04EA6FD8-CECD-4766-93B0-BE7137316669@local> Message-ID: <44E176EF.2080707@v.loewis.de> Alexander Belopolsky wrote: > Since on most platforms ssize_t is the same as long, the choice > between the two is just a matter of self-documenting code. No, it would be an actual change: on Win64, sizeof(Py_ssize_t)>sizeof(long). > Speaking > of which, I find it unfortunate that the name Py_ssize_t was selected > for the typedef. I would prefer Py_index_t. The first time I saw > Py_ssize_t, I did not notice the double 's' and thought it was an > unsigned type. Hmm. That you fail to read it correctly can hardly be an argument against it. > On the second look, I've realized that it is signed > and started wondering why not ptrdiff_t. I understand that ssize_t > is defined by POSIX as the return type of functions such as "read" > that can return either size or -1 for error. I don't think POSIX > mandates sizeof(size_t) == sizeof(ssize_t), but I may be wrong. In the rationale (XRAT) they say "This is intended to be a signed analog of size_t." They don't mandate it to have the same size, but it is the expectation that implementations typically will. > I would agree that ptrdiff_t, although standard C, is not a very > intuitive name, but ssize_t is even less clear. In the discussion, ptrdiff_t was never brought up as an alternative (intptr_t was; see the PEP for why it is unsuitable). ssize_t seemed most natural to me since size_t is the C "number of bytes" type, and we need a signed version of it. Regards, Martin From martin at v.loewis.de Tue Aug 15 09:30:12 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 15 Aug 2006 09:30:12 +0200 Subject: [Python-Dev] Arlington VA sprint on Sept. 23 In-Reply-To: <200608141834.08899.fdrake@acm.org> References: <20060814215106.GA851@Siri.local> <200608141834.08899.fdrake@acm.org> Message-ID: <44E17804.204@v.loewis.de> Fred L. Drake, Jr. wrote: > I can guess at Martin's thinking, but I'd rather let him speak for himself, > since I'm not a trained channeller. ;-) Essentially, I want to give patches more attention, since they are larger contributions. I don't care if bug reports get unnoticed for years; I do worry if patches get unnoticed/unprocessed for years. Regards, Martin From g.brandl at gmx.net Tue Aug 15 10:18:07 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 15 Aug 2006 10:18:07 +0200 Subject: [Python-Dev] Arlington VA sprint on Sept. 23 In-Reply-To: <44E17804.204@v.loewis.de> References: <20060814215106.GA851@Siri.local> <200608141834.08899.fdrake@acm.org> <44E17804.204@v.loewis.de> Message-ID: Martin v. L?wis wrote: > Fred L. Drake, Jr. wrote: >> I can guess at Martin's thinking, but I'd rather let him speak for himself, >> since I'm not a trained channeller. ;-) > > Essentially, I want to give patches more attention, since they are > larger contributions. I don't care if bug reports get unnoticed for > years; I do worry if patches get unnoticed/unprocessed for years. Then a "has-patch" flag would be okay, wouldn't it, along with assigning a higher priority? Georg From martin at v.loewis.de Tue Aug 15 10:43:41 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 15 Aug 2006 10:43:41 +0200 Subject: [Python-Dev] Arlington VA sprint on Sept. 23 In-Reply-To: References: <20060814215106.GA851@Siri.local> <200608141834.08899.fdrake@acm.org> <44E17804.204@v.loewis.de> Message-ID: <44E1893D.7030402@v.loewis.de> Georg Brandl wrote: >>> I can guess at Martin's thinking, but I'd rather let him speak for himself, >>> since I'm not a trained channeller. ;-) >> Essentially, I want to give patches more attention, since they are >> larger contributions. I don't care if bug reports get unnoticed for >> years; I do worry if patches get unnoticed/unprocessed for years. > > Then a "has-patch" flag would be okay, wouldn't it, along with assigning > a higher priority? Sure, that would work as well. Regards, Martin From theller at python.net Tue Aug 15 14:57:36 2006 From: theller at python.net (Thomas Heller) Date: Tue, 15 Aug 2006 14:57:36 +0200 Subject: [Python-Dev] no remaining issues blocking 2.5 release In-Reply-To: References: Message-ID: Neal Norwitz schrieb: > I just updated the PEP to remove all references to issues blocking > release of 2.5. > I don't know of any. I haven't heard of any issues with the fixes > that have been checked in. > > If you have issues, respond ASAP! The release candidate is planned to > be cut this Thursday/Friday. There are only a few more days before > code freeze. A branch will be made when the release candidate is cut. What is the policy for documentation changes? Am I allowed to check in changes/additions to the ctypes docs without release manager permission after the release candidate is out? I'l always make sure that the html docs can be built. Thanks, Thomas From mal at egenix.com Tue Aug 15 16:06:18 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 15 Aug 2006 16:06:18 +0200 Subject: [Python-Dev] no remaining issues blocking 2.5 release In-Reply-To: References: Message-ID: <44E1D4DA.2080002@egenix.com> Neal Norwitz wrote: > I just updated the PEP to remove all references to issues blocking > release of 2.5. > I don't know of any. I haven't heard of any issues with the fixes > that have been checked in. > > If you have issues, respond ASAP! The release candidate is planned to > be cut this Thursday/Friday. There are only a few more days before > code freeze. A branch will be made when the release candidate is cut. The distutils version number should be changed back to a static string literal. It's currently setup to get its version number from the Python version running it which pretty much defeats the whole purpose of having a version number and makes using the SVN distutils package with other Python versions problematic. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 15 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From alexander.belopolsky at gmail.com Tue Aug 15 16:32:45 2006 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 15 Aug 2006 10:32:45 -0400 Subject: [Python-Dev] Type of range object members In-Reply-To: References: <04EA6FD8-CECD-4766-93B0-BE7137316669@local> <80BC7ADD-01E0-41E5-B5B5-021765C6C865@local> Message-ID: <25F486F7-B449-4FE1-8C11-DCEFDDE4CBE2@local> On Aug 15, 2006, at 12:14 AM, Guido van Rossum wrote: > > Feel free to submit a patch for Python 2.6. Please see http://sourceforge.net/tracker/index.php? func=detail&aid=1540617&group_id=5470&atid=305470 > Perhaps it is in POSIX? Yes, "ssize_t - Used for a count of bytes or an error indication.", defined in sys/types.h . I promised to shut up, but let me respond to Martin: On Aug 15, 2006, at 3:25 AM, Martin v. L?wis wrote: > Alexander Belopolsky wrote: >> [snip] I did not notice the double 's' and thought it was an >> unsigned type. > > Hmm. That you fail to read it correctly can hardly be an argument > against it. Google tells me I am not the only one: . What happened to "readability counts"? > > In the rationale (XRAT) they say > > "This is intended to be a signed analog of size_t." That's POSIX rationale . PEP 353 says: "A new type Py_ssize_t is introduced, which has the same size as the compiler's size_t type, but is signed." > > They don't mandate it to have the same size, but it is the expectation > that implementations typically will. That's one of the reasons why POSIX' ssize_t is a wrong analogy. Another reason is that POSIX interprets negative values of ssize_t as "an error indication", not as an offset from the end. A better POSIX analogy would be off_t (as used in lseek). > > In the discussion, ptrdiff_t was never brought up as an alternative Yes, it was, by you :-). I was not suggesting Py_ptrdiff_t. My suggestion was "Py_index_t", but I am six month late :-(. From guido at python.org Tue Aug 15 17:21:59 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 15 Aug 2006 08:21:59 -0700 Subject: [Python-Dev] no remaining issues blocking 2.5 release In-Reply-To: <44E1D4DA.2080002@egenix.com> References: <44E1D4DA.2080002@egenix.com> Message-ID: On 8/15/06, M.-A. Lemburg wrote: > The distutils version number should be changed back to a static > string literal. > > It's currently setup to get its version number > from the Python version running it which pretty much defeats > the whole purpose of having a version number and makes using the > SVN distutils package with other Python versions problematic. I am sympathetic to this case. Is there any advantage to the *users* of distutils of the dynamic version number? If it's only done for the benefit of the release managers (with whose plight I also sympathesize) I think it must be rolled back, at least as long as distutils is officially listed as a package that needs to support older versions of Python, which pretty much implies that it's okay to extract it from the 2.5 release and distribute it separately for use with older Python versions. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Aug 15 17:27:17 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 15 Aug 2006 08:27:17 -0700 Subject: [Python-Dev] Type of range object members In-Reply-To: References: Message-ID: FWIW, I propose that in py3k, the int type uses Py_ssize_t instead of long. Already decided is that in py3k, the (x)range object will support arbitrary integer sizes, so that e.g. range(10**10, 10**10+10) is valid (it currently is, but for different reasons, it isn't with xrange, which will replace range in py3k). For 2.6 and onwards, I propose to let the issue rest *or* eventually backport the py3k xrange implementation; "fixing" xrange to use Py_ssize_t doesn't seem worth the churn. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From mal at egenix.com Tue Aug 15 17:45:37 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 15 Aug 2006 17:45:37 +0200 Subject: [Python-Dev] no remaining issues blocking 2.5 release In-Reply-To: References: <44E1D4DA.2080002@egenix.com> Message-ID: <44E1EC21.2080905@egenix.com> Guido van Rossum wrote: > On 8/15/06, M.-A. Lemburg wrote: >> The distutils version number should be changed back to a static >> string literal. >> >> It's currently setup to get its version number >> from the Python version running it which pretty much defeats >> the whole purpose of having a version number and makes using the >> SVN distutils package with other Python versions problematic. > > I am sympathetic to this case. Is there any advantage to the *users* > of distutils of the dynamic version number? If it's only done for the > benefit of the release managers (with whose plight I also > sympathesize) FWIW, I've already volunteered to do the version bumping to take that burden off the release managers. > I think it must be rolled back, at least as long as > distutils is officially listed as a package that needs to support > older versions of Python, which pretty much implies that it's okay to > extract it from the 2.5 release and distribute it separately for use > with older Python versions. Well, that's another point we should discuss after 2.5 is out the door: distutils was delisted from PEP 291 (without public discussion). I find it important to maintain distutils compatibility with a few Python versions back. Even if I can't volunteer to maintain distutils, like Martin suggested, due to lack of time, I don't really see the requirement to use the latest and greatest Python features in distutils, so preserving compatibility with the commonly used Python versions wouldn't hurt. It makes the life of Python extension writers a tad easier and Python users benefit as a result of being able to download pre-built binaries without being forced to upgrade (this is esp. true for Python users on Windows). Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 15 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From martin at v.loewis.de Tue Aug 15 17:51:38 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 15 Aug 2006 17:51:38 +0200 Subject: [Python-Dev] no remaining issues blocking 2.5 release In-Reply-To: References: <44E1D4DA.2080002@egenix.com> Message-ID: <44E1ED8A.8080308@v.loewis.de> Guido van Rossum schrieb: > I am sympathetic to this case. Is there any advantage to the *users* > of distutils of the dynamic version number? This series of commits was triggered by a user who wondered why Python 2.4.3 ships with distutils 2.4.1, yet Python 2.5bsomething ships with the older 2.4.0; he requested that the newer release of distutils should be included in Python 2.5. So the advantage to the user is that the distutils version numbers become less confusing, and more "correct" (as long as the packaged distutils is used). > If it's only done for the > benefit of the release managers (with whose plight I also > sympathesize) In principle, the release manager shouldn't be the one updating the distutils release number. Instead, whoever modifies distutils needs to decide whether this change is a bug fix (subminor change) or feature change (minor number changes). > I think it must be rolled back, at least as long as > distutils is officially listed as a package that needs to support > older versions of Python, which pretty much implies that it's okay to > extract it from the 2.5 release and distribute it separately for use > with older Python versions. See, it isn't listed as such anymore, not since AMK removed it from PEP 291. That said, I'm not terribly opposed to it being changed back (*) - but *I* won't change it back (having it changed already twice in two days). I just wish I could then delegate all bug reports "distutils doesn't work with older Python releases" to MAL. As far as I'm concerned, this isn't a distutils feature anymore. Regards, Martin (*) where "back" means 2.5.0, not 2.4.0. From martin at v.loewis.de Tue Aug 15 17:58:24 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 15 Aug 2006 17:58:24 +0200 Subject: [Python-Dev] Type of range object members In-Reply-To: <25F486F7-B449-4FE1-8C11-DCEFDDE4CBE2@local> References: <04EA6FD8-CECD-4766-93B0-BE7137316669@local> <80BC7ADD-01E0-41E5-B5B5-021765C6C865@local> <25F486F7-B449-4FE1-8C11-DCEFDDE4CBE2@local> Message-ID: <44E1EF20.2010204@v.loewis.de> Alexander Belopolsky schrieb: > Another reason is that POSIX interprets negative values of ssize_t as > "an error indication", not as an offset from the end. A better POSIX > analogy would be off_t (as used in lseek). That's subtle. By this reasoning, ptrdiff_t would be wrong, as well, since it really means "negative index", not "count from the end". >> In the discussion, ptrdiff_t was never brought up as an alternative > > Yes, it was, by you January/059562.html> :-). Ah, that's why it isn't ptrdiff_t: Tim Peters said he liked ssize_t > I was not suggesting Py_ptrdiff_t. My suggestion was "Py_index_t", > but I am six month late :-(. Indeed, it is too late for such a change. Regards, Martin From martin at v.loewis.de Tue Aug 15 18:01:51 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 15 Aug 2006 18:01:51 +0200 Subject: [Python-Dev] Type of range object members In-Reply-To: References: Message-ID: <44E1EFEF.20305@v.loewis.de> Guido van Rossum schrieb: > FWIW, I propose that in py3k, the int type uses Py_ssize_t instead of long. This is really a py3k issue, but: shouldn't the int and long types really get unified into a single type in Py3k? I suppose the "i" parameter to PyArg_ParseTuple would continue to parse int? Regards, Martin From martin at v.loewis.de Tue Aug 15 18:04:33 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 15 Aug 2006 18:04:33 +0200 Subject: [Python-Dev] no remaining issues blocking 2.5 release In-Reply-To: <44E1EC21.2080905@egenix.com> References: <44E1D4DA.2080002@egenix.com> <44E1EC21.2080905@egenix.com> Message-ID: <44E1F091.60306@v.loewis.de> M.-A. Lemburg schrieb: > I find it important to maintain distutils compatibility with > a few Python versions back. Even if I can't volunteer to > maintain distutils, like Martin suggested, due to lack of time, > I don't really see the requirement to use the latest and greatest > Python features in distutils, so preserving compatibility > with the commonly used Python versions wouldn't hurt. Actually, it would. bdist_msi uses extension modules which are only available in 2.5. Keeping support for the compiler-recognizing code in msvccompiler also hurts. It's either an official feature, with somebody maintaining it, or people should expect to break it anytime. Regards, Martin From mal at egenix.com Tue Aug 15 18:09:28 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 15 Aug 2006 18:09:28 +0200 Subject: [Python-Dev] no remaining issues blocking 2.5 release In-Reply-To: <44E1ED8A.8080308@v.loewis.de> References: <44E1D4DA.2080002@egenix.com> <44E1ED8A.8080308@v.loewis.de> Message-ID: <44E1F1B8.2070603@egenix.com> Martin v. L?wis wrote: > Guido van Rossum schrieb: >> I think it must be rolled back, at least as long as >> distutils is officially listed as a package that needs to support >> older versions of Python, which pretty much implies that it's okay to >> extract it from the 2.5 release and distribute it separately for use >> with older Python versions. > > See, it isn't listed as such anymore, not since AMK removed it from > PEP 291. > > That said, I'm not terribly opposed to it being changed back (*) - but > *I* won't change it back (having it changed already twice in two > days). I just wish I could then delegate all bug reports "distutils > doesn't work with older Python releases" to MAL. As far as I'm > concerned, this isn't a distutils feature anymore. How many of those are there ? We're running SVN distutils regularly for Python 2.4 and use the one that shipped with Python 2.3 (the last to support 1.5.2) for all versions between 1.5.2 and 2.3. We haven't hit any bugs related to distutils not being compatible to those Python versions anymore. The few I found over the years, I corrected directly in the repository. I just checked: SVN distutils (at least the parts that we use) still works with Python 2.2. It no longer works with Python 2.1 due to the use of "True" and probably a few other things. > Regards, > Martin > > (*) where "back" means 2.5.0, not 2.4.0. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 15 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From guido at python.org Tue Aug 15 18:10:12 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 15 Aug 2006 09:10:12 -0700 Subject: [Python-Dev] Type of range object members In-Reply-To: <44E1EFEF.20305@v.loewis.de> References: <44E1EFEF.20305@v.loewis.de> Message-ID: On 8/15/06, "Martin v. L?wis" wrote: > Guido van Rossum schrieb: > > FWIW, I propose that in py3k, the int type uses Py_ssize_t instead of long. > > This is really a py3k issue, but: shouldn't the int and long types > really get unified into a single type in Py3k? >From the Python *user*'s perspective, yes, as much as possible. But I'm still playing with the thought of having two implementation types, since otherwise we'd have to devote 4 bytes (8 on a 64-bit platform) to the single *bit* telling the difference between the two internal representations. I haven't decided whether to make 'int' an abstract base type and have 'short' and 'long' subtypes (perhaps using other names), or whether to make 'int' the base type and 'short' the subtype, or whether to cheat and hack type() so that type() of any integer always returns int. But in any case, this would mean that at the C level the distinction continues to exist. Maybe we can discuss this at the sprint next week? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Aug 15 18:14:49 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 15 Aug 2006 09:14:49 -0700 Subject: [Python-Dev] no remaining issues blocking 2.5 release In-Reply-To: <44E1F1B8.2070603@egenix.com> References: <44E1D4DA.2080002@egenix.com> <44E1ED8A.8080308@v.loewis.de> <44E1F1B8.2070603@egenix.com> Message-ID: Marc-Andre, I think the release managers might let you change this back if you volunteered, not to maintain all of distutils (I wouldn't wish that on my worst enemies :-) but at least to keep the version number up to date and to do the occasional work to keep it backwards compatible in the way you want it. I think it's fine of the new distutils contains new features that won't work with older Pythons, as long as the old features still work (and have perhaps updated functionality in some other way). How exactly to reconcile this with the black/white notion of PPE 291 I don't know; it seems a compromise is in order. --Guido On 8/15/06, M.-A. Lemburg wrote: > Martin v. L?wis wrote: > > Guido van Rossum schrieb: > >> I think it must be rolled back, at least as long as > >> distutils is officially listed as a package that needs to support > >> older versions of Python, which pretty much implies that it's okay to > >> extract it from the 2.5 release and distribute it separately for use > >> with older Python versions. > > > > See, it isn't listed as such anymore, not since AMK removed it from > > PEP 291. > > > > That said, I'm not terribly opposed to it being changed back (*) - but > > *I* won't change it back (having it changed already twice in two > > days). I just wish I could then delegate all bug reports "distutils > > doesn't work with older Python releases" to MAL. As far as I'm > > concerned, this isn't a distutils feature anymore. > > How many of those are there ? > > We're running SVN distutils regularly for Python 2.4 and use > the one that shipped with Python 2.3 (the last to support 1.5.2) > for all versions between 1.5.2 and 2.3. > > We haven't hit any bugs related to distutils not being > compatible to those Python versions anymore. The few I found > over the years, I corrected directly in the repository. > > I just checked: SVN distutils (at least the parts that we use) > still works with Python 2.2. It no longer works with Python 2.1 > due to the use of "True" and probably a few other things. > > > Regards, > > Martin > > > > (*) where "back" means 2.5.0, not 2.4.0. > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, Aug 15 2006) > >>> Python/Zope Consulting and Support ... http://www.egenix.com/ > >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ > >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ > ________________________________________________________________________ > > ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From mal at egenix.com Tue Aug 15 18:17:29 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 15 Aug 2006 18:17:29 +0200 Subject: [Python-Dev] no remaining issues blocking 2.5 release In-Reply-To: <44E1F091.60306@v.loewis.de> References: <44E1D4DA.2080002@egenix.com> <44E1EC21.2080905@egenix.com> <44E1F091.60306@v.loewis.de> Message-ID: <44E1F399.5080908@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg schrieb: >> I find it important to maintain distutils compatibility with >> a few Python versions back. Even if I can't volunteer to >> maintain distutils, like Martin suggested, due to lack of time, >> I don't really see the requirement to use the latest and greatest >> Python features in distutils, so preserving compatibility >> with the commonly used Python versions wouldn't hurt. > > Actually, it would. bdist_msi uses extension modules which > are only available in 2.5. bdist_msi is only a small part of distutils. While it would be great to have that distribution format for 2.3 and 2.4 as well, I don't think it's vital. It's simply a feature that is only available when using Python 2.5. > Keeping support for the > compiler-recognizing code in msvccompiler also hurts. I guess that's something to tell Microsoft ;-) However, I don't see that as argument for dropping the distutils support for Python versions prior to 2.5. > It's either an official feature, with somebody maintaining it, > or people should expect to break it anytime. I'll let you know when things break - is that good enough ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 15 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Tue Aug 15 18:24:01 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 15 Aug 2006 18:24:01 +0200 Subject: [Python-Dev] no remaining issues blocking 2.5 release In-Reply-To: References: <44E1D4DA.2080002@egenix.com> <44E1ED8A.8080308@v.loewis.de> <44E1F1B8.2070603@egenix.com> Message-ID: <44E1F521.2010908@egenix.com> Guido van Rossum wrote: > Marc-Andre, > > I think the release managers might let you change this back if you > volunteered, not to maintain all of distutils (I wouldn't wish that on > my worst enemies :-) but at least to keep the version number up to > date and to do the occasional work to keep it backwards compatible in > the way you want it. I guess it's Anthony's turn then... the patch is there and tested, just waiting in its shell window to get checked in ;-) I'll add a distutils checkin message filter to keep me informed of any changes to distutils. > I think it's fine of the new distutils contains new features that > won't work with older Pythons, as long as the old features still work > (and have perhaps updated functionality in some other way). How > exactly to reconcile this with the black/white notion of PPE 291 I > don't know; it seems a compromise is in order. +1 > --Guido > > On 8/15/06, M.-A. Lemburg wrote: >> Martin v. L?wis wrote: >> > Guido van Rossum schrieb: >> >> I think it must be rolled back, at least as long as >> >> distutils is officially listed as a package that needs to support >> >> older versions of Python, which pretty much implies that it's okay to >> >> extract it from the 2.5 release and distribute it separately for use >> >> with older Python versions. >> > >> > See, it isn't listed as such anymore, not since AMK removed it from >> > PEP 291. >> > >> > That said, I'm not terribly opposed to it being changed back (*) - but >> > *I* won't change it back (having it changed already twice in two >> > days). I just wish I could then delegate all bug reports "distutils >> > doesn't work with older Python releases" to MAL. As far as I'm >> > concerned, this isn't a distutils feature anymore. >> >> How many of those are there ? >> >> We're running SVN distutils regularly for Python 2.4 and use >> the one that shipped with Python 2.3 (the last to support 1.5.2) >> for all versions between 1.5.2 and 2.3. >> >> We haven't hit any bugs related to distutils not being >> compatible to those Python versions anymore. The few I found >> over the years, I corrected directly in the repository. >> >> I just checked: SVN distutils (at least the parts that we use) >> still works with Python 2.2. It no longer works with Python 2.1 >> due to the use of "True" and probably a few other things. >> >> > Regards, >> > Martin >> > >> > (*) where "back" means 2.5.0, not 2.4.0. >> >> -- >> Marc-Andre Lemburg >> eGenix.com >> >> Professional Python Services directly from the Source (#1, Aug 15 2006) >> >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >> >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >> >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ >> ________________________________________________________________________ >> >> ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: >> > > -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 15 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From gustavo at niemeyer.net Tue Aug 15 18:57:09 2006 From: gustavo at niemeyer.net (Gustavo Niemeyer) Date: Tue, 15 Aug 2006 13:57:09 -0300 Subject: [Python-Dev] no remaining issues blocking 2.5 release In-Reply-To: References: Message-ID: <20060815165709.GA29060@niemeyer.net> > If you have issues, respond ASAP! The release candidate is planned to > be cut this Thursday/Friday. There are only a few more days before > code freeze. A branch will be made when the release candidate is cut. I'd like to see problem #1531862 fixed. The bug is clear and the fix should be trivial. I can commit a fix tonight, if the subprocess module author/maintainer is unavailable to check it out. -- Gustavo Niemeyer http://niemeyer.net From faassen at infrae.com Tue Aug 15 19:39:51 2006 From: faassen at infrae.com (Martijn Faassen) Date: Tue, 15 Aug 2006 19:39:51 +0200 Subject: [Python-Dev] Elementtree and Namespaces in 2.5 In-Reply-To: References: Message-ID: <44E206E7.4040403@infrae.com> Fredrik Lundh wrote: > "Chris S" wrote: > >> and while most users and the w3 spec >> (http://www.w3.org/TR/2001/REC-xml-c14n-20010315#NoNSPrefixRewriting) >> agree this feature is actually a bug > > ET's not a canonicalization library, and doesn't claim to be one, so that reference isn't > very relevant. And "most users" know that ET uses a simplified infoset; it's not exactly > a secret. And if you need an XML library with ET's API that does retain namespace prefixes and has support for canonicalization and a boatload of other features, you could try lxml. http://codespeak.net/lxml Regards, Martijn From jimjjewett at gmail.com Tue Aug 15 22:19:43 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Tue, 15 Aug 2006 16:19:43 -0400 Subject: [Python-Dev] IDLE patches - bugfix or not? Message-ID: I just uploaded a series of IDLE patches, but I'm not quite sure how to classify them on the feature/bugfix scale now that the last beta is out. >From most to least buggish: python.org/sf/1540892 -- honor the new quit() behavior. On the other hand, it was documented that this didn't work in IDLE, and it is *possible* that someone was counting on this. python.org/sf/1540851 -- with is now a blockopener, and should be counted as such -- I *think* this one would be safe, but I know that changing a parser can be surprising, and I suppose it *could* wait until with no longer requires a future statement. python.org/sf/1540874 -- broken shortcut keys. On windows, only one entry per menu can be reached with the same shortcut letter, so advertising others is just an attractive nuisance. I'm not sure that other systems wouldn't be able to use the hidden shortcuts. python.org/sf/1540869 -- GUI fix. The current code puts in a separator using a magic number (and has XXX comments about it.) This changes the magic number so that the separator is more visible, but I'm not sure the old behavior rose to a bug, or that it wasn't platform dependent. python.org/sf/1540849 -- except too broad. I wouldn't suggest applying this late in the release cycle, except that it seems sort of like the memory errors that are still being patched. -jJ From martin at v.loewis.de Wed Aug 16 00:20:23 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 16 Aug 2006 00:20:23 +0200 Subject: [Python-Dev] Type of range object members In-Reply-To: References: <44E1EFEF.20305@v.loewis.de> Message-ID: <44E248A7.4080107@v.loewis.de> Guido van Rossum schrieb: > From the Python *user*'s perspective, yes, as much as possible. But > I'm still playing with the thought of having two implementation types, > since otherwise we'd have to devote 4 bytes (8 on a 64-bit platform) > to the single *bit* telling the difference between the two internal > representations. We had this discussion before; if you use ob_size==0 to indicate that it's an int, this space isn't needed in a long int. On a 32-bit platform, the size of an int would go up from 12 to 16; if we stop using a special-cased allocator (which we should (*)), there isn't any space increase on such a platform. On a 64-bit platform, the size of an int would go up from 24 bytes to 32 bytes. Regards, Martin (*) people have complained that the memory allocated for a large number of ints isn't ever reused. They consumed it by passing range() some large argument. From guido at python.org Wed Aug 16 00:29:13 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 15 Aug 2006 15:29:13 -0700 Subject: [Python-Dev] Type of range object members In-Reply-To: <44E248A7.4080107@v.loewis.de> References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> Message-ID: On 8/15/06, "Martin v. L?wis" wrote: > Guido van Rossum schrieb: > > From the Python *user*'s perspective, yes, as much as possible. But > > I'm still playing with the thought of having two implementation types, > > since otherwise we'd have to devote 4 bytes (8 on a 64-bit platform) > > to the single *bit* telling the difference between the two internal > > representations. > > We had this discussion before; if you use ob_size==0 to indicate > that it's an int, this space isn't needed in a long int. On a 32-bit > platform, the size of an int would go up from 12 to 16; if we stop > using a special-cased allocator (which we should (*)), there isn't > any space increase on such a platform. On a 64-bit platform, the > size of an int would go up from 24 bytes to 32 bytes. > > Regards, > Martin > > (*) people have complained that the memory allocated for a large > number of ints isn't ever reused. They consumed it by passing > range() some large argument. Since range() won't return a real list any more, this no longer is the case. I worry that dropping the special allocator will be too slow. And the space increase on 64-bit platforms is too much IMO. But clearly these issues can only be addressed by careful benchmarking. Perhaps we should do some of that to settle the issue. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Wed Aug 16 00:30:18 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 16 Aug 2006 00:30:18 +0200 Subject: [Python-Dev] no remaining issues blocking 2.5 release In-Reply-To: <44E1F399.5080908@egenix.com> References: <44E1D4DA.2080002@egenix.com> <44E1EC21.2080905@egenix.com> <44E1F091.60306@v.loewis.de> <44E1F399.5080908@egenix.com> Message-ID: <44E24AFA.1090102@v.loewis.de> M.-A. Lemburg schrieb: >> It's either an official feature, with somebody maintaining it, >> or people should expect to break it anytime. > > I'll let you know when things break - is that good enough ? That can't be an official policy; you seem to define "breaks" as "breaks in my (your) personal usage of distutils". While this is fine as an end-user point of view, it isn't useful as a maintenance guideline. In case you don't see what I mean: I said that something is already broken (bdist_msi), where "broken" means "doesn't work", and you said "it's only a small part". So if it's fine that only small parts break, I wonder where this gets us over time. For another example, you found that 2.1 compatibility is now broken (due to usage of True and False). When distutils was still listed in PEP 291, 2.1 compatibility was mentioned as a goal. Now, it doesn't seem to bother you very much that 2.1 compatibility is gone. Regards, Martin From foom at fuhm.net Wed Aug 16 00:34:43 2006 From: foom at fuhm.net (James Y Knight) Date: Tue, 15 Aug 2006 18:34:43 -0400 Subject: [Python-Dev] Type of range object members In-Reply-To: <44E248A7.4080107@v.loewis.de> References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> Message-ID: <7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net> On Aug 15, 2006, at 6:20 PM, Martin v. L?wis wrote: > Guido van Rossum schrieb: >> From the Python *user*'s perspective, yes, as much as possible. But >> I'm still playing with the thought of having two implementation >> types, >> since otherwise we'd have to devote 4 bytes (8 on a 64-bit platform) >> to the single *bit* telling the difference between the two internal >> representations. > > We had this discussion before; if you use ob_size==0 to indicate > that it's an int, this space isn't needed in a long int. On a 32-bit > platform, the size of an int would go up from 12 to 16; if we stop > using a special-cased allocator (which we should (*)), there isn't > any space increase on such a platform. On a 64-bit platform, the > size of an int would go up from 24 bytes to 32 bytes. But it's the short int that you probably really want to make size efficient. Which is of course also doable via something like: typedef struct { PyObject_HEAD long ob_islong : 1; long ob_ival_or_size : LONG_BITS - 1; long ob_digit[0]; } PyIntObject; There's no particular reason that a short int must be able to store the entire range of C "long", so, as many bits can be stolen from it as desired. James From mal at egenix.com Wed Aug 16 00:55:45 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 16 Aug 2006 00:55:45 +0200 Subject: [Python-Dev] no remaining issues blocking 2.5 release In-Reply-To: <44E24AFA.1090102@v.loewis.de> References: <44E1D4DA.2080002@egenix.com> <44E1EC21.2080905@egenix.com> <44E1F091.60306@v.loewis.de> <44E1F399.5080908@egenix.com> <44E24AFA.1090102@v.loewis.de> Message-ID: <44E250F1.40309@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg schrieb: >>> It's either an official feature, with somebody maintaining it, >>> or people should expect to break it anytime. >> I'll let you know when things break - is that good enough ? > > That can't be an official policy; you seem to define "breaks" > as "breaks in my (your) personal usage of distutils". While > this is fine as an end-user point of view, it isn't useful > as a maintenance guideline. It's all I can offer. We do use quite a few distutils features and also extend it in various ways, so I suppose that our build system covers quite a bit of what distutils has to offer. Think of it as a test-bot that runs a real-life test on the distutils code. > In case you don't see what I mean: I said that something > is already broken (bdist_msi), where "broken" means > "doesn't work", and you said "it's only a small part". > So if it's fine that only small parts break, I wonder > where this gets us over time. distutils is composed of many components. bdist_msi is one of them. Because it's new it can hardly "break" distutils in any way because it's a feature that didn't exist in the distutils version shipped with Python prior to 2.5. As a result, that distribution format cannot be used with older Python versions, but the rest of distutils continues to work as usual. I wouldn't call that breakage of distutils. It's just a feature that has external requirements, much like you can't run bdist_rpm without having the RPM tool installed. In general, if you have to rely on new features in Python such as extension modules which are not available in older Python versions, then that's fine - but again, if not really necessary, I'd rather see an approach being used that also works with older Python versions. What I do consider breakage, is if you use Python 2.5 features just to beef up some part of distutils core code. > For another example, you found that 2.1 compatibility is > now broken (due to usage of True and False). When distutils > was still listed in PEP 291, 2.1 compatibility was mentioned > as a goal. Now, it doesn't seem to bother you very much > that 2.1 compatibility is gone. Actually, it does bother me, but I understand that not many people use Python 2.1 anymore, so don't insist keeping that version requirement. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 16 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From guido at python.org Wed Aug 16 01:06:52 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 15 Aug 2006 16:06:52 -0700 Subject: [Python-Dev] Type of range object members In-Reply-To: <7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net> References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net> Message-ID: On 8/15/06, James Y Knight wrote: > > On Aug 15, 2006, at 6:20 PM, Martin v. L?wis wrote: > > Guido van Rossum schrieb: > >> From the Python *user*'s perspective, yes, as much as possible. But > >> I'm still playing with the thought of having two implementation > >> types, > >> since otherwise we'd have to devote 4 bytes (8 on a 64-bit platform) > >> to the single *bit* telling the difference between the two internal > >> representations. > > > > We had this discussion before; if you use ob_size==0 to indicate > > that it's an int, this space isn't needed in a long int. On a 32-bit > > platform, the size of an int would go up from 12 to 16; if we stop > > using a special-cased allocator (which we should (*)), there isn't > > any space increase on such a platform. On a 64-bit platform, the > > size of an int would go up from 24 bytes to 32 bytes. > > But it's the short int that you probably really want to make size > efficient. Which is of course also doable via something like: > > typedef struct { > PyObject_HEAD > long ob_islong : 1; > long ob_ival_or_size : LONG_BITS - 1; > long ob_digit[0]; > } PyIntObject; > > There's no particular reason that a short int must be able to store > the entire range of C "long", so, as many bits can be stolen from it > as desired. There isn't? Actually a lot of APIs currently assumen that. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Wed Aug 16 01:20:39 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 16 Aug 2006 01:20:39 +0200 Subject: [Python-Dev] Type of range object members In-Reply-To: <7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net> References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net> Message-ID: <44E256C7.5060808@v.loewis.de> James Y Knight schrieb: > But it's the short int that you probably really want to make size > efficient. Only if you have many of them. And if you do, you have the problem of the special-cased allocator: when the many ints go away, Python will hold onto their memory forever. Regards, Martin From guido at python.org Wed Aug 16 01:28:09 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 15 Aug 2006 16:28:09 -0700 Subject: [Python-Dev] Type of range object members In-Reply-To: <44E256C7.5060808@v.loewis.de> References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net> <44E256C7.5060808@v.loewis.de> Message-ID: On 8/15/06, "Martin v. L?wis" wrote: > James Y Knight schrieb: > > But it's the short int that you probably really want to make size > > efficient. > > Only if you have many of them. And if you do, you have the problem > of the special-cased allocator: when the many ints go away, Python > will hold onto their memory forever. But that's a bit of a corner case. There are plenty of cases where ints are allocated and deallocated at a fast rate without allocating tons of them at once, or where there's no need to reuse the same memory for something else later. I wonder if we could have a smarter int allocator that allocates some ints in a special array but switches to the standard allocator if too many are being allocated? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From foom at fuhm.net Wed Aug 16 01:33:12 2006 From: foom at fuhm.net (James Y Knight) Date: Tue, 15 Aug 2006 19:33:12 -0400 Subject: [Python-Dev] Type of range object members In-Reply-To: References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net> Message-ID: <0EC0B3AA-422B-4D88-BA28-F3BD06E40F6F@fuhm.net> On Aug 15, 2006, at 7:06 PM, Guido van Rossum wrote: >> There's no particular reason that a short int must be able to store >> the entire range of C "long", so, as many bits can be stolen from it >> as desired. > > There isn't? Actually a lot of APIs currently assumen that. I thought we were talking about Py3k. *IF* the idea is to integrate both short/long ints into a single type, with only an internal distinction (which is what is being discussed), all those APIs are broken already. The particular internal division of the new int object between short and long doesn't matter. Which is all I was saying. If combining all integers into a single type isn't actually desired, then neither my message nor Martin's is relevant. James From nas at arctrix.com Wed Aug 16 01:33:47 2006 From: nas at arctrix.com (Neil Schemenauer) Date: Tue, 15 Aug 2006 23:33:47 +0000 (UTC) Subject: [Python-Dev] no remaining issues blocking 2.5 release References: Message-ID: Neal Norwitz wrote: > I don't know of any. I haven't heard of any issues with the fixes > that have been checked in. It would be nice if someone could bytecompile Lib using Tools/compiler/compile.py and then run the test suite. I'd do it myself but can't spare the time at the moment (I started but ran into what seems to be a gcc bug along the way). Neil From guido at python.org Wed Aug 16 01:38:31 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 15 Aug 2006 16:38:31 -0700 Subject: [Python-Dev] Type of range object members In-Reply-To: <0EC0B3AA-422B-4D88-BA28-F3BD06E40F6F@fuhm.net> References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net> <0EC0B3AA-422B-4D88-BA28-F3BD06E40F6F@fuhm.net> Message-ID: On 8/15/06, James Y Knight wrote: > On Aug 15, 2006, at 7:06 PM, Guido van Rossum wrote: > >> There's no particular reason that a short int must be able to store > >> the entire range of C "long", so, as many bits can be stolen from it > >> as desired. > > > > There isn't? Actually a lot of APIs currently assumen that. > > I thought we were talking about Py3k. *IF* the idea is to integrate > both short/long ints into a single type, with only an internal > distinction (which is what is being discussed), all those APIs are > broken already. The particular internal division of the new int > object between short and long doesn't matter. Which is all I was saying. > > If combining all integers into a single type isn't actually desired, > then neither my message nor Martin's is relevant. As I said before, at the C level I expect the distinction between int and long to be alive and well. Changing it so that ints don't have the full range of the corresponding C type would be painful IMO. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Wed Aug 16 03:40:38 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 16 Aug 2006 13:40:38 +1200 Subject: [Python-Dev] Type of range object members In-Reply-To: <44E248A7.4080107@v.loewis.de> References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> Message-ID: <44E27796.20101@canterbury.ac.nz> Martin v. L?wis wrote: > We had this discussion before; if you use ob_size==0 to indicate > that it's an int, this space isn't needed in a long int. What about int subclasses? -- Greg From greg.ewing at canterbury.ac.nz Wed Aug 16 03:42:56 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 16 Aug 2006 13:42:56 +1200 Subject: [Python-Dev] Type of range object members In-Reply-To: References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> Message-ID: <44E27820.40205@canterbury.ac.nz> Guido van Rossum wrote: > I worry that dropping the special allocator will be too slow. Surely there's some compromise that would allow recently-used ints to be kept around, but reclaimed if memory becomes low? -- Greg From anthony at interlink.com.au Wed Aug 16 03:45:41 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed, 16 Aug 2006 11:45:41 +1000 Subject: [Python-Dev] no remaining issues blocking 2.5 release In-Reply-To: <44E1F521.2010908@egenix.com> References: <44E1F521.2010908@egenix.com> Message-ID: <200608161145.44938.anthony@interlink.com.au> I really don't care any more about this. My initial concern (and why I requested the change) was that there are no more official separate distutils releases. I don't see how keeping a bunch of version numbers in the stdlib that just track the main version number is a sane use of developer time - particularly when it's only being used for someone's private releases. If you're cutting releases out of the stdlib, it seems to me that the maintenance load should be on you (where 'you' is 'anyone doing something like this', not MAL specifically ) I'd also like to see idle's separate version number go away and have it start using the Python version number - maybe as of 2.6? This would then also mean we could pull idle's separate NEWS file into the master NEWS file. Right now, the "release notes" on the webpage miss all IDLE release notes, since it's just the main NEWS file. Since it's provoked so many complaints, change it back if you want. I won't be bothering to check that it's correct from now on, nor will I be updating it. Any user complaints will be ignored, and an incorrect version number will not be considered a reason for a bugfix release. Anthony From greg.ewing at canterbury.ac.nz Wed Aug 16 03:47:37 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 16 Aug 2006 13:47:37 +1200 Subject: [Python-Dev] Type of range object members In-Reply-To: References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net> Message-ID: <44E27939.9060505@canterbury.ac.nz> Guido van Rossum wrote: > On 8/15/06, James Y Knight wrote: > >>There's no particular reason that a short int must be able to store >>the entire range of C "long", so, as many bits can be stolen from it >>as desired. > > There isn't? Actually a lot of APIs currently assumen that. Also it means you'd pay a penalty every time you access it, whereas presumably short ints are the case we want to optimise for speed as well. -- Greg From anthony at interlink.com.au Wed Aug 16 03:49:20 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed, 16 Aug 2006 11:49:20 +1000 Subject: [Python-Dev] no remaining issues blocking 2.5 release In-Reply-To: References: Message-ID: <200608161149.22013.anthony@interlink.com.au> On Tuesday 15 August 2006 22:57, Thomas Heller wrote: > What is the policy for documentation changes? Am I allowed to check in > changes/additions to the ctypes docs without release manager permission > after the release candidate is out? I'l always make sure that the html > docs can be built. So long as you don't break anything, doc changes are fine. Remember that post RC1 you'll need to checkin to both release25-maint and trunk. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From anthony at interlink.com.au Wed Aug 16 04:10:43 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed, 16 Aug 2006 12:10:43 +1000 Subject: [Python-Dev] IDLE patches - bugfix or not? In-Reply-To: References: Message-ID: <200608161210.46100.anthony@interlink.com.au> On Wednesday 16 August 2006 06:19, Jim Jewett wrote: > I just uploaded a series of IDLE patches, but I'm not quite sure how > to classify them on the feature/bugfix scale now that the last beta is > out. > > >From most to least buggish: > > python.org/sf/1540892 -- honor the new quit() behavior. On the other > hand, it was documented that this didn't work in IDLE, and it is > *possible* that someone was counting on this. This seems like a sensible thing to add. > python.org/sf/1540851 -- with is now a blockopener, and should be > counted as such -- I *think* this one would be safe, but I know that > changing a parser can be surprising, and I suppose it *could* wait > until with no longer requires a future statement. If this can be done safely, it should be done. Preferably before RC1, so that we have time to fix any issues it shows up. > python.org/sf/1540874 -- broken shortcut keys. On windows, only one > entry per menu can be reached with the same shortcut letter, so > advertising others is just an attractive nuisance. I'm not sure that > other systems wouldn't be able to use the hidden shortcuts. Tough call. I guess it depends on the other systems - I will try this on Linux at least, and see if it works there. If it's broken everywhere, then changing it would seem the least offensive choice. > python.org/sf/1540869 -- GUI fix. The current code puts in a > separator using a magic number (and has XXX comments about it.) This > changes the magic number so that the separator is more visible, but > I'm not sure the old behavior rose to a bug, or that it wasn't > platform dependent. Let's leave this one for 2.6. > python.org/sf/1540849 -- except too broad. I wouldn't suggest > applying this late in the release cycle, except that it seems sort of > like the memory errors that are still being patched. I'd be concerned that this might cause other obscure behaviour changes, and so I'd prefer to leave this to 2.6. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From anthony at interlink.com.au Wed Aug 16 04:37:04 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed, 16 Aug 2006 12:37:04 +1000 Subject: [Python-Dev] TRUNK FREEZE for 2.5c1, 00:00 UTC, Thursday 17th August Message-ID: <200608161237.06556.anthony@interlink.com.au> Ok, here we go... I'm declaring the TRUNK FROZEN as of 00:00 UTC on Thursday the 17th of August. This is in about 22 hours time. At this time, I'll be cutting the release25-maint branch and 2.5c1 (and all future 2.5 releases) will be from that branch. I'll send another email once the release is done - at that point, the trunk is then available for 2.6 checkins. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From kbk at shore.net Wed Aug 16 04:44:40 2006 From: kbk at shore.net (Kurt B. Kaiser) Date: Tue, 15 Aug 2006 22:44:40 -0400 Subject: [Python-Dev] no remaining issues blocking 2.5 release In-Reply-To: <200608161145.44938.anthony@interlink.com.au> (Anthony Baxter's message of "Wed, 16 Aug 2006 11:45:41 +1000") References: <44E1F521.2010908@egenix.com> <200608161145.44938.anthony@interlink.com.au> Message-ID: <87mza5csbr.fsf@hydra.bayview.thirdcreek.com> Anthony Baxter writes: > I'd also like to see idle's separate version number go away and have > it start using the Python version number - maybe as of 2.6? +1 When we merged IDLEfork the consensus was to keep the versioning separate. But it seems confusing and is extra work for the release managers. > This would then also mean we could pull idle's separate NEWS file into > the master NEWS file. Right now, the "release notes" on the webpage > miss all IDLE release notes, since it's just the main NEWS file. -1 Right now IDLE has a button in the "About IDLE" dialog which accesses the IDLE NEWS.txt file, and another which accesses Python's NEWS. IDLE users may not want to grovel through all the Python changes to find out what changed in the IDE. It would be nice if the key IDLE changes could make it to the "What's New in Python X.X". If Andrew is interested, I could draft something for him. -- KBK From kbk at shore.net Wed Aug 16 04:53:55 2006 From: kbk at shore.net (Kurt B. Kaiser) Date: Tue, 15 Aug 2006 22:53:55 -0400 Subject: [Python-Dev] IDLE patches - bugfix or not? In-Reply-To: <200608161210.46100.anthony@interlink.com.au> (Anthony Baxter's message of "Wed, 16 Aug 2006 12:10:43 +1000") References: <200608161210.46100.anthony@interlink.com.au> Message-ID: <87irktcrwc.fsf@hydra.bayview.thirdcreek.com> Anthony Baxter writes: > On Wednesday 16 August 2006 06:19, Jim Jewett wrote: >> I just uploaded a series of IDLE patches, but I'm not quite sure how >> to classify them on the feature/bugfix scale now that the last beta is >> out. >> >> >From most to least buggish: >> >> python.org/sf/1540892 -- honor the new quit() behavior. On the other >> hand, it was documented that this didn't work in IDLE, and it is >> *possible* that someone was counting on this. > > This seems like a sensible thing to add. > >> python.org/sf/1540851 -- with is now a blockopener, and should be >> counted as such -- I *think* this one would be safe, but I know that >> changing a parser can be surprising, and I suppose it *could* wait >> until with no longer requires a future statement. > > If this can be done safely, it should be done. Preferably before RC1, so that > we have time to fix any issues it shows up. I will look at these two. RC1 seems reasonable. >> python.org/sf/1540874 -- broken shortcut keys. On windows, only one >> entry per menu can be reached with the same shortcut letter, so >> advertising others is just an attractive nuisance. I'm not sure that >> other systems wouldn't be able to use the hidden shortcuts. > > Tough call. I guess it depends on the other systems - I will try this on Linux > at least, and see if it works there. If it's broken everywhere, then changing > it would seem the least offensive choice. I would have been inclined to make the other choice, i.e. 'p' as the hotkey for print, rather than the rarely used "save copy as". >> python.org/sf/1540869 -- GUI fix. The current code puts in a >> separator using a magic number (and has XXX comments about it.) This >> changes the magic number so that the separator is more visible, but >> I'm not sure the old behavior rose to a bug, or that it wasn't >> platform dependent. > > Let's leave this one for 2.6. > >> python.org/sf/1540849 -- except too broad. I wouldn't suggest >> applying this late in the release cycle, except that it seems sort of >> like the memory errors that are still being patched. > > I'd be concerned that this might cause other obscure behaviour changes, and so > I'd prefer to leave this to 2.6. Yes, 2.6 for these two. -- KBK From brett at python.org Tue Aug 8 02:27:45 2006 From: brett at python.org (Brett Cannon) Date: Mon, 7 Aug 2006 17:27:45 -0700 Subject: [Python-Dev] Four issue trackers submitted for Infrastructue Committee's tracker search Message-ID: Back in June, the Python Software Foundation's Infrastructure Committee asked for help in the search for a new issue tracker to replace SourceForge (see http://wiki.python.org/moin/CallForTrackers for details). We asked people who wished to help with the search to install their favourite issue tracker and import a data dump of our issues from SourceForge. We placed a deadline of August 7th to have the installation up and going. We realized that this was a large request to make of the community, but we needed to make sure that we got to evaluate any candidate trackers with the amount of issues that Python has so as to get a real-world feel. Not surprisingly, the community stepped up to the challenge with four different test tracker installations by the deadline! We have one for JIRA, Roundup, Trac, and Launchpad. The current schedule calls for the Infrastructure Committee to spend the next month evaluating the four trackers and making a decision on which one to go with. This might slip, though, since this overlaps with traditional American and European vacation time. But the slippage will not go beyond October 1. On behalf of the Infrastructure committee I thank the people who took the time and effort to set up a test tracker to help improve the development of Python. - Brett Cannon Chairman, PSF Infrastructure Committee -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060807/dd080e69/attachment.htm From mal at lemburg.com Tue Aug 8 09:22:01 2006 From: mal at lemburg.com (mal) Date: Tue, 08 Aug 2006 09:22:01 +0200 Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys In-Reply-To: <44D73964.3050402@v.loewis.de> References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com> <2m3bccwopj.fsf@starship.python.net> <44D36B36.8030202@egenix.com> <44D73964.3050402@v.loewis.de> Message-ID: <44D83B99.2070603@lemburg.com> Martin v. L?wis wrote: > M.-A. Lemburg schrieb: >> Python just doesn't know the encoding of the 8-bit string, so can't >> make any assumptions on it. As result, it raises an exception to inform >> the programmer. > > Oh, Python does make an assumption what the encoding is: it assumes > it is the system encoding (i.e. "ascii"). Then invoking the ascii > codec raises an exception, because the string clearly isn't ascii. Right, and as consequence, Python raises an exception to let the programmer correct the problem. The subsequent solution to the problem may result in the string being decoded into Unicode and the two resulting Unicode objects being unequal, or it may also result in them being equal. Python doesn't have this knowledge, so always returning false is clearly wrong. Hiding programmer errors is not making life easier in the long run, so I'm -1 on having the equality comparison return False. Instead we should generate a warning in Python 2.5 and introduce the exception in Python 2.6. >> Note that you do have to interpret the string as characters >> > if you compare it to Unicode and there's nothing wrong with >> > that. > > Consider this: > py> int(3+4j) > Traceback (most recent call last): > File "", line 1, in ? > TypeError: can't convert complex to int; use int(abs(z)) > py> 3 == 3+4j > False > > So even though the conversion raises an exception, the > values are determined to be not equal. Again, because int > is a nearly true subset of complex, the conversion goes > the other way, but *if* it would use the complex->int > conversion, then the TypeError should be taken as > a guarantee that the objects don't compare equal. In the above example, you clearly know that the two are unequal due to the relationship between complex numbers having an imaginary part and integers.. The same is true for the overflow case: >>> 2**10000 == 1.23 False >>> float(2**10000) Traceback (most recent call last): File "", line 1, in ? OverflowError: long int too large to convert to float (Note that in Python 2.3 this used to raise an exception as well.) However, this is not the case for 8-bit string vs. Unicode, since you cannot use such extra knowledge if you find that ASCII encoding assumption obviously doesn't match the string in question. > Expanding this view to Unicode should mean that a unicode > string U equals a byte string B if > U.encode(system_encode) == B or B.decode(system_encoding) == U, > and that they don't equal otherwise Agreed. Note that Python always coerces to the "bigger" type. As a result, the second option is what is actually implemented in Python. > (e.g. if the conversion > fails with a "not convertible" exception). I disagree with this part. Failure to decode a string doesn't imply inequality. It implies that the programmer needs to step in and correct the problem by making an explicit and conscious decision. The alternative would be to decide that equal comparisons should never be allowed to raise exceptions and instead have the equal comparison return False. In which case, we'd have the revert the dict patch altogether and instead silence all exceptions that are generated during the equal comparison (not only in the dict implementation), replacing them with a False return value. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 07 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From oren at hishome.net Fri Aug 4 20:26:09 2006 From: oren at hishome.net (Oren Tirosh) Date: Fri, 4 Aug 2006 21:26:09 +0300 Subject: [Python-Dev] uuid module - byte order issue In-Reply-To: References: <7168d65a0608031311g3479824ds18a9ede4a33cae7e@mail.gmail.com> Message-ID: <7168d65a0608041126y6f02f127of48530b752afef74@mail.gmail.com> On 04/08/06, Ka-Ping Yee wrote: > On Thu, 3 Aug 2006, Oren Tirosh wrote: > > The UUID module uses network byte order, regardless of the platform > > byte order. On little-endian platforms like Windows the ".bytes" > > property of UUID objects is not compatible with the memory layout > > RFC 4122 says: > > In the absence of explicit application or presentation protocol > specification to the contrary, a UUID is encoded as a 128-bit > object, as follows: > > The fields are encoded as 16 octets, with the sizes and order of > the fields defined above, and with each field encoded with the > Most Significant Byte first (known as network byte order). RFC 4122 defines a canonical byte order for UUIDs but also makes explicit reference to the fact that UUIDs are stored locally in native byte order. The final step in the RFC 4122 UUID generation algorithm is: > o Convert the resulting UUID to local byte order. So this is not another case of the Microsoft-implements-RFC-incorrectly syndrome. After all, they are one of the co-authors of the document. Compatibility with Windows "GUIDs" may be one of the most important use cases for the UUID module. It's important to resolve this or users will have unpleasant surprises. I did. alternatives: 1. Default is big endian byte order. Little endian is explicit. 2. Default is native byte order. Little endian and big endian are explicit. 3. No default. Little endian and big endian are both explicit. All three are relevant for both the constructor and retrieving the byte representation. Oren From tim.peters at gmail.com Wed Aug 16 07:22:42 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 16 Aug 2006 01:22:42 -0400 Subject: [Python-Dev] Type of range object members In-Reply-To: <44E27820.40205@canterbury.ac.nz> References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <44E27820.40205@canterbury.ac.nz> Message-ID: <1f7befae0608152222n7403360ane3ece2174f7d7cb8@mail.gmail.com> [Greg Ewing] > Surely there's some compromise that would allow > recently-used ints to be kept around, but reclaimed > if memory becomes low? Not without losing /something/ we currently enjoy. The current int scheme has theoretically optimal memory use too, consuming 12 bytes per int object on a 32-bit box. That's minimal, because the object's type pointer, refcount, and integer value each require 4 bytes on a 32-bit box. It does this by allocating "big blocks" and carving them into 12-byte slices itself. But as with any "big block" scheme, a live integer anywhere in a block prevents the entire block from being freed, and objects can't /move/ in CPython either (so, e.g., if a single live integer is keeping a big block alive, we can't move it into some other block). That's the same kind of problem obmalloc has with its "very big block" arenas, and for which only heuristic help was added (with considerable increase in complexity) for 2.5 (before 2.5, obmalloc never freed an arena, same as ints and floats never free their versions of big blocks). From martin at v.loewis.de Wed Aug 16 08:23:53 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 16 Aug 2006 08:23:53 +0200 Subject: [Python-Dev] Type of range object members In-Reply-To: <44E27796.20101@canterbury.ac.nz> References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <44E27796.20101@canterbury.ac.nz> Message-ID: <44E2B9F9.3020004@v.loewis.de> Greg Ewing schrieb: > Martin v. L?wis wrote: >> We had this discussion before; if you use ob_size==0 to indicate >> that it's an int, this space isn't needed in a long int. > > What about int subclasses? It's what Guido proposes. It would still leave two types (perhaps three) at the C level, so C code might have to continue making conditional code depending on which of these it is. Also, Python code that dispatches by type still needs to make the distinction. Regards, Martin From martin at v.loewis.de Wed Aug 16 08:25:32 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 16 Aug 2006 08:25:32 +0200 Subject: [Python-Dev] Type of range object members In-Reply-To: <44E27820.40205@canterbury.ac.nz> References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <44E27820.40205@canterbury.ac.nz> Message-ID: <44E2BA5C.7010405@v.loewis.de> Greg Ewing schrieb: > Guido van Rossum wrote: >> I worry that dropping the special allocator will be too slow. > > Surely there's some compromise that would allow > recently-used ints to be kept around, but reclaimed > if memory becomes low? Hardly. The efficiency of the special-case allocator also comes from the fact that it doesn't ever have to release memory. Just try changing it to see what I mean. Regards, Martin From martin at v.loewis.de Wed Aug 16 08:28:13 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 16 Aug 2006 08:28:13 +0200 Subject: [Python-Dev] Type of range object members In-Reply-To: <44E27939.9060505@canterbury.ac.nz> References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net> <44E27939.9060505@canterbury.ac.nz> Message-ID: <44E2BAFD.50105@v.loewis.de> Greg Ewing schrieb: >> There isn't? Actually a lot of APIs currently assumen that. > > Also it means you'd pay a penalty every time you > access it, whereas presumably short ints are the > case we want to optimise for speed as well. That penalty is already paid today. Much code dealing with ints has a type test whether it's an int or a long. If int and long become subtypes of each other or of some abstract type, performance will decrease even more because a subtype test is quite expensive if the object is neither int nor long (it has to traverse the entire base type hierarchy to find out its not inherited from int). Regards, Martin From nnorwitz at gmail.com Wed Aug 16 08:46:26 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 15 Aug 2006 23:46:26 -0700 Subject: [Python-Dev] Type of range object members In-Reply-To: <44E2BAFD.50105@v.loewis.de> References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net> <44E27939.9060505@canterbury.ac.nz> <44E2BAFD.50105@v.loewis.de> Message-ID: On 8/15/06, "Martin v. L?wis" wrote: > > That penalty is already paid today. Much code dealing with > ints has a type test whether it's an int or a long. If > int and long become subtypes of each other or of some abstract > type, performance will decrease even more because a subtype > test is quite expensive if the object is neither int nor > long (it has to traverse the entire base type hierarchy to > find out its not inherited from int). I was playing around with a little patch to avoid that penalty. It doesn't take any additional memory, just a handful of bits we aren't using. :-) For the more common builtin types, it stores whether it's a subclass in tp_flags, so there's no function call necessary and it's a constant time operation. It was faster when doing simple stuff. Haven't thought much whether this is really worthwhile or not. n -------------- next part -------------- A non-text attachment was scrubbed... Name: fast-sub.diff Type: text/x-diff Size: 6669 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20060815/4a362cc4/attachment.bin From nnorwitz at gmail.com Wed Aug 16 09:06:39 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 16 Aug 2006 00:06:39 -0700 Subject: [Python-Dev] no remaining issues blocking 2.5 release In-Reply-To: References: Message-ID: On 8/15/06, Neil Schemenauer wrote: > > It would be nice if someone could bytecompile Lib using > Tools/compiler/compile.py and then run the test suite. I'd do it > myself but can't spare the time at the moment (I started but ran > into what seems to be a gcc bug along the way). Has this been done before? # This code causes python to segfault def foo(S): all(x > 42 for x in S) # around Python/ceval.c 2167: x = (*v->ob_type->tp_iternext)(v); # tp_iternext is NULL I added the changes below to Lib/compiler/pycodegen.py which are clearly wrong. It just crashes in a diff place. I think the changes to genxpr inner may be close to correct. The changes to _makeClosure and visitGenExpr are clearly wrong. I was just wondering how far it would go. There are a bunch of differences. Some are the bytecode optimizations or different ordering, but others are things dealing with co_names, co_varnames. Hopefully someone has time to look into this. Otherwise, it will have to wait for 2.5.1 n -- Index: Lib/compiler/pycodegen.py =================================================================== --- Lib/compiler/pycodegen.py (revision 51305) +++ Lib/compiler/pycodegen.py (working copy) @@ -628,9 +628,9 @@ self.newBlock() self.emit('POP_TOP') - def _makeClosure(self, gen, args): + def _makeClosure(self, gen, args, gen_outer=False): frees = gen.scope.get_free_vars() - if frees: + if frees and not gen_outer: for name in frees: self.emit('LOAD_CLOSURE', name) self.emit('BUILD_TUPLE', len(frees)) @@ -646,7 +646,7 @@ walk(node.code, gen) gen.finish() self.set_lineno(node) - self._makeClosure(gen, 0) + self._makeClosure(gen, 0, True) # precomputation of outmost iterable self.visit(node.code.quals[0].iter) self.emit('GET_ITER') @@ -655,6 +655,11 @@ def visitGenExprInner(self, node): self.set_lineno(node) # setup list + after = self.newBlock() + start = self.newBlock() + self.setups.push((LOOP, start)) + self.emit('SETUP_LOOP', after) + self.nextBlock(start) stack = [] for i, for_ in zip(range(len(node.quals)), node.quals): @@ -676,8 +681,12 @@ self.startBlock(cont) self.emit('POP_TOP') self.nextBlock(skip_one) + self.emit('POP_TOP') self.emit('JUMP_ABSOLUTE', start) self.startBlock(anchor) + self.emit('POP_BLOCK') + self.setups.pop() + self.nextBlock(after) self.emit('LOAD_CONST', None) def visitGenExprFor(self, node): From kbk at shore.net Wed Aug 16 09:16:58 2006 From: kbk at shore.net (Kurt B. Kaiser) Date: Wed, 16 Aug 2006 03:16:58 -0400 (EDT) Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200608160716.k7G7GwcP015299@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 404 open ( +2) / 3376 closed (+16) / 3780 total (+18) Bugs : 860 open ( -1) / 6131 closed (+17) / 6991 total (+16) RFE : 229 open ( +1) / 235 closed ( +1) / 464 total ( +2) New / Reopened Patches ______________________ option to leave tempfile.NamedTemporaryFile around on close (2006-08-10) http://python.org/sf/1537850 opened by djmdjm Patch to fix __index__() clipping (2006-08-11) CLOSED http://python.org/sf/1538606 opened by Travis Oliphant Patch cElementTree to export CurrentLineNumber (2006-08-11) http://python.org/sf/1538691 opened by Robin Bryce Replace unicode.__eq__ exceptions with a warning (2006-08-11) CLOSED http://python.org/sf/1538956 opened by M.-A. Lemburg Add readinto method to StringIO and cStringIO (2006-08-12) http://python.org/sf/1539381 opened by Alexander Belopolsky Backports from trunk to release24-maint (2006-08-15) http://python.org/sf/1540329 opened by flub tarfile __slots__ addition (2006-08-14) http://python.org/sf/1540385 opened by Brian Harring Patches for OpenBSD 4.0 (2006-08-15) http://python.org/sf/1540470 opened by djmdjm Use Py_ssize_t for rangeobject members (2006-08-15) http://python.org/sf/1540617 opened by Alexander Belopolsky except too broad (2006-08-15) http://python.org/sf/1540849 opened by Jim Jewett with is now a blockopener (2006-08-15) CLOSED http://python.org/sf/1540851 opened by Jim Jewett IOBinding overly broad except (2nd try) (2006-08-15) CLOSED http://python.org/sf/1540856 opened by Jim Jewett except too broad (2006-08-15) CLOSED http://python.org/sf/1540857 opened by Jim Jewett IOBinding broad except - try 3 (2006-08-15) CLOSED http://python.org/sf/1540859 opened by Jim Jewett CodeContext visibility (2006-08-15) http://python.org/sf/1540869 opened by Jim Jewett broken shortcut keys (2006-08-15) http://python.org/sf/1540874 opened by Jim Jewett make IDLE honor the quit() builtin (2006-08-15) CLOSED http://python.org/sf/1540892 opened by Jim Jewett Patches Closed ______________ Add notes on locale module changes to whatsnew25.tex (2006-08-03) http://python.org/sf/1534027 closed by akuchling Build ctypes on OpenBSD x86_64 (2006-08-08) http://python.org/sf/1536908 closed by theller Fix __index__() clipping really big numbers (2006-07-29) http://python.org/sf/1530738 closed by ncoghlan PyShell.recall - fix indentation logic (2006-07-25) http://python.org/sf/1528468 closed by kbk Patch to fix __index__() clipping (2006-08-11) http://python.org/sf/1538606 closed by nnorwitz Replace unicode.__eq__ exceptions with a warning (2006-08-11) http://python.org/sf/1538956 closed by lemburg Replace the ctypes internal '_as_parameter_' mechanism (2006-08-02) http://python.org/sf/1532975 closed by theller writelines() in bz2 module does not raise check for errors (2006-08-06) http://python.org/sf/1535500 closed by gbrandl trace.py on win32 has problems with lowercase drive names (2006-08-07) http://python.org/sf/1536071 closed by gbrandl Give Cookie.py its own _idmap (2006-07-13) http://python.org/sf/1521882 closed by gbrandl socket.gethostbyaddr fix for machines with incomplete DNS (2006-06-23) http://python.org/sf/1511317 closed by gbrandl with is now a blockopener (2006-08-15) http://python.org/sf/1540851 closed by kbk IOBinding overly broad except (2nd try) (2006-08-15) http://python.org/sf/1540856 closed by jimjjewett except too broad (2006-08-15) http://python.org/sf/1540857 closed by jimjjewett IOBinding broad except - try 3 (2006-08-15) http://python.org/sf/1540859 closed by jimjjewett make IDLE honor the quit() builtin (2006-08-15) http://python.org/sf/1540892 closed by kbk New / Reopened Bugs ___________________ 2nd issue with need for speed patch (2006-08-09) http://python.org/sf/1537167 opened by Robin Bryce Missing platform information in subprocess documentation (2006-08-09) CLOSED http://python.org/sf/1537195 opened by Aaron Bingham urllib2 httplib _read_chunked timeout (2006-08-09) http://python.org/sf/1537445 opened by devloop Installation on Windows Longhorn (2006-08-10) http://python.org/sf/1537601 opened by O.R.Senthil Kumaran import on cElementTree on Windows (2006-08-09) http://python.org/sf/1537685 reopened by thomasbhickey import on cElementTree on Windows (2006-08-09) http://python.org/sf/1537685 opened by Thomas B Hickey indent changes when copying command (2006-08-10) CLOSED http://python.org/sf/1538445 opened by mjd__ PyThreadState_SetAsyncExc bug (2006-08-11) http://python.org/sf/1538556 opened by ganges master pyo's are not overwritten by different optimization levels (2006-08-11) http://python.org/sf/1538778 opened by Toshio Kuratomi Long command lines don't work on Windows (2006-08-12) http://python.org/sf/1539295 opened by Albert Strasheim distutils example code missing imports (2006-08-13) CLOSED http://python.org/sf/1539336 opened by Albert Strasheim Identifiers begining with __ renamed (2006-08-14) CLOSED http://python.org/sf/1539847 opened by W Barnes warnings in interactive sessions (2006-08-14) http://python.org/sf/1539925 opened by M.-A. Lemburg Closing a pipe can lead to uncompleted command (2006-08-14) http://python.org/sf/1539954 opened by Stefan Sonnenberg Insane format string should cause ValueError (2006-08-14) CLOSED http://python.org/sf/1539955 opened by Nick Coghlan SocketServer.ForkingMixIn.collect_children() waits on pid 0 (2006-08-14) http://python.org/sf/1540386 opened by Neal Norwitz cgi.py error on parsing/handling content-disposition (2006-08-15) http://python.org/sf/1540529 opened by Dirk Holtwick Bugs Closed ___________ PyArg_ParseTupleAndKeywords potential core dump (2006-07-17) http://python.org/sf/1523610 closed by gbrandl Missing platform information in subprocess documentation (2006-08-09) http://python.org/sf/1537195 closed by gbrandl Concatenation on a long string breaks (2006-07-21) http://python.org/sf/1526585 closed by arigo import on cElementTree on Windows (2006-08-09) http://python.org/sf/1537685 closed by gbrandl cgi.FieldStorage memory usage can spike in line-oriented ops (2005-01-30) http://python.org/sf/1112549 closed by gvanrossum tokenize module does not detect inconsistent dedents (2005-06-21) http://python.org/sf/1224621 closed by gbrandl fix inplace assignment for immutable sequences (2006-02-21) http://python.org/sf/1436226 closed by gbrandl BROWSER path with capital letters (2006-07-27) http://python.org/sf/1529655 closed by sf-robot struct.pack raises TypeError where it used to convert (2006-07-28) http://python.org/sf/1530559 closed by nnorwitz indent changes when copying command (2006-08-11) http://python.org/sf/1538445 closed by kbk distutils example code missing imports (2006-08-12) http://python.org/sf/1539336 closed by nnorwitz Segmentation fault when importing expat from xml.parser (2005-07-27) http://python.org/sf/1246405 closed by sf-robot CSV regression in 2.5a1: multi-line cells (2006-04-05) http://python.org/sf/1465014 closed by sf-robot Installing documentation doesn't make it show up (2003-07-06) http://python.org/sf/766842 closed by sf-robot infinite __str__ recursion in thread causes seg fault (2003-07-31) http://python.org/sf/780714 closed by sf-robot Identifiers begining with __ renamed (2006-08-14) http://python.org/sf/1539847 closed by gbrandl Insane format string should cause ValueError (2006-08-14) http://python.org/sf/1539955 closed by gbrandl CTypes _as_parameter_ not working as documented (2006-08-03) http://python.org/sf/1533481 closed by theller PythonLauncher uses incorrect working directory (2006-07-23) http://python.org/sf/1527397 closed by sf-robot Quitter object masked (2006-05-01) http://python.org/sf/1479785 closed by kbk New / Reopened RFE __________________ csv module: add header row to DictWriter (2006-08-10) http://python.org/sf/1537721 opened by ed_abraham Allow choice of copy function in shutil.copytree (2006-08-14) http://python.org/sf/1540112 opened by Tony From thomas at python.org Wed Aug 16 13:37:47 2006 From: thomas at python.org (Thomas Wouters) Date: Wed, 16 Aug 2006 13:37:47 +0200 Subject: [Python-Dev] Making 'python -t' the default. Message-ID: <9e804ac0608160437h619af70ck8952850fa6330ae8@mail.gmail.com> Can I suggest making 'python -t' the default, in 2.6? It makes python warn about mixing tabs and spaces in indentation. In Py3k, '-tt' (the error-raising version) will be the default, instead. I see too many newsbies with problems they can't figure out because they mix tabs and spaces (usually not even consciously, but just because their editor isn't playing along.) There probably should be an option to turn the warning off again (although I won't fight for it ;) -- but I don't know what that option should be. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060816/8e9559f9/attachment.html From g.brandl at gmx.net Wed Aug 16 14:19:55 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 16 Aug 2006 14:19:55 +0200 Subject: [Python-Dev] Making 'python -t' the default. In-Reply-To: <9e804ac0608160437h619af70ck8952850fa6330ae8@mail.gmail.com> References: <9e804ac0608160437h619af70ck8952850fa6330ae8@mail.gmail.com> Message-ID: Thomas Wouters wrote: > > Can I suggest making 'python -t' the default, in 2.6? It makes python > warn about mixing tabs and spaces in indentation. +1. > In Py3k, '-tt' (the > error-raising version) will be the default, instead. Or disallow tabs altogether. > I see too many > newsbies with problems they can't figure out because they mix tabs and > spaces (usually not even consciously, but just because their editor > isn't playing along.) There probably should be an option to turn the > warning off again (although I won't fight for it ;) -- but I don't know > what that option should be. Perhaps -T. Georg From amk at amk.ca Wed Aug 16 14:43:09 2006 From: amk at amk.ca (A.M. Kuchling) Date: Wed, 16 Aug 2006 08:43:09 -0400 Subject: [Python-Dev] no remaining issues blocking 2.5 release In-Reply-To: <87mza5csbr.fsf@hydra.bayview.thirdcreek.com> References: <44E1F521.2010908@egenix.com> <200608161145.44938.anthony@interlink.com.au> <87mza5csbr.fsf@hydra.bayview.thirdcreek.com> Message-ID: <20060816124309.GA4841@localhost.localdomain> On Tue, Aug 15, 2006 at 10:44:40PM -0400, Kurt B. Kaiser wrote: > It would be nice if the key IDLE changes could make it to the "What's New > in Python X.X". If Andrew is interested, I could draft something for him. Sure! I can try to look through the IDLE NEWS file, but you'd certainly have a better idea of which changes are most significant to IDLE users. (You can commit your changes to whatsnew25.tex, module Anthony's trunk freeze in 12 hours or so, or just write plain text and let me add the LaTeX markup.) --amk From barry at python.org Wed Aug 16 14:48:17 2006 From: barry at python.org (Barry Warsaw) Date: Wed, 16 Aug 2006 08:48:17 -0400 Subject: [Python-Dev] Making 'python -t' the default. In-Reply-To: <9e804ac0608160437h619af70ck8952850fa6330ae8@mail.gmail.com> References: <9e804ac0608160437h619af70ck8952850fa6330ae8@mail.gmail.com> Message-ID: <580957E2-1635-44AB-97EF-9649008E241D@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 16, 2006, at 7:37 AM, Thomas Wouters wrote: > > Can I suggest making 'python -t' the default, in 2.6? It makes > python warn about mixing tabs and spaces in indentation. In Py3k, '- > tt' (the error-raising version) will be the default, +1. We even have Subversion hooks in our repository to reject any commits that have tabs in them (except for Makefiles and a few whitelisted 3rd-party apps that we don't want to reformat). - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBROMUEXEjvBPtnXfVAQJSuQQAlU/FqiDhVUXKiIIaH91s0iyyd8/NgHMn fuA5fO06vcpAWxDvZtpmF8xAqGbN9AUh87iISDEEYUdpe1xrs1hly2QEJ5xIBnmB rF+3FfDZI+YKCkCHzHhG01jJBH6j/4pW0ZxUs4DAKz5os0o1xor7Yiy1sI/t1cVR Ra0m8ByDssE= =D4hn -----END PGP SIGNATURE----- From skip at pobox.com Wed Aug 16 16:02:27 2006 From: skip at pobox.com (skip at pobox.com) Date: Wed, 16 Aug 2006 09:02:27 -0500 Subject: [Python-Dev] Type of range object members In-Reply-To: <44E2BA5C.7010405@v.loewis.de> References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <44E27820.40205@canterbury.ac.nz> <44E2BA5C.7010405@v.loewis.de> Message-ID: <17635.9587.937810.163966@montanaro.dyndns.org> Guido> I worry that dropping the special allocator will be too slow. Greg> Surely there's some compromise that would allow recently-used ints Greg> to be kept around, but reclaimed if memory becomes low? Martin> Hardly. The efficiency of the special-case allocator also comes Martin> from the fact that it doesn't ever have to release memory. Just Martin> try changing it to see what I mean. Wouldn't use of obmalloc offset much of that? Before obmalloc was available, the int free list was a huge win. Is it likely to be such a huge win today? Skip From guido at python.org Wed Aug 16 16:51:16 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 16 Aug 2006 07:51:16 -0700 Subject: [Python-Dev] Type of range object members In-Reply-To: <44E2B9F9.3020004@v.loewis.de> References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <44E27796.20101@canterbury.ac.nz> <44E2B9F9.3020004@v.loewis.de> Message-ID: On 8/15/06, "Martin v. L?wis" wrote: > Greg Ewing schrieb: > > Martin v. L?wis wrote: > >> We had this discussion before; if you use ob_size==0 to indicate > >> that it's an int, this space isn't needed in a long int. > > > > What about int subclasses? > > It's what Guido proposes. > > It would still leave two types (perhaps three) at the C level, > so C code might have to continue making conditional code depending > on which of these it is. Also, Python code that dispatches by type > still needs to make the distinction. I'm not sure that subclassing ints gives us much. We could make int and long "final" types, and then all we have to do is tweak type() and __class__ so that they always return the 'int' type. Alternatively, yes, there would be some minimal awareness of the two types in Python -- but nothing like we currently have; dispatching on exact type (which we discourage anyway) would be the only case. Would that be so bad? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From g.brandl at gmx.net Wed Aug 16 17:25:20 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 16 Aug 2006 17:25:20 +0200 Subject: [Python-Dev] Type of range object members In-Reply-To: References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net> <44E27939.9060505@canterbury.ac.nz> <44E2BAFD.50105@v.loewis.de> Message-ID: Neal Norwitz wrote: > On 8/15/06, "Martin v. L?wis" wrote: >> >> That penalty is already paid today. Much code dealing with >> ints has a type test whether it's an int or a long. If >> int and long become subtypes of each other or of some abstract >> type, performance will decrease even more because a subtype >> test is quite expensive if the object is neither int nor >> long (it has to traverse the entire base type hierarchy to >> find out its not inherited from int). > > I was playing around with a little patch to avoid that penalty. It > doesn't take any additional memory, just a handful of bits we aren't > using. :-) > > For the more common builtin types, it stores whether it's a subclass > in tp_flags, so there's no function call necessary and it's a constant > time operation. It was faster when doing simple stuff. Haven't > thought much whether this is really worthwhile or not. This might als be helpful when exceptions have to inherit from BaseException in Py3k. Georg From martin at v.loewis.de Wed Aug 16 17:29:37 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 16 Aug 2006 17:29:37 +0200 Subject: [Python-Dev] Type of range object members In-Reply-To: <17635.9587.937810.163966@montanaro.dyndns.org> References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <44E27820.40205@canterbury.ac.nz> <44E2BA5C.7010405@v.loewis.de> <17635.9587.937810.163966@montanaro.dyndns.org> Message-ID: <44E339E1.8030202@v.loewis.de> skip at pobox.com schrieb: > Guido> I worry that dropping the special allocator will be too slow. > > Greg> Surely there's some compromise that would allow recently-used ints > Greg> to be kept around, but reclaimed if memory becomes low? > > Martin> Hardly. The efficiency of the special-case allocator also comes > Martin> from the fact that it doesn't ever have to release memory. Just > Martin> try changing it to see what I mean. > > Wouldn't use of obmalloc offset much of that? Before obmalloc was > available, the int free list was a huge win. Is it likely to be such a huge > win today? That's my theory: it isn't a huge win. Guido has another theory: it's still faster. Only benchmarking can tell. Regards, Martin From martin at v.loewis.de Wed Aug 16 17:34:48 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 16 Aug 2006 17:34:48 +0200 Subject: [Python-Dev] Type of range object members In-Reply-To: References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <44E27796.20101@canterbury.ac.nz> <44E2B9F9.3020004@v.loewis.de> Message-ID: <44E33B18.9040403@v.loewis.de> Guido van Rossum schrieb: > I'm not sure that subclassing ints gives us much. We could make int > and long "final" types, and then all we have to do is tweak type() and > __class__ so that they always return the 'int' type. I don't think this can work - there would be too many ways for the "real" types to leak, anyway. People would come up with hacks like reload(sys), and then complain that they have to use such hacks. > Alternatively, yes, there would be some minimal awareness of the two > types in Python -- but nothing like we currently have; dispatching on > exact type (which we discourage anyway) would be the only case. Would > that be so bad? I thought it was the ultimate goal of PEP 237 to unify int and long, in all respects. I'll do some benchmarking. Regards, Martin From dinov at exchange.microsoft.com Wed Aug 16 17:31:43 2006 From: dinov at exchange.microsoft.com (Dino Viehland) Date: Wed, 16 Aug 2006 08:31:43 -0700 Subject: [Python-Dev] 2.4 & 2.5 beta 3 crash Message-ID: <7AD436E4270DD54A94238001769C2227372749F65E@DF-GRTDANE-MSG.exchange.corp.microsoft.com> We've been working on fixing some exception handling bugs in IronPython where we differ from CPython. Along the way we ran into this issue which causes CPython to crash when the code below is run. It crashes on both 2.4 and 2.5 beta 3. The code's technically illegal, but it probably shouldn't crash either :) def test(): for abc in range(10): try: pass finally: try: continue except: pass test() From martin at v.loewis.de Wed Aug 16 18:08:13 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 16 Aug 2006 18:08:13 +0200 Subject: [Python-Dev] Benchmarking the int allocator (Was: Type of range object members) In-Reply-To: <17635.9587.937810.163966@montanaro.dyndns.org> References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <44E27820.40205@canterbury.ac.nz> <44E2BA5C.7010405@v.loewis.de> <17635.9587.937810.163966@montanaro.dyndns.org> Message-ID: <44E342ED.6060507@v.loewis.de> skip at pobox.com schrieb: > Wouldn't use of obmalloc offset much of that? Before obmalloc was > available, the int free list was a huge win. Is it likely to be such a huge > win today? I have now some numbers. For the attached t.py, the unmodified svn python gives Test 1 3.25420880318 Test 2 1.86433696747 and the one with the attached patch gives Test 1 3.45080399513 Test 2 2.09729003906 So there apparently is a performance drop on int allocations of about 5-10%. On this machine (P4 3.2GHz) I could not find any difference in pystones from this patch. Notice that this test case is extremely focused on measuring int allocation (I just noticed I should have omitted the for loop in the second case, though). Regards, Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: t.py Type: text/x-python Size: 181 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20060816/239c9e63/attachment.py -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: intalloc.diff Url: http://mail.python.org/pipermail/python-dev/attachments/20060816/239c9e63/attachment.diff From jimjjewett at gmail.com Wed Aug 16 18:39:47 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed, 16 Aug 2006 12:39:47 -0400 Subject: [Python-Dev] IDLE patches - bugfix or not? In-Reply-To: <200608161210.46100.anthony@interlink.com.au> References: <200608161210.46100.anthony@interlink.com.au> Message-ID: > > python.org/sf/1540874 -- broken shortcut keys. On windows, only one > > entry per menu can be reached with the same shortcut letter, so > > advertising others is just an attractive nuisance. I'm not sure that > > other systems wouldn't be able to use the hidden shortcuts. On 8/15/06, Anthony Baxter wrote: > Tough call. I guess it depends on the other systems - I will try this on Linux > at least, and see if it works there. If it's broken everywhere, then changing > it would seem the least offensive choice. Thank you. Kurt Kaiser: > I would have been inclined to make the other choice, i.e. 'p' as the > hotkey for print, rather than the rarely used "save copy as". The existing functionality (at least on windows) is that p brings up a Path Browser window; print and save copy are *both* masked. I agree that p *should* be for print, but I didn't want to remove an existing (working) shortcut this late. So I just stopped advertising the (unusable) shortcut on print. I did add the (currently unused) 'y' for "save copy as", which I suppose might be considered a feature. Simply removing the claimed but broken shortcut would indeed fix the attractive nuisance problem. -jJ From nas at arctrix.com Wed Aug 16 18:59:31 2006 From: nas at arctrix.com (Neil Schemenauer) Date: Wed, 16 Aug 2006 16:59:31 +0000 (UTC) Subject: [Python-Dev] no remaining issues blocking 2.5 release References: Message-ID: Neal Norwitz wrote: > On 8/15/06, Neil Schemenauer wrote: >> >> It would be nice if someone could bytecompile Lib using >> Tools/compiler/compile.py and then run the test suite. > > Has this been done before? Obviously not. :-) > # This code causes python to segfault > def foo(S): > all(x > 42 for x in S) Hmm, it seems to work for me. The bytecode generated by Lib/compiler is the same as the normal compiler. Do you have a full test case? Neil From guido at python.org Wed Aug 16 19:08:00 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 16 Aug 2006 10:08:00 -0700 Subject: [Python-Dev] Benchmarking the int allocator (Was: Type of range object members) In-Reply-To: <44E342ED.6060507@v.loewis.de> References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <44E27820.40205@canterbury.ac.nz> <44E2BA5C.7010405@v.loewis.de> <17635.9587.937810.163966@montanaro.dyndns.org> <44E342ED.6060507@v.loewis.de> Message-ID: On 8/16/06, "Martin v. L?wis" wrote: > I have now some numbers. For the attached t.py, the unmodified svn > python gives > > Test 1 3.25420880318 > Test 2 1.86433696747 > > and the one with the attached patch gives > > Test 1 3.45080399513 > Test 2 2.09729003906 > > So there apparently is a performance drop on int allocations of about > 5-10%. > > On this machine (P4 3.2GHz) I could not find any difference in pystones > from this patch. > > Notice that this test case is extremely focused on measuring int > allocation (I just noticed I should have omitted the for loop in > the second case, though). I think the test isn't hardly focused enough on int allocation. I wonder if you could come up with a benchmark that repeatedly allocates 100s of 1000s of ints and then deletes them? What if it also allocates other small objects so the ints become more fragmented? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jcarlson at uci.edu Wed Aug 16 19:11:28 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Wed, 16 Aug 2006 10:11:28 -0700 Subject: [Python-Dev] 2.4 & 2.5 beta 3 crash In-Reply-To: <7AD436E4270DD54A94238001769C2227372749F65E@DF-GRTDANE-MSG.exchange.corp.microsoft.com> References: <7AD436E4270DD54A94238001769C2227372749F65E@DF-GRTDANE-MSG.exchange.corp.microsoft.com> Message-ID: <20060816100610.19DD.JCARLSON@uci.edu> Dino Viehland wrote: > > We've been working on fixing some exception handling bugs in > IronPython where we differ from CPython. Along the way we ran into > this issue which causes CPython to crash when the code below is run. > It crashes on both 2.4 and 2.5 beta 3. The code's technically illegal, > but it probably shouldn't crash either :) > def test(): > for abc in range(10): > try: pass > finally: > try: > continue > except: > pass > > > test() It also reliably crashes 2.3, though I don't see anything in there as being illegal, but maybe I don't understand the langauge as well as I think I do. Note that there is another segfaulting bug in CPython with regards to threads that was recently closed, but which is still a problem: http://python.org/sf/780714 Does IronPython survive in that case? - Josiah From g.brandl at gmx.net Wed Aug 16 19:15:54 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 16 Aug 2006 19:15:54 +0200 Subject: [Python-Dev] 2.4 & 2.5 beta 3 crash In-Reply-To: <20060816100610.19DD.JCARLSON@uci.edu> References: <7AD436E4270DD54A94238001769C2227372749F65E@DF-GRTDANE-MSG.exchange.corp.microsoft.com> <20060816100610.19DD.JCARLSON@uci.edu> Message-ID: Josiah Carlson wrote: > Dino Viehland wrote: >> >> We've been working on fixing some exception handling bugs in >> IronPython where we differ from CPython. Along the way we ran into >> this issue which causes CPython to crash when the code below is run. >> It crashes on both 2.4 and 2.5 beta 3. The code's technically illegal, >> but it probably shouldn't crash either :) > >> def test(): >> for abc in range(10): >> try: pass >> finally: >> try: >> continue >> except: >> pass >> >> >> test() > > It also reliably crashes 2.3, though I don't see anything in there as > being illegal, but maybe I don't understand the langauge as well as I > think I do. "continue" is not supported inside a finally clause. If you put the continue directly there or into the except clause, the compiler will tell you. It looks like nobody thought about that corner case when writing the compiler check for continue. Georg From guido at python.org Wed Aug 16 19:23:43 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 16 Aug 2006 10:23:43 -0700 Subject: [Python-Dev] Type of range object members In-Reply-To: References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net> <44E27939.9060505@canterbury.ac.nz> <44E2BAFD.50105@v.loewis.de> Message-ID: On 8/15/06, Neal Norwitz wrote: > I was playing around with a little patch to avoid that penalty. It > doesn't take any additional memory, just a handful of bits we aren't > using. :-) > > For the more common builtin types, it stores whether it's a subclass > in tp_flags, so there's no function call necessary and it's a constant > time operation. It was faster when doing simple stuff. Haven't > thought much whether this is really worthwhile or not. I like it! I wonder if you should use another bit for "inherits from BaseException". That would make catching and raising exceptions a bit faster. It applies cleanly to py3k -- perhaps you should just check it in there? +1 from me! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Wed Aug 16 20:29:30 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 16 Aug 2006 14:29:30 -0400 Subject: [Python-Dev] Type of range object members In-Reply-To: References: <44E2BAFD.50105@v.loewis.de> <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net> <44E27939.9060505@canterbury.ac.nz> <44E2BAFD.50105@v.loewis.de> Message-ID: <5.1.1.6.0.20060816142136.022968e8@sparrow.telecommunity.com> At 11:46 PM 8/15/2006 -0700, Neal Norwitz wrote: >On 8/15/06, "Martin v. L?wis" wrote: >> >>That penalty is already paid today. Much code dealing with >>ints has a type test whether it's an int or a long. If >>int and long become subtypes of each other or of some abstract >>type, performance will decrease even more because a subtype >>test is quite expensive if the object is neither int nor >>long (it has to traverse the entire base type hierarchy to >>find out its not inherited from int). > >I was playing around with a little patch to avoid that penalty. It >doesn't take any additional memory, just a handful of bits we aren't >using. :-) > >For the more common builtin types, it stores whether it's a subclass >in tp_flags, so there's no function call necessary and it's a constant >time operation. It was faster when doing simple stuff. Haven't >thought much whether this is really worthwhile or not. It seems to me that you could drop the FAST_SUBCLASS bit, since none of the other bits will be set if it is not a subclass of a builtin. That would free up one flag bit -- perhaps usable for that BaseException flag Guido wants. :) (Of course, if you can't inherit from both BaseException and one of the other builtin types, it can just be another enumeration value within the bit mask.) From martin at v.loewis.de Wed Aug 16 21:52:13 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 16 Aug 2006 21:52:13 +0200 Subject: [Python-Dev] Benchmarking the int allocator (Was: Type of range object members) In-Reply-To: References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <44E27820.40205@canterbury.ac.nz> <44E2BA5C.7010405@v.loewis.de> <17635.9587.937810.163966@montanaro.dyndns.org> <44E342ED.6060507@v.loewis.de> Message-ID: <44E3776D.90804@v.loewis.de> Guido van Rossum schrieb: > I think the test isn't hardly focused enough on int allocation. I > wonder if you could come up with a benchmark that repeatedly allocates > 100s of 1000s of ints and then deletes them? The question is: where to store them? In a pre-allocated list, or in a growing list? def t3(): m = [0]*100000 s = time.time() i = 0 for i in xrange(100): for k in xrange(100000): m[k] = k print "Test 3",time.time()-s def t4(): s = time.time() i = 0 for i in xrange(100): m = [] for k in xrange(100000): m.append(k) print "Test 4",time.time()-s This is 100s of 1000s of ints in the inner loop; test 3 puts them into a pre-allocated list, test 4 discards the list each time and lets it grow. > What if it also allocates > other small objects so the ints become more fragmented? This allocator doesn't bother much about fragmentation: it's constant-time most of the time on allocation, and often on deallocation (especially when the memory is fragmented). Also, it's hard to find an object that is as small as an int; I think a one-element tuple applies: def t5(): s = time.time() i = 0 for i in xrange(100): m = [] for k in xrange(100000): m.append((k,)) print "Test 5",time.time()-s The timings, for the best of three runs: Py2.5 +obmalloc-for-int slowdown Test 3 1.8s 2.1s 15% Test 4 3.6s 3.8s 5% test 5 7.5s 7.5s 0 Regards, Martin From allison at shasta.stanford.edu Wed Aug 16 19:22:00 2006 From: allison at shasta.stanford.edu (Dennis Allison) Date: Wed, 16 Aug 2006 10:22:00 -0700 (PDT) Subject: [Python-Dev] 2.4 & 2.5 beta 3 crash In-Reply-To: <20060816100610.19DD.JCARLSON@uci.edu> Message-ID: def test(): for abc in range(10): try: pass finally: try: pass except: pass test() does not raise a segmentation fault. On Wed, 16 Aug 2006, Josiah Carlson wrote: > > Dino Viehland wrote: > > > > We've been working on fixing some exception handling bugs in > > IronPython where we differ from CPython. Along the way we ran into > > this issue which causes CPython to crash when the code below is run. > > It crashes on both 2.4 and 2.5 beta 3. The code's technically illegal, > > but it probably shouldn't crash either :) > > > def test(): > > for abc in range(10): > > try: pass > > finally: > > try: > > continue > > except: > > pass > > > > > > test() > > It also reliably crashes 2.3, though I don't see anything in there as > being illegal, but maybe I don't understand the langauge as well as I > think I do. > > Note that there is another segfaulting bug in CPython with regards to > threads that was recently closed, but which is still a problem: > http://python.org/sf/780714 > > Does IronPython survive in that case? > > > - Josiah > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/allison%40shasta.stanford.edu > -- From martin at v.loewis.de Wed Aug 16 22:21:54 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 16 Aug 2006 22:21:54 +0200 Subject: [Python-Dev] Type of range object members In-Reply-To: References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net> <44E27939.9060505@canterbury.ac.nz> <44E2BAFD.50105@v.loewis.de> Message-ID: <44E37E62.4060501@v.loewis.de> Neal Norwitz schrieb: > I was playing around with a little patch to avoid that penalty. It > doesn't take any additional memory, just a handful of bits we aren't > using. :-) There are common schemes that allow constant-time issubclass tests, although they do require more memory: 1. give each base class a small unique number, starting with 1 (0 means no number has been assigned). Give each class a bitmap of all base classes, plus a field for the maximum-numbered base class. Then, def issubclass(C, B): return B.classnum and (B.classnum < C.maxbasenum) and\ bit_set(C.basenums, B.classnum) Supports multiple inheritance, space requirement linear with the number of classes that are used as base classes. Numbering should recycle class numbers when a class is gced. 2. restrict optimization to single-inheritance. Give each class a "depth" (distance from object, 0 for object and multiply-inherited classes). Also give each class an array of bases, ordered by depth. Then, def issubclass(C, B): if not C.depth: return expensive_issubclass(C, B) return B.depth < C.depth and (C.bases[B.depth] is B) Space requirement is linear with the depth of the class (but I think tp_mro could be used, if the formula is changed to (C.bases[C.depth-B.depth] is B)) Regards, Martin From dinov at exchange.microsoft.com Wed Aug 16 19:46:46 2006 From: dinov at exchange.microsoft.com (Dino Viehland) Date: Wed, 16 Aug 2006 10:46:46 -0700 Subject: [Python-Dev] 2.4 & 2.5 beta 3 crash In-Reply-To: References: <20060816100610.19DD.JCARLSON@uci.edu> Message-ID: <7AD436E4270DD54A94238001769C2227372749F734@DF-GRTDANE-MSG.exchange.corp.microsoft.com> Yeah, continue inside the finally block is illegal. If you don't have the extra try/except nesting then continue is detected as a syntax error. -----Original Message----- From: Dennis Allison [mailto:allison at shasta.stanford.edu] Sent: Wednesday, August 16, 2006 10:22 AM To: Josiah Carlson Cc: Dino Viehland; python-dev at python.org Subject: Re: [Python-Dev] 2.4 & 2.5 beta 3 crash def test(): for abc in range(10): try: pass finally: try: pass except: pass test() does not raise a segmentation fault. On Wed, 16 Aug 2006, Josiah Carlson wrote: > > Dino Viehland wrote: > > > > We've been working on fixing some exception handling bugs in > > IronPython where we differ from CPython. Along the way we ran into > > this issue which causes CPython to crash when the code below is run. > > It crashes on both 2.4 and 2.5 beta 3. The code's technically > > illegal, but it probably shouldn't crash either :) > > > def test(): > > for abc in range(10): > > try: pass > > finally: > > try: > > continue > > except: > > pass > > > > > > test() > > It also reliably crashes 2.3, though I don't see anything in there as > being illegal, but maybe I don't understand the langauge as well as I > think I do. > > Note that there is another segfaulting bug in CPython with regards to > threads that was recently closed, but which is still a problem: > http://python.org/sf/780714 > > Does IronPython survive in that case? > > > - Josiah > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/allison%40shasta.sta > nford.edu > -- From dinov at exchange.microsoft.com Wed Aug 16 19:49:22 2006 From: dinov at exchange.microsoft.com (Dino Viehland) Date: Wed, 16 Aug 2006 10:49:22 -0700 Subject: [Python-Dev] 2.4 & 2.5 beta 3 crash In-Reply-To: <20060816100610.19DD.JCARLSON@uci.edu> References: <7AD436E4270DD54A94238001769C2227372749F65E@DF-GRTDANE-MSG.exchange.corp.microsoft.com> <20060816100610.19DD.JCARLSON@uci.edu> Message-ID: <7AD436E4270DD54A94238001769C2227372749F736@DF-GRTDANE-MSG.exchange.corp.microsoft.com> IronPython actually enables / disables stack overflow checking through either a command line option or by calling sys.setrecursionlimit. By default we have no recursion limit. Without those command line options we'll end up having the CLR throw an unrecoverable stack overflow exception (and the process will be terminated). It's a little bit better than a seg fault, but not much. With stack overflow checking enabled we'll actually catch the recursion for this and throw a RuntimeError stating maximum recursion depth reached. -----Original Message----- From: Josiah Carlson [mailto:jcarlson at uci.edu] Sent: Wednesday, August 16, 2006 10:11 AM To: Dino Viehland; python-dev at python.org Subject: Re: [Python-Dev] 2.4 & 2.5 beta 3 crash Dino Viehland wrote: > > We've been working on fixing some exception handling bugs in > IronPython where we differ from CPython. Along the way we ran into > this issue which causes CPython to crash when the code below is run. > It crashes on both 2.4 and 2.5 beta 3. The code's technically > illegal, but it probably shouldn't crash either :) > def test(): > for abc in range(10): > try: pass > finally: > try: > continue > except: > pass > > > test() It also reliably crashes 2.3, though I don't see anything in there as being illegal, but maybe I don't understand the langauge as well as I think I do. Note that there is another segfaulting bug in CPython with regards to threads that was recently closed, but which is still a problem: http://python.org/sf/780714 Does IronPython survive in that case? - Josiah From mal at egenix.com Thu Aug 17 00:01:23 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 17 Aug 2006 00:01:23 +0200 Subject: [Python-Dev] Type of range object members In-Reply-To: <44E37E62.4060501@v.loewis.de> References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net> <44E27939.9060505@canterbury.ac.nz> <44E2BAFD.50105@v.loewis.de> <44E37E62.4060501@v.loewis.de> Message-ID: <44E395B3.6000805@egenix.com> Martin v. L?wis wrote: > Neal Norwitz schrieb: >> I was playing around with a little patch to avoid that penalty. It >> doesn't take any additional memory, just a handful of bits we aren't >> using. :-) > > There are common schemes that allow constant-time issubclass tests, > although they do require more memory: > > 1. give each base class a small unique number, starting with 1 > (0 means no number has been assigned). Give each class a bitmap > of all base classes, plus a field for the maximum-numbered > base class. Then, > > def issubclass(C, B): > return B.classnum and (B.classnum < C.maxbasenum) and\ > bit_set(C.basenums, B.classnum) > > Supports multiple inheritance, space requirement linear > with the number of classes that are used as base classes. > Numbering should recycle class numbers when a class is > gced. > > 2. restrict optimization to single-inheritance. Give each class > a "depth" (distance from object, 0 for object and > multiply-inherited classes). Also give each class an array > of bases, ordered by depth. Then, > > def issubclass(C, B): > if not C.depth: return expensive_issubclass(C, B) > return B.depth < C.depth and (C.bases[B.depth] is B) > > Space requirement is linear with the depth of the class > (but I think tp_mro could be used, if the formula is > changed to (C.bases[C.depth-B.depth] is B)) Two more: 3. Use a global cache of class objects that caches the issubclass() lookups. This is only amortized constant time, but easy to implement and has a few other nice features such as e.g. enabling traversal of the complete class inheritance forest (or tree if you just have new-style classes). Use weak references to the classes if you want to be able to GC them. 4. "Freeze" classes after they are constructed, meaning that all attributes from base-classes get bound to the inheriting class. This also speeds up method lookups considerably. Works great with classic classes, I'm not sure about new-style classes using e.g. staticmethods, slots and the like. mxTools has an implementation for classic classes called freeze(). If you add special attributes such as ._issubclass_XYZ in the process, issubclass() would then be a single attribute lookup which is constant time. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 16 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From kbk at shore.net Thu Aug 17 01:27:47 2006 From: kbk at shore.net (Kurt B. Kaiser) Date: Wed, 16 Aug 2006 19:27:47 -0400 Subject: [Python-Dev] no remaining issues blocking 2.5 release In-Reply-To: <20060816124309.GA4841@localhost.localdomain> (A. M. Kuchling's message of "Wed, 16 Aug 2006 08:43:09 -0400") References: <44E1F521.2010908@egenix.com> <200608161145.44938.anthony@interlink.com.au> <87mza5csbr.fsf@hydra.bayview.thirdcreek.com> <20060816124309.GA4841@localhost.localdomain> Message-ID: <87bqqkb6rw.fsf@hydra.bayview.thirdcreek.com> "A.M. Kuchling" writes: > On Tue, Aug 15, 2006 at 10:44:40PM -0400, Kurt B. Kaiser wrote: >> It would be nice if the key IDLE changes could make it to the "What's New >> in Python X.X". If Andrew is interested, I could draft something for him. > > Sure! I can try to look through the IDLE NEWS file, but you'd > certainly have a better idea of which changes are most significant to > IDLE users. > > (You can commit your changes to whatsnew25.tex, module Anthony's trunk > freeze in 12 hours or so, or just write plain text and let me add the > LaTeX markup.) %================================================================= \subsection{The IDLE Integrated Development Environment} Calltips have been greatly improved and a class attribute listbox provides completions. When the Shell cursor is on a previous command, retrieves that command. But instead of replacing the input line, the previous command is appended, preserving indentation. If there is an active selection, the selection will be appended to the input line. IDLE does a tabnanny and syntax check before every F5/Run. A number of changes were made to improve Mac OSX compatibility. The advanced keybinding dialog was enabled, allowing multiple complex key combinations to be bound to a given IDLE event. A number of improvements were made to the colorizer. The 'with' statement is now a Code Context block opener. IDLE honors the new quit() and exit() functions. Keybindings were added for del-word-left and del-word-right. %============================================================ -- KBK From greg.ewing at canterbury.ac.nz Thu Aug 17 03:25:46 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 17 Aug 2006 13:25:46 +1200 Subject: [Python-Dev] Type of range object members In-Reply-To: <44E2BAFD.50105@v.loewis.de> References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net> <44E27939.9060505@canterbury.ac.nz> <44E2BAFD.50105@v.loewis.de> Message-ID: <44E3C59A.1020208@canterbury.ac.nz> Martin v. L?wis wrote: > Greg Ewing schrieb: >>Also it means you'd pay a penalty every time you >>access it > > That penalty is already paid today. You'd still have that penalty, *plus* the overhead of bit masking to get at the value. Maybe that wouldn't be noticeable among all the other overheads, though. -- Greg From greg.ewing at canterbury.ac.nz Thu Aug 17 03:30:17 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 17 Aug 2006 13:30:17 +1200 Subject: [Python-Dev] Making 'python -t' the default. In-Reply-To: References: <9e804ac0608160437h619af70ck8952850fa6330ae8@mail.gmail.com> Message-ID: <44E3C6A9.3000200@canterbury.ac.nz> Georg Brandl wrote: > Or disallow tabs altogether. -1. I'd be annoyed if Python started telling me I wasn't allowed to write my source the way my preferred editor (BBEdit) works best. Very annoyed. -- Greg From jjl at pobox.com Thu Aug 17 03:58:22 2006 From: jjl at pobox.com (John J Lee) Date: Thu, 17 Aug 2006 01:58:22 +0000 (UTC) Subject: [Python-Dev] 2.5: recently introduced sgmllib regexp bug hangs Python Message-ID: Looks like revision 47154 introduced a regexp that hangs Python (Ctrl-C won't kill the process, CPU usage sits near 100%) under some circumstances. There's a test case here: http://python.org/sf/1541697 The problem isn't seen if you read the whole file at once (or almost the whole file at once). (But that doesn't make it a non-bug, AFAICS.) I'm not sure what the problem is, but presumably the relevant part of the patch is this: +starttag = re.compile(r'<[a-zA-Z][-_.:a-zA-Z0-9]*\s*(' + r'\s*([a-zA-Z_][-:.a-zA-Z_0-9]*)(\s*=\s*' + r'(\'[^\']*\'|"[^"]*"|[-a-zA-Z0-9./,:;+*%?!&$\(\)_#=~@]' + r'[][\-a-zA-Z0-9./,:;+*%?!&$\(\)_#=~\'"@]*(?=[\s>/<])))?' + r')*\s*/?\s*(?=[<>])') The patch attached to bug 1515142 (also from Sam Ruby -- claims to fix a regression introduced by his recent sgmllib patches, and has not yet been applied) does NOT fix the problem. If nobody has time to fix this, perhaps rev 47154 should be reverted? commit message for -r47154: """ SF bug #1504333: sgmlib should allow angle brackets in quoted values (modified patch by Sam Ruby; changed to use separate REs for start and end tags to reduce matching cost for end tags; extended tests; updated to avoid breaking previous changes to support IPv6 addresses in unquoted attribute values) """ John From jjl at pobox.com Thu Aug 17 04:13:06 2006 From: jjl at pobox.com (John J Lee) Date: Thu, 17 Aug 2006 02:13:06 +0000 (UTC) Subject: [Python-Dev] [wwwsearch-general] 2.5: recently introduced sgmllib regexp bug hangs Python In-Reply-To: References: Message-ID: On Thu, 17 Aug 2006, John J Lee wrote: [...] > If nobody has time to fix this, perhaps rev 47154 should be reverted? I should have put it more strongly: I think it *should* in fact be reverted if nobody has time to fix it before the release candidate / final release of 2.5. The revision in question is the most recent commit to sgmllib.py and certainly appears completely localised to that module. And a hung interpreter is worse than failing to parse some HTML, IMHO. [...] > commit message for -r47154: > > """ > SF bug #1504333: sgmlib should allow angle brackets in quoted values > (modified patch by Sam Ruby; changed to use separate REs for start and end > tags to reduce matching cost for end tags; extended tests; updated to > avoid > breaking previous changes to support IPv6 addresses in unquoted attribute > values) > """ [...] From nnorwitz at gmail.com Thu Aug 17 07:26:09 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 16 Aug 2006 22:26:09 -0700 Subject: [Python-Dev] 2.4 & 2.5 beta 3 crash In-Reply-To: <7AD436E4270DD54A94238001769C2227372749F65E@DF-GRTDANE-MSG.exchange.corp.microsoft.com> References: <7AD436E4270DD54A94238001769C2227372749F65E@DF-GRTDANE-MSG.exchange.corp.microsoft.com> Message-ID: Thanks Dino. The attached patch should fix the problem. Once RC1 is cut, I'll check this in unless someone beats me to it. Since the compiler changed, I can't backport this. If someone wants to make a similar fix for 2.4 go for it. n -- On 8/16/06, Dino Viehland wrote: > We've been working on fixing some exception handling bugs in IronPython where we differ from CPython. Along the way we ran into this issue which causes CPython to crash when the code below is run. It crashes on both 2.4 and 2.5 beta 3. The code's technically illegal, but it probably shouldn't crash either :) > > def test(): > for abc in range(10): > try: pass > finally: > try: > continue > except: > pass > > > test() > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com > -------------- next part -------------- A non-text attachment was scrubbed... Name: finally-cont.diff Type: text/x-patch Size: 1658 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20060816/a9670b69/attachment.bin From foom at fuhm.net Thu Aug 17 07:49:04 2006 From: foom at fuhm.net (James Y Knight) Date: Thu, 17 Aug 2006 01:49:04 -0400 Subject: [Python-Dev] 2.4 & 2.5 beta 3 crash In-Reply-To: References: <7AD436E4270DD54A94238001769C2227372749F65E@DF-GRTDANE-MSG.exchange.corp.microsoft.com> Message-ID: On Aug 17, 2006, at 1:26 AM, Neal Norwitz wrote: > Thanks Dino. > > The attached patch should fix the problem. Once RC1 is cut, I'll > check this in unless someone beats me to it. Since the compiler > changed, I can't backport this. If someone wants to make a similar > fix for 2.4 go for it. The attached patch is incorrect. It breaks the following perfectly valid code. Probably what would be right is to move (with appropriate changes) the while loop in the FINALLY_TRY case to outside the entire switch statement. Thus, if LOOP is reached, yay, add the jump, return success. If the FINALLY_END is reached first, boo, fail. If it's a FINALLY_TRY or EXCEPT, continue the loop. def test(): try: pass finally: for abc in range(10): continue James From nnorwitz at gmail.com Thu Aug 17 07:57:54 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 16 Aug 2006 22:57:54 -0700 Subject: [Python-Dev] Type of range object members In-Reply-To: <5.1.1.6.0.20060816142136.022968e8@sparrow.telecommunity.com> References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net> <44E27939.9060505@canterbury.ac.nz> <44E2BAFD.50105@v.loewis.de> <5.1.1.6.0.20060816142136.022968e8@sparrow.telecommunity.com> Message-ID: On 8/16/06, Phillip J. Eby wrote: > > It seems to me that you could drop the FAST_SUBCLASS bit, since none of the > other bits will be set if it is not a subclass of a builtin. That would > free up one flag bit -- perhaps usable for that BaseException flag Guido > wants. :) :-) Right, I'm not using the bit currently. I was thinking that it would be interesting to change the CheckExact versions to also use this. It's a little more work, but you lose the second comparison for Check. I expect that it would be slower, but I was just curious. So with the patch we currently have: #define PyInt_CheckExact(op) ((op)->ob_type == &PyInt_Type) #define PyInt_Check(op) (PyInt_CheckExact(op) || \ PyType_FastSubclass((op)->ob_type, Py_TPFLAGS_INT_SUBCLASS)) But we could have something like: #define PyInt_CheckExact(op) (PyType_FastClass(op,Py_TPFLAGS_INT_CLASS)) #define PyInt_Check(op) (PyType_FastSubclass(op,Py_TPFLAGS_INT_SUBCLASS)) It would change the CheckExact()s from: op->ob_type == global-variable, to: op->ob_type & CONSTANT == CONSTANT. Check would be the same as the CheckExact, just with different constants. The Check version would then drop the || condition. I might play with this at the sprint next week. It does seem to make sense to do BaseException too. It will take 4 or 5 bits to handle the current ones plus BaseException, which we can easily spare in tp_flags. n From martin at v.loewis.de Thu Aug 17 08:01:58 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 17 Aug 2006 08:01:58 +0200 Subject: [Python-Dev] Type of range object members In-Reply-To: <44E3C59A.1020208@canterbury.ac.nz> References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net> <44E27939.9060505@canterbury.ac.nz> <44E2BAFD.50105@v.loewis.de> <44E3C59A.1020208@canterbury.ac.nz> Message-ID: <44E40656.9050501@v.loewis.de> Greg Ewing schrieb: >>> Also it means you'd pay a penalty every time you >>> access it >> That penalty is already paid today. > > You'd still have that penalty, *plus* the > overhead of bit masking to get at the value. No, the penalty gets smaller if there is only a single type. For example, abstract.c now has if (res && (!PyInt_Check(res) && !PyLong_Check(res))) { PyErr_Format(PyExc_TypeError, "__int__ returned non-int (type %.200s)", res->ob_type->tp_name); Py_DECREF(res); return NULL; } Currently, if a long int is returned, it performs two subtype tests. If the long type is dropped, the second test can go away. In this specific code, there is no penalty for a representation flag, since the value is not accessed. Code that wants to support both int and long and needs the value often does PyLong_AsLong these days, which will support int as well. This currently reads if (vv == NULL || !PyLong_Check(vv)) { if (vv != NULL && PyInt_Check(vv)) return PyInt_AsLong(vv); PyErr_BadInternalCall(); return -1; } Notice that this has two checks if this is an int, and both are subtype checks. With a single type, this would become if (vv == NULL || !PyInt_Check(vv)) { PyErr_BadInternalCall(); return -1; } if (!vv->ob_size) return PyInt_AsLong(vv); Actually, the implementation of PyInt_AsLong might get inlined; it currently starts with a third PyInt_Check. So overall, I would expect that a single type would improve performance, not decrease it. As you say, any change is likely not noticeable in performance, though, either way. Regards, Martin From martin at v.loewis.de Thu Aug 17 08:27:02 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 17 Aug 2006 08:27:02 +0200 Subject: [Python-Dev] Type of range object members In-Reply-To: References: <44E1EFEF.20305@v.loewis.de> <44E248A7.4080107@v.loewis.de> <7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net> <44E27939.9060505@canterbury.ac.nz> <44E2BAFD.50105@v.loewis.de> <5.1.1.6.0.20060816142136.022968e8@sparrow.telecommunity.com> Message-ID: <44E40C36.4000603@v.loewis.de> Neal Norwitz schrieb: > It would change the CheckExact()s from: op->ob_type == > global-variable, to: op->ob_type & CONSTANT == CONSTANT. Check would > be the same as the CheckExact, just with different constants. The > Check version would then drop the || condition. Hmm. I don't see the for the FAST_SUBCLASS bit still. I would set the relevant bit in the type object itself, and then have #define PyInt_CheckExact(op) ((op)->ob_type == &PyInt_Type) #define PyInt_Check(op) \ PyType_FastSubclass((op)->ob_type, Py_TPFLAGS_INT_SUBCLASS) Then, in inherit_special, I'd do type->tp_flags |= base->tp_flags & Py_TPFLAGS_FAST_SUBCLASS_MASK; So you would have a pointer comparison for the exact check, and the bit mask check for the subtype check. It's likely that the pointer comparison is still more efficient: It does *not*, normally, need to read a global variable to get the address of PyInt_Type. Currently, on x86, with non-PIC code on Linux, the pointer check compiles as cmpl $PyInt_Type, 4(%eax) ; %eax is the object where the linker fills the address of PyInt_Type into the machine instruction. OTOH, the access to the flags compiles as movl 4(%eax), %eax ; %eax is the object movl 84(%eax), %eax andl $2013265920, %eax cmpl $2013265920, %eax Even with PIC code, the address check is still more efficient: movl PyInt_Type at GOT(%ecx), %eax cmpl %eax, 4(%edx) ; %edx is the object Regards, Martin From thomas at python.org Thu Aug 17 13:22:53 2006 From: thomas at python.org (Thomas Wouters) Date: Thu, 17 Aug 2006 13:22:53 +0200 Subject: [Python-Dev] Benchmarking the int allocator (Was: Type of range object members) In-Reply-To: <44E3776D.90804@v.loewis.de> References: <44E248A7.4080107@v.loewis.de> <44E27820.40205@canterbury.ac.nz> <44E2BA5C.7010405@v.loewis.de> <17635.9587.937810.163966@montanaro.dyndns.org> <44E342ED.6060507@v.loewis.de> <44E3776D.90804@v.loewis.de> Message-ID: <9e804ac0608170422w5a7fbbfelce61e5ba74c85971@mail.gmail.com> On 8/16/06, "Martin v. L?wis" wrote: > > Guido van Rossum schrieb: > > I think the test isn't hardly focused enough on int allocation. I > > wonder if you could come up with a benchmark that repeatedly allocates > > 100s of 1000s of ints and then deletes them? > > The question is: where to store them? In a pre-allocated list, or in a > growing list? Or you can expose Py_INCREF to Python code. I'm thinking about that for big-mem tests anyway ;-) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060817/ada243d2/attachment.htm From fredrik at pythonware.com Thu Aug 17 14:35:33 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 17 Aug 2006 14:35:33 +0200 Subject: [Python-Dev] recently introduced sgmllib regexp bughangs Python References: Message-ID: John J Lee wrote: > If nobody has time to fix this, perhaps rev 47154 should be reverted? yes please. From anthony at interlink.com.au Thu Aug 17 14:48:47 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu, 17 Aug 2006 22:48:47 +1000 Subject: [Python-Dev] [wwwsearch-general] 2.5: recently introduced sgmllib regexp bug hangs Python In-Reply-To: References: Message-ID: <200608172248.49046.anthony@interlink.com.au> On Thursday 17 August 2006 12:13, John J Lee wrote: > On Thu, 17 Aug 2006, John J Lee wrote: > [...] > > > If nobody has time to fix this, perhaps rev 47154 should be reverted? > > I should have put it more strongly: I think it *should* in fact be > reverted if nobody has time to fix it before the release candidate / final > release of 2.5. The revision in question is the most recent commit to > sgmllib.py and certainly appears completely localised to that module. > And a hung interpreter is worse than failing to parse some HTML, IMHO. Well, it's a bit late for the release candidate :-) but I agree - if it can't be fixed before 2.5 final, it should be reverted. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From benji at zope.com Thu Aug 17 15:50:54 2006 From: benji at zope.com (Benji York) Date: Thu, 17 Aug 2006 09:50:54 -0400 Subject: [Python-Dev] [wwwsearch-general] 2.5: recently introduced sgmllib regexp bug hangs Python In-Reply-To: References: Message-ID: <44E4743E.40807@zope.com> John J Lee wrote: > Looks like revision 47154 introduced a regexp that hangs Python Revision 47155 was a similar change to the 2.4 branch. -- Benji York Senior Software Engineer Zope Corporation From anthony at python.org Thu Aug 17 16:17:38 2006 From: anthony at python.org (Anthony Baxter) Date: Fri, 18 Aug 2006 00:17:38 +1000 Subject: [Python-Dev] RELEASED Python 2.5 (release candidate 1) Message-ID: <200608180017.52057.anthony@python.org> On behalf of the Python development team and the Python community, I'm happy to announce the first RELEASE CANDIDATE of Python 2.5. This is not yet the final release - it is not suitable for production use. It is being released to solicit feedback and hopefully expose bugs, as well as allowing you to determine how changes in 2.5 might impact you. As a release candidate, this is one of your last chances to test the new code in 2.5 before the final release. *Please* try this release out and let us know about any problems you find. In particular, note that changes to improve Python's support of 64 bit systems might require authors of C extensions to change their code. More information (as well as source distributions and Windows and Universal Mac OSX installers) are available from the 2.5 website: http://www.python.org/2.5/ As of this release, Python 2.5 is now in *feature freeze*. Unless absolutely necessary, no functionality changes will be made between now and the final release of Python 2.5. The plan now is to let the release candidate shake out any last-minute bugs in Python 2.5, leading to a 2.5 final release in early September. PEP 356 includes the schedule and will be updated as the schedule evolves. At this point, any testing you can do would be greatly, greatly appreciated. The new features in Python 2.5 are described in Andrew Kuchling's What's New In Python 2.5. It's available from the 2.5 web page. Amongst the language features added include conditional expressions, the with statement, the merge of try/except and try/finally into try/except/finally, enhancements to generators to produce a coroutine kind of functionality, and a brand new AST-based compiler implementation. New modules added include hashlib, ElementTree, sqlite3, wsgiref, uuid and ctypes. In addition, a new profiling module cProfile was added. Enjoy this new release, Anthony Anthony Baxter anthony at python.org Python Release Manager (on behalf of the entire python-dev team) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 191 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20060818/a331c29d/attachment.pgp From anthony at interlink.com.au Thu Aug 17 16:23:13 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri, 18 Aug 2006 00:23:13 +1000 Subject: [Python-Dev] TRUNK IS UNFROZEN, available for 2.6 work if you are so inclined Message-ID: <200608180023.14037.anthony@interlink.com.au> The release candidate is done - we creep ever closer to 2.5 final. Hoooray! All future 2.5 releases (including 2.5 final!) will now be done from the new release25-maint trunk (svn+ssh://pythondev at svn.python.org/python/branches/release25-maint) - so any changes you want to see after 2.5c1 and before 2.5 final will need to be applied to that branch as well as to the trunk. From now until the final release, I think we should say no checkins to the release25-maint branch without either myself or Neal signing off on it - for safety's sake, I'd recommend you email either the list here, or if you have to send private email, send it to both of us. If this policy offends you, please reply and let me know what you'd prefer. Right now, I don't really care about the trunk - although if you break the buildbot, I'm sure Neal will be very cranky at you :-) Anthony -- Anthony Baxter It's never too late to have a happy childhood. From martin at v.loewis.de Thu Aug 17 16:47:45 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 17 Aug 2006 16:47:45 +0200 Subject: [Python-Dev] [Python-checkins] TRUNK IS UNFROZEN, available for 2.6 work if you are so inclined In-Reply-To: <200608180023.14037.anthony@interlink.com.au> References: <200608180023.14037.anthony@interlink.com.au> Message-ID: <44E48191.6020008@v.loewis.de> Anthony Baxter schrieb: > Right now, I don't really care about the trunk - although if you break the > buildbot [...] Thanks for reminding me. I just added a buildbot builder for the 2.5 branch (actually, the builder was already there, but I connected it with the web server), so you can now see the status of the 2.5 branch at http://www.python.org/dev/buildbot/2.5/ Regards, Martin From anthony at interlink.com.au Thu Aug 17 16:50:51 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri, 18 Aug 2006 00:50:51 +1000 Subject: [Python-Dev] [Python-checkins] TRUNK IS UNFROZEN, available for 2.6 work if you are so inclined In-Reply-To: <44E48191.6020008@v.loewis.de> References: <200608180023.14037.anthony@interlink.com.au> <44E48191.6020008@v.loewis.de> Message-ID: <200608180050.55389.anthony@interlink.com.au> On Friday 18 August 2006 00:47, Martin v. L?wis wrote: > Anthony Baxter schrieb: > > Right now, I don't really care about the trunk - although if you break > > the buildbot [...] > > Thanks for reminding me. I just added a buildbot builder for the 2.5 > branch (actually, the builder was already there, but I connected it > with the web server), so you can now see the status of the 2.5 branch > at > > http://www.python.org/dev/buildbot/2.5/ You beat me to it by a couple of minutes - I'd added the entry to the master.cfg file but hadn't updated the apache config yet. Anthoiny -- Anthony Baxter It's never too late to have a happy childhood. From martin at v.loewis.de Thu Aug 17 19:03:05 2006 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 17 Aug 2006 19:03:05 +0200 Subject: [Python-Dev] Fixing 2.5 windows buildbots Message-ID: <44E4A149.7060409@v.loewis.de> I'd like to fix the two build failures that the Windows buildbots show for the 2.5 trunk. I'm not quite sure what the OpenSSL failure is, yet, but the sqlite error should be fixable with the patch below. Ok to work on this? Martin Index: Tools/buildbot/external.bat =================================================================== --- Tools/buildbot/external.bat (Revision 51339) +++ Tools/buildbot/external.bat (Arbeitskopie) @@ -28,6 +28,7 @@ cd tk8.4.12\win nmake -f makefile.vc TCLDIR=..\..\tcl8.4.12 nmake -f makefile.vc TCLDIR=..\..\tcl8.4.12 INSTALLDIR=..\..\tcltk install + cd ..\.. ) @rem sqlite From nnorwitz at gmail.com Thu Aug 17 19:12:46 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 17 Aug 2006 10:12:46 -0700 Subject: [Python-Dev] Fixing 2.5 windows buildbots In-Reply-To: <44E4A149.7060409@v.loewis.de> References: <44E4A149.7060409@v.loewis.de> Message-ID: On 8/17/06, "Martin v. L?wis" wrote: > I'd like to fix the two build failures that the Windows buildbots > show for the 2.5 trunk. I'm not quite sure what the OpenSSL failure > is, yet, but the sqlite error should be fixable with the patch > below. Ok to work on this? Please do. From martin at v.loewis.de Thu Aug 17 21:52:54 2006 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 17 Aug 2006 21:52:54 +0200 Subject: [Python-Dev] uuid tests failing on Windows Message-ID: <44E4C916.9040008@v.loewis.de> test_uuid1 and test_uuid4 fail consistently on Windows; apparently, the generated universally-unique identifiers aren't even unique within a set of 1000 identifiers. Can somebody please fix that? If not, should we remove the uuid module as being immature? Regards, Martin From g.brandl at gmx.net Thu Aug 17 21:56:49 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 17 Aug 2006 21:56:49 +0200 Subject: [Python-Dev] uuid tests failing on Windows In-Reply-To: <44E4C916.9040008@v.loewis.de> References: <44E4C916.9040008@v.loewis.de> Message-ID: Martin v. L?wis wrote: > test_uuid1 and test_uuid4 fail consistently on Windows; apparently, > the generated universally-unique identifiers aren't even unique within > a set of 1000 identifiers. > > Can somebody please fix that? If not, should we remove the uuid module > as being immature? Patch #1541863 supposedly solves this. Georg From jackdied at jackdied.com Thu Aug 17 22:00:46 2006 From: jackdied at jackdied.com (Jack Diederich) Date: Thu, 17 Aug 2006 16:00:46 -0400 Subject: [Python-Dev] [Python-checkins] TRUNK IS UNFROZEN, available for 2.6 work if you are so inclined In-Reply-To: References: <200608180023.14037.anthony@interlink.com.au> <20060817185049.GH5772@performancedrivers.com> Message-ID: <20060817200046.GI5772@performancedrivers.com> On Thu, Aug 17, 2006 at 09:07:53PM +0200, Georg Brandl wrote: > Jack Diederich wrote: > > > Looks good to me. While you are on that page do you want to change > > > > l = PyList_New(3); > > x = PyInt_FromLong(1L); > > PySequence_SetItem(l, 0, x); Py_DECREF(x); > > x = PyInt_FromLong(2L); > > PySequence_SetItem(l, 1, x); Py_DECREF(x); > > x = PyString_FromString("three"); > > PySequence_SetItem(l, 2, x); Py_DECREF(x); > > > > to > > > > l = PyList_New(3); > > x = PyInt_FromLong(1L); > > PyList_SetItem(l, 0, x); > > x = PyInt_FromLong(2L); > > PyList_SetItem(l, 1, x); > > x = PyString_FromString("three"); > > PyList_SetItem(l, 2, x); > > > > The example code causes segfaults and probably always has (at last to 2.2) > > Interesting! From a naive POV, the docs' example is quite right. > > The segfault occurs because list_ass_item Py_DECREFs the old item (which > segfaults because the old items are NULL in a newly created list). > PyList_SetItem does a Py_XDECREF. > > The docs to PyList_New, however, do not explicitly say that the new list > must only be filled using PyList_SetItem. > > So please, someone, decide what's broken here! The docs, this is from a thread yesterday and today on c.l.py http://groups.google.com/group/comp.lang.python/browse_frm/thread/158c8797ee2dccab/ -Jack From martin at v.loewis.de Thu Aug 17 22:06:24 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 17 Aug 2006 22:06:24 +0200 Subject: [Python-Dev] uuid tests failing on Windows In-Reply-To: References: <44E4C916.9040008@v.loewis.de> Message-ID: <44E4CC40.7030105@v.loewis.de> Georg Brandl schrieb: >> Can somebody please fix that? If not, should we remove the uuid module >> as being immature? > > Patch #1541863 supposedly solves this. Ah, good. I think it should go in. Regards, Martin From glyph at divmod.com Thu Aug 17 22:53:13 2006 From: glyph at divmod.com (glyph at divmod.com) Date: Thu, 17 Aug 2006 16:53:13 -0400 Subject: [Python-Dev] uuid tests failing on Windows In-Reply-To: <44E4CC40.7030105@v.loewis.de> Message-ID: <20060817205313.1717.1697957210.divmod.quotient.24160@ohm> On Thu, 17 Aug 2006 22:06:24 +0200, "\"Martin v. L?wis\"" wrote: >Georg Brandl schrieb: >>> Can somebody please fix that? If not, should we remove the uuid module >>> as being immature? >> >> Patch #1541863 supposedly solves this. > >Ah, good. I think it should go in. Uh, I may be misunderstanding here, but that patch looks like it changes that part of the test for test_uuid4 to stop calling uuid4 and call uuid1 instead? From fredrik at pythonware.com Thu Aug 17 23:03:08 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 17 Aug 2006 23:03:08 +0200 Subject: [Python-Dev] TRUNK IS UNFROZEN, available for 2.6 work if you are so inclined In-Reply-To: References: <200608180023.14037.anthony@interlink.com.au> <20060817185049.GH5772@performancedrivers.com> Message-ID: Georg Brandl wrote: >> The example code causes segfaults and probably always has (at last to 2.2) > > Interesting! From a naive POV, the docs' example is quite right. except that it doesn't work? the problem is that it's using a *high-level* API to manipulate objects that are not properly initialized. the documentation should be fixed (doctest for the C API, anyone?) From fredrik at pythonware.com Thu Aug 17 23:05:00 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 17 Aug 2006 23:05:00 +0200 Subject: [Python-Dev] TRUNK IS UNFROZEN, available for 2.6 work if you are so inclined In-Reply-To: References: <200608180023.14037.anthony@interlink.com.au> <20060817185049.GH5772@performancedrivers.com> Message-ID: Georg Brandl wrote: > Okay, now that I stumbled over and read the c.l.py thread, I think that we should > > * remove the faulty example (the correct one is already in there) > * add a note to PyList_New that the new list must be filled with items > before handing it to Python code or using it with abstract APIs such as > PySequence_SetItem. a blurb at the beginning of the abstract API section that mentions that the abstract API should only be used on fully initialized Python objects might also be a good idea. From nnorwitz at gmail.com Thu Aug 17 23:15:47 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 17 Aug 2006 14:15:47 -0700 Subject: [Python-Dev] Sprints next week at Google Message-ID: This is a reminder (I don't think anyone else sent one, I sure hope not). We are holding 4 days of sprints next week at Google offices in NY city and Mt View, CA. These are open if you'd like to attend. It would be very helpful to pre-register on the wiki as we can notify security and generally make things easier. Of the full-time Googlers, Guido and Alex will be sprinting in CA and Jeremy and I will be sprinting in NY. http://wiki.python.org/moin/GoogleSprint Cheers, n From g.brandl at gmx.net Thu Aug 17 23:30:59 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 17 Aug 2006 23:30:59 +0200 Subject: [Python-Dev] TRUNK IS UNFROZEN, available for 2.6 work if you are so inclined In-Reply-To: References: <200608180023.14037.anthony@interlink.com.au> <20060817185049.GH5772@performancedrivers.com> Message-ID: Fredrik Lundh wrote: > Georg Brandl wrote: > >>> The example code causes segfaults and probably always has (at last to 2.2) >> >> Interesting! From a naive POV, the docs' example is quite right. > > except that it doesn't work? I wanted to say: From looking at the docs only, it should be alright. > the problem is that it's using a *high-level* API to manipulate objects > that are not properly initialized. The docs for PyList_New don't tell the user that the list object isn't properly initialized, etc. > the documentation should be fixed Clearly. I'll do that (if the release managers give permission) Georg From martin at v.loewis.de Thu Aug 17 23:58:27 2006 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Thu, 17 Aug 2006 23:58:27 +0200 Subject: [Python-Dev] uuid tests failing on Windows In-Reply-To: <20060817205313.1717.1697957210.divmod.quotient.24160@ohm> References: <20060817205313.1717.1697957210.divmod.quotient.24160@ohm> Message-ID: <44E4E683.5090209@v.loewis.de> glyph at divmod.com schrieb: >>> Patch #1541863 supposedly solves this. >> Ah, good. I think it should go in. > > Uh, I may be misunderstanding here, but that patch looks like it > changes that part of the test for test_uuid4 to stop calling uuid4 > and call uuid1 instead? You misunderstand indeed: the chunk reads - for u in [uuid.uuid1() for i in range(1000)]: + for u in [uuid.uuid4() for i in range(1000)]: so it currently calls uuid1, and will call uuid4 when patched. test_uuid4 should have never failed, except that it uses uuid1 as the uniqueness test. Regards, Martin From glyph at divmod.com Fri Aug 18 00:22:40 2006 From: glyph at divmod.com (glyph at divmod.com) Date: Thu, 17 Aug 2006 18:22:40 -0400 Subject: [Python-Dev] uuid tests failing on Windows In-Reply-To: <44E4E683.5090209@v.loewis.de> Message-ID: <20060817222240.1717.1350793518.divmod.quotient.24211@ohm> On Thu, 17 Aug 2006 23:58:27 +0200, "\"Martin v. L?wis\"" wrote: >You misunderstand indeed: the chunk reads (...) >it currently calls uuid1, and will call uuid4 when patched. >test_uuid4 should have never failed, except that it uses uuid1 >as the uniqueness test. Whooops. I must have hit the "reverse diff" button in Emacs before reading it. Thanks for the correction. From arigo at tunes.org Fri Aug 18 00:26:33 2006 From: arigo at tunes.org (Armin Rigo) Date: Fri, 18 Aug 2006 00:26:33 +0200 Subject: [Python-Dev] 2.4 & 2.5 beta 3 crash In-Reply-To: References: <7AD436E4270DD54A94238001769C2227372749F65E@DF-GRTDANE-MSG.exchange.corp.microsoft.com> Message-ID: <20060817222633.GA26304@code0.codespeak.net> Hi Neal, On Wed, Aug 16, 2006 at 10:26:09PM -0700, Neal Norwitz wrote: > @@ -2367,7 +2374,9 @@ > compiler_use_next_block(c, end); > if (!compiler_push_fblock(c, FINALLY_END, end)) > return 0; > + c->c_in_finally = 1; > VISIT_SEQ(c, stmt, s->v.TryFinally.finalbody); > + c->c_in_finally = 0; > ADDOP(c, END_FINALLY); > compiler_pop_fblock(c, FINALLY_END, end); Without more inspection, I'd say that this looks like it won't do the right thing about nested finally's, as in: ... finally: try: ... finally: ... continue A bientot, Armin From guido at python.org Fri Aug 18 00:44:58 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 17 Aug 2006 15:44:58 -0700 Subject: [Python-Dev] 2.4 & 2.5 beta 3 crash In-Reply-To: <20060817222633.GA26304@code0.codespeak.net> References: <7AD436E4270DD54A94238001769C2227372749F65E@DF-GRTDANE-MSG.exchange.corp.microsoft.com> <20060817222633.GA26304@code0.codespeak.net> Message-ID: High-level remark: this seems to need more unit tests. Perhaps TDD (test-driven design) would be the best approach to obtaining a correct patch? --Guido On 8/17/06, Armin Rigo wrote: > Hi Neal, > > On Wed, Aug 16, 2006 at 10:26:09PM -0700, Neal Norwitz wrote: > > @@ -2367,7 +2374,9 @@ > > compiler_use_next_block(c, end); > > if (!compiler_push_fblock(c, FINALLY_END, end)) > > return 0; > > + c->c_in_finally = 1; > > VISIT_SEQ(c, stmt, s->v.TryFinally.finalbody); > > + c->c_in_finally = 0; > > ADDOP(c, END_FINALLY); > > compiler_pop_fblock(c, FINALLY_END, end); > > Without more inspection, I'd say that this looks like it won't do the > right thing about nested finally's, as in: > > ... > finally: > try: > ... > finally: > ... > continue > > > A bientot, > > Armin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From amk at amk.ca Fri Aug 18 02:26:17 2006 From: amk at amk.ca (A.M. Kuchling) Date: Thu, 17 Aug 2006 20:26:17 -0400 Subject: [Python-Dev] 2.4 & 2.5 beta 3 crash In-Reply-To: <20060817222633.GA26304@code0.codespeak.net> References: <7AD436E4270DD54A94238001769C2227372749F65E@DF-GRTDANE-MSG.exchange.corp.microsoft.com> <20060817222633.GA26304@code0.codespeak.net> Message-ID: <20060818002617.GA17684@Andrew-iBook2.local> On Fri, Aug 18, 2006 at 12:26:33AM +0200, Armin Rigo wrote: > Without more inspection, I'd say that this looks like it won't do the > right thing about nested finally's, as in: I guess it'll have to loop back up through the block stack: for (j=i- 1; j>=0; j--) { switch (c->u->u_fblock[j].fb_type) { case FINALLY_TRY: raise error; case LOOP: j=0; /* Exit the loop */ break; default: break; } } --amk From anthony at interlink.com.au Fri Aug 18 03:04:09 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri, 18 Aug 2006 11:04:09 +1000 Subject: [Python-Dev] TRUNK IS UNFROZEN, available for 2.6 work if you are so inclined In-Reply-To: References: <200608180023.14037.anthony@interlink.com.au> Message-ID: <200608181104.13441.anthony@interlink.com.au> Fixing docs is fine - please checkin. From anthony at interlink.com.au Fri Aug 18 03:05:36 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri, 18 Aug 2006 11:05:36 +1000 Subject: [Python-Dev] uuid tests failing on Windows In-Reply-To: <44E4CC40.7030105@v.loewis.de> References: <44E4C916.9040008@v.loewis.de> <44E4CC40.7030105@v.loewis.de> Message-ID: <200608181105.38897.anthony@interlink.com.au> On Friday 18 August 2006 06:06, Martin v. L?wis wrote: > Georg Brandl schrieb: > >> Can somebody please fix that? If not, should we remove the uuid module > >> as being immature? > > > > Patch #1541863 supposedly solves this. > > Ah, good. I think it should go in. Sounds fine to me. Making buildbot less red == goodness. -- Anthony Baxter It's never too late to have a happy childhood. From nnorwitz at gmail.com Fri Aug 18 09:29:00 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Fri, 18 Aug 2006 00:29:00 -0700 Subject: [Python-Dev] 2.4 & 2.5 beta 3 crash In-Reply-To: <20060818002617.GA17684@Andrew-iBook2.local> References: <7AD436E4270DD54A94238001769C2227372749F65E@DF-GRTDANE-MSG.exchange.corp.microsoft.com> <20060817222633.GA26304@code0.codespeak.net> <20060818002617.GA17684@Andrew-iBook2.local> Message-ID: I did something similar to what Andrew suggested. http://python.org/sf/1542451 Could everyone interested take a look and offer more test cases? n -- On 8/17/06, A.M. Kuchling wrote: > On Fri, Aug 18, 2006 at 12:26:33AM +0200, Armin Rigo wrote: > > Without more inspection, I'd say that this looks like it won't do the > > right thing about nested finally's, as in: > > I guess it'll have to loop back up through the block stack: > > for (j=i- 1; j>=0; j--) { > switch (c->u->u_fblock[j].fb_type) { > case FINALLY_TRY: > raise error; > case LOOP: > j=0; /* Exit the loop */ > break; > default: > break; > } > } > > --amk > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com > From duncan.booth at suttoncourtenay.org.uk Fri Aug 18 10:41:19 2006 From: duncan.booth at suttoncourtenay.org.uk (Duncan Booth) Date: Fri, 18 Aug 2006 09:41:19 +0100 Subject: [Python-Dev] __del__ unexpectedly being called twice Message-ID: There's a thread on comp.lang.python at the moment under the subject "It is __del__ calling twice for some instances?" which seems to show that when releasing a long chain of old-style classes every 50th approximately has its finaliser called twice. I've verified that this happens on both Python 1.4 and 1.5. My guess is that there's a bug in the trashcan mechanism: calling the __del__ method means creating a descriptor, and if that descriptor gets queued in the trashcan then releasing it calls the __del__ method a second time. I'm not sure if there is going to be a particularly easy fix for that. Would someone who knows this code (instance_dealloc in classobject.c) like to have a look at it, should I just submit a bug report, or isn't it worth bothering about? The code which exhibits the problem: #!/usr/local/bin/python -d # -*- coding: koi8-u -*- import sys class foo: def __init__(self, other): self.other = other self._deleted = False global ini_cnt ini_cnt +=1 def __del__(self): if self._deleted: print "aargh!" self._deleted = True global del_cnt del_cnt +=1 print "del",del_cnt,"at",id(self) def stat(): print "-"*20 print "ini_cnt = %d" % ini_cnt print "del_cnt = %d" % del_cnt print "difference = %d" % (ini_cnt-del_cnt) ini_cnt = 0 del_cnt = 0 loop_cnt = 55 a = foo(None) for i in xrange(loop_cnt): a = foo(a) stat() a = None stat() The original thread is at: http://groups.google.com/group/comp.lang.python/browse_thread/thread/293acf433a39583b/bfd4af9c6008a34e From g.brandl at gmx.net Fri Aug 18 10:48:48 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 18 Aug 2006 10:48:48 +0200 Subject: [Python-Dev] 2.4 & 2.5 beta 3 crash In-Reply-To: References: <7AD436E4270DD54A94238001769C2227372749F65E@DF-GRTDANE-MSG.exchange.corp.microsoft.com> <20060817222633.GA26304@code0.codespeak.net> <20060818002617.GA17684@Andrew-iBook2.local> Message-ID: Neal Norwitz wrote: > I did something similar to what Andrew suggested. > > http://python.org/sf/1542451 > > Could everyone interested take a look and offer more test cases? > Note that this needs to be fixed in the compiler package too. Georg From tjreedy at udel.edu Fri Aug 18 13:52:40 2006 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 18 Aug 2006 07:52:40 -0400 Subject: [Python-Dev] __del__ unexpectedly being called twice References: Message-ID: "Duncan Booth" wrote in message news:n2m-g.Xns9823615FCACE1duncanrcpcouk at 127.0.0.1... > There's a thread on comp.lang.python at the moment under the subject "It > is > __del__ calling twice for some instances?" which seems to show that when > releasing a long chain of old-style classes every 50th approximately has > its finaliser called twice. I've verified that this happens on both > Python > 1.4 and 1.5. Should we assume you meant 2.4 and 2.5? From duncan.booth at suttoncourtenay.org.uk Fri Aug 18 14:03:25 2006 From: duncan.booth at suttoncourtenay.org.uk (Duncan Booth) Date: Fri, 18 Aug 2006 13:03:25 +0100 Subject: [Python-Dev] __del__ unexpectedly being called twice References: Message-ID: "Terry Reedy" wrote in news:ec49m8$10q$1 at sea.gmane.org: > > "Duncan Booth" wrote in message > news:n2m-g.Xns9823615FCACE1duncanrcpcouk at 127.0.0.1... >> There's a thread on comp.lang.python at the moment under the subject >> "It is >> __del__ calling twice for some instances?" which seems to show that >> when releasing a long chain of old-style classes every 50th >> approximately has its finaliser called twice. I've verified that this >> happens on both Python >> 1.4 and 1.5. > > Should we assume you meant 2.4 and 2.5? > Probably. 2.5b3 to be a bit more precise. From theller at python.net Fri Aug 18 20:30:01 2006 From: theller at python.net (Thomas Heller) Date: Fri, 18 Aug 2006 20:30:01 +0200 Subject: [Python-Dev] ctypes and win64 Message-ID: The _ctypes extension module does currently not even build on Win64. I'm (slowly) working on this (for AMD64, not for itanium), but it may take a good while before it is stable - It is not even fully implemented currently. The win64 msi installer installs the ctypes package anyway, but it cannot be imported. I suggest that it should be removed from the 2.5 win64 msi installers, so that at least, when it is ready, can be installed as separate package. Thanks, Thomas From martin at v.loewis.de Fri Aug 18 20:47:06 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 18 Aug 2006 20:47:06 +0200 Subject: [Python-Dev] ctypes and win64 In-Reply-To: References: Message-ID: <44E60B2A.6020101@v.loewis.de> Thomas Heller schrieb: > I suggest that it should be removed from the 2.5 win64 msi installers, so that > at least, when it is ready, can be installed as separate package. Unfortunately, I won't be able to work on this until the release is made. Feel free to work on it, but I suggest to just not worry about it. There are many other problems with Win64 still, e.g. the test suite doesn't pass. Regards, Martin From theller at python.net Fri Aug 18 20:54:56 2006 From: theller at python.net (Thomas Heller) Date: Fri, 18 Aug 2006 20:54:56 +0200 Subject: [Python-Dev] ctypes and win64 In-Reply-To: <44E60B2A.6020101@v.loewis.de> References: <44E60B2A.6020101@v.loewis.de> Message-ID: <44E60D00.7090304@python.net> Martin v. L?wis schrieb: > Thomas Heller schrieb: >> I suggest that it should be removed from the 2.5 win64 msi installers, so that >> at least, when it is ready, can be installed as separate package. > > Unfortunately, I won't be able to work on this until the release is > made. Feel free to work on it, but I suggest to just not worry about > it. > > There are many other problems with Win64 still, e.g. the test suite > doesn't pass. And tons of compiler warnings, IIRC. If the state is still experimental, IMO it should be marked as such. Thomas From christopher at baus.net Fri Aug 18 21:00:50 2006 From: christopher at baus.net (christopher baus) Date: Fri, 18 Aug 2006 19:00:50 -0000 (GMT) Subject: [Python-Dev] Failed building 2.5rc1 pythoncore on VC8 Message-ID: <2174.69.181.106.91.1155927650.squirrel@webmail.tuffmail.net> Hi, I downloaded RC1 to give it a go. I failed to build pythoncore with VC8. The error is: Linking... Creating library .\./python25_d.lib and object .\./python25_d.exp config.obj : error LNK2001: unresolved external symbol _init_types ./python25_d.dll : fatal error LNK1120: 1 unresolved externals Thanks, Chris From jimjjewett at gmail.com Fri Aug 18 22:24:08 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri, 18 Aug 2006 16:24:08 -0400 Subject: [Python-Dev] How does this help? Re: [Python-checkins] r51366 - python/trunk/Lib/idlelib/NEWS.txt python/trunk/Lib/idlelib/idlever.py Message-ID: This makes things more consistent for today, but does it really ease maintenance? Why not just change it to: from sys import version as IDLE_VERSION so that it can be ignored from now on? On 8/18/06, anthony.baxter wrote: > Author: anthony.baxter > Date: Fri Aug 18 09:29:02 2006 > New Revision: 51366 > > Modified: > python/trunk/Lib/idlelib/NEWS.txt > python/trunk/Lib/idlelib/idlever.py > Log: > Updating IDLE's version number to match Python's (as per python-dev > discussion). ============================================================================== > --- python/trunk/Lib/idlelib/idlever.py (original) > +++ python/trunk/Lib/idlelib/idlever.py Fri Aug 18 09:29:02 2006 > @@ -1 +1 @@ > -IDLE_VERSION = "1.2c1" > +IDLE_VERSION = "2.6a0" From nnorwitz at gmail.com Fri Aug 18 23:35:40 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Fri, 18 Aug 2006 14:35:40 -0700 Subject: [Python-Dev] Failed building 2.5rc1 pythoncore on VC8 In-Reply-To: <2174.69.181.106.91.1155927650.squirrel@webmail.tuffmail.net> References: <2174.69.181.106.91.1155927650.squirrel@webmail.tuffmail.net> Message-ID: VC8 is not a supported compiler at this point. However, patches are greatly accepted. The _types module was added late and probably VC6 and VC8 project files did not get updated. You can search for the necessary mods to the VC7 proj file(s) on python-checkins. n -- On 8/18/06, christopher baus wrote: > Hi, > > I downloaded RC1 to give it a go. > > I failed to build pythoncore with VC8. > > The error is: > > Linking... > Creating library .\./python25_d.lib and object .\./python25_d.exp > config.obj : error LNK2001: unresolved external symbol _init_types > ./python25_d.dll : fatal error LNK1120: 1 unresolved externals > > Thanks, > > Chris > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com > From martin at v.loewis.de Fri Aug 18 23:48:18 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 18 Aug 2006 23:48:18 +0200 Subject: [Python-Dev] ctypes and win64 In-Reply-To: <44E60D00.7090304@python.net> References: <44E60B2A.6020101@v.loewis.de> <44E60D00.7090304@python.net> Message-ID: <44E635A2.3020000@v.loewis.de> Thomas Heller schrieb: >> There are many other problems with Win64 still, e.g. the test suite >> doesn't pass. > > And tons of compiler warnings, IIRC. If the state is still experimental, > IMO it should be marked as such. Most of the compiler warnings are irrelevant for most of the applications, and some are bogus (in the sense that the code is perfectly fine as it stands). Not sure what "experimental" would mean; it normally means that it may change or may be abandoned as a failure. I don't think this is the case with this port (although I do want to drop IA-64 binaries in 2.6, due to lack of interest). If you think it means "has known bugs", then this applies to all Python releases on all target systems, ever (just look at the open bugs list). Regards, Martin From christopher at baus.net Sat Aug 19 01:46:18 2006 From: christopher at baus.net (christopher baus) Date: Fri, 18 Aug 2006 23:46:18 -0000 (GMT) Subject: [Python-Dev] Failed building 2.5rc1 pythoncore on VC8 In-Reply-To: References: <2174.69.181.106.91.1155927650.squirrel@webmail.tuffmail.net> Message-ID: <2136.69.181.106.91.1155944778.squirrel@webmail.tuffmail.net> > VC8 is not a supported compiler at this point. However, patches are > greatly accepted. > > The _types module was added late and probably VC6 and VC8 project > files did not get updated. You can search for the necessary mods to > the VC7 proj file(s) on python-checkins. > Index: pythoncore.vcproj =================================================================== --- pythoncore.vcproj (revision 1020) +++ pythoncore.vcproj (working copy) @@ -1,7 +1,7 @@ + + From christopher at baus.net Sat Aug 19 02:54:53 2006 From: christopher at baus.net (christopher baus) Date: Sat, 19 Aug 2006 00:54:53 -0000 (GMT) Subject: [Python-Dev] patch to add socket module project to vc8 solution Message-ID: <2928.69.181.106.91.1155948893.squirrel@webmail.tuffmail.net> This is a patch to add the socket module project to the vc8 solution. Index: pcbuild.sln =================================================================== --- pcbuild.sln (revision 1020) +++ pcbuild.sln (working copy) @@ -2,8 +2,8 @@ # Visual Studio 2005 Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pythoncore", "pythoncore.vcproj", "{CF7AC3D1-E2DF-41D2-BEA6-1E2556CDEA26}" ProjectSection(ProjectDependencies) = postProject + {F0E0541E-F17D-430B-97C4-93ADF0DD284E} = {F0E0541E-F17D-430B-97C4-93ADF0DD284E} {C73F0EC1-358B-4177-940F-0846AC8B04CD} = {C73F0EC1-358B-4177-940F-0846AC8B04CD} - {F0E0541E-F17D-430B-97C4-93ADF0DD284E} = {F0E0541E-F17D-430B-97C4-93ADF0DD284E} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pythonw", "pythonw.vcproj", "{F4229CC3-873C-49AE-9729-DD308ED4CD4A}" @@ -70,6 +70,8 @@ EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "make_versioninfo", "make_versioninfo.vcproj", "{F0E0541E-F17D-430B-97C4-93ADF0DD284E}" EndProject +Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "_socket", "_socket.vcproj", "{324F66C2-44D0-4D50-B979-F9DAE7FD36DB}" +EndProject Global GlobalSection(SolutionConfigurationPlatforms) = preSolution Debug|Win32 = Debug|Win32 @@ -192,6 +194,14 @@ {F0E0541E-F17D-430B-97C4-93ADF0DD284E}.ReleaseAMD64|Win32.Build.0 = Release|Win32 {F0E0541E-F17D-430B-97C4-93ADF0DD284E}.ReleaseItanium|Win32.ActiveCfg = Release|Win32 {F0E0541E-F17D-430B-97C4-93ADF0DD284E}.ReleaseItanium|Win32.Build.0 = Release|Win32 + {324F66C2-44D0-4D50-B979-F9DAE7FD36DB}.Debug|Win32.ActiveCfg = Debug|Win32 + {324F66C2-44D0-4D50-B979-F9DAE7FD36DB}.Debug|Win32.Build.0 = Debug|Win32 + {324F66C2-44D0-4D50-B979-F9DAE7FD36DB}.Release|Win32.ActiveCfg = Release|Win32 + {324F66C2-44D0-4D50-B979-F9DAE7FD36DB}.Release|Win32.Build.0 = Release|Win32 + {324F66C2-44D0-4D50-B979-F9DAE7FD36DB}.ReleaseAMD64|Win32.ActiveCfg = ReleaseAMD64|Win32 + {324F66C2-44D0-4D50-B979-F9DAE7FD36DB}.ReleaseAMD64|Win32.Build.0 = ReleaseAMD64|Win32 + {324F66C2-44D0-4D50-B979-F9DAE7FD36DB}.ReleaseItanium|Win32.ActiveCfg = ReleaseItanium|Win32 + {324F66C2-44D0-4D50-B979-F9DAE7FD36DB}.ReleaseItanium|Win32.Build.0 = ReleaseItanium|Win32 EndGlobalSection GlobalSection(SolutionProperties) = preSolution HideSolutionNode = FALSE From christopher at baus.net Sat Aug 19 02:56:42 2006 From: christopher at baus.net (christopher baus) Date: Sat, 19 Aug 2006 00:56:42 -0000 (GMT) Subject: [Python-Dev] patch to add socket module project to vc8 solution In-Reply-To: <2928.69.181.106.91.1155948893.squirrel@webmail.tuffmail.net> References: <2928.69.181.106.91.1155948893.squirrel@webmail.tuffmail.net> Message-ID: <2931.69.181.106.91.1155949002.squirrel@webmail.tuffmail.net> > This is a patch to add the socket module project to the vc8 solution. > Sorry the patch got munged by my mail client. I attached it instead. -------------- next part -------------- A non-text attachment was scrubbed... Name: addsocket.patch Type: application/octet-stream Size: 2294 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20060819/5d64d98d/attachment.obj From jjl at pobox.com Sat Aug 19 04:05:14 2006 From: jjl at pobox.com (John J Lee) Date: Sat, 19 Aug 2006 02:05:14 +0000 (UTC) Subject: [Python-Dev] One-line fix for urllib2 regression Message-ID: Revision 50842 made a change to an undocumented interface of urllib2 that I'm sure will break real code. Patch 1542948 reverts the part of that commit that applied to urllib2, and adds a one-line fix in its place that addresses the problem that 50842 fixed. Details are on the patch tracker: http://python.org/sf/1542948 Can somebody commit this for 2.5? John From guido at python.org Sat Aug 19 04:21:10 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 18 Aug 2006 19:21:10 -0700 Subject: [Python-Dev] os.spawnlp() missing on Windows in 2.4? Message-ID: I just got a report from a Windows user that os.spawnlp() is missing from Python 2.4, despite being mentioned in the docs. Can someone confirm this? My Windows box is resting. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From scott+python-dev at scottdial.com Sat Aug 19 05:16:30 2006 From: scott+python-dev at scottdial.com (Scott Dial) Date: Fri, 18 Aug 2006 23:16:30 -0400 Subject: [Python-Dev] os.spawnlp() missing on Windows in 2.4? In-Reply-To: References: Message-ID: <44E6828E.4000505@scottdial.com> Guido van Rossum wrote: > I just got a report from a Windows user that os.spawnlp() is missing > from Python 2.4, despite being mentioned in the docs. Can someone > confirm this? My Windows box is resting. :-) > "Availability: Unix, Windows. spawnlp(), spawnlpe(), spawnvp() and spawnvpe() are not available on Windows. New in version 1.6." One could argue that it presented poorly, but it reads completely correct. Alternatively one could says "The 'p' variants are unavailable on Windows." but that would be assuming someone understand there was a scheme to the names :-) -- Scott Dial scott at scottdial.com scodial at indiana.edu From steve at holdenweb.com Sat Aug 19 05:27:53 2006 From: steve at holdenweb.com (Steve Holden) Date: Sat, 19 Aug 2006 04:27:53 +0100 Subject: [Python-Dev] ctypes and win64 In-Reply-To: <44E635A2.3020000@v.loewis.de> References: <44E60B2A.6020101@v.loewis.de> <44E60D00.7090304@python.net> <44E635A2.3020000@v.loewis.de> Message-ID: Martin v. L?wis wrote: > Thomas Heller schrieb: > >>>There are many other problems with Win64 still, e.g. the test suite >>>doesn't pass. >> >>And tons of compiler warnings, IIRC. If the state is still experimental, >>IMO it should be marked as such. > > > Most of the compiler warnings are irrelevant for most of the > applications, and some are bogus (in the sense that the code > is perfectly fine as it stands). > > Not sure what "experimental" would mean; it normally means > that it may change or may be abandoned as a failure. I don't > think this is the case with this port (although I do want > to drop IA-64 binaries in 2.6, due to lack of interest). > > If you think it means "has known bugs", then this applies to > all Python releases on all target systems, ever (just look > at the open bugs list). > Reasonable enough, but I suspect that Thomas' suggestion might save us from raising false hopes. I'd suggest that the final release announcement point out that this is the first release containing specific support for 64-bit architectures (if indeed it is), and that the developers would therefore expect a higher proportion of bug reports on this release to be associated with those platforms. Committed to fixing, blah blah blah ... regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden From steve at holdenweb.com Sat Aug 19 05:34:16 2006 From: steve at holdenweb.com (Steve Holden) Date: Sat, 19 Aug 2006 04:34:16 +0100 Subject: [Python-Dev] os.spawnlp() missing on Windows in 2.4? In-Reply-To: <44E6828E.4000505@scottdial.com> References: <44E6828E.4000505@scottdial.com> Message-ID: <44E686B8.6040609@holdenweb.com> Scott Dial wrote: > Guido van Rossum wrote: > >>I just got a report from a Windows user that os.spawnlp() is missing >>from Python 2.4, despite being mentioned in the docs. Can someone >>confirm this? My Windows box is resting. :-) >> > > > "Availability: Unix, Windows. spawnlp(), spawnlpe(), spawnvp() and > spawnvpe() are not available on Windows. New in version 1.6." > > One could argue that it presented poorly, but it reads completely > correct. Alternatively one could says "The 'p' variants are unavailable > on Windows." but that would be assuming someone understand there was a > scheme to the names :-) > How about: "Availability: Unix; Windows PARTIAL (spawnlp(), spawnlpe(), spawnvp() and spawnvpe() are not implemented). New in version 1.6" regards Steve PS: It's not resting, it has gone to meet its maker. It has joined the choir celestial, it has shuffled off this mortal coil, its clock has ceased to cycle, it is no more, it is an X-box. Oh, sorry, that's a Microsoft product isn't it. -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden From steve at holdenweb.com Sat Aug 19 05:34:16 2006 From: steve at holdenweb.com (Steve Holden) Date: Sat, 19 Aug 2006 04:34:16 +0100 Subject: [Python-Dev] os.spawnlp() missing on Windows in 2.4? In-Reply-To: <44E6828E.4000505@scottdial.com> References: <44E6828E.4000505@scottdial.com> Message-ID: <44E686B8.6040609@holdenweb.com> Scott Dial wrote: > Guido van Rossum wrote: > >>I just got a report from a Windows user that os.spawnlp() is missing >>from Python 2.4, despite being mentioned in the docs. Can someone >>confirm this? My Windows box is resting. :-) >> > > > "Availability: Unix, Windows. spawnlp(), spawnlpe(), spawnvp() and > spawnvpe() are not available on Windows. New in version 1.6." > > One could argue that it presented poorly, but it reads completely > correct. Alternatively one could says "The 'p' variants are unavailable > on Windows." but that would be assuming someone understand there was a > scheme to the names :-) > How about: "Availability: Unix; Windows PARTIAL (spawnlp(), spawnlpe(), spawnvp() and spawnvpe() are not implemented). New in version 1.6" regards Steve PS: It's not resting, it has gone to meet its maker. It has joined the choir celestial, it has shuffled off this mortal coil, its clock has ceased to cycle, it is no more, it is an X-box. Oh, sorry, that's a Microsoft product isn't it. -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden From collinw at gmail.com Sat Aug 19 05:40:30 2006 From: collinw at gmail.com (Collin Winter) Date: Fri, 18 Aug 2006 22:40:30 -0500 Subject: [Python-Dev] Questions on unittest behaviour Message-ID: <43aa6ff70608182040i43c28461i81011a18d5923ccc@mail.gmail.com> [Sorry for accidentally cross-posting this to python-list] While working on a test suite for unittest these past few weeks, I've run across some behaviours that, while not obviously wrong, don't strike me as quite right, either. Submitted for your consideration: 1) TestCase.tearDown() is only run if TestCase.setUp() succeeded. It seems to me that tearDown() should always be run, regardless of any failures in setUp() or the test method itself. The case I'm considering is something like this, ie, a multi-part setUp(): > def setUp(self) > lock_file(testfile) # open_socket(), connect_to_database(), etc > > something_that_raises_an_exception() > > def tearDown(self): > if file_is_locked(testfile): > unlock_file(testfile) In this pseudo-code example, the file won't be unlocked if some later operation in setUp() raises an exception. I propose that TestCase.run() be changed to always run tearDown(), even if setUp() raise an exception. I'm undecided if this is a new feature (so it should go in for 2.6) or a bug fix; I'm leaning toward the latter. 2) The TestLoader.testMethodPrefix attribute currently allows anything to be assigned to it, including invalid objects and the empty string. While the former will cause errors to be raised when one of TestLoader's loadTestsFrom*() methods is called, the empty string will raise no exception; rather, the loadTestsFrom*() methods will interpret every possible attribute as being a test method, e.g., meaning you get things like assertEqual(), failUnlessEqual(), etc, when TestLoader.loadTestsFromTestCase() is run. I propose protecting testMethodPrefix with a property that validates the assigned value, restricting input to non-empty instances of str. I see this as a bug fix that should go in before 2.5-final. 3) TestLoader.loadTestsFromTestCase() accepts objects that are not test cases and will happily look for appropriately-named methods on any object you give it. This flexibility should be documented, or proper input validation should be done (a bug fix for 2.5). 4) TestLoader.loadTestsFromName() (and by extension, loadTestsFromNames(), too) raises an AttributeError if the name is the empty string because -- as it correctly asserts -- the object does not contain an attribute named ''. I recommend that this be tested for and ValueError be raised (bug fix for 2.5). This of course leads into the question of how much input validation should be done on these names. Should loadTestsFrom{Name,Names}() make sure the names are valid attribute names, or is this overkill? 5) When TestLoader.loadTestsFrom{Name,Names}() are given a name that resolves to a classmethod on a TestCase subclass, the method is not invoked. From the docs: > The specifier name is a ``dotted name'' that may resolve either to a module, a test > case class, a TestSuite instance, a test method within a test case class, or a > callable object which returns a TestCase or TestSuite instance. It is not documented which of these tests takes priority: is the classmethod "a test method within a test case class" or is it a callable? The same issue applies to staticmethods as well. Once I get answers to these questions, I can finish off the last few bits of the test suite and have it ready for 2.5-final. Thanks, Collin Winter From g.brandl at gmx.net Sat Aug 19 07:28:56 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 19 Aug 2006 07:28:56 +0200 Subject: [Python-Dev] One-line fix for urllib2 regression In-Reply-To: References: Message-ID: John J Lee wrote: > Revision 50842 made a change to an undocumented interface of urllib2 that > I'm sure will break real code. > > Patch 1542948 reverts the part of that commit that applied to urllib2, and > adds a one-line fix in its place that addresses the problem that 50842 > fixed. > > Details are on the patch tracker: > > http://python.org/sf/1542948 > > > Can somebody commit this for 2.5? As the committer of the original patch, I have reviewed it and I'm for it. Georg From martin at v.loewis.de Sat Aug 19 07:51:36 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 19 Aug 2006 07:51:36 +0200 Subject: [Python-Dev] ctypes and win64 In-Reply-To: References: <44E60B2A.6020101@v.loewis.de> <44E60D00.7090304@python.net> <44E635A2.3020000@v.loewis.de> Message-ID: <44E6A6E8.3000101@v.loewis.de> Steve Holden schrieb: > Reasonable enough, but I suspect that Thomas' suggestion might save us > from raising false hopes. I'd suggest that the final release > announcement point out that this is the first release containing > specific support for 64-bit architectures (if indeed it is) It isn't. Python ran on 64-bit Alpha for nearly a decade now (I guess), and was released for Win64 throughout Python 2.4. ActiveState has been releasing an AMD64 package for some time now. Regards, Martin From ncoghlan at gmail.com Sat Aug 19 08:23:21 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 19 Aug 2006 16:23:21 +1000 Subject: [Python-Dev] Questions on unittest behaviour In-Reply-To: <43aa6ff70608182040i43c28461i81011a18d5923ccc@mail.gmail.com> References: <43aa6ff70608182040i43c28461i81011a18d5923ccc@mail.gmail.com> Message-ID: <44E6AE59.50602@gmail.com> Collin Winter wrote: > [Sorry for accidentally cross-posting this to python-list] > > While working on a test suite for unittest these past few weeks, I've > run across some behaviours that, while not obviously wrong, don't > strike me as quite right, either. Submitted for your consideration: > > 1) TestCase.tearDown() is only run if TestCase.setUp() succeeded. It > seems to me that tearDown() should always be run, regardless of any > failures in setUp() or the test method itself. > The case I'm considering is something like this, ie, a multi-part setUp(): > >> def setUp(self) >> lock_file(testfile) # open_socket(), connect_to_database(), etc >> >> something_that_raises_an_exception() >> >> def tearDown(self): >> if file_is_locked(testfile): >> unlock_file(testfile) > > In this pseudo-code example, the file won't be unlocked if some later > operation in setUp() raises an exception. I propose that > TestCase.run() be changed to always run tearDown(), even if setUp() > raise an exception. > > I'm undecided if this is a new feature (so it should go in for 2.6) or > a bug fix; I'm leaning toward the latter. On this point, I believe the current behaviour is correct and should be kept. If setUp() fails internally (such that the test step isn't going to be run) it needs to cleanup after itself, just like any other function. That way, the tearDown() method is allowed to assume that setUp() succeeded completely, instead of having to guard against the possibility that setUp() may have died partway through. IOW, I consider the setUp() method in your example to be buggy. It should be written something like this: def setUp(self) lock_file(testfile) # open_socket(), connect_to_database(), etc try: something_that_may_raise_an_exception() except: unlock_file(testfile) raise def tearDown(self): unlock_file(testfile) Alternatively, someone who prefers your style (with a tearDown() method that can handle a partially executed call to the setUp() method), can just write it as: def setUp(self) try: lock_file(testfile) # open_socket(), connect_to_database(), etc something_that_may_raise_an_exception() except: self.tearDown() raise def tearDown(self): if file_is_locked(testfile): unlock_file(testfile) Consider the parallel to PEP 343's __enter__ and __exit__ methods - __exit__ is allowed to assume that it will only be called if __enter__ succeeded, because that is part of the semantics of the with statement. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From jml at mumak.net Sat Aug 19 08:36:48 2006 From: jml at mumak.net (Jonathan Lange) Date: Sat, 19 Aug 2006 16:36:48 +1000 Subject: [Python-Dev] Questions on unittest behaviour In-Reply-To: <43aa6ff70608182040i43c28461i81011a18d5923ccc@mail.gmail.com> References: <43aa6ff70608182040i43c28461i81011a18d5923ccc@mail.gmail.com> Message-ID: On 8/19/06, Collin Winter wrote: > 1) TestCase.tearDown() is only run if TestCase.setUp() succeeded. It > seems to me that tearDown() should always be run, regardless of any > failures in setUp() or the test method itself. > > The case I'm considering is something like this, ie, a multi-part setUp(): > > def setUp(self) > > lock_file(testfile) # open_socket(), connect_to_database(), etc > > > > something_that_raises_an_exception() > > > > def tearDown(self): > > if file_is_locked(testfile): > > unlock_file(testfile) > > I'm undecided if this is a new feature (so it should go in for 2.6) or > a bug fix; I'm leaning toward the latter. > Most existing tearDown() code assumes that setUp() has been successfully called -- after all, that's what the docs say. It's also the behaviour of other xUnit frameworks. If this change is made, people will have to go through their tearDown() methods and add checks like the one in your example in order to make their tests correct. Further, it won't be obvious that their tearDown() methods are incorrect until something happens to make their setUp() methods incorrect. I don't think this change is a good one. However, if it does go in, it definitely shouldn't go in as a bug fix. cheers, jml From anthony at interlink.com.au Sat Aug 19 10:13:59 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Sat, 19 Aug 2006 18:13:59 +1000 Subject: [Python-Dev] How does this help? Re: [Python-checkins] r51366 - python/trunk/Lib/idlelib/NEWS.txt python/trunk/Lib/idlelib/idlever.py In-Reply-To: References: Message-ID: <200608191814.05404.anthony@interlink.com.au> On Saturday 19 August 2006 06:24, Jim Jewett wrote: > This makes things more consistent for today, but does it really ease > maintenance? > > Why not just change it to: > > from sys import version as IDLE_VERSION > > so that it can be ignored from now on? After the fuss about changing distutils' version number? :-) I'd very much like to do this, but I figured this was a good first step. -- Anthony Baxter It's never too late to have a happy childhood. From anthony at interlink.com.au Sat Aug 19 10:16:11 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Sat, 19 Aug 2006 18:16:11 +1000 Subject: [Python-Dev] One-line fix for urllib2 regression In-Reply-To: References: Message-ID: <200608191816.12174.anthony@interlink.com.au> On Saturday 19 August 2006 15:28, Georg Brandl wrote: > John J Lee wrote: > > Revision 50842 made a change to an undocumented interface of urllib2 that > > I'm sure will break real code. > > > > Patch 1542948 reverts the part of that commit that applied to urllib2, > > and adds a one-line fix in its place that addresses the problem that > > 50842 fixed. > > > > Details are on the patch tracker: > > > > http://python.org/sf/1542948 > > > > > > Can somebody commit this for 2.5? > > As the committer of the original patch, I have reviewed it and I'm for it. I agree with Neal's comment - this needs tests to make sure it doesn't break again. It's OK to apply it with the tests. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From tim.peters at gmail.com Sat Aug 19 11:19:40 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sat, 19 Aug 2006 05:19:40 -0400 Subject: [Python-Dev] ctypes and win64 In-Reply-To: <44E6A6E8.3000101@v.loewis.de> References: <44E60B2A.6020101@v.loewis.de> <44E60D00.7090304@python.net> <44E635A2.3020000@v.loewis.de> <44E6A6E8.3000101@v.loewis.de> Message-ID: <1f7befae0608190219m5429d505i8a155e943e76d3fd@mail.gmail.com> [Steve Holden] >> Reasonable enough, but I suspect that Thomas' suggestion might save us >> from raising false hopes. I'd suggest that the final release >> announcement point out that this is the first release containing >> specific support for 64-bit architectures (if indeed it is) [Martin v. L?wis] > It isn't. Python ran on 64-bit Alpha for nearly a decade now (I guess), > and was released for Win64 throughout Python 2.4. ActiveState has > been releasing an AMD64 package for some time now. Python has also been used on 64-bit Crays, and I actually did the first 64-bit port in 1993 (to a KSR Unix machine -- took less than a day to get it running fine! Guido's an excellent C coder.). Win64 is the first (and probably only forever more) where sizeof(long) < sizeof(void*), and that caused some Win64-unique pain, and may cause more. BTW, at least two of the people at the NFS sprint earlier this year were compiling and running Python on Win64 laptops. It's "solid enough", and surely nobody expects that Win64 users expect 100% perfection of anything they run <0.5 wink>. From steve at holdenweb.com Sat Aug 19 12:28:13 2006 From: steve at holdenweb.com (Steve Holden) Date: Sat, 19 Aug 2006 11:28:13 +0100 Subject: [Python-Dev] ctypes and win64 In-Reply-To: <1f7befae0608190219m5429d505i8a155e943e76d3fd@mail.gmail.com> References: <44E60B2A.6020101@v.loewis.de> <44E60D00.7090304@python.net> <44E635A2.3020000@v.loewis.de> <44E6A6E8.3000101@v.loewis.de> <1f7befae0608190219m5429d505i8a155e943e76d3fd@mail.gmail.com> Message-ID: <44E6E7BD.8000903@holdenweb.com> Tim Peters wrote: > [Steve Holden] > >>> Reasonable enough, but I suspect that Thomas' suggestion might save us >>> from raising false hopes. I'd suggest that the final release >>> announcement point out that this is the first release containing >>> specific support for 64-bit architectures (if indeed it is) > > > [Martin v. L?wis] > >> It isn't. Python ran on 64-bit Alpha for nearly a decade now (I guess), >> and was released for Win64 throughout Python 2.4. ActiveState has >> been releasing an AMD64 package for some time now. > > > Python has also been used on 64-bit Crays, and I actually did the > first 64-bit port in 1993 (to a KSR Unix machine -- took less than a > day to get it running fine! Guido's an excellent C coder.). Win64 is > the first (and probably only forever more) where sizeof(long) < > sizeof(void*), and that caused some Win64-unique pain, and may cause > more. > > BTW, at least two of the people at the NFS sprint earlier this year > were compiling and running Python on Win64 laptops. It's "solid > enough", and surely nobody expects that Win64 users expect 100% > perfection of anything they run <0.5 wink>. > I suppose we should be grateful thirteen years experience has gone into 64-bit support. I agree that "some breakage may occur" is true of any release and architecture though in the eyes of this lurker, at least, python-dev does a more-than-averagely conscientious job. It's going to be very interesting to see what comes out of the Google sprints. I am sure the 64-bitters will be out in force, so there'll be useful data about any such problems. 64 bits is, I am sure, as much as anyone is ever going to need, so the pain will finally be over. It's good to see the idea of industry support for open source sprints taking off. Tomorrow, the world ... :-) regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden From kbk at shore.net Sat Aug 19 17:40:03 2006 From: kbk at shore.net (Kurt B. Kaiser) Date: Sat, 19 Aug 2006 11:40:03 -0400 Subject: [Python-Dev] [Python-checkins] How does this help? Re: r51366 - python/trunk/Lib/idlelib/NEWS.txt python/trunk/Lib/idlelib/idlever.py In-Reply-To: <200608191814.05404.anthony@interlink.com.au> (Anthony Baxter's message of "Sat, 19 Aug 2006 18:13:59 +1000") References: <200608191814.05404.anthony@interlink.com.au> Message-ID: <87fyfs4tv0.fsf@hydra.bayview.thirdcreek.com> Anthony Baxter writes: > On Saturday 19 August 2006 06:24, Jim Jewett wrote: >> This makes things more consistent for today, but does it really ease >> maintenance? >> >> Why not just change it to: >> >> from sys import version as IDLE_VERSION >> >> so that it can be ignored from now on? > > After the fuss about changing distutils' version number? :-) > > I'd very much like to do this, but I figured this was a good first step. Jim Jewett's suggestion could actually be misleading, IMO. There are users, especially on Windows, who run IDLE out of svn or run their own version, but don't build Python. Here's my suggestion: we set IDLE_VERSION="" in idlever.py. If the IDLE hacker wants to set a specific version, he sets that string. I'll change the code so if the string is empty, IDLE will use the Python version, as Jim suggested; otherwise, the user supplied string. -- KBK From jack at psynchronous.com Sat Aug 19 16:27:45 2006 From: jack at psynchronous.com (Jack Diederich) Date: Sat, 19 Aug 2006 10:27:45 -0400 Subject: [Python-Dev] os.spawnlp() missing on Windows in 2.4? In-Reply-To: <44E686B8.6040609@holdenweb.com> References: <44E6828E.4000505@scottdial.com> <44E686B8.6040609@holdenweb.com> Message-ID: <20060819142745.GJ5772@performancedrivers.com> On Sat, Aug 19, 2006 at 04:34:16AM +0100, Steve Holden wrote: > Scott Dial wrote: > > Guido van Rossum wrote: > > > >>I just got a report from a Windows user that os.spawnlp() is missing > >>from Python 2.4, despite being mentioned in the docs. Can someone > >>confirm this? My Windows box is resting. :-) > >> > > > > > > "Availability: Unix, Windows. spawnlp(), spawnlpe(), spawnvp() and > > spawnvpe() are not available on Windows. New in version 1.6." > > > > One could argue that it presented poorly, but it reads completely > > correct. Alternatively one could says "The 'p' variants are unavailable > > on Windows." but that would be assuming someone understand there was a > > scheme to the names :-) > > > How about: > > "Availability: Unix; Windows PARTIAL (spawnlp(), spawnlpe(), spawnvp() > and spawnvpe() are not implemented). New in version 1.6" > Or *Availability: Unix: All, Windows: spawnl(), spawnle(), spawnv(), spawnve() only. New in version 1.6 Might as well positively list the half that is there instead of the half that isn't. -Jack From jack at psynchronous.com Sat Aug 19 16:40:42 2006 From: jack at psynchronous.com (Jack Diederich) Date: Sat, 19 Aug 2006 10:40:42 -0400 Subject: [Python-Dev] ctypes and win64 In-Reply-To: <1f7befae0608190219m5429d505i8a155e943e76d3fd@mail.gmail.com> References: <44E60B2A.6020101@v.loewis.de> <44E60D00.7090304@python.net> <44E635A2.3020000@v.loewis.de> <44E6A6E8.3000101@v.loewis.de> <1f7befae0608190219m5429d505i8a155e943e76d3fd@mail.gmail.com> Message-ID: <20060819144042.GK5772@performancedrivers.com> On Sat, Aug 19, 2006 at 05:19:40AM -0400, Tim Peters wrote: > [Steve Holden] > >> Reasonable enough, but I suspect that Thomas' suggestion might save us > >> from raising false hopes. I'd suggest that the final release > >> announcement point out that this is the first release containing > >> specific support for 64-bit architectures (if indeed it is) > > [Martin v. L?wis] > > It isn't. Python ran on 64-bit Alpha for nearly a decade now (I guess), > > and was released for Win64 throughout Python 2.4. ActiveState has > > been releasing an AMD64 package for some time now. > > Python has also been used on 64-bit Crays, and I actually did the > first 64-bit port in 1993 (to a KSR Unix machine -- took less than a > day to get it running fine! Guido's an excellent C coder.). Win64 is > the first (and probably only forever more) where sizeof(long) < > sizeof(void*), and that caused some Win64-unique pain, and may cause > more. > > BTW, at least two of the people at the NFS sprint earlier this year > were compiling and running Python on Win64 laptops. It's "solid > enough", and surely nobody expects that Win64 users expect 100% > perfection of anything they run <0.5 wink>. It has always "just worked" for me on Opterons + Debian. Python 2.4 (#1, May 31 2005, 10:19:45) [GCC 3.3.5 (Debian 1:3.3.5-12)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.maxint 9223372036854775807 Thanks to the Alphas for making AMD64 on *nix a no-brainer, -Jack From collinw at gmail.com Sat Aug 19 19:06:17 2006 From: collinw at gmail.com (Collin Winter) Date: Sat, 19 Aug 2006 12:06:17 -0500 Subject: [Python-Dev] Questions on unittest behaviour In-Reply-To: <44E6AE59.50602@gmail.com> References: <43aa6ff70608182040i43c28461i81011a18d5923ccc@mail.gmail.com> <44E6AE59.50602@gmail.com> Message-ID: <43aa6ff70608191006g5f9bf89fqfe56409792deb866@mail.gmail.com> On 8/19/06, Nick Coghlan wrote: > Alternatively, someone who prefers your style (with a tearDown() method that > can handle a partially executed call to the setUp() method), can just write it as: > > def setUp(self) > try: > lock_file(testfile) # open_socket(), connect_to_database(), etc > something_that_may_raise_an_exception() > except: > self.tearDown() > raise > > def tearDown(self): > if file_is_locked(testfile): > unlock_file(testfile) > > Consider the parallel to PEP 343's __enter__ and __exit__ methods - __exit__ > is allowed to assume that it will only be called if __enter__ succeeded, > because that is part of the semantics of the with statement. I can accept that. Any thoughts on the other four items? Collin Winter From tjreedy at udel.edu Sat Aug 19 19:17:07 2006 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 19 Aug 2006 13:17:07 -0400 Subject: [Python-Dev] os.spawnlp() missing on Windows in 2.4? References: <44E6828E.4000505@scottdial.com> <44E686B8.6040609@holdenweb.com> <20060819142745.GJ5772@performancedrivers.com> Message-ID: "Jack Diederich" wrote in message news:20060819142745.GJ5772 at performancedrivers.com... > *Availability: Unix: All, Windows: spawnl(), spawnle(), spawnv(), > spawnve() only. New in version 1.6 > > Might as well positively list the half that is there instead of the half > that isn't. Definitately. Succinct and clear. From ncoghlan at gmail.com Sat Aug 19 20:07:53 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 20 Aug 2006 04:07:53 +1000 Subject: [Python-Dev] Questions on unittest behaviour In-Reply-To: <43aa6ff70608191006g5f9bf89fqfe56409792deb866@mail.gmail.com> References: <43aa6ff70608182040i43c28461i81011a18d5923ccc@mail.gmail.com> <44E6AE59.50602@gmail.com> <43aa6ff70608191006g5f9bf89fqfe56409792deb866@mail.gmail.com> Message-ID: <44E75379.2020506@gmail.com> Collin Winter wrote: > Any thoughts on the other four items? Generally speaking, I'm not sure it's worth the effort to do the input validation. All of the cases you suggest ruling out do indeed seem to be insane, but someone may have defined a subclass that does something based on the current behaviour. Given the timing, I suggest just documenting and testing the current behaviour for Python 2.5, and then creating a tracker item targeting 2.6 to cover the lack of sanity checks (and whether or not more should be added). Some of the problems have multiple possible solutions (e.g., using inspect.getargspec() to check for zero-argument callables when searching for test methods, instead of trying to call everything unittest can lay its hands on) so we need more time to think about them. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From jjl at pobox.com Sun Aug 20 02:27:10 2006 From: jjl at pobox.com (John J Lee) Date: Sun, 20 Aug 2006 00:27:10 +0000 (UTC) Subject: [Python-Dev] String formatting / unicode 2.5 bug? Message-ID: Is this a bug? # run with 2.4 and then with 2.5 (I'm running release25-maint:51410) class a(object): def __getattribute__(self, name): print "accessing %r.%s" % (self, name) return object.__getattribute__(self, name) def __str__(self): print "__str__" return u'hi' print repr(str(a())) print print repr("%s" % a()) John From jackilyn at gmail.com Sun Aug 20 02:24:59 2006 From: jackilyn at gmail.com (Jackilyn Hoxworth) Date: Sat, 19 Aug 2006 19:24:59 -0500 Subject: [Python-Dev] SSH Key Added Message-ID: <58940d080608191724g6596e97av754ee61e30ed5e17@mail.gmail.com> Can I have this key added? -- Jackilyn Hoxworth -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060819/9da4c3a8/attachment.htm -------------- next part -------------- A non-text attachment was scrubbed... Name: jah_key.pub Type: application/octet-stream Size: 606 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20060819/9da4c3a8/attachment.obj From ncoghlan at gmail.com Sun Aug 20 07:03:21 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 20 Aug 2006 15:03:21 +1000 Subject: [Python-Dev] String formatting / unicode 2.5 bug? In-Reply-To: References: Message-ID: <44E7ED19.6090907@gmail.com> John J Lee wrote: > Is this a bug? I don't believe so - the string formatting documentation states that the result will be unicode if either the format string is unicode or any of the objects passed to a %s format code is unicode. That latter part has just been extended to include any object that returns Unicode from __str__, instead of being restricted to actual Unicode instances. Note that the following behaves the same way regardless of whether you use 2.4 or 2.5: "%s" % 'hi' "%s" % u'hi' And once the result has been promoted to unicode, __unicode__ is used directly: >>> print repr("%s%s" % (a(), a())) __str__ accessing <__main__.a object at 0x00AF66F0>.__unicode__ __str__ accessing <__main__.a object at 0x00AF6390>.__unicode__ __str__ u'hihi' Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From python-dev at zesty.ca Sun Aug 20 07:38:57 2006 From: python-dev at zesty.ca (Ka-Ping Yee) Date: Sun, 20 Aug 2006 00:38:57 -0500 (CDT) Subject: [Python-Dev] uuid module - byte order issue In-Reply-To: <7168d65a0608041126y6f02f127of48530b752afef74@mail.gmail.com> References: <7168d65a0608031311g3479824ds18a9ede4a33cae7e@mail.gmail.com> <7168d65a0608041126y6f02f127of48530b752afef74@mail.gmail.com> Message-ID: On Fri, 4 Aug 2006, Oren Tirosh wrote: > Compatibility with Windows "GUIDs" may be one of the most important > use cases for the UUID module. It's important to resolve this or users > will have unpleasant surprises. I did. [...] > alternatives: > > 1. Default is big endian byte order. Little endian is explicit. > 2. Default is native byte order. Little endian and big endian are explicit. > 3. No default. Little endian and big endian are both explicit. I implemented and committed the first alternative above (as a new attribute, 'bytes_le'), shortly before the freeze. -- ?!ng From thomas at python.org Sun Aug 20 08:05:39 2006 From: thomas at python.org (Thomas Wouters) Date: Sun, 20 Aug 2006 08:05:39 +0200 Subject: [Python-Dev] ctypes and win64 In-Reply-To: <20060819144042.GK5772@performancedrivers.com> References: <44E60B2A.6020101@v.loewis.de> <44E60D00.7090304@python.net> <44E635A2.3020000@v.loewis.de> <44E6A6E8.3000101@v.loewis.de> <1f7befae0608190219m5429d505i8a155e943e76d3fd@mail.gmail.com> <20060819144042.GK5772@performancedrivers.com> Message-ID: <9e804ac0608192305t56fdcf8fl3a3abe37ac296df5@mail.gmail.com> On 8/19/06, Jack Diederich wrote: > It has always "just worked" for me on Opterons + Debian. > Python 2.4 (#1, May 31 2005, 10:19:45) > [GCC 3.3.5 (Debian 1:3.3.5-12)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import sys > >>> sys.maxint > 9223372036854775807 While that's certainly useful behaviour, it isn't exactly the same thing as the 64-bit support in Python 2.5. 64-bit longs have always worked (as Tim said, CPython is good C), but large parts of CPython were using ints instead of longs -- and I'm fairly certain there are still quite a few bugs with container types and more than 31-bits worth of elements. (I say that because I found more than I hoped for, writing the bigmem tests a few months back. And those tests only consider bytestrings, lists and tuples.) So, while sys.maxint doesn't change, and any container with sane amounts of elements will almost certainly work (one would hope that's tested enough by now), real-world code that actually uses, say, more than 100Gb worth of memory in a single container may still break. Unless we write more bigmem tests :> Luxury-problem-anyone?'ly y'rs, -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060820/401fd8ee/attachment.htm From fredrik at pythonware.com Sun Aug 20 10:46:55 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sun, 20 Aug 2006 10:46:55 +0200 Subject: [Python-Dev] ctypes and win64 In-Reply-To: <44E6A6E8.3000101@v.loewis.de> References: <44E60B2A.6020101@v.loewis.de> <44E60D00.7090304@python.net> <44E635A2.3020000@v.loewis.de> <44E6A6E8.3000101@v.loewis.de> Message-ID: Martin v. L?wis wrote: > It isn't. Python ran on 64-bit Alpha for nearly a decade now (I guess) make that "over a decade". the first Python system I built was on tru64, back in 1995 (portions of the the initial prototype was written on a 286 box under MS-DOS, but the bulk was developed on tru64 and deployed on tru64...) From jjl at pobox.com Sun Aug 20 14:45:08 2006 From: jjl at pobox.com (John J Lee) Date: Sun, 20 Aug 2006 12:45:08 +0000 (UTC) Subject: [Python-Dev] String formatting / unicode 2.5 bug? In-Reply-To: <44E7ED19.6090907@gmail.com> References: <44E7ED19.6090907@gmail.com> Message-ID: On Sun, 20 Aug 2006, Nick Coghlan wrote: > John J Lee wrote: >> Is this a bug? > > I don't believe so - the string formatting documentation states that the > result will be unicode if either the format string is unicode or any of the > objects passed to a %s format code is unicode. > > That latter part has just been extended to include any object that returns > Unicode from __str__, instead of being restricted to actual Unicode > instances. > > Note that the following behaves the same way regardless of whether you use > 2.4 or 2.5: > "%s" % 'hi' > "%s" % u'hi' Given that, the following wording should be changed: http://docs.python.org/lib/typesseq-strings.html Conversion Meaning Notes ... s String (converts any python object using str()). (4) ... (4) If the object or format provided is a unicode string, the resulting string will also be unicode. The note (4) says that the result will be unicode, but it doesn't say how, in this case, that comes about. This case is confusing because the docs claim string formatting with %s "converts ... using str()", and yet str(a()) returns a bytestring. Does it *really* use str, or just __str__? Surely the latter? (given the observed behaviour, and not reading the C source) FWIW, this change broke epydoc (fails with an AssertionError -- so perhaps without the assert it would still "work", dunno). > And once the result has been promoted to unicode, __unicode__ is used > directly: > >> > > print repr("%s%s" % (a(), a())) > __str__ > accessing <__main__.a object at 0x00AF66F0>.__unicode__ > __str__ > accessing <__main__.a object at 0x00AF6390>.__unicode__ > __str__ > u'hihi' I don't understand this part. Why is __unicode__ called? Your example doesn't appear to show this happening "once [i.e., because?] the result has been promoted to unicode" -- if that were true, it would "stand to reason" that the interpreter would then conclude it should call __unicode__ for all remaining %s, and not bother with __str__. If OTOH __unicode__ is called because __str__ returned a unicode object, it makes (very slightly) more sense that it goes through the same __str__-then-__unicode__ rigmarole for each object on the RHS of the %. But none of that seems to make a huge amount of sense. I've now found the September 2004 discussion of this, and I'm none the wiser. John From guido at python.org Sun Aug 20 17:24:16 2006 From: guido at python.org (Guido van Rossum) Date: Sun, 20 Aug 2006 08:24:16 -0700 Subject: [Python-Dev] What should the focus for 2.6 be? Message-ID: I've been thinking a bit about a focus for the 2.6 release. We are now officially starting parallel development of 2.6 and 3.0. I really don't expect that we'll be able to merge the easily into the 3.0 branch much longer, so effectively 3.0 will be a fork of 2.5. I wonder if it would make sense to focus in 2.6 on making porting of 2.6 code to 3.0 easier, rather than trying to introduce new features in 2.6. We've done releases without new language features before; notable 2.3 didn't add anything new (except making a few __future__ imports redundant) and concentrated on bugfixes, performance, and library additions. Some projects that could be undertaken in 2.6: - add warnings when apply() is used - add warnings when string exceptions or non-BaseException-derived exceptions are used (this is already planned in PEP 252, which has a very specific roll-out plan) - add warnings when has_key() is used - add warnings when the result of dict.keys(), .values(), .items() is used for anything else than iterating over it - a warning if a class defines has_key() but not __contains__(). - add __contains__ to dbm and gdbm - add warnings to modules and built-ins that will die in 3.0 Some of these warnings should be suppressed by default, but enabled by a command line option. We should also do the work on the standard library to avoid the warnings: get rid of apply(), use 'x in d' instead of 'd.has_key(x)', etc. I've recently done some of this work in the 3.0 branch (e.g. dbm/gdbm are fresh in my memory). Another area that could use a lot of work (and from which 3.0 could also benefit directly) is converting all unit tests to using either unittest.py or doctest.py. There are still at least 40 tests written in the old "compare the output with one we prepared earlier" style. Of course, if people are chomping at the bit to implement certain new features (and those features are generally approved of) then I don't want to stop them; but I would recommend that our effort may better be focused on smoothing the 2.6/3.0 transition rather than looking for new features to add to 2.6. I am often approached by people who object against this or that feature proposal not because they dislike the proposed feature in particular, but because they believe Python is already large enough, and they worry that if we keep adding features at the current pace, it will soon become too unwieldy, and hence harder to learn or to keep in one's brain. I am very sympathetic to this concern (and you should be too!). This is one of the reasons that so much of the Python 3000 work is about ripping out old stuff and simplifying / unifiying things. Dropping two common data types (long and unicode -- of course they will really be merged into their simpler counterparts int and str, but it means much less to learn/remember) is one example. Ripping out classic classes and string exceptions another. Removing dead library modules a third. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Sun Aug 20 17:32:35 2006 From: barry at python.org (Barry Warsaw) Date: Sun, 20 Aug 2006 11:32:35 -0400 Subject: [Python-Dev] What should the focus for 2.6 be? In-Reply-To: References: Message-ID: <692A12F5-78C0-43BB-8041-C4CDAF4405B6@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 20, 2006, at 11:24 AM, Guido van Rossum wrote: > I wonder if it would make sense to focus in 2.6 on making porting of > 2.6 code to 3.0 easier, rather than trying to introduce new features > in 2.6. We've done releases without new language features before; > notable 2.3 didn't add anything new (except making a few __future__ > imports redundant) and concentrated on bugfixes, performance, and > library additions. +1, and there are other benefits to this approach too. First, the pace of change appears to slow, which addresses another source of complaints. Because instead of a slew of new features every 18 months, we really see that slew only every three years, with a stabilizing and bug fixing release in between. Another benefit is that with a de-emphasis on new features, we can spend more time improving the library and documentation. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBROiAk3EjvBPtnXfVAQKeMQP/QBfHJexDveKoj/nfjRjic3+HBvTupkoA bga7GmLV7Rn14AoHf+L6n3IhKkE1sIVXwzUmEoOeVN74h3trJSAeEYPjCF7Vt3// 3fZ4SgAlEy3nsOwRYufUtyYU9r36H7Fn7dKTtj+hJCVAzZdAOERy8ZMAEoSOw+Q4 vNfudLPznDQ= =7D1V -----END PGP SIGNATURE----- From rganesan at myrealbox.com Sun Aug 20 17:35:11 2006 From: rganesan at myrealbox.com (Ganesan Rajagopal) Date: Sun, 20 Aug 2006 21:05:11 +0530 Subject: [Python-Dev] ctypes and win64 References: <44E60B2A.6020101@v.loewis.de> <44E60D00.7090304@python.net> <44E635A2.3020000@v.loewis.de> <44E6A6E8.3000101@v.loewis.de> Message-ID: >>>>> Fredrik Lundh writes: > Martin v. L?wis wrote: >> It isn't. Python ran on 64-bit Alpha for nearly a decade now (I guess) > make that "over a decade". the first Python system I built was on > tru64, back in 1995 (portions of the the initial prototype was written > on a 286 box under MS-DOS, but the bulk was developed on tru64 and > deployed on tru64...) Well, there's 64-bit support and then there's 64-bit support. While all Unix or Unix like OSs follow LP64 model, I believe Win64 follows a P64 model. So it's a whole new ball game. Ganesan -- Ganesan Rajagopal From nas at arctrix.com Sun Aug 20 18:39:29 2006 From: nas at arctrix.com (Neil Schemenauer) Date: Sun, 20 Aug 2006 16:39:29 +0000 (UTC) Subject: [Python-Dev] String formatting / unicode 2.5 bug? References: <44E7ED19.6090907@gmail.com> Message-ID: John J Lee wrote: > The note (4) says that the result will be unicode, but it doesn't say how, > in this case, that comes about. This case is confusing because the docs > claim string formatting with %s "converts ... using str()", and yet > str(a()) returns a bytestring. Does it *really* use str, or just __str__? > Surely the latter? (given the observed behaviour, and not reading the C > source) It uses __str__ and confirms that the returned object is a 'str' or 'unicode'. The docs are not precise but they were not for 2.4 either. Note the following case: '%s' % u'Hello!' The operand is not forced to be a str. Neil From jjl at pobox.com Sun Aug 20 18:55:09 2006 From: jjl at pobox.com (John J Lee) Date: Sun, 20 Aug 2006 16:55:09 +0000 (UTC) Subject: [Python-Dev] String formatting / unicode 2.5 bug? In-Reply-To: References: <44E7ED19.6090907@gmail.com> Message-ID: On Sun, 20 Aug 2006, Neil Schemenauer wrote: > John J Lee wrote: >> The note (4) says that the result will be unicode, but it doesn't say how, >> in this case, that comes about. This case is confusing because the docs >> claim string formatting with %s "converts ... using str()", and yet >> str(a()) returns a bytestring. Does it *really* use str, or just __str__? >> Surely the latter? (given the observed behaviour, and not reading the C >> source) > > It uses __str__ and confirms that the returned object is a 'str' or > 'unicode'. The docs are not precise but they were not for 2.4 > either. Note the following case: [...] OK, but I assume you're not saying that the fact that the docs were broken in 2.4 implies they shouldn't be fixed now? I would suggest revised wording, but I'm clearly confused about what actually goes on under the hood... John From aleaxit at gmail.com Mon Aug 21 00:08:11 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Sun, 20 Aug 2006 15:08:11 -0700 Subject: [Python-Dev] ctypes and win64 In-Reply-To: <44E6E7BD.8000903@holdenweb.com> References: <44E60B2A.6020101@v.loewis.de> <44E60D00.7090304@python.net> <44E635A2.3020000@v.loewis.de> <44E6A6E8.3000101@v.loewis.de> <1f7befae0608190219m5429d505i8a155e943e76d3fd@mail.gmail.com> <44E6E7BD.8000903@holdenweb.com> Message-ID: On Aug 19, 2006, at 3:28 AM, Steve Holden wrote: ... > It's going to be very interesting to see what comes out of the Google > sprints. I am sure the 64-bitters will be out in force, so there'll be Hmmm, we'll be working on our laptops, as is typical of sprints, so I'm not sure how many 64-bit machines will in fact be around -- we'll see. > useful data about any such problems. 64 bits is, > I am > sure, as much as anyone is ever going to need, so the pain will > finally > be over. > > It's good to see the idea of industry support for open source sprints > taking off. Tomorrow, the world ... :-) Sprints are indeed a fascinating idea and have proven they work, in an open-source context -- I do wonder if they could be made to work in other contexts, and I'm sure many others are wondering similarly. Alex From thomas at python.org Mon Aug 21 03:31:45 2006 From: thomas at python.org (Thomas Wouters) Date: Mon, 21 Aug 2006 03:31:45 +0200 Subject: [Python-Dev] ctypes and win64 In-Reply-To: References: <44E60B2A.6020101@v.loewis.de> <44E60D00.7090304@python.net> <44E635A2.3020000@v.loewis.de> <44E6A6E8.3000101@v.loewis.de> <1f7befae0608190219m5429d505i8a155e943e76d3fd@mail.gmail.com> <44E6E7BD.8000903@holdenweb.com> Message-ID: <9e804ac0608201831q2b1f48e0qcb4164fd8ab927dd@mail.gmail.com> On 8/21/06, Alex Martelli wrote: > > > On Aug 19, 2006, at 3:28 AM, Steve Holden wrote: > ... > > It's going to be very interesting to see what comes out of the Google > > sprints. I am sure the 64-bitters will be out in force, so there'll be > > Hmmm, we'll be working on our laptops, as is typical of sprints, so > I'm not sure how many 64-bit machines will in fact be around -- we'll > see. > FWIW, I have access to a pair of AMD64 machines with 16Gb RAM and several hundred Gb swap. I can't give anyone else access, but I can run tests and debug. > Sprints are indeed a fascinating idea and have proven they work, in > an open-source context -- I do wonder if they could be made to work > in other contexts, and I'm sure many others are wondering similarly. > We've had commercial, private "sprints" at XS4ALL for years, every 6 months or so, until last year. We didn't call them sprints and they obviously weren't as freeform as the Python sprints, but the concept was the same (as well as the atmosphere.) We stopped having them because the number of concurrent projects was consistently larger than the number of programmers, which makes the sprinting setup somewhat useless. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060821/8571fe76/attachment-0001.htm From alexander.belopolsky at gmail.com Mon Aug 21 06:55:07 2006 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Mon, 21 Aug 2006 00:55:07 -0400 Subject: [Python-Dev] A cast from Py_ssize_t to long In-Reply-To: <44E174EB.1080305@v.loewis.de> References: <44E174EB.1080305@v.loewis.de> Message-ID: <6A93E89B-F2F6-4D5B-AACB-2EDC09A1D2CB@local> On Aug 15, 2006, at 3:16 AM, Martin v. L?wis wrote: > > Where does it assume that it is safe to case ssize_t -> long? > That would be a bug. Is this a bug? file_readinto(PyFileObject *f, PyObject *args) { ... Py_ssize_t ndone, nnow; ... return PyInt_FromLong((long)ndone); } See Objects/fileobject.c (revision 51420). From fredrik at pythonware.com Mon Aug 21 08:17:32 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Mon, 21 Aug 2006 08:17:32 +0200 Subject: [Python-Dev] ctypes and win64 In-Reply-To: References: <44E60B2A.6020101@v.loewis.de> <44E60D00.7090304@python.net> <44E635A2.3020000@v.loewis.de> <44E6A6E8.3000101@v.loewis.de> <1f7befae0608190219m5429d505i8a155e943e76d3fd@mail.gmail.com> <44E6E7BD.8000903@holdenweb.com> Message-ID: Alex Martelli wrote: > Sprints are indeed a fascinating idea and have proven they work, in > an open-source context -- I do wonder if they could be made to work > in other contexts, and I'm sure many others are wondering similarly. "war room" development and other colocation approaches are nothing new, and works very well in many contexts. From talin at acm.org Mon Aug 21 08:20:34 2006 From: talin at acm.org (Talin) Date: Sun, 20 Aug 2006 23:20:34 -0700 Subject: [Python-Dev] What should the focus for 2.6 be? In-Reply-To: References: Message-ID: <44E950B2.4060305@acm.org> Guido van Rossum wrote: > I've been thinking a bit about a focus for the 2.6 release. > > We are now officially starting parallel development of 2.6 and 3.0. I > really don't expect that we'll be able to merge the easily into the > 3.0 branch much longer, so effectively 3.0 will be a fork of 2.5. > > I wonder if it would make sense to focus in 2.6 on making porting of > 2.6 code to 3.0 easier, rather than trying to introduce new features > in 2.6. We've done releases without new language features before; > notable 2.3 didn't add anything new (except making a few __future__ > imports redundant) and concentrated on bugfixes, performance, and > library additions. I've been thinking about the transition to unicode strings, and I want to put forward a notion that might allow the transition to be done gradually instead of all at once. The idea would be to temporarily introduce a new name for 8-bit strings - let's call it "ascii". An "ascii" object would be exactly the same as today's 8-bit strings. The 'str' builtin symbol would be assigned to 'ascii' by default, but you could assign it to 'unicode' if you wanted to default to wide strings: str = ascii # Selects 8-bit strings by default str = unicode # Selects unicode strings by default In order to make the transition, what you would do is to temporarily undefine the 'str' symbol from the code base - in other words, remove 'str' from the builtin namespace, and then migrate all of the code -- replacing any library reference to 'str' with a reference to 'ascii' *or* updating that function to deal with unicode strings. Once you get all of the unit tests running again, you can re-introduce 'str', but now you know that since none of the libraries refer to 'str' directly, you can safely change its definition. All of this could be done while retaining compatibility with existing 3rd party code - as long as 'str = ascii' is defined. So you turn it on to run your Python programs, and turn it off when you want to work on 3.0 migration. The next step (which would not be backwards compatible) would be to gradually remove 'ascii' from the code base -- wherever that name occurs, it would be a signal that the function needs to be updated to use 'unicode' instead. Finally, once the last occurance of 'ascii' is removed, the final step is to do a search and replace of all occurances of 'unicode' with 'str'. I know this seems round-about, and is more work than doing it all in one shot. However, I know from past experience that the trickiest part of doing a pervasive change to a code base like this is just keeping track of what parts have been migrated and what parts have not. Many times in the past I've changed the definition of a ubiquitous type by temporarily renaming it, thus vacating the old name so that it can be defined anew, without conflict. -- Talin From g.brandl at gmx.net Mon Aug 21 10:56:33 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 21 Aug 2006 10:56:33 +0200 Subject: [Python-Dev] What should the focus for 2.6 be? In-Reply-To: <44E950B2.4060305@acm.org> References: <44E950B2.4060305@acm.org> Message-ID: Talin wrote: > Guido van Rossum wrote: >> I've been thinking a bit about a focus for the 2.6 release. >> >> We are now officially starting parallel development of 2.6 and 3.0. I >> really don't expect that we'll be able to merge the easily into the >> 3.0 branch much longer, so effectively 3.0 will be a fork of 2.5. >> >> I wonder if it would make sense to focus in 2.6 on making porting of >> 2.6 code to 3.0 easier, rather than trying to introduce new features >> in 2.6. We've done releases without new language features before; >> notable 2.3 didn't add anything new (except making a few __future__ >> imports redundant) and concentrated on bugfixes, performance, and >> library additions. > > I've been thinking about the transition to unicode strings, and I want > to put forward a notion that might allow the transition to be done > gradually instead of all at once. > > The idea would be to temporarily introduce a new name for 8-bit strings > - let's call it "ascii". An "ascii" object would be exactly the same as > today's 8-bit strings. > > The 'str' builtin symbol would be assigned to 'ascii' by default, but > you could assign it to 'unicode' if you wanted to default to wide strings: > > str = ascii # Selects 8-bit strings by default > str = unicode # Selects unicode strings by default This doesn't change the type of string literals. Georg From ncoghlan at gmail.com Mon Aug 21 11:37:13 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 21 Aug 2006 19:37:13 +1000 Subject: [Python-Dev] String formatting / unicode 2.5 bug? In-Reply-To: References: <44E7ED19.6090907@gmail.com> Message-ID: <44E97EC9.4040502@gmail.com> John J Lee wrote: >> And once the result has been promoted to unicode, __unicode__ is used >> directly: >> >>>>> print repr("%s%s" % (a(), a())) >> __str__ >> accessing <__main__.a object at 0x00AF66F0>.__unicode__ >> __str__ >> accessing <__main__.a object at 0x00AF6390>.__unicode__ >> __str__ >> u'hihi' > > I don't understand this part. Why is __unicode__ called? Your example > doesn't appear to show this happening "once [i.e., because?] the result > has been promoted to unicode" -- if that were true, it would "stand to > reason" that the interpreter would then conclude it should call > __unicode__ for all remaining %s, and not bother with __str__. It does try to call unicode directly, but because the example object doesn't supply __unicode__ it ends up falling back to __str__ instead. The behaviour is clearer when the example object provides both methods: >>> # Example (2.5b3) ... class a(object): ... def __str__(self): ... print "running __str__" ... return u'hi' ... def __unicode__(self): ... print "running __unicode__" ... return u'hi' ... >>> print repr("%s%s" % (a(), a())) running __str__ running __unicode__ running __unicode__ u'hihi' Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From thomas at python.org Mon Aug 21 13:23:46 2006 From: thomas at python.org (Thomas Wouters) Date: Mon, 21 Aug 2006 13:23:46 +0200 Subject: [Python-Dev] A cast from Py_ssize_t to long In-Reply-To: <6A93E89B-F2F6-4D5B-AACB-2EDC09A1D2CB@local> References: <44E174EB.1080305@v.loewis.de> <6A93E89B-F2F6-4D5B-AACB-2EDC09A1D2CB@local> Message-ID: <9e804ac0608210423r22541508q35f54a444b2ffab9@mail.gmail.com> On 8/21/06, Alexander Belopolsky wrote: > > > On Aug 15, 2006, at 3:16 AM, Martin v. L?wis wrote: > > > > > Where does it assume that it is safe to case ssize_t -> long? > > That would be a bug. > > Is this a bug? > > file_readinto(PyFileObject *f, PyObject *args) > { > ... > Py_ssize_t ndone, nnow; > ... > return PyInt_FromLong((long)ndone); > } > > See Objects/fileobject.c (revision 51420). Yes. Py_ssize_t can be bigger than a long (on LLP64 systems, such as Win64). It doesn't matter on other systems, and you have to read more than 31 bits worth of data to detect it even on Win64, but it's still a bug. file_readinto should be using PyInt_FromSsize_t() instead. (There is the SAFE_DOWNCAST macro for cases where we know the *value* of the (s)size_t will always fit in a long, on any supported system, but that isn't the case here.) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060821/4e09574e/attachment.htm From jimjjewett at gmail.com Mon Aug 21 16:03:46 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 21 Aug 2006 10:03:46 -0400 Subject: [Python-Dev] Questions on unittest behaviour Message-ID: Collin Winter: > [ improvements to unittest] These would all have been (mostly) reasonable when the module was first added to the standard library. But it has been there as an ugly fragile unchanging beast for several years, and is ironically not well endowed with unit tests. I therefore think it might be too late for these changes. > 2) The TestLoader.testMethodPrefix attribute currently allows anything > to be assigned to it, including invalid objects and the empty string. > While the former will cause errors ... you get things like assertEqual(), > failUnlessEqual(), etc, when TestLoader.loadTestsFromTestCase() is run. It makes the interface ugly, and learning harder, but for code this old, either people catch it with the first actual run of their tests suites, or a subclass is handling it (perhaps to trigger a default, or to call functions through a foreign interface). > 3) TestLoader.loadTestsFromTestCase() accepts objects that are not > test cases and will happily look for appropriately-named methods on > any object you give it. This flexibility should be documented, or > proper input validation should be done (a bug fix for 2.5). People are probably using this, so I think a deprecation in 2.6 is the fastest it should change. > 4) TestLoader.loadTestsFromName() (and by extension, > loadTestsFromNames(), too) raises an AttributeError if the name is the > empty string because -- as it correctly asserts -- the object does not > contain an attribute named ''. I recommend that this be tested for and > ValueError be raised (bug fix for 2.5). This seems reasonable for new code, but not this late in 2.5 -- if people are going through an auto-generated list of attributes, they may well be catching AttributeError and not ValueError. > 5) When TestLoader.loadTestsFrom{Name,Names}() are given a name that > resolves to a classmethod on a TestCase subclass, the method is not > invoked. ... > It is not documented which of these tests takes priority: is the > classmethod "a test method within a test case class" or is it a > callable? The same issue applies to staticmethods as well. Documentation probably should get added. -jJ From alexander.belopolsky at gmail.com Mon Aug 21 16:08:02 2006 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Mon, 21 Aug 2006 10:08:02 -0400 Subject: [Python-Dev] A cast from Py_ssize_t to long In-Reply-To: <9e804ac0608210423r22541508q35f54a444b2ffab9@mail.gmail.com> References: <44E174EB.1080305@v.loewis.de> <6A93E89B-F2F6-4D5B-AACB-2EDC09A1D2CB@local> <9e804ac0608210423r22541508q35f54a444b2ffab9@mail.gmail.com> Message-ID: On 8/21/06, Thomas Wouters wrote: [snip] > > Is this a bug? [snap] > Yes. Py_ssize_t can be bigger than a long (on LLP64 systems, such as Win64). Here is a similar problem: typedef struct { ... size_t pos; ... } mmap_object; ... mmap_tell_method(mmap_object *self, PyObject *unused) { CHECK_VALID(NULL); return PyInt_FromLong((long) self->pos); } See Modules/mmapmodule.c . Here a cast to ssize_t would, technically speaking, not be safe either, but it may be worth using ssize_t anyways. Is there a simple automated way to detect situations like this? Maybe there is a win64 compiler that would generate a warning. From thomas at python.org Mon Aug 21 16:34:25 2006 From: thomas at python.org (Thomas Wouters) Date: Mon, 21 Aug 2006 16:34:25 +0200 Subject: [Python-Dev] A cast from Py_ssize_t to long In-Reply-To: References: <44E174EB.1080305@v.loewis.de> <6A93E89B-F2F6-4D5B-AACB-2EDC09A1D2CB@local> <9e804ac0608210423r22541508q35f54a444b2ffab9@mail.gmail.com> Message-ID: <9e804ac0608210734m7786148ak34e3610d1523f8d0@mail.gmail.com> On 8/21/06, Alexander Belopolsky wrote: > Here is a similar problem: > typedef struct { > ... > size_t pos; > ... > } mmap_object; > ... > mmap_tell_method(mmap_object *self, PyObject *unused) > { > CHECK_VALID(NULL); > return PyInt_FromLong((long) self->pos); > } > > See Modules/mmapmodule.c . > > Here a cast to ssize_t would, technically speaking, not be safe > either, but it may be worth using ssize_t anyways. It should call PyInt_FromSize_t, without any casting. That will make it a PyLong if it's bigger than a Py_ssize_t, too. Is there a simple automated way to detect situations like this? Maybe > there is a win64 compiler that would generate a warning. > I doubt it. Explicit casts are meant to silence warnings (among other things.) Warning for all casts is bound to generate quite a lot of warnings. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060821/3f0f7968/attachment.htm From alexander.belopolsky at gmail.com Mon Aug 21 16:47:35 2006 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Mon, 21 Aug 2006 10:47:35 -0400 Subject: [Python-Dev] A cast from Py_ssize_t to long In-Reply-To: <9e804ac0608210734m7786148ak34e3610d1523f8d0@mail.gmail.com> References: <44E174EB.1080305@v.loewis.de> <6A93E89B-F2F6-4D5B-AACB-2EDC09A1D2CB@local> <9e804ac0608210423r22541508q35f54a444b2ffab9@mail.gmail.com> <9e804ac0608210734m7786148ak34e3610d1523f8d0@mail.gmail.com> Message-ID: On Aug 21, 2006, at 10:34 AM, Thomas Wouters wrote: > > Is there a simple automated way to detect situations like this? Maybe > there is a win64 compiler that would generate a warning. > > I doubt it. Explicit casts are meant to silence warnings (among > other things.) Warning for all casts is bound to generate quite a > lot of warnings. > There are also cases of implicit casts like this that were not caught so far: static Py_ssize_t mmap_buffer_getreadbuf(mmap_object *self, Py_ssize_t index, const void **ptr) { ... return self->size; } static Py_ssize_t mmap_buffer_getwritebuf(mmap_object *self, Py_ssize_t index, const void **ptr) { ... return self->size; } I don't have any system with sizeof(size_t) != sizeof(long), but it maybe worth the effort to review the warnings on such system. From thomas at python.org Mon Aug 21 16:56:23 2006 From: thomas at python.org (Thomas Wouters) Date: Mon, 21 Aug 2006 16:56:23 +0200 Subject: [Python-Dev] A cast from Py_ssize_t to long In-Reply-To: References: <44E174EB.1080305@v.loewis.de> <6A93E89B-F2F6-4D5B-AACB-2EDC09A1D2CB@local> <9e804ac0608210423r22541508q35f54a444b2ffab9@mail.gmail.com> <9e804ac0608210734m7786148ak34e3610d1523f8d0@mail.gmail.com> Message-ID: <9e804ac0608210756r1d9806efo84ed12a9848c1644@mail.gmail.com> On 8/21/06, Alexander Belopolsky wrote: > There are also cases of implicit casts like this that were not > caught so far: > > static Py_ssize_t > mmap_buffer_getreadbuf(mmap_object *self, Py_ssize_t index, const > void **ptr) > { > ... > return self->size; > } > > static Py_ssize_t > mmap_buffer_getwritebuf(mmap_object *self, Py_ssize_t index, const > void **ptr) > { > ... > return self->size; > } > > I don't have any system with sizeof(size_t) != sizeof(long), but it > maybe worth the effort to review the warnings on such system. GCC on a LP64 machine does not generate warnings for the above code. It doesn't have anything to do with 64-bit or 32-bit anyway, since Py_ssize_t and size_t are supposed to be the same size. They just different in signedness, and that's the case on any system. Even if those functions were defined to return longs instead of Py_ssize_t's, GCC wouldn't generate a warning; it's a valid (implicit) downcast that might lose bits. I believe there's a Windows compiler that goes warn for such cases, but if so, I'm sure it generates a whole lot of spurious ones at the moment. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060821/f6bef468/attachment.htm From guido at python.org Mon Aug 21 20:11:50 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 21 Aug 2006 11:11:50 -0700 Subject: [Python-Dev] What should the focus for 2.6 be? In-Reply-To: <44E950B2.4060305@acm.org> References: <44E950B2.4060305@acm.org> Message-ID: I'll keep this in mind -- with the caveat that Georg mentioned. For the next 96 hours I'm going to be severely limited in bandwidth due to the physical requirements of the sprint at Google. I'd appreciate not receiving too much email during this period... --Guido On 8/20/06, Talin wrote: > Guido van Rossum wrote: > > I've been thinking a bit about a focus for the 2.6 release. > > > > We are now officially starting parallel development of 2.6 and 3.0. I > > really don't expect that we'll be able to merge the easily into the > > 3.0 branch much longer, so effectively 3.0 will be a fork of 2.5. > > > > I wonder if it would make sense to focus in 2.6 on making porting of > > 2.6 code to 3.0 easier, rather than trying to introduce new features > > in 2.6. We've done releases without new language features before; > > notable 2.3 didn't add anything new (except making a few __future__ > > imports redundant) and concentrated on bugfixes, performance, and > > library additions. > > I've been thinking about the transition to unicode strings, and I want > to put forward a notion that might allow the transition to be done > gradually instead of all at once. > > The idea would be to temporarily introduce a new name for 8-bit strings > - let's call it "ascii". An "ascii" object would be exactly the same as > today's 8-bit strings. > > The 'str' builtin symbol would be assigned to 'ascii' by default, but > you could assign it to 'unicode' if you wanted to default to wide strings: > > str = ascii # Selects 8-bit strings by default > str = unicode # Selects unicode strings by default > > In order to make the transition, what you would do is to temporarily > undefine the 'str' symbol from the code base - in other words, remove > 'str' from the builtin namespace, and then migrate all of the code -- > replacing any library reference to 'str' with a reference to 'ascii' > *or* updating that function to deal with unicode strings. Once you get > all of the unit tests running again, you can re-introduce 'str', but now > you know that since none of the libraries refer to 'str' directly, you > can safely change its definition. > > All of this could be done while retaining compatibility with existing > 3rd party code - as long as 'str = ascii' is defined. So you turn it on > to run your Python programs, and turn it off when you want to work on > 3.0 migration. > > The next step (which would not be backwards compatible) would be to > gradually remove 'ascii' from the code base -- wherever that name > occurs, it would be a signal that the function needs to be updated to > use 'unicode' instead. > > Finally, once the last occurance of 'ascii' is removed, the final step > is to do a search and replace of all occurances of 'unicode' with 'str'. > > I know this seems round-about, and is more work than doing it all in one > shot. However, I know from past experience that the trickiest part of > doing a pervasive change to a code base like this is just keeping track > of what parts have been migrated and what parts have not. Many times in > the past I've changed the definition of a ubiquitous type by temporarily > renaming it, thus vacating the old name so that it can be defined anew, > without conflict. > > -- Talin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Mon Aug 21 21:24:54 2006 From: brett at python.org (Brett Cannon) Date: Mon, 21 Aug 2006 12:24:54 -0700 Subject: [Python-Dev] What should the focus for 2.6 be? In-Reply-To: <692A12F5-78C0-43BB-8041-C4CDAF4405B6@python.org> References: <692A12F5-78C0-43BB-8041-C4CDAF4405B6@python.org> Message-ID: On 8/20/06, Barry Warsaw wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Aug 20, 2006, at 11:24 AM, Guido van Rossum wrote: > > > I wonder if it would make sense to focus in 2.6 on making porting of > > 2.6 code to 3.0 easier, rather than trying to introduce new features > > in 2.6. We've done releases without new language features before; > > notable 2.3 didn't add anything new (except making a few __future__ > > imports redundant) and concentrated on bugfixes, performance, and > > library additions. > > +1, and there are other benefits to this approach too. > > First, the pace of change appears to slow, which addresses another > source of complaints. Because instead of a slew of new features > every 18 months, we really see that slew only every three years, with > a stabilizing and bug fixing release in between. Another benefit is > that with a de-emphasis on new features, we can spend more time > improving the library and documentation. I think fixing tests and documentation would be a great thing to focus 2.6on. Not glamourous, I know, but it is needed. For tests, I hope to get some decorators and such written that will help classify tests. Also adding a function to denote what module is being tested would be good (to avoid the issue of a dependent import for testing failing and then everyone just thinking the test was skipped). Lastly, testing the C API using ctypes would be really good since it is not thorougly tested. As for the docs, they just need a thorough updating. As to whether we should come up with some other format for Py3K with better semantic information and that is easier to read is another question entirely. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060821/7ee49990/attachment.htm From jcarlson at uci.edu Mon Aug 21 23:21:30 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Mon, 21 Aug 2006 14:21:30 -0700 Subject: [Python-Dev] What should the focus for 2.6 be? In-Reply-To: <44E950B2.4060305@acm.org> References: <44E950B2.4060305@acm.org> Message-ID: <20060821081944.1A0F.JCARLSON@uci.edu> Talin wrote: [snip] > I've been thinking about the transition to unicode strings, and I want > to put forward a notion that might allow the transition to be done > gradually instead of all at once. > > The idea would be to temporarily introduce a new name for 8-bit strings > - let's call it "ascii". An "ascii" object would be exactly the same as > today's 8-bit strings. There are two parts to the unicode conversion; all literals are unicode, and we don't have strings anymore, we have bytes. Without offering the bytes object, then people can't really convert their code. String literals can be handled with the -U command line option (and perhaps having the interpreter do the str=unicode assignment during startup). In any case, as I look at Py3k and the future of Python, in each release, I ask "what are the compelling features that make me want to upgrade?" In each of the 1.5-2.5 series that I've looked at, each has had some compelling feature or another that has basically required that I upgrade, or seriously consider upgrading (bugfixes for stuff that has bitten me, new syntax that I use, significant increases in speed, etc.) . As we approach Py3k, I again ask, "what are the compelling features?" Wholesale breakage of anything that uses ascii strings as text or binary data? A completely changed IO stack (requiring re-learning of everything known about Python IO)? Dictionary .keys(), .values(), and .items() being their .iter*() equivalents (making it just about impossible to optimize for Py3k dictionary behavior now)? I understand getting rid of the cruft, really I do (you should see some cruft I've been replacing lately). But some of that cruft is useful, or really, some of that cruft has no alternative currently, which will require significant rewrites of user code when Py3k is released. When everyone has to rewrite their code, they are going to ask, "Why don't I just stick with the maintenance 2.x? It's going to be maintained for a few more years yet, and I don't need to rewrite all of my disk IO, strings in dictionary code, etc. I will be right along with them (no offense intended to those currently working towards py3k). I can code defensively against buffer-sturating DOS attacks with my socket code, but I can't code defensively to handle some (never mind all) of the changes and incompatabilities that Py3k will bring. Here's my suggestion: every feature, syntax, etc., that is slated for Py3k, let us release bit by bit in the 2.x series. That lets the 2.x series evolve into the 3.x series in a somewhat more natural way than the currently proposed *everything breaks*. If it takes 1, 2, 3, or 10 more releases in the 2.x series to get to all of the 3.x features, great. At least people will have a chance to convert, or at least write correct code for the future. Say 2.6 gets bytes and special factories (or a special encoding argument) for file/socket to return bytes instead of strings, and only accept bytes objects to .write() methods (unless an encoding on the file, etc., was previously given). Given these bytes objects, it may even make sense to offer the .readinto() method that Alex B has been asking for (which would make 3 built-in objects that could reasonably support readinto: bytes, array, mmap). If the IO library is available for 2.6, toss that in there, or offer it in PyPI as an evolving library. I would suggest pushing off the dict changes until 2.7 or later, as there are 340+ examples of dict.keys() in the Python 2.5b2 standard library, at least half of which are going to need to be changed to list(dict.keys()) or otherwise. The breakage in user code will likely be at least as substantial. Those are just examples that come to mind now, but I'm sure there are others changes with similar issues. - Josiah From exarkun at divmod.com Mon Aug 21 23:38:17 2006 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Mon, 21 Aug 2006 17:38:17 -0400 Subject: [Python-Dev] [Python-3000] What should the focus for 2.6 be? In-Reply-To: <20060821081944.1A0F.JCARLSON@uci.edu> Message-ID: <20060821213817.1717.1725966885.divmod.quotient.28023@ohm> On Mon, 21 Aug 2006 14:21:30 -0700, Josiah Carlson wrote: > >Talin wrote: >[snip] >> I've been thinking about the transition to unicode strings, and I want >> to put forward a notion that might allow the transition to be done >> gradually instead of all at once. >> >> The idea would be to temporarily introduce a new name for 8-bit strings >> - let's call it "ascii". An "ascii" object would be exactly the same as >> today's 8-bit strings. > >There are two parts to the unicode conversion; all literals are unicode, >and we don't have strings anymore, we have bytes. Without offering the >bytes object, then people can't really convert their code. String >literals can be handled with the -U command line option (and perhaps >having the interpreter do the str=unicode assignment during startup). > A third step would ease this transition significantly: a unicode_literals __future__ import. > >Here's my suggestion: every feature, syntax, etc., that is slated for >Py3k, let us release bit by bit in the 2.x series. That lets the 2.x >series evolve into the 3.x series in a somewhat more natural way than >the currently proposed *everything breaks*. If it takes 1, 2, 3, or 10 >more releases in the 2.x series to get to all of the 3.x features, great. >At least people will have a chance to convert, or at least write correct >code for the future. This really seems like the right idea. "Shoot the moon" upgrades are almost always worse than incremental upgrades. The incremental path is better for everyone involved. For developers of Python, it gets more people using and providing feedback on the new features being developed. For developers with Python, it keeps the scope of a particular upgrade more manageable, letting them developer focus on a much smaller set of changes to be made to their application. Jean-Paul From ncoghlan at gmail.com Tue Aug 22 12:11:45 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 22 Aug 2006 20:11:45 +1000 Subject: [Python-Dev] [Python-checkins] r51458 - peps/trunk/pep-0000.txt peps/trunk/pep-0362.txt In-Reply-To: <20060821235408.8E80D1E4009@bag.python.org> References: <20060821235408.8E80D1E4009@bag.python.org> Message-ID: <44EAD861.7010201@gmail.com> brett.cannon wrote: > Added: peps/trunk/pep-0362.txt > +Relation With Other PEPs > +======================== > + > +"Keyword-Only Arguments [#pep-3102]_ > +------------------------------------ > + > +If keyword-only parameters come into existence, the Parameter object > +will require modification. A ``keyword_only`` attribute will be added > +that holds a boolean representing whether the parameter is > +keyword-only or not. Why an extra attribute instead of using "x.position is None"? Unlike default_value, None is not otherwise a legitimate value for the position attribute. > +Where should keyword-only parameters be stored in the Signature object? > +----------------------------------------------------------------------- > + > +If PEP 3102 [#pep-3102]_ ends up being accepted, there is the > +possibility that storing keyword-only parameters in a set instead of > +in the ``parameters`` attribute of the Signature object makes more > +sense. Since keyword-only parameters do not have any semantic meaning > +in terms of their position within the signature, there is no direct > +semantic gain in storing it in the parameter list. > + > +Storing keyword-only paramaters in a set makes it much more explicit > +that keyword-only parameters have no inherent order. It does have the > +drawback, though, that if one wants to process all parameters at once > +they would need to perform extra work to make sure to go through both > +the parameter list and set. Throwing the keyword arguments in a set still loses information that was present in the source code. Preserving the ordering in the signature object means that the author of the function can control the order in which the keyword arguments are displayed by introspection-based help utilities (e.g. putting "more important" arguments first. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From amk at amk.ca Tue Aug 22 14:41:20 2006 From: amk at amk.ca (A.M. Kuchling) Date: Tue, 22 Aug 2006 08:41:20 -0400 Subject: [Python-Dev] What should the focus for 2.6 be? In-Reply-To: References: <692A12F5-78C0-43BB-8041-C4CDAF4405B6@python.org> Message-ID: <20060822124120.GA15449@rogue.amk.ca> On Mon, Aug 21, 2006 at 12:24:54PM -0700, Brett Cannon wrote: > As for the docs, they just need a thorough updating. Michael Hudson was working on a new guide to extending/embedding Python. Incorporating that should be a goal for 2.6 (the document may still need to be finished -- I'm not sure what state it's in). The HOWTOs should be incorporated into the documentation build process; this will mean you need both LaTeX and Docutils to build all the documentation. I already have a Makefile patch for this; perhaps I should just check it into the trunk now. --amk From mwh at python.net Tue Aug 22 17:26:12 2006 From: mwh at python.net (Michael Hudson) Date: Tue, 22 Aug 2006 16:26:12 +0100 Subject: [Python-Dev] What should the focus for 2.6 be? In-Reply-To: <20060822124120.GA15449@rogue.amk.ca> (A. M. Kuchling's message of "Tue, 22 Aug 2006 08:41:20 -0400") References: <692A12F5-78C0-43BB-8041-C4CDAF4405B6@python.org> <20060822124120.GA15449@rogue.amk.ca> Message-ID: <2mpsesrduz.fsf@starship.python.net> "A.M. Kuchling" writes: > On Mon, Aug 21, 2006 at 12:24:54PM -0700, Brett Cannon wrote: >> As for the docs, they just need a thorough updating. > > Michael Hudson was working on a new guide to extending/embedding > Python. Incorporating that should be a goal for 2.6 (the document may > still need to be finished -- I'm not sure what state it's in). Oh my word, even I had nearly forgotten about that. I think the latest version is here: http://starship.python.net/crew/mwh/toext/ and source is here: http://codespeak.net/svn/user/mwh/toext/ It's certainly not even half-finished, and I'm not sure how much time or enthusiasm I'm likely to be able to find for it in the medium term. Cheers, mwh -- ZAPHOD: OK, so ten out of ten for style, but minus several million for good thinking, eh? -- The Hitch-Hikers Guide to the Galaxy, Episode 2 From barry at python.org Tue Aug 22 17:33:12 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 22 Aug 2006 11:33:12 -0400 Subject: [Python-Dev] What should the focus for 2.6 be? In-Reply-To: <2mpsesrduz.fsf@starship.python.net> References: <692A12F5-78C0-43BB-8041-C4CDAF4405B6@python.org> <20060822124120.GA15449@rogue.amk.ca> <2mpsesrduz.fsf@starship.python.net> Message-ID: <5BFC95A4-1202-40EA-9EA9-0397DB2A5F55@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 22, 2006, at 11:26 AM, Michael Hudson wrote: > Oh my word, even I had nearly forgotten about that. > > I think the latest version is here: > > http://starship.python.net/crew/mwh/toext/ > > and source is here: > > http://codespeak.net/svn/user/mwh/toext/ > > It's certainly not even half-finished, and I'm not sure how much time > or enthusiasm I'm likely to be able to find for it in the medium term. I'd be willing to help out with this, as well as flesh out some of the more, er, under-documented parts of the C API. I do a /ton/ of embedding/extending work for my Real Job so I think I could justify some work time devoted to improving things here. I'll try to take a look at MWH's stuff, but should we go ahead and check it into Python's svn at some point? - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBROsjuHEjvBPtnXfVAQLEcQQAiyH3LaJysEJrpbHPQhXUMBlQ/+ULDatc D1pBUmN02jY33fg14Z8Tw7AjJ5cOWsDFeF+gh6TBXvawpvF2q/XafdmtjkjwvQpa 8KfAl5JeNHSyTTd5tcKuZ1y2mQ1XpVVIqICT8PP24l+6RyCbnRhTe6MaLUHnUegm bSl728r3s64= =7WIt -----END PGP SIGNATURE----- From brett at python.org Tue Aug 22 19:16:01 2006 From: brett at python.org (Brett Cannon) Date: Tue, 22 Aug 2006 10:16:01 -0700 Subject: [Python-Dev] [Python-checkins] r51458 - peps/trunk/pep-0000.txt peps/trunk/pep-0362.txt In-Reply-To: <44EAD861.7010201@gmail.com> References: <20060821235408.8E80D1E4009@bag.python.org> <44EAD861.7010201@gmail.com> Message-ID: On 8/22/06, Nick Coghlan wrote: > > brett.cannon wrote: > > Added: peps/trunk/pep-0362.txt > > +Relation With Other PEPs > > +======================== > > + > > +"Keyword-Only Arguments [#pep-3102]_ > > +------------------------------------ > > + > > +If keyword-only parameters come into existence, the Parameter object > > +will require modification. A ``keyword_only`` attribute will be added > > +that holds a boolean representing whether the parameter is > > +keyword-only or not. > > Why an extra attribute instead of using "x.position is None"? Didn't think of that. =) Unlike > default_value, None is not otherwise a legitimate value for the position > attribute. > Makes sense. I will talk it over with Jiwon. Might want to rename 'position' to 'positional' then in order to make it even more obvious what the information stores. > +Where should keyword-only parameters be stored in the Signature object? > > +----------------------------------------------------------------------- > > + > > +If PEP 3102 [#pep-3102]_ ends up being accepted, there is the > > +possibility that storing keyword-only parameters in a set instead of > > +in the ``parameters`` attribute of the Signature object makes more > > +sense. Since keyword-only parameters do not have any semantic meaning > > +in terms of their position within the signature, there is no direct > > +semantic gain in storing it in the parameter list. > > + > > +Storing keyword-only paramaters in a set makes it much more explicit > > +that keyword-only parameters have no inherent order. It does have the > > +drawback, though, that if one wants to process all parameters at once > > +they would need to perform extra work to make sure to go through both > > +the parameter list and set. > > Throwing the keyword arguments in a set still loses information that was > present in the source code. Preserving the ordering in the signature > object > means that the author of the function can control the order in which the > keyword arguments are displayed by introspection-based help utilities (e.g > . > putting "more important" arguments first. That's a good enough argument for me. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060822/dd37116f/attachment.htm From brett at python.org Tue Aug 22 19:32:22 2006 From: brett at python.org (Brett Cannon) Date: Tue, 22 Aug 2006 10:32:22 -0700 Subject: [Python-Dev] [Python-checkins] r51458 - peps/trunk/pep-0000.txt peps/trunk/pep-0362.txt In-Reply-To: <44EAD861.7010201@gmail.com> References: <20060821235408.8E80D1E4009@bag.python.org> <44EAD861.7010201@gmail.com> Message-ID: On 8/22/06, Nick Coghlan wrote: > > brett.cannon wrote: > > Added: peps/trunk/pep-0362.txt > > +Relation With Other PEPs > > +======================== > > + > > +"Keyword-Only Arguments [#pep-3102]_ > > +------------------------------------ > > + > > +If keyword-only parameters come into existence, the Parameter object > > +will require modification. A ``keyword_only`` attribute will be added > > +that holds a boolean representing whether the parameter is > > +keyword-only or not. > > Why an extra attribute instead of using "x.position is None"? Unlike > default_value, None is not otherwise a legitimate value for the position > attribute. OK, talked over with Jiwon and the room here at the sprint and the preference was to keep it as-is. Reasons were it allowed the position in the signature to be kept with the object (and thus not be reliant on the Signature object to know its position in the actual output), and the possibility of disparate types being in the attribute. Plus Guido preferred the original way. =) -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060822/3233101e/attachment.html From jjl at pobox.com Tue Aug 22 19:42:02 2006 From: jjl at pobox.com (John J Lee) Date: Tue, 22 Aug 2006 18:42:02 +0100 (GMT Standard Time) Subject: [Python-Dev] String formatting / unicode 2.5 bug? In-Reply-To: <44E97EC9.4040502@gmail.com> References: <44E7ED19.6090907@gmail.com> <44E97EC9.4040502@gmail.com> Message-ID: On Mon, 21 Aug 2006, Nick Coghlan wrote: > John J Lee wrote: >>> And once the result has been promoted to unicode, __unicode__ is used >>> directly: >>> >>>>>> print repr("%s%s" % (a(), a())) >>> __str__ >>> accessing <__main__.a object at 0x00AF66F0>.__unicode__ >>> __str__ >>> accessing <__main__.a object at 0x00AF6390>.__unicode__ >>> __str__ >>> u'hihi' >> >> I don't understand this part. Why is __unicode__ called? Your example >> doesn't appear to show this happening "once [i.e., because?] the result has >> been promoted to unicode" -- if that were true, it would "stand to reason" >> that the interpreter would then conclude it should call >> __unicode__ for all remaining %s, and not bother with __str__. > > It does try to call unicode directly, but because the example object doesn't > supply __unicode__ it ends up falling back to __str__ instead. The behaviour > is clearer when the example object provides both methods: [...] If the interpreter is falling back from __unicode__ to __str__ (rather than the other way around, kind-of), that makes much more sense. I understood that __unicode__ was not provided, of course -- what wasn't clear to me was why the interpreter was calling/accessing those methods/attributes in the sequence it does. Still not sure I understand what the third __str__ above comes from, but until I've thought it through again, that's my problem. John From igor.bukanov at gmail.com Tue Aug 22 15:17:20 2006 From: igor.bukanov at gmail.com (Igor Bukanov) Date: Tue, 22 Aug 2006 15:17:20 +0200 Subject: [Python-Dev] GeneratorExit is unintuitive and uneccessary Message-ID: Consider the following example: for i in range(3): try: print i break except: print "Unexpected exception!" finally: print "Finally" When executed, it naturally prints 0 Finally since break does not use exceptions to transfer the control and as such can not be stopped using catch-all exception handler. Now consider a similar example using generators: def gen(): for i in range(3): try: yield i except: print "Unexpected exception!" finally: print "Finally" for i in gen(): print i break This example prints: 0 Unexpected exception! Finally Exception exceptions.RuntimeError: 'generator ignored GeneratorExit' in ignored Suddenly with generators a program can mess with control transfer since it uses explicit GeneratorExit which can be caught and ignored. This is unintuitive IMO. This example also suggests how to fix generators. One just need to change the close method so it would cause return executed right after the yield instead of throw. I.e. replace the current text from http://www.python.org/dev/peps/pep-0342/ 4. Add a close() method for generator-iterators, which raises GeneratorExit at the point where the generator was paused. If the generator then raises StopIteration (by exiting normally, or due to already being closed) or GeneratorExit (by not catching the exception), close() returns to its caller. If the generator yields a value, a RuntimeError is raised. If the generator raises any other exception, it is propagated to the caller. close() does nothing if the generator has already exited due to an exception or normal exit. by simpler one: 4. Add a close() method for generator-iterators, which executes normal return at the point where the generator was paused. If the generator then raises StopIteration (by exiting normally, or due to already being closed), close() returns to its caller. If the generator yields a value, a RuntimeError is raised. If the generator raises any other exception, it is propagated to the caller. close() does nothing if the generator has already exited due to an exception or normal exit. This not only fixes the above discrepancy between normal flow control and generators, removes GeneratorExit and simplifies the generator protocol, but also bring a new feature allowing to have easy to grasp feature table of the iterator methods: next: continue after yield throw: raise after yield close: return after yield Regards, Igor From ncoghlan at gmail.com Wed Aug 23 13:33:09 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 23 Aug 2006 21:33:09 +1000 Subject: [Python-Dev] GeneratorExit is unintuitive and uneccessary In-Reply-To: References: Message-ID: <44EC3CF5.8030502@gmail.com> Igor Bukanov wrote: > This example also suggests how to fix generators. One just need to > change the close method so it would cause return executed right after > the yield instead of throw. > > I.e. replace the current text from > http://www.python.org/dev/peps/pep-0342/ > by simpler one: Simpler is in the eye of the beholder - the current close() merely uses throw() to raise a particular kind of exception 'asynchronously' (which is already a familiar concept due to KeyboardInterrupt). What you're suggesting is a completely new flow control concept that doesn't exist anywhere else in the language. I suggested during the PEP 352 discussions that GeneratorExit should inherit directly from BaseException, so that the fix to your example would be to replace the bare "except:" with "except Exception:" (because it can swallow KeyboardInterrupt the version you posted is buggy regardless of how g.close() works). Guido didn't like that idea [1], so the correct way to write your exception handler is to either explicitly catch & reraise GeneratorExit before doing a general "except Exception:" catch-all clause, or to do the yield outside the try-except clause (but inside the try-finally) (obviously, this makes more sense when what you're yielding is an expression rather than a single variable). I think Guido is correct that GeneratorExit should be a normal exception from a technical point of view (as GE should never reach the outer layers of a program), but I also believe it is going to be a PITA from a practical point of view because I think the natural way of writing a generator with exception handling is to include the yield inside the try/except block instead of storing the result in a variable and yielding afterwards. However, since the status quo is the result of BDFL preference, it's going to be up to Guido as to whether or not he considers this enough of a potential pain to change his mind. As there isn't actually any new evidence here (it's all still hypothetical), I don't really expect that to happen :) That said, with the PEP 342 implementation now being a bit more mature, I am curious as to exactly *how* GeneratorExit is supposed to leak out of code that was supposed to catch it (which was Guido's main concern at the time he decided that GE should remain a normal Exception). GeneratorExit really shouldn't be getting raised by anything other than g.close(), which takes great pains to ensure that the exception gets handled correctly (and if it *doesn't* get handled correctly, the resulting exception that reports that fact is a RuntimeError, not GeneratorExit). > This not only fixes the above discrepancy between normal flow control > and generators, removes GeneratorExit and simplifies the generator > protocol, but also bring a new feature allowing to have easy to grasp > feature table of the iterator methods: > > next: continue after yield This isn't right, though. next()/send() just resume execution after a yield. They are only equivalent to continue if the yield is at the end of a loop body. > throw: raise after yield > close: return after yield In the current implementation, close is just a special case of throw (g.throw(GeneratorExit)), the same as next() is just a special case of send (g.send(None)). 2 concepts to understand (send(), throw()) is simpler to my mind than 3 (send(), throw(), close()). Cheers, Nick. [1] http://mail.python.org/pipermail/python-dev/2006-March/062825.html -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Wed Aug 23 13:44:44 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 23 Aug 2006 21:44:44 +1000 Subject: [Python-Dev] String formatting / unicode 2.5 bug? In-Reply-To: References: <44E7ED19.6090907@gmail.com> <44E97EC9.4040502@gmail.com> Message-ID: <44EC3FAC.3030202@gmail.com> John J Lee wrote: > On Mon, 21 Aug 2006, Nick Coghlan wrote: > >> John J Lee wrote: >>>> And once the result has been promoted to unicode, __unicode__ is used >>>> directly: >>>> >>>>>>> print repr("%s%s" % (a(), a())) >>>> __str__ >>>> accessing <__main__.a object at 0x00AF66F0>.__unicode__ >>>> __str__ >>>> accessing <__main__.a object at 0x00AF6390>.__unicode__ >>>> __str__ >>>> u'hihi' >>> I don't understand this part. Why is __unicode__ called? Your example >>> doesn't appear to show this happening "once [i.e., because?] the result has >>> been promoted to unicode" -- if that were true, it would "stand to reason" >>> that the interpreter would then conclude it should call >>> __unicode__ for all remaining %s, and not bother with __str__. >> It does try to call unicode directly, but because the example object doesn't >> supply __unicode__ it ends up falling back to __str__ instead. The behaviour >> is clearer when the example object provides both methods: > [...] > > If the interpreter is falling back from __unicode__ to __str__ (rather > than the other way around, kind-of), that makes much more sense. I > understood that __unicode__ was not provided, of course -- what wasn't > clear to me was why the interpreter was calling/accessing those > methods/attributes in the sequence it does. Still not sure I understand > what the third __str__ above comes from, but until I've thought it through > again, that's my problem. The sequence is effectively: x, y = a(), a() str(x) # calls x.__str__ unicode(x) # tries x.__unicode__, fails, falls back to x.__str__ unicode(y) # tries y.__unicode__, fails, falls back to y.__str__ The trick in 2.5 is that the '%s' format code, instead of actually calling str(x), calls x.__str__() directly, and promotes the result to Unicode if x.__str__() returns a Unicode result. I'll try to do something to clear up that section of the documentation before 2.5 final. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From guido at python.org Wed Aug 23 16:28:24 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 23 Aug 2006 07:28:24 -0700 Subject: [Python-Dev] GeneratorExit is unintuitive and uneccessary In-Reply-To: <44EC3CF5.8030502@gmail.com> References: <44EC3CF5.8030502@gmail.com> Message-ID: This discussion probably happens as a result of attempts to copy Python's design in JavaScript. I propose that JavaScript do whatever it wants and its developers leave Python alone -- maybe some time in the future we'll be able to compare the two approaches. I think it's impossible to do so at the moment. --Guido On 8/23/06, Nick Coghlan wrote: > Igor Bukanov wrote: > > This example also suggests how to fix generators. One just need to > > change the close method so it would cause return executed right after > > the yield instead of throw. > > > > I.e. replace the current text from > > http://www.python.org/dev/peps/pep-0342/ > > > > by simpler one: > > > Simpler is in the eye of the beholder - the current close() merely uses > throw() to raise a particular kind of exception 'asynchronously' (which is > already a familiar concept due to KeyboardInterrupt). What you're suggesting > is a completely new flow control concept that doesn't exist anywhere else in > the language. > > I suggested during the PEP 352 discussions that GeneratorExit should inherit > directly from BaseException, so that the fix to your example would be to > replace the bare "except:" with "except Exception:" (because it can swallow > KeyboardInterrupt the version you posted is buggy regardless of how g.close() > works). > > Guido didn't like that idea [1], so the correct way to write your exception > handler is to either explicitly catch & reraise GeneratorExit before doing a > general "except Exception:" catch-all clause, or to do the yield outside the > try-except clause (but inside the try-finally) (obviously, this makes more > sense when what you're yielding is an expression rather than a single variable). > > I think Guido is correct that GeneratorExit should be a normal exception from > a technical point of view (as GE should never reach the outer layers of a > program), but I also believe it is going to be a PITA from a practical point > of view because I think the natural way of writing a generator with exception > handling is to include the yield inside the try/except block instead of > storing the result in a variable and yielding afterwards. > > However, since the status quo is the result of BDFL preference, it's going to > be up to Guido as to whether or not he considers this enough of a potential > pain to change his mind. As there isn't actually any new evidence here (it's > all still hypothetical), I don't really expect that to happen :) > > That said, with the PEP 342 implementation now being a bit more mature, I am > curious as to exactly *how* GeneratorExit is supposed to leak out of code that > was supposed to catch it (which was Guido's main concern at the time he > decided that GE should remain a normal Exception). GeneratorExit really > shouldn't be getting raised by anything other than g.close(), which takes > great pains to ensure that the exception gets handled correctly (and if it > *doesn't* get handled correctly, the resulting exception that reports that > fact is a RuntimeError, not GeneratorExit). > > > This not only fixes the above discrepancy between normal flow control > > and generators, removes GeneratorExit and simplifies the generator > > protocol, but also bring a new feature allowing to have easy to grasp > > feature table of the iterator methods: > > > > next: continue after yield > > This isn't right, though. next()/send() just resume execution after a yield. > They are only equivalent to continue if the yield is at the end of a loop body. > > > throw: raise after yield > > close: return after yield > > In the current implementation, close is just a special case of throw > (g.throw(GeneratorExit)), the same as next() is just a special case of send > (g.send(None)). > > 2 concepts to understand (send(), throw()) is simpler to my mind than 3 > (send(), throw(), close()). > > Cheers, > Nick. > > [1] http://mail.python.org/pipermail/python-dev/2006-March/062825.html > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > --------------------------------------------------------------- > http://www.boredomandlaziness.org > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Wed Aug 23 16:35:42 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 23 Aug 2006 10:35:42 -0400 Subject: [Python-Dev] GeneratorExit is unintuitive and uneccessary In-Reply-To: Message-ID: <5.1.1.6.0.20060823102150.03a90638@sparrow.telecommunity.com> I rather like it, actually. I don't recall there being any real reasons to catch a GeneratorExit exception, but even if there were, you could do the equivalent like this: try: closed = True yield 1 closed = False finally: if closed: # special closing code Unfortunately, this is a pretty major change to be making to the specification (PEP 342) when we've just released 2.5c1; it seems like something that requires at least a beta release. But aside from that, it seems like a fairly neat solution to the problem. I'd like to hear from Jython and IronPython representatives regarding implementability for those platforms, though. Our original assumption was that if they could implement throw() then they could clearly implement close(), since close() was defined in terms of throw(). An asynchronous return might be another matter. But if that hurdle is crossable, then I agree that modelling close() as an early return is more intuitive, eliminates a built-in exception, etc. At 03:17 PM 8/22/2006 +0200, Igor Bukanov wrote: >Consider the following example: > >for i in range(3): > try: > print i > break > except: > print "Unexpected exception!" > finally: > print "Finally" > >When executed, it naturally prints > 0 > Finally >since break does not use exceptions to transfer the control and as >such can not be stopped using catch-all exception handler. > >Now consider a similar example using generators: > >def gen(): > for i in range(3): > try: > yield i > except: > print "Unexpected exception!" > finally: > print "Finally" > >for i in gen(): > print i > break > >This example prints: > 0 > Unexpected exception! > Finally > Exception exceptions.RuntimeError: 'generator ignored GeneratorExit' >in ignored > >Suddenly with generators a program can mess with control transfer >since it uses explicit GeneratorExit which can be caught and ignored. >This is unintuitive IMO. > >This example also suggests how to fix generators. One just need to >change the close method so it would cause return executed right after >the yield instead of throw. > >I.e. replace the current text from >http://www.python.org/dev/peps/pep-0342/ > >4. Add a close() method for generator-iterators, which raises >GeneratorExit at the point where the generator was paused. If the >generator then raises StopIteration (by exiting normally, or due to >already being closed) or GeneratorExit (by not catching the >exception), close() returns to its caller. If the generator yields a >value, a RuntimeError is raised. If the generator raises any other >exception, it is propagated to the caller. close() does nothing if the >generator has already exited due to an exception or normal exit. > >by simpler one: > >4. Add a close() method for generator-iterators, which executes normal >return at the point where the generator was paused. If the generator >then raises StopIteration (by exiting normally, or due to already >being closed), close() returns to its caller. If the generator yields >a value, a RuntimeError is raised. If the generator raises any other >exception, it is propagated to the caller. close() does nothing if the >generator has already exited due to an exception or normal exit. > >This not only fixes the above discrepancy between normal flow control >and generators, removes GeneratorExit and simplifies the generator >protocol, but also bring a new feature allowing to have easy to grasp >feature table of the iterator methods: > >next: continue after yield >throw: raise after yield >close: return after yield > >Regards, Igor >_______________________________________________ >Python-Dev mailing list >Python-Dev at python.org >http://mail.python.org/mailman/listinfo/python-dev >Unsubscribe: >http://mail.python.org/mailman/options/python-dev/pje%40telecommunity.com From cemerick at snowtide.com Wed Aug 23 17:11:32 2006 From: cemerick at snowtide.com (Chas Emerick) Date: Wed, 23 Aug 2006 11:11:32 -0400 Subject: [Python-Dev] Python + Java Integration Message-ID: This may seem like it's coming out of left field for a minute, but bear with me. There is no doubt that Ruby's success is a concern for anyone who sees it as diminishing Python's status. One of the reasons for Ruby's success is certainly the notion (originally advocated by Bruce Tate, if I'm not mistaken) that it is the "next Java" -- the language and environment that mainstream Java developers are, or will, look to as a natural next step. One thing that would help Python in this "debate" (or, perhaps simply put it in the running, at least as a "next Java" candidate) would be if Python had an easier migration path for Java developers that currently rely upon various third-party libraries. The wealth of third-party libraries available for Java has always been one of its great strengths. Ergo, if Python had an easy-to-use, recommended way to use those libraries within the Python environment, that would be a significant advantage to present to Java developers and those who would choose Ruby over Java. Platform compatibility is always a huge motivator for those looking to migrate or upgrade. In that vein, I would point to JPype (http://jpype.sourceforge.net). JPype is a module that gives "python programs full access to java class libraries". My suggestion would be to either: (a) include JPype in the standard library, or barring that, (b) make a very strong push to support JPype (a) might be difficult or cumbersome technically, as JPype does need to build against Java headers, which may or may not be possible given the way that Python is distributed, etc. However, (b) is very feasible. I can't really say what "supporting JPype" means exactly -- maybe GvR and/or other heavyweights in the Python community make public statements regarding its existence and functionality, maybe JPype gets a strong mention or placement on python.org....all those details are obviously not up to me, and I don't know the workings of the "official" Python organizations enough to make serious suggestions. Regardless of the form of support, I think raising people's awareness of JPype and what it adds to the Python environment would be a Good Thing (tm). For our part, we've used JPype to make PDFTextStream (our previously Java-only PDF text extraction library) available and supported for Python. You can read some about it here: http://snowtide.com/PDFTextStream.Python And I've blogged about how PDFTextStream.Python came about, and how we worked with Steve M?nard, the maintainer of JPype, to make it all happen (watch out for this URL wrapping): http://blog.snowtide.com/2006/08/21/working-together-pythonjava-open- sourcecommercial Cheers, Chas Emerick Founder, Snowtide Informatics Systems Enterprise-class PDF content extraction cemerick at snowtide.com http://snowtide.com | +1 413.519.6365 From jeremy.kloth at 4suite.org Wed Aug 23 17:21:27 2006 From: jeremy.kloth at 4suite.org (Jeremy Kloth) Date: Wed, 23 Aug 2006 09:21:27 -0600 Subject: [Python-Dev] [4suite] cDomlette deallocation bug? In-Reply-To: <44EB71F2.2080509@cs.washington.edu> References: <44EB71F2.2080509@cs.washington.edu> Message-ID: <200608230921.28027.jeremy.kloth@4suite.org> I'm CC python-dev as this may be a bug in the gc module. Although this may be more of a c.l.p. post, I think that this usage shouldn't cause segfaults regardless. On Tuesday 22 August 2006 3:06 pm, Robert Fleming wrote: > [step 1 unnecessary] > 2. test program (test.c): > /******** snip ********/ > #include > > #include > > int main(int argc, char* argv[]) > { > int i; > for (i = 0; i<10; i++) > { > printf("%d\n", i); > > Py_Initialize(); > (void)PyRun_SimpleString("import sys\n" > "sys.path.append('.')\n" > "import cDomlettec"); Change the string to just "import gc" and the segfault still happens. > Py_Finalize(); > } > > return EXIT_SUCCESS; > } > /****** snip ********/ > > 3. compile with: > gcc -I/usr/include/python2.4 test.c -lpython2.4 > > 4. run program; output is: > 0 > 1 > Segmentation fault > > Am I doing something wrong? I'm pretty new to embedding Python, but I'm > hoping to be able to cleanly load and unload 4Suite this way It just so happens that cDomlettec imports gc internally which, by the change above shows, causes the segfault. Hopefully someone with more knowledge of GC internals can comment on this. -- Jeremy Kloth http://4suite.org/ From theller at python.net Wed Aug 23 22:03:39 2006 From: theller at python.net (Thomas Heller) Date: Wed, 23 Aug 2006 22:03:39 +0200 Subject: [Python-Dev] ctypes and win64 In-Reply-To: <44E60B2A.6020101@v.loewis.de> References: <44E60B2A.6020101@v.loewis.de> Message-ID: Martin v. L?wis schrieb: > Thomas Heller schrieb: >> I suggest that it [the ctypes package] should be removed from the 2.5 win64 msi installers, so that >> at least, when it is ready, can be installed as separate package. > > Unfortunately, I won't be able to work on this until the release is > made. Feel free to work on it, but I suggest to just not worry about > it. Patch for Tools\msi\msi.py uploaded, and assigned to you. http://python.org/sf/1545507 Thomas From sreeram at tachyontech.net Wed Aug 23 23:22:55 2006 From: sreeram at tachyontech.net (K.S.Sreeram) Date: Thu, 24 Aug 2006 02:52:55 +0530 Subject: [Python-Dev] Can LOAD_GLOBAL be optimized to a simple array lookup? Message-ID: <44ECC72F.10303@tachyontech.net> Hi all, I noticed in Python/ceval.c that LOAD_GLOBAL uses a dictionary lookup, and was wondering if that can be optimized to a simple array lookup. If i'm right there are 3 kinds of name lookups: locals, outer scopes(closures), and globals. (not counting attribute lookup). Locals are identified by, either the presence of assignments, or their presence in the arg list. So all name lookups can be classified into the 3 types at compile/load time. Since we know, at load time, which names are global.. Can't we simply build a global name table and replace LOAD_GLOBALs with a lookup at the corresponding index into the global name table? The module's dict object will need to be special so that whenever a name gets 'set', the global name table should get updated. Is this optimization possible or have i missed something? cheers [sreeram;] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20060824/d936dba5/attachment.pgp From tjreedy at udel.edu Wed Aug 23 23:58:39 2006 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 23 Aug 2006 17:58:39 -0400 Subject: [Python-Dev] Python + Java Integration References: Message-ID: "Chas Emerick" wrote in message news:B53BCFBF-4FF1-42A8-B668-CB3E5513486E at snowtide.com... >One thing that would help Python in this "debate" In general, discussion of promoting/marketing python is more a topic for comp.lang.python than for this list. Also see below. >My suggestion would be to either: >(a) include JPype in the standard library, or barring that, Externally developed modules become part of the standard library by becoming sufficiently 'widely' tested and used in the Python community. One means for this is announcement and discussion in c.l.p. That and technical considerations aside, this strikes me as one of several specialized 'bridge' tools, none of which should be in that standard lib but all of which should be listed and accessible in the Python Package Index. >(b) make a very strong push to support JPype We are mostly all volunteers here. Go ahead and scratch this itch if you want. If it does not yet already, I would agree that python.org should have a page on bridging between Python and other languages (not just Java, though). C.l.py would be a place to discuss the idea and collect suggestions for inclusion on such a page >For our part, we've used JPype to make PDFTextStream (our previously >Java-only PDF text extraction library) available and supported for Python. Success strories that provide new information are welcome on c.l.py. Perhaps a couple of paragraphs summarizing the content that you link to. Terry Jan Reedy From brett at python.org Thu Aug 24 00:28:55 2006 From: brett at python.org (Brett Cannon) Date: Wed, 23 Aug 2006 15:28:55 -0700 Subject: [Python-Dev] Can LOAD_GLOBAL be optimized to a simple array lookup? In-Reply-To: <44ECC72F.10303@tachyontech.net> References: <44ECC72F.10303@tachyontech.net> Message-ID: On 8/23/06, K.S.Sreeram wrote: > > Hi all, > > I noticed in Python/ceval.c that LOAD_GLOBAL uses a dictionary lookup, > and was wondering if that can be optimized to a simple array lookup. No, not as the language stands now. If i'm right there are 3 kinds of name lookups: locals, outer > scopes(closures), and globals. (not counting attribute lookup). Locals > are identified by, either the presence of assignments, or their presence > in the arg list. So all name lookups can be classified into the 3 types > at compile/load time. > > Since we know, at load time, which names are global.. Can't we simply > build a global name table and replace LOAD_GLOBALs with a lookup at the > corresponding index into the global name table? But we don't know statically what the globals will be. You can import a module and put something in its global namespace externally. That is done after load time or compile time. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060823/0533d23a/attachment.htm From thomas at python.org Thu Aug 24 00:29:21 2006 From: thomas at python.org (Thomas Wouters) Date: Thu, 24 Aug 2006 00:29:21 +0200 Subject: [Python-Dev] [Python-checkins] r51525 - in python/trunk: Lib/test/test_float.py Objects/floatobject.c In-Reply-To: <9e804ac0608231516k3a08f0c5m3c4375fe0a09a24a@mail.gmail.com> References: <20060823204203.3C4911E4009@bag.python.org> <9e804ac0608231516k3a08f0c5m3c4375fe0a09a24a@mail.gmail.com> Message-ID: <9e804ac0608231529y7c8b941fyf1cb21a7a4ef5a6b@mail.gmail.com> Since Alex isn't on python-dev, forwarding this for his convenience (he said he wanted to reply.) Tim also replied to his checkin, but I didn't forward that message. ---------- Forwarded message ---------- From: Thomas Wouters Date: Aug 24, 2006 12:16 AM Subject: Re: [Python-checkins] r51525 - in python/trunk: Lib/test/test_float.py Objects/floatobject.c To: "alex. martelli" On 8/23/06, alex.martelli wrote: > > > --- python/trunk/Objects/floatobject.c (original) > +++ python/trunk/Objects/floatobject.c Wed Aug 23 22:42:02 2006 > @@ -821,12 +821,12 @@ > ix = pow(iv, iw); > PyFPE_END_PROTECT(ix) > Py_ADJUST_ERANGE1(ix); > - if (errno != 0) { > + /* we need to ignore ERANGE here and just return inf */ > + if (errno != 0 && errno != ERANGE) { > /* We don't expect any errno value other than ERANGE, but > * the range of libm bugs appears unbounded. > */ > - PyErr_SetFromErrno(errno == ERANGE ? PyExc_OverflowError : > - PyExc_ValueError); > + PyErr_SetFromErrno(PyExc_ValueError); > return NULL; I don't think this can be right. The returnvalue of pow() in the case of ERANGE isn't defined anywhere that I can find, at least. How can we assume it is +Infinity? As I tried to say over the visiphone, you can't even use compile-time or startup-time checks for the behaviour because 'undefined' also means it need not be consistent, either. The best we could do, if we really wanted to return +inf instead of raising OverflowError (which I don't, but don't really care about either), is generate +Infinity in some guaranteed way. I'm sure Tim can come up with a convenient, portable way < 0.9 wink>. But I'd prefer to keep it the way it was: x*x and x**2 don't always do the same thing. Floats have a lot more confusing features like that, such as 10*x - 9*x need not be x. I don't see the added value of trying to make it consistent in just this case, even if it's portable. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060824/5949f7c3/attachment.html From aleaxit at gmail.com Thu Aug 24 00:39:43 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Wed, 23 Aug 2006 15:39:43 -0700 Subject: [Python-Dev] Can LOAD_GLOBAL be optimized to a simple array lookup? In-Reply-To: <44ECC72F.10303@tachyontech.net> References: <44ECC72F.10303@tachyontech.net> Message-ID: On Aug 23, 2006, at 2:22 PM, K.S.Sreeram wrote: > Hi all, > > I noticed in Python/ceval.c that LOAD_GLOBAL uses a dictionary lookup, > and was wondering if that can be optimized to a simple array lookup. > > If i'm right there are 3 kinds of name lookups: locals, outer > scopes(closures), and globals. (not counting attribute lookup). Locals > are identified by, either the presence of assignments, or their > presence > in the arg list. So all name lookups can be classified into the 3 > types > at compile/load time. > > Since we know, at load time, which names are global.. Can't we simply > build a global name table and replace LOAD_GLOBALs with a lookup at > the > corresponding index into the global name table? At the time the function's body gets compiled, the global (or builtin) it's trying to access might or might not be there -- as long as it gets added afterwards, before the function's body gets _executed_, no problem (in today's scheme). It's not obvious to me how you could compile a ``corresponding index'' into the LOAD_GLOBAL opcode, since that index is in general unknown at compile time. > The module's dict object will need to be special so that whenever a > name > gets 'set', the global name table should get updated. It seems that you'd need to chase down and modify all of the LOAD_GLOBAL opcodes too, at every such modification. (the concept of modifying builtins becomes extremely scary...). Considering the amortized speed of a dict lookup for an interned string (hash value cached and thus immediately available, equality comparison with other interned string a single machine-level operation), it's not clear to me that the huge complexity (and potential performance impact) of all this could ever possibly be justified. A change in Python semantics allowing some level of "nailing down" of builtins (and perhaps globals too) *COULD* easily yield large performance benefits, but that's a subject for the Python-3000 mailing list; as long as builtins and globals stay as fluid as today, I'm skeptical on the optimization opportunities here. Alex From noamraph at gmail.com Thu Aug 24 00:50:34 2006 From: noamraph at gmail.com (Noam Raphael) Date: Thu, 24 Aug 2006 01:50:34 +0300 Subject: [Python-Dev] Can LOAD_GLOBAL be optimized to a simple array lookup? In-Reply-To: References: <44ECC72F.10303@tachyontech.net> Message-ID: 2006/8/24, Brett Cannon : > > > > On 8/23/06, K.S.Sreeram wrote: > > Hi all, > > > > I noticed in Python/ceval.c that LOAD_GLOBAL uses a dictionary lookup, > > and was wondering if that can be optimized to a simple array lookup. > > > No, not as the language stands now. > > > If i'm right there are 3 kinds of name lookups: locals, outer > > scopes(closures), and globals. (not counting attribute lookup). Locals > > are identified by, either the presence of assignments, or their presence > > in the arg list. So all name lookups can be classified into the 3 types > > at compile/load time. > > > > Since we know, at load time, which names are global.. Can't we simply > > build a global name table and replace LOAD_GLOBALs with a lookup at the > > corresponding index into the global name table? > > > But we don't know statically what the globals will be. You can import a > module and put something in its global namespace externally. That is done > after load time or compile time. > I think that it can be implemented for the language as it stands now. I don't know whether it will be a good thing or not. In principle, you can add a feature to dict implementation that will allow it to notify when the value of a specific key changes. If you have that, you can change LOAD_GLOBAL implementation to: 1. look for the global. 2. ask for notification when the global dict changes in a way that will change the meaning of the global. 3. change the LOAD_GLOBAL opcode to something like LOAD_CONST, and set the notification from the dict to update the LOAD_CONST opcode to the new object. In that way, LOAD_GLOBAL will cause a dict lookup only once. Changing the value of globals will require more work, though. Again, I'm not saying that it's desired, just that it's possible. Have a good day, Noam From brett at python.org Thu Aug 24 00:59:39 2006 From: brett at python.org (Brett Cannon) Date: Wed, 23 Aug 2006 15:59:39 -0700 Subject: [Python-Dev] PEP 362 open issues Message-ID: I have been spending my Google sprint time on writing and implementing PEP 362 (http://www.python.org/dev/peps/pep-0362/), which finally defines Signature objects for functions. With the implementation at a place I am happy with, I wanted to ask about the open issues with the PEP. The first question is whether to have Signature objects on all functions by default or to leave it in the 'inspect' module and have it be something totally optional. My vote is the latter for now and we can add them to all functions if they turn out to be useful and popular. The second open issue is how to handle binding arguments to tuple parameters for the Signature.bind() method. If you have the function ``def foo((a, b): pass``, how do you have the Signature.bind() method handle 'a', and 'b'? You can either take the proposed arguments and in the dict mapping arguments to parameters have an entry for 'a' and 'b' individually, or have an entry for '(a, (b,))'. My vote is the latter since you cannot address 'a' or 'b' individually as keyword arguments and this makes more sense with function annotations since they won't apply to 'a' and 'b' individually but instead to the entire tuple. Either way there is the issue of exhausting an iterator (like a generator) trying to find out if it is the proper length to unpack, for which I raise a TypeError instead of exhaust a possibly mutable iterator currently. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060823/926f341c/attachment.htm From guido at python.org Thu Aug 24 01:10:03 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 23 Aug 2006 16:10:03 -0700 Subject: [Python-Dev] GeneratorExit is unintuitive and uneccessary In-Reply-To: References: <5.1.1.6.0.20060823102150.03a90638@sparrow.telecommunity.com> Message-ID: IIUC this is how return currently works -- in some sense it's an exception, but it can't be caught, and it won't escape from the current frame. Ditto for break/continue. The generator extensions are still very young, and I'm not against changing them somewhat in 2.6 or py3k. Perhaps GeneratorExit can be replaced by a treatment similar to the treatment of return. But I'm not sure it helps with the original problem; you could still put a yield in a finally clause that intercepts a return, just as today you can write def f(): for i in range(10): print i try: return finally: break print 999 where the finally clause nullifies the return. In a generator, a yield in the finally clause still needs to be dealt with somehow. On 8/23/06, Igor Bukanov wrote: > On 8/23/06, Phillip J. Eby wrote: > > Our original > > assumption was that if they could implement throw() then they could clearly > > implement close(), since close() was defined in terms of throw(). An > > asynchronous return might be another matter. > > Such asynchronous return can always be implemented via a special > hidden exception that is invisible to catch statements. I.e an > implementation of generators can still use ___GeneratorExit > internally as long as it is not exposed to scripts. > > Regards, Igor > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Thu Aug 24 00:13:55 2006 From: martin at v.loewis.de (martin at v.loewis.de) Date: Thu, 24 Aug 2006 00:13:55 +0200 Subject: [Python-Dev] [Python-checkins] r51525 - in python/trunk:Lib/test/test_float.py Objects/floatobject.c In-Reply-To: <1f7befae0608231500n475cfed2x6cbfba20e1a2b480@mail.gmail.com> References: <20060823204203.3C4911E4009@bag.python.org> <1f7befae0608231500n475cfed2x6cbfba20e1a2b480@mail.gmail.com> Message-ID: <1156371234.44ecd32300e03@www.domainfactory-webmail.de> Zitat von Tim Peters : > Huh. It's been a (mildly controversial, but intentional all the same) > feature that Python tries to raise raise OverflowError on overflowing > libm operations. Doesn't work all that well, since there's no > consistency across platforms about when libm sets errno, or to what > (when it does). The other question is whether pow() is guaranteed to return inf when it sets errno to ERANGE; I believe there is no such guarantee. So I think the change should be reverted. Regards, Martin From g.brandl at gmx.net Thu Aug 24 01:13:54 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 24 Aug 2006 01:13:54 +0200 Subject: [Python-Dev] PEP 362 open issues In-Reply-To: References: Message-ID: Brett Cannon wrote: > I have been spending my Google sprint time on writing and implementing > PEP 362 (http://www.python.org/dev/peps/pep-0362/), which finally > defines Signature objects for functions. With the implementation at a > place I am happy with, I wanted to ask about the open issues with the PEP. While reading the PEP, I stumbled over the description of the bind() method. As I understand the implementation patch, the parameter names will be keys and the arguments values in the resulting dictionary, but this sentence looks to me as if it describes it the opposite way (it also doesn't clarify that "parameter" means "parameter name"): * bind(*args, **kwargs) -> dict Create a mapping from argument to parameter for the signature (see `Open Issues`_ for question of how to handle tuples). Another question: could it be helpful to make Parameter.default_value a weak reference? Georg From aleaxit at gmail.com Thu Aug 24 01:17:47 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Wed, 23 Aug 2006 16:17:47 -0700 Subject: [Python-Dev] [Python-checkins] r51525 - in python/trunk: Lib/test/test_float.py Objects/floatobject.c In-Reply-To: <9e804ac0608231529y7c8b941fyf1cb21a7a4ef5a6b@mail.gmail.com> References: <20060823204203.3C4911E4009@bag.python.org> <9e804ac0608231516k3a08f0c5m3c4375fe0a09a24a@mail.gmail.com> <9e804ac0608231529y7c8b941fyf1cb21a7a4ef5a6b@mail.gmail.com> Message-ID: <09CE8417-9F1F-4DEC-82DE-43AA0DAFD53E@gmail.com> On Aug 23, 2006, at 3:29 PM, Thomas Wouters wrote: > > Since Alex isn't on python-dev, forwarding this for his convenience > (he said he wanted to reply.) Thanks! I _am_ on Python-Dev (otherwise couldn't read what you're forwarding here), but not on Python-Checkins (where the discussion was started). > Tim also replied to his checkin, but I didn't forward that message. Yep, saw that (thanks Brett) -- if I read him right, Tim was basically suggesting better ways to check for infinity rather than saying it's better to keep today's inconsistency (he can surely correct me if I misread). Tim is of course right, but PEP 754 (the fpconst module) would appear to be the right way to perform the checking, anyway. > I don't think this can be right. The returnvalue of pow() in the > case of ERANGE isn't defined anywhere that I can find, at least. > How can we assume it is +Infinity? As I tried to say over the > visiphone, you can't even use compile- That's how I read ISO/IEC 9899:1999 7.12.1.4 , which is describing C's functions in general: """ If a floating result overflows and default rounding is in effect, or if the mathematical result is an exact infinity (for example log (0.0)), then the function returns the value of the macro HUGE_VAL, HUGE_VALF, or HUGE_VALL according to the return type, with the same sign as the correct value of the function; """ etc. Yes, that's a C language standard that's just 7 years old, rather than the 16-years-old standard Python can "definitely" count on; but we could presumably test (and rely on) __STDC_IEC_559__ if we need features in Appendix F, or just on __STDC_VERSION__ being defined and >= 199901L for more generic parts of the standard (such as the part I was quoting above from chapter 7). People who build Python with C compilers not meeting these macro criteria but still "good enough" to afford a minimally functional implementation of IEEE 754 floats might run configure with some appropriate -with-whatever options to assert the fact, if needed (i.e., if either because of C 1990 Standard language, or realworld brokenness of important compilers, automatic tests are in fact unreliable). And PEP 754 (and thus fpconst, or some other way for Python code to tell if it's running on a minimally functional implementation of IEEE 754 floats, and if so test for and generate NaNs and infinities) should also be there so that the Python-coded unittests can properly decide whether to test for compliance. > time or startup-time checks for the behaviour because 'undefined' > also means it need not be consistent, either. The Yes, but the behavior is not undefined in standard C (1999 standard; I don't have access to the 1990 standard) > best we could do, if we really wanted to return +inf instead of > raising OverflowError (which I don't, but don't really care about > either), is generate +Infinity in some guaranteed way. I'm sure Tim > can come up with a convenient, portable way < 0.9 wink>. The macro Py_HUGE_VAL seems to be the documented way to do it. And while I'd rather uniformly get infinity from operations producing infinity, what I'm really keen to avoid is having one such operation raise an exception while another one succeeds. > But I'd prefer to keep it the way it was: x*x and x**2 don't always > do the same thing. Floats have a lot more confusing features like > that, such as 10*x - 9*x need not be x. I don't see the added value > of trying to make it consistent in just this case, even if it's > portable. There's a huge conceptual gap between, on one side, two values produced by "roughly equivalent" expressions being only *ALMOST* equal rather than strictly equal, and one of the expressions raising an exception while the other one happily proceeds. The former case is reasonably easy to handle -- that's why we have an assertAlmostEqual method in unittest.TestCase; the latter can be a nightmare. The same goes for the other bug I recently added to the tracker, the """ x != y and [x] == [y] """ one. Of course, according to this "more confusing features" philosophy, I guess it can be considered quite OK to leave in Python absurdities like this one, as well. I abhor that whle "slippery slope" philosophy ("there's a tiny unavoidable problem in this corner so it's OK to leave a huge gaping hole all over the place") but I see no way to resolve such philosophical conflict without BDFL pronouncement. So, I have reverted r51525; I now believe it can't be done right (with proper unittests) without PEP 754 or something like that, anyway. Alex From aleaxit at gmail.com Thu Aug 24 01:24:11 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Wed, 23 Aug 2006 16:24:11 -0700 Subject: [Python-Dev] [Python-checkins] r51525 - in python/trunk:Lib/test/test_float.py Objects/floatobject.c In-Reply-To: <1156371234.44ecd32300e03@www.domainfactory-webmail.de> References: <20060823204203.3C4911E4009@bag.python.org> <1f7befae0608231500n475cfed2x6cbfba20e1a2b480@mail.gmail.com> <1156371234.44ecd32300e03@www.domainfactory-webmail.de> Message-ID: <38FC57F6-7F1A-4F4B-B6A6-9198F2BC0FC4@gmail.com> On Aug 23, 2006, at 3:13 PM, martin at v.loewis.de wrote: > Zitat von Tim Peters : > >> Huh. It's been a (mildly controversial, but intentional all the >> same) >> feature that Python tries to raise raise OverflowError on overflowing >> libm operations. Doesn't work all that well, since there's no >> consistency across platforms about when libm sets errno, or to what >> (when it does). > > The other question is whether pow() is guaranteed to return inf > when it sets errno to ERANGE; I believe there is no such guarantee. Summarizing my longer mail that crossed with yours: the guarantee is there in the 1999 C Standard; I don't have access any more to a copy of the 1990 C Standard to check how things were back then. > So I think the change should be reverted. I have reverted the change, but I think it (or rather "something like" it -- *SOME* fix for x**2 raising OverflowError while x*x proceeds happily for a certain range of floats) is important and should go back in again. Using Py_HUGE_VAL instead of trusting pow's result might be OK, for example, if the 1990 C Standard is that much weaker on this than 1999's and we need to stick to it anyway. But the problem is that testing this sensibly requires fpconst (from PEP 754) or the equivalent, otherwise it may be impossible to write unittest to show the presence or absence of this bug. Alex From brett at python.org Thu Aug 24 02:30:55 2006 From: brett at python.org (Brett Cannon) Date: Wed, 23 Aug 2006 17:30:55 -0700 Subject: [Python-Dev] PEP 362 open issues In-Reply-To: References: Message-ID: On 8/23/06, Georg Brandl wrote: > > Brett Cannon wrote: > > I have been spending my Google sprint time on writing and implementing > > PEP 362 (http://www.python.org/dev/peps/pep-0362/), which finally > > defines Signature objects for functions. With the implementation at a > > place I am happy with, I wanted to ask about the open issues with the > PEP. > > While reading the PEP, I stumbled over the description of the bind() > method. > > As I understand the implementation patch, the parameter names will be keys > and > the arguments values in the resulting dictionary, but this sentence looks > to > me as if it describes it the opposite way (it also doesn't clarify that > "parameter" means "parameter name"): > > * bind(*args, **kwargs) -> dict > Create a mapping from argument to parameter for the signature (see > `Open Issues`_ for question of how to handle tuples). You're right, it's a typo. Another question: could it be helpful to make Parameter.default_value a > weak reference? Perhaps, but I don't think it is necessarily required. I can change it if others want to go that way, but for now I am happy with the way it is. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060823/d557359c/attachment.html From pje at telecommunity.com Thu Aug 24 02:31:56 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 23 Aug 2006 20:31:56 -0400 Subject: [Python-Dev] GeneratorExit is unintuitive and uneccessary In-Reply-To: References: <5.1.1.6.0.20060823102150.03a90638@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20060823195222.022a1f40@sparrow.telecommunity.com> At 04:10 PM 8/23/2006 -0700, Guido van Rossum wrote: >IIUC this is how return currently works -- in some sense it's an >exception, but it can't be caught, and it won't escape from the >current frame. Ditto for break/continue. > >The generator extensions are still very young, and I'm not against >changing them somewhat in 2.6 or py3k. Perhaps GeneratorExit can be >replaced by a treatment similar to the treatment of return. But I'm >not sure it helps with the original problem; It depends on what you're referring to as "the original problem". If you mean the problem Bob Ippolito opened a bug for, then yes, it does not solve that problem. But I think we agree that *that* problem doesn't actually need solving. However, it *does* solve the problem(s) that Igor Bukanov brought up, namely that GeneratorExit is unintuitive and unnecessary. :) It also fixes the problem of calling close() or GC'ing a generator written for an earlier version of Python, that yields inside a try-except. >In a generator, a yield >in the finally clause still needs to be dealt with somehow. And we should keep the existing PEP-defined behavior for dealing with it. That is, my understanding is that the only part of close()'s definition that would change is the 'throw(GeneratorExit)' bit. I just wish this had been proposed some time much sooner, like maybe not long after I did the original implementation of throw()/close()/etc. During the original discussion, I'd have been more skeptical of a return-based approach. But knowing what I know now about the implementation, making it based on return instead would be trivial. A proof-of-concept patch I just whipped up adds 7 lines to the hook code in ceval.c (to determine whether WHY_RETURN or WHY_EXCEPTION should be used), and 5 lines to gen_close() in genobject.c. From pje at telecommunity.com Thu Aug 24 02:43:31 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 23 Aug 2006 20:43:31 -0400 Subject: [Python-Dev] GeneratorExit is unintuitive and uneccessary In-Reply-To: References: <5.1.1.6.0.20060823102150.03a90638@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20060823203943.026ce300@sparrow.telecommunity.com> At 01:44 AM 8/24/2006 +0200, Igor Bukanov wrote: >Regarding yield in the finally problem: > >The current implementation throws RuntimeException after the yield >transfer the control back to the close method. If that is changed to >reporting a bad behavior and rethrowing GeneratorExit or its hidden >analog as suggested at the point of yield inside the generator, that >would nicely unwind possible outer finally blocks and close the >generator in any case. This approach has already been proposed and rejected multiple times. A badly-behaving generator is broken and asking it to handle the error of its own brokenness will lead only to grief. Once the generator yields inappropriately, it has proven it cannot be trusted. It is only compounding the generator's insanity to ask it to handle this error! From sreeram at tachyontech.net Thu Aug 24 12:58:33 2006 From: sreeram at tachyontech.net (K.S.Sreeram) Date: Thu, 24 Aug 2006 16:28:33 +0530 Subject: [Python-Dev] Can LOAD_GLOBAL be optimized to a simple array lookup? In-Reply-To: References: <44ECC72F.10303@tachyontech.net> Message-ID: <44ED8659.7090602@tachyontech.net> Alex Martelli wrote: > > On Aug 23, 2006, at 2:22 PM, K.S.Sreeram wrote: > >> Hi all, >> >> I noticed in Python/ceval.c that LOAD_GLOBAL uses a dictionary lookup, >> and was wondering if that can be optimized to a simple array lookup. >> >> If i'm right there are 3 kinds of name lookups: locals, outer >> scopes(closures), and globals. (not counting attribute lookup). Locals >> are identified by, either the presence of assignments, or their presence >> in the arg list. So all name lookups can be classified into the 3 types >> at compile/load time. >> >> Since we know, at load time, which names are global.. Can't we simply >> build a global name table and replace LOAD_GLOBALs with a lookup at the >> corresponding index into the global name table? > > At the time the function's body gets compiled, the global (or builtin) > it's trying to access might or might not be there -- as long as it gets > added afterwards, before the function's body gets _executed_, no problem > (in today's scheme). It's not obvious to me how you could compile a > ``corresponding index'' into the LOAD_GLOBAL opcode, since that index is > in general unknown at compile time. How about this approach: (disclaimer: this is just a rough sketch!) At compile time: --------------- - When compiling the module, a list of global names is built. This list is populated in the order in which the names appear in the module. References to the global will be replaced by references to the index in this list. - Instead of generating LOAD_GLOBAL and STORE_GLOBAL opcodes, we generate LOAD_GLOBAL_BY_INDEX and STORE_GLOBAL_BY_INDEX, which are new opcodes. The indexes used with the opcode will be the index of the name from the list. - This list of names should be stored in the pyc file so that the ordering is remembered. At load time: ------------- - At module load time, we build an array of PyObject*, which is used to store the values for the globals. The values will be NULL, until the actual binding is created. - For each name in the global_name_list in the pyc file, we pass the pointer to the corresponding entry in the global table, to the dict. PyDict_SetLocationForName( module_dict, name, PyObject **ptr_to_entry_table ) This is done so that 'get' and 'set' of these names in the dict will update/fetch from the global table directly. This approach should result in faster access to globals defined within the module. It will still work if we try to shadow any builtins. As far as i see, this doesn't seem to change any existing semantics either. This does not result in faster access to existing builtins. But a similar scheme can be worked out for that too. > >> The module's dict object will need to be special so that whenever a name >> gets 'set', the global name table should get updated. > > It seems that you'd need to chase down and modify all of the LOAD_GLOBAL > opcodes too, at every such modification. (the concept of modifying > builtins becomes extremely scary...). Considering the amortized speed > of a dict lookup for an interned string (hash value cached and thus > immediately available, equality comparison with other interned string a > single machine-level operation), it's not clear to me that the huge > complexity (and potential performance impact) of all this could ever > possibly be justified. > > A change in Python semantics allowing some level of "nailing down" of > builtins (and perhaps globals too) *COULD* easily yield large > performance benefits, but that's a subject for the Python-3000 mailing > list; as long as builtins and globals stay as fluid as today, I'm > skeptical on the optimization opportunities here. [sreeram;] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20060824/c3c8699e/attachment.pgp From ncoghlan at gmail.com Thu Aug 24 13:55:16 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 24 Aug 2006 21:55:16 +1000 Subject: [Python-Dev] PEP 362 open issues In-Reply-To: References: Message-ID: <44ED93A4.3010601@gmail.com> Brett Cannon wrote: > Another question: could it be helpful to make Parameter.default_value a > weak reference? > > > Perhaps, but I don't think it is necessarily required. I can change it > if others want to go that way, but for now I am happy with the way it is. Leave it as a normal reference - if I have a reference to a Signature object, I'd like all it's attributes to stick around, thank you very much :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From martin at v.loewis.de Thu Aug 24 16:30:11 2006 From: martin at v.loewis.de (martin at v.loewis.de) Date: Thu, 24 Aug 2006 16:30:11 +0200 Subject: [Python-Dev] Can LOAD_GLOBAL be optimized to a simple arraylookup? In-Reply-To: <44ED8659.7090602@tachyontech.net> References: <44ECC72F.10303@tachyontech.net> <44ED8659.7090602@tachyontech.net> Message-ID: <1156429811.44edb7f34dbde@www.domainfactory-webmail.de> Zitat von "K.S.Sreeram" : > How about this approach: > (disclaimer: this is just a rough sketch!) This is actually the problem. There are many fine details which can affect performance and correctness. So about there are only two sensible ideas to treat such ideas: either implement them, or ignore them. If you think your approach could work, please try to implement it. It's not that anybody is objecting the goal; there is just debate about the feasibility of the implementation. So if you can come up with an implementation, we are in a much better position to praise or criticise the approach: it then becomes possible to study whether the implementation is really compatible with the current Python, and whether it really does improve performance. Regards, Martin From nnorwitz at gmail.com Thu Aug 24 17:43:38 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 24 Aug 2006 11:43:38 -0400 Subject: [Python-Dev] xrange accepting non-ints Message-ID: I've been working on enhancing xrange and there are a bunch of issues to consider. I've got pretty much complete implementations in both C and Python. Currently xrange is 2 objects: range and the iter. These only work on C longs. Here's what I propose: 2.6: * Add deprecation warning if a float is passed to xrange (currently silently truncates) * Allow int, long, float, or obj.__index__ * Implement xrange in python * Implement iterator over C longs (or Py_ssize_t) in C * Implement iterator over Python longs in Python (* may lose __length_hint__) * Capture the values on construction, so mutating objects wouldn't affect xrange The idea is to expand xrange's capabilities so that it can replace range in 3k. I've profiled various combinations. Here are the various results normalized doing xrange(0, 1e6, 1): Run on all integer (32-bit) values for start, step, end: C xrange and iter: 1 Py xrange w/C iter: 1 Py xrange w/Py iter (gen): 5-8 Py xrange w/Py iter (class): ~30 So having xrange in python is the same speed as if xrange is written in C. The important part is that the iterator needs to be written in C for speed. If we use a generator, something like: while value < end: yield value value += step The result is ~5 times slower in a release build and 8 times slower in a debug build than with an iterator implemented in C. Using a generator means that there is no __length_hint__. If we go with a full class that has a __length_hint__ the result was ~32 times slower in a debug build. The Python impl is about 1/10th the size of the C impl, though is lacking some comments. Run on Python longs the result is somewhat interesting. The Python based iterator is faster. There's probably a bug in the C version, but given that there is a lot of object allocation, I wouldn't expect the C version to ever be much faster than a similar Python version. Plus the Python version is trivial (same as above) for ints or longs. The C version for longs is quite a bit of code. Run on all Python longs (still 0..1e6, but sys.maxint..(sys.maxint + 1e6) is the same): C xrange and iter: 1.4 Py xrange w/C iter: not tested Py xrange w/Py iter (gen): 1 Py xrange w/Py iter (class): 4 Caveats: * The generator version above doesn't support floats. We could easily support floats with a different calculation that would be slightly more expensive, but not have accumulated error. * By using the generator version, __length_hint__ gets dropped. This means that converting the iterator into a sequence could be slightly more costly as we have to increase the allocation. This would only happen if any of start, step, end weren't an int. * With a python implementation there is a little bit of bootstraping that is necessary to get the iter implemented in C into the xrange object implemented in Python * Since xrange is implemented in Python, it can be changed. * The Python code is much easier to understand than the C code (there is at least one bug in the current C version where -sys.maxint -1 isn't always displayed properly). Hopefully this is all understandable. If I left anything out, Thomas will remind me. n From guido at python.org Thu Aug 24 17:58:19 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 24 Aug 2006 08:58:19 -0700 Subject: [Python-Dev] xrange accepting non-ints In-Reply-To: References: Message-ID: On 8/24/06, Neal Norwitz wrote: > I've been working on enhancing xrange and there are a bunch of issues > to consider. I've got pretty much complete implementations in both C > and Python. Currently xrange is 2 objects: range and the iter. > These only work on C longs. Here's what I propose: > > 2.6: > * Add deprecation warning if a float is passed to xrange (currently > silently truncates) > * Allow int, long, float, or obj.__index__ float? I thought the first bullet says no float? > * Implement xrange in python Since xrange is used in performance critical apps that may be a bad idea. Or maybe only if the args aren't all ints? > * Implement iterator over C longs (or Py_ssize_t) in C > * Implement iterator over Python longs in Python (* may lose __length_hint__) > * Capture the values on construction, so mutating objects wouldn't > affect xrange Right. So capture them as Python int or long only. > The idea is to expand xrange's capabilities so that it can replace range in 3k. > > I've profiled various combinations. Here are the various results > normalized doing xrange(0, 1e6, 1): > > Run on all integer (32-bit) values for start, step, end: > C xrange and iter: 1 > Py xrange w/C iter: 1 > Py xrange w/Py iter (gen): 5-8 > Py xrange w/Py iter (class): ~30 > > So having xrange in python is the same speed as if xrange is written > in C. I'm not sure I believe this benchmark -- to measure the cost of xrange itself (as opposed to the cost of iterating over the iterator) you should test xrange(0) or xrange(1). > The important part is that the iterator needs to be written in > C for speed. If we use a generator, something like: > > while value < end: > yield value > value += step > > The result is ~5 times slower in a release build and 8 times slower in > a debug build than with an iterator implemented in C. Nobody cares about the speed of the debug build. :-) > Using a > generator means that there is no __length_hint__. If we go with a > full class that has a __length_hint__ the result was ~32 times slower > in a debug build. I don't mind not having the length hint for longs. Doesn't current xrange also support a faster version of reversed? > The Python impl is about 1/10th the size of the C impl, though is > lacking some comments. > > Run on Python longs the result is somewhat interesting. The Python > based iterator is faster. There's probably a bug in the C version, > but given that there is a lot of object allocation, I wouldn't expect > the C version to ever be much faster than a similar Python version. > Plus the Python version is trivial (same as above) for ints or longs. > The C version for longs is quite a bit of code. If the longs are large enough, the addition is going to dominate the cost, yes. > Run on all Python longs (still 0..1e6, but sys.maxint..(sys.maxint + > 1e6) is the same): > C xrange and iter: 1.4 > Py xrange w/C iter: not tested > Py xrange w/Py iter (gen): 1 > Py xrange w/Py iter (class): 4 > > Caveats: > * The generator version above doesn't support floats. We could > easily support floats with a different calculation that would be > slightly more expensive, but not have accumulated error. Is there a good use case for supporting float? The problem with floats is that even apart from accumulated error, it's still ambiguous. E.g. will xrange(1.1, 1.9, 0.1) include the end point or not? That would depend on the details of decimal-to-binary conversion. > * By using the generator version, __length_hint__ gets dropped. This > means that converting the iterator into a sequence could be slightly > more costly as we have to increase the allocation. This would only > happen if any of start, step, end weren't an int. Fine with me -- you can't do that at all at the moment. :-) > * With a python implementation there is a little bit of bootstraping > that is necessary to get the iter implemented in C into the xrange > object implemented in Python Long-term, I'd rather see it implemented all in C. Short term, the Python implementation is great to experiment. > * Since xrange is implemented in Python, it can be changed. > * The Python code is much easier to understand than the C code (there > is at least one bug in the current C version where -sys.maxint -1 > isn't always displayed properly). > > Hopefully this is all understandable. If I left anything out, Thomas > will remind me. I'm about to head out to Google now... -- --Guido van Rossum (home page: http://www.python.org/~guido/) From thomas at python.org Thu Aug 24 18:10:32 2006 From: thomas at python.org (Thomas Wouters) Date: Thu, 24 Aug 2006 18:10:32 +0200 Subject: [Python-Dev] xrange accepting non-ints In-Reply-To: References: Message-ID: <9e804ac0608240910y5c95734cx841f92d42ecece73@mail.gmail.com> On 8/24/06, Guido van Rossum wrote: > > On 8/24/06, Neal Norwitz wrote: > > I've been working on enhancing xrange and there are a bunch of issues > > to consider. I've got pretty much complete implementations in both C > > and Python. Currently xrange is 2 objects: range and the iter. > > These only work on C longs. Here's what I propose: > > > > 2.6: > > * Add deprecation warning if a float is passed to xrange (currently > > silently truncates) > > * Allow int, long, float, or obj.__index__ > > float? I thought the first bullet says no float? No, the bullet says 'add warning' :) xrange() currently accepts floats, because it uses one of the integer getargs formats: >>> xrange(1.2, 2.5, 1.9999) xrange(1, 2) > * Implement xrange in python > > Since xrange is used in performance critical apps that may be a bad > idea. Or maybe only if the args aren't all ints? Is the cost of *calling* xrange() really a big issue? I don't think Neal measured this, but he could. I'd imagine performance-critical apps call xrange() once, then use that to iterate. > > > Caveats: > > * The generator version above doesn't support floats. We could > > easily support floats with a different calculation that would be > > slightly more expensive, but not have accumulated error. > > Is there a good use case for supporting float? The problem with floats > is that even apart from accumulated error, it's still ambiguous. E.g. > will xrange(1.1, 1.9, 0.1) include the end point or not? That would > depend on the details of decimal-to-binary conversion. Supporting floats is definately problematic. It would be nice if xrange() supported arbitrary numeric types, though, like decimals. That would quench the thirst people seem to have for float-ish xranges. > > * With a python implementation there is a little bit of bootstraping > > that is necessary to get the iter implemented in C into the xrange > > object implemented in Python > > Long-term, I'd rather see it implemented all in C. Short term, the > Python implementation is great to experiment. Why, other than performance? It's a lot simpler and much easier to get right in Python, which is quite good for maintenance, too. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060824/eca0c56f/attachment-0001.htm From fredrik at pythonware.com Thu Aug 24 18:14:01 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 24 Aug 2006 18:14:01 +0200 Subject: [Python-Dev] xrange accepting non-ints In-Reply-To: References: Message-ID: Neal Norwitz wrote: > I've profiled various combinations. Here are the various results > normalized doing xrange(0, 1e6, 1): > > Run on all integer (32-bit) values for start, step, end: > C xrange and iter: 1 > Py xrange w/C iter: 1 in real life, loops are a lot shorter than that. if you take that into account, you don't even have to run the benchmark to realize that calling a Python function and checking the arguments before calling a C function takes more time than calling a C function. even if you skip the "check the arguments" part, you take a 5% hit: > timeit -s"def myxrange(a,xrange=xrange): return xrange (a)" "for i in myxrange(100): pass" 100000 loops, best of 3: 5.28 usec per loop > timeit "for i in xrange(100): pass" 100000 loops, best of 3: 4.98 usec per loop > timeit -s"def myxrange(a,b=None,c=None,xrange=xrange): return xrange(a,b,c)" "for i in myxrange(0,100,1): pass" 100000 loops, best of 3: 5.58 usec per loop > timeit "for i in xrange(0,100,1): pass" 100000 loops, best of 3: 5.27 usec per loop I doubt adding more code to the myxrange function will speed it up... From tim.peters at gmail.com Thu Aug 24 18:29:50 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 24 Aug 2006 12:29:50 -0400 Subject: [Python-Dev] Can LOAD_GLOBAL be optimized to a simple array lookup? In-Reply-To: References: <44ECC72F.10303@tachyontech.net> Message-ID: <1f7befae0608240929x1f839d59wc9887748fd4143b3@mail.gmail.com> Note that there are already three PEPs related to speeding dict-based namespace access; start with: http://www.python.org/dev/peps/pep-0280/ which references the other two. The "normal path" through dict lookups got faster since there was a rash of those, to the extent that more complication elsewhere got much less attractive. It's possible that dict lookups got slower again since /then/, though. From sreeram at tachyontech.net Thu Aug 24 19:23:32 2006 From: sreeram at tachyontech.net (K.S.Sreeram) Date: Thu, 24 Aug 2006 22:53:32 +0530 Subject: [Python-Dev] Can LOAD_GLOBAL be optimized to a simple array lookup? In-Reply-To: <1f7befae0608240929x1f839d59wc9887748fd4143b3@mail.gmail.com> References: <44ECC72F.10303@tachyontech.net> <1f7befae0608240929x1f839d59wc9887748fd4143b3@mail.gmail.com> Message-ID: <44EDE094.7000201@tachyontech.net> Tim Peters wrote: > Note that there are already three PEPs related to speeding dict-based > namespace access; start with: > > http://www.python.org/dev/peps/pep-0280/ > > which references the other two. > > The "normal path" through dict lookups got faster since there was a > rash of those, to the extent that more complication elsewhere got much > less attractive. It's possible that dict lookups got slower again > since /then/, though. Thanks! Looks like i'm only proposing what is already there in pep-267 by Jeremy Hylton. http://www.python.org/dev/peps/pep-0267/ cheers [sreeram;] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20060824/76d6c88a/attachment.pgp From nnorwitz at gmail.com Thu Aug 24 20:52:12 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 24 Aug 2006 14:52:12 -0400 Subject: [Python-Dev] for 2.5 issues Message-ID: I don't want to make any more changes to 2.5 unless we absolutely have to. I also don't want to lose fixes. How about for anything that should be resolved in 2.5, but wait for 2.5.1 we set the tracker item to: Group 2.5, Resolution: Later, Priority 7. Then it should be easy to find these things. Thanks, n From brett at python.org Thu Aug 24 21:21:06 2006 From: brett at python.org (Brett Cannon) Date: Thu, 24 Aug 2006 12:21:06 -0700 Subject: [Python-Dev] draft for bug guidelines Message-ID: Below is the current draft of a set of bug guidelines for people to follow when they want to file a bug. The hope is that when we get an email asking "how do I file a bug?" we can point them towards these sets of guidelines for most issues. Let me know about any errors and such. This will (hopefully) end up going up on python.org/dev/ eventually. ------------------------- About /////////////////////////////////////////////////////////// These sets of guidelines are to help you file a bug report for the Python programming language on SourceForge_. If your bug is not for the language but for a third-party application, please report the bug to that third-party. *Please make sure to follow every step as it will make the lives of the Python developers much easier!!!* .. contents:: Steps to File a Bug Report /////////////////////////////////////////////////////////// Get a SourceForge account ======================================= In order to file a bug report, you must have an account_ on SourceForge_. We realize some people would like to have anonymous bug reporting for various reasons (anonymity, ease or reporting, etc.). But the Python developers made the decision to turn off anonymous posting so as to make sure you are notified by email when any action is taken on your report. This can be very important if a Python developer needs more information from you about the bug. Start a new bug ======================================= You must be logged into SourceForge to file a bug! See `Get a SourceForge account`_ if you do not have one. Go to the `SourceForge bug page`_ to start a new bug report. There you will find a link called `Submit New`_. Click on it and it will allow you to fill out a new bug report. Once you click on the link, you are presented with a page that has several fields for you to fill in. Here is what to do for each field: * Private Leave this unchecked. * Category Set this to the area that the bug is related to (e.g., documentation, build, etc.). * Group Usually this is set to the major.minor version of Python that you found the bug in. * Summary A one-liner describing the problem so as to make it easy for developers to spot whether they have the expertise needed to work on the bug; it also ends up being displayed as the title on index and search pages. * Detailed Description Following sections of this document discuss what should go in here. * Upload and Attach a File If you are going to upload a file, either type in the path to the file or click the ``Browse...`` button to find the file. * File Description A one-liner describing the file (e.g., how it is different from previously uploaded files); no date info is needed since the upload is timestamped. Specify the Python version ======================================= It is important that we have the most accurate version number of the interpreter you are using in order to best diagnose the issue. There are two ways to get us the version information. If you can run your Python interpreter, execute the following lines at at a shell prompt and paste the result into the ``Detailed Description`` field of the bug report:: python -c "import sys; print sys.version" If you are running a version of Python older than 2.5 and are working from a source checkout of Python, the please also report the Subversion revision number for the root of your checkout:: svnversion . If your bug is preventing you from running the interpreter, execute Python with the ``-V`` command-line flag and paste the output:: python -V Special settings for your Python interpreter ============================================ Sometimes your environment influences a bug and thus needs to be reported to help find the problem. This means we need to have reported: * Operating System * Environment Variables + ``PYTHONSTARTUP`` If this is set and might be causing the issue, please either upload the file or state what it does. + ``PYTHONCASEOK`` If your bug is on Windows and involves importing, please report if this environment variable is set or not. * ``site-packages`` If you have any third-party packages installed that may be contributing to the bug, please report those. * Custom Patches Any differences between your code and the code the Python developers work off of needs to be reported. Sample code to reproduce bug ======================================== If you can, please upload a file the demonstrates the bug. The more succinct the better! Submit! ======================================== At this point you should have a detailed bug report for developers to work off of. Click the ``Submit`` button and read on to see what you should do after the bug is reported. Respond to requests from developers ======================================== No matter how detailed the bug report, there is always the chance that a developer will need more information to fix a bug. Please be prompt in replying to requests for information by submitting a reply on the bug report. You may be asked to test out a patch. It is very important that you help with this, especially if the bug is not reproducible by the developer working on it. Write a patch! ======================================== If you are so inclined, patches for your own bug reports are always helpful! Please make sure to reference the tracker item number in the patch details. Also add a follow-up to the bug referencing the patch. .. _SourceForge: http://www.sourceforge.net/ .. _SourceForge bug page: http://sourceforge.net/tracker/?group_id=5470&atid=105470 .. _account: http://sourceforge.net/account/newuser_emailverify.php .. _Submit New: http://sourceforge.net/tracker/?func=add&group_id=5470&atid=105470 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060824/a246130c/attachment.html From brett at python.org Thu Aug 24 21:22:42 2006 From: brett at python.org (Brett Cannon) Date: Thu, 24 Aug 2006 12:22:42 -0700 Subject: [Python-Dev] draft of patch guidelines Message-ID: Below is a draft for a set of patch guidelines. This is meant for that times where people ask "how do a create a patch for a change I made" or where to point people if they created a patch but it comes up short (no tests, etc.). Hopefully this will go up on python.org/dev/ . Let me know of any errors and such. -------------------------- About ///// This set of guidelines is meant for people who have written code that they wish to have committed to the Python programming language. This means that this set of guidelines assumes you have already written code and thus just want to know the steps required to generate the patch necessary to submit your change for addition in Python. The assumption you are using a UNIX-style shell is also made. If you need help with any of these steps, please ask for it! Usually you can create a new patch missing a few steps and the Python developers will try to help you complete it so that your patch can eventuallly be accepted. Guidelines for committing code to Python will be contained in a separate document. .. contents:: Steps to Creating a Patch ///////////////////////// Read the Developer Intro to understand the scope of your proposed change ======================================================================== Changes to Python come in various levels of scope. You could be fixing a simple bug in the standard library. You might be trying to add a new feature to a module. You might even be suggesting a new feature to the language. All three have a different amount of scope and thus require varying amounts of work to get approval. Please read the `Developer Intro`_ doc to make sure you realize the proper scope of what you are doing. It also tells you the rough steps needed for varying types of changes to be accepted. Add the appropriate unit tests ============================== Python prides itself on being a very stable piece of software. We maintain this by having a regression test suite that contains a huge amount of unit tests that exercise both the language and the standard library. Any change you make to Python requires a unit test. If your patch is to fix a bug, then a test exercising that bug and your fix is needed. If it is a new feature then an entirely new regression test might be needed. The important thing is that your new code is exercised within Python's test suite. Add the proper document changes =============================== If your patch will change or add new semantics, you must update the documentation. If your code clarifies some ambiguous behaviour, you must update the docs. The only time you do not need to touch the documentation is if your patch fixes a bug or adds new unit tests. Make your code follow the style guidelines ========================================== `PEP 7`_ and `PEP 8`_ are the style guidelines for C and Python code, respectively. Your code must conform to the style guidelines before it is accepted. Generate a patch ================ In general, people get their copy of the Python source from two locations: a svn checkout or from downloaded source. Both have slightly different steps in terms of generating patches. Working from a svn checkout is the best way to create a patch. It makes it easier on the Python developers since there is less chance of any merge problems. It also raises the chances that a developer will be willing to put the time and effort into working with you and your patch. If you need help with working with svn and Python's repository, see the `Developer FAQ`_. Working from downloaded source is so inferior to working with a svn checkout that it will not be discussed here (although patches *will* be accepted against downloaded code). First step is to update your svn checkout with ``svn update``. Once you have done that and resolved the merge conflicts, generate a patch containing the files you have changed. You can see what files you have touched with ``svn status``. If it lists only modified files you want to include in your patch, you can then generate your patch:: svn diff > PATCH_FILE Make sure to name ``PATCH_FILE`` to a reasonable name and be prepared to upload it in a later step. Also, please note what revision number you are working against. This can be found by executing:: python -c "import sys; print sys.version" Create a tracker item on SourceForge ==================================== First make sure you have a `SourceForge account`_. Once you have one and havve logged in, go to the `SF patch tracker`_ for Python and create a new tracker item. In the new tracker item, set the item's fields as so: * Private Leave unchecked. * Category Set to what area of Python your patch affects. * Group Set to the version of Python that the patch was generated against. * Assigned To Leave as-is. * Priority: Leave as-is. * Summary Briefly describe what the patch does. * Detailed Description Describe in more detail what the patch does. Make sure to mention the revision number the patch is against here. Also mention any bugs that the patch fixes. * Upload and Attach a File Point to the patch you generated. * File Description Mention it is the initial version of the patch. If you end up uploading newer versions of your patch you then specify how it is different from the previous one. Reference the patch in proper bug reports ========================================= If your patch fixes a bug, add a follow-up to that bug report referencing the patch. If there is not bug report but your patch does fix a bug, then quickly create a bug report. Wait for a developer to contact you =================================== At this point you need to wait for a Python developer to come along and look at your patch. This might be a while as all Python developers are volunteers and thus have a limited amount of time. Do not be surprised if it takes weeks or more before your patch is examined. There is a standing offer that if you review five other patches (by commenting on its correctness, helping it meet these guidelines, etc.), then a developer will definitely have a look at any patch you specify. Just email python-dev with what five patches you reviewed (and what you did for the review) and the patch you want examined. .. _Developer Intro: http://www.python.org/dev/intro/ .. _PEP 7: http://www.python.org/dev/peps/pep-0007/ .. _PEP 8: http://www.python.org/dev/peps/pep-0008/ .. _Developer FAQ: http://www.python.org/dev/faq .. _SourceForge account: http://sourceforge.net/account/newuser_emailverify.php .. _SF patch tracker: http://sourceforge.net/tracker/?group_id=5470&atid=305470 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060824/85d3792f/attachment.htm From guido at python.org Thu Aug 24 21:22:52 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 24 Aug 2006 12:22:52 -0700 Subject: [Python-Dev] Need help with test_mutants.py Message-ID: There's a unit test "test_mutants" which I don't understand. If anyone remembers what it's doing, please contact me -- after ripping out dictionary ordering in Py3k, it stops working. In particular, the code in test_one() requires changes, but I don't know how... Please help! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From phd at mail2.phd.pp.ru Thu Aug 24 21:29:45 2006 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Thu, 24 Aug 2006 23:29:45 +0400 Subject: [Python-Dev] draft for bug guidelines In-Reply-To: References: Message-ID: <20060824192945.GA32475@phd.pp.ru> On Thu, Aug 24, 2006 at 12:21:06PM -0700, Brett Cannon wrote: > Start a new bug "Before starting a new bug please try to search if the bug has already been reported. It it has - do not start a new report, add your comments to the existing report." Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From phd at mail2.phd.pp.ru Thu Aug 24 21:33:58 2006 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Thu, 24 Aug 2006 23:33:58 +0400 Subject: [Python-Dev] draft of patch guidelines In-Reply-To: References: Message-ID: <20060824193358.GB32475@phd.pp.ru> On Thu, Aug 24, 2006 at 12:22:42PM -0700, Brett Cannon wrote: > Read the Developer Intro to understand the scope of your proposed change "Search through the PEPs, developer mailing lists and patches. Has a similar patch already been proposed? Has it been accepted, postponed or rejected? Learn the reasons why it's been postponed or rejected." Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From tim.peters at gmail.com Thu Aug 24 21:46:30 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 24 Aug 2006 15:46:30 -0400 Subject: [Python-Dev] Need help with test_mutants.py In-Reply-To: References: Message-ID: <1f7befae0608241246m7e6e54b9o265362752ab5dd61@mail.gmail.com> [Guido] > There's a unit test "test_mutants" which I don't understand. If anyone > remembers what it's doing, please contact me -- after ripping out > dictionary ordering in Py3k, Is any form of dictionary comparison still supported, and, if so, what does "dict1 cmp_op dict2" mean now? > it stops working. Traceback? > In particular, the code in test_one() requires changes, but I don't > know how... Please help! The keys and values of dict1 and dict2 are filled with objects of a user-defined class whose __cmp__ method randomly mutates dict1 and dict2. dict1 and dict2 are initially forced to have the same number of elements, so in current Python: c = cmp(dict1, dict2) triggers a world of pain, with the internal dict code doing fancy stuff comparing keys and values. However, every key and value comparison /may/ mutate the dicts in arbitrary ways, so this is testing whether the dict comparison implementation blows up (segfaults, etc) when the dicts it's comparing mutate during comparison. If it's only ordering comparisons that have gone away for dicts, then, e.g., replacing c = cmp(dict1, dict2) with c = dict1 == dict2 instead will still meet the test's intent. No particular /result/ is expected. The test passes if and only if Python doesn't crash. When the test was introduced, it uncovered at least six distinct failure (crashing) modes across the first 20 times it was run, so it's well worth keeping around in some form. From brett at python.org Thu Aug 24 22:46:35 2006 From: brett at python.org (Brett Cannon) Date: Thu, 24 Aug 2006 13:46:35 -0700 Subject: [Python-Dev] draft of patch guidelines In-Reply-To: <20060824193358.GB32475@phd.pp.ru> References: <20060824193358.GB32475@phd.pp.ru> Message-ID: On 8/24/06, Oleg Broytmann wrote: > > On Thu, Aug 24, 2006 at 12:22:42PM -0700, Brett Cannon wrote: > > Read the Developer Intro to understand the scope of your proposed change > > "Search through the PEPs, developer mailing lists and patches. Has a > similar patch already been proposed? Has it been accepted, postponed or > rejected? Learn the reasons why it's been postponed or rejected." > > Oleg. > -- > Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru > Programmers don't die, they just GOSUB without RETURN. > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060824/7e37de70/attachment.html From brett at python.org Thu Aug 24 22:49:25 2006 From: brett at python.org (Brett Cannon) Date: Thu, 24 Aug 2006 13:49:25 -0700 Subject: [Python-Dev] draft for bug guidelines In-Reply-To: <20060824192945.GA32475@phd.pp.ru> References: <20060824192945.GA32475@phd.pp.ru> Message-ID: Made it the first step. =) -Brett On 8/24/06, Oleg Broytmann wrote: > > On Thu, Aug 24, 2006 at 12:21:06PM -0700, Brett Cannon wrote: > > Start a new bug > > "Before starting a new bug please try to search if the bug has already > been reported. It it has - do not start a new report, add your comments to > the existing report." > > Oleg. > -- > Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru > Programmers don't die, they just GOSUB without RETURN. > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060824/685da05b/attachment.htm From guido at python.org Thu Aug 24 23:03:25 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 24 Aug 2006 14:03:25 -0700 Subject: [Python-Dev] Need help with test_mutants.py In-Reply-To: <1f7befae0608241246m7e6e54b9o265362752ab5dd61@mail.gmail.com> References: <1f7befae0608241246m7e6e54b9o265362752ab5dd61@mail.gmail.com> Message-ID: On 8/24/06, Tim Peters wrote: > [Guido] > > There's a unit test "test_mutants" which I don't understand. If anyone > > remembers what it's doing, please contact me -- after ripping out > > dictionary ordering in Py3k, > > Is any form of dictionary comparison still supported, and, if so, what > does "dict1 cmp_op dict2" mean now? Only == and != are supported between dicts. All the work is done by dict_equal(). > > it stops working. > > Traceback? Not particularly interesting: without changes, the code immediately bombs like this: trying w/ lengths 90 90 . Traceback (most recent call last): File "../Lib/test/test_mutants.py", line 152, in test(100) File "../Lib/test/test_mutants.py", line 149, in test test_one(random.randrange(1, 100)) File "../Lib/test/test_mutants.py", line 135, in test_one c = cmp(dict1, dict2) TypeError: unorderable types: dict() > dict() > > In particular, the code in test_one() requires changes, but I don't > > know how... Please help! > > The keys and values of dict1 and dict2 are filled with objects of a > user-defined class whose __cmp__ method randomly mutates dict1 and > dict2. dict1 and dict2 are initially forced to have the same number > of elements, so in current Python: > > c = cmp(dict1, dict2) > > triggers a world of pain, with the internal dict code doing fancy > stuff comparing keys and values. However, every key and value > comparison /may/ mutate the dicts in arbitrary ways, so this is > testing whether the dict comparison implementation blows up > (segfaults, etc) when the dicts it's comparing mutate during > comparison. > > If it's only ordering comparisons that have gone away for dicts, then, > e.g., replacing > > c = cmp(dict1, dict2) > > with > > c = dict1 == dict2 > > instead will still meet the test's intent. I made that change, and changed class Horrid to define __eq__ instead of __cmp__. Since dict_equal() only invokes PyObject_RichCompareBool() with op==Py_EQ that should be all that's needed. Now when I run it, it spits out an apaprently infinite number of dots. Putting a print in that __eq__ method suggests it is never called. Do you understand this? If I change Horrid.__hash__ to always return 42, I get output like this: trying w/ lengths 12 14 trying w/ lengths 48 52 trying w/ lengths 19 18 trying w/ lengths 10 9 trying w/ lengths 48 46 trying w/ lengths 58 55 trying w/ lengths 50 48 trying w/ lengths 45 50 trying w/ lengths 19 19 . Traceback (most recent call last): File "../Lib/test/test_mutants.py", line 158, in test(100) File "../Lib/test/test_mutants.py", line 155, in test test_one(random.randrange(1, 100)) File "../Lib/test/test_mutants.py", line 141, in test_one c = dict1 == dict2 File "../Lib/test/test_mutants.py", line 99, in __eq__ return self.i == other.i AttributeError: 'Horrid' object has no attribute 'i' Segmentation fault But it doesn't always end with a segfault -- most of the time, the AttributeError is the last thing printed. > No particular /result/ is expected. The test passes if and only if > Python doesn't crash. When the test was introduced, it uncovered at > least six distinct failure (crashing) modes across the first 20 times > it was run, so it's well worth keeping around in some form. Well, it looks like it did provoke another crash, so I'll play with it some more. Thanks! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Aug 24 23:49:15 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 24 Aug 2006 14:49:15 -0700 Subject: [Python-Dev] Need help with test_mutants.py In-Reply-To: References: <1f7befae0608241246m7e6e54b9o265362752ab5dd61@mail.gmail.com> Message-ID: On 8/24/06, Guido van Rossum wrote: > I made that change, and changed class Horrid to define __eq__ instead > of __cmp__. Since dict_equal() only invokes PyObject_RichCompareBool() > with op==Py_EQ that should be all that's needed. > > Now when I run it, it spits out an apaprently infinite number of dots. > Putting a print in that __eq__ method suggests it is never called. Do > you understand this? > > If I change Horrid.__hash__ to always return 42, I get output like this: > > trying w/ lengths 12 14 > trying w/ lengths 48 52 > trying w/ lengths 19 18 > trying w/ lengths 10 9 > trying w/ lengths 48 46 > trying w/ lengths 58 55 > trying w/ lengths 50 48 > trying w/ lengths 45 50 > trying w/ lengths 19 19 . > Traceback (most recent call last): > File "../Lib/test/test_mutants.py", line 158, in > test(100) > File "../Lib/test/test_mutants.py", line 155, in test > test_one(random.randrange(1, 100)) > File "../Lib/test/test_mutants.py", line 141, in test_one > c = dict1 == dict2 > File "../Lib/test/test_mutants.py", line 99, in __eq__ > return self.i == other.i > AttributeError: 'Horrid' object has no attribute 'i' > Segmentation fault > > But it doesn't always end with a segfault -- most of the time, the > AttributeError is the last thing printed. As a follow up to this story line, this appeared to be a refcount bug in dict_equal(). I believe the same bug is probably present in 2.5; it isn't triggered by test_mutants.py because that only exercises dict_compare, not dict_richcompare, and only the latter can call dict_equal (when op==Py_EQ or op==Py_NE). The bug is here: PyObject *key = a->ma_table[i].me_key; /* temporarily bump aval's refcount to ensure it stays alive until we're done with it */ Py_INCREF(aval); bval = PyDict_GetItem((PyObject *)b, key); The fix is to put Py_INCREF(key) // Py_DECREF(key) around the call to PyDict_GetItem(). Apparently what can happen is that the only reference to the key is in the dict, and the evil mutation from the comparison delete the object before the comparison is completely done with it. Should I attempt to reproduce this bug in 2.5 and fix it? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Aug 25 00:06:44 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 24 Aug 2006 15:06:44 -0700 Subject: [Python-Dev] Need help with test_mutants.py In-Reply-To: References: <1f7befae0608241246m7e6e54b9o265362752ab5dd61@mail.gmail.com> Message-ID: On 8/24/06, Guido van Rossum wrote: > Should I attempt to reproduce this bug in 2.5 and fix it? Couldn't help myself. The fix is python.org/sf/1546288 . I set the priority to 8 which means Neal and Anthony will look at it. It's probably okay to reduce the priority to 7 and fix it in 2.5.1. I suspect pre-2.5 versions may have the same bug, as long as they have dict_equal(). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Aug 25 01:08:26 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 24 Aug 2006 16:08:26 -0700 Subject: [Python-Dev] zip -> izip; is __length_hint__ required? Message-ID: At today's sprint, Brian Holmes contributed a patch that implements zip as an interator, a la izip. When reviewing Brian's code, I noticed that he added an implementation of __length_hint__. My gut feeling is that this isn't particularly useful given that zip() is almost exclusively used iteratively, and rarely if ever converted to a list or a tuple. (The one common exception is in the test suite, but there it's almost always a short list, and 3 out of 5 were actually tests for zip or izip.) Should we rip it out or keep it? Also, the existing convention for calling __length_hint__ (e.g. in _PyObject_LengthHint() in abstract.c) seems to be to use PyObject_CallMethod() and suppress TypeError and AttributeError coming out of the call. It would seem to make much more sense to check whether the attribute exists without calling it, and once it exists, just call it and not suppress any exceptions that come out of it. Is there any reason why this shouldn't work? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From chad at zetaweb.com Fri Aug 25 05:01:47 2006 From: chad at zetaweb.com (Chad Whitacre) Date: Thu, 24 Aug 2006 23:01:47 -0400 Subject: [Python-Dev] draft of patch guidelines In-Reply-To: References: Message-ID: <44EE681B.7010601@zetaweb.com> Brett, > Below is a draft for a set of patch guidelines. Thanks for getting around to this! > Wait for a developer to contact you > =================================== > > At this point you need to wait for a Python developer to come along > and look at your patch. This might be a while as all Python > developers are volunteers and thus have a limited amount of time. Do > not be surprised if it takes weeks or more before your patch is > examined. > > There is a standing offer that if you review five other patches (by > commenting on its correctness, helping it meet these guidelines, > etc.), then a developer will definitely have a look at any patch you > specify. Just email python-dev with what five patches you reviewed > (and what you did for the review) and the patch you want examined. May I suggest the following wording for this section? This is basically the feedback I found most helpful a few weeks ago; it's more specific about how notification and the 5-for-1 rule work. ================= When you submit your patch, the tracker notifies a mailing list that most core Python developers subscribe to. But remember that Open Source runs on volunteerism and meritocracy. If you are a new Python contributor, then your patch may languish for weeks or more before anyone volunteers a response. Python addresses this barrier to entry with a "5-for-1 rule." Pick five other patches or bugs, try to apply or reproduce them, and then report your results back to the tracker. Then, send a note to python-dev listing the five you reviewed and the patch id of your own patch, and one of Python's "patch angels" will get the ball rolling on your patch. ================= chad From brett at python.org Fri Aug 25 06:41:37 2006 From: brett at python.org (Brett Cannon) Date: Thu, 24 Aug 2006 21:41:37 -0700 Subject: [Python-Dev] draft of patch guidelines In-Reply-To: <44EE681B.7010601@zetaweb.com> References: <44EE681B.7010601@zetaweb.com> Message-ID: On 8/24/06, Chad Whitacre wrote: > > Brett, > > > Below is a draft for a set of patch guidelines. > > Thanks for getting around to this! Welcome. > Wait for a developer to contact you > > =================================== > > > > At this point you need to wait for a Python developer to come along > > and look at your patch. This might be a while as all Python > > developers are volunteers and thus have a limited amount of time. Do > > not be surprised if it takes weeks or more before your patch is > > examined. > > > > There is a standing offer that if you review five other patches (by > > commenting on its correctness, helping it meet these guidelines, > > etc.), then a developer will definitely have a look at any patch you > > specify. Just email python-dev with what five patches you reviewed > > (and what you did for the review) and the patch you want examined. > > > May I suggest the following wording for this section? This is basically > the feedback I found most helpful a few weeks ago; it's more specific > about how notification and the 5-for-1 rule work. > > ================= > > When you submit your patch, the tracker notifies a mailing list that > most core Python developers subscribe to. But remember that Open Source > runs on volunteerism and meritocracy. If you are a new Python > contributor, then your patch may languish for weeks or more before > anyone volunteers a response. > > Python addresses this barrier to entry with a "5-for-1 rule." Pick five > other patches or bugs, try to apply or reproduce them, and then report > your results back to the tracker. Then, send a note to python-dev > listing the five you reviewed and the patch id of your own patch, and > one of Python's "patch angels" will get the ball rolling on your patch. > > ================= Changed the wording slightly, but I mostly ended up using your text: When you submit your patch, the tracker notifies a mailing list that most core Python developers subscribe to of the creation of your new patch. But remember that Open Source runs on volunteerism and meritocracy. Your patch may languish for weeks or more before anyone volunteers a response because of limited time on the part of Python developers. Python addresses this barrier to entry with a "5-for-1 rule." Pick five other patches or bugs, review them, and then report your results back to the tracker. Then, send a note to python-dev listing the five you reviewed and the patch id of your own patch, and a Python developer will take a look at your patch. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060824/1cbfe71a/attachment.htm From guido at python.org Fri Aug 25 19:44:53 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 25 Aug 2006 10:44:53 -0700 Subject: [Python-Dev] Removing anachronisms from logging module Message-ID: The logging module currently contains some weird-looking code that is there to maintain 1.5.2 compatibility (*). I don't think that's needed any more; neither does Vinay. I propose as a project for someone to clean it up; perhaps compatibility could be maintained with 2.2 but that's probably as far back as it needs to go. Linux distros all come with 2.2 or newer these days. The logging module is currently listed in PEP 291 as requiring compatibility with 1.5.2; my proposal is to remove this or replace it with 2.2 (Vinay even proposed 2.4). (*) I found an example of code testing "if string.find(s, t) != 0", thinking it was a bug attempting to write "if t in s", but which Vinay identified as a 1.5.2 idiom for "if not s.startswith(t)"... -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik at pythonware.com Sat Aug 26 00:16:38 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sat, 26 Aug 2006 00:16:38 +0200 Subject: [Python-Dev] Removing anachronisms from logging module In-Reply-To: References: Message-ID: Guido van Rossum wrote: > (*) I found an example of code testing "if string.find(s, t) != 0", > thinking it was a bug attempting to write "if t in s", but which Vinay > identified as a 1.5.2 idiom for "if not s.startswith(t)"... and as noted on the py3k list, not a very good one, given that the usual way to write that under 1.5.2 is a lot faster (*). *) even faster than "startswith", in many cases. From guido at python.org Sat Aug 26 00:46:49 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 25 Aug 2006 15:46:49 -0700 Subject: [Python-Dev] Removing anachronisms from logging module In-Reply-To: References: Message-ID: I prefer to focus on "clearer" and "less error-prone" rather than "faster" in most cases. I do agree it's a minor embarrassment that s.startswith(t) is slower than s[:len(t)]==t; but I think we should make startswith() faster rather than recommending the other idiom. (Also, for longer substrings startswith wins -- but admittedly rather long, like 300 chracters.) --Guido On 8/25/06, Fredrik Lundh wrote: > Guido van Rossum wrote: > > > (*) I found an example of code testing "if string.find(s, t) != 0", > > thinking it was a bug attempting to write "if t in s", but which Vinay > > identified as a 1.5.2 idiom for "if not s.startswith(t)"... > > and as noted on the py3k list, not a very good one, given that the usual > way to write that under 1.5.2 is a lot faster (*). > > > > *) even faster than "startswith", in many cases. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik at pythonware.com Sat Aug 26 01:01:38 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sat, 26 Aug 2006 01:01:38 +0200 Subject: [Python-Dev] Removing anachronisms from logging module References: Message-ID: Guido van Rossum wrote: > I prefer to focus on "clearer" and "less error-prone" rather than > "faster" in most cases. well, given that "startwith" is by far the most common typo I make when writing Python programs these days, I'm not so sure about that error-proneness thing, but never mind... > I do agree it's a minor embarrassment that s.startswith(t) is slower > than s[:len(t)]==t; but I think we should make startswith() faster > rather than recommending the other idiom. absolutely (see the py3k list for some related discussion). From amk at amk.ca Sat Aug 26 17:58:22 2006 From: amk at amk.ca (A.M. Kuchling) Date: Sat, 26 Aug 2006 11:58:22 -0400 Subject: [Python-Dev] Removing anachronisms from logging module In-Reply-To: References: Message-ID: <20060826155822.GA5291@Siri.local> On Fri, Aug 25, 2006 at 10:44:53AM -0700, Guido van Rossum wrote: > any more; neither does Vinay. I propose as a project for someone to > clean it up ... There's a list of simple tasks in the wiki at . --amk From howarth at bromo.msbb.uc.edu Sat Aug 26 21:06:00 2006 From: howarth at bromo.msbb.uc.edu (Jack Howarth) Date: Sat, 26 Aug 2006 15:06:00 -0400 (EDT) Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows Message-ID: <20060826190600.0E75911002B@bromo.msbb.uc.edu> I discovered that gcc 4.2 exposes a flaw with signed integer overflows in python. This bug and the necessary fix has been discussed in detail on the gcc mailing list. I have filed a detailed bug report and the recommended patch proposed by the gcc developers. This problem should be addressed BEFORE python 2.5 is released. The bug report is... [ 1545668 ] gcc trunk (4.2) exposes a signed integer overflows in the python sourceforge bug tracker. Thanks in advance for attempting to fix this before Python 2.5 is released. Jack From guido at python.org Sat Aug 26 23:01:18 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 26 Aug 2006 14:01:18 -0700 Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows In-Reply-To: <20060826190600.0E75911002B@bromo.msbb.uc.edu> References: <20060826190600.0E75911002B@bromo.msbb.uc.edu> Message-ID: On 8/26/06, Jack Howarth wrote: > I discovered that gcc 4.2 exposes a flaw with > signed integer overflows in python. This bug and the > necessary fix has been discussed in detail on the gcc > mailing list. I have filed a detailed bug report and > the recommended patch proposed by the gcc developers. > This problem should be addressed BEFORE python 2.5 is > released. The bug report is... > > [ 1545668 ] gcc trunk (4.2) exposes a signed integer overflows > > in the python sourceforge bug tracker. Thanks in advance > for attempting to fix this before Python 2.5 is released. I'm not sure I follow why this isn't considered a regression in GCC. Clearly, on all current hardware, x == -x is also true for the most negative int (0x80000000 on a 32-bit box). Why is GCC attempting to break our code (and then blaming us for it!) by using the C standard's weaselwords that signed integer overflow is undefined, despite that it has had a traditional meaning on 2's complement hardware for many decades? If GCC starts to enforce everything that the C standard says is undefined then very few programs will still work... -- --Guido van Rossum (home page: http://www.python.org/~guido/) From howarth at bromo.msbb.uc.edu Sat Aug 26 23:14:44 2006 From: howarth at bromo.msbb.uc.edu (Jack Howarth) Date: Sat, 26 Aug 2006 17:14:44 -0400 (EDT) Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows Message-ID: <20060826211444.68419110019@bromo.msbb.uc.edu> Guido, You'll never win that argument with the gcc developers. If you rely on undefined behavior in the c language standard, they have in the past, and will continue to, feel free to ignore those cases. If you plan on ignoring this issue, just be prepared to see a testcase failure in the python testsuite when python is built with gcc 4.2. Jack From david.nospam.hopwood at blueyonder.co.uk Sun Aug 27 01:17:17 2006 From: david.nospam.hopwood at blueyonder.co.uk (David Hopwood) Date: Sun, 27 Aug 2006 00:17:17 +0100 Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows In-Reply-To: References: <20060826190600.0E75911002B@bromo.msbb.uc.edu> Message-ID: <44F0D67D.9000007@blueyonder.co.uk> Guido van Rossum wrote: > On 8/26/06, Jack Howarth wrote: > >> I discovered that gcc 4.2 exposes a flaw with >>signed integer overflows in python. This bug and the >>necessary fix has been discussed in detail on the gcc >>mailing list. I have filed a detailed bug report and >>the recommended patch proposed by the gcc developers. >>This problem should be addressed BEFORE python 2.5 is >>released. The bug report is... >> >>[ 1545668 ] gcc trunk (4.2) exposes a signed integer overflows >> >>in the python sourceforge bug tracker. Thanks in advance >>for attempting to fix this before Python 2.5 is released. > > I'm not sure I follow why this isn't considered a regression in GCC. > Clearly, on all current hardware, x == -x is also true for the most > negative int (0x80000000 on a 32-bit box). Why is GCC attempting to > break our code (and then blaming us for it!) by using the C standard's > weaselwords that signed integer overflow is undefined, despite that it > has had a traditional meaning on 2's complement hardware for many > decades? If GCC starts to enforce everything that the C standard says > is undefined then very few programs will still work... +1. It's clearly undefined behaviour, but that doesn't mean that it is a good idea for gcc to be making changes that could silently break a lot of code, in pursuit of a microoptimization that is unlikely to significantly help performance. CPython should be fixed anyway. The correct fix is "if (y == -1 && x < 0 && (unsigned long)x == -(unsigned long)x)". -- David Hopwood From thomas at python.org Sun Aug 27 01:54:08 2006 From: thomas at python.org (Thomas Wouters) Date: Sat, 26 Aug 2006 19:54:08 -0400 Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows In-Reply-To: <44F0D67D.9000007@blueyonder.co.uk> References: <20060826190600.0E75911002B@bromo.msbb.uc.edu> <44F0D67D.9000007@blueyonder.co.uk> Message-ID: <9e804ac0608261654y39cd2613i57b65a021c375263@mail.gmail.com> On 8/26/06, David Hopwood wrote: > CPython should be fixed anyway. The correct fix is > "if (y == -1 && x < 0 && (unsigned long)x == -(unsigned long)x)". Why not just "... && x == LONG_MIN"? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060826/11fa9509/attachment.html From dberlin at dberlin.org Sun Aug 27 01:36:34 2006 From: dberlin at dberlin.org (Daniel Berlin) Date: Sat, 26 Aug 2006 19:36:34 -0400 Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows In-Reply-To: <20060826211444.68419110019@bromo.msbb.uc.edu> References: <20060826211444.68419110019@bromo.msbb.uc.edu> Message-ID: <44F0DB02.6030804@dberlin.org> Jack Howarth wrote: > Guido, > You'll never win that argument with the gcc developers. If you > rely on undefined behavior in the c language standard, they have > in the past, and will continue to, feel free to ignore those cases. > If you plan on ignoring this issue, just be prepared to see a > testcase failure in the python testsuite when python is built with > gcc 4.2. In addition, i'm surprised that you don't see these failures on other compilers. I know XLC, for example, would treat signed integer overflow as undefined at all opt levels as well. I would not be surprised to find Intel's compiler doing the same thing. Speaking as a gcc optimization person, if we were to treat signed integer overflow as wrapping, you would inhibit a very large number of loop optimizations on a very large class of loops (due to a number of reasons, from no longer being able to prove termination of loops or estimation of number of iterations, to other things). Heck, XLC at -O3+ will ignore *unsigned* integer wraparound of loop indices unless you use -qstrict_induction. As the docs for this option say, using that option can cause severe performance degradation (I happen to know the numbers from when i worked at IBM, and they are surprisingly high). GCC is willing to take the performance hit relative to other compilers on things like that to be a standard conforming compiler, just like we are willing to optimize code based on the assumption that you do not invoke undefined behavior. --Dan From tim.peters at gmail.com Sun Aug 27 02:37:46 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sat, 26 Aug 2006 20:37:46 -0400 Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows In-Reply-To: <9e804ac0608261654y39cd2613i57b65a021c375263@mail.gmail.com> References: <20060826190600.0E75911002B@bromo.msbb.uc.edu> <44F0D67D.9000007@blueyonder.co.uk> <9e804ac0608261654y39cd2613i57b65a021c375263@mail.gmail.com> Message-ID: <1f7befae0608261737p595b805fnee9d65762fe1af19@mail.gmail.com> [David Hopwood] >> CPython should be fixed anyway. The correct fix is >> "if (y == -1 && x < 0 && (unsigned long)x == -(unsigned long)x)". Note that this was already suggested in the bug report. [Thomas Wouters] > Why not just "... && x == LONG_MIN"? In full, if (y == -1 && x == LONG_MIN) "should work" too. In practice we try to avoid numeric symbols from platform header files because so many platforms have screwed these up over the centuries (search for LONG_BIT or HUGE_VAL ;-)), and because it's better (when possible) not to tie the code to that `x` was specifically declared as type "long" (e.g., just more stuff that will break if Python decides to make its short int of type PY_LONG_LONG instead). In this specific case, there may also have been a desire to avoid generating a memory load for a fat constant. However, since this is integer division, in real life (outside the test suite) we'll never go beyond the "y == -1" test. From david.nospam.hopwood at blueyonder.co.uk Sun Aug 27 03:18:37 2006 From: david.nospam.hopwood at blueyonder.co.uk (David Hopwood) Date: Sun, 27 Aug 2006 02:18:37 +0100 Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows In-Reply-To: <9e804ac0608261654y39cd2613i57b65a021c375263@mail.gmail.com> References: <20060826190600.0E75911002B@bromo.msbb.uc.edu> <44F0D67D.9000007@blueyonder.co.uk> <9e804ac0608261654y39cd2613i57b65a021c375263@mail.gmail.com> Message-ID: <44F0F2ED.3020907@blueyonder.co.uk> Thomas Wouters wrote: > On 8/26/06, David Hopwood wrote: > >> CPython should be fixed anyway. The correct fix is >> "if (y == -1 && x < 0 && (unsigned long)x == -(unsigned long)x)". > > Why not just "... && x == LONG_MIN"? Because the intent is to check that x / y does not overflow a long, and x == LONG_MIN would not cause an overflow on 1's complement or sign-magnitude systems. (CPython has probably only been tested on 2's complement systems anyway, but if we're going to be pedantic about depending only on things in the C standard...) -- David Hopwood From tim.peters at gmail.com Sun Aug 27 05:24:51 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sat, 26 Aug 2006 23:24:51 -0400 Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows In-Reply-To: <44F0F2ED.3020907@blueyonder.co.uk> References: <20060826190600.0E75911002B@bromo.msbb.uc.edu> <44F0D67D.9000007@blueyonder.co.uk> <9e804ac0608261654y39cd2613i57b65a021c375263@mail.gmail.com> <44F0F2ED.3020907@blueyonder.co.uk> Message-ID: <1f7befae0608262024i4c712c3et12da59644607f526@mail.gmail.com> [David Hopwood] > (CPython has probably only been tested on 2's complement systems anyway, Definitely so. Are there any boxes using 1's-comp or sign-magnitude integers anymore? Python assumes 2's-comp in many places. > but if we're going to be pedantic about depending only on things in the > C standard...) No, in that respect we're driven by the silliest decisions made by C compiler writers ;-) From kbk at shore.net Sun Aug 27 05:27:23 2006 From: kbk at shore.net (Kurt B. Kaiser) Date: Sat, 26 Aug 2006 23:27:23 -0400 (EDT) Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200608270327.k7R3RNcT027925@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 407 open ( +3) / 3393 closed (+17) / 3800 total (+20) Bugs : 888 open (+28) / 6145 closed (+14) / 7033 total (+42) RFE : 232 open ( +3) / 236 closed ( +1) / 468 total ( +4) New / Reopened Patches ______________________ most missing from Windows distribution (2006-08-16) CLOSED http://python.org/sf/1541412 opened by Jim Jewett buffer overrun in repr() for unicode strings (2006-08-16) CLOSED http://python.org/sf/1541585 opened by Simon Law work around clocks with low resolution (uuid.py) (2006-08-17) http://python.org/sf/1541863 opened by Hirokazu Yamamoto patch in bug 1542016 (2006-08-17) CLOSED http://python.org/sf/1542019 opened by Santiago Gala fix crash with continue in nested try/finally (2006-08-18) http://python.org/sf/1542451 opened by Neal Norwitz Improve dynamic linking support on AIX (2006-08-18) http://python.org/sf/1542544 opened by G?ran Uddeborg Tweak pydoc to speak about with and CONTEXTMANAGERS (2006-08-18) http://python.org/sf/1542681 opened by Santiago Gala Fix the vc8 solution files (2006-08-18) http://python.org/sf/1542946 opened by baus urllib2 regression (2006-08-19) CLOSED http://python.org/sf/1542948 opened by John J Lee tarfile.py: fix for bug 1543303 (2006-08-21) CLOSED http://python.org/sf/1543897 opened by Lars Gust?bel Socket module is not thread-safe (2006-08-21) http://python.org/sf/1544279 opened by Maxim Sobolev Implementation of PEP 362 (2006-08-22) http://python.org/sf/1544909 opened by Brett Cannon Fixes SocketServer Bug 1531963 (2006-08-23) http://python.org/sf/1545011 opened by Damon Kohler new splicetee module (2006-08-23) http://python.org/sf/1545262 opened by Omar AitMous new directio module (2006-08-23) http://python.org/sf/1545275 opened by Omar AitMous xrange that supports longs, etc (2006-08-24) http://python.org/sf/1546078 opened by Neal Norwitz crash in dict_equal (2006-08-24) http://python.org/sf/1546288 opened by Guido van Rossum Create a real zip iterator object; not using itertools.izip (2006-08-24) CLOSED http://python.org/sf/1546297 opened by Brian Holmes broken error reporting when filename specified by -s missing (2006-08-25) http://python.org/sf/1546372 opened by lplatypus Patches Closed ______________ most missing from Windows distribution (2006-08-16) http://python.org/sf/1541412 closed by jimjjewett buffer overrun in repr() for unicode strings (2006-08-16) http://python.org/sf/1541585 closed by gbrandl broken shortcut keys (2006-08-15) http://python.org/sf/1540874 closed by kbk patch in bug 1542016 (2006-08-17) http://python.org/sf/1542019 closed by gbrandl urllib2 regression (2006-08-19) http://python.org/sf/1542948 closed by gbrandl Backports from trunk to release24-maint (2006-08-15) http://python.org/sf/1540329 closed by gbrandl tarfile.py: fix for bug 1543303 (2006-08-21) http://python.org/sf/1543897 closed by nnorwitz Add some explication to PEP 3100 (2006-07-13) http://python.org/sf/1522038 closed by bcannon Create a real zip iterator object; not using itertools.izip (2006-08-24) http://python.org/sf/1546297 closed by gvanrossum Add readinto method to StringIO and cStringIO (2006-08-12) http://python.org/sf/1539381 closed by bcannon Remove the repr()-backticks (2006-06-04) http://python.org/sf/1500623 closed by bcannon Move reduce() to functools (2006-06-28) http://python.org/sf/1513870 closed by gvanrossum New / Reopened Bugs ___________________ infinite __str__ recursion in thread causes seg fault (2003-07-31) http://python.org/sf/780714 reopened by gbrandl tools and demo missing from windows (2006-08-16) http://python.org/sf/1541420 opened by Jim Jewett bug in python 2.4.3 for windows? (2006-08-16) CLOSED http://python.org/sf/1541566 opened by tinkiwinky Compiler command percent-sign causes format string error (2006-08-16) CLOSED http://python.org/sf/1541642 opened by Matthew Cowles bsddb can't use unicode filenames (2006-08-17) http://python.org/sf/1541671 opened by Zooko O'Whielacronx Incorrect example calls to PyObject_SetItem (2006-08-17) CLOSED http://python.org/sf/1541682 opened by John Machin Recently introduced sgmllib regexp bug hangs Python (2006-08-17) http://python.org/sf/1541697 opened by John J Lee Exceptions don't call _PyObject_GC_UNTRACK(self) (2006-08-17) http://python.org/sf/1542051 opened by ?iga Seilnacht global variable: multiple id()-addresses (2006-08-17) CLOSED http://python.org/sf/1542166 opened by Frank R. Schaefer Nested finally in generators don't follow PEP 342 (2006-08-17) http://python.org/sf/1542308 opened by Bob Ippolito httplib reads one byte per system call (2006-08-18) http://python.org/sf/1542407 opened by Zoyd Wheeler python-2.5c1.msi contains ICE validation errors and warnings (2006-08-18) http://python.org/sf/1542432 opened by Bob Arnson Bug in definition of PyImport_ImportModuleEx macro (2006-08-18) CLOSED http://python.org/sf/1542693 opened by Chris Stawarz idle in python 2.5c1 freezes on macos 10.3.9 (2006-08-18) http://python.org/sf/1542949 opened by David Strozzi tarfile in mode w|gz adds padding that annoys gunzip (2006-08-19) CLOSED http://python.org/sf/1543303 opened by alexis "from __future__ import foobar;" causes wrong SyntaxError (2006-08-20) http://python.org/sf/1543306 opened by daniel hahler Operator precedence inconsistent for complex literal (2006-08-20) CLOSED http://python.org/sf/1543347 opened by [N/A] test_tempfile fails on cygwin (2006-08-20) http://python.org/sf/1543467 opened by Miki Tebeka test_subprocess fails on cygwin (2006-08-20) http://python.org/sf/1543469 opened by Miki Tebeka md5 sums are different between Solaris and Windows XP SP1 (2006-08-21) http://python.org/sf/1543801 reopened by sunmountain md5 sums are different between Solaris and Windows XP SP1 (2006-08-21) http://python.org/sf/1543801 opened by Stefan Sonnenberg ctypes unit test fails (test_macholib.py) under MacOS 10.4.7 (2006-08-21) http://python.org/sf/1544102 opened by M. J. Fromberger test_anydbm segmentation fault (2006-08-21) http://python.org/sf/1544106 opened by Clay Spence python compiler support for with stmt (2006-08-21) http://python.org/sf/1544277 opened by Neal Norwitz Fix Lib/test/test___all__.py (2006-08-21) http://python.org/sf/1544295 opened by Hasan Diwan checking size of int... configure: error: cannot compute siz (2006-08-21) http://python.org/sf/1544306 opened by arrecostao _ctypes fails to build on Solaris x86 32-bit (Sun compiler) (2006-08-21) http://python.org/sf/1544339 opened by Case Van Horsen Bad result of calculation (2006-08-22) CLOSED http://python.org/sf/1544381 opened by Jean-Christophe BERTOLINI x!=y and [x]=[y] (!) (2006-08-22) http://python.org/sf/1544762 opened by Alex Martelli setup() keyword have to be list (doesn't work with tuple) (2006-08-23) http://python.org/sf/1545341 opened by STINNER Victor New-style classes fail to cleanup attributes (2006-08-23) http://python.org/sf/1545463 opened by Alexander Belopolsky inconsistent treatment of NULs in int() (2006-08-23) http://python.org/sf/1545497 opened by Neal Norwitz distutils home scheme lacks python versioning (2006-08-24) http://python.org/sf/1545658 opened by John Levon distutils needs vendor-packages support (2006-08-24) http://python.org/sf/1545659 opened by John Levon gcc trunk (4.2) exposes a signed integer overflows (2006-08-24) http://python.org/sf/1545668 opened by Jack Howarth structmember T_LONG won't accept a python long (2006-08-24) http://python.org/sf/1545696 opened by Roger Upole Incomplete info in 7.18.1 ZipFile Objects (2006-08-24) http://python.org/sf/1545836 opened by Taco array.array borks on deepcopy (2006-08-24) http://python.org/sf/1545837 opened by V?clav Haisman PyString_FromString() clarification (2006-08-24) http://python.org/sf/1546052 opened by Christian Walther bcannon secure branch issues (2006-08-24) CLOSED http://python.org/sf/1546203 opened by Jim Jewett subprocess.Popen can't read file object as stdin after seek (2006-08-25) http://python.org/sf/1546442 opened by GaryD String methods don't support explicit None arguments (2006-08-25) http://python.org/sf/1546585 opened by Nick Coghlan urlparse.urljoin odd behaviour (2006-08-25) http://python.org/sf/1546628 opened by Andres Riancho Bugs Closed ___________ Python 2.5 svn crash in _elementtree.c (2006-08-04) http://python.org/sf/1534630 closed by effbot bug in python 2.4.3 for windows? (2006-08-16) http://python.org/sf/1541566 closed by gbrandl Compiler command percent-sign causes format string error (2006-08-16) http://python.org/sf/1541642 deleted by mdcowles Incorrect example calls to PyObject_SetItem (2006-08-17) http://python.org/sf/1541682 closed by gbrandl global variable: multiple id()-addresses (2006-08-17) http://python.org/sf/1542166 closed by gbrandl Bug in definition of PyImport_ImportModuleEx macro (2006-08-18) http://python.org/sf/1542693 closed by nnorwitz tarfile in mode w|gz adds padding that annoys gunzip (2006-08-19) http://python.org/sf/1543303 closed by nnorwitz Operator precedence inconsistent for complex literal (2006-08-20) http://python.org/sf/1543347 closed by gbrandl md5 sums are different between Solaris and Windows XP SP1 (2006-08-21) http://python.org/sf/1543801 closed by sunmountain Bad result of calculation (2006-08-22) http://python.org/sf/1544381 closed by gbrandl bcannon secure branch issues (2006-08-24) http://python.org/sf/1546203 closed by bcannon New / Reopened RFE __________________ Optimizations for cgi.FieldStorage methods (2006-08-16) http://python.org/sf/1541463 opened by Bob Kline wsgi.org link in wsgiref (2006-08-18) http://python.org/sf/1542920 opened by Ian Bicking RFE Closed __________ extend strptime to understand logging timestamps (2004-08-10) http://python.org/sf/1006786 closed by bcannon From howarth at bromo.msbb.uc.edu Sun Aug 27 16:36:04 2006 From: howarth at bromo.msbb.uc.edu (Jack Howarth) Date: Sun, 27 Aug 2006 10:36:04 -0400 (EDT) Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows Message-ID: <20060827143604.EB1A0110019@bromo.msbb.uc.edu> I believe some of the others here might be interested in some other postings on the gcc mailing list regarding this issue (which weren't cross-posted here)... http://gcc.gnu.org/ml/gcc/2006-08/msg00516.html http://gcc.gnu.org/ml/gcc/2006-08/msg00517.html It makes clear that the impact of this change in gcc was considered and the reasoning on their decision to follow the standard so closely. http://gcc.gnu.org/ml/gcc/2005-06/msg01238.html Just so other beside Guido see those. Jack From RHettinger at ewtllc.com Sun Aug 27 17:26:10 2006 From: RHettinger at ewtllc.com (Raymond Hettinger) Date: Sun, 27 Aug 2006 08:26:10 -0700 Subject: [Python-Dev] zip -> izip; is __length_hint__ required? References: Message-ID: Yes, please rip that out. The patch should be a direct copy of the code in the itertools module. The use of length-hint was deliberately left-out of izip(). Also, yes it would be fine to alter the code in abstract.c to LBYL instead of suppressing exceptions. Raymond ________________________________ From: gvanrossum at gmail.com on behalf of Guido van Rossum Sent: Thu 8/24/2006 4:08 PM To: Raymond Hettinger; python-dev at python.org; Brian Holmes Subject: zip -> izip; is __length_hint__ required? At today's sprint, Brian Holmes contributed a patch that implements zip as an interator, a la izip. When reviewing Brian's code, I noticed that he added an implementation of __length_hint__. My gut feeling is that this isn't particularly useful given that zip() is almost exclusively used iteratively, and rarely if ever converted to a list or a tuple. (The one common exception is in the test suite, but there it's almost always a short list, and 3 out of 5 were actually tests for zip or izip.) Should we rip it out or keep it? Also, the existing convention for calling __length_hint__ (e.g. in _PyObject_LengthHint() in abstract.c) seems to be to use PyObject_CallMethod() and suppress TypeError and AttributeError coming out of the call. It would seem to make much more sense to check whether the attribute exists without calling it, and once it exists, just call it and not suppress any exceptions that come out of it. Is there any reason why this shouldn't work? -- --Guido van Rossum (home page: http://www.python.org/~guido/) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060827/1360bfc8/attachment.htm From chad at zetaweb.com Sun Aug 27 20:12:28 2006 From: chad at zetaweb.com (Chad Whitacre) Date: Sun, 27 Aug 2006 14:12:28 -0400 Subject: [Python-Dev] draft of patch guidelines In-Reply-To: References: <324634B71B159D469BCEB616678A6B94F94BA7@ingdexs1.ingdirect.com> Message-ID: <44F1E08C.3080805@zetaweb.com> Brett, >> I liked the "patch angel" term in Chad's version. Stating "a Python >> developer will take a look at your patch" smacks of a guarantee, >> while Chad's use of "patch angel" and "get the ball rolling" better >> conveyed the fact that this 5-for-1 rule is simply a practice of some >> of the volunteers. > > Eh, I still don't like the term. Too hokey. But I did toss in a > "try" to not make it sound like a guarantee. Yeah, kind of hokey. I included it because it's succinct and accurate, and because I assumed there's history to the term. Putting it in quotes helped w/ the goofiness, I thought. I believe Skip used the term: any comment, Skip? chad From chad at zetaweb.com Sun Aug 27 20:41:21 2006 From: chad at zetaweb.com (Chad Whitacre) Date: Sun, 27 Aug 2006 14:41:21 -0400 Subject: [Python-Dev] draft of patch guidelines In-Reply-To: References: <44EE681B.7010601@zetaweb.com> Message-ID: <44F1E751.7080002@zetaweb.com> Brett, > When you submit your patch, the tracker notifies a mailing list that > most core Python developers subscribe to of the creation of your new > patch. Isn't "of the creation of your new patch" redundant? What else would it be notifying the list of? > Your patch may languish for weeks or more before anyone volunteers a > response because of limited time on the part of Python developers. I think this is less accurate. Patches languish because of limited time *and* because newbies don't have any social capital w/in the Python community. New patch contributors are volunteers too, so they understand that constraint. Their big problem is their outsider status, to which the patch angels/5-for-1 system is an elegant solution. And that's what should be emphasized in this FAQ. In other words, I still prefer my wording. But since I'm a newbie with little merit, I'll be content with whatever you decide. :) chad From brett at python.org Mon Aug 28 02:17:32 2006 From: brett at python.org (Brett Cannon) Date: Sun, 27 Aug 2006 17:17:32 -0700 Subject: [Python-Dev] draft of patch guidelines In-Reply-To: <44F1E751.7080002@zetaweb.com> References: <44EE681B.7010601@zetaweb.com> <44F1E751.7080002@zetaweb.com> Message-ID: On 8/27/06, Chad Whitacre wrote: > > Brett, > > > When you submit your patch, the tracker notifies a mailing list that > > most core Python developers subscribe to of the creation of your new > > patch. > > Isn't "of the creation of your new patch" redundant? What else would it > be notifying the list of? Yep, slightly. > Your patch may languish for weeks or more before anyone volunteers a > > response because of limited time on the part of Python developers. > > I think this is less accurate. Patches languish because of limited time > *and* because newbies don't have any social capital w/in the Python > community. New patch contributors are volunteers too, so they understand > that constraint. Their big problem is their outsider status, to which > the patch angels/5-for-1 system is an elegant solution. And that's what > should be emphasized in this FAQ. But that makes us sound elitist. In other words, I still prefer my wording. But since I'm a newbie with > little merit, I'll be content with whatever you decide. :) OK. =) -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060827/4ed0dd87/attachment.htm From david.nospam.hopwood at blueyonder.co.uk Mon Aug 28 02:46:10 2006 From: david.nospam.hopwood at blueyonder.co.uk (David Hopwood) Date: Mon, 28 Aug 2006 01:46:10 +0100 Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows In-Reply-To: <20060827143604.EB1A0110019@bromo.msbb.uc.edu> References: <20060827143604.EB1A0110019@bromo.msbb.uc.edu> Message-ID: <44F23CD2.5090609@blueyonder.co.uk> Jack Howarth wrote: > I believe some of the others here might be interested in > some other postings on the gcc mailing list regarding this issue > (which weren't cross-posted here)... > > http://gcc.gnu.org/ml/gcc/2006-08/msg00516.html > http://gcc.gnu.org/ml/gcc/2006-08/msg00517.html > > It makes clear that the impact of this change in gcc was considered > and the reasoning on their decision to follow the standard so closely. > > http://gcc.gnu.org/ml/gcc/2005-06/msg01238.html > > Just so other beside Guido see those. This discussion just highlights how different the perspective of much of the C community (especially its language committee and compiler writers) is on error handling and language safety, relative to the perspective of the community that uses Python and other memory-safe languages. The C perspective is that programs have to be correct, and if they're not correct, it basically doesn't matter what happens. The safe language perspective is that of course we try to write correct programs, but if a program is not correct, then it *really matters* what the error behaviour is. Ideally, it should be something that facilitates debugging and that doesn't silently propagate an incorrect result. Undefined behaviour is right out. For example, from : # The gcc developers always face two competing constituencies: do not # change the compiler behaviour so that existing code which relies on # undefined behaviour continues to work, and do change the compiler # behaviour so that new code which does not rely on undefined behaviour # runs faster. In general, being compiler developers, we come down on # the side of making code which does not rely on undefined behaviour run # faster. ... and rarely even consider whether the behaviour *should* have been undefined or not. (There are many people who use C but do not agree with that perspective on error handling -- I'm one of them. These people often devise elaborate error handling frameworks, and compile at low optimization levels when it doesn't hurt performance.) -- David Hopwood From igor.bukanov at gmail.com Wed Aug 23 16:06:01 2006 From: igor.bukanov at gmail.com (Igor Bukanov) Date: Wed, 23 Aug 2006 16:06:01 +0200 Subject: [Python-Dev] GeneratorExit is unintuitive and uneccessary In-Reply-To: <44EC3CF5.8030502@gmail.com> References: <44EC3CF5.8030502@gmail.com> Message-ID: On 8/23/06, Nick Coghlan wrote: > Simpler is in the eye of the beholder - the current close() merely uses > throw() to raise a particular kind of exception 'asynchronously' (which is > already a familiar concept due to KeyboardInterrupt). What you're suggesting > is a completely new flow control concept that doesn't exist anywhere else in > the language. Yes, this is a new feature, but IMO the extra complexity it brings to the implementation is worth simplifications for the user. > > I suggested during the PEP 352 discussions that GeneratorExit should inherit > directly from BaseException, so that the fix to your example would be to > replace the bare "except:" with "except Exception:" That is, the idea was to make writing code that catches GeneratorExit difficult, right? But then note that if the code never catches GeneratorExit, then throwing GeneratorExit after the yield is exactly equivalent to just returning after the yield. In either case only finally blocks would be executed and close then would ignore GeneratorExit or StopIteration exceptions. So this proposal is just an extension of the idea "it should be difficult to use catch to ignore GeneratorExit" to simpler "No catch can catch GeneratorExit". > (because it can swallow > KeyboardInterrupt the version you posted is buggy regardless of how g.close() > works). So there are 4 kinds of exceptions now in Python: 1. Normal exceptions that are OK to catch and ignore. 2. Interrupts like KeyboardInterrupt that can be caught and ignored, but this is not recommended and it is easy to avoid catching them accidentally. 3. Interrupts like GeneratorExit that can be caught and ignored, but this is not recommended and where one has to write a lot of code to avoid catching them in a generic catch block. 4. Control transfer "exceptions" like break and continue across finally blocks. With enough efforts one can catch and ignore them, but then one know what he is doing: for i in range(3): print "i:", i try: try: break finally: raise Exception except Exception: print "Ignoring break" print "Done" Ideally it would be nice to merge 2-3-4 into a single class of special exceptions where the normal catch statement never sees them and extra efforts are necessary to ignore them, but with this proposal at least the list would not contain the item 3. Please also note that these comments are born after playing with generators in JS 1.7, the next JavaScript that Firefox 2.0 would implement. The current JS implementation tries to follow PEP 352, but since the only catch available in JS1.7 is just catch-all as before, the problems with accidentally catching GeneratorExit are much more visible in JS [1]. Thus changing close in JS to just return after yield would address the issues nicely but perhaps this can help Python users as well. Regards, Igor [1] https://bugzilla.mozilla.org/show_bug.cgi?id=349331 From igor.bukanov at gmail.com Thu Aug 24 00:52:09 2006 From: igor.bukanov at gmail.com (Igor Bukanov) Date: Thu, 24 Aug 2006 00:52:09 +0200 Subject: [Python-Dev] GeneratorExit is unintuitive and uneccessary In-Reply-To: <5.1.1.6.0.20060823102150.03a90638@sparrow.telecommunity.com> References: <5.1.1.6.0.20060823102150.03a90638@sparrow.telecommunity.com> Message-ID: On 8/23/06, Phillip J. Eby wrote: > Our original > assumption was that if they could implement throw() then they could clearly > implement close(), since close() was defined in terms of throw(). An > asynchronous return might be another matter. Such asynchronous return can always be implemented via a special hidden exception that is invisible to catch statements. I.e an implementation of generators can still use ___GeneratorExit internally as long as it is not exposed to scripts. Regards, Igor From igor.bukanov at gmail.com Thu Aug 24 01:44:21 2006 From: igor.bukanov at gmail.com (Igor Bukanov) Date: Thu, 24 Aug 2006 01:44:21 +0200 Subject: [Python-Dev] GeneratorExit is unintuitive and uneccessary In-Reply-To: References: <5.1.1.6.0.20060823102150.03a90638@sparrow.telecommunity.com> Message-ID: Regarding yield in the finally problem: The current implementation throws RuntimeException after the yield transfer the control back to the close method. If that is changed to reporting a bad behavior and rethrowing GeneratorExit or its hidden analog as suggested at the point of yield inside the generator, that would nicely unwind possible outer finally blocks and close the generator in any case. With that close-as-return proposal it does not even require to change the interpreter as the close method can simply loop like in generator.triggerRetrunAfterTheYield while generatorYielding: reportBadYield generator.triggerRetrunAfterTheYield Of cause, a bad generator can still subvert even this in the same way as return can be canceled, but with such changes one has to put significant non-trivial efforts in writing such code. Regards, Igor On 8/24/06, Guido van Rossum wrote: > IIUC this is how return currently works -- in some sense it's an > exception, but it can't be caught, and it won't escape from the > current frame. Ditto for break/continue. > > The generator extensions are still very young, and I'm not against > changing them somewhat in 2.6 or py3k. Perhaps GeneratorExit can be > replaced by a treatment similar to the treatment of return. But I'm > not sure it helps with the original problem; you could still put a > yield in a finally clause that intercepts a return, just as today you > can write > > def f(): > for i in range(10): > print i > try: > return > finally: > break > print 999 > > where the finally clause nullifies the return. In a generator, a yield > in the finally clause still needs to be dealt with somehow. > > > On 8/23/06, Igor Bukanov wrote: > > On 8/23/06, Phillip J. Eby wrote: > > > Our original > > > assumption was that if they could implement throw() then they could clearly > > > implement close(), since close() was defined in terms of throw(). An > > > asynchronous return might be another matter. > > > > Such asynchronous return can always be implemented via a special > > hidden exception that is invisible to catch statements. I.e an > > implementation of generators can still use ___GeneratorExit > > internally as long as it is not exposed to scripts. > > > > Regards, Igor > > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > From anthony at interlink.com.au Mon Aug 28 03:54:43 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon, 28 Aug 2006 11:54:43 +1000 Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows In-Reply-To: <20060826190600.0E75911002B@bromo.msbb.uc.edu> References: <20060826190600.0E75911002B@bromo.msbb.uc.edu> Message-ID: <200608281154.45219.anthony@interlink.com.au> On Sunday 27 August 2006 05:06, Jack Howarth wrote: > I discovered that gcc 4.2 exposes a flaw with > signed integer overflows in python. This bug and the > necessary fix has been discussed in detail on the gcc > mailing list. I have filed a detailed bug report and > the recommended patch proposed by the gcc developers. > This problem should be addressed BEFORE python 2.5 is > released. The bug report is... > > [ 1545668 ] gcc trunk (4.2) exposes a signed integer overflows > > in the python sourceforge bug tracker. Thanks in advance > for attempting to fix this before Python 2.5 is released. > Jack Regardless of whether we consider gcc's behaviour to be correct or not, I do agree we need a fix for this in 2.5 final. That should also be backported to release24-maint for the 2.4.4 release, and maybe release23-maint, as Barry recently started talking about cutting a 2.3.6. Can I nominate Tim, with his terrifying knowledge of C compiler esoterica, as the person to pick the best fix? Thanks, Anthony From chad at zetaweb.com Mon Aug 28 04:10:34 2006 From: chad at zetaweb.com (Chad Whitacre) Date: Sun, 27 Aug 2006 22:10:34 -0400 Subject: [Python-Dev] draft of patch guidelines In-Reply-To: References: <44EE681B.7010601@zetaweb.com> <44F1E751.7080002@zetaweb.com> Message-ID: <44F2509A.1030506@zetaweb.com> Brett, >> I think this is less accurate. Patches languish because of limited time >> *and* because newbies don't have any social capital w/in the Python >> community. New patch contributors are volunteers too, so they understand >> that constraint. Their big problem is their outsider status, to which >> the patch angels/5-for-1 system is an elegant solution. And that's what >> should be emphasized in this FAQ. > > But that makes us sound elitist. To be honest I took away the opposite lesson. New patch contributors acutely feel their outsider status; this is to be expected in any social system. The fact that there is a clear, established path for newcomers to follow -- the 5-for-1 rule -- is precisely what makes the Python community non-elitist in this instance. Transparent documentation on this point would further erode any elitism. chad From tim.peters at gmail.com Mon Aug 28 04:32:52 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 27 Aug 2006 22:32:52 -0400 Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows In-Reply-To: <200608281154.45219.anthony@interlink.com.au> References: <20060826190600.0E75911002B@bromo.msbb.uc.edu> <200608281154.45219.anthony@interlink.com.au> Message-ID: <1f7befae0608271932r19358b5fsb36372f3cdb2097e@mail.gmail.com> [Anthony Baxter] > Regardless of whether we consider gcc's behaviour to be correct or not, It is correct, but more to the point it's, umm, /there/ ;-) > I do agree we need a fix for this in 2.5 final. That should also be backported to > release24-maint for the 2.4.4 release, and maybe release23-maint, as Barry > recently started talking about cutting a 2.3.6. > > Can I nominate Tim, with his terrifying knowledge of C compiler esoterica, as > the person to pick the best fix? It's a bitch. Changing to if (y == -1 && x < 0 && (unsigned long)x == -(unsigned long)x) is the obvious fix, but violates our "no warnings" policy: the MS compiler warns about applying unary minus to an unsigned operand -- it "looks insane" to /their/ compiler writers ;-). Elegant patch below -- LOL. Would be nice if someone verified it worked on a box where it matters. Would also be nice if people checked to see whether their compiler(s) warn about something else now. IIndex: Objects/intobject.c =================================================================== --- Objects/intobject.c (revision 51618) +++ Objects/intobject.c (working copy) @@ -564,8 +564,14 @@ "integer division or modulo by zero"); return DIVMOD_ERROR; } - /* (-sys.maxint-1)/-1 is the only overflow case. */ - if (y == -1 && x < 0 && x == -x) + /* (-sys.maxint-1)/-1 is the only overflow case. x is the most + * negative long iff x < 0 and, on a 2's-complement box, x == -x. + * However, -x is undefined (by C) if x /is/ the most negative long + * (it's a signed overflow case), and some compilers care. So we cast + * x to unsigned long first. However, then other compilers warn about + * applying unary minus to an unsigned operand. Hence the weird "0-". + */ + if (y == -1 && x < 0 && (unsigned long)x == 0-(unsigned long)x) return DIVMOD_OVERFLOW; xdivy = x / y; xmody = x - xdivy * y; From p.f.moore at gmail.com Mon Aug 28 12:18:38 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 28 Aug 2006 11:18:38 +0100 Subject: [Python-Dev] Doc suggestion for Elementtree (for 2.5? a bit late, I know...) Message-ID: <79990c6b0608280318s39c0ba41uda8377f3e8efc11@mail.gmail.com> One little addition to the elementtree docs. In the overview section, adding a paragraph explaining best practice for importing the module might be useful. Some suggested text, for the "overview" section: """ The ElementTree module comes in two forms - a pure-python version (xml.etree.ElementTree) and a C-coded implementation (xml.etree.cElementTree) which is faster. To import the faster code if possible, but fall back to the Python implementation, you can use try: from xml.etree import cElementTree as ET except ImportError: from xml.etree import ElementTree as ET ElementTree is also available as an external module for older Python versions. For portability to these versions, this pattern can be extended to try: from xml.etree import cElementTree as ET except ImportError: try: from xml.etree import ElementTree as ET except ImportError: try: import cElementTree as ET except ImportError: import ElementTree as ET """ I'd put a patch on SF, but guess what? It's down again :-( Paul. PS This actually begs the question - are there platforms where xml.etree.cElementTree is not available? If not, is there a need for both versions? If there are, the wording above should probably be modified to reflect this. From fredrik at pythonware.com Mon Aug 28 13:03:43 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Mon, 28 Aug 2006 13:03:43 +0200 Subject: [Python-Dev] Doc suggestion for Elementtree (for 2.5? a bit late, I know...) References: <79990c6b0608280318s39c0ba41uda8377f3e8efc11@mail.gmail.com> Message-ID: Paul Moore wrote: > One little addition to the elementtree docs. In the overview section, > adding a paragraph explaining best practice for importing the module > might be useful. good idea. > PS This actually begs the question - are there platforms where > xml.etree.cElementTree is not available? not really; I usually recommend checking for xml.etree.cElementTree cElementTree elementtree.ElementTree in that order, if you just want to get the "best" implementation. (there are some subtle differences between the C implementations and the Python implementations, so people who do non-standard stuff may want to use xml.etree.ElementTree, but I don't think that has to be mentioned in the overview) From rowen at cesmail.net Tue Aug 29 00:24:19 2006 From: rowen at cesmail.net (Russell E. Owen) Date: Mon, 28 Aug 2006 15:24:19 -0700 Subject: [Python-Dev] distutils patch Message-ID: A colleague stumbled across distutils bug/misfeature that he found had been reported with an associated patch: and I'm wondering if there's any plans to accept the patch or implement some other fix (such as raising an exception if one tries to use a blank package name). The current behavior is clearly unexpected and confusing. If there's something I can do to help get a fix of some kind into python 2.5, I'm happy to do it. Regards, -- Russell From martin at v.loewis.de Tue Aug 29 01:01:16 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 29 Aug 2006 01:01:16 +0200 Subject: [Python-Dev] distutils patch In-Reply-To: References: Message-ID: <44F375BC.20603@v.loewis.de> Russell E. Owen schrieb: > A colleague stumbled across distutils bug/misfeature that he found had > been reported with an associated patch: > d=5470&atid=305470> > > and I'm wondering if there's any plans to accept the patch or implement > some other fix (such as raising an exception if one tries to use a blank > package name). The current behavior is clearly unexpected and confusing. > If there's something I can do to help get a fix of some kind into python > 2.5, I'm happy to do it. As you can see from my comment, I think it really is a bug in the application. Without reviewing it again, I think disallowing empty package names might be the right thing; this can only be done in 2.6, though. Regards, Martin From greg.ewing at canterbury.ac.nz Tue Aug 29 04:05:22 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 29 Aug 2006 14:05:22 +1200 Subject: [Python-Dev] Doc suggestion for Elementtree (for 2.5? a bit late, I know...) In-Reply-To: <79990c6b0608280318s39c0ba41uda8377f3e8efc11@mail.gmail.com> References: <79990c6b0608280318s39c0ba41uda8377f3e8efc11@mail.gmail.com> Message-ID: <44F3A0E2.60006@canterbury.ac.nz> Paul Moore wrote: > The ElementTree module comes in two forms - a pure-python version > (xml.etree.ElementTree) and a C-coded implementation > (xml.etree.cElementTree) which is faster. If this is to be in the stdlib, is there any chance of tidying up the convoluted, uninituitive and non-pep8-compliant module naming structure? -- Greg From steve at holdenweb.com Tue Aug 29 06:36:05 2006 From: steve at holdenweb.com (Steve Holden) Date: Tue, 29 Aug 2006 05:36:05 +0100 Subject: [Python-Dev] Doc suggestion for Elementtree (for 2.5? a bit late, I know...) In-Reply-To: <44F3A0E2.60006@canterbury.ac.nz> References: <79990c6b0608280318s39c0ba41uda8377f3e8efc11@mail.gmail.com> <44F3A0E2.60006@canterbury.ac.nz> Message-ID: Greg Ewing wrote: > Paul Moore wrote: > > >>The ElementTree module comes in two forms - a pure-python version >>(xml.etree.ElementTree) and a C-coded implementation >>(xml.etree.cElementTree) which is faster. > > > If this is to be in the stdlib, is there any chance > of tidying up the convoluted, uninituitive and > non-pep8-compliant module naming structure? > I'm guessing the answer is "no" since the objection was only raised when the code had been incorporated into a release candidate. It was probably "no" from the beginning given that there was already substantial usage of the code before it was adopted for the stdlib. Standards, apparently, are for *other* people :-) It would be good if 3.0 was *much* more hard-nosed about naming conventions. How can we expect the community as a whole to take them seriously if we don't take them seriously ourselves? regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden From steve at holdenweb.com Tue Aug 29 06:38:36 2006 From: steve at holdenweb.com (Steve Holden) Date: Tue, 29 Aug 2006 05:38:36 +0100 Subject: [Python-Dev] draft for bug guidelines In-Reply-To: References: <20060824192945.GA32475@phd.pp.ru> Message-ID: Brett Cannon wrote: > Made it the first step. =) > > -Brett > > On 8/24/06, *Oleg Broytmann* > wrote: > > On Thu, Aug 24, 2006 at 12:21:06PM -0700, Brett Cannon wrote: > > Start a new bug > > "Before starting a new bug please try to search if the bug has > already > been reported. It it has - do not start a new report, add your > comments to > the existing report." > You probably want to include some advice about how. Isn't this rather an inappropriate time to be writing SF user notes, given that you are planning to provide us with an alternative to SF by the end of the year? The last thing we need is a "how to submit a bug" article that sends people to the wrong repository ... regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden From fredrik at pythonware.com Tue Aug 29 09:59:08 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue, 29 Aug 2006 09:59:08 +0200 Subject: [Python-Dev] Doc suggestion for Elementtree (for 2.5? a bitlate, I know...) References: <79990c6b0608280318s39c0ba41uda8377f3e8efc11@mail.gmail.com><44F3A0E2.60006@canterbury.ac.nz> Message-ID: Steve Holden wrote: > Standards, apparently, are for *other* people :-) ET was written years before the "certain modules should use camelcase" stuff was removed from PEP 8. as a refresher for those of you who have trouble remembering how things were back in early 2004, here's GvR's original style guide: http://www.python.org/doc/essays/styleguide/ Modules that export a single class (or a number of closely related classes, plus some additional support) are often named in MixedCase, with the module name being the same as the class name (e.g. the standard StringIO module). Modules that export a bunch of functions are usually named in all lowercase. /.../ There is an emerging convention that when an extension module written in C or C++ has an accompanying Python module that provides a higher level (e.g. more object oriented) interface, the Python module's name CapWords, while the C/C++ module is named in all lowercase and has a leading underscore (e.g. Tkinter/_tkinter). From greg.ewing at canterbury.ac.nz Tue Aug 29 11:19:10 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 29 Aug 2006 21:19:10 +1200 Subject: [Python-Dev] Doc suggestion for Elementtree (for 2.5? a bitlate, I know...) In-Reply-To: References: <79990c6b0608280318s39c0ba41uda8377f3e8efc11@mail.gmail.com> <44F3A0E2.60006@canterbury.ac.nz> Message-ID: <44F4068E.7030205@canterbury.ac.nz> Fredrik Lundh wrote: > ET was written years before the "certain modules should use camelcase" stuff > was removed from PEP 8. Maybe so, but I was hoping that additions to the stdlib in this day and age might be adapted to follow modern conventions instead of just being plonked in as-is. -- Greg From ncoghlan at iinet.net.au Tue Aug 29 11:29:28 2006 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Tue, 29 Aug 2006 19:29:28 +1000 Subject: [Python-Dev] Adding an rslice() builtin? Message-ID: <44F408F8.1070403@iinet.net.au> A discussion on the py3k list reminded me that translating a forward slice into a reversed slice is significantly less than obvious to many people. Not only do you have to negate the step value and swap the start and stop values, but you also need to subtract one from each of the step values, and ensure the new start value was actually in the original slice: reversed(seq[start:stop:step]) becomes seq[(stop-1)%abs(step):start-1:-step] An rslice builtin would make the latter version significantly easier to read: seq[rslice(start, stop, step)] Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From steve at holdenweb.com Tue Aug 29 11:37:54 2006 From: steve at holdenweb.com (Steve Holden) Date: Tue, 29 Aug 2006 10:37:54 +0100 Subject: [Python-Dev] Doc suggestion for Elementtree (for 2.5? a bitlate, I know...) In-Reply-To: <44F4068E.7030205@canterbury.ac.nz> References: <79990c6b0608280318s39c0ba41uda8377f3e8efc11@mail.gmail.com> <44F3A0E2.60006@canterbury.ac.nz> <44F4068E.7030205@canterbury.ac.nz> Message-ID: <44F40AF2.30504@holdenweb.com> Greg Ewing wrote: > Fredrik Lundh wrote: > > >>ET was written years before the "certain modules should use camelcase" stuff >>was removed from PEP 8. > > > Maybe so, but I was hoping that additions to the > stdlib in this day and age might be adapted to follow > modern conventions instead of just being plonked in > as-is. > Unfortunately this doesn't work if you want to encourage existing users to migrate to the built-in version. However, let's hope that Python 3000 will have enforcers patrolling the perimeter of the library with shotguns. It does make us look rather dumb when we set rules that our own code doesn't obey - or change the rules after we've already encouraged contributions under other terms. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden From greg.ewing at canterbury.ac.nz Tue Aug 29 11:37:47 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 29 Aug 2006 21:37:47 +1200 Subject: [Python-Dev] Adding an rslice() builtin? In-Reply-To: <44F408F8.1070403@iinet.net.au> References: <44F408F8.1070403@iinet.net.au> Message-ID: <44F40AEB.7010405@canterbury.ac.nz> Nick Coghlan wrote: > reversed(seq[start:stop:step]) becomes seq[(stop-1)%abs(step):start-1:-step] > > An rslice builtin would make the latter version significantly easier to read: > > seq[rslice(start, stop, step)] How would this deal with omitted start and/or stop values? -- Greg From steve at holdenweb.com Tue Aug 29 11:37:54 2006 From: steve at holdenweb.com (Steve Holden) Date: Tue, 29 Aug 2006 10:37:54 +0100 Subject: [Python-Dev] Doc suggestion for Elementtree (for 2.5? a bitlate, I know...) In-Reply-To: <44F4068E.7030205@canterbury.ac.nz> References: <79990c6b0608280318s39c0ba41uda8377f3e8efc11@mail.gmail.com> <44F3A0E2.60006@canterbury.ac.nz> <44F4068E.7030205@canterbury.ac.nz> Message-ID: <44F40AF2.30504@holdenweb.com> Greg Ewing wrote: > Fredrik Lundh wrote: > > >>ET was written years before the "certain modules should use camelcase" stuff >>was removed from PEP 8. > > > Maybe so, but I was hoping that additions to the > stdlib in this day and age might be adapted to follow > modern conventions instead of just being plonked in > as-is. > Unfortunately this doesn't work if you want to encourage existing users to migrate to the built-in version. However, let's hope that Python 3000 will have enforcers patrolling the perimeter of the library with shotguns. It does make us look rather dumb when we set rules that our own code doesn't obey - or change the rules after we've already encouraged contributions under other terms. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden From p.f.moore at gmail.com Tue Aug 29 14:46:33 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 29 Aug 2006 13:46:33 +0100 Subject: [Python-Dev] Adding an rslice() builtin? In-Reply-To: <44F40AEB.7010405@canterbury.ac.nz> References: <44F408F8.1070403@iinet.net.au> <44F40AEB.7010405@canterbury.ac.nz> Message-ID: <79990c6b0608290546h2542fd25raae256ebb3691b84@mail.gmail.com> On 8/29/06, Greg Ewing wrote: > Nick Coghlan wrote: > > > reversed(seq[start:stop:step]) becomes seq[(stop-1)%abs(step):start-1:-step] > > > > An rslice builtin would make the latter version significantly easier to read: > > > > seq[rslice(start, stop, step)] > > How would this deal with omitted start and/or stop values? More generally, given start/stop/step don't have to be numbers, how can this work in general? I don't actually see a need for this, given that reversed(seq[start:stop:step]) covers all of the real use cases I can think of... Paul. From guido at python.org Tue Aug 29 15:45:05 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 29 Aug 2006 06:45:05 -0700 Subject: [Python-Dev] Adding an rslice() builtin? In-Reply-To: <44F408F8.1070403@iinet.net.au> References: <44F408F8.1070403@iinet.net.au> Message-ID: That discussion on py3k is far from finished. We've also had a similar discussion in the past and ended up with reversed(), which solves the problem on the other end (using a forward slice but iterating backwards). Also, when I saw your subject I thought rslice() was related to rfind(), so the name is at best questuinable. Let's drop this until we've got clarity on what Py3k actually will do. --Guido On 8/29/06, Nick Coghlan wrote: > A discussion on the py3k list reminded me that translating a forward slice > into a reversed slice is significantly less than obvious to many people. Not > only do you have to negate the step value and swap the start and stop values, > but you also need to subtract one from each of the step values, and ensure the > new start value was actually in the original slice: > > reversed(seq[start:stop:step]) becomes seq[(stop-1)%abs(step):start-1:-step] > > An rslice builtin would make the latter version significantly easier to read: > > seq[rslice(start, stop, step)] > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > --------------------------------------------------------------- > http://www.boredomandlaziness.org > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz at pythoncraft.com Tue Aug 29 16:45:03 2006 From: aahz at pythoncraft.com (Aahz) Date: Tue, 29 Aug 2006 07:45:03 -0700 Subject: [Python-Dev] Doc suggestion for Elementtree (for 2.5? a bitlate, I know...) In-Reply-To: <44F4068E.7030205@canterbury.ac.nz> References: <79990c6b0608280318s39c0ba41uda8377f3e8efc11@mail.gmail.com> <44F3A0E2.60006@canterbury.ac.nz> <44F4068E.7030205@canterbury.ac.nz> Message-ID: <20060829144503.GA26161@panix.com> On Tue, Aug 29, 2006, Greg Ewing wrote: > Fredrik Lundh wrote: >> >> ET was written years before the "certain modules should use camelcase" stuff >> was removed from PEP 8. > > Maybe so, but I was hoping that additions to the stdlib in this day > and age might be adapted to follow modern conventions instead of just > being plonked in as-is. You have a point, but I think that for external libraries with a large following the best we can do is set things up so that it's both PEP8 compliant *and* has aliases to the existing setup. From my POV, it's critical to encourage people to switch to the stdlib version if possible (but often writing code that works with the external library is the only way to support multiple Python versions). That parenthetical bit is the real killer, and I don't think even Py3K can completely overcome it if PEP8 continues to evolve. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." --Brian W. Kernighan From ncoghlan at gmail.com Tue Aug 29 16:54:40 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 30 Aug 2006 00:54:40 +1000 Subject: [Python-Dev] Adding an rslice() builtin? In-Reply-To: <44F40AEB.7010405@canterbury.ac.nz> References: <44F408F8.1070403@iinet.net.au> <44F40AEB.7010405@canterbury.ac.nz> Message-ID: <44F45530.80108@gmail.com> Greg Ewing wrote: > Nick Coghlan wrote: > >> reversed(seq[start:stop:step]) becomes >> seq[(stop-1)%abs(step):start-1:-step] >> >> An rslice builtin would make the latter version significantly easier >> to read: >> >> seq[rslice(start, stop, step)] > > How would this deal with omitted start and/or stop values? Badly! (negative indices are screwed, too) This would be an awful lot easier if we could just subclass slice, and do the calculation in the indices() method where we have access to len(seq) in order to deal with an omitted stop value and negative indices. Ah well, never mind. I'll take it back to the Py3k list, and see if I can find some small changes we can make so that slice() doesn't feel like a bolted-on hack that exists solely to avoid the need for a new special method :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From barry at python.org Tue Aug 29 17:22:58 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 29 Aug 2006 11:22:58 -0400 Subject: [Python-Dev] Interest in a Python 2.3.6? Message-ID: <05D21426-7FF5-4AEF-B757-DF0BEC5D0D74@python.org> I am considering producing a Python 2.3.6 release, which would of course only be a bug fix maintenance release. The primary reason is that not all OS distributions have upgraded to Python 2.4 and I think it's worthwhile for us to bless a release that fixes known critical bugs. I'm willing to be the release manager for 2.3.6, but I'm hoping you all will be able to help identify the most important bugs that need fixing. First, is there interest in getting a 2.3.6 release out? I'd propose keeping things simple, by picking a date and releasing what we've got at that date (assuming of course all the unit tests pass). It's probably a good idea to add a wiki page tracking the fixes we want to get in there. 2.5 final is slated for 12-Sep-2006 and I know that Anthony is planning for a 2.4.4 soon after that. I'm thinking that we'd try to do a 2.3.6 a couple of weeks after 2.4.4 so that people who care about it aren't stacked up with fixing too many branches at once. My first thought was to shoot for Monday October 9th. What are the potential 2.3.6 fixes? Nothing on this page: http://www.python.org/download/releases/2.3.5/bugs/ seems critical to me. I know that I've added some important email package fixes that are already in Subversion. There are tons of bugs and patches reported against 2.3 in SF (lament: why won't SF just / tell/ me how many there are?), but I have no idea atm which, if any, are important enough to go into a 2.3.6. I haven't yet done an svn diff to see what changes are already in there. I don't have the cycles to fix things myself, so it would be up to everyone to help commit fixes. I'll ride herd a bit if necessary. Thoughts? I don't want to waste my time if nobody thinks a 2.3.6 would be useful, but I'm happy to do it if there's community support. I'll also need the usual help with Windows installers and documentation updates. Cheers, -Barry -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060829/d2b3650e/attachment.htm -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 304 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060829/d2b3650e/attachment.pgp From g.brandl at gmx.net Tue Aug 29 17:29:08 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 29 Aug 2006 17:29:08 +0200 Subject: [Python-Dev] Interest in a Python 2.3.6? In-Reply-To: <05D21426-7FF5-4AEF-B757-DF0BEC5D0D74@python.org> References: <05D21426-7FF5-4AEF-B757-DF0BEC5D0D74@python.org> Message-ID: Barry Warsaw wrote: > I am considering producing a Python 2.3.6 release, which would of > course only be a bug fix maintenance release. The primary reason is > that not all OS distributions have upgraded to Python 2.4 and I think > it's worthwhile for us to bless a release that fixes known critical > bugs. I'm willing to be the release manager for 2.3.6, but I'm hoping > you all will be able to help identify the most important bugs that need > fixing. > > First, is there interest in getting a 2.3.6 release out? I'd propose > keeping things simple, by picking a date and releasing what we've got at > that date (assuming of course all the unit tests pass). It's probably a > good idea to add a wiki page tracking the fixes we want to get in there. > > 2.5 final is slated for 12-Sep-2006 and I know that Anthony is planning > for a 2.4.4 soon after that. I'm thinking that we'd try to do a 2.3.6 a > couple of weeks after 2.4.4 so that people who care about it aren't > stacked up with fixing too many branches at once. My first thought was > to shoot for Monday October 9th. > > What are the potential 2.3.6 fixes? Nothing on this page: > > http://www.python.org/download/releases/2.3.5/bugs/ > > seems critical to me. I know that I've added some important email > package fixes that are already in Subversion. There are tons of bugs > and patches reported against 2.3 in SF (lament: why won't SF just /tell/ > me how many there are?), but I have no idea atm which, if any, are > important enough to go into a 2.3.6. Most of them are also in the 2.4 releases... > I haven't yet done an svn diff to > see what changes are already in there. There's one problem: it was thought for a long time that there wouldn't be any more 2.3 releases, so bug fixes were applied only in the head and 2.4 branch. If there will be a 2.3.6, it might be necessary to look at all 2.4 branch fixes and decide which of them are worth backporting. Georg From jcarlson at uci.edu Tue Aug 29 18:13:07 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 29 Aug 2006 09:13:07 -0700 Subject: [Python-Dev] Interest in a Python 2.3.6? In-Reply-To: <05D21426-7FF5-4AEF-B757-DF0BEC5D0D74@python.org> References: <05D21426-7FF5-4AEF-B757-DF0BEC5D0D74@python.org> Message-ID: <20060829084804.1B06.JCARLSON@uci.edu> Barry Warsaw wrote: > I am considering producing a Python 2.3.6 release, which would of > course only be a bug fix maintenance release. The primary reason is > that not all OS distributions have upgraded to Python 2.4 and I think > it's worthwhile for us to bless a release that fixes known critical > bugs. I'm willing to be the release manager for 2.3.6, but I'm > hoping you all will be able to help identify the most important bugs > that need fixing. > > What are the potential 2.3.6 fixes? Nothing on this page: These are both issues in 2.3 and 2.4: http://www.python.org/sf/780714 http://www.python.org/sf/1548687 Those are all I have off the top of my head. - Josiah From david.nospam.hopwood at blueyonder.co.uk Tue Aug 29 18:44:40 2006 From: david.nospam.hopwood at blueyonder.co.uk (David Hopwood) Date: Tue, 29 Aug 2006 17:44:40 +0100 Subject: [Python-Dev] Adding an rslice() builtin? In-Reply-To: <44F408F8.1070403@iinet.net.au> References: <44F408F8.1070403@iinet.net.au> Message-ID: <44F46EF8.9000909@blueyonder.co.uk> Nick Coghlan wrote: > A discussion on the py3k list reminded me that translating a forward slice > into a reversed slice is significantly less than obvious to many people. Not > only do you have to negate the step value and swap the start and stop values, > but you also need to subtract one from each of the step values, and ensure the > new start value was actually in the original slice: > > reversed(seq[start:stop:step]) becomes seq[(stop-1)%abs(step):start-1:-step] > > An rslice builtin would make the latter version significantly easier to read: > > seq[rslice(start, stop, step)] Or slice.reversed(). -- David Hopwood From exarkun at divmod.com Tue Aug 29 19:09:57 2006 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Tue, 29 Aug 2006 13:09:57 -0400 Subject: [Python-Dev] Adding an rslice() builtin? In-Reply-To: <44F46EF8.9000909@blueyonder.co.uk> Message-ID: <20060829170957.1717.385586324.divmod.quotient.33876@ohm> On Tue, 29 Aug 2006 17:44:40 +0100, David Hopwood wrote: >Nick Coghlan wrote: >> A discussion on the py3k list reminded me that translating a forward slice >> into a reversed slice is significantly less than obvious to many people. Not >> only do you have to negate the step value and swap the start and stop values, >> but you also need to subtract one from each of the step values, and ensure the >> new start value was actually in the original slice: >> >> reversed(seq[start:stop:step]) becomes seq[(stop-1)%abs(step):start-1:-step] >> >> An rslice builtin would make the latter version significantly easier to read: >> >> seq[rslice(start, stop, step)] > >Or slice.reversed(). > Better, slice.reversed(length). Jean-Paul From brett at python.org Tue Aug 29 19:15:51 2006 From: brett at python.org (Brett Cannon) Date: Tue, 29 Aug 2006 10:15:51 -0700 Subject: [Python-Dev] draft for bug guidelines In-Reply-To: References: <20060824192945.GA32475@phd.pp.ru> Message-ID: On 8/28/06, Steve Holden wrote: > > Brett Cannon wrote: > > Made it the first step. =) > > > > -Brett > > > > On 8/24/06, *Oleg Broytmann* > > wrote: > > > > On Thu, Aug 24, 2006 at 12:21:06PM -0700, Brett Cannon wrote: > > > Start a new bug > > > > "Before starting a new bug please try to search if the bug has > > already > > been reported. It it has - do not start a new report, add your > > comments to > > the existing report." > > > You probably want to include some advice about how. Isn't this rather an > inappropriate time to be writing SF user notes, given that you are > planning to provide us with an alternative to SF by the end of the year? No because we have people reporting bugs between now and the end of the year which is four months away. The last thing we need is a "how to submit a bug" article that sends > people to the wrong repository ... But the whole point is for these things to end up on www.python.org/dev/where I will update them as soon as we have a new tracker selected and before the new tracker is even used. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060829/5cc582fa/attachment.htm From arigo at tunes.org Tue Aug 29 22:10:22 2006 From: arigo at tunes.org (Armin Rigo) Date: Tue, 29 Aug 2006 22:10:22 +0200 Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows In-Reply-To: <1f7befae0608261737p595b805fnee9d65762fe1af19@mail.gmail.com> References: <20060826190600.0E75911002B@bromo.msbb.uc.edu> <44F0D67D.9000007@blueyonder.co.uk> <9e804ac0608261654y39cd2613i57b65a021c375263@mail.gmail.com> <1f7befae0608261737p595b805fnee9d65762fe1af19@mail.gmail.com> Message-ID: <20060829201022.GA22579@code0.codespeak.net> Hi Tim, On Sat, Aug 26, 2006 at 08:37:46PM -0400, Tim Peters wrote: > [Thomas Wouters] > > Why not just "... && x == LONG_MIN"? > it's better (when possible) not to tie the code to that `x` was > specifically declared as type "long" (e.g., just more stuff that will > break if Python decides to make its short int of type PY_LONG_LONG > instead). The proposed "correct fix" breaks this goal too: > >> "if (y == -1 && x < 0 && (unsigned long)x == -(unsigned long)x)". ^^^^^^^^^^^^^ ^^^^^^^^^^^^^ A bientot, Armin From barry at python.org Tue Aug 29 22:42:27 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 29 Aug 2006 16:42:27 -0400 Subject: [Python-Dev] Interest in a Python 2.3.6? In-Reply-To: References: <05D21426-7FF5-4AEF-B757-DF0BEC5D0D74@python.org> Message-ID: <4212B8D1-3152-4245-94C5-852EBFB820B3@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 29, 2006, at 11:29 AM, Georg Brandl wrote: > There's one problem: it was thought for a long time that there > wouldn't be > any more 2.3 releases, so bug fixes were applied only in the head > and 2.4 > branch. > > If there will be a 2.3.6, it might be necessary to look at all 2.4 > branch > fixes and decide which of them are worth backporting. Right, we'll definitely have to do that, and maybe not much more. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRPSmuXEjvBPtnXfVAQKyegQAgpzLpPddKE03wg3orKvlHPJXFKpbaLvd 4gIRIgvqwGjKwjxY4s8Gn55axHJM8YBwJ+IbCzYg3nmBB3JvljuKNxTZtz4w781Y BS/apc8Ng+2wv/W8Eo2zVbbRformZa24H/d8Gq5Mdst1yTrcAwwcO7k1hHjAl9w8 HFIWjechAVg= =4wXb -----END PGP SIGNATURE----- From laca at sun.com Tue Aug 29 23:04:25 2006 From: laca at sun.com (Laszlo (Laca) Peter) Date: Tue, 29 Aug 2006 17:04:25 -0400 Subject: [Python-Dev] 32-bit and 64-bit python on Solaris Message-ID: <1156885465.1622.42.camel@ferrari.sfbay.sun.com> Hi, I work in the team that delivers python on Solaris. Recently we've been getting requests for delivering python in 64-bit as well as in 32-bit. As you probably know, Solaris can run 64-bit and 32-bit binaries on the same system, but of course you can't mix and match shared objects with different ISAs. This seems to apply to python bytecode as well: the pyc files generated by a 64-bit build of python are incompatible with those generated by the 32-bit python. Note the caveat at http://aspn.activestate.com/ASPN/docs/ActivePython/2.3/python/lib/module-marshal.html I guess my first question is if there are any plans to make the bytecodes for different ISAs compatible. That would make most of our problems magically go away (; If not, then the issue we have is how to separate the 32-bit and 64-bit pyc's on the same system so that: - loading the right pyc is transparent to the user - you can't accidentally load a pyc with the wrong ISA (by setting PYTHONPATH, for example) So duplicating the libdir is not an option. The usual technique used for Solaris shared objects is placing the 64-bit objects in a 64/ subdirectory, while the 32-bit objects stay where they are. We considered various solutions. Our current proposal is this: .../some/dir/foo.py <- python source .../some/dir/foo.pyc <- 32-bit bytecode .../some/dir/foo.pyc64 <- 64-bit bytecode .../some/dir/foo.pyo <- optimised 32-bit bytecode .../some/dir/foo.pyo64 <- optimised 64-bit bytecode The 64-bit python would always look for pyc64/pyo64, so this would take care of accidentally loading the wrong ISA. PYTHONPATH could be the same for the 32-bit and the 64-bit python. Shared objects: .../some/dir/foo.so <- 32-bit shared object .../some/dir/64/foo.so <- 64-bit shared object This is kinda inconsistent, but simple enough. The 64-bit python would automatically look for shared objects in the 64/ subdir. ld.so.1 makes sure that the wrong share object is never loaded. So I'm looking for advice and wondering if it would be possible to support multiple ISAs out of the box. Would you be willing to accept patches that implement the above (or whatever solution we agree to) on Solaris? Thanks, Laca From guido at python.org Tue Aug 29 23:42:43 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 29 Aug 2006 14:42:43 -0700 Subject: [Python-Dev] Small Py3k task: fix modulefinder.py Message-ID: Is anyone familiar enough with modulefinder.py to fix its breakage in Py3k? It chokes in a nasty way (exceeding the recursion limit) on the relative import syntax. I suspect this is also a problem for 2.5, when people use that syntax; hence the cross-post. There's no unittest for modulefinder.py, but I believe py2exe depends on it (and of course freeze.py, but who uses that still?) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.peters at gmail.com Wed Aug 30 00:57:12 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 29 Aug 2006 18:57:12 -0400 Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows In-Reply-To: <20060829201022.GA22579@code0.codespeak.net> References: <20060826190600.0E75911002B@bromo.msbb.uc.edu> <44F0D67D.9000007@blueyonder.co.uk> <9e804ac0608261654y39cd2613i57b65a021c375263@mail.gmail.com> <1f7befae0608261737p595b805fnee9d65762fe1af19@mail.gmail.com> <20060829201022.GA22579@code0.codespeak.net> Message-ID: <1f7befae0608291557l5b04a8f6wd1371e62a5c9c69c@mail.gmail.com> [Thomas Wouters] >>> Why not just "... && x == LONG_MIN"? [Tim Peters] >> it's better (when possible) not to tie the code to that `x` was >> specifically declared as type "long" (e.g., just more stuff that will >> break if Python decides to make its short int of type PY_LONG_LONG >> instead). [Armin Rigo] > The proposed "correct fix" breaks this goal too: > > > >> "if (y == -1 && x < 0 && (unsigned long)x == -(unsigned long)x)". > > ^^^^^^^^^^^^^ ^^^^^^^^^^^^^ Yup, although as noted before the proposed fix actually writes == 0-(unsigned long)x at the tail end instead (to avoid compiler warnings at least under MS C). It doesn't run afoul of the other criterion you snipped from the start of the quoted paragraph: In practice we try to avoid numeric symbols from platform header files because so many platforms have screwed these up over the centuries (search for LONG_BIT or HUGE_VAL ;-)), .... This is a wrong time in the release process to take on chance on discovering a flaky LONG_MIN on some box, so I want to keep the code as much as possible like what's already there (which worked fine for > 10 years on all known boxes) for now. Speaking of which, I saw no feedback on the proposed patch in http://mail.python.org/pipermail/python-dev/2006-August/068502.html so I'll just check that in tomorrow. From tim.peters at gmail.com Wed Aug 30 01:07:33 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 29 Aug 2006 19:07:33 -0400 Subject: [Python-Dev] 32-bit and 64-bit python on Solaris In-Reply-To: <1156885465.1622.42.camel@ferrari.sfbay.sun.com> References: <1156885465.1622.42.camel@ferrari.sfbay.sun.com> Message-ID: <1f7befae0608291607p513f0201wda2465a5f4a73310@mail.gmail.com> [Laszlo (Laca) Peter] > I work in the team that delivers python on Solaris. Recently we've > been getting requests for delivering python in 64-bit as well > as in 32-bit. As you probably know, Solaris can run 64-bit and > 32-bit binaries on the same system, but of course you can't mix and > match shared objects with different ISAs. This seems to apply to > python bytecode as well: the pyc files generated by a 64-bit build > of python are incompatible with those generated by the 32-bit python. > Note the caveat at > http://aspn.activestate.com/ASPN/docs/ActivePython/2.3/python/lib/module-marshal.html Which caveat, specifically? As it says there, the only known problem was fixed in Python 2.2: This behavior is new in Python 2.2. In earlier versions, all but the least- significant 32 bits of the value were lost, and a warning message was printed > I guess my first question is if there are any plans to make the > bytecodes for different ISAs compatible. That would make most of > our problems magically go away (; I suspect they already have ;-) There are no plans to make marshal store a Python long object on a 64-bit box for integers that fit in 64 points but not in 32 bits, and there would be very little point to doing so. As the referenced page says, you get the same numeric value regardless. It's /possible/ to write Python code to detect the difference in type, but real code wouldn't do that. > ... From rhettinger at ewtllc.com Wed Aug 30 02:20:34 2006 From: rhettinger at ewtllc.com (Raymond Hettinger) Date: Tue, 29 Aug 2006 17:20:34 -0700 Subject: [Python-Dev] Py2.5 issue: decimal context manager misimplemented, misdesigned, and misdocumented Message-ID: <44F4D9D2.2040804@ewtllc.com> I would like to see the changes to the decimal module reverted for the Py2.5 release. Currently, the code in the decimal module implements the context manager as a separate class instead of incorporating it directly in decimal.Context. This makes the API unnecessarily complex and is not pretty compared to the code it was intended to replace. Worse still, the implementation saves a reference to the context instead of making a copy of it. Remember decimal.Context objects are mutable -- the current implementation does not fulfill its contract to restore the context to its original state at the conclusion of the with-statement. The right way to do it was presented in PEP343. The implementation was correct and the API was simple. Additionally: * The examples in WhatsNew don't work because the implementation uses a different method name to fetch to context (this is a shallow error except that the name in WhatsNew is better and we don't really want to have a new method for this). It doesn't bode well that none of the release candidate end users noticed this discrepancy -- it means they are not trying out the examples. * The implementation's doc string examples were not tested and don't work (this is a deep error). One reads: with decimal.getcontext() as ctx: ctx.prec += 2 s = ... return +s To get this to work with the current implementation, it should read with decimal.getcontext().copy().get_manager() as ctx: ctx.prec += 2 s = ... return +s This is horrid. Please either revert the patch or fix it to match PEP-343. Raymond From pje at telecommunity.com Wed Aug 30 02:40:00 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 29 Aug 2006 20:40:00 -0400 Subject: [Python-Dev] Py2.5 issue: decimal context manager misimplemented, misdesigned, and misdocumented In-Reply-To: <44F4D9D2.2040804@ewtllc.com> Message-ID: <5.1.1.6.0.20060829203433.00a04748@sparrow.telecommunity.com> At 05:20 PM 8/29/2006 -0700, Raymond Hettinger wrote: >* The implementation's doc string examples were not tested and don't >work (this is a deep error). One reads: > > with decimal.getcontext() as ctx: > ctx.prec += 2 > s = ... > return +s > > >To get this to work with the current implementation, it should read > > with decimal.getcontext().copy().get_manager() as ctx: > ctx.prec += 2 > s = ... > return +s > >This is horrid. Please either revert the patch or fix it to match PEP-343. Actually, as I read the code, that would be: with decimal.getcontext().get_manager() as ctx: Which is still horrible, but unfortunately Guido has already pronounced that __context__ must be removed from PEP 343, which is what caused this abomination to come about. The PEP currently offers the idea of a 'localcontext()' API that provides a nicer spelling, but it appears nobody implemented it. From laca at sun.com Wed Aug 30 02:42:23 2006 From: laca at sun.com (Laszlo (Laca) Peter) Date: Tue, 29 Aug 2006 20:42:23 -0400 Subject: [Python-Dev] 32-bit and 64-bit python on Solaris In-Reply-To: <1f7befae0608291607p513f0201wda2465a5f4a73310@mail.gmail.com> References: <1156885465.1622.42.camel@ferrari.sfbay.sun.com> <1f7befae0608291607p513f0201wda2465a5f4a73310@mail.gmail.com> Message-ID: <1156898543.1622.52.camel@ferrari.sfbay.sun.com> On Tue, 2006-08-29 at 19:07 -0400, Tim Peters wrote: > > I work in the team that delivers python on Solaris. Recently we've > > been getting requests for delivering python in 64-bit as well > > as in 32-bit. As you probably know, Solaris can run 64-bit and > > 32-bit binaries on the same system, but of course you can't mix and > > match shared objects with different ISAs. This seems to apply to > > python bytecode as well: the pyc files generated by a 64-bit build > > of python are incompatible with those generated by the 32-bit > python. > > Note the caveat at > > > http://aspn.activestate.com/ASPN/docs/ActivePython/2.3/python/lib/module-marshal.html > > Which caveat, specifically? As it says there, the only known problem > was fixed in Python 2.2: > > This behavior is new in Python 2.2. In earlier versions, all but > the least- > significant 32 bits of the value were lost, and a warning message > was printed Hmm... I must be going blind reading that wrong. So it's expected to Just Work and if not, that's a bug. That's cool, thanks. Now all that is left is taking care of the 64-bit shared objects. Thanks, Laca From rhettinger at ewtllc.com Wed Aug 30 02:51:13 2006 From: rhettinger at ewtllc.com (Raymond Hettinger) Date: Tue, 29 Aug 2006 17:51:13 -0700 Subject: [Python-Dev] Py2.5 issue: decimal context manager misimplemented, misdesigned, and misdocumented In-Reply-To: <44F4D9D2.2040804@ewtllc.com> References: <44F4D9D2.2040804@ewtllc.com> Message-ID: <44F4E101.6080309@ewtllc.com> Raymond Hettinger wrote: >I would like to see the changes to the decimal module reverted for the >Py2.5 release. > >Currently, the code in the decimal module implements the context manager >as a separate class instead of incorporating it directly in >decimal.Context. > > That should read "... as a separate class instantiated by a brand-new Context method instead of implementing it as a module level function as shown in section 9 of PEP343". The API in PEP343 is much cleaner. Also, it doesn't have the copy vs reference bug. Raymond From rhettinger at ewtllc.com Wed Aug 30 02:57:12 2006 From: rhettinger at ewtllc.com (Raymond Hettinger) Date: Tue, 29 Aug 2006 17:57:12 -0700 Subject: [Python-Dev] Py2.5 issue: decimal context manager misimplemented, misdesigned, and misdocumented In-Reply-To: <5.1.1.6.0.20060829203433.00a04748@sparrow.telecommunity.com> References: <5.1.1.6.0.20060829203433.00a04748@sparrow.telecommunity.com> Message-ID: <44F4E268.9000703@ewtllc.com> Phillip J. Eby wrote: > At 05:20 PM 8/29/2006 -0700, Raymond Hettinger wrote: > >> * The implementation's doc string examples were not tested and don't >> work (this is a deep error). One reads: >> >> with decimal.getcontext() as ctx: >> ctx.prec += 2 >> s = ... >> return +s >> >> >> To get this to work with the current implementation, it should read >> >> with decimal.getcontext().copy().get_manager() as ctx: >> ctx.prec += 2 >> s = ... >> return +s >> >> This is horrid. Please either revert the patch or fix it to match >> PEP-343. > > > Actually, as I read the code, that would be: > > with decimal.getcontext().get_manager() as ctx: Given the current mis-implementation, the copy() step is absolutely necessary. Since context objects are mutable, the current context would never get it precision and flags restored. Try running the example and printing out the current context precision before and after the with-suite. You'll see that the context has changed (which defeats the whole purpose). > > Which is still horrible, but unfortunately Guido has already > pronounced that __context__ must be removed from PEP 343, which is > what caused this abomination to come about. > > The PEP currently offers the idea of a 'localcontext()' API that > provides a nicer spelling, but it appears nobody implemented it. > Right. The PEP version was correct and desirable. But at this point, it is better to have nothing at all. Users can still write their own (a la example 8 in the PEP). I do not want the decimal API to be forever mucked-up; hence, we should revert the change. Since it is buggy, that makes the decision an easy one to swallow. Raymond From anthony at interlink.com.au Wed Aug 30 04:41:41 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed, 30 Aug 2006 12:41:41 +1000 Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows In-Reply-To: <1f7befae0608291557l5b04a8f6wd1371e62a5c9c69c@mail.gmail.com> References: <20060826190600.0E75911002B@bromo.msbb.uc.edu> <20060829201022.GA22579@code0.codespeak.net> <1f7befae0608291557l5b04a8f6wd1371e62a5c9c69c@mail.gmail.com> Message-ID: <200608301241.46420.anthony@interlink.com.au> On Wednesday 30 August 2006 08:57, Tim Peters wrote: > Speaking of which, I saw no feedback on the proposed patch in > > http://mail.python.org/pipermail/python-dev/2006-August/068502.html > > so I'll just check that in tomorrow. Fine with me! thanks, Anthony -- Anthony Baxter It's never too late to have a happy childhood. From anthony at interlink.com.au Wed Aug 30 04:42:26 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed, 30 Aug 2006 12:42:26 +1000 Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows In-Reply-To: <1f7befae0608291557l5b04a8f6wd1371e62a5c9c69c@mail.gmail.com> References: <20060826190600.0E75911002B@bromo.msbb.uc.edu> <20060829201022.GA22579@code0.codespeak.net> <1f7befae0608291557l5b04a8f6wd1371e62a5c9c69c@mail.gmail.com> Message-ID: <200608301242.28648.anthony@interlink.com.au> On Wednesday 30 August 2006 08:57, Tim Peters wrote: > Speaking of which, I saw no feedback on the proposed patch in > > http://mail.python.org/pipermail/python-dev/2006-August/068502.html > > so I'll just check that in tomorrow. This should also be backported to release24-maint and release23-maint. Let me know if you can't do the backport... -- Anthony Baxter It's never too late to have a happy childhood. From pje at telecommunity.com Wed Aug 30 05:59:00 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 29 Aug 2006 23:59:00 -0400 Subject: [Python-Dev] Py2.5 issue: decimal context manager misimplemented, misdesigned, and misdocumented In-Reply-To: <44F4E268.9000703@ewtllc.com> References: <5.1.1.6.0.20060829203433.00a04748@sparrow.telecommunity.com> <5.1.1.6.0.20060829203433.00a04748@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20060829235607.04aaa878@sparrow.telecommunity.com> At 05:57 PM 8/29/2006 -0700, Raymond Hettinger wrote: >Phillip J. Eby wrote: > >>At 05:20 PM 8/29/2006 -0700, Raymond Hettinger wrote: >> >>>* The implementation's doc string examples were not tested and don't >>>work (this is a deep error). One reads: >>> >>> with decimal.getcontext() as ctx: >>> ctx.prec += 2 >>> s = ... >>> return +s >>> >>> >>>To get this to work with the current implementation, it should read >>> >>> with decimal.getcontext().copy().get_manager() as ctx: >>> ctx.prec += 2 >>> s = ... >>> return +s >>> >>>This is horrid. Please either revert the patch or fix it to match PEP-343. >> >> >>Actually, as I read the code, that would be: >> >> with decimal.getcontext().get_manager() as ctx: > >Given the current mis-implementation, the copy() step is absolutely >necessary. Since context objects are mutable, the current context would >never get it precision and flags restored. > >Try running the example and printing out the current context precision >before and after the with-suite. You'll see that the context has changed >(which defeats the whole purpose). No need; now that you've explained the problem I see why the code is wrong. This is definitely a bug in the decimal module. It looks like the code is correct at first glance, but the .copy() definitely needs to be in the ContextManager class, not the get_manager() method. Yuck. From nmm1 at cus.cam.ac.uk Wed Aug 30 12:23:26 2006 From: nmm1 at cus.cam.ac.uk (Nick Maclaren) Date: Wed, 30 Aug 2006 11:23:26 +0100 Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows Message-ID: "Tim Peters" wrote: > > This is a wrong time in the release process to take on chance on > discovering a flaky LONG_MIN on some box, so I want to keep the code > as much as possible like what's already there (which worked fine for > > 10 years on all known boxes) for now. No, it didn't. I reported a bug a couple of years back. A blanket rule not to use symbols is clearly wrong, but there are good reasons not to want to rely on LONG_MIN (or INT_MIN for that matter). Because of some incredibly complex issues (which I only know some of), it hasn't been consistently -(1+LONG_MAX) on twos' complement machines. There are good reasons for making it -LONG_MAX, but they aren't the ones that actually cause it to be so. There are, however, very good reasons for using BOTH tests. I.e. if I have a C system which defines LONG_MIN to be -LONG_MAX because it uses -(1+LONG_MAX) for an integer NaN indicator in some contexts, you really DON'T want to create such a value. I don't know if there are any such C systems, but there have been other languages that did. I hope that Guido wasn't saying that Python should EVER rely on signed integer overflow wrapping in twos' complement. Despite the common usage, Java and all that, it is perhaps the worst systematic architectural change to have happened in 30 years, and accounts for a good 30% of all hard bugs in many classes of program. Simple buffer overflow is fairly easy to avoid by good programming style; integer overflow causing trashing of unpredictable data isn't. Any decent programming language (like Python!) regards integer overflow as an error, and the moves to make C copy Java semantics are yet another step away from software engineering in the direction of who-gives-a-damn hacking. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1 at cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679 From ncoghlan at gmail.com Wed Aug 30 13:11:52 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 30 Aug 2006 21:11:52 +1000 Subject: [Python-Dev] Py2.5 issue: decimal context manager misimplemented, misdesigned, and misdocumented In-Reply-To: <44F4D9D2.2040804@ewtllc.com> References: <44F4D9D2.2040804@ewtllc.com> Message-ID: <44F57278.5090408@gmail.com> Raymond Hettinger wrote: > I would like to see the changes to the decimal module reverted for the > Py2.5 release. I believe you may be overreacting - I don't consider the current behaviour buggy and the module level API can be added later. That said, the docstring is definitely wrong, and I can't find any unit tests for the feature (I thought there were some in test_with, but I appear to be mistaken). > Currently, the code in the decimal module implements the context manager > as a separate class instead of incorporating it directly in > decimal.Context. This makes the API unnecessarily complex and is not > pretty compared to the code it was intended to replace. The removal of the __context__ method made it impossible to permit context objects to be used directly in with statements. Even when that was the case, the separate ContextManager class was necessary in order to correctly handle the restoration as context objects may be shared between threads or nested within a single thread [1]. The localcontext() function in PEP 343 does exactly the same thing - it merely uses a generator context instead of a direct implementation of __enter__ and __exit__. The current syntax (the get_manager() method) can easily be made prettier in the future by adding a sugar function at the module level: def localcontext(ctx=None): if ctx is None: ctx = getcontext() return ctx.get_manager() > Worse still, the implementation saves a reference to the context instead > of making a copy of it. Remember decimal.Context objects are mutable -- > the current implementation does not fulfill its contract to restore the > context to its original state at the conclusion of the with-statement. The implementation doesn't forget that. The context to restore is determined by calling getcontext() in the __enter__ method. The restored context has nothing to do with the context passed to the ContextManager constructor. >>> from decimal import getcontext() >>> getcontext().prec 28 >>> with getcontext().get_manager() as ctx: ... ctx.prec += 2 ... >>> getcontext().prec 28 The only ways to break it are to call ContextManager directly with an existing context that someone else already has a reference to: >>> from decimal import getcontext() >>> getcontext().prec 28 >>> with ContextManager(getcontext()) as ctx: ... ctx.prec += 2 ... >>> getcontext().prec 30 Or to deliberately reuse a ContextManager instance: >>> mgr = getcontext().get_manager() >>> with mgr as ctx: ... ctx.prec += 2 ... print ctx.prec ... 32 >>> with mgr as ctx: ... ctx.prec += 2 ... print ctx.prec ... 34 Cheers, Nick. [1] In fact, get_manager() is merely a new name for the old __context__ method. This name was suggested by Neal after I initially called the method context_manager(), but was never separately discussed on python-dev (the original discussion occurred in one of the massive PEP 343 threads). http://mail.python.org/pipermail/python-checkins/2006-May/052083.html -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From rhettinger at ewtllc.com Wed Aug 30 14:35:51 2006 From: rhettinger at ewtllc.com (Raymond Hettinger) Date: Wed, 30 Aug 2006 05:35:51 -0700 Subject: [Python-Dev] Py2.5 issue: decimal context manager misimplemented, misdesigned, and misdocumented In-Reply-To: <44F57278.5090408@gmail.com> References: <44F4D9D2.2040804@ewtllc.com> <44F57278.5090408@gmail.com> Message-ID: <44F58627.5040504@ewtllc.com> Nick Coghlan wrote: > Raymond Hettinger wrote: > >> I would like to see the changes to the decimal module reverted for >> the Py2.5 release. > > > I believe you may be overreacting - I don't consider the current > behaviour buggy and the module level API can be added later. My preference is to remove the method entirely and then implement the approach in PEP 343 by adding a module level "localcontext()" function in Py2.6. If you keep the method, then at least: * fix the docstring (and make it doctestable) * rename the method to .localcontext() * move the .copy() step to inside the contextmanager instead of its caller * update the WhatsNew example to match * add a unittest From rhettinger at ewtllc.com Wed Aug 30 14:58:51 2006 From: rhettinger at ewtllc.com (Raymond Hettinger) Date: Wed, 30 Aug 2006 05:58:51 -0700 Subject: [Python-Dev] Py2.5 issue: decimal context manager misimplemented, misdesigned, and misdocumented In-Reply-To: <44F58627.5040504@ewtllc.com> References: <44F4D9D2.2040804@ewtllc.com> <44F57278.5090408@gmail.com> <44F58627.5040504@ewtllc.com> Message-ID: <44F58B8B.5060301@ewtllc.com> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060830/9476b59b/attachment.htm From ncoghlan at gmail.com Thu Aug 31 12:08:36 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 31 Aug 2006 20:08:36 +1000 Subject: [Python-Dev] Py2.5 issue: decimal context manager misimplemented, misdesigned, and misdocumented In-Reply-To: <44F4D9D2.2040804@ewtllc.com> References: <44F4D9D2.2040804@ewtllc.com> Message-ID: <44F6B524.6060504@gmail.com> Raymond Hettinger wrote: > The right way to do it was presented in PEP343. The implementation was > correct and the API was simple. Raymond's persuaded me that he's right on the API part at the very least. The current API was a mechanical replacement of the initial __context__ based API with a normal method, whereas I should have reverted back to the module-level localcontext() function from PEP343 and thrown the method on Context objects away entirely. I can fix it on the trunk (and add those missing tests!), but I'll need Anthony and/or Neal's permission to backport it and remove the get_manager() method from Python 2.5 before we get stuck with it forever. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From mugunth at gmail.com Thu Aug 31 12:36:54 2006 From: mugunth at gmail.com (Muguntharaj Subramanian) Date: Thu, 31 Aug 2006 16:06:54 +0530 Subject: [Python-Dev] Error while building 2.5rc1 pythoncore_pgo on VC8 Message-ID: <8dd9fd0608310336q45d2d3d3re203e871c7b384b8@mail.gmail.com> Hi All, I tried building python 2.5c1 using VC8. Getting the following errors while building pythoncore_pgo: Creating library pythoncore_pgo/python25.lib and object pythoncore_pgo/python25.exp config.obj : error LNK2001: unresolved external symbol _init_types .\pythoncore_pgo/python25.dll : fatal error LNK1120: 1 unresolved externals I checked pythoncore_pgo.vcproj file, it alreadu contains: RelativePath="..\Modules\_typesmodule.c" Can anyone give me some clue to solve this issue. Thanks & Regards, Mugunth http://mugunth.blogspot.com http://thamizha.com http://tamilblogs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060831/1b491f77/attachment.htm From fredrik at pythonware.com Thu Aug 31 12:55:12 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 31 Aug 2006 12:55:12 +0200 Subject: [Python-Dev] Error while building 2.5rc1 pythoncore_pgo on VC8 In-Reply-To: <8dd9fd0608310336q45d2d3d3re203e871c7b384b8@mail.gmail.com> References: <8dd9fd0608310336q45d2d3d3re203e871c7b384b8@mail.gmail.com> Message-ID: Muguntharaj Subramanian wrote: > Hi All, > I tried building python 2.5c1 using VC8. > > Getting the following errors while building pythoncore_pgo: > > Creating library pythoncore_pgo/python25.lib and object > pythoncore_pgo/python25.exp > config.obj : error LNK2001: unresolved external symbol _init_types > .\pythoncore_pgo/python25.dll : fatal error LNK1120: 1 unresolved externals > > I checked pythoncore_pgo.vcproj file, it alreadu contains: > > RelativePath="..\Modules\_typesmodule.c" > > Can anyone give me some clue to solve this issue. googling for error messages is always a good idea: http://thread.gmane.org/gmane.comp.python.devel/83341 (see the last post in that thread for what I believe is a workaround) From mugunth at gmail.com Thu Aug 31 13:46:23 2006 From: mugunth at gmail.com (Muguntharaj Subramanian) Date: Thu, 31 Aug 2006 17:16:23 +0530 Subject: [Python-Dev] Error while building 2.5rc1 pythoncore_pgo on VC8 In-Reply-To: References: <8dd9fd0608310336q45d2d3d3re203e871c7b384b8@mail.gmail.com> Message-ID: <8dd9fd0608310446o6008240x8bfa852b41595eab@mail.gmail.com> > > > > I checked pythoncore_pgo.vcproj file, it alreadu contains: > > > > RelativePath="..\Modules\_typesmodule.c" > > > > Can anyone give me some clue to solve this issue. > > googling for error messages is always a good idea: > > http://thread.gmane.org/gmane.comp.python.devel/83341 > > (see the last post in that thread for what I believe is a workaround) > > > Yes, I have already done a bit of searching before posting that message. I have seen that previous post. That error mentioned in that post was in "pythoncore" module. My error is while compiling "pythoncore_pgo" module. And the workaround solution mentioned there is already there in the pythoncore_pgo.vcproj file. Thats why i mentioned RelativePath="..\Modules\ > > _typesmodule.c" is already there. I am still getting the same error. I am not able to find the cause yet. Any clue regarding this. Thanks & Regards, Mugunth http://mugunth.blogspot.com http://thamizha.com http://tamilblogs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060831/5c211830/attachment.html From kristjan at ccpgames.com Thu Aug 31 13:46:46 2006 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_V=2E_J=F3nsson?=) Date: Thu, 31 Aug 2006 11:46:46 -0000 Subject: [Python-Dev] Error while building 2.5rc1 pythoncore_pgo on VC8 Message-ID: <129CEF95A523704B9D46959C922A280002FE98F0@nemesis.central.ccp.cc> Hello all. I am working on updating the PCBuild8 directory in the trunk. This should fix this issue, unify platform builds for win32, x64 and I64, and provide better support for PGO builds. Hopefully when this is checked in, I can backport it to 2.5, since PCBuild8 is not an "official" platform for 2.5 Cheers, Kristjan > -----Original Message----- > From: python-dev-bounces+kristjan=ccpgames.com at python.org > [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] > On Behalf Of Fredrik Lundh > Sent: 31. ?g?st 2006 10:55 > To: python-dev at python.org > Subject: Re: [Python-Dev] Error while building 2.5rc1 > pythoncore_pgo on VC8 > > Muguntharaj Subramanian wrote: > > Hi All, > > I tried building python 2.5c1 using VC8. > > > > Getting the following errors while building pythoncore_pgo: > > > > Creating library pythoncore_pgo/python25.lib and object > > pythoncore_pgo/python25.exp config.obj : error LNK2001: unresolved > > external symbol _init_types .\pythoncore_pgo/python25.dll : fatal > > error LNK1120: 1 unresolved externals > > > > I checked pythoncore_pgo.vcproj file, it alreadu contains: > > > > RelativePath="..\Modules\_typesmodule.c" > > > > Can anyone give me some clue to solve this issue. > > googling for error messages is always a good idea: > > http://thread.gmane.org/gmane.comp.python.devel/83341 > > (see the last post in that thread for what I believe is a workaround) > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/kristjan%40c cpgames.com > From fredrik at pythonware.com Thu Aug 31 13:58:21 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 31 Aug 2006 13:58:21 +0200 Subject: [Python-Dev] Error while building 2.5rc1 pythoncore_pgo on VC8 In-Reply-To: <8dd9fd0608310446o6008240x8bfa852b41595eab@mail.gmail.com> References: <8dd9fd0608310336q45d2d3d3re203e871c7b384b8@mail.gmail.com> <8dd9fd0608310446o6008240x8bfa852b41595eab@mail.gmail.com> Message-ID: Muguntharaj Subramanian wrote: > That error mentioned in that post was in "pythoncore" module. > My error is while compiling "pythoncore_pgo" module. iirc, that's a partially experimental alternative build for playing with performance guided optimizations. are you sure you need that module ? From kristjan at ccpgames.com Thu Aug 31 14:03:25 2006 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_V=2E_J=F3nsson?=) Date: Thu, 31 Aug 2006 12:03:25 -0000 Subject: [Python-Dev] Error while building 2.5rc1 pythoncore_pgo on VC8 Message-ID: <129CEF95A523704B9D46959C922A280002FE98F1@nemesis.central.ccp.cc> I see. There is a file, called pythoncore_pgo_link.txt where you have to add the object too. But pythoncore_pgo is a bit broken in other ways at the moment. I am working on making it better in the trunk. (I also think that MS should improve their tools to make PGO building a one step process. the current recommended solution involves two project configurations...) Kristj?n ________________________________ From: python-dev-bounces+kristjan=ccpgames.com at python.org [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Muguntharaj Subramanian Sent: 31. ?g?st 2006 11:46 To: Fredrik Lundh Cc: python-dev at python.org Subject: Re: [Python-Dev] Error while building 2.5rc1 pythoncore_pgo on VC8 > I checked pythoncore_pgo.vcproj file, it alreadu contains: > > RelativePath="..\Modules\_typesmodule.c" > > Can anyone give me some clue to solve this issue. googling for error messages is always a good idea: http://thread.gmane.org/gmane.comp.python.devel/83341 (see the last post in that thread for what I believe is a workaround) Yes, I have already done a bit of searching before posting that message. I have seen that previous post. That error mentioned in that post was in "pythoncore" module. My error is while compiling "pythoncore_pgo" module. And the workaround solution mentioned there is already there in the pythoncore_pgo.vcproj file. Thats why i mentioned RelativePath="..\Modules\ _typesmodule.c" is already there. I am still getting the same error. I am not able to find the cause yet. Any clue regarding this. Thanks & Regards, Mugunth http://mugunth.blogspot.com http://thamizha.com http://tamilblogs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060831/217a8a24/attachment.htm From ncoghlan at gmail.com Thu Aug 31 14:08:12 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 31 Aug 2006 22:08:12 +1000 Subject: [Python-Dev] Py2.5 issue: decimal context manager misimplemented, misdesigned, and misdocumented In-Reply-To: <44F6B524.6060504@gmail.com> References: <44F4D9D2.2040804@ewtllc.com> <44F6B524.6060504@gmail.com> Message-ID: <44F6D12C.4040808@gmail.com> Nick Coghlan wrote: > Raymond Hettinger wrote: >> The right way to do it was presented in PEP343. The implementation was >> correct and the API was simple. > > Raymond's persuaded me that he's right on the API part at the very least. The > current API was a mechanical replacement of the initial __context__ based API > with a normal method, whereas I should have reverted back to the module-level > localcontext() function from PEP343 and thrown the method on Context objects > away entirely. > > I can fix it on the trunk (and add those missing tests!), but I'll need > Anthony and/or Neal's permission to backport it and remove the get_manager() > method from Python 2.5 before we get stuck with it forever. I committed this fix as 51664 on the trunk (although the docstrings are still example free because doctest doesn't understand __future__ statements). Anthony, Neal: I'd like to backport this change to the 2.5 maintenance branch. I realise it is an API change between the release candidate and the actual release, but this really is a small tweak to something nobody is actually using yet. If that's not acceptable, I'd like to go with Raymond's original option: rip it out entirely for 2.5 so we don't get stuck maintaining it for the rest of the 2.x series. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From skip at pobox.com Thu Aug 31 16:06:28 2006 From: skip at pobox.com (skip at pobox.com) Date: Thu, 31 Aug 2006 09:06:28 -0500 Subject: [Python-Dev] link to community buildbot? Message-ID: <17654.60644.37459.683994@montanaro.dyndns.org> There was a buildbot set up recently to test various external packages (like twisted) against Python's source code. I don't recall where it is, but it would be nice if relevant links to it existed on this page: http://www.python.org/dev/buildbot/ In addition, if a 2.3.6 release is going to happen it seems like it would be a good idea to set up the relevant bits on both buildbots. Skip From alan.mcintyre at gmail.com Thu Aug 31 16:22:30 2006 From: alan.mcintyre at gmail.com (Alan McIntyre) Date: Thu, 31 Aug 2006 10:22:30 -0400 Subject: [Python-Dev] Windows build slave downtime Message-ID: <1d36917a0608310722i49cb9404ic5e90fe8248945bf@mail.gmail.com> Hi all, The Windows build slave ("x86 XP trunk") will be down for most of today while I move to a new apartment. As long as the phone company manages to provide me with an internet connection by the end of today, it should be available again some time this evening. Alan From grig.gheorghiu at gmail.com Thu Aug 31 16:34:37 2006 From: grig.gheorghiu at gmail.com (Grig Gheorghiu) Date: Thu, 31 Aug 2006 07:34:37 -0700 Subject: [Python-Dev] link to community buildbot? In-Reply-To: <17654.60644.37459.683994@montanaro.dyndns.org> References: <17654.60644.37459.683994@montanaro.dyndns.org> Message-ID: <3f09d5a00608310734k7c391fdbj8b26e9a08bb5bb7b@mail.gmail.com> On 8/31/06, skip at pobox.com wrote: > > > There was a buildbot set up recently to test various external packages > (like twisted) against Python's source code. I don't recall where it is, > but it would be nice if relevant links to it existed on this page: > > http://www.python.org/dev/buildbot/ > > In addition, if a 2.3.6 release is going to happen it seems like it would > be > a good idea to set up the relevant bits on both buildbots. > > Skip > _______________________________________________ The link is this one: http://www.python.org/dev/buildbot/community/all/ BTW, I haven't had many takers since I announced this project....So if anybody on these lists is interested in setting up a buildslave, please let me know. There's also a mailing list here: http://lists2.idyll.org/listinfo/pybots Thanks, Grig -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060831/ea00240d/attachment.html From rhettinger at ewtllc.com Thu Aug 31 19:01:16 2006 From: rhettinger at ewtllc.com (Raymond Hettinger) Date: Thu, 31 Aug 2006 10:01:16 -0700 Subject: [Python-Dev] Py2.5 issue: decimal context manager misimplemented, misdesigned, and misdocumented In-Reply-To: <44F6B524.6060504@gmail.com> References: <44F4D9D2.2040804@ewtllc.com> <44F6B524.6060504@gmail.com> Message-ID: <44F715DC.1090001@ewtllc.com> Nick Coghlan wrote: > Raymond Hettinger wrote: > >> The right way to do it was presented in PEP343. The implementation >> was correct and the API was simple. > > > Raymond's persuaded me that he's right on the API part at the very > least. The current API was a mechanical replacement of the initial > __context__ based API with a normal method, whereas I should have > reverted back to the module-level localcontext() function from PEP343 > and thrown the method on Context objects away entirely. > > I can fix it on the trunk (and add those missing tests!), but I'll > need Anthony and/or Neal's permission to backport it and remove the > get_manager() method from Python 2.5 before we get stuck with it forever. > Please go ahead and get the patch together for localcontext(). This should be an easy sell: * simple bugs can be fixed in Py2.5.1 but API mistakes are forever. * currently, all of the docs, docstrings, and whatsnew are incorrect. * the solution has already been worked-out in PEP343 -- it's nothing new. * nothing else, anywhere depends on this code -- it is as safe a change as we could hope for. Neal is tough, but he's not heartless ;-) Raymond From johann at rocholl.net Thu Aug 31 21:55:23 2006 From: johann at rocholl.net (Johann C. Rocholl) Date: Thu, 31 Aug 2006 21:55:23 +0200 Subject: [Python-Dev] That library reference, yet again Message-ID: <8233478f0608311255o7058a1feo55c710e7eb8e6b6c@mail.gmail.com> What is the status of http://effbot.org/lib/ ? I think it's a step in the right direction. Is it still in progress? Can I have access to toolchain source code and/or conversion output? I would like to set up something like gotapi.com for the Python library reference, or help the gotapi people to include it. But gotapi needs link targets for functions etc., but the original Python library reference doesn't have them. I really like the simple linking idea here: http://online.effbot.org/2006_01_01_archive.htm#link-doc Here's another working example with (ugly) link targets: http://www.zvon.org/other/python/PHP/search.php Cheers, Johann From jsceballos at gmail.com Thu Aug 31 22:18:04 2006 From: jsceballos at gmail.com (Javier Sanz) Date: Thu, 31 Aug 2006 22:18:04 +0200 Subject: [Python-Dev] That library reference, yet again In-Reply-To: <8233478f0608311255o7058a1feo55c710e7eb8e6b6c@mail.gmail.com> References: <8233478f0608311255o7058a1feo55c710e7eb8e6b6c@mail.gmail.com> Message-ID: <64c763bf0608311318n11afd9a7s6943c42905d16cad@mail.gmail.com> +1 to that idea -- Un saludo, Javier