From greg.ewing at canterbury.ac.nz Wed Mar 1 00:05:43 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 01 Mar 2006 12:05:43 +1300 Subject: [Python-Dev] defaultdict and on_missing() In-Reply-To: References: <20060222131328.nzferzk2xxk448cg@login.werra.lunarpages.com> <43FE2F5C.8090700@livinglogic.de> <327376ff0602231839h4c27c999la52039be37c28230@mail.gmail.com> <43FEAD58.2020208@canterbury.ac.nz> <015901c6395d$3b9fd4b0$2e2a960a@RaymondLaptop1> <43FFA4B2.7060303@canterbury.ac.nz> <4402F6A9.1030906@gmail.com> <4403E263.1060505@canterbury.ac.nz> <440411E1.6040401@gmail.com> Message-ID: <4404D747.2050402@canterbury.ac.nz> Guido van Rossum wrote: > On 2/28/06, Nick Coghlan wrote: > > > PEP 342 opted to extend the generator API instead (using "send") and leave the > > iterator protocol alone for the time being. > > One of the main reasons for this was the backwards compatibility > problems at the C level. I'm really quite happy either way. Having the functionality available in some way is the important thing. I'd still like to see next(x) / x.__next__() in some form in 3.0 for the sake of consistency, though. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Wed Mar 1 00:07:05 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 01 Mar 2006 12:07:05 +1300 Subject: [Python-Dev] bytes.from_hex() In-Reply-To: <06Feb28.090748pst.58633@synergy1.parc.xerox.com> References: <06Feb28.090748pst.58633@synergy1.parc.xerox.com> Message-ID: <4404D799.6080901@canterbury.ac.nz> Bill Janssen wrote: > bytes -> base64 -> text > text -> de-base64 -> bytes It's nice to hear I'm not out of step with the entire world on this. :-) -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Wed Mar 1 00:07:09 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 01 Mar 2006 12:07:09 +1300 Subject: [Python-Dev] str.count is slow In-Reply-To: References: <1141073696.107136.318600@j33g2000cwa.googlegroups.com> <1141083127.970403.147100@v46g2000cwv.googlegroups.com> <4403EFEE.80002@canterbury.ac.nz> Message-ID: <4404D79D.2070608@canterbury.ac.nz> Fredrik Lundh wrote: > > My personal goal in life right now is to stay as > > far away from C++ as I can get. > > so what C compiler are you using ? Gcc, mostly. I don't mind if it's capable of compiling C++, as long as I can choose not to write any. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From guido at python.org Wed Mar 1 00:14:10 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 28 Feb 2006 17:14:10 -0600 Subject: [Python-Dev] str.count is slow In-Reply-To: <4404D79D.2070608@canterbury.ac.nz> References: <1141073696.107136.318600@j33g2000cwa.googlegroups.com> <1141083127.970403.147100@v46g2000cwv.googlegroups.com> <4403EFEE.80002@canterbury.ac.nz> <4404D79D.2070608@canterbury.ac.nz> Message-ID: On 2/28/06, Greg Ewing wrote: > Fredrik Lundh wrote: > > > > My personal goal in life right now is to stay as > > > far away from C++ as I can get. > > > > so what C compiler are you using ? > > Gcc, mostly. I don't mind if it's capable of > compiling C++, as long as I can choose not to > write any. That's getting harder and harder though. Recent versions of GCC appear to be implementing C98 by default -- at least I didn't get complaints about declarations placed after non-declarations in the same block from any of the buildbot hosts... -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Mar 1 00:15:12 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 28 Feb 2006 17:15:12 -0600 Subject: [Python-Dev] bytes.from_hex() In-Reply-To: <4404D799.6080901@canterbury.ac.nz> References: <06Feb28.090748pst.58633@synergy1.parc.xerox.com> <4404D799.6080901@canterbury.ac.nz> Message-ID: On 2/28/06, Greg Ewing wrote: > Bill Janssen wrote: > > > bytes -> base64 -> text > > text -> de-base64 -> bytes > > It's nice to hear I'm not out of step with > the entire world on this. :-) What Bill proposes makes sense to me. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From janssen at parc.com Wed Mar 1 00:23:43 2006 From: janssen at parc.com (Bill Janssen) Date: Tue, 28 Feb 2006 15:23:43 PST Subject: [Python-Dev] bytes.from_hex() In-Reply-To: Your message of "Tue, 28 Feb 2006 15:07:05 PST." <4404D799.6080901@canterbury.ac.nz> Message-ID: <06Feb28.152344pst."58633"@synergy1.parc.xerox.com> Greg Ewing wrote: > Bill Janssen wrote: > > > bytes -> base64 -> text > > text -> de-base64 -> bytes > > It's nice to hear I'm not out of step with > the entire world on this. :-) Well, I can certainly understand the bytes->base64->bytes side of thing too. The "text" produced is specified as using "a 65-character subset of US-ASCII", so that's really bytes. Bill From raymond.hettinger at verizon.net Wed Mar 1 00:28:38 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Tue, 28 Feb 2006 18:28:38 -0500 Subject: [Python-Dev] defaultdict and on_missing() References: <20060222131328.nzferzk2xxk448cg@login.werra.lunarpages.com><43FE2F5C.8090700@livinglogic.de><327376ff0602231839h4c27c999la52039be37c28230@mail.gmail.com><43FEAD58.2020208@canterbury.ac.nz><015901c6395d$3b9fd4b0$2e2a960a@RaymondLaptop1><43FFA4B2.7060303@canterbury.ac.nz> <4402F6A9.1030906@gmail.com><4403E263.1060505@canterbury.ac.nz> <440411E1.6040401@gmail.com> <4404D747.2050402@canterbury.ac.nz> Message-ID: <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> [Greg Ewing] > I'd still like to see next(x) / x.__next__() in > some form in 3.0 for the sake of consistency, > though. +1 on modifying next() in Py3.0 to accept the send argument. -1 on the silly renaming to __next__ and adding __builtin__.next(). We have len() because it applies to many different object types. In contrast, next() would apply only to iterables. Foolish consistency is a somewhat weak reason to add a built-in. Also, calls to it.next() need to be as fast as possible. Adding a layer of indirection through __builtin__.next() is a waste. Raymond From fredrik at pythonware.com Wed Mar 1 00:26:17 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 1 Mar 2006 00:26:17 +0100 Subject: [Python-Dev] C++ for CPython 3? (Re: str.count is slow) References: <1141073696.107136.318600@j33g2000cwa.googlegroups.com><1141083127.970403.147100@v46g2000cwv.googlegroups.com> <4403EFEE.80002@canterbury.ac.nz> <4404D79D.2070608@canterbury.ac.nz> Message-ID: Guido van Rossum wrote: > > Fredrik Lundh wrote: > > > > > > My personal goal in life right now is to stay as > > > > far away from C++ as I can get. > > > > > > so what C compiler are you using ? > > > > Gcc, mostly. I don't mind if it's capable of > > compiling C++, as long as I can choose not to > > write any. > > That's getting harder and harder though. Recent versions of GCC appear > to be implementing C98 by default -- at least I didn't get complaints > about declarations placed after non-declarations in the same block > from any of the buildbot hosts... should we perhaps switch to (careful use of) C++ in 3.0 ? From foom at fuhm.net Wed Mar 1 03:00:49 2006 From: foom at fuhm.net (James Y Knight) Date: Tue, 28 Feb 2006 21:00:49 -0500 Subject: [Python-Dev] str.count is slow In-Reply-To: References: <1141073696.107136.318600@j33g2000cwa.googlegroups.com> <1141083127.970403.147100@v46g2000cwv.googlegroups.com> <4403EFEE.80002@canterbury.ac.nz> <4404D79D.2070608@canterbury.ac.nz> Message-ID: <2FD1EDB8-D01A-4C67-81E1-7B9AC1C22A43@fuhm.net> On Feb 28, 2006, at 6:14 PM, Guido van Rossum wrote: > On 2/28/06, Greg Ewing wrote: >> Fredrik Lundh wrote: >> >>>> My personal goal in life right now is to stay as >>>> far away from C++ as I can get. >>> >>> so what C compiler are you using ? >> >> Gcc, mostly. I don't mind if it's capable of >> compiling C++, as long as I can choose not to >> write any. > > That's getting harder and harder though. Recent versions of GCC appear > to be implementing C98 by default -- at least I didn't get complaints > about declarations placed after non-declarations in the same block > from any of the buildbot hosts... I don't know whether you meant "C++98" or "C99" in the above, but the default is (mostly) C99, now. If you like, you can still tell it to compile in C89 mode, with --std=c89. C99 contains some of the superficial C++ syntax changes such as // comments and declarations after non-declarations which have long been implemented as non-standard extensions, anyhow, but it's still nothing like C++. As for the question of whether to switch to C++ in 3.0, I'd say probably not, as it's much harder to interface with C++ from other languages than to C. James From janssen at parc.com Wed Mar 1 05:05:26 2006 From: janssen at parc.com (Bill Janssen) Date: Tue, 28 Feb 2006 20:05:26 PST Subject: [Python-Dev] str.count is slow In-Reply-To: Your message of "Tue, 28 Feb 2006 18:00:49 PST." <2FD1EDB8-D01A-4C67-81E1-7B9AC1C22A43@fuhm.net> Message-ID: <06Feb28.200535pst."58633"@synergy1.parc.xerox.com> > As for the question of whether to switch to C++ in 3.0, I'd say > probably not, as it's much harder to interface with C++ from other > languages than to C. An excellent point, and to my mind conclusive. Bill From greg.ewing at canterbury.ac.nz Wed Mar 1 05:13:07 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 01 Mar 2006 17:13:07 +1300 Subject: [Python-Dev] bytes.from_hex() In-Reply-To: <06Feb28.152344pst.58633@synergy1.parc.xerox.com> References: <06Feb28.152344pst.58633@synergy1.parc.xerox.com> Message-ID: <44051F53.5030307@canterbury.ac.nz> Bill Janssen wrote: > Well, I can certainly understand the bytes->base64->bytes side of > thing too. The "text" produced is specified as using "a 65-character > subset of US-ASCII", so that's really bytes. But it then goes on to say that these same characters are also a subset of EBCDIC. So it seems to be talking about characters as abstract entities here, not as bit patterns. Greg From greg.ewing at canterbury.ac.nz Wed Mar 1 05:16:29 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 01 Mar 2006 17:16:29 +1300 Subject: [Python-Dev] str.count is slow In-Reply-To: References: <1141073696.107136.318600@j33g2000cwa.googlegroups.com> <1141083127.970403.147100@v46g2000cwv.googlegroups.com> <4403EFEE.80002@canterbury.ac.nz> <4404D79D.2070608@canterbury.ac.nz> Message-ID: <4405201D.9040300@canterbury.ac.nz> Guido van Rossum wrote: > Recent versions of GCC appear > to be implementing C98 by default -- at least I didn't get complaints > about declarations placed after non-declarations in the same block > from any of the buildbot hosts... As long as it doesn't complain when I *do* put all my declarations before my non-declarations, I can live with that. :-) -- Greg From greg.ewing at canterbury.ac.nz Wed Mar 1 05:28:25 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 01 Mar 2006 17:28:25 +1300 Subject: [Python-Dev] C++ for CPython 3? (Re: str.count is slow) In-Reply-To: References: <1141073696.107136.318600@j33g2000cwa.googlegroups.com> <1141083127.970403.147100@v46g2000cwv.googlegroups.com> <4403EFEE.80002@canterbury.ac.nz> <4404D79D.2070608@canterbury.ac.nz> Message-ID: <440522E9.70106@canterbury.ac.nz> Fredrik Lundh wrote: > should we perhaps switch to (careful use of) C++ in 3.0 ? I worry that if the Python core becomes dependent on C++, it will force all extensions to be written in C++, too. Not only is this inconvenient for people who don't know C++ or prefer not to use it, but I suspect this would lead to some severe interoperability problems. On most platforms, C code will happily link to just about anything else, but in my experience, C++ code will only interoperate with other C++ code when it's compiled by exactly the same compiler and uses the same runtime system. For another thing, I'm not sure that C++ polymorphism is flexible enough to implement Python's style of polymorphism. Think of all the tricks we play with type objects -- allocating them on the heap, extending them dynamically at runtime, etc. In C++, classes aren't even available as program-accessible entities. So we'd end up implementing our own form of dynamic dispatch anyway, and it would probably look a lot like the current C-based type objects. I don't see that C++ would help much. -- Greg From greg.ewing at canterbury.ac.nz Wed Mar 1 05:42:30 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 01 Mar 2006 17:42:30 +1300 Subject: [Python-Dev] defaultdict and on_missing() In-Reply-To: <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> References: <20060222131328.nzferzk2xxk448cg@login.werra.lunarpages.com> <43FE2F5C.8090700@livinglogic.de> <327376ff0602231839h4c27c999la52039be37c28230@mail.gmail.com> <43FEAD58.2020208@canterbury.ac.nz> <015901c6395d$3b9fd4b0$2e2a960a@RaymondLaptop1> <43FFA4B2.7060303@canterbury.ac.nz> <4402F6A9.1030906@gmail.com> <4403E263.1060505@canterbury.ac.nz> <440411E1.6040401@gmail.com> <4404D747.2050402@canterbury.ac.nz> <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> Message-ID: <44052636.9090709@canterbury.ac.nz> Raymond Hettinger wrote: > -1 on the silly renaming to __next__ and adding __builtin__.next(). > We have len() because it applies to many different object types. > In contrast, next() would apply only to iterables. And you don't think there are many different types of iterables? You might as well argue that we don't need len() because it "only applies to sequences". There's a practical reason for using the xxx(obj)/obj.__xxx__() pattern -- it provides a lot more flexibility when it comes to future extensions of the protocol. We've used this to good effect already with iter(), which uses __iter__ if it's available, else falls back on another technique. If we had used this pattern with next() from the beginning then, for example, there would have been no problem adding an argument to next(), since the next() function could just have been updated to look for __send__ or something like that, and existing iterators need not have been affected. So the reason I'd like to see this changed in 3.0 is not because of consistency for its own sake, but because it would provide a better basis for future development. -- Greg From raymond.hettinger at verizon.net Wed Mar 1 06:43:06 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Wed, 01 Mar 2006 00:43:06 -0500 Subject: [Python-Dev] iterator API in Py3.0 References: <20060222131328.nzferzk2xxk448cg@login.werra.lunarpages.com> <43FE2F5C.8090700@livinglogic.de> <327376ff0602231839h4c27c999la52039be37c28230@mail.gmail.com> <43FEAD58.2020208@canterbury.ac.nz> <015901c6395d$3b9fd4b0$2e2a960a@RaymondLaptop1> <43FFA4B2.7060303@canterbury.ac.nz> <4402F6A9.1030906@gmail.com> <4403E263.1060505@canterbury.ac.nz> <440411E1.6040401@gmail.com> <4404D747.2050402@canterbury.ac.nz> <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> <44052636.9090709@canterbury.ac.nz> Message-ID: <004001c63cf3$043fd710$6a01a8c0@RaymondLaptop1> >> -1 on the silly renaming to __next__ and adding __builtin__.next(). >> We have len() because it applies to many different object types. >> In contrast, next() would apply only to iterables. [Greg Ewing] > And you don't think there are many different > types of iterables? Um, I meant iterators and suspect you meant the same -- iow, objects returned by iter(x) that are currently guaranteed to support __iter__() and next() methods (i.e. generator-iterators, not the generator-functions that create them). > So the reason I'd like to see this changed > in 3.0 is not because of consistency for its > own sake, but because it would provide a > better basis for future development. YAGNI. The hypothetical "future development" benefit is even more specious than the appeal for consistency. Let's try to resist mucking-up this simple, elegant, and fast iterface by adding another level of indirection. I'm strongly -1 on this proposal and will likely remain so until shown demonstrable benefits for real use cases that are compelling enough to warrant cluttering and slowing the iterator interface. I usually let this sort of thing slide, but the iterator API is too important for trivial, incompatible, and damaging changes. Looking back at Guido's original rationale for naming the method next() instead of __next__(), http://www.python.org/peps/pep-0234.html , it does not look like any of those reasons will cease to apply in Py3.0. If you want a next() function, it is trivial to write your own pure Python version and leave the API unmolested for everyone else. Since it is already possible to write such a function, I'm curious whether you've had occasion to do so in real code and whether it provided any benefits in terms of consistency or future development flexibility. Most likely, the answer is no. That should be a good indicator that the proposed benefits are illusory. Raymond From tim.peters at gmail.com Wed Mar 1 07:00:56 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 1 Mar 2006 00:00:56 -0600 Subject: [Python-Dev] New test failure on Windows Message-ID: <1f7befae0602282200k46ef03a5g24d5a029d1f0e0b3@mail.gmail.com> test test_pep352 failed -- Traceback (most recent call last): File "C:\Code\python\lib\test\test_pep352.py", line 54, in test_inheritance self.fail("%s not a built-in exception" % exc_name) AssertionError: WindowsError (Windows) not a built-in exception From tim.peters at gmail.com Wed Mar 1 07:17:16 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 1 Mar 2006 00:17:16 -0600 Subject: [Python-Dev] .py and .txt files missing svn:eol-style in trunk Message-ID: <1f7befae0602282217j1700cfb1j4b73ddf80ef1428a@mail.gmail.com> These .py and .txt files don't have the svn:eol-style property set. I'm not sure they all _should_, though. Some of them are particularly bizarre, e.g. Lib\email\test\data\msg_26.txt has the svn:mime-type property set to application/octet-stream (WTF?), and then svn refuses to set the eol-style property. .\Lib\bsddb\test\test_1413192.py .\Lib\email\test\data\msg_26.txt .\Lib\email\test\data\msg_44.txt .\Lib\encodings\utf_8_sig.py .\Lib\test\bad_coding2.py .\Lib\test\outstanding_bugs.py .\Lib\test\test_defaultdict.py .\Lib\test\test_exception_variations.py .\Lib\test\test_pep263.py .\Lib\test\test_platform.py .\Lib\test\crashers\coerce.py .\Lib\test\crashers\dangerous_subclassing.py .\Lib\test\crashers\infinite_rec_1.py .\Lib\test\crashers\infinite_rec_2.py .\Lib\test\crashers\infinite_rec_3.py .\Lib\test\crashers\infinite_rec_4.py .\Lib\test\crashers\infinite_rec_5.py .\Lib\test\crashers\loosing_dict_ref.py .\Lib\test\crashers\modify_dict_attr.py .\Lib\test\crashers\recursive_call.py .\Lib\test\crashers\weakref_in_del.py .\Lib\test\crashers\xml_parsers.py .\Lib\xmlcore\etree\cElementTree.py .\Modules\zlib\algorithm.txt .\PC\readme.txt .\PC\testpy.py .\PC\example_nt\readme.txt .\PC\os2vacpp\readme.txt OK, I tried to set eol-style on all of those. svn refused to change these: svn: File 'Lib\email\test\data\msg_26.txt' has binary mime type property svn: File 'Lib\test\test_pep263.py' has binary mime type property svn: File 'PC\readme.txt' has binary mime type property svn: File 'PC\testpy.py' has binary mime type property svn: File 'PC\example_nt\readme.txt' has binary mime type property svn: File 'PC\os2vacpp\readme.txt' has binary mime type property test_pepe263 probably should be binary (it contains "funny characters"). I'll wrestle with the ones under PC/. From thomas at python.org Wed Mar 1 07:51:04 2006 From: thomas at python.org (Thomas Wouters) Date: Wed, 1 Mar 2006 07:51:04 +0100 Subject: [Python-Dev] Webstats for www.python.org et al. Message-ID: <9e804ac0602282251n9d958a2kbd92a27ea3133a60@mail.gmail.com> I finally took the time to set up webalizer on dinsdale.python.org, after the move of the web content from creosote.python.org to dinsdale.python.org last October or November. The apache logs on dinsdale don't go back farther than 25 December, looks like, though. None of my backups have 'em either. I didn't bother regenerate the stats from the logfiles on creosote.python.org because of that, since there would be a two-month gap. We can still compare the last two month's of data to whatever we'll be generating once beta.python.org is final :-) And if anyone happens to have the older dinsdale logs somewhere, I can still regenerate the main stats. I added webstats for all subsites of python.org: http://www.python.org/webstats/ http://beta.python.org/webstats/ http://bugs.python.org/webstats/ http://planet.python.org/webstats/ http://docs.python.org/webstats/ http://doc.python.org/webstats/ http://svn.python.org/webstats/ and even http://www.pythonlabs.com/webstats/ (not working right now, not sure why) http://www.jpython.org/webstats/ (not working because of the global redirect) I think I'll merge doc.python.org and docs.python.org tomorrow. (And maybe add the doc/docs.python.org stats to www.python.org's stats after prepending '/doc' to all URLs, but that's a fair bit of work.) I'd also like to add a stats.python.org with direct listings of all those stats, unless someone sees a reason not to. The svn.python.org stats include only http- and https-traffic, not svn+ssh traffic (the authenticated, read-write traffic.) They're pretty interesting to look at, though, particularly the extreme dip in access on the 27th and 28th of February :) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From tim.peters at gmail.com Wed Mar 1 08:09:58 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 1 Mar 2006 01:09:58 -0600 Subject: [Python-Dev] Using and binding relative names (was Re: PEP forBetter Control of Nested Lexical Scopes) In-Reply-To: <4402C8F8.1040604@canterbury.ac.nz> References: <20060220204850.6004.JCARLSON@uci.edu> <7e9b97090602252315mf6d4686ud86dd5163ea76b37@mail.gmail.com> <440205C6.3030301@ronadam.com> <440245B7.5090903@ronadam.com> <1956B0C6-4FE7-41EC-BCBA-37C25CBE470D@gmail.com> <44025955.7090709@ronadam.com> <4402C8F8.1040604@canterbury.ac.nz> Message-ID: <1f7befae0602282309q21534789xdff87625975db226@mail.gmail.com> [Alex Martelli] >> We stole list comprehensions and genexps from Haskell [Greg Ewing] > The idea predates Haskell, I think. I first saw it in > Miranda, and it may have come from something even > earlier -- SETL, maybe? Haskell indeed took list comprehensions from SETL. SETL in turn took them from much earlier standard notation in set theory, related to the oddly named (but not universally so named) "axiom of comprehension". genexps were more directly taken from Icon, but tying them into Python's iteration protocol is a powerful twist not directly present in Icon. From tim.peters at gmail.com Wed Mar 1 08:09:58 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 1 Mar 2006 01:09:58 -0600 Subject: [Python-Dev] Using and binding relative names (was Re: PEP forBetter Control of Nested Lexical Scopes) In-Reply-To: <4402C8F8.1040604@canterbury.ac.nz> References: <20060220204850.6004.JCARLSON@uci.edu> <7e9b97090602252315mf6d4686ud86dd5163ea76b37@mail.gmail.com> <440205C6.3030301@ronadam.com> <440245B7.5090903@ronadam.com> <1956B0C6-4FE7-41EC-BCBA-37C25CBE470D@gmail.com> <44025955.7090709@ronadam.com> <4402C8F8.1040604@canterbury.ac.nz> Message-ID: <1f7befae0602282309q21534789xdff87625975db226@mail.gmail.com> [Alex Martelli] >> We stole list comprehensions and genexps from Haskell [Greg Ewing] > The idea predates Haskell, I think. I first saw it in > Miranda, and it may have come from something even > earlier -- SETL, maybe? Haskell indeed took list comprehensions from SETL. SETL in turn took them from much earlier standard notation in set theory, related to the oddly named (but not universally so named) "axiom of comprehension". genexps were more directly taken from Icon, but tying them into Python's iteration protocol is a powerful twist not directly present in Icon. From tim.peters at gmail.com Wed Mar 1 08:09:58 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 1 Mar 2006 01:09:58 -0600 Subject: [Python-Dev] Using and binding relative names (was Re: PEP forBetter Control of Nested Lexical Scopes) In-Reply-To: <4402C8F8.1040604@canterbury.ac.nz> References: <20060220204850.6004.JCARLSON@uci.edu> <7e9b97090602252315mf6d4686ud86dd5163ea76b37@mail.gmail.com> <440205C6.3030301@ronadam.com> <440245B7.5090903@ronadam.com> <1956B0C6-4FE7-41EC-BCBA-37C25CBE470D@gmail.com> <44025955.7090709@ronadam.com> <4402C8F8.1040604@canterbury.ac.nz> Message-ID: <1f7befae0602282309q21534789xdff87625975db226@mail.gmail.com> [Alex Martelli] >> We stole list comprehensions and genexps from Haskell [Greg Ewing] > The idea predates Haskell, I think. I first saw it in > Miranda, and it may have come from something even > earlier -- SETL, maybe? Haskell indeed took list comprehensions from SETL. SETL in turn took them from much earlier standard notation in set theory, related to the oddly named (but not universally so named) "axiom of comprehension". genexps were more directly taken from Icon, but tying them into Python's iteration protocol is a powerful twist not directly present in Icon. From tim.peters at gmail.com Wed Mar 1 08:09:58 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 1 Mar 2006 01:09:58 -0600 Subject: [Python-Dev] Using and binding relative names (was Re: PEP forBetter Control of Nested Lexical Scopes) In-Reply-To: <4402C8F8.1040604@canterbury.ac.nz> References: <20060220204850.6004.JCARLSON@uci.edu> <7e9b97090602252315mf6d4686ud86dd5163ea76b37@mail.gmail.com> <440205C6.3030301@ronadam.com> <440245B7.5090903@ronadam.com> <1956B0C6-4FE7-41EC-BCBA-37C25CBE470D@gmail.com> <44025955.7090709@ronadam.com> <4402C8F8.1040604@canterbury.ac.nz> Message-ID: <1f7befae0602282309q21534789xdff87625975db226@mail.gmail.com> [Alex Martelli] >> We stole list comprehensions and genexps from Haskell [Greg Ewing] > The idea predates Haskell, I think. I first saw it in > Miranda, and it may have come from something even > earlier -- SETL, maybe? Haskell indeed took list comprehensions from SETL. SETL in turn took them from much earlier standard notation in set theory, related to the oddly named (but not universally so named) "axiom of comprehension". genexps were more directly taken from Icon, but tying them into Python's iteration protocol is a powerful twist not directly present in Icon. From tim.peters at gmail.com Wed Mar 1 08:13:58 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 1 Mar 2006 01:13:58 -0600 Subject: [Python-Dev] Using and binding relative names (was Re: PEP forBetter Control of Nested Lexical Scopes) In-Reply-To: <4402C8F8.1040604@canterbury.ac.nz> References: <20060220204850.6004.JCARLSON@uci.edu> <7e9b97090602252315mf6d4686ud86dd5163ea76b37@mail.gmail.com> <440205C6.3030301@ronadam.com> <440245B7.5090903@ronadam.com> <1956B0C6-4FE7-41EC-BCBA-37C25CBE470D@gmail.com> <44025955.7090709@ronadam.com> <4402C8F8.1040604@canterbury.ac.nz> Message-ID: <1f7befae0602282313g29f897cu6cbfb27b216149bc@mail.gmail.com> [Alex Martelli] >> We stole list comprehensions and genexps from Haskell [Greg Ewing] > The idea predates Haskell, I think. I first saw it in > Miranda, and it may have come from something even > earlier -- SETL, maybe? Haskell indeed took list comprehensions from SETL. SETL in turn adopted them from pre-computer standard notation in set theory, related to the oddly named (but not universally so named) "axiom of comprehension". genexps were more directly taken from Icon (because of the "generator" part). From tjreedy at udel.edu Wed Mar 1 08:29:44 2006 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 1 Mar 2006 02:29:44 -0500 Subject: [Python-Dev] defaultdict and on_missing() References: <20060222131328.nzferzk2xxk448cg@login.werra.lunarpages.com><43FE2F5C.8090700@livinglogic.de><327376ff0602231839h4c27c999la52039be37c28230@mail.gmail.com><43FEAD58.2020208@canterbury.ac.nz><015901c6395d$3b9fd4b0$2e2a960a@RaymondLaptop1><43FFA4B2.7060303@canterbury.ac.nz> <4402F6A9.1030906@gmail.com><4403E263.1060505@canterbury.ac.nz> <440411E1.6040401@gmail.com><4404D747.2050402@canterbury.ac.nz><012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> <44052636.9090709@canterbury.ac.nz> Message-ID: "Greg Ewing" wrote in message news:44052636.9090709 at canterbury.ac.nz... > And you don't think there are many different > types of iterables? You might as well argue > that we don't need len() because it "only > applies to sequences". Since you mention it..., many people *have* asked on c.l.p why len() is a builtin function rather than a method of sequences (and other collections) (as .len, not .__len__). Some have suggested that it should be the latter. The answers justifying the status quo have been twofold. 1. Before 2.2, not all builtin sequence types had methods (str and tuple), so they could not have a .len method. (This begs the question of why not, but that is moot now.) - whereas .next came in with the universalization of methods. 2. Before the addition of list comprehensions, a function could be mapped much more easily than a method - whereas now we do have list comps and even this works >>> [i.__add__(2) for i in range(3)] [2, 3, 4] - I can imagine wanting to map len to, for instance, a list of strings more easily than I can imagine a reason to map next/.next to a list of iterators, and if I did, I am willing to use the list comp form. Terry Jan Reedy From tjreedy at udel.edu Wed Mar 1 08:41:26 2006 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 1 Mar 2006 02:41:26 -0500 Subject: [Python-Dev] Webstats for www.python.org et al. References: <9e804ac0602282251n9d958a2kbd92a27ea3133a60@mail.gmail.com> Message-ID: "Thomas Wouters" wrote in message news:9e804ac0602282251n9d958a2kbd92a27ea3133a60 at mail.gmail.com... > I added webstats for all subsites of python.org: > > http://www.python.org/webstats/ > http://beta.python.org/webstats/ > http://bugs.python.org/webstats/ > http://planet.python.org/webstats/ > http://docs.python.org/webstats/ > http://doc.python.org/webstats/ > http://svn.python.org/webstats/ Since the first and last months are incomplete, I would consider plotting daily averages for each month instead of totals. tjr From ulrich.berning at denviso.de Wed Mar 1 08:49:06 2006 From: ulrich.berning at denviso.de (Ulrich Berning) Date: Wed, 01 Mar 2006 08:49:06 +0100 Subject: [Python-Dev] C++ for CPython 3? (Re: str.count is slow) In-Reply-To: References: <1141073696.107136.318600@j33g2000cwa.googlegroups.com><1141083127.970403.147100@v46g2000cwv.googlegroups.com> <4403EFEE.80002@canterbury.ac.nz> <4404D79D.2070608@canterbury.ac.nz> Message-ID: <440551F2.4030702@denviso.de> Fredrik Lundh schrieb: >should we perhaps switch to (careful use of) C++ in 3.0 ? > > > I can't see many advantages in moving to C++, but a lot of disadvantages: - Size increase, especially when we start using templates - Performance decrease - Problems with name mangling together with dynamic loading and cross module API's - Everything has to be build with the same compiler, binaries created with different compilers can't interoperate - Possibly all extensions modules have to be (re)written in C++ - Moving to C++ will change Python's well known API substantially --- IMHO, if a Python major version change implies to forget everything that has been established over the years, ignoring backward compaibility, breaking nearly every Python script, than we should definitely find another name for it, or I will stay with Python 2 for the rest of my life. --- Ulli From fredrik at pythonware.com Wed Mar 1 08:54:28 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 1 Mar 2006 08:54:28 +0100 Subject: [Python-Dev] Webstats for www.python.org et al. References: <9e804ac0602282251n9d958a2kbd92a27ea3133a60@mail.gmail.com> Message-ID: Thomas Wouters wrote: > I added webstats for all subsites of python.org: > > http://www.python.org/webstats/ what's that "Java/1.4.2_03" user agent doing? (it's responsible for 10% of all hits in january/february, and 20% of the hits today...) From phd at mail2.phd.pp.ru Wed Mar 1 10:20:16 2006 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Wed, 1 Mar 2006 12:20:16 +0300 Subject: [Python-Dev] .py and .txt files missing svn:eol-style in trunk In-Reply-To: <1f7befae0602282217j1700cfb1j4b73ddf80ef1428a@mail.gmail.com> References: <1f7befae0602282217j1700cfb1j4b73ddf80ef1428a@mail.gmail.com> Message-ID: <20060301092016.GB12002@phd.pp.ru> On Wed, Mar 01, 2006 at 12:17:16AM -0600, Tim Peters wrote: > These .py and .txt files don't have the svn:eol-style property set. > I'm not sure they all _should_, though. My experience shows that if the developers use different OSes (our team uses Linux and Windows) it helps very much to set svn:eol-style to native for all text files - *.py, *.txt, etc, except for files with special requirements. So I use the following shell script #! /bin/sh svn add "$@" svn propset svn:eol-style native "$@" svn propset svn:keywords "Date Rev Id" "$@" to add *.py files to the repository. (I don't want to put it in a global configuration because I have different requirements for different projects.) For other text files the same except keywords: #! /bin/sh svn add "$@" svn propset svn:eol-style native "$@" > test_pepe263 probably should be binary (it contains "funny > characters"). It should be text with encoding $ svn propset svn:mime-type "text/plain; charset=koi8-r" test_pep263 Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From tds333+pydev at gmail.com Wed Mar 1 10:58:19 2006 From: tds333+pydev at gmail.com (Wolfgang Langner) Date: Wed, 1 Mar 2006 10:58:19 +0100 Subject: [Python-Dev] C++ for CPython 3? (Re: str.count is slow) In-Reply-To: <440551F2.4030702@denviso.de> References: <1141073696.107136.318600@j33g2000cwa.googlegroups.com> <1141083127.970403.147100@v46g2000cwv.googlegroups.com> <4403EFEE.80002@canterbury.ac.nz> <4404D79D.2070608@canterbury.ac.nz> <440551F2.4030702@denviso.de> Message-ID: <4c45c1530603010158q1d35aa11he41deed56eb6385e@mail.gmail.com> Hello, > >should we perhaps switch to (careful use of) C++ in 3.0 ? > I can't see many advantages in moving to C++, but a lot of disadvantages: > > - Size increase, especially when we start using templates > - Performance decrease > - Problems with name mangling together with dynamic loading and cross > module API's > - Everything has to be build with the same compiler, binaries created > with different compilers can't interoperate > - Possibly all extensions modules have to be (re)written in C++ > - Moving to C++ will change Python's well known API substantially Same opinion. C++ is evil, please don't use it. You get a lot of new problems and nearly not benefit. Better go to jython or consider the way of pypy. -- bye by Wolfgang From abo at minkirri.apana.org.au Wed Mar 1 12:37:43 2006 From: abo at minkirri.apana.org.au (Donovan Baarda) Date: Wed, 01 Mar 2006 11:37:43 +0000 Subject: [Python-Dev] bytes.from_hex() In-Reply-To: <06Feb28.152344pst."58633"@synergy1.parc.xerox.com> References: <06Feb28.152344pst."58633"@synergy1.parc.xerox.com> Message-ID: <1141213063.3666.14.camel@warna.dub.corp.google.com> On Tue, 2006-02-28 at 15:23 -0800, Bill Janssen wrote: > Greg Ewing wrote: > > Bill Janssen wrote: > > > > > bytes -> base64 -> text > > > text -> de-base64 -> bytes > > > > It's nice to hear I'm not out of step with > > the entire world on this. :-) > > Well, I can certainly understand the bytes->base64->bytes side of > thing too. The "text" produced is specified as using "a 65-character > subset of US-ASCII", so that's really bytes. Huh... just joining here but surely you don't mean a text string that doesn't use every character available in a particular encoding is "really bytes"... it's still a text string... If you base64 encode some bytes, you get a string. If you then want to access that base64 string as if it was a bunch of bytes, cast it to bytes. Be careful not to confuse "(type)cast" with "(type)convert"... A "convert" transforms the data from one type/class to another, modifying it to be a valid equivalent instance of the other type/class; ie int -> float. A "cast" does not modify the data in any way, it just changes its type/class to be the other type, and assumes that the data is a valid instance of the other type; eg int32 -> bytes[4]. Minor data munging under the hood to cleanly switch the type/class is acceptable (ie adding array length info etc) provided you keep to the spirit of the "cast". Keep these two concepts separate and you should be right :-) -- Donovan Baarda http://minkirri.apana.org.au/~abo/ From barry at python.org Wed Mar 1 14:10:10 2006 From: barry at python.org (Barry Warsaw) Date: Wed, 1 Mar 2006 08:10:10 -0500 Subject: [Python-Dev] C++ for CPython 3? (Re: str.count is slow) In-Reply-To: References: <1141073696.107136.318600@j33g2000cwa.googlegroups.com><1141083127.970403.147100@v46g2000cwv.googlegroups.com> <4403EFEE.80002@canterbury.ac.nz> <4404D79D.2070608@canterbury.ac.nz> Message-ID: <2BBD58CD-DD0C-441D-A592-DCB5A0C2CD76@python.org> On Feb 28, 2006, at 6:26 PM, Fredrik Lundh wrote: > > should we perhaps switch to (careful use of) C++ in 3.0 ? I hope not. It would make life more difficult for embedded/extended users like ourselves because it would force us to link our applications as C++ programs. That introduces lots of headaches on certain, shall we say, non-Windows platforms. -Barry From ncoghlan at gmail.com Wed Mar 1 14:10:22 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 01 Mar 2006 23:10:22 +1000 Subject: [Python-Dev] bytes.from_hex() In-Reply-To: <06Feb28.152344pst."58633"@synergy1.parc.xerox.com> References: <06Feb28.152344pst."58633"@synergy1.parc.xerox.com> Message-ID: <44059D3E.2010908@gmail.com> Bill Janssen wrote: > Greg Ewing wrote: >> Bill Janssen wrote: >> >>> bytes -> base64 -> text >>> text -> de-base64 -> bytes >> It's nice to hear I'm not out of step with >> the entire world on this. :-) > > Well, I can certainly understand the bytes->base64->bytes side of > thing too. The "text" produced is specified as using "a 65-character > subset of US-ASCII", so that's really bytes. If the base64 codec was a text<->bytes codec, and bytes did not have an encode method, then if you want to convert your original bytes to ascii bytes, you would do: ascii_bytes = orig_bytes.decode("base64").encode("ascii") "Use base64 to convert my byte sequence to characters, then give me the corresponding ascii byte sequence" To reverse the process: orig_bytes = ascii_bytes.decode("ascii").encode("base64") "Use ascii to convert my byte sequence to characters, then use base64 to convert those characters back to the original byte sequence" The only slightly odd aspect is that this inverts the conventional meaning of base64 encoding and decoding, where you expect to encode from bytes to characters and decode from characters to bytes. As strings currently have both methods, the existing codec is able to use the conventional sense for base64: encode goes from "str-as-bytes" to "str-as-text" (giving a longer string with characters that fit in the base64 subset) and decode goes from "str-as-text" to "str-as-bytes" (giving back the original string) All the unicode codecs, on the other hand, use encode to get from characters to bytes and decode to get from bytes to characters. So if bytes objects *did* have an encode method, it should still result in a unicode object, just the same as a decode method does (because you are encoding bytes as characters), and unicode objects would acquire a corresponding decode method (that decodes from a character format such as base64 to the original byte sequence). In the name of TOOWTDI, I'd suggest that we just eat the slight terminology glitch in the rare cases like base64, hex and oct (where the character format is technically the encoded format), and leave it so that there is a single method pair (bytes.decode to go from bytes to characters, and text.encode to go from characters to bytes). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From gmccaughan at synaptics-uk.com Wed Mar 1 14:32:39 2006 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Wed, 1 Mar 2006 13:32:39 +0000 Subject: [Python-Dev] Proposal: defaultdict In-Reply-To: <43F5EBB7.7080907@gmail.com> References: <43F5EBB7.7080907@gmail.com> Message-ID: <200603011332.40419.gmccaughan@synaptics-uk.com> > >> d.get(key, [], True).append(value) > > > > hmm. are you sure you didn't just reinvent setdefault ? > > I'm reasonably sure I copied it on purpose, only with a name that isn't 100% > misleading as to what it does ;) Heh. From the original Usenet posting that suggested the capability that was added in the form of "setdefault": | I suggest a minor change: another optional argument to | "get" so that | | dict.get(item,default,flag) | | is equivalent to | | if dict.has_key(item): | VALUE IS dict[item] | else: | if flag: dict[item] = default <-- This is all that's new | VALUE IS default | | but presumably more efficient. The response was a chorus of people saying "Not a bad idea, but that flag sucks. It needs a separate method." :-) -- g From walter at livinglogic.de Wed Mar 1 15:48:47 2006 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Wed, 01 Mar 2006 15:48:47 +0100 Subject: [Python-Dev] Stateful codecs [Was: str object going in Py3K] In-Reply-To: <43FB6E76.5050407@egenix.com> References: <43F5DFE0.6090806@livinglogic.de> <43F5E5E9.2040809@egenix.com> <43F5F24D.2000802@livinglogic.de> <43F5F5F4.2000906@egenix.com> <43F5FD88.8090605@livinglogic.de> <43F63C07.9030901@egenix.com> <61425.89.54.8.114.1140279099.squirrel@isar.livinglogic.de> <43F7585E.4080909@egenix.com> <61847.89.54.8.114.1140296899.squirrel@isar.livinglogic.de> <4f0b69dc0602190317g2b98df5cw2fcf42f948540b01@mail.gmail.com> <43FB6BBF.7030101@livinglogic.de> <43FB6E76.5050407@egenix.com> Message-ID: <4405B44F.40509@livinglogic.de> M.-A. Lemburg wrote: > Walter D?rwald wrote: >> Hye-Shik Chang wrote: >> >>> On 2/19/06, Walter D?rwald wrote: >>>> M.-A. Lemburg wrote: >>>>> Walter D?rwald wrote: >>>>>> Anyway, I've started implementing a patch that just adds >>>>>> codecs.StatefulEncoder/codecs.StatefulDecoder. UTF8, UTF8-Sig, >>>>>> UTF-16, UTF-16-LE and UTF-16-BE are already working. >>>>> Nice :-) >>>> gencodec.py is updated now too. The rest should be manageble too. >>>> I'll leave updating the CJKV codecs to Hye-Shik though. >>> Okay. I'll look whether how CJK codecs can be improved by the >>> new protocol soon. I guess it'll be not so difficult because CJK >>> codecs have a their own common stateful framework already. >> OK, here's the patch: bugs.python.org/1436130 (assigned to MAL). > > Thanks. I won't be able to look into it this week though, probably > next week. Any progress on this? I'd really like to get this into 2.5 and the feature freeze is approaching fast! Bye, Walter D?rwald From rrr at ronadam.com Wed Mar 1 15:49:09 2006 From: rrr at ronadam.com (Ron Adam) Date: Wed, 01 Mar 2006 08:49:09 -0600 Subject: [Python-Dev] bytes.from_hex() In-Reply-To: <44059D3E.2010908@gmail.com> References: <06Feb28.152344pst."58633"@synergy1.parc.xerox.com> <44059D3E.2010908@gmail.com> Message-ID: <4405B465.9060802@ronadam.com> Nick Coghlan wrote: > All the unicode codecs, on the other hand, use encode to get from characters > to bytes and decode to get from bytes to characters. > > So if bytes objects *did* have an encode method, it should still result in a > unicode object, just the same as a decode method does (because you are > encoding bytes as characters), and unicode objects would acquire a > corresponding decode method (that decodes from a character format such as > base64 to the original byte sequence). > > In the name of TOOWTDI, I'd suggest that we just eat the slight terminology > glitch in the rare cases like base64, hex and oct (where the character format > is technically the encoded format), and leave it so that there is a single > method pair (bytes.decode to go from bytes to characters, and text.encode to > go from characters to bytes). I think you have it pretty straight here. While playing around with the example bytes class I noticed code reads much better when I use methods called tounicode and tostring. b64ustring = b.tounicode('base64') b = bytes(b64ustring, 'base64') The bytes could then *not* ignore the string decode codec but use it for string to string decoding. b64string = b.tostring('base64') b = bytes(b64string, 'base64') b = bytes(hexstring, 'hex') hexstring = b.tostring('hex') hexstring = b.tounicode('hex') An exception could be raised if the codec does not support input or output type depending on the situation. This would allow for differnt types of codecs to live together without as much confusion I think. I'm not suggesting we start using to-type everywhere, just where it might make things clearer over decode and encode. Expecting it not to fly, but just maybe it could? Ron From nidoizo at yahoo.com Wed Mar 1 15:57:12 2006 From: nidoizo at yahoo.com (Nicolas Fleury) Date: Wed, 01 Mar 2006 09:57:12 -0500 Subject: [Python-Dev] Making staticmethod objects callable? Message-ID: Hi, I've posted this question on comp.lang.python, but nobody seems to conclude it is a bad idea, so I post it here. http://groups.google.com/group/comp.lang.python/browse_frm/thread/6082dae1deef9161/88bb8a26750dd8c6?lnk=raot&hl=en#88bb8a26750dd8c6 Basically, should staticmethods be made callable so that the following would not raise an exception: class A: @staticmethod def foo(): pass bar = foo() There's workarounds, but it's really just about usability. staticmethod could still return a descriptor, but additionnally callable. Is there something I'm missing? Is it error-prone in any way? Note that I'm not a big fan of static methods myself, but I understand users expecting this work. Regards, Nicolas From anthony at interlink.com.au Wed Mar 1 16:00:09 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu, 2 Mar 2006 02:00:09 +1100 Subject: [Python-Dev] Stateful codecs [Was: str object going in Py3K] In-Reply-To: <4405B44F.40509@livinglogic.de> References: <43FB6E76.5050407@egenix.com> <4405B44F.40509@livinglogic.de> Message-ID: <200603020200.12263.anthony@interlink.com.au> On Thursday 02 March 2006 01:48, Walter D?rwald wrote: > Any progress on this? I'd really like to get this into 2.5 and the > feature freeze is approaching fast! Remember, the feature freeze is as of beta1. Hopefully the major new features are all going to be in before alpha1, but they can continue to be added up until the first beta. Then (barring acts of Guido) we should be in feature-freeze mode. From walter at livinglogic.de Wed Mar 1 16:08:47 2006 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Wed, 01 Mar 2006 16:08:47 +0100 Subject: [Python-Dev] Stateful codecs [Was: str object going in Py3K] In-Reply-To: <200603020200.12263.anthony@interlink.com.au> References: <43FB6E76.5050407@egenix.com> <4405B44F.40509@livinglogic.de> <200603020200.12263.anthony@interlink.com.au> Message-ID: <4405B8FF.3070003@livinglogic.de> Anthony Baxter wrote: > On Thursday 02 March 2006 01:48, Walter D?rwald wrote: >> Any progress on this? I'd really like to get this into 2.5 and the >> feature freeze is approaching fast! > > Remember, the feature freeze is as of beta1. Hopefully the major new > features are all going to be in before alpha1, but they can continue > to be added up until the first beta. Then (barring acts of Guido) we > should be in feature-freeze mode. OK, it seems I misunderstood PEP 356: """ Planned features for 2.5 Target for inclusion of each feature by March 16. """ Bye, Walter D?rwald From mchermside at ingdirect.com Wed Mar 1 15:59:35 2006 From: mchermside at ingdirect.com (Chermside, Michael) Date: Wed, 1 Mar 2006 09:59:35 -0500 Subject: [Python-Dev] bytes.from_hex() Message-ID: Ron Adam writes: > While playing around with the example bytes class I noticed code reads > much better when I use methods called tounicode and tostring. [...] > I'm not suggesting we start using to-type everywhere, just where it > might make things clearer over decode and encode. +1 I always find myself slightly confused by encode() and decode() despite the fact that I understand (I think) the reason for the choice of those names and by rights ought to have no trouble. I'm not arguing that it's worth the gratuitous code breakage (I don't have enough code using encode() and decode() for my opinion to count in that matter.) But I will say that if there were no legacy I'd prefer the tounicode() and tostring() (but shouldn't it be 'tobytes()' instead?) names for Python 3.0. -- Michael Chermside ***************************************************************************** This email may contain confidential or privileged information. If you believe you have received the message in error, please notify the sender and delete the message without copying or disclosing it. ***************************************************************************** From mal at egenix.com Wed Mar 1 16:15:58 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 01 Mar 2006 16:15:58 +0100 Subject: [Python-Dev] Stateful codecs [Was: str object going in Py3K] In-Reply-To: <4405B44F.40509@livinglogic.de> References: <43F5DFE0.6090806@livinglogic.de> <43F5E5E9.2040809@egenix.com> <43F5F24D.2000802@livinglogic.de> <43F5F5F4.2000906@egenix.com> <43F5FD88.8090605@livinglogic.de> <43F63C07.9030901@egenix.com> <61425.89.54.8.114.1140279099.squirrel@isar.livinglogic.de> <43F7585E.4080909@egenix.com> <61847.89.54.8.114.1140296899.squirrel@isar.livinglogic.de> <4f0b69dc0602190317g2b98df5cw2fcf42f948540b01@mail.gmail.com> <43FB6BBF.7030101@livinglogic.de> <43FB6E76.5050407@egenix.com> <4405B44F.40509@livinglogic.de> Message-ID: <4405BAAE.7070800@egenix.com> Walter D?rwald wrote: > M.-A. Lemburg wrote: >> Walter D?rwald wrote: >>> Hye-Shik Chang wrote: >>> >>>> On 2/19/06, Walter D?rwald wrote: >>>>> M.-A. Lemburg wrote: >>>>>> Walter D?rwald wrote: >>>>>>> Anyway, I've started implementing a patch that just adds >>>>>>> codecs.StatefulEncoder/codecs.StatefulDecoder. UTF8, UTF8-Sig, >>>>>>> UTF-16, UTF-16-LE and UTF-16-BE are already working. >>>>>> Nice :-) >>>>> gencodec.py is updated now too. The rest should be manageble too. >>>>> I'll leave updating the CJKV codecs to Hye-Shik though. >>>> Okay. I'll look whether how CJK codecs can be improved by the >>>> new protocol soon. I guess it'll be not so difficult because CJK >>>> codecs have a their own common stateful framework already. >>> OK, here's the patch: bugs.python.org/1436130 (assigned to MAL). >> >> Thanks. I won't be able to look into it this week though, probably >> next week. > > Any progress on this? I'd really like to get this into 2.5 and the > feature freeze is approaching fast! I'll have a look this week. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 01 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From steven.bethard at gmail.com Wed Mar 1 16:50:21 2006 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed, 1 Mar 2006 08:50:21 -0700 Subject: [Python-Dev] Making staticmethod objects callable? In-Reply-To: References: Message-ID: On 3/1/06, Nicolas Fleury wrote: > Basically, should staticmethods be made callable so that the following > would not raise an exception: > > class A: > @staticmethod > def foo(): pass > bar = foo() > > There's workarounds, but it's really just about usability. staticmethod > could still return a descriptor, but additionnally callable. Is there > something I'm missing? Is it error-prone in any way? My only (mild) concern is that if staticmethod is going to get a __call__, I think classmethod should probably get one too. Inside a class this doesn't make much sense: class A(object): @classmethod def foo(cls): pass bar = foo(None) # ?? But I guess outside of a class maybe it's okay: @classmethod def foo(cls): pass class A(object): pass foo(A) Anyway, my feeling was that running into this behavior (that staticmethod is not callable) is a good oportunity to explain how descriptors work. And once you start playing around with staticmethod and classmethod, you're going to need to learn that pretty soon anyway. Hiding it a little bit longer with a __call__ method on staticmethod isn't going to help much in the long run. So I guess I'm -0 if classmethod gets a __call__ too. STeVe -- Grammar am for people who can't think for myself. --- Bucky Katt, Get Fuzzy From martin at v.loewis.de Wed Mar 1 17:10:18 2006 From: martin at v.loewis.de (martin at v.loewis.de) Date: Wed, 01 Mar 2006 17:10:18 +0100 Subject: [Python-Dev] C++ for CPython 3? (Re: str.count is slow) In-Reply-To: <440551F2.4030702@denviso.de> References: <1141073696.107136.318600@j33g2000cwa.googlegroups.com><1141083127.970403.147100@v46g2000cwv.googlegroups.com> <4403EFEE.80002@canterbury.ac.nz> <4404D79D.2070608@canterbury.ac.nz> <440551F2.4030702@denviso.de> Message-ID: <1141229418.4405c76a38edd@www.domainfactory-webmail.de> Zitat von Ulrich Berning : > I can't see many advantages in moving to C++, but a lot of disadvantages: There are a few advantages, though, mainly: - increased type-safety, in particular for API that isn't type-checked at all at the moment (e.g. PyArg_ParseTuple) - more reliable reference counting, due to destructors of local variables - "native" exception handling, making exceptions both less error-prone and possible more efficient. Regards, Martin From walter at livinglogic.de Wed Mar 1 17:30:37 2006 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Wed, 01 Mar 2006 17:30:37 +0100 Subject: [Python-Dev] C++ for CPython 3? (Re: str.count is slow) In-Reply-To: <1141229418.4405c76a38edd@www.domainfactory-webmail.de> References: <1141073696.107136.318600@j33g2000cwa.googlegroups.com><1141083127.970403.147100@v46g2000cwv.googlegroups.com> <4403EFEE.80002@canterbury.ac.nz> <4404D79D.2070608@canterbury.ac.nz> <440551F2.4030702@denviso.de> <1141229418.4405c76a38edd@www.domainfactory-webmail.de> Message-ID: <4405CC2D.5040903@livinglogic.de> martin at v.loewis.de wrote: > Zitat von Ulrich Berning : > >> I can't see many advantages in moving to C++, but a lot of disadvantages: > > There are a few advantages, though, mainly: > - increased type-safety, in particular for API that isn't type-checked > at all at the moment (e.g. PyArg_ParseTuple) > - more reliable reference counting, due to destructors of local > variables Indeed, smart pointers should get rid of most reference counting problems. > - "native" exception handling, making exceptions both less error-prone > and possible more efficient. Another advantage might be using inline functions instead of macros with do {foo} while(0). Bye, Walter D?rwald From janssen at parc.com Wed Mar 1 17:29:46 2006 From: janssen at parc.com (Bill Janssen) Date: Wed, 1 Mar 2006 08:29:46 PST Subject: [Python-Dev] bytes.from_hex() In-Reply-To: Your message of "Wed, 01 Mar 2006 03:37:43 PST." <1141213063.3666.14.camel@warna.dub.corp.google.com> Message-ID: <06Mar1.082951pst."58633"@synergy1.parc.xerox.com> > Huh... just joining here but surely you don't mean a text string that > doesn't use every character available in a particular encoding is > "really bytes"... it's still a text string... No, once it's in a particular encoding it's bytes, no longer text. As you say, > Keep these two concepts separate and you should be right :-) Bill From Scott.Daniels at Acm.Org Wed Mar 1 17:29:35 2006 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Wed, 01 Mar 2006 08:29:35 -0800 Subject: [Python-Dev] bytes.from_hex() In-Reply-To: References: Message-ID: Chermside, Michael wrote: > ... I will say that if there were no legacy I'd prefer the tounicode() > and tostring() (but shouldn't itbe 'tobytes()' instead?) names for Python 3.0. Wouldn't 'tobytes' and 'totext' be better for 3.0 where text == unicode? -- -- Scott David Daniels Scott.Daniels at Acm.Org From mcherm at mcherm.com Wed Mar 1 18:38:04 2006 From: mcherm at mcherm.com (Michael Chermside) Date: Wed, 01 Mar 2006 09:38:04 -0800 Subject: [Python-Dev] bytes.from_hex() Message-ID: <20060301093804.3vmmkrcpiw008sgw@login.werra.lunarpages.com> I wrote: > ... I will say that if there were no legacy I'd prefer the tounicode() > and tostring() (but shouldn't itbe 'tobytes()' instead?) names for Python 3.0. Scott Daniels replied: > Wouldn't 'tobytes' and 'totext' be better for 3.0 where text == unicode? Um... yes. Sorry, I'm not completely used to 3.0 yet. I'll need to borrow the time machine for a little longer before my fingers really pick up on the 3.0 names and idioms. -- Michael Chermside From talin at acm.org Wed Mar 1 20:00:22 2006 From: talin at acm.org (Talin) Date: Wed, 01 Mar 2006 11:00:22 -0800 Subject: [Python-Dev] Slightly OT: Replying to posts Message-ID: <4405EF46.7020100@acm.org> Just a quick question about the mechanics of replying to this list. I am a subscriber to the list, however I much prefer readiing the list archives on the web instead of having the postings delivered to my email account. Because of this, I have delivery turned off in the mailing list preferences. I particularly dislike the idea of wasting bandwidth and disk space on something that I am not going to read. However, I would like to be able to reply to posts in such a way as to have them appear in the appropriate place in the thread hierarchy. Since I don't have the original email, I cannot reply to it directly; instead I have to create a new, non-reply email and send it to the list. Simply editing the subject line to put "Re: subject" would seem to be insufficient. Does anyone have a trick for managing this? Or is there a FAQ that someone can point me to that addresses this issue? -- Talin From brett at python.org Wed Mar 1 20:02:19 2006 From: brett at python.org (Brett Cannon) Date: Wed, 1 Mar 2006 11:02:19 -0800 Subject: [Python-Dev] defaultdict and on_missing() In-Reply-To: References: <20060222131328.nzferzk2xxk448cg@login.werra.lunarpages.com> <43FFA4B2.7060303@canterbury.ac.nz> <4402F6A9.1030906@gmail.com> <4403E263.1060505@canterbury.ac.nz> <440411E1.6040401@gmail.com> <4404D747.2050402@canterbury.ac.nz> <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> <44052636.9090709@canterbury.ac.nz> Message-ID: On 2/28/06, Terry Reedy wrote: > > "Greg Ewing" wrote in message > news:44052636.9090709 at canterbury.ac.nz... > > And you don't think there are many different > > types of iterables? You might as well argue > > that we don't need len() because it "only > > applies to sequences". > > Since you mention it..., many people *have* asked on c.l.p why len() is a > builtin function rather than a method of sequences (and other collections) > (as .len, not .__len__). Some have suggested that it should be the latter. > The answers justifying the status quo have been twofold. > > 1. Before 2.2, not all builtin sequence types had methods (str and tuple), > so they could not have a .len method. (This begs the question of why not, > but that is moot now.) Well, up until 2.2 you didn't have new-style classes which have a common base class. And if you wanted to do the length compilation only when requested, you needed a method. But now with object, we could add extra smarts to __getattr__ or __getattribute__ so that if ``spam.len`` is requested it calls ``spam.__next__()`` for you, basically a poor-man's property. Or, if ``spam.len`` is defined, return that. But moving over to more attributes for how we access basic interfaces seems great to me. -Brett From brett at python.org Wed Mar 1 20:04:14 2006 From: brett at python.org (Brett Cannon) Date: Wed, 1 Mar 2006 11:04:14 -0800 Subject: [Python-Dev] Webstats for www.python.org et al. In-Reply-To: References: <9e804ac0602282251n9d958a2kbd92a27ea3133a60@mail.gmail.com> Message-ID: On 2/28/06, Fredrik Lundh wrote: > Thomas Wouters wrote: > > > I added webstats for all subsites of python.org: > > > > http://www.python.org/webstats/ > > what's that "Java/1.4.2_03" user agent doing? (it's responsible for > 10% of all hits in january/february, and 20% of the hits today...) > Most likely a crawler. -Brett From barry at python.org Wed Mar 1 20:10:04 2006 From: barry at python.org (Barry Warsaw) Date: Wed, 01 Mar 2006 14:10:04 -0500 Subject: [Python-Dev] Slightly OT: Replying to posts In-Reply-To: <4405EF46.7020100@acm.org> References: <4405EF46.7020100@acm.org> Message-ID: <1141240204.11040.0.camel@geddy.wooz.org> On Wed, 2006-03-01 at 11:00 -0800, Talin wrote: > However, I would like to be able to reply to posts in such a way as to > have them appear in the appropriate place in the thread hierarchy. Since > I don't have the original email, I cannot reply to it directly; instead > I have to create a new, non-reply email and send it to the list. Simply > editing the subject line to put "Re: subject" would seem to be insufficient. > > Does anyone have a trick for managing this? Or is there a FAQ that > someone can point me to that addresses this issue? Try a newsreader and reading/replying via news.gmane.org -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060301/42e6dad5/attachment.pgp From tjreedy at udel.edu Wed Mar 1 20:17:14 2006 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 1 Mar 2006 14:17:14 -0500 Subject: [Python-Dev] Slightly OT: Replying to posts References: <4405EF46.7020100@acm.org> Message-ID: "Talin" wrote in message news:4405EF46.7020100 at acm.org... > Just a quick question about the mechanics of replying to this list. > > I am a subscriber to the list, however I much prefer readiing the list > archives on the web instead of having the postings delivered to my email > account. Because of this, I have delivery turned off in the mailing list > preferences. I particularly dislike the idea of wasting bandwidth and > disk space on something that I am not going to read. > > However, I would like to be able to reply to posts in such a way as to > have them appear in the appropriate place in the thread hierarchy. Since > I don't have the original email, I cannot reply to it directly; instead > I have to create a new, non-reply email and send it to the list. Simply > editing the subject line to put "Re: subject" would seem to be > insufficient. > > Does anyone have a trick for managing this? Or is there a FAQ that > someone can point me to that addresses this issue? Another alternative to your mailbox is to read the list as newsgroup gmane.comp.python.devel at gmane.news.org (free). The gateway is bidirectional so properly threaded reply is easy, as this response indicates. tjr From nidoizo at yahoo.com Wed Mar 1 20:28:40 2006 From: nidoizo at yahoo.com (Nicolas Fleury) Date: Wed, 01 Mar 2006 14:28:40 -0500 Subject: [Python-Dev] Making staticmethod objects callable? In-Reply-To: References: Message-ID: Steven Bethard wrote: > My only (mild) concern is that if staticmethod is going to get a > __call__, I think classmethod should probably get one too. Inside a > class this doesn't make much sense: I agree, make sense or not, if "@staticmethod def foo()" and a simple "def foo(self)" can all be called inside class definition, "@classmethod def foo(cls)" should too. > Anyway, my feeling was that running into this behavior (that > staticmethod is not callable) is a good oportunity to explain how > descriptors work. And once you start playing around with staticmethod > and classmethod, you're going to need to learn that pretty soon > anyway. Hiding it a little bit longer with a __call__ method on > staticmethod isn't going to help much in the long run. The problem is that even after explaining descriptors (which IMHO can be a more advanced feature for newbies), you still need a workaround and you might end up with (or call __get__): class A: def foo(): pass bar = foo() foo = staticmethod(foo) Which I find sad considering all the energy debating @decorators;) Regards, Nicolas From jack at performancedrivers.com Wed Mar 1 20:52:40 2006 From: jack at performancedrivers.com (Jack Diederich) Date: Wed, 1 Mar 2006 14:52:40 -0500 Subject: [Python-Dev] Webstats for www.python.org et al. In-Reply-To: References: <9e804ac0602282251n9d958a2kbd92a27ea3133a60@mail.gmail.com> Message-ID: <20060301195240.GA14953@performancedrivers.com> On Wed, Mar 01, 2006 Brett Cannon wrote: > On 2/28/06, Fredrik Lundh wrote: > > Thomas Wouters wrote: > > > > > I added webstats for all subsites of python.org: > > > > > > http://www.python.org/webstats/ > > > > what's that "Java/1.4.2_03" user agent doing? (it's responsible for > > 10% of all hits in january/february, and 20% of the hits today...) > > Most likely a crawler. > Youch, if I'm reading it right it consumed fully half of the bandwidth for today on python.org. And what 1.6 million pages did it spider on the site last month? Something smells broken. -Jack From jason.orendorff at gmail.com Wed Mar 1 22:12:24 2006 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Wed, 1 Mar 2006 16:12:24 -0500 Subject: [Python-Dev] bytes thoughts Message-ID: 1. Maybe there should be a more obvious way to spell "bytes([0])*N". I went through "bytes([0]*N)" and "bytes('\0'*N)" before I realized there was a memory-efficient way to do it. 1a. Likewise, slice-assignment nicely handles memmove(), but there's no memset(). 2. Having a plural noun as a type name is awkward. I wish we could call it "buffer" (which, conveniently, also tells you that it's mutable, even if you don't know the word "mutable" :-). Alas. 3. I wrote a toy BytesIO class to go with the toy bytes object: http://wiki.python.org/moin/BytesIO (I hope this isn't considered wiki abuse -- it seemed as worthy and relevant as most of what's in there.) -j From aahz at pythoncraft.com Wed Mar 1 23:41:16 2006 From: aahz at pythoncraft.com (Aahz) Date: Wed, 1 Mar 2006 14:41:16 -0800 Subject: [Python-Dev] wiki as scratchpad In-Reply-To: References: Message-ID: <20060301224116.GA17350@panix.com> On Wed, Mar 01, 2006, Jason Orendorff wrote: > > 3. I wrote a toy BytesIO class to go with the toy bytes object: > http://wiki.python.org/moin/BytesIO > (I hope this isn't considered wiki abuse -- it seemed as worthy and > relevant as most of what's in there.) Au contraire! This is precisely why a wiki is useful, to serve as a community notepad. Perhaps this specific case isn't a "good use", but it absolutely does not count as abuse, and I encourage other people to make similar use. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From aahz at pythoncraft.com Wed Mar 1 23:48:14 2006 From: aahz at pythoncraft.com (Aahz) Date: Wed, 1 Mar 2006 14:48:14 -0800 Subject: [Python-Dev] Slightly OT: Replying to posts In-Reply-To: <4405EF46.7020100@acm.org> References: <4405EF46.7020100@acm.org> Message-ID: <20060301224814.GA22121@panix.com> On Wed, Mar 01, 2006, Talin wrote: > > However, I would like to be able to reply to posts in such a way as to > have them appear in the appropriate place in the thread hierarchy. Since > I don't have the original email, I cannot reply to it directly; instead > I have to create a new, non-reply email and send it to the list. Simply > editing the subject line to put "Re: subject" would seem to be insufficient. > > Does anyone have a trick for managing this? Or is there a FAQ that > someone can point me to that addresses this issue? If you're a die-hard hacker, you could parse the mailto: link at the top of the message. ;-) -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From thomas at python.org Wed Mar 1 23:58:57 2006 From: thomas at python.org (Thomas Wouters) Date: Wed, 1 Mar 2006 23:58:57 +0100 Subject: [Python-Dev] Webstats for www.python.org et al. In-Reply-To: <20060301195240.GA14953@performancedrivers.com> References: <9e804ac0602282251n9d958a2kbd92a27ea3133a60@mail.gmail.com> <20060301195240.GA14953@performancedrivers.com> Message-ID: <9e804ac0603011458k40fe2dbehd0b8e248873dc753@mail.gmail.com> On 3/1/06, Jack Diederich wrote: > On Wed, Mar 01, 2006 Brett Cannon wrote: > > On 2/28/06, Fredrik Lundh wrote: > > > Thomas Wouters wrote: > > > > > > > I added webstats for all subsites of python.org: > > > > > > > > http://www.python.org/webstats/ > > > > > > what's that "Java/1.4.2_03" user agent doing? (it's responsible for > > > 10% of all hits in january/february, and 20% of the hits today...) > > > > Most likely a crawler. > > > Youch, if I'm reading it right it consumed fully half of the bandwidth > for today on python.org. And what 1.6 million pages did it spider on > the site last month? Something smells broken. Well, here's a hint. The file almost all of them are retrieving is /topics/xml/dtds/xbel-1.0.dtd. They're all being redirected to pyxml.sf.net, though. It's a lot of hits, but www.python.org doesn't serve any actual pages, so the actual load is not that big (at least, not for us :-) It skewes the statistics somewhat, maybe I should ignore the whole /topics/xml tree in the stats. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From martin at v.loewis.de Thu Mar 2 01:40:44 2006 From: martin at v.loewis.de (martin at v.loewis.de) Date: Thu, 02 Mar 2006 01:40:44 +0100 Subject: [Python-Dev] test_compiler failure Message-ID: <1141260044.44063f0c9b106@www.domainfactory-webmail.de> I just found that test_compiler fails on test_contextlib.py. To reproduce the failure, run from compiler import compileFile compileFile("Lib/test/test_contextlib.py") Regards, Martin From baptiste13 at altern.org Thu Mar 2 01:59:04 2006 From: baptiste13 at altern.org (Baptiste Carvello) Date: Thu, 02 Mar 2006 01:59:04 +0100 Subject: [Python-Dev] bytes thoughts In-Reply-To: References: Message-ID: some more thoughts about the bytes object: 1) it would be nice to have an trivial way to change a bytes object to an int / long, and vice versa. Rationale: while manipulating binary data will happen mostly with bytes objects, some operations are better done with ints, like the bit manipulations with the &|~^ operators. So we should make sure there is no impedance mismatch between those 2 ways of editing binary data. Getting an individual byte at a time is not sufficient, because the part of the data you want to edit might span over a few bytes, or simply fall across a byte boundary. Toy implementation: >>> class bytes(list): ... def from_int(cls, value, length): ... return cls([(value >> 8*i) % 256 for i in range(length)[::-1]]) ... from_int=classmethod(from_int) ... def int(self): ... return sum([256**i*n for i,n in enumerate(self[::-1])]) ... >>> The length argument to from_int is necessary to create a fixed number of bytes, event if those bytes are 0. Use case: let's say you have a binary record made of 7 bits of padding and 3x3 bytes of unix permissions. You want to change the user permissions, and store the record back to a bytes object: >>> record=bytes([1,36]) # this could be a slice of a preexisting bytes object >>> perms=record.int() >>> print oct(perms) 0444 >>> perms &=~( 7 <<6 ) # clear the bits corresponding to user permissions >>> perms |= 6 <<6 # set the bits to the new value >>> print oct(perms) 0644 >>> record=bytes.from_int(perms,2) >>> 2) a common case of interactive use is to display a bytes string as a character string in order to spot which parts are text. In this case you ignore non-ASCII characters, and replace everything that cannot be printed with a space (as some hex editors do). So you don't need to care about encodings. >>> import string >>> def printable(c): ... if not c in string.printable: return ' ' ... if c.isspace(): return ' ' ... return c ... >>> class bytes(list): ... def printable_ascii(self): ... return u"".join([printable(chr(i)) for i in nb]) ... >>> nb=bytes([48,0,10,12,34,65,66]) >>> print nb.printable_ascii() 0 "AB >>> by the way, what will chr return in py3k ? Cheers, BC From guido at python.org Thu Mar 2 02:31:57 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 1 Mar 2006 19:31:57 -0600 Subject: [Python-Dev] Making staticmethod objects callable? In-Reply-To: References: Message-ID: On 3/1/06, Nicolas Fleury wrote: > The problem is that even after explaining descriptors (which IMHO can be > a more advanced feature for newbies), you still need a workaround and > you might end up with (or call __get__): > > class A: > def foo(): pass > bar = foo() > foo = staticmethod(foo) > > Which I find sad considering all the energy debating @decorators;) In which context did you find a need for defining a static method and calling it inside the class definition? I'm guessing that what you're playing dubious scoping games. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Thu Mar 2 03:07:33 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 02 Mar 2006 15:07:33 +1300 Subject: [Python-Dev] bytes.from_hex() In-Reply-To: <44059D3E.2010908@gmail.com> References: <06Feb28.152344pst.58633@synergy1.parc.xerox.com> <44059D3E.2010908@gmail.com> Message-ID: <44065365.9060906@canterbury.ac.nz> Nick Coghlan wrote: > ascii_bytes = orig_bytes.decode("base64").encode("ascii") > > orig_bytes = ascii_bytes.decode("ascii").encode("base64") > > The only slightly odd aspect is that this inverts the conventional meaning of > base64 encoding and decoding, -1. Whatever we do, we shouldn't design things so that it's necessary to write anything as unintuitive as that. We need to make up our minds whether the .encode() and .decode() methods are only meant for Unicode encodings, or whether they are for general transformations between bytes and characters. If they're only meant for Unicode, then bytes should only have .decode(), unicode strings should only have .encode(), and only Unicode codecs should be available that way. Things like base64 would need to have a different interface. If they're for general transformations, then both types should have both methods, with the return type depending on the codec you're using, and it's the programmer's responsibility to use codecs that make sense for what he's doing. But if they're for general transformations, why limit them to just bytes and characters? Following that through leads to giving *every* object .encode() and .decode() methods. I don't think we should go that far, but it's hard to see where to draw the line. Are bytes and strings special enough to justify them having their own peculiar methods for codec access? -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Thu Mar 2 03:31:41 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 02 Mar 2006 15:31:41 +1300 Subject: [Python-Dev] bytes.from_hex() In-Reply-To: <4405B465.9060802@ronadam.com> References: <06Feb28.152344pst.58633@synergy1.parc.xerox.com> <44059D3E.2010908@gmail.com> <4405B465.9060802@ronadam.com> Message-ID: <4406590D.7010108@canterbury.ac.nz> Ron Adam wrote: > While playing around with the example bytes class I noticed code reads > much better when I use methods called tounicode and tostring. > > b64ustring = b.tounicode('base64') > b = bytes(b64ustring, 'base64') I don't like that, because it creates a dependency (conceptually, at least) between the bytes type and the unicode type. And why unicode in particular? Why should it have a tounicode() method, but not a toint() or tofloat() or tolist() etc.? > I'm not suggesting we start using to-type everywhere, just where it > might make things clearer over decode and encode. Another thing is that it only works if the codec transforms between two different types. If you have a bytes-to-bytes transformation, for example, then b2 = b1.tobytes('some-weird-encoding') is ambiguous. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From murman at gmail.com Thu Mar 2 03:41:11 2006 From: murman at gmail.com (Michael Urman) Date: Wed, 1 Mar 2006 20:41:11 -0600 Subject: [Python-Dev] C++ for CPython 3? (Re: str.count is slow) In-Reply-To: <1141229418.4405c76a38edd@www.domainfactory-webmail.de> References: <1141073696.107136.318600@j33g2000cwa.googlegroups.com> <1141083127.970403.147100@v46g2000cwv.googlegroups.com> <4403EFEE.80002@canterbury.ac.nz> <4404D79D.2070608@canterbury.ac.nz> <440551F2.4030702@denviso.de> <1141229418.4405c76a38edd@www.domainfactory-webmail.de> Message-ID: On 3/1/06, martin at v.loewis.de wrote: > There are a few advantages, though, mainly: > - increased type-safety, in particular for API that isn't type-checked > at all at the moment (e.g. PyArg_ParseTuple) How would this be accomplished - by a function with a ton of optional templated arguments? By some sort of TupleParser(tuple) >> var1 >> var2 >> TupleParser::done? > - more reliable reference counting, due to destructors of local > variables Only true when the rules are consistent with what smart pointers or the like do. When there's more than a single rule, this goes out the window because you have to use the correct smart class... > - "native" exception handling, making exceptions both less error-prone > and possible more efficient. ...and exceptions make it impossible to not use smart classes. Since there isn't a nested level of C for each function call in Python, I don't see how exceptions in the implementation language would help exceptions in Python. Do I misunderstand your point, or is there some really cool trick I'm missing? (To explain my bias, I'm against the idea of the C++ rewrite as I also fail to see the advantages outweighing the disadvantages, especially in light of the amount of rewriting necessary to see the "advantages" cited so far.) Michael -- Michael Urman http://www.tortall.net/mu/blog From greg.ewing at canterbury.ac.nz Thu Mar 2 03:48:00 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 02 Mar 2006 15:48:00 +1300 Subject: [Python-Dev] bytes.from_hex() In-Reply-To: <06Mar1.082951pst.58633@synergy1.parc.xerox.com> References: <06Mar1.082951pst.58633@synergy1.parc.xerox.com> Message-ID: <44065CE0.1040505@canterbury.ac.nz> Bill Janssen wrote: > No, once it's in a particular encoding it's bytes, no longer text. The point at issue is whether the characters produced by base64 are in a particular encoding. According to my reading of the RFC, they're not. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Thu Mar 2 04:19:19 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 02 Mar 2006 16:19:19 +1300 Subject: [Python-Dev] iterator API in Py3. In-Reply-To: <004001c63cf3$043fd710$6a01a8c0@RaymondLaptop1> References: <20060222131328.nzferzk2xxk448cg@login.werra.lunarpages.com> <43FE2F5C.8090700@livinglogic.de> <327376ff0602231839h4c27c999la52039be37c28230@mail.gmail.com> <43FEAD58.2020208@canterbury.ac.nz> <015901c6395d$3b9fd4b0$2e2a960a@RaymondLaptop1> <43FFA4B2.7060303@canterbury.ac.nz> <4402F6A9.1030906@gmail.com> <4403E263.1060505@canterbury.ac.nz> <440411E1.6040401@gmail.com> <4404D747.2050402@canterbury.ac.nz> <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> <44052636.9090709@canterbury.ac.nz> <004001c63cf3$043fd710$6a01a8c0@RaymondLaptop1> Message-ID: <44066437.5010806@canterbury.ac.nz> Raymond Hettinger wrote: > [Greg Ewing] > > > And you don't think there are many different > > types of iterables? > > Um, I meant iterators and suspect you meant the same -- The same comment applies either way. Each type of iterable usually has its own corresponding type of iterator. > YAGNI. The hypothetical "future development" benefit is even more > specious than the appeal for consistency. I've pointed out one possible use that would have provided a benefit, so rather than YAGNI, I'd say WACHUI (We Already Could Have Used It). > Let's try to resist > mucking-up this simple, elegant, and fast iterface by adding another > level of indirection. There's already a level of indirection, because the type creation machinery adds a descriptor to allow the type slot to be called via .next() from Python. If there were a next() function, that descriptor would not be needed. So the total amount of indirection for built-in iterators would remain the same. > Looking > back at Guido's original rationale for naming the method next() instead > of __next__(), http://www.python.org/peps/pep-0234.html , it does not > look like any of those reasons will cease to apply in Py3.0. The discussion in that PEP makes no mention of the flexibility benefits of a next()/.__next__() combination. Mostly it doesn't even discuss the combination at all, only the idea of calling the method __next__(). The idea of a next() function is only mentioned in a parenthetical comment in which the sole reason for rejecting it is "it's too late, we've already implemented it". In 3.0 we will have a rare opportunity to revisit the idea without the "it's too late" constraint. I'm suggesting we give it careful consideration, because we probably won't have another opportunity for quite a long time. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From murman at gmail.com Thu Mar 2 04:25:20 2006 From: murman at gmail.com (Michael Urman) Date: Wed, 1 Mar 2006 21:25:20 -0600 Subject: [Python-Dev] bytes.from_hex() In-Reply-To: References: <06Feb28.152344pst.58633@synergy1.parc.xerox.com> <44059D3E.2010908@gmail.com> <4405B465.9060802@ronadam.com> <4406590D.7010108@canterbury.ac.nz> Message-ID: [My apologies Greg; I meant to send this to the whole list. I really need a list-reply button in GMail. ] On 3/1/06, Greg Ewing wrote: > I don't like that, because it creates a dependency > (conceptually, at least) between the bytes type and > the unicode type. I only find half of this bothersome. The unicode type has a pretty clear dependency on the bytestring type: all I/O needs to be done in bytes. Various APIs may mask this by accepting unicode values and transparently doing the right thing, but from the theoretical standpoint we pretend there is no simple serialization of unicode values. But the reverse is not true: the bytestring type has no dependency on unicode. As a practicality vs purity, however, I think it's a good choice to let the bytestring type have a tie to unicode, much like the str type implicitly does now. But you're absolutely right that adding a .tounicode begs the question why not a .tointeger? To try to step back and summarize the viewpoints I've seen so far, there are three main requirements. 1) We want things that are conceptually text to be stored in memory as unicode values. 2) We want there to be some unambiguous conversion via codecs between bytestrings and unicode values. This should help teaching, learning, and remembering unicode. 3) We want a way to apply and reverse compressions, encodings, encryptions, etc., which are not only between bytestrings and unicode values; they may be between any two arbitrary types. This allows writing practical programs. There seems to be little disagreement over 1, provided sufficiently efficient implementation, or sufficient string powers in the bytestring type. To satisfy both 2 and 3, there seem to be a couple options. What other requirements do we have? For (2): a) Restrict the existing helpers to be only bytestring.decode and unicode.encode, possibly enforcing output types of the opposite kind, and removing bytestring.encode b) Add new methods with these semantics, e.g. bytestring.udecode and unicode.uencode For (3): c) Create new helpers codecs.encode(obj, encoding, errors) and codecs.decode(obj, encoding, errors) d) [Keep existing bytestring and unicode helper methods as is, and] require use of codecs.getencoder() and codecs.getdecoder() for arbitrary starting object types Obviously 2a and 3d do not work together, but 2b and 3c work with either complementary option. What other options do we have? Michael -- Michael Urman http://www.tortall.net/mu/blog From martin at v.loewis.de Thu Mar 2 04:37:40 2006 From: martin at v.loewis.de (martin at v.loewis.de) Date: Thu, 02 Mar 2006 04:37:40 +0100 Subject: [Python-Dev] C++ for CPython 3? (Re: str.count is slow) In-Reply-To: References: <1141073696.107136.318600@j33g2000cwa.googlegroups.com> <1141083127.970403.147100@v46g2000cwv.googlegroups.com> <4403EFEE.80002@canterbury.ac.nz> <4404D79D.2070608@canterbury.ac.nz> <440551F2.4030702@denviso.de> <1141229418.4405c76a38edd@www.domainfactory-webmail.de> Message-ID: <1141270660.44066884b18dc@www.domainfactory-webmail.de> Zitat von Michael Urman : > On 3/1/06, martin at v.loewis.de wrote: > How would this be accomplished - by a function with a ton of optional > templated arguments? That's how I would do it. Actually, I would have PyArg_ParseTuple overloaded with different numbers of arguments, say 0..64. Each of them would be a template. > Only true when the rules are consistent with what smart pointers or > the like do. When there's more than a single rule, this goes out the > window because you have to use the correct smart class... Sure. You have to use PyObject* now, so changing to usage of PyObject_ptr wouldn't be that bad. Remember, we are talking about extension modules *to python* here. > ...and exceptions make it impossible to not use smart classes. Since > there isn't a nested level of C for each function call in Python, I > don't see how exceptions in the implementation language would help > exceptions in Python. Do I misunderstand your point, or is there some > really cool trick I'm missing? Instead of checking for a NULL return value on all functions, a Python exception could be expressed (on the C++ stack) as a C++ exception. So instead of writing n = PyString_FromString(name); if (n == NULL) return NULL; for (ml = methods; ml->ml_name != NULL; ml++) { if ((ml->ml_flags & METH_CLASS) || (ml->ml_flags & METH_STATIC)) { PyErr_SetString(PyExc_ValueError, "module functions cannot set" " METH_CLASS or METH_STATIC"); Py_DECREF(n); return NULL; } v = PyCFunction_NewEx(ml, passthrough, n); if (v == NULL) { Py_DECREF(n); return NULL; } if (PyDict_SetItemString(d, ml->ml_name, v) != 0) { Py_DECREF(v); Py_DECREF(n); return NULL; } Py_DECREF(v); } Py_DECREF(n); you would write n = PyString_FromString(name); for (ml = methods; ml->ml_name != NULL; ml++) { if ((ml->ml_flags & METH_CLASS) || (ml->ml_flags & METH_STATIC)) raise new PyExc_ValueError(PyExc_ValueError, "module functions cannot set" " METH_CLASS or METH_STATIC"); v = PyCFunction_NewEx(ml, passthrough, n); PyDict_SetItemString(d, ml->ml_name, v); } > (To explain my bias, I'm against the idea of the C++ rewrite as I also > fail to see the advantages outweighing the disadvantages, especially > in light of the amount of rewriting necessary to see the "advantages" > cited so far.) That's why I'm explaining the advantages to you. I'm not saying Python 3 should be written in C++, I'm only saying that doing so would have not just disadvantages. Regards, Martin From greg.ewing at canterbury.ac.nz Thu Mar 2 04:42:49 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 02 Mar 2006 16:42:49 +1300 Subject: [Python-Dev] bytes thoughts In-Reply-To: References: Message-ID: <440669B9.3000609@canterbury.ac.nz> Baptiste Carvello wrote: > while manipulating binary data will happen mostly with bytes objects, some > operations are better done with ints, like the bit manipulations with the &|~^ > operators. Why not just support bitwise operations directly on the bytes object? -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From guido at python.org Thu Mar 2 04:44:46 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 1 Mar 2006 21:44:46 -0600 Subject: [Python-Dev] defaultdict and on_missing() In-Reply-To: References: <20060222131328.nzferzk2xxk448cg@login.werra.lunarpages.com> <4402F6A9.1030906@gmail.com> <4403E263.1060505@canterbury.ac.nz> <440411E1.6040401@gmail.com> <4404D747.2050402@canterbury.ac.nz> <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> <44052636.9090709@canterbury.ac.nz> Message-ID: On 3/1/06, Brett Cannon wrote: > But moving over to more attributes for how we access basic interfaces > seems great to me. We shouldn't approach this as "methods good, functions bad" -- nor the opposite, for that matter. When all things are equal, I'd much rather do usability studies to find out what newbies (who haven't been given the OO kool-aid yet) find easier. For example, math functions are typically written as sin(x) -- I wouldn't want to switch that to x.sin() and I think most people would agree. But often, all things aren't equal. Sometimes, a single algorithm can be applied to a large set of different types, and then making it a method is a waste (forcing each type to reimplement the same algorithm). sin() is actually an example of this (since it applies to int, long and float). Other times, the same conceptual operation must be implemented differently for each type, and then a method makes more sense. I like to think of list.sort() as an example of this -- sorting algorithms are tightly coupled to internal data representation, and a generic sort function for mutable sequences would likely be of theoretical interest only -- in practice it would be much slower than an implementation that can make use of the internal representation directly. Operations with two or more arguments are often better expressed as function calls -- for example, map() and filter() don't make much sense as methods on callables or sequences. str.join() is an interesting case, where usability studies may be necessary. I've often been asked why this isn't a list method -- but of course that would be less general, since joining strings applies equally well to other types of sequeces (and iterables). Making it a string method is arguably the right thing to do, since this operation only makes sense for strings. But many people (not me!) find the ", ".join(seq) notation hard to read; some other languages use a built-in function join(seq, str) instead, which is arguably more readable. The type of such a polymorphic function is easily specified: join(sequence[T], T) -> T, where T is a string-ish type. (It should make sense for T==bytes as well; I'm not so sure about T==list though. :-) One problem with the methods approach is that there's less pressure to use the same API for all object types (especially with duck typing). For an example of methods gone horribly wrong, look at Java, where you have .length, String.length(), and Collection.size(). Give me len() any day. I believe Ruby has similar confusing diversity for looping (each/forEach). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From rrr at ronadam.com Thu Mar 2 05:01:10 2006 From: rrr at ronadam.com (Ron Adam) Date: Wed, 01 Mar 2006 22:01:10 -0600 Subject: [Python-Dev] bytes.from_hex() In-Reply-To: <4406590D.7010108@canterbury.ac.nz> References: <06Feb28.152344pst.58633@synergy1.parc.xerox.com> <44059D3E.2010908@gmail.com> <4405B465.9060802@ronadam.com> <4406590D.7010108@canterbury.ac.nz> Message-ID: <44066E06.6040900@ronadam.com> Greg Ewing wrote: > Ron Adam wrote: > >> While playing around with the example bytes class I noticed code reads >> much better when I use methods called tounicode and tostring. >> >> b64ustring = b.tounicode('base64') >> b = bytes(b64ustring, 'base64') > > I don't like that, because it creates a dependency > (conceptually, at least) between the bytes type and > the unicode type. And why unicode in particular? > Why should it have a tounicode() method, but not > a toint() or tofloat() or tolist() etc.? I don't think it creates a dependency between the types, but it does create a stronger relationship between them when a method that returns a fixed type is used. No reason not to other than avoiding having methods that really aren't needed. But if it makes sense to have them, sure. If a codec isn't needed probably using a regular constructor should be used instead. >> I'm not suggesting we start using to-type everywhere, just where it >> might make things clearer over decode and encode. > > Another thing is that it only works if the codec > transforms between two different types. If you > have a bytes-to-bytes transformation, for example, > then > > b2 = b1.tobytes('some-weird-encoding') > > is ambiguous. Are you asking if it's decoding or encoding? bytes to unicode -> decoding unicode to bytes -> encoding bytes to bytes -> ? Good point, I think this defines part the difficulty. 1. We can specify the operation and not be sure of the resulting type. *or* 2. We can specify the type and not always be sure of the operation. maybe there's a way to specify both so it's unambiguous? Ron From greg.ewing at canterbury.ac.nz Thu Mar 2 05:18:38 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 02 Mar 2006 17:18:38 +1300 Subject: [Python-Dev] defaultdict and on_missing() In-Reply-To: References: <20060222131328.nzferzk2xxk448cg@login.werra.lunarpages.com> <4402F6A9.1030906@gmail.com> <4403E263.1060505@canterbury.ac.nz> <440411E1.6040401@gmail.com> <4404D747.2050402@canterbury.ac.nz> <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> <44052636.9090709@canterbury.ac.nz> Message-ID: <4406721E.5000607@canterbury.ac.nz> Guido van Rossum wrote: > str.join() is an interesting case... > Making it a > string method is arguably the right thing to do, since this operation > only makes sense for strings. > The type of such a polymorphic function is easily specified: > join(sequence[T], T) -> T, where T is a string-ish type. I'd say it makes sense for any type that supports concatenation (maybe that's what you mean by "string-ish"?) This looks like a case where the xxx()/__xxx__() pattern could be of benefit. Suppose there were a function def join(seq, sep): if hasattr(sep, '__join__'): return sep.__join__(seq) else: # generic implementation Then you could get nice fast type-specific implementations for strings, bytes, etc., without being limited to those types. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From guido at python.org Thu Mar 2 05:24:34 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 1 Mar 2006 22:24:34 -0600 Subject: [Python-Dev] test_compiler failure In-Reply-To: <1141260044.44063f0c9b106@www.domainfactory-webmail.de> References: <1141260044.44063f0c9b106@www.domainfactory-webmail.de> Message-ID: Thanks! This was due to a bug in the code for creating the AST for an expr-less yield. Fixed in r42772. On 3/1/06, martin at v.loewis.de wrote: > I just found that test_compiler fails on test_contextlib.py. > To reproduce the failure, run > > from compiler import compileFile > compileFile("Lib/test/test_contextlib.py") -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Thu Mar 2 06:16:52 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 02 Mar 2006 18:16:52 +1300 Subject: [Python-Dev] bytes.from_hex() In-Reply-To: <44066E06.6040900@ronadam.com> References: <06Feb28.152344pst.58633@synergy1.parc.xerox.com> <44059D3E.2010908@gmail.com> <4405B465.9060802@ronadam.com> <4406590D.7010108@canterbury.ac.nz> <44066E06.6040900@ronadam.com> Message-ID: <44067FC4.2070704@canterbury.ac.nz> Ron Adam wrote: > 1. We can specify the operation and not be sure of the resulting type. > > *or* > > 2. We can specify the type and not always be sure of the operation. > > maybe there's a way to specify both so it's unambiguous? Here's another take on the matter. When we're doing Unicode encoding or decoding, we're performing a type conversion. The natural way to write a type conversion in Python is with a constructor. But we can't just say u = unicode(b) because that doesn't give enough information. We want to say that b is really of type e.g. "bytes containing utf8 encoded text": u = unicode(b, 'utf8') Here we're not thinking of the 'utf8' as selecting an encoder or decoder, but of giving extra information about the type of b, that isn't carried by b itself. Now, going in the other direction, we might think to write b = bytes(u, 'utf8') But that wouldn't be right, because if we interpret this consistently it would mean we're saying that u contains utf8-encoded information, which is nonsense. What we need is a way of saying "construct me something of type 'bytes containing utf8-encoded text'": b = bytes['utf8'](u) Here I've coined the notation t[enc] which evaluates to a callable object which constructs an object of type t by encoding its argument according to enc. Now let's consider base64. Here, the roles of bytes and unicode are reversed, because the bytes are just bytes without any further interpretation, whereas the unicode is really "unicode containing base64 encoded data". So we write u = unicode['base64'](b) # encoding b = bytes(u, 'base64') # decoding Note that this scheme is reasonably idiot-proof, e.g. u = unicode(b, 'base64') results in a type error, because this specifies a decoding operation, and the base64 decoder takes text as input, not bytes. What happens with transformations where the input and output types are the same? In this scheme, they're not really the same any more, because we're providing extra type information. Suppose we had a code called 'piglatin' which goes from unicode to unicode. The types involved are really "text" and "piglatin-encoded text", so we write u2 = unicode['piglatin'](u1) # encoding u1 = unicode(u2, 'piglatin') # decoding Here you won't get any type error if you get things backwards, but there's not much that can be done about that. You just have to keep straight which of your strings contain piglatin and which don't. Is this scheme any better than having encode and decode methods/functions? I'm not sure, but it shows that a suitably enhanced notion of "data type" can be used to replace the notions of encoding and decoding and maybe reduce potential confusion about which direction is which. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From tony.meyer at gmail.com Thu Mar 2 06:18:51 2006 From: tony.meyer at gmail.com (Tony Meyer) Date: Thu, 2 Mar 2006 18:18:51 +1300 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-16 through 2005-01-31 Message-ID: <6c63de570603012118q5491c1aai104ca37cd1735826@mail.gmail.com> Here's the draft for the second half of January. First half of February on its way soon. Any suggestions/corrections/additions/comments welcome. Thanks! -TAM ============= Announcements ============= ------------------------- Google summer internships ------------------------- Google is looking to fill an unprecedented number of `student intern positions`_ this (US) summer, at several US locations (Mountain View, Santa Monica, Kirkland (Wash.), and New York). The perks are incredible, and Google is not just looking for software development interns - there are also product management positions, and UI design and usability analyst positions. Contributing thread: - `Know anyone interested in a Google internship? `__ .. _student intern positions: http://www.google.com/jobs/intern.html [TAM] ----------------------- Possible Summer of PyPy ----------------------- Armin Rigo announced the possibility of a "Summer of PyPy", which would follow the style of Google's "Summer of Code" in funding students to work on various aspects of PyPy. The possibility has not been confirmed yet, but we'll let you know when there's more info. Contributing thread: - `Summer of PyPy `__ [SJB] ========= Summaries ========= --------------------------------------- Integers and strings in different bases --------------------------------------- Alex Martelli requested the inverse of ``int(, )`` that would convert an int into a string with digits in the appropriate base. There was a lot of discussion of exactly where such functionality should go. Among the suggested locations were: * The str constructor, e.g. ``str(, )`` * A str classmethod, e.g. ``str.from_int(, )`` * An encoding method, e.g. ``str().encode("base")`` * A method on ints, e.g. ``.to_base()`` * A format code, e.g. ``"%b" % `` * A builtin function, e.g. ``base(, )`` * A function in the math module, e.g. ``math.base(, )`` People seemed generally to like the builtin function or math module function options, though there was some debate as to the best name for the function. Guido suggested letting the proposal sit for a week or two to see if anyone could come up with a better name or suggest a better location for the function. (However, he seemed generally in favor of the proposal, suggesting that hex() and oct() should be deprecated and removed in a future version of Python.) No decisions had been made at the time this summary was written. Contributing threads: - `str with base `__ [SJB] ------------------------------------------------ PEP 355: Path - Object oriented filesystem paths ------------------------------------------------ Bj?rn Lindqvist resuscitated the idea of incorporating a Path class based on Jason Ordenorff's path module to the standard library by creating `PEP 355`_. There was some general discussion (and corresponding PEP changes), with much discussion centred on the use of "/" as a join-with-separator operator, which was eventually dropped from the PEP. More discussion considered whether Path should subclass string or not. Subclassing string provides the advantage that Paths can be used in the majority of places where strings are currently used, without modification. However, there are many methods of strings that do not seem appropriate for Path objects. Jason Orendorff would prefer for Paths to not subclass strings, and a new format specifier (e.g. for PyArg_ParseTuple()) be created for use with Paths. There was general agreement that the utility of the module would be highest when Path objects could be seamlessly used where string paths were previous used. The debate centred on whether subclassing string was the best way to do this or not. Path objects clearly are not string objects (e.g. __iter__ and join() are nonsensical with paths). Changing the C API so that Paths are accepted where necessary was the suggested solution, although the PEP (at the time of writing the summary) still subclasses Path from string. Changing the methods from the names used by the os module and Jason's module to ones that conform to PEP 8 was recommended. Jason explained that the reason that there is so much cruft in his path module is that the design is heavily skewed toward people already familiar with the existing standard library equivalents. He feels that a standard library Path class should have different design goals: less redundancy, fewer methods, and PEP 8 compliant names. .. _PEP 355: http://www.python.org/peps/pep-0355.html Contributing threads: - `The path module PEP `__ - `/ as path join operator (was: Re: The path module PEP) `__ - `/ as path join operator `__ - `Path inherits from string `__ - `The path module (class) PEP `__ [TAM] ------------------------------ Including ctypes in the stdlib ------------------------------ The inclusion of ctypes in the standard library hit a snag: although Guido pronounced that including it (with a strongly worded wording in the documentation) was fine, ctypes includes libffi, which is built with GPL'd tools, which is causes licensing issues. A lot of the typical legal-advice-from-everyone-but-lawyers discussion took place, but essentially what is required is a way to calculate the information necessary to build libffi without the GPL'd tools. Hye-Shik Chang did some work on this, and is working on integrating this into the ctypes repository, so ctypes may still may it into the standard library. Contributing threads: - `Include ctypes into core Python? `__ - `DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 `__ - `(libffi) Re: Copyright issue `__ [TAM] ----------------------------------- Improving the documentation process ----------------------------------- Georg Brandl pointed out the new `documentation effort`_ by Fredrik Lundh which uses a mix of HTML and PythonDoc-style markup and allows convenient single-method links like http://effbot.org/lib/os.path.join. Georg Brandl played around a bit with the `CSS styling`_, however the main focus of Fredrik's effort was to lower the threshold for user contributions. Along these lines, he was working on an alternate content management system for editing the Python documentation. He currently has a prototype_ available that mirrors some of the content on python.org, though whether or not this backend will be incorporated into python.org is as yet undecided. .. _documentation effort: http://www.effbot.org/lib/ .. _CSS styling: http://home.in.tum.de/~brandlg/zipfile.html .. _prototype: http://pydotorg.dyndns.org:8000/ Contributing threads: - `New Pythondoc by effbot `__ - `[Doc-SIG] that library reference, again `__ [SJB] --------------------- Updating ConfigParser --------------------- The discussion about possible ConfigParser improvements continued, with the typical failure to make any decisions. As always, the options include small modifications to the existing ConfigParser (e.g. allowing preservation of comments, order, and whitespace), re-writing ConfigParser (perhaps based on existing third-party versions), or providing a more sophisticated configuration file module (either leaving ConfigParser as-is, or rewriting it to use the new module). Contributing threads: - `ConfigParser to save with order `__ - `Extension to ConfigParser `__ - `YAML (was Re: Extension to ConfigParser) `__ - `JSON (was: YAML) `__ [TAM] --------------------------------------- Introducing a basenumber abstract class --------------------------------------- Alex Martelli proposed introducing a class, basenumber, along the lines of the basestring class. The intention was to make it easier to check whether or not something is a number. However, if inheriting from basenumber was required for all new number-like objects, then the duck-typing approach of simply implementing the appropriate interface would no longer work. Also, since inheriting from multiple builtin types is disallowed, a str subclass that needs to act like a number (e.g. a symbolic type) would no longer be able to do so since it could not also inherit from basenumber. Guido suggested an __index__ method that would identify a type as convertible into an integer, though without the loss of precision allowed by __int__. More on this suggestion in the next summary. Contributing threads: - `basenumber redux `__ - `index (was str with base) `__ [SJB] ----------------------------------- New Python website: beta.python.org ----------------------------------- Skip Montanaro pointed out that Tim Parkin is heading up a `new effort`_ to redesign the python.org site. The system is based on content formatted primarily in ReST_ for easy editing, though to fully render the pages a number of other tools are required. Tim hopes to get these all wrapped up in setuptools_ and eggs_ so that anyone can also see the fully rendered pages. Also on the topic of webpage redesign, Steve Holden looked into changing the sourceforge "Download Python" link to point to the right place: http://www.python.org/download/. This link, however, cannot be directly modified, so Martin v. L?wis put a small warning above it. .. _new effort: http://beta.python.org/ .. _ReST: http://docutils.sourceforge.net/rst.html .. _setuptools: http://peak.telecommunity.com/DevCenter/setuptools .. _eggs: http://peak.telecommunity.com/DevCenter/PythonEggs Contributing threads: - `Python icon `__ - `SourceForge Download Page, Subversion Home Page `__ [SJB] --------------------- Readline on OS X 10.4 --------------------- Thomas Heller has a problem building readline on OS X 10.4. This is a known issue, the result of Apple shipping libedit symlinked to readline rather than readline itself. You need to get a third party copy of readline - Bob Ippolito made a _readline egg`_. Alternatively, if LDFLAGS AND CPPFLAGS are set appropriately for /opt/local/lib and opt/local/include, and the DarwinPorts copy of readline is installed, then it will be found by setup.py. Concerns about linking with readline (which is released under the GPL) were raised, that explicitly requiring or checking for readline could violate the licence. A suggestion was made that Python 3000 could stop using readline and use an alternative instead. .. _readline egg: http://python.org/pypi/readline Contributing thread: - `Building on OS X 10.4 fails `__ [TAM] ----------------------------- Yielding from a sub-generator ----------------------------- Before PEP 342, there wasn't a big need for a shortcut to pass control to a "sub-generator" because the following for-loop works well enough: def main_generator(): ... for value in sub_generator(): yield value but now that yield can return a value, that value might have to be passed into sub_generator(), not to mention exceptions. Guido felt that a syntactic solution is needed, if the problem is important enough to be solved; a suggestion was "yield from sub_generator()". Contributing thread: - `yield back-and-forth? `__ [TAM] ---------------------------------------- Using the Win32 API for stat/fstat/wstat ---------------------------------------- Martin v. L?wis wanted to switch the code for stat/fstat/wstat from using msvcrt to the Win32 API. However, currently, when WindowsError is raised, the errno attribute must be interpreted as a Win32 error instead of as an errno.h error as in its superclass, OSError. The Win32 API provides different error codes, so using these for the errno attribute could cause some code breakage. The options seemed to be either to break WindowsError and use the new Win32 API error codes, or to introduce a new windows_error attribute to hold the additional information. No decision had been made at the time this summary was written. Contributing thread: - `"DOS" error codes, WindowsError, and errno `__ [SJB] --------------------------- Making timeit easier to use --------------------------- Connelly Barnes was concerned that the default behavior of the timeit module was not very useful. Since timing a function seems like the most common use of timeit, it should be a simple operation, not a method call with a string describing how to import the function and then another string describing how to call it. Also, the default behavior of 1 million iterations is not useful for any code that cannot be repeated that many times. Instead, the number of iterations should be calibrated as it is in the command-line version. Nick Coghlan provided an implementation for these improvements, but it was unclear whether or not those changes would be applied to the module. Contributing thread: - `timeit module `__ [SJB] --------------------------- Removing deprecated modules --------------------------- The regex, regsub and timing modules have been obsolete since Python 2.0, but are still importable in Python 2.4 (regex and regsub both generate a deprecation warning). Skip Montanaro asked when these would be removed, and Guido said that they should be as soon as possible. Contributing thread: - `When will regex really go away? `__ [TAM] ================ Deferred Threads ================ - `Compiler warnings `__ - `Octal literals `__ ================== Previous Summaries ================== - `os.path.getmtime on Windows `__ - `Birkenfeld's gone `__ - `Ph.D. dissertation ideas? `__ - `Names matter. `__ - `New PEP: Using ssize_t as the index type `__ - `[PATCH] Fix dictionary subclass semantics whenused as global dictionaries `__ - `from __future__ syntax changed `__ =============== Skipped Threads =============== - `PEP 247 and hashlib `__ - `[Python-checkins] r42064 - python/trunk/Lib/logging/handlers.py `__ - `pystate.c changes for Python 2.4.2 `__ - `computed goto's in ceval loop `__ - `subwcrev.exe `__ - `[Python-checkins] r42090 - in python/trunk: Modules/getbuildinfo.c PCbuild/make_buildinfo.vcproj PCbuild/pcbuild.sln PCbuild/pythoncore.vcproj `__ - `Additions to the functional module `__ - `PEP 343 and __context__() `__ - `site triggering a bug in urllib2 `__ - `[Python-checkins] r42116 - python/branches/release24-maint/Lib/unittest.py `__ - `stabilizing builds `__ - `building a module catalogue with buildbot `__ - `Weekly Python Patch/Bug Summary `__ - `Weekly Python Patch/Bug Summary (Revised) `__ - `Long-time shy failure in test_socket_ssl `__ - `[Python-checkins] r42185 - python/trunk/Lib/test/test_socket_ssl.py `__ - `(no subject) `__ From kbk at shore.net Thu Mar 2 06:28:24 2006 From: kbk at shore.net (Kurt B. Kaiser) Date: Thu, 2 Mar 2006 00:28:24 -0500 (EST) Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200603020528.k225SO9n008883@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 382 open ( -3) / 3079 closed (+12) / 3461 total ( +9) Bugs : 880 open (+16) / 5624 closed ( +3) / 6504 total (+19) RFE : 211 open ( +0) / 201 closed ( +1) / 412 total ( +1) New / Reopened Patches ______________________ PEP 357 --- adding nb_index (2006-02-21) http://python.org/sf/1436368 opened by Travis Oliphant Test of PEP 357 (2006-02-21) CLOSED http://python.org/sf/1436377 opened by Travis Oliphant Size of time_t changed to 64 bits in MSVC++ 2005 CRT (2006-02-23) http://python.org/sf/1437769 opened by Atul Varma dict.getdefault() implementation (2006-02-24) CLOSED http://python.org/sf/1438113 opened by Barry A. Warsaw PEP 0328 (relative import) implementation (2006-02-24) CLOSED http://python.org/sf/1438387 opened by Thomas Wouters Patch for bug 1438185: os.renames deletes junction points (2006-02-26) http://python.org/sf/1439312 opened by Grant Olson issue syntax warning for with becoming a keyword (2006-02-27) CLOSED http://python.org/sf/1439917 opened by Neal Norwitz Add col information to parse & ast nodes (2006-02-28) CLOSED http://python.org/sf/1440601 opened by John Ehresman asdl generation changes (2006-02-28) CLOSED http://python.org/sf/1440620 opened by John Ehresman Stop help() crashing with incorrect privileges (2006-03-01) http://python.org/sf/1440660 opened by splitscreen Inconsistency in os' function naming (2006-03-01) http://python.org/sf/1440928 opened by Frederik 'Freso' S. Olesen Updated Demo\parser\unpack.py (2006-03-02) http://python.org/sf/1441452 opened by John Ehresman Patches Closed ______________ Implement PEP 357 for real (2006-02-11) http://python.org/sf/1429591 deleted by teoliphant Test of PEP 357 (2006-02-21) http://python.org/sf/1436377 deleted by teoliphant add collections.defaultdict (2006-02-17) http://python.org/sf/1433928 closed by gvanrossum dict.getdefault() implementation (2006-02-24) http://python.org/sf/1438113 closed by gvanrossum PEP 0328 (relative import) implementation (2006-02-24) http://python.org/sf/1438387 closed by nnorwitz PyThreadState_Delete doesnt' clean autoTLSkey (2006-01-23) http://python.org/sf/1413181 closed by tim_one issue syntax warning for with becoming a keyword (2006-02-27) http://python.org/sf/1439917 closed by nnorwitz PEP 343 with statement (2006-02-21) http://python.org/sf/1435715 closed by gvanrossum Inline Lambda patch (2006-02-18) http://python.org/sf/1434008 closed by loewis More easily extensible logging module (2006-01-22) http://python.org/sf/1412054 closed by vsajip Add col information to parse & ast nodes (2006-02-28) http://python.org/sf/1440601 closed by loewis asdl generation changes (2006-02-28) http://python.org/sf/1440620 closed by jpe new-style exceptions (2005-01-18) http://python.org/sf/1104669 closed by bcannon unknown locale: lt_LT (patch) (2005-02-08) http://python.org/sf/1118341 closed by nerijus New / Reopened Bugs ___________________ urllib has trouble with Windows filenames (2006-02-21) http://python.org/sf/1436428 opened by Donovan Eastman length of unicode string changes print behaviour (2006-02-22) http://python.org/sf/1436532 opened by James urllib.py overwrite HTTPError code with 200 (2005-04-06) http://python.org/sf/1178141 reopened by gbrandl Problem parsing cmdline parameter quoted with a trailing \ (2006-02-22) CLOSED http://python.org/sf/1436900 opened by Wai Yip Tung "continue" in .pdbrc has no effect (2006-02-22) http://python.org/sf/1437051 opened by Coolheads Consulting can't send files via ftp on my MacOS X 10.3.9 (2006-02-23) http://python.org/sf/1437614 opened by Li Quid robotparser crashes if robots.txt contains non-ASCII (2006-02-23) http://python.org/sf/1437699 opened by osvenskan Problems w/ expat 1.95.7 vs. Python 2.4 (2006-02-23) http://python.org/sf/1437785 opened by Skip Montanaro os.renames() crashes NTFS junctions (2006-02-24) http://python.org/sf/1438185 opened by Daniel Rohlfing shutil.move raises OSError when copystat fails (2006-02-25) http://python.org/sf/1438480 opened by Joe Wreschnig modules search in help() crashes on insufficient file perms (2006-02-25) http://python.org/sf/1438537 opened by A. Coder test -e is not portable (Solaris 2.7) (2006-02-27) http://python.org/sf/1439538 opened by H??vard Tveite file to store pickled object should be opened in binary mode (2006-02-27) http://python.org/sf/1439659 opened by Raghuram Devarakonda email.Generator is not idempotent (2006-02-28) http://python.org/sf/1440472 opened by WonderClown UnicodeWriter: "utf-8" hardcoded instead of self.encoding (2006-03-01) http://python.org/sf/1440831 opened by Rodrigo Daunoravicius No simple example for pickle (2006-03-01) http://python.org/sf/1441072 opened by Kent Johnson set update method description is incorrect (2006-03-01) http://python.org/sf/1441264 opened by Richard King compiler module loses docstrings (2006-03-02) http://python.org/sf/1441397 opened by Michael Hudson svn head compiler fails to spot extended slicing (2006-03-02) http://python.org/sf/1441408 opened by Michael Hudson Bugs Closed ___________ Problem parsing cmdline parameter quoted with a trailing \ (2006-02-22) http://python.org/sf/1436900 closed by gbrandl Reading from a killed shell script with popen* under linux (2005-05-03) http://python.org/sf/1194328 closed by boukthor exception cannot be new-style class (2002-02-17) http://python.org/sf/518846 closed by bcannon Standard exceptions are hostile to Unicode (2004-08-20) http://python.org/sf/1012952 closed by bcannon New / Reopened RFE __________________ yday in datetime module (2006-02-21) http://python.org/sf/1436346 opened by Michael Toews RFE Closed __________ Extend pre-allocated integers to cover [0, 255] (2006-02-21) http://python.org/sf/1436243 closed by gbrandl From nnorwitz at gmail.com Thu Mar 2 06:46:59 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 1 Mar 2006 21:46:59 -0800 Subject: [Python-Dev] When will regex really go away? In-Reply-To: References: <17366.29087.747411.639080@montanaro.dyndns.org> Message-ID: On 1/24/06, Guido van Rossum wrote: > On 1/24/06, skip at pobox.com wrote: > > > > I ran Fredrik's listmodules script in my current sandbox and got a > > deprecation warning for the regex module. According to PEP 4 it is already > > obsolete. I saw nothing there about the timeframe for actual removal. Will > > it ever go away? > > ASAP please! > > PEP 4 lists these that were already obsolete in 2.0: > > addpack, cmp, cmpcache, codehack, dircmp, dump, fmt, lockfile, > newdir, Para, poly, regex, regsub, tb, timing, util, whatsound, > tzmod, find, grep, packmail, ni, rand, soundex, cl, sv > > of these, regex, regsub, and timing are still importable in 2.4 (I > don't have a 2.5 handy here at Google, and my home machine seems > inaccessible). ISTM these last three can safely be dropped now. I'll do this, except there are some issues: * Lib/reconvert.py imports regex. Ok to move regex,regsub,recovert to lib-old? * ./Demo/pdist/rcslib.py & ./Demo/sockets/mcast.py import regsub. I don't know how to convert the uses of regsub to re, any volunteers? Shall I just add doc that they are broken? * A whole mess of Demos and Tools use regex. What to do about them? ./Demo/tkinter/guido/mbox.py:import regex ./Demo/tkinter/guido/ManPage.py:import regex ./Demo/tkinter/guido/tkman.py:import regex ./Demo/pdist/makechangelog.py:import regex ./Demo/scripts/mboxconvert.py:import regex ./Demo/scripts/ftpstats.py:import regex ./Demo/scripts/eqfix.py:import regex ./Demo/scripts/update.py:import regex ./Tools/scripts/pathfix.py:import regex ./Tools/scripts/fixcid.py:import regex ./Tools/scripts/classfix.py:import regex ./Tools/scripts/objgraph.py:import regex ./Tools/scripts/methfix.py:import regex ./Tools/scripts/pdeps.py:import regex ./Tools/scripts/ifdef.py:import regex From jcarlson at uci.edu Thu Mar 2 07:31:17 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Wed, 01 Mar 2006 22:31:17 -0800 Subject: [Python-Dev] bytes.from_hex() In-Reply-To: <44067FC4.2070704@canterbury.ac.nz> References: <44066E06.6040900@ronadam.com> <44067FC4.2070704@canterbury.ac.nz> Message-ID: <20060301222824.6081.JCARLSON@uci.edu> Greg Ewing wrote: > u = unicode(b) > u = unicode(b, 'utf8') > b = bytes['utf8'](u) > u = unicode['base64'](b) # encoding > b = bytes(u, 'base64') # decoding > u2 = unicode['piglatin'](u1) # encoding > u1 = unicode(u2, 'piglatin') # decoding Your provided semantics feel cumbersome and confusing to me, as compared with str/unicode.encode/decode() . - Josiah From nidoizo at yahoo.com Thu Mar 2 07:55:03 2006 From: nidoizo at yahoo.com (Nicolas Fleury) Date: Thu, 02 Mar 2006 01:55:03 -0500 Subject: [Python-Dev] Making staticmethod objects callable? In-Reply-To: References: Message-ID: Guido van Rossum wrote: > In which context did you find a need for defining a static method and > calling it inside the class definition? I'm guessing that what you're > playing dubious scoping games. I'm not. I almost never use staticmethod actually. I find them not very pythonic, in my humble own definition of pythonic. But since staticmethod is a standard built-in, I considered valid the question of a programmer relatively new to Python (but obviously appreciating its dynamic nature) wondering why calling a static method inside a class definition doesn't work. A use case is not hard to imagine, especially a private static method called only to build a class attribute. I don't know the philosophy behind making staticmethod a built-in (instead of a function in a module only used in specific occasions), but my guess was that what is normal scoping/regrouping in Java/C++/C# was worth common use support in Python. But your comment about "dubious scoping games" makes me think I, again, didn't guess right;) So yes, I'm proposing something I'll probably never use, but I think would make Python more "welcoming". Regards, Nicolas From tim.peters at gmail.com Thu Mar 2 08:43:00 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 2 Mar 2006 01:43:00 -0600 Subject: [Python-Dev] Arena-freeing obmalloc ready for testing Message-ID: <1f7befae0603012343u1b65135bh7b36381806eb705d@mail.gmail.com> I spent most of my PyCon sprint time so far working on Evan Jones's arena-freeing obmalloc patch: http://www.python.org/sf/1123430 It's ready to test now! The work is on a branch: svn+ssh://svn.python.org/python/branches/tim-obmalloc Only obmalloc.c is changed in that branch, and you can get it directly from: You should be able to drop it into a current trunk checkout without problems. Why it's still on a branch: obmalloc has heroically optimized "hot paths" covering the most frequent cases, and Evan's patch kindly ;-) avoided slowing those paths directly. So I wasn't much worried about speed. However, "head review" revealed that the extremely hot-path Py_ADDRESS_IN_RANGE() macro was no longer correct in all cases (in the presence of free()'ing arenas), and I couldn't repair that without adding an additional test+branch to the macro's expansion. That means the jury is out on speed implications, and anyone who cares about that (including me) gets about a week to measure it (I don't want to drag this out beyond that). I'm optimistic, because the new test compares a quantity already being tested by the macro, a second time against 0, and it's hard to get cheaper than that. However, the new branch isn't predictable, so who knows? Other info: the debug "pymalloc stats" output is expanded, to track and report the current # of arenas allocated, the highwater mark (largest value ever seen) for the current # of arenas allocated, the total # of times an arena was allocated, and the total # of times an arena was free'd. Here's output from the end of a -uall debug-build test run(*) w/ the PYTHONMALLOCSTATS envar set, showing that arenas do get reclaimed: # times object malloc called = 138,387,107 # arenas allocated total = 1,841 # arenas reclaimed = 1,733 # arenas highwater mark = 212 # arenas allocated current = 108 (*) The branch was created before the sprints started, so this doesn't reflect the current trunk tests. test_subprocess was excluded, because it hangs with PYTHONMALLOCSTATS set (the debug output gets written to stderr, and at least one of the test_subprocess tests doesn't expect anything in the spawned process's stderr, the pipe it sets up for stderr fills, and both processes deadlock). For simpler fun, run this silly little program, and look at memory consumption at the prompts: """ x = [] for i in xrange(1000000): x.append([]) raw_input("full ") del x[:] raw_input("empty ") """ For example, in a release build on WinXP, VM size is about 48MB at the "full" prompt, and drops to 3MB at the "empty" prompt. In the trunk (without this patch), VM size falls relatively little from what it is at the "full" prompt (the contiguous vector holding a million PyObject* pointers is freed, but the obmalloc arenas holding a million+1 list objects are never freed). For more info about the patch, see Evan's slides from _last_ year's PyCon: http://evanjones.ca/memory-allocator.pdf From rrr at ronadam.com Thu Mar 2 09:03:20 2006 From: rrr at ronadam.com (Ron Adam) Date: Thu, 02 Mar 2006 02:03:20 -0600 Subject: [Python-Dev] bytes.from_hex() In-Reply-To: <20060301222824.6081.JCARLSON@uci.edu> References: <44066E06.6040900@ronadam.com> <44067FC4.2070704@canterbury.ac.nz> <20060301222824.6081.JCARLSON@uci.edu> Message-ID: <4406A6C8.4020101@ronadam.com> Josiah Carlson wrote: > Greg Ewing wrote: >> u = unicode(b) >> u = unicode(b, 'utf8') >> b = bytes['utf8'](u) >> u = unicode['base64'](b) # encoding >> b = bytes(u, 'base64') # decoding >> u2 = unicode['piglatin'](u1) # encoding >> u1 = unicode(u2, 'piglatin') # decoding > > Your provided semantics feel cumbersome and confusing to me, as compared > with str/unicode.encode/decode() . > > - Josiah This uses syntax to determine the direction of encoding. It would be easier and clearer to just require two arguments or a tuple. u = unicode(b, 'encode', 'base64') b = bytes(u, 'decode', 'base64') b = bytes(u, 'encode', 'utf-8') u = unicode(b, 'decode', 'utf-8') u2 = unicode(u1, 'encode', 'piglatin') u1 = unicode(u2, 'decode', 'piglatin') It looks somewhat cleaner if you combine them in a path style string. b = bytes(u, 'encode/utf-8') u = unicode(b, 'decode/utf-8') Ron From tim.peters at gmail.com Thu Mar 2 09:15:54 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 2 Mar 2006 02:15:54 -0600 Subject: [Python-Dev] Arena-freeing obmalloc ready for testing In-Reply-To: <1f7befae0603012343u1b65135bh7b36381806eb705d@mail.gmail.com> References: <1f7befae0603012343u1b65135bh7b36381806eb705d@mail.gmail.com> Message-ID: <1f7befae0603020015u13f377f9n56c4487a3e8119d1@mail.gmail.com> [Tim Peters] ... > Only obmalloc.c is changed in that branch, and you can get it directly from: > > Heck no -- sorry, that pins it to an out-of-date revision. Use the shorter to get the current revision. From fredrik at pythonware.com Thu Mar 2 09:40:23 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 2 Mar 2006 09:40:23 +0100 Subject: [Python-Dev] Arena-freeing obmalloc ready for testing References: <1f7befae0603012343u1b65135bh7b36381806eb705d@mail.gmail.com> Message-ID: Tim Peters wrote: > For simpler fun, run this silly little program, and look at memory > consumption at the prompts: > > """ > x = [] > for i in xrange(1000000): > x.append([]) > raw_input("full ") > del x[:] > raw_input("empty ") > """ > > For example, in a release build on WinXP, VM size is about 48MB at the > "full" prompt, and drops to 3MB at the "empty" prompt. hurray! can we have more Tim sprints, please ? From nnorwitz at gmail.com Thu Mar 2 09:45:04 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 2 Mar 2006 00:45:04 -0800 Subject: [Python-Dev] ref leak w/except hooks Message-ID: The following code leaks a reference. Original test case from Lib/test/test_sys.py in test_original_excepthook. import sys, StringIO eh = sys.__excepthook__ try: raise ValueError(42) except ValueError, exc: exc_type, exc_value, exc_tb = sys.exc_info() eh(exc_type, None, None) From just at letterror.com Thu Mar 2 09:57:57 2006 From: just at letterror.com (Just van Rossum) Date: Thu, 2 Mar 2006 09:57:57 +0100 Subject: [Python-Dev] bytes.from_hex() In-Reply-To: <4406A6C8.4020101@ronadam.com> Message-ID: Ron Adam wrote: > Josiah Carlson wrote: > > Greg Ewing wrote: > >> u = unicode(b) > >> u = unicode(b, 'utf8') > >> b = bytes['utf8'](u) > >> u = unicode['base64'](b) # encoding > >> b = bytes(u, 'base64') # decoding > >> u2 = unicode['piglatin'](u1) # encoding > >> u1 = unicode(u2, 'piglatin') # decoding > > > > Your provided semantics feel cumbersome and confusing to me, as > > compared with str/unicode.encode/decode() . > > > > - Josiah > > This uses syntax to determine the direction of encoding. It would be > easier and clearer to just require two arguments or a tuple. > > u = unicode(b, 'encode', 'base64') > b = bytes(u, 'decode', 'base64') > > b = bytes(u, 'encode', 'utf-8') > u = unicode(b, 'decode', 'utf-8') > > u2 = unicode(u1, 'encode', 'piglatin') > u1 = unicode(u2, 'decode', 'piglatin') > > > > It looks somewhat cleaner if you combine them in a path style string. > > b = bytes(u, 'encode/utf-8') > u = unicode(b, 'decode/utf-8') It gets from bad to worse :( I always liked the assymmetry between u = unicode(s, "utf8") and s = u.encode("utf8") which I think was the original design of the unicode API. Cudos for whoever came up with that. When I saw b = bytes(u, "utf8") mentioned for the first time, I thought: why on earth must the bytes constructor be coupled to the unicode API?!?! It makes no sense to me whatsoever. Bytes have so much more use besides encoded text. I believe (please correct me if I'm wrong) that the encoding argument of bytes() was invented to make it easier to write byte literals. Perhaps a true bytes literal notation is in order after all? My preference for bytes -> unicode -> bytes API would be this: u = unicode(b, "utf8") # just like we have now b = u.tobytes("utf8") # like u.encode(), but being explicit # about the resulting type As to base64, while it works as a codec ("Why a base64 codec? Because we can!"), I don't find it a natural API at all, for such conversions. (I do however agree with Greg Ewing that base64 encoded data is text, not ascii-encoded bytes ;-) Just-my-2-cts From ncoghlan at gmail.com Thu Mar 2 10:19:55 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 02 Mar 2006 19:19:55 +1000 Subject: [Python-Dev] bytes thoughts In-Reply-To: <440669B9.3000609@canterbury.ac.nz> References: <440669B9.3000609@canterbury.ac.nz> Message-ID: <4406B8BB.2090803@gmail.com> Greg Ewing wrote: > Baptiste Carvello wrote: > >> while manipulating binary data will happen mostly with bytes objects, some >> operations are better done with ints, like the bit manipulations with the &|~^ >> operators. > > Why not just support bitwise operations directly > on the bytes object? > +1! Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From aahz at pythoncraft.com Thu Mar 2 15:18:04 2006 From: aahz at pythoncraft.com (Aahz) Date: Thu, 2 Mar 2006 06:18:04 -0800 Subject: [Python-Dev] ref leak w/except hooks In-Reply-To: References: Message-ID: <20060302141804.GC17378@panix.com> On Thu, Mar 02, 2006, Neal Norwitz wrote: > > The following code leaks a reference. Original test case from > Lib/test/test_sys.py in test_original_excepthook. Did you submit a SF bug report? -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From aahz at pythoncraft.com Thu Mar 2 16:26:03 2006 From: aahz at pythoncraft.com (Aahz) Date: Thu, 2 Mar 2006 07:26:03 -0800 Subject: [Python-Dev] defaultdict and on_missing() In-Reply-To: References: <4402F6A9.1030906@gmail.com> <4403E263.1060505@canterbury.ac.nz> <440411E1.6040401@gmail.com> <4404D747.2050402@canterbury.ac.nz> <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> <44052636.9090709@canterbury.ac.nz> Message-ID: <20060302152603.GC11694@panix.com> On Wed, Mar 01, 2006, Guido van Rossum wrote: > > Operations with two or more arguments are often better expressed as > function calls -- for example, map() and filter() don't make much > sense as methods on callables or sequences. OTOH, my personal style is to always use re.compile() because I can never remember the order of arguments for re.match()/re.search(). -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From nick at craig-wood.com Thu Mar 2 15:54:38 2006 From: nick at craig-wood.com (Nick Craig-Wood) Date: Thu, 2 Mar 2006 14:54:38 +0000 Subject: [Python-Dev] Arena-freeing obmalloc ready for testing In-Reply-To: <1f7befae0603012343u1b65135bh7b36381806eb705d@mail.gmail.com> References: <1f7befae0603012343u1b65135bh7b36381806eb705d@mail.gmail.com> Message-ID: <20060302145438.GA5742@craig-wood.com> On Thu, Mar 02, 2006 at 01:43:00AM -0600, Tim Peters wrote: > I'm optimistic, because the new test compares a quantity already being > tested by the macro, a second time against 0, and it's hard to get > cheaper than that. However, the new branch isn't predictable, so who > knows? When compiling with gcc at least you could give the compiler a hint, eg http://kerneltrap.org/node/4705 > For example, in a release build on WinXP, VM size is about 48MB at the > "full" prompt, and drops to 3MB at the "empty" prompt. In the trunk > (without this patch), VM size falls relatively little from what it is > at the "full" prompt Excellent work! -- Nick Craig-Wood -- http://www.craig-wood.com/nick From barry at python.org Thu Mar 2 16:28:41 2006 From: barry at python.org (Barry Warsaw) Date: Thu, 02 Mar 2006 10:28:41 -0500 Subject: [Python-Dev] defaultdict and on_missing() In-Reply-To: <20060302152603.GC11694@panix.com> References: <4402F6A9.1030906@gmail.com> <4403E263.1060505@canterbury.ac.nz> <440411E1.6040401@gmail.com> <4404D747.2050402@canterbury.ac.nz> <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> <44052636.9090709@canterbury.ac.nz> <20060302152603.GC11694@panix.com> Message-ID: <1141313321.7665.0.camel@geddy.wooz.org> On Thu, 2006-03-02 at 07:26 -0800, Aahz wrote: > On Wed, Mar 01, 2006, Guido van Rossum wrote: > > > > Operations with two or more arguments are often better expressed as > > function calls -- for example, map() and filter() don't make much > > sense as methods on callables or sequences. > > OTOH, my personal style is to always use re.compile() because I can > never remember the order of arguments for re.match()/re.search(). Agreed. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060302/7c48205d/attachment.pgp From skip at pobox.com Thu Mar 2 16:43:30 2006 From: skip at pobox.com (skip at pobox.com) Date: Thu, 2 Mar 2006 09:43:30 -0600 Subject: [Python-Dev] When will regex really go away? In-Reply-To: References: <17366.29087.747411.639080@montanaro.dyndns.org> Message-ID: <17415.4770.779843.810801@montanaro.dyndns.org> Neal> I'll do this, except there are some issues: Neal> * Lib/reconvert.py imports regex. Ok to move regex,regsub,recovert to lib-old? Neal> * ./Demo/pdist/rcslib.py & ./Demo/sockets/mcast.py import regsub.... ... Neal> * A whole mess of Demos and Tools use regex. What to do about them? ... How about creating Demo/old and populating it with stuff that imports regex, regsub or reconvert? Neal> I don't know how to convert the uses of regsub to re, any Neal> volunteers? Whippersnapper... sheesh! I still remember when all we had was regex. And we were thankful for it, by golly. Now you'd think the young-uns never knew it existed. As for converting from regex to re that's what reconvert is for. Give it a whirl. The docstring shows how to use it. Yet another Andrew Kuchling gem as I recall (or maybe an effbot gem). Either way, I was happy it was there when I needed it. Go in peace, reconvert. Skip From guido at python.org Thu Mar 2 16:59:23 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 2 Mar 2006 07:59:23 -0800 Subject: [Python-Dev] defaultdict and on_missing() In-Reply-To: <1141313321.7665.0.camel@geddy.wooz.org> References: <4402F6A9.1030906@gmail.com> <4404D747.2050402@canterbury.ac.nz> <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> <44052636.9090709@canterbury.ac.nz> <20060302152603.GC11694@panix.com> <1141313321.7665.0.camel@geddy.wooz.org> Message-ID: On 3/2/06, Barry Warsaw wrote: > On Thu, 2006-03-02 at 07:26 -0800, Aahz wrote: > > OTOH, my personal style is to always use re.compile() because I can > > never remember the order of arguments for re.match()/re.search(). > > Agreed. I don't have that problem, because the order is the same either way: re.compile(pattern).match(line) re.match(pattern, line) :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik at pythonware.com Thu Mar 2 17:03:39 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 2 Mar 2006 17:03:39 +0100 Subject: [Python-Dev] C++ for CPython 3? (Re: str.count is slow) References: <1141073696.107136.318600@j33g2000cwa.googlegroups.com><1141083127.970403.147100@v46g2000cwv.googlegroups.com> <4403EFEE.80002@canterbury.ac.nz> <4404D79D.2070608@canterbury.ac.nz><440551F2.4030702@denviso.de><1141229418.4405c76a38edd@www.domainfactory-webmail.de> <1141270660.44066884b18dc@www.domainfactory-webmail.de> Message-ID: martin at v.loewis.de wrote: > I'm not saying Python 3 should be written in C++, I'm only saying > that doing so would have not just disadvantages. someone also pointed out in private mail (I think; it doesn't seem to have made it to this list) that CPython's extensive use of "inheritance by aggregation" is invalid C. switching to C++ would be one way to address that, of course. From fredrik at pythonware.com Thu Mar 2 17:10:30 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 2 Mar 2006 17:10:30 +0100 Subject: [Python-Dev] When will regex really go away? References: <17366.29087.747411.639080@montanaro.dyndns.org> <17415.4770.779843.810801@montanaro.dyndns.org> Message-ID: skip at pobox.com wrote: > The docstring shows how to use it. Yet another Andrew Kuchling gem > as I recall (or maybe an effbot gem). amk, most likely. and in 92.65% of all cases, switching from "regex" to "re" involves adding \ in front of (, | and ) if they don't already have them, and removing \ from any instances of (, | and ) that already have them. or something like that. From janssen at parc.com Thu Mar 2 17:25:02 2006 From: janssen at parc.com (Bill Janssen) Date: Thu, 2 Mar 2006 08:25:02 PST Subject: [Python-Dev] defaultdict and on_missing() In-Reply-To: Your message of "Wed, 01 Mar 2006 19:44:46 PST." Message-ID: <06Mar2.082502pst."58633"@synergy1.parc.xerox.com> > For an example of methods gone horribly wrong, look at Java, where you > have .length, String.length(), and Collection.size(). > Give me len() any day. I believe Ruby has similar confusing diversity > for looping (each/forEach). But Java is plagued with the same disease that hit Modula-3, distrust of inheritance. Done right, a base class called SomethingOfWhichTheLengthCanBeComputed would have been defined, with a method "length", and all these other classes would have inherited from it. Never too late to learn from the mistakes of the past... Bill From aahz at pythoncraft.com Thu Mar 2 17:45:56 2006 From: aahz at pythoncraft.com (Aahz) Date: Thu, 2 Mar 2006 08:45:56 -0800 Subject: [Python-Dev] defaultdict and on_missing() In-Reply-To: References: <4404D747.2050402@canterbury.ac.nz> <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> <44052636.9090709@canterbury.ac.nz> <20060302152603.GC11694@panix.com> <1141313321.7665.0.camel@geddy.wooz.org> Message-ID: <20060302164556.GA25392@panix.com> On Thu, Mar 02, 2006, Guido van Rossum wrote: > On 3/2/06, Barry Warsaw wrote: >> On Thu, 2006-03-02 at 07:26 -0800, Aahz wrote: >>> >>> OTOH, my personal style is to always use re.compile() because I can >>> never remember the order of arguments for re.match()/re.search(). >> >> Agreed. > > I don't have that problem, because the order is the same either way: > > re.compile(pattern).match(line) > re.match(pattern, line) But that would require thinking! ;-) More seriously, much as I hate the way ''.join() looks, I have never gotten mixed up about argument order as I used to with string.join(). -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From tim.peters at gmail.com Thu Mar 2 18:27:17 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 2 Mar 2006 11:27:17 -0600 Subject: [Python-Dev] Arena-freeing obmalloc ready for testing In-Reply-To: <20060302145438.GA5742@craig-wood.com> References: <1f7befae0603012343u1b65135bh7b36381806eb705d@mail.gmail.com> <20060302145438.GA5742@craig-wood.com> Message-ID: <1f7befae0603020927j778c4797x8899ed548d3cc64@mail.gmail.com> [Tim Peters] >> ... >> However, the new branch isn't predictable, so who knows? [Nick Craig-Wood] > When compiling with gcc at least you could give the compiler a hint, > eg > > http://kerneltrap.org/node/4705 By "the new branch isn't predictable", I mean that there's apparently no way to guess which way it's going to go that's better than flipping a coin. If I could annotate the branch with the probability of the branch being taken, my best guess would be "half the time". OTOH, if I rearranged the code a little, the new test could become highly predictable. OK, I'll do that :-) > ... > Excellent work! Thanks redirected to Evan, who worked hard for a long time on this. Thanks, Evan! From stephen at xemacs.org Thu Mar 2 18:40:00 2006 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 03 Mar 2006 02:40:00 +0900 Subject: [Python-Dev] C++ for CPython 3? (Re: str.count is slow) In-Reply-To: <1141229418.4405c76a38edd@www.domainfactory-webmail.de> (martin@v.loewis.de's message of "Wed, 01 Mar 2006 17:10:18 +0100") References: <1141073696.107136.318600@j33g2000cwa.googlegroups.com> <1141083127.970403.147100@v46g2000cwv.googlegroups.com> <4403EFEE.80002@canterbury.ac.nz> <4404D79D.2070608@canterbury.ac.nz> <440551F2.4030702@denviso.de> <1141229418.4405c76a38edd@www.domainfactory-webmail.de> Message-ID: <87r75k924v.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "martin" == martin writes: martin> There are a few advantages [to C++], though, mainly: martin> - increased type-safety, in particular for API that isn't martin> type-checked at all at the moment (e.g. PyArg_ParseTuple) That's merely an advantage to having a C++ *compiler*. No need to actually use the C++ *language*. :-) XEmacs has had a policy of compiling without warnings under *both* C and C++ for about 5 years now, and it catches a lot of stupidity before it leaves the developer's sandbox. The feature programmers are occasionally annoyed that a pet C idiom gets disallowed, but that's the only downside we've experienced. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From jcarlson at uci.edu Thu Mar 2 18:32:40 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu, 02 Mar 2006 09:32:40 -0800 Subject: [Python-Dev] bytes.from_hex() In-Reply-To: References: <4406A6C8.4020101@ronadam.com> Message-ID: <20060302090317.E36D.JCARLSON@uci.edu> Just van Rossum wrote: > > Ron Adam wrote: > > > Josiah Carlson wrote: > > > Greg Ewing wrote: > > >> u = unicode(b) > > >> u = unicode(b, 'utf8') > > >> b = bytes['utf8'](u) > > >> u = unicode['base64'](b) # encoding > > >> b = bytes(u, 'base64') # decoding > > >> u2 = unicode['piglatin'](u1) # encoding > > >> u1 = unicode(u2, 'piglatin') # decoding > > > > > > Your provided semantics feel cumbersome and confusing to me, as > > > compared with str/unicode.encode/decode() . > > > > > > - Josiah > > > > This uses syntax to determine the direction of encoding. It would be > > easier and clearer to just require two arguments or a tuple. > > > > u = unicode(b, 'encode', 'base64') > > b = bytes(u, 'decode', 'base64') > > > > b = bytes(u, 'encode', 'utf-8') > > u = unicode(b, 'decode', 'utf-8') > > > > u2 = unicode(u1, 'encode', 'piglatin') > > u1 = unicode(u2, 'decode', 'piglatin') > > > > > > > > It looks somewhat cleaner if you combine them in a path style string. > > > > b = bytes(u, 'encode/utf-8') > > u = unicode(b, 'decode/utf-8') > > It gets from bad to worse :( > > I always liked the assymmetry between > > u = unicode(s, "utf8") > > and > > s = u.encode("utf8") > > which I think was the original design of the unicode API. Cudos for > whoever came up with that. I personally have never used that mechanism. I always used s.decode('utf8') and u.encode('utf8'). I prefer the symmetry that .encode() and .decode() offer. > When I saw > > b = bytes(u, "utf8") > > mentioned for the first time, I thought: why on earth must the bytes > constructor be coupled to the unicode API?!?! It makes no sense to me > whatsoever. It's not a 'unicode API'. See integers for another example where a second argument to a type object defines how to interpret the other argument, or even arrays/structs where the first argument defines the interpretation. > Bytes have so much more use besides encoded text. Agreed. > I believe (please correct me if I'm wrong) that the encoding argument of > bytes() was invented to make it easier to write byte literals. Perhaps a > true bytes literal notation is in order after all? Maybe, but I think the other earlier use-case was for using: s2 = bytes(s1, 'base64') If bytes objects recieved an .encode() method, or even a .tobytes() method. I could be misremembering. > My preference for bytes -> unicode -> bytes API would be this: > > u = unicode(b, "utf8") # just like we have now > b = u.tobytes("utf8") # like u.encode(), but being explicit > # about the resulting type > > As to base64, while it works as a codec ("Why a base64 codec? Because we > can!"), I don't find it a natural API at all, for such conversions. Depending on whose definiton of codec you listen to (is it a compressor/decompressor, or a coder/decoder?), either very little of what we have as 'codecs' are actual codecs (only zlib, etc.), or all of them are. I would imagine that base64, etc., were made into codecs, or really encodings, because base64 is an 'encoding' of binary data in base64 format. Similar to the way you can think of utf8 is an 'encoding' of textual data in utf8 format. I would argue, due to the "one obvious way to do it", that using encodings/codecs should be preferred to one-shot encoding/decoding functions in various modules (with some exceptions). These exceptions are things like pickle, marshal, struct, etc., which may take a non-basestring object and convert it into a byte string, which is arguably an encoding of the object in a particular format. - Josiah From stephen at xemacs.org Thu Mar 2 18:51:19 2006 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 03 Mar 2006 02:51:19 +0900 Subject: [Python-Dev] bytes.from_hex() In-Reply-To: <4402E565.2070408@canterbury.ac.nz> (Greg Ewing's message of "Tue, 28 Feb 2006 00:41:25 +1300") References: <87wtfuv66i.fsf@tleepslib.sk.tsukuba.ac.jp> <20060217202813.5FA2.JCARLSON@uci.edu> <43f6abab.1091371449@news.gmane.org> <87zmknql7b.fsf@tleepslib.sk.tsukuba.ac.jp> <43F8C391.2070405@v.loewis.de> <87slqeqw0g.fsf@tleepslib.sk.tsukuba.ac.jp> <43FA3820.3070607@v.loewis.de> <87d5hhpf53.fsf@tleepslib.sk.tsukuba.ac.jp> <43FAE40B.8090406@canterbury.ac.nz> <87mzgjn2qn.fsf@tleepslib.sk.tsukuba.ac.jp> <43FC4C8B.6080300@canterbury.ac.nz> <87accimwy0.fsf@tleepslib.sk.tsukuba.ac.jp> <43FE528E.1040100@canterbury.ac.nz> <87bqwxj9t8.fsf@tleepslib.sk.tsukuba.ac.jp> <43FFA2C9.4010204@canterbury.ac.nz> <87lkvzgvpp.fsf@tleepslib.sk.tsukuba.ac.jp> <4400E5DE.3070102@canterbury.ac.nz> <87d5hagl5s.fsf@tleepslib.sk.tsukuba.ac.jp> <44024C1D.3020009@canterbury.ac.nz> <87irr1fijz.fsf@tleepslib.sk.tsukuba.ac.jp> <4402E565.2070408@canterbury.ac.nz> Message-ID: <87psl491m0.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Greg" == Greg Ewing writes: Greg> But the base64 string itself *does* have text semantics. What do you mean by that? The strings of abstract "characters" defined by RFC 3548 cannot be concatenated in general, they may only be split at 4-character intervals, they can't be reliably searched as text for a given octet or substring of the underlying binary object, and deletion or insertion of octets can't be done without decoding and re-encoding the whole string. And of course humans can make neither head nor tail of them in most cases. The only useful semantics that they have is "you can apply the base64 decoder" to them. In other words, by far the most important effect of endowing that string with "text semantics" is to force programmers to remember not to use them. Do you really mean to call that "text semantics"? Greg> To me this is no different than using a string of decimal Greg> digit characters to represent an integer, or a string of Greg> hexadecimal digit characters to represent a bit Greg> pattern. Would you say that those are not text, either? "No different"? OK, I'll take you at your word. T2YgY291cnNlIEkgd291bGQgY29uc2lkZXIgdGhvc2UgdGV4dC4gIFRoZXkncmUgaHVtYW4t cmVhZGFibGUu Greg> What about XML? What would you consider the proper data type Greg> for an XML document to be inside a Python program -- bytes Greg> or text? Neither. If I must chose one of those ... well, "I know I have a choice of programming languages, and I won't be using Python for this task." Fortunately, there's ElementTree. What you presumably meant was "what would you consider the proper type for (P)CDATA?" And my answer is "text" for text, and "bytes" for binary data (eg, image or audio). Let ElementTree handle the wire format: if an Element's text attribute has type "bytes", convert to base64 and then to the appropriate coded character set for the channel. I don't wanna know about the content transfer encoding, and I should have no need to. Greg> You seem to want to reserve the term "text" for data that Greg> doesn't ever have to be understood even a little bit by a Greg> computer program, but that seems far too restrictive to me, Greg> and a long way from established usage. What I want to reserve "text" for is data streams that nonprogrammer humans might want to manipulate with pencil, paper, scissors, and paste, or programmers with re and text[n:m] = text2. I have no objection to computers using it, too, and even asking us humans to respect some restrictions on the use of [:]= and +. But to tell us to give up those operations entirely makes it into non-text IMO. Greg> [The] assumption [that the channel is ASCII-compatible] could Greg> be very wrong. What happens if it turns out they really need Greg> to be encoded as UTF-16, or as EBCDIC? All hell breaks Greg> loose, as far as I can see, unless the programmer has kept Greg> very firmly in mind that there is an implicit ASCII encoding Greg> involved. Greg> It's exactly to avoid the need for those kinds of mental Greg> gymnastics Agreed, such bookkeeping would be annoying. But there's no _need_ for it any way you look at it: just leave binary objects as-is until you're ready to put them on the wire.[1] Attach a binary-to-wire codec to this end of the wire, and inject your data there. This puts the responsibility where it belongs: with the author of the wire driver. That's the point, which you already mentioned: nobody but authors of wire drivers[2] and introspective code will need to _explicitly_ call .encode('base64'). Greg> that Py3k will have a unified, encoding-agnostic data type Greg> for all character strings. Yeah, but if base64 produces character strings, Unicode becomes a unified, encoding-agnostic data type for all data. Just base64 everything, and now we don't need a bytes type, right? Note that this is precisely what Emacs/MULE does (with a variable width non-Unicode internal encoding and "base256" instead of base64), so as demented as it may sound, it's all too historically plausible. And it can be implemented, by accident, at the application program level. Why expose our users to increased risk of such trouble? Footnotes: [1] Of course you may want to manipulate the binary data, even as text. But who's going to use the base64 format for that purpose? [2] I mean to include those who are writing the git.object_id(), PGP_key.fingerprint(), and ElementTree.write() methods. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From brett at python.org Thu Mar 2 19:09:28 2006 From: brett at python.org (Brett Cannon) Date: Thu, 2 Mar 2006 10:09:28 -0800 Subject: [Python-Dev] ref leak w/except hooks In-Reply-To: References: Message-ID: On 3/2/06, Neal Norwitz wrote: > The following code leaks a reference. Original test case from > Lib/test/test_sys.py in test_original_excepthook. > > import sys, StringIO > eh = sys.__excepthook__ > try: > raise ValueError(42) > except ValueError, exc: > exc_type, exc_value, exc_tb = sys.exc_info() > eh(exc_type, None, None) Which can be simplified to:: from sys import __excepthook__ as eh try: raise BaseException except: eh(BaseException, None, None) It fails if the first argument to sys.__excepthook__ is either a built-in exception or a classic class. it looks like strings and new-style classes do not trigger it. -Brett From brett at python.org Thu Mar 2 19:36:10 2006 From: brett at python.org (Brett Cannon) Date: Thu, 2 Mar 2006 10:36:10 -0800 Subject: [Python-Dev] ref leak w/except hooks In-Reply-To: References: Message-ID: On 3/2/06, Brett Cannon wrote: > On 3/2/06, Neal Norwitz wrote: > > The following code leaks a reference. Original test case from > > Lib/test/test_sys.py in test_original_excepthook. > > > > import sys, StringIO > > eh = sys.__excepthook__ > > try: > > raise ValueError(42) > > except ValueError, exc: > > exc_type, exc_value, exc_tb = sys.exc_info() > > eh(exc_type, None, None) > > Which can be simplified to:: > > from sys import __excepthook__ as eh > > try: > raise BaseException > except: > eh(BaseException, None, None) > > It fails if the first argument to sys.__excepthook__ is either a > built-in exception or a classic class. it looks like strings and > new-style classes do not trigger it. And is now fixed in rev. 42794 . -Brett From guido at python.org Thu Mar 2 20:24:59 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 2 Mar 2006 11:24:59 -0800 Subject: [Python-Dev] Making staticmethod objects callable? In-Reply-To: References: Message-ID: On 3/1/06, Nicolas Fleury wrote: > Guido van Rossum wrote: > > In which context did you find a need for defining a static method and > > calling it inside the class definition? I'm guessing that what you're > > playing dubious scoping games. > > I'm not. I almost never use staticmethod actually. I find them not > very pythonic, in my humble own definition of pythonic. > > But since staticmethod is a standard built-in, I considered valid the > question of a programmer relatively new to Python (but obviously > appreciating its dynamic nature) wondering why calling a static method > inside a class definition doesn't work. A use case is not hard to > imagine, especially a private static method called only to build a class > attribute. Imagined use cases aren't particularly interesting. As to the "why" question, the answer is simply that it wasn't considered an important use case since nobody could come up with a reason why you'd want to do that apart from exploring the language. > I don't know the philosophy behind making staticmethod a built-in > (instead of a function in a module only used in specific occasions), but > my guess was that what is normal scoping/regrouping in Java/C++/C# was > worth common use support in Python. But your comment about "dubious > scoping games" makes me think I, again, didn't guess right;) At the time there wasn't much thought about it -- it just seemed reasonable to support something that was supported by syntax in other languages with at least a built-in decorator in Python. > So yes, I'm proposing something I'll probably never use, but I think > would make Python more "welcoming". I don't see how adding featuers that nobody uses helps. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik at pythonware.com Thu Mar 2 21:09:54 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 2 Mar 2006 21:09:54 +0100 Subject: [Python-Dev] Making staticmethod objects callable? References: Message-ID: Guido van Rossum wrote: > I don't see how adding featuers that nobody uses helps. the amount of odd staticmethod uses you see on comp.lang.python these days is staggering. From tdelaney at avaya.com Thu Mar 2 22:48:31 2006 From: tdelaney at avaya.com (Delaney, Timothy (Tim)) Date: Fri, 3 Mar 2006 08:48:31 +1100 Subject: [Python-Dev] bytes.from_hex() Message-ID: <2773CAC687FD5F4689F526998C7E4E5FF1E5D3@au3010avexu1.global.avaya.com> Just van Rossum wrote: > My preference for bytes -> unicode -> bytes API would be this: > > u = unicode(b, "utf8") # just like we have now > b = u.tobytes("utf8") # like u.encode(), but being explicit > # about the resulting type +1 - I was going to write exactly the same thing. The `bytes` type shouldn't know anything about unicode - conversions between bytes and unicode is entirely the responsibility of the unicode type. Alternatively, rather than as part of the constructor (though that seems the obvious place) some people may prefer a classmethod: unicode.frombytes(cls, encoding) It gives a nice symmetry. Tim Delaney From tdelaney at avaya.com Thu Mar 2 22:51:12 2006 From: tdelaney at avaya.com (Delaney, Timothy (Tim)) Date: Fri, 3 Mar 2006 08:51:12 +1100 Subject: [Python-Dev] bytes.from_hex() Message-ID: <2773CAC687FD5F4689F526998C7E4E5FF1E5D4@au3010avexu1.global.avaya.com> Delaney, Timothy (Tim) wrote: > unicode.frombytes(cls, encoding) unicode.frombytes(encoding) ... Tim Delaney From greg.ewing at canterbury.ac.nz Fri Mar 3 00:39:16 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 03 Mar 2006 12:39:16 +1300 Subject: [Python-Dev] bytes.from_hex() In-Reply-To: <4406A6C8.4020101@ronadam.com> References: <44066E06.6040900@ronadam.com> <44067FC4.2070704@canterbury.ac.nz> <20060301222824.6081.JCARLSON@uci.edu> <4406A6C8.4020101@ronadam.com> Message-ID: <44078224.9080107@canterbury.ac.nz> Ron Adam wrote: > This uses syntax to determine the direction of encoding. It would be > easier and clearer to just require two arguments or a tuple. > > u = unicode(b, 'encode', 'base64') > b = bytes(u, 'decode', 'base64') The point of the exercise was to avoid using the terms 'encode' and 'decode' entirely, since some people claim to be confused by them. While I succeeded in that, I concede that the result isn't particularly intuitive and is arguably even more confusing. If we're going to continue to use 'encode' and 'decode', why not just make them functions: b = encode(u, 'utf-8') u = decode(b, 'utf-8') In the case of Unicode encodings, if you get them backwards you'll get a type error. The advantage of using functions over methods or constructor arguments is that they can be applied uniformly to any input and output types. -- Greg From greg.ewing at canterbury.ac.nz Fri Mar 3 00:47:36 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 03 Mar 2006 12:47:36 +1300 Subject: [Python-Dev] C++ for CPython 3? (Re: str.count is slow) In-Reply-To: References: <1141073696.107136.318600@j33g2000cwa.googlegroups.com> <1141083127.970403.147100@v46g2000cwv.googlegroups.com> <4403EFEE.80002@canterbury.ac.nz> <4404D79D.2070608@canterbury.ac.nz> <440551F2.4030702@denviso.de> <1141229418.4405c76a38edd@www.domainfactory-webmail.de> <1141270660.44066884b18dc@www.domainfactory-webmail.de> Message-ID: <44078418.9030204@canterbury.ac.nz> Fredrik Lundh wrote: > someone also pointed out in private mail (I think; it doesn't seem to > have made it to this list) that CPython's extensive use of "inheritance > by aggregation" is invalid C. > > switching to C++ would be one way to address that, of course. A rather heavyweight solution to a problem that does not seem to have been a problem in practice so far, only in theory. Practicality beats purity once again... -- Greg From baptiste13 at altern.org Fri Mar 3 01:07:43 2006 From: baptiste13 at altern.org (Baptiste Carvello) Date: Fri, 03 Mar 2006 01:07:43 +0100 Subject: [Python-Dev] bytes thoughts In-Reply-To: <440669B9.3000609@canterbury.ac.nz> References: <440669B9.3000609@canterbury.ac.nz> Message-ID: Greg Ewing a ?crit : > Why not just support bitwise operations directly > on the bytes object? > Sure, what counts is that all the nice features that Python has for editing binary data are usable with the bytes object. These include bitwise operations, hex() and oct() representation functions and litterals, the struct module (as Paul Svensson kindly reminded me). Do I forget something ? From greg.ewing at canterbury.ac.nz Fri Mar 3 01:10:41 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 03 Mar 2006 13:10:41 +1300 Subject: [Python-Dev] bytes.from_hex() In-Reply-To: <87psl491m0.fsf@tleepslib.sk.tsukuba.ac.jp> References: <87wtfuv66i.fsf@tleepslib.sk.tsukuba.ac.jp> <20060217202813.5FA2.JCARLSON@uci.edu> <43f6abab.1091371449@news.gmane.org> <87zmknql7b.fsf@tleepslib.sk.tsukuba.ac.jp> <43F8C391.2070405@v.loewis.de> <87slqeqw0g.fsf@tleepslib.sk.tsukuba.ac.jp> <43FA3820.3070607@v.loewis.de> <87d5hhpf53.fsf@tleepslib.sk.tsukuba.ac.jp> <43FAE40B.8090406@canterbury.ac.nz> <87mzgjn2qn.fsf@tleepslib.sk.tsukuba.ac.jp> <43FC4C8B.6080300@canterbury.ac.nz> <87accimwy0.fsf@tleepslib.sk.tsukuba.ac.jp> <43FE528E.1040100@canterbury.ac.nz> <87bqwxj9t8.fsf@tleepslib.sk.tsukuba.ac.jp> <43FFA2C9.4010204@canterbury.ac.nz> <87lkvzgvpp.fsf@tleepslib.sk.tsukuba.ac.jp> <4400E5DE.3070102@canterbury.ac.nz> <87d5hagl5s.fsf@tleepslib.sk.tsukuba.ac.jp> <44024C1D.3020009@canterbury.ac.nz> <87irr1fijz.fsf@tleepslib.sk.tsukuba.ac.jp> <4402E565.2070408@canterbury.ac.nz> <87psl491m0.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <44078981.3040403@canterbury.ac.nz> Stephen J. Turnbull wrote: > What you presumably meant was "what would you consider the proper type > for (P)CDATA?" No, I mean the whole thing, including all the <...> tags etc. Like you see when you load an XML file into a text editor. (BTW, doesn't the fact that you *can* load an XML file into what we call a "text editor" say something?) > nobody but authors of > wire drivers[2] and introspective code will need to _explicitly_ call > .encode('base64'). Even a wire driver writer will only need it if he's trying to turn a text wire into a binary wire, as far as I can see. -- Greg From martin at v.loewis.de Fri Mar 3 05:36:41 2006 From: martin at v.loewis.de (martin at v.loewis.de) Date: Fri, 03 Mar 2006 05:36:41 +0100 Subject: [Python-Dev] C++ for CPython 3? (Re: str.count is slow) In-Reply-To: References: <1141073696.107136.318600@j33g2000cwa.googlegroups.com><1141083127.970403.147100@v46g2000cwv.googlegroups.com> <4403EFEE.80002@canterbury.ac.nz> <4404D79D.2070608@canterbury.ac.nz><440551F2.4030702@denviso.de><1141229418.4405c76a38edd@www.domainfactory-webmail.de> <1141270660.44066884b18dc@www.domainfactory-webmail.de> Message-ID: <1141360601.4407c7d97d033@www.domainfactory-webmail.de> Zitat von Fredrik Lundh : > > I'm not saying Python 3 should be written in C++, I'm only saying > > that doing so would have not just disadvantages. > > someone also pointed out in private mail (I think; it doesn't seem to > have made it to this list) that CPython's extensive use of "inheritance > by aggregation" is invalid C. > > switching to C++ would be one way to address that, of course. My preferred way of fixing it is to do it the "proper" C way, i.e. make PyObject the first member of each derived type. Regards, Martin From martin at v.loewis.de Fri Mar 3 05:40:33 2006 From: martin at v.loewis.de (martin at v.loewis.de) Date: Fri, 03 Mar 2006 05:40:33 +0100 Subject: [Python-Dev] C++ for CPython 3? (Re: str.count is slow) In-Reply-To: <44078418.9030204@canterbury.ac.nz> References: <1141073696.107136.318600@j33g2000cwa.googlegroups.com> <1141083127.970403.147100@v46g2000cwv.googlegroups.com> <4403EFEE.80002@canterbury.ac.nz> <4404D79D.2070608@canterbury.ac.nz> <440551F2.4030702@denviso.de> <1141229418.4405c76a38edd@www.domainfactory-webmail.de> <1141270660.44066884b18dc@www.domainfactory-webmail.de> <44078418.9030204@canterbury.ac.nz> Message-ID: <1141360833.4407c8c1a3dc1@www.domainfactory-webmail.de> Zitat von Greg Ewing : > A rather heavyweight solution to a problem that does > not seem to have been a problem in practice so far, > only in theory. The problem does exist in practice. Python is deliberately build with -fno-strict-aliasing when GCC is used, and might get compiled incorrectly on any other advanced C compiler. The problem with that bug is that it is both very hard to find when it exists, and very hard to dismiss as theoretical, unless an extensive source code review is performed. Have you done this review in the Python source code to know that there is no potential for misinterpretation to make the claim the problem is only theoretical? Regards, Martin From martin at v.loewis.de Fri Mar 3 05:45:06 2006 From: martin at v.loewis.de (martin at v.loewis.de) Date: Fri, 03 Mar 2006 05:45:06 +0100 Subject: [Python-Dev] C++ for CPython 3? (Re: str.count is slow) In-Reply-To: <87r75k924v.fsf@tleepslib.sk.tsukuba.ac.jp> References: <1141073696.107136.318600@j33g2000cwa.googlegroups.com> <1141083127.970403.147100@v46g2000cwv.googlegroups.com> <4403EFEE.80002@canterbury.ac.nz> <4404D79D.2070608@canterbury.ac.nz> <440551F2.4030702@denviso.de> <1141229418.4405c76a38edd@www.domainfactory-webmail.de> <87r75k924v.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <1141361106.4407c9d2b6b46@www.domainfactory-webmail.de> Zitat von "Stephen J. Turnbull" : > martin> - increased type-safety, in particular for API that isn't > martin> type-checked at all at the moment (e.g. PyArg_ParseTuple) > > That's merely an advantage to having a C++ *compiler*. No need to > actually use the C++ *language*. :-) I don't understand. How can you use a C++ compiler, but not the C++ language? Either a program is required to conform to the C++ syntax (in which case it is a C++ program), or it isn't. In the specific example of ParseTuple, I don't see a C++ solution without templates, FWIW. > XEmacs has had a policy of compiling without warnings under *both* C > and C++ for about 5 years now, and it catches a lot of stupidity > before it leaves the developer's sandbox. Right. It might be possible to write C++ programs that are also C programs, and it is then possible to release binaries of these through the C compiler. However, in the Python case, I doubt it would gain that much. As the recent const dilemma shows, C99 and C++98 have, unfortunately, different interpretations of "const" (with the C interpretation being more strict). Regards, Martin From stephen at xemacs.org Fri Mar 3 08:10:22 2006 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 03 Mar 2006 16:10:22 +0900 Subject: [Python-Dev] bytes.from_hex() In-Reply-To: <44078981.3040403@canterbury.ac.nz> (Greg Ewing's message of "Fri, 03 Mar 2006 13:10:41 +1300") References: <87wtfuv66i.fsf@tleepslib.sk.tsukuba.ac.jp> <87zmknql7b.fsf@tleepslib.sk.tsukuba.ac.jp> <43F8C391.2070405@v.loewis.de> <87slqeqw0g.fsf@tleepslib.sk.tsukuba.ac.jp> <43FA3820.3070607@v.loewis.de> <87d5hhpf53.fsf@tleepslib.sk.tsukuba.ac.jp> <43FAE40B.8090406@canterbury.ac.nz> <87mzgjn2qn.fsf@tleepslib.sk.tsukuba.ac.jp> <43FC4C8B.6080300@canterbury.ac.nz> <87accimwy0.fsf@tleepslib.sk.tsukuba.ac.jp> <43FE528E.1040100@canterbury.ac.nz> <87bqwxj9t8.fsf@tleepslib.sk.tsukuba.ac.jp> <43FFA2C9.4010204@canterbury.ac.nz> <87lkvzgvpp.fsf@tleepslib.sk.tsukuba.ac.jp> <4400E5DE.3070102@canterbury.ac.nz> <87d5hagl5s.fsf@tleepslib.sk.tsukuba.ac.jp> <44024C1D.3020009@canterbury.ac.nz> <87irr1fijz.fsf@tleepslib.sk.tsukuba.ac.jp> <4402E565.2070408@canterbury.ac.nz> <87psl491m0.fsf@tleepslib.sk.tsukuba.ac.jp> <44078981.3040403@canterbury.ac.nz> Message-ID: <87mzg86m1t.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Greg" == Greg Ewing writes: Greg> (BTW, doesn't the fact that you *can* load an XML file into Greg> what we call a "text editor" say something?) Why not answer that question for yourself, and then turn that answer into a description of "text semantics"? For me, it says that, just like a gzipped file or the Linux kernel, I can load an XML file into a text editor. But unlike the .gz or vmlinuz, I can easily find many useful things to do to the XML string in the text editor. Doesn't that make base64 non-text by analogy to other "look but don't touch" strings like a .gz or vmlinuz? -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From stephen at xemacs.org Fri Mar 3 09:51:49 2006 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 03 Mar 2006 17:51:49 +0900 Subject: [Python-Dev] C++ for CPython 3? (Re: str.count is slow) In-Reply-To: <1141361106.4407c9d2b6b46@www.domainfactory-webmail.de> (martin@v.loewis.de's message of "Fri, 03 Mar 2006 05:45:06 +0100") References: <1141073696.107136.318600@j33g2000cwa.googlegroups.com> <1141083127.970403.147100@v46g2000cwv.googlegroups.com> <4403EFEE.80002@canterbury.ac.nz> <4404D79D.2070608@canterbury.ac.nz> <440551F2.4030702@denviso.de> <1141229418.4405c76a38edd@www.domainfactory-webmail.de> <87r75k924v.fsf@tleepslib.sk.tsukuba.ac.jp> <1141361106.4407c9d2b6b46@www.domainfactory-webmail.de> Message-ID: <87irqv6hcq.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "martin" == martin writes: martin> I don't understand. How can you use a C++ compiler, but martin> not the C++ language? An abbreviation for "those features that aren't in C". martin> As the recent const dilemma shows, C99 and C++98 have, martin> unfortunately, different interpretations of "const" (with martin> the C interpretation being more strict). Yeah, that's really unfortunate. I suppose we'll probably run into it fairly quickly. Nevertheless, the restriction to programs that are both legal C89 and legal C++ of similar vintage has paid off for us, in the sense that we went from GCC 2.95 to GCC 3.4 without having significant problems where the compiler suddenly decided our code was unacceptable. (Similar experience with other vendors' compilers.) Recently we've seen a significant rise in warnings that need to be fixed, and occasional errors, as GCC 4.x has become more common. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From rrr at ronadam.com Fri Mar 3 10:22:44 2006 From: rrr at ronadam.com (Ron Adam) Date: Fri, 03 Mar 2006 03:22:44 -0600 Subject: [Python-Dev] bytes.from_hex() In-Reply-To: <44078224.9080107@canterbury.ac.nz> References: <44066E06.6040900@ronadam.com> <44067FC4.2070704@canterbury.ac.nz> <20060301222824.6081.JCARLSON@uci.edu> <4406A6C8.4020101@ronadam.com> <44078224.9080107@canterbury.ac.nz> Message-ID: <44080AE4.7030208@ronadam.com> Greg Ewing wrote: > Ron Adam wrote: > >> This uses syntax to determine the direction of encoding. It would be >> easier and clearer to just require two arguments or a tuple. >> >> u = unicode(b, 'encode', 'base64') >> b = bytes(u, 'decode', 'base64') > > The point of the exercise was to avoid using the terms > 'encode' and 'decode' entirely, since some people claim > to be confused by them. Yes, that was what I was trying for with the tounicode, tostring (tobyte) suggestion, but the direction could become ambiguous as you pointed out. The constructors above have 4 data items implied: 1: The source object which includes the source type and data 2: The codec to use 3: The direction of the operation 4: The destination type (determined by the constructor used) There isn't any ambiguity other than when to use encode or decode, but in this case that really is a documentation problem because there is no ambiguities in this form. Everything is explicit. Another version of the above was pointed out to me off line that might be preferable. u = unicode(b, encode='base64') b = bytes(u, decode='base64') Which would also work with the tostring and tounicode methods. u = b.tounicode(decode='base64') b = u.tobytes(incode='base64') > If we're going to continue to use 'encode' and 'decode', > why not just make them functions: > > b = encode(u, 'utf-8') > u = decode(b, 'utf-8') >>> import codecs >>> codecs.decode('abc', 'ascii') u'abc' There's that time machine again. ;-) > In the case of Unicode encodings, if you get them > backwards you'll get a type error. > > The advantage of using functions over methods or > constructor arguments is that they can be applied > uniformly to any input and output types. If codecs are to be more general, then there may be time when the returned type needs to be specified. This would apply to codecs that could return either bytes or strings, or strings or unicode, or bytes or unicode. Some inputs may equally work with more than one output type. Of course, the answer in these cases may be to just 'know' what you will get, and then convert it to what you want. Cheers, Ron From steve at holdenweb.com Fri Mar 3 10:49:38 2006 From: steve at holdenweb.com (Steve Holden) Date: Fri, 03 Mar 2006 04:49:38 -0500 Subject: [Python-Dev] Using and binding relative names (was Re: PEP forBetter Control of Nested Lexical Scopes) In-Reply-To: <1f7befae0602282313g29f897cu6cbfb27b216149bc@mail.gmail.com> References: <20060220204850.6004.JCARLSON@uci.edu> <7e9b97090602252315mf6d4686ud86dd5163ea76b37@mail.gmail.com> <440205C6.3030301@ronadam.com> <440245B7.5090903@ronadam.com> <1956B0C6-4FE7-41EC-BCBA-37C25CBE470D@gmail.com> <44025955.7090709@ronadam.com> <4402C8F8.1040604@canterbury.ac.nz> <1f7befae0602282313g29f897cu6cbfb27b216149bc@mail.gmail.com> Message-ID: Tim Peters wrote: > [Alex Martelli] > >>>We stole list comprehensions and genexps from Haskell > > > [Greg Ewing] > >>The idea predates Haskell, I think. I first saw it in >>Miranda, and it may have come from something even >>earlier -- SETL, maybe? > > > Haskell indeed took list comprehensions from SETL. SETL in turn > adopted them from pre-computer standard notation in set theory, > related to the oddly named (but not universally so named) "axiom of > comprehension". > > genexps were more directly taken from Icon (because of the "generator" part). SETL and Icon, of course, both have their roots in SNOBOL4, which was designed by Griswold when he worked at (IIRC) Bell Labs. Robert Dewar produced the machine-independent SPITBOL implementation (which I ported to DecSystem 10 as an undergraduate project at Leeds University). Griswold later went to the University of Arizona and developed Icon, Dewar went to Rutgers (I think) and developed SETL. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd www.holdenweb.com Love me, love my blog holdenweb.blogspot.com From steve at holdenweb.com Fri Mar 3 12:48:38 2006 From: steve at holdenweb.com (Steve Holden) Date: Fri, 03 Mar 2006 11:48:38 +0000 Subject: [Python-Dev] When will regex really go away? In-Reply-To: <17415.4770.779843.810801@montanaro.dyndns.org> References: <17366.29087.747411.639080@montanaro.dyndns.org> <17415.4770.779843.810801@montanaro.dyndns.org> Message-ID: skip at pobox.com wrote: > Neal> I'll do this, except there are some issues: > > Neal> * Lib/reconvert.py imports regex. Ok to move regex,regsub,recovert to lib-old? > Neal> * ./Demo/pdist/rcslib.py & ./Demo/sockets/mcast.py import regsub.... > ... > Neal> * A whole mess of Demos and Tools use regex. What to do about them? > ... > > How about creating Demo/old and populating it with stuff that imports regex, > regsub or reconvert? > > Neal> I don't know how to convert the uses of regsub to re, any > Neal> volunteers? > > Whippersnapper... sheesh! I still remember when all we had was regex. And > we were thankful for it, by golly. Now you'd think the young-uns never knew > it existed. > You had regex? You were lucky. (etc., etc.) yorkshireman-ly y'rs - steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd www.holdenweb.com Love me, love my blog holdenweb.blogspot.com From collinw at gmail.com Fri Mar 3 13:58:03 2006 From: collinw at gmail.com (Collin Winter) Date: Fri, 3 Mar 2006 13:58:03 +0100 Subject: [Python-Dev] Faster list comprehensions Message-ID: <43aa6ff70603030458xa88feeayde4c24afe2e0c63e@mail.gmail.com> Hello all, I've just posted a patch to SF (python.org/sf/1442442) that I thought I'd bring to everyone's attention. Executive summary: by changing the behaviour of the unused LIST_APPEND opcode, it's possible to get a 16% speed boost for list comprehensions. Currently, list comprehensions are built up by calling a bound method on the storage list. This is pretty inefficient, as it requires extra stack manipulation and a function call for each element in the final list. My patch takes the LIST_APPEND opcode (which existed in the eval loop but not the compiler) and changes its behaviour; rather than take its list argument off the stack, it takes a stack offset, allowing it to leave the list object in place. This also replaces the slow Python function call with a fast call directly to PyList_Append. In total, this results in a 16% speed improvement [1]. [1] Benchmarking was done by taking the list comprehension examples in Lib/test/test_grammar.py and running them in a loop a few thousand times. Python revision 42780 took 1.69 seconds to run the examples 10000 times; the updated version takes 1.41 seconds. Thanks, Collin Winter From raymond.hettinger at verizon.net Fri Mar 3 18:26:06 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Fri, 03 Mar 2006 12:26:06 -0500 Subject: [Python-Dev] Faster list comprehensions References: <43aa6ff70603030458xa88feeayde4c24afe2e0c63e@mail.gmail.com> Message-ID: <003801c63ee7$8e89c980$6a01a8c0@RaymondLaptop1> [Collin Winter] > I've just posted a patch to SF (python.org/sf/1442442) that I thought > I'd bring to everyone's attention. Executive summary: by changing the > behaviour of the unused LIST_APPEND opcode, it's possible to get a 16% > speed boost for list comprehensions. That was working fine in Py2.4. It appears to have gotten left out of the AST compiler. The patch as submitted needs to be revised to only modify the compiler. The other changes were both unnecessary and incorrect -- those components should be left as they were in Py2.4. Raymond From guido at python.org Fri Mar 3 18:56:09 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 3 Mar 2006 09:56:09 -0800 Subject: [Python-Dev] PEP 357 Message-ID: I think PEP 357 and the corresponding patch (python.org/sf/1436368) ae looking pretty good, except for one issue: Currently, when calling the __index__() method on a long with a value exceeding the boundaries of ssize_t will raise OverflowError. The C API chooses to clip such values, but the Python version thows an exception. I think the Python API ought to be allowed to return any long value and it's up to the C API to clip it. Apart from that, I'm ready to accept the PEP. Other opinions? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz at pythoncraft.com Fri Mar 3 20:15:12 2006 From: aahz at pythoncraft.com (Aahz) Date: Fri, 3 Mar 2006 11:15:12 -0800 Subject: [Python-Dev] iterator API in Py3.0 In-Reply-To: <004001c63cf3$043fd710$6a01a8c0@RaymondLaptop1> References: <015901c6395d$3b9fd4b0$2e2a960a@RaymondLaptop1> <43FFA4B2.7060303@canterbury.ac.nz> <4402F6A9.1030906@gmail.com> <4403E263.1060505@canterbury.ac.nz> <440411E1.6040401@gmail.com> <4404D747.2050402@canterbury.ac.nz> <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> <44052636.9090709@canterbury.ac.nz> <004001c63cf3$043fd710$6a01a8c0@RaymondLaptop1> Message-ID: <20060303191512.GA22242@panix.com> On Wed, Mar 01, 2006, Raymond Hettinger wrote: > > I usually let this sort of thing slide, but the iterator API is too > important for trivial, incompatible, and damaging changes. Looking > back at Guido's original rationale for naming the method next() > instead of __next__(), http://www.python.org/peps/pep-0234.html , it > does not look like any of those reasons will cease to apply in Py3.0. OTOH, there is a final comment that choosing next() was perhaps a mistake that cannot be rectified. However, it can certainly be fixed in Python 3.0. My argument in favor of switching to __next__() is rather simple: * As a writer/teacher, I want to be able to say, "All Python special methods have leading and trailing double-underscore." Period, end of story. * As a programmer, I want special methods to be VISIBLE. Python is a language for reading code. * Finally, as a debater in the future of the Python language, I don't want any time taken up discussing whether any future special method should get the same kind of special treatment we've given to next(). As for your point about efficiency, I don't much care whether we get a built-in next() function, but I think that anyone who is concerned about efficiency should do next = obj.__next__ which saves an attribute lookup, too. I still think the vast majority of cases will stick with for loops that automatically call __next__() directly. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From raymond.hettinger at verizon.net Fri Mar 3 21:14:06 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Fri, 03 Mar 2006 15:14:06 -0500 Subject: [Python-Dev] iterator API in Py3.0 References: <015901c6395d$3b9fd4b0$2e2a960a@RaymondLaptop1><43FFA4B2.7060303@canterbury.ac.nz> <4402F6A9.1030906@gmail.com><4403E263.1060505@canterbury.ac.nz> <440411E1.6040401@gmail.com><4404D747.2050402@canterbury.ac.nz><012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1><44052636.9090709@canterbury.ac.nz><004001c63cf3$043fd710$6a01a8c0@RaymondLaptop1> <20060303191512.GA22242@panix.com> Message-ID: <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1> [Aahz] > * As a writer/teacher, I want to be able to say, "All Python special > methods have leading and trailing double-underscore." Period, end of > story. When teaching your classes, do you sense an aversion to using double underscore methods in regular code? I sense an implied message that these methods are not intended to be called directly (i.e. the discomfort of typing x.__setitem__(k,v) serves as a cue to write x[k]=v instead; likewise, x.__int__() pushes towards int(x) instead). If so, then that is a good reason to leave it.next() as-is. Unlike __setitem__ or __int__, the next() method is intended to be called directly in normal code. Of course, for-loops are the most common case and it makes no difference there; however, in the rest of the cases where the iterator is accessed directly, the current naming is clean, readable, and doesn't provide an aversive cue. The double underscore convention is appropriate where the method is always invoked magically in normal code and not called directly. The next() method is differenct because it is a mixed case, sometimes called magically and sometimes called directly. In the latter case, the name is highly visible and therefore should not have double underscores. I suspect that those who feel differently are ones who usually avoid calling next() directly. That's okay, but we shouldn't muck-up the naming for the rest of us who often do have a need to use next(). This is doubly important because we're now expanding the protocol to include send() and throw(). Adding underscores around them too will only make those methods look harder to use than they actually are. Don't underestimate the psychological revulsion to calling code filled with piles of double underscores. > As for your point about efficiency, I don't much care whether we get a > built-in next() function, but I think that anyone who is concerned about > efficiency should do > > next = obj.__next__ There are plenty of places to blow-off efficiency. This iterator protocol is not one of them. Looping constructs need to be as lightweight as possible. Code like "next=obj.__next__" should be avoided as much as possible. It is ugly, it incidental to the expression of an algorithm, and it should not be encouraged. Raymond From mattjfleming at googlemail.com Fri Mar 3 21:10:49 2006 From: mattjfleming at googlemail.com (Matthew Fleming) Date: Fri, 3 Mar 2006 20:10:49 +0000 Subject: [Python-Dev] Lib/test/test_compiler.py fails Message-ID: <5ff4a1e50603031210w7cf07f3dm@mail.gmail.com> testCompileLibrary (__main__.CompilerTest) ... compiling /home/splitscreen/src/python/python/Lib/dircache.py compiling /home/splitscreen/src/python/python/Lib/smtplib.py compiling /home/splitscreen/src/python/python/Lib/pickle.py ERROR testFlatten (__main__.CompilerTest) ... ok testLineNo (__main__.CompilerTest) ... ok testNewClassSyntax (__main__.CompilerTest) ... ok testYieldExpr (__main__.CompilerTest) ... ok ====================================================================== ERROR: testCompileLibrary (__main__.CompilerTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "Lib/test/test_compiler.py", line 36, in testCompileLibrary compiler.compile(buf, basename, "exec") File "/home/splitscreen/src/python/python/Lib/compiler/pycodegen.py", line 64, in compile gen.compile() File "/home/splitscreen/src/python/python/Lib/compiler/pycodegen.py", line 112, in compile gen = ModuleCodeGenerator(tree) File "/home/splitscreen/src/python/python/Lib/compiler/pycodegen.py", line 1290, in __init__ walk(tree, self) File "/home/splitscreen/src/python/python/Lib/compiler/visitor.py", line 106, in walk walker.preorder(tree, visitor) File "/home/splitscreen/src/python/python/Lib/compiler/visitor.py", line 63, in preorder self.dispatch(tree, *args) # XXX *args make sense? File "/home/splitscreen/src/python/python/Lib/compiler/visitor.py", line 57, in dispatch return meth(node, *args) File "/home/splitscreen/src/python/python/Lib/compiler/pycodegen.py", line 350, in visitModule self.visit(node.node) File "/home/splitscreen/src/python/python/Lib/compiler/visitor.py", line 57, in dispatch return meth(node, *args) File "/home/splitscreen/src/python/python/Lib/compiler/visitor.py", line 40, in default self.dispatch(child, *args) File "/home/splitscreen/src/python/python/Lib/compiler/visitor.py", line 57, in dispatch return meth(node, *args) File "/home/splitscreen/src/python/python/Lib/compiler/pycodegen.py", line 362, in visitFunction self._visitFuncOrLambda(node, isLambda=0) File "/home/splitscreen/src/python/python/Lib/compiler/pycodegen.py", line 380, in _visitFuncOrLambda walk(node.code, gen) File "/home/splitscreen/src/python/python/Lib/compiler/visitor.py", line 106, in walk walker.preorder(tree, visitor) File "/home/splitscreen/src/python/python/Lib/compiler/visitor.py", line 63, in preorder self.dispatch(tree, *args) # XXX *args make sense? File "/home/splitscreen/src/python/python/Lib/compiler/visitor.py", line 57, in dispatch return meth(node, *args) File "/home/splitscreen/src/python/python/Lib/compiler/visitor.py", line 40, in default self.dispatch(child, *args) File "/home/splitscreen/src/python/python/Lib/compiler/visitor.py", line 57, in dispatch return meth(node, *args) File "/home/splitscreen/src/python/python/Lib/compiler/pycodegen.py", line 894, in visitImport level = 0 if "absolute_import" in self.futures else -1 AttributeError: FunctionCodeGenerator instance has no attribute 'futures'[in file pickle.py] ---------------------------------------------------------------------- Ran 5 tests in 0.878s FAILED (errors=1) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060303/71807953/attachment.html From nnorwitz at gmail.com Fri Mar 3 21:22:07 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Fri, 3 Mar 2006 12:22:07 -0800 Subject: [Python-Dev] Lib/test/test_compiler.py fails In-Reply-To: <5ff4a1e50603031210w7cf07f3dm@mail.gmail.com> References: <5ff4a1e50603031210w7cf07f3dm@mail.gmail.com> Message-ID: Yup. It's fixed. n -- On 3/3/06, Matthew Fleming wrote: > testCompileLibrary (__main__.CompilerTest) ... compiling > /home/splitscreen/src/python/python/Lib/dircache.py > compiling > /home/splitscreen/src/python/python/Lib/smtplib.py > compiling /home/splitscreen/src/python/python/Lib/pickle.py > ERROR > testFlatten (__main__.CompilerTest) ... ok > testLineNo (__main__.CompilerTest) ... ok > testNewClassSyntax (__main__.CompilerTest) ... ok > testYieldExpr (__main__.CompilerTest) ... ok > > ====================================================================== > ERROR: testCompileLibrary (__main__.CompilerTest) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "Lib/test/test_compiler.py", line 36, in testCompileLibrary > compiler.compile(buf, basename, "exec") > File > "/home/splitscreen/src/python/python/Lib/compiler/pycodegen.py", > line 64, in compile > gen.compile() > File > "/home/splitscreen/src/python/python/Lib/compiler/pycodegen.py", > line 112, in compile > gen = ModuleCodeGenerator(tree) > File > "/home/splitscreen/src/python/python/Lib/compiler/pycodegen.py", > line 1290, in __init__ > walk(tree, self) > File > "/home/splitscreen/src/python/python/Lib/compiler/visitor.py", > line 106, in walk > walker.preorder(tree, visitor) > File > "/home/splitscreen/src/python/python/Lib/compiler/visitor.py", > line 63, in preorder > self.dispatch(tree, *args) # XXX *args make sense? > File > "/home/splitscreen/src/python/python/Lib/compiler/visitor.py", > line 57, in dispatch > return meth(node, *args) > File > "/home/splitscreen/src/python/python/Lib/compiler/pycodegen.py", > line 350, in visitModule > self.visit(node.node) > File > "/home/splitscreen/src/python/python/Lib/compiler/visitor.py", > line 57, in dispatch > return meth(node, *args) > File > "/home/splitscreen/src/python/python/Lib/compiler/visitor.py", > line 40, in default > self.dispatch(child, *args) > File > "/home/splitscreen/src/python/python/Lib/compiler/visitor.py", > line 57, in dispatch > return meth(node, *args) > File > "/home/splitscreen/src/python/python/Lib/compiler/pycodegen.py", > line 362, in visitFunction > self._visitFuncOrLambda(node, isLambda=0) > File > "/home/splitscreen/src/python/python/Lib/compiler/pycodegen.py", > line 380, in _visitFuncOrLambda > walk(node.code, gen) > File > "/home/splitscreen/src/python/python/Lib/compiler/visitor.py", > line 106, in walk > walker.preorder(tree, visitor) > File > "/home/splitscreen/src/python/python/Lib/compiler/visitor.py", > line 63, in preorder > self.dispatch(tree, *args) # XXX *args make sense? > File > "/home/splitscreen/src/python/python/Lib/compiler/visitor.py", > line 57, in dispatch > return meth(node, *args) > File > "/home/splitscreen/src/python/python/Lib/compiler/visitor.py", > line 40, in default > self.dispatch(child, *args) > File > "/home/splitscreen/src/python/python/Lib/compiler/visitor.py", > line 57, in dispatch > return meth(node, *args) > File > "/home/splitscreen/src/python/python/Lib/compiler/pycodegen.py", > line 894, in visitImport > level = 0 if "absolute_import" in self.futures else -1 > AttributeError: FunctionCodeGenerator instance has no attribute 'futures'[in > file pickle.py] > > ---------------------------------------------------------------------- > Ran 5 tests in 0.878s > > FAILED (errors=1) > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com > > > From mcherm at mcherm.com Fri Mar 3 22:06:50 2006 From: mcherm at mcherm.com (Michael Chermside) Date: Fri, 03 Mar 2006 13:06:50 -0800 Subject: [Python-Dev] iterator API in Py3.0 Message-ID: <20060303130650.z9njqhpocgo0kog0@login.werra.lunarpages.com> Raymond writes: > The double underscore convention is appropriate where the method is always > invoked magically in normal code and not called directly. The next() method is > differenct because it is a mixed case, sometimes called magically and sometimes > called directly. In the latter case, the name is highly visible and therefore > should not have double underscores. I think it's clear that if a method is invoked magically, it ought to have underscores; if it is invoked directly then it ought not to. next() is invoked both ways, so the question is which of the following invariants we would rather maintain: * Any method that gets invoked 'magically' (by syntax) will have underscores. * Any method with underscores should be invoked directly only if you are practicing deep magic. What I would *really* like is to keep both invariants... but the only way to do that is to have a __next__() method paired with a next() builtin. (The pattern we use for things like len().) This is great for everyone except for (1) backward compatibility, (2) performance (but does the indirection really need to hurt, and how much?), and (3) proliferation of builtins (adding one more for next() won't hurt, but if we make a practice of it we'll eventually have too many). All told, I prefer using the underscores. I think that the first invariant is important. But it's a judgement call, and a close one, so I'll be happy with either decision. -- Michael Chermside From jeremy at alum.mit.edu Fri Mar 3 22:09:34 2006 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Fri, 3 Mar 2006 16:09:34 -0500 Subject: [Python-Dev] iterator API in Py3.0 In-Reply-To: <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1> References: <015901c6395d$3b9fd4b0$2e2a960a@RaymondLaptop1> <4403E263.1060505@canterbury.ac.nz> <440411E1.6040401@gmail.com> <4404D747.2050402@canterbury.ac.nz> <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> <44052636.9090709@canterbury.ac.nz> <004001c63cf3$043fd710$6a01a8c0@RaymondLaptop1> <20060303191512.GA22242@panix.com> <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1> Message-ID: On 3/3/06, Raymond Hettinger wrote: > The double underscore convention is appropriate where the method is always > invoked magically in normal code and not called directly. The next() method is > differenct because it is a mixed case, sometimes called magically and sometimes > called directly. In the latter case, the name is highly visible and therefore > should not have double underscores. > > I suspect that those who feel differently are ones who usually avoid calling > next() directly. That's okay, but we shouldn't muck-up the naming for the rest > of us who often do have a need to use next(). > > This is doubly important because we're now expanding the protocol to include > send() and throw(). Adding underscores around them too will only make those > methods look harder to use than they actually are. Don't underestimate the > psychological revulsion to calling code filled with piles of double underscores. I think it is a little odd that next is not spelled __next__, but I appreciate the reasons given here in particular. Every time I right .next(), I'm happy that it doesn't have underscores. Jeremy From barry at python.org Fri Mar 3 22:23:54 2006 From: barry at python.org (Barry Warsaw) Date: Fri, 03 Mar 2006 16:23:54 -0500 Subject: [Python-Dev] iterator API in Py3.0 In-Reply-To: <20060303130650.z9njqhpocgo0kog0@login.werra.lunarpages.com> References: <20060303130650.z9njqhpocgo0kog0@login.werra.lunarpages.com> Message-ID: <1141421034.10805.60.camel@resist.wooz.org> On Fri, 2006-03-03 at 13:06 -0800, Michael Chermside wrote: > I think it's clear that if a method is invoked magically, it ought to have > underscores; if it is invoked directly then it ought not to. next() is > invoked both ways, so the question is which of the following invariants > we would rather maintain: > > * Any method that gets invoked 'magically' (by syntax) will have > underscores. > > * Any method with underscores should be invoked directly only if > you are practicing deep magic. Practicality beats purity, so IMHO if the user should type it, it should not have underscores. If we grow a built-in next() -- which I'm -0 on, then clearly it should get renamed to __next__() because then the user would /not/ be expected to write it. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060303/cc787145/attachment.pgp From pje at telecommunity.com Fri Mar 3 23:08:05 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 03 Mar 2006 17:08:05 -0500 Subject: [Python-Dev] iterator API in Py3.0 In-Reply-To: References: <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1> <015901c6395d$3b9fd4b0$2e2a960a@RaymondLaptop1> <4403E263.1060505@canterbury.ac.nz> <440411E1.6040401@gmail.com> <4404D747.2050402@canterbury.ac.nz> <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> <44052636.9090709@canterbury.ac.nz> <004001c63cf3$043fd710$6a01a8c0@RaymondLaptop1> <20060303191512.GA22242@panix.com> <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1> Message-ID: <5.1.1.6.0.20060303170738.03723050@mail.telecommunity.com> At 04:09 PM 3/3/2006 -0500, Jeremy Hylton wrote: >I think it is a little odd that next is not spelled __next__, but I >appreciate the reasons given here in particular. Every time I right >.next(), I'm happy that it doesn't have underscores. But then next(ob) should make you even happier, because then there's no '.', either. :) From jeremy at alum.mit.edu Fri Mar 3 23:24:31 2006 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Fri, 3 Mar 2006 17:24:31 -0500 Subject: [Python-Dev] iterator API in Py3.0 In-Reply-To: <5.1.1.6.0.20060303170738.03723050@mail.telecommunity.com> References: <015901c6395d$3b9fd4b0$2e2a960a@RaymondLaptop1> <440411E1.6040401@gmail.com> <4404D747.2050402@canterbury.ac.nz> <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> <44052636.9090709@canterbury.ac.nz> <004001c63cf3$043fd710$6a01a8c0@RaymondLaptop1> <20060303191512.GA22242@panix.com> <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1> <5.1.1.6.0.20060303170738.03723050@mail.telecommunity.com> Message-ID: On 3/3/06, Phillip J. Eby wrote: > At 04:09 PM 3/3/2006 -0500, Jeremy Hylton wrote: > >I think it is a little odd that next is not spelled __next__, but I > >appreciate the reasons given here in particular. Every time I right > >.next(), I'm happy that it doesn't have underscores. > > But then next(ob) should make you even happier, because then there's no > '.', either. :) I've got no problem with dots. No shift key necessary :-). On the subject of predilections and builtins, I find myself typing .size() a lot lately. Too much C++, I guess, but it seems so natural as I type it. Jeremy From anthony at interlink.com.au Fri Mar 3 23:38:10 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Sat, 4 Mar 2006 09:38:10 +1100 Subject: [Python-Dev] C++ for CPython 3? (Re: str.count is slow) In-Reply-To: <1141361106.4407c9d2b6b46@www.domainfactory-webmail.de> References: <1141073696.107136.318600@j33g2000cwa.googlegroups.com> <87r75k924v.fsf@tleepslib.sk.tsukuba.ac.jp> <1141361106.4407c9d2b6b46@www.domainfactory-webmail.de> Message-ID: <200603040938.11052.anthony@interlink.com.au> It's probably worth mentioning that right now, we don't even come close to compiling with a C++ compiler. A bunch of the bugs are shallow (casting result from malloc, that sort of thing) but a bunch more look a tad uglier. Is this something worth trying to fix? Fixing the shallow bugs at least might be worthwhile... Anthony From greg.ewing at canterbury.ac.nz Sat Mar 4 00:46:56 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 04 Mar 2006 12:46:56 +1300 Subject: [Python-Dev] bytes.from_hex() In-Reply-To: <87mzg86m1t.fsf@tleepslib.sk.tsukuba.ac.jp> References: <87wtfuv66i.fsf@tleepslib.sk.tsukuba.ac.jp> <87zmknql7b.fsf@tleepslib.sk.tsukuba.ac.jp> <43F8C391.2070405@v.loewis.de> <87slqeqw0g.fsf@tleepslib.sk.tsukuba.ac.jp> <43FA3820.3070607@v.loewis.de> <87d5hhpf53.fsf@tleepslib.sk.tsukuba.ac.jp> <43FAE40B.8090406@canterbury.ac.nz> <87mzgjn2qn.fsf@tleepslib.sk.tsukuba.ac.jp> <43FC4C8B.6080300@canterbury.ac.nz> <87accimwy0.fsf@tleepslib.sk.tsukuba.ac.jp> <43FE528E.1040100@canterbury.ac.nz> <87bqwxj9t8.fsf@tleepslib.sk.tsukuba.ac.jp> <43FFA2C9.4010204@canterbury.ac.nz> <87lkvzgvpp.fsf@tleepslib.sk.tsukuba.ac.jp> <4400E5DE.3070102@canterbury.ac.nz> <87d5hagl5s.fsf@tleepslib.sk.tsukuba.ac.jp> <44024C1D.3020009@canterbury.ac.nz> <87irr1fijz.fsf@tleepslib.sk.tsukuba.ac.jp> <4402E565.2070408@canterbury.ac.nz> <87psl491m0.fsf@tleepslib.sk.tsukuba.ac.jp> <44078981.3040403@canterbury.ac.nz> <87mzg86m1t.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <4408D570.4060602@canterbury.ac.nz> Stephen J. Turnbull wrote: > Doesn't that make base64 non-text by analogy to other "look but don't > touch" strings like a .gz or vmlinuz? No, because I can take a piece of base64 encoded data and use a text editor to manually paste it in with some other text (e.g. a plain-text (not MIME) mail message). Then I can mail it to someone, or send it by text-mode ftp, or translate it from Unix to MSDOS line endings and give it to a Windows user, or translate it into EBCDIC and give it to someone who has an IBM mainframe, etc, etc. And the person at the other end can use their text editor to manually extract it and decode it and recover the original data. I can't do any of those directly with a .gz file or vmlinuz. I'm not just making those uses up, BTW. It's not very long ago people used to do things like that all the time with uuencode, binhex, etc -- because mail and news at the time were strictly text channels. They still are, really -- otherwise we wouldn't be using anything as hairy as MIME, we'd just mail our binary files as-is. -- Greg From greg.ewing at canterbury.ac.nz Sat Mar 4 01:02:26 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 04 Mar 2006 13:02:26 +1300 Subject: [Python-Dev] bytes.from_hex() In-Reply-To: <44080AE4.7030208@ronadam.com> References: <44066E06.6040900@ronadam.com> <44067FC4.2070704@canterbury.ac.nz> <20060301222824.6081.JCARLSON@uci.edu> <4406A6C8.4020101@ronadam.com> <44078224.9080107@canterbury.ac.nz> <44080AE4.7030208@ronadam.com> Message-ID: <4408D912.2060006@canterbury.ac.nz> Ron Adam wrote: > This would apply to codecs that > could return either bytes or strings, or strings or unicode, or bytes or > unicode. I'd need to see some concrete examples of such codecs before being convinced that they exist, or that they couldn't just as well return a fixed type that you then transform to what you want. I suspect that said transformation would involve some further encoding or decoding, in which case you really have more than one codec. -- Greg From rrr at ronadam.com Sat Mar 4 03:17:15 2006 From: rrr at ronadam.com (Ron Adam) Date: Fri, 03 Mar 2006 20:17:15 -0600 Subject: [Python-Dev] bytes.from_hex() In-Reply-To: <4408D912.2060006@canterbury.ac.nz> References: <44066E06.6040900@ronadam.com> <44067FC4.2070704@canterbury.ac.nz> <20060301222824.6081.JCARLSON@uci.edu> <4406A6C8.4020101@ronadam.com> <44078224.9080107@canterbury.ac.nz> <44080AE4.7030208@ronadam.com> <4408D912.2060006@canterbury.ac.nz> Message-ID: <4408F8AB.9070705@ronadam.com> Greg Ewing wrote: > Ron Adam wrote: > >> This would apply to codecs that >> could return either bytes or strings, or strings or unicode, or bytes or >> unicode. > > I'd need to see some concrete examples of such codecs > before being convinced that they exist, or that they > couldn't just as well return a fixed type that you > then transform to what you want. I think text some codecs that currently return 'ascii' encoded text would be candidates. If you use u'abc'.encode('rot13') you get an ascii string back and not a unicode string. And if you use decode to get back, you don't get the original unicode back, but an ascii representation of the original you then need to decode to unicode. > I suspect that said transformation would involve some > further encoding or decoding, in which case you really > have more than one codec. Yes, I can see that. So the following are probable better reasons to specify the type. Codecs are very close to types and they quite often result in a type change, having the change visible in the code adds to overall readability. This is probably my main desire for this. There is another reason for being explicit about types with codecs. If you store the codecs with a tuple of attributes as the keys, (name, in_type, out_type), then it makes it possible to look up the codec with the correct behavior and then just do it. The alternative is to test the input, try it, then test the output. The look up doesn't add much overhead, but does adds safety. Codecs don't seem to be the type of thing where you will want to be able to pass a wide variety of objects into. So a narrow slot is probably preferable to a wide one here. In cases where a codec might be useful in more than one combination of types, it could have an entry for each valid combination in the lookup table. The codec lookup also validates the desired operation for nearly free. Of course, the data will need to be valid as well. ;-) Cheers, Ron From tim.peters at gmail.com Sat Mar 4 03:53:03 2006 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 3 Mar 2006 21:53:03 -0500 Subject: [Python-Dev] .py and .txt files missing svn:eol-style in trunk In-Reply-To: <20060301092016.GB12002@phd.pp.ru> References: <1f7befae0602282217j1700cfb1j4b73ddf80ef1428a@mail.gmail.com> <20060301092016.GB12002@phd.pp.ru> Message-ID: <1f7befae0603031853m5456d95av2057131888e5c911@mail.gmail.com> [Oleg Broytmann[ > My experience shows that if the developers use different OSes (our team > uses Linux and Windows) it helps very much to set svn:eol-style to native > for all text files - *.py, *.txt, etc, except for files with special > requirements. Yes. > So I use the following shell script > > #! /bin/sh > > svn add "$@" > svn propset svn:eol-style native "$@" > svn propset svn:keywords "Date Rev Id" "$@" > > to add *.py files to the repository. (I don't want to put it in a global > configuration because I have different requirements for different projects.) > > For other text files the same except keywords: > > #! /bin/sh > > svn add "$@" > svn propset svn:eol-style native "$@" > >> test_pepe263 probably should be binary (it contains "funny >> characters"). > > It should be text with encoding > > $ svn propset svn:mime-type "text/plain; charset=koi8-r" test_pep263 "The reason" it's binary is given in a comment at the top of the file: ""|" # This file is marked as binary in the CVS, to prevent MacCVS from recoding it. """ So someone on a Mac should check whether the Mac flavor of SVN also screws it up. From Martin.Maly at microsoft.com Sat Mar 4 04:39:11 2006 From: Martin.Maly at microsoft.com (Martin Maly) Date: Fri, 3 Mar 2006 19:39:11 -0800 Subject: [Python-Dev] Bug in from __future__ processing? Message-ID: <5C0A6F919D675745BB1DBA7412DB68F501BA896B@df-foxhound-msg.exchange.corp.microsoft.com> Hello, The Python spec states that the "from __future__ import ..." statement can only occur at the beginning of the file, preceded only by doc strings, comments, empty lines or other future statements. The following code snippets, however, don't raise Syntax error in Python 2.4.2. Is it a bug? I am asking whether in IronPython we should try to match behavior of Python compiler, or stick to the language spec. In this case, I believe that we should stick to the spec and report a bug in Python compiler. Thanks Martin Example 1: ------------------ x = 10 / 2 from __future__ import division ------------------ Example 2: ------------------ from sys import * from __future__ import division ------------------ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060303/8f1905c8/attachment.htm From tim.peters at gmail.com Sat Mar 4 05:18:29 2006 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 3 Mar 2006 23:18:29 -0500 Subject: [Python-Dev] Bug in from __future__ processing? In-Reply-To: <5C0A6F919D675745BB1DBA7412DB68F501BA896B@df-foxhound-msg.exchange.corp.microsoft.com> References: <5C0A6F919D675745BB1DBA7412DB68F501BA896B@df-foxhound-msg.exchange.corp.microsoft.com> Message-ID: <1f7befae0603032018r9e10d7dh47e4d55e618263d@mail.gmail.com> [Martin Maly] > The Python spec states that the "from __future__ import ?" statement can > only occur at the beginning of the file, preceded only by doc strings, > comments, empty lines or other future statements. The following code > snippets, however, don't raise Syntax error in Python 2.4.2. Is it a bug? > > I am asking whether in IronPython we should try to match behavior of Python > compiler, or stick to the language spec. In this case, I believe that we > should stick to the spec and report a bug in Python compiler. Guido bumped into this about a month ago, noting that he thought it was a bug in current Python trunk that the PEP restrictions _are_ followed now. I replied that I thought it was a bug in released Pythons that the PEP restrictions were _not_ followed: http://mail.python.org/pipermail/python-dev/2006-January/060247.html Doesn't look like Guido responded, so I'll channel him and declare that he intended to agree with me after all ;-) From guido at python.org Sat Mar 4 05:21:55 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 3 Mar 2006 20:21:55 -0800 Subject: [Python-Dev] Bug in from __future__ processing? In-Reply-To: <1f7befae0603032018r9e10d7dh47e4d55e618263d@mail.gmail.com> References: <5C0A6F919D675745BB1DBA7412DB68F501BA896B@df-foxhound-msg.exchange.corp.microsoft.com> <1f7befae0603032018r9e10d7dh47e4d55e618263d@mail.gmail.com> Message-ID: On 3/3/06, Tim Peters wrote: > [Martin Maly] > > The Python spec states that the "from __future__ import ?" statement can > > only occur at the beginning of the file, preceded only by doc strings, > > comments, empty lines or other future statements. The following code > > snippets, however, don't raise Syntax error in Python 2.4.2. Is it a bug? > > > > I am asking whether in IronPython we should try to match behavior of Python > > compiler, or stick to the language spec. In this case, I believe that we > > should stick to the spec and report a bug in Python compiler. > > Guido bumped into this about a month ago, noting that he thought it > was a bug in current Python trunk that the PEP restrictions _are_ > followed now. I replied that I thought it was a bug in released > Pythons that the PEP restrictions were _not_ followed: > > http://mail.python.org/pipermail/python-dev/2006-January/060247.html > > Doesn't look like Guido responded, so I'll channel him and declare > that he intended to agree with me after all ;-) It was so obvious that you were right I didn't bother to agree at the time. But yes, I agree. And I swear on a stack of Python 1.5.2 language reference manuals that you didn't have to twist my arm. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Mar 4 05:26:21 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 3 Mar 2006 20:26:21 -0800 Subject: [Python-Dev] __exit__ API? Message-ID: A few days ago there were rumbling noises that requiring __exit__ to re-raise the exception (as I amended PEP 343 at the time) could lead to easily-missed bugs in __exit__ handlers. After thinking it over I think I agree and I think I'd like to change the API so that the exception is only ignored if __exit__ returns a "true" value. The easiest implementation is probably to just let the WITH_CLEANUP opcode do everything. This becomes a rather heavy opcode then but the alternative is to generate very hairy code (like the original patch did, full of ROT 4 choruses). Any objections? I probably won't get to this until Monday. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.peters at gmail.com Sat Mar 4 05:34:55 2006 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 3 Mar 2006 23:34:55 -0500 Subject: [Python-Dev] Bug in from __future__ processing? In-Reply-To: References: <5C0A6F919D675745BB1DBA7412DB68F501BA896B@df-foxhound-msg.exchange.corp.microsoft.com> <1f7befae0603032018r9e10d7dh47e4d55e618263d@mail.gmail.com> Message-ID: <1f7befae0603032034q41a04d49p597c809ecccfeee7@mail.gmail.com> [Tim] >> Doesn't look like Guido responded, so I'll channel him and declare >> that he intended to agree with me after all ;-) [Guido] > It was so obvious that you were right I didn't bother to agree at the > time. But yes, I agree. Of course you do. It was obvious to you, and therefore-- as your professional channeler --it was also obvious to me. For some inscrutable reason, though, others tend to imagine looming disasters in silence, instead of just plucking your comforting thoughts from the aether ;-) > And I swear on a stack of Python 1.5.2 language reference manuals that you > didn't have to twist my arm. :-) Indeed! But whose arm could we twist to get them to repair the compiler in 2.4? I'd settle for a blurb in the next 2.4 NEWS just noting that 2.5 will follow the documented syntax. That may even be desirable, to avoid breaking working (albeit by accident) code across a micro release. From barry at python.org Sat Mar 4 05:43:28 2006 From: barry at python.org (Barry Warsaw) Date: Fri, 03 Mar 2006 23:43:28 -0500 Subject: [Python-Dev] .py and .txt files missing svn:eol-style in trunk In-Reply-To: <1f7befae0602282217j1700cfb1j4b73ddf80ef1428a@mail.gmail.com> References: <1f7befae0602282217j1700cfb1j4b73ddf80ef1428a@mail.gmail.com> Message-ID: <1141447409.10805.77.camel@resist.wooz.org> On Wed, 2006-03-01 at 00:17 -0600, Tim Peters wrote: > OK, I tried to set eol-style on all of those. svn refused to change these: > > svn: File 'Lib\email\test\data\msg_26.txt' has binary mime type property Yeah, there's no reason for that, so I've fixed it. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060303/8d732aa8/attachment.pgp From barry at python.org Sat Mar 4 05:44:55 2006 From: barry at python.org (Barry Warsaw) Date: Fri, 03 Mar 2006 23:44:55 -0500 Subject: [Python-Dev] .py and .txt files missing svn:eol-style in trunk In-Reply-To: <1141447409.10805.77.camel@resist.wooz.org> References: <1f7befae0602282217j1700cfb1j4b73ddf80ef1428a@mail.gmail.com> <1141447409.10805.77.camel@resist.wooz.org> Message-ID: <1141447495.32263.79.camel@resist.wooz.org> On Fri, 2006-03-03 at 23:43 -0500, Barry Warsaw wrote: > On Wed, 2006-03-01 at 00:17 -0600, Tim Peters wrote: > > OK, I tried to set eol-style on all of those. svn refused to change these: > > > > svn: File 'Lib\email\test\data\msg_26.txt' has binary mime type property > > Yeah, there's no reason for that, so I've fixed it. Or wait, you fixed it there. I fixed it in the sandbox. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060303/a497c0f3/attachment.pgp From fredrik at pythonware.com Sat Mar 4 07:27:41 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sat, 4 Mar 2006 07:27:41 +0100 Subject: [Python-Dev] iterator API in Py3.0 References: <015901c6395d$3b9fd4b0$2e2a960a@RaymondLaptop1><43FFA4B2.7060303@canterbury.ac.nz><4402F6A9.1030906@gmail.com><4403E263.1060505@canterbury.ac.nz><440411E1.6040401@gmail.com><4404D747.2050402@canterbury.ac.nz><012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1><44052636.9090709@canterbury.ac.nz><004001c63cf3$043fd710$6a01a8c0@RaymondLaptop1><20060303191512.GA22242@panix.com> <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1> Message-ID: Raymond Hettinger wrote: > When teaching your classes, do you sense an aversion to using double > underscore methods in regular code? I sense an implied message that > these methods are not intended to be called directly (i.e. the discomfort > of typing x.__setitem__(k,v) serves as a cue to write x[k]=v instead; > likewise, x.__int__() pushes towards int(x) instead). the "if called magically, it should be __xxx__" rule is pretty worthless on its own; you also need to answer the questions "called by whom ?" and "never called by application code ?" (from the sound of it, one could suspect that some posters here would like all template methods in all standard library components to use the __xxx__ form). From anthony at interlink.com.au Sat Mar 4 09:49:31 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Sat, 4 Mar 2006 19:49:31 +1100 Subject: [Python-Dev] Bug in from __future__ processing? In-Reply-To: <1f7befae0603032034q41a04d49p597c809ecccfeee7@mail.gmail.com> References: <5C0A6F919D675745BB1DBA7412DB68F501BA896B@df-foxhound-msg.exchange.corp.microsoft.com> <1f7befae0603032034q41a04d49p597c809ecccfeee7@mail.gmail.com> Message-ID: <200603041949.34282.anthony@interlink.com.au> On Saturday 04 March 2006 15:34, Tim Peters wrote: > Indeed! But whose arm could we twist to get them to repair the > compiler in 2.4? I'd settle for a blurb in the next 2.4 NEWS just > noting that 2.5 will follow the documented syntax. That may even > be desirable, to avoid breaking working (albeit by accident) code > across a micro release. I don't think a change like this should go into a 2.4.x release. It stands a very very high chance of breaking someone's code. I _could_ be convinced about a warning being emitted about it, though I'm not going to have the time to figure out the new compiler to do the work. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From ncoghlan at gmail.com Sat Mar 4 10:46:56 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 04 Mar 2006 19:46:56 +1000 Subject: [Python-Dev] __exit__ API? In-Reply-To: References: Message-ID: <44096210.4070609@gmail.com> Guido van Rossum wrote: > A few days ago there were rumbling noises that requiring __exit__ to > re-raise the exception (as I amended PEP 343 at the time) could lead > to easily-missed bugs in __exit__ handlers. > > After thinking it over I think I agree and I think I'd like to change > the API so that the exception is only ignored if __exit__ returns a > "true" value. > > The easiest implementation is probably to just let the WITH_CLEANUP > opcode do everything. This becomes a rather heavy opcode then but the > alternative is to generate very hairy code (like the original patch > did, full of ROT 4 choruses). > > Any objections? I probably won't get to this until Monday. Sounds good to me - I believe the majority of explicit __exit__ handlers will be of a try/finally nature rather than try/except, and that situations where the exception is going to be suppressed are better handled using @contextmanager. The solution you propose means that __exit__ methods *can* suppress exceptions if they want to, but their default behaviour will still do the right thing. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From mal at egenix.com Fri Mar 3 23:39:54 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 03 Mar 2006 23:39:54 +0100 Subject: [Python-Dev] FrOSCon 2006 - Call for {Papers|Projects} Message-ID: <4408C5BA.8090900@egenix.com> Is anyone interested in having a Python track at this conference ? PS: The EuroPython conference takes place in Geneva, one week later. -------- Original Message -------- Subject: FrOSCon 2006 - Call for {Papers|Projects} Date: Fri, 03 Mar 2006 16:39:08 +0100 From: Sebastian Bergmann Organization: Free and Open Source Software Conference - Bonn/Rhein-Sieg, Germany FrOSCon is a two-day conference on free software and open source, which takes place on 24th and 25th June 2006 at the University of Applied Sciences Bonn-Rhein-Sieg, in St. Augustin near Bonn, Germany. Focus of the conference is a comprehensive range of talks about current topics in free software and open source. Furthermore, space will be provided for developers of free software and open source projects to organize their own developer meetings or even their own program. FrOSCon is organized for the first time in 2006 by the department of computer science in collaboration with the Linux/Unix User Group Sankt Augustin, the student body and the FrOSCon e.V., and aims to establish itself as the largest event of its kind in Rhineland. === Projects === Successful projects form the backbone of the open source and free software movements. FrOSCon wants to acknowledge this important role of projects for the community by offering special facilities to individual projects. === Developer Rooms === Open source and free software projects are often coordinated primarily via the internet, via email, the web and IRC. Nonetheless, personal contact between members of the project is of paramount importance. A conference like FrOSCon can serve as a meeting place, where distant members of the project can come together and meet. FrOSCon wants to offer dedicated developer rooms to individual projects, which are organized by the projects themselves. Thereby, we provide a space for projects to hold special-interest talks or meetings, to congregate, exchange views and to socialize. We have rooms in different sizes suitable for small developer meetings up to sub-conferences. We offer Internet access in all developer rooms. Besides tables and chairs, most rooms are or can be equipped with a video beamer. If necessary, an additional lecture hall can be assigned for talks or presentations where a larger audience is expected. === Application === Please send applications for a developer room via email to projects at froscon.org. Please include at least the name of your project, the URL of your website and the expected number of participants in your email. The Call for Projects ends on March 31st. === Call for Papers === Please note that our Call for Papers is still open until March 15th. Don't miss the chance to submit a talk. You can find all the details about the Call for Papers here: http://www.froscon.org/wiki/CallforPapers === Booth Space === If your project wants to man a booth at FrOSCon, feel free to contact projects at froscon.org as well. We are looking forward to hearing from you. Kind regards, The FrOSCon Team -- Sebastian Bergmann http://www.sebastian-bergmann.de/ GnuPG Key: 0xB85B5D69 / 27A7 2B14 09E4 98CD 6277 0E5B 6867 C514 B85B 5D69 -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 03 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From skip at pobox.com Sat Mar 4 15:44:30 2006 From: skip at pobox.com (skip at pobox.com) Date: Sat, 4 Mar 2006 08:44:30 -0600 Subject: [Python-Dev] Outdated Python Info on www.unicode.org (fwd) Message-ID: <17417.42958.864391.389414@montanaro.dyndns.org> Someone here (Martin, MAL, /F?) would probably be able to write a short, authoritative description of Python's Unicode powers. Any takers? Skip -------------- next part -------------- An embedded message was scrubbed... From: Dennis Benzinger Subject: Outdated Python Info on www.unicode.org Date: Fri, 03 Mar 2006 20:22:51 +0100 Size: 4635 Url: http://mail.python.org/pipermail/python-dev/attachments/20060304/472b117e/attachment.mht From skip at pobox.com Sat Mar 4 16:12:35 2006 From: skip at pobox.com (skip at pobox.com) Date: Sat, 4 Mar 2006 09:12:35 -0600 Subject: [Python-Dev] My buildbot host upgraded to OSX 10.4 Message-ID: <17417.44643.515636.179693@montanaro.dyndns.org> I upgraded my PowerMac G5 to OSX 10.4. ZopeInterface isn't recompiling after the upgrade to XCode 2.1. You probably ought to just pull "g5 osx.3" from the buildbot host list, certainly until I have time to figure this problem out. (Probably after as well. My machine configuration is now not substantially different from the loaner XServer.) Skip From skip at pobox.com Sat Mar 4 18:17:58 2006 From: skip at pobox.com (skip at pobox.com) Date: Sat, 4 Mar 2006 11:17:58 -0600 Subject: [Python-Dev] My buildbot host upgraded to OSX 10.4 In-Reply-To: <17417.44643.515636.179693@montanaro.dyndns.org> References: <17417.44643.515636.179693@montanaro.dyndns.org> Message-ID: <17417.52166.152446.783116@montanaro.dyndns.org> skip> I upgraded my PowerMac G5 to OSX 10.4. ZopeInterface isn't skip> recompiling after the upgrade to XCode 2.1. Okay, my buildbot slave is back online. Still, feel free to kick it out if you think it's not worth it to continue running it. Skip From martin at v.loewis.de Sat Mar 4 18:23:36 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 04 Mar 2006 18:23:36 +0100 Subject: [Python-Dev] My buildbot host upgraded to OSX 10.4 In-Reply-To: <17417.44643.515636.179693@montanaro.dyndns.org> References: <17417.44643.515636.179693@montanaro.dyndns.org> Message-ID: <4409CD18.9080109@v.loewis.de> skip at pobox.com wrote: > I upgraded my PowerMac G5 to OSX 10.4. ZopeInterface isn't recompiling > after the upgrade to XCode 2.1. You probably ought to just pull "g5 osx.3" > from the buildbot host list, certainly until I have time to figure this > problem out. (Probably after as well. My machine configuration is now not > substantially different from the loaner XServer.) Ok, I have now removed your machine from the list of slaves. Thanks for your contribution in getting this started! Martin From annaraven at gmail.com Sat Mar 4 18:34:50 2006 From: annaraven at gmail.com (Anna Ravenscroft) Date: Sat, 4 Mar 2006 09:34:50 -0800 Subject: [Python-Dev] iterator API in Py3.0 In-Reply-To: <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1> References: <015901c6395d$3b9fd4b0$2e2a960a@RaymondLaptop1> <4403E263.1060505@canterbury.ac.nz> <440411E1.6040401@gmail.com> <4404D747.2050402@canterbury.ac.nz> <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> <44052636.9090709@canterbury.ac.nz> <004001c63cf3$043fd710$6a01a8c0@RaymondLaptop1> <20060303191512.GA22242@panix.com> <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1> Message-ID: On 3/3/06, Raymond Hettinger wrote: > > [Aahz] > > * As a writer/teacher, I want to be able to say, "All Python special > > methods have leading and trailing double-underscore." Period, end of > > story. > > When teaching your classes, do you sense an aversion to using double > underscore > methods in regular code? I sense an implied message that these methods > are not > intended to be called directly (i.e. the discomfort of typing > x.__setitem__(k,v) > serves as a cue to write x[k]=v instead; likewise, x.__int__() pushes > towards > int(x) instead). > > If so, then that is a good reason to leave it.next() as-is. Unlike > __setitem__ > or __int__, the next() method is intended to be called directly in normal > code. > Of course, for-loops are the most common case and it makes no difference > there; > however, in the rest of the cases where the iterator is accessed directly, > the > current naming is clean, readable, and doesn't provide an aversive cue. > > The double underscore convention is appropriate where the method is always > invoked magically in normal code and not called directly. The next() > method is > differenct because it is a mixed case, sometimes called magically and > sometimes > called directly. In the latter case, the name is highly visible and > therefore > should not have double underscores. > > I suspect that those who feel differently are ones who usually avoid > calling > next() directly. That's okay, but we shouldn't muck-up the naming for the > rest > of us who often do have a need to use next(). > > This is doubly important because we're now expanding the protocol to > include > send() and throw(). Adding underscores around them too will only make > those > methods look harder to use than they actually are. Don't underestimate > the > psychological revulsion to calling code filled with piles of double > underscores. I think this is a really good point. next() is supposed to get used, by coders, in regular code - so it shouldn't be __next__. I can understand the desire for both forms, although that seems it would clutter things up unnecessarily - particularly if the two do the same thing. Anna -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060304/7ed63cdd/attachment.html From pje at telecommunity.com Sat Mar 4 21:45:03 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 04 Mar 2006 15:45:03 -0500 Subject: [Python-Dev] iterator API in Py3.0 In-Reply-To: References: <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1> <015901c6395d$3b9fd4b0$2e2a960a@RaymondLaptop1> <4403E263.1060505@canterbury.ac.nz> <440411E1.6040401@gmail.com> <4404D747.2050402@canterbury.ac.nz> <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> <44052636.9090709@canterbury.ac.nz> <004001c63cf3$043fd710$6a01a8c0@RaymondLaptop1> <20060303191512.GA22242@panix.com> <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1> Message-ID: <5.1.1.6.0.20060304154338.01ddd940@mail.telecommunity.com> At 09:34 AM 3/4/2006 -0800, Anna Ravenscroft wrote: >I think this is a really good point. next() is supposed to get used, by >coders, in regular code - so it shouldn't be __next__. I can understand >the desire for both forms, although that seems it would clutter things up >unnecessarily - particularly if the two do the same thing. By this argument, we should be using ob.len() instead of len(ob), and ob.iter() instead of iter(ob). From phd at mail2.phd.pp.ru Sat Mar 4 22:05:49 2006 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Sun, 5 Mar 2006 00:05:49 +0300 Subject: [Python-Dev] iterator API in Py3.0 In-Reply-To: <5.1.1.6.0.20060304154338.01ddd940@mail.telecommunity.com> References: <4403E263.1060505@canterbury.ac.nz> <440411E1.6040401@gmail.com> <4404D747.2050402@canterbury.ac.nz> <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> <44052636.9090709@canterbury.ac.nz> <004001c63cf3$043fd710$6a01a8c0@RaymondLaptop1> <20060303191512.GA22242@panix.com> <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1> <5.1.1.6.0.20060304154338.01ddd940@mail.telecommunity.com> Message-ID: <20060304210549.GA10150@phd.pp.ru> On Sat, Mar 04, 2006 at 03:45:03PM -0500, Phillip J. Eby wrote: > At 09:34 AM 3/4/2006 -0800, Anna Ravenscroft wrote: > >I think this is a really good point. next() is supposed to get used, by > >coders, in regular code - so it shouldn't be __next__. I can understand > >the desire for both forms, although that seems it would clutter things up > >unnecessarily - particularly if the two do the same thing. > > By this argument, we should be using ob.len() instead of len(ob), and > ob.iter() instead of iter(ob). Yes, I think it'd be more consistent and more object-oriented. After all we've switched from string.split(x, y) to x.split(y)... Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From raymond.hettinger at verizon.net Sat Mar 4 22:11:17 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sat, 04 Mar 2006 16:11:17 -0500 Subject: [Python-Dev] iterator API in Py3.0 References: <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1> <015901c6395d$3b9fd4b0$2e2a960a@RaymondLaptop1> <4403E263.1060505@canterbury.ac.nz> <440411E1.6040401@gmail.com> <4404D747.2050402@canterbury.ac.nz> <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> <44052636.9090709@canterbury.ac.nz> <004001c63cf3$043fd710$6a01a8c0@RaymondLaptop1> <20060303191512.GA22242@panix.com> <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1> <5.1.1.6.0.20060304154338.01ddd940@mail.telecommunity.com> Message-ID: <00ad01c63fd0$2e4c2f10$6a01a8c0@RaymondLaptop1> [Anna Ravenscroft] >>I think this is a really good point. next() is supposed to get used, by >>coders, in regular code - so it shouldn't be __next__. I can understand the >>desire for both forms, although that seems it would clutter things up >>unnecessarily - particularly if the two do the same thing. [Phillip Eby] > By this argument, we should be using ob.len() instead of len(ob), and > ob.iter() instead of iter(ob). Hey, no fair misstating her argument and then reducing it to the absurd. We're all aware that __len__ and __iter__ are supported by a wide variety of object types and are accessed through builtin functions provided expressly for that purpose. In contrast, we're all aware that next() applies only iterator objects, that it was designed for both direct and magic invocation, and that there is no corresponding builtin. This conversation is getting goofy. There is no underlying problem to be solved. Essentially, a couple of developers feel irritated by a perceived aberration from a naming convention. To assuage that irritation, they are willing to either 1) inflict pain on direct callers by cluttering the calling code with pairs of double underscores, or 2) add yet another builtin (needlessly bloating the namespace, adding another level of indirection, and burdening execution with an unnecessary global lookup). The cure is worse than the disease. IMO, the appropriate solution is to amend your understanding of the naming convention to only apply to methods whose normal invocation is exclusively magic and not called directly. Raymond From pje at telecommunity.com Sat Mar 4 22:27:15 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 04 Mar 2006 16:27:15 -0500 Subject: [Python-Dev] iterator API in Py3.0 In-Reply-To: <00ad01c63fd0$2e4c2f10$6a01a8c0@RaymondLaptop1> References: <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1> <015901c6395d$3b9fd4b0$2e2a960a@RaymondLaptop1> <4403E263.1060505@canterbury.ac.nz> <440411E1.6040401@gmail.com> <4404D747.2050402@canterbury.ac.nz> <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> <44052636.9090709@canterbury.ac.nz> <004001c63cf3$043fd710$6a01a8c0@RaymondLaptop1> <20060303191512.GA22242@panix.com> <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1> <5.1.1.6.0.20060304154338.01ddd940@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060304161723.01e05c30@mail.telecommunity.com> At 04:11 PM 3/4/2006 -0500, Raymond Hettinger wrote: >[Anna Ravenscroft] >>>I think this is a really good point. next() is supposed to get used, by >>>coders, in regular code - so it shouldn't be __next__. I can understand >>>the desire for both forms, although that seems it would clutter things >>>up unnecessarily - particularly if the two do the same thing. > >[Phillip Eby] >>By this argument, we should be using ob.len() instead of len(ob), and >>ob.iter() instead of iter(ob). > >Hey, no fair misstating her argument and then reducing it to the >absurd. We're all aware that __len__ and __iter__ are supported by a wide >variety of object types and are accessed through builtin functions >provided expressly for that purpose. In contrast, we're all aware that >next() applies only iterator objects, that it was designed for both direct >and magic invocation, and that there is no corresponding builtin. I didn't misstate her argument or reduce it to the absurd. I simply applied that argument consistently to similar features of Python. It's you who is concluding that this results in absurdity; I made no such conclusion. I'm simply pointing out that in 3.0 we should be consistent. Either we should have ob.iter() and ob.len(), or else we should have a builtin next(). >This conversation is getting goofy. There is no underlying problem to be >solved. Essentially, a couple of developers feel irritated by a perceived >aberration from a naming convention. To assuage that irritation, they are >willing to either 1) inflict pain on direct callers by cluttering the >calling code with pairs of double underscores, or 2) add yet another >builtin (needlessly bloating the namespace, adding another level of >indirection, and burdening execution with an unnecessary global lookup). You have it backwards. If those things are so important as to be the deciding factor, we should add .len() and .iter() methods in 3.0 and use them instead of the len() and iter() functions. Personally, I prefer functions to methods for these kinds of language-defined operations, but that has nothing to do with the issue of consistency. As far as I can understand your position, you seem to be arguing that whatever we have now is correct, and therefore changing it would be wrong. That is, since we use .next() now, this is correct by definition and a change to next() + __next__ would therefore be wrong. And *that* is the only "goofy" thing I see going on here. :) From pje at telecommunity.com Sat Mar 4 22:31:26 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 04 Mar 2006 16:31:26 -0500 Subject: [Python-Dev] iterator API in Py3.0 In-Reply-To: <20060304210549.GA10150@phd.pp.ru> References: <5.1.1.6.0.20060304154338.01ddd940@mail.telecommunity.com> <4403E263.1060505@canterbury.ac.nz> <440411E1.6040401@gmail.com> <4404D747.2050402@canterbury.ac.nz> <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> <44052636.9090709@canterbury.ac.nz> <004001c63cf3$043fd710$6a01a8c0@RaymondLaptop1> <20060303191512.GA22242@panix.com> <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1> <5.1.1.6.0.20060304154338.01ddd940@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060304162722.03ec7618@mail.telecommunity.com> At 12:05 AM 3/5/2006 +0300, Oleg Broytmann wrote: >On Sat, Mar 04, 2006 at 03:45:03PM -0500, Phillip J. Eby wrote: > > At 09:34 AM 3/4/2006 -0800, Anna Ravenscroft wrote: > > >I think this is a really good point. next() is supposed to get used, by > > >coders, in regular code - so it shouldn't be __next__. I can understand > > >the desire for both forms, although that seems it would clutter things up > > >unnecessarily - particularly if the two do the same thing. > > > > By this argument, we should be using ob.len() instead of len(ob), and > > ob.iter() instead of iter(ob). > > Yes, I think it'd be more consistent and more object-oriented. I'm not sure that "more object-oriented" should be equated with "good" in this context, or indeed any context. :) A function is no more or less polymorphic than a method in any case, especially if the function is normally delegating to a slot or special method in any case. From jcarlson at uci.edu Sun Mar 5 00:52:47 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat, 04 Mar 2006 15:52:47 -0800 Subject: [Python-Dev] Another Windows buildbot slave Message-ID: <20060304155215.F696.JCARLSON@uci.edu> It took me a while to get it set up, but we've got another Windows XP buildbot slave running. - Josiah From jcarlson at uci.edu Sun Mar 5 01:02:37 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat, 04 Mar 2006 16:02:37 -0800 Subject: [Python-Dev] Another Windows buildbot slave In-Reply-To: <20060304155215.F696.JCARLSON@uci.edu> References: <20060304155215.F696.JCARLSON@uci.edu> Message-ID: <20060304160221.F699.JCARLSON@uci.edu> Josiah Carlson wrote: > It took me a while to get it set up, but we've got another Windows XP > buildbot slave running. Spoke too soon, not quite ready yet. - Josiah From pje at telecommunity.com Sun Mar 5 03:03:26 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 04 Mar 2006 21:03:26 -0500 Subject: [Python-Dev] iterator API in Py3.0 In-Reply-To: <001001c63fd4$dd4ac7c0$6a01a8c0@RaymondLaptop1> References: <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1> <015901c6395d$3b9fd4b0$2e2a960a@RaymondLaptop1> <4403E263.1060505@canterbury.ac.nz> <440411E1.6040401@gmail.com> <4404D747.2050402@canterbury.ac.nz> <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> <44052636.9090709@canterbury.ac.nz> <004001c63cf3$043fd710$6a01a8c0@RaymondLaptop1> <20060303191512.GA22242@panix.com> <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1> <5.1.1.6.0.20060304154338.01ddd940@mail.telecommunity.com> <5.1.1.6.0.20060304161723.01e05c30@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060304205438.03ecc910@mail.telecommunity.com> At 04:44 PM 3/4/2006 -0500, Raymond Hettinger wrote: >> As far as I can understand your position, you seem to be arguing that >> whatever we have now is correct, and therefore changing it would be wrong. > >Yes, that's pretty much it. IMO, introducing __builtin__.next() would >suck. The sublime feeling of consistency isn't worth the cost of an extra >builtin and adding an unnecessary global lookup to an otherwise >lightweight inner-loop construct. You mean, versus an unnecessary attribute lookup? :) Guido has stated many times that future versions of Python should be able to optimize references to builtins so that the global cost isn't required. Optimization of attribute lookups is a bit more speculative at this point, and can currently involve *more* dictionary lookups than a global lookup, depending on the inheritance depth. IIRC, looking for a 'next' method on a new-style object requires a minimum of two dictionary accesses already: one to look in the class, and one in the instance to make sure it's not shadowed. In contrast, __next__() could be a fast slot access, since slots are copied directly to subclasses. (But neither the performance argument nor the "extra builtin" argument make sense to me anyway, since they would both apply equally to len() and iter().) From fredrik at pythonware.com Sun Mar 5 14:06:03 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sun, 5 Mar 2006 14:06:03 +0100 Subject: [Python-Dev] iterator API in Py3.0 References: <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1><015901c6395d$3b9fd4b0$2e2a960a@RaymondLaptop1><4403E263.1060505@canterbury.ac.nz> <440411E1.6040401@gmail.com><4404D747.2050402@canterbury.ac.nz><012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1><44052636.9090709@canterbury.ac.nz><004001c63cf3$043fd710$6a01a8c0@RaymondLaptop1><20060303191512.GA22242@panix.com><002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1><5.1.1.6.0.20060304154338.01ddd940@mail.telecommunity.com> <00ad01c63fd0$2e4c2f10$6a01a8c0@RaymondLaptop1> Message-ID: Raymond Hettinger wrote: > This conversation is getting goofy. indeed. let's pray that nobody that is considering picking up Python sees this thread. From phd at mail2.phd.pp.ru Sun Mar 5 15:44:17 2006 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Sun, 5 Mar 2006 17:44:17 +0300 Subject: [Python-Dev] .len() instead of __len__() (was: iterator API in Py3.0) Message-ID: <20060305144417.GB31296@phd.pp.ru> Hello! On Sat, Mar 04, 2006 at 04:31:26PM -0500, Phillip J. Eby wrote: > I'm not sure that "more object-oriented" should be equated with "good" in > this context, or indeed any context. :) I am sure it is. > A function is no more or less > polymorphic than a method in any case, especially if the function is > normally delegating to a slot or special method in any case. Advantages of a method over a function: -- a method is a more natural way to query or manipulate objects; -- a direct method call is faster than two-level call (len() => .__len__()); -- unnecessary cluttering of builtin namespace; the namespace should(1) contains only really global functions (__import__(), eval(), etc.) What are the advantages of a function over a method? What are disadvantages of a direct .len() instead of .__len__()? Footnote: (1) I am a perfectionist and I partially disagree with "practicality beats purity"; I would really love a small uncluttered builtin namespace in Python. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From tds333+2006 at gmail.com Wed Mar 1 10:12:49 2006 From: tds333+2006 at gmail.com (Wolfgang Langner) Date: Wed, 1 Mar 2006 10:12:49 +0100 Subject: [Python-Dev] C++ for CPython 3? (Re: str.count is slow) In-Reply-To: <440551F2.4030702@denviso.de> References: <1141073696.107136.318600@j33g2000cwa.googlegroups.com> <1141083127.970403.147100@v46g2000cwv.googlegroups.com> <4403EFEE.80002@canterbury.ac.nz> <4404D79D.2070608@canterbury.ac.nz> <440551F2.4030702@denviso.de> Message-ID: <4c45c1530603010112u594e680fo970ab0798e28d72b@mail.gmail.com> Hello, > >should we perhaps switch to (careful use of) C++ in 3.0 ? > I can't see many advantages in moving to C++, but a lot of disadvantages: > > - Size increase, especially when we start using templates > - Performance decrease > - Problems with name mangling together with dynamic loading and cross > module API's > - Everything has to be build with the same compiler, binaries created > with different compilers can't interoperate > - Possibly all extensions modules have to be (re)written in C++ > - Moving to C++ will change Python's well known API substantially Same opinion. C++ is evil, please don't use it. You get a lot of new problems and nearly not benefit. Better go to jython or consider the way of pypy. -- bye by Wolfgang From talin at maxis.com Wed Mar 1 19:59:44 2006 From: talin at maxis.com (Talin) Date: Wed, 01 Mar 2006 10:59:44 -0800 Subject: [Python-Dev] Slightly OT: Replying to posts Message-ID: <4405EF20.8090007@maxis.com> Just a quick question about the mechanics of replying to this list. I am a subscriber to the list, however I much prefer readiing the list archives on the web instead of having the postings delivered to my email account. Because of this, I have delivery turned off in the mailing list preferences. I particularly dislike the idea of wasting bandwidth and disk space on something that I am not going to read. However, I would like to be able to reply to posts in such a way as to have them appear in the appropriate place in the thread hierarchy. Since I don't have the original email, I cannot reply to it directly; instead I have to create a new, non-reply email and send it to the list. Simply editing the subject line to put "Re: subject" would seem to be insufficient. Does anyone have a trick for managing this? Or is there a FAQ that someone can point me to that addresses this issue? -- Talin From baptiste13 at altern.org Sun Mar 5 19:43:47 2006 From: baptiste13 at altern.org (Baptiste Carvello) Date: Sun, 05 Mar 2006 19:43:47 +0100 Subject: [Python-Dev] iterator API in Py3.0 In-Reply-To: <5.1.1.6.0.20060304161723.01e05c30@mail.telecommunity.com> References: <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1> <015901c6395d$3b9fd4b0$2e2a960a@RaymondLaptop1> <4403E263.1060505@canterbury.ac.nz> <440411E1.6040401@gmail.com> <4404D747.2050402@canterbury.ac.nz> <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> <44052636.9090709@canterbury.ac.nz> <004001c63cf3$043fd710$6a01a8c0@RaymondLaptop1> <20060303191512.GA22242@panix.com> <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1> <5.1.1.6.0.20060304154338.01ddd940@mail.telecommunity.com> <00ad01c63fd0$2e4c2f10$6a01a8c0@RaymondLaptop1> <5.1.1.6.0.20060304161723.01e05c30@mail.telecommunity.com> Message-ID: Phillip J. Eby a ?crit : > I didn't misstate her argument or reduce it to the absurd. I simply > applied that argument consistently to similar features of Python. It's you > who is concluding that this results in absurdity; I made no such > conclusion. I'm simply pointing out that in 3.0 we should be > consistent. Either we should have ob.iter() and ob.len(), or else we > should have a builtin next(). > There is a difference though: a next() builtin would change the state of its argument, what len() ant iter() don't do. I usually prefer methods to functions for operations that modify an object. But then we have setattr and delattr :) I'd say it makes sense to go for practicallity here. Let's keep the next() method as it is, and document this as an exception. The canonical place could be section 3.3 in the language reference, but for some reason, it does not list next() right now... Cheers, BC From nas at arctrix.com Sun Mar 5 19:48:22 2006 From: nas at arctrix.com (Neil Schemenauer) Date: Sun, 5 Mar 2006 18:48:22 +0000 (UTC) Subject: [Python-Dev] .len() instead of __len__() (was: iterator API in Py3.0) References: <20060305144417.GB31296@phd.pp.ru> Message-ID: Oleg Broytmann wrote: > What are disadvantages of a direct .len() instead of .__len__()? I can think of a few arguments against getting rid of double underscores in general. First, special methods are a little like keywords in that it would be nice to introduce new ones from time to time without breaking existing code. For example, using getattribute instead of __getattribute__ could have broken existing code. Another advantage to using __getattribute__ is that it's normally a method that users don't have to know about. Even if there was no problem with breaking existing code it still would be better to use underscores. Someone who doesn't yet know about the magic method could become confused when they use the name for some other purpose. Finally, there is an advantage to consistent naming. Since some special methods have double underscores, one could argue that they all should have them. Neil From nas at arctrix.com Sun Mar 5 19:53:38 2006 From: nas at arctrix.com (Neil Schemenauer) Date: Sun, 5 Mar 2006 18:53:38 +0000 (UTC) Subject: [Python-Dev] Slightly OT: Replying to posts References: <4405EF20.8090007@maxis.com> Message-ID: Talin wrote: > However, I would like to be able to reply to posts in such a way as to > have them appear in the appropriate place in the thread hierarchy. The message should have a References header that contains the Message-Id of the message that you are responding to. > Does anyone have a trick for managing this? Or is there a FAQ that > someone can point me to that addresses this issue? You could use gmane.org and read the list using NNTP. Neil From guido at python.org Sun Mar 5 20:04:55 2006 From: guido at python.org (Guido van Rossum) Date: Sun, 5 Mar 2006 11:04:55 -0800 Subject: [Python-Dev] Bug in from __future__ processing? In-Reply-To: <200603041949.34282.anthony@interlink.com.au> References: <5C0A6F919D675745BB1DBA7412DB68F501BA896B@df-foxhound-msg.exchange.corp.microsoft.com> <1f7befae0603032034q41a04d49p597c809ecccfeee7@mail.gmail.com> <200603041949.34282.anthony@interlink.com.au> Message-ID: On 3/4/06, Anthony Baxter wrote: > On Saturday 04 March 2006 15:34, Tim Peters wrote: > > Indeed! But whose arm could we twist to get them to repair the > > compiler in 2.4? I'd settle for a blurb in the next 2.4 NEWS just > > noting that 2.5 will follow the documented syntax. That may even > > be desirable, to avoid breaking working (albeit by accident) code > > across a micro release. > > I don't think a change like this should go into a 2.4.x release. It > stands a very very high chance of breaking someone's code. I _could_ > be convinced about a warning being emitted about it, though I'm not > going to have the time to figure out the new compiler to do the work. In fact I know for sure it will break code -- I believe I first ran into this with Django. Let's leave 2.4 alone (even adding a warning to 2.4.3 seems pointless) except for documenting this (not just in Misc/NEWS which is overwhelming but in the 2.4 docs for _future-_). In 2.5 it's already fixed; perhaps the 2.5 docs could mention that this was accidentally broken in 2.4. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From phd at mail2.phd.pp.ru Sun Mar 5 20:17:34 2006 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Sun, 5 Mar 2006 22:17:34 +0300 Subject: [Python-Dev] x.len() instead of len(x) in Py3.0 In-Reply-To: References: <20060305144417.GB31296@phd.pp.ru> Message-ID: <20060305191734.GA11420@phd.pp.ru> On Sun, Mar 05, 2006 at 06:48:22PM +0000, Neil Schemenauer wrote: > Oleg Broytmann wrote: > > What are disadvantages of a direct .len() instead of .__len__()? > > I can think of a few arguments against getting rid of double > underscores in general. The discussion is about moving from len(sequence) to sequence.len(), not from __len__() to .len(). Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From krumms at gmail.com Mon Mar 6 00:15:35 2006 From: krumms at gmail.com (Thomas Lee) Date: Mon, 06 Mar 2006 09:15:35 +1000 Subject: [Python-Dev] New Future Keywords Message-ID: <440B7117.3010009@gmail.com> Hi, Just a quick question: how can I add new future keywords to Python? I need to add a new (Python) keyword to the language, but there seems to be a few different source files that I need to modify. Just want to make sure I'm doing it the right way before I go unleashing a nasty broken patch into the world :) Cheers, Tom From thomas at python.org Mon Mar 6 00:39:26 2006 From: thomas at python.org (Thomas Wouters) Date: Mon, 6 Mar 2006 00:39:26 +0100 Subject: [Python-Dev] New Future Keywords In-Reply-To: <440B7117.3010009@gmail.com> References: <440B7117.3010009@gmail.com> Message-ID: <9e804ac0603051539i146cae0dj79bd896daf7fac4a@mail.gmail.com> On 3/6/06, Thomas Lee wrote: > Just a quick question: how can I add new future keywords to Python? I > need to add a new (Python) keyword to the language, but there seems to > be a few different source files that I need to modify. > > Just want to make sure I'm doing it the right way before I go unleashing > a nasty broken patch into the world :) I wouldn't bother with a future statement until your patch has a chance of being included in Python. Adding one later is just as easy as adding one now, and you have a good chance of getting someone else to do it for you. That's what Guido did with the with-statement patch; he got Neal to do it for him ;) But if you really want to do it yourself, I would suggest grokking the with-statement future-import patch I checked in last week (written mostly by Neal Norwitz, with inspiration stolen from Michael Hudson's patch): svn diff -r42666:42667 http://svn.python.org/projects/python/trunk -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060306/7356e9cb/attachment.htm From thomas at python.org Mon Mar 6 02:45:51 2006 From: thomas at python.org (Thomas Wouters) Date: Mon, 6 Mar 2006 02:45:51 +0100 Subject: [Python-Dev] [Python-checkins] Python Regression Test Failures refleak (1) In-Reply-To: <20060305215409.GA7545@python.psfb.org> References: <20060305215409.GA7545@python.psfb.org> Message-ID: <9e804ac0603051745i456a822cv4089aba6c3c8fc3d@mail.gmail.com> On 3/5/06, Neal Norwitz wrote: > test_generators leaked [255, 255, 255] references 254 of those 255 leaks seem to be caused by the two recursive-generator examples at the bottom of the fun_tests doctest: fib() and the last m235() implementation. The recursiveness isn't in the generators themselves, but rather in the use of their own return values through a nested scope -- the cell objects for those values hold tee-iterators which hold (through a tee-dataobject) a reference to the generator. It doesn't look like tee-objects participate in GC, so there's no way to break the cycle. I added a quick 'close' implementation to itertools.tee, which just calls the 'close' method of whatever iterator it is tee'ing, and that does solve the issue. Considering the fairly convoluted use of generators, I guess the latter solution is acceptable (although it would be nice if tee objects did do GC, at least enough so to let the GC do its work. I haven't looked at howmuch effort that would take.) The last leak is somewhere in the coroutines_tests doctest, but I haven't taken the time to figure out where, exactly. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060306/d3b2f449/attachment.html From fredrik at pythonware.com Mon Mar 6 07:17:00 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Mon, 6 Mar 2006 07:17:00 +0100 Subject: [Python-Dev] Bug in from __future__ processing? References: <5C0A6F919D675745BB1DBA7412DB68F501BA896B@df-foxhound-msg.exchange.corp.microsoft.com><1f7befae0603032034q41a04d49p597c809ecccfeee7@mail.gmail.com><200603041949.34282.anthony@interlink.com.au> Message-ID: Guido van Rossum wrote: > > I don't think a change like this should go into a 2.4.x release. It > > stands a very very high chance of breaking someone's code. I _could_ > > be convinced about a warning being emitted about it, though I'm not > > going to have the time to figure out the new compiler to do the work. > > In fact I know for sure it will break code -- I believe I first ran > into this with Django. Cheetah ? http://www.artima.com/weblogs/viewpost.jsp?thread=146606 > Let's leave 2.4 alone (even adding a warning to 2.4.3 seems pointless) > except for documenting this (not just in Misc/NEWS which is > overwhelming but in the 2.4 docs for _future-_). In 2.5 it's already > fixed; perhaps the 2.5 docs could mention that this was accidentally > broken in 2.4. agreed. From stephen at xemacs.org Mon Mar 6 10:54:13 2006 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 06 Mar 2006 18:54:13 +0900 Subject: [Python-Dev] C++ for CPython 3? (Re: str.count is slow) In-Reply-To: <200603040938.11052.anthony@interlink.com.au> (Anthony Baxter's message of "Sat, 4 Mar 2006 09:38:10 +1100") References: <1141073696.107136.318600@j33g2000cwa.googlegroups.com> <87r75k924v.fsf@tleepslib.sk.tsukuba.ac.jp> <1141361106.4407c9d2b6b46@www.domainfactory-webmail.de> <200603040938.11052.anthony@interlink.com.au> Message-ID: <87y7zn3nlm.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Anthony" == Anthony Baxter writes: Anthony> It's probably worth mentioning that right now, we don't Anthony> even come close to compiling with a C++ compiler. A bunch Anthony> of the bugs are shallow (casting result from malloc, that Anthony> sort of thing) but a bunch more look a tad uglier. Is Anthony> this something worth trying to fix? Um ... I really only know the results for XEmacs. C++ warnings and errors do catch a number of "real bugs". But most of the *work* was done before I got really involved in the C code. Martin Buchholz probably knows best, although he's been inactive for a couple of years. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From mcherm at mcherm.com Mon Mar 6 14:33:23 2006 From: mcherm at mcherm.com (Michael Chermside) Date: Mon, 06 Mar 2006 05:33:23 -0800 Subject: [Python-Dev] .len() instead of __len__() (was: iterator API inPy3.0) Message-ID: <20060306053323.ngne36ixrawwkw4g@login.werra.lunarpages.com> Oleg Broytmann writes that he thinks methods are inherently "better" than methods. He asks for advantages of a function over a method, after first presenting his arguments for methods over functions: > -- a method is a more natural way to query or manipulate objects; > -- a direct method call is faster than two-level call (len() => .__len__()); > -- unnecessary cluttering of builtin namespace; the namespace should(1) > contains only really global functions (__import__(), eval(), etc.) Responding to your first point, "a more natural way" is not an argument... it's an expression of your personal preference. Your second point (performance) has some merit -- although one would hope the difference is small in a good implementation. And your third point (cluttering the builtin namespace) is a valid and real reason to prefer methods. In defense of functions, my first exhibit is the following snippet of Java code: /** Returns b^2 - 4ac */ public BigDecimal determinant(BigDecimal a, BigDecimal b, BigDecimal c) { return b.multiply(b).subtract(new BigDecimal(4).multiply(a).multiply(c)); } In other words, some things are less clear when written using the notation used for method invocation. That is particularly true when there are multiple objects involved and no clear reason to prefer one over another as the "object performing the action". In languages with multiple dispatch, like CLOS, everything looks like a function for exactly this reason. My second exhibit is also some Java code: public int numItems(Object o) { if (o instanceof Collection) { // includes List, Set return ((Collection) o).size(); } else if (o instanceof Dictionary) { return ((Dictionary) o).size(); } else if (o instanceof CharSequence) { // includes String, StringBuffer return ((CharSequence) o).length(); } else if (o instanceof Object[]) { return ((Object[]) o).length; } else { throw new IllegalArgumentException(); } } The inconsistancy here is amazing... length, length(), size(). You could put it down to poor interface design by the Java team, but you must admit, individual objects are designed in isolation, but when there exists a common lanugage protocol like Python's builtin len() and __len__, that developers of individual objects go out of their way to make them conform with the common protocol... and in the end that makes them more usable. I'm sure there are additional arguments, but these are the first few I came up with. Of course, in nearly all cases, I prefer using methods... but len() is such a powerful concept applicable across MANY diverse kinds of objects, that I have no objection to granting it space in the builtin namespace. By the way... you write: > (1) I am a perfectionist and I partially disagree with "practicality beats > purity"; I would really love a small uncluttered builtin namespace in > Python. I just wanted to point out that one of the things I *LOVE* about Python is that the design of name resolution in Python ensures that no programmer need suffer from the size of the builtin namespace. If you never use certain builtins, perhaps "hex" or "super", then go ahead and use those as identifiers in your code. Avoid only that portion of the builtin namespace that you feel is worth keeping. -- Michael Chermside From phd at mail2.phd.pp.ru Mon Mar 6 15:19:10 2006 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Mon, 6 Mar 2006 17:19:10 +0300 Subject: [Python-Dev] .len() instead of __len__() in Py3.0 In-Reply-To: <20060306053323.ngne36ixrawwkw4g@login.werra.lunarpages.com> References: <20060306053323.ngne36ixrawwkw4g@login.werra.lunarpages.com> Message-ID: <20060306141910.GA14621@phd.pp.ru> On Mon, Mar 06, 2006 at 05:33:23AM -0800, Michael Chermside wrote: > Oleg Broytmann writes that he thinks methods are inherently "better" than > methods. "...better than *functions*." > In defense of functions, my first exhibit is the following snippet of Java > code: > > /** Returns b^2 - 4ac */ > public BigDecimal determinant(BigDecimal a, BigDecimal b, BigDecimal c) { > return b.multiply(b).subtract(new BigDecimal(4).multiply(a).multiply(c)); > } I never said methods are better than operators. Though to some extent they are. For example, if you want to grep your sources for ".addArray(" you have much better chances to find what you are looking for comparing with the case when you grep for "+=". In the latter case you will find too much. But I do not propose to get rid of operator overloading and always use methods. > public int numItems(Object o) { > if (o instanceof Collection) { // includes List, Set > return ((Collection) o).size(); > } else if (o instanceof Dictionary) { > return ((Dictionary) o).size(); > } else if (o instanceof CharSequence) { // includes String, StringBuffer > return ((CharSequence) o).length(); > } else if (o instanceof Object[]) { > return ((Object[]) o).length; > } else { > throw new IllegalArgumentException(); > } > } > > The inconsistancy here is amazing... The same can easily be written in Python. In absence of builtin function has_key() and documented method __has_key__() one can write class MyDict(object): def hasKey(self, key): ... other can spell it .has_key() or .haskey(). Still those who want to emulate dictionaries (to implement mapping protocol) spell it .has_key(). A real life example. SQLObject has a class SelectResults that partially implements sequence/iterator protocol. But it doesn't have __len__(), instead it has method .count(). Look at the first question at http://svn.colorstudy.com/SQLObject/docs/FAQ.txt , "Why there is no __len__?" So instead of len(sresults) one must write sresults.count(). So even the known protocol "len() => __len__()" doesn't prevent the inconsistency. Sometimes one really needs the inconsistency; in this particular case because Python internally calls len() on objects, and implementing __len__() here would be expensive. And those who want to be consistent are. They spell .has_key() and implement __len__(). And if __len__() becomes .len() they will write .len(), not .size() or .count(). > len() is such a powerful concept applicable across MANY diverse > kinds of objects Only to those already have method __len__(), so why not .len()? > I just wanted to point out that one of the things I *LOVE* about Python > is that the design of name resolution in Python ensures that no > programmer need suffer from the size of the builtin namespace. If you > never use certain builtins, perhaps "hex" or "super", then go ahead > and use those as identifiers in your code. Avoid only that portion of > the builtin namespace that you feel is worth keeping. Doesn't work. For example, PyChecker/PyLint would report warnings. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From rasky at develer.com Mon Mar 6 16:29:44 2006 From: rasky at develer.com (Giovanni Bajo) Date: Mon, 6 Mar 2006 16:29:44 +0100 Subject: [Python-Dev] .len() instead of __len__() in Py3.0 References: <20060306053323.ngne36ixrawwkw4g@login.werra.lunarpages.com> <20060306141910.GA14621@phd.pp.ru> Message-ID: <042601c64132$cb794350$bf03030a@trilan> Oleg Broytmann wrote: >> I just wanted to point out that one of the things I *LOVE* about Python >> is that the design of name resolution in Python ensures that no >> programmer need suffer from the size of the builtin namespace. If you >> never use certain builtins, perhaps "hex" or "super", then go ahead >> and use those as identifiers in your code. Avoid only that portion of >> the builtin namespace that you feel is worth keeping. > > Doesn't work. For example, PyChecker/PyLint would report warnings. ... which can be obviously shut down if you are not a newbie and know what you are doing. I want PyChecker/PyLine to find bugs in my code and help me enforce *my* personal coding standard. I don't want them to tell me what it is "generically better" to do. The fact that there is a warning for builtin shadowing, to be used by people which such a coding standard, doesn't mean that everybody must agree with it. -- Giovanni Bajo From ben at coverity.com Mon Mar 6 06:35:52 2006 From: ben at coverity.com (Ben Chelf) Date: Sun, 05 Mar 2006 21:35:52 -0800 Subject: [Python-Dev] Coverity Open Source Defect Scan of Python Message-ID: <440BCA38.5070509@coverity.com> Hello Python Developers, I'm the CTO of Coverity, Inc., a company that does static source code analysis to look for defects in code. You may have heard of us or of our technology from its days at Stanford (the "Stanford Checker"). The reason I'm writing is because we have set up a framework internally to continually scan open source projects and provide the results of our analysis back to the developers of those projects. Python is one of the 32 projects currently scanned at: http://scan.coverity.com My belief is that we (Coverity) must reach out to the developers of these packages (you) in order to make progress in actually fixing the defects that we happen to find, so this is my first step in that mission. Of course, I think Coverity technology is great, but I want to hear what you think and that's why I worked with folks at Coverity to put this infrastructure in place. The process is simple -- it checks out your code each night from your repository and scans it so you can always see the latest results. Right now, we're guarding access to the actual defects that we report for a couple of reasons: (1) We think that you, as developers of Python, should have the chance to look at the defects we find to patch them before random other folks get to see what we found and (2) From a support perspective, we want to make sure that we have the appropriate time to engage with those who want to use the results to fix the code. Because of this second point, I'd ask that if you are interested in really digging into the results a bit further for your project, please have a couple of core maintainers (or group nominated individuals) reach out to me to request access. As this is a new process for us and still involves a small number of packages, I want to make sure that I personally can be involved with the activity that is generated from this effort. So I'm basically asking for people who want to play around with some cool new technology to help make source code better. If this interests you, please feel free to reach out to me directly. And of course, if there are other packages you care about that aren't currently on the list, I want to know about those too. If this is the wrong list, my sincerest apologies and please let me know where would be a more appropriate forum for this type of message. Many thanks for reading this far... -ben Ben Chelf Chief Technology Officer Coverity, Inc. From jimjjewett at gmail.com Mon Mar 6 18:17:55 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 6 Mar 2006 12:17:55 -0500 Subject: [Python-Dev] conditional expressions - add parens? Message-ID: Now that we have started to see conditional expressions ... I would like to see them parenthesized. The parens aren't as important as they are with generator expressions, but ... they matter. >From a recent checkin -- level = 0 if "absolute_import" in self.futures else -1 Mentally, I can't help parsing that as "level = 0" plus comments that turn out to be code that triggers backracking. When the expressions (particularly the true case) are longer, it gets even worse. I think that adding parentheses would help, by at least signalling that the logic is longer than just the next (single) expression. level = (0 if "absolute_import" in self.futures else -1) -jJ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060306/3f86f17f/attachment.html From tim.peters at gmail.com Mon Mar 6 19:17:48 2006 From: tim.peters at gmail.com (Tim Peters) Date: Mon, 6 Mar 2006 13:17:48 -0500 Subject: [Python-Dev] [Python-checkins] Python humor In-Reply-To: <00c501c63fd1$d546cea0$6a01a8c0@RaymondLaptop1> References: <4403E263.1060505@canterbury.ac.nz> <4404D747.2050402@canterbury.ac.nz> <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> <44052636.9090709@canterbury.ac.nz> <004001c63cf3$043fd710$6a01a8c0@RaymondLaptop1> <20060303191512.GA22242@panix.com> <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1> <5.1.1.6.0.20060304154338.01ddd940@mail.telecommunity.com> <20060304210549.GA10150@phd.pp.ru> <00c501c63fd1$d546cea0$6a01a8c0@RaymondLaptop1> Message-ID: <1f7befae0603061017m3ca8ace2u18bd06a7143376cc@mail.gmail.com> [Anna Ravenscroft] >>>> I think this is a really good point. next() is supposed to get used, by >>>> coders, in regular code - so it shouldn't be __next__. I can understand the >>>> desire for both forms, although that seems it would clutter things up >>>> unnecessarily - particularly if the two do the same thing [Phillip] >>> By this argument, we should be using ob.len() instead of len(ob), and >>> ob.iter() instead of iter(ob). [Oleg] >> Yes, I think it'd be more consistent and more object-oriented. After all >> we've switched from string.split(x, y) to x.split(y)... [Raymond Hettinger] > LOL, Shakespearean comedy on python-dev: > > * Phillip mistates Anna's position and shoots down a straw-man. > * Oleg agrees with a literal reading of Phillips note, missing the sarcasm > entirely. Then * Raymond points out those foibles in a humorous way. * Uncle Timmy replies in fashion that's hard to understand, although some suspect he's insinuating that all missed the point ;-) How's everyone doing, BTW? I think I picked up the Texas Mystery Disease from Holger Krekel -- bed-ridden 20 hours Saturday, and most of Sunday, with dry cough and high fever. Or that's just a normal outcome of sprinting :-) From thomas at python.org Mon Mar 6 20:16:42 2006 From: thomas at python.org (Thomas Wouters) Date: Mon, 6 Mar 2006 20:16:42 +0100 Subject: [Python-Dev] conditional expressions - add parens? In-Reply-To: References: Message-ID: <9e804ac0603061116g438abfc6i5d0908b1d2a7e53@mail.gmail.com> On 3/6/06, Jim Jewett wrote: > From a recent checkin -- > > level = 0 if "absolute_import" in self.futures else -1 > > Mentally, I can't help parsing that as "level = 0" plus comments that turn > out to be code that triggers backracking. > > When the expressions (particularly the true case) are longer, it gets even > worse. > > I think that adding parentheses would help, by at least signalling that > the logic is longer than just the next (single) expression. > > level = (0 if "absolute_import" in self.futures else -1) > I'm not sure, are you suggesting the grammar/parser enforces it that way, or that it should be in PEP-8? (I still can't help but go 'yecchh' whenever I see the code. It looks like Perl's 'expr if expr' form, but it behaves quite different. Right now, removing if/else expressions still looks like the best solution, to me :-> But gauging reactions (my own and others) to the actual if-expr in use is the main reason I added those couple of lines to the compiler package.) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060306/48338b2a/attachment.html From tim.peters at gmail.com Mon Mar 6 20:26:58 2006 From: tim.peters at gmail.com (Tim Peters) Date: Mon, 6 Mar 2006 14:26:58 -0500 Subject: [Python-Dev] Coverity Open Source Defect Scan of Python In-Reply-To: <440BCA38.5070509@coverity.com> References: <440BCA38.5070509@coverity.com> Message-ID: <1f7befae0603061126m6b40e1d7yd27ced52c0f005c3@mail.gmail.com> [Ben Chelf ] > ... > I'd ask that if you are interested in really digging into the results a bit > further for your project, please have a couple of core maintainers (or > group nominated individuals) reach out to me to request access. Didn't we set up a "security swat team" some time ago? If not, we should. Regardless, since I have more free time these days, I'd like to be on it. think-of-it-as-john-kelly-reaching-out-to-andy-spowicz-ly y'rs - tim From barry at python.org Mon Mar 6 20:57:59 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 06 Mar 2006 14:57:59 -0500 Subject: [Python-Dev] Coverity Open Source Defect Scan of Python In-Reply-To: <1f7befae0603061126m6b40e1d7yd27ced52c0f005c3@mail.gmail.com> References: <440BCA38.5070509@coverity.com> <1f7befae0603061126m6b40e1d7yd27ced52c0f005c3@mail.gmail.com> Message-ID: <1141675079.10805.222.camel@resist.wooz.org> On Mon, 2006-03-06 at 14:26 -0500, Tim Peters wrote: > [Ben Chelf ] > > ... > > I'd ask that if you are interested in really digging into the results a bit > > further for your project, please have a couple of core maintainers (or > > group nominated individuals) reach out to me to request access. > > Didn't we set up a "security swat team" some time ago? If not, we > should. Regardless, since I have more free time these days, I'd like > to be on it. Yep, it's called security at python.org (with a semi-secret backing mailing list, which I'd be happy for you to join!). I definitely think that group of folks at the least should review the results. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060306/711696cd/attachment.pgp From thomas at python.org Mon Mar 6 21:11:37 2006 From: thomas at python.org (Thomas Wouters) Date: Mon, 6 Mar 2006 21:11:37 +0100 Subject: [Python-Dev] Coverity Open Source Defect Scan of Python In-Reply-To: <1141675079.10805.222.camel@resist.wooz.org> References: <440BCA38.5070509@coverity.com> <1f7befae0603061126m6b40e1d7yd27ced52c0f005c3@mail.gmail.com> <1141675079.10805.222.camel@resist.wooz.org> Message-ID: <9e804ac0603061211q18cbdd2di79ffb7053150dbe3@mail.gmail.com> On 3/6/06, Barry Warsaw wrote: > > Didn't we set up a "security swat team" some time ago? If not, we > > should. Regardless, since I have more free time these days, I'd like > > to be on it. > > Yep, it's called security at python.org (with a semi-secret backing mailing > list, which I'd be happy for you to join!). I definitely think that > group of folks at the least should review the results. Well, if we start volunteering here, I'll volunteer as well. (For either group.) Can't let Tim have all the fun! -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060306/48898db9/attachment.htm From jimjjewett at gmail.com Mon Mar 6 21:15:30 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 6 Mar 2006 15:15:30 -0500 Subject: [Python-Dev] conditional expressions - add parens? In-Reply-To: <9e804ac0603061116g438abfc6i5d0908b1d2a7e53@mail.gmail.com> References: <9e804ac0603061116g438abfc6i5d0908b1d2a7e53@mail.gmail.com> Message-ID: On 3/6/06, Thomas Wouters wrote: > On 3/6/06, Jim Jewett wrote: > > level = 0 if "absolute_import" in self.futures else -1 > > Mentally, I can't help parsing that as "level = 0" plus > > comments that turn out to be code that triggers > > backtracking. > > I think that adding parentheses would help ... > > level = (0 if "absolute_import" in self.futures else -1) > I'm not sure, are you suggesting the grammar/parser > enforces it that way, or that it should be in PEP-8? I would prefer that the grammar enforce it, but putting it in PEP 8 would be a half-way measure. I can't imagine an expression small enough that the parentheses really hurt. var = t if c else f var = (t if c else f) In PEP 8, I would even go so far as to recommend parentheses around the three sub-expressions if they contain any internal whitespace, but I don't think the grammar should enforce that. > (I still can't help but go 'yecchh' whenever I see the > code. It looks like Perl's 'expr if expr' form, but it > behaves quite different. Right now, removing if/else > expressions still looks like the best solution, to me :-> Yah, that sounds uncontroversial. :D -jJ From tim.peters at gmail.com Mon Mar 6 21:17:37 2006 From: tim.peters at gmail.com (Tim Peters) Date: Mon, 6 Mar 2006 15:17:37 -0500 Subject: [Python-Dev] Coverity Open Source Defect Scan of Python In-Reply-To: <1141675079.10805.222.camel@resist.wooz.org> References: <440BCA38.5070509@coverity.com> <1f7befae0603061126m6b40e1d7yd27ced52c0f005c3@mail.gmail.com> <1141675079.10805.222.camel@resist.wooz.org> Message-ID: <1f7befae0603061217ncb10700u576fd1355f2072e9@mail.gmail.com> [Barry] > Yep, it's called security at python.org (with a semi-secret backing mailing > list, which I'd be happy for you to join!). If guessing the right Mailman URL was the semi-secret test, I passed :-) > I definitely think that group of folks at the least should review the results. Yup! From allison at shasta.stanford.edu Mon Mar 6 21:42:41 2006 From: allison at shasta.stanford.edu (Dennis Allison) Date: Mon, 6 Mar 2006 12:42:41 -0800 (PST) Subject: [Python-Dev] Coverity Open Source Defect Scan of Python In-Reply-To: <1141675079.10805.222.camel@resist.wooz.org> Message-ID: On Mon, 6 Mar 2006, Barry Warsaw wrote: > On Mon, 2006-03-06 at 14:26 -0500, Tim Peters wrote: > > [Ben Chelf ] > > > ... > > > I'd ask that if you are interested in really digging into the results a bit > > > further for your project, please have a couple of core maintainers (or > > > group nominated individuals) reach out to me to request access. > > > > Didn't we set up a "security swat team" some time ago? If not, we > > should. Regardless, since I have more free time these days, I'd like > > to be on it. > > Yep, it's called security at python.org (with a semi-secret backing mailing > list, which I'd be happy for you to join!). I definitely think that > group of folks at the least should review the results. > > -Barry > >From their open source chart: OpenVPN 7 69,842 0.100 Sign in Register Perl 89 479,780 0.186 Sign in Register PHP 207 431,251 0.480 Sign in Register PostgreSQL 297 815,700 0.364 Sign in Register ProFTPD 26 89,650 0.290 Sign in Register Python 59 259,896 0.227 Sign in Register Samba 215 312,482 0.688 Sign in Register This is interesting stuff. See http://metacomp.stanford.edu for some background. The Coverty marketing droids need to be a bit less anal about getting people to register at the website. IMHO, the technology should be described openly and allowed to speak for itself. On the other hand, the policy of not disclosing discovered bugs until someone has had a chance to evaluate their significance and fix them is probably a good one. I'd also encourage Coventry to explain their business model a bit more clearly. Coventry seems to be supportive of open source projects. Coverty also seems to be targeting big companies as customers. It's not clear how arbitrary open source projects (and small companies and individuals) will be able to take advantage of Coventry's products and services. >From Ben's email: ... if you are interested in really digging into the results a bit further for your project, please have a couple of core maintainers (or group nominated individuals) reach out to me to request access. As this is a new process for us and still involves a small number of packages, I want to make sure that I personally can be involved with the activity that is generated from this effort. So I'm basically asking for people who want to play around with some cool new technology to help make source code better. If this interests you, please feel free to reach out to me directly. And of course, if there are other packages you care about that aren't currently on the list, I want to know about those too. This looks to me to be something worth doing. I wish I had the time to be one of the designated folks, but, sadly, I don't. From guido at python.org Mon Mar 6 21:52:14 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 6 Mar 2006 12:52:14 -0800 Subject: [Python-Dev] Coverity Open Source Defect Scan of Python In-Reply-To: <440BCA38.5070509@coverity.com> References: <440BCA38.5070509@coverity.com> Message-ID: FWIW, coverity has been busy marketing this already: http://www.pcpro.co.uk/news/84465/key-opensource-code-passes-muster.html -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steve at holdenweb.com Mon Mar 6 22:34:51 2006 From: steve at holdenweb.com (Steve Holden) Date: Mon, 06 Mar 2006 21:34:51 +0000 Subject: [Python-Dev] conditional expressions - add parens? In-Reply-To: References: Message-ID: <440CAAFB.5060702@holdenweb.com> Jim Jewett wrote: > Now that we have started to see conditional expressions ... I would > like to see them parenthesized. The parens aren't as important as they > are with generator expressions, but ... they matter. > > From a recent checkin -- > > level = 0 if "absolute_import" in self.futures else -1 > Personally I think the above code exemplifies exactly why Guido was correct in his reluctance to add conditional expressions. > Mentally, I can't help parsing that as "level = 0" plus comments that > turn out to be code that triggers backracking. > > When the expressions (particularly the true case) are longer, it gets > even worse. > > I think that adding parentheses would help, by at least signalling that > the logic is longer than just the next (single) expression. > > level = (0 if "absolute_import" in self.futures else -1) > Contrast with the bleeding obvious: level = 0 if "absolute_import" in self.futures: level = -1 or even, if a certain obscurity is desirable: level = - ("absolute_import" in self.futures) regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd www.holdenweb.com Love me, love my blog holdenweb.blogspot.com From nas at arctrix.com Mon Mar 6 22:36:44 2006 From: nas at arctrix.com (Neil Schemenauer) Date: Mon, 6 Mar 2006 21:36:44 +0000 (UTC) Subject: [Python-Dev] collections.idset and collections.iddict? Message-ID: I occasionally need dictionaries or sets that use object identity rather than __hash__ to store items. Would it be appropriate to add these to the collections module? Neil From mcherm at mcherm.com Mon Mar 6 23:12:06 2006 From: mcherm at mcherm.com (Michael Chermside) Date: Mon, 06 Mar 2006 14:12:06 -0800 Subject: [Python-Dev] (no subject) Message-ID: <20060306141206.6epyt2924e84kcos@login.werra.lunarpages.com> Jim Jewet writes: > I can't imagine [a conditional] expression small enough that [requiring] > parentheses really hurt. > > var = t if c else f > var = (t if c else f) This is a good example. I'm now +1 on adding this to PEP 8. I'm +0 on adding it to the grammer... in the long run it's unlikely to make any difference. Steve Holden writes: > Contrast with the bleeding obvious: > > level = 0 > if "absolute_import" in self.futures: > level = -1 Comparing this to the alternative: level = (0 if ("absolute_import" in self.futures) else -1) In the latter, level clearly takes on values of 0 or -1. In the former, level has a default value of 0, which is followed by what appears to be some branching logic. > or even, if a certain obscurity is desirable: > > level = - ("absolute_import" in self.futures) The only USE I have ever seen for code obscurity was in writing certain vote-counting software where the true behavior needed to remain obscure even in the face of source code publication.[1] -- Michael Chermside [1] http://graphics.stanford.edu/~danielrh/vote/scores.html From jeremy at alum.mit.edu Mon Mar 6 23:15:47 2006 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Mon, 6 Mar 2006 17:15:47 -0500 Subject: [Python-Dev] Coverity Open Source Defect Scan of Python In-Reply-To: <9e804ac0603061211q18cbdd2di79ffb7053150dbe3@mail.gmail.com> References: <440BCA38.5070509@coverity.com> <1f7befae0603061126m6b40e1d7yd27ced52c0f005c3@mail.gmail.com> <1141675079.10805.222.camel@resist.wooz.org> <9e804ac0603061211q18cbdd2di79ffb7053150dbe3@mail.gmail.com> Message-ID: On 3/6/06, Thomas Wouters wrote: > > > > On 3/6/06, Barry Warsaw wrote: > > > Didn't we set up a "security swat team" some time ago? If not, we > > > should. Regardless, since I have more free time these days, I'd like > > > to be on it. > > > > Yep, it's called security at python.org (with a semi-secret backing mailing > > list, which I'd be happy for you to join!). I definitely think that > > group of folks at the least should review the results. > > > Well, if we start volunteering here, I'll volunteer as well. (For either > group.) Can't let Tim have all the fun! I also sent mail to Ben volunteering. I expect the scope of defects recognized is larger than just security. In particular, the compiler has a large body of code that has never been released before. It would nice to catch a few of its bugs before a release :-). Jeremy From steve at holdenweb.com Mon Mar 6 23:25:05 2006 From: steve at holdenweb.com (Steve Holden) Date: Mon, 06 Mar 2006 22:25:05 +0000 Subject: [Python-Dev] conditional expressions - add parens? In-Reply-To: <440CB667.8080008@masklinn.net> References: <440CAAFB.5060702@holdenweb.com> <440CB667.8080008@masklinn.net> Message-ID: <440CB6C1.5030906@holdenweb.com> Morel Xavier wrote: > Steve Holden wrote: > >> Contrast with the bleeding obvious: >> >> level = 0 >> if "absolute_import" in self.futures: >> level = -1 >> >> regards >> Steve > > > > The issue that spawned the necessity of a ternary operator in the first > place was that this syntax is not usable at all in quite a few > situations like, say, list comprehensions... > Sure, and for those uses it's good enough. This isn't really a python-dev topic, though, so I'll avoid following up further on this list. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd www.holdenweb.com Love me, love my blog holdenweb.blogspot.com From brett at python.org Mon Mar 6 23:29:06 2006 From: brett at python.org (Brett Cannon) Date: Mon, 6 Mar 2006 14:29:06 -0800 Subject: [Python-Dev] collections.idset and collections.iddict? In-Reply-To: References: Message-ID: On 3/6/06, Neil Schemenauer wrote: > I occasionally need dictionaries or sets that use object identity > rather than __hash__ to store items. Would it be appropriate to add > these to the collections module? Yeah, that would be the place for them. But would it be more helpful to generalize this to having something in collections that worked like DSU for dicts and their hash value? -Brett From barry at python.org Tue Mar 7 00:16:24 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 06 Mar 2006 18:16:24 -0500 Subject: [Python-Dev] Two gcmodule patches Message-ID: <1141686984.32270.246.camel@resist.wooz.org> There are two patches on SF to add a couple of features to the gc module. Skip wrote one (which I reviewed) and I wrote the other. Neither is earth shattering, but they're helpful when debugging gc issues. Skips adds some timing output when DEBUG_STATS is set: https://sourceforge.net/tracker/index.php?func=detail&aid=1100294&group_id=5470&atid=305470 Mine adds gc.get_count() -- a parallel to .get_threshold() -- and also adds an optional 'generation' argument to gc.collect(). https://sourceforge.net/tracker/index.php?func=detail&aid=1443865&group_id=5470&atid=305470 The patch also cleans up a couple of gcc warnings. Thoughts? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060306/4b629ac6/attachment.pgp From nas at arctrix.com Tue Mar 7 00:30:00 2006 From: nas at arctrix.com (Neil Schemenauer) Date: Mon, 6 Mar 2006 23:30:00 +0000 (UTC) Subject: [Python-Dev] Two gcmodule patches References: <1141686984.32270.246.camel@resist.wooz.org> Message-ID: Barry Warsaw wrote: > There are two patches on SF to add a couple of features to the gc > module. Skip wrote one (which I reviewed) and I wrote the other. > Neither is earth shattering, but they're helpful when debugging gc > issues. I have no major objections to either patch. The library reference should be updated though. Neil From barry at python.org Tue Mar 7 00:32:17 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 06 Mar 2006 18:32:17 -0500 Subject: [Python-Dev] Two gcmodule patches In-Reply-To: References: <1141686984.32270.246.camel@resist.wooz.org> Message-ID: <1141687937.10806.249.camel@resist.wooz.org> On Mon, 2006-03-06 at 23:30 +0000, Neil Schemenauer wrote: > Barry Warsaw wrote: > > There are two patches on SF to add a couple of features to the gc > > module. Skip wrote one (which I reviewed) and I wrote the other. > > Neither is earth shattering, but they're helpful when debugging gc > > issues. > > I have no major objections to either patch. The library reference > should be updated though. Yes, definitely. Thanks. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060306/c28e11ad/attachment.pgp From python at rcn.com Tue Mar 7 01:08:02 2006 From: python at rcn.com (Raymond Hettinger) Date: Mon, 6 Mar 2006 16:08:02 -0800 Subject: [Python-Dev] collections.idset and collections.iddict? References: Message-ID: <006c01c6417b$344444f0$a600000a@RaymondLaptop1> [Neil Schemenauer] >I occasionally need dictionaries or sets that use object identity > rather than __hash__ to store items. Would it be appropriate to add > these to the collections module? Why not decorate the objects with a class adding a method: def __hash__(self): return id(self) That would seem to be more Pythonic than creating custom variants of other containers. Raymond From guido at python.org Tue Mar 7 01:14:54 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 6 Mar 2006 16:14:54 -0800 Subject: [Python-Dev] collections.idset and collections.iddict? In-Reply-To: <006c01c6417b$344444f0$a600000a@RaymondLaptop1> References: <006c01c6417b$344444f0$a600000a@RaymondLaptop1> Message-ID: On 3/6/06, Raymond Hettinger wrote: > [Neil Schemenauer] > >I occasionally need dictionaries or sets that use object identity > > rather than __hash__ to store items. Would it be appropriate to add > > these to the collections module? > > Why not decorate the objects with a class adding a method: > def __hash__(self): > return id(self) > > That would seem to be more Pythonic than creating custom variants of other > containers. I hate to second-guess the OP, but you'd have to override __eq__ too, and probably __ne__ and __cmp__ just to be sure. And probably that wouldn't do -- since the default __hash__ and __eq__ have the desired behavior, the OP is apparently talking about objects that override these operations to do something meaningful; overriding them back presumably breaks other functionality. I wonder if this use case and the frequently requested case-insensitive dict don't have some kind of generalization in common -- perhaps a dict that takes a key function a la list.sort()? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jack at performancedrivers.com Tue Mar 7 01:21:26 2006 From: jack at performancedrivers.com (Jack Diederich) Date: Mon, 6 Mar 2006 19:21:26 -0500 Subject: [Python-Dev] collections.idset and collections.iddict? In-Reply-To: References: <006c01c6417b$344444f0$a600000a@RaymondLaptop1> Message-ID: <20060307002126.GH6309@performancedrivers.com> On Mon, Mar 06, 2006, Guido van Rossum wrote: > On 3/6/06, Raymond Hettinger wrote: > > [Neil Schemenauer] > > >I occasionally need dictionaries or sets that use object identity > > > rather than __hash__ to store items. Would it be appropriate to add > > > these to the collections module? > > > > Why not decorate the objects with a class adding a method: > > def __hash__(self): > > return id(self) > > > > That would seem to be more Pythonic than creating custom variants of other > > containers. > > I hate to second-guess the OP, but you'd have to override __eq__ too, > and probably __ne__ and __cmp__ just to be sure. And probably that > wouldn't do -- since the default __hash__ and __eq__ have the desired > behavior, the OP is apparently talking about objects that override > these operations to do something meaningful; overriding them back > presumably breaks other functionality. I assumed Neil wanted a container that was id() sensitive, I've done this occasionally myself to see if an object is in a container and not just an object equivalent to the one I am checking for. >>> a = set([1,2,3,4]) >>> b = set([1,2,3,4]) >>> a == b True >>> a is b False >>> container = [a] >>> b in container True >>> container = [id(a)] >>> id(b) in container False >>> -Jack From bob at redivi.com Tue Mar 7 01:43:01 2006 From: bob at redivi.com (Bob Ippolito) Date: Mon, 6 Mar 2006 16:43:01 -0800 Subject: [Python-Dev] collections.idset and collections.iddict? In-Reply-To: References: <006c01c6417b$344444f0$a600000a@RaymondLaptop1> Message-ID: <9FB7D95A-6C7B-43A1-B60E-9A44C770CCC0@redivi.com> On Mar 6, 2006, at 4:14 PM, Guido van Rossum wrote: > On 3/6/06, Raymond Hettinger wrote: >> [Neil Schemenauer] >>> I occasionally need dictionaries or sets that use object identity >>> rather than __hash__ to store items. Would it be appropriate to add >>> these to the collections module? >> >> Why not decorate the objects with a class adding a method: >> def __hash__(self): >> return id(self) >> >> That would seem to be more Pythonic than creating custom variants >> of other >> containers. > > I hate to second-guess the OP, but you'd have to override __eq__ too, > and probably __ne__ and __cmp__ just to be sure. And probably that > wouldn't do -- since the default __hash__ and __eq__ have the desired > behavior, the OP is apparently talking about objects that override > these operations to do something meaningful; overriding them back > presumably breaks other functionality. > > I wonder if this use case and the frequently requested > case-insensitive dict don't have some kind of generalization in common > -- perhaps a dict that takes a key function a la list.sort()? +1. I've wanted such a thing a couple times, and there is some precedent in the stdlib (e.g. WeakKeyDictionary would be a lot shorter with such a base class). -bob From aleaxit at gmail.com Tue Mar 7 03:53:22 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Mon, 6 Mar 2006 18:53:22 -0800 Subject: [Python-Dev] collections.idset and collections.iddict? In-Reply-To: <9FB7D95A-6C7B-43A1-B60E-9A44C770CCC0@redivi.com> References: <006c01c6417b$344444f0$a600000a@RaymondLaptop1> <9FB7D95A-6C7B-43A1-B60E-9A44C770CCC0@redivi.com> Message-ID: <94713DFA-229A-4DA3-A6C2-98D16EF542D0@gmail.com> On Mar 6, 2006, at 4:43 PM, Bob Ippolito wrote: > > On Mar 6, 2006, at 4:14 PM, Guido van Rossum wrote: ... >> I wonder if this use case and the frequently requested >> case-insensitive dict don't have some kind of generalization in >> common >> -- perhaps a dict that takes a key function a la list.sort()? > > +1. I've wanted such a thing a couple times, and there is some > precedent in the stdlib (e.g. WeakKeyDictionary would be a lot > shorter with such a base class). Seconded -- though at least in the cases I remember better what I really needed was a _list_ with such functionality (order was significant, but I wanted 'in'-tests and find and remove calls to work by id, not by ==) -- I ended up with a somewhat more complicated solution (not just a list subclass, though that might perhaps be simpler). In the case where I needed a dict, I inherited from DictMixin and delegated some methods (with id(key) instead of key) to a self.somedict -- if a dict could be built with a key function, the solution would be both simpler and faster. Alex From hoffman at ebi.ac.uk Mon Mar 6 19:47:00 2006 From: hoffman at ebi.ac.uk (Michael Hoffman) Date: Mon, 6 Mar 2006 18:47:00 +0000 Subject: [Python-Dev] iterator API in Py3.0 In-Reply-To: References: <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1> <015901c6395d$3b9fd4b0$2e2a960a@RaymondLaptop1> <4403E263.1060505@canterbury.ac.nz> <440411E1.6040401@gmail.com> <4404D747.2050402@canterbury.ac.nz> <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> <44052636.9090709@canterbury.ac.nz> <004001c63cf3$043fd710$6a01a8c0@RaymondLaptop1> <20060303191512.GA22242@panix.com> <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1> <5.1.1.6.0.20060304154338.01ddd940@mail.telecommunity.com> <00ad01c63fd0$2e4c2f10$6a01a8c0@RaymondLaptop1> <5.1.1.6.0.20060304161723.01e05c30@mail.telecommunity.com> Message-ID: Another nice thing about having a next() built-in is that it makes getting the first item of a generator expression a lot more elegant, IMHO. I think this: next(item for item in items if item > 3) is a lot clearer than this: (item for item in items if item > 3).next() or alternatives that would break this into multiple statements. [inspired by a recent python-list question] -- Michael Hoffman European Bioinformatics Institute From aleaxit at gmail.com Tue Mar 7 04:50:56 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Mon, 6 Mar 2006 19:50:56 -0800 Subject: [Python-Dev] [Python-checkins] Python humor In-Reply-To: <1f7befae0603061017m3ca8ace2u18bd06a7143376cc@mail.gmail.com> References: <4403E263.1060505@canterbury.ac.nz> <4404D747.2050402@canterbury.ac.nz> <012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1> <44052636.9090709@canterbury.ac.nz> <004001c63cf3$043fd710$6a01a8c0@RaymondLaptop1> <20060303191512.GA22242@panix.com> <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1> <5.1.1.6.0.20060304154338.01ddd940@mail.telecommunity.com> <20060304210549.GA10150@phd.pp.ru> <00c501c63fd1$d546cea0$6a01a8c0@RaymondLaptop1> <1f7befae0603061017m3ca8ace2u18bd06a7143376cc@mail.gmail.com> Message-ID: <4B12D377-3E3D-4CD8-8DA6-EC3B7E79B321@gmail.com> On Mar 6, 2006, at 10:17 AM, Tim Peters wrote: ... > How's everyone doing, BTW? Swimmingly, thanks! Too busy to breathe, or come to pycon:-(, but, happy as a lark. > I think I picked up the Texas Mystery > Disease from Holger Krekel -- bed-ridden 20 hours Saturday, and most > of Sunday, with dry cough and high fever. Or that's just a normal > outcome of sprinting :-) I got vaccinated against flu in Oct last year, and as a result haven't had to take ANY sick day so far (darn, I'm wasting Google's policy of giving unlimited sick days...!!!). Alex From aleaxit at gmail.com Tue Mar 7 04:52:40 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Mon, 6 Mar 2006 19:52:40 -0800 Subject: [Python-Dev] conditional expressions - add parens? In-Reply-To: References: Message-ID: On Mar 6, 2006, at 9:17 AM, Jim Jewett wrote: ... > I think that adding parentheses would help, by at least signalling > that the logic is longer than just the next (single) expression. > > level = (0 if "absolute_import" in self.futures else -1) +1 (just because I can't give it +3.1415926...!!!). *Mandatory* parentheses make this form MUCH more readable. Alex From unknown_kev_cat at hotmail.com Tue Mar 7 05:26:53 2006 From: unknown_kev_cat at hotmail.com (Joe Smith) Date: Mon, 6 Mar 2006 23:26:53 -0500 Subject: [Python-Dev] conditional expressions - add parens? References: <440CAAFB.5060702@holdenweb.com> Message-ID: "Steve Holden" wrote in message news:440CAAFB.5060702 at holdenweb.com... > Jim Jewett wrote: >> I think that adding parentheses would help, by at least signalling that >> the logic is longer than just the next (single) expression. >> >> level = (0 if "absolute_import" in self.futures else -1) >> > Contrast with the bleeding obvious: > > level = 0 > if "absolute_import" in self.futures: > level = -1 > > or even, if a certain obscurity is desirable: > > level = - ("absolute_import" in self.futures) > Wait a second. I may be just a user but if the above is correct then that syntax needs to die! There is no logical reason for "XX if YY else ZZ" to be roughly equivlent to: "if (YY) then {ZZ} else {XX}" , but AFAICT that is pretty much the way you expanded that. I hope I misunderstood, or that there was a typo in a post. From tim.peters at gmail.com Tue Mar 7 05:47:06 2006 From: tim.peters at gmail.com (Tim Peters) Date: Mon, 6 Mar 2006 23:47:06 -0500 Subject: [Python-Dev] conditional expressions - add parens? In-Reply-To: References: <440CAAFB.5060702@holdenweb.com> Message-ID: <1f7befae0603062047g640d33femf27ae529596e2cd1@mail.gmail.com> [Jim Jewett] >>> I think that adding parentheses would help, by at least signalling that >>> the logic is longer than just the next (single) expression. >>> >>> level = (0 if "absolute_import" in self.futures else -1) [Steve Holden] >> Contrast with the bleeding obvious: >> >> level = 0 >> if "absolute_import" in self.futures: >> level = -1 >> >> or even, if a certain obscurity is desirable: >> >> level = - ("absolute_import" in self.futures) In honor of Peter Naur receiving the 2005 Turing Award: and remembering Python's Algol roots, I'd like to constrast it with the truly obvious: level = (if "absolute_import" in self.futures then 0 else -1) That way also has the minor advantage of computing the same value for `level` as Jim's code ;-) [Joe Smith] > Wait a second. > > I may be just a user but if the above is correct then that syntax needs to > die! > There is no logical reason for "XX if YY else ZZ" to be roughly equivlent > to: > "if (YY) then {ZZ} else {XX}" , but AFAICT that is pretty much the way you > expanded that. Ya, Steve just got it backwards. "(X if Y else Z)" is the same as "(if Y then X else Z)", except for the excessive novelty. The obvious spelling would require making "then" a keyword, which is actually OK with everyone :-) > I hope I misunderstood, or that there was a typo in a post. You were lucky this time, bucko, but don't ever go questioning a python-dev regular again ;-) From guido at python.org Tue Mar 7 06:25:10 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 6 Mar 2006 21:25:10 -0800 Subject: [Python-Dev] iterator API in Py3.0 In-Reply-To: References: <015901c6395d$3b9fd4b0$2e2a960a@RaymondLaptop1> <44052636.9090709@canterbury.ac.nz> <004001c63cf3$043fd710$6a01a8c0@RaymondLaptop1> <20060303191512.GA22242@panix.com> <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1> <5.1.1.6.0.20060304154338.01ddd940@mail.telecommunity.com> <00ad01c63fd0$2e4c2f10$6a01a8c0@RaymondLaptop1> <5.1.1.6.0.20060304161723.01e05c30@mail.telecommunity.com> Message-ID: On 3/6/06, Michael Hoffman wrote: > Another nice thing about having a next() built-in is that it makes > getting the first item of a generator expression a lot more elegant, > IMHO. > > I think this: > > next(item for item in items if item > 3) > > is a lot clearer than this: > > (item for item in items if item > 3).next() > > or alternatives that would break this into multiple statements. Why is putting everything on a single line considered elegant? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik at pythonware.com Tue Mar 7 06:29:05 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue, 7 Mar 2006 06:29:05 +0100 Subject: [Python-Dev] how about adding ping's uuid module to the standard lib ? Message-ID: see subject and http://python.org/sf/1368955 comments ? (note sure how my previous attempt to post this ended up on comp.lang.python instead, but I'll blame it on gmane... ;-) From fredrik at pythonware.com Tue Mar 7 06:38:55 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue, 7 Mar 2006 06:38:55 +0100 Subject: [Python-Dev] conditional expressions - add parens? References: Message-ID: Alex Martelli wrote: > On Mar 6, 2006, at 9:17 AM, Jim Jewett wrote: > ... > > I think that adding parentheses would help, by at least signalling > > that the logic is longer than just the next (single) expression. > > > > level = (0 if "absolute_import" in self.futures else -1) > > +1 (just because I can't give it +3.1415926...!!!). *Mandatory* > parentheses make this form MUCH more readable. but which parens ? level = 0 if "absolute_import" in self.futures else -1 level = 0 if ("absolute_import" in self.futures) else -1 level = (0 if "absolute_import" in self.futures else -1) level = (0 if ("absolute_import" in self.futures) else -1) level = ((0) if ("absolute_import" in self.futures) else (-1)) # ... if you're asking me, I'd say that the second alternative is most readable here. at least as long as level = if ("absolute_import" in self.futures) then 0 else -1 is not on the list... level = 0 or -1 depending on "absolute_import" in self.futures From ianb at colorstudy.com Tue Mar 7 07:04:45 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 07 Mar 2006 00:04:45 -0600 Subject: [Python-Dev] collections.idset and collections.iddict? In-Reply-To: References: <006c01c6417b$344444f0$a600000a@RaymondLaptop1> Message-ID: <440D227D.7080402@colorstudy.com> Guido van Rossum wrote: > On 3/6/06, Raymond Hettinger wrote: >> [Neil Schemenauer] >>> I occasionally need dictionaries or sets that use object identity >>> rather than __hash__ to store items. Would it be appropriate to add >>> these to the collections module? >> Why not decorate the objects with a class adding a method: >> def __hash__(self): >> return id(self) >> >> That would seem to be more Pythonic than creating custom variants of other >> containers. > > I hate to second-guess the OP, but you'd have to override __eq__ too, > and probably __ne__ and __cmp__ just to be sure. And probably that > wouldn't do -- since the default __hash__ and __eq__ have the desired > behavior, the OP is apparently talking about objects that override > these operations to do something meaningful; overriding them back > presumably breaks other functionality. > > I wonder if this use case and the frequently requested > case-insensitive dict don't have some kind of generalization in common > -- perhaps a dict that takes a key function a la list.sort()? That's what occurred to me as soon as I read Neil's post as well. I think it would have the added benefit that it would be case insensitive while still preserving case. Here's a rough idea of the semantics: from UserDict import DictMixin class KeyedDict(DictMixin): def __init__(self, keyfunc): self.keyfunc = keyfunc self.data = {} def __getitem__(self, key): return self.data[self.keyfunc(key)][1] def __setitem__(self, key, value): self.data[self.keyfunc(key)] = (key, value) def __delitem__(self, key): del self.data[self.keyfunc(key)] def keys(self): return [v[0] for v in self.data.values()] I definitely like this more than a key-normalizing dictionary -- the normalized key is never actually exposed anywhere. I didn't follow the defaultdict thing through to the end, so I didn't catch what the constructor was going to look like for that; but I assume those choices will apply here as well. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From tjreedy at udel.edu Tue Mar 7 08:06:26 2006 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 7 Mar 2006 02:06:26 -0500 Subject: [Python-Dev] iterator API in Py3.0 References: <002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1><015901c6395d$3b9fd4b0$2e2a960a@RaymondLaptop1><4403E263.1060505@canterbury.ac.nz> <440411E1.6040401@gmail.com><4404D747.2050402@canterbury.ac.nz><012201c63cbe$b7daa410$6a01a8c0@RaymondLaptop1><44052636.9090709@canterbury.ac.nz><004001c63cf3$043fd710$6a01a8c0@RaymondLaptop1><20060303191512.GA22242@panix.com><002601c63eff$06f80d20$6a01a8c0@RaymondLaptop1><5.1.1.6.0.20060304154338.01ddd940@mail.telecommunity.com><00ad01c63fd0$2e4c2f10$6a01a8c0@RaymondLaptop1><5.1.1.6.0.20060304161723.01e05c30@mail.telecommunity.com> Message-ID: "Michael Hoffman" wrote in message news:Pine.LNX.4.64.0603061844140.21386 at qnzvnan.rov.np.hx... > Another nice thing about having a next() built-in is that it makes > getting the first item of a generator expression a lot more elegant, > I think this: > next(item for item in items if item > 3) > is a lot clearer than this: > (item for item in items if item > 3).next() > or alternatives that would break this into multiple statements. > [inspired by a recent python-list question] Yuck. This bug-prone one-liner was an answer to the question: how do I (the OP) get just the first item of items meeting a conditiion instead of all, as with filter, with a one-line expression instead of the obvious for loop. Bad question; bad answer. As I pointed out, this 'solution' raises StopIteration if there is no first valid item. That in turn crashes the program unless one adds more lines to deal with the possible exception. In the same post, I also suggested the OP first define his desired no-item behavior (not answered that I have seen). To me, one-liner-itis can entertain, but also distract from complete, correct coding. Terry Jan Reedy From steve at holdenweb.com Tue Mar 7 10:57:33 2006 From: steve at holdenweb.com (Steve Holden) Date: Tue, 07 Mar 2006 09:57:33 +0000 Subject: [Python-Dev] conditional expressions - add parens? In-Reply-To: References: <440CAAFB.5060702@holdenweb.com> Message-ID: Joe Smith wrote: > "Steve Holden" wrote in message > news:440CAAFB.5060702 at holdenweb.com... > >>Jim Jewett wrote: >> >>>I think that adding parentheses would help, by at least signalling that >>>the logic is longer than just the next (single) expression. >>> >>> level = (0 if "absolute_import" in self.futures else -1) >>> >> >>Contrast with the bleeding obvious: >> >> level = 0 >> if "absolute_import" in self.futures: >> level = -1 >> >>or even, if a certain obscurity is desirable: >> >> level = - ("absolute_import" in self.futures) >> > > > Wait a second. > > I may be just a user but if the above is correct then that syntax needs to > die! > There is no logical reason for "XX if YY else ZZ" to be roughly equivlent > to: > "if (YY) then {ZZ} else {XX}" , but AFAICT that is pretty much the way you > expanded that. > I hope I misunderstood, or that there was a typo in a post. > There was a typo in the post. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd www.holdenweb.com Love me, love my blog holdenweb.blogspot.com From martin at v.loewis.de Tue Mar 7 11:03:45 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 07 Mar 2006 11:03:45 +0100 Subject: [Python-Dev] how about adding ping's uuid module to the standard lib ? In-Reply-To: References: Message-ID: <440D5A81.6060707@v.loewis.de> Fredrik Lundh wrote: > see subject and http://python.org/sf/1368955 > > comments ? Fine with me. Martin From jeremy at alum.mit.edu Tue Mar 7 14:06:46 2006 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Tue, 7 Mar 2006 08:06:46 -0500 Subject: [Python-Dev] conditional expressions - add parens? In-Reply-To: References: Message-ID: On 3/6/06, Alex Martelli wrote: > > On Mar 6, 2006, at 9:17 AM, Jim Jewett wrote: > ... > > I think that adding parentheses would help, by at least signalling > > that the logic is longer than just the next (single) expression. > > > > level = (0 if "absolute_import" in self.futures else -1) > > +1 (just because I can't give it +3.1415926...!!!). *Mandatory* > parentheses make this form MUCH more readable. Recent language features seem to be suffereing from excessive parenthesisitis. I worry that people will stop remembering which expressions requirement parens in which context. Perhaps the solution is to require parens around all expressions, a simple consistent rule. If so, then adding parens around all statements is a fairly natural extension, which solves a number of problems like how to make a richer lambda. Jeremy From p.f.moore at gmail.com Tue Mar 7 14:15:03 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 7 Mar 2006 13:15:03 +0000 Subject: [Python-Dev] conditional expressions - add parens? In-Reply-To: References: Message-ID: <79990c6b0603070515o579565c7t796b5e12a902a362@mail.gmail.com> On 3/7/06, Jeremy Hylton wrote: > On 3/6/06, Alex Martelli wrote: > > > > On Mar 6, 2006, at 9:17 AM, Jim Jewett wrote: > > ... > > > I think that adding parentheses would help, by at least signalling > > > that the logic is longer than just the next (single) expression. > > > > > > level = (0 if "absolute_import" in self.futures else -1) > > > > +1 (just because I can't give it +3.1415926...!!!). *Mandatory* > > parentheses make this form MUCH more readable. > > Recent language features seem to be suffereing from excessive > parenthesisitis. Agreed. Mandatory parentheses purely for readability are ultimately futile - I bet I can write unreadable code even if the parens were required :-))))))))))))))))))))))))) The parentheses around genexps were (AFAICT) different - without them, the grammar was ambiguous, so some way of disambiguating was needed. +0 for mentioning parens around conditional expressions in PEP 8. But it's aready covered by the general "code should be readable" in my view. Paul. From ncoghlan at gmail.com Tue Mar 7 15:03:19 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 08 Mar 2006 00:03:19 +1000 Subject: [Python-Dev] conditional expressions - add parens? In-Reply-To: <79990c6b0603070515o579565c7t796b5e12a902a362@mail.gmail.com> References: <79990c6b0603070515o579565c7t796b5e12a902a362@mail.gmail.com> Message-ID: <440D92A7.3030207@gmail.com> Paul Moore wrote: > On 3/7/06, Jeremy Hylton wrote: >> On 3/6/06, Alex Martelli wrote: >>> On Mar 6, 2006, at 9:17 AM, Jim Jewett wrote: >>> ... >>>> I think that adding parentheses would help, by at least signalling >>>> that the logic is longer than just the next (single) expression. >>>> >>>> level = (0 if "absolute_import" in self.futures else -1) >>> +1 (just because I can't give it +3.1415926...!!!). *Mandatory* >>> parentheses make this form MUCH more readable. >> Recent language features seem to be suffereing from excessive >> parenthesisitis. > > Agreed. Mandatory parentheses purely for readability are ultimately > futile - I bet I can write unreadable code even if the parens were > required :-))))))))))))))))))))))))) > > The parentheses around genexps were (AFAICT) different - without them, > the grammar was ambiguous, so some way of disambiguating was needed. > > +0 for mentioning parens around conditional expressions in PEP 8. But > it's aready covered by the general "code should be readable" in my > view. Agreed - and often the best way to make a conditional expression readable is to not use it in the first place, which is a concept that an official policy on parentheses may mask. And then there's the question of how to parenthesise chained conditional expressions. Mandatory parentheses in that case would make me wonder if I was writing a certain language that isn't Python. . . Given that the main reason PEP 308 was finally implemented was to provide OOWTDI for conditional expressions (instead of the handful of not-at-all-obvious and not-quite-right-either workarounds that had developed in their absence), I expect to see of rush of enthusiastic overuse of the new expression, which will eventually settle down as everyone remembers that not every problem is a nail to be hit with the shiny new hammer ;) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From g.brandl at gmx.net Tue Mar 7 15:15:42 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 07 Mar 2006 15:15:42 +0100 Subject: [Python-Dev] "as" mania Message-ID: Hi, while "as" is being made a keyword, I remembered parallels between "with" and a proposal made some time ago: with expr as f: do something with f while expr as f: do something with f if expr as f: do something with f elif expr as f: do something else with f What do you think? Georg From guido at python.org Tue Mar 7 16:14:24 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 7 Mar 2006 07:14:24 -0800 Subject: [Python-Dev] "as" mania In-Reply-To: References: Message-ID: I suggest you file those as products of an overactive imagination. :-) Have you even tried to define precise semantics for any of those, like the expansion of "with E as V: B" in PEP 343? --Guido On 3/7/06, Georg Brandl wrote: > Hi, > > while "as" is being made a keyword, I remembered parallels between "with" > and a proposal made some time ago: > > with expr as f: > do something with f > > while expr as f: > do something with f > > if expr as f: > do something with f > elif expr as f: > do something else with f > > What do you think? > > Georg > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From aleaxit at gmail.com Tue Mar 7 16:16:08 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Tue, 7 Mar 2006 07:16:08 -0800 Subject: [Python-Dev] "as" mania In-Reply-To: References: Message-ID: On Mar 7, 2006, at 6:15 AM, Georg Brandl wrote: > Hi, > > while "as" is being made a keyword, I remembered parallels between > "with" > and a proposal made some time ago: > > with expr as f: > do something with f > > while expr as f: > do something with f > > if expr as f: > do something with f > elif expr as f: > do something else with f > > What do you think? I think the best use cases for 'assignment inside an if or while' condition, as far as they go, require `capturing' a SUB-expression of the condition, rather than the whole condition. E.g., in C, while ( (x=next_x()) < threshold ) ... being able to capture (by `as') only the whole (true or false) condition meets a minority of the use cases, so I'm unenthusiastic about it. Alex From jimjjewett at gmail.com Tue Mar 7 16:19:06 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Tue, 7 Mar 2006 10:19:06 -0500 Subject: [Python-Dev] conditional expressions - add parens? In-Reply-To: <79990c6b0603070515o579565c7t796b5e12a902a362@mail.gmail.com> References: <79990c6b0603070515o579565c7t796b5e12a902a362@mail.gmail.com> Message-ID: On 3/7/06, Paul Moore wrote: > The parentheses around genexps were (AFAICT) > different - without them, the grammar was ambiguous, > so some way of disambiguating was needed. The out-of-order evaluation is a very large change, because now we have a situation where normal parsing completes an expression, but needs to avoid evaluating it, just in case. Currently, we can write: >>> if False: >>> print r >>> else: >>> print 6 6 >>> r Traceback (most recent call last): File "", line 1, in -toplevel- r NameError: name 'r' is not defined In the above example, r doesn't get evaluated because the if ahead of it says to skip that branch. But with conditional expressions, that flow control is changed from *later* in the program. I don't think we'll see the equivalent of Intercal Suck Points anywhere but intentionally obfuscated code, but I do expect to see: >>> side_effect() if condition In fact, I think the below examples are reasonable uses that do a better job of expressing intent than the if statement would. I just don't like the mental backtrack they require, and would like some sort of advance warning. Parentheses at least tell me "You're not done yet; keep reading." >>> ack(r,r) if r not in cache >>> log(message) if error_flag -jJ From steve at holdenweb.com Tue Mar 7 16:29:18 2006 From: steve at holdenweb.com (Steve Holden) Date: Tue, 07 Mar 2006 15:29:18 +0000 Subject: [Python-Dev] conditional expressions - add parens? In-Reply-To: References: <79990c6b0603070515o579565c7t796b5e12a902a362@mail.gmail.com> Message-ID: <440DA6CE.6040108@holdenweb.com> Jim Jewett wrote: > On 3/7/06, Paul Moore wrote: > > >>The parentheses around genexps were (AFAICT) >>different - without them, the grammar was ambiguous, >>so some way of disambiguating was needed. > > > The out-of-order evaluation is a very large change, > because now we have a situation where normal > parsing completes an expression, but needs to avoid > evaluating it, just in case. > > Currently, we can write: > > >>> if False: > >>> print r > >>> else: > >>> print 6 > > 6 > > >>> r > > Traceback (most recent call last): > File "", line 1, in -toplevel- > r > NameError: name 'r' is not defined > > In the above example, r doesn't get evaluated because > the if ahead of it says to skip that branch. But with > conditional expressions, that flow control is changed > from *later* in the program. > > I don't think we'll see the equivalent of Intercal Suck > Points anywhere but intentionally obfuscated code, > but I do expect to see: > > >>> side_effect() if condition > > In fact, I think the below examples are reasonable uses > that do a better job of expressing intent than the if > statement would. I just don't like the mental backtrack > they require, and would like some sort of advance > warning. Parentheses at least tell me "You're not done > yet; keep reading." > > >>> ack(r,r) if r not in cache > > >>> log(message) if error_flag > I think you've misunderstood the nature of the change. HTe new conditionals are *expressions*, they aren't postfix conditionals for statements. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd www.holdenweb.com Love me, love my blog holdenweb.blogspot.com From aleaxit at gmail.com Tue Mar 7 16:42:29 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Tue, 7 Mar 2006 07:42:29 -0800 Subject: [Python-Dev] conditional expressions - add parens? In-Reply-To: <440DA6CE.6040108@holdenweb.com> References: <79990c6b0603070515o579565c7t796b5e12a902a362@mail.gmail.com> <440DA6CE.6040108@holdenweb.com> Message-ID: On Mar 7, 2006, at 7:29 AM, Steve Holden wrote: ... >> In fact, I think the below examples are reasonable uses >> that do a better job of expressing intent than the if >> statement would. I just don't like the mental backtrack >> they require, and would like some sort of advance >> warning. Parentheses at least tell me "You're not done >> yet; keep reading." >> >>>>> ack(r,r) if r not in cache >> >>>>> log(message) if error_flag >> > I think you've misunderstood the nature of the change. HTe new > conditionals are *expressions*, they aren't postfix conditionals for > statements. Right, but the only effect of that on the examples is that they both need an 'else None' continuation or thereabouts. I also predict widespread over-use of the new toy and agree with Jim that mandatory parentheses around the whole (X if Y else Z) construct would ameliorate legibility a bit. Alex From jendroska at psychologie.uni-wuerzburg.de Tue Mar 7 16:42:32 2006 From: jendroska at psychologie.uni-wuerzburg.de (Dirk Jendroska) Date: Tue, 07 Mar 2006 16:42:32 +0100 Subject: [Python-Dev] Scientific Survey: Working Conditions in Open Source Projects Message-ID: <440DA9E8.9010407@psychologie.uni-wuerzburg.de> We like to invite you to a survey about the working conditions in Free/Open-Source Software development. This survey is conducted by the Open-Source Research Group of the University of W?rzburg (Germany). We will compare work design in open source and proprietary software development. Our findings should be used to adjust the working conditions of software development to the needs of the developers. The results will be published under the Creative Commons License. To start the survey, please follow the link: http://www.unipark.de/uc/open-source-task/ If you have any questions don't hesitate to contact us! Dirk Jendroska http://www.psychologie.uni-wuerzburg.de/ao/staff/jendroska.php?lang=en From pje at telecommunity.com Tue Mar 7 17:01:00 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 07 Mar 2006 11:01:00 -0500 Subject: [Python-Dev] how about adding ping's uuid module to the standard lib ? In-Reply-To: Message-ID: <5.1.1.6.0.20060307103122.042c4ae8@mail.telecommunity.com> At 06:29 AM 3/7/2006 +0100, Fredrik Lundh wrote: >see subject and http://python.org/sf/1368955 > >comments ? Why can't the UUIDs be immutable, so they can be dictionary keys? Also, it would be nice if you could just call UUID() to create a generic UUID in a platform-appropriate way. PEAK's uuid module does this such that if win32all is present, you get a Windows GUID, or if you have a FreeBSD 5+ or NetBSD 2+ kernel you use the local platform uuidgen API. See e.g.: http://svn.eby-sarna.com/PEAK/src/peak/util/_uuidgen.c?view=markup and: http://svn.eby-sarna.com/PEAK/src/peak/util/uuid.py?view=markup for details. This implementation also has unit tests: http://svn.eby-sarna.com/PEAK/src/peak/util/tests/uuid.py?view=markup and has been in production use for several years now. On the other hand, Ping's implementation is a lot simpler than ours; ours does a lot of stuff to support e.g. /dev/urandom or fallback seeding, whereas Python 2.4 offers os.urandom. We also use different fallback algorithms for getting the MAC address, including a fallback to a randomly-generated address if a system one could not be obtained. Ping's version will fail if there is no way to get the address (e.g. on Windows 9X, which has no ipconfig.exe), and does not cache the address for repeated uses, making its generation of verion-1 UUIDs *very* expensive. I like the idea of having RFC-compliant UUIDs in the stdlib, but I really want immutable ones, preferably without {} in their standard string representation. And efficient use of platform-local UUID generation APIs would also be preferable. (Note that using the Windows C API for UUIDs would make it unnecessary to execute a separate program in order to get version 1 UUIDs, and the same is true for the BSD variants with a UUID API.) (In addition to the PEAK UUID implementation, OSAF's Chandler also has a compliant UUID implementation that's written entirely in C, but I haven't played with it much. I don't know whether it supports using platform APIs for generation, but it does implement hashable, immutable UUID objects.) From p.f.moore at gmail.com Tue Mar 7 17:30:31 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 7 Mar 2006 16:30:31 +0000 Subject: [Python-Dev] how about adding ping's uuid module to the standard lib ? In-Reply-To: <5.1.1.6.0.20060307103122.042c4ae8@mail.telecommunity.com> References: <5.1.1.6.0.20060307103122.042c4ae8@mail.telecommunity.com> Message-ID: <79990c6b0603070830t2aaa3407vb48ce2b6675a3068@mail.gmail.com> On 3/7/06, Phillip J. Eby wrote: > At 06:29 AM 3/7/2006 +0100, Fredrik Lundh wrote: > >see subject and http://python.org/sf/1368955 > > > >comments ? > > would be nice if you could just call UUID() to create a generic UUID in a > platform-appropriate way. PEAK's uuid module does this such that if > win32all is present, you get a Windows GUID, or if you have a FreeBSD 5+ or > NetBSD 2+ kernel you use the local platform uuidgen API. See e.g.: Given that ctypes is going to be in Python 2.5, it might be reasonable to use it to access platform-specific APIs like this. It certainly makes some sense for Windows, where running an external process is slow, and the existence of suitable external utilities (and the format of their output!) is unreliable... (I haven't looked at the code, so I don't know if this applies to Ping's use of ipconfig). Paul From thomas at python.org Tue Mar 7 17:51:50 2006 From: thomas at python.org (Thomas Wouters) Date: Tue, 7 Mar 2006 17:51:50 +0100 Subject: [Python-Dev] _bsddb.c ownership Message-ID: <9e804ac0603070851r26c2db3eje0d61c1af0005029@mail.gmail.com> Who 'owns' Modules/_bsddb.c, if anyone? The file doesn't mention whether it's a fork of pybsddb maintained separately or not. I ask because it seems to contain a number of refleaks, and I wanted to fix some of the style issues while I'm at it (and maybe even Py_ssize_t it,) but I'll happily send the patch to pybsddb instead. (Georg and I already checked in some fixes, anyway.) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060307/a0985f5b/attachment.htm From janssen at parc.com Tue Mar 7 18:06:37 2006 From: janssen at parc.com (Bill Janssen) Date: Tue, 7 Mar 2006 09:06:37 PST Subject: [Python-Dev] how about adding ping's uuid module to the standard lib ? In-Reply-To: Your message of "Tue, 07 Mar 2006 08:01:00 PST." <5.1.1.6.0.20060307103122.042c4ae8@mail.telecommunity.com> Message-ID: <06Mar7.090642pst."58633"@synergy1.parc.xerox.com> Having UUID in the stdlib would be very helpful. Philip Eby writes: > I like the idea of having RFC-compliant UUIDs in the stdlib, but I really > want immutable ones, preferably without {} in their standard string > representation. And efficient use of platform-local UUID generation APIs > would also be preferable. I completely agree with Philip. Bill From janssen at parc.com Tue Mar 7 18:42:59 2006 From: janssen at parc.com (Bill Janssen) Date: Tue, 7 Mar 2006 09:42:59 PST Subject: [Python-Dev] how about adding ping's uuid module to the standard lib ? In-Reply-To: Your message of "Tue, 07 Mar 2006 09:06:37 PST." <06Mar7.090642pst."58633"@synergy1.parc.xerox.com> Message-ID: <06Mar7.094303pst."58633"@synergy1.parc.xerox.com> > Philip Eby writes: > ... > I completely agree with Philip. Sorry, I mean of course "Phillip". Bill From g.brandl at gmx.net Tue Mar 7 18:52:16 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 07 Mar 2006 18:52:16 +0100 Subject: [Python-Dev] "as" mania In-Reply-To: References: Message-ID: [HPH the BDFL] > I suggest you file those as products of an overactive imagination. :-) At least not only mine. ;) > Have you even tried to define precise semantics for any of those, like > the expansion of "with E as V: B" in PEP 343? Easily. if expr as name: BLOCK would be equivalent to name = expr if name: BLOCK del name Same for while. Georg From guido at python.org Tue Mar 7 19:00:18 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 7 Mar 2006 10:00:18 -0800 Subject: [Python-Dev] "as" mania In-Reply-To: References: Message-ID: On 3/7/06, Georg Brandl wrote: > > Have you even tried to define precise semantics for any of those, like > > the expansion of "with E as V: B" in PEP 343? > > Easily. > > if expr as name: > BLOCK > > would be equivalent to > > name = expr > if name: > BLOCK > del name You need to be a little more formal. What happens when there are elif or else clauses? Why the del at all? (with doesn't delete VAR). You also need to present a better motivation. What would be the point of having a separate name for the value True or False when you already know its value based on which branch you execute? And what exactly is the big savings? If you look at the motivation for with E as V, V is *not* assigned the value of E (but rather E.__context__().__enter__()) and the with statement as a whole has a very good reason for its existence. Your proposal here lacks that. > Same for while. You aren't really trying, are you? What would the point be of NAME = EXPR while NAME: BLOCK del NAME (Hint: while needs EXPR to be reevaluated each time through the loop.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From g.brandl at gmx.net Tue Mar 7 18:54:17 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 07 Mar 2006 18:54:17 +0100 Subject: [Python-Dev] "as" mania In-Reply-To: References: Message-ID: Alex Martelli wrote: > I think the best use cases for 'assignment inside an if or while' > condition, as far as they go, require `capturing' a SUB-expression of > the condition, rather than the whole condition. E.g., in C, > > while ( (x=next_x()) < threshold ) ... > > being able to capture (by `as') only the whole (true or false) > condition meets a minority of the use cases, so I'm unenthusiastic > about it. There are use cases (because every value can be tested for true-ness), such as the likes of while file.readline() as line: BLOCK (which is not needed for builtin file objects, I know) or something like m = re.match(...) if m.group(1) as filename: do something with filename Georg From jcarlson at uci.edu Tue Mar 7 19:25:54 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 07 Mar 2006 10:25:54 -0800 Subject: [Python-Dev] "as" mania In-Reply-To: References: Message-ID: <20060307102337.F6BB.JCARLSON@uci.edu> Georg Brandl wrote: > or something like > > m = re.match(...) > if m.group(1) as filename: > do something with filename Except that m could be None, which would raise an exception during the .group(1) call. Perhaps you meant... m = re.match(...) if m and m.group(1) as filename: do something with filename I'm -1 on the 'as'-itis. - Josiah From g.brandl at gmx.net Tue Mar 7 19:34:18 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 07 Mar 2006 19:34:18 +0100 Subject: [Python-Dev] "as" mania In-Reply-To: References: Message-ID: Guido van Rossum wrote: > On 3/7/06, Georg Brandl wrote: >> > Have you even tried to define precise semantics for any of those, like >> > the expansion of "with E as V: B" in PEP 343? >> >> Easily. >> >> if expr as name: >> BLOCK >> >> would be equivalent to >> >> name = expr >> if name: >> BLOCK >> del name > > You need to be a little more formal. What happens when there are elif > or else clauses? Why the del at all? (with doesn't delete VAR). > > You also need to present a better motivation. What would be the point > of having a separate name for the value True or False when you already > know its value based on which branch you execute? And what exactly is > the big savings? If you look at the motivation for with E as V, V is > *not* assigned the value of E (but rather E.__context__().__enter__()) > and the with statement as a whole has a very good reason for its > existence. Your proposal here lacks that. Thinking over it, this is too much a difference between the with-"as" and my "as", so I'm abandoning this idea. My "as" would just have been a shortcut to avoid writing longish expressions that have to be checked for true-ness and then tinkered with. >> Same for while. > > You aren't really trying, are you? No ;) I would have written a PEP anyway. Cheers, Georg From ark at acm.org Tue Mar 7 20:00:48 2006 From: ark at acm.org (Andrew Koenig) Date: Tue, 7 Mar 2006 14:00:48 -0500 Subject: [Python-Dev] "as" mania In-Reply-To: Message-ID: <01c801c64219$757a6910$6402a8c0@arkdesktop> > Thinking over it, this is too much a difference between the with-"as" and > my "as", so I'm abandoning this idea. My "as" would just have been a > shortcut to avoid writing longish expressions that have to be checked for > true-ness and then tinkered with. ML has a similar feature, which you may consider as an argument either for or against it depending on your taste. The point is that ML lets you write "patterns" that decompose data structures, and the "as" usage lets you talk about the whole data structure and the decomposed one at the same time. For example, in ML, :: works like "cons" in Lisp, so that x::y is a list with a first element of x and a tail of y, which must be a list. In other words, [3, 4, 5] is equivalent to (3::4::5::nil) in ML. Now consider the following: fun merge(nil, y) = y | merge(x, nil) = x | merge (x as xh::ht, y as yh::yt) = if xh < yh then xh::merge(xt, y) else xt::merge(x, yt) Without the "as" clause, we would have had to write fun merge(nil, y) = y | merge(x, nil) = x | merge(x, y) = let val xh::xt = x val yh::yt = y in if xh < yh then xh::merge(xt, y) else xt::merge(x, yt) end which is somewhat longer and harder to follow. As it turns out, Python has similar ways of decomposing data structures: (x, y) = foo or def bar((x, y)): # etc. and I have sometimes wished I could write z as (x, y) = foo or def bar(z as (x, y)): # etc. However, it's not real high on my list of priorities, and I suspect that many Pythonists consider these usages to be a frill anyway. From martin at v.loewis.de Tue Mar 7 20:16:51 2006 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Tue, 07 Mar 2006 20:16:51 +0100 Subject: [Python-Dev] _bsddb.c ownership In-Reply-To: <9e804ac0603070851r26c2db3eje0d61c1af0005029@mail.gmail.com> References: <9e804ac0603070851r26c2db3eje0d61c1af0005029@mail.gmail.com> Message-ID: <440DDC23.5080301@v.loewis.de> Thomas Wouters wrote: > Who 'owns' Modules/_bsddb.c, if anyone? It's a fork of pybsddb, originally contributed by Gregory Smith (*). For all practical purposes, he also "owns" it, but hasn't objected to others making changes in the past. At the time it was imported, I recall the plan was to out-phase pybsddb, and only keep the version in Python. It appears that things have developed differently. http://mail.python.org/pipermail/python-dev/2002-June/025616.html http://mail.python.org/pipermail/python-dev/2002-June/025806.html http://mail.python.org/pipermail/python-dev/2002-October/029788.html http://mail.python.org/pipermail/python-dev/2002-November/030247.html Regards, Martin (*) although I did the original import. From p.f.moore at gmail.com Tue Mar 7 21:49:27 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 7 Mar 2006 20:49:27 +0000 Subject: [Python-Dev] "as" mania In-Reply-To: <01c801c64219$757a6910$6402a8c0@arkdesktop> References: <01c801c64219$757a6910$6402a8c0@arkdesktop> Message-ID: <79990c6b0603071249s30c7ec20x5be1dc50830a2383@mail.gmail.com> On 3/7/06, Andrew Koenig wrote: > As it turns out, Python has similar ways of decomposing data structures: > > (x, y) = foo > > or > > def bar((x, y)): > # etc. > > and I have sometimes wished I could write > > z as (x, y) = foo > > or > > def bar(z as (x, y)): > # etc. > > However, it's not real high on my list of priorities, and I suspect that > many Pythonists consider these usages to be a frill anyway. For the assignment case, you can do this: >>> foo = (1,2) >>> (x,y) = z = foo >>> x 1 >>> y 2 >>> z (1, 2) >>> Function arguments are not covered by this trick, but def bar(z): (x,y) = z probably isn't too much overhead... (Or did I miss your point?) Paul. From ark at acm.org Tue Mar 7 22:09:17 2006 From: ark at acm.org (Andrew Koenig) Date: Tue, 7 Mar 2006 16:09:17 -0500 Subject: [Python-Dev] "as" mania In-Reply-To: <79990c6b0603071249s30c7ec20x5be1dc50830a2383@mail.gmail.com> Message-ID: <023d01c6422b$68c844a0$6402a8c0@arkdesktop> > Function arguments are not covered by this trick, but > > def bar(z): > (x,y) = z > > probably isn't too much overhead... It's not the machine overhead, it's the intellectual overhead. I know there are some who will disagree with me, but I would find it easier to read def foo(origin as (x1, y1), corner as (x2, y2)): than to read def foo(origin, corner): (x1, y1) = origin (x2, y2) = corner It's not a big deal, but it is not completely negligible either. From theller at python.net Tue Mar 7 22:14:19 2006 From: theller at python.net (Thomas Heller) Date: Tue, 07 Mar 2006 22:14:19 +0100 Subject: [Python-Dev] str(Exception) changed, is that intended? Message-ID: I know that my unittests should not rely on this, but is this change intended? c:\sf\ctypes_head>py24 Python 2.4.2 (#67, Sep 28 2005, 12:41:11) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> str(Exception) 'exceptions.Exception' >>> ^Z c:\sf\ctypes_head>py Python 2.5a0 (trunk:42903M, Mar 7 2006, 22:01:07) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> str(Exception) "" >>> ^Z From brett at python.org Tue Mar 7 22:21:10 2006 From: brett at python.org (Brett Cannon) Date: Tue, 7 Mar 2006 13:21:10 -0800 Subject: [Python-Dev] str(Exception) changed, is that intended? In-Reply-To: References: Message-ID: On 3/7/06, Thomas Heller wrote: > I know that my unittests should not rely on this, but is this change > intended? > > c:\sf\ctypes_head>py24 > Python 2.4.2 (#67, Sep 28 2005, 12:41:11) [MSC v.1310 32 bit (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> str(Exception) > 'exceptions.Exception' > >>> ^Z > > > c:\sf\ctypes_head>py > Python 2.5a0 (trunk:42903M, Mar 7 2006, 22:01:07) [MSC v.1310 32 bit (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> str(Exception) > "" > >>> ^Z It's a side-effect of making built-in exceptions new-style classes. Not sure how you would override the string representation of a class anyway to fix this. -Brett From theller at python.net Tue Mar 7 22:19:03 2006 From: theller at python.net (Thomas Heller) Date: Tue, 07 Mar 2006 22:19:03 +0100 Subject: [Python-Dev] how about adding ping's uuid module to the standard lib ? In-Reply-To: <79990c6b0603070830t2aaa3407vb48ce2b6675a3068@mail.gmail.com> References: <5.1.1.6.0.20060307103122.042c4ae8@mail.telecommunity.com> <79990c6b0603070830t2aaa3407vb48ce2b6675a3068@mail.gmail.com> Message-ID: Paul Moore wrote: > On 3/7/06, Phillip J. Eby wrote: >> At 06:29 AM 3/7/2006 +0100, Fredrik Lundh wrote: >>> see subject and http://python.org/sf/1368955 >>> >>> comments ? >> would be nice if you could just call UUID() to create a generic UUID in a >> platform-appropriate way. PEAK's uuid module does this such that if >> win32all is present, you get a Windows GUID, or if you have a FreeBSD 5+ or >> NetBSD 2+ kernel you use the local platform uuidgen API. See e.g.: > > Given that ctypes is going to be in Python 2.5, it might be reasonable > to use it to access platform-specific APIs like this. It certainly > makes some sense for Windows, where running an external process is > slow, and the existence of suitable external utilities (and the format > of their output!) is unreliable... (I haven't looked at the code, so I > don't know if this applies to Ping's use of ipconfig). > IMO that is a perfect use-case for ctypes - access 2 or 3 platform-specific api functions. Too bad that ctypes whill be an optional module, so I'm not sure if it could be used in the Python library itself. Thomas From phd at mail2.phd.pp.ru Tue Mar 7 22:23:29 2006 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Wed, 8 Mar 2006 00:23:29 +0300 Subject: [Python-Dev] how about adding ping's uuid module to the standard lib ? In-Reply-To: References: <5.1.1.6.0.20060307103122.042c4ae8@mail.telecommunity.com> <79990c6b0603070830t2aaa3407vb48ce2b6675a3068@mail.gmail.com> Message-ID: <20060307212329.GB30899@phd.pp.ru> On Tue, Mar 07, 2006 at 10:19:03PM +0100, Thomas Heller wrote: > Too bad that ctypes whill be an optional module, so I'm > not sure if it could be used in the Python library itself. try: import ctypes except ImportError: def fallback(): ... else: def real_thing(): ... Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From theller at python.net Tue Mar 7 22:29:31 2006 From: theller at python.net (Thomas Heller) Date: Tue, 07 Mar 2006 22:29:31 +0100 Subject: [Python-Dev] how about adding ping's uuid module to the standard lib ? In-Reply-To: <20060307212329.GB30899@phd.pp.ru> References: <5.1.1.6.0.20060307103122.042c4ae8@mail.telecommunity.com> <79990c6b0603070830t2aaa3407vb48ce2b6675a3068@mail.gmail.com> <20060307212329.GB30899@phd.pp.ru> Message-ID: Oleg Broytmann wrote: > On Tue, Mar 07, 2006 at 10:19:03PM +0100, Thomas Heller wrote: >> Too bad that ctypes whill be an optional module, so I'm >> not sure if it could be used in the Python library itself. > > try: > import ctypes > except ImportError: > def fallback(): > ... > else: > def real_thing(): > ... Personally, I don't like standard modules to behave different if I install other stuff, so please change 'could' to 'should' in this sentence: >> not sure if it could be used in the Python library itself. From guido at python.org Tue Mar 7 22:35:54 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 7 Mar 2006 13:35:54 -0800 Subject: [Python-Dev] str(Exception) changed, is that intended? In-Reply-To: References: Message-ID: On 3/7/06, Brett Cannon wrote: > On 3/7/06, Thomas Heller wrote: > > I know that my unittests should not rely on this, but is this change > > intended? > > > > c:\sf\ctypes_head>py24 > > Python 2.4.2 (#67, Sep 28 2005, 12:41:11) [MSC v.1310 32 bit (Intel)] on win32 > > Type "help", "copyright", "credits" or "license" for more information. > > >>> str(Exception) > > 'exceptions.Exception' > > >>> ^Z > > > > > > c:\sf\ctypes_head>py > > Python 2.5a0 (trunk:42903M, Mar 7 2006, 22:01:07) [MSC v.1310 32 bit (Intel)] on win32 > > Type "help", "copyright", "credits" or "license" for more information. > > >>> str(Exception) > > "" > > >>> ^Z > > It's a side-effect of making built-in exceptions new-style classes. > Not sure how you would override the string representation of a class > anyway to fix this. IMO it shouldn't be fixed. Classic classes define their str to print the module name and class name with a dot in between; new-style classes use the same format as their repr. Making exceptions new-style classes is going to break a number of things; that's just inevitable. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From theller at python.net Tue Mar 7 22:33:03 2006 From: theller at python.net (Thomas Heller) Date: Tue, 07 Mar 2006 22:33:03 +0100 Subject: [Python-Dev] str(Exception) changed, is that intended? In-Reply-To: References: Message-ID: Brett Cannon wrote: > On 3/7/06, Thomas Heller wrote: >> I know that my unittests should not rely on this, but is this change >> intended? >> >> c:\sf\ctypes_head>py24 >> Python 2.4.2 (#67, Sep 28 2005, 12:41:11) [MSC v.1310 32 bit (Intel)] on win32 >> Type "help", "copyright", "credits" or "license" for more information. >>>>> str(Exception) >> 'exceptions.Exception' >>>>> ^Z >> >> c:\sf\ctypes_head>py >> Python 2.5a0 (trunk:42903M, Mar 7 2006, 22:01:07) [MSC v.1310 32 bit (Intel)] on win32 >> Type "help", "copyright", "credits" or "license" for more information. >>>>> str(Exception) >> "" >>>>> ^Z > > It's a side-effect of making built-in exceptions new-style classes. > Not sure how you would override the string representation of a class > anyway to fix this. With metaclasses, of course ;-). Seriously, I can live with the change - I just wanted to ask if it will stay, or was an oversight. Thomas From ianb at colorstudy.com Tue Mar 7 22:44:49 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 07 Mar 2006 15:44:49 -0600 Subject: [Python-Dev] quit() on the prompt Message-ID: <440DFED1.1000708@colorstudy.com> Frederick suggested a change to quit/exit a while ago, so it wasn't just a string with slight instructional purpose, but actually useful. The discussion was surprisingly involved, despite the change really trully not being that big. And everyone drifted off, too tired from the discussion to make a change. I suppose it didn't help that the original proposal struck some people as too magic, while there were some more substantive problems brought up as well, and when you mix aesthetic with technical concerns everyone gets all distracted and worked up. Anyway, I would like to re-propose one of the ideas that came up (originally from Ping?): class Quitter(object): def __init__(self, name): self.name = name def __repr__(self): return 'Use %s() to exit' % self.name def __call__(self): raise SystemExit() quit = Quitter('quit') exit = Quitter('exit') This is not very magical, but I think is more helpful than the current behavior. It does not satisfy the "just do what I said" argument for not requiring the call (quit() not quit), but eh -- I guess it seemed like everything that didn't require a call had some scary corner case where the interpreter would abruptly exit. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From guido at python.org Tue Mar 7 22:53:00 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 7 Mar 2006 13:53:00 -0800 Subject: [Python-Dev] quit() on the prompt In-Reply-To: <440DFED1.1000708@colorstudy.com> References: <440DFED1.1000708@colorstudy.com> Message-ID: Works for me. On 3/7/06, Ian Bicking wrote: > Frederick suggested a change to quit/exit a while ago, so it wasn't just > a string with slight instructional purpose, but actually useful. The > discussion was surprisingly involved, despite the change really trully > not being that big. And everyone drifted off, too tired from the > discussion to make a change. I suppose it didn't help that the original > proposal struck some people as too magic, while there were some more > substantive problems brought up as well, and when you mix aesthetic with > technical concerns everyone gets all distracted and worked up. Anyway, > I would like to re-propose one of the ideas that came up (originally > from Ping?): > > class Quitter(object): > def __init__(self, name): > self.name = name > def __repr__(self): > return 'Use %s() to exit' % self.name > def __call__(self): > raise SystemExit() > quit = Quitter('quit') > exit = Quitter('exit') > > This is not very magical, but I think is more helpful than the current > behavior. It does not satisfy the "just do what I said" argument for > not requiring the call (quit() not quit), but eh -- I guess it seemed > like everything that didn't require a call had some scary corner case > where the interpreter would abruptly exit. > > -- > Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Tue Mar 7 22:56:11 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 07 Mar 2006 16:56:11 -0500 Subject: [Python-Dev] str(Exception) changed, is that intended? In-Reply-To: References: Message-ID: <1141768571.32270.274.camel@resist.wooz.org> On Tue, 2006-03-07 at 13:35 -0800, Guido van Rossum wrote: > IMO it shouldn't be fixed. Classic classes define their str to print > the module name and class name with a dot in between; new-style > classes use the same format as their repr. Making exceptions new-style > classes is going to break a number of things; that's just inevitable. What else do you expect to break? Should we at least try to describe expected breakage in PEP 352? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060307/a3b91d14/attachment.pgp From guido at python.org Tue Mar 7 23:11:14 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 7 Mar 2006 14:11:14 -0800 Subject: [Python-Dev] str(Exception) changed, is that intended? In-Reply-To: <1141768571.32270.274.camel@resist.wooz.org> References: <1141768571.32270.274.camel@resist.wooz.org> Message-ID: On 3/7/06, Barry Warsaw wrote: > On Tue, 2006-03-07 at 13:35 -0800, Guido van Rossum wrote: > > > IMO it shouldn't be fixed. Classic classes define their str to print > > the module name and class name with a dot in between; new-style > > classes use the same format as their repr. Making exceptions new-style > > classes is going to break a number of things; that's just inevitable. > > What else do you expect to break? Should we at least try to describe > expected breakage in PEP 352? Anything that depends on the differences in behavior between classic and new-style classes, e.g. multiple inheritance if the inheritance graph contains a diamond, or type(exc) having a specific value (namely, the metaclass for classic classes), or certain broken behaviors (like read-only properties not being really read-only). It's hard to make a complete list. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Tue Mar 7 23:36:48 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 08 Mar 2006 08:36:48 +1000 Subject: [Python-Dev] quit() on the prompt In-Reply-To: <440DFED1.1000708@colorstudy.com> References: <440DFED1.1000708@colorstudy.com> Message-ID: <440E0B00.6000001@gmail.com> Ian Bicking wrote: > class Quitter(object): > def __init__(self, name): > self.name = name > def __repr__(self): > return 'Use %s() to exit' % self.name > def __call__(self): > raise SystemExit() > quit = Quitter('quit') > exit = Quitter('exit') > > This is not very magical, but I think is more helpful than the current > behavior. It does not satisfy the "just do what I said" argument for > not requiring the call (quit() not quit), but eh -- I guess it seemed > like everything that didn't require a call had some scary corner case > where the interpreter would abruptly exit. > It also nicely aligns with the way that license() and help() already work at the interactive prompt. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From bjourne at gmail.com Wed Mar 8 00:20:19 2006 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Wed, 8 Mar 2006 00:20:19 +0100 Subject: [Python-Dev] quit() on the prompt In-Reply-To: <440DFED1.1000708@colorstudy.com> References: <440DFED1.1000708@colorstudy.com> Message-ID: <740c3aec0603071520n23cab2e3xf976108de7f508ea@mail.gmail.com> do { cmd = readline() do_stuff_with_cmd(cmd); } while (!strcmp(cmd, "quit")); printf("Bye!"); exit(0); KISS? -- mvh Bj?rn From crutcher at gmail.com Wed Mar 8 00:26:17 2006 From: crutcher at gmail.com (Crutcher Dunnavant) Date: Tue, 7 Mar 2006 15:26:17 -0800 Subject: [Python-Dev] quit() on the prompt In-Reply-To: <740c3aec0603071520n23cab2e3xf976108de7f508ea@mail.gmail.com> References: <440DFED1.1000708@colorstudy.com> <740c3aec0603071520n23cab2e3xf976108de7f508ea@mail.gmail.com> Message-ID: I am probably the biggest proponent of magic variables, but this just won't work. First, commands and lines are not the same thing, so: print \ exit breaks your propossal. Second, quit and exit are bindable variables, and you need to be sure that they still mean _quit_, and not something else. On 3/7/06, BJ?rn Lindqvist wrote: > do { > cmd = readline() > do_stuff_with_cmd(cmd); > } while (!strcmp(cmd, "quit")); > printf("Bye!"); > exit(0); > > KISS? > > -- > mvh Bj?rn > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/crutcher%40gmail.com > -- Crutcher Dunnavant littlelanguages.com monket.samedi-studios.com From ianb at colorstudy.com Wed Mar 8 00:31:01 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 07 Mar 2006 17:31:01 -0600 Subject: [Python-Dev] quit() on the prompt In-Reply-To: <740c3aec0603071520n23cab2e3xf976108de7f508ea@mail.gmail.com> References: <440DFED1.1000708@colorstudy.com> <740c3aec0603071520n23cab2e3xf976108de7f508ea@mail.gmail.com> Message-ID: <440E17B5.3020108@colorstudy.com> BJ?rn Lindqvist wrote: > do { > cmd = readline() > do_stuff_with_cmd(cmd); > } while (!strcmp(cmd, "quit")); > printf("Bye!"); > exit(0); > > KISS? I believe there were concerns that rebinding quit would cause strange behavior. E.g.: >>> quit = False >>> while not quit: ... >>> quit $ Or: >>> if raw_input('quit?') == 'yes': ... quit will that work? Should it? Functions are pretty predictable in comparison to these other options. So, at least to me, quit() == KISS -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From brett at python.org Wed Mar 8 00:33:05 2006 From: brett at python.org (Brett Cannon) Date: Tue, 7 Mar 2006 15:33:05 -0800 Subject: [Python-Dev] quit() on the prompt In-Reply-To: <440DFED1.1000708@colorstudy.com> References: <440DFED1.1000708@colorstudy.com> Message-ID: On 3/7/06, Ian Bicking wrote: > Frederick suggested a change to quit/exit a while ago, so it wasn't just > a string with slight instructional purpose, but actually useful. The > discussion was surprisingly involved, despite the change really trully > not being that big. And everyone drifted off, too tired from the > discussion to make a change. I suppose it didn't help that the original > proposal struck some people as too magic, while there were some more > substantive problems brought up as well, and when you mix aesthetic with > technical concerns everyone gets all distracted and worked up. Anyway, > I would like to re-propose one of the ideas that came up (originally > from Ping?): > > class Quitter(object): > def __init__(self, name): > self.name = name > def __repr__(self): > return 'Use %s() to exit' % self.name > def __call__(self): > raise SystemExit() > quit = Quitter('quit') > exit = Quitter('exit') > > This is not very magical, but I think is more helpful than the current > behavior. It does not satisfy the "just do what I said" argument for > not requiring the call (quit() not quit), but eh -- I guess it seemed > like everything that didn't require a call had some scary corner case > where the interpreter would abruptly exit. +1 from me. Only change I would make is pass an argument to SystemExit() such as "%s() called", although the chances of this exception being caught is very slim. -Brett From thomas at python.org Wed Mar 8 00:37:47 2006 From: thomas at python.org (Thomas Wouters) Date: Wed, 8 Mar 2006 00:37:47 +0100 Subject: [Python-Dev] quit() on the prompt In-Reply-To: References: <440DFED1.1000708@colorstudy.com> Message-ID: <9e804ac0603071537p10cb7ae0k25f97cca9048656f@mail.gmail.com> On 3/8/06, Brett Cannon wrote: > > On 3/7/06, Ian Bicking wrote: > > class Quitter(object): > > def __init__(self, name): > > self.name = name > > def __repr__(self): > > return 'Use %s() to exit' % self.name > > def __call__(self): > > raise SystemExit() > > quit = Quitter('quit') > > exit = Quitter('exit') > > +1 from me. Only change I would make is pass an argument to > SystemExit() such as "%s() called", although the chances of this > exception being caught is very slim. Raising SystemExit("quit() called") has an additional benefit (although the wording could use some work): >>> raise SystemExit("quit() called") quit() called (At least, I consider that a benefit :-) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060308/a3808aba/attachment.html From brett at python.org Wed Mar 8 00:38:31 2006 From: brett at python.org (Brett Cannon) Date: Tue, 7 Mar 2006 15:38:31 -0800 Subject: [Python-Dev] str(Exception) changed, is that intended? In-Reply-To: References: <1141768571.32270.274.camel@resist.wooz.org> Message-ID: On 3/7/06, Guido van Rossum wrote: > On 3/7/06, Barry Warsaw wrote: > > On Tue, 2006-03-07 at 13:35 -0800, Guido van Rossum wrote: > > > > > IMO it shouldn't be fixed. Classic classes define their str to print > > > the module name and class name with a dot in between; new-style > > > classes use the same format as their repr. Making exceptions new-style > > > classes is going to break a number of things; that's just inevitable. > > > > What else do you expect to break? Should we at least try to describe > > expected breakage in PEP 352? > > Anything that depends on the differences in behavior between classic > and new-style classes, e.g. multiple inheritance if the inheritance > graph contains a diamond, or type(exc) having a specific value > (namely, the metaclass for classic classes), or certain broken > behaviors (like read-only properties not being really read-only). > > It's hard to make a complete list. Right, stuff dealing with the type could break. Instance-related stuff dealing with the interface will continue to work as expected and be fully backwards-compatible (unless someone complains about now having a proper __unicode__ method, in which case it will definitely go in one ear and out the other for me). -Brett From jimjjewett at gmail.com Wed Mar 8 01:07:22 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Tue, 7 Mar 2006 19:07:22 -0500 Subject: [Python-Dev] quit() on the prompt Message-ID: Ian reproposed: class Quitter(object): def __init__(self, name): self.name = name def __repr__(self): return 'Use %s() to exit' % self.name def __call__(self): raise SystemExit() The one change I would suggest is the string used in repr. Some of the objections to the current behavior boil down to "If it knows what I mean, why didn't it do it?" We probably can't explain first-class callables in a short message, but we could emphasize that it has to be *call*ed. Perhaps def __repr__(self): return 'Calling %s() will exit python' % self.name -jJ From greg.ewing at canterbury.ac.nz Wed Mar 8 01:32:20 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 08 Mar 2006 13:32:20 +1300 Subject: [Python-Dev] conditional expressions - add parens? In-Reply-To: References: Message-ID: <440E2614.8050508@canterbury.ac.nz> Jeremy Hylton wrote: > Perhaps the solution > is to require parens around all expressions, a simple consistent rule. I actually designed a language with that feature once. It was an exercise in minimality, with hardly anything built-in -- all the arithmetic operators, etc. were defined in the language. A result was that there was no built-in notion of precedence, and my solution was to require parentheses around every infix operation. So instead of dsq = b * b - 4 * a * c you would have had to write dsq = ((b * b) - ((4 * a) * c)) I never got an implementation working well enough to find out how much of a disaster this would have been to use, though. :-) -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Wed Mar 8 01:36:01 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 08 Mar 2006 13:36:01 +1300 Subject: [Python-Dev] conditional expressions - add parens? In-Reply-To: <79990c6b0603070515o579565c7t796b5e12a902a362@mail.gmail.com> References: <79990c6b0603070515o579565c7t796b5e12a902a362@mail.gmail.com> Message-ID: <440E26F1.9020102@canterbury.ac.nz> Paul Moore wrote: > +0 for mentioning parens around conditional expressions in PEP 8. But > it's aready covered by the general "code should be readable" in my > view. Indeed. Writing readable code is ultimately the responsibility of the person doing the writing. It's enough that you *can* put parentheses around things if it helps readability. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Wed Mar 8 01:42:11 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 08 Mar 2006 13:42:11 +1300 Subject: [Python-Dev] "as" mania In-Reply-To: References: Message-ID: <440E2863.4060209@canterbury.ac.nz> Alex Martelli wrote: > On Mar 7, 2006, at 6:15 AM, Georg Brandl wrote: >>with expr as f: >> do something with f I expect the "with" example here is a red herring, not intended to have anything to do with the new "with" statement that's just been added. > I think the best use cases for 'assignment inside an if or while' > condition, as far as they go, require `capturing' a SUB-expression of > the condition, rather than the whole condition. E.g., in C, > > while ( (x=next_x()) < threshold ) ... IIUC, that would be while (next_x() as x) < threshold: ... i.e. 'x as y' would be an expression. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From fumanchu at amor.org Wed Mar 8 02:04:30 2006 From: fumanchu at amor.org (Robert Brewer) Date: Tue, 7 Mar 2006 17:04:30 -0800 Subject: [Python-Dev] conditional expressions - add parens? Message-ID: <435DF58A933BA74397B42CDEB8145A8666CFA5@ex9.hostedexchange.local> Greg Ewing wrote: > Jeremy Hylton wrote: > > Perhaps the solution > > is to require parens around all expressions, a simple > consistent rule. > > I actually designed a language with that feature once. > It was an exercise in minimality, with hardly anything > built-in -- all the arithmetic operators, etc. were > defined in the language. > > A result was that there was no built-in notion of > precedence, and my solution was to require parentheses > around every infix operation. So instead of > > dsq = b * b - 4 * a * c > > you would have had to write > > dsq = ((b * b) - ((4 * a) * c)) > > I never got an implementation working well enough > to find out how much of a disaster this would > have been to use, though. :-) I already do that anyway, and even update other people's code in any open-source projects I contribute to, because I find it *far* easier to read and write 'unnecessary' parens than remember precedence rules. But I can understand why some people would balk at it, so +0.5 from me. ;) Robert Brewer System Architect Amor Ministries fumanchu at amor.org From thomas at python.org Wed Mar 8 03:03:48 2006 From: thomas at python.org (Thomas Wouters) Date: Wed, 8 Mar 2006 03:03:48 +0100 Subject: [Python-Dev] _bsddb.c ownership In-Reply-To: <440DDC23.5080301@v.loewis.de> References: <9e804ac0603070851r26c2db3eje0d61c1af0005029@mail.gmail.com> <440DDC23.5080301@v.loewis.de> Message-ID: <9e804ac0603071803u2e071929hcabd85a36ddea5b@mail.gmail.com> On 3/7/06, "Martin v. L?wis" wrote: > > Thomas Wouters wrote: > > Who 'owns' Modules/_bsddb.c, if anyone? > > It's a fork of pybsddb, originally contributed by Gregory Smith (*). > For all practical purposes, he also "owns" it, but hasn't objected > to others making changes in the past. > > At the time it was imported, I recall the plan was to out-phase > pybsddb, and only keep the version in Python. It appears that things > have developed differently. Alright. It seems someone already made backward-incompatible changes to _bsddb.c (adding Py_ssize_t use), and various other things were fixed, too, so I just went ahead and checked in a bunch of small fixes. I'm not done yet, though. After I am done (which might be a while still, but hopefully before alpha1), I'll fix backward compatibility and send in a patch to the pybsddb project :) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060308/c1ce7c62/attachment.html From nnorwitz at gmail.com Wed Mar 8 08:02:24 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 7 Mar 2006 23:02:24 -0800 Subject: [Python-Dev] Long-time shy failure in test_socket_ssl In-Reply-To: <1f7befae0602272048x50d260b3i29819389709e0ab6@mail.gmail.com> References: <1f7befae0601241422sd783c37u5d676893de2f6055@mail.gmail.com> <1f7befae0601241553v10f7dd41i364e92b98f9de4a2@mail.gmail.com> <1f7befae0602272048x50d260b3i29819389709e0ab6@mail.gmail.com> Message-ID: On 2/27/06, Tim Peters wrote: > > Neal plugged another hole later, but-- alas --I have seen the same shy > failure since then on WinXP. One of the most recent buildbot test > runs saw it too, on a non-Windows box: > > http://www.python.org/dev/buildbot/trunk/g5%20osx.3%20trunk/builds/204/step-test/0 > > test_socket_ssl > test test_socket_ssl crashed -- > exceptions.TypeError: 'NoneType' object is not callable > > in the second test run there. For closure, I believe this problem was addressed by revs 42842 and 42844 to Lib/test/test_importhooks.py. If anyone sees spurious failures with the buildbot (one time failures, crashes, etc), please report the problems to python-dev. It would be great to see if you can reproduce the results with the same tests that failed. We need to determine if it is architecture specific, test-order related, or something else. n From steve at holdenweb.com Wed Mar 8 08:05:07 2006 From: steve at holdenweb.com (Steve Holden) Date: Wed, 08 Mar 2006 07:05:07 +0000 Subject: [Python-Dev] "as" mania In-Reply-To: <023d01c6422b$68c844a0$6402a8c0@arkdesktop> References: <79990c6b0603071249s30c7ec20x5be1dc50830a2383@mail.gmail.com> <023d01c6422b$68c844a0$6402a8c0@arkdesktop> Message-ID: Andrew Koenig wrote: >>Function arguments are not covered by this trick, but >> >> def bar(z): >> (x,y) = z >> >>probably isn't too much overhead... > > > It's not the machine overhead, it's the intellectual overhead. I know there > are some who will disagree with me, but I would find it easier to read > > def foo(origin as (x1, y1), corner as (x2, y2)): > > than to read > > def foo(origin, corner): > (x1, y1) = origin > (x2, y2) = corner > > It's not a big deal, but it is not completely negligible either. > On the third hand, what about >>> def foo((x1, y1), (x2, y2)): ... print x1, y1, x2, y2 ... >>> origin = (10, 1) >>> corner = (20, 2) >>> foo(origin, corner) 10 1 20 2 >>> Follow-ups set to comp.lang.python :-) regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd www.holdenweb.com Love me, love my blog holdenweb.blogspot.com From thomas at python.org Wed Mar 8 12:06:40 2006 From: thomas at python.org (Thomas Wouters) Date: Wed, 8 Mar 2006 12:06:40 +0100 Subject: [Python-Dev] mail.python.org disruption Message-ID: <9e804ac0603080306g6dfca8e7j94c9b4966ae9f171@mail.gmail.com> [ Mailed to python-dev and python-list, as that should cover most of the users ;P ] There was a slight disruption on mail.python.org this morning. For about three and a half hours, it was rejecting most of its mail with the message: Client host [....] blocked using singlehop.dsbl.org; Your mail has been rejected because the server you are sending to is misconfigured. The error means mail.python.org was using singlehop.dsbl.org as a DNSBL list, but that list doesn't exist, so it rejects all mail. Someone (not me :) added that blacklist at 08:15 local time (07:15 GMT) and I fixed it at 11:46 (10:46 GMT). Blame lingering PyCon-jetlag for me not catching it earlier, sorry. About 7759 mails were bounced, although a decent portion of them will have been actual spam (of which python.org gets massive amounts.) If you sent legitimate mail in that period, and got a bounce back with a message like the one above, it's safe to re-send it now. Sorry for the inconvenience. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060308/24f96825/attachment.htm From g.brandl at gmx.net Wed Mar 8 12:10:40 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 08 Mar 2006 12:10:40 +0100 Subject: [Python-Dev] quit() on the prompt In-Reply-To: References: Message-ID: Jim Jewett wrote: > Ian reproposed: > > class Quitter(object): > def __init__(self, name): > self.name = name > def __repr__(self): > return 'Use %s() to exit' % self.name > def __call__(self): > raise SystemExit() > > The one change I would suggest is the string used in repr. > > Some of the objections to the current behavior boil down to > "If it knows what I mean, why didn't it do it?" We probably > can't explain first-class callables in a short message, but > we could emphasize that it has to be *call*ed. Perhaps > > def __repr__(self): > return 'Calling %s() will exit python' % self.name +1. I also think that the old way should still be mentioned, as it is much shorter to type. Georg From theller at python.net Wed Mar 8 12:20:48 2006 From: theller at python.net (Thomas Heller) Date: Wed, 08 Mar 2006 12:20:48 +0100 Subject: [Python-Dev] mail.python.org disruption In-Reply-To: <9e804ac0603080306g6dfca8e7j94c9b4966ae9f171@mail.gmail.com> References: <9e804ac0603080306g6dfca8e7j94c9b4966ae9f171@mail.gmail.com> Message-ID: Thomas Wouters wrote: > [ Mailed to python-dev and python-list, as that should cover most of the > users ;P ] > > There was a slight disruption on mail.python.org this morning. For about > three and a half hours, it was rejecting most of its mail with the message: > > Client host [....] blocked using singlehop.dsbl.org; Your mail has been > rejected because the server you are sending to is misconfigured. > > The error means mail.python.org was using singlehop.dsbl.org as a DNSBL > list, but that list doesn't exist, so it rejects all mail. Someone (not me > :) added that blacklist at 08:15 local time (07:15 GMT) and I fixed it at > 11:46 (10:46 GMT). Blame lingering PyCon-jetlag for me not catching it > earlier, sorry. About 7759 mails were bounced, although a decent portion of > them will have been actual spam (of which python.org gets massive amounts.) > If you sent legitimate mail in that period, and got a bounce back with a > message like the one above, it's safe to re-send it now. > > Sorry for the inconvenience. Has this also to do with that http://svn.python.org/ doesn't work? From thomas at python.org Wed Mar 8 12:33:48 2006 From: thomas at python.org (Thomas Wouters) Date: Wed, 8 Mar 2006 12:33:48 +0100 Subject: [Python-Dev] mail.python.org disruption In-Reply-To: References: <9e804ac0603080306g6dfca8e7j94c9b4966ae9f171@mail.gmail.com> Message-ID: <9e804ac0603080333p25498e8y5705b1fce6bdf01a@mail.gmail.com> On 3/8/06, Thomas Heller wrote: > > Thomas Wouters wrote: > > There was a slight disruption on mail.python.org this morning. > > Has this also to do with that http://svn.python.org/ doesn't work? No, it doesn't, although my fixing that (almost) is what made me notice mail.python.org wasn't working right. svn.python.org should be working fine for SVN checkouts and the like, it's just viewcvs that isn't working yet. I'll look at it. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060308/fb564e23/attachment.htm From thomas at python.org Wed Mar 8 12:39:19 2006 From: thomas at python.org (Thomas Wouters) Date: Wed, 8 Mar 2006 12:39:19 +0100 Subject: [Python-Dev] mail.python.org disruption In-Reply-To: <9e804ac0603080333p25498e8y5705b1fce6bdf01a@mail.gmail.com> References: <9e804ac0603080306g6dfca8e7j94c9b4966ae9f171@mail.gmail.com> <9e804ac0603080333p25498e8y5705b1fce6bdf01a@mail.gmail.com> Message-ID: <9e804ac0603080339n6644a986y6ea541da6ddb07dc@mail.gmail.com> On 3/8/06, Thomas Wouters wrote: > > svn.python.org should be working fine for SVN checkouts and the like, it's > just viewcvs that isn't working yet. I'll look at it. > Actually, it all seems to work fine now. Let me know of any specific problems if you see any :) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060308/de986649/attachment.html From phd at mail2.phd.pp.ru Wed Mar 8 12:45:55 2006 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Wed, 8 Mar 2006 14:45:55 +0300 Subject: [Python-Dev] quit() on the prompt In-Reply-To: <9e804ac0603071537p10cb7ae0k25f97cca9048656f@mail.gmail.com> References: <440DFED1.1000708@colorstudy.com> <9e804ac0603071537p10cb7ae0k25f97cca9048656f@mail.gmail.com> Message-ID: <20060308114555.GA23359@phd.pp.ru> On Wed, Mar 08, 2006 at 12:37:47AM +0100, Thomas Wouters wrote: > Raising SystemExit("quit() called") has an additional benefit (although the > wording could use some work): > > >>> raise SystemExit("quit() called") > quit() called > > (At least, I consider that a benefit :-) It has a bad side-effect of returning an error code to the calling shell: $ python && echo Ok || echo Error\! Python 2.4.2 (#1, Oct 3 2005, 20:57:52) [GCC 3.3.5 (Debian 1:3.3.5-13)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> raise SystemExit("quit() called") quit() called Error! (At least, I consider that a bad side-effect.) Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From steve at holdenweb.com Wed Mar 8 13:39:51 2006 From: steve at holdenweb.com (Steve Holden) Date: Wed, 08 Mar 2006 12:39:51 +0000 Subject: [Python-Dev] quit() on the prompt In-Reply-To: <20060308114555.GA23359@phd.pp.ru> References: <440DFED1.1000708@colorstudy.com> <9e804ac0603071537p10cb7ae0k25f97cca9048656f@mail.gmail.com> <20060308114555.GA23359@phd.pp.ru> Message-ID: Oleg Broytmann wrote: > On Wed, Mar 08, 2006 at 12:37:47AM +0100, Thomas Wouters wrote: > >>Raising SystemExit("quit() called") has an additional benefit (although the >>wording could use some work): >> >> >>>>>raise SystemExit("quit() called") >> >>quit() called >> >>(At least, I consider that a benefit :-) > > > It has a bad side-effect of returning an error code to the calling > shell: > > $ python && echo Ok || echo Error\! > Python 2.4.2 (#1, Oct 3 2005, 20:57:52) > [GCC 3.3.5 (Debian 1:3.3.5-13)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>>>raise SystemExit("quit() called") > > quit() called > Error! > > (At least, I consider that a bad side-effect.) I should imagine the use cases for running an interactive Python shell as a part of a script are fairly few and far between, though. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd www.holdenweb.com Love me, love my blog holdenweb.blogspot.com From phd at mail2.phd.pp.ru Wed Mar 8 13:52:50 2006 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Wed, 8 Mar 2006 15:52:50 +0300 Subject: [Python-Dev] quit() on the prompt In-Reply-To: References: <440DFED1.1000708@colorstudy.com> <9e804ac0603071537p10cb7ae0k25f97cca9048656f@mail.gmail.com> <20060308114555.GA23359@phd.pp.ru> Message-ID: <20060308125250.GA25869@phd.pp.ru> On Wed, Mar 08, 2006 at 12:39:51PM +0000, Steve Holden wrote: > Oleg Broytmann wrote: > >>>>raise SystemExit("quit() called") > > > > quit() called > > Error! > > > I should imagine the use cases for running an interactive Python shell > as a part of a script are fairly few and far between, though. IDEs. Edit a code in an editor, run python -i script.py, investigate the environment, return to the editor, get error message. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From guido at python.org Wed Mar 8 16:39:46 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 8 Mar 2006 07:39:46 -0800 Subject: [Python-Dev] quit() on the prompt In-Reply-To: <9e804ac0603071537p10cb7ae0k25f97cca9048656f@mail.gmail.com> References: <440DFED1.1000708@colorstudy.com> <9e804ac0603071537p10cb7ae0k25f97cca9048656f@mail.gmail.com> Message-ID: On 3/7/06, Thomas Wouters wrote: > +1 from me. Only change I would make is pass an argument to > SystemExit() such as "%s() called", although the chances of this > exception being caught is very slim. > > > Raising SystemExit("quit() called") has an additional benefit (although the > wording could use some work): > > >>> raise SystemExit("quit() called") > quit() called > > (At least, I consider that a benefit :-) Bad idea, as several pointed out -- quit() should return a 0 exit to the shell. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Mar 8 16:52:05 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 8 Mar 2006 07:52:05 -0800 Subject: [Python-Dev] __exit__ API? In-Reply-To: References: Message-ID: I have a patch for this now -- python.org/sf/1445739. Please review. --Guido On 3/3/06, Guido van Rossum wrote: > A few days ago there were rumbling noises that requiring __exit__ to > re-raise the exception (as I amended PEP 343 at the time) could lead > to easily-missed bugs in __exit__ handlers. > > After thinking it over I think I agree and I think I'd like to change > the API so that the exception is only ignored if __exit__ returns a > "true" value. > > The easiest implementation is probably to just let the WITH_CLEANUP > opcode do everything. This becomes a rather heavy opcode then but the > alternative is to generate very hairy code (like the original patch > did, full of ROT 4 choruses). > > Any objections? I probably won't get to this until Monday. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz at pythoncraft.com Wed Mar 8 16:58:49 2006 From: aahz at pythoncraft.com (Aahz) Date: Wed, 8 Mar 2006 07:58:49 -0800 Subject: [Python-Dev] 2.5 release schedule? Message-ID: <20060308155848.GA11872@panix.com> Have we reached final agreement on the 2.5 release schedule? The last message was on Feb 15, which said: alpha 1: May 6, 2006 [planned] alpha 2: June 3, 2006 [planned] alpha 3: July 1, 2006 [planned] beta 1: July 29, 2006 [planned] beta 2: August 26, 2006 [planned] rc 1: September 16, 2006 [planned] final: September 30, 2006 [planned] but http://www.python.org/doc/peps/pep-0356/ still has alpha 1: April 1, 2006 [planned] alpha 2: April 29, 2006 [planned] alpha 3: May 27, 2006 [planned] beta 1: June 24, 2006 [planned] beta 2: July 15, 2006 [planned] rc 1: August 5, 2006 [planned] final: August 19, 2006 [planned] Can we complete this? (Or does the actual PEP have the correct schedule and the web site re-org didn't get the published version updated?) -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From guido at python.org Wed Mar 8 18:39:10 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 8 Mar 2006 09:39:10 -0800 Subject: [Python-Dev] 2.5 release schedule? In-Reply-To: <20060308155848.GA11872@panix.com> References: <20060308155848.GA11872@panix.com> Message-ID: If anything, we're probably going to do it earlier even. All schedules are tentative, BTW; the PSF is not responsible for losses due to schedule changes. :-) --Guido On 3/8/06, Aahz wrote: > Have we reached final agreement on the 2.5 release schedule? The last > message was on Feb 15, which said: > > alpha 1: May 6, 2006 [planned] > alpha 2: June 3, 2006 [planned] > alpha 3: July 1, 2006 [planned] > beta 1: July 29, 2006 [planned] > beta 2: August 26, 2006 [planned] > rc 1: September 16, 2006 [planned] > final: September 30, 2006 [planned] > > but http://www.python.org/doc/peps/pep-0356/ still has > > alpha 1: April 1, 2006 [planned] > alpha 2: April 29, 2006 [planned] > alpha 3: May 27, 2006 [planned] > beta 1: June 24, 2006 [planned] > beta 2: July 15, 2006 [planned] > rc 1: August 5, 2006 [planned] > final: August 19, 2006 [planned] > > Can we complete this? (Or does the actual PEP have the correct schedule > and the web site re-org didn't get the published version updated?) > -- > Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ > > "19. A language that doesn't affect the way you think about programming, > is not worth knowing." --Alan Perlis > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From nas at arctrix.com Wed Mar 8 19:00:14 2006 From: nas at arctrix.com (Neil Schemenauer) Date: Wed, 8 Mar 2006 18:00:14 +0000 (UTC) Subject: [Python-Dev] quit() on the prompt References: <440DFED1.1000708@colorstudy.com> <9e804ac0603071537p10cb7ae0k25f97cca9048656f@mail.gmail.com> Message-ID: Guido van Rossum wrote: > Bad idea, as several pointed out -- quit() should return a 0 exit > to the shell. I like the idea of making "quit" callable. One small concern I have is that people will use it in scripts to exit (rather than one of the other existing ways to exit). OTOH, maybe that's a feature. Neil From ianb at colorstudy.com Wed Mar 8 19:11:31 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 08 Mar 2006 12:11:31 -0600 Subject: [Python-Dev] quit() on the prompt In-Reply-To: References: <440DFED1.1000708@colorstudy.com> <9e804ac0603071537p10cb7ae0k25f97cca9048656f@mail.gmail.com> Message-ID: <440F1E53.9020100@colorstudy.com> Neil Schemenauer wrote: >>Bad idea, as several pointed out -- quit() should return a 0 exit >>to the shell. > > > I like the idea of making "quit" callable. One small concern I have > is that people will use it in scripts to exit (rather than one of > the other existing ways to exit). OTOH, maybe that's a feature. I actually thought it was only defined for interactive sessions, but a brief test shows I was wrong. It doesn't bother me, but it does make me think that exit(1) should exit with a code of one. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From tim.peters at gmail.com Wed Mar 8 20:44:03 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 8 Mar 2006 14:44:03 -0500 Subject: [Python-Dev] Long-time shy failure in test_socket_ssl In-Reply-To: <1f7befae0603072357i4347ceeme670db49a20f15eb@mail.gmail.com> References: <1f7befae0601241422sd783c37u5d676893de2f6055@mail.gmail.com> <1f7befae0601241553v10f7dd41i364e92b98f9de4a2@mail.gmail.com> <1f7befae0602272048x50d260b3i29819389709e0ab6@mail.gmail.com> <1f7befae0603072357i4347ceeme670db49a20f15eb@mail.gmail.com> Message-ID: <1f7befae0603081144t1debefc9yf8af0d2ee646f458@mail.gmail.com> [Tim] >> Neal plugged another hole later, but-- alas --I have seen the same shy >> failure since then on WinXP. One of the most recent buildbot test >> runs saw it too, on a non-Windows box: >> >> http://www.python.org/dev/buildbot/trunk/g5%20osx.3%20trunk/builds/204/step-test/0 >> >> test_socket_ssl >> test test_socket_ssl crashed -- >> exceptions.TypeError: 'NoneType' object is not callable >> >> in the second test run there. [Neal] > For closure, I believe this problem was addressed by revs 42842 and > 42844 to Lib/test/test_importhooks.py. I agree the mystery-change there made the problem go away (yay!), but would prefer to reserve "addressed" for some notion of "understood". Who understands test_importhooks? Without Neal's change to delete "urllib" from sys.modules there, and when running tests in a specific order, test_socket_ssl dies because the module-global `urlparse` magically becomes None on line 147 of urlparse.py's urljoin() function. What sins does test_importhooks commit to provoke this? What else does running test_importhooks distort? That is, what else should test_importhooks be doing to clean up after itself that it's not currently doing (in the absence of understanding the original problem, there's no apparent answer)? > If anyone sees spurious failures with the buildbot (one time failures, > crashes, etc), please report the problems to python-dev. It would be > great to see if you can reproduce the results with the same tests that > failed. We need to determine if it is architecture specific, > test-order related, or something else. Yes, it's loads of fun :-) From jcarlson at uci.edu Wed Mar 8 20:47:26 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Wed, 08 Mar 2006 11:47:26 -0800 Subject: [Python-Dev] quit() on the prompt In-Reply-To: <440F1E53.9020100@colorstudy.com> References: <440F1E53.9020100@colorstudy.com> Message-ID: <20060308114610.F6D2.JCARLSON@uci.edu> Ian Bicking wrote: > > Neil Schemenauer wrote: > >>Bad idea, as several pointed out -- quit() should return a 0 exit > >>to the shell. > > > > > > I like the idea of making "quit" callable. One small concern I have > > is that people will use it in scripts to exit (rather than one of > > the other existing ways to exit). OTOH, maybe that's a feature. > > I actually thought it was only defined for interactive sessions, but a > brief test shows I was wrong. It doesn't bother me, but it does make me > think that exit(1) should exit with a code of one. Sounds like the quit/exit bits are becoming aliases for sys.exit with special docstrings. That would be fine with me. - Josiah From g.brandl at gmx.net Wed Mar 8 22:23:52 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 08 Mar 2006 22:23:52 +0100 Subject: [Python-Dev] Bug Day? Message-ID: Hi, I know, PyCon's just been, but not many bugs were closed and there really ought to be some issues resolved before 2.4.3 happens. The number of open bugs is again crawling to 900. I myself are looking at many bugs and patches over time, but with most of them I can't decide alone what to do. Writing here every time doesn't seem like the most sensible thing too. So, is there interest? Can I do anything to make it easier? Cheers, Georg From brett at python.org Wed Mar 8 23:17:06 2006 From: brett at python.org (Brett Cannon) Date: Wed, 8 Mar 2006 14:17:06 -0800 Subject: [Python-Dev] Bug Day? In-Reply-To: References: Message-ID: On 3/8/06, Georg Brandl wrote: > Hi, > > I know, PyCon's just been, but not many bugs were closed and > there really ought to be some issues resolved before 2.4.3 happens. > The number of open bugs is again crawling to 900. > > I myself are looking at many bugs and patches over time, but with > most of them I can't decide alone what to do. Writing here every > time doesn't seem like the most sensible thing too. > > So, is there interest? Pending time, I would be interested, especially to see if we can at least narrow down the persistent reference leaks. > Can I do anything to make it easier? Do my homework and TA work for me to an acceptable level of quality. =) Other than that, just holding at a time that works for me (they tend to fall in the morning for PSF on the weekend which means I am still asleep, although I am working on actually getting up at reasonable hours these days. =) -Brett From jeremy at alum.mit.edu Wed Mar 8 23:50:59 2006 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Wed, 8 Mar 2006 17:50:59 -0500 Subject: [Python-Dev] Bug Day? In-Reply-To: References: Message-ID: On 3/8/06, Georg Brandl wrote: > I know, PyCon's just been, but not many bugs were closed and > there really ought to be some issues resolved before 2.4.3 happens. > The number of open bugs is again crawling to 900. > > I myself are looking at many bugs and patches over time, but with > most of them I can't decide alone what to do. Writing here every > time doesn't seem like the most sensible thing too. > > So, is there interest? Can I do anything to make it easier? Would anyone be interested in a bug day on a weekday? I don't have time on weekends, but could probably find some time on a Wed or Fri for bug fixing. I'll probably do it regardless, but I'd have more fun if a few others were online. Jeremy From brett at python.org Thu Mar 9 00:05:01 2006 From: brett at python.org (Brett Cannon) Date: Wed, 8 Mar 2006 15:05:01 -0800 Subject: [Python-Dev] Bug Day? In-Reply-To: References: Message-ID: On 3/8/06, Jeremy Hylton wrote: > On 3/8/06, Georg Brandl wrote: > > I know, PyCon's just been, but not many bugs were closed and > > there really ought to be some issues resolved before 2.4.3 happens. > > The number of open bugs is again crawling to 900. > > > > I myself are looking at many bugs and patches over time, but with > > most of them I can't decide alone what to do. Writing here every > > time doesn't seem like the most sensible thing too. > > > > So, is there interest? Can I do anything to make it easier? > > Would anyone be interested in a bug day on a weekday? I don't have > time on weekends, but could probably find some time on a Wed or Fri > for bug fixing. I'll probably do it regardless, but I'd have more fun > if a few others were online. > I could find time on Fridays. Wednesday as well, but not quite as much. -Brett From greg.ewing at canterbury.ac.nz Thu Mar 9 00:30:15 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 09 Mar 2006 12:30:15 +1300 Subject: [Python-Dev] quit() on the prompt In-Reply-To: <20060308125250.GA25869@phd.pp.ru> References: <440DFED1.1000708@colorstudy.com> <9e804ac0603071537p10cb7ae0k25f97cca9048656f@mail.gmail.com> <20060308114555.GA23359@phd.pp.ru> <20060308125250.GA25869@phd.pp.ru> Message-ID: <440F6907.70703@canterbury.ac.nz> Oleg Broytmann wrote: > IDEs. Edit a code in an editor, run python -i script.py, investigate the > environment, return to the editor, get error message. An IDE is likely to want to catch SystemExits in the debugged script and handle them specially anyway. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From guido at python.org Thu Mar 9 00:42:13 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 8 Mar 2006 15:42:13 -0800 Subject: [Python-Dev] quit() on the prompt In-Reply-To: <440F6907.70703@canterbury.ac.nz> References: <440DFED1.1000708@colorstudy.com> <9e804ac0603071537p10cb7ae0k25f97cca9048656f@mail.gmail.com> <20060308114555.GA23359@phd.pp.ru> <20060308125250.GA25869@phd.pp.ru> <440F6907.70703@canterbury.ac.nz> Message-ID: We seem to have a consensus. Is anybody working on a patch yet? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From olsongt at verizon.net Thu Mar 9 00:55:01 2006 From: olsongt at verizon.net (Grant Olson) Date: Wed, 8 Mar 2006 23:55:01 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?Generated_code_in_test=5Fast=2Epy?= Message-ID: I?m finishing up a patch for bug 1441408. I had to change the asdl definitions which in turn caused a failure in test_ast.py. A comment in the file indicates that #### EVERYTHING BELOW IS GENERATED ##### and has a definition for Slice() that is no longer valid. Can anyone tell me how this is generated? Should it be automatically generated when I run asdl_c.py? I'm developing on Windows. I know in the past some of the autogenerated code from the ast-branch worked better via make. Would this have fixed itself on a linux box, or is this a manual process? -Grant P.S. Originally sent this on Monday, still waiting moderator approval. Thought I'd give gmane a try. From tim.peters at gmail.com Thu Mar 9 03:30:42 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 8 Mar 2006 21:30:42 -0500 Subject: [Python-Dev] Bug Day? In-Reply-To: References: Message-ID: <1f7befae0603081830p6dca69a5sd3c010dce833c385@mail.gmail.com> [Georg Brandl] > I know, PyCon's just been, but not many bugs were closed Ya, it was very much a development sprint this year -- "new features". > and there really ought to be some issues resolved before 2.4.3 happens. > The number of open bugs is again crawling to 900. > > I myself are looking at many bugs and patches over time, but with > most of them I can't decide alone what to do. Writing here every > time doesn't seem like the most sensible thing too. > > So, is there interest? Can I do anything to make it easier? Some bug days would be an excellent idea. Given my current schedule, I'm afraid the only days I could make time are those whose names end with "y" ;-) From nas at arctrix.com Thu Mar 9 05:31:39 2006 From: nas at arctrix.com (Neil Schemenauer) Date: Thu, 9 Mar 2006 04:31:39 +0000 (UTC) Subject: [Python-Dev] Bug Day? References: Message-ID: I think it would be a good idea to follow the Plone project and try to encourage new developers by offering assistance to get them up and running. AFAIK, we've done that for the other bug days but it might help to publish the fact that no prior Python development experience is necessary. Neil From selliott4 at austin.rr.com Thu Mar 9 05:48:11 2006 From: selliott4 at austin.rr.com (Steven Elliott) Date: Wed, 08 Mar 2006 22:48:11 -0600 Subject: [Python-Dev] Making builtins more efficient Message-ID: <1141879691.11091.78.camel@grey> I'm interested in how builtins could be more efficient. I've read over some of the PEPs having to do with making global variables more efficient (search for "global"): http://www.python.org/doc/essays/pepparade.html But I think the problem can be simplified by focusing strictly on builtins. One of my assumptions is that only a small fractions of modules override the default builtins with something like: import mybuiltins __builtins__ = mybuiltins As you probably know each access of a builtin requires two hash table lookups. First, the builtin is not found in the list of globals. It is then found in the list of builtins. Why not have a means of referencing the default builtins with some sort of index the way the LOAD_FAST op code currently works? In other words, by default each module gets the default set of builtins indexed (where the index indexes into an array) in a certain order. The version stored in the pyc file would be bumped each time the set of default builtins is changed. I don't have very strong feelings whether things like True = (1 == 1) would be a syntax error, but assigning to a builtin could just do the equivalent of STORE_FAST. I also don't have very strong feelings about whether the array of default builtins would be shared between modules. To simulate the current behavior where attempting to assign to builtin actually alters that module's global hashtable a separate array of builtins could be used for each module. As to assigning to __builtins__ (like I mentioned at the beginning of this post) perhaps it could assign to the builtin array for those items that have a name that matches a default builtin (such as "True" or "len"). Those items that don't match a default builtin would just create global variables. Perhaps what I'm suggesting isn't feasible for reasons that have already been discussed. But it seems like it should be possible to make "while True" as efficient as "while 1". -- ----------------------------------------------------------------------- | Steven Elliott | selliott4 at austin.rr.com | ----------------------------------------------------------------------- From xavier.morel at masklinn.net Mon Mar 6 23:23:35 2006 From: xavier.morel at masklinn.net (Morel Xavier) Date: Mon, 06 Mar 2006 23:23:35 +0100 Subject: [Python-Dev] conditional expressions - add parens? In-Reply-To: <440CAAFB.5060702@holdenweb.com> References: <440CAAFB.5060702@holdenweb.com> Message-ID: <440CB667.8080008@masklinn.net> Steve Holden wrote: > Contrast with the bleeding obvious: > > level = 0 > if "absolute_import" in self.futures: > level = -1 > > regards > Steve > The issue that spawned the necessity of a ternary operator in the first place was that this syntax is not usable at all in quite a few situations like, say, list comprehensions... From tzot at mediconsa.com Thu Mar 9 00:50:58 2006 From: tzot at mediconsa.com (Christos Georgiou) Date: Thu, 9 Mar 2006 01:50:58 +0200 Subject: [Python-Dev] fixing log messages References: Message-ID: [Fredrik Lundh] >> (but alright, as long as you don't call me "Fred"...) [Steve Holden] > Did I *ever* do that? That would have been an embarrassing slip ;-) I know I'm extremely late, but there should be a POTF (Pun Of The Fortnight) from now on. A member of the Mund-SIG From kbk at shore.net Thu Mar 9 09:07:15 2006 From: kbk at shore.net (Kurt B. Kaiser) Date: Thu, 9 Mar 2006 03:07:15 -0500 (EST) Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200603090807.k2987F9n020936@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 386 open ( +4) / 3089 closed (+10) / 3475 total (+14) Bugs : 889 open ( +9) / 5636 closed (+12) / 6525 total (+21) RFE : 212 open ( +1) / 201 closed ( +0) / 413 total ( +1) New / Reopened Patches ______________________ db4.4 support for bsddb (2006-03-02) CLOSED http://python.org/sf/1441660 opened by J?rome Laheurte Faster list comprehensions (2006-03-03) CLOSED http://python.org/sf/1442442 opened by Collin Winter Add support for the If-Modified-Since header (2006-03-03) http://python.org/sf/1442867 opened by Shannon -jj Behrens PyLong_FromString optimization (2006-03-04) http://python.org/sf/1442927 opened by Alan McIntyre Incremental codecs for CJKCodecs (2006-03-05) http://python.org/sf/1443155 opened by Hye-Shik Chang Remove bad PREDICT in ceval.c (2006-03-04) http://python.org/sf/1443159 opened by Collin Winter gc.get_count() and other changes (2006-03-05) CLOSED http://python.org/sf/1443865 opened by Barry A. Warsaw email/charset.py convert() patch (2006-03-06) http://python.org/sf/1443875 opened by Tokio Kikuchi Various minor fixes for NULL checks, etc. (2006-03-06) http://python.org/sf/1444030 opened by Hye-Shik Chang Make itertools.tee participate in GC (2006-03-06) http://python.org/sf/1444398 opened by Thomas Wouters kwdargs for compile/__import__ (2006-03-07) http://python.org/sf/1444529 opened by Thomas Wouters New with semantics (2006-03-08) http://python.org/sf/1445739 opened by Guido van Rossum Patches Closed ______________ db4.4 support for bsddb (2006-03-02) http://python.org/sf/1441660 closed by nnorwitz Inconsistency in os' function naming (2006-03-01) http://python.org/sf/1440928 closed by gvanrossum PEP 357 --- adding nb_index (2006-02-21) http://python.org/sf/1436368 closed by gvanrossum Faster list comprehensions (2006-03-03) http://python.org/sf/1442442 closed by nnorwitz The email package needs an "application" type (2006-02-04) http://python.org/sf/1424065 closed by bwarsaw Size of time_t changed to 64 bits in MSVC++ 2005 CRT (2006-02-24) http://python.org/sf/1437769 closed by loewis gc.get_count() and other changes (2006-03-05) http://python.org/sf/1443865 closed by bwarsaw StreamReader.readline with size reading multiple lines (2005-12-13) http://python.org/sf/1379332 closed by doerwalter Make property use the getter's __doc__ if no doc argument (2006-02-18) http://python.org/sf/1434038 closed by gbrandl New / Reopened Bugs ___________________ yet another svn head compiler change (2006-03-02) http://python.org/sf/1441486 opened by Michael Hudson socket read() can cause MemoryError in Windows (2006-03-02) http://python.org/sf/1441530 opened by taukki Omission in docs for urllib2.urlopen() (2006-03-02) http://python.org/sf/1441864 opened by Kent Johnson A (treaded) Python crash only on dual core machines (2006-03-02) CLOSED http://python.org/sf/1441884 opened by Robert Kiendl Multiple simultaneous sendtos on AF_UNIX socket broken. (2006-03-02) http://python.org/sf/1441984 opened by Brian Brunswick Traceback in pydoc (2006-03-02) CLOSED http://python.org/sf/1442012 opened by Thomas Heller IDLE shell window gets very slow when displaying long lines (2006-03-03) http://python.org/sf/1442493 opened by Heiko Selber os.listdir doesn't release GIL (2006-02-15) CLOSED http://python.org/sf/1432525 reopened by gvanrossum docs for os.statvfs miss f_bsize parameter (2006-03-03) CLOSED http://python.org/sf/1442767 opened by Chris Cogdon handling comments with markupbase and HTMLParser (2006-03-03) http://python.org/sf/1442874 opened by Daniel Bad Coroutine link (2006-03-04) CLOSED http://python.org/sf/1442937 opened by Paddy McCarthy Pickle protocol 2 fails on private slots. (2006-03-05) http://python.org/sf/1443328 opened by Daniele Varrazzo locale.getpreferredencoding() dies when setlocale fails (2006-03-05) http://python.org/sf/1443504 opened by Catherine Devlin email 3.0+ stops parsing headers prematurely (2006-03-05) http://python.org/sf/1443866 opened by Mark Sapiro os.open() Documentation (2006-03-06) http://python.org/sf/1444104 opened by Thomas Guettler subprocess test cases fail with noexec-mounted /tmp (2006-03-06) http://python.org/sf/1444408 opened by Wummel Pointer freed twice in Py_InitializeEx() (2006-03-07) http://python.org/sf/1444893 opened by athorp getpass.getpass queries on STDOUT not STERR (*nix) (2006-03-07) http://python.org/sf/1445068 opened by Jon Lasser embedding Python causes memory leaks (2006-03-08) http://python.org/sf/1445210 opened by Andrew Trevorrow install fails on hard link (2006-03-08) http://python.org/sf/1445781 opened by goldenautumnday install fails on hard link (2006-03-08) CLOSED http://python.org/sf/1445807 opened by goldenautumnday os.path.realpath works on windows (2006-03-08) CLOSED http://python.org/sf/1445901 opened by Jonathan Ellis unicode('foo', '.utf99') does not raise LookupError (2006-03-08) http://python.org/sf/1446043 opened by osvenskan subprocess interpreted double quotation mark wrong on window (2006-03-09) http://python.org/sf/1446119 opened by simon Bugs Closed ___________ A (treaded) Python crash only on dual core machines (2006-03-02) http://python.org/sf/1441884 closed by loewis Traceback in pydoc (2006-03-02) http://python.org/sf/1442012 closed by gbrandl os.listdir doesn't release GIL (2006-02-15) http://python.org/sf/1432525 closed by gbrandl docs for os.statvfs miss f_bsize parameter (2006-03-03) http://python.org/sf/1442767 closed by nnorwitz Bad Coroutine link (2006-03-03) http://python.org/sf/1442937 closed by nnorwitz test_struct crashed, py2.3.5, solaris 10 (12/08/05) http://python.org/sf/1376400 closed by sf-robot test test_nis crashed -- nis.error: no such key in map (07/02/03) http://python.org/sf/764493 closed by sf-robot file to store pickled object should be opened in binary mode (2006-02-27) http://python.org/sf/1439659 closed by gbrandl set update method description is incorrect (2006-03-01) http://python.org/sf/1441264 closed by tjreedy test_tarfile fails with readonly source dir for Python 2.4.2 (12/23/05) http://python.org/sf/1389157 closed by sf-robot cannot build SVN trunk on old systems (12/29/05) http://python.org/sf/1393109 closed by sf-robot building readline module fails on Irix 6.5 (04/28/03) http://python.org/sf/729236 closed by sf-robot Python 2.5 CVS broken for HP-UX platform? (06/01/05) http://python.org/sf/1212703 closed by sf-robot MemoryError on AIX52 (08/03/04) http://python.org/sf/1002465 closed by sf-robot 2.3.4 can not be installed over 2.3.3 (05/29/04) http://python.org/sf/962918 closed by sf-robot Rebuilding from source on RH9 fails (_tkinter.so missing) (04/22/05) http://python.org/sf/1188231 closed by sf-robot configure problem on HP-UX 11.11 (11/23/04) http://python.org/sf/1071597 closed by sf-robot dbm Error (10/09/03) http://python.org/sf/820953 closed by sf-robot bsddb hangs with recno/source sync (01/26/03) http://python.org/sf/675187 closed by sf-robot ImportError: No module named _socket (10/01/03) http://python.org/sf/815999 closed by sf-robot Python 2.3.3 urllib proxy support fails (01/07/04) http://python.org/sf/872736 closed by sf-robot UnicodeWriter: "utf-8" hardcoded instead of self.encoding (2006-03-01) http://python.org/sf/1440831 closed by gbrandl CHM file contains proprietary link format (2006-02-18) http://python.org/sf/1434298 closed by alexanderweb install fails on hard link (2006-03-08) http://python.org/sf/1445807 deleted by goldenautumnday os.path.realpath works on windows (2006-03-08) http://python.org/sf/1445901 closed by gbrandl RFE Closed __________ lib-deprecated (02/02/06) http://python.org/sf/1423082 closed by sf-robot From steve at holdenweb.com Thu Mar 9 09:13:32 2006 From: steve at holdenweb.com (Steve Holden) Date: Thu, 09 Mar 2006 08:13:32 +0000 Subject: [Python-Dev] conditional expressions - add parens? In-Reply-To: <440CB667.8080008@masklinn.net> References: <440CAAFB.5060702@holdenweb.com> <440CB667.8080008@masklinn.net> Message-ID: Morel Xavier wrote: > Steve Holden wrote: > >>Contrast with the bleeding obvious: >> >> level = 0 >> if "absolute_import" in self.futures: >> level = -1 >> >>regards >> Steve > > > > The issue that spawned the necessity of a ternary operator in the first > place was that this syntax is not usable at all in quite a few > situations like, say, list comprehensions... And of course the example I criticized was a list comprehension (not). The fact that ternary operators improve some features isn't an excuse for using them when they actually decrease readability. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd www.holdenweb.com Love me, love my blog holdenweb.blogspot.com From seojiwon at gmail.com Thu Mar 9 09:25:24 2006 From: seojiwon at gmail.com (Jiwon Seo) Date: Thu, 9 Mar 2006 00:25:24 -0800 Subject: [Python-Dev] conditional expressions - add parens? In-Reply-To: References: Message-ID: With parentheses, we can use "if cond then val1 else val2" form without the burden of hacking the parser, although the cost of the keyword "then" is still there. so, some possible forms that prompts in my mind are level = (if "absolute_import" in self.future then 0 else -1) level = (if "absolute_import" in self.future: 0 else: -1) level = (0 if "absolute_import" in self.future else -1) -Jiwon On 3/7/06, Jeremy Hylton wrote: > On 3/6/06, Alex Martelli wrote: > > > > On Mar 6, 2006, at 9:17 AM, Jim Jewett wrote: > > ... > > > I think that adding parentheses would help, by at least signalling > > > that the logic is longer than just the next (single) expression. > > > > > > level = (0 if "absolute_import" in self.futures else -1) > > > > +1 (just because I can't give it +3.1415926...!!!). *Mandatory* > > parentheses make this form MUCH more readable. > > Recent language features seem to be suffereing from excessive > parenthesisitis. I worry that people will stop remembering which > expressions requirement parens in which context. Perhaps the solution > is to require parens around all expressions, a simple consistent rule. > If so, then adding parens around all statements is a fairly natural > extension, which solves a number of problems like how to make a richer > lambda. > > Jeremy > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/seojiwon%40gmail.com > From ncoghlan at gmail.com Thu Mar 9 10:47:55 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 09 Mar 2006 19:47:55 +1000 Subject: [Python-Dev] Making builtins more efficient In-Reply-To: <1141879691.11091.78.camel@grey> References: <1141879691.11091.78.camel@grey> Message-ID: <440FF9CB.5030407@gmail.com> Steven Elliott wrote: > I'm interested in how builtins could be more efficient. I've read over > some of the PEPs having to do with making global variables more > efficient (search for "global"): > http://www.python.org/doc/essays/pepparade.html > But I think the problem can be simplified by focusing strictly on > builtins. Unfortunately, builtins can currently be shadowed in the module global namespace from outside the module (via constructs like "import mod; mod.str = my_str"). Unless/until that becomes illegal, focusing solely on builtins doesn't help - the difficulties lie in optimising builtin access while preserving the existing name shadowing semantics. PEP 280 seems to be about as simple as an approach can get while still possessing the correct semantics. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From p.f.moore at gmail.com Thu Mar 9 13:00:57 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 9 Mar 2006 12:00:57 +0000 Subject: [Python-Dev] Making builtins more efficient In-Reply-To: <440FF9CB.5030407@gmail.com> References: <1141879691.11091.78.camel@grey> <440FF9CB.5030407@gmail.com> Message-ID: <79990c6b0603090400h25dd2c7ev3d5c379f6529f3c2@mail.gmail.com> On 3/9/06, Nick Coghlan wrote: > Steven Elliott wrote: > > I'm interested in how builtins could be more efficient. I've read over > > some of the PEPs having to do with making global variables more > > efficient (search for "global"): > > http://www.python.org/doc/essays/pepparade.html > > But I think the problem can be simplified by focusing strictly on > > builtins. > > Unfortunately, builtins can currently be shadowed in the module global > namespace from outside the module (via constructs like "import mod; mod.str = > my_str"). Unless/until that becomes illegal, focusing solely on builtins > doesn't help - the difficulties lie in optimising builtin access while > preserving the existing name shadowing semantics. Is there any practical way of detecting and flagging constructs like the above (remotely shadowing a builtin in another module)? I can't see a way of doing it (but I know very little about this area...). If it *is* possible, I'd say it's worth implementing at least a warning sooner rather than later - the practice seems questionable at best, and any progress towards outlawing it would help in work on optimising builtins. Paul. From g.brandl at gmx.net Thu Mar 9 13:38:12 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 09 Mar 2006 13:38:12 +0100 Subject: [Python-Dev] quit() on the prompt In-Reply-To: References: <440DFED1.1000708@colorstudy.com> <9e804ac0603071537p10cb7ae0k25f97cca9048656f@mail.gmail.com> <20060308114555.GA23359@phd.pp.ru> <20060308125250.GA25869@phd.pp.ru> <440F6907.70703@canterbury.ac.nz> Message-ID: Guido van Rossum wrote: > We seem to have a consensus. Is anybody working on a patch yet? http://python.org/sf/1446372 Georg From selliott4 at austin.rr.com Thu Mar 9 15:50:06 2006 From: selliott4 at austin.rr.com (Steven Elliott) Date: Thu, 09 Mar 2006 08:50:06 -0600 Subject: [Python-Dev] Making builtins more efficient In-Reply-To: <79990c6b0603090400h25dd2c7ev3d5c379f6529f3c2@mail.gmail.com> References: <1141879691.11091.78.camel@grey> <440FF9CB.5030407@gmail.com> <79990c6b0603090400h25dd2c7ev3d5c379f6529f3c2@mail.gmail.com> Message-ID: <1141915806.11091.127.camel@grey> On Thu, 2006-03-09 at 12:00 +0000, Paul Moore wrote: > On 3/9/06, Nick Coghlan wrote: > > Steven Elliott wrote: > > > I'm interested in how builtins could be more efficient. I've read over > > > some of the PEPs having to do with making global variables more > > > efficient (search for "global"): > > > http://www.python.org/doc/essays/pepparade.html > > > But I think the problem can be simplified by focusing strictly on > > > builtins. > > > > Unfortunately, builtins can currently be shadowed in the module global > > namespace from outside the module (via constructs like "import mod; mod.str = > > my_str"). Unless/until that becomes illegal, focusing solely on builtins > > doesn't help - the difficulties lie in optimising builtin access while > > preserving the existing name shadowing semantics. > > Is there any practical way of detecting and flagging constructs like > the above (remotely shadowing a builtin in another module)? I can't > see a way of doing it (but I know very little about this area...). It may be possible to flag it, or it may be possible it make it work. In my post I mentioned one special case that needs to be addressed (assigning to __builtins__). What Nick mentioned in his post ("import mod; mod.str = my_str") is another special case that needs to be addressed. If we can assume that all pyc files are compiled with the same set of default bulitins (which should be assured by the by the version in the pyc file) then there are two ways that things like "mod.str = my_str" could be handled. I believe that currently "mod.str = my_str" alters the module's global hash table (f->f_globals in the code). One way of handling it is to alter STORE_ATTR (op code for assigning to mod.str) to always check to see if the key being assigned is one of the default builtins. If it is, then the module's indexed array of builtins is assigned to. Alternatively if we also wanted to optimize "mod.str = my_str" then there could be a new opcode like STORE_ATTR that would take an index into the array of builtins instead of an index into the names. PEP 280, which Nick mentioned, talks about a "cells", a hybrid data structure that can do both hash table lookups and lookups by index efficiently. That's great, but I'm curious if additional gains can be made be focusing just on builtins. -- ----------------------------------------------------------------------- | Steven Elliott | selliott4 at austin.rr.com | ----------------------------------------------------------------------- From pje at telecommunity.com Thu Mar 9 16:27:18 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 09 Mar 2006 10:27:18 -0500 Subject: [Python-Dev] Making builtins more efficient In-Reply-To: <1141915806.11091.127.camel@grey> References: <79990c6b0603090400h25dd2c7ev3d5c379f6529f3c2@mail.gmail.com> <1141879691.11091.78.camel@grey> <440FF9CB.5030407@gmail.com> <79990c6b0603090400h25dd2c7ev3d5c379f6529f3c2@mail.gmail.com> Message-ID: <5.1.1.6.0.20060309101604.04c1fa78@mail.telecommunity.com> At 08:50 AM 3/9/2006 -0600, Steven Elliott wrote: >I believe that currently "mod.str = my_str" alters the module's global >hash table (f->f_globals in the code). One way of handling it is to >alter STORE_ATTR (op code for assigning to mod.str) to always check to >see if the key being assigned is one of the default builtins. If it is, >then the module's indexed array of builtins is assigned to. It's not the opcode that would change, it's the C function referenced by the module type's tp_setattro function slot. This has already been attempted before, in order to implement a warning for this behavior. You might want to research that, because the patch ended up being backed out for some reason. I think it ended up being too strict of a check to be accepted for Python 2.4. If some version of it could come back, however, it's possible we could use this in Python 2.5 to allow -O compilation to assume that unshadowed builtins are static, making it potentially possible to convert some of them to specialized bytecodes, or perhaps a "BUILTIN_OP n" bytecode, where 'n' is a combination of an operation selector and the number of arguments to be taken from the stack. The determination of "unshadowed" would have to be conservative, in that 'from foo import *' might shadow a builtin, so using 'import *' would disable optimization of all builtins. However, if that were not present, and there's no statically detectable assignment shadowing a particular builtin, that builtin could be optimized. (Assuming -O, of course.) From theller at python.net Thu Mar 9 16:34:55 2006 From: theller at python.net (Thomas Heller) Date: Thu, 09 Mar 2006 16:34:55 +0100 Subject: [Python-Dev] ctypes is in SVN now. Message-ID: ctypes is in SVN now, and the buildbot is green, after I disabled a test that dumped core on sparc solaris. The crash was apparently caused Missing are .vcproj files for Windows, both for the _ctypes.pyd extension and the _ctypes_test.pyd extension needed for testing. IIRC, Martin promised to create them - is this offer still valid? I could do that myself, but only for x86, while other .pyd files also have build settings for Itanium and x86_64. (I find it always very painful to click through the settings dialog in MSVC - isn't there any other way to create these files?) Thomas From rasky at develer.com Thu Mar 9 17:03:44 2006 From: rasky at develer.com (Giovanni Bajo) Date: Thu, 9 Mar 2006 17:03:44 +0100 Subject: [Python-Dev] ctypes is in SVN now. References: Message-ID: <06aa01c64393$0ab8bb10$bf03030a@trilan> Thomas Heller wrote: > Missing are .vcproj files for Windows, both for the _ctypes.pyd extension > and the _ctypes_test.pyd extension needed for testing. IIRC, Martin > promised to create them - is this offer still valid? I could do that > myself, but only for x86, while other .pyd files also have build settings > for Itanium and x86_64. (I find it always very painful to click through > the settings dialog in MSVC - isn't there any other way to create these > files?) I discussed with Martin a bit about the opportunity of generating .vcproj files with a script-driven tool. I'm going to try and setup something as soon as I find some spare time. The goal of this work would exactly fit your need: make the creation of extension modules trivially easy (just as easy as adding the modules to the main python.dll). My personal goal, in fact, is to move many of those builtin extension modules from python.dll out into their own .pyd files where they'd belong (were not for this technical annoyance of being forced to use the settings dialog in MSVC). -- Giovanni Bajo From theller at python.net Thu Mar 9 17:17:01 2006 From: theller at python.net (Thomas Heller) Date: Thu, 09 Mar 2006 17:17:01 +0100 Subject: [Python-Dev] ctypes is in SVN now. In-Reply-To: <06aa01c64393$0ab8bb10$bf03030a@trilan> References: <06aa01c64393$0ab8bb10$bf03030a@trilan> Message-ID: <441054FD.9020704@python.net> Giovanni Bajo wrote: > Thomas Heller wrote: > >> Missing are .vcproj files for Windows, both for the _ctypes.pyd extension >> and the _ctypes_test.pyd extension needed for testing. IIRC, Martin >> promised to create them - is this offer still valid? I could do that >> myself, but only for x86, while other .pyd files also have build settings >> for Itanium and x86_64. (I find it always very painful to click through >> the settings dialog in MSVC - isn't there any other way to create these >> files?) > > > I discussed with Martin a bit about the opportunity of generating .vcproj > files with a script-driven tool. I'm going to try and setup something as > soon as I find some spare time. > > The goal of this work would exactly fit your need: make the creation of > extension modules trivially easy (just as easy as adding the modules to the > main python.dll). My personal goal, in fact, is to move many of those > builtin extension modules from python.dll out into their own .pyd files > where they'd belong (were not for this technical annoyance of being forced > to use the settings dialog in MSVC). Ideally this would be integrated with distutils because the setup-script has most of the information that's needed. OTOH, extensions could be built with distutils even for core Python, and even on Windows. For extensions that are *not* builtin or not included with Python itself distutils works good enough. Oh, IIRC the pywin32 setup script has code that is able to parse MSVC6 project files and create a distutils Extension instance from them - maybe this can help you. Thomas From rasky at develer.com Thu Mar 9 17:27:56 2006 From: rasky at develer.com (Giovanni Bajo) Date: Thu, 9 Mar 2006 17:27:56 +0100 Subject: [Python-Dev] ctypes is in SVN now. References: <06aa01c64393$0ab8bb10$bf03030a@trilan> <441054FD.9020704@python.net> Message-ID: <06e501c64396$6c200130$bf03030a@trilan> Thomas Heller wrote: >> I discussed with Martin a bit about the opportunity of generating .vcproj >> files with a script-driven tool. I'm going to try and setup something as >> soon as I find some spare time. > > Ideally this would be integrated with distutils because the setup-script > has most of the information that's needed. > > OTOH, extensions could be built with distutils even for core Python, and > even on Windows. For extensions that are *not* builtin or not included with > Python itself distutils works good enough. I fear this is an orthogonal change. Alas, using distutils to build core extensions is not something I'm ready to tackle at the moment. I was just thiking of something much more naive like using a free tool to build .vcproj/.sln (see www.cmake.org for instance) from a script description. With such a tool, it would be very easy to build a .pyd file around a self-contained .c file (like _heapqmodule.c, etc.), it would mostly be a couple of line changes in the script file, and then re-run the tool executable to regenerate vcproj/sln. OTOH, both the tool executable (cmake.exe in this example) and the current version of the generated vcproj/sln files would be committed in SVN under PCbuild, so to have a minimal impact on developer habits. -- Giovanni Bajo From raymond.hettinger at verizon.net Thu Mar 9 17:51:16 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Thu, 09 Mar 2006 08:51:16 -0800 Subject: [Python-Dev] Making builtins more efficient References: <1141879691.11091.78.camel@grey> Message-ID: <007c01c64399$af03ffd0$a600000a@RaymondLaptop1> [Steven Elliott] > As you probably know each access of a builtin requires two hash table > lookups. First, the builtin is not found in the list of globals. It is > then found in the list of builtins. If someone really cared about the double lookup, they could flatten a level by starting their modules with: from __builtin__ import * However, we don't see people writing this kind of code. That could mean that the double lookup hasn't been a big concern. > Why not have a means of referencing the default builtins with some sort > of index the way the LOAD_FAST op code currently works? FWIW, there was a PEP proposing a roughly similar idea, but the PEP encountered a great deal of resistance: http://www.python.org/doc/peps/pep-0329/ When it comes time to write your PEP, it would helpful to highlight how it differs from PEP 329 (i.e. implemented through the compiler rather than as a bytecode hack, etc.). > Perhaps what I'm suggesting isn't feasible for reasons that have already > been discussed. But it seems like it should be possible to make "while > True" as efficient as "while 1". That is going to be difficult as long as it is legal to write: True = 0 Raymond From theller at python.net Thu Mar 9 17:54:03 2006 From: theller at python.net (Thomas Heller) Date: Thu, 09 Mar 2006 17:54:03 +0100 Subject: [Python-Dev] ctypes is in SVN now. In-Reply-To: <06e501c64396$6c200130$bf03030a@trilan> References: <06aa01c64393$0ab8bb10$bf03030a@trilan> <441054FD.9020704@python.net> <06e501c64396$6c200130$bf03030a@trilan> Message-ID: <44105DAB.9040900@python.net> Giovanni Bajo wrote: > Thomas Heller wrote: > >>> I discussed with Martin a bit about the opportunity of generating >>> .vcproj files with a script-driven tool. I'm going to try and >>> setup something as soon as I find some spare time. >> Ideally this would be integrated with distutils because the >> setup-script has most of the information that's needed. >> >> OTOH, extensions could be built with distutils even for core >> Python, and even on Windows. For extensions that are *not* builtin >> or not included with Python itself distutils works good enough. > > I fear this is an orthogonal change. Alas, using distutils to build > core extensions is not something I'm ready to tackle at the moment. > > I was just thiking of something much more naive like using a free > tool to build .vcproj/.sln (see www.cmake.org for instance) from a > script description. With such a tool, it would be very easy to build > a .pyd file around a self-contained .c file (like _heapqmodule.c, > etc.), it would mostly be a couple of line changes in the script > file, and then re-run the tool executable to regenerate vcproj/sln. > OTOH, both the tool executable (cmake.exe in this example) and the > current version of the generated vcproj/sln files would be committed > in SVN under PCbuild, so to have a minimal impact on developer > habits. While I don't understand *why* you would want to do this (building Python interpreters with a customized list of builtin modules could be a reason for this, I've had thought about something like that for py2exe a long time ago), I've clicked myself through the MSVC dialog and I'm happy now because the two .vcproj files are ready. I'll commit them and let others (Martin, probably) extend them for 64-bit Windows versions ;-). Thomas From rasky at develer.com Thu Mar 9 18:25:17 2006 From: rasky at develer.com (Giovanni Bajo) Date: Thu, 9 Mar 2006 18:25:17 +0100 Subject: [Python-Dev] ctypes is in SVN now. References: <06aa01c64393$0ab8bb10$bf03030a@trilan> <441054FD.9020704@python.net> <06e501c64396$6c200130$bf03030a@trilan> <44105DAB.9040900@python.net> Message-ID: <06f401c6439e$6f4f7590$bf03030a@trilan> Thomas Heller wrote: >>> Ideally this would be integrated with distutils because the >>> setup-script has most of the information that's needed. >>> >>> OTOH, extensions could be built with distutils even for core >>> Python, and even on Windows. For extensions that are *not* builtin >>> or not included with Python itself distutils works good enough. >> >> I fear this is an orthogonal change. Alas, using distutils to build >> core extensions is not something I'm ready to tackle at the moment. >> >> I was just thiking of something much more naive like using a free >> tool to build .vcproj/.sln (see www.cmake.org for instance) from a >> script description. With such a tool, it would be very easy to build >> a .pyd file around a self-contained .c file (like _heapqmodule.c, >> etc.), it would mostly be a couple of line changes in the script >> file, and then re-run the tool executable to regenerate vcproj/sln. >> OTOH, both the tool executable (cmake.exe in this example) and the >> current version of the generated vcproj/sln files would be committed >> in SVN under PCbuild, so to have a minimal impact on developer >> habits. > > While I don't understand *why* you would want to do this (building > Python interpreters with a customized list of builtin modules could be > a reason for this, I've had thought about something like that for py2exe > a long time ago), That's exactly the reason: packaged executables. I'm sure there is still some weird encoding in world, with 2Mb of cute codec data tables, which is only waiting for people to find out and merge it into python.dll -- because it's harder to maintain it as an external .pyd with the current PCbuild. That's unreasonable IMVHO. CJK codecs *ought* to be distributed as an external .pyd. -- Giovanni Bajo From jcarlson at uci.edu Thu Mar 9 19:16:54 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu, 09 Mar 2006 10:16:54 -0800 Subject: [Python-Dev] Making builtins more efficient In-Reply-To: <1141879691.11091.78.camel@grey> References: <1141879691.11091.78.camel@grey> Message-ID: <20060309100812.F6E5.JCARLSON@uci.edu> Steven Elliott wrote: > I'm interested in how builtins could be more efficient. I've read over > some of the PEPs having to do with making global variables more > efficient (search for "global"): > http://www.python.org/doc/essays/pepparade.html > But I think the problem can be simplified by focusing strictly on > builtins. Whether or not it is a good idea, it is not wholly uncommon for users to manipulate variables in the builtin or other module namespaces. From very simple import hooks replacing __import__, to module.QUIT = 1, these things happen, and I don't think that making them produce a warning as suggested by Paul, necessarily helps us. I personally think the problem can be simplified (and solved) by applying decorators where such builtin lookups could/should be more efficient. Raymond has a recipe for doing such optimization over at the Python cookbook: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/277940 Toss that little guy into a decorators module; if people want their runtime calling to be fast (at the expense of a slightly increased compilation time), they'll start using it. - Josiah From nd at perlig.de Thu Mar 9 19:17:48 2006 From: nd at perlig.de (=?iso-8859-1?q?Andr=E9_Malo?=) Date: Thu, 9 Mar 2006 19:17:48 +0100 Subject: [Python-Dev] Making builtins more efficient In-Reply-To: <79990c6b0603090400h25dd2c7ev3d5c379f6529f3c2@mail.gmail.com> References: <1141879691.11091.78.camel@grey> <440FF9CB.5030407@gmail.com> <79990c6b0603090400h25dd2c7ev3d5c379f6529f3c2@mail.gmail.com> Message-ID: <200603091917.48597@news.perlig.de> * Paul Moore wrote: > If it *is* possible, I'd say it's worth implementing at least a > warning sooner rather than later - the practice seems questionable at > best, and any progress towards outlawing it would help in work on > optimising builtins. FWIW, this practice is very handy for unit tests. For example, I often shadow builtins like ``file`` for my tests with a mock placed from the outside into the global namespace. nd -- "Das Verhalten von Gates hatte mir bewiesen, dass ich auf ihn und seine beiden Gef?hrten nicht zu z?hlen brauchte" -- Karl May, "Winnetou III" Im Westen was neues: From thomas at python.org Thu Mar 9 19:28:23 2006 From: thomas at python.org (Thomas Wouters) Date: Thu, 9 Mar 2006 19:28:23 +0100 Subject: [Python-Dev] ctypes is in SVN now. In-Reply-To: References: Message-ID: <9e804ac0603091028q56e8171dx31a130c247635532@mail.gmail.com> On 3/9/06, Thomas Heller wrote: > > ctypes is in SVN now, and the buildbot is green, Well, only by accident; Neal's amd64 machine has been offline, or you would have seen yellow blocks for warnings: ctypes has yet to be Py_ssize_t'ed. Do you want to do that yourself, or do you want me to submit a patch? (Or I could just check it in ;) > after I disabled a test that dumped coreon sparc solaris. The crash was > apparently caused > Ohh, mystery. Do we get to guess what it was apparently caused by? :) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060309/02ae2c9d/attachment.html From pje at telecommunity.com Thu Mar 9 19:35:00 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 09 Mar 2006 13:35:00 -0500 Subject: [Python-Dev] Making builtins more efficient In-Reply-To: <007c01c64399$af03ffd0$a600000a@RaymondLaptop1> References: <1141879691.11091.78.camel@grey> Message-ID: <5.1.1.6.0.20060309133030.01fb5ca8@mail.telecommunity.com> At 08:51 AM 3/9/2006 -0800, Raymond Hettinger wrote: > > Perhaps what I'm suggesting isn't feasible for reasons that have already > > been discussed. But it seems like it should be possible to make "while > > True" as efficient as "while 1". > >That is going to be difficult as long as it is legal to write: > > True = 0 In that case, it can obviously be determined statically that the builtin is shadowed, and optimization is impossible for that builtin in that module. What *is* difficult is the somemodule.True = 0 case, and the somemodule.__dict__['True']=0 case, and the globals()['True'] = 0 case, and other such fiddling. Personally, I think that such shenanigans (in the case where somemodule doesn't already have a module-level binding for the builtin) should not be allowed to change the semantics of optimized functions using that dictionary as a global namespace. I think this is a reasonable implementation limit on dynamicity, although to be conservative I think we should only do this with -O in Python 2.5, if we do it at all. From theller at python.net Thu Mar 9 19:44:49 2006 From: theller at python.net (Thomas Heller) Date: Thu, 09 Mar 2006 19:44:49 +0100 Subject: [Python-Dev] ctypes is in SVN now. In-Reply-To: <9e804ac0603091028q56e8171dx31a130c247635532@mail.gmail.com> References: <9e804ac0603091028q56e8171dx31a130c247635532@mail.gmail.com> Message-ID: <441077A1.4050503@python.net> Thomas Wouters wrote: > On 3/9/06, Thomas Heller wrote: >> ctypes is in SVN now, and the buildbot is green, > > > Well, only by accident; Neal's amd64 machine has been offline, or you would > have seen yellow blocks for warnings: ctypes has yet to be Py_ssize_t'ed. Do Do compiler warnings create yellow blocks? There are other warnings on other machines as well. OTOH, testing ctypes (see below) can cause core dumps, these should create orange blinking blocks ;-) > you want to do that yourself, or do you want me to submit a patch? (Or I > could just check it in ;) You can do it faster then me, I assume - so go ahead and check it in. I'll backport it to the upstream ctypes CVS repository. >> after I disabled a test that dumped coreon sparc solaris. The crash was >> apparently caused >> > > Ohh, mystery. Do we get to guess what it was apparently caused by? :) It was caused by a test that did access misaligned integer fields in a structure. ctypes isn't yet able to handle this, but probably should steal the code for it from the struct module. But the change is not trivial I fear because of the access macros that also handle bit fields in structures. And I never had tried it before on a sparc machine - all the intel and ppc processors seem to have no problems with it. Thomas From just at letterror.com Thu Mar 9 19:49:23 2006 From: just at letterror.com (Just van Rossum) Date: Thu, 9 Mar 2006 19:49:23 +0100 Subject: [Python-Dev] Making builtins more efficient In-Reply-To: <5.1.1.6.0.20060309133030.01fb5ca8@mail.telecommunity.com> Message-ID: Phillip J. Eby wrote: > I think this is a reasonable implementation limit on dynamicity, Absolutely. > although to be conservative I think we should only do this with -O in > Python 2.5, if we do it at all. Or with a __future__ directive? Just From tim.peters at gmail.com Thu Mar 9 21:30:14 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 9 Mar 2006 15:30:14 -0500 Subject: [Python-Dev] [Python-checkins] r42929 - python/trunk/Tools/scripts/svneol.py In-Reply-To: <44106221.3090001@python.net> References: <20060309014225.418ED1E4002@bag.python.org> <44106221.3090001@python.net> Message-ID: <1f7befae0603091230r47ef9755p2c43cb7b5d03ff81@mail.gmail.com> [Tim Peters] >> Added: >> python/trunk/Tools/scripts/svneol.py (contents, props changed) >> Log: >> Simple utility to add svn:eol-style to text files under >> SVN control. Like reindent.py, I expect to run this >> mindlessly from time to time, checking in whatever it >> happens to do ;-) [Thomas Heller] > Should 'sln' and 'vcproj' be added to the extensions list? I think these are > text-files too. Although PCBuild\pcbuild.sln has a binary mime-type property, > so the script would not change that. I don't know whether they're text files in the SVN eol-style "native" sense. The corresponding file types under VC 6 were not: in SVN terms, they should have svn:eol-style set to CRLF (Visual Studio required \r\n line ends for some inscrutable reason -- it would not tolerate \n line ends, and this mattered when, e.g., some Linux-head tried to run Visual Studio from a Linux-native checkout of Python; CRLF was still required, and we "faked that" under CVS by calling those files binary; SVN offers finer control; I don't know how picky the .NET 2003 MSDev is about this). From theller at python.net Thu Mar 9 21:43:07 2006 From: theller at python.net (Thomas Heller) Date: Thu, 09 Mar 2006 21:43:07 +0100 Subject: [Python-Dev] [Python-checkins] r42929 - python/trunk/Tools/scripts/svneol.py In-Reply-To: <1f7befae0603091230r47ef9755p2c43cb7b5d03ff81@mail.gmail.com> References: <20060309014225.418ED1E4002@bag.python.org> <44106221.3090001@python.net> <1f7befae0603091230r47ef9755p2c43cb7b5d03ff81@mail.gmail.com> Message-ID: <4410935B.40007@python.net> Tim Peters wrote: > [Tim Peters] >>> Added: >>> python/trunk/Tools/scripts/svneol.py (contents, props changed) >>> Log: >>> Simple utility to add svn:eol-style to text files under >>> SVN control. Like reindent.py, I expect to run this >>> mindlessly from time to time, checking in whatever it >>> happens to do ;-) > > [Thomas Heller] >> Should 'sln' and 'vcproj' be added to the extensions list? I think these are >> text-files too. Although PCBuild\pcbuild.sln has a binary mime-type property, >> so the script would not change that. > > I don't know whether they're text files in the SVN eol-style "native" > sense. The corresponding file types under VC 6 were not: in SVN > terms, they should have svn:eol-style set to CRLF (Visual Studio > required \r\n line ends for some inscrutable reason -- it would not I know. In CVS, they had to be added using the -kb flag, IIRC. > tolerate \n line ends, and this mattered when, e.g., some Linux-head > tried to run Visual Studio from a Linux-native checkout of Python; > CRLF was still required, and we "faked that" under CVS by calling > those files binary; SVN offers finer control; I don't know how picky > the .NET 2003 MSDev is about this). I remember that someone has tested that the .vcproj files actually did work even if they had the wrong line endings, so it should be ok. AFAIK, noone tried that with the .sln file - anyway, I have removed the svn:mem-type property and the script then set svn:eol-style native. Maybe a different set of svn properties were better, I have to learn about them. The problem that I saw is that 'svn diff' refused to provide a diff when pcbuild.sln had changed - I would consider this diff useful, both in commit-emails as well as run on the command line before checking in. The Linux-head is the release manager, prepaing the release on a unixish platform, in combination with the windows guy who downloads the source distribution and tries to compile that on Windows. Thomas From tim.peters at gmail.com Thu Mar 9 21:43:38 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 9 Mar 2006 15:43:38 -0500 Subject: [Python-Dev] ctypes is in SVN now. In-Reply-To: <441077A1.4050503@python.net> References: <9e804ac0603091028q56e8171dx31a130c247635532@mail.gmail.com> <441077A1.4050503@python.net> Message-ID: <1f7befae0603091243r4f7a55f3n35a3c753e2e31308@mail.gmail.com> [Thomas Heller] > ... > And I never had tried it before on a sparc machine - all the intel and ppc processors > seem to have no problems with it. Pentiums don't enforce "natural" alignment restrictions, but run much slower on unaligned access (varying by specific chip model, and generally more heavily penalized as time goes on). In the good old days, Pentium was one of dozens of competing architectures, and was the oddball in catering to unaligned access. Now it's eternal "backward compatibility" with an early implementation accident. Most other architectures never catered to unaligned access, or did so only at the cost of generating an interrupt so that kernel-mode software could fake unaligned access. Bottom line is that unaligned access isn't portable and never was, and even on architectures where "it works" it can be extremely expensive to use it. From tim.peters at gmail.com Thu Mar 9 21:46:10 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 9 Mar 2006 15:46:10 -0500 Subject: [Python-Dev] ctypes is in SVN now. In-Reply-To: References: Message-ID: <1f7befae0603091246g1cfc1b55j57b962f2be27022f@mail.gmail.com> [Thomas Heller] > ... > I could do that myself, but only for x86, while other .pyd files also have build > settings for Itanium and x86_64. (I find it always very painful to click through > the settings dialog in MSVC - isn't there any other way to create these files?) Under VC 6 I generally just renamed a copy of "a similar" existing project file and used a text editor to change the paths. I haven't tried that under the 7.1 MSDev. From theller at python.net Thu Mar 9 21:50:31 2006 From: theller at python.net (Thomas Heller) Date: Thu, 09 Mar 2006 21:50:31 +0100 Subject: [Python-Dev] ctypes is in SVN now. In-Reply-To: <1f7befae0603091243r4f7a55f3n35a3c753e2e31308@mail.gmail.com> References: <9e804ac0603091028q56e8171dx31a130c247635532@mail.gmail.com> <441077A1.4050503@python.net> <1f7befae0603091243r4f7a55f3n35a3c753e2e31308@mail.gmail.com> Message-ID: <44109517.1030707@python.net> Tim Peters wrote: > [Thomas Heller] >> ... >> And I never had tried it before on a sparc machine - all the intel and ppc processors >> seem to have no problems with it. > > Pentiums don't enforce "natural" alignment restrictions, but run much > slower on unaligned access (varying by specific chip model, and > generally more heavily penalized as time goes on). In the good old > days, Pentium was one of dozens of competing architectures, and was > the oddball in catering to unaligned access. Now it's eternal > "backward compatibility" with an early implementation accident. Most > other architectures never catered to unaligned access, or did so only > at the cost of generating an interrupt so that kernel-mode software > could fake unaligned access. Bottom line is that unaligned access > isn't portable and never was, and even on architectures where "it > works" it can be extremely expensive to use it. I'm old enough to know this, but thanks anyway. I'm not so speed paranoid to care about these nanoseconds, maybe timeit can tell if there's a measurable difference. From theller at python.net Thu Mar 9 21:51:17 2006 From: theller at python.net (Thomas Heller) Date: Thu, 09 Mar 2006 21:51:17 +0100 Subject: [Python-Dev] ctypes is in SVN now. In-Reply-To: <1f7befae0603091246g1cfc1b55j57b962f2be27022f@mail.gmail.com> References: <1f7befae0603091246g1cfc1b55j57b962f2be27022f@mail.gmail.com> Message-ID: <44109545.6060405@python.net> Tim Peters wrote: > [Thomas Heller] >> ... >> I could do that myself, but only for x86, while other .pyd files also have build >> settings for Itanium and x86_64. (I find it always very painful to click through >> the settings dialog in MSVC - isn't there any other way to create these files?) > > Under VC 6 I generally just renamed a copy of "a similar" existing > project file and used a text editor to change the paths. I haven't > tried that under the 7.1 MSDev. That's what I did when I created _ctypes_test.vcproj from _ctypes.vcproj. I had to insert a new guid though. From tim.peters at gmail.com Thu Mar 9 22:00:09 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 9 Mar 2006 16:00:09 -0500 Subject: [Python-Dev] [Python-checkins] r42929 - python/trunk/Tools/scripts/svneol.py In-Reply-To: <4410935B.40007@python.net> References: <20060309014225.418ED1E4002@bag.python.org> <44106221.3090001@python.net> <1f7befae0603091230r47ef9755p2c43cb7b5d03ff81@mail.gmail.com> <4410935B.40007@python.net> Message-ID: <1f7befae0603091300n40355d23k24c1617fb484670@mail.gmail.com> [Thomas Heller] > ... > The Linux-head is the release manager, prepaing the release on > a unixish platform, in combination with the windows guy who downloads > the source distribution and tries to compile that on Windows. That sounds like an artificial (process-created) problem, then. Since a release should be made from a tag, there doesn't seem a real reason to make the Windows release wait for a source-release tarball to get made (the Windows dweeb should be able to do a native checkout of the tag, and do their stuff in parallel -- that's how I used to do Windows releases, and it worked fine -- the only thing the Windows release serialized on was waiting for the release's final docs to get cut). Setting project files to some flavor of text mode makes sense for your other reasons regardless (like getting useful diffs). From theller at python.net Thu Mar 9 22:05:35 2006 From: theller at python.net (Thomas Heller) Date: Thu, 09 Mar 2006 22:05:35 +0100 Subject: [Python-Dev] [Python-checkins] r42929 - python/trunk/Tools/scripts/svneol.py In-Reply-To: <1f7befae0603091300n40355d23k24c1617fb484670@mail.gmail.com> References: <20060309014225.418ED1E4002@bag.python.org> <44106221.3090001@python.net> <1f7befae0603091230r47ef9755p2c43cb7b5d03ff81@mail.gmail.com> <4410935B.40007@python.net> <1f7befae0603091300n40355d23k24c1617fb484670@mail.gmail.com> Message-ID: <4410989F.9030805@python.net> Tim Peters wrote: > [Thomas Heller] >> ... >> The Linux-head is the release manager, prepaing the release on >> a unixish platform, in combination with the windows guy who downloads >> the source distribution and tries to compile that on Windows. > > That sounds like an artificial (process-created) problem, then. Since > a release should be made from a tag, there doesn't seem a real reason > to make the Windows release wait for a source-release tarball to get > made (the Windows dweeb should be able to do a native checkout of the > tag, and do their stuff in parallel -- that's how I used to do Windows > releases, and it worked fine -- the only thing the Windows release > serialized on was waiting for the release's final docs to get cut). I did it the same way for the 2.3 releases that I made. But I did not mean the official binary Windows installer, I mean someone who downloads the official source tarball, and compiles that himself. There was at least one 2.3 release where one could not compile with MSVC6 from the source tarball because of wrong line endings. > Setting project files to some flavor of text mode makes sense for your > other reasons regardless (like getting useful diffs). Sure. From martin at v.loewis.de Thu Mar 9 23:54:53 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 09 Mar 2006 23:54:53 +0100 Subject: [Python-Dev] ctypes is in SVN now. In-Reply-To: References: Message-ID: <4410B23D.5020606@v.loewis.de> Thomas Heller wrote: > ctypes is in SVN now, and the buildbot is green, after I disabled a test that dumped core > on sparc solaris. The crash was apparently caused > > Missing are .vcproj files for Windows, both for the _ctypes.pyd extension and the _ctypes_test.pyd > extension needed for testing. IIRC, Martin promised to create them - is this offer still valid? It is. I usually create the vcproj files by copying an existing one, and replacing everything that matters in a text editor. Regards, Martin From martin at v.loewis.de Thu Mar 9 23:57:42 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 09 Mar 2006 23:57:42 +0100 Subject: [Python-Dev] [Python-checkins] r42929 - python/trunk/Tools/scripts/svneol.py In-Reply-To: <1f7befae0603091230r47ef9755p2c43cb7b5d03ff81@mail.gmail.com> References: <20060309014225.418ED1E4002@bag.python.org> <44106221.3090001@python.net> <1f7befae0603091230r47ef9755p2c43cb7b5d03ff81@mail.gmail.com> Message-ID: <4410B2E6.7020902@v.loewis.de> Tim Peters wrote: >>Should 'sln' and 'vcproj' be added to the extensions list? I think these are >>text-files too. Although PCBuild\pcbuild.sln has a binary mime-type property, >>so the script would not change that. > > > I don't know whether they're text files in the SVN eol-style "native" > sense. Last I tried, VS 2003 would tolerate \n line endings in vcproj files (these being plain XML files). I don't remember whether it would tolerate them in .sln files (as these are *not* XML files); this is why the .sln file is set to binary. Regards, Martin From martin at v.loewis.de Thu Mar 9 23:59:57 2006 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Thu, 09 Mar 2006 23:59:57 +0100 Subject: [Python-Dev] Generated code in test_ast.py In-Reply-To: References: Message-ID: <4410B36D.6030806@v.loewis.de> Grant Olson wrote: > I?m finishing up a patch for bug 1441408. I had to change the asdl definitions > which in turn caused a failure in test_ast.py. A comment in the file indicates > that #### EVERYTHING BELOW IS GENERATED ##### and has a definition for Slice() > that is no longer valid. Can anyone tell me how this is generated? Should it be > automatically generated when I run asdl_c.py? You run it with -g, then manually put the output into the test case. > I'm developing on Windows. I know in the past some of the autogenerated code > from the ast-branch worked better via make. Would this have fixed itself on a > linux box, or is this a manual process? It's a manual process. Regards, Martin From tim.peters at gmail.com Fri Mar 10 00:48:03 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 9 Mar 2006 18:48:03 -0500 Subject: [Python-Dev] [Python-checkins] r42929 - python/trunk/Tools/scripts/svneol.py In-Reply-To: <4410B2E6.7020902@v.loewis.de> References: <20060309014225.418ED1E4002@bag.python.org> <44106221.3090001@python.net> <1f7befae0603091230r47ef9755p2c43cb7b5d03ff81@mail.gmail.com> <4410B2E6.7020902@v.loewis.de> Message-ID: <1f7befae0603091548x31169c00jbe6b1d8e2511cfc8@mail.gmail.com> [Martin v. L?wis] > Last I tried, VS 2003 would tolerate \n line endings in vcproj files > (these being plain XML files). I don't remember whether it would > tolerate them in .sln files (as these are *not* XML files); this is > why the .sln file is set to binary. FYI, I tried changing pcbuild.sln to use plain \n, and it worked fine. From greg.ewing at canterbury.ac.nz Fri Mar 10 00:46:49 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 10 Mar 2006 12:46:49 +1300 Subject: [Python-Dev] Making builtins more efficient In-Reply-To: <1141915806.11091.127.camel@grey> References: <1141879691.11091.78.camel@grey> <440FF9CB.5030407@gmail.com> <79990c6b0603090400h25dd2c7ev3d5c379f6529f3c2@mail.gmail.com> <1141915806.11091.127.camel@grey> Message-ID: <4410BE69.3080004@canterbury.ac.nz> Steven Elliott wrote: > One way of handling it is to > alter STORE_ATTR (op code for assigning to mod.str) to always check to > see if the key being assigned is one of the default builtins. If it is, > then the module's indexed array of builtins is assigned to. As long as you're going to all that trouble, it doesn't seem like it would be much harder to treat all global names that way, instead of just a predefined set. The compiler already knows all of the names that are used as globals in the module's code. > That's great, but I'm curious if additional gains can be > made be focusing just on builtins. As long as builtins can be shadowed, I can't see how to make any extra use of the fact that it's a builtin. A semantic change would be needed, such as forbidding shadowing of builtins, or at least forbidding this from outside the module. Greg From greg.ewing at canterbury.ac.nz Fri Mar 10 00:51:58 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 10 Mar 2006 12:51:58 +1300 Subject: [Python-Dev] Making builtins more efficient In-Reply-To: <007c01c64399$af03ffd0$a600000a@RaymondLaptop1> References: <1141879691.11091.78.camel@grey> <007c01c64399$af03ffd0$a600000a@RaymondLaptop1> Message-ID: <4410BF9E.1080301@canterbury.ac.nz> Raymond Hettinger wrote: > That is going to be difficult as long as it is legal to write: > > True = 0 BTW, are there any plans to make True and False hard constants in 3.0 (like None is now)? Maybe also others like Ellipsis, NotImplemented, etc. Greg From thomas at python.org Fri Mar 10 01:14:50 2006 From: thomas at python.org (Thomas Wouters) Date: Fri, 10 Mar 2006 01:14:50 +0100 Subject: [Python-Dev] ctypes is in SVN now. In-Reply-To: <441077A1.4050503@python.net> References: <9e804ac0603091028q56e8171dx31a130c247635532@mail.gmail.com> <441077A1.4050503@python.net> Message-ID: <9e804ac0603091614o7d527cb4s7ea2f583f3eec4c2@mail.gmail.com> On 3/9/06, Thomas Heller wrote: > > Thomas Wouters wrote: > > On 3/9/06, Thomas Heller wrote: > >> ctypes is in SVN now, and the buildbot is green, > > > > > > Well, only by accident; Neal's amd64 machine has been offline, or you > would > > have seen yellow blocks for warnings: ctypes has yet to be > Py_ssize_t'ed. Do > > Do compiler warnings create yellow blocks? There are other warnings on > other machines > as well. OTOH, testing ctypes (see below) can cause core dumps, these > should create > orange blinking blocks ;-) I thought they did, but I guess I was wrong :) Or maybe just the header turns orange (not yellow.) > you want to do that yourself, or do you want me to submit a patch? (Or I > > could just check it in ;) > > You can do it faster then me, I assume - so go ahead and check it in. > I'll backport it to the upstream ctypes CVS repository. I'm not sure if I'll be faster. It's quite a lot of work to make it completely Py_ssize_t-aware (but worth it, IMHO; ints just don't cut it on 64-bit platforms. :) I have a simple patch to silence the warnings, but it doesn't support indexing beyond int-size and such. I'm also going through all of the code and changing most appropriate things into Py_ssize_t's, and that would be much nicer. I'll probably have some XXX markers left when I'm done, though. Also, how stable should the C API be? Does the C code have any direct users besides Python? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060310/b77eac73/attachment.html From guido at python.org Fri Mar 10 02:26:35 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 9 Mar 2006 17:26:35 -0800 Subject: [Python-Dev] Making builtins more efficient In-Reply-To: <4410BF9E.1080301@canterbury.ac.nz> References: <1141879691.11091.78.camel@grey> <007c01c64399$af03ffd0$a600000a@RaymondLaptop1> <4410BF9E.1080301@canterbury.ac.nz> Message-ID: On 3/9/06, Greg Ewing wrote: > BTW, are there any plans to make True and False hard > constants in 3.0 (like None is now)? Maybe also > others like Ellipsis, NotImplemented, etc. I don't think we should make any of these keywords. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Fri Mar 10 02:29:21 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 09 Mar 2006 20:29:21 -0500 Subject: [Python-Dev] Making builtins more efficient In-Reply-To: <4410BE69.3080004@canterbury.ac.nz> References: <1141915806.11091.127.camel@grey> <1141879691.11091.78.camel@grey> <440FF9CB.5030407@gmail.com> <79990c6b0603090400h25dd2c7ev3d5c379f6529f3c2@mail.gmail.com> <1141915806.11091.127.camel@grey> Message-ID: <5.1.1.6.0.20060309201636.01fd50e0@mail.telecommunity.com> At 12:46 PM 3/10/2006 +1300, Greg Ewing wrote: >Steven Elliott wrote: > > One way of handling it is to > > alter STORE_ATTR (op code for assigning to mod.str) to always check to > > see if the key being assigned is one of the default builtins. If it is, > > then the module's indexed array of builtins is assigned to. > >As long as you're going to all that trouble, it >doesn't seem like it would be much harder to treat >all global names that way, instead of just a predefined >set. The compiler already knows all of the names that >are used as globals in the module's code. But knowing that an operation is a builtin allows for the possibility of invoking the equivalent C operation directly in the interpreter (e.g. via a jump table), thus letting us translate something like "len(foo)" from: LOAD_GLOBAL len LOAD_FAST foo CALL_FUNCTION 1 into something like: LOAD_FAST foo BUILTIN_OP len, 1 And, the BUILTIN_OP could invoke a C function passing in a pointer to the arguments on the stack, so as to bypass tuple allocation, and the C function would use PySequence_Length() rather than going through the Python calling convention to PyObject_Call() the 'len' object. Now, whether that'll actually speed real programs up much, I don't know. But there are certainly a lot of things being cut out of the process using this approach, including an opcode fetch/decode, two dictionary lookups (one failing, one successful), and perhaps some tuplefying (only for C funcs w argcount>1, since those builtins don't need an argtuple IIRC). And, even if it does speed things up a good deal, there's still a question of whether it should be done, when some real systems hack modules' builtins for testing. However, if BUILTIN_OP were to verify at runtime that __builtins__ is the interpreter standard builtins (i.e., the restricted-mode check), then it could dynamically choose to do the slower operation lookup. That would allow you to hack a module's builtins by giving it a fresh __builtins__ dictionary to implement that kind of monkeypatching. From tim.peters at gmail.com Fri Mar 10 03:37:07 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 9 Mar 2006 21:37:07 -0500 Subject: [Python-Dev] unicodedata.c no longer compiles on Windows Message-ID: <1f7befae0603091837u1f4ea5ecn1ab8660cc0e6bba5@mail.gmail.com> Compiling... unicodedata.c \Code\python\Modules\unicodedata.c(74) : error C2133: 'unicodedata_functions' : unknown size It's griping about this: /* Forward declaration */ static PyMethodDef unicodedata_functions[]; From theller at python.net Fri Mar 10 08:27:57 2006 From: theller at python.net (Thomas Heller) Date: Fri, 10 Mar 2006 08:27:57 +0100 Subject: [Python-Dev] ctypes is in SVN now. In-Reply-To: <9e804ac0603091614o7d527cb4s7ea2f583f3eec4c2@mail.gmail.com> References: <9e804ac0603091028q56e8171dx31a130c247635532@mail.gmail.com> <441077A1.4050503@python.net> <9e804ac0603091614o7d527cb4s7ea2f583f3eec4c2@mail.gmail.com> Message-ID: <44112A7D.2070102@python.net> Thomas Wouters wrote: (about ctypes and Py_ssize_t) >> you want to do that yourself, or do you want me to submit a patch? (Or I >>> could just check it in ;) >> You can do it faster then me, I assume - so go ahead and check it in. >> I'll backport it to the upstream ctypes CVS repository. > > > I'm not sure if I'll be faster. It's quite a lot of work to make it > completely Py_ssize_t-aware (but worth it, IMHO; ints just don't cut it on > 64-bit platforms. :) I have a simple patch to silence the warnings, but it > doesn't support indexing beyond int-size and such. I'm also going through > all of the code and changing most appropriate things into Py_ssize_t's, and > that would be much nicer. I'll probably have some XXX markers left when I'm > done, though. Also, how stable should the C API be? Does the C code have any > direct users besides Python? Apart from a few functions that are used as foreign functions from ctypes itself (memmove and memset currently), the C api is not used from the outside. The important thing when changing something in the ctypes sources is to run the unittests (*) on as many platforms and Python versions as possible - currently I do this on all the sourceforge compile farm machines that have either Python >= 2.3, or where I have been able to compile Python myself. Plus WindowsXP 32 and OSX 10.4. This is for the ctypes upstream version, of course. If your patch is going to change more than a few int -> Py_ssize_t replacements it would probably be less work to do it in ctypes CVS instead of Python SVN. Maintaining code in multiple repositories is no fun ;) (*) ctypes unittests are run by 'python ctypes/test/runtests.py', for example. Thomas From theller at python.net Fri Mar 10 08:48:32 2006 From: theller at python.net (Thomas Heller) Date: Fri, 10 Mar 2006 08:48:32 +0100 Subject: [Python-Dev] Developing/patching ctypes (was: Re: integrating ctypes into python) In-Reply-To: <4f0b69dc0603091623jefdb8fhf4b13695bbb491dc@mail.gmail.com> References: <44082068.5010301@python.net> <440E9614.3000504@python.net> <4f0b69dc0603080104g6053ae47p59be98f81e65ddad@mail.gmail.com> <440F463E.7050009@python.net> <4f0b69dc0603081535q5e2ef69ft99e339a920dbc25e@mail.gmail.com> <44107267.9090806@python.net> <4f0b69dc0603091623jefdb8fhf4b13695bbb491dc@mail.gmail.com> Message-ID: <44112F50.3020506@python.net> [CC to python-dev again] Hye-Shik Chang wrote: > On 3/10/06, Thomas Heller wrote: >> Hye-Shik Chang wrote: >>> On 3/9/06, Thomas Heller wrote: >>>> I've now committed ctypes 0.9.9.4 into SVN. It seems to build, at least on the buildbots >>>> that are online. >>> Good work! It's all green on my machines (FreeBSD amd64/sparc64). >>> But gcc outputs some warnings: >>> >>> 1) libffi/include/ffi.h.in: void (*fn)() -> void (*void)(void) >> This is not the only file that would have to be changed. I wonder why they don't change >> it in gcc itself. > > This *is* the only change to calm gcc warnings down except Py_ssize_t > stuff. :-) > >>> 2) many of Py_ssize_t / int prototype mismatches. (eg. sq_repeat) >>> Could we make ctypes Py_ssize_t clean? >>> 3) ctypes doesn't support Py_ssize_t nor ssize_t yet. Because major >>> portion of our pythonapi uses it now, it would be better to have it. >> This should be done in ctypes CVS, and the result imported into python SVN after that. >> At least ctypes (the Python module) has an c_size_t type now ;-). > > I'll do it in this weekend. I saw that branch_1_0 is merged into HEAD. > Which branch are you using for Python integration at now? I did 'cvs export -r release_0_9_9_4' from the SF repository and then imported this into python SVN external/ctypes-0.9.9.4. The 'release_0_9_9_4' tag is on the 'branch_1_0' branch. I moved on a little bit on that branch, later merged everything into SF CVS HEAD, and continued to work there. We'll have to coordinate this. Different people working in different repositories will become a maintaince nightmare, so there should be an official 'master' repository for ctypes. Question for python-dev: Would it be a solution to move the 'official' ctypes development into Python SVN external/ctypes, or would this be considered abuse? Another location in SVN could be used as well, if external is though to contain only vendor drops... Thomas From martin at v.loewis.de Fri Mar 10 09:01:00 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 10 Mar 2006 09:01:00 +0100 Subject: [Python-Dev] Developing/patching ctypes In-Reply-To: <44112F50.3020506@python.net> References: <44082068.5010301@python.net> <440E9614.3000504@python.net> <4f0b69dc0603080104g6053ae47p59be98f81e65ddad@mail.gmail.com> <440F463E.7050009@python.net> <4f0b69dc0603081535q5e2ef69ft99e339a920dbc25e@mail.gmail.com> <44107267.9090806@python.net> <4f0b69dc0603091623jefdb8fhf4b13695bbb491dc@mail.gmail.com> <44112F50.3020506@python.net> Message-ID: <4411323C.3000306@v.loewis.de> Thomas Heller wrote: > Would it be a solution to move the 'official' ctypes development into > Python SVN external/ctypes, or would this be considered abuse? Another > location in SVN could be used as well, if external is though to contain > only vendor drops... external indeed is meant only for vendor drops. I personally don't mind having the "upstream" ctypes repository also in svn.python.org; this would be similar to distutils, setuptools, and (I think) the email package. Currently, some of them live in sandbox/trunk, but I wouldn't mind it living in /ctypes/trunk, either. Be aware that the set of committers to svn.python.org/projects is currently restricted to Python committers. Regards, Martin From thomas at python.org Fri Mar 10 09:50:12 2006 From: thomas at python.org (Thomas Wouters) Date: Fri, 10 Mar 2006 09:50:12 +0100 Subject: [Python-Dev] Developing/patching ctypes (was: Re: integrating ctypes into python) In-Reply-To: <44112F50.3020506@python.net> References: <440E9614.3000504@python.net> <4f0b69dc0603080104g6053ae47p59be98f81e65ddad@mail.gmail.com> <440F463E.7050009@python.net> <4f0b69dc0603081535q5e2ef69ft99e339a920dbc25e@mail.gmail.com> <44107267.9090806@python.net> <4f0b69dc0603091623jefdb8fhf4b13695bbb491dc@mail.gmail.com> <44112F50.3020506@python.net> Message-ID: <9e804ac0603100050g6aeeabebi96392e93b4c2eea1@mail.gmail.com> On 3/10/06, Thomas Heller wrote: > Would it be a solution to move the 'official' ctypes development into > Python SVN external/ctypes, or would this be considered abuse? Another > location in SVN could be used as well, if external is though to contain > only vendor drops... If all the developers are also python-developers (or at least have svn access), and the source is supposed to be identical, SVN obviously beats SVN :) I can work equally well with the main repository in CVS, though, and I've had quite a bit of practice in merging changes. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060310/82a40e71/attachment-0001.htm From theller at python.net Fri Mar 10 12:16:59 2006 From: theller at python.net (Thomas Heller) Date: Fri, 10 Mar 2006 12:16:59 +0100 Subject: [Python-Dev] Developing/patching ctypes In-Reply-To: <9e804ac0603100050g6aeeabebi96392e93b4c2eea1@mail.gmail.com> References: <440E9614.3000504@python.net> <4f0b69dc0603080104g6053ae47p59be98f81e65ddad@mail.gmail.com> <440F463E.7050009@python.net> <4f0b69dc0603081535q5e2ef69ft99e339a920dbc25e@mail.gmail.com> <44107267.9090806@python.net> <4f0b69dc0603091623jefdb8fhf4b13695bbb491dc@mail.gmail.com> <44112F50.3020506@python.net> <9e804ac0603100050g6aeeabebi96392e93b4c2eea1@mail.gmail.com> Message-ID: <4411602B.5050809@python.net> Thomas Wouters wrote: > On 3/10/06, Thomas Heller wrote: > >> Would it be a solution to move the 'official' ctypes development into >> Python SVN external/ctypes, or would this be considered abuse? Another >> location in SVN could be used as well, if external is though to contain >> only vendor drops... > > > If all the developers are also python-developers (or at least have svn > access), and the source is supposed to be identical, SVN obviously beats SVN I didn't know that SVN is *that great* :) ! > :) I can work equally well with the main repository in CVS, though, and I've > had quite a bit of practice in merging changes. Cool. Martin v. L?wis wrote: > external indeed is meant only for vendor drops. > > I personally don't mind having the "upstream" ctypes repository also > in svn.python.org; this would be similar to distutils, setuptools, > and (I think) the email package. Currently, some of them live in > sandbox/trunk, but I wouldn't mind it living in /ctypes/trunk, either. > Be aware that the set of committers to svn.python.org/projects is > currently restricted to Python committers. I think the currently active ctypes developers are also Python committers, additionally I've asked on the ctypes-users list for objections. I'll think about this stuff over the weekend, currently I tend to moving ctypes to Python SVN. Whether it will be /ctypes/trunk or sandbox/ctypes I don't really care. In the meantime, please: If anyone is going to make fixes to the ctypes source code (apart from Tim's regular whitespace cleanup), please do this in the ctypes CVS repository on sourceforge, in the trunk. If anyone (of the Python developers) wants to do this and doesn't currently have commit access, please mail me your SF userid privately. Note that both perky and Thomas Wouters have promised to do Py_ssize_t fixes - please coordinate that to avoid double work. Thomas From barry at python.org Fri Mar 10 14:20:36 2006 From: barry at python.org (Barry Warsaw) Date: Fri, 10 Mar 2006 08:20:36 -0500 Subject: [Python-Dev] Developing/patching ctypes In-Reply-To: <4411323C.3000306@v.loewis.de> References: <44082068.5010301@python.net> <440E9614.3000504@python.net> <4f0b69dc0603080104g6053ae47p59be98f81e65ddad@mail.gmail.com> <440F463E.7050009@python.net> <4f0b69dc0603081535q5e2ef69ft99e339a920dbc25e@mail.gmail.com> <44107267.9090806@python.net> <4f0b69dc0603091623jefdb8fhf4b13695bbb491dc@mail.gmail.com> <44112F50.3020506@python.net> <4411323C.3000306@v.loewis.de> Message-ID: <1141996836.15512.10.camel@geddy.wooz.org> On Fri, 2006-03-10 at 09:01 +0100, "Martin v. L?wis" wrote: > I personally don't mind having the "upstream" ctypes repository also > in svn.python.org; this would be similar to distutils, setuptools, > and (I think) the email package. Currently, some of them live in > sandbox/trunk, but I wouldn't mind it living in /ctypes/trunk, either. > Be aware that the set of committers to svn.python.org/projects is > currently restricted to Python committers. Actually, for the most part email pkg lives inside Python and I external it into the sandbox for the standalone releases. email 4.0 is a little different because I wanted to do the development outside Python to reduce the chance of breakage, but I am planning on merging the code back into the Python trunk, possibly this weekend. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060310/8813bb4d/attachment.pgp From tim.peters at gmail.com Fri Mar 10 16:59:53 2006 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 10 Mar 2006 10:59:53 -0500 Subject: [Python-Dev] Developing/patching ctypes In-Reply-To: <4411602B.5050809@python.net> References: <4f0b69dc0603080104g6053ae47p59be98f81e65ddad@mail.gmail.com> <440F463E.7050009@python.net> <4f0b69dc0603081535q5e2ef69ft99e339a920dbc25e@mail.gmail.com> <44107267.9090806@python.net> <4f0b69dc0603091623jefdb8fhf4b13695bbb491dc@mail.gmail.com> <44112F50.3020506@python.net> <9e804ac0603100050g6aeeabebi96392e93b4c2eea1@mail.gmail.com> <4411602B.5050809@python.net> Message-ID: <1f7befae0603100759u2de80260u9c351fc472cf63e2@mail.gmail.com> [Thomas Heller] > ... > In the meantime, please: If anyone is going to make fixes to the ctypes source > code (apart from Tim's regular whitespace cleanup), please do this in the > ctypes CVS repository on sourceforge, in the trunk. FYI, my regular whitespace cleanup consists of running reindent.py from the root of a Python checkout ("cd python; reindent.py -r ."). There's nothing specific to the Python project about that little procedure, and, e.g., I regularly did the same for Zope Corp's ZODB project. From theller at python.net Fri Mar 10 17:40:27 2006 From: theller at python.net (Thomas Heller) Date: Fri, 10 Mar 2006 17:40:27 +0100 Subject: [Python-Dev] Developing/patching ctypes In-Reply-To: <1f7befae0603100759u2de80260u9c351fc472cf63e2@mail.gmail.com> References: <4f0b69dc0603080104g6053ae47p59be98f81e65ddad@mail.gmail.com> <440F463E.7050009@python.net> <4f0b69dc0603081535q5e2ef69ft99e339a920dbc25e@mail.gmail.com> <44107267.9090806@python.net> <4f0b69dc0603091623jefdb8fhf4b13695bbb491dc@mail.gmail.com> <44112F50.3020506@python.net> <9e804ac0603100050g6aeeabebi96392e93b4c2eea1@mail.gmail.com> <4411602B.5050809@python.net> <1f7befae0603100759u2de80260u9c351fc472cf63e2@mail.gmail.com> Message-ID: Tim Peters wrote: > [Thomas Heller] >> ... >> In the meantime, please: If anyone is going to make fixes to the ctypes source >> code (apart from Tim's regular whitespace cleanup), please do this in the >> ctypes CVS repository on sourceforge, in the trunk. > > FYI, my regular whitespace cleanup consists of running reindent.py > from the root of a Python checkout ("cd python; reindent.py -r ."). > There's nothing specific to the Python project about that little > procedure, and, e.g., I regularly did the same for Zope Corp's ZODB > project. I've nothing against that, and I should myself get used to do the same in my own repositories from time to time. BTW: The buildbot reports ctypes test failures on the gentoo amd64 machine: http://www.python.org/dev/buildbot/trunk/amd64%20gentoo%20trunk/builds/277/step-test/0 Is there a way to get the actual failures somehow? Running the tests in verbose mode by default is probably not a good idea, because it may make tests fail, AFAIK. If not, I have no other choice (since I don't have a 64-bit machine myself) than to test this in the sourceforge compilefarm amd64 machine, which is a pain since they don't allow outbound network from there. Thomas From ianb at colorstudy.com Fri Mar 10 18:25:26 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 10 Mar 2006 11:25:26 -0600 Subject: [Python-Dev] multidict API Message-ID: <4411B686.1040502@colorstudy.com> I'm not really making any actionable proposal here, so maybe this is off-topic; if so, sorry. Back during the defaultdict discussion I proposed a multidict object (http://mail.python.org/pipermail/python-dev/2006-February/061264.html) -- right now I need to implement one to represent web form submissions. It would also be ordered in that case. The question then is what the API should look like for such an object -- an ordered, multi-value dictionary. I would really like if this object was in the collections module, but I'm too lazy to try to pursue that now. But if it did show up, I'd like the class I write to look the same. There's some open questions I see: * Does __getitem__ return a list of all matching keys (never a KeyError, though possibly returning []), or does it return the first matching key? * Either way, I assume there will be another method, like getfirst or getall, that will present the other choice. What would it be named? Should it have a default? * Should there be a method to get a single value, that implicitly asserts that there is only one matching key? * Should the default for .get() be None, or something else? * Does __setitem__ overwrite any or all values with matching keys? * If so, there should be another method like .add(key, value) which does not overwrite. Or, if __setitem__ does not overwrite, then there should be a method that does. * Does __delitem__ raise a KeyError if the key is not found? * Does .keys() return all unique keys, or all keys in order (meaning a key may show up more than once in the list)? I really could go either way on all of these questions, though I think there's constraints -- answer one of the questions and another becomes obvious. But you can answer them in whatever order you want. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From raymond.hettinger at verizon.net Fri Mar 10 18:53:05 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Fri, 10 Mar 2006 09:53:05 -0800 Subject: [Python-Dev] multidict API References: <4411B686.1040502@colorstudy.com> Message-ID: <005501c6446b$7c152db0$a600000a@RaymondLaptop1> [Ian Bicking] > The question then is what the API should look like for such an object -- > an ordered, multi-value dictionary. May I suggest that multidict begin it's life as a cookbook recipe so that its API can mature. Raymond From ianb at colorstudy.com Fri Mar 10 19:12:49 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 10 Mar 2006 12:12:49 -0600 Subject: [Python-Dev] multidict API In-Reply-To: <005501c6446b$7c152db0$a600000a@RaymondLaptop1> References: <4411B686.1040502@colorstudy.com> <005501c6446b$7c152db0$a600000a@RaymondLaptop1> Message-ID: <4411C1A1.8010209@colorstudy.com> Raymond Hettinger wrote: > [Ian Bicking] > >>The question then is what the API should look like for such an object -- >>an ordered, multi-value dictionary. > > > May I suggest that multidict begin it's life as a cookbook recipe so that its > API can mature. There's already quite a few recipes out there. But I should probably collect them as well. http://www.voidspace.org.uk/python/odict.html http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/107747 http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/438823 http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/173072 http://urchin.earth.li/~twic/odict.py http://www.astro.washington.edu/owen/ROPython.html http://home.arcor.de/wolfgang.grafen/Python/Modules/Modules.html email.Message.Message http://cvs.eby-sarna.com/wsgiref/src/wsgiref/headers.py?view=markup Well, there's a few, mostly ordered, some multivalue. A comparison would be helpful, but maybe a little later. odict is probably the most filled-out, though it is probably more listish than I really would like. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From nnorwitz at gmail.com Fri Mar 10 19:43:17 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Fri, 10 Mar 2006 10:43:17 -0800 Subject: [Python-Dev] Developing/patching ctypes In-Reply-To: References: <440F463E.7050009@python.net> <4f0b69dc0603081535q5e2ef69ft99e339a920dbc25e@mail.gmail.com> <44107267.9090806@python.net> <4f0b69dc0603091623jefdb8fhf4b13695bbb491dc@mail.gmail.com> <44112F50.3020506@python.net> <9e804ac0603100050g6aeeabebi96392e93b4c2eea1@mail.gmail.com> <4411602B.5050809@python.net> <1f7befae0603100759u2de80260u9c351fc472cf63e2@mail.gmail.com> Message-ID: On 3/10/06, Thomas Heller wrote: > > BTW: The buildbot reports ctypes test failures on the gentoo amd64 machine: > > http://www.python.org/dev/buildbot/trunk/amd64%20gentoo%20trunk/builds/277/step-test/0 > > Is there a way to get the actual failures somehow? Running the tests in verbose > mode by default is probably not a good idea, because it may make tests fail, AFAIK. I want to modify regrtest so the traceback info is stored in a file, so you can still retrieve the data after a test run. I haven't started making this mod yet. > If not, I have no other choice (since I don't have a 64-bit machine myself) than > to test this in the sourceforge compilefarm amd64 machine, which is a pain > since they don't allow outbound network from there. I ran the test manually last night, there were 12 errors IIRC. It was stuff like strings not being equal. I can't send you the verbose run right now. I'll try to do it when I get a chance, but that won't be before Sunday. n From theller at python.net Fri Mar 10 22:01:28 2006 From: theller at python.net (Thomas Heller) Date: Fri, 10 Mar 2006 22:01:28 +0100 Subject: [Python-Dev] ctypes is in SVN now. In-Reply-To: <1f7befae0603091243r4f7a55f3n35a3c753e2e31308@mail.gmail.com> References: <9e804ac0603091028q56e8171dx31a130c247635532@mail.gmail.com> <441077A1.4050503@python.net> <1f7befae0603091243r4f7a55f3n35a3c753e2e31308@mail.gmail.com> Message-ID: Tim Peters wrote: > [Thomas Heller] >> ... >> And I never had tried it before on a sparc machine - all the intel and ppc processors >> seem to have no problems with it. > > Pentiums don't enforce "natural" alignment restrictions, but run much > slower on unaligned access (varying by specific chip model, and > generally more heavily penalized as time goes on). In the good old > days, Pentium was one of dozens of competing architectures, and was > the oddball in catering to unaligned access. Now it's eternal > "backward compatibility" with an early implementation accident. Most > other architectures never catered to unaligned access, or did so only > at the cost of generating an interrupt so that kernel-mode software > could fake unaligned access. Bottom line is that unaligned access > isn't portable and never was, and even on architectures where "it > works" it can be extremely expensive to use it. I was astonished to find out that also the ARM processor on Windows CE doesn't support unaligned accesses. When one thinks about it, it probably makes sense on small devices. Thomas From martin at v.loewis.de Fri Mar 10 22:04:44 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 10 Mar 2006 22:04:44 +0100 Subject: [Python-Dev] Developing/patching ctypes In-Reply-To: References: <4f0b69dc0603080104g6053ae47p59be98f81e65ddad@mail.gmail.com> <440F463E.7050009@python.net> <4f0b69dc0603081535q5e2ef69ft99e339a920dbc25e@mail.gmail.com> <44107267.9090806@python.net> <4f0b69dc0603091623jefdb8fhf4b13695bbb491dc@mail.gmail.com> <44112F50.3020506@python.net> <9e804ac0603100050g6aeeabebi96392e93b4c2eea1@mail.gmail.com> <4411602B.5050809@python.net> <1f7befae0603100759u2de80260u9c351fc472cf63e2@mail.gmail.com> Message-ID: <4411E9EC.7050408@v.loewis.de> Thomas Heller wrote: > BTW: The buildbot reports ctypes test failures on the gentoo amd64 machine: FWIW, ctypes doesn't even build on a Windows AMD64 machine. It wants to use the Windows x86 FFI code, which does not compile with the AMD64 compiler. Regards, Martin From martin at v.loewis.de Fri Mar 10 22:05:17 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 10 Mar 2006 22:05:17 +0100 Subject: [Python-Dev] unicodedata.c no longer compiles on Windows In-Reply-To: <1f7befae0603091837u1f4ea5ecn1ab8660cc0e6bba5@mail.gmail.com> References: <1f7befae0603091837u1f4ea5ecn1ab8660cc0e6bba5@mail.gmail.com> Message-ID: <4411EA0D.3000005@v.loewis.de> Tim Peters wrote: > It's griping about this: > > /* Forward declaration */ > static PyMethodDef unicodedata_functions[]; This is now fixed. Martin From barry at python.org Fri Mar 10 22:05:40 2006 From: barry at python.org (Barry Warsaw) Date: Fri, 10 Mar 2006 16:05:40 -0500 Subject: [Python-Dev] multidict API In-Reply-To: <4411B686.1040502@colorstudy.com> References: <4411B686.1040502@colorstudy.com> Message-ID: <1142024740.23864.116.camel@resist.wooz.org> On Fri, 2006-03-10 at 11:25 -0600, Ian Bicking wrote: > I'm not really making any actionable proposal here, so maybe this is > off-topic; if so, sorry. > > Back during the defaultdict discussion I proposed a multidict object > (http://mail.python.org/pipermail/python-dev/2006-February/061264.html) > -- right now I need to implement one to represent web form submissions. > It would also be ordered in that case. FWIW, the email package's Message class implements something similar in its mapping API, to represent message headers. I'm not saying that it's a good general approach to the problem, but I do think the way we solved it works well for Messages. The implementation is also probably not very good for general purposes, since it can involve linear searches of lists, but for Messages, which don't typically have that many headers, I think it's fine. Messages keep track of the order in which the headers are added, and you may have multiple values for each header. If you delete a header and re-add it, it gets added to the end. So iterating over the headers gives you each header in order, including duplicates. __getitem__() however will return only one of those headers; which exactly you get is technically undefined. get() has the same semantics. There's a new get_all() method that takes a key and returns a list of all the values for that key (header). __delitem__() removes all matching keys. > * Does __getitem__ return a list of all matching keys (never a KeyError, > though possibly returning []), or does it return the first matching key? See above -- it returns one matching key, but the spec doesn't specify which one (see the implementation for the obvious answer ;). One other semantic difference with Messages is that __getitem__() on a missing header returns None -- it does not raise a KeyError. Practicality beats purity. > * Either way, I assume there will be another method, like getfirst or > getall, that will present the other choice. What would it be named? > Should it have a default? In Message, it's called get_all() and it does take an optional default, just like get(). > * Should there be a method to get a single value, that implicitly > asserts that there is only one matching key? Message doesn't have this. > * Should the default for .get() be None, or something else? Message.get() defaults to None. > * Does __setitem__ overwrite any or all values with matching keys? Message's __setitem__() appends the header (there's a separate add_header() method that takes a bunch of arguments specific to email headers) but it does not overwrite any existing header. The idiom used to replace a header is usually: del msg['some-header'] msg['Some-Header'] = 'Wizzy Mailerified' Oh yeah, keys are case-preserving but case-insensitive because in email messages 'Some-Header: Foo' is the same as 'some-header: Foo', but we want to be as idempotent as possible. > * If so, there should be another method like .add(key, value) which does > not overwrite. Or, if __setitem__ does not overwrite, then there should > be a method that does. Message.replace_header() preserves header order, and it replaces the value for the first header found. > * Does __delitem__ raise a KeyError if the key is not found? For Message, the answer is "no". > * Does .keys() return all unique keys, or all keys in order (meaning a > key may show up more than once in the list)? Message.keys() returns them in order. Likewise .values() and .items(). > I really could go either way on all of these questions, though I think > there's constraints -- answer one of the questions and another becomes > obvious. But you can answer them in whatever order you want. Cool, thanks! For simplicity, I've answered them in the order they were asked, just like Message would. :) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060310/e1b69e06/attachment.pgp From barry at python.org Fri Mar 10 22:07:35 2006 From: barry at python.org (Barry Warsaw) Date: Fri, 10 Mar 2006 16:07:35 -0500 Subject: [Python-Dev] multidict API In-Reply-To: <4411C1A1.8010209@colorstudy.com> References: <4411B686.1040502@colorstudy.com> <005501c6446b$7c152db0$a600000a@RaymondLaptop1> <4411C1A1.8010209@colorstudy.com> Message-ID: <1142024855.23864.118.camel@resist.wooz.org> On Fri, 2006-03-10 at 12:12 -0600, Ian Bicking wrote: > There's already quite a few recipes out there. But I should probably > collect them as well. > > http://www.voidspace.org.uk/python/odict.html > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/107747 > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/438823 > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/173072 > http://urchin.earth.li/~twic/odict.py > http://www.astro.washington.edu/owen/ROPython.html > http://home.arcor.de/wolfgang.grafen/Python/Modules/Modules.html > email.Message.Message > http://cvs.eby-sarna.com/wsgiref/src/wsgiref/headers.py?view=markup > > Well, there's a few, mostly ordered, some multivalue. A comparison > would be helpful, but maybe a little later. odict is probably the most > filled-out, though it is probably more listish than I really would like. Actually, my suspicion is that there won't be a general enough solution to warrant inclusion in the stdlib. But hey, if you can create an 80% solution, that would be nice. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060310/f2ab0abb/attachment.pgp From theller at python.net Fri Mar 10 22:12:49 2006 From: theller at python.net (Thomas Heller) Date: Fri, 10 Mar 2006 22:12:49 +0100 Subject: [Python-Dev] Developing/patching ctypes In-Reply-To: <4411E9EC.7050408@v.loewis.de> References: <4f0b69dc0603080104g6053ae47p59be98f81e65ddad@mail.gmail.com> <440F463E.7050009@python.net> <4f0b69dc0603081535q5e2ef69ft99e339a920dbc25e@mail.gmail.com> <44107267.9090806@python.net> <4f0b69dc0603091623jefdb8fhf4b13695bbb491dc@mail.gmail.com> <44112F50.3020506@python.net> <9e804ac0603100050g6aeeabebi96392e93b4c2eea1@mail.gmail.com> <4411602B.5050809@python.net> <1f7befae0603100759u2de80260u9c351fc472cf63e2@mail.gmail.com> <4411E9EC.7050408@v.loewis.de> Message-ID: Martin v. L?wis wrote: > Thomas Heller wrote: >> BTW: The buildbot reports ctypes test failures on the gentoo amd64 machine: > > FWIW, ctypes doesn't even build on a Windows AMD64 machine. It wants to > use the Windows x86 FFI code, which does not compile with the AMD64 > compiler. On such a machine probably other source files should be used. I have no such machine - is it possible to build the 64-bit version on a 32-bit machine (with the platform SDK)? Thomas From martin at v.loewis.de Fri Mar 10 22:28:43 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 10 Mar 2006 22:28:43 +0100 Subject: [Python-Dev] Developing/patching ctypes In-Reply-To: References: <440F463E.7050009@python.net> <4f0b69dc0603081535q5e2ef69ft99e339a920dbc25e@mail.gmail.com> <44107267.9090806@python.net> <4f0b69dc0603091623jefdb8fhf4b13695bbb491dc@mail.gmail.com> <44112F50.3020506@python.net> <9e804ac0603100050g6aeeabebi96392e93b4c2eea1@mail.gmail.com> <4411602B.5050809@python.net> <1f7befae0603100759u2de80260u9c351fc472cf63e2@mail.gmail.com> Message-ID: <4411EF8B.2040501@v.loewis.de> Neal Norwitz wrote: > I want to modify regrtest so the traceback info is stored in a file, > so you can still retrieve the data after a test run. I haven't > started making this mod yet. I took a different approach now, adding an option to regrtest to re-run failed tests in verbose mode; this is what the buildbot now uses. Regards, Martin From theller at python.net Fri Mar 10 22:15:47 2006 From: theller at python.net (Thomas Heller) Date: Fri, 10 Mar 2006 22:15:47 +0100 Subject: [Python-Dev] Developing/patching ctypes In-Reply-To: References: <440F463E.7050009@python.net> <4f0b69dc0603081535q5e2ef69ft99e339a920dbc25e@mail.gmail.com> <44107267.9090806@python.net> <4f0b69dc0603091623jefdb8fhf4b13695bbb491dc@mail.gmail.com> <44112F50.3020506@python.net> <9e804ac0603100050g6aeeabebi96392e93b4c2eea1@mail.gmail.com> <4411602B.5050809@python.net> <1f7befae0603100759u2de80260u9c351fc472cf63e2@mail.gmail.com> Message-ID: Neal Norwitz wrote: > On 3/10/06, Thomas Heller wrote: >> BTW: The buildbot reports ctypes test failures on the gentoo amd64 machine: >> >> http://www.python.org/dev/buildbot/trunk/amd64%20gentoo%20trunk/builds/277/step-test/0 >> >> Is there a way to get the actual failures somehow? Running the tests in verbose >> mode by default is probably not a good idea, because it may make tests fail, AFAIK. > > I want to modify regrtest so the traceback info is stored in a file, > so you can still retrieve the data after a test run. I haven't > started making this mod yet. > There are also sporadic test failures on the OSX machine: http://www.python.org/dev/buildbot/trunk/g4%20osx.4%20trunk/builds/345/step-test/0 I think the test is broken - but what I find interesting is that in this case the actual failure is printed: """ test_set test_ctypes test test_ctypes failed -- Traceback (most recent call last): File "/Users/buildslave/bb/trunk.4/build/Lib/ctypes/test/test_leaks.py", line 68, in test_cycles_refcount self.fail("leaking refcounts") AssertionError: leaking refcounts test_long_future """ Why is that? Thomas From martin at v.loewis.de Fri Mar 10 22:42:35 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 10 Mar 2006 22:42:35 +0100 Subject: [Python-Dev] Developing/patching ctypes In-Reply-To: References: <4f0b69dc0603080104g6053ae47p59be98f81e65ddad@mail.gmail.com> <440F463E.7050009@python.net> <4f0b69dc0603081535q5e2ef69ft99e339a920dbc25e@mail.gmail.com> <44107267.9090806@python.net> <4f0b69dc0603091623jefdb8fhf4b13695bbb491dc@mail.gmail.com> <44112F50.3020506@python.net> <9e804ac0603100050g6aeeabebi96392e93b4c2eea1@mail.gmail.com> <4411602B.5050809@python.net> <1f7befae0603100759u2de80260u9c351fc472cf63e2@mail.gmail.com> <4411E9EC.7050408@v.loewis.de> Message-ID: <4411F2CB.1010708@v.loewis.de> Thomas Heller wrote: >>FWIW, ctypes doesn't even build on a Windows AMD64 machine. It wants to >>use the Windows x86 FFI code, which does not compile with the AMD64 >>compiler. > > > On such a machine probably other source files should be used. I have no such > machine - is it possible to build the 64-bit version on a 32-bit machine (with > the platform SDK)? Indeed, this is possible. I don't know of anybody who does so, so far, but in principle, the procedure is described in PCbuild/readme.txt. In essence, you install the platform SDK and vsextcomp, and then build the ReleaseAMD64 solution setting. OTOH, the distutils included with Python 2.5 also honor SDK environments (again), so if you install the platform SDK, open an IA64 or AMD64 build environment, and run upstream's setup.py, it /should/ just use the compiler, header files, etc in the path. If that would help, I could provide you with a Python 2.5 MSI file (for x86), to run distutils, and get the 2.5 headers. OTTH, a lot of things don't work on Win64, so people could probably readily accept the lack of ctypes. Regards, Martin From martin at v.loewis.de Fri Mar 10 22:43:45 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 10 Mar 2006 22:43:45 +0100 Subject: [Python-Dev] Developing/patching ctypes In-Reply-To: References: <440F463E.7050009@python.net> <4f0b69dc0603081535q5e2ef69ft99e339a920dbc25e@mail.gmail.com> <44107267.9090806@python.net> <4f0b69dc0603091623jefdb8fhf4b13695bbb491dc@mail.gmail.com> <44112F50.3020506@python.net> <9e804ac0603100050g6aeeabebi96392e93b4c2eea1@mail.gmail.com> <4411602B.5050809@python.net> <1f7befae0603100759u2de80260u9c351fc472cf63e2@mail.gmail.com> Message-ID: <4411F311.1090402@v.loewis.de> Thomas Heller wrote: > """ > test_set > test_ctypes > test test_ctypes failed -- Traceback (most recent call last): > File "/Users/buildslave/bb/trunk.4/build/Lib/ctypes/test/test_leaks.py", line 68, in test_cycles_refcount > self.fail("leaking refcounts") > AssertionError: leaking refcounts > > test_long_future > """ > > Why is that? Because it is just a single exception. For that, a traceback is printed. Not so for multiple failures. Regards, Martin From martin at v.loewis.de Fri Mar 10 22:52:18 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 10 Mar 2006 22:52:18 +0100 Subject: [Python-Dev] Developing/patching ctypes In-Reply-To: References: <4f0b69dc0603080104g6053ae47p59be98f81e65ddad@mail.gmail.com> <440F463E.7050009@python.net> <4f0b69dc0603081535q5e2ef69ft99e339a920dbc25e@mail.gmail.com> <44107267.9090806@python.net> <4f0b69dc0603091623jefdb8fhf4b13695bbb491dc@mail.gmail.com> <44112F50.3020506@python.net> <9e804ac0603100050g6aeeabebi96392e93b4c2eea1@mail.gmail.com> <4411602B.5050809@python.net> <1f7befae0603100759u2de80260u9c351fc472cf63e2@mail.gmail.com> Message-ID: <4411F512.7020701@v.loewis.de> Thomas Heller wrote: > BTW: The buildbot reports ctypes test failures on the gentoo amd64 machine: > > http://www.python.org/dev/buildbot/trunk/amd64%20gentoo%20trunk/builds/277/step-test/0 > > Is there a way to get the actual failures somehow? They are now in http://www.python.org/dev/buildbot/all/amd64%20gentoo%20trunk/builds/280/step-test/0 Regards, Martin From greg.ewing at canterbury.ac.nz Sat Mar 11 00:35:26 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 11 Mar 2006 12:35:26 +1300 Subject: [Python-Dev] Making builtins more efficient In-Reply-To: References: <1141879691.11091.78.camel@grey> <007c01c64399$af03ffd0$a600000a@RaymondLaptop1> <4410BF9E.1080301@canterbury.ac.nz> Message-ID: <44120D3E.4030904@canterbury.ac.nz> Guido van Rossum wrote: > I don't think we should make any of these keywords. Not even True and False? The only good reasons I can see for anyone wanting to shadow these are backwards compatibility ones. Greg From guido at python.org Sat Mar 11 00:56:50 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 10 Mar 2006 15:56:50 -0800 Subject: [Python-Dev] Making builtins more efficient In-Reply-To: <44120D3E.4030904@canterbury.ac.nz> References: <1141879691.11091.78.camel@grey> <007c01c64399$af03ffd0$a600000a@RaymondLaptop1> <4410BF9E.1080301@canterbury.ac.nz> <44120D3E.4030904@canterbury.ac.nz> Message-ID: On 3/10/06, Greg Ewing wrote: > Guido van Rossum wrote: > > > I don't think we should make any of these keywords. > > Not even True and False? The only good reasons > I can see for anyone wanting to shadow these > are backwards compatibility ones. Not even True and False. I don't see why everything that doesn't make sense to be shadowed ought to become a keyword. That way lies madness. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Sat Mar 11 01:20:11 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 11 Mar 2006 01:20:11 +0100 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot Message-ID: <441217BB.1000801@v.loewis.de> Josiah Carlson told me had has given up getting a Windows buildbot running, because every time he installed VS.NET on his machine, the installation would immediately crash. So if anybody wants to contribute both a machine and time to operate it (including the likely very tedious phase to get any results out of this at all), please contact me. Regards, Martin From nas at arctrix.com Sat Mar 11 01:42:41 2006 From: nas at arctrix.com (Neil Schemenauer) Date: Sat, 11 Mar 2006 00:42:41 +0000 (UTC) Subject: [Python-Dev] Making builtins more efficient References: <1141879691.11091.78.camel@grey> <007c01c64399$af03ffd0$a600000a@RaymondLaptop1> <4410BF9E.1080301@canterbury.ac.nz> <44120D3E.4030904@canterbury.ac.nz> Message-ID: Guido van Rossum wrote: > Not even True and False. > > I don't see why everything that doesn't make sense to be shadowed > ought to become a keyword. That way lies madness. Have you considered whether P3K will disallow names from being shadowed in such as way as to prevent the compiler from determining the namespace to look for a given name? I seen nothing in PEP 3000 related to this issue. Neil From guido at python.org Sat Mar 11 02:21:46 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 10 Mar 2006 17:21:46 -0800 Subject: [Python-Dev] Making builtins more efficient In-Reply-To: References: <1141879691.11091.78.camel@grey> <007c01c64399$af03ffd0$a600000a@RaymondLaptop1> <4410BF9E.1080301@canterbury.ac.nz> <44120D3E.4030904@canterbury.ac.nz> Message-ID: On 3/10/06, Neil Schemenauer wrote: > Guido van Rossum wrote: > > Not even True and False. > > > > I don't see why everything that doesn't make sense to be shadowed > > ought to become a keyword. That way lies madness. > > Have you considered whether P3K will disallow names from being > shadowed in such as way as to prevent the compiler from determining > the namespace to look for a given name? I seen nothing in PEP 3000 > related to this issue. Yes, that's one of my goals. PEP 280 (clumsily) expresses some of the ideas. Currently, my main idea is to allow the compiler to assume that if a known built-in is used in a module, and inspection of the source code for that module reveals no assignment to a global with the same name, the built-in can be assumed to have the standard semantics, and may be hardcoded in any way the compiler seems fit (for example, an opcode that computes len() would be fair game). Exceptions would be made for 'open' and '__import__' (since these are commonly hacked). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From trentm at ActiveState.com Sat Mar 11 03:43:59 2006 From: trentm at ActiveState.com (Trent Mick) Date: Fri, 10 Mar 2006 18:43:59 -0800 Subject: [Python-Dev] Developing/patching ctypes In-Reply-To: <4411F2CB.1010708@v.loewis.de> References: <44107267.9090806@python.net> <4f0b69dc0603091623jefdb8fhf4b13695bbb491dc@mail.gmail.com> <44112F50.3020506@python.net> <9e804ac0603100050g6aeeabebi96392e93b4c2eea1@mail.gmail.com> <4411602B.5050809@python.net> <1f7befae0603100759u2de80260u9c351fc472cf63e2@mail.gmail.com> <4411E9EC.7050408@v.loewis.de> <4411F2CB.1010708@v.loewis.de> Message-ID: <20060311024359.GB17238@activestate.com> > > On such a machine probably other source files should be used. I have no such > > machine - is it possible to build the 64-bit version on a 32-bit machine (with > > the platform SDK)? > > Indeed, this is possible. I don't know of anybody who does so, so far, > but in principle, the procedure is described in PCbuild/readme.txt. In > essence, you install the platform SDK and vsextcomp, and then build the > ReleaseAMD64 solution setting. I do this for ActivePython builds... by setting up the Platform SDK compiler I want in the environment and then using: devenv.com .../pcbuild.sln /useenv /build Release Trent -- Trent Mick TrentM at ActiveState.com From martin at v.loewis.de Sat Mar 11 12:17:58 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 11 Mar 2006 12:17:58 +0100 Subject: [Python-Dev] Developing/patching ctypes In-Reply-To: <20060311024359.GB17238@activestate.com> References: <44107267.9090806@python.net> <4f0b69dc0603091623jefdb8fhf4b13695bbb491dc@mail.gmail.com> <44112F50.3020506@python.net> <9e804ac0603100050g6aeeabebi96392e93b4c2eea1@mail.gmail.com> <4411602B.5050809@python.net> <1f7befae0603100759u2de80260u9c351fc472cf63e2@mail.gmail.com> <4411E9EC.7050408@v.loewis.de> <4411F2CB.1010708@v.loewis.de> <20060311024359.GB17238@activestate.com> Message-ID: <4412B1E6.9000208@v.loewis.de> Trent Mick wrote: > I do this for ActivePython builds... by setting up the Platform SDK > compiler I want in the environment and then using: > > devenv.com .../pcbuild.sln /useenv /build Release Right - that might be the easiest thing to do. Regards, Martin From ben at coverity.com Tue Mar 7 09:58:01 2006 From: ben at coverity.com (Ben Chelf) Date: Tue, 07 Mar 2006 00:58:01 -0800 Subject: [Python-Dev] Coverity Open Source Defect Scan of Python In-Reply-To: References: Message-ID: <440D4B19.8010605@coverity.com> > The Coverty marketing droids need to be a bit less anal about getting > people to register at the website. IMHO, the technology should be Honestly, I laughed out loud when I read this. ;) So thanks for that. > > I'd also encourage Coventry to explain their business model a bit more > clearly. Coventry seems to be supportive of open source projects. > Coverty also seems to be targeting big companies as customers. It's not > clear how arbitrary open source projects (and small companies and > individuals) will be able to take advantage of Coventry's products and > services. > Here's my take on this -- in the last couple of years, I've personally been to hundreds of companies (some big, some small) in an effort to get our technology out there. Of course it's no surprise that I see open source projects everywhere -- as part of infrastructure or part of code bases that people are developing. So from a Coverity perspective, clearly we want to provide source code analysis for the projects that our customers care about (their own as well as open source). Putting on my idealistic hat and remembering back my grad school days, I think we're on to something very new in the world of source code analysis. I really just want every developer to use source code analysis while they write code (remember, idealistic :)). We got a lot of the good publicity in the research lab because there existed this big open source OS that we could test our theories on. So from that angle, I think it makes sense for Coverity to have a strong relationship with the open source community since that community has been helping us pretty much since day 1. This project is just the next step in that...it's certainly not the last. There's plenty more to do to target every developer. -ben From theller at python.net Sat Mar 11 19:08:14 2006 From: theller at python.net (Thomas Heller) Date: Sat, 11 Mar 2006 19:08:14 +0100 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <441217BB.1000801@v.loewis.de> References: <441217BB.1000801@v.loewis.de> Message-ID: Martin v. L?wis wrote: > Josiah Carlson told me had has given up getting a Windows > buildbot running, because every time he installed VS.NET > on his machine, the installation would immediately crash. > > So if anybody wants to contribute both a machine and time > to operate it (including the likely very tedious phase to > get any results out of this at all), please contact me. > Are there any estimates how much network traffic a buildbot would generate? And how must it be connected to the internet - I assume it must be reachable from the outside. Thomas From tim.peters at gmail.com Sat Mar 11 21:17:32 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sat, 11 Mar 2006 15:17:32 -0500 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: References: <441217BB.1000801@v.loewis.de> Message-ID: <1f7befae0603111217x28bd0afdrf7e1a9fecb1c938d@mail.gmail.com> [Thomas Heller] > Are there any estimates how much network traffic a buildbot would generate? It should be trivial except for the initial checkout of the Python code base. > And how must it be connected to the internet - I assume it must be reachable > from the outside. The slave opens a socket connection to the master, so you must be able to reach the python.org box _from_ the slave. I don't expect you'll have serious problems if you can do that much. For example, my home box has a dynamic IP (assigned by my ISP to my router), my router gives a different dynamic IP to my box (NAT'ed), and I'm running behind both software and hardware (SPI) firewalls. Because my box initiated the connection, nothing in the chain objects. From thomas at python.org Sat Mar 11 23:16:08 2006 From: thomas at python.org (Thomas Wouters) Date: Sat, 11 Mar 2006 23:16:08 +0100 Subject: [Python-Dev] Coverity Open Source Defect Scan of Python In-Reply-To: <440D4B19.8010605@coverity.com> References: <440D4B19.8010605@coverity.com> Message-ID: <9e804ac0603111416p78ed9048n3aad598959186cf5@mail.gmail.com> On 3/7/06, Ben Chelf wrote: > Putting on my idealistic hat and remembering back my grad school days, I > think we're on to something very new in the world of source code > analysis. I really just want every developer to use source code analysis > while they write code (remember, idealistic :)). We got a lot of the > good publicity in the research lab because there existed this big open > source OS that we could test our theories on. So from that angle, I > think it makes sense for Coverity to have a strong relationship with the > open source community since that community has been helping us pretty > much since day 1. This project is just the next step in that...it's > certainly not the last. > > There's plenty more to do to target every developer. Well, as long as we're talking idealistically, I wonder how easy it would be to add reference-counting tracking to Coverity Prevent. Python, Perl and (I believe) PHP all have their own kind of refcounting, but the base semantics are pretty much the same: a function can return a new or a borrowed reference, and it can borrow or steal references passed to it. Without having seen how Prevent works, it feels to me like it would be a small addition to keep track of these application-specific details. Or, perhaps more generic, add a few markers to keep track of them; in Python, you'd only have to mark Py_INCREF and Py_DECREF, and possibly manual fidgeting with an objects' refcount (which is hopefully extremely rare.) I say 'idealistically', though, because I don't know how much business sense it makes to cater to refcounting mechanisms. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060311/398bdff5/attachment.htm From g.brandl at gmx.net Sun Mar 12 00:16:38 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 12 Mar 2006 00:16:38 +0100 Subject: [Python-Dev] Google ads on python.org? Message-ID: Okay, if they were sensible, but: http://www.ph.tum.de/~gbrandl/python-vb.png Not that we want them to use Python... Georg From thomas at python.org Sun Mar 12 00:49:55 2006 From: thomas at python.org (Thomas Wouters) Date: Sun, 12 Mar 2006 00:49:55 +0100 Subject: [Python-Dev] Developing/patching ctypes In-Reply-To: <4411F512.7020701@v.loewis.de> References: <4f0b69dc0603081535q5e2ef69ft99e339a920dbc25e@mail.gmail.com> <44107267.9090806@python.net> <4f0b69dc0603091623jefdb8fhf4b13695bbb491dc@mail.gmail.com> <44112F50.3020506@python.net> <9e804ac0603100050g6aeeabebi96392e93b4c2eea1@mail.gmail.com> <4411602B.5050809@python.net> <1f7befae0603100759u2de80260u9c351fc472cf63e2@mail.gmail.com> <4411F512.7020701@v.loewis.de> Message-ID: <9e804ac0603111549q3508661dua3c033136ad247b9@mail.gmail.com> On 3/10/06, "Martin v. L?wis" wrote: > > Thomas Heller wrote: > > BTW: The buildbot reports ctypes test failures on the gentoo amd64 > machine: > > > > > http://www.python.org/dev/buildbot/trunk/amd64%20gentoo%20trunk/builds/277/step-test/0 > > > > Is there a way to get the actual failures somehow? > > They are now in > > > http://www.python.org/dev/buildbot/all/amd64%20gentoo%20trunk/builds/280/step-test/0 I suspect that some of those failures probably aren't ctypes failures, but re-running-ctype-tests-in-the-same-process failures (like the very first one) -- I get more errors when running -R:: than I do when running the test verbosely directly. Maybe re-running the test could be done in a freshly spawned Python? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060312/1af5014f/attachment.htm From tim at pollenation.net Sun Mar 12 01:22:32 2006 From: tim at pollenation.net (Tim Parkin) Date: Sun, 12 Mar 2006 00:22:32 +0000 Subject: [Python-Dev] Google ads on python.org? In-Reply-To: References: Message-ID: <441369C8.1020103@pollenation.net> Georg Brandl wrote: >Okay, if they were sensible, but: > >http://www.ph.tum.de/~gbrandl/python-vb.png > >Not that we want them to use Python... > >Georg > > > > It's a beta that Google have asked Python to roadtest. I'm hoping that the relevance level will increase soon though ... Tim Parkin From martin at v.loewis.de Sun Mar 12 01:24:56 2006 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 12 Mar 2006 01:24:56 +0100 Subject: [Python-Dev] Developing/patching ctypes In-Reply-To: <9e804ac0603111549q3508661dua3c033136ad247b9@mail.gmail.com> References: <4f0b69dc0603081535q5e2ef69ft99e339a920dbc25e@mail.gmail.com> <44107267.9090806@python.net> <4f0b69dc0603091623jefdb8fhf4b13695bbb491dc@mail.gmail.com> <44112F50.3020506@python.net> <9e804ac0603100050g6aeeabebi96392e93b4c2eea1@mail.gmail.com> <4411602B.5050809@python.net> <1f7befae0603100759u2de80260u9c351fc472cf63e2@mail.gmail.com> <4411F512.7020701@v.loewis.de> <9e804ac0603111549q3508661dua3c033136ad247b9@mail.gmail.com> Message-ID: <44136A58.1010000@v.loewis.de> Thomas Wouters wrote: > I suspect that some of those failures probably aren't ctypes failures, > but re-running-ctype-tests-in-the-same-process failures (like the very > first one) -- I get more errors when running -R:: than I do when running > the test verbosely directly. Maybe re-running the test could be done in > a freshly spawned Python? Feel free to implement that; buildbot will immediately pick it up. Getting the command line right might be a challenge though (plus finding the right Python interpreter, etc.) Regards, Martin From martin at v.loewis.de Sun Mar 12 01:44:45 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 12 Mar 2006 01:44:45 +0100 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: References: <441217BB.1000801@v.loewis.de> Message-ID: <44136EFD.1000603@v.loewis.de> Thomas Heller wrote: > Are there any estimates how much network traffic a buildbot would generate? It will need to download the entire source code twice (once for the trunk, and once for 2.4). After that, it currently does only svn up, on each commit. > And how must it be connected to the internet - I assume it must be reachable > from the outside. No: it opens a connection itself, and then keeps that connection permanently open. Regards, Martin From alan.mcintyre at gmail.com Sun Mar 12 01:46:39 2006 From: alan.mcintyre at gmail.com (Alan McIntyre) Date: Sat, 11 Mar 2006 19:46:39 -0500 Subject: [Python-Dev] libbzip2 version? Message-ID: <44136F6F.1040104@gmail.com> The current PCBuild/readme.txt directs people to http://sources.redhat.com/bzip2 to get the 1.0.2 version of libbz2. The RedHat link redirects to http://www.bzip.org, which shows that 1.0.3 was released in February 2005. I suggest that Python 2.5 should move up to 1.0.3, and that readme.txt should point to bzip.org instead of redhat.com. I'll make & submit a patch if that would help. Here's the changelog entry for 1.0.3: 1.0.3 (15 Feb 05) ~~~~~~~~~~~~~~~~~ Fixes some minor bugs since the last version, 1.0.2. * Further robustification against corrupted compressed data. There are currently no known bitstreams which can cause the decompressor to crash, loop or access memory which does not belong to it. If you are using bzip2 or the library to decompress bitstreams from untrusted sources, an upgrade to 1.0.3 is recommended. * The documentation has been converted to XML, from which html and pdf can be derived. * Various minor bugs in the documentation have been fixed. * Fixes for various compilation warnings with newer versions of gcc, and on 64-bit platforms. * The BZ_NO_STDIO cpp symbol was not properly observed in 1.0.2. This has been fixed. From martin at v.loewis.de Sun Mar 12 01:49:08 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 12 Mar 2006 01:49:08 +0100 Subject: [Python-Dev] libbzip2 version? In-Reply-To: <44136F6F.1040104@gmail.com> References: <44136F6F.1040104@gmail.com> Message-ID: <44137004.2080105@v.loewis.de> Alan McIntyre wrote: > The current PCBuild/readme.txt directs people to > http://sources.redhat.com/bzip2 to get the 1.0.2 version of libbz2. The > RedHat link redirects to http://www.bzip.org, which shows that 1.0.3 was > released in February 2005. I suggest that Python 2.5 should move up to > 1.0.3, and that readme.txt should point to bzip.org instead of > redhat.com. I'll make & submit a patch if that would help. There is no need to submit a patch. I had planned to update it myself for some time now. I also want to update versions of various other libraries (Tcl and bsddb in particular). On bzip2, I wonder whether 2.4 should also update to the newer library; on bsddb, I wonder what version to update to. Regards, Martin From rasky at develer.com Sun Mar 12 02:00:25 2006 From: rasky at develer.com (Giovanni Bajo) Date: Sun, 12 Mar 2006 02:00:25 +0100 Subject: [Python-Dev] libbzip2 version? References: <44136F6F.1040104@gmail.com> <44137004.2080105@v.loewis.de> Message-ID: <14ed01c64570$58fac140$c8b02997@bagio> Martin v. L?wis wrote: > On bzip2, I wonder whether > 2.4 should also update to the newer library; +1, I seem to remember of exploits with corrupted data fed to the bz2 decompressor. Giovanni Bajo From tim.peters at gmail.com Sun Mar 12 03:49:24 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sat, 11 Mar 2006 21:49:24 -0500 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <20060312002336.GA3935@activestate.com> References: <441217BB.1000801@v.loewis.de> <20060312002336.GA3935@activestate.com> Message-ID: <1f7befae0603111849n67c40694u7eebf38410b8788e@mail.gmail.com> [Trent Mick] > I'm keen. However, it looks like Tim is most of the way there already: The first 100% clean (ignoring _ctypes warnings) Windows builbot test run just finished. Yippee! Setup is hellish, although you've already done the worst of it if you regularly build+test full Python on Windows from a checkout ("full" == everything we ship on Windows, including bsddb, ssl, Tcl/Tk and bz2). The second-worst part was figuring out which parts of various software docs could be ignored. I recorded all that remained here: http://wiki.python.org/moin/BuilbotOnWindows Reading that should save you several months ;-) > "x86 XP-2 trunk". I'd still like to give it a go. The machine I'd use > (initially at least) would be Win2k -- so not just a dupe of Tim's WinXP > box. That would be great. A dupe of WinXP would also be great: I'm not going to keep my buildbot slave up all the time, or anywhere near all the time. From unknown_kev_cat at hotmail.com Sun Mar 12 05:50:06 2006 From: unknown_kev_cat at hotmail.com (Joe Smith) Date: Sat, 11 Mar 2006 23:50:06 -0500 Subject: [Python-Dev] conditional expressions - add parens? References: <440E2614.8050508@canterbury.ac.nz> Message-ID: "Greg Ewing" wrote in message news:440E2614.8050508 at canterbury.ac.nz... > Jeremy Hylton wrote: >> Perhaps the solution >> is to require parens around all expressions, a simple consistent rule. > > I actually designed a language with that feature once. > It was an exercise in minimality, with hardly anything > built-in -- all the arithmetic operators, etc. were > defined in the language. > > A result was that there was no built-in notion of > precedence, and my solution was to require parentheses > around every infix operation. So instead of > > dsq = b * b - 4 * a * c > > you would have had to write > > dsq = ((b * b) - ((4 * a) * c)) > > I never got an implementation working well enough > to find out how much of a disaster this would > have been to use, though. :-) Well the original was almost certainly a tongue-in-cheek reference to LISP. LISP was a disaster to use, so I doubt your language would have been any worse. The way one identifies a lisp programmer is to find the person whose paren keys have worn competely off their keyboard. :D From selliott4 at austin.rr.com Sun Mar 12 06:41:29 2006 From: selliott4 at austin.rr.com (Steven Elliott) Date: Sat, 11 Mar 2006 23:41:29 -0600 Subject: [Python-Dev] Making builtins more efficient In-Reply-To: <4410BE69.3080004@canterbury.ac.nz> References: <1141879691.11091.78.camel@grey> <440FF9CB.5030407@gmail.com> <79990c6b0603090400h25dd2c7ev3d5c379f6529f3c2@mail.gmail.com> <1141915806.11091.127.camel@grey> <4410BE69.3080004@canterbury.ac.nz> Message-ID: <1142142089.11091.197.camel@grey> On Fri, 2006-03-10 at 12:46 +1300, Greg Ewing wrote: > Steven Elliott wrote: > > One way of handling it is to > > alter STORE_ATTR (op code for assigning to mod.str) to always check to > > see if the key being assigned is one of the default builtins. If it is, > > then the module's indexed array of builtins is assigned to. > > As long as you're going to all that trouble, it > doesn't seem like it would be much harder to treat > all global names that way, instead of just a predefined > set. The compiler already knows all of the names that > are used as globals in the module's code. The important difference between builtins and globals is that with builtins both the compiler and the runtime can enumerate all references to builtins in a single consistent way. That is "True" can always be builtin #3 and "len" can always be builtin #17, or whatever. This isn't true of globals in that a pyc file referencing a global in a module may have been compiled with a different version of that module (that is "some_module.some_global" can't compiled to single fixed index since stuff may shift around in "some_module"). With globals you have the same kind of problem that you have with operating systems that use ordinals to refer to symbols in shared libraries. So in the case of a static reference to a builtin ("while True", or whatever) the compiler would generate something that refers to it with that builtin's index (such as a new "BUILTIN_OP" opcode, as Philip suggested). Ordinary globals (non-builtins) would continue to be generated as the same code (the "LOAD_GLOBAL" opcode (I'll only refer to the loading opcodes in this email)). In the case of a dynamic reference to a builtin ("eval('True = 7')" or "from foo import *" or whatever) would generate the opcode that indicates that the runtime needs to figure out what do to (the same "LOAD_NAME" opcode). The second part of the the "LOAD_NAME" opcode is similar to the current "LOAD_GLOBAL" opcode - it first checks the hash tables of globals and then checks the hash table of builtins. However, the second part of the "LOAD_NAME" opcode could be implemented such that it first checks against a list of default builtins (which could be a hash table that returns the index of that builtin) and then indexes into the array of builtins if it is found, or retrieves it from the single hash table of globals otherwise. So the "LOAD_NAME" opcode (or similar attempts to dynamically get a name) would almost be as efficient as it currently it. > > That's great, but I'm curious if additional gains can be > > made be focusing just on builtins. > > As long as builtins can be shadowed, I can't see how > to make any extra use of the fact that it's a builtin. > A semantic change would be needed, such as forbidding > shadowing of builtins, or at least forbidding this > from outside the module. One way of looking at is rather than having a clear distinction between builtins and globals (as there currently is) there would be a single global name space that internally in Python is implemented in two data structures. An array for frequently used names and a hash table for infrequently used names. And the division between the two wouldn't even have two be between globals and builtins like we've been talking about so far. What distinguishes the builtins is you get them for free (initialized on startup). So, it would be possible to insert infrequently used builtins into the hash table of infrequently used names only when the module refers to it. Conversely, names that aren't builtins, but that are used frequently in many different modules, such as "sys" or "os", could have indexes set aside for for them in the array of frequently used names. Later, when when it gets a value (because "sys" is imported, or whatever) it just gets stuck into the predetermined slot in the array of frequently used names. Since builtins can be shadowed, as you point out, there would have to be one array of frequently used names per module. But often it would be the same as other modules. So internally, as a matter of efficiency, the interpreter could use a copy on write strategy where a global array of frequently used names is used by the module until it assigns to "True", or something like that, at which point it gets its own copy. -- ----------------------------------------------------------------------- | Steven Elliott | selliott4 at austin.rr.com | ----------------------------------------------------------------------- From g.brandl at gmx.net Sat Mar 11 14:12:08 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 11 Mar 2006 14:12:08 +0100 Subject: [Python-Dev] libbzip2 version? In-Reply-To: <44137004.2080105@v.loewis.de> References: <44136F6F.1040104@gmail.com> <44137004.2080105@v.loewis.de> Message-ID: Martin v. L?wis wrote: > Alan McIntyre wrote: >> The current PCBuild/readme.txt directs people to >> http://sources.redhat.com/bzip2 to get the 1.0.2 version of libbz2. The >> RedHat link redirects to http://www.bzip.org, which shows that 1.0.3 was >> released in February 2005. I suggest that Python 2.5 should move up to >> 1.0.3, and that readme.txt should point to bzip.org instead of >> redhat.com. I'll make & submit a patch if that would help. > > There is no need to submit a patch. I had planned to update it myself > for some time now. I also want to update versions of various other > libraries (Tcl and bsddb in particular). There seems to be a new version of expat too: http://python.org/sf/1433435 The namespace problem with pyexpat persists: http://python.org/sf/1295808 Georg From martin at v.loewis.de Sun Mar 12 10:55:56 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 12 Mar 2006 10:55:56 +0100 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <1f7befae0603111849n67c40694u7eebf38410b8788e@mail.gmail.com> References: <441217BB.1000801@v.loewis.de> <20060312002336.GA3935@activestate.com> <1f7befae0603111849n67c40694u7eebf38410b8788e@mail.gmail.com> Message-ID: <4413F02C.4040307@v.loewis.de> Tim Peters wrote: > That would be great. A dupe of WinXP would also be great: I'm not > going to keep my buildbot slave up all the time, or anywhere near all > the time. It seems like the buildbot needs even more hand-holding on Windows: it apparently doesn't survive a master stop/start cycle. While the Unix buildbots reconnect, the Windows one doesn't. Regards, Martin From theller at python.net Sun Mar 12 12:13:07 2006 From: theller at python.net (Thomas Heller) Date: Sun, 12 Mar 2006 12:13:07 +0100 Subject: [Python-Dev] Developing/patching ctypes In-Reply-To: <9e804ac0603111549q3508661dua3c033136ad247b9@mail.gmail.com> References: <4f0b69dc0603081535q5e2ef69ft99e339a920dbc25e@mail.gmail.com> <44107267.9090806@python.net> <4f0b69dc0603091623jefdb8fhf4b13695bbb491dc@mail.gmail.com> <44112F50.3020506@python.net> <9e804ac0603100050g6aeeabebi96392e93b4c2eea1@mail.gmail.com> <4411602B.5050809@python.net> <1f7befae0603100759u2de80260u9c351fc472cf63e2@mail.gmail.com> <4411F512.7020701@v.loewis.de> <9e804ac0603111549q3508661dua3c033136ad247b9@mail.gmail.com> Message-ID: Thomas Wouters wrote: > On 3/10/06, "Martin v. L?wis" wrote: >> Thomas Heller wrote: >>> BTW: The buildbot reports ctypes test failures on the gentoo amd64 >> machine: >>> >> http://www.python.org/dev/buildbot/trunk/amd64%20gentoo%20trunk/builds/277/step-test/0 >>> Is there a way to get the actual failures somehow? >> They are now in >> >> >> http://www.python.org/dev/buildbot/all/amd64%20gentoo%20trunk/builds/280/step-test/0 > > > I suspect that some of those failures probably aren't ctypes failures, but > re-running-ctype-tests-in-the-same-process failures (like the very first > one) -- I get more errors when running -R:: than I do when running the test > verbosely directly. I can reproduce that now - thanks for finding the problem. IMO, some of the ctypes tests are kind-of broken, since they keep state although they should not. > Maybe re-running the test could be done in a freshly > spawned Python? I'll try to fix the ctypes-tests starting on Monday, so I assume that would not be necessary. Thomas From g.brandl at gmx.net Sun Mar 12 14:38:19 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 12 Mar 2006 14:38:19 +0100 Subject: [Python-Dev] decorator module patch Message-ID: Hi, to underlay my proposals with facts, I've written a simple decorator module containing at the moment only the "decorator" decorator. http://python.org/sf/1448297 It is implemented as a C extension module _decorator which contains the decorator object (modelled after the functional.partial object) and a Lib/decorator.py to allow further decorators added as Python code. Comes with docs and unit test. Please review! Cheers, Georg From ncoghlan at gmail.com Sun Mar 12 15:10:58 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 13 Mar 2006 00:10:58 +1000 Subject: [Python-Dev] decorator module patch In-Reply-To: References: Message-ID: <44142BF2.1040301@gmail.com> Georg Brandl wrote: > Hi, > > to underlay my proposals with facts, I've written a simple decorator > module containing at the moment only the "decorator" decorator. > > http://python.org/sf/1448297 > > It is implemented as a C extension module _decorator which contains the > decorator object (modelled after the functional.partial object) and a > Lib/decorator.py to allow further decorators added as Python code. > > Comes with docs and unit test. Given that @decorator is a definition time only operation to modify a function's __name__, __doc__ and __dict__ attributes, and doesn't actually introduce any extra levels of run-time nesting to function calls, I'm not clear on why you bothered with a hybrid implementation instead of sticking with pure Python. (To clarify what I mean: using the example in the doc patch, the extra layer of run-time nesting from @decorator's wrapper function applies only to the @logged decorator, not to the function 'print_nested'. If an application has a decorated function definition in a performance critical path, a little bit of extra overhead from @decorator is the least of its worries.) Also, I thought we were trying to move away from modules that shared a name with one of their public functions or classes. As it is, I'm not even sure that a name like "decorator" gives the right emphasis. In general, decorators belong in the appropriate domain-specific module (similar to context managers). In this case, though, the domain is the manipulation of Python functions - maybe the module should be called "metafunctions" or "functools" to reflect its application domain, rather than the coincidental fact that its first member happens to be a decorator. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Sun Mar 12 15:25:34 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 13 Mar 2006 00:25:34 +1000 Subject: [Python-Dev] decorator module patch In-Reply-To: <44142BF2.1040301@gmail.com> References: <44142BF2.1040301@gmail.com> Message-ID: <44142F5E.60502@gmail.com> Nick Coghlan wrote: > Georg Brandl wrote: >> Hi, >> >> to underlay my proposals with facts, I've written a simple decorator >> module containing at the moment only the "decorator" decorator. Sorry, I forgot the initial comment which was meant to be "Thanks for moving this proposal forward" :) It's currently all too easy to write decorators that don't play nicely with introspection. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From g.brandl at gmx.net Sun Mar 12 15:36:57 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 12 Mar 2006 15:36:57 +0100 Subject: [Python-Dev] decorator module patch In-Reply-To: <44142BF2.1040301@gmail.com> References: <44142BF2.1040301@gmail.com> Message-ID: Nick Coghlan wrote: > Georg Brandl wrote: >> Hi, >> >> to underlay my proposals with facts, I've written a simple decorator >> module containing at the moment only the "decorator" decorator. >> >> http://python.org/sf/1448297 >> >> It is implemented as a C extension module _decorator which contains the >> decorator object (modelled after the functional.partial object) and a >> Lib/decorator.py to allow further decorators added as Python code. >> >> Comes with docs and unit test. > > Given that @decorator is a definition time only operation to modify a > function's __name__, __doc__ and __dict__ attributes, and doesn't actually > introduce any extra levels of run-time nesting to function calls, I'm not > clear on why you bothered with a hybrid implementation instead of sticking > with pure Python. Good question... partly because I wanted to make myself more intimate with the C API and extension modules. Everyone can write that in Python... > (To clarify what I mean: using the example in the doc patch, the extra layer > of run-time nesting from @decorator's wrapper function applies only to the > @logged decorator, not to the function 'print_nested'. If an application has a > decorated function definition in a performance critical path, a little bit of > extra overhead from @decorator is the least of its worries.) Right. I'm not opposed to a Python-only module, I've had my fun :) > Also, I thought we were trying to move away from modules that shared a name > with one of their public functions or classes. As it is, I'm not even sure > that a name like "decorator" gives the right emphasis. I thought about "decorators" too, that would make "decorators.decorator". Hm. > In general, decorators belong in the appropriate domain-specific module > (similar to context managers). In this case, though, the domain is the > manipulation of Python functions - maybe the module should be called > "metafunctions" or "functools" to reflect its application domain, rather than > the coincidental fact that its first member happens to be a decorator. Depends on what else will end up there. If it's "memoize" or "deprecated" then the name "functools" doesn't sound too good either. Georg From arigo at tunes.org Sun Mar 12 16:26:13 2006 From: arigo at tunes.org (Armin Rigo) Date: Sun, 12 Mar 2006 16:26:13 +0100 Subject: [Python-Dev] str.count is slow In-Reply-To: References: <1141073696.107136.318600@j33g2000cwa.googlegroups.com> <1141083127.970403.147100@v46g2000cwv.googlegroups.com> Message-ID: <20060312152613.GA16848@code0.codespeak.net> Hi Ben, On Mon, Feb 27, 2006 at 06:50:28PM -0500, Ben Cartwright wrote: > > It seems to me that str.count is awfully slow. Is there some reason > > for this? stringobject.c could do with a good clean-up. It contains very similar algorithms multiple times, in slightly different styles and with different performance characteristics. If I find some motivation I'll try to come up with a patch. A bientot, Armin From arigo at tunes.org Sun Mar 12 16:32:36 2006 From: arigo at tunes.org (Armin Rigo) Date: Sun, 12 Mar 2006 16:32:36 +0100 Subject: [Python-Dev] Making staticmethod objects callable? In-Reply-To: References: Message-ID: <20060312153236.GB16848@code0.codespeak.net> Hi Nicolas, On Thu, Mar 02, 2006 at 01:55:03AM -0500, Nicolas Fleury wrote: > (...) A use case is not hard to > imagine, especially a private static method called only to build a class > attribute. Uh. I do this all the time, and the answer is simply: don't make that a staticmethod. Staticmethods are for the rare case where you need dynamic class-based dispatch but don't have an instance around. class A: def _myinitializer(): do strange stuff here _myinitializer() del _myinitializer # optional A bientot, Armin From thomas at python.org Sun Mar 12 17:51:34 2006 From: thomas at python.org (Thomas Wouters) Date: Sun, 12 Mar 2006 17:51:34 +0100 Subject: [Python-Dev] Making staticmethod objects callable? In-Reply-To: <20060312153236.GB16848@code0.codespeak.net> References: <20060312153236.GB16848@code0.codespeak.net> Message-ID: <9e804ac0603120851h27f0e632i17ba870654f37161@mail.gmail.com> On 3/12/06, Armin Rigo wrote: > > Staticmethods are for the rare case where you need > dynamic class-based dispatch but don't have an instance around. Actually, I would argue that's what classmethods are for, not staticmethods. You may not envision a desire for having the class in the method right now, but it won't hurt, either. The only real use-case for staticmethods that I know of is one Jp Caldrone pointed out once: storing arbitrary callables as class attributes: class MailHandleThingy(object): sendmail = mymaillib.mailsend ... Without wrapping that in a staticmethod, 'sendmail' may or may not become a bound method -- and that might change without touching any of the MailHandleThingy code. All cases where the callable is under your direct control, it's more rewarding (same buck, way more bang) to use classmethods, IMHO. (And no, calling a function during class-definition isn't a usecase for staticmethods :) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060312/f2b1509d/attachment.htm From martin at v.loewis.de Sun Mar 12 18:24:29 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 12 Mar 2006 18:24:29 +0100 Subject: [Python-Dev] PythonCore\CurrentVersion In-Reply-To: <1f7befae0510101542tf65c250ybf93eb7f11c187ae@mail.gmail.com> References: <4347A020.2050008@v.loewis.de> <1f7befae0510101542tf65c250ybf93eb7f11c187ae@mail.gmail.com> Message-ID: <4414594D.3090906@v.loewis.de> Tim Peters wrote: >>I believe I documented it many moons ago. I don't think CurrentVersion was >>ever implemented (or possibly was for a very short time before being >>removed). The "registered modules" concept was misguided and AFAIK is not >>used by anyone - IMO it should be deprecated (if not just removed!). >>Further, I believe the documentation in the file for PYTHONPATH is, as said >>in those docs, out of date, but that the comments in getpathp.c are correct. > > > It would be good to update that web page ;-) I have since done that: http://www.python.org/windows/registry.html Regards, Martin From tim.peters at gmail.com Sun Mar 12 18:28:22 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 12 Mar 2006 12:28:22 -0500 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <4413F02C.4040307@v.loewis.de> References: <441217BB.1000801@v.loewis.de> <20060312002336.GA3935@activestate.com> <1f7befae0603111849n67c40694u7eebf38410b8788e@mail.gmail.com> <4413F02C.4040307@v.loewis.de> Message-ID: <1f7befae0603120928p5e61a15eg8f9ec66c1d4ad970@mail.gmail.com> [Martin v. L?wis] > It seems like the buildbot needs even more hand-holding on Windows: > it apparently doesn't survive a master stop/start cycle. While the > Unix buildbots reconnect, the Windows one doesn't. What makes us believe that? My box was hibernating from 03:12 to 11:54 EST today, and couldn't have responded to anything during that period. When I woke the box up, I was pleased to see that Twisted figured out that it hadn't heard from the master in 8+ hours, and established a new connection, all by itself. OTOH, I see the new connection soon got wedged all by itself too ;-), while running trunk tests just now. From martin at v.loewis.de Sun Mar 12 18:38:13 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 12 Mar 2006 18:38:13 +0100 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <1f7befae0603120928p5e61a15eg8f9ec66c1d4ad970@mail.gmail.com> References: <441217BB.1000801@v.loewis.de> <20060312002336.GA3935@activestate.com> <1f7befae0603111849n67c40694u7eebf38410b8788e@mail.gmail.com> <4413F02C.4040307@v.loewis.de> <1f7befae0603120928p5e61a15eg8f9ec66c1d4ad970@mail.gmail.com> Message-ID: <44145C85.6010306@v.loewis.de> Tim Peters wrote: >>It seems like the buildbot needs even more hand-holding on Windows: >>it apparently doesn't survive a master stop/start cycle. While the >>Unix buildbots reconnect, the Windows one doesn't. > > > What makes us believe that? The slave was reported as "idle" just before I restarted the master, and "offline" after the restart. From that, I inferred that the restart broke the connection. > My box was hibernating from 03:12 to > 11:54 EST today, and couldn't have responded to anything during that > period. I see. So it is rather that the master doesn't see the slaves go away. Regards, Martin From ianb at colorstudy.com Sun Mar 12 20:16:04 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Sun, 12 Mar 2006 13:16:04 -0600 Subject: [Python-Dev] decorator module patch In-Reply-To: References: <44142BF2.1040301@gmail.com> Message-ID: <44147374.8080803@colorstudy.com> Georg Brandl wrote: >> Also, I thought we were trying to move away from modules that shared a name >> with one of their public functions or classes. As it is, I'm not even sure >> that a name like "decorator" gives the right emphasis. > > I thought about "decorators" too, that would make "decorators.decorator". Hm. I personally like pluralized modules for exactly the reason that they don't clash as much with members or likely local variables. datetime.datetime frequently leads me to make mistakes. >> In general, decorators belong in the appropriate domain-specific module >> (similar to context managers). In this case, though, the domain is the >> manipulation of Python functions - maybe the module should be called >> "metafunctions" or "functools" to reflect its application domain, rather than >> the coincidental fact that its first member happens to be a decorator. > > Depends on what else will end up there. If it's "memoize" or "deprecated" then > the name "functools" doesn't sound too good either. memoize seems to fit into functools fairly well, though deprecated not so much. functools is similarly named to itertools, another module that is kind of vague in scope (though functools is much more vague). partial would make just as much sense in functools as in functional. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From guido at python.org Sun Mar 12 21:24:19 2006 From: guido at python.org (Guido van Rossum) Date: Sun, 12 Mar 2006 12:24:19 -0800 Subject: [Python-Dev] conditional expressions - add parens? In-Reply-To: References: <440E2614.8050508@canterbury.ac.nz> Message-ID: On 3/11/06, Joe Smith wrote: > Well the original was almost certainly a tongue-in-cheek reference to LISP. > LISP was a disaster to use, so I doubt your language would have been any > worse. > The way one identifies a lisp programmer is to find the person whose paren > keys have > worn competely off their keyboard. :D Given how much Python is indebted to Lisp, digs at Lisp seem particularly off-topic for this list. Fully expecting this to bounce, -- --Guido van Rossum (home page: http://www.python.org/~guido/) From aleaxit at gmail.com Sun Mar 12 22:03:07 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Sun, 12 Mar 2006 13:03:07 -0800 Subject: [Python-Dev] decorator module patch In-Reply-To: <44147374.8080803@colorstudy.com> References: <44142BF2.1040301@gmail.com> <44147374.8080803@colorstudy.com> Message-ID: On Mar 12, 2006, at 11:16 AM, Ian Bicking wrote: ... > memoize seems to fit into functools fairly well, though deprecated not > so much. functools is similarly named to itertools, another module > that > is kind of vague in scope (though functools is much more vague). > partial would make just as much sense in functools as in functional. Couldn't we merge functools and functional into just one (user- visible) module? The distinction between what goes into one vs the other is exceedingly subtle and poor users will be guessing as to what's where. If we need a mixed module with something in C and something in Python, we can do it the usual way, func.py wrapping _func.pyd (or .so or whatever)... Alex From raymond.hettinger at verizon.net Sun Mar 12 22:14:42 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sun, 12 Mar 2006 16:14:42 -0500 Subject: [Python-Dev] decorator module patch References: <44142BF2.1040301@gmail.com> <44147374.8080803@colorstudy.com> Message-ID: <005e01c64619$fba83770$6a01a8c0@RaymondLaptop1> [Ian Bicking] >> memoize seems to fit into functools fairly well, though deprecated not >> so much. functools is similarly named to itertools, another module >> that >> is kind of vague in scope (though functools is much more vague). >> partial would make just as much sense in functools as in functional. [Alex] > Couldn't we merge functools and functional into just one (user- > visible) module? The distinction between what goes into one vs the > other is exceedingly subtle and poor users will be guessing as to > what's where. If we need a mixed module with something in C and > something in Python, we can do it the usual way, func.py wrapping > _func.pyd (or .so or whatever)... +1 on putting the tools all in one module. With respect to decorator entries, I would like to see python-dev collectively decide to show restraint. There are so many ways to write and use decorators that best-of-the-best are not yet obvious. Hopefully, collections of decorators will be allowed to grow-in-the-wild as recipes and as third-party modules before being put into the core. Georg's proposal seems like a good candidate for a first entry -- its chief virtue being that it may help people avoid writing crummy decorators. If his goes in, hopefully it will not fall down a slippery slope and trigger an avalance of immature decorators being added to the core. my-two-cents, Raymond From ncoghlan at gmail.com Sun Mar 12 22:29:49 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 13 Mar 2006 07:29:49 +1000 Subject: [Python-Dev] decorator module patch In-Reply-To: References: <44142BF2.1040301@gmail.com> <44147374.8080803@colorstudy.com> Message-ID: <441492CD.8000802@gmail.com> Alex Martelli wrote: > On Mar 12, 2006, at 11:16 AM, Ian Bicking wrote: > ... >> memoize seems to fit into functools fairly well, though deprecated not >> so much. functools is similarly named to itertools, another module >> that >> is kind of vague in scope (though functools is much more vague). >> partial would make just as much sense in functools as in functional. > > Couldn't we merge functools and functional into just one (user- > visible) module? The distinction between what goes into one vs the > other is exceedingly subtle and poor users will be guessing as to > what's where. If we need a mixed module with something in C and > something in Python, we can do it the usual way, func.py wrapping > _func.pyd (or .so or whatever)... I agree it makes sense to have "decorator", "memoize", "deprecated" and "partial" all being members of the same module, whether the name be "functools" or "functional" (although I have a slight preference for "functools" due to the parallel with "itertools"). On the question of whether or not deprecated fits in as a function tool, I know I'd tend to only use it on functions (to deprecate a class, I'd simply decorate the class's __init__ or __new__ method). Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From raymond.hettinger at verizon.net Sun Mar 12 23:03:58 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sun, 12 Mar 2006 17:03:58 -0500 Subject: [Python-Dev] decorator module patch References: <44142BF2.1040301@gmail.com> <44147374.8080803@colorstudy.com> <441492CD.8000802@gmail.com> Message-ID: <007f01c64620$dd3c4450$6a01a8c0@RaymondLaptop1> [Nick Coghlan] > I agree it makes sense to have "decorator", "memoize", "deprecated" and > "partial" all being members of the same module, whether the name be > "functools" or "functional" (although I have a slight preference for > "functools" due to the parallel with "itertools"). I like "functools" for a different reason -- the name is sufficiently broad so that we don't have fret about whether a particular tool fits within the module's scope. In contrast, a name like "functional" suggests that some of these tools don't quite fit. Raymond From greg.ewing at canterbury.ac.nz Sun Mar 12 23:00:40 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 13 Mar 2006 11:00:40 +1300 Subject: [Python-Dev] Making builtins more efficient In-Reply-To: References: <1141879691.11091.78.camel@grey> <007c01c64399$af03ffd0$a600000a@RaymondLaptop1> <4410BF9E.1080301@canterbury.ac.nz> <44120D3E.4030904@canterbury.ac.nz> Message-ID: <44149A08.4060502@canterbury.ac.nz> Guido van Rossum wrote: > I don't see why everything that doesn't make sense to be shadowed > ought to become a keyword. That wasn't the reason. I was thinking it would be nice if one could use True and False instead of 1 and 0 in the knowledge that it wasn't costing a couple of dictionary lookups. However, if a more general way is found of optimising global lookups, this may become a non-issue. Greg From greg.ewing at canterbury.ac.nz Sun Mar 12 23:20:16 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 13 Mar 2006 11:20:16 +1300 Subject: [Python-Dev] Making builtins more efficient In-Reply-To: <1142142089.11091.197.camel@grey> References: <1141879691.11091.78.camel@grey> <440FF9CB.5030407@gmail.com> <79990c6b0603090400h25dd2c7ev3d5c379f6529f3c2@mail.gmail.com> <1141915806.11091.127.camel@grey> <4410BE69.3080004@canterbury.ac.nz> <1142142089.11091.197.camel@grey> Message-ID: <44149EA0.1090700@canterbury.ac.nz> Steven Elliott wrote: > a pyc file referencing a global in a module may > have been compiled with a different version of that module (that is > "some_module.some_global" can't compiled to single fixed index since > stuff may shift around in "some_module"). Not sure I quite follow that. Since the code in the .pyc file is what sets the module up in the first place, it can set up the array to suit its own use of global symbols. The only way this could go wrong was if you extracted the code out of a function compiled for one module and surgically implanted it in another, but if you're hacking at that level you deserve what you get. I think it's actually possible to combine all the ideas suggested so far. Builtins are assigned predefined indexes for a particular bytecode version, and each module assigns indexes its own way for other globals it uses. Builtins can have dedicated opcodes which perform a fast check for shadowing in the module and builtin arrays. Everything goes as fast as possible while still allowing anything to be overridden. Greg From trentm at ActiveState.com Sun Mar 12 23:41:48 2006 From: trentm at ActiveState.com (Trent Mick) Date: Sun, 12 Mar 2006 14:41:48 -0800 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <1f7befae0603111849n67c40694u7eebf38410b8788e@mail.gmail.com> References: <441217BB.1000801@v.loewis.de> <20060312002336.GA3935@activestate.com> <1f7befae0603111849n67c40694u7eebf38410b8788e@mail.gmail.com> Message-ID: <20060312224148.GB32648@activestate.com> [Tim Peters wrote] > Setup is hellish, Agreed, though I have everything going with my own testing buildbot master (everything for the trunk build that is). My only remaining problem is that I can't connect to python.org's master. (Following up with Martin.) > The second-worst part was figuring out which parts of various software > docs could be ignored. :) Did you apply the Berkeley DB patches to your db-4.2.52 sources? > I recorded all that remained here: > > http://wiki.python.org/moin/BuilbotOnWindows correction for others: http://wiki.python.org/moin/BuildbotOnWindows > > "x86 XP-2 trunk". I'd still like to give it a go. The machine I'd use > > (initially at least) would be Win2k -- so not just a dupe of Tim's WinXP > > box. > > That would be great. A dupe of WinXP would also be great: I'm not > going to keep my buildbot slave up all the time, or anywhere near all > the time. I'm worried about the load this is going to cause on this machine with a new build for every checkin (granted they are serialized so not for *every* checkin). The full (doubled) test suite takes a *long* time to run on Windows. It looks like it took about 25 minutes on your box. It is taking over 40 minutes on one of my machines here. :( This Windows box needs an enema. Remaining TODOs: - get connection to python.org master working - make sure the python24-maint branch one works - see about the load issue - get the build slaves running as a Windows service - update PCbuild/readme.txt Trent -- Trent Mick TrentM at ActiveState.com From greg.ewing at canterbury.ac.nz Sun Mar 12 23:34:29 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 13 Mar 2006 11:34:29 +1300 Subject: [Python-Dev] conditional expressions - add parens? In-Reply-To: References: <440E2614.8050508@canterbury.ac.nz> Message-ID: <4414A1F5.1000202@canterbury.ac.nz> Joe Smith wrote: > LISP was a disaster to use, so I doubt your language would have been any > worse. At least Lisp would let you say (* 4 a c) and not force you to write (* (* 4 a) c) My language would not have been so forgiving, unless you were willing to define a bunch of different * functions with different numbers of parameters. BTW, I did use Lisp and Scheme fairly heavily for a period, and didn't find them to be disasters -- at least not because of the parens. Greg From g.brandl at gmx.net Sun Mar 12 23:11:31 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 12 Mar 2006 23:11:31 +0100 Subject: [Python-Dev] decorator module patch In-Reply-To: <441492CD.8000802@gmail.com> References: <44142BF2.1040301@gmail.com> <44147374.8080803@colorstudy.com> <441492CD.8000802@gmail.com> Message-ID: Nick Coghlan wrote: > Alex Martelli wrote: >> On Mar 12, 2006, at 11:16 AM, Ian Bicking wrote: >> ... >>> memoize seems to fit into functools fairly well, though deprecated not >>> so much. functools is similarly named to itertools, another module >>> that >>> is kind of vague in scope (though functools is much more vague). >>> partial would make just as much sense in functools as in functional. >> >> Couldn't we merge functools and functional into just one (user- >> visible) module? The distinction between what goes into one vs the >> other is exceedingly subtle and poor users will be guessing as to >> what's where. If we need a mixed module with something in C and >> something in Python, we can do it the usual way, func.py wrapping >> _func.pyd (or .so or whatever)... > > > I agree it makes sense to have "decorator", "memoize", "deprecated" and > "partial" all being members of the same module, whether the name be > "functools" or "functional" (although I have a slight preference for > "functools" due to the parallel with "itertools"). +1 from me. I'll happily make the according changes if that reaches a consensus. > On the question of whether or not deprecated fits in as a function tool, I > know I'd tend to only use it on functions (to deprecate a class, I'd simply > decorate the class's __init__ or __new__ method). I suppose it would be okay, since as a decorator it can only be applied to functions. In PEP 356, there is even a suggestion to "add builtin @deprecated decorator?". Cheers, Georg From raymond.hettinger at verizon.net Mon Mar 13 00:00:48 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sun, 12 Mar 2006 18:00:48 -0500 Subject: [Python-Dev] decorator module patch References: <44142BF2.1040301@gmail.com> <44147374.8080803@colorstudy.com> <441492CD.8000802@gmail.com> Message-ID: <00a401c64628$cdb03070$6a01a8c0@RaymondLaptop1> > In PEP 356, there is even a suggestion to "add builtin @deprecated > decorator?". Restraint please. Go easy on the decorator additions. Raymond From steven.bethard at gmail.com Mon Mar 13 00:12:46 2006 From: steven.bethard at gmail.com (Steven Bethard) Date: Sun, 12 Mar 2006 16:12:46 -0700 Subject: [Python-Dev] decorator module patch In-Reply-To: <007f01c64620$dd3c4450$6a01a8c0@RaymondLaptop1> References: <44142BF2.1040301@gmail.com> <44147374.8080803@colorstudy.com> <441492CD.8000802@gmail.com> <007f01c64620$dd3c4450$6a01a8c0@RaymondLaptop1> Message-ID: On 3/12/06, Raymond Hettinger wrote: > [Nick Coghlan] > > I agree it makes sense to have "decorator", "memoize", "deprecated" and > > "partial" all being members of the same module, whether the name be > > "functools" or "functional" (although I have a slight preference for > > "functools" due to the parallel with "itertools"). > > I like "functools" for a different reason -- the name is sufficiently broad so > that we don't have fret about whether a particular tool fits within the module's > scope. In contrast, a name like "functional" suggests that some of these tools > don't quite fit. FWIW, +1 here. Especially if we're only going to add two functions -- ``partial``, which is already accepted, and Georg's ``decorator`` -- it seems like overkill to introduce a module for each. I agree that "functools" is a better module name if both ``partial`` and ``decorator`` are going in there. STeVe -- Grammar am for people who can't think for myself. --- Bucky Katt, Get Fuzzy From trentm at ActiveState.com Mon Mar 13 00:20:12 2006 From: trentm at ActiveState.com (Trent Mick) Date: Sun, 12 Mar 2006 15:20:12 -0800 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <1f7befae0603111849n67c40694u7eebf38410b8788e@mail.gmail.com> References: <441217BB.1000801@v.loewis.de> <20060312002336.GA3935@activestate.com> <1f7befae0603111849n67c40694u7eebf38410b8788e@mail.gmail.com> Message-ID: <20060312232012.GA11347@activestate.com> [Tim Peters wrote] > Setup is hellish Any objections to: Index: Tools/buildbot/build.bat =================================================================== --- build.bat (revision 42982) +++ build.bat (working copy) @@ -1,3 +1,3 @@ @rem Used by the buildbot "compile" step. call "%VS71COMNTOOLS%vsvars32.bat" -devenv.com /build Debug PCbuild\pcbuild.sln +devenv.com /useenv /build Debug PCbuild\pcbuild.sln Adding the /useenv means that one's PATH actually gets through. This is important for the _ssl.vproj build. It calls build_ssl.py which tries to find a Perl to use. Without "/useenv" Visual Studio is getting a PATH from somewhere else (presumably from its internal environment configuration). The result is that build_ssl.py fallsback to its "well-known" locations for a Perl install. On one mahcine I was trying this on I didn't have Perl installed to C:\Perl but to C:\Perl58. People who install to an alternate drive might also get surprised. Trent -- Trent Mick TrentM at ActiveState.com From tdelaney at avaya.com Mon Mar 13 00:25:21 2006 From: tdelaney at avaya.com (Delaney, Timothy (Tim)) Date: Mon, 13 Mar 2006 10:25:21 +1100 Subject: [Python-Dev] Making builtins more efficient Message-ID: <2773CAC687FD5F4689F526998C7E4E5F07435C@au3010avexu1.global.avaya.com> Steven Elliott wrote: > The important difference between builtins and globals is that with > builtins both the compiler and the runtime can enumerate all > references > to builtins in a single consistent way. That is "True" can always be > builtin #3 and "len" can always be builtin #17, or whatever. __slots__ for modules? Tim Delaney From martin at v.loewis.de Mon Mar 13 00:29:51 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 13 Mar 2006 00:29:51 +0100 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <20060312232012.GA11347@activestate.com> References: <441217BB.1000801@v.loewis.de> <20060312002336.GA3935@activestate.com> <1f7befae0603111849n67c40694u7eebf38410b8788e@mail.gmail.com> <20060312232012.GA11347@activestate.com> Message-ID: <4414AEEF.6020505@v.loewis.de> Trent Mick wrote: > Adding the /useenv means that one's PATH actually gets through. This is > important for the _ssl.vproj build. It calls build_ssl.py which tries to > find a Perl to use. Without "/useenv" Visual Studio is getting a PATH > from somewhere else (presumably from its internal environment > configuration). The result is that build_ssl.py fallsback to its > "well-known" locations for a Perl install. Go ahead. The above makes a good check-in message. Regards, Martin From tim.peters at gmail.com Mon Mar 13 00:48:13 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 12 Mar 2006 18:48:13 -0500 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <20060312224148.GB32648@activestate.com> References: <441217BB.1000801@v.loewis.de> <20060312002336.GA3935@activestate.com> <1f7befae0603111849n67c40694u7eebf38410b8788e@mail.gmail.com> <20060312224148.GB32648@activestate.com> Message-ID: <1f7befae0603121548k29d18135mbcf0e3365f23c93@mail.gmail.com> [Tim] >> Setup is hellish, [Trent] > Agreed, though I have everything going with my own testing buildbot > master (everything for the trunk build that is). My only remaining > problem is that I can't connect to python.org's master. (Following up > with Martin.) Looks like that got fixed. >> The second-worst part was figuring out which parts of various software >> docs could be ignored. > :) > Did you apply the Berkeley DB patches to your db-4.2.52 sources? Ah, _which_ patches? As with my buildbot Wiki page, I write down everything I do if there's a good chance I may need to do it again. So, e.g., these are my words in PCbuild\readme.txt: As of 11-Apr-2004, you also need to download and manually apply two patches before proceeding (and the sleepycat download page tells you about this). Cygwin patch worked for me. cd to dist\db-4.2.52 and use "patch -p0 < patchfile" once for each downloaded patchfile. It's possible that there are more patches needed since then, but if so I wouldn't know about that (last time I built the externals in the _bsddb part was indeed 11-Apr-2004). This touches on something we (including Martin) should think about: it's very painful to build a full Python on Windows because of these external packages, each with its own unique and involved compile dance, and I expect it's a _huge_ barrier for getting Windows buildbot volunteers. You have to do real, messy, tedious work to get beyond this part. Something we might be able to do instead is have just one person per external project endure the pain of building it, and then have them check in the whole post-compilation post-test project directory tree. Everyone else (presumably including Windows buildbot slaves) using the same compiler could then just check out the result. >> I recorded all that remained here: >> >> http://wiki.python.org/moin/BuilbotOnWindows > correction for others: > http://wiki.python.org/moin/BuildbotOnWindows The URL I gave was correct at the time I sent it ;-) You're right that I renamed the page later, and good eye! > I'm worried about the load this is going to cause on this machine with a > new build for every checkin (granted they are serialized so not for > *every* checkin). The full (doubled) test suite takes a *long* time to > run on Windows. It looks like it took about 25 minutes on your box. It > is taking over 40 minutes on one of my machines here. :( This Windows > box needs an enema. Its current second test run is pretty pointless. Would be fine by me if we changed test.bat from call rt.bat -d -uall -rw to call rt.bat -d -q -uall -rw "-q" would cause it to run the tests only once. What's worse is when checkins cause both trunk and branch rebuilds. That can bring my zippy box to its knees, particularly when two of the disk-intensive tests (like test_largefile) run at the same time. > Remaining TODOs: > - get connection to python.org master working > - make sure the python24-maint branch one works Works for me, although (as the Wiki page says) you also need to copy zlib-1.2.3 into the branch build area. > - see about the load issue > - get the build slaves running as a Windows service Mark Hammond has a patch to the buildbot project toward this end: > - update PCbuild/readme.txt Anything in particular need changing there? From g.brandl at gmx.net Mon Mar 13 08:56:34 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 13 Mar 2006 08:56:34 +0100 Subject: [Python-Dev] decorator module patch In-Reply-To: <00a401c64628$cdb03070$6a01a8c0@RaymondLaptop1> References: <44142BF2.1040301@gmail.com> <44147374.8080803@colorstudy.com> <441492CD.8000802@gmail.com> <00a401c64628$cdb03070$6a01a8c0@RaymondLaptop1> Message-ID: Raymond Hettinger wrote: >> In PEP 356, there is even a suggestion to "add builtin @deprecated >> decorator?". > > Restraint please. Well, that sentence wasn't meant in the sense of "we should add it" but in the sense of "why shouldn't we put it in functools _if_ we add it, when it's even suggested as a builtin" ;) Georg From oliphant.travis at ieee.org Mon Mar 13 09:28:56 2006 From: oliphant.travis at ieee.org (Travis E. Oliphant) Date: Mon, 13 Mar 2006 01:28:56 -0700 Subject: [Python-Dev] Strange behavior in Python 2.5a0 (trunk) --- possible error in AST? Message-ID: I'm seeing strange behavior in the Python 2.5a0 trunk that is causing the tests for numpy to fail. Apparently obj[...] = 1 is not calling PyObject_SetItem Here is a minimal example to show the error. Does anyone else see this? class temp(object): def __setitem__(self, obj, v): print obj, v mine = temp() mine[...] = 1 mine[0] = 1 Output from Python 2.4.2: Ellipsis 1 0 1 Output from Python 2.5a0: 0 1 In other words, the first line does nothing in Python 2.5a0. Does anybody else see this? Thanks, -Travis From mwh at python.net Mon Mar 13 10:47:46 2006 From: mwh at python.net (Michael Hudson) Date: Mon, 13 Mar 2006 09:47:46 +0000 Subject: [Python-Dev] Strange behavior in Python 2.5a0 (trunk) --- possible error in AST? In-Reply-To: (Travis E. Oliphant's message of "Mon, 13 Mar 2006 01:28:56 -0700") References: Message-ID: <2mk6ay65h9.fsf@starship.python.net> "Travis E. Oliphant" writes: > I'm seeing strange behavior in the Python 2.5a0 trunk that is causing > the tests for numpy to fail. Apparently obj[...] = 1 is not calling > PyObject_SetItem > > Here is a minimal example to show the error. Does anyone else see this? > > class temp(object): > def __setitem__(self, obj, v): > print obj, v > > mine = temp() > mine[...] = 1 It's a compiler problem: >>> dis.dis(compile("mine[...] = 1", '', 'single')) 1 0 LOAD_CONST 0 (1) 3 LOAD_NAME 0 (mine) 6 LOAD_CONST 1 (Ellipsis) 9 LOAD_CONST 2 (None) 12 RETURN_VALUE Cheers, mwh -- 6. The code definitely is not portable - it will produce incorrect results if run from the surface of Mars. -- James Bonfield, http://www.ioccc.org/2000/rince.hint From ncoghlan at gmail.com Mon Mar 13 13:11:38 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 13 Mar 2006 22:11:38 +1000 Subject: [Python-Dev] Strange behavior in Python 2.5a0 (trunk) --- possible error in AST? In-Reply-To: <2mk6ay65h9.fsf@starship.python.net> References: <2mk6ay65h9.fsf@starship.python.net> Message-ID: <4415617A.8030707@gmail.com> Michael Hudson wrote: > "Travis E. Oliphant" writes: > >> I'm seeing strange behavior in the Python 2.5a0 trunk that is causing >> the tests for numpy to fail. Apparently obj[...] = 1 is not calling >> PyObject_SetItem >> >> Here is a minimal example to show the error. Does anyone else see this? >> >> class temp(object): >> def __setitem__(self, obj, v): >> print obj, v >> >> mine = temp() >> mine[...] = 1 > > It's a compiler problem: > >>>> dis.dis(compile("mine[...] = 1", '', 'single')) > 1 0 LOAD_CONST 0 (1) > 3 LOAD_NAME 0 (mine) > 6 LOAD_CONST 1 (Ellipsis) > 9 LOAD_CONST 2 (None) > 12 RETURN_VALUE And how... case Ellipsis_kind: ADDOP_O(c, LOAD_CONST, Py_Ellipsis, consts) break; Just a couple of minor details missing, like, oh, compiling the actual subscript operation :) Bug here: http://www.python.org/sf/1448804 (assigned to myself, since I already wrote a test for it and worked out where to fix it) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From mcherm at mcherm.com Mon Mar 13 15:16:46 2006 From: mcherm at mcherm.com (Michael Chermside) Date: Mon, 13 Mar 2006 06:16:46 -0800 Subject: [Python-Dev] Coverity Open Source Defect Scan of Python Message-ID: <20060313061646.6re2ufpu8yo08k4s@login.werra.lunarpages.com> Dennis Allison writes: > I'd also encourage Coventry to explain their business model a bit more > clearly. Ben Chelf writes: > Of course it's no surprise that I see open > source projects everywhere -- as part of infrastructure or part of code > bases that people are developing. So from a Coverity perspective, > clearly we want to provide source code analysis for the projects that > our customers care about [...] > I really just want every developer to use source code analysis > while they write code [...] > We got a lot of the > good publicity in the research lab because there existed this big open > source OS that we could test our theories on. [...] > I think it makes sense for Coverity to have a strong relationship with > the open source community since that community has been helping us > pretty much since day 1 I work for a business... and we pay full price for the tools that we use in our work. I am aware of and follow the work that I and my colleagues do -- when someone has a good idea, I tend to learn from that and introduce the same idea in future projects. I also am aware of and follow the work of quite a few open source projects (to different degrees depending on the project). In fact, I see far more open source projects than I do other projects. I learn a lot of good ideas from these projects also, and I use them in my paid work. For example, it was my experience with open source projects that convinced me that extensive unit tests that are expected to always pass was a feasable and useful idea. Many of the ways that open source projects are managed are also beginning to work their way into my professional life also. It's just that I see far more open source projects than others, and frankly the open source projects are almost always better run, with higher standards. I doubt I'm the only one... I think open source will be leading the way in software development standards for some time now. So I think offering a software development tool FREE to open source projects in hopes of selling it for money to comercial projects is a WONDERFUL business model. Good luck! -- Michael Chermside (PS: too bad I can't buy stock in Coverity. How come all the GOOD companies are private? I had to wait around 6 years before I could buy stock in Google.) From sf at nuxeo.com Mon Mar 13 16:05:55 2006 From: sf at nuxeo.com (fermigier) Date: Mon, 13 Mar 2006 15:05:55 +0000 Subject: [Python-Dev] About "Coverity Study Ranks LAMP Code Quality" Message-ID: <44158A53.90909@nuxeo.com> http://www.internetnews.com/dev-news/article.php/3589361 "Perl had a defect density of only 0.186. In comparison Python had a defect density of 0.372 and PHP was actually above both the baseline and LAMP averages at 0.474." This is of course a PR stunt. But I'm wondering if the actual "bugs" list was transmitted to Python developers, and verified / acted upon. Because according to http://www.washingtontechnology.com/news/1_1/daily_news/28134-1.html : "The maintainers of the source codes can register with Coverity to see the full results. (End users cannot see the bug lists themselves; they will be able to see how buggy a particular program may be.)" Regards, S. From fredrik at pythonware.com Mon Mar 13 16:37:25 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Mon, 13 Mar 2006 16:37:25 +0100 Subject: [Python-Dev] About "Coverity Study Ranks LAMP Code Quality" References: <44158A53.90909@nuxeo.com> Message-ID: "fermigier" wrote: > "Perl had a defect density of only 0.186. In comparison Python had a > defect density of 0.372 and PHP was actually above both the baseline and > LAMP averages at 0.474." > > This is of course a PR stunt. But I'm wondering if the actual "bugs" > list was transmitted to Python developers, and verified / acted upon. according to http://scan.coverity.com perl is currently at 0.151 and Python 2.5 trunk is at 0.055. looks like the perlheads decided to write press releases rather than fixing their core... (given how Coverity works, the "raw" unreviewed defect density figure is relatively useless, and anyone using that for marketing is a complete idiot. I've no time to read the article, but I do hope it's not some Coverity guy you've quoted...) (also note that most true Python bugs found by Coverity are of the "may leak or misbehave under extreme pressure, in the unlikely event that you get here under such circumstances...") From fredrik at pythonware.com Mon Mar 13 17:53:06 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Mon, 13 Mar 2006 17:53:06 +0100 Subject: [Python-Dev] About "Coverity Study Ranks LAMP Code Quality" References: <44158A53.90909@nuxeo.com> Message-ID: "fermigier" wrote: > But I'm wondering if the actual "bugs" list was transmitted to Python developers, > and verified / acted upon. and in case it wasn't clear from my previous post, the answer to your specific question is "yes" ;-) From oliphant.travis at ieee.org Mon Mar 13 18:22:03 2006 From: oliphant.travis at ieee.org (Travis E. Oliphant) Date: Mon, 13 Mar 2006 10:22:03 -0700 Subject: [Python-Dev] Strange behavior in Python 2.5a0 (trunk) --- possible error in AST? In-Reply-To: <4415617A.8030707@gmail.com> References: <2mk6ay65h9.fsf@starship.python.net> <4415617A.8030707@gmail.com> Message-ID: Nick Coghlan wrote: > > And how... > > case Ellipsis_kind: > ADDOP_O(c, LOAD_CONST, Py_Ellipsis, consts) > break; > > Just a couple of minor details missing, like, oh, compiling the actual > subscript operation :) > > Bug here: http://www.python.org/sf/1448804 > > (assigned to myself, since I already wrote a test for it and worked out where > to fix it) Fabulous! The fix committed to SVN seems to work. Now, all of numpy's unit tests are passing with Python 2.5a0. Great work, thank you. -Travis From Scott.Daniels at Acm.Org Mon Mar 13 18:22:57 2006 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Mon, 13 Mar 2006 09:22:57 -0800 Subject: [Python-Dev] Coverity Open Source Defect Scan of Python In-Reply-To: <20060313061646.6re2ufpu8yo08k4s@login.werra.lunarpages.com> References: <20060313061646.6re2ufpu8yo08k4s@login.werra.lunarpages.com> Message-ID: Michael Chermside wrote: > (PS: too bad I can't buy stock in Coverity. How come all the GOOD > companies are private? I had to wait around 6 years before I could > buy stock in Google.) Maybe because the companies whose stock is available early are companies bent on producing stock profits, rather than a solid value proposition. Trying to satisfy the profit-lust of angels has redirected more than one company. --Scott David Daniels Scott.Daniels at Acm.Org From trentm at ActiveState.com Mon Mar 13 18:28:16 2006 From: trentm at ActiveState.com (Trent Mick) Date: Mon, 13 Mar 2006 09:28:16 -0800 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <1f7befae0603121548k29d18135mbcf0e3365f23c93@mail.gmail.com> References: <441217BB.1000801@v.loewis.de> <20060312002336.GA3935@activestate.com> <1f7befae0603111849n67c40694u7eebf38410b8788e@mail.gmail.com> <20060312224148.GB32648@activestate.com> <1f7befae0603121548k29d18135mbcf0e3365f23c93@mail.gmail.com> Message-ID: <20060313172816.GA9763@activestate.com> [Tim Peters wrote] > This touches on something we (including Martin) should think about: > it's very painful to build a full Python on Windows because of these > external packages... Yup. That is part of what I meant by updating PCBuild\readme.txt below: to improve the instructions so the build-dance for all these external packages is a little more cut n' paste. > Something we might be able to do instead is have just one person per > external project endure the pain of building it, and then have them > check in the whole post-compilation post-test project directory tree. > Everyone else (presumably including Windows buildbot slaves) using the > same compiler could then just check out the result. If people don't mind having prebuilt binaries checked in, yes I suppose we could do this. For some projects here at work we have a "prebuilt" dir at the root of the source tree: prebuilt/ win32-x86/ foo/ # Windows prebuilt bits for package foo linux-x86/ foo/ solaris8-sparc/ foo/ ... Whoa, someone is way ahead of me here. :) > What's worse is when checkins cause both trunk and branch rebuilds. > That can bring my zippy box to its knees, particularly when two of the > disk-intensive tests (like test_largefile) run at the same time. That would require some kind of Scheduler work in the build master config that knows how to queue up scheduled builds from multiple sources to only allow one per machine at a time. I'd find this very useful for other projects at work and I'm sure other buildbot users would as well. Maybe buildbot may already have something like this? > > Remaining TODOs: > > - make sure the python24-maint branch one works > > Works for me, although (as the Wiki page says) you also need to copy > zlib-1.2.3 into the branch build area. Yup. Looks like my build worked. Another TODO now though: - Figure out why usage of: winsound.PlaySound(, winsound.SND_ALIAS) fails on my Win2k box. This is why the test suite fails on that box. > > - get the build slaves running as a Windows service > > Mark Hammond has a patch to the buildbot project toward this end: > > Cool. I'll look into that. Trent -- Trent Mick TrentM at ActiveState.com From theller at python.net Mon Mar 13 18:48:23 2006 From: theller at python.net (Thomas Heller) Date: Mon, 13 Mar 2006 18:48:23 +0100 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <20060313172816.GA9763@activestate.com> References: <441217BB.1000801@v.loewis.de> <20060312002336.GA3935@activestate.com> <1f7befae0603111849n67c40694u7eebf38410b8788e@mail.gmail.com> <20060312224148.GB32648@activestate.com> <1f7befae0603121548k29d18135mbcf0e3365f23c93@mail.gmail.com> <20060313172816.GA9763@activestate.com> Message-ID: Trent Mick wrote: > Yup. Looks like my build worked. Another TODO now though: > > - Figure out why usage of: > winsound.PlaySound(, winsound.SND_ALIAS) > fails on my Win2k box. This is why the test suite fails on that box. > Doesn't that always fail when there is no soundcard in the machine? Thomas From trentm at ActiveState.com Mon Mar 13 20:04:06 2006 From: trentm at ActiveState.com (Trent Mick) Date: Mon, 13 Mar 2006 11:04:06 -0800 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: References: <441217BB.1000801@v.loewis.de> <20060312002336.GA3935@activestate.com> <1f7befae0603111849n67c40694u7eebf38410b8788e@mail.gmail.com> <20060312224148.GB32648@activestate.com> <1f7befae0603121548k29d18135mbcf0e3365f23c93@mail.gmail.com> <20060313172816.GA9763@activestate.com> Message-ID: <20060313190406.GB9763@activestate.com> [Thomas Heller wrote] > Trent Mick wrote: > > > Yup. Looks like my build worked. Another TODO now though: > > > > - Figure out why usage of: > > winsound.PlaySound(, winsound.SND_ALIAS) > > fails on my Win2k box. This is why the test suite fails on that box. > > > > Doesn't that always fail when there is no soundcard in the machine? I don't know. The *other* winsound tests work and I *can* hear that rising tone goop coming out of the box's tinny speakers when the test suite is run. http://www.python.org/dev/buildbot/all/x86%20W2k%202.4/builds/5/step-test/0 http://msdn.microsoft.com/library/default.asp?url=/library/en-us/multimed/htm/_win32_playsound.asp PlaySound ... SND_ALIAS The pszSound parameter is a system-event alias in the registry or the WIN.INI file. ... These aliases are apparently supposed to be in the registry or WIN.INI, but the docs don't say where. Perhaps my Win2k box is missing some registry entries. No, I think it is the config of the box -- read on. I do have a sound card in that box, however, the "Sounds and Multimedia Properties" dialog (off Control Panel) says that there are "No Playback Devices" for Sound Playback. So I guess that is it. Maybe the sound card in that box is not hooked up. Grrr. I certainly don't care about the sound card for that box but I don't want the test suite to keep reporting a spurious failure. Trent -- Trent Mick TrentM at ActiveState.com From jimjjewett at gmail.com Mon Mar 13 20:07:51 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 13 Mar 2006 14:07:51 -0500 Subject: [Python-Dev] decorator module patch Message-ID: Raymond Hettinger wrote: > In contrast, a name like "functional" suggests that > some of these tools don't quite fit. The original intent was that the functional module become the home of typical utilities for functional style programming. partial was there were patches (such as 1412451) providing quite a few additional tools and improved documentation on the more functional builtins (such as map and reduce). That particular patch was closed to spend time as an independent module first, but there is a fair chance that parts of it will eventually be added. (OTOH, it might be just a few parts which fit well under functools, rather tha just "functional") -jJ From jimjjewett at gmail.com Mon Mar 13 20:47:19 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 13 Mar 2006 14:47:19 -0500 Subject: [Python-Dev] Making builtins more efficient Message-ID: Paul Moore wrote: > Is there any practical way of detecting and flagging > constructs like the above (remotely shadowing a > builtin in another module)? Phillip J. Eby wrote: > the patch ended up being backed out ... too strict > of a check to be accepted for Python 2.4. http://svn.python.org/view/python/trunk/Objects/moduleobject.c It was revision 33054, backed out in 33084. The patch warned about any shadowing of builtins, which probably is too strict. An alternative might be to warn only about modifying *another* module's globals. (And perhaps not just when they shadow builtins?) For example, modules could grow a __sealed__ attribute which gets set at the end of the import; instead of using PyObject_GenericSetAttr directly, the tp_setattro slot would check the __sealed__ attribute (and maybe squawk) before deferring. -jJ From pje at telecommunity.com Mon Mar 13 21:14:09 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 13 Mar 2006 15:14:09 -0500 Subject: [Python-Dev] Making builtins more efficient In-Reply-To: Message-ID: <5.1.1.6.0.20060313150537.02218e20@mail.telecommunity.com> At 02:47 PM 3/13/2006 -0500, Jim Jewett wrote: >Paul Moore wrote: > > > Is there any practical way of detecting and flagging > > constructs like the above (remotely shadowing a > > builtin in another module)? > >Phillip J. Eby wrote: > > the patch ended up being backed out ... too strict > > of a check to be accepted for Python 2.4. > >http://svn.python.org/view/python/trunk/Objects/moduleobject.c > >It was revision 33054, backed out in 33084. > >The patch warned about any shadowing of builtins, which >probably is too strict. I'm curious: what actually happened? What things were causing warnings? From pedronis at strakt.com Mon Mar 13 21:28:52 2006 From: pedronis at strakt.com (Samuele Pedroni) Date: Mon, 13 Mar 2006 21:28:52 +0100 Subject: [Python-Dev] Making builtins more efficient In-Reply-To: <5.1.1.6.0.20060313150537.02218e20@mail.telecommunity.com> References: <5.1.1.6.0.20060313150537.02218e20@mail.telecommunity.com> Message-ID: <4415D604.8070104@strakt.com> Phillip J. Eby wrote: > At 02:47 PM 3/13/2006 -0500, Jim Jewett wrote: > >>Paul Moore wrote: >> >> >>>Is there any practical way of detecting and flagging >>>constructs like the above (remotely shadowing a >>>builtin in another module)? >> >>Phillip J. Eby wrote: >> >>>the patch ended up being backed out ... too strict >>>of a check to be accepted for Python 2.4. >> >>http://svn.python.org/view/python/trunk/Objects/moduleobject.c >> >>It was revision 33054, backed out in 33084. >> >>The patch warned about any shadowing of builtins, which >>probably is too strict. > > > I'm curious: what actually happened? What things were causing warnings? http://mail.python.org/pipermail/python-dev/2003-July/036879.html > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/pedronis%40strakt.com From fabianosidler at gmail.com Mon Mar 13 21:29:06 2006 From: fabianosidler at gmail.com (Fabiano Sidler) Date: Mon, 13 Mar 2006 21:29:06 +0100 Subject: [Python-Dev] Why are so many built-in types inheritable? Message-ID: Hi folks! Let me explain the above question: For debugging purpose I tried this: --- snip --- def foo(): pass function = type(foo) class PrintingFunction(function): def __init__(self, func): self.func = func def __call__(self, *args, **kwargs): print args, kwargs return function.__call__(self, args, kwargs) class DebugMeta(type): def __new__(self, name, bases, dict): for name in dict: if type(dict[name]) is function: dict[name] = PrintingFunction(dict[name]) --- snap --- Now I tought I were able to let all methods of classes with DebugMeta as metaclass print out their arguments. But I got the following sad error: TypeError: Error when calling the metaclass bases type 'function' is not an acceptable base type That's a pity, isn't it? What could I do to get the above code to work? (No, I don't want to reimplement without this unpleasant behaviour in Python. Greetings, F. Sidler From ncoghlan at gmail.com Mon Mar 13 23:18:29 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 14 Mar 2006 08:18:29 +1000 Subject: [Python-Dev] Strange behavior in Python 2.5a0 (trunk) --- possible error in AST? In-Reply-To: References: <2mk6ay65h9.fsf@starship.python.net> <4415617A.8030707@gmail.com> Message-ID: <4415EFB5.8040200@gmail.com> Travis E. Oliphant wrote: > Nick Coghlan wrote: >> And how... >> >> case Ellipsis_kind: >> ADDOP_O(c, LOAD_CONST, Py_Ellipsis, consts) >> break; >> >> Just a couple of minor details missing, like, oh, compiling the actual >> subscript operation :) >> >> Bug here: http://www.python.org/sf/1448804 >> >> (assigned to myself, since I already wrote a test for it and worked out where >> to fix it) > > Fabulous! The fix committed to SVN seems to work. > > Now, all of numpy's unit tests are passing with Python 2.5a0. > > Great work, thank you. Unfortunately my new test case breaks test_compiler. I didn't notice because I didn't use -uall before checking it in :( If no-one else gets to it, I'll try to sort it out tonight. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From martin at v.loewis.de Mon Mar 13 23:32:03 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 13 Mar 2006 23:32:03 +0100 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <20060313172816.GA9763@activestate.com> References: <441217BB.1000801@v.loewis.de> <20060312002336.GA3935@activestate.com> <1f7befae0603111849n67c40694u7eebf38410b8788e@mail.gmail.com> <20060312224148.GB32648@activestate.com> <1f7befae0603121548k29d18135mbcf0e3365f23c93@mail.gmail.com> <20060313172816.GA9763@activestate.com> Message-ID: <4415F2E3.5040909@v.loewis.de> Trent Mick wrote: >>This touches on something we (including Martin) should think about: >>it's very painful to build a full Python on Windows because of these >>external packages... > > > Yup. That is part of what I meant by updating PCBuild\readme.txt below: > to improve the instructions so the build-dance for all these external > packages is a little more cut n' paste. I took an approach with a little more automation: Tools/buildbot/external.bat gradually learns how to fetch and build the necessary prerequisites; to avoid moving URLs, these come from the external/ directory of the projects svn (in case of bsddb, this already has the patches, and the VS project files converted). > If people don't mind having prebuilt binaries checked in Well, I do mind having prebuilt binaries checked in. They take up space for little gain, plus they might increase the maintenance burden. > That would require some kind of Scheduler work in the build master > config that knows how to queue up scheduled builds from multiple sources > to only allow one per machine at a time. I'd find this very useful for > other projects at work and I'm sure other buildbot users would as well. > Maybe buildbot may already have something like this? I could not easily find it. There is the AnyBranchScheduler, but it seems to serve a different purpose. Regards, Martin From trentm at ActiveState.com Mon Mar 13 23:44:51 2006 From: trentm at ActiveState.com (Trent Mick) Date: Mon, 13 Mar 2006 14:44:51 -0800 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <4415F2E3.5040909@v.loewis.de> References: <441217BB.1000801@v.loewis.de> <20060312002336.GA3935@activestate.com> <1f7befae0603111849n67c40694u7eebf38410b8788e@mail.gmail.com> <20060312224148.GB32648@activestate.com> <1f7befae0603121548k29d18135mbcf0e3365f23c93@mail.gmail.com> <20060313172816.GA9763@activestate.com> <4415F2E3.5040909@v.loewis.de> Message-ID: <20060313224451.GA5986@activestate.com> [Martin v. Loewis wrote] > I took an approach with a little more automation: > Tools/buildbot/external.bat gradually learns how to fetch and build > the necessary prerequisites; to avoid moving URLs, these come from > the external/ directory of the projects svn (in case of bsddb, > this already has the patches, and the VS project files converted). > > > If people don't mind having prebuilt binaries checked in > > Well, I do mind having prebuilt binaries checked in. They take up space > for little gain, plus they might increase the maintenance burden. I like you approach better. Very nice. Trent -- Trent Mick TrentM at ActiveState.com From martin at v.loewis.de Mon Mar 13 23:57:34 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 13 Mar 2006 23:57:34 +0100 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <20060313190406.GB9763@activestate.com> References: <441217BB.1000801@v.loewis.de> <20060312002336.GA3935@activestate.com> <1f7befae0603111849n67c40694u7eebf38410b8788e@mail.gmail.com> <20060312224148.GB32648@activestate.com> <1f7befae0603121548k29d18135mbcf0e3365f23c93@mail.gmail.com> <20060313172816.GA9763@activestate.com> <20060313190406.GB9763@activestate.com> Message-ID: <4415F8DE.5050401@v.loewis.de> Trent Mick wrote: > I do have a sound card in that box, however, the "Sounds and Multimedia > Properties" dialog (off Control Panel) says that there are "No Playback > Devices" for Sound Playback. So I guess that is it. Maybe the sound card > in that box is not hooked up. Grrr. I certainly don't care about the > sound card for that box but I don't want the test suite to keep > reporting a spurious failure. Now, if there was a reliable check whether a soundcard is present, that check could be run as a prerequisite, then raising TestSkipped if no soundcard is present. Regards, Martin From greg.ewing at canterbury.ac.nz Tue Mar 14 00:09:22 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 14 Mar 2006 12:09:22 +1300 Subject: [Python-Dev] About "Coverity Study Ranks LAMP Code Quality" In-Reply-To: References: <44158A53.90909@nuxeo.com> Message-ID: <4415FBA2.5090308@canterbury.ac.nz> Fredrik Lundh wrote: > > But I'm wondering if the actual "bugs" list was transmitted to Python developers, > > and verified / acted upon. > > and in case it wasn't clear from my previous post, the answer to > your specific question is "yes" ;-) Could whoever did this perhaps post a brief description of what sort of information their tool produces, and how good a hit rate they have of finding real problems? Greg From jepler at unpythonic.net Tue Mar 14 00:05:07 2006 From: jepler at unpythonic.net (Jeff Epler) Date: Mon, 13 Mar 2006 17:05:07 -0600 Subject: [Python-Dev] About "Coverity Study Ranks LAMP Code Quality" In-Reply-To: <44158A53.90909@nuxeo.com> References: <44158A53.90909@nuxeo.com> Message-ID: <20060313230507.GA18532@unpythonic.net> On Mon, Mar 13, 2006 at 03:05:55PM +0000, fermigier wrote: > Because according to > http://www.washingtontechnology.com/news/1_1/daily_news/28134-1.html : > > "The maintainers of the source codes can register with Coverity to see > the full results. (End users cannot see the bug lists themselves; they > will be able to see how buggy a particular program may be.)" This distinction tweaks me a bit. One strength of Open Source software is that any "end user" may choose to be a developer. But it sounds like I have little chance of seeing Coverity's scan results on Open Source software, because I'm not a "maintainer" of any project they've chosen to scan. This despite the fact that over the years I've made (admittedly minor) contributions to a handful of the projects on their list, and might choose to fix some of the scan-discovered defects if only I could know what they were. Jeff From martin at v.loewis.de Tue Mar 14 00:55:52 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 14 Mar 2006 00:55:52 +0100 Subject: [Python-Dev] About "Coverity Study Ranks LAMP Code Quality" In-Reply-To: <20060313230507.GA18532@unpythonic.net> References: <44158A53.90909@nuxeo.com> <20060313230507.GA18532@unpythonic.net> Message-ID: <44160688.3000209@v.loewis.de> Jeff Epler wrote: >>Because according to >>http://www.washingtontechnology.com/news/1_1/daily_news/28134-1.html : >> >>"The maintainers of the source codes can register with Coverity to see >>the full results. (End users cannot see the bug lists themselves; they >>will be able to see how buggy a particular program may be.)" > > > This distinction tweaks me a bit. I can understand that position. The bugs they find include potential security flaws, for which exploits could be created if the results are freely available. While its clearly impossible to keep that information only with trusted people, they need to make a faithful attempt to restrict it. If you have contributed to open source projects, you should ask the maintainers of these projects to copy you the reports they produced. If these maintainers consider you trustworthy, they will reveal it to you. Regards, Martin From jepler at unpythonic.net Tue Mar 14 01:55:11 2006 From: jepler at unpythonic.net (Jeff Epler) Date: Mon, 13 Mar 2006 18:55:11 -0600 Subject: [Python-Dev] checkin r43015 Message-ID: <20060314005511.GA22066@unpythonic.net> After the recent discussion about Coverity, I took a look at one of the checkins made, apparently based on output from their tool. http://svn.python.org/view/python/branches/release24-maint/Objects/object.c?&r1=43015&r2=43014&rev=43015&view=diff&diff_format=l This change, a backport of a similar change made to HEAD, doesn't seem to fix the flaw: the PyUnicode_CheckExact() call is now guarded against a NULL return, but the subsequent PyUnicode_Check() and PyString_Check() calls don't seem to be. I'm not 100% sure what's going on here, but it still looks a bit fishy. The API reference says that PyObject_AsUnicode may return NULL, so why doesn't the function just call PyErr_BadInternalCall() and return NULL? Jeff From trentm at ActiveState.com Tue Mar 14 02:08:19 2006 From: trentm at ActiveState.com (Trent Mick) Date: Mon, 13 Mar 2006 17:08:19 -0800 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <4415F8DE.5050401@v.loewis.de> References: <441217BB.1000801@v.loewis.de> <20060312002336.GA3935@activestate.com> <1f7befae0603111849n67c40694u7eebf38410b8788e@mail.gmail.com> <20060312224148.GB32648@activestate.com> <1f7befae0603121548k29d18135mbcf0e3365f23c93@mail.gmail.com> <20060313172816.GA9763@activestate.com> <20060313190406.GB9763@activestate.com> <4415F8DE.5050401@v.loewis.de> Message-ID: <20060314010819.GB29489@activestate.com> [Martin v. Loewis wrote] > Trent Mick wrote: > > I do have a sound card in that box, however, the "Sounds and Multimedia > > Properties" dialog (off Control Panel) says that there are "No Playback > > Devices" for Sound Playback. So I guess that is it. Maybe the sound card > > in that box is not hooked up. Grrr. I certainly don't care about the > > sound card for that box but I don't want the test suite to keep > > reporting a spurious failure. > > Now, if there was a reliable check whether a soundcard is present, that > check could be run as a prerequisite, then raising TestSkipped if no > soundcard is present. Roger on python-win32 had an answer which works for me: [Roger Upole wrote] > WMI can list sound devices. > > import win32com.client > wmi=win32com.client.GetObject('winmgmts:') > scs=wmi.InstancesOf('win32_sounddevice') > for sc in scs: > print sc.Properties_('Name'), sc.Properties_('Status') However, that requires PyWin32 so can't really use that for test_winsound.py. My understanding of ctypes is that it can NOT replace win32com, but I'd be happy to be wrong here. Thomas? Trent -- Trent Mick TrentM at ActiveState.com From mhammond at skippinet.com.au Tue Mar 14 02:30:15 2006 From: mhammond at skippinet.com.au (Mark Hammond) Date: Tue, 14 Mar 2006 12:30:15 +1100 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <20060314010819.GB29489@activestate.com> Message-ID: > Roger on python-win32 had an answer which works for me: > > [Roger Upole wrote] > > WMI can list sound devices. > > > > import win32com.client > > wmi=win32com.client.GetObject('winmgmts:') > > scs=wmi.InstancesOf('win32_sounddevice') > > for sc in scs: > > print sc.Properties_('Name'), sc.Properties_('Status') > > However, that requires PyWin32 so can't really use that for > test_winsound.py. My understanding of ctypes is that it can NOT replace > win32com, but I'd be happy to be wrong here. Thomas? Maybe the following VBScript "port" of the above will work: -- check_soundcard.vbs rem Check for a working sound-card - exit with 0 if OK, 1 otherwise. set wmi = GetObject("winmgmts:") set scs = wmi.InstancesOf("win32_sounddevice") for each sc in scs set status = sc.Properties_("Status") wscript.Echo(sc.Properties_("Name") + "/" + status) if status = "OK" then wscript.Quit 0 rem normal exit end if next rem No sound card found - exit with status code of 1 wscript.Quit 1 -- eof Running "cscript.exe check_soundcard.vbs" and checking the return code should work. cscript.exe comes with all modern Windows variants, and although there may be ways to install Windows without it, I think we can safely assume it exists for these purposes. Cheers, Mark From amk at amk.ca Tue Mar 14 04:22:19 2006 From: amk at amk.ca (A.M. Kuchling) Date: Mon, 13 Mar 2006 22:22:19 -0500 Subject: [Python-Dev] Topic suggestions from the PyCon feedback Message-ID: <20060314032219.GA3629@rogue.amk.ca> Those of you on python-checkins will have noticed the recent fiddling around with the feedback from PyCon. I'd like to draw the attention of the python-dev readership to the answers for the question "What 3 topics should have been covered at PyCon?" I split out core Python and web-related topics into their own lists. Here's the core Python list: Usage of new/advanced python features (12) Python Best Practices (3) Design Patterns in Python (3) Anything Alex Martelli wants to talk about. (3) Advanced Python: metaclasses, descriptors, decorators (3) Strategies for integrating C and Python (2) Problems with python (2) Iterators and Generators (2) Ways to improve performance of Python code (1) Standard library gems/ unsung modules (1) Python Language perspective from non-Guido PEP members (1) Py3K (1) More on Python internals (1) Language howtos (I really enjoyed Alex Martelli's talk last year on itertools) (1) Getting started hacking python source (1) Decimal for Experts (1) Better Development Practices with Python (1) Being more productive with Python (1) In the first entry, "new/advanced features" = a fuzzy set containing generators, iterators, metaclasses, __slots__, and decorators. When 2.5 is released, this set will probably grow to include the 'with' statement's context managers and coroutines. Some of the other entries in the above list overlap with the first entry. The conclusion I draw from these results: intermediate- or advanced-level topics of program design are not covered enough, whether in the Python documentation, in published books and articles, or in PyCon talks. Please feel free to mine the above list, or the rest of the PyCon feedback, for topic ideas. In particular: if you're going to attend PyCon 2007, EuroPython, or some other conference (even a non-Python one), please consider submitting a talk proposal covering one of the above topics. Such presentations would find a receptive audience, I think. --amk From raymond.hettinger at verizon.net Tue Mar 14 03:57:59 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Mon, 13 Mar 2006 21:57:59 -0500 Subject: [Python-Dev] Threading idea -- exposing a global thread lock Message-ID: <000701c64713$1aaa7fd0$6a01a8c0@RaymondLaptop1> A user on comp.lang.python has twisted himself into knots writing multi-threaded code that avoids locks and queues but fails when running code with non-atomic access to a shared resource. While his specific design is somewhat flawed, it does suggest that we could offer an easy way to make a block of code atomic without the complexity of other synchronization tools: gil.acquire() try: #do some transaction that needs to be atomic finally: gil.release() The idea is to temporarily suspend thread switches (either using the GIL or a global variable in the eval-loop). Think of it as "non-cooperative" multi-threading. While this is a somewhat rough approach, it is dramatically simpler than the alternatives (i.e. wrapping locks around every access to a resource or feeding all resource requests to a separate thread via a Queue). While I haven't tried it yet, I think the implementation is likely to be trivial. FWIW, the new with-statement makes the above fragment even more readable: with atomic_transaction(): # do a series of steps without interruption Raymond From hyeshik at gmail.com Tue Mar 14 04:20:15 2006 From: hyeshik at gmail.com (Hye-Shik Chang) Date: Tue, 14 Mar 2006 12:20:15 +0900 Subject: [Python-Dev] checkin r43015 In-Reply-To: <20060314005511.GA22066@unpythonic.net> References: <20060314005511.GA22066@unpythonic.net> Message-ID: <4f0b69dc0603131920y62ccbe55l892e8698998c73d3@mail.gmail.com> On 3/14/06, Jeff Epler wrote: > After the recent discussion about Coverity, I took a look at one of the > checkins made, apparently based on output from their tool. > > http://svn.python.org/view/python/branches/release24-maint/Objects/object.c?&r1=43015&r2=43014&rev=43015&view=diff&diff_format=l > > This change, a backport of a similar change made to HEAD, doesn't seem > to fix the flaw: the PyUnicode_CheckExact() call is now guarded against > a NULL return, but the subsequent PyUnicode_Check() and PyString_Check() > calls don't seem to be. Agreed. That change doesn't fix the real problem. I bet it'll still segfault for PyObject_Unicode(NULL). In fact, I and Neal talked about the problem and have a correct patch. But the patch looks bit messy so we hated it. :-) http://python.org/sf/1444030 > I'm not 100% sure what's going on here, but it still looks a bit fishy. > The API reference says that PyObject_AsUnicode may return NULL, so why > doesn't the function just call PyErr_BadInternalCall() and return NULL? For the consistency with PyObject_String(NULL) which returns a string "". Hye-Shik From aleaxit at gmail.com Tue Mar 14 04:22:11 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Mon, 13 Mar 2006 19:22:11 -0800 Subject: [Python-Dev] Why are so many built-in types inheritable? In-Reply-To: References: Message-ID: On Mar 13, 2006, at 12:29 PM, Fabiano Sidler wrote: > Hi folks! Hello Fabiano! The proper venue for your interesting issues is comp.lang.python (or the equivalent mailing list), where all sorts of people will be able to "hear" you, discuss things, and help out. python-dev is strictly for developers of Python itself, so please re- post your issues to comp.lang.python. Thanks, Alex From trentm at ActiveState.com Tue Mar 14 04:48:31 2006 From: trentm at ActiveState.com (Trent Mick) Date: Mon, 13 Mar 2006 19:48:31 -0800 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: References: <20060314010819.GB29489@activestate.com> Message-ID: <20060314034831.GE10510@activestate.com> [Mark Hammond wrote] > Maybe the following VBScript "port" of the above will work: > ... Cool, yes that works. > Running "cscript.exe check_soundcard.vbs" and checking the return code > should work. cscript.exe comes with all modern Windows variants, and > although there may be ways to install Windows without it, I think we can > safely assume it exists for these purposes. I have a patch in the works that defaults to "yes, this machine does have a soundcard" if cscript.exe cannot be found on the PATH. However, one wrinkle: test_winsound.py is made up of three test cases: BeepTest MessageBeepTest PlaySoundTest only the last need be skipped if there is not soundcard. However, TestSkipped only works add the module level. So, which is better: 1. Use TestSkipped and skip all three test cases if there is not sound card. Running the test suite will actually show that something is being skipped. 2. Conditionally define class PlaySoundTest only if there is a soundcard. Running the test suite on a machine without a soundcard will not show that something is being skipped but *will* run the Beep tests. 3. Break out test_winsound.py into two test modules: one with the beep tests and one with PlaySoundTest (the latter using TestSkipped). Trent -- Trent Mick TrentM at ActiveState.com From aleaxit at gmail.com Tue Mar 14 04:52:20 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Mon, 13 Mar 2006 19:52:20 -0800 Subject: [Python-Dev] Topic suggestions from the PyCon feedback In-Reply-To: <20060314032219.GA3629@rogue.amk.ca> References: <20060314032219.GA3629@rogue.amk.ca> Message-ID: On Mar 13, 2006, at 7:22 PM, A.M. Kuchling wrote: ... > Design Patterns in Python (3) > Anything Alex Martelli wants to talk about. (3) ... > Language howtos (I really enjoyed Alex Martelli's talk last year on > itertools) (1) Wow, I'm blushing;-). I promise and swear I'll do the utmost to attend Pycon 2007 -- this year, what between work-related issues AND the 2nd edition of the Nutshell, it was, alas, really unfeasible:-(. The *ONE* thing I dislike about working in the US is vacations -- I get about half of what I would expect in Europe, and that's with my employer being reasonably generous... in practice, given I NEED some time to go visit family and friends back in Italy, this means I can't really take vacations to do conferences, but rather I must convince my boss that conference X is worth my time (basically, this means that by attending I can hope to help *HIRE* somebody -- otherwise, I'm fighting uphill!-). > In the first entry, "new/advanced features" = a fuzzy set containing > generators, iterators, metaclasses, __slots__, and decorators. When > 2.5 is released, this set will probably grow to include the 'with' > statement's context managers and coroutines. Some of the other > entries in the above list overlap with the first entry. Wow, I'm RARIN' for a go at that -- I did the former more than once, and 'with' and coroutines sound right up my alley -- if I get any chance to practice them, I might even be able to present related *patterns*...;-) > In particular: if you're going to attend PyCon 2007, EuroPython, or > some other conference (even a non-Python one), please consider > submitting a talk proposal covering one of the above topics. Such > presentations would find a receptive audience, I think. A-yup. Most presentations cover specific project, which is fine, but there's an unsatisfied demand for talks on how best to use certain language features, and design patterns around them -- I suspect that's a big reason why I was singled out by name (that kind of thing is generally what I address in my talks). Alex From pje at telecommunity.com Tue Mar 14 05:06:49 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 13 Mar 2006 23:06:49 -0500 Subject: [Python-Dev] Threading idea -- exposing a global thread lock In-Reply-To: <000701c64713$1aaa7fd0$6a01a8c0@RaymondLaptop1> Message-ID: <5.1.1.6.0.20060313230035.03f93610@mail.telecommunity.com> At 09:57 PM 3/13/2006 -0500, Raymond Hettinger wrote: >FWIW, the new with-statement makes the above fragment even more readable: > > with atomic_transaction(): > # do a series of steps without interruption +1 on the idea, -1000 on the name. It's neither atomic nor a transaction. I believe that "critical section" is a more common term for what you're proposing. Probably the primitive could be placed in the thread or threading module, so that it would be something like: with threading.critical_section(): # ... It might be nice to be able to escape out of the critical section using a nested with: statement, as this would allow you to treat much of a program as single-threaded, and then selectively allow task switching. But I'm not sure what you'd call that API. :) From nnorwitz at gmail.com Tue Mar 14 06:03:55 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 13 Mar 2006 21:03:55 -0800 Subject: [Python-Dev] About "Coverity Study Ranks LAMP Code Quality" In-Reply-To: <4415FBA2.5090308@canterbury.ac.nz> References: <44158A53.90909@nuxeo.com> <4415FBA2.5090308@canterbury.ac.nz> Message-ID: On 3/13/06, Greg Ewing wrote: > Fredrik Lundh wrote: > > > > But I'm wondering if the actual "bugs" list was transmitted to Python developers, > > > and verified / acted upon. > > > > and in case it wasn't clear from my previous post, the answer to > > your specific question is "yes" ;-) 16 people have access, 2-3 of which I haven't heard of. 6+ different developers have reviewed at least some of the reports. > Could whoever did this perhaps post a brief description > of what sort of information their tool produces, and > how good a hit rate they have of finding real problems? I can post info, but it won't be brief. :-) Their reports were high quality and accurate. Of the false positives, it was difficult for the tool to determine that the condition they are checking really doesn't occur. One example is doing a PyString_Check() then using PyString_AsString() (or Size). The tool complained that they could return NULL/negative number, which is true in general. However, since we already know the object is a string since we already verified it, we know those APIs can't return error conditions. Current results: Uninspected 12 False Positives 13 Ignore 24 Resolved 3 Total 51 For our purposes, False, Ignore, and Resolved are all pretty much the same in that the warnings are no longer pertinent. Actually, one of the resolved is still outstanding as Jeff Epler? pointed out. http://scan.coverity.com/ shows Python originally had 96 reported issues, currently 15 are outstanding as of 2006-Mar-13 01:16:18 (presumably PST). This does not include all C code checked in, since their report includes code: 272,268, comment: 41,269, blank 38,496 (total: 352033) and: $ wc -1 `find . -name '*.[ch]'` | tail -n 1 425810 The public report says 15, but the current developer report shows 12. I'm not sure why there is a discrepancy. All 12 are in ctypes which was recently imported. The reports provide information about what condition is violated and where. Usually this means there are at least 2, somtimes more, annotations in the code to point out the originating condition and problem area. The types of problems reported in Python include: unintialized variables, resource leak, negative return values, using a NULL pointer, dead code, use after free and some other similar conditions. Here's a report of the current disposition of issues: Name Uninspected Bug False Ignore Pending Resolved Total 0 0 0 1 0 0 1 Everything 12 0 13 23 0 3 51 Modules 12 0 2 8 0 0 22 Modules/expat 0 0 1 0 0 0 1 Objects 0 0 6 8 0 1 15 Parser 0 0 0 3 0 0 3 Python 0 0 5 6 0 2 13 Include 0 0 0 0 0 0 0 Modules/cjkcodecs 0 0 0 0 0 0 0 usr 0 0 0 0 0 0 0 usr/X11R6 0 0 0 0 0 0 0 usr/X11R6/include 0 0 0 0 0 0 0 usr/include 0 0 0 0 0 0 0 usr/include/arpa 0 0 0 0 0 0 0 usr/include/asm 0 0 0 0 0 0 0 usr/include/bits 0 0 0 0 0 0 0 usr/include/bluetooth 0 0 0 0 0 0 0 usr/include/et 0 0 0 0 0 0 0 usr/include/gdbm 0 0 0 0 0 0 0 usr/include/gnu 0 0 0 0 0 0 0 usr/include/linux 0 0 0 0 0 0 0 usr/include/ncurses 0 0 0 0 0 0 0 usr/include/net 0 0 0 0 0 0 0 usr/include/netinet 0 0 0 0 0 0 0 usr/include/netpacket 0 0 0 0 0 0 0 usr/include/openssl 0 0 0 0 0 0 0 usr/include/readline 0 0 0 0 0 0 0 usr/include/rpc 0 0 0 0 0 0 0 usr/include/rpcsvc 0 0 0 0 0 0 0 usr/include/sys 0 0 0 0 0 0 0 usr/lib 0 0 0 0 0 0 0 usr/lib/gcc 0 0 0 0 0 0 0 From guido at python.org Tue Mar 14 06:06:00 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 13 Mar 2006 21:06:00 -0800 Subject: [Python-Dev] Threading idea -- exposing a global thread lock In-Reply-To: <000701c64713$1aaa7fd0$6a01a8c0@RaymondLaptop1> References: <000701c64713$1aaa7fd0$6a01a8c0@RaymondLaptop1> Message-ID: Oh, no! Please! I just had to dissuade someone inside Google from the same idea. IMO it's fatally flawed for several reasons: it doesn't translate reasonably to Jython or IronPython, it's really tricky to implement, and it's an invitation for deadlocks. The danger of this thing in the wrong hands is too big to warrant the (rare) use case that can only be solved elegantly using direct GIL access. --Guido On 3/13/06, Raymond Hettinger wrote: > A user on comp.lang.python has twisted himself into knots writing multi-threaded > code that avoids locks and queues but fails when running code with non-atomic > access to a shared resource. While his specific design is somewhat flawed, it > does suggest that we could offer an easy way to make a block of code atomic > without the complexity of other synchronization tools: > > gil.acquire() > try: > #do some transaction that needs to be atomic > finally: > gil.release() > > The idea is to temporarily suspend thread switches (either using the GIL or a > global variable in the eval-loop). Think of it as "non-cooperative" > multi-threading. While this is a somewhat rough approach, it is dramatically > simpler than the alternatives (i.e. wrapping locks around every access to a > resource or feeding all resource requests to a separate thread via a Queue). > > While I haven't tried it yet, I think the implementation is likely to be > trivial. > > FWIW, the new with-statement makes the above fragment even more readable: > > with atomic_transaction(): > # do a series of steps without interruption > > > Raymond > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Tue Mar 14 06:08:45 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 14 Mar 2006 00:08:45 -0500 Subject: [Python-Dev] Threading idea -- exposing a global thread lock In-Reply-To: <5.1.1.6.0.20060313230035.03f93610@mail.telecommunity.com> References: <5.1.1.6.0.20060313230035.03f93610@mail.telecommunity.com> Message-ID: <1142312925.2091.6.camel@geddy.wooz.org> On Mon, 2006-03-13 at 23:06 -0500, Phillip J. Eby wrote: > +1 on the idea, -1000 on the name. It's neither atomic nor a > transaction. I believe that "critical section" is a more common term for > what you're proposing. > > Probably the primitive could be placed in the thread or threading module, > so that it would be something like: > > with threading.critical_section(): > # ... Or even threading.synchronized(). But in any event, +1 on the idea and on sticking the primitive in threading. > It might be nice to be able to escape out of the critical section using a > nested with: statement, as this would allow you to treat much of a program > as single-threaded, and then selectively allow task switching. But I'm not > sure what you'd call that API. :) threading.unsynchronize()? Yikes. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060314/715a3fb8/attachment.pgp From raymond.hettinger at verizon.net Tue Mar 14 06:36:19 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Tue, 14 Mar 2006 00:36:19 -0500 Subject: [Python-Dev] Threading idea -- exposing a global thread lock References: <000701c64713$1aaa7fd0$6a01a8c0@RaymondLaptop1> Message-ID: <007301c64729$393f7890$6a01a8c0@RaymondLaptop1> [Guido] > Oh, no! Before shooting this one down, consider a simpler incarnation not involving the GIL. The idea is to allow an active thread to temporarily suspend switching for a few steps: threading.stop_switching() # step1 # step2 # setp3 theading.resume_switching() To me, this version is innocuous -- it doesn't violate any existing guarantees (no thread knows when it is going to be switched out or how long it is going to be suspended) and allows the user to easily create an atomic transaction (i.e. swapping two global variables or printing a debug value). While it could be abused, it has the offsetting virtue of being much simpler than the alternatives. I disagree that the need is rare. My own use case is that I sometimes add some debugging print statements that need to execute atomically -- it is a PITA because PRINT_ITEM and PRINT_NEWLINE are two different opcodes and are not guaranteed to pair atomically. The current RightWay(tm) is for me to create a separate daemon thread for printing and to send lines to it via the queue module (even that is tricky because you don't want the main thread to exit before a print queued item is completed). I suggest that that is too complex for a simple debugging print statement. It would be great to simply write: with other_threads_suspended(): print value I haven't worked through the IronPython and Jython points of view but think there is probably a way to implement the simpler incarnation. Also, unlike the gil.aquire() version, there is no blocking of the currently active thread. IOW, the above incarnation won't deadlock (because the main thread never goes into a wait mode). Raymond ----- Original Message ----- From: "Guido van Rossum" To: "Raymond Hettinger" Cc: Sent: Tuesday, March 14, 2006 12:06 AM Subject: Re: [Python-Dev] Threading idea -- exposing a global thread lock Oh, no! Please! I just had to dissuade someone inside Google from the same idea. IMO it's fatally flawed for several reasons: it doesn't translate reasonably to Jython or IronPython, it's really tricky to implement, and it's an invitation for deadlocks. The danger of this thing in the wrong hands is too big to warrant the (rare) use case that can only be solved elegantly using direct GIL access. --Guido On 3/13/06, Raymond Hettinger wrote: > A user on comp.lang.python has twisted himself into knots writing > multi-threaded > code that avoids locks and queues but fails when running code with non-atomic > access to a shared resource. While his specific design is somewhat flawed, it > does suggest that we could offer an easy way to make a block of code atomic > without the complexity of other synchronization tools: > > gil.acquire() > try: > #do some transaction that needs to be atomic > finally: > gil.release() > > The idea is to temporarily suspend thread switches (either using the GIL or a > global variable in the eval-loop). Think of it as "non-cooperative" > multi-threading. While this is a somewhat rough approach, it is dramatically > simpler than the alternatives (i.e. wrapping locks around every access to a > resource or feeding all resource requests to a separate thread via a Queue). > > While I haven't tried it yet, I think the implementation is likely to be > trivial. > > FWIW, the new with-statement makes the above fragment even more readable: > > with atomic_transaction(): > # do a series of steps without interruption > > > Raymond > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From nnorwitz at gmail.com Tue Mar 14 06:38:48 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 13 Mar 2006 21:38:48 -0800 Subject: [Python-Dev] Developing/patching ctypes (was: Re: integrating ctypes into python) In-Reply-To: <44112F50.3020506@python.net> References: <440E9614.3000504@python.net> <4f0b69dc0603080104g6053ae47p59be98f81e65ddad@mail.gmail.com> <440F463E.7050009@python.net> <4f0b69dc0603081535q5e2ef69ft99e339a920dbc25e@mail.gmail.com> <44107267.9090806@python.net> <4f0b69dc0603091623jefdb8fhf4b13695bbb491dc@mail.gmail.com> <44112F50.3020506@python.net> Message-ID: On 3/9/06, Thomas Heller wrote: > Would it be a solution to move the 'official' ctypes development into > Python SVN external/ctypes, or would this be considered abuse? Another > location in SVN could be used as well, if external is though to contain > only vendor drops... Thomas, I'd be fine with the official ctypes repo being Python SVN. The attached patch fixes all the ctypes tests so they pass on amd64. It also fixes several warnings. I'm not sure what else to do with the patch. Let me know how you want to handle these in the future. I'm not sure the patch is 100% correct. You will need to decide what can be 64 bits and what can't. I believe sq_{item,slice,ass_item,ass_slice} all need to use Py_ssize_t. The types in ctypes.h may not require all the changes I made. I don't know how you want to support older version, so I unconditionally changed the types to Py_ssize_t. n -------------- next part -------------- A non-text attachment was scrubbed... Name: ctypes.patch Type: text/x-patch Size: 11275 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20060313/736d2550/attachment-0001.bin From raymond.hettinger at verizon.net Tue Mar 14 07:01:36 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Tue, 14 Mar 2006 01:01:36 -0500 Subject: [Python-Dev] Threading idea -- exposing a global thread lock References: <000701c64713$1aaa7fd0$6a01a8c0@RaymondLaptop1> Message-ID: <00c501c6472c$c1121a40$6a01a8c0@RaymondLaptop1> > it doesn't translate reasonably to Jython or IronPython, it's really tricky to > implement, FWIW, someone on the newsgroup suggested implementing this via a slight modification to sys.checkinterval(). The idea was that a None argument would translate to "stop-checking" and the active thread would do a few steps atomically and then restore the original value. The new with-statement makes that a piece of cake. Raymond From nnorwitz at gmail.com Tue Mar 14 07:04:40 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 13 Mar 2006 22:04:40 -0800 Subject: [Python-Dev] checkin r43015 In-Reply-To: <4f0b69dc0603131920y62ccbe55l892e8698998c73d3@mail.gmail.com> References: <20060314005511.GA22066@unpythonic.net> <4f0b69dc0603131920y62ccbe55l892e8698998c73d3@mail.gmail.com> Message-ID: On 3/13/06, Hye-Shik Chang wrote: > On 3/14/06, Jeff Epler wrote: > > After the recent discussion about Coverity, I took a look at one of the > > checkins made, apparently based on output from their tool. > > > > http://svn.python.org/view/python/branches/release24-maint/Objects/object.c?&r1=43015&r2=43014&rev=43015&view=diff&diff_format=l > > > > This change, a backport of a similar change made to HEAD, doesn't seem > > to fix the flaw: the PyUnicode_CheckExact() call is now guarded against > > a NULL return, but the subsequent PyUnicode_Check() and PyString_Check() > > calls don't seem to be. > > Agreed. That change doesn't fix the real problem. I bet it'll still segfault > for PyObject_Unicode(NULL). > > In fact, I and Neal talked about the problem and have a correct patch. > But the patch looks bit messy so we hated it. :-) > http://python.org/sf/1444030 Right. I realized it would just be easier to inline the code, which is what i wound up doing. This problem should be addressed fully now. n From pedronis at strakt.com Tue Mar 14 08:34:59 2006 From: pedronis at strakt.com (Samuele Pedroni) Date: Tue, 14 Mar 2006 08:34:59 +0100 Subject: [Python-Dev] Threading idea -- exposing a global thread lock In-Reply-To: <00c501c6472c$c1121a40$6a01a8c0@RaymondLaptop1> References: <000701c64713$1aaa7fd0$6a01a8c0@RaymondLaptop1> <00c501c6472c$c1121a40$6a01a8c0@RaymondLaptop1> Message-ID: <44167223.1080603@strakt.com> Raymond Hettinger wrote: >>it doesn't translate reasonably to Jython or IronPython, it's really tricky to >>implement, > > > FWIW, someone on the newsgroup suggested implementing this via a slight > modification to sys.checkinterval(). The idea was that a None argument would > translate to "stop-checking" and the active thread would do a few steps > atomically and then restore the original value. The new with-statement makes > that a piece of cake. > > there's no sys.checkinterval in Jython. Implementing this would need the introduction of some kind of GIL implementation in Jython, the JVM has no primitive for global critical sections. A GIL is something users of Jython don't want. Even with that is way too easy to have non-Jython threads around that could manipulates Java objects you are seamlessy dealing with. That is of course true for CPython and C extension initiated threads too, but is probably harder to be oblivious of such things in CPython context. From fredrik at pythonware.com Tue Mar 14 08:45:27 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue, 14 Mar 2006 08:45:27 +0100 Subject: [Python-Dev] About "Coverity Study Ranks LAMP Code Quality" References: <44158A53.90909@nuxeo.com> <4415FBA2.5090308@canterbury.ac.nz> Message-ID: Neal Norwitz wrote: > Their reports were high quality and accurate. absolutely (which is why I'm surprised that someone's using the un- reviewed numbers are a quality measure; guess I have to go back and read the article to see who that was...) > Of the false positives, it was difficult for the tool to determine that > the condition they are checking really doesn't occur. One example > is doing a PyString_Check() then using PyString_AsString() (or Size). > The tool complained that they could return NULL/negative number, > which is true in general. one favourite was a function that used both a return value and an output argument, and used the following combinations to represent the different outcomes: return=NULL; output=junk => out of memory return=junk; output=-1 => cannot do this return=pointer; output=value => did this, returned value bytes i.e. p = function(..., &out); if (!p) return "out of memory"; if (out == -1) use fallback algorithm return result; else { generate result free(p); return result; } Coverity pointed out that in the out == -1 case, the calling code never released the p pointer. while it was easy to flag this as INVALID, I agree that the design is a bit questionable; I'd probably change the function slightly to make sure it always returns either NULL or a valid pointer: p = function(..., &out); if (!p) { if (out == -1) use fallback algorithm return result; } return "out of memory"; } generate result free(p); return result; anyway, this is of course related to why the raw coverity figures can be grossly misleading: code that uses certain coding patterns may get really bad figures, without being broken in any way whatsoever. (on the other hand, you can treat the Coverity result as "even more warnings than your (current) compiler can provide", and fix as many false reports as you can, just for stylistic reasons). From martin at v.loewis.de Tue Mar 14 08:53:46 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 14 Mar 2006 08:53:46 +0100 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <20060314034831.GE10510@activestate.com> References: <20060314010819.GB29489@activestate.com> <20060314034831.GE10510@activestate.com> Message-ID: <4416768A.7080900@v.loewis.de> Trent Mick wrote: > 1. Use TestSkipped and skip all three test cases if there is not sound > card. Running the test suite will actually show that something is > being skipped. This is best. The sound tests are not that important that they absolutely need to be run. Regards, Martin From theller at python.net Tue Mar 14 09:06:04 2006 From: theller at python.net (Thomas Heller) Date: Tue, 14 Mar 2006 09:06:04 +0100 Subject: [Python-Dev] Developing/patching ctypes In-Reply-To: References: <440E9614.3000504@python.net> <4f0b69dc0603080104g6053ae47p59be98f81e65ddad@mail.gmail.com> <440F463E.7050009@python.net> <4f0b69dc0603081535q5e2ef69ft99e339a920dbc25e@mail.gmail.com> <44107267.9090806@python.net> <4f0b69dc0603091623jefdb8fhf4b13695bbb491dc@mail.gmail.com> <44112F50.3020506@python.net> Message-ID: <4416796C.3080600@python.net> Neal Norwitz wrote: > On 3/9/06, Thomas Heller wrote: >> Would it be a solution to move the 'official' ctypes development into >> Python SVN external/ctypes, or would this be considered abuse? Another >> location in SVN could be used as well, if external is though to contain >> only vendor drops... > > Thomas, > > I'd be fine with the official ctypes repo being Python SVN. > > The attached patch fixes all the ctypes tests so they pass on amd64. > It also fixes several warnings. I'm not sure what else to do with the > patch. Let me know how you want to handle these in the future. > > I'm not sure the patch is 100% correct. You will need to decide what > can be 64 bits and what can't. I believe > sq_{item,slice,ass_item,ass_slice} all need to use Py_ssize_t. The > types in ctypes.h may not require all the changes I made. I don't > know how you want to support older version, so I unconditionally > changed the types to Py_ssize_t. > > n Thanks, Neal, I'll look into that this night. In the future I hope to have access to a amd64-linux system, and I'll try to keep this stuff up-to-date myself. Thomas From raymond.hettinger at verizon.net Tue Mar 14 09:08:02 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Tue, 14 Mar 2006 03:08:02 -0500 Subject: [Python-Dev] Threading idea -- exposing a global thread lock References: <000701c64713$1aaa7fd0$6a01a8c0@RaymondLaptop1> <00c501c6472c$c1121a40$6a01a8c0@RaymondLaptop1> <44167223.1080603@strakt.com> Message-ID: <001701c6473e$6b232e50$6a01a8c0@RaymondLaptop1> [Samuele Pedroni] > there's no sys.checkinterval in Jython. Implementing this would need the > introduction of some kind of GIL implementation in Jython, the JVM has no > primitive for global critical sections. Wouldn't Java implement this directly by suspending and resuming the other threads (being careful to avoid access to monitored resources and to pair the suspend/resume operations in a try/finally or with-statement to prevent deadlocks)? Raymond From pedronis at strakt.com Tue Mar 14 09:59:59 2006 From: pedronis at strakt.com (Samuele Pedroni) Date: Tue, 14 Mar 2006 09:59:59 +0100 Subject: [Python-Dev] Threading idea -- exposing a global thread lock In-Reply-To: <001701c6473e$6b232e50$6a01a8c0@RaymondLaptop1> References: <000701c64713$1aaa7fd0$6a01a8c0@RaymondLaptop1> <00c501c6472c$c1121a40$6a01a8c0@RaymondLaptop1> <44167223.1080603@strakt.com> <001701c6473e$6b232e50$6a01a8c0@RaymondLaptop1> Message-ID: <4416860F.8090403@strakt.com> Raymond Hettinger wrote: > [Samuele Pedroni] > >> there's no sys.checkinterval in Jython. Implementing this would need the >> introduction of some kind of GIL implementation in Jython, the JVM >> has no primitive for global critical sections. > > > Wouldn't Java implement this directly by suspending and resuming the > other threads (being careful to avoid access to monitored resources > and to pair the suspend/resume operations in a try/finally or > with-statement to prevent deadlocks)? suspending a thread is a deprecated operation because it can cause deadlocks. From barry at python.org Tue Mar 14 14:22:04 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 14 Mar 2006 08:22:04 -0500 Subject: [Python-Dev] Developing/patching ctypes (was: Re: integrating ctypes into python) In-Reply-To: References: <440E9614.3000504@python.net> <4f0b69dc0603080104g6053ae47p59be98f81e65ddad@mail.gmail.com> <440F463E.7050009@python.net> <4f0b69dc0603081535q5e2ef69ft99e339a920dbc25e@mail.gmail.com> <44107267.9090806@python.net> <4f0b69dc0603091623jefdb8fhf4b13695bbb491dc@mail.gmail.com> <44112F50.3020506@python.net> Message-ID: <1142342524.2091.9.camel@geddy.wooz.org> On Mon, 2006-03-13 at 21:38 -0800, Neal Norwitz wrote: > On 3/9/06, Thomas Heller wrote: > > Would it be a solution to move the 'official' ctypes development into > > Python SVN external/ctypes, or would this be considered abuse? Another > > location in SVN could be used as well, if external is though to contain > > only vendor drops... > > Thomas, > > I'd be fine with the official ctypes repo being Python SVN. The sandbox seems a fine place for this. It's what I'm currently doing with the email package. Two of the three versions are actually external'd from Python branches and contain extra stuff to enable standalone releases. The third is being developed first in the sandbox, but will soon be merged back into the trunk and then managed in the same way as the other two. Except for the usual headaches of managing three versions of a package, it's working out quite well. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060314/b9a7e1af/attachment.pgp From ncoghlan at gmail.com Tue Mar 14 14:26:24 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 14 Mar 2006 23:26:24 +1000 Subject: [Python-Dev] Strange behavior in Python 2.5a0 (trunk) --- possible error in AST? In-Reply-To: <4415EFB5.8040200@gmail.com> References: <2mk6ay65h9.fsf@starship.python.net> <4415617A.8030707@gmail.com> <4415EFB5.8040200@gmail.com> Message-ID: <4416C480.9060904@gmail.com> Nick Coghlan wrote: > Unfortunately my new test case breaks test_compiler. I didn't notice because I > didn't use -uall before checking it in :( > > If no-one else gets to it, I'll try to sort it out tonight. OK, as of rev 43025 the compiler module also understands augmented assignment to tuple subscripts, so test_compiler can cope with the new test case. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From abo at minkirri.apana.org.au Tue Mar 14 15:58:58 2006 From: abo at minkirri.apana.org.au (Donovan Baarda) Date: Tue, 14 Mar 2006 14:58:58 +0000 Subject: [Python-Dev] Threading idea -- exposing a global thread lock In-Reply-To: References: <000701c64713$1aaa7fd0$6a01a8c0@RaymondLaptop1> Message-ID: <1142348338.11593.51.camel@warna.dub.corp.google.com> On Mon, 2006-03-13 at 21:06 -0800, Guido van Rossum wrote: > Oh, no! Please! > > I just had to dissuade someone inside Google from the same idea. Heh... that was me... I LOL'ed when I saw this... and no, I didn't put Raymond up to it :-) > IMO it's fatally flawed for several reasons: it doesn't translate > reasonably to Jython or IronPython, it's really tricky to implement, > and it's an invitation for deadlocks. The danger of this thing in the > wrong hands is too big to warrant the (rare) use case that can only be > solved elegantly using direct GIL access. I didn't bother pursuing it because I'm not that attached to it... I'm not sure that a language like Python really needs it, and I don't do that kind of programming much any more. When I did, I was programming in Ada. The Ada language has a global thread-lock used as a primitive to implement all other atomic operations and thread-synchronisation stuff... (it's been a while... this may have been a particular Ada compiler extension, though I think the Ada concurrency model pretty much required it). And before that it was in assembler; an atomic section was done by disabling all interrupts. At that low-level, atomic sections were the building-block for all the other higher level synchronisation tools. I believe the original semaphore relied on an atomic test-and-set operation. The main place where something like this would be useful in Python is in writing thread-safe code that uses non-thread safe resources. Examples are; a chunk of code that redirects then restores sys.stdout, something that changes then restores TZ using time.settz(), etc. I think the deadlock risk argument is bogus... any locking has deadlock risks. The "danger in the wrong hands" I'm also unconvinced about; non-threadsafe resource use worries me far more than a strong lock. I'd rather debug a deadlock than a race condition any day. But the hard to implement for other VMs is a breaker, and suggests there a damn good reasons those VMs disallow it that I haven't thought of :-) So I'm +0, probably -0.5... > --Guido > > On 3/13/06, Raymond Hettinger wrote: > > A user on comp.lang.python has twisted himself into knots writing multi-threaded > > code that avoids locks and queues but fails when running code with non-atomic > > access to a shared resource. While his specific design is somewhat flawed, it > > does suggest that we could offer an easy way to make a block of code atomic > > without the complexity of other synchronization tools: > > > > gil.acquire() > > try: > > #do some transaction that needs to be atomic > > finally: > > gil.release() > > > > The idea is to temporarily suspend thread switches (either using the GIL or a > > global variable in the eval-loop). Think of it as "non-cooperative" > > multi-threading. While this is a somewhat rough approach, it is dramatically > > simpler than the alternatives (i.e. wrapping locks around every access to a > > resource or feeding all resource requests to a separate thread via a Queue). > > > > While I haven't tried it yet, I think the implementation is likely to be > > trivial. > > > > FWIW, the new with-statement makes the above fragment even more readable: > > > > with atomic_transaction(): > > # do a series of steps without interruption > > > > > > Raymond > > > > _______________________________________________ > > Python-Dev mailing list > > Python-Dev at python.org > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/abo%40minkirri.apana.org.au -- Donovan Baarda http://minkirri.apana.org.au/~abo/ From tim.peters at gmail.com Tue Mar 14 16:55:21 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 14 Mar 2006 10:55:21 -0500 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <20060314034831.GE10510@activestate.com> References: <20060314010819.GB29489@activestate.com> <20060314034831.GE10510@activestate.com> Message-ID: <1f7befae0603140755k66ff91c1jb41917d21efb5d41@mail.gmail.com> [Trent Mick] > I have a patch in the works that defaults to "yes, this machine does > have a soundcard" if cscript.exe cannot be found on the PATH. > > However, one wrinkle: test_winsound.py is made up of three test cases: > BeepTest > MessageBeepTest > PlaySoundTest > only the last need be skipped if there is not soundcard. I'd say instead that they should never be skipped: the real difference on your box is the expected _outcome_ in the third category. After umpteen years we've got a universe of one machine where PlaySoundTest is known to fail, and now a little mound of VB code that presumably returns something different on that machine than on other machines. In reality, that's more code to test. We seem to be assuming here that "the VB code says no sound device" means "PlaySoundTest will fail in a particular way", and have one box on which that's known to be true. So sure, skip the tests on that box, and the immediate buildbot failure on that box will go away. Other possiblities include that the test will also be skipped on boxes where it would actually work, because the VB code isn't actually a definitive test for some reason. Since we can't be sure from a universe of one exception, better to test that assumption too, by reworking the tests to say "oh, but if the VB code thinks we don't have a sound card, then this test should raise RuntimeError instead". There's still a testable outcome here. From tim.peters at gmail.com Tue Mar 14 16:57:13 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 14 Mar 2006 10:57:13 -0500 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: References: <20060314010819.GB29489@activestate.com> Message-ID: <1f7befae0603140757g76e5842el46265a803b3698f2@mail.gmail.com> [Mark Hammond] > Maybe the following VBScript "port" of the above will work: > > -- check_soundcard.vbs > rem Check for a working sound-card - exit with 0 if OK, 1 otherwise. > set wmi = GetObject("winmgmts:") > set scs = wmi.InstancesOf("win32_sounddevice") > for each sc in scs > set status = sc.Properties_("Status") > wscript.Echo(sc.Properties_("Name") + "/" + status) > if status = "OK" then > wscript.Quit 0 rem normal exit > end if > next > rem No sound card found - exit with status code of 1 > wscript.Quit 1 > > -- eof > > Running "cscript.exe check_soundcard.vbs" and checking the return > code should work. FYI, "it works" on my main box: C:\Code>cscript.exe csc.vbs Microsoft (R) Windows Script Host Version 5.6 Copyright (C) Microsoft Corporation 1996-2001. All rights reserved. Creative Audigy Audio Processor (WDM)/OK C:\Code>echo %errorlevel% 0 From abo at minkirri.apana.org.au Tue Mar 14 17:05:04 2006 From: abo at minkirri.apana.org.au (Donovan Baarda) Date: Tue, 14 Mar 2006 16:05:04 +0000 Subject: [Python-Dev] Threading idea -- exposing a global thread lock In-Reply-To: <007301c64729$393f7890$6a01a8c0@RaymondLaptop1> References: <000701c64713$1aaa7fd0$6a01a8c0@RaymondLaptop1> <007301c64729$393f7890$6a01a8c0@RaymondLaptop1> Message-ID: <1142352304.11593.64.camel@warna.dub.corp.google.com> On Tue, 2006-03-14 at 00:36 -0500, Raymond Hettinger wrote: > [Guido] > > Oh, no! > > Before shooting this one down, consider a simpler incarnation not involving the > GIL. The idea is to allow an active thread to temporarily suspend switching for > a few steps: [...] > I disagree that the need is rare. My own use case is that I sometimes add some > debugging print statements that need to execute atomically -- it is a PITA > because PRINT_ITEM and PRINT_NEWLINE are two different opcodes and are not > guaranteed to pair atomically. The current RightWay(tm) is for me to create a > separate daemon thread for printing and to send lines to it via the queue module > (even that is tricky because you don't want the main thread to exit before a > print queued item is completed). I suggest that that is too complex for a > simple debugging print statement. It would be great to simply write: You don't need to use queue... that has the potentially nasty side affect of allowing threads to run ahead before their debugging has been output. A better way is to have all your debugging go through a print_debug() method that acquires and releases a debug_lock threading.Lock. This is simpler as it avoids the separate thread, and ensures that threads "pause" until their debugging output is done. -- Donovan Baarda http://minkirri.apana.org.au/~abo/ From jcarlson at uci.edu Tue Mar 14 17:58:10 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 14 Mar 2006 08:58:10 -0800 Subject: [Python-Dev] Threading idea -- exposing a global thread lock In-Reply-To: <4416860F.8090403@strakt.com> References: <001701c6473e$6b232e50$6a01a8c0@RaymondLaptop1> <4416860F.8090403@strakt.com> Message-ID: <20060314083749.F717.JCARLSON@uci.edu> Samuele Pedroni wrote: > > Raymond Hettinger wrote: > > > [Samuele Pedroni] > > > >> there's no sys.checkinterval in Jython. Implementing this would need the > >> introduction of some kind of GIL implementation in Jython, the JVM > >> has no primitive for global critical sections. > > > > > > Wouldn't Java implement this directly by suspending and resuming the > > other threads (being careful to avoid access to monitored resources > > and to pair the suspend/resume operations in a try/finally or > > with-statement to prevent deadlocks)? > > suspending a thread is a deprecated operation because it can cause > deadlocks. There are two assumptions that one can make about code using the "gil", or the equivalent of suspending all threads but the current one, or in Python, just disabling thread switching; I'll call it a (global) 'critical section'. Either the user is going to rely on just the critical section for locking, or the user is going to mix locks too. If the user doesn't mix (even implicitly with Queue, etc.), then there can be no deadlocks caused by the critical section. If the user _is_ mixing standard locks with critical sections, the only _new_ potential cause of deadlocks is if the user attempts to acquire locks within the critical section which have already been acquired by another thread. Deadlocks of this particular type, however, can be generally prevented by making locks aware of critical sections and raising an exception whenever a lock acquisition is taking place within a critical section. You wouldn't want the exception to only be raised if the acquisition would block, as this would result in intermittant errors; just make it an error. It would be nice if Jython or IronPython could (and would) implement these 'critical sections'. Whether they can or not, I think that it would be a useful feature in the CPython runtime. It could be considered a platform-specific feature, similar to how you can use select on any file handle on *nix, but you need to jump through hoops to get a similar thing on Windows. I'm +1, but only because I've spent more than my share of time digging around with threads, locks, Rlocks, conditions, events, etc. - Josiah From mcherm at mcherm.com Tue Mar 14 19:07:40 2006 From: mcherm at mcherm.com (Michael Chermside) Date: Tue, 14 Mar 2006 10:07:40 -0800 Subject: [Python-Dev] Threading idea -- exposing a global thread lock Message-ID: <20060314100740.t0qcxptbek1wkok4@login.werra.lunarpages.com> Josiah Carlson writes: > It would be nice if Jython or IronPython could (and would) implement > these 'critical sections'. Whether they can or not, I think that it > would be a useful feature in the CPython runtime. The issue is not whether Jython and IronPython "will", it's whether they "can". Essentially, Jython and IronPython both use locking within their fundamental data structures. This then allows them to freely allow threads to run on multiple processors. Meanwhile, CPython lacks locks in its fundamental data structures, so it uses the GIL to ensure that code which might touch Python data structures executes on only one CPU at a time. The concept of a "critical section" makes great sense when there is effectively only one CPU: just stop switching threads. But if code is using multiple CPUs, what does it mean? Shut down the other CPUs? To do such a thing cooperatively would require checking some master lock at every step... (a price which CPython pays, but which the implementations built on good thread-supporting VMs don't have to). To do such a thing non-cooperatively is not supported in either VM. Since I doubt we're going to convince Sun or Microsoft to change their approach to threading, I think it is unwise to build such a feature into the Python language. Supporting it in CPython only requires (I think) no more than a very simple C extension. I think it should stay as an extension and not become part of the language. -- Michael Chermside From raymond.hettinger at verizon.net Tue Mar 14 19:30:54 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Tue, 14 Mar 2006 13:30:54 -0500 Subject: [Python-Dev] Threading idea -- exposing a global thread lock References: <20060314100740.t0qcxptbek1wkok4@login.werra.lunarpages.com> Message-ID: <000601c64795$6e9f7140$6a01a8c0@RaymondLaptop1> [Nice analysis from Michael Chermside] > The concept of a "critical section" makes great sense when there is > effectively only one CPU: just stop switching threads. But if code > is using multiple CPUs, what does it mean? Shut down the other CPUs? . . . > I think it is unwise to build such a > feature into the Python language. Supporting it in CPython only > requires (I think) no more than a very simple C extension. I think > it should stay as an extension and not become part of the language. That makes sense. Once place where we already have CPython specific support is in sys.setcheckinterval(). That suggests adapting that function or adding a new one to temporarily stop switching, almost the same as sys.setcheckinterval(sys.maxint) but continuing to perform other periodic checks for control-break and such. Raymond From thomas at python.org Tue Mar 14 19:45:11 2006 From: thomas at python.org (Thomas Wouters) Date: Tue, 14 Mar 2006 19:45:11 +0100 Subject: [Python-Dev] Threading idea -- exposing a global thread lock In-Reply-To: <000601c64795$6e9f7140$6a01a8c0@RaymondLaptop1> References: <20060314100740.t0qcxptbek1wkok4@login.werra.lunarpages.com> <000601c64795$6e9f7140$6a01a8c0@RaymondLaptop1> Message-ID: <9e804ac0603141045l30ea3649lacfade171cdb8a0e@mail.gmail.com> On 3/14/06, Raymond Hettinger wrote: > Once place where we already have CPython specific support is in > sys.setcheckinterval(). That suggests adapting that function or adding a > new > one to temporarily stop switching, almost the same as > sys.setcheckinterval(sys.maxint) but continuing to perform other periodic > checks > for control-break and such. Don't forget that sys.setcheckinterval() is more of a hint than a requirement. It's easy to wrap sys.setcheckinterval() in a try/except or an if-hasattr, and just ignore the case where it doesn't exist ('it won't be necessary'). Doing the same thing with a 'critical section' would be a lot harder. I would also assume a 'critical section' should not allow threads in extension modules, even if they explicitly allow threads. That's quite a bit different from making the check-interval infinitely high for the duration of the block. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060314/9e73c71c/attachment.htm From tim.peters at gmail.com Tue Mar 14 20:21:14 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 14 Mar 2006 14:21:14 -0500 Subject: [Python-Dev] Threading idea -- exposing a global thread lock In-Reply-To: <5.1.1.6.0.20060313230035.03f93610@mail.telecommunity.com> References: <000701c64713$1aaa7fd0$6a01a8c0@RaymondLaptop1> <5.1.1.6.0.20060313230035.03f93610@mail.telecommunity.com> Message-ID: <1f7befae0603141121k29ff8c36xbece20e6d421b1c8@mail.gmail.com> [Raymond Hettinger] >> FWIW, the new with-statement makes the above fragment even more >> readable: >> >> with atomic_transaction(): >> # do a series of steps without interruption [Phillip J. Eby] > +1 on the idea, -1000 on the name. It's neither atomic nor a > transaction. I believe that "critical section" is a more common term for > what you're proposing. No, there is no common term for this idea, no "standard" threading model supports it directly (which is bad news for portability, of course), and it's a Bad Idea to start calling it "critical section" here. There _is_ some variation in what "critical section" means, exactly, to different thread programming cultures, but in none does it mean: a section of code such that, once a thread enters it, all other threads are blocked from doing anything for the duration The common meaning is: a section of code such that, once a thread enters it, all other threads are blocked from entering the section for the duration which is a very far cry from getting blocked from doing anything. In some thread cultures, "critical section" also implies that a thread won't migrate across processors (on a multi-CPU box) while that thread is in a critical section, and that's in addition to the "other threads are blocked from entering the section for the duration" meaning. In some thread cultures, "critical section" isn't distinguished from the obvious implementation in terms of acquiring and releasing a mutex around the code section, but that gets muddy. For example, on Win32 using a native mutex actually implments a cross-*process* "critical section", while the term "critical section" is reserved for cross-thread-within-a-process but not-cross-process mutual exclusion. From pje at telecommunity.com Tue Mar 14 20:30:57 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 14 Mar 2006 14:30:57 -0500 Subject: [Python-Dev] Threading idea -- exposing a global thread lock In-Reply-To: <1f7befae0603141121k29ff8c36xbece20e6d421b1c8@mail.gmail.co m> References: <5.1.1.6.0.20060313230035.03f93610@mail.telecommunity.com> <000701c64713$1aaa7fd0$6a01a8c0@RaymondLaptop1> <5.1.1.6.0.20060313230035.03f93610@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060314142533.01ff21e0@mail.telecommunity.com> At 02:21 PM 3/14/2006 -0500, Tim Peters wrote: >There _is_ some variation in what "critical section" means, exactly, >to different thread programming cultures, but in none does it mean: > > a section of code such that, once a thread enters it, all other > threads are blocked from doing anything for the duration Well, I'm showing my age here, but in the good ol' days of the 8086 processor, I recall it frequently being used to describe a block of assembly code which ran with interrupts disabled - ensuring that no task switching would occur. Obviously I haven't been doing a lot of threaded programming *since* those days, except in Python. :) >The common meaning is: > > a section of code such that, once a thread enters it, all other > threads are blocked from entering the section for the duration That doesn't seem like a very useful definition, since it describes any piece of code that's protected by a statically-determined mutex. But you clearly have more experience in this than I. From tim.peters at gmail.com Tue Mar 14 20:40:47 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 14 Mar 2006 14:40:47 -0500 Subject: [Python-Dev] Threading idea -- exposing a global thread lock In-Reply-To: <007301c64729$393f7890$6a01a8c0@RaymondLaptop1> References: <000701c64713$1aaa7fd0$6a01a8c0@RaymondLaptop1> <007301c64729$393f7890$6a01a8c0@RaymondLaptop1> Message-ID: <1f7befae0603141140t7be4dd9dj14b6c95c4cd3f5c4@mail.gmail.com> [Raymond Hettinger] > ... > I disagree that the need is rare. My own use case is that I sometimes > add some debugging print statements that need to execute > atomically -- it is a PITA because PRINT_ITEM and PRINT_NEWLINE > are two different opcodes and are not guaranteed to pair atomically. Well, it's much worse than that, right? If you have a print list with N items, there are N PRINT_ITEM opcodes. > The current RightWay(tm) is for me to create a separate daemon > thread for printing and to send lines to it via the queue module > (even that is tricky because you don't want the main thread to exit > before a print queued item is completed). I suggest that that is too > complex for a simple debugging print statement. It sure is. You're welcome to use my thread-safe debug-print function :-): def msg(fmt, *args): s = fmt % args + '\n' for stream in sys.stdout, logfile: stream.write(s) stream.flush() I use that for long-running (days) multi-threaded apps, where I want to see progress messages on stdout but save them to a log file too. It assumes that the underlying C library writes a single string atomically. If I couldn't assume that, it would be easy to acquire/release a lock inside the function. For example, as-is the order of lines displayed on stdout isn't always exactly the same as the order in the log file, and when I care about that (I rarely do) adding a lock can make it deterministic. I also have minor variants of that function, some that prepend a timestamp to each message, and/or prepend the id or name of the current thread. Because all such decisions are hiding inside the msg() function, it's very esay to change the debug output as needed. Or to do def msg(*args): pass when I don't want to see output at all. From guido at python.org Tue Mar 14 21:26:35 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 14 Mar 2006 12:26:35 -0800 Subject: [Python-Dev] Threading idea -- exposing a global thread lock In-Reply-To: <5.1.1.6.0.20060314142533.01ff21e0@mail.telecommunity.com> References: <000701c64713$1aaa7fd0$6a01a8c0@RaymondLaptop1> <5.1.1.6.0.20060313230035.03f93610@mail.telecommunity.com> <5.1.1.6.0.20060314142533.01ff21e0@mail.telecommunity.com> Message-ID: On 3/14/06, Phillip J. Eby wrote: > At 02:21 PM 3/14/2006 -0500, Tim Peters wrote: > >The common meaning is: > > > > a section of code such that, once a thread enters it, all other > > threads are blocked from entering the section for the duration > > That doesn't seem like a very useful definition, since it describes any > piece of code that's protected by a statically-determined mutex. But you > clearly have more experience in this than I. Trust Tim. That's what "critical section" means in most places. And yes, indeed, a static mutex is the obvious way to implement it. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.peters at gmail.com Tue Mar 14 21:33:43 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 14 Mar 2006 15:33:43 -0500 Subject: [Python-Dev] Threading idea -- exposing a global thread lock In-Reply-To: <5.1.1.6.0.20060314142533.01ff21e0@mail.telecommunity.com> References: <000701c64713$1aaa7fd0$6a01a8c0@RaymondLaptop1> <5.1.1.6.0.20060313230035.03f93610@mail.telecommunity.com> <5.1.1.6.0.20060314142533.01ff21e0@mail.telecommunity.com> Message-ID: <1f7befae0603141233m5557c9a8ic9d2bed070048a39@mail.gmail.com> [Phillip J. Eby] > Well, I'm showing my age here, but in the good ol' days of the 8086 > processor, I recall it frequently being used to describe a block of > assembly code which ran with interrupts disabled - ensuring that no task > switching would occur. According to Wikipedia's current article on "critical section" (which is pretty good!), that still may be common usage for kernel-level programmers. "The rest of us" don't get to run privileged instructions anymore, and in Wikipedia terms I'm talking about what they call "application level" critical sections: http://en.wikipedia.org/wiki/Critical_section A little Googling confirms that almost everyone has the application-level sense in mind these days. > Obviously I haven't been doing a lot of threaded programming *since* > those days, except in Python. :) And for all the whining about it, threaded programming in Python is both much easier than elsewhere, _and_ still freaking hard ;-) >> The common meaning is: >> >> a section of code such that, once a thread enters it, all other >> threads are blocked from entering the section for the duration > That doesn't seem like a very useful definition, since it describes any > piece of code that's protected by a statically-determined mutex. But you > clearly have more experience in this than I. As I tried to explain the first time, a mutex is a common implementation technique, but even saying "mutex" doesn't define the semantics (see the original msg for one distinction between cross-thread and cross-process exclusion). There are ways to implement critical sections other than via a mutex. The "common meaning" I gave above tries to describe the semantics (visible behavior), not an implementation. A Python-level lock is an obvious and straightforward way to implement those semantics. From lists at janc.be Tue Mar 14 21:44:16 2006 From: lists at janc.be (Jan Claeys) Date: Tue, 14 Mar 2006 21:44:16 +0100 Subject: [Python-Dev] Topic suggestions from the PyCon feedback In-Reply-To: References: <20060314032219.GA3629@rogue.amk.ca> Message-ID: <1142369057.12100.80.camel@localhost.localdomain> Op ma, 13-03-2006 te 19:52 -0800, schreef Alex Martelli: > The *ONE* thing I dislike about working in the US is vacations -- I > get about half of what I would expect in Europe, and that's with my > employer being reasonably generous... in practice, given I NEED some > time to go visit family and friends back in Italy, this means I can't > really take vacations to do conferences, but rather I must convince > my boss that conference X is worth my time [...]. Well, I'm sure (from a previous c.l.py encounter) that you know enough about (European) civilisation to explain history to an American savage (~= employer)? ;-) -- Jan Claeys From martin at v.loewis.de Tue Mar 14 22:01:44 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 14 Mar 2006 22:01:44 +0100 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <1f7befae0603140755k66ff91c1jb41917d21efb5d41@mail.gmail.com> References: <20060314010819.GB29489@activestate.com> <20060314034831.GE10510@activestate.com> <1f7befae0603140755k66ff91c1jb41917d21efb5d41@mail.gmail.com> Message-ID: <44172F38.1090909@v.loewis.de> Tim Peters wrote: > I'd say instead that they should never be skipped: the real > difference on your box is the expected _outcome_ in the third > category. That is indeed more reasonable than what I proposed. Regards, Martin From martin at v.loewis.de Tue Mar 14 22:17:10 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 14 Mar 2006 22:17:10 +0100 Subject: [Python-Dev] Threading idea -- exposing a global thread lock In-Reply-To: <000601c64795$6e9f7140$6a01a8c0@RaymondLaptop1> References: <20060314100740.t0qcxptbek1wkok4@login.werra.lunarpages.com> <000601c64795$6e9f7140$6a01a8c0@RaymondLaptop1> Message-ID: <441732D6.9090607@v.loewis.de> Raymond Hettinger wrote: > Once place where we already have CPython specific support is in > sys.setcheckinterval(). That suggests adapting that function or adding a new > one to temporarily stop switching, almost the same as > sys.setcheckinterval(sys.maxint) but continuing to perform other periodic checks > for control-break and such. I object to the notion that Python does "thread switching". It doesn't. Instead, it releases the GIL under certain circumstances, which might make the operating system switch threads. Whether the operating system does that is its own choice. I don't see a value to disabling the "release the GIL from time to time". This does *not* mean there couldn't be thread-switching, anymore. E.g. inside a PRINT_* opcode, thread-switching may still occur, as file_write releases the GIL around the fwrite() call. So if you set the checkinterval to "no check", you cannot trust that there won't be any thread switching. Regards, Martin From raymond.hettinger at verizon.net Tue Mar 14 22:33:06 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Tue, 14 Mar 2006 16:33:06 -0500 Subject: [Python-Dev] Another threading idea Message-ID: <002301c647ae$e522c2e0$6a01a8c0@RaymondLaptop1> FWIW, I've been working on a way to simplify the use of queues with daemon consumer threads Sometimes, I launch one or more consumer threads that wait for a task to enter a queue and then work on the task. A recurring problem is that I sometimes need to know if all of the tasks have been completed so I can exit or do something with the result. If each thread only does a single task, I can use t.join() to wait until the task is done. However, if the thread stays alive and waits for more Queue entries, then there doesn't seem to be a good way to tell when all the processing is done. So, the idea is to create a subclass of Queue that increments a counter when objects are enqueued, that provides a method for worker threads to decrement the counter when the work is done, and offers a blocking join() method that waits until the counter is zero # main thread q = TaskQueue() for t in worker_threads(): t.start() for task in tasklist: q.put(task) # increments the counter and enqueues a task q.join() # all of the tasks are done (counter is zero) do_work_on_results() # worker thread while 1: task = q.get() # task is popped but the counter is unchanged do_work(task) q.decrement() # now the counter gets reduced The idea is still in its infancy (no implementation and it hasn't been tried in real-world code) but I would like to get some feedback. If it works out, I'll post a recipe to ASPN and see how it goes. Raymond From 2005a at usenet.alexanderweb.de Tue Mar 14 22:42:15 2006 From: 2005a at usenet.alexanderweb.de (Alexander Schremmer) Date: Tue, 14 Mar 2006 22:42:15 +0100 Subject: [Python-Dev] Threading idea -- exposing a global thread lock References: <000701c64713$1aaa7fd0$6a01a8c0@RaymondLaptop1> Message-ID: <1fsqsdav3cqgj$.dlg@usenet.alexanderweb.de> On Mon, 13 Mar 2006 21:57:59 -0500, Raymond Hettinger wrote: > Think of it as "non-cooperative" > multi-threading. While this is a somewhat rough approach, it is dramatically > simpler than the alternatives (i.e. wrapping locks around every access to a > resource or feeding all resource requests to a separate thread via a Queue). Why is that actually more difficult to write? Consider res_lock = Lock() res = ... with locked(res_lock): do_something(res) It is only about supplying the correct lock at the right time. Or even this could work: res = ... # implements lock()/unlock() with locked(res): do_something(res) Directly exposing the GIL (or some related system) for such matters does not seem to be a good reason for a novice to let him stop all threads. Kind regards, Alexander From 2005a at usenet.alexanderweb.de Tue Mar 14 22:48:24 2006 From: 2005a at usenet.alexanderweb.de (Alexander Schremmer) Date: Tue, 14 Mar 2006 22:48:24 +0100 Subject: [Python-Dev] About "Coverity Study Ranks LAMP Code Quality" References: <44158A53.90909@nuxeo.com> <20060313230507.GA18532@unpythonic.net> <44160688.3000209@v.loewis.de> Message-ID: On Tue, 14 Mar 2006 00:55:52 +0100, "Martin v. L?wis" wrote: > I can understand that position. The bugs they find include potential > security flaws, for which exploits could be created if the results are > freely available. On the other hand, the exploit could be crafted based on reading the SVN check-ins ... Kind regards, Alexander From trentm at ActiveState.com Tue Mar 14 22:55:50 2006 From: trentm at ActiveState.com (Trent Mick) Date: Tue, 14 Mar 2006 13:55:50 -0800 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <44172F38.1090909@v.loewis.de> References: <20060314010819.GB29489@activestate.com> <20060314034831.GE10510@activestate.com> <1f7befae0603140755k66ff91c1jb41917d21efb5d41@mail.gmail.com> <44172F38.1090909@v.loewis.de> Message-ID: <20060314215550.GC5358@activestate.com> [Martin v. Loewis wrote] > Tim Peters wrote: > > I'd say instead that they should never be skipped: the real > > difference on your box is the expected _outcome_ in the third > > category. > > That is indeed more reasonable than what I proposed. I'll do this tonight or tomorrow. Trent -- Trent Mick TrentM at ActiveState.com From guido at python.org Tue Mar 14 23:08:09 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 14 Mar 2006 14:08:09 -0800 Subject: [Python-Dev] Another threading idea In-Reply-To: <002301c647ae$e522c2e0$6a01a8c0@RaymondLaptop1> References: <002301c647ae$e522c2e0$6a01a8c0@RaymondLaptop1> Message-ID: Isn't this a job for threading.BoundedSpemaphore()? On 3/14/06, Raymond Hettinger wrote: > FWIW, I've been working on a way to simplify the use of queues with daemon > consumer threads > > Sometimes, I launch one or more consumer threads that wait for a task to enter a > queue and then work on the task. A recurring problem is that I sometimes need to > know if all of the tasks have been completed so I can exit or do something with > the result. > > If each thread only does a single task, I can use t.join() to wait until the > task is done. However, if the thread stays alive and waits for more Queue > entries, then there doesn't seem to be a good way to tell when all the > processing is done. > > So, the idea is to create a subclass of Queue that increments a counter when > objects are enqueued, that provides a method for worker threads to decrement the > counter when the work is done, and offers a blocking join() method that waits > until the counter is zero > > # main thread > q = TaskQueue() > for t in worker_threads(): > t.start() > for task in tasklist: > q.put(task) # increments the counter and enqueues a task > q.join() # all of the tasks are done (counter is > zero) > do_work_on_results() > > > > # worker thread > while 1: > task = q.get() # task is popped but the counter is > unchanged > do_work(task) > q.decrement() # now the counter gets reduced > > > The idea is still in its infancy (no implementation and it hasn't been tried in > real-world code) but I would like to get some feedback. If it works out, I'll > post a recipe to ASPN and see how it goes. > > > Raymond > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From raymond.hettinger at verizon.net Tue Mar 14 23:10:17 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Tue, 14 Mar 2006 17:10:17 -0500 Subject: [Python-Dev] Threading idea -- exposing a global thread lock References: <000701c64713$1aaa7fd0$6a01a8c0@RaymondLaptop1> <1fsqsdav3cqgj$.dlg@usenet.alexanderweb.de> Message-ID: <005801c647b4$142d5b40$6a01a8c0@RaymondLaptop1> [Raymond] >> While this is a somewhat rough approach, it is dramatically >> simpler than the alternatives (i.e. wrapping locks around every access to a >> resource or feeding all resource requests to a separate thread via a Queue). [Alexander] > Why is that actually more difficult to write? Consider > > res_lock = Lock() > res = ... > with locked(res_lock): > do_something(res) > > It is only about supplying the correct lock at the right time. In the case on the newsgroup, the resource was a mapping or somesuch. Getitem and setitem accesses were spread throughout the program and it would have been a mess to put locks everywhere. All he wanted was to simply freeze task-switching for a moment so he could loop over the mapping and have it be in a consistent state from the start of the loop to the end. His situation was further complicated because the looping construct was buried in library code which used iterkeys() instead of keys() -- IOW, the code bombed if the mapping changed during iteration While the guy had an odd design, the generalization was clear. Sometimes you want to make one little section uninterruptable and don't want to put locks around everything that touches a resource. If the resource access occurs in fifty places throughout your code and you only need one little section to have uninterrupted access, then the lock approach requires way too much effort. Further, if you miss putting locks around any one of the accesses, you lose reliability. Also, with locks scatterred all over the place, it is not easy to tell at a glance that you haven't introduced the possibility of dead-lock. In contrast, with the temporary suspension of thread-switching, it is pretty easy to look inside the uninterruptable block to make sure that none of the statements block. Why make life unnecessarily hard. Raymond From p.f.moore at gmail.com Tue Mar 14 23:10:47 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 14 Mar 2006 22:10:47 +0000 Subject: [Python-Dev] Another threading idea In-Reply-To: <002301c647ae$e522c2e0$6a01a8c0@RaymondLaptop1> References: <002301c647ae$e522c2e0$6a01a8c0@RaymondLaptop1> Message-ID: <79990c6b0603141410j145bc120ibfb473195044352d@mail.gmail.com> On 3/14/06, Raymond Hettinger wrote: > FWIW, I've been working on a way to simplify the use of queues with daemon > consumer threads > > Sometimes, I launch one or more consumer threads that wait for a task to enter a > queue and then work on the task. A recurring problem is that I sometimes need to > know if all of the tasks have been completed so I can exit or do something with > the result. [...] > So, the idea is to create a subclass of Queue that increments a counter when > objects are enqueued, that provides a method for worker threads to decrement the > counter when the work is done, and offers a blocking join() method that waits > until the counter is zero I've also hit this problem, and would find this pattern useful. FWIW, in my code, I bypassed the problem by spawning one worker per task, and waiting on them all. This works, but is sub-optimal (there's no need for 100+ subthreads, when 20 or so generic workers would have done...) Paul. From p.f.moore at gmail.com Tue Mar 14 23:28:26 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 14 Mar 2006 22:28:26 +0000 Subject: [Python-Dev] Another threading idea In-Reply-To: References: <002301c647ae$e522c2e0$6a01a8c0@RaymondLaptop1> Message-ID: <79990c6b0603141428i2108026dk96f314316b135773@mail.gmail.com> On 3/14/06, Guido van Rossum wrote: > Isn't this a job for threading.BoundedSpemaphore()? Not sure I see how. What I think Raymond's after (and certainly what I want) is to queue N tasks, set a counter to N, then wait until the counter goes to zero. I suppose counter = Semaphore(-N) # Queue N tasks counter.acquire() with each task (or the queue) saying counter.release() when it finishes. But the logic seems backwards, and highly prone to off-by-one errors, and I'm not entirely convinced that a negative semaphore value is officially supported... (BoundedSemaphore seems a red herring here - the blocking semantics of Semaphore and BoundedSemaphore are identical). Paul. From raymond.hettinger at verizon.net Tue Mar 14 23:33:19 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Tue, 14 Mar 2006 17:33:19 -0500 Subject: [Python-Dev] Another threading idea References: <002301c647ae$e522c2e0$6a01a8c0@RaymondLaptop1> Message-ID: <008d01c647b7$4bb30ad0$6a01a8c0@RaymondLaptop1> > Isn't this a job for threading.BoundedSpemaphore()? I don't see how that would work. ISTM that we need an inverse of a BoundedSemaphore. If it understand it correctly, a BS blocks after some pre-set maximum number of acquires and is used for resources with limited capacity (i.e. a number of connections that can be served). With the TaskQueue, there is no pre-set number, the queue can grow to any size, and the join() method will block until the counter falls back to zero. IOW, a BS is about potentially blocking new requests and a TaskQueue is about blocking other work until outstanding requests are complete. Raymond From tim.peters at gmail.com Wed Mar 15 00:07:13 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 14 Mar 2006 18:07:13 -0500 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <20060314215550.GC5358@activestate.com> References: <20060314010819.GB29489@activestate.com> <20060314034831.GE10510@activestate.com> <1f7befae0603140755k66ff91c1jb41917d21efb5d41@mail.gmail.com> <44172F38.1090909@v.loewis.de> <20060314215550.GC5358@activestate.com> Message-ID: <1f7befae0603141507u7e7f45eet68fdaaa453c8f94e@mail.gmail.com> [Trent Mick, on test_winsound] > I'll do this tonight or tomorrow. Cool! I see that your Win2K buildbot slave always dies in the compile step now, with """ ------ Build started: Project: pythoncore, Configuration: Debug Win32 ------ Compiling resources... generate buildinfo cl.exe -c -D_WIN32 -DUSE_DL_EXPORT -D_WINDOWS -DWIN32 -D_WINDLL -D_DEBUG -MDd ..\Modules\getbuildinfo.c -Fogetbuildinfo.o -I..\Include -I..\PC Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077 for 80x86 Copyright (C) Microsoft Corporation 1984-2002. All rights reserved. getbuildinfo.c Linking... LINK : fatal error LNK1104: cannot open file './python25_d.dll' """ That happened to me once, but I still don't understand it. It turned out that the corresponding python_d.exe was still running (for hours, and hours, and hours, ...), and I had to manually kill the process. I'm not sure that was enough, because I coincidentally rebooted the box before the buildbot tests ran again. I am pretty sure that the symptom above won't fix itself. Possibly related: since we upgraded to a new bsddb (and this may be coincidence), I've seen two failure modes in test_shelve: test_bool (which is the first test) never completes, and test_bool does complete but fails. Turns out both are rare failure modes, and they haven't happened again since I prepared myself to dig into them <0.5 wink>. From guido at python.org Wed Mar 15 00:09:46 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 14 Mar 2006 15:09:46 -0800 Subject: [Python-Dev] Another threading idea In-Reply-To: <008d01c647b7$4bb30ad0$6a01a8c0@RaymondLaptop1> References: <002301c647ae$e522c2e0$6a01a8c0@RaymondLaptop1> <008d01c647b7$4bb30ad0$6a01a8c0@RaymondLaptop1> Message-ID: I think I was thinking of the following: create a semaphore set to zero; the main thread does N acquire operations; each of N workers releases it once after it's done. When the main thread proceeds it knows all workers are done. Doesn't that work? Also, I believe Tim once implemented a barrier lock but I can't find it right now. --Guido On 3/14/06, Raymond Hettinger wrote: > > Isn't this a job for threading.BoundedSpemaphore()? > > I don't see how that would work. ISTM that we need an inverse of a > BoundedSemaphore. If it understand it correctly, a BS blocks after some pre-set > maximum number of acquires and is used for resources with limited capacity (i.e. > a number of connections that can be served). With the TaskQueue, there is no > pre-set number, the queue can grow to any size, and the join() method will block > until the counter falls back to zero. IOW, a BS is about potentially blocking > new requests and a TaskQueue is about blocking other work until outstanding > requests are complete. > > > Raymond > > > > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.peters at gmail.com Wed Mar 15 00:24:56 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 14 Mar 2006 18:24:56 -0500 Subject: [Python-Dev] Another threading idea In-Reply-To: <002301c647ae$e522c2e0$6a01a8c0@RaymondLaptop1> References: <002301c647ae$e522c2e0$6a01a8c0@RaymondLaptop1> Message-ID: <1f7befae0603141524s6e8f3ff6u161772334b3125ee@mail.gmail.com> [Raymond Hettinger] > FWIW, I've been working on a way to simplify the use of queues with > daemon consumer threads > > Sometimes, I launch one or more consumer threads that wait for a task > to enter a queue and then work on the task. A recurring problem is that > I sometimes need to know if all of the tasks have been completed so I > can exit or do something with the result. FWIW, instead of: # main thread q = TaskQueue() for t in worker_threads(): t.start() for task in tasklist: q.put(task) # increments the counter and enqueues a task q.join() # all of the tasks are done (counter is zero) do_work_on_results() I've sometimes used a separate "work finished" queue, like so: # main thread q = TaskQueue() finished = Queue.Queue() for t in worker_threads(): t.start() for task in tasklist: q.put(task) for task in tasklist: finished.get() do_work_on_results() When a worker thread is done with a task, it simply does: finished.put(None) # `None` can just as well be 42 or "done" No explicit count is needed, although it's easy to add one if desired. The only trick is for the main thread to insist on doing finished.get() exactly as many times as it does q.put(). This is easy and natural -- once you've seen it :-) BTW, it's more common for me to put something meaningful on the `finished` queue, as my main thread often wants to accumulate some info about the outcomes of tasks. From greg.ewing at canterbury.ac.nz Wed Mar 15 00:12:42 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 15 Mar 2006 12:12:42 +1300 Subject: [Python-Dev] About "Coverity Study Ranks LAMP Code Quality" In-Reply-To: References: <44158A53.90909@nuxeo.com> <4415FBA2.5090308@canterbury.ac.nz> Message-ID: <44174DEA.2020205@canterbury.ac.nz> Fredrik Lundh wrote: > return=NULL; output=junk => out of memory > return=junk; output=-1 => cannot do this > return=pointer; output=value => did this, returned value bytes > I agree that the design is a bit questionable; It sure is. If you get both NULL and -1 returned, how are you supposed to know which one is the junk? Greg From guido at python.org Wed Mar 15 00:30:05 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 14 Mar 2006 15:30:05 -0800 Subject: [Python-Dev] Py3k branch - please stay out :-) Message-ID: So I created a Py3K branch in subversion. (Due to my slippery fingers it's actually called p3yk -- that's fine, it may keep bystanders out, and it means we can rename it to the proper name when it's more ready for public consumption. :-) My current plans for this branch are simple: I'm going to rip out some obvious stuff (string exceptions, classic classes) and get a "feel" for what Python 3000 might look like. I'm not particularly looking for help -- if all goes well this is going to be my personal hobby project for the next few months, and then we'll see where we stand. I promised OSCON a keynote on Python 3000 so that's a convenient deadline. In other news, I'd like to nominate Neal Norwitz as the Python 2.5 "release coordinator". He's already doing a great job doing exactly what I think a coordinator should be doing. Anthony will remain release manager, Tim, Martin, Fred and others will do their stuff; but Neal can be the public face. He and Anthony should probably get together on IM and decide on the actual release schedule. For all Python 2.5 issues, please look to Neal. Also, if your proposal doesn't already have a PEP number, it shouldn't be going into Python 2.5. It's time to start releasing and stop evolving for a few months... -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Wed Mar 15 01:08:14 2006 From: brett at python.org (Brett Cannon) Date: Tue, 14 Mar 2006 16:08:14 -0800 Subject: [Python-Dev] Py3k branch - please stay out :-) In-Reply-To: References: Message-ID: On 3/14/06, Guido van Rossum wrote: [SNIP] > In other news, I'd like to nominate Neal Norwitz as the Python 2.5 > "release coordinator". He's already doing a great job doing exactly > what I think a coordinator should be doing. Anthony will remain > release manager, Tim, Martin, Fred and others will do their stuff; but > Neal can be the public face. He and Anthony should probably get > together on IM and decide on the actual release schedule. For all > Python 2.5 issues, please look to Neal. Also, if your proposal doesn't > already have a PEP number, it shouldn't be going into Python 2.5. It's > time to start releasing and stop evolving for a few months... +1 for Neal being the release manager. -Brett From trentm at ActiveState.com Wed Mar 15 03:17:34 2006 From: trentm at ActiveState.com (Trent Mick) Date: Tue, 14 Mar 2006 18:17:34 -0800 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <1f7befae0603141507u7e7f45eet68fdaaa453c8f94e@mail.gmail.com> References: <20060314010819.GB29489@activestate.com> <20060314034831.GE10510@activestate.com> <1f7befae0603140755k66ff91c1jb41917d21efb5d41@mail.gmail.com> <44172F38.1090909@v.loewis.de> <20060314215550.GC5358@activestate.com> <1f7befae0603141507u7e7f45eet68fdaaa453c8f94e@mail.gmail.com> Message-ID: <20060315021734.GA21778@activestate.com> [Tim Peters wrote] >... > I see that your Win2K buildbot slave always dies in the compile step now, with > > """ > ------ Build started: Project: pythoncore, Configuration: Debug Win32 ------ > > Compiling resources... > generate buildinfo > cl.exe -c -D_WIN32 -DUSE_DL_EXPORT -D_WINDOWS -DWIN32 -D_WINDLL > -D_DEBUG -MDd ..\Modules\getbuildinfo.c -Fogetbuildinfo.o -I..\Include > -I..\PC > Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077 for 80x86 > Copyright (C) Microsoft Corporation 1984-2002. All rights reserved. > getbuildinfo.c > Linking... > LINK : fatal error LNK1104: cannot open file './python25_d.dll' > """ > > That happened to me once, but I still don't understand it. It turned > out that the corresponding python_d.exe was still running (for hours, > and hours, and hours, ...), and I had to manually kill the process. > I'm not sure that was enough, because I coincidentally rebooted the > box before the buildbot tests ran again. I am pretty sure that the > symptom above won't fix itself. Yes I've noticed it too. I've had to kill python_d.exe a few times. I haven't yet had the chance to look into it. I am NOT getting this error on another Windows Python build slave that I am running in-house for play. Trent -- Trent Mick TrentM at ActiveState.com From anthony at interlink.com.au Wed Mar 15 05:44:13 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed, 15 Mar 2006 15:44:13 +1100 Subject: [Python-Dev] Topic suggestions from the PyCon feedback In-Reply-To: <20060314032219.GA3629@rogue.amk.ca> References: <20060314032219.GA3629@rogue.amk.ca> Message-ID: <200603151544.23281.anthony@interlink.com.au> On Tuesday 14 March 2006 14:22, A.M. Kuchling wrote: > The conclusion I draw from these results: intermediate- or > advanced-level topics of program design are not covered enough, > whether in the Python documentation, in published books and > articles, or in PyCon talks. Please feel free to mine the above > list, or the rest of the PyCon feedback, for topic ideas. > > In particular: if you're going to attend PyCon 2007, EuroPython, or > some other conference (even a non-Python one), please consider > submitting a talk proposal covering one of the above topics. Such > presentations would find a receptive audience, I think. Just as another data point - at OSDC (australian open source conference) I've presented a "What's New In Python" talk the two years the conference has run, and it's gotten good responses from the audience. It's fairly brief race through the world - only 1/2 an hour - but I try to hit all the good points. -- Anthony Baxter It's never too late to have a happy childhood. From guido at python.org Wed Mar 15 05:48:22 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 14 Mar 2006 20:48:22 -0800 Subject: [Python-Dev] Octal literals In-Reply-To: References: <43E47DBC.9000703@v.loewis.de> <43e60b15.1685113@news.gmane.org> <20060205091611.10F7.JCARLSON@uci.edu> <43e657a1.21281501@news.gmane.org> <43e6bfbc.47932262@news.gmane.org> <20060206080501.GA10226@xs4all.nl> <43e75b8e.87822602@news.gmane.org> Message-ID: (I'm shedding load; cleaning up my inbox in preparation for moving on to Py3K. I'll try to respond to some old mail in the process.) On 2/6/06, Alex Martelli wrote: > Essentially, you need to decide: does type(x) mostly refer to the > protocol that x respects ("interface" plus semantics and pragmatics), > or to the underlying implementation? If the latter, as your > observation about "the philosophy" suggests, then it would NOT be nice > if int was an exception wrt other types. > > If int is to be a concrete type, then I'd MUCH rather it didn't get > subclassed, for all sorts of both pratical and principled reasons. > So, to me, the best solution would be the abstract base class with > concrete implementation subclasses. Besides being usable for > isinstance checks, like basestring, it should also work as a factory > when called, returning an instance of the appropriate concrete > subclass. I like this approach, and I'd like to make it happen. (Not tomorrow. :-) > AND it would let me have (part of) what I was pining for a > while ago -- an abstract base class that type gmpy.mpz can subclass to > assert "I _am_ an integer type!", so lists will accept mpz instances > as indices, etc etc. I'm still dead set against this. Using type checks instead of interface checks is too big a deviation from the language's philosophy. It would be the end of duck typing as we know it! Using __index__ makes much more sense to me. > Now consider how nice it would be, on occasion, to be able to operate > on an integer that's guaranteed to be 8, 16, 32, or 64 bits, to > ensured the desired shifting/masking behavior for certain kinds of > low-level programming; and also on one that's unsigned, in each of > these sizes. Python could have a module offering signed8, unsigned16, > and so forth (all combinations of size and signedness supported by the > underlying C compiler), all subclassing the abstract int, and > guarantee much happiness to people who are, for example, writing a > Python prototype of code that's going to become C or assembly... Why should these have to subclass int? They behave quite differently! I still don't see the incredible value of such types compared to simply doing standard arithmetic and adding "& 0xFF" or "& 0xFFFF" at the end, etc. (Slightly more complicated for signed arithmetic, but who really wants signed clipped arithmetic except if you're simulating a microprocessor?) You can write these things in Python 2.5, and as long as they implement __index__ and do their own mixed-mode arithmetic when combined with regular int or long, all should well. (BTW a difficult design choice may be: if an int8 and an int meet, should the result be an int8 or an int?) > Similarly, it would help a slightly different kind of prototyping a > lot if another Python module could offer 32-bit, 64-bit, 80-bit and > 128-bit floating point types (if supported by the underlying C > compiler) -- all subclassing an ABSTRACT 'float'; the concrete > implementation that one gets by calling float or using a float literal > would also subclass it... and so would the decimal type (why not? it's > floating point -- 'float' doesn't mean 'BINARY fp';-). And I'd be > happy, because gmpy.mpf could also subclass the abstract float! I'd like concrete indications that the implementation of such a module runs into serious obstacles with the current approach. I'm not aware of any, apart from the occasional isinstance(x, float) check in the standard library. If that's all you're fighting, perhaps those occurrences should be fixed? They violate duck typing. > And then finally we could have an abstract superclass 'number', whose > subclasses are the abstract int and the abstract float (dunno 'bout > complex, I'd be happy either way), and Python's typesystem would > finally start being nice and cleanly organized instead of > grand-prarie-level flat ...!-) I think you can have families of numbers separate from subclassing relationships. I'm not at all sure that subclassing doesn't create more problems than it solves here. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.peters at gmail.com Wed Mar 15 06:00:06 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 15 Mar 2006 00:00:06 -0500 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <20060315021734.GA21778@activestate.com> References: <20060314010819.GB29489@activestate.com> <20060314034831.GE10510@activestate.com> <1f7befae0603140755k66ff91c1jb41917d21efb5d41@mail.gmail.com> <44172F38.1090909@v.loewis.de> <20060314215550.GC5358@activestate.com> <1f7befae0603141507u7e7f45eet68fdaaa453c8f94e@mail.gmail.com> <20060315021734.GA21778@activestate.com> Message-ID: <1f7befae0603142100r6f9e0dddr1ad3e5be4e2a669f@mail.gmail.com> [Trent Mick] > Yes I've noticed it too. I've had to kill python_d.exe a few times. I > haven't yet had the chance to look into it. I am NOT getting this error > on another Windows Python build slave that I am running in-house for > play. The last run on your Win2K slave that got beyond the compile step: http://www.python.org/dev/buildbot/all/x86%20W2k%20trunk/builds/16/step-test/0 Looks like it was running test_bsddb at the time, and the test framework gave up after waiting 20 minutes for more output. I had one of those "recently" that waited 20 minutes for output after starting test_shelve, but it's scrolled off the page. Berkeley DB is fishy. Looks like the buildbot doesn't know how to kill a process on Windows either (SIGKILL sure ain't gonna do it ;-)). The good news is that at least we're not seeing the random segfaults plaguing the Mac slave :-) From selliott4 at austin.rr.com Wed Mar 15 06:07:27 2006 From: selliott4 at austin.rr.com (Steven Elliott) Date: Tue, 14 Mar 2006 23:07:27 -0600 Subject: [Python-Dev] Making builtins more efficient In-Reply-To: <007c01c64399$af03ffd0$a600000a@RaymondLaptop1> References: <1141879691.11091.78.camel@grey> <007c01c64399$af03ffd0$a600000a@RaymondLaptop1> Message-ID: <1142399247.11091.229.camel@grey> On Thu, 2006-03-09 at 08:51 -0800, Raymond Hettinger wrote: > [Steven Elliott] > > As you probably know each access of a builtin requires two hash table > > lookups. First, the builtin is not found in the list of globals. It is > > then found in the list of builtins. > > If someone really cared about the double lookup, they could flatten a level by > starting their modules with: > > from __builtin__ import * > > However, we don't see people writing this kind of code. That could mean that > the double lookup hasn't been a big concern. It could mean that. I think what you are suggesting is sufficiently cleaver that the average Python coder may not have thought of it. In any case, many people are willing to do "while 1" instead of "while True" to avoid the double lookup. And the "from __builtin__ import *" additionally imposes a startup cost and memory cost (at least a word per builtin, I would guess). > > Why not have a means of referencing the default builtins with some sort > > of index the way the LOAD_FAST op code currently works? > > FWIW, there was a PEP proposing a roughly similar idea, but the PEP encountered > a great deal of resistance: > > http://www.python.org/doc/peps/pep-0329/ > > When it comes time to write your PEP, it would helpful to highlight how it > differs from PEP 329 (i.e. implemented through the compiler rather than as a > bytecode hack, etc.). I'm flattered that you think it might be worthy of a PEP. I'll look into doing that. > > Perhaps what I'm suggesting isn't feasible for reasons that have already > > been discussed. But it seems like it should be possible to make "while > > True" as efficient as "while 1". > > That is going to be difficult as long as it is legal to write: > > True = 0 "LOAD_BUILTIN" (or whatever we want to call it) should be as fast as "LOAD_FAST" (locals) or "LOAD_CONST" in that they each index into an array where the index is the argument to the opcode. I'll look into writing a PEP. -- ----------------------------------------------------------------------- | Steven Elliott | selliott4 at austin.rr.com | ----------------------------------------------------------------------- From tim.peters at gmail.com Wed Mar 15 06:13:12 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 15 Mar 2006 00:13:12 -0500 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <1f7befae0603142100r6f9e0dddr1ad3e5be4e2a669f@mail.gmail.com> References: <20060314010819.GB29489@activestate.com> <20060314034831.GE10510@activestate.com> <1f7befae0603140755k66ff91c1jb41917d21efb5d41@mail.gmail.com> <44172F38.1090909@v.loewis.de> <20060314215550.GC5358@activestate.com> <1f7befae0603141507u7e7f45eet68fdaaa453c8f94e@mail.gmail.com> <20060315021734.GA21778@activestate.com> <1f7befae0603142100r6f9e0dddr1ad3e5be4e2a669f@mail.gmail.com> Message-ID: <1f7befae0603142113jf891024ra3861d41ab6f2cf0@mail.gmail.com> [Uncle Timmy] ... > Looks like it was running test_bsddb at the time, and the test > framework gave up after waiting 20 minutes for more output. I had one > of those "recently" that waited 20 minutes for output after starting > test_shelve, but it's scrolled off the page. Berkeley DB is fishy. Well speak of the devil, and the Canadians appear ;-) Your _current_ Win2K test run crapped out after waiting 20 minutes for test_shelve to finish: http://www.python.org/dev/buildbot/all/x86%20W2k%20trunk/builds/23/step-test/0 I don't recall this ever happening before we moved to the newer bsddb, Now it's happened on at least two machines. From guido at python.org Wed Mar 15 06:16:17 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 14 Mar 2006 21:16:17 -0800 Subject: [Python-Dev] Keep default comparisons - or add a second set? In-Reply-To: <6949EC6CD39F97498A57E0FA55295B214115C5@ex9.hostedexchange.local> References: <6949EC6CD39F97498A57E0FA55295B214115C5@ex9.hostedexchange.local> Message-ID: On 12/28/05, Robert Brewer wrote: > Noam Raphael wrote: > > I don't think that every type that supports equality > > comparison should support order comparison. I think > > that if there's no meaningful comparison (whether > > equality or order), an exception should be raised. > > Just to keep myself sane... > > def date_range(start=None, end=None): > if start == None: > start = datetime.date.today() > if end == None: > end = datetime.date.today() > return end - start > > Are you saying the "if" statements will raise TypeError if start or end are > dates? That would be a sad day for Python. Perhaps you're saying that there > is a "meaningful comparison" between None and anything else, but please > clarify if so. Not to worry. My plans for Py3K are to ditch />= unless explicitly defined, but to define == and != on all objects -- if not explicitly defined, == will be false and != will be true. Types can still override == and != to raise exceptions if they really want to guard against certain comparisons; but equality is too important an operation to drop. It should still be possible to use dicts with mixed-type keys! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From nnorwitz at gmail.com Wed Mar 15 06:35:17 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 14 Mar 2006 21:35:17 -0800 Subject: [Python-Dev] Deprecated modules going away in 2.5 Message-ID: Unless I hear shouts *soon*, the following modules will be removed in 2.5: reconvert.py regex # regexmodule.c regex_syntax.py regsub.py lib-old/* # these are the modules under lib-old Para.py codehack.py fmt.py ni.py statcache.py whatsound.py addpack.py dircmp.py grep.py packmail.py tb.py whrandom.py cmp.py dump.py lockfile.py poly.py tzparse.py zmod.py cmpcache.py find.py newdir.py rand.py util.py In addition, I will swap sre and re. This will make help(re) work properly. Let me know if you disagree with any of these changes. n From fredrik at pythonware.com Wed Mar 15 06:56:02 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 15 Mar 2006 06:56:02 +0100 Subject: [Python-Dev] About "Coverity Study Ranks LAMP Code Quality" References: <44158A53.90909@nuxeo.com> <4415FBA2.5090308@canterbury.ac.nz> <44174DEA.2020205@canterbury.ac.nz> Message-ID: Greg Ewing wrote: > Fredrik Lundh wrote: > > > return=NULL; output=junk => out of memory > > return=junk; output=-1 => cannot do this > > return=pointer; output=value => did this, returned value bytes > > > I agree that the design is a bit questionable; > > It sure is. If you get both NULL and -1 returned, how are > you supposed to know which one is the junk? I was about to say "by doing the tests in the prescribed order", but you're right that it's not obvious that the function check that it returns the right kind of junk... (it's possible that the junk in the second line is actually "pointer to some other ob- ject"). I'll have a look. From tim.peters at gmail.com Wed Mar 15 07:11:18 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 15 Mar 2006 01:11:18 -0500 Subject: [Python-Dev] [Python-checkins] r43022 - in python/trunk: Modules/xxmodule.c Objects/object.c In-Reply-To: References: <20060314060219.856311E4007@bag.python.org> <44168F42.6040102@egenix.com> Message-ID: <1f7befae0603142211p6bc6302du701b6417fa9dd09d@mail.gmail.com> [M.-A. Lemburg] >> Why do you add these things to the xx module and not the >> _testcapi module where these things should live ? [Neal Norwitz] > Because I'm an idiot? Ah, so _that's_ why you were made the release coordinator ;-) > Thanks for pointing it out, I moved the code. Or maybe that was why. People who clean up after themselves are certainly welcome to coordinate me! From tim.peters at gmail.com Wed Mar 15 07:42:24 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 15 Mar 2006 01:42:24 -0500 Subject: [Python-Dev] [Python-checkins] Python Regression Test Failures refleak (1) In-Reply-To: <9e804ac0603131533n6156f6e3gc85728c5235435c1@mail.gmail.com> References: <20060313215525.GA4007@python.psfb.org> <9e804ac0603131533n6156f6e3gc85728c5235435c1@mail.gmail.com> Message-ID: <1f7befae0603142242g5ede4c2ftc7d4a49ffe359f9a@mail.gmail.com> [Thomas Wouters] > I did the same narrowing-down last week, and submitted a patch to add > cycle-GC support to itertools.tee . It really needs it. I agree. > Come to think of it, now that I remember how to properly do GC, I think > the patch cuts some corners, but it solved the problem. You mean because it didn't supply tp_clear? That's a funny one. Some people take pride in not supplying tp_clear unless it's absolutely necessary. For example, despite that tuples can be in cycles, the tuple type doesn't supply a tp_clear. This is "because" it's possible to prove that any cycle involving tuples must involve a non-tuple gc'ed type too, and that clearing the latter is always sufficient to break the cycles (which is all tp_clear _needs_ to do: we just need that the aggregate of all tp_clear slots that are implemented suffice to break all possible cycles). I never saw a point to that cleverness, though. It makes gc more obscure, and I'm not sure what it buys. Maybe the (typically teensy) bit of code needed to implement a tp_clear slot? That's all the (dubious) benefit I can think of. > Raymond is on it, anyway: > > http://python.org/sf/1444398 You found it, you fix it :-) From raymond.hettinger at verizon.net Wed Mar 15 07:49:51 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Wed, 15 Mar 2006 01:49:51 -0500 Subject: [Python-Dev] [Python-checkins] Python Regression Test Failuresrefleak (1) References: <20060313215525.GA4007@python.psfb.org><9e804ac0603131533n6156f6e3gc85728c5235435c1@mail.gmail.com> <1f7befae0603142242g5ede4c2ftc7d4a49ffe359f9a@mail.gmail.com> Message-ID: <000d01c647fc$ad4d8ff0$6a01a8c0@RaymondLaptop1> >> Raymond is on it, anyway: >> >> http://python.org/sf/1444398 > > You found it, you fix it :-) I've got this one. Raymond From tim.peters at gmail.com Wed Mar 15 08:43:37 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 15 Mar 2006 02:43:37 -0500 Subject: [Python-Dev] About "Coverity Study Ranks LAMP Code Quality" In-Reply-To: References: <44158A53.90909@nuxeo.com> <4415FBA2.5090308@canterbury.ac.nz> Message-ID: <1f7befae0603142343x3749ece2nfa1fde24c23e7cfc@mail.gmail.com> [Neal Norwitz] > ... > The public report says 15, but the current developer report shows 12. > I'm not sure why there is a discrepancy. All 12 are in ctypes which > was recently imported. I'm having a really hard time making sense of the UI on this. When I looked at the Python project just now (I can log in, so guess that's what you called the "developer report" above), I see 13 "error" rows, and none of them referencing ctypes. OTOH, maybe you'd count this as zero rows, since there are none left with BUG or UNINSPECTED status. I dug into one of them, a claim by the tool that after marshal.c's: int one = 1; int is_little_endian = (int)*(char*)&one; we have: Event const: After this line, the value of "is_little_endian" is equal to 1 but of course that's not so on a big-endian box, and it goes on to claim that there's dead code because of this. I'm not much inclined to look at more of these -- I probably waited so long that all we have left are false positives? If not, and somebody wants me to look at one, point it out specifically ;-) From nnorwitz at gmail.com Wed Mar 15 08:56:43 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 14 Mar 2006 23:56:43 -0800 Subject: [Python-Dev] About "Coverity Study Ranks LAMP Code Quality" In-Reply-To: <1f7befae0603142343x3749ece2nfa1fde24c23e7cfc@mail.gmail.com> References: <44158A53.90909@nuxeo.com> <4415FBA2.5090308@canterbury.ac.nz> <1f7befae0603142343x3749ece2nfa1fde24c23e7cfc@mail.gmail.com> Message-ID: On 3/14/06, Tim Peters wrote: > [Neal Norwitz] > > ... > > The public report says 15, but the current developer report shows 12. > > I'm not sure why there is a discrepancy. All 12 are in ctypes which > > was recently imported. > > I'm having a really hard time making sense of the UI on this. When I The UI is, um, a little less than intuitive. > looked at the Python project just now (I can log in, so guess that's > what you called the "developer report" above), I see 13 "error" rows, Yes, the reports developers can see when they log in. > and none of them referencing ctypes. OTOH, maybe you'd count this as > zero rows, since there are none left with BUG or UNINSPECTED status. After you login, you can click View Runs. Then click the link in the Results column, currently it's 50 for Run 19 (the top row). Now you should be looking at all the results. For me the top 10 rows or so are UNCONFIRMED all for ctypes. But to make the categories clearer, use the Group By option menu at the top and select Status. Then the table will provide a table where each category is shown a little clearer. Click on the View links to see the actual code with the warnings annotated inline. > I'm not much inclined to look at more of these -- I probably waited so > long that all we have left are false positives? If not, and somebody > wants me to look at one, point it out specifically ;-) Yes, most of the problems have been resolved. The one you pointed out is bogus. There's another dead code one, but it's in generated code (an extra if (! value) return NULL;) so who cares. Since there's no problem in any of your code AFAIK, I'll let you off the hook. :-) There really weren't that many reports and I believe most have been reviewed by more than one person. n From g.brandl at gmx.net Wed Mar 15 09:40:28 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 15 Mar 2006 09:40:28 +0100 Subject: [Python-Dev] Bug Day? In-Reply-To: References: Message-ID: Neil Schemenauer wrote: > I think it would be a good idea to follow the Plone project and try > to encourage new developers by offering assistance to get them up > and running. AFAIK, we've done that for the other bug days but it > might help to publish the fact that no prior Python development > experience is necessary. That's right, though I guess it'll be hard to find new core developers. Unfortunately, for most people developing _with_ Python is much more fun than developing Python. Settling on Friday, how about the 31st of March? If that's okay for the VIPs, I will send out announcements on python-list, the non-english newsgroups and lists and ask Tim to place it on the website. Georg From tim.peters at gmail.com Wed Mar 15 09:41:20 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 15 Mar 2006 03:41:20 -0500 Subject: [Python-Dev] [Python-checkins] r43028 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: References: <20060314203927.EA3181E4008@bag.python.org> Message-ID: <1f7befae0603150041u4e0e6e7ds37450ba19293cfcc@mail.gmail.com> >> Author: thomas.heller >> Date: Tue Mar 14 21:39:27 2006 >> New Revision: 43028 >> >> Modified: >> python/trunk/Modules/_ctypes/cfield.c >> Log: >> Cast an Py_ssize_t to int, to avoid a compiler warning. >> >> Modified: python/trunk/Modules/_ctypes/cfield.c >> ============================================================================== >> --- python/trunk/Modules/_ctypes/cfield.c (original) >> +++ python/trunk/Modules/_ctypes/cfield.c Tue Mar 14 21:39:27 2006 >> @@ -251,10 +251,10 @@ >> >> if (bits) >> result = PyString_FromFormat("", >> - name, self->offset, size, bits); >> + name, (int)self->offset, size, bits); >> else >> result = PyString_FromFormat("", >> - name, self->offset, size); >> + name, (int)self->offset, size); >> return result; >> } [Neal Norwitz] > This isn't exactly correct. On a 64-bit system, the value will be > cast into a 32-bit integer. This is true for both Win64 and Unix. If > you change the cast to a long and use %ld (lowercase ell), that will > work correctly on Unix, but not Win64. To always display the correct > value on all platforms, you need an #ifdef MS_WIN64. For Windows, you > use %Id (that's a capital letter eye) and reference the value without > a cast. For Unix, you use %ld (lowercase ell), and cast the value to > a (long) to avoid a warning. I'm copying this to python-dev because it's important people understand this :-): all of the above is kinda true but irrelevant. Martin already checked in a patch so that PyString_FromFormat() understands the C99 "z" qualifier on _all_ platforms. So the correct way to repair this one wasn't to add a cast, and much less to add an #ifdef, it was to change ofs=%d to ofs=%zd in the format. I'm going to check that change in now, but don't make me do it again :-) From martin at v.loewis.de Wed Mar 15 10:00:30 2006 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 15 Mar 2006 10:00:30 +0100 Subject: [Python-Dev] About "Coverity Study Ranks LAMP Code Quality" In-Reply-To: References: <44158A53.90909@nuxeo.com> <20060313230507.GA18532@unpythonic.net> <44160688.3000209@v.loewis.de> Message-ID: <4417D7AE.7030103@v.loewis.de> Alexander Schremmer wrote: >>I can understand that position. The bugs they find include potential >>security flaws, for which exploits could be created if the results are >>freely available. > > > On the other hand, the exploit could be crafted based on reading the SVN > check-ins ... Sure. However, at that point, the bug is fixed (atleast in SVN); crackers need to act comparatively fast then to exploit it. OTOH, if only the report was available, the project might not take any action for some time, increasing the risk of an exploit. Only telling the developers is an established tradition for security-relevant bugs, and a reasonable one IMO. Regards, Martin From martin at v.loewis.de Wed Mar 15 10:03:09 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 15 Mar 2006 10:03:09 +0100 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <20060315021734.GA21778@activestate.com> References: <20060314010819.GB29489@activestate.com> <20060314034831.GE10510@activestate.com> <1f7befae0603140755k66ff91c1jb41917d21efb5d41@mail.gmail.com> <44172F38.1090909@v.loewis.de> <20060314215550.GC5358@activestate.com> <1f7befae0603141507u7e7f45eet68fdaaa453c8f94e@mail.gmail.com> <20060315021734.GA21778@activestate.com> Message-ID: <4417D84D.1090800@v.loewis.de> Trent Mick wrote: > Yes I've noticed it too. I've had to kill python_d.exe a few times. I > haven't yet had the chance to look into it. I am NOT getting this error > on another Windows Python build slave that I am running in-house for > play. I believe this originated from a test run not terminating. buildbot then tried to kill the test run after the timeout; that also failed. Subsequent build now fail to compile. It just happened again: http://www.python.org/dev/buildbot/trunk/x86%20W2k%20trunk/builds/23/step-test/0 http://www.python.org/dev/buildbot/trunk/x86%20W2k%20trunk/builds/23/step-test/1 http://www.python.org/dev/buildbot/trunk/x86%20W2k%20trunk/builds/23/step-test/2 Regards, Martin From tim.peters at gmail.com Wed Mar 15 10:03:12 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 15 Mar 2006 04:03:12 -0500 Subject: [Python-Dev] [Python-checkins] r43041 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: <4417D654.3030600@v.loewis.de> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> Message-ID: <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> [tim.peters] >> CField_repr(): PyString_FromFormat() understands the >> C99 "z" qualifier on all platforms. [Martin v. L?wis] > Unfortunately, only so in Python 2.5. If the code is also > meant to be used for earlier versions, it won't work there > at all. Does that matter? I checked the patch in on the Python trunk, and that's 2.5 to me ;-) The change was in: static void CField_repr(CFieldObject *self) and: typedef struct { PyObject_HEAD Py_ssize_t offset; Py_ssize_t size; Py_ssize_t index; /* Index into CDataObject's object array */ PyObject *proto; /* a type or NULL */ GETFUNC getfunc; /* getter function if proto is NULL */ SETFUNC setfunc; /* setter function if proto is NULL */ } CFieldObject; has Py_ssize_t all over the place, and that's unique to 2.5 (so far) too, right? From fredrik at pythonware.com Wed Mar 15 10:07:42 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 15 Mar 2006 10:07:42 +0100 Subject: [Python-Dev] About "Coverity Study Ranks LAMP Code Quality" References: <44158A53.90909@nuxeo.com><20060313230507.GA18532@unpythonic.net> <44160688.3000209@v.loewis.de> <4417D7AE.7030103@v.loewis.de> Message-ID: Martin v. Löwis wrote: > > On the other hand, the exploit could be crafted based on reading the SVN > > check-ins ... > > Sure. However, at that point, the bug is fixed (atleast in SVN); > crackers need to act comparatively fast then to exploit it. OTOH, if > only the report was available, the project might not take any action > for some time, increasing the risk of an exploit. it should also be mentioned that Python has an established procedure for dealing with more serious security problems, and "go check it in" is not part of that procedure. (there's still a possibility that someone checks in a fix without realizing that the original bug is an attack vector, but I don't think Coverity has discovered anything like that in the Python code base; we're mainly talking about leaks and null-pointer references here). From martin at v.loewis.de Wed Mar 15 10:13:27 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 15 Mar 2006 10:13:27 +0100 Subject: [Python-Dev] [Python-checkins] r43041 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> Message-ID: <4417DAB7.7030308@v.loewis.de> Tim Peters wrote: >>Unfortunately, only so in Python 2.5. If the code is also >>meant to be used for earlier versions, it won't work there >>at all. > > > Does that matter? I believe it does: the ctypes maintainer wants to keep the code identical across releases (AFAICT). > has Py_ssize_t all over the place, and that's unique to 2.5 (so far) too, right? Not necessarily. For example, Modules/_elementtree.c has /* compatibility macros */ #if (PY_VERSION_HEX < 0x02050000) typedef int Py_ssize_t; #define lenfunc inquiry #endif I believe ctypes "wants" to take that approach also. Regards, Martin From tim.peters at gmail.com Wed Mar 15 10:20:14 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 15 Mar 2006 04:20:14 -0500 Subject: [Python-Dev] [Python-checkins] r43041 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: <4417DAB7.7030308@v.loewis.de> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> Message-ID: <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> [Martin] > ... > I believe it does: the ctypes maintainer wants to keep the > code identical across releases (AFAICT). Fair enough -- I reverted the checkin. It's going to need #if'ery on the Python version, though, if it wants to match a Python-version-dependent data width with an appropriate format code. From fredrik at pythonware.com Wed Mar 15 10:12:52 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 15 Mar 2006 10:12:52 +0100 Subject: [Python-Dev] About "Coverity Study Ranks LAMP Code Quality" References: <44158A53.90909@nuxeo.com><20060313230507.GA18532@unpythonic.net> <44160688.3000209@v.loewis.de><4417D7AE.7030103@v.loewis.de> Message-ID: > (there's still a possibility that someone checks in a fix without realizing that > the original bug is an attack vector, but I don't think Coverity has discovered > anything like that in the Python code base; we're mainly talking about leaks > and null-pointer references here). to clarify, "potential leaks and null-pointer references". we already have Neal for leaks, and a large user base for null-pointer references... From mal at egenix.com Wed Mar 15 10:22:50 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 15 Mar 2006 10:22:50 +0100 Subject: [Python-Dev] Using relative imports in std lib packages ([Python-checkins] r43033 - in python/trunk/Lib: distutils/sysconfig.py encodings/__init__.py) In-Reply-To: <20060315043355.C82D71E4009@bag.python.org> References: <20060315043355.C82D71E4009@bag.python.org> Message-ID: <4417DCEA.1090707@egenix.com> guido.van.rossum wrote: > Author: guido.van.rossum > Date: Wed Mar 15 05:33:54 2006 > New Revision: 43033 > > Modified: > python/trunk/Lib/distutils/sysconfig.py > python/trunk/Lib/encodings/__init__.py > Log: > Use relative imports in a few places where I noticed the need. > (Ideally, all packages in Python 2.5 will use the relative import > syntax for all their relative import needs.) Instead of adding relative imports to packages in the standard lib, I'd suggest to use absolute imports instead. These are much easier to manage, maintain and read. There's also no "need" for relative imports in std lib packages since these won't be subject to possible relocation. > Modified: python/trunk/Lib/distutils/sysconfig.py > ============================================================================== > --- python/trunk/Lib/distutils/sysconfig.py (original) > +++ python/trunk/Lib/distutils/sysconfig.py Wed Mar 15 05:33:54 2006 > @@ -16,7 +16,7 @@ > import string > import sys > > -from errors import DistutilsPlatformError > +from .errors import DistutilsPlatformError > > # These are needed in a couple of spots, so just compute them once. > PREFIX = os.path.normpath(sys.prefix) > > Modified: python/trunk/Lib/encodings/__init__.py > ============================================================================== > --- python/trunk/Lib/encodings/__init__.py (original) > +++ python/trunk/Lib/encodings/__init__.py Wed Mar 15 05:33:54 2006 > @@ -27,7 +27,8 @@ > > """#" > > -import codecs, types, aliases > +import codecs, types > +from . import aliases > > _cache = {} > _unknown = '--unknown--' > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 15 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From ncoghlan at iinet.net.au Wed Mar 15 12:18:42 2006 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Wed, 15 Mar 2006 21:18:42 +1000 Subject: [Python-Dev] PEP 338 implemented in SVN Message-ID: <4417F812.6000904@iinet.net.au> Following Guido's acceptance of the PEP in SVN, PEP 338 has now been checked in for 2.5. (PEP's 0, 338 and 356 have been updated to reflect this). Cheers, Nick. P.S. Has the change to the new look site broken the automatic update of pydotorg after PEP checkins? (PEP 356 looks fairly different from the latest SVN version) -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From fredrik at pythonware.com Wed Mar 15 12:13:03 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 15 Mar 2006 12:13:03 +0100 Subject: [Python-Dev] About "Coverity Study Ranks LAMP Code Quality" References: <44158A53.90909@nuxeo.com><4415FBA2.5090308@canterbury.ac.nz><44174DEA.2020205@canterbury.ac.nz> Message-ID: >> > return=NULL; output=junk => out of memory >> > return=junk; output=-1 => cannot do this >> > return=pointer; output=value => did this, returned value bytes >> >> > I agree that the design is a bit questionable; >> >> It sure is. If you get both NULL and -1 returned, how are >> you supposed to know which one is the junk? > > I was about to say "by doing the tests in the prescribed order", > but you're right that it's not obvious that the function checks > that it returns the right kind of junk... (it's possible that the > junk in the second line is actually "pointer to some other ob- > ject"). footnote: it is; the table should read return=NULL; output=junk => out of memory return=old pointer; output=-1 => no need do this; returning old pointer return=new pointer; output=value => did this, returned pointer to newly allocated area containing 'value' bytes From ncoghlan at gmail.com Wed Mar 15 14:32:15 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 15 Mar 2006 23:32:15 +1000 Subject: [Python-Dev] PEP 338 implemented in SVN In-Reply-To: <4417F812.6000904@iinet.net.au> References: <4417F812.6000904@iinet.net.au> Message-ID: <4418175F.80803@gmail.com> Nick Coghlan wrote: > Following Guido's acceptance of the PEP in SVN, PEP 338 has now been checked > in for 2.5. (PEP's 0, 338 and 356 have been updated to reflect this). And thanks to buildbot, the test code that was broken on Windows has been fixed, too. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From rodsenra at gpr.com.br Wed Mar 15 14:47:32 2006 From: rodsenra at gpr.com.br (Rodrigo Dias Arruda Senra) Date: Wed, 15 Mar 2006 10:47:32 -0300 Subject: [Python-Dev] Making builtins more efficient In-Reply-To: <1142399247.11091.229.camel@grey> References: <1141879691.11091.78.camel@grey> <007c01c64399$af03ffd0$a600000a@RaymondLaptop1> <1142399247.11091.229.camel@grey> Message-ID: <20060315104732.694a8d68@Fenix.gpr.com.br> | [ Raymond Hettinger ]: | > If someone really cared about the double lookup, they could | > flatten a level by starting their modules with: | > | > from __builtin__ import * | > | > However, we don't see people writing this kind of code. That | > could mean that the double lookup hasn't been a big concern. [ Steven Elliott ]: ---------------------------------------- | It could mean that. I think what you are suggesting is sufficiently | cleaver that the average Python coder may not have thought of it. | | # small cut | | And the "from __builtin__ import *" additionally imposes a startup | cost and memory cost (at least a word per builtin, I would guess). I suppose that if someone decided to use "from __builtin__ import *" to avoid double lookup, this person *knows* what builtins should be optmized, and therefore could use import specific builtins "from __builtin__ import len, max" avoiding a startup/memory penalty. Otherwise, the startup/memory penalty might be non-issues. cheers, Rod Senra From exarkun at divmod.com Wed Mar 15 16:15:44 2006 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Wed, 15 Mar 2006 10:15:44 -0500 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <1f7befae0603142100r6f9e0dddr1ad3e5be4e2a669f@mail.gmail.com> Message-ID: <20060315151544.22481.539132731.divmod.quotient.503@ohm> On Wed, 15 Mar 2006 00:00:06 -0500, Tim Peters wrote: >[Trent Mick] >> Yes I've noticed it too. I've had to kill python_d.exe a few times. I >> haven't yet had the chance to look into it. I am NOT getting this error >> on another Windows Python build slave that I am running in-house for >> play. > >The last run on your Win2K slave that got beyond the compile step: > >http://www.python.org/dev/buildbot/all/x86%20W2k%20trunk/builds/16/step-test/0 > >Looks like it was running test_bsddb at the time, and the test >framework gave up after waiting 20 minutes for more output. I had one >of those "recently" that waited 20 minutes for output after starting >test_shelve, but it's scrolled off the page. Berkeley DB is fishy. >Looks like the buildbot doesn't know how to kill a process on Windows >either (SIGKILL sure ain't gonna do it ;-)). It should actually be using TerminateProcess (depending on the Twisted version being used, the relevant code is either in twisted/internet/_dumbwin32proc.py or twisted/internet/win32eventreactor.py, in the signalProcess method in either case), but even this doesn't seem to be a completely reliable way to end a process. Twisted's buildbot has run into this problem as well, but we haven't figure out how to fix it yet. Suggestions welcome - patches even more so :) Jean-Paul From thomas at python.org Wed Mar 15 19:14:47 2006 From: thomas at python.org (Thomas Wouters) Date: Wed, 15 Mar 2006 19:14:47 +0100 Subject: [Python-Dev] [Python-checkins] r43033 - in python/trunk/Lib: distutils/sysconfig.py encodings/__init__.py In-Reply-To: <20060315043355.C82D71E4009@bag.python.org> References: <20060315043355.C82D71E4009@bag.python.org> Message-ID: <9e804ac0603151014sce7c1aq6eeb2add15e893e4@mail.gmail.com> On 3/15/06, guido.van.rossum wrote: > > > Use relative imports in a few places where I noticed the need. > (Ideally, all packages in Python 2.5 will use the relative import > syntax for all their relative import needs.) You should be aware that using relative imports (or absolute imports) in the standard library may cause custom __import__'s that don't take the optional fifth argument to break, whereas using 'old-style' imports does not. I don't know how common custom __import__ hooks are, and adding the optional fifth argument is backwardly compatible, but I would personally not add absolute or relative imports to the oft-used parts of the stdlib until after 2.5. On the other hand, code using sys.path_hooks (like zipimport) don't need any tweaking, as the absolute/relative choice is made before they are queried, so I don't know howmuch impact this has, all in all. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060315/6c0220cc/attachment.html From guido at python.org Wed Mar 15 19:33:14 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 15 Mar 2006 10:33:14 -0800 Subject: [Python-Dev] [Python-checkins] r43033 - in python/trunk/Lib: distutils/sysconfig.py encodings/__init__.py In-Reply-To: <9e804ac0603151014sce7c1aq6eeb2add15e893e4@mail.gmail.com> References: <20060315043355.C82D71E4009@bag.python.org> <9e804ac0603151014sce7c1aq6eeb2add15e893e4@mail.gmail.com> Message-ID: Well, absolute imports without the future statement will not use the 5th argument, so they won't break, right? That's what MAL also says. Someone please fix this. On 3/15/06, Thomas Wouters wrote: > > > On 3/15/06, guido.van.rossum wrote: > > > > Use relative imports in a few places where I noticed the need. > > (Ideally, all packages in Python 2.5 will use the relative import > > syntax for all their relative import needs.) > > > You should be aware that using relative imports (or absolute imports) in the > standard library may cause custom __import__'s that don't take the optional > fifth argument to break, whereas using 'old-style' imports does not. I don't > know how common custom __import__ hooks are, and adding the optional fifth > argument is backwardly compatible, but I would personally not add absolute > or relative imports to the oft-used parts of the stdlib until after 2.5. > > On the other hand, code using sys.path_hooks (like zipimport) don't need any > tweaking, as the absolute/relative choice is made before they are queried, > so I don't know howmuch impact this has, all in all. > > -- > Thomas Wouters > > Hi! I'm a .signature virus! copy me into your .signature file to help me > spread! > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Wed Mar 15 20:00:55 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 15 Mar 2006 20:00:55 +0100 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <20060315151544.22481.539132731.divmod.quotient.503@ohm> References: <20060315151544.22481.539132731.divmod.quotient.503@ohm> Message-ID: <44186467.9060102@v.loewis.de> Jean-Paul Calderone wrote: > It should actually be using TerminateProcess (depending on the > Twisted version being used, the relevant code is either in > twisted/internet/_dumbwin32proc.py or > twisted/internet/win32eventreactor.py, in the signalProcess method in > either case) So PythonWin needs to be installed on a Windows buildbot slave, right? Regards, Martin From fredrik at pythonware.com Wed Mar 15 20:18:28 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 15 Mar 2006 20:18:28 +0100 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot References: <20060315151544.22481.539132731.divmod.quotient.503@ohm> <44186467.9060102@v.loewis.de> Message-ID: Martin v. Löwis wrote: > Jean-Paul Calderone wrote: > > It should actually be using TerminateProcess (depending on the > > Twisted version being used, the relevant code is either in > > twisted/internet/_dumbwin32proc.py or > > twisted/internet/win32eventreactor.py, in the signalProcess method in > > either case) > > So PythonWin needs to be installed on a Windows buildbot slave, right? unless someone hacks Twisted to use _subprocess.TerminateProcess instead of the win32all version... From pje at telecommunity.com Wed Mar 15 20:23:39 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 15 Mar 2006 14:23:39 -0500 Subject: [Python-Dev] [Python-checkins] r43033 - in python/trunk/Lib: distutils/sysconfig.py encodings/__init__.py In-Reply-To: References: <9e804ac0603151014sce7c1aq6eeb2add15e893e4@mail.gmail.com> <20060315043355.C82D71E4009@bag.python.org> <9e804ac0603151014sce7c1aq6eeb2add15e893e4@mail.gmail.com> Message-ID: <5.1.1.6.0.20060315142201.01fb8a18@mail.telecommunity.com> At 10:33 AM 3/15/2006 -0800, Guido van Rossum wrote: >Well, absolute imports without the future statement will not use the >5th argument, so they won't break, right? That's what MAL also says. >Someone please fix this. Why is a 5th argument needed to do absolute imports? Shouldn't it suffice to supply a globals argument with no __path__ and an undotted __name__? ISTM that passing in the builtins dictionary as the globals argument ought to do the trick. From trentm at ActiveState.com Wed Mar 15 20:28:22 2006 From: trentm at ActiveState.com (Trent Mick) Date: Wed, 15 Mar 2006 11:28:22 -0800 Subject: [Python-Dev] Deprecated modules going away in 2.5 In-Reply-To: References: Message-ID: <20060315192822.GB26023@activestate.com> [Neal Norwitz wrote] > In addition, I will swap sre and re. This will make help(re) work properly. Yay! Trent -- Trent Mick TrentM at ActiveState.com From exarkun at divmod.com Wed Mar 15 21:30:39 2006 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Wed, 15 Mar 2006 15:30:39 -0500 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: Message-ID: <20060315203039.22481.1702215618.divmod.quotient.675@ohm> On Wed, 15 Mar 2006 20:18:28 +0100, Fredrik Lundh wrote: >Martin v. L? wrote: > >> Jean-Paul Calderone wrote: >> > It should actually be using TerminateProcess (depending on the >> > Twisted version being used, the relevant code is either in >> > twisted/internet/_dumbwin32proc.py or >> > twisted/internet/win32eventreactor.py, in the signalProcess method in >> > either case) >> >> So PythonWin needs to be installed on a Windows buildbot slave, right? > >unless someone hacks Twisted to use _subprocess.TerminateProcess >instead of the win32all version... Twisted's Win32 process support also uses win32api, win32con, win32event, win32file, win32pipe, and win32security. ;P So the new subprocess module alone isn't quite a sufficient replacement... Jean-Paul From nnorwitz at gmail.com Tue Mar 14 07:55:23 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 13 Mar 2006 22:55:23 -0800 Subject: [Python-Dev] Coverity report Message-ID: Attached is an image from a coverity report for ctypes. Here is the text: 3866 val = Simple_get_value(self); 3867 if (val == NULL) 3868 return NULL; 3869 3870 name = PyString_FromString(self->ob_type->tp_name); Event var_compare_op: Added "name" due to comparison "name == 0" Also see events: [var_deref_op] At conditional (1): "name == 0" taking true path 3871 if (name == NULL) { Event var_deref_op: Variable "name" tracked as NULL was dereferenced. Also see events: [var_compare_op] 3872 Py_DECREF(name); 3873 return NULL; 3874 } 3875 3876 args = PyTuple_Pack(2, name, val); 3877 Py_DECREF(name); 3878 Py_DECREF(val); 3879 if (args == NULL) 3880 return NULL; What is being described is that name is known to be NULL and it is being free()d which is incorrect. As you can see from the context, line 3872 (of Modules/_ctypes/_ctypes.c) should be Py_DECREF(val); n -------------- next part -------------- A non-text attachment was scrubbed... Name: ctypes-coverity.jpg Type: image/jpeg Size: 40797 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20060313/77fda0ac/attachment-0001.jpg From guido at python.org Wed Mar 15 22:34:38 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 15 Mar 2006 13:34:38 -0800 Subject: [Python-Dev] [Python-checkins] r43033 - in python/trunk/Lib: distutils/sysconfig.py encodings/__init__.py In-Reply-To: <5.1.1.6.0.20060315142201.01fb8a18@mail.telecommunity.com> References: <20060315043355.C82D71E4009@bag.python.org> <9e804ac0603151014sce7c1aq6eeb2add15e893e4@mail.gmail.com> <5.1.1.6.0.20060315142201.01fb8a18@mail.telecommunity.com> Message-ID: Because Thomas designed it this way. :-) I believe his design makes sense though: "import foo" translates to __import__(foo, ...). There's a separate setting, only known to the compiler, that says whether "from __future__ import absolute_import" is in effect (there's no way to slip a flag into globals to convey this setting, since code is compiled independently from globals, and there are ways to pass flags to the compiler without explicitly doing the future import). So the compiler emits different code when the future syntax is in effect, and that opcode must pass its knowledge to __import__. This is done trough the 5th argument, which also tells us how many leading dots there were. Without the future import, the 5th argument is omitted (as long as there aren't any leading dots). --Guido On 3/15/06, Phillip J. Eby wrote: > At 10:33 AM 3/15/2006 -0800, Guido van Rossum wrote: > >Well, absolute imports without the future statement will not use the > >5th argument, so they won't break, right? That's what MAL also says. > >Someone please fix this. > > Why is a 5th argument needed to do absolute imports? Shouldn't it suffice > to supply a globals argument with no __path__ and an undotted > __name__? ISTM that passing in the builtins dictionary as the globals > argument ought to do the trick. > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From nnorwitz at gmail.com Wed Mar 15 22:41:37 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 15 Mar 2006 13:41:37 -0800 Subject: [Python-Dev] [Python-checkins] r43028 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: <1f7befae0603150041u4e0e6e7ds37450ba19293cfcc@mail.gmail.com> References: <20060314203927.EA3181E4008@bag.python.org> <1f7befae0603150041u4e0e6e7ds37450ba19293cfcc@mail.gmail.com> Message-ID: On 3/15/06, Tim Peters wrote: > > [Neal Norwitz] > > This isn't exactly correct. On a 64-bit system, the value will be > > cast into a 32-bit integer. This is true for both Win64 and Unix. If > > you change the cast to a long and use %ld (lowercase ell), that will > > work correctly on Unix, but not Win64. To always display the correct > > value on all platforms, you need an #ifdef MS_WIN64. For Windows, you > > use %Id (that's a capital letter eye) and reference the value without > > a cast. For Unix, you use %ld (lowercase ell), and cast the value to > > a (long) to avoid a warning. > > I'm copying this to python-dev because it's important people > understand this :-): all of the above is kinda true but irrelevant. Whoops, my comment only applies to *printf(), not PyString's as you correctly point out. For an example, see the definition for PRINT_TOTAL_REFS() in http://svn.python.org/view/python/trunk/Python/pythonrun.c?rev=42876&view=markup n -- > Martin already checked in a patch so that PyString_FromFormat() > understands the C99 "z" qualifier on _all_ platforms. So the correct > way to repair this one wasn't to add a cast, and much less to add an > #ifdef, it was to change > > ofs=%d > > to > > ofs=%zd > > in the format. I'm going to check that change in now, but don't make > me do it again :-) > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com > From theller at python.net Wed Mar 15 23:08:55 2006 From: theller at python.net (Thomas Heller) Date: Wed, 15 Mar 2006 23:08:55 +0100 Subject: [Python-Dev] [Python-checkins] r43041 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> Message-ID: Tim Peters wrote: > [Martin] >> ... >> I believe it does: the ctypes maintainer wants to keep the >> code identical across releases (AFAICT). Correct, that's why I listed it in PEP 291. > Fair enough -- I reverted the checkin. It's going to need #if'ery on > the Python version, though, if it wants to match a > Python-version-dependent data width with an appropriate format code. You could have left it in, I'm very thankful for any help that I get. Making the patch version independent is my job - unless someone else takes care of it, of course. BTW: Is a "porting guide" to make extension modules compatible with 2.5 available somewhere? PEP 353 scratches only the surface... Thomas From pje at telecommunity.com Wed Mar 15 23:18:43 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 15 Mar 2006 17:18:43 -0500 Subject: [Python-Dev] [Python-checkins] r43033 - in python/trunk/Lib: distutils/sysconfig.py encodings/__init__.py In-Reply-To: References: <5.1.1.6.0.20060315142201.01fb8a18@mail.telecommunity.com> <20060315043355.C82D71E4009@bag.python.org> <9e804ac0603151014sce7c1aq6eeb2add15e893e4@mail.gmail.com> <5.1.1.6.0.20060315142201.01fb8a18@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060315170442.01fff668@mail.telecommunity.com> At 01:34 PM 3/15/2006 -0800, Guido van Rossum wrote: >Because Thomas designed it this way. :-) > >I believe his design makes sense though: "import foo" translates to >__import__(foo, ...). > >There's a separate setting, only known to the compiler, that says >whether "from __future__ import absolute_import" is in effect (there's >no way to slip a flag into globals to convey this setting, since code >is compiled independently from globals, and there are ways to pass >flags to the compiler without explicitly doing the future import). > >So the compiler emits different code when the future syntax is in >effect, and that opcode must pass its knowledge to __import__. This is >done trough the 5th argument, which also tells us how many leading >dots there were. Without the future import, the 5th argument is >omitted (as long as there aren't any leading dots). Ah, so it's *relative* imports that require a 5th argument. I was thinking it was there to support absolute imports. I was thinking that relative imports could be implemented by popping bits off of __name__ to get an absolute location. It seems to me that backward compatibility would be greatly enhanced by having the interpreter convert everything but "legacy" imports into absolute imports, as this would then work with the existing __import__ code in the field, even when new relative/absolute code was doing the importing. Otherwise, this forces __import__ hooks to be rewritten. (I personally avoid writing __import__ hooks if at all possible, but there are certainly some out there.) The mechanism I have in mind would be to just have an IMPORT_EXACT opcode that takes a relative or absolute name. This opcode would process relative names relative to the __name__ in globals to produce an exact module name, and leave absolute names alone. It would then invoke __import__ using the builtins or sys module dictionary as the "globals" argument *instead of the current globals*, so that __import__ will not do any legacy-style fallback. When absolute imports are in effect, or when an explicit relative import is used, it would be compiled such that IMPORT_NAME is replaced by IMPORT_EXACT. This mechanism doesn't require any change to the __import__() signature, and so remains backward compatible with any existing import hook that doesn't do weird things to the globals dictionary of the module that invoked it. On the other hand, perhaps it would be better to fail loudly by breaking on the 5th argument, than to fail silently for really weird __import__ hooks. That is, if it breaks, it will force people to make sure their __import__ code is safe for use with absolute imports. So, the existing approach might well be better than what I had in mind. From guido at python.org Wed Mar 15 23:25:14 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 15 Mar 2006 14:25:14 -0800 Subject: [Python-Dev] [Python-checkins] r43033 - in python/trunk/Lib: distutils/sysconfig.py encodings/__init__.py In-Reply-To: <5.1.1.6.0.20060315170442.01fff668@mail.telecommunity.com> References: <20060315043355.C82D71E4009@bag.python.org> <9e804ac0603151014sce7c1aq6eeb2add15e893e4@mail.gmail.com> <5.1.1.6.0.20060315142201.01fb8a18@mail.telecommunity.com> <5.1.1.6.0.20060315170442.01fff668@mail.telecommunity.com> Message-ID: On 3/15/06, Phillip J. Eby wrote: > Ah, so it's *relative* imports that require a 5th argument. I was thinking > it was there to support absolute imports. I was thinking that relative > imports could be implemented by popping bits off of __name__ to get an > absolute location. Well, also absolute imports when the future statement is in effect -- __import__ needs to know whether to interpret "import foo" as "first try __path__, then sys.path" or as "only look in sys.path". > It seems to me that backward compatibility would be greatly enhanced by > having the interpreter convert everything but "legacy" imports into > absolute imports, as this would then work with the existing __import__ code > in the field, even when new relative/absolute code was doing the > importing. Otherwise, this forces __import__ hooks to be rewritten. (I > personally avoid writing __import__ hooks if at all possible, but there are > certainly some out there.) How would the conversion be done? The compiler can't tell whether a classic "import foo" is intended to be a relative or absolute import. > The mechanism I have in mind would be to just have an IMPORT_EXACT opcode > that takes a relative or absolute name. This opcode would process relative > names relative to the __name__ in globals to produce an exact module name, > and leave absolute names alone. It would then invoke __import__ using the > builtins or sys module dictionary as the "globals" argument *instead of the > current globals*, so that __import__ will not do any legacy-style fallback. Yes, but it would defeat the purpose of import hooks -- import hooks need to be able to assign their own semantics. (Long, long ago, there was no built-in support for packages, but you could install an import hook that enabled it.) > When absolute imports are in effect, or when an explicit relative import is > used, it would be compiled such that IMPORT_NAME is replaced by IMPORT_EXACT. > > This mechanism doesn't require any change to the __import__() signature, > and so remains backward compatible with any existing import hook that > doesn't do weird things to the globals dictionary of the module that > invoked it. > > On the other hand, perhaps it would be better to fail loudly by breaking on > the 5th argument, than to fail silently for really weird __import__ > hooks. That is, if it breaks, it will force people to make sure their > __import__ code is safe for use with absolute imports. So, the existing > approach might well be better than what I had in mind. I think so. Import hooks requires a lot of careful work. Requiring import hooks to be explicitly ported probably provides better guarantees that they actually work. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy at alum.mit.edu Wed Mar 15 23:33:32 2006 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Wed, 15 Mar 2006 17:33:32 -0500 Subject: [Python-Dev] [Python-checkins] r43033 - in python/trunk/Lib: distutils/sysconfig.py encodings/__init__.py In-Reply-To: References: <20060315043355.C82D71E4009@bag.python.org> <9e804ac0603151014sce7c1aq6eeb2add15e893e4@mail.gmail.com> Message-ID: On 3/15/06, Guido van Rossum wrote: > Well, absolute imports without the future statement will not use the > 5th argument, so they won't break, right? That's what MAL also says. > Someone please fix this. I'd much rather see us change imports to use absolute imports than to use relative imports in count-the-dots fashion. Could we add that to PEP 8? Jeremy > > On 3/15/06, Thomas Wouters wrote: > > > > > > On 3/15/06, guido.van.rossum wrote: > > > > > > Use relative imports in a few places where I noticed the need. > > > (Ideally, all packages in Python 2.5 will use the relative import > > > syntax for all their relative import needs.) > > > > > > You should be aware that using relative imports (or absolute imports) in the > > standard library may cause custom __import__'s that don't take the optional > > fifth argument to break, whereas using 'old-style' imports does not. I don't > > know how common custom __import__ hooks are, and adding the optional fifth > > argument is backwardly compatible, but I would personally not add absolute > > or relative imports to the oft-used parts of the stdlib until after 2.5. > > > > On the other hand, code using sys.path_hooks (like zipimport) don't need any > > tweaking, as the absolute/relative choice is made before they are queried, > > so I don't know howmuch impact this has, all in all. > > > > -- > > Thomas Wouters > > > > Hi! I'm a .signature virus! copy me into your .signature file to help me > > spread! > > _______________________________________________ > > Python-Dev mailing list > > Python-Dev at python.org > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > > http://mail.python.org/mailman/options/python-dev/guido%40python.org > > > > > > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu > From barry at python.org Wed Mar 15 23:41:25 2006 From: barry at python.org (Barry Warsaw) Date: Wed, 15 Mar 2006 17:41:25 -0500 Subject: [Python-Dev] [Python-checkins] r43033 - in python/trunk/Lib: distutils/sysconfig.py encodings/__init__.py In-Reply-To: References: <20060315043355.C82D71E4009@bag.python.org> <9e804ac0603151014sce7c1aq6eeb2add15e893e4@mail.gmail.com> Message-ID: <1142462485.2149.27.camel@resist.wooz.org> On Wed, 2006-03-15 at 17:33 -0500, Jeremy Hylton wrote: > On 3/15/06, Guido van Rossum wrote: > > Well, absolute imports without the future statement will not use the > > 5th argument, so they won't break, right? That's what MAL also says. > > Someone please fix this. > > I'd much rather see us change imports to use absolute imports than to > use relative imports in count-the-dots fashion. Could we add that to > PEP 8? +1 -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060315/b00ccee7/attachment.pgp From guido at python.org Wed Mar 15 23:46:32 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 15 Mar 2006 14:46:32 -0800 Subject: [Python-Dev] [Python-checkins] r43033 - in python/trunk/Lib: distutils/sysconfig.py encodings/__init__.py In-Reply-To: <1142462485.2149.27.camel@resist.wooz.org> References: <20060315043355.C82D71E4009@bag.python.org> <9e804ac0603151014sce7c1aq6eeb2add15e893e4@mail.gmail.com> <1142462485.2149.27.camel@resist.wooz.org> Message-ID: Done. Index: pep-0008.txt =================================================================== --- pep-0008.txt (revision 42952) +++ pep-0008.txt (working copy) @@ -156,8 +156,9 @@ - Relative imports for intra-package imports are highly discouraged. Always use the absolute package path for all imports. - (However, once PEP 328 [7] is fully implemented, its style of - explicit relative imports will be recommended.) + Even now that PEP 328 [7] is fully implemented in Python 2.5, + its style of explicit relative imports is actively discouraged; + absolute imports are more portable and usually more readable. - When importing a class from a class-containing module, it's usually okay to spell this On 3/15/06, Barry Warsaw wrote: > On Wed, 2006-03-15 at 17:33 -0500, Jeremy Hylton wrote: > > On 3/15/06, Guido van Rossum wrote: > > > Well, absolute imports without the future statement will not use the > > > 5th argument, so they won't break, right? That's what MAL also says. > > > Someone please fix this. > > > > I'd much rather see us change imports to use absolute imports than to > > use relative imports in count-the-dots fashion. Could we add that to > > PEP 8? > > +1 > -Barry > > > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.2.2 (GNU/Linux) > > iQCVAwUARBiYFXEjvBPtnXfVAQJPzQP/aOcipFhTU+Z6SYB7aeIwOIl+TcUObHzo > xdp3ufscF1R4emX7vgw/BChJfCwt6vcOzKQsyjVgAFYnDuh6XdRdtL3gjex/Z2L0 > OLBKMhOjd3OScyhypefruhXJJ8o1SvQn4eQeGacIsYaqbohs3OEEqv3RW5o1h3Qo > azV3CQn+2Fs= > =qcFp > -----END PGP SIGNATURE----- > > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From dave.brueck at protoven.com Tue Mar 14 23:30:40 2006 From: dave.brueck at protoven.com (Dave Brueck) Date: Tue, 14 Mar 2006 15:30:40 -0700 Subject: [Python-Dev] Another threading idea In-Reply-To: <002301c647ae$e522c2e0$6a01a8c0@RaymondLaptop1> References: <002301c647ae$e522c2e0$6a01a8c0@RaymondLaptop1> Message-ID: <44174410.3040104@protoven.com> Raymond Hettinger wrote: > FWIW, I've been working on a way to simplify the use of queues with daemon > consumer threads > > Sometimes, I launch one or more consumer threads that wait for a task to enter a > queue and then work on the task. A recurring problem is that I sometimes need to > know if all of the tasks have been completed so I can exit or do something with > the result. > > If each thread only does a single task, I can use t.join() to wait until the > task is done. However, if the thread stays alive and waits for more Queue > entries, then there doesn't seem to be a good way to tell when all the > processing is done. The pattern I use is to use the None object to indicate that no more tasks are coming - just add it to your queue once for every worker thread that exists and have your worker threads exit when they get a task of None. Your setup code to start the threads and add tasks to the queue wouldn't need to change; once all the tasks have been enqueued you'd wait for completion like this: # Tell all the workers to quit for t in worker_threads(): q.put(None) # Wait for all the workers to quit for t in worker_threads(): t.join() do_work_on_results() The worker thread looks like this: while 1: task = q.get() if task is None: break do_work(task) -Dave From martin at v.loewis.de Thu Mar 16 00:33:49 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 16 Mar 2006 00:33:49 +0100 Subject: [Python-Dev] [Python-checkins] r43041 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> Message-ID: <4418A45D.7090500@v.loewis.de> Thomas Heller wrote: > BTW: Is a "porting guide" to make extension modules compatible with 2.5 > available somewhere? PEP 353 scratches only the surface... Wrt. ssize_t changes, PEP 353 is meant to be comprehensive. Which particular aspect are you missing? Regards, Martin From tim.peters at gmail.com Thu Mar 16 01:06:06 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 15 Mar 2006 19:06:06 -0500 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <44186467.9060102@v.loewis.de> References: <20060315151544.22481.539132731.divmod.quotient.503@ohm> <44186467.9060102@v.loewis.de> Message-ID: <1f7befae0603151606i6f5368aege63af2912cd4e0da@mail.gmail.com> [Martin v. L?wis] > So PythonWin needs to be installed on a Windows buildbot slave, right? It must, since that's what my wiki page says ;-): http://wiki.python.org/moin/BuildbotOnWindows ... o Install a matching version of pywin32. ... From greg.ewing at canterbury.ac.nz Thu Mar 16 03:30:40 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 16 Mar 2006 15:30:40 +1300 Subject: [Python-Dev] Py3k: Except clause syntax Message-ID: <4418CDD0.8020803@canterbury.ac.nz> For Py3k, any thoughts on changing the syntax of the except clause from except , : to except as : so that things like except TypeError, ValueError: will do what is expected? Greg From greg.ewing at canterbury.ac.nz Thu Mar 16 03:53:24 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 16 Mar 2006 15:53:24 +1300 Subject: [Python-Dev] open() mode is lax Message-ID: <4418D324.60001@canterbury.ac.nz> I've just noticed that (in 2.3.4) open() seems to accept just about anything after the first character of the mode argument: Python 2.3.4 (#1, Jun 30 2004, 16:47:37) [GCC 3.2 20020903 (Red Hat Linux 8.0 3.2-7)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> f = open("DU", "rqwerty") >>> -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From tim.peters at gmail.com Thu Mar 16 04:21:07 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 15 Mar 2006 22:21:07 -0500 Subject: [Python-Dev] open() mode is lax In-Reply-To: <4418D324.60001@canterbury.ac.nz> References: <4418D324.60001@canterbury.ac.nz> Message-ID: <1f7befae0603151921m3c2465f3h5691b8c9a3aef037@mail.gmail.com> [Greg Ewing] > I've just noticed that (in 2.3.4) open() seems to accept > just about anything after the first character of the > mode argument: > > Python 2.3.4 (#1, Jun 30 2004, 16:47:37) > [GCC 3.2 20020903 (Red Hat Linux 8.0 3.2-7)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> f = open("DU", "rqwerty") > >>> Yes. That's because virtually all C libraries accept just about anything in the mode string, so that code using mode-string extensions specific to other platforms doesn't just blow up. Python passes the mode string on to the platform C, and complains if and only if the platform C complains. From guido at python.org Thu Mar 16 04:58:53 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 15 Mar 2006 19:58:53 -0800 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: <4418CDD0.8020803@canterbury.ac.nz> References: <4418CDD0.8020803@canterbury.ac.nz> Message-ID: On 3/15/06, Greg Ewing wrote: > For Py3k, any thoughts on changing the syntax of > the except clause from > > except , : > > to > > except as : > > so that things like > > except TypeError, ValueError: > > will do what is expected? Not a bad idea. The trend seems to be that each use of 'as' is some kind of assignment to the name after 'as' but the relationship with the thing before 'as' is different in each case. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From nidoizo at yahoo.com Thu Mar 16 06:10:57 2006 From: nidoizo at yahoo.com (Nicolas Fleury) Date: Thu, 16 Mar 2006 00:10:57 -0500 Subject: [Python-Dev] Making staticmethod objects callable? In-Reply-To: <20060312153236.GB16848@code0.codespeak.net> References: <20060312153236.GB16848@code0.codespeak.net> Message-ID: Armin Rigo wrote: > Hi Nicolas, > > On Thu, Mar 02, 2006 at 01:55:03AM -0500, Nicolas Fleury wrote: >> (...) A use case is not hard to >> imagine, especially a private static method called only to build a class >> attribute. > > Uh. I do this all the time, and the answer is simply: don't make that a > staticmethod. Staticmethods are for the rare case where you need > dynamic class-based dispatch but don't have an instance around. I think we all agree on this list that there's no point using a staticmethod for that use case. My suggestion was for some comp.lang.python people, a lot coming from other languages. Their reflex would be much more to define a staticmethod. This issue has been pointed a lot of times on comp.lang.python. Regards, Nicolas From guido at python.org Thu Mar 16 06:20:21 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 15 Mar 2006 21:20:21 -0800 Subject: [Python-Dev] Making staticmethod objects callable? In-Reply-To: References: <20060312153236.GB16848@code0.codespeak.net> Message-ID: On 3/15/06, Nicolas Fleury wrote: > I think we all agree on this list that there's no point using a > staticmethod for that use case. My suggestion was for some > comp.lang.python people, a lot coming from other languages. Their > reflex would be much more to define a staticmethod. This issue has been > pointed a lot of times on comp.lang.python. There's no need to change Python so that people coming from other languages won't make silly mistakes, is there? BTW I question the claimed reflex -- assuming by "other languages" you mean Java or C++ (the only languages I know that *have* static methods) -- since those languages don't have the ability to call methods (static or otherwise) at class definition time. So perhaps you need to dig deeper to find out *why* this is a recurring issue. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Thu Mar 16 10:27:32 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 16 Mar 2006 19:27:32 +1000 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: <4418CDD0.8020803@canterbury.ac.nz> References: <4418CDD0.8020803@canterbury.ac.nz> Message-ID: <44192F84.9030903@gmail.com> Greg Ewing wrote: > For Py3k, any thoughts on changing the syntax of > the except clause from > > except , : > > to > > except as : > > so that things like > > except TypeError, ValueError: > > will do what is expected? +1 here I actually had a go at figuring how to do this a long while back, but I have no idea what I ended up doing with the code (I think it died along with the old hard drive in my laptop). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From fredrik at pythonware.com Thu Mar 16 12:02:27 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 16 Mar 2006 12:02:27 +0100 Subject: [Python-Dev] towards a stricter definition of sys.executable Message-ID: the definition of sys.executable is a bit unclear, something that has led to incompatible use in deployed code. the docstring for sys.executable says "pathname of this Python interpreter", which can be interpreted as either a) sys.executable points to the executable that was used to load the Python interpreter library/dll. this use is supported by the docstring and the implementation, and is quite common in the wild. an application using this interpretation may - call sys.executable to run another instance of itself - extract data from resources embedded in (or attached to) sys.executable - locate configuration data etc via os.path.dirname(sys.executable) etc. or b) sys.executable points to a standard Python interpreter executable of the same version, and having the same library, as the currently running interpreter instance. this use is supported by more strict interpretation of the word "interpreter" (as an executable, rather than an implementation DLL), and is quite common in the wild. an application using this interpretation may - call sys.executable to run a Python script in the same environment as itself. etc. or c) sys.executable points to the file containing the actual ("this") interpreter. I haven't seen any code that assumes this, probably because no implementation uses this interpretation... for programs that are invoked via the standard interpreter, case (a) and (b) are of course identical. the problem is when the interpreter library is embedded in some other application; should sys.executable be set to the actual EXE used to start the program, or to something else ? if it's set to something else, how can that application do the things that's described under (a) ? to fix this, I propose adding another sys variable; for example, let sys.executable keep behaving like case (a) (which is how it's implemented today), and add a new sys.python_executable for case (b). the latter can then be set to None if a proper interpreter cannot be located. From g.brandl at gmx.net Thu Mar 16 12:48:25 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 16 Mar 2006 12:48:25 +0100 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: <4418CDD0.8020803@canterbury.ac.nz> References: <4418CDD0.8020803@canterbury.ac.nz> Message-ID: Greg Ewing wrote: > For Py3k, any thoughts on changing the syntax of > the except clause from > > except , : > > to > > except as : > > so that things like > > except TypeError, ValueError: > > will do what is expected? +1. Fits well with the current use of "as". Georg From tzot at mediconsa.com Thu Mar 16 14:08:33 2006 From: tzot at mediconsa.com (Christos Georgiou) Date: Thu, 16 Mar 2006 15:08:33 +0200 Subject: [Python-Dev] bytes thoughts References: <440669B9.3000609@canterbury.ac.nz> Message-ID: "Greg Ewing" wrote in message news:440669B9.3000609 at canterbury.ac.nz... > Baptiste Carvello wrote: > [Baptiste] >> while manipulating binary data will happen mostly with bytes objects, >> some >> operations are better done with ints, like the bit manipulations with the >> &|~^ >> operators. [Greg] > Why not just support bitwise operations directly > on the bytes object? Well, what's the result of bytes([1,0,0])^ bytes([1,0]) ? Is it bytes([0,0,0]) (? la little-endian) or is it bytes([1,1,0]) (straight conversion to base-256)? Or perhaps throw a ValueError if the sizes differ? These details should be considered in the PEP. From tzot at mediconsa.com Thu Mar 16 15:11:14 2006 From: tzot at mediconsa.com (Christos Georgiou) Date: Thu, 16 Mar 2006 16:11:14 +0200 Subject: [Python-Dev] Switch to MS VC++ 2005 ?! References: <44033992.8040805@egenix.com> Message-ID: "M.-A. Lemburg" wrote in message news:44033992.8040805 at egenix.com... > Microsoft has recently released their express version of the Visual C++. > Given that this version is free for everyone, wouldn't it make sense > to ship Python 2.5 compiled with this version ?! > > http://msdn.microsoft.com/vstudio/express/default.aspx Link to the middle of a previous thread about the same option: http://mail.python.org/pipermail/python-dev/2005-November/058052.html Link describes hands-on experience by Paul Moore. From murman at gmail.com Thu Mar 16 15:57:37 2006 From: murman at gmail.com (Michael Urman) Date: Thu, 16 Mar 2006 08:57:37 -0600 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: References: <4418CDD0.8020803@canterbury.ac.nz> Message-ID: On 3/16/06, Georg Brandl wrote: > +1. Fits well with the current use of "as". I like it, but wonder about the false parallels below. My initial reaction is it's not worth worrying about as it's easy to learn as part of the import or except statements, and should do the right thing. Nobody would expect the second import to rename both items to q, and the first except clause would be a SyntaxError. from foo import bar as b, quux as q except TypeError as te, IndexError as ie from foo import bar, quux as q except TypeError, IndexError as e Michael -- Michael Urman http://www.tortall.net/mu/blog From guido at python.org Thu Mar 16 16:36:52 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 16 Mar 2006 07:36:52 -0800 Subject: [Python-Dev] towards a stricter definition of sys.executable In-Reply-To: References: Message-ID: When I added this my intention was a mixture of (b) and (c) -- I wasn't thinking of situations where there was a difference. (If you remember Python's very early history, embedding wasn't something I had anticipated -- hence the "Great Renaming".) The use that I had in mind does things like os.system(sys.executable + " foo.py") to run foo.py in a separate process. Any additional use (e.g. digging data out of it or finding related files by parsing its pathname) is a highly platform dependent activity; embedding can be seen as a change in platform. For finding related files, sys.exec_prefix and sys.prefix should be used. Digging data out of the file itself never even occurred to me -- it assumes things like executable format etc. that are typically beyond my understanding as a portable language designer. When Python is embedded in another app (e.g. mod_python), I would expect sys.executable to be meaningless and its use to be undefined. None would be a good value in that case. sys.prefix / exec_prefix may or may not have a useful meaning in such an environment, depending on how the standard library is made accessible. Can you say more about the motivation for wanting this reinterpreted? --Guido On 3/16/06, Fredrik Lundh wrote: > the definition of sys.executable is a bit unclear, something that has led to > incompatible use in deployed code. > > the docstring for sys.executable says "pathname of this Python interpreter", > which can be interpreted as either > > a) sys.executable points to the executable that was used to load the > Python interpreter library/dll. > > this use is supported by the docstring and the implementation, and is quite > common in the wild. an application using this interpretation may > > - call sys.executable to run another instance of itself > - extract data from resources embedded in (or attached to) sys.executable > - locate configuration data etc via os.path.dirname(sys.executable) > > etc. > > or > > b) sys.executable points to a standard Python interpreter executable of > the same version, and having the same library, as the currently running > interpreter instance. > > this use is supported by more strict interpretation of the word "interpreter" > (as an executable, rather than an implementation DLL), and is quite common > in the wild. an application using this interpretation may > > - call sys.executable to run a Python script in the same environment as itself. > > etc. > > or > > c) sys.executable points to the file containing the actual ("this") interpreter. I > haven't seen any code that assumes this, probably because no implementation > uses this interpretation... > > for programs that are invoked via the standard interpreter, case (a) and (b) are of > course identical. > > the problem is when the interpreter library is embedded in some other application; > should sys.executable be set to the actual EXE used to start the program, or to > something else ? if it's set to something else, how can that application do the things > that's described under (a) ? > > to fix this, I propose adding another sys variable; for example, let sys.executable > keep behaving like case (a) (which is how it's implemented today), and add a new > sys.python_executable for case (b). the latter can then be set to None if a proper > interpreter cannot be located. > > > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik at pythonware.com Thu Mar 16 17:12:42 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 16 Mar 2006 17:12:42 +0100 Subject: [Python-Dev] towards a stricter definition of sys.executable References: Message-ID: Guido van Rossum wrote: > For finding related files, sys.exec_prefix and sys.prefix should be used except that they're defined in terms of where the standard library is: prefix -- prefix used to find the Python library exec_prefix -- prefix used to find the machine-specific Python library there are scenarios (e.g. with exemaker) where several applications share a Python library / Python DLL, but there's no python.exe in sight. > When Python is embedded in another app (e.g. mod_python), I would > expect sys.executable to be meaningless and its use to be undefined. > None would be a good value in that case. sys.prefix / exec_prefix may > or may not have a useful meaning in such an environment, depending on > how the standard library is made accessible. > > Can you say more about the motivation for wanting this reinterpreted? well, I'm not asking for a reinterpretation as much as a clarification, and possibly some implementation tweaks (since the current code implements approach (a) by default). how about this alternative ? (extended (b)). d) If Python was started from a standard Python interpreter, sys.executable contains the full path to this interpreter. If not, or if the path could not be determined, sys.executable is set to None. If Python is embedded in another environment, that environment may set the executable to a corresponding interpreter, or leave it set to None. An embedding application may set sys.app_executable to point to the application executable. _PySys_Init should probably be modified to set sys.executable to None instead of Py_GetProgramFullPath(); the latter should be done in Py_Main (or in main() ?) when running under a standard interpreter, sys.app_executable could be left undefined, set to None, or set to the same thing as sys.executable. leaving it undefined is good enough for me. From theller at python.net Thu Mar 16 17:35:11 2006 From: theller at python.net (Thomas Heller) Date: Thu, 16 Mar 2006 17:35:11 +0100 Subject: [Python-Dev] towards a stricter definition of sys.executable In-Reply-To: References: Message-ID: Fredrik Lundh wrote: > the definition of sys.executable is a bit unclear, something that has led to > incompatible use in deployed code. > > the docstring for sys.executable says "pathname of this Python interpreter", > which can be interpreted as either > > a) sys.executable points to the executable that was used to load the > Python interpreter library/dll. > > this use is supported by the docstring and the implementation, and is quite > common in the wild. an application using this interpretation may > > - call sys.executable to run another instance of itself > - extract data from resources embedded in (or attached to) sys.executable > - locate configuration data etc via os.path.dirname(sys.executable) > > etc. > > or > > b) sys.executable points to a standard Python interpreter executable of > the same version, and having the same library, as the currently running > interpreter instance. > > this use is supported by more strict interpretation of the word "interpreter" > (as an executable, rather than an implementation DLL), and is quite common > in the wild. an application using this interpretation may > > - call sys.executable to run a Python script in the same environment as itself. > > etc. > > or > > c) sys.executable points to the file containing the actual ("this") interpreter. I > haven't seen any code that assumes this, probably because no implementation > uses this interpretation... > > for programs that are invoked via the standard interpreter, case (a) and (b) are of > course identical. py2exe used the a) interpretation. It uses sys.executable to find the exe that is currently running, for registration of COM servers, and for finding resources in the exe: the manifest file that's needed for GUI applications on WindowsXP to give the native XP look and feel, icons, typelibs, and more. > the problem is when the interpreter library is embedded in some other application; > should sys.executable be set to the actual EXE used to start the program, or to > something else ? if it's set to something else, how can that application do the things > that's described under (a) ? The use case for b) 'call sys.executable to run a Python script' makes no sense for a py2exe'd application. Thomas From fdrake at acm.org Thu Mar 16 17:52:34 2006 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Thu, 16 Mar 2006 11:52:34 -0500 Subject: [Python-Dev] towards a stricter definition of sys.executable In-Reply-To: References: Message-ID: <200603161152.34271.fdrake@acm.org> Guido van Rossum wrote: > Can you say more about the motivation for wanting this reinterpreted? Fredrik Lundh wrote: > d) If Python was started from a standard Python interpreter, My understanding matches Guido's description, so I'm not sure any changes are needed. Since an embedding application can provide an alternate API to the needed information, that seems sufficient. That said, option "d" seems like a reasonable approach as well, and I'd have no objection to the change. I suggest the sys.application instead of sys.app_executable for the new variable, but that's minor. -Fred -- Fred L. Drake, Jr. From ndbecker2 at gmail.com Thu Mar 16 18:06:16 2006 From: ndbecker2 at gmail.com (Neal Becker) Date: Thu, 16 Mar 2006 12:06:16 -0500 Subject: [Python-Dev] Problem with module loading on multi-arch? Message-ID: I just installed TwistedSumo-2006-02-12 on x86_64, and noticed a problem. It installs arch-dep stuff into /usr/lib64/python2.4/site-packages/twisted, and arch-indep into /usr/lib/python2.4/site-packages/twisted as it should. But: from twisted.web import html exceptions.ImportError: No module named web I'm guessing that what's happening is that since there is an /twisted, we never find the module /twisted/web. If my analysis (guess) is correct, I think we have a problem with the module search. From jcarlson at uci.edu Thu Mar 16 18:13:29 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu, 16 Mar 2006 09:13:29 -0800 Subject: [Python-Dev] bytes thoughts In-Reply-To: References: <440669B9.3000609@canterbury.ac.nz> Message-ID: <20060316090941.F73B.JCARLSON@uci.edu> "Christos Georgiou" wrote: > Well, what's the result of > > bytes([1,0,0])^ bytes([1,0]) > > ? Is it bytes([0,0,0]) (? la little-endian) or is it bytes([1,1,0]) > (straight conversion to base-256)? Or perhaps throw a ValueError if the > sizes differ? It's a ValueError. If the sizes matched, it would be a per-element bitwise xor. > These details should be considered in the PEP. They aren't considered because they are *obvious* to most (if not all) sane people who use Python. Think of future bytes behavior to be similar to current array behavior, with a few bits and pieces added to make life easier (like all of the current string methods, etc.) - Josiah From trentm at ActiveState.com Thu Mar 16 18:45:38 2006 From: trentm at ActiveState.com (Trent Mick) Date: Thu, 16 Mar 2006 09:45:38 -0800 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <20060314215550.GC5358@activestate.com> References: <20060314010819.GB29489@activestate.com> <20060314034831.GE10510@activestate.com> <1f7befae0603140755k66ff91c1jb41917d21efb5d41@mail.gmail.com> <44172F38.1090909@v.loewis.de> <20060314215550.GC5358@activestate.com> Message-ID: <20060316174538.GA924@activestate.com> [Trent Mick wrote] > [Martin v. Loewis wrote] > > Tim Peters wrote: > > > I'd say instead that they should never be skipped: the real > > > difference on your box is the expected _outcome_ in the third > > > category. > > > > That is indeed more reasonable than what I proposed. > > I'll do this tonight or tomorrow. Done now (finally). Trent -- Trent Mick TrentM at ActiveState.com From edcjones at comcast.net Thu Mar 16 18:53:55 2006 From: edcjones at comcast.net (Edward C. Jones) Date: Thu, 16 Mar 2006 12:53:55 -0500 Subject: [Python-Dev] Python Library Reference top page too long Message-ID: <4419A633.6010607@comcast.net> The contents page for the Python Library Reference ("http://docs.python.org/dev/lib/lib.html") has become much too long. I suggest that it should be designed like the top page for portal web sites. For example see "http://www.dmoz.org/". I suggest that "lib.html" be replaced by "lib_index.html" and "lib_contents.html". The latter is the current "lib.html". "lib_index.html" should look something like: Front Matter Full Contents 1. Introduction 2. Built-in Objects functions, types, exceptions, constants 3. File and Directory Access open, file objects, os, os.path, ... 4. Generic Operating System Services os, time, optparse, ... 5. Python Runtime Services sys, __main__, __future__, ... 6. String Services type str, string, re, ... 7. Numeric and Mathematical Modules math, random, decimal, ... ... Both "lib_index.html" and "lib_contents.html" should be accessible from the top Python Documentation page. Here are two principles I feel would be useful to follow. References are to the Python 2.5 documentation at "http://docs.python.org/dev/lib/lib.html". 1. Important, commonly used, or confusing modules should be near the top. The things I look up the most are os.path, os (files and directories), and string attributes. "sys" should also be near the top because of "sys.argv", "sys.stderr", and "sys.exit". Etc, etc. 2. Put cross-links everywhere. For example the documentation for built in function "file" has a link to "File Objects". There should also be a link to "open()" and to sections 13.1.2, 13.1.3, 13.1.4, and chapter 10. (What became of the often proposed and long overdue reorganization of "os"?) From trentm at ActiveState.com Thu Mar 16 19:00:28 2006 From: trentm at ActiveState.com (Trent Mick) Date: Thu, 16 Mar 2006 10:00:28 -0800 Subject: [Python-Dev] towards a stricter definition of sys.executable In-Reply-To: References: Message-ID: <20060316180028.GB924@activestate.com> > Fredrik Lundh wrote: > > a) sys.executable points to the executable that was used to load the > > Python interpreter library/dll. > > > > this use is supported by the docstring and the implementation, and is quite > > common in the wild. an application using this interpretation may > > > > - call sys.executable to run another instance of itself > > - extract data from resources embedded in (or attached to) sys.executable > > - locate configuration data etc via os.path.dirname(sys.executable) > > > > etc. [Thomas Heller wrote] > py2exe used the a) interpretation. It uses sys.executable to find the exe that is currently > running, for registration of COM servers, and for finding resources in the exe: the manifest > file that's needed for GUI applications on WindowsXP to give the native XP look and feel, > icons, typelibs, and more. > > The use case for b) 'call sys.executable to run a Python script' makes no sense for > a py2exe'd application. Ditto on both counts for PyXPCOM (Python embedded in Mozilla). Trent -- Trent Mick TrentM at ActiveState.com From fredrik at pythonware.com Thu Mar 16 19:40:02 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 16 Mar 2006 19:40:02 +0100 Subject: [Python-Dev] towards a stricter definition of sys.executable References: <20060316180028.GB924@activestate.com> Message-ID: > > > a) sys.executable points to the executable that was used to load the > > > Python interpreter library/dll. > > > > > > this use is supported by the docstring and the implementation, and is quite > > > common in the wild. an application using this interpretation may Thomas: > > py2exe used the a) interpretation. /.../ > > The use case for b) 'call sys.executable to run a Python script' makes > > no sense for a py2exe'd application. Trent: > Ditto on both counts for PyXPCOM (Python embedded in Mozilla). Looks like I might have to withdraw my (d) proposal, and, once again, suggest that we stick to the "GetProgramFullPath" sense, as implemented, and add a new variable for the originally intended but not really implemented "GetInter- preterPath" sense... Guido ? From tim.peters at gmail.com Thu Mar 16 20:22:59 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 16 Mar 2006 14:22:59 -0500 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <20060316174538.GA924@activestate.com> References: <20060314010819.GB29489@activestate.com> <20060314034831.GE10510@activestate.com> <1f7befae0603140755k66ff91c1jb41917d21efb5d41@mail.gmail.com> <44172F38.1090909@v.loewis.de> <20060314215550.GC5358@activestate.com> <20060316174538.GA924@activestate.com> Message-ID: <1f7befae0603161122p55e89e9egce49afd683aecef8@mail.gmail.com> [Trent Mick, on changing test_winsound to expect RuntimeError on a box without a sound card] > Done now (finally). Cool -- thanks! I merged that to the 2.4 branch, and (of course) your buildbot slave passes the tests on that too now. Green is a lovely color :-) From pje at telecommunity.com Thu Mar 16 20:29:50 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 16 Mar 2006 14:29:50 -0500 Subject: [Python-Dev] towards a stricter definition of sys.executable In-Reply-To: References: <20060316180028.GB924@activestate.com> Message-ID: <5.1.1.6.0.20060316142527.02fb38b0@mail.telecommunity.com> At 07:40 PM 3/16/2006 +0100, Fredrik Lundh wrote: >Looks like I might have to withdraw my (d) proposal, and, once again, suggest >that we stick to the "GetProgramFullPath" sense, as implemented, and add a >new variable for the originally intended but not really implemented "GetInter- >preterPath" sense... Note that the stdlib and tools often use the (b) interpretation. For example, the distutils does byte-compilation of files by invoking sys.executable. So these would need to be changed. (This in fact is why I couldn't use exemaker to wrap easy_install on Windows and had to create my own wrapper that exec()'s Python rather than linking to it.) Basically, no matter how this is clarified, some code somewhere is going to have to change. (On the bright side, we could get rid of the distutils hack in 2.5 if compile() allowed you to set the optimization level.) From ulrich.berning at t-online.de Thu Mar 16 21:38:50 2006 From: ulrich.berning at t-online.de (Ulrich Berning) Date: Thu, 16 Mar 2006 21:38:50 +0100 Subject: [Python-Dev] towards a stricter definition of sys.executable In-Reply-To: References: Message-ID: <4419CCDA.7020500@t-online.de> Fredrik Lundh schrieb: >how about this alternative ? (extended (b)). > > d) If Python was started from a standard Python interpreter, > sys.executable contains the full path to this interpreter. If not, > or if the path could not be determined, sys.executable is set to > None. > > Our registration code for Windows services and COM servers and some other specific things rely on the fact, that sys.executable contains the name of the binary, that is actually running (either the full path of python[.exe] or the full path of the frozen application executable), so please don't touch sys.executable. Ulli From theller at python.net Thu Mar 16 21:45:46 2006 From: theller at python.net (Thomas Heller) Date: Thu, 16 Mar 2006 21:45:46 +0100 Subject: [Python-Dev] [Python-checkins] r43041 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: <4418A45D.7090500@v.loewis.de> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> Message-ID: <4419CE7A.2030500@python.net> Martin v. L?wis wrote: > Thomas Heller wrote: >> BTW: Is a "porting guide" to make extension modules compatible with 2.5 >> available somewhere? PEP 353 scratches only the surface... > > Wrt. ssize_t changes, PEP 353 is meant to be comprehensive. Which > particular aspect are you missing? I suggest to change this: #if PY_VERSION_HEX < 0x02050000 typedef int Py_ssize_t; #endif with this: #if (PY_VERSION_HEX < 0x02050000) typedef int Py_ssize_t; #define lenfunc inquiry #define readbufferproc getreadbufferproc #define writebufferproc getwritebufferproc #define segcountproc getsegcountproc #define charbufferproc getcharbufferproc #define ssizeargfunc intargfunc #define ssizessizeargfunc intintargfunc #define ssizeobjargproc intobjargproc #define ssizessizeobjargproc intintobjargproc ... more defines #endif Maybe a complete list of defines needed can be given? Then, from only reading the PEP without looking up the sources, it is not clear to me what the PY_SIZE_T_CLEAN definition does. Finally, the format codes to use for Py_ssize_t arguments passed to PyBuild_Value, PyString_FromFormat, PyObject_CallFunction (and other functions) are not mentioned at all. Thanks, Thomas From martin at v.loewis.de Thu Mar 16 22:34:38 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 16 Mar 2006 22:34:38 +0100 Subject: [Python-Dev] Problem with module loading on multi-arch? In-Reply-To: References: Message-ID: <4419D9EE.4040603@v.loewis.de> Neal Becker wrote: > I'm guessing that what's happening is that since there is an > /twisted, we never find the module /twisted/web. If > my analysis (guess) is correct, I think we have a problem with the module > search. That is quite possible. I keep applying patches from people who claim to know how things on AMD64 linux work, much without questioning them. It is quite possible that one of these patches broke something. I'm happy to revert or augment them if needed. Somebody should define how things ought to work, implement a patch that both includes a documentation and an implementation of these decisions, and then keep an eye on patches that might break this design. Regards, Martin From baptiste13 at altern.org Thu Mar 16 22:32:09 2006 From: baptiste13 at altern.org (Baptiste Carvello) Date: Thu, 16 Mar 2006 22:32:09 +0100 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: <4418CDD0.8020803@canterbury.ac.nz> References: <4418CDD0.8020803@canterbury.ac.nz> Message-ID: Greg Ewing a ?crit : > except as : > what about except with : a program dies "with" an error message, not "as" an error message. ciao, BC From martin at v.loewis.de Thu Mar 16 22:36:28 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 16 Mar 2006 22:36:28 +0100 Subject: [Python-Dev] Python Library Reference top page too long In-Reply-To: <4419A633.6010607@comcast.net> References: <4419A633.6010607@comcast.net> Message-ID: <4419DA5C.2000101@v.loewis.de> Edward C. Jones wrote: > The contents page for the Python Library Reference > ("http://docs.python.org/dev/lib/lib.html") has become much too long. I disagree. It serves my purposes very well: I usually search in the page for a keywork I think should be there. If the page was broken into multiple pages, that would break. Regards, Martin From rowen at cesmail.net Thu Mar 16 22:56:52 2006 From: rowen at cesmail.net (Russell E. Owen) Date: Thu, 16 Mar 2006 13:56:52 -0800 Subject: [Python-Dev] Py3k: Except clause syntax References: <4418CDD0.8020803@canterbury.ac.nz> Message-ID: In article <4418CDD0.8020803 at canterbury.ac.nz>, Greg Ewing wrote: > For Py3k, any thoughts on changing the syntax of > the except clause from > > except , : > > to > > except as : > > so that things like > > except TypeError, ValueError: > > will do what is expected? Brilliant. I may be showing my clumsiness, but I catch multiple exceptions so rarely that I often stumble over this. -- Russell From barry at python.org Thu Mar 16 23:25:39 2006 From: barry at python.org (Barry Warsaw) Date: Thu, 16 Mar 2006 17:25:39 -0500 Subject: [Python-Dev] towards a stricter definition of sys.executable In-Reply-To: References: Message-ID: <1142547940.13342.30.camel@resist.wooz.org> On Thu, 2006-03-16 at 12:02 +0100, Fredrik Lundh wrote: > a) sys.executable points to the executable that was used to load the > Python interpreter library/dll. > > this use is supported by the docstring and the implementation, and is quite > common in the wild. an application using this interpretation may > > - call sys.executable to run another instance of itself > - extract data from resources embedded in (or attached to) sys.executable > - locate configuration data etc via os.path.dirname(sys.executable) Yep, that's how our embedded apps use sys.executable. > to fix this, I propose adding another sys variable; for example, let sys.executable > keep behaving like case (a) (which is how it's implemented today), and add a new > sys.python_executable for case (b). the latter can then be set to None if a proper > interpreter cannot be located. As long as sys.executable behaving like a) doesn't change, that seems fine with me. ;) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060316/62ca093a/attachment.pgp From rowen at cesmail.net Thu Mar 16 23:41:37 2006 From: rowen at cesmail.net (Russell E. Owen) Date: Thu, 16 Mar 2006 14:41:37 -0800 Subject: [Python-Dev] Python Library Reference top page too long References: <4419A633.6010607@comcast.net> Message-ID: In article <4419A633.6010607 at comcast.net>, "Edward C. Jones" wrote: > The contents page for the Python Library Reference > ("http://docs.python.org/dev/lib/lib.html") has become much too long. I > suggest that it should be designed like the top page for portal web > sites. For example see "http://www.dmoz.org/". I suggest that "lib.html" > be replaced by "lib_index.html" and "lib_contents.html". The latter is > the current "lib.html". "lib_index.html" should look something like: > > Front Matter > Full Contents > 1. Introduction > 2. Built-in Objects > functions, types, exceptions, constants > 3. File and Directory Access > open, file objects, os, os.path, ... > 4. Generic Operating System Services > os, time, optparse, ... > 5. Python Runtime Services > sys, __main__, __future__, ... > 6. String Services > type str, string, re, ... > 7. Numeric and Mathematical Modules > math, random, decimal, ... I agree it needs work and your suggestion sounds promising. With the manual as it stands, I find it *very* difficult to find what I want. Here are two typical queries I find painful: 1) Look up a string method. - I usually think "OK, a string is a sequence type, so jump there". - Jump to that link and you get a page that is completely irrelevant *except* that at the *bottom* (if one gets down that far) is a nice list of subtopics, one of which is the "right" link. The real problem here is that if one jumps too deeply on the first go, the page doesn't list sub-topics and related topics at the beginning. Instead maybe there's a "subtopics" section at the end, but that's a lousy place for it. (Thus if I was less clever and just jumped to "Built-in Classes" I'd get a nicely detailed TOC that has exactly the link I want.) 2) Look up the special methods for a sequence object (the __xxx___ methods that make one's own class look like a sequence). I've found them before but every time I want them it's another @#$%@#$-laden wade through the manual (I just tried to find them again--no luck). Thank god for Python Essential Reference, dated as it is. Fundamentally I think what's wanted is: - Another level of sub-TOCs, e.g. one for "Sequence Types", "Mapping Types", etc. Every page that has sub-topics or intimately related should have a list of them at the beginning. - The special methods for a given type of class would make a really good "intimately related" topic for "Sequence Types" and "Mapping Types". Another thing that would be *really* useful is to list the actual built-in types with the category. For example: Sequence Types (str, unicode, list, tuple, buffer, xrange) Mapping Types (dict) -- Russell From edloper at gradient.cis.upenn.edu Thu Mar 16 23:28:55 2006 From: edloper at gradient.cis.upenn.edu (Edward Loper) Date: Thu, 16 Mar 2006 17:28:55 -0500 Subject: [Python-Dev] Python Library Reference top page too long In-Reply-To: <4419DA5C.2000101@v.loewis.de> References: <4419A633.6010607@comcast.net> <4419DA5C.2000101@v.loewis.de> Message-ID: <4419E6A7.2060008@gradient.cis.upenn.edu> Martin v. L?wis wrote: > Edward C. Jones wrote: >> The contents page for the Python Library Reference >> ("http://docs.python.org/dev/lib/lib.html") has become much too long. > > I disagree. It serves my purposes very well: I usually search in the > page for a keywork I think should be there. If the page was broken > into multiple pages, that would break. Perhaps the lib page could have a portal-like index like the one Edward Jones suggested at the top, followed the full list w/ descriptions? That way, it would be easier to pick out a module with a fairly quick glance, but searching the page would still work. -Edward From baptiste13 at altern.org Thu Mar 16 23:51:00 2006 From: baptiste13 at altern.org (Baptiste Carvello) Date: Thu, 16 Mar 2006 23:51:00 +0100 Subject: [Python-Dev] bytes thoughts In-Reply-To: <20060316090941.F73B.JCARLSON@uci.edu> References: <440669B9.3000609@canterbury.ac.nz> <20060316090941.F73B.JCARLSON@uci.edu> Message-ID: Josiah Carlson a ?crit : > They aren't considered because they are *obvious* to most (if not all) > sane people who use Python. They are not *that* obvious. Logical operations on ints have allowed users to forget about size (and shoot themselves in the foot from time to time). Or is 1^(~1) == -1 obvious ? Well, maybe that's not sane either :-) > Think of future bytes behavior to be > similar to current array behavior, with a few bits and pieces added to > make life easier (like all of the current string methods, etc.) > I didn't get before that the array behavior really *defined* the bytes behavior. OK, I'll keep that in mind. Cheers, BC From nidoizo at yahoo.com Thu Mar 16 23:58:27 2006 From: nidoizo at yahoo.com (Nicolas Fleury) Date: Thu, 16 Mar 2006 17:58:27 -0500 Subject: [Python-Dev] Making staticmethod objects callable? In-Reply-To: References: <20060312153236.GB16848@code0.codespeak.net> Message-ID: Guido van Rossum wrote: > There's no need to change Python so that people coming from other > languages won't make silly mistakes, is there? Is that really a mistake... Yes, it's a mistake since staticmethod is a descriptor, but isn't it in a sense an implementation detail, particularly for a newbie? As Steven pointed, it is forcing to learn about descriptors. But I don't feel comfortable with that; I've always seen Python as a language that you can use with minimal knowledge. Again, I'm not thinking about anyone on this list. > BTW I question the claimed reflex -- assuming by "other languages" you > mean Java or C++ (the only languages I know that *have* static > methods) -- since those languages don't have the ability to call > methods (static or otherwise) at class definition time. Java, C++, C#. Yes, you're right, but the way I see it the first thing you learn in Python is that everything is dynamic. So I understand the reflex to quickly adapt the way you code to the new capabilities of Python. > So perhaps you need to dig deeper to find out *why* this is a recurring issue. I think I understand why this is a recurring issue, but maybe I'm not good at explaining why. In the end, on that specific issue I think there's something to improve for newbies. The error message "'staticmethod' object is not callable" could also be changed to something like "'staticmethod' object is a descriptor and is not callable". Personally, I prefer the "it just works to ease your life" compromise (it doesn't hurt much, no?). Note that it's not a big deal anyway and I hope it doesn't look like I want to argue; I just want to explain the issue. Regards, Nicolas From mal at egenix.com Fri Mar 17 00:11:03 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 17 Mar 2006 00:11:03 +0100 Subject: [Python-Dev] [Python-checkins] r43041 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: <4419CE7A.2030500@python.net> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> Message-ID: <4419F087.5080207@egenix.com> Thomas Heller wrote: > Martin v. L?wis wrote: >> Thomas Heller wrote: >>> BTW: Is a "porting guide" to make extension modules compatible with 2.5 >>> available somewhere? PEP 353 scratches only the surface... >> Wrt. ssize_t changes, PEP 353 is meant to be comprehensive. Which >> particular aspect are you missing? > > I suggest to change this: > > #if PY_VERSION_HEX < 0x02050000 > typedef int Py_ssize_t; > #endif > > with this: > > #if (PY_VERSION_HEX < 0x02050000) > typedef int Py_ssize_t; > #define lenfunc inquiry > #define readbufferproc getreadbufferproc > #define writebufferproc getwritebufferproc > #define segcountproc getsegcountproc > #define charbufferproc getcharbufferproc > #define ssizeargfunc intargfunc > #define ssizessizeargfunc intintargfunc > #define ssizeobjargproc intobjargproc > #define ssizessizeobjargproc intintobjargproc > ... more defines > #endif > > Maybe a complete list of defines needed can be given? > > Then, from only reading the PEP without looking up the sources, > it is not clear to me what the PY_SIZE_T_CLEAN definition does. > > Finally, the format codes to use for Py_ssize_t arguments passed to PyBuild_Value, > PyString_FromFormat, PyObject_CallFunction (and other functions) are not mentioned at all. Since this change is going to affect a lot of 3rd party extensions, I'd also like to see a complete list of public APIs that changed and how they changed (including the type slots) Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 16 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From martin at v.loewis.de Fri Mar 17 00:21:28 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 17 Mar 2006 00:21:28 +0100 Subject: [Python-Dev] [Python-checkins] r43041 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: <4419F087.5080207@egenix.com> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> Message-ID: <4419F2F8.9040900@v.loewis.de> M.-A. Lemburg wrote: > Since this change is going to affect a lot of 3rd party extensions, > I'd also like to see a complete list of public APIs that changed and > how they changed (including the type slots) You can construct this list easily by comparing the header files of the previous and the current release. Please contribute a patch that presents these changes in a form that you would consider acceptable. Regards, Martin From greg.ewing at canterbury.ac.nz Fri Mar 17 01:09:46 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 17 Mar 2006 13:09:46 +1300 Subject: [Python-Dev] bytes thoughts In-Reply-To: References: <440669B9.3000609@canterbury.ac.nz> Message-ID: <4419FE4A.9060902@canterbury.ac.nz> Christos Georgiou wrote: > Well, what's the result of > > bytes([1,0,0])^ bytes([1,0]) > > ? Is it bytes([0,0,0]) (? la little-endian) or is it bytes([1,1,0]) > (straight conversion to base-256)? Or perhaps throw a ValueError if the > sizes differ? In the interests of refusing the temptation to guess, I'd go for the ValueError. Greg From jcarlson at uci.edu Fri Mar 17 01:47:37 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu, 16 Mar 2006 16:47:37 -0800 Subject: [Python-Dev] bytes thoughts In-Reply-To: References: <20060316090941.F73B.JCARLSON@uci.edu> Message-ID: <20060316163605.F753.JCARLSON@uci.edu> Baptiste Carvello wrote: > Josiah Carlson a ?crit : > > > They aren't considered because they are *obvious* to most (if not all) > > sane people who use Python. > > They are not *that* obvious. Logical operations on ints have allowed users to > forget about size (and shoot themselves in the foot from time to time). Or is > 1^(~1) == -1 obvious ? Well, maybe that's not sane either :-) I find most operations involving ~ to be insane (just a personal opinion), but in this case, if one assumes two's compliment arithmetic (which seems fairly reasonable considering current standard platforms), an integer, all of whose bits are 1, dictates that the value be -1. An insane use of ~, but a sane result. *shrug* - Josiah From ndbecker2 at gmail.com Fri Mar 17 01:45:33 2006 From: ndbecker2 at gmail.com (Neal Becker) Date: Thu, 16 Mar 2006 19:45:33 -0500 Subject: [Python-Dev] Problem with module loading on multi-arch? References: <4419D9EE.4040603@v.loewis.de> Message-ID: "Martin v. L?wis" wrote: > Neal Becker wrote: >> I'm guessing that what's happening is that since there is an >> /twisted, we never find the module /twisted/web. >> If my analysis (guess) is correct, I think we have a problem with the >> module search. > > That is quite possible. I keep applying patches from people who claim to > know how things on AMD64 linux work, much without questioning them. It > is quite possible that one of these patches broke something. I'm happy > to revert or augment them if needed. > > Somebody should define how things ought to work, implement a patch that > both includes a documentation and an implementation of these decisions, > and then keep an eye on patches that might break this design. > OK, let's talk about it. 1. Does it make sense to have both /app/subpackage and /app/subpackage ? My answer: definitely yes. We already agree that we should have both site-dep for binary code and site-indep for python code, so there is no way to avoid this. 2. What should the module search do? I don't know the details of the current algorithm, but clearly it's going to have to deal correctly with the above. One possibility (and maybe this is how it already works?) is that the module search doesn't know anything about site-dep/site-indep, it simply has a list of paths to search. If I did want to try to fix this myself, where would I find the code for it? From greg.ewing at canterbury.ac.nz Fri Mar 17 01:40:06 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 17 Mar 2006 13:40:06 +1300 Subject: [Python-Dev] bytes thoughts In-Reply-To: References: <440669B9.3000609@canterbury.ac.nz> <20060316090941.F73B.JCARLSON@uci.edu> Message-ID: <441A0566.3040201@canterbury.ac.nz> Baptiste Carvello wrote: > They are not *that* obvious. Logical operations on ints have allowed users to > forget about size (and shoot themselves in the foot from time to time). Or is > 1^(~1) == -1 obvious ? Well, maybe that's not sane either :-) It's about as sane as you can get in a world where ints don't have any fixed size. Bytes objects will have a size, on the other hand, and it makes sense to take notice of it. BTW, is anyone else bothered that the term "bytes object" is a bit cumbersome? Also confusing as to whether it's singular or plural. Could we perhaps call it a bytevector or bytearray or something? Greg From greg.ewing at canterbury.ac.nz Fri Mar 17 01:47:09 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 17 Mar 2006 13:47:09 +1300 Subject: [Python-Dev] Python Library Reference top page too long In-Reply-To: References: <4419A633.6010607@comcast.net> Message-ID: <441A070D.10002@canterbury.ac.nz> Russell E. Owen wrote: > Fundamentally I think what's wanted is: > - Another level of sub-TOCs, e.g. one for "Sequence Types", "Mapping > Types", etc. Every page that has sub-topics or intimately related should > have a list of them at the beginning. > - The special methods for a given type of class would make a really good > "intimately related" topic for "Sequence Types" and "Mapping Types". +1 to all that. > Another thing that would be *really* useful is to list the actual > built-in types with the category. For example: > Sequence Types (str, unicode, list, tuple, buffer, xrange) > Mapping Types (dict) +1 Greg From greg.ewing at canterbury.ac.nz Fri Mar 17 01:47:16 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 17 Mar 2006 13:47:16 +1300 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: References: <4418CDD0.8020803@canterbury.ac.nz> Message-ID: <441A0714.7040200@canterbury.ac.nz> Baptiste Carvello wrote: > what about > > except with : > > a program dies "with" an error message, not "as" an error message. No. The exception object you're catching *is* the value, not something which *has* a value. I maintain that "as" is the correct word to use here. Greg From guido at python.org Fri Mar 17 02:39:50 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 16 Mar 2006 17:39:50 -0800 Subject: [Python-Dev] bytes thoughts In-Reply-To: <441A0566.3040201@canterbury.ac.nz> References: <440669B9.3000609@canterbury.ac.nz> <20060316090941.F73B.JCARLSON@uci.edu> <441A0566.3040201@canterbury.ac.nz> Message-ID: On 3/16/06, Greg Ewing wrote: > BTW, is anyone else bothered that the term "bytes object" > is a bit cumbersome? Also confusing as to whether it's > singular or plural. Could we perhaps call it a bytevector > or bytearray or something? Doesn't really bother me. You could call it a bytes array (especially since I expect it will start off as a subclass of the array.array class). I'd still like the built-in name to be 'bytes'. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tjreedy at udel.edu Fri Mar 17 03:22:17 2006 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 16 Mar 2006 21:22:17 -0500 Subject: [Python-Dev] Py3k: Except clause syntax References: <4418CDD0.8020803@canterbury.ac.nz> <441A0714.7040200@canterbury.ac.nz> Message-ID: "Greg Ewing" wrote in message news:441A0714.7040200 at canterbury.ac.nz... > Baptiste Carvello wrote: > >> what about >> >> except with : >> >> a program dies "with" an error message, not "as" an error message. > > No. The exception object you're catching *is* the value, > not something which *has* a value. I maintain that "as" > is the correct word to use here. Besides which, 'with' is a (new) main (compound statement) keyword while 'as' is always subordinate and denoting a name to be used in the sequel. And I also like using it to separate classes to be caught from name binding of instance. tjr From mhammond at skippinet.com.au Fri Mar 17 03:44:40 2006 From: mhammond at skippinet.com.au (Mark Hammond) Date: Fri, 17 Mar 2006 13:44:40 +1100 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <44186467.9060102@v.loewis.de> Message-ID: > So PythonWin needs to be installed on a Windows buildbot slave, right? FWIW, we are having reasonable success with buildbot service packaged as a py2exe application - just unzip into a directory and go! This has the primary advantage (to me!) of not using the Python installed on the box, thereby avoiding inconvenient "file is in use" errors when you do need to update that Python or similar. As I've been chatting with Trent about recently, I intend updating my buildbot patch to use the win32 "Job" related functions to manage the build subtasks more effectively and potentially guard against the buildbot using too many resources. Sadly though, my buildbot patch has been languishing in their collector without comment, so my motivation is fairly low. Cheers, Mark From skip at pobox.com Fri Mar 17 04:05:33 2006 From: skip at pobox.com (skip at pobox.com) Date: Thu, 16 Mar 2006 21:05:33 -0600 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: References: <4418CDD0.8020803@canterbury.ac.nz> Message-ID: <17434.10109.675484.243338@montanaro.dyndns.org> Greg> except as : Baptiste> except with : Can I catch multiple exceptions with a single value in this case? Today, I write: try: foo() except (TypeError, KeyError), msg: print msg Either of the above seem like they'd require me to repeat the value, e.g: try: foo() except TypeError with msg, KeyError with msg: print msg Not very Pythonic methinks. Skip From brett at python.org Fri Mar 17 04:30:01 2006 From: brett at python.org (Brett Cannon) Date: Thu, 16 Mar 2006 19:30:01 -0800 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: <441A0714.7040200@canterbury.ac.nz> References: <4418CDD0.8020803@canterbury.ac.nz> <441A0714.7040200@canterbury.ac.nz> Message-ID: On 3/16/06, Greg Ewing wrote: > Baptiste Carvello wrote: > > > what about > > > > except with : > > > > a program dies "with" an error message, not "as" an error message. > > No. The exception object you're catching *is* the value, > not something which *has* a value. I maintain that "as" > is the correct word to use here. I agree. "as" is taking on the use of assignment in statements that are not ``=`` and I say we just keep on with that. Plus Greg's above explanation also makes sense to me; you are binding the exception to a name and treating as if it was called . -Brett From rrr at ronadam.com Fri Mar 17 04:32:54 2006 From: rrr at ronadam.com (Ron Adam) Date: Thu, 16 Mar 2006 21:32:54 -0600 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: <441A0714.7040200@canterbury.ac.nz> References: <4418CDD0.8020803@canterbury.ac.nz> <441A0714.7040200@canterbury.ac.nz> Message-ID: <441A2DE6.2010203@ronadam.com> Greg Ewing wrote: > Baptiste Carvello wrote: > >> what about >> >> except with : >> >> a program dies "with" an error message, not "as" an error message. > > No. The exception object you're catching *is* the value, > not something which *has* a value. I maintain that "as" > is the correct word to use here. > > Greg I think it reads well with "as" also. +1 To me it's a filter statement. So would catching multiple exceptions have the form: except (, ) as : The value is bound to the name if it's type is in the sequence. This would be an indirect assignment similar to. if value in list: name = value Which you can't do directly because you don't have access to the value yet. That's not too different than import which is also an indirect name binding operation of a value you don't have yet. Cheers, Ron From aleaxit at gmail.com Fri Mar 17 06:04:36 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Thu, 16 Mar 2006 21:04:36 -0800 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: References: <4418CDD0.8020803@canterbury.ac.nz> <441A0714.7040200@canterbury.ac.nz> Message-ID: <50372CD0-B314-4FED-8C57-3F563F459E09@gmail.com> On Mar 16, 2006, at 7:30 PM, Brett Cannon wrote: ... > I agree. "as" is taking on the use of assignment in statements that > are not ``=`` and I say we just keep on with that. Plus Greg's above Hmmm, if we allowed '( as )' for generic expr's we'd make a lot of people pining for 'assignment as an expression' very happy, wouldn't we...? Alex From kbk at shore.net Fri Mar 17 06:36:20 2006 From: kbk at shore.net (Kurt B. Kaiser) Date: Fri, 17 Mar 2006 00:36:20 -0500 (EST) Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200603170536.k2H5aK9n007030@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 392 open ( +6) / 3094 closed ( +5) / 3486 total (+11) Bugs : 907 open (+18) / 5646 closed (+10) / 6553 total (+28) RFE : 213 open ( +1) / 202 closed ( +1) / 415 total ( +2) New / Reopened Patches ______________________ new exit and quit objects (2006-03-09) CLOSED http://python.org/sf/1446372 opened by Georg Brandl zipfile: support for ZIP64 (2006-03-09) http://python.org/sf/1446489 opened by Ronald Oussoren patch for 1441408 compiler fails to spot extended slice (2006-03-09) http://python.org/sf/1446847 opened by Grant Olson Patch for bug 1441486: bad unary minus folding in compiler (2006-03-09) http://python.org/sf/1446922 opened by Grant Olson logging: findCaller() sometimes raises AttributeError (2006-03-10) CLOSED http://python.org/sf/1447410 opened by Kevin J Bluck ConnectRegistry blocks all threads (2006-03-12) http://python.org/sf/1448199 opened by Lars L decorator module (2006-03-12) http://python.org/sf/1448297 opened by Georg Brandl declspec for ssize_t (2006-03-12) http://python.org/sf/1448484 opened by Zooko O'Whielacronx const PyDict_Type ? (2006-03-12) http://python.org/sf/1448488 opened by Zooko O'Whielacronx email.message.py charset can be unicode instance (2006-03-13) http://python.org/sf/1449244 opened by Tokio Kikuchi Splitting CJK codecs from pythoncore dll (2006-03-14) http://python.org/sf/1449471 opened by Hye-Shik Chang Patches Closed ______________ new exit and quit objects (2006-03-09) http://python.org/sf/1446372 closed by gbrandl logging: findCaller() sometimes raises AttributeError (2006-03-10) http://python.org/sf/1447410 closed by vsajip Several minor fixes for NULL checks, etc. (2006-03-06) http://python.org/sf/1444030 closed by nnorwitz New with semantics (2006-03-08) http://python.org/sf/1445739 closed by nnorwitz Python memory allocator: Free memory (2005-02-15) http://python.org/sf/1123430 closed by tim_one New / Reopened Bugs ___________________ extended slice behavior inconsistent with docs (2006-03-09) http://python.org/sf/1446619 opened by Steven Bethard bug with xmlrpclib (2006-03-09) http://python.org/sf/1446690 opened by varun bhansaly whatsnew 2.5: pep 353, python -m (2006-03-09) http://python.org/sf/1447031 opened by Gene Tani tkinter Dialog fails when more than four buttons are used (2006-03-10) http://python.org/sf/1447222 opened by Hernan P. Dacharry tkinter Dialog fails when more than four buttons are used (2006-03-10) CLOSED http://python.org/sf/1447234 opened by Hernan P. Dacharry Mac Framework build fails on intel (2006-03-10) http://python.org/sf/1447587 opened by Michael Mondragon make frameworkinstall fails on Intel Macs (2006-03-10) http://python.org/sf/1447607 opened by Michael Mondragon "reindent.py" exposes bug in tokenize (2006-03-10) CLOSED http://python.org/sf/1447633 opened by Edward C. Jones traceback.format_exception_only() and SyntaxError (2006-03-11) http://python.org/sf/1447885 opened by Remy Blank Unable to stringify datetime with tzinfo (2006-03-11) http://python.org/sf/1447945 opened by Ilpo Nyyss?nen Defining a class with __dict__ brakes attributes assignment (2006-03-11) http://python.org/sf/1448042 opened by Michal Kwiatkowski problems with too many sockets in FreeBSD (2006-03-12) http://python.org/sf/1448058 opened by aix-d gettext.py breaks on plural-forms header (2006-03-12) http://python.org/sf/1448060 opened by Danilo Segan re search infinite loop (2006-03-12) http://python.org/sf/1448325 opened by Don Allen Convertion error for latin1 characters with iso-2022-jp-2 (2006-03-12) CLOSED http://python.org/sf/1448490 opened by Francois Duranleau asyncore dispatcher.__getattr__() masks self._map (2006-03-12) http://python.org/sf/1448639 opened by Doug White datetime.__init__ cannot be overridden (2006-03-13) CLOSED http://python.org/sf/1448640 opened by Martin Blais Subscript operations generating incorrect code (2006-03-13) CLOSED http://python.org/sf/1448804 opened by Nick Coghlan urllib2+https+proxy not working (2006-03-13) http://python.org/sf/1448934 opened by Rui Martins urllib2+https+proxy not working (2006-03-14) CLOSED http://python.org/sf/1449397 opened by Rui Martins optparse: extending actions missing ALWAYS_TYPES_ACTIONS (2006-03-13) http://python.org/sf/1449311 opened by Christopher Build of readline fails (2006-03-14) CLOSED http://python.org/sf/1450019 opened by Sydney Weidman int() and isdigit() accept non-digit numbers when using unic (2006-03-15) CLOSED http://python.org/sf/1450212 opened by Pierre-Fr?d?ric Caillaud windows python truncates files when reading them (2006-03-15) CLOSED http://python.org/sf/1450456 opened by tom berger Python build fails for gcc 4.x from Gnu (2006-03-15) http://python.org/sf/1450807 opened by John W. Grove msgfmt.py: fuzzy messages not correctly found (2006-03-16) http://python.org/sf/1451341 opened by Hanno Schlichting reading very large files (2006-03-16) http://python.org/sf/1451466 opened by christen os.startfile() still doesn't work with Unicode filenames (2006-03-16) http://python.org/sf/1451503 opened by roee88 segfault in optimize_code() (2006-03-16) CLOSED http://python.org/sf/1451641 opened by Kristj?n Valur OS/X on i386 framework build (2006-03-17) http://python.org/sf/1451717 opened by Ian Holsman Bugs Closed ___________ handling comments with markupbase and HTMLParser (2006-03-04) http://python.org/sf/1442874 closed by gbrandl tkinter Dialog fails when more than four buttons are used (2006-03-10) http://python.org/sf/1447234 closed by gbrandl "reindent.py" exposes bug in tokenize (2006-03-10) http://python.org/sf/1447633 closed by goodger PCbuild/readme.txt description not quite right (2005-04-15) http://python.org/sf/1183896 closed by gbrandl Convertion error for latin1 characters with iso-2022-jp-2 (2006-03-13) http://python.org/sf/1448490 closed by perky datetime.__init__ cannot be overridden (2006-03-13) http://python.org/sf/1448640 closed by mwh Subscript operations generating incorrect code (2006-03-13) http://python.org/sf/1448804 closed by ncoghlan urllib2+https+proxy not working (2006-03-14) http://python.org/sf/1449397 deleted by ruibmartins Build of readline fails (2006-03-14) http://python.org/sf/1450019 closed by nnorwitz int() and isdigit() accept non-digit unicode numbers (2006-03-15) http://python.org/sf/1450212 closed by lemburg windows python truncates files when reading them (2006-03-15) http://python.org/sf/1450456 deleted by object segfault in optimize_code() (2006-03-16) http://python.org/sf/1451641 closed by rhettinger New / Reopened RFE __________________ Python should use 3GB Address Space on Windows (2006-03-14) http://python.org/sf/1449496 opened by Martin Gfeller bisect should support a custom comparison function (2006-03-16) http://python.org/sf/1451588 opened by Jonathan S. Joseph RFE Closed __________ friendly quit object (2006-01-13) http://python.org/sf/1404859 closed by gbrandl From g.brandl at gmx.net Fri Mar 17 07:32:43 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 17 Mar 2006 07:32:43 +0100 Subject: [Python-Dev] Python Library Reference top page too long In-Reply-To: <441A070D.10002@canterbury.ac.nz> References: <4419A633.6010607@comcast.net> <441A070D.10002@canterbury.ac.nz> Message-ID: Greg Ewing wrote: > Russell E. Owen wrote: > >> Fundamentally I think what's wanted is: >> - Another level of sub-TOCs, e.g. one for "Sequence Types", "Mapping >> Types", etc. Every page that has sub-topics or intimately related should >> have a list of them at the beginning. >> - The special methods for a given type of class would make a really good >> "intimately related" topic for "Sequence Types" and "Mapping Types". > > +1 to all that. I agree that the docs on __special__ methods really are hard to find if you don't know where to look. >> Another thing that would be *really* useful is to list the actual >> built-in types with the category. For example: >> Sequence Types (str, unicode, list, tuple, buffer, xrange) >> Mapping Types (dict) > > +1 If I understand correctly what you mean, that's already done. Georg From fredrik at pythonware.com Fri Mar 17 09:20:33 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Fri, 17 Mar 2006 09:20:33 +0100 Subject: [Python-Dev] towards a stricter definition of sys.executable References: <200603161152.34271.fdrake@acm.org> Message-ID: Fred L. Drake, Jr. wrote: > Fredrik Lundh wrote: > > d) If Python was started from a standard Python interpreter, > > My understanding matches Guido's description, so I'm not sure any changes are > needed. the problem with that is that your understanding doesn't match the implementation (which implements (a), as the GetProgramFullPath name indicates). as we've seen, lots of people (mainly application builders on windows) relies on the implemented behaviour, while some people (mostly toolmakers, from what I can tell) use guido's original interpretation. I don't think many people embed setup.py scripts, so alternative (e) would pro- bably cause the least problems: e) sys.executable contains the full path to the program used to invoke this interpreter instance, or None if this could not be determined. A new variable, sys.python_executable, is set to the name of the python executable used to invoke this interpreter instance. If this instance is embedded, this variable may be set to a corresponding interpreter, or to None if no such interpreter is available. If Python is started from a standard Python interpreter, both variables are set to the same value. any application that uses sys.executable to start another application will still work, as long as the application is started via a standard interpreter. embedded apps on windows will still work as today. From tzot at mediconsa.com Fri Mar 17 09:26:26 2006 From: tzot at mediconsa.com (Christos Georgiou) Date: Fri, 17 Mar 2006 10:26:26 +0200 Subject: [Python-Dev] bytes thoughts References: <440669B9.3000609@canterbury.ac.nz> <20060316090941.F73B.JCARLSON@uci.edu> Message-ID: "Josiah Carlson" wrote in message news:20060316090941.F73B.JCARLSON at uci.edu... "Christos Georgiou" wrote: [Christos] > Well, what's the result of > > bytes([1,0,0])^ bytes([1,0]) > > ? Is it bytes([0,0,0]) (? la little-endian) or is it bytes([1,1,0]) > (straight conversion to base-256)? Or perhaps throw a ValueError if the > sizes differ? [Josiah] > It's a ValueError. If the sizes matched, it would be a per-element > bitwise xor. I agree ("in the face of ambiguity..."), although I understand that it wasn't obvious that I do from the way I asked the question, which I was expecting Greg to answer :) [Christos] > These details should be considered in the PEP. [Josiah] > They aren't considered because they are *obvious* to most (if not all) > sane people who use Python. I beg to disagree. I don't know whether you are Dutch or not, but most of the Python users aren't; one of the reason PEPs exist is to explain what *should* "be obvious at first" when one is Dutch ;-) Apart from joking, PEPs should be a record of beating/thinking the PEP subject to its death: "The PEP author is responsible for building consensus within the community and documenting dissenting opinions." From martin at v.loewis.de Fri Mar 17 09:31:44 2006 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Fri, 17 Mar 2006 09:31:44 +0100 Subject: [Python-Dev] Problem with module loading on multi-arch? In-Reply-To: References: <4419D9EE.4040603@v.loewis.de> Message-ID: <441A73F0.1090404@v.loewis.de> Neal Becker wrote: > 1. Does it make sense to have both > > /app/subpackage > and > /app/subpackage > ? > > My answer: definitely yes. We already agree that we should have both > site-dep for binary code and site-indep for python code, so there is no way > to avoid this. I'm not sure I understand what you are talking about. What is site-dep and site-indep? Where do we have that for binary code and for python code? I know what prefix and exec_prefix are. > 2. What should the module search do? > > I don't know the details of the current algorithm, but clearly it's going to > have to deal correctly with the above. It's very simple. Importing traverses sys.path. > One possibility (and maybe this is how it already works?) is that the module > search doesn't know anything about site-dep/site-indep, it simply has a > list of paths to search. Well, I don't know about site-dep/site-indep, so I wouldn't be surprised if module search didn't, either. > If I did want to try to fix this myself, where would I find the code for it? The code for importing is in Modules/import.c. sys.path is defined in Modules/getpath.c Regards, Martin From oliphant.travis at ieee.org Fri Mar 17 09:39:45 2006 From: oliphant.travis at ieee.org (Travis E. Oliphant) Date: Fri, 17 Mar 2006 01:39:45 -0700 Subject: [Python-Dev] Expose the array interface in Python 2.5? Message-ID: Last year, during development of NumPy, the concept of the array interface was developed as a means to share array_like data. It was realized that it is this concept that needs to be explored further and pushed into Python itself, rather than a particular incarnation of an array. It is very important for many people to get access to memory with some description of how that memory should be interpreted as an array. Several Python packages could benefit if Python had some notion of an array interface that was at least supported in a duck-typing fashion. The description of what we've come up with so far and is implemented in NumPy (and Numarray and last released Numeric) is at http://numeric.scipy.org/#array_interface Quite a few of us would love to see this get into Python proper, but have very little free-time to spare to make sure it happens. Would it be possible to add at least the C-struct array interface to the Python arrayobject in time for Python 2.5? Is someone on this list willing to help make it happen? In NumPy, there is also a reasonably good way to describe the "data-type" of arbitrary data, that fell out of the discussions over the array interface. I think something like this could eventually find its way into Python as well. We would love any feedback from the Python community on the array interface. Especially because we'd like to see it in Python itself and supported and used by every relevant Python package sooner rather than later. Thanks, -Travis Oliphant From martin at v.loewis.de Fri Mar 17 09:40:31 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 17 Mar 2006 09:40:31 +0100 Subject: [Python-Dev] towards a stricter definition of sys.executable In-Reply-To: References: <200603161152.34271.fdrake@acm.org> Message-ID: <441A75FF.6020709@v.loewis.de> Fredrik Lundh wrote: > I don't think many people embed setup.py scripts, so alternative (e) would pro- > bably cause the least problems: > > e) sys.executable contains the full path to the program used to invoke > this interpreter instance, or None if this could not be determined. It seems that you indeed are trying to solve a problem you encountered. Can you please explain what the problem is? ISTM that the current definition doesn't really cause problems, despite potentially being fuzzy. People that start sys.executable typically *do* get a Python interpreter - in an embedded interpreter, they just don't want to start a new interpreter, as that couldn't work, anyway. Regards, Martin From mwh at python.net Fri Mar 17 10:06:13 2006 From: mwh at python.net (Michael Hudson) Date: Fri, 17 Mar 2006 09:06:13 +0000 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: <17434.10109.675484.243338@montanaro.dyndns.org> (skip@pobox.com's message of "Thu, 16 Mar 2006 21:05:33 -0600") References: <4418CDD0.8020803@canterbury.ac.nz> <17434.10109.675484.243338@montanaro.dyndns.org> Message-ID: <2m3bhh30fu.fsf@starship.python.net> skip at pobox.com writes: > Greg> except as : > > Baptiste> except with : > > Can I catch multiple exceptions with a single value in this case? Today, I > write: > > try: > foo() > except (TypeError, KeyError), msg: > print msg > > Either of the above seem like they'd require me to repeat the value, e.g: > > try: > foo() > except TypeError with msg, KeyError with msg: > print msg > > Not very Pythonic methinks. except TypeError or KeyError as msg: ! not-serious-ly y'rs, mwh -- That's why the smartest companies use Common Lisp, but lie about it so all their competitors think Lisp is slow and C++ is fast. (This rumor has, however, gotten a little out of hand. :) -- Erik Naggum, comp.lang.lisp From fuzzyman at voidspace.org.uk Fri Mar 17 10:08:26 2006 From: fuzzyman at voidspace.org.uk (Fuzzyman) Date: Fri, 17 Mar 2006 09:08:26 +0000 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: <2m3bhh30fu.fsf@starship.python.net> References: <4418CDD0.8020803@canterbury.ac.nz> <17434.10109.675484.243338@montanaro.dyndns.org> <2m3bhh30fu.fsf@starship.python.net> Message-ID: <441A7C8A.9030404@voidspace.org.uk> Michael Hudson wrote: > skip at pobox.com writes: > > >> Greg> except as : >> >> Baptiste> except with : >> >> Can I catch multiple exceptions with a single value in this case? Today, I >> write: >> >> try: >> foo() >> except (TypeError, KeyError), msg: >> print msg >> >> Either of the above seem like they'd require me to repeat the value, e.g: >> >> try: >> foo() >> except TypeError with msg, KeyError with msg: >> print msg >> >> Not very Pythonic methinks. >> > > Wasn't the proposal : try: something except NameError, OtherError as e: something... ? With e being bound for any of the exceptions... Michael Foord > except TypeError or KeyError as msg: ! > > not-serious-ly y'rs, > mwh > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060317/69155b7e/attachment.html From ncoghlan at gmail.com Fri Mar 17 10:58:32 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 17 Mar 2006 19:58:32 +1000 Subject: [Python-Dev] Expose the array interface in Python 2.5? In-Reply-To: References: Message-ID: <441A8848.2090004@gmail.com> Travis E. Oliphant wrote: > Would it be possible to add at least the C-struct array interface to the > Python arrayobject in time for Python 2.5? Do you mean simply adding an __array_shape__ attribute that consists of a tuple with the array length, and an __array_type__ attribute set to 'O'? Or trying to expose the array object's data? The former seems fairly pointless, and the latter difficult (since it has implications for moving the data store when the array gets resized). > We would love any feedback from the Python community on the array > interface. Especially because we'd like to see it in Python itself and > supported and used by every relevant Python package sooner rather than > later. I've spent a fair bit of time looking at this interface, and while I'm a big fan of the basic idea, I'm not convinced that it makes sense to include the interface in the core without *also* adopting a common convention for multi-dimensional fixed shape indexing (e.g. by introducing a simple dimensioned array type as something like array.dimarray). The fact that array.array is a mutable sequence rather than a fixed shape array means that it doesn't mesh particularly well with the ideas behind the array interface. numpy arrays can have their shape changed via reshape, but they impose the rule that the total number of elements can't change so that the allocated memory doesn't need to be moved - the standard library's array type has no such limitation. Aside from the obvious (the use of Ellipsis and permitting multiple dimensions), there are a number of ways in which the semantics of numpy array subscripts differ from normal sequence subcripts, and which of these should be part of the common multi-dimensional indexing conventions needs to be thrashed out in a PEP: - numpy array slices are views that permit mutation of the original object (slicing a sequence creates a copy of the sliced section) - assignment to slices is not allowed to change the shape of a numpy array (assigning to a slice of a normal sequence may change the total length) - deletion of slices is not permitted by numpy arrays (deleting a slice of a sequence changes the total length) - NewAxis is a novel use of subscript notation - there are sophisticated rules to try to align numpy array shapes - assignment of a sequence to a numpy array section is rather disconcerting, as the checks to determine what should and should not be repeated to fit into the available space are type based For something in the standard library, much of the complexity should be stripped out, with the clever bits of programmer convenience left for numpy to provide. However, decided which bits to remove and which to keep is a non-trivial task. Given that even the bytes type has been deferred to 2.6 to allow further consideration of the appropriate API, my vote is to do the same for an array.dimarray type and allow more time to figure out the appropriate *Python* interface. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From mal at egenix.com Fri Mar 17 10:59:59 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 17 Mar 2006 10:59:59 +0100 Subject: [Python-Dev] [Python-checkins] r43041 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: <4419F2F8.9040900@v.loewis.de> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> Message-ID: <441A889F.9080305@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg wrote: >> Since this change is going to affect a lot of 3rd party extensions, >> I'd also like to see a complete list of public APIs that changed and >> how they changed (including the type slots) > > You can construct this list easily by comparing the header files of > the previous and the current release. Please contribute a patch that > presents these changes in a form that you would consider acceptable. Sorry, but I'd rather spend that time on getting my extensions working again for both Python 2.4 and 2.5. I think it's only fair that I ask the patch authors to complete the PEP, since the ssize_t patch is causing extension authors enough trouble already. If you want quick adoption of the changes, you have to make it as easy as possible for the authors to port their extensions to the new API. Otherwise, we'll end up having quite a large number of users who can't switch to Python 2.5 simply because their favorite extensions don't work with it. The argument that it "only" affects 64-bit platforms simply ignores reality. Most new machines are 64-bit machines, so by the time Python 2.5 goes public, 64-bit will have a large enough market share to make a difference. It's also not good enough to simply suggest to ignore compiler warnings - this falls back on the extension authors and the quality of their code without them really having done anything wrong. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 17 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From oliphant.travis at ieee.org Fri Mar 17 11:40:38 2006 From: oliphant.travis at ieee.org (Travis E. Oliphant) Date: Fri, 17 Mar 2006 03:40:38 -0700 Subject: [Python-Dev] Expose the array interface in Python 2.5? In-Reply-To: <441A8848.2090004@gmail.com> References: <441A8848.2090004@gmail.com> Message-ID: Nick Coghlan wrote: > Travis E. Oliphant wrote: >> Would it be possible to add at least the C-struct array interface to the >> Python arrayobject in time for Python 2.5? > > Do you mean simply adding an __array_shape__ attribute that consists of a > tuple with the array length, and an __array_type__ attribute set to 'O'? > > Or trying to expose the array object's data? I was thinking more the __array_struct__ (in particular the C-structure that defines it). > > The former seems fairly pointless, and the latter difficult (since it has > implications for moving the data store when the array gets resized). Sure, it's the same problem as exposing through the buffer protocol. Since, we already have that problem, why try to pretend we don't? > > I've spent a fair bit of time looking at this interface, and while I'm a big > fan of the basic idea, I'm not convinced that it makes sense to > include the interface in the core without *also* adopting a common convention > for multi-dimensional fixed shape indexing (e.g. by introducing a simple > dimensioned array type as something like array.dimarray). True, such a thing would be great, but it could also be written in Python fairly quickly building on top of the array and serve as a simple example. My big quest is to get PIL, PyVox, WxPython, PyOpenGL, and so forth to be able to use the same interface. Blessing the interface by including it in the Python core would help. I'm also just wanting people in py-dev to get the concept of an array interface on their radar, as discussions of new bytes types emerges. Sometimes, there is not enough cross-talk between numpy-discussions and pydev. This is our fault, of course, but we're often swamped (I know I am...), and it can take some effort for us "array" people to figure out what's going on in the depths of Python sufficiently to comprehend some of the discussions here. > > The fact that array.array is a mutable sequence rather than a fixed shape > array means that it doesn't mesh particularly well with the ideas behind the > array interface. numpy arrays can have their shape changed via reshape, but > they impose the rule that the total number of elements can't change so that > the allocated memory doesn't need to be moved - the standard library's array > type has no such limitation. This is not really a limitation of numpy arrays either. Check the resize method... But, I understand your point that array.array's are more-like lists. Of course, when they behave that way, their buffer interface is presently broken. So, maybe the array.array is sufficiently broken to not be worth "fixing", but what else should be done? I'm kind of tired of this problem dragging on and on. The Numeric header (essentially what the __array_struct__ exposes) is now basically unchanged for over 10 years and yet it's direct support by Python is still not their. The Python community has been very helpful over the years, but we need more direct discussion with Python developers to help things along. I'm grateful Nick has responded. If anyone else has any interest in these ideas, please sound off. > > Aside from the obvious (the use of Ellipsis and permitting multiple > dimensions), there are a number of ways in which the semantics of numpy array > subscripts differ from normal sequence subcripts, and which of these should be > part of the common multi-dimensional indexing conventions needs to be thrashed > out in a PEP: While these are interesting academic issues. The problem with most of these comments is that you will get load voices of disapproval if any of these conventions changes significantly from what has become standard via Numeric's use over 10 years. I think no one is up to the task of trying to re-concile Numeric behavior with Python-dev opinions of what 'ought' to be, unless the basic usage does not change too much. > > - numpy array slices are views that permit mutation of the original object > (slicing a sequence creates a copy of the sliced section) Not really open for discussion among Numeric Python users as it's been debated for years always coming to the same (keep the current behavior) conclusion. > > - assignment to slices is not allowed to change the shape of a numpy array > (assigning to a slice of a normal sequence may change the total length) People might be open to this idea, as it adds a new feature and doesn't signficantly change other usages. > > - deletion of slices is not permitted by numpy arrays > (deleting a slice of a sequence changes the total length) > Also something people might accept. > - NewAxis is a novel use of subscript notation True, but not something we can really change. > > - there are sophisticated rules to try to align numpy array shapes You are speaking of broadcasting. These could of course be discussed, but current behavior is "entrenched" > > - assignment of a sequence to a numpy array section is rather disconcerting, > as the checks to determine what should and should not be repeated to fit > into the available space are type based I'm not sure what this means... Please elaborate. > > For something in the standard library, much of the complexity should be > stripped out, with the clever bits of programmer convenience left for numpy to > provide. However, decided which bits to remove and which to keep is a > non-trivial task. > I agree. I suppose your itemization above was really to come to this conclusion as well. But, I think a stripped-down array that doesn't try to guess what to do with these interfaces is a good start. In other words, I disagree that you need to implement multidimensional indexing in order for Python to support the array interface. All you need is a simple object that supports the buffer protocol and has the __array_struct__ method and has a C-structure very similar to the current NumPy array (which is very similar to the old Numeric C-structure). If such a thing were in Python, then NumPy could inherit from it (as could other array-like objects), with the big advantage that there is at least one common memory model for arrays. Others could still exist, of course, but at least there would be a very useful common one. > Given that even the bytes type has been deferred to 2.6 to allow further > consideration of the appropriate API, my vote is to do the same for an > array.dimarray type and allow more time to figure out the appropriate *Python* > interface. I was afraid of that. But, unless people in pydev actually care to discuss these matters, I fear that yet again nothing will be done. The problem is that for most of us array users, it's only community outreach and a desire to get people using Python talking the same array language that makes us really care about these things. The NumPy library works fine for what we really need it to do, and it's hard to get motivated to convince people that haven't used an array-language like IDL or MATLAB in the past to understand the reasons for NumPy's behavior. The big difference with the bytes type, is that Numeric has 10 years of history behind it. There is a lot of experience with an appropriate array type. It's not like we just came up with this a few days ago :-) As the bytes type is developed please keep in mind it's uses as the memory for an N-dimensional array. Perhaps the bytes object could be a default way (or built on a default way) to allocate memory. A simple reference-counted memory object would certainly belay the problems of the buffer interface that the array object currently has problems with. In other words, the array object should not malloc it's own memory but create a memory object which is nothing more than a reference-counted pointer to memory. Surely this has been talked about. Is there a reason it has not been implemented? It would not be that hard. Even something like that would be a first step. Thanks for the comments. I'm glad there is another voice here that cares about the issues involved. -Travis > > Regards, > Nick. > From thomas at python.org Fri Mar 17 11:48:39 2006 From: thomas at python.org (Thomas Wouters) Date: Fri, 17 Mar 2006 11:48:39 +0100 Subject: [Python-Dev] Problem with module loading on multi-arch? In-Reply-To: References: <4419D9EE.4040603@v.loewis.de> Message-ID: <9e804ac0603170248l7af8eeccub959f55154a60470@mail.gmail.com> On 3/17/06, Neal Becker wrote: > OK, let's talk about it. > > 1. Does it make sense to have both > > /app/subpackage > and > /app/subpackage > ? > > My answer: definitely yes. > Nope. It would make sense to have .py files in site-indep and .pyc, .pyo and .so files in site-dep, but Python doesn't allow that (in 2.x, anyway; 3.xfeature?) I believe it should all be in site-dep. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060317/a59bec00/attachment.htm From ndbecker2 at gmail.com Fri Mar 17 12:50:36 2006 From: ndbecker2 at gmail.com (Neal Becker) Date: Fri, 17 Mar 2006 06:50:36 -0500 Subject: [Python-Dev] Problem with module loading on multi-arch? References: <4419D9EE.4040603@v.loewis.de> <441A73F0.1090404@v.loewis.de> Message-ID: "Martin v. L?wis" wrote: > Neal Becker wrote: >> 1. Does it make sense to have both >> >> /app/subpackage >> and >> /app/subpackage >> ? >> >> My answer: definitely yes. We already agree that we should have both >> site-dep for binary code and site-indep for python code, so there is no >> way to avoid this. > > I'm not sure I understand what you are talking about. What is site-dep > and site-indep? Where do we have that for binary code and for python > code? I know what prefix and exec_prefix are. > >> 2. What should the module search do? >> >> I don't know the details of the current algorithm, but clearly it's going >> to have to deal correctly with the above. > > It's very simple. Importing traverses sys.path. > >> One possibility (and maybe this is how it already works?) is that the >> module search doesn't know anything about site-dep/site-indep, it simply >> has a list of paths to search. > > Well, I don't know about site-dep/site-indep, so I wouldn't be surprised > if module search didn't, either. > Sorry, maybe I used confusing terminology. A reference is here: http://fedoraproject.org/wiki/Packaging/Python This is the current setup. For example, this is a standard macro used by Redhat in RPM SPEC files for python: %define python_sitearch %(%{__python} -c "from distutils.sysconfig import get_python_lib; print get_python_lib(1)")} %define python_sitelib %(%{__python} -c "from distutils.sysconfig import get_python_lib; print get_python_lib()")} Clearly this practice is widespread. It would seem that module search needs some modification to fully support it. From tds333+pydev at gmail.com Fri Mar 17 14:28:44 2006 From: tds333+pydev at gmail.com (Wolfgang Langner) Date: Fri, 17 Mar 2006 14:28:44 +0100 Subject: [Python-Dev] Bug 1184112 still valid Message-ID: <4c45c1530603170528m4600d4a3v@mail.gmail.com> Hello, today I got my first real python bug. Problem is described in: Bug with ID: 1184112 http://sourceforge.net/tracker/index.php?func=detail&aid=1184112&group_id=5470&atid=105470 I use python 2.3.5 in an embedded C++ Application which uses function: PyRun_SimpleString. Is this bug still present in current python or the 2.4 line ? I found nothing about it. -- bye by Wolfgang From ncoghlan at gmail.com Fri Mar 17 14:53:48 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 17 Mar 2006 23:53:48 +1000 Subject: [Python-Dev] Bug 1184112 still valid In-Reply-To: <4c45c1530603170528m4600d4a3v@mail.gmail.com> References: <4c45c1530603170528m4600d4a3v@mail.gmail.com> Message-ID: <441ABF6C.2040903@gmail.com> Wolfgang Langner wrote: > Hello, > > today I got my first real python bug. Problem is described in: > > Bug with ID: 1184112 > > http://sourceforge.net/tracker/index.php?func=detail&aid=1184112&group_id=5470&atid=105470 > > I use python 2.3.5 in an embedded C++ Application which uses function: > PyRun_SimpleString. > > Is this bug still present in current python or the 2.4 line ? > I found nothing about it. Confirmed on SVN trunk. I don't know enough about the tokenizer to determine if the proposed fix is the best solution, though. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From barry at python.org Fri Mar 17 15:40:17 2006 From: barry at python.org (Barry Warsaw) Date: Fri, 17 Mar 2006 09:40:17 -0500 Subject: [Python-Dev] Merging email 4.0 to Python 2.5 svn trunk Message-ID: <1142606417.32074.21.camel@geddy.wooz.org> Last night I merged email 4.0 from the sandbox into my working copy of the Python svn trunk and ran "make testall". I hit one failure, in test_pyclbr.py. The test was importing email.Parser and expecting a real module, however in email 4.0 email.Parser is a placeholder object which exposes a backward compatible API for email.parser. The fix is simple, change 'P' to 'p' in the test, but I want to make sure that nobody cares that __import__('email.Parser') now returns an object that isn't a module, but acts enough like the original module for all intents and purposes. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060317/c4d6778a/attachment.pgp From fredrik at pythonware.com Fri Mar 17 15:56:48 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Fri, 17 Mar 2006 15:56:48 +0100 Subject: [Python-Dev] Bug 1184112 still valid References: <4c45c1530603170528m4600d4a3v@mail.gmail.com> Message-ID: Wolfgang Langner wrote: > today I got my first real python bug. Problem is described in: > > Bug with ID: 1184112 > > http://sourceforge.net/tracker/index.php?func=detail&aid=1184112&group_id=5470&atid=105470 > > I use python 2.3.5 in an embedded C++ Application which uses function: > PyRun_SimpleString. the "missing trailing newline may cause errors for multiline statements" is documented behaviour; from the compile() docs: "When compiling multi-line statements, two caveats apply: line endings must be represented by a single newline character ('\n'), and the input must be terminated by at least one newline character." see the codeop.py sources for library code that depends on this behaviour. From tjreedy at udel.edu Fri Mar 17 17:49:16 2006 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 17 Mar 2006 11:49:16 -0500 Subject: [Python-Dev] Bug 1184112 still valid References: <4c45c1530603170528m4600d4a3v@mail.gmail.com> Message-ID: "Fredrik Lundh" wrote in message news:dveini$ntv$1 at sea.gmane.org... >> http://sourceforge.net/tracker/index.php?func=detail&aid=1184112&group_id=5470&atid=105470 > "When compiling multi-line statements, two caveats apply: line > endings must be represented by a single newline character ('\n'), > and the input must be terminated by at least one newline character." Relevant part of quote added to bug report. tjr From ncoghlan at gmail.com Fri Mar 17 18:19:35 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 18 Mar 2006 03:19:35 +1000 Subject: [Python-Dev] Bug 1184112 still valid In-Reply-To: References: <4c45c1530603170528m4600d4a3v@mail.gmail.com> Message-ID: <441AEFA7.4090007@gmail.com> Terry Reedy wrote: > "Fredrik Lundh" wrote in message > news:dveini$ntv$1 at sea.gmane.org... >>> http://sourceforge.net/tracker/index.php?func=detail&aid=1184112&group_id=5470&atid=105470 >> "When compiling multi-line statements, two caveats apply: line >> endings must be represented by a single newline character ('\n'), >> and the input must be terminated by at least one newline character." > > Relevant part of quote added to bug report. One quirk though, is that omitting the newline actually works for some other statements (e.g. exec "if 1:\n print 'Hello world!'" works just fine) However, I don't think the quote is as relevant as one might think, because in the code that fails the multi-line statement is not only terminated by a newline, it's actually followed by a blank line as well. The line missing the newline is only a single-line statement (specifically, a line with only a comment). Compare: Py> exec """ ... if 1: ... print 'This works!' ... ... pass # Do nothing at all""" This works! With: Py> exec """ ... if 1: ... print 'This breaks!' ... ... # Do nothing at all""" Traceback (most recent call last): File "", line 1, in ? File "", line 5 # Do nothing at all ^ SyntaxError: invalid syntax There's an easy workaround (adding the missing newline), but I think there's definitely something going wrong in the tokenizer. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From skip at pobox.com Fri Mar 17 18:27:41 2006 From: skip at pobox.com (skip at pobox.com) Date: Fri, 17 Mar 2006 11:27:41 -0600 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: <441A7C8A.9030404@voidspace.org.uk> References: <4418CDD0.8020803@canterbury.ac.nz> <17434.10109.675484.243338@montanaro.dyndns.org> <2m3bhh30fu.fsf@starship.python.net> <441A7C8A.9030404@voidspace.org.uk> Message-ID: <17434.61837.491103.995641@montanaro.dyndns.org> Skip> try: Skip> foo() Skip> except TypeError with msg, KeyError with msg: Skip> print msg fuzz> Wasn't the proposal : fuzz> try: fuzz> something fuzz> except NameError, OtherError as e: fuzz> something... I'm not sure. I only saw as|with . In your formulation the comma binds more tightly than the as keyword. In import statements it's the other way around. That seems like it might be a source of confusion. Skip From ncoghlan at iinet.net.au Fri Mar 17 18:53:52 2006 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Sat, 18 Mar 2006 03:53:52 +1000 Subject: [Python-Dev] Making runpy.run_module *not* thread-safe Message-ID: <441AF7B0.10602@iinet.net.au> While it may seem like going backwards, I actually have a legitimate reason for getting rid of the current thread safety mechanism in runpy.run_module. It appears Guido's doubts about the approach I used are entirely justified. When the alter_sys flag is set, runpy.run_module pokes around a bit in the sys module. Because one of the things it modifies is sys.modules (putting a partially initialised module there), it currently uses the import lock to make this tinkering thread-safe. However, if you try using "-m test.regrtest" you'll notice that test_threaded_import gets skipped because the import lock is held in the main application script. This is *not* the case when invoking regrtest directly via the filesystem - in that case, the runtime simply puts the partially initialised __main__ module into sys.modules, and leaves it to the application to decide whether or not that module is safe to import. That's all well and good for regrtest.py, but any program that actually spawned additional threads that attempted to import modules would likely see deadlocks in very short order. Removing the locking from run_module() makes it more like the PyRun C APIs - it puts the extra module in sys.modules, but leaves it up to the invoking code to decide whether or not to grab the import lock (or some other synchronisation object) during execution. The import lock will still be held while trying to *find* the requested module and the various optimisation caches in sys are updated. If I don't hear any objections, I'll switch SVN (along with PEP 338 and the docs patch) to the new (non-thread-safe) semantics sometime this weekend. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From martin at v.loewis.de Fri Mar 17 19:26:29 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 17 Mar 2006 19:26:29 +0100 Subject: [Python-Dev] [Python-checkins] r43041 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: <441A889F.9080305@egenix.com> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> Message-ID: <441AFF55.2030209@v.loewis.de> M.-A. Lemburg wrote: > I think it's only fair that I ask the patch authors to complete > the PEP, since the ssize_t patch is causing extension authors > enough trouble already. Well, the PEP is complete as it stands. It's possible to provide more guidelines, but the specification part of it says precisely what I intend it to say. Also, the API documentation ought to be in the Python documentation, and, for these changes, it is. > If you want quick adoption of the changes, you have > to make it as easy as possible for the authors to port their > extensions to the new API. Otherwise, we'll end up having > quite a large number of users who can't switch to Python 2.5 > simply because their favorite extensions don't work with it. I don't see how giving a complete list of all changed functions helps in any way. > It's also not good enough to simply suggest to ignore compiler > warnings - this falls back on the extension authors and the > quality of their code without them really having done anything > wrong. Sure. Compiler warnings should be corrected. That's why the compiler emits them. However, I don't see how this is related to the text of the PEP. Regards, Martin From jcarlson at uci.edu Fri Mar 17 19:38:34 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Fri, 17 Mar 2006 10:38:34 -0800 Subject: [Python-Dev] bytes thoughts In-Reply-To: References: <20060316090941.F73B.JCARLSON@uci.edu> Message-ID: <20060317102938.F765.JCARLSON@uci.edu> "Christos Georgiou" wrote: > [Christos] > > These details should be considered in the PEP. > > [Josiah] > > They aren't considered because they are *obvious* to most (if not all) > > sane people who use Python. > > I beg to disagree. I don't know whether you are Dutch or not, but most of There's probably a bit of dutch in me, but likely not sufficient to alter my language intuitions. > the Python users aren't; one of the reason PEPs exist is to explain what > *should* "be obvious at first" when one is Dutch ;-) Apart from joking, > PEPs should be a record of beating/thinking the PEP subject to its death: > "The PEP author is responsible for building consensus within the > community and documenting dissenting opinions." Considering your quote, I have thusfar not seen any opinion dissenting from what I stated as 'obvious' behavior. As I told Michael Chermside off-list: "Sure, great, clarify it. I wasn't saying anything in regards to whether it should or shouldn't [be clarified], just why it probably wasn't already." Also considering that I lack SVN commit permission, it's not within my power to clarify the PEP. - Josiah From theller at python.net Fri Mar 17 19:37:05 2006 From: theller at python.net (Thomas Heller) Date: Fri, 17 Mar 2006 19:37:05 +0100 Subject: [Python-Dev] [Python-checkins] r43041 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: <441AFF55.2030209@v.loewis.de> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> Message-ID: Martin v. L?wis wrote: > M.-A. Lemburg wrote: >> I think it's only fair that I ask the patch authors to complete >> the PEP, since the ssize_t patch is causing extension authors >> enough trouble already. > > Well, the PEP is complete as it stands. It's possible to provide > more guidelines, but the specification part of it says precisely > what I intend it to say. Also, the API documentation ought to > be in the Python documentation, and, for these changes, it is. > >> If you want quick adoption of the changes, you have >> to make it as easy as possible for the authors to port their >> extensions to the new API. Otherwise, we'll end up having >> quite a large number of users who can't switch to Python 2.5 >> simply because their favorite extensions don't work with it. > > I don't see how giving a complete list of all changed functions > helps in any way. I'm not sure if this is what Marc-Andre means, but maybe these definitions could go into a new include file: #if (PY_VERSION_HEX < 0x02050000) typedef int Py_ssize_t; #define lenfunc inquiry #define readbufferproc getreadbufferproc #define writebufferproc getwritebufferproc #define segcountproc getsegcountproc #define charbufferproc getcharbufferproc #define ssizeargfunc intargfunc #define ssizessizeargfunc intintargfunc #define ssizeobjargproc intobjargproc #define ssizessizeobjargproc intintobjargproc ... more defines #endif From rowen at cesmail.net Fri Mar 17 19:44:59 2006 From: rowen at cesmail.net (Russell E. Owen) Date: Fri, 17 Mar 2006 10:44:59 -0800 Subject: [Python-Dev] Python Library Reference top page too long References: <4419A633.6010607@comcast.net> <441A070D.10002@canterbury.ac.nz> Message-ID: In article , Georg Brandl wrote: > >> Another thing that would be *really* useful is to list the actual > >> built-in types with the category. For example: > >> Sequence Types (str, unicode, list, tuple, buffer, xrange) > >> Mapping Types (dict) > > > > +1 > > If I understand correctly what you mean, that's already done. You're right! I was using a local copy of 2.4.0 documentation and it did not have this feature. -- Russell From edcjones at comcast.net Fri Mar 17 20:10:47 2006 From: edcjones at comcast.net (Edward C. Jones) Date: Fri, 17 Mar 2006 14:10:47 -0500 Subject: [Python-Dev] Expose the array interface in Python 2.5? In-Reply-To: References: Message-ID: <441B09B7.6070403@comcast.net> "Travis E. Oliphant" wrote: > It is very important for many people to get access to memory with some > description of how that memory should be interpreted as an array. > Several Python packages could benefit if Python had some notion of an > array interface that was at least supported in a duck-typing fashion. Which packages? Which people? Which constituencies? "Travis E. Oliphant" also wrote: > My big quest is to get PIL, PyVox, WxPython, PyOpenGL, and so forth to > be able to use the same interface. Blessing the interface by > including it in the Python core would help. I'm also just wanting > people in py-dev to get the concept of an array interface on their > radar, as discussions of new bytes types emerges. I use PIL and numarray a lot. It would be nice if they used a common array format so I would not need to convert back and forth. But I survive quite well with the current arrangement. Many other packages besides PIL and Numeric / numarray / Numpy are involved here: byte, struct, ctypes, SWIG, PyTables, Psyco, PyPy, Pyrex, etc. There are some major issues that need to be dealt with which I will state concisely in an abstract way. A data structure without an API and thorough documentation is useless. Any proposal needs to include them from the very start. How should Python interact with low level data? By "low level data" I mean data as seen by C, C++, and FORTRAN as well as linear arrays of bytes. What changes in Python would make the packages listed above easier to write and use? Easier enough to write that the package owners would be willing to give up control of part of their packages. Does anyone know of any academic-type papers that have been written in the last few years on these matters? From oliphant.travis at ieee.org Fri Mar 17 20:32:47 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Fri, 17 Mar 2006 12:32:47 -0700 Subject: [Python-Dev] Expose the array interface in Python 2.5? In-Reply-To: <441B09B7.6070403@comcast.net> References: <441B09B7.6070403@comcast.net> Message-ID: Edward C. Jones wrote: > "Travis E. Oliphant" wrote: > > > It is very important for many people to get access to memory with some > > description of how that memory should be interpreted as an array. > > Several Python packages could benefit if Python had some notion of an > > array interface that was at least supported in a duck-typing fashion. > > Which packages? Which people? Which constituencies? > I think I spell it out later. Do you really need to argue about whether or not an array interface is a useful thing? I thought we were beyond that and to the point of trying to figure out how to get the many groups to agree at least on a common interface. > "Travis E. Oliphant" also wrote: > > > My big quest is to get PIL, PyVox, WxPython, PyOpenGL, and so forth to > > be able to use the same interface. Blessing the interface by > > including it in the Python core would help. I'm also just wanting > > people in py-dev to get the concept of an array interface on their > > radar, as discussions of new bytes types emerges. > > I use PIL and numarray a lot. It would be nice if they used a common > array format so I would not need to convert back and forth. But I > survive quite well with the current arrangement. > We all "survive", but saying it is "quite well" is a bit optimistic as it means many very useful applications are harder to write than they really need to be. > Many other packages besides PIL and Numeric / numarray / Numpy are > involved here: byte, struct, ctypes, SWIG, PyTables, Psyco, PyPy, Pyrex, > etc. There are some major issues that need to be dealt with which I will Sure they are involved, but I would argue the other ones you list care less about the multidimensional aspect of the array interface. (Actually PyTables just uses NumPy and so it should not be discussed as a "separate" package --- i.e. PyTables already tries to get along with NumPy as do many other packages...) > > A data structure without an API and thorough documentation is useless. > Any proposal needs to include them from the very start. Again, I restate. The Numeric structure has been documented and has been around for a *long* time. I'm just trying to get this basic interface into Python as a very simple object. Let's not try to make it so complicated that no body can agree on what it should do. To be specific, I want to see a type object with almost none of the Type structure filled in with specific behavior. I'm mainly interested in an array structure that other packages can rely on (and inherit from if they so choose). Because the C-structure of the Numeric PyArrayObject (which NumPy also uses) is so widely known and used and documented for over 10 years, I argue it ought to form the foundation for this simple Python object. We can argue about explicit multidimensional indexing behavior, but to hold hostage the introduction of a simple inheritable object to disagreements about those more complicated issues seems to be missing the mark. > > How should Python interact with low level data? By "low level data" I > mean data as seen by C, C++, and FORTRAN as well as linear arrays of bytes. This is already known about in Numeric. That's what I'm saying. Numeric handles this well, let's just bring over this basic memory model for an array over to Python itself and not worry about the other TypeObject function-pointer tables until later. Everybody I talked to at SciPy was very enthused about this concept. There is a large number of people who don't read Python-dev that I'm speaking for here. > > What changes in Python would make the packages listed above easier to > write and use? Easier enough to write that the package owners would be > willing to give up control of part of their packages. They don't have to give up control if we just introduce a simple memory model for an array. Thanks for your comments, -Travis From brett at python.org Fri Mar 17 20:40:50 2006 From: brett at python.org (Brett Cannon) Date: Fri, 17 Mar 2006 11:40:50 -0800 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: <50372CD0-B314-4FED-8C57-3F563F459E09@gmail.com> References: <4418CDD0.8020803@canterbury.ac.nz> <441A0714.7040200@canterbury.ac.nz> <50372CD0-B314-4FED-8C57-3F563F459E09@gmail.com> Message-ID: On 3/16/06, Alex Martelli wrote: > > On Mar 16, 2006, at 7:30 PM, Brett Cannon wrote: > ... > > I agree. "as" is taking on the use of assignment in statements that > > are not ``=`` and I say we just keep on with that. Plus Greg's above > > Hmmm, if we allowed '( as )' for generic expr's we'd make > a lot of people pining for 'assignment as an expression' very happy, > wouldn't we...? Yes, but I don't want them to be happy. =) -Brett From jjl at pobox.com Fri Mar 17 20:44:51 2006 From: jjl at pobox.com (John J Lee) Date: Fri, 17 Mar 2006 19:44:51 +0000 (UTC) Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: <17434.61837.491103.995641@montanaro.dyndns.org> References: <4418CDD0.8020803@canterbury.ac.nz> <17434.10109.675484.243338@montanaro.dyndns.org> <2m3bhh30fu.fsf@starship.python.net> <441A7C8A.9030404@voidspace.org.uk> <17434.61837.491103.995641@montanaro.dyndns.org> Message-ID: On Fri, 17 Mar 2006, skip at pobox.com wrote: [...] > fuzz> Wasn't the proposal : > > fuzz> try: > fuzz> something > fuzz> except NameError, OtherError as e: > fuzz> something... > > I'm not sure. I only saw as|with . Fuzzyman is right. > In your formulation the comma binds more tightly than the as keyword. > In import statements it's the other way around. That seems like it > might be a source of confusion. Perhaps parentheses around the exception list should be mandatory for the 2-or-more exceptions case? except NameError as e: --> fine except (NameError) as e: --> fine except (NameError,) as e: --> fine except NameError, OtherError as e: --> SyntaxError except (NameError, OtherError) as e: --> fine John From g.brandl at gmx.net Fri Mar 17 20:49:59 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 17 Mar 2006 20:49:59 +0100 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: References: <4418CDD0.8020803@canterbury.ac.nz> <17434.10109.675484.243338@montanaro.dyndns.org> <2m3bhh30fu.fsf@starship.python.net> <441A7C8A.9030404@voidspace.org.uk> <17434.61837.491103.995641@montanaro.dyndns.org> Message-ID: John J Lee wrote: >> In your formulation the comma binds more tightly than the as keyword. >> In import statements it's the other way around. That seems like it >> might be a source of confusion. > > Perhaps parentheses around the exception list should be mandatory for the > 2-or-more exceptions case? > > except NameError as e: --> fine > except (NameError) as e: --> fine > except (NameError,) as e: --> fine > > except NameError, OtherError as e: --> SyntaxError > except (NameError, OtherError) as e: --> fine I don't like that particularly, but I guess that for consistency's sake it would have to be done this way. Cheers, Georg From theller at python.net Fri Mar 17 20:55:58 2006 From: theller at python.net (Thomas Heller) Date: Fri, 17 Mar 2006 20:55:58 +0100 Subject: [Python-Dev] Expose the array interface in Python 2.5? In-Reply-To: References: Message-ID: Travis E. Oliphant wrote: > Last year, during development of NumPy, the concept of the array > interface was developed as a means to share array_like data. It was > realized that it is this concept that needs to be explored further and > pushed into Python itself, rather than a particular incarnation of an > array. > > It is very important for many people to get access to memory with some > description of how that memory should be interpreted as an array. > Several Python packages could benefit if Python had some notion of an > array interface that was at least supported in a duck-typing fashion. > > The description of what we've come up with so far and is implemented in > NumPy (and Numarray and last released Numeric) is at > > http://numeric.scipy.org/#array_interface > > Quite a few of us would love to see this get into Python proper, but > have very little free-time to spare to make sure it happens. > > Would it be possible to add at least the C-struct array interface to the > Python arrayobject in time for Python 2.5? I'm very much for that. > Is someone on this list willing to help make it happen? Unfortunately not me - I'm too busy with ctypes, and if the array interface makes it into the core I will have to implement/use that in ctypes too. Accessing Python arrays (Numeric arrays, Numeric array, or Numpy array) as ctypes arrays, and vice versa, without copying any memory, would be a good thing. > In NumPy, there is also a reasonably good way to describe the > "data-type" of arbitrary data, that fell out of the discussions over the > array interface. I think something like this could eventually find its > way into Python as well. > > We would love any feedback from the Python community on the array > interface. Especially because we'd like to see it in Python itself and > supported and used by every relevant Python package sooner rather than > later. Thomas From phd at mail2.phd.pp.ru Fri Mar 17 21:23:45 2006 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Fri, 17 Mar 2006 23:23:45 +0300 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: References: <4418CDD0.8020803@canterbury.ac.nz> <17434.10109.675484.243338@montanaro.dyndns.org> <2m3bhh30fu.fsf@starship.python.net> <441A7C8A.9030404@voidspace.org.uk> <17434.61837.491103.995641@montanaro.dyndns.org> Message-ID: <20060317202345.GA19451@phd.pp.ru> On Fri, Mar 17, 2006 at 07:44:51PM +0000, John J Lee wrote: > Perhaps parentheses around the exception list should be mandatory for the > 2-or-more exceptions case? > > except NameError as e: --> fine > except (NameError) as e: --> fine > except (NameError,) as e: --> fine > > except NameError, OtherError as e: --> SyntaxError > except (NameError, OtherError) as e: --> fine If parens are mandatory what is the difference from the current syntax? Why do we need "as" here? Why not stay with except (NameError, OtherError), e: ??? Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From mal at egenix.com Fri Mar 17 21:49:09 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 17 Mar 2006 21:49:09 +0100 Subject: [Python-Dev] [Python-checkins] r43041 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: <441AFF55.2030209@v.loewis.de> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> Message-ID: <441B20C5.1080009@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg wrote: >> I think it's only fair that I ask the patch authors to complete >> the PEP, since the ssize_t patch is causing extension authors >> enough trouble already. > > Well, the PEP is complete as it stands. It's possible to provide > more guidelines, but the specification part of it says precisely > what I intend it to say. Also, the API documentation ought to > be in the Python documentation, and, for these changes, it is. Changes to the public API must be documented somewhere, either in Misc/NEWS or in the PEP. How else do you expect users to find out about these changes ??? >> If you want quick adoption of the changes, you have >> to make it as easy as possible for the authors to port their >> extensions to the new API. Otherwise, we'll end up having >> quite a large number of users who can't switch to Python 2.5 >> simply because their favorite extensions don't work with it. > > I don't see how giving a complete list of all changed functions > helps in any way. It documents those changes, allows extension authors and extension users to check their extensions for possible problems and is (or at least should be) standard procedure. The list of changes is also necessary in order to be able to write code which allows a module to work in both Python 2.4 and 2.5. The code snippet that Thomas suggested should be part of the conversion guidelines. Again, if you don't make it easy for the developers, they are going to have a hard time upgrading their modules and getting them stable again after the changes. Telling them to read the header files or API docs and suggesting that they should compare 2.4 and 2.5 versions of these should not be the attitude with which such changes are approached. >> It's also not good enough to simply suggest to ignore compiler >> warnings - this falls back on the extension authors and the >> quality of their code without them really having done anything >> wrong. > > Sure. Compiler warnings should be corrected. That's why the compiler > emits them. However, I don't see how this is related to the text of > the PEP. """ Module authors have the choice whether they support this PEP in their code or not; if they support it, they have the choice of different levels of compatibility. If a module is not converted to support this PEP, it will continue to work unmodified on a 32-bit system. On a 64-bit system, compile-time errors and warnings might be issued, and the module might crash the interpreter if the warnings are ignored. """ Do you really think module authors do have a choice given that last sentence ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 17 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From martin at v.loewis.de Fri Mar 17 21:54:50 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 17 Mar 2006 21:54:50 +0100 Subject: [Python-Dev] [Python-checkins] r43041 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> Message-ID: <441B221A.5090900@v.loewis.de> Thomas Heller wrote: > I'm not sure if this is what Marc-Andre means, but maybe these definitions > could go into a new include file: How would that include file be used? You would have to copy it into your own source base, and include it, right? Regards, Martin From martin at v.loewis.de Fri Mar 17 22:03:06 2006 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Fri, 17 Mar 2006 22:03:06 +0100 Subject: [Python-Dev] Problem with module loading on multi-arch? In-Reply-To: References: <4419D9EE.4040603@v.loewis.de> <441A73F0.1090404@v.loewis.de> Message-ID: <441B240A.9000909@v.loewis.de> Neal Becker wrote: > Sorry, maybe I used confusing terminology. > > A reference is here: http://fedoraproject.org/wiki/Packaging/Python > This is the current setup. For example, this is a standard macro used by > Redhat in RPM SPEC files for python: > > %define python_sitearch %(%{__python} -c "from distutils.sysconfig import > get_python_lib; print get_python_lib(1)")} > > %define python_sitelib %(%{__python} -c "from distutils.sysconfig import > get_python_lib; print get_python_lib()")} > > Clearly this practice is widespread. It would seem that module search needs > some modification to fully support it. Ah. That isn't supported at all, at the moment. Redhat should not be using it. Instead, there shouldn't be a difference between sitearch and sitelib. Regards, Martin From brett at python.org Fri Mar 17 22:08:19 2006 From: brett at python.org (Brett Cannon) Date: Fri, 17 Mar 2006 13:08:19 -0800 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: References: <4418CDD0.8020803@canterbury.ac.nz> <17434.10109.675484.243338@montanaro.dyndns.org> <2m3bhh30fu.fsf@starship.python.net> <441A7C8A.9030404@voidspace.org.uk> <17434.61837.491103.995641@montanaro.dyndns.org> Message-ID: On 3/17/06, Georg Brandl wrote: > John J Lee wrote: > > >> In your formulation the comma binds more tightly than the as keyword. > >> In import statements it's the other way around. That seems like it > >> might be a source of confusion. > > > > Perhaps parentheses around the exception list should be mandatory for the > > 2-or-more exceptions case? > > > > except NameError as e: --> fine > > except (NameError) as e: --> fine > > except (NameError,) as e: --> fine > > > > except NameError, OtherError as e: --> SyntaxError > > except (NameError, OtherError) as e: --> fine > > I don't like that particularly, but I guess that for consistency's sake > it would have to be done this way. > I don't like it period. What consistency problem is there? This is Python 3 we are talking about, so if something makes good sense such as not having parentheses, I say leave them off. There are multiple places where parens could be required but are not, like multiple assignment. -Brett From mal at egenix.com Fri Mar 17 22:11:36 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 17 Mar 2006 22:11:36 +0100 Subject: [Python-Dev] [Python-checkins] r43041 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> Message-ID: <441B2608.2010900@egenix.com> Thomas Heller wrote: > Martin v. L?wis wrote: >> M.-A. Lemburg wrote: >>> I think it's only fair that I ask the patch authors to complete >>> the PEP, since the ssize_t patch is causing extension authors >>> enough trouble already. >> Well, the PEP is complete as it stands. It's possible to provide >> more guidelines, but the specification part of it says precisely >> what I intend it to say. Also, the API documentation ought to >> be in the Python documentation, and, for these changes, it is. >> >>> If you want quick adoption of the changes, you have >>> to make it as easy as possible for the authors to port their >>> extensions to the new API. Otherwise, we'll end up having >>> quite a large number of users who can't switch to Python 2.5 >>> simply because their favorite extensions don't work with it. >> I don't see how giving a complete list of all changed functions >> helps in any way. > > I'm not sure if this is what Marc-Andre means, but maybe these definitions > could go into a new include file: > > #if (PY_VERSION_HEX < 0x02050000) > typedef int Py_ssize_t; > #define lenfunc inquiry > #define readbufferproc getreadbufferproc > #define writebufferproc getwritebufferproc > #define segcountproc getsegcountproc > #define charbufferproc getcharbufferproc > #define ssizeargfunc intargfunc > #define ssizessizeargfunc intintargfunc > #define ssizeobjargproc intobjargproc > #define ssizessizeobjargproc intintobjargproc > ... more defines > #endif I think it would be enough to have this code snippet (completed, of course) in the PEP, so that authors can grab it and paste it into their extensions header files. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 17 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Mar 17 22:14:05 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 17 Mar 2006 22:14:05 +0100 Subject: [Python-Dev] [Python-checkins] r43041 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: <441B221A.5090900@v.loewis.de> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> <441B221A.5090900@v.loewis.de> Message-ID: <441B269D.20504@egenix.com> Martin v. L?wis wrote: > Thomas Heller wrote: >> I'm not sure if this is what Marc-Andre means, but maybe these definitions >> could go into a new include file: > > How would that include file be used? You would have to copy it into your > own source base, and include it, right? We could put it into a b/w compatibility header file, e.g. #include "pycompat.h" -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 17 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From g.brandl at gmx.net Fri Mar 17 22:21:26 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 17 Mar 2006 22:21:26 +0100 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: References: <4418CDD0.8020803@canterbury.ac.nz> <17434.10109.675484.243338@montanaro.dyndns.org> <2m3bhh30fu.fsf@starship.python.net> <441A7C8A.9030404@voidspace.org.uk> <17434.61837.491103.995641@montanaro.dyndns.org> Message-ID: Brett Cannon wrote: > On 3/17/06, Georg Brandl wrote: >> John J Lee wrote: >> >> >> In your formulation the comma binds more tightly than the as keyword. >> >> In import statements it's the other way around. That seems like it >> >> might be a source of confusion. >> > >> > Perhaps parentheses around the exception list should be mandatory for the >> > 2-or-more exceptions case? >> > >> > except NameError as e: --> fine >> > except (NameError) as e: --> fine >> > except (NameError,) as e: --> fine >> > >> > except NameError, OtherError as e: --> SyntaxError >> > except (NameError, OtherError) as e: --> fine >> >> I don't like that particularly, but I guess that for consistency's sake >> it would have to be done this way. >> > > I don't like it period. What consistency problem is there? This is > Python 3 we are talking about, so if something makes good sense such > as not having parentheses, I say leave them off. There are multiple > places where parens could be required but are not, like multiple > assignment. I only think of consistency with the usage of "as" in the import statement. I know how it works, but I predict people will come and write except NameError as e, OtherError as f: Georg From brett at python.org Fri Mar 17 22:28:31 2006 From: brett at python.org (Brett Cannon) Date: Fri, 17 Mar 2006 13:28:31 -0800 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: References: <4418CDD0.8020803@canterbury.ac.nz> <17434.10109.675484.243338@montanaro.dyndns.org> <2m3bhh30fu.fsf@starship.python.net> <441A7C8A.9030404@voidspace.org.uk> <17434.61837.491103.995641@montanaro.dyndns.org> Message-ID: On 3/17/06, Georg Brandl wrote: > Brett Cannon wrote: > > On 3/17/06, Georg Brandl wrote: > >> John J Lee wrote: > >> > >> >> In your formulation the comma binds more tightly than the as keyword. > >> >> In import statements it's the other way around. That seems like it > >> >> might be a source of confusion. > >> > > >> > Perhaps parentheses around the exception list should be mandatory for the > >> > 2-or-more exceptions case? > >> > > >> > except NameError as e: --> fine > >> > except (NameError) as e: --> fine > >> > except (NameError,) as e: --> fine > >> > > >> > except NameError, OtherError as e: --> SyntaxError > >> > except (NameError, OtherError) as e: --> fine > >> > >> I don't like that particularly, but I guess that for consistency's sake > >> it would have to be done this way. > >> > > > > I don't like it period. What consistency problem is there? This is > > Python 3 we are talking about, so if something makes good sense such > > as not having parentheses, I say leave them off. There are multiple > > places where parens could be required but are not, like multiple > > assignment. > > I only think of consistency with the usage of "as" in the import statement. > I know how it works, but I predict people will come and write > > except NameError as e, OtherError as f: > Really? I don't. 'while' statement doesn't work that way either. I don't personally associate 'as' with any specific statement that uses it. Plus the above is bad since those should be separate 'except' clauses if people need to differentiate between what exceptions they caught. Otherwise it should be ambiguous and assigned to a single variable name. -Brett From theller at python.net Fri Mar 17 22:29:53 2006 From: theller at python.net (Thomas Heller) Date: Fri, 17 Mar 2006 22:29:53 +0100 Subject: [Python-Dev] [Python-checkins] r43041 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: <441B269D.20504@egenix.com> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> <441B221A.5090900@v.loewis.de> <441B269D.20504@egenix.com> Message-ID: M.-A. Lemburg wrote: > Martin v. L?wis wrote: >> Thomas Heller wrote: >>> I'm not sure if this is what Marc-Andre means, but maybe these definitions >>> could go into a new include file: >> How would that include file be used? You would have to copy it into your >> own source base, and include it, right? > > We could put it into a b/w compatibility header file, e.g. > > #include "pycompat.h" > #include "pep353.h" From p.f.moore at gmail.com Fri Mar 17 22:45:52 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 17 Mar 2006 21:45:52 +0000 Subject: [Python-Dev] bytes thoughts In-Reply-To: <20060317102938.F765.JCARLSON@uci.edu> References: <20060316090941.F73B.JCARLSON@uci.edu> <20060317102938.F765.JCARLSON@uci.edu> Message-ID: <79990c6b0603171345l4ac1cf55w6bd8f9ce874cf004@mail.gmail.com> On 3/17/06, Josiah Carlson wrote: > Considering your quote, I have thusfar not seen any opinion dissenting > from what I stated as 'obvious' behavior. FWIW, I have no objections to the behaviour you state as "obvious", but I didn't find it immediately obvious (although admittedly it didn't take me long to see why it *was* obvious :-)), so I'd appreciate the clarification being added to the PEP. Paul. From p.f.moore at gmail.com Fri Mar 17 23:02:07 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 17 Mar 2006 22:02:07 +0000 Subject: [Python-Dev] Expose the array interface in Python 2.5? In-Reply-To: References: Message-ID: <79990c6b0603171402pd7441aam69b66b92095b7566@mail.gmail.com> On 3/17/06, Thomas Heller wrote: > Travis E. Oliphant wrote: > > Would it be possible to add at least the C-struct array interface to the > > Python arrayobject in time for Python 2.5? > > I'm very much for that. > > > Is someone on this list willing to help make it happen? > > Unfortunately not me - I'm too busy with ctypes, and if the array interface > makes it into the core I will have to implement/use that in ctypes too. > Accessing Python arrays (Numeric arrays, Numeric array, or Numpy array) > as ctypes arrays, and vice versa, without copying any memory, would be a good thing. This seems to expose the key point here. What *exactly* is the work that Travis is requesting? (I have very little spare time, no VC7.1 to do builds, but I do have an interest and would possibly be willing to help, if it was clear what was involved...) If all that is required is a simple C definition added to the Python code, then it seems to me that would be (a) fairly easy, but (b) pretty useless (on the basis that a standards-style PEP like the DB API ones would do much the same, without any code changes). But as ctypes is now in the core, it seems likely that any addition of an array type to the core would mean that ctypes should support it - which means that the work Thomas mentions is required. There are quite likely other issues, as well. On the third hand, the PEP referenced from the page you quote (at http://svn.scipy.org/svn/PEP/PEP_genarray.txt) seems to specify an implementation - are you just asking for that to be added to the core? If so, why not submit this as a formal PEP, and see what happens from there? (For this route, one of the Numeric people *needs* to champion the PEP, IMHO, as only they are qualified to address any points that get raised in general discussion). Paul. From bob at redivi.com Fri Mar 17 23:15:38 2006 From: bob at redivi.com (Bob Ippolito) Date: Fri, 17 Mar 2006 14:15:38 -0800 Subject: [Python-Dev] towards a stricter definition of sys.executable In-Reply-To: <441A75FF.6020709@v.loewis.de> References: <200603161152.34271.fdrake@acm.org> <441A75FF.6020709@v.loewis.de> Message-ID: <12D2B122-BA57-4654-8207-16C31AF85C3A@redivi.com> On Mar 17, 2006, at 12:40 AM, Martin v. L?wis wrote: > Fredrik Lundh wrote: >> I don't think many people embed setup.py scripts, so alternative >> (e) would pro- >> bably cause the least problems: >> >> e) sys.executable contains the full path to the program used >> to invoke >> this interpreter instance, or None if this could not be >> determined. > > It seems that you indeed are trying to solve a problem you > encountered. > Can you please explain what the problem is? > > ISTM that the current definition doesn't really cause problems, > despite > potentially being fuzzy. People that start sys.executable typically > *do* get a Python interpreter - in an embedded interpreter, they just > don't want to start a new interpreter, as that couldn't work, anyway. I've seen cases where people want to start worker processes from bundled apps (as in py2app/py2exe). The bootstrap executable (sys.executable) is not suitable for this purpose, as it runs a specific script. Forking doesn't quite do the right thing either because it's not safe to fork without exec'ing in all cases due to state that persists that shouldn't across processes with certain platform libraries (in Mac OS X especially). For py2app, we can bundle a Python interpreter that links to the same Python framework and has the same set of modules and extensions that the bundled application does, so we can support this use case. I'd definitely like to see something like sys.python_executable become standard, and I think I'll go ahead and support it in the next release of py2app. It's possible to degrade gracefully with this approach too: def get_python_executable(): python_executable = getattr(sys, 'python_executable', None) if python_executable is not None: return python_executable if not sys.frozen and sys.executable: # launched from a standard interpreter return sys.executable # frozen without python_executable support raise RuntimeError -bob From martin at v.loewis.de Fri Mar 17 23:16:45 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 17 Mar 2006 23:16:45 +0100 Subject: [Python-Dev] [Python-checkins] r43041 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: <441B20C5.1080009@egenix.com> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> <441B20C5.1080009@egenix.com> Message-ID: <441B354D.1030906@v.loewis.de> M.-A. Lemburg wrote: > Changes to the public API must be documented somewhere, > either in Misc/NEWS or in the PEP. Sure. They are documented in the PEP, which says # All occurrences of index and length parameters and results are changed # to use Py_ssize_t, including the sequence slots in type objects, and # the buffer interface. The relevant Misc/NEWS entry is - PEP 353: Using ssize_t as the index type. > It documents those changes, allows extension authors and extension > users to check their extensions for possible problems and is (or > at least should be) standard procedure. The changes are documented in the PEP. The current API is documented in the API docs. > The list of changes is also necessary in order to be able > to write code which allows a module to work in both Python > 2.4 and 2.5. The code snippet that Thomas suggested > should be part of the conversion guidelines. It is (atleast some form of it). > """ > Module authors have the choice whether they support this PEP in their > code or not; if they support it, they have the choice of different > levels of compatibility. > > If a module is not converted to support this PEP, it will continue to > work unmodified on a 32-bit system. On a 64-bit system, compile-time > errors and warnings might be issued, and the module might crash the > interpreter if the warnings are ignored. > """ > > Do you really think module authors do have a choice given that last > sentence ? I really do. Most developers will not be confronted with 64-bit systems for several years to come. That current hardware supports a 64-bit mode is only one aspect: Most operating system installations on such hardware will continue to operate in 32-bit mode for quite some time. Of course, developers of widely-used extension modules will need to support the PEP in some way. They should follow the guidelines in the PEP. Regards, Martin From amk at amk.ca Sat Mar 18 00:26:49 2006 From: amk at amk.ca (A.M. Kuchling) Date: Fri, 17 Mar 2006 18:26:49 -0500 Subject: [Python-Dev] Py3K thought: use external library for client-side HTTP Message-ID: <20060317232649.GA7358@rogue.amk.ca> Thought: We should drop all of httplib, urllib, urllib2, and ftplib, and instead adopt some third-party library for HTTP/FTP/whatever, write a Python wrapper, and use it instead. (The only such library I know of is libcurl, but doubtless there are other candidates; see http://curl.haxx.se/libcurl/competitors.html for a list.) Rationale: * HTTP client-side support is pretty complicated. HTTP itself has many corners (httplib.py alone is 1420 lines long, and urllib2 is 1300 lines). * There are many possible permutations of proxying, SSL on/off, and authentication. We probably haven't tested every permutation, and probably lack the volunteer effort to test them all. If you search for 'http' in the bug tracker, you find about 16 or so bugs submitted for httplib/urllib/urllib2, most of them for one permutation or another. With a third-party library, the work of maintaining RFC compliance falls to someone else. * A third-party library might support more features than we have time to implement. A downside: these libraries would be in C, and might be the source of security bugs. Python code may be buggy, but probably won't fall prey to buffer overflow. We'd also have to keep in sync with the library. Similar arguments could be made for a server-side solution, but here I have no idea what we might choose. A server-side HTTP implementation + a WSGI gateway might be all that Python 3000 needs. Good idea? Dumb idea? --amk From martin at v.loewis.de Fri Mar 17 23:36:16 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 17 Mar 2006 23:36:16 +0100 Subject: [Python-Dev] Py3K thought: use external library for client-side HTTP In-Reply-To: <20060317232649.GA7358@rogue.amk.ca> References: <20060317232649.GA7358@rogue.amk.ca> Message-ID: <441B39E0.2000500@v.loewis.de> A.M. Kuchling wrote: > Good idea? Dumb idea? I have no idea about urllib, but httplib certainly should stay, and its bugs should get fixed (and will get fixed, over time). It is very easy to write a httplib application, since you can follow its source code. It would be much more complicated to use a native library. Regards, Marttin From brett at python.org Fri Mar 17 23:43:24 2006 From: brett at python.org (Brett Cannon) Date: Fri, 17 Mar 2006 14:43:24 -0800 Subject: [Python-Dev] Py3K thought: use external library for client-side HTTP In-Reply-To: <20060317232649.GA7358@rogue.amk.ca> References: <20060317232649.GA7358@rogue.amk.ca> Message-ID: On 3/17/06, A.M. Kuchling wrote: > Thought: We should drop all of httplib, urllib, urllib2, and ftplib, > and instead adopt some third-party library for HTTP/FTP/whatever, > write a Python wrapper, and use it instead. (The only such library I > know of is libcurl, but doubtless there are other candidates; see > http://curl.haxx.se/libcurl/competitors.html for a list.) > > Rationale: > > * HTTP client-side support is pretty complicated. HTTP itself > has many corners (httplib.py alone is 1420 lines long, and urllib2 > is 1300 lines). > > * There are many possible permutations of proxying, SSL on/off, > and authentication. We probably haven't tested every permutation, > and probably lack the volunteer effort to test them all. > If you search for 'http' in the bug tracker, you find about 16 or so > bugs submitted for httplib/urllib/urllib2, most of them for one > permutation or another. > > With a third-party library, the work of maintaining RFC compliance falls > to someone else. > > * A third-party library might support more features than we have time > to implement. > > A downside: these libraries would be in C, and might be the source of > security bugs. Python code may be buggy, but probably won't fall prey > to buffer overflow. We'd also have to keep in sync with the library. > There is also the issue that PyPy could have problems since they have always preferred we keep pure Python versions of stuff around when possible (I assume IronPython has .NET Internet libraries to use). > Similar arguments could be made for a server-side solution, but here I > have no idea what we might choose. A server-side HTTP implementation > + a WSGI gateway might be all that Python 3000 needs. > > Good idea? Dumb idea? Possibly good. We have the precendent of zlib, expat, etc. The key is probably the license is compatible with ours (which libcurl seems to be: MIT/X derivative). I know that having fixed urllib bugs I sure wouldn't mind if I didn't have to read another RFC on URL formats. =) But maybe this also poses a larger question of where for Py3K we want to take the stdlib. Ignoring its needed cleanup and nesting of the namespace, do we want to try to use more external tools by importing them and writing a Pythonic wrapper? Or do we want to not do that and try to keep more things under our control and go with the status quo? Or do we want to really prune down the stdlib and use more dynamic downloading ala Cheeseshop and setuptools? I support the first even though it makes problems for PyPy since it should allow us to have more quality code with less effort on our part. I also support the second since we seem to be able to pull it off. For the third option I would want to be very careful with what is and is not included since Python's "batteries included" solution is an important part of Python and I would not want that to suffer. -Brett From brett at python.org Sat Mar 18 00:44:16 2006 From: brett at python.org (Brett Cannon) Date: Fri, 17 Mar 2006 15:44:16 -0800 Subject: [Python-Dev] dealing with decorators hiding metadata of decorated functions Message-ID: With the discussion of a possible @decorator to help set the metadata of the decorator to that of what the wrapped function has, I had an idea that I wanted to toss out there (this dicussion stems from a blog post I made: http://sayspy.blogspot.com/2006/03/how-to-handle-object-identity-issues.html). The problem I have with the current practice of setting __name__ and __doc__ (updating __dict__ I have no problem with) is that is destroys metadata on the decorator. This seems kind of bad to destroy metadata for introspection purposes. So how about a __decorates__ attribute that points to what something is decorating? Then help() along with 'inspect' can be changed to look for that attribute, and if it is there, use that for introspection. This way advanced users can still poke at the decorator for information while more common introspection will still point to the decorated function instead of the decorator itself, all without data loss! And on top of it all, it's simple to add support for (obviously the actual code in 'inspect' would be different than what is below =) !:: _help = help def help(obj): while hasattr(obj, "__decorates__"): obj = obj.__decorates__ _help(obj) Worst case is advanced users at the command line will be put out by not being able to directly look at obj.__doc__ for the docstring they care about and such. But this should be a minor issue. But if people don't like that, I do have another proposal to add a __signature__ attribute that would hold an object representing the call signature of a function. This could then be put on a decorator so that introspection on the calling requirements of a decorated function are correct instead of the typical ``*args, **kwargs``. Anyway, what do people think of a __decorates__ attribute? -Brett From jcarlson at uci.edu Sat Mar 18 01:18:22 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Fri, 17 Mar 2006 16:18:22 -0800 Subject: [Python-Dev] dealing with decorators hiding metadata of decorated functions In-Reply-To: References: Message-ID: <20060317161349.F76D.JCARLSON@uci.edu> "Brett Cannon" wrote: > > With the discussion of a possible @decorator to help set the metadata > of the decorator to that of what the wrapped function has, I had an > idea that I wanted to toss out there (this dicussion stems from a blog > post I made: http://sayspy.blogspot.com/2006/03/how-to-handle-object-identity-issues.html). [snip] Edward Loper suggested this way back on September 5, 2004. http://mail.python.org/pipermail/python-dev/2004-September/048626.html I was and continue to be +1 on this, though I would go farther and state, like I did at the time, that one shouldn't copy any of the function attributes, they should come 'free', similar to the way that class attributes are 'free' on subclasses. http://mail.python.org/pipermail/python-dev/2004-September/048631.html What would make this _really_ nice is if one didn't need to do anything manually; that the attribute that pointed to the decorated function/object would be automatically applied - though I realize that this may not be generally possible. - Josiah From ndbecker2 at gmail.com Sat Mar 18 01:38:29 2006 From: ndbecker2 at gmail.com (Neal Becker) Date: Fri, 17 Mar 2006 19:38:29 -0500 Subject: [Python-Dev] Problem with module loading on multi-arch? References: <4419D9EE.4040603@v.loewis.de> <441A73F0.1090404@v.loewis.de> <441B240A.9000909@v.loewis.de> Message-ID: "Martin v. L?wis" wrote: > Neal Becker wrote: >> Sorry, maybe I used confusing terminology. >> >> A reference is here: http://fedoraproject.org/wiki/Packaging/Python >> This is the current setup. For example, this is a standard macro used by >> Redhat in RPM SPEC files for python: >> >> %define python_sitearch %(%{__python} -c "from distutils.sysconfig import >> get_python_lib; print get_python_lib(1)")} >> >> %define python_sitelib %(%{__python} -c "from distutils.sysconfig import >> get_python_lib; print get_python_lib()")} >> >> Clearly this practice is widespread. It would seem that module search >> needs some modification to fully support it. > > Ah. That isn't supported at all, at the moment. Redhat should not be > using it. Instead, there shouldn't be a difference between sitearch and > sitelib. > x86_64 is multiarch. That means, we allow both i386 and x86_64 binaries to coexits. Is the proposal that python should not support this? That would be unfortunate. I suspect is would not be that difficult to correctly support multiarch platforms. As it is, this usually works - but the example I gave above shows where it seems to break. From thomas at python.org Sat Mar 18 02:45:21 2006 From: thomas at python.org (Thomas Wouters) Date: Sat, 18 Mar 2006 02:45:21 +0100 Subject: [Python-Dev] Problem with module loading on multi-arch? In-Reply-To: References: <4419D9EE.4040603@v.loewis.de> <441A73F0.1090404@v.loewis.de> <441B240A.9000909@v.loewis.de> Message-ID: <9e804ac0603171745i134ac7dbpf8365ed9466ab897@mail.gmail.com> On 3/18/06, Neal Becker wrote: > x86_64 is multiarch. That means, we allow both i386 and x86_64 binaries > to > coexits. Is the proposal that python should not support this? That would > be unfortunate. Not within a single interpreter, I'd say. Extensions compiled for x86_64 will not load into an interpreter compiled for x86, and vice-versa, so that's out of the question. What else should python support? You can, of course, have different interpreters live next to each other -- python already supports that just fine. Python shouldn't care about architectures, though, and just use 'prefix' and 'exec_prefix' as they were given at compiletime; they contain the properly compiled extensions for that interpreter. For the purpose of multi-archedness, python and all its modules and extensions are a single application. Perhaps Py3K should differentiate more between .so files and .py files (and .pyc/.pyo files), but I'm not entirely convinced of the value of installing a module once for multiple interpreters. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060318/e8102c1f/attachment.html From bob at redivi.com Sat Mar 18 02:52:22 2006 From: bob at redivi.com (Bob Ippolito) Date: Fri, 17 Mar 2006 17:52:22 -0800 Subject: [Python-Dev] Problem with module loading on multi-arch? In-Reply-To: References: <4419D9EE.4040603@v.loewis.de> <441A73F0.1090404@v.loewis.de> <441B240A.9000909@v.loewis.de> Message-ID: <9598D041-5C3C-419F-80D3-B5581648142E@redivi.com> On Mar 17, 2006, at 4:38 PM, Neal Becker wrote: > "Martin v. L?wis" wrote: > >> Neal Becker wrote: >>> Sorry, maybe I used confusing terminology. >>> >>> A reference is here: http://fedoraproject.org/wiki/Packaging/Python >>> This is the current setup. For example, this is a standard macro >>> used by >>> Redhat in RPM SPEC files for python: >>> >>> %define python_sitearch %(%{__python} -c "from >>> distutils.sysconfig import >>> get_python_lib; print get_python_lib(1)")} >>> >>> %define python_sitelib %(%{__python} -c "from distutils.sysconfig >>> import >>> get_python_lib; print get_python_lib()")} >>> >>> Clearly this practice is widespread. It would seem that module >>> search >>> needs some modification to fully support it. >> >> Ah. That isn't supported at all, at the moment. Redhat should not be >> using it. Instead, there shouldn't be a difference between >> sitearch and >> sitelib. >> > > x86_64 is multiarch. That means, we allow both i386 and x86_64 > binaries to > coexits. Is the proposal that python should not support this? > That would > be unfortunate. > > I suspect is would not be that difficult to correctly support > multiarch > platforms. As it is, this usually works - but the example I gave > above > shows where it seems to break. All the difficult issues supporting multi-arch are going to be with distutils, not Python itself. On OS X it isn't all that hard to support (beyond backwards compatibility issues) because you run gcc once with the right options and get a single universal binary as output. It would be a lot more invasive if GCC had to be run multiple times and the products had to be put in different places. -bob From ndarray at mac.com Sat Mar 18 03:19:11 2006 From: ndarray at mac.com (Sasha) Date: Sat, 18 Mar 2006 02:19:11 +0000 (UTC) Subject: [Python-Dev] Expose the array interface in Python 2.5? References: <441B09B7.6070403@comcast.net> Message-ID: "Travis E. Oliphant" ieee.org> wrote: > It is very important for many people to get access to memory with some > description of how that memory should be interpreted as an array. > Several Python packages could benefit if Python had some notion of an > array interface that was at least supported in a duck-typing fashion. I have just submited a patch to SF that implements __array_shape__ and __array_typestr__ attributes for the array.array class. These are the only two attributes that are marked "required" in the "N-D array interface" document. I think it makes sense to discuss these two first. Note that these methods do not add any new functionality. Shape is just (len(a),) and typestr just packs endiness information together with a.typecode and a.itemsize. The only benefit of adding these two methods is to create "one true way" to access this information. I am fairly happy with __array_shape__, but __array_typestr__ feels somewhat archaic. I would rather see ctypes style data description instead of this struct style. I believe unifying the way different python modules describe binary data is a worthwhile first step in bringing array interface to the core. From tim.peters at gmail.com Sat Mar 18 04:24:01 2006 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 17 Mar 2006 22:24:01 -0500 Subject: [Python-Dev] All green! Message-ID: <1f7befae0603171924w37d1ce21o3f118dcaa6ea9885@mail.gmail.com> After backporting a patch from Neal (to make test_timeout less likely to fail on the Mac buildbot slave living at python.org), all 14 columns in http://www.python.org/dev/buildbot/all/ are green. Yippee! Let's keep 'em that way. You can all take the weekend off ;-) I believe it's possible to fiddle the buildbot master to send email when a test run fails, and maybe it's time to start doing that. BTW, there's still one compiler warning on Windows, on the trunk: """ Compiling... _ctypes_test.c \buildbot_py25\trunk.mcintyre-windows\build\Modules\_ctypes\_ctypes_test.c(102) : warning C4267: 'initializing' : conversion from 'size_t' to 'int', possible loss of data """ From ncoghlan at gmail.com Sat Mar 18 04:47:05 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 18 Mar 2006 13:47:05 +1000 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: <20060317202345.GA19451@phd.pp.ru> References: <4418CDD0.8020803@canterbury.ac.nz> <17434.10109.675484.243338@montanaro.dyndns.org> <2m3bhh30fu.fsf@starship.python.net> <441A7C8A.9030404@voidspace.org.uk> <17434.61837.491103.995641@montanaro.dyndns.org> <20060317202345.GA19451@phd.pp.ru> Message-ID: <441B82B9.4030607@gmail.com> Oleg Broytmann wrote: > On Fri, Mar 17, 2006 at 07:44:51PM +0000, John J Lee wrote: >> Perhaps parentheses around the exception list should be mandatory for the >> 2-or-more exceptions case? >> >> except NameError as e: --> fine >> except (NameError) as e: --> fine >> except (NameError,) as e: --> fine >> >> except NameError, OtherError as e: --> SyntaxError >> except (NameError, OtherError) as e: --> fine > > If parens are mandatory what is the difference from the current syntax? > Why do we need "as" here? Why not stay with > > except (NameError, OtherError), e: Currently the comma is used for two very different purposes within the same clause, so it's far too easy to make the mistake: except NameError, OtherError: and then have to deal with the fact that OtherError isn't caught (and given how often unit tests fail to cover all error cases, it wouldn't be unusual for this bug to escape a unit testing regime, too). If this code is made to "do the right thing", then there needs to be some other syntax to indicate the name that should be used for the value of the caught exception. "as" is already used as an embedded naming syntax in two places: - import statements use it for "locate an object or module using one name, but bind it to a different name in this module" In this case, the binding is restricted to a single name and is applied individually to each element of a comma separated list: import foo as a, bar as b, baz as c - with statements use it for "retrieve the context manager for this context and bind the result of the manager's __enter__ method to this name" In this case, the binding may be either a single name or a tuple of names, and the expression itself is limited to a single term - parentheses are required around both comma separated lists and unpacking to separate names is optional: with (foo, bar, baz) as (a, b, c): pass with (foo, bar, baz) as x: pass The proposal to use it for except clauses encounters a subtle difference - in the except clause, even if multiple exception types are listed in the clause, there is still only a single exception value to be bound to the name. The list of exception names merely limits the possible types for that exception, rather than their names. So, as a somewhat novel approach, what about putting the "as" *before* the list of exceptions types? After all, the thing being bound is the exception caught, not the possible list of types: except NameError, OtherError: # Exception value not bound except as ex, NameError, OtherError: # Exception value bound to ex I don't actually have a problem with simply tacking the clause on the end (with or without mandatory parentheses) but figured I'd throw this idea out as an option. I'm not sure whether the parallel with "print" and "print >> stream," is a positive or a negative, though. . . Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Sat Mar 18 05:09:13 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 18 Mar 2006 14:09:13 +1000 Subject: [Python-Dev] Expose the array interface in Python 2.5? In-Reply-To: References: <441A8848.2090004@gmail.com> Message-ID: <441B87E9.3050703@gmail.com> Travis E. Oliphant wrote: >> Given that even the bytes type has been deferred to 2.6 to allow further >> consideration of the appropriate API, my vote is to do the same for an >> array.dimarray type and allow more time to figure out the appropriate *Python* >> interface. > > I was afraid of that. But, unless people in pydev actually care to > discuss these matters, I fear that yet again nothing will be done. The > problem is that for most of us array users, it's only community outreach > and a desire to get people using Python talking the same array language > that makes us really care about these things. The NumPy library works > fine for what we really need it to do, and it's hard to get motivated to > convince people that haven't used an array-language like IDL or > MATLAB in the past to understand the reasons for NumPy's behavior. Hmm, we could have the builtin type support only access to individual elements (raising an IndexError or ValueError for any slice operations, or failures to specify all dimensions). It wouldn't be that easy to *manipulate* data in that format, but it would meet the needs of a common memory format for the various array implementions that *do* make the data easy to manipulate. Doing that would allow us to get a basic type while ducking most (all?) of the potentially controversial behaviour. Such a type would definitely belong in the array module rather than being a builtin, though. > As the bytes type is developed please keep in mind it's uses as the > memory for an N-dimensional array. Perhaps the bytes object could be a > default way (or built on a default way) to allocate memory. A simple > reference-counted memory object would certainly belay the problems of > the buffer interface that the array object currently has problems with. Even if array.dimarray was implemented as a fairly dumb implementation of the multi-dimensional array interface, then other types could either inherit from it or contain it and manipulate the memory directly. > In other words, the array object should not malloc it's own memory but > create a memory object which is nothing more than a reference-counted > pointer to memory. Surely this has been talked about. Is there a reason > it has not been implemented? It would not be that hard. A simple implementation of array.dimarray could certainly serve as such an object. Note that we don't have to do everything at once. It would be possible to put a transition plan in a PEP whereby array.dimarray was added in Python 2.5 (allowing external modules to start relying on it), while converting things like ctypes and array.array to use it could be deferred until 2.6. If we defer the whole lot, then even if the standard library used a common bulk data format in 2.6, extension modules probably wouldn't be using it until 2.7. Of course, there's a whole 'nother question of where Jython, IronPython and PyPy would fit into this. . . Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Sat Mar 18 05:15:27 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 18 Mar 2006 14:15:27 +1000 Subject: [Python-Dev] All green! In-Reply-To: <1f7befae0603171924w37d1ce21o3f118dcaa6ea9885@mail.gmail.com> References: <1f7befae0603171924w37d1ce21o3f118dcaa6ea9885@mail.gmail.com> Message-ID: <441B895F.2050500@gmail.com> Tim Peters wrote: > After backporting a patch from Neal (to make test_timeout less likely > to fail on the Mac buildbot slave living at python.org), all 14 > columns in > > http://www.python.org/dev/buildbot/all/ > > are green. Yippee! Let's keep 'em that way. You can all take the > weekend off ;-) > > I believe it's possible to fiddle the buildbot master to send email > when a test run fails, and maybe it's time to start doing that. Neal's already set it up to email the Python checkins list when something breaks. It already told me about a bug in the PEP 338 tests that meant they wouldn't work on Windows. The combination of -uall and rerunning with -v when a test fails is really quite effective. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From greg.ewing at canterbury.ac.nz Sat Mar 18 05:17:50 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 18 Mar 2006 17:17:50 +1300 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: References: <4418CDD0.8020803@canterbury.ac.nz> <17434.10109.675484.243338@montanaro.dyndns.org> <2m3bhh30fu.fsf@starship.python.net> <441A7C8A.9030404@voidspace.org.uk> <17434.61837.491103.995641@montanaro.dyndns.org> Message-ID: <441B89EE.8000303@canterbury.ac.nz> Georg Brandl wrote: > I predict people will come and write > > except NameError as e, OtherError as f: Then they'll learn very fast not to write that, because they'll get a SyntaxError. Greg From greg.ewing at canterbury.ac.nz Sat Mar 18 05:46:14 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 18 Mar 2006 17:46:14 +1300 Subject: [Python-Dev] Expose the array interface in Python 2.5? In-Reply-To: <441A8848.2090004@gmail.com> References: <441A8848.2090004@gmail.com> Message-ID: <441B9096.6040000@canterbury.ac.nz> Nick Coghlan wrote: > The former seems fairly pointless, and the latter difficult (since it has > implications for moving the data store when the array gets resized). I don't see why that should be a serious problem, as long as it's understood that the address reported by the array interface is only for temporary use. I also don't see what relevance the semantics of Python indexing or mutation has. The array interface should just report the location and shape of the data as it happens to be when the call is made. What happens to it in between times is entirely up to the object. Greg From aleaxit at gmail.com Sat Mar 18 06:02:17 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Fri, 17 Mar 2006 21:02:17 -0800 Subject: [Python-Dev] All green! In-Reply-To: <1f7befae0603171924w37d1ce21o3f118dcaa6ea9885@mail.gmail.com> References: <1f7befae0603171924w37d1ce21o3f118dcaa6ea9885@mail.gmail.com> Message-ID: Can't think of a more Pythonic way to celebrate St Patrick's day!-) Alex From ncoghlan at gmail.com Sat Mar 18 06:40:09 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 18 Mar 2006 15:40:09 +1000 Subject: [Python-Dev] dealing with decorators hiding metadata of decorated functions In-Reply-To: <20060317161349.F76D.JCARLSON@uci.edu> References: <20060317161349.F76D.JCARLSON@uci.edu> Message-ID: <441B9D39.9090000@gmail.com> Josiah Carlson wrote: > "Brett Cannon" wrote: >> With the discussion of a possible @decorator to help set the metadata >> of the decorator to that of what the wrapped function has, I had an >> idea that I wanted to toss out there (this dicussion stems from a blog >> post I made: http://sayspy.blogspot.com/2006/03/how-to-handle-object-identity-issues.html). > > [snip] > > Edward Loper suggested this way back on September 5, 2004. > > http://mail.python.org/pipermail/python-dev/2004-September/048626.html > > I was and continue to be +1 on this, +1 here, too. Unlike Brett, though, I have no problem with overwriting __name__ and updating __dict__ unconditionally, and overwriting __doc__ if it hasn't already been set. The first two are needed if we expect "print f" and "f.a" to work properly. The function's name is set on the 'def' line, and I'd be much happier seeing that at all levels of the decorator chain, rather than seeing something like "wrapper", "wrapper", ... "wrapper", "f". Annotating decorators will modify the functions' attributes, and this needs to be visible in the final function's dictionary. If a wrapper doesn't set a docstring explicitly, it makes a lot more sense to me to re-uses the original function's docstring rather than leave it at None. My real interest is that it should be possible to get at all the details of the original function (such as its code object), and the obvious way to do that is with a standard attribute that links to the original. > though I would go farther and state, > like I did at the time, that one shouldn't copy any of the function > attributes, they should come 'free', similar to the way that class > attributes are 'free' on subclasses. Well, that's the idea behind the decorator decorator - simply put @decorator on your decorator function and it will automatically do the right thing. > http://mail.python.org/pipermail/python-dev/2004-September/048631.html > > What would make this _really_ nice is if one didn't need to do anything > manually; that the attribute that pointed to the decorated > function/object would be automatically applied - though I realize that > this may not be generally possible. A slight problem is that not all decorators will wrap the function they decorate - some will only annotate it. However, here's an idea for the @decorator decorator that would make it pretty much automatic, leaves the docstring alone if the decorator has already set it on the wrapper, and builds up a record of the decorators that are wrapping the the original function: def _link_decorated(decorated, orig, decorator): """Link a decorated function with the original""" decorated.__name__ = orig.__name__ decorated.__dict__.update(orig.__dict__) if decorated.__doc__ is None: decorated.__doc__ = orig.__doc__ decorated.__decorates__ = orig decorated.__decorator__ = decorator def decorator(orig_decorator): """Decorator to create a well-behaved decorator""" # Wrapper function that links a decorated function # to the original function if necessary def wrapper(f): decorated = orig_decorator(f) if decorated is not f: # Link wrapper function to the original _link_decorated(decorated, f, wrapper) return decorated _link_decorated(wrapper, orig_decorator, decorator) return wrapper Decorators that only do annotations aren't recorded because there isn't anywhere to record them. Wrapping decorators, on the other hand, allow the references to both the decorated function and the applied decorator to be stored on the new function object. Cheers, Nick. P.S. Example usage: Py> @decorator ... def annotated(f): ... f.note = 1 ... return f ... Py> @decorator ... def wrapped(f): ... def wrapper(*args, **kwds): ... return f(*args, **kwds) ... return wrapper ... Py> @wrapped ... @annotated ... @wrapped ... @wrapped ... def show(*args, **kwds): ... print args, kwds ... Py> while hasattr(obj, "__decorates__"): ... print obj ... print " Decorates:\t%s" % obj.__decorates__ ... print " Using:\t%s" % obj.__decorator__ ... print " Annotated?:\t%s" % hasattr(obj, "note") ... print ... obj = obj.__decorates__ ... Decorates: Using: Annotated?: True Decorates: Using: Annotated?: True Decorates: Using: Annotated?: False -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ronaldoussoren at mac.com Sat Mar 18 08:04:47 2006 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Sat, 18 Mar 2006 08:04:47 +0100 Subject: [Python-Dev] [Python-checkins] r43041 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: <441B269D.20504@egenix.com> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> <441B221A.5090900@v.loewis.de> <441B269D.20504@egenix.com> Message-ID: On 17-mrt-2006, at 22:14, M.-A. Lemburg wrote: > Martin v. L?wis wrote: >> Thomas Heller wrote: >>> I'm not sure if this is what Marc-Andre means, but maybe these >>> definitions >>> could go into a new include file: >> >> How would that include file be used? You would have to copy it >> into your >> own source base, and include it, right? > > We could put it into a b/w compatibility header file, e.g. > > #include "pycompat.h" But wouldn't this header be needed on versions of python before 2.5? That would make inclusion of a pycompat.h header with python 2.5 less useful. Including the completed block into the pep would be useful. Ronald > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, Mar 17 > 2006) >>>> Python/Zope Consulting and Support ... http:// >>>> www.egenix.com/ >>>> mxODBC.Zope.Database.Adapter ... http:// >>>> zope.egenix.com/ >>>> mxODBC, mxDateTime, mxTextTools ... http:// >>>> python.egenix.com/ > ______________________________________________________________________ > __ > > ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for > free ! :::: > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ > ronaldoussoren%40mac.com From nnorwitz at gmail.com Sat Mar 18 08:48:35 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Fri, 17 Mar 2006 23:48:35 -0800 Subject: [Python-Dev] Python 2.5 Schedule Message-ID: I posted a message to c.l.p about the upcoming alpha 1. Just in case anybody here's been snoozing, 2.5 alpha 1 is coming up real quick, hopefully within a couple of weeks. If you have any *major* features (particularly implemented in C) that you want to see in 2.5, bring it up now. I want to strive for feature completeness by alpha 1. I know we will have some .py modules that won't make it into alpha 1, but they really should make it in by alpha 2 or be deferred to 2.6. For any important bugs or patches, add comments to them. There's still a bunch that I want to see go in. If all goes well, with alphas 1 & 2, maybe we could even skip an alpha 3? We can make a beta 3 if things don't go as well as I anticipate. I'm hoping the buildbot helped shake out a lot of the bugs we might have seen in the alphas. We'll have to wait and see what makes sense. n From martin at v.loewis.de Sat Mar 18 08:59:36 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 18 Mar 2006 08:59:36 +0100 Subject: [Python-Dev] Py_ssize_t backwards compatibility In-Reply-To: <4419CE7A.2030500@python.net> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> Message-ID: <441BBDE8.40402@v.loewis.de> Thomas Heller wrote: > #if (PY_VERSION_HEX < 0x02050000) > typedef int Py_ssize_t; > #define lenfunc inquiry > #define readbufferproc getreadbufferproc > #define writebufferproc getwritebufferproc > #define segcountproc getsegcountproc > #define charbufferproc getcharbufferproc > #define ssizeargfunc intargfunc > #define ssizessizeargfunc intintargfunc > #define ssizeobjargproc intobjargproc > #define ssizessizeobjargproc intintobjargproc > ... more defines > #endif Thinking about this, I now believe (again) that this isn't what people should do. Instead, they should avoid these type names altogether, and should avoid casting function pointers. The reason they currently do that is because of the self pointer: the self pointer should have the specific type, instead of being PyObject*, so that you can access the fields of the object. The reason they should avoid this is that this casting is error-prone: If they make a mistake in the signature (e.g. forget to replace int with Py_ssize_t), the compiler will not tell them. This happened to me a few times while converting modules to Py_ssize_t; you need to be very careful when using these casts. OTOH, the solution to avoid them is pretty mechanic, and straight-forward. Just drop the casts, and replace Py_ssize_t foo_len(Foo* self) { ... } with Py_ssize_t foo_len(PyObject* _self) { Foo* self = (Foo*)_self; ... } Then, foo_len will already have the type lenfunc, and no cast is required. If you have also #if (PY_VERSION_HEX < 0x02050000) typedef int Py_ssize_t; #endif then this code will work unmodified in earlier versions: foo_len will be of type int (*)(PyObject*), which is precisely the type inquiry that it should have in the earlier versions. See http://sourceforge.net/tracker/index.php?func=detail&aid=1453037&group_id=71702&atid=532156 for an example of applying this approach to ctypes. Regards, Martin From martin at v.loewis.de Sat Mar 18 09:04:38 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 18 Mar 2006 09:04:38 +0100 Subject: [Python-Dev] PY_SSIZE_T_CLEAN In-Reply-To: <4419CE7A.2030500@python.net> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> Message-ID: <441BBF16.60802@v.loewis.de> Thomas Heller wrote: > Then, from only reading the PEP without looking up the sources, > it is not clear to me what the PY_SIZE_T_CLEAN definition does. The description says The conversion codes 's#' and 't#' will output Py_ssize_t if the macro PY_SSIZE_T_CLEAN is defined before Python.h is included, and continue to output int if that macro isn't defined. which (IMO, of course), specifies precisely what PY_SSIZE_T_CLEAN does. What information do you find lacking here? Regards, Martin From martin at v.loewis.de Sat Mar 18 09:19:13 2006 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 18 Mar 2006 09:19:13 +0100 Subject: [Python-Dev] Problem with module loading on multi-arch? In-Reply-To: References: <4419D9EE.4040603@v.loewis.de> <441A73F0.1090404@v.loewis.de> <441B240A.9000909@v.loewis.de> Message-ID: <441BC281.5040409@v.loewis.de> Neal Becker wrote: > x86_64 is multiarch. That means, we allow both i386 and x86_64 binaries to > coexits. Is the proposal that python should not support this? That would > be unfortunate. No. You should install the Python library to the architecture-specific directory (/usr/lib64/python2.3 for AMD64 and /usr/lib/python2.3 for x86, I guess). The library destination shouldn't be split into to locations. > I suspect is would not be that difficult to correctly support multiarch > platforms. If you think you can fix it - go ahead. As I said: I keep applying patches from people who claim they know how things ought to work, much without questioning them. Regards, Martin From martin at v.loewis.de Sat Mar 18 10:08:12 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 18 Mar 2006 10:08:12 +0100 Subject: [Python-Dev] All green! In-Reply-To: <1f7befae0603171924w37d1ce21o3f118dcaa6ea9885@mail.gmail.com> References: <1f7befae0603171924w37d1ce21o3f118dcaa6ea9885@mail.gmail.com> Message-ID: <441BCDFC.1070403@v.loewis.de> Tim Peters wrote: > I believe it's possible to fiddle the buildbot master to send email > when a test run fails, and maybe it's time to start doing that. I just changed to do that, let's see whether that works. Regards, Martin From mal at egenix.com Sat Mar 18 10:28:22 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat, 18 Mar 2006 10:28:22 +0100 Subject: [Python-Dev] [Python-checkins] r43041 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> <441B221A.5090900@v.loewis.de> <441B269D.20504@egenix.com> Message-ID: <441BD2B6.3030407@egenix.com> Ronald Oussoren wrote: > On 17-mrt-2006, at 22:14, M.-A. Lemburg wrote: > >> Martin v. L?wis wrote: >>> Thomas Heller wrote: >>>> I'm not sure if this is what Marc-Andre means, but maybe these >>>> definitions >>>> could go into a new include file: >>> How would that include file be used? You would have to copy it >>> into your >>> own source base, and include it, right? >> We could put it into a b/w compatibility header file, e.g. >> >> #include "pycompat.h" > > But wouldn't this header be needed on versions of python before 2.5? Yes. Ideally it should work on more Python versions than just Python 2.5. I have such a compatibility header file for the mx Extensions (called mxpyapi.h and included in egenix-mx-base). It includes #defines such as the one Thomas proposed for Python back to version 1.5. > That > would make inclusion of a pycompat.h header with python 2.5 less useful. Why is that ? For older versions you can copy it into your extension's include directory. With the usual #ifdef PYCOMPAT_H wrapper this won't get included if Python already includes the header file via #include "Python.h". > Including the completed block into the pep would be useful. > > Ronald > > >> -- >> Marc-Andre Lemburg >> eGenix.com >> >> Professional Python Services directly from the Source (#1, Mar 17 >> 2006) >>>>> Python/Zope Consulting and Support ... http:// >>>>> www.egenix.com/ >>>>> mxODBC.Zope.Database.Adapter ... http:// >>>>> zope.egenix.com/ >>>>> mxODBC, mxDateTime, mxTextTools ... http:// >>>>> python.egenix.com/ >> ______________________________________________________________________ >> __ >> >> ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for >> free ! :::: >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ >> ronaldoussoren%40mac.com > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/mal%40egenix.com -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From jcarlson at uci.edu Sat Mar 18 10:37:04 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat, 18 Mar 2006 01:37:04 -0800 Subject: [Python-Dev] Python 2.5 Schedule In-Reply-To: References: Message-ID: <20060318013021.F779.JCARLSON@uci.edu> "Neal Norwitz" wrote: > > I posted a message to c.l.p about the upcoming alpha 1. > > Just in case anybody here's been snoozing, 2.5 alpha 1 is coming up > real quick, hopefully within a couple of weeks. If you have any > *major* features (particularly implemented in C) that you want to see > in 2.5, bring it up now. I want to strive for feature completeness by > alpha 1. I know we will have some .py modules that won't make it into > alpha 1, but they really should make it in by alpha 2 or be deferred > to 2.6. (as previously requested August 12, 2005) (http://mail.python.org/pipermail/python-dev/2005-August/055356.html) > For 2.5a1... > > Some exposure of _PyLong_AsByteArray() and _PyLong_FromByteArray() to > Python. There was a discussion about this almost a year ago > (http://python.org/sf/1023290), and no mechanism (struct format code > addition, binascii.tolong/fromlong, long.tostring/fromstring, ...) > actually made it into Python 2.4 . At this point, I'd be happy to get > /any/ mechanism, with a preference to struct and/or binascii (I'd put > them in both, if only because different groups of people people may look > for them in both places, and people who use one tend to like to use that > one for as much as possible, and because the code additions in both are > minor). Raymond followed up with the following: (http://mail.python.org/pipermail/python-dev/2005-August/055358.html) > Assign 1023290 to me and I'll get it done in the next month or so. It was assigned, but he didn't get around to it at the time. I can easily update the patch/test/documentation for struct, but my CPython abilities are somewhat lacking, and I wouldn't be comfortable writing two new functions in the binascii module. - Josiah From fredrik at pythonware.com Sat Mar 18 10:46:44 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sat, 18 Mar 2006 10:46:44 +0100 Subject: [Python-Dev] Python 2.5 Schedule References: Message-ID: Neal Norwitz wrote: > Just in case anybody here's been snoozing, 2.5 alpha 1 is coming up > real quick, hopefully within a couple of weeks. If you have any > *major* features (particularly implemented in C) that you want to see > in 2.5, bring it up now. I want to strive for feature completeness by > alpha 1. I know we will have some .py modules that won't make it into > alpha 1, but they really should make it in by alpha 2 or be deferred > to 2.6. the xmlplus/xmlcore situation needs to be sorted out. From ncoghlan at iinet.net.au Sat Mar 18 13:53:28 2006 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Sat, 18 Mar 2006 22:53:28 +1000 Subject: [Python-Dev] GeneratorExit inheriting from Exception Message-ID: <441C02C8.5080504@iinet.net.au> Should GeneratorExit inherit from Exception or BaseException? Currently, a generator that catches Exception and continues on to yield another value can't be closed properly (you get a runtime error pointing out that the generator ignored GeneratorExit). The only decent reference I could find to it in the old PEP 348/352 discussions is Guido writing [1]: > when GeneratorExit or StopIteration > reach the outer level of an app, it's a bug like all the others that > bare 'except:' WANTS to catch. (at that point in the conversation, I believe bare except was considered the equivalent of "except Exception:") While I agree with what Guido says about GeneratorExit being a bug if it reaches the outer level of an app, it seems like a bit of a trap that a correctly written generator can't write "except Exception:" without preceding it with an "except GeneratorExit:" that reraises the exception. Isn't that exactly the idiom we're trying to get rid of for SystemExit and KeyboardInterrupt? Regards, Nick. [1] http://mail.python.org/pipermail/python-dev/2005-August/055173.html -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From barry at python.org Sat Mar 18 16:10:22 2006 From: barry at python.org (Barry Warsaw) Date: Sat, 18 Mar 2006 10:10:22 -0500 Subject: [Python-Dev] Merging email 4.0 to Python 2.5 svn trunk In-Reply-To: <1142606417.32074.21.camel@geddy.wooz.org> References: <1142606417.32074.21.camel@geddy.wooz.org> Message-ID: <1142694622.10599.0.camel@resist.wooz.org> On Fri, 2006-03-17 at 09:40 -0500, Barry Warsaw wrote: > The fix is simple, change 'P' to 'p' in the test, but I want to make > sure that nobody cares that __import__('email.Parser') now returns an > object that isn't a module, but acts enough like the original module for > all intents and purposes. I think the answer is "no", so I'm going to commit these changes. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060318/769c87f9/attachment.pgp From barry at python.org Sat Mar 18 16:26:45 2006 From: barry at python.org (Barry Warsaw) Date: Sat, 18 Mar 2006 10:26:45 -0500 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <441C02C8.5080504@iinet.net.au> References: <441C02C8.5080504@iinet.net.au> Message-ID: <1142695605.32074.49.camel@geddy.wooz.org> On Sat, 2006-03-18 at 22:53 +1000, Nick Coghlan wrote: > Should GeneratorExit inherit from Exception or BaseException? Actually, this prompts me to write about an issue I have with PEP 352. I actually don't think it's necessary (yes, I know it's already in the tree). What I would much rather is is for StandardError to be renamed Error, for Exception to remain the base class of the exception hierarchy, and for KeyboardInterrupt to be moved to inherit directly from Exception. GeneratorExit, SystemExit, and StopIteration would continue to inherit from Exception. The reasoning is this: anything that can be raised is an Exception. Not all Exceptions are Errors. Anything that signals an error condition is an Error, and anything that signals a warning condition is a Warning. Thus the basic hierarchy /ought/ to be: Exception +- KeyboardInterrupt +- GeneratorExit +- SystemExit +- StopIteration +- Error | +- ImportError | +- (etc.) | +- Warning +- UserWarning +- (etc.) Use defined errors should inherit from Error, not Exception. With this, "except Exception" would be a synonym for bare except, while "except Error" would be the standard idiom for letting non-error exceptions pass through. I don't know whether this is possible for Python 2.5, but I think it should be what we strive for for Py3K, and I do not think BaseException is at all necessary. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060318/c03643b9/attachment.pgp From g.brandl at gmx.net Sat Mar 18 16:31:13 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 18 Mar 2006 16:31:13 +0100 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <1142695605.32074.49.camel@geddy.wooz.org> References: <441C02C8.5080504@iinet.net.au> <1142695605.32074.49.camel@geddy.wooz.org> Message-ID: Barry Warsaw wrote: > On Sat, 2006-03-18 at 22:53 +1000, Nick Coghlan wrote: >> Should GeneratorExit inherit from Exception or BaseException? > > Actually, this prompts me to write about an issue I have with PEP 352. > I actually don't think it's necessary (yes, I know it's already in the > tree). > > What I would much rather is is for StandardError to be renamed Error, > for Exception to remain the base class of the exception hierarchy, and > for KeyboardInterrupt to be moved to inherit directly from Exception. > GeneratorExit, SystemExit, and StopIteration would continue to inherit > from Exception. > > The reasoning is this: anything that can be raised is an Exception. Not > all Exceptions are Errors. Anything that signals an error condition is > an Error, and anything that signals a warning condition is a Warning. > Thus the basic hierarchy /ought/ to be: > > Exception > +- KeyboardInterrupt > +- GeneratorExit > +- SystemExit > +- StopIteration > +- Error > | +- ImportError > | +- (etc.) > | > +- Warning > +- UserWarning > +- (etc.) Cool! That's so far the clearest solution. An additional bonus is that except statements look nicer: except: # still catches all Exceptions, just like except Exception: except Error: # is what you normally should do Cheers, Georg From mattjfleming at googlemail.com Sat Mar 18 16:39:28 2006 From: mattjfleming at googlemail.com (Matthew Fleming) Date: Sat, 18 Mar 2006 15:39:28 +0000 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: References: <441C02C8.5080504@iinet.net.au> <1142695605.32074.49.camel@geddy.wooz.org> Message-ID: <5ff4a1e50603180739i72c946dbg@mail.gmail.com> > > except Error: # is what you normally should do +1 This definatley makes more sense and is easier to understand just from glancing at it. Thanks, Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060318/8afe9bce/attachment.htm From pedronis at strakt.com Sat Mar 18 16:50:18 2006 From: pedronis at strakt.com (Samuele Pedroni) Date: Sat, 18 Mar 2006 16:50:18 +0100 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <1142695605.32074.49.camel@geddy.wooz.org> References: <441C02C8.5080504@iinet.net.au> <1142695605.32074.49.camel@geddy.wooz.org> Message-ID: <441C2C3A.8070003@strakt.com> Barry Warsaw wrote: > On Sat, 2006-03-18 at 22:53 +1000, Nick Coghlan wrote: > >>Should GeneratorExit inherit from Exception or BaseException? > > > Actually, this prompts me to write about an issue I have with PEP 352. > I actually don't think it's necessary (yes, I know it's already in the > tree). > > What I would much rather is is for StandardError to be renamed Error, > for Exception to remain the base class of the exception hierarchy, and > for KeyboardInterrupt to be moved to inherit directly from Exception. > GeneratorExit, SystemExit, and StopIteration would continue to inherit > from Exception. > > The reasoning is this: anything that can be raised is an Exception. Not > all Exceptions are Errors. Anything that signals an error condition is > an Error, and anything that signals a warning condition is a Warning. > Thus the basic hierarchy /ought/ to be: > > Exception > +- KeyboardInterrupt > +- GeneratorExit > +- SystemExit > +- StopIteration > +- Error > | +- ImportError > | +- (etc.) > | > +- Warning > +- UserWarning > +- (etc.) > > Use defined errors should inherit from Error, not Exception. With this, > "except Exception" would be a synonym for bare except, while "except > Error" would be the standard idiom for letting non-error exceptions pass > through. > > I don't know whether this is possible for Python 2.5, well, one thing to consider is all the class MyException(Exception): in current code. > but I think it > should be what we strive for for Py3K, and I do not think BaseException > is at all necessary. > > -Barry > > > > ------------------------------------------------------------------------ > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/pedronis%40strakt.com From barry at python.org Sat Mar 18 16:51:44 2006 From: barry at python.org (Barry Warsaw) Date: Sat, 18 Mar 2006 10:51:44 -0500 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <441C2C3A.8070003@strakt.com> References: <441C02C8.5080504@iinet.net.au> <1142695605.32074.49.camel@geddy.wooz.org> <441C2C3A.8070003@strakt.com> Message-ID: <1142697104.32089.58.camel@geddy.wooz.org> On Sat, 2006-03-18 at 16:50 +0100, Samuele Pedroni wrote: > > I don't know whether this is possible for Python 2.5, > > well, one thing to consider is all the > > class MyException(Exception): > > in current code. Yep, which is why I'm not sure we can do this for Py2.5. However as PEP 352 talks about a transition plan for Py3k, I think we should document the ultimate desired hierarchy (and maybe implement that in the p3yk branch ;), and then think about how to transition to it in Py2.5. One possible approach is to revert BaseException out of Py2.5, re-position KeyboardInterrupt, and add Error as an alias for StandardError. Then we can encourage people to start using Error as the base classes for their own errors. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060318/cc54f9ea/attachment.pgp From python at rcn.com Sat Mar 18 17:48:39 2006 From: python at rcn.com (Raymond Hettinger) Date: Sat, 18 Mar 2006 11:48:39 -0500 Subject: [Python-Dev] Python 2.5 Schedule References: Message-ID: <005f01c64aab$cf0e1ef0$b83efea9@RaymondLaptop1> [Neal] >I posted a message to c.l.p about the upcoming alpha 1. > > Just in case anybody here's been snoozing, 2.5 alpha 1 is coming up > real quick, hopefully within a couple of weeks. If you have any > *major* features (particularly implemented in C) that you want to see > in 2.5, bring it up now. I want to strive for feature completeness by > alpha 1. I know we will have some .py modules that won't make it into > alpha 1, but they really should make it in by alpha 2 or be deferred > to 2.6. FYI, I have several non-major C components to go in but not in time for alpha 1. They include some minor fix-ups in the sets module, the str.partition function, add gc to itertools.tee, a couple of functions in binascii, add itertools.izip_longest, Crutcher's patch to make exec take dictionary arguments, move the peepholer to just before the assembler, and implement Alex's iterator copier for a number of iterables (xrange, repeat, count, reversed, list, tuple, deque, dict, and set). All of these have been previously discussed/approved and should go in to the second alpha. The only one that is borderline is Crutcher's patch. I will try to free up some time to get that into alpha 1. It touches critical parts of the interpreted and needs to be reviewed, tested, timed, and exercised thoroughly. Raymond From fperez.net at gmail.com Sat Mar 18 18:28:23 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 18 Mar 2006 10:28:23 -0700 Subject: [Python-Dev] [Python-checkins] r43041 - python/trunk/Modules/_ctypes/cfield.c References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> <441B20C5.1080009@egenix.com> <441B354D.1030906@v.loewis.de> Message-ID: "Martin v. L?wis" wrote: > M.-A. Lemburg wrote: >> Do you really think module authors do have a choice given that last >> sentence ? > > I really do. Most developers will not be confronted with 64-bit systems > for several years to come. That current hardware supports a 64-bit mode > is only one aspect: Most operating system installations on such hardware > will continue to operate in 32-bit mode for quite some time. I think it's worth pointing out that the above is not true in a fairly significant market: AMD hardware under Linux. Just about any AMD chip you can buy for a desktop these days is 64-bit, and all major linux distributions have out-of-the box native x86-64 support (via a native build downloadable as a separate install CD, typically). So while it may well be true that most Win32 users who have 64 bit hardware will still be using it in 32 bit mode, in the Linux world it is /extremely/ common to find native 64 bit users. If you want confirmation, stop by the scipy list anytime for any of the recurrent battles being fought on the 64 bit front (often related to obscure compilation problems with Fortran code). So I think M.A. is right on the money here with his statement. Unless you consider the Linux/64bit camp insignificant. But if that is the case, it might be worth putting a big statement in the 2.5 release notes indicating "there is a good chance, if you use third party extensions and a 64 bit OS, that this won't work for you". Which will mean that a fraction of the scientific userbase (a big, enthusiastic and growing community of python users) will have to stick to 2.4. Regards, f From tzot at mediconsa.com Sat Mar 18 18:28:36 2006 From: tzot at mediconsa.com (Christos Georgiou) Date: Sat, 18 Mar 2006 19:28:36 +0200 Subject: [Python-Dev] Patch or feature? Tix.Grid working for 2.5 Message-ID: I made a plea for help months ago (just checked, and it was Jan 2004! time flies like a fruit or something, ref http://mail.python.org/pipermail/python-list/2004-January/202704.html ) about directions to fix the borken Tix.Grid widget; I had no replies. I finally found some spare time (too little, though, much work, wife expecting etc) and I started work on it, and at the moment it can be added in a window and has some functionality, but I need a lot more finding my way about using tcl/tk --I have almost zero knowledge of the language. I would like to know if supplying a patch for it sometime in the next couple of weeks would be considered a patch (since the widget currently is not working at all, its class in Tix.py contains just a pass statement) or a feature (ie extra functionality) for the 2.5 branch... From g.brandl at gmx.net Sat Mar 18 18:57:12 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 18 Mar 2006 18:57:12 +0100 Subject: [Python-Dev] __dict__ strangeness In-Reply-To: <1hce2nh.hnzrzu1qt4yv2N%aleaxit@yahoo.com> References: <482lr6Fhs1jvU1@individual.net> <1hce2nh.hnzrzu1qt4yv2N%aleaxit@yahoo.com> Message-ID: [moving to python-dev] Alex Martelli wrote: > Georg Brandl wrote: > >> can someone please tell me that this is correct and why: > > IMHO, it is not correct: it is a Python bug (and it would be nice to fix > it in 2.5). Fine. Credits go to Michal Kwiatkowski for discovering that in bug #1448042 which I closed out of ignorance ;) >> >>> class C(object): >> ... pass >> ... >> >>> c = C() >> >>> c.a = 1 >> >>> c.__dict__ >> {'a': 1} >> >>> c.__dict__ = {} >> >>> c.a >> Traceback (most recent call last): >> File "", line 1, in ? >> AttributeError: 'C' object has no attribute 'a' > > So far so good, I think we agree;-). Yes. >> >>> class D(object): >> ... __dict__ = {} >> ... >> >>> d = D() >> >>> d.a = 1 >> >>> d.__dict__ >> {} >> >>> d.__dict__ = {} >> >>> d.a >> 1 > > Yep, that's the bug, fully reproducible in 2.3 and 2.4. FWIW, mucking > around with gc.getreferrers (with a more uniquely identifiable value for > d.a;-) shows a dictionary "somewhere" with keys 'a' and '__dict__'... Georg From barry at python.org Sat Mar 18 19:06:14 2006 From: barry at python.org (Barry Warsaw) Date: Sat, 18 Mar 2006 13:06:14 -0500 Subject: [Python-Dev] Python 2.5 Schedule In-Reply-To: References: Message-ID: <1142705174.32089.69.camel@geddy.wooz.org> On Fri, 2006-03-17 at 23:48 -0800, Neal Norwitz wrote: > Just in case anybody here's been snoozing, 2.5 alpha 1 is coming up > real quick, hopefully within a couple of weeks. If you have any > *major* features (particularly implemented in C) that you want to see > in 2.5, bring it up now. I want to strive for feature completeness by > alpha 1. I know we will have some .py modules that won't make it into > alpha 1, but they really should make it in by alpha 2 or be deferred > to 2.6. I'd like to get some feedback on my PEP 352 comments, and if there's general agreement on the hierarchy I proposed (so far so good :), then I'd like to take a crack at implementing them. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060318/0393b4c6/attachment.pgp From barry at python.org Sat Mar 18 19:07:33 2006 From: barry at python.org (Barry Warsaw) Date: Sat, 18 Mar 2006 13:07:33 -0500 Subject: [Python-Dev] Python 2.5 Schedule In-Reply-To: References: Message-ID: <1142705253.32074.72.camel@geddy.wooz.org> On Fri, 2006-03-17 at 23:48 -0800, Neal Norwitz wrote: > Just in case anybody here's been snoozing, 2.5 alpha 1 is coming up > real quick, hopefully within a couple of weeks. If you have any > *major* features (particularly implemented in C) that you want to see > in 2.5, bring it up now. I want to strive for feature completeness by > alpha 1. I know we will have some .py modules that won't make it into > alpha 1, but they really should make it in by alpha 2 or be deferred > to 2.6. Oh, also, we have a couple of additions to the PySet C API. I'll work on putting together an SF patch for them over the weekend. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060318/cda7d83e/attachment.pgp From Scott.Daniels at Acm.Org Sat Mar 18 19:13:25 2006 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Sat, 18 Mar 2006 10:13:25 -0800 Subject: [Python-Dev] Expose the array interface in Python 2.5? In-Reply-To: <441B9096.6040000@canterbury.ac.nz> References: <441A8848.2090004@gmail.com> <441B9096.6040000@canterbury.ac.nz> Message-ID: Greg Ewing wrote: > Nick Coghlan wrote: > >> The former seems fairly pointless, and the latter difficult (since it has >> implications for moving the data store when the array gets resized). > > I don't see why that should be a serious problem, as > long as it's understood that the address reported by > the array interface is only for temporary use. > > I also don't see what relevance the semantics of > Python indexing or mutation has. The array interface > should just report the location and shape of the > data as it happens to be when the call is made. > What happens to it in between times is entirely > up to the object. But it might be nice to be able to determine if the object has a way of promising a constant address while a reference is held. The array.array implementation could simply say, "nope." I certainly have use for implementations that can give better guarantees, and I'd like to be able to distinguish the two. --Scott David Daniels Scott.Daniels at Acm.Org From rasky at develer.com Sat Mar 18 19:15:10 2006 From: rasky at develer.com (Giovanni Bajo) Date: Sat, 18 Mar 2006 19:15:10 +0100 Subject: [Python-Dev] Python 2.5 Schedule References: <005f01c64aab$cf0e1ef0$b83efea9@RaymondLaptop1> Message-ID: <02aa01c64ab7$e4a35da0$bf03030a@trilan> Raymond Hettinger wrote: > They include [...] the str.partition function, Where is the current version of this patch? After reharsing the archives, I have an additional suggestion which I didn't find already mentioned in the discussion. What about: str.partition() -> (left, right or None) str.rparition() -> (left or None, right) which means: "foo:bar".partition(":") -> ("foo", "bar") "foo:bar".rpartition(":") -> ("foo", "bar") "foo:".partition(":") -> ("foo", "") "foo:".rpartition(":") -> ("foo", "") ":foo".partition(":") -> ("", "foo") ":foo".rpartition(":") -> ("", "foo") "foo".partition(":") -> ("foo", None) "foo".rpartition(":") -> (None, "foo") Notice that None-checking can be done as a way to know if the separator was found. I mentally went through the diff here (http://mail.python.org/pipermail/python-dev/2005-August/055781.html) and found out that most (all?) the usages of '_' disappears with this semantic. -- Giovanni Bajo From rasky at develer.com Sat Mar 18 19:32:36 2006 From: rasky at develer.com (Giovanni Bajo) Date: Sat, 18 Mar 2006 19:32:36 +0100 Subject: [Python-Dev] GeneratorExit inheriting from Exception References: <441C02C8.5080504@iinet.net.au><1142695605.32074.49.camel@geddy.wooz.org> Message-ID: <038501c64aba$545d2200$bf03030a@trilan> Georg Brandl wrote: >> Exception >> +- KeyboardInterrupt >> +- GeneratorExit >> +- SystemExit >> +- StopIteration >> +- Error >>> +- ImportError >>> +- (etc.) >>> >> +- Warning >> +- UserWarning >> +- (etc.) > > Cool! That's so far the clearest solution. An additional bonus is that > except > statements look nicer: > > except: # still catches all Exceptions, just like > except Exception: > > except Error: # is what you normally should do +1 on the general idea, I just don't specifically like that "except:" is the "wrong" thing to do: part of the PEP352 idea was that people writing "except:" out of ignorance would still not cause their program to intercept KeyboardInterrupt, or StopIteration. Unless this new proposal also includes changing the meaning of "except:" to "except Error". Also, under this new proposal, we could even remove Exception from the builtins namespace in Py3k. It's almost always wrong to use it, and if you really really need it, it's spelled exceptions.Exception. -- Giovanni Bajo From barry at python.org Sat Mar 18 19:50:13 2006 From: barry at python.org (Barry Warsaw) Date: Sat, 18 Mar 2006 13:50:13 -0500 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <038501c64aba$545d2200$bf03030a@trilan> References: <441C02C8.5080504@iinet.net.au> <1142695605.32074.49.camel@geddy.wooz.org> <038501c64aba$545d2200$bf03030a@trilan> Message-ID: <1142707813.10599.15.camel@resist.wooz.org> On Sat, 2006-03-18 at 19:32 +0100, Giovanni Bajo wrote: > +1 on the general idea, I just don't specifically like that "except:" is the > "wrong" thing to do: part of the PEP352 idea was that people writing > "except:" out of ignorance would still not cause their program to intercept > KeyboardInterrupt, or StopIteration. > > Unless this new proposal also includes changing the meaning of "except:" to > "except Error". It's worth debating. OT1H, it's a semantic different for Python 2.x (although +1 on the idea for Py3K). OTOH, I think people generally want to just catch errors and not all exceptions. Going along with that, maybe the interpreter should do something different when an Exception that's not an Error reaches the top (e.g. not print a traceback if KeyboardInterrupt is seen -- we usually just catch that, print "Interrupted" and exit). > Also, under this new proposal, we could even remove > Exception from the builtins namespace in Py3k. It's almost always wrong to > use it, and if you really really need it, it's spelled exceptions.Exception. I'm not sure I'd go as far as hiding Exception, since I don't think the penalty is that great and it makes it easier to document. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060318/e487e46d/attachment.pgp From rasky at develer.com Sat Mar 18 20:13:50 2006 From: rasky at develer.com (Giovanni Bajo) Date: Sat, 18 Mar 2006 20:13:50 +0100 Subject: [Python-Dev] GeneratorExit inheriting from Exception References: <441C02C8.5080504@iinet.net.au> <1142695605.32074.49.camel@geddy.wooz.org> <038501c64aba$545d2200$bf03030a@trilan> <1142707813.10599.15.camel@resist.wooz.org> Message-ID: <0a4e01c64ac0$174b39a0$524d2697@bagio> Barry Warsaw wrote: > >Unless this new proposal also includes changing the meaning of >> "except:" to "except Error". > It's worth debating. OT1H, it's a semantic different for Python 2.x > (although +1 on the idea for Py3K). I was speaking of Py3K here, yes. > Going along with that, maybe the interpreter should do something > different when an Exception that's not an Error reaches the top > (e.g. not print a traceback if KeyboardInterrupt is seen -- > we usually just catch that, print "Interrupted" and exit). SystemExit is already special-cased, as far as I can tell. KeyboardInterrupt could be in fact be special cased as well (I saw many Python newbies -- but otherwise experienced -- being disgusted at first when they interrupt their code with CTRL+C: they expect the program to exit "almost silently"). >> Also, under this new proposal, we could even remove >> Exception from the builtins namespace in Py3k. It's almost >> always wrong to use it, and if you really really need it, it's >> spelled exceptions.Exception. > I'm not sure I'd go as far as hiding Exception, since I don't think the > penalty is that great and it makes it easier to document. The situation (in Py3k) I was thinking is when people see this code: except: # something and want to change it so to get a name to the exception object. I *think* many could get confused and write: except Exception, e: # something which changes the meaning. It "sounds" correct, but it's wrong. Of course, it's easy to argue that "Exception" is just that, and people actually meant "Error". In a way, the current PEP352 is superior here because it makes harder to do the "bad" thing by giving it a complex name (BaseException). Giovanni Bajo From tim.peters at gmail.com Sat Mar 18 20:14:38 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sat, 18 Mar 2006 14:14:38 -0500 Subject: [Python-Dev] Two buildbot slaves wedged Message-ID: <1f7befae0603181114u7089fb28j154a38b522778d5e@mail.gmail.com> svn got confused trying to incorporate updates to the email pkg. Slaves sparc solaris10 gcc trunk and x86 gentoo trunk got stuck in their trunk "updating" steps for 7 hours, presumably as a result. I killed those builds, and tried to start new builds on those slaves. There was no effect apart from their "pending" count rising from 6 to 7. They respond to pings, but won't start another build. I'm not sure how they get into such a state (but saw it at Zope Corp at times too). It's possible that their buildbot processes need to be restarted, and/or that the master process needs to be restarted. From tds333+pydev at gmail.com Sat Mar 18 20:31:22 2006 From: tds333+pydev at gmail.com (Wolfgang Langner) Date: Sat, 18 Mar 2006 20:31:22 +0100 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: <441B89EE.8000303@canterbury.ac.nz> References: <4418CDD0.8020803@canterbury.ac.nz> <17434.10109.675484.243338@montanaro.dyndns.org> <2m3bhh30fu.fsf@starship.python.net> <441A7C8A.9030404@voidspace.org.uk> <17434.61837.491103.995641@montanaro.dyndns.org> <441B89EE.8000303@canterbury.ac.nz> Message-ID: <4c45c1530603181131s58b3d598i@mail.gmail.com> Hello, what about: try: something except NameError or OtherError as e: Only a thought. -- bye by Wolfgang From raymond.hettinger at verizon.net Sat Mar 18 20:51:27 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sat, 18 Mar 2006 14:51:27 -0500 Subject: [Python-Dev] Python 2.5 Schedule References: <1142705253.32074.72.camel@geddy.wooz.org> Message-ID: <005c01c64ac5$589859b0$b83efea9@RaymondLaptop1> [Barry Warsaw] > Oh, also, we have a couple of additions to the PySet C API. > I'll work on putting together an SF patch for them over the weekend. What are you proposing to add to the PySet API? Raymond From aahz at pythoncraft.com Sat Mar 18 21:21:51 2006 From: aahz at pythoncraft.com (Aahz) Date: Sat, 18 Mar 2006 12:21:51 -0800 Subject: [Python-Dev] Python Library Reference top page too long In-Reply-To: References: <4419A633.6010607@comcast.net> <441A070D.10002@canterbury.ac.nz> Message-ID: <20060318202151.GA27514@panix.com> On Fri, Mar 17, 2006, Georg Brandl wrote: > Greg Ewing wrote: >> Russell E. Owen wrote: >>> >>> Another thing that would be *really* useful is to list the actual >>> built-in types with the category. For example: >>> Sequence Types (str, unicode, list, tuple, buffer, xrange) >>> Mapping Types (dict) >> >> +1 > > If I understand correctly what you mean, that's already done. Partially, by my standard. That is, the names of the types are now listed, but you still have to drill down two layers to get to the string methods. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From nnorwitz at gmail.com Sat Mar 18 23:05:42 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sat, 18 Mar 2006 14:05:42 -0800 Subject: [Python-Dev] Two buildbot slaves wedged In-Reply-To: <1f7befae0603181114u7089fb28j154a38b522778d5e@mail.gmail.com> References: <1f7befae0603181114u7089fb28j154a38b522778d5e@mail.gmail.com> Message-ID: On 3/18/06, Tim Peters wrote: > svn got confused trying to incorporate updates to the email pkg. Slaves > > sparc solaris10 gcc trunk > and > x86 gentoo trunk > > got stuck in their trunk "updating" steps for 7 hours, presumably as a result. > > I killed those builds, and tried to start new builds on those slaves. > There was no effect apart from their "pending" count rising from 6 to > 7. They respond to pings, but won't start another build. I've seen this happen a few times. It seems the only way to fix this is to restart the master which I did. After the restart, all the pending changes are lost, so I forced a rebuild on both machines. n From martin at v.loewis.de Sat Mar 18 23:18:50 2006 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 18 Mar 2006 23:18:50 +0100 Subject: [Python-Dev] [Python-checkins] r43041 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> <441B20C5.1080009@egenix.com> <441B354D.1030906@v.loewis.de> Message-ID: <441C874A.1020305@v.loewis.de> Fernando Perez wrote: > So I think M.A. is right on the money here with his statement. Unless you > consider the Linux/64bit camp insignificant. But if that is the case, it > might be worth putting a big statement in the 2.5 release notes indicating > "there is a good chance, if you use third party extensions and a 64 bit OS, > that this won't work for you". Which will mean that a fraction of the > scientific userbase (a big, enthusiastic and growing community of python > users) will have to stick to 2.4. It's more intricate than that. If certain extensions are of interest to the scientific community, I certainly expect the authors of them to fix them. However, to "fix" the extension can still be done in different levels: you can easily fix it to not crash, by just honoring all compiler warnings that gcc produces. That doesn't mean you fully support Py_ssize_t, since you might not support collections with more than 2**31 elements. In these cases, the extension will not crash, but it will compute incorrect results if you ever encounter such a collection. Even on all these AMD64 Linux machines, this is still fairly unlikely, since you need considerably more than 16GiB main memory to do something useful on such collections - very few machines have that much memory today. Again, people that have such machines and need to run Python programs with that many data need to make sure that the extensions work for them. Sticking with 2.4 is no option for these people, since 2.4 doesn't support such large collections, anyway. To really fix such extensions, I recommend building with either Microsoft C or Intel C. The Microsoft compiler is available free of charge, but runs on Windows only. It gives good warnings, and if you fix them all (in a careful way), your code should fully support 64 bits. Likewise for the Intel compiler: it is available for free only for a month, but it runs on Linux as well. OTOH, I'm still doubtful how many extensions will be really affected by the change in the first place. Your code *breaks* with the change only if you implement the sequence or buffer protocols. I'm doubtful that this is an issue for most applications, since many extensions (I believe) work without implementing these protocols. Regards, Martin From martin at v.loewis.de Sat Mar 18 23:21:21 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 18 Mar 2006 23:21:21 +0100 Subject: [Python-Dev] Patch or feature? Tix.Grid working for 2.5 In-Reply-To: References: Message-ID: <441C87E1.3020709@v.loewis.de> Christos Georgiou wrote: > I would like to know if supplying a patch for it sometime in the next couple > of weeks would be considered a patch (since the widget currently is not > working at all, its class in Tix.py contains just a pass statement) or a > feature (ie extra functionality) for the 2.5 branch... I wouldn't object to including it before beta 1. Regards, Martin From martin at v.loewis.de Sat Mar 18 23:23:46 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 18 Mar 2006 23:23:46 +0100 Subject: [Python-Dev] Two buildbot slaves wedged In-Reply-To: <1f7befae0603181114u7089fb28j154a38b522778d5e@mail.gmail.com> References: <1f7befae0603181114u7089fb28j154a38b522778d5e@mail.gmail.com> Message-ID: <441C8872.8000707@v.loewis.de> Tim Peters wrote: > I'm not sure how they get into such a state (but saw it at Zope Corp > at times too). It's possible that their buildbot processes need to be > restarted, and/or that the master process needs to be restarted. It's a buildbot 0.7.1 bug (according to the 0.7.2 changelog); the sighup command sometimes gets confused. I thought it would survive the configuration change when no build is ongoing, but apparently I was wrong. Regards, Martin From nnorwitz at gmail.com Sat Mar 18 23:27:05 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sat, 18 Mar 2006 14:27:05 -0800 Subject: [Python-Dev] Patch or feature? Tix.Grid working for 2.5 In-Reply-To: <441C87E1.3020709@v.loewis.de> References: <441C87E1.3020709@v.loewis.de> Message-ID: On 3/18/06, "Martin v. L?wis" wrote: > Christos Georgiou wrote: > > I would like to know if supplying a patch for it sometime in the next couple > > of weeks would be considered a patch (since the widget currently is not > > working at all, its class in Tix.py contains just a pass statement) or a > > feature (ie extra functionality) for the 2.5 branch... > > I wouldn't object to including it before beta 1. Nor would I. I'm not sure if this helps or not, but a general rule of thumb is if it's 100 lines it's probably a patch. If it's 1000 lines, it's probably a major feature. Given such a patch is pure python and probably much closer to 100 lines, this should be ok. BTW, have you contacted the Tix people? It is a separate project and looks like people are working on it. http://tix.sourceforge.net/ n From nnorwitz at gmail.com Sat Mar 18 23:37:47 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sat, 18 Mar 2006 14:37:47 -0800 Subject: [Python-Dev] Python 2.5 Schedule In-Reply-To: <005f01c64aab$cf0e1ef0$b83efea9@RaymondLaptop1> References: <005f01c64aab$cf0e1ef0$b83efea9@RaymondLaptop1> Message-ID: On 3/18/06, Raymond Hettinger wrote: > > FYI, I have several non-major C components to go in but not in time for alpha 1. > They include some minor fix-ups in the sets module, the str.partition function, > add gc to itertools.tee, a couple of functions in binascii, add > itertools.izip_longest, Crutcher's patch to make exec take dictionary arguments, > move the peepholer to just before the assembler, and implement Alex's iterator > copier for a number of iterables (xrange, repeat, count, reversed, list, tuple, > deque, dict, and set). All of these have been previously discussed/approved and > should go in to the second alpha. > > The only one that is borderline is Crutcher's patch. I will try to free up some > time to get that into alpha 1. It touches critical parts of the interpreted and > needs to be reviewed, tested, timed, and exercised thoroughly. All those seem reasonable. Do the functions you mention address Josiah's patch? n From brett at python.org Sun Mar 19 00:37:14 2006 From: brett at python.org (Brett Cannon) Date: Sat, 18 Mar 2006 15:37:14 -0800 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <1142695605.32074.49.camel@geddy.wooz.org> References: <441C02C8.5080504@iinet.net.au> <1142695605.32074.49.camel@geddy.wooz.org> Message-ID: On 3/18/06, Barry Warsaw wrote: > On Sat, 2006-03-18 at 22:53 +1000, Nick Coghlan wrote: > > Should GeneratorExit inherit from Exception or BaseException? > > Actually, this prompts me to write about an issue I have with PEP 352. > I actually don't think it's necessary (yes, I know it's already in the > tree). > Much to personal pain and sprint time. Are you trying to make me shed a manly tear, Barry?!? =) > What I would much rather is is for StandardError to be renamed Error, > for Exception to remain the base class of the exception hierarchy, and > for KeyboardInterrupt to be moved to inherit directly from Exception. > GeneratorExit, SystemExit, and StopIteration would continue to inherit > from Exception. > So it isn't that PEP 352 is unnecessary since there is nothing here about deprecating string execptions and making built-in exceptions new-style. You just want to change the renaming of the exceptions. > The reasoning is this: anything that can be raised is an Exception. Not > all Exceptions are Errors. Anything that signals an error condition is > an Error, and anything that signals a warning condition is a Warning. > Thus the basic hierarchy /ought/ to be: > > Exception > +- KeyboardInterrupt > +- GeneratorExit > +- SystemExit > +- StopIteration > +- Error > | +- ImportError > | +- (etc.) > | > +- Warning > +- UserWarning > +- (etc.) > > Use defined errors should inherit from Error, not Exception. With this, > "except Exception" would be a synonym for bare except, while "except > Error" would be the standard idiom for letting non-error exceptions pass > through. > I still like my idea of a ControlFlowException, but that died with PEP 348 (along with part of my innocence). > I don't know whether this is possible for Python 2.5, but I think it > should be what we strive for for Py3K, and I do not think BaseException > is at all necessary. > I view PEP 352 as the PEP that fixed an issue with overreaching 'except' clauses, officially getting us off of string exceptions, and making built-in exceptions new-style classes. I do not view it as the be-all-end-all hierarchy for Py3K. With that said, I am not changing the PEP. Another PEP that suggests the hierarchy for Py3K can be suggested, but I am not doing it. I am totally burned out on exceptions after two PEPs on the subject matter. Just remember that PEP 348 was meant for Py3K when we are supposed to break stuff and how much resistence I hit. Granted my changes were more radical, but even in the end, small changes were resisted heavily. In other words, make sure you have the time and energy to take this on. =) But the above suggestions seem reasonable, especially if bare 'except' statements go away. So I am supportive of the idea. -Brett From python at rcn.com Sun Mar 19 00:38:34 2006 From: python at rcn.com (Raymond Hettinger) Date: Sat, 18 Mar 2006 18:38:34 -0500 Subject: [Python-Dev] Python 2.5 Schedule References: <005f01c64aab$cf0e1ef0$b83efea9@RaymondLaptop1> Message-ID: <00a001c64ae5$12aef560$b83efea9@RaymondLaptop1> [Raymond] > FYI, I have several non-major C components to go in but not in time for alpha > 1. > They include some minor fix-ups in the sets module, the str.partition > function, > add gc to itertools.tee, a couple of functions in binascii, add > itertools.izip_longest, Crutcher's patch to make exec take dictionary > arguments, > move the peepholer to just before the assembler, and implement Alex's iterator > copier for a number of iterables (xrange, repeat, count, reversed, list, > tuple, > deque, dict, and set). All of these have been previously discussed/approved > and > should go in to the second alpha. > > The only one that is borderline is Crutcher's patch. I will try to free up > some > time to get that into alpha 1. It touches critical parts of the interpreted > and > needs to be reviewed, tested, timed, and exercised thoroughly. [Neal] > All those seem reasonable. Do the functions you mention address Josiah's > patch? I believe so. They are binascii.b2long() and binascii.long2b(). And, no, the name of the latter wasn't taken from a song ;-) Raymond From trentm at ActiveState.com Sun Mar 19 01:07:57 2006 From: trentm at ActiveState.com (Trent Mick) Date: Sat, 18 Mar 2006 16:07:57 -0800 Subject: [Python-Dev] pyexpat namespace problem (Was: libbzip2 version?) In-Reply-To: References: <44136F6F.1040104@gmail.com> <44137004.2080105@v.loewis.de> Message-ID: <20060319000757.GA10959@activestate.com> [Georg Brandl wrote] > There seems to be a new version of expat too: http://python.org/sf/1433435 > > The namespace problem with pyexpat persists: http://python.org/sf/1295808 http://python.org/sf/1295808 [My lastest comment to that bug] > As Neal points out this is a duplicate of > . The patch on *that* bug > fixes the specific issue, but does not solve the general > problem (as Michael, 'mwh', pointed out in the comments > there). > > I think we should > 1. apply this patch > 2. then perhaps update our expat version (see > http://python.org/sf/1433435, > http://mail.python.org/pipermail/python-dev/2006-March/062287.html), > 3. then update Modules/expat/pyexpatns.h (that this patch > adds) for any new symbols in the new version of Expat. I'm testing my patch to the trunk right now to make sure it passes everything. If you all think this is a good idea I could apply it and then also give a go an upgrading out expat version (I haven't looked at whether that would be very difficult). Cheers, Trent -- Trent Mick TrentM at ActiveState.com From trentm at ActiveState.com Sun Mar 19 01:09:35 2006 From: trentm at ActiveState.com (Trent Mick) Date: Sat, 18 Mar 2006 16:09:35 -0800 Subject: [Python-Dev] pyexpat namespace problem (Was: libbzip2 version?) In-Reply-To: <20060319000757.GA10959@activestate.com> References: <44136F6F.1040104@gmail.com> <44137004.2080105@v.loewis.de> <20060319000757.GA10959@activestate.com> Message-ID: <20060319000935.GB10959@activestate.com> [Trent Mick wrote] > I'm testing my patch to the trunk right now to make sure it passes > everything. If you all think this is a good idea I could apply it and > then also give a go an upgrading out expat version (I haven't looked at > whether that would be very difficult). "give a go at upgrading our (i.e. Python's) expat version" Trent -- Trent Mick TrentM at ActiveState.com From martin at v.loewis.de Sun Mar 19 01:15:29 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 19 Mar 2006 01:15:29 +0100 Subject: [Python-Dev] pyexpat namespace problem (Was: libbzip2 version?) In-Reply-To: <20060319000757.GA10959@activestate.com> References: <44136F6F.1040104@gmail.com> <44137004.2080105@v.loewis.de> <20060319000757.GA10959@activestate.com> Message-ID: <441CA2A1.8060802@v.loewis.de> Trent Mick wrote: > I'm testing my patch to the trunk right now to make sure it passes > everything. If you all think this is a good idea I could apply it and > then also give a go an upgrading out expat version (I haven't looked at > whether that would be very difficult). I would like to see an update to the current Expat release before that. Regards, Martin From python at rcn.com Sun Mar 19 01:22:55 2006 From: python at rcn.com (Raymond Hettinger) Date: Sat, 18 Mar 2006 19:22:55 -0500 Subject: [Python-Dev] PySet API References: <1142705253.32074.72.camel@geddy.wooz.org> <005c01c64ac5$589859b0$b83efea9@RaymondLaptop1> Message-ID: <002b01c64aeb$44eb7750$b83efea9@RaymondLaptop1> > [Barry Warsaw] >> Oh, also, we have a couple of additions to the PySet C API. >> I'll work on putting together an SF patch for them over the weekend. What are you proposing to add to the PySet API? I designed an API that was both minimal and complete. The idea was to provide direct access to fined grained functions and access the rest through the existing abstract API for PyObject and PyNumber as detailed in the Set API docs. I tried out the API to translate a number of set algorithms and found that the API was easy-to-use and sufficient as-is. There may be room for variants of the type checking macros, but I would like the rest of the C API to remain as-is unless some compelling deficiency can be shown. It is easy to expand the API later but almost impossible to take anything back once in the field. IOW, if I have I still have a say in the matter, the patch will most likely not be accepted. Raymond From trentm at ActiveState.com Sun Mar 19 02:10:48 2006 From: trentm at ActiveState.com (Trent Mick) Date: Sat, 18 Mar 2006 17:10:48 -0800 Subject: [Python-Dev] pyexpat namespace problem (Was: libbzip2 version?) In-Reply-To: <441CA2A1.8060802@v.loewis.de> References: <44136F6F.1040104@gmail.com> <44137004.2080105@v.loewis.de> <20060319000757.GA10959@activestate.com> <441CA2A1.8060802@v.loewis.de> Message-ID: <20060319011048.GA24906@activestate.com> [Martin v. Loewis wrote] > Trent Mick wrote: > > I'm testing my patch to the trunk right now to make sure it passes > > everything. If you all think this is a good idea I could apply it and > > then also give a go an upgrading out expat version (I haven't looked at > > whether that would be very difficult). > > I would like to see an update to the current Expat release before that. Okay. I'll see if I can try that tomorrow. Trent -- Trent Mick TrentM at ActiveState.com From greg.ewing at canterbury.ac.nz Sun Mar 19 02:48:54 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 19 Mar 2006 13:48:54 +1200 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <1142707813.10599.15.camel@resist.wooz.org> References: <441C02C8.5080504@iinet.net.au> <1142695605.32074.49.camel@geddy.wooz.org> <038501c64aba$545d2200$bf03030a@trilan> <1142707813.10599.15.camel@resist.wooz.org> Message-ID: <441CB886.4000401@canterbury.ac.nz> Barry Warsaw wrote: > On Sat, 2006-03-18 at 19:32 +0100, Giovanni Bajo wrote: >>Unless this new proposal also includes changing the meaning of "except:" to >>"except Error". Then maybe it should be called "error:" rather than "except:". :-) Greg From greg.ewing at canterbury.ac.nz Sun Mar 19 02:49:21 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 19 Mar 2006 13:49:21 +1200 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <1142697104.32089.58.camel@geddy.wooz.org> References: <441C02C8.5080504@iinet.net.au> <1142695605.32074.49.camel@geddy.wooz.org> <441C2C3A.8070003@strakt.com> <1142697104.32089.58.camel@geddy.wooz.org> Message-ID: <441CB8A1.8040205@canterbury.ac.nz> Barry Warsaw wrote: > One possible approach is to revert BaseException out of Py2.5, > re-position KeyboardInterrupt, and add Error as an alias for > StandardError. Then we can encourage people to start using Error as the > base classes for their own errors. Also maybe start issuing warnings whenever you inherit directly from Exception. Greg From greg.ewing at canterbury.ac.nz Sun Mar 19 02:49:32 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 19 Mar 2006 13:49:32 +1200 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <1142695605.32074.49.camel@geddy.wooz.org> References: <441C02C8.5080504@iinet.net.au> <1142695605.32074.49.camel@geddy.wooz.org> Message-ID: <441CB8AC.4010300@canterbury.ac.nz> Barry Warsaw wrote: > Exception > +- KeyboardInterrupt > +- GeneratorExit > +- SystemExit > +- StopIteration > +- Error > | +- ImportError > | +- (etc.) > | > +- Warning > +- UserWarning > +- (etc.) +42! This is beautifully clear and simple, especially compared to some of the other exception hierarchy reorganisations that have been proposed. Greg From greg.ewing at canterbury.ac.nz Sun Mar 19 02:49:40 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 19 Mar 2006 13:49:40 +1200 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: <441B82B9.4030607@gmail.com> References: <4418CDD0.8020803@canterbury.ac.nz> <17434.10109.675484.243338@montanaro.dyndns.org> <2m3bhh30fu.fsf@starship.python.net> <441A7C8A.9030404@voidspace.org.uk> <17434.61837.491103.995641@montanaro.dyndns.org> <20060317202345.GA19451@phd.pp.ru> <441B82B9.4030607@gmail.com> Message-ID: <441CB8B4.9030201@canterbury.ac.nz> Nick Coghlan wrote: > So, as a somewhat novel approach, what about putting the "as" *before* the > list of exceptions types? -1. When you're scanning down a series of except clauses, what you're looking for foremost is the types of exceptions being caught. The bound name is of secondary importance and only relevant to the code in the except clause. Greg From greg.ewing at canterbury.ac.nz Sun Mar 19 02:49:57 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 19 Mar 2006 13:49:57 +1200 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: <4c45c1530603181131s58b3d598i@mail.gmail.com> References: <4418CDD0.8020803@canterbury.ac.nz> <17434.10109.675484.243338@montanaro.dyndns.org> <2m3bhh30fu.fsf@starship.python.net> <441A7C8A.9030404@voidspace.org.uk> <17434.61837.491103.995641@montanaro.dyndns.org> <441B89EE.8000303@canterbury.ac.nz> <4c45c1530603181131s58b3d598i@mail.gmail.com> Message-ID: <441CB8C5.3000403@canterbury.ac.nz> Wolfgang Langner wrote: > try: > something > except NameError or OtherError as e: I don't see that this really helps anything, since it's no clearer how "or" and "as" should bind than it is how "," and "as" should bind. Also it has the disadvantage that except E1 or E2 as e: would *not* be equivalent to except (E1 or E2) as e: On the other hand, in except E1, E2 as e: the E1, E2 is just a tuple expression, so it's exactly equivalent to except (E1, E2) as e: Greg From greg.ewing at canterbury.ac.nz Sun Mar 19 02:50:13 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 19 Mar 2006 13:50:13 +1200 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <0a4e01c64ac0$174b39a0$524d2697@bagio> References: <441C02C8.5080504@iinet.net.au> <1142695605.32074.49.camel@geddy.wooz.org> <038501c64aba$545d2200$bf03030a@trilan> <1142707813.10599.15.camel@resist.wooz.org> <0a4e01c64ac0$174b39a0$524d2697@bagio> Message-ID: <441CB8D5.5090601@canterbury.ac.nz> Giovanni Bajo wrote: > The situation (in Py3k) I was thinking is when people see this code: > > except: > # something > > and want to change it so to get a name to the exception object. I *think* many > could get confused and write: > > except Exception, e: > # something If except clauses are changed to use "as", then as long as we're still allowing bare excepts, that could become except as e: ... Greg From greg.ewing at canterbury.ac.nz Sun Mar 19 02:50:19 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 19 Mar 2006 13:50:19 +1200 Subject: [Python-Dev] Expose the array interface in Python 2.5? In-Reply-To: <441B87E9.3050703@gmail.com> References: <441A8848.2090004@gmail.com> <441B87E9.3050703@gmail.com> Message-ID: <441CB8DB.8060901@canterbury.ac.nz> Things are getting a bit confusing, because it seems that there are two different things being discussed here: (1) A C-level array interface that's a generalisation of the buffer interface. (2) A new multi-dimensional array object type which happens to implement (1). As I understand the proposals, the important thing for interoperability between different types is to have (1). Whereas (2) would be useful to have, it's not a prerequisite for (1). Do I understand correctly? Greg From tjreedy at udel.edu Sun Mar 19 03:33:26 2006 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 18 Mar 2006 21:33:26 -0500 Subject: [Python-Dev] GeneratorExit inheriting from Exception References: <441C02C8.5080504@iinet.net.au> <1142695605.32074.49.camel@geddy.wooz.org> Message-ID: Exception +- KeyboardInterrupt +- GeneratorExit +- SystemExit +- StopIteration This would look even better to me and be easier to learn and remember if the above specifics were gathered under one general category parallel to Error and Warning. Not sure what. Not NonErrorNonWarning though. SystemException is too long and too specific. Maybe Control? No, I don't have a specific use other than didactic, but that is worth something, and I can imagine that someone else might. +- Error | +- ImportError | +- (etc.) | +- Warning +- UserWarning +- (etc.) Otherwise, looks good to me. Terry Jan Reedy From ncoghlan at gmail.com Sun Mar 19 03:35:04 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 19 Mar 2006 12:35:04 +1000 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <1142695605.32074.49.camel@geddy.wooz.org> References: <441C02C8.5080504@iinet.net.au> <1142695605.32074.49.camel@geddy.wooz.org> Message-ID: <441CC358.2020305@gmail.com> Barry Warsaw wrote: > Exception > +- KeyboardInterrupt > +- GeneratorExit > +- SystemExit > +- StopIteration > +- Error > | +- ImportError > | +- (etc.) > | > +- Warning > +- UserWarning > +- (etc.) > > Use defined errors should inherit from Error, not Exception. With this, > "except Exception" would be a synonym for bare except, while "except > Error" would be the standard idiom for letting non-error exceptions pass > through. > > I don't know whether this is possible for Python 2.5, but I think it > should be what we strive for for Py3K, and I do not think BaseException > is at all necessary. The main problems I have with the idea are the fact that a large fraction of the user-defined exceptions out there already inherit from Exception (and this has been pushed for a long time as the right thing to do) and the fact that "except Exception:" has also been pushed for years as the preferred alternative to a bare "except:". Rather than trying to change course midstream, I *like* the fact that the PEP 352 hierarchy introduces BaseException to bring the language itself into line with what people have already been taught. Breaking things in Py3k is all well and good, but breaking them gratuitously is something else entirely :) I also agree with the point made during the PEP 348/352 discussions that StopIteration reaching any except clause that isn't specifically expecting it *is* an error. Direct calls to an iterator's next method need to expect the StopIteration and decide how to handle it - not handling it is a bug. For KeyboardInterrupt/SystemExit/GeneratorExit, it is clear that the standard behaviour should be to reraise them - the entire point of those exceptions is to shut down an entire call stack. StopIteration, on the other hand, merely indicates that a particular iterator has completed, not that the entire stack of iterators should be shut down. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Sun Mar 19 03:52:22 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 19 Mar 2006 12:52:22 +1000 Subject: [Python-Dev] Expose the array interface in Python 2.5? In-Reply-To: <441CB8DB.8060901@canterbury.ac.nz> References: <441A8848.2090004@gmail.com> <441B87E9.3050703@gmail.com> <441CB8DB.8060901@canterbury.ac.nz> Message-ID: <441CC766.2060209@gmail.com> Greg Ewing wrote: > Things are getting a bit confusing, because it seems > that there are two different things being discussed > here: > > (1) A C-level array interface that's a generalisation > of the buffer interface. > > (2) A new multi-dimensional array object type which > happens to implement (1). > > As I understand the proposals, the important thing for > interoperability between different types is to have > (1). Whereas (2) would be useful to have, it's not > a prerequisite for (1). > > Do I understand correctly? That sounds right to me. Although option 2 is more a matter of "have an exemplar of the interface in the standard library that can be used by inheritance or containment". The interface is actually already defined by numpy - Option 1 would involve transferring that info to a PEP on python.org without necessarily providing any examples of the interface in the standard library. This pushes more work onto those wishing to support the interface (since they will need to either implement the interface independently or become dependent on another library such as numpy). Option 2 would involve either modifying array.array to expose the interface, or adding a new type, array.dimarray, that allowed such structures to be accessed, but not necessarily easily manipulated. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Sun Mar 19 03:58:13 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 19 Mar 2006 12:58:13 +1000 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: References: <441C02C8.5080504@iinet.net.au> <1142695605.32074.49.camel@geddy.wooz.org> Message-ID: <441CC8C5.9080508@gmail.com> Terry Reedy wrote: > Exception > +- KeyboardInterrupt > +- GeneratorExit > +- SystemExit > +- StopIteration > > This would look even better to me and be easier to learn and remember if > the above specifics were gathered under one general category parallel to > Error and Warning. Not sure what. Not NonErrorNonWarning though. > SystemException is too long and too specific. Maybe Control? > > No, I don't have a specific use other than didactic, but that is worth > something, and I can imagine that someone else might. I'd vote for ControlFlowException if StopIteration is included in the category, and TerminatingException if it isn't. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From skip at pobox.com Sun Mar 19 04:47:17 2006 From: skip at pobox.com (skip at pobox.com) Date: Sat, 18 Mar 2006 21:47:17 -0600 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: <441CB8C5.3000403@canterbury.ac.nz> References: <4418CDD0.8020803@canterbury.ac.nz> <17434.10109.675484.243338@montanaro.dyndns.org> <2m3bhh30fu.fsf@starship.python.net> <441A7C8A.9030404@voidspace.org.uk> <17434.61837.491103.995641@montanaro.dyndns.org> <441B89EE.8000303@canterbury.ac.nz> <4c45c1530603181131s58b3d598i@mail.gmail.com> <441CB8C5.3000403@canterbury.ac.nz> Message-ID: <17436.54341.20232.333509@montanaro.dyndns.org> Greg> On the other hand, in Greg> except E1, E2 as e: Greg> the E1, E2 is just a tuple expression, so it's Greg> exactly equivalent to Greg> except (E1, E2) as e: The comma and "as" have different precedence in your proposed except clause than they currently do in the import statement. I think that can lead to confusion. In particular, if someone is used to from foo import bar, baz as flurp I think they might expect that in this try/except statement: try: something() except E1, E2 as e print "something bad happened" if E1 is raised, e will not be (re)bound, but if E2 is raised, then it will be. I know Python is not as heavily an operator-oriented language as C or Perl, but I think it makes sense to maintain the same relative binding precedence between operators in different contexts wherever possible. There seem to be other places where Python is beginning to require parens even though they aren't strictly necessary to resolve syntactic ambiguity. This might be one of them. Skip From oliphant.travis at ieee.org Sun Mar 19 06:32:44 2006 From: oliphant.travis at ieee.org (Travis E. Oliphant) Date: Sat, 18 Mar 2006 22:32:44 -0700 Subject: [Python-Dev] Expose the array interface in Python 2.5? In-Reply-To: <79990c6b0603171402pd7441aam69b66b92095b7566@mail.gmail.com> References: <79990c6b0603171402pd7441aam69b66b92095b7566@mail.gmail.com> Message-ID: Paul Moore wrote: > On 3/17/06, Thomas Heller wrote: > >> Travis E. Oliphant wrote: >>> Would it be possible to add at least the C-struct array interface to the >>> Python arrayobject in time for Python 2.5? >> I'm very much for that. >> >>> Is someone on this list willing to help make it happen? >> Unfortunately not me - I'm too busy with ctypes, and if the array interface >> makes it into the core I will have to implement/use that in ctypes too. The ctypes interface is also another reason I really want a basic array interface into the core. As the very clever people on this list noticed, I was actually asking two different things because I was really looking for support from people who wanted either one... What I would really like to see is a very-simple C-object defined (perhaps not even exposed to the Python user but only the extension-module writer). This C-object would have the structure of NumPy arrays, be inheritable, and define an __array_struct__ method (notice I'm not even asking at this point for a new function-pointer table like the buffer protocol has --- although I think at some point that could be useful and maybe it is even better right now to push for that). The purpose of the C-object would be so that all extension writers to Python can rely on a simple but general-purpose description of an array that Numeric has established over the past decade. This C-object already has been somewhat fleshed out and the only real work would be to add support for it on other objects. > If all that is required is a simple C definition added to the Python > code, then it seems to me that would be (a) fairly easy, but (b) > pretty useless (on the basis that a standards-style PEP like the DB > API ones would do much the same, without any code changes). I think I would like to see a real object that can be inherited from (at least in C extension modules). Again perhaps in Python 2.5, this object is not even intentionally exposed to the Python level. > > On the third hand, the PEP referenced from the page you quote (at > http://svn.scipy.org/svn/PEP/PEP_genarray.txt) seems to specify an > implementation - are you just asking for that to be added to the core? > If so, why not submit this as a formal PEP, and see what happens from > there? (For this route, one of the Numeric people *needs* to champion > the PEP, IMHO, as only they are qualified to address any points that > get raised in general discussion). > The big problem is the release schedule of Python 2.5, I've really wanted to get "something" along these lines going for Python 2.5, but I've not had the time to finish the PEP. So, perhaps I'll tone down the request to a simple C-object with even less defined and put that out. Frankly, I'm really looking for help from an avid Python-dev person who may have an interest but perhaps not much experience with arrays. -Travis From just at letterror.com Sun Mar 19 08:05:22 2006 From: just at letterror.com (Just van Rossum) Date: Sun, 19 Mar 2006 08:05:22 +0100 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <441CB8A1.8040205@canterbury.ac.nz> Message-ID: Greg Ewing wrote: > Barry Warsaw wrote: > > > One possible approach is to revert BaseException out of Py2.5, > > re-position KeyboardInterrupt, and add Error as an alias for > > StandardError. Then we can encourage people to start using Error > > as the base classes for their own errors. > > Also maybe start issuing warnings whenever you inherit > directly from Exception. Ugh. I hate it when it's made (virtually) impossible to write code that runs warnings-free on both Python X.Y and X.(Y+1). Just From ncoghlan at gmail.com Sun Mar 19 08:31:53 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 19 Mar 2006 17:31:53 +1000 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: References: Message-ID: <441D08E9.10503@gmail.com> Just van Rossum wrote: > Greg Ewing wrote: > >> Barry Warsaw wrote: >> >>> One possible approach is to revert BaseException out of Py2.5, >>> re-position KeyboardInterrupt, and add Error as an alias for >>> StandardError. Then we can encourage people to start using Error >>> as the base classes for their own errors. >> Also maybe start issuing warnings whenever you inherit >> directly from Exception. > > Ugh. I hate it when it's made (virtually) impossible to write code that > runs warnings-free on both Python X.Y and X.(Y+1). This was one of the key things that led to the approach in PEP 352. Barry's hierarchy is very similar to some of the suggestions made during the PEP 348 discussion, and they were rejected because they made the transition a hell of a lot harder without any concomitant benefit. I know this because I was one of the people making those suggestions. With PEP 352 (tweaked to move GeneratorExit out from under Exception): - "except:" continues to mean catch everything - "except Exception:" now does the right thing - inheriting from Exception continues to be correct for user exceptions - errors may choose to inherit from one of the existing Errors instead With Barry's proposed hierarchy: - "except:" continues to mean catch everything - "except Exception:" continues to do the wrong thing - all code has to change to do "except Error:" instead - inheriting from Exception becomes incorrect for user exceptions - all code has to change to inherit from Error instead - non-error user exceptions (such as completion of a search using nested loops) have no clear parent to inherit from (both Error and Exception are wrong, albeit for different reasons. The additional pain required in order to have 'Exception' at the root of the hierarchy just isn't worth it. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From rasky at develer.com Sun Mar 19 12:43:33 2006 From: rasky at develer.com (Giovanni Bajo) Date: Sun, 19 Mar 2006 12:43:33 +0100 Subject: [Python-Dev] GeneratorExit inheriting from Exception References: <441C02C8.5080504@iinet.net.au><1142695605.32074.49.camel@geddy.wooz.org> <441CC358.2020305@gmail.com> Message-ID: <002e01c64b4a$5a3f2a70$88472597@bagio> Nick Coghlan wrote: > Rather than trying to change course midstream, I *like* the fact that > the PEP 352 hierarchy introduces BaseException to bring the language > itself into line with what people have already been taught. Breaking > things in Py3k is all well and good, but breaking them gratuitously > is something else entirely :) I really like this too, but Barry's proposal doesn't really *break* anything. Existing Python code that creates a FooBar(Exception) and then catches it with either "except FooBar:" or "except Exception, e:" + check for FooBar, will still work as expected. At *worse*, it would be catching too much, like SystemExit or GeneratorExit, which are still pretty uncommon exception. OTOH, I also understand that people have been told that deriving from Exception is the right thing to do forever now. Giovanni Bajo From python-dev at python.org Sun Mar 19 02:01:16 2006 From: python-dev at python.org (python-dev at python.org) Date: Sun, 19 Mar 2006 01:01:16 +0000 Subject: [Python-Dev] buildbot failure in sparc solaris10 gcc trunk Message-ID: <20060319010117.44A191E400E@bag.python.org> The Buildbot has detected a new failure of sparc solaris10 gcc trunk. Full details are available at: http://www.python.org:9010/sparc%20solaris10%20gcc%20trunk/builds/68 Buildbot URL: http://www.python.org:9010/ Build Reason: The web-page 'rebuild' button was pressed by '': Force compile error Build Source Stamp: [branch trunk] HEAD Blamelist: BUILD FAILED: failed compile sincerely, -The Buildbot From python-dev at python.org Sun Mar 19 09:46:16 2006 From: python-dev at python.org (python-dev at python.org) Date: Sun, 19 Mar 2006 09:46:16 +0100 (CET) Subject: [Python-Dev] Test Message-ID: <20060319084616.CC5F21E400F@bag.python.org> Testing submission from dinsdale. From tzot at mediconsa.com Sun Mar 19 17:25:58 2006 From: tzot at mediconsa.com (Christos Georgiou) Date: Sun, 19 Mar 2006 18:25:58 +0200 Subject: [Python-Dev] Patch or feature? Tix.Grid working for 2.5 References: <441C87E1.3020709@v.loewis.de> Message-ID: "Neal Norwitz" wrote in message news:ee2a432c0603181427x31d31c6foe39ab01d3a6cf2d9 at mail.gmail.com... On 3/18/06, "Martin v. L?wis" wrote: > Christos Georgiou wrote: > > I would like to know if supplying a patch for it sometime in the next > > couple > > of weeks would be considered a patch (since the widget currently is not > > working at all, its class in Tix.py contains just a pass statement) or a > > feature (ie extra functionality) for the 2.5 branch... [Martin] > I wouldn't object to including it before beta 1. Good to have a general deadline (ObRef Douglas Adams: ""I like deadlines; I especially like the whooshing sound they make as they go flying by.") [Neal] > Nor would I. I'm not sure if this helps or not, but a general rule of > thumb is if it's 100 lines it's probably a patch. If it's 1000 lines, > it's probably a major feature. Given such a patch is pure python and > probably much closer to 100 lines, this should be ok. It should amount to 100-200 lines as far as I can see, I'm just playing with .tk.call and writing down how to do stuff, and trying to grok the general style of classes interfacing to Tkinter/Tix. DisplayStyle is already there, which is good. Thankfully the tix docs seem to be complete, so it should be only a matter of time. I will try to update tixwidgets.py with a demo page too. [Neal] > BTW, have you contacted the Tix people? It is a separate project and > looks like people are working on it. http://tix.sourceforge.net/ That's where I got the documentation. I haven't contacted them, though, because I saw that Tix.py hasn't been modified for 4 years now. I will offer the patch both to the Python and the Tix Tix.py though, thanks for the suggestion. From tjreedy at udel.edu Sun Mar 19 18:05:21 2006 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 19 Mar 2006 12:05:21 -0500 Subject: [Python-Dev] buildbot failure in sparc solaris10 gcc trunk References: <20060319010117.44A191E400E@bag.python.org> Message-ID: wrote in message news:20060319010117.44A191E400E at bag.python.org... > The Buildbot has detected a new failure of sparc solaris10 gcc trunk. > Full details are available at: > http://www.python.org:9010/sparc%20solaris10%20gcc%20trunk/builds/68 > > Buildbot URL: http://www.python.org:9010/ Both links failed with Cannot Find Server (Winxp/IE). Removed :9010 and latter worked fine. By now, Sparc is green and Winxp has compile failure tjr From martin at v.loewis.de Sun Mar 19 19:04:51 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 19 Mar 2006 19:04:51 +0100 Subject: [Python-Dev] buildbot failure in sparc solaris10 gcc trunk In-Reply-To: References: <20060319010117.44A191E400E@bag.python.org> Message-ID: <441D9D43.6090004@v.loewis.de> Terry Reedy wrote: >> Buildbot URL: http://www.python.org:9010/ > > Both links failed with Cannot Find Server (Winxp/IE). Right. Buildbot doesn't know what its URL is; it is http://www.python.org/dev/buildbot/ Regards, Martin From ronaldoussoren at mac.com Sun Mar 19 20:39:23 2006 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Sun, 19 Mar 2006 20:39:23 +0100 Subject: [Python-Dev] [Python-checkins] r43041 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: <441BD2B6.3030407@egenix.com> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> <441B221A.5090900@v.loewis.de> <441B269D.20504@egenix.com> <441BD2B6.3030407@egenix.com> Message-ID: <9250CF38-C683-4E61-B220-2317DF53B3EE@mac.com> On 18-mrt-2006, at 10:28, M.-A. Lemburg wrote: > Ronald Oussoren wrote: >> On 17-mrt-2006, at 22:14, M.-A. Lemburg wrote: >> >>> Martin v. L?wis wrote: >>>> Thomas Heller wrote: >>>>> I'm not sure if this is what Marc-Andre means, but maybe these >>>>> definitions >>>>> could go into a new include file: >>>> How would that include file be used? You would have to copy it >>>> into your >>>> own source base, and include it, right? >>> We could put it into a b/w compatibility header file, e.g. >>> >>> #include "pycompat.h" >> >> But wouldn't this header be needed on versions of python before 2.5? > > Yes. Ideally it should work on more Python versions than just > Python 2.5. > > I have such a compatibility header file for the mx Extensions > (called mxpyapi.h and included in egenix-mx-base). > > It includes #defines such as the one Thomas proposed for > Python back to version 1.5. > >> That >> would make inclusion of a pycompat.h header with python 2.5 less >> useful. > > Why is that ? For older versions you can copy it into your > extension's include directory. With the usual #ifdef PYCOMPAT_H > wrapper this won't get included if Python already includes > the header file via #include "Python.h". The actual contents of the file would be useful, but having a header file should be copied into your own projects seems odd to me. I'd expect information like this to be in the documentation (either the size_t PEP or the extending & embedding document). Ronald From jjl at pobox.com Sun Mar 19 21:14:56 2006 From: jjl at pobox.com (John J Lee) Date: Sun, 19 Mar 2006 20:14:56 +0000 (UTC) Subject: [Python-Dev] Py3K timescale and stdlib philosophy (was: Re: Py3K thought: use...) In-Reply-To: References: <20060317232649.GA7358@rogue.amk.ca> Message-ID: On Fri, 17 Mar 2006, Brett Cannon wrote: > On 3/17/06, A.M. Kuchling wrote: >> Thought: We should drop all of httplib, urllib, urllib2, and ftplib, >> and instead adopt some third-party library for HTTP/FTP/whatever, >> write a Python wrapper, and use it instead. (The only such library I [...] > But maybe this also poses a larger question of where for Py3K we want > to take the stdlib. Ignoring its needed cleanup and nesting of the > namespace, do we want to try to use more external tools by importing > them and writing a Pythonic wrapper? Or do we want to not do that and > try to keep more things under our control and go with the status quo? > Or do we want to really prune down the stdlib and use more dynamic > downloading ala Cheeseshop and setuptools? [...] Do we have any idea yet what sort of timescale we're talking about for Python 3.0 (or should I call it Py3K still)? I have a personal interest in these particular modules, but the questions that seem to need answering first are more general ones about the stdlib post-3.0. Brett asks some good questions. ISTM that another important question must be: What do each of the small set of people like yourself (Brett), Andrew, Martin, Georg, Raymond (etc.!) who bear most of the burden of maintaining the stdlib at present, intend to do after Python 3.0 is out? I assume that it would only be useful to drop parts of the stdlib in this way if that group of people were then to stop working on them. That makes sense, but I don't want to make assumptions about what each of the group of people referred to above intend to do post-3.0: a. Drop 2.x right away to concentrate on developing and maintaining the 3.0 stdlib (and/or the 3.0 interpreter)? b. Spend at least some effort maintaining 2.x for a few years? c. Carry on maintaining 2.x for a few years? d. Ignore 3.x and continue with 2.x indefinitely? e. Watch and see how the Python community at large responds to 3.0? f. Wait and see what you feel like doing at the time? g. Some combination of the above? h. Quit Python to take up pig farming? These sorts of questions are often quite hard to answer, I understand, because many people often want to see what everybody else will do before making up their minds. But I guess people who post here frequently are less likely to do that than are the rest of us sheep ;-) [BTW, I assume much of the stdlib will remain essentially the same (if not without backwards-incompatibilities), one hopes people will step in to backport 3.0 fixes (and perhaps forward-port: I make no judgement about which of 2.x and 3.x will have the larger user community in the short or long term). People will presumably be more motivated to do that than currently, since I assume many people will not port all (or any) of their code to 3.0.] John From arigo at tunes.org Sun Mar 19 21:44:01 2006 From: arigo at tunes.org (Armin Rigo) Date: Sun, 19 Mar 2006 21:44:01 +0100 Subject: [Python-Dev] __dict__ strangeness In-Reply-To: References: <482lr6Fhs1jvU1@individual.net> <1hce2nh.hnzrzu1qt4yv2N%aleaxit@yahoo.com> Message-ID: <20060319204401.GA9241@code0.codespeak.net> Hi, On Sat, Mar 18, 2006 at 06:57:12PM +0100, Georg Brandl wrote: > >> >>> class D(object): > >> ... __dict__ = {} > >> ... > >> >>> d = D() > >> >>> d.a = 1 > >> >>> d.__dict__ > >> {} > >> >>> d.__dict__ = {} > >> >>> d.a > >> 1 > > > > Yep, that's the bug, fully reproducible in 2.3 and 2.4. IMHO this is not a bug, it's expected behavior. I don't see what you could "fix" without hacking the Python 2.2 descriptor model -- which is possible, but probably not what we *should* do. See zseil's explanations on the SF tracker. A bientot, Armin From brett at python.org Sun Mar 19 22:29:46 2006 From: brett at python.org (Brett Cannon) Date: Sun, 19 Mar 2006 13:29:46 -0800 Subject: [Python-Dev] dealing with decorators hiding metadata of decorated functions In-Reply-To: <441B9D39.9090000@gmail.com> References: <20060317161349.F76D.JCARLSON@uci.edu> <441B9D39.9090000@gmail.com> Message-ID: On 3/17/06, Nick Coghlan wrote: > Josiah Carlson wrote: > > "Brett Cannon" wrote: > >> With the discussion of a possible @decorator to help set the metadata > >> of the decorator to that of what the wrapped function has, I had an > >> idea that I wanted to toss out there (this dicussion stems from a blog > >> post I made: http://sayspy.blogspot.com/2006/03/how-to-handle-object-identity-issues.html). > > > > [snip] > > > > Edward Loper suggested this way back on September 5, 2004. > > > > http://mail.python.org/pipermail/python-dev/2004-September/048626.html > > > > I was and continue to be +1 on this, > > +1 here, too. Unlike Brett, though, I have no problem with overwriting > __name__ and updating __dict__ unconditionally, and overwriting __doc__ if it > hasn't already been set. > I have no problem with updating __dict__ since decorators that just do annotations will need to have that information passed forward. But if Josiah's suggestion is taken and decorators are just part of a lookup chain of attributes and they just fall through then this would not be needed. > The first two are needed if we expect "print f" and "f.a" to work properly. I don't know if having ``print f`` work like that is important. And if it is print can be tweaked to follow __decorates__ properly. > The function's name is set on the 'def' line, and I'd be much happier seeing > that at all levels of the decorator chain, rather than seeing something like > "wrapper", "wrapper", ... "wrapper", "f". Annotating decorators will modify > the functions' attributes, and this needs to be visible in the final > function's dictionary. > > If a wrapper doesn't set a docstring explicitly, it makes a lot more sense to > me to re-uses the original function's docstring rather than leave it at None. > But this is still a loss of information since you will no longer know that the decorator had no docstring. > My real interest is that it should be possible to get at all the details of > the original function (such as its code object), and the obvious way to do > that is with a standard attribute that links to the original. > Exactly. __decorates__ would provide this. Otherwise the last key piece of information I think missing that decorators cannot copy from the decorated function is the function parameters, and that is where my __signature__ object proposal (I think Philip proposed the main idea originally) steps in if the __decorates__ idea doesn't go anywhere. > > though I would go farther and state, > > like I did at the time, that one shouldn't copy any of the function > > attributes, they should come 'free', similar to the way that class > > attributes are 'free' on subclasses. > > Well, that's the idea behind the decorator decorator - simply put @decorator > on your decorator function and it will automatically do the right thing. > > > http://mail.python.org/pipermail/python-dev/2004-September/048631.html > > > > What would make this _really_ nice is if one didn't need to do anything > > manually; that the attribute that pointed to the decorated > > function/object would be automatically applied - though I realize that > > this may not be generally possible. > > A slight problem is that not all decorators will wrap the function they > decorate - some will only annotate it. > > However, here's an idea for the @decorator decorator that would make it pretty > much automatic, leaves the docstring alone if the decorator has already set it > on the wrapper, and builds up a record of the decorators that are wrapping the > the original function: > > def _link_decorated(decorated, orig, decorator): > """Link a decorated function with the original""" > decorated.__name__ = orig.__name__ > decorated.__dict__.update(orig.__dict__) > if decorated.__doc__ is None: > decorated.__doc__ = orig.__doc__ > decorated.__decorates__ = orig > decorated.__decorator__ = decorator > > def decorator(orig_decorator): > """Decorator to create a well-behaved decorator""" > # Wrapper function that links a decorated function > # to the original function if necessary > def wrapper(f): > decorated = orig_decorator(f) > if decorated is not f: > # Link wrapper function to the original > _link_decorated(decorated, f, wrapper) > return decorated > _link_decorated(wrapper, orig_decorator, decorator) > return wrapper > > Decorators that only do annotations aren't recorded because there isn't > anywhere to record them. Wrapping decorators, on the other hand, allow the > references to both the decorated function and the applied decorator to be > stored on the new function object. > If you do the copying of data and provide a __signature__ object you have 90% of the metadata one would want for introspection on a decorated function, so I don't know if __decorates__ will be that important in that situation (but one extra pointer to the function object is not that expensive, so it can still be provided very cheaply). I guess we need to decide if we want to promote the copying of metadata from the decorated function into the decorator or not. If we do support copying the metadata, then should we provide a __signature__ object as well to help with that or not. __decorates__ is an innocuous suggestion that it should probably be promoted regardless of what we end up suggesting for use. The real difference will be whether 'inspect' and friends get tweaked to follow __decorates__ or just to use the metadata on the decorator. -Brett > Cheers, > Nick. > > P.S. Example usage: > > Py> @decorator > ... def annotated(f): > ... f.note = 1 > ... return f > ... > Py> @decorator > ... def wrapped(f): > ... def wrapper(*args, **kwds): > ... return f(*args, **kwds) > ... return wrapper > ... > > Py> @wrapped > ... @annotated > ... @wrapped > ... @wrapped > ... def show(*args, **kwds): > ... print args, kwds > ... > > Py> while hasattr(obj, "__decorates__"): > ... print obj > ... print " Decorates:\t%s" % obj.__decorates__ > ... print " Using:\t%s" % obj.__decorator__ > ... print " Annotated?:\t%s" % hasattr(obj, "note") > ... print > ... obj = obj.__decorates__ > ... > > Decorates: > Using: > Annotated?: True > > > Decorates: > Using: > Annotated?: True > > > Decorates: > Using: > Annotated?: False > > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > --------------------------------------------------------------- > http://www.boredomandlaziness.org > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org > From aahz at pythoncraft.com Sun Mar 19 22:37:27 2006 From: aahz at pythoncraft.com (Aahz) Date: Sun, 19 Mar 2006 13:37:27 -0800 Subject: [Python-Dev] dealing with decorators hiding metadata of decorated functions In-Reply-To: References: <20060317161349.F76D.JCARLSON@uci.edu> <441B9D39.9090000@gmail.com> Message-ID: <20060319213726.GA24510@panix.com> On Sun, Mar 19, 2006, Brett Cannon wrote: > > I guess we need to decide if we want to promote the copying of > metadata from the decorated function into the decorator or not. Short answer: absolutely yes I had to deal with a related issue recently, where embedded doctests no longer worked on the decorated function. There were two separate problems: the first was non-copying of __doc__, which was easy enough to fix; the second (which I still haven't fixed) is the fact that __name__ wasn't set, so doctest didn't auto-find the function. (The decorator came from a different module.) (This was Python 2.2/2.3, so decorators weren't actually involved, but I think it's the same problem.) I don't understand decorators well enough to propose a good solution given the other constraints, but it seems clear to me that we need a good solution of some kind. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From brett at python.org Sun Mar 19 23:02:19 2006 From: brett at python.org (Brett Cannon) Date: Sun, 19 Mar 2006 14:02:19 -0800 Subject: [Python-Dev] Py3K timescale and stdlib philosophy (was: Re: Py3K thought: use...) In-Reply-To: References: <20060317232649.GA7358@rogue.amk.ca> Message-ID: On 3/19/06, John J Lee wrote: > On Fri, 17 Mar 2006, Brett Cannon wrote: > > > On 3/17/06, A.M. Kuchling wrote: > >> Thought: We should drop all of httplib, urllib, urllib2, and ftplib, > >> and instead adopt some third-party library for HTTP/FTP/whatever, > >> write a Python wrapper, and use it instead. (The only such library I > [...] > > But maybe this also poses a larger question of where for Py3K we want > > to take the stdlib. Ignoring its needed cleanup and nesting of the > > namespace, do we want to try to use more external tools by importing > > them and writing a Pythonic wrapper? Or do we want to not do that and > > try to keep more things under our control and go with the status quo? > > Or do we want to really prune down the stdlib and use more dynamic > > downloading ala Cheeseshop and setuptools? > [...] > > Do we have any idea yet what sort of timescale we're talking about for > Python 3.0 (or should I call it Py3K still)? > Py3K. It's shorter and since Python 3.0 is still just a PEP and Guido's neurons it really has not materalized yet to be an upcoming version of Python yet. =) > I have a personal interest in these particular modules, but the questions > that seem to need answering first are more general ones about the stdlib > post-3.0. Brett asks some good questions. > > ISTM that another important question must be: What do each of the small > set of people like yourself (Brett), Andrew, Martin, Georg, Raymond > (etc.!) who bear most of the burden of maintaining the stdlib at present, > intend to do after Python 3.0 is out? I assume that it would only be > useful to drop parts of the stdlib in this way if that group of people > were then to stop working on them. That makes sense, but I don't want to > make assumptions about what each of the group of people referred to above > intend to do post-3.0: > > a. Drop 2.x right away to concentrate on developing and maintaining the > 3.0 stdlib (and/or the 3.0 interpreter)? > > b. Spend at least some effort maintaining 2.x for a few years? > > c. Carry on maintaining 2.x for a few years? > > d. Ignore 3.x and continue with 2.x indefinitely? > > e. Watch and see how the Python community at large responds to 3.0? > > f. Wait and see what you feel like doing at the time? > > g. Some combination of the above? > > h. Quit Python to take up pig farming? > Py3K will most likely be just another release of Python with a lot of changes. The final 2.x release will be maintained for a while just because we always maintain the last stable release while the next version is being developed. But since the 2.x series will be depended upon by people for quite a while I suspect we will continue to patch it and release it as long as Anthony is willing to do micro releases and developers plan to continue to backport fixes. Personally, I plan to help to maintain the 2.x series, but once Python 3.0 becomes a reality, it won't be my focus. One would hope that bugs in the 2.x series will get closed up over time and will require less and less maintenance. But backporting might be a problem from 3.x to 2.x because of fundamental differences of how things are structured on top of people just losing interest in 2.x since it isn't bleeding edge. > > These sorts of questions are often quite hard to answer, I understand, > because many people often want to see what everybody else will do before > making up their minds. But I guess people who post here frequently are > less likely to do that than are the rest of us sheep ;-) > > [BTW, I assume much of the stdlib will remain essentially the same (if not > without backwards-incompatibilities), one hopes people will step in to > backport 3.0 fixes (and perhaps forward-port: I make no judgement about > which of 2.x and 3.x will have the larger user community in the short or > long term). People will presumably be more motivated to do that than > currently, since I assume many people will not port all (or any) of their > code to 3.0.] > Well, I don't know if the stdlib will stay the same. It will definitely get pruned down and cleaned up (wouldn't be shocked if we have a Great Renaming like the C codebase did way back in the day). So I have no clue where the stdlib will go compared to 2.x . -Brett From martin at v.loewis.de Sun Mar 19 23:34:14 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 19 Mar 2006 23:34:14 +0100 Subject: [Python-Dev] Py3K timescale and stdlib philosophy In-Reply-To: References: <20060317232649.GA7358@rogue.amk.ca> Message-ID: <441DDC66.2050100@v.loewis.de> John J Lee wrote: > a. Drop 2.x right away to concentrate on developing and maintaining the > 3.0 stdlib (and/or the 3.0 interpreter)? I expect the same to happen as with all previous releases: the current and the previous release (say, 3.0 and 2.5) are maintained; anything older is unmaintained. So when 3.1 is released, 2.x is dead. Regards, Martin From janssen at parc.com Mon Mar 20 00:32:21 2006 From: janssen at parc.com (Bill Janssen) Date: Sun, 19 Mar 2006 15:32:21 PST Subject: [Python-Dev] Py3K timescale and stdlib philosophy (was: Re: Py3K thought: use...) In-Reply-To: Your message of "Sun, 19 Mar 2006 12:14:56 PST." Message-ID: <06Mar19.153223pst."58633"@synergy1.parc.xerox.com> I was sort of hoping that Python would approach Py3K asymptotically... :-). PEP 328, for instance, talks about Python 2.5, 2.6, 2.7. Bill From greg.ewing at canterbury.ac.nz Mon Mar 20 03:13:51 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 20 Mar 2006 14:13:51 +1200 Subject: [Python-Dev] Expose the array interface in Python 2.5? In-Reply-To: References: <79990c6b0603171402pd7441aam69b66b92095b7566@mail.gmail.com> Message-ID: <441E0FDF.7000608@canterbury.ac.nz> Travis E. Oliphant wrote: > The purpose of the C-object would be so that all extension writers to > Python can rely on a simple but general-purpose description of an array > that Numeric has established over the past decade. I'm not sure that this is the right direction to approach things from. It might be all right for writers of new extensions, but there are existing modules (PIL, ctypes, etc.) that already have their own way of storing data, and it seems to me it would be easier for the maintainers of those modules to add a new interface to the existing data than to rearrange their internal structure to use this new C-object. So I still think the standardised interface is more important and should have a higher priority than the new object. Greg From greg.ewing at canterbury.ac.nz Mon Mar 20 03:16:43 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 20 Mar 2006 14:16:43 +1200 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: References: Message-ID: <441E108B.4040808@canterbury.ac.nz> Just van Rossum wrote: > Greg Ewing wrote: > > > Also maybe start issuing warnings whenever you inherit > > directly from Exception. > > Ugh. I hate it when it's made (virtually) impossible to write code that > runs warnings-free on both Python X.Y and X.(Y+1). Yes, that could be a problem. Maybe there could be a __future__ import which says "I know about the new exception scheme, trust me when I inherit from Exception." Greg From tim.peters at gmail.com Mon Mar 20 03:35:27 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 19 Mar 2006 21:35:27 -0500 Subject: [Python-Dev] Long-time shy failure in test_socket_ssl In-Reply-To: References: <1f7befae0601241422sd783c37u5d676893de2f6055@mail.gmail.com> <1f7befae0601241553v10f7dd41i364e92b98f9de4a2@mail.gmail.com> <1f7befae0602272048x50d260b3i29819389709e0ab6@mail.gmail.com> Message-ID: <1f7befae0603191835i20ba2de7w6f68b8ce883af159@mail.gmail.com> [Tim Peters] >> ... >> test_socket_ssl >> test test_socket_ssl crashed -- >> exceptions.TypeError: 'NoneType' object is not callable [Neal Norwitz] > For closure, I believe this problem was addressed by revs 42842 and > 42844 to Lib/test/test_importhooks.py. Amazingly, the same thing popped up 3(!) times today in the 2.4 buildbot slaves (on two Windows boxes, and the "amd64 gentoo 2.4" box). I merged those revs to the 2.4 branch. > If anyone sees spurious failures with the buildbot (one time failures, > crashes, etc), please report the problems to python-dev. It would be > great to see if you can reproduce the results with the same tests that > failed. We need to determine if it is architecture specific, > test-order related, or something else. One-shot segfaults are still common enough on the Mac box, like the very recent: From guido at python.org Mon Mar 20 03:40:49 2006 From: guido at python.org (Guido van Rossum) Date: Sun, 19 Mar 2006 18:40:49 -0800 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <441E108B.4040808@canterbury.ac.nz> References: <441E108B.4040808@canterbury.ac.nz> Message-ID: Sigh. Enough already. PEP 352 was chosen to minimize incompatibilities and maximize gain with minimal changes in the tree. Also note that Warnings can sometimes be raised and should then treated as errors, so Warning would have to inherit from Error. I vote for the status quo in HEAD, except I've got to think more about the pros and cons of making GeneratorExit as special as KeyboardInterrupt. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Mon Mar 20 03:45:18 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 20 Mar 2006 14:45:18 +1200 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <002e01c64b4a$5a3f2a70$88472597@bagio> References: <441C02C8.5080504@iinet.net.au> <1142695605.32074.49.camel@geddy.wooz.org> <441CC358.2020305@gmail.com> <002e01c64b4a$5a3f2a70$88472597@bagio> Message-ID: <441E173E.8050109@canterbury.ac.nz> Giovanni Bajo wrote: > OTOH, I also understand that people have been told that deriving from Exception > is the right thing to do forever now. Have we really being telling them to derive *directly* from Exception, or just that deriving somehow from Exception will become mandatory? For the purpose of minimising bare-except problems, recommending direct derivation from Exception seems like a particularly bad idea, whether the exception hierarchy is changed or not. Greg From nnorwitz at gmail.com Mon Mar 20 04:09:35 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 19 Mar 2006 19:09:35 -0800 Subject: [Python-Dev] Long-time shy failure in test_socket_ssl In-Reply-To: <1f7befae0603191835i20ba2de7w6f68b8ce883af159@mail.gmail.com> References: <1f7befae0601241422sd783c37u5d676893de2f6055@mail.gmail.com> <1f7befae0601241553v10f7dd41i364e92b98f9de4a2@mail.gmail.com> <1f7befae0602272048x50d260b3i29819389709e0ab6@mail.gmail.com> <1f7befae0603191835i20ba2de7w6f68b8ce883af159@mail.gmail.com> Message-ID: On 3/19/06, Tim Peters wrote: > > > If anyone sees spurious failures with the buildbot (one time failures, > > crashes, etc), please report the problems to python-dev. It would be > > great to see if you can reproduce the results with the same tests that > > failed. We need to determine if it is architecture specific, > > test-order related, or something else. > > One-shot segfaults are still common enough on the Mac box, like the very recent: > Most of the recent failures seem to be related to threads. the most recent info is below. There's a lot more in case if it can help someone debug. It seems that one thread is in pthread_cond_wait and another thread is in PyObject_Call. n -- Exception: EXC_BAD_ACCESS (0x0001) Codes: KERN_INVALID_ADDRESS (0x0001) at 0xdbdbdc23 Thread 0: 0 libSystem.B.dylib 0x9002b8a8 semaphore_wait_signal_trap + 8 1 libSystem.B.dylib 0x9003001c pthread_cond_wait + 488 2 python.exe 0x0010a778 PyThread_acquire_lock + 240 (thread_pthread.h:416) 3 python.exe 0x000cdb84 PyEval_RestoreThread + 116 (ceval.c:306) 4 python.exe 0x000e0d58 file_dealloc + 160 (fileobject.c:310) 5 python.exe 0x00030b10 _Py_Dealloc + 68 (object.c:1872) 6 python.exe 0x000a02a4 tupledealloc + 348 (tupleobject.c:168) 7 python.exe 0x00030b10 _Py_Dealloc + 68 (object.c:1872) 8 python.exe 0x000dcb04 call_function + 1912 (ceval.c:3565) 9 python.exe 0x000d681c PyEval_EvalFrame + 34752 (ceval.c:2163) 10 python.exe 0x000dcfe0 fast_function + 464 (ceval.c:3645) 11 python.exe 0x000dccb0 call_function + 2340 (ceval.c:3584) 12 python.exe 0x000d681c PyEval_EvalFrame + 34752 (ceval.c:2163) 13 python.exe 0x000d9680 PyEval_EvalCodeEx + 4480 (ceval.c:2736) 14 python.exe 0x00118114 function_call + 556 (funcobject.c:548) 15 python.exe 0x00028e18 PyObject_Call + 96 (abstract.c:1795) 16 python.exe 0x000ddd24 ext_do_call + 764 (ceval.c:3840) 17 python.exe 0x000d6b10 PyEval_EvalFrame + 35508 (ceval.c:2203) 18 python.exe 0x000d9680 PyEval_EvalCodeEx + 4480 (ceval.c:2736) 19 python.exe 0x00118114 function_call + 556 (funcobject.c:548) 20 python.exe 0x00028e18 PyObject_Call + 96 (abstract.c:1795) 21 python.exe 0x000cadd4 instancemethod_call + 736 (classobject.c:2447) 22 python.exe 0x00028e18 PyObject_Call + 96 (abstract.c:1795) 23 python.exe 0x0005681c slot_tp_call + 112 (typeobject.c:4533) 24 python.exe 0x00028e18 PyObject_Call + 96 (abstract.c:1795) 25 python.exe 0x000dd90c do_call + 168 (ceval.c:3771) 26 python.exe 0x000dccd0 call_function + 2372 (ceval.c:3586) 27 python.exe 0x000d681c PyEval_EvalFrame + 34752 (ceval.c:2163) 28 python.exe 0x000d9680 PyEval_EvalCodeEx + 4480 (ceval.c:2736) 29 python.exe 0x00118114 function_call + 556 (funcobject.c:548) 30 python.exe 0x00028e18 PyObject_Call + 96 (abstract.c:1795) 31 python.exe 0x000ddd24 ext_do_call + 764 (ceval.c:3840) 32 python.exe 0x000d6b10 PyEval_EvalFrame + 35508 (ceval.c:2203) 33 python.exe 0x000d9680 PyEval_EvalCodeEx + 4480 (ceval.c:2736) 34 python.exe 0x00118114 function_call + 556 (funcobject.c:548) 35 python.exe 0x00028e18 PyObject_Call + 96 (abstract.c:1795) 36 python.exe 0x000cadd4 instancemethod_call + 736 (classobject.c:2447) 37 python.exe 0x00028e18 PyObject_Call + 96 (abstract.c:1795) 38 python.exe 0x0005681c slot_tp_call + 112 (typeobject.c:4533) 39 python.exe 0x00028e18 PyObject_Call + 96 (abstract.c:1795) 40 python.exe 0x000dd90c do_call + 168 (ceval.c:3771) 41 python.exe 0x000dccd0 call_function + 2372 (ceval.c:3586) 42 python.exe 0x000d681c PyEval_EvalFrame + 34752 (ceval.c:2163) 43 python.exe 0x000d9680 PyEval_EvalCodeEx + 4480 (ceval.c:2736) 44 python.exe 0x00118114 function_call + 556 (funcobject.c:548) 45 python.exe 0x00028e18 PyObject_Call + 96 (abstract.c:1795) 46 python.exe 0x000ddd24 ext_do_call + 764 (ceval.c:3840) 47 python.exe 0x000d6b10 PyEval_EvalFrame + 35508 (ceval.c:2203) 48 python.exe 0x000d9680 PyEval_EvalCodeEx + 4480 (ceval.c:2736) 49 python.exe 0x00118114 function_call + 556 (funcobject.c:548) 50 python.exe 0x00028e18 PyObject_Call + 96 (abstract.c:1795) 51 python.exe 0x000cadd4 instancemethod_call + 736 (classobject.c:2447) 52 python.exe 0x00028e18 PyObject_Call + 96 (abstract.c:1795) 53 python.exe 0x0005681c slot_tp_call + 112 (typeobject.c:4533) 54 python.exe 0x00028e18 PyObject_Call + 96 (abstract.c:1795) 55 python.exe 0x000dd90c do_call + 168 (ceval.c:3771) 56 python.exe 0x000dccd0 call_function + 2372 (ceval.c:3586) 57 python.exe 0x000d681c PyEval_EvalFrame + 34752 (ceval.c:2163) 58 python.exe 0x000dcfe0 fast_function + 464 (ceval.c:3645) 59 python.exe 0x000dccb0 call_function + 2340 (ceval.c:3584) 60 python.exe 0x000d681c PyEval_EvalFrame + 34752 (ceval.c:2163) 61 python.exe 0x000d9680 PyEval_EvalCodeEx + 4480 (ceval.c:2736) 62 python.exe 0x000dd150 fast_function + 832 (ceval.c:3656) 63 python.exe 0x000dccb0 call_function + 2340 (ceval.c:3584) 64 python.exe 0x000d681c PyEval_EvalFrame + 34752 (ceval.c:2163) 65 python.exe 0x000d9680 PyEval_EvalCodeEx + 4480 (ceval.c:2736) 66 python.exe 0x000dd150 fast_function + 832 (ceval.c:3656) 67 python.exe 0x000dccb0 call_function + 2340 (ceval.c:3584) 68 python.exe 0x000d681c PyEval_EvalFrame + 34752 (ceval.c:2163) 69 python.exe 0x000dcfe0 fast_function + 464 (ceval.c:3645) 70 python.exe 0x000dccb0 call_function + 2340 (ceval.c:3584) 71 python.exe 0x000d681c PyEval_EvalFrame + 34752 (ceval.c:2163) 72 python.exe 0x000d9680 PyEval_EvalCodeEx + 4480 (ceval.c:2736) 73 python.exe 0x000dd150 fast_function + 832 (ceval.c:3656) 74 python.exe 0x000dccb0 call_function + 2340 (ceval.c:3584) 75 python.exe 0x000d681c PyEval_EvalFrame + 34752 (ceval.c:2163) 76 python.exe 0x000d9680 PyEval_EvalCodeEx + 4480 (ceval.c:2736) 77 python.exe 0x000dd150 fast_function + 832 (ceval.c:3656) 78 python.exe 0x000dccb0 call_function + 2340 (ceval.c:3584) 79 python.exe 0x000d681c PyEval_EvalFrame + 34752 (ceval.c:2163) 80 python.exe 0x000d9680 PyEval_EvalCodeEx + 4480 (ceval.c:2736) 81 python.exe 0x000ce040 PyEval_EvalCode + 84 (ceval.c:484) 82 python.exe 0x00013c7c run_node + 120 (pythonrun.c:1265) 83 python.exe 0x00013be0 run_err_node + 88 (pythonrun.c:1252) 84 python.exe 0x00013b6c PyRun_FileExFlags + 180 (pythonrun.c:1243) 85 python.exe 0x00011fbc PyRun_SimpleFileExFlags + 712 (pythonrun.c:860) 86 python.exe 0x00011408 PyRun_AnyFileExFlags + 168 (pythonrun.c:664) 87 python.exe 0x00008750 Py_Main + 3820 (main.c:493) 88 python.exe 0x000028d8 main + 40 (python.c:23) 89 python.exe 0x00002108 _start + 340 (crt.c:272) 90 python.exe 0x00001fb0 start + 60 Thread 1 Crashed: 0 python.exe 0x00028de8 PyObject_Call + 48 (abstract.c:1794) 1 python.exe 0x0002901c PyObject_CallFunction + 360 (abstract.c:1837) 2 _testcapi.so 0x049903d4 _make_call + 64 (_testcapimodule.c:569) 3 libSystem.B.dylib 0x9002b200 _pthread_body + 96 From barry at python.org Mon Mar 20 04:10:36 2006 From: barry at python.org (Barry Warsaw) Date: Sun, 19 Mar 2006 22:10:36 -0500 Subject: [Python-Dev] PySet API In-Reply-To: <002b01c64aeb$44eb7750$b83efea9@RaymondLaptop1> References: <1142705253.32074.72.camel@geddy.wooz.org> <005c01c64ac5$589859b0$b83efea9@RaymondLaptop1> <002b01c64aeb$44eb7750$b83efea9@RaymondLaptop1> Message-ID: <1142824236.10599.29.camel@resist.wooz.org> On Sat, 2006-03-18 at 19:22 -0500, Raymond Hettinger wrote: > > [Barry Warsaw] > >> Oh, also, we have a couple of additions to the PySet C API. > >> I'll work on putting together an SF patch for them over the weekend. > > What are you proposing to add to the PySet API? PySet_Clear(), PySet_Next(), PySet_Update(), and PySet_AsList(). > I designed an API that was both minimal and complete. The idea was to provide > direct access to fined grained functions and access the rest through the > existing abstract API for PyObject and PyNumber as detailed in the Set API docs. We use the above functions quite a bit in our embedded app, so we want them to be as efficient as possible. They should also be obvious to C programmers (e.g. using PyNumber_InPlaceSubtract() is much less obvious than PySet_Clear()). > I tried out the API to translate a number of set algorithms and found that the > API was easy-to-use and sufficient as-is. There may be room for variants of the > type checking macros, but I would like the rest of the C API to remain as-is > unless some compelling deficiency can be shown. It is easy to expand the API > later but almost impossible to take anything back once in the field. The above mirrors what's available for dict objects, and if you are using sets for collections of objects, I believe they make the most sense. > IOW, if I have I still have a say in the matter, the patch will most likely not > be accepted. Can you explain why the additions above would not be obvious improvements? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060319/c32f0cd4/attachment.pgp From guido at python.org Mon Mar 20 04:18:59 2006 From: guido at python.org (Guido van Rossum) Date: Sun, 19 Mar 2006 19:18:59 -0800 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <441E173E.8050109@canterbury.ac.nz> References: <441C02C8.5080504@iinet.net.au> <1142695605.32074.49.camel@geddy.wooz.org> <441CC358.2020305@gmail.com> <002e01c64b4a$5a3f2a70$88472597@bagio> <441E173E.8050109@canterbury.ac.nz> Message-ID: On 3/19/06, Greg Ewing wrote: > Have we really being telling them to derive *directly* > from Exception, or just that deriving somehow from > Exception will become mandatory? It doesn't matter. Most code that tries to be a good citizen today derives its exceptions from Exception. > For the purpose of minimising bare-except problems, > recommending direct derivation from Exception seems > like a particularly bad idea, whether the exception > hierarchy is changed or not. Why? With PEP 352 (== HEAD status quo) this works just fine. I have a problem with using Error as the focal point since so many exceptions (user-defined or otherwise) aren't errors. Not to mention the Warnings which sometimes can be raised as Errors (but without changing their type). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Mar 20 04:42:24 2006 From: guido at python.org (Guido van Rossum) Date: Sun, 19 Mar 2006 19:42:24 -0800 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: <17436.54341.20232.333509@montanaro.dyndns.org> References: <4418CDD0.8020803@canterbury.ac.nz> <17434.61837.491103.995641@montanaro.dyndns.org> <441B89EE.8000303@canterbury.ac.nz> <4c45c1530603181131s58b3d598i@mail.gmail.com> <441CB8C5.3000403@canterbury.ac.nz> <17436.54341.20232.333509@montanaro.dyndns.org> Message-ID: On 3/18/06, skip at pobox.com wrote: > The comma and "as" have different precedence in your proposed except clause > than they currently do in the import statement. I think that can lead to > confusion. In particular, if someone is used to > > from foo import bar, baz as flurp > > I think they might expect that in this try/except statement: > > try: > something() > except E1, E2 as e > print "something bad happened" > > if E1 is raised, e will not be (re)bound, but if E2 is raised, then it will > be. I know Python is not as heavily an operator-oriented language as C or > Perl, but I think it makes sense to maintain the same relative binding > precedence between operators in different contexts wherever possible. This is as good a point as any to explain that 'as' is not an operator. It's a piece of syntax. It behaves different in each context where it is used -- because the context determines what it does. The parallel you suggest simply doesn't make sense: 'as' in the import context causes a simple renaming, and the name should be different for each renamed module, of course. But 'as' in an except clause is not a renaming -- it simply separates the exception "matching pattern" from the variable that is to receive the exception *value*. Having different exceptions bind different variables doesn't make sense -- how would the code know which variable to use? > There seem to be other places where Python is beginning to require parens > even though they aren't strictly necessary to resolve syntactic ambiguity. In the style guide only, I hope. The parens that are mandatory in a few places *are* necessary to resolve ambiguity. > This might be one of them. No, the original proposal (which was already in PEP 3000 anyway) is fine. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Mar 20 04:45:12 2006 From: guido at python.org (Guido van Rossum) Date: Sun, 19 Mar 2006 19:45:12 -0800 Subject: [Python-Dev] Python 2.5 Schedule In-Reply-To: <1142705174.32089.69.camel@geddy.wooz.org> References: <1142705174.32089.69.camel@geddy.wooz.org> Message-ID: On 3/18/06, Barry Warsaw wrote: > On Fri, 2006-03-17 at 23:48 -0800, Neal Norwitz wrote: > > > Just in case anybody here's been snoozing, 2.5 alpha 1 is coming up > > real quick, hopefully within a couple of weeks. If you have any > > *major* features (particularly implemented in C) that you want to see > > in 2.5, bring it up now. I want to strive for feature completeness by > > alpha 1. I know we will have some .py modules that won't make it into > > alpha 1, but they really should make it in by alpha 2 or be deferred > > to 2.6. > > I'd like to get some feedback on my PEP 352 comments, and if there's > general agreement on the hierarchy I proposed (so far so good :), then > I'd like to take a crack at implementing them. -1. See my response in the other thread. The focus on 'Error' is mistaken, and we have a large body of existing code that derives from Exception. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Mon Mar 20 04:51:11 2006 From: barry at python.org (Barry Warsaw) Date: Sun, 19 Mar 2006 22:51:11 -0500 Subject: [Python-Dev] Python 2.5 Schedule In-Reply-To: References: <1142705174.32089.69.camel@geddy.wooz.org> Message-ID: <1142826671.10599.33.camel@resist.wooz.org> On Sun, 2006-03-19 at 19:45 -0800, Guido van Rossum wrote: > -1. See my response in the other thread. The focus on 'Error' is > mistaken, and we have a large body of existing code that derives from > Exception. Just to be clear, are you saying -1 only for Python 2.5 or -1 also for Python 3.0? If the former, as I mentioned before, that would be fine with me. If the latter too, then I won't waste my time following up on the thread or writing a PEP. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060319/49d433c7/attachment.pgp From aleaxit at gmail.com Mon Mar 20 05:08:48 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Sun, 19 Mar 2006 20:08:48 -0800 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: References: <4418CDD0.8020803@canterbury.ac.nz> <17434.61837.491103.995641@montanaro.dyndns.org> <441B89EE.8000303@canterbury.ac.nz> <4c45c1530603181131s58b3d598i@mail.gmail.com> <441CB8C5.3000403@canterbury.ac.nz> <17436.54341.20232.333509@montanaro.dyndns.org> Message-ID: On Mar 19, 2006, at 7:42 PM, Guido van Rossum wrote: ... >> There seem to be other places where Python is beginning to require >> parens >> even though they aren't strictly necessary to resolve syntactic >> ambiguity. > > In the style guide only, I hope. The parens that are mandatory in a > few places *are* necessary to resolve ambiguity. Technically, I believe the first place where this did not apply was list comprehensions, back in 2.2, since (for example) [x, y for y, x in whatever] could have been syntactically disambiguated but (quite reasonably, IMHO) [(x,y) for y,x in whatever] was mandated instead. Alex From guido at python.org Mon Mar 20 05:30:13 2006 From: guido at python.org (Guido van Rossum) Date: Sun, 19 Mar 2006 20:30:13 -0800 Subject: [Python-Dev] Python 3000 Process Message-ID: I see increased activity related to Python 3000. This is great, but there is some danger involved. Some of the dangers are: overloading developers; setting unrealistic expectations for what can be accomplished; scaring the more conservative user community; paralyzing developers who can't decide whether to wait for Python 3000 or not. I'd like to address all these issues, but it may be better to first spend some time on creating a more streamlined process. Perhaps it's finally time to introduce a separate mailing list? If people agree, let's call it python-3000 at python.org. For many developers this won't make much of a difference (since they'll subscribe to both lists), but it will give people who are only concerned with Python 2.x a way to opt-out, and perhaps more importantly, it will make it clear whether any particular proposal is intended for Python 3000 or for Python 2.x. I don't want to encourage people who are only interested in Python 3000 to opt-out from the 2.5 python-dev list, since Python 3000 is not being developed in a vacuum. It must be "a better Python" and that means it is informed to a large extent by recurring issues on python-dev (and c.l.py!). The mailing list is only a small part of the new strategy. We need to start deciding on important meta-issues like: - What's the timeline? I don't expect to be setting a schedule now and sticking to it for the next five years. But we owe everybody out there who is watching some clarity about when Python 3000 can be expected, and how we plan to get there; there are widely differing estimates of how long it will take, and I don't want to scare users away or cause developers to hold their breath waiting for it (some of which I imagine is happening with Perl 6). - What's the upgrade path? Do we provide a conversion tool, or a compatibility mode, or both, or what? Will it be at all possible to write code that runs in Python 2.x (for large enough values of x) as well as in 3.0? This also touches upon the issue of parallel releases of Python 2.x and 3.x. My personal expectation (contrary to what MvL said recently) is that there will be several 2.x releases issued even after 3.0 is out; possibly 3.0 and 2.6 may coexist, and 2.7-2.9 may continue to evolve 2.x while 3.x is maturing. I've seen this used successfully in Perl (with 4->5) and Apache, and closer to home in Zope. Again, this is important in the light of how the transition is perceived in the world outside python-dev. - Will we do a grand library reform at the same time? Personally I see that as quite a separate issue; apart from some specific things like the stdio redesign, we could start the library reform in 2.6, or post-3.0, depending on how much energy there it. - What's the implementation strategy? I've started a branch where I plan to do some weeding out; but I've already found that the large amount of legacy code makes the weeding difficult. I may yet decide to switch to a sandbox model where only new code or carefully modernized old code is added (this is how Zope 3 was developed). - What's the procedure for proposing and new features? It may be time to start a new series of PEPs that focus exclusively on Python 3000. I'd like to reserve the numbers 3000-3099 for meta-PEPs (e.g. addressing the above questions) and 3100-3999 for feature PEPs. Please don't respond with answers to these questions -- each of them is worth several threads. Instead, ponder them, and respond with a +1 or -1 on the creation of the python-3000 mailing list. We'll start discussing the issues there -- or here, if the general sense is not to split the list. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Mar 20 05:40:20 2006 From: guido at python.org (Guido van Rossum) Date: Sun, 19 Mar 2006 20:40:20 -0800 Subject: [Python-Dev] Python 2.5 Schedule In-Reply-To: <1142826671.10599.33.camel@resist.wooz.org> References: <1142705174.32089.69.camel@geddy.wooz.org> <1142826671.10599.33.camel@resist.wooz.org> Message-ID: On 3/19/06, Barry Warsaw wrote: > On Sun, 2006-03-19 at 19:45 -0800, Guido van Rossum wrote: > > > -1. See my response in the other thread. The focus on 'Error' is > > mistaken, and we have a large body of existing code that derives from > > Exception. > > Just to be clear, are you saying -1 only for Python 2.5 or -1 also for > Python 3.0? If the former, as I mentioned before, that would be fine > with me. If the latter too, then I won't waste my time following up on > the thread or writing a PEP. I'm -0 for changing this in 3.0; a larger-scale reorganization could be undertaken but it's not a big priority. Before you spend more energy on this, I'd like to address the process for Python 3000, which is too chaotic right now. See my new thread titled "Python 3000 Process". -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy at alum.mit.edu Mon Mar 20 05:42:03 2006 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Sun, 19 Mar 2006 23:42:03 -0500 Subject: [Python-Dev] Bug Day? In-Reply-To: References: Message-ID: On 3/15/06, Georg Brandl wrote: > Neil Schemenauer wrote: > > I think it would be a good idea to follow the Plone project and try > > to encourage new developers by offering assistance to get them up > > and running. AFAIK, we've done that for the other bug days but it > > might help to publish the fact that no prior Python development > > experience is necessary. > > That's right, though I guess it'll be hard to find new core developers. > Unfortunately, for most people developing _with_ Python is much more fun than > developing Python. > > Settling on Friday, how about the 31st of March? Sounds good to me. Jeremy > If that's okay for the VIPs, I will send out announcements on python-list, the > non-english newsgroups and lists and ask Tim to place it on the website. > > Georg > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu > From anthony at interlink.com.au Mon Mar 20 06:49:49 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon, 20 Mar 2006 16:49:49 +1100 Subject: [Python-Dev] Python 2.5 Schedule In-Reply-To: References: Message-ID: <200603201649.51438.anthony@interlink.com.au> On Saturday 18 March 2006 18:48, Neal Norwitz wrote: > Just in case anybody here's been snoozing, 2.5 alpha 1 is coming up > real quick, hopefully within a couple of weeks. If you have any > *major* features (particularly implemented in C) that you want to > see in 2.5, bring it up now. I want to strive for feature > completeness by alpha 1. I know we will have some .py modules that > won't make it into alpha 1, but they really should make it in by > alpha 2 or be deferred to 2.6. +1. We shouldn't be making feature changes once we hit beta. I'd still like to push 2.4.3rc1 out in a couple of days time, with 2.4.3 final next week, and then maybe aim for 2.5a1 a week or two later? How does that work for everyone? Anthony -- Anthony Baxter It's never too late to have a happy childhood. From gmccaughan at synaptics-uk.com Mon Mar 20 07:00:02 2006 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Mon, 20 Mar 2006 06:00:02 +0000 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: <50372CD0-B314-4FED-8C57-3F563F459E09@gmail.com> References: <4418CDD0.8020803@canterbury.ac.nz> <50372CD0-B314-4FED-8C57-3F563F459E09@gmail.com> Message-ID: <200603200600.03110.gmccaughan@synaptics-uk.com> On Friday 2006-03-17 05:04, Alex Martelli wrote: > Hmmm, if we allowed '( as )' for generic expr's we'd make > a lot of people pining for 'assignment as an expression' very happy, > wouldn't we...? I hope not. It looks a lot more like a binding construct than an assigning one. But what about "pattern-matching"? Currently, you can say def f((x0,y0), (x1,y1)): ... but wouldn't if be nice if you could say def f((x0,y0) as p0, (x1,y1) as p1): ... and have both the "packed" and "unpacked" forms available? (Prior art: some functional languages that let you define a function with multiple "pattern-matching" clauses have a similar feature.) -- g From tismer at stackless.com Mon Mar 20 06:49:29 2006 From: tismer at stackless.com (Christian Tismer) Date: Sun, 19 Mar 2006 21:49:29 -0800 Subject: [Python-Dev] Python 3000 Process In-Reply-To: References: Message-ID: <441E4269.7090509@stackless.com> Guido van Rossum wrote: ... > Please don't respond with answers to these questions -- each of them > is worth several threads. Instead, ponder them, and respond with a +1 > or -1 on the creation of the python-3000 mailing list. We'll start > discussing the issues there -- or here, if the general sense is not to > split the list. I think this list has grown already too much activity, so I'd appreciate a split. +1 -- chris From fdrake at acm.org Mon Mar 20 07:22:55 2006 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Mon, 20 Mar 2006 01:22:55 -0500 Subject: [Python-Dev] Python 2.5 Schedule In-Reply-To: <200603201649.51438.anthony@interlink.com.au> References: <200603201649.51438.anthony@interlink.com.au> Message-ID: <200603200122.55698.fdrake@acm.org> On Monday 20 March 2006 00:49, Anthony Baxter wrote: > I'd still like to push 2.4.3rc1 out in a couple of days time, with > 2.4.3 final next week, and then maybe aim for 2.5a1 a week or two > later? How does that work for everyone? I should be fine to build the documentation Wednesday night (US Eastern time). -Fred -- Fred L. Drake, Jr. From brett at python.org Mon Mar 20 09:16:00 2006 From: brett at python.org (Brett Cannon) Date: Mon, 20 Mar 2006 00:16:00 -0800 Subject: [Python-Dev] Python 3000 Process In-Reply-To: <441E4269.7090509@stackless.com> References: <441E4269.7090509@stackless.com> Message-ID: On 3/19/06, Christian Tismer wrote: > Guido van Rossum wrote: > ... > > > Please don't respond with answers to these questions -- each of them > > is worth several threads. Instead, ponder them, and respond with a +1 > > or -1 on the creation of the python-3000 mailing list. We'll start > > discussing the issues there -- or here, if the general sense is not to > > split the list. > > I think this list has grown already too much activity, so I'd > appreciate a split. > > +1 -- chris +1 (maybe even with top-posting being acceptable on the list! =) . -Brett From raymond.hettinger at verizon.net Mon Mar 20 09:44:02 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Mon, 20 Mar 2006 03:44:02 -0500 Subject: [Python-Dev] PySet API References: <1142705253.32074.72.camel@geddy.wooz.org> <005c01c64ac5$589859b0$b83efea9@RaymondLaptop1> <002b01c64aeb$44eb7750$b83efea9@RaymondLaptop1> <1142824236.10599.29.camel@resist.wooz.org> Message-ID: <00a001c64bfa$71a0ab20$b83efea9@RaymondLaptop1> [Raymond] >> What are you proposing to add to the PySet API? [Barry] > PySet_Clear(), PySet_Next(), PySet_Update(), and PySet_AsList(). PySet_Clear() ------------- Use PyObject_CallMethod(s, "clear", NULL). Or if you need to save a millisecond on an O(n) operation, use PyNumber_InPlaceSubtract(s,s) as shown in the docs. If the name bugs you, it only takes a one-line macro to define a wrapper. The set API should not be cluttered with unnecessary and redundant functions. PySet_Next() ------------ This is also redundant. The preferred way to iterate over a set should be PyObject_GetIter(s). The iter api is generic and works for all containers. It ought to be the one-way-to-do-it. Further, it doesn't make sense to model this after the dictionary API where the next function is needed to avoid double lookups by returning pointers to both the key and value fields at the same time (allowing for modification of the value field). In contrast, for sets, there is no value field to look-up or mutate (the key should not be touched). So, we shouldn't be passing around pointers to the internal structure. I want to keep the internal structure of sets much more private than they were for dictionaries -- all access should be through the provided C API functions -- that keeps the implementation flexible and allows for future improvements without worrying that we've broken code for someone who has touched the internal structure directly. Also, the _Next() api is not as safe as the _GetIter api which checks for mutation during iteration. The safety should not be tossed aside without good reason. PySet_Update() --------------- Use PyObject_CallMethod(s, "update", "O", iterable). That is the preferred way to access all of the high volume methods. Only the fine grained methods (like contains, add, pop, or discard) have a need for a direct call. Adding unnecessary functions for the many-at-once methods gains you nothing -- perhaps saving a tiny O(1) look-up step in an O(n) operation. FWIW, the same reasoning also applies to why the list API defines PyList_Append() but not PyList_Extend(). PySet_AsList() --------------- There is already a function expressly for this purpose, PySequence_List(s). It is clear, readable, and is the one-way-to-do-it for turning arbitrary iterables into a list. FWIW, that function already has an optimization to pre-size the result list to the correct size, so it runs pretty fast (no over-allocate / resize dance). I had considered putting a further optimization inside PySequence_List to have a special case path for sets (like it does for tuples); however, it occurred to me that this can never be the time critical part of a program since the time to convert a set to a list is small compared to the time to construct the set in the first place (many times longer). IOW, further micro-optimization here is pointless. [Raymond] >> IOW, if I have I still have a say in the matter, the patch will most likely >> not >> be accepted. [Barry] > Can you explain why the additions above would not be obvious improvements? Yes. A fatter api is not a better api. The PyObject_CallMethod() approach is highly readable and assures direct correspondence with a Python set method of the same name. Trying to save the method name lookup time on O(n) methods is a false optimization. The concrete API should avoid duplicating services provided by the abstract API such as PySequence_List(). Also, the set API should not model parts of the dict API that grant direct access to the internal structure or were needed because dictionaries had a value field. As it stands now, it is possible to use sets in C programs and access them in a way that has a direct correspondence to pure Python code -- using PyObject_CallMethod() for things we would usually access by name, using the PyNumber API for things we would access using operators, using other parts of the abstract API exactly as we would in Python (PyObject_Repr, PyObject_GetIter, PySequence_List, PyObject_Print, etc.), and using a handful of direct access functions for the fine grained methods like (add, pop, contains, etc.). IOW, I like the way the C code looks now and I like the minimal, yet complete API. Let's don't muck it up. FWIW, the C implementation in Py2.5 already provides nice speed-ups for many operations. Likewise, its memory requirements have been reduced by a third. Try to enjoy the improvements without gilding the lily. Cheers, Raymond From raymond.hettinger at verizon.net Mon Mar 20 09:49:24 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Mon, 20 Mar 2006 03:49:24 -0500 Subject: [Python-Dev] GeneratorExit inheriting from Exception References: <441E108B.4040808@canterbury.ac.nz> Message-ID: <00ad01c64bfb$308561c0$b83efea9@RaymondLaptop1> [Guido] > Sigh. Enough already. PEP 352 was chosen to minimize incompatibilities > and maximize gain with minimal changes in the tree. Also note that > Warnings can sometimes be raised and should then treated as errors, so > Warning would have to inherit from Error. > > I vote for the status quo in HEAD, except I've got to think more about > the pros and cons of making GeneratorExit as special as > KeyboardInterrupt. While Guido is thinking, could one of the proponents please enumerate the reasons for treating GeneratorExit like KeyboardInterrupt and SystemExit. To me, they obviously should be under Exception, and not treated like KeyboardInterrupt or SystemExit, so that probably means that I'm missing something about GeneratorExit. Raymond From ncoghlan at gmail.com Mon Mar 20 10:19:01 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 20 Mar 2006 19:19:01 +1000 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <00ad01c64bfb$308561c0$b83efea9@RaymondLaptop1> References: <441E108B.4040808@canterbury.ac.nz> <00ad01c64bfb$308561c0$b83efea9@RaymondLaptop1> Message-ID: <441E7385.9030100@gmail.com> Raymond Hettinger wrote: > While Guido is thinking, could one of the proponents please enumerate the > reasons for treating GeneratorExit like KeyboardInterrupt and SystemExit. > > To me, they obviously should be under Exception, and not treated like > KeyboardInterrupt or SystemExit, so that probably means that I'm missing > something about GeneratorExit. If GeneratorExit *isn't* special, then correct catch-all exception handling around a yield expression in a generator looks like: def run_tasks(tasks, failed_tasks) for task in tasks: try: yield task() except GeneratorExit: # don't break gen.close() raise except Exception: failed_tasks.append((task, sys.exc_info())) Having to explicitly reraise a terminating exception like that is the exact idiom we're trying to get rid of for SystemExit and KeyboardInterrupt, so it makes sense to me to avoid it for GeneratorExit as well. Given that the default system-level excepthook *won't* ignore GeneratorExit (and so an error message will still be printed), I see moving it out with the other two terminating exceptions as the best option. The only way for that approach to do the wrong thing is if GeneratorExit is raised outside a generator's pseudothread, at which point I'd be blaming the code that raised it explicitly (the interpreter core only ever raises it as a result of a call to gen.close()). Note that this argument does *not* apply to StopIteration - that should stay under Exception because it should never accidentally leak beyond the code that is calling the next() method. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From anthony at interlink.com.au Mon Mar 20 10:19:32 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon, 20 Mar 2006 20:19:32 +1100 Subject: [Python-Dev] release24-maint FREEZE for 2.4.3c1, this Thursday Message-ID: <200603202019.37243.anthony@interlink.com.au> I'd like to freeze the release24-maint branch for 2.4.3c1 this Thursday 23rd March from 0000 UTC. I'll post a note once that's done, but as usual, please please only check in critical fixes in the week between 2.4.3c1 and 2.4.3 (which will follow next week). Please let me know if you have any issues with this. Should I also put this sort of information somewhere on the web? Maybe a slot at the top of the buildbot page? Thanks, Anthony -- Anthony Baxter It's never too late to have a happy childhood. From ncoghlan at gmail.com Mon Mar 20 10:27:00 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 20 Mar 2006 19:27:00 +1000 Subject: [Python-Dev] Python 3000 Process In-Reply-To: References: Message-ID: <441E7564.3040305@gmail.com> Guido van Rossum wrote: > I see increased activity related to Python 3000. This is great, but > there is some danger involved. Some of the dangers are: overloading > developers; setting unrealistic expectations for what can be > accomplished; scaring the more conservative user community; paralyzing > developers who can't decide whether to wait for Python 3000 or not. > > I'd like to address all these issues, but it may be better to first > spend some time on creating a more streamlined process. Perhaps it's > finally time to introduce a separate mailing list? If people agree, > let's call it python-3000 at python.org. For many developers this won't > make much of a difference (since they'll subscribe to both lists), but > it will give people who are only concerned with Python 2.x a way to > opt-out, and perhaps more importantly, it will make it clear whether > any particular proposal is intended for Python 3000 or for Python 2.x. +1 on the separate mailing list (and PEP numbers). If Barry (or another mailing list admin) is feeling energetic, some branch-specific topics on python-checkins would be nice, too. While I'm interested in tracking trunk checkins, I'm not so concerned about the exact details of what's happening on the p3yk branch. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Mon Mar 20 10:53:33 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 20 Mar 2006 19:53:33 +1000 Subject: [Python-Dev] Long-time shy failure in test_socket_ssl In-Reply-To: References: <1f7befae0601241422sd783c37u5d676893de2f6055@mail.gmail.com> <1f7befae0601241553v10f7dd41i364e92b98f9de4a2@mail.gmail.com> <1f7befae0602272048x50d260b3i29819389709e0ab6@mail.gmail.com> <1f7befae0603191835i20ba2de7w6f68b8ce883af159@mail.gmail.com> Message-ID: <441E7B9D.6030702@gmail.com> Neal Norwitz wrote: > On 3/19/06, Tim Peters wrote: >>> If anyone sees spurious failures with the buildbot (one time failures, >>> crashes, etc), please report the problems to python-dev. It would be >>> great to see if you can reproduce the results with the same tests that >>> failed. We need to determine if it is architecture specific, >>> test-order related, or something else. >> One-shot segfaults are still common enough on the Mac box, like the very recent: >> > > Most of the recent failures seem to be related to threads. the most > recent info is below. There's a lot more in case if it can help > someone debug. It seems that one thread is in pthread_cond_wait and > another thread is in PyObject_Call. Hrrm, test_thread_state hands an object pointer off to a different thread without INCREF'ing it first. Could there be a race condition with the deallocation of arguments at the end of test_thread_state? Specifically, that trace looks to me like the spawned thread is trying to call the function passed in while the main thread is trying to delete the argument tuple passed to test_thread_state. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From bjourne at gmail.com Mon Mar 20 10:53:52 2006 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Mon, 20 Mar 2006 09:53:52 +0000 Subject: [Python-Dev] Python 3000 Process In-Reply-To: References: Message-ID: <740c3aec0603200153l6f42dc70yeda46129b6d5495d@mail.gmail.com> > Please don't respond with answers to these questions -- each of them > is worth several threads. Instead, ponder them, and respond with a +1 > or -1 on the creation of the python-3000 mailing list. We'll start > discussing the issues there -- or here, if the general sense is not to > split the list. -0 on separate mailing list and PEP numbers. Because to much separation between python 2.x and python 3.x makes 3.x seem more like vaporware that will never happen (Perl 6). I would have hoped that the transition to 3.x would be evolutionary in small steps; removal of string exceptions in one major release, replacement of print with a function in the next and so on. A total redesign from bottom up in one release has a very high probability of "failing" IMHO. Or at least very annoying for end users that who to deal with two parallel and incompatible versions. The reality is that discussions about python 3000 has a high probability of being a total waste of time. I think all those of us who have for lengths argued whether lambda should stay or be dropped knows that. :) -- mvh Bj?rn From mattjfleming at googlemail.com Mon Mar 20 12:56:52 2006 From: mattjfleming at googlemail.com (Matthew Fleming) Date: Mon, 20 Mar 2006 11:56:52 +0000 Subject: [Python-Dev] Python 3000 Process In-Reply-To: <441E7564.3040305@gmail.com> References: <441E7564.3040305@gmail.com> Message-ID: <5ff4a1e50603200356i18bfe6fao@mail.gmail.com> On 20/03/06, Nick Coghlan wrote: > > > +1 on the separate mailing list (and PEP numbers). > > If Barry (or another mailing list admin) is feeling energetic, some > branch-specific topics on python-checkins would be nice, too. While I'm > interested in tracking trunk checkins, I'm not so concerned about the > exact > details of what's happening on the p3yk branch. +1 for the separate mailing list. Reading anything from 50 to 100 emails first thing every morning, I need all the help I can get trying to distinguish different topics. Also, as Nick said, a separate mailing list for p3yk check-ins would be nice but naturally, only if someone has the time to maintain it. Thanks, Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060320/040beabc/attachment.htm From skip at pobox.com Mon Mar 20 13:03:44 2006 From: skip at pobox.com (skip at pobox.com) Date: Mon, 20 Mar 2006 06:03:44 -0600 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: References: <4418CDD0.8020803@canterbury.ac.nz> <17434.61837.491103.995641@montanaro.dyndns.org> <441B89EE.8000303@canterbury.ac.nz> <4c45c1530603181131s58b3d598i@mail.gmail.com> <441CB8C5.3000403@canterbury.ac.nz> <17436.54341.20232.333509@montanaro.dyndns.org> Message-ID: <17438.39456.858553.490354@montanaro.dyndns.org> Skip> There seem to be other places where Python is beginning to require Skip> parens even though they aren't strictly necessary to resolve Skip> syntactic ambiguity. Guido> In the style guide only, I hope. Alex> Technically, I believe the first place where this did not apply Alex> was list comprehensions, ... This was the instance I was thinking of, but it was far enough back that I didn't remember if it was for readability or disambiguation. Skip From thomas at python.org Mon Mar 20 14:32:21 2006 From: thomas at python.org (Thomas Wouters) Date: Mon, 20 Mar 2006 14:32:21 +0100 Subject: [Python-Dev] Python 3000 Process In-Reply-To: References: Message-ID: <9e804ac0603200532h6ddea030k3574585a7d6dfc9a@mail.gmail.com> On 3/20/06, Guido van Rossum wrote: > Please don't respond with answers to these questions -- each of them > is worth several threads. Instead, ponder them, and respond with a +1 > or -1 on the creation of the python-3000 mailing list. We'll start > discussing the issues there -- or here, if the general sense is not to > split the list. Definately very +1 on a python-3000 list. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060320/a8d471df/attachment.html From g.brandl at gmx.net Mon Mar 20 15:28:17 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 20 Mar 2006 15:28:17 +0100 Subject: [Python-Dev] Bug Day on Friday, 31st of March Message-ID: Hello, it's time for the 7th Python Bug Day. The aim of the bug day is to close as many bugs, patches and feature requests as possible, this time with a special focus on new features that can still go into the upcoming 2.5 alpha release. When? ^^^^^ The bug day will take place on Friday, March 31st, running from 1PM to 7PM GMT (9AM to 3PM Eastern time). You don't need to be around all day; feel free to stop by for a few hours and contribute. Where and How? ^^^^^^^^^^^^^^ To join, stop by the IRC channel #python-dev on irc.freenode.net, where efforts will be discussed and coordinated. We'll collaboratively go through the Python bug database at SourceForge and fix things as they come up. IMPORTANT: *No* prior knowledge of the Python source is necessary to participate! You'll get all assistance the developers can offer for starting up with helping, this is in fact a good opportunity to learn the basics. Bug day participation helps the developers and makes Python 2.5 a better release by reducing the backlog of bugs and patches. Plus, it's fun! More information ^^^^^^^^^^^^^^^^ For instructions and more information, see the Wiki page at http://www.python.org/cgi-bin/moinmoin/PythonBugDay Cheers, Georg From barry at python.org Mon Mar 20 15:04:47 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 20 Mar 2006 09:04:47 -0500 Subject: [Python-Dev] Python 3000 Process In-Reply-To: <441E7564.3040305@gmail.com> References: <441E7564.3040305@gmail.com> Message-ID: <1142863487.10599.39.camel@resist.wooz.org> On Mon, 2006-03-20 at 19:27 +1000, Nick Coghlan wrote: > +1 on the separate mailing list (and PEP numbers). I'm neutral on whether a new mailing list should be added, but +1 on separate PEP numbers. > If Barry (or another mailing list admin) is feeling energetic, some > branch-specific topics on python-checkins would be nice, too. While I'm > interested in tracking trunk checkins, I'm not so concerned about the exact > details of what's happening on the p3yk branch. If we agree on adding a separate discussion list, then I think we should have a separate checkins list. Plumbing that through the subversion machinery shouldn't be too hard. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060320/195e5310/attachment.pgp From barry at python.org Mon Mar 20 15:06:57 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 20 Mar 2006 09:06:57 -0500 Subject: [Python-Dev] Python 2.5 Schedule In-Reply-To: References: <1142705174.32089.69.camel@geddy.wooz.org> <1142826671.10599.33.camel@resist.wooz.org> Message-ID: <1142863617.10598.42.camel@resist.wooz.org> On Sun, 2006-03-19 at 20:40 -0800, Guido van Rossum wrote: > I'm -0 for changing this in 3.0; a larger-scale reorganization could > be undertaken but it's not a big priority. > > Before you spend more energy on this, I'd like to address the process > for Python 3000, which is too chaotic right now. See my new thread > titled "Python 3000 Process". Fair enough. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060320/a59ddadd/attachment.pgp From barry at python.org Mon Mar 20 16:32:28 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 20 Mar 2006 10:32:28 -0500 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: References: <441C02C8.5080504@iinet.net.au> <1142695605.32074.49.camel@geddy.wooz.org> Message-ID: <1142868748.10625.58.camel@resist.wooz.org> On Sat, 2006-03-18 at 15:37 -0800, Brett Cannon wrote: > > Actually, this prompts me to write about an issue I have with PEP 352. > > I actually don't think it's necessary (yes, I know it's already in the > > tree). > > > > Much to personal pain and sprint time. Are you trying to make me shed > a manly tear, Barry?!? =) > So it isn't that PEP 352 is unnecessary since there is nothing here > about deprecating string execptions and making built-in exceptions > new-style. You just want to change the renaming of the exceptions. Yes, sorry Brett! No the other things about PEP 352 are all good, and I was only commenting about the new hierarchy changes. I still don't like that part of the PEP, but I appreciate the competing compromises being made so I can live with it for Python 2.x. > I still like my idea of a ControlFlowException, but that died with PEP > 348 (along with part of my innocence). Good! The sooner you replace that with soul-crushing defeatism the happier you will be! > Just remember that PEP 348 was meant for Py3K when we are supposed to > break stuff and how much resistence I hit. Granted my changes were > more radical, but even in the end, small changes were resisted > heavily. In other words, make sure you have the time and energy to > take this on. =) I know all about soul-crushing PEP championship, which is why I'm such a happy person. :) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060320/9f5a1b5d/attachment.pgp From barry at python.org Mon Mar 20 16:33:46 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 20 Mar 2006 10:33:46 -0500 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <441CB8AC.4010300@canterbury.ac.nz> References: <441C02C8.5080504@iinet.net.au> <1142695605.32074.49.camel@geddy.wooz.org> <441CB8AC.4010300@canterbury.ac.nz> Message-ID: <1142868826.10625.59.camel@resist.wooz.org> On Sun, 2006-03-19 at 13:49 +1200, Greg Ewing wrote: > Barry Warsaw wrote: > > > Exception > > +- KeyboardInterrupt > > +- GeneratorExit > > +- SystemExit > > +- StopIteration > > +- Error > > | +- ImportError > > | +- (etc.) > > | > > +- Warning > > +- UserWarning > > +- (etc.) > > +42! This is beautifully clear and simple, especially > compared to some of the other exception hierarchy > reorganisations that have been proposed. I still believe in this, and I'm thankful for the support I've seen. It won't happen for Python 2.x, but I do plan on addressing this for Py3K. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060320/cb317bdf/attachment.pgp From guido at python.org Mon Mar 20 16:38:57 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 20 Mar 2006 07:38:57 -0800 Subject: [Python-Dev] Python 3000 Process In-Reply-To: <740c3aec0603200153l6f42dc70yeda46129b6d5495d@mail.gmail.com> References: <740c3aec0603200153l6f42dc70yeda46129b6d5495d@mail.gmail.com> Message-ID: On 3/20/06, BJ?rn Lindqvist wrote: > -0 on separate mailing list and PEP numbers. You seem to be the minority so far... > Because to much separation between python 2.x and python 3.x makes 3.x > seem more like vaporware that will never happen (Perl 6). I'm actually trying to reenergize things to make sure Python 3000 *will* happen. > I would have > hoped that the transition to 3.x would be evolutionary in small steps; > removal of string exceptions in one major release, replacement of > print with a function in the next and so on. To the extent possible that's certainly the plan. But there are certain changes that are too hard to do without introducing a major incompatibility -- you can't add warnings for everything, especially not where the semantics of an existing API changes (e.g. range(), or dict.keys(), or int/int). Future statements don't help for some of these cases either (especially not for dict.keys() and similar, since dicts are passed between modules). > A total redesign from > bottom up in one release has a very high probability of "failing" > IMHO. There's not going to be a total redesign. This is exactly the kind of misunderstanding that I would like to address by specifying a process for getting there rather than just starting to discuss features. 3.0 will be the opportunity to radically clean up mistakes I made 16 years ago, by allowing the language to change backwards incompatibly. It is *not* intended to be as an invitation to start adding random new ideas. (In fact, there's something to say for *limiting* Python 3000 to the incompatibilities, and to continue introducing new ideas in 2.x, as long as they don't require incompatibilities beyond the occasional new keyword, which can be handled with a warning and a future statement. But that's for the Python 3000 process to be decided.) > Or at least very annoying for end users that who to deal with > two parallel and incompatible versions. Well that can't be helped, really, unless you want to keep the language backwards compatible forever, can it? Some necessary changes *are* incompatible, so they are going to cause some pain during the transition. The alternative would be years and years of pain as various changes are introduced in different versions (if we did it all at once it would *be* Python 3.0 of course :-). Python 3000 hopes to ease the pain by making it all happen at once (ripping the band-aid off quickly, so to speak :-) while at the same time giving users a chance to decide *when* to do it -- since (in my vision, anyway) there will be a fair amount of maintenance applied to the 2.x branch even after 3.0 has been released. > The reality is that > discussions about python 3000 has a high probability of being a total > waste of time. I think all those of us who have for lengths argued > whether lambda should stay or be dropped knows that. :) Now that sounds like an insult. I hope you didn't mean it that way. Discussing Python 3000 will *not* be a waste of time. I fully intend to carry through the changes and release Python 3.0 as I have envisioned it for a long time. If anything's a waste of time, it's discussing random Python 3000 proposals without having a good process in place first. That's what I want to change right now. Barry, if you could create that mailing list, please? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Mon Mar 20 16:39:04 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 20 Mar 2006 10:39:04 -0500 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <441D08E9.10503@gmail.com> References: <441D08E9.10503@gmail.com> Message-ID: <1142869144.10599.64.camel@resist.wooz.org> On Sun, 2006-03-19 at 17:31 +1000, Nick Coghlan wrote: > With PEP 352 (tweaked to move GeneratorExit out from under Exception): > - "except:" continues to mean catch everything > - "except Exception:" now does the right thing > - inheriting from Exception continues to be correct for user exceptions > - errors may choose to inherit from one of the existing Errors instead > > With Barry's proposed hierarchy: > - "except:" continues to mean catch everything > - "except Exception:" continues to do the wrong thing > - all code has to change to do "except Error:" instead > - inheriting from Exception becomes incorrect for user exceptions > - all code has to change to inherit from Error instead > - non-error user exceptions (such as completion of a search using nested > loops) have no clear parent to inherit from (both Error and Exception > are wrong, albeit for different reasons. > > The additional pain required in order to have 'Exception' at the root of the > hierarchy just isn't worth it. One quibble. Since the term used for the general concept of something that is raised and caught is "exception" and since all the raise-able objects live in a module called "exceptions", it is confusing that "except Exception" will not catch all exceptions. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060320/1ac1cdde/attachment.pgp From barry at python.org Mon Mar 20 16:43:30 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 20 Mar 2006 10:43:30 -0500 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: References: <441C02C8.5080504@iinet.net.au> <1142695605.32074.49.camel@geddy.wooz.org> <441CC358.2020305@gmail.com> <002e01c64b4a$5a3f2a70$88472597@bagio> <441E173E.8050109@canterbury.ac.nz> Message-ID: <1142869410.10598.69.camel@resist.wooz.org> On Sun, 2006-03-19 at 19:18 -0800, Guido van Rossum wrote: > I have a problem with using Error as the focal point since so many > exceptions (user-defined or otherwise) aren't errors. I'm not sure that's totally true in practice. I think most user-defined exceptions are actually errors. Ideally, StandardError would be called Error (or there'd be an alias of that name) and people should be deriving their error exceptions from Error. Their non-error exceptions would be derived from Exception. The last thing I'll suggest in this thread for Python 2.5 is to add an alias called Error for StandardError. Then, users can begin to do the sensible thing (as above). -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060320/353e55fa/attachment.pgp From aahz at pythoncraft.com Mon Mar 20 17:18:33 2006 From: aahz at pythoncraft.com (Aahz) Date: Mon, 20 Mar 2006 08:18:33 -0800 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <441CB8A1.8040205@canterbury.ac.nz> References: <441C02C8.5080504@iinet.net.au> <1142695605.32074.49.camel@geddy.wooz.org> <441C2C3A.8070003@strakt.com> <1142697104.32089.58.camel@geddy.wooz.org> <441CB8A1.8040205@canterbury.ac.nz> Message-ID: <20060320161833.GA8328@panix.com> On Sun, Mar 19, 2006, Greg Ewing wrote: > Barry Warsaw wrote: >> >> One possible approach is to revert BaseException out of Py2.5, >> re-position KeyboardInterrupt, and add Error as an alias for >> StandardError. Then we can encourage people to start using Error as the >> base classes for their own errors. > > Also maybe start issuing warnings whenever you inherit directly from > Exception. -1 -- I occasionally use exceptions as a multi-loop break. That's a perfectly valid Python practice, those exceptions should inherit from Exception, and there should not be any warnings raised. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From barry at python.org Mon Mar 20 17:48:21 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 20 Mar 2006 11:48:21 -0500 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <20060320161833.GA8328@panix.com> References: <441C02C8.5080504@iinet.net.au> <1142695605.32074.49.camel@geddy.wooz.org> <441C2C3A.8070003@strakt.com> <1142697104.32089.58.camel@geddy.wooz.org> <441CB8A1.8040205@canterbury.ac.nz> <20060320161833.GA8328@panix.com> Message-ID: <1142873301.10625.78.camel@resist.wooz.org> On Mon, 2006-03-20 at 08:18 -0800, Aahz wrote: > -1 -- I occasionally use exceptions as a multi-loop break. That's a > perfectly valid Python practice, those exceptions should inherit from > Exception, and there should not be any warnings raised. Exactly! But they're not errors, so "except Exception" should catch them but "except Error" should not. This does speak to a generic ControlFlow (but not ControlFlowError) base class. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060320/3b4b02e4/attachment.pgp From python-dev at python.org Mon Mar 20 06:28:35 2006 From: python-dev at python.org (python-dev at python.org) Date: Mon, 20 Mar 2006 05:28:35 +0000 Subject: [Python-Dev] buildbot failure in x86 W2k trunk Message-ID: <20060320052836.1BADB1E400F@bag.python.org> The Buildbot has detected a new failure of x86 W2k trunk. Full details are available at: http://www.python.org:9010/x86%20W2k%20trunk/builds/83 Buildbot URL: http://www.python.org:9010/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: anthony.baxter BUILD FAILED: failed compile sincerely, -The Buildbot From python-dev at python.org Mon Mar 20 06:28:44 2006 From: python-dev at python.org (python-dev at python.org) Date: Mon, 20 Mar 2006 05:28:44 +0000 Subject: [Python-Dev] buildbot failure in x86 XP-2 trunk Message-ID: <20060320052844.253121E400F@bag.python.org> The Buildbot has detected a new failure of x86 XP-2 trunk. Full details are available at: http://www.python.org:9010/x86%20XP-2%20trunk/builds/69 Buildbot URL: http://www.python.org:9010/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: anthony.baxter BUILD FAILED: failed compile sincerely, -The Buildbot From python-dev at python.org Mon Mar 20 07:40:39 2006 From: python-dev at python.org (python-dev at python.org) Date: Mon, 20 Mar 2006 06:40:39 +0000 Subject: [Python-Dev] buildbot failure in x86 XP-2 trunk Message-ID: <20060320064039.4503B1E4003@bag.python.org> The Buildbot has detected a new failure of x86 XP-2 trunk. Full details are available at: http://www.python.org:9010/x86%20XP-2%20trunk/builds/71 Buildbot URL: http://www.python.org:9010/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: anthony.baxter,neal.norwitz BUILD FAILED: failed compile sincerely, -The Buildbot From python-dev at python.org Mon Mar 20 07:40:53 2006 From: python-dev at python.org (python-dev at python.org) Date: Mon, 20 Mar 2006 06:40:53 +0000 Subject: [Python-Dev] buildbot failure in x86 W2k trunk Message-ID: <20060320064053.B403F1E4003@bag.python.org> The Buildbot has detected a new failure of x86 W2k trunk. Full details are available at: http://www.python.org:9010/x86%20W2k%20trunk/builds/85 Buildbot URL: http://www.python.org:9010/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: anthony.baxter,neal.norwitz BUILD FAILED: failed compile sincerely, -The Buildbot From python-dev at python.org Mon Mar 20 10:30:05 2006 From: python-dev at python.org (python-dev at python.org) Date: Mon, 20 Mar 2006 09:30:05 +0000 Subject: [Python-Dev] buildbot failure in x86 XP 2.4 Message-ID: <20060320093006.06B761E4010@bag.python.org> The Buildbot has detected a new failure of x86 XP 2.4. Full details are available at: http://www.python.org:9010/x86%20XP%202.4/builds/20 Buildbot URL: http://www.python.org:9010/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: vinay.sajip BUILD FAILED: failed failed slave lost sincerely, -The Buildbot From tjreedy at udel.edu Mon Mar 20 19:30:34 2006 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 20 Mar 2006 13:30:34 -0500 Subject: [Python-Dev] Python 3000 Process References: <740c3aec0603200153l6f42dc70yeda46129b6d5495d@mail.gmail.com> Message-ID: "Guido van Rossum" wrote in message >Barry, if you could create that mailing list, please? And please mirror it on gmane the same way as this list is. Terry From rrr at ronadam.com Mon Mar 20 19:49:19 2006 From: rrr at ronadam.com (Ron Adam) Date: Mon, 20 Mar 2006 12:49:19 -0600 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: <17438.39456.858553.490354@montanaro.dyndns.org> References: <4418CDD0.8020803@canterbury.ac.nz> <17434.61837.491103.995641@montanaro.dyndns.org> <441B89EE.8000303@canterbury.ac.nz> <4c45c1530603181131s58b3d598i@mail.gmail.com> <441CB8C5.3000403@canterbury.ac.nz> <17436.54341.20232.333509@montanaro.dyndns.org> <17438.39456.858553.490354@montanaro.dyndns.org> Message-ID: <441EF92F.5010406@ronadam.com> skip at pobox.com wrote: > Skip> There seem to be other places where Python is beginning to require > Skip> parens even though they aren't strictly necessary to resolve > Skip> syntactic ambiguity. > > Guido> In the style guide only, I hope. > > Alex> Technically, I believe the first place where this did not apply > Alex> was list comprehensions, ... > > This was the instance I was thinking of, but it was far enough back that I > didn't remember if it was for readability or disambiguation. There is also the cases of types needing parens such as tuple((1,2,3)). The same is true for dict, list, and sets. They could have been written to accept individual items, (not a suggestion as it is slower), so requiring tuples, sets, and/or sequences where they are used as such isn't unexpected. But as long as I can do the following I will be happy. some_exc_group = (some_except, some_except, some_except) try: ... except excgroup as value: ... Cheers, Ron From mal at egenix.com Mon Mar 20 20:29:55 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 20 Mar 2006 20:29:55 +0100 Subject: [Python-Dev] [Python-checkins] r43041 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: <441C874A.1020305@v.loewis.de> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> <441B20C5.1080009@egenix.com> <441B354D.1030906@v.loewis.de> <441C874A.1020305@v.loewis.de> Message-ID: <441F02B3.9030507@egenix.com> Martin v. L?wis wrote: > Fernando Perez wrote: >> So I think M.A. is right on the money here with his statement. Unless you >> consider the Linux/64bit camp insignificant. But if that is the case, it >> might be worth putting a big statement in the 2.5 release notes indicating >> "there is a good chance, if you use third party extensions and a 64 bit OS, >> that this won't work for you". Which will mean that a fraction of the >> scientific userbase (a big, enthusiastic and growing community of python >> users) will have to stick to 2.4. > > It's more intricate than that. If certain extensions are of interest to > the scientific community, I certainly expect the authors of them to fix > them. > > However, to "fix" the extension can still be done in different levels: > you can easily fix it to not crash, by just honoring all compiler > warnings that gcc produces. That doesn't mean you fully support > Py_ssize_t, since you might not support collections with more than > 2**31 elements. In these cases, the extension will not crash, but it > will compute incorrect results if you ever encounter such a collection. Sorry Martin, but such an advice is simply not good enough. You can't expect people to go chasing compiler warnings to "fix" their code without at the same time giving them a complete list of APIs that changed and which is needed for the still required manual inspection. I really don't understand why you put so much effort into trying to argue that the ssize_t patch isn't going to break extensions or that fixing compiler warnings is good enough. The interface to the Python API should not be a compiler giving you warnings, it should be a document where users (extension authors) can read about the changes, check their code using grep an editor or some other tool and then *know* that their code works rather than relying on the compiler warnings which only *might* catch all the cases where breakage occurs. Like Fernando said, 64-bit is becoming the de-facto standard for Unix systems very quickly these days. It is becoming hard to buy systems not supporting 64-bit and, of course, people are anxious to use the capabilities of the systems they bought (whether they need them or not). If you look at the market for root servers as second example of users of 64-bit machines next to the scientific users, several big hosters have completely switched over to 64-bit. You can't even get a 32-bit install of Linux on those machines - because they use custom kernels patched for their particular hardware. Ignoring this is just foolish. I consider the fact that it's currently not possible to have a look at the changed APIs a documentation which is in the responsibility of the patch authors to provide (just like it always is). Please provide such a list. > Even on all these AMD64 Linux machines, this is still fairly unlikely, > since you need considerably more than 16GiB main memory to do something > useful on such collections - very few machines have that much memory > today. Again, people that have such machines and need to run Python > programs with that many data need to make sure that the extensions > work for them. Sticking with 2.4 is no option for these people, since > 2.4 doesn't support such large collections, anyway. > > To really fix such extensions, I recommend building with either > Microsoft C or Intel C. The Microsoft compiler is available free > of charge, but runs on Windows only. It gives good warnings, and > if you fix them all (in a careful way), your code should fully > support 64 bits. Likewise for the Intel compiler: it is available > for free only for a month, but it runs on Linux as well. > > OTOH, I'm still doubtful how many extensions will be really affected > by the change in the first place. Your code *breaks* with the change > only if you implement the sequence or buffer protocols. I'm doubtful > that this is an issue for most applications, since many extensions > (I believe) work without implementing these protocols. You know that it's not only the sequence and buffer protocol that changed. If you grep through Include, you get at these reference to output variables which are going to cause breakage regardless of whether you use 64-bit or not simply due to the fact that the function is writing into memory owned by the caller: ./dictobject.h: -- PyAPI_FUNC(int) PyDict_Next( -- PyObject *mp, Py_ssize_t *pos, PyObject **key, PyObject **value); ./pyerrors.h: -- PyAPI_FUNC(int) PyUnicodeEncodeError_GetStart(PyObject *, Py_ssize_t *); -- PyAPI_FUNC(int) PyUnicodeDecodeError_GetStart(PyObject *, Py_ssize_t *); -- PyAPI_FUNC(int) PyUnicodeTranslateError_GetStart(PyObject *, Py_ssize_t *); -- PyAPI_FUNC(int) PyUnicodeEncodeError_GetEnd(PyObject *, Py_ssize_t *); -- PyAPI_FUNC(int) PyUnicodeDecodeError_GetEnd(PyObject *, Py_ssize_t *); -- PyAPI_FUNC(int) PyUnicodeTranslateError_GetEnd(PyObject *, Py_ssize_t *); ./sliceobject.h: -- PyAPI_FUNC(int) PySlice_GetIndices(PySliceObject *r, Py_ssize_t length, -- Py_ssize_t *start, Py_ssize_t *stop, Py_ssize_t *step); -- PyAPI_FUNC(int) PySlice_GetIndicesEx(PySliceObject *r, Py_ssize_t length, -- Py_ssize_t *start, Py_ssize_t *stop, -- Py_ssize_t *step, Py_ssize_t *slicelength); ./stringobject.h: -- PyAPI_FUNC(int) PyString_AsStringAndSize( -- register PyObject *obj, /* string or Unicode object */ -- register char **s, /* pointer to buffer variable */ -- register Py_ssize_t *len /* pointer to length variable or NULL -- (only possible for 0-terminated -- strings) */ -- ); ./abstract.h: -- PyAPI_FUNC(int) PyObject_AsCharBuffer(PyObject *obj, -- const char **buffer, -- Py_ssize_t *buffer_len); -- PyAPI_FUNC(int) PyObject_AsReadBuffer(PyObject *obj, -- const void **buffer, -- Py_ssize_t *buffer_len); -- PyAPI_FUNC(int) PyObject_AsWriteBuffer(PyObject *obj, -- void **buffer, -- Py_ssize_t *buffer_len); ./unicodeobject.h: -- PyAPI_FUNC(PyObject*) PyUnicode_DecodeUTF8Stateful( -- const char *string, /* UTF-8 encoded string */ -- Py_ssize_t length, /* size of string */ -- const char *errors, /* error handling */ -- Py_ssize_t *consumed /* bytes consumed */ -- ); ./ceval.h: -- PyAPI_FUNC(int) _PyEval_SliceIndex(PyObject *, Py_ssize_t *); Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 20 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mcherm at mcherm.com Mon Mar 20 20:43:28 2006 From: mcherm at mcherm.com (Michael Chermside) Date: Mon, 20 Mar 2006 11:43:28 -0800 Subject: [Python-Dev] GeneratorExit inheriting from Exception Message-ID: <20060320114328.2wldxhwo7fs48gco@login.werra.lunarpages.com> Barry writes: > I still believe in this, and I'm thankful for the support I've seen. It > won't happen for Python 2.x, but I do plan on addressing this for Py3K. When you do, I'd like you to consider one change to the names. You are proposing this: > Exception > +- KeyboardInterrupt > +- GeneratorExit > +- SystemExit > +- StopIteration > +- Error > | +- ImportError > | +- (etc.) [...] If we use that structure, I still strongly feel that what you call "Error" ought to be named "Exception" (and what you call "Exception" should be named "BaseException" or perhaps "Raisable"). The reason is this: Most people coming to Python today are coming from languages more modern than Fortran and C, and (if they have any programming background at all) they are familiar with the idea of an exception. When chatting around the water cooler, they probably call it "an exception", rather than "a dingus", "one of those thingys that unwinds the stack until reaching a handler", or even "an error". (The term "an error" is usually used around the water cooler for any kind of mistake, whether human or programatic.) This is encouraged in Python by the fact that the keyword for handling exceptions in Python is "except". But we don't want people using the dingus at the top of the hierarchy -- only a few experts who know what they're doing should care about that one. We want them using the one that you call "Error". So by naming that one "Exception", we take advantage of the meaning of the word, the experience from other languages, and the keywords of the language to encourage correct usage. Naming the top level dingus something slightly odd is a small price to pay (although both "BaseException" and "Raisable" seem instantly comprehensible to me). I present Java as an example. Given it's position as the new Cobol, there are literally hundreds of thousands of not-very-skillful Java programmers out there. Yet it is rare to see them write "catch Throwable" when they should use "catch Exception". That mistake is far more common among beginning Python users. Surely we can do at least as well as Java! -- Michael Chermside PS: I've intentionally ignored the question Guido raised about whether Warning belongs under your "Error" or your "Exception". From barry at python.org Mon Mar 20 21:13:43 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 20 Mar 2006 15:13:43 -0500 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <20060320114328.2wldxhwo7fs48gco@login.werra.lunarpages.com> References: <20060320114328.2wldxhwo7fs48gco@login.werra.lunarpages.com> Message-ID: <1142885623.10625.109.camel@resist.wooz.org> On Mon, 2006-03-20 at 11:43 -0800, Michael Chermside wrote: > When you do, I'd like you to consider one change to the names. You are > proposing this: I'll keep this in mind, but won't comment further here for two reasons. I want to think about it some more (you throw, er raise some good points ;) and I'd rather carry the discussion forward on the soon to be announced python-3000 mailing list. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060320/6417cf2a/attachment.pgp From barry at python.org Mon Mar 20 21:22:56 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 20 Mar 2006 15:22:56 -0500 Subject: [Python-Dev] Python 3000 Process In-Reply-To: References: <740c3aec0603200153l6f42dc70yeda46129b6d5495d@mail.gmail.com> Message-ID: <1142886176.10598.111.camel@resist.wooz.org> On Mon, 2006-03-20 at 13:30 -0500, Terry Reedy wrote: > "Guido van Rossum" wrote in message > >Barry, if you could create that mailing list, please? > > And please mirror it on gmane the same way as this list is. Subscription requests have been made. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060320/6edb9cd2/attachment.pgp From p.f.moore at gmail.com Mon Mar 20 21:30:18 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 20 Mar 2006 20:30:18 +0000 Subject: [Python-Dev] Python 3000 Process In-Reply-To: <1142886176.10598.111.camel@resist.wooz.org> References: <740c3aec0603200153l6f42dc70yeda46129b6d5495d@mail.gmail.com> <1142886176.10598.111.camel@resist.wooz.org> Message-ID: <79990c6b0603201230t16aa4e13r14d3af06f43cc9ed@mail.gmail.com> On 3/20/06, Barry Warsaw wrote: > On Mon, 2006-03-20 at 13:30 -0500, Terry Reedy wrote: > > "Guido van Rossum" wrote in message > > >Barry, if you could create that mailing list, please? > > > > And please mirror it on gmane the same way as this list is. > > Subscription requests have been made. > -Barry I assume an appropriate signup URL/address will be announced here (and added to the python.org site) when the list is created? Paul. From martin at v.loewis.de Mon Mar 20 22:10:37 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 20 Mar 2006 22:10:37 +0100 Subject: [Python-Dev] [Python-checkins] r43041 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: <441F02B3.9030507@egenix.com> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> <441B20C5.1080009@egenix.com> <441B354D.1030906@v.loewis.de> <441C874A.1020305@v.loewis.de> <441F02B3.9030507@egenix.com> Message-ID: <441F1A4D.8010509@v.loewis.de> M.-A. Lemburg wrote: > I really don't understand why you put so much effort into > trying to argue that the ssize_t patch isn't going to break > extensions or that fixing compiler warnings is good enough. Well, in *this* thread, my main point is that I don't want to provide the list that you demand, mainly because I consider it a waste of my time to create it. > I consider the fact that it's currently not possible to have > a look at the changed APIs a documentation which is in the > responsibility of the patch authors to provide (just like it > always is). It is possible to look at the changed APIs, see http://docs.python.org/dev/api/sequence.html Regards, Martin From jimjjewett at gmail.com Mon Mar 20 22:38:45 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 20 Mar 2006 16:38:45 -0500 Subject: [Python-Dev] supported platforms OS2? Message-ID: Is there a list of currently supported platforms? Is OS2 still actively supported? From martin at v.loewis.de Mon Mar 20 22:44:32 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 20 Mar 2006 22:44:32 +0100 Subject: [Python-Dev] supported platforms OS2? In-Reply-To: References: Message-ID: <441F2240.1080608@v.loewis.de> Jim Jewett wrote: > Is there a list of currently supported platforms? No, only a negative list: PEP 11. > Is OS2 still actively supported? It's not listed as unsupported. Whether it actually works, nobody knows. Regards, Martin From jimjjewett at gmail.com Mon Mar 20 22:49:33 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 20 Mar 2006 16:49:33 -0500 Subject: [Python-Dev] supported platforms OS2? In-Reply-To: References: Message-ID: (Sorry about the early send) Is there a list of currently supported platforms? Is OS2 still actively supported? The reason I ask is that PYOS_OS2 (in Modules/posixmodule.c and Include/pyport.h) triggers enough ifdeffery that I'm certain it missed out on some updates, and I'm not certain it is even still correct. If no one is checking, should it be added to the unsupported systems list along with Windows 95? -jJ From mal at egenix.com Mon Mar 20 22:53:09 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 20 Mar 2006 22:53:09 +0100 Subject: [Python-Dev] Documenting the ssize_t Python C API changes In-Reply-To: <441F1A4D.8010509@v.loewis.de> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> <441B20C5.1080009@egenix.com> <441B354D.1030906@v.loewis.de> <441C874A.1020305@v.loewis.de> <441F02B3.9030507@egenix.com> <441F1A4D.8010509@v.loewis.de> Message-ID: <441F2445.6040504@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg wrote: >> I really don't understand why you put so much effort into >> trying to argue that the ssize_t patch isn't going to break >> extensions or that fixing compiler warnings is good enough. > > Well, in *this* thread, my main point is that I don't want > to provide the list that you demand, mainly because I consider > it a waste of my time to create it. It's not a waste of time at all: you'd be helping lots and lots of developers out there who want to fix their extensions. Apart from that I can only repeat what I said before: you expect developers to put lots of time into fixing their extensions to run with Python 2.5, so it's only fair that you invest some time into making it as easy as possible for them. The ssize_t patch is the single most disruptive patch in Python 2.5, so it deserves special attention. >> I consider the fact that it's currently not possible to have >> a look at the changed APIs a documentation which is in the >> responsibility of the patch authors to provide (just like it >> always is). > > It is possible to look at the changed APIs, see > > http://docs.python.org/dev/api/sequence.html That's neither a complete list (see the grep I sent), nor does it highlight the changes. The API documentation is not the right place for the documentation of this change, IMHO, since it always refers to the current state in a specific Python release and not the changes applied to an API compared to an older release. The PEP is the perfect place for such a list, I guess. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 20 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From jimjjewett at gmail.com Mon Mar 20 23:21:25 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 20 Mar 2006 17:21:25 -0500 Subject: [Python-Dev] r43041 - python/trunk/Modules/_ctypes/cfield.c Message-ID: (Regarding changes caused by ssize_t0 Martin v. L?wis wrote: > It is possible to look at the changed APIs, see > http://docs.python.org/dev/api/sequence.html I think the problem is more with APIs that are not part of the sequence page. For instance, in http://mail.python.org/pipermail/python-dev/2006-March/062679.html M.-A. Lemburg mentioned PyAPI_FUNC(int) PyDict_Next( PyObject *mp, Py_ssize_t *pos, PyObject **key, PyObject **value); PyDict_Next isn't on the sequence page, but if you call it, it could write a Py_ssize_t into pos, even though your pointer is only large enough for an int. To do things correctly, you really have to change your call to the new version (and add the #ifdef) #if PY_VERSION_HEX < 0x02050000 typedef int Py_ssize_t; #endif You can downcast if you aren't ready to support 64-bits, but ... setting up a possible buffer overflow is a bit worse than simply not supporting. -jJ From martin at v.loewis.de Mon Mar 20 23:48:33 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 20 Mar 2006 23:48:33 +0100 Subject: [Python-Dev] Documenting the ssize_t Python C API changes In-Reply-To: <441F2445.6040504@egenix.com> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> <441B20C5.1080009@egenix.com> <441B354D.1030906@v.loewis.de> <441C874A.1020305@v.loewis.de> <441F02B3.9030507@egenix.com> <441F1A4D.8010509@v.loewis.de> <441F2445.6040504@egenix.com> Message-ID: <441F3141.4000301@v.loewis.de> M.-A. Lemburg wrote: > It's not a waste of time at all: you'd be helping lots and > lots of developers out there who want to fix their extensions. This is free software, anybody is free to decide what they do. I don't believe that developers would be helped a lot - they can easily search for Py_ssize_t in the header files, and find all the APIs that have changed. However, they *should* not have to do that. Instead, they should look at it from a conceptual point of view: Does that variable "count" something (memory, number of elements, size of some structure). If it does, and it currently counts that using an int, it should be changed to use a Py_ssize_t instead. So just review all occurrences of int in your code, and you are done. No need to look at API lists. > The ssize_t patch is the single most disruptive patch in > Python 2.5, so it deserves special attention. I can believe you that you would have preferred not to see that patch at all, not at this time, and preferably never. I have a different view. I don't see it as a problem, but as a solution. Again, if you think the documentation should be improved, go ahead and improve it. Regards, Martin From buildbot at python.org Mon Mar 20 22:23:17 2006 From: buildbot at python.org (buildbot at python.org) Date: Mon, 20 Mar 2006 21:23:17 +0000 Subject: [Python-Dev] buildbot failure in sparc solaris10 gcc trunk Message-ID: <20060320212317.B1CCA1E4009@bag.python.org> The Buildbot has detected a new failure of sparc solaris10 gcc trunk. Full details are available at: http://www.python.org/dev/buildbot/sparc%20solaris10%20gcc%20trunk/builds/85 Buildbot URL: http://www.python.org/dev/buildbot/ Build Reason: The web-page 'rebuild' button was pressed by 'Martin von Loewis': Trigger a build failure Build Source Stamp: [branch trunk] HEAD Blamelist: raymond.hettinger BUILD FAILED: failed compile sincerely, -The Buildbot From martin at v.loewis.de Tue Mar 21 00:24:44 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 21 Mar 2006 00:24:44 +0100 Subject: [Python-Dev] r43041 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: References: Message-ID: <441F39BC.4050805@v.loewis.de> Jim Jewett wrote: >> It is possible to look at the changed APIs, see > >> http://docs.python.org/dev/api/sequence.html > > I think the problem is more with APIs that are not part of the > sequence page. For instance, in > > http://mail.python.org/pipermail/python-dev/2006-March/062679.html > > M.-A. Lemburg mentioned > > PyAPI_FUNC(int) PyDict_Next( > PyObject *mp, Py_ssize_t *pos, PyObject **key, PyObject **value); > > PyDict_Next isn't on the sequence page, but if you call it, it could > write a Py_ssize_t into pos, even though your pointer is only > large enough for an int. I gave the sequence page just as an example. The new signature of PyDict_Next is also documented, see http://docs.python.org/dev/api/dictObjects.html#l2h-629 > To do things correctly, you really have to change your call to the > new version (and add the #ifdef) > > #if PY_VERSION_HEX < 0x02050000 > typedef int Py_ssize_t; > #endif > > You can downcast if you aren't ready to support 64-bits, but ... setting > up a possible buffer overflow is a bit worse than simply not supporting. Sure. In that case, the compiler will give you a compiler warning, on a 64-bit system. You certainly should react to that warning. I personally doubt that somebody would go through the list of the 100-something APIs that changed, memorize them, and then go to his source code, to find out what to change; I know that I don't work that way. That said, if somebody things it is worthwhile to create such a list: contributions are welcome. Regards, Martin From tim.peters at gmail.com Tue Mar 21 01:10:41 2006 From: tim.peters at gmail.com (Tim Peters) Date: Mon, 20 Mar 2006 19:10:41 -0500 Subject: [Python-Dev] All green! In-Reply-To: <1f7befae0603171924w37d1ce21o3f118dcaa6ea9885@mail.gmail.com> References: <1f7befae0603171924w37d1ce21o3f118dcaa6ea9885@mail.gmail.com> Message-ID: <1f7befae0603201610k7bc04d30kdcdc83b778418fe5@mail.gmail.com> [Tim, last Friday] > ... all 14 columns in > > http://www.python.org/dev/buildbot/all/ > > are green. Yippee! Let's keep 'em that way. You can all take the > weekend off ;-) But you didn't, and they haven't been all green again since then . With the next 2.4 release candidate imminent, remember that the buildbot isn't testing release-mode builds, or -O runs. There are 8 combinations you can make out of: debug vs release -O or not with/without deleting .py[co] files first I'll run all 8 on WinXP Pro. It would be prudent for someone to do the same on at least one flavor of Linux. From buildbot at python.org Tue Mar 21 01:12:06 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 21 Mar 2006 00:12:06 +0000 Subject: [Python-Dev] buildbot warnings in x86 gentoo trunk Message-ID: <20060321001207.98BAB1E403C@bag.python.org> The Buildbot has detected a new failure of x86 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/x86%20gentoo%20trunk/builds/95 Buildbot URL: http://www.python.org/dev/buildbot/ Build Reason: The web-page 'force build' button was pressed by 'Tim': someone forgot to say "trunk" last time a build was forced Build Source Stamp: [branch trunk] HEAD Blamelist: Build Had Warnings: warnings test sincerely, -The Buildbot From greg.ewing at canterbury.ac.nz Tue Mar 21 02:12:43 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 21 Mar 2006 13:12:43 +1200 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: References: <4418CDD0.8020803@canterbury.ac.nz> <17434.61837.491103.995641@montanaro.dyndns.org> <441B89EE.8000303@canterbury.ac.nz> <4c45c1530603181131s58b3d598i@mail.gmail.com> <441CB8C5.3000403@canterbury.ac.nz> <17436.54341.20232.333509@montanaro.dyndns.org> Message-ID: <441F530B.8060105@canterbury.ac.nz> Guido van Rossum wrote: > In the style guide only, I hope. The parens that are mandatory in a > few places *are* necessary to resolve ambiguity. That's not entirely true. My original list comprehension implementation didn't require parens around unpacked loop variables -- Guido added them because he thought it looked better that way. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Tue Mar 21 02:16:43 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 21 Mar 2006 13:16:43 +1200 Subject: [Python-Dev] Python 3000 Process In-Reply-To: References: Message-ID: <441F53FB.5040605@canterbury.ac.nz> Guido van Rossum wrote: > respond with a +1 > or -1 on the creation of the python-3000 mailing list. +1 -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Tue Mar 21 02:20:53 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 21 Mar 2006 13:20:53 +1200 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: <200603200600.03110.gmccaughan@synaptics-uk.com> References: <4418CDD0.8020803@canterbury.ac.nz> <50372CD0-B314-4FED-8C57-3F563F459E09@gmail.com> <200603200600.03110.gmccaughan@synaptics-uk.com> Message-ID: <441F54F5.3000004@canterbury.ac.nz> Gareth McCaughan wrote: > but wouldn't if be nice if you could say > > def f((x0,y0) as p0, (x1,y1) as p1): I'm not sure that it would. Currently you can look at a function header and get a picture of its calling signature, but this would clutter it up with implementation details of the function that aren't relevant to the caller. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Tue Mar 21 02:55:06 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 21 Mar 2006 13:55:06 +1200 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <1142869144.10599.64.camel@resist.wooz.org> References: <441D08E9.10503@gmail.com> <1142869144.10599.64.camel@resist.wooz.org> Message-ID: <441F5CFA.8090704@canterbury.ac.nz> Barry Warsaw wrote: > One quibble. Since the term used for the general concept of something > that is raised and caught is "exception" and since all the raise-able > objects live in a module called "exceptions", it is confusing that > "except Exception" will not catch all exceptions. My thoughts exactly. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Tue Mar 21 02:58:50 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 21 Mar 2006 13:58:50 +1200 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <1142869410.10598.69.camel@resist.wooz.org> References: <441C02C8.5080504@iinet.net.au> <1142695605.32074.49.camel@geddy.wooz.org> <441CC358.2020305@gmail.com> <002e01c64b4a$5a3f2a70$88472597@bagio> <441E173E.8050109@canterbury.ac.nz> <1142869410.10598.69.camel@resist.wooz.org> Message-ID: <441F5DDA.4090708@canterbury.ac.nz> Barry Warsaw wrote: > Ideally, StandardError would be called > Error ... Their non-error exceptions > would be derived from Exception. Having something called StandardError suggests that there are "non-standard errors" around somewhere. Are we to have Error Police going around making sure everyone's errors are standardised? :=) Also, "standard error" sounds like some sort of statistical term to me... -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Tue Mar 21 03:05:03 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 21 Mar 2006 14:05:03 +1200 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <20060320161833.GA8328@panix.com> References: <441C02C8.5080504@iinet.net.au> <1142695605.32074.49.camel@geddy.wooz.org> <441C2C3A.8070003@strakt.com> <1142697104.32089.58.camel@geddy.wooz.org> <441CB8A1.8040205@canterbury.ac.nz> <20060320161833.GA8328@panix.com> Message-ID: <441F5F4F.7050309@canterbury.ac.nz> Aahz wrote: > > Also maybe start issuing warnings whenever you inherit directly from > > Exception. > > -1 -- I occasionally use exceptions as a multi-loop break. That's a > perfectly valid Python practice, those exceptions should inherit from > Exception, and there should not be any warnings raised. There probably should be a ControlFlowException category for these that would also include StopIteration and GeneratorExit. I don't think it should include *all* exceptions other than Errors or Warnings, though. SystemExit and KeyboardInterrupt remain two things that you will almost always not want to catch, even in a top-level catch-almost-everything loop. So I'd leave these two out on their own. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From tim.peters at gmail.com Tue Mar 21 03:38:11 2006 From: tim.peters at gmail.com (Tim Peters) Date: Mon, 20 Mar 2006 21:38:11 -0500 Subject: [Python-Dev] segfaults on Mac (was Re: Long-time shy failure in test_socket_ssl)) Message-ID: <1f7befae0603201838w5168e8c4gbe65e95dafdfcad8@mail.gmail.com> [Nick Coghlan] > Hrrm, test_thread_state hands an object pointer off to a different thread > without INCREF'ing it first. Could there be a race condition with the > deallocation of arguments at the end of test_thread_state? > > Specifically, that trace looks to me like the spawned thread is trying to call > the function passed in while the main thread is trying to delete the argument > tuple passed to test_thread_state. Good eye! I haven't been able to _provoke_ that failure on Windows yet, but it sure looks promising. Compounding the problem is that the thread tests in test_capi.py are set up incorrectly: due to "hidden" fighting over Python's import lock, the second TestThreadState here (the one run in the new thread) can't really start before test_capi.py (when run via regrtest.py) finishes: if have_thread_state: TestThreadState() import threading t=threading.Thread(target=TestThreadState) THIS ONE t.start() The dramatic way to show that is to add "t.join()". test_capi.py deadlocks then (not a problem with t.join(), the problem is that TestThreadState is hung trying to do its imports, which it can't do before regrtest.py's import of test_capi.py completes, because of the hidden import lock; so the thread never makes progress, and t.join() waits forever). I can fix that easily (and will), but I want to provoke "a problem" on my box first. The consequence is that the instance of TestThreadState run in the thread ends up running in parallel with later tests run by regrtest.py. Therefore it's not surprising to see a segfault due to test_thread_state occur a few tests after regrtest.py believes test_capi has completed. Later: by adding strategic little counters and conditional sleeps all over the place in the C code, I was able to reproduce a segfault on Windows. I'm not yet sure how to fix it (repairing the structure of test_capi.py would make it more likely that the segfault would occur closer to the time test_capi runs, but the segfault actually occurs in a non-threading.py thread spawned by the _C_ test code, and there's nothing we can do in test_capi.py to wait for that thread to finish). From rod at nabble.com Tue Mar 21 04:26:37 2006 From: rod at nabble.com (R Morris) Date: Mon, 20 Mar 2006 19:26:37 -0800 (PST) Subject: [Python-Dev] Python 3000 Process In-Reply-To: <1142886176.10598.111.camel@resist.wooz.org> References: <740c3aec0603200153l6f42dc70yeda46129b6d5495d@mail.gmail.com> <1142886176.10598.111.camel@resist.wooz.org> Message-ID: <3506703.post@talk.nabble.com> Barry Warsaw wrote: > > On Mon, 2006-03-20 at 13:30 -0500, Terry Reedy wrote: >> "Guido van Rossum" wrote in message >> >Barry, if you could create that mailing list, please? >> >> And please mirror it on gmane the same way as this list is. > > Subscription requests have been made. > -Barry > BTW: It's already archived here and searchable back to the end of August: http://www.nabble.com/Python---python-dev-f2960.html Sorry for the intrustion. Regards, Rod Morris Nabble.com -- View this message in context: http://www.nabble.com/Python-3000-Process-t1309231.html#a3506703 Sent from the Python - python-dev forum at Nabble.com. From guido at python.org Tue Mar 21 05:06:28 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 20 Mar 2006 20:06:28 -0800 Subject: [Python-Dev] Python 3000 Process In-Reply-To: References: Message-ID: The Python 3000 mailing list has been created. From now on let's have all discussions related to Python 3000 on that list. Please subscribe here: http://mail.python.org/mailman/listinfo/python-3000 Let's start by attempting to answer the process questions: timeline; upgrade path; library reform; implementation strategy; and feature proposal process. I'll be away tomorrow so there's plenty of opportunity to start without me. :-) There's also a list for Python-3000-related checkins. Don't subscribe to that one yet, it's very boring and uninteresting. Also, Barry hasn't done the svn magic yet. :-) BTW please be patient with inquiries about Python 3000 on python-dev. While *proposals* of new features etc. can safely be referred to the python-3000 list, *questions* on python-dev about Python 3000 (e.g. "when will it be released" or "will I have to rewrite all my code" or even "what's the fate of existing feature X") are best answered, initially directly on python-dev, eventually by referring to the appropriate PEPs once they have been written (and are on-line -- a sore point right now since the new website has broken the process for updating the on-line copies of PEPs automatically upon checkin into svn). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jimjjewett at gmail.com Tue Mar 21 05:19:03 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 20 Mar 2006 23:19:03 -0500 Subject: [Python-Dev] r43041 - python/trunk/Modules/_ctypes/cfield.c In-Reply-To: <441F39BC.4050805@v.loewis.de> References: <441F39BC.4050805@v.loewis.de> Message-ID: On 3/20/06, "Martin v. L?wis" wrote: > Jim Jewett wrote: > > To do things correctly, you really have to change > > your call to the new version (and add the #ifdef) > > #if PY_VERSION_HEX < 0x02050000 > > typedef int Py_ssize_t; > > #endif > > You can downcast if you aren't ready to > > support 64-bits, but ... setting up a possible > > buffer overflow is a bit worse than simply > > not supporting. > Sure. In that case, the compiler will give you a > compiler warning, on a 64-bit system. You > certainly should react to that warning. The person compiling on a 64-bit system may not be the developer. The buildbot was started in part because the Windows build didn't work -- many developers didn't have a windows environment to test with. This change will put 64 bit systems in the same awkward position -- they need different code, which many developers can't really test properly. > I personally doubt that somebody would go > through the list of the 100-something APIs > that changed, memorize them, and then go > to his source code, to find out what to change; Agreed; they would be more likely to run a grep for a one-time conversion. Catching the error as it creeps back in later would be ... something people learn the hard way. But not having that list for a grep makes the situation even worse. -jJ From jimjjewett at gmail.com Tue Mar 21 05:25:45 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 20 Mar 2006 23:25:45 -0500 Subject: [Python-Dev] Problem with module loading on multi-arch? Message-ID: > x86_64 is multiarch. That means, we allow both > i386 and x86_64 binaries to coexits. Is the proposal > that python should not support this? That would > be unfortunate. The problem is that at the moment, a package has to live all in one place. This doesn't mean you can't have both types of binaries for python as a whole. (Though whether you can run them in the same process is another issue.) It does mean that you can't have them in the same package. This is already awkward for packages split between python and extension modules. As a workaround, you can add a .pth file; eggs and namespace packages may improve the situation some more. -jJ From jcarlson at uci.edu Tue Mar 21 09:38:01 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 21 Mar 2006 00:38:01 -0800 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <20060320114328.2wldxhwo7fs48gco@login.werra.lunarpages.com> References: <20060320114328.2wldxhwo7fs48gco@login.werra.lunarpages.com> Message-ID: <20060321003544.F794.JCARLSON@uci.edu> Michael Chermside wrote: > > Barry writes: > > I still believe in this, and I'm thankful for the support I've seen. It > > won't happen for Python 2.x, but I do plan on addressing this for Py3K. > > When you do, I'd like you to consider one change to the names. You are > proposing this: > > > Exception > > +- KeyboardInterrupt > > +- GeneratorExit > > +- SystemExit > > +- StopIteration > > +- Error > > | +- ImportError > > | +- (etc.) > [...] > > If we use that structure, I still strongly feel that what you call "Error" > ought to be named "Exception" (and what you call "Exception" should be > named "BaseException" or perhaps "Raisable"). The reason is this: [snip some good reasons] +1 for those good reasons and others. - Josiah From buildbot at python.org Tue Mar 21 10:10:54 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 21 Mar 2006 09:10:54 +0000 Subject: [Python-Dev] buildbot warnings in x86 gentoo trunk Message-ID: <20060321091054.3E3A51E4009@bag.python.org> The Buildbot has detected a new failure of x86 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%20gentoo%20trunk/builds/97 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl Build Had Warnings: warnings test sincerely, -The Buildbot From mwh at python.net Tue Mar 21 10:52:25 2006 From: mwh at python.net (Michael Hudson) Date: Tue, 21 Mar 2006 09:52:25 +0000 Subject: [Python-Dev] Documenting the ssize_t Python C API changes In-Reply-To: <441F2445.6040504@egenix.com> (M.'s message of "Mon, 20 Mar 2006 22:53:09 +0100") References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> <441B20C5.1080009@egenix.com> <441B354D.1030906@v.loewis.de> <441C874A.1020305@v.loewis.de> <441F02B3.9030507@egenix.com> <441F1A4D.8010509@v.loewis.de> <441F2445.6040504@egenix.com> Message-ID: <2mek0wxgyu.fsf@starship.python.net> "M.-A. Lemburg" writes: > The ssize_t patch is the single most disruptive patch in > Python 2.5, so it deserves special attention. >From your POV, maybe: from mine, it's definitely the new compiler. Cheers, mwh -- I reject that approach. It has a suspicious lack of internet. -- from Twisted.Quotes From fredrik at pythonware.com Tue Mar 21 11:34:07 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue, 21 Mar 2006 11:34:07 +0100 Subject: [Python-Dev] Documenting the ssize_t Python C API changes References: <20060315084115.90E641E4078@bag.python.org><4417D654.3030600@v.loewis.de><1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com><4417DAB7.7030308@v.loewis.de><1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de><4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com><4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com><441AFF55.2030209@v.loewis.de> <441B20C5.1080009@egenix.com><441B354D.1030906@v.loewis.de> <441C874A.1020305@v.loewis.de> <441F02B3.9030507@egenix.com><441F1A4D.8010509@v.loewis.de> <441F2445.6040504@egenix.com> <2mek0wxgyu.fsf@starship.python.net> Message-ID: Michael Hudson wrote: > > The ssize_t patch is the single most disruptive patch in > > Python 2.5, so it deserves special attention. > > From your POV, maybe: from mine, it's definitely the new compiler. in what way does the new compiler affect third-party developers ? From mwh at python.net Tue Mar 21 11:59:10 2006 From: mwh at python.net (Michael Hudson) Date: Tue, 21 Mar 2006 10:59:10 +0000 Subject: [Python-Dev] Documenting the ssize_t Python C API changes In-Reply-To: (Fredrik Lundh's message of "Tue, 21 Mar 2006 11:34:07 +0100") References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> <441B20C5.1080009@egenix.com> <441B354D.1030906@v.loewis.de> <441C874A.1020305@v.loewis.de> <441F02B3.9030507@egenix.com> <441F1A4D.8010509@v.loewis.de> <441F2445.6040504@egenix.com> <2mek0wxgyu.fsf@starship.python.net> Message-ID: <2macbkxdvl.fsf@starship.python.net> "Fredrik Lundh" writes: > Michael Hudson wrote: > >> > The ssize_t patch is the single most disruptive patch in >> > Python 2.5, so it deserves special attention. >> >> From your POV, maybe: from mine, it's definitely the new compiler. > > in what way does the new compiler affect third-party developers ? It has subtle bugs in it. I guess when it's working it has little effect and in that way it's less disruptive than the ssize_t branch. Cheers, mwh -- today's lesson don't strace X in an xterm -- from Twisted.Quotes From mal at egenix.com Tue Mar 21 12:26:41 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 21 Mar 2006 12:26:41 +0100 Subject: [Python-Dev] Documenting the ssize_t Python C API changes In-Reply-To: <441F3141.4000301@v.loewis.de> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> <441B20C5.1080009@egenix.com> <441B354D.1030906@v.loewis.de> <441C874A.1020305@v.loewis.de> <441F02B3.9030507@egenix.com> <441F1A4D.8010509@v.loewis.de> <441F2445.6040504@egenix.com> <441F3141.4000301@v.loewis.de> Message-ID: <441FE2F1.5050307@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg wrote: >> It's not a waste of time at all: you'd be helping lots and >> lots of developers out there who want to fix their extensions. > > This is free software, anybody is free to decide what they do. With due respect for other developers, yes. > I don't believe that developers would be helped a lot - they > can easily search for Py_ssize_t in the header files, and > find all the APIs that have changed. Of course they can. We could also stop writing documentation and tell users to read the code instead - after all, it's all there, ready to be consumed by interested parties. Oh, and for changes: we'll just point them to Subversion and tell them to run a 'svn diff'. > However, they *should* not have to do that. Instead, they > should look at it from a conceptual point of view: Does > that variable "count" something (memory, number of elements, > size of some structure). If it does, and it currently counts > that using an int, it should be changed to use a Py_ssize_t > instead. > > So just review all occurrences of int in your code, and you > are done. No need to look at API lists. Just did a grep on the mx Extensions: 17000 cases of 'int' being used. Sounds like a nice weekend activity... Seriously, your suggestion on how to port the extensions to Py_ssize_t is certainly true, but this may not be what all extension authors would want to do (or at least not right away). Instead, they'll want to know what changed and then check their code for uses of the changed APIs, in particular those APIs where output parameters are used. I think that documenting these changes is part of doing responsible development. You seem to disagree. >> The ssize_t patch is the single most disruptive patch in >> Python 2.5, so it deserves special attention. > > I can believe you that you would have preferred not to see > that patch at all, not at this time, and preferably never. > I have a different view. I don't see it as a problem, but > as a solution. You are right in that I would have rather seen this change go into Py3k than into the 2.x series. You're wrong in saying that I would have preferred not to get such a change into Python at all. I've given up believing that there would be a possibility of having code that works in both Py3k and Py2.x. I've also given up, believing that code written for Py2.x will continue to work in Py3k. I still holding on to the belief that the 2.x will not introduce major breakage between the versions and that there'll always be some way to write software that works in 2.n and 2.n+1 for any n. However, I feel that at least some Python developers seem to be OK with breaking this possibility, ignoring all the existing working code that's out there. > Again, if you think the documentation should be improved, > go ahead and improve it. Here's a grep of all the changed/new APIs, please include it in the PEP. ./dictobject.h: -- PyAPI_FUNC(int) PyDict_Next( -- PyObject *mp, Py_ssize_t *pos, PyObject **key, PyObject **value); -- PyAPI_FUNC(Py_ssize_t) PyDict_Size(PyObject *mp); ./pyerrors.h: -- PyAPI_FUNC(PyObject *) PyUnicodeDecodeError_Create( -- const char *, const char *, Py_ssize_t, Py_ssize_t, Py_ssize_t, const char *); -- PyAPI_FUNC(PyObject *) PyUnicodeEncodeError_Create( -- const char *, const Py_UNICODE *, Py_ssize_t, Py_ssize_t, Py_ssize_t, const char *); -- PyAPI_FUNC(PyObject *) PyUnicodeTranslateError_Create( -- const Py_UNICODE *, Py_ssize_t, Py_ssize_t, Py_ssize_t, const char *); -- PyAPI_FUNC(int) PyUnicodeEncodeError_GetStart(PyObject *, Py_ssize_t *); -- PyAPI_FUNC(int) PyUnicodeDecodeError_GetStart(PyObject *, Py_ssize_t *); -- PyAPI_FUNC(int) PyUnicodeTranslateError_GetStart(PyObject *, Py_ssize_t *); -- PyAPI_FUNC(int) PyUnicodeEncodeError_SetStart(PyObject *, Py_ssize_t); -- PyAPI_FUNC(int) PyUnicodeDecodeError_SetStart(PyObject *, Py_ssize_t); -- PyAPI_FUNC(int) PyUnicodeTranslateError_SetStart(PyObject *, Py_ssize_t); -- PyAPI_FUNC(int) PyUnicodeEncodeError_GetEnd(PyObject *, Py_ssize_t *); -- PyAPI_FUNC(int) PyUnicodeDecodeError_GetEnd(PyObject *, Py_ssize_t *); -- PyAPI_FUNC(int) PyUnicodeTranslateError_GetEnd(PyObject *, Py_ssize_t *); -- PyAPI_FUNC(int) PyUnicodeEncodeError_SetEnd(PyObject *, Py_ssize_t); -- PyAPI_FUNC(int) PyUnicodeDecodeError_SetEnd(PyObject *, Py_ssize_t); -- PyAPI_FUNC(int) PyUnicodeTranslateError_SetEnd(PyObject *, Py_ssize_t); ./tupleobject.h: -- PyAPI_FUNC(PyObject *) PyTuple_New(Py_ssize_t size); -- PyAPI_FUNC(Py_ssize_t) PyTuple_Size(PyObject *); -- PyAPI_FUNC(PyObject *) PyTuple_GetItem(PyObject *, Py_ssize_t); -- PyAPI_FUNC(int) PyTuple_SetItem(PyObject *, Py_ssize_t, PyObject *); -- PyAPI_FUNC(PyObject *) PyTuple_GetSlice(PyObject *, Py_ssize_t, Py_ssize_t); -- PyAPI_FUNC(int) _PyTuple_Resize(PyObject **, Py_ssize_t); -- PyAPI_FUNC(PyObject *) PyTuple_Pack(Py_ssize_t, ...); ./sliceobject.h: -- PyAPI_FUNC(int) PySlice_GetIndices(PySliceObject *r, Py_ssize_t length, -- Py_ssize_t *start, Py_ssize_t *stop, Py_ssize_t *step); -- PyAPI_FUNC(int) PySlice_GetIndicesEx(PySliceObject *r, Py_ssize_t length, -- Py_ssize_t *start, Py_ssize_t *stop, -- Py_ssize_t *step, Py_ssize_t *slicelength); ./bufferobject.h: -- PyAPI_FUNC(PyObject *) PyBuffer_FromObject(PyObject *base, -- Py_ssize_t offset, Py_ssize_t size); -- PyAPI_FUNC(PyObject *) PyBuffer_FromReadWriteObject(PyObject *base, -- Py_ssize_t offset, -- Py_ssize_t size); -- PyAPI_FUNC(PyObject *) PyBuffer_FromMemory(void *ptr, Py_ssize_t size); -- PyAPI_FUNC(PyObject *) PyBuffer_FromReadWriteMemory(void *ptr, Py_ssize_t size); -- PyAPI_FUNC(PyObject *) PyBuffer_New(Py_ssize_t size); ./marshal.h: -- PyAPI_FUNC(PyObject *) PyMarshal_ReadObjectFromString(char *, Py_ssize_t); ./stringobject.h: -- PyAPI_FUNC(PyObject *) PyString_FromStringAndSize(const char *, Py_ssize_t); -- PyAPI_FUNC(Py_ssize_t) PyString_Size(PyObject *); -- PyAPI_FUNC(int) _PyString_Resize(PyObject **, Py_ssize_t); -- PyAPI_FUNC(PyObject *) PyString_DecodeEscape(const char *, Py_ssize_t, -- const char *, Py_ssize_t, -- const char *); -- PyAPI_FUNC(PyObject*) PyString_Decode( -- const char *s, /* encoded string */ -- Py_ssize_t size, /* size of buffer */ -- const char *encoding, /* encoding */ -- const char *errors /* error handling */ -- ); -- PyAPI_FUNC(PyObject*) PyString_Encode( -- const char *s, /* string char buffer */ -- Py_ssize_t size, /* number of chars to encode */ -- const char *encoding, /* encoding */ -- const char *errors /* error handling */ -- ); -- PyAPI_FUNC(int) PyString_AsStringAndSize( -- register PyObject *obj, /* string or Unicode object */ -- register char **s, /* pointer to buffer variable */ -- register Py_ssize_t *len /* pointer to length variable or NULL -- (only possible for 0-terminated -- strings) */ -- ); ./longobject.h: -- PyAPI_FUNC(Py_ssize_t) _PyLong_AsSsize_t(PyObject *); -- PyAPI_FUNC(PyObject *) _PyLong_FromSsize_t(Py_ssize_t); -- PyAPI_FUNC(PyObject *) PyLong_FromUnicode(Py_UNICODE*, Py_ssize_t, int); ./object.h: -- PyAPI_FUNC(PyObject *) PyType_GenericAlloc(PyTypeObject *, Py_ssize_t); ./intobject.h: -- PyAPI_FUNC(PyObject *) PyInt_FromUnicode(Py_UNICODE*, Py_ssize_t, int); -- PyAPI_FUNC(PyObject *) PyInt_FromSsize_t(Py_ssize_t); -- PyAPI_FUNC(Py_ssize_t) PyInt_AsSsize_t(PyObject *); ./objimpl.h: -- PyAPI_FUNC(PyVarObject *) PyObject_InitVar(PyVarObject *, -- PyTypeObject *, Py_ssize_t); -- PyAPI_FUNC(PyVarObject *) _PyObject_NewVar(PyTypeObject *, Py_ssize_t); -- PyAPI_FUNC(PyVarObject *) _PyObject_GC_Resize(PyVarObject *, Py_ssize_t); -- PyAPI_FUNC(PyVarObject *) _PyObject_GC_NewVar(PyTypeObject *, Py_ssize_t); ./abstract.h: -- PyAPI_FUNC(Py_ssize_t) PyObject_Size(PyObject *o); -- PyAPI_FUNC(Py_ssize_t) PyObject_Length(PyObject *o); -- PyAPI_FUNC(Py_ssize_t) _PyObject_LengthHint(PyObject *o); -- PyAPI_FUNC(int) PyObject_AsCharBuffer(PyObject *obj, -- const char **buffer, -- Py_ssize_t *buffer_len); -- PyAPI_FUNC(int) PyObject_AsReadBuffer(PyObject *obj, -- const void **buffer, -- Py_ssize_t *buffer_len); -- PyAPI_FUNC(int) PyObject_AsWriteBuffer(PyObject *obj, -- void **buffer, -- Py_ssize_t *buffer_len); -- PyAPI_FUNC(Py_ssize_t) PySequence_Size(PyObject *o); -- PyAPI_FUNC(Py_ssize_t) PySequence_Length(PyObject *o); -- PyAPI_FUNC(PyObject *) PySequence_Repeat(PyObject *o, Py_ssize_t count); -- PyAPI_FUNC(PyObject *) PySequence_GetItem(PyObject *o, Py_ssize_t i); -- PyAPI_FUNC(PyObject *) PySequence_GetSlice(PyObject *o, Py_ssize_t i1, Py_ssize_t i2); -- PyAPI_FUNC(int) PySequence_SetItem(PyObject *o, Py_ssize_t i, PyObject *v); -- PyAPI_FUNC(int) PySequence_DelItem(PyObject *o, Py_ssize_t i); -- PyAPI_FUNC(int) PySequence_SetSlice(PyObject *o, Py_ssize_t i1, Py_ssize_t i2, -- PyObject *v); -- PyAPI_FUNC(int) PySequence_DelSlice(PyObject *o, Py_ssize_t i1, Py_ssize_t i2); -- PyAPI_FUNC(PyObject *) PySequence_InPlaceRepeat(PyObject *o, Py_ssize_t count); -- PyAPI_FUNC(Py_ssize_t) PyMapping_Size(PyObject *o); -- PyAPI_FUNC(Py_ssize_t) PyMapping_Length(PyObject *o); ./unicodeobject.h: -- PyAPI_FUNC(PyObject*) PyUnicode_FromUnicode( -- const Py_UNICODE *u, /* Unicode buffer */ -- Py_ssize_t size /* size of buffer */ -- ); -- PyAPI_FUNC(Py_ssize_t) PyUnicode_GetSize( -- PyObject *unicode /* Unicode object */ -- ); -- PyAPI_FUNC(int) PyUnicode_Resize( -- PyObject **unicode, /* Pointer to the Unicode object */ -- Py_ssize_t length /* New length */ -- ); -- PyAPI_FUNC(PyObject*) PyUnicode_FromWideChar( -- register const wchar_t *w, /* wchar_t buffer */ -- Py_ssize_t size /* size of buffer */ -- ); -- PyAPI_FUNC(Py_ssize_t) PyUnicode_AsWideChar( -- PyUnicodeObject *unicode, /* Unicode object */ -- register wchar_t *w, /* wchar_t buffer */ -- Py_ssize_t size /* size of buffer */ -- ); -- PyAPI_FUNC(PyObject*) PyUnicode_Decode( -- const char *s, /* encoded string */ -- Py_ssize_t size, /* size of buffer */ -- const char *encoding, /* encoding */ -- const char *errors /* error handling */ -- ); -- PyAPI_FUNC(PyObject*) PyUnicode_Encode( -- const Py_UNICODE *s, /* Unicode char buffer */ -- Py_ssize_t size, /* number of Py_UNICODE chars to encode */ -- const char *encoding, /* encoding */ -- const char *errors /* error handling */ -- ); -- PyAPI_FUNC(PyObject*) PyUnicode_DecodeUTF7( -- const char *string, /* UTF-7 encoded string */ -- Py_ssize_t length, /* size of string */ -- const char *errors /* error handling */ -- ); -- PyAPI_FUNC(PyObject*) PyUnicode_EncodeUTF7( -- const Py_UNICODE *data, /* Unicode char buffer */ -- Py_ssize_t length, /* number of Py_UNICODE chars to encode */ -- int encodeSetO, /* force the encoder to encode characters in -- Set O, as described in RFC2152 */ -- int encodeWhiteSpace, /* force the encoder to encode space, tab, -- carriage return and linefeed characters */ -- const char *errors /* error handling */ -- ); -- PyAPI_FUNC(PyObject*) PyUnicode_DecodeUTF8( -- const char *string, /* UTF-8 encoded string */ -- Py_ssize_t length, /* size of string */ -- const char *errors /* error handling */ -- ); -- PyAPI_FUNC(PyObject*) PyUnicode_DecodeUTF8Stateful( -- const char *string, /* UTF-8 encoded string */ -- Py_ssize_t length, /* size of string */ -- const char *errors, /* error handling */ -- Py_ssize_t *consumed /* bytes consumed */ -- ); -- PyAPI_FUNC(PyObject*) PyUnicode_EncodeUTF8( -- const Py_UNICODE *data, /* Unicode char buffer */ -- Py_ssize_t length, /* number of Py_UNICODE chars to encode */ -- const char *errors /* error handling */ -- ); -- PyAPI_FUNC(PyObject*) PyUnicode_DecodeUTF16( -- const char *string, /* UTF-16 encoded string */ -- Py_ssize_t length, /* size of string */ -- const char *errors, /* error handling */ -- int *byteorder /* pointer to byteorder to use -- 0=native;-1=LE,1=BE; updated on -- PyAPI_FUNC(PyObject*) PyUnicode_DecodeUTF16Stateful( -- const char *string, /* UTF-16 encoded string */ -- Py_ssize_t length, /* size of string */ -- const char *errors, /* error handling */ -- int *byteorder, /* pointer to byteorder to use -- 0=native;-1=LE,1=BE; updated on -- PyAPI_FUNC(PyObject*) PyUnicode_EncodeUTF16( -- const Py_UNICODE *data, /* Unicode char buffer */ -- Py_ssize_t length, /* number of Py_UNICODE chars to encode */ -- const char *errors, /* error handling */ -- int byteorder /* byteorder to use 0=BOM+native;-1=LE,1=BE */ -- PyAPI_FUNC(PyObject*) PyUnicode_DecodeUnicodeEscape( -- const char *string, /* Unicode-Escape encoded string */ -- Py_ssize_t length, /* size of string */ -- const char *errors /* error handling */ -- ); -- PyAPI_FUNC(PyObject*) PyUnicode_EncodeUnicodeEscape( -- const Py_UNICODE *data, /* Unicode char buffer */ -- Py_ssize_t length /* Number of Py_UNICODE chars to encode */ -- ); -- PyAPI_FUNC(PyObject*) PyUnicode_DecodeRawUnicodeEscape( -- const char *string, /* Raw-Unicode-Escape encoded string */ -- Py_ssize_t length, /* size of string */ -- const char *errors /* error handling */ -- ); -- PyAPI_FUNC(PyObject*) PyUnicode_EncodeRawUnicodeEscape( -- const Py_UNICODE *data, /* Unicode char buffer */ -- Py_ssize_t length /* Number of Py_UNICODE chars to encode */ -- ); -- PyAPI_FUNC(PyObject*) PyUnicode_DecodeLatin1( -- const char *string, /* Latin-1 encoded string */ -- Py_ssize_t length, /* size of string */ -- const char *errors /* error handling */ -- ); -- PyAPI_FUNC(PyObject*) PyUnicode_EncodeLatin1( -- const Py_UNICODE *data, /* Unicode char buffer */ -- Py_ssize_t length, /* Number of Py_UNICODE chars to encode */ -- const char *errors /* error handling */ -- ); -- PyAPI_FUNC(PyObject*) PyUnicode_DecodeASCII( -- const char *string, /* ASCII encoded string */ -- Py_ssize_t length, /* size of string */ -- const char *errors /* error handling */ -- ); -- PyAPI_FUNC(PyObject*) PyUnicode_EncodeASCII( -- const Py_UNICODE *data, /* Unicode char buffer */ -- Py_ssize_t length, /* Number of Py_UNICODE chars to encode */ -- const char *errors /* error handling */ -- ); -- PyAPI_FUNC(PyObject*) PyUnicode_DecodeCharmap( -- const char *string, /* Encoded string */ -- Py_ssize_t length, /* size of string */ -- PyObject *mapping, /* character mapping -- (char ordinal -> unicode ordinal) */ -- const char *errors /* error handling */ -- ); -- PyAPI_FUNC(PyObject*) PyUnicode_EncodeCharmap( -- const Py_UNICODE *data, /* Unicode char buffer */ -- Py_ssize_t length, /* Number of Py_UNICODE chars to encode */ -- PyObject *mapping, /* character mapping -- (unicode ordinal -> char ordinal) */ -- const char *errors /* error handling */ -- ); -- PyAPI_FUNC(PyObject *) PyUnicode_TranslateCharmap( -- const Py_UNICODE *data, /* Unicode char buffer */ -- Py_ssize_t length, /* Number of Py_UNICODE chars to encode */ -- PyObject *table, /* Translate table */ -- const char *errors /* error handling */ -- ); -- PyAPI_FUNC(PyObject*) PyUnicode_DecodeMBCS( -- const char *string, /* MBCS encoded string */ -- Py_ssize_t length, /* size of string */ -- const char *errors /* error handling */ -- ); -- PyAPI_FUNC(PyObject*) PyUnicode_EncodeMBCS( -- const Py_UNICODE *data, /* Unicode char buffer */ -- Py_ssize_t length, /* Number of Py_UNICODE chars to encode */ -- const char *errors /* error handling */ -- ); -- PyAPI_FUNC(int) PyUnicode_EncodeDecimal( -- Py_UNICODE *s, /* Unicode buffer */ -- Py_ssize_t length, /* Number of Py_UNICODE chars to encode */ -- char *output, /* Output buffer; must have size >= length */ -- PyAPI_FUNC(PyObject*) PyUnicode_Split( -- PyObject *s, /* String to split */ -- PyObject *sep, /* String separator */ -- Py_ssize_t maxsplit /* Maxsplit count */ -- ); -- PyAPI_FUNC(PyObject*) PyUnicode_RSplit( -- PyObject *s, /* String to split */ -- PyObject *sep, /* String separator */ -- Py_ssize_t maxsplit /* Maxsplit count */ -- ); -- PyAPI_FUNC(Py_ssize_t) PyUnicode_Tailmatch( -- PyObject *str, /* String */ -- PyObject *substr, /* Prefix or Suffix string */ -- Py_ssize_t start, /* Start index */ -- Py_ssize_t end, /* Stop index */ -- int direction /* Tail end: -1 prefix, +1 suffix */ -- ); -- PyAPI_FUNC(Py_ssize_t) PyUnicode_Find( -- PyObject *str, /* String */ -- PyObject *substr, /* Substring to find */ -- Py_ssize_t start, /* Start index */ -- Py_ssize_t end, /* Stop index */ -- int direction /* Find direction: +1 forward, -1 backward */ -- ); -- PyAPI_FUNC(Py_ssize_t) PyUnicode_Count( -- PyObject *str, /* String */ -- PyObject *substr, /* Substring to count */ -- Py_ssize_t start, /* Start index */ -- Py_ssize_t end /* Stop index */ -- ); -- PyAPI_FUNC(PyObject *) PyUnicode_Replace( -- PyObject *str, /* String */ -- PyObject *substr, /* Substring to find */ -- PyObject *replstr, /* Substring to replace */ -- Py_ssize_t maxcount /* Max. number of replacements to apply; ./ceval.h: -- PyAPI_FUNC(int) _PyEval_SliceIndex(PyObject *, Py_ssize_t *); ./listobject.h: -- PyAPI_FUNC(PyObject *) PyList_New(Py_ssize_t size); -- PyAPI_FUNC(Py_ssize_t) PyList_Size(PyObject *); -- PyAPI_FUNC(PyObject *) PyList_GetItem(PyObject *, Py_ssize_t); -- PyAPI_FUNC(int) PyList_SetItem(PyObject *, Py_ssize_t, PyObject *); -- PyAPI_FUNC(int) PyList_Insert(PyObject *, Py_ssize_t, PyObject *); -- PyAPI_FUNC(PyObject *) PyList_GetSlice(PyObject *, Py_ssize_t, Py_ssize_t); -- PyAPI_FUNC(int) PyList_SetSlice(PyObject *, Py_ssize_t, Py_ssize_t, PyObject *); ./longintrepr.h: -- PyAPI_FUNC(PyLongObject *) _PyLong_New(Py_ssize_t); This is an excerpt of just the changes to output parameters causing buffer overflows if not corrected in all extensions using them: ./dictobject.h: -- PyAPI_FUNC(int) PyDict_Next( -- PyObject *mp, Py_ssize_t *pos, PyObject **key, PyObject **value); ./pyerrors.h: -- PyAPI_FUNC(int) PyUnicodeEncodeError_GetStart(PyObject *, Py_ssize_t *); -- PyAPI_FUNC(int) PyUnicodeDecodeError_GetStart(PyObject *, Py_ssize_t *); -- PyAPI_FUNC(int) PyUnicodeTranslateError_GetStart(PyObject *, Py_ssize_t *); -- PyAPI_FUNC(int) PyUnicodeEncodeError_GetEnd(PyObject *, Py_ssize_t *); -- PyAPI_FUNC(int) PyUnicodeDecodeError_GetEnd(PyObject *, Py_ssize_t *); -- PyAPI_FUNC(int) PyUnicodeTranslateError_GetEnd(PyObject *, Py_ssize_t *); ./sliceobject.h: -- PyAPI_FUNC(int) PySlice_GetIndices(PySliceObject *r, Py_ssize_t length, -- Py_ssize_t *start, Py_ssize_t *stop, Py_ssize_t *step); -- PyAPI_FUNC(int) PySlice_GetIndicesEx(PySliceObject *r, Py_ssize_t length, -- Py_ssize_t *start, Py_ssize_t *stop, -- Py_ssize_t *step, Py_ssize_t *slicelength); ./stringobject.h: -- PyAPI_FUNC(int) PyString_AsStringAndSize( -- register PyObject *obj, /* string or Unicode object */ -- register char **s, /* pointer to buffer variable */ -- register Py_ssize_t *len /* pointer to length variable or NULL -- (only possible for 0-terminated -- strings) */ -- ); ./abstract.h: -- PyAPI_FUNC(int) PyObject_AsCharBuffer(PyObject *obj, -- const char **buffer, -- Py_ssize_t *buffer_len); -- PyAPI_FUNC(int) PyObject_AsReadBuffer(PyObject *obj, -- const void **buffer, -- Py_ssize_t *buffer_len); -- PyAPI_FUNC(int) PyObject_AsWriteBuffer(PyObject *obj, -- void **buffer, -- Py_ssize_t *buffer_len); ./unicodeobject.h: -- PyAPI_FUNC(PyObject*) PyUnicode_DecodeUTF8Stateful( -- const char *string, /* UTF-8 encoded string */ -- Py_ssize_t length, /* size of string */ -- const char *errors, /* error handling */ -- Py_ssize_t *consumed /* bytes consumed */ -- ); ./ceval.h: -- PyAPI_FUNC(int) _PyEval_SliceIndex(PyObject *, Py_ssize_t *); Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 21 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From fredrik at pythonware.com Tue Mar 21 13:20:44 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue, 21 Mar 2006 13:20:44 +0100 Subject: [Python-Dev] Documenting the ssize_t Python C API changes References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> <441B20C5.1080009@egenix.com> <441B354D.1030906@v.loewis.de> <441C874A.1020305@v.loewis.de><441F02B3.9030507@egenix.com> <441F1A4D.8010509@v.loewis.de><441F2445.6040504@egenix.com> <441F3141.4000301@v.loewis.de> <441FE2F1.5050307@egenix.com> Message-ID: M.-A. Lemburg wrote: > Here's a grep of all the changed/new APIs, please include it > in the PEP. I've posted a simple-minded source scanner here: http://svn.effbot.python-hosting.com/stuff/sandbox/python/ssizecheck.py From barry at python.org Tue Mar 21 14:34:36 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 21 Mar 2006 08:34:36 -0500 Subject: [Python-Dev] Python 3000 Process In-Reply-To: References: Message-ID: <1142948076.5179.4.camel@geddy.wooz.org> On Mon, 2006-03-20 at 20:06 -0800, Guido van Rossum wrote: > There's also a list for Python-3000-related checkins. Don't subscribe > to that one yet, it's very boring and uninteresting. Also, Barry > hasn't done the svn magic yet. :-) We're having a small problem on the svn box which requires Thomas's intervention, so it might be a little while. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060321/0f2491ff/attachment-0001.pgp From buildbot at python.org Tue Mar 21 14:41:10 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 21 Mar 2006 13:41:10 +0000 Subject: [Python-Dev] buildbot warnings in amd64 gentoo trunk Message-ID: <20060321134110.E81471E4009@bag.python.org> The Buildbot has detected a new failure of amd64 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%20gentoo%20trunk/builds/100 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: martin.v.loewis Build Had Warnings: warnings test sincerely, -The Buildbot From martin at v.loewis.de Tue Mar 21 15:28:35 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 21 Mar 2006 15:28:35 +0100 Subject: [Python-Dev] Documenting the ssize_t Python C API changes In-Reply-To: <441FE2F1.5050307@egenix.com> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> <441B20C5.1080009@egenix.com> <441B354D.1030906@v.loewis.de> <441C874A.1020305@v.loewis.de> <441F02B3.9030507@egenix.com> <441F1A4D.8010509@v.loewis.de> <441F2445.6040504@egenix.com> <441F3141.4000301@v.loewis.de> <441FE2F1.5050307@egenix.com> Message-ID: <44200D93.6020002@v.loewis.de> M.-A. Lemburg wrote: > Here's a grep of all the changed/new APIs, please include it > in the PEP. You want me to include that *literally*? Are you serious? Please go ahead and commit that change yourself: I consider it completely unreadable and entirely worthless. Regards, Martin From mal at egenix.com Tue Mar 21 16:00:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 21 Mar 2006 16:00:24 +0100 Subject: [Python-Dev] Documenting the ssize_t Python C API changes In-Reply-To: <44200D93.6020002@v.loewis.de> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> <441B20C5.1080009@egenix.com> <441B354D.1030906@v.loewis.de> <441C874A.1020305@v.loewis.de> <441F02B3.9030507@egenix.com> <441F1A4D.8010509@v.loewis.de> <441F2445.6040504@egenix.com> <441F3141.4000301@v.loewis.de> <441FE2F1.5050307@egenix.com> <44200D93.6020002@v.loewis.de> Message-ID: <44201508.5060701@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg wrote: >> Here's a grep of all the changed/new APIs, please include it >> in the PEP. > > You want me to include that *literally*? Are you serious? Feel free to format it in a different way. > Please go ahead and commit that change yourself: I consider > it completely unreadable and entirely worthless. Interesting: A few mails ago you suggested that developers do exactly what I did to get the list of changes. Now you gripe about the output format of the grep. I really don't understand what your problem is with documenting the work that you and others have put into this. Why is there so much resistance ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 21 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From martin at v.loewis.de Tue Mar 21 16:11:35 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 21 Mar 2006 16:11:35 +0100 Subject: [Python-Dev] Documenting the ssize_t Python C API changes In-Reply-To: References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> <441B20C5.1080009@egenix.com> <441B354D.1030906@v.loewis.de> <441C874A.1020305@v.loewis.de><441F02B3.9030507@egenix.com> <441F1A4D.8010509@v.loewis.de><441F2445.6040504@egenix.com> <441F3141.4000301@v.loewis.de> <441FE2F1.5050307@egenix.com> Message-ID: <442017A7.6010605@v.loewis.de> Fredrik Lundh wrote: > I've posted a simple-minded source scanner here: > > http://svn.effbot.python-hosting.com/stuff/sandbox/python/ssizecheck.py Great job! I've added a link to that to the PEP. There are two improvements you could make: - Some of the functions in the first list return Py_ssize_t; calling them can cause truncation if the result value is larger than INT_MAX (and it is assigned to an int). To find those functions, do grep 'PyAPI_FUNC(Py_ssize_t)' Include/*.h - Usage of the "old" function pointer type name (e.g. intobjargproc) likely also indicates an error, since many of these pointers have changed there types. I believe the only exception for this is inquiry, which still has legit uses, in nb_nonzero, tp_clear, and tp_is_gc. Regards, Martin From martin at v.loewis.de Tue Mar 21 16:22:21 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 21 Mar 2006 16:22:21 +0100 Subject: [Python-Dev] Documenting the ssize_t Python C API changes In-Reply-To: <44201508.5060701@egenix.com> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> <441B20C5.1080009@egenix.com> <441B354D.1030906@v.loewis.de> <441C874A.1020305@v.loewis.de> <441F02B3.9030507@egenix.com> <441F1A4D.8010509@v.loewis.de> <441F2445.6040504@egenix.com> <441F3141.4000301@v.loewis.de> <441FE2F1.5050307@egenix.com> <44200D93.6020002@v.loewis.de> <44201508.5060701@egenix.com> Message-ID: <44201A2D.90600@v.loewis.de> M.-A. Lemburg wrote: > Interesting: A few mails ago you suggested that developers > do exactly what I did to get the list of changes. Now you > gripe about the output format of the grep. Developers which use grep can read the output of grep. Developers which use other methods of searching (e.g. Visual Studio) can understand the output of these tools. I don't say they *should* search for Py_ssize_t, I said they *can* if they want to. I still don't think anybody would *want* to read such a list. > I really don't understand what your problem is with documenting > the work that you and others have put into this. Why is there > so much resistance ? Because I think it is pointless, confusing, and redundant in a dangerous way. There is absolutely NO problem with API changes where a function consumes Py_ssize_t. Why should anybody care that PyString_FromStringAndSize now consumes a Py_ssize_t? Passing an int works just fine, and if the int fits the length of the string, it will absolutely do the right thing. There is no need to touch that code, or worry about the change. Putting PyString_FromStringAndSize into such a list *will* make developers worry, because they now think they have to change their code somehow, when there is absolutely for action. Furthermore, it is redundant because there are now *four* places where the signature of the API is mentioned: in the header, in the implementation, in the API docs, and in the PEP. The compiler watches for consistency of the first two; consistency of the others is a manual process, and thus error-prone. Regards, Martin From mal at egenix.com Tue Mar 21 17:20:48 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 21 Mar 2006 17:20:48 +0100 Subject: [Python-Dev] Documenting the ssize_t Python C API changes In-Reply-To: References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> <441B20C5.1080009@egenix.com> <441B354D.1030906@v.loewis.de> <441C874A.1020305@v.loewis.de><441F02B3.9030507@egenix.com> <441F1A4D.8010509@v.loewis.de><441F2445.6040504@egenix.com> <441F3141.4000301@v.loewis.de> <441FE2F1.5050307@egenix.com> Message-ID: <442027E0.5010405@egenix.com> Fredrik Lundh wrote: > M.-A. Lemburg wrote: > >> Here's a grep of all the changed/new APIs, please include it >> in the PEP. > > I've posted a simple-minded source scanner here: > > http://svn.effbot.python-hosting.com/stuff/sandbox/python/ssizecheck.py Very nice. Thanks ! -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 21 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Tue Mar 21 18:07:31 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 21 Mar 2006 18:07:31 +0100 Subject: [Python-Dev] Documenting the ssize_t Python C API changes In-Reply-To: <44201A2D.90600@v.loewis.de> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> <441B20C5.1080009@egenix.com> <441B354D.1030906@v.loewis.de> <441C874A.1020305@v.loewis.de> <441F02B3.9030507@egenix.com> <441F1A4D.8010509@v.loewis.de> <441F2445.6040504@egenix.com> <441F3141.4000301@v.loewis.de> <441FE2F1.5050307@egenix.com> <44200D93.6020002@v.loewis.de> <44201508.5060701@egenix.com> <44201A2D.90600@v.loewis.de> Message-ID: <442032D3.8020609@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg wrote: >> Interesting: A few mails ago you suggested that developers >> do exactly what I did to get the list of changes. Now you >> gripe about the output format of the grep. > > Developers which use grep can read the output of grep. Developers > which use other methods of searching (e.g. Visual Studio) can > understand the output of these tools. I don't say they *should* > search for Py_ssize_t, I said they *can* if they want to. > > I still don't think anybody would *want* to read such a list. They don't necessarily want to read it, but like Fredrik did, use as input for their testing tools. >> I really don't understand what your problem is with documenting >> the work that you and others have put into this. Why is there >> so much resistance ? > > Because I think it is pointless, confusing, and redundant in a dangerous > way. There is absolutely NO problem with API changes where a function > consumes Py_ssize_t. Why should anybody care that > PyString_FromStringAndSize now consumes a Py_ssize_t? Passing an int > works just fine, and if the int fits the length of the string, it will > absolutely do the right thing. There is no need to touch that code, > or worry about the change. > > Putting PyString_FromStringAndSize into such a list *will* make > developers worry, because they now think they have to change their code > somehow, when there is absolutely for action. Don't you think developers are capable enough to judge for themselves ? They might also want to change their extensions to make use of the new possibilities, so a list of APIs taking Py_ssize_t parameters on input would be handy to check where there's potential for such a change in their code. Perhaps we should have three lists: 1. Py_ssize_t output parameters (these need changes) 2. Py_ssize_t return values (these need overflow checks) 3. Py_ssize_t input parameters (these can be used to enhance the extension) Here's the list for 2 (I already provided the list for 1 in the other mail): ./dictobject.h: -- PyAPI_FUNC(Py_ssize_t) PyDict_Size(PyObject *mp); ./tupleobject.h: -- PyAPI_FUNC(Py_ssize_t) PyTuple_Size(PyObject *); ./stringobject.h: -- PyAPI_FUNC(Py_ssize_t) PyString_Size(PyObject *); ./longobject.h: -- PyAPI_FUNC(Py_ssize_t) _PyLong_AsSsize_t(PyObject *); ./intobject.h: -- PyAPI_FUNC(Py_ssize_t) PyInt_AsSsize_t(PyObject *); ./abstract.h: -- PyAPI_FUNC(Py_ssize_t) PyObject_Size(PyObject *o); -- PyAPI_FUNC(Py_ssize_t) PyObject_Length(PyObject *o); -- PyAPI_FUNC(Py_ssize_t) _PyObject_LengthHint(PyObject *o); -- PyAPI_FUNC(Py_ssize_t) PySequence_Size(PyObject *o); -- PyAPI_FUNC(Py_ssize_t) PySequence_Length(PyObject *o); -- PyAPI_FUNC(Py_ssize_t) PyMapping_Size(PyObject *o); -- PyAPI_FUNC(Py_ssize_t) PyMapping_Length(PyObject *o); ./unicodeobject.h: -- PyAPI_FUNC(Py_ssize_t) PyUnicode_GetSize( -- PyObject *unicode /* Unicode object */ -- ); -- PyAPI_FUNC(Py_ssize_t) PyUnicode_AsWideChar( -- PyUnicodeObject *unicode, /* Unicode object */ -- register wchar_t *w, /* wchar_t buffer */ -- Py_ssize_t size /* size of buffer */ -- ); -- PyAPI_FUNC(Py_ssize_t) PyUnicode_Tailmatch( -- PyObject *str, /* String */ -- PyObject *substr, /* Prefix or Suffix string */ -- Py_ssize_t start, /* Start index */ -- Py_ssize_t end, /* Stop index */ -- int direction /* Tail end: -1 prefix, +1 suffix */ -- ); -- PyAPI_FUNC(Py_ssize_t) PyUnicode_Find( -- PyObject *str, /* String */ -- PyObject *substr, /* Substring to find */ -- Py_ssize_t start, /* Start index */ -- Py_ssize_t end, /* Stop index */ -- int direction /* Find direction: +1 forward, -1 backward */ -- ); -- PyAPI_FUNC(Py_ssize_t) PyUnicode_Count( -- PyObject *str, /* String */ -- PyObject *substr, /* Substring to count */ -- Py_ssize_t start, /* Start index */ -- Py_ssize_t end /* Stop index */ -- ); ./listobject.h: -- PyAPI_FUNC(Py_ssize_t) PyList_Size(PyObject *); > Furthermore, it is redundant because there are now *four* places where > the signature of the API is mentioned: in the header, in the > implementation, in the API docs, and in the PEP. The compiler watches > for consistency of the first two; consistency of the others is a manual > process, and thus error-prone. It's a one-time operation, documenting the changes between Python 2.4 and 2.5 - much like the grand renaming in the C API a few years ago. You'd only create the lists once in the PEP, namely when Python 2.5 is released. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 21 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From tjreedy at udel.edu Tue Mar 21 18:45:44 2006 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 21 Mar 2006 12:45:44 -0500 Subject: [Python-Dev] Documenting the ssize_t Python C API changes References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> <441B20C5.1080009@egenix.com> <441B354D.1030906@v.loewis.de> <441C874A.1020305@v.loewis.de> <441F02B3.9030507@egenix.com> <441F1A4D.8010509@v.loewis.de> <441F2445.6040504@egenix.com> <441F3141.4000301@v.loewis.de> <441FE2F1.5050307@egenix.com> <44200D93.6020002@v.loewis.de><44201508.5060701@egenix.com> <44201A2D.90600@v.loewis.de> <442032D3.8020609@egenix.com> Message-ID: "M.-A. Lemburg" wrote in message news:442032D3.8020609 at egenix.com... >Perhaps we should have three lists: >1. Py_ssize_t output parameters (these need changes) >2. Py_ssize_t return values (these need overflow checks) >3. Py_ssize_t input parameters (these can be used to enhance the extension) If I were an extension writer in a position to need these, I am sure I would prefer the three lists separated. (Where housed would otherwise be pretty irrelevant.) I know I would strongly prefer to have them be separate plain ascii text files linked to as appendixes rather than incorporated into the PEP itself. I might also like a separate doc (or wiki entry/ies) on updating extension to use Py_ssize_t, with more than one persons experiences, if indeed it it not trivial and straightforward. Terry Jan Reedy From fredrik at pythonware.com Tue Mar 21 18:47:39 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue, 21 Mar 2006 18:47:39 +0100 Subject: [Python-Dev] Documenting the ssize_t Python C API changes References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> <441B20C5.1080009@egenix.com> <441B354D.1030906@v.loewis.de> <441C874A.1020305@v.loewis.de> <441F02B3.9030507@egenix.com> <441F1A4D.8010509@v.loewis.de> <441F2445.6040504@egenix.com> <441F3141.4000301@v.loewis.de> <441FE2F1.5050307@egenix.com> <44200D93.6020002@v.loewis.de><44201508.5060701@egenix.com> <44201A2D.90600@v.loewis.de> <442032D3.8020609@egenix.com> Message-ID: M.-A. Lemburg wrote: > Perhaps we should have three lists: > > 1. Py_ssize_t output parameters (these need changes) > 2. Py_ssize_t return values (these need overflow checks) > 3. Py_ssize_t input parameters (these can be used to enhance > the extension) > > Here's the list for 2 (I already provided the list for 1 in the > other mail): thanks! I've updated the ssizecheck tool. From tim.peters at gmail.com Tue Mar 21 18:57:29 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 21 Mar 2006 12:57:29 -0500 Subject: [Python-Dev] [Python-checkins] r43126 - in python/trunk: Doc/lib/libsocket.tex Lib/socket.py Lib/test/test_socket.py Misc/NEWS Modules/socketmodule.c In-Reply-To: References: <20060317191736.A13151E4006@bag.python.org> <441F36D9.80708@v.loewis.de> Message-ID: <1f7befae0603210957h7ec228b0hc8439344238fdb1b@mail.gmail.com> [attribution lost] >>>> Looking into the library, I see many accessor >>>> functions starting with either get_ or get; however, I don't >>>> recall why that could have been relevant for socket objects. [attribution lost] >>> If you think that should be changed to attributes, we should >>> do that. [Guido] >> Accessor functions are typical for APIs translated too literally from >> Java. (threading.py being an example :-( ) >> >> I'd like to change this as long as we're doing greenfield API design. [Georg Brandl] > Does that mean to change it to attributes? > (since I'm no professional BDFL channeler I could have misread your > sentence ;) Yup, that's what it means. In addition, the "greenfield" part meant specifically they should be changed to read-only attributes. One trick to professional channeling is to do the Pythonic thing, and then just pretend you read Guido's mind. Of course I'm _way_ beyond that :-) From fredrik at pythonware.com Tue Mar 21 18:51:03 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue, 21 Mar 2006 18:51:03 +0100 Subject: [Python-Dev] Documenting the ssize_t Python C API changes References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> <441B20C5.1080009@egenix.com> <441B354D.1030906@v.loewis.de> <441C874A.1020305@v.loewis.de><441F02B3.9030507@egenix.com> <441F1A4D.8010509@v.loewis.de><441F2445.6040504@egenix.com> <441F3141.4000301@v.loewis.de><441FE2F1.5050307@egenix.com> <442017A7.6010605@v.loewis.de> Message-ID: Martin v. Löwis wrote: > There are two improvements you could make: > - Some of the functions in the first list return Py_ssize_t; calling > them can cause truncation if the result value is larger than INT_MAX > (and it is assigned to an int). To find those functions, do > grep 'PyAPI_FUNC(Py_ssize_t)' Include/*.h fixed (based on the list MAL just posted). > - Usage of the "old" function pointer type name (e.g. intobjargproc) > likely also indicates an error, since many of these pointers have > changed there types. I believe the only exception for this is inquiry, > which still has legit uses, in nb_nonzero, tp_clear, and tp_is_gc. is there a complete list of these somewhere ? From jeremy at alum.mit.edu Tue Mar 21 19:06:29 2006 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Tue, 21 Mar 2006 13:06:29 -0500 Subject: [Python-Dev] Documenting the ssize_t Python C API changes In-Reply-To: <442032D3.8020609@egenix.com> References: <20060315084115.90E641E4078@bag.python.org> <441F02B3.9030507@egenix.com> <441F1A4D.8010509@v.loewis.de> <441F2445.6040504@egenix.com> <441F3141.4000301@v.loewis.de> <441FE2F1.5050307@egenix.com> <44200D93.6020002@v.loewis.de> <44201508.5060701@egenix.com> <44201A2D.90600@v.loewis.de> <442032D3.8020609@egenix.com> Message-ID: On 3/21/06, M.-A. Lemburg wrote: > Don't you think developers are capable enough to judge for > themselves ? > > They might also want to change their extensions to make use > of the new possibilities, so a list of APIs taking Py_ssize_t > parameters on input would be handy to check where there's > potential for such a change in their code. > > Perhaps we should have three lists: > > 1. Py_ssize_t output parameters (these need changes) > 2. Py_ssize_t return values (these need overflow checks) > 3. Py_ssize_t input parameters (these can be used to enhance > the extension) It sounds more like the PEP should list these three categories and tell developers that they need to inspect their code for instances of them and fix them in the appropriate way. Perhaps with an example of each. Jeremy From g.brandl at gmx.net Tue Mar 21 19:19:36 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 21 Mar 2006 19:19:36 +0100 Subject: [Python-Dev] [Python-checkins] r43126 - in python/trunk: Doc/lib/libsocket.tex Lib/socket.py Lib/test/test_socket.py Misc/NEWS Modules/socketmodule.c In-Reply-To: <1f7befae0603210957h7ec228b0hc8439344238fdb1b@mail.gmail.com> References: <20060317191736.A13151E4006@bag.python.org> <441F36D9.80708@v.loewis.de> <1f7befae0603210957h7ec228b0hc8439344238fdb1b@mail.gmail.com> Message-ID: Tim Peters wrote: > [Guido] >>> Accessor functions are typical for APIs translated too literally from >>> Java. (threading.py being an example :-( ) >>> >>> I'd like to change this as long as we're doing greenfield API design. > > [Georg Brandl] >> Does that mean to change it to attributes? >> (since I'm no professional BDFL channeler I could have misread your >> sentence ;) > > Yup, that's what it means. In addition, the "greenfield" part meant > specifically they should be changed to read-only attributes. Thank you! Changed in rev. 43196. > One trick to professional channeling is to do the Pythonic thing, and > then just pretend you read Guido's mind. Of course I'm _way_ beyond > that :-) One day, I want to become like you! ;) Cheers, Georg From buildbot at python.org Tue Mar 21 19:44:19 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 21 Mar 2006 18:44:19 +0000 Subject: [Python-Dev] buildbot warnings in x86 XP-2 trunk Message-ID: <20060321184419.C81FF1E4009@bag.python.org> The Buildbot has detected a new failure of x86 XP-2 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%20XP-2%20trunk/builds/83 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: barry.warsaw Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot From buildbot at python.org Tue Mar 21 19:46:14 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 21 Mar 2006 18:46:14 +0000 Subject: [Python-Dev] buildbot failure in x86 XP-2 trunk Message-ID: <20060321184614.F15341E4009@bag.python.org> The Buildbot has detected a new failure of x86 XP-2 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%20XP-2%20trunk/builds/84 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: The web-page 'force build' button was pressed by 'Tim': lost connection Build Source Stamp: [branch trunk] HEAD Blamelist: BUILD FAILED: failed compile sincerely, -The Buildbot From martin at v.loewis.de Tue Mar 21 19:51:27 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 21 Mar 2006 19:51:27 +0100 Subject: [Python-Dev] Documenting the ssize_t Python C API changes In-Reply-To: References: <20060315084115.90E641E4078@bag.python.org> <441F02B3.9030507@egenix.com> <441F1A4D.8010509@v.loewis.de> <441F2445.6040504@egenix.com> <441F3141.4000301@v.loewis.de> <441FE2F1.5050307@egenix.com> <44200D93.6020002@v.loewis.de> <44201508.5060701@egenix.com> <44201A2D.90600@v.loewis.de> <442032D3.8020609@egenix.com> Message-ID: <44204B2F.6070803@v.loewis.de> Jeremy Hylton wrote: > It sounds more like the PEP should list these three categories and > tell developers that they need to inspect their code for instances of > them and fix them in the appropriate way. Perhaps with an example of > each. +1; I will do that. Regards, Martin From buildbot at python.org Tue Mar 21 19:55:20 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 21 Mar 2006 18:55:20 +0000 Subject: [Python-Dev] buildbot warnings in g4 osx.4 trunk Message-ID: <20060321185520.9C41F1E4009@bag.python.org> The Buildbot has detected a new failure of g4 osx.4 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/g4%20osx.4%20trunk/builds/94 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl Build Had Warnings: warnings test sincerely, -The Buildbot From martin at v.loewis.de Tue Mar 21 20:09:25 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 21 Mar 2006 20:09:25 +0100 Subject: [Python-Dev] Documenting the ssize_t Python C API changes In-Reply-To: References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <4419F087.5080207@egenix.com> <4419F2F8.9040900@v.loewis.de> <441A889F.9080305@egenix.com> <441AFF55.2030209@v.loewis.de> <441B20C5.1080009@egenix.com> <441B354D.1030906@v.loewis.de> <441C874A.1020305@v.loewis.de><441F02B3.9030507@egenix.com> <441F1A4D.8010509@v.loewis.de><441F2445.6040504@egenix.com> <441F3141.4000301@v.loewis.de><441FE2F1.5050307@egenix.com> <442017A7.6010605@v.loewis.de> Message-ID: <44204F65.1010400@v.loewis.de> Fredrik Lundh wrote: >> - Usage of the "old" function pointer type name (e.g. intobjargproc) >> likely also indicates an error, since many of these pointers have >> changed there types. I believe the only exception for this is inquiry, >> which still has legit uses, in nb_nonzero, tp_clear, and tp_is_gc. > > is there a complete list of these somewhere ? This is actually in the PEP: the new types are ssizeargfunc ssizessizeargfunc ssizeobjargproc ssizessizeobjargproc lenfunc readbufferproc writebufferproc segcountproc charbufferproc These replace intargfunc intintargfunc intobjargproc intintobjargproc inquiry (partial) getreadbufferproc getwritebufferproc getsegcountproc getcharbufferproc Regards, Martin From andymac at bullseye.apana.org.au Tue Mar 21 14:33:56 2006 From: andymac at bullseye.apana.org.au (Andrew MacIntyre) Date: Tue, 21 Mar 2006 23:33:56 +1000 Subject: [Python-Dev] supported platforms OS2? In-Reply-To: References: Message-ID: <442000C4.5010609@bullseye.apana.org.au> Jim Jewett wrote: > Is OS2 still actively supported? If you mean, is Python still actively supported on OS/2, then the answer is yes. ------------------------------------------------------------------------- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac at bullseye.apana.org.au (pref) | Snail: PO Box 370 andymac at pcug.org.au (alt) | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From andymac at bullseye.apana.org.au Tue Mar 21 14:41:37 2006 From: andymac at bullseye.apana.org.au (Andrew MacIntyre) Date: Tue, 21 Mar 2006 23:41:37 +1000 Subject: [Python-Dev] supported platforms OS2? In-Reply-To: <442000C4.5010609@bullseye.apana.org.au> References: <442000C4.5010609@bullseye.apana.org.au> Message-ID: <44200291.6050006@bullseye.apana.org.au> Andrew MacIntyre wrote: > Jim Jewett wrote: > >> Is OS2 still actively supported? > > If you mean, is Python still actively supported on OS/2, then the answer > is yes. Just to clarify, I haven't been actively monitoring other changes for impact on the OS/2 port for the last few months (too much happening in the rest of my life), but am getting back into the swing of things now that I have a SVN checkout to work from. Barring disaster, there will be a 2.5 release of the OS/2 port. ------------------------------------------------------------------------- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac at bullseye.apana.org.au (pref) | Snail: PO Box 370 andymac at pcug.org.au (alt) | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From barry at python.org Tue Mar 21 20:48:44 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 21 Mar 2006 14:48:44 -0500 Subject: [Python-Dev] svn checkins are now split Message-ID: <1142970524.639.160.camel@resist.wooz.org> I believe I've figured out the magic to split checkin messages so that python/branches/p3yk goes to python-3000-checkins and everything else goes to python-checkins. Let me know if you notice any problems, and feel free to subscribe to whichever list(s) you're interested in. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060321/fc406c03/attachment.pgp From buildbot at python.org Tue Mar 21 21:17:50 2006 From: buildbot at python.org (buildbot at python.org) Date: Tue, 21 Mar 2006 20:17:50 +0000 Subject: [Python-Dev] buildbot warnings in amd64 gentoo trunk Message-ID: <20060321201750.98A611E4009@bag.python.org> The Buildbot has detected a new failure of amd64 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%20gentoo%20trunk/builds/105 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: barry.warsaw Build Had Warnings: warnings test sincerely, -The Buildbot From barry at python.org Wed Mar 22 01:24:39 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 21 Mar 2006 19:24:39 -0500 Subject: [Python-Dev] PySet API In-Reply-To: <00a001c64bfa$71a0ab20$b83efea9@RaymondLaptop1> References: <1142705253.32074.72.camel@geddy.wooz.org> <005c01c64ac5$589859b0$b83efea9@RaymondLaptop1> <002b01c64aeb$44eb7750$b83efea9@RaymondLaptop1> <1142824236.10599.29.camel@resist.wooz.org> <00a001c64bfa$71a0ab20$b83efea9@RaymondLaptop1> Message-ID: <1142987079.10599.232.camel@resist.wooz.org> Is it your intent to push for more use of the abstract API instead of the concrete APIs for all of Python's C data structures? Current API aside, are you advocating this approach for all new built-in types? Would you argue that Python 3.0's C API be stripped of everything but the abstract API and the bare essentials of the concrete API? If so, then I think this is extremely misguided. C is not Python, and while the abstract API is useful for some things, so is the concrete API. In fact, the Python C API's clarity, utility, completeness, and discoverability has made Python one of the nicest languages to embed and extend, and I see no reason to deviate from that for the sake of blind TOOWTDI worship. We have a rich tradition of providing both concrete and abstract APIs at the C layer, and I think that's a good thing that we should continue here. On Mon, 2006-03-20 at 03:44 -0500, Raymond Hettinger wrote: > PySet_Clear() > ------------- > Use PyObject_CallMethod(s, "clear", NULL). > > Or if you need to save a millisecond on an O(n) operation, use > PyNumber_InPlaceSubtract(s,s) as shown in the docs. If the name bugs you, it > only takes a one-line macro to define a wrapper. The set API should not be > cluttered with unnecessary and redundant functions. This is a great example of what I'm talking about. You lose some static C compiler checks when you use either of these alternatives. C is not Python and we shouldn't try to make it so. The documentation is much less concise too, and if macros are encouraged, then every extension will invent their own name, further reducing readability, or use the obvious choice of PySet_Clear() and then question why Python doesn't provide this itself. This also has a detrimental effect on debugging. Macros suck for debugging and going through all the abstract API layers sucks. A nice, clean, direct call is so much more embedder-friendly. In addition, you essentially have all the pieces for PySet_Clear() right there in front of you, so why not expose them to embedders and make their lives easier? Forcing them to go through the abstract API or use obscure alternatives does not improve the API. It seems a false economy to not include concrete API calls just to end up back in setobject.c after layers of indirection. > PySet_Next() > ------------ > This is also redundant. The preferred way to iterate over a set should be > PyObject_GetIter(s). The iter api is generic and works for all containers. It > ought to be the one-way-to-do-it. For the C API, I disagree for the reasons stated above. In this specific case, using the iterator API actually imposes more pain on embedders because there are more things you have to keep track of and that can go wrong. PyDict_Next() is a very nice and direct API, where you often don't have to worry about reference counting (borrowed refs in this case are the right thing to return). You also don't have to worry about error conditions, and both of these things reduce bugs because it usually means less code. PySet_Next() would provide the same benefits. I don't buy the safety argument against PyDict_Next()/PySet_Next() because they are clearly documented as requiring no modification during iteration. Again, this is what I mean by useful concrete vs. abstract APIs. When you /know/ you have a set and you /know/ you won't be modifying it, PySet_Next() is the perfect interface. If you will be modifying the set, or don't know what kind of sequence you have, then the abstract API is the right thing to use. > Further, it doesn't make sense to model this after the dictionary API where the > next function is needed to avoid double lookups by returning pointers to both > the key and value fields at the same time (allowing for modification of the > value field). In contrast, for sets, there is no value field to look-up or > mutate (the key should not be touched). So, we shouldn't be passing around > pointers to the internal structure. I want to keep the internal structure of > sets much more private than they were for dictionaries -- all access should be > through the provided C API functions -- that keeps the implementation flexible > and allows for future improvements without worrying that we've broken code for > someone who has touched the internal structure directly. The implementation of PySet_Next() would not return setentrys, it would return PyObjects. Yes, those would be borrowed refs to setentry.keys, but you still avoid direct access to internal structures. > Also, the _Next() api is not as safe as the _GetIter api which checks for > mutation during iteration. The safety should not be tossed aside without good > reason. > > > PySet_Update() > --------------- > Use PyObject_CallMethod(s, "update", "O", iterable). That is the preferred way > to access all of the high volume methods. Again, I disagree, but I don't think I need to restate my reasons. > Only the fine grained methods (like > contains, add, pop, or discard) have a need for a direct call. Adding > unnecessary functions for the many-at-once methods gains you nothing -- perhaps > saving a tiny O(1) look-up step in an O(n) operation. > > FWIW, the same reasoning also applies to why the list API defines > PyList_Append() but not PyList_Extend(). Personally, I think that's a bug in the PyList C API. I haven't complained because I've rarely needed it, but it /is/ a deficiency. > PySet_AsList() > --------------- > There is already a function expressly for this purpose, PySequence_List(s). I'll grant you this one. ;) Forget PySet_AsList(). I'll try to answer the rest of your message without repeating myself too much. ;) > As it stands now, it is possible to use sets in C programs and access them in a > way that has a direct correspondence to pure Python code -- using > PyObject_CallMethod() for things we would usually access by name, using the > PyNumber API for things we would access using operators, using other parts of > the abstract API exactly as we would in Python (PyObject_Repr, PyObject_GetIter, > PySequence_List, PyObject_Print, etc.), and using a handful of direct access > functions for the fine grained methods like (add, pop, contains, etc.). IOW, I > like the way the C code looks now and I like the minimal, yet complete API. > Let's don't muck it up. This is where you and I disagree. Again, C is not Python. I actually greatly dislike having to use things like PyObject_Call() for concrete objects. First, the C code does not look like Python at all, and is actually /less/ readable because now you have to look in two places to understand what the code does. Second, it imposes much more pain when debugging because of all the extra layers you have to step through. But of course, with a rich concrete and abstract API, as most Python types have, we both get to appease our aesthetic demons, and chose the right tool for the job. > FWIW, the C implementation in Py2.5 already provides nice speed-ups for many > operations. Likewise, its memory requirements have been reduced by a third. > Try to enjoy the improvements without gilding the lily. Let's embrace C and continue to make life easier for the C coder. You can't argue that going through all the rigamarole of the iterator API would be faster than PySet_Next(), and it certainly won't be more readable or easier to debug. A foolish consistency, and all that... Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060321/48295e94/attachment.pgp From raymond.hettinger at verizon.net Wed Mar 22 03:31:13 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Tue, 21 Mar 2006 21:31:13 -0500 Subject: [Python-Dev] PySet API References: <1142705253.32074.72.camel@geddy.wooz.org><005c01c64ac5$589859b0$b83efea9@RaymondLaptop1><002b01c64aeb$44eb7750$b83efea9@RaymondLaptop1><1142824236.10599.29.camel@resist.wooz.org><00a001c64bfa$71a0ab20$b83efea9@RaymondLaptop1> <1142987079.10599.232.camel@resist.wooz.org> Message-ID: <004001c64d58$b0ebebf0$b83efea9@RaymondLaptop1> [Barry] > Is it your intent to push for more use of the abstract API instead of > the concrete APIs for all of Python's C data structures? Current API > aside, are you advocating this approach for all new built-in types? > Would you argue that Python 3.0's C API be stripped of everything but > the abstract API and the bare essentials of the concrete API? It's not up to me. Perhaps someone else can chime-in about the philosophy of how the C API is supposed to balance abstract and concrete APIs. For concrete APIs, it does make sense to have methods for fine-grained access to containers (for the O(1) ops that tend to appear inside loops). I know that the more one uses the abstract API, the more likely the code is going to be able to accept duck typed inputs. Also, most things that have tp_slots have a corresponding abstract method instead of tons a concrete access points; hence, I would be supportive if you proposed a PyObject_Clear(o) function (for calling tp_clear slots when they exist and returning an error code when they don't). For setobject.c, if I still have a say in the matter, my strong preference is to keep the API minimal, expose fine-grained functions for efficiency, use PyNumber methods for direct access to operator style set operations, and use the abstract API for everything else. Though you have a different world view about fat vs thin APIs and on whether the C API should track the Python API, perhaps you can agree with my reasoning in this case. There is a semantic difference between code like s+=t and s.update(t). The former only works when t is a set and the latter works for any iterable. When the C code corresponds to the Python code, that knowledge is kept intact and there is no confusion between PyNumber_InPlaceAdd(s,t) vs PyObject_CallMethod(s, "update", "(O)", t). I did not want to fill-up the C API with methods corresponding to every named method in the Python API -- besides making the API unnecessarily fat, IMO, it introduces ambiguity about whether PySet_Update would be the set-only operation or the generator iteratable version. With respect to clear(), I do not not want to further enshrine that method. IMO, most uses of it are misguided and the code would be better-off allowing the existing set to be decreffed away and using PySet_New() instead. I realize that is a matter of taste, but it is a basic of API design that the module author provide clues about how the module is intended to be used. With respect to PySet_Next(), I do agree that it is both a bit faster and easier to use than the iterator API. That is more of a reflection of issues with the iterator API than it is an indication that PySet_Next() would be worthwhile. That being said, I stand by my rejection of that method. For dictionaries, the problems with the approach were some offset by the advantages of getting a simultaneous key/value lookup and by direct access to the memory where the value was stored. Sets, of course, do not have these offseting benefits. The problems with the approach however do apply to sets. I do not want to be passing around pointers to the private internal hash table -- programs have no business altering that memory directly. You have to be careful with even simple cases such as using PyString_AS_STRING which returns a pointer and then the underlying object can get decreffed away leaving a pointer to an invalid memory location (that happened to me recently but I caught it during code review). Also, there is the safety issue of having the table mutate during iteration. While your note downplayed the issue noting that the risks are fully disclosed, I speak from experience in saying that it is all too easy to accidentally allow the table to mutate during iteration -- I learned this the hard-way and had to undo a number of my uses of set_next() which had been used for speed and for being simpler than the iteator api. A single decref or call to a key's PyObject_Hash() is enough to trigger arbitrary Python code running during the middle of iteration, possibly resulting in a resize or value mutation. Take a look at the code for set_clear() to see the dance required to avoid this risk. IOW, the safety considerations alone preclude PySet_Next(). Instead, use PyObject_GetIter() and enjoy the benefits of safety, duck typing, and re-usable code. Because of the safety issues and passing internal pointers, I prefer that the _Next() api NOT get replicated throughout the language. It was needed for dicts, but shouldn't start sprouting up everywhere. All that being said, I understand that some of it comes down to taste and philosophical differences. As the module author, the existing API of course reflects my tastes and world-view. I have given you simple, one line alternatives to each proposal and listed the reasons for each choice, so it can't be argued that the API is somehow crippling your work. I'm sympathethic to your reluctance to use PyObject_CallMethod(). But, do understand that it is simply an aversion. It works just fine and makes no speed difference on coarse-grained, O(n) methods. I like its clarity and direct correspondence to the pure Python api for sets. Because the methods in question would all be METH_O or METH_NOARGS, the only static type checking you've lost is verifying the number of arguments. I would suggest that if you have the wrong number of arguments for s.update(t) or s.clear(), then you have problems a C API can't solve ;-) Cheers, Raymond P.S. One other thought: I don't want to crystalize the API in a way that precludes future development of the module. One possibility for the future is for updates to take multiple arguments such as s.update(t,u,v) causing three updates to be folded-in at once. From python at rcn.com Wed Mar 22 04:01:39 2006 From: python at rcn.com (Raymond Hettinger) Date: Tue, 21 Mar 2006 22:01:39 -0500 Subject: [Python-Dev] PySet API References: <1142705253.32074.72.camel@geddy.wooz.org><005c01c64ac5$589859b0$b83efea9@RaymondLaptop1><002b01c64aeb$44eb7750$b83efea9@RaymondLaptop1><1142824236.10599.29.camel@resist.wooz.org><00a001c64bfa$71a0ab20$b83efea9@RaymondLaptop1><1142987079.10599.232.camel@resist.wooz.org> <004001c64d58$b0ebebf0$b83efea9@RaymondLaptop1> Message-ID: <006101c64d5c$f09c6870$b83efea9@RaymondLaptop1> [Me] > There is a semantic difference between > code like s+=t and s.update(t). The former only works when t is a set > and the latter works for any iterable. When the C code corresponds to > the Python code, that knowledge is kept intact and there is no confusion > between > PyNumber_InPlaceAdd(s,t) vs PyObject_CallMethod(s, "update", "(O)", t). Of course, that should have been s|=t and PyNumber_InPlaceOr(). Raymond From buildbot at python.org Wed Mar 22 04:44:11 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 22 Mar 2006 03:44:11 +0000 Subject: [Python-Dev] buildbot warnings in amd64 gentoo trunk Message-ID: <20060322034411.010CD1E4004@bag.python.org> The Buildbot has detected a new failure of amd64 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%20gentoo%20trunk/builds/108 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: tim.peters Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Wed Mar 22 08:34:27 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 22 Mar 2006 07:34:27 +0000 Subject: [Python-Dev] buildbot warnings in x86 gentoo trunk Message-ID: <20060322073427.55A261E4004@bag.python.org> The Buildbot has detected a new failure of x86 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%20gentoo%20trunk/builds/109 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings test sincerely, -The Buildbot From fredrik at pythonware.com Wed Mar 22 10:27:35 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 22 Mar 2006 10:27:35 +0100 Subject: [Python-Dev] r43214 - peps/trunk/pep-3000.txt References: <20060322052432.419F01E4004@bag.python.org> Message-ID: neal.norwitz wrote: > +Outstanding Issues > +================== > + > +* Require C99, so we can use // comments, named initializers, declare variables > + without introducing a new scope, among other benefits. gcc only, in other words ? > +* Remove support for old systems, including: OS2, BeOS, RISCOS, (SGI) Irix, Tru64 what's old with tru64 ? it's not that uncommon in places where Python has a strong presence, you can still buy AXP hardware throughout 2006, and HP says they'll keep developing and supporting the software platform at least through 2011. From mwh at python.net Wed Mar 22 10:34:25 2006 From: mwh at python.net (Michael Hudson) Date: Wed, 22 Mar 2006 09:34:25 +0000 Subject: [Python-Dev] r43214 - peps/trunk/pep-3000.txt In-Reply-To: (Fredrik Lundh's message of "Wed, 22 Mar 2006 10:27:35 +0100") References: <20060322052432.419F01E4004@bag.python.org> Message-ID: <2mlkv2x1pa.fsf@starship.python.net> "Fredrik Lundh" writes: > neal.norwitz wrote: > >> +Outstanding Issues >> +================== >> + >> +* Require C99, so we can use // comments, named initializers, declare variables >> + without introducing a new scope, among other benefits. > > gcc only, in other words ? Heh, I was going to make this point as well: it's not clear that MSVC will ever support C99. It supports some of the features listed here, of course, but probably won't support everything. >> +* Remove support for old systems, including: OS2, BeOS, RISCOS, (SGI) Irix, Tru64 > > what's old with tru64 ? it's not that uncommon in places where Python > has a strong presence, you can still buy AXP hardware throughout 2006, > and HP says they'll keep developing and supporting the software platform > at least through 2011. And we still have someone actively interested in maintaining the OS2 port, it seems. Cheers, mwh -- I never disputed the Perl hacking skill of the Slashdot creators. My objections are to the editors' taste, the site's ugly visual design, and the Slashdot community's raging stupidity. -- http://www.cs.washington.edu/homes/klee/misc/slashdot.html#faq From fredrik at pythonware.com Wed Mar 22 10:53:28 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 22 Mar 2006 10:53:28 +0100 Subject: [Python-Dev] r43214 - peps/trunk/pep-3000.txt References: <20060322052432.419F01E4004@bag.python.org> <2mlkv2x1pa.fsf@starship.python.net> Message-ID: Michael Hudson wrote: > >> +* Remove support for old systems, including: OS2, BeOS, RISCOS, (SGI) Irix, Tru64 > > > > what's old with tru64 ? it's not that uncommon in places where Python > > has a strong presence, you can still buy AXP hardware throughout 2006, > > and HP says they'll keep developing and supporting the software platform > > at least through 2011. > > And we still have someone actively interested in maintaining the OS2 > port, it seems. fwict, OS/2 will be officially supported throughout this year. not sure about Irix; the roadmaps don't seem to be available to non-paying customers. From mal at egenix.com Wed Mar 22 11:02:14 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 22 Mar 2006 11:02:14 +0100 Subject: [Python-Dev] r43214 - peps/trunk/pep-3000.txt In-Reply-To: <2mlkv2x1pa.fsf@starship.python.net> References: <20060322052432.419F01E4004@bag.python.org> <2mlkv2x1pa.fsf@starship.python.net> Message-ID: <442120A6.6050106@egenix.com> Michael Hudson wrote: > "Fredrik Lundh" writes: > >> neal.norwitz wrote: >> >>> +Outstanding Issues >>> +================== >>> + >>> +* Require C99, so we can use // comments, named initializers, declare variables >>> + without introducing a new scope, among other benefits. >> gcc only, in other words ? > > Heh, I was going to make this point as well: it's not clear that MSVC > will ever support C99. It supports some of the features listed here, > of course, but probably won't support everything. Given the list of features Neal mentioned, I don't think these really warrant going for full C99. >>> +* Remove support for old systems, including: OS2, BeOS, RISCOS, (SGI) Irix, Tru64 >> what's old with tru64 ? it's not that uncommon in places where Python >> has a strong presence, you can still buy AXP hardware throughout 2006, >> and HP says they'll keep developing and supporting the software platform >> at least through 2011. > > And we still have someone actively interested in maintaining the OS2 > port, it seems. Dito for BeOS, now under the name Zeta OS. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 22 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mithrandi-python-dev at mithrandi.za.net Wed Mar 22 11:04:34 2006 From: mithrandi-python-dev at mithrandi.za.net (Tristan Seligmann) Date: Wed, 22 Mar 2006 12:04:34 +0200 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: <441F54F5.3000004@canterbury.ac.nz> References: <4418CDD0.8020803@canterbury.ac.nz> <50372CD0-B314-4FED-8C57-3F563F459E09@gmail.com> <200603200600.03110.gmccaughan@synaptics-uk.com> <441F54F5.3000004@canterbury.ac.nz> Message-ID: <20060322100434.GC7966@mithrandi.za.net> * Greg Ewing [2006-03-21 13:20:53 +1200]: > Gareth McCaughan wrote: > > > but wouldn't if be nice if you could say > > > > def f((x0,y0) as p0, (x1,y1) as p1): > > I'm not sure that it would. Currently you can look at > a function header and get a picture of its calling > signature, but this would clutter it up with > implementation details of the function that aren't > relevant to the caller. For maximal utility, this would affect the calling signature of the function, too: it would now have keyword arguments named p0 and p1. -- mithrandi, i Ainil en-Balandor, a faer Ambar -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20060322/2395471a/attachment.pgp From buildbot at python.org Wed Mar 22 11:34:28 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 22 Mar 2006 10:34:28 +0000 Subject: [Python-Dev] buildbot warnings in x86 gentoo trunk Message-ID: <20060322103428.7FA601E4004@bag.python.org> The Buildbot has detected a new failure of x86 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%20gentoo%20trunk/builds/111 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: thomas.heller Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Wed Mar 22 14:37:51 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 22 Mar 2006 13:37:51 +0000 Subject: [Python-Dev] buildbot warnings in g4 osx.4 trunk Message-ID: <20060322133751.240261E4005@bag.python.org> The Buildbot has detected a new failure of g4 osx.4 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/g4%20osx.4%20trunk/builds/105 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: thomas.heller Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Wed Mar 22 14:42:54 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 22 Mar 2006 13:42:54 +0000 Subject: [Python-Dev] buildbot warnings in x86 gentoo trunk Message-ID: <20060322134255.177BD1E4005@bag.python.org> The Buildbot has detected a new failure of x86 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%20gentoo%20trunk/builds/113 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: thomas.heller Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Wed Mar 22 15:13:36 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 22 Mar 2006 14:13:36 +0000 Subject: [Python-Dev] buildbot warnings in sparc solaris10 gcc trunk Message-ID: <20060322141336.B636D1E4005@bag.python.org> The Buildbot has detected a new failure of sparc solaris10 gcc trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%20solaris10%20gcc%20trunk/builds/108 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: thomas.heller Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Wed Mar 22 16:18:20 2006 From: buildbot at python.org (buildbot at python.org) Date: Wed, 22 Mar 2006 15:18:20 +0000 Subject: [Python-Dev] buildbot warnings in amd64 gentoo trunk Message-ID: <20060322151820.672AA1E4005@bag.python.org> The Buildbot has detected a new failure of amd64 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%20gentoo%20trunk/builds/115 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: barry.warsaw Build Had Warnings: warnings test sincerely, -The Buildbot From theller at python.net Wed Mar 22 18:02:28 2006 From: theller at python.net (Thomas Heller) Date: Wed, 22 Mar 2006 18:02:28 +0100 Subject: [Python-Dev] Building Python for AMD64 (Windows) Message-ID: I'm trying out to build Python for 64-bit Windows, running the tools on WinXP SP2 (32-bit). I have installed: - MSVC .NET 2003 - Platform SDK Feb 2003 - vsextcomp 0.6 Then I fire up MSVC, select the ReleaseAMD64 configuration, and try to build the pythoncore project. After lots of compiler warnings, the build fails with this message: cl.exe -c -D_WIN32 -DUSE_DL_EXPORT -D_WINDOWS -DWIN32 -D_WINDLL -MD -MD /USECL:MS_OPTERON getbuildinfo2.c -DSUBWCREV -Fogetbuildinfo.o -I..\Include -I..\PC Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077 for 80x86 Copyright (C) Microsoft Corporation 1984-2002. All rights reserved. getbuildinfo2.c Linking... LINK : warning LNK4044: unrecognized option '/USELINK:MS_SDK'; ignored getbuildinfo.o : fatal error LNK1112: module machine type 'X86' conflicts with target machine type 'AMD64' Build log was saved at "file://c:\svn\trunk\PCbuild\amd64-temp-release\pythoncore\BuildLog.htm" pythoncore - 1 error(s), 229 warning(s) Is there something wrong on my side? Thanks, Thomas From martin at v.loewis.de Wed Mar 22 19:37:45 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 22 Mar 2006 19:37:45 +0100 Subject: [Python-Dev] r43214 - peps/trunk/pep-3000.txt In-Reply-To: <442120A6.6050106@egenix.com> References: <20060322052432.419F01E4004@bag.python.org> <2mlkv2x1pa.fsf@starship.python.net> <442120A6.6050106@egenix.com> Message-ID: <44219979.8050406@v.loewis.de> M.-A. Lemburg wrote: >> And we still have someone actively interested in maintaining the OS2 >> port, it seems. > > Dito for BeOS, now under the name Zeta OS. Who is the one interested in maintaining the BeOS port? the last checkins related to BeOS seem to originate from the 2001/2002 period. Regards, Martin From martin at v.loewis.de Wed Mar 22 19:43:14 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 22 Mar 2006 19:43:14 +0100 Subject: [Python-Dev] Building Python for AMD64 (Windows) In-Reply-To: References: Message-ID: <44219AC2.2050502@v.loewis.de> Thomas Heller wrote: > Then I fire up MSVC, select the ReleaseAMD64 configuration, and try to build the > pythoncore project. > After lots of compiler warnings, the build fails with this message: > > cl.exe -c -D_WIN32 -DUSE_DL_EXPORT -D_WINDOWS -DWIN32 -D_WINDLL -MD -MD /USECL:MS_OPTERON getbuildinfo2.c -DSUBWCREV -Fogetbuildinfo.o -I..\Include -I..\PC > Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077 for 80x86 > Copyright (C) Microsoft Corporation 1984-2002. All rights reserved. > getbuildinfo2.c > Linking... > LINK : warning LNK4044: unrecognized option '/USELINK:MS_SDK'; ignored > getbuildinfo.o : fatal error LNK1112: module machine type 'X86' conflicts with target machine type 'AMD64' > > Build log was saved at "file://c:\svn\trunk\PCbuild\amd64-temp-release\pythoncore\BuildLog.htm" > pythoncore - 1 error(s), 229 warning(s) > > Is there something wrong on my side? For some reason, VS.NET hasn't picked up the PATH modifications, so that it doesn't invoke the linker wrapper (which would have invoked the right compiler and linker). This sometimes happens, and I don't know why. As the first measure, open Tools/External compiler configuration, and see whether this reports changes to the path. If it does, throw all object files away, and recompile. Otherwise, please try editing the executable path manually: Tools/Options/Projects/VC++ Directories/Executable Files should start with c:\program files\vsextcompiler. Please let me know whether that works. Martin From mal at egenix.com Wed Mar 22 20:21:56 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 22 Mar 2006 20:21:56 +0100 Subject: [Python-Dev] [Fwd: buildbot warnings in amd64 gentoo trunk] Message-ID: <4421A3D4.5080507@egenix.com> Would it be possible to redirect these buildbot messages to the python-checkins or a separate python-buildbot list ? -------- Original Message -------- Subject: [Python-Dev] buildbot warnings in amd64 gentoo trunk Date: Wed, 22 Mar 2006 15:18:20 +0000 From: buildbot at python.org Reply-To: python-dev at python.org To: python-dev at python.org The Buildbot has detected a new failure of amd64 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%20gentoo%20trunk/builds/115 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: barry.warsaw Build Had Warnings: warnings test sincerely, -The Buildbot _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/mal%40egenix.com -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 22 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From theller at python.net Wed Mar 22 20:29:19 2006 From: theller at python.net (Thomas Heller) Date: Wed, 22 Mar 2006 20:29:19 +0100 Subject: [Python-Dev] Building Python for AMD64 (Windows) In-Reply-To: <44219AC2.2050502@v.loewis.de> References: <44219AC2.2050502@v.loewis.de> Message-ID: <4421A58F.20709@python.net> Martin v. L?wis wrote: > Thomas Heller wrote: >> Then I fire up MSVC, select the ReleaseAMD64 configuration, and try to >> build the >> pythoncore project. >> After lots of compiler warnings, the build fails with this message: >> ... > > For some reason, VS.NET hasn't picked up the PATH modifications, so that > it doesn't invoke the linker wrapper (which would have invoked the right > compiler and linker). This sometimes happens, and I don't know why. > > As the first measure, open Tools/External compiler configuration, and > see whether this reports changes to the path. If it does, throw all > object files away, and recompile. > > Otherwise, please try editing the executable path manually: > > Tools/Options/Projects/VC++ Directories/Executable Files > > should start with c:\program files\vsextcompiler. > > Please let me know whether that works. Yes, that worked. vsextcomp complained that the PATH was not correct, but it did then repair it itself. Ok, so now I'm able to make itanium builds (which I don't want because I don't have such amachine). The Feb 2003 Platform SDK apparently doesn't have an AMD64 compiler. In the meantime, I installed a fresh copy of MSVC on a second machine (Athlon64, running Windows XP 64bit), plus the Platform SDK for Win Server 2003 SP1 - April 2005 Edition. Same problems with vsextcomp, but vsextcomp repairs the path itself again. But here, starting the build it complains: CL.EXE Wrapper for VSExtCompiler Plugin Error 2 while trying to open SDK path key from registry ... Error: Could not determine installation directory of Microsoft SDK. Guess I'll install the 2003 SP1 SDK on the first machine and try it there. Thomas From theller at python.net Wed Mar 22 20:52:53 2006 From: theller at python.net (Thomas Heller) Date: Wed, 22 Mar 2006 20:52:53 +0100 Subject: [Python-Dev] PY_SSIZE_T_CLEAN In-Reply-To: <441BBF16.60802@v.loewis.de> References: <20060315084115.90E641E4078@bag.python.org> <4417D654.3030600@v.loewis.de> <1f7befae0603150103t2c3121fdm86034f6a0174a667@mail.gmail.com> <4417DAB7.7030308@v.loewis.de> <1f7befae0603150120t51f425e0t1f470718f3b222a4@mail.gmail.com> <4418A45D.7090500@v.loewis.de> <4419CE7A.2030500@python.net> <441BBF16.60802@v.loewis.de> Message-ID: <4421AB15.5060100@python.net> Martin v. L?wis wrote: > Thomas Heller wrote: >> Then, from only reading the PEP without looking up the sources, >> it is not clear to me what the PY_SIZE_T_CLEAN definition does. > > The description says > > The conversion codes 's#' and 't#' will output Py_ssize_t > if the macro PY_SSIZE_T_CLEAN is defined before Python.h > is included, and continue to output int if that macro > isn't defined. > > which (IMO, of course), specifies precisely what PY_SSIZE_T_CLEAN > does. What information do you find lacking here? Nothing lacking. It is fine - I didn't read it carefully enough. Thanks, Thomas From theller at python.net Wed Mar 22 21:26:16 2006 From: theller at python.net (Thomas Heller) Date: Wed, 22 Mar 2006 21:26:16 +0100 Subject: [Python-Dev] Building Python for AMD64 (Windows) In-Reply-To: <4421A58F.20709@python.net> References: <44219AC2.2050502@v.loewis.de> <4421A58F.20709@python.net> Message-ID: <4421B2E8.7010309@python.net> Thomas Heller wrote: > Martin v. L?wis wrote: >> Thomas Heller wrote: >>> Then I fire up MSVC, select the ReleaseAMD64 configuration, and try to >>> build the >>> pythoncore project. >>> After lots of compiler warnings, the build fails with this message: >>> > ... >> For some reason, VS.NET hasn't picked up the PATH modifications, so that >> it doesn't invoke the linker wrapper (which would have invoked the right >> compiler and linker). This sometimes happens, and I don't know why. >> >> As the first measure, open Tools/External compiler configuration, and >> see whether this reports changes to the path. If it does, throw all >> object files away, and recompile. >> >> Otherwise, please try editing the executable path manually: >> >> Tools/Options/Projects/VC++ Directories/Executable Files >> >> should start with c:\program files\vsextcompiler. >> >> Please let me know whether that works. > > Yes, that worked. vsextcomp complained that the PATH was not correct, but it > did then repair it itself. > > Ok, so now I'm able to make itanium builds (which I don't want because I don't have > such amachine). The Feb 2003 Platform SDK apparently doesn't have an AMD64 compiler. > > In the meantime, I installed a fresh copy of MSVC on a second machine (Athlon64, running > Windows XP 64bit), plus the Platform SDK for Win Server 2003 SP1 - April 2005 Edition. > > Same problems with vsextcomp, but vsextcomp repairs the path itself again. > But here, starting the build it complains: > CL.EXE Wrapper for VSExtCompiler Plugin > Error 2 while trying to open SDK path key from registry ... > Error: Could not determine installation directory of Microsoft SDK. > > Guess I'll install the 2003 SP1 SDK on the first machine and try it there. Ok, the Win32 Server 2003 SP1 April 2005 SDK is installed on the first machine, the build now fails with this: ------ Build started: Project: pythoncore, Configuration: ReleaseAMD64 Win32 ------ Compiling resources... generate buildinfo "C:\Programme\TortoiseSVN\bin\subwcrev.exe" .. ..\Modules\getbuildinfo.c getbuildinfo2.c SubWCRev: 'c:\svn\trunk' Last committed at revision 43226 Updated to revision 43226 Local modifications found cl.exe -c -D_WIN32 -DUSE_DL_EXPORT -D_WINDOWS -DWIN32 -D_WINDLL -MD -MD /USECL:MS_OPTERON getbuildinfo2.c -DSUBWCREV -Fogetbuildinfo.o -I..\Include -I..\PC getbuildinfo2.c ------ CL.EXE Wrapper for VSExtCompiler Plugin ------ Linking... Creating library .\./python25.lib and object .\./python25.exp zipimport.obj : error LNK2001: unresolved external symbol __security_cookie traceback.obj : error LNK2001: unresolved external symbol __security_cookie typeobject.obj : error LNK2001: unresolved external symbol __security_cookie unicodeobject.obj : error LNK2001: unresolved external symbol __security_cookie (lots of similar errors...) gzio.obj : error LNK2001: unresolved external symbol __security_check_cookie inftrees.obj : error LNK2001: unresolved external symbol __security_check_cookie trees.obj : error LNK2001: unresolved external symbol __security_check_cookie _codecs_cn.obj : error LNK2001: unresolved external symbol __security_check_cookie ./python25.dll : fatal error LNK1120: 2 unresolved externals ------ LINK.EXE Wrapper for VSExtCompiler Plugin ------ Build log was saved at "file://c:\svn\trunk\PCbuild\amd64-temp-release\pythoncore\BuildLog.htm" pythoncore - 107 error(s), 0 warning(s) ---------------------- Done ---------------------- Build: 2 succeeded, 1 failed, 0 skipped Is there a library missing? BTW: When this eventually builds, will this run on an Athlon64, or does it need an Opteron processor? Thomas From theller at python.net Wed Mar 22 21:26:16 2006 From: theller at python.net (Thomas Heller) Date: Wed, 22 Mar 2006 21:26:16 +0100 Subject: [Python-Dev] Building Python for AMD64 (Windows) In-Reply-To: <4421A58F.20709@python.net> References: <44219AC2.2050502@v.loewis.de> <4421A58F.20709@python.net> Message-ID: <4421B2E8.7010309@python.net> Thomas Heller wrote: > Martin v. L?wis wrote: >> Thomas Heller wrote: >>> Then I fire up MSVC, select the ReleaseAMD64 configuration, and try to >>> build the >>> pythoncore project. >>> After lots of compiler warnings, the build fails with this message: >>> > ... >> For some reason, VS.NET hasn't picked up the PATH modifications, so that >> it doesn't invoke the linker wrapper (which would have invoked the right >> compiler and linker). This sometimes happens, and I don't know why. >> >> As the first measure, open Tools/External compiler configuration, and >> see whether this reports changes to the path. If it does, throw all >> object files away, and recompile. >> >> Otherwise, please try editing the executable path manually: >> >> Tools/Options/Projects/VC++ Directories/Executable Files >> >> should start with c:\program files\vsextcompiler. >> >> Please let me know whether that works. > > Yes, that worked. vsextcomp complained that the PATH was not correct, but it > did then repair it itself. > > Ok, so now I'm able to make itanium builds (which I don't want because I don't have > such amachine). The Feb 2003 Platform SDK apparently doesn't have an AMD64 compiler. > > In the meantime, I installed a fresh copy of MSVC on a second machine (Athlon64, running > Windows XP 64bit), plus the Platform SDK for Win Server 2003 SP1 - April 2005 Edition. > > Same problems with vsextcomp, but vsextcomp repairs the path itself again. > But here, starting the build it complains: > CL.EXE Wrapper for VSExtCompiler Plugin > Error 2 while trying to open SDK path key from registry ... > Error: Could not determine installation directory of Microsoft SDK. > > Guess I'll install the 2003 SP1 SDK on the first machine and try it there. Ok, the Win32 Server 2003 SP1 April 2005 SDK is installed on the first machine, the build now fails with this: ------ Build started: Project: pythoncore, Configuration: ReleaseAMD64 Win32 ------ Compiling resources... generate buildinfo "C:\Programme\TortoiseSVN\bin\subwcrev.exe" .. ..\Modules\getbuildinfo.c getbuildinfo2.c SubWCRev: 'c:\svn\trunk' Last committed at revision 43226 Updated to revision 43226 Local modifications found cl.exe -c -D_WIN32 -DUSE_DL_EXPORT -D_WINDOWS -DWIN32 -D_WINDLL -MD -MD /USECL:MS_OPTERON getbuildinfo2.c -DSUBWCREV -Fogetbuildinfo.o -I..\Include -I..\PC getbuildinfo2.c ------ CL.EXE Wrapper for VSExtCompiler Plugin ------ Linking... Creating library .\./python25.lib and object .\./python25.exp zipimport.obj : error LNK2001: unresolved external symbol __security_cookie traceback.obj : error LNK2001: unresolved external symbol __security_cookie typeobject.obj : error LNK2001: unresolved external symbol __security_cookie unicodeobject.obj : error LNK2001: unresolved external symbol __security_cookie (lots of similar errors...) gzio.obj : error LNK2001: unresolved external symbol __security_check_cookie inftrees.obj : error LNK2001: unresolved external symbol __security_check_cookie trees.obj : error LNK2001: unresolved external symbol __security_check_cookie _codecs_cn.obj : error LNK2001: unresolved external symbol __security_check_cookie ./python25.dll : fatal error LNK1120: 2 unresolved externals ------ LINK.EXE Wrapper for VSExtCompiler Plugin ------ Build log was saved at "file://c:\svn\trunk\PCbuild\amd64-temp-release\pythoncore\BuildLog.htm" pythoncore - 107 error(s), 0 warning(s) ---------------------- Done ---------------------- Build: 2 succeeded, 1 failed, 0 skipped Is there a library missing? BTW: When this eventually builds, will this run on an Athlon64, or does it need an Opteron processor? Thomas From trentm at ActiveState.com Wed Mar 22 22:45:46 2006 From: trentm at ActiveState.com (Trent Mick) Date: Wed, 22 Mar 2006 13:45:46 -0800 Subject: [Python-Dev] Building Python for AMD64 (Windows) In-Reply-To: <4421B2E8.7010309@python.net> References: <44219AC2.2050502@v.loewis.de> <4421A58F.20709@python.net> <4421B2E8.7010309@python.net> Message-ID: <20060322214546.GB28488@activestate.com> [Thomas Heller wrote] > Linking... > Creating library .\./python25.lib and object .\./python25.exp > zipimport.obj : error LNK2001: unresolved external symbol __security_cookie > traceback.obj : error LNK2001: unresolved external symbol __security_cookie > typeobject.obj : error LNK2001: unresolved external symbol __security_cookie > unicodeobject.obj : error LNK2001: unresolved external symbol __security_cookie > > (lots of similar errors...) I have this patch to the python 2.4 sources to get win64-ia64 to build (that I haven't yet had the chance to submit to SF). I think it will require a little massaging for the trunk. Does this work for you? Trent -- Trent Mick TrentM at ActiveState.com -------------- next part -------------- diff Must link with bufferoverflowU.lib for Win64 builds of some binaries with the latest Platform SDK to avoid errors a la (c.f. http://support.microsoft.com/?id=894573): 'unresolved external symbol __security_cookie' errors. diff -ur python/PCbuild.original/_bsddb.vcproj python/PCbuild/_bsddb.vcproj --- python/PCbuild.original/_bsddb.vcproj Thu Aug 12 07:07:49 2004 +++ python/PCbuild/_bsddb.vcproj Thu Jun 02 16:41:43 2005 @@ -179,7 +179,7 @@ References: <44219AC2.2050502@v.loewis.de> <4421A58F.20709@python.net> Message-ID: <4421CC99.2070801@v.loewis.de> Thomas Heller wrote: > Ok, so now I'm able to make itanium builds (which I don't want because I > don't have such amachine). The Feb 2003 Platform SDK apparently doesn't have an > AMD64 compiler. Right. The Itanium compiler was included in the November 2004 (or so) SDK (which called itself still Feb 2003); this was available only to MSDN subscribers. > In the meantime, I installed a fresh copy of MSVC on a second machine > (Athlon64, running > Windows XP 64bit), plus the Platform SDK for Win Server 2003 SP1 - April > 2005 Edition. That one does have the AMD64 compiler. > Same problems with vsextcomp, but vsextcomp repairs the path itself again. > But here, starting the build it complains: > CL.EXE Wrapper for VSExtCompiler Plugin > Error 2 while trying to open SDK path key from registry ... > Error: Could not determine installation directory of Microsoft SDK. > > Guess I'll install the 2003 SP1 SDK on the first machine and try it there. No. I thought I had released that version of vsextcomp... MS changed the registry keys to locate the SDK, and vsextcomp had to be updated for that. Will check tomorrow. Regards, Martin From martin at v.loewis.de Wed Mar 22 23:19:04 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 22 Mar 2006 23:19:04 +0100 Subject: [Python-Dev] [Fwd: buildbot warnings in amd64 gentoo trunk] In-Reply-To: <4421A3D4.5080507@egenix.com> References: <4421A3D4.5080507@egenix.com> Message-ID: <4421CD58.2050201@v.loewis.de> M.-A. Lemburg wrote: > Would it be possible to redirect these buildbot messages to the > python-checkins or a separate python-buildbot list ? Sure. They are sent to python-dev, because I think Tim Peters wanted them here. Regards, Martin From martin at v.loewis.de Wed Mar 22 23:25:01 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 22 Mar 2006 23:25:01 +0100 Subject: [Python-Dev] Building Python for AMD64 (Windows) In-Reply-To: <4421B2E8.7010309@python.net> References: <44219AC2.2050502@v.loewis.de> <4421A58F.20709@python.net> <4421B2E8.7010309@python.net> Message-ID: <4421CEBD.1000206@v.loewis.de> Thomas Heller wrote: > Is there a library missing? Either that (although I forgot the library name - google knows it), or a command line option missing: there is an option to suppress usage of these security check symbols. It appears that VS 2005 has them disabled by default, whereas the SDK compilers have enabled them by default. I think Python should not link with these libraries, but disable the feature instead. > BTW: When this eventually builds, will this run on an Athlon64, or > does it need an Opteron processor? It should run on any AMD64 processor, including Intel processors with EM64T. Regards, Martin From mal at egenix.com Wed Mar 22 23:31:16 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 22 Mar 2006 23:31:16 +0100 Subject: [Python-Dev] [Fwd: buildbot warnings in amd64 gentoo trunk] In-Reply-To: <4421CD58.2050201@v.loewis.de> References: <4421A3D4.5080507@egenix.com> <4421CD58.2050201@v.loewis.de> Message-ID: <4421D034.2040705@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg wrote: >> Would it be possible to redirect these buildbot messages to the >> python-checkins or a separate python-buildbot list ? > > Sure. They are sent to python-dev, because I think Tim Peters > wanted them here. For the Snake-Farm we had a separate mailing list, so I'd prefer that if possible. This lets you opt-in to the messages and also makes it easier to search via the python.org search facility. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 22 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From greg.ewing at canterbury.ac.nz Thu Mar 23 03:21:59 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 23 Mar 2006 14:21:59 +1200 Subject: [Python-Dev] Py3k: Except clause syntax In-Reply-To: <20060322100434.GC7966@mithrandi.za.net> References: <4418CDD0.8020803@canterbury.ac.nz> <50372CD0-B314-4FED-8C57-3F563F459E09@gmail.com> <200603200600.03110.gmccaughan@synaptics-uk.com> <441F54F5.3000004@canterbury.ac.nz> <20060322100434.GC7966@mithrandi.za.net> Message-ID: <44220647.7090204@canterbury.ac.nz> Tristan Seligmann wrote: > Greg Ewing [2006-03-21 13:20:53 +1200]: > > Gareth McCaughan wrote: > > > > > > def f((x0,y0) as p0, (x1,y1) as p1): > For maximal utility, this would affect the calling signature of the > function, too: it would now have keyword arguments named p0 and p1. I still think it hurts readability of the header too much to be worth the benefit. To me it would seem much clearer all round to write def f(p0, p1): x0, y0 = p0 x1, y1 = p1 -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From buildbot at python.org Thu Mar 23 07:18:57 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 23 Mar 2006 06:18:57 +0000 Subject: [Python-Dev] buildbot warnings in x86 gentoo trunk Message-ID: <20060323061857.C588B1E4009@bag.python.org> The Buildbot has detected a new failure of x86 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%20gentoo%20trunk/builds/116 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings test sincerely, -The Buildbot From martin at v.loewis.de Thu Mar 23 07:23:44 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 23 Mar 2006 07:23:44 +0100 Subject: [Python-Dev] [Fwd: buildbot warnings in amd64 gentoo trunk] In-Reply-To: <4421D034.2040705@egenix.com> References: <4421A3D4.5080507@egenix.com> <4421CD58.2050201@v.loewis.de> <4421D034.2040705@egenix.com> Message-ID: <44223EF0.2070701@v.loewis.de> M.-A. Lemburg wrote: > For the Snake-Farm we had a separate mailing list, so I'd prefer > that if possible. This lets you opt-in to the messages and also > makes it easier to search via the python.org search facility. I'll wait for other people to voice an opinion. Creating a new mailing list is an effort I'd like to avoid. You still can opt-out, by filtering messages that originate from buildbot at python.org. Regards, Martin From nnorwitz at gmail.com Thu Mar 23 07:28:49 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 22 Mar 2006 22:28:49 -0800 Subject: [Python-Dev] [Fwd: buildbot warnings in amd64 gentoo trunk] In-Reply-To: <44223EF0.2070701@v.loewis.de> References: <4421A3D4.5080507@egenix.com> <4421CD58.2050201@v.loewis.de> <4421D034.2040705@egenix.com> <44223EF0.2070701@v.loewis.de> Message-ID: On 3/22/06, "Martin v. L?wis" wrote: > M.-A. Lemburg wrote: > > For the Snake-Farm we had a separate mailing list, so I'd prefer > > that if possible. This lets you opt-in to the messages and also > > makes it easier to search via the python.org search facility. > > I'll wait for other people to voice an opinion. Creating a new > mailing list is an effort I'd like to avoid. Hopefully we will get it tuned better soon so there aren't so many mails. Unfortunately, it may take a while due to the ssize_t issues getting fixed as we find them. I'd be happy if the messages were on python-checkins. n From buildbot at python.org Thu Mar 23 07:56:55 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 23 Mar 2006 06:56:55 +0000 Subject: [Python-Dev] buildbot warnings in sparc solaris10 gcc trunk Message-ID: <20060323065655.D3C071E4009@bag.python.org> The Buildbot has detected a new failure of sparc solaris10 gcc trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%20solaris10%20gcc%20trunk/builds/111 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings test sincerely, -The Buildbot From theller at python.net Thu Mar 23 09:48:46 2006 From: theller at python.net (Thomas Heller) Date: Thu, 23 Mar 2006 09:48:46 +0100 Subject: [Python-Dev] Building Python for AMD64 (Windows) In-Reply-To: <20060322214546.GB28488@activestate.com> References: <44219AC2.2050502@v.loewis.de> <4421A58F.20709@python.net> <4421B2E8.7010309@python.net> <20060322214546.GB28488@activestate.com> Message-ID: <442260EE.6030703@python.net> Trent Mick wrote: > [Thomas Heller wrote] >> Linking... >> Creating library .\./python25.lib and object .\./python25.exp >> zipimport.obj : error LNK2001: unresolved external symbol __security_cookie >> traceback.obj : error LNK2001: unresolved external symbol __security_cookie >> typeobject.obj : error LNK2001: unresolved external symbol __security_cookie >> unicodeobject.obj : error LNK2001: unresolved external symbol __security_cookie >> >> (lots of similar errors...) > > I have this patch to the python 2.4 sources to get win64-ia64 to build > (that I haven't yet had the chance to submit to SF). I think it will > require a little massaging for the trunk. Does this work for you? Yes, I was able to compile pythoncore when I added bufferoverflowU.lib to the project. However, starting python.exe immediately crashes with an access violation, in sys_getwindowsversion, in the PyBuild_Value() call. Thomas From ncoghlan at gmail.com Thu Mar 23 11:25:02 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 23 Mar 2006 20:25:02 +1000 Subject: [Python-Dev] [Fwd: buildbot warnings in amd64 gentoo trunk] In-Reply-To: References: <4421A3D4.5080507@egenix.com> <4421CD58.2050201@v.loewis.de> <4421D034.2040705@egenix.com> <44223EF0.2070701@v.loewis.de> Message-ID: <4422777E.7010200@gmail.com> Neal Norwitz wrote: > On 3/22/06, "Martin v. L?wis" wrote: >> M.-A. Lemburg wrote: >>> For the Snake-Farm we had a separate mailing list, so I'd prefer >>> that if possible. This lets you opt-in to the messages and also >>> makes it easier to search via the python.org search facility. >> I'll wait for other people to voice an opinion. Creating a new >> mailing list is an effort I'd like to avoid. > > Hopefully we will get it tuned better soon so there aren't so many > mails. Unfortunately, it may take a while due to the ssize_t issues > getting fixed as we find them. > > I'd be happy if the messages were on python-checkins. +1 on simply changing the messages to python-checkins instead of here - it gives the build failure better context than it gets on python-dev, and subscribing to python-checkins is a fair indication of willingness to receive frequent automated mail ;) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From mal at egenix.com Thu Mar 23 11:47:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 23 Mar 2006 11:47:17 +0100 Subject: [Python-Dev] r43214 - peps/trunk/pep-3000.txt In-Reply-To: <44219979.8050406@v.loewis.de> References: <20060322052432.419F01E4004@bag.python.org> <2mlkv2x1pa.fsf@starship.python.net> <442120A6.6050106@egenix.com> <44219979.8050406@v.loewis.de> Message-ID: <44227CB5.8020508@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg wrote: >>> And we still have someone actively interested in maintaining the OS2 >>> port, it seems. >> >> Dito for BeOS, now under the name Zeta OS. > > Who is the one interested in maintaining the BeOS port? the last > checkins related to BeOS seem to originate from the 2001/2002 period. That would be Donn Cave: http://bebits.com/app/4232 He's also the one who wrote the Bethon wrapper for the BeOS C++ API. BTW, the fact that you don't see new checkins doesn't necessarily mean that a port is no longer used. It may just be that the existing port still works without changes. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 23 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From anthony at python.org Thu Mar 23 13:31:46 2006 From: anthony at python.org (Anthony Baxter) Date: Thu, 23 Mar 2006 23:31:46 +1100 Subject: [Python-Dev] RELEASED Python 2.4.3, release candidate 1 Message-ID: <200603232331.52570.anthony@python.org> On behalf of the Python development team and the Python community, I'm happy to announce the release of Python 2.4.3 (release candidate 1). Python 2.4.3 is a bug-fix release. See the release notes at the website (also available as Misc/NEWS in the source distribution) for details of the more than 50 bugs squished in this release, including a number found by the Coverity Scan project. Assuming no major problems crop up, a final release of Python 2.4.3 will follow in about a week's time. For more information on Python 2.4.3, including download links for various platforms, release notes, and known issues, please see: http://www.python.org/2.4.3/ Highlights of this new release include: - Bug fixes. According to the release notes, at least 50 have been fixed since 2.4.2. Highlights of the previous major Python release (2.4) are available from the Python 2.4 page, at http://www.python.org/2.4/highlights.html On a personal note, according to my records this is the 25th release of Python I've made as release manager. So enjoy this silver anniversary release, Anthony Anthony Baxter anthony at python.org Python Release Manager (on behalf of the entire python-dev team) From buildbot at python.org Thu Mar 23 13:41:34 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 23 Mar 2006 12:41:34 +0000 Subject: [Python-Dev] buildbot warnings in x86 gentoo trunk Message-ID: <20060323124134.EF9171E400A@bag.python.org> The Buildbot has detected a new failure of x86 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%20gentoo%20trunk/builds/119 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: hyeshik.chang Build Had Warnings: warnings test sincerely, -The Buildbot From anthony at interlink.com.au Thu Mar 23 13:41:47 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu, 23 Mar 2006 23:41:47 +1100 Subject: [Python-Dev] release24-maint unfrozen, kinda Message-ID: <200603232341.49072.anthony@interlink.com.au> 2.4.3c1 is out, so the release24-maint branch is kinda unfrozen. I want to cut a 2.4.3 final next week, so please be very very cautious about checkins to the branch. A 2.4.4 will follow sometime in 6 months, or after 2.5 final is out, whichever is sooner. -- Anthony Baxter It's never too late to have a happy childhood. From buildbot at python.org Thu Mar 23 14:17:26 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 23 Mar 2006 13:17:26 +0000 Subject: [Python-Dev] buildbot warnings in sparc solaris10 gcc trunk Message-ID: <20060323131727.1D7821E400A@bag.python.org> The Buildbot has detected a new failure of sparc solaris10 gcc trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%20solaris10%20gcc%20trunk/builds/113 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: hyeshik.chang Build Had Warnings: warnings test sincerely, -The Buildbot From aahz at pythoncraft.com Thu Mar 23 17:38:11 2006 From: aahz at pythoncraft.com (Aahz) Date: Thu, 23 Mar 2006 08:38:11 -0800 Subject: [Python-Dev] [Fwd: buildbot warnings in amd64 gentoo trunk] In-Reply-To: <4422777E.7010200@gmail.com> References: <4421A3D4.5080507@egenix.com> <4421CD58.2050201@v.loewis.de> <4421D034.2040705@egenix.com> <44223EF0.2070701@v.loewis.de> <4422777E.7010200@gmail.com> Message-ID: <20060323163811.GB3297@panix.com> On Thu, Mar 23, 2006, Nick Coghlan wrote: > > +1 on simply changing the messages to python-checkins instead of > here - it gives the build failure better context than it gets on > python-dev, and subscribing to python-checkins is a fair indication of > willingness to receive frequent automated mail ;) +1 python-checkins -- I'm not subscribed to python-checkins, and the buildbot messages mean nothing to me. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From buildbot at python.org Thu Mar 23 19:42:08 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 23 Mar 2006 18:42:08 +0000 Subject: [Python-Dev] buildbot warnings in x86 W2k trunk Message-ID: <20060323184208.5BD201E400A@bag.python.org> The Buildbot has detected a new failure of x86 W2k trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%20W2k%20trunk/builds/122 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: martin.v.loewis Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Thu Mar 23 19:51:50 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 23 Mar 2006 18:51:50 +0000 Subject: [Python-Dev] buildbot warnings in x86 XP trunk Message-ID: <20060323185150.3933A1E400A@bag.python.org> The Buildbot has detected a new failure of x86 XP trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%20XP%20trunk/builds/120 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: martin.v.loewis Build Had Warnings: warnings test sincerely, -The Buildbot From buildbot at python.org Thu Mar 23 20:02:22 2006 From: buildbot at python.org (buildbot at python.org) Date: Thu, 23 Mar 2006 19:02:22 +0000 Subject: [Python-Dev] buildbot warnings in x86 XP-2 trunk Message-ID: <20060323190222.C4F971E400A@bag.python.org> The Buildbot has detected a new failure of x86 XP-2 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%20XP-2%20trunk/builds/96 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: hyeshik.chang,martin.v.loewis Build Had Warnings: warnings test sincerely, -The Buildbot From nnorwitz at gmail.com Thu Mar 23 20:10:03 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 23 Mar 2006 11:10:03 -0800 Subject: [Python-Dev] r43214 - peps/trunk/pep-3000.txt In-Reply-To: <44227CB5.8020508@egenix.com> References: <20060322052432.419F01E4004@bag.python.org> <2mlkv2x1pa.fsf@starship.python.net> <442120A6.6050106@egenix.com> <44219979.8050406@v.loewis.de> <44227CB5.8020508@egenix.com> Message-ID: On 3/23/06, M.-A. Lemburg wrote: > Martin v. L?wis wrote: > > M.-A. Lemburg wrote: > >>> And we still have someone actively interested in maintaining the OS2 > >>> port, it seems. > >> > >> Dito for BeOS, now under the name Zeta OS. > > > > Who is the one interested in maintaining the BeOS port? the last > > checkins related to BeOS seem to originate from the 2001/2002 period. > > That would be Donn Cave: > > http://bebits.com/app/4232 > > He's also the one who wrote the Bethon wrapper for the BeOS C++ API. > > BTW, the fact that you don't see new checkins doesn't necessarily > mean that a port is no longer used. It may just be that the existing > port still works without changes. True, but note that he says there are patches against 2.4.2. So he apparently is maintaining this outside the Python source tree. If that's already the case, is it worth us keeping all the #ifdef'ery etc in the trunk? I know Donn from his c.l.p postings, but I don't recall him on python-dev or seeing a patch. Donn if you are here, care to explain what direction you think we should take? n From chris at atlee.ca Thu Mar 23 21:12:29 2006 From: chris at atlee.ca (Chris AtLee) Date: Thu, 23 Mar 2006 15:12:29 -0500 Subject: [Python-Dev] Patch to add timestamp() method to datetime objects Message-ID: <7790b6530603231212m1b688db0s6885adf3f987ef5c@mail.gmail.com> Hi, I've just submitted patch #1457227 which adds a convenience method to datetime objects to get the timestamp. It's equivalent to time.mktime(d.timetuple()), I just wanted to save myself some typing and be able to write d.timestamp() instead. I hope I have the dst code right. Would d.utctimestamp() be useful as well? Cheers, Chris From martin at v.loewis.de Thu Mar 23 23:07:30 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 23 Mar 2006 23:07:30 +0100 Subject: [Python-Dev] r43214 - peps/trunk/pep-3000.txt In-Reply-To: <44227CB5.8020508@egenix.com> References: <20060322052432.419F01E4004@bag.python.org> <2mlkv2x1pa.fsf@starship.python.net> <442120A6.6050106@egenix.com> <44219979.8050406@v.loewis.de> <44227CB5.8020508@egenix.com> Message-ID: <44231C22.2070606@v.loewis.de> M.-A. Lemburg wrote: > That would be Donn Cave: > > http://bebits.com/app/4232 > > He's also the one who wrote the Bethon wrapper for the BeOS C++ API. > > BTW, the fact that you don't see new checkins doesn't necessarily > mean that a port is no longer used. It may just be that the existing > port still works without changes. From that page: "Includes Bethon 0.5.3 libraries, and patches to build Python 2.4.2 from source." So it looks like the patches are not being contributed back. OTOH, looking at the patches, some of them might not be suitable for inclusion, anyway (but some are, of course). Regards, Martin From martin at v.loewis.de Thu Mar 23 23:40:17 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 23 Mar 2006 23:40:17 +0100 Subject: [Python-Dev] test_quopri, test_wait3, and test_popen2 Message-ID: <442323D1.3070801@v.loewis.de> I have been looking into the (seemingly random) test_popen2 failures today, and found that it fails when the tests are run in the order given in the subject. Here is what happens: - test_quopri uses os.popen2, which in turn creates a popen2.Popen3 object. It processes stdin/stdout, but never calls wait on that the process (it can't: os.popen2 doesn't return the pid). This is a flaw (the process should be waited for) which I fixed. - test_wait3 has a loop to wait for child processes, which invokes wait repeatedly until the expected child pid is returned. In the process, it might receive the wait status from processes it didn't start. This is a flaw which I didn't address - I wonder why there is a loop, instead of just waiting once, and expecting the PID of the process that had been created just before. - test_popen2 invokes _cleanup at the end, which waits for all _active processes. Now, since test_quopri added an active process, and test_wait3 deleted its pid, _cleanup fails with the error that the pid is no longer valid. I made that failure more obvious. So here is what I did to address these: - I changed test_quopri to use subprocess.Popen().communicate() to invoke quopri as a program. communicate will wait() for the child process. - I changed test_popen2 to check at the beginning that there are no active processes. To print out the commands that are still active, I added the cmd string to the Popen objects. It turned out that this broke the Windows builds, which, in turn, resolves as a hidden bug/feature of quopri command line mode. When "python -mquopri References: <4421A3D4.5080507@egenix.com> <4421CD58.2050201@v.loewis.de> <4421D034.2040705@egenix.com> <44223EF0.2070701@v.loewis.de> <4422777E.7010200@gmail.com> Message-ID: <4423248B.8040405@v.loewis.de> Nick Coghlan wrote: > +1 on simply changing the messages to python-checkins instead of here - it > gives the build failure better context than it gets on python-dev, and > subscribing to python-checkins is a fair indication of willingness to receive > frequent automated mail ;) Ok, I changed the address to python-checkins. It might take some delay initially, because messages will have to be approved by a list moderator. Regards, Martin From nnorwitz at gmail.com Fri Mar 24 00:00:53 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 23 Mar 2006 15:00:53 -0800 Subject: [Python-Dev] test_quopri, test_wait3, and test_popen2 In-Reply-To: <442323D1.3070801@v.loewis.de> References: <442323D1.3070801@v.loewis.de> Message-ID: On 3/23/06, "Martin v. L?wis" wrote: > I have been looking into the (seemingly random) test_popen2 > failures today, and found that it fails when the tests I played with this some last night and found the same ordering. I have a different patch that also fixes the problem. It also fixes 2-3 bugs I think. Basically the child could be waited on from outside popen (or from 2 threads). The question is what should we do if that happens? I wrapped some calls and handled the exceptions. Threads can be handled fine since we have the return result. But if the child was waited on from outside popen, we don't have the status info. I'm not sure what to do about that. There is a problem with the patch that I should use self.sts if it's not -1. See bug #1183780 for the details. n From martin at v.loewis.de Fri Mar 24 08:47:18 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 24 Mar 2006 08:47:18 +0100 Subject: [Python-Dev] test_quopri, test_wait3, and test_popen2 In-Reply-To: References: <442323D1.3070801@v.loewis.de> Message-ID: <4423A406.1070503@v.loewis.de> Neal Norwitz wrote: > I played with this some last night and found the same ordering. I > have a different patch that also fixes the problem. It also fixes 2-3 > bugs I think. Basically the child could be waited on from outside > popen (or from 2 threads). The question is what should we do if that > happens? As discussed in the patch: If the application calls wait() at some point, or poll() (in the same thread or a different thread), then it is correct if .wait() will raise an os.error (ECHILD) - the pid was already waited on. OTOH, _cleanup shouldn't mess with pids that the application still might want to wait on; _cleanup should also clean out objects that already have been waited on. So my proposal is that Popen3 objects should be added to _active only in __del__. Regards, Martin From ncoghlan at gmail.com Fri Mar 24 11:15:32 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 24 Mar 2006 20:15:32 +1000 Subject: [Python-Dev] [Python-3000] Iterators for dict keys, values, and items == annoying :) In-Reply-To: <4423BF55.7030608@canterbury.ac.nz> References: <4422FC96.2020409@zope.com> <4422FFE1.8050807@colorstudy.com> <4423BF55.7030608@canterbury.ac.nz> Message-ID: <4423C6C4.8030507@gmail.com> Greg Ewing wrote: > Guido van Rossum wrote: > >> Its maps have methods to >> return keys, values and items, but these return neither new lists nor >> iterators; they return "views" which obey set (or multiset, in the >> case of items) semantics. > >> I'd like to explore this as an alternative to making keys() etc. >> return iterators. > > This sounds like a really really good idea! > > It would solve Jim's problem, because the result of > d.keys() would print out just like a real list, and > then he could backspace over the .keys() and do > something else. If these things returned iterables rather than actual iterators (i.e. similar to what xrange currently does), it would also address the fact that caching references to iterators just doesn't work. For example: keys = d.keys() big_key = max(keys) little_key = max(keys) If keys() returns a list or other iterable, all is good. If it returns an actual iterator though. . . I do find it somewhat interesting that we're considering moving the standard containers to a more numpy-ish view of the world, though (i.e. one where multiple mutable views of a data structure are common in order to avoid unnecessary copying) Would that philosophy be extended to slicing operations, though? Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From guido at python.org Fri Mar 24 16:19:04 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 24 Mar 2006 07:19:04 -0800 Subject: [Python-Dev] [Python-3000] Iterators for dict keys, values, and items == annoying :) In-Reply-To: <4423C6C4.8030507@gmail.com> References: <4422FC96.2020409@zope.com> <4422FFE1.8050807@colorstudy.com> <4423BF55.7030608@canterbury.ac.nz> <4423C6C4.8030507@gmail.com> Message-ID: On 3/24/06, Nick Coghlan wrote: > I do find it somewhat interesting that we're considering moving the standard > containers to a more numpy-ish view of the world, though (i.e. one where > multiple mutable views of a data structure are common in order to avoid > unnecessary copying) > > Would that philosophy be extended to slicing operations, though? That's a question I hadn't though about yet. I'm not sure I like it, but it sure is consistent with other view APIs; like everywhere, if you need a separate object, you can place a list() around it. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From Chris.Barker at noaa.gov Mon Mar 20 23:52:29 2006 From: Chris.Barker at noaa.gov (Chris Barker) Date: Mon, 20 Mar 2006 14:52:29 -0800 Subject: [Python-Dev] Expose the array interface in Python 2.5? Message-ID: <441F322D.5090107@noaa.gov> On 3/17/06, Thomas Heller python.net> wrote: > Accessing Python arrays (Numeric arrays, Numeric array, or Numpy > array) as ctypes arrays, and vice versa, without copying any memory, > would be a good thing. This does bring up a point. I was thinking that a really bare-bones nd-array type would be very useful by itself as a way to exchange data. However, if ctypes were to return nd-arrays from C function calls, then we would want at least a rudimentary way of directly accessing the data. I'd sure like to see indexing and slicing, at at least. Also if ctypes were to allow nd-arrays as a way to pass data into a C function, then you'd need a build-in python way to create the data in the first place. If that's too much for now, I'd still like to have the basic data structure defined in the standard library (or built-in) as soon as possible. Greg Ewing wrote: > It might be all right for writers of new > extensions, but there are existing modules (PIL, ctypes, etc.) > that already have their own way of storing data, and it > seems to me it would be easier for the maintainers of > those modules to add a new interface to the existing > data than to rearrange their internal structure to use > this new C-object. Can we have both? A defined interface, that existing code can be adapted to provide, and a new C-Object, that future code can just use. If the goal is to have as many extension types as possible use the same base object, the sooner a standard object is provided the better. There are those of us in the scientific computing community that would love to just have numpy be part of the standard library. It really is big, and maybe too special purpose for that, but at least having the core array in there would be great. For years, I've been dealing with modules like wxPython that don't understand numpy arrays. As a result, passing in a list of points to be drawn is actually faster than passing in a numpy array of points, even though the numpy array of points has the data in the exact binary representation that wxWidgets expects. The problem is that the wrapper code doesn't know that, because Robin (quite reasonably) didn't want to have wxPython depend on Numeric. While a standard interface could support this, it would be great if wxPython could also do things like pass an image buffer back to Python efficiently. Another point is that n-dimensional arrays really are very useful for all sorts of stuff that have nothing to do with high-performance Numeric computing. I find myself using numpy for almost every little script I write, even though most of it is not performance bounded at all. I suspect that if we get a n-dimensional array type into Python, one that allows n-d slicing, it will see a LOT of use by people who now think they have no use for numpy. My guess is that a 2-dimensional object array would get the most use, but why not support n-d while you're at it? Having an easy and natural way to store and manipulate a "table" of objects, for instance, would be handy for many, many users. I'm still a tiny bit confused about the proposed individual pieces involved, but I'd like to see, in order of priority (and difficulty): 1) A standard n-d array interface (numpy already defines this, but outside of the numpy community, who knows about it?) 2) A standard, n-d array C-Object, inheritable for use by various extension packages. 3) A standard n-d array object, with a python interface for indexing and slicing for access and assignment (modeled after, or better yet taken directly from, numpy). 4) A standard n-d array object that supports array-wise arithmetic and functions (ufuncs, in numpy parlance). There is no reason it has to have anything to do with the current array module. For instance, having the arrays be of fixed size is just fine. It's really not that big a deal to make a new one when you need to change the size, after all, we live with that for immutable objects, like strings. just my $0.02 -Chris From nnorwitz at gmail.com Wed Mar 22 10:47:54 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 22 Mar 2006 01:47:54 -0800 Subject: [Python-Dev] r43214 - peps/trunk/pep-3000.txt In-Reply-To: <2mlkv2x1pa.fsf@starship.python.net> References: <20060322052432.419F01E4004@bag.python.org> <2mlkv2x1pa.fsf@starship.python.net> Message-ID: [bcc python-dev to move to python-3000] On 3/22/06, Michael Hudson wrote: > "Fredrik Lundh" writes: > > > neal.norwitz wrote: > > > >> +Outstanding Issues > >> +================== > >> + > >> +* Require C99, so we can use // comments, named initializers, declare variables > >> + without introducing a new scope, among other benefits. > > > > gcc only, in other words ? > > Heh, I was going to make this point as well: it's not clear that MSVC > will ever support C99. It supports some of the features listed here, > of course, but probably won't support everything. Note this is the outstanding issues section. So everything here is a question. By the time 3k is released, who knows what compilers will support. > >> +* Remove support for old systems, including: OS2, BeOS, RISCOS, (SGI) Irix, Tru64 > > > > what's old with tru64 ? it's not that uncommon in places where Python > > has a strong presence, you can still buy AXP hardware throughout 2006, > > and HP says they'll keep developing and supporting the software platform > > at least through 2011. > > And we still have someone actively interested in maintaining the OS2 > port, it seems. OS2 was a mistake. I created this list a few days ago before Alan said he was interested in maintaining it. Tru64 is difficult (I think there are still some open bugs that go back years) because no developer has access to any of these boxes. It would be good for people interested in these platforms to speak up and offer their time or at least access to the platform so we can test. I haven't researched Tru64 status, but I remember hearing a bunch of times how it wasn't going to be supported. If we support these platforms, we still need to decide what versions. Again remember all of these are questions. n From merkosh at hadiko.de Wed Mar 22 11:12:52 2006 From: merkosh at hadiko.de (Uwe Mayer) Date: Wed, 22 Mar 2006 11:12:52 +0100 Subject: [Python-Dev] howto return malloc()ed memory from C -> Python Message-ID: <200603221112.57950.merkosh@hadiko.de> Hi, I am wrapping libsndfile to Python, using SWIG and don't know howto return a malloc()ed memory area back to Python. I've found CObjects, which would let me pass free() as cleanup function, but I'd rather return a buffer object (PyBuffer_FromReadWriteMemory) so that I could prevent copying of data, but I don't know how to prevent the memory leak then. Can I install some kind of cleanup hook? Please also reply to me personally, as I am not on the python-dev list. Thanks, Uwe -- Conway's Law: In any organization there will always be one person who knows what is going on. This person must be fired. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 191 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20060322/0b7cd7b4/attachment.pgp From charlie at begeistert.org Thu Mar 23 08:36:44 2006 From: charlie at begeistert.org (Charlie Clark) Date: Thu, 23 Mar 2006 08:36:44 +0100 Subject: [Python-Dev] [Fwd: Re: r43214 - peps/trunk/pep-3000.txt] In-Reply-To: <4421A412.80802@egenix.com> References: <4421A412.80802@egenix.com> Message-ID: <20060323083644.1928.5@Zeta.1143098496.fake> On 2006-03-22 at 20:22:58 [+0100], M.-A. Lemburg wrote: > Charlie, > > you might want to reply to this one... > > -------- Original Message -------- > Subject: Re: [Python-Dev] r43214 - peps/trunk/pep-3000.txt > Date: Wed, 22 Mar 2006 19:37:45 +0100 > From: "Martin v. L?wis" > To: M.-A. Lemburg > CC: Michael Hudson , python-dev at python.org, Fredrik > Lundh > References: <20060322052432.419F01E4004 at bag.python.org> > <2mlkv2x1pa.fsf at starship.python.net> > <442120A6.6050106 at egenix.com> > > M.-A. Lemburg wrote: > >> And we still have someone actively interested in maintaining the OS2 > >> port, it seems. > > > > Dito for BeOS, now under the name Zeta OS. > > Who is the one interested in maintaining the BeOS port? the last > checkins related to BeOS seem to originate from the 2001/2002 period. yellowTAB includes Python 2.4 as part of their ZETA OS which, as Marc-Andr? notes is a further development of BeOS. Francois Revol has done the work to keep Python up to date and working with the existing maintainer, Donn Cave, on this. It's a much more solid release than all previous versions with much greater support for Python extensions. Even Zope runs with only one slight change as flock() is not yet supported. Charlie From aahz at pythoncraft.com Fri Mar 24 18:08:50 2006 From: aahz at pythoncraft.com (Aahz) Date: Fri, 24 Mar 2006 09:08:50 -0800 Subject: [Python-Dev] howto return malloc()ed memory from C -> Python In-Reply-To: <200603221112.57950.merkosh@hadiko.de> References: <200603221112.57950.merkosh@hadiko.de> Message-ID: <20060324170850.GA10568@panix.com> On Wed, Mar 22, 2006, Uwe Mayer wrote: > > I am wrapping libsndfile to Python, using SWIG and don't know howto > return a malloc()ed memory area back to Python. Please post this question to comp.lang.python; it's not appropriate for python-dev. Note that you're unlikely to get many useful responses if you don't subscribe to c.l.py. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Look, it's your affair if you want to play with five people, but don't go calling it doubles." --John Cleese anticipates Usenet From theller at python.net Fri Mar 24 18:27:13 2006 From: theller at python.net (Thomas Heller) Date: Fri, 24 Mar 2006 18:27:13 +0100 Subject: [Python-Dev] Expose the array interface in Python 2.5? In-Reply-To: References: Message-ID: <44242BF1.50301@python.net> Travis E. Oliphant wrote: > Last year, during development of NumPy, the concept of the array > interface was developed as a means to share array_like data. It was > realized that it is this concept that needs to be explored further and > pushed into Python itself, rather than a particular incarnation of an > array. > > It is very important for many people to get access to memory with some > description of how that memory should be interpreted as an array. > Several Python packages could benefit if Python had some notion of an > array interface that was at least supported in a duck-typing fashion. I'm astonished that our BDFL doesn't speak up on this issue. IIRC, it was the (his?) plan to let the numpy community develop an interface, which would then be included into Python. > The description of what we've come up with so far and is implemented in > NumPy (and Numarray and last released Numeric) is at > > http://numeric.scipy.org/#array_interface > > Quite a few of us would love to see this get into Python proper, but > have very little free-time to spare to make sure it happens. > > Would it be possible to add at least the C-struct array interface to the > Python arrayobject in time for Python 2.5? > > Is someone on this list willing to help make it happen? What excatly would you like to see in the core? If I read the url above correctly, it would only be required to attach some attributes to existing array-like objects. Are ctypes and arrayobject the only candidates, or do you want these attributes on other data types as well (strings, unicode)? OTOH, with a 2.5 alpha release next week, it may be too late for anything... Thomas From guido at python.org Fri Mar 24 18:38:48 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 24 Mar 2006 09:38:48 -0800 Subject: [Python-Dev] Expose the array interface in Python 2.5? In-Reply-To: <44242BF1.50301@python.net> References: <44242BF1.50301@python.net> Message-ID: On 3/24/06, Thomas Heller wrote: > I'm astonished that our BDFL doesn't speak up on this issue. IIRC, > it was the (his?) plan to let the numpy community develop an interface, > which would then be included into Python. Where's the PEP? You don't expect me to argue the point for the numpy community do you? Once there's a PEP I'll read it and comment. Last I saw this thread Travis was asking for volunteers to write the PEP. Personally I think this is too close to the Python 2.5 release schedule to go ahead; but that's really up to the release managers. However, without a PEP there's an automatic "no". -- --Guido van Rossum (home page: http://www.python.org/~guido/) From theller at python.net Fri Mar 24 18:40:29 2006 From: theller at python.net (Thomas Heller) Date: Fri, 24 Mar 2006 18:40:29 +0100 Subject: [Python-Dev] Expose the array interface in Python 2.5? In-Reply-To: References: <44242BF1.50301@python.net> Message-ID: <44242F0D.9090506@python.net> Guido van Rossum wrote: > On 3/24/06, Thomas Heller wrote: >> I'm astonished that our BDFL doesn't speak up on this issue. IIRC, >> it was the (his?) plan to let the numpy community develop an interface, >> which would then be included into Python. > > Where's the PEP? You don't expect me to argue the point for the numpy > community do you? Once there's a PEP I'll read it and comment. Last I > saw this thread Travis was asking for volunteers to write the PEP. > Personally I think this is too close to the Python 2.5 release > schedule to go ahead; but that's really up to the release managers. > However, without a PEP there's an automatic "no". > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) Ok, sounds fair. Thomas From pje at telecommunity.com Fri Mar 24 22:48:17 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 24 Mar 2006 16:48:17 -0500 Subject: [Python-Dev] PEP 343: A messy contextmanager corner case Message-ID: <5.1.1.6.0.20060324162743.02056ca8@mail.telecommunity.com> I was just updating an experimental context library for the latest PEP 343 protocol, and I ran into a rather messy situation with the recently added error-suppression feature, when used in combination with @contextmanager, that caused some of my tests to fail. Basically, the problem is this. The way contextmanager is implemented in the PEP, __exit__ will always either throw an exception or return True in the exception case. It will never return a false value to reraise the original exception. The problem with this is that it requires chained __exit__ handlers to trap errors in any other __exit__ methods they call, unless contextmanager can somehow detect that the reraised exception is the same one, and suppress it. This case doesn't affect nominal use of a single contextmanager in a 'with' statement. It only affects contexts that need to call other contexts' __exit__ methods. But it seems to me that it might be a good idea to make GeneratorContextManager.__exit__() simply return False if the generator reraises an exception. That way, an __exit__() that raises an error can always be considered broken, which simplifies context chaining quite a bit. This would basically consist of adding another "except:" clause to the __exit__() method, to check if the raised value is the same value as the one that was passed in to throw(). The test_contextlib tests all pass with this change, but of course I would add a test to ensure that __exit__() returns False if the same value is reraised. I would also update the PEP to reflect the updated implementation, and to explain that __exit__() methods should not propagate exceptions unless they are unable to fulfill their contract. Any objections? From pje at telecommunity.com Fri Mar 24 23:24:09 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 24 Mar 2006 17:24:09 -0500 Subject: [Python-Dev] PEP 342 support for string exceptions in throw() Message-ID: <5.1.1.6.0.20060324171438.03bba400@mail.telecommunity.com> Should geniter.throw() issue a deprecation warning for string exceptions? My first thought was yes, since that's what raise() does. On the other hand, one of the key motivating uses for throw() is to allow exception propagation on a coroutine stack. In this use case, the original "raise" of the string exception is what should be warned about. Issuing a warning for the line of code calling "throw()" is misleading, since that is not the line that is "wrong". So, here's what I propose to do instead. Rather than support arbitrary string exceptions, which are deprecated anyway, they should only be allowed in the case where a non-None traceback argument is provided. Thus, string exceptions could be *propagated* using throw(), but not *initiated*. Meanwhile, if you throw() a string exception with no traceback argument, you would receive the same error you do now. Any objections? From guido at python.org Fri Mar 24 23:36:22 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 24 Mar 2006 14:36:22 -0800 Subject: [Python-Dev] PEP 342 support for string exceptions in throw() In-Reply-To: <5.1.1.6.0.20060324171438.03bba400@mail.telecommunity.com> References: <5.1.1.6.0.20060324171438.03bba400@mail.telecommunity.com> Message-ID: On 3/24/06, Phillip J. Eby wrote: > Should geniter.throw() issue a deprecation warning for string exceptions? > > My first thought was yes, since that's what raise() does. > > On the other hand, one of the key motivating uses for throw() is to allow > exception propagation on a coroutine stack. In this use case, the original > "raise" of the string exception is what should be warned about. Issuing a > warning for the line of code calling "throw()" is misleading, since that is > not the line that is "wrong". > > So, here's what I propose to do instead. Rather than support arbitrary > string exceptions, which are deprecated anyway, they should only be allowed > in the case where a non-None traceback argument is provided. Thus, string > exceptions could be *propagated* using throw(), but not *initiated*. > > Meanwhile, if you throw() a string exception with no traceback argument, > you would receive the same error you do now. > > Any objections? -0. I think it's overkill to warn for any string exceptions thrown this way. Since the only use case for using throw() is to pass an exception you just caught, I don't see that putting the warning is useful -- it's just more code that in practice is never triggered. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Fri Mar 24 23:40:21 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 24 Mar 2006 17:40:21 -0500 Subject: [Python-Dev] Another PEP 343 contextmanager glitch Message-ID: <5.1.1.6.0.20060324173724.03bc26d0@mail.telecommunity.com> It appears that the contextmanager implementation (both in the stdlib and as described in the PEP) doesn't correctly handle the case where the generator yields a new value in response to throw(). It responds by suppressing the original exception, but IIRC earlier versions of the PEP threw a RuntimeError for this scenario. Unless there are any objections, I'd like to restore this by replacing the "return True" that follows the throw(), with: raise RuntimeError("generator didn't stop after throw()") and again, update the PEP accordingly. From pje at telecommunity.com Fri Mar 24 23:56:39 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 24 Mar 2006 17:56:39 -0500 Subject: [Python-Dev] PEP 342 support for string exceptions in throw() In-Reply-To: References: <5.1.1.6.0.20060324171438.03bba400@mail.telecommunity.com> <5.1.1.6.0.20060324171438.03bba400@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060324175346.072c4400@mail.telecommunity.com> At 02:36 PM 3/24/2006 -0800, Guido van Rossum wrote: >I think it's overkill to warn for any string exceptions thrown this >way. Since the only use case for using throw() is to pass an exception >you just caught, I don't see that putting the warning is useful -- >it's just more code that in practice is never triggered. My proposal was that throw() should only succeed or fail, never warn. If you throw() a string exception with a traceback, it Just Works. If you throw() a string exception without a traceback, you get an immediate TypeError, just like in the 2.5 trunk now. Is that acceptable? i.e., was that what you were "-0"-ing? The only change is that throw() would now *accept* string exceptions without warning or error, if and only if you supply a traceback. That is, if you are effectively re-raising an existing exception. From guido at python.org Sat Mar 25 00:04:11 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 24 Mar 2006 15:04:11 -0800 Subject: [Python-Dev] PEP 342 support for string exceptions in throw() In-Reply-To: <5.1.1.6.0.20060324175346.072c4400@mail.telecommunity.com> References: <5.1.1.6.0.20060324171438.03bba400@mail.telecommunity.com> <5.1.1.6.0.20060324175346.072c4400@mail.telecommunity.com> Message-ID: On 3/24/06, Phillip J. Eby wrote: > At 02:36 PM 3/24/2006 -0800, Guido van Rossum wrote: > >I think it's overkill to warn for any string exceptions thrown this > >way. Since the only use case for using throw() is to pass an exception > >you just caught, I don't see that putting the warning is useful -- > >it's just more code that in practice is never triggered. > > My proposal was that throw() should only succeed or fail, never warn. > > If you throw() a string exception with a traceback, it Just Works. > > If you throw() a string exception without a traceback, you get an immediate > TypeError, just like in the 2.5 trunk now. > > Is that acceptable? i.e., was that what you were "-0"-ing? Actually that's slightly worse; I'm not sure we guarantee that exceptions you catch *always* have a traceback object associated with them. So -1. > The only change is that throw() would now *accept* string exceptions > without warning or error, if and only if you supply a traceback. That is, > if you are effectively re-raising an existing exception. The current state is that it always allows them, right? I don't see a reason to change that, and I don't think you've made a strong argument for changing it -- I don't think the use case matters. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Sat Mar 25 00:07:08 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 24 Mar 2006 18:07:08 -0500 Subject: [Python-Dev] PEP 342 support for string exceptions in throw() In-Reply-To: References: <5.1.1.6.0.20060324175346.072c4400@mail.telecommunity.com> <5.1.1.6.0.20060324171438.03bba400@mail.telecommunity.com> <5.1.1.6.0.20060324175346.072c4400@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060324180628.0732ada8@mail.telecommunity.com> At 03:04 PM 3/24/2006 -0800, Guido van Rossum wrote: >The current state is that it always allows them, right? No. It doesn't allow them. Support for string exceptions was never actually implemented; I'm trying to implement it now. From guido at python.org Sat Mar 25 00:05:46 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 24 Mar 2006 15:05:46 -0800 Subject: [Python-Dev] Another PEP 343 contextmanager glitch In-Reply-To: <5.1.1.6.0.20060324173724.03bc26d0@mail.telecommunity.com> References: <5.1.1.6.0.20060324173724.03bc26d0@mail.telecommunity.com> Message-ID: Oops, that's definitely a bug. Please go ahead and fix in both places. (I'm still trying to fathom the other PEP 343 issue you brought up.) --Guido On 3/24/06, Phillip J. Eby wrote: > It appears that the contextmanager implementation (both in the stdlib and > as described in the PEP) doesn't correctly handle the case where the > generator yields a new value in response to throw(). It responds by > suppressing the original exception, but IIRC earlier versions of the PEP > threw a RuntimeError for this scenario. > > Unless there are any objections, I'd like to restore this by replacing the > "return True" that follows the throw(), with: > > raise RuntimeError("generator didn't stop after throw()") > > and again, update the PEP accordingly. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Mar 25 00:06:58 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 24 Mar 2006 15:06:58 -0800 Subject: [Python-Dev] PEP 342 support for string exceptions in throw() In-Reply-To: <5.1.1.6.0.20060324180628.0732ada8@mail.telecommunity.com> References: <5.1.1.6.0.20060324171438.03bba400@mail.telecommunity.com> <5.1.1.6.0.20060324175346.072c4400@mail.telecommunity.com> <5.1.1.6.0.20060324180628.0732ada8@mail.telecommunity.com> Message-ID: On 3/24/06, Phillip J. Eby wrote: > At 03:04 PM 3/24/2006 -0800, Guido van Rossum wrote: > >The current state is that it always allows them, right? > > No. It doesn't allow them. Support for string exceptions was never > actually implemented; I'm trying to implement it now. Oops, I see. I'm +1 for allowing them, -1 for doing any kind of detection of situations that might not allow them in the future. (You're not warning about classic classes either are you?) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Sat Mar 25 00:16:23 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 24 Mar 2006 18:16:23 -0500 Subject: [Python-Dev] PEP 342 support for string exceptions in throw() In-Reply-To: References: <5.1.1.6.0.20060324180628.0732ada8@mail.telecommunity.com> <5.1.1.6.0.20060324171438.03bba400@mail.telecommunity.com> <5.1.1.6.0.20060324175346.072c4400@mail.telecommunity.com> <5.1.1.6.0.20060324180628.0732ada8@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060324180943.073365b0@mail.telecommunity.com> At 03:06 PM 3/24/2006 -0800, Guido van Rossum wrote: >On 3/24/06, Phillip J. Eby wrote: > > At 03:04 PM 3/24/2006 -0800, Guido van Rossum wrote: > > >The current state is that it always allows them, right? > > > > No. It doesn't allow them. Support for string exceptions was never > > actually implemented; I'm trying to implement it now. > >Oops, I see. I'm +1 for allowing them, -1 for doing any kind of >detection of situations that might not allow them in the future. Okay - so allow them without warning or error, even if somebody does 'gen.throw("abc")'? I was proposing to only add support for this if a non-None traceback is supplied. I guess I'm confused as to how you could end up with sys.exc_info()[2] not containing a traceback object. The idea here was that the only valid use case I can think of for throw()-ing a string exception is when using throw() to propagate a previously-raised exception. Since throw() is a new feature, it seems reasonable to me to not provide string exception support -- *except* for backward compatibility needs, such as propagating a string exception that was already raised elsewhere. Thus, it's not a case of removing support for string exceptions, but rather introducing a new feature that doesn't support creating new code that raises string exceptions. But, string exceptions definitely should be supported for propagation, because you a coroutine could call some old code that uses string exceptions, so you need to be able to re-throw() them to outer coroutines. >(You're not warning about classic classes either are you?) Nope. From pje at telecommunity.com Sat Mar 25 00:22:49 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 24 Mar 2006 18:22:49 -0500 Subject: [Python-Dev] Another PEP 343 contextmanager glitch In-Reply-To: References: <5.1.1.6.0.20060324173724.03bc26d0@mail.telecommunity.com> <5.1.1.6.0.20060324173724.03bc26d0@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060324181631.03b28620@mail.telecommunity.com> At 03:05 PM 3/24/2006 -0800, Guido van Rossum wrote: >Oops, that's definitely a bug. Please go ahead and fix in both places. Will do. >(I'm still trying to fathom the other PEP 343 issue you brought up.) Here's the paragraph I'm proposing to add to the PEP to explain that issue: NOTE: to facilitate chaining of contexts, __exit__() methods should *not* reraise the error that is passed in, because it is always the responsibility of the *caller* to reraise. If an __exit__() method exits with *any* exception, the caller of __exit__() should interpret this as meaning that the __exit__() method itself failed to perform the desired cleanup or other operations. Currently, the PEP is ambiguous on this point. When "with" calls __exit__() this point is irrelevant because either way, the same exception will be raised/propagated. But if you are writing Python code that explicitly uses the context protocol (e.g. the 'nested' context manager), it should be clear as to whether a given __exit__() invocation failed or succeeded. Thus, __exit__() should only raise an exception if it *failed*. (This then implies that the @contextmanager decorator must catch any re-raising done by the wrapped generator.) From guido at python.org Sat Mar 25 00:23:48 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 24 Mar 2006 15:23:48 -0800 Subject: [Python-Dev] PEP 342 support for string exceptions in throw() In-Reply-To: <5.1.1.6.0.20060324180943.073365b0@mail.telecommunity.com> References: <5.1.1.6.0.20060324171438.03bba400@mail.telecommunity.com> <5.1.1.6.0.20060324175346.072c4400@mail.telecommunity.com> <5.1.1.6.0.20060324180628.0732ada8@mail.telecommunity.com> <5.1.1.6.0.20060324180943.073365b0@mail.telecommunity.com> Message-ID: On 3/24/06, Phillip J. Eby wrote: > Okay - so allow them without warning or error, even if somebody does > 'gen.throw("abc")'? Correct. There's no current code that does this, and I doubt that few people will write any, so I don't see the value of trying to trap or warn for this case. > I was proposing to only add support for this if a non-None traceback is > supplied. I guess I'm confused as to how you could end up with > sys.exc_info()[2] not containing a traceback object. I'm not sure either -- but there's so much convoluted code that I can't prove to myself it can't happen either. > The idea here was that the only valid use case I can think of for > throw()-ing a string exception is when using throw() to propagate a > previously-raised exception. > > Since throw() is a new feature, it seems reasonable to me to not provide > string exception support -- *except* for backward compatibility needs, such > as propagating a string exception that was already raised elsewhere. > > Thus, it's not a case of removing support for string exceptions, but rather > introducing a new feature that doesn't support creating new code that > raises string exceptions. > > But, string exceptions definitely should be supported for propagation, > because you a coroutine could call some old code that uses string > exceptions, so you need to be able to re-throw() them to outer coroutines. And I see no reason to complicate the code with an additional check that doesn't serve a purpose. The purpose of reminding people not to write g.throw("abc") seems artificial to me. I'd rather see less code, meaning less maintenance, and no need to remove the check later (or when I merge your changes into the p3yk branch). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Mar 25 00:27:28 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 24 Mar 2006 15:27:28 -0800 Subject: [Python-Dev] Another PEP 343 contextmanager glitch In-Reply-To: <5.1.1.6.0.20060324181631.03b28620@mail.telecommunity.com> References: <5.1.1.6.0.20060324173724.03bc26d0@mail.telecommunity.com> <5.1.1.6.0.20060324181631.03b28620@mail.telecommunity.com> Message-ID: On 3/24/06, Phillip J. Eby wrote: > At 03:05 PM 3/24/2006 -0800, Guido van Rossum wrote: > >Oops, that's definitely a bug. Please go ahead and fix in both places. > > Will do. > > > >(I'm still trying to fathom the other PEP 343 issue you brought up.) > > Here's the paragraph I'm proposing to add to the PEP to explain that issue: > > NOTE: to facilitate chaining of contexts, __exit__() methods should > *not* reraise the error that is passed in, because it is always the > responsibility of the *caller* to reraise. If an __exit__() method > exits with *any* exception, the caller of __exit__() should > interpret this as meaning that the __exit__() method itself > failed to perform the desired cleanup or other operations. > > Currently, the PEP is ambiguous on this point. When "with" calls > __exit__() this point is irrelevant because either way, the same exception > will be raised/propagated. > > But if you are writing Python code that explicitly uses the context > protocol (e.g. the 'nested' context manager), it should be clear as to > whether a given __exit__() invocation failed or succeeded. Thus, > __exit__() should only raise an exception if it *failed*. > > (This then implies that the @contextmanager decorator must catch any > re-raising done by the wrapped generator.) I guess I like the ambiguity -- to the outer __exit__, it shouldn't make any difference whether the exception was re-raised by the inner __exit__ or by the finally clause containing it. After all, if there *wasn't* an outer __exit__, there wouldn't be any difference to the user either, whether the re-raise came from __exit__ or from finally. If you still disagree, can you produce a test case that's currently broken? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Sat Mar 25 00:43:14 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 24 Mar 2006 18:43:14 -0500 Subject: [Python-Dev] Another PEP 343 contextmanager glitch In-Reply-To: References: <5.1.1.6.0.20060324181631.03b28620@mail.telecommunity.com> <5.1.1.6.0.20060324173724.03bc26d0@mail.telecommunity.com> <5.1.1.6.0.20060324181631.03b28620@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060324183840.03bc6d78@mail.telecommunity.com> At 03:27 PM 3/24/2006 -0800, Guido van Rossum wrote: >I guess I like the ambiguity -- to the outer __exit__, it shouldn't >make any difference whether the exception was re-raised by the inner >__exit__ or by the finally clause containing it. After all, if there >*wasn't* an outer __exit__, there wouldn't be any difference to the >user either, whether the re-raise came from __exit__ or from finally. > >If you still disagree, can you produce a test case that's currently broken? I discovered the issue when I updated to the latest implementation and it broke the tests for a context manager I had written. This context manager is basically a transaction manager that accumulates context managers for resources used in the transaction, then calls all their __exit__() methods from its __exit__. It has to be able to distinguish between an __exit__() that failed (which means a critical failure of the overall transaction!) and an exit that's merely allowing the original exception to propagate (which means that the other handlers should still be invoked, and that everything's working normally). From guido at python.org Sat Mar 25 00:50:33 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 24 Mar 2006 15:50:33 -0800 Subject: [Python-Dev] Another PEP 343 contextmanager glitch In-Reply-To: <5.1.1.6.0.20060324183840.03bc6d78@mail.telecommunity.com> References: <5.1.1.6.0.20060324173724.03bc26d0@mail.telecommunity.com> <5.1.1.6.0.20060324181631.03b28620@mail.telecommunity.com> <5.1.1.6.0.20060324183840.03bc6d78@mail.telecommunity.com> Message-ID: On 3/24/06, Phillip J. Eby wrote: > At 03:27 PM 3/24/2006 -0800, Guido van Rossum wrote: > >I guess I like the ambiguity -- to the outer __exit__, it shouldn't > >make any difference whether the exception was re-raised by the inner > >__exit__ or by the finally clause containing it. After all, if there > >*wasn't* an outer __exit__, there wouldn't be any difference to the > >user either, whether the re-raise came from __exit__ or from finally. > > > >If you still disagree, can you produce a test case that's currently broken? > > I discovered the issue when I updated to the latest implementation and it > broke the tests for a context manager I had written. This context manager > is basically a transaction manager that accumulates context managers for > resources used in the transaction, then calls all their __exit__() methods > from its __exit__. It has to be able to distinguish between an __exit__() > that failed (which means a critical failure of the overall transaction!) > and an exit that's merely allowing the original exception to propagate > (which means that the other handlers should still be invoked, and that > everything's working normally). It seems you're proposing something that is *not* equivalent to with A: with B: ... since in that case a failure of the inner __exit__ (whether a re-raise or a real failure) would still invoke the outer __exit__. Is that a good idea? I'm not against recommending in the PEP that __exit__ shouldn't re-raise but instead should return False to signal a re-raise, and fixing any existing code that re-raises in __exit__. But I'm still questioning your use case; why is it important not to call the outer __exit__ methods in your case? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Sat Mar 25 00:52:21 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 24 Mar 2006 18:52:21 -0500 Subject: [Python-Dev] PEP 342 support for string exceptions in throw() In-Reply-To: References: <5.1.1.6.0.20060324180943.073365b0@mail.telecommunity.com> <5.1.1.6.0.20060324171438.03bba400@mail.telecommunity.com> <5.1.1.6.0.20060324175346.072c4400@mail.telecommunity.com> <5.1.1.6.0.20060324180628.0732ada8@mail.telecommunity.com> <5.1.1.6.0.20060324180943.073365b0@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060324184328.038ed4a8@mail.telecommunity.com> At 03:23 PM 3/24/2006 -0800, Guido van Rossum wrote: >And I see no reason to complicate the code with an additional check >that doesn't serve a purpose. The purpose of reminding people not to >write g.throw("abc") seems artificial to me. I'd rather see less code, >meaning less maintenance, and no need to remove the check later I think you're still thinking as though there was support, and I'm proposing adding new code to restrict the support. There is *no* support for string exceptions now, so we are going to be adding code, period. :) For comparison purposes, here is the patch for doing it my way: Index: Objects/genobject.c =================================================================== --- Objects/genobject.c (revision 43297) +++ Objects/genobject.c (working copy) @@ -249,7 +249,10 @@ Py_INCREF(typ); } } - else { + + /* Allow reraise of builtin string exceptions if traceback was given */ + + else if (!PyString_CheckExact(typ) || !tb) { /* Not something you can raise. throw() fails. */ PyErr_Format(PyExc_TypeError, "exceptions must be classes, or instances, not %s", And now, here's the patch for doing it your way: Index: Objects/genobject.c =================================================================== --- Objects/genobject.c (revision 43297) +++ Objects/genobject.c (working copy) @@ -249,7 +249,10 @@ Py_INCREF(typ); } } - else { + + /* Allow raising builtin string exceptions */ + + else if (!PyString_CheckExact(typ)) { /* Not something you can raise. throw() fails. */ PyErr_Format(PyExc_TypeError, "exceptions must be classes, or instances, not %s", As you can see, there are only a few characters difference between the two, so I'm not sure there's a significant maintenance overhead involved either way. Indeed, we've probably used more time discussing it than will ever be spent maintaining it. :) > (or >when I merge your changes into the p3yk branch). Actually, you won't need to merge them if Python 3000 doesn't support string exceptions. The code you've already got should be correct for that, because again, there is *no* support for string exceptions in the trunk right now. From guido at python.org Sat Mar 25 00:58:42 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 24 Mar 2006 15:58:42 -0800 Subject: [Python-Dev] PEP 342 support for string exceptions in throw() In-Reply-To: <5.1.1.6.0.20060324184328.038ed4a8@mail.telecommunity.com> References: <5.1.1.6.0.20060324171438.03bba400@mail.telecommunity.com> <5.1.1.6.0.20060324175346.072c4400@mail.telecommunity.com> <5.1.1.6.0.20060324180628.0732ada8@mail.telecommunity.com> <5.1.1.6.0.20060324180943.073365b0@mail.telecommunity.com> <5.1.1.6.0.20060324184328.038ed4a8@mail.telecommunity.com> Message-ID: OK, I give up. I'm giving you a -1 on the test for a non-empty traceback without explaining it further. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Sat Mar 25 01:01:32 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 24 Mar 2006 19:01:32 -0500 Subject: [Python-Dev] Another PEP 343 contextmanager glitch In-Reply-To: References: <5.1.1.6.0.20060324183840.03bc6d78@mail.telecommunity.com> <5.1.1.6.0.20060324173724.03bc26d0@mail.telecommunity.com> <5.1.1.6.0.20060324181631.03b28620@mail.telecommunity.com> <5.1.1.6.0.20060324183840.03bc6d78@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060324185430.07267cb8@mail.telecommunity.com> At 03:50 PM 3/24/2006 -0800, Guido van Rossum wrote: >I'm not against recommending in the PEP that __exit__ shouldn't >re-raise but instead should return False to signal a re-raise, and >fixing any existing code that re-raises in __exit__. But I'm still >questioning your use case; why is it important not to call the outer >__exit__ methods in your case? I didn't say it was important not to call them; I said it was important to be able to *tell* whether any of the __exit__ methods had actually failed, since this indicates a critical failure of transaction rollback. That is, there was some part of the transaction that could not be rolled back. This is a distinct issue from whatever caused the transaction to be aborted. Also, the __exit__ methods that failed should be logged, but not the __exit__ methods that are simply propagating an exception after performing their rollback behavior. From pje at telecommunity.com Sat Mar 25 01:08:31 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 24 Mar 2006 19:08:31 -0500 Subject: [Python-Dev] PEP 342 support for string exceptions in throw() In-Reply-To: References: <5.1.1.6.0.20060324184328.038ed4a8@mail.telecommunity.com> <5.1.1.6.0.20060324171438.03bba400@mail.telecommunity.com> <5.1.1.6.0.20060324175346.072c4400@mail.telecommunity.com> <5.1.1.6.0.20060324180628.0732ada8@mail.telecommunity.com> <5.1.1.6.0.20060324180943.073365b0@mail.telecommunity.com> <5.1.1.6.0.20060324184328.038ed4a8@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060324190721.0726c050@mail.telecommunity.com> At 03:58 PM 3/24/2006 -0800, Guido van Rossum wrote: >OK, I give up. I'm giving you a -1 on the test for a non-empty >traceback without explaining it further. Not a problem; I just wanted to make sure you were rejecting the same thing that I was proposing. :) I've checked in the "your way" version. From guido at python.org Sat Mar 25 01:34:36 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 24 Mar 2006 16:34:36 -0800 Subject: [Python-Dev] Another PEP 343 contextmanager glitch In-Reply-To: <5.1.1.6.0.20060324185430.07267cb8@mail.telecommunity.com> References: <5.1.1.6.0.20060324173724.03bc26d0@mail.telecommunity.com> <5.1.1.6.0.20060324181631.03b28620@mail.telecommunity.com> <5.1.1.6.0.20060324183840.03bc6d78@mail.telecommunity.com> <5.1.1.6.0.20060324185430.07267cb8@mail.telecommunity.com> Message-ID: On 3/24/06, Phillip J. Eby wrote: > At 03:50 PM 3/24/2006 -0800, Guido van Rossum wrote: > >I'm not against recommending in the PEP that __exit__ shouldn't > >re-raise but instead should return False to signal a re-raise, and > >fixing any existing code that re-raises in __exit__. But I'm still > >questioning your use case; why is it important not to call the outer > >__exit__ methods in your case? > > I didn't say it was important not to call them; I said it was important to > be able to *tell* whether any of the __exit__ methods had actually failed, > since this indicates a critical failure of transaction rollback. That is, > there was some part of the transaction that could not be rolled back. This > is a distinct issue from whatever caused the transaction to be > aborted. Also, the __exit__ methods that failed should be logged, but not > the __exit__ methods that are simply propagating an exception after > performing their rollback behavior. You could probably tell the difference by comparing the exception it raises to the exception you pass it. But by all means update the PEP to discourage or disallow re-raising and fix any re-raising code you find. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Sat Mar 25 02:04:50 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 24 Mar 2006 20:04:50 -0500 Subject: [Python-Dev] Another PEP 343 contextmanager glitch In-Reply-To: References: <5.1.1.6.0.20060324185430.07267cb8@mail.telecommunity.com> <5.1.1.6.0.20060324173724.03bc26d0@mail.telecommunity.com> <5.1.1.6.0.20060324181631.03b28620@mail.telecommunity.com> <5.1.1.6.0.20060324183840.03bc6d78@mail.telecommunity.com> <5.1.1.6.0.20060324185430.07267cb8@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060324195200.038ec488@mail.telecommunity.com> At 04:34 PM 3/24/2006 -0800, Guido van Rossum wrote: >You could probably tell the difference by comparing the exception it >raises to the exception you pass it. That's what I changed @contextmanager to do, because it's the only place where you have a good reason to re-raise (i.e., the generator re-raises in order to request propagation). Thus, this inspection code should only have to appear in one place: @contextmanager, rather than it having to be in every other context that needs to tell the difference. >But by all means update the PEP >to discourage or disallow re-raising and fix any re-raising code you >find. Done. By the way, I just got approval earlier this week to work on various Python 2.5 tasks as a "20% project" at OSAF, so today's my day this week to try to burn through as many outstanding Python 2.5 issues as possible. Hence all my activity on PEPs 342 and 343. My current rough plans for wsgiref, setuptools and the rest can be found at: http://peak.telecommunity.com/DevCenter/Python25 if anybody's interested. From ncoghlan at gmail.com Sat Mar 25 04:15:01 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 25 Mar 2006 13:15:01 +1000 Subject: [Python-Dev] Expose the array interface in Python 2.5? In-Reply-To: <441F322D.5090107@noaa.gov> References: <441F322D.5090107@noaa.gov> Message-ID: <4424B5B5.3050003@gmail.com> Chris Barker wrote: > I'm still a tiny bit confused about the proposed individual pieces > involved, but I'd like to see, in order of priority (and difficulty): > > 1) A standard n-d array interface (numpy already defines this, but > outside of the numpy community, who knows about it?) This is pretty much a matter of taking Travis's array interface and checking it in as a PEP. Checking it in is the easy part, but I'm not clear on what actually constitutes the array interface. The array interface is apparently here: http://numeric.scipy.org/array_interface.html but it only talks about exposing this info to Python code - it doesn't discuss how to expose or use it at the C level (unless the intent is that it should be accessed via the PyObject abstract API, in which case a PEP would need to say that explicitly). However, the top of that page references the genarray PEP: http://svn.scipy.org/svn/PEP/PEP_genarray.txt Here it gets even *more* confusing, because the dimarray objects in that PEP don't appear to expose the array interface described by the page above. If this PEP is out of date, then the reference needs to be removed until it is brought up to speed (a disclaimer on the draft PEP probably wouldn't hurt either). A useful PEP for a C-level array interface would need to cover the following three things that an extension author would need to know in order to either produce or consume generic arrays: 1. The C-level protocol for exposing and retrieving the array interface (potentially using the PyObject API to access Python-level attributes) 2. How to use the array interface to access an extension type's data 3. How to use the array interface to modify an extension type's data This is doable in whatever time frame we like, since it is documenting a convention, rather than an implementation, and so isn't coupled directly to the release cycle. However, if it's available soon (before the second alpha?), it may be possible to update arrayobject and ctypes to expose this interface for Python 2.5. > 2) A standard, n-d array C-Object, inheritable for use by various > extension packages. > > 3) A standard n-d array object, with a python interface for indexing and > slicing for access and assignment (modeled after, or better yet taken > directly from, numpy). For the sake of test writers' sanity, these would probably need to be done together so that at least some of the tests could be written in Python. Due to the conflicting slice semantics between the standard library (slices are copies) and numpy (slices are mutable views), I'd actually suggest that the Python interface for this simple object *shouldn't* support slicing (at least, not in its first incarnation). Regardless, I can't see either of these steps happening before Python 2.6. If we try to do step 2 without doing step 3, then we're stuck either writing all the test code in C, or else shipping an untested component. Neither seems like a good idea. > 4) A standard n-d array object that supports array-wise arithmetic and > functions (ufuncs, in numpy parlance). Definitely not before 2.6 :) > There is no reason it has to have anything to do with the current array > module. For instance, having the arrays be of fixed size is just fine. > It's really not that big a deal to make a new one when you need to > change the size, after all, we live with that for immutable objects, > like strings. The array module is the easiest place to put the code - then you would have "array.array" as a 1 dimensional resizeable array, and "array.dimarray" as a multi-dimensional array. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From guido at python.org Sat Mar 25 05:13:42 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 24 Mar 2006 20:13:42 -0800 Subject: [Python-Dev] Another PEP 343 contextmanager glitch In-Reply-To: <5.1.1.6.0.20060324195200.038ec488@mail.telecommunity.com> References: <5.1.1.6.0.20060324173724.03bc26d0@mail.telecommunity.com> <5.1.1.6.0.20060324181631.03b28620@mail.telecommunity.com> <5.1.1.6.0.20060324183840.03bc6d78@mail.telecommunity.com> <5.1.1.6.0.20060324185430.07267cb8@mail.telecommunity.com> <5.1.1.6.0.20060324195200.038ec488@mail.telecommunity.com> Message-ID: On 3/24/06, Phillip J. Eby wrote: > At 04:34 PM 3/24/2006 -0800, Guido van Rossum wrote: > >You could probably tell the difference by comparing the exception it > >raises to the exception you pass it. > > That's what I changed @contextmanager to do, because it's the only place > where you have a good reason to re-raise (i.e., the generator re-raises in > order to request propagation). > > Thus, this inspection code should only have to appear in one place: > @contextmanager, rather than it having to be in every other context that > needs to tell the difference. Ah. After reading what you checked in I understand it better. The issue is the slight impedance mismatch between throw() and __exit__(): throw() always has to re-raise the exception, because returning a value already has a different meaning (the generator yielded again). That's what I originally implemented for __exit__() after I realized the need for a way to swallow exceptions, but then after reasonable opposition that this would make writing a "vanilla" __exit__() a bit tricky in a way that encourages buggy code, I changed it to returning True to swallow the exception, and returning False *or* re-raising to re-raise. (returning in any way also meant to continue a non-local goto.) I think that the code you added: + except: + if sys.exc_info()[1] is not value: + raise could use a comment explaining what's going on; the difference between returning None and re-raising the exception is really quite subtle here, and only ever matters if something else besides the with-statement framework is calling __exit__. (I would add this distinction to the PEP too.) > By the way, I just got approval earlier this week to work on various Python > 2.5 tasks as a "20% project" at OSAF, so today's my day this week to try to > burn through as many outstanding Python 2.5 issues as possible. Hence all > my activity on PEPs 342 and 343. My current rough plans for wsgiref, > setuptools and the rest can be found at: > > http://peak.telecommunity.com/DevCenter/Python25 > > if anybody's interested. Great news! I'm glad you're planning to finish up the PEP 302 stuff. I'd like to see some discussion of your proposed change to .pth file processing; this could cause some surprises and will slow down all Python's start-up time due to the necessity to list every directory on sys.path. This is IMO a bad thing; there must be nothing to keep people from writing tiny scripts in Python, and slow startup time is an inhibitor there. (If you don't believe this could be a problem, try it with a long path containing several directories on NFS. This is the standard setup of Google developer workstations, and it gives one a different outlook on life.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Sat Mar 25 05:38:07 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 24 Mar 2006 23:38:07 -0500 Subject: [Python-Dev] Another PEP 343 contextmanager glitch In-Reply-To: References: <5.1.1.6.0.20060324195200.038ec488@mail.telecommunity.com> <5.1.1.6.0.20060324173724.03bc26d0@mail.telecommunity.com> <5.1.1.6.0.20060324181631.03b28620@mail.telecommunity.com> <5.1.1.6.0.20060324183840.03bc6d78@mail.telecommunity.com> <5.1.1.6.0.20060324185430.07267cb8@mail.telecommunity.com> <5.1.1.6.0.20060324195200.038ec488@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060324232344.0205dc00@mail.telecommunity.com> At 08:13 PM 3/24/2006 -0800, Guido van Rossum wrote: >I think that the code you added: > >+ except: >+ if sys.exc_info()[1] is not value: >+ raise > >could use a comment explaining what's going on; Done. > the difference between >returning None and re-raising the exception is really quite subtle >here, and only ever matters if something else besides the >with-statement framework is calling __exit__. (I would add this >distinction to the PEP too.) Also done. >I'd like to see some discussion of your proposed change to .pth file >processing; this could cause some surprises and will slow down all >Python's start-up time due to the necessity to list every directory on >sys.path. This is IMO a bad thing; there must be nothing to keep >people from writing tiny scripts in Python, and slow startup time is >an inhibitor there. Okay, I'll cross that off then. Setuptools will just have to keep using its "install a special site.py on PYTHONPATH" trick to support PYTHONPATH installs. > (If you don't believe this could be a problem, try >it with a long path containing several directories on NFS. This is the >standard setup of Google developer workstations, and it gives one a >different outlook on life.) Ouch. All that computing power and you can't even copy files to your own machines? Tsk, tsk. ;) From ncoghlan at gmail.com Sat Mar 25 16:13:25 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 26 Mar 2006 01:13:25 +1000 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <441C02C8.5080504@iinet.net.au> References: <441C02C8.5080504@iinet.net.au> Message-ID: <44255E15.5000502@gmail.com> Nick Coghlan wrote: > Should GeneratorExit inherit from Exception or BaseException? > > Currently, a generator that catches Exception and continues on to yield > another value can't be closed properly (you get a runtime error pointing out > that the generator ignored GeneratorExit). > > The only decent reference I could find to it in the old PEP 348/352 > discussions is Guido writing [1]: > >> when GeneratorExit or StopIteration >> reach the outer level of an app, it's a bug like all the others that >> bare 'except:' WANTS to catch. > > (at that point in the conversation, I believe bare except was considered the > equivalent of "except Exception:") > > While I agree with what Guido says about GeneratorExit being a bug if it > reaches the outer level of an app, it seems like a bit of a trap that a > correctly written generator can't write "except Exception:" without preceding > it with an "except GeneratorExit:" that reraises the exception. Isn't that > exactly the idiom we're trying to get rid of for SystemExit and KeyboardInterrupt? The last comment I heard from Guido on this topic was that he was still thinking about it. However, I now have an additional data point - if GeneratorExit inherits directly from BaseException, it makes it much easier to write exception handling code in generators that does the right thing on both Python 2.4 and 2.5. In 2.4, PEP 342 hasn't happened, so "except Exception:" can't misbehave in response to GeneratorExit (the latter doesn't exist, and nor does generator finalisation). If GeneratorExit inherits directly from BaseException, the code still does the right thing since the exception isn't caught. OTOH, if GeneratorExit inherits from Exception (as in current SVN), then two things will be needed to make the generator work correctly: 1. add a preceding exception clause to fix Python 2.5 behaviour: except GeneratorExit: raise except Exception: # whatever 2. add header code to the module to make it work again on Python 2.4: try: GeneratorExit except NameError: class GeneratorExit(Exception): pass IMO, that would be an ugly bit of backwards incompatibility (even though I wouldn't expect such broad exception handling in generators to be at all common). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From guido at python.org Sat Mar 25 16:43:17 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 25 Mar 2006 07:43:17 -0800 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <44255E15.5000502@gmail.com> References: <441C02C8.5080504@iinet.net.au> <44255E15.5000502@gmail.com> Message-ID: On 3/25/06, Nick Coghlan wrote: > The last comment I heard from Guido on this topic was that he was still > thinking about it. Not exactly. I'm delegating the thinking mostly to others. > However, I now have an additional data point - if GeneratorExit inherits > directly from BaseException, it makes it much easier to write exception > handling code in generators that does the right thing on both Python 2.4 and 2.5. > > In 2.4, PEP 342 hasn't happened, so "except Exception:" can't misbehave in > response to GeneratorExit (the latter doesn't exist, and nor does generator > finalisation). If GeneratorExit inherits directly from BaseException, the code > still does the right thing since the exception isn't caught. > > OTOH, if GeneratorExit inherits from Exception (as in current SVN), then two > things will be needed to make the generator work correctly: > > 1. add a preceding exception clause to fix Python 2.5 behaviour: > except GeneratorExit: > raise > except Exception: > # whatever > > 2. add header code to the module to make it work again on Python 2.4: > > try: > GeneratorExit > except NameError: > class GeneratorExit(Exception): pass > > IMO, that would be an ugly bit of backwards incompatibility (even though I > wouldn't expect such broad exception handling in generators to be at all common). I can't see all that much use for GeneratorExit in code that needs to be compatible with 2.4, since the rest of the machinery that makes exception handling around yield feasible doesn't exist. Rather than speaking of "data points" which are really just "ideas", try to come up with a data point that represents an actual (not made-up) use case to show the difference. I'm saying this because, while I believe there *may* be something here, I also believe that the decision to derive an exception from BaseException instead of Exception should not be taken lightly -- lest we set the wrong example and render the nice feature we're trying to create (that "except Exception"does the right thing almost all of the time) useless. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Sat Mar 25 19:04:23 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 26 Mar 2006 04:04:23 +1000 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: References: <441C02C8.5080504@iinet.net.au> <44255E15.5000502@gmail.com> Message-ID: <44258627.4090308@gmail.com> Guido van Rossum wrote: > On 3/25/06, Nick Coghlan wrote: >> OTOH, if GeneratorExit inherits from Exception (as in current SVN), then two >> things will be needed to make the generator work correctly: >> >> 1. add a preceding exception clause to fix Python 2.5 behaviour: >> except GeneratorExit: >> raise >> except Exception: >> # whatever >> >> 2. add header code to the module to make it work again on Python 2.4: >> >> try: >> GeneratorExit >> except NameError: >> class GeneratorExit(Exception): pass >> >> IMO, that would be an ugly bit of backwards incompatibility (even though I >> wouldn't expect such broad exception handling in generators to be at all common). > > I can't see all that much use for GeneratorExit in code that needs to > be compatible with 2.4, since the rest of the machinery that makes > exception handling around yield feasible doesn't exist. I agree entirely - my goal is to make sure it stays that way. The kind of code I'm talking about would be an *existing* Python 2.4 generator that happens to do something like: def gen(tasks): """yield the results of a bunch of task functions""" for task in tasks: try: yield (task, task()) except Exception, ex: yield ExceptionOccurred(task, ex) If you run such a generator on Python 2.5, but don't run it to completion before it is garbage collected, you will get an error message printed on stderr saying that an exception was ignored when this generator was cleaned up. If you use the new PEP 342 features to try to explicitly close it before it is garbage collected, you'll get the exception directly. The culprit is the RuntimeError raised when the generator's close() method gets upset because the generator swallowed GeneratorExit. If GeneratorExit inherits directly from BaseException, such unexpected behaviour won't happen - the only way for an existing generator to break is if it contained a bare except clause, and that code was *already* dubious (e.g. it probably swallowed KeyboardInterrupt). I don't have any actual live examples of a generator with a broad exception clause like the one above, but toy generators like the one above are legal in 2.4 and result in spurious errors with current SVN. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From python at rcn.com Sat Mar 25 19:12:26 2006 From: python at rcn.com (Raymond Hettinger) Date: Sat, 25 Mar 2006 13:12:26 -0500 Subject: [Python-Dev] GeneratorExit inheriting from Exception References: <441C02C8.5080504@iinet.net.au> <44255E15.5000502@gmail.com> <44258627.4090308@gmail.com> Message-ID: <002201c65037$ac9cac10$b83efea9@RaymondLaptop1> >> I can't see all that much use for GeneratorExit in code that needs to >> be compatible with 2.4, since the rest of the machinery that makes >> exception handling around yield feasible doesn't exist. > > I agree entirely - my goal is to make sure it stays that way. > > The kind of code I'm talking about would be an *existing* Python 2.4 generator > that happens to do something like: > > def gen(tasks): > """yield the results of a bunch of task functions""" > for task in tasks: > try: > yield (task, task()) > except Exception, ex: > yield ExceptionOccurred(task, ex) > > > If you run such a generator on Python 2.5, but don't run it to completion > before it is garbage collected, you will get an error message printed on > stderr saying that an exception was ignored when this generator was cleaned > up. If you use the new PEP 342 features to try to explicitly close it before > it is garbage collected, you'll get the exception directly. > > The culprit is the RuntimeError raised when the generator's close() method > gets upset because the generator swallowed GeneratorExit. > > If GeneratorExit inherits directly from BaseException, such unexpected > behaviour won't happen - the only way for an existing generator to break is if > it contained a bare except clause, and that code was *already* dubious (e.g. > it probably swallowed KeyboardInterrupt). > > I don't have any actual live examples of a generator with a broad exception > clause like the one above, but toy generators like the one above are legal in > 2.4 and result in spurious errors with current SVN. I can't say that I care enough about this hypothetical inter-version flimflam to warrant mucking-up the otherwise useful distinction between Exception and BaseException. special-cases-aren't-special-enough ... Raymond From guido at python.org Sat Mar 25 20:56:09 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 25 Mar 2006 11:56:09 -0800 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: <44258627.4090308@gmail.com> References: <441C02C8.5080504@iinet.net.au> <44255E15.5000502@gmail.com> <44258627.4090308@gmail.com> Message-ID: On 3/25/06, Nick Coghlan wrote: > The kind of code I'm talking about would be an *existing* Python 2.4 generator > that happens to do something like: > > def gen(tasks): > """yield the results of a bunch of task functions""" > for task in tasks: > try: > yield (task, task()) > except Exception, ex: > yield ExceptionOccurred(task, ex) This is purely hypothetical. It doesn't look like good style at all. > If you run such a generator on Python 2.5, but don't run it to completion > before it is garbage collected, you will get an error message printed on > stderr saying that an exception was ignored when this generator was cleaned > up. If you use the new PEP 342 features to try to explicitly close it before > it is garbage collected, you'll get the exception directly. I think this is fine. The code breaks with the new yield semantics. But that's because the except clause was overly broad. It's easy to rewrite it like this, which is better style anyway because the scope of the try/except is limited. try: value = (task, task()) except Exception, ex: value = ExceptionOccurred(task, ex) yield value > The culprit is the RuntimeError raised when the generator's close() method > gets upset because the generator swallowed GeneratorExit. > > If GeneratorExit inherits directly from BaseException, such unexpected > behaviour won't happen - the only way for an existing generator to break is if > it contained a bare except clause, and that code was *already* dubious (e.g. > it probably swallowed KeyboardInterrupt). > > I don't have any actual live examples of a generator with a broad exception > clause like the one above, but toy generators like the one above are legal in > 2.4 and result in spurious errors with current SVN. I don't want to cater to hypotheticals. Unless you find real code out there doing this kind of thing I don't believe the problem is real. I like to resolve corner cases so that *likely* situations are handled reasonably. Just in case you feel inclined to argue this further, let me argue that there's also a *downside* to making GeneratorExit inherit from BaseException: if it ever "leaks" out of some code that was supposed to catch it but somehow didn't, and there's an outer "except Exception:" trying to protect against buggy code, that except clause is bypassed. So perhaps we can turn this into a requirement for exceptions that inherit from BaseException instead of Exception: the chance that they get raised by buggy code should be nihil. I think that SystemExit and KeyboardExit both qualify -- the former is raised by *non-buggy* code with the intention of falling all the way through; the latter is not raised by code at all but by the end user. I don't think GeneratorExit qualifies. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Sun Mar 26 00:03:35 2006 From: barry at python.org (Barry Warsaw) Date: Sat, 25 Mar 2006 18:03:35 -0500 Subject: [Python-Dev] PySet API In-Reply-To: <004001c64d58$b0ebebf0$b83efea9@RaymondLaptop1> References: <1142705253.32074.72.camel@geddy.wooz.org> <005c01c64ac5$589859b0$b83efea9@RaymondLaptop1> <002b01c64aeb$44eb7750$b83efea9@RaymondLaptop1> <1142824236.10599.29.camel@resist.wooz.org> <00a001c64bfa$71a0ab20$b83efea9@RaymondLaptop1> <1142987079.10599.232.camel@resist.wooz.org> <004001c64d58$b0ebebf0$b83efea9@RaymondLaptop1> Message-ID: <1143327815.10799.42.camel@resist.wooz.org> On Tue, 2006-03-21 at 21:31 -0500, Raymond Hettinger wrote: > [Barry] > > Is it your intent to push for more use of the abstract API instead of > > the concrete APIs for all of Python's C data structures? Current API > > aside, are you advocating this approach for all new built-in types? > > Would you argue that Python 3.0's C API be stripped of everything but > > the abstract API and the bare essentials of the concrete API? > > It's not up to me. Perhaps someone else can chime-in about the philosophy > of how the C API is supposed to balance abstract and concrete APIs. I think it's an important point to discuss, both for trying to resolve this impasse and for helping to direct future API designs, especially as we get into Python 3.0. Maybe it will help you to understand why I want a richer concrete API. I work on an app that is deeply integrated with Python. It's hard to say whether we embed or extend -- it's a lot of both. We use Python data structures such as lists, dicts, and sets in many places as our fundamental tracking objects. So we know what we have, i.e. it's definitely a set here and another set there, and we want to merge one into the other. Or we know we have a set of foo's here and we need to iterate over them quickly (yes, in a tight loop) to count things or whatever. So there's no question that a concrete API is very useful to us. And there's no questions that snaking through the abstract API causes us real debugging pain (a point which you mostly glossed over). We understand the gotchas about reference counting and the possibilities and implications about calling back into Python. Remember, we're all consenting adults here. I don't think we're unique here, as the rich concrete API of other fundamental Python objects attests to. Your comments lead me to think that you aren't taking this important use case into account. You talk about duck typing, but I don't care about that here. I absolutely know I have a PySet, so why cause me pain to use it? > I know that the more one uses the abstract API, the more likely the code > is going to be able to accept duck typed inputs. Also, most things that > have tp_slots have a corresponding abstract method instead of tons > a concrete access points; hence, I would be supportive if you proposed a > PyObject_Clear(o) function (for calling tp_clear slots when they exist and > returning an error code when they don't). I wouldn't object to that, but it wouldn't change my mind about PySet_Clear(). I'm not arguing against a rich abstract API, I'm arguing for having a richer concrete API too. And in this case, only slightly richer. > For setobject.c, if I still have a say in the matter, my strong preference is to > keep the API minimal, expose fine-grained functions for efficiency, use > PyNumber methods for direct access to operator style set operations, > and use the abstract API for everything else. I think this is a silly stance. You agree that PySet_Next() is easier to use than the iterator API. We will definitely not use the latter, and if your position stands, then we'll just have to hack Python to add it (or implement it in an auxiliary module). But I don't want to have to do that, so I really don't understand your reluctance to add three obviously useful functions. Another point: these don't expose internal bits of the set implementation. Well, except for the opaque position pointer, but that's still enough data hiding for me because you're never supposed to /do/ anything with that variable except pass it right back to PySet_Next(). PySet_Clear() and PySet_Update() don't expose any implementation details -- that's the whole point! > P.S. One other thought: I don't want to crystalize the API in a way that > precludes > future development of the module. One possibility for the future is for updates > to take > multiple arguments such as s.update(t,u,v) causing three updates to be > folded-in at once. I don't see any way that my proposals preclude that. And besides, the three API calls I'm proposing are useful /today/. But just so we all know what we're talking about, I've uploaded the patch to SourceForge: http://sourceforge.net/tracker/index.php?func=detail&aid=1458476&group_id=5470&atid=305470 As with all good patches, there's (almost) more test code than implementation. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060325/dc65d315/attachment.pgp From barry at python.org Sun Mar 26 00:05:05 2006 From: barry at python.org (Barry Warsaw) Date: Sat, 25 Mar 2006 18:05:05 -0500 Subject: [Python-Dev] PySet API In-Reply-To: <006101c64d5c$f09c6870$b83efea9@RaymondLaptop1> References: <1142705253.32074.72.camel@geddy.wooz.org> <005c01c64ac5$589859b0$b83efea9@RaymondLaptop1> <002b01c64aeb$44eb7750$b83efea9@RaymondLaptop1> <1142824236.10599.29.camel@resist.wooz.org> <00a001c64bfa$71a0ab20$b83efea9@RaymondLaptop1> <1142987079.10599.232.camel@resist.wooz.org> <004001c64d58$b0ebebf0$b83efea9@RaymondLaptop1> <006101c64d5c$f09c6870$b83efea9@RaymondLaptop1> Message-ID: <1143327905.10793.43.camel@resist.wooz.org> On Tue, 2006-03-21 at 22:01 -0500, Raymond Hettinger wrote: > [Me] > > There is a semantic difference between > > code like s+=t and s.update(t). The former only works when t is a set > > and the latter works for any iterable. When the C code corresponds to > > the Python code, that knowledge is kept intact and there is no confusion > > between > > PyNumber_InPlaceAdd(s,t) vs PyObject_CallMethod(s, "update", "(O)", t). > > Of course, that should have been s|=t and PyNumber_InPlaceOr(). Heh, my point exactly. You wouldn't have gotten confused about PySet_Update(). :) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060325/06c891a6/attachment-0001.pgp From greg.ewing at canterbury.ac.nz Sun Mar 26 03:13:22 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 26 Mar 2006 13:13:22 +1200 Subject: [Python-Dev] Pickling problems are hard to debug Message-ID: <4425EAB2.6000504@canterbury.ac.nz> There seems to be a need for better diagnostics when pickle encounters something that can't be pickled. Recently when attempting to pickle a rather large and complicated data structure, I got the following incomprehensible message: cPickle.PicklingError: args[0] from __newobj__ args has the wrong class Trying again with protocol 1 instead of 2, I get TypeError: can't pickle function objects which I'm *guessing* is because somewhere I've tried to pickle a nested function or a bound method. But it still doesn't give me any idea *which* function I tried to pickle or where abouts it turns up in the data structure. Anyone have any ideas how the situation could be improved? At the very least, it could include some info about the type and identity of the offending object. Greg From gary at modernsongs.com Sun Mar 26 04:07:20 2006 From: gary at modernsongs.com (Gary Poster) Date: Sat, 25 Mar 2006 21:07:20 -0500 Subject: [Python-Dev] Pickling problems are hard to debug In-Reply-To: <4425EAB2.6000504@canterbury.ac.nz> References: <4425EAB2.6000504@canterbury.ac.nz> Message-ID: <9D2CB3E5-EBC1-419E-9263-992AA1B91C31@modernsongs.com> On Mar 25, 2006, at 8:13 PM, Greg Ewing wrote: > There seems to be a need for better diagnostics > when pickle encounters something that can't be > pickled. > > Recently when attempting to pickle a rather > large and complicated data structure, I got > the following incomprehensible message: > > cPickle.PicklingError: args[0] > from __newobj__ args has the wrong class > > Trying again with protocol 1 instead of 2, > I get > > TypeError: can't pickle function objects > > which I'm *guessing* is because somewhere I've > tried to pickle a nested function or a bound > method. But it still doesn't give me any idea > *which* function I tried to pickle or where > abouts it turns up in the data structure. > > Anyone have any ideas how the situation could > be improved? At the very least, it could > include some info about the type and identity > of the offending object. You are asking for ideas on how to change the pickle story to help. However, just reading your issue, I thought I might have done a debugging hack like this, at least for the protocol 1 traceback. We'll assume that the error is more mysterious than what I've manufactured here. >>> import cPickle >>> cPickle.dumps({'foo': lambda: 42}) Traceback (most recent call last): File "", line 1, in ? File "/Library/Frameworks/Python.framework/Versions/2.4/lib/ python2.4/copy_reg.py", line 69, in _reduce_ex raise TypeError, "can't pickle %s objects" % base.__name__ TypeError: can't pickle function objects >>> import copy_reg >>> def debug(obj): ... import pdb; pdb.set_trace() ... >>> import types >>> copy_reg.pickle(types.FunctionType, debug) >>> cPickle.dumps({'foo': lambda: 42}) --Return-- > (2)debug()->None (Pdb) p obj at 0x63230> (Pdb) p obj.__module__ '__main__' I also might have used pickle, rather than cPickle, to try and see what happened, if that ended up being necessary. I don't use protocol 2 much: that error message in particular looked a bit difficult, and my hack might not be any help there. I agree that it would be nice to have a better message there, in particular. back to lurking... Gary From ncoghlan at gmail.com Sun Mar 26 04:59:29 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 26 Mar 2006 12:59:29 +1000 Subject: [Python-Dev] GeneratorExit inheriting from Exception In-Reply-To: References: <441C02C8.5080504@iinet.net.au> <44255E15.5000502@gmail.com> <44258627.4090308@gmail.com> Message-ID: <44260391.6080505@gmail.com> Guido van Rossum wrote: > On 3/25/06, Nick Coghlan wrote: >> The kind of code I'm talking about would be an *existing* Python 2.4 generator >> that happens to do something like: >> >> def gen(tasks): >> """yield the results of a bunch of task functions""" >> for task in tasks: >> try: >> yield (task, task()) >> except Exception, ex: >> yield ExceptionOccurred(task, ex) > > This is purely hypothetical. It doesn't look like good style at all. > >> If you run such a generator on Python 2.5, but don't run it to completion >> before it is garbage collected, you will get an error message printed on >> stderr saying that an exception was ignored when this generator was cleaned >> up. If you use the new PEP 342 features to try to explicitly close it before >> it is garbage collected, you'll get the exception directly. > > I think this is fine. The code breaks with the new yield semantics. > But that's because the except clause was overly broad. It's easy to > rewrite it like this, which is better style anyway because the scope > of the try/except is limited. > > try: > value = (task, task()) > except Exception, ex: > value = ExceptionOccurred(task, ex) > yield value Works for me. Consider the issue dropped :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From raymond.hettinger at verizon.net Sun Mar 26 05:05:59 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sat, 25 Mar 2006 22:05:59 -0500 Subject: [Python-Dev] PySet API References: <1142705253.32074.72.camel@geddy.wooz.org> <005c01c64ac5$589859b0$b83efea9@RaymondLaptop1> <002b01c64aeb$44eb7750$b83efea9@RaymondLaptop1> <1142824236.10599.29.camel@resist.wooz.org> <00a001c64bfa$71a0ab20$b83efea9@RaymondLaptop1> <1142987079.10599.232.camel@resist.wooz.org> <004001c64d58$b0ebebf0$b83efea9@RaymondLaptop1> <1143327815.10799.42.camel@resist.wooz.org> Message-ID: <001a01c65082$35a4c840$b83efea9@RaymondLaptop1> [Barry] > Maybe it will help you to understand why I want a richer concrete API. > I work on an app that is deeply integrated with Python. It's hard to > say whether we embed or extend -- it's a lot of both. We use Python > data structures such as lists, dicts, and sets in many places as our > fundamental tracking objects. In such an app, it would be trival to write a header: #define BarrySet_Clear(s) PyObject_CallMethod(s, "clear", NULL) Still, PyObject_Clear(s) would be better. Better still would be to examine the actual uses in the app. I suspect that most code that clears a set and then rebuilds it would be better-off starting with a new empty set (and because of freelisting, that is a very fast operation). Likewise, it only takes a one-line header to define BarrySet_Update(s). I do not want that part of the C API exposed yet. It is still under development and may eventually become a function with a variable length argument list. It's bogus to say there is some app critical need. Afterall, these are both one-line defines if you personally crave them. There's no speed argument here either -- saving an O(1) dispatch step in an O(n) operation. > there's no questions that snaking through the abstract API causes us > real debugging pain I honestly don't follow you here. Doesn't your debugger have options for step-over and step-into? Are you debugging the set module or your client code? Besides, these are all high volume functions -- do you really want to trace through the internal mechanics of set_clear? Internally, this code has special cases for small and large table sizes, it does a pointer swap with an empty table to avoid mid-stream resize issues, it treats dummy entries and active entries as being the same, and it's not at all beautiful. Ergo, it is not something you want to be tracing through. The debugging argument is bogus. > You agree that PySet_Next() is easier to use than the iterator API. > We will definitely not use the latter, and if your position stands, then > we'll just have to hack Python to add it (or implement it in an auxiliary > module). If you're dead-set against using the iterator API, then maybe there is something wrong with the API. You should probably start a new thread on why you detest the iterator API and see if there are ways to improve it. Avoidance of the iterator protocol is no reason to proliferate the _Next() api across other collections. That would be a mistake. It is a bug-factory. Any operation which could potentially call back arbitrary Python code can also potentially trigger a resize or table update, leaving an invalid pointer. Something as simple as PyObject_Hash(k) can trigger a callback. Usually with code like this, it would take Armin less than five minutes to write a pure Python crasher. If you absolutely must go against my recommendation, can we compromise with a semi-private _PySet_Next() so that you have a hook but without mucking-up the public API for the rest of the world? > You talk about duck typing, but I don't care about that here. It's one of the virtues of Python that gets reflected in the abstract API. IMO, it's nice that PyObject_Dir(o) corresponds to "dir(o)" and the same for hash(o), repr(o), etc. I just hope that by hardwiring data types in stone, that your app doesn't become rigid and impossible to change. I certainly do not recommend that other people adopt this coding style (avoidance of iterators, duplication of abstact api functions in concrete form, etc.) If you're experiencing debugging pain, it may be that avoidance of abstraction is the root cause. >> I would be supportive if you proposed a PyObject_Clear(o) function >> (for calling tp_clear slots when they exist and >> returning an error code when they don't). > > I wouldn't object to that, but it wouldn't change my mind about > PySet_Clear(). This is plain evidence that something is wrong with your approach. While possibly necessary in your environment, the rest of mankind should not have to stomach this kind of API clutter. From tim.peters at gmail.com Sun Mar 26 07:20:09 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 26 Mar 2006 00:20:09 -0500 Subject: [Python-Dev] Prevalence of low-level memory abuse? Message-ID: <1f7befae0603252120w630604dchea878e5db5eb2a71@mail.gmail.com> When Python's small-object memory allocator was introduced, some horrid hacks came with it to map PyMem_{Del, DEL} and PyMem_{Free, FREE} to PyObject_{Free, FREE}. This was to cater to less than a handful of extension modules found at the time that obtained memory for an object via PyObject_{New, NEW}, but released that memory via the insanely mismatched PyMem_{Del, DEL} or PyMem_{Free, FREE}. Since such combinations were found rarely in real life, have been officially forbidden for years, the hacks are ugly & hard to understand, and the hacks needlessly slow PyMem_{Del, DEL, Free, FREE}, I'm trying to get rid of them now. Alas, in a release(*) build, Python's test suite segfaulted all over the place. So far I found one smoking gun: in _subprocess.c, sp_handle_new() gets memory via PyObject_NEW(), but sp_handle_dealloc() releases that memory via PyMem_DEL(). That's nuts, and after removing the now-ancient hacks obtains the memory from obmalloc but releases it directly to the system free(). That ends up corrupting both obmalloc's _and_ the platform C library's ultra-low-level memory bookkeeping bytes. Since this wasn't common before, has it become common since then :-)? I checked Zope and ZODB a long time ago, and there were no PyMem/PyObject mismatches there. See any in your code? (*) Nothing failed in a debug build, since all PyObject_ and PyMem_ calls go thru obmalloc in a debug build. Note that, because of this, the buildbot test runs wouldn't have detected anything wrong. From aahz at pythoncraft.com Sun Mar 26 07:57:36 2006 From: aahz at pythoncraft.com (Aahz) Date: Sat, 25 Mar 2006 21:57:36 -0800 Subject: [Python-Dev] PySet API In-Reply-To: <001a01c65082$35a4c840$b83efea9@RaymondLaptop1> References: <1142705253.32074.72.camel@geddy.wooz.org> <005c01c64ac5$589859b0$b83efea9@RaymondLaptop1> <002b01c64aeb$44eb7750$b83efea9@RaymondLaptop1> <1142824236.10599.29.camel@resist.wooz.org> <00a001c64bfa$71a0ab20$b83efea9@RaymondLaptop1> <1142987079.10599.232.camel@resist.wooz.org> <004001c64d58$b0ebebf0$b83efea9@RaymondLaptop1> <1143327815.10799.42.camel@resist.wooz.org> <001a01c65082$35a4c840$b83efea9@RaymondLaptop1> Message-ID: <20060326055736.GA24140@panix.com> I'd really like to see someone else who understands the issues (i.e. using the Python C-API) weigh in. Both Barry and Raymond are clever programmers who generally understand what's Pythonic, and I find myself agreeing with whoever posted last. ;-) Having another perspective would probably shed some light here. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Look, it's your affair if you want to play with five people, but don't go calling it doubles." --John Cleese anticipates Usenet From aleaxit at gmail.com Sun Mar 26 08:57:32 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Sat, 25 Mar 2006 22:57:32 -0800 Subject: [Python-Dev] PySet API In-Reply-To: <20060326055736.GA24140@panix.com> References: <1142705253.32074.72.camel@geddy.wooz.org> <005c01c64ac5$589859b0$b83efea9@RaymondLaptop1> <002b01c64aeb$44eb7750$b83efea9@RaymondLaptop1> <1142824236.10599.29.camel@resist.wooz.org> <00a001c64bfa$71a0ab20$b83efea9@RaymondLaptop1> <1142987079.10599.232.camel@resist.wooz.org> <004001c64d58$b0ebebf0$b83efea9@RaymondLaptop1> <1143327815.10799.42.camel@resist.wooz.org> <001a01c65082$35a4c840$b83efea9@RaymondLaptop1> <20060326055736.GA24140@panix.com> Message-ID: On Mar 25, 2006, at 9:57 PM, Aahz wrote: > I'd really like to see someone else who understands the issues (i.e. > using the Python C-API) weigh in. Both Barry and Raymond are clever > programmers who generally understand what's Pythonic, and I find > myself > agreeing with whoever posted last. ;-) Having another perspective > would probably shed some light here. My general preference is rather well-known, and I quote the advice I gave in "Python in a Nutshell"...: """ Some of the functions callable on specifically-typed objects [...] duplicate functionality that is also available from PyObject_ functions; in these cases, you should almost invariably use the more general PyObject_ function instead. I don?t cover such almost- redundant functions in this book. """ However, I don't go as far as suggesting PyObject_CallMethod and the like... I'd much rather have abstract-layer PyObject_... functions, as long as they're applicable to two or more concrete built-in types (for example, IMHO adding PyObject_Clear is a no-brainer -- it's obviously right). And I'm on the fence regarding the specific issue of PySet_Next. So, having carefully staked out a position smack in the middle, I cheerfully now expect to be fired upon from both sides!-) Alex From martin at v.loewis.de Sun Mar 26 11:55:31 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 26 Mar 2006 11:55:31 +0200 Subject: [Python-Dev] Pickling problems are hard to debug In-Reply-To: <4425EAB2.6000504@canterbury.ac.nz> References: <4425EAB2.6000504@canterbury.ac.nz> Message-ID: <44266513.10104@v.loewis.de> Greg Ewing wrote: > Anyone have any ideas how the situation could > be improved? As always: on a case-by-case basis. If you find a specific case where you think the diagnosis should be better, make it better for this case. Perhaps some generalization arises while doing so, but if not, atleast this specific case gets improved. While doing so, try to cover similar cases in the process, e.g. different pickle protocols, and different pickle implementations. Regards, Martin From greg.ewing at canterbury.ac.nz Sun Mar 26 14:39:07 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 27 Mar 2006 00:39:07 +1200 Subject: [Python-Dev] Class decorators Message-ID: <44268B6B.9070305@canterbury.ac.nz> I've just been playing around with metaclasses, and I think I've stumbled across a reason for having class decorators as an alternative to metaclasses for some purposes. The metaclass I wrote was for the purpose of adding a class to a registry, the reason for which isn't important here. It worked, but I was surprised to find that it not only registered the classes that I made it the metaclass of, but all subclasses of those classes as well. I'm not sure whether that's really the behaviour I want, and I can imagine some cases in which it's definitely not what I'd want. The general principle brought out here is that when you use a metaclass, it gets inherited by subclasses, but if we had class decorators, they would only affect to the classes that you explicitly applied them to. I think there are uses for both behaviours. Greg From raymond.hettinger at verizon.net Sun Mar 26 17:38:30 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sun, 26 Mar 2006 10:38:30 -0500 Subject: [Python-Dev] PySet API References: <1142705253.32074.72.camel@geddy.wooz.org><005c01c64ac5$589859b0$b83efea9@RaymondLaptop1><002b01c64aeb$44eb7750$b83efea9@RaymondLaptop1><1142824236.10599.29.camel@resist.wooz.org><00a001c64bfa$71a0ab20$b83efea9@RaymondLaptop1><1142987079.10599.232.camel@resist.wooz.org><004001c64d58$b0ebebf0$b83efea9@RaymondLaptop1><1143327815.10799.42.camel@resist.wooz.org><001a01c65082$35a4c840$b83efea9@RaymondLaptop1><20060326055736.GA24140@panix.com> Message-ID: <006401c650eb$55e140c0$b83efea9@RaymondLaptop1> [Alex] > And I'm on the fence regarding the specific issue of PySet_Next. > > So, having carefully staked out a position smack in the middle, I > cheerfully now expect to be fired upon from both sides!-) Okay, here's the first cheap shot ;-) Which of the following pieces of code is preferable? The first loops with the iterator protocol and the second loops with the _next protocol. static long frozenset_hash(PyObject *self) { PySetObject *so = (PySetObject *)self; long h, hash = 0; PyObject *it, *key; if (so->hash != -1) return so->hash; it = PyObject_GetIter(self); if (it == NULL) return -1; while ((key = PyIter_Next(it)) != NULL) { h = PyObject_Hash(key); Py_DECREF(key); if (h == -1) { Py_DECREF(it); return -1; } hash ^= h * 3644798167; } Py_DECREF(it); if (PyErr_Occurred()) return -1; if (hash == -1) hash = 590923713L; so->hash = hash; return hash; } static long frozenset_hash(PyObject *self) { PySetObject *so = (PySetObject *)self; long h, hash = 0; PyObject *key; Py_ssize_t pos = 0; if (so->hash != -1) return so->hash; while (set_next(so, &pos, &key)) { h = PyObject_Hash(key); if (h == -1) { return -1; } hash ^= h * 3644798167; } if (hash == -1) hash = 590923713L; so->hash = hash; return hash; } From aahz at pythoncraft.com Sun Mar 26 18:07:28 2006 From: aahz at pythoncraft.com (Aahz) Date: Sun, 26 Mar 2006 08:07:28 -0800 Subject: [Python-Dev] PySet API In-Reply-To: <006401c650eb$55e140c0$b83efea9@RaymondLaptop1> References: <006401c650eb$55e140c0$b83efea9@RaymondLaptop1> Message-ID: <20060326160728.GA6403@panix.com> On Sun, Mar 26, 2006, Raymond Hettinger wrote: > [Alex] >> >> And I'm on the fence regarding the specific issue of PySet_Next. >> >> So, having carefully staked out a position smack in the middle, I >> cheerfully now expect to be fired upon from both sides!-) > > Okay, here's the first cheap shot ;-) Which of the following pieces of > code is preferable? The first loops with the iterator protocol and > the second loops with the _next protocol. Speaking as a person who does relatively little C programming, I don't see much difference between them. The first example is more Pythonic -- for Python. I agree with Barry that it's not much of a virtue for C code. However, I do have one nitpick with both your examples; I don't know whether this is an artifact of them being examples: > hash ^= h * 3644798167; Seems to me that magic numbers like this need to be made constants and explained with a comment. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Look, it's your affair if you want to play with five people, but don't go calling it doubles." --John Cleese anticipates Usenet From raymond.hettinger at verizon.net Sun Mar 26 18:43:35 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sun, 26 Mar 2006 11:43:35 -0500 Subject: [Python-Dev] PySet API References: <006401c650eb$55e140c0$b83efea9@RaymondLaptop1> <20060326160728.GA6403@panix.com> Message-ID: <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> [Aahz] > Speaking as a person who does relatively little C programming, I don't > see much difference between them. The first example is more Pythonic -- > for Python. I agree with Barry that it's not much of a virtue for C > code. It was a trick question. Everyone is supposed to be attracted to the _next version because it is shorter, faster, and takes less ref counting management. However, the _next version has a hard-to-find bug. The call to PyObject_Hash() can trigger arbitrary Python code and possibly mutate the table, leaving pointers to invalid memory addresses. It would likely take Armin less than five minutes to write a pure Python crasher for the code. And THAT is why PySet_Next() should never come into being. The iterator form is more duck-typable and re-usable than the set specific _next version, but the example was chosen to take that issue off of the table and just focus on mutation issues. > However, I do have one nitpick with both your examples; I don't know > whether this is an artifact of them being examples: > >> hash ^= h * 3644798167; > > Seems to me that magic numbers like this need to be made constants and > explained with a comment FWIW, the actual code does have comments. I stripped them out of the posting because they weren't relevant to the code comparison. Raymond From aleaxit at gmail.com Sun Mar 26 18:55:03 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Sun, 26 Mar 2006 08:55:03 -0800 Subject: [Python-Dev] PySet API In-Reply-To: <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> References: <006401c650eb$55e140c0$b83efea9@RaymondLaptop1> <20060326160728.GA6403@panix.com> <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> Message-ID: On Mar 26, 2006, at 8:43 AM, Raymond Hettinger wrote: > [Aahz] >> Speaking as a person who does relatively little C programming, I >> don't >> see much difference between them. The first example is more >> Pythonic -- >> for Python. I agree with Barry that it's not much of a virtue for C >> code. > > It was a trick question. Everyone is supposed to be attracted to > the _next > version because it is shorter, faster, and takes less ref counting > management. > However, the _next version has a hard-to-find bug. The call to > PyObject_Hash() > can trigger arbitrary Python code and possibly mutate the table, > leaving > pointers to invalid memory addresses. It would likely take Armin > less than five > minutes to write a pure Python crasher for the code. And THAT is why > PySet_Next() should never come into being. Sure, accidentally mutating underlying iterables is a subtle (but alas frequent) bug, but I don't see why it should be any harsher when the loop is using a hypothetical PySet_Next than when it is using PyIter_Next -- whatever precautions the latter takes to detect the bug and raise an exception instead of crashing, wouldn't it be at least as feasible for PySet_Next to take similar precautions (probably easier, since PySet_Next need only worry about one concrete case rather than an arbitrary variety)? What does PyDict_Next do in similar cases, and why couldn't PySet_Next behave similarly? (Yes, I could/should look it up myself, but I'm supposed to be working on the 2nd Ed of the Nutshell, whose deadline is getting worryingly close...;-). Alex From raymond.hettinger at verizon.net Sun Mar 26 19:24:19 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sun, 26 Mar 2006 12:24:19 -0500 Subject: [Python-Dev] PySet API References: <006401c650eb$55e140c0$b83efea9@RaymondLaptop1> <20060326160728.GA6403@panix.com> <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> Message-ID: <009e01c650fa$1e6e0290$b83efea9@RaymondLaptop1> [Alex] > Sure, accidentally mutating underlying iterables is a subtle (but alas > frequent) bug, but I don't see why it should be any harsher when the loop is > using a hypothetical PySet_Next than when it is using PyIter_Next -- whatever > precautions the latter takes to detect the bug and raise an exception instead > of crashing, wouldn't it be at least as feasible for PySet_Next to take > similar precautions The difference is that the PySet_Next returns pointers to the table keys and that the mutation occurs AFTER the call to PySet_Next, leaving pointers to invalid addresses. IOW, the function cannot detect the mutation. PyIter_Next on the other hand returns an object (not a pointer to an object such as those in the hash table). If the table has mutated before the function is called, then it simply raises an exception instead of returning an object. If the table mutates afterwards, it is no big deal because the returned object is still valid. FWIW, here's an easier to understand example of the same ilk (taken from real code): s = PyString_AS_STRING(item); Py_DECREF(item); if (s == NULL) break; x = strtol(s, &endptr, 10); The problem, of course, is that the decref can render the string pointer invalid. The correct code moves the decref after the strtol() call and inside the conditional. This is at the core of the issue. I don't want the set iteration API to return pointers inside the table. The PyIter_Next API takes a couple more lines but is easy to get correct and has nice duck-typing properties. For dicts, the _next api is worth the risk because it saves a double lookup and because there are legitimate use cases for changing the contents of the value field directly inside the hash table. For sets, those arguments don't apply. We have a safe way that takes a couple more lines and a proposed second-way-to-do-it that is dangerously attractive, yet somewhat unsafe. For that reason, I say no to PySet_Next(). Hopefully, as the module author and principal maintainer, I get some say in the matter. Raymond Nothing is more conducive to peace of mind than not having any opinions at all. -- Georg Christoph Lichtenberg From martin at v.loewis.de Sun Mar 26 19:59:26 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 26 Mar 2006 19:59:26 +0200 Subject: [Python-Dev] PySet_Next (Was: PySet API) In-Reply-To: <009e01c650fa$1e6e0290$b83efea9@RaymondLaptop1> References: <006401c650eb$55e140c0$b83efea9@RaymondLaptop1> <20060326160728.GA6403@panix.com> <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> <009e01c650fa$1e6e0290$b83efea9@RaymondLaptop1> Message-ID: <4426D67E.6040303@v.loewis.de> Raymond Hettinger wrote: > The difference is that the PySet_Next returns pointers to the table keys and > that the mutation occurs AFTER the call to PySet_Next, leaving pointers to > invalid addresses. IOW, the function cannot detect the mutation. I'm coming late to the discussion: where did anybody ever suggest that PySet_Next should return a pointer into the set? Looking over the entire discussion, I could not find any mentioning of a specific API. If it is similar to PyDict_Next, it will have PyObject** /input/ variables, which are really meant as PyObject* /output/ variables. But yes, PyDict_Next returns a borrowed reference, so if the dictionary mutates between calls, your borrowed reference might become stale. > PyIter_Next on the other hand returns an object (not a pointer to an > object such as those in the hash table). PyIter_Next behaves identical wrt. to result types to PyDict_Next. The difference is that PyIter_Next always returns a new reference (or NULL in case of an exception). For the caller, a clear usage strategy follows from this: either discard the references before making a potentially-mutating call, or Py_INCREF the set element before making that mutating call. Of course, *after* you made the mutating call, your iteration position might be bogus, as the set might have been reorganized. If the position is represented as a Py_ssize_t (as it is for PyDict_Next), the only consequence of continuing the iteration is that you might see elements twice or not at all - you cannot cause a crash with that. Regards, Martin From raymond.hettinger at verizon.net Sun Mar 26 20:24:25 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sun, 26 Mar 2006 13:24:25 -0500 Subject: [Python-Dev] PySet_Next (Was: PySet API) References: <006401c650eb$55e140c0$b83efea9@RaymondLaptop1> <20060326160728.GA6403@panix.com> <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> <009e01c650fa$1e6e0290$b83efea9@RaymondLaptop1> <4426D67E.6040303@v.loewis.de> Message-ID: <00cd01c65102$83d90460$b83efea9@RaymondLaptop1> >> The difference is that the PySet_Next returns pointers to the table keys and >> that the mutation occurs AFTER the call to PySet_Next, leaving pointers to >> invalid addresses. IOW, the function cannot detect the mutation. > > I'm coming late to the discussion: where did anybody ever suggest that > PySet_Next should return a pointer into the set? Looking over the entire > discussion, I could not find any mentioning of a specific API. Pardon, I bungled the terminology. PySet_Next returns a borrowed reference. That is problematic is arbitrary Python code can be run afterwards (such as PyObject_Hash in the example). We could make a version that returns a new reference or immediately Py_INCREF the reference but then PySet_Next() loses its charm and you might as well be using PyIter_Next(). Aside from bad pointers, the issue of mid-stream table mutation has other reliability issues stemming from the contents of the table potentially changing in the an arbitrary way as the iteration proceeds. That means you can make very few guarantees about the meaningfulness of the results even if you don't crash due to a bad pointer. We have a perfectly good way to iterate with PyIter_Next(). It may take a couple of extra lines, but it is easy to get correct and has no surprises. It seems that the only issue is that Barry says that he refuses to use the iterator protocol. Heck, just turn it into a list and index directly. There is no need to muck-up the set api for this. Raymond From martin at v.loewis.de Sun Mar 26 21:50:43 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 26 Mar 2006 21:50:43 +0200 Subject: [Python-Dev] PySet_Next (Was: PySet API) In-Reply-To: <00cd01c65102$83d90460$b83efea9@RaymondLaptop1> References: <006401c650eb$55e140c0$b83efea9@RaymondLaptop1> <20060326160728.GA6403@panix.com> <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> <009e01c650fa$1e6e0290$b83efea9@RaymondLaptop1> <4426D67E.6040303@v.loewis.de> <00cd01c65102$83d90460$b83efea9@RaymondLaptop1> Message-ID: <4426F093.3040808@v.loewis.de> Raymond Hettinger wrote: > Pardon, I bungled the terminology. PySet_Next returns a borrowed > reference. That is problematic is arbitrary Python code can be run > afterwards (such as PyObject_Hash in the example). Not really. It is under the control of the caller of PySet_Next what (if any) Python code is invoked, and getting this correct is straight-forward (once you know that it yields borrowed references). I don't know what specific application Barry has in mind, but I'm sure he can get it right (although it might be an interesting experiment to test that theory :-) In general, I would expect that people find it easier to get code involving PyDict_Next right than code dealing with iterators - primarily because of the error cases you have to consider. > We have a perfectly good way to iterate with PyIter_Next(). It may take > a couple of extra lines, but it is easy to get correct and has no > surprises. It seems that the only issue is that Barry says that he > refuses to use the iterator protocol. Heck, just turn it into a list > and index directly. There is no need to muck-up the set api for this. I don't care that much either way, although I would prefer to see an actual, current use case for PySet_Next, rather than theoretical, made-up examples. I don't expect to use Python sets in C code at all, personally. Regards, Martin From tim.peters at gmail.com Mon Mar 27 01:29:42 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 26 Mar 2006 18:29:42 -0500 Subject: [Python-Dev] Prevalence of low-level memory abuse? In-Reply-To: <1f7befae0603252120w630604dchea878e5db5eb2a71@mail.gmail.com> References: <1f7befae0603252120w630604dchea878e5db5eb2a71@mail.gmail.com> Message-ID: <1f7befae0603261529m3dc2989ftcf4f4c919517916d@mail.gmail.com> [Tim, on PyMem vs PyObject memory mismatches] > ... Alas, in a release(*) build, Python's test suite segfaulted all over the place. > > So far I found one smoking gun: in _subprocess.c, ... And Python's front end was chock full o' these, sometimes even getting memory via PyMem and releasing it via PyObject. I checked the change in, under the theory that there's never going to be a better time to clean up this mess. As the checkin comment says, If extension modules in real life prove as sloppy as Python's front end, we'll have to revert the objimpl.h + pymem.h part of this patch. Note that no problems will show up in a debug build (all calls still go thru obmalloc then). Problems will show up only in a release build, most likely segfaults. From tim.peters at gmail.com Mon Mar 27 01:58:02 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 26 Mar 2006 18:58:02 -0500 Subject: [Python-Dev] Prevalence of low-level memory abuse? In-Reply-To: <44272813.2090407@gradient.cis.upenn.edu> References: <1f7befae0603252120w630604dchea878e5db5eb2a71@mail.gmail.com> <1f7befae0603261529m3dc2989ftcf4f4c919517916d@mail.gmail.com> <44272813.2090407@gradient.cis.upenn.edu> Message-ID: <1f7befae0603261558t37f5b91cv31e68f91c7f3f997@mail.gmail.com> [Edward Loper] > Could the debug build's macros for PyMem/PyObject_new/free be modified > to check for mismatches? Or would storing information about which > method was used to allocate each pointer be too expensive? Perhaps a > special build could be used to check for mismatches? It's partly possible (e.g., it's impossible to know whether a blob of memory was obtained by calling malloc() directly). If someone wants to do it (I do not), the debug build adds 8 bytes to each side of each memory block obtained via each PyMem and PyObject malloc/realloc call, and one of the (current) 8 FORBIDDEN_BYTEs could be used to store flags without significant loss of functionality. It would make a decent enhancement. From edloper at gradient.cis.upenn.edu Mon Mar 27 01:47:31 2006 From: edloper at gradient.cis.upenn.edu (Edward Loper) Date: Sun, 26 Mar 2006 18:47:31 -0500 Subject: [Python-Dev] Prevalence of low-level memory abuse? In-Reply-To: <1f7befae0603261529m3dc2989ftcf4f4c919517916d@mail.gmail.com> References: <1f7befae0603252120w630604dchea878e5db5eb2a71@mail.gmail.com> <1f7befae0603261529m3dc2989ftcf4f4c919517916d@mail.gmail.com> Message-ID: <44272813.2090407@gradient.cis.upenn.edu> Tim Peters wrote: > If extension modules in real life prove as sloppy as Python's > front end, we'll have to revert the objimpl.h + pymem.h part of this > patch. Note that no problems will show up in a debug build (all > calls still go thru obmalloc then). Problems will show up only in a > release build, most likely segfaults. Could the debug build's macros for PyMem/PyObject_new/free be modified to check for mismatches? Or would storing information about which method was used to allocate each pointer be too expensive? Perhaps a special build could be used to check for mismatches? -Edward From adam.deprince at gmail.com Mon Mar 27 06:35:05 2006 From: adam.deprince at gmail.com (Adam DePrince) Date: Sun, 26 Mar 2006 23:35:05 -0500 Subject: [Python-Dev] Prevalence of low-level memory abuse? In-Reply-To: <1f7befae0603261558t37f5b91cv31e68f91c7f3f997@mail.gmail.com> References: <1f7befae0603252120w630604dchea878e5db5eb2a71@mail.gmail.com> <1f7befae0603261529m3dc2989ftcf4f4c919517916d@mail.gmail.com> <44272813.2090407@gradient.cis.upenn.edu> <1f7befae0603261558t37f5b91cv31e68f91c7f3f997@mail.gmail.com> Message-ID: <1143434105.14391.71.camel@localhost.localdomain> On Sun, 2006-03-26 at 18:58 -0500, Tim Peters wrote: > [Edward Loper] > > Could the debug build's macros for PyMem/PyObject_new/free be modified > > to check for mismatches? Or would storing information about which > > method was used to allocate each pointer be too expensive? Perhaps a > > special build could be used to check for mismatches? > > It's partly possible (e.g., it's impossible to know whether a blob of > memory was obtained by calling malloc() directly). > > If someone wants to do it (I do not), the debug build adds 8 bytes to > each side of each memory block obtained via each PyMem and PyObject > malloc/realloc call, and one of the (current) 8 FORBIDDEN_BYTEs could > be used to store flags without significant loss of functionality. It > would make a decent enhancement. Sounds fun. I'll do it. - Adam DePrince > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/adam.deprince%40gmail.com From nnorwitz at gmail.com Mon Mar 27 06:47:18 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 26 Mar 2006 20:47:18 -0800 Subject: [Python-Dev] daily releases? Message-ID: Now that the buildbot is in place and seems to be running relatively smoothly, maybe should consider making daily (or periodic) builds and releasing them. We've got a system in place to build on many platforms automatically. How much more difficult would it be to package up the results and make them available for download? Maybe we could get some users testing these builds? Would users accept a pure debug build? If not, what about a build with asserts enabled? n From nnorwitz at gmail.com Mon Mar 27 07:04:25 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 26 Mar 2006 21:04:25 -0800 Subject: [Python-Dev] Changing -Q to warn for 2.5? Message-ID: http://python.org/sf/1458927 asks if -Q warn option should become the default in 2.5. PEP 238 (http://www.python.org/dev/peps/pep-0238/) says: """ The -Q command line option takes a string argument that can take four values: "old", "warn", "warnall", or "new". The default is "old" in Python 2.2 but will change to "warn" in later 2.x versions. """ Is this still accurate? Do we want to change the default in 2.x? If so, does x == 5? I'm not sure this is worth in 2.x. If we aren't going to change it, we should update the PEP. OTOH, people ask how they can find integer division in their code. Even though they can use the flag themselves, I wonder if anyone who wants help finding integer division really uses the flag. n From martin at v.loewis.de Mon Mar 27 07:38:02 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 27 Mar 2006 07:38:02 +0200 Subject: [Python-Dev] daily releases? In-Reply-To: References: Message-ID: <44277A3A.1030707@v.loewis.de> Neal Norwitz wrote: > Now that the buildbot is in place and seems to be running relatively > smoothly, maybe should consider making daily (or periodic) builds and > releasing them. We've got a system in place to build on many > platforms automatically. How much more difficult would it be to > package up the results and make them available for download? Maybe we > could get some users testing these builds? Would users accept a pure > debug build? If not, what about a build with asserts enabled? Depends on what precisely you mean by "release". If it is a tarball of the sources, we already have these: http://svn.python.org/snapshots/ In a release, the most prominent binary is the Windows installer. While it would be possible to define buildbot build steps to create a Windows installer (and msi.py even has the notion of "snapshot" builds already), fetching the results from the buildbot slave isn't really supported. Whether daily RPMs or Mac installers would be possible, I don't know. Regards, Martin From anthony at interlink.com.au Mon Mar 27 08:20:24 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon, 27 Mar 2006 17:20:24 +1100 Subject: [Python-Dev] BRANCH release24-maint FREEZE from 00:00 UTC on Wednesday 29th Message-ID: <200603271720.26407.anthony@interlink.com.au> Ok, pending a fix for http://www.python.org/sf/1459029, I'm declaring a freeze for release24-maint starting at 0000 UTC on Wednesday 29th of March. This is for Python 2.4.3. Please don't checkin after that time to the branch without checking with me first. I can be reached via email, or on #python-dev on Freenode. Thanks, Anthony -- Anthony Baxter It's never too late to have a happy childhood. From anthony at interlink.com.au Mon Mar 27 08:30:28 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon, 27 Mar 2006 17:30:28 +1100 Subject: [Python-Dev] TRUNK FREEZE for 2.5a1: 0000 UTC, Thursday 30th Message-ID: <200603271730.30582.anthony@interlink.com.au> Ok, it's time to rock and roll. The SVN trunk is FROZEN for 2.5a1 from 00:00 UTC on Thursday 30th of March. I'll post again once it's open. Note that new features can keep going in during the alpha cycle, the feature freeze only happens once we hit beta. And we're not going to hit beta until the features we want are in. Please help in making this release as painless as possible by not checking in while the trunk is frozen. Thanks! Anthony -- Anthony Baxter It's never too late to have a happy childhood. From rasky at develer.com Mon Mar 27 09:18:56 2006 From: rasky at develer.com (Giovanni Bajo) Date: Mon, 27 Mar 2006 09:18:56 +0200 Subject: [Python-Dev] Changing -Q to warn for 2.5? References: Message-ID: <042501c6516e$b5e8c0a0$f44c2597@bagio> Neal Norwitz wrote: > """ > The -Q command line option takes a string argument that can take four > values: "old", "warn", "warnall", or "new". The default is "old" in > Python 2.2 but will change to "warn" in later 2.x versions. > """ > > I'm not sure this is worth in 2.x. If we aren't going to change it, > we should update the PEP. OTOH, people ask how they can find integer > division in their code. Even though they can use the flag themselves, > I wonder if anyone who wants help finding integer division really uses > the flag. -1 gratuitous breakage. There's noting really wrong or dangerous about the old semantic, it just won't be the one used by Python 3.0. While it's nice to have an option to help forward porting, I don't think we should force it. Giovanni Bajo From anthony at interlink.com.au Mon Mar 27 09:25:36 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon, 27 Mar 2006 18:25:36 +1100 Subject: [Python-Dev] Changing -Q to warn for 2.5? In-Reply-To: References: Message-ID: <200603271825.40175.anthony@interlink.com.au> On Monday 27 March 2006 16:04, Neal Norwitz wrote: > http://python.org/sf/1458927 asks if -Q warn option should become > the default in 2.5. PEP 238 > (http://www.python.org/dev/peps/pep-0238/) says: > > """ > The -Q command line option takes a string argument that can take > four values: "old", "warn", "warnall", or "new". The default is > "old" in Python 2.2 but will change to "warn" in later 2.x > versions. """ > > Is this still accurate? Do we want to change the default in 2.x? > If so, does x == 5? -1 PITA. ;) -- Anthony Baxter It's never too late to have a happy childhood. From nnorwitz at gmail.com Mon Mar 27 09:39:50 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 26 Mar 2006 23:39:50 -0800 Subject: [Python-Dev] refleaks in 2.4 Message-ID: There are 5 tests that leak references that are present in 2.4.3c1, but not on HEAD. It would be great if someone can diagnose these and suggest a fix. test_doctest leaked [1, 1, 1] references test_pkg leaked [10, 10, 10] references test_pkgimport leaked [2, 2, 2] references test_traceback leaked [11, 11, 11] references test_unicode leaked [7, 7, 7] references test_traceback leaks due to test_bug737473. n From hyeshik at gmail.com Mon Mar 27 09:40:30 2006 From: hyeshik at gmail.com (Hye-Shik Chang) Date: Mon, 27 Mar 2006 16:40:30 +0900 Subject: [Python-Dev] Inconsistency in 2.4.3 for __repr__() returning unicode Message-ID: <4f0b69dc0603262340l705ee84dr9cd8f356c464438e@mail.gmail.com> We got an inconsistency for __repr__() returning unicode as reported in http://python.org/sf/1459029 : class s1: def __repr__(self): return '\\n' class s2: def __repr__(self): return u'\\n' print repr(s1()), repr(s2()) Until 2.4.2: \n \n 2.4.3: \n \\n \\n looks bit weird but it's correct. As once discussed[1] in python-dev before, if __repr__ returns unicode object, PyObject_Repr encodes it via unicode-escape codec. So, non-latin character also could be in repr neutrally. But our unicode-escape had a bug since when it is introduced. The bug was that it doesn't escape backslashes. Therefore, backslashes wasn't escaped in repr while it sholud be escaped because we used the unicode-escape codec. So, fixing the bug made a behavior inconsistency. How do we resolve the problem? Hye-Shik [1] http://mail.python.org/pipermail/python-dev/2000-July/005353.html From mal at egenix.com Mon Mar 27 12:14:50 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 27 Mar 2006 12:14:50 +0200 Subject: [Python-Dev] Inconsistency in 2.4.3 for __repr__() returning unicode In-Reply-To: <4f0b69dc0603262340l705ee84dr9cd8f356c464438e@mail.gmail.com> References: <4f0b69dc0603262340l705ee84dr9cd8f356c464438e@mail.gmail.com> Message-ID: <4427BB1A.3000201@egenix.com> Hye-Shik Chang wrote: > We got an inconsistency for __repr__() returning unicode as > reported in http://python.org/sf/1459029 : > > class s1: > def __repr__(self): > return '\\n' > > class s2: > def __repr__(self): > return u'\\n' > > print repr(s1()), repr(s2()) > > Until 2.4.2: \n \n > 2.4.3: \n \\n > > \\n looks bit weird but it's correct. As once discussed[1] in > python-dev before, if __repr__ returns unicode object, > PyObject_Repr encodes it via unicode-escape codec. So, > non-latin character also could be in repr neutrally. I don't think that using unicode-escape is the right choice for converting a string returned by __repr__ to a string - why would you want to escape a Unicode string that was specifically prepared to provide the representation of an object ? > But our unicode-escape had a bug since when it is introduced. > The bug was that it doesn't escape backslashes. Therefore, > backslashes wasn't escaped in repr while it sholud be escaped > because we used the unicode-escape codec. > > So, fixing the bug made a behavior inconsistency. > How do we resolve the problem? Change PyObject_Repr() to use the default encoding (again) which is also consistent with how PyObject_Str() works. To make repr() conversion more robust, we could have PyObject_Repr() apply the conversion using the 'replace' error strategy - after all, repr() is usually only used for debugging, where it's more important that you do get an output rather than an exception. > Hye-Shik > > > [1] http://mail.python.org/pipermail/python-dev/2000-July/005353.html -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 27 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From pedronis at strakt.com Mon Mar 27 15:31:08 2006 From: pedronis at strakt.com (Samuele Pedroni) Date: Mon, 27 Mar 2006 15:31:08 +0200 Subject: [Python-Dev] Next PyPy Sprint: Tokyo 23/4 - 29/4 Message-ID: <4427E91C.2090603@strakt.com> Tokyo PyPy Sprint: 23rd - 29th April 2006 ============================================================ The next PyPy sprint is scheduled to take place 23rd- 29th April 2006 (Sunday-Saturday) in Akihabara, Tokyo, Japan. We will together with FSIJ (Free Software Initiative of Japan) aim to promote Python and PyPy. We therefor invite Japanese hackers knowledgeable in Python to join our sprint! We'll give newcomer-friendly introductions. To learn more about the new Python-in-Python implementation look here: http://codespeak.net/pypy For this sprint we are particularly interested in meeting and coding on PyPy together with interested Japanese Python hackers. Please register your interest at pypy-sprint at codespeak.net as soon as possible and we will help with any questions regarding getting started, pointing to relevant documentation etc. The PyPy team is curious and interested in the experience of hacking code for embedded devices and would love to discuss and get feedback on optimisation efforts and the current state of PyPy. Goals and topics of the sprint ------------------------------ Possible suggestions for topics are: - Work on gensqueak (our Squeak backend) or possibly other backends. - Implementing Python 2.5 features in PyPy. - Progress further on an 'rctypes' module aiming at letting us use a ctypes implementation of an extension module from the compiled pypy-c. - Writing ctypes implementations of modules to be used by the above tool. - Experimenting and improving performance of our garbage collectors. - Experiment with PyPy flexibility or other aspects of the implementation. - Possibly experiment with writing modules translatable for use both in PyPy and CPython. - Whatever participants want to do with PyPy or particular areas of PyPy (please send suggestions to the mailing list before to allow us to plan and give feedback) Location & Accomodation ------------------------ The sprint will be held at National Institute of AIST (National Institute of Advanced Industrial Science and Technology, http://www.aist.go.jp/index_en.html), Akihahabara (the technical gadget district in Tokyo). Yutaka Niibe is our contact person there, helping with arranging facilities. Niibe is the chairman of FSIJ and they have invited us to sprint in Tokyo and we are very grateful for the help and interest we have recieved so far. The facilities we are sprinting in are located here: http://www.gtrc.aist.go.jp/en/access/index.html#Akihabara The actual address is: Akihabara Dai Bldg , 1-18-13 Sotokanda, Chiyoda-ku, Tokyo 101-0021 Japan Phone: +81-3-5298-4729 Hotel areas - we are recommended to book hotels in Ueno and Asakusa (old town), from those areas there are only two metro stops to Akihabara. Please note that hotelrooms in Tokyo are often very small. http://www.wh-rsv.com/english/akihabara/index.html (nearest hotel to sprint location) http://www.greenhotel.co.jp/ochanomizu_e.html http://www.ohgai.co.jp/index-e.html (Ueno) http://www.toyoko-inn.com/e_hotel/00012/index.html (Asakusa) http://www.hotelnewkanda.com/ (second nearest, but no english page) Here is a url for booking hotels with not too unreasonable rates (see map): http://japan-hotelguide.com/hotels/Japan/Tokyo/index.htm For more general tourist information about travelling to Japan and Tokyo - please see: http://www.jnto.go.jp/eng/ http://www.japantravelinfo.com/ (really useful information regarding airfares, hotels, currency, phones etc etc) Comments on the weather: In end April it is ca 20 degrees Celsius. Exact times ----------- The public PyPy sprint is held Sunday 23rd - Saturday 29th April 2006. Hours will be from 10:00 until people have had enough. It's a good idea to arrive a day before the sprint starts and leave a day later. Sometimes people cannot stay for the whole sprint - you are welcome even if you can only stay for a day or a few days. Sunday: Starting at 10:00. This day is focused on getting to know PyPy enought to start to participate. We will hold a PyPy tutorial and an architectural overview. Planning meeting for the work to be done during the week and grouping of developers (pairs or groups mixing new participants with core developers). Dinner in the evening (Yutaka will arrange a place for us to go to). Monday-Tuesday: Starting at 10:00 with status meetings. Possible regrouping depending on the interest and progress of the various teams. Wednesday: Breakday (coding is allowed although we recommend taking a break). Thursday-Saturday: Starting at 10:00 with status meetings. Possible regrouping depending on the interest and progress of the various teams. Ending on Saturday with a Closure session - summing of the work and planning work to be done until the next sprint. Network, Food, currency ------------------------ We will have access to WiFi at AIST - please make sure you have wlan capabilities. Electricity outlets: 100V (adapters needed for european standard). Currency is Japanese yen. There are Citibank cash machines that accepts cash withdrawals from the major cards such as VISA and Mastercard. But it is a good idea to bring cash. Also note that cell phones (european) are not very compatible with the Japanese network. There are possibilities for 3G phones to hire a phone and put your simcard in there. At the airport (both Kansai and Narita) there are information and places were this can be arranged (to a cost of course). Food: well - japanese food is great (wether it is sushi, sashimi, tempura, sukiyaki, teriyaki.... Eating out is not that much differently prized than any large european town. There are of course restaurants serving other food than japanese (chinese, korean, McDonalds ;-). Please also note that vegetables and fruit is quite expensive in Japan. For more information - see tourist url:s above. Registration etc.pp. -------------------- Please subscribe to the `PyPy sprint mailing list`_, introduce yourself and post a note that you want to come. Feel free to ask any questions there! There also is a separate `Tokyo people`_ page tracking who is already thought to come. If you have commit rights on codespeak then you can modify yourself a checkout of http://codespeak.net/svn/pypy/extradoc/sprintinfo/tokyo/people.txt .. _`PyPy sprint mailing list`: http://codespeak.net/mailman/listinfo/pypy-sprint .. _`Tokyo people`: http://codespeak.net/pypy/extradoc/sprintinfo/tokyo/people.html _______________________________________________ pypy-dev at codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev From barry at python.org Mon Mar 27 16:07:26 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 27 Mar 2006 09:07:26 -0500 Subject: [Python-Dev] PySet API In-Reply-To: <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> References: <006401c650eb$55e140c0$b83efea9@RaymondLaptop1> <20060326160728.GA6403@panix.com> <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> Message-ID: <1143468446.10799.50.camel@resist.wooz.org> On Sun, 2006-03-26 at 11:43 -0500, Raymond Hettinger wrote: > It was a trick question. Everyone is supposed to be attracted to the _next > version because it is shorter, faster, and takes less ref counting management. > However, the _next version has a hard-to-find bug. The call to PyObject_Hash() > can trigger arbitrary Python code and possibly mutate the table, leaving > pointers to invalid memory addresses. It would likely take Armin less than five > minutes to write a pure Python crasher for the code. And THAT is why > PySet_Next() should never come into being. We're clearly going in circles here, and it's obvious we're not going to agree. The fact that PySet_Next() can be used incorrectly is no reason not to include it. There are /lots/ of things in Python that if you use incorrectly will screw you. So you document them, teach people when not to use them, and teach them how to use them correctly when they /are/ the right thing to use. I don't want to be babied into using inappropriate and cumbersome APIs which, yes, can be a source of their own subtle bugs. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060327/199640a0/attachment.pgp From barry at python.org Mon Mar 27 16:11:04 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 27 Mar 2006 09:11:04 -0500 Subject: [Python-Dev] PySet_Next (Was: PySet API) In-Reply-To: <4426D67E.6040303@v.loewis.de> References: <006401c650eb$55e140c0$b83efea9@RaymondLaptop1> <20060326160728.GA6403@panix.com> <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> <009e01c650fa$1e6e0290$b83efea9@RaymondLaptop1> <4426D67E.6040303@v.loewis.de> Message-ID: <1143468664.10792.54.camel@resist.wooz.org> On Sun, 2006-03-26 at 19:59 +0200, "Martin v. L?wis" wrote: > If it is similar to PyDict_Next, it will have PyObject** /input/ > variables, which are really meant as PyObject* /output/ variables. Yep, that's exactly what my posted patch does. > For the caller, a clear usage strategy follows from this: either discard > the references before making a potentially-mutating call, or Py_INCREF > the set element before making that mutating call. Yep. Nice and simple. And if you're not making a potentially-mutating call, you don't have to worry about even that. These /are/ valid use cases. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060327/df3d2da2/attachment.pgp From barry at python.org Mon Mar 27 16:24:03 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 27 Mar 2006 09:24:03 -0500 Subject: [Python-Dev] PySet_Next (Was: PySet API) In-Reply-To: <00cd01c65102$83d90460$b83efea9@RaymondLaptop1> References: <006401c650eb$55e140c0$b83efea9@RaymondLaptop1> <20060326160728.GA6403@panix.com> <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> <009e01c650fa$1e6e0290$b83efea9@RaymondLaptop1> <4426D67E.6040303@v.loewis.de> <00cd01c65102$83d90460$b83efea9@RaymondLaptop1> Message-ID: <1143469443.10799.58.camel@resist.wooz.org> On Sun, 2006-03-26 at 13:24 -0500, Raymond Hettinger wrote: > We have a perfectly good way to iterate with PyIter_Next(). It may take a > couple of extra lines, but it is easy to get correct and has no surprises. It > seems that the only issue is that Barry says that he refuses to use the iterator > protocol. Heck, just turn it into a list and index directly. There is no need > to muck-up the set api for this. I just think you have a narrow vision of how Python sets can be used in a C application. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060327/7de55752/attachment.pgp From barry at python.org Mon Mar 27 16:32:26 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 27 Mar 2006 09:32:26 -0500 Subject: [Python-Dev] PySet_Next (Was: PySet API) In-Reply-To: <4426F093.3040808@v.loewis.de> References: <006401c650eb$55e140c0$b83efea9@RaymondLaptop1> <20060326160728.GA6403@panix.com> <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> <009e01c650fa$1e6e0290$b83efea9@RaymondLaptop1> <4426D67E.6040303@v.loewis.de> <00cd01c65102$83d90460$b83efea9@RaymondLaptop1> <4426F093.3040808@v.loewis.de> Message-ID: <1143469946.10793.67.camel@resist.wooz.org> On Sun, 2006-03-26 at 21:50 +0200, "Martin v. L?wis" wrote: > I don't know what specific application Barry has in mind, but I'm > sure he can get it right (although it might be an interesting experiment > to test that theory :-) In general, I would expect that people find > it easier to get code involving PyDict_Next right than code dealing > with iterators - primarily because of the error cases you have to > consider. I can't post the code because it's proprietary, but I gave a general feel to the types of things we do in a previous response. Imagine you have application objects that are also PyObjects. They have application specific state and behavior. They can be put in Python sets and they can be iterated over to check that state, invoke that behavior (which won't involve trips into Python), or perhaps add them to other collections. Do this 50 to 60 times in your application and I think you'll start to see why the iterator protocol is incredibly cumbersome to use. > I don't care that much either way, although I would prefer to see an > actual, current use case for PySet_Next, rather than theoretical, > made-up examples. I don't expect to use Python sets in C code at all, > personally. We really, honestly do use PySet_Next in many places. We implemented that API for Python 2.4 exactly because the iterator protocol was way too much overhead. The posted patch is a port to Python 2.5. We obviously can't add this to Python 2.4, but I had really hoped that we wouldn't have to maintain this extension for subsequent versions. I'm frankly astonished to get so much pushback from Raymond about it. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060327/f4d4ba37/attachment.pgp From arigo at tunes.org Mon Mar 27 16:58:24 2006 From: arigo at tunes.org (Armin Rigo) Date: Mon, 27 Mar 2006 16:58:24 +0200 Subject: [Python-Dev] refleaks in 2.4 In-Reply-To: References: Message-ID: <20060327145824.GA23444@code0.codespeak.net> Hi Neal, On Sun, Mar 26, 2006 at 11:39:50PM -0800, Neal Norwitz wrote: > test_pkg leaked [10, 10, 10] references This one at least appears to be caused by dummy (deleted) entries in the dictionary of interned strings. So it is not really a leak. It is a pain that it is so hard to figure this out, though. Wouldn't it make sense to find a trick to exclude these dummy entries from the total reference count? E.g. by subtracting the refcount of the dummy object... A bientot, Armin From python at rcn.com Mon Mar 27 17:20:33 2006 From: python at rcn.com (Raymond Hettinger) Date: Mon, 27 Mar 2006 10:20:33 -0500 Subject: [Python-Dev] PySet API References: <006401c650eb$55e140c0$b83efea9@RaymondLaptop1> <20060326160728.GA6403@panix.com> <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> <1143468446.10799.50.camel@resist.wooz.org> Message-ID: <003701c651b1$ff0c8670$1e2810ac@RaymondLaptop1> Why don't we expose _PySet_Next() for Barry and leave it out of the public API for everyone else. Raymond From python at rcn.com Mon Mar 27 18:30:22 2006 From: python at rcn.com (Raymond Hettinger) Date: Mon, 27 Mar 2006 11:30:22 -0500 Subject: [Python-Dev] PySet API References: <006401c650eb$55e140c0$b83efea9@RaymondLaptop1> <20060326160728.GA6403@panix.com> <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> <1143468446.10799.50.camel@resist.wooz.org> <003701c651b1$ff0c8670$1e2810ac@RaymondLaptop1> <6EFE2418-2EEA-42DF-9E89-9886BEF71C24@mac.com> Message-ID: <003101c651bb$bfa53f90$d200000a@RaymondLaptop1> >> Why don't we expose _PySet_Next() for Barry and leave it out of the public >> API >> for everyone else. [Alex] > There are precedents for adding some functionality to the C API but not > documenting it to ensure "non advanced users" don't get hurt -- that's how we > added the ability to raise exceptions in different threads, in particular. > Not sure if this is the best solution here, but I'm just pointing out that > it's definitely not "unthinkable", procedurally speaking. Thank would be nice. It gives me the ability to keep a clean, sane API while at the same time making sure that my most important customer (Barry) gets his needs met. Raymond From jeremy at alum.mit.edu Mon Mar 27 21:38:37 2006 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Mon, 27 Mar 2006 14:38:37 -0500 Subject: [Python-Dev] [Python-checkins] TRUNK FREEZE for 2.5a1: 0000 UTC, Thursday 30th In-Reply-To: <200603271730.30582.anthony@interlink.com.au> References: <200603271730.30582.anthony@interlink.com.au> Message-ID: On 3/27/06, Anthony Baxter wrote: > Ok, it's time to rock and roll. > > The SVN trunk is FROZEN for 2.5a1 from 00:00 UTC on > Thursday 30th of March. > > I'll post again once it's open. Note that new features can keep going > in during the alpha cycle, the feature freeze only happens once we > hit beta. And we're not going to hit beta until the features we want > are in. > > Please help in making this release as painless as possible by not > checking in while the trunk is frozen. There's a bug day on Friday. http://wiki.python.org/moin/PythonBugDay It seems a bit senseless to freeze the trunk the day before the bug day. All the bugs I'm planning to work on are important for the alpha. How much inconvenience will it case to delay the freeze until after the bug day? I'd say we should push the alpha a whole week rather than screw up the bug day. Jeremy Jeremy From g.brandl at gmx.net Mon Mar 27 22:14:34 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 27 Mar 2006 22:14:34 +0200 Subject: [Python-Dev] [Python-checkins] TRUNK FREEZE for 2.5a1: 0000 UTC, Thursday 30th In-Reply-To: References: <200603271730.30582.anthony@interlink.com.au> Message-ID: Jeremy Hylton wrote: > On 3/27/06, Anthony Baxter wrote: >> Ok, it's time to rock and roll. >> >> The SVN trunk is FROZEN for 2.5a1 from 00:00 UTC on >> Thursday 30th of March. >> >> I'll post again once it's open. Note that new features can keep going >> in during the alpha cycle, the feature freeze only happens once we >> hit beta. And we're not going to hit beta until the features we want >> are in. >> >> Please help in making this release as painless as possible by not >> checking in while the trunk is frozen. > > There's a bug day on Friday. > http://wiki.python.org/moin/PythonBugDay > > It seems a bit senseless to freeze the trunk the day before the bug > day. All the bugs I'm planning to work on are important for the > alpha. How much inconvenience will it case to delay the freeze until > after the bug day? I'd say we should push the alpha a whole week > rather than screw up the bug day. As the one who scheduled the bug day, I second that. (I really should have read the freeze message more closely, thanks Jeremy) Georg From martin at v.loewis.de Mon Mar 27 23:21:34 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 27 Mar 2006 23:21:34 +0200 Subject: [Python-Dev] PySet API In-Reply-To: <003701c651b1$ff0c8670$1e2810ac@RaymondLaptop1> References: <006401c650eb$55e140c0$b83efea9@RaymondLaptop1> <20060326160728.GA6403@panix.com> <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> <1143468446.10799.50.camel@resist.wooz.org> <003701c651b1$ff0c8670$1e2810ac@RaymondLaptop1> Message-ID: <4428575E.8010409@v.loewis.de> Raymond Hettinger wrote: > Why don't we expose _PySet_Next() for Barry and leave it out of the public API > for everyone else. That is stupid. If Barry wants a "private" PySet_Next function, he can just implement it himself, no need to include it in the release. It should be included only if it is meant to be public. Regards, Martin From thomas at python.org Mon Mar 27 23:21:40 2006 From: thomas at python.org (Thomas Wouters) Date: Mon, 27 Mar 2006 23:21:40 +0200 Subject: [Python-Dev] [Python-checkins] r43358 - python/trunk/Modules/itertoolsmodule.c In-Reply-To: <007e01c651e3$1d863ac0$d200000a@RaymondLaptop1> References: <20060327210214.2E8991E4004@bag.python.org> <007e01c651e3$1d863ac0$d200000a@RaymondLaptop1> Message-ID: <9e804ac0603271321x2aa99f58vf6013dafd0ca9a0c@mail.gmail.com> On 3/27/06, Raymond Hettinger wrote: > > > Modified: > > python/trunk/Modules/itertoolsmodule.c > > Log: > > > > Make itertools.tee and its internal teedataobject participate in GC. > This > > alone does not solve the leak in test_generators, unfortunately, but it > is > > part of test_generators' problem and it does solve other cycles. > > Thanks for getting this in. > > To get the leak in test_generators, I think you make need to add GC to the > teeobject as well as the teedataobject. The teeobject has GC (hence the word 'and' in 'itertools.tee and its internal teedataobject' ;-) The problem with test_generators is that this also leaks: def leak(): def gen(): while True: yield g g = gen() leak() It doesn't leak in 2.4. I'm using a little shell script to work backwards through the 2.5 changes to find out which one introduced this, although I somehow suspect it's the coroutine stuff... ;P -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060327/20b24644/attachment.html From pje at telecommunity.com Mon Mar 27 23:26:23 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 27 Mar 2006 16:26:23 -0500 Subject: [Python-Dev] Libref sections to put new modules under? Message-ID: <5.1.1.6.0.20060327161706.05c64fc0@mail.telecommunity.com> Any thoughts on where documentation for the new "contextlib" module should go in the library reference? Some candidates: * Program Frameworks (???) * Development Tools (??) * Python Runtime Services (?) * Python Language Services (??) * Miscellaneous Services (??) A similar issue exists for "ctypes" module, although I imagine an argument could easily be made for putting it with "Optional Operating System Services". wsgiref can probably go in "Internet Protocols and Support", while ElementTree obviously goes under "Structured Markup Processing Tools". What other new modules do we have going in 2.5 that haven't found a home in the documentation yet? I seem to recall some discussion about a decorator library or functools or something of that sort. From nnorwitz at gmail.com Mon Mar 27 23:30:27 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 27 Mar 2006 13:30:27 -0800 Subject: [Python-Dev] [Python-checkins] r43358 - python/trunk/Modules/itertoolsmodule.c In-Reply-To: <9e804ac0603271321x2aa99f58vf6013dafd0ca9a0c@mail.gmail.com> References: <20060327210214.2E8991E4004@bag.python.org> <007e01c651e3$1d863ac0$d200000a@RaymondLaptop1> <9e804ac0603271321x2aa99f58vf6013dafd0ca9a0c@mail.gmail.com> Message-ID: On 3/27/06, Thomas Wouters wrote: > > The teeobject has GC (hence the word 'and' in 'itertools.tee and its > internal teedataobject' ;-) The problem with test_generators is that this > also leaks: > > def leak(): > def gen(): > while True: > yield g > g = gen() > > leak() Please add a test for this to the leakers and remove the tee test in leakers if that no longer leaks. Thanks, n From guido at python.org Mon Mar 27 23:33:16 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 27 Mar 2006 13:33:16 -0800 Subject: [Python-Dev] Libref sections to put new modules under? In-Reply-To: <5.1.1.6.0.20060327161706.05c64fc0@mail.telecommunity.com> References: <5.1.1.6.0.20060327161706.05c64fc0@mail.telecommunity.com> Message-ID: On 3/27/06, Phillip J. Eby wrote: > Any thoughts on where documentation for the new "contextlib" module should > go in the library reference? > > Some candidates: > > * Program Frameworks (???) > * Development Tools (??) > * Python Runtime Services (?) > * Python Language Services (??) > * Miscellaneous Services (??) Runtime Services is by far the best candidate. The Language services are more for code that reads/writes Python source code or bytecode. The others are even less likely matches. (What's development tools?) > A similar issue exists for "ctypes" module, although I imagine an argument > could easily be made for putting it with "Optional Operating System Services". +1 > wsgiref can probably go in "Internet Protocols and Support", while > ElementTree obviously goes under "Structured Markup Processing Tools". +1 > What other new modules do we have going in 2.5 that haven't found a home in > the documentation yet? I seem to recall some discussion about a decorator > library or functools or something of that sort. Is the with statement documented yet? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake at acm.org Mon Mar 27 23:35:39 2006 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Mon, 27 Mar 2006 16:35:39 -0500 Subject: [Python-Dev] Libref sections to put new modules under? In-Reply-To: <5.1.1.6.0.20060327161706.05c64fc0@mail.telecommunity.com> References: <5.1.1.6.0.20060327161706.05c64fc0@mail.telecommunity.com> Message-ID: <200603271635.39335.fdrake@acm.org> On Monday 27 March 2006 16:26, Phillip J. Eby wrote: > Any thoughts on where documentation for the new "contextlib" module should > go in the library reference? Most definately in "Python Runtime Services." > A similar issue exists for "ctypes" module, although I imagine an argument > could easily be made for putting it with "Optional Operating System > Services". I'm less sure of that one. It could reasonably go in "Python Runtime Services," "Generic Operating System Services," or "Optional Operating System Services." > wsgiref can probably go in "Internet Protocols and Support", while > ElementTree obviously goes under "Structured Markup Processing Tools". Yes to both. -Fred -- Fred L. Drake, Jr. From martin at v.loewis.de Mon Mar 27 23:39:42 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 27 Mar 2006 23:39:42 +0200 Subject: [Python-Dev] refleaks in 2.4 In-Reply-To: References: Message-ID: <44285B9E.9050704@v.loewis.de> Neal Norwitz wrote: > test_unicode leaked [7, 7, 7] references This one is lacking this checkin: r41530 | neal.norwitz | 2005-11-24 23:00:56 +0100 (Do, 24 Nov 2005) | 6 lines Ge?nderte Pfade: M /python/trunk/Lib/test/test_unicode.py Move registration of the codec search function to the module scope so it is only executed once. Otherwise the same search function is repeated added to the codec search path when regrtest is run with -R and leaks are reported. :-) Applying it gets the number of leaked references down to 1. Regards, Martin From guido at python.org Mon Mar 27 23:44:02 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 27 Mar 2006 13:44:02 -0800 Subject: [Python-Dev] I'm not getting email from SF when assigned a bug/patch Message-ID: Folks, When you assign a bug/patch to me, somehow SourceForge doesn't send me an email. (Is this understood behavior? Can it be changed?) Since I don't monitor my SF personal page regularly any more, that means that the issue will remain in limbo forever or until someone points me to it. So if you want me to take any particular action (even rejecting something) please send me a separate email! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Mon Mar 27 23:49:40 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 27 Mar 2006 16:49:40 -0500 Subject: [Python-Dev] Libref sections to put new modules under? In-Reply-To: References: <5.1.1.6.0.20060327161706.05c64fc0@mail.telecommunity.com> <5.1.1.6.0.20060327161706.05c64fc0@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060327164534.03ad4d50@mail.telecommunity.com> At 01:33 PM 3/27/2006 -0800, Guido van Rossum wrote: >Is the with statement documented yet? Sort of. I've added stuff to the language reference, but the "What's New" for it is blank right now. There's nothing in the tutorial, either, and of course contextlib isn't in the libref yet (which is what triggered my question). PEP 342 is also virtually undocumented outside of the "What's New". So far all I've done is correct the part about yield not being allowed in try/finally; apart from that I'm a bit hung up on how to document the new hybrid nature of "yield" in the language reference. It's like "New YIELD! It's a statement! It's an expression! It's an operator! It's... well, we don't know what it is. But it's great!" :) Suggestions on how to handle this appreciated. :) From martin at v.loewis.de Mon Mar 27 23:48:54 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 27 Mar 2006 23:48:54 +0200 Subject: [Python-Dev] refleaks in 2.4 In-Reply-To: References: Message-ID: <44285DC6.4030105@v.loewis.de> Neal Norwitz wrote: > test_unicode leaked [7, 7, 7] references And the last one came from r41531 | neal.norwitz | 2005-11-24 23:09:18 +0100 (Do, 24 Nov 2005) | 1 line Ge?nderte Pfade: M /python/trunk/Python/codecs.c M /python/trunk/Python/compile.c M /python/trunk/Python/symtable.c Fix a few more ref leaks. Backport candidate Only codecs.c backported, so this is what I did. Regards, Martin From martin at v.loewis.de Mon Mar 27 23:57:19 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 27 Mar 2006 23:57:19 +0200 Subject: [Python-Dev] I'm not getting email from SF when assigned a bug/patch In-Reply-To: References: Message-ID: <44285FBF.9090407@v.loewis.de> Guido van Rossum wrote: > When you assign a bug/patch to me, somehow SourceForge doesn't send me > an email. (Is this understood behavior? Can it be changed?) I believe it broke at some point, I'm pretty certain it used to work. Regards, Martin From anthony at interlink.com.au Tue Mar 28 00:05:38 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue, 28 Mar 2006 09:05:38 +1100 Subject: [Python-Dev] [Python-checkins] Cancelled! TRUNK FREEZE for 2.5a1: 0000 UTC, Thursday 30th In-Reply-To: References: <200603271730.30582.anthony@interlink.com.au> Message-ID: <200603280905.40989.anthony@interlink.com.au> On Tuesday 28 March 2006 06:38, Jeremy Hylton wrote: > There's a bug day on Friday. > http://wiki.python.org/moin/PythonBugDay Whoops! I _thought_ there was some vague reason in the back of my head to put it off til next week! Ok, this will be rescheduled for next week (the trunk freeze, not the bug day ) - exact date to follow once the appropriate people are OK with the date. Sorry about that, Anthony -- Anthony Baxter It's never too late to have a happy childhood. From guido at python.org Tue Mar 28 00:12:24 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 27 Mar 2006 14:12:24 -0800 Subject: [Python-Dev] I'm not getting email from SF when assigned a bug/patch In-Reply-To: <44285FBF.9090407@v.loewis.de> References: <44285FBF.9090407@v.loewis.de> Message-ID: On 3/27/06, "Martin v. L?wis" wrote: > Guido van Rossum wrote: > > When you assign a bug/patch to me, somehow SourceForge doesn't send me > > an email. (Is this understood behavior? Can it be changed?) > > I believe it broke at some point, I'm pretty certain it used to work. More reason to move away from SF ASAP. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From thomas at python.org Tue Mar 28 00:27:44 2006 From: thomas at python.org (Thomas Wouters) Date: Tue, 28 Mar 2006 00:27:44 +0200 Subject: [Python-Dev] I'm not getting email from SF when assigned a bug/patch In-Reply-To: References: Message-ID: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> On 3/27/06, Guido van Rossum wrote: > > Folks, > > When you assign a bug/patch to me, somehow SourceForge doesn't send me > an email. (Is this understood behavior? Can it be changed?) Since I > don't monitor my SF personal page regularly any more, that means that > the issue will remain in limbo forever or until someone points me to > it. So if you want me to take any particular action (even rejecting > something) please send me a separate email! Ah, glad to hear I wasn't the only one bitten by that. For a while, I thought I was going senile :-P Speaking of which, perhaps we should designate the running roundup instance on python.org as the issuetracker for py3k ? Gives us a good chance to evaluate, and if it doesn't work out, it wouldn't matter too much. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060328/fb40c818/attachment.htm From greg.ewing at canterbury.ac.nz Tue Mar 28 00:59:01 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 28 Mar 2006 10:59:01 +1200 Subject: [Python-Dev] PySet API In-Reply-To: <1143468446.10799.50.camel@resist.wooz.org> References: <006401c650eb$55e140c0$b83efea9@RaymondLaptop1> <20060326160728.GA6403@panix.com> <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> <1143468446.10799.50.camel@resist.wooz.org> Message-ID: <44286E35.4060607@canterbury.ac.nz> Barry Warsaw wrote: > We're clearly going in circles here, and it's obvious we're not going to > agree. Would it perhaps help if there were a better API for using the iterator protocol from C code? Something that's as easy to use as the proposed set iterating API, but which uses the general iterator protocol underneath. Greg From greg.ewing at canterbury.ac.nz Tue Mar 28 01:31:38 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 28 Mar 2006 11:31:38 +1200 Subject: [Python-Dev] Expose the array interface in Python 2.5? In-Reply-To: <441F322D.5090107@noaa.gov> References: <441F322D.5090107@noaa.gov> Message-ID: <442875DA.3010504@canterbury.ac.nz> Chris Barker wrote: > Can we have both? A defined interface, that existing code can be adapted > to provide, and a new C-Object, that future code can just use. If the > goal is to have as many extension types as possible use the same base > object, the sooner a standard object is provided the better. Having many extension types provide the same *interface* is what I think the main goal should be, not to have them use the same object. So getting the interface defined should be the first priority. > I'd sure like to see indexing and slicing, at at least. The interface itself doesn't need to provide indexing and slicing -- these could be provided by a view object that used the array interface of the underlying object. This would also fit in nicely with the "views" philosophy that seems to be shaping up for Py3k. > Another point is that n-dimensional arrays really are very useful for > all sorts of stuff that have nothing to do with high-performance Numeric > computing. I'm all in favour of including such an object, as long as we keep in mind that this is an orthogonal issue to having an array interface. The discussion still seems to be a bit muddled on this point. Greg From oliphant.travis at ieee.org Tue Mar 28 03:08:00 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 27 Mar 2006 18:08:00 -0700 Subject: [Python-Dev] INPLACE_ADD and INPLACE_MULTIPLY oddities in ceval.c Message-ID: If you have Numeric or numpy installed try this: #import Numeric as N import numpy as N a = range(10) b = N.arange(10) a.__iadd__(b) print a Result: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9] Contrast the returned output with import numpy as N a = range(10) b = N.arange(10) a += b print a Result: [ 0 2 4 6 8 10 12 14 16 18] Having "a+=b" and "a.__iadd__(b)" do different things seems like an unfortunate bug. It seems to me that the problem is that the INPLACE_ADD and INPLACE_MULTIPLY cases in ceval.c use PyNumber_InPlace instead of trying PySequence_InPlace when the object doesn't support the in-place number protocol. I could submit a patch if there is agreement that this is a problem. -Travis From guido at python.org Tue Mar 28 03:21:18 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 27 Mar 2006 17:21:18 -0800 Subject: [Python-Dev] INPLACE_ADD and INPLACE_MULTIPLY oddities in ceval.c In-Reply-To: References: Message-ID: On 3/27/06, Travis Oliphant wrote: > > If you have Numeric or numpy installed try this: > > #import Numeric as N > import numpy as N > > a = range(10) > b = N.arange(10) > > a.__iadd__(b) > > print a > > Result: > > [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9] > > > Contrast the returned output with > > import numpy as N > > a = range(10) > b = N.arange(10) > > a += b > > print a > > Result: > > [ 0 2 4 6 8 10 12 14 16 18] > > > Having "a+=b" and "a.__iadd__(b)" do different things seems like an > unfortunate bug. > > It seems to me that the problem is that the INPLACE_ADD and > INPLACE_MULTIPLY cases in ceval.c use PyNumber_InPlace instead of > trying PySequence_InPlace when the object doesn't support the > in-place number protocol. > > I could submit a patch if there is agreement that this is a problem. Well how is the interpreter to know whether += meanse numeric add or sequence add in any particular case? Shouldn't it be numpy's responsibility to implement both operations identically? (It's a design bug in Python that the + operation has two implementation slots. But it's difficult to fix that until Python 3000. It *will* be fixed there.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python.10.laughingboy at spamgourmet.com Mon Mar 27 21:20:19 2006 From: python.10.laughingboy at spamgourmet.com (Mike Krell) Date: Mon, 27 Mar 2006 19:20:19 +0000 (UTC) Subject: [Python-Dev] Class decorators References: <44268B6B.9070305@canterbury.ac.nz> Message-ID: Greg Ewing canterbury.ac.nz> writes: > > I've just been playing around with metaclasses, and > I think I've stumbled across a reason for having > class decorators as an alternative to metaclasses > for some purposes. There has also been discussion on the IronPython mailing list that class decorators would be a very useful syntax for expressing .NET attributes. http://lists.ironpython.com/pipermail/users-ironpython.com/2006-March/002007.html Mike From pje at telecommunity.com Tue Mar 28 04:23:50 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 27 Mar 2006 21:23:50 -0500 Subject: [Python-Dev] Class decorators In-Reply-To: References: <44268B6B.9070305@canterbury.ac.nz> Message-ID: <5.1.1.6.0.20060327211817.05cd3288@mail.telecommunity.com> At 07:20 PM 3/27/2006 +0000, Mike Krell wrote: >Greg Ewing canterbury.ac.nz> writes: > > > > > I've just been playing around with metaclasses, and > > I think I've stumbled across a reason for having > > class decorators as an alternative to metaclasses > > for some purposes. > >There has also been discussion on the IronPython mailing list that class >decorators would be a very useful syntax for expressing .NET attributes. > >http://lists.ironpython.com/pipermail/users-ironpython.com/2006-March/002007.html > PyProtocols and the zope.interface package both support inline class decorators called "class advisors". They don't require any special syntax, and aren't much more complex than regular decorators. By defining an advisor like this: from protocols.advice import addClassAdvisor def some_advisor(whatever_args): def callback(cls): print "I can modify",cls,"or replace it" return cls addClassAdvisor(callback) you can then use it in a class body like so: class SomeClass: some_advisor("something") And the return value from 'callback' will replace SomeClass, just like a decorator replaces the function it's called on. The implementation should work with any Python version from 2.2 up. I'm not sure if it would work with IronPython. But if it doesn't, that would be a good indication of feature(s) that IronPython is missing. ;) From robert.kern at gmail.com Tue Mar 28 04:45:54 2006 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 27 Mar 2006 20:45:54 -0600 Subject: [Python-Dev] INPLACE_ADD and INPLACE_MULTIPLY oddities in ceval.c In-Reply-To: References: Message-ID: Guido van Rossum wrote: > On 3/27/06, Travis Oliphant wrote: > >>If you have Numeric or numpy installed try this: >> >>#import Numeric as N >>import numpy as N >> >>a = range(10) >>b = N.arange(10) >> >>a.__iadd__(b) >> >>print a >> >>Result: >> >>[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9] >> >> >>Contrast the returned output with >> >>import numpy as N >> >>a = range(10) >>b = N.arange(10) >> >>a += b >> >>print a >> >>Result: >> >>[ 0 2 4 6 8 10 12 14 16 18] >> >>Having "a+=b" and "a.__iadd__(b)" do different things seems like an >>unfortunate bug. >> >>It seems to me that the problem is that the INPLACE_ADD and >>INPLACE_MULTIPLY cases in ceval.c use PyNumber_InPlace instead of >>trying PySequence_InPlace when the object doesn't support the >>in-place number protocol. >> >>I could submit a patch if there is agreement that this is a problem. > > Well how is the interpreter to know whether += meanse numeric add or > sequence add in any particular case? > > Shouldn't it be numpy's responsibility to implement both operations identically? "a" is a list object. Travis's question is, "why doesn't the list object implement both operations identically?" I think the answer is, "it never gets a chance to." When a numpy array is on the LHS of an inplace operation, we always want the numeric add. numpy arrays can't be extended inplace like a list, and a lone "+" is always a numeric operation for arrays, never a sequence operation. When numpy arrays are on the LHS of an inplace operation, everything works consistently. The issue seems to arise from the fact that PyNumber_InPlaceAdd tries to coerce the LHS and the RHS to a common numerical type first before trying PySequence_InPlaceConcat. Lists of numbers coerce to numpy arrays just fine, so this code path is taken, and nothing tests for the sq_inplace_concat or sq_concat slots. Personally, I'm willing to just document it in the numpy documentation as a rarely-seen gotcha. This behavior pretty much affects just us. -- Robert Kern robert.kern at gmail.com "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From oliphant.travis at ieee.org Tue Mar 28 05:10:10 2006 From: oliphant.travis at ieee.org (Travis E. Oliphant) Date: Mon, 27 Mar 2006 20:10:10 -0700 Subject: [Python-Dev] INPLACE_ADD and INPLACE_MULTIPLY oddities in ceval.c In-Reply-To: References: Message-ID: Guido van Rossum wrote: > On 3/27/06, Travis Oliphant wrote: >> If you have Numeric or numpy installed try this: >> >> #import Numeric as N >> import numpy as N >> >> a = range(10) >> b = N.arange(10) >> >> a.__iadd__(b) >> >> print a >> >> Result: >> >> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9] >> >> >> Contrast the returned output with >> >> import numpy as N >> >> a = range(10) >> b = N.arange(10) >> >> a += b >> >> print a >> >> Result: >> >> [ 0 2 4 6 8 10 12 14 16 18] >> >> >> Having "a+=b" and "a.__iadd__(b)" do different things seems like an >> unfortunate bug. >> >> It seems to me that the problem is that the INPLACE_ADD and >> INPLACE_MULTIPLY cases in ceval.c use PyNumber_InPlace instead of >> trying PySequence_InPlace when the object doesn't support the >> in-place number protocol. >> >> I could submit a patch if there is agreement that this is a problem. > > Well how is the interpreter to know whether += meanse numeric add or > sequence add in any particular case? > I can see that '+' (and '*') having two different implementation slots is a design bug. However, it seems prudent that both += and .__iadd__ should have identical behavior regardless. Whatever mechanism is used to resolve the conflict for __iadd__, the same mechanism should be used for for +=. I don't think this is a NumPy-only issue. Any object that defines addition that works with lists will have similar problems. I think this can be fixed easily by first checking the sequence slot for a sq_concat function before calling PyNumber_InPlaceAdd. All I'm asking for is that a += b have the same behavior as a.__iadd__(b). That seems like desireable behavior to me. -Travis From barry at python.org Tue Mar 28 05:25:46 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 27 Mar 2006 22:25:46 -0500 Subject: [Python-Dev] PySet API In-Reply-To: <001a01c65082$35a4c840$b83efea9@RaymondLaptop1> References: <1142705253.32074.72.camel@geddy.wooz.org> <005c01c64ac5$589859b0$b83efea9@RaymondLaptop1> <002b01c64aeb$44eb7750$b83efea9@RaymondLaptop1> <1142824236.10599.29.camel@resist.wooz.org> <00a001c64bfa$71a0ab20$b83efea9@RaymondLaptop1> <1142987079.10599.232.camel@resist.wooz.org> <004001c64d58$b0ebebf0$b83efea9@RaymondLaptop1> <1143327815.10799.42.camel@resist.wooz.org> <001a01c65082$35a4c840$b83efea9@RaymondLaptop1> Message-ID: <1143516346.2317.80.camel@geddy.wooz.org> On Sat, 2006-03-25 at 22:05 -0500, Raymond Hettinger wrote: > Still, PyObject_Clear(s) would be better. Although not ideal, in the interest of compromise, I could support this option. There's a problem with this though: I don't think you want to be able to clear a frozen set. My PySet_Clear() raises a SystemError and returns -1 when the object is a frozen set. If PyObject_Clear() is implemented something like int PyObject_Clear(PyObject *o) { return (o->ob_type->tp_clear ? o->ob_type->tp_clear(o) : -1); } then you /would/ be able to clear a frozen set. For that matter, it would be the case that any immutable collection would be clearable if it had a tp_clear (which it probably would). That isn't the semantics I'd expect though. That may not be solvable unless you make PyObject_Clear() an alias for PyObject_CallMethod("clear"). Although I'm sure you'll disagree, I think this is less than ideal. For one thing, you're requiring objects that work with PyObject_Clear() to implement an exact Python-level protocol (it must have a method, it must be called "clear" and it must take zero arguments). You also have to implement PyObject_Clear() with a hasattr test, because I don't think you want PyObject_Clear() raising AttributeErrors. That raises the constant overhead cost, which can make clearing small sets more expensive. > Better still would be to examine the > actual uses in the app. I suspect that most code that clears a set and then > rebuilds it would be better-off starting with a new empty set (and because of > freelisting, that is a very fast operation). That may not be possible. Imagine a complex application where the set is passed through many layers of calls. The set hangs off of other application level objects which you don't have access to at the point where you're deciding whether to clear the set or not. You can't create a new set because you have no way to pass the new set back to the several application level objects that would need to get their pointers updated. So the most obvious, simple approach is to just clear the set you have right there. > Likewise, it only takes a one-line header to define BarrySet_Update(s). I do > not want that part of the C API exposed yet. It is still under development and > may eventually become a function with a variable length argument list. Really? That would be odd and not at all parallel with established convention (e.g. PyDict_Update()). I would think that a vararg update should be named something different in order to preserve the principle of least surprise. > If you're dead-set against using the iterator API, then maybe there is something > wrong with the API. You should probably start a new thread on why you detest > the iterator API and see if there are ways to improve it. I'm not saying there's anything wrong with the iterator API, I'm saying that it's not always appropriate. It's the nail/hammer argument. But I ran out of clever when I tried to propose the simplest, most direct fix for our most pressing issues, so I'm not going to take the bait. > > You talk about duck typing, but I don't care about that here. > > It's one of the virtues of Python that gets reflected in the abstract API. IMO, > it's nice that PyObject_Dir(o) corresponds to "dir(o)" and the same for hash(o), > repr(o), etc. I just hope that by hardwiring data types in stone, that your app > doesn't become rigid and impossible to change. I certainly do not recommend > that other people adopt this coding style (avoidance of iterators, duplication > of abstact api functions in concrete form, etc.) If you're experiencing > debugging pain, it may be that avoidance of abstraction is the root cause. Trust me Raymond, it's not the cause. I keep trying to explain this but I must be completely inept because you're just not getting it. Let me try this way: we're using Python's collection types (sets, lists, dicts) as our fundamental collection data structures internally in our application. There's no duck typing going on. There's no need for abstraction because we know exactly what we have and there's no chance we'll have something that smells like a set that isn't exactly a PySet. As I've said many times, I'm all for an abstract API because it's darn useful in many applications. It's the lack of a concrete API that is limiting. > > I wouldn't object to that, but it wouldn't change my mind about > > PySet_Clear(). > > This is plain evidence that something is wrong with your approach. While > possibly necessary in your environment, the rest of mankind should not have to > stomach this kind of API clutter. Please, that's a bit extreme. I haven't heard anybody scream about the PyDict's API clutter and I don't see my PySet proposal as being any different. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060327/9fa20e5f/attachment.pgp From barry at python.org Tue Mar 28 05:29:18 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 27 Mar 2006 22:29:18 -0500 Subject: [Python-Dev] PySet API In-Reply-To: <003701c651b1$ff0c8670$1e2810ac@RaymondLaptop1> References: <006401c650eb$55e140c0$b83efea9@RaymondLaptop1> <20060326160728.GA6403@panix.com> <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> <1143468446.10799.50.camel@resist.wooz.org> <003701c651b1$ff0c8670$1e2810ac@RaymondLaptop1> Message-ID: <1143516558.2309.84.camel@geddy.wooz.org> On Mon, 2006-03-27 at 10:20 -0500, Raymond Hettinger wrote: > Why don't we expose _PySet_Next() for Barry and leave it out of the public API > for everyone else. Just so I understand exactly what you mean by "leave it out of the public API", let me ask: are you saying you don't want to document the function? Do you not want to include it in setobject.h? No tests for it? If all you're suggesting is to stick an underscore in front of the name to indicate "it's something special", I could live with that, although I don't see much benefit. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060327/f810a59c/attachment.pgp From barry at python.org Tue Mar 28 05:59:13 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 27 Mar 2006 22:59:13 -0500 Subject: [Python-Dev] PySet API In-Reply-To: <4428575E.8010409@v.loewis.de> References: <006401c650eb$55e140c0$b83efea9@RaymondLaptop1> <20060326160728.GA6403@panix.com> <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> <1143468446.10799.50.camel@resist.wooz.org> <003701c651b1$ff0c8670$1e2810ac@RaymondLaptop1> <4428575E.8010409@v.loewis.de> Message-ID: <1143518353.2317.117.camel@geddy.wooz.org> On Mon, 2006-03-27 at 23:21 +0200, "Martin v. L?wis" wrote: > Raymond Hettinger wrote: > > Why don't we expose _PySet_Next() for Barry and leave it out of the public API > > for everyone else. > > That is stupid. If Barry wants a "private" PySet_Next function, he can > just implement it himself, no need to include it in the release. It > should be included only if it is meant to be public. The one thing I'm trying to avoid is requiring us to patch Python when we move to 2.5. The most obvious and straightforward implementation of the 3 API calls are very simple wrappers around static functions defined in setobject.c (see the patch I posted). So if the patch gets rejected, then implementing them as a hack on standard Python 2.5 will require patching setobject.c or cracking open the PySet structure and /really/ violating the principle of data hiding. Neither option is very appealing. That's why I said in a previous message that I could live with this as a compromise, although I agree with your (Martin's) sentiment. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060327/7070fa31/attachment-0001.pgp From guido at python.org Tue Mar 28 06:00:09 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 27 Mar 2006 20:00:09 -0800 Subject: [Python-Dev] INPLACE_ADD and INPLACE_MULTIPLY oddities in ceval.c In-Reply-To: References: Message-ID: On 3/27/06, Travis E. Oliphant wrote: > Guido van Rossum wrote: > > On 3/27/06, Travis Oliphant wrote: > >> If you have Numeric or numpy installed try this: > >> > >> #import Numeric as N > >> import numpy as N > >> > >> a = range(10) > >> b = N.arange(10) > >> > >> a.__iadd__(b) > >> > >> print a > >> > >> Result: > >> > >> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9] > >> > >> > >> Contrast the returned output with > >> > >> import numpy as N > >> > >> a = range(10) > >> b = N.arange(10) > >> > >> a += b > >> > >> print a > >> > >> Result: > >> > >> [ 0 2 4 6 8 10 12 14 16 18] > >> > >> > >> Having "a+=b" and "a.__iadd__(b)" do different things seems like an > >> unfortunate bug. > >> > >> It seems to me that the problem is that the INPLACE_ADD and > >> INPLACE_MULTIPLY cases in ceval.c use PyNumber_InPlace instead of > >> trying PySequence_InPlace when the object doesn't support the > >> in-place number protocol. > >> > >> I could submit a patch if there is agreement that this is a problem. > > > > Well how is the interpreter to know whether += meanse numeric add or > > sequence add in any particular case? > > I can see that '+' (and '*') having two different implementation slots > is a design bug. However, it seems prudent that both += and .__iadd__ > should have identical behavior regardless. Whatever mechanism is used > to resolve the conflict for __iadd__, the same mechanism should be used > for for +=. > > I don't think this is a NumPy-only issue. Any object that defines > addition that works with lists will have similar problems. > > I think this can be fixed easily by first checking the sequence slot for > a sq_concat function before calling PyNumber_InPlaceAdd. > > All I'm asking for is that a += b have the same behavior as > a.__iadd__(b). That seems like desireable behavior to me. Correct. Robert Kern's analysis is correct: when you make the call explicitly, a.__iadd__(b) invokes the list.__iadd__ method; this is effectively an alias for list.extend. I take back whatever I said about numpy. So for consistency we want a += b to also execute a.__iadd__. The opcode calls PyNumber_InplaceAdd; I think that PyNumber_InplaceAdd (and PySequence_InplaceConcat, if it exists) should test for both the numeric and the sequence augmented slot of the left argument first; then they should try both the numeric and sequence non-augmented slot of the left argument; and then the numeric non-augmented slot of the right argument. Coercion should not be attempted at all. The question is, can we do this in 2.5 without breaking backwards compatibility? Someone else with more time should look into the details of that. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From skip at pobox.com Tue Mar 28 06:00:34 2006 From: skip at pobox.com (skip at pobox.com) Date: Mon, 27 Mar 2006 22:00:34 -0600 Subject: [Python-Dev] I'm not getting email from SF when assigned a bug/patch In-Reply-To: <44285FBF.9090407@v.loewis.de> References: <44285FBF.9090407@v.loewis.de> Message-ID: <17448.46306.112637.1041@montanaro.dyndns.org> >> When you assign a bug/patch to me, somehow SourceForge doesn't send >> me an email. (Is this understood behavior? Can it be changed?) Martin> I believe it broke at some point, I'm pretty certain it used to Martin> work. I would expect that if it broke for Guido it broke for everybody. While we consider him to be the BDFL I suspect the accolades are lost on the SF folks. Skip From guido at python.org Tue Mar 28 06:02:20 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 27 Mar 2006 20:02:20 -0800 Subject: [Python-Dev] Class decorators In-Reply-To: <5.1.1.6.0.20060327211817.05cd3288@mail.telecommunity.com> References: <44268B6B.9070305@canterbury.ac.nz> <5.1.1.6.0.20060327211817.05cd3288@mail.telecommunity.com> Message-ID: On 3/27/06, Phillip J. Eby wrote: > PyProtocols and the zope.interface package both support inline class > decorators called "class advisors". They don't require any special syntax, > and aren't much more complex than regular decorators. By defining an > advisor like this: > > from protocols.advice import addClassAdvisor > > def some_advisor(whatever_args): > def callback(cls): > print "I can modify",cls,"or replace it" > return cls > addClassAdvisor(callback) > > you can then use it in a class body like so: > > class SomeClass: > some_advisor("something") > > And the return value from 'callback' will replace SomeClass, just like a > decorator replaces the function it's called on. > > The implementation should work with any Python version from 2.2 up. I'm > not sure if it would work with IronPython. But if it doesn't, that would > be a good indication of feature(s) that IronPython is missing. ;) Just curious (and lazy): what magic is the implementation using that makes this work without a custom metaclass? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Tue Mar 28 06:41:50 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 27 Mar 2006 23:41:50 -0500 Subject: [Python-Dev] Class decorators In-Reply-To: References: <5.1.1.6.0.20060327211817.05cd3288@mail.telecommunity.com> <44268B6B.9070305@canterbury.ac.nz> <5.1.1.6.0.20060327211817.05cd3288@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060327233216.01fbc5e8@mail.telecommunity.com> At 08:02 PM 3/27/2006 -0800, Guido van Rossum wrote: >Just curious (and lazy): what magic is the implementation using that >makes this work without a custom metaclass? It registers a function as the __metaclass__ by poking it into the f_locals of the frame that's defining the class. This function has two attributes that denote 1) the callback to be invoked, and 2) the previous metaclass, forming a linked list of callbacks. When the class is created, this dummy metaclass function invokes the previous metaclass (or metaclass function) in the chain, then calls its callback on the result and returns that value, thus recursively unwinding the chain of advisors in the right order. (i.e., the same order as the advisors were called in the class body). The only part of the code that's even mildly complex is simulating Python's builtin rules for determining the default metaclass, so that when the recursion bottoms out you end up with an actual class with the right metaclass. And that's not so much complex as tedious. Anyway, the mechanism is orthogonal to the chosen metaclass, and thus can be used with classic classes or even ExtensionClasses if you're sufficiently masochistic. :) And AFAIK it's been in use by Zope 3 for some years now; in fact I think you were still working at Zope Corp. when Steve Alexander came up with the basic idea and I made the first practical implementation. From greg.ewing at canterbury.ac.nz Tue Mar 28 07:19:58 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 28 Mar 2006 17:19:58 +1200 Subject: [Python-Dev] INPLACE_ADD and INPLACE_MULTIPLY oddities in ceval.c In-Reply-To: References: Message-ID: <4428C77E.5020005@canterbury.ac.nz> Travis E. Oliphant wrote: > I think this can be fixed easily by first checking the sequence slot for > a sq_concat function before calling PyNumber_InPlaceAdd. However, if this *is* fixed, it looks like it's going to break NumPy, in the sense that it will no longer be able to force an arithmetic interpretation for "+" when adding a sequence to an array. Greg From greg.ewing at canterbury.ac.nz Tue Mar 28 07:28:57 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 28 Mar 2006 17:28:57 +1200 Subject: [Python-Dev] PySet API In-Reply-To: <1143516346.2317.80.camel@geddy.wooz.org> References: <1142705253.32074.72.camel@geddy.wooz.org> <005c01c64ac5$589859b0$b83efea9@RaymondLaptop1> <002b01c64aeb$44eb7750$b83efea9@RaymondLaptop1> <1142824236.10599.29.camel@resist.wooz.org> <00a001c64bfa$71a0ab20$b83efea9@RaymondLaptop1> <1142987079.10599.232.camel@resist.wooz.org> <004001c64d58$b0ebebf0$b83efea9@RaymondLaptop1> <1143327815.10799.42.camel@resist.wooz.org> <001a01c65082$35a4c840$b83efea9@RaymondLaptop1> <1143516346.2317.80.camel@geddy.wooz.org> Message-ID: <4428C999.5010801@canterbury.ac.nz> Barry Warsaw wrote: > My PySet_Clear() raises a SystemError > and returns -1 when the object is a frozen set. Isn't SystemError a bit drastic? TypeError would be sufficient here, surely. > > If PyObject_Clear() is implemented something like > > int PyObject_Clear(PyObject *o) > { > return (o->ob_type->tp_clear ? o->ob_type->tp_clear(o) : -1); > } > > then you /would/ be able to clear a frozen set. Hmmm, the problem here, I think, is that tp_clear is really only designed for use by the garbage collector. Giving anything else access to it is probably wrong. Clearability is not a general feature in Python land -- a few types have a clear() method, but this is an ad hoc feature of the type concerned. I don't think it makes sense to have a general PyObject_Clear function at all. -- Greg From greg.ewing at canterbury.ac.nz Tue Mar 28 07:40:52 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 28 Mar 2006 17:40:52 +1200 Subject: [Python-Dev] Class decorators In-Reply-To: <5.1.1.6.0.20060327233216.01fbc5e8@mail.telecommunity.com> References: <5.1.1.6.0.20060327211817.05cd3288@mail.telecommunity.com> <44268B6B.9070305@canterbury.ac.nz> <5.1.1.6.0.20060327211817.05cd3288@mail.telecommunity.com> <5.1.1.6.0.20060327233216.01fbc5e8@mail.telecommunity.com> Message-ID: <4428CC64.8020405@canterbury.ac.nz> Phillip J. Eby wrote: > It registers a function as the __metaclass__ by poking it into the f_locals > of the frame that's defining the class. That is stunningly brilliant! I'd nominate it for Hack of the Year if there were such an award. It's far too magical for me to feel like actually using it, though. Class decorators would be so much more straightforward and explicit. Greg From martin at v.loewis.de Tue Mar 28 08:07:08 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 28 Mar 2006 08:07:08 +0200 Subject: [Python-Dev] I'm not getting email from SF when assigned a bug/patch In-Reply-To: <17448.46306.112637.1041@montanaro.dyndns.org> References: <44285FBF.9090407@v.loewis.de> <17448.46306.112637.1041@montanaro.dyndns.org> Message-ID: <4428D28C.1000004@v.loewis.de> skip at pobox.com wrote: > Martin> I believe it broke at some point, I'm pretty certain it used to > Martin> work. > > I would expect that if it broke for Guido it broke for everybody. While we > consider him to be the BDFL I suspect the accolades are lost on the SF > folks. Indeed, it broke for me as well. I didn't notice until Guido's post that this is what happened - I had assumed I had filtered out/deleted these messages. Regards, Martin From martin at v.loewis.de Tue Mar 28 08:10:35 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 28 Mar 2006 08:10:35 +0200 Subject: [Python-Dev] PySet API In-Reply-To: <4428C999.5010801@canterbury.ac.nz> References: <1142705253.32074.72.camel@geddy.wooz.org> <005c01c64ac5$589859b0$b83efea9@RaymondLaptop1> <002b01c64aeb$44eb7750$b83efea9@RaymondLaptop1> <1142824236.10599.29.camel@resist.wooz.org> <00a001c64bfa$71a0ab20$b83efea9@RaymondLaptop1> <1142987079.10599.232.camel@resist.wooz.org> <004001c64d58$b0ebebf0$b83efea9@RaymondLaptop1> <1143327815.10799.42.camel@resist.wooz.org> <001a01c65082$35a4c840$b83efea9@RaymondLaptop1> <1143516346.2317.80.camel@geddy.wooz.org> <4428C999.5010801@canterbury.ac.nz> Message-ID: <4428D35B.7070208@v.loewis.de> Greg Ewing wrote: > Hmmm, the problem here, I think, is that tp_clear is > really only designed for use by the garbage collector. > Giving anything else access to it is probably wrong. > > Clearability is not a general feature in Python land -- > a few types have a clear() method, but this is an > ad hoc feature of the type concerned. I don't think > it makes sense to have a general PyObject_Clear > function at all. I agree. Barry's PySet_Clear and Raymond's PyObject_Clear would be two completely unrelated functions (one invoking the "clear" method, the other invoking tp_clear). Regards, Martin From anthony at interlink.com.au Tue Mar 28 08:24:38 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue, 28 Mar 2006 17:24:38 +1100 Subject: [Python-Dev] Inconsistency in 2.4.3 for __repr__() returning unicode In-Reply-To: <4427BB1A.3000201@egenix.com> References: <4f0b69dc0603262340l705ee84dr9cd8f356c464438e@mail.gmail.com> <4427BB1A.3000201@egenix.com> Message-ID: <200603281724.40352.anthony@interlink.com.au> On Monday 27 March 2006 21:14, M.-A. Lemburg wrote: > Change PyObject_Repr() to use the default encoding (again) > which is also consistent with how PyObject_Str() works. For 2.4.3, I plan to just revert the following patch (and supply a test case) Index: Objects/object.c =================================================================== --- Objects/object.c (revision 16197) +++ Objects/object.c (revision 16198) @@ -267,7 +267,7 @@ return NULL; if (PyUnicode_Check(res)) { PyObject* str; - str = PyUnicode_AsEncodedString(res, NULL, NULL); + str = PyUnicode_AsUnicodeEscapeString(res); Py_DECREF(res); if (str) res = str; Does anyone have any objections to this? The test suite passes with this (including the new test) as do various random tests I could string together. I need to apply this in the next short while, so if you have an issue with it, please speak up now! Thanks, Anthony > To make repr() conversion more robust, we could have > PyObject_Repr() apply the conversion using the 'replace' > error strategy - after all, repr() is usually only > used for debugging, where it's more important that > you do get an output rather than an exception. -- Anthony Baxter It's never too late to have a happy childhood. From nnorwitz at gmail.com Tue Mar 28 08:53:58 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 27 Mar 2006 22:53:58 -0800 Subject: [Python-Dev] improving quality Message-ID: We've made a lot of improvement with testing over the years. Recently, we've gotten even more serious with the buildbot, Coverity, and coverage (http://coverage.livinglogic.de). However, in order to improve quality even further, we need to do a little more work. This is especially important with the upcoming 2.5. Python 2.5 is the most fundamental set of changes to Python since 2.2. If we're to make this release work, we need to be very careful about it. In order to do the best possible job and avoid silly errors, there shouldn't be any checkins which could change behaviour that do not include a test. I'm not talking about updating comments or string constants. But even trivial changes can cause regressions or incompatible changes. Just like failing tests, code checked in without tests is fair game for being reverted if there is anything questionable. If you really can't figure out any way to test the change, please describe why in your checkin message. Just make sure it's true. It would be quite embarrassing to have your whole theory trashed when Uncle Timmy comes along 5 minutes later and checks in the test you just claimed was impossible. :-) Cheers, n From anthony at interlink.com.au Tue Mar 28 08:59:17 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue, 28 Mar 2006 17:59:17 +1100 Subject: [Python-Dev] pysqlite for 2.5? Message-ID: <200603281759.18756.anthony@interlink.com.au> This came up before (back in October 2004!) but didn't go anywhere since, AFAICR. Do we want to consider including pysqlite in Python 2.5? It's the only DB adaptor that I'd really consider suitable for shipping with the distribution, because it's self-contained. What's people's thoughts? Anthony -- Anthony Baxter It's never too late to have a happy childhood. From anthony at interlink.com.au Tue Mar 28 09:00:32 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue, 28 Mar 2006 18:00:32 +1100 Subject: [Python-Dev] improving quality In-Reply-To: References: Message-ID: <200603281800.33894.anthony@interlink.com.au> On Tuesday 28 March 2006 17:53, Neal Norwitz wrote: > In order to do the best possible job and avoid silly errors, there > shouldn't be any checkins which could change behaviour that do not > include a test. I'm not talking about updating comments or string > constants. But even trivial changes can cause regressions or > incompatible changes. Just like failing tests, code checked in > without tests is fair game for being reverted if there is anything > questionable. +1 from me. Anyone disagree? Anthony -- Anthony Baxter It's never too late to have a happy childhood. From tim.peters at gmail.com Tue Mar 28 09:01:15 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 28 Mar 2006 02:01:15 -0500 Subject: [Python-Dev] improving quality In-Reply-To: References: Message-ID: <1f7befae0603272301p42e0df85q2313b3eefa01d8f0@mail.gmail.com> [Neal Norwitz] ... > If you really can't figure out any way to test the change, please > describe why in your checkin message. Just make sure it's true. It > would be quite embarrassing to have your whole theory trashed when > Uncle Timmy comes along 5 minutes later and checks in the test you > just claimed was impossible. :-) And I, for one, find that especially embarrassing when Uncle Timmy did the original checkin ;-) BTW, I just noticed that the Mac buildbot died with a segfault on the 2.4 branch, a few tests after test_capi "finished". I'll backport the fix for that from the trunk now. BTW2, has anyone tested their favorite extension module on the trunk with a release build? If you don't start reporting horrible problems, I'm not going to revert the patch that causes them :-) From theller at python.net Tue Mar 28 09:04:43 2006 From: theller at python.net (Thomas Heller) Date: Tue, 28 Mar 2006 09:04:43 +0200 Subject: [Python-Dev] Libref sections to put new modules under? In-Reply-To: <200603271635.39335.fdrake@acm.org> References: <5.1.1.6.0.20060327161706.05c64fc0@mail.telecommunity.com> <200603271635.39335.fdrake@acm.org> Message-ID: <4428E00B.2070301@python.net> Fred L. Drake, Jr. wrote: > On Monday 27 March 2006 16:26, Phillip J. Eby wrote: > > Any thoughts on where documentation for the new "contextlib" module should > > go in the library reference? > > Most definately in "Python Runtime Services." > > > A similar issue exists for "ctypes" module, although I imagine an argument > > could easily be made for putting it with "Optional Operating System > > Services". > > I'm less sure of that one. It could reasonably go in "Python Runtime > Services," "Generic Operating System Services," or "Optional Operating System > Services." > I had the same thought as Philip. For a preview of what it could look like, see http://starship.python.net/crew/theller/python-docs/html/lib/ , especially http://starship.python.net/crew/theller/python-docs/html/lib/module-ctypes.html . Thomas From oliphant.travis at ieee.org Tue Mar 28 09:28:12 2006 From: oliphant.travis at ieee.org (Travis E. Oliphant) Date: Tue, 28 Mar 2006 00:28:12 -0700 Subject: [Python-Dev] INPLACE_ADD and INPLACE_MULTIPLY oddities in ceval.c In-Reply-To: <4428C77E.5020005@canterbury.ac.nz> References: <4428C77E.5020005@canterbury.ac.nz> Message-ID: Greg Ewing wrote: > Travis E. Oliphant wrote: > >> I think this can be fixed easily by first checking the sequence slot for >> a sq_concat function before calling PyNumber_InPlaceAdd. > > However, if this *is* fixed, it looks like it's going to > break NumPy, in the sense that it will no longer be able > to force an arithmetic interpretation for "+" when adding > a sequence to an array. Well, it won't break any "documented" behavior of NumPy which does not use the sq_concat (or sq_concat_inplace) slot at all. I can't imaging anybody relying on an in-place operations to return a "different" object, but we could make the change and run all the NumPy/SciPy tests to see what happens. But, you are right that w = [1,2,3,4] w += array([1,2,3,4]) would no longer make w equal to array([2,4,6,8]). But, this seems like a very good thing to me, as such behavior is actually quite hard to explain except as "its a bug" -Travis From anthony at interlink.com.au Tue Mar 28 09:41:27 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue, 28 Mar 2006 18:41:27 +1100 Subject: [Python-Dev] Inconsistency in 2.4.3 for __repr__() returning unicode In-Reply-To: <200603281724.40352.anthony@interlink.com.au> References: <4f0b69dc0603262340l705ee84dr9cd8f356c464438e@mail.gmail.com> <4427BB1A.3000201@egenix.com> <200603281724.40352.anthony@interlink.com.au> Message-ID: <200603281841.30774.anthony@interlink.com.au> Never mind. For 2.4.3, I reverted perky's patch for the unicode-escape, and reverted the old patch for PyObject_Repr on the trunk. After talking to perky and Neal, this seemed like the safest option for 2.4.3. Anthony From g.brandl at gmx.net Tue Mar 28 09:45:02 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 28 Mar 2006 09:45:02 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <200603281759.18756.anthony@interlink.com.au> References: <200603281759.18756.anthony@interlink.com.au> Message-ID: Anthony Baxter wrote: > This came up before (back in October 2004!) but didn't go anywhere > since, AFAICR. Do we want to consider including pysqlite in Python > 2.5? It's the only DB adaptor that I'd really consider suitable for > shipping with the distribution, because it's self-contained. > > What's people's thoughts? OTOH, +1 for a simple DB wrapper that makes it easy to start with DB-enabled applications. The trouble with it can't be worse than the BSDDB issues ;) OTOH, pysqlite2 seems to have had a fairly rapid sequence of releases in the past. I don't know whether it is now bug-free (the website claims that the 2.1 branch should be stable, and the 2.0 branch has proven stable). There also have been some API changes in the 2.0.x line, like the introduction of executemany() which broke e.g. SQLObject. Anyway, almost all popular web frameworks rely on PySQLite and seem to work well with it. Of course, speaking with Gerhard will be the way to find out more. Georg From raymond.hettinger at verizon.net Tue Mar 28 09:53:00 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Mon, 27 Mar 2006 23:53:00 -0800 Subject: [Python-Dev] PySet API Message-ID: <005f01c6523c$a2f5f020$1e2810ac@RaymondLaptop1> >> Thank would be nice. It gives me the ability to keep a clean, sane API while >> at >> the same time making sure that my most important customer (Barry) gets his >> needs >> met. > I'm having a hard time deciphering whether you're being condescending or > trying to be funny. I'm going to go with the latter. Good one! :) Neither. I was serious. You are a principal customer for setobjects. It seems that you make extensive use of them in your code. While I don't favor the proposed API, I think is essential that you not be left hanging without good options. Raymond From rasky at develer.com Tue Mar 28 10:13:24 2006 From: rasky at develer.com (Giovanni Bajo) Date: Tue, 28 Mar 2006 10:13:24 +0200 Subject: [Python-Dev] I'm not getting email from SF when assigned abug/patch References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> Message-ID: <00ff01c6523f$7c583290$bf03030a@trilan> Thomas Wouters wrote: >> Ah, glad to hear I wasn't the only one bitten by that. For a while, I >> thought I was going senile :-P Speaking of which, perhaps we should >> designate the running roundup instance on python.org as the issuetracker >> for py3k ? Gives us a good chance to evaluate, and if it doesn't work out, it >> wouldn't matter too much. Another option would be Bugzilla, which is proven to be stable, maintained and used succesfully by large open source projects (like GCC+RedHat+Binutils+Classpath). -- Giovanni Bajo From jcarlson at uci.edu Tue Mar 28 10:18:32 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 28 Mar 2006 00:18:32 -0800 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <200603281759.18756.anthony@interlink.com.au> References: <200603281759.18756.anthony@interlink.com.au> Message-ID: <20060328000835.05C2.JCARLSON@uci.edu> Anthony Baxter wrote: > This came up before (back in October 2004!) but didn't go anywhere > since, AFAICR. Do we want to consider including pysqlite in Python > 2.5? It's the only DB adaptor that I'd really consider suitable for > shipping with the distribution, because it's self-contained. > > What's people's thoughts? I can't help but be +1. Though I have found various uses for the btree, hash, and recno databases available via bsddb, having a built-in SQL database would be quite convenient; saving people from having to install any one of the dozen+ SQL variants for their platform in order to prototype or even develop a full system using SQL. While I haven't used pysqlite extensively, when I did, I found that it worked quite well for what I was doing. - Josiah From gh at ghaering.de Tue Mar 28 10:26:11 2006 From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=) Date: Tue, 28 Mar 2006 10:26:11 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: References: <200603281759.18756.anthony@interlink.com.au> Message-ID: <4428F323.6040605@ghaering.de> Georg Brandl wrote: > Anthony Baxter wrote: > >>This came up before (back in October 2004!) but didn't go anywhere >>since, AFAICR. Do we want to consider including pysqlite in Python >>2.5? It's the only DB adaptor that I'd really consider suitable for >>shipping with the distribution, because it's self-contained. >> >>What's people's thoughts? > > OTOH, +1 for a simple DB wrapper that makes it easy to start with DB-enabled > applications. The trouble with it can't be worse than the BSDDB issues ;) > > OTOH, pysqlite2 seems to have had a fairly rapid sequence of releases in the > past. That's because I decided for a more rapid release cycle than I used in the past. If bugs are fixed and no features planned to implement in the near future, I made a release. > I don't know whether it is now bug-free (the website claims that the > 2.1 branch should be stable, and the 2.0 branch has proven stable). There have been no more bug reports since 2.1, so I'm confident that all the glitches the switch to transparent compiled statements in 2.1 introduced are fixed now. > There also have been some API changes in the 2.0.x line, like the introduction > of executemany() which broke e.g. SQLObject. I missed that, can you provide a link please? pysqlite 2 was announced to be incompatible with pysqlite 1. I don't think there were any backwards incompatible API changes in the 2.x line. > Anyway, almost all popular web frameworks rely on PySQLite and seem to work > well with it. > > Of course, speaking with Gerhard will be the way to find out more. I'll try to throw in a bit more information that will be necessary for this discussion: pysqlite 2.x is (almost) feature complete now. I've a few more changes sitting in SVN trunk that are waiting for the pysqlite 2.2 release. These are all about wrapping more of the SQLite API, like custom collations. I *am* willing to be a maintainer of an SQLite module for Python. I will gladly help writing a PEP for it. But I won't be the champion for the idea, because I'm only +0 on adding external libraries to Python, like elementtree, or ctypes, or pysqlite instead of relying on setuptools/Cheese Shop. I could probably be convinced that a fat Python is still a good idea nowadays, though :-) -- Gerhard From anthony at interlink.com.au Tue Mar 28 10:30:58 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue, 28 Mar 2006 19:30:58 +1100 Subject: [Python-Dev] I'm not getting email from SF when assigned abug/patch In-Reply-To: <00ff01c6523f$7c583290$bf03030a@trilan> References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> Message-ID: <200603281931.00973.anthony@interlink.com.au> On Tuesday 28 March 2006 19:13, Giovanni Bajo wrote: > Another option would be Bugzilla, which is proven to be stable, > maintained and used succesfully by large open source projects (like > GCC+RedHat+Binutils+Classpath). Please god no. No bugzilla, no no no. Please! From rasky at develer.com Tue Mar 28 10:35:03 2006 From: rasky at develer.com (Giovanni Bajo) Date: Tue, 28 Mar 2006 10:35:03 +0200 Subject: [Python-Dev] I'm not getting email from SF when assigned abug/patch References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> <200603281931.00973.anthony@interlink.com.au> Message-ID: <018001c65242$828dad40$bf03030a@trilan> Anthony Baxter wrote: >> Another option would be Bugzilla, which is proven to be stable, >> maintained and used succesfully by large open source projects (like >> GCC+RedHat+Binutils+Classpath). > > Please god no. No bugzilla, no no no. Please! Care to elaborate? -- Giovanni Bajo From anthony at interlink.com.au Tue Mar 28 10:39:41 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue, 28 Mar 2006 19:39:41 +1100 Subject: [Python-Dev] I'm not getting email from SF when assigned abug/patch In-Reply-To: <018001c65242$828dad40$bf03030a@trilan> References: <200603281931.00973.anthony@interlink.com.au> <018001c65242$828dad40$bf03030a@trilan> Message-ID: <200603281939.43235.anthony@interlink.com.au> On Tuesday 28 March 2006 19:35, Giovanni Bajo wrote: > Anthony Baxter wrote: > >> Another option would be Bugzilla, which is proven to be stable, > >> maintained and used succesfully by large open source projects > >> (like GCC+RedHat+Binutils+Classpath). > > > > Please god no. No bugzilla, no no no. Please! > > Care to elaborate? Unbelievably complicated to setup and run. Awful default user interface (how many form fields can you get??) Awful code. -- Anthony Baxter It's never too late to have a happy childhood. From thomas at python.org Tue Mar 28 11:02:31 2006 From: thomas at python.org (Thomas Wouters) Date: Tue, 28 Mar 2006 11:02:31 +0200 Subject: [Python-Dev] [Python-checkins] r43358 - python/trunk/Modules/itertoolsmodule.c In-Reply-To: References: <20060327210214.2E8991E4004@bag.python.org> <007e01c651e3$1d863ac0$d200000a@RaymondLaptop1> <9e804ac0603271321x2aa99f58vf6013dafd0ca9a0c@mail.gmail.com> Message-ID: <9e804ac0603280102l4e6b42eena5619efc917aaed7@mail.gmail.com> On 3/27/06, Neal Norwitz wrote: > > On 3/27/06, Thomas Wouters wrote: > > > > The teeobject has GC (hence the word 'and' in 'itertools.tee and its > > internal teedataobject' ;-) The problem with test_generators is that > this > > also leaks: > > > > def leak(): > > def gen(): > > while True: > > yield g > > g = gen() > > > > leak() > > Please add a test for this to the leakers and remove the tee test in > leakers if that no longer leaks. I added this test, but I didn't remove test_tee, because it still leaks. I also checked in a test of code that used to leak, but doesn't anymore. The cycle this nested generator creates, which is also involved in the test_tee leak, is not cleanable by the cycle-gc, and it looks like it hasn't been since the yield-expr/coroutine patch was included in the trunk. I believe the culprit to be this part of that patch: Index: Modules/gcmodule.c =================================================================== --- Modules/gcmodule.c (revision 39238) +++ Modules/gcmodule.c (revision 39239) @@ -413,10 +413,8 @@ assert(delstr != NULL); return _PyInstance_Lookup(op, delstr) != NULL; } - else if (PyType_HasFeature(op->ob_type, Py_TPFLAGS_HEAPTYPE)) + else return op->ob_type->tp_del != NULL; - else - return 0; } /* Move the objects in unreachable with __del__ methods into `finalizers`. Now, reverting that part of the patch in revision 39239 triggers an assertion failure, but removing the assertion makes it work right; the above nested-generator cycle gets cleaned up again. Later in the trunk, the assertion was changed anyway, and it no longer fails if I just revert the gcmodule change. Of course, the reason the cycle is uncleanable is because generators grew a tp_del method, and that throws cycle-gc in a hissy fit; reverting the above patch just makes cycle-gc ignore the tp_del of heaptypes. I don't know enough about the cycle-gc to say whether that's good or bad, but not having generators be cleanable is not very good. For what it's worth, reverting the gcmodule patch doesn't seem to break any tests. However, fixing _both_ those things (itertools.tee lack of GC, and GC's inability to clean generators) *still does not fix the 254 refleaks in test_generators*. When I backport the itertools.tee-GC changes to r39238 (one checkin before coroutines), test_generators doesn't leak, neither the r39238 version of test_generators, nor the trunk version. One revision later, r39239, either test_generators leaks 15 references. 'Fixing' gcmodule, which fixes the nested-generator leak, does not change the number of leaks. And, as you all may be aware of, in the trunk, test_generators leaks 254 references; this is also the case with the gcmodule fix. So there's more huntin' afoot. In the mean time, if people knowledgeable about the cycle-GC care to comment about the gcmodule change wrt. heaptypes, please do. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060328/c31c1407/attachment.htm From tds333+pydev at gmail.com Tue Mar 28 11:51:17 2006 From: tds333+pydev at gmail.com (Wolfgang Langner) Date: Tue, 28 Mar 2006 11:51:17 +0200 Subject: [Python-Dev] I'm not getting email from SF when assigned abug/patch In-Reply-To: <200603281939.43235.anthony@interlink.com.au> References: <200603281931.00973.anthony@interlink.com.au> <018001c65242$828dad40$bf03030a@trilan> <200603281939.43235.anthony@interlink.com.au> Message-ID: <4c45c1530603280151s4b64e073ybd192f2d045b69e2@mail.gmail.com> Hello, what about trac: http://www.edgewall.com/trac/ It is based on python and has a very good svn integration. -- bye by Wolfgang From g.brandl at gmx.net Tue Mar 28 11:53:14 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 28 Mar 2006 11:53:14 +0200 Subject: [Python-Dev] I'm not getting email from SF when assigned abug/patch In-Reply-To: <200603281939.43235.anthony@interlink.com.au> References: <200603281931.00973.anthony@interlink.com.au> <018001c65242$828dad40$bf03030a@trilan> <200603281939.43235.anthony@interlink.com.au> Message-ID: Anthony Baxter wrote: > On Tuesday 28 March 2006 19:35, Giovanni Bajo wrote: >> Anthony Baxter wrote: >> >> Another option would be Bugzilla, which is proven to be stable, >> >> maintained and used succesfully by large open source projects >> >> (like GCC+RedHat+Binutils+Classpath). >> > >> > Please god no. No bugzilla, no no no. Please! >> >> Care to elaborate? > > Unbelievably complicated to setup and run. > Awful default user interface (how many form fields can you get??) > Awful code. Perl. Georg ;) From ncoghlan at gmail.com Tue Mar 28 12:11:48 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 28 Mar 2006 20:11:48 +1000 Subject: [Python-Dev] Libref sections to put new modules under? In-Reply-To: <5.1.1.6.0.20060327161706.05c64fc0@mail.telecommunity.com> References: <5.1.1.6.0.20060327161706.05c64fc0@mail.telecommunity.com> Message-ID: <44290BE4.4080208@gmail.com> Phillip J. Eby wrote: > What other new modules do we have going in 2.5 that haven't found a home in > the documentation yet? I seem to recall some discussion about a decorator > library or functools or something of that sort. The suggestion with functools was to rename functional.partial to functools.partial, and add a functools.decorator function. The idea of the latter being that sticking @decorator on the front would be enough to make a decorator function well-behaved (and future-proof). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From just at letterror.com Tue Mar 28 13:25:01 2006 From: just at letterror.com (Just van Rossum) Date: Tue, 28 Mar 2006 12:25:01 +0100 Subject: [Python-Dev] I'm not getting email from SF when assigned abug/patch In-Reply-To: <4c45c1530603280151s4b64e073ybd192f2d045b69e2@mail.gmail.com> Message-ID: Wolfgang Langner wrote: > what about trac: > > http://www.edgewall.com/trac/ > > It is based on python and has a very good svn integration. We started using it recently and so far it's working really well. I love the svn (and wiki!) integration. However, I have no idea how well it scales to a project the size of Python. Just From g.brandl at gmx.net Tue Mar 28 12:38:32 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 28 Mar 2006 12:38:32 +0200 Subject: [Python-Dev] Error msgs for new-style division Message-ID: Hi, (this makes test_ctypes fail, therefore I noticed) currently with -Qnew: >>> 2/0 Traceback (most recent call last): File "", line 1, in ? ZeroDivisionError: float division >>> 2L/0 Traceback (most recent call last): File "", line 1, in ? ZeroDivisionError: long division or modulo by zero >>> 2/0L Traceback (most recent call last): File "", line 1, in ? ZeroDivisionError: long division or modulo by zero This seems a little inconsistent. Do we care? Georg From thomas at python.org Tue Mar 28 12:54:07 2006 From: thomas at python.org (Thomas Wouters) Date: Tue, 28 Mar 2006 12:54:07 +0200 Subject: [Python-Dev] I'm not getting email from SF when assigned abug/patch In-Reply-To: <4c45c1530603280151s4b64e073ybd192f2d045b69e2@mail.gmail.com> References: <200603281931.00973.anthony@interlink.com.au> <018001c65242$828dad40$bf03030a@trilan> <200603281939.43235.anthony@interlink.com.au> <4c45c1530603280151s4b64e073ybd192f2d045b69e2@mail.gmail.com> Message-ID: <9e804ac0603280254w2ae72044s289d6facdc2b076b@mail.gmail.com> On 3/28/06, Wolfgang Langner wrote: > what about trac: > > http://www.edgewall.com/trac/ > > It is based on python and has a very good svn integration. Sorry, I should have realized more than half of python-dev lacked the context in which I made my suggestion. At PyCon and in a few other select groups, there's been talk about swapping SF for 'something else'. (Actually, there's been a lot of talk about that everywhere, but I'm talking about concrete proposals ;) The problem is 'which 'something else''. There is a PSF committee that is charged with proposing and trying out, but it got somewhat stuck at trying to get information from SourceForge. My suggestion is to use the existing, installed, configured roundup installation so we can give it a whirl. Obviously, we will also want to give trac and jira and any other serious alternatives a try. (But Bugzilla and RT have already been veto'd.) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060328/e51f1e2d/attachment.html From g.brandl at gmx.net Tue Mar 28 12:55:28 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 28 Mar 2006 12:55:28 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <4428F323.6040605@ghaering.de> References: <200603281759.18756.anthony@interlink.com.au> <4428F323.6040605@ghaering.de> Message-ID: Gerhard H?ring wrote: > Georg Brandl wrote: >> Anthony Baxter wrote: >> >>>This came up before (back in October 2004!) but didn't go anywhere >>>since, AFAICR. Do we want to consider including pysqlite in Python >>>2.5? It's the only DB adaptor that I'd really consider suitable for >>>shipping with the distribution, because it's self-contained. >>> >>>What's people's thoughts? >> >> OTOH, +1 for a simple DB wrapper that makes it easy to start with DB-enabled >> applications. The trouble with it can't be worse than the BSDDB issues ;) >> >> OTOH, pysqlite2 seems to have had a fairly rapid sequence of releases in the >> past. > > That's because I decided for a more rapid release cycle than I used in > the past. If bugs are fixed and no features planned to implement in the > near future, I made a release. Sounds fair. >> I don't know whether it is now bug-free (the website claims that the >> 2.1 branch should be stable, and the 2.0 branch has proven stable). > > There have been no more bug reports since 2.1, so I'm confident that all > the glitches the switch to transparent compiled statements in 2.1 > introduced are fixed now. > >> There also have been some API changes in the 2.0.x line, like the introduction >> of executemany() which broke e.g. SQLObject. > > I missed that, can you provide a link please? pysqlite 2 was announced > to be incompatible with pysqlite 1. I don't think there were any > backwards incompatible API changes in the 2.x line. Never mind, you're right. I had another piece of software in my head ;) >> Anyway, almost all popular web frameworks rely on PySQLite and seem to work >> well with it. >> >> Of course, speaking with Gerhard will be the way to find out more. > > I'll try to throw in a bit more information that will be necessary for > this discussion: > > pysqlite 2.x is (almost) feature complete now. I've a few more changes > sitting in SVN trunk that are waiting for the pysqlite 2.2 release. > These are all about wrapping more of the SQLite API, like custom collations. In what timeframe will those be completed? > I *am* willing to be a maintainer of an SQLite module for Python. I will > gladly help writing a PEP for it. But I won't be the champion for the > idea, because I'm only +0 on adding external libraries to Python, like > elementtree, or ctypes, or pysqlite instead of relying on > setuptools/Cheese Shop. > > I could probably be convinced that a fat Python is still a good idea > nowadays, though :-) Even though setuptools are a very good concept and implementation, "ships with Python" is still a different kind of statement. Many people think that every external package to maintain is one too much... Georg From theller at python.net Tue Mar 28 12:57:10 2006 From: theller at python.net (Thomas Heller) Date: Tue, 28 Mar 2006 12:57:10 +0200 Subject: [Python-Dev] Error msgs for new-style division In-Reply-To: References: Message-ID: <44291686.5060507@python.net> Georg Brandl wrote: > Hi, > > (this makes test_ctypes fail, therefore I noticed) > > currently with -Qnew: > >>>> 2/0 > Traceback (most recent call last): > File "", line 1, in ? > ZeroDivisionError: float division >>>> 2L/0 > Traceback (most recent call last): > File "", line 1, in ? > ZeroDivisionError: long division or modulo by zero >>>> 2/0L > Traceback (most recent call last): > File "", line 1, in ? > ZeroDivisionError: long division or modulo by zero > I'll fix that in the ctypes tests. Thomas From arigo at tunes.org Tue Mar 28 13:28:56 2006 From: arigo at tunes.org (Armin Rigo) Date: Tue, 28 Mar 2006 13:28:56 +0200 Subject: [Python-Dev] INPLACE_ADD and INPLACE_MULTIPLY oddities in ceval.c In-Reply-To: References: Message-ID: <20060328112856.GA21294@code0.codespeak.net> Hi, On Mon, Mar 27, 2006 at 08:00:09PM -0800, Guido van Rossum wrote: > So for consistency we want a += b to also execute a.__iadd__. The > opcode calls PyNumber_InplaceAdd; I think that PyNumber_InplaceAdd > (and PySequence_InplaceConcat, if it exists) should test for both the > numeric and the sequence augmented slot of the left argument first; > then they should try both the numeric and sequence non-augmented slot > of the left argument; and then the numeric non-augmented slot of the > right argument. Coercion should not be attempted at all. > > The question is, can we do this in 2.5 without breaking backwards > compatibility? Someone else with more time should look into the > details of that. I agree that there is a bug. There is more than one inconsistency left around here, though. Fixing one might expose the next one... For example, if -- as the documention says -- the expression 'a + b' would really try all slots corresponding to a.__add__(b) first and then fall back only if the slots return NotImplemented, then we'd also have to fix the following to return NotImplemented: >>> [].__add__(5) TypeError: can only concatenate list (not "int") to list and then we have no place to put that nice error message. Nevertheless I think we should fix all this for consistency. I can try to give it a good look. I don't think many programs would break if the change goes into 2.5, but there are some C extension modules out there abusing the inner details of the type slots in unpredictable ways... A bientot, Armin From gh at ghaering.de Tue Mar 28 13:41:21 2006 From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=) Date: Tue, 28 Mar 2006 13:41:21 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: References: <200603281759.18756.anthony@interlink.com.au> <4428F323.6040605@ghaering.de> Message-ID: <442920E1.70509@ghaering.de> Georg Brandl wrote: > Gerhard H?ring wrote: >>I'll try to throw in a bit more information that will be necessary for >>this discussion: >> >>pysqlite 2.x is (almost) feature complete now. I've a few more changes >>sitting in SVN trunk that are waiting for the pysqlite 2.2 release. >>These are all about wrapping more of the SQLite API, like custom collations. > > In what timeframe will those be completed? I would have waited for a pysqlite 2.2 release until I found a good API for logging for pysqlite (mainly for reporting exceptions in Python callbacks). I will probably defer the logging stuff to pysqlite 2.3 and release pysqlite 2.2 soon with the features currently in SVN. I know that pushing new things into Python 2.5 should happen soon, if at all. So *if* pysqlite should go into Python, I propose that I release pysqlite 2.2.0 and we integrate that into the Python alpha release. If this is going to happen, I want it to happen under a different package name than "pysqlite2" and it's probably a good idea to skip "sqlite" too, because there are still users of the SQLite 1.x API (*) and this is the package name of that API. OTOH, "sqlite" would be the natural name if it ends up in Python. Perhaps "sqlitedb" ... Extensive unit tests are available, as is a reference manual in ReST format. -- Gerhard (*) Although if they're really using pysqlite 0.x/1.x then it's unlikely they'll switch to Python 2.5 with their applications. From barry at python.org Tue Mar 28 14:37:08 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 28 Mar 2006 07:37:08 -0500 Subject: [Python-Dev] improving quality In-Reply-To: <200603281800.33894.anthony@interlink.com.au> References: <200603281800.33894.anthony@interlink.com.au> Message-ID: <1143549428.2309.137.camel@geddy.wooz.org> On Tue, 2006-03-28 at 18:00 +1100, Anthony Baxter wrote: > On Tuesday 28 March 2006 17:53, Neal Norwitz wrote: > > In order to do the best possible job and avoid silly errors, there > > shouldn't be any checkins which could change behaviour that do not > > include a test. I'm not talking about updating comments or string > > constants. But even trivial changes can cause regressions or > > incompatible changes. Just like failing tests, code checked in > > without tests is fair game for being reverted if there is anything > > questionable. > > +1 from me. > > Anyone disagree? Not me! -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060328/2c9ee991/attachment-0001.pgp From barry at python.org Tue Mar 28 15:59:11 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 28 Mar 2006 08:59:11 -0500 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <200603281759.18756.anthony@interlink.com.au> References: <200603281759.18756.anthony@interlink.com.au> Message-ID: <1143554351.2309.158.camel@geddy.wooz.org> On Tue, 2006-03-28 at 17:59 +1100, Anthony Baxter wrote: > This came up before (back in October 2004!) but didn't go anywhere > since, AFAICR. Do we want to consider including pysqlite in Python > 2.5? It's the only DB adaptor that I'd really consider suitable for > shipping with the distribution, because it's self-contained. SQLite/pysqlite is an awesome combination, and I do think it would make for a more compelling Python distribution. I've used it in several projects, usually with great success. My only hesitation is that in one project, we've had to stick with SQLite 3.2.1 and pysqlite 1.1.6 because we encountered some concurrency problems when trying to upgrade the former to 3.2.8. We couldn't upgrade both to the latest versions due to external project management constraints. But I'm not sure that's enough of a reason to vote against inclusion, so I guess I'm +0. Having Gerhard accept maintainership definitely pushes it from -0 to +0 in my mind. It's got lots of unit tests, so the only thing I'd probably want to see are Python docs for the module. I think Gerhard could still do more frequent releases if he wanted to -- I do that with the email package, and while it's a bit of a PITA, it's doable. Either sqlite or sqlitedb are fine names. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060328/765dc9cf/attachment.pgp From barry at python.org Tue Mar 28 16:02:56 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 28 Mar 2006 09:02:56 -0500 Subject: [Python-Dev] I'm not getting email from SF when assigned abug/patch In-Reply-To: <00ff01c6523f$7c583290$bf03030a@trilan> References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> Message-ID: <1143554576.2310.161.camel@geddy.wooz.org> On Tue, 2006-03-28 at 10:13 +0200, Giovanni Bajo wrote: > Another option would be Bugzilla, which is proven to be stable, maintained > and used succesfully by large open source projects (like > GCC+RedHat+Binutils+Classpath). The infrastructure committee (of which I'm a member but not the chair) is examining the alternatives and trying to put up some live demos for people to check out. Getting the data out of SourceForge has been the sticky issue though, and last I recall, we were waiting for some response from them about export bugs we'd encountered. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060328/9026921b/attachment.pgp From gmccaughan at synaptics-uk.com Tue Mar 28 16:07:17 2006 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Tue, 28 Mar 2006 14:07:17 +0000 Subject: [Python-Dev] PySet API In-Reply-To: <1143468446.10799.50.camel@resist.wooz.org> References: <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> <1143468446.10799.50.camel@resist.wooz.org> Message-ID: <200603281507.18175.gmccaughan@synaptics-uk.com> > We're clearly going in circles here, and it's obvious we're not going to > agree. > > The fact that PySet_Next() can be used incorrectly is no reason not to > include it. [etc] For what it's worth[1], I think Raymond is absolutely on crack here. [1] Not necessarily very much. There is none of my code in Python, so far as I know. * Simple API: The complexity of an API is not determined by the number of methods in it but by the variety of different things you can ask it to do, and it's not any simpler to have PyObject_CallMethod(x, "foo") PyObject_CallMethod(x, "bar") PyObject_CallMethod(x, "baz") than to have PyObject_foo(x) PyObject_bar(x) PyObject_baz(x) API complexity is measured in brain cells, not in methods. * Ease of making mistakes: The Python API is absolutely stuffed with places where you can go wrong by forgetting about subtle refcounting issues. Sure, it's nice to minimize that pain, but it's never going to be possible to write much code that uses the C API without being alert to such issues. (Incidentally, the more things you have that can only be done by invoking PyObject_CallMethod, the more places you have where you have to assume that arbitrary Python code may have been called and that reference counts may have changed behind your back.) * Duck typing: Yup, supporting duck typing is good. That's why we have an abstract API. There are concrete APIs for all sorts of particular kinds of Python object; it seems pretty clear to me that this isn't a mistake, and that sets should be one such type. Clients get to choose how to trade off the benefits in efficiency, conciseness and clarity from using the concrete API against the benefits in generality from using the abstract one. And when PySet_Add is the obvious way to add items to sets, how much C code using sets is likely to work with things that merely walk and quack like sets, anyway? * Efficiency: Anyone measured this? The mere fact that the overhead of (say) emptying a set using PyObject_CallMethod is O(1) doesn't mean it's insignificant. For many applications the size of your sets is O(1) too. (Often with quite a small implicit constant, too.) -- Gareth McCaughan From anthony at interlink.com.au Tue Mar 28 16:51:59 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed, 29 Mar 2006 01:51:59 +1100 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <1143554351.2309.158.camel@geddy.wooz.org> References: <200603281759.18756.anthony@interlink.com.au> <1143554351.2309.158.camel@geddy.wooz.org> Message-ID: <200603290152.00598.anthony@interlink.com.au> I'm happy to work with Gerhard to make this happen. Does it need a PEP? I'd say "no", but only because things like ElementTree didn't, either. Does it need a BDFL pronouncement? I'd say yes. Anthony From rasky at develer.com Tue Mar 28 16:59:41 2006 From: rasky at develer.com (Giovanni Bajo) Date: Tue, 28 Mar 2006 16:59:41 +0200 Subject: [Python-Dev] I'm not getting email from SF when assignedabug/patch References: <4c45c1530603280151s4b64e073ybd192f2d045b69e2@mail.gmail.com> Message-ID: <086c01c65278$3de52b60$bf03030a@trilan> Just van Rossum wrote: >> http://www.edgewall.com/trac/ >> >> It is based on python and has a very good svn integration. > > We started using it recently and so far it's working really well. I love > the svn (and wiki!) integration. However, I have no idea how well it > scales to a project the size of Python. Having extensively used both Trac and Bugzilla, let me say that the ticket tracker in Trac is a child-play version of Bugzilla. It might be enough for Python, though, if SF was enough till now. I thought that a large project like Python required something more advanced. Anyway, I'll shut up as I see there is a committee for this decision. The integration between tickets/svn/wiki in Trac is cute though, even if, after a while, you'd really want that mailman parsed that syntax as well (and maybe your MUA too :) -- Giovanni Bajo From skip at pobox.com Tue Mar 28 17:23:48 2006 From: skip at pobox.com (skip at pobox.com) Date: Tue, 28 Mar 2006 09:23:48 -0600 Subject: [Python-Dev] I'm not getting email from SF when assigned abug/patch In-Reply-To: <1143554576.2310.161.camel@geddy.wooz.org> References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> <1143554576.2310.161.camel@geddy.wooz.org> Message-ID: <17449.21764.475497.457592@montanaro.dyndns.org> Barry> The infrastructure committee (of which I'm a member but not the Barry> chair) is examining the alternatives and trying to put up some Barry> live demos for people to check out. Roundup is there now, right (sans SF export)? Trac is being used by the folks doing the new website. What other candidates are being considered? Based on my brief experience as a Bugzilla user (just trying to be a good citizen and report Mozilla bugs a few years ago), I would vote -1. I'd hate to think the bug reporting interface was *so* bad that it alone would discourage people from reporting bugs. Still, I gave up reporting Mozilla bugs because of it. Skip From chris at atlee.ca Tue Mar 28 17:39:18 2006 From: chris at atlee.ca (Chris AtLee) Date: Tue, 28 Mar 2006 10:39:18 -0500 Subject: [Python-Dev] improving quality In-Reply-To: References: Message-ID: <7790b6530603280739y423098a1v8c83c2558f7fdd3c@mail.gmail.com> On 3/28/06, Neal Norwitz wrote: > We've made a lot of improvement with testing over the years. > Recently, we've gotten even more serious with the buildbot, Coverity, > and coverage (http://coverage.livinglogic.de). However, in order to > improve quality even further, we need to do a little more work. This > is especially important with the upcoming 2.5. Python 2.5 is the most > fundamental set of changes to Python since 2.2. If we're to make this > release work, we need to be very careful about it. This reminds me of something I've been wanting to ask for a while: does anybody run python through valgrind on a regular basis? I've noticed that valgrind complains a lot about invalid reads in PyObject_Free. I know that valgrind can warn about things that turn out not to be problems, but would generating a suppresion file and running all or part of the test suite through valgrind on the buildbots be useful? Cheers, Chris From thomas at python.org Tue Mar 28 18:28:46 2006 From: thomas at python.org (Thomas Wouters) Date: Tue, 28 Mar 2006 18:28:46 +0200 Subject: [Python-Dev] improving quality In-Reply-To: <7790b6530603280739y423098a1v8c83c2558f7fdd3c@mail.gmail.com> References: <7790b6530603280739y423098a1v8c83c2558f7fdd3c@mail.gmail.com> Message-ID: <9e804ac0603280828g78435fa9hf29e9e3f6cfe6a8@mail.gmail.com> On 3/28/06, Chris AtLee wrote: > > On 3/28/06, Neal Norwitz wrote: > > We've made a lot of improvement with testing over the years. > > Recently, we've gotten even more serious with the buildbot, Coverity, > > and coverage (http://coverage.livinglogic.de). However, in order to > > improve quality even further, we need to do a little more work. This > > is especially important with the upcoming 2.5. Python 2.5 is the most > > fundamental set of changes to Python since 2.2. If we're to make this > > release work, we need to be very careful about it. > > This reminds me of something I've been wanting to ask for a while: > does anybody run python through valgrind on a regular basis? I've > noticed that valgrind complains a lot about invalid reads in > PyObject_Free. I know that valgrind can warn about things that turn > out not to be problems, but would generating a suppresion file and > running all or part of the test suite through valgrind on the > buildbots be useful? See Misc/README.valgrind and Misc/valgrind-python.supp. I ran Python with valgrind two weeks ago, and didn't encounter any new problems. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060328/a604abe6/attachment.html From lists at janc.be Tue Mar 28 18:56:53 2006 From: lists at janc.be (Jan Claeys) Date: Tue, 28 Mar 2006 18:56:53 +0200 Subject: [Python-Dev] I'm not getting email from SF when assigned abug/patch In-Reply-To: <17449.21764.475497.457592@montanaro.dyndns.org> References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> <1143554576.2310.161.camel@geddy.wooz.org> <17449.21764.475497.457592@montanaro.dyndns.org> Message-ID: <1143565014.3712.26.camel@localhost.localdomain> Op di, 28-03-2006 te 09:23 -0600, schreef skip at pobox.com: > Based on my brief experience as a Bugzilla user (just trying to be a good > citizen and report Mozilla bugs a few years ago), I would vote -1. I'd hate > to think the bug reporting interface was *so* bad that it alone would > discourage people from reporting bugs. Still, I gave up reporting Mozilla > bugs because of it. GNOME uses a Bugzilla "fork" that's more user-friendly... (But other people say Bugzilla is also difficult for admins, so that's probably not the only important thing.) -- Jan Claeys From pedronis at strakt.com Tue Mar 28 19:41:12 2006 From: pedronis at strakt.com (Samuele Pedroni) Date: Tue, 28 Mar 2006 19:41:12 +0200 Subject: [Python-Dev] Class decorators In-Reply-To: References: <44268B6B.9070305@canterbury.ac.nz> Message-ID: <44297538.8060002@strakt.com> Mike Krell wrote: >Greg Ewing canterbury.ac.nz> writes: > > > >>I've just been playing around with metaclasses, and >>I think I've stumbled across a reason for having >>class decorators as an alternative to metaclasses >>for some purposes. >> >> > >There has also been discussion on the IronPython mailing list that class >decorators would be a very useful syntax for expressing .NET attributes. > >http://lists.ironpython.com/pipermail/users-ironpython.com/2006-March/002007.html > > the same is true for Jython and Java annotations. From guido at python.org Tue Mar 28 19:50:49 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 28 Mar 2006 09:50:49 -0800 Subject: [Python-Dev] INPLACE_ADD and INPLACE_MULTIPLY oddities in ceval.c In-Reply-To: <20060328112856.GA21294@code0.codespeak.net> References: <20060328112856.GA21294@code0.codespeak.net> Message-ID: On 3/28/06, Armin Rigo wrote: > Hi, > > On Mon, Mar 27, 2006 at 08:00:09PM -0800, Guido van Rossum wrote: > > So for consistency we want a += b to also execute a.__iadd__. The > > opcode calls PyNumber_InplaceAdd; I think that PyNumber_InplaceAdd > > (and PySequence_InplaceConcat, if it exists) should test for both the > > numeric and the sequence augmented slot of the left argument first; > > then they should try both the numeric and sequence non-augmented slot > > of the left argument; and then the numeric non-augmented slot of the > > right argument. Coercion should not be attempted at all. > > > > The question is, can we do this in 2.5 without breaking backwards > > compatibility? Someone else with more time should look into the > > details of that. > > I agree that there is a bug. There is more than one inconsistency left > around here, though. Fixing one might expose the next one... For > example, if -- as the documention says -- the expression 'a + b' would > really try all slots corresponding to a.__add__(b) first and then fall > back only if the slots return NotImplemented, then we'd also have to fix > the following to return NotImplemented: > > >>> [].__add__(5) > TypeError: can only concatenate list (not "int") to list > > and then we have no place to put that nice error message. > > Nevertheless I think we should fix all this for consistency. I can try > to give it a good look. I don't think many programs would break if the > change goes into 2.5, but there are some C extension modules out there > abusing the inner details of the type slots in unpredictable ways... Thanks for looking into this! C extensions are my main worry -- OTOH if += for a list can already passes arbitrary types as the argument, then any extension types should be ready to expect this, right? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Mar 28 20:01:44 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 28 Mar 2006 10:01:44 -0800 Subject: [Python-Dev] Class decorators In-Reply-To: <44297538.8060002@strakt.com> References: <44268B6B.9070305@canterbury.ac.nz> <44297538.8060002@strakt.com> Message-ID: On 3/28/06, Samuele Pedroni wrote: > Mike Krell wrote: > >Greg Ewing canterbury.ac.nz> writes: > >>I've just been playing around with metaclasses, and > >>I think I've stumbled across a reason for having > >>class decorators as an alternative to metaclasses > >>for some purposes. > > > >There has also been discussion on the IronPython mailing list that class > >decorators would be a very useful syntax for expressing .NET attributes. > > > >http://lists.ironpython.com/pipermail/users-ironpython.com/2006-March/002007.html > the same is true for Jython and Java annotations. OK, I'm convinced (mostly by the awful hackery that Phillip so proudly exposed :-). I propose that someone start writing a Py3k PEP for class decorators. I don't think it's fair to the 2.5 release team to want to push this into 2.5 though; how about 2.6? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jcarlson at uci.edu Tue Mar 28 20:17:16 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 28 Mar 2006 10:17:16 -0800 Subject: [Python-Dev] I'm not getting email from SF when assigned abug/patch In-Reply-To: <200603281931.00973.anthony@interlink.com.au> References: <00ff01c6523f$7c583290$bf03030a@trilan> <200603281931.00973.anthony@interlink.com.au> Message-ID: <20060328101649.05C5.JCARLSON@uci.edu> Anthony Baxter wrote: > > On Tuesday 28 March 2006 19:13, Giovanni Bajo wrote: > > Another option would be Bugzilla, which is proven to be stable, > > maintained and used succesfully by large open source projects (like > > GCC+RedHat+Binutils+Classpath). > > Please god no. No bugzilla, no no no. Please! I second the "god no. No bugzilla" sentiment. - Josiah From nnorwitz at gmail.com Tue Mar 28 20:16:01 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 28 Mar 2006 10:16:01 -0800 Subject: [Python-Dev] Class decorators In-Reply-To: References: <44268B6B.9070305@canterbury.ac.nz> <44297538.8060002@strakt.com> Message-ID: On 3/28/06, Guido van Rossum wrote: > > I propose that someone start writing a Py3k PEP for class decorators. > I don't think it's fair to the 2.5 release team to want to push this > into 2.5 though; how about 2.6? Wasn't there already a (pretty small) patch? I guess it would be different now with the AST though. n From jack at performancedrivers.com Tue Mar 28 20:27:54 2006 From: jack at performancedrivers.com (Jack Diederich) Date: Tue, 28 Mar 2006 13:27:54 -0500 Subject: [Python-Dev] Class decorators In-Reply-To: References: <44268B6B.9070305@canterbury.ac.nz> <44297538.8060002@strakt.com> Message-ID: <20060328182754.GE5787@performancedrivers.com> On Tue, Mar 28, 2006 at 10:16:01AM -0800, Neal Norwitz wrote: > On 3/28/06, Guido van Rossum wrote: > > > > I propose that someone start writing a Py3k PEP for class decorators. > > I don't think it's fair to the 2.5 release team to want to push this > > into 2.5 though; how about 2.6? > > Wasn't there already a (pretty small) patch? I guess it would be > different now with the AST though. > I submitted one a couple years ago. The AST makes it obsolete though. I'd be happy to make a new AST friendly one (time to learn the AST..) -Jack From fredrik at pythonware.com Tue Mar 28 20:41:54 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue, 28 Mar 2006 20:41:54 +0200 Subject: [Python-Dev] pysqlite for 2.5? References: <200603281759.18756.anthony@interlink.com.au> <4428F323.6040605@ghaering.de> <442920E1.70509@ghaering.de> Message-ID: Gerhard Häring wrote: > I know that pushing new things into Python 2.5 should happen soon, if at > all. So *if* pysqlite should go into Python, I propose that I release > pysqlite 2.2.0 and we integrate that into the Python alpha release. +1 ! > If this is going to happen, I want it to happen under a different > package name than "pysqlite2" and it's probably a good idea to skip > "sqlite" too, because there are still users of the SQLite 1.x API (*) > and this is the package name of that API. OTOH, "sqlite" would be the > natural name if it ends up in Python. Perhaps "sqlitedb" ... db.sqlite3 ? (it wraps version 3 of the sqlite engine, after all...) From pje at telecommunity.com Tue Mar 28 20:46:50 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 28 Mar 2006 13:46:50 -0500 Subject: [Python-Dev] Class decorators In-Reply-To: References: <44297538.8060002@strakt.com> <44268B6B.9070305@canterbury.ac.nz> <44297538.8060002@strakt.com> Message-ID: <5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com> At 10:01 AM 3/28/2006 -0800, Guido van Rossum wrote: >OK, I'm convinced (mostly by the awful hackery that Phillip so proudly >exposed :-). Just as a historical note, here's where you previously rejected the same hackery as an argument for supporting class decorators: http://mail.python.org/pipermail/python-dev/2004-March/043462.html Ironically, the subsequent discussion following the above message brought me around to your point of view. :) Or more precisely, the subsequent discussion and examples convinced me that putting class decorators on top of the class was bad for readability, vs. putting them in the body just after the docstring. As you said, "the use cases are certainly very *different* than those for function/method decorators". So at this point I'd rather see a way to make my hackery go away (or become part of the standard library in some fashion) rather than simply mimic @decorators for classes. From martin at v.loewis.de Tue Mar 28 21:00:40 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 28 Mar 2006 21:00:40 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <200603281759.18756.anthony@interlink.com.au> References: <200603281759.18756.anthony@interlink.com.au> Message-ID: <442987D8.4010903@v.loewis.de> Anthony Baxter wrote: > This came up before (back in October 2004!) but didn't go anywhere > since, AFAICR. Do we want to consider including pysqlite in Python > 2.5? It's the only DB adaptor that I'd really consider suitable for > shipping with the distribution, because it's self-contained. > > What's people's thoughts? We shouldn't include anything without the explicit permission of the author(s); if possible, the authors should grant the PSF permission to change the license (i.e. fill out the contributor agreement). Even better, the authors should be willing to keep the version in Python synchronized with the separate release. Regards, Martin From guido at python.org Tue Mar 28 21:04:54 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 28 Mar 2006 11:04:54 -0800 Subject: [Python-Dev] Class decorators In-Reply-To: <5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com> References: <44268B6B.9070305@canterbury.ac.nz> <44297538.8060002@strakt.com> <5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com> Message-ID: On 3/28/06, Phillip J. Eby wrote: > At 10:01 AM 3/28/2006 -0800, Guido van Rossum wrote: > >OK, I'm convinced (mostly by the awful hackery that Phillip so proudly > >exposed :-). > > Just as a historical note, here's where you previously rejected the same > hackery as an argument for supporting class decorators: > > http://mail.python.org/pipermail/python-dev/2004-March/043462.html > > Ironically, the subsequent discussion following the above message brought > me around to your point of view. :) > > Or more precisely, the subsequent discussion and examples convinced me that > putting class decorators on top of the class was bad for readability, vs. > putting them in the body just after the docstring. As you said, "the use > cases are certainly very *different* than those for function/method > decorators". > > So at this point I'd rather see a way to make my hackery go away (or become > part of the standard library in some fashion) rather than simply mimic > @decorators for classes. That's fine. But there's also the C#/Java POV. I'm somehow concerned that any mechanism that puts the syntax inside the class body is somehow going to have a hackish implementation, but I'd be glad to be proven wrong, so please come up with a concrete proposal! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Tue Mar 28 21:07:00 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 28 Mar 2006 21:07:00 +0200 Subject: [Python-Dev] I'm not getting email from SF when assigned abug/patch In-Reply-To: <17449.21764.475497.457592@montanaro.dyndns.org> References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> <1143554576.2310.161.camel@geddy.wooz.org> <17449.21764.475497.457592@montanaro.dyndns.org> Message-ID: <44298954.4040705@v.loewis.de> skip at pobox.com wrote: > Roundup is there now, right (sans SF export)? Richard Jones has an SF importer for one of the two XML-like formats, the one that is correct XML but with incomplete data. The other format, which has complete data but is ill-formed XML, has no importer into roundup at the moment. > Trac is being used by the > folks doing the new website. What other candidates are being considered? My view is that nothing should be "considered" unless there is a) a volunteer to operate the tracker (or, failing that, somebody who does it for money), and b) an importer of whatever data SF can provide. Regards, Martin From g.brandl at gmx.net Tue Mar 28 21:22:58 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 28 Mar 2006 21:22:58 +0200 Subject: [Python-Dev] What about PEP 299? Message-ID: Hi, since I found myself writing "if __name__ == '__main__'" often these days, I wondered whether PEP 299 could be pronounced upon. I'm not proposing putting it into 2.5, but it should be relatively small a change. Cheers, Georg From pje at telecommunity.com Tue Mar 28 21:41:52 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 28 Mar 2006 14:41:52 -0500 Subject: [Python-Dev] Class decorators In-Reply-To: References: <5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com> <44268B6B.9070305@canterbury.ac.nz> <44297538.8060002@strakt.com> <5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060328142333.01f82410@mail.telecommunity.com> At 11:04 AM 3/28/2006 -0800, Guido van Rossum wrote: >That's fine. But there's also the C#/Java POV. Can someone point me to examples of C# class "attributes" and Java annotations that they'd like to use with this mechanism? I would indeed like to see how those use cases compare with mine. > I'm somehow concerned >that any mechanism that puts the syntax inside the class body is >somehow going to have a hackish implementation, That depends on whether there's syntax support, or it's just something that happens at runtime. I'm not opposed to having syntax, I just don't think that putting the syntax outside the class is the best thing for most of my class decorator use cases. Nonetheless, neither special syntax nor hackiness is really required. For example, your proposal here: http://mail.python.org/pipermail/python-dev/2004-April/043913.html was to use a '__decorators__' attribute, which would then be processed by a metaclass. However, this could simply become a standard magic attribute ala __metaclass__, and be processed by the class creation machinery after first invoking the correct metaclass. I don't really *like* that approach, but it requires neither syntax nor hacks, (nor a custom metaclass) and it could be syntax-sugared later. For that matter, if this __decorators__ machinery existed, the existing advisor implementation could become a lot less hacky. The only magical bit at that point would be doing something like: locals = sys._getframe(n).f_locals. locals.setdefault('__decorators__',[]).append(decorator) And this of course could go in a function in the stdlib, rather than have everybody writing their own _getframe() hacks. From g.brandl at gmx.net Tue Mar 28 21:40:08 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 28 Mar 2006 21:40:08 +0200 Subject: [Python-Dev] Reminder: Bug Day this Friday, 31st of March Message-ID: Hello, it's time for the 7th Python Bug Day, just before 2.5 alpha 1 is released. The aim of the bug day is to close as many bugs, patches and feature requests as possible, this time with a focus on small feature additions that can still go into the upcoming 2.5 alpha release. When? ^^^^^ The bug day will take place on Friday, March 31st, running from approximately 1PM to 8PM GMT (9AM to 4PM Eastern time). You don't need to be around all day; feel free to stop by for a few hours and contribute. Where and How? ^^^^^^^^^^^^^^ To join, stop by the IRC channel #python-dev on irc.freenode.net, where efforts will be discussed and coordinated. We'll collaboratively go through the Python bug database at SourceForge and fix things as they come up. IMPORTANT: *No* prior knowledge of the Python source is necessary to participate! You'll get all assistance the developers can offer for starting up with helping, this is in fact a good opportunity to learn the basics. Bug day participation helps the developers and makes Python 2.5 a better release by reducing the backlog of bugs and patches. Plus, it's fun! More information ^^^^^^^^^^^^^^^^ For instructions and more information, see the Wiki page at http://www.python.org/cgi-bin/moinmoin/PythonBugDay Cheers, Georg From g.brandl at gmx.net Tue Mar 28 21:31:20 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 28 Mar 2006 21:31:20 +0200 Subject: [Python-Dev] I'm not getting email from SF when assigned abug/patch In-Reply-To: <44298954.4040705@v.loewis.de> References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> <1143554576.2310.161.camel@geddy.wooz.org> <17449.21764.475497.457592@montanaro.dyndns.org> <44298954.4040705@v.loewis.de> Message-ID: Martin v. L?wis wrote: > skip at pobox.com wrote: >> Roundup is there now, right (sans SF export)? > > Richard Jones has an SF importer for one of the two XML-like formats, > the one that is correct XML but with incomplete data. The other format, > which has complete data but is ill-formed XML, has no importer into > roundup at the moment. > >> Trac is being used by the >> folks doing the new website. What other candidates are being considered? > > My view is that nothing should be "considered" unless there is > a) a volunteer to operate the tracker (or, failing that, somebody who > does it for money), and > b) an importer of whatever data SF can provide. There exists a Trac importer for SF data at http://projects.edgewall.com/trac/browser/trunk/contrib/sourceforge2trac.py though I don't know which format it does expect. Generally, I like Trac very much, especially for its interconnected subsystems. I've used it with smaller projects, and there it works perfectly. Having said that, I don't know if the Trac ticket system (which would be the most important subsystem for us) scales up well enough. Of course, if there are only a few bits missing, instead of paying someone to operate a complicated tracker, perhaps the money could be used to pay someone to improve Trac... Georg From pje at telecommunity.com Tue Mar 28 21:47:07 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 28 Mar 2006 14:47:07 -0500 Subject: [Python-Dev] What about PEP 299? In-Reply-To: Message-ID: <5.1.1.6.0.20060328144418.01fae7f0@mail.telecommunity.com> At 09:22 PM 3/28/2006 +0200, Georg Brandl wrote: >Hi, > >since I found myself writing "if __name__ == '__main__'" >often these days, I wondered whether PEP 299 could be pronounced >upon. I'm not proposing putting it into 2.5, but it should be >relatively small a change. A couple of issues that the PEP doesn't address: * How can you write code that runs in multiple Python versions with this? If you use the current idiom to invoke __main__, it's going to get invoked twice. * A module that imports __main__ (using "import __main__") is going to get a TypeError unless the implementation checks that __main__ is not in fact the __main__ module. From g.brandl at gmx.net Tue Mar 28 21:33:34 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 28 Mar 2006 21:33:34 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: References: <200603281759.18756.anthony@interlink.com.au> <4428F323.6040605@ghaering.de> <442920E1.70509@ghaering.de> Message-ID: Fredrik Lundh wrote: > Gerhard H?ring wrote: > >> I know that pushing new things into Python 2.5 should happen soon, if at >> all. So *if* pysqlite should go into Python, I propose that I release >> pysqlite 2.2.0 and we integrate that into the Python alpha release. > > +1 ! > >> If this is going to happen, I want it to happen under a different >> package name than "pysqlite2" and it's probably a good idea to skip >> "sqlite" too, because there are still users of the SQLite 1.x API (*) >> and this is the package name of that API. OTOH, "sqlite" would be the >> natural name if it ends up in Python. Perhaps "sqlitedb" ... > > db.sqlite3 ? That would make sense if inclusion of more database-related modules was planned. Whatever name is chosen: please no "from sqlite3 import dbapi2 as sqlite"! Georg From barry at python.org Tue Mar 28 22:12:47 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 28 Mar 2006 15:12:47 -0500 Subject: [Python-Dev] PySet API In-Reply-To: <4428C999.5010801@canterbury.ac.nz> References: <1142705253.32074.72.camel@geddy.wooz.org> <005c01c64ac5$589859b0$b83efea9@RaymondLaptop1> <002b01c64aeb$44eb7750$b83efea9@RaymondLaptop1> <1142824236.10599.29.camel@resist.wooz.org> <00a001c64bfa$71a0ab20$b83efea9@RaymondLaptop1> <1142987079.10599.232.camel@resist.wooz.org> <004001c64d58$b0ebebf0$b83efea9@RaymondLaptop1> <1143327815.10799.42.camel@resist.wooz.org> <001a01c65082$35a4c840$b83efea9@RaymondLaptop1> <1143516346.2317.80.camel@geddy.wooz.org> <4428C999.5010801@canterbury.ac.nz> Message-ID: <1143576767.10792.115.camel@resist.wooz.org> On Tue, 2006-03-28 at 17:28 +1200, Greg Ewing wrote: > Barry Warsaw wrote: > > My PySet_Clear() raises a SystemError > > and returns -1 when the object is a frozen set. > > Isn't SystemError a bit drastic? TypeError would be > sufficient here, surely. Possibly, but all the other PySet_*() functions call PyErr_BadInternalCall() when they get a type they don't accept, so PySet_Clear() should be consistent. > > If PyObject_Clear() is implemented something like > > > > int PyObject_Clear(PyObject *o) > > { > > return (o->ob_type->tp_clear ? o->ob_type->tp_clear(o) : -1); > > } > > > > then you /would/ be able to clear a frozen set. > > Hmmm, the problem here, I think, is that tp_clear is > really only designed for use by the garbage collector. > Giving anything else access to it is probably wrong. Exactly. > Clearability is not a general feature in Python land -- > a few types have a clear() method, but this is an > ad hoc feature of the type concerned. I don't think > it makes sense to have a general PyObject_Clear > function at all. I'm thinking the same thing, which is why I'm now favoring PySet_Clear() again. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060328/1ff0835a/attachment.pgp From python at discworld.dyndns.org Tue Mar 28 22:21:01 2006 From: python at discworld.dyndns.org (Charles Cazabon) Date: Tue, 28 Mar 2006 14:21:01 -0600 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: References: <200603281759.18756.anthony@interlink.com.au> <442920E1.70509@ghaering.de> Message-ID: <20060328202101.GA8345@discworld.dyndns.org> Georg Brandl wrote: > Fredrik Lundh wrote: > > > > db.sqlite3 ? +1 > That would make sense if inclusion of more database-related modules was > planned. Yup; I'd be happy to see db.mysql and db.pgsql or whatnot added as appropriate, and having a nice new namespace ready for them is a good idea. > Whatever name is chosen: please no "from sqlite3 import dbapi2 as sqlite"! Amen. db.sqlite3 is the perfect name. Charles -- ----------------------------------------------------------------------- Charles Cazabon GPL'ed software available at: http://pyropus.ca/software/ ----------------------------------------------------------------------- From barry at python.org Tue Mar 28 22:36:55 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 28 Mar 2006 15:36:55 -0500 Subject: [Python-Dev] PySet API In-Reply-To: <005f01c6523c$a2f5f020$1e2810ac@RaymondLaptop1> References: <005f01c6523c$a2f5f020$1e2810ac@RaymondLaptop1> Message-ID: <1143578215.10793.138.camel@resist.wooz.org> On Mon, 2006-03-27 at 23:53 -0800, Raymond Hettinger wrote: > While I don't favor the proposed API, I think is essential that > you not be left hanging without good options. Thank you. So where does this leave us? BTW, Guido made a suggestion in private email (which he okayed to mention publicly). Quoting: "OTOH, our experience with dicts have shown that the C dict API causes major constraints on dict subclassing (since the C APIs mostly ignore overridden methods) and the proliferation of their use makes it hard to implement features like allowing arbitrary mappings as globals/locals for exec/eval." "Perhaps the PySet API can raise an error if it's ever called on something that's not *exactly* a set? No subclassing allowed. Shouldn't affect you, and should be an effective deterrent against abuse of the kind that made the PyDict API an albatross." This would involve changing the type test in PySet_Next() from a PyAnySet_Check() to a PyAnySet_CheckExact() so that it only works for sets and frozen sets. Guido's exactly right, it wouldn't affect us so it seems like it would be fine. I'm not sure if this directly addresses your concerns though, since they seem mostly centered around the safety (or lack thereof) of PySet_Next(). -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060328/58162fd1/attachment-0001.pgp From barry at python.org Tue Mar 28 22:40:46 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 28 Mar 2006 15:40:46 -0500 Subject: [Python-Dev] PySet API In-Reply-To: <200603281507.18175.gmccaughan@synaptics-uk.com> References: <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> <1143468446.10799.50.camel@resist.wooz.org> <200603281507.18175.gmccaughan@synaptics-uk.com> Message-ID: <1143578446.10793.141.camel@resist.wooz.org> Excerpting... On Tue, 2006-03-28 at 14:07 +0000, Gareth McCaughan wrote: > * Simple API: > > API complexity is measured in brain cells, not in methods. > > * Ease of making mistakes: > > The Python API is absolutely stuffed with places where you can go wrong > by forgetting about subtle refcounting issues. Sure, it's nice to minimize > that pain, but it's never going to be possible to write much code that > uses the C API without being alert to such issues. > * Duck typing: > > Yup, supporting duck typing is good. That's why we have an abstract API. > There are concrete APIs for all sorts of particular kinds of Python object; > it seems pretty clear to me that this isn't a mistake, and that sets should > be one such type. Clients get to choose how to trade off the benefits in > efficiency, conciseness and clarity from using the concrete API against > the benefits in generality from using the abstract one. > * Efficiency: > > Anyone measured this? The mere fact that the overhead of (say) emptying > a set using PyObject_CallMethod is O(1) doesn't mean it's insignificant. > For many applications the size of your sets is O(1) too. (Often with > quite a small implicit constant, too.) My sentiments exactly Gareth. Thanks for putting it so much more eloquently than I have. :) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060328/85fa6019/attachment.pgp From barry at python.org Tue Mar 28 22:48:36 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 28 Mar 2006 15:48:36 -0500 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <200603290152.00598.anthony@interlink.com.au> References: <200603281759.18756.anthony@interlink.com.au> <1143554351.2309.158.camel@geddy.wooz.org> <200603290152.00598.anthony@interlink.com.au> Message-ID: <1143578916.10793.143.camel@resist.wooz.org> On Wed, 2006-03-29 at 01:51 +1100, Anthony Baxter wrote: > I'm happy to work with Gerhard to make this happen. Does it need a > PEP? I'd say "no", Agreed. pysqlite is solid and widely accepted, and AFAIK has no competition. > but only because things like ElementTree didn't, > either. Does it need a BDFL pronouncement? I'd say yes. If Guido even cares ;), but if not, RM pronouncement is good enough for me. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060328/06ceddee/attachment.pgp From exarkun at divmod.com Tue Mar 28 23:22:17 2006 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Tue, 28 Mar 2006 16:22:17 -0500 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <1143578916.10793.143.camel@resist.wooz.org> Message-ID: <20060328212217.22481.256092693.divmod.quotient.6808@ohm> On Tue, 28 Mar 2006 15:48:36 -0500, Barry Warsaw wrote: >On Wed, 2006-03-29 at 01:51 +1100, Anthony Baxter wrote: >> I'm happy to work with Gerhard to make this happen. Does it need a >> PEP? I'd say "no", > >Agreed. pysqlite is solid and widely accepted, and AFAIK has no >competition. FWIW: http://www.rogerbinns.com/apsw.html Jean-Paul From guido at python.org Tue Mar 28 23:47:03 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 28 Mar 2006 13:47:03 -0800 Subject: [Python-Dev] What about PEP 299? In-Reply-To: References: Message-ID: On 3/28/06, Georg Brandl wrote: > since I found myself writing "if __name__ == '__main__'" > often these days, I wondered whether PEP 299 could be pronounced > upon. I'm not proposing putting it into 2.5, but it should be > relatively small a change. If you're asking for a quick pronouncement, it's going to be "no". It's not worth the change (in docs, user habits, etc.) and there's nothing particularly broken. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at discworld.dyndns.org Wed Mar 29 00:44:00 2006 From: python at discworld.dyndns.org (Charles Cazabon) Date: Tue, 28 Mar 2006 16:44:00 -0600 Subject: [Python-Dev] What about PEP 299? In-Reply-To: References: Message-ID: <20060328224400.GA9042@discworld.dyndns.org> Guido van Rossum wrote: > On 3/28/06, Georg Brandl wrote: > > since I found myself writing "if __name__ == '__main__'" > > often these days, I wondered whether PEP 299 could be pronounced > > upon. I'm not proposing putting it into 2.5, but it should be > > relatively small a change. > > If you're asking for a quick pronouncement, it's going to be "no". > It's not worth the change (in docs, user habits, etc.) and there's > nothing particularly broken. It might be worth instead adding an option flag to the executable that implies "from the loaded module, run __main__() with sys.argv as its argument(s)", so the user can get this behaviour with `python -X somemodule.py`. Charles -- ----------------------------------------------------------------------- Charles Cazabon GPL'ed software available at: http://pyropus.ca/software/ ----------------------------------------------------------------------- From guido at python.org Wed Mar 29 00:53:31 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 28 Mar 2006 14:53:31 -0800 Subject: [Python-Dev] What about PEP 299? In-Reply-To: <20060328224400.GA9042@discworld.dyndns.org> References: <20060328224400.GA9042@discworld.dyndns.org> Message-ID: On 3/28/06, Charles Cazabon wrote: > It might be worth instead adding an option flag to the executable that implies > "from the loaded module, run __main__() with sys.argv as its argument(s)", so > the user can get this behaviour with `python -X somemodule.py`. You can do "python -x somemodule" as long as somemodule.py uses the if __name__=='__main__' convention. What does your proposal add? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Mar 29 00:54:11 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 28 Mar 2006 14:54:11 -0800 Subject: [Python-Dev] What about PEP 299? In-Reply-To: References: <20060328224400.GA9042@discworld.dyndns.org> Message-ID: Sorry, I meant "python -m somemodule". On 3/28/06, Guido van Rossum wrote: > On 3/28/06, Charles Cazabon wrote: > > It might be worth instead adding an option flag to the executable that implies > > "from the loaded module, run __main__() with sys.argv as its argument(s)", so > > the user can get this behaviour with `python -X somemodule.py`. > > You can do "python -x somemodule" as long as somemodule.py uses the if > __name__=='__main__' convention. What does your proposal add? > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at discworld.dyndns.org Wed Mar 29 00:58:58 2006 From: python at discworld.dyndns.org (Charles Cazabon) Date: Tue, 28 Mar 2006 16:58:58 -0600 Subject: [Python-Dev] What about PEP 299? In-Reply-To: References: <20060328224400.GA9042@discworld.dyndns.org> Message-ID: <20060328225858.GA9229@discworld.dyndns.org> Guido van Rossum wrote: > On 3/28/06, Charles Cazabon wrote: > > It might be worth instead adding an option flag to the executable that implies > > "from the loaded module, run __main__() with sys.argv as its argument(s)", so > > the user can get this behaviour with `python -X somemodule.py`. > > You can do "python -m somemodule" as long as somemodule.py uses the if > __name__=='__main__' convention. What does your proposal add? Well, it's not really my proposal. I'm just suggesting the behaviour in PEP299 could be added via a commandline option so that it doesn't introduce backward-incompatibility. I don't see any particular benefit to PEP299 myself -- I'd rather just add an entry to the style guide about standardizing the "executable stanza" and main() function. Charles -- ----------------------------------------------------------------------- Charles Cazabon GPL'ed software available at: http://pyropus.ca/software/ ----------------------------------------------------------------------- From guido at python.org Wed Mar 29 01:06:57 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 28 Mar 2006 15:06:57 -0800 Subject: [Python-Dev] What about PEP 299? In-Reply-To: <20060328225858.GA9229@discworld.dyndns.org> References: <20060328224400.GA9042@discworld.dyndns.org> <20060328225858.GA9229@discworld.dyndns.org> Message-ID: OK. -1 on PEP 299 it is. On 3/28/06, Charles Cazabon wrote: > Guido van Rossum wrote: > > On 3/28/06, Charles Cazabon wrote: > > > It might be worth instead adding an option flag to the executable that implies > > > "from the loaded module, run __main__() with sys.argv as its argument(s)", so > > > the user can get this behaviour with `python -X somemodule.py`. > > > > You can do "python -m somemodule" as long as somemodule.py uses the if > > __name__=='__main__' convention. What does your proposal add? > > Well, it's not really my proposal. I'm just suggesting the behaviour in > PEP299 could be added via a commandline option so that it doesn't introduce > backward-incompatibility. > > I don't see any particular benefit to PEP299 myself -- I'd rather just add an > entry to the style guide about standardizing the "executable stanza" and main() function. > > Charles > -- > ----------------------------------------------------------------------- > Charles Cazabon > GPL'ed software available at: http://pyropus.ca/software/ > ----------------------------------------------------------------------- > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.peters at gmail.com Wed Mar 29 01:13:47 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 28 Mar 2006 18:13:47 -0500 Subject: [Python-Dev] [Python-checkins] r43358 - python/trunk/Modules/itertoolsmodule.c In-Reply-To: <9e804ac0603280102l4e6b42eena5619efc917aaed7@mail.gmail.com> References: <20060327210214.2E8991E4004@bag.python.org> <007e01c651e3$1d863ac0$d200000a@RaymondLaptop1> <9e804ac0603271321x2aa99f58vf6013dafd0ca9a0c@mail.gmail.com> <9e804ac0603280102l4e6b42eena5619efc917aaed7@mail.gmail.com> Message-ID: <1f7befae0603281513r415b46g7996e218642b2c8b@mail.gmail.com> [Thomas Wouters] > ... > The cycle this nested generator creates, which is also involved in the test_tee > leak, is not cleanable by the cycle-gc, and it looks like it hasn't been > since the yield-expr/coroutine patch was included in the trunk. That could very well be. Adding finalizers to generators could legitimately make some cycles "suddenly" uncollectable. There was a burst of list traffic about this on the 18th and 19th of June, 2005, under subject: gcmodule issue w/adding __del__ to generator objects I starred it in gmail at the time (which is why I was able to find it again ;-)), but never had time to pay attention. > I believe the culprit to be this part of that patch: > > Index: Modules/gcmodule.c >=================================================================== > --- Modules/gcmodule.c (revision 39238) > +++ Modules/gcmodule.c (revision 39239) > @@ -413,10 +413,8 @@ > assert(delstr != NULL); > return _PyInstance_Lookup(op, delstr) != NULL; > } > - else if (PyType_HasFeature(op->ob_type, Py_TPFLAGS_HEAPTYPE)) > + else > return op->ob_type->tp_del != NULL; > - else > - return 0; > } Right, that's the patch that taught gc that generators now have finalizers. The original code can be read in two ways: 1. As a whole, it was an implementation of: Only instances of old- or new-style classes can have finalizers. An instance of an old-style class has a finalizer iff it has a method named "__del__". An instance of a new-style class (PyType_HasFeature(op->ob_type, Py_TPFLAGS_HEAPTYPE) has a finalizer iff its tp_del is non-NULL. 2. Because of the obscure gimmicks that try to cater to using old binary extension modules across new Python releases without recompiling, there's no guarantee that the tp_del slot even exists, and therefore we don't try to access tp_del at all unless PyType_HasFeature says the type object was compiled recently enough so that we know tp_del does exist. When generators grew finalizers, the "Only instances of ... classes can have finalizers" part of #1 became wrong, and I expect that's why Phillip removed the conditional. It was the right thing to do from that point of view. I don't understand the #2 gimmicks well enough to guess whether it was actually safe to remove all guards before trying to access tp_del (I run on Windows, and compiled extensions can never be reused across Python minor releases on Windows -- if a problem was introduced here, I'll never see it). > Now, reverting that part of the patch in revision 39239 There may be a reason to change that patch (due to #2 above), but generators do have finalizers now and the #1 part must not be reverted (although it may be fruitful to complicate it ;-)). > triggers an assertion failure, but removing the assertion makes it work right; No, it's not right if has_finalizer(g) returns 0 for all generators g. > the above nested-generator cycle gets cleaned up again. Later in the trunk, the > assertion was changed anyway, and it no longer fails if I just revert the > gcmodule change. Of course, the reason the cycle is uncleanable is because > generators grew a tp_del method, and that throws cycle-gc in a hissy fit; If by "hissy fit" you mean "gcmodule properly moves generators to the list of objects with finalizers, thereby preventing catastrophes", right, that's an intended hissy fit ;-) > reverting the above patch just makes cycle-gc ignore the tp_del of > heaptypes. I don't know enough about the cycle-gc to say whether that's good > or bad, Ignoring all generators' tp_del would be disastrous (opens pure-Python code to segfaults, etc). > but not having generators be cleanable is not very good. Not disastrous, though. > For what it's worth, reverting the gcmodule patch doesn't seem to break any > tests. I believe that. gc disasters are typically very difficult to provoke --, the first time :-) > However, fixing _both_ those things (itertools.tee lack of GC, and GC's > inability to clean generators) *still does not fix the 254 refleaks in > test_generators*. When I backport the itertools.tee-GC changes to r39238 > (one checkin before coroutines), test_generators doesn't leak, neither the > r39238 version of test_generators, nor the trunk version. One revision > later, r39239, either test_generators leaks 15 references. 'Fixing' > gcmodule, which fixes the nested-generator leak, does not change the number > of leaks. And, as you all may be aware of, in the trunk, test_generators > leaks 254 references; this is also the case with the gcmodule fix. So > there's more huntin' afoot. > > In the mean time, if people knowledgeable about the cycle-GC care to comment > about the gcmodule change wrt. heaptypes, please do. Did the best I could above, short of explaining exactly how failing to identify an object with a finalizer can lead to disaster. Short course: gc obviously needs to know which objects are and are not trash. If a finalizer is associated with an object in cyclic trash, then trash objects are potentially visible _from_ the finalizer, and therefore can be resurrected by running the finalizer. gc must therefore identify all trash objects reachable from trash objects with finalizers, because running finalizers _may_ make all such objects "not trash" again. Getting any part of that wrong can lead to calling tp_clear on an object that a finalizer actually resurrects, leading to symptoms from "hey, all the attributes on my object suddenly vanished by magic, for no reason at all" to segfaults. From pje at telecommunity.com Wed Mar 29 01:24:49 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 28 Mar 2006 18:24:49 -0500 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <20060328212217.22481.256092693.divmod.quotient.6808@ohm> References: <1143578916.10793.143.camel@resist.wooz.org> Message-ID: <5.1.1.6.0.20060328181905.01f92a68@mail.telecommunity.com> At 04:22 PM 3/28/2006 -0500, Jean-Paul Calderone wrote: >On Tue, 28 Mar 2006 15:48:36 -0500, Barry Warsaw wrote: > >On Wed, 2006-03-29 at 01:51 +1100, Anthony Baxter wrote: > >> I'm happy to work with Gerhard to make this happen. Does it need a > >> PEP? I'd say "no", > > > >Agreed. pysqlite is solid and widely accepted, and AFAIK has no > >competition. > >FWIW: http://www.rogerbinns.com/apsw.html That implementation doesn't support the DBAPI, although it sounds as though the differences are mostly shallow and easy to add implementations for. (e.g., implementing the fetch* methods, a description property, etc.). More likely to be an issue is that the author hasn't volunteered to contribute/support it. On the plus side, it sounds like ASPW is a more general wrapping of SQLite, which seems to me to lean in its favor for the stdlib, if it can also be brought into DBAPI compliance. From pje at telecommunity.com Wed Mar 29 01:36:00 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 28 Mar 2006 18:36:00 -0500 Subject: [Python-Dev] [Python-checkins] r43358 - python/trunk/Modules/itertoolsmodule.c In-Reply-To: <1f7befae0603281513r415b46g7996e218642b2c8b@mail.gmail.com> References: <9e804ac0603280102l4e6b42eena5619efc917aaed7@mail.gmail.com> <20060327210214.2E8991E4004@bag.python.org> <007e01c651e3$1d863ac0$d200000a@RaymondLaptop1> <9e804ac0603271321x2aa99f58vf6013dafd0ca9a0c@mail.gmail.com> <9e804ac0603280102l4e6b42eena5619efc917aaed7@mail.gmail.com> Message-ID: <5.1.1.6.0.20060328182758.01fb58d0@mail.telecommunity.com> At 06:13 PM 3/28/2006 -0500, Tim Peters wrote: >2. Because of the obscure gimmicks that try to cater to using old > binary extension modules across new Python releases without > recompiling, there's no guarantee that the tp_del slot even exists, > and therefore we don't try to access tp_del at all unless > PyType_HasFeature says the type object was compiled recently > enough so that we know tp_del does exist. > >When generators grew finalizers, the "Only instances of ... classes >can have finalizers" part of #1 became wrong, and I expect that's why >Phillip removed the conditional. It was the right thing to do from >that point of view. > >I don't understand the #2 gimmicks well enough to guess whether it was >actually safe to remove all guards before trying to access tp_del (I >run on Windows, and compiled extensions can never be reused across >Python minor releases on Windows -- if a problem was introduced here, >I'll never see it). By that reasoning, binary compatibility won't be an issue anywhere else, either, since the change was made on the 2.5 alpha trunk, and ISTM that 2.5 will require recompiling extensions anyway. Now, the trick could potentially be made a bit smarter if there were a slot by which gcmodule could ask the object for a finalizer *dynamically*. A generator without an active frame (or an active frame with no "try" blocks on the frame's block stack), has no need to run Python code for finalization. Calling tp_clear on such a generator will do anything that the actual deletion would, only faster. :) So, that approach could be used to get rid of most new leaks caused by generator cycles, at the expense of a bit more complexity in the gc and in generators. It won't fix leaks caused by cycles in active generators that have active try/finally or try/except blocks, since these are the things that actually need finalizing. From thomas at python.org Wed Mar 29 01:36:47 2006 From: thomas at python.org (Thomas Wouters) Date: Wed, 29 Mar 2006 01:36:47 +0200 Subject: [Python-Dev] Fwd: [Python-checkins] r43358 - python/trunk/Modules/itertoolsmodule.c In-Reply-To: <9e804ac0603281534g13ee19f1i9c955cbb55db23aa@mail.gmail.com> References: <20060327210214.2E8991E4004@bag.python.org> <007e01c651e3$1d863ac0$d200000a@RaymondLaptop1> <9e804ac0603271321x2aa99f58vf6013dafd0ca9a0c@mail.gmail.com> <9e804ac0603280102l4e6b42eena5619efc917aaed7@mail.gmail.com> <1f7befae0603281513r415b46g7996e218642b2c8b@mail.gmail.com> <9e804ac0603281534g13ee19f1i9c955cbb55db23aa@mail.gmail.com> Message-ID: <9e804ac0603281536r56c88cfx2e1c5704e57d42e5@mail.gmail.com> Wupsie, this was meant for all of python-dev ;P ---------- Forwarded message ---------- From: Thomas Wouters Date: Mar 29, 2006 1:34 AM Subject: Re: [Python-Dev] [Python-checkins] r43358 - python/trunk/Modules/itertoolsmodule.c To: Tim Peters On 3/29/06, Tim Peters wrote: > > ... that throws cycle-gc in a hissy fit; > If by "hissy fit" you mean "gcmodule properly moves generators to the list of objects with finalizers, thereby preventing catastrophes", right, that's an intended hissy fit ;-) Well, I meant 'the durned thing refuses to do what I want it to do', which is clear cycles. I guess the hissy fit was mine ;) Did the best I could above, short of explaining exactly how failing to > identify an object with a finalizer can lead to disaster. Much appreciated. Short course: gc obviously needs to know which objects are and are not > trash. If a finalizer is associated with an object in cyclic trash, > then trash objects are potentially visible _from_ the finalizer, and > therefore can be resurrected by running the finalizer. gc must > therefore identify all trash objects reachable from trash objects with > finalizers, because running finalizers _may_ make all such objects > "not trash" again. Getting any part of that wrong can lead to calling > tp_clear on an object that a finalizer actually resurrects, leading to > symptoms from "hey, all the attributes on my object suddenly vanished > by magic, for no reason at all" to segfaults. > So does that make all cycles involving only objects with finalizers impervious to cycle-gc? I guess it'd have to be that way. That also means I can stop searching for leaks in the C code, as we'll never be able to fix them all, anyway (although 'good code' could manually break many of the cycles.) At least not while keeping the tp_del of generator objects, not to mention avoiding finalizers on other builtin objects. (None seem to have them at the moment, they all do their stuff in tp_dealloc, which I guess doesn't allow objects to reincarnate themselves.) I'll see about fixing the Python code to avoid or explicitly break the cycles. Maybe we can patch regrtest.py to take into account uncollectable cycles, so that it could report them separately from refleaks. I'm going to think about all this first, though. Hrrm-hrrm'ly y'rs, -- Thomas Wouters Hi! I'm a finalizer virus! copy me into your tp_del slot to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060329/e1fc1748/attachment.html From thomas at python.org Wed Mar 29 01:44:18 2006 From: thomas at python.org (Thomas Wouters) Date: Wed, 29 Mar 2006 01:44:18 +0200 Subject: [Python-Dev] [Python-checkins] r43358 - python/trunk/Modules/itertoolsmodule.c In-Reply-To: <9e804ac0603281536r56c88cfx2e1c5704e57d42e5@mail.gmail.com> References: <20060327210214.2E8991E4004@bag.python.org> <007e01c651e3$1d863ac0$d200000a@RaymondLaptop1> <9e804ac0603271321x2aa99f58vf6013dafd0ca9a0c@mail.gmail.com> <9e804ac0603280102l4e6b42eena5619efc917aaed7@mail.gmail.com> <1f7befae0603281513r415b46g7996e218642b2c8b@mail.gmail.com> <9e804ac0603281534g13ee19f1i9c955cbb55db23aa@mail.gmail.com> <9e804ac0603281536r56c88cfx2e1c5704e57d42e5@mail.gmail.com> Message-ID: <9e804ac0603281544k64f3e7f5labdb0bcea304516d@mail.gmail.com> On 3/29/06, Thomas Wouters wrote: > > So does that make all cycles involving only objects with finalizers > impervious to cycle-gc? I guess it'd have to be that way. Er, I meant to say 'does that make all cycles involving just one object with a finalizer impervious to cycle-gc'. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060329/4b1105af/attachment.htm From barry at python.org Wed Mar 29 01:57:07 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 28 Mar 2006 18:57:07 -0500 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <5.1.1.6.0.20060328181905.01f92a68@mail.telecommunity.com> References: <1143578916.10793.143.camel@resist.wooz.org> <5.1.1.6.0.20060328181905.01f92a68@mail.telecommunity.com> Message-ID: <1143590227.10799.151.camel@resist.wooz.org> On Tue, 2006-03-28 at 18:24 -0500, Phillip J. Eby wrote: > On the plus side, it sounds like ASPW is a more general wrapping of SQLite, > which seems to me to lean in its favor for the stdlib, if it can also be > brought into DBAPI compliance. If there's some general uncertainty about which to add, then it might make better sense to wait and not add either for Python 2.5. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060328/56eb7dcd/attachment.pgp From tim.peters at gmail.com Wed Mar 29 02:13:58 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 28 Mar 2006 19:13:58 -0500 Subject: [Python-Dev] [Python-checkins] r43358 - python/trunk/Modules/itertoolsmodule.c In-Reply-To: <9e804ac0603281544k64f3e7f5labdb0bcea304516d@mail.gmail.com> References: <20060327210214.2E8991E4004@bag.python.org> <007e01c651e3$1d863ac0$d200000a@RaymondLaptop1> <9e804ac0603271321x2aa99f58vf6013dafd0ca9a0c@mail.gmail.com> <9e804ac0603280102l4e6b42eena5619efc917aaed7@mail.gmail.com> <1f7befae0603281513r415b46g7996e218642b2c8b@mail.gmail.com> <9e804ac0603281534g13ee19f1i9c955cbb55db23aa@mail.gmail.com> <9e804ac0603281536r56c88cfx2e1c5704e57d42e5@mail.gmail.com> <9e804ac0603281544k64f3e7f5labdb0bcea304516d@mail.gmail.com> Message-ID: <1f7befae0603281613r6cb6fe19ld80283ef5111f35d@mail.gmail.com> [Thomas Wouters] >> So does that make all cycles involving only objects with finalizers >> impervious to cycle-gc? I guess it'd have to be that way. [again] > Er, I meant to say 'does that make all cycles involving just one object with > a finalizer impervious to cycle-gc'. Both are true, and both are implied by this sharper variant: all cycles containing at least one object with a finalizer [etc] From janssen at parc.com Wed Mar 29 02:21:58 2006 From: janssen at parc.com (Bill Janssen) Date: Tue, 28 Mar 2006 16:21:58 PST Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: Your message of "Tue, 28 Mar 2006 12:21:01 PST." <20060328202101.GA8345@discworld.dyndns.org> Message-ID: <06Mar28.162205pst."58633"@synergy1.parc.xerox.com> > Yup; I'd be happy to see db.mysql and db.pgsql or whatnot added as > appropriate, and having a nice new namespace ready for them is a good idea. I really wish that that namespace would be "database", not "db". Bill From tim.peters at gmail.com Wed Mar 29 02:26:27 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 28 Mar 2006 19:26:27 -0500 Subject: [Python-Dev] Fwd: [Python-checkins] r43358 - python/trunk/Modules/itertoolsmodule.c In-Reply-To: <9e804ac0603281536r56c88cfx2e1c5704e57d42e5@mail.gmail.com> References: <20060327210214.2E8991E4004@bag.python.org> <007e01c651e3$1d863ac0$d200000a@RaymondLaptop1> <9e804ac0603271321x2aa99f58vf6013dafd0ca9a0c@mail.gmail.com> <9e804ac0603280102l4e6b42eena5619efc917aaed7@mail.gmail.com> <1f7befae0603281513r415b46g7996e218642b2c8b@mail.gmail.com> <9e804ac0603281534g13ee19f1i9c955cbb55db23aa@mail.gmail.com> <9e804ac0603281536r56c88cfx2e1c5704e57d42e5@mail.gmail.com> Message-ID: <1f7befae0603281626x37805db9ya62fcfa65fcd8be6@mail.gmail.com> [Thomas Wouters] ... > not to mention avoiding finalizers on other builtin objects. (None seem to have > them at the moment, they all do their stuff in tp_dealloc, Right, generators are the only exception now (and there were no exceptions before 2.5). > which I guess doesn't allow objects to reincarnate themselves.) This is the actual rule: tp_dealloc must not do anything that could confuse cyclic gc <0.5 wink>. I'm afraid that's the best writeup there is. If you have a few spare weeks, try to figure out exactly why subtype_dealloc() does what it does ;-) > ... > I'll see about fixing the Python code to avoid or explicitly break the > cycles. Or giving gc more smarts about generators specifically (see Phillip's msg). > Maybe we can patch regrtest.py to take into account uncollectable > cycles, so that it could report them separately from refleaks. I'm going to > think about all this first, though. > > Hrrm-hrrm'ly y'rs, > -- > Hi! I'm a finalizer virus! copy me into your tp_del slot to help me spread! LOL! Nice variation. From tim.peters at gmail.com Wed Mar 29 02:37:20 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 28 Mar 2006 19:37:20 -0500 Subject: [Python-Dev] [Python-checkins] r43358 - python/trunk/Modules/itertoolsmodule.c In-Reply-To: <5.1.1.6.0.20060328182758.01fb58d0@mail.telecommunity.com> References: <20060327210214.2E8991E4004@bag.python.org> <007e01c651e3$1d863ac0$d200000a@RaymondLaptop1> <9e804ac0603271321x2aa99f58vf6013dafd0ca9a0c@mail.gmail.com> <9e804ac0603280102l4e6b42eena5619efc917aaed7@mail.gmail.com> <1f7befae0603281513r415b46g7996e218642b2c8b@mail.gmail.com> <5.1.1.6.0.20060328182758.01fb58d0@mail.telecommunity.com> Message-ID: <1f7befae0603281637t6e60a973w3297ea6984f3c01d@mail.gmail.com> [Phillip J. Eby] > By that reasoning, binary compatibility won't be an issue anywhere else, > either, since the change was made on the 2.5 alpha trunk, and ISTM that 2.5 > will require recompiling extensions anyway. I don't know how people work on Linux; that's why I brought it up. The binary API version changed to 1013 on the trunk (see modsupport.h), but that's only "advisory" (it produces a message in case of mismatch but does not stop the mismatched module from loading). For example, I know that Guido has been known not to bother recompiling when an API mismatch warning is given on Linux. > Now, the trick could potentially be made a bit smarter if there were a slot > by which gcmodule could ask the object for a finalizer *dynamically*. A > generator without an active frame (or an active frame with no "try" blocks > on the frame's block stack), has no need to run Python code for > finalization. Calling tp_clear on such a generator will do anything that > the actual deletion would, only faster. :) > > So, that approach could be used to get rid of most new leaks caused by > generator cycles, at the expense of a bit more complexity in the gc and in > generators. It won't fix leaks caused by cycles in active generators that > have active try/finally or try/except blocks, since these are the things > that actually need finalizing. Simpler: forget generalizing this (YAGNI). Restore the previous code, but add a new if-test specific to generators. Then it will be bulletproof wrt accessing tp_del, and the generator-specific branch can be as fast as possible since it _knows_ it's dealing with a generator. Generalize it when and only when this bad idea spreads to other builtin types :-) From greg.ewing at canterbury.ac.nz Wed Mar 29 02:38:55 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 29 Mar 2006 12:38:55 +1200 Subject: [Python-Dev] INPLACE_ADD and INPLACE_MULTIPLY oddities in ceval.c In-Reply-To: References: <4428C77E.5020005@canterbury.ac.nz> Message-ID: <4429D71F.1010601@canterbury.ac.nz> Travis E. Oliphant wrote: > I can't imaging anybody relying on an in-place operations to return a > "different" object, but we could make the change and run all the > NumPy/SciPy tests to see what happens. I'm really thinking more about the non-inplace operators. If nb_add and sq_concat are collapsed into a single slot, it seems to me that if you do a = [1, 2, 3] b = array([4, 5, 6]) c = a + b then a will be asked "Please add yourself to b", and a will say "Okay, I know how to do that!" and promptly concatenate itself with b. This would be very different from the current behaviour of Numeric arrays. I don't know whether Numeric users would consider it a serious problem or not, but I think we need to consider the implications before charging ahead too fast with slot unification. -- Greg From jcarlson at uci.edu Wed Mar 29 03:23:21 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 28 Mar 2006 17:23:21 -0800 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <1143590227.10799.151.camel@resist.wooz.org> References: <5.1.1.6.0.20060328181905.01f92a68@mail.telecommunity.com> <1143590227.10799.151.camel@resist.wooz.org> Message-ID: <20060328171919.05C8.JCARLSON@uci.edu> Barry Warsaw wrote: > On Tue, 2006-03-28 at 18:24 -0500, Phillip J. Eby wrote: > > > On the plus side, it sounds like ASPW is a more general wrapping of SQLite, > > which seems to me to lean in its favor for the stdlib, if it can also be > > brought into DBAPI compliance. > > If there's some general uncertainty about which to add, then it might > make better sense to wait and not add either for Python 2.5. I'm not sure that there is uncertainty about *which* to add. So far we've seen a handful of +1s for pysqlite, but a link and short discussion about aspw. In my opinion, because pysqlite already has a DBAPI compliant interface to sqlite, and of us who have used it have had positive experiences, I don't see how aspw is even competition, let alone influential enough to push the adoption of an embedded SQL server to Python 2.6 or later. - Josiah From greg.ewing at canterbury.ac.nz Wed Mar 29 04:38:41 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 29 Mar 2006 14:38:41 +1200 Subject: [Python-Dev] PySet API In-Reply-To: <1143578215.10793.138.camel@resist.wooz.org> References: <005f01c6523c$a2f5f020$1e2810ac@RaymondLaptop1> <1143578215.10793.138.camel@resist.wooz.org> Message-ID: <4429F331.9090700@canterbury.ac.nz> Barry Warsaw wrote: > "Perhaps the PySet API can raise an error if it's ever called on > something that's not *exactly* a set? No subclassing allowed. > Shouldn't affect you, and should be an effective deterrent against > abuse of the kind that made the PyDict API an albatross." And perhaps in Py3k the same could be done for dicts, and any other builtin types with concrete access functions? -- Greg From greg.ewing at canterbury.ac.nz Wed Mar 29 04:51:13 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 29 Mar 2006 14:51:13 +1200 Subject: [Python-Dev] INPLACE_ADD and INPLACE_MULTIPLY oddities in ceval.c In-Reply-To: <20060328112856.GA21294@code0.codespeak.net> References: <20060328112856.GA21294@code0.codespeak.net> Message-ID: <4429F621.20303@canterbury.ac.nz> Armin Rigo wrote: > >>> [].__add__(5) > TypeError: can only concatenate list (not "int") to list Would that be much of a loss? It doesn't really give you much more information than something like Unsupported operand types for '+': list, int and what it does give is based on the assumption that concatenation is what the user has in mind. He might just as easily have been thinking of addition, or something else entirely. -- Greg From greg.ewing at canterbury.ac.nz Wed Mar 29 04:52:20 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 29 Mar 2006 14:52:20 +1200 Subject: [Python-Dev] PySet API In-Reply-To: <200603281507.18175.gmccaughan@synaptics-uk.com> References: <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> <1143468446.10799.50.camel@resist.wooz.org> <200603281507.18175.gmccaughan@synaptics-uk.com> Message-ID: <4429F664.3080706@canterbury.ac.nz> Gareth McCaughan wrote: > For what it's worth[1], I think Raymond is absolutely on crack here. +1 on a good concrete set API from me, too. Being such similar types, sets should have about the same API richness as dicts, IMO. -- Greg From greg.ewing at canterbury.ac.nz Wed Mar 29 04:55:32 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 29 Mar 2006 14:55:32 +1200 Subject: [Python-Dev] Class decorators In-Reply-To: <5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com> References: <44297538.8060002@strakt.com> <44268B6B.9070305@canterbury.ac.nz> <44297538.8060002@strakt.com> <5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com> Message-ID: <4429F724.4060202@canterbury.ac.nz> Phillip J. Eby wrote: > http://mail.python.org/pipermail/python-dev/2004-March/043462.html > Or more precisely, the subsequent discussion and examples convinced me that > putting class decorators on top of the class was bad for readability, vs. > putting them in the body just after the docstring. I just looked at that discussion, and I didn't see any argument to refute the thesis that if decorators-on-top is good enough for functions, it should be good enough for classes. In fact it seems to have been written before the on-top decorator syntax existed, so there's no discussion of the issue at all. Seems to me that, since we now already have @-decorators for functions, a *very* good reason will be needed for requiring a completely different syntax to get exactly the same effect for classes. -- Greg From pje at telecommunity.com Wed Mar 29 05:06:05 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 28 Mar 2006 22:06:05 -0500 Subject: [Python-Dev] Class decorators In-Reply-To: <4429F724.4060202@canterbury.ac.nz> References: <5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com> <44297538.8060002@strakt.com> <44268B6B.9070305@canterbury.ac.nz> <44297538.8060002@strakt.com> <5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> At 02:55 PM 3/29/2006 +1200, Greg Ewing wrote: >Phillip J. Eby wrote: > > > http://mail.python.org/pipermail/python-dev/2004-March/043462.html > > > Or more precisely, the subsequent discussion and examples convinced me > that > > putting class decorators on top of the class was bad for readability, vs. > > putting them in the body just after the docstring. > >I just looked at that discussion, and I didn't see any >argument to refute the thesis that if decorators-on-top >is good enough for functions, it should be good enough >for classes. > >In fact it seems to have been written before the on-top >decorator syntax existed, so there's no discussion of >the issue at all. > >Seems to me that, since we now already have @-decorators >for functions, a *very* good reason will be needed for >requiring a completely different syntax to get exactly >the same effect for classes. And here it is: because the use cases for class decorators are different. I routinely use them with things that take numerous keyword arguments, but this isn't nearly as common of a scenario for function decorators. Also, class decorators are far more likely to be just registering the class with something -- which means they don't deserve so prominent a location as to obscure the class itself. ObDisclaimer: this is my personal experience and opinion. Others may have different use cases in mind. I'm just pointing out that if @decorator support were added for classes, I wouldn't use it, because it's not actually an improvement over what I'm doing now. Function decorators, OTOH, *are* an improvement over what I did before, so I use them. From fdrake at acm.org Wed Mar 29 06:35:09 2006 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Tue, 28 Mar 2006 23:35:09 -0500 Subject: [Python-Dev] Class decorators In-Reply-To: <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> References: <5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com> <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> Message-ID: <200603282335.09341.fdrake@acm.org> On Tuesday 28 March 2006 22:06, Phillip J. Eby wrote: > And here it is: because the use cases for class decorators are > different. This is vague. > I routinely use them with things that take numerous keyword > arguments, but this isn't nearly as common of a scenario for function > decorators. The zope.formlib decorators are commonly used with many arguments; those construct the replacements and register with a class-specific registry. > Also, class decorators are far more likely to be just > registering the class with something -- which means they don't deserve so > prominent a location as to obscure the class itself. I've not looked at the Java and C# examples, so I don't claim anything about those examples. For all the cases where I'm registering classes, however, it's not a local registry, but something that lives elsewhere in the system; it doesn't affect the class itself at all. I'm happy for that use case to be supported in other ways than prefixing the class with decorator-syntax stuff. > ObDisclaimer: this is my personal experience and opinion. Others may have > different use cases in mind. I'm just pointing out that if @decorator > support were added for classes, I wouldn't use it, because it's not > actually an improvement over what I'm doing now. So it doesn't apply. I suspect this is something for which familiarity with the use cases for the Java and C# precedents would help. For Zope 3, we have decorators that work with the component architecture (I'm sure Phillip is familiar with these). They're used with functions to indicate that the function adapts a particular kind of object, or that it implements or provides a particular interface. We have different functions that get used for this purpose in classes that are executed within the body of the class. There's some merit to being able to use a single set of functions in both cases, since the use cases are the same. I'm not sure I'd want to change the existing pattern, though, since it's already so widespread within the Zope 3 codebase (including 3rd-party components). -Fred -- Fred L. Drake, Jr. From kbk at shore.net Wed Mar 29 06:59:16 2006 From: kbk at shore.net (Kurt B. Kaiser) Date: Tue, 28 Mar 2006 23:59:16 -0500 (EST) Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200603290459.k2T4xG9n011371@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 389 open ( -3) / 3117 closed (+23) / 3506 total (+20) Bugs : 901 open ( -6) / 5687 closed (+41) / 6588 total (+35) RFE : 214 open ( +1) / 206 closed ( +4) / 420 total ( +5) New / Reopened Patches ______________________ Minimalist fix for bug 1446847 (2006-03-17) CLOSED http://python.org/sf/1452332 opened by Nick Coghlan N-d array interface for array object (2006-03-17) http://python.org/sf/1452906 opened by Alexander Belopolsky IDNA codec simplification (2006-03-18) http://python.org/sf/1453235 opened by Walter D?rwald popen2 new parameter and setpgid (2006-03-20) CLOSED http://python.org/sf/1454425 opened by Oskar Ander? 2 Tools for easy inter-thread communication->Queue,threading (2006-03-20) http://python.org/sf/1454452 opened by kxroberto Make thread stack size runtime tunable (2006-03-20) http://python.org/sf/1454481 opened by Andrew I MacIntyre patch for SIGSEGV in arraymodule.c (2006-03-20) http://python.org/sf/1454485 reopened by tbmetin patch for SIGSEGV in arraymodule.c (2006-03-20) http://python.org/sf/1454485 opened by Baris Metin Use dlopen() to load extensions on Darwin, where possible (2006-03-20) http://python.org/sf/1454844 opened by Zach Pincus patch for mbcs codecs (2006-03-22) http://python.org/sf/1455898 opened by ocean-city timestamp() method for datetime objects (2006-03-23) CLOSED http://python.org/sf/1457227 opened by Chris AtLee Support different GPG keys in upload --sign (2006-03-23) http://python.org/sf/1457316 opened by Stefan Behnel patch for building trunk with VC6 (2006-03-24) http://python.org/sf/1457736 opened by ocean-city floor division (2006-03-25) CLOSED http://python.org/sf/1458419 opened by Andy Improved PySet C API (2006-03-25) CLOSED http://python.org/sf/1458476 opened by Barry A. Warsaw Mutable Iterators PEP (2006-03-26) http://python.org/sf/1459011 opened by Adam DePrince Install PKG-INFO with packages (2006-03-27) http://python.org/sf/1459476 opened by Phillip J. Eby Add docs for zlib.decompressobj.flush optional param (2006-03-27) CLOSED http://python.org/sf/1459630 opened by Chris AtLee Update docs for zlib.decompressobj.flush() (2006-03-27) http://python.org/sf/1459631 opened by Chris AtLee convenient Message.as_string to use mangle_from_=unixfrom ? (2006-03-28) http://python.org/sf/1459867 opened by kxroberto Patches Closed ______________ PEP 338 implementation (2006-02-11) http://python.org/sf/1429601 closed by ncoghlan patch for 1441408 compiler fails to spot extended slice (2006-03-10) http://python.org/sf/1446847 closed by ncoghlan Minimalist fix for bug 1441408 (2006-03-17) http://python.org/sf/1452332 closed by ncoghlan PEP 338 documentation (2006-02-11) http://python.org/sf/1429605 closed by ncoghlan error checking after PyXXX_New (2006-03-08) http://python.org/sf/1445505 closed by gbrandl audioop - alaw encoding/decoding added, code updated (2005-07-02) http://python.org/sf/1231053 closed by anthonybaxter Method for cell objects to access contents (2005-03-25) http://python.org/sf/1170323 closed by gbrandl Incremental codecs for CJKCodecs (2006-03-05) http://python.org/sf/1443155 closed by perky A wait4() implementation (2005-09-30) http://python.org/sf/1309579 closed by nnorwitz Make itertools.tee participate in GC (2006-03-06) http://python.org/sf/1444398 closed by twouters more leaky, leaky (2006-03-07) http://python.org/sf/1445431 closed by nnorwitz Some fixes for the binary distribution builder (2005-12-14) http://python.org/sf/1380777 closed by ronaldoussoren popen2 new parameter and setpgid (2006-03-20) http://python.org/sf/1454425 closed by oskar_andero CALL_ATTR opcode (2003-03-26) http://python.org/sf/709744 closed by twouters FreeBSD is system scope threads capable (2006-01-04) http://python.org/sf/1396919 closed by perky timestamp() method for datetime objects (2006-03-23) http://python.org/sf/1457227 closed by gvanrossum Add method to function objects to simplify decoration (2005-03-12) http://python.org/sf/1161819 closed by ncoghlan Simplify using Queues with consumer threads (2006-03-21) http://python.org/sf/1455676 closed by rhettinger floor division (2006-03-25) http://python.org/sf/1458419 closed by nnorwitz Improved PySet C API (2006-03-25) http://python.org/sf/1458476 closed by gvanrossum Add docs for zlib.decompressobj.flush optional param (2006-03-27) http://python.org/sf/1459630 closed by gbrandl New / Reopened Bugs ___________________ xmlrpclib.py problem solved (2006-03-17) CLOSED http://python.org/sf/1452174 opened by varun bhansaly htmllib doesn't properly substitute entities (2006-03-17) http://python.org/sf/1452246 opened by Helmut Grohne broken string on mbcs (2006-03-18) CLOSED http://python.org/sf/1452697 opened by ocean-city Defining a class with __dict__ brakes attributes assignment (2006-03-11) CLOSED http://python.org/sf/1448042 reopened by gbrandl Unexpected module reloading (2006-03-18) CLOSED http://python.org/sf/1453145 opened by Ali Gholami Rudi try-except-finally in the tutorial (2006-03-19) CLOSED http://python.org/sf/1453570 opened by George Yoshida conditional expression documentation missing/incorrect (2006-03-19) CLOSED http://python.org/sf/1453579 opened by Alan makesetup fails to tolerate valid linker options (2006-03-20) http://python.org/sf/1454227 opened by Jeff Blaine test_parsedate_acceptable_to_time_functions+DST = :-( (2006-03-20) http://python.org/sf/1454285 opened by Anthony Baxter Explanation of pow() in lib (2006-03-20) CLOSED http://python.org/sf/1454855 opened by Christoph Zwerschke import email fails (2006-03-20) CLOSED http://python.org/sf/1454912 opened by Sjoerd Mullender Incompatible size for long (int) in Python Win and Linux? (2006-03-21) CLOSED http://python.org/sf/1455357 opened by Michael Yanowitz pyport.h freebsd inconsistent (2006-03-21) CLOSED http://python.org/sf/1455641 opened by Jim Jewett Simplify use of Queues with consumer threads (2006-03-21) CLOSED http://python.org/sf/1455676 opened by Raymond Hettinger dictobject.c:dictresize() vulnerability (2006-03-22) http://python.org/sf/1456209 opened by Armin Rigo Traceback error when compiling Regex (2006-03-22) http://python.org/sf/1456280 opened by Wolfgang Grafen sliceobject ssize_t (and index) not completed (2006-03-22) http://python.org/sf/1456470 opened by Jim Jewett PEP Links broken on new website (2006-03-22) CLOSED http://python.org/sf/1456609 opened by Grant Olson round not rounding correctly (2006-03-23) CLOSED http://python.org/sf/1456775 opened by Norbert Sebok Unifying pickle and cPickle exception class hierarchies (2006-03-23) http://python.org/sf/1457119 opened by Ori Peleg urllib.splithost parses incorrectly (2006-03-23) CLOSED http://python.org/sf/1457264 opened by Steve AttributeError in upload_file on interrupted connection (2006-03-23) http://python.org/sf/1457312 opened by Stefan Behnel floor division not documented (2006-03-23) CLOSED http://python.org/sf/1457358 opened by Christoph Zwerschke byext.py errors (2006-03-23) CLOSED http://python.org/sf/1457411 opened by Jeff Lowery regs attribute on regex objects not documented (2006-03-24) CLOSED http://python.org/sf/1457470 opened by Jeff Lowery Malloc error on MacOSX/imaplib (2006-03-24) http://python.org/sf/1457783 opened by Andreas Jung cgi.FormContentDict constructor should support parse options (2006-03-25) http://python.org/sf/1457823 opened by John Belmonte Log._log needs to be more forgiving... (2006-03-24) http://python.org/sf/1458017 opened by Skip Montanaro sys.maxint should be mentioned in explanation of int type (2006-03-25) CLOSED http://python.org/sf/1458220 opened by Christoph Zwerschke lambda with a parenthesized argument raises TypeError (2006-03-27) CLOSED http://python.org/sf/1458903 opened by Jeong-Min Lee -Q warn option is still not default in 2.4 (2006-03-26) CLOSED http://python.org/sf/1458927 opened by Christoph Zwerschke Doubled backslash in repr() method for unicode (2006-03-27) CLOSED http://python.org/sf/1459029 opened by Christoph Zwerschke missing links beetween strftime()'s... (2006-03-27) CLOSED http://python.org/sf/1459103 opened by Jens Diemer inspect.getargspec() is wrong for def foo((x)): (2006-03-27) http://python.org/sf/1459159 opened by Neal Norwitz sgmllib.SGMLparser and hexadecimal numeric character refs (2006-03-27) http://python.org/sf/1459279 opened by Francesco Ricciardi time.timezone / time.strftime('%z') problem (2006-03-27) CLOSED http://python.org/sf/1459642 opened by Simon Lanzmich sets.Set can't be subclassed (2006-03-28) CLOSED http://python.org/sf/1459733 opened by Michal Kwiatkowski test suite should pass with -Qnew ? (2006-03-28) CLOSED http://python.org/sf/1459808 opened by Anthony Baxter urllib2 changes header names (2006-03-28) CLOSED http://python.org/sf/1459963 opened by Michael Vert random.sample can raise KeyError (2006-03-29) http://python.org/sf/1460340 opened by paul rubin Bugs Closed ___________ make frameworkinstall fails on Intel Macs (2006-03-10) http://python.org/sf/1447607 closed by gbrandl xmlrpclib.py problem solved (2006-03-17) http://python.org/sf/1452174 deleted by gbrandl bug with xmlrpclib (2006-03-09) http://python.org/sf/1446690 closed by gbrandl asyncore dispatcher.__getattr__() masks self._map (2006-03-13) http://python.org/sf/1448639 closed by gbrandl svn head compiler fails to spot extended slicing (2006-03-02) http://python.org/sf/1441408 closed by ncoghlan (XMLRPC) multitude of sockets ending up in TIME_WAIT (2005-02-25) http://python.org/sf/1151968 closed by gbrandl README build instructions for fpectl (2004-01-07) http://python.org/sf/872175 closed by gbrandl Built-in compile function with PEP 0263 encoding bug (2005-02-03) http://python.org/sf/1115379 closed by nnorwitz broken string on mbcs (2006-03-18) http://python.org/sf/1452697 deleted by ocean-city Defining a class with __dict__ brakes attributes assignment (2006-03-11) http://python.org/sf/1448042 closed by gbrandl Python drops core when stdin is bogus (2005-11-10) http://python.org/sf/1353504 closed by gbrandl Inconsistency between StringIO and cStringIO (2006-01-27) http://python.org/sf/1416477 closed by gbrandl optparse docs double-dash confusion (2006-02-16) http://python.org/sf/1432838 closed by gbrandl Http redirection error in urllib2.py (2005-11-10) http://python.org/sf/1353433 closed by gbrandl Unexpected module reloading (2006-03-18) http://python.org/sf/1453145 closed by loewis try-except-finally in the tutorial (2006-03-19) http://python.org/sf/1453570 closed by gbrandl conditional expression documentation missing/incorrect (2006-03-19) http://python.org/sf/1453579 closed by gbrandl shelve/bsddb crash on db close (2005-06-26) http://python.org/sf/1227955 closed by nnorwitz Explanation of pow() in lib (2006-03-20) http://python.org/sf/1454855 closed by rhettinger import email fails (2006-03-20) http://python.org/sf/1454912 closed by bwarsaw Incompatible size for long (int) in Python Win and Linux? (2006-03-21) http://python.org/sf/1455357 closed by perky pyport.h freebsd inconsistent (2006-03-22) http://python.org/sf/1455641 closed by perky problems with too many sockets in FreeBSD (2006-03-11) http://python.org/sf/1448058 closed by nnorwitz PEP Links broken on new website (2006-03-23) http://python.org/sf/1456609 closed by gbrandl Pointer freed twice in Py_InitializeEx() (03/07/06) http://python.org/sf/1444893 closed by sf-robot Popen4 wait() fails sporadically with threads (2005-04-15) http://python.org/sf/1183780 closed by loewis Python 2.2.3 crashes using popen2 to spawn lots of children (2004-01-28) http://python.org/sf/886492 closed by nnorwitz round not rounding correctly (2006-03-23) http://python.org/sf/1456775 closed by gbrandl urllib.splithost parses incorrectly (2006-03-23) http://python.org/sf/1457264 closed by gbrandl floor division not documented (2006-03-23) http://python.org/sf/1457358 closed by nnorwitz byext.py errors (2006-03-23) http://python.org/sf/1457411 closed by nnorwitz regs attribute on regex objects not documented (2006-03-24) http://python.org/sf/1457470 closed by gbrandl build doesn't pick up bsddb w/Mandrake 9.2 (2004-09-23) http://python.org/sf/1033390 closed by nnorwitz sys.maxint should be mentioned in explanation of int type (2006-03-25) http://python.org/sf/1458220 closed by gbrandl lambda with a parenthesized argument raises TypeError (2006-03-26) http://python.org/sf/1458903 closed by nnorwitz -Q warn option is still not default in 2.4 (2006-03-26) http://python.org/sf/1458927 closed by gbrandl Doubled backslash in repr() method for unicode (2006-03-27) http://python.org/sf/1459029 closed by anthonybaxter missing links beetween strftime()'s... (2006-03-27) http://python.org/sf/1459103 closed by gbrandl time.timezone / time.strftime('%z') problem (2006-03-27) http://python.org/sf/1459642 closed by gbrandl sets.Set can't be subclassed (2006-03-28) http://python.org/sf/1459733 closed by gbrandl test suite should pass with -Qnew ? (2006-03-28) http://python.org/sf/1459808 closed by gbrandl urllib2 changes header names (2006-03-28) http://python.org/sf/1459963 closed by gbrandl New / Reopened RFE __________________ sys.setatomicexecution - for critical sections (2006-03-18) http://python.org/sf/1453341 opened by kxroberto addheaders for urlopen / open / xxxx_open (2006-03-19) http://python.org/sf/1453973 opened by kxroberto Make more out of 'as' : general ad-hoc right-assignment (2006-03-20) CLOSED http://python.org/sf/1454344 opened by kxroberto Python space syntax & pythonic long lambda's (2006-03-20) CLOSED http://python.org/sf/1454364 opened by kxroberto RFE Closed __________ Missing 4 socket object properties (2002-06-12) http://python.org/sf/567972 closed by gbrandl Constructor for counting things (2003-05-17) http://python.org/sf/739029 closed by gbrandl Make more out of 'as' : general ad-hoc right-assignment (2006-03-20) http://python.org/sf/1454344 closed by gbrandl Python space syntax & pythonic long lambda's (2006-03-20) http://python.org/sf/1454364 closed by gbrandl From pje at telecommunity.com Wed Mar 29 07:01:55 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 29 Mar 2006 00:01:55 -0500 Subject: [Python-Dev] Class decorators In-Reply-To: <200603282335.09341.fdrake@acm.org> References: <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> <5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com> <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> At 11:35 PM 3/28/2006 -0500, Fred L. Drake, Jr. wrote: >For Zope 3, we have decorators that work with the component architecture (I'm >sure Phillip is familiar with these). They're used with functions to >indicate that the function adapts a particular kind of object, or that it >implements or provides a particular interface. We have different functions >that get used for this purpose in classes that are executed within the body >of the class. There's some merit to being able to use a single set of >functions in both cases, since the use cases are the same. I'm not sure I'd >want to change the existing pattern, though, since it's already so widespread >within the Zope 3 codebase (including 3rd-party components). If we're using Zope 3 as an example, I personally find that: class Foo: """Docstring here, blah blah blah """ implements(IFoo) is easier to read than: @implements(IFoo) class Foo: """Docstring here, blah blah blah """ And it only gets worse when you have lots of arguments or multiple decorators. For some reason, this doesn't bother me with functions. But then, I can't remember how often I've actually needed to use two decorators on the same function, or how many times a function decorator's arguments took multiple lines to list. Both of these are routine occurrences for my class use cases. It's too bad this syntax is ambiguous: class Foo: """Docstring here, blah blah blah """ @implements(IFoo) As this achieves a desirable highlighting of the specialness, without forcing the decorator outside the class. Oh well. From fdrake at acm.org Wed Mar 29 07:48:52 2006 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Wed, 29 Mar 2006 00:48:52 -0500 Subject: [Python-Dev] Class decorators In-Reply-To: <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> References: <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> Message-ID: <200603290048.52669.fdrake@acm.org> On Wednesday 29 March 2006 00:01, Phillip J. Eby wrote: > If we're using Zope 3 as an example, I personally find that: > > class Foo: > """Docstring here, blah blah blah > """ > implements(IFoo) > > is easier to read than: I think the existing usage for classes is perfectly readable. The @-syntax works well for functions as well. > For some reason, this doesn't bother me with functions. But then, I can't > remember how often I've actually needed to use two decorators on the same > function, or how many times a function decorator's arguments took multiple > lines to list. For zope.formlib actions, I find there's usually only one decorator. Sometimes it fits comfortably on one line, and sometimes it takes two or three. For component architecture decorators, we find we commonly use two (zope.interface.implementer and zope.component.adapter) in tandem. This can be fairly verbose with multi-object adaptation, or really long package names. > It's too bad this syntax is ambiguous: > > class Foo: > """Docstring here, blah blah blah > """ > @implements(IFoo) > > As this achieves a desirable highlighting of the specialness, without > forcing the decorator outside the class. Oh well. Agreed, but... guess we can't have everything. On the other hand, something like: class Foo: """Documentation is good.""" @class implements(IFoo) is not ambiguous. Hmm. It even says what it means. :-) -Fred -- Fred L. Drake, Jr. From fdrake at acm.org Wed Mar 29 08:11:06 2006 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Wed, 29 Mar 2006 01:11:06 -0500 Subject: [Python-Dev] Class decorators In-Reply-To: <200603290048.52669.fdrake@acm.org> References: <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <200603290048.52669.fdrake@acm.org> Message-ID: <200603290111.06247.fdrake@acm.org> On Wednesday 29 March 2006 00:48, Fred L. Drake, Jr. wrote: > I think the existing usage for classes is perfectly readable. The > @-syntax works well for functions as well. On re-reading what I wrote, I don't think I actually clarified the point I was trying to make originally. My point wasn't that I desparately need @-syntax for class decorators (I don't), or see it as inherantly superior in some way. It's much simpler than that: I just want to be able to use the same syntax for a group of use cases regardless of whether the target is a function or a class. This fits into the nice-to-have category for me, since the use case can be the same regardless of whether I'm decorating a class or a function. (I will note that when this use case applies to a function, it's usually a module-level function I'm decorating rather than a method.) My other example, the zope.formlib example, has only ever involved a single decorator, and is always a method. It could conceivably be applied to a nested class without much of a stretch, however. -Fred -- Fred L. Drake, Jr. From tjreedy at udel.edu Wed Mar 29 08:48:43 2006 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 29 Mar 2006 01:48:43 -0500 Subject: [Python-Dev] PySet API References: <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1><1143468446.10799.50.camel@resist.wooz.org><200603281507.18175.gmccaughan@synaptics-uk.com> <4429F664.3080706@canterbury.ac.nz> Message-ID: "Greg Ewing" wrote in message news:4429F664.3080706 at canterbury.ac.nz... > Gareth McCaughan wrote: > >> For what it's worth[1], I think Raymond is absolutely on crack here. > > +1 on a good concrete set API from me, too. For what it's worth, I think Gareth's crack at Raymond is childish and out of place here. tjr From gh at ghaering.de Wed Mar 29 09:35:22 2006 From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=) Date: Wed, 29 Mar 2006 09:35:22 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <442987D8.4010903@v.loewis.de> References: <200603281759.18756.anthony@interlink.com.au> <442987D8.4010903@v.loewis.de> Message-ID: <442A38BA.5000400@ghaering.de> Martin v. L?wis wrote: > Anthony Baxter wrote: > >>This came up before (back in October 2004!) but didn't go anywhere >>since, AFAICR. Do we want to consider including pysqlite in Python >>2.5? It's the only DB adaptor that I'd really consider suitable for >>shipping with the distribution, because it's self-contained. >> >>What's people's thoughts? > > We shouldn't include anything without the explicit permission of the > author(s); I'm the only author of the pysqlite 2 codebase, so no problem here. > if possible, the authors should grant the PSF permission > to change the license (i.e. fill out the contributor agreement). That would be no problem either. > Even better, the authors should be willing to keep the version in > Python synchronized with the separate release. In particular, I would then synchronize changes that have proven stable in the standalone release to the Python core sqlite module. I think this is how Barry does it with the email module, too. -- Gerhard From brett at python.org Wed Mar 29 09:45:55 2006 From: brett at python.org (Brett Cannon) Date: Tue, 28 Mar 2006 23:45:55 -0800 Subject: [Python-Dev] I'm not getting email from SF when assigned abug/patch In-Reply-To: <44298954.4040705@v.loewis.de> References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> <1143554576.2310.161.camel@geddy.wooz.org> <17449.21764.475497.457592@montanaro.dyndns.org> <44298954.4040705@v.loewis.de> Message-ID: On 3/28/06, "Martin v. L?wis" wrote: > skip at pobox.com wrote: > > Roundup is there now, right (sans SF export)? > > Richard Jones has an SF importer for one of the two XML-like formats, > the one that is correct XML but with incomplete data. The other format, > which has complete data but is ill-formed XML, has no importer into > roundup at the moment. > > > Trac is being used by the > > folks doing the new website. What other candidates are being considered? > > My view is that nothing should be "considered" unless there is > a) a volunteer to operate the tracker (or, failing that, somebody who > does it for money), and > b) an importer of whatever data SF can provide. > And as the chair of the infrastructure committee, those are the base requirements for a tracker to be considered. Once we have the complete SF data I will put out a call for proposals for trackers along with who has volunteered to manage them so that people can band together to help push their favorite tracker. But as of right now the committe just wants to get a clean dump of our SF data to work off of (yes, we could start a tracker from scratch but we want the SF data anyway and so we are going through the hassle of getting it off now and as a test data set for the various trackers). -Brett From nnorwitz at gmail.com Wed Mar 29 10:10:57 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 29 Mar 2006 00:10:57 -0800 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <442A38BA.5000400@ghaering.de> References: <200603281759.18756.anthony@interlink.com.au> <442987D8.4010903@v.loewis.de> <442A38BA.5000400@ghaering.de> Message-ID: On 3/28/06, Gerhard H?ring wrote: > > > Even better, the authors should be willing to keep the version in > > Python synchronized with the separate release. > > In particular, I would then synchronize changes that have proven stable > in the standalone release to the Python core sqlite module. I think this > is how Barry does it with the email module, too. Everything Gerhard has said sounds good. From what I read it seems that it might be good to add pysqlite to the stdlib eventually. Overall, I'm +0 on the idea. It seems everyone is pretty positive on the concept. However, I'm -0 on adding this to 2.5. We've already got a lot of changes. I don't want us to keep piling more on. Also I thought I saw Gerhard say that there were some other things he wanted to finish and the timing might work better for him to defer a bit. Some of these things sounded like API changes which may be more problematic once in the core as we may have stricter rules on backwards compatibility. We also have to convert the doc from ReST to latex. None of these are show stoppers, but it adds to the amount of work we need to do before release. And there's already more work than we can handle. n From gh at ghaering.de Wed Mar 29 10:44:50 2006 From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=) Date: Wed, 29 Mar 2006 10:44:50 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: References: <200603281759.18756.anthony@interlink.com.au> <442987D8.4010903@v.loewis.de> <442A38BA.5000400@ghaering.de> Message-ID: <442A4902.10506@ghaering.de> Neal Norwitz wrote: > On 3/28/06, Gerhard H?ring wrote: > >>>Even better, the authors should be willing to keep the version in >>>Python synchronized with the separate release. >> >>In particular, I would then synchronize changes that have proven stable >>in the standalone release to the Python core sqlite module. I think this >>is how Barry does it with the email module, too. > > Everything Gerhard has said sounds good. From what I read it seems > that it might be good to add pysqlite to the stdlib eventually. > Overall, I'm +0 on the idea. It seems everyone is pretty positive on > the concept. > > However, I'm -0 on adding this to 2.5. We've already got a lot of > changes. I don't want us to keep piling more on. Also I thought I > saw Gerhard say that there were some other things he wanted to finish > and the timing might work better for him to defer a bit. My current and future plans for pysqlite are really only additional features, like wrapping the rest of the SQLite API. > Some of these things sounded like API changes which may be more problematic > once in the core as we may have stricter rules on backwards > compatibility. All my plans for pysqlite are adding more methods to the API, so I see no backwards compatibility problems. > We also have to convert the doc from ReST to latex. None of these are > show stoppers, but it adds to the amount of work we need to do before > release. And there's already more work than we can handle. I understand your concern for keeping the amount of work for a 2.5 release manageable. So as Anthony Baxter said he'd like to work with me to make this happen, then I think he and I can try to not overload other people with more work. Creating latex docs sounds like I could do it, too. What I'd personally like to offload are these two tasks: - integreting pysqlite into the Python build process - in particular the win32 build process I would have access to Linux and win32 development machines with MS VS2003, but I don't have enough experience with the Python build process to not make stupid mistakes here. -- Gerhard From anthony at interlink.com.au Wed Mar 29 10:47:32 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed, 29 Mar 2006 19:47:32 +1100 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: References: <200603281759.18756.anthony@interlink.com.au> Message-ID: <200603291947.34461.anthony@interlink.com.au> On Wednesday 29 March 2006 06:33, Georg Brandl wrote: > Fredrik Lundh wrote: > > db.sqlite3 ? > > That would make sense if inclusion of more database-related modules > was planned. My only concern about this is that it wouldn't be possible for other authors to provide 3rd party packages as (for instance) db.mysqldb because of the way package importing works. And I'd prefer 'database.sqlite' rather than 'db.sqlite'. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From g.brandl at gmx.net Wed Mar 29 11:04:45 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 29 Mar 2006 11:04:45 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: References: <200603281759.18756.anthony@interlink.com.au> <442987D8.4010903@v.loewis.de> <442A38BA.5000400@ghaering.de> Message-ID: Neal Norwitz wrote: > On 3/28/06, Gerhard H?ring wrote: >> >> > Even better, the authors should be willing to keep the version in >> > Python synchronized with the separate release. >> >> In particular, I would then synchronize changes that have proven stable >> in the standalone release to the Python core sqlite module. I think this >> is how Barry does it with the email module, too. > > Everything Gerhard has said sounds good. From what I read it seems > that it might be good to add pysqlite to the stdlib eventually. > Overall, I'm +0 on the idea. It seems everyone is pretty positive on > the concept. > > However, I'm -0 on adding this to 2.5. We've already got a lot of > changes. I don't want us to keep piling more on. Also I thought I > saw Gerhard say that there were some other things he wanted to finish > and the timing might work better for him to defer a bit. Some of > these things sounded like API changes which may be more problematic > once in the core as we may have stricter rules on backwards > compatibility. > > We also have to convert the doc from ReST to latex. None of these are > show stoppers, but it adds to the amount of work we need to do before > release. And there's already more work than we can handle. FWIW, I'd volunteer to convert the doc format. Georg From fredrik at pythonware.com Wed Mar 29 11:26:01 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 29 Mar 2006 11:26:01 +0200 Subject: [Python-Dev] pysqlite for 2.5? References: <200603281759.18756.anthony@interlink.com.au> <200603291947.34461.anthony@interlink.com.au> Message-ID: Anthony Baxter wrote: > > > db.sqlite3 ? > > > > That would make sense if inclusion of more database-related modules > > was planned. > > My only concern about this is that it wouldn't be possible for other > authors to provide 3rd party packages as (for instance) db.mysqldb > because of the way package importing works. that could be addressed by a plugin facility in db/__init__.py (or by allowing installation tools to add redirection modules to the db directory...) > And I'd prefer 'database.sqlite' rather than 'db.sqlite'. and extensible_markup_language.document_object_model over xml.dom, I presume ? ;-) From arigo at tunes.org Wed Mar 29 12:45:40 2006 From: arigo at tunes.org (Armin Rigo) Date: Wed, 29 Mar 2006 12:45:40 +0200 Subject: [Python-Dev] INPLACE_ADD and INPLACE_MULTIPLY oddities in ceval.c In-Reply-To: <4429D71F.1010601@canterbury.ac.nz> References: <4428C77E.5020005@canterbury.ac.nz> <4429D71F.1010601@canterbury.ac.nz> Message-ID: <20060329104540.GA26145@code0.codespeak.net> Hi Greg, On Wed, Mar 29, 2006 at 12:38:55PM +1200, Greg Ewing wrote: > I'm really thinking more about the non-inplace operators. > If nb_add and sq_concat are collapsed into a single slot, > it seems to me that if you do > > a = [1, 2, 3] > b = array([4, 5, 6]) > c = a + b > > then a will be asked "Please add yourself to b", and a > will say "Okay, I know how to do that!" and promptly > concatenate itself with b. No: there is a difference between + and += for lists. You can only concatenate exactly a list to a list. Indeed: >>> [].__add__((2, 3)) TypeError: can only concatenate list (not "tuple") to list By contrast, list += is like extend() and accepts any iterable. So if we provide a complete fix, [].__add__(x) will be modified to return NotImplemented instead of raising TypeError if x is not a list, and then [1,2,3]+array([4,5,6]) will fall back to array.__radd__() as before. I'll try harder to see if there is a reasonable example whose behavior would change... A bientot, Armin From ncoghlan at gmail.com Wed Mar 29 13:15:21 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 29 Mar 2006 21:15:21 +1000 Subject: [Python-Dev] [Python-3000] Iterators for dict keys, values, and items == annoying :) In-Reply-To: <79990c6b0603290129m7bed22cft3e242cd009efe9e0@mail.gmail.com> References: <4422FC96.2020409@zope.com> <1143306134.3186.1.camel@localhost.localdomain> <44258376.1080709@gmx.net> <1143432484.14391.67.camel@localhost.localdomain> <79990c6b0603271117l4c69372h11362dd5d2d0ca32@mail.gmail.com> <1143565628.3305.82.camel@localhost.localdomain> <4429F6BD.60704@canterbury.ac.nz> <79990c6b0603290129m7bed22cft3e242cd009efe9e0@mail.gmail.com> Message-ID: <442A6C49.5020106@gmail.com> Paul Moore wrote: > On 3/29/06, Brett Cannon wrote: >> Without a direct reason in terms of the language needing a >> standardization of an interface, perhaps we just don't need views. If >> people want their iterator to have a __len__ method, then fine, they >> can add it without breaking anything, just realize it isn't part of >> the iterator protocol and thus may limit what objects a function can >> accept, but that is there choice. > > Good point. I think we need to start from strong use cases. With > these, I agree that the view concept is a good implementation > technique to consider. But let's not implement views just for the sake > of having them - I'm pretty sure that was never Guido's intention. There are three big use cases: dict.keys dict.values dict.items Currently these all return lists, which may be expensive in terms of copying. They all have iter* variants which while memory efficient, are far less convenient to work with. For Py3k, the intent is to have only one version which produces a view with the memory efficiency of an iterator, but the convenience of a list. To give these views the benefits of having a real list, the following is all that's really needed: 1. implement __len__ (allows bool() and len() to work) - all delegate to dict.__len__ 2. implement __contains__ (allows containment tests to work) - delegate to dict.__contains__ for dict.keys() - use (or fallback to) linear search for dict.values() - check "dict[item[0]] == item[1]" for dict.items() 3. implement __iter__ (allows iteration to work) - make iter(dict.keys()) equivalent to current dict.iterkeys() - make iter(dict.values()) equivalent to current dict.itervalues() - make iter(dict.items()) equivalent to current dict.iteritems() For an immutable view, that's all you need. IOW, take the iterable protocol (an __iter__ that returns a new iterator when invoked) and add __len__ and __contains__ to get a "container" protocol. Given that containment falls back on __iter__ anyway, __len__ is the only essential addition to turn an iterable into a container. Note that adding __len__ to an *iterator* does NOT give you something that would satisfy such a container protocol - invoking __iter__ again does not give you a fresh iterator, so you can't easily iterate repeatedly. With reiterability as a defining characteristic, other niceties become possible (potentially available as a mixin): 1. a generic container __str__ (not __repr__!) implementation: def __str__(self): # keep default __repr__ since eval(repr(x)) won't round trip name = self.__name__ guts = ", ".join(repr(x) for x in self) return "%s([%s])" % guts 2. generic container value based equality testing: def __eq__(self, other): if len(self) != len(other): return False for this, that in izip(self, other): if this != that: return False return True Further refinement of such a container protocol to the minimal requirements for a sequence protocol is already defined by such things as the requirements of the reversed() builtin: for i, x in enumerate(seq): assert seq[i] == x Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Wed Mar 29 13:19:00 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 29 Mar 2006 21:19:00 +1000 Subject: [Python-Dev] [Python-3000] Iterators for dict keys, values, and items == annoying :) In-Reply-To: <442A6C49.5020106@gmail.com> References: <4422FC96.2020409@zope.com> <1143306134.3186.1.camel@localhost.localdomain> <44258376.1080709@gmx.net> <1143432484.14391.67.camel@localhost.localdomain> <79990c6b0603271117l4c69372h11362dd5d2d0ca32@mail.gmail.com> <1143565628.3305.82.camel@localhost.localdomain> <4429F6BD.60704@canterbury.ac.nz> <79990c6b0603290129m7bed22cft3e242cd009efe9e0@mail.gmail.com> <442A6C49.5020106@gmail.com> Message-ID: <442A6D24.2090301@gmail.com> Nick Coghlan wrote: Darn, I'd hoped I'd caught that in time :( Sorry folks. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Wed Mar 29 14:17:40 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 29 Mar 2006 22:17:40 +1000 Subject: [Python-Dev] What about PEP 299? In-Reply-To: References: <20060328224400.GA9042@discworld.dyndns.org> Message-ID: <442A7AE4.2040702@gmail.com> Guido van Rossum wrote: > On 3/28/06, Charles Cazabon wrote: >> It might be worth instead adding an option flag to the executable that implies >> "from the loaded module, run __main__() with sys.argv as its argument(s)", so >> the user can get this behaviour with `python -X somemodule.py`. > > You can do "python -x somemodule" as long as somemodule.py uses the if > __name__=='__main__' convention. What does your proposal add? FWIW, you can already (Python 2.4) do something like: ------- x.py --------- import sys if __name__ == __main__: del sys.argv[0] # Get rid of reference to ourselves mod_name = sys.argv[0] # First arg is now module to be run mod = __import__(mod_name) # Run the top level module code main = mod.__main__ # Grab the main function sys.modules["__main__"] = mod # Make that module the __main__ one try: sys.argv[0] = mod.__file__ # Try to set argv[0] properly except AttributeError: pass sys.exit(main(*sys.argv)) # Run the function ------------------------- Put that in site-packages and "python -mx somemodule" will do exactly as Charles describes. Getting it to work with a filename instead of a module name would be a bit trickier, but not a lot. However, I think PEP 299 is mainly a holdover from C/C++/Java where the top level of a module is a play area for the compiler that the runtime never really gets to see. I know I found PEP 299 appealing when I first seriously started using Python, but the appeal faded over time as I got used to the idea of being able to have control logic at the top level of a module (to the point where the idea is now thoroughly *un*appealing). PEP 299's other 'use case' (trying to run another program's main function from within the current program) seems like a recipe for disaster - far better to use the subprocess module instead (since, strangely enough, application initialisation code has this tendency to assume it has sole control of the interpreter). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From python at rcn.com Wed Mar 29 08:20:01 2006 From: python at rcn.com (Raymond Hettinger) Date: Tue, 28 Mar 2006 22:20:01 -0800 Subject: [Python-Dev] PySet API References: <1142705253.32074.72.camel@geddy.wooz.org><005c01c64ac5$589859b0$b83efea9@RaymondLaptop1><002b01c64aeb$44eb7750$b83efea9@RaymondLaptop1><1142824236.10599.29.camel@resist.wooz.org><00a001c64bfa$71a0ab20$b83efea9@RaymondLaptop1><1142987079.10599.232.camel@resist.wooz.org><004001c64d58$b0ebebf0$b83efea9@RaymondLaptop1><1143327815.10799.42.camel@resist.wooz.org><001a01c65082$35a4c840$b83efea9@RaymondLaptop1><1143516346.2317.80.camel@geddy.wooz.org><4428C999.5010801@canterbury.ac.nz> <1143576767.10792.115.camel@resist.wooz.org> Message-ID: <000201c6532c$33e4b1d0$a341a8c0@RaymondLaptop1> Barry, go ahead with PySet_Clear(). Raymond From barry at python.org Wed Mar 29 14:44:42 2006 From: barry at python.org (Barry Warsaw) Date: Wed, 29 Mar 2006 07:44:42 -0500 Subject: [Python-Dev] Class decorators In-Reply-To: <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> References: <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> <5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com> <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> Message-ID: <1143636282.2310.168.camel@geddy.wooz.org> On Wed, 2006-03-29 at 00:01 -0500, Phillip J. Eby wrote: > For some reason, this doesn't bother me with functions. But then, I can't > remember how often I've actually needed to use two decorators on the same > function, or how many times a function decorator's arguments took multiple > lines to list. Both of these are routine occurrences for my class use cases. We have a couple of instances where we use multiple decorators, and even a case where one of them takes an argument. It's not too bad. I agree with your comments about class decorator readability though. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060329/6d8890a1/attachment.pgp From arigo at tunes.org Wed Mar 29 14:51:48 2006 From: arigo at tunes.org (Armin Rigo) Date: Wed, 29 Mar 2006 14:51:48 +0200 Subject: [Python-Dev] INPLACE_ADD and INPLACE_MULTIPLY oddities in ceval.c In-Reply-To: References: <20060328112856.GA21294@code0.codespeak.net> Message-ID: <20060329125148.GA32251@code0.codespeak.net> Hi all, On Tue, Mar 28, 2006 at 09:50:49AM -0800, Guido van Rossum wrote: > C extensions are my main worry -- OTOH if += for a list can already > passes arbitrary types as the argument, then any extension types > should be ready to expect this, right? Yes, I don't think C extensions are going to segfault. My worry is about returning a different result than before. Actually I believe the problem is not specific to C extensions. Here are some typical behavior changes that could be observed in pure Python already: class X(object): def __radd__(self, other): return 42 def __iter__(self): return iter("xyz") def __rmul__(self, other): return 42 def __index__(self): return 5 t = [] t += X() print t # current: 42 new: ['x', 'y', 'z'] print [1] * X() # current: 42 new: [1, 1, 1, 1, 1] Another visible difference is that the __add__/__iadd__/__mul__/__imul__ methods of lists, tuples, strings etc., will return NotImplemented instead of raising the TypeError themselves. This could impact user subclasses of these built-in types trying to override and call the super methods, not expecting a NotImplemented result (a reason why NotImplemented should have been an exception in the first place IMHO). (A different bug I found is that [1].__mul__(X()) with an __index__able class X currently raises TypeError, even though [1]*X() works just fine.) This seems to be it on the incompatibility side. I'd vote for the change anyway because the language specs -- as well as PyPy and probably all Python implementations other than CPython -- don't have this double-slot inconsistency and already show the "new" behavior. For what it's worth no CPython test breaks on top of PyPy because of this. If this change is accepted I'll submit a patch for 2.5. A bientot, Armin From gmccaughan at synaptics-uk.com Wed Mar 29 14:57:59 2006 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Wed, 29 Mar 2006 12:57:59 +0000 Subject: [Python-Dev] PySet API In-Reply-To: References: <4429F664.3080706@canterbury.ac.nz> Message-ID: <200603291357.59405.gmccaughan@synaptics-uk.com> Terry Reedy wrote: [me:] > >> For what it's worth[1], I think Raymond is absolutely on crack here. [Greg Ewing:] > > +1 on a good concrete set API from me, too. [Terry:] > For what it's worth, I think Gareth's crack at Raymond is childish and out > of place here. Er, it wasn't a crack at Raymond, it was a crack at a particular position he's taking on a particular issue. What I intended (but may have failed) to convey was: "Raymond's a clever and sensible chap, and this is a very weird position for a clever and sensible person to be taking: must be the drugs." And, just in case it's still not clear, I wasn't in fact suggesting that Raymond *is* on drugs either. However: if Raymond, or anyone else, is offended, then I'm sorry. Now, what about the technical issues, as opposed to the way I happened to introduce my comments? -- g From barry at python.org Wed Mar 29 15:13:19 2006 From: barry at python.org (Barry Warsaw) Date: Wed, 29 Mar 2006 08:13:19 -0500 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <442A38BA.5000400@ghaering.de> References: <200603281759.18756.anthony@interlink.com.au> <442987D8.4010903@v.loewis.de> <442A38BA.5000400@ghaering.de> Message-ID: <1143637999.2309.182.camel@geddy.wooz.org> On Wed, 2006-03-29 at 09:35 +0200, Gerhard H?ring wrote: > In particular, I would then synchronize changes that have proven stable > in the standalone release to the Python core sqlite module. I think this > is how Barry does it with the email module, too. I do things a little differently, at least for the maintenance releases. The email packages in the sandbox svn:external the library from the appropriate Python branch. The rest of the sandbox serves as a repository for all the chrome around releases, e.g. generated docs, setup.py, etc. For email 4.0 I did things a little different, treating it more like a dev branch until it was stable enough to merge back into the trunk (which reminds me, I have to twiddle the sandbox to svn:external it again). That works well for email, but you may want to do something differently. In general though, I think the sandbox is a great place to develop and release standalone packages that are also integrated with Python. So if you wanted to do something similar for pysqlite I'd be totally fine with that. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060329/a1586ced/attachment-0001.pgp From barry at python.org Wed Mar 29 15:17:32 2006 From: barry at python.org (Barry Warsaw) Date: Wed, 29 Mar 2006 08:17:32 -0500 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <200603291947.34461.anthony@interlink.com.au> References: <200603281759.18756.anthony@interlink.com.au> <200603291947.34461.anthony@interlink.com.au> Message-ID: <1143638252.2317.184.camel@geddy.wooz.org> On Wed, 2006-03-29 at 19:47 +1100, Anthony Baxter wrote: > My only concern about this is that it wouldn't be possible for other > authors to provide 3rd party packages as (for instance) db.mysqldb > because of the way package importing works. And I'd prefer > 'database.sqlite' rather than 'db.sqlite'. +1 on 'database' as the top-level package name. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060329/e7d5cd5b/attachment.pgp From gh at ghaering.de Wed Mar 29 15:35:48 2006 From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=) Date: Wed, 29 Mar 2006 15:35:48 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <1143638252.2317.184.camel@geddy.wooz.org> References: <200603281759.18756.anthony@interlink.com.au> <200603291947.34461.anthony@interlink.com.au> <1143638252.2317.184.camel@geddy.wooz.org> Message-ID: <442A8D34.8060200@ghaering.de> Barry Warsaw wrote: > On Wed, 2006-03-29 at 19:47 +1100, Anthony Baxter wrote: > >>My only concern about this is that it wouldn't be possible for other >>authors to provide 3rd party packages as (for instance) db.mysqldb >>because of the way package importing works. And I'd prefer >>'database.sqlite' rather than 'db.sqlite'. > > +1 on 'database' as the top-level package name. I think short names are more more consistent with the existing naming in the standard library. +1 on db.sqlite from me. db.sql.sqlite is another possibility, if adding something like Durus or ZODB in the same top-level namespace could be considered for 2.6. -- Gerhard From gjcarneiro at gmail.com Wed Mar 29 15:41:08 2006 From: gjcarneiro at gmail.com (Gustavo Carneiro) Date: Wed, 29 Mar 2006 13:41:08 +0000 Subject: [Python-Dev] Class decorators In-Reply-To: <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> References: <5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com> <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> <200603282335.09341.fdrake@acm.org> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> Message-ID: On 3/29/06, Phillip J. Eby wrote: > > At 11:35 PM 3/28/2006 -0500, Fred L. Drake, Jr. wrote: > >For Zope 3, we have decorators that work with the component architecture > (I'm > >sure Phillip is familiar with these). They're used with functions to > >indicate that the function adapts a particular kind of object, or that it > >implements or provides a particular interface. We have different > functions > >that get used for this purpose in classes that are executed within the > body > >of the class. There's some merit to being able to use a single set of > >functions in both cases, since the use cases are the same. I'm not sure > I'd > >want to change the existing pattern, though, since it's already so > widespread > >within the Zope 3 codebase (including 3rd-party components). > > If we're using Zope 3 as an example, I personally find that: > > class Foo: > """Docstring here, blah blah blah > """ > implements(IFoo) > > is easier to read than: > > @implements(IFoo) > class Foo: > """Docstring here, blah blah blah > """ Yeah, but in the first case implements(IFoo) has to do dirty hacks involving sys.getframe(), so you win in once place but lose in the other one. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060329/b8cb0c5d/attachment.html From mcherm at mcherm.com Wed Mar 29 17:22:01 2006 From: mcherm at mcherm.com (Michael Chermside) Date: Wed, 29 Mar 2006 07:22:01 -0800 Subject: [Python-Dev] pysqlite for 2.5? Message-ID: <20060329072201.xkuv6llu9wr484c8@login.werra.lunarpages.com> Gerhard H?ring writes: > db.sql.sqlite is another possibility, if adding something like Durus or > ZODB in the same top-level namespace could be considered for 2.6. Flat is better than nested. I see no reason why we couldn't have all of this: database.sqllite database.zodb database.duras database.oracle there's no need to group the SQL databases. -- Michael Chermside From phd at mail2.phd.pp.ru Wed Mar 29 17:28:44 2006 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Wed, 29 Mar 2006 19:28:44 +0400 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <20060329072201.xkuv6llu9wr484c8@login.werra.lunarpages.com> References: <20060329072201.xkuv6llu9wr484c8@login.werra.lunarpages.com> Message-ID: <20060329152844.GE29855@phd.pp.ru> On Wed, Mar 29, 2006 at 07:22:01AM -0800, Michael Chermside wrote: > Flat is better than nested. I see no reason why we couldn't have all of > this: > > database.sqllite > database.zodb > database.duras > database.oracle > > there's no need to group the SQL databases. If flat is really so much better than nested there is no need to create a "database" namespace. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From aahz at pythoncraft.com Wed Mar 29 17:33:44 2006 From: aahz at pythoncraft.com (Aahz) Date: Wed, 29 Mar 2006 07:33:44 -0800 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: References: <200603291947.34461.anthony@interlink.com.au> Message-ID: <20060329153344.GC7531@panix.com> On Wed, Mar 29, 2006, Fredrik Lundh wrote: > Anthony Baxter wrote: >> >> And I'd prefer 'database.sqlite' rather than 'db.sqlite'. > > and extensible_markup_language.document_object_model over > xml.dom, I presume ? ;-) While I see your point, from my POV "xml" feels different from "db". Part of it is that XML is *always* referred to that way in texts except for an initial expansion so people know what it stands for, whereas database is only abbreviated in limited circumstances. In addition, three characters is more visible than two. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Look, it's your affair if you want to play with five people, but don't go calling it doubles." --John Cleese anticipates Usenet From tim.hochberg at ieee.org Wed Mar 29 17:45:10 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Wed, 29 Mar 2006 08:45:10 -0700 Subject: [Python-Dev] INPLACE_ADD and INPLACE_MULTIPLY oddities in ceval.c In-Reply-To: <20060329104540.GA26145@code0.codespeak.net> References: <4428C77E.5020005@canterbury.ac.nz> <4429D71F.1010601@canterbury.ac.nz> <20060329104540.GA26145@code0.codespeak.net> Message-ID: <442AAB86.4010008@ieee.org> Armin Rigo wrote: > Hi Greg, > > On Wed, Mar 29, 2006 at 12:38:55PM +1200, Greg Ewing wrote: > >>I'm really thinking more about the non-inplace operators. >>If nb_add and sq_concat are collapsed into a single slot, >>it seems to me that if you do >> >> a = [1, 2, 3] >> b = array([4, 5, 6]) >> c = a + b >> >>then a will be asked "Please add yourself to b", and a >>will say "Okay, I know how to do that!" and promptly >>concatenate itself with b. > > > No: there is a difference between + and += for lists. You can only > concatenate exactly a list to a list. Indeed: > > >>> [].__add__((2, 3)) > TypeError: can only concatenate list (not "tuple") to list > > By contrast, list += is like extend() and accepts any iterable. > So if we provide a complete fix, [].__add__(x) will be modified to > return NotImplemented instead of raising TypeError if x is not a list, > and then [1,2,3]+array([4,5,6]) will fall back to array.__radd__() as > before. Ouch. Assuming the same path is followed with tuples, I think that this means the following behaviour will continue: >>> t = (1,2,3) >>> a = array([4,5,6]) >>> t += a >>> t array([5, 7, 9]) That's not particularly desirable. There's not much to be done about it short of adding __iadd__s everywhere, which is probably brittle and unfriendly. And, admittedly this is a corner case that's very rarely going to cause trouble. Still, perhaps for Py3K it's worth considering if PyNumber_InplaceAdd should only call __iadd__ and __add__, not __radd__. Thus giving the target object complete control during inplace adds. Similarly for other inplace operations, of course. I'm not certain that all of the consequences of this change would be benign, but it's something to consider. > I'll try harder to see if there is a reasonable example whose behavior > would change... > Regards, Tim Hochberg From fredrik at pythonware.com Wed Mar 29 17:52:07 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 29 Mar 2006 17:52:07 +0200 Subject: [Python-Dev] I'm not getting email from SF when assignedabug/patch References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> <1143554576.2310.161.camel@geddy.wooz.org> <17449.21764.475497.457592@montanaro.dyndns.org><44298954.4040705@v.loewis.de> Message-ID: Georg Brandl wrote: > Generally, I like Trac very much, especially for its interconnected subsystems. > I've used it with smaller projects, and there it works perfectly. > Having said that, I don't know if the Trac ticket system (which would be the > most important subsystem for us) scales up well enough. I'm completely fail to see why a Trac server shouldn't scale up at least as well as the SF-hosted Python tracker... (I mean, we're talking about one project, not 116,757 ...) > Of course, if there are only a few bits missing, instead of paying someone > to operate a complicated tracker, perhaps the money could be used to pay > someone to improve Trac... I cannot find the message right now, but I'm quite sure that someone recently suggested that the right way to try out a new tracker was to use it for the Python 3000 activity. my suggestion is to ask the python-hosting folks if they're willing to set up a free pyk3 account: http://www.python-hosting.com/freetrac if this works well for Python 3000, the next step would be to ask them if they're willing to host the 2.X tracker as well (and optionally the SVN archive, as well). PSF might not be the Mozilla Foundation, but I'm sure there's enough funds to pay for suitably large commercial account. From barry at python.org Wed Mar 29 18:06:49 2006 From: barry at python.org (Barry Warsaw) Date: Wed, 29 Mar 2006 11:06:49 -0500 Subject: [Python-Dev] I'm not getting email from SF when assignedabug/patch In-Reply-To: References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> <1143554576.2310.161.camel@geddy.wooz.org> <17449.21764.475497.457592@montanaro.dyndns.org> <44298954.4040705@v.loewis.de> Message-ID: <1143648409.10799.159.camel@resist.wooz.org> On Wed, 2006-03-29 at 17:52 +0200, Fredrik Lundh wrote: > if this works well for Python 3000, the next step would be to ask them > if they're willing to host the 2.X tracker as well (and optionally the SVN > archive, as well). PSF might not be the Mozilla Foundation, but I'm sure > there's enough funds to pay for suitably large commercial account. I'll just point out that Atlassian has offered us free hosting for a Jira/Confluence solution (plus svn and other stuff we may or may not want). I personally support this option, but I know (and accept!) that there are differing opinions. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060329/0bda47d8/attachment-0001.pgp From exarkun at divmod.com Wed Mar 29 18:24:07 2006 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Wed, 29 Mar 2006 11:24:07 -0500 Subject: [Python-Dev] I'm not getting email from SF when assignedabug/patch In-Reply-To: Message-ID: <20060329162407.22481.1766520559.divmod.quotient.7276@ohm> On Wed, 29 Mar 2006 17:52:07 +0200, Fredrik Lundh wrote: >Georg Brandl wrote: > >> Generally, I like Trac very much, especially for its interconnected subsystems. >> I've used it with smaller projects, and there it works perfectly. > >> Having said that, I don't know if the Trac ticket system (which would be the >> most important subsystem for us) scales up well enough. > >I'm completely fail to see why a Trac server shouldn't scale up at least as >well as the SF-hosted Python tracker... (I mean, we're talking about one >project, not 116,757 ...) One reason might be the lack of paging for tickets. Viewing ~600 tickets at once (approximately the number of open tickets in Twisted's tracker) serves up a 2MB page. How many open tickets does Python have? :) Another reason is that the currently released version hemorrhage memory pretty badly (sometimes ~30MB for a single HTTP request). This is better in trunk at HEAD, but I don't think it's entirely resolved yet. Not that I am suggesting these and the other problems like them are unfixable, but someone will have to fix them, and whoever is going to have to admin the installation should be aware that there will be some issues. Jean-Paul From tds333+pydev at gmail.com Wed Mar 29 18:57:43 2006 From: tds333+pydev at gmail.com (Wolfgang Langner) Date: Wed, 29 Mar 2006 18:57:43 +0200 Subject: [Python-Dev] I'm not getting email from SF when assignedabug/patch In-Reply-To: <1143648409.10799.159.camel@resist.wooz.org> References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> <1143554576.2310.161.camel@geddy.wooz.org> <17449.21764.475497.457592@montanaro.dyndns.org> <44298954.4040705@v.loewis.de> <1143648409.10799.159.camel@resist.wooz.org> Message-ID: <4c45c1530603290857o50a10d0cnef21922927d5810@mail.gmail.com> Hello, > I'll just point out that Atlassian has offered us free hosting for a > Jira/Confluence solution (plus svn and other stuff we may or may not > want). I personally support this option, but I know (and accept!) that > there are differing opinions. It is a Java system. Why promote Java solutions for python ? I think there are good python solutions for a bug tracker and we should prefer them. -- bye by Wolfgang From pje at telecommunity.com Wed Mar 29 19:27:29 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 29 Mar 2006 12:27:29 -0500 Subject: [Python-Dev] Class decorators In-Reply-To: <200603290048.52669.fdrake@acm.org> References: <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060329121635.0390fce8@mail.telecommunity.com> At 12:48 AM 3/29/2006 -0500, Fred L. Drake, Jr. wrote: >Agreed, but... guess we can't have everything. On the other hand, something >like: > > class Foo: > """Documentation is good.""" > > @class implements(IFoo) > >is not ambiguous. Hmm. It even says what it means. :-) Interesting. I might have to rename some of my decorators to read well that way, though. I'll play around with the idea a bit. Of course, one of my use cases for class-level declarations is to work around the fact that decorators can't be applied to arbitrary attributes of a class, only to methods. Hm. Let me see: class SomeThing: """Some docs""" @class security.permissions( foo = security.Anybody, bar = myapp.User, ... ) class SomeCommand: """usage: some_command [-v|-q] [-f FILENAME] ...""" @class options.accept( verbose = [ options.Set('-v', '--verbose', value=True, help="Be loud"), options.Set('-q', '--quiet', value=False, help="Be quiet") ], filename = [ options.Set('-f', '--file', type=str, metavar="FILENAME") ] ) Not bad. Not bad at all. From pje at telecommunity.com Wed Mar 29 19:32:36 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 29 Mar 2006 12:32:36 -0500 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <442A4902.10506@ghaering.de> References: <200603281759.18756.anthony@interlink.com.au> <442987D8.4010903@v.loewis.de> <442A38BA.5000400@ghaering.de> Message-ID: <5.1.1.6.0.20060329122933.0390cef8@mail.telecommunity.com> At 10:44 AM 3/29/2006 +0200, Gerhard H?ring wrote: >Creating latex docs sounds like I could do it, too. FYI, there's a reST->PythonDoc converter somebody wrote: http://www.rexx.com/~dkuhlman/rstpythonlatex_intro.html I'm planning to try it for porting the setuptools docs. I'm sure that editing the result will be required, but it might be a good way to get the mechanical parts of the translation done. From pje at telecommunity.com Wed Mar 29 19:34:44 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 29 Mar 2006 12:34:44 -0500 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <200603291947.34461.anthony@interlink.com.au> References: <200603281759.18756.anthony@interlink.com.au> Message-ID: <5.1.1.6.0.20060329123254.037662c8@mail.telecommunity.com> At 07:47 PM 3/29/2006 +1100, Anthony Baxter wrote: >My only concern about this is that it wouldn't be possible for other >authors to provide 3rd party packages as (for instance) db.mysqldb >because of the way package importing works. See the stdlib module 'pkgutil' for one way around this, that works with existing Python versions. And the 'pkg_resources' module, slated for 2.5a2 inclusion, offers another way, that also works with modules supplied in eggs. So this bit is going to be more of a policy decision (pkgutil vs. pkg_resources) rather than a technical problem per se. From jack at performancedrivers.com Wed Mar 29 19:45:06 2006 From: jack at performancedrivers.com (Jack Diederich) Date: Wed, 29 Mar 2006 12:45:06 -0500 Subject: [Python-Dev] Class decorators In-Reply-To: <200603290111.06247.fdrake@acm.org> References: <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <200603290048.52669.fdrake@acm.org> <200603290111.06247.fdrake@acm.org> Message-ID: <20060329174506.GG5787@performancedrivers.com> On Wed, Mar 29, 2006 at 01:11:06AM -0500, Fred L. Drake, Jr. wrote: > On Wednesday 29 March 2006 00:48, Fred L. Drake, Jr. wrote: > > I think the existing usage for classes is perfectly readable. The > > @-syntax works well for functions as well. > > On re-reading what I wrote, I don't think I actually clarified the point I was > trying to make originally. > > My point wasn't that I desparately need @-syntax for class decorators (I > don't), or see it as inherantly superior in some way. It's much simpler than > that: I just want to be able to use the same syntax for a group of use cases > regardless of whether the target is a function or a class. > > This fits into the nice-to-have category for me, since the use case can be the > same regardless of whether I'm decorating a class or a function. (I will > note that when this use case applies to a function, it's usually a > module-level function I'm decorating rather than a method.) Agreed, let's not have the decorator syntax argument all over again. Once someone knows how a function decorator works they should be able to guess how a class decorator works. In my old patch[1] the grammar production for decorators was: decorated_thing: decorators (funcdef|classdef) Which makes sense, once you know how to decorate one thing you know how to decorate all things. -jackdied [1] http://python.org/sf/1007991 From fumanchu at amor.org Wed Mar 29 19:48:57 2006 From: fumanchu at amor.org (Robert Brewer) Date: Wed, 29 Mar 2006 09:48:57 -0800 Subject: [Python-Dev] [Python-3000] Iterators for dict keys, values, and items == annoying :) Message-ID: <435DF58A933BA74397B42CDEB8145A86010CBC5E@ex9.hostedexchange.local> Nick Coghlan wrote: > There are three big use cases: > > dict.keys > dict.values > dict.items > > Currently these all return lists, which may be expensive in > terms of copying. They all have iter* variants which while > memory efficient, are far less convenient to work with. I'm still wondering what "far less convenient" means. Is it simply the 4 extra key presses? I find the iter* variants to be a great solution. Robert Brewer System Architect Amor Ministries fumanchu at amor.org From janssen at parc.com Wed Mar 29 19:57:09 2006 From: janssen at parc.com (Bill Janssen) Date: Wed, 29 Mar 2006 09:57:09 PST Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: Your message of "Wed, 29 Mar 2006 05:35:48 PST." <442A8D34.8060200@ghaering.de> Message-ID: <06Mar29.095711pst."58633"@synergy1.parc.xerox.com> > I think short names are more more consistent with the existing naming in > the standard library. Which doesn't make it a good idea. +1 on adding longer top-level package names as aliases for existing shorter top-level package names. Bill From g.brandl at gmx.net Wed Mar 29 20:06:11 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 29 Mar 2006 20:06:11 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <06Mar29.095711pst."58633"@synergy1.parc.xerox.com> References: <442A8D34.8060200@ghaering.de> <06Mar29.095711pst."58633"@synergy1.parc.xerox.com> Message-ID: Bill Janssen wrote: >> I think short names are more more consistent with the existing naming in >> the standard library. > > Which doesn't make it a good idea. +1 on adding longer top-level > package names as aliases for existing shorter top-level package names. Which existing short names do you have in mind? os? xml? email? Let's not get excited over names here. I say: let the RM or the BDFL, whoever feels responsible to pronounce whether PySQLite actually gets in, decide which name it will have. Georg From martin at v.loewis.de Wed Mar 29 20:16:24 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 29 Mar 2006 20:16:24 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: References: <200603281759.18756.anthony@interlink.com.au> <442987D8.4010903@v.loewis.de> <442A38BA.5000400@ghaering.de> Message-ID: <442ACEF8.90100@v.loewis.de> Neal Norwitz wrote: > However, I'm -0 on adding this to 2.5. We've already got a lot of > changes. I don't want us to keep piling more on. Also I thought I > saw Gerhard say that there were some other things he wanted to finish > and the timing might work better for him to defer a bit. Some of > these things sounded like API changes which may be more problematic > once in the core as we may have stricter rules on backwards > compatibility. > > We also have to convert the doc from ReST to latex. None of these are > show stoppers, but it adds to the amount of work we need to do before > release. And there's already more work than we can handle. This should all depend on volunteers being willing to contribute. If Gerhard wants to do this all on his own, and finds the time - that would be fine with me. Also, if others are willing to help, that could work (e.g. conversion of the documentation could be done by a different contributor). However, the release schedule for 2.5 should not change because of that. Regards, Martin From martin at v.loewis.de Wed Mar 29 20:18:22 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 29 Mar 2006 20:18:22 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <442A4902.10506@ghaering.de> References: <200603281759.18756.anthony@interlink.com.au> <442987D8.4010903@v.loewis.de> <442A38BA.5000400@ghaering.de> <442A4902.10506@ghaering.de> Message-ID: <442ACF6E.3010803@v.loewis.de> Gerhard H?ring wrote: > Creating latex docs sounds like I could do it, too. What I'd personally > like to offload are these two tasks: > > - integreting pysqlite into the Python build process > - in particular the win32 build process > > I would have access to Linux and win32 development machines with MS > VS2003, but I don't have enough experience with the Python build process > to not make stupid mistakes here. I could create the VC project file if desired. Regards, Martin From benji at benjiyork.com Wed Mar 29 20:39:16 2006 From: benji at benjiyork.com (Benji York) Date: Wed, 29 Mar 2006 13:39:16 -0500 Subject: [Python-Dev] I'm not getting email from SF when assignedabug/patch In-Reply-To: <4c45c1530603290857o50a10d0cnef21922927d5810@mail.gmail.com> References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> <1143554576.2310.161.camel@geddy.wooz.org> <17449.21764.475497.457592@montanaro.dyndns.org> <44298954.4040705@v.loewis.de> <1143648409.10799.159.camel@resist.wooz.org> <4c45c1530603290857o50a10d0cnef21922927d5810@mail.gmail.com> Message-ID: <442AD454.3010302@benjiyork.com> Wolfgang Langner wrote: > It is a Java system. Why promote Java solutions for python ? > I think there are good python solutions for a bug tracker and we > should prefer them. It is an application. Why worry about its implementation language? If there are good Python solutions they should be used, if not it's better to use something that works well regardless of what it's written in. (Not that I qualify to have an opinion. I don't have time to contribute to Python, other than with snarky emails.) -- Benji York From martin at v.loewis.de Wed Mar 29 20:38:50 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 29 Mar 2006 20:38:50 +0200 Subject: [Python-Dev] PySet API In-Reply-To: <200603291357.59405.gmccaughan@synaptics-uk.com> References: <4429F664.3080706@canterbury.ac.nz> <200603291357.59405.gmccaughan@synaptics-uk.com> Message-ID: <442AD43A.4000703@v.loewis.de> Gareth McCaughan wrote: > However: if Raymond, or anyone else, is offended, then I'm sorry. > Now, what about the technical issues, as opposed to the way I > happened to introduce my comments? Proposing that a certain API in an open source project is introduced for a single "customer" is indeed a surprising notion, and I don't think it should be done. Either there is a need for the API, in which case it should be added, or there isn't (and the user is mistaken requesting it), then it shouldn't be added. Given that Barry insists so firmly that there is a need, and that this need arises from a significant code simplification that can be achieved through the API, the natural conclusion is to add the API. That, of course, assumes that you believe Barry's testimony. Regards, Martin From martin at v.loewis.de Wed Mar 29 20:39:17 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 29 Mar 2006 20:39:17 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <442A38BA.5000400@ghaering.de> References: <200603281759.18756.anthony@interlink.com.au> <442987D8.4010903@v.loewis.de> <442A38BA.5000400@ghaering.de> Message-ID: <442AD455.3060802@v.loewis.de> Gerhard H?ring wrote: > In particular, I would then synchronize changes that have proven stable > in the standalone release to the Python core sqlite module. I think this > is how Barry does it with the email module, too. Sounds all fine to me. Regards, Martin From guido at python.org Wed Mar 29 20:55:51 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 29 Mar 2006 10:55:51 -0800 Subject: [Python-Dev] What about PEP 299? In-Reply-To: <442A7AE4.2040702@gmail.com> References: <20060328224400.GA9042@discworld.dyndns.org> <442A7AE4.2040702@gmail.com> Message-ID: Die, thread. Do I personally have to go into svn and reject this PEP? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Mar 29 21:01:59 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 29 Mar 2006 11:01:59 -0800 Subject: [Python-Dev] INPLACE_ADD and INPLACE_MULTIPLY oddities in ceval.c In-Reply-To: <20060329125148.GA32251@code0.codespeak.net> References: <20060328112856.GA21294@code0.codespeak.net> <20060329125148.GA32251@code0.codespeak.net> Message-ID: On 3/29/06, Armin Rigo wrote: > Hi all, > > On Tue, Mar 28, 2006 at 09:50:49AM -0800, Guido van Rossum wrote: > > C extensions are my main worry -- OTOH if += for a list can already > > passes arbitrary types as the argument, then any extension types > > should be ready to expect this, right? > > Yes, I don't think C extensions are going to segfault. My worry is > about returning a different result than before. Actually I believe the > problem is not specific to C extensions. Here are some typical behavior > changes that could be observed in pure Python already: > > class X(object): > def __radd__(self, other): > return 42 > def __iter__(self): > return iter("xyz") > def __rmul__(self, other): > return 42 > def __index__(self): > return 5 > > t = [] > t += X() > print t # current: 42 new: ['x', 'y', 'z'] > print [1] * X() # current: 42 new: [1, 1, 1, 1, 1] > > Another visible difference is that the __add__/__iadd__/__mul__/__imul__ > methods of lists, tuples, strings etc., will return NotImplemented > instead of raising the TypeError themselves. This could impact user > subclasses of these built-in types trying to override and call the super > methods, not expecting a NotImplemented result (a reason why > NotImplemented should have been an exception in the first place IMHO). > > (A different bug I found is that [1].__mul__(X()) with an __index__able > class X currently raises TypeError, even though [1]*X() works just > fine.) > > This seems to be it on the incompatibility side. I'd vote for the > change anyway because the language specs -- as well as PyPy and probably > all Python implementations other than CPython -- don't have this > double-slot inconsistency and already show the "new" behavior. For what > it's worth no CPython test breaks on top of PyPy because of this. > > If this change is accepted I'll submit a patch for 2.5. I trust you in these matters. Go for it. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Mar 29 21:03:21 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 29 Mar 2006 11:03:21 -0800 Subject: [Python-Dev] INPLACE_ADD and INPLACE_MULTIPLY oddities in ceval.c In-Reply-To: <442AAB86.4010008@ieee.org> References: <4428C77E.5020005@canterbury.ac.nz> <4429D71F.1010601@canterbury.ac.nz> <20060329104540.GA26145@code0.codespeak.net> <442AAB86.4010008@ieee.org> Message-ID: On 3/29/06, Tim Hochberg wrote: > Ouch. Assuming the same path is followed with tuples, I think that this > means the following behaviour will continue: > > >>> t = (1,2,3) > >>> a = array([4,5,6]) > >>> t += a > >>> t > array([5, 7, 9]) > > That's not particularly desirable. Why not? It doesn't really differ much from the following example IMO: >>> x = 3 >>> x += 0.5 >>> x 3.5 >>> -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Mar 29 21:07:12 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 29 Mar 2006 11:07:12 -0800 Subject: [Python-Dev] Class decorators In-Reply-To: <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> References: <5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com> <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> <200603282335.09341.fdrake@acm.org> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> Message-ID: On 3/28/06, Phillip J. Eby wrote: > If we're using Zope 3 as an example, I personally find that: > > class Foo: > """Docstring here, blah blah blah > """ > implements(IFoo) > > is easier to read than: > > @implements(IFoo) > class Foo: > """Docstring here, blah blah blah > """ But the former also smells more of magic. A function putting a magic variable into the scope in which it is called isn't exactly elegant use of Python. You've probably read too much Zope code to be sensitive to this argument; but to me it still grates. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Mar 29 21:13:45 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 29 Mar 2006 11:13:45 -0800 Subject: [Python-Dev] I'm not getting email from SF when assignedabug/patch In-Reply-To: <4c45c1530603290857o50a10d0cnef21922927d5810@mail.gmail.com> References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> <1143554576.2310.161.camel@geddy.wooz.org> <17449.21764.475497.457592@montanaro.dyndns.org> <44298954.4040705@v.loewis.de> <1143648409.10799.159.camel@resist.wooz.org> <4c45c1530603290857o50a10d0cnef21922927d5810@mail.gmail.com> Message-ID: On 3/29/06, Wolfgang Langner wrote: [Barry] > > I'll just point out that Atlassian has offered us free hosting for a > > Jira/Confluence solution (plus svn and other stuff we may or may not > > want). I personally support this option, but I know (and accept!) that > > there are differing opinions. > > It is a Java system. Why promote Java solutions for python ? > I think there are good python solutions for a bug tracker and we > should prefer them. Watch out for the parochialism! I like Python as much as the next guy (probably more :-) but I'm sensitive to choosing the best solution. In this case I think the criteria should be saving volunteer sysadmin time in maintenance, conveniece for users, and convenience in developer time (not necessarily in that order). I don't know any of the suggested systems well enough to know how they score, so I don't want to side any particular proposal; but I think we should not base our argument on "it's written in competing language X". Beware NIH. Also, we're supposed to be friendly with Java, as we have a major product in that arena. What if Java folks were to reject a Python solution because it's not written in Python? Wouldn't we be upset about the parochialism? The language choice should only be used as an argument if all else is equal. Of course, "hackability" of a particular solution may be a criterion too, and there the language choice could matter. But the above response sounded like a knee-jerk to me, and IMO needs to be rebutted. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From g.brandl at gmx.net Wed Mar 29 21:17:42 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 29 Mar 2006 21:17:42 +0200 Subject: [Python-Dev] What about PEP 299? In-Reply-To: References: <20060328224400.GA9042@discworld.dyndns.org> <442A7AE4.2040702@gmail.com> Message-ID: Guido van Rossum wrote: > Die, thread. > > Do I personally have to go into svn and reject this PEP? After my latest channeling disaster, I was cautious about this one ;) I'll reject it now. Georg From theller at python.net Wed Mar 29 21:22:31 2006 From: theller at python.net (Thomas Heller) Date: Wed, 29 Mar 2006 21:22:31 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <5.1.1.6.0.20060329122933.0390cef8@mail.telecommunity.com> References: <200603281759.18756.anthony@interlink.com.au> <442987D8.4010903@v.loewis.de> <442A38BA.5000400@ghaering.de> <442A4902.10506@ghaering.de> <5.1.1.6.0.20060329122933.0390cef8@mail.telecommunity.com> Message-ID: Phillip J. Eby wrote: > At 10:44 AM 3/29/2006 +0200, Gerhard H?ring wrote: >> Creating latex docs sounds like I could do it, too. > > FYI, there's a reST->PythonDoc converter somebody wrote: > > http://www.rexx.com/~dkuhlman/rstpythonlatex_intro.html > > I'm planning to try it for porting the setuptools docs. I'm sure that > editing the result will be required, but it might be a good way to get the > mechanical parts of the translation done. > I am currently using a somewhat hacked version of Greg Ward's mkpydoc script to convert the ctypes tutorial into libctypes.tex. So far it works great, except that I'm having problems converting a table. I'm in contact with someone from the docutils users list who has helped me with some TeX stuff. The hacked version of mkpydoc is here: http://cvs.sourceforge.net/viewcvs.py/ctypes/ctypes/docs/manual/mkpydoc.py?rev=1.7&only_with_tag=HEAD&view=markup Thomas From fredrik at pythonware.com Wed Mar 29 21:27:28 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 29 Mar 2006 21:27:28 +0200 Subject: [Python-Dev] I'm not getting email from SF whenassignedabug/patch References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com><00ff01c6523f$7c583290$bf03030a@trilan><1143554576.2310.161.camel@geddy.wooz.org><17449.21764.475497.457592@montanaro.dyndns.org><44298954.4040705@v.loewis.de> <1143648409.10799.159.camel@resist.wooz.org><4c45c1530603290857o50a10d0cnef21922927d5810@mail.gmail.com> Message-ID: Guido van Rossum wrote: > Watch out for the parochialism! I like Python as much as the next guy > (probably more :-) but I'm sensitive to choosing the best solution. you better make that "good enough", or we'll be stuck with SF for an- other hundred years. > The language choice should only be used as an argument if all else is > equal. since it's a lot easier to get Pythoneers to volunteer time to work on (develop, hack, keep running, create add-ons for) a solution written in Python, the criteria ought to be "the language choice is only irrelevant if there's no Python solution that's good enough". it's also a marketing thing; if the developers don't want to eat Python dogfood, why should anyone else do that ? From guido at python.org Wed Mar 29 21:36:22 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 29 Mar 2006 11:36:22 -0800 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <200603290152.00598.anthony@interlink.com.au> References: <200603281759.18756.anthony@interlink.com.au> <1143554351.2309.158.camel@geddy.wooz.org> <200603290152.00598.anthony@interlink.com.au> Message-ID: On 3/28/06, Anthony Baxter wrote: > I'm happy to work with Gerhard to make this happen. Does it need a > PEP? I'd say "no", but only because things like ElementTree didn't, > either. Does it need a BDFL pronouncement? I'd say yes. Unless you've recanted on that already, let me point out that I've never seen sqlite, and I've ignored this thread, so I don't know what the disagreement is all about. Perhaps one person in favor and one person against could summarize the argument for me? Otherwise I'll have to go with "no" just to err on the side of safety. I have strong feelings about the language. Sometimes I have strong feelings about the library. This doesn't seem to be one of those cases though... -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Mar 29 21:40:40 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 29 Mar 2006 11:40:40 -0800 Subject: [Python-Dev] I'm not getting email from SF whenassignedabug/patch In-Reply-To: References: <1143554576.2310.161.camel@geddy.wooz.org> <17449.21764.475497.457592@montanaro.dyndns.org> <44298954.4040705@v.loewis.de> <1143648409.10799.159.camel@resist.wooz.org> <4c45c1530603290857o50a10d0cnef21922927d5810@mail.gmail.com> Message-ID: On 3/29/06, Fredrik Lundh wrote: > Guido van Rossum wrote: > > > Watch out for the parochialism! I like Python as much as the next guy > > (probably more :-) but I'm sensitive to choosing the best solution. > > you better make that "good enough", or we'll be stuck with SF for an- > other hundred years. Fair enough. I like "good enough" as a criterion; it's served me well in real life for many years, since it reduces the time I waste pondering decisions. Unlike language design issues, tool selection choices aren't forever. > > The language choice should only be used as an argument if all else is > > equal. > > since it's a lot easier to get Pythoneers to volunteer time to work on > (develop, hack, keep running, create add-ons for) a solution written in > Python, the criteria ought to be "the language choice is only irrelevant > if there's no Python solution that's good enough". > > it's also a marketing thing; if the developers don't want to eat Python > dogfood, why should anyone else do that ? Sure. There are plenty of reasons to prefer Python, making all else "not equal". I was just warning against knee-jerk parochialism, which I don't think will serve us well. There's Perl code in the Python source tree, and the only reason to get rid of it IMO should be if it no longer serves our purpose. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From foom at fuhm.net Wed Mar 29 21:42:01 2006 From: foom at fuhm.net (James Y Knight) Date: Wed, 29 Mar 2006 14:42:01 -0500 Subject: [Python-Dev] PySet API In-Reply-To: <442AD43A.4000703@v.loewis.de> References: <4429F664.3080706@canterbury.ac.nz> <200603291357.59405.gmccaughan@synaptics-uk.com> <442AD43A.4000703@v.loewis.de> Message-ID: <18DE251C-1661-4036-9A2F-9473647D590A@fuhm.net> On Mar 29, 2006, at 1:38 PM, Martin v. L?wis wrote: > Given that Barry insists so firmly that there is a need, and that > this need arises from a significant code simplification that can > be achieved through the API, the natural conclusion is to add > the API. That, of course, assumes that you believe Barry's testimony. It doesn't seem to me that there really is a significant code simplification, looking at the diff between Raymond's code examples. @@ -7,2 +7 @@ - PyObject *key; - Py_ssize_t pos = 0; + PyObject *it, *key; @@ -13 +12,5 @@ - while (set_next(so, &pos, &key)) { + it = PyObject_GetIter(self); + if (it == NULL) + return -1; + + while ((key = PyIter_Next(it)) != NULL) { @@ -14,0 +18 @@ + Py_DECREF(key); @@ -16 +20,2 @@ - return -1; + Py_DECREF(it); + return -1; @@ -19,0 +25,3 @@ + Py_DECREF(it); + if (PyErr_Occurred()) + return -1; James From brett at python.org Wed Mar 29 21:19:14 2006 From: brett at python.org (Brett Cannon) Date: Wed, 29 Mar 2006 11:19:14 -0800 Subject: [Python-Dev] What about PEP 299? In-Reply-To: References: <20060328224400.GA9042@discworld.dyndns.org> <442A7AE4.2040702@gmail.com> Message-ID: On 3/29/06, Guido van Rossum wrote: > Die, thread. > > Do I personally have to go into svn and reject this PEP? No, just get a procrastinating student to do it. -Brett From pje at telecommunity.com Wed Mar 29 21:53:40 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 29 Mar 2006 14:53:40 -0500 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: References: <200603290152.00598.anthony@interlink.com.au> <200603281759.18756.anthony@interlink.com.au> <1143554351.2309.158.camel@geddy.wooz.org> <200603290152.00598.anthony@interlink.com.au> Message-ID: <5.1.1.6.0.20060329144142.0390e5d8@mail.telecommunity.com> At 11:36 AM 3/29/2006 -0800, Guido van Rossum wrote: >On 3/28/06, Anthony Baxter wrote: > > I'm happy to work with Gerhard to make this happen. Does it need a > > PEP? I'd say "no", but only because things like ElementTree didn't, > > either. Does it need a BDFL pronouncement? I'd say yes. > >Unless you've recanted on that already, let me point out that I've >never seen sqlite, and I've ignored this thread, so I don't know what >the disagreement is all about. Perhaps one person in favor and one >person against could summarize the argument for me? Pro: * SQLite is really nice to have for writing simple applications with small data needs, especially client-side software. It's probably the best-of-breed open source embedded SQL DB right now. * So, having a wrapper would be a big "Batteries included" plus for Python Con: * Competing Python wrappers exist * SQLite itself is updated frequently, let alone the wrappers * Build integration risks unknown, possible delay of 2.5? * Another external library to track and maybe have emergency updates of I personally lean somewhat toward the con side because to me it's just as easy to "easy_install pysqlite" after the fact, or get it from the appropriate packager (RPM, Debian, whatever). However, we can't please everybody. If we go for more "batteries included", one group will complain about how much we have linked in and don't have proper system dependencies. If we go for "easy to install add-ons", the same people will gripe that it's the job of the packaging system to do those add-ons, and another group will chime in that they don't have or don't want the packaging system. So we might as well flip a coin. :) From g.brandl at gmx.net Wed Mar 29 22:08:34 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 29 Mar 2006 22:08:34 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <5.1.1.6.0.20060329144142.0390e5d8@mail.telecommunity.com> References: <200603290152.00598.anthony@interlink.com.au> <200603281759.18756.anthony@interlink.com.au> <1143554351.2309.158.camel@geddy.wooz.org> <200603290152.00598.anthony@interlink.com.au> <5.1.1.6.0.20060329144142.0390e5d8@mail.telecommunity.com> Message-ID: Phillip J. Eby wrote: > At 11:36 AM 3/29/2006 -0800, Guido van Rossum wrote: >>On 3/28/06, Anthony Baxter wrote: >> > I'm happy to work with Gerhard to make this happen. Does it need a >> > PEP? I'd say "no", but only because things like ElementTree didn't, >> > either. Does it need a BDFL pronouncement? I'd say yes. >> >>Unless you've recanted on that already, let me point out that I've >>never seen sqlite, and I've ignored this thread, so I don't know what >>the disagreement is all about. Perhaps one person in favor and one >>person against could summarize the argument for me? > > Pro: > > * SQLite is really nice to have for writing simple applications with small > data needs, especially client-side software. It's probably the > best-of-breed open source embedded SQL DB right now. > > * So, having a wrapper would be a big "Batteries included" plus for Python > > Con: > > * Competing Python wrappers exist Which aren't DBAPI compliant, and I think not nearly as popular. > * SQLite itself is updated frequently, let alone the wrappers That's a point. > * Build integration risks unknown, possible delay of 2.5? There could be an sqlite-integration branch. If it's ready for beta 1, it is merged then, if not, it is merged to trunk after 2.5 final happened. Georg From martin at v.loewis.de Wed Mar 29 22:53:04 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 29 Mar 2006 22:53:04 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: References: <200603281759.18756.anthony@interlink.com.au> <1143554351.2309.158.camel@geddy.wooz.org> <200603290152.00598.anthony@interlink.com.au> Message-ID: <442AF3B0.3060905@v.loewis.de> Guido van Rossum wrote: > Unless you've recanted on that already, let me point out that I've > never seen sqlite, and I've ignored this thread, so I don't know what > the disagreement is all about. Perhaps one person in favor and one > person against could summarize the argument for me? Otherwise I'll > have to go with "no" just to err on the side of safety. I have strong > feelings about the language. Sometimes I have strong feelings about > the library. This doesn't seem to be one of those cases though... Let me try to take both sides simultaneously: For: would add an SQL library to the standard distribution, and one that doesn't depend on additional infrastructure on the target machine (such as an existing database server); the author of that library is fine with including it in Python Against: Adds work-load on the release process, adding more libraries to the already-large list of new libraries for 2.5. Choice of naming things is ad-hoc, but gets cast in stone by the release; likewise, choice of specific SQL library might inhibit addition of different libraries later. I'm sure people will add to the list if they think I omitted important points. Regards, Martin From fumanchu at amor.org Wed Mar 29 22:57:57 2006 From: fumanchu at amor.org (Robert Brewer) Date: Wed, 29 Mar 2006 12:57:57 -0800 Subject: [Python-Dev] pysqlite for 2.5? Message-ID: <435DF58A933BA74397B42CDEB8145A86010CC12A@ex9.hostedexchange.local> Martin v. L?wis wrote: > Guido van Rossum wrote: > > Unless you've recanted on that already, let me point out that I've > > never seen sqlite, and I've ignored this thread, so I don't > know what > > the disagreement is all about. Perhaps one person in favor and one > > person against could summarize the argument for me? Otherwise I'll > > have to go with "no" just to err on the side of safety. I > have strong > > feelings about the language. Sometimes I have strong feelings about > > the library. This doesn't seem to be one of those cases though... > > Let me try to take both sides simultaneously: > > For: would add an SQL library to the standard distribution, and one > that doesn't depend on additional infrastructure on the target machine > (such as an existing database server); the author of that library is > fine with including it in Python > > Against: Adds work-load on the release process, adding more libraries > to the already-large list of new libraries for 2.5. Choice of naming > things is ad-hoc, but gets cast in stone by the release; likewise, > choice of specific SQL library might inhibit addition of different > libraries later. More Against?: Explaining "database is locked" errors (due to SQLite's exposed multiple-readers/one-writer design) on a daily basis (FAQ entries notwithstanding). Robert Brewer System Architect Amor Ministries fumanchu at amor.org From fredrik at pythonware.com Wed Mar 29 23:08:59 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 29 Mar 2006 23:08:59 +0200 Subject: [Python-Dev] pysqlite for 2.5? References: <435DF58A933BA74397B42CDEB8145A86010CC12A@ex9.hostedexchange.local> Message-ID: Robert Brewer wrote: > More Against?: > Explaining "database is locked" errors (due to SQLite's exposed > multiple-readers/one-writer design) on a daily basis (FAQ entries > notwithstanding). wow. that's one quality argument. what's wrong with you ? From fredrik at pythonware.com Wed Mar 29 23:15:14 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 29 Mar 2006 23:15:14 +0200 Subject: [Python-Dev] pysqlite for 2.5? References: <200603281759.18756.anthony@interlink.com.au><1143554351.2309.158.camel@geddy.wooz.org><200603290152.00598.anthony@interlink.com.au> Message-ID: Guido van Rossum wrote: > Unless you've recanted on that already, let me point out that I've > never seen sqlite, and I've ignored this thread, so I don't know what > the disagreement is all about. what disagreement ? sqlite is a widely used light-weight SQL library (http://www.sqlite.org) that's an excellent choice for many kind of applications. it has no com- petition in this space. gerald's pysqlite binding is a second-generation implementation of the full DB-API (http://www.python.org/dev/peps/pep-0249/) for sqlite. from a user perspective, adding this to the standard library is a no-brainer. the only reason not to add it would be if the release managers don't have time to sort out the build issues. From gh at ghaering.de Wed Mar 29 23:17:40 2006 From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=) Date: Wed, 29 Mar 2006 23:17:40 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <5.1.1.6.0.20060329144142.0390e5d8@mail.telecommunity.com> References: <200603290152.00598.anthony@interlink.com.au> <200603281759.18756.anthony@interlink.com.au> <1143554351.2309.158.camel@geddy.wooz.org> <200603290152.00598.anthony@interlink.com.au> <5.1.1.6.0.20060329144142.0390e5d8@mail.telecommunity.com> Message-ID: <442AF974.70307@ghaering.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Phillip J. Eby wrote: > At 11:36 AM 3/29/2006 -0800, Guido van Rossum wrote: >> [...] Perhaps one person in favor and one >> person against could summarize the argument for me? > > Pro: > > * SQLite is really nice to have for writing simple applications with small > data needs, especially client-side software. It's probably the > best-of-breed open source embedded SQL DB right now. > * So, having a wrapper would be a big "Batteries included" plus for Python That would be my arguments, too. > Con: > > * Competing Python wrappers exist There is really only one other relevant wrapper: APSW. It was purposefully designed to *not* use the DB-API 2.0 (though a similar one), and being a "thinner" wrapper of SQLite. It wraps a few more functions of the SQLite API, though less and less, because pysqlite is catching up here. While there might be arguments for a "thinner" wrapper, I think that pysqlite has the advantage of being DB-API compliant (even with most optional DB-API extensions) and thus offers a good way to prototype database apps with a smooth upgrade path to other, more powerful, databases. > * SQLite itself is updated frequently, let alone the wrappers SQLite being updated regularly is not really a problem, because we can link dynamically against SQLite. And we probably *should* do this on Windows, too, so users can replace a SQLite.DLL with an updated version if they wish to. > * Build integration risks unknown, possible delay of 2.5? > * Another external library to track and maybe have emergency updates of Emergency updates are only for security problems, right? I don't think this would apply to pysqlite. I don't think that would apply to SQLite either, but if it's conceivable, it's another argument for dynamic linking. - -- Gerhard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFEKvlzdIO4ozGCH14RAuLKAJ9BGnHz4Tym60xOGSwSuqXlqRaAdwCdFeqx +vo5eC0aBu4S2sttb/iZPOc= =bJKK -----END PGP SIGNATURE----- From raymond.hettinger at verizon.net Wed Mar 29 23:29:02 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Wed, 29 Mar 2006 16:29:02 -0500 Subject: [Python-Dev] PySet API References: <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> <1143468446.10799.50.camel@resist.wooz.org> <200603281507.18175.gmccaughan@synaptics-uk.com> Message-ID: <001a01c65377$cd099a10$6601a8c0@RaymondLaptop1> [Gareth McCaughan] > For what it's worth[1], I think Raymond is absolutely on crack here. Nope. No mind-altering drugs here. Based on real-word experience, I have found PySet_Next() to be a bug factory and do not want it included in the API. The story is different for PySet_Update(). Defining it now could get in the way of possible future development for the module (the function may end-up taking a variable length argument list instead of a single argument). Neither of these proposals are necessary. Both have safe, simple, workable alternatives. It is not my problem if those alternatives do not suit your tastes. A personal aversion to the abstract api is no reason to forgo safety or to interfere with future development of the module. Quality and flexibility considerations trump micro-optimizations and personal style biases. Most of the push has been predicated on being in a snit about the existing iterator API. However, in the course of writing itertools and other Python enchancements, I've had occassion to thoroughly exercise the iterator API and have not found it to be a problem in practice. Raymond From fredrik at pythonware.com Wed Mar 29 23:32:59 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 29 Mar 2006 23:32:59 +0200 Subject: [Python-Dev] pysqlite for 2.5? References: <200603281759.18756.anthony@interlink.com.au><1143554351.2309.158.camel@geddy.wooz.org><200603290152.00598.anthony@interlink.com.au> Message-ID: > gerald's pysqlite binding sorry, gerhard. From brett at python.org Wed Mar 29 23:39:03 2006 From: brett at python.org (Brett Cannon) Date: Wed, 29 Mar 2006 13:39:03 -0800 Subject: [Python-Dev] Discussing the Great Library Reorganization Message-ID: While this is going to require a PEP (which I am willing to write), the discussion of adding pysqlite has brought forth some discussion on naming and packaging in the stdlub. Perhaps it's time to start discussing the Great Library Reorganization that has been discussed for eons. Here is a place I think we can take a queue from Java. I think we should have a root package, 'py', and then have subpackages within that. Those subpackages would group the existing modules that are not already in a nice package hierarchy. In other words, try to make it so that importing an actual module takes no more than three dots in the general case: ``from py.dev import pdb``, etc. I do think it is okay to put things without proper classification just under 'py' without being in a subpackage. The three dots idea is not hard. We could, for instance, have a py.dist subpackage and have pkgutil and distutils under it. That will make the modules in distutils take four dots, but that's just life and I think within reason for something that is not used directly by a large number of people. I also have no issue taking certain names from existing modules and making them both a module directly (as in putting what exists in a module into __init__.py for the subpackage with the same name) on top of putting more modules in the subpackage. The issue I see with this, though, is people doing something like ``from py import pickle; pickle.pickletools.dis()``, not realizing they need to import pickletools directly instead of getting to it through py.pickle . I don't want to spark a subpackage grouping discussion yet since we should decide on a general strategy of how we want the basic stdlib organized. I also don't want to argue over module renaming directly since that can be based on what subpackages there might be. -Brett From gh at ghaering.de Wed Mar 29 23:47:02 2006 From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=) Date: Wed, 29 Mar 2006 23:47:02 +0200 Subject: [Python-Dev] Discussing the Great Library Reorganization In-Reply-To: References: Message-ID: <442B0056.5040509@ghaering.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Brett Cannon wrote: > While this is going to require a PEP (which I am willing to write), > the discussion of adding pysqlite has brought forth some discussion on > naming and packaging in the stdlub. Perhaps it's time to start > discussing the Great Library Reorganization that has been discussed > for eons. [...] Wouldn't the newly founded python-3000 mailing list be the perfect place for such major changes? - -- Gerhard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFEKwBWdIO4ozGCH14RAnIkAJ9L/m8J4aiIisJKVimIv15mvSQApgCgnvP4 H/aV/ZuLs0DLScvnyrfsGPo= =Dm7c -----END PGP SIGNATURE----- From thomas at python.org Wed Mar 29 23:47:10 2006 From: thomas at python.org (Thomas Wouters) Date: Wed, 29 Mar 2006 23:47:10 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <5.1.1.6.0.20060329144142.0390e5d8@mail.telecommunity.com> References: <200603281759.18756.anthony@interlink.com.au> <1143554351.2309.158.camel@geddy.wooz.org> <200603290152.00598.anthony@interlink.com.au> <5.1.1.6.0.20060329144142.0390e5d8@mail.telecommunity.com> Message-ID: <9e804ac0603291347t2f4e1d94l4fa022fb0e2aa738@mail.gmail.com> On 3/29/06, Phillip J. Eby wrote: > Pro: > [...] Con: > > * Competing Python wrappers exist > * SQLite itself is updated frequently, let alone the wrappers > * Build integration risks unknown, possible delay of 2.5? > * Another external library to track and maybe have emergency updates of All of these con arguments go for bsddb, too, and without sounding too negative about bsddb, I believe SQLite is a *much* better solution than BerkeleyDB, for roughly the same problem space. The same goes for pysqlite vs. bsddb. IMNSHO, SQLite and pysqlite are much easier to use correctly than BerkelyDB and bsddb, for simple and complex tasks. I may be biased against bsddb because I spent too much time hunting refleaks in it, but I'm not biased in favour of SQLite -- I'm a PostgreSQL user myself. ;-P I personally lean somewhat toward the con side because to me it's just as > easy to "easy_install pysqlite" after the fact, or get it from the > appropriate packager (RPM, Debian, whatever). Actually, I have no doubt that all the package managers will split the 'bundled' pysqlite (whatever the name will be) in a separate package, just like it's done for Tkinter and bsddb and most other stdlib modules with extra dependencies. Nevertheless, adding it to the standard library is probably a good thing. I would probably choose sqlite instead of shelve/anydbm/bsddb if it were part of the standard library, even though it's probably installed on all my machines anyway. I guess it's a psych thing. As for people asking about deadlocks, well, I much rather explain about sqlite deadlocks than about BerkelyDB transactions. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060329/cb1624d7/attachment.htm From brett at python.org Wed Mar 29 23:56:00 2006 From: brett at python.org (Brett Cannon) Date: Wed, 29 Mar 2006 13:56:00 -0800 Subject: [Python-Dev] Discussing the Great Library Reorganization In-Reply-To: <442B0056.5040509@ghaering.de> References: <442B0056.5040509@ghaering.de> Message-ID: On 3/29/06, Gerhard H?ring wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Brett Cannon wrote: > > While this is going to require a PEP (which I am willing to write), > > the discussion of adding pysqlite has brought forth some discussion on > > naming and packaging in the stdlub. Perhaps it's time to start > > discussing the Great Library Reorganization that has been discussed > > for eons. [...] > > Wouldn't the newly founded python-3000 mailing list be the perfect place > for such major changes? If you go back and look at Guido's Python 3000 Process email he said that the change could occur in 2.6 and then be done for 3000. Renaming modules is not that hard to make backwards-compatible by having old names fully import the new name directly (``from py.dev.pdb import *``), especially if we put everything under the 'py' package and thus remove any possible name clashing with the existing arrangement. So I posted to python-dev since I think we could do it in 2.6 and then remove a ton of old modules we don't want anymore in 3000. -Brett From skip at pobox.com Thu Mar 30 00:00:18 2006 From: skip at pobox.com (skip at pobox.com) Date: Wed, 29 Mar 2006 16:00:18 -0600 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <5.1.1.6.0.20060329144142.0390e5d8@mail.telecommunity.com> References: <200603290152.00598.anthony@interlink.com.au> <200603281759.18756.anthony@interlink.com.au> <1143554351.2309.158.camel@geddy.wooz.org> <5.1.1.6.0.20060329144142.0390e5d8@mail.telecommunity.com> Message-ID: <17451.882.842146.979286@montanaro.dyndns.org> Phillip> Pro: Phillip> * SQLite is really nice to have for writing simple applications Phillip> with small data needs, especially client-side software. It's Phillip> probably the best-of-breed open source embedded SQL DB right Phillip> now. Phillip> * So, having a wrapper would be a big "Batteries included" plus Phillip> for Python Phillip> Con: Phillip> * Competing Python wrappers exist Phillip> * SQLite itself is updated frequently, let alone the wrappers Phillip> * Build integration risks unknown, possible delay of 2.5? Phillip> * Another external library to track and maybe have emergency Phillip> updates of I haven't been tracking the pysqlite discussion either, but one con you missed is that regardless of pro #1 people will almost certainly apply it to problems for which it is ill-suited, reflectly poorly on both Python and SQLite. Of course, that can and does happen today. Including pysqlite with Python just means it will happen more frequently. Phillip> I personally lean somewhat toward the con side because to me Phillip> it's just as easy to "easy_install pysqlite" after the fact, or Phillip> get it from the appropriate packager (RPM, Debian, whatever). Is it not possible to distribute an empty db package which is then populated with various database eggs (or other similar installation things)? Skip From allison at shasta.stanford.edu Thu Mar 30 00:04:49 2006 From: allison at shasta.stanford.edu (Dennis Allison) Date: Wed, 29 Mar 2006 14:04:49 -0800 (PST) Subject: [Python-Dev] Discussing the Great Library Reorganization In-Reply-To: Message-ID: Not quite on the same topic, but perhaps it belong there. I think most of use use both the stdlib and some selection of other libraries (MySQL-Python, ReportLab Toolkit, PyChart, and PyXML, for example). These libraries have to be managed independently and installed independently. It would be nice if there were a central repository of "blessed" extensions to the standard library which could be selectively loaded when a new version of Python is installed. There'd also need to be a tool for checking the extensions for new versions and upgrading since they would most likely not be maintained in lockstep with Python and its standard library. On Wed, 29 Mar 2006, Brett Cannon wrote: > While this is going to require a PEP (which I am willing to write), > the discussion of adding pysqlite has brought forth some discussion on > naming and packaging in the stdlub. Perhaps it's time to start > discussing the Great Library Reorganization that has been discussed > for eons. > -- From fredrik at pythonware.com Thu Mar 30 00:05:35 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 30 Mar 2006 00:05:35 +0200 Subject: [Python-Dev] pysqlite for 2.5? References: <200603290152.00598.anthony@interlink.com.au><200603281759.18756.anthony@interlink.com.au><1143554351.2309.158.camel@geddy.wooz.org><5.1.1.6.0.20060329144142.0390e5d8@mail.telecommunity.com> <17451.882.842146.979286@montanaro.dyndns.org> Message-ID: skip at pobox.com wrote: > I haven't been tracking the pysqlite discussion either, but one con you > missed is that regardless of pro #1 people will almost certainly apply it to > problems for which it is ill-suited, reflectly poorly on both Python and > SQLite. the arguments keep getting more and more weird. is there *any* part of the standard Python distribution that cannot be applied to problems for which it is ill-suited? From fredrik at pythonware.com Thu Mar 30 00:07:46 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 30 Mar 2006 00:07:46 +0200 Subject: [Python-Dev] Discussing the Great Library Reorganization References: <442B0056.5040509@ghaering.de> Message-ID: Brett Cannon wrote: > > Wouldn't the newly founded python-3000 mailing list be the perfect place > > for such major changes? > > If you go back and look at Guido's Python 3000 Process email he said > that the change could occur in 2.6 and then be done for 3000. > Renaming modules is not that hard to make backwards-compatible by > having old names fully import the new name directly (``from py.dev.pdb > import *``) hmm. I'm starting to think that the 3000 project may cause the core 2.X development to derail, even before 2.5 is out... From anthony at python.org Thu Mar 30 00:10:33 2006 From: anthony at python.org (Anthony Baxter) Date: Thu, 30 Mar 2006 09:10:33 +1100 Subject: [Python-Dev] RELEASED Python 2.4.3, final. Message-ID: <200603300910.48322.anthony@python.org> On behalf of the Python development team and the Python community, I'm happy to announce the release of Python 2.4.3 (final). Python 2.4.3 is a bug-fix release. See the release notes at the website (also available as Misc/NEWS in the source distribution) for details of the more than 50 bugs squished in this release, including a number found by the Coverity Scan project. Assuming no major bugs pop up, the next release of Python will be Python 2.5 (alpha 1), with a final 2.4.4 release of Python shortly after the final version of Python 2.5. The release plan for Python 2.5 is documented in PEP-0356. For more information on Python 2.4.3, including download links for various platforms, release notes, and known issues, please see: http://www.python.org/2.4.3/ Highlights of this new release include: - Bug fixes. According to the release notes, at least 50 have been fixed. - A small number of bugs, regressions and reference leaks have been fixed since Python 2.4.3 release candidate 1. See NEWS.txt for more. Highlights of the previous major Python release (2.4) are available from the Python 2.4 page, at http://www.python.org/2.4/highlights.html Enjoy this new release, Anthony Anthony Baxter anthony at python.org Python Release Manager (on behalf of the entire python-dev team) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20060330/85958038/attachment.pgp From janssen at parc.com Thu Mar 30 00:14:02 2006 From: janssen at parc.com (Bill Janssen) Date: Wed, 29 Mar 2006 14:14:02 PST Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: Your message of "Wed, 29 Mar 2006 13:15:14 PST." Message-ID: <06Mar29.141411pst."58633"@synergy1.parc.xerox.com> > from a user perspective, adding this to the standard library is a no-brainer. > the only reason not to add it would be if the release managers don't have > time to sort out the build issues. I agree with Fredrik here. On the package naming issue: using "em" for "email" would be wrong, just as "db" for "database" would be wrong. I might change my mind if all stdlib packages were under some toplevel package, like "stdlib". Bill From anthony at interlink.com.au Thu Mar 30 00:15:46 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu, 30 Mar 2006 09:15:46 +1100 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <20060328212217.22481.256092693.divmod.quotient.6808@ohm> References: <20060328212217.22481.256092693.divmod.quotient.6808@ohm> Message-ID: <200603300915.49106.anthony@interlink.com.au> On Wednesday 29 March 2006 08:22, Jean-Paul Calderone wrote: > >Agreed. pysqlite is solid and widely accepted, and AFAIK has no > >competition. > > FWIW: http://www.rogerbinns.com/apsw.html Looks interesting, but not being DB-API compliant is a huge issue for the stdlib. Part of the reason I want to see pysqlite in 2.5 is that it follows the standard DB-API. People can start off using it, then look at switching to a larger database if their application needs grow. -- Anthony Baxter It's never too late to have a happy childhood. From pje at telecommunity.com Thu Mar 30 00:31:51 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 29 Mar 2006 17:31:51 -0500 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <17451.882.842146.979286@montanaro.dyndns.org> References: <5.1.1.6.0.20060329144142.0390e5d8@mail.telecommunity.com> <200603290152.00598.anthony@interlink.com.au> <200603281759.18756.anthony@interlink.com.au> <1143554351.2309.158.camel@geddy.wooz.org> <5.1.1.6.0.20060329144142.0390e5d8@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060329172917.041345d8@mail.telecommunity.com> At 04:00 PM 3/29/2006 -0600, skip at pobox.com wrote: >Is it not possible to distribute an empty db package which is then populated >with various database eggs (or other similar installation things)? I don't think I understand your question. If you are asking whether it's possible to have Java-like "namespace packages" in Python, the answer is yes. The stdlib module "pkgutil" supports this for regular filesystem packages, and the "pkg_resources" module in setuptools extends this support to zipfiles and eggs. From p.f.moore at gmail.com Thu Mar 30 00:41:35 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 29 Mar 2006 23:41:35 +0100 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: References: <200603281759.18756.anthony@interlink.com.au> <1143554351.2309.158.camel@geddy.wooz.org> <200603290152.00598.anthony@interlink.com.au> Message-ID: <79990c6b0603291441i2f74882cw3574565c1b188357@mail.gmail.com> On 3/29/06, Fredrik Lundh wrote: > from a user perspective, adding this to the standard library is a no-brainer. > the only reason not to add it would be if the release managers don't have > time to sort out the build issues. Agreed. As a SQL user, it feels like a no-brainer to me, as well. I'd love to see a DB-API compliant module, and an embedded SQL database, in the core. I don't believe that there's anything comparable to (py)sqlite on that score. And as Anthony pointed out, it provides a nice upgrade path to "bigger" SQL/DB-API solutions. +1 from me. Paul. From brett at python.org Thu Mar 30 00:42:46 2006 From: brett at python.org (Brett Cannon) Date: Wed, 29 Mar 2006 14:42:46 -0800 Subject: [Python-Dev] Discussing the Great Library Reorganization In-Reply-To: References: Message-ID: On 3/29/06, Dennis Allison wrote: > > Not quite on the same topic, but perhaps it belong there. I think most of > use use both the stdlib and some selection of other libraries > (MySQL-Python, ReportLab Toolkit, PyChart, and PyXML, for example). These > libraries have to be managed independently and installed independently. > It would be nice if there were a central repository of "blessed" > extensions to the standard library which could be selectively loaded when > a new version of Python is installed. There'd also need to be a tool for > checking the extensions for new versions and upgrading since they would > most likely not be maintained in lockstep with Python and its standard > library. I think we should discuss this, but not necessarily here. Whether we want a blessed Cheeseshop section and have anything from there be automatically downloaded and installed as needed for the installed interpreter for things that we feel are important (ala pysqlite) but not necessarily in the core distribution should be discussed. But since I don't distribute individual modules often enough I am not in a position to lead that discussion. -Brett From greg.ewing at canterbury.ac.nz Thu Mar 30 00:42:57 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 30 Mar 2006 10:42:57 +1200 Subject: [Python-Dev] Class decorators In-Reply-To: <200603290048.52669.fdrake@acm.org> References: <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <200603290048.52669.fdrake@acm.org> Message-ID: <442B0D71.2060101@canterbury.ac.nz> Fred L. Drake, Jr. wrote: > class Foo: > """Documentation is good.""" > > @class implements(IFoo) That's an interesting idea. It could be applied to functions, too: def myfunc(myargs): """Documentation is hoopy" @def biguglydecorator(longconvolutedarglist) Someone is going to object that the evaluation time and environment of the decorator is screwy. But I don't care. :-) Greg From tdelaney at avaya.com Thu Mar 30 00:46:12 2006 From: tdelaney at avaya.com (Delaney, Timothy (Tim)) Date: Thu, 30 Mar 2006 09:46:12 +1100 Subject: [Python-Dev] pysqlite for 2.5? Message-ID: <2773CAC687FD5F4689F526998C7E4E5FF1E625@au3010avexu1.global.avaya.com> Anthony Baxter wrote: > My only concern about this is that it wouldn't be possible for other > authors to provide 3rd party packages as (for instance) db.mysqldb > because of the way package importing works. And I'd prefer > 'database.sqlite' rather than 'db.sqlite'. Perhaps dbapi2.sqlite? Tim Delaney From anthony at interlink.com.au Thu Mar 30 00:47:04 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu, 30 Mar 2006 09:47:04 +1100 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: References: <200603281759.18756.anthony@interlink.com.au> Message-ID: <200603300947.06439.anthony@interlink.com.au> On Thursday 30 March 2006 08:15, Fredrik Lundh wrote: > from a user perspective, adding this to the standard library is a > no-brainer. the only reason not to add it would be if the release > managers don't have time to sort out the build issues. Ok - well, I'm willing to work with Gerhard to do this work (for alpha2), Martin's willing to do the Windows build project - so I'm going to say "it's going to be in 2.5". I've really not seen any arguments that convince me otherwise. Martin: > Against: Adds work-load on the release process, adding more > libraries to the already-large list of new libraries for 2.5. Choice > of naming things is ad-hoc, but gets cast in stone by the release; > likewise, choice of specific SQL library might inhibit addition of > different libraries later. I'm happy to do the work (and you've said you're ok to do the windows part). All naming in the stdlib is adhoc by it's nature. We choose a name, and then that's it's name. I'm pretty happy with either 'db.sqlite' or 'database.sqlite', really. I don't think there's an alternative implementation of pysqlite bindings that could be considered for the stdlib. If an alternative to sqlite comes out some time, I don't have a problem with adding it. Phillip: > * Competing Python wrappers exist There's one - and it's not DB-API compliant. I know a lot of people who use the pysqlite wrapper, I've not come across anything that uses APSW. > * SQLite itself is updated frequently, let alone the wrappers > * Another external library to track and maybe have emergency updates > of Only an issue on platforms where we're not using the system-installed version. While sqlite gets new versions, very very few of these are security-related (I can't recall one lately) > * Build integration risks unknown, possible delay of 2.5? If it's going to cause a delay, it slips until 2.6. Easy. :) Skip: > I haven't been tracking the pysqlite discussion either, but one con > you missed is that regardless of pro #1 people will almost certainly > apply it to problems for which it is ill-suited, reflectly poorly on > both Python and SQLite. Of course, that can and does happen today. > Including pysqlite with Python just means it will happen more > frequently. Er - what? Right now, people are far more likely to use bsddb or anydbm for an inappropriate problem space. Adding a _better_ solution makes this better, not worse. I mean, adding ElementTree could also mean people will use XML in more places that are inappropriate, too, but I didn't see that raised as a problem. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From Scott.Daniels at Acm.Org Thu Mar 30 00:57:05 2006 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Wed, 29 Mar 2006 14:57:05 -0800 Subject: [Python-Dev] Discussing the Great Library Reorganization In-Reply-To: References: Message-ID: Brett Cannon wrote: > Here is a place I think we can take a queue from Java. I think we > should have a root package, 'py', and then have subpackages within > that.... At one point Tim Peters and I thought the right spot for python equivalents of C-coded modules belonged in package "py". Would 'std' do as well for the top level, or should we use "python" for the python-coded versions? -- -- Scott David Daniels Scott.Daniels at Acm.Org From fredrik at pythonware.com Thu Mar 30 01:00:04 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 30 Mar 2006 01:00:04 +0200 Subject: [Python-Dev] pysqlite for 2.5? References: <06Mar29.141411pst."58633"@synergy1.parc.xerox.com> Message-ID: Bill Janssen wrote: > On the package naming issue: using "em" for "email" would be wrong, > just as "db" for "database" would be wrong. are you aware of the fact that the module implements the "db-api" ? From pje at telecommunity.com Thu Mar 30 01:08:37 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 29 Mar 2006 18:08:37 -0500 Subject: [Python-Dev] Class decorators In-Reply-To: <442B0D71.2060101@canterbury.ac.nz> References: <200603290048.52669.fdrake@acm.org> <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <200603290048.52669.fdrake@acm.org> Message-ID: <5.1.1.6.0.20060329180627.04003120@mail.telecommunity.com> At 10:42 AM 3/30/2006 +1200, Greg Ewing wrote: >Fred L. Drake, Jr. wrote: > > > class Foo: > > """Documentation is good.""" > > > > @class implements(IFoo) > >That's an interesting idea. It could be applied to >functions, too: > > def myfunc(myargs): > """Documentation is hoopy" > @def biguglydecorator(longconvolutedarglist) -1; there should be only one obvious way to do it. Plus, @class is a noun and reads like it's saying something about the class; @def doesn't. It's too "verby". :) From python at discworld.dyndns.org Thu Mar 30 01:10:06 2006 From: python at discworld.dyndns.org (Charles Cazabon) Date: Wed, 29 Mar 2006 17:10:06 -0600 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <06Mar29.141411pst."58633"@synergy1.parc.xerox.com> References: <06Mar29.141411pst."58633"@synergy1.parc.xerox.com> Message-ID: <20060329231006.GA13930@discworld.dyndns.org> Bill Janssen wrote: > > On the package naming issue: using "em" for "email" would be wrong, Eh, that should be "import electronic_mail", then. And "import simple_mail_transport_protocol_lib". > just as "db" for "database" would be wrong. People who are familiar with Extensible Markup Language abbreviate it "xml". People who are familiar with electronic mail abbreviate it "email". And people who are familiar with the concept of a database abbreviate it "db". Why are two right for the stdlib (sorry, "standard library" :), and the other wrong? Charles -- ----------------------------------------------------------------------- Charles Cazabon GPL'ed software available at: http://pyropus.ca/software/ ----------------------------------------------------------------------- From anthony at interlink.com.au Thu Mar 30 01:21:57 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu, 30 Mar 2006 10:21:57 +1100 Subject: [Python-Dev] Discussing the Great Library Reorganization In-Reply-To: References: Message-ID: <200603301022.02352.anthony@interlink.com.au> On Thursday 30 March 2006 08:39, Brett Cannon wrote: > Here is a place I think we can take a queue from Java. I think we > should have a root package, 'py', and then have subpackages within > that. org.python.stdlib, surely? I don't have a problem with reorganising the standard library, but what's the motivation for moving everything under a new root? Is it just to allow people to unambigiously get hold of something from the stdlib, rather than following the normal search path? Doesn't the absolute/relative import PEP solve this problem? And what does 'from py import *' do, anyway? Anthony -- Anthony Baxter It's never too late to have a happy childhood. From guido at python.org Thu Mar 30 01:31:04 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 29 Mar 2006 15:31:04 -0800 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <200603300947.06439.anthony@interlink.com.au> References: <200603281759.18756.anthony@interlink.com.au> <200603300947.06439.anthony@interlink.com.au> Message-ID: On 3/29/06, Anthony Baxter wrote: > On Thursday 30 March 2006 08:15, Fredrik Lundh wrote: > > from a user perspective, adding this to the standard library is a > > no-brainer. the only reason not to add it would be if the release > > managers don't have time to sort out the build issues. > > Ok - well, I'm willing to work with Gerhard to do this work (for > alpha2), Martin's willing to do the Windows build project - so I'm > going to say "it's going to be in 2.5". I've really not seen any > arguments that convince me otherwise. Sounds like an excellent decision. Having it standard in the Windows build is actually going to be a big plus (though not for me personally :-). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Thu Mar 30 01:31:33 2006 From: brett at python.org (Brett Cannon) Date: Wed, 29 Mar 2006 15:31:33 -0800 Subject: [Python-Dev] Discussing the Great Library Reorganization In-Reply-To: <200603301022.02352.anthony@interlink.com.au> References: <200603301022.02352.anthony@interlink.com.au> Message-ID: On 3/29/06, Anthony Baxter wrote: > On Thursday 30 March 2006 08:39, Brett Cannon wrote: > > Here is a place I think we can take a queue from Java. I think we > > should have a root package, 'py', and then have subpackages within > > that. > > org.python.stdlib, surely? > > I don't have a problem with reorganising the standard library, but > what's the motivation for moving everything under a new root? Is it > just to allow people to unambigiously get hold of something from the > stdlib, rather than following the normal search path? Yes, it's to make it obvious the module came from the stdlib instead of another package. > Doesn't the > absolute/relative import PEP solve this problem? > Basically, but I think it wouldn't hurt to have a specific package name for the stdlib for in-code documenting instead of thinking that perhaps someone just stuck a module directly on sys.path . > And what does 'from py import *' do, anyway? Not much. =) It would import the top-level of a bunch of subpackages which will most likely not get you to a module, class, or function and thus couldn't be used to resolve to anything. -Brett From janssen at parc.com Thu Mar 30 01:32:21 2006 From: janssen at parc.com (Bill Janssen) Date: Wed, 29 Mar 2006 15:32:21 PST Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: Your message of "Wed, 29 Mar 2006 15:00:04 PST." Message-ID: <06Mar29.153222pst."58633"@synergy1.parc.xerox.com> Fredrik writes: > are you aware of the fact that the module implements the "db-api" ? "db-api" is just an earlier version of the same naming mistake. I'd be happy with "database_api" instead of database. Bill From greg.ewing at canterbury.ac.nz Thu Mar 30 01:34:46 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 30 Mar 2006 11:34:46 +1200 Subject: [Python-Dev] INPLACE_ADD and INPLACE_MULTIPLY oddities in ceval.c In-Reply-To: <20060329104540.GA26145@code0.codespeak.net> References: <4428C77E.5020005@canterbury.ac.nz> <4429D71F.1010601@canterbury.ac.nz> <20060329104540.GA26145@code0.codespeak.net> Message-ID: <442B1996.2040101@canterbury.ac.nz> Armin Rigo wrote: > So if we provide a complete fix, [].__add__(x) will be modified to > return NotImplemented instead of raising TypeError if x is not a list, > and then [1,2,3]+array([4,5,6]) will fall back to array.__radd__() as > before. Ah, okay. That seems like it would work. -- Greg From janssen at parc.com Thu Mar 30 01:36:34 2006 From: janssen at parc.com (Bill Janssen) Date: Wed, 29 Mar 2006 15:36:34 PST Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: Your message of "Wed, 29 Mar 2006 15:10:06 PST." <20060329231006.GA13930@discworld.dyndns.org> Message-ID: <06Mar29.153637pst."58633"@synergy1.parc.xerox.com> Charles Cabazon writes: > > On the package naming issue: using "em" for "email" would be wrong, > > Eh, that should be "import electronic_mail", then. And > "import simple_mail_transport_protocol_lib". > > > just as "db" for "database" would be wrong. > > People who are familiar with Extensible Markup Language abbreviate it "xml". > People who are familiar with electronic mail abbreviate it "email". And > people who are familiar with the concept of a database abbreviate it "db". > Why are two right for the stdlib (sorry, "standard library" :), and the other > wrong? Wow, what a world of black-and-white people we've got :-). Here's what I was thinking: "db" and "em" are too short to be useful context-free abbreviations, because there's too much chance of either conflicting with variable names in existing programs, or being confused with some other meaning for those two letters. "email" and "xml", on the other hand, have enough real-world emphasis to be suitable. "stdlib" might or might not be OK ("python_stdlib" might be better, or "pystdlib"); "py" surely isn't. Bill From brett at python.org Thu Mar 30 01:38:01 2006 From: brett at python.org (Brett Cannon) Date: Wed, 29 Mar 2006 15:38:01 -0800 Subject: [Python-Dev] warnings in libffi Message-ID: gcc 4.0.1 on OS X is spitting out some warnings about libffi: build/temp.darwin-8.5.0-Power_Macintosh-2.5/libffi/include/ffi.h:191: warning: function declaration isn't a prototype build/temp.darwin-8.5.0-Power_Macintosh-2.5/libffi/include/ffi.h:204: warning: function declaration isn't a prototype build/temp.darwin-8.5.0-Power_Macintosh-2.5/libffi/include/ffi.h:273: warning: function declaration isn't a prototype /Users/drifty/Code/Trees/svn/python/trunk/Modules/_ctypes/libffi/src/powerpc/ffi_darwin.c:383: warning: function declaration isn't a prototype /Users/drifty/Code/Trees/svn/python/trunk/Modules/_ctypes/libffi/src/powerpc/ffi_darwin.c:384: warning: function declaration isn't a prototype /Users/drifty/Code/Trees/svn/python/trunk/Modules/_ctypes/libffi/src/powerpc/ffi_darwin.c:388: warning: function declaration isn't a prototype /Users/drifty/Code/Trees/svn/python/trunk/Modules/_ctypes/libffi/src/powerpc/ffi_darwin.c:389: warning: function declaration isn't a prototype /Users/drifty/Code/Trees/svn/python/trunk/Modules/_ctypes/libffi/src/powerpc/ffi_darwin.c:394: warning: function declaration isn't a prototype All of them are for function parameters of function pointers (``void (*fn)(void)`` and such) when used in both function prototypes and function declarations. Do we fix these ourselves, or do we report them to the libffi maintainers (or are whom)? -Brett From anthony at interlink.com.au Thu Mar 30 01:39:01 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu, 30 Mar 2006 10:39:01 +1100 Subject: [Python-Dev] Discussing the Great Library Reorganization In-Reply-To: References: <200603301022.02352.anthony@interlink.com.au> Message-ID: <200603301039.04140.anthony@interlink.com.au> On Thursday 30 March 2006 10:31, Brett Cannon wrote: > > I don't have a problem with reorganising the standard library, > > but what's the motivation for moving everything under a new root? > > Is it just to allow people to unambigiously get hold of something > > from the stdlib, rather than following the normal search path? > > Yes, it's to make it obvious the module came from the stdlib > instead of another package. In that case, I don't see why this couldn't be added to import, rather than moving all the files around. > Basically, but I think it wouldn't hurt to have a specific package > name for the stdlib for in-code documenting instead of thinking > that perhaps someone just stuck a module directly on sys.path . I'm not convinced it buys us anything over just using the absolute import mechanism in the PEP. > > And what does 'from py import *' do, anyway? > > Not much. =) It would import the top-level of a bunch of > subpackages which will most likely not get you to a module, class, > or function and thus couldn't be used to resolve to anything. So you're saying that the toplevel of 'stdlib' wouldn't contain any real modules, but instead they'd be grouped under sub-packages? Good luck finding a home for everything... trying to categorise everything will be nearly impossible. And 'from stdlib.misc import foo' will make me very very unhappy. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From guido at python.org Thu Mar 30 01:40:56 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 29 Mar 2006 15:40:56 -0800 Subject: [Python-Dev] Discussing the Great Library Reorganization In-Reply-To: References: <200603301022.02352.anthony@interlink.com.au> Message-ID: On 3/29/06, Brett Cannon wrote: > On 3/29/06, Anthony Baxter wrote: > > On Thursday 30 March 2006 08:39, Brett Cannon wrote: > > > Here is a place I think we can take a queue from Java. I think we > > > should have a root package, 'py', and then have subpackages within > > > that. > > > > org.python.stdlib, surely? > > > > I don't have a problem with reorganising the standard library, but > > what's the motivation for moving everything under a new root? Is it > > just to allow people to unambigiously get hold of something from the > > stdlib, rather than following the normal search path? > > Yes, it's to make it obvious the module came from the stdlib instead > of another package. Dream on. The Java "standard" namespace is polluted with weirdnesses like "javax" (some kind of extensions) "org.xml", etc. > > Doesn't the > > absolute/relative import PEP solve this problem? > > Basically, but I think it wouldn't hurt to have a specific package > name for the stdlib for in-code documenting instead of thinking that > perhaps someone just stuck a module directly on sys.path . Actually it doesn't. > > And what does 'from py import *' do, anyway? > > Not much. =) It would import the top-level of a bunch of subpackages > which will most likely not get you to a module, class, or function and > thus couldn't be used to resolve to anything. I'd like to nip this discussion in the bud; it's just going to waste a lot of developers time. We need more people thinking seriously about the process and meta issues for Python 3000. (Yes, I know, I need to catch up with some threads myself. Hopefully next week when I'm no longer a single parent.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From rasky at develer.com Thu Mar 30 01:53:59 2006 From: rasky at develer.com (Giovanni Bajo) Date: Thu, 30 Mar 2006 01:53:59 +0200 Subject: [Python-Dev] Discussing the Great Library Reorganization References: <200603301022.02352.anthony@interlink.com.au> Message-ID: <09e601c6538c$0c646230$f44c2597@bagio> Anthony Baxter wrote: > I don't have a problem with reorganising the standard library, but > what's the motivation for moving everything under a new root? Is it > just to allow people to unambigiously get hold of something from the > stdlib, rather than following the normal search path? Doesn't the > absolute/relative import PEP solve this problem? I don't think so. For instance, if I have a package called "db" in my application (which I import with absolute imports from other packages), I might have problems with the newly added "db.sqlite" package in Python 2.5. In fact, I guess my "db" will shadow the stdlib one, making it impossible to access. An unique prefix for stdlib would solve this. Giovanni Bajo From greg.ewing at canterbury.ac.nz Thu Mar 30 02:00:46 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 30 Mar 2006 12:00:46 +1200 Subject: [Python-Dev] INPLACE_ADD and INPLACE_MULTIPLY oddities in ceval.c In-Reply-To: <442AAB86.4010008@ieee.org> References: <4428C77E.5020005@canterbury.ac.nz> <4429D71F.1010601@canterbury.ac.nz> <20060329104540.GA26145@code0.codespeak.net> <442AAB86.4010008@ieee.org> Message-ID: <442B1FAE.2010003@canterbury.ac.nz> Tim Hochberg wrote: > Still, perhaps for Py3K it's worth considering > if PyNumber_InplaceAdd should only call __iadd__ and __add__, not > __radd__. Thus giving the target object complete control during inplace > adds. That's probably reasonable, although it would break the conceptual notion that a += b is equivalent to a = a + b when a can't be modified in-place. Greg From pje at telecommunity.com Thu Mar 30 02:23:03 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 29 Mar 2006 19:23:03 -0500 Subject: [Python-Dev] Class decorators In-Reply-To: References: <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com> <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> <200603282335.09341.fdrake@acm.org> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060329142653.037617a8@mail.telecommunity.com> At 11:07 AM 3/29/2006 -0800, Guido van Rossum wrote: >On 3/28/06, Phillip J. Eby wrote: > > If we're using Zope 3 as an example, I personally find that: > > > > class Foo: > > """Docstring here, blah blah blah > > """ > > implements(IFoo) > > > > is easier to read than: > > > > @implements(IFoo) > > class Foo: > > """Docstring here, blah blah blah > > """ > >But the former also smells more of magic. My comment above was only about readable *placement* of the decorators, not the actual syntax. Many approaches to the actual syntax in the body are possible. For example, what did you think of Fred Drakes's "@class" proposal? To specify it formally, one could say that this: @class EXPR in a class scope would expand to the equivalent of: locals().setdefault('__decorators__',[]).append(EXPR) and is a syntax error if placed anywhere else. That, combined with support for processing __decorators__ at class creation time, would fulfill the desired semantics without any implicit "magic". (The locals() part could of course be implemented in bytecode as LOAD_LOCALS, since class scopes implement their locals as a dictionary. That would avoid the need for adding any new bytecodes, since this isn't a performance-sensitive feature.) From barry at python.org Thu Mar 30 02:31:42 2006 From: barry at python.org (Barry Warsaw) Date: Wed, 29 Mar 2006 19:31:42 -0500 Subject: [Python-Dev] PySet API In-Reply-To: <000201c6532c$33e4b1d0$a341a8c0@RaymondLaptop1> References: <1142705253.32074.72.camel@geddy.wooz.org> <005c01c64ac5$589859b0$b83efea9@RaymondLaptop1> <002b01c64aeb$44eb7750$b83efea9@RaymondLaptop1> <1142824236.10599.29.camel@resist.wooz.org> <00a001c64bfa$71a0ab20$b83efea9@RaymondLaptop1> <1142987079.10599.232.camel@resist.wooz.org> <004001c64d58$b0ebebf0$b83efea9@RaymondLaptop1> <1143327815.10799.42.camel@resist.wooz.org> <001a01c65082$35a4c840$b83efea9@RaymondLaptop1> <1143516346.2317.80.camel@geddy.wooz.org> <4428C999.5010801@canterbury.ac.nz> <1143576767.10792.115.camel@resist.wooz.org> <000201c6532c$33e4b1d0$a341a8c0@RaymondLaptop1> Message-ID: <1143678702.7204.48.camel@resist.wooz.org> On Tue, 2006-03-28 at 22:20 -0800, Raymond Hettinger wrote: > Barry, go ahead with PySet_Clear(). Cool thanks. I think we've also compromised on _PySet_Next(), correct? I'll follow up on PySet_Update() in a moment. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060329/22ee10a5/attachment-0001.pgp From barry at python.org Thu Mar 30 02:34:09 2006 From: barry at python.org (Barry Warsaw) Date: Wed, 29 Mar 2006 19:34:09 -0500 Subject: [Python-Dev] PySet API In-Reply-To: <001a01c65377$cd099a10$6601a8c0@RaymondLaptop1> References: <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> <1143468446.10799.50.camel@resist.wooz.org> <200603281507.18175.gmccaughan@synaptics-uk.com> <001a01c65377$cd099a10$6601a8c0@RaymondLaptop1> Message-ID: <1143678849.6694.52.camel@resist.wooz.org> On Wed, 2006-03-29 at 16:29 -0500, Raymond Hettinger wrote: > The story is different for PySet_Update(). Defining it now could get in the way > of possible future development for the module (the function may end-up taking a > variable length argument list instead of a single argument). So why not just go ahead and do that now? If you know that's what you want eventually, why wait? From my perspective, adding a NULL at the end of the argument list wouldn't be that big of a deal. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060329/a60a0799/attachment.pgp From barry at python.org Thu Mar 30 02:36:43 2006 From: barry at python.org (Barry Warsaw) Date: Wed, 29 Mar 2006 19:36:43 -0500 Subject: [Python-Dev] PySet API In-Reply-To: <1143678849.6694.52.camel@resist.wooz.org> References: <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> <1143468446.10799.50.camel@resist.wooz.org> <200603281507.18175.gmccaughan@synaptics-uk.com> <001a01c65377$cd099a10$6601a8c0@RaymondLaptop1> <1143678849.6694.52.camel@resist.wooz.org> Message-ID: <1143679003.6687.56.camel@resist.wooz.org> On Wed, 2006-03-29 at 19:34 -0500, Barry Warsaw wrote: > On Wed, 2006-03-29 at 16:29 -0500, Raymond Hettinger wrote: > > > The story is different for PySet_Update(). Defining it now could get in the way > > of possible future development for the module (the function may end-up taking a > > variable length argument list instead of a single argument). > > So why not just go ahead and do that now? If you know that's what you > want eventually, why wait? From my perspective, adding a NULL at the > end of the argument list wouldn't be that big of a deal. BTW, I'm willing to do the work on this. I'm already going to update my patch anyway to reflect our current decisions, so I'm happy to do this while I'm at it. I'll try to get a new patch posted in a day or so. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060329/ef57911a/attachment.pgp From janssen at parc.com Thu Mar 30 02:42:43 2006 From: janssen at parc.com (Bill Janssen) Date: Wed, 29 Mar 2006 16:42:43 PST Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: Your message of "Wed, 29 Mar 2006 15:36:34 PST." <06Mar29.153637pst."58633"@synergy1.parc.xerox.com> Message-ID: <06Mar29.164252pst."58633"@synergy1.parc.xerox.com> > Charles Cabazon writes: Whoops! Should be "Cazabon". Bill From aleaxit at gmail.com Thu Mar 30 02:45:06 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Wed, 29 Mar 2006 16:45:06 -0800 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <79990c6b0603291441i2f74882cw3574565c1b188357@mail.gmail.com> References: <200603281759.18756.anthony@interlink.com.au> <1143554351.2309.158.camel@geddy.wooz.org> <200603290152.00598.anthony@interlink.com.au> <79990c6b0603291441i2f74882cw3574565c1b188357@mail.gmail.com> Message-ID: On 3/29/06, Paul Moore wrote: > On 3/29/06, Fredrik Lundh wrote: > > > from a user perspective, adding this to the standard library is a no-brainer. > > the only reason not to add it would be if the release managers don't have > > time to sort out the build issues. > > Agreed. As a SQL user, it feels like a no-brainer to me, as well. I'd > love to see a DB-API compliant module, and an embedded SQL database, > in the core. I don't believe that there's anything comparable to > (py)sqlite on that score. > > And as Anthony pointed out, it provides a nice upgrade path to > "bigger" SQL/DB-API solutions. > > +1 from me. +1 here, too; just as /f said, the only real issue is time (as in, is it too late to get such a package into 2.5), but with the volunteering I've seen that should be feasible. I just hope we get a BDFL +1 soon so I can add the good news to the 2nd ed of "Python in a Nutshell", which is due to go "printers-wards" about 8-9 days from now (in order to be ready for OSCON -- the times from final text version to paper are NOT brief:-). Alex From jack at performancedrivers.com Thu Mar 30 03:00:59 2006 From: jack at performancedrivers.com (Jack Diederich) Date: Wed, 29 Mar 2006 20:00:59 -0500 Subject: [Python-Dev] Class decorators In-Reply-To: <5.1.1.6.0.20060329142653.037617a8@mail.telecommunity.com> References: <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com> <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> <200603282335.09341.fdrake@acm.org> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <5.1.1.6.0.20060329142653.037617a8@mail.telecommunity.com> Message-ID: <20060330010059.GH5787@performancedrivers.com> On Wed, Mar 29, 2006 at 07:23:03PM -0500, Phillip J. Eby wrote: > At 11:07 AM 3/29/2006 -0800, Guido van Rossum wrote: > >On 3/28/06, Phillip J. Eby wrote: > > > If we're using Zope 3 as an example, I personally find that: > > > > > > class Foo: > > > """Docstring here, blah blah blah > > > """ > > > implements(IFoo) > > > > > > is easier to read than: > > > > > > @implements(IFoo) > > > class Foo: > > > """Docstring here, blah blah blah > > > """ > > > >But the former also smells more of magic. > > My comment above was only about readable *placement* of the decorators, not > the actual syntax. Many approaches to the actual syntax in the body are > possible. > > For example, what did you think of Fred Drakes's "@class" proposal? To > specify it formally, one could say that this: > > @class EXPR > > in a class scope would expand to the equivalent of: > > locals().setdefault('__decorators__',[]).append(EXPR) > > and is a syntax error if placed anywhere else. That, combined with support > for processing __decorators__ at class creation time, would fulfill the > desired semantics without any implicit "magic". > A function decorator takes a function as an argument and returns something (probably a function and maybe even the very same function). This is exactly what class decorators should do or we should call them something else and give them a distinct syntax. A function decorator is there to replace code like: def myfunc(a, b, c): # half a screen of code myfunc = mangle(myfunc) Likewise class decorators would save me from typing class MyClass: # many functions taking half a screen of code each register(MyClass, db_id=20) I used to do this with metaclasses but stopped because it requires making 'db_id' a member of the class which is magically noticed by a metaclass two files away. Using metaclasses also required gross hacks like checking for a 'DO_NOT_REGISTER' member for subclasses that wanted to inherit from a class that had a Register metaclass but didn't want to be registered. Yuck. If you want to do lots of Zopeish stuff mostly inside the class write a decorator that looks for it in the class body. @zope.find_magic_in_attr('mymagic') class MyClass: mymagic = [] # some long hairy thing Talking about something other than a decorator or proposing all new syntax is just going to get this pronounced out of existence. If-I-wanted-zope-I'd-use-zope-ly, -jackdied From skip at pobox.com Thu Mar 30 03:07:06 2006 From: skip at pobox.com (skip at pobox.com) Date: Wed, 29 Mar 2006 19:07:06 -0600 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: References: <200603290152.00598.anthony@interlink.com.au> <200603281759.18756.anthony@interlink.com.au> <1143554351.2309.158.camel@geddy.wooz.org> <5.1.1.6.0.20060329144142.0390e5d8@mail.telecommunity.com> <17451.882.842146.979286@montanaro.dyndns.org> Message-ID: <17451.12090.257751.636336@montanaro.dyndns.org> >> I haven't been tracking the pysqlite discussion either, but one con >> you missed is that regardless of pro #1 people will almost certainly >> apply it to problems for which it is ill-suited, reflectly poorly on >> both Python and SQLite. Fredrik> the arguments keep getting more and more weird. Fredrik> is there *any* part of the standard Python distribution that Fredrik> cannot be applied to problems for which it is ill-suited? To many people "SQL" in the name implies "big databases". I know from personal experience at work. The powers-that-be didn't want to support another database server (we already have Sybase) and didn't want our group's experimental data "polluting" the production database, so the folks who wanted it went the SQLite/pysqlite route. They were immediately bitten by the multiple reader/single writer limitation and they tried to cram too much data into it, so performance further sucked. Skip From skip at pobox.com Thu Mar 30 03:15:40 2006 From: skip at pobox.com (skip at pobox.com) Date: Wed, 29 Mar 2006 19:15:40 -0600 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <5.1.1.6.0.20060329172917.041345d8@mail.telecommunity.com> References: <5.1.1.6.0.20060329144142.0390e5d8@mail.telecommunity.com> <200603290152.00598.anthony@interlink.com.au> <200603281759.18756.anthony@interlink.com.au> <1143554351.2309.158.camel@geddy.wooz.org> <5.1.1.6.0.20060329172917.041345d8@mail.telecommunity.com> Message-ID: <17451.12604.993926.654756@montanaro.dyndns.org> >> Is it not possible to distribute an empty db package which is then >> populated with various database eggs (or other similar installation >> things)? Phillip> I don't think I understand your question. Someone was throwing around names like db.sqlite as the place to install pysqlite. That suggests other database interface modules like db.mysql, db.postgresql, db.sybase, etc. Given that we probably won't include all those as standard modules, we should make it easy for someone to install one or more of those modules via normal external mechanisms and have them appear seamlessly to the Python programmer. Then I begin to wonder why bother with db.sqlite at all. Why not just create an empty db package that does the pkgutil or pkg_resources dance and let people install all N database interfaces instead of just N-1? Skip From skip at pobox.com Thu Mar 30 03:17:40 2006 From: skip at pobox.com (skip at pobox.com) Date: Wed, 29 Mar 2006 19:17:40 -0600 Subject: [Python-Dev] Discussing the Great Library Reorganization In-Reply-To: References: Message-ID: <17451.12724.644388.340734@montanaro.dyndns.org> Brett> While this is going to require a PEP (which I am willing to Brett> write), the discussion of adding pysqlite has brought forth some Brett> discussion on naming and packaging in the stdlub. Perhaps it's Brett> time to start discussing the Great Library Reorganization that Brett> has been discussed for eons. I think this belongs on the py3k list. Stdlib reorganization should be done with Py3k in mind, then backported to 2.x if deemed worthwhile and doable. Skip From skip at pobox.com Thu Mar 30 03:32:31 2006 From: skip at pobox.com (skip at pobox.com) Date: Wed, 29 Mar 2006 19:32:31 -0600 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <200603300947.06439.anthony@interlink.com.au> References: <200603281759.18756.anthony@interlink.com.au> <200603300947.06439.anthony@interlink.com.au> Message-ID: <17451.13615.74425.528082@montanaro.dyndns.org> Anthony> All naming in the stdlib is adhoc by it's nature. We choose a Anthony> name, and then that's it's name. I'm pretty happy with either Anthony> 'db.sqlite' or 'database.sqlite', really. Let's slow down here. If we are really going to start putting together a package infrastructure shouldn't it be done in the context of Brett's great stdlib restructuring PEP? If everything included in the stdlib is under some core package (core, stdlib, py, etc), that suggests that a package containing database wrappers would be named something like stdlib.db or py.database. The existence of a stdlib package certainly isn't a given. Still, let's assume for the moment that's how things fall out and we deliver pysqlite in the stdlib as stdlib.db.sqlite. If I want to install Object Craft's Sybase wrapper the logical place for it seems like stdlib.db.sybase. But that's not right because the Sybase module's not part of the stdlib. Okay, it belongs in site.db.sybase. But now we have two different db packages and the programmer has to care about stdlib vs external. (Maybe that's okay.) I'm sure there are a number of different workable solutions, but shouldn't they be considered before rather arbitrarily deciding that pysqlite belongs in db.sqlite or database.sqlite? If you're not willing to resolve the future naming scheme for 3.x or 2.6 at this point I see no reason start adding any package hierarchy(ies). Just toss it in as the sqlite module (no packaging) and be done with it. Don't start carving up the package namespace without considering the bigger picture. Skip From greg.ewing at canterbury.ac.nz Thu Mar 30 03:38:54 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 30 Mar 2006 13:38:54 +1200 Subject: [Python-Dev] Name for python package repository Message-ID: <442B36AE.4040508@canterbury.ac.nz> I just thought of a possible name for the Python package repository. We could call it the PIPE - Python Index of Packages and Extensions. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From pje at telecommunity.com Thu Mar 30 03:43:41 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 29 Mar 2006 20:43:41 -0500 Subject: [Python-Dev] Class decorators In-Reply-To: <20060330010059.GH5787@performancedrivers.com> References: <5.1.1.6.0.20060329142653.037617a8@mail.telecommunity.com> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com> <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> <200603282335.09341.fdrake@acm.org> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <5.1.1.6.0.20060329142653.037617a8@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060329201639.01e43970@mail.telecommunity.com> At 08:00 PM 3/29/2006 -0500, Jack Diederich wrote: >A function decorator takes a function as an argument and returns something >(probably a function and maybe even the very same function). So would class decorators. >This is exactly what class decorators should do or we should call them >something else and give them a distinct syntax. Yep. >A function decorator is there to replace code like: > >def myfunc(a, b, c): > # half a screen of code >myfunc = mangle(myfunc) > >Likewise class decorators would save me from typing > >class MyClass: > # many functions taking half a screen of code each >register(MyClass, db_id=20) Yep. >Talking about something other than a decorator You lost me there. Nobody's argued (AFAIK) for class decorators being anything other than single-argument functions that take a class as input and return a class as output. > or proposing all new >syntax is just going to get this pronounced out of existence. Which wouldn't be a bad thing, IMO. There's no point in adding them if they're not an actual improvement over what we can do now. Comments about the hackishness of the implementation used by Zope and PEAK are also off-base; nobody proposed that everyone should go off and implement their own such hacks! Meanwhile, inclusion of such a facility in the stdlib isn't without precedent; there are IIRC at least *6* sys._getframe hacks in the stdlib already. Nothing stops us from adding a 'decorate_class' function to the stdlib that's used like this, for example: class MyClass: decorate_class( register(db_id=20), implements(IFoo), ... ) or to just make the class machinery interpret a __decorators__ attribute (a variant on an old suggestion of Guido's): class MyClass: __decorators__ = [register(db_id=20), implements(IFoo)] The only *implementation* reason to give this special syntax, IIUC, is to allow implementations like Jython and IronPython and Pyrex to statically recognize certain decorators at compile-time. There are of course also non-implementation reasons to have class decorator syntax (such as EIBTI), and I agree with that. But readability also counts, and the readability of @decorators on the outside of a class tends to suck as the number of decorators and arguments increases. What's more, I haven't seen anybody posting any counterexamples to show that it doesn't suck for common use cases. Indeed, at the moment I don't even recall seeing any examples of class decorators being used without arguments! I also suspect that any actual Jython/IronPython examples are likely to be at least as verbose as Zope and PEAK's, and probably more likely to include multiple decorators. (Depending on how Java annotations and .Net attribs would be translated to decorators.) So, I'm personally not in favor of adding class decorators with a syntax that blindly imitates that of function decorators, without a proper examination of the use cases. This is precisely the argument that Guido used to veto class decorators in 2.4, and nothing about the issue has changed since then, except for the subject being reopened to consideration. From greg.ewing at canterbury.ac.nz Thu Mar 30 04:21:29 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 30 Mar 2006 14:21:29 +1200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <06Mar29.153637pst.58633@synergy1.parc.xerox.com> References: <06Mar29.153637pst.58633@synergy1.parc.xerox.com> Message-ID: <442B40A9.3090204@canterbury.ac.nz> Bill Janssen wrote: > "db" and "em" are too short to be useful context-free abbreviations, There's a big difference between "db" and "em": "db" is an extremely well-known abbreviation, whereas "em" isn't. At the top level of a reorganised package namespace, I don't think it would be out of place to reserve "db" for database stuff. It can always be renamed on import if it happens to conflict with anything in code, and I wouldn't object to not being able to have my own top-level package called "db". -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From skip at pobox.com Thu Mar 30 04:26:10 2006 From: skip at pobox.com (skip at pobox.com) Date: Wed, 29 Mar 2006 20:26:10 -0600 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <442B40A9.3090204@canterbury.ac.nz> References: <06Mar29.153637pst.58633@synergy1.parc.xerox.com> <442B40A9.3090204@canterbury.ac.nz> Message-ID: <17451.16834.487862.108957@montanaro.dyndns.org> Greg> There's a big difference between "db" and "em": "db" is an Greg> extremely well-known abbreviation, whereas "em" isn't. Unless you're a typesetter or a TeX hound... :-) Skip From greg.ewing at canterbury.ac.nz Thu Mar 30 04:27:59 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 30 Mar 2006 14:27:59 +1200 Subject: [Python-Dev] Class decorators In-Reply-To: <5.1.1.6.0.20060329142653.037617a8@mail.telecommunity.com> References: <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com> <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> <200603282335.09341.fdrake@acm.org> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <5.1.1.6.0.20060329142653.037617a8@mail.telecommunity.com> Message-ID: <442B422F.4030407@canterbury.ac.nz> Phillip J. Eby wrote: > My comment above was only about readable *placement* of the decorators, not > the actual syntax. The placement is part of the syntax... -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Thu Mar 30 04:32:02 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 30 Mar 2006 14:32:02 +1200 Subject: [Python-Dev] PySet API In-Reply-To: <1143678849.6694.52.camel@resist.wooz.org> References: <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> <1143468446.10799.50.camel@resist.wooz.org> <200603281507.18175.gmccaughan@synaptics-uk.com> <001a01c65377$cd099a10$6601a8c0@RaymondLaptop1> <1143678849.6694.52.camel@resist.wooz.org> Message-ID: <442B4322.8010206@canterbury.ac.nz> Barry Warsaw wrote: > On Wed, 2006-03-29 at 16:29 -0500, Raymond Hettinger wrote: > >>The story is different for PySet_Update(). Defining it now could get in the way >>of possible future development for the module (the function may end-up taking a >>variable length argument list instead of a single argument). Would that really buy you anything much over just making multiple PySet_Update() calls? Is it just syntactic sugar, or is there some optimisation you can do with multiple updates presented all at once? -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Thu Mar 30 04:42:22 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 30 Mar 2006 14:42:22 +1200 Subject: [Python-Dev] Class decorators In-Reply-To: <20060330010059.GH5787@performancedrivers.com> References: <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com> <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> <200603282335.09341.fdrake@acm.org> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <5.1.1.6.0.20060329142653.037617a8@mail.telecommunity.com> <20060330010059.GH5787@performancedrivers.com> Message-ID: <442B458E.6020302@canterbury.ac.nz> Jack Diederich wrote: > Using metaclasses also required gross hacks like checking > for a 'DO_NOT_REGISTER' member for subclasses that wanted to inherit from > a class that had a Register metaclass but didn't want to be registered. I've just done something like this myself in the last few days, and my solution was to have the registration metaclass look for an attribute in the class's __dict__, instead of using attribute access on the class, to decide whether to register it. That way the registerability doesn't get inherited, and I get to choose individually for each class. This of course may not be such a good idea if you want registerability to *usually* be inherited, but not always. In that case, you need some way of saying when you don't want it, in which case a do-not-register attribute doesn't seem so bad. How else would you provide this information? Still want class decorators, though. :-) BTW, the system I'm working on is a response to some hair-tearing problems I was experiencing trying to pickle some of my data, and also to address some disadvantages I see in using pickle for long-term data storage. I'll write something up about it if it works out... -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Thu Mar 30 04:47:50 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 30 Mar 2006 14:47:50 +1200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <17451.12090.257751.636336@montanaro.dyndns.org> References: <200603290152.00598.anthony@interlink.com.au> <200603281759.18756.anthony@interlink.com.au> <1143554351.2309.158.camel@geddy.wooz.org> <5.1.1.6.0.20060329144142.0390e5d8@mail.telecommunity.com> <17451.882.842146.979286@montanaro.dyndns.org> <17451.12090.257751.636336@montanaro.dyndns.org> Message-ID: <442B46D6.70703@canterbury.ac.nz> skip at pobox.com wrote: > The powers-that-be didn't want to support > another database server (we already have Sybase) and didn't want our group's > experimental data "polluting" the production database, so the folks who > wanted it went the SQLite/pysqlite route. They were immediately bitten by > the multiple reader/single writer limitation and they tried to cram too much > data into it, so performance further sucked. Firebird could be a solution to this. It can be used in a mode that doesn't need a server, and it has no trouble at all with concurrency or large amounts of data that I know of. In fact, a Firebird interface might be an alternative worth considering for the library. It would have most of the advantages of SQLite without these disadvantages. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Thu Mar 30 04:55:42 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 30 Mar 2006 14:55:42 +1200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <17451.13615.74425.528082@montanaro.dyndns.org> References: <200603281759.18756.anthony@interlink.com.au> <200603300947.06439.anthony@interlink.com.au> <17451.13615.74425.528082@montanaro.dyndns.org> Message-ID: <442B48AE.6010105@canterbury.ac.nz> skip at pobox.com wrote: > If I want to install Object Craft's Sybase wrapper the > logical place for it seems like stdlib.db.sybase. But that's not right > because the Sybase module's not part of the stdlib. Okay, it belongs in > site.db.sybase. But now we have two different db packages and the > programmer has to care about stdlib vs external. (Maybe that's okay.) This seems to be an instance of the general problem of trying to fit a multidimensional classification into a hierarchical structure. Database systems solved this long ago by ditching the hierarchy completely and going relational. Maybe a single dotted hierarchy of package names is too restrictive? Should we be able to import things by specifying attributes instead of a pathname? import db where db.stdlib == True and db.language == "SQL" \ and db.interface == "DBAPI2.0" ?-) -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From fdrake at acm.org Thu Mar 30 05:07:31 2006 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Wed, 29 Mar 2006 22:07:31 -0500 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <442B48AE.6010105@canterbury.ac.nz> References: <200603281759.18756.anthony@interlink.com.au> <17451.13615.74425.528082@montanaro.dyndns.org> <442B48AE.6010105@canterbury.ac.nz> Message-ID: <200603292207.31378.fdrake@acm.org> On Wednesday 29 March 2006 21:55, Greg Ewing wrote: > import db where db.stdlib == True and db.language == "SQL" \ > and db.interface == "DBAPI2.0" While we're at it, we could spell import "select". :-) -Fred -- Fred L. Drake, Jr. From anthony at interlink.com.au Thu Mar 30 05:08:47 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu, 30 Mar 2006 14:08:47 +1100 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <17451.12090.257751.636336@montanaro.dyndns.org> References: <200603290152.00598.anthony@interlink.com.au> <17451.12090.257751.636336@montanaro.dyndns.org> Message-ID: <200603301408.49965.anthony@interlink.com.au> On Thursday 30 March 2006 12:07, skip at pobox.com wrote: > To many people "SQL" in the name implies "big databases". I know > from personal experience at work. The powers-that-be didn't want > to support another database server (we already have Sybase) and > didn't want our group's experimental data "polluting" the > production database, so the folks who wanted it went the > SQLite/pysqlite route. They were immediately bitten by the > multiple reader/single writer limitation and they tried to cram too > much data into it, so performance further sucked. And people trying to build production systems on SimpleHTTPServer, SimpleXMLRPCServer, smptd, or dumbdbm will also find their performance sucks. What's your point? Anthony -- Anthony Baxter It's never too late to have a happy childhood. From greg.ewing at canterbury.ac.nz Thu Mar 30 05:09:54 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 30 Mar 2006 15:09:54 +1200 Subject: [Python-Dev] Class decorators In-Reply-To: <5.1.1.6.0.20060329201639.01e43970@mail.telecommunity.com> References: <5.1.1.6.0.20060329142653.037617a8@mail.telecommunity.com> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com> <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> <200603282335.09341.fdrake@acm.org> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <5.1.1.6.0.20060329142653.037617a8@mail.telecommunity.com> <5.1.1.6.0.20060329201639.01e43970@mail.telecommunity.com> Message-ID: <442B4C02.2020708@canterbury.ac.nz> Phillip J. Eby wrote: > the readability of @decorators on the outside of a class tends > to suck as the number of decorators and arguments increases. So do decorators outside a function. > What's more, I haven't seen anybody posting any counterexamples to show > that it doesn't suck for common use cases. Indeed, at the moment I don't > even recall seeing any examples of class decorators being used without > arguments! Well, here's how my use case would look if I had class decorators: @IOClass class MyClass: ... Does that count? My decorator wouldn't need any arguments, because it looks inside the class for all the information it needs. [1] That's actually a general solution to Phillip's concern: the decorator can always look for attributes in the class (or the class's __dict__ if you don't want them inherited) for large amounts of information that wouldn't comfortably fit up the top. That's an extra degree of freedom that we don't have with functions. - - - - - [1] Actually I would probably give it one optional argument, the name to register under if different from the class name. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From anthony at interlink.com.au Thu Mar 30 05:11:24 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu, 30 Mar 2006 14:11:24 +1100 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <17451.12604.993926.654756@montanaro.dyndns.org> References: <5.1.1.6.0.20060329144142.0390e5d8@mail.telecommunity.com> <5.1.1.6.0.20060329172917.041345d8@mail.telecommunity.com> <17451.12604.993926.654756@montanaro.dyndns.org> Message-ID: <200603301411.27471.anthony@interlink.com.au> On Thursday 30 March 2006 12:15, skip at pobox.com wrote: > Someone was throwing around names like db.sqlite as the place to > install pysqlite. Dunno who originally suggested it, but the theory was that there's some issue with toplevel library namespace pollution. I'm not too stressed out one way or the other - but starting off with 'db.sqlite' (and then maybe moving/aliasing db.bsddb &c at a future point) doesn't seem like a bad idea. > That suggests other database interface modules > like db.mysql, db.postgresql, db.sybase, etc. Given that we > probably won't include all those as standard modules, we should > make it easy for someone to install one or more of those modules > via normal external mechanisms and have them appear seamlessly to > the Python programmer. Then I begin to wonder why bother with > db.sqlite at all. Why not just create an empty db package that > does the pkgutil or pkg_resources dance and let people install all > N database interfaces instead of just N-1? The same could be said of vast amounts of the standard library. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From greg.ewing at canterbury.ac.nz Thu Mar 30 05:12:33 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 30 Mar 2006 15:12:33 +1200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <17451.16834.487862.108957@montanaro.dyndns.org> References: <06Mar29.153637pst.58633@synergy1.parc.xerox.com> <442B40A9.3090204@canterbury.ac.nz> <17451.16834.487862.108957@montanaro.dyndns.org> Message-ID: <442B4CA1.9070503@canterbury.ac.nz> skip at pobox.com wrote: > Greg> There's a big difference between "db" and "em": "db" is an > Greg> extremely well-known abbreviation, whereas "em" isn't. > > Unless you're a typesetter or a TeX hound... :-) Good point! Still, the fact remains that it's not a well-known abbreviation for *email*. :-) -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Thu Mar 30 05:21:36 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 30 Mar 2006 15:21:36 +1200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <200603292207.31378.fdrake@acm.org> References: <200603281759.18756.anthony@interlink.com.au> <17451.13615.74425.528082@montanaro.dyndns.org> <442B48AE.6010105@canterbury.ac.nz> <200603292207.31378.fdrake@acm.org> Message-ID: <442B4EC0.8050809@canterbury.ac.nz> Fred L. Drake, Jr. wrote: > On Wednesday 29 March 2006 21:55, Greg Ewing wrote: > > import db where db.stdlib == True and db.language == "SQL" \ > > and db.interface == "DBAPI2.0" > > While we're at it, we could spell import "select". :-) Getting off on a tangent here, but I would actually like some decent way of writing SQL queries in Python -- not for importing, but for database access. Constructing bits of SQL out of character strings sucks *extremely* badly. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From aleax at mac.com Mon Mar 27 18:17:24 2006 From: aleax at mac.com (Alex Martelli) Date: Mon, 27 Mar 2006 08:17:24 -0800 Subject: [Python-Dev] PySet API In-Reply-To: <003701c651b1$ff0c8670$1e2810ac@RaymondLaptop1> References: <006401c650eb$55e140c0$b83efea9@RaymondLaptop1> <20060326160728.GA6403@panix.com> <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> <1143468446.10799.50.camel@resist.wooz.org> <003701c651b1$ff0c8670$1e2810ac@RaymondLaptop1> Message-ID: <6EFE2418-2EEA-42DF-9E89-9886BEF71C24@mac.com> On Mar 27, 2006, at 7:20 AM, Raymond Hettinger wrote: > Why don't we expose _PySet_Next() for Barry and leave it out of the > public API > for everyone else. There are precedents for adding some functionality to the C API but not documenting it to ensure "non advanced users" don't get hurt -- that's how we added the ability to raise exceptions in different threads, in particular. Not sure if this is the best solution here, but I'm just pointing out that it's definitely not "unthinkable", procedurally speaking. Alex From tds333+2006 at gmail.com Wed Mar 29 17:24:32 2006 From: tds333+2006 at gmail.com (Wolfgang Langner) Date: Wed, 29 Mar 2006 17:24:32 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <442A8D34.8060200@ghaering.de> References: <200603281759.18756.anthony@interlink.com.au> <200603291947.34461.anthony@interlink.com.au> <1143638252.2317.184.camel@geddy.wooz.org> <442A8D34.8060200@ghaering.de> Message-ID: <4c45c1530603290724g12f656d6j295841be2cf78f1a@mail.gmail.com> Hello, > I think short names are more more consistent with the existing naming in > the standard library. > > +1 on db.sqlite from me. same for me +1 on db.sqlite > db.sql.sqlite is another possibility, if adding something like Durus or > ZODB in the same top-level namespace could be considered for 2.6. -1 on db.sql.sqlite. Keep structure flat. Or we are eventually in a Java world with org.something.this.andthat .... -- bye by Wolfgang From tim.hochberg at ieee.org Wed Mar 29 17:45:10 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Wed, 29 Mar 2006 08:45:10 -0700 Subject: [Python-Dev] INPLACE_ADD and INPLACE_MULTIPLY oddities in ceval.c In-Reply-To: <20060329104540.GA26145@code0.codespeak.net> References: <4428C77E.5020005@canterbury.ac.nz> <4429D71F.1010601@canterbury.ac.nz> <20060329104540.GA26145@code0.codespeak.net> Message-ID: <442AAB86.4010008@ieee.org> Armin Rigo wrote: > Hi Greg, > > On Wed, Mar 29, 2006 at 12:38:55PM +1200, Greg Ewing wrote: > >>I'm really thinking more about the non-inplace operators. >>If nb_add and sq_concat are collapsed into a single slot, >>it seems to me that if you do >> >> a = [1, 2, 3] >> b = array([4, 5, 6]) >> c = a + b >> >>then a will be asked "Please add yourself to b", and a >>will say "Okay, I know how to do that!" and promptly >>concatenate itself with b. > > > No: there is a difference between + and += for lists. You can only > concatenate exactly a list to a list. Indeed: > > >>> [].__add__((2, 3)) > TypeError: can only concatenate list (not "tuple") to list > > By contrast, list += is like extend() and accepts any iterable. > So if we provide a complete fix, [].__add__(x) will be modified to > return NotImplemented instead of raising TypeError if x is not a list, > and then [1,2,3]+array([4,5,6]) will fall back to array.__radd__() as > before. Ouch. Assuming the same path is followed with tuples, I think that this means the following behaviour will continue: >>> t = (1,2,3) >>> a = array([4,5,6]) >>> t += a >>> t array([5, 7, 9]) That's not particularly desirable. There's not much to be done about it short of adding __iadd__s everywhere, which is probably brittle and unfriendly. And, admittedly this is a corner case that's very rarely going to cause trouble. Still, perhaps for Py3K it's worth considering if PyNumber_InplaceAdd should only call __iadd__ and __add__, not __radd__. Thus giving the target object complete control during inplace adds. Similarly for other inplace operations, of course. I'm not certain that all of the consequences of this change would be benign, but it's something to consider. > I'll try harder to see if there is a reasonable example whose behavior > would change... > Regards, Tim Hochberg From jack at performancedrivers.com Thu Mar 30 06:09:56 2006 From: jack at performancedrivers.com (Jack Diederich) Date: Wed, 29 Mar 2006 23:09:56 -0500 Subject: [Python-Dev] Class decorators In-Reply-To: <5.1.1.6.0.20060329201639.01e43970@mail.telecommunity.com> References: <5.1.1.6.0.20060329142653.037617a8@mail.telecommunity.com> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com> <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> <200603282335.09341.fdrake@acm.org> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <5.1.1.6.0.20060329142653.037617a8@mail.telecommunity.com> <5.1.1.6.0.20060329201639.01e43970@mail.telecommunity.com> Message-ID: <20060330040956.GJ5787@performancedrivers.com> [promted by Phillip Eby's post, but not in response so content snipped] I think we both want class decorators as a more fine grained substitute for __metaclass__ (fine grained as in declared per-class-instance instead of this-class-and-all-its-children). I can think of three ways class decorators are used: 1) register pattern, use a class attribute or two to stick the class in a lookup table and optionally delete the meta-attribs from the class 2) __init__ pattern, examine the class and maybe munge it based on attribs 3) __new__ pattern, consider the class a namespace and use the attribs to generate an essentially new class based on the attribs (the main difference between 2 and 3 is that the __new__ caseis a total tear down and rebuild where 2 tends towards tweaks on the parent) Classes have a unique property in that they are the easiest way to make little namespaces in python. Modules require lots of file clutter, functions as namespaces require digging into the internals, dicts as namespaces require more finger typing and don't support attribute access. It turns out I have two use cases for class decorators and didn't even know it. One is the 'register' pattern that started this thread. In that case I just want to move the metadata outside the class (the @register(db_id=20) case) and the rest of the class definition is honest to goodness overriding of a method or two from the parent class to change the behavior of its instances. The other pattern I hadn't thought of would be a '@namespace' decorator. A @namespace decorator would strip the attribs of all class-like qualities - it would strip the class of all descriptor magic (using descriptors, of course!). I just want a convenient bag to stick related items in. The first time I ever used metaclasses was to make PLY[1] (a SLR parser) use per-class namespaces for lexers and grammars instead of per-module. The metaclass chewed through all the class attributes and returned a class that shared no attributes with the original - each class was basically a config file. The decorator version would be spelled '@make_a_lexer_class' or '@make_a_grammar_class'. PEAK and Zope seem like they do a mix of __init__ and __new__, my current use cases are just 'notice' (I'm not a user, so feel free to correct). I like the func-like decorator syntax because I have just a bit of metadata that I'd like to move outside the class. PEAK/Zope sounds like they use classes as a mix of class and namespace. Their class decorator would return a hybrid class that has applied the namespace parts (post processing) to the class parts. A partly new class. I'd like that spelled: @tweak_this_class class MyClass: namespacey_stuff = ... def class_thing(self, foo): pass That leaves a class decorator behaving like a function decorator. It is a declaration that the final product (maybe a completely different thing as in the PLY case) is the result of the tweak_this_class function. Writing the above certainly helped me clear up my thoughts on the issue, I hope it hasn't gone and muddied it for anyone else :) -jackdied [1] http://savannah.nongnu.org/projects/ply/ From raymond.hettinger at verizon.net Thu Mar 30 06:09:45 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Wed, 29 Mar 2006 23:09:45 -0500 Subject: [Python-Dev] PySet API References: <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> <1143468446.10799.50.camel@resist.wooz.org> <200603281507.18175.gmccaughan@synaptics-uk.com> <001a01c65377$cd099a10$6601a8c0@RaymondLaptop1> <1143678849.6694.52.camel@resist.wooz.org> Message-ID: <004c01c653af$c8017a00$6601a8c0@RaymondLaptop1> [Raymond Hettinger] > > Barry, go ahead with PySet_Clear(). [Barry] Cool thanks. I think we've also compromised on _PySet_Next(), correct? Yes, _PySet_Next() is a good compromise for you and me -- it saves you from writing a hack and saves my API from including a bug factory. The only issue is that Martin thinks it to be a crummy idea. Personally, I have no problem with putting-in an undocumented hook for daring people who aspire to swim in quicksand ;-) [Raymond Hettinger] > > The story is different for PySet_Update(). Defining it now could get in the > > way > > of possible future development for the module (the function may end-up > > taking a > > variable length argument list instead of a single argument). > > So why not just go ahead and do that now? If you know that's what you > want eventually, why wait? From my perspective, adding a NULL at the > end of the argument list wouldn't be that big of a deal. [Barry] > BTW, I'm willing to do the work on this. I'm already going to update my > patch anyway to reflect our current decisions, so I'm happy to do this > while I'm at it. I'll try to get a new patch posted in a day or so. The idea is not yet ready for prime-time. If I do it for one of the named operations, I will do it for all (to keep the interface uniform). I haven't yet had the time to work-out the math on whether it would be worthwhile and provide some differential advantage over simply repeating the same operation several times over. My research question is whether work can be saved by controlling the order of operations -- the concept is somewhat like optimizing multi-term matrix multiplication where the total work effort can vary dramatically depending on which matrices are multiplied together first, A((BC)D) vs (AB)(CD) vs (A(BC))D etc. Put in business terms, the question is whether I'm able to leverage the associative and commutative properties of some chained set operations. FWIW, the module already has optimizations to take advantage of the commutative property of binary AND, OR, and SYMMETRIC_DIFFERENCE operations. However, the multi-term optimization probably wait until Py2.6 -- it is too experimental for now. Raymond From guido at python.org Thu Mar 30 06:24:55 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 29 Mar 2006 20:24:55 -0800 Subject: [Python-Dev] INPLACE_ADD and INPLACE_MULTIPLY oddities in ceval.c In-Reply-To: <442B1FAE.2010003@canterbury.ac.nz> References: <4428C77E.5020005@canterbury.ac.nz> <4429D71F.1010601@canterbury.ac.nz> <20060329104540.GA26145@code0.codespeak.net> <442AAB86.4010008@ieee.org> <442B1FAE.2010003@canterbury.ac.nz> Message-ID: On 3/29/06, Greg Ewing wrote: > Tim Hochberg wrote: > > > Still, perhaps for Py3K it's worth considering > > if PyNumber_InplaceAdd should only call __iadd__ and __add__, not > > __radd__. Thus giving the target object complete control during inplace > > adds. > > That's probably reasonable, although it would break > the conceptual notion that a += b is equivalent to > a = a + b when a can't be modified in-place. No, it's not reasonable, because this would fail: x = 5 x += 0.5 -- --Guido van Rossum (home page: http://www.python.org/~guido/) From nnorwitz at gmail.com Thu Mar 30 07:37:21 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 29 Mar 2006 21:37:21 -0800 Subject: [Python-Dev] alpha problems -- need input Message-ID: These issues are on HEAD. There might be some others I missed. With cc there are at least 2 issues: * test_file causes interpreter exit due to sys.stdin.seek(-1) * test_pty fails apparently due to whitespace differences http://www.python.org/dev/buildbot/all/alpha%20Tru64%205.1%20trunk/builds/18/step-test/0 Should we skip the test for an invalid seek on stdin on osf1? I haven't investigated the test_pty failure further. With gcc, there are also several issues: * test_float and test_struct fail due to NaN handling * test_long fails * test_ctypes The question is how to fix these. test_float and test_struct fail due to a Floating Point Exception signal (SIGFPE). test_long fails due to float(shuge) raising a ValueError. There is a comment: # XXX Perhaps float(shuge) can raise OverflowError on some box? # The comparison should not. Note: it raises value error, not OverflowError. Should I just wrap the float(shuge) in a try/except and skip the test if an exception is raised? The patches below fix the float and struct problems. I'm not sure if we should ignore SIGFPE in test_float. I'm also not sure if the change to floatobject.c should be made. It's possible there is a compiler flag that would help. I tried -mieee and a few others, but none seemed to make a difference for gcc. test_ctypes fails because _findLib() doesn't seem to work properly on OSF1. Note: libc.so => /usr/shlib/libc.so ERROR: test_find (ctypes.test.test_loading.LoaderTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/net/ringneck/scratch1/nnorwitz/python/trunk/Lib/ctypes/test/test_loading.py", line 40, in test_find cdll.find(name) File "/net/ringneck/scratch1/nnorwitz/python/trunk/Lib/ctypes/_loader.py", line 205, in find raise OSError("Library %r not found" % name) OSError: Library 'c' not found ====================================================================== ERROR: test_load (ctypes.test.test_loading.LoaderTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/net/ringneck/scratch1/nnorwitz/python/trunk/Lib/ctypes/test/test_loading.py", line 25, in test_load cdll.load(name) File "/net/ringneck/scratch1/nnorwitz/python/trunk/Lib/ctypes/_loader.py", line 112, in load return self._load(libname, mode) File "/net/ringneck/scratch1/nnorwitz/python/trunk/Lib/ctypes/_loader.py", line 124, in load_library return self._dlltype(libname, mode) File "/net/ringneck/scratch1/nnorwitz/python/trunk/Lib/ctypes/__init__.py", line 292, in __init__ self._handle = _dlopen(self._name, mode) OSError: dlopen: Cannot map library libc.so.6 ====================================================================== Below are the patches. Index: Objects/floatobject.c =================================================================== --- Objects/floatobject.c (revision 43416) +++ Objects/floatobject.c (working copy) @@ -1430,10 +1430,19 @@ return -1; } else { - float y = (float)x; + float y; const char *s = (char*)&y; int i, incr = 1; + /* The conversion of x to a float causes a floating exception + on some platforms (at least OSF1 v5.1 gcc-4.0.1) + */ +#ifndef MAXFLOAT +#define MAXFLOAT ((float)3.40282346638528860e+38) +#endif + if (x > MAXFLOAT || x < -MAXFLOAT) + goto Overflow; + y = (float)x; if ((float_format == ieee_little_endian_format && !le) || (float_format == ieee_big_endian_format && le)) { p += 3; Index: Lib/test/test_float.py =================================================================== --- Lib/test/test_float.py (revision 43416) +++ Lib/test/test_float.py (working copy) @@ -1,4 +1,5 @@ +import signal import unittest, struct from test import test_support @@ -83,6 +84,12 @@ # is accident (today). class IEEEFormatTestCase(unittest.TestCase): + def setUp(self): + signal.signal(signal.SIGFPE, signal.SIG_IGN) + + def tearDown(self): + signal.signal(signal.SIGFPE, signal.SIG_DFL) + if float.__getformat__("double").startswith("IEEE"): def test_double_specials_do_unpack(self): for fmt, data in [('>d', BE_DOUBLE_INF), Index: Lib/test/test_file.py =================================================================== --- Lib/test/test_file.py (revision 43416) +++ Lib/test/test_file.py (working copy) @@ -100,12 +100,14 @@ print "writelines accepted sequence of non-string objects" f.close() -try: - sys.stdin.seek(-1) -except IOError: - pass -else: - print "should not be able to seek on sys.stdin" +# This causes the interpreter to exit on OSF1 v5.1. +if sys.platform != 'osf1V5': + try: + sys.stdin.seek(-1) + except IOError: + pass + else: + print "should not be able to seek on sys.stdin" try: sys.stdin.truncate() Index: Lib/test/test_long.py =================================================================== --- Lib/test/test_long.py (revision 43416) +++ Lib/test/test_long.py (working copy) @@ -372,10 +372,15 @@ self.assertRaises(OverflowError, eval, test, namespace) - # XXX Perhaps float(shuge) can raise OverflowError on some box? - # The comparison should not. - self.assertNotEqual(float(shuge), int(shuge), - "float(shuge) should not equal int(shuge)") + # float(shuge) can raise OverflowError on an Alpha built with gcc. + try: + fshuge = float(shuge) + except ValueError: + pass + else: + # The comparison should not. + self.assertNotEqual(fshuge, int(shuge), + "float(shuge) should not equal int(shuge)") def test_logs(self): import math Index: Lib/test/test_struct.py =================================================================== --- Lib/test/test_struct.py (revision 43416) +++ Lib/test/test_struct.py (working copy) @@ -429,11 +429,12 @@ # The same, but tack on a 1 bit so it rounds up to infinity. big = (1 << 25) - 1 big = math.ldexp(big, 127 - 24) - try: - packed = struct.pack(">f", big) - except OverflowError: - pass - else: - TestFailed("expected OverflowError") + for value in (big, -big): + try: + packed = struct.pack(">f", value) + except OverflowError: + pass + else: + TestFailed("expected OverflowError") test_705836() From greg at electricrain.com Thu Mar 30 08:18:01 2006 From: greg at electricrain.com (Gregory P. Smith) Date: Wed, 29 Mar 2006 22:18:01 -0800 Subject: [Python-Dev] _bsddb.c ownership In-Reply-To: <9e804ac0603071803u2e071929hcabd85a36ddea5b@mail.gmail.com> References: <9e804ac0603070851r26c2db3eje0d61c1af0005029@mail.gmail.com> <440DDC23.5080301@v.loewis.de> <9e804ac0603071803u2e071929hcabd85a36ddea5b@mail.gmail.com> Message-ID: <20060330061801.GA31651@zot.electricrain.com> On Wed, Mar 08, 2006 at 03:03:48AM +0100, Thomas Wouters wrote: > On 3/7/06, "Martin v. L??wis" wrote: > > > > Thomas Wouters wrote: > > > Who 'owns' Modules/_bsddb.c, if anyone? > > > > It's a fork of pybsddb, originally contributed by Gregory Smith (*). > > For all practical purposes, he also "owns" it, but hasn't objected > > to others making changes in the past. > > > > At the time it was imported, I recall the plan was to out-phase > > pybsddb, and only keep the version in Python. It appears that things > > have developed differently. > > > Alright. It seems someone already made backward-incompatible changes > to _bsddb.c (adding Py_ssize_t use), and various other things were fixed, > too, so I just went ahead and checked in a bunch of small fixes. I'm not > done yet, though. After I am done (which might be a while still, but > hopefully before alpha1), I'll fix backward compatibility and send in a > patch to the pybsddb project :) Yeah I "own" it for whatever thats worth. The pybsddb project only continues to exist as a skeleton to provide a more up to date BerkeleyDB module for: (a) use with a new BerkeleyDB version (b) use with older versions of python The pybsddb project does not have its own copy of the code, it just pulls Modules/_bsddb.c and the Lib/bsddb/ directly from the python repository using a script. Its appreciated if people keep Modules/_bsddb.c and Lib/bsddb/ backwards compatible with the few revs of python. Otherwise I end up finding out and fixing it myself with ifdefs on the rare occasions when I attempt to make another pybsddb module release and find it doesn't work with an old version. ;) -greg From martin at v.loewis.de Thu Mar 30 08:29:14 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 30 Mar 2006 08:29:14 +0200 Subject: [Python-Dev] PySet API In-Reply-To: <004c01c653af$c8017a00$6601a8c0@RaymondLaptop1> References: <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> <1143468446.10799.50.camel@resist.wooz.org> <200603281507.18175.gmccaughan@synaptics-uk.com> <001a01c65377$cd099a10$6601a8c0@RaymondLaptop1> <1143678849.6694.52.camel@resist.wooz.org> <004c01c653af$c8017a00$6601a8c0@RaymondLaptop1> Message-ID: <442B7ABA.8010405@v.loewis.de> Raymond Hettinger wrote: > Yes, _PySet_Next() is a good compromise for you and me -- it saves you from > writing a hack and saves my API from including a bug factory. The only issue is > that Martin thinks it to be a crummy idea. If it makes everyone happy, I shouldn't be in the way. Of course, it might be that not only Barry will use it, but other people as well. Regards, Martin From martin at v.loewis.de Thu Mar 30 08:31:26 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 30 Mar 2006 08:31:26 +0200 Subject: [Python-Dev] warnings in libffi In-Reply-To: References: Message-ID: <442B7B3E.9050609@v.loewis.de> Brett Cannon wrote: > All of them are for function parameters of function pointers (``void > (*fn)(void)`` and such) when used in both function prototypes and > function declarations. Do we fix these ourselves, or do we report > them to the libffi maintainers (or are whom)? If you don't have write access to the ctypes CVS, you should report that (with a patch) to sf.net/projects/ctypes. Thomas will then decide how to process it. Regards, Martin From greg at electricrain.com Thu Mar 30 08:32:21 2006 From: greg at electricrain.com (Gregory P. Smith) Date: Wed, 29 Mar 2006 22:32:21 -0800 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: References: <441217BB.1000801@v.loewis.de> Message-ID: <20060330063221.GB31651@zot.electricrain.com> On Sat, Mar 11, 2006 at 07:08:14PM +0100, Thomas Heller wrote: > Martin v. L?wis wrote: > > Josiah Carlson told me had has given up getting a Windows > > buildbot running, because every time he installed VS.NET > > on his machine, the installation would immediately crash. > > > > So if anybody wants to contribute both a machine and time > > to operate it (including the likely very tedious phase to > > get any results out of this at all), please contact me. This is an ideal job for VMWare on an existing linux build machine if someone can ante up a win xp and msvc++ license to the cause. A vmware license for the build host probably isn't even necessary assuming vmware player works and one of us with a license can setup the vmware image (http://www.vmware.com/products/player/). (theres also a chance we could get vmware inc. to donate a license to python if one was needed) From martin at v.loewis.de Thu Mar 30 08:35:59 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 30 Mar 2006 08:35:59 +0200 Subject: [Python-Dev] alpha problems -- need input In-Reply-To: References: Message-ID: <442B7C4F.1030207@v.loewis.de> Neal Norwitz wrote: > The question is how to fix these. test_float and test_struct fail due > to a Floating Point Exception signal (SIGFPE). I would hope that there is some way to control the floating point error mode of the CPU (*). Changing it would be one option; Tim hopefully can tell us whether it's a good idea to do so. Regards, Martin (*) I believe you use some #pragma for that in C99; before, there might be some libc or libm API to manipulate the error mode. From greg at electricrain.com Thu Mar 30 08:38:28 2006 From: greg at electricrain.com (Gregory P. Smith) Date: Wed, 29 Mar 2006 22:38:28 -0800 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <1f7befae0603121548k29d18135mbcf0e3365f23c93@mail.gmail.com> References: <441217BB.1000801@v.loewis.de> <20060312002336.GA3935@activestate.com> <1f7befae0603111849n67c40694u7eebf38410b8788e@mail.gmail.com> <20060312224148.GB32648@activestate.com> <1f7befae0603121548k29d18135mbcf0e3365f23c93@mail.gmail.com> Message-ID: <20060330063828.GC31651@zot.electricrain.com> On Sun, Mar 12, 2006 at 06:48:13PM -0500, Tim Peters wrote: > [Trent] > > :) > > Did you apply the Berkeley DB patches to your db-4.2.52 sources? > > Ah, _which_ patches? As with my buildbot Wiki page, I write down > everything I do if there's a good chance I may need to do it again. > So, e.g., these are my words in PCbuild\readme.txt: > > As of 11-Apr-2004, you also need to download and manually apply two > patches before proceeding (and the sleepycat download page tells you > about this). Cygwin patch worked for me. cd to dist\db-4.2.52 and > use "patch -p0 < patchfile" once for each downloaded patchfile. > > It's possible that there are more patches needed since then, but if so > I wouldn't know about that (last time I built the externals in the > _bsddb part was indeed 11-Apr-2004). FWIW, thats an old BerkeleyDB (yes it'll still work). Python 2.5 should ship built with BerkeleyDB 4.4.20 so thats what buildbot should use if it builds the module. From theller at python.net Thu Mar 30 08:44:17 2006 From: theller at python.net (Thomas Heller) Date: Thu, 30 Mar 2006 08:44:17 +0200 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <20060330063221.GB31651@zot.electricrain.com> References: <441217BB.1000801@v.loewis.de> <20060330063221.GB31651@zot.electricrain.com> Message-ID: Gregory P. Smith wrote: > On Sat, Mar 11, 2006 at 07:08:14PM +0100, Thomas Heller wrote: >> Martin v. L?wis wrote: >>> Josiah Carlson told me had has given up getting a Windows >>> buildbot running, because every time he installed VS.NET >>> on his machine, the installation would immediately crash. >>> >>> So if anybody wants to contribute both a machine and time >>> to operate it (including the likely very tedious phase to >>> get any results out of this at all), please contact me. > > This is an ideal job for VMWare on an existing linux build machine if > someone can ante up a win xp and msvc++ license to the cause. A > vmware license for the build host probably isn't even necessary > assuming vmware player works and one of us with a license can setup > the vmware image (http://www.vmware.com/products/player/). (theres > also a chance we could get vmware inc. to donate a license to python > if one was needed) Right. I have setup 64-bit Ubuntu on a new Athon64 machine, created WinXP and WinXP64 empty virtual machines with VMWare workstation, then installed XP and XP64 into these images. The images run on the Ubuntu machine in VMWare player. I don't think we need more x86 or x86-64 buildbots, so I made no attempt to install the buildbots. Thomas From pje at telecommunity.com Thu Mar 30 08:49:07 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 30 Mar 2006 01:49:07 -0500 Subject: [Python-Dev] Class decorators In-Reply-To: <442B4C02.2020708@canterbury.ac.nz> References: <5.1.1.6.0.20060329201639.01e43970@mail.telecommunity.com> <5.1.1.6.0.20060329142653.037617a8@mail.telecommunity.com> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com> <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> <200603282335.09341.fdrake@acm.org> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <5.1.1.6.0.20060329142653.037617a8@mail.telecommunity.com> <5.1.1.6.0.20060329201639.01e43970@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060330014323.01ed8e68@mail.telecommunity.com> At 03:09 PM 3/30/2006 +1200, Greg Ewing wrote: >Well, here's how my use case would look if I had >class decorators: > > @IOClass > class MyClass: > ... > >Does that count? My decorator wouldn't need any >arguments, because it looks inside the class for >all the information it needs. [1] > >That's actually a general solution to Phillip's >concern: the decorator can always look for attributes >in the class (or the class's __dict__ if you don't >want them inherited) for large amounts of information >that wouldn't comfortably fit up the top. That's >an extra degree of freedom that we don't have with >functions. I fail to see how this is an improvement. It sounds like the worst of all possible worlds to me -- *much* worse than the status quo. I use class decorators to declare metadata *about* attributes, so having to add a bunch of attributes for that *and* having to stick a funny hat on top of the class is definitely not an improvement for me. Are you actually *using* this IOClass thing, or is this just a hypothetical proposal? From nnorwitz at gmail.com Thu Mar 30 08:50:12 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 29 Mar 2006 22:50:12 -0800 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <20060330063221.GB31651@zot.electricrain.com> References: <441217BB.1000801@v.loewis.de> <20060330063221.GB31651@zot.electricrain.com> Message-ID: On 3/29/06, Gregory P. Smith wrote: > > This is an ideal job for VMWare on an existing linux build machine if > someone can ante up a win xp and msvc++ license to the cause. It probably isn't great from a practical point of view if you wanted to run buildbot for both the server and host platforms. The problem is that jobs for both platforms will be scheduled concurrently. If checkins occur in both 2.5 and 2.4 as is somewhat common, you would be running 4 test suites simultaneously. The full suite already takes quite a bit of time (over 20 minutes depending on the box). The machine could be crushed if we run 4. If you just wanted to run Windows in VMware, that would be fine. We already have 3 windows buildbots of the XP and 2k variety. Other varieties might be desirable. n From pje at telecommunity.com Thu Mar 30 08:52:41 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 30 Mar 2006 01:52:41 -0500 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <442B4EC0.8050809@canterbury.ac.nz> References: <200603292207.31378.fdrake@acm.org> <200603281759.18756.anthony@interlink.com.au> <17451.13615.74425.528082@montanaro.dyndns.org> <442B48AE.6010105@canterbury.ac.nz> <200603292207.31378.fdrake@acm.org> Message-ID: <5.1.1.6.0.20060330015012.01ed8fb0@mail.telecommunity.com> At 03:21 PM 3/30/2006 +1200, Greg Ewing wrote: >Fred L. Drake, Jr. wrote: > > On Wednesday 29 March 2006 21:55, Greg Ewing wrote: > > > import db where db.stdlib == True and db.language == "SQL" \ > > > and db.interface == "DBAPI2.0" > > > > While we're at it, we could spell import "select". :-) > >Getting off on a tangent here, but I would actually >like some decent way of writing SQL queries in Python -- >not for importing, but for database access. Listcomps and generator expressions are effectively equivalent to the nested domain relational calculus -- which is actually more powerful than SQL (which is based more or less on the tuple relational calculus). I've been putting off writing a genexp-to-SQL translator for some time now, but I think somebody else wrote one that actually works; I don't remember if it's SQLAlchemy or one of the other new Python ORM programs. I think it only does the subset of genexps that are easily translated to SQL, but it's certainly a start. From nnorwitz at gmail.com Thu Mar 30 08:56:18 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 29 Mar 2006 22:56:18 -0800 Subject: [Python-Dev] unicode vs buffer (array) design issue can crash interpreter Message-ID: See http://python.org/sf/1454485 for the gory details. Basically if you create a unicode array (array.array('u')) and try to append an 8-bit string (ie, not unicode), you can crash the interpreter. The problem is that the string is converted without question to a unicode buffer. Within unicode, it assumes the data to be valid, but this isn't necessarily the case. We wind up accessing an array with a negative index and boom. This is perhaps somewhat simplified and not exactly accurate. Probably best to look over the patch and make comments there on how best to fix this issue. n From pje at telecommunity.com Thu Mar 30 09:06:52 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 30 Mar 2006 02:06:52 -0500 Subject: [Python-Dev] Class decorators In-Reply-To: <20060330040956.GJ5787@performancedrivers.com> References: <5.1.1.6.0.20060329201639.01e43970@mail.telecommunity.com> <5.1.1.6.0.20060329142653.037617a8@mail.telecommunity.com> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com> <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> <200603282335.09341.fdrake@acm.org> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <5.1.1.6.0.20060329142653.037617a8@mail.telecommunity.com> <5.1.1.6.0.20060329201639.01e43970@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060330015322.0545ff88@mail.telecommunity.com> At 11:09 PM 3/29/2006 -0500, Jack Diederich wrote: >I think we both want class decorators as a more fine grained substitute >for __metaclass__ (fine grained as in declared per-class-instance instead >of this-class-and-all-its-children). I can think of three ways class >decorators are used: > >1) register pattern, use a class attribute or two to stick the class in > a lookup table and optionally delete the meta-attribs from the class >2) __init__ pattern, examine the class and maybe munge it based on attribs >3) __new__ pattern, consider the class a namespace and use the attribs > to generate an essentially new class based on the attribs My uses of class decorators don't fit any of your descriptions -- neither do Zope's for that matter, at least not the ones I'm familiar with. "implements(IFoo)" doesn't use any class attributes or delete meta attributes from the class. The whole point of using these decorators is to *not* interfere with the class by putting metadata in attributes. This is especially true for PEAK's class decorators, which are used to do things like make security declarations about attributes offered by the class (see the examples I posted previously in response to Fred's @class proposal). >It turns out I have two use cases for class decorators and didn't even >know it. One is the 'register' pattern that started this thread. In >that case I just want to move the metadata outside the class >(the @register(db_id=20) case) and the rest of the class definition is >honest to goodness overriding of a method or two from the parent class >to change the behavior of its instances. The other pattern I hadn't thought >of would be a '@namespace' decorator. A @namespace decorator would strip >the attribs of all class-like qualities - it would strip the class of all >descriptor magic (using descriptors, of course!). I just want a convenient >bag to stick related items in. This seems to me like a perfectly good place to use a metaclass. The place where metaclasses tend to quickly meet their downfall is in trying to support inheritance, especially multiple inheritance. If you just have a bag of data, however, inheritance doesn't enter into it and a metaclass is *perfect*. It doesn't even have to inherit from 'type', it can even be a classic class or a function, just as long as it takes a name, bases tuple, and dictionary. >PEAK and Zope seem like they do a mix of __init__ and __new__, my current >use cases are just 'notice' (I'm not a user, so feel free to correct). >I like the func-like decorator syntax because I have just a bit of >metadata that I'd like to move outside the class. PEAK/Zope sounds >like they use classes as a mix of class and namespace. Their class >decorator would return a hybrid class that has applied the namespace >parts (post processing) to the class parts. A partly new class. > >I'd like that spelled: >@tweak_this_class >class MyClass: > namespacey_stuff = ... > def class_thing(self, foo): pass You can do this quite easily now. Try running this code and see what happens: class Tweaker(object): def __class__(self, name, bases, cdict): print name, bases, cdict # put code here that returns something using name,bases,cdict return "I'm a string!" Tweak = Tweaker() class MyClass(Tweak): namespacey_stuff = "foo" def class_thing(self, foo): pass assert MyClass == "I'm a string!" If what you're making isn't a class, metaclasses work just fine. The return value of the __class__ method shown here will be bound to the name 'MyClass' when this runs. >That leaves a class decorator behaving like a function decorator. They always did, and as far as I know, nobody has proposed they behave otherwise. >It >is a declaration that the final product (maybe a completely different >thing as in the PLY case) is the result of the tweak_this_class function. Again, if you don't want a class to be the result, you can do this sort of thing quite easily now. From greg at electricrain.com Thu Mar 30 09:22:02 2006 From: greg at electricrain.com (Gregory P. Smith) Date: Wed, 29 Mar 2006 23:22:02 -0800 Subject: [Python-Dev] I'm not getting email from SF when assignedabug/patch In-Reply-To: References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> <1143554576.2310.161.camel@geddy.wooz.org> <17449.21764.475497.457592@montanaro.dyndns.org> <44298954.4040705@v.loewis.de> <1143648409.10799.159.camel@resist.wooz.org> <4c45c1530603290857o50a10d0cnef21922927d5810@mail.gmail.com> Message-ID: <20060330072202.GD31651@zot.electricrain.com> > The language choice should only be used as an argument if all else is > equal. Of course, "hackability" of a particular solution may be a > criterion too, and there the language choice could matter. But the > above response sounded like a knee-jerk to me, and IMO needs to be > rebutted. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) Include in "hackability" the ability to get all ticket data out accurately in a reasonable format to auto-import into a different system 5 years from now when the new one, whatever it may be, is deemed annoying. :) i like the idea of trac. i like the experience of jira/confluence. anything beats sourceforge. From arigo at tunes.org Thu Mar 30 09:27:02 2006 From: arigo at tunes.org (Armin Rigo) Date: Thu, 30 Mar 2006 09:27:02 +0200 Subject: [Python-Dev] INPLACE_ADD and INPLACE_MULTIPLY oddities in ceval.c In-Reply-To: <442AAB86.4010008@ieee.org> References: <4428C77E.5020005@canterbury.ac.nz> <4429D71F.1010601@canterbury.ac.nz> <20060329104540.GA26145@code0.codespeak.net> <442AAB86.4010008@ieee.org> Message-ID: <20060330072702.GA21393@code0.codespeak.net> Hi Tim, On Wed, Mar 29, 2006 at 08:45:10AM -0700, Tim Hochberg wrote: > Ouch. Assuming the same path is followed with tuples, I think that this > means the following behaviour will continue: > > >>> t = (1,2,3) > >>> a = array([4,5,6]) > >>> t += a > >>> t > array([5, 7, 9]) I fell into the same trap at first, but no: in fact, only lists have a special in-place addition among all the built-in objects. Tuples fall back to the normal addition, which means that you can only add tuples to tuples: >>> t = (1,2,3) >>> t += [4,5,6] TypeError: can only concatenate tuple (not "list") to tuple >>> t += array([4,5,6]) TypeError: ... This is current behavior and it wouldn't change. A bientot, Armin. From nnorwitz at gmail.com Thu Mar 30 09:33:22 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 29 Mar 2006 23:33:22 -0800 Subject: [Python-Dev] I'm not getting email from SF when assignedabug/patch In-Reply-To: <1143648409.10799.159.camel@resist.wooz.org> References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> <1143554576.2310.161.camel@geddy.wooz.org> <17449.21764.475497.457592@montanaro.dyndns.org> <44298954.4040705@v.loewis.de> <1143648409.10799.159.camel@resist.wooz.org> Message-ID: On 3/29/06, Barry Warsaw wrote: > > I'll just point out that Atlassian has offered us free hosting for a > Jira/Confluence solution (plus svn and other stuff we may or may not > want). I personally support this option, but I know (and accept!) that > there are differing opinions. I'm in favor of having Atlassian setup a system to be used for 3k. It would be completely experimental and could be completely thrown away which should be made clear to Atlassian if we were to do this. I would use the system for evaluation. If we did this, we should also try to ensure we can get all the data out, just in case we want to preserve any of it. Most of my time will probably be spent cleaning up 2.5 for the next couple of weeks at least. So I'm not going to have time to rip out more stuff from 3k. coerce is almost gone. Low hanging fruit for someone interested in helping out: * find warnings with icc that are important to fix * remove unnecessary casts for PyMethodDefs The first item requires icc obviously. I haven't had time to set up my version yet. The second item could be done by anyone. There may be some issues resulting from the ssize_t merge that are being masked due to casting. If we remove the unnecessary casts (e.g., PyCFunction, unaryfunc, binaryfunc, ternaryfunc), we may find some things to cleanup. We still need builbots for: FreeBSD, AIX, HP-UX, and NetBSD. n From greg at electricrain.com Thu Mar 30 09:33:51 2006 From: greg at electricrain.com (Gregory P. Smith) Date: Wed, 29 Mar 2006 23:33:51 -0800 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <442A8D34.8060200@ghaering.de> References: <200603281759.18756.anthony@interlink.com.au> <200603291947.34461.anthony@interlink.com.au> <1143638252.2317.184.camel@geddy.wooz.org> <442A8D34.8060200@ghaering.de> Message-ID: <20060330073351.GE31651@zot.electricrain.com> On Wed, Mar 29, 2006 at 03:35:48PM +0200, Gerhard H?ring wrote: > Barry Warsaw wrote: > > On Wed, 2006-03-29 at 19:47 +1100, Anthony Baxter wrote: > > > >>My only concern about this is that it wouldn't be possible for other > >>authors to provide 3rd party packages as (for instance) db.mysqldb > >>because of the way package importing works. And I'd prefer > >>'database.sqlite' rather than 'db.sqlite'. > > > > +1 on 'database' as the top-level package name. > > I think short names are more more consistent with the existing naming in > the standard library. > > +1 on db.sqlite from me. > > db.sql.sqlite is another possibility, if adding something like Durus or > ZODB in the same top-level namespace could be considered for 2.6. two letter names seem short but database is a long word. long words make me thing of java. so i like 'db' ;) +1 on db.sql.sqlite. I was also about to suggest just plain sql.sqlite along the same lines as all of the one people had mentioned so far had really been SQL interfaces rather than general databases. With the bsddb module a common thing for code to do is "from bsddb import db" to have easy to type access to the full bsddb.db.DB object interface. that interface could migrate to live under db.berkeleydb or db.bsddb at some future point in time. -greg From fredrik at pythonware.com Thu Mar 30 09:39:52 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 30 Mar 2006 09:39:52 +0200 Subject: [Python-Dev] pysqlite for 2.5? References: <200603290152.00598.anthony@interlink.com.au><200603281759.18756.anthony@interlink.com.au><1143554351.2309.158.camel@geddy.wooz.org><5.1.1.6.0.20060329144142.0390e5d8@mail.telecommunity.com><17451.882.842146.979286@montanaro.dyndns.org><17451.12090.257751.636336@montanaro.dyndns.org> <442B46D6.70703@canterbury.ac.nz> Message-ID: Greg Ewing wrote: > Firebird could be a solution to this. It can be > used in a mode that doesn't need a server, and it > has no trouble at all with concurrency or large > amounts of data that I know of. so a library that doesn't support multiple independent readers/writers on a single file at all is much better than one that does, at least if the latter immediately reports conflicts if you use a zero second timeout ? From greg at electricrain.com Thu Mar 30 09:48:43 2006 From: greg at electricrain.com (Gregory P. Smith) Date: Wed, 29 Mar 2006 23:48:43 -0800 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <9e804ac0603291347t2f4e1d94l4fa022fb0e2aa738@mail.gmail.com> References: <200603281759.18756.anthony@interlink.com.au> <1143554351.2309.158.camel@geddy.wooz.org> <200603290152.00598.anthony@interlink.com.au> <5.1.1.6.0.20060329144142.0390e5d8@mail.telecommunity.com> <9e804ac0603291347t2f4e1d94l4fa022fb0e2aa738@mail.gmail.com> Message-ID: <20060330074843.GF31651@zot.electricrain.com> On Wed, Mar 29, 2006 at 11:47:10PM +0200, Thomas Wouters wrote: > Con: > > * Competing Python wrappers exist > > * SQLite itself is updated frequently, let alone the wrappers > > * Build integration risks unknown, possible delay of 2.5? > > * Another external library to track and maybe have emergency updates of > > All of these con arguments go for bsddb, too, and without sounding too > negative about bsddb, I believe SQLite is a *much* better solution than > BerkeleyDB, for roughly the same problem space. The same goes for pysqlite > vs. bsddb. IMNSHO, SQLite and pysqlite are much easier to use correctly than > BerkelyDB and bsddb, for simple and complex tasks. I may be biased against > bsddb because I spent too much time hunting refleaks in it, but I'm not > biased in favour of SQLite -- I'm a PostgreSQL user myself. ;-P Agreed. sqlite is a joy to use. Its simple. It provides table structure that anyone can understand. BerkeleyDB is very powerful but requires a much more serious time investment to use usefully than sqlite for anything other than simple dictionary-like data storage. I wanted sqlite to exist for years. The intentionally undocumented bsddb.db.dbtables module i hacked together in early 2000 would never have been written had sqlite existed at the time. other things available at the time (gadfly anyone?) just didn't seem right. > probably a good thing. I would probably choose sqlite instead of > shelve/anydbm/bsddb if it were part of the standard library, even though > it's probably installed on all my machines anyway. I guess it's a psych > thing. > > As for people asking about deadlocks, well, I much rather explain about > sqlite deadlocks than about BerkelyDB transactions. yep. From arigo at tunes.org Thu Mar 30 09:56:00 2006 From: arigo at tunes.org (Armin Rigo) Date: Thu, 30 Mar 2006 09:56:00 +0200 Subject: [Python-Dev] INPLACE_ADD and INPLACE_MULTIPLY oddities in ceval.c In-Reply-To: <20060330072702.GA21393@code0.codespeak.net> References: <4428C77E.5020005@canterbury.ac.nz> <4429D71F.1010601@canterbury.ac.nz> <20060329104540.GA26145@code0.codespeak.net> <442AAB86.4010008@ieee.org> <20060330072702.GA21393@code0.codespeak.net> Message-ID: <20060330075600.GA21954@code0.codespeak.net> Hi Tim, Oups, sorry. I only just realized my mistake and the meaning of your message. On Thu, Mar 30, 2006 at 09:27:02AM +0200, Armin Rigo wrote: > >>> t = (1,2,3) > >>> t += [4,5,6] > TypeError: can only concatenate tuple (not "list") to tuple > > >>> t += array([4,5,6]) > TypeError: ... > > This is current behavior and it wouldn't change. I'm pasting untested bits of code. Indeed, as you point out: >>> t = (1,2,3) >>> t += array([4,5,6]) >>> t array([5, 7, 9]) and it would remain so after the fix. I still think the fix is a good thing, and the above is an issue at a different level. It's somehow the "fault" of list.__iadd__ and list.__imul__, which are oddballs -- before the introduction of set objects, it was the single place in the whole library of built-in types where in-place behavior was different from normal behavior. It would require an official language extension to say that for all sequences, += is supposed to accept any iterable (which may or may not be a good thing, I have no opinion here). Otherwise, I'd just ignore the whole sub-issue, given that 'tuple += array' returning an array is just correct language-wise and doesn't look like a trap for bad surprises -- if the user expected a tuple but gets an array, most tuple-like operations will work just fine on the array, except hashing, which gives a clean TypeError. A bientot, Armin From fredrik at pythonware.com Thu Mar 30 09:55:37 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 30 Mar 2006 09:55:37 +0200 Subject: [Python-Dev] I'm not getting email from SF whenassignedabug/patch References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com><00ff01c6523f$7c583290$bf03030a@trilan><1143554576.2310.161.camel@geddy.wooz.org><17449.21764.475497.457592@montanaro.dyndns.org><44298954.4040705@v.loewis.de> <1143648409.10799.159.camel@resist.wooz.org> Message-ID: Neal Norwitz wrote: > I'm in favor of having Atlassian setup a system to be used for 3k. It > would be completely experimental and could be completely thrown away > which should be made clear to Atlassian if we were to do this. I > would use the system for evaluation. so what's the advantage of a freely hosted Atlassian setup compared to a freely hosted Trac setup ? from what I can tell, no big contemporary Python project use Atlassian. they all use Trac. there are thousands of Python developers out there that are used to working with Trac. I'm obviously missing something here. From nnorwitz at gmail.com Thu Mar 30 10:04:10 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 30 Mar 2006 00:04:10 -0800 Subject: [Python-Dev] I'm not getting email from SF whenassignedabug/patch In-Reply-To: References: <00ff01c6523f$7c583290$bf03030a@trilan> <1143554576.2310.161.camel@geddy.wooz.org> <17449.21764.475497.457592@montanaro.dyndns.org> <44298954.4040705@v.loewis.de> <1143648409.10799.159.camel@resist.wooz.org> Message-ID: On 3/29/06, Fredrik Lundh wrote: > Neal Norwitz wrote: > > > I'm in favor of having Atlassian setup a system to be used for 3k. It > > would be completely experimental and could be completely thrown away > > which should be made clear to Atlassian if we were to do this. I > > would use the system for evaluation. > > so what's the advantage of a freely hosted Atlassian setup compared > to a freely hosted Trac setup ? Someone else's time to setup and maintain it. In particular, it won't take any time away from any python developer. Plus it's just a chance for me to learn. > from what I can tell, no big contemporary Python project use Atlassian. > they all use Trac. there are thousands of Python developers out there > that are used to working with Trac. If someone setups and maintains an instance of Trac, I'd use that too. I suppose I could use Roundup on python.org, but I don't know if it's working. I've used Roundup before, but it was an old version and obviously not tailored to python's setup. Like I said, I won't be prepared to use any new tracker for some time, at least a couple of weeks. If people want to setup systems for me to test out, I'm willing to be a guinea pig. The only thing I really care about is what makes my life the easiest. n From greg at electricrain.com Thu Mar 30 10:09:51 2006 From: greg at electricrain.com (Gregory P. Smith) Date: Thu, 30 Mar 2006 00:09:51 -0800 Subject: [Python-Dev] building sql queries in python In-Reply-To: <442B4EC0.8050809@canterbury.ac.nz> References: <200603281759.18756.anthony@interlink.com.au> <17451.13615.74425.528082@montanaro.dyndns.org> <442B48AE.6010105@canterbury.ac.nz> <200603292207.31378.fdrake@acm.org> <442B4EC0.8050809@canterbury.ac.nz> Message-ID: <20060330080951.GH31651@zot.electricrain.com> > Getting off on a tangent here, but I would actually > like some decent way of writing SQL queries in Python -- > not for importing, but for database access. > > Constructing bits of SQL out of character strings > sucks *extremely* badly. Have you looked at SqlObject? (and its associated modules sqlobject.sqlbuilder in particular) http://www.sqlobject.org/module-sqlobject.sqlbuilder.html -greg From thomas at python.org Thu Mar 30 10:16:49 2006 From: thomas at python.org (Thomas Wouters) Date: Thu, 30 Mar 2006 10:16:49 +0200 Subject: [Python-Dev] _bsddb.c ownership In-Reply-To: <20060330061801.GA31651@zot.electricrain.com> References: <9e804ac0603070851r26c2db3eje0d61c1af0005029@mail.gmail.com> <440DDC23.5080301@v.loewis.de> <9e804ac0603071803u2e071929hcabd85a36ddea5b@mail.gmail.com> <20060330061801.GA31651@zot.electricrain.com> Message-ID: <9e804ac0603300016p3e94efa2n88f6711275a9d6a2@mail.gmail.com> On 3/30/06, Gregory P. Smith wrote: > The pybsddb project does not have its own copy of the code, it just > pulls Modules/_bsddb.c and the Lib/bsddb/ directly from the python > repository using a script. Ah, excellent, that makes it a lot easier for the rest of python-dev. > Its appreciated if people keep Modules/_bsddb.c and Lib/bsddb/ > backwards compatible with the few revs of python. Otherwise I end up > finding out and fixing it myself with ifdefs on the rare occasions > when I attempt to make another pybsddb module release and find it > doesn't work with an old version. ;) I'll see about making all the recent changes backwardly-compatible. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060330/13e1260b/attachment.htm From thomas at python.org Thu Mar 30 10:38:19 2006 From: thomas at python.org (Thomas Wouters) Date: Thu, 30 Mar 2006 10:38:19 +0200 Subject: [Python-Dev] building sql queries in python In-Reply-To: <20060330080951.GH31651@zot.electricrain.com> References: <200603281759.18756.anthony@interlink.com.au> <17451.13615.74425.528082@montanaro.dyndns.org> <442B48AE.6010105@canterbury.ac.nz> <200603292207.31378.fdrake@acm.org> <442B4EC0.8050809@canterbury.ac.nz> <20060330080951.GH31651@zot.electricrain.com> Message-ID: <9e804ac0603300038g3fe3aa36l396d0c0fc7abdfb2@mail.gmail.com> On 3/30/06, Gregory P. Smith wrote: > > > Getting off on a tangent here, but I would actually > > like some decent way of writing SQL queries in Python -- > > not for importing, but for database access. > > > > Constructing bits of SQL out of character strings > > sucks *extremely* badly. > > Have you looked at SqlObject? (and its associated modules > sqlobject.sqlbuilder in particular) SQLAlchemy (www.sqlalchemy.org) is also nice, in particular for more complex setups. I found it scales much better to the insanely complex SQL queries that our home-grown PostgreSQL/Perl setup needs, and it handles transactions the same way as our home-grown ORM too. There's plenty of ways to reliably and sanely avoid typing actual SQL in Python. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060330/df1f8cb3/attachment.html From g.brandl at gmx.net Thu Mar 30 10:39:40 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 30 Mar 2006 10:39:40 +0200 Subject: [Python-Dev] I'm not getting email from SF whenassignedabug/patch In-Reply-To: References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com><00ff01c6523f$7c583290$bf03030a@trilan><1143554576.2310.161.camel@geddy.wooz.org><17449.21764.475497.457592@montanaro.dyndns.org><44298954.4040705@v.loewis.de> <1143648409.10799.159.camel@resist.wooz.org> Message-ID: Fredrik Lundh wrote: > Neal Norwitz wrote: > >> I'm in favor of having Atlassian setup a system to be used for 3k. It >> would be completely experimental and could be completely thrown away >> which should be made clear to Atlassian if we were to do this. I >> would use the system for evaluation. > > so what's the advantage of a freely hosted Atlassian setup compared > to a freely hosted Trac setup ? > > from what I can tell, no big contemporary Python project use Atlassian. > they all use Trac. there are thousands of Python developers out there > that are used to working with Trac. > > I'm obviously missing something here. Perhaps that Jira is commercial, so it is out of the question for most open-source Python applications. Georg From phd at mail2.phd.pp.ru Thu Mar 30 10:45:10 2006 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Thu, 30 Mar 2006 12:45:10 +0400 Subject: [Python-Dev] building sql queries in python In-Reply-To: <20060330080951.GH31651@zot.electricrain.com> References: <200603281759.18756.anthony@interlink.com.au> <17451.13615.74425.528082@montanaro.dyndns.org> <442B48AE.6010105@canterbury.ac.nz> <200603292207.31378.fdrake@acm.org> <442B4EC0.8050809@canterbury.ac.nz> <20060330080951.GH31651@zot.electricrain.com> Message-ID: <20060330084509.GA25840@phd.pp.ru> On Thu, Mar 30, 2006 at 12:09:51AM -0800, Gregory P. Smith wrote: > http://www.sqlobject.org/module-sqlobject.sqlbuilder.html This automatically generated doc concentrates on the interface instead of usage. SQLBuilder is really a capable and very pythonic SQL generator. But it has an issue - it can generate SQL *strings* only for specific databases - because some SQL constructs are DB-dependent, quoting e.g. Some examples are in the __main__ part of sqlbuilder.py: http://svn.colorstudy.com/SQLObject/trunk/sqlobject/sqlbuilder.py Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From theller at python.net Thu Mar 30 11:02:23 2006 From: theller at python.net (Thomas Heller) Date: Thu, 30 Mar 2006 11:02:23 +0200 Subject: [Python-Dev] Win64 AMD64 (aka x64) binaries available64 In-Reply-To: <43FA4A1B.3030209@v.loewis.de> References: <43FA4A1B.3030209@v.loewis.de> Message-ID: Martin v. L?wis wrote: > I have now produces a snapshot of a Win64 build for AMD64 > processors (also known as EM64T or x64); this is different > from IA-64 (which is also known as Itanium)... > > Anyway, the binaries are > > http://www.dcl.hpi.uni-potsdam.de/home/loewis/python-2.5.13199.amd64.msi > > This is from today's trunk. If you have general remarks/discussion, > please post to python-dev. If you have specific bug reports, file > them on SF. Bug fixes are particularly welcome. Is this no longer available? BTW: When I build Python for ReleaseAMD64 myself, the exe crashes at the first Py_BuildValue call. Thomas From tds333+pydev at gmail.com Thu Mar 30 11:48:21 2006 From: tds333+pydev at gmail.com (Wolfgang Langner) Date: Thu, 30 Mar 2006 11:48:21 +0200 Subject: [Python-Dev] I'm not getting email from SF when assignedabug/patch In-Reply-To: References: <00ff01c6523f$7c583290$bf03030a@trilan> <1143554576.2310.161.camel@geddy.wooz.org> <17449.21764.475497.457592@montanaro.dyndns.org> <44298954.4040705@v.loewis.de> <1143648409.10799.159.camel@resist.wooz.org> <4c45c1530603290857o50a10d0cnef21922927d5810@mail.gmail.com> Message-ID: <4c45c1530603300148l63450c2eh9132ba687f515448@mail.gmail.com> Hello, On 3/29/06, Guido van Rossum wrote: > On 3/29/06, Wolfgang Langner wrote: > > It is a Java system. Why promote Java solutions for python ? > > I think there are good python solutions for a bug tracker and we > > should prefer them. > ... > Also, we're supposed to be friendly with Java, as we have a major > product in that arena. What if Java folks were to reject a Python > solution because it's not written in Python? Wouldn't we be upset > about the parochialism? I have no problem with Java, neither with other languages. Most time I have to program in C++ and sometimes in Java. I only suggest to give python solutions a try even if they are not yet equal in quality. But they should be better than SF. > The language choice should only be used as an argument if all else is > equal. Of course, "hackability" of a particular solution may be a > criterion too, and there the language choice could matter. But the > above response sounded like a knee-jerk to me, and IMO needs to be > rebutted. I think there are good python solutions and please don't underrate the marketing effect. My opinion is only subjective and to choose one system we must define some criteria's to decide the "best" one. But something other :-) Even Ruby on Rails uses Trac: http://dev.rubyonrails.org/wiki -- bye by Wolfgang From greg.ewing at canterbury.ac.nz Thu Mar 30 12:38:30 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 30 Mar 2006 22:38:30 +1200 Subject: [Python-Dev] Class decorators In-Reply-To: <20060330040956.GJ5787@performancedrivers.com> References: <5.1.1.6.0.20060329142653.037617a8@mail.telecommunity.com> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com> <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> <200603282335.09341.fdrake@acm.org> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <5.1.1.6.0.20060329142653.037617a8@mail.telecommunity.com> <5.1.1.6.0.20060329201639.01e43970@mail.telecommunity.com> <20060330040956.GJ5787@performancedrivers.com> Message-ID: <442BB526.9070504@canterbury.ac.nz> Jack Diederich wrote: > Classes have a unique property in that they are the easiest way to make > little namespaces in python. For a while now, I've been wondering whether it would be worth having a construct purely for creating little namespaces, instead of abusing a class for this. I've been thinking about an 'instance' statement that creates an instance of a class: instance my_thing(MyClass): # attribute assignments go here -- Greg From greg.ewing at canterbury.ac.nz Thu Mar 30 12:44:32 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 30 Mar 2006 22:44:32 +1200 Subject: [Python-Dev] Class decorators In-Reply-To: <5.1.1.6.0.20060330014323.01ed8e68@mail.telecommunity.com> References: <5.1.1.6.0.20060329201639.01e43970@mail.telecommunity.com> <5.1.1.6.0.20060329142653.037617a8@mail.telecommunity.com> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com> <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> <200603282335.09341.fdrake@acm.org> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <5.1.1.6.0.20060329142653.037617a8@mail.telecommunity.com> <5.1.1.6.0.20060329201639.01e43970@mail.telecommunity.com> <5.1.1.6.0.20060330014323.01ed8e68@mail.telecommunity.com> Message-ID: <442BB690.1090509@canterbury.ac.nz> Phillip J. Eby wrote: > Are you actually *using* this IOClass thing, or is this just a > hypothetical proposal? I'm using it. It's not hypothetical. Putting all the info I want in the decorator itself wouldn't be very nice in my case, or at least that's my opinion. One issue is that I'm also abusing a class statement to create a namespace for my extra info. Inserting that as a decorator argument obviously wouldn't be possible... -- Greg From raymond.hettinger at verizon.net Thu Mar 30 12:56:23 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Thu, 30 Mar 2006 05:56:23 -0500 Subject: [Python-Dev] Class decorators References: <5.1.1.6.0.20060329142653.037617a8@mail.telecommunity.com><5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com><5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com><5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com><200603282335.09341.fdrake@acm.org><5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com><5.1.1.6.0.20060329142653.037617a8@mail.telecommunity.com><5.1.1.6.0.20060329201639.01e43970@mail.telecommunity.com><20060330040956.GJ5787@performancedrivers.com> <442BB526.9070504@canterbury.ac.nz> Message-ID: <009e01c653e8$965b9420$6601a8c0@RaymondLaptop1> [Jack Diederich] >> Classes have a unique property in that they are the easiest way to make >> little namespaces in python. [Greg Ewing] > For a while now, I've been wondering whether it would > be worth having a construct purely for creating little > namespaces, instead of abusing a class for this. FWIW, I do not consider it an abuse to use a class to create a small namespace. Essentially that is what it is for -- it matters not whether the class has no methods. What I've been wanting is the ability to easily define functions directly into an existing namespace: class A: pass def A.mymethod(x): return x+1 The latter definition is equivalent to: A.mymethod = lambda(x): x+1 This ability to inject function definitions into an existing namespace is the one missing piece to enable clean use of Protocol OO techniques: a = Object.copy() b = a.copy() def b.f(x): return x+1 c = b.copy() def c.f(x): return x+2 Raymond From greg.ewing at canterbury.ac.nz Thu Mar 30 13:11:19 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 30 Mar 2006 23:11:19 +1200 Subject: [Python-Dev] building sql queries in python In-Reply-To: <9e804ac0603300038g3fe3aa36l396d0c0fc7abdfb2@mail.gmail.com> References: <200603281759.18756.anthony@interlink.com.au> <17451.13615.74425.528082@montanaro.dyndns.org> <442B48AE.6010105@canterbury.ac.nz> <200603292207.31378.fdrake@acm.org> <442B4EC0.8050809@canterbury.ac.nz> <20060330080951.GH31651@zot.electricrain.com> <9e804ac0603300038g3fe3aa36l396d0c0fc7abdfb2@mail.gmail.com> Message-ID: <442BBCD7.3070901@canterbury.ac.nz> Thomas Wouters wrote: > > Have you looked at SqlObject? (and its associated modules > sqlobject.sqlbuilder in particular) > > SQLAlchemy (www.sqlalchemy.org ) is also > nice, in particular for more complex setups. > > There's plenty of ways to reliably and sanely > avoid typing actual SQL in Python. Well, yes, but they all seem to be a bit hackish in one way or another -- e.g. SQLAlchemy's requirement to use &, |, ~ instead of and, or, not. (That's one of the main motivations behind my Overloadable Boolean Operators PEP, by the way.) I'd like to see some way of attacking this problem head-on, rather than endlessly looking for convoluted ways around it. -- Greg From Ben.Young at risk.sungard.com Thu Mar 30 13:04:39 2006 From: Ben.Young at risk.sungard.com (Ben.Young at risk.sungard.com) Date: Thu, 30 Mar 2006 11:04:39 +0000 Subject: [Python-Dev] Class decorators In-Reply-To: <442BB526.9070504@canterbury.ac.nz> Message-ID: python-dev-bounces+python=theyoungfamily.co.uk at python.org wrote on 30/03/2006 11:38:30: > Jack Diederich wrote: > > > Classes have a unique property in that they are the easiest way to make > > little namespaces in python. > > For a while now, I've been wondering whether it would > be worth having a construct purely for creating little > namespaces, instead of abusing a class for this. > > I've been thinking about an 'instance' statement that > creates an instance of a class: > > instance my_thing(MyClass): > > # attribute assignments go here Maybe this would be a use for the proposal a while back where: 'statement' name(args): ... implied name = 'statement'("name", args, namespace) then we could have: namespace foo: ... and interface IFoo(IBar): ... and your instance my_thing(MyClass): ... ? Cheers, Ben > > -- > Greg > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python- > dev/python%40theyoungfamily.co.uk > From greg.ewing at canterbury.ac.nz Thu Mar 30 13:20:58 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 30 Mar 2006 23:20:58 +1200 Subject: [Python-Dev] Class decorators In-Reply-To: <009e01c653e8$965b9420$6601a8c0@RaymondLaptop1> References: <5.1.1.6.0.20060329142653.037617a8@mail.telecommunity.com> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <5.1.1.6.0.20060328133015.0374d9f8@mail.telecommunity.com> <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> <200603282335.09341.fdrake@acm.org> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <5.1.1.6.0.20060329142653.037617a8@mail.telecommunity.com> <5.1.1.6.0.20060329201639.01e43970@mail.telecommunity.com> <20060330040956.GJ5787@performancedrivers.com> <442BB526.9070504@canterbury.ac.nz> <009e01c653e8$965b9420$6601a8c0@RaymondLaptop1> Message-ID: <442BBF1A.7090904@canterbury.ac.nz> Raymond Hettinger wrote: > FWIW, I do not consider it an abuse to use a class to create a small > namespace. Essentially that is what it is for -- it matters not whether > the class has no methods. Two problems with that: * The word "class" in front of it is a misnomer if you've no intention of using it as a class. * It's not a completely blank slate -- it comes with various baggage in the form of __xxx__ attributes (my registration function has to filter those out). Plus some unwanted behaviour such as being callable, and doing magic things with attribute access when the attribute is a descriptor. -- Greg From greg.ewing at canterbury.ac.nz Thu Mar 30 13:30:06 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 30 Mar 2006 23:30:06 +1200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: References: <200603290152.00598.anthony@interlink.com.au> <200603281759.18756.anthony@interlink.com.au> <1143554351.2309.158.camel@geddy.wooz.org> <5.1.1.6.0.20060329144142.0390e5d8@mail.telecommunity.com> <17451.882.842146.979286@montanaro.dyndns.org> <17451.12090.257751.636336@montanaro.dyndns.org> <442B46D6.70703@canterbury.ac.nz> Message-ID: <442BC13E.5000503@canterbury.ac.nz> Fredrik Lundh wrote: > Greg Ewing wrote: > > > Firebird could be a solution to this. > > so a library that doesn't support multiple independent > readers/writers on a single file at all is much better > than one that does, Where do you get that from? Firebird supports multiple readers/writers perfectly well. Greg From mal at egenix.com Thu Mar 30 13:31:04 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 30 Mar 2006 13:31:04 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <06Mar29.141411pst."58633"@synergy1.parc.xerox.com> References: <06Mar29.141411pst."58633"@synergy1.parc.xerox.com> Message-ID: <442BC178.3000704@egenix.com> Bill Janssen wrote: >> from a user perspective, adding this to the standard library is a no-brainer. >> the only reason not to add it would be if the release managers don't have >> time to sort out the build issues. > > I agree with Fredrik here. > > On the package naming issue: using "em" for "email" would be wrong, > just as "db" for "database" would be wrong. I might change my mind if > all stdlib packages were under some toplevel package, like "stdlib". I don't really care about the name, but please be aware that you are talking about adding a *very* popular module name to the top-level Python namespace if you go for "db" or "database". Why can't we just have the pysqlite package as top-level package, with a slight change in name to prevent clashes with the external distribution, e.g. sqlite ?! Such a module name is less likely to cause problems. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 30 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From fredrik at pythonware.com Thu Mar 30 13:41:21 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 30 Mar 2006 13:41:21 +0200 Subject: [Python-Dev] pysqlite for 2.5? References: <200603290152.00598.anthony@interlink.com.au><200603281759.18756.anthony@interlink.com.au><1143554351.2309.158.camel@geddy.wooz.org><5.1.1.6.0.20060329144142.0390e5d8@mail.telecommunity.com><17451.882.842146.979286@montanaro.dyndns.org><17451.12090.257751.636336@montanaro.dyndns.org><442B46D6.70703@canterbury.ac.nz> <442BC13E.5000503@canterbury.ac.nz> Message-ID: Greg Ewing wrote: >> > Firebird could be a solution to this. >> >> so a library that doesn't support multiple independent > > readers/writers on a single file at all is much better > > than one that does, > > Where do you get that from? Firebird supports > multiple readers/writers perfectly well. not according to the documentation (which says that the embedded library locks the database file, to prevent other independent processes from accessing the data). but that might of course be misleading or outdated information. do you have a URL that explains how the embedded Firebird component handles concurrency ? (something comparable to http://www.sqlite.org/lockingv3.html, that is.) From andrew-pydev at lexical.org.uk Thu Mar 30 13:25:44 2006 From: andrew-pydev at lexical.org.uk (Andrew Walkingshaw) Date: Thu, 30 Mar 2006 12:25:44 +0100 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <4c45c1530603290724g12f656d6j295841be2cf78f1a@mail.gmail.com> References: <200603281759.18756.anthony@interlink.com.au> <200603291947.34461.anthony@interlink.com.au> <1143638252.2317.184.camel@geddy.wooz.org> <442A8D34.8060200@ghaering.de> <4c45c1530603290724g12f656d6j295841be2cf78f1a@mail.gmail.com> Message-ID: <20060330112544.GA60869@colon.colondot.net> On Wed, Mar 29, 2006 at 05:24:32PM +0200, Wolfgang Langner wrote: > -1 on db.sql.sqlite. > Keep structure flat. Or we are eventually in a Java world with > org.something.this.andthat .... xml.dom.minidom? -- http://www.lexical.org.uk/ | http://covertmusic.com/ | work: adw27 at cam.ac.uk "interrupting my train of thought" From g.brandl at gmx.net Thu Mar 30 14:01:25 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 30 Mar 2006 14:01:25 +0200 Subject: [Python-Dev] Class decorators In-Reply-To: References: <442BB526.9070504@canterbury.ac.nz> Message-ID: Ben.Young at risk.sungard.com wrote: > python-dev-bounces+python=theyoungfamily.co.uk at python.org wrote on > 30/03/2006 11:38:30: > >> Jack Diederich wrote: >> >> > Classes have a unique property in that they are the easiest way to > make >> > little namespaces in python. >> >> For a while now, I've been wondering whether it would >> be worth having a construct purely for creating little >> namespaces, instead of abusing a class for this. >> >> I've been thinking about an 'instance' statement that >> creates an instance of a class: >> >> instance my_thing(MyClass): >> >> # attribute assignments go here > > Maybe this would be a use for the proposal a while back where: > > 'statement' name(args): > ... > > implied > > name = 'statement'("name", args, namespace) [...] I like that generalization (since a class definition statement currently does about the same anyway). However, please post that to the python-3000 list as this would be a change for Python 3. Cheers, Georg From anthony at interlink.com.au Thu Mar 30 14:03:23 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu, 30 Mar 2006 23:03:23 +1100 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <442BC178.3000704@egenix.com> References: <06Mar29.141411pst."58633"@synergy1.parc.xerox.com> <442BC178.3000704@egenix.com> Message-ID: <200603302303.28587.anthony@interlink.com.au> On Thursday 30 March 2006 22:31, M.-A. Lemburg wrote: > I don't really care about the name, but please be aware that > you are talking about adding a *very* popular module name to > the top-level Python namespace if you go for "db" or "database". > > Why can't we just have the pysqlite package as top-level > package, with a slight change in name to prevent clashes > with the external distribution, e.g. sqlite ?! > > Such a module name is less likely to cause problems. Excellent point. Hm. Maybe we should just go with 'sqlite', instead. Anyway, at the moment, there's 'db.sqlite' all checked in and working on a branch, at svn+ssh://pythondev at svn.python.org/python/branches/sqlite-integration or, if you use a readonly version http://svn.python.org/python/branches/sqlite-integration (you can use 'svn switch URL' to change a current trunk checkout to the branch without having to checkout a whole new version). It is from sqlite 2.1.3. Gerhard is cutting a 2.2.0 which reduces the requirement for the version of sqlite3 that is required. Currently, it needs 3.2.2 or later. There's tests (which pass), setup.py magic to find a correct sqlite3 version, and the like. Still to do: Windows buildproj Documentation Upgrade to the updated pysqlite once it's out maybe switch from db.sqlite to just sqlite (trivial enough change). Anthony -- Anthony Baxter It's never too late to have a happy childhood. From anthony at interlink.com.au Thu Mar 30 14:05:24 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu, 30 Mar 2006 23:05:24 +1100 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <20060330112544.GA60869@colon.colondot.net> References: <200603281759.18756.anthony@interlink.com.au> <4c45c1530603290724g12f656d6j295841be2cf78f1a@mail.gmail.com> <20060330112544.GA60869@colon.colondot.net> Message-ID: <200603302305.25863.anthony@interlink.com.au> On Thursday 30 March 2006 22:25, Andrew Walkingshaw wrote: > On Wed, Mar 29, 2006 at 05:24:32PM +0200, Wolfgang Langner wrote: > > -1 on db.sql.sqlite. > > Keep structure flat. Or we are eventually in a Java world with > > org.something.this.andthat .... > > xml.dom.minidom? given the horror of _xmlplus/xmlcore and whatnot, I'd be hesitant to use the xml package as an example of _anything_ From fredrik at pythonware.com Thu Mar 30 14:07:39 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 30 Mar 2006 14:07:39 +0200 Subject: [Python-Dev] pysqlite for 2.5? References: <200603281759.18756.anthony@interlink.com.au><4c45c1530603290724g12f656d6j295841be2cf78f1a@mail.gmail.com><20060330112544.GA60869@colon.colondot.net> <200603302305.25863.anthony@interlink.com.au> Message-ID: Anthony Baxter wrote: >> xml.dom.minidom? > > given the horror of _xmlplus/xmlcore and whatnot, I'd be hesitant to > use the xml package as an example of _anything_ which reminds me -- is that issue still open ? martin? fred? From anthony at interlink.com.au Thu Mar 30 14:48:36 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu, 30 Mar 2006 23:48:36 +1100 Subject: [Python-Dev] _xmlplus fixup for 2.5 In-Reply-To: References: <200603281759.18756.anthony@interlink.com.au> <200603302305.25863.anthony@interlink.com.au> Message-ID: <200603302348.41991.anthony@interlink.com.au> On Thursday 30 March 2006 23:07, Fredrik Lundh wrote: > Anthony Baxter wrote: > >> xml.dom.minidom? > > > > given the horror of _xmlplus/xmlcore and whatnot, I'd be hesitant > > to use the xml package as an example of _anything_ > > which reminds me -- is that issue still open ? martin? fred? It looks to me like it's fixed in SVN. http://mail.python.org/pipermail/python-dev/2005-December/058710.html Interestingly, 15 minutes ago I was helping the Ubuntu python packager diagnose exactly the test failure mentioned in this email... Anthony -- Anthony Baxter It's never too late to have a happy childhood. From fredrik at pythonware.com Thu Mar 30 14:55:37 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 30 Mar 2006 14:55:37 +0200 Subject: [Python-Dev] _xmlplus fixup for 2.5 References: <200603281759.18756.anthony@interlink.com.au><200603302305.25863.anthony@interlink.com.au> <200603302348.41991.anthony@interlink.com.au> Message-ID: Anthony Baxter wrote: > It looks to me like it's fixed in SVN. > > http://mail.python.org/pipermail/python-dev/2005-December/058710.html the issue isn't the one in that message though; it's the one in this message: http://mail.python.org/pipermail/python-dev/2005-December/058752.html "I don't agree with the change. You just broke source compatibility between the core package and PyXML." which, I fear, is still unresolved. or ? From anthony at interlink.com.au Thu Mar 30 15:26:50 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri, 31 Mar 2006 00:26:50 +1100 Subject: [Python-Dev] (finally) getting around to killing __private names from stdlib Message-ID: <200603310026.52009.anthony@interlink.com.au> I originally raised this a metric age or four ago, and promptly forgot about it - http://mail.python.org/pipermail/python-dev/2004-August/048444.html The concensus seemed to be "ok, remove them, but carefully!" I've started doing this, and will post a patch on SF when it's done. Before I do this, does anyone want to scream? (see also http://mail.python.org/pipermail/python-dev/2005-December/058555.html ) -- Anthony Baxter It's never too late to have a happy childhood. From barry at python.org Thu Mar 30 16:02:11 2006 From: barry at python.org (Barry Warsaw) Date: Thu, 30 Mar 2006 09:02:11 -0500 Subject: [Python-Dev] I'm not getting email from SF when assignedabug/patch In-Reply-To: References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> <1143554576.2310.161.camel@geddy.wooz.org> <17449.21764.475497.457592@montanaro.dyndns.org> <44298954.4040705@v.loewis.de> <1143648409.10799.159.camel@resist.wooz.org> Message-ID: <1143727331.2310.218.camel@geddy.wooz.org> On Wed, 2006-03-29 at 23:33 -0800, Neal Norwitz wrote: > I'm in favor of having Atlassian setup a system to be used for 3k. It > would be completely experimental and could be completely thrown away > which should be made clear to Atlassian if we were to do this. I > would use the system for evaluation. If we did this, we should also > try to ensure we can get all the data out, just in case we want to > preserve any of it. I'm sure we can get the data out (unlike SF :), and my guess would be that they'd be fine with such an experimental relationship. They're very cool folks. I'll leave it up to Brett to decide though, since he's chair of the infrastructure committee. I'm happy to ask them though. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060330/01bef677/attachment.pgp From barry at python.org Thu Mar 30 16:06:43 2006 From: barry at python.org (Barry Warsaw) Date: Thu, 30 Mar 2006 09:06:43 -0500 Subject: [Python-Dev] I'm not getting email from SF whenassignedabug/patch In-Reply-To: References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> <1143554576.2310.161.camel@geddy.wooz.org> <17449.21764.475497.457592@montanaro.dyndns.org> <44298954.4040705@v.loewis.de> <1143648409.10799.159.camel@resist.wooz.org> Message-ID: <1143727603.2317.224.camel@geddy.wooz.org> On Thu, 2006-03-30 at 09:55 +0200, Fredrik Lundh wrote: > so what's the advantage of a freely hosted Atlassian setup compared > to a freely hosted Trac setup ? Dunno. I'm sure both will accomplish the job and both will be better than the current situation. I've used Jira and Confluence for almost two years now on a daily basis, and I can personally say that they are among the most intuitive and pleasant to use systems of their kind that I've used. > from what I can tell, no big contemporary Python project use Atlassian. > they all use Trac. there are thousands of Python developers out there > that are used to working with Trac. FWIW, Atlassian has made the same generous offer for Mailman, and I am going to accept it (assuming we can ever get the freakin' data out of SF). -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060330/99dc8e23/attachment.pgp From barry at python.org Thu Mar 30 16:08:13 2006 From: barry at python.org (Barry Warsaw) Date: Thu, 30 Mar 2006 09:08:13 -0500 Subject: [Python-Dev] I'm not getting email from SF whenassignedabug/patch In-Reply-To: References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> <1143554576.2310.161.camel@geddy.wooz.org> <17449.21764.475497.457592@montanaro.dyndns.org> <44298954.4040705@v.loewis.de> <1143648409.10799.159.camel@resist.wooz.org> Message-ID: <1143727693.2317.226.camel@geddy.wooz.org> On Thu, 2006-03-30 at 10:39 +0200, Georg Brandl wrote: > Perhaps that Jira is commercial, so it is out of the question for most > open-source Python applications. Sorry, I don't follow. Why is a commercial product out of the question for Python? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060330/cbf850f1/attachment-0001.pgp From g.brandl at gmx.net Thu Mar 30 16:20:36 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 30 Mar 2006 16:20:36 +0200 Subject: [Python-Dev] I'm not getting email from SF whenassignedabug/patch In-Reply-To: <1143727693.2317.226.camel@geddy.wooz.org> References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> <1143554576.2310.161.camel@geddy.wooz.org> <17449.21764.475497.457592@montanaro.dyndns.org> <44298954.4040705@v.loewis.de> <1143648409.10799.159.camel@resist.wooz.org> <1143727693.2317.226.camel@geddy.wooz.org> Message-ID: Barry Warsaw wrote: > On Thu, 2006-03-30 at 10:39 +0200, Georg Brandl wrote: > >> Perhaps that Jira is commercial, so it is out of the question for most >> open-source Python applications. > > Sorry, I don't follow. Why is a commercial product out of the question > for Python? What I answered to was: """ from what I can tell, no big contemporary Python project use Atlassian. they all use Trac. there are thousands of Python developers out there that are used to working with Trac. I'm obviously missing something here. """ I'm not saying it's out of the question for Python, I'm saying that it's out of the question for most open-source projects, which don't have the money or don't want to spend the money on a mere bug tracker, and that this may be the reason that Jira isn't used widely by Python programmers. Georg From mal at egenix.com Thu Mar 30 17:04:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 30 Mar 2006 17:04:24 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <200603302303.28587.anthony@interlink.com.au> References: <06Mar29.141411pst."58633"@synergy1.parc.xerox.com> <442BC178.3000704@egenix.com> <200603302303.28587.anthony@interlink.com.au> Message-ID: <442BF378.5060708@egenix.com> Anthony Baxter wrote: > On Thursday 30 March 2006 22:31, M.-A. Lemburg wrote: >> I don't really care about the name, but please be aware that >> you are talking about adding a *very* popular module name to >> the top-level Python namespace if you go for "db" or "database". >> >> Why can't we just have the pysqlite package as top-level >> package, with a slight change in name to prevent clashes >> with the external distribution, e.g. sqlite ?! >> >> Such a module name is less likely to cause problems. > > Excellent point. Hm. Maybe we should just go with 'sqlite', instead. Anything, but please no "db" or "database" top-level module or package :-) > Anyway, at the moment, there's 'db.sqlite' all checked in and working > on a branch, at > svn+ssh://pythondev at svn.python.org/python/branches/sqlite-integration > or, if you use a readonly version > http://svn.python.org/python/branches/sqlite-integration > > (you can use 'svn switch URL' to change a current trunk checkout to > the branch without having to checkout a whole new version). > > It is from sqlite 2.1.3. Gerhard is cutting a 2.2.0 which reduces the > requirement for the version of sqlite3 that is required. Currently, > it needs 3.2.2 or later. There's tests (which pass), setup.py magic > to find a correct sqlite3 version, and the like. Still to do: > > Windows buildproj > Documentation > Upgrade to the updated pysqlite once it's out > maybe switch from db.sqlite to just sqlite (trivial enough change). I take it that this is not going to go into 2.5a1 ?! Also your statement regarding sqlite3 suggests that sqlite itself is not included - why not ? Isn't the main argument for having pysqlite included in the core to be able to play around with SQL without relying on external libraries ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 30 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From gh at ghaering.de Thu Mar 30 17:28:24 2006 From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=) Date: Thu, 30 Mar 2006 17:28:24 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <442BF378.5060708@egenix.com> References: <06Mar29.141411pst."58633"@synergy1.parc.xerox.com> <442BC178.3000704@egenix.com> <200603302303.28587.anthony@interlink.com.au> <442BF378.5060708@egenix.com> Message-ID: <442BF918.5060706@ghaering.de> M.-A. Lemburg wrote: > [...] > Also your statement regarding sqlite3 suggests that sqlite > itself is not included - why not ? - SQLite sources are 1.57 MiB uncompressed, we wouldn't want to add that to the Python sources download size, would we? - I personally would not want to have the job to "bless" a certain version of SQLite for being bugfree enough to be used until the next Python minor release. And we wouldn't want to push a Python minor release just somebody found an obscure data corroption bug in a SQLite release - SQLite might not compile on some less common platforms (AIX, HP/UX, Win64, whatever) that Python compiles fine on. - I believe Python is written in more portable C than SQLite. So it might be certain compilers that fail for compiling SQLite. - At some point you might also want a sqlite commandline shell instead of just the shared library, too. All of these are non-issues if we just compile against an installed SQLite on Unix-like system if it can be found. On Windows, I also prefer to have a dynamically linked SQLite Python module. We can distribute the SQLITE3.DLL with Python, and then people could just download an updated SQLITE.DLL from http://sqlite.org/ and overwrite the existing one of the Python installation, *if* an important bug is fixed in SQLite. > Isn't the main argument for having pysqlite included in the > core to be able to play around with SQL without relying > on external libraries ? This, and that you can prototype without having to install and configure a database server. For many applications, the prototype can be the final version. -- Gerhard From fredrik at pythonware.com Thu Mar 30 17:30:14 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 30 Mar 2006 17:30:14 +0200 Subject: [Python-Dev] I'm not getting email from SFwhenassignedabug/patch References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> <1143554576.2310.161.camel@geddy.wooz.org> <17449.21764.475497.457592@montanaro.dyndns.org> <44298954.4040705@v.loewis.de> <1143648409.10799.159.camel@resist.wooz.org> <1143727693.2317.226.camel@geddy.wooz.org> Message-ID: Georg Brandl wrote: > What I answered to was: > > """ > from what I can tell, no big contemporary Python project use Atlassian. > they all use Trac. there are thousands of Python developers out there > that are used to working with Trac. > > I'm obviously missing something here. > """ > > I'm not saying it's out of the question for Python, I'm saying that it's > out of the question for most open-source projects, which don't have the > money or don't want to spend the money on a mere bug tracker, and that > this may be the reason that Jira isn't used widely by Python programmers. Like most commercial tool providers that complete on a market full of open source tools of various quality, Atlassian offers free licenses to non-profits and established open source projects: http://www.atlassian.com/software/jira/pricing.jsp#nonprofit Perforce (who makes an excellent version management system) is another example. We went through very similar discussions back when we moved python from cvs to svn, and ended up picking an open source system that was good enough over a better commercial product, despite the fact that several key developers had extensive experiences of using perforce for large-scale projects. And that's the whole point, of course: if something is widely used by others and is good enough, a project needs very good reasons to pick another tool. From barry at python.org Thu Mar 30 17:36:20 2006 From: barry at python.org (Barry Warsaw) Date: Thu, 30 Mar 2006 10:36:20 -0500 Subject: [Python-Dev] I'm not getting email from SF whenassignedabug/patch In-Reply-To: References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> <1143554576.2310.161.camel@geddy.wooz.org> <17449.21764.475497.457592@montanaro.dyndns.org> <44298954.4040705@v.loewis.de> <1143648409.10799.159.camel@resist.wooz.org> <1143727693.2317.226.camel@geddy.wooz.org> Message-ID: <1143732980.6694.64.camel@resist.wooz.org> On Thu, 2006-03-30 at 16:20 +0200, Georg Brandl wrote: > I'm not saying it's out of the question for Python, I'm saying that it's > out of the question for most open-source projects, which don't have the > money or don't want to spend the money on a mere bug tracker, and that > this may be the reason that Jira isn't used widely by Python programmers. Gotcha, sorry I misunderstood. BTW, Atlassian does have a program to provide free licenses to open source projects. Here's their FAQ item on it: http://www.atlassian.com/about/licensing/faq.jsp#open_source -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060330/606bfe83/attachment.pgp From anthony at interlink.com.au Thu Mar 30 17:44:28 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri, 31 Mar 2006 02:44:28 +1100 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <442BF378.5060708@egenix.com> References: <06Mar29.141411pst."58633"@synergy1.parc.xerox.com> <200603302303.28587.anthony@interlink.com.au> <442BF378.5060708@egenix.com> Message-ID: <200603310244.33110.anthony@interlink.com.au> On Friday 31 March 2006 02:04, M.-A. Lemburg wrote: > > Excellent point. Hm. Maybe we should just go with 'sqlite', > > instead. > > Anything, but please no "db" or "database" top-level module or > package :-) How about "sql"? I can't think of a better name right now - can anyone else, or should it just go in the top level as 'sqlite'? > I take it that this is not going to go into 2.5a1 ?! Well, right now the major missing bits for landing it right now are the windows build project and the documentation. I'm pretty comfortable with landing it for a1. It has tests, I've knitted these into the Python regression testing suite and they're all passing fine. I've tested building on systems with a version of sqlite that is acceptable, with no sqlite, and with an old version of sqlite, and the build process handles it all correctly. > Also your statement regarding sqlite3 suggests that sqlite > itself is not included - why not ? For the same reasons we don't include the BerkeleyDB library. Many, many modern operating systems now ship with libsqlite3 (just as they ship with bsddb). While sqlite is nowhere near the size of BerkeleyDB, it's still a non-trivial amount of code. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From barry at python.org Thu Mar 30 17:54:09 2006 From: barry at python.org (Barry Warsaw) Date: Thu, 30 Mar 2006 10:54:09 -0500 Subject: [Python-Dev] PySet API In-Reply-To: <004c01c653af$c8017a00$6601a8c0@RaymondLaptop1> References: <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> <1143468446.10799.50.camel@resist.wooz.org> <200603281507.18175.gmccaughan@synaptics-uk.com> <001a01c65377$cd099a10$6601a8c0@RaymondLaptop1> <1143678849.6694.52.camel@resist.wooz.org> <004c01c653af$c8017a00$6601a8c0@RaymondLaptop1> Message-ID: <1143734049.7204.79.camel@resist.wooz.org> On Wed, 2006-03-29 at 23:09 -0500, Raymond Hettinger wrote: > Yes, _PySet_Next() is a good compromise for you and me -- it saves you from > writing a hack and saves my API from including a bug factory. The only issue is > that Martin thinks it to be a crummy idea. Personally, I have no problem with > putting-in an undocumented hook for daring people who aspire to swim in > quicksand ;-) Of course if it was "just" a bug factory I might agree. But since it's instead a powerful tool that can be misused if misunderstood, I'd tend to want to document it and explain to people where and why it might or might not be the right hammer for the nail you're trying to pound in. But that's just me. :) > The idea is not yet ready for prime-time. If I do it for one of the named > operations, I will do it for all (to keep the interface uniform). Which named operations are you thinking of? > I haven't yet > had the time to work-out the math on whether it would be worthwhile and provide > some differential advantage over simply repeating the same operation several > times over. My research question is whether work can be saved by controlling > the order of operations -- the concept is somewhat like optimizing multi-term > matrix multiplication where the total work effort can vary dramatically > depending on which matrices are multiplied together first, A((BC)D) vs (AB)(CD) > vs (A(BC))D etc. Put in business terms, the question is whether I'm able to > leverage the associative and commutative properties of some chained set > operations. FWIW, the module already has optimizations to take advantage of > the commutative property of binary AND, OR, and SYMMETRIC_DIFFERENCE operations. > However, the multi-term optimization probably wait until Py2.6 -- it is too > experimental for now. Does that mean you want to make sure the function is insanely fast before you'll add it? Shouldn't you instead decide whether there's even a need for vararg update first and then figure out how to optimize it? IOW, if there's a need for vararg update, let's add the API now so that people can start using it, even if it's not as fast as it could be. Then they'll be especially grateful when you figure out how to make it insanely fast in Python 2.6. If vararg update isn't useful, then there's no point in adding the API, even if it can be made insanely fast. You'd just be wasting your time because no one would use it. It seems backwards to design the implementation first and then the API. An API represents how you want people to use your objects, what operations and semantics you want it to have, what contracts you're guaranteeing and so on. Optimization then is a very nice side benefit. Let me ask this: if you can't make vararg PySet_Update() insanely fast, does that mean you won't add a vararg version? Or you won't add the function at all? I'm all for making things fast, but I just don't believe that in general that should be the primary driver for how you want people to /use/ your objects. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060330/5b9db9d8/attachment.pgp From eric.frost at mp2kmag.com Thu Mar 30 17:55:23 2006 From: eric.frost at mp2kmag.com (Eric Frost) Date: Thu, 30 Mar 2006 09:55:23 -0600 Subject: [Python-Dev] new article - MapPoint and Python Message-ID: http://www.mp2kmag.com/a131--python.pythonwin.mappoint.html Let me know if anyone would like to publish this short article on their web site. website @ MP2Kmag dot com for permission and instructions. Thanks, Eric From fredrik at pythonware.com Thu Mar 30 17:50:13 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 30 Mar 2006 17:50:13 +0200 Subject: [Python-Dev] pysqlite for 2.5? References: <06Mar29.141411pst."58633"@synergy1.parc.xerox.com><442BC178.3000704@egenix.com> <200603302303.28587.anthony@interlink.com.au> Message-ID: Anthony Baxter wrote: > > Such a module name is less likely to cause problems. > > Excellent point. Hm. Maybe we should just go with 'sqlite', instead. except that "sqlite" was the name used by the first pysqlite generation: $ python2.3 >>> import sqlite >>> sqlite.version '1.1.6' I'm not sure how much trouble reusing this name would cause, but I'm quite sure the amount of trouble is larger than "none at all". From mal at egenix.com Thu Mar 30 18:08:40 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 30 Mar 2006 18:08:40 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <200603310244.33110.anthony@interlink.com.au> References: <06Mar29.141411pst."58633"@synergy1.parc.xerox.com> <200603302303.28587.anthony@interlink.com.au> <442BF378.5060708@egenix.com> <200603310244.33110.anthony@interlink.com.au> Message-ID: <442C0288.8080502@egenix.com> Anthony Baxter wrote: > On Friday 31 March 2006 02:04, M.-A. Lemburg wrote: >>> Excellent point. Hm. Maybe we should just go with 'sqlite', >>> instead. >> Anything, but please no "db" or "database" top-level module or >> package :-) > > > How about "sql"? > > I can't think of a better name right now - can anyone else, or should > it just go in the top level as 'sqlite'? I think sqlite is just fine. >> I take it that this is not going to go into 2.5a1 ?! > > Well, right now the major missing bits for landing it right now are > the windows build project and the documentation. I'm pretty > comfortable with landing it for a1. It has tests, I've knitted these > into the Python regression testing suite and they're all passing > fine. I've tested building on systems with a version of sqlite that > is acceptable, with no sqlite, and with an old version of sqlite, and > the build process handles it all correctly. Will it also work with e.g. sqlite 2.8.15 (ie. sqlite < v3) - this is the standard version on SuSE 9.2. >> Also your statement regarding sqlite3 suggests that sqlite >> itself is not included - why not ? > > For the same reasons we don't include the BerkeleyDB library. Many, > many modern operating systems now ship with libsqlite3 (just as they > ship with bsddb). While sqlite is nowhere near the size of > BerkeleyDB, it's still a non-trivial amount of code. If it works with sqlite2 then I agree: these versions are usually available on Unixes. sqlite3 is not as wide-spread yet. What about the Windows build ? Will that contain the necessary DLLs ? Regards, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 30 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From gh at ghaering.de Thu Mar 30 18:09:43 2006 From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=) Date: Thu, 30 Mar 2006 18:09:43 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: References: <06Mar29.141411pst."58633"@synergy1.parc.xerox.com><442BC178.3000704@egenix.com> <200603302303.28587.anthony@interlink.com.au> Message-ID: <442C02C7.8050706@ghaering.de> Fredrik Lundh wrote: > Anthony Baxter wrote: > > >>>Such a module name is less likely to cause problems. >> >>Excellent point. Hm. Maybe we should just go with 'sqlite', instead. > > > except that "sqlite" was the name used by the first pysqlite > generation: > > $ python2.3 > >>>>import sqlite >>>>sqlite.version > > '1.1.6' > > I'm not sure how much trouble reusing this name would cause, but I'm > quite sure the amount of trouble is larger than "none at all". "sqlitedb" is not taken yet. -- Gerhard From gh at ghaering.de Thu Mar 30 18:14:12 2006 From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=) Date: Thu, 30 Mar 2006 18:14:12 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <442C0288.8080502@egenix.com> References: <06Mar29.141411pst."58633"@synergy1.parc.xerox.com> <200603302303.28587.anthony@interlink.com.au> <442BF378.5060708@egenix.com> <200603310244.33110.anthony@interlink.com.au> <442C0288.8080502@egenix.com> Message-ID: <442C03D4.1090000@ghaering.de> M.-A. Lemburg wrote: > Will it also work with e.g. sqlite 2.8.15 (ie. sqlite < v3) - > this is the standard version on SuSE 9.2. No, SQLite 3 has a completely different API than SQLite 2.x. If you need a Python module for it, you can use pysqlite 1.0.1. >>>Also your statement regarding sqlite3 suggests that sqlite >>>itself is not included - why not ? >> >>For the same reasons we don't include the BerkeleyDB library. Many, >>many modern operating systems now ship with libsqlite3 (just as they >>ship with bsddb). While sqlite is nowhere near the size of >>BerkeleyDB, it's still a non-trivial amount of code. > > > If it works with sqlite2 then I agree: these versions are > usually available on Unixes. sqlite3 is not as wide-spread > yet. > > What about the Windows build ? Will that contain the necessary > DLLs ? I proposed to link dynamically on Windows, and ship the Windows SQLite3.DLL. This has two advantages: - Python users can upgrade the SQLite3.DLL by a simple download from in case of emergency - we could make it so that win32 Python core developers (or the Python win32 build process) don't have to build SQLite when building Python -- Gerhard From martin at v.loewis.de Thu Mar 30 18:24:19 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 30 Mar 2006 18:24:19 +0200 Subject: [Python-Dev] Still looking for volunteer to run Windows buildbot In-Reply-To: <20060330063828.GC31651@zot.electricrain.com> References: <441217BB.1000801@v.loewis.de> <20060312002336.GA3935@activestate.com> <1f7befae0603111849n67c40694u7eebf38410b8788e@mail.gmail.com> <20060312224148.GB32648@activestate.com> <1f7befae0603121548k29d18135mbcf0e3365f23c93@mail.gmail.com> <20060330063828.GC31651@zot.electricrain.com> Message-ID: <442C0633.8010800@v.loewis.de> Gregory P. Smith wrote: > FWIW, thats an old BerkeleyDB (yes it'll still work). Python 2.5 > should ship built with BerkeleyDB 4.4.20 so thats what buildbot should > use if it builds the module. The buildbots now fetch bsddb automatically from http://svn.python.org/projects/external/db-4.4.20/ so the buildbot slave admin doesn't need to do anything here. This also has some 4.4.20 patches already applied. Regards, Martin From martin at v.loewis.de Thu Mar 30 18:56:52 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 30 Mar 2006 18:56:52 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: References: <200603281759.18756.anthony@interlink.com.au><4c45c1530603290724g12f656d6j295841be2cf78f1a@mail.gmail.com><20060330112544.GA60869@colon.colondot.net> <200603302305.25863.anthony@interlink.com.au> Message-ID: <442C0DD4.6010003@v.loewis.de> Fredrik Lundh wrote: >>> xml.dom.minidom? >> given the horror of _xmlplus/xmlcore and whatnot, I'd be hesitant to >> use the xml package as an example of _anything_ > > which reminds me -- is that issue still open ? martin? fred? It still is. I wonder whether I should just revert the change. Regards, Martin From martin at v.loewis.de Thu Mar 30 18:59:31 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 30 Mar 2006 18:59:31 +0200 Subject: [Python-Dev] _xmlplus fixup for 2.5 In-Reply-To: <200603302348.41991.anthony@interlink.com.au> References: <200603281759.18756.anthony@interlink.com.au> <200603302305.25863.anthony@interlink.com.au> <200603302348.41991.anthony@interlink.com.au> Message-ID: <442C0E73.7020301@v.loewis.de> Anthony Baxter wrote: > It looks to me like it's fixed in SVN. > > http://mail.python.org/pipermail/python-dev/2005-December/058710.html > > Interestingly, 15 minutes ago I was helping the Ubuntu python packager > diagnose exactly the test failure mentioned in this email... The problem is that now PyXML is no longer maintainable (not that it has been maintained very well, though): The files that used to be identical in PyXML and Python no longer are identical, so keeping them synchronized adds unreasonable maintenance costs (IMO). Regards, Martin From martin at v.loewis.de Thu Mar 30 19:08:37 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 30 Mar 2006 19:08:37 +0200 Subject: [Python-Dev] unicode vs buffer (array) design issue can crash interpreter In-Reply-To: References: Message-ID: <442C1095.9070905@v.loewis.de> Neal Norwitz wrote: > See http://python.org/sf/1454485 for the gory details. Basically if > you create a unicode array (array.array('u')) and try to append an > 8-bit string (ie, not unicode), you can crash the interpreter. > > The problem is that the string is converted without question to a > unicode buffer. Within unicode, it assumes the data to be valid, but > this isn't necessarily the case. We wind up accessing an array with a > negative index and boom. There are several problems combined here, which might need discussion: - why does the 'u#' converter use the buffer interface if available? it should just support Unicode objects. The buffer object makes no promise that the buffer actually is meaningful UCS-2/UCS-4, so u# shouldn't guess that it is. (FWIW, it currently truncates the buffer size to the next-smaller multiple of sizeof(Py_UNICODE), and silently so) I think that part should just go: u# should be restricted to unicode objects. - should Python guarantee that all characters in a Unicode object are between 0 and sys.maxunicode? Currently, it is possible to create Unicode strings with either negative or very large Py_UNICODE elements. - if the answer to the last question is no (i.e. if it is intentional that a unicode object can contain arbitrary Py_UNICODE values): should Python then guarantee that Py_UNICODE is an unsigned type? Regards, Martin From martin at v.loewis.de Thu Mar 30 19:14:10 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 30 Mar 2006 19:14:10 +0200 Subject: [Python-Dev] I'm not getting email from SF whenassignedabug/patch In-Reply-To: References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com><00ff01c6523f$7c583290$bf03030a@trilan><1143554576.2310.161.camel@geddy.wooz.org><17449.21764.475497.457592@montanaro.dyndns.org><44298954.4040705@v.loewis.de> <1143648409.10799.159.camel@resist.wooz.org> Message-ID: <442C11E2.6050106@v.loewis.de> Fredrik Lundh wrote: > so what's the advantage of a freely hosted Atlassian setup compared > to a freely hosted Trac setup ? > > I'm obviously missing something here. One thing that you are *obviously* missing (there might be more): Nobody has stepped forward and said "I make trac happen". Without somebody (specific) saying that, all technical arguments in favour of that software are futile. Regards, Martin From ianb at colorstudy.com Thu Mar 30 19:14:36 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 30 Mar 2006 11:14:36 -0600 Subject: [Python-Dev] Class decorators In-Reply-To: <200603290048.52669.fdrake@acm.org> References: <5.1.1.6.0.20060328220253.01fb6c88@mail.telecommunity.com> <5.1.1.6.0.20060328235303.01fbd3c8@mail.telecommunity.com> <200603290048.52669.fdrake@acm.org> Message-ID: <442C11FC.30809@colorstudy.com> Fred L. Drake, Jr. wrote: > > It's too bad this syntax is ambiguous: > > > > class Foo: > > """Docstring here, blah blah blah > > """ > > @implements(IFoo) > > > > As this achieves a desirable highlighting of the specialness, without > > forcing the decorator outside the class. Oh well. > > Agreed, but... guess we can't have everything. On the other hand, something > like: > > class Foo: > """Documentation is good.""" > > @class implements(IFoo) > > is not ambiguous. Hmm. It even says what it means. :-) This is quite reminiscent of Ruby to me, where: class Foo: implements(IFoo) basically means: class Foo: pass Foo.implements(IFoo) For a variety of reasons that doesn't work for Python, but what you propose accomplishes the same basic thing. I'm coming in a little late on all this, but I find moving the decorator inside the class statement to be a substantial improvement, even if it is also a trivial improvement ;) Anytime I've done thought experiments about using class decorators, the results is very hard to read. That classes are inherently declarative and open, while functions are imperative and closed, makes the constructs very different. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From martin at v.loewis.de Thu Mar 30 19:19:31 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 30 Mar 2006 19:19:31 +0200 Subject: [Python-Dev] Win64 AMD64 (aka x64) binaries available64 In-Reply-To: References: <43FA4A1B.3030209@v.loewis.de> Message-ID: <442C1323.9000109@v.loewis.de> Thomas Heller wrote: > Is this no longer available? Sorry, I just deleted that. I now replaced it with python-2.5.13231.amd64.msi > BTW: When I build Python for ReleaseAMD64 myself, the exe crashes at the first > Py_BuildValue call. That doesn't happen for me... can you find out why it crashes? (and what is the buildvalue call it crashes on)? Regards, Martin From theller at python.net Thu Mar 30 19:22:23 2006 From: theller at python.net (Thomas Heller) Date: Thu, 30 Mar 2006 19:22:23 +0200 Subject: [Python-Dev] alpha problems -- need input In-Reply-To: References: Message-ID: <442C13CF.8050608@python.net> Neal Norwitz wrote: > These issues are on HEAD. There might be some others I missed. > > With cc there are at least 2 issues: > * test_file causes interpreter exit due to sys.stdin.seek(-1) > * test_pty fails apparently due to whitespace differences > > http://www.python.org/dev/buildbot/all/alpha%20Tru64%205.1%20trunk/builds/18/step-test/0 > > Should we skip the test for an invalid seek on stdin on osf1? > I haven't investigated the test_pty failure further. > > With gcc, there are also several issues: > * test_float and test_struct fail due to NaN handling > * test_long fails > * test_ctypes > ... > > test_ctypes fails because _findLib() doesn't seem to work properly on > OSF1. Note: libc.so => /usr/shlib/libc.so > > ERROR: test_find (ctypes.test.test_loading.LoaderTest) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/net/ringneck/scratch1/nnorwitz/python/trunk/Lib/ctypes/test/test_loading.py", > line 40, in test_find > cdll.find(name) > File "/net/ringneck/scratch1/nnorwitz/python/trunk/Lib/ctypes/_loader.py", > line 205, in find > raise OSError("Library %r not found" % name) > OSError: Library 'c' not found The ctypes findLib stuff for 'os.name == "posix"' was written by Andreas Degert, he used fancy calls to gcc, /sbin/ldconfig, and objdump to find the shared lib that would be used for the linker's -l flag. Apparently this only works on *some* systems. On OSX, Bob Ippolito's macholib is used. If someone could implement this for other systems it would be great, otherwise it would probably be best to enable this test only on linux . Thomas From theller at python.net Thu Mar 30 19:35:41 2006 From: theller at python.net (Thomas Heller) Date: Thu, 30 Mar 2006 19:35:41 +0200 Subject: [Python-Dev] Win64 AMD64 (aka x64) binaries available64 In-Reply-To: <442C1323.9000109@v.loewis.de> References: <43FA4A1B.3030209@v.loewis.de> <442C1323.9000109@v.loewis.de> Message-ID: <442C16ED.8080502@python.net> Martin v. L?wis wrote: > Thomas Heller wrote: >> Is this no longer available? > > Sorry, I just deleted that. I now replaced it with > python-2.5.13231.amd64.msi Thanks, I'll try that. >> BTW: When I build Python for ReleaseAMD64 myself, the exe crashes at the first >> Py_BuildValue call. > > That doesn't happen for me... can you find out why it crashes? (and what > is the buildvalue call it crashes on)? In sys_getwindowsversion: return Py_BuildValue("HHHHs", ver.dwMajorVersion, ver.dwMinorVersion, ver.dwBuildNumber, ver.dwPlatformId, ver.szCSDVersion); The crash disappears if I change the first parameter in the Py_BuildValue call to "LLLLs". No idea why. With this change, I can start the exe without a crash, but sys.versioninfo starts with (IIRC) (2, 0, 5,...). Thomas From fredrik at pythonware.com Thu Mar 30 19:37:00 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 30 Mar 2006 19:37:00 +0200 Subject: [Python-Dev] I'm not getting email from SF when assigned a bug/patch References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com><00ff01c6523f$7c583290$bf03030a@trilan><1143554576.2310.161.camel@geddy.wooz.org><17449.21764.475497.457592@montanaro.dyndns.org><44298954.4040705@v.loewis.de> <1143648409.10799.159.camel@resist.wooz.org> <442C11E2.6050106@v.loewis.de> Message-ID: Martin v. Löwis wrote: > > I'm obviously missing something here. > > One thing that you are *obviously* missing (there might be more): > > Nobody has stepped forward and said "I make trac happen". Without > somebody (specific) saying that, all technical arguments in favour > of that software are futile. I'm quite sure I've already pointed to http://www.python-hosting.com/freetrac which could be used by the Python 3000 activity in order to evaluate Trac, and to http://www.python-hosting.com/shared_hosting plus a small fraction of the PSF budget for the Python code, if Trac is found to be good enough, and provided that the PH folks think that it's a good idea to host core development on their platform (I assume we have pretty good stats for how svn.python.org is being used). I can ask them for a test py3k account, if there's any interest. From g.brandl at gmx.net Thu Mar 30 19:40:54 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 30 Mar 2006 19:40:54 +0200 Subject: [Python-Dev] I'm not getting email from SFwhenassignedabug/patch In-Reply-To: References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> <1143554576.2310.161.camel@geddy.wooz.org> <17449.21764.475497.457592@montanaro.dyndns.org> <44298954.4040705@v.loewis.de> <1143648409.10799.159.camel@resist.wooz.org> <1143727693.2317.226.camel@geddy.wooz.org> Message-ID: Fredrik Lundh wrote: > Georg Brandl wrote: > >> What I answered to was: >> >> """ >> from what I can tell, no big contemporary Python project use Atlassian. >> they all use Trac. there are thousands of Python developers out there >> that are used to working with Trac. >> >> I'm obviously missing something here. >> """ >> >> I'm not saying it's out of the question for Python, I'm saying that it's >> out of the question for most open-source projects, which don't have the >> money or don't want to spend the money on a mere bug tracker, and that >> this may be the reason that Jira isn't used widely by Python programmers. > > Like most commercial tool providers that complete on a market full of > open source tools of various quality, Atlassian offers free licenses to > non-profits and established open source projects: > > http://www.atlassian.com/software/jira/pricing.jsp#nonprofit Okay, I wasn't aware of that. Georg From g.brandl at gmx.net Thu Mar 30 19:44:55 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 30 Mar 2006 19:44:55 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <442C0288.8080502@egenix.com> References: <06Mar29.141411pst."58633"@synergy1.parc.xerox.com> <200603302303.28587.anthony@interlink.com.au> <442BF378.5060708@egenix.com> <200603310244.33110.anthony@interlink.com.au> <442C0288.8080502@egenix.com> Message-ID: M.-A. Lemburg wrote: > Anthony Baxter wrote: >> On Friday 31 March 2006 02:04, M.-A. Lemburg wrote: >>>> Excellent point. Hm. Maybe we should just go with 'sqlite', >>>> instead. >>> Anything, but please no "db" or "database" top-level module or >>> package :-) >> >> >> How about "sql"? >> >> I can't think of a better name right now - can anyone else, or should >> it just go in the top level as 'sqlite'? > > I think sqlite is just fine. sqlite is problematic since the old PySQLite 1.0 (2.x wrapper) had this name. But I'm in favour of a flat name too -- let's use "sqlite3". Georg From martin at v.loewis.de Thu Mar 30 19:51:34 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 30 Mar 2006 19:51:34 +0200 Subject: [Python-Dev] Win64 AMD64 (aka x64) binaries available64 In-Reply-To: <442C16ED.8080502@python.net> References: <43FA4A1B.3030209@v.loewis.de> <442C1323.9000109@v.loewis.de> <442C16ED.8080502@python.net> Message-ID: <442C1AA6.9030408@v.loewis.de> Thomas Heller wrote: > In sys_getwindowsversion: > > return Py_BuildValue("HHHHs", > ver.dwMajorVersion, > ver.dwMinorVersion, > ver.dwBuildNumber, > ver.dwPlatformId, > ver.szCSDVersion); > > The crash disappears if I change the first parameter in the > Py_BuildValue call to "LLLLs". No idea why. > With this change, I can start the exe without a crash, but > sys.versioninfo starts with (IIRC) (2, 0, 5,...). Very strange. What is your compiler version (first line of cl /?)? 'H' isn't exactly right, since these are DWORDs, so they are unsigned longs, not unsigned ints (so 'k' should be right), however, the actually bug apparently is elsewhere - something like memory corruption. Can you find out what the value of szCSDVersion is (e.g. by putting a printf right before that)? Regards, Martin From martin at v.loewis.de Thu Mar 30 19:59:26 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 30 Mar 2006 19:59:26 +0200 Subject: [Python-Dev] I'm not getting email from SF when assigned a bug/patch In-Reply-To: References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com><00ff01c6523f$7c583290$bf03030a@trilan><1143554576.2310.161.camel@geddy.wooz.org><17449.21764.475497.457592@montanaro.dyndns.org><44298954.4040705@v.loewis.de> <1143648409.10799.159.camel@resist.wooz.org> <442C11E2.6050106@v.loewis.de> Message-ID: <442C1C7E.4050502@v.loewis.de> Fredrik Lundh wrote: > I can ask them for a test py3k account, if there's any interest. I'm personally not very much interested in a Py3k tracker; I don't see myself using it. So I'm not interested in a trac-based one, either. As for python-hosting.com: Somebody would *still* have to set this up, manage to import the data, manage accounts, manage to make it known etc. The actual hardware to run it on is the least of my concerns - xs4all provides that very nicely. I don't see anything that could be saved by moving things from xs4all to python-hosting. Regards, Martin From guido at python.org Thu Mar 30 20:01:30 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 30 Mar 2006 10:01:30 -0800 Subject: [Python-Dev] I'm not getting email from SF when assigned a bug/patch In-Reply-To: <442C1C7E.4050502@v.loewis.de> References: <44298954.4040705@v.loewis.de> <1143648409.10799.159.camel@resist.wooz.org> <442C11E2.6050106@v.loewis.de> <442C1C7E.4050502@v.loewis.de> Message-ID: On 3/30/06, "Martin v. L?wis" wrote: > Fredrik Lundh wrote: > > I can ask them for a test py3k account, if there's any interest. > > I'm personally not very much interested in a Py3k tracker; I don't > see myself using it. So I'm not interested in a trac-based one, > either. Me neither. It's too early. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From raymond.hettinger at verizon.net Thu Mar 30 20:09:40 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Thu, 30 Mar 2006 13:09:40 -0500 Subject: [Python-Dev] PySet API References: <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1><1143468446.10799.50.camel@resist.wooz.org><200603281507.18175.gmccaughan@synaptics-uk.com><001a01c65377$cd099a10$6601a8c0@RaymondLaptop1><1143678849.6694.52.camel@resist.wooz.org><004c01c653af$c8017a00$6601a8c0@RaymondLaptop1> <1143734049.7204.79.camel@resist.wooz.org> Message-ID: <008f01c65425$1d64db70$6601a8c0@RaymondLaptop1> >> The idea is not yet ready for prime-time. If I do it for one of the named >> operations, I will do it for all (to keep the interface uniform). > Which named operations are you thinking of? s.union(t), s.intersection(t), s.difference(t), s.symmetric_difference(t), s.update(t), s.intersection_update(t), s.difference_update(t), s.symmetric_difference_update(t) > if you can't make vararg PySet_Update() insanely fast, > does that mean you won't add a vararg version? Right. Please leave this one alone. I still need to work on this part of the API and do not currently have the spare clock cycles to do it right now. You don't HAVE to jam something in right away. Please let it continue to cook and not muck it up through over-enthusiasm. If I had time to explain/debate every darned aspect of what is under consideration, then it would have been done already. The fierce insistence for the patch is pre-mature and is grotesquely out-of-proportion to any potential benefit. Please do not jam this one down my throat -- the function is not necessary to have right away -- you're talking about nanoseconds of efficiency and a microscopically shorter call. Sorry, I need to stop wasting time on this thread. It has consumed far too much development time already. Please write a one-line macro for yourself and leave this alone so I can continue the development efforts at a thoughtful pace. Raymond From theller at python.net Thu Mar 30 20:27:45 2006 From: theller at python.net (Thomas Heller) Date: Thu, 30 Mar 2006 20:27:45 +0200 Subject: [Python-Dev] Win64 AMD64 (aka x64) binaries available64 In-Reply-To: <442C1AA6.9030408@v.loewis.de> References: <43FA4A1B.3030209@v.loewis.de> <442C1323.9000109@v.loewis.de> <442C16ED.8080502@python.net> <442C1AA6.9030408@v.loewis.de> Message-ID: <442C2321.9000105@python.net> Martin v. L?wis wrote: > Thomas Heller wrote: >> In sys_getwindowsversion: >> >> return Py_BuildValue("HHHHs", >> ver.dwMajorVersion, >> ver.dwMinorVersion, >> ver.dwBuildNumber, >> ver.dwPlatformId, >> ver.szCSDVersion); >> >> The crash disappears if I change the first parameter in the >> Py_BuildValue call to "LLLLs". No idea why. >> With this change, I can start the exe without a crash, but >> sys.versioninfo starts with (IIRC) (2, 0, 5,...). > > Very strange. What is your compiler version (first line of cl /?)? > > 'H' isn't exactly right, since these are DWORDs, so they > are unsigned longs, not unsigned ints (so 'k' should be right), > however, the actually bug apparently is elsewhere - something > like memory corruption. Can you find out what the value of > szCSDVersion is (e.g. by putting a printf right before that)? I would prefer to examine this further when I'm able to compile Python on the AMD64 machine. Currently I'm not, because the VSExtCompiler Plugin cannot determine the SDK path from the registry. This is vsextcomp 0.6. I have installed the 2003 Server SP1 SDK. I tried installing the Feb 2003 SDK (since that did work on a 32-bit machine), but this won't install on XP-64. Thomas From g.brandl at gmx.net Thu Mar 30 20:52:06 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 30 Mar 2006 20:52:06 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <200603310244.33110.anthony@interlink.com.au> References: <06Mar29.141411pst."58633"@synergy1.parc.xerox.com> <200603302303.28587.anthony@interlink.com.au> <442BF378.5060708@egenix.com> <200603310244.33110.anthony@interlink.com.au> Message-ID: <442C28D6.10701@gmx.net> Anthony Baxter wrote: > On Friday 31 March 2006 02:04, M.-A. Lemburg wrote: >> > Excellent point. Hm. Maybe we should just go with 'sqlite', >> > instead. >> >> Anything, but please no "db" or "database" top-level module or >> package :-) > > > How about "sql"? > > I can't think of a better name right now - can anyone else, or should > it just go in the top level as 'sqlite'? > > >> I take it that this is not going to go into 2.5a1 ?! > > Well, right now the major missing bits for landing it right now are > the windows build project and the documentation. I'm pretty > comfortable with landing it for a1. It has tests, I've knitted these > into the Python regression testing suite and they're all passing > fine. I've tested building on systems with a version of sqlite that > is acceptable, with no sqlite, and with an old version of sqlite, and > the build process handles it all correctly. Apart from the tests issue (see python-checkins), doesn't version 1.0 of PySQLite also use the name "_sqlite" for its C module? If so, ours should be renamed too (_sqlite3?) to avoid breaking old apps. Georg From chris at atlee.ca Thu Mar 30 21:12:51 2006 From: chris at atlee.ca (Chris AtLee) Date: Thu, 30 Mar 2006 14:12:51 -0500 Subject: [Python-Dev] xmlrpclib.binary missing? Message-ID: <7790b6530603301112v411e35f7rc4c2e2a453d59ee0@mail.gmail.com> The current 2.4 and 2.5 docs mention that the xmlrpclib has a function called binary which converts any python value to a Binary object. However, this function does not exist in either 2.4.3 or 2.5. The Binary constructor accepts a data parameter, so I would say just remove mention of the binary function from the docs and leave the implementation as-is. Cheers, Chris From janssen at parc.com Thu Mar 30 21:29:02 2006 From: janssen at parc.com (Bill Janssen) Date: Thu, 30 Mar 2006 11:29:02 PST Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: Your message of "Thu, 30 Mar 2006 08:08:40 PST." <442C0288.8080502@egenix.com> Message-ID: <06Mar30.112908pst."58633"@synergy1.parc.xerox.com> > I think sqlite is just fine. I do, too. Bill From aahz at pythoncraft.com Thu Mar 30 21:41:58 2006 From: aahz at pythoncraft.com (Aahz) Date: Thu, 30 Mar 2006 11:41:58 -0800 Subject: [Python-Dev] xmlrpclib.binary missing? In-Reply-To: <7790b6530603301112v411e35f7rc4c2e2a453d59ee0@mail.gmail.com> References: <7790b6530603301112v411e35f7rc4c2e2a453d59ee0@mail.gmail.com> Message-ID: <20060330194158.GB13161@panix.com> On Thu, Mar 30, 2006, Chris AtLee wrote: > > The current 2.4 and 2.5 docs mention that the xmlrpclib has a function > called binary which converts any python value to a Binary object. > > However, this function does not exist in either 2.4.3 or 2.5. > > The Binary constructor accepts a data parameter, so I would say just > remove mention of the binary function from the docs and leave the > implementation as-is. Please file a bug on SF and report it back here so that we don't lose track of this. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Look, it's your affair if you want to play with five people, but don't go calling it doubles." --John Cleese anticipates Usenet From chris at atlee.ca Thu Mar 30 21:55:27 2006 From: chris at atlee.ca (Chris AtLee) Date: Thu, 30 Mar 2006 14:55:27 -0500 Subject: [Python-Dev] xmlrpclib.binary missing? In-Reply-To: <20060330194158.GB13161@panix.com> References: <7790b6530603301112v411e35f7rc4c2e2a453d59ee0@mail.gmail.com> <20060330194158.GB13161@panix.com> Message-ID: <7790b6530603301155x27904349s5a323fe12ae2f760@mail.gmail.com> On 3/30/06, Aahz wrote: > On Thu, Mar 30, 2006, Chris AtLee wrote: > > > > The current 2.4 and 2.5 docs mention that the xmlrpclib has a function > > called binary which converts any python value to a Binary object. > > > > However, this function does not exist in either 2.4.3 or 2.5. > > > > The Binary constructor accepts a data parameter, so I would say just > > remove mention of the binary function from the docs and leave the > > implementation as-is. > > Please file a bug on SF and report it back here so that we don't lose > track of this. Sure, this is filed as bug #1461610. Cheers, Chris From blais at furius.ca Thu Mar 30 22:21:30 2006 From: blais at furius.ca (Martin Blais) Date: Thu, 30 Mar 2006 15:21:30 -0500 Subject: [Python-Dev] Fwd: Python 2.5 update In-Reply-To: <8393fff0603291932la505a7cx99241a4a55cb5cf3@mail.gmail.com> References: <8393fff0603291932la505a7cx99241a4a55cb5cf3@mail.gmail.com> Message-ID: <8393fff0603301221i556b35b0w7e0dd591432b0e7f@mail.gmail.com> ---------- Forwarded message ---------- From: Martin Blais Date: Mar 29, 2006 10:32 PM Subject: Python 2.5 update To: optik-users at lists.sourceforge.net Hi I was thinking of a new action "append_const" to optparse, so I googled it to check if anybody else had been thinking about the same idea, and *surprise* -- not really -- I discover that it's already been implemented and included in Optik 1.5. Now, I checked in the upcoming Python 2.5 source code -- the Subversion repository -- and it still is at 1.5a2, which does not contain this code. And the stdlib uses an alpha version? Hmmm. What is the process for upgrading the stdlib version of optparse? Are there any plans to upgrade the stdlib's 1.5a2 to the stable 1.5? It seems like it has been released for a while... If there are no plans to do that, what is the process exactly, and can I nudge it with a little bit of armwork somehow? cheers, From g.brandl at gmx.net Thu Mar 30 22:38:20 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 30 Mar 2006 22:38:20 +0200 Subject: [Python-Dev] 2.5 trunk built for Windows available? Message-ID: Hi, for the Bug Day, someone asked me if there is a prebuilt trunk for Windows available somewhere so that he could participate. Martin: I read you've built for a snapshot AMD64, is there one for x86 too? Georg From guido at python.org Thu Mar 30 22:41:45 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 30 Mar 2006 12:41:45 -0800 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <-4432802660741965007@unknownmsgid> References: <442C0288.8080502@egenix.com> <-4432802660741965007@unknownmsgid> Message-ID: So do I. On 3/30/06, Bill Janssen wrote: > > I think sqlite is just fine. > > I do, too. > > Bill -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.peters at gmail.com Thu Mar 30 22:59:06 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 30 Mar 2006 15:59:06 -0500 Subject: [Python-Dev] Nasty trunk test failures Message-ID: <1f7befae0603301259s1bb4c3e3lc7c616943dce5f99@mail.gmail.com> test_tokenize started failing on all the trunk buildbots immediately after this seemingly unrelated checkin: > Author: armin.rigo > Date: Thu Mar 30 16:04:02 2006 > New Revision: 43458 > > Modified: > python/trunk/Lib/test/test_index.py > python/trunk/Objects/abstract.c > python/trunk/Objects/typeobject.c > Log: > Minor bugs in the __index__ code (PEP 357), with tests. I haven't gotten anywhere with this. test_tokenize doesn't fail in isolation on my box, and at one point didn't even fail after using -f to reproduce all the steps up to and including test_tokenize from a failing -r run. I do have a long sequence of tests now ending with test_tokenize that fails on my box provided I use -uall (and I should note that test_index is not in this sequence). I'm using a debug build here, and don't know about a release build. Alas, it takes a long time to run, so progress this way is slow. Worse, since this is so flaky I doubt it's going to lead to something interesting anyway. More eyeballs? I didn't see anything obviously wrong in Armin's checkin. Maybe someone else would. Or maybe it's just managing to provoke a pre-existing C problem. ... Eww! test_tokenize _does_ fail in isolation here, but only if I use -uall(!): C:\Code\python\PCbuild>python_d ../Lib/test/regrtest.py test_tokenize test_tokenize 1 test OK. [25131 refs] C:\Code\python\PCbuild>python_d ../Lib/test/regrtest.py -uall test_tokenize test_tokenize test test_tokenize crashed -- : 1 test failed: test_tokenize [15147 refs] OK, test_tokenize does a hell of a lot more work when -uall is specified. After applying this patch, test_tokenize fails very quickly (although you still need -uall): """ Index: test_tokenize.py =================================================================== --- test_tokenize.py (revision 43462) +++ test_tokenize.py (working copy) @@ -34,6 +34,7 @@ testdir = os.path.dirname(f) or os.curdir testfiles = glob.glob(testdir + os.sep + 'test*.py') +testfiles = [testdir + os.sep + 'test_index.py'] if not is_resource_enabled('compiler'): testfiles = random.sample(testfiles, 10) """ IOW, it's specifically the new test_index.py that test_tokenize fails on. I'm not sure what test_tokenize is doing, but if someone else is, that seems like a good hint ;-) From skip at pobox.com Thu Mar 30 23:01:45 2006 From: skip at pobox.com (skip at pobox.com) Date: Thu, 30 Mar 2006 15:01:45 -0600 Subject: [Python-Dev] I'm not getting email from SF whenassignedabug/patch In-Reply-To: <442C11E2.6050106@v.loewis.de> References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> <1143554576.2310.161.camel@geddy.wooz.org> <17449.21764.475497.457592@montanaro.dyndns.org> <44298954.4040705@v.loewis.de> <1143648409.10799.159.camel@resist.wooz.org> <442C11E2.6050106@v.loewis.de> Message-ID: <17452.18233.415289.251479@montanaro.dyndns.org> Martin> Nobody has stepped forward and said "I make trac happen". Hasn't trac already happened in the sense that it's installed (by Tim Parkin on the Pollenation website) and in use by the website maintainers? Seems the only major hurdle is the extraction of history from SF. Skip From guido at python.org Thu Mar 30 23:43:33 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 30 Mar 2006 13:43:33 -0800 Subject: [Python-Dev] Nasty trunk test failures In-Reply-To: <1f7befae0603301259s1bb4c3e3lc7c616943dce5f99@mail.gmail.com> References: <1f7befae0603301259s1bb4c3e3lc7c616943dce5f99@mail.gmail.com> Message-ID: On 3/30/06, Tim Peters wrote: > test_tokenize started failing on all the trunk buildbots immediately > after this seemingly unrelated checkin: tokenize seems to be mishandling this line: assert 6 .__index__() == 6 Note the space between '6' and '.'. I'm guessing that the untokenization of this somehow drops the space; this seems to be a bug in untokenize() which probably should add a safety space after names as well as numbers. Yes, this fixes the problem: Index: tokenize.py =================================================================== --- tokenize.py (revision 43463) +++ tokenize.py (working copy) @@ -182,7 +182,7 @@ for tok in iterable: toknum, tokval = tok[:2] - if toknum == NAME: + if toknum in (NAME, NUMBER): tokval += ' ' if toknum == INDENT: I'll check this in. Then there's another (shallow) problem that only occurs when I run test_tokenize.py directly -- the doctest for decistmt() has-3.21716034272e-007 but (on my box) this outputs -3.21716034272e-07. That doesn't seem to bother it when run via regrtest.py. I'm not sure what's at fault here. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Thu Mar 30 23:49:12 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 30 Mar 2006 23:49:12 +0200 Subject: [Python-Dev] I'm not getting email from SF whenassignedabug/patch In-Reply-To: <17452.18233.415289.251479@montanaro.dyndns.org> References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> <1143554576.2310.161.camel@geddy.wooz.org> <17449.21764.475497.457592@montanaro.dyndns.org> <44298954.4040705@v.loewis.de> <1143648409.10799.159.camel@resist.wooz.org> <442C11E2.6050106@v.loewis.de> <17452.18233.415289.251479@montanaro.dyndns.org> Message-ID: <442C5258.7040009@v.loewis.de> skip at pobox.com wrote: > Hasn't trac already happened in the sense that it's installed (by Tim Parkin > on the Pollenation website) and in use by the website maintainers? Seems > the only major hurdle is the extraction of history from SF. That isn't actually the case. Test data would be available; somebody would need to import them. Regards, Martin From martin at v.loewis.de Fri Mar 31 00:15:42 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 31 Mar 2006 00:15:42 +0200 Subject: [Python-Dev] Fwd: Python 2.5 update In-Reply-To: <8393fff0603301221i556b35b0w7e0dd591432b0e7f@mail.gmail.com> References: <8393fff0603291932la505a7cx99241a4a55cb5cf3@mail.gmail.com> <8393fff0603301221i556b35b0w7e0dd591432b0e7f@mail.gmail.com> Message-ID: <442C588E.5040807@v.loewis.de> Martin Blais wrote: > What is the process for upgrading the stdlib version of optparse? Very simple: the author of the library would have to contribute it. Actually, if the changes where contributed by Optik contributors, they would have to contribute them to Python. > Are there any plans to upgrade the stdlib's 1.5a2 to the stable 1.5? I don't have any such plans: I cannot just incorporate somebody else's library into Python without his permission. > If there are no plans to do that, what is the process exactly, and can > I nudge it with a little bit of armwork somehow? Ask the authors to act. Regards, Martin From martin at v.loewis.de Fri Mar 31 00:16:21 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 31 Mar 2006 00:16:21 +0200 Subject: [Python-Dev] 2.5 trunk built for Windows available? In-Reply-To: References: Message-ID: <442C58B5.70704@v.loewis.de> Georg Brandl wrote: > for the Bug Day, someone asked me if there is a prebuilt trunk > for Windows available somewhere so that he could participate. > > Martin: I read you've built for a snapshot AMD64, is there one for x86 too? Not at the moment, and I might not have time tomorrow to produce one. Regards, Martin From barry at python.org Fri Mar 31 00:19:00 2006 From: barry at python.org (Barry Warsaw) Date: Thu, 30 Mar 2006 17:19:00 -0500 Subject: [Python-Dev] PySet API In-Reply-To: <008f01c65425$1d64db70$6601a8c0@RaymondLaptop1> References: <004301c650f4$6d2966f0$b83efea9@RaymondLaptop1> <1143468446.10799.50.camel@resist.wooz.org> <200603281507.18175.gmccaughan@synaptics-uk.com> <001a01c65377$cd099a10$6601a8c0@RaymondLaptop1> <1143678849.6694.52.camel@resist.wooz.org> <004c01c653af$c8017a00$6601a8c0@RaymondLaptop1> <1143734049.7204.79.camel@resist.wooz.org> <008f01c65425$1d64db70$6601a8c0@RaymondLaptop1> Message-ID: <1143757140.7204.91.camel@resist.wooz.org> On Thu, 2006-03-30 at 13:09 -0500, Raymond Hettinger wrote: > Please leave this one alone. I still need to work on this part of the API and > do not currently have the spare clock cycles to do it right now. You don't HAVE > to jam something in right away. Please let it continue to cook and not muck it > up through over-enthusiasm. If I had time to explain/debate every darned aspect > of what is under consideration, then it would have been done already. The fierce > insistence for the patch is pre-mature and is grotesquely out-of-proportion to > any potential benefit. Please do not jam this one down my throat -- the > function is not necessary to have right away -- you're talking about nanoseconds > of efficiency and a microscopically shorter call. Sorry, I need to stop wasting > time on this thread. It has consumed far too much development time already. > Please write a one-line macro for yourself and leave this alone so I can > continue the development efforts at a thoughtful pace. As per your comment in patch 1458476, I will add _PySet_Update() and consider this thread closed. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060330/692097e3/attachment.pgp From tjreedy at udel.edu Fri Mar 31 00:53:42 2006 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 30 Mar 2006 17:53:42 -0500 Subject: [Python-Dev] pysqlite for 2.5? References: <06Mar29.141411pst."58633"@synergy1.parc.xerox.com><200603302303.28587.anthony@interlink.com.au><442BF378.5060708@egenix.com> <200603310244.33110.anthony@interlink.com.au> Message-ID: "Anthony Baxter" wrote in message news:200603310244.33110.anthony at interlink.com.au... >> Anything, but please no "db" or "database" top-level module or >> package :-) > > How about "sql"? Whereas I am quite happy with a 'db' package, and would like to see other db stuff put under it. tjr From greg.ewing at canterbury.ac.nz Fri Mar 31 02:11:22 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 31 Mar 2006 12:11:22 +1200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: References: <200603290152.00598.anthony@interlink.com.au> <200603281759.18756.anthony@interlink.com.au> <1143554351.2309.158.camel@geddy.wooz.org> <5.1.1.6.0.20060329144142.0390e5d8@mail.telecommunity.com> <17451.882.842146.979286@montanaro.dyndns.org> <17451.12090.257751.636336@montanaro.dyndns.org> <442B46D6.70703@canterbury.ac.nz> <442BC13E.5000503@canterbury.ac.nz> Message-ID: <442C73AA.5080602@canterbury.ac.nz> Fredrik Lundh wrote: > not according to the documentation (which says that the embedded library locks > the database file, to prevent other independent processes from accessing the data). I think that means other *non-Firebird* processes. Firebird itself uses a system of file locks and transactions to manage concurrent access by different Firebird applications. Just to make sure I wasn't imagining things, I ran two instances of a program of mine that uses Firebird, without server, accessing the same database, and both worked fine. They can independently access different parts of the database, and if they try to update the same part at the same time, deadlocks are reported as appropriate. On the whole I'm very impressed with Firebird. I've found it to be nothing less than sleek, efficient, powerful and reliable. I'd encourage anyone looking for a database, embedded or otherwise, to give it a go. -- Greg From greg.ewing at canterbury.ac.nz Fri Mar 31 02:16:51 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 31 Mar 2006 12:16:51 +1200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <442BC178.3000704@egenix.com> References: <06Mar29.141411pst.58633@synergy1.parc.xerox.com> <442BC178.3000704@egenix.com> Message-ID: <442C74F3.1010100@canterbury.ac.nz> M.-A. Lemburg wrote: > I don't really care about the name, but please be aware that > you are talking about adding a *very* popular module name to > the top-level Python namespace if you go for "db" or "database". This would only be an issue for an application that had a private module calle db, since nobody will be trying to publish a top-level module for general use with such a generic name. In that case the application's module would just shadow the db package and the app would continue to work. If the app's author at some point wanted to start using stuff from the new db package, he would just have to rename his module. -- Greg From brett at python.org Fri Mar 31 02:40:43 2006 From: brett at python.org (Brett Cannon) Date: Thu, 30 Mar 2006 16:40:43 -0800 Subject: [Python-Dev] I'm not getting email from SF when assigned a bug/patch In-Reply-To: References: <1143648409.10799.159.camel@resist.wooz.org> <442C11E2.6050106@v.loewis.de> <442C1C7E.4050502@v.loewis.de> Message-ID: On 3/30/06, Guido van Rossum wrote: > On 3/30/06, "Martin v. L?wis" wrote: > > Fredrik Lundh wrote: > > > I can ask them for a test py3k account, if there's any interest. > > > > I'm personally not very much interested in a Py3k tracker; I don't > > see myself using it. So I'm not interested in a trac-based one, > > either. > > Me neither. It's too early. Same here. Please move any more comments about infrastructure to the infrastructure list (http://mail.python.org/mailman/listinfo/infrastructure/). But do realize the committee is not discussing trackers yet. We are still trying to get our SF data out so that can be imported into a tracker to test its use (we need large scale bugs in it to stress it and get a feel for its search abilities, etc.). I am going to be picky about keeping the intfrastructure email traffic focued on the task at hand. Tracker discussions will be mostly ignored by me until I send out the official call for tracker suggestions with volunteer maintainers. At that point we can figure out what trackers to consider and who will be in charge of getting the test setup going. -Brett From jimjjewett at gmail.com Fri Mar 31 03:07:46 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu, 30 Mar 2006 20:07:46 -0500 Subject: [Python-Dev] reference leaks, __del__, and annotations Message-ID: The checkins list has been struggling with generator reference leaks; the latest conclusion was that some are unavoidable because of __del__ cycles. That sort of defeats the purpose of resource managers. (Yes, it can be worked around on a case-by-case basis.) As I see it, part of the problem is that (1) When there is a cycle, python refuses to guess. (2) There is no way for a __del__ method to hint at ordering constraints. (3) There is no lightweight version of __del__ to say "I don't care about ordering constraints." How about using an (optional) annotation on __del__ methods, to indicate how cycles should be broken? As a strawman proposal: deletes = [obj for obj in cycle if hasattr(obj, "cycle")] deletes.sort() for obj in deletes: obj.__del__() Lightweight __del__ methods (such as most resource managers) could set the cycle attribute to True, and thereby ensure that they don't cause unbreakable cycles. Fancier object frameworks could use different values for the cycle attribute. Any object whose __del__ is not annotated will still be at least as likely to get finalized as it is today. -jJ From jimjjewett at gmail.com Fri Mar 31 03:16:35 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu, 30 Mar 2006 20:16:35 -0500 Subject: [Python-Dev] reference leaks, __del__, and annotations In-Reply-To: References: Message-ID: (Apologies for the thinko -- corrected because it was in the example code.) The checkins list has been struggling with generator reference leaks; the latest conclusion was that some are unavoidable because of __del__ cycles. That sort of defeats the purpose of resource managers. (Yes, it can be worked around on a case-by-case basis.) As I see it, part of the problem is that (1) When there is a cycle, python refuses to guess. (2) There is no way for a __del__ method to hint at ordering constraints. (3) There is no lightweight version of __del__ to say "I don't care about ordering constraints." How about using an (optional) annotation on __del__ methods, to indicate how cycles should be broken? As a strawman proposal: deletes = [(obj.__del__.cycle, obj) for obj in cycle if hasattr(obj, "__del__") and hasattr(obj.__del__, "cycle")] deletes.sort() for (cycle, obj) in deletes: obj.__del__() Lightweight __del__ methods (such as most resource managers) could set the cycle attribute to True, and thereby ensure that they won't cause unbreakable cycles. Fancier object frameworks could use different values for the cycle attribute. Any object whose __del__ is not annotated will still be at least as likely to get finalized as it is today. -jJ From jimjjewett at gmail.com Fri Mar 31 03:21:13 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu, 30 Mar 2006 20:21:13 -0500 Subject: [Python-Dev] Class decorators Message-ID: Phillip J. Eby wrote: > I don't even recall seeing any examples of class decorators > being used without arguments! I have often started with a function, and ended up replacing it with a callable object so that I could save state without resorting to "defalt args" or worse. I would prefer to decorate these exactly like the functions they replace. -jJ From greg.ewing at canterbury.ac.nz Fri Mar 31 04:21:13 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 31 Mar 2006 14:21:13 +1200 Subject: [Python-Dev] reference leaks, __del__, and annotations In-Reply-To: References: Message-ID: <442C9219.4060702@canterbury.ac.nz> Jim Jewett wrote: > The checkins list has been struggling with generator reference leaks; > the latest conclusion was that some are unavoidable because of __del__ > cycles. That sort of defeats the purpose of resource managers. Seems to me we need a whole new approach to finalization that's friendly to cyclic gc, such as a way of registering a finalizer that doesn't depend on the original object. If such a mechanism were available, could it be used instead of a __del__ method to clean up after a generator? (I'm asking because I'm not sure exactly what a generator __del__ needs to do.) > As a strawman proposal: > > deletes = [(obj.__del__.cycle, obj) for obj in cycle > if hasattr(obj, "__del__") and > hasattr(obj.__del__, "cycle")] > deletes.sort() > for (cycle, obj) in deletes: > obj.__del__() I think we need to be very careful about doing anything like this. From what Tim said recently, the consequences of an object getting its __del__ annotation wrong could be as bad as crashing the interpreter. -- Greg From tim.peters at gmail.com Fri Mar 31 04:34:04 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 30 Mar 2006 21:34:04 -0500 Subject: [Python-Dev] Nasty trunk test failures In-Reply-To: References: <1f7befae0603301259s1bb4c3e3lc7c616943dce5f99@mail.gmail.com> Message-ID: <1f7befae0603301834j510df900x571d3a20c6467eea@mail.gmail.com> [Guido] > tokenize seems to be mishandling this line: > > assert 6 .__index__() == 6 > > Note the space between '6' and '.'. > > I'm guessing that the untokenization of this somehow drops the space; > this seems to be a bug in untokenize() which probably should add a > safety space after names as well as numbers. Yes, this fixes the > problem: Thanks! It's nice to get a break from the "you found it, you fix it" rule ;-) > Then there's another (shallow) problem that only occurs when I run > test_tokenize.py directly -- the doctest for decistmt() > has-3.21716034272e-007 but (on my box) this outputs > -3.21716034272e-07. That doesn't seem to bother it when run via > regrtest.py. I'm not sure what's at fault here. I'll return the favor, then: decistmt's doctests don't run at all when test_tokenize is run indirectly via regrtest.py. I'll fix that (and worm around the Windows-specific expected output). From tim.peters at gmail.com Fri Mar 31 04:38:11 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 30 Mar 2006 21:38:11 -0500 Subject: [Python-Dev] 2.5 trunk built for Windows available? In-Reply-To: References: Message-ID: <1f7befae0603301838p29955094y3819d07dbcf1ee92@mail.gmail.com> [Georg Brandl] > for the Bug Day, someone asked me if there is a prebuilt trunk > for Windows available somewhere so that he could participate. > > Martin: I read you've built for a snapshot AMD64, is there one for x86 too? If someone can explain what "prebuilt trunk" means, exactly, maybe I can do it. From tim.peters at gmail.com Fri Mar 31 07:12:52 2006 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 31 Mar 2006 00:12:52 -0500 Subject: [Python-Dev] alpha problems -- need input In-Reply-To: References: Message-ID: <1f7befae0603302112g1a391bc6p970d11c5bb7b69d4@mail.gmail.com> Disabling a test on a platform is usually a bad thing, overall. The purpose of the test suite isn't to get a lot of green buildbot boxes <0.5 wink>, it's to determine whether Python works as expected. If a platform bug causes some test to fail, then that test _should_ fail on that platform -- it's not a Python bug, after all, and users on that platform aren't served by hiding that platform bugs cause Python tests to fail. If they try the same things in their programs they'll fail there too, and the test suite is supposed to warn them about that. If the primary goal here is really to "make the alpha Tru64 5.1 buildbot columns green", then maybe the Alpha needs a different test runner, to exclude the tests that are doomed to fail due to Alpha bugs. > ... > With gcc, there are also several issues: > * test_float and test_struct fail due to NaN handling > * test_long fails Which version of gcc is in use? Alpha hardware has incomplete support for IEEE endcase semantics, and "it usually" requires a special compiler option to generate code that hides the HW limitations. That you didn't list the above as failures using the native cc strongly suggests that we're missing a necessary gcc Alpha trick. You can go to: http://gcc.gnu.org/onlinedocs/ pick the version of gcc, and drill down to the "DEC Alpha Options" chapter for that version to see the Alpha tricks available. Looks like compiling with -mieee would be an excellent idea for Python, and looks like we're not using that now. I've never used an Alpha, but I recall that this suggestion fixed other peoples' problems :-) From martin at v.loewis.de Fri Mar 31 07:47:10 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 31 Mar 2006 07:47:10 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: References: <06Mar29.141411pst."58633"@synergy1.parc.xerox.com> <200603302303.28587.anthony@interlink.com.au> <442BF378.5060708@egenix.com> <200603310244.33110.anthony@interlink.com.au><442C0288.8080502@egenix.com> <442C03D4.1090000@ghaering.de> Message-ID: <442CC25E.1060605@v.loewis.de> Terry Reedy wrote: > "Gerhard H?ring" wrote in message > news:442C03D4.1090000 at ghaering.de... >> I proposed to link dynamically on Windows, and ship the Windows >> SQLite3.DLL. This has two advantages: >> >> - Python users can upgrade the SQLite3.DLL by a simple download from in >> case of emergency > > +1 and thanks from a Windows user This binary depends on msvcrt.dll. Does anybody know whether this could cause a problem? IOW, is any of the forbidden API used across the DLL boundary (in particular: memory management, stdio, locales)? Regards, Martin From martin at v.loewis.de Fri Mar 31 07:51:03 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 31 Mar 2006 07:51:03 +0200 Subject: [Python-Dev] 2.5 trunk built for Windows available? In-Reply-To: <1f7befae0603301838p29955094y3819d07dbcf1ee92@mail.gmail.com> References: <1f7befae0603301838p29955094y3819d07dbcf1ee92@mail.gmail.com> Message-ID: <442CC347.1090002@v.loewis.de> Tim Peters wrote: >> for the Bug Day, someone asked me if there is a prebuilt trunk >> for Windows available somewhere so that he could participate. >> >> Martin: I read you've built for a snapshot AMD64, is there one for x86 too? > > If someone can explain what "prebuilt trunk" means, exactly, maybe I can do it. If you have a release build of a Python trunk working copy, cd Tools\msi, run msi.py (with a Python version that has win32com), and make the resulting msi file available somewhere. Regards, Martin From nnorwitz at gmail.com Fri Mar 31 07:56:39 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 30 Mar 2006 21:56:39 -0800 Subject: [Python-Dev] _xmlplus fixup for 2.5 In-Reply-To: <442C0E73.7020301@v.loewis.de> References: <200603281759.18756.anthony@interlink.com.au> <200603302305.25863.anthony@interlink.com.au> <200603302348.41991.anthony@interlink.com.au> <442C0E73.7020301@v.loewis.de> Message-ID: On 3/30/06, "Martin v. L?wis" wrote: > > The problem is that now PyXML is no longer maintainable (not that it > has been maintained very well, though): The files that used to be > identical in PyXML and Python no longer are identical, so keeping > them synchronized adds unreasonable maintenance costs (IMO). I don't know what the differences are. Are they large? Can we copy the changes from Python back into PyXML? Or modify both PyXML and Python so they are the same? Could we create a patch that would be applied on importing PyXML to make things easier? n From fredrik at pythonware.com Fri Mar 31 08:03:45 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Fri, 31 Mar 2006 08:03:45 +0200 Subject: [Python-Dev] I'm not getting email from SF when assigned abug/patch References: <1143648409.10799.159.camel@resist.wooz.org> <442C11E2.6050106@v.loewis.de> <442C1C7E.4050502@v.loewis.de> Message-ID: Brett Cannon wrote: > Same here. Please move any more comments about infrastructure to the > infrastructure list (http://mail.python.org/mailman/listinfo/infrastructure/). But > do realize the committee is not discussing trackers yet. We are still > trying to get our SF data out so that can be imported into a tracker > to test its use oh, I forgot that the Procrastination & Stop energy Foundation was involved in this. see you all in 2015. From martin at v.loewis.de Fri Mar 31 08:23:36 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 31 Mar 2006 08:23:36 +0200 Subject: [Python-Dev] _xmlplus fixup for 2.5 In-Reply-To: References: <200603281759.18756.anthony@interlink.com.au> <200603302305.25863.anthony@interlink.com.au> <200603302348.41991.anthony@interlink.com.au> <442C0E73.7020301@v.loewis.de> Message-ID: <442CCAE8.1010608@v.loewis.de> Neal Norwitz wrote: > I don't know what the differences are. Are they large? Can we copy > the changes from Python back into PyXML? Or modify both PyXML and > Python so they are the same? Could we create a patch that would be > applied on importing PyXML to make things easier? Primarily, things got renamed from "xml" to "xmlcore" in different places. Porting them to PyXML is besides the point of the renaming, because PyXML then would have to provide an xmlcore package, too. As for actual solution strategies: I haven't yet researched any. Fred indicated he has something in mind, but hasn't been forthcoming over the past few months. Regards, Martin From nnorwitz at gmail.com Fri Mar 31 08:51:21 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 30 Mar 2006 22:51:21 -0800 Subject: [Python-Dev] alpha problems -- need input In-Reply-To: <1f7befae0603302112g1a391bc6p970d11c5bb7b69d4@mail.gmail.com> References: <1f7befae0603302112g1a391bc6p970d11c5bb7b69d4@mail.gmail.com> Message-ID: On 3/30/06, Tim Peters wrote: > Disabling a test on a platform is usually a bad thing, overall. The Agreed. > purpose of the test suite isn't to get a lot of green buildbot boxes > <0.5 wink>, it's to determine whether Python works as expected. If a > platform bug causes some test to fail, then that test _should_ fail on > that platform -- it's not a Python bug, after all, and users on that > platform aren't served by hiding that platform bugs cause Python tests > to fail. If they try the same things in their programs they'll fail > there too, and the test suite is supposed to warn them about that. Fair enough. However, it's not just that the test fails. It's that Python exits. Hell, if I'm in bash, bash dies too! If I'm running python in gdb, gdb dies. I tried building a trace (supposedly like truss), but that crashes while python is starting up. So we can never get results on which tests failed, since we can never finish reasonably. When I single stepped in gdb, the code died when executing the equivalent of fseek(stdin, -1, 0); I tried to seek(-1) on a different file and that was ok. The test is meant to stress a failure condition, but fseek is expected to return. Upgrading the machine isn't an option (it's not mine), and I don't know that upgrading would fix the problem. > If the primary goal here is really to "make the alpha Tru64 5.1 > buildbot columns green", then maybe the Alpha needs a different test > runner, to exclude the tests that are doomed to fail due to Alpha > bugs. That's not my goal. I just want to get as much information as possible. I also don't want to skip all of test_file, only one test case. > > ... > > With gcc, there are also several issues: > > * test_float and test_struct fail due to NaN handling > > * test_long fails > > Which version of gcc is in use? Alpha hardware has incomplete support 4.0.1. > Looks like compiling with > > -mieee > > would be an excellent idea for Python, and looks like we're not using > that now. I've never used an Alpha, but I recall that this suggestion > fixed other peoples' problems :-) Since we are already using -ieee for the platform cc, I thought this was an excellent idea as well. I almost put in my message that I tried it along with a couple other options. But I guess I was on crack or something, cause I tried it again, being very careful (ie, not trying to do 4 things at once) and voila! it worked. Thanks Tim! It fixed the problem with test_float, test_long, and test_struct. The thing that seems really odd is that I was sure test_pty passed before with gcc, now it's failing. Given the problem with -mieee, I was probably just very confused when I did everything the first time. So now the failures with gcc are: test_file, test_pty, test_ctypes. I believe this is the same as with the platform cc. Additional ctypes failure info for Thomas: AssertionError: ('dlopen: Cannot map library libc.so.6', 'libc.so.6', ('posix', 'osf1V5')) Note: there is no version info on osf1, ie no 6. ldd python shows: libc.so => /usr/shlib/libc.so n From fredrik at pythonware.com Fri Mar 31 08:57:56 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Fri, 31 Mar 2006 08:57:56 +0200 Subject: [Python-Dev] 2.5 trunk built for Windows available? References: <1f7befae0603301838p29955094y3819d07dbcf1ee92@mail.gmail.com> Message-ID: Tim Peters wrote: > [Georg Brandl] > > for the Bug Day, someone asked me if there is a prebuilt trunk > > for Windows available somewhere so that he could participate. > > > > Martin: I read you've built for a snapshot AMD64, is there one for x86 too? > > If someone can explain what "prebuilt trunk" means, exactly, maybe I can do it. a compiled version of the current development trunk ? I guess an installer would be nice, but a ZIP archive is probably good enough. From robert.kern at gmail.com Fri Mar 31 09:05:26 2006 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 31 Mar 2006 01:05:26 -0600 Subject: [Python-Dev] I'm not getting email from SF when assigned abug/patch In-Reply-To: References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> <1143554576.2310.161.camel@geddy.wooz.org> <17449.21764.475497.457592@montanaro.dyndns.org> <44298954.4040705@v.loewis.de> Message-ID: Brett Cannon wrote: > On 3/28/06, "Martin v. L?wis" wrote: > >>skip at pobox.com wrote: >> >>>Roundup is there now, right (sans SF export)? >> >>Richard Jones has an SF importer for one of the two XML-like formats, >>the one that is correct XML but with incomplete data. The other format, >>which has complete data but is ill-formed XML, has no importer into >>roundup at the moment. >> >> >>>Trac is being used by the >>>folks doing the new website. What other candidates are being considered? >> >>My view is that nothing should be "considered" unless there is >>a) a volunteer to operate the tracker (or, failing that, somebody who >> does it for money), and >>b) an importer of whatever data SF can provide. >> > > > And as the chair of the infrastructure committee, those are the base > requirements for a tracker to be considered. > > Once we have the complete SF data I will put out a call for proposals > for trackers along with who has volunteered to manage them so that > people can band together to help push their favorite tracker. > > But as of right now the committe just wants to get a clean dump of our > SF data to work off of (yes, we could start a tracker from scratch but > we want the SF data anyway and so we are going through the hassle of > getting it off now and as a test data set for the various trackers). > > -Brett > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org -- Robert Kern robert.kern at gmail.com "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From fredrik at pythonware.com Fri Mar 31 09:03:00 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Fri, 31 Mar 2006 09:03:00 +0200 Subject: [Python-Dev] I'm not getting email from SF when assigned abug/patch References: <44298954.4040705@v.loewis.de> <1143648409.10799.159.camel@resist.wooz.org> <442C11E2.6050106@v.loewis.de> <442C1C7E.4050502@v.loewis.de> Message-ID: Guido van Rossum wrote: > > > I can ask them for a test py3k account, if there's any interest. > > > > I'm personally not very much interested in a Py3k tracker; I don't > > see myself using it. So I'm not interested in a trac-based one, > > either. > > Me neither. It's too early. I wasn't really expecting you to start collecting *bug reports* for Py3k at this time; you can use Trac for a lot more than that, if you want (including todo/totry/tothinkabout lists, milestones, pep editing sandboxes, etc). From robert.kern at gmail.com Fri Mar 31 09:09:56 2006 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 31 Mar 2006 01:09:56 -0600 Subject: [Python-Dev] I'm not getting email from SF when assigned abug/patch In-Reply-To: References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> <1143554576.2310.161.camel@geddy.wooz.org> <17449.21764.475497.457592@montanaro.dyndns.org> <44298954.4040705@v.loewis.de> Message-ID: Brett Cannon wrote: > On 3/28/06, "Martin v. L?wis" wrote: >>My view is that nothing should be "considered" unless there is >>a) a volunteer to operate the tracker (or, failing that, somebody who >> does it for money), and >>b) an importer of whatever data SF can provide. > > And as the chair of the infrastructure committee, those are the base > requirements for a tracker to be considered. > > Once we have the complete SF data I will put out a call for proposals > for trackers along with who has volunteered to manage them so that > people can band together to help push their favorite tracker. FWIW: Trac has a Sourceforge bug tracker import script: http://projects.edgewall.com/trac/browser/trunk/contrib/sourceforge2trac.py Apologies: for the other blank reply. -- Robert Kern robert.kern at gmail.com "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From taroso at gmail.com Fri Mar 31 06:23:43 2006 From: taroso at gmail.com (Taro Ogawa) Date: Fri, 31 Mar 2006 04:23:43 +0000 (UTC) Subject: [Python-Dev] [Python-3000] Iterators for dict keys, values, and items == annoying :) References: <4422FC96.2020409@zope.com> <1143306134.3186.1.camel@localhost.localdomain> <44258376.1080709@gmx.net> <1143432484.14391.67.camel@localhost.localdomain> <79990c6b0603271117l4c69372h11362dd5d2d0ca32@mail.gmail.com> <1143565628.3305.82.camel@localhost.localdomain> <4429F6BD.60704@canterbury.ac.nz> <79990c6b0603290129m7bed22cft3e242cd009efe9e0@mail.gmail.com> <442A6C49.5020106@gmail.com> Message-ID: Nick Coghlan gmail.com> writes: > There are three big use cases: > dict.keys > dict.values > dict.items > Currently these all return lists, which may be expensive in terms of copying. > They all have iter* variants which while memory efficient, are far less > convenient to work with. Is there any reason why they can't be view objects - a dictionary has keys, has values, has items - rather than methods returning view objects: for k in mydict.keys: ... for v in mydict.values: ... for k, v in mydict.items: ... For backward compatibility with Py2.x, calling them would raise a DeprecationWarning and return a list. This could even be introduced in 2.x (with a PendingDeprecationWarning instead?). Cheers, -T. From fredrik at pythonware.com Fri Mar 31 09:37:14 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Fri, 31 Mar 2006 09:37:14 +0200 Subject: [Python-Dev] I'm not getting email from SF when assignedabug/patch References: <9e804ac0603271427p332b5a9cid84380b44aca1a33@mail.gmail.com> <00ff01c6523f$7c583290$bf03030a@trilan> <1143554576.2310.161.camel@geddy.wooz.org> <17449.21764.475497.457592@montanaro.dyndns.org> <44298954.4040705@v.loewis.de> Message-ID: Robert Kern wrote: > Apologies: for the other blank reply. oh, I don't know about that -- the eco quote made perfect sense ;-) From duncan.booth at suttoncourtenay.org.uk Fri Mar 31 10:12:56 2006 From: duncan.booth at suttoncourtenay.org.uk (Duncan Booth) Date: Fri, 31 Mar 2006 09:12:56 +0100 Subject: [Python-Dev] reference leaks, __del__, and annotations References: Message-ID: "Jim Jewett" wrote in news:fb6fbf560603301716x13c4cda7x7fd5e462850b5a03 at mail.gmail.com: > As a strawman proposal: > > deletes = [(obj.__del__.cycle, obj) for obj in cycle > if hasattr(obj, "__del__") and > hasattr(obj.__del__, "cycle")] > deletes.sort() > for (cycle, obj) in deletes: > obj.__del__() > > Lightweight __del__ methods (such as most resource managers) could set > the cycle attribute to True, and thereby ensure that they won't cause > unbreakable cycles. Fancier object frameworks could use different > values for the cycle attribute. Any object whose __del__ is not > annotated will still be at least as likely to get finalized as it is That doesn't look right to me. Surely if you have a cycle what you want to do is to pick just *one* of the objects in the cycle and break the link which makes it participate in the cycle. That should be sufficient to cause the rest of the cycle to collapse with __del__ methods being called from the normal refcounting mechanism. So something like this: for obj in cycle: if hasattr(obj, "__breakcycle__"): obj.__breakcycle__() break Every object which knows it can participate in a cycle then has the option of defining a method which it can use to tear down the cycle. e.g. by releasing the resource and then deleting all of its attributes, but no guarantees are made over which obj has this method called. An object with a __breakcycle__ method would have to be extra careful as its methods could still be called after it has broken the cycle, but it does mean that the responsibilities are in the right place (i.e. defining the method implies taking that into account). From gh at ghaering.de Fri Mar 31 10:26:11 2006 From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=) Date: Fri, 31 Mar 2006 10:26:11 +0200 Subject: [Python-Dev] pysqlite for 2.5? In-Reply-To: <442CC25E.1060605@v.loewis.de> References: <06Mar29.141411pst."58633"@synergy1.parc.xerox.com> <200603302303.28587.anthony@interlink.com.au> <442BF378.5060708@egenix.com> <200603310244.33110.anthony@interlink.com.au><442C0288.8080502@egenix.com> <442C03D4.1090000@ghaering.de> <442CC25E.1060605@v.loewis.de> Message-ID: <442CE7A3.5070404@ghaering.de> Martin v. L?wis wrote: > Terry Reedy wrote: > >>"Gerhard H?ring" wrote in message >>news:442C03D4.1090000 at ghaering.de... >> >>>I proposed to link dynamically on Windows, and ship the Windows >>>SQLite3.DLL. This has two advantages: >>> >>>- Python users can upgrade the SQLite3.DLL by a simple download from in >>>case of emergency >> >>+1 and thanks from a Windows user > > This binary depends on msvcrt.dll. Does anybody know whether this could > cause a problem? IOW, is any of the forbidden API used across the DLL > boundary (in particular: memory management, stdio, locales)? AFAIK SQLite does not use stdio or locales. Memory management is normally never done explicitly, but by API calls using opaque pointers to the SQLite structure, for example: - sqlite3_compile will return a SQLite statement. - sqlite3_finalize will clean up the SQLite statement, and return any memory used by it. The only explicit memory management function I know is void sqlite3_free(char *z); which looks ok, too. And pysqlite doesn't use it, anyway. -- Gerhard From mal at egenix.com Fri Mar 31 11:04:59 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 31 Mar 2006 11:04:59 +0200 Subject: [Python-Dev] unicode vs buffer (array) design issue can crash interpreter In-Reply-To: <442C1095.9070905@v.loewis.de> References: <442C1095.9070905@v.loewis.de> Message-ID: <442CF0BB.6040901@egenix.com> Martin v. L?wis wrote: > Neal Norwitz wrote: >> See http://python.org/sf/1454485 for the gory details. Basically if >> you create a unicode array (array.array('u')) and try to append an >> 8-bit string (ie, not unicode), you can crash the interpreter. >> >> The problem is that the string is converted without question to a >> unicode buffer. Within unicode, it assumes the data to be valid, but >> this isn't necessarily the case. We wind up accessing an array with a >> negative index and boom. > > There are several problems combined here, which might need discussion: > > - why does the 'u#' converter use the buffer interface if available? > it should just support Unicode objects. The buffer object makes > no promise that the buffer actually is meaningful UCS-2/UCS-4, so > u# shouldn't guess that it is. > (FWIW, it currently truncates the buffer size to the next-smaller > multiple of sizeof(Py_UNICODE), and silently so) > > I think that part should just go: u# should be restricted to unicode > objects. 'u#' is intended to match 's#' which also uses the buffer interface. It expects the buffer returned by the object to a be a Py_UNICODE* buffer, hence the calculation of the length. However, we already have 'es#' which is a lot safer to use in this respect: you can explicity define the encoding you want to see, e.g. 'unicode-internal' and the associated codec also takes care of range checks, etc. So, I'm +1 on restricting 'u#' to Unicode objects. > - should Python guarantee that all characters in a Unicode object > are between 0 and sys.maxunicode? Currently, it is possible to > create Unicode strings with either negative or very large Py_UNICODE > elements. > > - if the answer to the last question is no (i.e. if it is intentional > that a unicode object can contain arbitrary Py_UNICODE values): should > Python then guarantee that Py_UNICODE is an unsigned type? Py_UNICODE must always be unsigned. The whole implementation relies on this and has been designed with this in mind (see PEP 100). AFAICT, the configure does check that Py_UNICODE is always unsigned. Regarding the permitted range of values, I think the necessary overhead to check that all Py_UNICODE* array values are within the currently permitted range would unnecessarily slow down the implementation. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 31 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From g.brandl at gmx.net Fri Mar 31 11:44:10 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 31 Mar 2006 11:44:10 +0200 Subject: [Python-Dev] New-style icons, .desktop file Message-ID: Hi, some time ago, someone posted in python-list about icons using the Python logo from the new site design [1]. IMO they are looking great and would be a good replacement for the old non-scaling snakes on Windows in 2.5. While we're at it, Python (and IDLE) .desktop files could be added to the distribution, making it easier for newbies on *ix platforms to start up a shell or the IDE, together with .png versions of the new icons. A pre- liminary .desktop file is available at [2], I have already created a corresponding Makefile rule locally. What do people think? Georg [1] http://mail.python.org/pipermail/python-list/2006-March/332132.html [2] https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1353344&group_id=5470 From taroso at gmail.com Fri Mar 31 13:20:40 2006 From: taroso at gmail.com (Taro Ogawa) Date: Fri, 31 Mar 2006 11:20:40 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?=5BPython-3000=5D_Iterators_for_dict_keys?= =?utf-8?q?=2C_values=2C=09and_items_=3D=3D_annoying_=3A=29?= References: <4422FC96.2020409@zope.com> <1143306134.3186.1.camel@localhost.localdomain> <44258376.1080709@gmx.net> <1143432484.14391.67.camel@localhost.localdomain> <79990c6b0603271117l4c69372h11362dd5d2d0ca32@mail.gmail.com> <1143565628.3305.82.camel@localhost.localdomain> <4429F6BD.60704@canterbury.ac.nz> <79990c6b0603290129m7bed22cft3e242cd009efe9e0@mail.gmail.com> <442A6C49.5020106@gmail.com> Message-ID: Taro Ogawa gmail.com> writes: > Nick Coghlan gmail.com> writes: > > There are three big use cases: > > ... > > ... Apologies - this was posted via gmane and the post I responded to appeared in the gmane.comp.python.devel.3000 tree... I'll repost there (and check gmane a little more carefully in future). -T. From mcherm at mcherm.com Fri Mar 31 14:47:28 2006 From: mcherm at mcherm.com (Michael Chermside) Date: Fri, 31 Mar 2006 04:47:28 -0800 Subject: [Python-Dev] Class decorators Message-ID: <20060331044728.5ux7gbq0nb0gsowo@login.werra.lunarpages.com> In the discussion over class decorators, Jim Jewett writes: > I have often started with a function, and ended up replacing it with a > callable object so that I could save state without resorting to > "defalt args" or worse. > > I would prefer to decorate these exactly like the functions they replace. I have observed the entire discussion about class decorators with absolutely no opinion, until I read Jim's brief post quoted above. I am now completely convinced that class decorators ought to exist and behave exactly like function decorators. Thanks, Jim for pointing out what should have been obvious to me from the start. The ability to use callable objects as functions is a powerful tool in Python, and ought not be broken by decorator inconsistencies. -- Michael Chermside From ncoghlan at gmail.com Fri Mar 31 15:27:35 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 31 Mar 2006 23:27:35 +1000 Subject: [Python-Dev] reference leaks, __del__, and annotations In-Reply-To: References: Message-ID: <442D2E47.8070809@gmail.com> Duncan Booth wrote: > Surely if you have a cycle what you want to do is to pick just *one* of the > objects in the cycle and break the link which makes it participate in the > cycle. That should be sufficient to cause the rest of the cycle to collapse > with __del__ methods being called from the normal refcounting mechanism. > > So something like this: > > for obj in cycle: > if hasattr(obj, "__breakcycle__"): > obj.__breakcycle__() > break > > Every object which knows it can participate in a cycle then has the option > of defining a method which it can use to tear down the cycle. e.g. by > releasing the resource and then deleting all of its attributes, but no > guarantees are made over which obj has this method called. An object with a > __breakcycle__ method would have to be extra careful as its methods could > still be called after it has broken the cycle, but it does mean that the > responsibilities are in the right place (i.e. defining the method implies > taking that into account). Unfortunately, there's two problems with that idea: a. it's broken, since we now have a partially torn down object at the tail end of our former cycle. What happens if the penultimate object's finaliser tries to access that broken one? b.it doesn't actually help in the case of generators (which are the ones causing all the grief). The generator object itself (which implements the __del__ method) knows nothing about what caused the cycle (the cycle is going to be due to the Python code in the body of the generator). As PJE posted the other day, the problem is that the GC assumes that because the *type* has a __del__ method, the *instance* needs finalisation. And for objects with an explicit close method (like generators), context management semantics (like generator-based context managers), or the ability to be finalised in the normal course of events (like generator-iterators), most instances *don't* need finalisation, as they'll have already been finalised in the normal course of events. Generators are even more special, in that they only require finalisation in the first place if they're stopped on a yield statement inside a try-finally block. A simple Boolean attribute (e.g. __finalized__) should be enough. If the type has a __del__ method, then the GC would check the __finalized__ attribute. If it's both present and true, the GC can ignore the finaliser on that instance (i.e. never invokes it, and doesn't treat cycles as uncollectable because of it) I don't know the GC well enough to know how hard that would be to implement, but I suspect we need to do it (or something like it) if PEP 342 isn't going to cause annoying memory leaks in real applications. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From thomas at python.org Fri Mar 31 15:40:47 2006 From: thomas at python.org (Thomas Wouters) Date: Fri, 31 Mar 2006 15:40:47 +0200 Subject: [Python-Dev] reference leaks, __del__, and annotations In-Reply-To: References: Message-ID: <9e804ac0603310540u714e4f2fj8725313c4e5dc251@mail.gmail.com> On 3/31/06, Jim Jewett wrote: > > The checkins list has been struggling with generator reference leaks; > the latest conclusion was that some are unavoidable because of __del__ > cycles. That was Tim's conclusion, but I wasn't quite done thinking about it ;) > That sort of defeats the purpose of resource managers. (Yes, > it can be worked around on a case-by-case basis.) > > As I see it, part of the problem is that > > (1) When there is a cycle, python refuses to guess. > (2) There is no way for a __del__ method to hint at ordering constraints. > (3) There is no lightweight version of __del__ to say "I don't care > about ordering constraints." An additional (and much more complicating) problem is that __del__ can (and is allowed to) revive 'self'. That means that halfway through your cleanup, however you decided to do it, you can suddenly find out that you shouldn't be cleaning up at all. And of course any given __del__ can rely on all of the parts of the cycle, even if one of those parts _claims_ it can safely break the cycle. I think there are three different scenarios involving cycles and/or __del__ methods: - An object may conciously create a cycle, and know how to resolve it. A '__breakcycle__' method or such may be the right way to handle those cases. It would have to be pretty sure that no one outside itself can (or should) rely on the attribute or closure it breaks the cycle with, though. (Sensible exceptions ought to be fine, IMHO.) - An object may not care about cycles, but want to do some cleanup when it is deleted. The C types have 'tp_dealloc' for this, and that's what PyFile uses to close files. If you want to emulate this behaviour in Python, you are forced to use __del__, creating the unreclaimable cycle problem. Perhaps a __dealloc__ class/staticmethod makes sense; it would be passed a dictionary of instancedata, but not 'self', so it can never revive 'self' and can be sanely used in cycles -- that is, some of its instancedata may still suddenly be None, but that's a problem with __del__ methods and global variables, too. - An object may care about cycles, actually need a __del__ method that can revive the object, but not have a sane way to say 'break the cycle at this point'. Generators may be an example of that kind of object, although I'm not sure: they could throw() an exception to end the cycle. I don't think we can reclaim cycles where none of the objects can fairly break the cycle. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060331/624508fa/attachment.html From ncoghlan at gmail.com Fri Mar 31 15:58:03 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 31 Mar 2006 23:58:03 +1000 Subject: [Python-Dev] New-style icons, .desktop file In-Reply-To: References: Message-ID: <442D356B.9080505@gmail.com> Georg Brandl wrote: > Hi, > > some time ago, someone posted in python-list about icons using the Python > logo from the new site design [1]. IMO they are looking great and would > be a good replacement for the old non-scaling snakes on Windows in 2.5. Those are *really* pretty. And the self-referential PIL source code and disassembly is just plain brilliant. . . You could even use a similar style for a Python egg icon by placing the plus-Python logo in front of a file folder picture. However, the concerns raised on python-list about the similarities between the .exe and .pyc icons are valid, IMO. I also agree with Andrew Clover's own comment that having the Windows shortcut symbol cover the Python logo on the .exe is a bad thing. One way around that would be to simply use the logo itself as the icon for the executable, and talk to Andrew about using his icons for .py/.pyw and .pyc files. > While we're at it, Python (and IDLE) .desktop files could be added to the > distribution, making it easier for newbies on *ix platforms to start up > a shell or the IDE, together with .png versions of the new icons. A pre- > liminary .desktop file is available at [2], I have already created a > corresponding Makefile rule locally. Kubuntu has a Python shell as one of the options on its Terminal Sessions menu, so this part doesn't really matter to me. However, given that that box is my CPython dev machine, I probably don't qualify as one of the newbies you're targeting with this bit, anyway ;) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Fri Mar 31 16:55:54 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 01 Apr 2006 00:55:54 +1000 Subject: [Python-Dev] Class decorators In-Reply-To: <20060331044728.5ux7gbq0nb0gsowo@login.werra.lunarpages.com> References: <20060331044728.5ux7gbq0nb0gsowo@login.werra.lunarpages.com> Message-ID: <442D42FA.8060204@gmail.com> Michael Chermside wrote: > In the discussion over class decorators, Jim Jewett writes: >> I have often started with a function, and ended up replacing it with a >> callable object so that I could save state without resorting to >> "defalt args" or worse. >> >> I would prefer to decorate these exactly like the functions they replace. > > I have observed the entire discussion about class decorators with absolutely > no opinion, until I read Jim's brief post quoted above. I am now completely > convinced that class decorators ought to exist and behave exactly like > function decorators. Thanks, Jim for pointing out what should have been > obvious to me from the start. The ability to use callable objects as > functions is a powerful tool in Python, and ought not be broken by decorator > inconsistencies. While I agree with you, I don't think this conclusion is as obviously correct as it might first appear, because you don't want to decorate the class itself in such cases. You don't even want to decorate the class's __call__ method - you want to decorate an *instance* of the class, as that is what will mimic the interface of the original function. Compare: Py> def f(): ... print "Hi World from !" ... Py> f() Hi World from ! and: Py> class f(object): ... def __call__(self): ... print "Hi world from %s!" % self ... Py> f() <__main__.f object at 0x00AE1F70> Py> f()() Hi world from <__main__.f object at 0x00AE7130>! Clearly, these two definitions of 'f' are _not_ equivalent - the first one is a callable, but the latter is a callable factory. Applying the original function's decorators to the class or its __call__ method will yield nonsense. To get an object from the second approach with an interface equivalent to the original f (only with an automatically supplied mutable data store as its first argument), you instead want to write: Py> class f(object): ... def __call__(self): ... print "Hi world from %s!" % self ... Py> f = f() Py> f() Hi world from <__main__.f object at 0x00AE7190>! If the original "f" had decorators applied to it, then you would have to apply those to the final resulting bound method in the example, not to the class definition. OTOH, all is not lost, as a simple class decorator would allow Jim's use case to be satisfied quite handily: def instance(cls): return cls() Then: @deco1 @deco2 def f(): pass Could easily be replaced with: @deco1 @deco2 @instance class f(object): def __call__(self): pass Cheers, Nick. P.S. If all you want is somewhere to store mutable state between invocations, you can always use the function's own attribute space: Py> def f(): ... print "Hi world from %s!" % f ... Py> f() Hi world from ! -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From martin at v.loewis.de Fri Mar 31 17:58:34 2006 From: martin at v.loewis.de (martin at v.loewis.de) Date: Fri, 31 Mar 2006 17:58:34 +0200 Subject: [Python-Dev] x86 trunk MSI preview Message-ID: <1143820714.442d51aa5ec55@www.domainfactory-webmail.de> I just created a snapshot MSI release from the trunk (I actually recreated the 25a0 tag for that, to find that subversion will not update $HeadURL$ automatically when switching branches). Please find the file at http://www.dcl.hpi.uni-potsdam.de/home/loewis/python-2.5.13238.msi Regards, Martin From jeremy at alum.mit.edu Fri Mar 31 18:28:22 2006 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Fri, 31 Mar 2006 11:28:22 -0500 Subject: [Python-Dev] [Python-checkins] Python Regression Test Failures all (1) In-Reply-To: <20060302044631.GA22293@python.psfb.org> References: <20060302044631.GA22293@python.psfb.org> Message-ID: On 3/1/06, Neal Norwitz wrote: > test_bsddb3 > Exception in thread reader 4: > Traceback (most recent call last): > File "/home/neal/python/trunk/Lib/threading.py", line 473, in __bootstrap > self.run() > File "/home/neal/python/trunk/Lib/threading.py", line 453, in run > self.__target(*self.__args, **self.__kwargs) > File "/home/neal/python/trunk/Lib/bsddb/test/test_thread.py", line 275, in readerThread > rec = dbutils.DeadlockWrap(c.next, max_retries=10) > File "/home/neal/python/trunk/Lib/bsddb/dbutils.py", line 62, in DeadlockWrap > return function(*_args, **_kwargs) > DBLockDeadlockError: (-30996, 'DB_LOCK_DEADLOCK: Locker killed to resolve a deadlock') > > Exception in thread writer 0: > Traceback (most recent call last): > File "/home/neal/python/trunk/Lib/threading.py", line 473, in __bootstrap > self.run() > File "/home/neal/python/trunk/Lib/threading.py", line 453, in run > self.__target(*self.__args, **self.__kwargs) > File "/home/neal/python/trunk/Lib/bsddb/test/test_thread.py", line 254, in writerThread > self.assertEqual(data, self.makeData(key)) > File "/home/neal/python/trunk/Lib/unittest.py", line 334, in failUnlessEqual > (msg or '%r != %r' % (first, second)) > AssertionError: None != '0003-0003-0003-0003-0003' > > Exception in thread writer 1: > Traceback (most recent call last): > File "/home/neal/python/trunk/Lib/threading.py", line 473, in __bootstrap > self.run() > File "/home/neal/python/trunk/Lib/threading.py", line 453, in run > self.__target(*self.__args, **self.__kwargs) > File "/home/neal/python/trunk/Lib/bsddb/test/test_thread.py", line 254, in writerThread > self.assertEqual(data, self.makeData(key)) > File "/home/neal/python/trunk/Lib/unittest.py", line 334, in failUnlessEqual > (msg or '%r != %r' % (first, second)) > AssertionError: None != '1000-1000-1000-1000-1000' I haven't run regrtest.py -u all in a while. Are these sorts of errors just tolerated? Jeremy From tim.peters at gmail.com Fri Mar 31 18:47:59 2006 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 31 Mar 2006 11:47:59 -0500 Subject: [Python-Dev] [Python-checkins] Python Regression Test Failures all (1) In-Reply-To: References: <20060302044631.GA22293@python.psfb.org> Message-ID: <1f7befae0603310847t67af30a0w8c4aaaac6c61633b@mail.gmail.com> [Neal Norwitz] """ test_bsddb3 Exception in thread reader 4: Traceback (most recent call last): File "/home/neal/python/trunk/Lib/threading.py", line 473, in __bootstrap self.run() File "/home/neal/python/trunk/Lib/threading.py", line 453, in run self.__target(*self.__args, **self.__kwargs) File "/home/neal/python/trunk/Lib/bsddb/test/test_thread.py", line 275, in readerThread rec = dbutils.DeadlockWrap(c.next, max_retries=10) File "/home/neal/python/trunk/Lib/bsddb/dbutils.py", line 62, in DeadlockWrap return function(*_args, **_kwargs) DBLockDeadlockError: (-30996, 'DB_LOCK_DEADLOCK: Locker killed to resolve a deadlock') Exception in thread writer 0: Traceback (most recent call last): File "/home/neal/python/trunk/Lib/threading.py", line 473, in __bootstrap self.run() File "/home/neal/python/trunk/Lib/threading.py", line 453, in run self.__target(*self.__args, **self.__kwargs) File "/home/neal/python/trunk/Lib/bsddb/test/test_thread.py", line 254, in writerThread self.assertEqual(data, self.makeData(key)) File "/home/neal/python/trunk/Lib/unittest.py", line 334, in failUnlessEqual (msg or '%r != %r' % (first, second)) AssertionError: None != '0003-0003-0003-0003-0003' [and so on] """ [Jeremy Hylton] > I haven't run regrtest.py -u all in a while. Are these sorts of > errors just tolerated? test_bsddb3 has always failed in these ways, and I believe on all platforms, although on my WinXP box it only happens if my box is heavily loaded with other stuff. Normally a DBLockDeadlockError doesn't even cause the test to fail, presumably because the exception occurs in a thread unittest doesn't know about (you'll recall the convolutions we endured to get unittest to notice errors in ZODB/ZEO test threads). For example, here's a recent buildbot run where test_bsddb3 passed despite a DBLockDeadlockError: http://www.python.org/dev/buildbot/all/x86%20gentoo%20trunk/builds/223/step-test/0 The prospects for fixing it appear dim; PCbuild/readme.txt has noted for years "I'm told that DBLockDeadlockError is expected at times". From pje at telecommunity.com Fri Mar 31 18:52:05 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 31 Mar 2006 11:52:05 -0500 Subject: [Python-Dev] Class decorators In-Reply-To: <20060331044728.5ux7gbq0nb0gsowo@login.werra.lunarpages.com > Message-ID: <5.1.1.6.0.20060331114747.01e0a210@mail.telecommunity.com> At 04:47 AM 3/31/2006 -0800, Michael Chermside wrote: >In the discussion over class decorators, Jim Jewett writes: > > I have often started with a function, and ended up replacing it with a > > callable object so that I could save state without resorting to > > "defalt args" or worse. > > > > I would prefer to decorate these exactly like the functions they replace. > >I have observed the entire discussion about class decorators with absolutely >no opinion, until I read Jim's brief post quoted above. I am now completely >convinced that class decorators ought to exist and behave exactly like >function decorators. Thanks, Jim for pointing out what should have been >obvious to me from the start. The ability to use callable objects as >functions is a powerful tool in Python, and ought not be broken by decorator >inconsistencies. Unless the class has a metaclass implementing __call__, or you mean that you want instance creation to be a call, I don't understand what you mean. Nonetheless, the discussion has only been about *where* the decorators go and what syntax they use. Nobody has proposed any change in decorator semantics, so please stop attacking this meaningless strawman. Moving from: @foo def bar(...): ... to: class bar: @class foo def __init___(...): ... instead of: @foo class bar: def __init___(...): ... is a trivial difference in editing: you type "class " one more time. From pje at telecommunity.com Fri Mar 31 18:57:21 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 31 Mar 2006 11:57:21 -0500 Subject: [Python-Dev] reference leaks, __del__, and annotations In-Reply-To: <442D2E47.8070809@gmail.com> References: Message-ID: <5.1.1.6.0.20060331115351.039bc6b8@mail.telecommunity.com> At 11:27 PM 3/31/2006 +1000, Nick Coghlan wrote: >Generators are even more special, in that they only require finalisation in >the first place if they're stopped on a yield statement inside a try-finally >block. Or a try-except block. Or a 'with' statement. It's only loop blocks that are exempt. >A simple Boolean attribute (e.g. __finalized__) should be enough. If the type >has a __del__ method, then the GC would check the __finalized__ attribute. If >it's both present and true, the GC can ignore the finaliser on that instance >(i.e. never invokes it, and doesn't treat cycles as uncollectable because >of it) > >I don't know the GC well enough to know how hard that would be to implement, >but I suspect we need to do it (or something like it) if PEP 342 isn't going >to cause annoying memory leaks in real applications. As Tim suggested, it'd be better to have the code be generator-specific, at least for now. That had actually been my original plan, to make it generator-specific, but I was afraid of breaking encapsulation in the garbage collector by having it know about generators. But now that Uncle Timmy has blessed the approach, I'll go back and add it in. (On Monday, unless somebody gets to it before me.) From tim.peters at gmail.com Fri Mar 31 19:14:36 2006 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 31 Mar 2006 12:14:36 -0500 Subject: [Python-Dev] reference leaks, __del__, and annotations In-Reply-To: <5.1.1.6.0.20060331115351.039bc6b8@mail.telecommunity.com> References: <442D2E47.8070809@gmail.com> <5.1.1.6.0.20060331115351.039bc6b8@mail.telecommunity.com> Message-ID: <1f7befae0603310914q644aafb4t2ffc4efaa0718415@mail.gmail.com> [Phillip J. Eby] > ... > As Tim suggested, it'd be better to have the code be generator-specific, at > least for now. That had actually been my original plan, to make it > generator-specific, but I was afraid of breaking encapsulation in the > garbage collector by having it know about generators. It sucks in a way, but so would adding yet another new slot just for (at present, and possibly forever) making gc and generators play nicer together. "Practicality beats purity" here. > But now that Uncle Timmy has blessed the approach, I'll go back and add it in. > (On Monday, unless somebody gets to it before me.) It won't be me: I wasn't even able to make enough time to understand the new generator features at the Python level, let alone the implementation. At PyCon, when Guido showed his slide with a new yield-as-expression example, for the rest of his talk I was wondering what the heck the example meant <0.3 wink>. From pje at telecommunity.com Fri Mar 31 19:24:18 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 31 Mar 2006 12:24:18 -0500 Subject: [Python-Dev] reference leaks, __del__, and annotations In-Reply-To: <1f7befae0603310914q644aafb4t2ffc4efaa0718415@mail.gmail.co m> References: <5.1.1.6.0.20060331115351.039bc6b8@mail.telecommunity.com> <442D2E47.8070809@gmail.com> <5.1.1.6.0.20060331115351.039bc6b8@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060331121741.01e345e8@mail.telecommunity.com> At 12:14 PM 3/31/2006 -0500, Tim Peters wrote: >[Phillip J. Eby] > > ... > > As Tim suggested, it'd be better to have the code be generator-specific, at > > least for now. That had actually been my original plan, to make it > > generator-specific, but I was afraid of breaking encapsulation in the > > garbage collector by having it know about generators. > >It sucks in a way, but so would adding yet another new slot just for >(at present, and possibly forever) making gc and generators play nicer >together. "Practicality beats purity" here. > > > But now that Uncle Timmy has blessed the approach, I'll go back and add > it in. > > (On Monday, unless somebody gets to it before me.) > >It won't be me: I wasn't even able to make enough time to understand >the new generator features at the Python level, let alone the >implementation. At PyCon, when Guido showed his slide with a new >yield-as-expression example, for the rest of his talk I was wondering >what the heck the example meant <0.3 wink>. Think of yield as being a sort of IPC operator: it sends a value out of the generator, and then replaces it with a "return value" that defaults to None. On the opposite side of the communication, calling send(value) allows you to put a value "into" the generator, and returns the next value from the generator. The existing next() operation is effectively send(None). So, it's effectively an asymmetric IPC system, using a yield operator and a send() method. On either side, the operation takes a value to give to the other side of the communication, and then returns the value that's passed "back" by the other side. that takes an expression, sends it out of the generator, and then brings back in a "return value" - sort of like sending some kind of IPC message out of the process, and receiving a response. From fdrake at acm.org Fri Mar 31 20:48:50 2006 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Fri, 31 Mar 2006 13:48:50 -0500 Subject: [Python-Dev] Class decorators In-Reply-To: <5.1.1.6.0.20060331114747.01e0a210@mail.telecommunity.com> References: <5.1.1.6.0.20060331114747.01e0a210@mail.telecommunity.com> Message-ID: <200603311348.51165.fdrake@acm.org> On Friday 31 March 2006 11:52, Phillip J. Eby wrote: > class bar: > @class foo > def __init___(...): > ... The more I think about it, the more I like the "@class foo" syntax. The existing syntax for functions doesn't have anything between the decorators and the "def"; having class decorators embedded within the class should still allow the docstring to be the first thing, so there's more distance between the decorator and the name being decorated. The extra hint about what's being decorated is nice. -Fred -- Fred L. Drake, Jr. From brett at python.org Fri Mar 31 21:48:37 2006 From: brett at python.org (Brett Cannon) Date: Fri, 31 Mar 2006 11:48:37 -0800 Subject: [Python-Dev] I'm not getting email from SF when assigned abug/patch In-Reply-To: References: <1143648409.10799.159.camel@resist.wooz.org> <442C11E2.6050106@v.loewis.de> <442C1C7E.4050502@v.loewis.de> Message-ID: On 3/30/06, Fredrik Lundh wrote: > Brett Cannon wrote: > > > Same here. Please move any more comments about infrastructure to the > > infrastructure list (http://mail.python.org/mailman/listinfo/infrastructure/). But > > do realize the committee is not discussing trackers yet. We are still > > trying to get our SF data out so that can be imported into a tracker > > to test its use > > oh, I forgot that the Procrastination & Stop energy Foundation was involved > in this. > Fredrik, if you would like to help move this all forward, great; I would appreciate the help. You can write a page scraper to get the data out of SF if you don't believe SF will cooperate fast enough for your tastes or I am giving them too long to try to fix things on their end (I don't expect they will fix things fast enough either, but because of school ending soon I don't have time right now to start working on a scraper myself plus I am willing to give them a little time to try to fix the XML export so we can minimize headaches on our end). If you would rather contribute by collecting a list of possible trackers along with who will maintain it, then please do. I am not going to dive into that quite yet, but if you want to parallelize the work needed then I would appreciate the help. The tracker will need to be able to import the SF data somehow (probably will require a custom tool so the volunteers need to be aware of this), be able to export data (so we can back it up on a regular basis so we don't have to go through this again), and an email interface for at least replying to tracker items. A community-wide announcement will probably be needed to get a good group of volunteers together for any one non-commercial tracker. But I am not procrastinating. I don't think I have ever come off as a procrastinator on this list and I don't think I deserve the label. I am not putting more time in now because I am near the end of term here at school and thus passing my courses takes precendent over a new bug tracker. I ended up the chairman at the infrastructure committee during one of my busiest school terms I have ever had. But I will see this through and it will be done in a timely manner. -Brett From tjreedy at udel.edu Fri Mar 31 22:13:23 2006 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 31 Mar 2006 15:13:23 -0500 Subject: [Python-Dev] gmane.comp.python.devel.3000 has disappeared Message-ID: For about a week, I have been reading and occasionally posting to the new pydev-3000 mailing list via the gmane mirror gmane.comp.lang.devel.3000. Today, it has disappeared and was still gone after reloading their newsgroup list. Was this intentional on the part of the mail list maintainers? (I certainly hope not!) Or is it a repairable glitch somewhere between the list and gmane? Terry Jan Reedy From guido at python.org Fri Mar 31 22:38:52 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 31 Mar 2006 12:38:52 -0800 Subject: [Python-Dev] gmane.comp.python.devel.3000 has disappeared In-Reply-To: References: Message-ID: Wasn't my intention. gmane is black magic to me (I've never used it) so I can't be much help debugging this... I did add 3 new admins and changed the list password. --Guido On 3/31/06, Terry Reedy wrote: > For about a week, I have been reading and occasionally posting to the new > pydev-3000 mailing list via the gmane mirror gmane.comp.lang.devel.3000. > Today, it has disappeared and was still gone after reloading their > newsgroup list. Was this intentional on the part of the mail list > maintainers? (I certainly hope not!) Or is it a repairable glitch > somewhere between the list and gmane? -- --Guido van Rossum (home page: http://www.python.org/~guido/)