From noamraph at gmail.com Sun Jan 1 00:08:26 2006 From: noamraph at gmail.com (Noam Raphael) Date: Sun, 1 Jan 2006 01:08:26 +0200 Subject: [Python-Dev] When do sets shrink? In-Reply-To: <002f01c60e4c$23b97d60$2827a044@oemcomputer> References: <002f01c60e4c$23b97d60$2827a044@oemcomputer> Message-ID: On 12/31/05, Raymond Hettinger wrote: > [Noam] > > For example, iteration over a set which once had > > 1,000,000 members and now has 2 can take 1,000,000 operations every > > time you traverse all the (2) elements. > > Do you find that to be a common or plausible use case? I don't have a concrete example in this minute, but a set which is repeatedly filled with elements and then emptied by pop operations doesn't seem to me that far-fetched. > > Was Guido's suggestion of s=set(s) unworkable for some reason? dicts > and sets emphasize fast lookups over fast iteration -- apps requiring > many iterations over a collection may be better off converting to a list > (which has no dummy entries or empty gaps between entries). It's workable, but I think that most Python programmers haven't read that specific message, and are expecting operations which should take a short time to take a short time. Converting to a list won't help the use-case above, and anyway, it's another thing that I wouldn't expect anyone to do - there's no reason that iteration over a set should take a long time. (I'm speaking of my point of view, which I believe is common. I don't expect programs I write in Python to be super-fast - if that were the case, I would write them in C. I do expect them to take a reasonable amount of time, and in the case of iteration over a set, that means a time proportional to the number of elements in the set.) > > Would the case be improved by incurring the time cost of 999,998 tests > for possible resizing (one for each pop) and some non-trivial number of > resize operations along the way (each requiring a full-iteration over > the then current size)? > I believe it would. It seems to me that those 999,998 tests take not much more than a machine clock, which means about 1 milisecond on todays computers. Those resize operations will take some more miliseconds. It all doesn't really matter, since probably all other things will take much more. I now run this code >>> s = set() >>> for j in xrange(1000000): ... s.add(j) ... >>> while s: ... tmp = s.pop() ... And it took 2.4 seconds to finish. And it's okay - I'm just saying that a few additional clock ticks per operation will usually not matter when the overall complexity is the same, but changes in order of complexity can matter much more. > Even if this unique case could be improved, what is the impact on common > cases? Would a downsizing scheme risk thrashing with the > over-allocation scheme in cases with mixed adds and pops? > I think that there shouldn't be additional damage beyond those clock ticks. The simple method I took from "Introduction to Algorithms" works no matter what sequence of adds and pops you have. > Is there any new information/research beyond what has been obvious from > the moment the dict resizing scheme was born? I wanted to say that there isn't any new information, and yet I don't think that I have to assume that everything in current Python is the best that can be. All I did was finding another reason why a downsizing scheme might be good, and posting it to ask if people have thought about it. If you have a document listing all the design decisions that went into dict implementation, then please send it to me and I won't ask about things that were already thought about. But the answer is, yes. I beleive that the current dict resizing scheme was born before the iterator protocol was introduced, and it may be a reason why the current scheme doesn't try to minimize the number of empty hash table entries. Noam From fperez.net at gmail.com Sun Jan 1 01:27:50 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 31 Dec 2005 17:27:50 -0700 Subject: [Python-Dev] When do sets shrink? References: <002f01c60e4c$23b97d60$2827a044@oemcomputer> Message-ID: Raymond Hettinger wrote: > Was Guido's suggestion of s=set(s) unworkable for some reason? dicts > and sets emphasize fast lookups over fast iteration -- apps requiring > many iterations over a collection may be better off converting to a list > (which has no dummy entries or empty gaps between entries). > > Would the case be improved by incurring the time cost of 999,998 tests > for possible resizing (one for each pop) and some non-trivial number of > resize operations along the way (each requiring a full-iteration over > the then current size)? Note that this is not a comment on the current discussion per se, but rather a small request/idea in the docs department: I think it would be a really useful thing to have a summary page/table indicating the complexities of the various operations on all the builtin types, including at least _mention_ of subtleties and semi-gotchas. Python is growing in popularity, and it is being used for more and more demanding tasks all the time. Such a 'complexity overview' of the language's performance would, I think, be very valuable to many. I know that much of this information is available, but I'm talking about a specific summary, which also discusses things like Noam's issue. For example, I had never realized that on dicts, for some O(N) operations, N would mean "largest N in the dict's history" instead of "current number of elements". While I'm not arguing for any changes, I think it's good to _know_ this, so I can plan for it if I am ever in a situation where it may be a problem. Just my 1e-2. And Happy New Year to the python-dev team, with many thanks for all your fantastic work on making the most pleasant, useful programming language out there. Cheers, f From jcarlson at uci.edu Sun Jan 1 01:42:19 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat, 31 Dec 2005 16:42:19 -0800 Subject: [Python-Dev] When do sets shrink? In-Reply-To: References: <002f01c60e4c$23b97d60$2827a044@oemcomputer> Message-ID: <20051231155926.BF5D.JCARLSON@uci.edu> Noam Raphael wrote: > > On 12/31/05, Raymond Hettinger wrote: > > [Noam] > > > For example, iteration over a set which once had > > > 1,000,000 members and now has 2 can take 1,000,000 operations every > > > time you traverse all the (2) elements. > > > > Do you find that to be a common or plausible use case? > > I don't have a concrete example in this minute, but a set which is > repeatedly filled with elements and then emptied by pop operations > doesn't seem to me that far-fetched. It doesn't seem far-fetched, but I've not seen anything like it. List appending and popping, yeah, set differences and intersections and unions, yeah, but not set insertion then removal for large numbers of items. Note that you provide insertion into a set then repeated popping as an example, which is done faster by other methods. > (I'm speaking of my point of view, which I believe is common. I don't > expect programs I write in Python to be super-fast - if that were the > case, I would write them in C. I do expect them to take a reasonable > amount of time, and in the case of iteration over a set, that means a > time proportional to the number of elements in the set.) That is a reasonable point of view. But realize that depending on the shrinking strategy, popping/deletion will take ammortized 2+ times longer than they do now, and whose benefits include (and are basically limited to): memory cam be freed to the operating system, repeated iteration over a resized-smaller dictionary can be faster. > > Would the case be improved by incurring the time cost of 999,998 tests > > for possible resizing (one for each pop) and some non-trivial number of > > resize operations along the way (each requiring a full-iteration over > > the then current size)? > > > I believe it would. It seems to me that those 999,998 tests take not > much more than a machine clock, which means about 1 milisecond on > todays computers. Those resize operations will take some more > miliseconds. It all doesn't really matter, since probably all other > things will take much more. I now run this code > > >>> s = set() > >>> for j in xrange(1000000): > ... s.add(j) > ... > >>> while s: > ... tmp = s.pop() > ... > > And it took 2.4 seconds to finish. And it's okay - I'm just saying > that a few additional clock ticks per operation will usually not > matter when the overall complexity is the same, but changes in order > of complexity can matter much more. Doing that while loop will take _longer_ with a constantly resizing set. The only way that resizing a dict/set as it gets smaller will increase overall running speed is if iteration over the dict/set occurs anywhere between 2-100 times (depending on the resizing factor) > > Even if this unique case could be improved, what is the impact on common > > cases? Would a downsizing scheme risk thrashing with the > > over-allocation scheme in cases with mixed adds and pops? > > > I think that there shouldn't be additional damage beyond those clock > ticks. The simple method I took from "Introduction to Algorithms" > works no matter what sequence of adds and pops you have. You may get more memory fragmentation, depending on the underlying memory manager. > > Is there any new information/research beyond what has been obvious from > > the moment the dict resizing scheme was born? > > I wanted to say that there isn't any new information, and yet I don't > think that I have to assume that everything in current Python is the > best that can be. All I did was finding another reason why a > downsizing scheme might be good, and posting it to ask if people have > thought about it. If you have a document listing all the design > decisions that went into dict implementation, then please send it to > me and I won't ask about things that were already thought about. See the source for dictobject.c and dictnotes.txt: http://svn.python.org/view/python/trunk/Objects/dictobject.c?rev=39608&view=auto http://svn.python.org/view/python/trunk/Objects/dictnotes.txt?rev=35428&view=auto - Josiah From tim.peters at gmail.com Sun Jan 1 02:21:23 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sat, 31 Dec 2005 20:21:23 -0500 Subject: [Python-Dev] When do sets shrink? In-Reply-To: References: <002f01c60e4c$23b97d60$2827a044@oemcomputer> Message-ID: <1f7befae0512311721k27ba3582p7ad64d6f37a66f74@mail.gmail.com> [Noam Raphael] >>> For example, iteration over a set which once had >>> 1,000,000 members and now has 2 can take 1,000,000 operations every >>> time you traverse all the (2) elements. [Raymond Hettinger] >> Do you find that to be a common or plausible use case? [Naom] > I don't have a concrete example in this minute, but a set which is > repeatedly filled with elements and then emptied by pop operations > doesn't seem to me that far-fetched. Ah, but that's an entirely different example than the one you started with: every detail counts when you're looking for "bad" cases. In this new example, the set _does_ get resized, as soon as you start adding elements again. OTOH, in the absence of repeated iteration too, it's not clear that this resizing helps more than it hurts. ... > I wanted to say that there isn't any new information, and yet I don't > think that I have to assume that everything in current Python is the > best that can be. It was in 2005; 2006 is an entirely different year ;-) > All I did was finding another reason why a downsizing scheme might > be good, and posting it to ask if people have thought about it. Not all that much -- sets whose sizes bounce around a lot, and which are also iterated over a lot, haven't stuck out as an important use case. Typically, if a set or dict gets iterated over at all, that happens once near the end of its life. > If you have a document listing all the design decisions that went into > dict implementation, then please send it to me and I won't ask about > things that were already thought about. Lots of info in the source; Josiah already pointed at the most useful dict docs. > But the answer is, yes. I beleive that the current dict resizing > scheme was born before the iterator protocol was introduced, and it > may be a reason why the current scheme doesn't try to minimize the > number of empty hash table entries. Dict resizing was designed before the Python-level iteration protocol, but under the covers dicts offered the PyDict_Next() C-level iteration protocol "forever". It's not the iteration protocol (or lack thereof) that drives this. Far more important is that dicts have always been heavily used by Python itself, in its own implementation, for a variety of namespaces: the global dict, originally the local dict too, for class dicts and instance dicts, and to pass keyword arguments. Note that all those cases use strings for keys, and in fact Python originally supported only string-keyed dicts. In all those cases too, deletions are at worst very rare, and iteration a minor use case. From raymond.hettinger at verizon.net Sun Jan 1 02:23:06 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sat, 31 Dec 2005 20:23:06 -0500 Subject: [Python-Dev] When do sets shrink? In-Reply-To: <20051231155926.BF5D.JCARLSON@uci.edu> Message-ID: <000101c60e71$eda5d7c0$e916c797@oemcomputer> > > > [Noam] > > > > For example, iteration over a set which once had > > > > 1,000,000 members and now has 2 can take 1,000,000 operations every > > > > time you traverse all the (2) elements. > > > > > > Do you find that to be a common or plausible use case? > > > > I don't have a concrete example in this minute, but a set which is > > repeatedly filled with elements and then emptied by pop operations > > doesn't seem to me that far-fetched. > > It doesn't seem far-fetched, but I've not seen anything like it. It's more far-fetched when fully spelled-out: Build a VERY large list, ALMOST empty it with pop operations, then iterate over it MANY times (enough to offset the cost of multiple resize operations with their attendant memory allocator interactions and the expensive block copies (cache misses are a certitude and each miss is as expensive as a floating point divide)). Also note, this example was not selected from a real-world use-case; it was contrived for purposes of supporting an otherwise weak proposal. > > > Would the case be improved by incurring the time cost of 999,998 tests > > > for possible resizing (one for each pop) and some non-trivial number > of > > > resize operations along the way (each requiring a full-iteration over > > > the then current size)? > > > > > I believe it would. It seems to me that those 999,998 tests take not > > much more than a machine clock, which means about 1 milisecond on > > todays computers. Those resize operations will take some more > > miliseconds. It all doesn't really matter, since probably all other > > things will take much more. I now run this code > > > Doing that while loop will take _longer_ with a constantly resizing set. > The only way that resizing a dict/set as it gets smaller will increase > overall running speed is if iteration over the dict/set occurs anywhere > between 2-100 times (depending on the resizing factor) Josiah is exactly correct. The resize operations are enormously expensive compared to the cost of an iteration. You would have to do the latter many times to make up for the costs of repeatedly downsizing the set. > > > Even if this unique case could be improved, what is the impact on > common > > > cases? Would a downsizing scheme risk thrashing with the > > > over-allocation scheme in cases with mixed adds and pops? > > > > > I think that there shouldn't be additional damage beyond those clock > > ticks. The simple method I took from "Introduction to Algorithms" > > works no matter what sequence of adds and pops you have. > > You may get more memory fragmentation, depending on the underlying > memory manager. There's more a risk than fragmentation. Trashing is a basic concern. There is no way around it -- some combination of adds and pops always triggers it when both upsizing and downsizing logic are present. The code in listobject.c works hard to avoid this but there are still patterns which would trigger horrid behavior with a resize occurring every few steps. > > > Is there any new information/research beyond what has been obvious > from > > > the moment the dict resizing scheme was born? > > > > I wanted to say that there isn't any new information, and yet I don't > > think that I have to assume that everything in current Python is the > > best that can be. All I did was finding another reason why a > > downsizing scheme might be good, and posting it to ask if people have > > thought about it. If you have a document listing all the design > > decisions that went into dict implementation, then please send it to > > me and I won't ask about things that were already thought about. > > See the source for dictobject.c and dictnotes.txt: > http://svn.python.org/view/python/trunk/Objects/dictobject.c?rev=39608&v ie > w=auto > http://svn Those are both good references. The code for general purpose dicts has been fine-tuned, reviewed, and field-tested to a highly polished level. It is at a point where most attempts to improve it will make it worse-off. There may be some room for development in special versions of dictionaries for specific use cases. For instance, it may be worthwhile to create a version emphasizing size or access speed over insertion time (using Brent's variation to optimizing search order). Raymond From raymond.hettinger at verizon.net Sun Jan 1 04:09:22 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sat, 31 Dec 2005 22:09:22 -0500 Subject: [Python-Dev] When do sets shrink? In-Reply-To: Message-ID: <000201c60e80$c3c35900$e916c797@oemcomputer> [Fernando Perez] > Note that this is not a comment on the current discussion per se, but > rather a > small request/idea in the docs department: I think it would be a really > useful > thing to have a summary page/table indicating the complexities of the > various > operations on all the builtin types, including at least _mention_ of > subtleties > and semi-gotchas. The wiki might be the place to cover this sort of thing. Unlike infrequent doc releases, the medium is good at being responsive to whatever someone thought important enough to write an entry for. Also, it is more easily kept up-to-date for variations between versions (Py2.4, Py2.5, etc.) and implementations (CPython, Jython, etc.). The relevant list of these ideas may be somewhat short: * mystring += frag # use ''.join() instead * mylist.insert(0, obj) # takes O(n) time to move all the elements * if x in y: # runs in O(n) time if y is a sequence I think the number one performance gotcha is adopting a COBOL-like code writing mentality and failing to creatively use Python's powerful collection types: list, tuple, dict, set, str and an occasional array, deque, or cStringIO object. > For example, I had never realized that on dicts, for some O(N) operations, > N > would mean "largest N in the dict's history" instead of "current number of > elements". It might be better to give more generic advice that tends to be true across implementations and versions: "Dense collections like lists tuples iterate faster than sparse structures like dicts and sets. Whenever repeated iteration starts to dominate application run-time, consider converting to a dense representation for faster iteration and better memory/cache utilization." A statement like this remains true whether or not a down-sizing algorithm is present. > Cheers, > > f Hmm, your initial may be infringing on another developer's trademarked signature ;-) Raymond Side note: To some degree, ignorance is bliss. Most of my code was written in AWK and I was content to know only one non-algorithmic optimization ("exp" vs /exp/). Time was spent thinking about the problem at hand rather than how to outsmart the interpreter. Knowing too much about the implementation can be a distraction. Besides, when timing does become critical, people seem to suddenly become spontaneously ingenious. From kbk at shore.net Sun Jan 1 06:08:59 2006 From: kbk at shore.net (Kurt B. Kaiser) Date: Sun, 1 Jan 2006 00:08:59 -0500 (EST) Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200601010508.k0158xMh031318@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 382 open ( +3) / 3003 closed ( +1) / 3385 total ( +4) Bugs : 903 open (-11) / 5479 closed (+27) / 6382 total (+16) RFE : 203 open ( -1) / 195 closed ( +2) / 398 total ( +1) New / Reopened Patches ______________________ NotImplemented->TypeError in __add__ and __mul__ (2005-12-26) CLOSED http://python.org/sf/1390657 opened by Armin Rigo dict.merge (2005-12-27) http://python.org/sf/1391204 opened by Nicolas Lehuen cookielib LWPCookieJar and MozillaCookieJar exceptions (2005-02-06) http://python.org/sf/1117398 reopened by jjlee Optional second argument for startfile (2005-12-29) http://python.org/sf/1393157 opened by Thomas Heller Add restart debugger command to pdb.py (2005-12-30) http://python.org/sf/1393667 opened by Rocky Bernstein Patches Closed ______________ NotImplemented->TypeError in __add__ and __mul__ (2005-12-26) http://python.org/sf/1390657 closed by arigo weakref callbacks are called only if the weakref is alive (2005-12-12) http://python.org/sf/1379023 closed by arigo New / Reopened Bugs ___________________ Incorrectly docs for return values of set update methods (2005-12-24) CLOSED http://python.org/sf/1389673 opened by Collin Winter Fxn call in _elementtree.c has incorrect signedness (2005-12-24) CLOSED http://python.org/sf/1389809 opened by Brett Cannon _winreg specifies EnvironmentError instead of WindowsError (2005-12-21) CLOSED http://python.org/sf/1386675 reopened by birkenfeld ScrolledText hungs up in some conditions (2005-12-25) http://python.org/sf/1390086 opened by dani_filth README mention --without-cxx (2005-12-25) http://python.org/sf/1390321 opened by Aahz time docs lack %F in strftime! (2005-12-26) CLOSED http://python.org/sf/1390605 opened by Nikos Kouremenos split() breaks no-break spaces (2005-12-26) CLOSED http://python.org/sf/1390608 opened by MvR time.strftime('%F', local_time) is okay but time.strptime no (2005-12-26) CLOSED http://python.org/sf/1390629 opened by Nikos Kouremenos lambda functions confused when mapped in dictionary object (2005-12-27) CLOSED http://python.org/sf/1390991 opened by Samuel Hsiung missing module names in email package (2005-12-27) http://python.org/sf/1391608 opened by Gabriel Genellina floating point literals don't work in non-US locale in 2.5 (2005-12-28) http://python.org/sf/1391872 opened by Fredrik Lundh build fails on BSD 3.8 (2005-12-30) http://python.org/sf/1392915 opened by George Yoshida cannot build SVN trunk on old systems (2005-12-29) http://python.org/sf/1393109 opened by Fredrik Lundh Deleting first item causes anydbm.first() to fail (2005-12-30) http://python.org/sf/1394135 opened by Dan Bisalputra urllib2 raises exception when page redirects to itself (2005-12-31) CLOSED http://python.org/sf/1394453 opened by Ren?? Pijlman SimpleHTTPServer doesn't understand query arguments (2005-12-31) http://python.org/sf/1394565 opened by Aaron Swartz 'Plus' filemode exposes uninitialized memory on win32 (2005-12-31) http://python.org/sf/1394612 opened by Cory Dodt Bugs Closed ___________ Decimal sqrt() ignores rounding (2005-12-23) http://python.org/sf/1388949 closed by facundobatista Incorrect docs for return values of set update methods (2005-12-24) http://python.org/sf/1389673 closed by birkenfeld Fxn call in _elementtree.c has incorrect signedness (2005-12-25) http://python.org/sf/1389809 closed by effbot _winreg specifies EnvironmentError instead of WindowsError (2005-12-21) http://python.org/sf/1386675 closed by birkenfeld time docs lack %F in strftime! (2005-12-26) http://python.org/sf/1390605 closed by birkenfeld split() breaks no-break spaces (2005-12-26) http://python.org/sf/1390608 closed by lemburg time.strftime('%F', local_time) is okay but time.strptime no (2005-12-26) http://python.org/sf/1390629 closed by birkenfeld metaclasses, __getattr__, and special methods (2003-04-29) http://python.org/sf/729913 closed by arigo special methods become static (2004-11-15) http://python.org/sf/1066490 closed by arigo len() on class broken (2005-12-16) http://python.org/sf/1382740 closed by arigo urllib.url2pathname, pathname2url doc strings inconsistent (2002-12-07) http://python.org/sf/649974 closed by birkenfeld PyLong_AsVoidPtr()/PyLong_FromVoidPtr() (2002-12-14) http://python.org/sf/653542 closed by birkenfeld Acrobat Reader 5 compatibility (2003-04-14) http://python.org/sf/721160 closed by birkenfeld Calling socket.recv() with a large number breaks (2003-06-17) http://python.org/sf/756104 closed by birkenfeld Automated daily documentation builds (2002-06-26) http://python.org/sf/574241 closed by birkenfeld String formatting operator % badly documented (2003-11-10) http://python.org/sf/839585 closed by birkenfeld Document that highly recursive data cannot be pickled (2003-11-10) http://python.org/sf/839075 closed by birkenfeld os.exec* and first 'arg' (2003-11-19) http://python.org/sf/845342 closed by birkenfeld Error in urllib2 Examples (2004-05-16) http://python.org/sf/954981 closed by birkenfeld Lib/profile.doc should be updated (2001-12-05) http://python.org/sf/489256 closed by birkenfeld exec statement link in index broken (2005-12-19) http://python.org/sf/1385004 closed by birkenfeld Tutorial errors (2005-12-07) http://python.org/sf/1375599 closed by birkenfeld Dict docstring error Python-2.3.5 (2005-02-26) http://python.org/sf/1152424 closed by birkenfeld lambda functions confused when mapped in dictionary object (2005-12-27) http://python.org/sf/1390991 closed by birkenfeld Can't build Python on POSIX w/o $HOME (2004-05-24) http://python.org/sf/959576 closed by birkenfeld Setup.local ignored by setup.py (2004-07-29) http://python.org/sf/999767 closed by birkenfeld PyXxx_Check(x) trusts x->ob_type->tp_mro (2005-02-27) http://python.org/sf/1153075 closed by arigo urllib2 raises exception when page redirects to itself (2005-12-31) http://python.org/sf/1394453 closed by rpijlman New / Reopened RFE __________________ tempfile misses usecase which requirs renaming (2005-12-25) http://python.org/sf/1390197 opened by Dennis Allison RFE Closed __________ split() string method has two splitting algorithms (2005-11-28) http://python.org/sf/1367936 closed by rhettinger future warning in commets (2005-02-18) http://python.org/sf/1144057 closed by mcherm From stephen at xemacs.org Sun Jan 1 20:19:45 2006 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 02 Jan 2006 04:19:45 +0900 Subject: [Python-Dev] [Doc-SIG] that library reference, again In-Reply-To: <43B700A2.70701@colorstudy.com> (Ian Bicking's message of "Sat, 31 Dec 2005 16:05:22 -0600") References: <4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com> <43B4A238.2030101@python.org> <43B55FE8.1040401@python.org> <43B58F09.6090803@colorstudy.com> <43B61A1E.9080300@gmail.com> <43B700A2.70701@colorstudy.com> Message-ID: <87ek3rpxgu.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Ian" == Ian Bicking writes: Ian> Nick Coghlan wrote: >> While I quite like this idea, would it make it more difficult >> when the bug tracking for the main source code is eventually >> migrated off SF? And what would happen to existing >> documentation bug reports/patches on the SF trackers? Ian> I think the requirements for documentation are a bit lighter, Ian> so it's not as big a deal. E.g., the history of bug reports Ian> on documentation isn't as important, either the ones from SF, Ian> or if all of Python moves to a new system then the history of Ian> the transitional system. This is almost true, but Ian> Documentation is mostly self-describing. is not. Python maintains several "supported" releases simultaneously. There are important differences between the Python language and the implementation being discussed on this list (cPython). These issues (and their historical tracking) are not particularly important in the current workflow because most of the people actually touching the docs are highly experienced experts. They will get this stuff right more or less naturally, in part because they're working on documents that are directly associated with a particular version of a particular implementation. But as the documentation process moves to a more fluid, user-driven process, that expertise will be diluted (even with proposals that the last phase be edited by a dedicated expert, that expert's time commitment will surely be strained). That means that history tracking will become more important, and documentation meta-data (like which version of which implementation does this change apply to) much more so. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From fperez.net at gmail.com Sun Jan 1 21:06:03 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 01 Jan 2006 13:06:03 -0700 Subject: [Python-Dev] When do sets shrink? References: <000201c60e80$c3c35900$e916c797@oemcomputer> Message-ID: Raymond Hettinger wrote: > It might be better to give more generic advice that tends to be true > across implementations and versions: "Dense collections like lists > tuples iterate faster than sparse structures like dicts and sets. > Whenever repeated iteration starts to dominate application run-time, > consider converting to a dense representation for faster iteration and > better memory/cache utilization." A statement like this remains true > whether or not a down-sizing algorithm is present. Thanks. While I certainly wasn't advocating an early optimization approach, I think that part of using a tool effectively is also knowing its dark corners. Sometimes you _do_ need them, so it's handy to have the little 'break the glass in case of an emergency' box :) >> Cheers, >> >> f > > Hmm, your initial may be infringing on another developer's trademarked > signature ;-) Well, tough. It happens to be my name, and I've been signing like this since long before I started using python. I'll think about changing when the lawsuits come knocking, if I can't get the EFF to defend me ;-) Thanks again for your feedback. Until a suitable wiki comes along, I've kept your message in my python-info folder as a potentially useful nugget. Regards, f From martin at v.loewis.de Mon Jan 2 15:14:04 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 02 Jan 2006 15:14:04 +0100 Subject: [Python-Dev] slight inconsistency in svn checkin email subject lines In-Reply-To: <20051230184534.GC30751@ActiveState.com> References: <20051230184534.GC30751@ActiveState.com> Message-ID: <43B9352C.9080800@v.loewis.de> Trent Mick wrote: > Is this intentional? If not, could someone point me to where the svn > trigger scripts are maintained so I could poke around for a fix? (Or > just fix it themselves. :) It was not my intention. They are in dinsdale.python.org:/data/repos/projects Regards, Martin From barry at python.org Tue Jan 3 00:03:04 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 02 Jan 2006 18:03:04 -0500 Subject: [Python-Dev] [Python-checkins] commit of r41880 - python/trunk/Python/Python-ast.c In-Reply-To: <20060102211856.1D1F61E4002@bag.python.org> References: <20060102211856.1D1F61E4002@bag.python.org> Message-ID: <1136242984.2309.109.camel@geddy.wooz.org> I think we have a fundamental problem with Python-ast.c and Python-ast.h. These files should not be both auto-generated and checked into Subversion. The problem is that if you do a "make distclean", these files will get removed and "svn stat" will give you a ! flag. Of course, you can "svn up" to get them back, but that just masks the problem. Worse, if you distclean, then configure, then make, it is possible these files will change. Say you're working on a patch that touches a lot of files and you do an "svn commit". It's certainly possible that you'll accidentally commit new versions of the Python-ast.* files, possibly even broken ones. The general rule should be that no file that is ever generated can be checked into Subversion. Probably the right approach is to check in a template file that will not get removed by a distclean, and modify the build process to generate Python-ast.* from those template files. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060102/4624949e/attachment.pgp From nnorwitz at gmail.com Tue Jan 3 00:16:19 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 2 Jan 2006 15:16:19 -0800 Subject: [Python-Dev] [Python-checkins] commit of r41880 - python/trunk/Python/Python-ast.c In-Reply-To: <1136242984.2309.109.camel@geddy.wooz.org> References: <20060102211856.1D1F61E4002@bag.python.org> <1136242984.2309.109.camel@geddy.wooz.org> Message-ID: On 1/2/06, Barry Warsaw wrote: > I think we have a fundamental problem with Python-ast.c and > Python-ast.h. These files should not be both auto-generated and checked > into Subversion. I agree with the problem statement. > The general rule should be that no file that is ever generated can be > checked into Subversion. Probably the right approach is to check in a > template file that will not get removed by a distclean, and modify the > build process to generate Python-ast.* from those template files. I'm not sure about your proposed solution, though. There's a bootstrapping issue. Python-ast.[ch] are generated by a python 2.2+ script. /f created a bug report if only 2.1 is available. The Python-ast.[ch] should probably not be removed by distclean. This is similar to configure. Would that make you happy? What else would improve the current situation? If you go the template route, you would just copy the files. That doesn't seem to gain anything. n From barry at python.org Tue Jan 3 00:22:50 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 02 Jan 2006 18:22:50 -0500 Subject: [Python-Dev] [Python-checkins] commit of r41880 - python/trunk/Python/Python-ast.c In-Reply-To: References: <20060102211856.1D1F61E4002@bag.python.org> <1136242984.2309.109.camel@geddy.wooz.org> Message-ID: <1136244170.2309.112.camel@geddy.wooz.org> On Mon, 2006-01-02 at 15:16 -0800, Neal Norwitz wrote: > The Python-ast.[ch] should probably not be removed by distclean. This > is similar to configure. Would that make you happy? What else would > improve the current situation? I think it would, without causing bootstrapping issues. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060102/e981d725/attachment.pgp From nnorwitz at gmail.com Tue Jan 3 02:06:36 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 2 Jan 2006 17:06:36 -0800 Subject: [Python-Dev] buildbot Message-ID: Here's some info on setting up buildbot . These instructions should be cleaned up and put in the developers faq. Right now, the only person that can generate/assign buildbot names/passwords is Martin AFAIK. We currently have 4 slaves setup: Linux: x86 and amd64 (gentoo 2.6, glibc 2.3) Solaris10 sparc OS X: G5 (not yet working, need to setup path to svn) It would be nice to get HP-UX, AIX, and Windows in various flavors. Ideally, any common platform we support should be added. This includes O/S, version, H/W, and various configurations. Not sure we care about much more Linux variants. Note the x86 is the same one I use for generating doc and more detailed build results. I'll put that info into a separate mail. We also are only running the slaves on the trunk. Thanks to Martin for setting this up. n -- Instructions: Request a buildbot name and password from python-dev at python.org. Please state the machine type, operating system, compiler, and branch. For example, "amd64 gentoo gcc trunk" or "powerpc64 aix43 xlC 2.4". Download buildbot (we currently use 0.7.1) Install buildbot (and dependencies like Twisted-1.3 if necesary) Create buildslave account [optional] If adding a slave for the trunk: # example directory, choose whatever you like DIR=`pwd`/buildslave/python-trunk mkdir -p $DIR cd $DIR buildbot slave `pwd` www.python.org:9020 "assigned-name" "assigned-password" If adding a slave for the 2.4 branch: # example directory, choose whatever you like DIR=buildslave/python-2.4 mkdir -p $DIR cd $DIR buildbot slave `pwd` www.python.org:9024 "assigned-name" "assigned-password" Edit info/admin and info/host appropriately. You can use this command for info/host if you'd like: uname -a > info/host To start the buildbot: buildbot start `pwd` To ensure buildbot is started on reboot: (crontab -l | egrep -v '^#' ; echo "@reboot /usr/bin/buildbot start $DIR") | crontab - Make sure the buildslave account is able run cron jobs. From nnorwitz at gmail.com Tue Jan 3 02:09:08 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 2 Jan 2006 17:09:08 -0800 Subject: [Python-Dev] current test problems Message-ID: test_bsddb3 fails for me with BSD DB 4.1 and 4.2 on gentoo Linux. I think it has for a long time. Is anyone else seeing these failures? You must use -u bsddb when running regrtest. 4.2 has 2 failures: test02_associateAfterDB (bsddb.test.test_associate.ShelveAssociateRecnoTestCase) test02_associateAfterDB (bsddb.test.test_associate.ThreadedAssociateRecnoTestCase) 4.1 fails like this: Traceback (most recent call last): File "Lib/threading.py", line 444, in __bootstrap self.run() File "Lib/threading.py", line 424, in run self.__target(*self.__args, **self.__kwargs) File "Lib/bsddb/test/test_thread.py", line 275, in readerThread rec = dbutils.DeadlockWrap(c.next, max_retries=10) File "Lib/bsddb/dbutils.py", line 62, in DeadlockWrap return function(*_args, **_kwargs) DBLockDeadlockError: (-30996, 'DB_LOCK_DEADLOCK: Locker killed to resolve a deadlock') When looking for leaks, the following tests fail: test_decimal test_difflib test_optparse test_warnings test_zipimport To test: ./python -E -tt ./Lib/test/regrtest.py -R 3:2: \ test_decimal test_difflib test_optparse test_warnings test_zipimport The following test has actual leaks which was reported a while ago. Armin found the problem to be that itertools.tee needs to participate with GC IIRC: test_generators The following tests do not report stable leaks (at least sometimes): test_capi test_cfgparser test_charmapcodec test_cmd_line test_compiler test_filecmp test_logging test_threadedtempfile test_threading test_threading_local test_urllib2 n From jeremy at alum.mit.edu Tue Jan 3 02:23:20 2006 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Mon, 2 Jan 2006 20:23:20 -0500 Subject: [Python-Dev] [Python-checkins] commit of r41880 - python/trunk/Python/Python-ast.c In-Reply-To: References: <20060102211856.1D1F61E4002@bag.python.org> <1136242984.2309.109.camel@geddy.wooz.org> Message-ID: On 1/2/06, Neal Norwitz wrote: > On 1/2/06, Barry Warsaw wrote: > > I think we have a fundamental problem with Python-ast.c and > > Python-ast.h. These files should not be both auto-generated and checked > > into Subversion. > > I agree with the problem statement. > > > The general rule should be that no file that is ever generated can be > > checked into Subversion. Probably the right approach is to check in a > > template file that will not get removed by a distclean, and modify the > > build process to generate Python-ast.* from those template files. > > I'm not sure about your proposed solution, though. > > There's a bootstrapping issue. Python-ast.[ch] are generated by a > python 2.2+ script. /f created a bug report if only 2.1 is available. > > The Python-ast.[ch] should probably not be removed by distclean. This > is similar to configure. Would that make you happy? What else would > improve the current situation? I think this solution is better. It's relatively rare for people to change the ast definition, so for most purposes these should be static files. Jeremy From nnorwitz at gmail.com Tue Jan 3 02:27:13 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 2 Jan 2006 17:27:13 -0800 Subject: [Python-Dev] automated builds and tests Message-ID: I've updated the script that builds python and the docs, runs the tests, and copies everything up to docs.python.org. Currently, this only works for the trunk. I'm not sure if there's much of a benefit to set this up for branch(es) also. The docs are here: http://docs.python.org/dev/ The results of the build are here: http://docs.python.org/dev/results/ The script is checked in to Misc/build.sh. If the script is modified, it doesn't pick up the changes until the next run. There is a hook to pickup the changes on each run, but I didn't finish that. Look for one of the several FIXMEs. A brief description of what the script does is: svn up ; ./configure ; make ; make test ; make install ; cd Doc ; make ; rsync Below is a description of everything that's done in the script. Most commands are checked for errors and the results are emailed iff there are problems. Right now I'm the only one that gets spammed. This ran all last week without any problems. I made a bunch of changes and probably sometime this week, I'll change the mail to go to python-checkins. svn update svn status ./configure --prefix=$INSTALL_DIR --with-pydebug make | grep warning if make fails, goto building docs make install make test | grep fail ./python ./Lib/test/regrtest.py -R 4:3:$REFLOG -u network | grep leak ./python -E -tt ./Lib/test/regrtest.py -uall -x test_curses test_linuxaudiodev test_ossaudiodev | grep fail cd Doc ; make rsync doc rsync build results --- test_curses is ignored because this script is run from cron and there is no terminal. The test stops prematurely. The sound tests are ignored because the sound device is not configured on this computer. Some of the failure counts are off because it does a simple grep and make test runs everything twice. I've generally tried to filter out stuff we don't care about. There is potential for the tests to fail due to them running at the same time as the buildbot test run. These tests are run every 12 hours. It currently takes over 1.25 hours to build and run everything. There is no make clean step for either the code or doc, perhaps there should be? n From skip at pobox.com Tue Jan 3 04:12:27 2006 From: skip at pobox.com (skip at pobox.com) Date: Mon, 2 Jan 2006 21:12:27 -0600 Subject: [Python-Dev] buildbot In-Reply-To: References: Message-ID: <17337.60315.618431.480677@montanaro.dyndns.org> Neal> We currently have 4 slaves setup: Neal> Linux: x86 and amd64 (gentoo 2.6, glibc 2.3) Neal> Solaris10 sparc Neal> OS X: G5 (not yet working, need to setup path to svn) The G5 *was* working. I changed nothing at my end. Got a mail yesterday from Martin. It looks like PATH lost /usr/local/bin (where the Metissian installer puts the svn executable). I added a /usr/bin/svn symlink, but would really rather see /usr/local/bin in PATH. Skip From nnorwitz at gmail.com Tue Jan 3 07:30:01 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 2 Jan 2006 22:30:01 -0800 Subject: [Python-Dev] buildbot In-Reply-To: <17337.60315.618431.480677@montanaro.dyndns.org> References: <17337.60315.618431.480677@montanaro.dyndns.org> Message-ID: On 1/2/06, skip at pobox.com wrote: > > The G5 *was* working. I changed nothing at my end. Got a mail yesterday > from Martin. It looks like PATH lost /usr/local/bin (where the Metissian > installer puts the svn executable). I added a /usr/bin/svn symlink, but > would really rather see /usr/local/bin in PATH. It seems reasonable for /usr/local/bin to be in PATH. But isn't this be done on the slave (ie your) side? It looks like the PATH comes from your env't based on how my PATH in twistd.log. So if you started this from cron, you may need to add it there. If nothing else works, you could probably update PATH in buildbot.tac. I forced a build, but it failed since svnversion was not found. BTW, I noticed this warning: Modules/getpath.c:405: warning: passing arg 2 of `_NSGetExecutablePath' from incompatible pointer type Can somebody with a Mac fix this warning? n From bob at redivi.com Tue Jan 3 07:39:25 2006 From: bob at redivi.com (Bob Ippolito) Date: Mon, 2 Jan 2006 22:39:25 -0800 Subject: [Python-Dev] buildbot In-Reply-To: References: <17337.60315.618431.480677@montanaro.dyndns.org> Message-ID: <727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com> On Jan 2, 2006, at 10:30 PM, Neal Norwitz wrote: > On 1/2/06, skip at pobox.com wrote: >> >> The G5 *was* working. I changed nothing at my end. Got a mail >> yesterday >> from Martin. It looks like PATH lost /usr/local/bin (where the >> Metissian >> installer puts the svn executable). I added a /usr/bin/svn >> symlink, but >> would really rather see /usr/local/bin in PATH. > > It seems reasonable for /usr/local/bin to be in PATH. But isn't this > be done on the slave (ie your) side? It looks like the PATH comes > from your env't based on how my PATH in twistd.log. So if you started > this from cron, you may need to add it there. If nothing else works, > you could probably update PATH in buildbot.tac. > > I forced a build, but it failed since svnversion was not found. > > BTW, I noticed this warning: > Modules/getpath.c:405: warning: passing arg 2 of > `_NSGetExecutablePath' from incompatible pointer type > > Can somebody with a Mac fix this warning? This warning depends on the version of Mac OS X you're using. The type signature for arg 2 is "unsigned long" up to 10.3, and "uint32_t" on 10.4. The version I see in svn is current with Mac OS X 10.4. The easy fix is to upgrade your OS. I don't think anyone is going to bother with the preprocessor hackery necessary to make that (harmless) warning go away on older versions of the OS. -bob From ianb at colorstudy.com Tue Jan 3 08:08:26 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 03 Jan 2006 01:08:26 -0600 Subject: [Python-Dev] [Doc-SIG] that library reference, again In-Reply-To: <43B58F09.6090803@colorstudy.com> References: <4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com> <43B4A238.2030101@python.org> <43B55FE8.1040401@python.org> <43B58F09.6090803@colorstudy.com> Message-ID: <43BA22EA.5010603@colorstudy.com> I've put an instance of Commentary up against the full 2.4 documentation: http://pythonpaste.org/comment/python24/ It writes to a Subversion repository so you can see what the backend data looks like: http://pythonpaste.org/comment/svn/python24/ -- not much there yet. Obviously things like notification and reports would be useful, but I don't know what the state of the art for Subversion is either, so potentially these tools already exist. But last I really looked at the tools for Subversion, I wasn't overly impressed. Things like self-signup for pages you were interested in would be useful. But anyway, there's nothing too difficult about those features. Anyway, it's just meant as a demo for now, but give it a look, and if you have problems or suggestions write a ticket: http://pythonpaste.org/commentary/trac/newticket Cheers. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From nnorwitz at gmail.com Tue Jan 3 08:58:31 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 2 Jan 2006 23:58:31 -0800 Subject: [Python-Dev] mac memory leaks Message-ID: I had a friend run regrtest -L on Mac OSX a while ago. There are several memory leaks which still appear to be an issue. There are a bunch of leaks reported from putenv which I'm not sure how to fix The attached patch should correct one of the problems. Can someone with a Mac test it? I'll add to SF later if I don't get a response. n -- I'm not sure about how to fix these or if they are real problems: Leak: 0x02541470 size=368 instance of 'NSCFData' Call stack: call_function | ici_ICFindPrefHandle | ICFindPrefHandle | OpaqueICInstance::FindPrefHandle(ICKeyInfo const&, unsigned long*, char**) | OpaqueICInstance::GetPref(ICKeyInfo const&, long, void*, long*, unsigned long*) | OpaqueICInstance::GetPrefCore(ICKeyInfo const&, long, void*, long*, unsigned long*) | OpaqueICInstance::LazyLoad() | ICCFPrefWrapper::ContainsPrefs() | ICCFPrefWrapper::GetPrefDictionary() | fetchXMLValue | _loadXMLDomainIfStale | _CFPropertyListCreateFromXMLData | __CFTryParseBinaryPlist | __CFBinaryPlistCreateObject | __CFBinaryPlistCreateObject | __CFBinaryPlistCreateObject | __CFBinaryPlistCreateObject | __CFBinaryPlistCreateObject | __CFDataInit | _CFRuntimeCreateInstance | CFAllocatorAllocate Leak: 0x02506640 size=256 Call stack: call_function | do_call | PyObject_Call | parser_st2tuple | node2tuple | node2tuple | node2tuple | node2tuple | node2tuple | node2tuple | node2tuple | node2tuple | node2tuple | node2tuple | node2tuple | node2tuple | node2tuple | node2tuple | node2tuple | node2tuple | node2tuple | node2tuple | node2tuple | node2tuple | node2tuple | PyTuple_New | _PyObject_GC_NewVar | _PyObject_GC_Malloc | PyObject_Malloc | new_arena Leak: 0x0118ad10 size=80 Call stack: call_function | AE_AECreateDesc | AECreateDesc | operator new(unsigned long) -------------- next part -------------- A non-text attachment was scrubbed... Name: mac-leak.diff Type: text/x-patch Size: 812 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20060102/49128aa2/attachment.bin From martin at v.loewis.de Tue Jan 3 09:23:23 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 03 Jan 2006 09:23:23 +0100 Subject: [Python-Dev] buildbot In-Reply-To: References: Message-ID: <43BA347B.90002@v.loewis.de> Neal Norwitz wrote: > If adding a slave for the trunk: [...] > If adding a slave for the 2.4 branch: Currently, branches are not supported, because buildbot is somewhat limited. When I get a solution for this problem, I plan to have all buildbots build both the trunk and the 2.4 branch; such builds would only be initiated whenever something is committed on the branch. > To ensure buildbot is started on reboot: > > (crontab -l | egrep -v '^#' ; echo "@reboot /usr/bin/buildbot start > $DIR") | crontab - AFAICT, this assumes Vixie cron, which is fine for Linux, but I don't think it would work on, say, Solaris (atleast I couldn't find anything in the documentation of Solaris cron that suggests "@reboot" lines are supported). Regards, Martin From martin at v.loewis.de Tue Jan 3 09:38:59 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 03 Jan 2006 09:38:59 +0100 Subject: [Python-Dev] [Python-checkins] commit of r41880 - python/trunk/Python/Python-ast.c In-Reply-To: References: <20060102211856.1D1F61E4002@bag.python.org> <1136242984.2309.109.camel@geddy.wooz.org> Message-ID: <43BA3823.9000305@v.loewis.de> Jeremy Hylton wrote: > I think this solution is better. It's relatively rare for people to > change the ast definition, so for most purposes these should be static > files. Interestingly enough, I found yesterday that Python-ast.c did change for me, even though I had not touched the AST grammar at all. Assuming somebody forgot to commit this file, I just did. I now investigated where this change originated from, and it was this commit r41812 | tim.peters | 2005-12-26 00:18:31 +0100 (Mo, 26 Dez 2005) | 2 lines Whitespace normalization. This indicates two problems: apparently, the whitespace normalization also normalizes string literals; not sure whether this is a good idea or not. Furthermore, people should be made aware that something deserves their attention when they find that Python-ast.c has changed for them (but I guess Tim never saw Python-ast.c change, because this build step isn't executed on Windows). Regards, Martin From abo at minkirri.apana.org.au Tue Jan 3 13:13:23 2006 From: abo at minkirri.apana.org.au (Donovan Baarda) Date: Tue, 03 Jan 2006 12:13:23 +0000 Subject: [Python-Dev] [Python-checkins] commit of r41880 - python/trunk/Python/Python-ast.c In-Reply-To: References: <20060102211856.1D1F61E4002@bag.python.org> <1136242984.2309.109.camel@geddy.wooz.org> Message-ID: <1136290403.18340.22.camel@warna.dub.corp.google.com> On Mon, 2006-01-02 at 15:16 -0800, Neal Norwitz wrote: > On 1/2/06, Barry Warsaw wrote: > > I think we have a fundamental problem with Python-ast.c and > > Python-ast.h. These files should not be both auto-generated and checked > > into Subversion. > > I agree with the problem statement. > > > The general rule should be that no file that is ever generated can be > > checked into Subversion. Probably the right approach is to check in a > > template file that will not get removed by a distclean, and modify the > > build process to generate Python-ast.* from those template files. > > I'm not sure about your proposed solution, though. > > There's a bootstrapping issue. Python-ast.[ch] are generated by a > python 2.2+ script. /f created a bug report if only 2.1 is available. > > The Python-ast.[ch] should probably not be removed by distclean. This > is similar to configure. Would that make you happy? What else would > improve the current situation? > > If you go the template route, you would just copy the files. That > doesn't seem to gain anything. The solution I use is to never have anything auto-generated in CVS/SVN, but have "make dist" generate and include anything needed for bootstrapping in the distribution tarball (or whatever). Doing "make distclean" should delete enough to bring you back to a freshly extracted distribution tarball, and "make maintainer-clean" should delete all auto-generated files to bring you back to a clean CVS/SVN checkout. I tend to include quite a few generated files in the distribution tarball that are not in CVS/RCS. Things like ChangeList (generated by cvs2cl), all the autotools autogen'ed files, generated datafiles, etc. This way your source distributions don't have any bootstrap problems, but you also don't have any auto-generated files in CVS/SVN and the problems they create. It does however mean that a fresh CVS/SVN checkout does have additional build requirements above and beyond building from a source distribution. -- Donovan Baarda http://minkirri.apana.org.au/~abo/ From Jack.Jansen at cwi.nl Tue Jan 3 15:55:38 2006 From: Jack.Jansen at cwi.nl (Jack Jansen) Date: Tue, 3 Jan 2006 15:55:38 +0100 Subject: [Python-Dev] mac memory leaks In-Reply-To: References: Message-ID: <357EDE66-9CE4-4E11-99EC-565D7E6B239D@cwi.nl> On 3 Jan 2006, at 08:58, Neal Norwitz wrote: > I had a friend run regrtest -L on Mac OSX a while ago. There are > several memory leaks which still appear to be an issue. There are a > bunch of leaks reported from putenv which I'm not sure how to fix > > The attached patch should correct one of the problems. Can someone > with a Mac test it? I'll add to SF later if I don't get a response. > > n > -- > > I'm not sure about how to fix these or if they are real problems: > > Leak: 0x02541470 size=368 instance of 'NSCFData' > Call stack: call_function | ici_ICFindPrefHandle | ICFindPrefHandle | > OpaqueICInstance::FindPrefHandle(ICKeyInfo const&, unsigned long*, > char**) | OpaqueICInstance::GetPref(ICKeyInfo const&, long, void*, > long*, unsigned long*) | OpaqueICInstance::GetPrefCore(ICKeyInfo > const&, long, void*, long*, unsigned long*) | > OpaqueICInstance::LazyLoad() | ICCFPrefWrapper::ContainsPrefs() | > ICCFPrefWrapper::GetPrefDictionary() | fetchXMLValue | > _loadXMLDomainIfStale | _CFPropertyListCreateFromXMLData | > __CFTryParseBinaryPlist | __CFBinaryPlistCreateObject | > __CFBinaryPlistCreateObject | __CFBinaryPlistCreateObject | > __CFBinaryPlistCreateObject | __CFBinaryPlistCreateObject | > __CFDataInit | _CFRuntimeCreateInstance | CFAllocatorAllocate This is somewhere down in the Internet Config builtin module, which would point to the webbrowser library module. My guess is that it's a singleton leak. > > Leak: 0x02506640 size=256 > Call stack: call_function | do_call | PyObject_Call | parser_st2tuple > | node2tuple | node2tuple | node2tuple | node2tuple | node2tuple | > node2tuple | node2tuple | node2tuple | node2tuple | node2tuple | > node2tuple | node2tuple | node2tuple | node2tuple | node2tuple | > node2tuple | node2tuple | node2tuple | node2tuple | node2tuple | > node2tuple | PyTuple_New | _PyObject_GC_NewVar | _PyObject_GC_Malloc | > PyObject_Malloc | new_arena No idea. There don't seem to be any mac-specific modules involved... > Leak: 0x0118ad10 size=80 > Call stack: call_function | AE_AECreateDesc | AECreateDesc | operator > new(unsigned long) Hmm, the first candidates here would be test_aepack and test_scriptpackages, but neither one has an obvious leak (on cursory inspection). And actually, if there was a leak problem in either I would expect more than one AEDesc to be leaked. -- Jack Jansen, , http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman From skip at pobox.com Tue Jan 3 16:39:03 2006 From: skip at pobox.com (skip at pobox.com) Date: Tue, 3 Jan 2006 09:39:03 -0600 Subject: [Python-Dev] Including zlib... In-Reply-To: <20060103064313.DA3A21E4002@bag.python.org> References: <20060103064313.DA3A21E4002@bag.python.org> Message-ID: <17338.39575.904122.821741@montanaro.dyndns.org> Martin checked in zlib to the Python svn repository. Are we really sure that including zlib is the only path to whatever it is that it achieves? If security holes in zlib turn up (they have in the past), new Python releases will have to be released quickly. Skip From skip at pobox.com Tue Jan 3 16:46:52 2006 From: skip at pobox.com (skip at pobox.com) Date: Tue, 3 Jan 2006 09:46:52 -0600 Subject: [Python-Dev] buildbot In-Reply-To: <727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com> References: <17337.60315.618431.480677@montanaro.dyndns.org> <727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com> Message-ID: <17338.40044.896652.583651@montanaro.dyndns.org> Bob> The easy fix is to upgrade your OS. I don't think anyone is going Bob> to bother with the preprocessor hackery necessary to make that Bob> (harmless) warning go away on older versions of the OS. Excuse me, but this really pisses me off. I delayed upgrading from 10.2 for awhile and was given the same "advice". I was further told (check the mac sig archives) that "you'd be able to stick with 10.3 for much longer" because new apps wouldn't need to require 10.4. So I upgraded. Now you're telling me that it's somehow "obsolete" and that I should upgrade because "we can't be bothered to fix a compiler warning"? Python supports much older versions of other platforms. What makes Mac OSX so special in this regard? Skip From theller at python.net Tue Jan 3 16:58:37 2006 From: theller at python.net (Thomas Heller) Date: Tue, 03 Jan 2006 16:58:37 +0100 Subject: [Python-Dev] API changes Message-ID: <4q4l9uc2.fsf@python.net> The ast-branch merge apparently changed some formerly public functions to macros. The two that I found out about are PyRun_SimpleString and PyRun_InteractiveLoop, I have not checked if there are more or not. This breaks apps which dynamically link at runtime to the Python dll (the latest py2exe does this). Was this change intentional, or can it be reverted? Thanks, Thomas From ronaldoussoren at mac.com Tue Jan 3 17:04:05 2006 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Tue, 3 Jan 2006 17:04:05 +0100 Subject: [Python-Dev] buildbot In-Reply-To: <17338.40044.896652.583651@montanaro.dyndns.org> References: <17337.60315.618431.480677@montanaro.dyndns.org> <727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com> <17338.40044.896652.583651@montanaro.dyndns.org> Message-ID: On 3-jan-2006, at 16:46, skip at pobox.com wrote: > > Bob> The easy fix is to upgrade your OS. I don't think anyone > is going > Bob> to bother with the preprocessor hackery necessary to make > that > Bob> (harmless) warning go away on older versions of the OS. > > Excuse me, but this really pisses me off. I delayed upgrading from > 10.2 for > awhile and was given the same "advice". I was further told (check > the mac > sig archives) that "you'd be able to stick with 10.3 for much longer" > because new apps wouldn't need to require 10.4. So I upgraded. > Now you're > telling me that it's somehow "obsolete" and that I should upgrade > because > "we can't be bothered to fix a compiler warning"? Python supports > much > older versions of other platforms. What makes Mac OSX so special > in this > regard? This is a completely harmless warning, but I agree it is annoying that older versions of the OS are neglected. At least for me, supporting older versions of OSX is hard. Both my system at work and my private system run 10.4 by default, which means I'll have to reboot and leave my normal work environment to work on older versions. Thanks to the age of my machines "older versions" is limited to 10.3.0 or later, neither machine can boot 10.2. Ronald > > Skip > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ > ronaldoussoren%40mac.com From jeremy at alum.mit.edu Tue Jan 3 17:18:56 2006 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Tue, 3 Jan 2006 11:18:56 -0500 Subject: [Python-Dev] API changes In-Reply-To: <4q4l9uc2.fsf@python.net> References: <4q4l9uc2.fsf@python.net> Message-ID: The intent was to provide binary compatibility, but redirect all newly linked code to the newer variants. We did this correctly for PyParser_SimpleParseFile and PyParser_SimpleParseString, but didn't do it for the rest of the changed functions. Can you file a bug report? (Or just fix the ones that bother you.) We ought to update the docs, too. Jeremy On 1/3/06, Thomas Heller wrote: > The ast-branch merge apparently changed some formerly public functions > to macros. The two that I found out about are PyRun_SimpleString and > PyRun_InteractiveLoop, I have not checked if there are more or not. > > This breaks apps which dynamically link at runtime to the Python dll > (the latest py2exe does this). > > Was this change intentional, or can it be reverted? > > Thanks, > > Thomas > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu > From stephen at xemacs.org Tue Jan 3 17:27:43 2006 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 04 Jan 2006 01:27:43 +0900 Subject: [Python-Dev] buildbot In-Reply-To: <17338.40044.896652.583651@montanaro.dyndns.org> (skip@pobox.com's message of "Tue, 3 Jan 2006 09:46:52 -0600") References: <17337.60315.618431.480677@montanaro.dyndns.org> <727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com> <17338.40044.896652.583651@montanaro.dyndns.org> Message-ID: <871wzpmg3k.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "skip" == skip writes: Bob> The easy fix is to upgrade your OS. I don't think anyone is going Bob> to bother with the preprocessor hackery necessary to make that Bob> (harmless) warning go away on older versions of the OS. skip> Excuse me, but this really pisses me off. Oh, c'mon, Skip. I'm a Panther hold-out, too, but this is just a (soi-disant harmless) warning. Have you tried to build anything from DarwinPorts recently? I have at least 20 modified PortFiles plus at least two patched source files because they can't be bothered to make sure things _build_ on Panther, let alone eliminate warnings. Sure, Python can be expected to do much better than random port maintainers, but.... Discussions with the port maintainers makes it hard to blame them. They say the development environment has changed enormously---you really need multiple systems because of significant changes in the standard headers, it's nowhere near enough to just keep a couple or six versions of GCC around. And you need multiple dev environments, because Xcode 1.5 doesn't cut it on Tiger, and Xcode 2.x doesn't work at all on Panther. Yes, it would be nice to make the warnings go away, but in view of all that, is it something to get one's shorts in a knot about? -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From theller at python.net Tue Jan 3 17:44:47 2006 From: theller at python.net (Thomas Heller) Date: Tue, 03 Jan 2006 17:44:47 +0100 Subject: [Python-Dev] API changes References: <4q4l9uc2.fsf@python.net> Message-ID: Jeremy Hylton writes: > On 1/3/06, Thomas Heller wrote: >> The ast-branch merge apparently changed some formerly public functions >> to macros. The two that I found out about are PyRun_SimpleString and >> PyRun_InteractiveLoop, I have not checked if there are more or not. >> >> This breaks apps which dynamically link at runtime to the Python dll >> (the latest py2exe does this). >> >> Was this change intentional, or can it be reverted? >> >> Thanks, >> >> Thomas >> > The intent was to provide binary compatibility, but redirect all newly > linked code to the newer variants. We did this correctly for > PyParser_SimpleParseFile and PyParser_SimpleParseString, but didn't do > it for the rest of the changed functions. Can you file a bug report? > (Or just fix the ones that bother you.) > > We ought to update the docs, too. I'm not sure I understand the 'we did this correctly for ...'. PyParser_SimpleParseFile is a function in 2.4, and a macro in 2.5. Thomas From jeremy at alum.mit.edu Tue Jan 3 18:40:14 2006 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Tue, 3 Jan 2006 12:40:14 -0500 Subject: [Python-Dev] API changes In-Reply-To: References: <4q4l9uc2.fsf@python.net> Message-ID: On 1/3/06, Thomas Heller wrote: > Jeremy Hylton writes: > > On 1/3/06, Thomas Heller wrote: > >> The ast-branch merge apparently changed some formerly public functions > >> to macros. The two that I found out about are PyRun_SimpleString and > >> PyRun_InteractiveLoop, I have not checked if there are more or not. > >> > >> This breaks apps which dynamically link at runtime to the Python dll > >> (the latest py2exe does this). > >> > >> Was this change intentional, or can it be reverted? > >> > >> Thanks, > >> > >> Thomas > >> > > > The intent was to provide binary compatibility, but redirect all newly > > linked code to the newer variants. We did this correctly for > > PyParser_SimpleParseFile and PyParser_SimpleParseString, but didn't do > > it for the rest of the changed functions. Can you file a bug report? > > (Or just fix the ones that bother you.) > > > > We ought to update the docs, too. > > I'm not sure I understand the 'we did this correctly for ...'. > PyParser_SimpleParseFile is a function in 2.4, and a macro in 2.5. There's a function by the same name at the end of pythonrun.c. Nothing in the Python core should link to it, but it's still available for third-party code. Now we can mark PyParser_SimpleParseFile as deprecated remove it in some future release. Jeremy From theller at python.net Tue Jan 3 19:18:21 2006 From: theller at python.net (Thomas Heller) Date: Tue, 03 Jan 2006 19:18:21 +0100 Subject: [Python-Dev] API changes References: <4q4l9uc2.fsf@python.net> Message-ID: Jeremy Hylton writes: > On 1/3/06, Thomas Heller wrote: >> Jeremy Hylton writes: >> > On 1/3/06, Thomas Heller wrote: >> >> The ast-branch merge apparently changed some formerly public functions >> >> to macros. The two that I found out about are PyRun_SimpleString and >> >> PyRun_InteractiveLoop, I have not checked if there are more or not. >> >> >> >> This breaks apps which dynamically link at runtime to the Python dll >> >> (the latest py2exe does this). >> >> > The intent was to provide binary compatibility, but redirect all newly >> > linked code to the newer variants. We did this correctly for >> > PyParser_SimpleParseFile and PyParser_SimpleParseString, but didn't do >> > it for the rest of the changed functions. Can you file a bug report? >> > (Or just fix the ones that bother you.) >> >> I'm not sure I understand the 'we did this correctly for ...'. >> PyParser_SimpleParseFile is a function in 2.4, and a macro in 2.5. > > There's a function by the same name at the end of pythonrun.c. > Nothing in the Python core should link to it, but it's still available > for third-party code. Now we can mark PyParser_SimpleParseFile as > deprecated remove it in some future release. I see. I doesn't help on Windows, though, since the functions are not declared DL_EXPORT(). All in all, I'm unsure if it is worth fixing this - maybe it would be better to change py2exe. Thomas From tim.peters at gmail.com Tue Jan 3 19:38:03 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 3 Jan 2006 13:38:03 -0500 Subject: [Python-Dev] [Python-checkins] commit of r41906 - python/branches/ssize_t/Objects/obmalloc.c In-Reply-To: <20060103131655.201471E400A@bag.python.org> References: <20060103131655.201471E400A@bag.python.org> Message-ID: <1f7befae0601031038w6f6c6f03o4549e039dafd19a8@mail.gmail.com> > Author: martin.v.loewis > Date: Tue Jan 3 14:16:53 2006 > New Revision: 41906 > > Modified: > python/branches/ssize_t/Objects/obmalloc.c > Log: > Disable 32-bit size limitation for 64-bit mode. > > > Modified: python/branches/ssize_t/Objects/obmalloc.c > ============================================================================== > --- python/branches/ssize_t/Objects/obmalloc.c (original) > +++ python/branches/ssize_t/Objects/obmalloc.c Tue Jan 3 14:16:53 2006 > @@ -1005,6 +1005,8 @@ > > bumpserialno(); > total = nbytes + 16; > +#if SIZEOF_SIZE_T < 8 > + /* XXX do this check only on 32-bit machines */ > if (total < nbytes || (total >> 31) > 1) { > /* overflow, or we can't represent it in 4 bytes */ > /* Obscure: can't do (total >> 32) != 0 instead, because > @@ -1013,6 +1015,7 @@ > size_t is an unsigned type. */ > return NULL; > } > +#endif This checkin should be reverted for now. It's in _PyObject_DebugMalloc, and at present the layout of the extra debug info in a PYMALLOC_DEBUG build is hard-coded to use 4-byte fields, no matter what sizeof(size_t) may be. One of the extra fields recorded in a PYMALLOC_DEBUG build is the number of bytes requested, and at present it's simply not capable of recording a value that doesn't fit in 4 bytes. Even after (if ever ;-)) this is changed to support recording 8-byte values on a box where sizeof(size_t) == 8, the "total < nbytes" part of the test would still be appropriate: PyObject_DebugMalloc requests more memory (`total`) than the user asked for (`nbytes`), and the computation of `total` may have overflowed. That's what "total < nbytes" is checking, and that's the right way to spell the overflow check on any box. From tim.peters at gmail.com Tue Jan 3 19:58:52 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 3 Jan 2006 13:58:52 -0500 Subject: [Python-Dev] [Python-checkins] commit of r41880 - python/trunk/Python/Python-ast.c In-Reply-To: <43BA3823.9000305@v.loewis.de> References: <20060102211856.1D1F61E4002@bag.python.org> <1136242984.2309.109.camel@geddy.wooz.org> <43BA3823.9000305@v.loewis.de> Message-ID: <1f7befae0601031058h6f5b1615tbea25bf7acc4835c@mail.gmail.com> [Jeremy Hylton] >> I think this solution is better. It's relatively rare for people to >> change the ast definition, so for most purposes these should be static >> files. [Martin v. L?wis] > Interestingly enough, I found yesterday that Python-ast.c did change for > me, even though I had not touched the AST grammar at all. Assuming > somebody forgot to commit this file, I just did. > > I now investigated where this change originated from, and it was this > commit > > r41812 | tim.peters | 2005-12-26 00:18:31 +0100 (Mo, 26 Dez 2005) | 2 lines > > Whitespace normalization. > > This indicates two problems: apparently, the whitespace normalization > also normalizes string literals; not sure whether this is a good idea > or not. reindent.py never leaves a hard tab character behind, or a line with trailing whitespace. Those are features, because they're invisible, and _someone's_ editor is going to destroy either or both of those sooner or later anyway. No core code should rely on them (inside string literals or not). I don't believe that asdl_c.py requires either hard tab characters or trailing whitespace on lines (both of which reindent.py replaced/removed), and reindent.py's output is a fixed-point for reindent.py, so those shouldn't happen again (unless/until someone sticks hard tabs or trailing whitespace into asdl_c.py again). If someone is really keen to have hard tab characters inside the CODE strings written by this file, then "the right" way to do that is to use \t escapes inside the triple-quoted strings (instead of using embedded hard tab characters). > Furthermore, people should be made aware that something > deserves their attention when they find that Python-ast.c has changed > for them (but I guess Tim never saw Python-ast.c change, because this > build step isn't executed on Windows). I can do better than guess about that ;-) From bob at redivi.com Tue Jan 3 20:49:09 2006 From: bob at redivi.com (Bob Ippolito) Date: Tue, 3 Jan 2006 11:49:09 -0800 Subject: [Python-Dev] buildbot In-Reply-To: <17338.40044.896652.583651@montanaro.dyndns.org> References: <17337.60315.618431.480677@montanaro.dyndns.org> <727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com> <17338.40044.896652.583651@montanaro.dyndns.org> Message-ID: <1F024C8F-E68E-40F7-991A-AE25B6616CFF@redivi.com> On Jan 3, 2006, at 7:46 AM, skip at pobox.com wrote: > Bob> The easy fix is to upgrade your OS. I don't think anyone > is going > Bob> to bother with the preprocessor hackery necessary to make > that > Bob> (harmless) warning go away on older versions of the OS. > > Excuse me, but this really pisses me off. I delayed upgrading from > 10.2 for > awhile and was given the same "advice". I was further told (check > the mac > sig archives) that "you'd be able to stick with 10.3 for much longer" > because new apps wouldn't need to require 10.4. So I upgraded. > Now you're > telling me that it's somehow "obsolete" and that I should upgrade > because > "we can't be bothered to fix a compiler warning"? Python supports > much > older versions of other platforms. What makes Mac OSX so special > in this > regard? The compiler warning is harmless. It's probably about 10 lines of code and an hour of testing to fix it, then you'd need to backport to 2.4, etc. Also, in order to do it properly you need to have access to older versions of the OS in order to see the warning and test to make sure you didn't break anything. Who's going to bother? It's just not practical to fix something that isn't broken. The thing that makes Mac OS X "special" is that there don't seem to be Python developers who are both knowledgeable and willing to make sure everything works on all versions of the OS. Since you seem to care, why don't you take up the task? -bob From gmccaughan at synaptics-uk.com Tue Jan 3 19:43:10 2006 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Tue, 3 Jan 2006 18:43:10 +0000 Subject: [Python-Dev] a quit that actually quits In-Reply-To: References: <20051229051008.GA2459@panix.com> Message-ID: <200601031843.11183.gmccaughan@synaptics-uk.com> > class _Quitter(str): > def __call__(self): raise SystemExit > quit = _Quitter('The quit command. Type "quit()" to exit') > exit = _Quitter('The exit command. Type "exit()" to exit') I think you meant class _Quitter(str): def __call__(self): raise SystemExit quit = _Quitter('The quit command. Type "quit()" to quit') exit = _Quitter('The exit command. Type "exit()" to exit') :-) -- g From trentm at ActiveState.com Tue Jan 3 21:02:29 2006 From: trentm at ActiveState.com (Trent Mick) Date: Tue, 3 Jan 2006 12:02:29 -0800 Subject: [Python-Dev] slight inconsistency in svn checkin email subject lines In-Reply-To: <43B9352C.9080800@v.loewis.de> References: <20051230184534.GC30751@ActiveState.com> <43B9352C.9080800@v.loewis.de> Message-ID: <20060103200229.GC31965@activestate.com> [Martin v. Loewis wrote] > Trent Mick wrote: > > Is this intentional? If not, could someone point me to where the svn > > trigger scripts are maintained so I could poke around for a fix? (Or > > just fix it themselves. :) > > It was not my intention. They are in > dinsdale.python.org:/data/repos/projects I don't have access to that box. Is this a machine used for maintaining the pydotorg stuff? Do you want me in that "club"? Or would it be easier for you the fix this little inconsistency... or have me make a bug on SF and assign to you (or someone else)? Thanks, Trent -- Trent Mick trentm at activestate.com From guido at python.org Tue Jan 3 21:35:03 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 3 Jan 2006 12:35:03 -0800 Subject: [Python-Dev] Including zlib... In-Reply-To: <17338.39575.904122.821741@montanaro.dyndns.org> References: <20060103064313.DA3A21E4002@bag.python.org> <17338.39575.904122.821741@montanaro.dyndns.org> Message-ID: Hear, hear. On 1/3/06, skip at pobox.com wrote: > > Martin checked in zlib to the Python svn repository. Are we really sure > that including zlib is the only path to whatever it is that it achieves? If > security holes in zlib turn up (they have in the past), new Python releases > will have to be released quickly. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From theller at python.net Tue Jan 3 21:36:39 2006 From: theller at python.net (Thomas Heller) Date: Tue, 03 Jan 2006 21:36:39 +0100 Subject: [Python-Dev] buildbot References: <17337.60315.618431.480677@montanaro.dyndns.org> <727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com> <17338.40044.896652.583651@montanaro.dyndns.org> Message-ID: <7j9h82w8.fsf@python.net> skip at pobox.com writes: > I delayed upgrading from 10.2 for awhile and was given the same > "advice". I was further told (check the mac sig archives) that "you'd > be able to stick with 10.3 for much longer" because new apps wouldn't > need to require 10.4. So I upgraded. Now you're telling me that it's > somehow "obsolete" and that I should upgrade because "we can't be > bothered to fix a compiler warning"? Python supports much older > versions of other platforms. What makes Mac OSX so special in this > regard? There are 10.1 and 10.2 boxes in the sourceforge compile farm which could probably used for testing. Thomas From tim.peters at gmail.com Tue Jan 3 21:58:10 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 3 Jan 2006 15:58:10 -0500 Subject: [Python-Dev] Including zlib... In-Reply-To: References: <20060103064313.DA3A21E4002@bag.python.org> <17338.39575.904122.821741@montanaro.dyndns.org> Message-ID: <1f7befae0601031258t20401cc6q2c4e676dedc09d2d@mail.gmail.com> [Guido] > Hear, hear. [Skip] >> Martin checked in zlib to the Python svn repository. Are we really sure >> that including zlib is the only path to whatever it is that it achieves? If >> security holes in zlib turn up (they have in the past), new Python releases >> will have to be released quickly. Martin & Thomas Heller proposed it here about two weeks ago: http://mail.python.org/pipermail/python-dev/2005-December/058873.html It's pimarily for the benefit of py2exe on Windows. AFAICT, he didn't change Python's treatment of zlib on non-Windows boxes; on Windows zlib is now compiled in to the core DLL. From guido at python.org Tue Jan 3 22:13:00 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 3 Jan 2006 13:13:00 -0800 Subject: [Python-Dev] Including zlib... In-Reply-To: <1f7befae0601031258t20401cc6q2c4e676dedc09d2d@mail.gmail.com> References: <20060103064313.DA3A21E4002@bag.python.org> <17338.39575.904122.821741@montanaro.dyndns.org> <1f7befae0601031258t20401cc6q2c4e676dedc09d2d@mail.gmail.com> Message-ID: OK. As long a typical Unix build still links with whatever shared zlib is present on the box I'm fine with this. --Guido On 1/3/06, Tim Peters wrote: > [Guido] > > Hear, hear. > > [Skip] > >> Martin checked in zlib to the Python svn repository. Are we really sure > >> that including zlib is the only path to whatever it is that it achieves? If > >> security holes in zlib turn up (they have in the past), new Python releases > >> will have to be released quickly. > > Martin & Thomas Heller proposed it here about two weeks ago: > > http://mail.python.org/pipermail/python-dev/2005-December/058873.html > > It's pimarily for the benefit of py2exe on Windows. AFAICT, he didn't > change Python's treatment of zlib on non-Windows boxes; on Windows > zlib is now compiled in to the core DLL. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.peters at gmail.com Tue Jan 3 22:22:09 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 3 Jan 2006 16:22:09 -0500 Subject: [Python-Dev] Including zlib... In-Reply-To: References: <20060103064313.DA3A21E4002@bag.python.org> <17338.39575.904122.821741@montanaro.dyndns.org> <1f7befae0601031258t20401cc6q2c4e676dedc09d2d@mail.gmail.com> Message-ID: <1f7befae0601031322u2b3ac77ej195141b1e112d559@mail.gmail.com> [Guido] > OK. As long a typical Unix build still links with whatever shared zlib > is present on the box I'm fine with this. [Guido] >>> Hear, hear. [Skip] >>>> Martin checked in zlib to the Python svn repository. Are we really sure >>>> that including zlib is the only path to whatever it is that it achieves? If >>>> security holes in zlib turn up (they have in the past), new Python releases >>>> will have to be released quickly. [Tim] >> Martin & Thomas Heller proposed it here about two weeks ago: >> >> http://mail.python.org/pipermail/python-dev/2005-December/058873.html >> >> It's pimarily for the benefit of py2exe on Windows. AFAICT, he didn't >> change Python's treatment of zlib on non-Windows boxes; on Windows >> zlib is now compiled in to the core DLL. I figure that if we keep this conversation going long enough, we can eventually shame Guido out of top-posting before it becomes a habit ;-). From nnorwitz at gmail.com Tue Jan 3 22:27:11 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 3 Jan 2006 13:27:11 -0800 Subject: [Python-Dev] Including zlib... In-Reply-To: <1f7befae0601031322u2b3ac77ej195141b1e112d559@mail.gmail.com> References: <20060103064313.DA3A21E4002@bag.python.org> <17338.39575.904122.821741@montanaro.dyndns.org> <1f7befae0601031258t20401cc6q2c4e676dedc09d2d@mail.gmail.com> <1f7befae0601031322u2b3ac77ej195141b1e112d559@mail.gmail.com> Message-ID: Hear, hear. Er.... On 1/3/06, Tim Peters wrote: > [Guido] > > OK. As long a typical Unix build still links with whatever shared zlib > > is present on the box I'm fine with this. > > [Guido] > >>> Hear, hear. > > [Skip] > ... > > [Tim] > ... > I figure that if we keep this conversation going long enough, we can > eventually shame Guido out of top-posting before it becomes a habit > ;-). Here, here! Google has already taught him bad habits. :-) n From martin at v.loewis.de Tue Jan 3 22:59:42 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 03 Jan 2006 22:59:42 +0100 Subject: [Python-Dev] [Python-checkins] commit of r41906 - python/branches/ssize_t/Objects/obmalloc.c In-Reply-To: <1f7befae0601031038w6f6c6f03o4549e039dafd19a8@mail.gmail.com> References: <20060103131655.201471E400A@bag.python.org> <1f7befae0601031038w6f6c6f03o4549e039dafd19a8@mail.gmail.com> Message-ID: <43BAF3CE.7040506@v.loewis.de> Tim Peters wrote: >>Modified: python/branches/ssize_t/Objects/obmalloc.c [...] > This checkin should be reverted for now. Not sure whether you've noticed this is "just" on the ssize_t branch. Without this patch, it is not possible to allocate 4GiB or more for a string object in debug mode, which kind of defeated my attempts to test that. I certainly plan to remove all XXX marks I have introduced in that branch before suggesting to integrate it back into the trunk. So "for now", I would prefer to keep it, and only revert it if I have a complete fix. > It's in > _PyObject_DebugMalloc, and at present the layout of the extra debug > info in a PYMALLOC_DEBUG build is hard-coded to use 4-byte fields, no > matter what sizeof(size_t) may be. One of the extra fields recorded > in a PYMALLOC_DEBUG build is the number of bytes requested, and at > present it's simply not capable of recording a value that doesn't fit > in 4 bytes. Well, AFAICT, it "works" even if it records only records the lower 4 bytes of the requested size. Upon freeing, it just won't put enough DEADBYTEs in, which I cannot see having further unfortunate consequences (except that it won't diagnose errors as good anymore as it could). > Even after (if ever ;-)) this is changed to support recording 8-byte > values on a box where sizeof(size_t) == 8, the "total < nbytes" part > of the test would still be appropriate: PyObject_DebugMalloc requests > more memory (`total`) than the user asked for (`nbytes`), and the > computation of `total` may have overflowed. That's what "total < > nbytes" is checking, and that's the right way to spell the overflow > check on any box. Certainly; I did not mean to completely disable this test. Regards, Martin From tim.peters at gmail.com Tue Jan 3 23:13:21 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 3 Jan 2006 17:13:21 -0500 Subject: [Python-Dev] [Python-checkins] commit of r41906 - python/branches/ssize_t/Objects/obmalloc.c In-Reply-To: <43BAF3CE.7040506@v.loewis.de> References: <20060103131655.201471E400A@bag.python.org> <1f7befae0601031038w6f6c6f03o4549e039dafd19a8@mail.gmail.com> <43BAF3CE.7040506@v.loewis.de> Message-ID: <1f7befae0601031413hc93e132gb64022f7e8eb727b@mail.gmail.com> [Tim] > >>Modified: python/branches/ssize_t/Objects/obmalloc.c > [...] >> This checkin should be reverted for now. [Martin] > Not sure whether you've noticed this is "just" on the ssize_t branch. Right, I noticed. > Without this patch, it is not possible to allocate 4GiB or more for > a string object in debug mode, which kind of defeated my attempts to > test that. > > I certainly plan to remove all XXX marks I have introduced in that > branch before suggesting to integrate it back into the trunk. > > So "for now", I would prefer to keep it, and only revert it if > I have a complete fix. If it's expedient, that's fine for now. I'd like to help fix it "for real", but time is a problem for now. In a week or two I may have more to give to this. Would probably also like to change the dict object's ma_{fill,used,mask} members to Py_ssize_t (while other people don't seem to do this much, I routinely write first-stab programs where a single dict blows more than 2GB of VM ;-)). > ... From martin at v.loewis.de Tue Jan 3 23:14:42 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 03 Jan 2006 23:14:42 +0100 Subject: [Python-Dev] slight inconsistency in svn checkin email subject lines In-Reply-To: <20060103200229.GC31965@activestate.com> References: <20051230184534.GC30751@ActiveState.com> <43B9352C.9080800@v.loewis.de> <20060103200229.GC31965@activestate.com> Message-ID: <43BAF752.4040504@v.loewis.de> Trent Mick wrote: > I don't have access to that box. Is this a machine used for maintaining > the pydotorg stuff? Yes, it's also www.python.org (but not mail.python.org) > Or would it be easier > for you the fix this little inconsistency... or have me make a bug on SF > and assign to you (or someone else)? I tried to, but couldn't. I'll be sending the mailer.py and mailer.conf files in a private mail. I can't see anything wrong in that code. Perhaps mailman is adding the spaces? Regards, Martin From nnorwitz at gmail.com Tue Jan 3 23:25:22 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 3 Jan 2006 14:25:22 -0800 Subject: [Python-Dev] slight inconsistency in svn checkin email subject lines In-Reply-To: <43BAF752.4040504@v.loewis.de> References: <20051230184534.GC30751@ActiveState.com> <43B9352C.9080800@v.loewis.de> <20060103200229.GC31965@activestate.com> <43BAF752.4040504@v.loewis.de> Message-ID: On 1/3/06, "Martin v. L?wis" wrote: > > I tried to, but couldn't. I'll be sending the mailer.py and mailer.conf > files in a private mail. I can't see anything wrong in that code. > > Perhaps mailman is adding the spaces? I looked into this briefly. I saw something like: if prefix: subject = prefix + ' ' + self.subject else: subject = self.subject My guess is that there needs to be stripping for prefix and self.subject. I don't know the code well, but that's all I could come up with. I didn't see any extra spaces in mailer.conf IIRC. n From martin at v.loewis.de Tue Jan 3 23:45:03 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 03 Jan 2006 23:45:03 +0100 Subject: [Python-Dev] [Python-checkins] commit of r41906 - python/branches/ssize_t/Objects/obmalloc.c In-Reply-To: <1f7befae0601031413hc93e132gb64022f7e8eb727b@mail.gmail.com> References: <20060103131655.201471E400A@bag.python.org> <1f7befae0601031038w6f6c6f03o4549e039dafd19a8@mail.gmail.com> <43BAF3CE.7040506@v.loewis.de> <1f7befae0601031413hc93e132gb64022f7e8eb727b@mail.gmail.com> Message-ID: <43BAFE6F.5010305@v.loewis.de> Tim Peters wrote: > If it's expedient, that's fine for now. I'd like to help fix it "for > real", but time is a problem for now. In a week or two I may have > more to give to this. Would probably also like to change the dict > object's ma_{fill,used,mask} members to Py_ssize_t (while other people > don't seem to do this much, I routinely write first-stab programs > where a single dict blows more than 2GB of VM ;-)). Yes, I just noticed I mostly ignored dicts so far; that should also be part of the final patch. Regards, Martin From martin at v.loewis.de Wed Jan 4 00:01:30 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 04 Jan 2006 00:01:30 +0100 Subject: [Python-Dev] slight inconsistency in svn checkin email subject lines In-Reply-To: References: <20051230184534.GC30751@ActiveState.com> <43B9352C.9080800@v.loewis.de> <20060103200229.GC31965@activestate.com> <43BAF752.4040504@v.loewis.de> Message-ID: <43BB024A.6030109@v.loewis.de> Neal Norwitz wrote: > I looked into this briefly. I saw something like: > > if prefix: > subject = prefix + ' ' + self.subject > else: > subject = self.subject Hmm - this shouldn't matter: There is only a single prefix defined, in [defaults]: commit_subject_prefix = [Python-checkins] commit of There is no other occurrence of that string in any other spelling elsewhere, and neither of the "commit of" part. So if prefix was empty, the entire "commit of" should be absent. Anyway, I just killed the "[Python-checkins]" part from the prefix (expecting that mailman would add it, anyway), and regenerated the r41848 commit message, and it seem that worked this time. So I still suspect this to be a mailman bug. There is just too much Python code involved in delivering Python commit messages :-) Regards, Martin From martin at v.loewis.de Wed Jan 4 00:11:07 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 04 Jan 2006 00:11:07 +0100 Subject: [Python-Dev] buildbot In-Reply-To: <871wzpmg3k.fsf@tleepslib.sk.tsukuba.ac.jp> References: <17337.60315.618431.480677@montanaro.dyndns.org> <727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com> <17338.40044.896652.583651@montanaro.dyndns.org> <871wzpmg3k.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <43BB048B.7080909@v.loewis.de> Stephen J. Turnbull wrote: > Sure, Python can be expected to do much better than random port > maintainers, but.... Discussions with the port maintainers makes it > hard to blame them. Well, it's the same attitude that Fredrik Lundh just critized. It's ok for a port maintainer to say "I don't have any energy to fix it myself" (although I would hope he still says "patches will be accepted"). It's not ok for a port maintainer to say "it's not a bug - just upgrade your system". It *is* a bug for Python to emit warnings on "major platforms" (PEP 7). Also, it would be good if PEP 11 was considered before breaking platforms. If some version of the system is not supported anymore, fine - but *atleast* that should be mentioned in PEP 11, and better, the process suggested in PEP 11 should be followed. Regards, Martin From martin at v.loewis.de Wed Jan 4 00:12:22 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 04 Jan 2006 00:12:22 +0100 Subject: [Python-Dev] buildbot In-Reply-To: <1F024C8F-E68E-40F7-991A-AE25B6616CFF@redivi.com> References: <17337.60315.618431.480677@montanaro.dyndns.org> <727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com> <17338.40044.896652.583651@montanaro.dyndns.org> <1F024C8F-E68E-40F7-991A-AE25B6616CFF@redivi.com> Message-ID: <43BB04D6.5080302@v.loewis.de> Bob Ippolito wrote: > Who's going to bother? It violates PEP 7, unless you argue that OS X/gcc is not a "major compiler". Regards, Martin From martin at v.loewis.de Wed Jan 4 00:17:50 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 04 Jan 2006 00:17:50 +0100 Subject: [Python-Dev] Including zlib... In-Reply-To: <17338.39575.904122.821741@montanaro.dyndns.org> References: <20060103064313.DA3A21E4002@bag.python.org> <17338.39575.904122.821741@montanaro.dyndns.org> Message-ID: <43BB061E.5020401@v.loewis.de> skip at pobox.com wrote: > Martin checked in zlib to the Python svn repository. Are we really sure > that including zlib is the only path to whatever it is that it achieves? If > security holes in zlib turn up (they have in the past), new Python releases > will have to be released quickly. As Tim explains: this change doesn't change anything wrt. having to rerelease. On Unix, nothing changes because the included zlib isn't used (but, personally, I would accept contributions of a patch to use it if there is no platform zlib). On Windows, nothing changes really because now you have to update python24.dll, instead of updating zlib.pyd - either requires a recompilation of the Python sources. Regards, Martin From bob at redivi.com Wed Jan 4 00:51:39 2006 From: bob at redivi.com (Bob Ippolito) Date: Tue, 3 Jan 2006 15:51:39 -0800 Subject: [Python-Dev] buildbot In-Reply-To: <43BB04D6.5080302@v.loewis.de> References: <17337.60315.618431.480677@montanaro.dyndns.org> <727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com> <17338.40044.896652.583651@montanaro.dyndns.org> <1F024C8F-E68E-40F7-991A-AE25B6616CFF@redivi.com> <43BB04D6.5080302@v.loewis.de> Message-ID: <3E7B7F37-D4F0-4835-9323-89BFA100149B@redivi.com> On Jan 3, 2006, at 3:12 PM, Martin v. L?wis wrote: > Bob Ippolito wrote: >> Who's going to bother? > > It violates PEP 7, unless you argue that OS X/gcc is not > a "major compiler". Clearly, but that still doesn't answer the question of who's going to do it. Writing two code paths with availability macros and all the testing involved is a whole lot of work to fix a harmless warning on older versions of the OS. If it actually caused a problem I'm sure someone would go through the trouble, but in this case it's probably not worth the effort. -bob From martin at v.loewis.de Wed Jan 4 01:00:09 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 04 Jan 2006 01:00:09 +0100 Subject: [Python-Dev] buildbot In-Reply-To: <3E7B7F37-D4F0-4835-9323-89BFA100149B@redivi.com> References: <17337.60315.618431.480677@montanaro.dyndns.org> <727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com> <17338.40044.896652.583651@montanaro.dyndns.org> <1F024C8F-E68E-40F7-991A-AE25B6616CFF@redivi.com> <43BB04D6.5080302@v.loewis.de> <3E7B7F37-D4F0-4835-9323-89BFA100149B@redivi.com> Message-ID: <43BB1009.7080008@v.loewis.de> Bob Ippolito wrote: > Clearly, but that still doesn't answer the question of who's going to > do it. Writing two code paths with availability macros and all the > testing involved is a whole lot of work to fix a harmless warning on > older versions of the OS. If it actually caused a problem I'm sure > someone would go through the trouble, but in this case it's probably > not worth the effort. I can fully understand that it is not worth your effort (it isn't worth mine, either). However, if Skip (or somebody else) wanted to fix it, I wouldn't mind. So I wouldn't agree to "not worth the effort". Regards, Martin From skip at pobox.com Wed Jan 4 04:15:03 2006 From: skip at pobox.com (skip at pobox.com) Date: Tue, 3 Jan 2006 21:15:03 -0600 Subject: [Python-Dev] buildbot In-Reply-To: <43BB048B.7080909@v.loewis.de> References: <17337.60315.618431.480677@montanaro.dyndns.org> <727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com> <17338.40044.896652.583651@montanaro.dyndns.org> <871wzpmg3k.fsf@tleepslib.sk.tsukuba.ac.jp> <43BB048B.7080909@v.loewis.de> Message-ID: <17339.15799.408323.601253@montanaro.dyndns.org> >>>>> "Martin" == Martin v L?wis writes: Martin> Stephen J. Turnbull wrote: >> Sure, Python can be expected to do much better than random port >> maintainers, but.... Discussions with the port maintainers makes it >> hard to blame them. Martin> Well, it's the same attitude that Fredrik Lundh just critized. Martin> It's ok for a port maintainer to say "I don't have any energy to Martin> fix it myself" (although I would hope he still says "patches Martin> will be accepted"). It's not ok for a port maintainer to say Martin> "it's not a bug - just upgrade your system". It *is* a bug for Martin> Python to emit warnings on "major platforms" (PEP 7). Thank you. I was beginning to think I was going crazy. Skip From walter at livinglogic.de Wed Jan 4 07:27:03 2006 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Wed, 04 Jan 2006 07:27:03 +0100 Subject: [Python-Dev] slight inconsistency in svn checkin email subject lines In-Reply-To: <43BB024A.6030109@v.loewis.de> References: <20051230184534.GC30751@ActiveState.com> <43B9352C.9080800@v.loewis.de> <20060103200229.GC31965@activestate.com> <43BAF752.4040504@v.loewis.de> <43BB024A.6030109@v.loewis.de> Message-ID: <43BB6AB7.8000301@livinglogic.de> Martin v. L?wis wrote: > [...] > commit_subject_prefix = [Python-checkins] commit of > > [...] > > Anyway, I just killed the "[Python-checkins]" part from the prefix > (expecting that mailman would add it, anyway), and regenerated > the r41848 commit message, and it seem that worked this time. And while we're at it, could you remove the "commit of" too? IMHO it just obscures the real content of the subject. Bye, Walter D?rwald From raymond.hettinger at verizon.net Wed Jan 4 07:37:51 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Wed, 04 Jan 2006 01:37:51 -0500 Subject: [Python-Dev] slight inconsistency in svn checkin email subject lines In-Reply-To: <43BB6AB7.8000301@livinglogic.de> Message-ID: <000801c610f9$63acadc0$741cc797@oemcomputer> > And while we're at it, could you remove the "commit of" too? IMHO it > just obscures the real content of the subject. +1 Raymond From warner at lothar.com Wed Jan 4 09:30:58 2006 From: warner at lothar.com (Brian Warner) Date: Wed, 04 Jan 2006 00:30:58 -0800 (PST) Subject: [Python-Dev] buildbot In-Reply-To: ee2a432c0601021706p43cd8001j1b7e1b1547a7bc2a@mail.gmail.com Message-ID: <20060104.003058.48524852.warner@lothar.com> > Currently, branches are not supported, because buildbot is > somewhat limited. When I get a solution for this problem, > I plan to have all buildbots build both the trunk and the > 2.4 branch; such builds would only be initiated whenever > something is committed on the branch. Branch support appeared fairly recently (buildbot-0.7.0, about two months ago), and I believe it should be sufficient to trigger builds for both the trunk and the 2.4 branch. The last part of the setup involves modifying the svn_buildbot.py hook script, though, so I won't claim this support is complete yet. The basic idea is that you create multiple Scheduler instances, one for each branch you want to monitor. Both Schedulers are configured to trigger all the Builders. You configure the SVN-checkout step of the build process with a baseURL= argument to provide the common portion of the SVN URL, and add a defaultBranch= argument to point to the trunk. Each Build is done for a specific branch, and the SVN checkout step just appends the branch name to the baseURL to figure out what to pass to 'svn co %s' Assuming the ChangeSource is providing branch information properly, this will result in any change to the trunk causing a trunk build, and any change to the 2.4 branch causing a 2.4 build. Your master.cfg file will probably want to have some of the following elements: ---BEGIN--- from buildbot.changes.pb import PBChangeSource c['sources'] = PBChangeSource() # N.B.: don't set prefix= from buildbot.scheduler import Scheduler all_builders=["sparc solaris10 gcc trunk", "g5 osx trunk", "amd64 gentoo trunk", "x86 gentoo trunk"] s1 = Scheduler("trunk scheduler", "trunk", 2*60, all_builders) s2 = Scheduler("2.4 scheduler", "branches/release24-maint", 2*60, all_builders) c['schedulers'] = [s1, s2] ... from buildbot.process.factory import s from buildbot.process.step import SVN checkoutstep = s(SVN, baseURL="http://svn.python.org/projects/python/", defaultBranch="trunk", mode="copy") steps = [checkoutstep, ...] ---END--- The tricky part is getting branch-annotated Changes into the buildbot in the first place. I assume you're using the svn_buildbot.py script. The version provided in the current release always sets the branch to 'None', which implies the default branch. You will have to edit this script to give it the ability to decide which branch each change is for (by adding some code which does a test like filename.startswith("trunk/"), perhaps), then include the branch name in the RPC call that informs the buildmaster about the new Change. Basically you'll need to change this call: return persp.callRemote('addChange', {'who': who, 'files': changed, 'comments': message, 'revision': revision}) into one like: branchname = "trunk" # or "branches/release24-maint" return persp.callRemote('addChange', {'who': who, 'branch': branchname, 'files': changed, 'comments': message, 'revision': revision}) There's been some discussion on the buildbot-devel list about the best way to implement branch-determination. One school of thought says it belongs close to the repository (hence inside a hook script like svn_buildbot.py). Another says it is easier to configure from the buildbot side, in which case you'd probably want to modify buildbot.changes.pb.PBChangeSource to analyze the filenames and decide which branch they live on there. There was also a patch[1] to add some regexps to svn_buildbot.py to do this kind of branch determination. I haven't been able to review it properly yet, but it may give you some ideas. svn_buildbot.py certainly needs help here. The rest of the Buildbot is branch-enabled and ready to go. The only lingering issues are with status delivery: the main HTML "Waterfall" HTML will interleave builds of both branches on the same page, which could be a bit confusing (if the top line is red, does that mean the trunk is broken, or the 2.4 branch?). Fixing this (by creating separate pages for each branch) is high on the TODO list. If there's any way I can help out further, please let me know. I'm ecstatic that Python is using a buildbot.. if you have any feature suggestions or bug reports, please send them to me, to the buildbot-devel mailing list[2], or file them in the sf.net bugtracker[3]. Or corner me at PyCon, or in #twisted (where I can sometimes be found as ). thanks! -Brian (author of Buildbot) [1]: http://sourceforge.net/mailarchive/message.php?msg_id=13977004 [2]: http://sourceforge.net/mailarchive/forum.php?forum=buildbot-devel [3]: http://sourceforge.net/projects/buildbot/ From martin at v.loewis.de Wed Jan 4 09:47:02 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 04 Jan 2006 09:47:02 +0100 Subject: [Python-Dev] slight inconsistency in svn checkin email subject lines In-Reply-To: <43BB6AB7.8000301@livinglogic.de> References: <20051230184534.GC30751@ActiveState.com> <43B9352C.9080800@v.loewis.de> <20060103200229.GC31965@activestate.com> <43BAF752.4040504@v.loewis.de> <43BB024A.6030109@v.loewis.de> <43BB6AB7.8000301@livinglogic.de> Message-ID: <43BB8B86.7000509@v.loewis.de> Walter D?rwald wrote: > And while we're at it, could you remove the "commit of" too? IMHO it > just obscures the real content of the subject. Done. FYI, the rationale for this prefix was that post-commit distinguishes between "commit" and "revprop", where revprop would indicate that properties changed, not the contents. I just noticed that this is not actually post-commit, but that there is a second hook, post-revprop-change, which I hadn't filled out yet. I just created the hook; because I dropped the prefix, commit and revprop mails will have the same subject. Regards, Martin From gregoire.weber at schmid-telecom.ch Wed Jan 4 12:17:32 2006 From: gregoire.weber at schmid-telecom.ch (Weber, Gregoire) Date: Wed, 4 Jan 2006 12:17:32 +0100 Subject: [Python-Dev] Real time behaviour of Pythons memory management; WAS: RE: Documentation about Python's GC,python-dev list messages referenced in Modules/gcmodule.c notreachable anymore Message-ID: <52D00EBDF4ECC74389E3B4F3D39F8A290FC575@ZHEXCHANGE.schmid-telecom.com> Garbage Collector findings To understand pythons garbage collector better and to get a picture about the runtime behaviour and performance I did some experiments: The attached script allocates a lot of circularly self referencing lists. Then it instantiates a one item list. I tuned the loop counter in such a way that the subsequent instantiation of the one item list triggers the garbage collector. The number of the circularly self referencing objects is around 90000. Results (Pentium4 3GHz, Python 2.4.1 on cygwin/Win2k) ----------------------------------------------------- gc gen 1) | time msec 2) | unreachable 3) -----------+--------------+---------------- None | 0.000 | All 0 | 0.002 | All 1 | 0.007 | All 2 | 0.065 | All None | 0.000 | None 0 | 0.001 | None 1 | 0.003 | None 2 | 0.018 | None --> Collecting a self referencing list costs about 0.7 usec (micro seconds) --> Checking a non collectable self ref list costs about 0.2 usec (micro seconds) Legend: 1) the generation the garbage collector has been triggered to collect (None means that GC wasn't triggered) 2) time for instantiating a list with one entry in msec 3) All: All of the circularly self referencing lists were **unreachable** (thus got collected by the gc) None: None of the circularly self referencing lists were **unreachable** (no garbage available to collect) Questions --------- 1. Am I correct that in a system which instantiates a lot*) of containerish objects without destructing them again the GC may be triggered to evaluate all generations which may be very costy (see above)? 2. In a system where only generation 0 and 1 get evaluated (because not so much object got instantiated without beeing destructed) the costs are relatively low. Correct? *) a lot means here: more than ``threshold0 * threshold1 * threshold2`` objects, Python 2.4 default values are: 700*10*10 = 70000 Gregoire P.S.: Interestingely the counters found by experimenting seem to depend on the platform (my platform: Python 2.4.1, cygwin under Win2k, the counters aren't valid for native Python 2.4.2 under Win2k). -------------- next part -------------- A non-text attachment was scrubbed... Name: gctest.py Type: application/octet-stream Size: 3936 bytes Desc: gctest.py Url : http://mail.python.org/pipermail/python-dev/attachments/20060104/a19e29c2/attachment.obj From stephen at xemacs.org Wed Jan 4 13:47:41 2006 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 04 Jan 2006 21:47:41 +0900 Subject: [Python-Dev] buildbot In-Reply-To: =?iso-8859-1?q?=3C43BB048B=2E7080909=40v=2Eloewis=2Ede=3E_?= =?iso-8859-1?q?=28Martin_v=2E_L=F6wis's_message_of_=22Wed=2C_04_Jan_200?= =?iso-8859-1?q?6_00=3A11=3A07_+0100=22=29?= References: <17337.60315.618431.480677@montanaro.dyndns.org> <727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com> <17338.40044.896652.583651@montanaro.dyndns.org> <871wzpmg3k.fsf@tleepslib.sk.tsukuba.ac.jp> <43BB048B.7080909@v.loewis.de> Message-ID: <87vex0jh1u.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Martin" == Martin v L?wis writes: Martin> It *is* a bug for Python to emit warnings on "major Martin> platforms" (PEP 7). OK, I'm as big a standards bigot as the next guy, you hooked me. After some consideration, I can't write the patch, though. I'm sorry that all I can offer is time and trouble testing on a Mac G4 running Panther (10.3.9), but for what it's worth, I offer it. I'll get the workspace ready, and when there's a patch, I'll test it. Martin> Also, it would be good if PEP 11 was considered before Martin> breaking platforms. I don't think Apple is ever likely to consider Python PEPs when revising its standard headers in incompatible ways.<0.5 wink> -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From thomas at xs4all.net Wed Jan 4 17:34:19 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Wed, 4 Jan 2006 17:34:19 +0100 Subject: [Python-Dev] file.next() vs. file.readline() Message-ID: <20060104163419.GF18916@xs4all.nl> Twice now, I've been bitten by the 'mixing file-iteration with readline' issue. (Yeah, twice.. Good thing I don't write anything important in Python ;) I've also seen a number of newbies bitten by the same thing. The issue, for those that aren't aware, is that when mixing file-iteration and readline (or readlines or such), you run the risk of getting data in the wrong order. This is because file-iteration uses an 8k buffer, while readline doesn't. It's not really a big deal, once you understand it's not wise to mix iteration and the readline(s) methods. I do wonder, though, why Python doesn't take care to raise an exception when readline is called with 'stuff' in the iteration-buffer. A cursory look at the source doesn't reveal any glaring problems. It's a single check, possibly two, with good locality of reference. Also, raising an exception when there is stuff in the buffer would only make mixing iteration/readline an error when you would actually, in fact, lose or mess up data. In other words, it would only raise exceptions for existing cases that are already broken. Is there something I've missed that makes the check undesireable or unfeasible? Or should I just assume no on has gotten around to it (or could be bothered), and just submit a patch? :) (Making readline and friends use the same buffer may seem the better solution to some, but I'm sure there's a whole discussion behind that, about whether to buffer or not. All non-iteration routines in fileobject.c take pretty good care not to read too much, and I choose to see that as explicitly designed that way.) Absent-ly y'rs, -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From guido at python.org Wed Jan 4 19:10:07 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 4 Jan 2006 10:10:07 -0800 Subject: [Python-Dev] file.next() vs. file.readline() In-Reply-To: <20060104163419.GF18916@xs4all.nl> References: <20060104163419.GF18916@xs4all.nl> Message-ID: I'd say go right ahead and submit a change to SF (and then after it's reviewed you can check it in yourself :-). The only reason I can think of why this isn't done yet would be that nobody thought of it. Of course there are other ill-advised combinations, like write followed by read for which stdio doesn't guarantee any result in particular. In Py3K I want to revise the whole I/O stack to be independent from C stdio (except on those platforms where that's the only I/O you have.) --Guido On 1/4/06, Thomas Wouters wrote: > > Twice now, I've been bitten by the 'mixing file-iteration with readline' > issue. (Yeah, twice.. Good thing I don't write anything important in > Python ;) I've also seen a number of newbies bitten by the same thing. The > issue, for those that aren't aware, is that when mixing file-iteration and > readline (or readlines or such), you run the risk of getting data in the > wrong order. This is because file-iteration uses an 8k buffer, while > readline doesn't. It's not really a big deal, once you understand it's not > wise to mix iteration and the readline(s) methods. > > I do wonder, though, why Python doesn't take care to raise an exception when > readline is called with 'stuff' in the iteration-buffer. A cursory look at > the source doesn't reveal any glaring problems. It's a single check, > possibly two, with good locality of reference. Also, raising an exception > when there is stuff in the buffer would only make mixing iteration/readline > an error when you would actually, in fact, lose or mess up data. In other > words, it would only raise exceptions for existing cases that are already > broken. > > Is there something I've missed that makes the check undesireable or > unfeasible? Or should I just assume no on has gotten around to it (or could > be bothered), and just submit a patch? :) > > (Making readline and friends use the same buffer may seem the better > solution to some, but I'm sure there's a whole discussion behind that, about > whether to buffer or not. All non-iteration routines in fileobject.c take > pretty good care not to read too much, and I choose to see that as > explicitly designed that way.) > > Absent-ly y'rs, > -- > Thomas Wouters > > Hi! I'm a .signature virus! copy me into your .signature file to help me spread! > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From lists at core-sdi.com Wed Jan 4 18:43:15 2006 From: lists at core-sdi.com (Gabriel Becedillas) Date: Wed, 04 Jan 2006 14:43:15 -0300 Subject: [Python-Dev] Sharing between multiple interpreters and restricted mode Message-ID: <43BC0933.2060705@core-sdi.com> Hi, At the company I work for we've embedded Python 2.4.1 in a C++ application. We execute multiple scripts concurrenlty, each one in its own interpreter (created using Py_NewInterpreter()). We are sharing a certain instance between interpreters because its to expensive to instantiate that class every time an interpreter is created. The class is instantiated in the main interpreter (that is always alive) and every time a new interpreter is created, that instance is added to the interpreter's __builtins__ dict. The problem I'm having is that some methods of that class (when executed in an interpreter different from the one it was created in) complain about being in restricted execution mode. Assuming that there are no issues with this instance lifetime (coz it will always be referenced by the main interpreter), is there a safe way to do some sharing between interpreters ?. Thanks. -- Gabriel Becedillas Developer CORE SECURITY TECHNOLOGIES Florida 141 - 2? cuerpo - 7? piso C1005AAC Buenos Aires - Argentina Tel/Fax: (54 11) 5032-CORE (2673) http://www.corest.com From guido at python.org Wed Jan 4 19:29:09 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 4 Jan 2006 10:29:09 -0800 Subject: [Python-Dev] Sharing between multiple interpreters and restricted mode In-Reply-To: <43BC0933.2060705@core-sdi.com> References: <43BC0933.2060705@core-sdi.com> Message-ID: This is more a question for c.l.py. My own suggestion is to go back to a single shared interpreter; what's the value of separate interpreters if you're sharing objects anyway? Sharing Python code between interpreters is definitely not supported. If you insist on separate interpreters, an alternative solution to your heavyweight shared object might be a lighter-weight facade for that object which can be instantiated separately in each interpreter? --Guido On 1/4/06, Gabriel Becedillas wrote: > Hi, > At the company I work for we've embedded Python 2.4.1 in a C++ > application. We execute multiple scripts concurrenlty, each one in its > own interpreter (created using Py_NewInterpreter()). > We are sharing a certain instance between interpreters because its to > expensive to instantiate that class every time an interpreter is > created. The class is instantiated in the main interpreter (that is > always alive) and every time a new interpreter is created, that > instance is added to the interpreter's __builtins__ dict. > The problem I'm having is that some methods of that class (when > executed in an interpreter different from the one it was created in) > complain about being in restricted execution mode. > Assuming that there are no issues with this instance lifetime (coz it > will always be referenced by the main interpreter), is there a safe way > to do some sharing between interpreters ?. > Thanks. > > -- > > Gabriel Becedillas > Developer > CORE SECURITY TECHNOLOGIES > > Florida 141 - 2? cuerpo - 7? piso > C1005AAC Buenos Aires - Argentina > Tel/Fax: (54 11) 5032-CORE (2673) > http://www.corest.com > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ronaldoussoren at mac.com Wed Jan 4 21:10:44 2006 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Wed, 4 Jan 2006 21:10:44 +0100 Subject: [Python-Dev] buildbot In-Reply-To: <3E7B7F37-D4F0-4835-9323-89BFA100149B@redivi.com> References: <17337.60315.618431.480677@montanaro.dyndns.org> <727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com> <17338.40044.896652.583651@montanaro.dyndns.org> <1F024C8F-E68E-40F7-991A-AE25B6616CFF@redivi.com> <43BB04D6.5080302@v.loewis.de> <3E7B7F37-D4F0-4835-9323-89BFA100149B@redivi.com> Message-ID: <5D707323-97AF-4939-A95F-EFC48DBC6D3D@mac.com> On 4-jan-2006, at 0:51, Bob Ippolito wrote: > > On Jan 3, 2006, at 3:12 PM, Martin v. L?wis wrote: > >> Bob Ippolito wrote: >>> Who's going to bother? >> >> It violates PEP 7, unless you argue that OS X/gcc is not >> a "major compiler". > > Clearly, but that still doesn't answer the question of who's going to > do it. Writing two code paths with availability macros and all the > testing involved is a whole lot of work to fix a harmless warning on > older versions of the OS. If it actually caused a problem I'm sure > someone would go through the trouble, but in this case it's probably > not worth the effort. More energy seems to be spent on talking about this than implementing the required patch :-) This should do it, although I haven't tested this on OSX 10.3: diff -u Python-2.4.2/Modules/getpath.c Python-2.4.2-orig/Modules/ getpath.c --- Python-2.4.2/Modules/getpath.c 2006-01-04 21:06:17.000000000 +0100 +++ Python-2.4.2-orig/Modules/getpath.c 2004-08-08 03:00:47.000000000 +0200 @@ -381,12 +381,8 @@ NSModule pythonModule; #endif #ifdef __APPLE__ -#ifdef MAC_OS_X_VERSION_10_4 - uint32_t nsexeclength = MAXPATHLEN; -#else unsigned long nsexeclength = MAXPATHLEN; #endif -#endif /* If there is no slash in the argv0 path, then we have to * assume python is on the user's $PATH, since there's no > > -bob > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ > ronaldoussoren%40mac.com From reinhold-birkenfeld-nospam at wolke7.net Wed Jan 4 22:01:21 2006 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Wed, 04 Jan 2006 22:01:21 +0100 Subject: [Python-Dev] TAR problems under Solaris Message-ID: Recently, someone on dclpy posted about an error he got when he tried to unpack the Python distribution tarball with Sparc Solaris 9's tar: tar: directory checksum error With GNU tar, it worked correctly. Is this a known issue, or is it irrelevant? Reinhold -- Mail address is perfectly valid! From tim.peters at gmail.com Wed Jan 4 22:07:37 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 4 Jan 2006 16:07:37 -0500 Subject: [Python-Dev] TAR problems under Solaris In-Reply-To: References: Message-ID: <1f7befae0601041307u3720fe18p485ee3ed2239aab7@mail.gmail.com> [Reinhold Birkenfeld] > Recently, someone on dclpy posted about an error he got > when he tried to unpack the Python distribution tarball > with Sparc Solaris 9's tar: > > tar: directory checksum error > > With GNU tar, it worked correctly. > > Is this a known issue, or is it irrelevant? It's a known issue for just about every tarball-based distribution of every project on the planet. For Python, Solaris tar woes are noted on this page: http://www.python.org/2.4.2/bugs.html From reinhold-birkenfeld-nospam at wolke7.net Wed Jan 4 22:10:58 2006 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Wed, 04 Jan 2006 22:10:58 +0100 Subject: [Python-Dev] TAR problems under Solaris In-Reply-To: <1f7befae0601041307u3720fe18p485ee3ed2239aab7@mail.gmail.com> References: <1f7befae0601041307u3720fe18p485ee3ed2239aab7@mail.gmail.com> Message-ID: Tim Peters wrote: > [Reinhold Birkenfeld] >> Recently, someone on dclpy posted about an error he got >> when he tried to unpack the Python distribution tarball >> with Sparc Solaris 9's tar: >> >> tar: directory checksum error >> >> With GNU tar, it worked correctly. >> >> Is this a known issue, or is it irrelevant? > > It's a known issue for just about every tarball-based distribution of > every project on the planet. For Python, Solaris tar woes are noted > on this page: > > http://www.python.org/2.4.2/bugs.html Ah ok, so never mind. Reinhold -- Mail address is perfectly valid! From barry at python.org Wed Jan 4 22:23:41 2006 From: barry at python.org (Barry Warsaw) Date: Wed, 04 Jan 2006 16:23:41 -0500 Subject: [Python-Dev] TAR problems under Solaris In-Reply-To: References: Message-ID: <1136409821.10363.59.camel@geddy.wooz.org> On Wed, 2006-01-04 at 22:01 +0100, Reinhold Birkenfeld wrote: > Recently, someone on dclpy posted about an error he got > when he tried to unpack the Python distribution tarball > with Sparc Solaris 9's tar: > > tar: directory checksum error > > With GNU tar, it worked correctly. > > Is this a known issue, or is it irrelevant? Dunno, but I'm always having problems w/ Solaris tar, so I just use GNU tar on Solaris. ;) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060104/5249bed0/attachment.pgp From martin at v.loewis.de Thu Jan 5 00:36:40 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 05 Jan 2006 00:36:40 +0100 Subject: [Python-Dev] buildno (Was: [Python-checkins] commit of r41907- python/trunk/Makefile.pre.in) In-Reply-To: <1136406410.10342.36.camel@geddy.wooz.org> References: <20060103143057.4DB8E1E401C@bag.python.org> <43BB9562.2070404@acm.org> <200601050238.51766.anthony@interlink.com.au> <1136406410.10342.36.camel@geddy.wooz.org> Message-ID: <43BC5C08.1010600@v.loewis.de> Barry Warsaw wrote: > Unfortunately, /usr/bin/type doesn't seem to accept the -t flag for me > on Solaris 9. Okay, so what's the best (read: portable) way to do this? The portable way would be to check for svnversion in configure, and then only use it if it was found. You could also check for .svn in configure, and generate the entire buildno generation. OTOH, I also think we should get rid of buildno entirely. Instead, svnversion should be compiled into the object file, or, if it is absent, $Revision$ should be used; the release process should be updated to force a commit to the tag/Modules/buildno.c right after creating the tag. sys.build_number should go, and be replaced with sys.svn_info, which should also include the branch from which the checkout/export was made. $Revision$ should only be trusted if it comes from a tag/. Should I write a PEP for that? Regards, Martin From martin at v.loewis.de Thu Jan 5 00:45:34 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 05 Jan 2006 00:45:34 +0100 Subject: [Python-Dev] Real time behaviour of Pythons memory management; WAS: RE: Documentation about Python's GC,python-dev list messages referenced in Modules/gcmodule.c notreachable anymore In-Reply-To: <52D00EBDF4ECC74389E3B4F3D39F8A290FC575@ZHEXCHANGE.schmid-telecom.com> References: <52D00EBDF4ECC74389E3B4F3D39F8A290FC575@ZHEXCHANGE.schmid-telecom.com> Message-ID: <43BC5E1E.5020304@v.loewis.de> Weber, Gregoire wrote: > Questions > --------- Notice that this article is not appropriate for python-dev. If you ask "is it the case that Python behaves such and such?", you have a python-list (comp.lang.python) question. For python-dev, the question should be formulated as "because it is such and such, would it be acceptable to change it so and so?" > 1. Am I correct that in a system which instantiates a lot*) > of containerish objects without destructing them again the > GC may be triggered to evaluate all generations which may > be very costy (see above)? Correct. > 2. In a system where only generation 0 and 1 get evaluated > (because not so much object got instantiated without beeing > destructed) the costs are relatively low. Correct? Correct. If the objects are *really* short-lived, a generation 0 collection might never get initiated, even though millions of containers are created. Regards, Martin From trentm at ActiveState.com Thu Jan 5 03:28:55 2006 From: trentm at ActiveState.com (Trent Mick) Date: Wed, 4 Jan 2006 18:28:55 -0800 Subject: [Python-Dev] buildbot In-Reply-To: <20060104.003058.48524852.warner@lothar.com> References: <20060104.003058.48524852.warner@lothar.com> Message-ID: <20060105022855.GB5183@activestate.com> [Brian Warner wrote] >... > The only lingering issues are with status delivery: the main HTML > "Waterfall" HTML will interleave builds of both branches on the same > page, which could be a bit confusing (if the top line is red, does > that mean the trunk is broken, or the 2.4 branch?). Or for separate logic projects being built with the same builtbot master. For example, say Python's buildbot wanted to do regular builds and tests of the distutils tree (http://svn.python.org/view/distutils/trunk/). > Fixing this (by creating separate pages for each branch) is high on > the TODO list. I'm keen to help with that if I can. I'm now subscribed to buildbot-devel (as trentm at gmail) so I can pester you about that there further. Cheers, Trent -- Trent Mick trentm at activestate.com From trentm at ActiveState.com Thu Jan 5 03:35:55 2006 From: trentm at ActiveState.com (Trent Mick) Date: Wed, 4 Jan 2006 18:35:55 -0800 Subject: [Python-Dev] buildbot In-Reply-To: <20060105022855.GB5183@activestate.com> References: <20060104.003058.48524852.warner@lothar.com> <20060105022855.GB5183@activestate.com> Message-ID: <20060105023555.GC5183@activestate.com> [Trent Mick wrote] > Or for separate logic projects being built with the same builtbot s/logic/logical/ Trent -- Trent Mick trentm at activestate.com From skip at pobox.com Thu Jan 5 04:23:07 2006 From: skip at pobox.com (skip at pobox.com) Date: Wed, 4 Jan 2006 21:23:07 -0600 Subject: [Python-Dev] buildbot In-Reply-To: <5D707323-97AF-4939-A95F-EFC48DBC6D3D@mac.com> References: <17337.60315.618431.480677@montanaro.dyndns.org> <727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com> <17338.40044.896652.583651@montanaro.dyndns.org> <1F024C8F-E68E-40F7-991A-AE25B6616CFF@redivi.com> <43BB04D6.5080302@v.loewis.de> <3E7B7F37-D4F0-4835-9323-89BFA100149B@redivi.com> <5D707323-97AF-4939-A95F-EFC48DBC6D3D@mac.com> Message-ID: <17340.37147.878933.687304@montanaro.dyndns.org> Ronald> This should do it, although I haven't tested this on OSX 10.3: Not quite. On my 10.3 system MAC_OS_X_VERSION_10_ for in 0, 1, 2, 3, 4 is defined. However, MAC_OS_X_VERSION_MAX_ALLOWED is defined to be MAC_OS_X_VERSION_10_3. This works for me (compiles with no warnings, passes all tests). Skip % svn diff Modules/getpath.c Index: Modules/getpath.c =================================================================== --- Modules/getpath.c (revision 41914) +++ Modules/getpath.c (working copy) @@ -381,8 +381,12 @@ NSModule pythonModule; #endif #ifdef __APPLE__ +#if MAC_OS_X_VERSION_MAX_ALLOWED >= MAC_OS_X_VERSION_10_4 uint32_t nsexeclength = MAXPATHLEN; +#else + unsigned long nsexeclength = MAXPATHLEN; #endif +#endif /* If there is no slash in the argv0 path, then we have to * assume python is on the user's $PATH, since there's no From anthony at interlink.com.au Thu Jan 5 04:54:44 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu, 5 Jan 2006 14:54:44 +1100 Subject: [Python-Dev] TAR problems under Solaris In-Reply-To: <1136409821.10363.59.camel@geddy.wooz.org> References: <1136409821.10363.59.camel@geddy.wooz.org> Message-ID: <200601051454.46472.anthony@interlink.com.au> On Thursday 05 January 2006 08:23, Barry Warsaw wrote: > Dunno, but I'm always having problems w/ Solaris tar, so I just use > GNU tar on Solaris. ;) Maybe we should switch to cpio-based distributions? Anthony From nnorwitz at gmail.com Thu Jan 5 09:39:39 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 5 Jan 2006 00:39:39 -0800 Subject: [Python-Dev] bsddb broken Message-ID: Anyone wanna give bsddb some tlc? Head and 2.4 work with 4.2 on amd64 and x86. Neither python version works when using BSD db 4.1 or 3.2. I don't know anything about bsddb, so any help fixing this would be appreciated. In 4.1 it seems associate doesn't work. http://python.org/sf/1332873 3.2 has other problems. n From martin at v.loewis.de Thu Jan 5 10:33:29 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 05 Jan 2006 10:33:29 +0100 Subject: [Python-Dev] TAR problems under Solaris In-Reply-To: <200601051454.46472.anthony@interlink.com.au> References: <1136409821.10363.59.camel@geddy.wooz.org> <200601051454.46472.anthony@interlink.com.au> Message-ID: <43BCE7E9.90602@v.loewis.de> Anthony Baxter wrote: >>Dunno, but I'm always having problems w/ Solaris tar, so I just use >>GNU tar on Solaris. ;) > > > Maybe we should switch to cpio-based distributions? Peace, brother... Err, pax(1). -bash-3.00$ uname -a SunOS tb3 5.10 Generic sun4u sparc SUNW,Sun-Fire-V440 -bash-3.00$ bunzip2 -c Python-2.4.2.tar.bz2 |pax -r pax: ././@LongLink : Unknown filetype This message refers to the links in Mac/OSXResources/app/Resources/English.lproj/Documentation/ide Apart from that, pax extracts it the same way as gtar. Regards, Martin P.S. I always ROTFL when I see mentioning of the "tar wars" in the POSIX standard. From skip at pobox.com Thu Jan 5 11:53:00 2006 From: skip at pobox.com (skip at pobox.com) Date: Thu, 5 Jan 2006 04:53:00 -0600 Subject: [Python-Dev] buildbot In-Reply-To: <17340.37147.878933.687304@montanaro.dyndns.org> References: <17337.60315.618431.480677@montanaro.dyndns.org> <727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com> <17338.40044.896652.583651@montanaro.dyndns.org> <1F024C8F-E68E-40F7-991A-AE25B6616CFF@redivi.com> <43BB04D6.5080302@v.loewis.de> <3E7B7F37-D4F0-4835-9323-89BFA100149B@redivi.com> <5D707323-97AF-4939-A95F-EFC48DBC6D3D@mac.com> <17340.37147.878933.687304@montanaro.dyndns.org> Message-ID: <17340.64140.68114.605704@montanaro.dyndns.org> me> This works for me (compiles with no warnings, passes all tests). ... The bagpipe didn't say "no", so I checked this in on trunk and the 2.4 branch. Skip From ronaldoussoren at mac.com Thu Jan 5 12:07:46 2006 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Thu, 5 Jan 2006 12:07:46 +0100 Subject: [Python-Dev] buildbot In-Reply-To: <17340.64140.68114.605704@montanaro.dyndns.org> References: <17337.60315.618431.480677@montanaro.dyndns.org> <727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com> <17338.40044.896652.583651@montanaro.dyndns.org> <1F024C8F-E68E-40F7-991A-AE25B6616CFF@redivi.com> <43BB04D6.5080302@v.loewis.de> <3E7B7F37-D4F0-4835-9323-89BFA100149B@redivi.com> <5D707323-97AF-4939-A95F-EFC48DBC6D3D@mac.com> <17340.37147.878933.687304@montanaro.dyndns.org> <17340.64140.68114.605704@montanaro.dyndns.org> Message-ID: <66D28935-C68C-4E92-9DDE-D41B308A5CC3@mac.com> On 5-jan-2006, at 11:53, skip at pobox.com wrote: > > me> This works for me (compiles with no warnings, passes all > tests). > ... > > The bagpipe didn't say "no", so I checked this in on trunk and the 2.4 > branch. I haven't tested this on 10.4 yet, but it should work. The heuristic is false for anyone that will try to build a 10.3-only binary on 10.4, but I'd be surprised if that would work anyway. Ronald > > Skip > > From xavier.morel at masklinn.net Wed Jan 4 00:31:49 2006 From: xavier.morel at masklinn.net (Morel Xavier) Date: Wed, 04 Jan 2006 00:31:49 +0100 Subject: [Python-Dev] Buildbot questions Message-ID: <43BB0965.5080301@masklinn.net> I currently have a (quite weak) computer that mostly sits idle (shares the web connection), Tbird 750; 640Mb RAM; Windows Server 2003 Standard Edition. Since the computer sits there doing nothing, I could probably put a buildbot on it if needed (since the python-dev thread states that many windows flavour would be appreciable and that Win2003 may not be extremely common), but i'd like to know how often it'll try to build, and how long the build itself may take on such a platform. Morel Xavier From skip at pobox.com Thu Jan 5 14:38:21 2006 From: skip at pobox.com (skip at pobox.com) Date: Thu, 5 Jan 2006 07:38:21 -0600 Subject: [Python-Dev] Buildbot questions In-Reply-To: <43BB0965.5080301@masklinn.net> References: <43BB0965.5080301@masklinn.net> Message-ID: <17341.8525.39858.703311@montanaro.dyndns.org> Morel> but i'd like to know how often it'll try to build, and how long Morel> the build itself may take on such a platform. It should build every time there's a checkin on trunk or the 2.4 branch. As for performance, take a look at http://www.python.org/dev/buildbot/ to see how long some of the other build bots take and extrapolate from that. Skip From barry at python.org Thu Jan 5 14:38:18 2006 From: barry at python.org (Barry Warsaw) Date: Thu, 05 Jan 2006 08:38:18 -0500 Subject: [Python-Dev] buildno (Was: [Python-checkins] commit of r41907- python/trunk/Makefile.pre.in) In-Reply-To: <43BC5C08.1010600@v.loewis.de> References: <20060103143057.4DB8E1E401C@bag.python.org> <43BB9562.2070404@acm.org> <200601050238.51766.anthony@interlink.com.au> <1136406410.10342.36.camel@geddy.wooz.org> <43BC5C08.1010600@v.loewis.de> Message-ID: <1136468298.15418.43.camel@geddy.wooz.org> On Thu, 2006-01-05 at 00:36 +0100, "Martin v. L?wis" wrote: > The portable way would be to check for svnversion in configure, and then > only use it if it was found. You could also check for .svn in configure, > and generate the entire buildno generation. > > OTOH, I also think we should get rid of buildno entirely. Instead, > svnversion should be compiled into the object file, or, if it is absent, > $Revision$ should be used; the release process should be updated to > force a commit to the tag/Modules/buildno.c right after creating the > tag. sys.build_number should go, and be replaced with sys.svn_info, > which should also include the branch from which the checkout/export > was made. $Revision$ should only be trusted if it comes from a > tag/. > > Should I write a PEP for that? To be honest I don't think we need a PEP for that. I saw that you checked in a bunch of stuff here, and that's great, thanks! I was working on a patch to add a PY_BUILDNO macro to Include/patchlevel.h, which would have "$Revision$" as its value. patchlevel.h seems like the natural place to put this, since any release manager is going to be modifying this file anyway. PEP 101 should be updated so that updating patchlevel.h be the last commit before an svn export is done (but that PEP needs an overhaul anyway to change the cvs references into svn commands). Thoughts? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060105/f8c0e376/attachment.pgp From anthony at interlink.com.au Thu Jan 5 15:53:45 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri, 6 Jan 2006 01:53:45 +1100 Subject: [Python-Dev] Buildbot questions In-Reply-To: <17341.8525.39858.703311@montanaro.dyndns.org> References: <43BB0965.5080301@masklinn.net> <17341.8525.39858.703311@montanaro.dyndns.org> Message-ID: <200601060153.47011.anthony@interlink.com.au> FWIW, I have an older box running Ubuntu 05.10 that spends most of it's days pining for stuff to do (at the moment, it does DHCP and DNS for the house. Yes, I know I have too many damn computers here). I can set up a buildbot on it easily enough. It's something like a 600MHz P3 or something. Is it going to be useful to have this in the mix? Heck, there's a pile of 500MHz P3s sitting here that I could drop a random free unix onto if someone wants to nominate something that's a) useful b) not a total pain in the clacker to install. From tim.peters at gmail.com Thu Jan 5 16:43:56 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 5 Jan 2006 10:43:56 -0500 Subject: [Python-Dev] Buildbot questions In-Reply-To: <43BB0965.5080301@masklinn.net> References: <43BB0965.5080301@masklinn.net> Message-ID: <1f7befae0601050743p32e542f4gb899153b2d3ae94c@mail.gmail.com> [Morel Xavier] > I currently have a (quite weak) computer that mostly sits idle (shares > the web connection), Tbird 750; 640Mb RAM; Windows Server 2003 Standard > Edition. > > Since the computer sits there doing nothing, I could probably put a > buildbot on it if needed (since the python-dev thread states that many > windows flavour would be appreciable and that Win2003 may not be > extremely common), but i'd like to know how often it'll try to build, > and how long the build itself may take on such a platform. A problem for Windows buildbot slaves is that they need an appropriate compiler. Does this machine have MS VC 7.1 installed? If not, it can't compile the code. The Windows Python would also like to build several other packages (like bz2 and Tcl/Tk). An "update" style of slave does an svn update rather than a full checkout, and that usually goes very fast after the first time. Likewise compiling if binaries are left behind across runs. For the rest, open a DOS box on this machine, cd to root of a Python 2.4.2 installation, and time python Lib\test\regrtest.py -uall That's about how long it will take on that machine to run all the tests from the current trunk too. Really thorough tests take 8x as long (with and without purging .pyc/.pyo files first, with and without -O, and under release and debug builds: 2*2*2 = 8). From fwierzbicki at gmail.com Thu Jan 5 19:17:50 2006 From: fwierzbicki at gmail.com (Frank Wierzbicki) Date: Thu, 5 Jan 2006 13:17:50 -0500 Subject: [Python-Dev] Jython and CPython In-Reply-To: References: <439F2A3F.4000703@v.loewis.de> Message-ID: <4dab5f760601051017m4fdaf4f1l905c18b2b9f29b50@mail.gmail.com> > > If the portability problem can be solved by checking things into Jython > > instead, I think I would prefer that. I'm definitely interested in bringing ElementTree into Jython at some point, though I probably won't have time to look into it until after the next Jython release. I'm quite sure that we can work something out so that Jython-specific portability code can reside in Jython only. Though whenever it is cleanly do-able, I'd like to be able to use the python libraries unchanged. It sounds like this is a case where it is not that clean to do... -Frank From thomas at xs4all.net Thu Jan 5 19:30:08 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Thu, 5 Jan 2006 19:30:08 +0100 Subject: [Python-Dev] file.next() vs. file.readline() In-Reply-To: References: <20060104163419.GF18916@xs4all.nl> Message-ID: <20060105183008.GG18916@xs4all.nl> On Wed, Jan 04, 2006 at 10:10:07AM -0800, Guido van Rossum wrote: > I'd say go right ahead and submit a change to SF (and then after it's > reviewed you can check it in yourself :-). http://www.python.org/sf?id=1397960 The patch comments and source should explain it all. The diff is quite big (just over 16k) because of the 16k testfile. As for checking in myself, I think my write-access was (rightly, considering my absense) removed sometime last year :) > In Py3K I want to revise the whole I/O stack to be independent from C > stdio (except on those platforms where that's the only I/O you have.) When I first read that, I thought "yuuuuck". Then I went and actually read all of fileobject.c, went "yuuuuck" more times than I cared to count, and now I wholeheartedly agree ;) Changing the read* methods to use the file-iteration buffer would be a lot simpler than I thought, by the way. And it would simplify a lot of the code (not just because of the code duplication currently necessary.) I'm sure there'll be people who like having direct control over howmuch gets read though. Which is fine, I think that should stay possible, but I don't think those people deserve to get the full force of Tim's optimizations either. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From arigo at tunes.org Thu Jan 5 20:38:33 2006 From: arigo at tunes.org (Armin Rigo) Date: Thu, 5 Jan 2006 20:38:33 +0100 Subject: [Python-Dev] buildno (Was: [Python-checkins] commit of r41907- python/trunk/Makefile.pre.in) In-Reply-To: <43BC5C08.1010600@v.loewis.de> References: <20060103143057.4DB8E1E401C@bag.python.org> <43BB9562.2070404@acm.org> <200601050238.51766.anthony@interlink.com.au> <1136406410.10342.36.camel@geddy.wooz.org> <43BC5C08.1010600@v.loewis.de> Message-ID: <20060105193833.GA11736@code1.codespeak.net> Hi Martin, On Thu, Jan 05, 2006 at 12:36:40AM +0100, "Martin v. L?wis" wrote: > OTOH, I also think we should get rid of buildno entirely. Instead, > svnversion should be compiled into the object file, or, if it is absent, > $Revision$ should be used; the release process should be updated to > force a commit to the tag/Modules/buildno.c right after creating the > tag. sys.build_number should go, and be replaced with sys.svn_info, > which should also include the branch from which the checkout/export > was made. $Revision$ should only be trusted if it comes from a > tag/. All this sounds good. > Should I write a PEP for that? I agree with Barry that it's overkill to ask for PEPs for too many small details. A bientot, Armin From arigo at tunes.org Thu Jan 5 20:45:38 2006 From: arigo at tunes.org (Armin Rigo) Date: Thu, 5 Jan 2006 20:45:38 +0100 Subject: [Python-Dev] New PEP: Using ssize_t as the index type In-Reply-To: <43B50B64.20904@v.loewis.de> References: <43B3ECEE.9060300@v.loewis.de> <20051229173041.GA19575@code1.codespeak.net> <43B50B64.20904@v.loewis.de> Message-ID: <20060105194538.GB11736@code1.codespeak.net> Hi Martin, On Fri, Dec 30, 2005 at 11:26:44AM +0100, "Martin v. L?wis" wrote: > > Hum. It would be much cleaner to introduce a new format character to > > replace '#' and deprecate '#'... > > That would certainly be clearer. What character would you suggest? > > I see two drawbacks with that approach: > 1. writing backwards-compatible modules will become harder. > Users have to put ifdefs around the ParseTuple calls (or atleast > around the format strings) Ok, I see your point. In theory we could reuse a macro-based trick in C extensions: #include #ifndef Py_SIZE_CHR typedef int Py_Ssize_t; #define Py_SIZE_CHR "#" #endif And then we can replace -- say -- the format string "is#s#" with "is" Py_SIZE_CHR "s" Py_SIZE_CHR But it's rather cumbersome. An equally strange alternative would be to start C modules like this: #define Py_Ssize_t int /* compatibility with Python <= 2.4 */ #include This would do the right thing for <= 2.4, using ints everywhere; and the Python.h version 2.5 would detect the #define and assume it's a 2.5-compatible module, so it would override the #define with the real thing *and* turn on the ssize_t interpretation of the '#' format character. Not that I think that this is a good idea. Just an idea. I still don't like the idea of a magic #define that changes the behavior of '#include ', but I admit I don't find any better solution. I suppose I'll just blame C. A bientot, Armin From jjl at pobox.com Thu Jan 5 20:52:22 2006 From: jjl at pobox.com (John J Lee) Date: Thu, 5 Jan 2006 19:52:22 +0000 (UTC) Subject: [Python-Dev] Buildbot questions In-Reply-To: <1f7befae0601050743p32e542f4gb899153b2d3ae94c@mail.gmail.com> References: <43BB0965.5080301@masklinn.net> <1f7befae0601050743p32e542f4gb899153b2d3ae94c@mail.gmail.com> Message-ID: On Thu, 5 Jan 2006, Tim Peters wrote: [...] > A problem for Windows buildbot slaves is that they need an appropriate > compiler. Does this machine have MS VC 7.1 installed? If not, it > can't compile the code. The Windows Python would also like to build > several other packages (like bz2 and Tcl/Tk). Might a buildbot running this setup of David Munman's (free MS compiler + NAnt interpreting the MS project file) be useful? http://groups.google.co.uk/group/comp.lang.python/browse_frm/thread/226584dd47047bb6/1e33ad19197bee20 (I'm not offering a buildbot myself, just pointing out the possibility of using this tool) There always the chance it gets something wrong or falls over while trying to interpret the project file, of course. That in itself would be beneficial, though, if there's a committer willing to fix it when it breaks -- I currently have access to MS compilers, but I remember many happy :-/ hours as a graduate student trying to compile Fortran and C extensions on win32 with free compilers. > An "update" style of slave does an svn update rather than a full > checkout, and that usually goes very fast after the first time. > Likewise compiling if binaries are left behind across runs. [...] Much though I like SVN, it seems its working copy management still leaves a little to be desired: Even quite recently (fairly sure it was client version 1.2.*, on Win XP) and with read-only checkouts used only for builds, I've still seen it end up in an incorrect state. I suspect 'svn switch' or 'svn up -r xxxxx' was the culprit, though, so perhaps it's not a problem if exactly 'svn up' is the only svn command ever executed on the checkout. Still, perhaps it's wise to wipe the checkout every so often? John From martin at v.loewis.de Thu Jan 5 21:41:08 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 05 Jan 2006 21:41:08 +0100 Subject: [Python-Dev] New PEP: Using ssize_t as the index type In-Reply-To: <20060105194538.GB11736@code1.codespeak.net> References: <43B3ECEE.9060300@v.loewis.de> <20051229173041.GA19575@code1.codespeak.net> <43B50B64.20904@v.loewis.de> <20060105194538.GB11736@code1.codespeak.net> Message-ID: <43BD8464.4060801@v.loewis.de> Armin Rigo wrote: > This would do the right thing for <= 2.4, using ints everywhere; and the > Python.h version 2.5 would detect the #define and assume it's a > 2.5-compatible module, so it would override the #define with the real > thing *and* turn on the ssize_t interpretation of the '#' format > character. This would be very similar to the PY_SIZE_T_CLEAN approach, except that it would also help to detect spelling mistakes. >From an implementation point of view, the real challenge is to give PyArg_ParseTuple a different meaning; I do this be #defining it to PyArg_ParseTupleSsize_t (to preserve binary compatibility for the original interpretation of ParseTuple). Putting additional flags arguments in the entire code is also quite hackish. > I still don't like the idea of a magic #define that changes the behavior > of '#include ', but I admit I don't find any better solution. > I suppose I'll just blame C. More precisely, the printf style of function calling, and varargs functions. ISO C is pretty type safe, but with varargs functions, you lose that completely. I still hope I can write a C parser some day that does ParseTuple/BuildValue checking. Regards, Martin From martin at v.loewis.de Thu Jan 5 21:44:32 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 05 Jan 2006 21:44:32 +0100 Subject: [Python-Dev] Buildbot questions In-Reply-To: <200601060153.47011.anthony@interlink.com.au> References: <43BB0965.5080301@masklinn.net> <17341.8525.39858.703311@montanaro.dyndns.org> <200601060153.47011.anthony@interlink.com.au> Message-ID: <43BD8530.1000606@v.loewis.de> Anthony Baxter wrote: > FWIW, I have an older box running Ubuntu 05.10 that spends most of > it's days pining for stuff to do (at the moment, it does DHCP and DNS > for the house. Yes, I know I have too many damn computers here). I > can set up a buildbot on it easily enough. It's something like a > 600MHz P3 or something. Is it going to be useful to have this in the > mix? With the gentoo installation, I think we have "enough" linux for the moment. Somebody noticed that the Waterfall view of buildbot quickly becomes unreadable if there are too many builds. > Heck, there's a pile of 500MHz P3s sitting here that I could drop a > random free unix onto if someone wants to nominate something that's > > a) useful > b) not a total pain in the clacker to install. For a), I think one of the BSDs might be useful. Whether they qualify for b), I don't know. Regards, Martin From martin at v.loewis.de Thu Jan 5 21:47:21 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 05 Jan 2006 21:47:21 +0100 Subject: [Python-Dev] Buildbot questions In-Reply-To: References: <43BB0965.5080301@masklinn.net> <1f7befae0601050743p32e542f4gb899153b2d3ae94c@mail.gmail.com> Message-ID: <43BD85D9.5030301@v.loewis.de> John J Lee wrote: > Might a buildbot running this setup of David Munman's (free MS compiler + > NAnt interpreting the MS project file) be useful? I feel that any contribution here takes quite some time initially, so somebody making that offer should accept some pain until it really works self-contained. I would need to know the exact sequence of commands that are necessary for the build; I assume the autoconf builder would be useless. Regards, Martin From anthony at interlink.com.au Thu Jan 5 22:08:08 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri, 6 Jan 2006 08:08:08 +1100 Subject: [Python-Dev] Buildbot questions In-Reply-To: <43BD8530.1000606@v.loewis.de> References: <43BB0965.5080301@masklinn.net> <200601060153.47011.anthony@interlink.com.au> <43BD8530.1000606@v.loewis.de> Message-ID: <200601060808.13439.anthony@interlink.com.au> On Friday 06 January 2006 07:44, Martin v. L?wis wrote: > With the gentoo installation, I think we have "enough" linux for > the moment. Somebody noticed that the Waterfall view of buildbot > quickly becomes unreadable if there are too many builds. My only concern is that it's gentoo, not just linux. I know that for a couple of my other open source projects I usually don't spend too long debugging bizarrely broken apparent bugs, because it ends up being some strange build flag or some such on the gentoo box in question. On the other hand, this box is unlikely to have been built with a selection of gcc flags that Neal just selected randomly from the gcc manual so it's probably going to be better than that. > > Heck, there's a pile of 500MHz P3s sitting here that I could drop > > a random free unix onto if someone wants to nominate something > > that's > > > > a) useful > > b) not a total pain in the clacker to install. > > For a), I think one of the BSDs might be useful. Whether they > qualify for b), I don't know. Anyone else have an opinion on the ease of installation for the various BSDs? Last time I tried one (which was several years ago) it was Not Very Good. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From bob at redivi.com Thu Jan 5 22:12:03 2006 From: bob at redivi.com (Bob Ippolito) Date: Thu, 5 Jan 2006 13:12:03 -0800 Subject: [Python-Dev] Buildbot questions In-Reply-To: <200601060808.13439.anthony@interlink.com.au> References: <43BB0965.5080301@masklinn.net> <200601060153.47011.anthony@interlink.com.au> <43BD8530.1000606@v.loewis.de> <200601060808.13439.anthony@interlink.com.au> Message-ID: <6ECB8452-9DC1-44FE-846F-885677FE1B9F@redivi.com> On Jan 5, 2006, at 1:08 PM, Anthony Baxter wrote: > On Friday 06 January 2006 07:44, Martin v. L?wis wrote: >> With the gentoo installation, I think we have "enough" linux for >> the moment. Somebody noticed that the Waterfall view of buildbot >> quickly becomes unreadable if there are too many builds. > > My only concern is that it's gentoo, not just linux. I know that for a > couple of my other open source projects I usually don't spend too > long debugging bizarrely broken apparent bugs, because it ends up > being some strange build flag or some such on the gentoo box in > question. On the other hand, this box is unlikely to have been built > with a selection of gcc flags that Neal just selected randomly from > the gcc manual so it's probably going to be better than that. > >>> Heck, there's a pile of 500MHz P3s sitting here that I could drop >>> a random free unix onto if someone wants to nominate something >>> that's >>> >>> a) useful >>> b) not a total pain in the clacker to install. >> >> For a), I think one of the BSDs might be useful. Whether they >> qualify for b), I don't know. > > Anyone else have an opinion on the ease of installation for the > various BSDs? Last time I tried one (which was several years ago) it > was Not Very Good. FreeBSD and OpenBSD are painless these days, assuming that you're comfortable reading text during installation. No experience with NetBSD. -bob From jjl at pobox.com Thu Jan 5 22:58:25 2006 From: jjl at pobox.com (John J Lee) Date: Thu, 5 Jan 2006 21:58:25 +0000 (UTC) Subject: [Python-Dev] Buildbot questions In-Reply-To: References: <43BB0965.5080301@masklinn.net> <1f7befae0601050743p32e542f4gb899153b2d3ae94c@mail.gmail.com> Message-ID: On Thu, 5 Jan 2006, John J Lee wrote: > Might a buildbot running this setup of David Munman's (free MS compiler + > NAnt interpreting the MS project file) be useful? s/Munman/Murmann/ John From martin at v.loewis.de Thu Jan 5 23:07:31 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 05 Jan 2006 23:07:31 +0100 Subject: [Python-Dev] Buildbot questions In-Reply-To: <200601060808.13439.anthony@interlink.com.au> References: <43BB0965.5080301@masklinn.net> <200601060153.47011.anthony@interlink.com.au> <43BD8530.1000606@v.loewis.de> <200601060808.13439.anthony@interlink.com.au> Message-ID: <43BD98A3.2060005@v.loewis.de> Anthony Baxter wrote: > My only concern is that it's gentoo, not just linux. I know that for a > couple of my other open source projects I usually don't spend too > long debugging bizarrely broken apparent bugs, because it ends up > being some strange build flag or some such on the gentoo box in > question. For buildbot, failures to build are good (if they are reproducable); if it succeeds for gentoo, but fails on some other system on which it ought to work, then adding this system as a build slave would be useful. Regards, Martin P.S. That's assuming we get in a state where the tests "usually" pass, of course. From alexander.kozlovsky at gmail.com Thu Jan 5 22:56:01 2006 From: alexander.kozlovsky at gmail.com (Alexander Kozlovsky) Date: Fri, 6 Jan 2006 00:56:01 +0300 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 Message-ID: <611494241.20060106005601@gmail.com> Hello! I have some proposal for Python 3.0 (interesting one, from my point of view). I'm sorry for my English, it is not very good. Abstract ======== There are three different peculiarity in Python 2.x in respect of 'self' method argument: 1. Each method must have explicit 'self' argument in its argument list 2. This argument must be used explicitly for instance's attribute value assignment 3. The count of explicit arguments in method definition differs from count of explicit parameters when method is called Example 1 (Python 2.x): ----------------------- class Foo: def __init__(self, x): # 1: Explicit 'self' argument self.x = x # 2: 'self' must be used explicitly def bar(self, a, b): # 3: There are three arguments... print self.x + a + b Foo(10).bar(20, 30) # ...but only two explicit parameters # is presented This document proposes to change this, as the next example shows: Example 2 (Python 3.0): ----------------------- class Foo: def __init__(x): # 1: Implicit self .x = x # 2: Brief form of: self.x = x def bar(a, b): # 3: Two arguments... print .x + a + b Foo(10).bar(20, 30) # ...and exactly two parameters According Python Zen, "Explicit is better then implicit", but "practicality beats purity" and "Readability counts" ;-) This draft document describes high-level semantic of proposed changes and doesn't discuss details of C implementation. Rationale ========= When programmer tries to pick up some new programming language from different possibilities (e.g. Smalltalk, Python, Perl, ...), he often bases his choice on secondary and non-essential details. Almost any language has peculiarities, which can distract potential users from language evaluation. Examples of such warts may be #@$&% perlish syntax in Ruby, abundance of parenthesis in Lisp and Schema, and explicit 'self' argument in Python. Of course, from technical point of view, such peculiarities are completely non-issue. Parenthesis is not problem in Lisp if you use a good text editor, perlisms in Ruby can be avoided, etc. But from sociological and usability points of view, such warts can cause remarkable hurt, because they affect first impression about language. In many superficial language comparisons Python's OO approach dismissed as after-thought because of the explicit 'self'. Example: But the most important aspect, why I am using Ruby instead of Python or Perl are the object-orientated features of Ruby, and that Ruby was designed object-oriented right from beginning, unlike Python and Perl where object-orientation was added on later. You can recognize this in e.g. in Python very good, because the first parameter (often named self) of every method of a class is the object on which the method is called (http://www.ntecs.de/old-hp/s-direktnet/rb/ruby.pdf) Of course, this words about Python are not true. Python was object-oriented from the beginning, and explicit 'self' is intentional design decision, which is elegant in some aspects. But from pragmatical point of view, implicit 'self' in Python 3.0 may lower entry barrier for many people and assist Python's wide adoption. The proposed change is not backward-compatible with Python 2.x Detailed proposals ================== 1. 'self' becomes a keyword, and may be used inside function definition to denote a special implicit function argument 2. New syntax shortcut is introduced: '.attr' inside a function is exact equivalent to 'self.attr'. Full syntax can be used as well 3. 'class' keyword can be used in two different syntactical construction: a) If 'class' keyword immediately followed by identifier, then it is usual class definition. b) In all other cases (inside a function) 'class' keyword denotes value of implicit function argument >>> # in Python 3.0: >>> def sample(a, b): ... class C: pass # ordinary class definition ... print class.__name__ # 'class' is implicit argument ... 4. Each function (not only class methods) accepts two implicit arguments, 'class' and 'self'. With ordinary function call, this arguments has 'None' value >>> def f(a, b): ... return [class, self, a, b] ... >>> f(1, 2) [None, None, 1, 2] Implicit 'class' and 'self' attributes don't participates in partial function application 5. Each function have two constant attributes, __class__ and __self__, both of them have value 'None' >>> f.__class__, f.__self__ (None, None) 6. New builtin function bind(func, self_, [class_]) is introduced. The result is a new function with the same name, __self__ and __class__ attributes of which are equal to self_ and class_, correspondly. If bind function is called without class_ argument, then __class__ value in new function remains the same as in previous ones. >>> f >>> g = bind(f, 'foo', 'bar') >>> g >>> g.__class__, g.__self__ ('bar', 'foo') When function is called, its __class__ and __self__ attributes are accessible inside function body as 'class' and 'self': >>> g(10, 20) ['foo', 'bar', 10, 20] Once bounded function can be rebinded easily: >>> bind(g, 100, 200) 7. In process of function extraction from class or instance dictionary it became bound automatically (the machinery is in object.__getattribute__ and type.__getattribute__ methods): >>> def func(a, b): ... return [class, self, a, b] ... >>> def Class: ... func = func ... >>> instance = Class() >>> func(10, 20) [None, None, 10, 20] >>> Class.func(10, 20) [, None, 10, 20] >>> instance.func(10, 20) [, <__main__.Class object at 0x016434F0>, 10, 20] More examples with metaclasses: >>> def func(a, b): ... return [class, self, a, b] ... >>> class Meta(type): ... func = func ... >>> class Class: ... __metaclass__ = Meta ... >>> instance = Class() >>> func(10, 20) [None, None, 10, 20] >>> Meta.func(10, 20) [, None, 10, 20] >>> Class.func(10, 20) [, , 10, 20] >>> instance.func(10, 20) Traceback (most recent call last): ... AttributeError: 'Class' object has no attribute 'func' 9. Methods can be overloaded this way: >>> class MyList(list): ... def __getitem__(key): ... print "overloaded method!" ... return super(MyList, self).__getitem__(key) ... def __setitem__(key, value): ... print "overloaded without super" ... bind(list.__setitem__, self)(key, value) ... 10. It is interesting that with this semantics there are no need in @classmethod and @staticmethod decorators: - If you want to create class method, simply don't use 'self' argument - If you want to create static method, don't use 'class' and 'self' argument >>> class Example: ... class_var = 100 ... def my_class_method(): # no need for @classmethod ... class.class_var += 1 ... def my_static_method(): # no need for @staticmethod ... print "hello!" ... def usual_method(): ... print .name ... What do you think about this? Best regards, Alexander mailto:kozlovsky at mail.spbnit.ru From martin at v.loewis.de Thu Jan 5 23:18:26 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 05 Jan 2006 23:18:26 +0100 Subject: [Python-Dev] buildbot In-Reply-To: <20060104.003058.48524852.warner@lothar.com> References: <20060104.003058.48524852.warner@lothar.com> Message-ID: <43BD9B32.3030700@v.loewis.de> Brian Warner wrote: > There was also a patch[1] to add some regexps to svn_buildbot.py to do this > kind of branch determination. I haven't been able to review it properly yet, > but it may give you some ideas. The patch itself is completely broken. It removes random parts of svn_buildbot.py, so after applying it, svn_buildbot isn't even syntactically correct anymore. However, I adjusted the patch, so it now works fine. (the other issue was that the regex was not entirely correct: it assumes a trunk/ structure, where we have a /trunk structure). So at the moment, I'm quite happy with buildbot. My only wish is that it would do static pages, rather than being a web server. I set it up as a reverse proxy (so nobody knows what the port number is, but still). Regards, Martin From martin at v.loewis.de Thu Jan 5 23:21:58 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 05 Jan 2006 23:21:58 +0100 Subject: [Python-Dev] buildbot In-Reply-To: <20060105022855.GB5183@activestate.com> References: <20060104.003058.48524852.warner@lothar.com> <20060105022855.GB5183@activestate.com> Message-ID: <43BD9C06.3080401@v.loewis.de> Trent Mick wrote: > Or for separate logic projects being built with the same builtbot > master. For example, say Python's buildbot wanted to do regular builds > and tests of the distutils tree > (http://svn.python.org/view/distutils/trunk/). I believe you could always get it arranged the way you like by properly setting up an "uninteresting changes" filter. You would have several builders, one change source, and each builder would filter out the changes it is interested in. Regards, Martin From alexander.kozlovsky at gmail.com Fri Jan 6 00:34:23 2006 From: alexander.kozlovsky at gmail.com (Alexander Kozlovsky) Date: Fri, 6 Jan 2006 02:34:23 +0300 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <611494241.20060106005601@gmail.com> References: <611494241.20060106005601@gmail.com> Message-ID: <1542680550.20060106023423@gmail.com> I wrote: > 5. Each function have two constant attributes, __class__ and __self__, > both of them have value 'None' Of course, this attributes have names 'im_class' and 'im_self', as before, but can be used with any function. I have not sleep enough last night :) Best regards, Alexander mailto:alexander.kozlovsky at gmail.com From martin at v.loewis.de Fri Jan 6 01:01:00 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 06 Jan 2006 01:01:00 +0100 Subject: [Python-Dev] [Buildbot-devel] Re: buildbot In-Reply-To: <6db0ea510601051432g5791c209j57fc6810664a7f25@mail.gmail.com> References: <20060104.003058.48524852.warner@lothar.com> <20060105022855.GB5183@activestate.com> <43BD9C06.3080401@v.loewis.de> <6db0ea510601051432g5791c209j57fc6810664a7f25@mail.gmail.com> Message-ID: <43BDB33C.5060203@v.loewis.de> Trent Mick wrote: > I meant that more as a justification for improving the Waterfall > status receiver to support separate summary pages for separate > projects and trunks... all with the same buildbot master server. > python.org/dev/buildbot/python/... > python.org/dev/buildbot/python-release24-maint/... > python.org/dev/buildbot/distutils/... Ah, right. On result presentation, there is definitely room for improvement. Regards, Martin From martin at v.loewis.de Fri Jan 6 01:33:53 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 06 Jan 2006 01:33:53 +0100 Subject: [Python-Dev] buildno (Was: [Python-checkins] commit of r41907- python/trunk/Makefile.pre.in) In-Reply-To: <1136468298.15418.43.camel@geddy.wooz.org> References: <20060103143057.4DB8E1E401C@bag.python.org> <43BB9562.2070404@acm.org> <200601050238.51766.anthony@interlink.com.au> <1136406410.10342.36.camel@geddy.wooz.org> <43BC5C08.1010600@v.loewis.de> <1136468298.15418.43.camel@geddy.wooz.org> Message-ID: <43BDBAF1.2040506@v.loewis.de> Barry Warsaw wrote: > I was working on a patch to add a PY_BUILDNO macro to > Include/patchlevel.h, which would have "$Revision$" as its value. As you can see, this is done now. The value appears at the Python level only for tags, though, because it will be unreliable for the trunk and for branches. > patchlevel.h seems like the natural place to put this, since any release > manager is going to be modifying this file anyway. PEP 101 should be > updated so that updating patchlevel.h be the last commit before an svn > export is done (but that PEP needs an overhaul anyway to change the cvs > references into svn commands). It would still be one level behind: patchlevel.h gets N, then 'svn cp' creates the tag, producing N+1. OTOH, for a tag, the revision number is nearly irrelevant. Regards, Martin From ncoghlan at gmail.com Fri Jan 6 01:55:29 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 06 Jan 2006 10:55:29 +1000 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <611494241.20060106005601@gmail.com> References: <611494241.20060106005601@gmail.com> Message-ID: <43BDC001.3070602@gmail.com> Alexander Kozlovsky wrote: > Hello! > > I have some proposal for Python 3.0 (interesting one, from my point > of view). I'm sorry for my English, it is not very good. Your English seems fine. About the only thing I noticed is that you have the meaning of 'function arguments' vs 'function parameters' switched from a Python point of view - the parameters are defined as part of the function definition, while the arguments are provided at call time. This is really a minor semantic quibble though - I expect most people wouldn't have any real trouble figuring out what you meant. Even though I still disagree with it, this is one of the better reasoned "no explicit self" proposals I've encountered - even if Guido ends up not liking it, I believe it should still be recorded as a PEP on python.org. To sum the proposal up in my own words: Eliminate the need for explicit class and self slots in class and instance methods by implicitly providing those slots on all functions. The main concern I have is with the answer to the question "How many positional arguments does the function have if I retrieve it from the class, rather than from an instance?" (this is the common concern for almost all proposals to remove the explicit self and class_ slots). That is, the rationale for requiring the explicit self is an outgrowth of the fact that methods can be retrieved directly from the class: class Foo: def __init__(self, x): # 1: Explicit 'self' argument self.x = x # 2: 'self' must be used explicitly def bar(self, a, b): # 3: There are three parameters... print self.x + a + b f = Foo.bar # We retrieve the unbound method f(Foo(10), 20, 30) # And there are three arguments at call time f = Foo().bar # Using an instance binds the first argument f(20, 30) # Which means there are two arguments left You can also bypass the type machinery entirely, and retrieve the raw function object from the class dictionary: f = Foo.__dict__["bar"] # This gives a function. . . f(Foo(10), 20, 30) # which takes three arguments as written Under the proposal being discussed, things become far less clear: class Foo: def __init__(x): # 1: Implicit self .x = x # 2: Brief form of: self.x = x def bar(a, b): # 3: Two arguments... print .x + a + b f = Foo(10).bar # We agree this accepts 2 arguments f = Foo.bar # How many arguments does f accept? f = Foo.__dict__["bar"] # How many arguments does it accept now? The answer to the first question *has* to be 3, or we lose too much functionality - but that's seriously surprising (the method appears to require two arguments when its defined, but actually requires 3 due to its being retrieved from a class). And it still requires that a distinction be made between instance, class and static methods in order to control the signature of the retrieved method. However, that answer creates its own problems - what if we have a 3-argument function that does exactly what we want our method to do? We'd need some way of signalling to the class that the function should be left alone when being retrieved from the class, but have the first argument bound automatically when being retrieved from an instance. This is where the "Explicit is better than implicit" and "Practicality beats purity" *both* kick in in favour of explicit self and class_ parameters - the signature of the retrieved function is exactly what the source code says it is, because there aren't any implicit parameters getting slipped into the parameter list when you aren't looking. As I see it, the real issue is that people are often coming from a Java or C++ background where you can't manipulate functions and classes as first-class objects - the idea that the instance method signature could be written to describe the signature of the unbound method returned by "Foo.bar" rather than the bound method returned by "Foo().bar" is an entirely foreign concept. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From barry at python.org Fri Jan 6 02:59:01 2006 From: barry at python.org (Barry Warsaw) Date: Thu, 05 Jan 2006 20:59:01 -0500 Subject: [Python-Dev] buildno (Was: [Python-checkins] commit of r41907- python/trunk/Makefile.pre.in) In-Reply-To: <43BDBAF1.2040506@v.loewis.de> References: <20060103143057.4DB8E1E401C@bag.python.org> <43BB9562.2070404@acm.org> <200601050238.51766.anthony@interlink.com.au> <1136406410.10342.36.camel@geddy.wooz.org> <43BC5C08.1010600@v.loewis.de> <1136468298.15418.43.camel@geddy.wooz.org> <43BDBAF1.2040506@v.loewis.de> Message-ID: <1136512741.15418.86.camel@geddy.wooz.org> On Fri, 2006-01-06 at 01:33 +0100, "Martin v. L?wis" wrote: > As you can see, this is done now. The value appears at the Python > level only for tags, though, because it will be unreliable for the > trunk and for branches. Cool, thanks. I can chuck my local diffs now. :) > > patchlevel.h seems like the natural place to put this, since any release > > manager is going to be modifying this file anyway. PEP 101 should be > > updated so that updating patchlevel.h be the last commit before an svn > > export is done (but that PEP needs an overhaul anyway to change the cvs > > references into svn commands). > > It would still be one level behind: patchlevel.h gets N, then 'svn cp' > creates the tag, producing N+1. OTOH, for a tag, the revision number > is nearly irrelevant. Unless we tagged and then modified the file in that tag as the very last thing we do before we create the tarball. Or is that too evil? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060105/cf8ba7be/attachment.pgp From tim.peters at gmail.com Fri Jan 6 04:10:36 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 5 Jan 2006 22:10:36 -0500 Subject: [Python-Dev] Buildbot questions In-Reply-To: References: <43BB0965.5080301@masklinn.net> <1f7befae0601050743p32e542f4gb899153b2d3ae94c@mail.gmail.com> Message-ID: <1f7befae0601051910g189d9689te2e6f9532cc3d5b3@mail.gmail.com> [John J Lee] > Might a buildbot running this setup of David Munman's (free MS compiler + > NAnt interpreting the MS project file) be useful? > > http://groups.google.co.uk/group/comp.lang.python/browse_frm/thread/226584dd47047bb6/1e33ad19197bee20 No comment from me about that (don't know anything about it, and don't have time to learn). Someone who wants to see a platform exercised needs to volunteer that platform themself (and realize that every stinkin' little step has to be so well defined that the people running the buildbot _master_ can send the exact steps needed to the slave). > ... > Much though I like SVN, it seems its working copy management still leaves > a little to be desired: Even quite recently (fairly sure it was client > version 1.2.*, on Win XP) and with read-only checkouts used only for > builds, I've still seen it end up in an incorrect state. I suspect 'svn > switch' or 'svn up -r xxxxx' was the culprit, though, so perhaps it's not > a problem if exactly 'svn up' is the only svn command ever executed on the > checkout. I doubt anyone slings more svn projects, branches and tags than Zope Corp, and I've had no problems with svn on WinXP there except when a project switches from making copies of externals to getting them via svn:externals instead -- and then everyone has problems, regardless of platform. What I _have_ had problems with is PuTTY, and recently discovered that all my months-long "svn+ssh" problems went away after backing off to the older PuTTY 0.57 (and come back again immediately upon switching to 0.58). > Still, perhaps it's wise to wipe the checkout every so often? I think it is. And while I haven't seen this under MS VC7.1 yet, a few times I caught VC 6.0 failing to recompile after a relevant header file changed. Certainly from-scratch checkout + build should be done before a release. From trentm at gmail.com Thu Jan 5 23:32:14 2006 From: trentm at gmail.com (Trent Mick) Date: Thu, 5 Jan 2006 14:32:14 -0800 Subject: [Python-Dev] [Buildbot-devel] Re: buildbot In-Reply-To: <43BD9C06.3080401@v.loewis.de> References: <20060104.003058.48524852.warner@lothar.com> <20060105022855.GB5183@activestate.com> <43BD9C06.3080401@v.loewis.de> Message-ID: <6db0ea510601051432g5791c209j57fc6810664a7f25@mail.gmail.com> > > Or for separate logic projects being built with the same builtbot > > master. For example, say Python's buildbot wanted to do regular builds > > and tests of the distutils tree > > (http://svn.python.org/view/distutils/trunk/). > > I believe you could always get it arranged the way you like by properly > setting up an "uninteresting changes" filter. You would have several > builders, one change source, and each builder would filter out the > changes it is interested in. I meant that more as a justification for improving the Waterfall status receiver to support separate summary pages for separate projects and trunks... all with the same buildbot master server. python.org/dev/buildbot/python/... python.org/dev/buildbot/python-release24-maint/... python.org/dev/buildbot/distutils/... Trent -- Trent Mick trentm at gmail.com From kay.schluehr at gmx.net Fri Jan 6 07:26:36 2006 From: kay.schluehr at gmx.net (Kay Schluehr) Date: Fri, 06 Jan 2006 07:26:36 +0100 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <611494241.20060106005601@gmail.com> References: <611494241.20060106005601@gmail.com> Message-ID: Just one short comment of your ( well written ) rationale and PEP. Ruby might be hyped at the moment but at least in the german labour market and the biggest german freelancer mediation Ruby is quite inexistent. Even when the nerdish blogosphere is setting different priorities SAP and Intel made million $ investments in Zend Corp. and PHP but not in RoR. The slashdot sociology is not all important. If its just about rhetorical battles, hype and popularity Pythonistas would do a better service for themselves not to change Pythons object model but write a J2EE-like spec, standardize it and provide a reference implementation ( "THE agile alternative!... "). Kay From martin at v.loewis.de Fri Jan 6 08:27:52 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 06 Jan 2006 08:27:52 +0100 Subject: [Python-Dev] buildno (Was: [Python-checkins] commit of r41907- python/trunk/Makefile.pre.in) In-Reply-To: <1136512741.15418.86.camel@geddy.wooz.org> References: <20060103143057.4DB8E1E401C@bag.python.org> <43BB9562.2070404@acm.org> <200601050238.51766.anthony@interlink.com.au> <1136406410.10342.36.camel@geddy.wooz.org> <43BC5C08.1010600@v.loewis.de> <1136468298.15418.43.camel@geddy.wooz.org> <43BDBAF1.2040506@v.loewis.de> <1136512741.15418.86.camel@geddy.wooz.org> Message-ID: <43BE1BF8.3020606@v.loewis.de> Barry Warsaw wrote: >>It would still be one level behind: patchlevel.h gets N, then 'svn cp' >>creates the tag, producing N+1. OTOH, for a tag, the revision number >>is nearly irrelevant. > > > Unless we tagged and then modified the file in that tag as the very last > thing we do before we create the tarball. Or is that too evil? That would work, and I wouldn't see anything wrong with it. I believe it would also work to modify the working copy, and then svn cp it (instead of svn committing it) - atleast the svn docs suggest that you can copy a working copy into a repo URL. Regards, Martin From martin at v.loewis.de Fri Jan 6 08:39:15 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 06 Jan 2006 08:39:15 +0100 Subject: [Python-Dev] Buildbot questions In-Reply-To: <1f7befae0601051910g189d9689te2e6f9532cc3d5b3@mail.gmail.com> References: <43BB0965.5080301@masklinn.net> <1f7befae0601050743p32e542f4gb899153b2d3ae94c@mail.gmail.com> <1f7befae0601051910g189d9689te2e6f9532cc3d5b3@mail.gmail.com> Message-ID: <43BE1EA3.8040405@v.loewis.de> Tim Peters wrote: >>Still, perhaps it's wise to wipe the checkout every so often? > > > I think it is. And while I haven't seen this under MS VC7.1 yet, a > few times I caught VC 6.0 failing to recompile after a relevant > header file changed. Certainly from-scratch checkout + build should > be done before a release. I would like to do this in buildbot, but I'm not sure how to (i.e. wipe the build occasionally, but not every time). For example, I could imagine completely cleaning the build directory every time the build number % 10 == 0. Still, what the precise buildbot incantation would be, I don't know. Regards, Martin From anthony at interlink.com.au Fri Jan 6 09:16:32 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri, 6 Jan 2006 19:16:32 +1100 Subject: [Python-Dev] Buildbot questions In-Reply-To: <43BE1EA3.8040405@v.loewis.de> References: <43BB0965.5080301@masklinn.net> <1f7befae0601051910g189d9689te2e6f9532cc3d5b3@mail.gmail.com> <43BE1EA3.8040405@v.loewis.de> Message-ID: <200601061916.38944.anthony@interlink.com.au> On Friday 06 January 2006 18:39, Martin v. L?wis wrote: > I would like to do this in buildbot, but I'm not sure how to > (i.e. wipe the build occasionally, but not every time). > > For example, I could imagine completely cleaning the build > directory every time the build number % 10 == 0. Still, what the > precise buildbot incantation would be, I don't know. At least with the way Twisted is set up, the buildbot also sits in an IRC channel and sends updates there. It can also be controlled from there. Is this worth doing? A 'force clean build' command could be added... Anthony -- Anthony Baxter It's never too late to have a happy childhood. From mwh at python.net Fri Jan 6 12:34:35 2006 From: mwh at python.net (Michael Hudson) Date: Fri, 06 Jan 2006 11:34:35 +0000 Subject: [Python-Dev] Automated Python testing In-Reply-To: <43B308E0.20205@v.loewis.de> ( =?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Wed, 28 Dec 2005 22:51:28 +0100") References: <20051228194543.1217.425460361.divmod.quotient.8597@ohm> <43B308E0.20205@v.loewis.de> Message-ID: <2mmzi9zj1w.fsf@starship.python.net> "Martin v. L?wis" writes: > Currently, my buildbot isn't connected to IRC at all. If I ever > enable that aspect, I'll use allowForce=False again to disable > remotely invoking builds. #python-dev on freenode is ready and waiting should you decide to activate this :) Cheers, mwh -- well, take it from an old hand: the only reason it would be easier to program in C is that you can't easily express complex problems in C, so you don't. -- Erik Naggum, comp.lang.lisp From skink at evhr.net Fri Jan 6 13:10:41 2006 From: skink at evhr.net (Fabien Schwob) Date: Fri, 06 Jan 2006 13:10:41 +0100 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <611494241.20060106005601@gmail.com> References: <611494241.20060106005601@gmail.com> Message-ID: <43BE5E41.10301@evhr.net> > Example 1 (Python 2.x): > ----------------------- > > class Foo: > def __init__(self, x): # 1: Explicit 'self' argument > self.x = x # 2: 'self' must be used explicitly > def bar(self, a, b): # 3: There are three arguments... > print self.x + a + b > > Foo(10).bar(20, 30) # ...but only two explicit parameters > # is presented > > This document proposes to change this, as the next example shows: > > Example 2 (Python 3.0): > ----------------------- > > class Foo: > def __init__(x): # 1: Implicit self > .x = x # 2: Brief form of: self.x = x > def bar(a, b): # 3: Two arguments... > print .x + a + b > > Foo(10).bar(20, 30) # ...and exactly two parameters In my case, I think that the problem of _self_ is mainly in the method definition. It's a little "hard" to understand why you have to use myFunction(self, otherArgs) when you create a class method. But the use of self in the code of the method is a good thing because it allow you to clearly say that you are working on a class property. In my case, I would like to have the following syntax in Python 3.0 : class Foo: def __init__(x): self.x = x def bar(a, b): print self.x + a + b My 0.2? ;) -- Fabien SCHWOB From ianb at colorstudy.com Fri Jan 6 18:14:28 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 06 Jan 2006 11:14:28 -0600 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <43BDC001.3070602@gmail.com> References: <611494241.20060106005601@gmail.com> <43BDC001.3070602@gmail.com> Message-ID: <43BEA574.8010607@colorstudy.com> Nick Coghlan wrote: [...] > Under the proposal being discussed, things become far less clear: > > class Foo: > def __init__(x): # 1: Implicit self > .x = x # 2: Brief form of: self.x = x > def bar(a, b): # 3: Two arguments... > print .x + a + b > > f = Foo(10).bar # We agree this accepts 2 arguments > f = Foo.bar # How many arguments does f accept? > f = Foo.__dict__["bar"] # How many arguments does it accept now? I think this might return things very similar to what are returned now. The only reasonable alternative I can think of is that Foo.bar would return something that simply could not be called until it was bound, so this: inst = Foo() f = Foo.bar print f(inst, 1, 2) would have to be translated to this this: inst = Foo() f = Foo.bar meth = bind(f, inst) print meth(1, 2) It seems clear given the rest of the proposal that this second form will work, it's just a question of whether the first form will work at all. Personally I find the first form is rather uncommon anyway, and 50% of the time I write it it's a bug. So the more explicit second form would be fine with me. Well... it becomes more complex for decorators, I guess: def printargs(func): def replacement(*args, **kw): print args, kw return func(*args, **kw) return replacement class Foo: @printargs def null(a): pass What is printargs going to print? Will it even work? I'd guess you'd have to do: def printargs(func): def replacement(*args, **kw): print args, kw return bind(func, self)(*args, **kw) return replacement But what does that do if you apply printargs to a function that isn't a method? I generally have found it awkard to apply decorators to both methods and functions, except when the decorators pass through all the arguments unchanged (as printargs does). But I don't know if this is an improvement; I guess it depends on what bind(func, self) does outside of a method invocation. I think self should be undefined in that case. Well, I guess you could do: def printargs(func): def replacement(*args, **kw): print args, kw try: bound = bind(func, self, class) except NameError: try: bound = bind(func, None, class) except NameError: bound = func return bound(*args, **kw) return replacement Ugh. > The answer to the first question *has* to be 3, or we lose too much > functionality - but that's seriously surprising (the method appears to require > two arguments when its defined, but actually requires 3 due to its being > retrieved from a class). And it still requires that a distinction be made > between instance, class and static methods in order to control the signature > of the retrieved method. > > However, that answer creates its own problems - what if we have a 3-argument > function that does exactly what we want our method to do? We'd need some way > of signalling to the class that the function should be left alone when being > retrieved from the class, but have the first argument bound automatically when > being retrieved from an instance. > > This is where the "Explicit is better than implicit" and "Practicality beats > purity" *both* kick in in favour of explicit self and class_ parameters - the > signature of the retrieved function is exactly what the source code says it > is, because there aren't any implicit parameters getting slipped into the > parameter list when you aren't looking. > > As I see it, the real issue is that people are often coming from a Java or C++ > background where you can't manipulate functions and classes as first-class > objects - the idea that the instance method signature could be written to > describe the signature of the unbound method returned by "Foo.bar" rather than > the bound method returned by "Foo().bar" is an entirely foreign concept. I disagree; I'm 100% comfortable and have internalized Python's use of an explicit self parameter, but it still doesn't feel more explicit or practical than the traditional way of writing out methods in other languages. I'm inclined to say that this might be too substantial of a change to Python, but I don't think the current way method definitions work is particular elegant, and it hasn't really grown on me over all these years. I still frequently spell method signatures incorrectly. And I can't explain the current situation to a new user, except to say "this is just the way it is", because the explanation only makes sense when you are much further into the language. And don't get me started on the error messages when you get the parameter count wrong... (As an aside directed at the original PEP, I think discussion of leaving self out of expressions, e.g., ".x" for "self.x", should be separate from the rest of this PEP). -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From trentm at ActiveState.com Fri Jan 6 18:31:10 2006 From: trentm at ActiveState.com (Trent Mick) Date: Fri, 6 Jan 2006 09:31:10 -0800 Subject: [Python-Dev] Buildbot questions In-Reply-To: <43BE1EA3.8040405@v.loewis.de> References: <43BB0965.5080301@masklinn.net> <1f7befae0601050743p32e542f4gb899153b2d3ae94c@mail.gmail.com> <1f7befae0601051910g189d9689te2e6f9532cc3d5b3@mail.gmail.com> <43BE1EA3.8040405@v.loewis.de> Message-ID: <20060106173110.GD15276@activestate.com> [Martin v. Loewis wrote] > I would like to do this in buildbot, but I'm not sure how to > (i.e. wipe the build occasionally, but not every time). > > For example, I could imagine completely cleaning the build directory > every time the build number % 10 == 0. Still, what the precise > buildbot incantation would be, I don't know. (Still learning my buildbot mojo.) One idea would be to do what Mozilla's Tinderbox does: they have one set of builds that are incremental and one set that are full. Actually looking around on tinderbox.mozilla.org I could only find incremental builds so I'm not sure what they are doing. To wipe out the build occassionally you could (presumably) add a starting step to the Python 'builder' (in the build master.cfg) to rm -rf $builddir every, say, Sunday night. Trent -- Trent Mick trentm at activestate.com From Martin.vonLoewis at hpi.uni-potsdam.de Fri Jan 6 19:40:07 2006 From: Martin.vonLoewis at hpi.uni-potsdam.de (=?iso-8859-1?Q?von_L=F6wis_Martin?=) Date: Fri, 6 Jan 2006 19:40:07 +0100 Subject: [Python-Dev] Compiler warnings for 64-bit portability problems Message-ID: <5AC0F51A926CD442BC08D55DA7DAB1D40129F4AB@3mxma1r.hpi.uni-potsdam.de> I just found that the intel compiler (icc 9.0) also supports compiler warnings for portability problems. For the file #include int foo(size_t x) { return x; } it says (with -Wall) a.c(3): remark #1418: external definition with no prior declaration int foo(size_t x) ^ a.c(5): remark #1682: implicit conversion of a 64-bit integral type to a smaller integral type (potential portability problem) return x; ^ If you just want the latter warning, pass -Wp64. Regards, Martin P.S. In case you want to install it on a Debian AMD64 system, here is the procedure: 1. register at https://registrationcenter.intel.com/EvalCenter/EvalForm.aspx?ProductID=408 2. download and unpack the l_cc_c_9.0.030 tarfile. 3. don't try alien, it won't work. 4. don't try the Intel install.sh, it won't work (atleast, it didn't work for me) 5. Instead, install with 'rpm -i --nodeps *.em64t.rpm' 6. edit /opt/intel/cce/9.0/bin/icc, replacing all occurrences of with /opt/intel/cce/9.0 7. Put the *.lic file that Intel sent you into /opt/intel/cce/9.0/bin From jimjjewett at gmail.com Fri Jan 6 20:14:21 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri, 6 Jan 2006 14:14:21 -0500 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 Message-ID: Nick Coghlan wrote: > Eliminate the need for explicit class and self > slots in class and instance methods by > implicitly providing those slots on all functions. > How many positional arguments does the function > have if I retrieve it from the class, rather than from > an instance? To keep things simple, it should have the same number of positional arguments no matter how you retrieve it -- think of self and cls as keyword arguments that we currently write in an odd manner. > class Foo: > def __init__(x): # 1: Implicit self > .x = x # 2: Brief form of: self.x = x > def bar(a, b): # 3: Two arguments... > print .x + a + b > f = Foo(10).bar # We agree this accepts 2 arguments > f = Foo.bar # How many arguments does f accept? > f = Foo.__dict__["bar"] # How many arguments does it accept now? > The answer to the first question *has* to be 3, > or we lose too much functionality No, it should be 2. If you use it as a staticmethod, there isn't any problem. If you really want a method outside of its context, then pass self (or cls) as a keyword. f(4,5,self=Foo(10)) (I also think we could use class as the keyword, since class=expression is different from class Name: but that is a separate question.) If you like having self at the front -- well, wasn't the "keywords must be last" restriction likely to go away even before python 3.0? > ... a Java or C++ background where > you can't manipulate functions and classes as > first-class objects - the idea that the instance > method signature could be written to describe > the signature of the unbound method returned > by "Foo.bar" rather than the bound method > returned by "Foo().bar" is an entirely foreign > concept. I find it sort of strange that the signatures are different in the first place. I prefer the one without boilerplate. -jJ From arigo at tunes.org Fri Jan 6 20:34:48 2006 From: arigo at tunes.org (Armin Rigo) Date: Fri, 6 Jan 2006 20:34:48 +0100 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <611494241.20060106005601@gmail.com> References: <611494241.20060106005601@gmail.com> Message-ID: <20060106193448.GA656@code1.codespeak.net> Hi Alexander, On Fri, Jan 06, 2006 at 12:56:01AM +0300, Alexander Kozlovsky wrote: > There are three different peculiarity in Python 2.x > in respect of 'self' method argument: Yuk! This has been discussed again and again already. *Please* move this discussion to comp.lang.python. A bientot, Armin From alexander.kozlovsky at gmail.com Fri Jan 6 21:26:55 2006 From: alexander.kozlovsky at gmail.com (Alexander Kozlovsky) Date: Fri, 6 Jan 2006 23:26:55 +0300 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 Message-ID: <1325280794.20060106232655@gmail.com> Hello! Ian Bicking wrote: > (As an aside directed at the original PEP, I think discussion of leaving > self out of expressions, e.g., ".x" for "self.x", should be separate > from the rest of this PEP). Yes, I'm fully agree. Nick Coghlan wrote: > The main concern I have is with the answer to the question "How many > positional arguments does the function have if I retrieve it from the class, > rather than from an instance?" (this is the common concern for almost all > proposals to remove the explicit self and class_ slots). In this case, the function always (!!!) has fixed number of positional arguments equal to number of positional parameters inside the function definition: >>> def func(a, b): ... print class, self, a, b ... >>> class Foo: ... bar = func ... >>> instance = Foo() >>> func(1, 2) # there are 2 arguments None None 1 2 # <- class and self both equal to None >>> Foo.bar(1, 2) # 2 arguments again!! None 1 2 # <- self is equal to None >>> instance.bar(1, 2) # 2 arguments always!!! 1 2 Nick Coghlan wrote: > To sum the proposal up in my own words: > Eliminate the need for explicit class and self slots in class > and instance methods by implicitly providing those slots on all > functions. Yes, I think, it is what I mean. With my proposal, 'self' is no longer the first explicit or implicit positional argument (!), instead, it is 'pseudo-argument', the value of which is equal to function's 'im_self' attribute. Any function contains two special read-only attributes: 'im_class' and 'im_self', and values of this attributes are accessible inside function body as values of 'class' and 'self' pseudo-arguments. Any function has this attributes, for ordinary function their values are set to 'None'. Example: >>> # Python 3.0 >>> def f(): return class, self >>> f() (None, None) There is new built-in function: bind(old_function, self_, class_=None) -> new_function The result is the new function with the same code object which the old function had, but with different im_class and im_self attributes values. - If both self_ and class_ arguments are None, then 'im_class' and 'im_self' attributes of new function is set to equal to None. - If self_ is not None, then class_ must be None or self_.__class__. 'im_self' attribute of new function is set to self_, and im_class attribute is set to self_.__class__ - If self_ is None but class_ is not None, then 'im_self' attribute is set to None, and 'im_class' attribute is set to class_ Consider this expression (extraction of a method from a class): x = Foo.bar This expression is equivalent of: x = bind(Foo.__dict__["bar"], None, Foo) After this, x.im_class is Foo and x.im_self is None The type.__getattribute__ contains implementation of this Consider next expression (extraction of a method from an instance): y = instance.bar This expression is equivalent of: y = bind(instance.__class__.__dict__["bar"], instance) After this, y.im_class is instance.__class__ and y.im_self is instance. The object.__getattribute__ is responsible for this Ian Bicking wrote: > Well... it becomes more complex for decorators, I guess: > > def printargs(func): > def replacement(*args, **kw): > print args, kw > return func(*args, **kw) > return replacement > class Foo: > @printargs > def null(a): pass > > What is printargs going to print? Will it even work? I'd guess you'd > have to do: > > def printargs(func): > def replacement(*args, **kw): > print args, kw > return bind(func, self)(*args, **kw) > return replacement I think, it should be: def printargs(func): def replacement(*args, **kw): print args, kw return bind(func, self, class)(*args, **kw) return replacement Yep, the code in decorators will be more complicated than it is today. I did not get it before... > I guess it depends on what bind(func, self) does outside of > a method invocation. I think self should be undefined in that case. 'self' will be None in that case. Any function has 'self' pseudo-argument, but in a plain function (not a method) 'self' and 'class' both equal to None. -- Best regards, Alexander mailto:alexander.kozlovsky at gmail.com From guido at python.org Fri Jan 6 22:19:16 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 6 Jan 2006 13:19:16 -0800 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <20060106193448.GA656@code1.codespeak.net> References: <611494241.20060106005601@gmail.com> <20060106193448.GA656@code1.codespeak.net> Message-ID: On 1/6/06, Armin Rigo wrote: > On Fri, Jan 06, 2006 at 12:56:01AM +0300, Alexander Kozlovsky wrote: > > There are three different peculiarity in Python 2.x > > in respect of 'self' method argument: > > Yuk! This has been discussed again and again already. *Please* move > this discussion to comp.lang.python. Yes please. This won't change. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Fri Jan 6 23:02:28 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 06 Jan 2006 23:02:28 +0100 Subject: [Python-Dev] Buildbot questions In-Reply-To: <20060106173110.GD15276@activestate.com> References: <43BB0965.5080301@masklinn.net> <1f7befae0601050743p32e542f4gb899153b2d3ae94c@mail.gmail.com> <1f7befae0601051910g189d9689te2e6f9532cc3d5b3@mail.gmail.com> <43BE1EA3.8040405@v.loewis.de> <20060106173110.GD15276@activestate.com> Message-ID: <43BEE8F4.5080304@v.loewis.de> Trent Mick wrote: > (Still learning my buildbot mojo.) One idea would be to do what > Mozilla's Tinderbox does: they have one set of builds that are > incremental and one set that are full. Actually looking around on > tinderbox.mozilla.org I could only find incremental builds so I'm not > sure what they are doing. I know how to set up full vs. incremental *builders*; they would appear in different lanes in the Waterfall, and they would operate on different build directories. I don't think I want additional lanes to the Waterfall. > To wipe out the build occassionally you could (presumably) add a > starting step to the Python 'builder' (in the build master.cfg) to > rm -rf $builddir > every, say, Sunday night. Sure, that would be the idea. How to formulate it? Regards, Martin From martin at v.loewis.de Sat Jan 7 00:01:32 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 07 Jan 2006 00:01:32 +0100 Subject: [Python-Dev] Buildbot questions In-Reply-To: <200601061916.38944.anthony@interlink.com.au> References: <43BB0965.5080301@masklinn.net> <1f7befae0601051910g189d9689te2e6f9532cc3d5b3@mail.gmail.com> <43BE1EA3.8040405@v.loewis.de> <200601061916.38944.anthony@interlink.com.au> Message-ID: <43BEF6CC.1020603@v.loewis.de> Anthony Baxter wrote: > At least with the way Twisted is set up, the buildbot also sits in an > IRC channel and sends updates there. It can also be controlled from > there. Is this worth doing? A 'force clean build' command could be > added... The problem with irc-enabling (or web-enabling) such things is that there is a potential for abuse. Of course, in this case, we could wait for the abuse to happen. Regards, Martin From martin at v.loewis.de Sat Jan 7 00:03:46 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 07 Jan 2006 00:03:46 +0100 Subject: [Python-Dev] Automated Python testing In-Reply-To: <2mmzi9zj1w.fsf@starship.python.net> References: <20051228194543.1217.425460361.divmod.quotient.8597@ohm> <43B308E0.20205@v.loewis.de> <2mmzi9zj1w.fsf@starship.python.net> Message-ID: <43BEF752.8010506@v.loewis.de> Michael Hudson wrote: > #python-dev on freenode is ready and waiting should you decide to > activate this :) Ok, I added "him"; "his" nick is py-bb. Commands include "hello", "status", "version". "force" is disabled. Regards, Martin From trentm at ActiveState.com Sat Jan 7 01:12:21 2006 From: trentm at ActiveState.com (Trent Mick) Date: Fri, 6 Jan 2006 16:12:21 -0800 Subject: [Python-Dev] Buildbot questions In-Reply-To: <43BEE8F4.5080304@v.loewis.de> References: <43BB0965.5080301@masklinn.net> <1f7befae0601050743p32e542f4gb899153b2d3ae94c@mail.gmail.com> <1f7befae0601051910g189d9689te2e6f9532cc3d5b3@mail.gmail.com> <43BE1EA3.8040405@v.loewis.de> <20060106173110.GD15276@activestate.com> <43BEE8F4.5080304@v.loewis.de> Message-ID: <20060107001221.GA13845@activestate.com> > > To wipe out the build occassionally you could (presumably) add a > > starting step to the Python 'builder' (in the build master.cfg) to > > rm -rf $builddir > > every, say, Sunday night. > > Sure, that would be the idea. How to formulate it? I think I'm part of the way there with the following. I've subclassed the "SVN" source build step to add support for new source mode: "update_and_clobber_occassionally". Basically it (hackily) changes the source type btwn "update", which we usually want, and "clobber", which we sometimes want. The current problem with this is that I don't know if the build step objects have access to a local data store -- so it could, say, count how many builds have been done to know to clobber every tenth one. The current code just chooses to clobber for any build on Sunday. Not a very satisfying solution. Brian, Is there a way for build steps to maintain some state? ------------ clipped from the build master's master.cfg ----------------- ... class SVNEx(step.SVN): """A SVN Source build step that adds the ability for the Source "mode" to be a combination of "update" and "clobber". This is useful if your general just want to update the source tree from source code control, but occassionally want to clobber and start fresh. To use this functionality use mode="update_and_clobber_occassionally". To control when "occassionally" is now override the "should_clobber" method. The default implementation is currently hardcoded to "every Sunday". (This is a HACK. Perhaps there should be constructor options to clobber every Nth time or to have cron-like arguments -- see "Nightly" in scheduler.py. I don't know if "steps" have access to a local data store to be able to do this -- e.g. keep a counter.) Ideally this would be an option of the base "Source" class in buildbot/process/step.py. """ def __init__(self, **kwargs): if kwargs.has_key("mode") \ and kwargs["mode"] == "update_and_clobber_occassionally": self.update_and_clobber_occassionally = True kwargs["mode"] = "update" else: self.update_and_clobber_occassionally = False step.SVN.__init__(self, **kwargs) def should_clobber(self): from datetime import date is_sunday = date.today().weekday() == 6 # it is Sunday from twisted.python import log log.msg("should_clobber? %s", (is_sunday and "yes" or "no")) return is_sunday def startVC(self, *args, **kwargs): if self.update_and_clobber_occassionally: if self.should_clobber(): self.args["mode"] = "clobber" else: self.args["mode"] = "update" step.SVN.startVC(self, *args, **kwargs) python_factory = factory.GNUAutoconf( s(SVNEx, svnurl="http://svn.python.org/projects/python/trunk", mode="update_and_clobber_occassionally"), test=["make", "test"], # use `make testall`? ) # Then use python_factory as something like so: #c['builders'] = [ # {'name': "linux-x86", # 'slavename': "...", # 'builddir': "python", # 'factory': python_factory, # }, # ... #] ------------ clipped from the build master's master.cfg ----------------- Cheers, Trent -- Trent Mick trentm at activestate.com From martin at v.loewis.de Sat Jan 7 01:13:53 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 07 Jan 2006 01:13:53 +0100 Subject: [Python-Dev] Buildbot questions In-Reply-To: <20060107001221.GA13845@activestate.com> References: <43BB0965.5080301@masklinn.net> <1f7befae0601050743p32e542f4gb899153b2d3ae94c@mail.gmail.com> <1f7befae0601051910g189d9689te2e6f9532cc3d5b3@mail.gmail.com> <43BE1EA3.8040405@v.loewis.de> <20060106173110.GD15276@activestate.com> <43BEE8F4.5080304@v.loewis.de> <20060107001221.GA13845@activestate.com> Message-ID: <43BF07C1.3060608@v.loewis.de> Trent Mick wrote: > I think I'm part of the way there with the following. I've subclassed > the "SVN" source build step to add support for new source mode: > "update_and_clobber_occassionally". Basically it (hackily) changes the > source type btwn "update", which we usually want, and "clobber", which > we sometimes want. Ah, but that would require changes to the slaves, right? I would prefer a solution that avoids that. Regards, Martin From trentm at ActiveState.com Sat Jan 7 01:16:51 2006 From: trentm at ActiveState.com (Trent Mick) Date: Fri, 6 Jan 2006 16:16:51 -0800 Subject: [Python-Dev] Buildbot questions In-Reply-To: <43BF07C1.3060608@v.loewis.de> References: <43BB0965.5080301@masklinn.net> <1f7befae0601050743p32e542f4gb899153b2d3ae94c@mail.gmail.com> <1f7befae0601051910g189d9689te2e6f9532cc3d5b3@mail.gmail.com> <43BE1EA3.8040405@v.loewis.de> <20060106173110.GD15276@activestate.com> <43BEE8F4.5080304@v.loewis.de> <20060107001221.GA13845@activestate.com> <43BF07C1.3060608@v.loewis.de> Message-ID: <20060107001651.GB13845@activestate.com> [Martin v. Loewis wrote] > Trent Mick wrote: > > I think I'm part of the way there with the following. I've subclassed > > the "SVN" source build step to add support for new source mode: > > "update_and_clobber_occassionally". Basically it (hackily) changes the > > source type btwn "update", which we usually want, and "clobber", which > > we sometimes want. > > Ah, but that would require changes to the slaves, right? I would > prefer a solution that avoids that. I don't think so. In my little test setup I didn't have to make any change to the slave. Trent -- Trent Mick trentm at activestate.com From murman at gmail.com Sat Jan 7 04:11:34 2006 From: murman at gmail.com (Michael Urman) Date: Fri, 6 Jan 2006 21:11:34 -0600 Subject: [Python-Dev] New PEP: Using ssize_t as the index type In-Reply-To: <43BD8464.4060801@v.loewis.de> References: <43B3ECEE.9060300@v.loewis.de> <20051229173041.GA19575@code1.codespeak.net> <43B50B64.20904@v.loewis.de> <20060105194538.GB11736@code1.codespeak.net> <43BD8464.4060801@v.loewis.de> Message-ID: [I just noticed that I sent this mail to just Martin when I meant it for the list. Sorry Martin!] On 1/5/06, "Martin v. L?wis" wrote: > More precisely, the printf style of function calling, and varargs > functions. ISO C is pretty type safe, but with varargs functions, > you lose that completely. > > I still hope I can write a C parser some day that does > ParseTuple/BuildValue checking. I put together a non-parsing checker last month to help me feel more secure after http://python.org/sf/1365916. It's awful code, but the simple things are easy to change or extend. Fixing the false positives and other misinterpretations is harder. http://www.tortall.net/mu/static/fmtcheck.py?raw - it takes a list of directories to os.walk for c source files. Michael -- Michael Urman http://www.tortall.net/mu/blog From nnorwitz at gmail.com Sat Jan 7 04:18:37 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Fri, 6 Jan 2006 19:18:37 -0800 Subject: [Python-Dev] New PEP: Using ssize_t as the index type In-Reply-To: References: <43B3ECEE.9060300@v.loewis.de> <20051229173041.GA19575@code1.codespeak.net> <43B50B64.20904@v.loewis.de> <20060105194538.GB11736@code1.codespeak.net> <43BD8464.4060801@v.loewis.de> Message-ID: On 1/6/06, Michael Urman wrote: > > I put together a non-parsing checker last month to help me feel more > secure after http://python.org/sf/1365916. It's awful code, but the > simple things are easy to change or extend. Fixing the false positives > and other misinterpretations is harder. Perhaps you could use Synopsis to parse the C code? http://synopsis.fresco.org/ I'm hoping to look into this more. IIRC, it generates an AST from the C source which you can iterate over to find problems. It's got Python APIs. I was hoping it could provide the basis for a static analysis tool to help find local memory (ref) leaks among other things. n From anthony at interlink.com.au Sat Jan 7 04:48:56 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Sat, 7 Jan 2006 14:48:56 +1100 Subject: [Python-Dev] Buildbot questions In-Reply-To: <43BEF6CC.1020603@v.loewis.de> References: <43BB0965.5080301@masklinn.net> <200601061916.38944.anthony@interlink.com.au> <43BEF6CC.1020603@v.loewis.de> Message-ID: <200601071449.01642.anthony@interlink.com.au> On Saturday 07 January 2006 10:01, Martin v. L?wis wrote: > The problem with irc-enabling (or web-enabling) such things is that > there is a potential for abuse. Of course, in this case, we could > wait for the abuse to happen. That would be my vote. Worst comes to worst, we lock it down to a list of known users. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From kay.schluehr at gmx.net Sat Jan 7 08:57:48 2006 From: kay.schluehr at gmx.net (Kay Schluehr) Date: Sat, 07 Jan 2006 08:57:48 +0100 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: References: <611494241.20060106005601@gmail.com> <20060106193448.GA656@code1.codespeak.net> Message-ID: Guido van Rossum wrote: >>Yuk! This has been discussed again and again already. *Please* move >>this discussion to comp.lang.python. > > > Yes please. This won't change. Then simply reject the PEP and the discussion can be stopped on comp.lang.python too. Or why do you think it should be discussed there again and again or elsewhere on the web ( e.g. in Bruce Eckels blog on Artima )? Regards From fredrik at pythonware.com Sat Jan 7 18:20:14 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sat, 7 Jan 2006 18:20:14 +0100 Subject: [Python-Dev] [Doc-SIG] that library reference, again References: <4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com> <43B4A238.2030101@python.org> <51A7717B-3832-477E-BED5-3C36DCA20336@lag.net> <43B56EEE.9090904@egenix.com> Message-ID: M.-A. Lemburg wrote: > I haven't followed the thread, so many I'm repeating things. > > Has anyone considered using e.g. MediaWiki (the wiki used for > Wikipedia) for Python documentation ? > > I'm asking because this wiki has proven to be ideally suited > for creating complex documentation tasks and offers many features > which would make Python documentation a lot easier and more > accessible: > > * people wouldn't have to learn LaTeX to commit doc-patches > * it's easy to monitor and revert changes, discuss changes > * there's version history available > * new docs would be instantly available on the web > * builtin search facility, categories and all the other nifty > wiki stuff > * it's one of the more popular wikis around and due to Wikipedia > it's here to stay > * conversion to XML and DocBook is possible, providing > entry points for conversion to other formats (including > LaTeX) > * more following means more tools (over time) > > Just a thought. anyone knows anything about support for semantic markup ? (afaict, the current moinmoin wiki would solve 80% of the issues involved here; I'm aiming for those 80% plus 80% of the remaining 20%). From facundobatista at gmail.com Sat Jan 7 19:17:15 2006 From: facundobatista at gmail.com (Facundo Batista) Date: Sat, 7 Jan 2006 15:17:15 -0300 Subject: [Python-Dev] ConfigParser to save with order Message-ID: ConfigParser saves the data in a not-predefined order. This is because it keeps, internally, the information in dictionaries. I opened a patch in SF 1399309 that orders the info to be saved in the file through the ConfigParser write() method. This patch does not let the user to specify the order, but allows to predict the order in which it'll be saved. There's another patch, the 1371075, that allows to specify the order through an "ordered dict". What do we want ConfigParser to do? Regards, . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From nnorwitz at gmail.com Sat Jan 7 21:45:27 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sat, 7 Jan 2006 12:45:27 -0800 Subject: [Python-Dev] Checking in a broken test was: Re: [Python-checkins] r41940 - python/trunk/Lib/test/test_compiler.py Message-ID: [moving to python-dev] > On 1/7/06, Reinhold Birkenfeld wrote: > > Well, it is not the test that's broken... it's compiler. [In reference to: http://mail.python.org/pipermail/python-checkins/2006-January/048715.html] In the past, we haven't checked in tests which are known to be broken. There are several good reasons for this. I would prefer you, 1) also fix the code so the test doesn't fail, 2) revert the change (there's still a bug report open, right?), or 3) generalize tests for known bugs. I strongly prefer #1, but have been thinking about adding #3. There are many open bug reports that fall into two broad categories: incorrect behaviour and crashers. I've been thinking about adding two tests which incorporate these bugs as a way of consolidating where the known problems are. Also, it's great when we have test cases that can be moved to the proper place once the fix has been checked in. I'm proposing something like add two files to Lib/test: outstanding_bugs.py and outstanding_crashes.py. Both would be normal test files with info about the bug report and the code that causes problems. This test in test_compiler should be moved to outstanding_bugs.py. And for a different discussion: On 1/7/06, Jim Jewett wrote: > Maybe. Guido's statement (maybe short of a pronouncement) > was that keyword-only arguments were OK in principle, and > that *args could follow keywords. It wasn't true yet because > no one had put in the work, but it would be an acceptable > change. > > I interpret this to mean that > > def f(a=1, b): pass > > should not necessarily raise an error, but I would like to see what > it does to > > def f(a=1, b): > print a,b > f(b=7) > > before saying that it is OK. From pedronis at strakt.com Sat Jan 7 22:26:01 2006 From: pedronis at strakt.com (Samuele Pedroni) Date: Sat, 07 Jan 2006 22:26:01 +0100 Subject: [Python-Dev] some interesting readings Message-ID: <43C031E9.6070704@strakt.com> because I was reminded of them recently, because they may be useful landmarks in the prospective of future discussions, because expanding one's understanding of the problem/solution space of language design is quite a good thing if one is interested in such things... 1) Gilad Bracha. Pluggable Type Systems . OOPSLA04 Workshop on Revival of Dynamic Languages ( http://pico.vub.ac.be/%7Ewdmeuter/RDL04/papers/Bracha.pdf ) As a talk: Pluggable Types, originally given at Aarhus University in March 2003, and repeated since at Berne and elsewhere. ( http://bracha.org/pluggable-types.pdf ) 2) http://homepages.cwi.nl/~ralf/OOHaskell/ state of the art experiment on trying to reconcile object orientation, type inference and as much as possible expressiveness PS: I think 1 is much more relevant than 2 for Python as we know it. happy reading. From tim.peters at gmail.com Sun Jan 8 00:24:06 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sat, 7 Jan 2006 18:24:06 -0500 Subject: [Python-Dev] Checking in a broken test was: Re: [Python-checkins] r41940 - python/trunk/Lib/test/test_compiler.py In-Reply-To: References: Message-ID: <1f7befae0601071524x55fc0652pbfebc5212d1bc6f9@mail.gmail.com> [Neal Norwitz] > ... > In the past, we haven't checked in tests which are known to be broken. It's an absolute rule that you never check in a change (whether a test or anything else) that causes ``regretst.py -uall`` to fail. Even if it passes on your development box, but fails on someone else's box, it's fair game to revert that change. Since this particular change caused -uall to fail on all boxes, I had no qualms about reverting rev 41940. Reinhold, if you're working on a fix for the problem too, feel free to check that test in on a branch instead (short-lived branches are easy & fast under SVN). Branches aren't required to pass any tests. The test on the branch can be merged into the trunk later, after a fix has also been checked in on that branch. ... > There are many open bug reports that fall into two broad categories: > incorrect behaviour and crashers. I've been thinking about adding two > tests which incorporate these bugs as a way of consolidating where the > known problems are. Also, it's great when we have test cases that can > be moved to the proper place once the fix has been checked in. > > I'm proposing something like add two files to Lib/test: > outstanding_bugs.py and outstanding_crashes.py. Both would be normal > test files with info about the bug report and the code that causes > problems. > > This test in test_compiler should be moved to outstanding_bugs.py. That would be fine provided that the failing tests aren't normally run via -uall. From guido at python.org Sun Jan 8 01:43:49 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 7 Jan 2006 16:43:49 -0800 Subject: [Python-Dev] Checking in a broken test was: Re: [Python-checkins] r41940 - python/trunk/Lib/test/test_compiler.py In-Reply-To: References: Message-ID: On 1/7/06, Neal Norwitz wrote: > I'm proposing something like add two files to Lib/test: > outstanding_bugs.py and outstanding_crashes.py. Both would be normal > test files with info about the bug report and the code that causes > problems. I like this approach. regrtest.py won't runthem, but you can run them manually to help in finding the bug (if the cause isn't known yet) or verifying it's been fixed (if it is). > And for a different discussion: > > On 1/7/06, Jim Jewett wrote: > > Maybe. Guido's statement (maybe short of a pronouncement) > > was that keyword-only arguments were OK in principle, and > > that *args could follow keywords. It wasn't true yet because > > no one had put in the work, but it would be an acceptable > > change. > > > > I interpret this to mean that > > > > def f(a=1, b): pass > > > > should not necessarily raise an error, but I would like to see what > > it does to > > > > def f(a=1, b): > > print a,b > > f(b=7) > > > > before saying that it is OK. I suspect you've got that wrong. I don't recall saying that you should be able to put positional arguments after keyword arguments in a call -- the meaning would be ambiguous to humans even if it can be given a clear meaning by the compiler. Similarly, in a def it would also be unclear to the human reader, and I'm not sure what the point would be. (In the above example, the real question is what f(7) would mean -- does it set b to 7 or a?) However, I *do* recall saying something about the relative ordering of *args and **kwds. In particular, I think we should be allowed to write f(1, 2, 3, *args, x=1, y=2, **kwds) # also without **kwds and perhaps also f(1, 2, 3, *args, 4, 5, 6) (though I'm less interested in supporting the latter). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From nnorwitz at gmail.com Sun Jan 8 01:58:30 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sat, 7 Jan 2006 16:58:30 -0800 Subject: [Python-Dev] Compiler warnings for 64-bit portability problems In-Reply-To: <5AC0F51A926CD442BC08D55DA7DAB1D40129F4AB@3mxma1r.hpi.uni-potsdam.de> References: <5AC0F51A926CD442BC08D55DA7DAB1D40129F4AB@3mxma1r.hpi.uni-potsdam.de> Message-ID: On 1/6/06, von L?wis Martin wrote: > I just found that the intel compiler (icc 9.0) > also supports compiler warnings for portability > problems. Cool. Thanks for the info. It would be nice if Intel would provide Python developers with a permanent icc license for Python. Can anyone help with that? > 4. don't try the Intel install.sh, it won't work > (atleast, it didn't work for me) I don't follow directions very well (I'm also a doubter), so I had to try it myself. :-) It worked for me on gentoo amd64 non-root install. I needed to set LD_LIBRARY_PATH to ~/intel/cce/9.0/lib, but otherwise it was pretty painless. This worked: CC=icc ./configure --without-cxx But I think this should work too: CC=icc CXX=icc ./configure I've fixed a bunch of problems on HEAD. I think I will move to ssize branch since that will be a much bigger help. I have some outstanding modifications, but here are the results of make: http://python.org/neal/icc-warnings.txt Summary (type, warning count, description): warning.#170: 2 pointer points outside of underlying object (obmalloc.c PT(0)) warning.#188: 14 enumerated type mixed with another type warning.#810: 1478 conversion from "long" to "int" may lose significant bits warning.#1418: 137 external definition with no prior declaration warning.#1419: 53 external declaration in primary source file warning.#1684: 285 conversion from pointer to same-sized integral type (potential portability problem) -- all due to offsetof() remark.#111: 121 statement is unreachable remark.#177: 1 func "marshal_write_mod" was declared but never referenced remark.#181: 9 arg is incompatible w/corresponding format string conversion remark.#279: 862 controlling expression is constant remark.#593: 17 variable set, but not used remark.#869: 1317 parameter not used remark.#981: 130 operands are evaluated in unspecified order remark.#1469: 10 "cc" clobber ignored (wtf? use of htons() causes this msg) remark.#1572: 111 floating-point equality/inequality comparisons are unreliable remark.#1599: 30 declaration hides parameter Note there are a lot of warning even when we mask off the higher bits (e.g., x = y & 0xff still generats a warning if x is a char). n From guido at python.org Sun Jan 8 02:04:39 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 7 Jan 2006 17:04:39 -0800 Subject: [Python-Dev] Compiler warnings for 64-bit portability problems In-Reply-To: References: <5AC0F51A926CD442BC08D55DA7DAB1D40129F4AB@3mxma1r.hpi.uni-potsdam.de> Message-ID: On 1/7/06, Neal Norwitz wrote: > Cool. Thanks for the info. It would be nice if Intel would provide > Python developers with a permanent icc license for Python. Can anyone > help with that? I'll try. A dutch friend from long ago (CWI) is now working for Intel's compiler group and has tried to interest me in providing feedback about their compiler (and also their alternative to valgrind). I'll contact him. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sun Jan 8 02:10:16 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 7 Jan 2006 17:10:16 -0800 Subject: [Python-Dev] ConfigParser to save with order In-Reply-To: References: Message-ID: I think it's moot unless you also preserve comments. Ideally would be something that prserved everything (ordering, blank lines, comments etc.) from how it was read in. Modifying a value should keep its position. Adding a value should add it to the end of the section it's in (unless there are cases where ordering is important to the semantics -- are there?). --Guido On 1/7/06, Facundo Batista wrote: > ConfigParser saves the data in a not-predefined order. This is because > it keeps, internally, the information in dictionaries. > > I opened a patch in SF 1399309 that orders the info to be saved in the > file through the ConfigParser write() method. > > This patch does not let the user to specify the order, but allows to > predict the order in which it'll be saved. > > There's another patch, the 1371075, that allows to specify the order > through an "ordered dict". > > What do we want ConfigParser to do? > > Regards, > > . Facundo > > Blog: http://www.taniquetil.com.ar/plog/ > PyAr: http://www.python.org/ar/ > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sun Jan 8 02:12:06 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 7 Jan 2006 17:12:06 -0800 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: References: <611494241.20060106005601@gmail.com> <20060106193448.GA656@code1.codespeak.net> Message-ID: On 1/6/06, Kay Schluehr wrote: > Then simply reject the PEP and the discussion can be stopped on > comp.lang.python too. Only in the most severe cases does it make sense to create a PEP specifically to be rejected. > Or why do you think it should be discussed there > again and again or elsewhere on the web ( e.g. in Bruce Eckels blog on > Artima )? Perhaps because people don't understand it? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Sun Jan 8 05:01:20 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 08 Jan 2006 14:01:20 +1000 Subject: [Python-Dev] some interesting readings In-Reply-To: <43C031E9.6070704@strakt.com> References: <43C031E9.6070704@strakt.com> Message-ID: <43C08E90.1070308@gmail.com> Samuele Pedroni wrote: > because I was reminded of them recently, because they may be useful > landmarks in the prospective of future discussions, because expanding > one's understanding of the problem/solution space of language design > is quite a good thing if one is interested in such things... > > 1) > Gilad Bracha. Pluggable Type Systems . OOPSLA04 Workshop on Revival of > Dynamic Languages ( > http://pico.vub.ac.be/%7Ewdmeuter/RDL04/papers/Bracha.pdf ) > > As a talk: > Pluggable Types, originally given at Aarhus University in March 2003, > and repeated since at Berne and elsewhere. > ( http://bracha.org/pluggable-types.pdf ) > > 2) > http://homepages.cwi.nl/~ralf/OOHaskell/ > state of the art experiment on trying to reconcile object orientation, > type inference and as much as possible expressiveness > > PS: I think 1 is much more relevant than 2 for Python as we know it. I'd have to agree with that - I didn't actually make it all the way through the second one, but an awful of lot of what I did read seemed to taken up with clever workarounds designed to trick the Haskell type inferencer into letting the authors of the paper do some fairly basic things (like having a heterogeneous collection of subtypes). There are some fascinating ideas in the first paper, though. It actually had me wondering about the possibilities of PyPy's object spaces, but I don't really know enough about those to determine whether or not such a connection is actually meaningful. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From pje at telecommunity.com Sun Jan 8 05:27:14 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 07 Jan 2006 23:27:14 -0500 Subject: [Python-Dev] some interesting readings In-Reply-To: <43C08E90.1070308@gmail.com> References: <43C031E9.6070704@strakt.com> <43C031E9.6070704@strakt.com> Message-ID: <5.1.1.6.0.20060107232224.02266cb8@mail.telecommunity.com> At 02:01 PM 1/8/2006 +1000, Nick Coghlan wrote: >Samuele Pedroni wrote: > > 2) > > http://homepages.cwi.nl/~ralf/OOHaskell/ > > state of the art experiment on trying to reconcile object orientation, > > type inference and as much as possible expressiveness > > > > PS: I think 1 is much more relevant than 2 for Python as we know it. > >I'd have to agree with that - I didn't actually make it all the way through >the second one, but an awful of lot of what I did read seemed to taken up >with >clever workarounds designed to trick the Haskell type inferencer into letting >the authors of the paper do some fairly basic things (like having a >heterogeneous collection of subtypes). Yeah, I thought the second one was a really strong argument for *not* trying to transpose "traditional" OO code directly into Haskell, but rather focusing on polymorphic functions instead. In Python terms, functions like len() and copy() rather than focusing on building methods like __len__() and __copy__(). Since Haskell allows you to provide multiple definitions for a function based on matching types or values, there's really no point to trying to make actual methods. It seemed to me a lot like the whole "implicit self" argument; i.e., imposing a foreign paradigm on the language for familiarity's sake. From kay.schluehr at gmx.net Sun Jan 8 10:56:34 2006 From: kay.schluehr at gmx.net (Kay Schluehr) Date: Sun, 08 Jan 2006 10:56:34 +0100 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: References: <611494241.20060106005601@gmail.com> <20060106193448.GA656@code1.codespeak.net> Message-ID: Guido van Rossum wrote: > On 1/6/06, Kay Schluehr wrote: > >>Then simply reject the PEP and the discussion can be stopped on >>comp.lang.python too. > > > Only in the most severe cases does it make sense to create a PEP > specifically to be rejected. >>Or why do you think it should be discussed there >>again and again or elsewhere on the web ( e.g. in Bruce Eckels blog on >>Artima )? > > > Perhaps because people don't understand it? They don't accept it even more than they don't understand it. This leads to repeated guesses that the object model might perhaps change in Py3K with respect to explicit self [1]. It's not all just an educational issue. If this is not a severe case where an autoritarian decision should be made and also be made visible at a central and recoverable place I don't know what? Regards Kay [1] See e.g. Bruce Eckels postings on this page: http://www.artima.com/forums/flat.jsp?forum=106&thread=141312&start=75&msRange=15 From g.brandl-nospam at gmx.net Sun Jan 8 11:21:03 2006 From: g.brandl-nospam at gmx.net (Georg Brandl) Date: Sun, 08 Jan 2006 11:21:03 +0100 Subject: [Python-Dev] Birkenfeld's gone Message-ID: Hello, today, when two Python developers here had approached me about the PSF, I realized that it is time to correct a mistake which I had made over three years ago, when I discovered Linux, free software, Usenet etc (I was sixteen at that time). I then assumed a different name, partly to anonymize myself as others had advised. When I came across Python, I instantly liked it, and since I had interest in Open Source development in general, I wanted to try and contribute to the development. And, as a matter of course, I used my different name for that. When I realized that I liked the community and the development of Python very much, I decided to "unveil" myself, but I could never overcome myself -- till now. I'm really sorry, and I hope you could at least partly understand what caused me to act so irrational. regards, Georg PS: I'm quite prepared now for the PSU to visit my place and interrog From martin at v.loewis.de Sun Jan 8 12:03:55 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 08 Jan 2006 12:03:55 +0100 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <43BEA574.8010607@colorstudy.com> References: <611494241.20060106005601@gmail.com> <43BDC001.3070602@gmail.com> <43BEA574.8010607@colorstudy.com> Message-ID: <43C0F19B.2000005@v.loewis.de> Ian Bicking wrote: > would have to be translated to this this: > > inst = Foo() > f = Foo.bar > meth = bind(f, inst) > print meth(1, 2) +1 for an explicit "bind unbound method operation", although I would spell it as inst = Foo() f = Foo.bar meth = f.bind(inst) print meth(1, 2) Regards, Martin From munna_tank at yahoo.co.in Sun Jan 8 13:00:24 2006 From: munna_tank at yahoo.co.in (RASHMI TANK) Date: Sun, 8 Jan 2006 12:00:24 +0000 (GMT) Subject: [Python-Dev] Python-Dev Digest, Vol 29, Issue 111 In-Reply-To: Message-ID: <20060108120024.77871.qmail@web8703.mail.in.yahoo.com> Respected sir, iam a un knowledged and un experienced person. i have an email to you but no reply. How you find me? i had joined icash business through icashline.com but i failed. you always giving emails to me where as idoesnt have any know ledge of web and free hosting and linking web pages and ee business and mass mailing softwere. please reply first and then give other supports. thanking you. please stop mails from you . i will inform to you then you send. ok. due to i am unable to view my computor for some period. may be3 months from now. thanking you. Send instant messages to your online friends http://in.messenger.yahoo.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060108/ee0fecf6/attachment.html From thomas at xs4all.net Sun Jan 8 13:25:28 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Sun, 8 Jan 2006 13:25:28 +0100 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: References: <611494241.20060106005601@gmail.com> <20060106193448.GA656@code1.codespeak.net> Message-ID: <20060108122527.GH18916@xs4all.nl> On Sat, Jan 07, 2006 at 05:12:06PM -0800, Guido van Rossum wrote: > On 1/6/06, Kay Schluehr wrote: > > Then simply reject the PEP and the discussion can be stopped on > > comp.lang.python too. > Only in the most severe cases does it make sense to create a PEP > specifically to be rejected. Yet if it isn't recorded, people will keep bringing it up. How about a 'rejected ideas' PEP for ideas that are right out no matter how people argue? A single PEP, with oneliners to describe ideas, one or two lines to explain 'why not', references to the python-list or python-dev discussions, if any, and a sign in big friendly letters saying "if you really must discuss these subjects, do it on python-list, not python-dev". Some of the stuff that could be in there: - implicit rather than explicit self: invalidates too many tricks - omitting () on functioncalls: like implicit self - changing all statements into expressions (e.g. logix): python isn't (going to be) a functional language - methods for tuples: tuples are records, not collections; use lists instead - sigils to indicate 'self.' (e.g. @foo): loses readability, wins too little - '?' and other non-alphanumerical characters in identifiers: like sigils - strict private/public accessing: 'fake' protection; use closures instead etc. No need to come up with them all, I'm sure all the prize ideas will pop back up eventually ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From martin at v.loewis.de Sun Jan 8 13:35:16 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 08 Jan 2006 13:35:16 +0100 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <20060108122527.GH18916@xs4all.nl> References: <611494241.20060106005601@gmail.com> <20060106193448.GA656@code1.codespeak.net> <20060108122527.GH18916@xs4all.nl> Message-ID: <43C10704.80404@v.loewis.de> Thomas Wouters wrote: > Yet if it isn't recorded, people will keep bringing it up. How about a > 'rejected ideas' PEP for ideas that are right out no matter how people > argue? Recorded it is, in the mailing list archive. However, a central place might be better, preferably with referrals to pronouncements or other discussion. A Wiki comes to mind... Regards, Martin From fredrik at pythonware.com Sun Jan 8 15:09:14 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sun, 8 Jan 2006 15:09:14 +0100 Subject: [Python-Dev] [Doc-SIG] that library reference, again References: <4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com> <43B4A238.2030101@python.org> <1135943776.8971.77.camel@warna.dub.corp.google.com> Message-ID: Donovan Baarda wrote: > > No, it's a fundamental goal: to support light-weight generation of rendered > > markup directly from source code, to enable simple tools (CGI scripts, etc) > > to be able to render reference documentation. > > Python is run-time interpreted, but I don't think we need its > documentation to be. Are you sure ? >>> help(help) >From the pythondoc/lib goal list: Support tighter (and 2-way) integration between the library reference and help tools (pydoc, help, inspect, etc) and documentation generators (epydoc, pythondoc, etc). > "trivial" is a relative term. When I say "trivial" in this context, I mean that any programmer should be capable of writing a Python tool that does some kind of basic processing of the source format from scratch. The easier it is to do this, the more likely it is that someone will come up with a new way to use the information ("Web 2.0", you know...) > As a some-times developer not really interested in writing > documentation, I must admit I really like ReST. It looks like plain > text, so I can write and use it as plain text, but it also can magically > transform to other formats and look pretty good. Lots of "looks like plain text if you squint" formats can produce output that looks pretty good. The problem is semantics; I haven't seen any format that looks like plain text but provides the amount of machine-readable semantic information that we have today. And I want more semantics, not less. > I hate writing XML/HTML, and my only experience with LaTex was brief and > painful too. If I was forced to choose, I'd probably pick LaTex even > though I don't know it as well as XML/HTML (I'd rather face an unknown > evil than that particular devil I know). Familiarity breeds contempt ? We're not talking about well-formed XML here, nor 1996-era hand-rolled web sites. If you look at typical Java- Doc or PythonDoc markup, or the translated samples, you'll notice that the source code contains very few HTML tags. (I seriously doubt that your problems with XML/HTML is the use of < in- stead of {...) > I'm not going to write docs for stuff I didn't write myself. I'm not > going to submit patches to Javadoc, LaTex or XML/HTML, but I might post > vague "change this to..." bug reports. > Writing good documentation is hard (which is why I prefer to leave it to > others if I can), but fixing almost good documentation doesn't have to > be /.../ Since I'm focussed on the library reference, I'm a lot more interested in the fixers than the essay writers. Give people a way to find things to fix, and a way to fix it, and they will. > > It's the workflow that's the real problem here, but you cannot fix the workflow > > without fixing the markup. > > I must say I disagree with this statement... changing the markup will > not change the workflow at all, just shave 3-6 mins off the compile-test > step. I don't do this enough to really know if that's worth it. Do you think wikipedia would be what it is today if the edit/save process included a 3-6 minute "regenerate site" pass ? Heck, I've discovered dozens of bogosities in the current library reference while working on the translation tools, and I have the sources and full SVN access, but I find that the overhead discourages me from doing anything about them ("maybe later"). I simply don't have time to fix anything but very minor typos within a standard 15-minute "let's improve python" time slot, and I sure don't think I'm unique in this respect. From rwgk at yahoo.com Sun Jan 8 15:28:57 2006 From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve) Date: Sun, 8 Jan 2006 06:28:57 -0800 (PST) Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <611494241.20060106005601@gmail.com> Message-ID: <20060108142857.89587.qmail@web31506.mail.mud.yahoo.com> --- Alexander Kozlovsky wrote: > What do you think about this? I (who writes Python code for a living) love it! See also: http://cci.lbl.gov/~rwgk/python/adopt_init_args_2005_07_02.html ***Please*** make Python more selfish. Note that this is also an obvious avenue for significant performance increases. If self is implicit you don't have to do the dictionary lookup for "self" all the time as is the case now. Good luck with your battle! Cheers, Ralf __________________________________________ Yahoo! DSL ? Something to write home about. Just $16.99/mo. or less. dsl.yahoo.com From fredrik at pythonware.com Sun Jan 8 15:25:01 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sun, 8 Jan 2006 15:25:01 +0100 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 References: <611494241.20060106005601@gmail.com><20060106193448.GA656@code1.codespeak.net> <20060108122527.GH18916@xs4all.nl> Message-ID: Thomas Wouters wrote: > > Only in the most severe cases does it make sense to create a PEP > > specifically to be rejected. > > Yet if it isn't recorded, people will keep bringing it up. How about a > 'rejected ideas' PEP for ideas that are right out no matter how people > argue? A single PEP, with oneliners to describe ideas, one or two lines to > explain 'why not', references to the python-list or python-dev discussions, > if any, and a sign in big friendly letters saying "if you really must > discuss these subjects, do it on python-list, not python-dev". Some of the > stuff that could be in there: > > - implicit rather than explicit self: invalidates too many tricks > - omitting () on functioncalls: like implicit self > - changing all statements into expressions (e.g. logix): python isn't > (going to be) a functional language > - methods for tuples: tuples are records, not collections; use lists instead > - sigils to indicate 'self.' (e.g. @foo): loses readability, wins too little > - '?' and other non-alphanumerical characters in identifiers: like sigils > - strict private/public accessing: 'fake' protection; use closures instead > > etc. No need to come up with them all, I'm sure all the prize ideas will pop > back up eventually ;) the FAQ touches many of these. maybe a couple of clarifications to the relevant FAQ texts (explaining things in terms of design tradeoffs, rather than absolute truths) would be good enough ? From g.brandl-nospam at gmx.net Sun Jan 8 15:36:42 2006 From: g.brandl-nospam at gmx.net (Georg Brandl) Date: Sun, 08 Jan 2006 15:36:42 +0100 Subject: [Python-Dev] Checking in a broken test was: Re: [Python-checkins] r41940 - python/trunk/Lib/test/test_compiler.py In-Reply-To: References: Message-ID: Neal Norwitz wrote: > [moving to python-dev] > >> On 1/7/06, Reinhold Birkenfeld wrote: >> > Well, it is not the test that's broken... it's compiler. > > [In reference to: > http://mail.python.org/pipermail/python-checkins/2006-January/048715.html] > > In the past, we haven't checked in tests which are known to be broken. Okay. I apologize. I originally intended to fix it, but I found that I don't have the time to search the bug. > There are several good reasons for this. I would prefer you, 1) also > fix the code so the test doesn't fail, 2) revert the change (there's > still a bug report open, right?), or 3) generalize tests for known > bugs. > > I strongly prefer #1, but have been thinking about adding #3. There > are many open bug reports that fall into two broad categories: > incorrect behaviour and crashers. I've been thinking about adding two > tests which incorporate these bugs as a way of consolidating where the > known problems are. Also, it's great when we have test cases that can > be moved to the proper place once the fix has been checked in. > > I'm proposing something like add two files to Lib/test: > outstanding_bugs.py and outstanding_crashes.py. Both would be normal > test files with info about the bug report and the code that causes > problems. > > This test in test_compiler should be moved to outstanding_bugs.py. I added outstanding_bugs.py and this initial test. regards, Georg From fredrik at pythonware.com Sun Jan 8 15:42:29 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sun, 8 Jan 2006 15:42:29 +0100 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 References: <611494241.20060106005601@gmail.com> <20060108142857.89587.qmail@web31506.mail.mud.yahoo.com> Message-ID: Ralf W. Grosse-Kunstleve wrote: > ***Please*** make Python more selfish. Note that this is also an obvious avenue > for significant performance increases. If self is implicit you don't have to do > the dictionary lookup for "self" all the time as is the case now. what dictionary lookup ? From skip at pobox.com Sun Jan 8 15:49:10 2006 From: skip at pobox.com (skip at pobox.com) Date: Sun, 8 Jan 2006 08:49:10 -0600 Subject: [Python-Dev] Birkenfeld's gone In-Reply-To: References: Message-ID: <17345.9830.144828.412332@montanaro.dyndns.org> Georg> I then assumed a different name, partly to anonymize myself as Georg> others had advised.... When I realized that I liked the Georg> community and the development of Python very much, I decided to Georg> "unveil" myself, but I could never overcome myself -- till now. Hmmm... How do we know Reinhold Birkenfeld and Georg Brandl are the same person? In fact, how do we know GB didn't "overcome" RB in some nefarious way and that RB isn't swimming with the fishes in the Rhein or buried deep in the Black Forest? Different first name, different last name, different ISP. From here it appears your hair color may have changed as well. Only the "-nospam" in your email addresses is the same... I'm sure the PSU will be able to From fredrik at pythonware.com Sun Jan 8 15:57:06 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sun, 8 Jan 2006 15:57:06 +0100 Subject: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py References: Message-ID: Neal Norwitz wrote: > In the past, we haven't checked in tests which are known to be broken. > There are several good reasons for this. I would prefer you, 1) also > fix the code so the test doesn't fail, 2) revert the change (there's > still a bug report open, right?), or 3) generalize tests for known > bugs. > > I strongly prefer #1, but have been thinking about adding #3. There > are many open bug reports that fall into two broad categories: > incorrect behaviour and crashers. I've been thinking about adding two > tests which incorporate these bugs as a way of consolidating where the > known problems are. Also, it's great when we have test cases that can > be moved to the proper place once the fix has been checked in. many test frameworks support "expected failures" for this purpose. how hard would it be to add a unittest.FailingTestCase class that runs a TestCase, catches any errors in it, and signals an error ("test foo passed unexpectedly") if it runs cleanly ? From rwgk at yahoo.com Sun Jan 8 16:35:53 2006 From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve) Date: Sun, 8 Jan 2006 07:35:53 -0800 (PST) Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: Message-ID: <20060108153553.78685.qmail@web31510.mail.mud.yahoo.com> --- Fredrik Lundh wrote: > Ralf W. Grosse-Kunstleve wrote: > > > ***Please*** make Python more selfish. Note that this is also an obvious > avenue > > for significant performance increases. If self is implicit you don't have > to do > > the dictionary lookup for "self" all the time as is the case now. > > what dictionary lookup ? IIUC, "self" is first looked up in the local dictionary. Please try the code below to see the performance impact. As an alternative to a copy/paste exercise try this: wget http://cci.lbl.gov/~rwgk/python/self_lookup.py python self_lookup.py The output with Python 2.4.1/Fedora3/Opteron is: loop_function: 13.10 loop_class: 17.11 Cheers, Ralf import time def loop_function(x, n): result = 0 for i in xrange(n): result += x return result class loop_class: def __init__(self, x, n): self.x = x self.n = n def __call__(self): result = 0 for i in xrange(self.n): result += self.x return result def run(x=3.1415, n=10**8): t0 = time.time() loop_function(x=x, n=n) print "loop_function: %.2f" % (time.time() - t0) t0 = time.time() loop_class(x=x, n=n)() print " loop_class: %.2f" % (time.time() - t0) if (__name__ == "__main__"): run() __________________________________________ Yahoo! DSL ? Something to write home about. Just $16.99/mo. or less. dsl.yahoo.com From fredrik at pythonware.com Sun Jan 8 16:49:36 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sun, 8 Jan 2006 16:49:36 +0100 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 References: <20060108153553.78685.qmail@web31510.mail.mud.yahoo.com> Message-ID: Ralf W. Grosse-Kunstleve wrote: > > what dictionary lookup ? > > IIUC, "self" is first looked up in the local dictionary. no, self is a local variable. self.x means that "x" is looked up in the in- stance dictionary, though. > Please try the code below to see the performance impact. oh, please. do you seriously think that if you don't have to type "self" yourself, Python will suddenly be able to turn all instance variables into local function variables without any performance overhead ? From rwgk at yahoo.com Sun Jan 8 16:54:34 2006 From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve) Date: Sun, 8 Jan 2006 07:54:34 -0800 (PST) Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: Message-ID: <20060108155434.24567.qmail@web31511.mail.mud.yahoo.com> --- Guido van Rossum wrote: > On 1/6/06, Armin Rigo wrote: > > On Fri, Jan 06, 2006 at 12:56:01AM +0300, Alexander Kozlovsky wrote: > > > There are three different peculiarity in Python 2.x > > > in respect of 'self' method argument: > > > > Yuk! This has been discussed again and again already. *Please* move > > this discussion to comp.lang.python. > > Yes please. This is a death penalty, even without your explicit rejection. No professional person has the time to sift through that volume of noise. > This won't change. The outside world is trying to tell you something; and they are not all dumb as you suggest in your other posting. I suggest creating python-pep at python.org. Cheers, Ralf __________________________________________ Yahoo! DSL ? Something to write home about. Just $16.99/mo. or less. dsl.yahoo.com From thomas at xs4all.net Sun Jan 8 16:58:16 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Sun, 8 Jan 2006 16:58:16 +0100 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <20060108153553.78685.qmail@web31510.mail.mud.yahoo.com> References: <20060108153553.78685.qmail@web31510.mail.mud.yahoo.com> Message-ID: <20060108155816.GI18916@xs4all.nl> On Sun, Jan 08, 2006 at 07:35:53AM -0800, Ralf W. Grosse-Kunstleve wrote: > IIUC, "self" is first looked up in the local dictionary. No. Local variables are stored in a tuple (more or less,) and indexed by, er, index. Loading a local variable onto the stack is a fairly fast operation. > Please try the code below to see the performance impact. As an alternative to a > copy/paste exercise try this: > > wget http://cci.lbl.gov/~rwgk/python/self_lookup.py > python self_lookup.py > The output with Python 2.4.1/Fedora3/Opteron is: > > loop_function: 13.10 > loop_class: 17.11 The main difference isn't the lookup of 'self', it's the attribute retrieval of 'x' from 'self'. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From rwgk at yahoo.com Sun Jan 8 16:58:29 2006 From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve) Date: Sun, 8 Jan 2006 07:58:29 -0800 (PST) Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: Message-ID: <20060108155829.69611.qmail@web31513.mail.mud.yahoo.com> --- Fredrik Lundh wrote: > > Please try the code below to see the performance impact. > > oh, please. do you seriously think that if you don't have to type "self" > yourself, Python will suddenly be able to turn all instance variables into > local function variables without any performance overhead ? Yes, exactly, since you can make the local namespace vs. instance attribute distinction in C. Cheers, Ralf __________________________________________ Yahoo! DSL ? Something to write home about. Just $16.99/mo. or less. dsl.yahoo.com From pedronis at strakt.com Sun Jan 8 17:13:27 2006 From: pedronis at strakt.com (Samuele Pedroni) Date: Sun, 08 Jan 2006 17:13:27 +0100 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <20060108155829.69611.qmail@web31513.mail.mud.yahoo.com> References: <20060108155829.69611.qmail@web31513.mail.mud.yahoo.com> Message-ID: <43C13A27.5030702@strakt.com> Ralf W. Grosse-Kunstleve wrote: > --- Fredrik Lundh wrote: > >>>Please try the code below to see the performance impact. >> >>oh, please. do you seriously think that if you don't have to type "self" >>yourself, Python will suddenly be able to turn all instance variables into >>local function variables without any performance overhead ? > > > Yes, exactly, since you can make the local namespace vs. instance attribute > distinction in C. > > Cheers, > Ralf > > that indeed proves why this discussion for whatever value it has, belongs to comp.lang.python. This is not a place to clarify people how Python currently works, or to have discussions made of guesses and half-formed speculations. From rwgk at yahoo.com Sun Jan 8 17:09:22 2006 From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve) Date: Sun, 8 Jan 2006 08:09:22 -0800 (PST) Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <20060108155816.GI18916@xs4all.nl> Message-ID: <20060108160922.88708.qmail@web31507.mail.mud.yahoo.com> --- Thomas Wouters wrote: > The main difference isn't the lookup of 'self', it's the attribute retrieval > of 'x' from 'self'. I see. Thanks! If you put 'self' into a special category (with corresponding C code), couldn't you use the same "indexing" tricks as for local variables and make 'self.x' just as fast as 'x'? Cheers, Ralf __________________________________________ Yahoo! DSL ? Something to write home about. Just $16.99/mo. or less. dsl.yahoo.com From amk at amk.ca Sun Jan 8 18:21:16 2006 From: amk at amk.ca (A.M. Kuchling) Date: Sun, 8 Jan 2006 12:21:16 -0500 Subject: [Python-Dev] PyCon TX 2006: Early-bird registration ends Jan. 15! Message-ID: <20060108172116.GB8928@rogue.amk.ca> Early bird registration for PyCon TX 2006 ends on January 15th, so there's only a week left. To register, please go to: http://us.pycon.org/TX2006/Registration You can still register after Jan. 15th, but the cost will go up by US$65 (US$25 for students). This year PyCon will feature a day of tutorials before the three days of regular presentations. Course outlines for all the tutorials have been posted; see http://wiki.python.org/moin/PyCon2006/Tutorials All of the PyCon tutorials are still open for new registrations, but space is limited. Don't forget to book your hotel room, too. PyCon TX 2006 is being held at a Dallas/Addison hotel, and we have negotiated a special low rate of $79 plus applicable taxes: http://us.pycon.org/Addison/Hotels We hope to see you in Texas! Regards, A.M. Kuchling amk at amk.ca Chair, PyCon 2006 http://us.pycon.org From thomas at xs4all.net Sun Jan 8 17:26:59 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Sun, 8 Jan 2006 17:26:59 +0100 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <20060108160922.88708.qmail@web31507.mail.mud.yahoo.com> References: <20060108155816.GI18916@xs4all.nl> <20060108160922.88708.qmail@web31507.mail.mud.yahoo.com> Message-ID: <20060108162659.GJ18916@xs4all.nl> On Sun, Jan 08, 2006 at 08:09:22AM -0800, Ralf W. Grosse-Kunstleve wrote: > --- Thomas Wouters wrote: > > The main difference isn't the lookup of 'self', it's the attribute retrieval > > of 'x' from 'self'. > I see. Thanks! > If you put 'self' into a special category (with corresponding C code), couldn't > you use the same "indexing" tricks as for local variables and make 'self.x' > just as fast as 'x'? No need to make 'self' magical for that. You do, however, need to make the *object* magical for that. It works for local namespaces because the Python compiler knows exactly which names are local and which aren't. There are two cases where the compiler can't know for sure (using 'exec' without 'in' and using 'from module import *') and in those cases, Python doesn't optimize local variable access all the way. This all won't work for objects and their attributes because too much is possible at runtime for the current compiler to handle. The compiler could, of course, check everything in the whole program, keeping track of what could possibly adjust which attributes of what object where. That would be 'type inferencing' and has been (and is being, I guess) attempted by various people in various projets. However, if your compiler is so smart, it won't have _any_ problems taking an exlicit self into account :) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From pje at telecommunity.com Sun Jan 8 17:27:16 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 08 Jan 2006 11:27:16 -0500 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <20060108160922.88708.qmail@web31507.mail.mud.yahoo.com> References: <20060108155816.GI18916@xs4all.nl> Message-ID: <5.1.1.6.0.20060108111702.01b12308@mail.telecommunity.com> At 08:09 AM 1/8/2006 -0800, Ralf W. Grosse-Kunstleve wrote: >--- Thomas Wouters wrote: > > > The main difference isn't the lookup of 'self', it's the attribute > retrieval > > of 'x' from 'self'. > >I see. Thanks! >If you put 'self' into a special category (with corresponding C code), >couldn't >you use the same "indexing" tricks as for local variables and make 'self.x' >just as fast as 'x'? It's not nearly that simple. See e.g.: http://mail.python.org/pipermail/python-dev/2002-February/019854.html Note, however, that such a speedup is entirely independent of the syntax. Trying to link the syntax with the performance is completely bogus. From pje at telecommunity.com Sun Jan 8 17:31:00 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 08 Jan 2006 11:31:00 -0500 Subject: [Python-Dev] Rejected ideas PEP (was re: Draft proposal: Implicit self in Python 3.0) In-Reply-To: <20060108122527.GH18916@xs4all.nl> References: <611494241.20060106005601@gmail.com> <20060106193448.GA656@code1.codespeak.net> Message-ID: <5.1.1.6.0.20060108112847.043ff7e8@mail.telecommunity.com> At 01:25 PM 1/8/2006 +0100, Thomas Wouters wrote: >On Sat, Jan 07, 2006 at 05:12:06PM -0800, Guido van Rossum wrote: > > On 1/6/06, Kay Schluehr wrote: > > > Then simply reject the PEP and the discussion can be stopped on > > > comp.lang.python too. > > > Only in the most severe cases does it make sense to create a PEP > > specifically to be rejected. > >Yet if it isn't recorded, people will keep bringing it up. How about a >'rejected ideas' PEP for ideas that are right out no matter how people >argue? A single PEP, with oneliners to describe ideas, one or two lines to >explain 'why not', references to the python-list or python-dev discussions, >if any, and a sign in big friendly letters saying "if you really must >discuss these subjects, do it on python-list, not python-dev". Some of the >stuff that could be in there: > > - implicit rather than explicit self: invalidates too many tricks > - omitting () on functioncalls: like implicit self > - changing all statements into expressions (e.g. logix): python isn't > (going to be) a functional language > - methods for tuples: tuples are records, not collections; use lists instead > - sigils to indicate 'self.' (e.g. @foo): loses readability, wins too little > - '?' and other non-alphanumerical characters in identifiers: like sigils > - strict private/public accessing: 'fake' protection; use closures instead > >etc. No need to come up with them all, I'm sure all the prize ideas will pop >back up eventually ;) +1, along with a modification to the PEP procedures to require that people check the rejected ideas PEP before submitting a new proposal. The "rejected ideas" PEP should presumably be a low-numbered process PEP. From thomas at xs4all.net Sun Jan 8 17:37:12 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Sun, 8 Jan 2006 17:37:12 +0100 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <43C10704.80404@v.loewis.de> References: <611494241.20060106005601@gmail.com> <20060106193448.GA656@code1.codespeak.net> <20060108122527.GH18916@xs4all.nl> <43C10704.80404@v.loewis.de> Message-ID: <20060108163712.GK18916@xs4all.nl> On Sun, Jan 08, 2006 at 01:35:16PM +0100, "Martin v. L?wis" wrote: > Thomas Wouters wrote: > > Yet if it isn't recorded, people will keep bringing it up. How about a > > 'rejected ideas' PEP for ideas that are right out no matter how people > > argue? > Recorded it is, in the mailing list archive. > However, a central place might be better, preferably with referrals to > pronouncements or other discussion. A Wiki comes to mind... My point isn't that it isn't archived somewhere (mailinglists, wiki, FAQ, the minds of many, many people, not just Python developers) but that it isn't easily findable and it isn't easily accessible in a single location. I thought PEP's where supposed to be that, and if I have a particular idea for new syntax or new semantics, PEPs would be the place I'd look, not the FAQ or a Wiki. I guess a Wiki'd work, but then I wonder, why aren't all PEPs in a Wiki instead? At least until they are actually accepted, after which they get finalized and formalized and implemented and what not. (I'm not a fan of Wiki's, but that may be because I've seen too many chaotic, unstructured, messy, unmaintained ones.) And I would like to point out how hard it is to google or grep for ideas like this. For instance, the 'x, y, *rest = someseq' syntax. I wouldn't know what to call it (or what others would call it, and you can't google for the syntax (since the variables can be named anything). Grepping through downloaded mailinglist archives might find you some discussion, but probably without any indication whether anything is final or current. Grepping through the PEPs might find you something, if it's PEP'ed, but it's a lot easier to browse the list of (non-accepted/implemented) PEPs. You may not know the name of the construct you're looking for, but someone else's name for it would likely ring a bell. And a PEP titled 'Rejected Ideas' would certainly be the first place to look ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From tim.peters at gmail.com Sun Jan 8 17:42:09 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 8 Jan 2006 11:42:09 -0500 Subject: [Python-Dev] [Python-checkins] r41972 - python/branches/ssize_t/Objects/funcobject.c In-Reply-To: <43C0DF8C.7050104@v.loewis.de> References: <20060108060915.A69E01E4002@bag.python.org> <43C0DF8C.7050104@v.loewis.de> Message-ID: <1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com> [neal.norwitz] >> PyErr_Format(PyExc_ValueError, >> - "%s() requires a code object with %d free vars," >> - " not %d", >> + "%s() requires a code object with %ld free vars," >> + " not %ld", >> PyString_AsString(op->func_name), >> nclosure, nfree); [Martin v. L?wis] > I'm not sure whether this is the right fix. This says nclosure and nfree > are longs; this is true on most 32-bit systems (where ssize_t is int > and int and long have the same width), and on some 64-bit systems (where > ssize_t and long are the same). It does not work for Win64 (where > ssize_t is larger than long). Ya, that doesn't work. We'll have similar problems with obmalloc debug output if obmalloc is changed to escape 32-bitness on Win64 -- and anywhere else we need to format one of these things. > I'm tempted to use %zd in these places, but fear that it isn't portable > enough (it *is* part of C99). MS VC 7.1 doesn't support the z length qualifier, and I expect that kills it right there. So we need a macro, say Py_SIZE_T_WIDTH, in pyport.h: /* Length qualifier for use in formats when printing size_t or Py_ssize_t values. */ #ifndef Py_SIZE_T_WIDTH # define Py_SIZE_T_WIDTH "z" #endif That handles C99-ish platforms by defalt. Other platforms (like Windows) would need to supply their own expansion in their pyconfig.h. Neal's format would become the god-awful but platform-neutral: "%s() requires a code object with %" Py_SIZE_T_WIDTH "d free vars," " not %" Py_SIZE_T_WIDTH "d" I suspect we're going to have other problems when someone gets around to passing a size_t-ish value to PyString_Format() or PyErr_Format(). Maybe we could teach those about the "z" qualifier on all platforms. From tim.peters at gmail.com Sun Jan 8 18:16:13 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 8 Jan 2006 12:16:13 -0500 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <20060108163712.GK18916@xs4all.nl> References: <611494241.20060106005601@gmail.com> <20060106193448.GA656@code1.codespeak.net> <20060108122527.GH18916@xs4all.nl> <43C10704.80404@v.loewis.de> <20060108163712.GK18916@xs4all.nl> Message-ID: <1f7befae0601080916n2dc1388et14578ddc33c5563d@mail.gmail.com> [Thomas Wouters] > My point isn't that it isn't archived somewhere (mailinglists, wiki, FAQ, > the minds of many, many people, not just Python developers) but that it > isn't easily findable and it isn't easily accessible in a single location. I > thought PEP's where supposed to be that, and if I have a particular idea for > new syntax or new semantics, PEPs would be the place I'd look, not the FAQ > or a Wiki. Luckily, in his benevolent infinite wisdom, I expect Guido reserved PEP number 13 for exactly this purpose: for a meta-PEP to record the unlucky PEP ideas that are so unlikely to get accepted that it's not worth anyone's time to write an actual PEP for them. I like the title: Don't Bother: PEPs Rejected Before Being Written No, I'm not kidding. At least I don't think I am. > ... > And I would like to point out how hard it is to google or grep for ideas > like this. For instance, the 'x, y, *rest = someseq' syntax. I wouldn't know > what to call it (or what others would call it, and you can't google for the > syntax (since the variables can be named anything). If PEP 13 grows large enough, won't be easy to find there either. But it will be easy to remember _where_ to look. > ... > And a PEP titled 'Rejected Ideas' would certainly be the first place to look ;) Too bland ;-) From martin at v.loewis.de Sun Jan 8 18:19:19 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 08 Jan 2006 18:19:19 +0100 Subject: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py In-Reply-To: References: Message-ID: <43C14997.30309@v.loewis.de> Fredrik Lundh wrote: > many test frameworks support "expected failures" for this purpose. > how hard would it be to add a > > unittest.FailingTestCase > > class that runs a TestCase, catches any errors in it, and signals an > error ("test foo passed unexpectedly") if it runs cleanly ? I don't know how hard it would be, but I would also consider this appropriate. Of course, this should work on a case-by-case basis: if there are multiple test methods in a class, unexpected passes of each method should be reported. Regards, Martin From martin at v.loewis.de Sun Jan 8 18:22:56 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 08 Jan 2006 18:22:56 +0100 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <20060108155434.24567.qmail@web31511.mail.mud.yahoo.com> References: <20060108155434.24567.qmail@web31511.mail.mud.yahoo.com> Message-ID: <43C14A70.1050206@v.loewis.de> Ralf W. Grosse-Kunstleve wrote: > The outside world is trying to tell you something; and they are not all dumb as > you suggest in your other posting. But they might suffer from misunderstandings, such as your misunderstanding in how local variables work and whether 'self' is looked up in a dictionary. So it's being dumb - just being uninformed. Regards, Martin From martin at v.loewis.de Sun Jan 8 18:31:35 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 08 Jan 2006 18:31:35 +0100 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <20060108163712.GK18916@xs4all.nl> References: <611494241.20060106005601@gmail.com> <20060106193448.GA656@code1.codespeak.net> <20060108122527.GH18916@xs4all.nl> <43C10704.80404@v.loewis.de> <20060108163712.GK18916@xs4all.nl> Message-ID: <43C14C77.8010308@v.loewis.de> Thomas Wouters wrote: > My point isn't that it isn't archived somewhere (mailinglists, wiki, FAQ, > the minds of many, many people, not just Python developers) but that it > isn't easily findable and it isn't easily accessible in a single location. Why would a single Wiki page not be accessible in a single location? Why is the FAQ not accessible in a single location? > I > thought PEP's where supposed to be that, and if I have a particular idea for > new syntax or new semantics, PEPs would be the place I'd look, not the FAQ > or a Wiki. Right. However, I doubt you would look in the "rejected ideas" PEP. > I guess a Wiki'd work, but then I wonder, why aren't all PEPs in > a Wiki instead? There might be several reasons - one is that a PEP is supposed to get posted to mailing lists and newsgroups, something that is not readily possible with Wiki pages. > And I would like to point out how hard it is to google or grep for ideas > like this. For instance, the 'x, y, *rest = someseq' syntax. I agree that collecting them in a single place is a good idea. Whether this is a PEP, a Wiki page, or the FAQ is nearly irrelevant, except: - it is strange to call it a "Python Enhancement Proposal" - in either the FAQ or the PEP, it will stay in its initial form forever, since nobody but the original author will edit it (that could be true of a Wiki page as well, but on a Wiki page, people *know* they are meant to edit it if they want to say something) Regards, Martin From martin at v.loewis.de Sun Jan 8 18:32:41 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 08 Jan 2006 18:32:41 +0100 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <43C14A70.1050206@v.loewis.de> References: <20060108155434.24567.qmail@web31511.mail.mud.yahoo.com> <43C14A70.1050206@v.loewis.de> Message-ID: <43C14CB9.8050906@v.loewis.de> Martin v. L?wis wrote: > But they might suffer from misunderstandings, such as your > misunderstanding in how local variables work and whether > 'self' is looked up in a dictionary. > > So it's being dumb - just being uninformed. Sorry: "*not* being dumb" is what I wanted to say. Regards, Martin From tim.peters at gmail.com Sun Jan 8 18:42:09 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 8 Jan 2006 12:42:09 -0500 Subject: [Python-Dev] [Python-checkins] r41972 - python/branches/ssize_t/Objects/funcobject.c In-Reply-To: <1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com> References: <20060108060915.A69E01E4002@bag.python.org> <43C0DF8C.7050104@v.loewis.de> <1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com> Message-ID: <1f7befae0601080942o3a7dc194ub790b616bb79c2fe@mail.gmail.com> [Tim] > ... > I suspect we're going to have other problems when someone gets around > to passing a size_t-ish value to PyString_Format() or PyErr_Format(). > Maybe we could teach those about the "z" qualifier on all platforms. Yup. Because the supporting PyString_FromFormatV() doesn't know anything about z qualifiers now, if you _tried_ to use %zd then even on a C99-platform PyString_FromFormatV() wouldn't do any conversions from that point on: it would just copy the format string from the first "z" onward into the output buffer, and ignore the trailing inputs. The good news is that we could teach it about "z" (but would still need a macro or some such to implement "z" correctly across platforms). From fdrake at acm.org Sun Jan 8 18:34:59 2006 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Sun, 8 Jan 2006 12:34:59 -0500 Subject: [Python-Dev] =?iso-8859-1?q?Checking_in_a_broken_test_was=3A_Re?= =?iso-8859-1?q?=3A=09=5BPython-checkins=5Dr41940_-_python/trunk/Lib/test/?= =?iso-8859-1?q?test=5Fcompiler=2Epy?= In-Reply-To: <43C14997.30309@v.loewis.de> References: <43C14997.30309@v.loewis.de> Message-ID: <200601081234.59951.fdrake@acm.org> On Sunday 08 January 2006 12:19, Martin v. L?wis wrote: > I don't know how hard it would be, but I would also consider this > appropriate. Of course, this should work on a case-by-case basis: > if there are multiple test methods in a class, unexpected passes > of each method should be reported. I like the way trial (from twisted) supports this. The test method is written normally, in whatever class makes sense. Then the test is marked with an attribute to say it isn't expected to pass yet. When the code is fixed and the test passes, you get that information in trial's output, and can unmark the test. This avoids having to refactor test classes just to update the status of a test. -Fred -- Fred L. Drake, Jr. From martin at v.loewis.de Sun Jan 8 18:52:39 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 08 Jan 2006 18:52:39 +0100 Subject: [Python-Dev] [Python-checkins] r41972 - python/branches/ssize_t/Objects/funcobject.c In-Reply-To: <1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com> References: <20060108060915.A69E01E4002@bag.python.org> <43C0DF8C.7050104@v.loewis.de> <1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com> Message-ID: <43C15167.7000702@v.loewis.de> Tim Peters wrote: > That handles C99-ish platforms by defalt. Other platforms (like > Windows) would need to supply their own expansion in their pyconfig.h. > Neal's format would become the god-awful > but platform-neutral: > > "%s() requires a code object with %" Py_SIZE_T_WIDTH "d free vars," > " not %" Py_SIZE_T_WIDTH "d" That's too ugly, IMO. So I could accept to write "%s() requires a code object with %ld free vars," " not %ld", PyString_AsString(op->func_name), (long)nclosure, (long)nfree); instead. Alternatively, if there is a real chance that it overflows LONG_MAX (which there isn't, AFAIK - we don't support that many free vars), we could also write "%s() requires a code object with %ld free vars," " not %ld", PyString_AsString(op->func_name), Py_long_ceil(nclosure), Py_long_ceil(nfree)); with #define Py_long_ceil(n) ((n) > LONG_MAX ? LONG_MAX : \ (n) < LONG_MIN ? LONG_MIN : (n)) On most platforms, the compiler should notice that the conditions are both known false, and collaps this to (n). If this ever overflows, somebody will have to remember that this is really +/- infinity. Regards, Martin P.S. Proposals for a different macro name are thankfully accepted. From martin at v.loewis.de Sun Jan 8 18:59:35 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 08 Jan 2006 18:59:35 +0100 Subject: [Python-Dev] [Python-checkins] r41972 - python/branches/ssize_t/Objects/funcobject.c In-Reply-To: <1f7befae0601080942o3a7dc194ub790b616bb79c2fe@mail.gmail.com> References: <20060108060915.A69E01E4002@bag.python.org> <43C0DF8C.7050104@v.loewis.de> <1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com> <1f7befae0601080942o3a7dc194ub790b616bb79c2fe@mail.gmail.com> Message-ID: <43C15307.2060600@v.loewis.de> Tim Peters wrote: > Yup. Because the supporting PyString_FromFormatV() doesn't know > anything about z qualifiers now, if you _tried_ to use %zd then even > on a C99-platform PyString_FromFormatV() wouldn't do any conversions > from that point on: it would just copy the format string from the > first "z" onward into the output buffer, and ignore the trailing > inputs. The good news is that we could teach it about "z" (but would > still need a macro or some such to implement "z" correctly across > platforms). On VC7.1, we could use 'L', right? On other platforms, we could check whether sizeof(size_t) is sizeof(long), and use 'l', else we could refuse compilation (I doubt this would rule out any relevant system). Regards, Martin From martin at v.loewis.de Sun Jan 8 19:01:20 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 08 Jan 2006 19:01:20 +0100 Subject: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py In-Reply-To: <200601081234.59951.fdrake@acm.org> References: <43C14997.30309@v.loewis.de> <200601081234.59951.fdrake@acm.org> Message-ID: <43C15370.2070700@v.loewis.de> Fred L. Drake, Jr. wrote: > I like the way trial (from twisted) supports this. The test method is written > normally, in whatever class makes sense. Then the test is marked with an > attribute to say it isn't expected to pass yet. When the code is fixed and > the test passes, you get that information in trial's output, and can unmark > the test. This avoids having to refactor test classes just to update the > status of a test. So how is the mark added? I would suggest @xfail def test_foo(self): self.assertEquals(0,1) Regards, Martin From foom at fuhm.net Sun Jan 8 19:09:27 2006 From: foom at fuhm.net (James Y Knight) Date: Sun, 8 Jan 2006 13:09:27 -0500 Subject: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py In-Reply-To: <43C15370.2070700@v.loewis.de> References: <43C14997.30309@v.loewis.de> <200601081234.59951.fdrake@acm.org> <43C15370.2070700@v.loewis.de> Message-ID: On Jan 8, 2006, at 1:01 PM, Martin v. L?wis wrote: > Fred L. Drake, Jr. wrote: > >> I like the way trial (from twisted) supports this. The test >> method is written >> normally, in whatever class makes sense. Then the test is marked >> with an >> attribute to say it isn't expected to pass yet. When the code is >> fixed and >> the test passes, you get that information in trial's output, and >> can unmark >> the test. This avoids having to refactor test classes just to >> update the >> status of a test. >> > > So how is the mark added? I would suggest > > @xfail > def test_foo(self): > self.assertEquals(0,1) def test_foo(self): self.assertEquals(0,1) test_foo.todo = "Why this test fails description message." James From g.brandl-nospam at gmx.net Sun Jan 8 20:04:45 2006 From: g.brandl-nospam at gmx.net (Georg Brandl) Date: Sun, 08 Jan 2006 20:04:45 +0100 Subject: [Python-Dev] test_curses Message-ID: The call to curses.setupterm() leaves my terminal in a bad state. The reset program outputs: Erase set to delete. Kill set to control-U (^U). Interrupt set to control-C (^C). Doesn't the setupterm() have to be paired with something like shutdownterm()? regards, Georg From thomas at xs4all.net Sun Jan 8 20:30:18 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Sun, 8 Jan 2006 20:30:18 +0100 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <43C14C77.8010308@v.loewis.de> References: <611494241.20060106005601@gmail.com> <20060106193448.GA656@code1.codespeak.net> <20060108122527.GH18916@xs4all.nl> <43C10704.80404@v.loewis.de> <20060108163712.GK18916@xs4all.nl> <43C14C77.8010308@v.loewis.de> Message-ID: <20060108193018.GL18916@xs4all.nl> On Sun, Jan 08, 2006 at 06:31:35PM +0100, "Martin v. L?wis" wrote: > Thomas Wouters wrote: > > My point isn't that it isn't archived somewhere (mailinglists, wiki, FAQ, > > the minds of many, many people, not just Python developers) but that it > > isn't easily findable and it isn't easily accessible in a single location. > Why would a single Wiki page not be accessible in a single location? Why > is the FAQ not accessible in a single location? Unless either of those would replace PEPs (or include all information from all PEPs), they would be a second location. > > I thought PEP's where supposed to be that, and if I have a particular > > idea for new syntax or new semantics, PEPs would be the place I'd look, > > not the FAQ or a Wiki. > Right. However, I doubt you would look in the "rejected ideas" PEP. You may have the luxury of that doubt, but I don't :) I'm sorry if my comments sounded hypothetical; they weren't. I did wonder about (among other things) 'x, y, *rest = l', and because I didn't know the name, I did browse the PEP list. A 'Rejected Ideas' PEP would've caught my eye even if I didn't have a hunch Guido would reject 'x, y, *r = l'. > I agree that collecting them in a single place is a good idea. Whether > this is a PEP, a Wiki page, or the FAQ is nearly irrelevant, except: > - it is strange to call it a "Python Enhancement Proposal" They are proposals to enhance Python, though. That's exactly why I argue they should be in a PEP, not in some other location: it's all neatly bundled together. It's just not a PEP per single proposal. > - in either the FAQ or the PEP, it will stay in its initial form > forever, since nobody but the original author will edit it > (that could be true of a Wiki page as well, but on a Wiki page, > people *know* they are meant to edit it if they want to say > something) I don't think the rejected-ideas collection should be publically editable either. It's not a discussion forum, it's not meant to get people to persuade Guido to change his mind, it's a list of "these things just aren't going to happen, deal with it". A Meta-PEP like Tim suggested seems most appropriate to me, even if it isn't purely Meta; practicality beats purity and all that. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From g.brandl-nospam at gmx.net Sun Jan 8 20:39:08 2006 From: g.brandl-nospam at gmx.net (Georg Brandl) Date: Sun, 08 Jan 2006 20:39:08 +0100 Subject: [Python-Dev] locale and LC_NUMERIC Message-ID: Hi, >>> import locale >>> locale.setlocale(locale.LC_NUMERIC, "") 'de_DE at euro' >>> "%f" % 1.0 '1.000000' >>> u"%f" % 1.0 u'1,000000' >>> Is this intended? This breaks test_format on my box when test_builtin (method test_float_with_comma) is executed first. regards, Georg From ianb at colorstudy.com Sun Jan 8 21:43:17 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Sun, 08 Jan 2006 14:43:17 -0600 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <1f7befae0601080916n2dc1388et14578ddc33c5563d@mail.gmail.com> References: <611494241.20060106005601@gmail.com> <20060106193448.GA656@code1.codespeak.net> <20060108122527.GH18916@xs4all.nl> <43C10704.80404@v.loewis.de> <20060108163712.GK18916@xs4all.nl> <1f7befae0601080916n2dc1388et14578ddc33c5563d@mail.gmail.com> Message-ID: <43C17965.5070408@colorstudy.com> Tim Peters wrote: > [Thomas Wouters] > >>My point isn't that it isn't archived somewhere (mailinglists, wiki, FAQ, >>the minds of many, many people, not just Python developers) but that it >>isn't easily findable and it isn't easily accessible in a single location. I >>thought PEP's where supposed to be that, and if I have a particular idea for >>new syntax or new semantics, PEPs would be the place I'd look, not the FAQ >>or a Wiki. > > > Luckily, in his benevolent infinite wisdom, I expect Guido reserved > PEP number 13 for exactly this purpose: for a meta-PEP to record the > unlucky PEP ideas that are so unlikely to get accepted that it's not > worth anyone's time to write an actual PEP for them. I like the > title: > > Don't Bother: PEPs Rejected Before Being Written > > No, I'm not kidding. At least I don't think I am. +1. I think it's rather patronizing to send things back to python-list when you know people are wasting their time discussing them because they will never be accepted. People on python-list have useful things to do too, they don't just read to waste their time. I would prefer PEP form over a wiki page, as I'd rather this be truly authoritative, and only Guido can really completely reject an idea. Justifying the rejections can be done by anyone though; maybe each idea could link to a wiki page on the topic. I think it's also important to be clear on what's being rejected. Often these rejected ideas address a real issue, and if you think about the issue from another angle you might be able to solve that without falling into the trap that the oft-rejected proposal fell into. But it's easy to confuse that the issue or use case is being explicitly ignored, rather than the particulars. For instance, Thomas said "changing all statements into expressions (e.g. logix): python isn't (going to be) a functional language" -- and that's what shouldn't be in the PEP. All statements aren't going to be expressions; the editorialization that Python isn't going to be a functional language is both rather inaccurate, misses the real reason for statements, and needlessly alienates people who like functional programming (and they have been *needlessly* alienated by discussions about lambda and filter). So... maybe Guido or python-dev should write/vet the justifications too. When you are putting up walls and specifically discouraging community participation on certain issues, it should be done in a sensitive way. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From pedronis at strakt.com Sun Jan 8 22:19:17 2006 From: pedronis at strakt.com (Samuele Pedroni) Date: Sun, 08 Jan 2006 22:19:17 +0100 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <43C17965.5070408@colorstudy.com> References: <611494241.20060106005601@gmail.com> <20060106193448.GA656@code1.codespeak.net> <20060108122527.GH18916@xs4all.nl> <43C10704.80404@v.loewis.de> <20060108163712.GK18916@xs4all.nl> <1f7befae0601080916n2dc1388et14578ddc33c5563d@mail.gmail.com> <43C17965.5070408@colorstudy.com> Message-ID: <43C181D5.8040702@strakt.com> Ian Bicking wrote: > Tim Peters wrote: > >>[Thomas Wouters] >> >> >>>My point isn't that it isn't archived somewhere (mailinglists, wiki, FAQ, >>>the minds of many, many people, not just Python developers) but that it >>>isn't easily findable and it isn't easily accessible in a single location. I >>>thought PEP's where supposed to be that, and if I have a particular idea for >>>new syntax or new semantics, PEPs would be the place I'd look, not the FAQ >>>or a Wiki. >> >> >>Luckily, in his benevolent infinite wisdom, I expect Guido reserved >>PEP number 13 for exactly this purpose: for a meta-PEP to record the >>unlucky PEP ideas that are so unlikely to get accepted that it's not >>worth anyone's time to write an actual PEP for them. I like the >>title: >> >> Don't Bother: PEPs Rejected Before Being Written >> >>No, I'm not kidding. At least I don't think I am. > > > +1. I think it's rather patronizing to send things back to python-list > when you know people are wasting their time discussing them because they > will never be accepted. Well, the problem is that most of these proposers think that their proposal deserve attention, is valuable no matter what, but python-dev is not the place to defuse/clarify that in a gentle way for the proposer. Of course some people do exactly that here out of kindness etc., I personally don't think it's a good approach. python-list is a better place to get clarifications about the way Python is, will stay. Half-baked, not well thought out ideas, as huge threads in the past going nowhere have proven, are not healthy to python-dev s/n ratio. Even with such a valuable PEP in place, I expect some people to need clarification discussions and python-list will still be the right place to have them. > People on python-list have useful things to do > too, they don't just read to waste their time. > > I would prefer PEP form over a wiki page, as I'd rather this be truly > authoritative, and only Guido can really completely reject an idea. > Justifying the rejections can be done by anyone though; maybe each idea > could link to a wiki page on the topic. > > I think it's also important to be clear on what's being rejected. Often > these rejected ideas address a real issue, and if you think about the > issue from another angle you might be able to solve that without falling > into the trap that the oft-rejected proposal fell into. But it's easy > to confuse that the issue or use case is being explicitly ignored, > rather than the particulars. For instance, Thomas said "changing all > statements into expressions (e.g. logix): python isn't (going to be) a > functional language" -- and that's what shouldn't be in the PEP. All > statements aren't going to be expressions; the editorialization that > Python isn't going to be a functional language is both rather > inaccurate, misses the real reason for statements, and needlessly > alienates people who like functional programming (and they have been > *needlessly* alienated by discussions about lambda and filter). > > So... maybe Guido or python-dev should write/vet the justifications too. > When you are putting up walls and specifically discouraging community > participation on certain issues, it should be done in a sensitive way. > > From fredrik at pythonware.com Sun Jan 8 22:02:19 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sun, 8 Jan 2006 22:02:19 +0100 Subject: [Python-Dev] [Doc-SIG] that library reference, again References: <4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com> <43B4A238.2030101@python.org> <51A7717B-3832-477E-BED5-3C36DCA20336@lag.net><43B56EEE.9090904@egenix.com> Message-ID: > anyone knows anything about support for semantic markup ? http://meta.wikimedia.org/wiki/Semantic_MediaWiki not sure a full-blown RDF-in-wiki-syntax is really that optimal, though ;-) From nnorwitz at gmail.com Sun Jan 8 22:47:12 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 8 Jan 2006 13:47:12 -0800 Subject: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py In-Reply-To: References: Message-ID: On 1/8/06, Fredrik Lundh wrote: > > many test frameworks support "expected failures" for this purpose. > how hard would it be to add a > > unittest.FailingTestCase > > class that runs a TestCase, catches any errors in it, and signals an > error ("test foo passed unexpectedly") if it runs cleanly ? http://python.org/sf/1399935 From thomas at xs4all.net Sun Jan 8 22:47:47 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Sun, 8 Jan 2006 22:47:47 +0100 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <43C17965.5070408@colorstudy.com> References: <611494241.20060106005601@gmail.com> <20060106193448.GA656@code1.codespeak.net> <20060108122527.GH18916@xs4all.nl> <43C10704.80404@v.loewis.de> <20060108163712.GK18916@xs4all.nl> <1f7befae0601080916n2dc1388et14578ddc33c5563d@mail.gmail.com> <43C17965.5070408@colorstudy.com> Message-ID: <20060108214747.GM18916@xs4all.nl> On Sun, Jan 08, 2006 at 02:43:17PM -0600, Ian Bicking wrote: > [T]he editorialization that Python isn't going to be a functional language > is both rather inaccurate, misses the real reason for statements, and > needlessly alienates people who like functional programming > So... maybe Guido or python-dev should write/vet the justifications too. Oh, none of my examples were anything but that. Particularly the justification. These were just quick ramblings from my side; all actual pronouncements should be made channeling or being Guido, and the editorial review to ease sensitive minds is a group effort. (On the topic you quoted, though, I do feel that if people are using Python while they are waiting for Python to turn into a functional language, they're waiting for the wrong thing. Or at least, I hope so, or *I'm* waiting for the wrong thing. So I'm not sure what's wrong about the editorialization, but perhaps I'm just too insensitive towards functional programming disciples. And I'm usually quite in tune with sensitive minds. But we can discuss that off-list ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From ianb at colorstudy.com Sun Jan 8 23:59:57 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Sun, 08 Jan 2006 16:59:57 -0600 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <20060108214747.GM18916@xs4all.nl> References: <611494241.20060106005601@gmail.com> <20060106193448.GA656@code1.codespeak.net> <20060108122527.GH18916@xs4all.nl> <43C10704.80404@v.loewis.de> <20060108163712.GK18916@xs4all.nl> <1f7befae0601080916n2dc1388et14578ddc33c5563d@mail.gmail.com> <43C17965.5070408@colorstudy.com> <20060108214747.GM18916@xs4all.nl> Message-ID: <43C1996D.8060902@colorstudy.com> Thomas Wouters wrote: >>[T]he editorialization that Python isn't going to be a functional language >>is both rather inaccurate, misses the real reason for statements, and >>needlessly alienates people who like functional programming > > >>So... maybe Guido or python-dev should write/vet the justifications too. > > > Oh, none of my examples were anything but that. Particularly the > justification. These were just quick ramblings from my side; all actual > pronouncements should be made channeling or being Guido, and the editorial > review to ease sensitive minds is a group effort. I was also just using your example as an example ;) As another similar example, around the discussions of the deprecation of map and filter I see a lot of misunderstandings. People confused the deprecation of two Lisp functions with Guido wanting to discourage functional program in general. The whole thing is based on several misconceptions, but the misunderstanding has become very widespread. A big part of what happened (I think) is that people who defended the decision added editorialization that became confused with the original intentions. And that consisted of things like "functional programming is hard/confusing, so it's not appropriate in Python". I think putting these kinds of decisions on a wiki would generally lead to these kinds of confusions. When rejecting an idea altogether I think it should be clear exactly what is being rejected, and not imply that anything extra is being rejected. People read a lot I think there is a valid perception that the Python community is not very open to many ideas about changing the language. I think that's okay -- it's this way *because* of the participation and discussion that has occurred in the past, because these are old ideas that have been rejected and we are trying to move on. But it leaves people in an awkward situation, because how can you really know what is open for discussion? I don't think people on python-list really know this either, so asking there won't clarify that. I doubt there's general consensus on python-dev about what is open for change. I just don't want people to feel discouraged when they try to contribute to the Python community and a PEP 13 could help direct people towards areas where their contributions are more likely to be useful. Also I think it is unfair to use python-list to clarify things that python-dev is not willing to clarify itself. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From martin at v.loewis.de Mon Jan 9 00:31:46 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 09 Jan 2006 00:31:46 +0100 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <43C1996D.8060902@colorstudy.com> References: <611494241.20060106005601@gmail.com> <20060106193448.GA656@code1.codespeak.net> <20060108122527.GH18916@xs4all.nl> <43C10704.80404@v.loewis.de> <20060108163712.GK18916@xs4all.nl> <1f7befae0601080916n2dc1388et14578ddc33c5563d@mail.gmail.com> <43C17965.5070408@colorstudy.com> <20060108214747.GM18916@xs4all.nl> <43C1996D.8060902@colorstudy.com> Message-ID: <43C1A0E2.6030605@v.loewis.de> Ian Bicking wrote: > I just don't want people to feel discouraged when they try to contribute > to the Python community and a PEP 13 could help direct people towards > areas where their contributions are more likely to be useful. Also I > think it is unfair to use python-list to clarify things that python-dev > is not willing to clarify itself. But now: who is going to write it? "Guido should write it" clearly won't work. And no, I'm explicitly not volunteering either. Regards, Martin From martin at v.loewis.de Mon Jan 9 00:33:27 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 09 Jan 2006 00:33:27 +0100 Subject: [Python-Dev] locale and LC_NUMERIC In-Reply-To: References: Message-ID: <43C1A147.60409@v.loewis.de> Georg Brandl wrote: > >>> import locale > >>> locale.setlocale(locale.LC_NUMERIC, "") > 'de_DE at euro' > >>> "%f" % 1.0 > '1.000000' > >>> u"%f" % 1.0 > u'1,000000' > >>> > > > Is this intended? This breaks test_format on my box when test_builtin (method > test_float_with_comma) is executed first. No. %-formatting should always use the C locale. One should use locale.format to get locale-aware formatting. Regards, Martin From pedronis at strakt.com Mon Jan 9 00:51:42 2006 From: pedronis at strakt.com (Samuele Pedroni) Date: Mon, 09 Jan 2006 00:51:42 +0100 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <43C1996D.8060902@colorstudy.com> References: <611494241.20060106005601@gmail.com> <20060106193448.GA656@code1.codespeak.net> <20060108122527.GH18916@xs4all.nl> <43C10704.80404@v.loewis.de> <20060108163712.GK18916@xs4all.nl> <1f7befae0601080916n2dc1388et14578ddc33c5563d@mail.gmail.com> <43C17965.5070408@colorstudy.com> <20060108214747.GM18916@xs4all.nl> <43C1996D.8060902@colorstudy.com> Message-ID: <43C1A58E.1050200@strakt.com> Ian Bicking wrote: > I just don't want people to feel discouraged when they try to contribute > to the Python community and a PEP 13 could help direct people towards > areas where their contributions are more likely to be useful. but people have a lot of options, probably more effective, ranging from writing great applications in Python, great libraries ... plus implementation work before they are left with the hard option that is language design to contribute to the community. > Also I > think it is unfair to use python-list to clarify things that python-dev > is not willing to clarify itself. > notice that the intersection is not empty yet. Also PEP 1 contains """ PEP authors are responsible for collecting community feedback on a PEP before submitting it for review. A PEP that has not been discussed on python-list at python.org and/or python-dev at python.org will not be accepted. However, wherever possible, long open-ended discussions on public mailing lists should be avoided. Strategies to keep the discussions efficient include: setting up a separate SIG mailing list for the topic, having the PEP author accept private comments in the early design phases, setting up a wiki page, etc. PEP authors should use their discretion here. """ in the past it often happened that water testing, honing for a PEP happend on python-list before any long discussion went on on python-dev. Basically I think that throwing half-cooked ideas at python-dev, especially for people with no track-record of language design contributions to Python, is just a recipe for frustration. From tim.peters at gmail.com Mon Jan 9 00:47:12 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 8 Jan 2006 18:47:12 -0500 Subject: [Python-Dev] [Python-checkins] r41972 - python/branches/ssize_t/Objects/funcobject.c In-Reply-To: <43C15167.7000702@v.loewis.de> References: <20060108060915.A69E01E4002@bag.python.org> <43C0DF8C.7050104@v.loewis.de> <1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com> <43C15167.7000702@v.loewis.de> Message-ID: <1f7befae0601081547h50cbce19w328e30aa6b1e1f5a@mail.gmail.com> [Tim] >> That handles C99-ish platforms by defalt. Other platforms (like >> Windows) would need to supply their own expansion in their pyconfig.h. >> Neal's format would become the god-awful >> but platform-neutral: >> >> "%s() requires a code object with %" Py_SIZE_T_WIDTH "d free vars," >> " not %" Py_SIZE_T_WIDTH "d" [Martin] > That's too ugly, IMO. So I could accept to write > > "%s() requires a code object with %ld free vars," > " not %ld", PyString_AsString(op->func_name), > (long)nclosure, (long)nfree); > > instead. Alternatively, if there is a real chance that it overflows > LONG_MAX (which there isn't, AFAIK - we don't support that many free > vars), we could also write > > "%s() requires a code object with %ld free vars," > " not %ld", PyString_AsString(op->func_name), > Py_long_ceil(nclosure), Py_long_ceil(nfree)); > > with > > #define Py_long_ceil(n) ((n) > LONG_MAX ? LONG_MAX : \ > (n) < LONG_MIN ? LONG_MIN : (n)) > > On most platforms, the compiler should notice that the conditions > are both known false, and collaps this to (n). > > If this ever overflows, somebody will have to remember that > this is really +/- infinity. I think you're trying to solve a different problem. I'm not trying to special-case the snot out of Neil's specific line of code, along the lines of "well, yes, the types here were changed from int to Py_ssize_t, but that was basically bizarre since in fact plain old int was big enough to hold all plausible values to begin with, so let's find a way to pass Py_ssize_t arguments in cases where they're really ints or longs". If that's what's going on here, it's a symptom of changing type declarations with little thought just to shut up compiler warnings. If `int` or `long` are in fact big enough to hold all plausible values for these variables, then that's what they should be declared as. The problem I am trying to solve is formatting values that are of types size_t or Py_ssize_t for _good_ reason: in effect, they really need to be 64 bits on 64-bit boxes, and "unsigned long"/"long" aren't good enough for that on all 64-bit platforms. In these cases, using %ld is wrong, and truncating bits is also wrong. These cases need a platform-correct format code. The pain of separating out the width modifier could be reduced by defining a few type-oriented macros instead; e.g., #define PyFORMAT_ssize_t "%zd" /* on C99 boxes */ If it made sense to use Py_ssize_t to begin with in that code, then: "%s() requires a code object with " PyFORMAT_ssize_t " free vars," " not " PyFORMAT_ssize_t, PyString_AsString(op->func_name), nclosure, free); That's no more or less painful than using C99's huge pile of PRId8, PRId16, PRId32 (etc, etc) macros, defined there for similar purposes. From tim.peters at gmail.com Mon Jan 9 01:00:54 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 8 Jan 2006 19:00:54 -0500 Subject: [Python-Dev] [Python-checkins] r41972 - python/branches/ssize_t/Objects/funcobject.c In-Reply-To: <43C15307.2060600@v.loewis.de> References: <20060108060915.A69E01E4002@bag.python.org> <43C0DF8C.7050104@v.loewis.de> <1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com> <1f7befae0601080942o3a7dc194ub790b616bb79c2fe@mail.gmail.com> <43C15307.2060600@v.loewis.de> Message-ID: <1f7befae0601081600g42ea19bdtfac2368cdd51ba51@mail.gmail.com> [Martin v. L?wis] > On VC7.1, we could use 'L', right? We could use the "I" (capital letter eye) length modifier under VC7.1. That's good for both size_t and ptrdiff_t formats under VC7.1, where ptrdiff_t under VC7.1 is really the same concept as Py_ssize_t. On 32-bit boxes, "I" means 4 bytes, and on 64-bit boxes "I" means 8 bytes. > On other platforms, we could check whether sizeof(size_t) is sizeof(long), and > use 'l', else we could refuse compilation (I doubt this would rule out any relevant > system). Assuming that's a lowercase letter el (I really can't see any difference between upper-eye and lower-el in gmail's font), I think that would be fine. From thomas at xs4all.net Mon Jan 9 01:51:23 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Mon, 9 Jan 2006 01:51:23 +0100 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <43C1A0E2.6030605@v.loewis.de> References: <20060108122527.GH18916@xs4all.nl> <43C10704.80404@v.loewis.de> <20060108163712.GK18916@xs4all.nl> <1f7befae0601080916n2dc1388et14578ddc33c5563d@mail.gmail.com> <43C17965.5070408@colorstudy.com> <20060108214747.GM18916@xs4all.nl> <43C1996D.8060902@colorstudy.com> <43C1A0E2.6030605@v.loewis.de> Message-ID: <20060109005123.GN18916@xs4all.nl> On Mon, Jan 09, 2006 at 12:31:46AM +0100, "Martin v. L?wis" wrote: > Ian Bicking wrote: > > I just don't want people to feel discouraged when they try to contribute > > to the Python community and a PEP 13 could help direct people towards > > areas where their contributions are more likely to be useful. Also I > > think it is unfair to use python-list to clarify things that python-dev > > is not willing to clarify itself. > But now: who is going to write it? "Guido should write it" clearly won't > work. And no, I'm explicitly not volunteering either. Well, the PEP will be mostly boilerplate anyway (unless there's a sudden influx of old ideas) so I'm sure I can whip something up before next weekend. I'll probably keep the actual list of rejected items to implicit self (which started the whole discussion), upgrading the Python parser beyond a simple LL(1), and maybe implicit ()'s for functioncalls, if I feel inspired. The exact wording, and additional rejects, can be discussed at length by those interested. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From tim.peters at gmail.com Mon Jan 9 01:59:43 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 8 Jan 2006 19:59:43 -0500 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <20060109005123.GN18916@xs4all.nl> References: <20060108122527.GH18916@xs4all.nl> <43C10704.80404@v.loewis.de> <20060108163712.GK18916@xs4all.nl> <1f7befae0601080916n2dc1388et14578ddc33c5563d@mail.gmail.com> <43C17965.5070408@colorstudy.com> <20060108214747.GM18916@xs4all.nl> <43C1996D.8060902@colorstudy.com> <43C1A0E2.6030605@v.loewis.de> <20060109005123.GN18916@xs4all.nl> Message-ID: <1f7befae0601081659h3567dfcbs1283d3c8ef82dc94@mail.gmail.com> [Martin] >> But now: who is going to write it? "Guido should write it" clearly won't >> work. And no, I'm explicitly not volunteering either. [Thomas] > Well, the PEP will be mostly boilerplate anyway (unless there's a sudden > influx of old ideas) so I'm sure I can whip something up before next > weekend. I'll probably keep the actual list of rejected items to implicit > self (which started the whole discussion), upgrading the Python parser > beyond a simple LL(1), and maybe implicit ()'s for functioncalls, if I feel > inspired. The exact wording, and additional rejects, can be discussed at > length by those interested. I'll help with this. That means I want to and intend to, and will discover that I can't actually make any time for it. With two of us putting it off indefinitely, it should get done in half the time ;-). From Scott.Daniels at Acm.Org Mon Jan 9 02:10:59 2006 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Sun, 08 Jan 2006 17:10:59 -0800 Subject: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py In-Reply-To: References: <43C14997.30309@v.loewis.de> <200601081234.59951.fdrake@acm.org> <43C15370.2070700@v.loewis.de> Message-ID: James Y Knight wrote: > On Jan 8, 2006, at 1:01 PM, Martin v. L?wis wrote: > >> Fred L. Drake, Jr. wrote: >> >>> I like the way trial (from twisted) supports this. The test >>> method is written >>> normally, in whatever class makes sense. Then the test is marked >>> with an >>> attribute to say it isn't expected to pass yet. When the code is >>> fixed and >>> the test passes, you get that information in trial's output, and >>> can unmark >>> the test. This avoids having to refactor test classes just to >>> update the >>> status of a test. >>> >> So how is the mark added? I would suggest >> >> @xfail >> def test_foo(self): >> self.assertEquals(0,1) > > def test_foo(self): > self.assertEquals(0,1) > test_foo.todo = "Why this test fails description message." Looks pretty good. Here's some code to critique: import unittest class BrokenTest(unittest.TestCase.failureException): def __repr__(self): return '%s: %s: %s works now' % ( (self.__class__.__name__,) + self.args) def known_broken_XXX(reason): def wrapper(test_method): def replacement(*args, **kwargs): try: test_method(*args, **kwargs) except unittest.TestCase.failureException: pass else: raise BrokenTest(test_method.__name__, reason) return replacement wrapper.todo = reason return wrapper So your use looks like: class SomeTests(unittest.TestCase): ... @known_broken_XXX('Someday the naive expect this to work') def test_distinct_ones(self): self.failIfEqual(1, 1.0) -- Scott David Daniels Scott.Daniels at Acm.Org From brett at python.org Mon Jan 9 02:24:26 2006 From: brett at python.org (Brett Cannon) Date: Sun, 8 Jan 2006 17:24:26 -0800 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <1f7befae0601081659h3567dfcbs1283d3c8ef82dc94@mail.gmail.com> References: <43C10704.80404@v.loewis.de> <20060108163712.GK18916@xs4all.nl> <1f7befae0601080916n2dc1388et14578ddc33c5563d@mail.gmail.com> <43C17965.5070408@colorstudy.com> <20060108214747.GM18916@xs4all.nl> <43C1996D.8060902@colorstudy.com> <43C1A0E2.6030605@v.loewis.de> <20060109005123.GN18916@xs4all.nl> <1f7befae0601081659h3567dfcbs1283d3c8ef82dc94@mail.gmail.com> Message-ID: On 1/8/06, Tim Peters wrote: > [Martin] > >> But now: who is going to write it? "Guido should write it" clearly won't > >> work. And no, I'm explicitly not volunteering either. > > [Thomas] > > Well, the PEP will be mostly boilerplate anyway (unless there's a sudden > > influx of old ideas) so I'm sure I can whip something up before next > > weekend. I'll probably keep the actual list of rejected items to implicit > > self (which started the whole discussion), upgrading the Python parser > > beyond a simple LL(1), and maybe implicit ()'s for functioncalls, if I feel > > inspired. The exact wording, and additional rejects, can be discussed at > > length by those interested. > > I'll help with this. That means I want to and intend to, and will > discover that I can't actually make any time for it. With two of us > putting it off indefinitely, it should get done in half the time ;-). Make it a third. I am willing to help out as needed since I already keep an eye out for Py3K stuff. Watching for stuff we don't want to discuss again is not that much harder. =) -Brett From warner at lothar.com Mon Jan 9 02:38:10 2006 From: warner at lothar.com (Brian Warner) Date: Sun, 08 Jan 2006 17:38:10 -0800 (PST) Subject: [Python-Dev] buildbot In-Reply-To: <43BD9B32.3030700@v.loewis.de> References: <20060104.003058.48524852.warner@lothar.com> <43BD9B32.3030700@v.loewis.de> Message-ID: <20060108.173810.91561560.warner@lothar.com> > From: "Martin v. L?wis" > > So at the moment, I'm quite happy with buildbot. My only wish is that > it would do static pages, rather than being a web server. I set it up > as a reverse proxy (so nobody knows what the port number is, but still). As a bit of history, the buildbot's predecessor used static pages.. it just dumped everything into HTML files any time something changed, and then I pointed apache at the directory containing these files. By the time I started the current Buildbot project, I had become familiar with Twisted, and elected to use its built-in web server to provide status information. This makes it much easier to implement things like the "Force Build" button, and allows the same status-retrieving API to be used by all the different status-delivery modules: web, IRC, email, PB. On the other hand, it makes it slightly harder to merge the buildbot's web space into that of another web server (unless that other web server also happens to be Twisted: the buildbot can run as a twisted.web.distrib sub-server, which is how the Twisted buildbot presents status). The reverse proxy is a great way to improve the integration of the buildbot's web pages into the rest of your site's. I'm hoping to improve the web-status plugin to make it easier to change around the page layout from the buildbot side, but even with that you'll want the reverse-proxy to remove the port-number from the URL. I should mention a feature that you might find useful: you can add some query arguments to the top-level Waterfall page's URL to restrict the set of Builders that are displayed. If you append "?show=full-2.3", you'll wind up with a waterfall display that only contains the "full-2.3" Builder's column. Appending "?show=full-2.3&show=full-2.4" will show two columns, etc. You can also create multiple Waterfall instances, each with a different set of "categories", and use that to manage the complexity of displaying status for lots of Builders. (with the current release these multiple Waterfalls must all listen on different TCP ports, unfortunately, but you can always hide this with the reverse-proxy's URL mapping). So you could have one page that shows *all* the builders, and another set of pages that only show specific ones (perhaps one primary builder for each architecture). If the reverse-proxy or some sort of Apache URL-rewriting allows you to add query parameters to the target URLs, you might be able to hide these filtered views behind simple-looking status URLs. It's not the greatest solution, but it might let you add more Builders without making the display unworkably complex. cheers, -Brian From ianb at colorstudy.com Mon Jan 9 02:41:02 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Sun, 08 Jan 2006 19:41:02 -0600 Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0 In-Reply-To: <43C1A58E.1050200@strakt.com> References: <611494241.20060106005601@gmail.com> <20060106193448.GA656@code1.codespeak.net> <20060108122527.GH18916@xs4all.nl> <43C10704.80404@v.loewis.de> <20060108163712.GK18916@xs4all.nl> <1f7befae0601080916n2dc1388et14578ddc33c5563d@mail.gmail.com> <43C17965.5070408@colorstudy.com> <20060108214747.GM18916@xs4all.nl> <43C1996D.8060902@colorstudy.com> <43C1A58E.1050200@strakt.com> Message-ID: <43C1BF2E.1040506@colorstudy.com> Samuele Pedroni wrote: > Ian Bicking wrote: > >>I just don't want people to feel discouraged when they try to contribute >>to the Python community and a PEP 13 could help direct people towards >>areas where their contributions are more likely to be useful. > > > but people have a lot of options, probably more effective, ranging > from writing great applications in Python, great libraries ... plus > implementation work before they are left with the hard option that is > language design to contribute to the community. I completely agree. >>Also I >>think it is unfair to use python-list to clarify things that python-dev >>is not willing to clarify itself. >> > > > notice that the intersection is not empty yet. > > Also PEP 1 contains > > """ > PEP authors are responsible for collecting community feedback on a PEP > before submitting it for review. A PEP that has not been discussed on > python-list at python.org and/or python-dev at python.org will not be > accepted. However, wherever possible, long open-ended discussions on > public mailing lists should be avoided. Strategies to keep the > discussions efficient include: setting up a separate SIG mailing list > for the topic, having the PEP author accept private comments in the > early design phases, setting up a wiki page, etc. PEP authors should use > their discretion here. > """ > > in the past it often happened that water testing, honing > for a PEP happend on python-list before any long discussion went on > on python-dev. Basically I think that throwing half-cooked ideas at > python-dev, especially for people with no track-record of language > design contributions to Python, is just a recipe for frustration. Sure; lots of PEPs can go to python-list, and it's fine if people ask that a PEP that goes to python-dev be discussed on python-list. But only PEPs that have some chance of being accepted. In fact, these things come up on python-list far more often than python-dev, so a PEP 13 could also help clean those conversations out of that list too. Or at least if people want to discuss those things, they'll be aware that the discussion is only academic. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From martin at v.loewis.de Mon Jan 9 08:23:37 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 09 Jan 2006 08:23:37 +0100 Subject: [Python-Dev] [Python-checkins] r41972 - python/branches/ssize_t/Objects/funcobject.c In-Reply-To: <1f7befae0601081547h50cbce19w328e30aa6b1e1f5a@mail.gmail.com> References: <20060108060915.A69E01E4002@bag.python.org> <43C0DF8C.7050104@v.loewis.de> <1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com> <43C15167.7000702@v.loewis.de> <1f7befae0601081547h50cbce19w328e30aa6b1e1f5a@mail.gmail.com> Message-ID: <43C20F79.4020908@v.loewis.de> Tim Peters wrote: > That's no more or less painful than using C99's huge pile of PRId8, > PRId16, PRId32 (etc, etc) macros, defined there for similar purposes. Right - and I consider them just as painful. If you believe that this is really what we should be doing, then, well, let's do it. Regards, Martin From nnorwitz at gmail.com Mon Jan 9 08:41:29 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 8 Jan 2006 23:41:29 -0800 Subject: [Python-Dev] locale and LC_NUMERIC In-Reply-To: <43C1A147.60409@v.loewis.de> References: <43C1A147.60409@v.loewis.de> Message-ID: On 1/8/06, "Martin v. L?wis" wrote: > Georg Brandl wrote: > > >>> "%f" % 1.0 > > '1.000000' > > >>> u"%f" % 1.0 > > u'1,000000' > > > > Is this intended? This breaks test_format on my box when test_builtin (method > > test_float_with_comma) is executed first. > > No. %-formatting should always use the C locale. One should use > locale.format to get locale-aware formatting. I know very little about locale's. /f assigned me a bug http://python.org/sf/1391872 which suggests I run all the tests in a different locale than C. I think this is a good idea, but when I set LANG or LC_ALL or LC_CTYPE in the env't the locale isn't set without me calling locale.setlocale(). Shouldn't we maintain the value from the env't? The problem I found is in Python/pythonrun.c. At least I think that's the problem. Around line 230-240 we call setlocale(NULL), then call setlocale(""). Then later on we restore the locale by calling setlocale() with the return result from setlocale(NULL). The man page on my box (gentoo) says that passing NULL returns the current locale() while passing "" sets the locale from the env't variable. Should we just capture the results of setlocale("")? If my understanding is wrong, how can I start python with the correct locale? It seems like that I'm confused since this problem hasn't seemed to come up before. n From martin at v.loewis.de Mon Jan 9 08:48:47 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 09 Jan 2006 08:48:47 +0100 Subject: [Python-Dev] [Python-checkins] r41972 - python/branches/ssize_t/Objects/funcobject.c In-Reply-To: <1f7befae0601081600g42ea19bdtfac2368cdd51ba51@mail.gmail.com> References: <20060108060915.A69E01E4002@bag.python.org> <43C0DF8C.7050104@v.loewis.de> <1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com> <1f7befae0601080942o3a7dc194ub790b616bb79c2fe@mail.gmail.com> <43C15307.2060600@v.loewis.de> <1f7befae0601081600g42ea19bdtfac2368cdd51ba51@mail.gmail.com> Message-ID: <43C2155F.8010305@v.loewis.de> Tim Peters wrote: > We could use the "I" (capital letter eye) length modifier under VC7.1. > That's good for both size_t and ptrdiff_t formats under VC7.1, where > ptrdiff_t under VC7.1 is really the same concept as Py_ssize_t. ptrdiff_t has the advantage of being available on all platforms, being part of C89 (IIRC). Should we use ptrdiff_t instead of Py_ssize_t? Formally, ptrdiff_t could be different from size_t (in width); reportedly, there are 8086 compilers which had a 16-bit size_t (maximum size of a segment), but a 32-bit ptrdiff_t (allowing for cross-segment differences, something that apparently became undefined in C99). Regards, Martin From fredrik at pythonware.com Mon Jan 9 08:51:02 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Mon, 9 Jan 2006 08:51:02 +0100 Subject: [Python-Dev] locale and LC_NUMERIC References: <43C1A147.60409@v.loewis.de> Message-ID: Neal Norwitz wrote: > > No. %-formatting should always use the C locale. One should use > > locale.format to get locale-aware formatting. > > I know very little about locale's. /f assigned me a bug > http://python.org/sf/1391872 which suggests I run all the tests in a > different locale than C. I think this is a good idea, but when I set > LANG or LC_ALL or LC_CTYPE in the env't the locale isn't set without > me calling locale.setlocale(). Shouldn't we maintain the value from > the env't? the locale is a global program setting, and should only be "activated" when it's actually needed. my rationale for running the tests with a non-US locale was to increase the chance of catching bugs where the external locale setting affects Python's behaviour. maybe it would be a good idea to add a "use setlocale" flag to the test- runner ? From martin at v.loewis.de Mon Jan 9 08:57:13 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 09 Jan 2006 08:57:13 +0100 Subject: [Python-Dev] locale and LC_NUMERIC In-Reply-To: References: <43C1A147.60409@v.loewis.de> Message-ID: <43C21759.2090301@v.loewis.de> Neal Norwitz wrote: > I know very little about locale's. /f assigned me a bug > http://python.org/sf/1391872 which suggests I run all the tests in a > different locale than C. I think this is a good idea, but when I set > LANG or LC_ALL or LC_CTYPE in the env't the locale isn't set without > me calling locale.setlocale(). Shouldn't we maintain the value from > the env't? I feel I'm lacking some link here: why do you think we should do that? > The problem I found is in Python/pythonrun.c. At least I think that's > the problem. Around line 230-240 we call setlocale(NULL), then call > setlocale(""). Then later on we restore the locale by calling > setlocale() with the return result from setlocale(NULL). Right. > The man page on my box (gentoo) says that passing NULL returns the > current locale() while passing "" sets the locale from the env't > variable. Should we just capture the results of setlocale("")? Again: why? > If my understanding is wrong, how can I start python with the correct > locale? What does "start with the correct locale" mean? What is the "correct locale"? Python should work in any locale, and locale.format should always give the locale-specific formatting, whereas "%f" % value should always give the result from the "C" locale (regardless of whether the application has called setlocale or not). Regards, Martin From martin at v.loewis.de Mon Jan 9 08:58:30 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 09 Jan 2006 08:58:30 +0100 Subject: [Python-Dev] locale and LC_NUMERIC In-Reply-To: References: <43C1A147.60409@v.loewis.de> Message-ID: <43C217A6.1080906@v.loewis.de> Fredrik Lundh wrote: > maybe it would be a good idea to add a "use setlocale" flag to the test- > runner ? Sure! Martin From nnorwitz at gmail.com Mon Jan 9 09:24:18 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 9 Jan 2006 00:24:18 -0800 Subject: [Python-Dev] locale and LC_NUMERIC In-Reply-To: <43C21759.2090301@v.loewis.de> References: <43C1A147.60409@v.loewis.de> <43C21759.2090301@v.loewis.de> Message-ID: On 1/8/06, "Martin v. L?wis" wrote: > Neal Norwitz wrote: > > I know very little about locale's. /f assigned me a bug > > http://python.org/sf/1391872 which suggests I run all the tests in a > > different locale than C. I think this is a good idea, but when I set > > LANG or LC_ALL or LC_CTYPE in the env't the locale isn't set without > > me calling locale.setlocale(). Shouldn't we maintain the value from > > the env't? > > I feel I'm lacking some link here: why do you think we should do that? neal at janus ~/build/python/svn/clean-ish $ LC_ALL=de_DE ./python >>> import locale >>> locale.setlocale(locale.LC_ALL) 'C' >>> locale.setlocale(locale.LC_ALL, 'de_DE') 'de_DE' I would have expected the first call to setlocale() to return de_DE. /f solution of adding a flag to the test runner should work for the bug report I mentioned. Part of the reason why I asked was because I'm confused by the current behaviour. Is the first call to setlocale() supposed to return C? If so, always? I was looking for some way for the initial call to setlocale() to return de_DE or whatever other locale I set. > What does "start with the correct locale" mean? What is the "correct > locale"? Python should work in any locale, Hopefully my explanation above answered all those questions. > and locale.format > should always give the locale-specific formatting, whereas "%f" % value > should always give the result from the "C" locale (regardless of > whether the application has called setlocale or not). Right. I'm working on a patch for this problem which is independent of what I was asking. I really should have started a new thread, but I wasn't completely sure when I asked if there was a relationship. Also, in Georg's snippet, he didn't show that he was setting the locale, only what it was set to. n From fredrik at pythonware.com Mon Jan 9 09:54:09 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Mon, 9 Jan 2006 09:54:09 +0100 Subject: [Python-Dev] locale and LC_NUMERIC References: <43C1A147.60409@v.loewis.de><43C21759.2090301@v.loewis.de> Message-ID: Neal Norwitz wrote: > > I feel I'm lacking some link here: why do you think we should do that? > > neal at janus ~/build/python/svn/clean-ish $ LC_ALL=de_DE ./python > >>> import locale > >>> locale.setlocale(locale.LC_ALL) > 'C' > >>> locale.setlocale(locale.LC_ALL, 'de_DE') > 'de_DE' > > I would have expected the first call to setlocale() to return de_DE. the locale is a program-specific setting (handled by the C runtime library), and is set to "C" by default. if you want to use a specific locale, you have to call setlocale to indicate that you really want a non-default locale. the "" argument is just a con- venience; it tells the C runtime library to set the locale based on OS-level language settings (e.g. LANG on Unix, control panel settings on Windows, etc). LANG in itself isn't the locale; it's just a way to tell a locale-aware program what the preferred locale is. From nnorwitz at gmail.com Mon Jan 9 10:12:31 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 9 Jan 2006 01:12:31 -0800 Subject: [Python-Dev] locale and LC_NUMERIC In-Reply-To: <43C1A147.60409@v.loewis.de> References: <43C1A147.60409@v.loewis.de> Message-ID: On 1/8/06, "Martin v. L?wis" wrote: > Georg Brandl wrote: > > >>> import locale > > >>> locale.setlocale(locale.LC_NUMERIC, "") > > 'de_DE at euro' > > >>> u"%f" % 1.0 > > u'1,000000' > > > > Is this intended? This breaks test_format on my box when test_builtin (method > > test_float_with_comma) is executed first. > > No. %-formatting should always use the C locale. One should use > locale.format to get locale-aware formatting. http://python.org/sf/1400181 fixes this problem I hope. n From mwh at python.net Mon Jan 9 10:19:46 2006 From: mwh at python.net (Michael Hudson) Date: Mon, 09 Jan 2006 09:19:46 +0000 Subject: [Python-Dev] test_curses In-Reply-To: (Georg Brandl's message of "Sun, 08 Jan 2006 20:04:45 +0100") References: Message-ID: <2m1wzhsqq5.fsf@starship.python.net> Georg Brandl writes: > The call to curses.setupterm() leaves my terminal in a bad state. Hmm. > The reset program outputs: > Erase set to delete. > Kill set to control-U (^U). > Interrupt set to control-C (^C). It always says that :) (unless you've messed with stty, I guess) > Doesn't the setupterm() have to be paired with something like shutdownterm()? Not as far as my memory of curses goes. From the man page: The setupterm routine reads in the terminfo database, initializing the terminfo structures, but does not set up the output virtualization structures used by curses. What platform are you on? Cheers, mwh -- #ifndef P_tmpdir printf( "Go buy a better computer" ); exit( ETHESKYISFALLINGANDIWANTMYMAMA ); -- Dimitri Maziuk on writing secure code, asr From benji at benjiyork.com Mon Jan 9 14:02:42 2006 From: benji at benjiyork.com (Benji York) Date: Mon, 09 Jan 2006 08:02:42 -0500 Subject: [Python-Dev] locale and LC_NUMERIC In-Reply-To: References: <43C1A147.60409@v.loewis.de> Message-ID: <43C25EF2.4050401@benjiyork.com> Fredrik Lundh wrote: > my rationale for running the tests with a non-US locale was to increase > the chance of catching bugs where the external locale setting affects > Python's behaviour. > > maybe it would be a good idea to add a "use setlocale" flag to the test- > runner ? Sounds like a good use for buildbot. After running the tests "normally" they could be re-run with a different locale. -- Benji York From facundobatista at gmail.com Mon Jan 9 18:08:15 2006 From: facundobatista at gmail.com (Facundo Batista) Date: Mon, 9 Jan 2006 14:08:15 -0300 Subject: [Python-Dev] ConfigParser to save with order In-Reply-To: References: Message-ID: 2006/1/7, Guido van Rossum : > I think it's moot unless you also preserve comments. Ideally would be > something that prserved everything (ordering, blank lines, comments > etc.) from how it was read in. Modifying a value should keep its > position. Adding a value should add it to the end of the section it's > in (unless there are cases where ordering is important to the > semantics -- are there?). Not in my case. We can rewrite ConfigParser to store everything and write it back in the exact order it took it, with new values at the end of each section (or where the user inserted it), but it'll took a big rewrite of the module: right now it's using dictionaries and should use lists to achieve that behaviour. What I wanted to add to the module was predicatibility: a very needed feature when you're writing test cases (and that's where I got bite). . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From fdrake at acm.org Mon Jan 9 18:17:50 2006 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Mon, 9 Jan 2006 12:17:50 -0500 Subject: [Python-Dev] ConfigParser to save with order In-Reply-To: References: Message-ID: <200601091217.51071.fdrake@acm.org> On Monday 09 January 2006 12:08, Facundo Batista wrote: > What I wanted to add to the module was predicatibility: a very needed > feature when you're writing test cases (and that's where I got bite). In that case, would sorting the keys within each section be sufficient when writing it back out? I had a class to do the really-careful editing bit like Guido described once, but lost it in a disk crash several years ago. I really wish I'd gotten that into revision control somewhere, but it's too late now. I've not needed that badly enough to re-write it. -Fred -- Fred L. Drake, Jr. From tim.peters at gmail.com Mon Jan 9 18:48:41 2006 From: tim.peters at gmail.com (Tim Peters) Date: Mon, 9 Jan 2006 12:48:41 -0500 Subject: [Python-Dev] [Python-checkins] r41972 - python/branches/ssize_t/Objects/funcobject.c In-Reply-To: <43C20F79.4020908@v.loewis.de> References: <20060108060915.A69E01E4002@bag.python.org> <43C0DF8C.7050104@v.loewis.de> <1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com> <43C15167.7000702@v.loewis.de> <1f7befae0601081547h50cbce19w328e30aa6b1e1f5a@mail.gmail.com> <43C20F79.4020908@v.loewis.de> Message-ID: <1f7befae0601090948k47f782baq1e46ed6306be778e@mail.gmail.com> ... [Tim] >> That's no more or less painful than using C99's huge pile of PRId8, >> PRId16, PRId32 (etc, etc) macros, defined there for similar purposes. [Martin] > Right - and I consider them just as painful. > > If you believe that this is really what we should be doing, then, well, > let's do it. I think it's clear that in a 64-bit world, we absolutely need a way to format 64-bit integers. I'm dubious that Neil's specific line of code needed Py_ssize_t to begin with, but increasing amounts of other code will. Maybe this wasn't clear: I think Neil's code _should_ be able to put "%zd" in the format string, not using format macros at all. IOW, I believe the supporting routines for PyErr_Format() and PyString_Format() should be taught to recognize the C99-standard "z" qualifier and implement its semantics in a platform-correct way. That's where the irritating macros would be used, _inside_ the implementations of Py{Err,String}_Format(). Code passing a Py_ssize_t to one of those would just use "%zd" directly in its format string, and on all platforms. It's only code calling printf/fprintf/sprintf directly with a Py_ssize_t or size_t output value that would need to use the format macros, and I don't think there's a lot of that -- looks like most relevant messages are constructed by Py{Err,String}_Format(). From tim.peters at gmail.com Mon Jan 9 19:14:10 2006 From: tim.peters at gmail.com (Tim Peters) Date: Mon, 9 Jan 2006 13:14:10 -0500 Subject: [Python-Dev] [Python-checkins] r41972 - python/branches/ssize_t/Objects/funcobject.c In-Reply-To: <43C2155F.8010305@v.loewis.de> References: <20060108060915.A69E01E4002@bag.python.org> <43C0DF8C.7050104@v.loewis.de> <1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com> <1f7befae0601080942o3a7dc194ub790b616bb79c2fe@mail.gmail.com> <43C15307.2060600@v.loewis.de> <1f7befae0601081600g42ea19bdtfac2368cdd51ba51@mail.gmail.com> <43C2155F.8010305@v.loewis.de> Message-ID: <1f7befae0601091014ic0cdebcv2d4ad0983c995aa8@mail.gmail.com> [Tim] >> We could use the "I" (capital letter eye) length modifier under VC7.1. >> That's good for both size_t and ptrdiff_t formats under VC7.1, where >> ptrdiff_t under VC7.1 is really the same concept as Py_ssize_t. [Martin] > ptrdiff_t has the advantage of being available on all platforms, > being part of C89 (IIRC). Should we use ptrdiff_t instead of > Py_ssize_t? Formally, ptrdiff_t could be different from size_t > (in width); reportedly, there are 8086 compilers which had > a 16-bit size_t (maximum size of a segment), but a 32-bit > ptrdiff_t (allowing for cross-segment differences, something > that apparently became undefined in C99). I grew up on 60- and 64-bit boxes, but all I've worked on for the last decade is 32-bit Pentium chips. If there's a Python platform where sizeof(size_t) != sizeof(ptrdiff_t), I don't know about it, but I'm not sure I have a reason to _expect_ to be aware of such things anymore. Regardless, I like Py_ssize_t: it clearly says "same width as size_t, but signed". "Difference between two pointers" is obscurely related to that at best. From nnorwitz at gmail.com Mon Jan 9 19:40:44 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 9 Jan 2006 10:40:44 -0800 Subject: [Python-Dev] [Python-checkins] r41972 - python/branches/ssize_t/Objects/funcobject.c In-Reply-To: <1f7befae0601090948k47f782baq1e46ed6306be778e@mail.gmail.com> References: <20060108060915.A69E01E4002@bag.python.org> <43C0DF8C.7050104@v.loewis.de> <1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com> <43C15167.7000702@v.loewis.de> <1f7befae0601081547h50cbce19w328e30aa6b1e1f5a@mail.gmail.com> <43C20F79.4020908@v.loewis.de> <1f7befae0601090948k47f782baq1e46ed6306be778e@mail.gmail.com> Message-ID: On 1/9/06, Tim Peters wrote: > ... > > [Tim] > >> That's no more or less painful than using C99's huge pile of PRId8, > >> PRId16, PRId32 (etc, etc) macros, defined there for similar purposes. > > [Martin] > > Right - and I consider them just as painful. > > > > If you believe that this is really what we should be doing, then, well, > > let's do it. > > I think it's clear that in a 64-bit world, we absolutely need a way to > format 64-bit integers. I'm dubious that Neil's specific line of code > needed Py_ssize_t to begin with, but increasing amounts of other code > will. I wasn't always sure which choice to make. I generally tried to leave the warnings when I wasn't sure. Like you and Martin I agree this is a problem, though I don't have any good ideas how to fix. They all seem kinda painful. I often chose to use Py_ssize_t rather than int if it was capturing the result of a sequence length. But often this is larger than what will be allowed (like in this case). The problem then comes that we have cases everywhere if we leave things as ints. Stuff like this: len = (int) PyTuple_Size(co_consts); How do we know that is safe or not? Also, maybe it would be nice to have defined maximum sizes enforced by the compiler for lots of fields in code objects, etc. Like 32k for # local variables, parameters, free variables, etc. But when we add these together in the stack for extras, we would still need to use at least 32 bits. And there's still the possibility of overflow when we add these pieces together. n From g.brandl-nospam at gmx.net Mon Jan 9 19:55:52 2006 From: g.brandl-nospam at gmx.net (Georg Brandl) Date: Mon, 09 Jan 2006 19:55:52 +0100 Subject: [Python-Dev] test_curses In-Reply-To: <2m1wzhsqq5.fsf@starship.python.net> References: <2m1wzhsqq5.fsf@starship.python.net> Message-ID: Michael Hudson wrote: > Georg Brandl writes: > >> The call to curses.setupterm() leaves my terminal in a bad state. > > Hmm. > >> The reset program outputs: >> Erase set to delete. >> Kill set to control-U (^U). >> Interrupt set to control-C (^C). > > It always says that :) (unless you've messed with stty, I guess) Well, when I do a reset without meddling with the terminal, it says nothing, at least on my box. And, there's more: Ctrl+D doesn't work, Ctrl+C doesn't work. I just looked, my .profile contains "stty sane cr0 pass8 dec". >> Doesn't the setupterm() have to be paired with something like shutdownterm()? > > Not as far as my memory of curses goes. From the man page: > > The setupterm routine reads in the terminfo database, > initializing the terminfo structures, but does not set up the > output virtualization structures used by curses. > > What platform are you on? Linux 2.6, ncurses 5.5, TERM=xterm. Georg From g.brandl-nospam at gmx.net Mon Jan 9 20:17:31 2006 From: g.brandl-nospam at gmx.net (Georg Brandl) Date: Mon, 09 Jan 2006 20:17:31 +0100 Subject: [Python-Dev] locale and LC_NUMERIC In-Reply-To: <43C1A147.60409@v.loewis.de> References: <43C1A147.60409@v.loewis.de> Message-ID: Martin v. L?wis wrote: > Georg Brandl wrote: >> >>> import locale >> >>> locale.setlocale(locale.LC_NUMERIC, "") >> 'de_DE at euro' >> >>> "%f" % 1.0 >> '1.000000' >> >>> u"%f" % 1.0 >> u'1,000000' >> >>> >> >> >> Is this intended? This breaks test_format on my box when test_builtin (method >> test_float_with_comma) is executed first. > > No. %-formatting should always use the C locale. One should use > locale.format to get locale-aware formatting. While we're at it: test_builtin does import locale orig_locale = locale.setlocale(locale.LC_NUMERIC, '') locale.setlocale(locale.LC_NUMERIC, 'fr_FR') and later finally: locale.setlocale(locale.LC_NUMERIC, orig_locale) Shouldn't the first setlocale have no second argument? regards, Georg From shane at hathawaymix.org Mon Jan 9 21:48:36 2006 From: shane at hathawaymix.org (Shane Hathaway) Date: Mon, 09 Jan 2006 13:48:36 -0700 Subject: [Python-Dev] Logging enhancements Message-ID: <43C2CC24.4090405@hathawaymix.org> I'd like to create a patch for the logging package, but before I do, I need to know whether someone else is working on the package and whether my patch is likely to be accepted. Is there another group I should talk to? Specifically, I want to change the following: - The logging.config.fileConfig function has bare excepts that hide configuration errors. I'd like to remove the bare excepts and break up fileConfig for clarity. - I'd like to make it possible to configure the format of exception tracebacks. Some logs want compressed tracebacks like Medusa; others want extra information like Zope. - There is no support for external log rotation. There is plenty of support for Python rotating logs directly, but permission settings often make that impossible. Long-running Python processes need to accept a signal which notifies them when an external tool rotates the logs. - The logging documentation is lacking somewhat. For example, it fails to mention that the logging handlers are in the 'handlers' module, which I only learned by searching the source code. Shane From martin at v.loewis.de Mon Jan 9 22:56:12 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 09 Jan 2006 22:56:12 +0100 Subject: [Python-Dev] locale and LC_NUMERIC In-Reply-To: References: <43C1A147.60409@v.loewis.de> <43C21759.2090301@v.loewis.de> Message-ID: <43C2DBFC.1010201@v.loewis.de> Neal Norwitz wrote: > neal at janus ~/build/python/svn/clean-ish $ LC_ALL=de_DE ./python > >>>>import locale >>>>locale.setlocale(locale.LC_ALL) > > 'C' > >>>>locale.setlocale(locale.LC_ALL, 'de_DE') > > 'de_DE' > > I would have expected the first call to setlocale() to return de_DE. No, it shouldn't. We are providing C semantics here, which is that no locale functionality is activated unless the application explicitly asks for it. > Is the first call to > setlocale() supposed to return C? If so, always? The return value is implementation-defined, but yes, it is supposed to return the "C" locale (which is synonymous to the "POSIX" locale on a POSIX system). > I was looking for > some way for the initial call to setlocale() to return de_DE or > whatever other locale I set. That shouldn't happen. Regards, Martin From martin at v.loewis.de Mon Jan 9 23:11:03 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 09 Jan 2006 23:11:03 +0100 Subject: [Python-Dev] [Python-checkins] r41972 - python/branches/ssize_t/Objects/funcobject.c In-Reply-To: References: <20060108060915.A69E01E4002@bag.python.org> <43C0DF8C.7050104@v.loewis.de> <1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com> <43C15167.7000702@v.loewis.de> <1f7befae0601081547h50cbce19w328e30aa6b1e1f5a@mail.gmail.com> <43C20F79.4020908@v.loewis.de> <1f7befae0601090948k47f782baq1e46ed6306be778e@mail.gmail.com> Message-ID: <43C2DF77.1060409@v.loewis.de> Neal Norwitz wrote: > I often chose to use Py_ssize_t rather than int if it was capturing > the result of a sequence length. But often this is larger than what > will be allowed (like in this case). The problem then comes that we > have cases everywhere if we leave things as ints. This is also the strategy I have been following. The alternative would be to add error handling at the places where truncation occurs (unless you can *really* determine statically that there never ever will be truncation). This is too painful, so it is IMO better to carry the wide values throughout, until the place where the range check is carried out. > Stuff like this: > len = (int) PyTuple_Size(co_consts); > > How do we know that is safe or not? Well, you know that LOAD_CONST only supports 2**31 constants, so truncation to int is currently safe (I hope that the compiler detects cases where somebody tries to create more than 2**16 constants). Still, this is only the *current* state. Going further, it might be that some of the restrictions are lifted someday, so we would have to review all casts then to see whether they are still safe. Therefore, I think having fewer casts is better. Regards, Martin From martin at v.loewis.de Mon Jan 9 23:19:39 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 09 Jan 2006 23:19:39 +0100 Subject: [Python-Dev] locale and LC_NUMERIC In-Reply-To: References: <43C1A147.60409@v.loewis.de> Message-ID: <43C2E17B.1050003@v.loewis.de> Georg Brandl wrote: > import locale > orig_locale = locale.setlocale(locale.LC_NUMERIC, '') > locale.setlocale(locale.LC_NUMERIC, 'fr_FR') > > and later > > finally: > locale.setlocale(locale.LC_NUMERIC, orig_locale) > > > Shouldn't the first setlocale have no second argument? Oops, right - it certainly should. Martin From nnorwitz at gmail.com Tue Jan 10 00:06:04 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 9 Jan 2006 15:06:04 -0800 Subject: [Python-Dev] Logging enhancements In-Reply-To: <43C2CC24.4090405@hathawaymix.org> References: <43C2CC24.4090405@hathawaymix.org> Message-ID: On 1/9/06, Shane Hathaway wrote: > I'd like to create a patch for the logging package, but before I do, I > need to know whether someone else is working on the package and whether > my patch is likely to be accepted. Is there another group I should talk to? Vinay (copied) maintains it separately. He's the best person to talk to AFAIK. He maintains compatability with older pythons, possibly 1.5, I don't recall. Check PEP 291. > Specifically, I want to change the following: There are several bugs (any maybe patches) on SF related to logging. AFAIK, they are all assigned to Vinay. It would be great if you could fix those as well. n -- > - The logging.config.fileConfig function has bare excepts that hide > configuration errors. I'd like to remove the bare excepts and break up > fileConfig for clarity. > > - I'd like to make it possible to configure the format of exception > tracebacks. Some logs want compressed tracebacks like Medusa; others > want extra information like Zope. > > - There is no support for external log rotation. There is plenty of > support for Python rotating logs directly, but permission settings often > make that impossible. Long-running Python processes need to accept a > signal which notifies them when an external tool rotates the logs. > > - The logging documentation is lacking somewhat. For example, it fails > to mention that the logging handlers are in the 'handlers' module, which > I only learned by searching the source code. > > Shane From facundobatista at gmail.com Tue Jan 10 00:11:43 2006 From: facundobatista at gmail.com (Facundo Batista) Date: Mon, 9 Jan 2006 20:11:43 -0300 Subject: [Python-Dev] ConfigParser to save with order In-Reply-To: <200601091217.51071.fdrake@acm.org> References: <200601091217.51071.fdrake@acm.org> Message-ID: 2006/1/9, Fred L. Drake, Jr. : > On Monday 09 January 2006 12:08, Facundo Batista wrote: > > What I wanted to add to the module was predicatibility: a very needed > > feature when you're writing test cases (and that's where I got bite). > > In that case, would sorting the keys within each section be sufficient when > writing it back out? Yes, because giving a set of sections and values inside them, you know in advance how they'll finish in the file. > I had a class to do the really-careful editing bit like Guido described once, > but lost it in a disk crash several years ago. I really wish I'd gotten that > into revision control somewhere, but it's too late now. I've not needed that > badly enough to re-write it. Somebody (me, for example) can write it if we all decide that that's the desired behaviour. We just need to take a side. Me, I just need the predicatibility that sort() gives. . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From nnorwitz at gmail.com Tue Jan 10 00:14:48 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 9 Jan 2006 15:14:48 -0800 Subject: [Python-Dev] locale and LC_NUMERIC In-Reply-To: <43C2DBFC.1010201@v.loewis.de> References: <43C1A147.60409@v.loewis.de> <43C21759.2090301@v.loewis.de> <43C2DBFC.1010201@v.loewis.de> Message-ID: On 1/9/06, "Martin v. L?wis" wrote: > > > > I would have expected the first call to setlocale() to return de_DE. > > No, it shouldn't. We are providing C semantics here, which is that > no locale functionality is activated unless the application explicitly > asks for it. Thanks (to /f too). That was the part that I was missing. It makes sense now. I made the patch to do this. It's called before each class' test run in regrtest.py. I would prefer to set the locale before each test method run, but I need to muck with unittest for that IIRC. There were 2 failures. One was in test_email which needs the C locale for generating the proper timestamp. The other was (IIRC) in test_re, 2 methods failed. So we are hopefully in decent shape. n From shane at hathawaymix.org Tue Jan 10 00:29:38 2006 From: shane at hathawaymix.org (Shane Hathaway) Date: Mon, 09 Jan 2006 16:29:38 -0700 Subject: [Python-Dev] Logging enhancements In-Reply-To: References: <43C2CC24.4090405@hathawaymix.org> Message-ID: <43C2F1E2.8030102@hathawaymix.org> Neal Norwitz wrote: > On 1/9/06, Shane Hathaway wrote: > >>I'd like to create a patch for the logging package, but before I do, I >>need to know whether someone else is working on the package and whether >>my patch is likely to be accepted. Is there another group I should talk to? > > > Vinay (copied) maintains it separately. He's the best person to talk > to AFAIK. He maintains compatability with older pythons, possibly > 1.5, I don't recall. Check PEP 291. > > >>Specifically, I want to change the following: > > > There are several bugs (any maybe patches) on SF related to logging. > AFAIK, they are all assigned to Vinay. It would be great if you could > fix those as well. What is the time frame for Python 2.5? If I finish soon and the changes are not disruptive, do I have enough time for my changes to make it into 2.5? Shane From aahz at pythoncraft.com Tue Jan 10 00:42:32 2006 From: aahz at pythoncraft.com (Aahz) Date: Mon, 9 Jan 2006 15:42:32 -0800 Subject: [Python-Dev] Logging enhancements In-Reply-To: <43C2F1E2.8030102@hathawaymix.org> References: <43C2CC24.4090405@hathawaymix.org> <43C2F1E2.8030102@hathawaymix.org> Message-ID: <20060109234231.GA7090@panix.com> On Mon, Jan 09, 2006, Shane Hathaway wrote: > > What is the time frame for Python 2.5? If I finish soon and the changes > are not disruptive, do I have enough time for my changes to make it into > 2.5? There is still no official timeframe for 2.5. Alpha is hoped for March/April; final release for June/July. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From tim.peters at gmail.com Tue Jan 10 00:45:15 2006 From: tim.peters at gmail.com (Tim Peters) Date: Mon, 9 Jan 2006 18:45:15 -0500 Subject: [Python-Dev] Logging enhancements In-Reply-To: <43C2F1E2.8030102@hathawaymix.org> References: <43C2CC24.4090405@hathawaymix.org> <43C2F1E2.8030102@hathawaymix.org> Message-ID: <1f7befae0601091545u13e7c3c7ha852ff439d2f20e5@mail.gmail.com> [Shane Hathaway] > What is the time frame for Python 2.5? If I finish soon and the changes > are not disruptive, do I have enough time for my changes to make it into > 2.5? Yup! 2.5 hasn't even had an alpha release yet, and I don't expect 2.5a1 before March: http://mail.python.org/pipermail/python-dev/2005-August/055342.html From shane at hathawaymix.org Tue Jan 10 00:50:17 2006 From: shane at hathawaymix.org (Shane Hathaway) Date: Mon, 09 Jan 2006 16:50:17 -0700 Subject: [Python-Dev] Logging enhancements In-Reply-To: <1f7befae0601091545u13e7c3c7ha852ff439d2f20e5@mail.gmail.com> References: <43C2CC24.4090405@hathawaymix.org> <43C2F1E2.8030102@hathawaymix.org> <1f7befae0601091545u13e7c3c7ha852ff439d2f20e5@mail.gmail.com> Message-ID: <43C2F6B9.1060108@hathawaymix.org> Tim Peters wrote: > [Shane Hathaway] > >>What is the time frame for Python 2.5? If I finish soon and the changes >>are not disruptive, do I have enough time for my changes to make it into >>2.5? > > > Yup! 2.5 hasn't even had an alpha release yet, and I don't expect > 2.5a1 before March: > > http://mail.python.org/pipermail/python-dev/2005-August/055342.html That's good news. Thanks. Shane From vinay_sajip at red-dove.com Tue Jan 10 01:01:31 2006 From: vinay_sajip at red-dove.com (Vinay Sajip) Date: Tue, 10 Jan 2006 00:01:31 -0000 Subject: [Python-Dev] Logging enhancements References: <43C2CC24.4090405@hathawaymix.org> Message-ID: <001001c61579$05fb01c0$0200a8c0@alpha> Hi Shane, On 1/9/06, Shane Hathaway wrote: > I'd like to create a patch for the logging package, but before I do, I > need to know whether someone else is working on the package and whether > my patch is likely to be accepted. Is there another group I should talk to? I maintain the logging package, and I will happily review patches and apply them where I think they're fine. I have accepted many patches to the package, some via SF and some sent privately, but also rejected a few. For example, the TimedRotatingFileHandler was submitted by someone. Patches need to be compatible with Python 1.5.2. > - The logging.config.fileConfig function has bare excepts that hide > configuration errors. I'd like to remove the bare excepts and break up > fileConfig for clarity. I've no problem with this in principle, though there have been cases where ConfigParser behaviour is not what you would expect (e.g. a non-existent file leads to a "missing section" exception). I'd be grateful for more specifics as to where you have encountered problems. > - I'd like to make it possible to configure the format of exception > tracebacks. Some logs want compressed tracebacks like Medusa; others > want extra information like Zope. Again, no problem in principle, but it's probably an unusual use case so any change should not require extra work for users without specialised requirements. > - There is no support for external log rotation. There is plenty of > support for Python rotating logs directly, but permission settings often > make that impossible. Long-running Python processes need to accept a > signal which notifies them when an external tool rotates the logs. I'd be interested in your approach to solve this. > - The logging documentation is lacking somewhat. For example, it fails > to mention that the logging handlers are in the 'handlers' module, which > I only learned by searching the source code. Not true for this specific case - for example, http://docs.python.org/lib/module-logging.html has a list of 12 handlers, followed by a statement to the effect that all but StreamHandler and FileHandler are defined in the handlers module. In general, patches are gratefully received! If you don't want to invest too much time up front, send me a mail indicating your general approach and we'll discuss. Best regards, Vinay Sajip From shane at hathawaymix.org Tue Jan 10 01:29:48 2006 From: shane at hathawaymix.org (Shane Hathaway) Date: Mon, 09 Jan 2006 17:29:48 -0700 Subject: [Python-Dev] Logging enhancements In-Reply-To: <001001c61579$05fb01c0$0200a8c0@alpha> References: <43C2CC24.4090405@hathawaymix.org> <001001c61579$05fb01c0$0200a8c0@alpha> Message-ID: <43C2FFFC.9000202@hathawaymix.org> Vinay Sajip wrote: > On 1/9/06, Shane Hathaway wrote: >>- The logging.config.fileConfig function has bare excepts that hide >>configuration errors. I'd like to remove the bare excepts and break up >>fileConfig for clarity. > > I've no problem with this in principle, though there have been cases where > ConfigParser behaviour is not what you would expect (e.g. a non-existent > file leads to a "missing section" exception). I'd be grateful for more > specifics as to where you have encountered problems. I wrote a config file with an invalid handler section. Because of the bare except, not only was my error not reported anywhere, but the daemon started with no logging at all. It took me a while to realize what had happened. The logging package needs to report erroneous configurations. >>- I'd like to make it possible to configure the format of exception >>tracebacks. Some logs want compressed tracebacks like Medusa; others >>want extra information like Zope. > > Again, no problem in principle, but it's probably an unusual use case so any > change should not require extra work for users without specialised > requirements. Agreed. >>- There is no support for external log rotation. There is plenty of >>support for Python rotating logs directly, but permission settings often >>make that impossible. Long-running Python processes need to accept a >>signal which notifies them when an external tool rotates the logs. > > I'd be interested in your approach to solve this. To be compatible with an external tool like logrotate, all a daemon needs to do is close and reopen log files at about the same time log rotation happens. To handle this use case, I suggest the logging module needs a function named reopen(), which daemons can call upon receipt of a signal. The reopen function will find all handlers with open files and ask them to reopen. >>- The logging documentation is lacking somewhat. For example, it fails >>to mention that the logging handlers are in the 'handlers' module, which >>I only learned by searching the source code. > > > Not true for this specific case - for example, > http://docs.python.org/lib/module-logging.html has a list of 12 handlers, > followed by a statement to the effect that all but StreamHandler and > FileHandler are defined in the handlers module. Oops. I guess I skipped over that because I was using the documentation as a reference rather than as a tutorial. I suggest the documentation needs to become a better reference. > In general, patches are gratefully received! If you don't want to invest too > much time up front, send me a mail indicating your general approach and > we'll discuss. I'll get started soon. I just wanted to be sure I'm not duplicating someone else's efforts. Shane From barry at python.org Tue Jan 10 01:37:19 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 09 Jan 2006 19:37:19 -0500 Subject: [Python-Dev] Logging enhancements In-Reply-To: <001001c61579$05fb01c0$0200a8c0@alpha> References: <43C2CC24.4090405@hathawaymix.org> <001001c61579$05fb01c0$0200a8c0@alpha> Message-ID: <1136853439.1035.110.camel@geddy.wooz.org> On Tue, 2006-01-10 at 00:01 +0000, Vinay Sajip wrote: > I maintain the logging package, and I will happily review patches and apply > them where I think they're fine. I have accepted many patches to the > package, some via SF and some sent privately, but also rejected a few. For > example, the TimedRotatingFileHandler was submitted by someone. > > Patches need to be compatible with Python 1.5.2. I've been thinking about this compatibility thing too, especially w.r.t. the email package. For example, I still want to maintain email 2.5 which has b/c to Python 2.1. But email 3.0 need only be compatible with Python 2.3. Given that we now have everything in Subversion, I think it makes it much easier to maintain multiple versions of separately distributed packages, modulo the time it takes to do so. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060109/e481130b/attachment.pgp From martin at v.loewis.de Tue Jan 10 09:15:56 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 10 Jan 2006 09:15:56 +0100 Subject: [Python-Dev] buildbot In-Reply-To: <20060108.173810.91561560.warner@lothar.com> References: <20060104.003058.48524852.warner@lothar.com> <43BD9B32.3030700@v.loewis.de> <20060108.173810.91561560.warner@lothar.com> Message-ID: <43C36D3C.7050202@v.loewis.de> Brian Warner wrote: > This makes it much easier to implement things like the "Force Build" > button, and allows the same status-retrieving API to be used by all > the different status-delivery modules: web, IRC, email, PB. Certainly: I can easily believe that this is easier from a development point of view. I can also accept that this is the only way if you want to trigger things through the Web (such as the Force Build interface). However, this interface was the first thing I turned off, because there is too much potential for abuse. The slaves are (in our case) donated resources, sometimes in machines that also serve other purposes. The donors need to know what precisely will run on their machines, and when precisely that will happen, so "Force build" is disabled. The reason I want static pages is for security concerns. It is not easy whether buildbot can be trusted to have no security flaws, which might allow people to start new processes on the master, or (perhaps worse) on any of the slaves. I know I could limit the Twisted webserver to localhost using firewalling/iptables (and I will need to if there is no other option); just having it generate static pages would have been more convenient. BTW, the same security concern applies to the IRC listener and the slave listeners: how do I know that nobody can take over my infrastructure using carefully-crafted messages? I could study the entire protocol stack to convince myself, but I would prefer if there was an easy answer, like: this is the set of operations callable through twisted, and these are the preconditions for calling them. For the IRC, this is easy to see (just inspect all command_ procedures), but what about the slaves port? > It's not the greatest solution, but it might let you add more > Builders without making the display unworkably complex. Thanks! so far, we only have 8 lanes, which is still managable. If size increases, I come back to these ideas. Regards, Martin From mhammond at skippinet.com.au Tue Jan 10 09:10:06 2006 From: mhammond at skippinet.com.au (Mark Hammond) Date: Tue, 10 Jan 2006 19:10:06 +1100 Subject: [Python-Dev] Logging enhancements In-Reply-To: <43C2FFFC.9000202@hathawaymix.org> Message-ID: Shane: > Vinay Sajip wrote: > > I'd be interested in your approach to solve this. > > To be compatible with an external tool like logrotate, all a daemon > needs to do is close and reopen log files at about the same time log > rotation happens. To handle this use case, I suggest the logging module > needs a function named reopen(), which daemons can call upon receipt of > a signal. The reopen function will find all handlers with open files > and ask them to reopen. FWIW, such an operation is generally useless on a Windows system, as a file can not be renamed while it is in use. zope-dev went through this discussion recently, and in the interests of still allowing an external tool (similar to logrotate) to have the rotation logic instead of Python or Zope, we came up with a slightly different scheme for Windows; when requested, Zope will temporarily close the file and rename it to the original name + ".last" (ignoring any errors.) It then reopens the original name. In most cases, this will leave a .last file around which can be processed by an external tool, but is also safe in the face of failure. IMO, we should consider implementing something like the above; it is still preferable to have a standard solution for Windows, even if different than other platforms, rather than requiring everyone to solve this problem themselves. I'd even suggest sticking with the name 'reopen()' is still OK, noting the differences in the documentation, but that is a secondary issue. Cheers, Mark From andrew-pythondev at puzzling.org Tue Jan 10 09:49:55 2006 From: andrew-pythondev at puzzling.org (Andrew Bennetts) Date: Tue, 10 Jan 2006 19:49:55 +1100 Subject: [Python-Dev] buildbot In-Reply-To: <43C36D3C.7050202@v.loewis.de> References: <20060104.003058.48524852.warner@lothar.com> <43BD9B32.3030700@v.loewis.de> <20060108.173810.91561560.warner@lothar.com> <43C36D3C.7050202@v.loewis.de> Message-ID: <20060110084955.GF10373@home.puzzling.org> On Tue, Jan 10, 2006 at 09:15:56AM +0100, "Martin v. L?wis" wrote: [...] > > I know I could limit the Twisted webserver to localhost using > firewalling/iptables (and I will need to if there is no other > option); just having it generate static pages would have been > more convenient. For this part at least, it's easier than that; you can pass interface='127.0.0.1' to the relevant listenTCP or TCPServer call, so that it will only bind that port to localhost. -Andrew. From mal at egenix.com Tue Jan 10 10:18:58 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 10 Jan 2006 10:18:58 +0100 Subject: [Python-Dev] New PEP: Using ssize_t as the index type In-Reply-To: <43BD8464.4060801@v.loewis.de> References: <43B3ECEE.9060300@v.loewis.de> <20051229173041.GA19575@code1.codespeak.net> <43B50B64.20904@v.loewis.de> <20060105194538.GB11736@code1.codespeak.net> <43BD8464.4060801@v.loewis.de> Message-ID: <43C37C02.8030005@egenix.com> Martin v. L?wis wrote: > Armin Rigo wrote: >> This would do the right thing for <= 2.4, using ints everywhere; and the >> Python.h version 2.5 would detect the #define and assume it's a >> 2.5-compatible module, so it would override the #define with the real >> thing *and* turn on the ssize_t interpretation of the '#' format >> character. > > This would be very similar to the PY_SIZE_T_CLEAN approach, except > that it would also help to detect spelling mistakes. > >>From an implementation point of view, the real challenge is to > give PyArg_ParseTuple a different meaning; I do this be #defining > it to PyArg_ParseTupleSsize_t (to preserve binary compatibility > for the original interpretation of ParseTuple). Putting > additional flags arguments in the entire code is also quite > hackish. > >> I still don't like the idea of a magic #define that changes the behavior >> of '#include ', but I admit I don't find any better solution. >> I suppose I'll just blame C. > > More precisely, the printf style of function calling, and varargs > functions. ISO C is pretty type safe, but with varargs functions, > you lose that completely. > > I still hope I can write a C parser some day that does > ParseTuple/BuildValue checking. At least gcc does check the parameter types and generates warnings for wrong vararg parameters passed to printf() et al. We definitely need a clean solution for this. Otherwise, authors who want to support more than just the latest Python release will run into deep trouble. Note that the branch also has other changes of output parameter types which will cause problems in extensions (not only extensions implementing the sequence protocol as the PEP suggests, but also ones using it as well as extensions implementing or using the buffer protocol and the slicing protocol). These are not as easily avoidable as the PyArg_ParseTuple() problem which could be dealt with by a a new parser marker for ssize_t lengths (with the '#' marker casting the argument to an int). I don't see a good solution for these other than introducing a set of new APIs (and maybe doing some macro magic as Martin did for PyArg_ParseTuple()). Due to the fact that changes in the types of output parameters require changes in the extension variable type layout itself, they introduce a large number of type changes in the extension and make writing backwards compatible extensions harder than necessary. Furthermore, all extensions for Python 2.4 would have to be ported to the new Python API and porting is not going to be a simple recompile, but will require C skills. We'd also have to make sure that old extensions don't just import with a warning, since the change will introduce buffer overflows and seg faults in extensions that are not aware of the change. Martin, please add the above points to the PEP. I'd also like to see it published, because it's hard to track a PEP in the mailing Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 10 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mwh at python.net Tue Jan 10 10:28:53 2006 From: mwh at python.net (Michael Hudson) Date: Tue, 10 Jan 2006 09:28:53 +0000 Subject: [Python-Dev] [Python-checkins] r41972 - python/branches/ssize_t/Objects/funcobject.c In-Reply-To: <43C2DF77.1060409@v.loewis.de> ( =?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Mon, 09 Jan 2006 23:11:03 +0100") References: <20060108060915.A69E01E4002@bag.python.org> <43C0DF8C.7050104@v.loewis.de> <1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com> <43C15167.7000702@v.loewis.de> <1f7befae0601081547h50cbce19w328e30aa6b1e1f5a@mail.gmail.com> <43C20F79.4020908@v.loewis.de> <1f7befae0601090948k47f782baq1e46ed6306be778e@mail.gmail.com> <43C2DF77.1060409@v.loewis.de> Message-ID: <2m64osqvmy.fsf@starship.python.net> "Martin v. L?wis" writes: > Well, you know that LOAD_CONST only supports 2**31 constants, so > truncation to int is currently safe (I hope that the compiler detects > cases where somebody tries to create more than 2**16 constants). Easy enough to check: >>> eval(repr(range(100000))) (It works). Cheers, mwh -- Q: What are 1000 lawyers at the bottom of the ocean? A: A good start. (A lawyer told me this joke.) -- Michael Str?der, comp.lang.python From mwh at python.net Tue Jan 10 10:32:22 2006 From: mwh at python.net (Michael Hudson) Date: Tue, 10 Jan 2006 09:32:22 +0000 Subject: [Python-Dev] test_curses In-Reply-To: (Georg Brandl's message of "Mon, 09 Jan 2006 19:55:52 +0100") References: <2m1wzhsqq5.fsf@starship.python.net> Message-ID: <2m1wzgqvh5.fsf@starship.python.net> Georg Brandl writes: > Michael Hudson wrote: >> Georg Brandl writes: >> >>> The call to curses.setupterm() leaves my terminal in a bad state. >> >> Hmm. >> >>> The reset program outputs: >>> Erase set to delete. >>> Kill set to control-U (^U). >>> Interrupt set to control-C (^C). >> >> It always says that :) (unless you've messed with stty, I guess) > > Well, when I do a reset without meddling with the terminal, it says nothing, > at least on my box. Oh. Maybe I'm out of date. > And, there's more: Ctrl+D doesn't work, Ctrl+C doesn't work. Right, so it definitely sounds like the terminal is in raw mode. >>> Doesn't the setupterm() have to be paired with something like shutdownterm()? >> >> Not as far as my memory of curses goes. From the man page: >> >> The setupterm routine reads in the terminfo database, >> initializing the terminfo structures, but does not set up the >> output virtualization structures used by curses. >> >> What platform are you on? > > Linux 2.6, ncurses 5.5, TERM=xterm. Well, this still has the faint whiff of impossibility about it. Are you sure it's setupterm() that's doing the damage? Can you reproduce interactively? Cheers, mwh -- You can remove divmod() when I'm dead. Before then, it stays. I'm sure all will agree that's a reasonable compromise. -- Tim Peters negotiating on python-dev From skip at pobox.com Tue Jan 10 15:46:22 2006 From: skip at pobox.com (skip at pobox.com) Date: Tue, 10 Jan 2006 08:46:22 -0600 Subject: [Python-Dev] sudo security hole w/ potential Python connection Message-ID: <17347.51390.16470.505543@montanaro.dyndns.org> Got this from a Google alert overnight. It's not really a Python problem (it's a sudo problem), but it's probably not a bad idea to understand the implications. >> SUDO Python Environment Cleaning Privilege Escalation ... >> Secunia - UK >> ... This can be exploited by a user with sudo access to a python script >> to gain access to an interactive python prompt via the "PYTHONINSPECT" >> environment variable ... >> Skip From munna_tank at yahoo.co.in Tue Jan 10 16:51:12 2006 From: munna_tank at yahoo.co.in (RASHMI TANK) Date: Tue, 10 Jan 2006 15:51:12 +0000 (GMT) Subject: [Python-Dev] Python-Dev Digest, Vol 30, Issue 32 In-Reply-To: Message-ID: <20060110155112.92853.qmail@web8514.mail.in.yahoo.com> please stop email upto further request. thanking you Send instant messages to your online friends http://in.messenger.yahoo.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060110/9aac8e8e/attachment.htm From jorodrig at sfwmd.gov Tue Jan 10 17:35:45 2006 From: jorodrig at sfwmd.gov (Rodrigues, Joseph) Date: Tue, 10 Jan 2006 11:35:45 -0500 Subject: [Python-Dev] Tkinter Message-ID: <14A2A120D369B6469BB154B2D2DC34D203537050@EXCHVS01.ad.sfwmd.gov> Hello, I would like to know if Tkinter is being developed/maintained. It appears the latest book on Tkinter by John E. Grayson published in 2000 is the only book resource on Tkinter. The publisher suggested that QtPy (http://www.riverbankcomputing.co.uk/pyqt/) and wxPython ( http://www.wxpython.org/ ) have replaced Tkinter. Could you provide an update on Tkinter Joseph -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060110/776eac80/attachment.html From anothermax at gmail.com Mon Jan 9 20:40:00 2006 From: anothermax at gmail.com (Jeremy Maxfield) Date: Mon, 9 Jan 2006 20:40:00 +0100 Subject: [Python-Dev] Sharing between multiple interpreters and restricted mode In-Reply-To: <43BC0933.2060705@core-sdi.com> References: <43BC0933.2060705@core-sdi.com> Message-ID: <93dc9c320601091140h3338b085rb739f39e98ea1213@mail.gmail.com> Hi Gabriel, Apart from the problem of sharing code between interpreters (sounds pretty dangerous to me) there's a bug in 2.4.1 (and 2.3.5 for that matter) that causes problems with multiple interpreters in certain siutations (esp. threads) which results in this 'restricted mode' message - I encountered it when trying to do some work with JEP (Java Embedded Python). Python 2.4.2 fixed this problem for me. I would suggest giving 2.4.2 a try too. Good luck, Max On 1/4/06, Gabriel Becedillas wrote: > Hi, > At the company I work for we've embedded Python 2.4.1 in a C++ > application. We execute multiple scripts concurrenlty, each one in its > own interpreter (created using Py_NewInterpreter()). > We are sharing a certain instance between interpreters because its to > expensive to instantiate that class every time an interpreter is > created. The class is instantiated in the main interpreter (that is > always alive) and every time a new interpreter is created, that > instance is added to the interpreter's __builtins__ dict. > The problem I'm having is that some methods of that class (when > executed in an interpreter different from the one it was created in) > complain about being in restricted execution mode. > Assuming that there are no issues with this instance lifetime (coz it > will always be referenced by the main interpreter), is there a safe way > to do some sharing between interpreters ?. > Thanks. > > -- > > Gabriel Becedillas > Developer > CORE SECURITY TECHNOLOGIES > > Florida 141 - 2? cuerpo - 7? piso > C1005AAC Buenos Aires - Argentina > Tel/Fax: (54 11) 5032-CORE (2673) > http://www.corest.com > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/anothermax%40gmail.com > From aahz at pythoncraft.com Tue Jan 10 17:54:23 2006 From: aahz at pythoncraft.com (Aahz) Date: Tue, 10 Jan 2006 08:54:23 -0800 Subject: [Python-Dev] Tkinter In-Reply-To: <14A2A120D369B6469BB154B2D2DC34D203537050@EXCHVS01.ad.sfwmd.gov> References: <14A2A120D369B6469BB154B2D2DC34D203537050@EXCHVS01.ad.sfwmd.gov> Message-ID: <20060110165423.GA10810@panix.com> On Tue, Jan 10, 2006, Rodrigues, Joseph wrote: > > I would like to know if Tkinter is being developed/maintained. It > appears the latest book on Tkinter by John E. Grayson published in 2000 > is the only book resource on Tkinter. This message is not appropriate for python-dev. Please use comp.lang.python (or python-list). -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From warner at lothar.com Tue Jan 10 19:07:46 2006 From: warner at lothar.com (Brian Warner) Date: Tue, 10 Jan 2006 10:07:46 -0800 (PST) Subject: [Python-Dev] buildbot In-Reply-To: <20060110084955.GF10373@home.puzzling.org> References: <20060108.173810.91561560.warner@lothar.com> <43C36D3C.7050202@v.loewis.de> <20060110084955.GF10373@home.puzzling.org> Message-ID: <20060110.100746.13886364.warner@lothar.com> > For this part at least, it's easier than that; you can pass > interface='127.0.0.1' to the relevant listenTCP or TCPServer call, so that it > will only bind that port to localhost. And I intend to add an option to the buildbot master.cfg file to make it easy to do just that. It should be in the next release. -Brian (running busy today, hoping to respond to the rest tonight) From guido at python.org Tue Jan 10 19:10:50 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 10 Jan 2006 10:10:50 -0800 Subject: [Python-Dev] Tkinter In-Reply-To: <20060110165423.GA10810@panix.com> References: <14A2A120D369B6469BB154B2D2DC34D203537050@EXCHVS01.ad.sfwmd.gov> <20060110165423.GA10810@panix.com> Message-ID: On 1/10/06, Aahz wrote: > On Tue, Jan 10, 2006, Rodrigues, Joseph wrote: > > I would like to know if Tkinter is being developed/maintained. It > > appears the latest book on Tkinter by John E. Grayson published in 2000 > > is the only book resource on Tkinter. > > This message is not appropriate for python-dev. Please use > comp.lang.python (or python-list). But for sure, Tkinter is still being maintained by the Python developers (mainly MvL I believe). If it appears stagnant that's probably because Tcl/Tk itself isn't changing much. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From warner at lothar.com Tue Jan 10 19:16:11 2006 From: warner at lothar.com (Brian Warner) Date: Tue, 10 Jan 2006 10:16:11 -0800 (PST) Subject: [Python-Dev] [Buildbot-devel] Re: buildbot In-Reply-To: References: <20060108.173810.91561560.warner@lothar.com> <43C36D3C.7050202@v.loewis.de> Message-ID: <20060110.101611.23734793.warner@lothar.com> > I have security concerns as well, but not in buildbot itself. My > project is restricted even withinz the company I work for so I need > the buildbot web server to only be available to certain people. > HTTPS access would be nice too. TwistedWeb doesn't seem to have > support for either HTTPS or authentication so I've been forced to > "hide" it by putting it on a non-standard port. Very weak. Actually twisted-web provides pretty good support for both SSL and password-based authentication, but my own web skills are too weak to really figure out how to take advantage of them. I've been hoping to address this in the "web status page cleanup" item on the TODO list, but other bugs and features keep getting in the way. There are a couple of examples of setting up an HTTPS website in the twisted docs, in case you're interested. I think the changes would only have to go into Waterfall.setup, probably just replacing the internet.TCPServer() call to something involving SSL. The authentication stuff is called "guard", and I think it would involve inserting an additional password-checking resource between the top-level server.Site and the primary html.StatusResource object. But, as I said, my twisted-web-foo is not strong, so I could be completely wrong about this. cheers, -Brian From guido at python.org Tue Jan 10 19:31:16 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 10 Jan 2006 10:31:16 -0800 Subject: [Python-Dev] sudo security hole w/ potential Python connection In-Reply-To: <17347.51390.16470.505543@montanaro.dyndns.org> References: <17347.51390.16470.505543@montanaro.dyndns.org> Message-ID: Methinks anyone using sudo to allow non-root-users to execute specific scripts without giving them full root perms is relying on security by obscurity at this point. (Ditto for setuid Python scripts BTW.) --Guido On 1/10/06, skip at pobox.com wrote: > > Got this from a Google alert overnight. It's not really a Python problem > (it's a sudo problem), but it's probably not a bad idea to understand the > implications. > > >> SUDO Python Environment Cleaning Privilege Escalation ... > >> Secunia - UK > >> ... This can be exploited by a user with sudo access to a python script > >> to gain access to an interactive python prompt via the "PYTHONINSPECT" > >> environment variable ... > >> > > Skip > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From lists at core-sdi.com Tue Jan 10 19:27:02 2006 From: lists at core-sdi.com (Gabriel Becedillas) Date: Tue, 10 Jan 2006 15:27:02 -0300 Subject: [Python-Dev] PyThreadState_Delete vs PyThreadState_DeleteCurrent Message-ID: <43C3FC76.4010305@core-sdi.com> Hi, I started hitting Py_FatalError("Invalid thread state for this thread") in pystate.c when a thread bootstraps since my upgrade to Python 2.4.2 (was using 2.4.1 previously). I'm embedding Python in C++, using multiple interpreters and threads. I think the problem is that PyThreadState_Delete (which is used when I destroy thread states and interpreters) is not making the same clean up that PyThreadState_DeleteCurrent is doing: if (autoTLSkey && PyThread_get_key_value(autoTLSkey) == tstate) PyThread_delete_key_value(autoTLSkey); If a thread id is reused (and this often happens), and the previous thread used PyThreadState_Delete instead of PyThreadState_DeleteCurrent, an old and invalid value for autoTLSkey is stored, and that is triggering the Py_FatalError. Thanks. -- Gabriel Becedillas Developer CORE SECURITY TECHNOLOGIES Florida 141 - 2? cuerpo - 7? piso C1005AAC Buenos Aires - Argentina Tel/Fax: (54 11) 5032-CORE (2673) http://www.corest.com From g.brandl-nospam at gmx.net Tue Jan 10 20:06:48 2006 From: g.brandl-nospam at gmx.net (Georg Brandl) Date: Tue, 10 Jan 2006 20:06:48 +0100 Subject: [Python-Dev] test_curses In-Reply-To: <2m1wzgqvh5.fsf@starship.python.net> References: <2m1wzhsqq5.fsf@starship.python.net> <2m1wzgqvh5.fsf@starship.python.net> Message-ID: Michael Hudson wrote: > Georg Brandl writes: > >> Michael Hudson wrote: >>> Georg Brandl writes: >>> >>>> The call to curses.setupterm() leaves my terminal in a bad state. >>> >>> Hmm. >>> >>>> The reset program outputs: >>>> Erase set to delete. >>>> Kill set to control-U (^U). >>>> Interrupt set to control-C (^C). >>> >>> It always says that :) (unless you've messed with stty, I guess) >> >> Well, when I do a reset without meddling with the terminal, it says nothing, >> at least on my box. > > Oh. Maybe I'm out of date. > >> And, there's more: Ctrl+D doesn't work, Ctrl+C doesn't work. > > Right, so it definitely sounds like the terminal is in raw mode. > >>>> Doesn't the setupterm() have to be paired with something like shutdownterm()? >>> >>> Not as far as my memory of curses goes. From the man page: >>> >>> The setupterm routine reads in the terminfo database, >>> initializing the terminfo structures, but does not set up the >>> output virtualization structures used by curses. >>> >>> What platform are you on? >> >> Linux 2.6, ncurses 5.5, TERM=xterm. > > Well, this still has the faint whiff of impossibility about it. Are > you sure it's setupterm() that's doing the damage? Can you reproduce > interactively? Yep. Alone, the setupterm call [curses.setupterm(sys.__stdout__.fileno())] does nothing remarkable, but when it is done inside of curses.initscr() / curses.endwin() the terminal is not restored properly. If setupterm() is to be tested, I think it should be executed before initscr(). regards, Georg From martin at v.loewis.de Tue Jan 10 20:17:19 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 10 Jan 2006 20:17:19 +0100 Subject: [Python-Dev] Py_ssize_t output parameters (Was: [Python-checkins] r41971 ...) In-Reply-To: <43C37C92.3060302@egenix.com> References: <20060108054820.15F321E4002@bag.python.org> <43C25CE1.4070106@egenix.com> <43C2DD44.2040908@v.loewis.de> <43C37C92.3060302@egenix.com> Message-ID: <43C4083F.3070001@v.loewis.de> M.-A. Lemburg wrote: >>I don't believe the change is major. It only affects a few extensions, >>and for these, it is only a minor change. A single line of changing >>will be enough. > > > This is true for all the changes related to parameters passed by > value. It is not true for output parameters. For these, the > change will propagate into the extension and make quite a few > changes necessary (for the same reason you have to change > so many variables to Py_ssize_t). For the output parameter, just the variable receiving the value needs to be changed, and you are done. > We should make it possible to have a simple recompile of old > extensions continue to work with Python 2.5 (albeit without > them supporting 64-bit indexes) and without introducing > segfaults or buffer overflows that way. I cannot see how this could reasonably achieved for arbitrary extension modules. Regards, Martin From fredrik at pythonware.com Tue Jan 10 20:29:26 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue, 10 Jan 2006 20:29:26 +0100 Subject: [Python-Dev] Tkinter References: <14A2A120D369B6469BB154B2D2DC34D203537050@EXCHVS01.ad.sfwmd.gov><20060110165423.GA10810@panix.com> Message-ID: Guido van Rossum wrote: > But for sure, Tkinter is still being maintained by the Python > developers (mainly MvL I believe). If it appears stagnant that's > probably because Tcl/Tk itself isn't changing much. afaict, Tcl/Tk 8.5 isn't moving quite as fast as Python 2.5, but they're probably a lot closer to a release than the Perl 6 guys... http://www.tcl.tk/software/tcltk/8.5.html From fredrik at pythonware.com Tue Jan 10 21:00:51 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue, 10 Jan 2006 21:00:51 +0100 Subject: [Python-Dev] r42003 - python/trunk/Lib/test/outstanding_bugs.pypython/trunk/Lib/test/outstanding_crashes.py References: <20060110192928.F114B1E4002@bag.python.org> Message-ID: time to synchronize your activities a bit ? > Author: neal.norwitz > Date: Tue Jan 10 08:49:41 2006 > New Revision: 42000 > > Added: > python/trunk/Lib/test/crashers/ > python/trunk/Lib/test/crashers/README > python/trunk/Lib/test/crashers/coerce.py > python/trunk/Lib/test/crashers/weakref_in_del.py > python/trunk/Lib/test/crashers/xml_parsers.py > Log: > As I threatened on python-dev, add a directory which contains all known > bugs which cause the interpreter to crash. I'm sure we can find a few > more. Many missing bugs deal with variations on unchecked infinite recursion > (like coerce.py). > Author: georg.brandl > Date: Tue Jan 10 20:29:24 2006 > New Revision: 42003 > > Added: > python/trunk/Lib/test/outstanding_crashes.py > Modified: > python/trunk/Lib/test/outstanding_bugs.py > Log: > Add outstanding_crashes.py with tests for crashes. From martin at v.loewis.de Tue Jan 10 21:05:45 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 10 Jan 2006 21:05:45 +0100 Subject: [Python-Dev] buildbot In-Reply-To: <20060110084955.GF10373@home.puzzling.org> References: <20060104.003058.48524852.warner@lothar.com> <43BD9B32.3030700@v.loewis.de> <20060108.173810.91561560.warner@lothar.com> <43C36D3C.7050202@v.loewis.de> <20060110084955.GF10373@home.puzzling.org> Message-ID: <43C41399.9090707@v.loewis.de> Andrew Bennetts wrote: > For this part at least, it's easier than that; you can pass > interface='127.0.0.1' to the relevant listenTCP or TCPServer call, so that it > will only bind that port to localhost. Thanks for the suggestion! works fine for me. Regards, Martin From martin at v.loewis.de Tue Jan 10 21:28:43 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 10 Jan 2006 21:28:43 +0100 Subject: [Python-Dev] New PEP: Using ssize_t as the index type In-Reply-To: <43C37C02.8030005@egenix.com> References: <43B3ECEE.9060300@v.loewis.de> <20051229173041.GA19575@code1.codespeak.net> <43B50B64.20904@v.loewis.de> <20060105194538.GB11736@code1.codespeak.net> <43BD8464.4060801@v.loewis.de> <43C37C02.8030005@egenix.com> Message-ID: <43C418FB.7010007@v.loewis.de> M.-A. Lemburg wrote: > I don't see a good solution for these other than introducing > a set of new APIs (and maybe doing some macro magic as Martin > did for PyArg_ParseTuple()). Due to the fact that changes in > the types of output parameters require changes in the > extension variable type layout itself, they introduce a large > number of type changes in the extension and make writing > backwards compatible extensions harder than necessary. That's not true. It is very easy to write extensions that receive such values and are still backwards-compatible. Suppose you had int pos; PyObject *k, *v; PyDict_Next(dict, &pos, &k, &v); You just change this to /* beginning of file */ #ifdef Py_HEX_VERSION < 2.5 typedef int Py_ssize_t; #endif /* later */ Py_ssize_t pos; PyObject *k, *v; PyDict_Next(dict, &pos, &k, &v); That's it! > Furthermore, all extensions for Python 2.4 would have to be > ported to the new Python API and porting is not going to > be a simple recompile, but will require C skills. Not all extensions. Only those that call functions that expect int* output parameters - which is fairly uncommon. > Martin, please add the above points to the PEP. I'd also > like to see it published, because it's hard to track a PEP > in the mailing It's very difficult to get a PEP number assigned. I wrote peps at python.org with no response. Regards, Martin From guido at python.org Tue Jan 10 21:34:47 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 10 Jan 2006 12:34:47 -0800 Subject: [Python-Dev] Tkinter In-Reply-To: References: <14A2A120D369B6469BB154B2D2DC34D203537050@EXCHVS01.ad.sfwmd.gov> <20060110165423.GA10810@panix.com> Message-ID: On 1/10/06, Fredrik Lundh wrote: > Guido van Rossum wrote: > > > But for sure, Tkinter is still being maintained by the Python > > developers (mainly MvL I believe). If it appears stagnant that's > > probably because Tcl/Tk itself isn't changing much. > > afaict, Tcl/Tk 8.5 isn't moving quite as fast as Python 2.5, but they're > probably a lot closer to a release than the Perl 6 guys... > > http://www.tcl.tk/software/tcltk/8.5.html Well to compare to Perl 6 you should really look at the progress of Tcl/Tk 9.0, which sounds like it's never going to happen. So Perl 6 still wins. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Tue Jan 10 21:37:53 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 10 Jan 2006 21:37:53 +0100 Subject: [Python-Dev] Tkinter In-Reply-To: References: <14A2A120D369B6469BB154B2D2DC34D203537050@EXCHVS01.ad.sfwmd.gov><20060110165423.GA10810@panix.com> Message-ID: <43C41B21.40500@v.loewis.de> Fredrik Lundh wrote: >>But for sure, Tkinter is still being maintained by the Python >>developers (mainly MvL I believe). If it appears stagnant that's >>probably because Tcl/Tk itself isn't changing much. > > > afaict, Tcl/Tk 8.5 isn't moving quite as fast as Python 2.5, but they're > probably a lot closer to a release than the Perl 6 guys... > > http://www.tcl.tk/software/tcltk/8.5.html My maintenance focuses on integrating user contributions, plus looking for the latest Tcl release when doing a Windows build. So new options to existing widgets, or entire new widgets don't get "automatically" wrapper classes in Tkinter. Instead, users wanting to make use of such features are expected to contribute a patch. I usually catch up with such patches before the next major release, atleast if the patches are straight-forward. Regards, Martin From theller at python.net Tue Jan 10 21:55:02 2006 From: theller at python.net (Thomas Heller) Date: Tue, 10 Jan 2006 21:55:02 +0100 Subject: [Python-Dev] Include ctypes into core Python? Message-ID: I would like to suggest to include ctypes into core Python, starting with the 2.5 release. ctypes is nearing the 1.0 release, and works on Windows, OS X, linux, possibly other platforms (provided that the processor is supported by libffi), and just recently even on Windows CE. ctypes does not depend on a libfii library installed on the system, instead the libfii source code is compiled into the extension module. libffi is copyright Red Hat, Inc, and released under a MIT style license. Up-to date docs for ctypes still have to be written, but I assume that the 2.5 release timeframe leaves enough time for that. Thomas From mal at egenix.com Tue Jan 10 22:05:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 10 Jan 2006 22:05:39 +0100 Subject: [Python-Dev] Py_ssize_t output parameters (Was: [Python-checkins] r41971 ...) In-Reply-To: <43C4083F.3070001@v.loewis.de> References: <20060108054820.15F321E4002@bag.python.org> <43C25CE1.4070106@egenix.com> <43C2DD44.2040908@v.loewis.de> <43C37C92.3060302@egenix.com> <43C4083F.3070001@v.loewis.de> Message-ID: <43C421A3.2030806@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg wrote: >>> I don't believe the change is major. It only affects a few extensions, >>> and for these, it is only a minor change. A single line of changing >>> will be enough. >> >> This is true for all the changes related to parameters passed by >> value. It is not true for output parameters. For these, the >> change will propagate into the extension and make quite a few >> changes necessary (for the same reason you have to change >> so many variables to Py_ssize_t). > > For the output parameter, just the variable receiving > the value needs to be changed, and you are done. ... and then the type change of that variable propagates throughout the extension. You basically end up having to convert the whole extension to Py_ssize_t. Don't get me wrong: I don't mind doing this for the eGenix extensions, but I'm worried about the gazillion other useful extensions out there which probably won't get upgraded in time to be used with Python 2.5. >> We should make it possible to have a simple recompile of old >> extensions continue to work with Python 2.5 (albeit without >> them supporting 64-bit indexes) and without introducing >> segfaults or buffer overflows that way. > > I cannot see how this could reasonably achieved for arbitrary > extension modules. I think all it takes is a set of new APIs for functions that use Py_ssize_t as output parameter and which are mapped to the regular API names if and only if the extension #defines PY_SSIZE_T_CLEAN (or some other capability flag). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 10 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From skip at pobox.com Tue Jan 10 22:07:48 2006 From: skip at pobox.com (skip at pobox.com) Date: Tue, 10 Jan 2006 15:07:48 -0600 Subject: [Python-Dev] Tkinter In-Reply-To: References: <14A2A120D369B6469BB154B2D2DC34D203537050@EXCHVS01.ad.sfwmd.gov> <20060110165423.GA10810@panix.com> Message-ID: <17348.8740.582103.283482@montanaro.dyndns.org> Guido> Well to compare to Perl 6 you should really look at the progress Guido> of Tcl/Tk 9.0, which sounds like it's never going to happen. So Guido> Perl 6 still wins. :-) So the grand race is between Tcl/Tk 9.0, Perl 6 and Python 3? Maybe you, Larry and whoever the Tcl/Tk master is these days (still Ousterhout?) could agree to release all three on the same day. Skip From guido at python.org Tue Jan 10 22:11:33 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 10 Jan 2006 13:11:33 -0800 Subject: [Python-Dev] Tkinter In-Reply-To: <17348.8740.582103.283482@montanaro.dyndns.org> References: <14A2A120D369B6469BB154B2D2DC34D203537050@EXCHVS01.ad.sfwmd.gov> <20060110165423.GA10810@panix.com> <17348.8740.582103.283482@montanaro.dyndns.org> Message-ID: On 1/10/06, skip at pobox.com wrote: > > Guido> Well to compare to Perl 6 you should really look at the progress > Guido> of Tcl/Tk 9.0, which sounds like it's never going to happen. So > Guido> Perl 6 still wins. :-) > > So the grand race is between Tcl/Tk 9.0, Perl 6 and Python 3? Maybe you, > Larry and whoever the Tcl/Tk master is these days (still Ousterhout?) could > agree to release all three on the same day. Or we could have another bake-off and pie-throwing contest. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Jan 10 22:14:13 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 10 Jan 2006 13:14:13 -0800 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: References: Message-ID: On 1/10/06, Thomas Heller wrote: > I would like to suggest to include ctypes into core Python, starting > with the 2.5 release. On the one hand I agree that this is a useful module, popular, mature etc. On the other hand it breaks one of the most fundamental Python guidelines: if you get a core dump (segfault etc.) it's a bug in Python or in a 3rd party extension, not in *your* Python code. An exception would have to be made for any code that uses ctypes, as it is usually trivial to cause core dumps with ctypes (I'd venture it's hard to avoid them ;-). I don't expect this to count against including ctypes; but I do want it to be dealt with somehow! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From trentm at ActiveState.com Tue Jan 10 22:15:16 2006 From: trentm at ActiveState.com (Trent Mick) Date: Tue, 10 Jan 2006 13:15:16 -0800 Subject: [Python-Dev] Tkinter In-Reply-To: <17348.8740.582103.283482@montanaro.dyndns.org> References: <14A2A120D369B6469BB154B2D2DC34D203537050@EXCHVS01.ad.sfwmd.gov> <20060110165423.GA10810@panix.com> <17348.8740.582103.283482@montanaro.dyndns.org> Message-ID: <20060110211516.GA8360@activestate.com> [skip at pobox.com wrote] > ...and whoever the Tcl/Tk master is these days (still Ousterhout?)... That's Jeff Hobbs. He sits behind me. I'll see if I can pester him to give some Tcl/Tk and _tkinter thoughts. Cheers, Trent -- Trent Mick trentm at activestate.com From jorodrig at sfwmd.gov Tue Jan 10 22:17:12 2006 From: jorodrig at sfwmd.gov (Rodrigues, Joseph) Date: Tue, 10 Jan 2006 16:17:12 -0500 Subject: [Python-Dev] Tkinter Message-ID: <14A2A120D369B6469BB154B2D2DC34D203537052@EXCHVS01.ad.sfwmd.gov> Thanks for your reply. (I won't bug you on this with lots of email) but I would really like a comment or two on the other python GUIs viz. wxPython and QtPy. Would you say wxPython and QtPy are competing with TKinter or vice versa? Why wxPyton and/or QtPy when TKinter is alive and well? Why am I asking about this? I am working with an application that my agency started with TKinter and is investigating if we should stay with it or consider the others and have not been able to get the kind of direction anywhere. In addition, we purchased the John E. Grayson book but realized it is from 2000 and is wondering what is missing that has been developed in TKinter since 2000? Why has the book become so dated with no updates? This led me to consider that TKinter may be on its way out and we need to consider the other GUI toolkits. I am glad for you response and understand if you cannot dwell too long on this topic. Sincerely Joseph -----Original Message----- From: "Martin v. L?wis" [mailto:martin at v.loewis.de] Sent: Tuesday, January 10, 2006 3:38 PM To: Fredrik Lundh Cc: python-dev at python.org; Rodrigues, Joseph Subject: Re: [Python-Dev] Tkinter Fredrik Lundh wrote: >>But for sure, Tkinter is still being maintained by the Python >>developers (mainly MvL I believe). If it appears stagnant that's >>probably because Tcl/Tk itself isn't changing much. > > > afaict, Tcl/Tk 8.5 isn't moving quite as fast as Python 2.5, but they're > probably a lot closer to a release than the Perl 6 guys... > > http://www.tcl.tk/software/tcltk/8.5.html My maintenance focuses on integrating user contributions, plus looking for the latest Tcl release when doing a Windows build. So new options to existing widgets, or entire new widgets don't get "automatically" wrapper classes in Tkinter. Instead, users wanting to make use of such features are expected to contribute a patch. I usually catch up with such patches before the next major release, atleast if the patches are straight-forward. Regards, Martin From mal at egenix.com Tue Jan 10 22:20:36 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 10 Jan 2006 22:20:36 +0100 Subject: [Python-Dev] New PEP: Using ssize_t as the index type In-Reply-To: <43C418FB.7010007@v.loewis.de> References: <43B3ECEE.9060300@v.loewis.de> <20051229173041.GA19575@code1.codespeak.net> <43B50B64.20904@v.loewis.de> <20060105194538.GB11736@code1.codespeak.net> <43BD8464.4060801@v.loewis.de> <43C37C02.8030005@egenix.com> <43C418FB.7010007@v.loewis.de> Message-ID: <43C42524.7000802@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg wrote: >> I don't see a good solution for these other than introducing >> a set of new APIs (and maybe doing some macro magic as Martin >> did for PyArg_ParseTuple()). Due to the fact that changes in >> the types of output parameters require changes in the >> extension variable type layout itself, they introduce a large >> number of type changes in the extension and make writing >> backwards compatible extensions harder than necessary. > > That's not true. It is very easy to write extensions that > receive such values and are still backwards-compatible. > > Suppose you had > > int pos; > PyObject *k, *v; > > PyDict_Next(dict, &pos, &k, &v); > > You just change this to > > /* beginning of file */ > #ifdef Py_HEX_VERSION < 2.5 > typedef int Py_ssize_t; > #endif > > /* later */ > Py_ssize_t pos; > PyObject *k, *v; > > PyDict_Next(dict, &pos, &k, &v); > > That's it! If it were this easy, I wouldn't have objections. But it's not. The variables you use with these APIs tend to propagate through the extension, you use them in other calls, make assignments, etc. If you implement extension types, you end up having to convert all the length related struct variables to Py_ssize_t. If you're writing against 3rd party APIs which don't use ssize_t or size_t, you have to convert Py_ssize_t to int where necessary. All this is fine, but it's also a lot of work which can be made easier. Recompiling an extension is well within range of many Python users, manually checking, fixing and porting it to the new API is certainly not. >> Furthermore, all extensions for Python 2.4 would have to be >> ported to the new Python API and porting is not going to >> be a simple recompile, but will require C skills. > > Not all extensions. Only those that call functions that expect > int* output parameters - which is fairly uncommon. The set of functions that will require Py_ssize_t is getting larger in your branch which is why I started this discussion. In the first checkin you only had the rarely used slice APIs converted. In the meantime the buffer APIs, the Unicode APIs and others have been added to the list. These APIs are used a lot more often than the slice APIs. I'm not saying that it's a bad idea to adjust these to Py_ssize_t, it's just the backwards incompatible way this is done which bothers me. >> Martin, please add the above points to the PEP. I'd also >> like to see it published, because it's hard to track a PEP >> in the mailing > > It's very difficult to get a PEP number assigned. I wrote > peps at python.org with no response. Would it be possible to host the PEP in the python.org wiki or maybe in the sandbox on svn.python.org ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 10 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From martin at v.loewis.de Tue Jan 10 22:23:42 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 10 Jan 2006 22:23:42 +0100 Subject: [Python-Dev] Py_ssize_t output parameters (Was: [Python-checkins] r41971 ...) In-Reply-To: <43C421A3.2030806@egenix.com> References: <20060108054820.15F321E4002@bag.python.org> <43C25CE1.4070106@egenix.com> <43C2DD44.2040908@v.loewis.de> <43C37C92.3060302@egenix.com> <43C4083F.3070001@v.loewis.de> <43C421A3.2030806@egenix.com> Message-ID: <43C425DE.2050001@v.loewis.de> M.-A. Lemburg wrote: > ... and then the type change of that variable propagates > throughout the extension. That depends on the usage of the code. If the variable is passed by value, no further changes are necessary. If a pointer to the variable is passed, you could replace it with Py_ssize_t x64; int x; foo(&x64); x = x64; Then use x as you did with the original code. > You basically end up having to convert the whole extension > to Py_ssize_t. That is not necessary. Can you give in example of a module where you think it is necessary? > Don't get me wrong: I don't mind doing this for the eGenix > extensions, but I'm worried about the gazillion other > useful extensions out there which probably won't get > upgraded in time to be used with Python 2.5. I agree that it is not acceptable to require immediate whole-sale updates of every modules. However, I do believe that the number of modules that need any change at all is small, and that those modules can be modified with minimal effort to "get them working again, backwards-compatible" (i.e. with the only exception that they would fail if indices run above 2**31). > I think all it takes is a set of new APIs for functions > that use Py_ssize_t as output parameter and which are > mapped to the regular API names if and only if the > extension #defines PY_SSIZE_T_CLEAN (or some other > capability flag). That is not enough. You also need to deal with the function pointers that change. Also, others have rejected/expressed dislike of the PY_SIZE_T_CLEAN macro already, so they would probably dislike further hackishness in that direction. Anyway, I have installed the PEP onsite, and added an "Open Issues" section, recording your comments. Regards, Martin From martin at v.loewis.de Tue Jan 10 22:28:52 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 10 Jan 2006 22:28:52 +0100 Subject: [Python-Dev] Tkinter In-Reply-To: <14A2A120D369B6469BB154B2D2DC34D203537052@EXCHVS01.ad.sfwmd.gov> References: <14A2A120D369B6469BB154B2D2DC34D203537052@EXCHVS01.ad.sfwmd.gov> Message-ID: <43C42714.6060702@v.loewis.de> Rodrigues, Joseph wrote: > Would you say wxPython and QtPy are competing with TKinter or vice > versa? No. I never say that software "competes". I don't even say that products compete. People may compete, or perhaps companies, but not things. > Why wxPyton and/or QtPy when TKinter is alive and well? Because they have their user communities. > Why am I asking about this? Not sure :-) > I am working with an application that my agency started with TKinter > and is investigating if we should stay with it or consider the others > and have not been able to get the kind of direction anywhere. In > addition, we purchased the John E. Grayson book but realized it is > from 2000 and is wondering what is missing that has been developed in > TKinter since 2000? Not much - code in that book should continue to work just fine. > Why has the book become so dated with no updates? You would have to ask John Grayson for that. > This led me to consider that TKinter may be on its way out > and we need to consider the other GUI toolkits. Well, this is certainly off-topic for python-dev. Regards, Martin From martin at v.loewis.de Tue Jan 10 22:35:30 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 10 Jan 2006 22:35:30 +0100 Subject: [Python-Dev] New PEP: Using ssize_t as the index type In-Reply-To: <43C42524.7000802@egenix.com> References: <43B3ECEE.9060300@v.loewis.de> <20051229173041.GA19575@code1.codespeak.net> <43B50B64.20904@v.loewis.de> <20060105194538.GB11736@code1.codespeak.net> <43BD8464.4060801@v.loewis.de> <43C37C02.8030005@egenix.com> <43C418FB.7010007@v.loewis.de> <43C42524.7000802@egenix.com> Message-ID: <43C428A2.9000703@v.loewis.de> M.-A. Lemburg wrote: > If it were this easy, I wouldn't have objections. But it's > not. It is so easy. Can you should me an example where it isn't? > The variables you use with these APIs tend to propagate > through the extension, you use them in other calls, > make assignments, etc. They only propage if you make them. You don't have to, if you don't want to. > If you implement extension types, you end up having to > convert all the length related struct variables to > Py_ssize_t. Only if you want to. If not, the module will work (nearly) unmodified. Of course, it will be limited to 32-bit indices. > All this is fine, but it's also a lot of work which > can be made easier. Recompiling an extension is well > within range of many Python users, manually checking, > fixing and porting it to the new API is certainly not. Sure. However, most users will compile it on 32-bit systems. If they find they cannot get it to work on a 64-bit system, they should ask the author for help, or just use it in 32-bit mode (as 64-bit mode won't gain them anything, anyway). > The set of functions that will require Py_ssize_t > is getting larger in your branch which is why I started > this discussion. How so? I did not add a single function that has int* output values, AFAICT. I am talking about the entirety of these functions, and claim that they are rarely used (including the Unicode and buffer APIs). > Would it be possible to host the PEP in the python.org > wiki or maybe in the sandbox on svn.python.org ? I pinged the PEP editors again, and they assigned the PEP number. Hosting it in a Wiki would be besides the point of the PEP process. Regards, Martin From tim.peters at gmail.com Tue Jan 10 22:49:11 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 10 Jan 2006 16:49:11 -0500 Subject: [Python-Dev] New PEP: Using ssize_t as the index type In-Reply-To: <43C428A2.9000703@v.loewis.de> References: <43B3ECEE.9060300@v.loewis.de> <20051229173041.GA19575@code1.codespeak.net> <43B50B64.20904@v.loewis.de> <20060105194538.GB11736@code1.codespeak.net> <43BD8464.4060801@v.loewis.de> <43C37C02.8030005@egenix.com> <43C418FB.7010007@v.loewis.de> <43C42524.7000802@egenix.com> <43C428A2.9000703@v.loewis.de> Message-ID: <1f7befae0601101349l57e3ae35o65913fecc7112710@mail.gmail.com> [Martin v. L?wis] > ... > I am talking about the entirety of these functions, > and claim that they are rarely used (including the > Unicode and buffer APIs). This reminded me that I still owe you a reply about s# and t# format codes. It occurred to me that I've never used them, and probably never will, so I really don't care how they work: I'm only really worried about widespread ugliness, meaning wide enough that it touches me ;-) From tdelaney at avaya.com Wed Jan 11 00:08:27 2006 From: tdelaney at avaya.com (Delaney, Timothy (Tim)) Date: Wed, 11 Jan 2006 10:08:27 +1100 Subject: [Python-Dev] Include ctypes into core Python? Message-ID: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com> Guido van Rossum wrote: > On 1/10/06, Thomas Heller wrote: >> I would like to suggest to include ctypes into core Python, starting >> with the 2.5 release. > > On the one hand I agree that this is a useful module, popular, mature > etc. > > On the other hand it breaks one of the most fundamental Python > guidelines: if you get a core dump (segfault etc.) it's a bug in > Python or in a 3rd party extension, not in *your* Python code. An > exception would have to be made for any code that uses ctypes, as it > is usually trivial to cause core dumps with ctypes (I'd venture it's > hard to avoid them ;-). > > I don't expect this to count against including ctypes; but I do want > it to be dealt with somehow! As was pointed out on c.l.py, the `dl` module suffers the exact same problem (I don't know myself, as I've never used it). There are no warnings about this in the `dl` module documentation. I can't see how it would be possible to guarantee that such a module could not cause crashes. I'm of the opinion that having a big red warning at the top of the module documentation that this is a contributed module, and incorrect use could cause segmentation faults/crashes, etc would be sufficient. Tim Delaney From g.brandl-nospam at gmx.net Wed Jan 11 00:10:22 2006 From: g.brandl-nospam at gmx.net (Georg Brandl) Date: Wed, 11 Jan 2006 00:10:22 +0100 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com> References: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com> Message-ID: Delaney, Timothy (Tim) wrote: > Guido van Rossum wrote: > >> On 1/10/06, Thomas Heller wrote: >>> I would like to suggest to include ctypes into core Python, starting >>> with the 2.5 release. >> >> On the one hand I agree that this is a useful module, popular, mature >> etc. >> >> On the other hand it breaks one of the most fundamental Python >> guidelines: if you get a core dump (segfault etc.) it's a bug in >> Python or in a 3rd party extension, not in *your* Python code. An >> exception would have to be made for any code that uses ctypes, as it >> is usually trivial to cause core dumps with ctypes (I'd venture it's >> hard to avoid them ;-). >> >> I don't expect this to count against including ctypes; but I do want >> it to be dealt with somehow! > > As was pointed out on c.l.py, the `dl` module suffers the exact same > problem (I don't know myself, as I've never used it). There are no > warnings about this in the `dl` module documentation. > > I can't see how it would be possible to guarantee that such a module > could not cause crashes. I'm of the opinion that having a big red > warning at the top of the module documentation that this is a > contributed module, and incorrect use could cause segmentation > faults/crashes, etc would be sufficient. +1. A warning for dl might not be the worst thing. Georg From guido at python.org Wed Jan 11 00:12:43 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 10 Jan 2006 15:12:43 -0800 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com> References: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com> Message-ID: On 1/10/06, Delaney, Timothy (Tim) wrote: > Guido van Rossum wrote: > > > On 1/10/06, Thomas Heller wrote: > >> I would like to suggest to include ctypes into core Python, starting > >> with the 2.5 release. > > > > On the one hand I agree that this is a useful module, popular, mature > > etc. > > > > On the other hand it breaks one of the most fundamental Python > > guidelines: if you get a core dump (segfault etc.) it's a bug in > > Python or in a 3rd party extension, not in *your* Python code. An > > exception would have to be made for any code that uses ctypes, as it > > is usually trivial to cause core dumps with ctypes (I'd venture it's > > hard to avoid them ;-). > > > > I don't expect this to count against including ctypes; but I do want > > it to be dealt with somehow! > > As was pointed out on c.l.py, the `dl` module suffers the exact same > problem (I don't know myself, as I've never used it). There are no > warnings about this in the `dl` module documentation. Good point. I think there should be such warnings though. > I can't see how it would be possible to guarantee that such a module > could not cause crashes. And I'm not asking for that. > I'm of the opinion that having a big red > warning at the top of the module documentation that this is a > contributed module, and incorrect use could cause segmentation > faults/crashes, etc would be sufficient. Works for me. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tdelaney at avaya.com Wed Jan 11 00:22:22 2006 From: tdelaney at avaya.com (Delaney, Timothy (Tim)) Date: Wed, 11 Jan 2006 10:22:22 +1100 Subject: [Python-Dev] Include ctypes into core Python? Message-ID: <2773CAC687FD5F4689F526998C7E4E5F074324@au3010avexu1.global.avaya.com> Guido van Rossum wrote: >> As was pointed out on c.l.py, the `dl` module suffers the exact same >> problem (I don't know myself, as I've never used it). There are no >> warnings about this in the `dl` module documentation. > > Good point. I think there should be such warnings though. Added documentation patch 1402224: http://sourceforge.net/tracker/index.php?func=detail&aid=1402224&group_i d=5470&atid=305470 May need a little more work on the wording. >> I can't see how it would be possible to guarantee that such a module >> could not cause crashes. > > And I'm not asking for that. > >> I'm of the opinion that having a big red >> warning at the top of the module documentation that this is a >> contributed module, and incorrect use could cause segmentation >> faults/crashes, etc would be sufficient. > > Works for me. Sounds like we agree on everything :) Maybe I should start channelling ... . Tim Delaney From thomas at xs4all.net Wed Jan 11 00:23:24 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Wed, 11 Jan 2006 00:23:24 +0100 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: References: Message-ID: <20060110232324.GO18916@xs4all.nl> On Tue, Jan 10, 2006 at 01:14:13PM -0800, Guido van Rossum wrote: > On the other hand it breaks one of the most fundamental Python > guidelines: if you get a core dump (segfault etc.) it's a bug in > Python or in a 3rd party extension, not in *your* Python code. An > exception would have to be made for any code that uses ctypes, as it > is usually trivial to cause core dumps with ctypes (I'd venture it's > hard to avoid them ;-). Aside from 'dl', what was also pointed out in c.l.py was the crashability of Python in general, even from pure Python code: centurion:~ > python < . Segmentation fault [...] >>> sys.setrecursionlimit(1<<30) >>> f = lambda f:f(f) >>> f(f) Segmentation fault There's more, all from Python itself. And sure, "well, don't do that then" is a perfectly valid response to most of these harebrained tricks, but it does put a lie to the 'uncrashable python' idea :) Not-for-or-against-including-ctypes-anyway'ly y'rs, -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From guido at python.org Wed Jan 11 00:35:08 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 10 Jan 2006 15:35:08 -0800 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: <20060110232324.GO18916@xs4all.nl> References: <20060110232324.GO18916@xs4all.nl> Message-ID: On 1/10/06, Thomas Wouters wrote: > Aside from 'dl', what was also pointed out in c.l.py was the crashability of > Python in general, even from pure Python code: > > centurion:~ > python < . > Segmentation fault This I think ought to be fixed; it's just (I presume) the parser stumbling over extremely invalid input. > [...] > >>> sys.setrecursionlimit(1<<30) > >>> f = lambda f:f(f) > >>> f(f) > Segmentation fault OK, point taken. > There's more, all from Python itself. And sure, "well, don't do that then" > is a perfectly valid response to most of these harebrained tricks, but it > does put a lie to the 'uncrashable python' idea :) I'm not saying it's uncrashable. I'm saying that if you crash it, it's a bug unless proven harebrained. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From skip at pobox.com Wed Jan 11 00:42:13 2006 From: skip at pobox.com (skip at pobox.com) Date: Tue, 10 Jan 2006 17:42:13 -0600 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com> References: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com> Message-ID: <17348.18005.164441.346058@montanaro.dyndns.org> Tim> I'm of the opinion that having a big red warning at the top of the Tim> module documentation ... Do we have semantic markup to support that? Skip From fredrik at pythonware.com Wed Jan 11 00:42:13 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 11 Jan 2006 00:42:13 +0100 Subject: [Python-Dev] building a module catalogue with buildbot Message-ID: I've been playing with "automatic module discovery" in an attempt to figure out how complete the library reference really is. I've created a simple script that tweaks the path to get rid of site-package stuff, and then scans the path for python modules and extensions, and writes the result to a text file. By combining module lists from different sources, you can trivially generate all sorts of custom module indexes (e.g. modules available on typical linux machines, modules not available in 2.5 on windows, etc). For some examples etc, see: http://effbot.org/lib/index-24.html http://effbot.org/lib/index-23.html http://effbot.org/lib/index-win32.html http://online.effbot.org/2006_01_01_archive.htm#20060110 http://online.effbot.org/2006_01_01_archive.htm#module-count Most importantly, this makes it easy to track down missing reference pages, and generate stubs for pages that "should exist". My initial thought was that we could ask alpha testers to run this script on their alpha builds, and report back, but it just struck me that the "buildbot" already builds stuff on a couple of interesting platforms. Can buildbot deal with custom test/validation scripts, and collect the output somewhere ? From raymond.hettinger at verizon.net Wed Jan 11 00:42:25 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Tue, 10 Jan 2006 18:42:25 -0500 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: Message-ID: <000801c6163f$833558e0$e427a044@oemcomputer> [Guido] > I'm not saying it's uncrashable. I'm saying that if you crash it, it's > a bug unless proven harebrained. +1 QOTW From crutcher at gmail.com Wed Jan 11 00:47:18 2006 From: crutcher at gmail.com (Crutcher Dunnavant) Date: Tue, 10 Jan 2006 15:47:18 -0800 Subject: [Python-Dev] [PATCH] Fix dictionary subclass semantics when used as global dictionaries Message-ID: There is an inconsistancy in the way that dictionary subclasses behave when they are used as as namespaces in execs. Basically, while python 2.4 permits the usage of dictionary subclasses for local environments, it still bypasses the subclass functions and uses the C API for global environments. The attached patch (and unittest!) addresses this issue. I'm pretty sure we keep the fast path in this. -- Crutcher Dunnavant monket.samedi-studios.com -------------- next part -------------- A non-text attachment was scrubbed... Name: dictsubclassexec.patch Type: application/octet-stream Size: 6621 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20060110/34be979a/attachment.obj From jcarlson at uci.edu Wed Jan 11 00:57:28 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 10 Jan 2006 15:57:28 -0800 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: <20060110232324.GO18916@xs4all.nl> References: <20060110232324.GO18916@xs4all.nl> Message-ID: <20060110154626.C003.JCARLSON@uci.edu> Thomas Wouters wrote: > centurion:~ > python < . > Segmentation fault > > [...] > >>> sys.setrecursionlimit(1<<30) > >>> f = lambda f:f(f) > >>> f(f) > Segmentation fault > > There's more, all from Python itself. And sure, "well, don't do that then" > is a perfectly valid response to most of these harebrained tricks, but it > does put a lie to the 'uncrashable python' idea :) Many of the "segfaulting Python" issues are platform specific. On a platform with a sane malloc, you get a MemoryError for the recursion. On a platform which doesn't see the current path as a readable file, you get "Access is denied." on the redirection attempt. As always, I'm sure that reasonable patches which work around such segfaulting cases will be acceptable, though remember that it may not be clear how to work around them. - Josiah From thomas at xs4all.net Wed Jan 11 01:03:10 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Wed, 11 Jan 2006 01:03:10 +0100 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: <20060110154626.C003.JCARLSON@uci.edu> References: <20060110232324.GO18916@xs4all.nl> <20060110154626.C003.JCARLSON@uci.edu> Message-ID: <20060111000310.GP18916@xs4all.nl> On Tue, Jan 10, 2006 at 03:57:28PM -0800, Josiah Carlson wrote: > As always, I'm sure that reasonable patches which work around such > segfaulting cases will be acceptable, though remember that it may not be > clear how to work around them. Sorry, I missed the point I was aiming at, I guess. I wasn't aiming for fixable bugs; I see these things as, with great effort, holding up your foot at an awkward angle so that it ends up right at the business end of your primed and lit medieval cannon. We can jump through hoops to stop those particular examples, but I'd rather spend time and effort to fix *real* bugs instead. I was just showing that if you want to crash Python, you need neither 'ctypes' nor 'dl'. I do agree a warning is suitable to both of those modules, though if anyone wants to nitpick, similar modules might be added to marshal and cPickle (and pickle, too!), when feeding invalid input. Those modules are more in the awkward-foot-angle terrain for me, though. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From crutcher at gmail.com Wed Jan 11 01:04:52 2006 From: crutcher at gmail.com (Crutcher Dunnavant) Date: Tue, 10 Jan 2006 16:04:52 -0800 Subject: [Python-Dev] Hello Message-ID: Sorry, sent a patch without an intro. My name is Crutcher Dunnavant I'm working on a doctorate in computer science (in modeling language practises). Before I got my master's degree, I used to work for Red Hat in North Carolina as an OS developer, and I now work in the San Fransisco Bay Area as a system analyst. -- Crutcher Dunnavant monket.samedi-studios.com From aahz at pythoncraft.com Wed Jan 11 01:08:44 2006 From: aahz at pythoncraft.com (Aahz) Date: Tue, 10 Jan 2006 16:08:44 -0800 Subject: [Python-Dev] [PATCH] Fix dictionary subclass semantics when used as global dictionaries In-Reply-To: References: Message-ID: <20060111000844.GA11919@panix.com> On Tue, Jan 10, 2006, Crutcher Dunnavant wrote: > > There is an inconsistancy in the way that dictionary subclasses behave > when they are used as as namespaces in execs. > > Basically, while python 2.4 permits the usage of dictionary subclasses > for local environments, it still bypasses the subclass functions and > uses the C API for global environments. The attached patch (and > unittest!) addresses this issue. Please submit the patch to SourceForge and report back with the SF ID. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From crutcher at gmail.com Wed Jan 11 01:25:07 2006 From: crutcher at gmail.com (Crutcher Dunnavant) Date: Tue, 10 Jan 2006 16:25:07 -0800 Subject: [Python-Dev] [PATCH] Fix dictionary subclass semantics when used as global dictionaries In-Reply-To: <20060111000844.GA11919@panix.com> References: <20060111000844.GA11919@panix.com> Message-ID: 1402289 On 1/10/06, Aahz wrote: > On Tue, Jan 10, 2006, Crutcher Dunnavant wrote: > > > > There is an inconsistancy in the way that dictionary subclasses behave > > when they are used as as namespaces in execs. > > > > Basically, while python 2.4 permits the usage of dictionary subclasses > > for local environments, it still bypasses the subclass functions and > > uses the C API for global environments. The attached patch (and > > unittest!) addresses this issue. > > Please submit the patch to SourceForge and report back with the SF ID. > -- > Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ > > "19. A language that doesn't affect the way you think about programming, > is not worth knowing." --Alan Perlis > -- Crutcher Dunnavant monket.samedi-studios.com From guido at python.org Wed Jan 11 01:38:56 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 10 Jan 2006 16:38:56 -0800 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: <20060111000310.GP18916@xs4all.nl> References: <20060110232324.GO18916@xs4all.nl> <20060110154626.C003.JCARLSON@uci.edu> <20060111000310.GP18916@xs4all.nl> Message-ID: On 1/10/06, Thomas Wouters wrote: > Sorry, I missed the point I was aiming at, I guess. I wasn't aiming for > fixable bugs; I see these things as, with great effort, holding up your foot > at an awkward angle so that it ends up right at the business end of your > primed and lit medieval cannon. We can jump through hoops to stop those > particular examples, but I'd rather spend time and effort to fix *real* bugs > instead. I'm not sure what makes those particular examples less than real bugs. Users *will* trip over these things. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From crutcher at gmail.com Wed Jan 11 02:09:27 2006 From: crutcher at gmail.com (Crutcher Dunnavant) Date: Tue, 10 Jan 2006 17:09:27 -0800 Subject: [Python-Dev] [PATCH] Fix dictionary subclass semantics when used as global dictionaries In-Reply-To: References: <20060111000844.GA11919@panix.com> Message-ID: This is the unittest that checks it. On trunk, the global and fall through cases will fail, but they pass with the patch. import unittest from test import test_support class DictSubclass(dict): get_notify_func = None def __getitem__(self, key): if self.get_notify_func: self.get_notify_func(key) return dict.__getitem__(self, key) set_notify_func = None def __setitem__(self, key, value): if self.set_notify_func: self.set_notify_func(key) return dict.__setitem__(self, key, value) del_notify_func = None def __delitem__(self, key): if self.del_notify_func: self.del_notify_func(key) return dict.__delitem__(self, key) class DictSubclassExecTest(unittest.TestCase): def test_subclassexec_setlocal(self): d = DictSubclass() l = [] d.set_notify_func = l.append exec "a = 1" in d d.set_notify_func = None self.assert_(d['a'] == 1) self.assert_('a' in l) def test_subclassexec_getlocal(self): d = DictSubclass() l = [] d.get_notify_func = l.append exec "a = 1; a" in d d.get_notify_func = None self.assert_(d['a'] == 1) self.assert_('a' in l) def test_subclassexec_dellocal(self): d = DictSubclass() l = [] d.del_notify_func = l.append exec "a = 1; del a" in d d.del_notify_func = None self.assert_(not d.has_key('a')) self.assert_('a' in l) def test_subclassexec_setglobal(self): d = DictSubclass() l = [] d.set_notify_func = l.append exec "a = 1\ndef foo():\n\tglobal a\n\tpass" in d d.set_notify_func = None self.assert_(d['a'] == 1) self.assert_('a' in l) def test_subclassexec_getglobal(self): d = DictSubclass() l = [] d.get_notify_func = l.append exec "a = 1; a\ndef foo():\n\tglobal a\n\tpass" in d d.get_notify_func = None self.assert_(d['a'] == 1) self.assert_('a' in l) def test_subclassexec_delglobal(self): d = DictSubclass() l = [] d.del_notify_func = l.append exec "a = 1; del a\ndef foo():\n\tglobal a\n\tpass" in d d.del_notify_func = None self.assert_(not d.has_key('a')) self.assert_('a' in l) def test_subclassexec_getfallthrough(self): ld = DictSubclass() ll = [] ld.get_notify_func = ll.append gd = DictSubclass() gl = [] gd.get_notify_func = gl.append gd['a'] = 1 exec 'b = a' in gd, ld gd.del_notify_func = None ld.del_notify_func = None self.assert_(ld['b'] == 1) self.assert_('a' in ll) self.assert_('a' in gl) def test_main(): test_support.run_unittest( DictSubclassExecTest, ) if __name__ == "__main__": test_main() On 1/10/06, Crutcher Dunnavant wrote: > 1402289 > > On 1/10/06, Aahz wrote: > > On Tue, Jan 10, 2006, Crutcher Dunnavant wrote: > > > > > > There is an inconsistancy in the way that dictionary subclasses behave > > > when they are used as as namespaces in execs. > > > > > > Basically, while python 2.4 permits the usage of dictionary subclasses > > > for local environments, it still bypasses the subclass functions and > > > uses the C API for global environments. The attached patch (and > > > unittest!) addresses this issue. > > > > Please submit the patch to SourceForge and report back with the SF ID. > > -- > > Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ > > > > "19. A language that doesn't affect the way you think about programming, > > is not worth knowing." --Alan Perlis > > > > > -- > Crutcher Dunnavant > monket.samedi-studios.com > -- Crutcher Dunnavant monket.samedi-studios.com From trentm at ActiveState.com Wed Jan 11 02:16:01 2006 From: trentm at ActiveState.com (Trent Mick) Date: Tue, 10 Jan 2006 17:16:01 -0800 Subject: [Python-Dev] building a module catalogue with buildbot In-Reply-To: References: Message-ID: <20060111011601.GF4721@activestate.com> [Fredrik Lundh wrote] > Can buildbot deal with custom test/validation scripts, and collect the output > somewhere ? Yes, it should be able to. However, it might change the part of the the master.cfg file that defines the build steps from being trivial (probably something like: building_python_factory = factory.GNUAutoconf( s(SVN, svnurl="http://svn.python.org/projects/python/trunk"), test=["make", "test"], ) ) to something a little less trivial. The standard GNUAutoconf class would have to be subclassed to add that extra step ("make check_for_missing_docs"?). That probably would be fairly straightforward. Neil and Martin would probably know better. I don't have access to the buildbot setup. Trent -- Trent Mick trentm at activestate.com From skip at pobox.com Wed Jan 11 02:54:19 2006 From: skip at pobox.com (skip at pobox.com) Date: Tue, 10 Jan 2006 19:54:19 -0600 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: References: <20060110232324.GO18916@xs4all.nl> <20060110154626.C003.JCARLSON@uci.edu> <20060111000310.GP18916@xs4all.nl> Message-ID: <17348.25931.469196.19964@montanaro.dyndns.org> Guido> I'm not sure what makes those particular examples less than real Guido> bugs. Users *will* trip over these things. Feel free to hack away at: http://wiki.python.org/moin/CrashingPython I have no idea where (if at all) it should be linked from elsewhere on the wiki. Skip From barry at python.org Wed Jan 11 03:09:49 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 10 Jan 2006 21:09:49 -0500 Subject: [Python-Dev] ConfigParser to save with order In-Reply-To: References: Message-ID: <1136945389.10328.7.camel@geddy.wooz.org> On Mon, 2006-01-09 at 14:08 -0300, Facundo Batista wrote: > We can rewrite ConfigParser to store everything and write it back in > the exact order it took it, with new values at the end of each section > (or where the user inserted it), but it'll took a big rewrite of the > module: right now it's using dictionaries and should use lists to > achieve that behaviour. > > What I wanted to add to the module was predicatibility: a very needed > feature when you're writing test cases (and that's where I got bite). It's also important when you do crazy things like keep your configuration files in a revision control system. That's something that I do as much as possible. In general it works great for some thing, and pretty poorly for others, but when it works it makes keeping systems in sync or backed up much easier. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060110/bb0509a7/attachment.pgp From facundobatista at gmail.com Wed Jan 11 03:49:18 2006 From: facundobatista at gmail.com (Facundo Batista) Date: Tue, 10 Jan 2006 23:49:18 -0300 Subject: [Python-Dev] ConfigParser to save with order In-Reply-To: <1136945389.10328.7.camel@geddy.wooz.org> References: <1136945389.10328.7.camel@geddy.wooz.org> Message-ID: 2006/1/10, Barry Warsaw : > It's also important when you do crazy things like keep your > configuration files in a revision control system. That's something that Good point, Barry. So, unless somebody screams a blatant "no", I'll apply my patch to gain predictability and reject the other one because, if further work is made, it should be done towards a controlled position structure (a.k.a. use lists instead of dictionaries). Thanks. . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From t-meyer at ihug.co.nz Wed Jan 11 05:08:29 2006 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Wed, 11 Jan 2006 17:08:29 +1300 Subject: [Python-Dev] ConfigParser to save with order In-Reply-To: References: Message-ID: <089A37AE-E802-4903-98A5-33BBA0D6E635@ihug.co.nz> [Guido] >> I think it's moot unless you also preserve comments. Ideally would be >> something that prserved everything (ordering, blank lines, comments >> etc.) from how it was read in. Modifying a value should keep its >> position. Adding a value should add it to the end of the section it's >> in (unless there are cases where ordering is important to the >> semantics -- are there?). [Facundo Batista] > We can rewrite ConfigParser to store everything and write it back in > the exact order it took it, with new values at the end of each section > (or where the user inserted it), but it'll took a big rewrite of the > module: right now it's using dictionaries and should use lists to > achieve that behaviour. It's not at all that complicated. Simply running through the original file when writing gets you the original order and comments. This is simple to do (a simple example is in the SpamBayes source code in the OptionsClass.py module). Remember that there has been a lot of discussion about how ConfigParser should work in the past; for example (ignoring c.l.p): =Tony.Meyer From ronaldoussoren at mac.com Wed Jan 11 07:45:03 2006 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Wed, 11 Jan 2006 07:45:03 +0100 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: <20060110154626.C003.JCARLSON@uci.edu> References: <20060110232324.GO18916@xs4all.nl> <20060110154626.C003.JCARLSON@uci.edu> Message-ID: On 11-jan-2006, at 0:57, Josiah Carlson wrote: > > On a platform which doesn't see the current path as a readable > file, you > get "Access is denied." on the redirection attempt. On my osx box using python 2.4.2: $ cat < . cat: stdin: Is a directory $ python < . Bus error And finally: $ python -c 'execfile(".")' Traceback (most recent call last): File "", line 1, in ? IOError: [Errno 21] Is a directory: '.' It seems there is a bug somewhere. Annoyingly 'gdb -tty=. python' doesn't crash. Ronald > > As always, I'm sure that reasonable patches which work around such > segfaulting cases will be acceptable, though remember that it may > not be > clear how to work around them. > > > - Josiah > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ > ronaldoussoren%40mac.com From theller at python.net Wed Jan 11 07:59:03 2006 From: theller at python.net (Thomas Heller) Date: Wed, 11 Jan 2006 07:59:03 +0100 Subject: [Python-Dev] Include ctypes into core Python? References: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com> Message-ID: Guido van Rossum writes: > On 1/10/06, Delaney, Timothy (Tim) wrote: >> Guido van Rossum wrote: >> >> > On 1/10/06, Thomas Heller wrote: >> >> I would like to suggest to include ctypes into core Python, starting >> >> with the 2.5 release. >> > >> > On the one hand I agree that this is a useful module, popular, mature >> > etc. >> > >> > On the other hand it breaks one of the most fundamental Python >> > guidelines: if you get a core dump (segfault etc.) it's a bug in >> > Python or in a 3rd party extension, not in *your* Python code. An >> > exception would have to be made for any code that uses ctypes, as it >> > is usually trivial to cause core dumps with ctypes (I'd venture it's >> > hard to avoid them ;-). >> > >> > I don't expect this to count against including ctypes; but I do want >> > it to be dealt with somehow! >> >> As was pointed out on c.l.py, the `dl` module suffers the exact same >> problem (I don't know myself, as I've never used it). There are no >> warnings about this in the `dl` module documentation. > > Good point. I think there should be such warnings though. > >> I can't see how it would be possible to guarantee that such a module >> could not cause crashes. > > And I'm not asking for that. > >> I'm of the opinion that having a big red >> warning at the top of the module documentation that this is a >> contributed module, and incorrect use could cause segmentation >> faults/crashes, etc would be sufficient. > > Works for me. Another possibility would be to emit a warning when the module (dl or ctypes, if included) is imported. warnings.warn("Incorrect usage of this module may crash Python", RuntimeWarning, stacklevel=2) Thomas From nyamatongwe at gmail.com Wed Jan 11 08:14:55 2006 From: nyamatongwe at gmail.com (Neil Hodgson) Date: Wed, 11 Jan 2006 18:14:55 +1100 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: References: Message-ID: <50862ebd0601102314r228a0d6hd977eb431ee0ec5c@mail.gmail.com> Won't ctypes completely replace dl? dl provides only a small subset of the functionality of ctypes and is very restricted in the set of argument types allowed. Neil From nnorwitz at gmail.com Wed Jan 11 08:22:56 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 10 Jan 2006 23:22:56 -0800 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: <20060110232324.GO18916@xs4all.nl> References: <20060110232324.GO18916@xs4all.nl> Message-ID: On 1/10/06, Thomas Wouters wrote: > > centurion:~ > python < . > Segmentation fault I fixed that in Oct in head and 2.4 branch. Although Skip filed a bug since Py_FatalError() is called which generates a core dump in debug builds at least. http://python.org/sf/1353504 I'm not sure what else can be done except perhaps just print to stderr and exit() rather than abort(). > >>> sys.setrecursionlimit(1<<30) > >>> f = lambda f:f(f) > >>> f(f) > Segmentation fault Added as Lib/test/crashers/recursive_call.py I'm sure there are many more similar to this one though. > There's more, all from Python itself. We should at least try to clearly document limitations such as these. That's why I added the crashers directory. Maybe we can even find volunteers to fix these. n From nnorwitz at gmail.com Wed Jan 11 08:49:23 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 10 Jan 2006 23:49:23 -0800 Subject: [Python-Dev] building a module catalogue with buildbot In-Reply-To: <20060111011601.GF4721@activestate.com> References: <20060111011601.GF4721@activestate.com> Message-ID: On 1/10/06, Trent Mick wrote: > [Fredrik Lundh wrote] > > Can buildbot deal with custom test/validation scripts, and collect the output > > somewhere ? > > Yes, it should be able to. However, it might change the part of the the > master.cfg file that defines the build steps from being trivial > to something a little less trivial. > > The standard GNUAutoconf class would have to be subclassed to add that > extra step ("make check_for_missing_docs"?). That probably would be > fairly straightforward. Neil and Martin would probably know better. I > don't have access to the buildbot setup. It looks like we could define a class similar to Test, such as: class Catalog(ShellCommand): name = "catalog" warnOnFailure = 0 # this was 1, not sure which we want description = ["catalog"] descriptionDone = ["catalog"] command = ["/f cmd here", "and", "other", "flags"] The factory is created like this: f = factory.GNUAutoconf(source, test=["make","testall"]) ISTM, we can just add this line: f.steps.append(s(Catalog, command=Catalog.command)) Does that make sense? We would just need /f's script in SVN. n PS If there was a method on ShellCommand, adding a step would be simpler, e.g., def addStep(self, cls, **kwds): self.steps.append((cls, kwds)) then we could do: f.addStep(Catalog, command=Catalog.command) which would simplify a bunch of code in process/factory.py since it is called this way 10 times. Didn't look elsewhere. From warner at lothar.com Wed Jan 11 09:36:09 2006 From: warner at lothar.com (Brian Warner) Date: Wed, 11 Jan 2006 00:36:09 -0800 (PST) Subject: [Python-Dev] Buildbot: doing occasional full builds In-Reply-To: <20060107001221.GA13845@activestate.com> References: <20060106173110.GD15276@activestate.com> <43BEE8F4.5080304@v.loewis.de> <20060107001221.GA13845@activestate.com> Message-ID: <20060111.003609.21623845.warner@lothar.com> > > > To wipe out the build occassionally you could (presumably) add a > > > starting step to the Python 'builder' (in the build master.cfg) to > > > rm -rf $builddir > > > every, say, Sunday night. That would work, although to be honest the buildbot is more about repeatable builds. My first suggestion would be to have two separate Builders, one of which does incremental builds, the other which does full (from-scratch) builds. Both modes are useful to test: I've seen build problems that affected one but not the other (deleted source files that leave .o files in place, Makefile changes that don't notice modified files correctly, etc). If you want to minimize the compile load placed on the build slaves, you could use the Schedulers to reduce the frequency of triggering the "full" builder. A subclass of the default Scheduler class could enforce a minimum-time-between-builds timer, and you could crank it up to only allow one full build per hour, or per day, or whatever. The reason I'd encourage you to let every build of a given Builder behave the same way is that I hope to add more problem-tracking code to the Buildbot, and that code will work better if builds really are reproducible and stable. If the tests pass in build #9 and fail in build #10, you should be able to blame the failure on the changes that went into build #10. If instead the failure is due to the fact that build #10 just happened to be the one where you do a clean build instead of an incremental one, then it will be harder to automatically isolate the cause of the failure. It isn't a big issue, though. Just thought I'd share what my design philosophy is so you can decide which direction you want to go. The code is flexible enough to handle it. > The current problem with this is that I don't know if the build step > objects have access to a local data store -- so it could, say, count how > many builds have been done to know to clobber every tenth one. The > current code just chooses to clobber for any build on Sunday. Not a very > satisfying solution. > > Brian, > Is there a way for build steps to maintain some state? In general, no.. the long-term status is kept in a separate BuildStatus object, the idea is that BuildStatus gets persisted but Builds do not. You can always have your Build subclass use the regular status interface to look up an earlier BuildStatus, and base some decisions upon the data you find there, although I haven't really provided a way to set arbitrary attributes on the BuildStatus. For this case, however, you can have the Build reach into its BuildStatus object and look at the .number attribute, and then do a simple %10==0 test. Not especially elegant but it would get the job done. From a BuildStep you would use 'self.build.build_status.number % 10 == 0'. Apart from that, your SVNEx subclass seems feasible. I'd be inclined to have the subclass always implement this update_and_clobber_occassionally behavior (rather than providing a special mode= switch for it), but that's just a usage detail. cheers, -Brian From warner at lothar.com Wed Jan 11 09:39:50 2006 From: warner at lothar.com (Brian Warner) Date: Wed, 11 Jan 2006 00:39:50 -0800 (PST) Subject: [Python-Dev] Buildbot questions In-Reply-To: <20060107001651.GB13845@activestate.com> References: <20060107001221.GA13845@activestate.com> <43BF07C1.3060608@v.loewis.de> <20060107001651.GB13845@activestate.com> Message-ID: <20060111.003950.48501461.warner@lothar.com> > > Ah, but that would require changes to the slaves, right? I would > > prefer a solution that avoids that. > > I don't think so. In my little test setup I didn't have to make any > change to the slave. The "update" and "clobber" mode parameters are implemented on the slave side. Trent's patch changes which mode setting is used on the master side before anything is sent to the slaves, so the slaves don't have to know anything about it. In general there is as little slave-side code as possible, to reduce the need to upgrade or change the code on that side. The VC checkout/update operations are an exception, since they may do several operations in a row (which would mean a lot more latency to do them without support from code on the slave side), and because I'm lazy and prefer to implement the algorithms in Python instead of in Bourne shell :). -Brian From warner at lothar.com Wed Jan 11 10:24:29 2006 From: warner at lothar.com (Brian Warner) Date: Wed, 11 Jan 2006 01:24:29 -0800 (PST) Subject: [Python-Dev] buildbot In-Reply-To: <43C36D3C.7050202@v.loewis.de> References: <43BD9B32.3030700@v.loewis.de> <20060108.173810.91561560.warner@lothar.com> <43C36D3C.7050202@v.loewis.de> Message-ID: <20060111.012429.45715174.warner@lothar.com> > The reason I want static pages is for security concerns. It is not > easy whether buildbot can be trusted to have no security flaws, > which might allow people to start new processes on the master, > or (perhaps worse) on any of the slaves. These are excellent points. While it would take a complete security audit to satisfy the kind of paranoia *I* tend to carry around, I can mention a couple of points about the buildbot's security design that might help you make some useful decisions about it: The buildmaster "owns" (spelled "pwns" or "0wnz0red" these days, according to my "leet-speak phrasebook" :) the buildslaves. It can make them run whatever shell commands it likes, therefore it has full control of the buildslave accounts. It is appropriate to give the buildslaves their own account, with limited privileges. The codebase "owns" the buildslaves too: most build commands will wind up running './configure' or 'make' or something which executes commands that are provided by the checked out source tree. Nobody is supposed to "own" the buildmaster: it performs build scheduling and status reporting according to its own design and configuration file. A compromised codebase cannot make the buildmaster do anything unusual, nor can a compromised buildslave. The worst that a buildslave can do is cause a DoS attack by sending overly large command-status messages, which can prevent the buildmaster from doing anything useful (in the worst case causing it to run out of memory), but cannot make it do anything it isn't supposed to. The top-level IRC functions can be audited by inspecting the command_ methods, as you've already seen. The HTTP status page can be audited similarly, once you know how Twisted-Web works (there is a hierarchy of Resource objects, each component of the URL path uses Resource.getChild to obtain the child node in this tree, once the final child is retrieved then the 'render' method is called to produce HTML). The Waterfall resource and all its children get their capabilities from two objects: Status (which provides read-only status information about all builds) and Control (which is the piece that allows things like "force build"). The knob that disables the "Force Build" button does so by creating the Waterfall instance with control=None. If you can verify that the code doesn't go out of its way to acquire a Control reference through some private-use-only attribute, then you can be reasonably confident that it isn't possible to make the web server do anything to trigger a build. It's not restricted-execution mode or anything, but it's designed with capability-based security in mind, and that may help someone who wishes to audit it. The PBListener status interface is similar: PB guarantees that only remote_* methods can be invoked by a remote client, and the PBListener object only has a reference to the top-level Status object. The slave->master connection (via the 'slaveport') uses PB, so it can be audited the same way. Only the remote_* (and perspective_*) methods of objects which are provided to the buildslave can be invoked. The buildslaves are allowed to do two things to the top-level buildmaster: force a build that is run on their own machine, and invoke an empty 'keepalive' method. During a build, they can send remote_update and remote_complete messages to the current BuildStep: this is how they deliver status information (specifically the output of shell commands). By inspecting buildbot.process.step.RemoteCommand.remote_update, you can verify that the update is appended to a logfile and nothing else. PB's serialization is designed specifically to be safe (in explicit contrast to pickle). Arbitrary classes cannot be sent over the wire. The worst-case attack is DoS, specifically memory exhaustion. Any application which can talk to the outside world is a security concern. The tools that we have to insure that these applications only do what we intended them to do are not as fully developed as we would like (I hang out with the developers of E, and would love to implement the buildbot in a capability-secure form of Python, but such a beast is not available right now, and I'm spending too much time writing Buildbot code to get around to writing a more secureable language too). So we write our programs in as clear a way as possible, and take advantage of tools that have been developed or inspected by people we respect. These days my paranoia tells me to trust a webserver written in Python more than one written in C. Buffer overruns are the obvious thing, but another important distinction is how Twisted's web server architecture treats the URL as a path of edges in a tree of Resource instances rather than as a pathname to a file on the disk. I don't need to worry about what kind of URLs might give access to the master.cfg file (which could contain debugging passwords or something), as long as I can tell that none of the Resource instances give access to it. This also makes preventing things like http://foo/../../oops.txt much much easier. Preferring a Twisted web server over Apache reveals my bias, both in favor of Python and the developers of Twisted and Twisted's web server, and I quite understand if you don't share that bias. I think it would be quite possible to create a 'StaticWaterfall' status class, which would write HTML to a tree of files each time something changed. There are a number of status event delivery APIs in the buildbot which could cause a method to be called each time a Step was started or finished, and these could just write new HTML to a file. It would consume a bit more disk space, but would allow an external webserver to provide truly read-only access to build status. If you'd like me to spend some cycles on this, please let me know.. perhaps others would prefer this style of status delivery too. cheers, -Brian From mwh at python.net Wed Jan 11 12:56:32 2006 From: mwh at python.net (Michael Hudson) Date: Wed, 11 Jan 2006 11:56:32 +0000 Subject: [Python-Dev] test_curses In-Reply-To: (Georg Brandl's message of "Tue, 10 Jan 2006 20:06:48 +0100") References: <2m1wzhsqq5.fsf@starship.python.net> <2m1wzgqvh5.fsf@starship.python.net> Message-ID: <2mslrvrn9r.fsf@starship.python.net> Georg Brandl writes: >> Well, this still has the faint whiff of impossibility about it. Are >> you sure it's setupterm() that's doing the damage? Can you reproduce >> interactively? > > Yep. > Alone, the setupterm call [curses.setupterm(sys.__stdout__.fileno())] does > nothing remarkable, but when it is done inside of curses.initscr() / curses.endwin() > the terminal is not restored properly. Ah! That does indeed make some small amount of sense. > If setupterm() is to be tested, I think it should be executed before initscr(). Sounds good. What are you waiting for? :) Cheers, mwh -- Jokes around here tend to get followed by implementations. -- from Twisted.Quotes From barry at python.org Wed Jan 11 13:59:50 2006 From: barry at python.org (Barry Warsaw) Date: Wed, 11 Jan 2006 07:59:50 -0500 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: References: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com> Message-ID: <1136984390.12149.6.camel@geddy.wooz.org> On Wed, 2006-01-11 at 07:59 +0100, Thomas Heller wrote: > Another possibility would be to emit a warning when the module (dl or > ctypes, if included) is imported. > > warnings.warn("Incorrect usage of this module may crash Python", > RuntimeWarning, stacklevel=2) BTW, although I'm pretty sure the answer is "no" (at least, I hope it is), is anyone aware of a situation where the mere importation of a module can cause Python to crash? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060111/0eedc77f/attachment.pgp From thomas at xs4all.net Wed Jan 11 14:54:40 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Wed, 11 Jan 2006 14:54:40 +0100 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: <1136984390.12149.6.camel@geddy.wooz.org> References: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com> <1136984390.12149.6.camel@geddy.wooz.org> Message-ID: <20060111135440.GQ18916@xs4all.nl> On Wed, Jan 11, 2006 at 07:59:50AM -0500, Barry Warsaw wrote: > BTW, although I'm pretty sure the answer is "no" (at least, I hope it > is), is anyone aware of a situation where the mere importation of a > module can cause Python to crash? Well, I assume you aren't importing any 'hostile' code, nor running in an uncontrolled environment so I guess you mean other than running out of memory, or the module you are importing actually executing one of the ways to crash Python? Or the module being an extension module that crashes on import? Or another extension module having corrupted the Python environment to a point where a simple import crashes Python? Or a non-extension module using one of the vulnerabilities (in, say, marshal, or pickle) to corrupt the Python environment? Or stuff in os.environ, like LD_* variables, that interfere with library linking? Ponder, ponder, no, can't think of any. :) The pickle vulnerability came up last year, when someone on #python was subclassing a builtin type (string or dict, I think the latter) that was using a magical invocation of (IIRC) __new__ on unpickle. The subclassed __new__ didn't handle this right, so the baseclass __new__ wasn't getting called right, and the new object's addressspace was not initialized. This lead to crashes. I don't remember the details exactly, and my continuous advice of not subclassing builtin types unless you know what you're doing solved the issue (there was no actual need to subclass, there), and I have no idea whether that specific issue was solved or not, but I'm trying to find it again :) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From barry at python.org Wed Jan 11 15:02:50 2006 From: barry at python.org (Barry Warsaw) Date: Wed, 11 Jan 2006 09:02:50 -0500 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: <20060111135440.GQ18916@xs4all.nl> References: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com> <1136984390.12149.6.camel@geddy.wooz.org> <20060111135440.GQ18916@xs4all.nl> Message-ID: <1136988170.12149.12.camel@geddy.wooz.org> On Wed, 2006-01-11 at 14:54 +0100, Thomas Wouters wrote: > On Wed, Jan 11, 2006 at 07:59:50AM -0500, Barry Warsaw wrote: > > > BTW, although I'm pretty sure the answer is "no" (at least, I hope it > > is), is anyone aware of a situation where the mere importation of a > > module can cause Python to crash? > > Well, I assume you aren't importing any 'hostile' code, nor running in an > uncontrolled environment so I guess you mean other than running out of > memory, or the module you are importing actually executing one of the ways > to crash Python? Correct. > Or the module being an extension module that crashes on > import? Or another extension module having corrupted the Python environment > to a point where a simple import crashes Python? Or a non-extension module > using one of the vulnerabilities (in, say, marshal, or pickle) to corrupt > the Python environment? Or stuff in os.environ, like LD_* variables, that > interfere with library linking? Ponder, ponder, no, can't think of any. :) Let's keep it to modules in the standard library, although that includes extension modules, and let's further say that it's a controlled enough environment that you won't have stray evil modules floating around in your sys.path. > The pickle vulnerability came up last year, when someone on #python was > subclassing a builtin type (string or dict, I think the latter) that was > using a magical invocation of (IIRC) __new__ on unpickle. The subclassed > __new__ didn't handle this right, so the baseclass __new__ wasn't getting > called right, and the new object's addressspace was not initialized. This > lead to crashes. I don't remember the details exactly, and my continuous > advice of not subclassing builtin types unless you know what you're doing > solved the issue (there was no actual need to subclass, there), and I have > no idea whether that specific issue was solved or not, but I'm trying to > find it again :) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060111/0d8a82fb/attachment.pgp From mwh at python.net Wed Jan 11 15:03:48 2006 From: mwh at python.net (Michael Hudson) Date: Wed, 11 Jan 2006 14:03:48 +0000 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: (Guido van Rossum's message of "Tue, 10 Jan 2006 16:38:56 -0800") References: <20060110232324.GO18916@xs4all.nl> <20060110154626.C003.JCARLSON@uci.edu> <20060111000310.GP18916@xs4all.nl> Message-ID: <2moe2isvy3.fsf@starship.python.net> Guido van Rossum writes: > On 1/10/06, Thomas Wouters wrote: >> Sorry, I missed the point I was aiming at, I guess. I wasn't aiming for >> fixable bugs; I see these things as, with great effort, holding up your foot >> at an awkward angle so that it ends up right at the business end of your >> primed and lit medieval cannon. We can jump through hoops to stop those >> particular examples, but I'd rather spend time and effort to fix *real* bugs >> instead. > > I'm not sure what makes those particular examples less than real bugs. I think the sys.setrecursionlimit(1<<30) one is fairly well into the terriotory of "don't do that", isn't it? Cheers, mwh -- today's lesson don't strace X in an xterm -- from Twisted.Quotes From thomas at xs4all.net Wed Jan 11 15:20:04 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Wed, 11 Jan 2006 15:20:04 +0100 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: <20060111135440.GQ18916@xs4all.nl> References: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com> <1136984390.12149.6.camel@geddy.wooz.org> <20060111135440.GQ18916@xs4all.nl> Message-ID: <20060111142004.GR18916@xs4all.nl> On Wed, Jan 11, 2006 at 02:54:40PM +0100, Thomas Wouters wrote: > The pickle vulnerability came up last year, when someone on #python was > subclassing a builtin type (string or dict, I think the latter) that was > using a magical invocation of (IIRC) __new__ on unpickle. The subclassed > __new__ didn't handle this right, so the baseclass __new__ wasn't getting > called right, and the new object's addressspace was not initialized. This > lead to crashes. I don't remember the details exactly, and my continuous > advice of not subclassing builtin types unless you know what you're doing > solved the issue (there was no actual need to subclass, there), and I have > no idea whether that specific issue was solved or not, but I'm trying to > find it again :) Ah, found it: it was one of the datetime types. It has guards in place (some back then, python2.3, more since 2.4) so I wasn't able to figure out why it actually crashed Python, rather than produce a weird date. I couldn't find anything obviously wrong with the data's handling (although the extra guards are good.) I'll see if I can reproduce it anyway. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From skip at pobox.com Wed Jan 11 15:37:49 2006 From: skip at pobox.com (skip at pobox.com) Date: Wed, 11 Jan 2006 08:37:49 -0600 Subject: [Python-Dev] Limiting the recursion limit In-Reply-To: References: <20060110232324.GO18916@xs4all.nl> Message-ID: <17349.6205.976505.883903@montanaro.dyndns.org> >>> sys.setrecursionlimit(1<<30) >>> f = lambda f:f(f) >>> f(f) Segmentation fault Is there some way that Python can determine that 1<<30 is an unreasonable recursion limit? Skip From guido at python.org Wed Jan 11 18:34:28 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 11 Jan 2006 09:34:28 -0800 Subject: [Python-Dev] Limiting the recursion limit In-Reply-To: <17349.6205.976505.883903@montanaro.dyndns.org> References: <20060110232324.GO18916@xs4all.nl> <17349.6205.976505.883903@montanaro.dyndns.org> Message-ID: On 1/11/06, skip at pobox.com wrote: > > >>> sys.setrecursionlimit(1<<30) > >>> f = lambda f:f(f) > >>> f(f) > Segmentation fault > > Is there some way that Python can determine that 1<<30 is an unreasonable > recursion limit? Yes, but that doesn't help -- there's some value in the middle where you may or may not get a crash depending on random other stuff that is going on, and the only way to know is to try. It's easy enough to code a loop that tries higher and higher values until it finds a crash. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ronaldoussoren at mac.com Wed Jan 11 19:04:51 2006 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Wed, 11 Jan 2006 19:04:51 +0100 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: References: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com> Message-ID: On 11-jan-2006, at 7:59, Thomas Heller wrote: >> >>> I'm of the opinion that having a big red >>> warning at the top of the module documentation that this is a >>> contributed module, and incorrect use could cause segmentation >>> faults/crashes, etc would be sufficient. >> >> Works for me. > > Another possibility would be to emit a warning when the module (dl or > ctypes, if included) is imported. > > warnings.warn("Incorrect usage of this module may crash Python", > RuntimeWarning, stacklevel=2) Yuck. Just adding a warning to the documentation should be good enough. Other modules allow you to make your machine unuseable (such a fork bomb using os.fork) and those don't warn either. Ronald > > Thomas > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ > ronaldoussoren%40mac.com From nick at craig-wood.com Wed Jan 11 19:16:41 2006 From: nick at craig-wood.com (Nick Craig-Wood) Date: Wed, 11 Jan 2006 18:16:41 +0000 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: References: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com> Message-ID: <20060111181641.GA8149@craig-wood.com> On Wed, Jan 11, 2006 at 07:59:03AM +0100, Thomas Heller wrote: > Another possibility would be to emit a warning when the module (dl or > ctypes, if included) is imported. > > warnings.warn("Incorrect usage of this module may crash Python", > RuntimeWarning, stacklevel=2) Arrgggh! No!! Warnings are one of the things I really hate about perl and I'd hate to see them infecting python any more than they already have. I want my programming language to tell me, the programmer, about stuff not go blurting to the user behind my back. Death to warnings ;-) (Yes I know about the warnings module!) -- Nick Craig-Wood -- http://www.craig-wood.com/nick From turnbull at sk.tsukuba.ac.jp Sun Jan 8 17:10:50 2006 From: turnbull at sk.tsukuba.ac.jp (Stephen J. Turnbull) Date: Mon, 09 Jan 2006 01:10:50 +0900 Subject: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py In-Reply-To: (Fredrik Lundh's message of "Sun, 8 Jan 2006 15:57:06 +0100") References: Message-ID: <87zmm6d7jp.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Fredrik" == Fredrik Lundh writes: Fredrik> many test frameworks support "expected failures" for this Fredrik> purpose. how hard would it be to add a Fredrik> unittest.FailingTestCase Fredrik> class that runs a TestCase, catches any errors in it, and Fredrik> signals an error ("test foo passed unexpectedly") if it Fredrik> runs cleanly ? One can do even better than that. unittest.FailingTestCase should (except possibly for platform dependencies) know _how_ the TestCase is expected to fail. You also want to know if the error changes. -- Graduate School of Systems and Information Engineering University of Tsukuba http://turnbull.sk.tsukuba.ac.jp/ Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Economics of Information Communication and Computation Systems Experimental Economics, Microeconomic Theory, Game Theory From stephendavis at mac.com Tue Jan 10 18:18:59 2006 From: stephendavis at mac.com (Stephen Davis) Date: Tue, 10 Jan 2006 09:18:59 -0800 Subject: [Python-Dev] [Buildbot-devel] Re: buildbot In-Reply-To: <43C36D3C.7050202@v.loewis.de> References: <20060104.003058.48524852.warner@lothar.com> <43BD9B32.3030700@v.loewis.de> <20060108.173810.91561560.warner@lothar.com> <43C36D3C.7050202@v.loewis.de> Message-ID: > The reason I want static pages is for security concerns. It is not > easy whether buildbot can be trusted to have no security flaws, > which might allow people to start new processes on the master, > or (perhaps worse) on any of the slaves. I have security concerns as well, but not in buildbot itself. My project is restricted even withinz the company I work for so I need the buildbot web server to only be available to certain people. HTTPS access would be nice too. TwistedWeb doesn't seem to have support for either HTTPS or authentication so I've been forced to "hide" it by putting it on a non-standard port. Very weak. I am no networking expert so the suggestions for using a reverse proxy are very welcome and I will look into that right away. Just wanted to add my voice to the security concerns. stephen From trentm at ActiveState.com Wed Jan 11 20:59:00 2006 From: trentm at ActiveState.com (Trent Mick) Date: Wed, 11 Jan 2006 11:59:00 -0800 Subject: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py In-Reply-To: <87zmm6d7jp.fsf@tleepslib.sk.tsukuba.ac.jp> References: <87zmm6d7jp.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <20060111195900.GB17269@activestate.com> [Stephen J. Turnbull wrote] > >>>>> "Fredrik" == Fredrik Lundh writes: > > Fredrik> many test frameworks support "expected failures" for this > Fredrik> purpose. how hard would it be to add a > > Fredrik> unittest.FailingTestCase > > Fredrik> class that runs a TestCase, catches any errors in it, and > Fredrik> signals an error ("test foo passed unexpectedly") if it > Fredrik> runs cleanly ? > > One can do even better than that. unittest.FailingTestCase should > (except possibly for platform dependencies) know _how_ the TestCase is > expected to fail. You also want to know if the error changes. How about this: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/307970 a better assertRaises() for unittest.py When writing unit tests for Python using the standard unittest.py system the assertRaises() (aka failUnlessRaises()) method is used to test that a particular call raises the given exception. This recipe if for assertRaisesEx() that adds three things: (1) the ability to assert the raised exception's args; (2) the ability to test that the stringified exception matches a given regular expression; and (3) much better failure messages. I haven't read this thread, so apologies is this doesn't really apply to the discussion. Cheers, Trent -- Trent Mick trentm at activestate.com From trentm at ActiveState.com Wed Jan 11 21:05:28 2006 From: trentm at ActiveState.com (Trent Mick) Date: Wed, 11 Jan 2006 12:05:28 -0800 Subject: [Python-Dev] building a module catalogue with buildbot In-Reply-To: References: <20060111011601.GF4721@activestate.com> Message-ID: <20060111200528.GA23752@activestate.com> [Neal Norwitz wrote] > > [Fredrik Lundh wrote] > > > Can buildbot deal with custom test/validation scripts, and collect the output > > > somewhere ? > > > ... > It looks like we could define a class similar to Test, such as: > > class Catalog(ShellCommand): > name = "catalog" > warnOnFailure = 0 # this was 1, not sure which we want > description = ["catalog"] > descriptionDone = ["catalog"] > command = ["/f cmd here", "and", "other", "flags"] > > The factory is created like this: > f = factory.GNUAutoconf(source, test=["make","testall"]) > > ISTM, we can just add this line: > f.steps.append(s(Catalog, command=Catalog.command)) > > Does that make sense? We would just need /f's script in SVN. Yup. Sounds right to me. > PS If there was a method on ShellCommand, adding a step would be simpler, e.g., > > def addStep(self, cls, **kwds): > self.steps.append((cls, kwds)) > > then we could do: f.addStep(Catalog, command=Catalog.command) > which would simplify a bunch of code in process/factory.py since it is > called this way 10 times. Didn't look elsewhere. Add a buildbot patch/bug? http://sourceforge.net/tracker/?func=add&group_id=73177&atid=537003 Trent -- Trent Mick trentm at activestate.com From martin at v.loewis.de Wed Jan 11 21:06:59 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 11 Jan 2006 21:06:59 +0100 Subject: [Python-Dev] Buildbot: doing occasional full builds In-Reply-To: <20060111.003609.21623845.warner@lothar.com> References: <20060106173110.GD15276@activestate.com> <43BEE8F4.5080304@v.loewis.de> <20060107001221.GA13845@activestate.com> <20060111.003609.21623845.warner@lothar.com> Message-ID: <43C56563.9060107@v.loewis.de> Brian Warner wrote: > That would work, although to be honest the buildbot is more about repeatable > builds. My first suggestion would be to have two separate Builders, one of > which does incremental builds, the other which does full (from-scratch) > builds. My concern is that then the number of builders would go up from 8 to 16, even though it is just four different machines that we use. Regards, Martin From trentm at ActiveState.com Wed Jan 11 21:45:33 2006 From: trentm at ActiveState.com (Trent Mick) Date: Wed, 11 Jan 2006 12:45:33 -0800 Subject: [Python-Dev] Buildbot: doing occasional full builds In-Reply-To: <43C56563.9060107@v.loewis.de> References: <20060106173110.GD15276@activestate.com> <43BEE8F4.5080304@v.loewis.de> <20060107001221.GA13845@activestate.com> <20060111.003609.21623845.warner@lothar.com> <43C56563.9060107@v.loewis.de> Message-ID: <20060111204533.GB30010@activestate.com> [Martin v. Loewis wrote] > Brian Warner wrote: > > That would work, although to be honest the buildbot is more about repeatable > > builds. My first suggestion would be to have two separate Builders, one of > > which does incremental builds, the other which does full (from-scratch) > > builds. > > My concern is that then the number of builders would go up from 8 to 16, > even though it is just four different machines that we use. Specifically are you concerned about the readability of the waterfall page or other things (maintainability or something)? If the former, perhaps we could get some mileage out of the query args that Brian mentioned: [Brian Warner wrote] > I should mention a feature that you might find useful: you can add > some query arguments to the top-level Waterfall page's URL to > restrict the set of Builders that are displayed. If you append > "?show=full-2.3", you'll wind up with a waterfall display that > only contains the "full-2.3" Builder's column. Appending > "?show=full-2.3&show=full-2.4" will show two columns, etc. You can > also create multiple Waterfall instances, each with a different > set of "categories", and use that to manage the complexity of > displaying status for lots of Builders. (with the current release > these multiple Waterfalls must all listen on different TCP ports, > unfortunately, but you can always hide this with the > reverse-proxy's URL mapping). Trent -- Trent Mick trentm at activestate.com From skip at pobox.com Wed Jan 11 21:53:50 2006 From: skip at pobox.com (skip at pobox.com) Date: Wed, 11 Jan 2006 14:53:50 -0600 Subject: [Python-Dev] Buildbot: doing occasional full builds In-Reply-To: <20060111204533.GB30010@activestate.com> References: <20060106173110.GD15276@activestate.com> <43BEE8F4.5080304@v.loewis.de> <20060107001221.GA13845@activestate.com> <20060111.003609.21623845.warner@lothar.com> <43C56563.9060107@v.loewis.de> <20060111204533.GB30010@activestate.com> Message-ID: <17349.28766.364593.662110@montanaro.dyndns.org> Trent> Specifically are you concerned about the readability of the Trent> waterfall page or other things (maintainability or something)? If Trent> the former, perhaps we could get some mileage out of the query Trent> args that Brian mentioned: One concern might be performance. All buildbot slaves are contributed hardware. I don't mind the load on my Mac (it's a dual processor g5), but it may be an issue for other people. Does/can buildbot run at a lower priority or is it just a matter of nice-ing the slave process at startup? Skip From nnorwitz at gmail.com Wed Jan 11 22:02:03 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 11 Jan 2006 13:02:03 -0800 Subject: [Python-Dev] Buildbot: doing occasional full builds In-Reply-To: <17349.28766.364593.662110@montanaro.dyndns.org> References: <20060106173110.GD15276@activestate.com> <43BEE8F4.5080304@v.loewis.de> <20060107001221.GA13845@activestate.com> <20060111.003609.21623845.warner@lothar.com> <43C56563.9060107@v.loewis.de> <20060111204533.GB30010@activestate.com> <17349.28766.364593.662110@montanaro.dyndns.org> Message-ID: On 1/11/06, skip at pobox.com wrote: > > One concern might be performance. All buildbot slaves are contributed > hardware. I don't mind the load on my Mac (it's a dual processor g5), but > it may be an issue for other people. I've "contributed" 2 machines. One is my personal box, the other is the PSFs. The PSF box is very lightly used except by automated stuff, so it's not a problem. My box is a dual opteron and it really isn't a problem for me right now. I suppose if it was building all the time, I might mind, but so far, there's no issues. Hopefully over time we will get more people contributing other boxes and this might become a bigger issue. But right now I don't think it's a problem. n From trentm at ActiveState.com Wed Jan 11 22:25:30 2006 From: trentm at ActiveState.com (Trent Mick) Date: Wed, 11 Jan 2006 13:25:30 -0800 Subject: [Python-Dev] Buildbot: doing occasional full builds In-Reply-To: <17349.28766.364593.662110@montanaro.dyndns.org> References: <20060106173110.GD15276@activestate.com> <43BEE8F4.5080304@v.loewis.de> <20060107001221.GA13845@activestate.com> <20060111.003609.21623845.warner@lothar.com> <43C56563.9060107@v.loewis.de> <20060111204533.GB30010@activestate.com> <17349.28766.364593.662110@montanaro.dyndns.org> Message-ID: <20060111212530.GA6642@activestate.com> [skip at pobox.com wrote] > > Trent> Specifically are you concerned about the readability of the > Trent> waterfall page or other things (maintainability or something)? If > Trent> the former, perhaps we could get some mileage out of the query > Trent> args that Brian mentioned: > > One concern might be performance. All buildbot slaves are contributed > hardware. I don't mind the load on my Mac (it's a dual processor g5), but > it may be an issue for other people. Does/can buildbot run at a lower > priority or is it just a matter of nice-ing the slave process at startup? As (I think) Brian mentioned, the scheduler for the (proposed) "full builder" could be setup to just build, say, once per day or only a couple of times per day where as the "incremental builder" would trigger for every SVN checkin. Nicing could probably also be done. Trent -- Trent Mick trentm at activestate.com From fredrik at pythonware.com Wed Jan 11 22:45:06 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 11 Jan 2006 22:45:06 +0100 Subject: [Python-Dev] building a module catalogue with buildbot References: <20060111011601.GF4721@activestate.com> Message-ID: Neal Norwitz wrote: > Does that make sense? We would just need /f's script in SVN. in python/Tools/something or sandbox/something ? From martin at v.loewis.de Wed Jan 11 22:50:48 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 11 Jan 2006 22:50:48 +0100 Subject: [Python-Dev] Buildbot: doing occasional full builds In-Reply-To: <17349.28766.364593.662110@montanaro.dyndns.org> References: <20060106173110.GD15276@activestate.com> <43BEE8F4.5080304@v.loewis.de> <20060107001221.GA13845@activestate.com> <20060111.003609.21623845.warner@lothar.com> <43C56563.9060107@v.loewis.de> <20060111204533.GB30010@activestate.com> <17349.28766.364593.662110@montanaro.dyndns.org> Message-ID: <43C57DB8.7060702@v.loewis.de> skip at pobox.com wrote: > One concern might be performance. All buildbot slaves are contributed > hardware. I don't mind the load on my Mac (it's a dual processor g5), but > it may be an issue for other people. Does/can buildbot run at a lower > priority or is it just a matter of nice-ing the slave process at startup? nice-ing the slave process at startup would be the way to do it, right? Regards, Martin From martin at v.loewis.de Wed Jan 11 22:53:20 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 11 Jan 2006 22:53:20 +0100 Subject: [Python-Dev] Buildbot: doing occasional full builds In-Reply-To: <20060111204533.GB30010@activestate.com> References: <20060106173110.GD15276@activestate.com> <43BEE8F4.5080304@v.loewis.de> <20060107001221.GA13845@activestate.com> <20060111.003609.21623845.warner@lothar.com> <43C56563.9060107@v.loewis.de> <20060111204533.GB30010@activestate.com> Message-ID: <43C57E50.2090700@v.loewis.de> Trent Mick wrote: > Specifically are you concerned about the readability of the waterfall > page or other things (maintainability or something)? If the former, > perhaps we could get some mileage out of the query args that Brian > mentioned: Indeed - that would be a solution. However, I wonder how many people would look at the (say) nightly builds. Anyway, I'll try that. Regards, Martin From martin at v.loewis.de Wed Jan 11 22:59:16 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 11 Jan 2006 22:59:16 +0100 Subject: [Python-Dev] New PEP: Using ssize_t as the index type In-Reply-To: <1f7befae0601101349l57e3ae35o65913fecc7112710@mail.gmail.com> References: <43B3ECEE.9060300@v.loewis.de> <20051229173041.GA19575@code1.codespeak.net> <43B50B64.20904@v.loewis.de> <20060105194538.GB11736@code1.codespeak.net> <43BD8464.4060801@v.loewis.de> <43C37C02.8030005@egenix.com> <43C418FB.7010007@v.loewis.de> <43C42524.7000802@egenix.com> <43C428A2.9000703@v.loewis.de> <1f7befae0601101349l57e3ae35o65913fecc7112710@mail.gmail.com> Message-ID: <43C57FB4.4050900@v.loewis.de> Tim Peters wrote: > This reminded me that I still owe you a reply about s# and t# format > codes. It occurred to me that I've never used them, and probably > never will, so I really don't care how they work: I'm only really > worried about widespread ugliness, meaning wide enough that it touches > me ;-) Ok, given that Armin also "gave in", I think I keep that at a macro. Regards, Martin From martin at v.loewis.de Wed Jan 11 23:09:04 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 11 Jan 2006 23:09:04 +0100 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: References: Message-ID: <43C58200.20803@v.loewis.de> Guido van Rossum wrote: > On the other hand it breaks one of the most fundamental Python > guidelines: if you get a core dump (segfault etc.) it's a bug in > Python or in a 3rd party extension, not in *your* Python code. An > exception would have to be made for any code that uses ctypes, as it > is usually trivial to cause core dumps with ctypes (I'd venture it's > hard to avoid them ;-). > > I don't expect this to count against including ctypes; but I do want > it to be dealt with somehow! I think the strongest point *for* ctypes is the inclusion of dl in the source distribution, which has the very same flaws as ctypes in terms of crashability. So as for dealing with it "somehow": I would make ctypes a dynamically loaded module (ctypes.pyd), so administrators could remove it, and I could also make it a separate option in the Windows installer, so administrators could reject to install it. Regards, Martin From warner at lothar.com Wed Jan 11 23:10:18 2006 From: warner at lothar.com (Brian Warner) Date: Wed, 11 Jan 2006 14:10:18 -0800 (PST) Subject: [Python-Dev] Buildbot: doing occasional full builds In-Reply-To: <43C57DB8.7060702@v.loewis.de> References: <20060111204533.GB30010@activestate.com> <17349.28766.364593.662110@montanaro.dyndns.org> <43C57DB8.7060702@v.loewis.de> Message-ID: <20060111.141018.74733221.warner@lothar.com> > nice-ing the slave process at startup would be the way to do it, right? Yup. We run the twisted buildslaves under 'nice', and it works pretty well. It also reveals problems in tests that use absolute timeouts which fail when the test runs slower than the author thought it was supposed to. cheers, -Brian From trentm at ActiveState.com Wed Jan 11 23:30:39 2006 From: trentm at ActiveState.com (Trent Mick) Date: Wed, 11 Jan 2006 14:30:39 -0800 Subject: [Python-Dev] building a module catalogue with buildbot In-Reply-To: References: Message-ID: <20060111223039.GC6642@activestate.com> > > Does that make sense? We would just need /f's script in SVN. > > in python/Tools/something or sandbox/something ? python/Doc/tools/something? Trent -- Trent Mick trentm at activestate.com From tdelaney at avaya.com Thu Jan 12 00:08:25 2006 From: tdelaney at avaya.com (Delaney, Timothy (Tim)) Date: Thu, 12 Jan 2006 10:08:25 +1100 Subject: [Python-Dev] Include ctypes into core Python? Message-ID: <2773CAC687FD5F4689F526998C7E4E5F4DB8B3@au3010avexu1.global.avaya.com> "Martin v. L?wis" wrote: > So as for dealing with it "somehow": I would make ctypes a dynamically > loaded module (ctypes.pyd), so administrators could remove it, and > I could also make it a separate option in the Windows installer, > so administrators could reject to install it. I like this solution. Of course, Thomas (author of both py2exe and ctypes) may like the ability to have ctypes built into python.dll ... Tim Delaney From facundobatista at gmail.com Thu Jan 12 00:14:04 2006 From: facundobatista at gmail.com (Facundo Batista) Date: Wed, 11 Jan 2006 20:14:04 -0300 Subject: [Python-Dev] ConfigParser to save with order In-Reply-To: <089A37AE-E802-4903-98A5-33BBA0D6E635@ihug.co.nz> References: <089A37AE-E802-4903-98A5-33BBA0D6E635@ihug.co.nz> Message-ID: 2006/1/11, Tony Meyer : > Remember that there has been a lot of discussion about how > ConfigParser should work in the past; for example (ignoring c.l.p): > > > > My god. Read it all (which took me near a thermo flask (or whatever you call to an insulated bottle to keep liquids cold or hot) of "mate"!). I see two paths here: - Good one - Bad one No, sorry, seriously: - Rewrite ConfigParser entirely. - Apply my patch. The first option is the better, though it's the one which will take more work. Someone needs to make a PEP. Michael Chermside, in the first mail of the first link that you sent, was working in a solution, but don't know what is done. In this case, both patchs I talked before should be rejected. The second one is the quickest. Both paths solves my problem. I think we should go with the second option in the short term, and with the first one in the long term. What do you think? . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From crutcher at gmail.com Thu Jan 12 01:03:11 2006 From: crutcher at gmail.com (Crutcher Dunnavant) Date: Wed, 11 Jan 2006 16:03:11 -0800 Subject: [Python-Dev] [PATCH] Fix dictionary subclass semantics when used as global dictionaries In-Reply-To: References: <20060111000844.GA11919@panix.com> Message-ID: Is there any objection to this patch? Any support? On 1/10/06, Crutcher Dunnavant wrote: > 1402289 > > On 1/10/06, Aahz wrote: > > On Tue, Jan 10, 2006, Crutcher Dunnavant wrote: > > > > > > There is an inconsistancy in the way that dictionary subclasses behave > > > when they are used as as namespaces in execs. > > > > > > Basically, while python 2.4 permits the usage of dictionary subclasses > > > for local environments, it still bypasses the subclass functions and > > > uses the C API for global environments. The attached patch (and > > > unittest!) addresses this issue. > > > > Please submit the patch to SourceForge and report back with the SF ID. > > -- > > Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ > > > > "19. A language that doesn't affect the way you think about programming, > > is not worth knowing." --Alan Perlis > > > > > -- > Crutcher Dunnavant > monket.samedi-studios.com > -- Crutcher Dunnavant monket.samedi-studios.com From warner at lothar.com Thu Jan 12 01:04:27 2006 From: warner at lothar.com (Brian Warner) Date: Wed, 11 Jan 2006 16:04:27 -0800 (PST) Subject: [Python-Dev] building a module catalogue with buildbot Message-ID: <20060111.160427.07674399.warner@lothar.com> > PS If there was a method on ShellCommand, adding a step would be simpler, > e.g., > > def addStep(self, cls, **kwds): > self.steps.append((cls, kwds)) Ooh! I like that. > then we could do: f.addStep(Catalog, command=Catalog.command) It would be even simpler: f.addStep(Catalog). The command= argument is provided by the class-level attributes of your Catalog class (it can be set by either an attribute or by a constructor argument, but if it's always going to be the same thing then the attribute is easier). If you make a patch for this, I'll commit it. If not, I'll write one before I get the next release out. thanks, -Brian From goodger at python.org Thu Jan 12 04:51:19 2006 From: goodger at python.org (David Goodger) Date: Wed, 11 Jan 2006 22:51:19 -0500 Subject: [Python-Dev] r42015 - peps/trunk In-Reply-To: <20060112033316.E24C31E4002@bag.python.org> References: <20060112033316.E24C31E4002@bag.python.org> Message-ID: <43C5D237.3040006@python.org> > Author: david.goodger > Date: Thu Jan 12 04:33:16 2006 > New Revision: 42015 > > Modified: > peps/trunk/ (props changed) > Log: > add external link to Docutils public repo -- always up-to-date I just deleted the static copy of the "docutils" directory from the "peps" repository, and added in an external link (svn:externals 'docutils svn://svn.berlios.de/docutils/trunk/docutils/docutils'). This way, the external code should always be up-to-date. You may need to manually delete your peps/trunk/docutils directory to get this to work though -- SVN leaves subdirectories behind which hinder the externals update. Please let me know if this causes any problems. Thanks. -- David Goodger -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 249 bytes Desc: OpenPGP digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20060111/43d05d63/attachment.pgp From exarkun at divmod.com Thu Jan 12 06:35:11 2006 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Thu, 12 Jan 2006 00:35:11 -0500 Subject: [Python-Dev] [Buildbot-devel] Re: buildbot In-Reply-To: Message-ID: <20060112053511.26200.797800731.divmod.quotient.127@ohm> On Tue, 10 Jan 2006 09:18:59 -0800, Stephen Davis wrote: >> The reason I want static pages is for security concerns. It is not >> easy whether buildbot can be trusted to have no security flaws, >> which might allow people to start new processes on the master, >> or (perhaps worse) on any of the slaves. > >I have security concerns as well, but not in buildbot itself. My >project is restricted even withinz the company I work for so I need >the buildbot web server to only be available to certain people. >HTTPS access would be nice too. TwistedWeb doesn't seem to have >support for either HTTPS or authentication so I've been forced to >"hide" it by putting it on a non-standard port. Very weak. Inexcusably weak. Fortunately not exactly accurate, though: Twisted's web server supports both HTTPS and authentication. I won't bore the list with the details, but you can read more here: and Jean-Paul From martin at v.loewis.de Thu Jan 12 07:19:08 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 12 Jan 2006 07:19:08 +0100 Subject: [Python-Dev] building a module catalogue with buildbot In-Reply-To: References: Message-ID: <43C5F4DC.4010604@v.loewis.de> Fredrik Lundh wrote: > My initial thought was that we could ask alpha testers to run this script on > their alpha builds, and report back, but it just struck me that the "buildbot" > already builds stuff on a couple of interesting platforms. > > Can buildbot deal with custom test/validation scripts, and collect the output > somewhere ? Collecting the output is the real challenge. Basically, you are restricted to putting stdout of a process into a file. So a result that is a set of linked HTML files is not supported. Regards, Martin From andrew-pythondev at puzzling.org Thu Jan 12 07:42:20 2006 From: andrew-pythondev at puzzling.org (Andrew Bennetts) Date: Thu, 12 Jan 2006 17:42:20 +1100 Subject: [Python-Dev] building a module catalogue with buildbot In-Reply-To: <43C5F4DC.4010604@v.loewis.de> References: <43C5F4DC.4010604@v.loewis.de> Message-ID: <20060112064220.GE4555@home.puzzling.org> On Thu, Jan 12, 2006 at 07:19:08AM +0100, "Martin v. L?wis" wrote: > Fredrik Lundh wrote: > > My initial thought was that we could ask alpha testers to run this script on > > their alpha builds, and report back, but it just struck me that the "buildbot" > > already builds stuff on a couple of interesting platforms. > > > > Can buildbot deal with custom test/validation scripts, and collect the output > > somewhere ? > > Collecting the output is the real challenge. Basically, you are > restricted to putting stdout of a process into a file. So a result > that is a set of linked HTML files is not supported. A limited solution is just to make the script put the files where they will be published by something other than buildbot. e.g. Twisted's docs are built from SVN by one of our buildslaves, and placed in a directory published at http://twistedmatrix.com/users/warner/doc-latest/ -Andrew. From martin at v.loewis.de Thu Jan 12 07:58:00 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 12 Jan 2006 07:58:00 +0100 Subject: [Python-Dev] building a module catalogue with buildbot In-Reply-To: <20060112064220.GE4555@home.puzzling.org> References: <43C5F4DC.4010604@v.loewis.de> <20060112064220.GE4555@home.puzzling.org> Message-ID: <43C5FDF8.9010002@v.loewis.de> Andrew Bennetts wrote: > A limited solution is just to make the script put the files where they will be > published by something other than buildbot. e.g. Twisted's docs are built from > SVN by one of our buildslaves, and placed in a directory published at > http://twistedmatrix.com/users/warner/doc-latest/ Right. Whether this could work for us depends on how the files could be transferred. I would not want to see rsync, ftp, scp or some such to put files on python.org, but I could imagine getting them "through" the twisted connection, if that were possible somehow (i.e. with the master itself putting the files into the places where I want them). Regards, Martin From raymond.hettinger at verizon.net Thu Jan 12 10:01:37 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Thu, 12 Jan 2006 04:01:37 -0500 Subject: [Python-Dev] [PATCH] Fix dictionary subclass semantics whenused as global dictionaries In-Reply-To: Message-ID: <001201c61756$cc2d8780$5916c797@oemcomputer> > Is there any objection to this patch? Any support? It is assigned to me. When I have time, will go through it in detail (the given use cases, more detailed timings, and a close reading of the code). If accepted, it will be for Py2.5, as it is a new feature. There is nothing broken about the existing eval() version, it just doesn't apply as broadly as you would have liked. Raymond From mal at egenix.com Thu Jan 12 11:50:00 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 12 Jan 2006 11:50:00 +0100 Subject: [Python-Dev] Py_ssize_t output parameters (Was: [Python-checkins] r41971 ...) In-Reply-To: <43C425DE.2050001@v.loewis.de> References: <20060108054820.15F321E4002@bag.python.org> <43C25CE1.4070106@egenix.com> <43C2DD44.2040908@v.loewis.de> <43C37C92.3060302@egenix.com> <43C4083F.3070001@v.loewis.de> <43C421A3.2030806@egenix.com> <43C425DE.2050001@v.loewis.de> Message-ID: <43C63458.10804@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg wrote: >> ... and then the type change of that variable propagates >> throughout the extension. > > That depends on the usage of the code. If the variable > is passed by value, no further changes are necessary. > If a pointer to the variable is passed, you could replace > it with > > Py_ssize_t x64; int x; > foo(&x64); > x = x64; > > Then use x as you did with the original code. What if x64 has a 64-bit value ? How do you catch and process the truncation error ? To do this down-casting correctly, you'd need to write code that does bounds checks and integrate into the functions error handling. >> You basically end up having to convert the whole extension >> to Py_ssize_t. > > That is not necessary. Can you give in example of a module > where you think it is necessary? If you want to port the extension to Py_ssize_t, this is essentially the case. You might want to take the unicodeobject.c file as example. >> Don't get me wrong: I don't mind doing this for the eGenix >> extensions, but I'm worried about the gazillion other >> useful extensions out there which probably won't get >> upgraded in time to be used with Python 2.5. > > I agree that it is not acceptable to require immediate > whole-sale updates of every modules. However, I do > believe that the number of modules that need any change > at all is small, and that those modules can be modified > with minimal effort to "get them working again, > backwards-compatible" (i.e. with the only exception that > they would fail if indices run above 2**31). > >> I think all it takes is a set of new APIs for functions >> that use Py_ssize_t as output parameter and which are >> mapped to the regular API names if and only if the >> extension #defines PY_SSIZE_T_CLEAN (or some other >> capability flag). > > That is not enough. You also need to deal with the > function pointers that change. We could use the type flags for these. much like we do for the new style numbers. > Also, others have rejected/expressed dislike of the > PY_SIZE_T_CLEAN macro already, so they would probably > dislike further hackishness in that direction. That's easy to say... by not providing an easy way to upgrade extensions, you basically just move the requirement for hacks into the extensions. I wouldn't consider that a good solution. If you don't like the macro approach, why not simply leave the two separate sets of APIs visible. Old extensions would then use and link against the existing APIs. All Py_ssize_t aware and compatible extensions would use the new APIs instead. The old-style APIs should then take care of the proper down-casting and error reporting. In Py3K we could then remove the old-style ones and rename the new ones to the old names. Porting an already Py_ssize_t compatible extension to the renamed new style APIs would then become a simple task of search&replace. > Anyway, I have installed the PEP onsite, and > added an "Open Issues" section, recording your > comments. Thanks. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 12 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Thu Jan 12 12:12:09 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 12 Jan 2006 12:12:09 +0100 Subject: [Python-Dev] New PEP: Using ssize_t as the index type In-Reply-To: <43C428A2.9000703@v.loewis.de> References: <43B3ECEE.9060300@v.loewis.de> <20051229173041.GA19575@code1.codespeak.net> <43B50B64.20904@v.loewis.de> <20060105194538.GB11736@code1.codespeak.net> <43BD8464.4060801@v.loewis.de> <43C37C02.8030005@egenix.com> <43C418FB.7010007@v.loewis.de> <43C42524.7000802@egenix.com> <43C428A2.9000703@v.loewis.de> Message-ID: <43C63989.6070500@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg wrote: >> If it were this easy, I wouldn't have objections. But it's >> not. > > It is so easy. Can you should me an example where it isn't? > >> The variables you use with these APIs tend to propagate >> through the extension, you use them in other calls, >> make assignments, etc. > > They only propage if you make them. You don't have to, > if you don't want to. > >> If you implement extension types, you end up having to >> convert all the length related struct variables to >> Py_ssize_t. > > Only if you want to. If not, the module will work > (nearly) unmodified. Of course, it will be limited > to 32-bit indices. See my other reply on this topic. >> All this is fine, but it's also a lot of work which >> can be made easier. Recompiling an extension is well >> within range of many Python users, manually checking, >> fixing and porting it to the new API is certainly not. > > Sure. However, most users will compile it on 32-bit > systems. If they find they cannot get it to work on > a 64-bit system, they should ask the author for help, > or just use it in 32-bit mode (as 64-bit mode won't > gain them anything, anyway). I wonder how you are going to import a 32-bit extension into a 64-bit binary of Python. It simply doesn't work. >> The set of functions that will require Py_ssize_t >> is getting larger in your branch which is why I started >> this discussion. > > How so? I did not add a single function that has > int* output values, AFAICT. No, but there are quite a few APIs with Py_ssize_t* output values. > I am talking about the entirety of these functions, > and claim that they are rarely used (including the > Unicode and buffer APIs). I wouldn't say that PyString_AsStringAndSize() is rarely used and neither is PyArg_ParseTuple(). I agree that other APIs are certainly more domain specific and can be dealt with in the extension, but those two APIs need special care and so do the type slot functions. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 12 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Thu Jan 12 12:27:46 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 12 Jan 2006 12:27:46 +0100 Subject: [Python-Dev] r42015 - peps/trunk In-Reply-To: <43C5D237.3040006@python.org> References: <20060112033316.E24C31E4002@bag.python.org> <43C5D237.3040006@python.org> Message-ID: <43C63D32.9070206@egenix.com> David Goodger wrote: >> Author: david.goodger >> Date: Thu Jan 12 04:33:16 2006 >> New Revision: 42015 >> >> Modified: >> peps/trunk/ (props changed) >> Log: >> add external link to Docutils public repo -- always up-to-date > > I just deleted the static copy of the "docutils" directory from the "peps" > repository, and added in an external link (svn:externals 'docutils > svn://svn.berlios.de/docutils/trunk/docutils/docutils'). This way, the external > code should always be up-to-date. You may need to manually delete your > peps/trunk/docutils directory to get this to work though -- SVN leaves > subdirectories behind which hinder the externals update. > > Please let me know if this causes any problems. Thanks. Question: why do we need docutils in the peps/trunk/ directory in the first place ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 12 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From goodger at python.org Thu Jan 12 14:31:25 2006 From: goodger at python.org (David Goodger) Date: Thu, 12 Jan 2006 08:31:25 -0500 Subject: [Python-Dev] r42015 - peps/trunk In-Reply-To: <43C63D32.9070206@egenix.com> References: <20060112033316.E24C31E4002@bag.python.org> <43C5D237.3040006@python.org> <43C63D32.9070206@egenix.com> Message-ID: <43C65A2D.2050807@python.org> [M.-A. Lemburg] > Question: why do we need docutils in the peps/trunk/ directory > in the first place ? It's a convenience, so that a separate Docutils download & install (& maintain) isn't necessary for those who process reST-format PEPs. -- David Goodger -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 249 bytes Desc: OpenPGP digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20060112/1d6be3a8/attachment.pgp From mal at egenix.com Thu Jan 12 16:37:49 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 12 Jan 2006 16:37:49 +0100 Subject: [Python-Dev] r42015 - peps/trunk In-Reply-To: <43C65A2D.2050807@python.org> References: <20060112033316.E24C31E4002@bag.python.org> <43C5D237.3040006@python.org> <43C63D32.9070206@egenix.com> <43C65A2D.2050807@python.org> Message-ID: <43C677CD.3030709@egenix.com> David Goodger wrote: > [M.-A. Lemburg] >> Question: why do we need docutils in the peps/trunk/ directory >> in the first place ? > > It's a convenience, so that a separate Docutils download & install > (& maintain) isn't necessary for those who process reST-format PEPs. Hmm, shouldn't these things be tracked under external/ ?! -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 12 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From goodger at python.org Thu Jan 12 16:56:18 2006 From: goodger at python.org (David Goodger) Date: Thu, 12 Jan 2006 10:56:18 -0500 Subject: [Python-Dev] r42015 - peps/trunk In-Reply-To: <43C677CD.3030709@egenix.com> References: <20060112033316.E24C31E4002@bag.python.org> <43C5D237.3040006@python.org> <43C63D32.9070206@egenix.com> <43C65A2D.2050807@python.org> <43C677CD.3030709@egenix.com> Message-ID: <4335d2c40601120756l34c4a16fvf4b1d1b7b376d939@mail.gmail.com> On 1/12/06, M.-A. Lemburg wrote: > Hmm, shouldn't these things be tracked under external/ ?! What do you mean exactly? A new "external" directory? SVN provides a built-in mechanism for tracking external repositories, via the "svn:externals" property, and that's what I used. -- David Goodger From mal at egenix.com Thu Jan 12 17:15:26 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 12 Jan 2006 17:15:26 +0100 Subject: [Python-Dev] r42015 - peps/trunk In-Reply-To: <4335d2c40601120756l34c4a16fvf4b1d1b7b376d939@mail.gmail.com> References: <20060112033316.E24C31E4002@bag.python.org> <43C5D237.3040006@python.org> <43C63D32.9070206@egenix.com> <43C65A2D.2050807@python.org> <43C677CD.3030709@egenix.com> <4335d2c40601120756l34c4a16fvf4b1d1b7b376d939@mail.gmail.com> Message-ID: <43C6809E.306@egenix.com> David Goodger wrote: > On 1/12/06, M.-A. Lemburg wrote: >> Hmm, shouldn't these things be tracked under external/ ?! > > What do you mean exactly? A new "external" directory? Yes. > SVN provides a built-in mechanism for tracking external > repositories, via the "svn:externals" property, and that's > what I used. I know, but I wouldn't expect SVN to query other servers than svn.python.org inside the standard repository directories. AFAIK, this is a first in the Python repository. Not sure if it's such a good idea. Branching and tagging doesn't work with external resources in Subversion, so things can become inconsistent. Also, what if you break the code in the berlios repo or the server is not reachable ? A release copy in the external/ dir would solve all these issues. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 12 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From fredrik at pythonware.com Thu Jan 12 17:53:13 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 12 Jan 2006 17:53:13 +0100 Subject: [Python-Dev] r42015 - peps/trunk References: <20060112033316.E24C31E4002@bag.python.org> <43C5D237.3040006@python.org> <43C63D32.9070206@egenix.com> <43C65A2D.2050807@python.org> <43C677CD.3030709@egenix.com><4335d2c40601120756l34c4a16fvf4b1d1b7b376d939@mail.gmail.com> <43C6809E.306@egenix.com> Message-ID: M.-A. Lemburg wrote: > AFAIK, this is a first in the Python repository. Not > sure if it's such a good idea. Branching and tagging > doesn't work with external resources in Subversion, > so things can become inconsistent. > > Also, what if you break the code in the berlios repo > or the server is not reachable ? A release copy in the > external/ dir would solve all these issues. using svn:external is usually a lousy idea. using it in a public repository is a really, really, really lousy idea. From theller at python.net Thu Jan 12 17:55:34 2006 From: theller at python.net (Thomas Heller) Date: Thu, 12 Jan 2006 17:55:34 +0100 Subject: [Python-Dev] Include ctypes into core Python? References: <2773CAC687FD5F4689F526998C7E4E5F4DB8B3@au3010avexu1.global.avaya.com> Message-ID: "Delaney, Timothy (Tim)" writes: > "Martin v. L?wis" wrote: > >> So as for dealing with it "somehow": I would make ctypes a dynamically >> loaded module (ctypes.pyd), so administrators could remove it, and >> I could also make it a separate option in the Windows installer, >> so administrators could reject to install it. > > I like this solution. Of course, Thomas (author of both py2exe and > ctypes) may like the ability to have ctypes built into python.dll ... It is getting offtopic, but I don't care too much about that. I requested that zlib be changed to a builtin module too allow easier bootstrapping of py2exe'd apps which have a compressed library archive. The nearly only reason for me to implement the single-file option for py2exe was that the implementation simulates a static linked Python-dll, which allows for several totally-isolated interpreters in the same process. Thomas From goodger at python.org Thu Jan 12 18:04:51 2006 From: goodger at python.org (David Goodger) Date: Thu, 12 Jan 2006 12:04:51 -0500 Subject: [Python-Dev] r42015 - peps/trunk In-Reply-To: <43C6809E.306@egenix.com> References: <20060112033316.E24C31E4002@bag.python.org> <43C5D237.3040006@python.org> <43C63D32.9070206@egenix.com> <43C65A2D.2050807@python.org> <43C677CD.3030709@egenix.com> <4335d2c40601120756l34c4a16fvf4b1d1b7b376d939@mail.gmail.com> <43C6809E.306@egenix.com> Message-ID: <4335d2c40601120904rd45400cy2c09bcc9d3d66197@mail.gmail.com> On 1/12/06, M.-A. Lemburg wrote: > I know, but I wouldn't expect SVN to query other servers > than svn.python.org inside the standard repository > directories. > > AFAIK, this is a first in the Python repository. True, and worth discussing. Luckily the PEPs repository is a low-traffic, non-core area, so it's not urgent. > Not sure if it's such a good idea. >From my point of view, it's a time-saver. No more manual updates! They were a pain, and rarely got done. > Branching and tagging > doesn't work with external resources in Subversion, > so things can become inconsistent. How so? The "svn:externals" property is treated the same as any other, and is copied along with everything else by "svn copy". > Also, what if you break the code in the berlios repo > or the server is not reachable ? If the code in the repo ever breaks, it will be fixed within minutes. The server being unreachable is only a problem for initial checkouts; updates will just keep the code that was already there. In my experience, the berlios.de SVN server has rarely been unreachable. If there's a problem, we can always back out the svn:externals property and install the package. That having been said, I do see the value of installing a packaged release. We just released Docutils 0.4 a few days ago; I could install that statically. An alternative is to use svn:externals to link to a specific revision (via the -r option); I could link to the 0.4 release revision. This would solve the repo-breakage issue, but not the server-unreachability issue. I don't think these issues are major, but I wouldn't mind terribly if we decide to go with a static install. > A release copy in the > external/ dir would solve all these issues. That's basically what we had before (except no explicity "external" directory), but it was always out of date. The "docutils" package was directly in the trunk, for ease of import by the pep2html.py front end (easy to work around, but why bother?). Another minor nit with the old way: updates "polluted" the python-checkins list. -- David Goodger From theller at python.net Thu Jan 12 17:58:15 2006 From: theller at python.net (Thomas Heller) Date: Thu, 12 Jan 2006 17:58:15 +0100 Subject: [Python-Dev] Include ctypes into core Python? References: <43C58200.20803@v.loewis.de> Message-ID: "Martin v. L?wis" writes: > Guido van Rossum wrote: >> On the other hand it breaks one of the most fundamental Python >> guidelines: if you get a core dump (segfault etc.) it's a bug in >> Python or in a 3rd party extension, not in *your* Python code. An >> exception would have to be made for any code that uses ctypes, as it >> is usually trivial to cause core dumps with ctypes (I'd venture it's >> hard to avoid them ;-). >> >> I don't expect this to count against including ctypes; but I do want >> it to be dealt with somehow! > > I think the strongest point *for* ctypes is the inclusion of dl > in the source distribution, which has the very same flaws as > ctypes in terms of crashability. > > So as for dealing with it "somehow": I would make ctypes a dynamically > loaded module (ctypes.pyd), so administrators could remove it, and > I could also make it a separate option in the Windows installer, > so administrators could reject to install it. Sounds good, although it should be noted that ctypes is a package now, with a ctypes.wrap subpackage (contains the code generator), a ctypes.macholib subpackage (contains Bob Ippolitos macholib for OS X), and ctypes.test subpackage whcih contains several test modules. Plus the _ctypes.(dll|so) extension module. Thomas From fredrik at pythonware.com Thu Jan 12 18:08:57 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 12 Jan 2006 18:08:57 +0100 Subject: [Python-Dev] building a module catalogue with buildbot References: <43C5F4DC.4010604@v.loewis.de> Message-ID: Martin v. Löwis wrote: > > My initial thought was that we could ask alpha testers to run this script on > > their alpha builds, and report back, but it just struck me that the "buildbot" > > already builds stuff on a couple of interesting platforms. > > > > Can buildbot deal with custom test/validation scripts, and collect the output > > somewhere ? > > Collecting the output is the real challenge. Basically, you are > restricted to putting stdout of a process into a file. So a result > that is a set of linked HTML files is not supported. The listmodule utility only outputs a plain list of available modules. Just being to able to scrape the lists off http://www.python.org/dev/buildbot/ would be good enough. From skink at evhr.net Thu Jan 12 18:29:16 2006 From: skink at evhr.net (Fabien Schwob) Date: Thu, 12 Jan 2006 18:29:16 +0100 Subject: [Python-Dev] PEP and stdlib Message-ID: <43C691EC.3000503@evhr.net> Hello, I've often read new PEP that are published, and they are often about new additions to the core language. Why not using them with the standard library. Guido often say that he don't want to include new module that aren't widely used in the community. It's a good thing, but it lead to the creation of a lot of API incompatible modules. Why not using PEP in order to create standard API like the Java world do with JSRs (Java Specification Requests) ? What do you think about that ? -- Fabien From mal at egenix.com Thu Jan 12 20:34:54 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 12 Jan 2006 20:34:54 +0100 Subject: [Python-Dev] r42015 - peps/trunk In-Reply-To: <4335d2c40601120904rd45400cy2c09bcc9d3d66197@mail.gmail.com> References: <20060112033316.E24C31E4002@bag.python.org> <43C5D237.3040006@python.org> <43C63D32.9070206@egenix.com> <43C65A2D.2050807@python.org> <43C677CD.3030709@egenix.com> <4335d2c40601120756l34c4a16fvf4b1d1b7b376d939@mail.gmail.com> <43C6809E.306@egenix.com> <4335d2c40601120904rd45400cy2c09bcc9d3d66197@mail.gmail.com> Message-ID: <43C6AF5E.7020803@egenix.com> David Goodger wrote: > On 1/12/06, M.-A. Lemburg wrote: >> I know, but I wouldn't expect SVN to query other servers >> than svn.python.org inside the standard repository >> directories. >> >> AFAIK, this is a first in the Python repository. > > True, and worth discussing. Luckily the PEPs repository is a > low-traffic, non-core area, so it's not urgent. True. >> Not sure if it's such a good idea. > >>From my point of view, it's a time-saver. No more manual updates! > They were a pain, and rarely got done. > >> Branching and tagging >> doesn't work with external resources in Subversion, >> so things can become inconsistent. > > How so? The "svn:externals" property is treated the same as any > other, and is copied along with everything else by "svn copy". Right, but Subversion doesn't store the revision of the external resource at the time you made the copy, so when you checkout the branch or tag, you'll still get the most recent version of the external resource which can break things, e.g. say you upgrade docutils to Python 2.5, then a checkout of a tag built at a time when Python 2.3 was current would probably no longer work (with Python 2.3). At least that's how things were documented the last time I had a look at svn:externals - could be that they modified the behavior to make it a little more clever since. >> Also, what if you break the code in the berlios repo >> or the server is not reachable ? > > If the code in the repo ever breaks, it will be fixed within minutes. > The server being unreachable is only a problem for initial checkouts; > updates will just keep the code that was already there. In my > experience, the berlios.de SVN server has rarely been unreachable. If > there's a problem, we can always back out the svn:externals property > and install the package. > > That having been said, I do see the value of installing a packaged > release. We just released Docutils 0.4 a few days ago; I could > install that statically. An alternative is to use svn:externals to > link to a specific revision (via the -r option); I could link to the > 0.4 release revision. This would solve the repo-breakage issue, but > not the server-unreachability issue. Interesting. Last time I looked these external revisions were not possible (or I overread the possibility). > I don't think these issues are major, but I wouldn't mind terribly if > we decide to go with a static install. There are two other nits with the external reference: * connecting to a server that people probably don't recognize as being an official Python source and thus probably don't trust right away (though this is well hidden by subversion, firewall software will alarm the user) * being able to download the complete repository of Python with all the history - since the external resource is not part of the history, users would have to track down the repository of the external resource (and this might not be readily available for download) >> A release copy in the >> external/ dir would solve all these issues. > > That's basically what we had before (except no explicity "external" > directory), but it was always out of date. The "docutils" package was > directly in the trunk, for ease of import by the pep2html.py front end > (easy to work around, but why bother?). I guess that only PSF license covered software should go into the main directories of the repository. Everything else should first go in externals/ and then get copied into the main trunks. It's only one additional step, but will help a lot in the future if we ever have to track the copyright status of things in the main trunks. This is not an issue for docutils, I guess, since it's public domain, but then again, there are lawyers who believe that there's no such thing as public domain... http://www.linuxjournal.com/article/6225 > Another minor nit with the old way: updates "polluted" the > python-checkins list. I guess if you just update to new release versions, then this will be minimal. I, for one, don't mind at all. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 12 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From tim.peters at gmail.com Thu Jan 12 20:41:56 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 12 Jan 2006 14:41:56 -0500 Subject: [Python-Dev] r42015 - peps/trunk In-Reply-To: <43C6AF5E.7020803@egenix.com> References: <20060112033316.E24C31E4002@bag.python.org> <43C5D237.3040006@python.org> <43C63D32.9070206@egenix.com> <43C65A2D.2050807@python.org> <43C677CD.3030709@egenix.com> <4335d2c40601120756l34c4a16fvf4b1d1b7b376d939@mail.gmail.com> <43C6809E.306@egenix.com> <4335d2c40601120904rd45400cy2c09bcc9d3d66197@mail.gmail.com> <43C6AF5E.7020803@egenix.com> Message-ID: <1f7befae0601121141m313d836etb0ba912821844bdf@mail.gmail.com> [David Goodger] ... > An alternative is to use svn:externals to link to a specific > revision (via the -r option); Yes. please. svn:externals should always pin to a specific revision (or at least to a "frozen" tag). Updating to a new revision then is just a propedit away (to change the revision number); ditto for backing off to an older revision if the newer one has problems. As to the rest, you're using externals in a vanilla way for their intended purpose, and I have no problems with that. From fredrik at pythonware.com Thu Jan 12 21:13:05 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 12 Jan 2006 21:13:05 +0100 Subject: [Python-Dev] r42015 - peps/trunk References: <20060112033316.E24C31E4002@bag.python.org><43C5D237.3040006@python.org> <43C63D32.9070206@egenix.com><43C65A2D.2050807@python.org> <43C677CD.3030709@egenix.com><4335d2c40601120756l34c4a16fvf4b1d1b7b376d939@mail.gmail.com><43C6809E.306@egenix.com><4335d2c40601120904rd45400cy2c09bcc9d3d66197@mail.gmail.com><43C6AF5E.7020803@egenix.com> <1f7befae0601121141m313d836etb0ba912821844bdf@mail.gmail.com> Message-ID: Tim Peters wrote: > Yes. please. svn:externals should always pin to a specific revision > (or at least to a "frozen" tag). Updating to a new revision then is > just a propedit away (to change the revision number); ditto for > backing off to an older revision if the newer one has problems. wasn't the whole point for this exercise to make sure that the included version was always up to date. if you pin it to a given version, all we got from this was a dependence on another SVN server. (and a can of SVN worms, including issues with locking/cleanup, etc) From tim.peters at gmail.com Thu Jan 12 21:24:23 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 12 Jan 2006 15:24:23 -0500 Subject: [Python-Dev] r42015 - peps/trunk In-Reply-To: References: <20060112033316.E24C31E4002@bag.python.org> <43C63D32.9070206@egenix.com> <43C65A2D.2050807@python.org> <43C677CD.3030709@egenix.com> <4335d2c40601120756l34c4a16fvf4b1d1b7b376d939@mail.gmail.com> <43C6809E.306@egenix.com> <4335d2c40601120904rd45400cy2c09bcc9d3d66197@mail.gmail.com> <43C6AF5E.7020803@egenix.com> <1f7befae0601121141m313d836etb0ba912821844bdf@mail.gmail.com> Message-ID: <1f7befae0601121224g1e9efc6ew33bb0791d2d9c701@mail.gmail.com> [Tim Peters] >> Yes. please. svn:externals should always pin to a specific revision >> (or at least to a "frozen" tag). Updating to a new revision then is >> just a propedit away (to change the revision number); ditto for >> backing off to an older revision if the newer one has problems. [Fredrik Lundh] > wasn't the whole point for this exercise Sorry, I don't know what the whole point was. > to make sure that the included version was always up to date. if you pin > it to a given version, all we got from this was a dependence on another SVN > server. And very easy updating to a new version ("a propedit away ..."), and transparency about exactly what it is we're depending on ("revision 12345 of project XYZ"). > (and a can of SVN worms, including issues with locking/cleanup, etc) I haven't experienced any of that. I've experienced plenty of locking/cleanup problems when a project switches _between_ making its own copies and using svn:externals, but none so long as it sticks to one way of doing that. Even on Windows ;-) From crutcher at gmail.com Thu Jan 12 22:54:24 2006 From: crutcher at gmail.com (Crutcher Dunnavant) Date: Thu, 12 Jan 2006 13:54:24 -0800 Subject: [Python-Dev] [PATCH] Fix dictionary subclass semantics whenused as global dictionaries In-Reply-To: <001201c61756$cc2d8780$5916c797@oemcomputer> References: <001201c61756$cc2d8780$5916c797@oemcomputer> Message-ID: I sorta disagree about it not being broken. Adding a feature which works for eval but not for exec seems pretty broken. It's difficult to reason about what will happen in the exec context, so I can't see what fixing it would endanger; but I'd deffinately like to see it for 2.5. I've run rough timings on the code, ('test make time'), detailed in the patch discussion, and it seems completely lost in the noise. On 1/12/06, Raymond Hettinger wrote: > > Is there any objection to this patch? Any support? > > It is assigned to me. When I have time, will go through it in detail > (the given use cases, more detailed timings, and a close reading of the > code). > > If accepted, it will be for Py2.5, as it is a new feature. There is > nothing broken about the existing eval() version, it just doesn't apply > as broadly as you would have liked. > > > > Raymond > > -- Crutcher Dunnavant monket.samedi-studios.com From t-meyer at ihug.co.nz Thu Jan 12 22:57:03 2006 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Fri, 13 Jan 2006 10:57:03 +1300 Subject: [Python-Dev] ConfigParser to save with order In-Reply-To: References: <089A37AE-E802-4903-98A5-33BBA0D6E635@ihug.co.nz> Message-ID: <56E4F12D-9477-4F58-9D85-3D159B3F1B71@ihug.co.nz> > I see two paths here: > > - Rewrite ConfigParser entirely. > - Apply my patch. Allowing 'surgical' editing of configuration files, as has been proposed many times both here and c.l.p would not require ConfigParser to be entirely rewritten (just more extensive modification of the write() method). > Someone needs to make a PEP. Michael Chermside, in the > first mail of the first link that you sent, was working in a solution, > but don't know what is done. I believe that this always trails off without anything being achieved. IMO what would be best would be a pronouncement (or something of similar weight) about the direction that should be taken wrt to configuration file parsing support in the stdlib. * The status quo. * The status quo, but output files in a fixed order (but unrelated to the read order); i.e. http://python.org/sf/1399309 * 'Surgical' editing of files; i.e. preserve comments, whitespace, and ordering, appending new options/sections as needed. * A more complex configuration system (perhaps like others that exist in the wild). I'm happy to either submit or help with a patch for the third option (and maybe the fourth) if need be. I haven't personally found the lack of a more complex system in the stdlib a problem (although I suppose that the various more complex systems I have worked with could have been simplified if there was one). I do believe that failing to preserve comments and ordering is a problem. =Tony.Meyer From martin at v.loewis.de Thu Jan 12 23:36:01 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 12 Jan 2006 23:36:01 +0100 Subject: [Python-Dev] Py_ssize_t output parameters (Was: [Python-checkins] r41971 ...) In-Reply-To: <43C63458.10804@egenix.com> References: <20060108054820.15F321E4002@bag.python.org> <43C25CE1.4070106@egenix.com> <43C2DD44.2040908@v.loewis.de> <43C37C92.3060302@egenix.com> <43C4083F.3070001@v.loewis.de> <43C421A3.2030806@egenix.com> <43C425DE.2050001@v.loewis.de> <43C63458.10804@egenix.com> Message-ID: <43C6D9D1.7040101@v.loewis.de> M.-A. Lemburg wrote: > What if x64 has a 64-bit value ? How do you catch > and process the truncation error ? We were *both* discussing a scenario where no sizes exceed 2**31, right? Under such a scenario, this just won't happen. OTOH, if you were discussing a scenario where sizes might exceed 2**31, then I don't see why you are worried about Py_ssize_t* parameters alone: Even PyString_Size() might (of course!) return a value > 2**31 - so it is not just output parameters, but also return values. For more information, please read "Conversion guidelines" in http://www.python.org/peps/pep-0353.html >>That is not necessary. Can you give in example of a module >>where you think it is necessary? > > > If you want to port the extension to Py_ssize_t, this > is essentially the case. > > You might want to take the unicodeobject.c file as > example. unicodeobject.c is not an extension. We were discussing existing extension modules. > We could use the type flags for these. much like we > do for the new style numbers. Would you like to write a specification for that? > If you don't like the macro approach, why not simply > leave the two separate sets of APIs visible. To simplify the porting. > All Py_ssize_t aware and compatible extensions > would use the new APIs instead. The old-style APIs > should then take care of the proper down-casting and > error reporting. That is not possible. Some API does not support error reporting (like PyString_Size()). So callers don't expect it to fail. Regards, Martin From brett at python.org Thu Jan 12 23:36:51 2006 From: brett at python.org (Brett Cannon) Date: Thu, 12 Jan 2006 14:36:51 -0800 Subject: [Python-Dev] Ph.D. dissertation ideas? Message-ID: It is time once again in my educational career to come to python-dev for help for major project ideas. This time, though, it is for my Ph.D. dissertation (and thus can have larger scope than my masters thesis) but there are funding restrictions (and thus only certain areas I have possible funding for). First off, there are two areas where I have possible funding: XML integration into programming languages and game scripting support (I have a third, AOP for anything, but I don't think AOP is a good match for Python and thus not considering it for Python work). The XML integration I have no direct ideas since I don't use XML. There is the possibility of doing something like the LINQ project (http://msdn.microsoft.com/netframework/future/linq/default.aspx?pull=/library/en-us/dndotnet/html/linqprojectovw.asp) or something, but I don't know how useful that could be. It would have to be just as generic, though, as the LINQ project in terms of working with intefaces to be Pythonic so it might turn out to be more about language support for querying data. For the game scripting, trying out the active objects for Python is a possibility (or something else that would help with concurrency). Since that could allow for agent objects to be more concurrent it can be argued that it would help with game scripting. I don't have any other ideas for the area of game scripting support. Now I am not locked down to these areas, but they are what I have possible grant money for. If through some miracle someone out there wants to fund my Ph.D. and wants something else worked on for Python, don't be shy. =) Unfortunately I don't think the PSF is up for funding my Ph.D., else I would say I would work on whatever python-dev decided they wanted to worked on that could be done as a dissertation. Anyway, any ideas are appreciated. There is no rush on this, either. Just getting this out there now while I am tentatively exploring possible topics. -Brett From martin at v.loewis.de Thu Jan 12 23:44:27 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 12 Jan 2006 23:44:27 +0100 Subject: [Python-Dev] New PEP: Using ssize_t as the index type In-Reply-To: <43C63989.6070500@egenix.com> References: <43B3ECEE.9060300@v.loewis.de> <20051229173041.GA19575@code1.codespeak.net> <43B50B64.20904@v.loewis.de> <20060105194538.GB11736@code1.codespeak.net> <43BD8464.4060801@v.loewis.de> <43C37C02.8030005@egenix.com> <43C418FB.7010007@v.loewis.de> <43C42524.7000802@egenix.com> <43C428A2.9000703@v.loewis.de> <43C63989.6070500@egenix.com> Message-ID: <43C6DBCB.70305@v.loewis.de> M.-A. Lemburg wrote: >>Sure. However, most users will compile it on 32-bit >>systems. If they find they cannot get it to work on >>a 64-bit system, they should ask the author for help, >>or just use it in 32-bit mode (as 64-bit mode won't >>gain them anything, anyway). > > > I wonder how you are going to import a 32-bit > extension into a 64-bit binary of Python. > It simply doesn't work. No. They will need to run the 32-bit binary of Python on the 64-bit system. > I wouldn't say that PyString_AsStringAndSize() is rarely > used and neither is PyArg_ParseTuple(). I can't find much usage of PyString_AsStringAndSize. Even in the Python core, it is only called 8 times outside string and strop. > I agree that other APIs are certainly more domain > specific and can be dealt with in the extension, but > those two APIs need special care and so do the type > slot functions. For PyArg_ParseTuple, there is special care, because there won't be compiler warnings otherwise. For PyString_AsStringAndSize, the compiler will warn about incorrect pointer types. Regards, Martin From pje at telecommunity.com Fri Jan 13 00:00:58 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 12 Jan 2006 18:00:58 -0500 Subject: [Python-Dev] Ph.D. dissertation ideas? In-Reply-To: Message-ID: <5.1.1.6.0.20060112174613.01e4ce10@mail.telecommunity.com> At 02:36 PM 1/12/2006 -0800, Brett Cannon wrote: >(I have a third, AOP for anything, but I don't think AOP is a good match >for Python and thus not considering it for Python work) For what it's worth, I've been doing what I'll loosely refer to as "research" on implementing a non-traditional form of AOP in Python via the combination of extensible generic functions and contextual variables -- two concepts that already exist in Python but which can be extended far beyond what is in current use. The combination can be used to implement an "explicit is better than implicit" form of AOP, that is usually handled by a "weaving language" in classic AOP implementations. I don't know of any published AOP work that focuses on the use of these kinds of tools; the communities that talk about the respective technologies appear to be disjoint. Anyway, extensible generic operations in Python like 'len()', 'copy()' and so on are fairly common, but it's rare that people define their own such functions, as opposed to using ones in the stdlib. A typical workaround in larger frameworks is to use interfaces and adaptation, but extensible generic operations make it easier to separate concerns orthogonally. So, although you might think that AOP isn't "a good match for Python", the truth is that there's actually a fair amount of AOP-like things being done with Python. It's just that classic structuring of AOP into things like aspects and weavers and all that crud is overcomplication. If you view it simply in terms of extensible operations, which can also make decisions based on context, then you can achieve all the same goals of "classic" AOP without using any of its concepts or terminology. One definition of AOP claims that it allows asserting qualified statements over oblivious code. Python has this basic ability natively -- monkeypatching suffices to meet that definition. The interesting question is, how can you do it *without* monkeypatching? From martin at v.loewis.de Fri Jan 13 00:06:09 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 13 Jan 2006 00:06:09 +0100 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: References: <43C58200.20803@v.loewis.de> Message-ID: <43C6E0E1.5010705@v.loewis.de> Thomas Heller wrote: > Sounds good, although it should be noted that ctypes is a package now, > with a ctypes.wrap subpackage (contains the code generator), a > ctypes.macholib subpackage (contains Bob Ippolitos macholib for OS X), > and ctypes.test subpackage whcih contains several test modules. Plus > the _ctypes.(dll|so) extension module. Ok. The installer should then combine them all into a feature. Still, the admin could disable all of this just by removing _ctypes.dll, right? Regards, Martin From martin at v.loewis.de Fri Jan 13 00:12:31 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 13 Jan 2006 00:12:31 +0100 Subject: [Python-Dev] [PATCH] Fix dictionary subclass semantics whenused as global dictionaries In-Reply-To: References: <001201c61756$cc2d8780$5916c797@oemcomputer> Message-ID: <43C6E25F.2040606@v.loewis.de> Crutcher Dunnavant wrote: > I sorta disagree about it not being broken. Adding a feature which > works for eval but not for exec seems pretty broken. You "seem" to have a different notion of "to be broken", then. Python is broken, if and only if - the interpreter crashes, for any Python input - the implementation does not do what the documentation says it would do - the BDFL pronounces it is broken In this specific change, the change did precisely what the requestor of the feature wanted it to do (that eval could accept non-exact dicts was a new feature back then also) > It's difficult to > reason about what will happen in the exec context, so I can't see what > fixing it would endanger; but I'd deffinately like to see it for 2.5. It would make Python code run which is currently rejected with an exception. Therefore, it is a new feature (a behaviour change). Applications relying on that feature would have to specify that they require "2.4.3"; people would find that code that runs fine in 2.4.3 fails in 2.4.2. This is not acceptable. Regards, Martin From martin at v.loewis.de Fri Jan 13 00:14:24 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 13 Jan 2006 00:14:24 +0100 Subject: [Python-Dev] PEP and stdlib In-Reply-To: <43C691EC.3000503@evhr.net> References: <43C691EC.3000503@evhr.net> Message-ID: <43C6E2D0.9040407@v.loewis.de> Fabien Schwob wrote: > What do you think about that ? See PEP 2. Regards, Martin From Scott.Daniels at Acm.Org Fri Jan 13 01:27:29 2006 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Thu, 12 Jan 2006 16:27:29 -0800 Subject: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py In-Reply-To: <20060111195900.GB17269@activestate.com> References: <87zmm6d7jp.fsf@tleepslib.sk.tsukuba.ac.jp> <20060111195900.GB17269@activestate.com> Message-ID: OK I carried the code I offered earlier in this whole thread (tweaked in reaction to some comments) over to comp.lang.python, gathered some feedback, and put up a recipe on the cookbook. After a week or so for more comment, I'll be happy to submit a patch to include the broken_test decorator function in unittest. Here is where the recipe is, for those who want to comment further: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/466288 --Scott David Daniels Scott.Daniels at Acm.Org From brett at python.org Fri Jan 13 01:58:20 2006 From: brett at python.org (Brett Cannon) Date: Thu, 12 Jan 2006 16:58:20 -0800 Subject: [Python-Dev] Ph.D. dissertation ideas? In-Reply-To: References: Message-ID: On 1/12/06, Dennis Allison wrote: > > Brett, > > Where are you doing your Phd and who will be your likely supervisor? > It does make a difference. > University of British Columbia under Eric Wohlstadter in the Software Practices Lab. > Your dissertation idea list seems to me to focus on implementation > projects and not on research. Usually a dissertation proceeds from a > hypothesis leading to an experiment, some measurements, and conclusions. > Many universities tacitly expect a theorem and associated proof. The goal > of dissertation research is a completed approved dissertation, not some > intergalactic all encompassing contribution to human knowledge. > Dissertation research should, IMHO, focus on a small, manageable problem. > The topics are a little on the implementation side partially from my personal taste in projects; I have always had a major bent towards the practical over theoretical (reason I love Python so much =). Plus the lab I am in tends towards the practical and not into the theoretical (e.g., the number of Eclipse plug-ins being developed as part of theses and dissertations here). But I am open for theoretical work (as long as it does not involve type inference). > Frequently, universities have a large cooperative project involving many > graduate students, each of whom research a small, well defined topic areas > related to the larger project. Is this the case for your dissertation? > Nope. Nothing going on at the moment here in terms of a large scoped project with multiple parts to it. > What are your interests? Are you interested in language structure, > language implementation, the formal interface between languages and > applications, or what. > Tough question. Language design is the over-reaching area that I love. Basically how can powerful language ideas be presented in a easy-to-use fashion. Could even generalize even farther to just making programming easier through the language. And for some reason networking protocols and concurrent programming have always caught my eye (have not delved deep into these topics but I always catch myself doing stuff such as reading the ICMP standard or reading all of the various arguments for or against threading). > Like most folks who lurk on this list, I have my list of things I'd like > to see someone research. Tell us a bit more and I am sure you'll get lots > of us to share. > That's the hope. =) -Brett From janssen at parc.com Fri Jan 13 02:10:41 2006 From: janssen at parc.com (Bill Janssen) Date: Thu, 12 Jan 2006 17:10:41 PST Subject: [Python-Dev] Ph.D. dissertation ideas? In-Reply-To: Your message of "Thu, 12 Jan 2006 16:58:20 PST." Message-ID: <06Jan12.171050pst."58633"@synergy1.parc.xerox.com> Brett, How about building a system that compiles a Python program (possibly annotated) to an AJAX program? That is, it analyzes the program to determine what's appropriate and possible for client-side and server-side, figures out the optimal network API (reduce latency, reduce calls, reduce data transfer, increase responsiveness), generates server-side Jython to run in Tomcat (or Python for mod_python), and client-side XHTML, CSS, and Javascript to implement the browser-based portion of the application. Oddities of browser implementations and Javascript VMs would be contained in the compiler and automatically pushed into the generated code. This would be a tremendously useful advance in building distributed systems that work with the Web. The current approach to AJAX is a bit like carpentry with old hand tools. You'd probably also push Python to higher levels of language-ness. > Tough question. Language design is the over-reaching area that I > love. Basically how can powerful language ideas be presented in a > easy-to-use fashion. Could even generalize even farther to just > making programming easier through the language. And for some reason > networking protocols and concurrent programming have always caught my > eye Bill From brett at python.org Fri Jan 13 04:29:33 2006 From: brett at python.org (Brett Cannon) Date: Thu, 12 Jan 2006 19:29:33 -0800 Subject: [Python-Dev] Ph.D. dissertation ideas? In-Reply-To: <-8397309676539309595@unknownmsgid> References: <-8397309676539309595@unknownmsgid> Message-ID: On 1/12/06, Bill Janssen wrote: > Brett, > > How about building a system that compiles a Python program (possibly > annotated) to an AJAX program? That is, it analyzes the program to > determine what's appropriate and possible for client-side and > server-side, figures out the optimal network API (reduce latency, > reduce calls, reduce data transfer, increase responsiveness), > generates server-side Jython to run in Tomcat (or Python for > mod_python), and client-side XHTML, CSS, and Javascript to implement > the browser-based portion of the application. Oddities of browser > implementations and Javascript VMs would be contained in the compiler > and automatically pushed into the generated code. > > This would be a tremendously useful advance in building distributed > systems that work with the Web. The current approach to AJAX is a bit > like carpentry with old hand tools. You'd probably also push Python > to higher levels of language-ness. > Hmm. It's an idea. I also thought of Python -> JavaScript compiler since JavaScript is not fun and getting to test using Python might be cool. But not sure how useful that would be. Plus I bet someone has does this with Java or something. -Brett From steve at holdenweb.com Fri Jan 13 04:40:34 2006 From: steve at holdenweb.com (Steve Holden) Date: Fri, 13 Jan 2006 03:40:34 +0000 Subject: [Python-Dev] Ph.D. dissertation ideas? In-Reply-To: References: <-8397309676539309595@unknownmsgid> Message-ID: Brett Cannon wrote: > On 1/12/06, Bill Janssen wrote: > >>Brett, >> >>How about building a system that compiles a Python program (possibly >>annotated) to an AJAX program? That is, it analyzes the program to >>determine what's appropriate and possible for client-side and >>server-side, figures out the optimal network API (reduce latency, >>reduce calls, reduce data transfer, increase responsiveness), >>generates server-side Jython to run in Tomcat (or Python for >>mod_python), and client-side XHTML, CSS, and Javascript to implement >>the browser-based portion of the application. Oddities of browser >>implementations and Javascript VMs would be contained in the compiler >>and automatically pushed into the generated code. >> >>This would be a tremendously useful advance in building distributed >>systems that work with the Web. The current approach to AJAX is a bit >>like carpentry with old hand tools. You'd probably also push Python >>to higher levels of language-ness. >> > > > Hmm. It's an idea. I also thought of Python -> JavaScript compiler > since JavaScript is not fun and getting to test using Python might be > cool. But not sure how useful that would be. Plus I bet someone has > does this with Java or something. > If you can persuade your supervisor to be interested in educational games, a Python learning system might be an interesting project with lots of scope for user modelling and goal-directed behaviour. There's been some work done on bomb-proofing the interpreter to offer a "try-python" type site, but nobody seems to have thought about how to engage a neophyte with an specific teaching strategy (possibly based on personality analysis) and then present problems to verify comprehension and offer appropriate feedback. That would be a terrific game! regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/ From tony.meyer at gmail.com Thu Jan 12 05:03:35 2006 From: tony.meyer at gmail.com (Tony Meyer) Date: Thu, 12 Jan 2006 17:03:35 +1300 Subject: [Python-Dev] DRAFT: python-dev Summary for 2005-12-01 through 2005-12-15 Message-ID: <6c63de570601112003oa772cc3l4eb31cf3aad314f3@mail.gmail.com> ============= Announcements ============= ----------------------------------------------------- Reminder: plain text documentation fixes are accepted ----------------------------------------------------- Want to help out with the Python documentation? Don't know LaTeX? No problem! Plain text or ReST fixes are also welcome. You won't be able to produce a diff file like with a normal patch, but comments that explain how to fix the docs are just as good. A form like "in section XXX right before the paragraph starting with YYY, the documentation should say ZZZ" will make it easy for the doc maintainers to apply your fix. Contributing thread: - `c.l.p post on docs `__ [SJB] -------------------------------------------- New-style exceptions patch requesting review -------------------------------------------- With `PEP 352`_ ready_ for Guido's pronouncement, Michael Hudson has asked for a few more developers to review his patch_ to make all exceptions new-style. It should be basically ready, but it would be nice to have a few eyes for sanity checks, documentation, and compilations on the various platforms. .. _PEP 352: http://www.python.org/peps/pep-0352.html .. _ready: http://www.python.org/dev/summary/2005-10-16_2005-10-31.html#required-superclass-for-exceptions .. _patch: http://bugs.python.org/1104669 Contributing threads: - `New-style exceptions patch (was Re: PEP 8 updates/clarifications) `__ - `New-style exceptions patch `__ [SJB] ---------------------------------------- Debugging lib available from ActiveState ---------------------------------------- Trent Mick confirmed that ActiveState do distribute a separate distribution of debug DLLs for each Python release. These can be found by filling in the version number in the URL ftp://ftp.activestate.com/ActivePython/etc/ActivePython--win32-ix86-debug.zip Contributing thread: - `Plea to distribute debugging lib `__ [TAM] ========= Summaries ========= ------------------------------- Updating the Python style guide ------------------------------- Ian Bicking kicked off a lengthy discussion about updating various aspects of `PEP 8`_ (the Python code style guide), which resulted in a substantial revision of the PEP. PEP 8 stated that if a module defines a single exception raised for all sorts of conditions, it is generally called "error" or "Error". Ian noted that other than some outlying cases (e.g. os.error, socket.error), CapWords are always used. He also wondered if exceptions should have names that are relatively unique, or simply unique within their namespace. Finally, he requested a position be taken on acronyms (e.g. HHTPRedirect or HttpRedirect). Barry Warsaw pointed out that since exceptions are now classes, the rules for naming classes should really apply; his preference is CapWordsEndingInError rather than Error or error. He also suggested that acronym letters should be capitalised. There was mostly consensus on this, although some prefer shorter exception class names. Guido wondered whether the function/method naming convention (lower_case, mixedCase, or CapWords) was so controversial that the PEP should not make a recommendation other than of consistency. In the end, however, the status quo (lower_case except for consistency reasons) won out. He was definite about module, package, and class names, however, which should be all-lowercase without underscores (modules/packages), or CapWords (class names). (He noted that standard library modules such as StringIO.py and UserDict.py that break this rule should be changed). PEP 8 recommended appending an underscore rather than corrupted spelling when a name conflicts with a reserved word e.g. class_ rather than klass). Ian suggested that a specific recommendation for the class argument to classmethods be made; out of cls, klass and class_ he preferred cls. He further suggested that, as self is not used outside of the first argument of instance methods, whatever spelling of the class argument is used should not be used elsewhere (e.g. cls for the class argument, and class_ elsewhere). This was generally accepted. A further suggestion from Barry's `Mailman style guide`_ was also made, that from-imports should follow non-from imports, and dotted imports should follow non-dotted imports. Non-dotted imports should be grouped by increasing length, while dotted imports should be grouped roughly alphabetically. However, this was felt to be too prescriptive; users should follow their own aesthetic taste. Various other small modifications and improvements were suggested and made, such as:: * PEP 8 stated that modules designed for use via "from M import *" should prefix their globals, internal functions, and internal classes with an underscore. Ian suggested that __all__ should be recommended over using a leading underscore, which was done. * PEP 8 was fairly dated in discussing inheritance and public/non-public methods. Ian suggested that the language should be updated, and that property() should be discussed, and this was done. * PEP 8 recommended that class-based exceptions are used rather than string-based exceptions. Ian suggested that language be changed to recommend more strongly against string-based exceptions, which was done. * To make the PEP more concise and succinct, references to Emacs were removed. There was also a sub-thread that considered when when to use normal instance variables and when to use accessor methods. Jim Fulton suggested that attributes should be used if the accessors are trivial, whereas Skip Montanaro preferred more frequent use of accessor methods, to avoid incorrect usage of classes. Phillip J. Eby's opinion was that normal instance variables should be used, until there is a need to do something when they change, at which point properties should be introduced. PEP 8 now recommends simple attribute access over accessors. One distinction is how people interpret "public" and "private". Skip's opinion is that just because an attribute doesn't start with an underscore doesn't mean that it is implicitly declared public, and that people should familiarise themselves with the callable methods of an object and only use direct access if the necessary methods aren't provided. Jim felt that the API should be documented, and people should use that API (he considered that prepending an underscore documents that the attribute is internal; although this does not cover the "subclassing API"). Ian also brought up the section of PEP 8 that discussed private attributes and double leading underscores. PEP 8 was unclear whether double leading attributes should only be used to avoid name conflicts when subclassing, or whether indicating that an attribute was meant to be private was a valid use. This spun off a very long discussion about private variables. Several people spoke up against double-underscore prefixes. On the other hand, Raymond Hettinger felt that the PEP should not dictate how variables should be named, especially for a convention with a long history and language support. Jim Fulton went so far as to suggest that __private be deprecated, which gained some support. However, Guido stated that he not only liked __private, he liked the current implementation. Specifically, he felt that the typically-shallow class hierarchies found in Python reduces the likelihood of accidental reuse of class names (he also advocated that subclasses should not share names with parent classes), and he liked that the name-mangling algorithm was well-defined and simple (e.g. so that when in the debugger it is easy to manually mangle/unmangle names). Interestingly, this is the main reason that Jeremy Hylton dislikes mangled names: because the debugger is unaware of the names and he can't remember how to type them, and also because it's annoying to have to change every instance of __var to _var if the intended usage changes (as compared to C++ private variables, where only the declaration of visibility must change). He noted, though, that these are problems that tools (debuggers, editors, IDEs) could solve. Others felt that keeping mangling was fine, but that it should be changed so that collisions were harder (e.g. use a GUID instead of the class name). However, this violates one of Guido's reasons for liking the current implementation (although it's possible that having better support for the feature in common tools would remove this necessity). Tim Peters gave a nice explanation of why the name mangling is provided:: If your utility or mixin base class `A` has a data member named `n`, nobody deriving from `A` dare name one of their data members `n` too, and it's unreasonable to expect everyone deriving from `A` to learn and avoid all the names `A` uses internally. It's even more unreasonable for A's author to have to promise, after A's first release, never to change the name of, or introduce any new, attribute (A's author dare not, lest the new name conflict with a name someone else's subclass used). If A's author names the attribute `__n` instead, all those problems go away, provided only that some subclass doesn't also name itself `A`. Contributing threads: - `PEP 8 updates/clarifications `__ - `Deprecate __ private (was Re: PEP 8 updates/clarifications) `__ - `Import order (was Re: PEP 8 updates/clarifications) `__ - `Deprecate __ private (was Re: PEP 8updates/clarifications) `__ - `PEP 8 updates/clarifications, function/method style `__ .. _PEP 8: http://www.python.org/dev/peps/pep-0008.html .. _Mailman style guide: http://barry.warsaw.us/software/STYLEGUIDE.txt [TAM] ------------------------------------ ElementTree in the Python 2.5 stdlib ------------------------------------ Skip Montanaro forwarded to python-dev a bit of a comp.lang.python thread asking why ElementTree wasn't part of the standard library. He and others argued that it was "best of breed" with a wide user base, and that it met the other requirements_ for stdlib inclusion like having an active maintainer, Fredrik Lundh. After a relatively brief discussion, Fredrik got the ball rolling, setting things up so that: * The ``external`` directory at the top of the SVN tree contains snapshots of his celementtree and elementtree packages * The ``etree`` subpackage of the ``xml`` module contains the ElementTree, cElementTree, ElementPath and ElementInclude modules Mike Brown started a brief follow-up thread concerned that the XML-SIG hadn't been consulted on this inclusion. Martin v. L?wis and others indicated that sidestepping XML-SIG had not been intensional, but also that carrying the discussion on XML-SIG as well would not likely have changed the outcome due to the strong community support for ElementTree. As a side effect of this discussion, magic in ``xml/__init__`` was removed in favor of introducing an ``xmlcore`` package containing all the xml modules found in the basic standard library, and an ``xml`` module which imports things from ``xmlcore`` or PyXML if it's available. .. _requirements: http://mail.python.org/pipermail/python-dev/2003-April/034881.html Contributing threads: - `ElementTree - Why not part of the core? (fwd) `__ - `stupid package tricks `__ - `ElementTree in stdlib `__ - `Sharing expat instances `__ - `"xml"; package in standard library `__ [SJB] -------------------- More work on the AST -------------------- This fortnight saw implementations for the two main models proposed for revamping the AST memory management. Martin v. L?wis created a branch_ that converted all AST objects to PyObjects and used the normal Python reference counting to manage them. This requires that AST code: * initialize all PyObjects to NULL * Py_XDECREF() each PyObject on exit * goto an error block for Py_DECREF()ing if there is a failure Jeremy Hylton worked up a separate version of the AST code using an arena API. This requires that AST code: * call PyArena_AddPyObject() for any allocated PyObject * call PyArena_AddMallocPointer() for any malloc()ed memory The arena code was merged into the trunk, though it seemed that work on the PyObject branch would continue in order to be able to compare the final outcomes of both strategies. In a related issue, because the AST code is generated from Parser/asdl_c.py, and because subversion sets the timestamp for each file to the checkout time, trying to build the current trunk on a machine without Python installed failed even when the AST C files had been checked into subversion. It looked like the best solution would be to introduce a separate make rule for generating the AST code. .. _branch: http://svn.python.org/projects/python/branches/ast-objects/ Contributing threads: - `ast-objects branch created `__ - `Memory management in the AST parser & compiler `__ - `PyAST_FromNode returning PyTypeObject* ? `__ - `should I really have to install Python before I can build it ? `__ - `should I really have to install Python before Ican build it ? `__ - `should I really have to install Python before Icanbuild it ? `__ [SJB] ------------------- Python GC Refresher ------------------- Gregoire Weber asked for a little more information on Python's garbage collector since the links in ``Modules/gcmodule.c`` were broken. Python's garbage collector is a combination of reference counting and the periodic invocation of a cyclic garbage collector. Python's reference counting means that each time an object P is referenced by another object, P's refcount is incremented, and each time one of the references to P is removed, P's refcount is decremented. When P's refcount reaches zero, the interpreter pauses to reclaim P and all objects that were reachable only from P. In addition to this reference counting, Python's cyclic garbage collector means that after a large number of objects have been allocated (and not deallocated though reference counting), the interpreter will pause to try to clear out any unreferenced cycles. Contributing thread: - `Documentation about Python's GC, python-dev list messages referenced in Modules/gcmodule.c not reachable anymore `__ [SJB] ----------------------------------- Improving the documentation process ----------------------------------- Skip Montanaro suggested that, in order to lower the barrier for submitting documentation patches, it might be worth allowing anonymous bug reports. Guido was against this, indicating that he thought the problem was more the sourceforge sign-up hassle than the need to provide an email address, and suggested that it might be time to switch to roundup_. Martin v. L?wis was concerned about the transition process - whether or not roundup could properly import the sourceforge bug reports, and whether or not the python.org/sf/ redirect would continue to work. The discussion trailed off before any final decisions were made. .. _roundup: http://roundup.sourceforge.net/ Contributing threads: - `Tracker anonymity `__ [SJB] ------------------------ hasattr() and properties ------------------------ Thomas Lotze noticed that when applied to a class instance (but not an object of that class), hasattr calls getattr and decides that the attribute doesn't exist if the call raises any exception. For example:: >>> class Foo(object): ... def get(self): ... raise Exception ... bar = property(get) ... >>> hasattr(Foo, "bar") True >>> hasattr(Foo(), "bar") False He asked whether it would make more sense to only report a missing attribute if an AttributeError is raised. Greg Ewing agreed that this would be an improvement, but felt that calling the property access code as a side effect of hasattr seems like a misfeature. Thomas also wondered whether it would make sense for properties to look up the attribute in the same way that getattr would rather than calling getattr. Greg wondered if descripters need a forth slot for hasattr customisation, removing the need to rely on catching exceptions, so that the logic would be:: if there is a descriptor for the attribute: if the descriptor's hasattr slot is populated: return the result of calling it else: return True else: look in the instance dict for the attribute Guido indicated that he believes that a property that has a side effect (other than caching, statistics, logging, and so forth) is a misfeature anyway, so he doesn't have a problem with hasattr() trying getattr() and reporting False if that raises an exception. Discussion died out before anything was resolved. Contributing thread: - `hasattr and properties `__ [TAM] --------------------------------- Supporting iterables in xmlrpclib --------------------------------- Skip Montanaro `presented a patch`_ to allow all currently supported iterables (e.g. sets and arrays) to be marshalled as lists with xmlrpclib. The goals are to extend the set of sequences that can be marshalled transparently, and keep the caller from caring as much about the limitations of the XML-RPC datatypes. Guido felt that this was a bad idea, however; he feels that it's better to be aware of the limitations in what XML-RPC can handle and try to handle it. Contributing thread: - `Broader iterable support for xmlrpclib `__ [TAM] ----------------------------------------- Checking Jython support code into CPython ----------------------------------------- Fredrik Lundh asked what the policy with respect to Jython-specific modules in the standard library was. Guido felt that as long as it didn't do any harm (likely causing unit tests to fail) to CPython, it would be fine. He noted that traditionally Jython-specific code has been checked into Jython's own source control, however, and Martin v. L?wis indicated that this is what he would prefer. - `Jython and CPython `__ [TAM] ----------------------------------------- Getting rid of __builtins__ in Python 3.0 ----------------------------------------- Neal Norwitz asked Guido whether improving the confusing system of having both __builtins__ and __builtin__ could be begun. Guido clarified that having both is clearly a bad idea, that he's not sure that renaming builtin to __builtin__ was correct (and that perhaps it should be changed back), that __builtins__ always being a dict would simplify some code but need special handling of vars() in interactive mode, and that another alternative might be to merge the __builtin__ and __builtins__ functionality (under the __builtin__ name). Guido asked that people mull this over, but hasn't had any responses so far. - `__builtin__ vs __builtins__ `__ [TAM] ================ Deferred Threads ================ - `Linked lists `__ =============== Skipped Threads =============== - `Python bug day this Sunday `__ - `Subject lines of commit email `__ - `Proposed additional keyword argument in logging calls `__ - `os.normpath may change the meaning of the path if it contains symbolic links? `__ - `SVN backup? `__ - `svn problem - can't get log info for a specific revision `__ - `Patch reviews & request for patch review `__ - `Dynamic Link Library `__ - `[Python-checkins] commit of r41586 - in python/trunk: Lib/SimpleXMLRPCServer.py Misc/NEWS `__ - `Short-circuiting iterators `__ - `Bug bz2.BZ2File(...).seek(0,2) + patch `__ - `imaplib module with IDLE implememted via threads `__ - `Exception type on handling closed files `__ - `A missing piece of information in weakref documentation `__ - `Directory for packages maintained outside the core (was Re: ElementTree - Why not part of the core?) `__ - `Incorporating external packages into Python's std distribution `__ - `On moving to new-style classes `__ - `Weekly Python Patch/Bug Summary `__ - `Website cruft `__ - `Add timeout to subprocess.py? `__ - `patch tracker ping: cross compile and mingw support `__ - `Needed tester for patch in urllib.py module `__ - `Location of .so files (Was: Sharing expat instances) `__ From tony.meyer at gmail.com Thu Jan 12 05:08:08 2006 From: tony.meyer at gmail.com (Tony Meyer) Date: Thu, 12 Jan 2006 17:08:08 +1300 Subject: [Python-Dev] DRAFT: python-dev Summary for 2005-12-01 through 2005-12-15 Message-ID: <6c63de570601112008o69989401n99487d15c8f7cd5e@mail.gmail.com> Opps. I just sent out the draft summary for the first half of December (which might only make it to the list after this one, since it's very long) but forgot to say anything at the top. No-doubt everyone knows the pitch by now, but if anyone is able to take a look at the summary (or parts of it) and send any comments/suggestions/corrections/additions to either me or Steve, that would be great. Responding off-list would be best, to keep noise down (unless you want to revitalise one of the threads, of course). As always, thanks to all for your help! The second half of December draft should follow tomorrow, and hopefully the first January one next week. =Tony.Meyer From rkiendl at gmx.net Thu Jan 12 11:18:43 2006 From: rkiendl at gmx.net (Robert Kiendl) Date: Thu, 12 Jan 2006 11:18:43 +0100 Subject: [Python-Dev] Unicode print/stdoutput exceptions are not nice Message-ID: <43C62D03.9050909@gmx.net> <1137059888.538391.119110 at o13g2000cwo.googlegroups.com> Martin v. L?wis schrieb: > Robert wrote: > > is in a PythonWin Interactive session - ok results for cyrillic chars > > (tolerant mbcs/utf-8 encoding!). > > But if I do this on Win console (as you probably mean), I get also > > encoding Errors - no matter if chcp1251, because cyrillic chars raise > > the encoding errors also. > > If you do "chcp 1251" (not "chcp1251") in the console, and then > run python.exe in the same console, what is the value of > sys.stdout.encoding? correctly: 'cp1252' in my case; cyrillic-chars break "print" (on PC linux 2.2 tty py24 sys.stdout.encoding is None (?); only locale.getdef... useful? ) I live with this in site(customize): import codecs _stdout=sys.stdout if sys.platform=='win32' and not sys.modules.has_key('pywin.framework.startup'): _stdoutenc=getattr(_stdout,'encoding',None) or sys.getdefaultencoding() try: codecs.lookup(_stdoutenc) except LookupError: _stdoutenc=sys.getdefaultencoding() class StdOut: def write(self,s): _stdout.write(s.encode(_stdoutenc,'backslashreplace')) sys.stdout=StdOut() elif sys.platform.startswith('linux'): import locale _stdoutenc=locale.getdefaultlocale()[1] try: codecs.lookup(_stdoutenc) except LookupError: _stdoutenc=sys.getdefaultencoding() class StdOut: def write(self,s): _stdout.write(s.encode(_stdoutenc,'backslashreplace')) sys.stdout=StdOut() > > I think this is not a good behaviour of python to be so picky. > > I think it it is good. > > Errors should never pass silently. > Unless explicitly silenced. A political question. Arguments: * Webbrowsers for example have to display defective HTML as good as possible, unknown unicode chars as "?" and so on... Users got very angry in the beginning of browsers when 'strict' programmers displayed their exception error boxes ... * at least the "print" statement has to go through - the costs (for angry users and developers; e.g. http://blog.ianbicking.org/do-i-hate-unicode-or-do-i-hate-ascii.html) are much higher when apps suddenly break in simple print/display-output when the system picks up alien unicode chars somewhere (e.g. occasionally in filenames,...). No one is really angry when occasionally chinese chars are displayed cryptically on non-chinese computers. One can investigate, add fonts, ... to improve, or do nothing in most cases, but apps do not break on every print statement! This is not only true for tty-output, but also for log-file redirect and almost any common situation for print/normal stdout/file-(write)-output. * anything is nice-printable in python by default, why not unicode-strings!? If the decision for default 'strict' encoding on stdout stands, we have at least to discuss about print-repr for unicode. * the need for having technical strings 'strict' is much more rare. And programmers are anyway very aware in such situations . e.g. by asciifile.write( us.encode(xy,'strict') ) . * on Windows for example the (good) mbcs_encode is anyway tolerant as it: unkown chars are mapped to '?' . I never had any objection to this. Some recommendations - soft to hard: * make print-repr for unicode strings tolerant (and in PythonWin alwasy tolerant 'mbcs' encoding) * make stdout/files to have 'replace'-mode encoding by default. (similar as done with my code above) * set site.py/encoding=('ascii', 'replace') # if not utf-8/mbcs/locale ;enable a tuple * save sys._setdefaultencoding by default * I would also live perfectly with .encode(enc) to run 'replace' by default, and 'strict' on demand. None of my apps and scripts would break because of this, but win. A programmer is naturally very aware when he wants 'strict'. Can you name realistic cases where 'replace' behavior would be so critical that a program damages something? > > In > > 1136925967.990106.299760 at g44g2000cwa.googlegroups.com I showed, how I > > solved this so far. Any better/portable idea? > > Not sure why you aren't using sys.stdout.encoding on Linux. I would do > > try: > c = codecs.getwriter(sys.stdout.encoding) > except: > c = codecs.getwriter('ascii') > sys.stdout = c(sys.stdout, 'replace') > > Also, I wouldn't edit site.py, but instead add sitecustomize.py. I have more problems with the shape of sys.path in different situations, multiple sitecustomize.py on other apps, environments, OS / users, cxfreeze,py2exe ... sitecustomize not stackable easily: a horror solution. The need is for a callable _function_ or for general change in python behaviour. modifiying site.py is better and stable for me (I have my patch/module-todo-list handy each time i install a new python), as I always want tolerant behaviour. in code i check for site.encoding/_setdefaultencoding (I save this). Thus i get one central error if setup is not correct, but not evil unicode-errors somewhere deep in the app once on a russian computer in the future... > > Yes. But the original problem is, that occasionally unicode strings > > (filenames in my case) arise which are not defined in the local > > platform encodings, but have to be displayed (in 'replace' encoding mode) > > without breaking the app flow. Thats the pain of the default behaviour > > of current python - and there is no simple switch. Why should "print > > xy" not print something _always_ as good and as far as possible? > > Because the author of the application wouldn't know that there > is a bug in the application, and that information was silently > discarded. Users might only find out much later that they have > question marks in places where users originally entered data, > and they would have no way of retrieving the original data. > > If you can accept that data loss: fine, but you should silence > the errors explicitly. this is black/white theoretical - not real and practical (as python wants to be). see above. Robert From gabriel.becedillas at corest.com Thu Jan 12 17:47:31 2006 From: gabriel.becedillas at corest.com (Gabriel Becedillas) Date: Thu, 12 Jan 2006 13:47:31 -0300 Subject: [Python-Dev] pystate.c changes for Python 2.4.2 Message-ID: <43C68823.3000105@corest.com> Hi, At the company I work for, we've embedded Python in C++ application we develop. Since our upgrade to Python 2.4.2 from 2.4.1 we started hitting Py_FatalError("Invalid thread state for this thread") when using debug builds. We use both multiple interpreters and thread states. I think the problem is that PyThreadState_Delete (which is used when I destroy thread states and interpreters) is not making the same clean up that PyThreadState_DeleteCurrent is doing (which is used when we create threads from python code). If a thread id is reused (obviously not between two running threads), and the previous thread used PyThreadState_Delete instead of PyThreadState_DeleteCurrent, then an old and invalid value for autoTLSkey is still stored, and that is triggering the Py_FatalError when a call to PyThreadState_Swap is made. If I add this code at the end of PyThreadState_Delete: if (autoTLSkey && PyThread_get_key_value(autoTLSkey) == tstate) PyThread_delete_key_value(autoTLSkey); then everything works fine. Could you please confirm if this is a bug ? Thanks a lot and have a nice day. From tony.meyer at gmail.com Thu Jan 12 23:46:02 2006 From: tony.meyer at gmail.com (Tony Meyer) Date: Fri, 13 Jan 2006 11:46:02 +1300 Subject: [Python-Dev] DRAFT: python-dev Summary for 2005-12-16 through 2005-12-31 Message-ID: <6c63de570601121446p390477a7q97a1486bada53d0d@mail.gmail.com> Here's the second December summary. As always, if anyone can spare some time to take a look over it and send any comments/suggestions/corrections/additions to me or Steve that would be great. I'm not all that confident about the "default comparisons" thread, so particular attention to that would be great. Thanks! ============= Announcements ============= ---------------------------- QOTF: Quote of the Fortnight ---------------------------- Python-dev is in love with Python, though sometimes too much, Fredrik Lundh contends: ...in reality, some things are carefully thought out and craftily implemented, some things are engineering tradeoffs made at a certain time, and some things are just accidents -- but python-dev will happily defend the current solution with the same energy, no matter what it is. Contributing thread: - `a quit that actually quits `__ [SJB] ------------------------------------------------------- Reminder: plain text documentation patches are accepted ------------------------------------------------------- Want to help out with the Python documentation? Don't know LaTeX? No problem! Plain text or ReST fixes are also welcome. You won't be able to produce a diff file like with a normal patch, but comments that explain how to fix the docs are just as good. A form like "in section XXX right before the paragraph starting with YYY, the documentation should say ZZZ" will make it easy for the doc maintainers to apply your fix. Contributing thread: - `LaTeX and Python doc contributions `__ [SJB] --------------------------------------------------------------- PyPy Sprint: Palma de Mallorca (Spain) 23rd - 29th January 2006 --------------------------------------------------------------- The next PyPy_ sprint is scheduled to take place from the 23rd to the 29th of January 2006 in Palma De Mallorca, Balearic Isles, Spain. There will be newcomer-friendly introductions and the focus will mainly be on current JIT work, garbage collection, alternative threading models, logic programming and on improving the interface with external functions. .. _PyPy: http://codespeak.net/pypy Contributing thread: - `Next PyPy Sprint: Palma de Mallorca (Spain) 23rd - 29th January 2006 `__ [TAM] -------------------------------------------------- SHA-224, -256, -384 and -512 support in Python 2.5 -------------------------------------------------- Ronald L. Rivest asked about the cryptographic hash function support in Python, now that md5 and sha1 are broken in terms of collision-resistance. The new-in-Python-2.5 hashlib module was pointed out (and somewhat belatedly added to the NEWS file), which includes new implementations for SHA-224, -256, -384 and -512 (these, including tests, are already in SVN). Gregory P. Smith noted that although the code isn't currently available for earlier versions of Python, he does plan to release a standalone package for Python 2.3 and Python 2.4, when time permits. Contributing thread: - `hashlib - faster md5/sha, adds sha256/512 support `__ [TAM] ========= Summaries ========= ----------------------------------- Improving the documentation process ----------------------------------- Fredrik Lundh asked about automatically updating the development docs, so that users wouldn't be required to have a latex toolchain installed to get an HTML version of the current docs. This spawned a long discussion about using something other than LaTeX (e.g. microformats_ or ReST_) for markup. Though HTML has the advantage that many Python users are already familiar with it, a number of people voiced concerns about the ease of reading and writing it. ReST is generally easy to read and write, but some people suggested that for complicated structured text (like Python function and class definitions) it was no better than LaTeX. Fredrik Lundh presented some example code side-by-side_ and argued that using HTML with something like PythonDoc_ could better represent the documentation's intent. He also produced an initial conversion_ of the Python docs to this format. Fredrik also pointed out that currently the doc patch submission procedure is rather tedious, and listed_ the rather large number of steps required for end-users and developers to get their documentation fixes added to the current Python docs. He claimed that a simple wiki, discussion board, or "user edit" mechanism like that of roundup_, combined with automated updates of the documentation from the Python SVN trunk, could reduce this procedure to two or three simple steps. As a partial effort towards this goal, Trent Mick started posting `daily builds`_ of the Python HTML. This prompted Neal Norwitz to set up the docs server in a similar manner so that it now produces development documentation builds twice daily at http://docs.python.org/dev/. .. _ReST: http://docutils.sourceforge.net/rst.html .. _microformats: http://microformats.org/wiki/microformats .. _side-by-side: http://mail.python.org/pipermail/python-dev/2005-December/059022.html .. _PythonDoc: http://effbot.org/zone/pythondoc.htm .. _conversion: http://effbot.org/zone/pythondoc-lib.htm .. _listed: http://mail.python.org/pipermail/python-dev/2005-December/059311.html .. _roundup: http://roundup.sourceforge.net .. _daily builds: http://trentm.com/python/ Contributing threads: - `status of development documentation `__ - `documentation comments `__ - `[Doc-SIG] status of development documentation `__ - `reST limitations? (was Re: status of development documentation) `__ - `that library reference, again `__ - `[Doc-SIG] that library reference, again `__ [SJB] -------------------------------------- Making quit and exit more command-like -------------------------------------- Fredrik Lundh kicked off a surprisingly long thread when he proposed that typing "quit" or "exit" in the interactive prompt actually exit (i.e. raises SystemError) rather than printing a message informing the user how they can exit, to avoid the "if you knew what I wanted, why didn't you just do it?" situation. His proposal was to install a custom excepthook that looks for the appropriate NameErrors at the top level (in interactive mode only). Early discussion revolved around the implementation. Skip Montanaro worried that multiple code sources wanting to change sys.excepthook would step on one another's toes; Fredrik felt that if you add your own excepthook, you take responsibility. Martin was also concerned about this; although Fredrik believed that the change was so early that no code would try to add a hook prior to this, Martin suggested that the code stand as an example of good practice and pass execution on to any existing excepthook (if the command wasn't "exit" or "quit"). Reinhold Birkenfeld suggested that exit and quit could be instances of a class whose repr() raises SystemExit; however, this would cause an exit to be incorrectly performed whenever the exit or quit objects were examined (e.g. calling vars()). Reinhold suggested a variation where the SystemExit would only be raised if sys._getframe(1).f_code.co_names was "exit" or "quit"). Having quit and exit be functions (i.e. one would type "exit()" to exit) was also suggested, with a plain "exit" printing a help message (as today). However, while this shortens the platform-independent method of exiting ("raise SystemExit") to "exit()", it does nothing to address the "why didn't you just do it?" problem. Ka-Ping Yee was concerned that Fredrik's implementation would cause an exit in surprising situations, such as :: print exit # seems surprising [3, 4, 5, exit] # seems surprising 'a' in 'xyz' or exit # desirable or undesirable? del exit # actually happened to me x = exit # seems surprising And that Reinhold's modified proposal would do likewise:: print exit # seems surprising [3, 4, 5, exit] # seems surprising 'a' in 'xyz' or exit # desirable or undesirable? def foo(): print exit foo() # seems surprising As such, Fredrik proposed adding a new sys attribute that contains the most recent interpreter line, which could be examined to check that the line entered was simply "exit" or "quit" by itself. Skip suggested that such an attribute be made clearly private magic (e.g. by naming it sys._last_input). Michael Hudson suggested that the clever hacks could be avoided by simply having the equivalent of "if input.rstrip() == "exit": raise SystemExit" in the implementation of the interactive interpreter. Fredrik felt that this would elevate "exit" and "quit" into reserved keywords, in that this could occur:: >>> def exit(): ... print "bye" >>> # what is it? >>> exit $ oops! (However, one could simply type "repr(exit)" in this case). Walter D?rwald suggested adding "sys.inputhook" (c.f. sys.displayhook, sys.excepthook), which gets each statement entered and returns True if it processed the line or False if normal handling should be done; Alex Martelli made a similar suggestion. This would not only allow special handling for "exit", "quit", and "help", but also might make implementing alternative shells easier. Nick Coghlan added a default handler to this suggestion, which checked a new dictionary, "sys.aliases", for special statements of this type and called an appropriate function if found. Fredrik's concern with this method was the same as with Michael Hudson's - that typing "exit" as a shortcut for "print repr(exit)" (when the user had assigned a value to "exit") would cause the shell to exit. As a result, Nick modified his default inputhook to check that the statement didn't exist in __main__'s vars(). Fernando Perez pointed out that this begins to come close to reimplementing ipython_. Fernando also commented that he felt that the amount of code he thought would be necessary to avoid any side-effects when determining whether "exit" should exit or not would make the standard interpreter too brittle, and that this should be left to third-party interpreters. He gave a lengthy explanation of the process that ipython went through, including linking to `the _prefilter method` that performs the appropriate magic in ipython. His opinion was that interpreters should have a separate command mode, entered by a particular character (e.g. the '%' used in ipython); while this would simply code, '%exit' would be even harder for people to guess, however. Some people were concerned that "quit" and "exit" would be treated specially (i.e. the standard way to do something is to "call a function" and a special case for exiting the interpreter is inappropriate); others pointed out that this would be there especially for people who don't know how to program in Python, and might not know that typing an EOF would exit the shell (or what the EOF character was for their platform). Fredrik's proposal gained a lot of early support, but later posters were not so keen. Stephen J. Turnbull suggested that the problem was simply that the current "help"/"exit" messages are impolite, and changing the language would solve the problem. Similarly, Aahz suggested that adding an explanation of how to quit to the startup message would suffice, although others disagreed that users would notice this, particularly when it was needed. Despite the strong support, however, in the end, Guido stepped in and stated that, in his opinion, nothing was wrong wih the status quo (or that if anything were to be changed, the hints provided by typing "exit" or "quit" should be removed). This pretty much ended the discussion. .. _ipython: http://ipython.scipy.org .. _`the _prefilter method`: http://projects.scipy.org/ipython/ipython/file/ipython/trunk/IPython/iplib.py Contributing threads: - `a quit that actually quits `__ --------------------------------------- Exposing the Subversion revision number --------------------------------------- Barry Warsaw proposed `a fairly simple patch`_ to expose the Subversion revision number to Python, both in the Py_GetBuildInfo() text, in a new Py_GetBuildNumber() C API function, and via a new sys.build_number attribute. A lengthy discussion took place about the correct method of determining the "revision number" that should be used. There are many possibilities, from the originally proposed revision number in which the directory was last modified, to the latest revision number within the tree, to using subversion itself. Each method requires a different amount of processing in order to determine the correct value. Barry indicated a preference for the original, simple, method, because he wished to keep the patch as simple as possible, didn't want to depend on Python already being built, and felt that generally developers will just 'svn up' at the top of the source tree then rebuild, rather than 'svn up' a buried sub-tree, change back to the top directory, and rebuild from there. Phillip J. Eby's concern with this was that this would not indicate if a change has been made locally and committed, unless the developer also did a 'svn up'. He also pointed out that the "Revision" from 'svn info' can change when the code doesn't change (because it will change when code in branches, the sandbox, PEPs, and so forth change). In the end, using the svnversion command was agreed on as a suitable method. The two main advantages are that this is the officially sanctioned (by Subversion) method, and that it indicates when local changes have been made. Armin Rigo voiced a concern that people would use sys.build_number to check for features in their programs, instead of using sys.version_info (or duck-typing methods such as hasattr()) because it seems that comparing a single number is easier than comparing a tuple. This was of especial concern considering that this value would only be present in CPython. He suggested that a new build_info attribute could be added to sys instead, which would be a tuple of ("CPython", "", "trunk"); that is, it would also include which Python implementation the program is using (for particular use by the test suite). The name also has advantages in that the number is actually a string (because 'M' will be present if there are local modifications) and it is not tied to a specific version control system. Michael Hudson pointed out that this value will not be able to be obtained when Python is built from an 'svn export' (where the .svn files do not exist), which is probably the case for many Python distributions. He wondered whether a subversion trigger could put the revision number into some file in the repository to cover this case, but Martin indicated that this would be difficult (although if the distribution is built from a tag, that the information would not be necessary). Barry suggested that the number would be the latest revision at which getbuildinfo was changed if a .svn directory isn't found. As an aside to this patch, Martin suggested that the build number should be dropped, as it stopped working on Windows some time ago and no longer provides any useful information. There was no opposition to this proposal, and some support. .. _a fairly simple patch: http://python.org/sf/1382163 Contributing threads: - `Expose Subversion revision number to Python `__ [TAM] --------------------------------------- Python's lists, linked lists and deques --------------------------------------- Martin Blais asked why Python didn't have native singly-linked or doubly-linked list types. Generally, it seemed that people couldn't find use cases for them that weren't covered by Python's builtin lists, collections.deque objects, or head-plus-tail-style two-tuples. The proposal was redirected to comp.lang.python until it was more developed. In a related thread, Christian Tismer asked if Python's list type could do a little usage-tracking and switch between the current array-based implementation and a deque-based implementation if, say, a large number of inserts or deletes began to occur at the beginning of the list. He got a few responses suggesting that it would be more complication that it was worth, but mostly his question got drowned out by the linked-list discussion. Contributing threads: - `Linked lists `__ - `deque alternative (was: Linked lists) `__ - `deque alternative `__ - `(back to): Linked lists -- (was: deque alternative) `__ [SJB] ------------------------ set() memory consumption ------------------------ Noam Raphael noted that sets (and dicts) do not release allocated memory after calls to pop(). He suggested that to keep pop() as amortized O(1) per delete, all that was needed was to resize the table to half its current size whenever it dropped to less that one quarter full. People seemed generally to think that the dict implementation (and therefore the set implementation which is based on it) were already highly optimized for real-world performance, and that there were too few use cases where a set would be filled and then emptied, but not filled up again. Contributing thread: - `When do sets shrink? `__ [SJB] --------------------------------------------------------- Changing the naming conventions of builtins in Python 3.0 --------------------------------------------------------- Ka-Ping Yee provocatively suggested that, to be consistent with the recommended style guide, in Python 3.0 built-in constants (None, True, False) should be in all caps (NONE, TRUE, FALSE), and built-in classes (object, int, float, str, unicode, set, list, tuple, dict) should be in CapWords (Object, Int, Float, Str, Unicode, Set, List, Tuple, Dict). However, Michael Chermside noted that there is a common convention that the fundamental built-in types (not standard library types; e.g. set, not sets.Set) are all in lowercase. Almost no-one liked this idea, and many people were vehemently opposed. Guido quickly pronounced this dead for built-in types (cleaning up and making the more consistent the standard library classes is another matter). Contributing thread: - `Naming conventions in Py3K `__ [TAM] ------------------- Default comparisons ------------------- Jim Jewett pointed out that `PEP 3000`_ now suggests that dropping default comparison has become more than an idle what-if, which means that the common use case of comparisons to get a canonical order (not necessarily one that makes sense) will no longer work in many cases. An alternative solution would be to promote a standard way to say "just get me a canonical ordering of some sort". "Comparison" would raise a TypeError if the pair of objects were not comparable; "Ordering" would continue on to "something consistent", just like sorts do today. Josiah Carlson suggested that this could be achieved by new superclasses for all built-in types (except string and unicode, which already subclass from basestring). int, float, and complex would subclass from basenumber; tuple and list would subclass from basesequence; dict and set would subclass from basemapping. Each of these base classes define a group in which items are comparable; if you end up in a situation in which the base classes of the compared object differ, you compare their base class name. If a user-defined classes wanted to be compared against (e.g.) ints, floats, or complex, the user would subclass from basenumber; if the user only wanted their objects to compare against objects of its own type, they define their own __cmp__. However, Marc-Andre Lemburg pointed out that Python already uses this 'trick' based on the type name (falling back to id(), which is the problem), and that inheriting from (e.g.) basenumber could result in unexpected side-effects. .. _PEP 3000: http://www.python.org/peps/pep-3000.html Contributing thread: - `Keep default comparisons - or add a second set? `__ [TAM] --------------------------------------- Converting open() to a factory function --------------------------------------- Guido had previously_ indicated that while file() will always stay the name of the file object, open() may be changed in the future to a factory function (instead of just being an alias for file). Noting that ``help(open)`` now just displays the file object documentation, Aahz suggested that open() become a factory function now. Fredrik Lundh chimed in to suggest that a textopen() function should also be introduced, which would call codecs.open() or file() depending on whether or not an encoding was supplied. There was mild support for both proposals, but no patches were available at the time of this writing. .. _previously: http://mail.python.org/pipermail/python-dev/2004-July/045921.html Contributing thread: - `file() vs open(), round 7 `__ [SJB] ----------------------------- NotImplemented and __iadd__() ----------------------------- Facundo Batista presented a bug_ where using += on Decimal objects returned NotImplemented instead of raising an exception. Armin Rigo was able to supply a patch after adding the following conditions to the documentation: * PySequence_Concat(a, b) and operator.concat(a, b) mean "a + b", with the difference that we check that both arguments appear to be sequences (as checked with operator.isSequenceType()). * PySequence_Repeat(a, b) and operator.repeat(a, b) mean "a * b", where "a" is checked to be a sequence and "b" is an integer. Some bounds can be enforced on "b" -- for CPython, it means that it must fit in a C int. .. _bug: http://www.python.org/sf/1355842 Contributing thread: - `NotImplemented reaching top-level `__ [SJB] --------------------------------------------------- Using buildbot to automate testing of Python builds --------------------------------------------------- Contributing thread: - `Automated Python testing (was Re: status of development documentation) `__ ----------------------------------------------------- Importing C++ extensions compiled with Visual C++ 8.0 ----------------------------------------------------- Ralf W. Grosse-Kunstleve indicated that while they could compile their C++ extensions with Visual C++ 8.0, importing any such extensions resulted in an error:: This application has failed to start because MSVCP80.dll was not found. Re-installing the application may fix this problem. Though the combination of compiling Python with VS.NET2003 and an extension with VS2005 is not supported, Ralf was able to get things to work by adding ``/manifest`` to the linker options and envoking ``mt.exe`` to embed the manifest. Contributing thread: - `Python + Visual C++ 8.0? `__ [SJB] ------------------------------- Using ssize_t as the index type ------------------------------- Martin v. L?wis presented a new (as yet unnumbered) PEP to use ssize_t as the index type in the CPython implementation. In Python 2.4, indices of sequences are restricted to the C type int. On 64-bit machines, sequences therefore cannot use the full address space, and are restricted to 2**31 elements. The PEP proposes to change this, introducing a platform-specific index type Py_ssize_t. An implementation of the proposed change is in `an SVN branch`_; the Py_ssize_t will have the same size as the compiler's size_t type, but is signed (it will be a typedef for ssize_t where available). Although at the moment one needs 16GiB to just hold the pointers of a 2GiElement list, these changes also allow strings and mmap objects longer than 2GiB (and also improve the scipy_core (ne? Numarray and Numerical) objects, which are already 64-bit clean). Since the proposed change will cause incompatibilities on 64-bit machines, the reasoning is that this change should be done as early as possible (i.e. while such machines are not in wide use). Guido approved the PEP, indicating he believed it was long overdue. Several other people also indicated support for the PEP. Note that people felt that Martin's PEP was of a particularly high standard; people considering writing their own PEP should consider reading this (once it has a number and is available online!) to gain an understanding of how a good PEP is both extremely readable (even when including arcane C programming points!) and well-organised. .. _an SVN branch: http://svn.python.org/projects/python/branches/ssize_t Contributing thread: - `New PEP: Using ssize_t as the index type `__ [TAM] ----------------------------------------------------------------------------- Garbage collection based on object memory consumption instead of object count ----------------------------------------------------------------------------- Andrea Arcangeli suggested that Python's garbage collection could be improved by invoking a gc.collect() at least once every time the amount of anonymous memory allocated by the interpreter increases by a tunable factor (defaulting to 2, meaning the memory doubles, and with a factor of 1 mimicking the current state, where collection is done if 1000 new objects are allocated). Aahz and Martin v. L?wis indicated that it was extremely unlikely that anything would be done about this unless Andrea submitted a patch (with doc patches and tests). Contributing thread: - `suggestion for smarter garbage collection in function of size (gc.set_collect_mem_growth(2)) `__ [TAM] ---------------------------------- Adding zlib module to python25.dll ---------------------------------- Thomas Heller and Martin Heller had been discussing whether the zlib module should become builtin, at least on Windows (i.e. part of python25.dll). This would simplify both the build process and py2exe, the latter could then bootstrap extraction from the compressed file just with pythonxy.dll, but this is currently not done, because the pythoncore.vcproj would then not be buildable anymore unless you also have the right version of zlib on disk. To solve this problem, Thomas proposed that the Python release could incorporate a copy of zlib, primarily for use on Windows (with the project files appropriately adjusted). This change was later made. Contributing thread: - `Incorporation of zlib sources into Python subversion `__ [TAM] ---------------------------------------- Procedure for fixing bugs in ElementTree ---------------------------------------- Neal Norwitz found a few bugs in ElementTree and asked what the procedure was, since the ElementTree module is maintained externally by Fredrik Lundh. The answer was to post the bug to sourceforge as usual, and and assign it to Fredrik Lundh. The ElementTree in the Python SVN trunk should only have critical patches committed - all other patches would be applied by Fredrik to his externally distributed ElementTree package, and imported to the Python SVN trunk after the next release of ElementTree. Contributing thread: - `ref leak in element tree/pyexpat `__ [SJB] =============== Skipped Threads =============== - `PEP 8 updates/clarifications, function/method style `__ - `Test branch for ssize_t changes `__ - `DRAFT: python-dev summary for 2005-11-16 to 2005-11-31 `__ - `[Python-checkins] commit of r41497 -python/trunk/Lib/test `__ - `fresh checkout won't build `__ - `fixing log messages `__ - `synchronized enumerate `__ - `(no subject) `__ - `Build failure and problem on Windows `__ - `os.startfile with optional second parameter `__ - `[OT] Fwd: a new python port: iPod `__ - `Sets are mappings? `__ - `Patch to make unittest.TestCase easier to subclass `__ - `timeout options in high-level networking modules `__ - `Patch reviews & request for patch review `__ - `Weekly Python Patch/Bug Summary `__ - `A few questions about setobject `__ ) - `Small any/all enhancement `__ - `floating point literals don't work in non-US locale in 2.5 `__ - `JOB OPENING: Implementor for Python and Search `__ - `PyCon TX 2006: Early-bird registration ends Dec. 31! `__ - `set.copy documentation string `__ - `Bug in Py_InitModule4 `__ - `floating point literals don't work in non-USlocale in 2.5 `__ - `slight inconsistency in svn checkin email subject lines `__ - `Gaming on Sunday (Jan 1) `__ From allison at shasta.stanford.edu Fri Jan 13 00:07:21 2006 From: allison at shasta.stanford.edu (Dennis Allison) Date: Thu, 12 Jan 2006 15:07:21 -0800 (PST) Subject: [Python-Dev] Ph.D. dissertation ideas? In-Reply-To: Message-ID: Brett, Where are you doing your Phd and who will be your likely supervisor? It does make a difference. Your dissertation idea list seems to me to focus on implementation projects and not on research. Usually a dissertation proceeds from a hypothesis leading to an experiment, some measurements, and conclusions. Many universities tacitly expect a theorem and associated proof. The goal of dissertation research is a completed approved dissertation, not some intergalactic all encompassing contribution to human knowledge. Dissertation research should, IMHO, focus on a small, manageable problem. Frequently, universities have a large cooperative project involving many graduate students, each of whom research a small, well defined topic areas related to the larger project. Is this the case for your dissertation? What are your interests? Are you interested in language structure, language implementation, the formal interface between languages and applications, or what. Like most folks who lurk on this list, I have my list of things I'd like to see someone research. Tell us a bit more and I am sure you'll get lots of us to share. -d On Thu, 12 Jan 2006, Brett Cannon wrote: > It is time once again in my educational career to come to python-dev > for help for major project ideas. This time, though, it is for my > Ph.D. dissertation (and thus can have larger scope than my masters > thesis) but there are funding restrictions (and thus only certain > areas I have possible funding for). > > First off, there are two areas where I have possible funding: XML > integration into programming languages and game scripting support (I > have a third, AOP for anything, but I don't think AOP is a good match > for Python and thus not considering it for Python work). The XML > integration I have no direct ideas since I don't use XML. There is > the possibility of doing something like the LINQ project > (http://msdn.microsoft.com/netframework/future/linq/default.aspx?pull=/library/en-us/dndotnet/html/linqprojectovw.asp) > or something, but I don't know how useful that could be. It would > have to be just as generic, though, as the LINQ project in terms of > working with intefaces to be Pythonic so it might turn out to be more > about language support for querying data. > > For the game scripting, trying out the active objects for Python is a > possibility (or something else that would help with concurrency). > Since that could allow for agent objects to be more concurrent it can > be argued that it would help with game scripting. I don't have any > other ideas for the area of game scripting support. > > Now I am not locked down to these areas, but they are what I have > possible grant money for. If through some miracle someone out there > wants to fund my Ph.D. and wants something else worked on for Python, > don't be shy. =) Unfortunately I don't think the PSF is up for > funding my Ph.D., else I would say I would work on whatever python-dev > decided they wanted to worked on that could be done as a dissertation. > > Anyway, any ideas are appreciated. There is no rush on this, either. > Just getting this out there now while I am tentatively exploring > possible topics. > > -Brett > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/allison%40sumeru.stanford.edu > -- From goodger at python.org Fri Jan 13 05:29:19 2006 From: goodger at python.org (David Goodger) Date: Thu, 12 Jan 2006 23:29:19 -0500 Subject: [Python-Dev] r42015 - peps/trunk In-Reply-To: <1f7befae0601121141m313d836etb0ba912821844bdf@mail.gmail.com> References: <20060112033316.E24C31E4002@bag.python.org> <43C5D237.3040006@python.org> <43C63D32.9070206@egenix.com> <43C65A2D.2050807@python.org> <43C677CD.3030709@egenix.com> <4335d2c40601120756l34c4a16fvf4b1d1b7b376d939@mail.gmail.com> <43C6809E.306@egenix.com> <4335d2c40601120904rd45400cy2c09bcc9d3d66197@mail.gmail.com> <43C6AF5E.7020803@egenix.com> <1f7befae0601121141m313d836etb0ba912821844bdf@mail.gmail.com> Message-ID: <43C72C9F.6040700@python.org> > [David Goodger] >> An alternative is to use svn:externals to link to a specific >> revision (via the -r option); [Tim Peters] > Yes. please. Done. -- David Goodger -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 249 bytes Desc: OpenPGP digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20060112/19055cab/attachment.pgp From aahz at pythoncraft.com Fri Jan 13 05:36:49 2006 From: aahz at pythoncraft.com (Aahz) Date: Thu, 12 Jan 2006 20:36:49 -0800 Subject: [Python-Dev] pystate.c changes for Python 2.4.2 In-Reply-To: <43C68823.3000105@corest.com> References: <43C68823.3000105@corest.com> Message-ID: <20060113043649.GA16672@panix.com> On Thu, Jan 12, 2006, Gabriel Becedillas wrote: > > If I add this code at the end of PyThreadState_Delete: > > if (autoTLSkey && PyThread_get_key_value(autoTLSkey) == tstate) > PyThread_delete_key_value(autoTLSkey); > > then everything works fine. > Could you please confirm if this is a bug ? Regardless of whether you get any other responses, please go ahead and file a SourceForge bug report so that we don't lose track of this. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From theller at python.net Fri Jan 13 07:51:20 2006 From: theller at python.net (Thomas Heller) Date: Fri, 13 Jan 2006 07:51:20 +0100 Subject: [Python-Dev] Include ctypes into core Python? References: <43C58200.20803@v.loewis.de> <43C6E0E1.5010705@v.loewis.de> Message-ID: <1wzc62on.fsf@python.net> "Martin v. L?wis" writes: > Thomas Heller wrote: >> Sounds good, although it should be noted that ctypes is a package now, >> with a ctypes.wrap subpackage (contains the code generator), a >> ctypes.macholib subpackage (contains Bob Ippolitos macholib for OS X), >> and ctypes.test subpackage whcih contains several test modules. Plus >> the _ctypes.(dll|so) extension module. > > Ok. The installer should then combine them all into a feature. > > Still, the admin could disable all of this just by removing _ctypes.dll, > right? Right. Thomas From theller at python.net Fri Jan 13 07:56:56 2006 From: theller at python.net (Thomas Heller) Date: Fri, 13 Jan 2006 07:56:56 +0100 Subject: [Python-Dev] DRAFT: python-dev Summary for 2005-12-16 through 2005-12-31 References: <6c63de570601121446p390477a7q97a1486bada53d0d@mail.gmail.com> Message-ID: Tony Meyer writes: > ---------------------------------- > Adding zlib module to python25.dll > ---------------------------------- > > Thomas Heller and Martin Heller had been discussing whether the zlib Should be 'Martin v. L?wis', as we all know. > module should become builtin, at least on Windows (i.e. part of > python25.dll). This would simplify both the build process and py2exe, > the latter could then bootstrap extraction from the compressed file > just with pythonxy.dll, but this is currently not done, because the > pythoncore.vcproj would then not be buildable anymore unless you also > have the right version of zlib on disk. To solve this problem, Thomas > proposed that the Python release could incorporate a copy of zlib, > primarily for use on Windows (with the project files appropriately > adjusted). This change was later made. > > Contributing thread: > > - `Incorporation of zlib sources into Python subversion > `__ From ianb at colorstudy.com Fri Jan 13 08:05:44 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 13 Jan 2006 01:05:44 -0600 Subject: [Python-Dev] Ph.D. dissertation ideas? In-Reply-To: References: <-8397309676539309595@unknownmsgid> Message-ID: <43C75148.1060003@colorstudy.com> Brett Cannon wrote: > Hmm. It's an idea. I also thought of Python -> JavaScript compiler > since JavaScript is not fun and getting to test using Python might be > cool. But not sure how useful that would be. Plus I bet someone has > does this with Java or something. There's a Python project that compiles to Javascript: http://www.aminus.org/blogs/index.php/phunt/2005/10/06/subway_s_new_ajax_framework However, I think this is a bad idea. The world doesn't need another compiled language with leaky abstractions that is next to impossible to debug. Anyway, to chime in about something I'd really like to see, that maybe fits in somewhere between graphics and language implementation, is the ideas of Boxer (http://dewey.soe.berkeley.edu/boxer.html/) implemented for Python. Basically meaning that some of what we represent with indentation in Python would be represented graphically (Boxer uses, unsurprisingly, boxes). Also a kind of window in the runtime of the system -- variables in Boxer aren't names, but visual elements you can see as the program runs. From what I've seen of HyperCard -- or at least what I choose to pay attention to ;) -- I think this kind of concrete runtime is part of its appeal. I have a lot of more specific ideas related to this, if the idea interests you. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From mal at egenix.com Fri Jan 13 10:59:49 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 13 Jan 2006 10:59:49 +0100 Subject: [Python-Dev] Py_ssize_t output parameters (Was: [Python-checkins] r41971 ...) In-Reply-To: <43C6D9D1.7040101@v.loewis.de> References: <20060108054820.15F321E4002@bag.python.org> <43C25CE1.4070106@egenix.com> <43C2DD44.2040908@v.loewis.de> <43C37C92.3060302@egenix.com> <43C4083F.3070001@v.loewis.de> <43C421A3.2030806@egenix.com> <43C425DE.2050001@v.loewis.de> <43C63458.10804@egenix.com> <43C6D9D1.7040101@v.loewis.de> Message-ID: <43C77A15.5050803@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg wrote: >> What if x64 has a 64-bit value ? How do you catch >> and process the truncation error ? > > We were *both* discussing a scenario where no sizes > exceed 2**31, right? Right, but I don't see the point of each and every extension having to go through these hoops when you could add support for these checks (including error reporting) to the few APIs in question, in particular the PyArg_ParseTuple() API. > Under such a scenario, this just won't happen. > > OTOH, if you were discussing a scenario where sizes > might exceed 2**31, then I don't see why you are worried > about Py_ssize_t* parameters alone: Even > > PyString_Size() > > might (of course!) return a value > 2**31 - so it > is not just output parameters, but also return values. Indeed - which is why I'm going to convert our tools to Py_ssize_t throughout. I don't expect this to happen any time soon for the ten or twenty other 3rd party extensions we regularly use and this would prevent an upgrade to Python 2.5. > For more information, please read "Conversion guidelines" > in > > http://www.python.org/peps/pep-0353.html BTW, the open issue should be reworded: In particular, functions that currently take int* output parameters should continue to do so. New functions should be revised to enable Py_ssize_t* output arguments and preseve backwards compatibility. (This also includes the strategy to be used, so you can remove the note on strategy) Please also add a comment on the fact that extensions which haven't been recompiled for the Python 2.5 API will not get imported (the API_LEVEL check should consistently fail for these instead of just issuing a warning). >>> That is not necessary. Can you give in example of a module >>> where you think it is necessary? >> >> If you want to port the extension to Py_ssize_t, this >> is essentially the case. >> >> You might want to take the unicodeobject.c file as >> example. > > unicodeobject.c is not an extension. We were discussing > existing extension modules. > >> We could use the type flags for these. much like we >> do for the new style numbers. > > Would you like to write a specification for that? Sure, if there's hope to get this into the code. >> If you don't like the macro approach, why not simply >> leave the two separate sets of APIs visible. > > To simplify the porting. Sorry, I lost you there: I'm saying that two sets of APIs (one for int*, one for Py_ssize_t*) will make porting easier since updated code will only have to rename the APIs used. Furthermore, it will allow to write code that easily works in Python 2.1-2.4 as well as Python 2.5. >> All Py_ssize_t aware and compatible extensions >> would use the new APIs instead. The old-style APIs >> should then take care of the proper down-casting and >> error reporting. > > That is not possible. Some API does not support > error reporting (like PyString_Size()). So callers > don't expect it to fail. I'm talking about the few APIs that use output parameters. Those do support error reporting. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From ncoghlan at gmail.com Fri Jan 13 10:59:58 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 13 Jan 2006 19:59:58 +1000 Subject: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py In-Reply-To: References: <87zmm6d7jp.fsf@tleepslib.sk.tsukuba.ac.jp> <20060111195900.GB17269@activestate.com> Message-ID: <43C77A1E.7010808@gmail.com> Scott David Daniels wrote: > OK I carried the code I offered earlier in this whole thread (tweaked in > reaction to some comments) over to comp.lang.python, gathered some > feedback, and put up a recipe on the cookbook. After a week or so for > more comment, I'll be happy to submit a patch to include the broken_test > decorator function in unittest. > > Here is where the recipe is, for those who want to comment further: > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/466288 Looks pretty handy to me - I've certainly experienced the "I know what the problem is and exactly how to provoke it, I just don't have time to fix it right now" problem myself. I added a minor comment about testing for whether there were user supplied exceptions at definition time, rather than run time. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From fredrik at pythonware.com Fri Jan 13 11:04:35 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Fri, 13 Jan 2006 11:04:35 +0100 Subject: [Python-Dev] DRAFT: python-dev Summary for 2005-12-16 through2005-12-31 References: <6c63de570601121446p390477a7q97a1486bada53d0d@mail.gmail.com> Message-ID: Tony Meyer wrote: > Fredrik Lundh kicked off a surprisingly long thread when he proposed > that typing "quit" or "exit" in the interactive prompt actually exit > (i.e. raises SystemError) SystemExit > >>> def exit(): > ... print "bye" > > >>> # what is it? > >>> exit > > $ oops! > > (However, one could simply type "repr(exit)" in this case). too late: >>> exit $ repr(exit) bash: syntax error near unexpected token `repr(exit)' From mwh at python.net Fri Jan 13 11:07:53 2006 From: mwh at python.net (Michael Hudson) Date: Fri, 13 Jan 2006 10:07:53 +0000 Subject: [Python-Dev] pystate.c changes for Python 2.4.2 In-Reply-To: <43C68823.3000105@corest.com> (Gabriel Becedillas's message of "Thu, 12 Jan 2006 13:47:31 -0300") References: <43C68823.3000105@corest.com> Message-ID: <2mwth4qw3q.fsf@starship.python.net> Gabriel Becedillas writes: > Hi, > At the company I work for, we've embedded Python in C++ application we > develop. Since our upgrade to Python 2.4.2 from 2.4.1 we started hitting > Py_FatalError("Invalid thread state for this thread") when using debug > builds. > We use both multiple interpreters and thread states. I know you've said this to me in email already, but I have no idea how you managed to get this to work with 2.4.1 :) > I think the problem is that PyThreadState_Delete (which is used when I > destroy thread states and interpreters) is not making the same clean up > that PyThreadState_DeleteCurrent is doing (which is used when we create > threads from python code). If a thread id is reused (obviously not > between two running threads), and the previous thread used > PyThreadState_Delete instead of PyThreadState_DeleteCurrent, then an old > and invalid value for autoTLSkey is still stored, and that is > triggering the Py_FatalError when a call to PyThreadState_Swap is made. > If I add this code at the end of PyThreadState_Delete: > > if (autoTLSkey && PyThread_get_key_value(autoTLSkey) == tstate) > PyThread_delete_key_value(autoTLSkey); > > then everything works fine. I think I begin to understand the problem... but this is still fragile: it relies on the fact that you delete a thread state from the OS level thread that created it, but while a thread belonging to a different *interpreter state* has the GIL (or at least: the interpreter state of the thread state being deleted *doesn't* hold the GIL). Can you guarantee that? It seems to me that to be robust, you need a way of saying "delete the thread local value of autoTLSKey from thread $FOO". But this is all very confusing... > Could you please confirm if this is a bug ? Yes, I think it's a bug. Cheers, mwh -- Jokes around here tend to get followed by implementations. -- from Twisted.Quotes From ncoghlan at gmail.com Fri Jan 13 11:16:12 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 13 Jan 2006 20:16:12 +1000 Subject: [Python-Dev] PEP and stdlib In-Reply-To: <43C691EC.3000503@evhr.net> References: <43C691EC.3000503@evhr.net> Message-ID: <43C77DEC.70404@gmail.com> Fabien Schwob wrote: > Hello, > > I've often read new PEP that are published, and they are often about new > additions to the core language. Why not using them with the standard > library. PEPs are used to make changes to the Standard Library - there's a bunch of them listed in PEP 0. However, a formal PEP is often not required when there is no significant dissent on c.l.p or python-dev that requires a BDFL pronouncement - in such cases, the relevant module maintainer (or python-dev collectively) may just accept the change and check it in. If there's no controversy about a change, the overhead involved in making a PEP just isn't worth it. Making changes to the core language (especially syntax changes, but also adding or significantly altering builtins) always requires a BDFL pronouncement, is almost always controversial, and hence almost always involves a full PEP. (The only syntax change I know of that Guido has approved without a PEP is to fix the minor irritation where keyword arguments can't come after a *argument. However, even that should probably be made into a full PEP so that some potential extensions to define keyword-only arguments and function default values can be debated). > Guido often say that he don't want to include new module that aren't > widely used in the community. It's a good thing, but it lead to the > creation of a lot of API incompatible modules. Why not using PEP in > order to create standard API like the Java world do with JSRs (Java > Specification Requests) ? > > What do you think about that ? Some PEP's are already about exactly such issues - there are a few listed under "Other Informational PEP's" in PEP 0. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From fredrik at pythonware.com Fri Jan 13 11:32:47 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Fri, 13 Jan 2006 11:32:47 +0100 Subject: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py References: <87zmm6d7jp.fsf@tleepslib.sk.tsukuba.ac.jp><20060111195900.GB17269@activestate.com> Message-ID: Scott David Daniels wrote: > OK I carried the code I offered earlier in this whole thread (tweaked in > reaction to some comments) over to comp.lang.python, gathered some > feedback, and put up a recipe on the cookbook. After a week or so for > more comment, I'll be happy to submit a patch to include the broken_test > decorator function in unittest. > > Here is where the recipe is, for those who want to comment further: > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/466288 my main nit is the name: the test isn't broken in itself, and doesn't need to be fixed; it's just not expected to succeed at this time. the usual term for this is "expected failure" (sometimes called XFAIL). for the developer, this means that a failure is not a regression, and is pro- bably not caused by something that the developer just did. From steve at holdenweb.com Fri Jan 13 14:16:01 2006 From: steve at holdenweb.com (Steve Holden) Date: Fri, 13 Jan 2006 13:16:01 +0000 Subject: [Python-Dev] TAR problems under Solaris In-Reply-To: <1136409821.10363.59.camel@geddy.wooz.org> References: <1136409821.10363.59.camel@geddy.wooz.org> Message-ID: Barry Warsaw wrote: > On Wed, 2006-01-04 at 22:01 +0100, Reinhold Birkenfeld wrote: > >>Recently, someone on dclpy posted about an error he got >>when he tried to unpack the Python distribution tarball >>with Sparc Solaris 9's tar: >> >>tar: directory checksum error >> >>With GNU tar, it worked correctly. >> >>Is this a known issue, or is it irrelevant? > > > Dunno, but I'm always having problems w/ Solaris tar, so I just use GNU > tar on Solaris. ;) > Yes. Back in the 1980's Sun's consulting group made good money selling an "extended" version of tar (that coped with longer paths than the standard version) to customers who could afford it. Guess they never fed the changes back into the trunk ... regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/ From kbk at shore.net Fri Jan 13 22:24:11 2006 From: kbk at shore.net (Kurt B. Kaiser) Date: Fri, 13 Jan 2006 16:24:11 -0500 (EST) Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200601132124.k0DLOBMh006701@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 384 open ( +2) / 3016 closed (+13) / 3400 total (+15) Bugs : 909 open ( +6) / 5500 closed (+21) / 6409 total (+27) RFE : 208 open ( +5) / 196 closed ( +1) / 404 total ( +6) New / Reopened Patches ______________________ add support for thread function result storage (2006-01-02) http://python.org/sf/1395552 opened by tosi-in Nit: incorrect subsection LaTeX label in cookielib docs (2006-01-03) CLOSED http://python.org/sf/1395715 opened by John J Lee Further .vcproj cleanups (2006-01-03) http://python.org/sf/1396093 opened by Adal Chiriliuc FreeBSD is system scope threads capable (2006-01-04) http://python.org/sf/1396919 opened by Valts Fix dict and set docs, re: immutability (2006-01-05) http://python.org/sf/1397711 opened by Collin Winter dictnotes.txt (2006-01-05) http://python.org/sf/1397848 opened by Jim Jewett File-iteration and read* method protection (2006-01-05) http://python.org/sf/1397960 opened by Thomas Wouters ConfigParser to save with order (2006-01-07) http://python.org/sf/1399309 opened by Facundo Batista enhance unittest to define tests that *should* fail (2006-01-08) http://python.org/sf/1399935 opened by Neal Norwitz unicode formats floats according to locale (2006-01-09) CLOSED http://python.org/sf/1400181 opened by Neal Norwitz Need warning that `dl` module can cause segfaults/crashes (2006-01-10) http://python.org/sf/1402224 opened by Tim Delaney Fix dictionary subclass semantics when used as global dicts. (2006-01-10) http://python.org/sf/1402289 opened by crutcher Suggested patch for #1403410 (2006-01-13) CLOSED http://python.org/sf/1404357 opened by Peter van Kampen Patch for #1394565 (2006-01-13) CLOSED http://python.org/sf/1404374 opened by Peter van Kampen Patches Closed ______________ dict.merge (2005-12-27) http://python.org/sf/1391204 closed by gvanrossum cookielib LWPCookieJar and MozillaCookieJar exceptions (2005-02-06) http://python.org/sf/1117398 closed by nnorwitz Nit: incorrect subsection LaTeX label in cookielib docs (2006-01-03) http://python.org/sf/1395715 closed by birkenfeld mingw compile (2004-10-25) http://python.org/sf/1053879 closed by loewis skip winsound for Windows/IA64 build (2005-03-09) http://python.org/sf/1160169 closed by loewis PCbuild vcproj project files need a cleanup (2005-09-29) http://python.org/sf/1307806 closed by loewis look in libbsd.a also for openpty and forkpty (2004-01-22) http://python.org/sf/881820 closed by loewis UTF-8-Sig codec (2005-04-05) http://python.org/sf/1177307 closed by loewis fix for distutils "upload" command (2005-09-23) http://python.org/sf/1299675 closed by loewis unicode formats floats according to locale (2006-01-09) http://python.org/sf/1400181 closed by nnorwitz Python crashes in pyexpat.c if malformed XML is parsed (2005-03-31) http://python.org/sf/1173998 closed by nnorwitz Suggested patch for #1403410 (2006-01-13) http://python.org/sf/1404357 closed by birkenfeld Patch for #1394565 (2006-01-13) http://python.org/sf/1404374 closed by birkenfeld New / Reopened Bugs ___________________ errata (2006-01-01) CLOSED http://python.org/sf/1394868 opened by Chad Whitacre os.remove should behave like rm, not unlink (2006-01-02) CLOSED http://python.org/sf/1395442 opened by Chad Whitacre Please document pyc format guarantees (2006-01-02) CLOSED http://python.org/sf/1395511 opened by Joe Wreschnig module os, very small doc inconsistency (2006-01-02) CLOSED http://python.org/sf/1395597 opened by tiissa Can't build Python on POSIX w/o $HOME (2004-05-24) http://python.org/sf/959576 reopened by birkenfeld make fails trying to run svnversion (2006-01-03) CLOSED http://python.org/sf/1395926 opened by M.-A. Lemburg TimedRotatingFileHandler at midnight rolls over at 01:00 (2006-01-03) http://python.org/sf/1396030 opened by Andrew Waters TimedRotatingFileHandler does not recover from open error (2006-01-03) http://python.org/sf/1396040 opened by Andrew Waters KeyboardInterrupt prevents return to Windows console (2006-01-03) http://python.org/sf/1396258 opened by Vernon Cole file.tell() returns illegal value on Windows (2006-01-03) http://python.org/sf/1396471 opened by Tom Goddard urlparse is confused by / (2006-01-04) http://python.org/sf/1396543 opened by John Hansen TimedRotatingFileHandler midnight rollover time increases (2006-01-04) http://python.org/sf/1396622 opened by Andrew Waters bsddb.__init__ causes error (2006-01-04) http://python.org/sf/1396678 opened by Fabian_M %ehrntDRT support for time.strptime (2006-01-04) http://python.org/sf/1396946 opened by Johan Dahlin no handler base classes in xml.sax (2006-01-04) CLOSED http://python.org/sf/1397205 opened by Alan G timeit execution enviroment (2006-01-05) http://python.org/sf/1397474 opened by rurpy libpython2.4.so get not installed (2006-01-05) http://python.org/sf/1397850 opened by hajo Example in section 5.3 "Pure Embedding" doesn't work. (2006-01-06) http://python.org/sf/1398781 opened by Bill Studenmund i get a memory leak after using split() function on windows (2006-01-07) CLOSED http://python.org/sf/1399099 opened by Leo Jay segfault in curses.panel with userptr() (2006-01-09) CLOSED http://python.org/sf/1400115 opened by Tom Quetchenbach 'is' does not work on methods (2006-01-09) CLOSED http://python.org/sf/1400227 opened by Collin Winter 'is' does not work for methods (2006-01-09) CLOSED http://python.org/sf/1400235 opened by Collin Winter Extended version of _curses over{lay,write} does not work. (2006-01-09) CLOSED http://python.org/sf/1400822 opened by J. Sipprell Win32COM policy.py unicode exceptions (2006-01-10) CLOSED http://python.org/sf/1401642 opened by rbell01824 null source chars handled oddly (2005-01-20) CLOSED http://python.org/sf/1105770 reopened by birkenfeld segfault when using mmap(-1,...) [+PATCH] (2006-01-11) http://python.org/sf/1402308 opened by Ralf Schmitt cannot import extension module with Purify (2006-01-11) http://python.org/sf/1403068 opened by Amaury Forgeot d'Arc 2.3.5 source RPM install fails w/o tk-devel (2006-01-11) http://python.org/sf/1403221 opened by Nathan Zook 2.3.5 RPM provides incompatibility (2006-01-11) http://python.org/sf/1403225 opened by Nathan Zook in email can't get attachments' filenames using get_filename (2006-01-11) http://python.org/sf/1403349 opened by Michal P. TypeError unsubscriptable object caused by warnings.py (2006-01-12) CLOSED http://python.org/sf/1403410 opened by FreyP os.getlogin() goes **poof** (2006-01-12) CLOSED http://python.org/sf/1404213 opened by Jim Wilson subprocess.Popen inside thread locks the thread in some case (2006-01-13) http://python.org/sf/1404925 opened by Toon Verstraelen Bugs Closed ___________ 'Plus' filemode exposes uninitialized memory on win32 (2005-12-31) http://python.org/sf/1394612 closed by tim_one doc errata (2006-01-01) http://python.org/sf/1394868 closed by birkenfeld os.remove should behave like rm, not unlink (2006-01-02) http://python.org/sf/1395442 deleted by whit537 Please document pyc format guarantees (2006-01-02) http://python.org/sf/1395511 closed by piman module os, very small doc inconsistency (2006-01-02) http://python.org/sf/1395597 closed by birkenfeld make fails trying to run svnversion (2006-01-03) http://python.org/sf/1395926 closed by bwarsaw no handler base classes in xml.sax (2006-01-04) http://python.org/sf/1397205 closed by birkenfeld SimpleHTTPServer doesn't understand query arguments (2005-12-31) http://python.org/sf/1394565 closed by birkenfeld Float marshaling problems with test_colorsys (2005-11-30) http://python.org/sf/1370322 closed by birkenfeld i get a memory leak after using split() function on windows (2006-01-07) http://python.org/sf/1399099 closed by perky 2.3.3 make fails build posix_openpty' (2004-01-21) http://python.org/sf/880996 closed by loewis SegFault in interactive mode when cursor is over a non-ASCII (2004-11-15) http://python.org/sf/1066545 closed by nnorwitz segfault in curses.panel with userptr() (2006-01-08) http://python.org/sf/1400115 closed by nnorwitz 2.4.1 build fails on OpenBSD 3.7 (2005-07-25) http://python.org/sf/1244610 closed by nnorwitz build fails on BSD 3.8 (2005-12-29) http://python.org/sf/1392915 closed by nnorwitz 'is' does not work on methods (2006-01-09) http://python.org/sf/1400227 closed by mwh 'is' does not work for methods (2006-01-09) http://python.org/sf/1400235 deleted by collinwinter Extended version of _curses over{lay,write} does not work. (2006-01-09) http://python.org/sf/1400822 closed by nnorwitz Win32COM policy.py unicode exceptions (2006-01-10) http://python.org/sf/1401642 closed by birkenfeld simple attribute access causing segfault in certain cases (2004-07-01) http://python.org/sf/983311 closed by birkenfeld Another dealloc stack killer (2002-08-26) http://python.org/sf/600007 closed by birkenfeld PyOS_snprintf segfaults on missing native snprintf (2004-01-05) http://python.org/sf/871026 closed by tim_one null source chars handled oddly (2005-01-20) http://python.org/sf/1105770 closed by birkenfeld TypeError unsubscriptable object caused by warnings.py (2006-01-12) http://python.org/sf/1403410 closed by birkenfeld os.getlogin() goes **poof** (2006-01-12) http://python.org/sf/1404213 closed by birkenfeld New / Reopened RFE __________________ subprocess: wait for a period of time (2006-01-04) http://python.org/sf/1396825 opened by Ragnar Ouchterlony python.org - add RFE tracker (2006-01-05) http://python.org/sf/1397806 opened by Jim Jewett Add str.partition (2006-01-08) http://python.org/sf/1399936 opened by Georg Brandl invoking an external Java program from Python script (2006-01-09) CLOSED http://python.org/sf/1400252 opened by Laure friendly quit object (2006-01-13) http://python.org/sf/1404859 opened by Jim Jewett RFE Closed __________ invoking an external Java program from Python script (2006-01-09) http://python.org/sf/1400252 closed by birkenfeld From martin at v.loewis.de Sat Jan 14 00:29:51 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 14 Jan 2006 00:29:51 +0100 Subject: [Python-Dev] Py_ssize_t output parameters (Was: [Python-checkins] r41971 ...) In-Reply-To: <43C77A15.5050803@egenix.com> References: <20060108054820.15F321E4002@bag.python.org> <43C25CE1.4070106@egenix.com> <43C2DD44.2040908@v.loewis.de> <43C37C92.3060302@egenix.com> <43C4083F.3070001@v.loewis.de> <43C421A3.2030806@egenix.com> <43C425DE.2050001@v.loewis.de> <43C63458.10804@egenix.com> <43C6D9D1.7040101@v.loewis.de> <43C77A15.5050803@egenix.com> Message-ID: <43C837EF.30903@v.loewis.de> M.-A. Lemburg wrote: > Right, but I don't see the point of each and every > extension having to go through these hoops when you > could add support for these checks (including error > reporting) to the few APIs in question, in particular > the PyArg_ParseTuple() API. I don't want to rename the existing APIs, so porters to ssize_t have an easier life (just fix the compiler warnings). As for PyArg_ParseTuple: this already works and requires no changes either way. > I don't expect this to happen any time soon for the ten or > twenty other 3rd party extensions we regularly use and this > would prevent an upgrade to Python 2.5. It shouldn't. It could sensibly prevent an upgrade to 64-bit machines, but there would be no reason not to upgrade on 32-bit machines. > In particular, functions that currently take int* output > parameters should continue to do so. New functions should be > revised to enable Py_ssize_t* output arguments and preseve > backwards compatibility. > > (This also includes the strategy to be used, so you can > remove the note on strategy) Will do. > Please also add a comment on the fact that extensions > which haven't been recompiled for the Python 2.5 API > will not get imported (the API_LEVEL check should consistently > fail for these instead of just issuing a warning). I will add it as an open issue. I disagree that this should actually be done. >>Would you like to write a specification for that? > > > Sure, if there's hope to get this into the code. I don't know. So far, you are the only one wanting this kind of change. If the change actually existed, more people might be willing to voice an opinion - whether that will be in favour of more hackishness and backwards compatibility, or against, I cannot guess. I would personally want to restrict hackishness to places that the compiler doesn't warn about. >>To simplify the porting. > > > Sorry, I lost you there: > > I'm saying that two sets of APIs (one for int*, one > for Py_ssize_t*) will make porting easier since > updated code will only have to rename the APIs > used. Furthermore, it will allow to write code > that easily works in Python 2.1-2.4 as well as > Python 2.5. Yes, but users of the new API (who would conceptually do the right thing) would need more effort, just to protect users who do nothing (which conceptually do the wrong thing). By just editing the APIs (instead of also renaming them), I simplify the porting. > I'm talking about the few APIs that use output parameters. > Those do support error reporting. Hmm. I think I could personally accept a change that uses macro tricks to do all that, i.e. keep the output parameters at int* if some macro (Py_SIZE_T_CLEAN) is not defined, and use Py_ssize_t* if it is defined. People would then get the right warnings still, but no changes would be necessary to "old" code. Regards, Martin From Scott.Daniels at Acm.Org Sat Jan 14 05:22:49 2006 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Fri, 13 Jan 2006 20:22:49 -0800 Subject: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py In-Reply-To: References: <87zmm6d7jp.fsf@tleepslib.sk.tsukuba.ac.jp><20060111195900.GB17269@activestate.com> Message-ID: Fredrik Lundh wrote: > Scott David Daniels wrote: >> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/466288 > > my main nit is the name: the test isn't broken in itself, and doesn't need > to be fixed; it's just not expected to succeed at this time. > > the usual term for this is "expected failure" (sometimes called XFAIL). Would "expect_fail", "expect_failure", "expected_fail", or "expected_failure", work for you? If so, could you rank them? I don't get anything from "xfail", and I'm not sure others will either. --Scott David Daniels Scott.Daniels at Acm.Org From ncoghlan at gmail.com Sat Jan 14 05:40:10 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 14 Jan 2006 14:40:10 +1000 Subject: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py In-Reply-To: References: <87zmm6d7jp.fsf@tleepslib.sk.tsukuba.ac.jp><20060111195900.GB17269@activestate.com> Message-ID: <43C880AA.3060608@gmail.com> Scott David Daniels wrote: > Fredrik Lundh wrote: >> Scott David Daniels wrote: >>> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/466288 >> my main nit is the name: the test isn't broken in itself, and doesn't need >> to be fixed; it's just not expected to succeed at this time. >> >> the usual term for this is "expected failure" (sometimes called XFAIL). > > Would "expect_fail", "expect_failure", "expected_fail", or > "expected_failure", > work for you? > > If so, could you rank them? I don't get anything from "xfail", and I'm > not sure others will either. I'd be happy with either "expect_fail" (as the shortest) or "expected_failure" (as the actual English term), with a slight preference for the former as being just as clear, and requiring slightly less typing. There's also the fact that unittest has a large number of test case methods that start with "failIf" or "failUnless", so the "expect_fail" term aligns nicely with those. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From fdrake at acm.org Sat Jan 14 05:51:42 2006 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Fri, 13 Jan 2006 23:51:42 -0500 Subject: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py In-Reply-To: <43C880AA.3060608@gmail.com> References: <43C880AA.3060608@gmail.com> Message-ID: <200601132351.42410.fdrake@acm.org> Scott David Daniels wrote: > Would "expect_fail", "expect_failure", "expected_fail", or > "expected_failure", work for you? None of these use the same naming convention as the other unittest object attributes. Perhaps something like failureExpected? I'd definately prefer something that reads cleanly; mirroring the exact form of the word "fail" doesn't make sense; making it readable does. -Fred -- Fred L. Drake, Jr. From arigo at tunes.org Sat Jan 14 11:14:54 2006 From: arigo at tunes.org (Armin Rigo) Date: Sat, 14 Jan 2006 11:14:54 +0100 Subject: [Python-Dev] Ph.D. dissertation ideas? In-Reply-To: References: <-8397309676539309595@unknownmsgid> Message-ID: <20060114101454.GA29087@code1.codespeak.net> Hi Brett, If by any chance PyPy continues to be funded beyond 2006, we would definitely welcome you around :-) (If our funding model doesn't change, it might be difficult for us to give you money oversea, though... just asking, just in case, would you consider moving to a European university?) PyPy contains several open language research areas that you mentioned: network-distributed language support, concurrent programming... and we even already have a Python -> JavaScript compiler :-) Making it useful is an open challange, though. A bientot, Armin From Scott.Daniels at Acm.Org Sat Jan 14 15:17:55 2006 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Sat, 14 Jan 2006 06:17:55 -0800 Subject: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py In-Reply-To: <200601132351.42410.fdrake@acm.org> References: <43C880AA.3060608@gmail.com> <200601132351.42410.fdrake@acm.org> Message-ID: Fred L. Drake, Jr. wrote: > Scott David Daniels wrote: > > Would "expect_fail", "expect_failure", "expected_fail", or > > "expected_failure", work for you? > > None of these use the same naming convention as the other unittest object > attributes. Perhaps something like failureExpected? > > I'd definately prefer something that reads cleanly; mirroring the exact form > of the word "fail" doesn't make sense; making it readable does. Hmmm.... I'd like to avoid anything looking like "failIf" or whatever (I'm afraid it will be attempted as a method), but the point about matching styles does make sense. I see my idea of XXX has inspired a huge groundswell of "no comment" yet. How about: @expectFailure('some reason for the failure') def test_whatever(self): ... Nick Coghlan (on the recipe) wrote: > While embedding the 'or' in the except clause header is cute, its also > a little subtle and obscure, and causes the check to be executed every > time the test is run, rather than when the test is defined. I wasn't striving for cute or clever there. I did want the pick up of TestCase.failureException to occur at test time, however. My reasoning is that failureException became a class variable for a reason, and test running frameworks other than unittest.main might well be stomping their own exception in there. Certainly the "failIf" methods all raise whatever exception is stored as a class variable of TestCase. Do we want to include the equivalent of find_broken_tests(module) in unittest or is leaving it in the recipe for report writers better? I suppose it wants a better name as well. findExpectedFailures(module)? I'm off to see family, I'll check back end of next week. --Scott David Daniels Scott.Daniels at Acm.Org From g.brandl-nospam at gmx.net Sun Jan 15 00:19:30 2006 From: g.brandl-nospam at gmx.net (Georg Brandl) Date: Sun, 15 Jan 2006 00:19:30 +0100 Subject: [Python-Dev] Python icon Message-ID: Hi, does Python have an official icon? Not py.ico from PC/, that's a bit ugly and does not scale. Has no designerhead ever done such a thing? Georg From jason.orendorff at gmail.com Sun Jan 15 01:17:19 2006 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Sat, 14 Jan 2006 19:17:19 -0500 Subject: [Python-Dev] Ph.D. dissertation ideas? In-Reply-To: References: Message-ID: Brett, You could create a downloadable corpus of Python source code, and maybe a web site through which people can easily browse/search it, contribute to it, and maintain it. The point would be to support language designers, tool developers, and researchers. Several python-dev folks have their own corpuses; I think other people would be happy to use a free one if it were out there. Of course there's no need to limit it to Python... Creating a really *good* corpus is maybe not super-easy; I imagine there are myriad linguistics papers explaining the nuances. Hey, cross-discipline research--cool points! Once this exists, there's no shortage of research questions you can quickly and easily answer with it. What percentage of Python programs use functional programming techniques? How often are list comprehensions used? What do people use generators for? And if you do something web-based, you can certainly work XML in there somewhere. :) -j From python-dev at zesty.ca Sun Jan 15 01:36:31 2006 From: python-dev at zesty.ca (Ka-Ping Yee) Date: Sat, 14 Jan 2006 18:36:31 -0600 (CST) Subject: [Python-Dev] Python icon In-Reply-To: References: Message-ID: On Sun, 15 Jan 2006, Georg Brandl wrote: > does Python have an official icon? Not py.ico from PC/, that's a bit > ugly and does not scale. Has no designerhead ever done such a thing? There have been a couple of proposed logos -- i found some images at http://www.pythonology.com/logos -- but i don't know what their official status is. -- ?!ng From tismer at stackless.com Sun Jan 15 02:37:47 2006 From: tismer at stackless.com (Christian Tismer) Date: Sun, 15 Jan 2006 02:37:47 +0100 Subject: [Python-Dev] os.path.getmtime on Windows Message-ID: <43C9A76B.8070801@stackless.com> Hi Python developers, today I got a complaint from the python.de IRC channel about os.path.getmtime and time zone. How to produce the weird behavior: 1. create a file 2. get it's os.path.getmtime() 3. change your time zone 4. get os.path.getmtime again compare - the time stamps are different. Change the time zone back, and they are identical, again. I was not ableto produce an identity, neither by time.gmtime nor by time.localtime, so I'm a bit confused. I checked the sources, and this is probably not a Python problem. It uses the suggested win32 function properly. But the win32 documentation seems to have no hints about this. I assumend the value would be in UTC, but it is obviously not. Is there a way to circumvent this problem, or am I missing something? If this is not the expected behavior, then it might make sense to find a patch. thanks -- chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From brett at python.org Sun Jan 15 02:51:25 2006 From: brett at python.org (Brett Cannon) Date: Sat, 14 Jan 2006 17:51:25 -0800 Subject: [Python-Dev] Ph.D. dissertation ideas? In-Reply-To: <20060114101454.GA29087@code1.codespeak.net> References: <-8397309676539309595@unknownmsgid> <20060114101454.GA29087@code1.codespeak.net> Message-ID: On 1/14/06, Armin Rigo wrote: > Hi Brett, > > If by any chance PyPy continues to be funded beyond 2006, we would > definitely welcome you around :-) (If our funding model doesn't change, > it might be difficult for us to give you money oversea, though... just > asking, just in case, would you consider moving to a European > university?) > That would be cool! I definitely would not mind working on PyPy. Unfortunately I would not consider changing universities; I really like it here. > PyPy contains several open language research areas that you mentioned: > network-distributed language support, concurrent programming... and we > even already have a Python -> JavaScript compiler :-) Making it useful > is an open challange, though. > Well, if PyPy picked up the grants tab I would work on whatever you guys wanted that could be turned into a dissertation topic. =) -Brett From pje at telecommunity.com Sun Jan 15 03:41:46 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 14 Jan 2006 21:41:46 -0500 Subject: [Python-Dev] os.path.getmtime on Windows In-Reply-To: <43C9A76B.8070801@stackless.com> Message-ID: <5.1.1.6.0.20060114214008.022459d8@mail.telecommunity.com> At 02:37 AM 1/15/2006 +0100, Christian Tismer wrote: >I assumend the value would be in UTC, but it is obviously not. > >Is there a way to circumvent this problem, or am I missing something? >If this is not the expected behavior, then it might make sense >to find a patch. Windows doesn't store UTC timestamps, at least not on older FAT filesystems and maybe not even on NTFS. Changing Python won't help. :) From tim.peters at gmail.com Sun Jan 15 04:31:36 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sat, 14 Jan 2006 22:31:36 -0500 Subject: [Python-Dev] os.path.getmtime on Windows In-Reply-To: <43C9A76B.8070801@stackless.com> References: <43C9A76B.8070801@stackless.com> Message-ID: <1f7befae0601141931u5dbd38e5nd2e24338f50776f1@mail.gmail.com> [Christian Tismer] > Hi Python developers, > > today I got a complaint from the python.de IRC channel > about os.path.getmtime and time zone. > > How to produce the weird behavior: > > 1. create a file > 2. get it's os.path.getmtime() > 3. change your time zone > 4. get os.path.getmtime again > > compare - the time stamps are different. > Change the time zone back, and they are identical, again. > > I was not ableto produce an identity, neither by time.gmtime > nor by time.localtime, so I'm a bit confused. > > I checked the sources, and this is probably not a Python > problem. It uses the suggested win32 function properly. > But the win32 documentation seems to have no hints about this. > > I assumend the value would be in UTC, but it is obviously not. See: http://www.codeproject.com/datetime/dstbugs.asp and just be grateful you didn't bump into the additional mysteries MS added around "daylight time" switches. > Is there a way to circumvent this problem, or am I missing something? > If this is not the expected behavior, then it might make sense > to find a patch. While I love the article linked to above, I've never had the will to force myself to read it all the way to the end ;-) From Martin.vonLoewis at hpi.uni-potsdam.de Sun Jan 15 08:43:02 2006 From: Martin.vonLoewis at hpi.uni-potsdam.de (=?iso-8859-1?Q?Martin_v._L=F6wis?=) Date: Sun, 15 Jan 2006 08:43:02 +0100 Subject: [Python-Dev] Installing PEPs Message-ID: <43C9FD06.6@hpi.uni-potsdam.de> I finally arranged for an automatic installation of PEPs: they will now get published in the subversion post-commit. Regards, Martin From martin at v.loewis.de Sun Jan 15 09:21:47 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 15 Jan 2006 09:21:47 +0100 Subject: [Python-Dev] os.path.getmtime on Windows In-Reply-To: <5.1.1.6.0.20060114214008.022459d8@mail.telecommunity.com> References: <5.1.1.6.0.20060114214008.022459d8@mail.telecommunity.com> Message-ID: <43CA061B.3080604@v.loewis.de> Phillip J. Eby wrote: > Windows doesn't store UTC timestamps, at least not on older FAT filesystems > and maybe not even on NTFS. Changing Python won't help. :) Windows definitely stores UTC timestamps on NTFS, in units of 100ns since Jan 1, 1601. Regards, Martin From martin at v.loewis.de Sun Jan 15 09:38:25 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 15 Jan 2006 09:38:25 +0100 Subject: [Python-Dev] os.path.getmtime on Windows In-Reply-To: <43C9A76B.8070801@stackless.com> References: <43C9A76B.8070801@stackless.com> Message-ID: <43CA0A01.4020406@v.loewis.de> Christian Tismer wrote: > Is there a way to circumvent this problem, or am I missing something? > If this is not the expected behavior, then it might make sense > to find a patch. I have meant to work on a patch for several years now. I would like to drop usage of msvcrt's stat(3), and instead implement os.stat in terms of Windows API directly. That would also have the advantage that subsecond time-stamps can be exposed. There are several issues involved in implementing such a patch, though. One is that you need to do it twice: once for Win9x, and once for NT+, because you have to use Unicode file names on one system, and ANSI file names on the other. Regards, Martin From collinw at gmail.com Sun Jan 15 12:00:53 2006 From: collinw at gmail.com (Collin Winter) Date: Sun, 15 Jan 2006 12:00:53 +0100 Subject: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py In-Reply-To: References: <87zmm6d7jp.fsf@tleepslib.sk.tsukuba.ac.jp> <20060111195900.GB17269@activestate.com> Message-ID: <43aa6ff70601150300sacfa02fvc6709964061c19af@mail.gmail.com> On 1/13/06, Fredrik Lundh wrote: > my main nit is the name: the test isn't broken in itself, and doesn't need > to be fixed; it's just not expected to succeed at this time. > > the usual term for this is "expected failure" (sometimes called XFAIL). > > for the developer, this means that a failure is not a regression, and is pro- > bably not caused by something that the developer just did. When I've implemented this kind of thing in the past, I've generally called the decorator/marker/whatever "TODO" (or some variation of caps/lowercase). Thanks, Collin Winter From skip at pobox.com Sun Jan 15 14:41:06 2006 From: skip at pobox.com (skip at pobox.com) Date: Sun, 15 Jan 2006 07:41:06 -0600 Subject: [Python-Dev] Python icon In-Reply-To: References: Message-ID: <17354.20722.560136.55434@montanaro.dyndns.org> >> does Python have an official icon? Ping> i found some images at http://www.pythonology.com/logos... It appears the yin/yang Python's on that page are being used in the new site (beta.python.org). I don't know if that makes it official or not though. Skip From tismer at stackless.com Sun Jan 15 15:13:21 2006 From: tismer at stackless.com (Christian Tismer) Date: Sun, 15 Jan 2006 15:13:21 +0100 Subject: [Python-Dev] os.path.getmtime on Windows In-Reply-To: <43CA0A01.4020406@v.loewis.de> References: <43C9A76B.8070801@stackless.com> <43CA0A01.4020406@v.loewis.de> Message-ID: <43CA5881.3020305@stackless.com> Hi Martin, >> Is there a way to circumvent this problem, or am I missing something? >> If this is not the expected behavior, then it might make sense >> to find a patch. > > I have meant to work on a patch for several years now. I would like to > drop usage of msvcrt's stat(3), and instead implement os.stat in terms > of Windows API directly. That would also have the advantage that > subsecond time-stamps can be exposed. I see! Still trying to understand the story. I'm working through the article Tim pointed us at. http://www.codeproject.com/datetime/dstbugs.asp Does it mean that msvcrt does extra magic to modify the existing correct UTC entries? And would usage of the Windows API heal this immediately, or are extra steps involved? As I understand the article, things are different when a file is stored in a FAT or NTFS drive. Do you think the provided solution is worthwhile to be adapted for Python? http://www.codeproject.com/datetime/DstBugs/DstBugs.zip > There are several issues involved in implementing such a patch, though. > One is that you need to do it twice: once for Win9x, and once for > NT+, because you have to use Unicode file names on one system, and > ANSI file names on the other. Correcting it just for NT/XP would make the majority of people happy, IMHO. cheers - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From gabriel.becedillas at corest.com Fri Jan 13 15:37:19 2006 From: gabriel.becedillas at corest.com (Gabriel Becedillas) Date: Fri, 13 Jan 2006 11:37:19 -0300 Subject: [Python-Dev] pystate.c changes for Python 2.4.2 In-Reply-To: <2mwth4qw3q.fsf@starship.python.net> References: <43C68823.3000105@corest.com> <2mwth4qw3q.fsf@starship.python.net> Message-ID: <43C7BB1F.3000904@corest.com> Michael Hudson wrote: > Gabriel Becedillas writes: > > >>Hi, >>At the company I work for, we've embedded Python in C++ application we >>develop. Since our upgrade to Python 2.4.2 from 2.4.1 we started hitting >>Py_FatalError("Invalid thread state for this thread") when using debug >>builds. >>We use both multiple interpreters and thread states. > > > I know you've said this to me in email already, but I have no idea how > you managed to get this to work with 2.4.1 :) > > >>I think the problem is that PyThreadState_Delete (which is used when I >>destroy thread states and interpreters) is not making the same clean up >>that PyThreadState_DeleteCurrent is doing (which is used when we create >>threads from python code). If a thread id is reused (obviously not >>between two running threads), and the previous thread used >>PyThreadState_Delete instead of PyThreadState_DeleteCurrent, then an old >>and invalid value for autoTLSkey is still stored, and that is >>triggering the Py_FatalError when a call to PyThreadState_Swap is made. >>If I add this code at the end of PyThreadState_Delete: >> >>if (autoTLSkey && PyThread_get_key_value(autoTLSkey) == tstate) >> PyThread_delete_key_value(autoTLSkey); >> >>then everything works fine. > > > I think I begin to understand the problem... but this is still > fragile: it relies on the fact that you delete a thread state from the > OS level thread that created it, but while a thread belonging to a > different *interpreter state* has the GIL (or at least: the > interpreter state of the thread state being deleted *doesn't* hold the > GIL). Can you guarantee that? Mmm... it doesn't have to do with a race condition or something. Let me try to show you with an example (keep in mind that this relies on the fact that at some moment the operating system gives you a thread with the same id of another thread that allready finished executing): // Initialize python. Py_Initialize(); PyEval_InitThreads(); PyThreadState* p_main = PyThreadState_Swap(NULL); PyEval_ReleaseLock(); /* Asume this block is executed in a separate thread, and that has been asigned thread id 10. It doesn't matter what the main thread (the thread that initialized Python) is doing. Lets assume its waiting for this one to finish. */ { PyEval_AcquireLock(); PyThreadState_Swap(p_main); PyThreadState* p_child = PyThreadState_New(p_main->interp); PyThreadState_Swap(p_child); PyRun_SimpleString("print 'mi vieja mula ya no es lo que era'"); PyThreadState_Clear(p_child); PyThreadState_Swap(NULL); PyThreadState_Delete(p_child); PyEval_ReleaseLock(); // The os level thread exits here. } /* When this thread state gets deleted (using PyThreadState_Delete), the autoTLSkey stored for thread id number 10 is not removed, because PyThreadState_Delete doesn't have the necesary cleanup code (check pystate.c): if (autoTLSkey && PyThread_get_key_value(autoTLSkey) == tstate) PyThread_delete_key_value(autoTLSkey); PyThreadState_DeleteCurrent behaves correctly because it does have this code. I think that this code should be added to tstate_delete_common (I've done this and everything worked just fine). If a new thread executes the same code, and that thread is assigned the same thread id, you get the Py_FatalError I'm talking about. A workaround for this would be to use PyThreadState_DeleteCurrent instead of PyThreadState_Delete. If you use the following code instead of the one above, the you have no way out to the Py_FatalError: */ { PyEval_AcquireLock(); PyThreadState* ret = Py_NewInterpreter(); PyRun_SimpleString("print 'mi vieja mula ya no es lo que era'"); Py_EndInterpreter(ret); PyThreadState_Swap(NULL); PyEval_ReleaseLock(); } /* When this interpreter gets deleted (using Py_EndInterpreter), its thread state gets deleted using PyThreadState_Delete, and you are back to the beginning of the problem. */ I hope this helps to clarify the problem. Thanks a lot and have a nice day. > It seems to me that to be robust, you need a way of saying "delete the > thread local value of autoTLSKey from thread $FOO". But this is all > very confusing... > > >>Could you please confirm if this is a bug ? > > > Yes, I think it's a bug. > > Cheers, > mwh > -- Gabriel Becedillas Developer CORE SECURITY TECHNOLOGIES Florida 141 - 2? cuerpo - 7? piso C1005AAC Buenos Aires - Argentina Tel/Fax: (54 11) 5032-CORE (2673) http://www.corest.com From martin at v.loewis.de Sun Jan 15 20:23:39 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 15 Jan 2006 20:23:39 +0100 Subject: [Python-Dev] os.path.getmtime on Windows In-Reply-To: <43CA5881.3020305@stackless.com> References: <43C9A76B.8070801@stackless.com> <43CA0A01.4020406@v.loewis.de> <43CA5881.3020305@stackless.com> Message-ID: <43CAA13B.2050902@v.loewis.de> Christian Tismer wrote: > Does it mean that msvcrt does extra magic to modify the existing > correct UTC entries? Mostly, yes. For FAT, the system does also some conversion. Those conversions I don't fully understand, http://msdn.microsoft.com/library/default.asp?url=/library/en-us/sysinfo/base/file_times.asp suggests that there are "cached UTC times" on FAT somewhere. > And would usage of the Windows API heal this > immediately, or are extra steps involved? If we use GetFileTime, apparently yes. FindFirstFile apparently also suffers from daylight conversion bugs when reading time stamps from FAT. > As I understand the article, things are different when a file is > stored in a FAT or NTFS drive. Correct. > Do you think the provided solution is worthwhile to be adapted > for Python? > http://www.codeproject.com/datetime/DstBugs/DstBugs.zip No. If anything is done about this, it should be to drop msvcrt. >> There are several issues involved in implementing such a patch, though. >> One is that you need to do it twice: once for Win9x, and once for >> NT+, because you have to use Unicode file names on one system, and >> ANSI file names on the other. > > > Correcting it just for NT/XP would make the majority of people > happy, IMHO. Right - the question would be whether completely breaking W9x support in the process would be acceptable. We use the very same binaries for W9x and NT. Regards, Martin From martin at v.loewis.de Sun Jan 15 20:45:32 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 15 Jan 2006 20:45:32 +0100 Subject: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py In-Reply-To: <43aa6ff70601150300sacfa02fvc6709964061c19af@mail.gmail.com> References: <87zmm6d7jp.fsf@tleepslib.sk.tsukuba.ac.jp> <20060111195900.GB17269@activestate.com> <43aa6ff70601150300sacfa02fvc6709964061c19af@mail.gmail.com> Message-ID: <43CAA65C.8060005@v.loewis.de> Collin Winter wrote: > When I've implemented this kind of thing in the past, I've generally > called the decorator/marker/whatever "TODO" (or some variation of > caps/lowercase). I usually call things TODO if they need to be done. The test case is not "TODO", since it is already done. "TODO" would be the place in the code that needs to change; for an expected failure, you might not even know where that code is. Regards, Martin From 2005a at usenet.alexanderweb.de Sun Jan 15 22:18:02 2006 From: 2005a at usenet.alexanderweb.de (Alexander Schremmer) Date: Sun, 15 Jan 2006 22:18:02 +0100 Subject: [Python-Dev] os.path.getmtime on Windows References: <43C9A76B.8070801@stackless.com> <43CA0A01.4020406@v.loewis.de> <43CA5881.3020305@stackless.com> <43CAA13B.2050902@v.loewis.de> Message-ID: On Sun, 15 Jan 2006 20:23:39 +0100, "Martin v. L?wis" wrote: >>> There are several issues involved in implementing such a patch, though. >>> One is that you need to do it twice: once for Win9x, and once for >>> NT+, because you have to use Unicode file names on one system, and >>> ANSI file names on the other. > Right - the question would be whether completely breaking W9x support > in the process would be acceptable. We use the very same binaries > for W9x and NT. How about accessing/calling the wide char versions just if the OS is NT, i.e. compiling both versions into the dll and just using one? Looking at the file object, the open function uses "_wfopen" which needs Windows NT according to the MSDN lib. So, how is 9x compat ensured here? Kind regards, Alexander From martin at v.loewis.de Sun Jan 15 22:32:01 2006 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 15 Jan 2006 22:32:01 +0100 Subject: [Python-Dev] os.path.getmtime on Windows In-Reply-To: References: <43C9A76B.8070801@stackless.com> <43CA0A01.4020406@v.loewis.de> <43CA5881.3020305@stackless.com> <43CAA13B.2050902@v.loewis.de> Message-ID: <43CABF51.4020108@v.loewis.de> Alexander Schremmer wrote: >>>>There are several issues involved in implementing such a patch, though. >>>>One is that you need to do it twice: once for Win9x, and once for >>>>NT+, because you have to use Unicode file names on one system, and >>>>ANSI file names on the other. > > >>Right - the question would be whether completely breaking W9x support >>in the process would be acceptable. We use the very same binaries >>for W9x and NT. > > > How about accessing/calling the wide char versions just if the OS is NT, > i.e. compiling both versions into the dll and just using one? Right. That's what I meant when I said: "you need to do it twice". > Looking at the file object, the open function uses "_wfopen" which needs > Windows NT according to the MSDN lib. So, how is 9x compat ensured here? There is a GetVersion check in there; not sure how effective it is, though. However, if you pass byte strings, as the file name, _wfopen is not used. Regards, Martin From martin at v.loewis.de Mon Jan 16 01:06:16 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 16 Jan 2006 01:06:16 +0100 Subject: [Python-Dev] os.path.getmtime on Windows In-Reply-To: <43C9A76B.8070801@stackless.com> References: <43C9A76B.8070801@stackless.com> Message-ID: <43CAE378.6050901@v.loewis.de> Christian Tismer wrote: > 1. create a file > 2. get it's os.path.getmtime() > 3. change your time zone > 4. get os.path.getmtime again > > compare - the time stamps are different. > Change the time zone back, and they are identical, again. Just to add an important detail here: I assume you did not exit Python between step 2 and step 4, right? (please confirm whether this is the case). If so, it has nothing to do with daylight saving time. If I start a new Python interpreter and fetch the mtime again, it will report the value I got at step 2. This is on NTFS, so the time stamps the system reports (in FindFirstFile) are true UTC. What happens is apparently this: msvcrt converts the UTC time to local time, using FileTimeToLocalFileTime; this gets the new time zone offset from the system. It then converts it back through __loctotime_t. This invokes __tzset, however, __tzset caches the _timezone offset, so it is unaffected by the time zone change. So *this* specific problem goes away as soon as we start dropping msvcrt. Regards, Martin P.S. The other problem (FindFirstFile gives bad UTC time stamps on FAT, for files created in a different DST period) has no real solution, AFAICT. There is no portable way to determine whether the underlying file system stores time stamps in local time or in UTC, and even if you know that file stamps were in local time, you still couldn't reliably convert that to UTC (because of the repeated hour, and because of potential time zone changes). So I would rather return to the user what the system gives me, knowing that a) the problem exists on FAT only, and people should use NTFS if they care about time stamps b) it is better to document the limitation instead of trying to work around it: msvcrt shows that attempts to work around it make the problem worse, not better. So I would like to use FindFirstFile for stat(), and GetFileType+GetFileInformationByHandle for fstat(). From tim.peters at gmail.com Mon Jan 16 03:46:19 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 15 Jan 2006 21:46:19 -0500 Subject: [Python-Dev] Birkenfeld's gone In-Reply-To: References: Message-ID: <1f7befae0601151846h57bbdfa0s2609bf0d2bf06f44@mail.gmail.com> [Georg Brandl -- or so he claimed on January 8] > today, when two Python developers here had approached me about the PSF, I > realized that it is time to correct a mistake which I had made over three years > ago, when I discovered Linux, free software, Usenet etc (I was sixteen at that time). > > I then assumed a different name, partly to anonymize myself as others had advised. > When I came across Python, I instantly liked it, and since I had interest in > Open Source development in general, I wanted to try and contribute to the > development. > > And, as a matter of course, I used my different name for that. When I realized that > I liked the community and the development of Python very much, I decided to > "unveil" myself, but I could never overcome myself -- till now. > > I'm really sorry, and I hope you could at least partly understand what caused > me to act so irrational. It doesn't really matter, Georg: by default, Python compares by object identity, not name. If _you_ haven't changed, your name doesn't matter. Still, it takes real courage to reveal one's true identity, and you're to be commended for it. If it will make you feel better, "Aahz" was born Bartholomew Fortescue Chillington the Third, and I was christened Griselda Dolores Bignose. Curiously, I'm not sure Guido even remembers his true name (but, obviously, nobody would name their kid "Guido"). I hope that revealing my true name too, and outing Aahz, will inspire everyone to reveal their secret powers and home planets too. I won't be the first, though -- as you've discovered, when you're the first you get ridiculed ;-). From aleaxit at gmail.com Mon Jan 16 05:20:10 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Sun, 15 Jan 2006 20:20:10 -0800 Subject: [Python-Dev] Birkenfeld's gone In-Reply-To: <1f7befae0601151846h57bbdfa0s2609bf0d2bf06f44@mail.gmail.com> References: <1f7befae0601151846h57bbdfa0s2609bf0d2bf06f44@mail.gmail.com> Message-ID: <5C666F20-EB46-401F-B685-299BE5FD9ED7@gmail.com> On Jan 15, 2006, at 6:46 PM, Tim Peters wrote: ... > I hope that revealing my true name too, and outing Aahz, will inspire > everyone to reveal their secret powers and home planets too. I won't OK, there I go -- "Alessandro Ferruccio Raffaele Martelli-Fortini". Good thing it's spelled in full only in one obscure church register and an even obscurer "Italian Register of Gentry", because I even feel it an imposition to have to spell out my legal first name "Alessandro" in some documents (it's that way in my passport, alas!) rather than the "Alex" I prefer... besides, no American ever spelled "Alessandro" right, I think they'd lose US citizenship if they did, or something, so I wanna stick with "Alex", which even THEY hardly ever manage to mis-spell (so they take their revenge by spelling my surname "Martinelli", like a popular local brand of [NON-alcoholic, yeck] cider... can't win, can't break even...). Unfortunately none of my secret powers has to do with forcing Americans to spell correctly (indeed, that's rather my secret _weakness_ -- just seeing "its" and "it's" freely exchanged for each other instantly saps my will to live!)... Alex From allison at shasta.stanford.edu Mon Jan 16 05:29:12 2006 From: allison at shasta.stanford.edu (Dennis Allison) Date: Sun, 15 Jan 2006 20:29:12 -0800 (PST) Subject: [Python-Dev] Ph.D. dissertation ideas? In-Reply-To: Message-ID: On Thu, 12 Jan 2006, Brett Cannon wrote: > It is time once again in my educational career to come to python-dev > for help for major project ideas. This time, though, it is for my > Ph.D. dissertation (and thus can have larger scope than my masters > thesis) but there are funding restrictions (and thus only certain > areas I have possible funding for). > > First off, there are two areas where I have possible funding: XML > integration into programming languages and game scripting support (I > have a third, AOP for anything, but I don't think AOP is a good match > for Python and thus not considering it for Python work). The XML > integration I have no direct ideas since I don't use XML. There is > the possibility of doing something like the LINQ project > (http://msdn.microsoft.com/netframework/future/linq/default.aspx?pull=/library/en-us/dndotnet/html/linqprojectovw.asp) > or something, but I don't know how useful that could be. It would > have to be just as generic, though, as the LINQ project in terms of > working with intefaces to be Pythonic so it might turn out to be more > about language support for querying data. > > For the game scripting, trying out the active objects for Python is a > possibility (or something else that would help with concurrency). > Since that could allow for agent objects to be more concurrent it can > be argued that it would help with game scripting. I don't have any > other ideas for the area of game scripting support. Brett-- Financial considerations aside, neither of the ideas you mention see ready to be a dissertation topic. I'd encourge you to look to other topics. You might research interpreter structures for languages like Python which do not use a global interpreter lock (GIL). There's been discussion about this from time to time on this list, even some suggestions as to what might be done. Given the trend toward multi-core processors, Python would benefit if all processors did not need to block on the GIL. Guido told me once that someone had build a Python interpreter with multiple locks, but that the peformance was not very good and that the interpreter was fragile. This is, of course, a hard problem. In 1974, Dijkstra wrote a fascinating paper on self-stabalizing algorithms; since that time a considerable literature has grown up. Managing a Python system these days is a management problem when the standard system is extended with libraries from many different sources, each with different release schedules, dependencies, compatibilities, and documentation. Automating the management of Python and its libraries might make a good dissertation topic, particualarly if there is some advice giving system that helps find an appropriate library for a particular problem. Testing, optimization, refactopring tools, etc. Dave Altel, following others, suggested in a provacatively title talks-- From brett at python.org Mon Jan 16 05:54:44 2006 From: brett at python.org (Brett Cannon) Date: Sun, 15 Jan 2006 20:54:44 -0800 Subject: [Python-Dev] Ph.D. dissertation ideas? In-Reply-To: References: Message-ID: On 1/15/06, Dennis Allison wrote: > On Thu, 12 Jan 2006, Brett Cannon wrote: > > > It is time once again in my educational career to come to python-dev > > for help for major project ideas. This time, though, it is for my > > Ph.D. dissertation (and thus can have larger scope than my masters > > thesis) but there are funding restrictions (and thus only certain > > areas I have possible funding for). > > > > First off, there are two areas where I have possible funding: XML > > integration into programming languages and game scripting support (I > > have a third, AOP for anything, but I don't think AOP is a good match > > for Python and thus not considering it for Python work). The XML > > integration I have no direct ideas since I don't use XML. There is > > the possibility of doing something like the LINQ project > > (http://msdn.microsoft.com/netframework/future/linq/default.aspx?pull=/library/en-us/dndotnet/html/linqprojectovw.asp) > > or something, but I don't know how useful that could be. It would > > have to be just as generic, though, as the LINQ project in terms of > > working with intefaces to be Pythonic so it might turn out to be more > > about language support for querying data. > > > > For the game scripting, trying out the active objects for Python is a > > possibility (or something else that would help with concurrency). > > Since that could allow for agent objects to be more concurrent it can > > be argued that it would help with game scripting. I don't have any > > other ideas for the area of game scripting support. > > Brett-- > > Financial considerations aside, neither of the ideas you mention see ready > to be a dissertation topic. I'd encourge you to look to other topics. > I'm open to other topics and in no way settled or locked into anything mentioned so far. Just need to be able to feed myself. =) > You might research interpreter structures for languages like Python which > do not use a global interpreter lock (GIL). There's been discussion about > this from time to time on this list, even some suggestions as to what > might be done. Given the trend toward multi-core processors, Python would > benefit if all processors did not need to block on the GIL. Guido told me > once that someone had build a Python interpreter with multiple locks, but > that the peformance was not very good and that the interpreter was > fragile. This is, of course, a hard problem. > It was Greg Stein who worked on it while at Microsoft. He didn't start to approach breaking even until two processors (if I remember correctly). None of the code was ever released. This could fit under the concurrency or gaming funding if there is some way to make threading just work better for Python. So it's a possibility. > In 1974, Dijkstra wrote a fascinating paper on self-stabalizing > algorithms; since that time a considerable literature has grown up. > > Managing a Python system these days is a management problem when the > standard system is extended with libraries from many different sources, > each with different release schedules, dependencies, compatibilities, and > documentation. Automating the management of Python and its libraries > might make a good dissertation topic, particualarly if there is some > advice giving system that helps find an appropriate library for a > particular problem. > Possibilty. Does tie into my lab's area of focus. > Testing, optimization, refactopring tools, etc. > All possible general areas. > Dave Altel, following others, suggested in a provacatively title talks-- > Was there more to this line? Seems like the email was cut off. Thanks Dennis (and everyone, for that matter) for the continued help and support! -Brett From tinuviel at sparcs.kaist.ac.kr Mon Jan 16 05:37:18 2006 From: tinuviel at sparcs.kaist.ac.kr (Seo Sanghyeon) Date: Mon, 16 Jan 2006 13:37:18 +0900 Subject: [Python-Dev] PEP 247 and hashlib Message-ID: <20060116043718.GA1129@sparcs.kaist.ac.kr> "PEP 247 -- API for Cryptographic Hash Functions" specifies a standard API for hashing modules. new([string]) ... the optional 'string' parameter, if supplied, will be immediately hashed into the object's starting state, as if obj.update(string) was called. But hashlib.new() takes the algorithm name... Does PEP need an update? By the way, I am thinking about mapping .NET's System.Security.Cryptography.HashAlgorithm.Create and created object's ComputeHash method to Python's hash API for IronPython. For that, I'd like to see a clarification. Seo Sanghyeon From aleaxit at gmail.com Mon Jan 16 06:59:20 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Sun, 15 Jan 2006 21:59:20 -0800 Subject: [Python-Dev] basenumber redux Message-ID: For the last 2+ years I've been occasionally arguing for the introduction of a basenumber (and ideally a baseinteger, but that, to me, is a slightly lesser issue) analogous to basestring. Google search fo [basenumber site:python.org] for several messages on the subject, by me and others; it will also find the recent thread about more general abstract baseclasses, which seems to have bogged down on such issues as whether sets are mappings. Now, today, I have _again_ been bit by the lack of basenumber (by a bug of mine, fixed by adding decimal.Decimal to a long tuple of classes to be passed to an isinstance call -- I hadn't run that particular numeric code of mine since the time of Python 2.3, apparently), so I'm back to pining for it. The previous discussion was short but pretty exhaustive, so I'd ask further discussants to refer back to it, rather than repeating it; no blocking issue appears to have emerged at that time, plenty of use cases were pointed out, etc. Can we PLEASE have basenumber (and maybe baseinteger, so sequences can typecheck against that for their indices -- that's the key usecase of baseinteger) rather than have them "hijacked" by wider consideration of basesequence, basemapping, and so on...? Pretty please....? Let's be pragmatic: basenumber isn't at all complicated nor controversial, baseinteger hardly at all, so let's accept them while pondering on other potential base* classes for as long as it takes for the dust to settle.... I'll be happy to draft a PEP if needed (and just as happy to eventually provide an implementation patch if the PEP's accepted), but wanted to doublecheck on the general issue first! Alex From martin at v.loewis.de Mon Jan 16 07:39:41 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 16 Jan 2006 07:39:41 +0100 Subject: [Python-Dev] basenumber redux In-Reply-To: References: Message-ID: <43CB3FAD.2090602@v.loewis.de> Alex Martelli wrote: > I'll be happy to draft a PEP if needed (and just as happy to > eventually provide an implementation patch if the PEP's accepted), > but wanted to doublecheck on the general issue first! Please do so. I've browsed somewhat through past discussions, but wasn't able to find a proposed specification of basenumber. Also, I only found half of a rationale for it: it is meant to be used along with isinstance, but I couldn't find out why you wanted to do that. In http://mail.python.org/pipermail/python-dev/2003-November/039916.html you explain that you wanted to multiply all items of self with other if other is a number; why couldn't this be written as def __mul__(self, other): try: return self.__class__([ x*other for x in self.items ]) except TypeError: # various other multiplication cases You give performance as the rationale; this is unconvincing as it would rather indicate that performance of exceptions should be improved (also, I think it is unpythonic to change the language for performance reasons, except in really common cases). Also, in that example, I wonder if the use of multiplication is flawed. If you have so many multiplication cases, perhaps you abuse the notion of multiplication? Users will need to understand the different cases, as well, and they will be surprised when it works in one case, but not in a (seemingly similar) othercase. Regards, Martin From ncoghlan at gmail.com Mon Jan 16 10:05:24 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 16 Jan 2006 19:05:24 +1000 Subject: [Python-Dev] basenumber redux In-Reply-To: References: Message-ID: <43CB61D4.4000908@gmail.com> Alex Martelli wrote: > I'll be happy to draft a PEP if needed (and just as happy to > eventually provide an implementation patch if the PEP's accepted), > but wanted to doublecheck on the general issue first! I haven't really followed the earlier basenumber discussions (aside from the sidetrack into the nature of mappings), but why would we want this ability as a typecheck and not some form of interface check? For example, what's wrong with "hasattr(x, __int__)"? That works for all the builtin types, and, IMO, anyone defining a direct conversion to an integer for a non-numeric type deserves whatever happens to them. Something like: def is_number(x): return hasattr(x, '__int__') def is_integer(x): return x == int(x) Requiring inheritance from "basenumber" in order to make something behave like a real number seems antithetical to both duck-typing and the adaptation PEP. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From anthony at interlink.com.au Mon Jan 16 10:53:06 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon, 16 Jan 2006 20:53:06 +1100 Subject: [Python-Dev] basenumber redux In-Reply-To: <43CB61D4.4000908@gmail.com> References: <43CB61D4.4000908@gmail.com> Message-ID: <200601162053.08253.anthony@interlink.com.au> On Monday 16 January 2006 20:05, Nick Coghlan wrote: > For example, what's wrong with "hasattr(x, __int__)"? That works > for all the builtin types, and, IMO, anyone defining a direct > conversion to an integer for a non-numeric type deserves whatever > happens to them. What about something that's got something like: def __int__(self): raise TypeError("This type is not a number!") I don't see a problem with defining basenumber. For the use cases, pretty much the same set as basesstring. -- Anthony Baxter It's never too late to have a happy childhood. From theller at python.net Mon Jan 16 10:53:35 2006 From: theller at python.net (Thomas Heller) Date: Mon, 16 Jan 2006 10:53:35 +0100 Subject: [Python-Dev] Include ctypes into core Python? References: <43C58200.20803@v.loewis.de> Message-ID: "Martin v. L?wis" writes: > Guido van Rossum wrote: >> On the other hand it breaks one of the most fundamental Python >> guidelines: if you get a core dump (segfault etc.) it's a bug in >> Python or in a 3rd party extension, not in *your* Python code. An >> exception would have to be made for any code that uses ctypes, as it >> is usually trivial to cause core dumps with ctypes (I'd venture it's >> hard to avoid them ;-). >> >> I don't expect this to count against including ctypes; but I do want >> it to be dealt with somehow! > > I think the strongest point *for* ctypes is the inclusion of dl > in the source distribution, which has the very same flaws as > ctypes in terms of crashability. > > So as for dealing with it "somehow": I would make ctypes a dynamically > loaded module (ctypes.pyd), so administrators could remove it, and > I could also make it a separate option in the Windows installer, > so administrators could reject to install it. It looks like we need a pronouncement now. Thomas From jim at zope.com Mon Jan 16 12:51:27 2006 From: jim at zope.com (Jim Fulton) Date: Mon, 16 Jan 2006 06:51:27 -0500 Subject: [Python-Dev] Names matter. In-Reply-To: <1f7befae0601151846h57bbdfa0s2609bf0d2bf06f44@mail.gmail.com> References: <1f7befae0601151846h57bbdfa0s2609bf0d2bf06f44@mail.gmail.com> Message-ID: <43CB88BF.4080501@zope.com> Tim Peters wrote: > [Georg Brandl -- or so he claimed on January 8] > >>today, when two Python developers here had approached me about the PSF, I >>realized that it is time to correct a mistake which I had made over three years >>ago, when I discovered Linux, free software, Usenet etc (I was sixteen at that time). >> >>I then assumed a different name, partly to anonymize myself as others had advised. >>When I came across Python, I instantly liked it, and since I had interest in >>Open Source development in general, I wanted to try and contribute to the >>development. >> >>And, as a matter of course, I used my different name for that. When I realized that >>I liked the community and the development of Python very much, I decided to >>"unveil" myself, but I could never overcome myself -- till now. >> >>I'm really sorry, and I hope you could at least partly understand what caused >>me to act so irrational. > > > It doesn't really matter, Georg: by default, Python compares by > object identity, not name. If _you_ haven't changed, your name > doesn't matter. I realize that folks are, rightly, trying to encourage Georg. I think protection of the identity of a minor on the Internet is understandable and justifiable. In general though, for adults, truthfulness and non-anonymity *do* matter. At least they matter to me, a *lot*. I don't think members of the PSF should be allowed to hide their identity and certainly, it should not be acceptable to contribute to Python under a false name. Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From rganesan at myrealbox.com Mon Jan 16 12:59:50 2006 From: rganesan at myrealbox.com (Ganesan Rajagopal) Date: Mon, 16 Jan 2006 17:29:50 +0530 Subject: [Python-Dev] Include ctypes into core Python? References: <43C58200.20803@v.loewis.de> Message-ID: >>>>> Thomas Heller writes: > It looks like we need a pronouncement now. +1 I am a lurker in this list. I maintain ctypes for Debian and I would love to see it in core python. Debian now includes ctypes for 10 Linux architectures and kFreeBSD. The only missing significant architecture is ARM because of lack of FFI support. Ganesan -- Ganesan Rajagopal (rganesan at debian.org) | GPG Key: 1024D/5D8C12EA Web: http://employees.org/~rganesan | http://rganesan.blogspot.com From connellybarnes at yahoo.com Mon Jan 16 06:27:23 2006 From: connellybarnes at yahoo.com (Connelly Barnes) Date: Sun, 15 Jan 2006 21:27:23 -0800 (PST) Subject: [Python-Dev] timeit module Message-ID: <20060116052723.31287.qmail@web54314.mail.yahoo.com> Hi, Perhaps I am the only one bothered by the timeit module, but it seems poorly designed to me. First of all, it should use a geometric series with a timeout value to detect how many iterations it should perform. Currently, the user is required to manually specify the number of iterations (the default is 1 million). If the user optimizes his or her code, then the number of iterations must be changed. If the user moves to a slower or faster computer, then the number of iterations must be changed again. This is annoying. Secondly, there should be a way to time a callable directly. That is, without finding the string name of the callable and using a string "import X" statement. These contortions violate rules #1 and #3 of the Zen of Python. Yes, there is function call overhead in timing a callable as opposed to a fragment of code. However, when I'm benchmarking code I am often timing functions, so it seems logical to make this easy to do. I have three (mutually exclusive) ideas for improving the current situation: * Add a time_callable() or time_func() function to the timeit module. These would take the callable, args, kwargs, and maximum time* to spend timing the function as arguments. * Add a FuncTimer class (function/callable timer) to the existing timeit module. The constructor takes a callable, args, kwargs as arguments. The class has instance methods timeit(max_time=4.0) and repeat(repeat=3, max_time=4.0). * Create a new timeit module (e.g. timeit2) which uses geometric series to implement both code fragment timing (which timeit does) and callable timing. I have a simple implementation of geometric series timing routines for callables and code fragments. This is recipe 440657. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440657 This recipe leaves the GC enabled. Clearly, this is simple to implement. If python-dev likes the idea of improving Python's timeit functionality, I volunteer to implement whichever interface is decided upon. Opinions? Thanks, Connelly Barnes * The maximum time will be exceeded only if a single call to the callable takes more than max_time seconds. __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From tismer at stackless.com Mon Jan 16 13:56:13 2006 From: tismer at stackless.com (Christian Tismer) Date: Mon, 16 Jan 2006 13:56:13 +0100 Subject: [Python-Dev] os.path.getmtime on Windows In-Reply-To: <43CAE378.6050901@v.loewis.de> References: <43C9A76B.8070801@stackless.com> <43CAE378.6050901@v.loewis.de> Message-ID: <43CB97ED.4020307@stackless.com> Martin v. L?wis wrote: > Christian Tismer wrote: >> 1. create a file >> 2. get it's os.path.getmtime() >> 3. change your time zone >> 4. get os.path.getmtime again >> >> compare - the time stamps are different. >> Change the time zone back, and they are identical, again. > > Just to add an important detail here: I assume > you did not exit Python between step 2 and step 4, right? > (please confirm whether this is the case). No, I did not exit Python. The problem only appeared when I changed time zone. Changing the date had no effect. -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From steve at holdenweb.com Mon Jan 16 13:57:51 2006 From: steve at holdenweb.com (Steve Holden) Date: Mon, 16 Jan 2006 12:57:51 +0000 Subject: [Python-Dev] Names matter. In-Reply-To: <43CB88BF.4080501@zope.com> References: <1f7befae0601151846h57bbdfa0s2609bf0d2bf06f44@mail.gmail.com> <43CB88BF.4080501@zope.com> Message-ID: Jim Fulton wrote: > Tim Peters wrote: > >>[Georg Brandl -- or so he claimed on January 8] >> >> >>>today, when two Python developers here had approached me about the PSF, I >>>realized that it is time to correct a mistake which I had made over three years >>>ago, when I discovered Linux, free software, Usenet etc (I was sixteen at that time). >>> >>>I then assumed a different name, partly to anonymize myself as others had advised. >>>When I came across Python, I instantly liked it, and since I had interest in >>>Open Source development in general, I wanted to try and contribute to the >>>development. >>> >>>And, as a matter of course, I used my different name for that. When I realized that >>>I liked the community and the development of Python very much, I decided to >>>"unveil" myself, but I could never overcome myself -- till now. >>> >>>I'm really sorry, and I hope you could at least partly understand what caused >>>me to act so irrational. >> >> >>It doesn't really matter, Georg: by default, Python compares by >>object identity, not name. If _you_ haven't changed, your name >>doesn't matter. > > > I realize that folks are, rightly, trying to encourage Georg. I think > protection of the identity of a minor on the Internet is understandable > and justifiable. > > In general though, for adults, truthfulness and non-anonymity *do* > matter. At least they matter to me, a *lot*. I don't think members > of the PSF should be allowed to hide their identity and certainly, > it should not be acceptable to contribute to Python under a false name. > In principle I think you are correct. In practice it's difficult to see how to nsure this without insistence on some digital certificate from an acceptable CA - by which I mean one that actually checks a driving license or passport before asserting a user's identity. Both CACert and Thawte have web-of-trust-like schemes to support this. In the meantine what do we do? Ask for a signed statement that contributors are contributing under their own identities? regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/ From steve at holdenweb.com Mon Jan 16 14:05:18 2006 From: steve at holdenweb.com (Steve Holden) Date: Mon, 16 Jan 2006 13:05:18 +0000 Subject: [Python-Dev] timeit module In-Reply-To: <20060116052723.31287.qmail@web54314.mail.yahoo.com> References: <20060116052723.31287.qmail@web54314.mail.yahoo.com> Message-ID: Connelly Barnes wrote: > Hi, > > Perhaps I am the only one bothered by the timeit > module, but it seems poorly designed to me. > > First of all, it should use a geometric series with a > timeout value to detect how many iterations it should > perform. Currently, the user is required to manually > specify the number of iterations (the default is 1 The provision of a default value generally seems to chop the legs from your argument that the number of iterations is required. > million). If the user optimizes his or her code, then > the number of iterations must be changed. If the user > moves to a slower or faster computer, then the number > of iterations must be changed again. This is > annoying. > What? The purpose of timeit is to give an approximation to the length of time to run a specific piece ofcode. Why must the number of iterations be changed when moving to a slower or faster computer? > Secondly, there should be a way to time a callable > directly. That is, without finding the string name of > the callable and using a string "import X" statement. > These contortions violate rules #1 and #3 of the Zen > of Python. > Presumably for benchmarking purposes the function call overhead would be present for all compaered techniques. Do you mean rules #0 and #2? > [...] regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/ From amk at amk.ca Mon Jan 16 15:32:29 2006 From: amk at amk.ca (A.M. Kuchling) Date: Mon, 16 Jan 2006 09:32:29 -0500 Subject: [Python-Dev] PEP 247 and hashlib In-Reply-To: <20060116043718.GA1129@sparcs.kaist.ac.kr> References: <20060116043718.GA1129@sparcs.kaist.ac.kr> Message-ID: <20060116143229.GB12888@rogue.amk.ca> On Mon, Jan 16, 2006 at 01:37:18PM +0900, Seo Sanghyeon wrote: > But hashlib.new() takes the algorithm name... Does PEP need an update? The new() function in hashlib's interface doesn't conform to PEP 247; that's all. I don't think the PEP needs to be updated. --amk From guido at python.org Mon Jan 16 16:45:55 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 16 Jan 2006 07:45:55 -0800 Subject: [Python-Dev] Names matter. In-Reply-To: References: <1f7befae0601151846h57bbdfa0s2609bf0d2bf06f44@mail.gmail.com> <43CB88BF.4080501@zope.com> Message-ID: On 1/16/06, Steve Holden wrote: > Jim Fulton wrote: > > In general though, for adults, truthfulness and non-anonymity *do* > > matter. At least they matter to me, a *lot*. I don't think members > > of the PSF should be allowed to hide their identity and certainly, > > it should not be acceptable to contribute to Python under a false name. Agreed. Also, I find it difficult to exchange emails with someone who is known by an alias only. > In principle I think you are correct. > > In practice it's difficult to see how to nsure this without insistence > on some digital certificate from an acceptable CA - by which I mean one > that actually checks a driving license or passport before asserting a > user's identity. Both CACert and Thawte have web-of-trust-like schemes > to support this. > > In the meantine what do we do? Ask for a signed statement that > contributors are contributing under their own identities? I think that's unnecessary, and not any more effective than what we already do -- ask for a signed contributor form. Someone who signs that form "Tim Peters" won't be stopped by a clause asking them to use their real name, *really*. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Jan 16 16:48:34 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 16 Jan 2006 07:48:34 -0800 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: References: <43C58200.20803@v.loewis.de> Message-ID: On 1/16/06, Thomas Heller wrote: > It looks like we need a pronouncement now. Sorry. It appeared to me that there was general agreement to using a strongly worded warning in the docs, so I tuned out of the rest of the discussion. So for the record, I'm fine with that. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Jan 16 16:53:20 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 16 Jan 2006 07:53:20 -0800 Subject: [Python-Dev] basenumber redux In-Reply-To: References: Message-ID: On 1/15/06, Alex Martelli wrote: > Now, today, I have _again_ been bit by the lack of basenumber (by a > bug of mine, fixed by adding decimal.Decimal to a long tuple of > classes to be passed to an isinstance call -- I hadn't run that > particular numeric code of mine since the time of Python 2.3, > apparently), so I'm back to pining for it. As you already suspected, I think a PEP is needed. The intent of basestring was to *only* be used as the base class for *built-in* string types. Clearly what you're proposing is different (Decimal is not built-in -- not yet anyway). Like other posters, I suspect that the best way of detecting numbers might be some other kind of test, not necessarily a call to isinstance(). It would also help to explain the user case more. ("I've been bitten" doesn't convey a lot of information. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.peters at gmail.com Mon Jan 16 17:09:20 2006 From: tim.peters at gmail.com (Tim Peters) Date: Mon, 16 Jan 2006 11:09:20 -0500 Subject: [Python-Dev] [Python-checkins] r42064 - python/trunk/Lib/logging/handlers.py In-Reply-To: <17355.49089.866569.489467@montanaro.dyndns.org> References: <20060116090810.7D2581E4002@bag.python.org> <17355.49089.866569.489467@montanaro.dyndns.org> Message-ID: <1f7befae0601160809n7c7888bg2534402750d03cf6@mail.gmail.com> [vinay] >> Log: >> Fixed bug in time-to-midnight calculation. [Skip] > Is there some reason not to use datetime to do this? PEP 291 says logging intends to be compatible with Python 1.5.2, which leaves datetime out. Vinay, rather than: if (currentMinute == 0) and (currentSecond == 0): r = (24 - currentHour) * 60 * 60 # number of hours in seconds else: r = (23 - currentHour) * 60 * 60 r = r + (59 - currentMinute) * 60 # plus the number of minutes (in secs) r = r + (60 - currentSecond) # pl for clarity I suggest this instead: # A module-level constant _MIDNIGHT = 24 * 60 * 60 # number of seconds in a day r = _MIDNIGHT - ((currentHour * 60 + currentMinute) * 60 + currentSecond) That's "obviously correct" instead of "huh?" . From lists at core-sdi.com Mon Jan 16 17:27:11 2006 From: lists at core-sdi.com (Gabriel Becedillas) Date: Mon, 16 Jan 2006 13:27:11 -0300 Subject: [Python-Dev] pystate.c changes for Python 2.4.2 In-Reply-To: <43C7BB1F.3000904@corest.com> References: <43C68823.3000105@corest.com> <2mwth4qw3q.fsf@starship.python.net> <43C7BB1F.3000904@corest.com> Message-ID: <43CBC95F.3020907@core-sdi.com> Gabriel Becedillas wrote: > Michael Hudson wrote: >> Gabriel Becedillas writes: >> >> >>> Hi, >>> At the company I work for, we've embedded Python in C++ application >>> we develop. Since our upgrade to Python 2.4.2 from 2.4.1 we started >>> hitting Py_FatalError("Invalid thread state for this thread") when >>> using debug builds. >>> We use both multiple interpreters and thread states. >> >> >> I know you've said this to me in email already, but I have no idea how >> you managed to get this to work with 2.4.1 :) >> >> >>> I think the problem is that PyThreadState_Delete (which is used when I >>> destroy thread states and interpreters) is not making the same clean up >>> that PyThreadState_DeleteCurrent is doing (which is used when we >>> create threads from python code). If a thread id is reused (obviously >>> not between two running threads), and the previous thread used >>> PyThreadState_Delete instead of PyThreadState_DeleteCurrent, then an >>> old and invalid value for autoTLSkey is still stored, and that is >>> triggering the Py_FatalError when a call to PyThreadState_Swap is made. >>> If I add this code at the end of PyThreadState_Delete: >>> >>> if (autoTLSkey && PyThread_get_key_value(autoTLSkey) == tstate) >>> PyThread_delete_key_value(autoTLSkey); >>> >>> then everything works fine. >> >> >> I think I begin to understand the problem... but this is still >> fragile: it relies on the fact that you delete a thread state from the >> OS level thread that created it, but while a thread belonging to a >> different *interpreter state* has the GIL (or at least: the >> interpreter state of the thread state being deleted *doesn't* hold the >> GIL). Can you guarantee that? > > Mmm... it doesn't have to do with a race condition or something. Let me > try to show you with an example (keep in mind that this relies on the > fact that at some moment the operating system gives you a thread with > the same id of another thread that allready finished executing): > > // Initialize python. > Py_Initialize(); > PyEval_InitThreads(); > PyThreadState* p_main = PyThreadState_Swap(NULL); > PyEval_ReleaseLock(); > > /* > Asume this block is executed in a separate thread, and that has been > asigned thread id 10. > It doesn't matter what the main thread (the thread that initialized > Python) is doing. Lets assume its waiting for this one to finish. > */ > { > PyEval_AcquireLock(); > PyThreadState_Swap(p_main); > PyThreadState* p_child = PyThreadState_New(p_main->interp); > > PyThreadState_Swap(p_child); > PyRun_SimpleString("print 'mi vieja mula ya no es lo que era'"); > > PyThreadState_Clear(p_child); > PyThreadState_Swap(NULL); > PyThreadState_Delete(p_child); > PyEval_ReleaseLock(); > // The os level thread exits here. > } > /* > When this thread state gets deleted (using PyThreadState_Delete), the > autoTLSkey stored for thread id number 10 is not removed, because > PyThreadState_Delete doesn't have the necesary cleanup code (check > pystate.c): > > if (autoTLSkey && PyThread_get_key_value(autoTLSkey) == tstate) > PyThread_delete_key_value(autoTLSkey); > > PyThreadState_DeleteCurrent behaves correctly because it does have this > code. > I think that this code should be added to tstate_delete_common (I've > done this and everything worked just fine). > If a new thread executes the same code, and that thread is assigned the > same thread id, you get the Py_FatalError I'm talking about. > A workaround for this would be to use PyThreadState_DeleteCurrent > instead of PyThreadState_Delete. > > If you use the following code instead of the one above, the you have no > way out to the Py_FatalError: > */ > { > PyEval_AcquireLock(); > PyThreadState* ret = Py_NewInterpreter(); > > PyRun_SimpleString("print 'mi vieja mula ya no es lo que era'"); > > Py_EndInterpreter(ret); > PyThreadState_Swap(NULL); > PyEval_ReleaseLock(); > } > /* > When this interpreter gets deleted (using Py_EndInterpreter), its thread > state gets deleted using PyThreadState_Delete, and you are back to the > beginning of the problem. > */ > > I hope this helps to clarify the problem. > Thanks a lot and have a nice day. > >> It seems to me that to be robust, you need a way of saying "delete the >> thread local value of autoTLSKey from thread $FOO". But this is all >> very confusing... >> >> >>> Could you please confirm if this is a bug ? >> >> >> Yes, I think it's a bug. >> >> Cheers, >> mwh >> > > Can anybody tell me if the patch I suggested is ok ? That will be to add the following code at the end of PyThreadState_Delete: if (autoTLSkey && PyThread_get_key_value(autoTLSkey) == tstate) PyThread_delete_key_value(autoTLSkey); Thank you very much. -- Gabriel Becedillas Developer CORE SECURITY TECHNOLOGIES Florida 141 - 2? cuerpo - 7? piso C1005AAC Buenos Aires - Argentina Tel/Fax: (54 11) 5032-CORE (2673) http://www.corest.com From aleaxit at gmail.com Mon Jan 16 18:27:31 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Mon, 16 Jan 2006 09:27:31 -0800 Subject: [Python-Dev] basenumber redux In-Reply-To: References: Message-ID: <17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com> On Jan 16, 2006, at 7:53 AM, Guido van Rossum wrote: > On 1/15/06, Alex Martelli wrote: >> Now, today, I have _again_ been bit by the lack of basenumber (by a >> bug of mine, fixed by adding decimal.Decimal to a long tuple of >> classes to be passed to an isinstance call -- I hadn't run that >> particular numeric code of mine since the time of Python 2.3, >> apparently), so I'm back to pining for it. > > As you already suspected, I think a PEP is needed. The intent of I'll be happy to write it, if it stands any chance. > basestring was to *only* be used as the base class for *built-in* > string types. Clearly what you're proposing is different (Decimal is > not built-in -- not yet anyway). I can't find a PEP describing this restriction of basestring, and I don't see why a coder who needs to implement another kind of character string shouldn't subclass basestring, so that those instances pass an isinstance test on basestring which is quite likely to be found e.g. in the standard library. Implementing different kinds of numbers is more likely than implementing different kinds of strings, of course. > Like other posters, I suspect that the best way of detecting numbers > might be some other kind of test, not necessarily a call to > isinstance(). I've tended to use a try/except around x+0 to detect if "x is a number". But that's NOT how the standard library does it -- rather, it has isinstance tests (often forgetting long in the tuple of types), as I pointed out in my mails on the subject back in 2003 (google for [basenumber site:python.org], there aren't many). I will reproduce those in the PEP, of course, if I do write one. The x+0 test has been criticized in the past because x COULD be an instance of a type which defines an __add__ which has side effects, or a very inclusive __add__ which happens to accept an int argument even though the type is NOT meant to be a number. These could be seen as design defects of x's type, of course. A second argument against the x+0 test is performance. A third argument against it is asymmetry: why should I use completely different approaches to check if x is "some kind of string", vs checking if x is "some kind of number"? Once upon a time I used x +'' (with try/except around it) to check for stringness, but the introduction of basestring strongly signaled that this was not the "one obvious way" any more. > > It would also help to explain the user case more. ("I've been bitten" > doesn't convey a lot of information. :-) isinstance with a tuple of number types, where the tuple did not include Decimal (because when I developed and tested that module, Decimal wasn't around yet). That's the problem of using isinstance without a "flag class" like basestring: one is hardwiring a specific tuple of types as being "singled out" for different treatment. If a new type comes up (be it for the standard library or some extension) there's no way to respect the "open-closed principle", leaving the affected module "closed to changes" yet "open for extension". In this way, using isinstance with a "hardwired" tuple of types is open to the same objections as "type-switching": it produces code that is not extensible to new types beyond those specific ones it had considered at coding time. I have the same issue in the C-coded extension gmpy: I want (e.g.) a gmpy.mpq to be able to be constructed by passing any number as the argument, but I have no good way to say "what's a number", so I use rather dirty tricks -- in particular, I've had to tweak things in a weird direction in the latest gmpy to accomodate Python 2.4 (specifically Decimal). Since "being a number" is a protocol (albeit, like "being a string", a rather peculiar one -- "the verb TO BE" is always fraught;-), other traditional possibilities for supporting it are introducing a special method or flag attribute such as "__isanumber__" (either callable, or flagging numberhood just by its presence). But introducing flag- abstract-class basenumber is more consistent with what was done for strings and affords a simpler test via isinstance. Of course _if_ PEP 246 was accepted, anybody could add more types to the set of those which "can be adapted to being numbers", but an object to denote that protocol should still be there (since the standard library does need to test for numberhood in a few places). If I do write the PEP, should it be just about basenumber, or should it include baseinteger as well? The case for the latter is IMHO still good but a bit weaker; it would be nice to be able to code 'xy' * Z without having str.__rmul__ perform a hardcoded test on Z being specifically an int or a long, making "other kinds of integers" (e.g. gmpy.mpz instances) smoothly substitutable for ints everywhere (similarly for somelist[Z], of course). Right now I have to pepper my code with int(Z) casts when Z is a gmpy.mpz and I need to use it to index or multiply a sequence, which isn't nice for either readability or performance, but (in my experience so far, at least) these aren't _frequent_ needs, just occasional ones. Thanks, Alex From fredrik at pythonware.com Mon Jan 16 19:03:34 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Mon, 16 Jan 2006 19:03:34 +0100 Subject: [Python-Dev] basenumber redux References: <17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com> Message-ID: Alex Martelli wrote: > > As you already suspected, I think a PEP is needed. The intent of > > I'll be happy to write it, if it stands any chance. > > > basestring was to *only* be used as the base class for *built-in* > > string types. Clearly what you're proposing is different (Decimal is > > not built-in -- not yet anyway). > > I can't find a PEP describing this restriction of basestring that's how it's documented, at least: http://effbot.org/lib/builtin.basestring > Implementing different kinds of numbers is more likely than > implementing different kinds of strings, of course. indeed. implementing a new string type is not a trivial task in today's CPython. From steve at holdenweb.com Mon Jan 16 19:48:23 2006 From: steve at holdenweb.com (Steve Holden) Date: Mon, 16 Jan 2006 18:48:23 +0000 Subject: [Python-Dev] Names matter. In-Reply-To: References: <1f7befae0601151846h57bbdfa0s2609bf0d2bf06f44@mail.gmail.com> <43CB88BF.4080501@zope.com> Message-ID: <43CBEA77.5080105@holdenweb.com> Guido van Rossum wrote: [...] > > I think that's unnecessary, and not any more effective than what we > already do -- ask for a signed contributor form. Someone who signs > that form "Tim Peters" won't be stopped by a clause asking them to use > their real name, *really*. > Sorry. My previous message suffered from a smiley-deficiency. Clearly there is little point in a signed piece of paper saying "I am me, signed 'me'". regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/ From martin at v.loewis.de Mon Jan 16 23:01:29 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 16 Jan 2006 23:01:29 +0100 Subject: [Python-Dev] basenumber redux In-Reply-To: <17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com> References: <17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com> Message-ID: <43CC17B9.3060200@v.loewis.de> Alex Martelli wrote: > I can't find a PEP describing this restriction of basestring, and I > don't see why a coder who needs to implement another kind of > character string shouldn't subclass basestring, so that those > instances pass an isinstance test on basestring which is quite likely > to be found e.g. in the standard library. People could do that, but they would be on their own. basestring could be interpreted as "immutable sequence of characterish things", but it was really meant to be "union of str and unicode". There is no PEP because it was introduced by BDFL pronouncement. That it is not a generic base class for stringish types can be seen by looking at UserString.UserString, which doesn't inherit from basestring. For most practical purposes, those two definitions actually define the same thing - there simply aren't any stringish data types in praxis: you always have Unicode, and if you don't, you have bytes. > Implementing different kinds of numbers is more likely than > implementing different kinds of strings, of course. Right. That's why a PEP is needed here, but not for basestring. > A third argument against it is asymmetry: why should I use completely > different approaches to check if x is "some kind of string", vs > checking if x is "some kind of number"? I guess that's for practicality which beats purity. People often support interfaces that either accept both an individual string and a list of strings, and they need the test in that case. It would be most natural to look for whether it is a sequence; unfortunately, strings are also sequences. > isinstance with a tuple of number types, where the tuple did not > include Decimal (because when I developed and tested that module, > Decimal wasn't around yet). As I suggested in a different message: Why are you doing that in the first place? > I have the same issue in the C-coded extension gmpy: I want (e.g.) a > gmpy.mpq to be able to be constructed by passing any number as the > argument, but I have no good way to say "what's a number", so I use > rather dirty tricks -- in particular, I've had to tweak things in a > weird direction in the latest gmpy to accomodate Python 2.4 > (specifically Decimal). Not sure what a gmpy.mpq is, but I would expect that can only work if the parameter belongs to some algebraic ring homomorphic with the real numbers, or some such. Are complex numbers also numbers? Is it meaningful to construct gmpy.mpqs out of them? What about Z/nZ? > If I do write the PEP, should it be just about basenumber, or should > it include baseinteger as well? I think it should only do the case you care about. If others have other use cases, they might get integrated, or they might have to write another PEP. Regards, Martin From martin at v.loewis.de Mon Jan 16 23:02:35 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 16 Jan 2006 23:02:35 +0100 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: References: <43C58200.20803@v.loewis.de> Message-ID: <43CC17FB.5020904@v.loewis.de> Guido van Rossum wrote: > On 1/16/06, Thomas Heller wrote: > >>It looks like we need a pronouncement now. > > > Sorry. It appeared to me that there was general agreement to using a > strongly worded warning in the docs, so I tuned out of the rest of the > discussion. So for the record, I'm fine with that. Ok. If Thomas checks in the code and the documentation, I'll do the Windows/setup part of it. Regards, Martin From martin at v.loewis.de Mon Jan 16 23:07:15 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 16 Jan 2006 23:07:15 +0100 Subject: [Python-Dev] Names matter. In-Reply-To: <43CB88BF.4080501@zope.com> References: <1f7befae0601151846h57bbdfa0s2609bf0d2bf06f44@mail.gmail.com> <43CB88BF.4080501@zope.com> Message-ID: <43CC1913.1050306@v.loewis.de> Jim Fulton wrote: > In general though, for adults, truthfulness and non-anonymity *do* > matter. At least they matter to me, a *lot*. I don't think members > of the PSF should be allowed to hide their identity and certainly, > it should not be acceptable to contribute to Python under a false name. Absolutely right. The encouragement is not for the past pseudonym, but for actually revealing his true identity when it mattered (i.e. when asked to "officially join"). For contributors, we should really track copyright forms more thoroughly (and indeed, Georg said he sent one right after I asked him). Regards, Martin From t-meyer at ihug.co.nz Mon Jan 16 23:10:54 2006 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Tue, 17 Jan 2006 11:10:54 +1300 Subject: [Python-Dev] Names matter. In-Reply-To: References: <1f7befae0601151846h57bbdfa0s2609bf0d2bf06f44@mail.gmail.com> <43CB88BF.4080501@zope.com> Message-ID: <12C84BD6-0F38-4D02-9807-47A4BDFAFFC8@ihug.co.nz> [Jim Fulton] > certainly, it should not be acceptable to contribute to Python > under a false name. What do you mean "contribute to Python"? Do you mean become one of the developers listed on sourceforge? Contribute any patches, even simple documentation ones? Review, comment and test patches? Work on pydotorg? Write python-dev summaries? Give money to the PSF? (These, and many other things, are all contributing to Python, IMO). I can understand that for legal purposes having some sort legally valid name might be required in some cases (e.g. contributing significant amounts of code), but wouldn't having that name known to the PSF suffice? I don't see how a pseudonym hurts when dealing with the "public". =Tony.Meyer From aahz at pythoncraft.com Tue Jan 17 01:43:34 2006 From: aahz at pythoncraft.com (Aahz) Date: Mon, 16 Jan 2006 16:43:34 -0800 Subject: [Python-Dev] Names matter. In-Reply-To: <43CB88BF.4080501@zope.com> References: <1f7befae0601151846h57bbdfa0s2609bf0d2bf06f44@mail.gmail.com> <43CB88BF.4080501@zope.com> Message-ID: <20060117004334.GC12718@panix.com> On Mon, Jan 16, 2006, Jim Fulton wrote: > > In general though, for adults, truthfulness and non-anonymity *do* > matter. At least they matter to me, a *lot*. I don't think members > of the PSF should be allowed to hide their identity and certainly, it > should not be acceptable to contribute to Python under a false name. And to me it's important for both political and personal reasons that people be accepted -- nay, encouraged -- to use pseudonyms as they please (or even choose to be anonymous). I won't write a long screed on this subject because it's off-topic here, but I will oppose any attempt to limit access to the Python community from people who do not use their legal names. (See Alex's excellent post for more detail.) One point where I do agree with you: I have little taste for morphing aliases that refer to a single person. If someone chooses to be declaratively anonymous, that's fine; if someone chooses to use a stable pseudonym (or a "real name" that happens to not be zir legal name), that's fine, too. But constantly changing identity does strike me as abusing community. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From aleaxit at gmail.com Tue Jan 17 03:18:10 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Mon, 16 Jan 2006 18:18:10 -0800 Subject: [Python-Dev] basenumber redux In-Reply-To: <43CC17B9.3060200@v.loewis.de> References: <17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com> <43CC17B9.3060200@v.loewis.de> Message-ID: <37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com> On Jan 16, 2006, at 2:01 PM, Martin v. L?wis wrote: > Alex Martelli wrote: >> I can't find a PEP describing this restriction of basestring, and I >> don't see why a coder who needs to implement another kind of >> character string shouldn't subclass basestring, so that those >> instances pass an isinstance test on basestring which is quite likely >> to be found e.g. in the standard library. > > People could do that, but they would be on their own. basestring > could be interpreted as "immutable sequence of characterish things", > but it was really meant to be "union of str and unicode". There > is no PEP because it was introduced by BDFL pronouncement. Unfortunately the lack of a PEP leaves this a bit underdocumented. > That it is not a generic base class for stringish types can be > seen by looking at UserString.UserString, which doesn't inherit > from basestring. Raymond Hettinger's reason for not wanting to add basestring as a base for UserString was: UserString is a legacy class, since today people can inherit from str directly, and it cannot be changed from a classic class to a new-style one without breaking backwards compatibility, which for a legacy class would be a big booboo. Nothing was said about "different design intent for basestring", as I recall (that discussion comes up among the few hits for [basestring site:python.org] if you want to check the details). > For most practical purposes, those two definitions actually > define the same thing - there simply aren't any stringish data > types in praxis: you always have Unicode, and if you don't, > you have bytes. But not necessarily in one big blob that's consecutive (==compact) in memory. mmap instances are "almost" strings and could easily be made into a closer match, at least for the immutable variants, for example; other implementations such as SGI STL's "Rope" also come to mind. In the context of a current struggle (a different and long story) between Python builds with 2-bytes Unicode and ones with 4-bytes Unicode, I've sometimes found myself dreaming of a string type that's GUARANTEED to be 2-bytes, say, and against which extension modules could be written that don't need recompilation to move among such different builds, for example. It's not (yet) hurting enough to make me hunker down and write such an extension (presumably mostly by copy- past-edit from Python's own sources;-), but if somebody did it would sure be nice if they could have that type "assert it's a string" by inheriting from basestring, no? >> Implementing different kinds of numbers is more likely than >> implementing different kinds of strings, of course. > > Right. That's why a PEP is needed here, but not for basestring. OK, I've mailed requesting a number. > >> A third argument against it is asymmetry: why should I use completely >> different approaches to check if x is "some kind of string", vs >> checking if x is "some kind of number"? > > I guess that's for practicality which beats purity. People often > support interfaces that either accept both an individual string > and a list of strings, and they need the test in that case. > It would be most natural to look for whether it is a sequence; > unfortunately, strings are also sequences. Sure, isinstance-tests with basestring are a fast and handy way to typetest that. But so would similar tests with basenumber be. > >> isinstance with a tuple of number types, where the tuple did not >> include Decimal (because when I developed and tested that module, >> Decimal wasn't around yet). > > As I suggested in a different message: Why are you doing that > in the first place? Because isinstance is faster and handier than testing with try/except around (say) "x+0". As to why I want to distinguish numbers from non-numbers, let me quote from a message I sent in 2003 (one of the few you'll find by searching for [basestring site:python.org] as I have repeatedly recommended, but apparently there's no way to avoid just massively copying and pasting...): """ def __mul__(self, other): if isinstance(other, self.KnownNumberTypes): return self.__class__([ x*other for x in self.items ]) else: # etc etc, various other multiplication cases right now, that (class, actually) attribute KnownNumberTypes starts out "knowing" about int, long, float, gmpy.mpz, etc, and may require user customization (e.g by subclassing) if any other "kind of (scalar) number" needs to be supported; besides, the isinstance check must walk linearly down the tuple of known number types each time. (I originally had quite a different test structure: try: other + 0 except TypeError: # other is not a number # various other multiplication cases else: # other is a number, so... return self.__class__([ x*other for x in self.items ]) but the performance for typical benchmarks improved with the isinstance test, so, reluctantly, that's what I changed to). If an abstract basetype 'basenumber' caught many useful cases, I'd put it right at the start of the KnownNumberTypes tuple, omit all subclasses thereof from it, get better performance, AND be able to document very simply what the user must do to ensure his own custom type is known to me as "a number". """ Other use cases, still quoted from the very same message: """ in Python/bltinmodule.c , function builtin_sum uses C-coded typechecking to single out strings as an error case: /* reject string values for 'start' parameter */ if (PyObject_TypeCheck(result, &PyBaseString_Type)) { PyErr_SetString(PyExc_TypeError, "sum() can't sum strings [use ''.join(seq) instea [etc]. Now, what builtin_sum really "wants" to do is to accept numbers, only -- it's _documented_ as being meant for "numbers": it uses +, NOT +=, so its performance on sequences, matrix and array-ish things, etc, is not going to be good. But -- it can't easily _test_ whether something "is a number". If we had a PyBaseNumber_Type to use here, it would be smooth, easy, and fast to check for it. """ and yet another, which is the directly gmpy-related one: """ I see a few other cases in the standard library which want to treat "numbers" in some specific way different from other types (often forgetting longs:-), e.g. Lib/plat-mac/plistlib.py has one. In gmpy, I would often like some operations to be able to accept "a number", perhaps by letting it try to transform itself into a float as a worst case (so complex numbers would fail there), but I definitely do NOT want to accept non-number objects which "happen to be able to return a value from float(x)", such as strings. In all such cases of wanting to check if something "is a number", an abstract basetype might be handy, smooth, fast. """ > >> I have the same issue in the C-coded extension gmpy: I want (e.g.) a >> gmpy.mpq to be able to be constructed by passing any number as the >> argument, but I have no good way to say "what's a number", so I use >> rather dirty tricks -- in particular, I've had to tweak things in a >> weird direction in the latest gmpy to accomodate Python 2.4 >> (specifically Decimal). > > Not sure what a gmpy.mpq is, but I would expect that can only work A fast rational number type, see http://gmpy.sourceforge.net for details (gmpy wraps LGPL'd library GMP, and gets a lot of speed and functionality thereby). > if the parameter belongs to some algebraic ring homomorphic > with the real numbers, or some such. Are complex numbers also numbers? > Is it meaningful to construct gmpy.mpqs out of them? What about > Z/nZ? If I could easily detect "this is a number" about an argument x, I'd then ask x to change itself into a float, so complex would be easily rejected (while decimals would mostly work fine, although a bit slowly without some specialcasing, due to the Stern-Brocot-tree algorithm I use to build gmpy.mpq's from floats). I can't JUST ask x to "make itself into a float" (without checking for x's "being a number") because that would wrongfully succeed for many cases such as strings. >> If I do write the PEP, should it be just about basenumber, or should >> it include baseinteger as well? > > I think it should only do the case you care about. If others have > other > use cases, they might get integrated, or they might have to write > another PEP. Good idea. I'll check around -- if anybody feels strongly about baseinteger they may choose to co-author with me (and the PEP will propose both), otherwise we'll hassle on basenumber only;-) Alex From guido at python.org Tue Jan 17 04:00:23 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 16 Jan 2006 19:00:23 -0800 Subject: [Python-Dev] basenumber redux In-Reply-To: <37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com> References: <17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com> <43CC17B9.3060200@v.loewis.de> <37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com> Message-ID: On 1/16/06, Alex Martelli wrote: > Nothing was said about "different design intent for basestring", as I > recall (that discussion comes up among the few hits for [basestring > site:python.org] if you want to check the details). Please. How many times do I have to say this. *I added this with the stated intent.* Maybe I didn't communicate that intent clearly. But my intent was exactly as it is documented -- basestring == (str, unicode) and nothing else. Why this intent? Because I felt the need to be able to distinguish the *built-in* string types from *anything* else; they have many special properties and purposes. Also note that basestring was introduced in 2.3, a whole release *after* inheritance from str was made possible. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From aleaxit at gmail.com Tue Jan 17 04:44:44 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Mon, 16 Jan 2006 19:44:44 -0800 Subject: [Python-Dev] str with base Message-ID: Is it finally time in Python 2.5 to allow the "obvious" use of, say, str(5,2) to give '101', just the converse of the way int('101',1) gives 5? I'm not sure why str has never allowed this obvious use -- any bright beginner assumes it's there and it's awkward to explain why it's not!-). I'll be happy to propose a patch if the BDFL blesses this, but I don't even think it's worth a PEP... it's an inexplicable though long-standing omission (given the argumentative nature of this crowd I know I'll get pushback, but I still hope the BDFL can Pronounce about it anyway;-). Alex From python at rcn.com Tue Jan 17 04:37:19 2006 From: python at rcn.com (Raymond Hettinger) Date: Mon, 16 Jan 2006 22:37:19 -0500 Subject: [Python-Dev] basenumber redux In-Reply-To: <37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com> Message-ID: <001901c61b17$52c0c280$e221cb97@oemcomputer> > Because isinstance is faster and handier than testing with try/except > around (say) "x+0". Are you sure? The former has two builtin lookups (which also entail two failed global lookups), a function call, and a test/jump for the result. The latter approach has no lookups (just a load constant), a try-block setup, an add operation (optimized for integers, a fast slot lookup otherwise), and a block end. Even if there were a performance edge, I suppose that the type checking is the time critical part of most scripts. > If an abstract > basetype > 'basenumber' caught many useful cases, I'd put it right at the start of > the KnownNumberTypes tuple, omit all subclasses thereof from it, get > better performance, AND be able to document very simply what the user > must do to ensure his own custom type is known to me as "a number". So, this would essentially become a required API? It would no longer be enough to duck-type a number, you would also have to subclass from basenumber? Wouldn't this preclude custom number types that also need to derive from another builtin such as str? For instance, somewhere I have Gram-Schmidt orthogonalization transformation code for computing orthonormal bases and the only requirement for the input basis is that it be an inner-product space -- accordingly, the inputs can be functions, symbols (a subclass of str), or vectors (pretty-much anything supporting multiplication and subtraction). Similar conflicts arise for any pure computation function -- I should be able to supply either numbers or symbolic inputs (a subclass of str) that act like numbers. > in Python/bltinmodule.c , function builtin_sum uses C-coded > typechecking > to single out strings as an error case: > > /* reject string values for 'start' parameter */ > if (PyObject_TypeCheck(result, &PyBaseString_Type)) { > PyErr_SetString(PyExc_TypeError, > "sum() can't sum strings [use ''.join(seq) instea > > [etc]. Now, what builtin_sum really "wants" to do is to accept numbers, > only -- it's _documented_ as being meant for "numbers": it uses +, NOT > +=, so its performance on sequences, matrix and array-ish things, etc, > is not going to be good. But -- it can't easily _test_ whether > something > "is a number". If we had a PyBaseNumber_Type to use here, it would > be smooth, easy, and fast to check for it. > """ I think this a really a counter-example. We wanted to preclude sequences so that sum() didn't become a low performance synonym for ''.join(). However, there's no reason to preclude user-defined matrix or vector classes. And, if you're suggesting that the type-check should have been written with as instance(result, basenumber), one consequence would be that existing number-like classes would start to fail unless retro-fitted with a basenumber superclass. Besides being a PITA, retro-fitting existing, working code could also entail changing from an old to new-style class (possibly changing the semantics of the class). > A fast rational number type, see http://gmpy.sourceforge.net for > details (gmpy wraps LGPL'd library GMP, and gets a lot of speed and > functionality thereby). > > > if the parameter belongs to some algebraic ring homomorphic > > with the real numbers, or some such. Are complex numbers also numbers? > > Is it meaningful to construct gmpy.mpqs out of them? What about > > Z/nZ? > > If I could easily detect "this is a number" about an argument x, I'd > then ask x to change itself into a float, so complex would be easily > rejected (while decimals would mostly work fine, although a bit > slowly without some specialcasing, due to the Stern-Brocot-tree > algorithm I use to build gmpy.mpq's from floats). I can't JUST ask x > to "make itself into a float" (without checking for x's "being a > number") because that would wrongfully succeed for many cases such as > strings. Part of the rationale for basestring was convenience of writing isinstance(x,basestring) instead of isinstance(x,(str,unicode)). No existing code needed to be changed for this to work. This proposal, on the other hand, is different. To get the purported benefits, everyone has to play along. All existing number-look-a-like classes would have to be changed in order to work with functions testing for basenumber. That would be too bad for existing, work code. Likewise, it would be impossible for symbolic code which already subclassed from str (as discussed above). > >> If I do write the PEP, should it be just about basenumber, or should > >> it include baseinteger as well? Introducing a baseinteger type is likely to create confusion because all integers are real, but baseintegers are not a subclass of floats. I don't see a way around creating an integer recognition tool that doesn't conflate its terminology with broadly-held, pre-existing math knowledge: complex is a superset of reals, reals include rationals and irrationals some of which are trancendental, and rationals include integers which are an infinite superset of non-negative integers, whole numbers, negative numbers, etc. The decimal class only makes this more complicated. All binary floats can be translated exactly to decimal but not vice-versa. I'm not sure where they would fit into a inheritance hierarchy. On the one hand, decimals are very much like floats. On the other hand, they can be used as integers and should not be rejected by some code testing for baseinteger unless the value actually has a fractional part. The latter isn't a nit in the same league as wanting float('2.0') to be acceptable as an integer argument; rather, the decimal class was specifically designed to usable in integer-only situations (the spec holds itself out as encompassing integer math, fixed-point, and floating-point models). Raymond (my real name is unpronounceable by humans) From brett at python.org Tue Jan 17 04:50:55 2006 From: brett at python.org (Brett Cannon) Date: Mon, 16 Jan 2006 19:50:55 -0800 Subject: [Python-Dev] str with base In-Reply-To: References: Message-ID: On 1/16/06, Alex Martelli wrote: > Is it finally time in Python 2.5 to allow the "obvious" use of, say, > str(5,2) to give '101', just the converse of the way int('101',1) I think you mean ``int('101' 2)``. =) > gives 5? I'm not sure why str has never allowed this obvious use -- > any bright beginner assumes it's there and it's awkward to explain > why it's not!-). I'll be happy to propose a patch if the BDFL > blesses this, but I don't even think it's worth a PEP... it's an > inexplicable though long-standing omission (given the argumentative > nature of this crowd I know I'll get pushback, but I still hope the > BDFL can Pronounce about it anyway;-). > I'm +0. Not a big thing for me, but having the symmetry seems reasonable. -Brett From python at rcn.com Tue Jan 17 04:54:47 2006 From: python at rcn.com (Raymond Hettinger) Date: Mon, 16 Jan 2006 22:54:47 -0500 Subject: [Python-Dev] str with base In-Reply-To: Message-ID: <002d01c61b19$c2cec160$e221cb97@oemcomputer> > Is it finally time in Python 2.5 to allow the "obvious" use of, say, > str(5,2) to give '101', just the converse of the way int('101',2) > gives 5? +1 The only downside is that like list.reverse(), it will deprive students of being assigned to write the roll-your-own version. Raymond From python at rcn.com Tue Jan 17 04:57:30 2006 From: python at rcn.com (Raymond Hettinger) Date: Mon, 16 Jan 2006 22:57:30 -0500 Subject: [Python-Dev] basenumber redux In-Reply-To: <001901c61b17$52c0c280$e221cb97@oemcomputer> Message-ID: <002e01c61b1a$24014340$e221cb97@oemcomputer> [Me] > Even if there were a performance edge, I suppose that the type checking > is the time critical part of most scripts. That should be: RARELY the time critical part of most scripts. From jeremy at alum.mit.edu Tue Jan 17 05:03:17 2006 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Mon, 16 Jan 2006 23:03:17 -0500 Subject: [Python-Dev] str with base In-Reply-To: References: Message-ID: It never occured to me that str() would behave like int() for this case. It makes complete sense to me that a factory for numbers would ask about the base of the number. What would the base of a string be, except in a few limited cases? str([1, 2], 4) doesn't make any sense. You might argue that I wasn't all that bright as a beginner <0.5 wink>. I think it shouldn't be changed, because the second positional argument only works for a small number of the panoply types that can be passed to str(). It would be fine to have a function for this hiding somewhere, perhaps even as a method on numbers, but str() is too generic. Jeremy On 1/16/06, Alex Martelli wrote: > Is it finally time in Python 2.5 to allow the "obvious" use of, say, > str(5,2) to give '101', just the converse of the way int('101',1) > gives 5? I'm not sure why str has never allowed this obvious use -- > any bright beginner assumes it's there and it's awkward to explain > why it's not!-). I'll be happy to propose a patch if the BDFL > blesses this, but I don't even think it's worth a PEP... it's an > inexplicable though long-standing omission (given the argumentative > nature of this crowd I know I'll get pushback, but I still hope the > BDFL can Pronounce about it anyway;-). > > > Alex > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu > From guido at python.org Tue Jan 17 05:04:27 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 16 Jan 2006 20:04:27 -0800 Subject: [Python-Dev] str with base In-Reply-To: References: Message-ID: On 1/16/06, Alex Martelli wrote: > Is it finally time in Python 2.5 to allow the "obvious" use of, say, > str(5,2) to give '101', just the converse of the way int('101',1) [I'm sure you meant int('101', 2) here] > gives 5? I'm not sure why str has never allowed this obvious use -- > any bright beginner assumes it's there and it's awkward to explain > why it's not!-). I'll be happy to propose a patch if the BDFL > blesses this, but I don't even think it's worth a PEP... it's an > inexplicable though long-standing omission (given the argumentative > nature of this crowd I know I'll get pushback, but I still hope the > BDFL can Pronounce about it anyway;-). I wish you had an argument better than "every bright beginner assumes it exists". :-) But (unlike for some other things that bright beginners might assume) I don't think there's a deep reason why this couldn't exist. The only reasons I can come up with is "because input and output are notoriously asymmetric in Python" and "because nobody submitted a patch". :-) There are some corner cases to consider though. - Should repr() support the same convention? I think not. - Should str(3.1, n) be allowed? I think not. - Should str(x, n) call x.__str__(n)? Neither. - Should bases other than 2..36 be considered? I don't think so. Unfortunately this doesn't obsolete oct() and hex() -- oct(10) returns '012', while str(10, 8) soulc return '12'; hex(10) returns '0xa' while str(10, 16) would return 'a'. I do think that before we add this end-user nicety, it's more important to implement __index__() in Python 2.5. This behaves like __int__() for integral types, but is not defined for float or Decimal. Operations that intrinsically require an integral argument, like indexing and slicing, should call __index__() on their argument rather than __int__(), so as to support non-built-in integral arguments while still complaining about float arguments. This is currently implemented by explicitly checking for float in a few places, which I find repulsing. __index__() won't be requested by bright beginners, but it is important e.g. to the Numeric Python folks, who like to implement their own integral types but are suffering from that their integers aren't usable everywhere. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From andrew-pythondev at puzzling.org Tue Jan 17 05:08:02 2006 From: andrew-pythondev at puzzling.org (Andrew Bennetts) Date: Tue, 17 Jan 2006 15:08:02 +1100 Subject: [Python-Dev] str with base In-Reply-To: References: Message-ID: <20060117040802.GG7771@home.puzzling.org> On Mon, Jan 16, 2006 at 07:44:44PM -0800, Alex Martelli wrote: > Is it finally time in Python 2.5 to allow the "obvious" use of, say, > str(5,2) to give '101', My reaction having read this far was "huh?". It took some time (several seconds) before it occurred to me what you wanted str(5,2) to mean, and why it should give '101'. If you'd proposed, say (5).as_binary() == '101', or "5".encode("base2"), I wouldn't have been as baffled. Or perhaps even str(5, base=2), but frankly the idea of the string type doing numeric base conversions seems weird to me, rather than symmetric. I wouldn't mind seeing arbitrary base encoding of integers included somewhere, but as a method of str -- let alone the constructor! -- it feels quite wrong. -Andrew. From raymond.hettinger at verizon.net Tue Jan 17 05:13:27 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Mon, 16 Jan 2006 23:13:27 -0500 Subject: [Python-Dev] str with base In-Reply-To: Message-ID: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> > I wish you had an argument better than "every bright beginner assumes > it exists". :-) > > But (unlike for some other things that bright beginners might assume) > I don't think there's a deep reason why this couldn't exist. > > The only reasons I can come up with is "because input and output are > notoriously asymmetric in Python" and "because nobody submitted a > patch". :-) My reason is that I've rolled-my-own more times than I can count but infrequently enough to where it was easier to re-write than to search for the previous use. Another quick thought: I presume that only the str() builtin would change and that the underlying __str__ slot would continue to be hard-wired to the (reprfunc) signature. Raymond From exarkun at divmod.com Tue Jan 17 05:16:37 2006 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Mon, 16 Jan 2006 23:16:37 -0500 Subject: [Python-Dev] str with base In-Reply-To: Message-ID: <20060117041637.26200.1650489643.divmod.quotient.1321@ohm> On Mon, 16 Jan 2006 19:44:44 -0800, Alex Martelli wrote: >Is it finally time in Python 2.5 to allow the "obvious" use of, say, >str(5,2) to give '101', just the converse of the way int('101',1) >gives 5? I'm not sure why str has never allowed this obvious use -- >any bright beginner assumes it's there and it's awkward to explain >why it's not!-). I'll be happy to propose a patch if the BDFL >blesses this, but I don't even think it's worth a PEP... it's an >inexplicable though long-standing omission (given the argumentative >nature of this crowd I know I'll get pushback, but I still hope the >BDFL can Pronounce about it anyway;-). > -1. Confusing and non-obvious. The functionality may be valuable but it is mis-placed as a feature of str() or a method of the str type. I work with a lot of Python beginners too, and while they occassionally ask for this functionality, I've never heard anyone wonder why str() didn't provide it or suggest that it should. Jean-Paul From barry at python.org Tue Jan 17 05:18:11 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 16 Jan 2006 23:18:11 -0500 Subject: [Python-Dev] str with base In-Reply-To: <20060117040802.GG7771@home.puzzling.org> References: <20060117040802.GG7771@home.puzzling.org> Message-ID: <1137471491.5352.87.camel@geddy.wooz.org> On Tue, 2006-01-17 at 15:08 +1100, Andrew Bennetts wrote: > My reaction having read this far was "huh?". It took some time (several > seconds) before it occurred to me what you wanted str(5,2) to mean, and why it > should give '101'. > > If you'd proposed, say (5).as_binary() == '101', or "5".encode("base2"), I > wouldn't have been as baffled. Or perhaps even str(5, base=2), but frankly the > idea of the string type doing numeric base conversions seems weird to me, rather > than symmetric. > > I wouldn't mind seeing arbitrary base encoding of integers included somewhere, > but as a method of str -- let alone the constructor! -- it feels quite wrong. Hear, hear. I was similarly perplexed when I first read that! -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060116/ce26803a/attachment.pgp From aleaxit at gmail.com Tue Jan 17 05:25:54 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Mon, 16 Jan 2006 20:25:54 -0800 Subject: [Python-Dev] str with base In-Reply-To: References: Message-ID: On Jan 16, 2006, at 8:03 PM, Jeremy Hylton wrote: > It never occured to me that str() would behave like int() for this > case. It makes complete sense to me that a factory for numbers would > ask about the base of the number. What would the base of a string be, > except in a few limited cases? str([1, 2], 4) doesn't make any sense. > You might argue that I wasn't all that bright as a beginner <0.5 > wink>. > > I think it shouldn't be changed, because the second positional > argument only works for a small number of the panoply types that can > be passed to str(). Identically the same situation as for int: the base argument is only accepted if the first argument is a str (not a float, etc). Just the same way, the base argument to str will only be accepted if the first argument is an int (not a float, etc). Alex From jeremy at alum.mit.edu Tue Jan 17 05:47:50 2006 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Mon, 16 Jan 2006 23:47:50 -0500 Subject: [Python-Dev] str with base In-Reply-To: References: Message-ID: On 1/16/06, Alex Martelli wrote: > > On Jan 16, 2006, at 8:03 PM, Jeremy Hylton wrote: > > I think it shouldn't be changed, because the second positional > > argument only works for a small number of the panoply types that can > > be passed to str(). > > Identically the same situation as for int: the base argument is only > accepted if the first argument is a str (not a float, etc). Just the > same way, the base argument to str will only be accepted if the first > argument is an int (not a float, etc). The concept of base is closely related to ints, and the base argument is useful for a large percentage of the types that int accepts. It is not related to strings, in general, and applies to only one of the types it accepts. In one case "not a float, etc." applies to a very limited set of types, in the other case it applies to every conceivable type (except int). If str() were to take two argument, the analogy with int suggests it should be an encoding, where the second argument describes how to interpret the representation of the first (it's base 7 or it's utf-8). Jeremy From bob at redivi.com Tue Jan 17 05:49:26 2006 From: bob at redivi.com (Bob Ippolito) Date: Mon, 16 Jan 2006 20:49:26 -0800 Subject: [Python-Dev] str with base In-Reply-To: <1137471491.5352.87.camel@geddy.wooz.org> References: <20060117040802.GG7771@home.puzzling.org> <1137471491.5352.87.camel@geddy.wooz.org> Message-ID: <501A02B9-738E-4334-8621-50E0E1AB3736@redivi.com> On Jan 16, 2006, at 8:18 PM, Barry Warsaw wrote: > On Tue, 2006-01-17 at 15:08 +1100, Andrew Bennetts wrote: > >> My reaction having read this far was "huh?". It took some time >> (several >> seconds) before it occurred to me what you wanted str(5,2) to >> mean, and why it >> should give '101'. >> >> If you'd proposed, say (5).as_binary() == '101', or "5".encode >> ("base2"), I >> wouldn't have been as baffled. Or perhaps even str(5, base=2), >> but frankly the >> idea of the string type doing numeric base conversions seems weird >> to me, rather >> than symmetric. >> >> I wouldn't mind seeing arbitrary base encoding of integers >> included somewhere, >> but as a method of str -- let alone the constructor! -- it feels >> quite wrong. > > Hear, hear. I was similarly perplexed when I first read that! The only bases I've ever really had a good use for are 2, 8, 10, and 16. There are currently formatting codes for 8 (o), 10 (d, u), and 16 (x, X). Why not just add a string format code for unsigned binary? The obvious choice is probably "b". For example: >>> '%08b' % (12) '00001100' >>> '%b' % (12) '1100' I'd probably expect "5".encode("base2") to return '00110101', because "5".encode("hex") returns '35' -bob From raymond.hettinger at verizon.net Tue Jan 17 05:54:05 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Mon, 16 Jan 2006 23:54:05 -0500 Subject: [Python-Dev] str with base In-Reply-To: Message-ID: <003b01c61b22$0bbfe040$e221cb97@oemcomputer> [Jeremy Hylton] > The concept of base is closely related to ints, and the base argument > is useful for a large percentage of the types that int accepts. It is > not related to strings, in general, and applies to only one of the > types it accepts. In one case "not a float, etc." applies to a very > limited set of types, in the other case it applies to every > conceivable type (except int). That suggests that it would be better to simply add an int method: x.convert_to_base(7) Raymond From andrew-pythondev at puzzling.org Tue Jan 17 06:12:53 2006 From: andrew-pythondev at puzzling.org (Andrew Bennetts) Date: Tue, 17 Jan 2006 16:12:53 +1100 Subject: [Python-Dev] str with base In-Reply-To: <003b01c61b22$0bbfe040$e221cb97@oemcomputer> References: <003b01c61b22$0bbfe040$e221cb97@oemcomputer> Message-ID: <20060117051253.GH7771@home.puzzling.org> On Mon, Jan 16, 2006 at 11:54:05PM -0500, Raymond Hettinger wrote: [...] > That suggests that it would be better to simply add an int method: > > x.convert_to_base(7) This seems clear and simple to me. I like it. I strongly suspect the "bright beginners" Alex is interested in would have no trouble using it or finding it. -Andrew. From bob at redivi.com Tue Jan 17 06:28:10 2006 From: bob at redivi.com (Bob Ippolito) Date: Mon, 16 Jan 2006 21:28:10 -0800 Subject: [Python-Dev] str with base In-Reply-To: <20060117051253.GH7771@home.puzzling.org> References: <003b01c61b22$0bbfe040$e221cb97@oemcomputer> <20060117051253.GH7771@home.puzzling.org> Message-ID: <771D4186-062D-446A-9621-95FD6048FCEB@redivi.com> On Jan 16, 2006, at 9:12 PM, Andrew Bennetts wrote: > On Mon, Jan 16, 2006 at 11:54:05PM -0500, Raymond Hettinger wrote: > [...] >> That suggests that it would be better to simply add an int method: >> >> x.convert_to_base(7) > > This seems clear and simple to me. I like it. I strongly suspect > the "bright > beginners" Alex is interested in would have no trouble using it or > finding it. I don't know about that, all of the methods that int and long currently have are __special__. They'd really need to start with Python 2.5 (assuming int/long grow "public methods" in 2.5) to even think to look there. A format code or a built-in would be more likely to be found, since that's how you convert integers to hex and oct string representations with current Python. >>> [name for name in dir(0)+dir(0L) if not name.startswith('__')] [] -bob From andrew-pythondev at puzzling.org Tue Jan 17 06:34:32 2006 From: andrew-pythondev at puzzling.org (Andrew Bennetts) Date: Tue, 17 Jan 2006 16:34:32 +1100 Subject: [Python-Dev] str with base In-Reply-To: <771D4186-062D-446A-9621-95FD6048FCEB@redivi.com> References: <003b01c61b22$0bbfe040$e221cb97@oemcomputer> <20060117051253.GH7771@home.puzzling.org> <771D4186-062D-446A-9621-95FD6048FCEB@redivi.com> Message-ID: <20060117053432.GI7771@home.puzzling.org> On Mon, Jan 16, 2006 at 09:28:10PM -0800, Bob Ippolito wrote: > On Jan 16, 2006, at 9:12 PM, Andrew Bennetts wrote: [...] > >> x.convert_to_base(7) > > > >This seems clear and simple to me. I like it. I strongly suspect > >the "bright > >beginners" Alex is interested in would have no trouble using it or > >finding it. > > I don't know about that, all of the methods that int and long > currently have are __special__. They'd really need to start with > Python 2.5 (assuming int/long grow "public methods" in 2.5) to even > think to look there. A format code or a built-in would be more > likely to be found, since that's how you convert integers to hex and > oct string representations with current Python. > > >>> [name for name in dir(0)+dir(0L) if not name.startswith('__')] > [] I should have said, I'm equally happy with the format code as well (although it doesn't allow arbitary base conversions, I've never had need for that, so I'm not too worried about that case). Either option is better than making the str constructor do relatively rarely used mathematics! -Andrew. From barry at python.org Tue Jan 17 07:03:32 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 17 Jan 2006 01:03:32 -0500 Subject: [Python-Dev] str with base In-Reply-To: <501A02B9-738E-4334-8621-50E0E1AB3736@redivi.com> References: <20060117040802.GG7771@home.puzzling.org> <1137471491.5352.87.camel@geddy.wooz.org> <501A02B9-738E-4334-8621-50E0E1AB3736@redivi.com> Message-ID: <1137477812.5351.91.camel@geddy.wooz.org> On Mon, 2006-01-16 at 20:49 -0800, Bob Ippolito wrote: > The only bases I've ever really had a good use for are 2, 8, 10, and > 16. There are currently formatting codes for 8 (o), 10 (d, u), and > 16 (x, X). Why not just add a string format code for unsigned > binary? The obvious choice is probably "b". > > For example: > > >>> '%08b' % (12) > '00001100' > >>> '%b' % (12) > '1100' +1 -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060117/14da01fc/attachment.pgp From brett at python.org Tue Jan 17 07:41:06 2006 From: brett at python.org (Brett Cannon) Date: Mon, 16 Jan 2006 22:41:06 -0800 Subject: [Python-Dev] str with base In-Reply-To: References: Message-ID: On 1/16/06, Guido van Rossum wrote: > On 1/16/06, Alex Martelli wrote: > > Is it finally time in Python 2.5 to allow the "obvious" use of, say, > > str(5,2) to give '101', just the converse of the way int('101',1) > [I'm sure you meant int('101', 2) here] > > gives 5? I'm not sure why str has never allowed this obvious use -- > > any bright beginner assumes it's there and it's awkward to explain > > why it's not!-). I'll be happy to propose a patch if the BDFL > > blesses this, but I don't even think it's worth a PEP... it's an > > inexplicable though long-standing omission (given the argumentative > > nature of this crowd I know I'll get pushback, but I still hope the > > BDFL can Pronounce about it anyway;-). > > I wish you had an argument better than "every bright beginner assumes > it exists". :-) > > But (unlike for some other things that bright beginners might assume) > I don't think there's a deep reason why this couldn't exist. > > The only reasons I can come up with is "because input and output are > notoriously asymmetric in Python" and "because nobody submitted a > patch". :-) > > There are some corner cases to consider though. > > - Should repr() support the same convention? I think not. > - Should str(3.1, n) be allowed? I think not. > - Should str(x, n) call x.__str__(n)? Neither. > - Should bases other than 2..36 be considered? I don't think so. > > Unfortunately this doesn't obsolete oct() and hex() -- oct(10) returns > '012', while str(10, 8) soulc return '12'; hex(10) returns '0xa' while > str(10, 16) would return 'a'. > > I do think that before we add this end-user nicety, it's more > important to implement __index__() in Python 2.5. This behaves like > __int__() for integral types, but is not defined for float or Decimal. > Operations that intrinsically require an integral argument, like > indexing and slicing, should call __index__() on their argument rather > than __int__(), so as to support non-built-in integral arguments while > still complaining about float arguments. This is currently implemented > by explicitly checking for float in a few places, which I find > repulsing. __index__() won't be requested by bright beginners, but it > is important e.g. to the Numeric Python folks, who like to implement > their own integral types but are suffering from that their integers > aren't usable everywhere. > +1 from me (feel like this has come up before, but not totally sure). Be nice to add an abstraction for indexing. Added to the PyCon wiki as a possible sprint topic. -Brett From martin at v.loewis.de Tue Jan 17 07:41:13 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 17 Jan 2006 07:41:13 +0100 Subject: [Python-Dev] basenumber redux In-Reply-To: <37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com> References: <17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com> <43CC17B9.3060200@v.loewis.de> <37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com> Message-ID: <43CC9189.2060206@v.loewis.de> Alex Martelli wrote: >> As I suggested in a different message: Why are you doing that >> in the first place? > > > Because isinstance is faster and handier than testing with try/except > around (say) "x+0". Nit: I think you should not test. Instead, you should starting you mean to do if the test passes, and then expect TypeErrors from doing so. > As to why I want to distinguish numbers from non-numbers, let me quote > from a message I sent in 2003 (one of the few you'll find by searching > for [basestring site:python.org] as I have repeatedly recommended, but > apparently there's no way to avoid just massively copying and pasting...): As I said in that other message, I found this one, and still don't understand what the use case is, because the example you give still reads hypothetical (I'm sure there is an actual example behind it, but you don't say what that is). > """ > def __mul__(self, other): > if isinstance(other, self.KnownNumberTypes): > return self.__class__([ x*other for x in self.items ]) > else: > # etc etc, various other multiplication cases So what *are* the "various other multiplication cases"? You just shouldn't be doing that: multiplication shouldn't mean "item multiplication" sometimes, and various other things if it can't mean item multiplication. > in Python/bltinmodule.c , function builtin_sum uses C-coded typechecking > to single out strings as an error case: > > /* reject string values for 'start' parameter */ > if (PyObject_TypeCheck(result, &PyBaseString_Type)) { > PyErr_SetString(PyExc_TypeError, > "sum() can't sum strings [use ''.join(seq) instea This is indeed confusing: why is it again that sum refuses to sum up strings? > [etc]. Now, what builtin_sum really "wants" to do is to accept numbers, > only -- it's _documented_ as being meant for "numbers": it uses +, NOT > +=, so its performance on sequences, matrix and array-ish things, etc, > is not going to be good. But -- it can't easily _test_ whether something > "is a number". If we had a PyBaseNumber_Type to use here, it would > be smooth, easy, and fast to check for it. There shouldn't be a check at all. It should just start doing the summing, and it will "work" if PyNumber_Add works for the individual items. Of course, there is an education value of ruling out string addition, since there is a better way to do that, and there should be only one obvious way. I see nothing wrong in summing up sequences, matrices, and arrayish things, using sum. > A fast rational number type, see http://gmpy.sourceforge.net for > details (gmpy wraps LGPL'd library GMP, and gets a lot of speed and > functionality thereby). Ok, so mpq are rational numbers. >> if the parameter belongs to some algebraic ring homomorphic >> with the real numbers, or some such. Are complex numbers also numbers? >> Is it meaningful to construct gmpy.mpqs out of them? What about >> Z/nZ? > > > If I could easily detect "this is a number" about an argument x, I'd > then ask x to change itself into a float, so complex would be easily > rejected (while decimals would mostly work fine, although a bit slowly > without some specialcasing, due to the Stern-Brocot-tree algorithm I > use to build gmpy.mpq's from floats). I can't JUST ask x to "make > itself into a float" (without checking for x's "being a number") > because that would wrongfully succeed for many cases such as strings. Hmm. If you want to do the generic conversion from numbers to rationals by going through float, then this is what you should do. Convert to float, and don't bother with checking whether it will succeed. Instead, if the type conversion gives an error, just return that to the caller. However, it also sounds odd that you are trying to do the to-rational-with-arbitrary-precision conversion by going through floats. Instead, if the argument is decimal, you really should do the division by the approprate base of 10, no? Regards, Martin From brett at python.org Tue Jan 17 07:45:23 2006 From: brett at python.org (Brett Cannon) Date: Mon, 16 Jan 2006 22:45:23 -0800 Subject: [Python-Dev] str with base In-Reply-To: <771D4186-062D-446A-9621-95FD6048FCEB@redivi.com> References: <003b01c61b22$0bbfe040$e221cb97@oemcomputer> <20060117051253.GH7771@home.puzzling.org> <771D4186-062D-446A-9621-95FD6048FCEB@redivi.com> Message-ID: On 1/16/06, Bob Ippolito wrote: > > On Jan 16, 2006, at 9:12 PM, Andrew Bennetts wrote: > > > On Mon, Jan 16, 2006 at 11:54:05PM -0500, Raymond Hettinger wrote: > > [...] > >> That suggests that it would be better to simply add an int method: > >> > >> x.convert_to_base(7) > > > > This seems clear and simple to me. I like it. I strongly suspect > > the "bright > > beginners" Alex is interested in would have no trouble using it or > > finding it. > > I don't know about that, all of the methods that int and long > currently have are __special__. They'd really need to start with > Python 2.5 (assuming int/long grow "public methods" in 2.5) to even > think to look there. A format code or a built-in would be more > likely to be found, since that's how you convert integers to hex and > oct string representations with current Python. > > >>> [name for name in dir(0)+dir(0L) if not name.startswith('__')] If a method is the best solution, then fine, 2.5 is the beginning of methods on int/long. We could do a static method like int.from_str("101", 2) and str.from_int(5, 2) if people don't like the overloading of the constructors. Otherwise add methods like '101'.to_int(2) or 5 .to_str(2) . -Brett From martin at v.loewis.de Tue Jan 17 07:46:17 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 17 Jan 2006 07:46:17 +0100 Subject: [Python-Dev] basenumber redux In-Reply-To: <001901c61b17$52c0c280$e221cb97@oemcomputer> References: <001901c61b17$52c0c280$e221cb97@oemcomputer> Message-ID: <43CC92B9.2030709@v.loewis.de> Raymond Hettinger wrote: > Are you sure? The former has two builtin lookups (which also entail two > failed global lookups), a function call, and a test/jump for the result. > The latter approach has no lookups (just a load constant), a try-block > setup, an add operation (optimized for integers, a fast slot lookup > otherwise), and a block end. > > Even if there were a performance edge, I suppose that the type checking > is the time critical part of most scripts. My guess is that it depends on the common case. If the common case is that the type test fails (i.e. element-wise operations are the exception), then you also have the exception creation and storing in the exception approach, compared to returning only existing objects in the type test case. If the common case is that x+0 succeeds, x+0 may or may not create new objects. However, I have long ago learned not to guess about performance, so I won't do further guessing until I see the actual code. Regards, Martin From martin at v.loewis.de Tue Jan 17 09:21:59 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 17 Jan 2006 09:21:59 +0100 Subject: [Python-Dev] str with base In-Reply-To: References: Message-ID: <43CCA927.8030600@v.loewis.de> Alex Martelli wrote: > Is it finally time in Python 2.5 to allow the "obvious" use of, say, > str(5,2) to give '101', just the converse of the way int('101',1) > gives 5? I'm not sure why str has never allowed this obvious use -- > any bright beginner assumes it's there and it's awkward to explain > why it's not!-). My main concern is what the impact on __str__ would be. It seems "obvious" that def str(obj, *args): return obj.__str__(*args) because it is ultimately int's responsibility to interpret the base argument, not str's. People would then come up with use cases like class Color: msg = {'en':['red', 'green', 'blue'], 'de':['rot','gr?n','blau']} def __str__(self, language='en'): return self.msg[language][self.value] red = Color(0) so you could say print str(red, 'de') I don't think I like this direction. Regards, Martin From abo at minkirri.apana.org.au Tue Jan 17 09:52:55 2006 From: abo at minkirri.apana.org.au (Donovan Baarda) Date: Tue, 17 Jan 2006 08:52:55 +0000 Subject: [Python-Dev] str with base In-Reply-To: <1137477812.5351.91.camel@geddy.wooz.org> References: <20060117040802.GG7771@home.puzzling.org> <1137471491.5352.87.camel@geddy.wooz.org> <501A02B9-738E-4334-8621-50E0E1AB3736@redivi.com> <1137477812.5351.91.camel@geddy.wooz.org> Message-ID: <1137487975.4342.0.camel@warna.dub.corp.google.com> On Tue, 2006-01-17 at 01:03 -0500, Barry Warsaw wrote: > On Mon, 2006-01-16 at 20:49 -0800, Bob Ippolito wrote: > > > The only bases I've ever really had a good use for are 2, 8, 10, and > > 16. There are currently formatting codes for 8 (o), 10 (d, u), and > > 16 (x, X). Why not just add a string format code for unsigned > > binary? The obvious choice is probably "b". > > > > For example: > > > > >>> '%08b' % (12) > > '00001100' > > >>> '%b' % (12) > > '1100' > > +1 +1 me too. -- Donovan Baarda http://minkirri.apana.org.au/~abo/ From mal at egenix.com Tue Jan 17 11:02:22 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 17 Jan 2006 11:02:22 +0100 Subject: [Python-Dev] basenumber redux In-Reply-To: <37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com> References: <17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com> <43CC17B9.3060200@v.loewis.de> <37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com> Message-ID: <43CCC0AE.5070800@egenix.com> Alex, I think you're missing a point here: what you are looking for is an interface, not a base class - simply because the assumptions you make when finding a "KnownNumberTypes" instance are only related to an interface you expect them to provide. A common case class won't really help all that much with this, since the implementations of the different types will vary a lot (unlike, for example, strings and Unicode, which implement a very common interface) and not necessarily provide a common interface. If you look at the Python C API, you'll find that "a number" is actually never tested. The tests always ask for either integers or floats. The addition of a basenumber base class won't make these any simpler. Here's a snippet which probably does what you're looking for using Python's natural way of hooking up to an implicit interface: import UserString STRING_TYPES = (basestring, UserString.UserString) def floatnumber(obj): if isinstance(obj, STRING_TYPES): raise TypeError('strings are not numbers') # Convert to a float try: return float(obj) except (AttributeError, TypeError, ValueError): raise TypeError('%r is not a float' % obj) def intnumber(obj): if isinstance(obj, STRING_TYPES): raise TypeError('strings are not numbers') # Convert to an integer try: value = int(obj) except (AttributeError, TypeError, ValueError): raise TypeError('%r is not an integer' % obj) # Double check so that we don't lose precision try: floatvalue = floatnumber(obj) except TypeError: return value if floatvalue != value: raise TypeError('%r is not an integer' % obj) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 17 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From nick at craig-wood.com Tue Jan 17 11:05:44 2006 From: nick at craig-wood.com (Nick Craig-Wood) Date: Tue, 17 Jan 2006 10:05:44 +0000 Subject: [Python-Dev] str with base In-Reply-To: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> Message-ID: <20060117100544.GC1419@craig-wood.com> On Mon, Jan 16, 2006 at 11:13:27PM -0500, Raymond Hettinger wrote: > My reason is that I've rolled-my-own more times than I can count but > infrequently enough to where it was easier to re-write than to search > for the previous use. Me too! The assymetry is annoying. Its easy to consume base 2..36 integers, but its hard to generate them. However str() seems far too important to monkey with to me. I like a method on int that would be great. That keeps all the base conversions in int (either in __init__ or in as_yet_unnamed_method()). Another suggestion would be to give hex() and oct() another parameter, base, so you'd do hex(123123123, 2). Perhaps a little counter-intuitive, but if you were looking for base conversion functions you'd find hex() pretty quickly and the documentation would mention the other parameter. -- Nick Craig-Wood -- http://www.craig-wood.com/nick From mal at egenix.com Tue Jan 17 11:30:37 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 17 Jan 2006 11:30:37 +0100 Subject: [Python-Dev] str with base In-Reply-To: References: Message-ID: <43CCC74D.80105@egenix.com> Alex Martelli wrote: > Is it finally time in Python 2.5 to allow the "obvious" use of, say, > str(5,2) to give '101', just the converse of the way int('101',1) > gives 5? I'm not sure why str has never allowed this obvious use -- > any bright beginner assumes it's there and it's awkward to explain > why it's not!-). I'll be happy to propose a patch if the BDFL > blesses this, but I don't even think it's worth a PEP... it's an > inexplicable though long-standing omission (given the argumentative > nature of this crowd I know I'll get pushback, but I still hope the > BDFL can Pronounce about it anyway;-). Hmm, how about this: str(obj, ifisunicode_decode_using_encoding='ascii', ifisinteger_use_base=10, ifisfile_open_and_read_it=False, isdecimal_use_precision=10, ismoney_use_currency='EUR', isdatetime_use_format='%c') and so on ?! Or even better: str(obj, **kws) and then call obj.__str__(**kws) instead of just obj.__str__() ?! Seriously, shouldn't these more specific "convert to a string" functions be left to specific object methods or helper functions ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 17 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From ianb at colorstudy.com Tue Jan 17 11:36:54 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 17 Jan 2006 04:36:54 -0600 Subject: [Python-Dev] str with base In-Reply-To: <771D4186-062D-446A-9621-95FD6048FCEB@redivi.com> References: <003b01c61b22$0bbfe040$e221cb97@oemcomputer> <20060117051253.GH7771@home.puzzling.org> <771D4186-062D-446A-9621-95FD6048FCEB@redivi.com> Message-ID: <43CCC8C6.8060503@colorstudy.com> Bob Ippolito wrote: > On Jan 16, 2006, at 9:12 PM, Andrew Bennetts wrote: > > >>On Mon, Jan 16, 2006 at 11:54:05PM -0500, Raymond Hettinger wrote: >>[...] >> >>>That suggests that it would be better to simply add an int method: >>> >>> x.convert_to_base(7) >> >>This seems clear and simple to me. I like it. I strongly suspect >>the "bright >>beginners" Alex is interested in would have no trouble using it or >>finding it. > > > I don't know about that, all of the methods that int and long > currently have are __special__. They'd really need to start with > Python 2.5 (assuming int/long grow "public methods" in 2.5) to even > think to look there. A format code or a built-in would be more > likely to be found, since that's how you convert integers to hex and > oct string representations with current Python. How about just stuffing some function in the math module? Everything in that module works on floats, but it seems incidental to me; I'm pretty sure I've even looked in that module for such a function before. But it's such an obscure need that only comes up in the kind of algorithms students write, that it just seems odd and unnecessary to put it in str() which is *so* much more general than int() is. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From bob at redivi.com Tue Jan 17 11:44:05 2006 From: bob at redivi.com (Bob Ippolito) Date: Tue, 17 Jan 2006 02:44:05 -0800 Subject: [Python-Dev] str with base In-Reply-To: <43CCC8C6.8060503@colorstudy.com> References: <003b01c61b22$0bbfe040$e221cb97@oemcomputer> <20060117051253.GH7771@home.puzzling.org> <771D4186-062D-446A-9621-95FD6048FCEB@redivi.com> <43CCC8C6.8060503@colorstudy.com> Message-ID: On Jan 17, 2006, at 2:36 AM, Ian Bicking wrote: > Bob Ippolito wrote: >> On Jan 16, 2006, at 9:12 PM, Andrew Bennetts wrote: >>> On Mon, Jan 16, 2006 at 11:54:05PM -0500, Raymond Hettinger wrote: >>> [...] >>> >>>> That suggests that it would be better to simply add an int method: >>>> >>>> x.convert_to_base(7) >>> >>> This seems clear and simple to me. I like it. I strongly >>> suspect the "bright >>> beginners" Alex is interested in would have no trouble using it >>> or finding it. >> I don't know about that, all of the methods that int and long >> currently have are __special__. They'd really need to start with >> Python 2.5 (assuming int/long grow "public methods" in 2.5) to >> even think to look there. A format code or a built-in would be >> more likely to be found, since that's how you convert integers to >> hex and oct string representations with current Python. > > How about just stuffing some function in the math module? > Everything in that module works on floats, but it seems incidental > to me; I'm pretty sure I've even looked in that module for such a > function before. But it's such an obscure need that only comes up > in the kind of algorithms students write, that it just seems odd > and unnecessary to put it in str() which is *so* much more general > than int() is. I want binary all the time when I'm dealing with bitflags and such. Of course, I'm trained to be able to read bits in hex format, but it would be nicer to see the flags as-is. Even worse when you have to deal with some kind of file format where fields are N bits long, where N is not a multiple of 8. -bob From fredrik at pythonware.com Tue Jan 17 11:48:04 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue, 17 Jan 2006 11:48:04 +0100 Subject: [Python-Dev] str with base References: <003b01c61b22$0bbfe040$e221cb97@oemcomputer> <20060117051253.GH7771@home.puzzling.org><771D4186-062D-446A-9621-95FD6048FCEB@redivi.com><43CCC8C6.8060503@colorstudy.com> Message-ID: Bob Ippolito wrote: > I want binary all the time when I'm dealing with bitflags and such. > Of course, I'm trained to be able to read bits in hex format, but it > would be nicer to see the flags as-is. Even worse when you have to > deal with some kind of file format where fields are N bits long, > where N is not a multiple of 8. so you want flags for bit order and fill order too, I assume ? From arigo at tunes.org Tue Jan 17 11:14:02 2006 From: arigo at tunes.org (Armin Rigo) Date: Tue, 17 Jan 2006 11:14:02 +0100 Subject: [Python-Dev] Summer of PyPy In-Reply-To: References: <-8397309676539309595@unknownmsgid> <20060114101454.GA29087@code1.codespeak.net> Message-ID: <20060117101402.GA6458@code1.codespeak.net> Hi Brett, hi all, On Sat, Jan 14, 2006 at 05:51:25PM -0800, Brett Cannon wrote: > That would be cool! I definitely would not mind working on PyPy. > Unfortunately I would not consider changing universities; I really > like it here. We are looking at the possibility to do a "Summer of PyPy" in the same style as last year's Google's Summer of Code. It might be a way for you (or anybody else interested!) to get to work a bit on PyPy first :-) http://codespeak.net/pipermail/pypy-dev/2006q1/002721.html A bientot, Armin From bob at redivi.com Tue Jan 17 12:12:17 2006 From: bob at redivi.com (Bob Ippolito) Date: Tue, 17 Jan 2006 03:12:17 -0800 Subject: [Python-Dev] str with base In-Reply-To: References: <003b01c61b22$0bbfe040$e221cb97@oemcomputer> <20060117051253.GH7771@home.puzzling.org><771D4186-062D-446A-9621-95FD6048FCEB@redivi.com><43CCC8C6.8060503@colorstudy.com> Message-ID: On Jan 17, 2006, at 2:48 AM, Fredrik Lundh wrote: > Bob Ippolito wrote: > >> I want binary all the time when I'm dealing with bitflags and such. >> Of course, I'm trained to be able to read bits in hex format, but it >> would be nicer to see the flags as-is. Even worse when you have to >> deal with some kind of file format where fields are N bits long, >> where N is not a multiple of 8. > > so you want flags for bit order and fill order too, I assume ? Not really, big endian covers almost everything I need.. and when it doesn't, then I can just flip and/or pad the string accordingly. -bob From abo at minkirri.apana.org.au Tue Jan 17 12:54:58 2006 From: abo at minkirri.apana.org.au (Donovan Baarda) Date: Tue, 17 Jan 2006 11:54:58 +0000 Subject: [Python-Dev] str with base In-Reply-To: <20060117100544.GC1419@craig-wood.com> References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> <20060117100544.GC1419@craig-wood.com> Message-ID: <1137498898.4342.68.camel@warna.dub.corp.google.com> On Tue, 2006-01-17 at 10:05 +0000, Nick Craig-Wood wrote: > On Mon, Jan 16, 2006 at 11:13:27PM -0500, Raymond Hettinger wrote: [...] > Another suggestion would be to give hex() and oct() another parameter, > base, so you'd do hex(123123123, 2). Perhaps a little > counter-intuitive, but if you were looking for base conversion > functions you'd find hex() pretty quickly and the documentation would > mention the other parameter. Ugh! I still favour extending % format strings. I really like '%b' for binary, but if arbitary bases are really wanted, then perhaps also leverage off the "precision" value for %d to indicate base such that '% 3.3d' % 5 = " 12" If people think that using "." is for "precision" and is too ambiguous for "base", you could do something like extend the whole conversion specifier to (in EBNF) conversion=%[mapping][flags][width][.precision][@base][modifier]type which would allow for weird things like "%8.4 at 3f" % 5.5 == " 12.1111" Note: it is possible for floats to be represented in non-decimal number systems, its just extremely rare for anyone to do it. I have in my distant past used base 16 float notation for fixed-point numbers. I personally think %b would be adding enough. The other suggestions are just me being silly :-) -- Donovan Baarda http://minkirri.apana.org.au/~abo/ From mwh at python.net Tue Jan 17 13:00:52 2006 From: mwh at python.net (Michael Hudson) Date: Tue, 17 Jan 2006 12:00:52 +0000 Subject: [Python-Dev] str with base In-Reply-To: <1137498898.4342.68.camel@warna.dub.corp.google.com> (Donovan Baarda's message of "Tue, 17 Jan 2006 11:54:58 +0000") References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> <20060117100544.GC1419@craig-wood.com> <1137498898.4342.68.camel@warna.dub.corp.google.com> Message-ID: <2mek37oyh7.fsf@starship.python.net> Donovan Baarda writes: > I personally think %b would be adding enough. The other suggestions are > just me being silly :-) Yeah, the whole area is just crying out for the simplicity and restraint that is common lisp's #'format function :) Cheers, mwh -- INEFFICIENT CAPITALIST YOUR OPULENT TOILET WILL BE YOUR UNDOING -- from Twisted.Quotes From collinw at gmail.com Tue Jan 17 14:12:53 2006 From: collinw at gmail.com (Collin Winter) Date: Tue, 17 Jan 2006 14:12:53 +0100 Subject: [Python-Dev] basenumber redux In-Reply-To: <001901c61b17$52c0c280$e221cb97@oemcomputer> References: <37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com> <001901c61b17$52c0c280$e221cb97@oemcomputer> Message-ID: <43aa6ff70601170512y30dfe3d0ib420827ef5d9df5@mail.gmail.com> On 1/17/06, Raymond Hettinger wrote: [snip] > I don't see a way around creating an integer recognition tool that > doesn't conflate its terminology with broadly-held, pre-existing math > knowledge: complex is a superset of reals, reals include rationals and > irrationals some of which are trancendental, and rationals include > integers which are an infinite superset of non-negative integers, whole > numbers, negative numbers, etc. > > The decimal class only makes this more complicated. All binary floats > can be translated exactly to decimal but not vice-versa. I'm not sure > where they would fit into a inheritance hierarchy. To repeat a popular suggestion these days, python might borrow a page from Haskell. Haskell's Prelude_ defines a number (pardon the pun) of numeric typeclasses, each of which requires certain members. The inheritance graph shapes up roughly like this: Num - the ur-base class for all numbers Real - inherits from Num Integral - inherits from Real. Integral numbers support integer division Fractional - inherits from Num. Fractionals support true division Floating - inherits from Fractional. Floating-class objects support trigonometric and hyperbolic functions and related functions. RealFrac - inherits from Real and Fractional. This is used to operate on the components of fractions. RealFloat - inherits from RealFrac and Floating, providing efficient, machine-independent access to the components of a floating-point number. While it may not be necessary to concoct that many base classes for python, having a solid selection of such classes to subclass would reduce the need for heuristics like attribute testing. Moreover, subclassing a (or several) numeric type classes makes your intentions explicit, rather than relying on testing for an "implicit interface". Given the impact this would have on legacy code, and the need to refit the built-in types and standard library, such a chance might be better put off until Python 3k. _Prelude - http://www.haskell.org/ghc/docs/latest/html/libraries/base/Prelude.html Thanks, Collin Winter From jason.orendorff at gmail.com Tue Jan 17 15:23:29 2006 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Tue, 17 Jan 2006 09:23:29 -0500 Subject: [Python-Dev] str with base In-Reply-To: <2mek37oyh7.fsf@starship.python.net> References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> <20060117100544.GC1419@craig-wood.com> <1137498898.4342.68.camel@warna.dub.corp.google.com> <2mek37oyh7.fsf@starship.python.net> Message-ID: It seems dumb to support *parsing* integers in weird bases, but not *formatting* them in weird bases. Not a big deal, but if you're going to give me a toy, at least give me the whole toy! The %b idea is a little disappointing in two ways. Even with %b, Python is still dumb by the above criterion. And, I suspect users that don't know about %b are unlikely to find it when they want it. I know I've never looked for it there. I think a method 5664400.to_base(13) sounds nice. -j From andrew-pythondev at puzzling.org Tue Jan 17 15:30:48 2006 From: andrew-pythondev at puzzling.org (Andrew Bennetts) Date: Wed, 18 Jan 2006 01:30:48 +1100 Subject: [Python-Dev] str with base In-Reply-To: References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> <20060117100544.GC1419@craig-wood.com> <1137498898.4342.68.camel@warna.dub.corp.google.com> <2mek37oyh7.fsf@starship.python.net> Message-ID: <20060117143048.GO7771@home.puzzling.org> On Tue, Jan 17, 2006 at 09:23:29AM -0500, Jason Orendorff wrote: > It seems dumb to support *parsing* integers in weird bases, but not > *formatting* them in weird bases. Not a big deal, but if you're going > to give me a toy, at least give me the whole toy! > > The %b idea is a little disappointing in two ways. Even with %b, > Python is still dumb by the above criterion. And, I suspect users > that don't know about %b are unlikely to find it when they want it. I > know I've never looked for it there. > > I think a method 5664400.to_base(13) sounds nice. It's also a SyntaxError. With the current syntax, it would need to be "(5664400).to_base(13)" or "5664400 .to_base(13)". Why not just make the %b take a base, e.g.: '%13b' % 5664400 -Andrew. From jimjjewett at gmail.com Tue Jan 17 15:49:46 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Tue, 17 Jan 2006 09:49:46 -0500 Subject: [Python-Dev] index (was str with base) Message-ID: Guido wrote: > more important to implement __index__() in Python 2.5. > This behaves like __int__() for integral types, but is not > defined for float or Decimal. Why not for Decimal, or even float? I would not be surprised if 10.798 failed, but I would expect 1000D to work. If indexing worked with more arbitrary extension integers, then I would expect it work with Decimal, and possibly float, when the number == a whole number, and to raise ValueError otherwise. -jJ From jimjjewett at gmail.com Tue Jan 17 15:51:58 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Tue, 17 Jan 2006 09:51:58 -0500 Subject: [Python-Dev] str with base Message-ID: Raymond wrote: > I presume that only the str() builtin would > change and that the underlying __str__ slot > would continue to be hard-wired to the > (reprfunc) signature. Are you saying that subclasses of str should behave differently from the builtin str, and not in the slots that were added or changed? -jJ From skip at pobox.com Tue Jan 17 16:17:36 2006 From: skip at pobox.com (skip at pobox.com) Date: Tue, 17 Jan 2006 09:17:36 -0600 Subject: [Python-Dev] str with base In-Reply-To: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> Message-ID: <17357.2704.438248.183461@montanaro.dyndns.org> Raymond> My reason is that I've rolled-my-own more times than I can Raymond> count but infrequently enough to where it was easier to Raymond> re-write than to search for the previous use. Maybe a bin() builtin would be better. Even better it seems to me would be to add a method to ints and longs that returns a string formatted in a base between 2 and 36 (then deprecate hex and oct). Like Jeremy, I wonder what str([1,2], 4) means. Skip From skip at pobox.com Tue Jan 17 16:19:35 2006 From: skip at pobox.com (skip at pobox.com) Date: Tue, 17 Jan 2006 09:19:35 -0600 Subject: [Python-Dev] str with base In-Reply-To: References: Message-ID: <17357.2823.765518.514875@montanaro.dyndns.org> Alex> Identically the same situation as for int: the base argument is Alex> only accepted if the first argument is a str (not a float, etc). Alex> Just the same way, the base argument to str will only be accepted Alex> if the first argument is an int (not a float, etc). A shortcoming in int() hardly seems like a good reason to mess with str(). Skip From gmccaughan at synaptics-uk.com Tue Jan 17 16:22:37 2006 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Tue, 17 Jan 2006 15:22:37 +0000 Subject: [Python-Dev] str with base In-Reply-To: <17357.2823.765518.514875@montanaro.dyndns.org> References: <17357.2823.765518.514875@montanaro.dyndns.org> Message-ID: <200601171522.38184.gmccaughan@synaptics-uk.com> On Tuesday 2006-01-17 15:19, skip at pobox.com wrote: > Alex> Identically the same situation as for int: the base argument is > Alex> only accepted if the first argument is a str (not a float, etc). > Alex> Just the same way, the base argument to str will only be accepted > Alex> if the first argument is an int (not a float, etc). > > A shortcoming in int() hardly seems like a good reason to mess with str(). How's it a shortcoming in int() that it doesn't do anything with, say, int(2.345,19)? (What would you like it to do?) Or are you saying that the fact that int() lets you specify a base to interpret the string in is itself a shortcoming, and if so why? -- g From guido at python.org Tue Jan 17 16:26:57 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 17 Jan 2006 07:26:57 -0800 Subject: [Python-Dev] index (was str with base) In-Reply-To: References: Message-ID: On 1/17/06, Jim Jewett wrote: > Guido wrote: > > > more important to implement __index__() in Python 2.5. > > This behaves like __int__() for integral types, but is not > > defined for float or Decimal. > > Why not for Decimal, or even float? I would not be surprised > if 10.798 failed, but I would expect 1000D to work. > > If indexing worked with more arbitrary extension integers, > then I would expect it work with Decimal, and possibly float, > when the number == a whole number, and to raise ValueError > otherwise. Sorry, I forgot to explain this. There's a very good reason why this should not be allowed. You don't know if the 1000D was the result of an exact or of an approximate calculation. It could be a pure coincidence that it's 1000D instead of 999.9D; and we certainly don't want the latter to be allowed for indexing. Floating point (including Decimal floating point) calculations are fraught with uncertainties about the precision / accuracy of the outcome. Even if when you test your algorithm you always get exact outcomes, there's no guarantee that with real input data the same will hold. Requiring the index to be an int by *type* solves this by forcing you to think about rounding/truncating. I talked to some Numeric folks and they understand and agree. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From skip at pobox.com Tue Jan 17 16:38:42 2006 From: skip at pobox.com (skip at pobox.com) Date: Tue, 17 Jan 2006 09:38:42 -0600 Subject: [Python-Dev] str with base In-Reply-To: <200601171522.38184.gmccaughan@synaptics-uk.com> References: <17357.2823.765518.514875@montanaro.dyndns.org> <200601171522.38184.gmccaughan@synaptics-uk.com> Message-ID: <17357.3970.537715.827114@montanaro.dyndns.org> Alex> Identically the same situation as for int: the base argument is Alex> only accepted if the first argument is a str (not a float, etc). Alex> Just the same way, the base argument to str will only be accepted Alex> if the first argument is an int (not a float, etc). >> Skip> A shortcoming in int() hardly seems like a good reason to mess Skip> with str(). Gareth> How's it a shortcoming in int() that it doesn't do anything Gareth> with, say, int(2.345,19)? My reasoning was that just because int() was written to ignore the second arg depending on type (the "shortcoming") doesn't mean that str() should as well. Skip From fredrik at pythonware.com Tue Jan 17 17:27:38 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue, 17 Jan 2006 17:27:38 +0100 Subject: [Python-Dev] str with base References: <17357.2823.765518.514875@montanaro.dyndns.org><200601171522.38184.gmccaughan@synaptics-uk.com> <17357.3970.537715.827114@montanaro.dyndns.org> Message-ID: skip at pobox.com wrote: > Skip> A shortcoming in int() hardly seems like a good reason to mess > Skip> with str(). > > Gareth> How's it a shortcoming in int() that it doesn't do anything > Gareth> with, say, int(2.345,19)? > > My reasoning was that just because int() was written to ignore the second > arg depending on type (the "shortcoming") doesn't mean that str() should as > well. "ignore" is perhaps the wrong word: >>> int(1.0, 1) Traceback (most recent call last): File "", line 1, in ? TypeError: int() can't convert non-string with explicit base From rhamph at gmail.com Tue Jan 17 17:48:08 2006 From: rhamph at gmail.com (Adam Olsen) Date: Tue, 17 Jan 2006 09:48:08 -0700 Subject: [Python-Dev] index (was str with base) In-Reply-To: References: Message-ID: On 1/17/06, Guido van Rossum wrote: > On 1/17/06, Jim Jewett wrote: > > Guido wrote: > > > > > more important to implement __index__() in Python 2.5. > > > This behaves like __int__() for integral types, but is not > > > defined for float or Decimal. > > > > Why not for Decimal, or even float? I would not be surprised > > if 10.798 failed, but I would expect 1000D to work. > > > > If indexing worked with more arbitrary extension integers, > > then I would expect it work with Decimal, and possibly float, > > when the number == a whole number, and to raise ValueError > > otherwise. > > Sorry, I forgot to explain this. There's a very good reason why this > should not be allowed. > > You don't know if the 1000D was the result of an exact or of an > approximate calculation. It could be a pure coincidence that it's > 1000D instead of 999.9D; and we certainly don't want the latter to be > allowed for indexing. Floating point (including Decimal floating > point) calculations are fraught with uncertainties about the precision > / accuracy of the outcome. Even if when you test your algorithm you > always get exact outcomes, there's no guarantee that with real input > data the same will hold. > > Requiring the index to be an int by *type* solves this by forcing you > to think about rounding/truncating. I talked to some Numeric folks and > they understand and agree. IMO this is better done by replacing float with an interval type, or atleast float context objects with explicit rounding and precision control. The former would tell you when the result is inexact (the gap or "length" is non-zero), the latter would let you pick rounding behavior when you know enough math to back it up. And rather than leave you all hanging I'll say that I believe it could be done by using LLVM (circumventing C entierly.) http://rhamphoryncus.blogspot.com/2006/01/floating-point-and-nans-in-python.html (Why have float at all if the documentation essentially says it channels /dev/urandom?) -- Adam Olsen, aka Rhamphoryncus From rhamph at gmail.com Tue Jan 17 18:04:35 2006 From: rhamph at gmail.com (Adam Olsen) Date: Tue, 17 Jan 2006 10:04:35 -0700 Subject: [Python-Dev] str with base In-Reply-To: <43CCA927.8030600@v.loewis.de> References: <43CCA927.8030600@v.loewis.de> Message-ID: On 1/17/06, "Martin v. L?wis" wrote: > class Color: > msg = {'en':['red', 'green', 'blue'], 'de':['rot','gr?n','blau']} > def __str__(self, language='en'): > return self.msg[language][self.value] > > red = Color(0) > > so you could say > > print str(red, 'de') > > I don't think I like this direction. I agree that *args makes the code non-obvious. However, if **kwargs is used instead: def str(obj, **kwargs): return obj.__str__(**kwargs) class Color: msg = {'en':['red', 'green', 'blue'], 'de':['rot','gr?n','blau']} def __str__(self, language='en'): return self.msg[language][self.value] red = Color(0) print str(red, language='de') I find that quite readable. -- Adam Olsen, aka Rhamphoryncus From aahz at pythoncraft.com Tue Jan 17 18:04:47 2006 From: aahz at pythoncraft.com (Aahz) Date: Tue, 17 Jan 2006 09:04:47 -0800 Subject: [Python-Dev] str with base In-Reply-To: References: Message-ID: <20060117170447.GA5411@panix.com> On Mon, Jan 16, 2006, Alex Martelli wrote: > > Is it finally time in Python 2.5 to allow the "obvious" use of, say, > str(5,2) to give '101', just the converse of the way int('101',1) > gives 5? I'm not sure why str has never allowed this obvious use -- > any bright beginner assumes it's there and it's awkward to explain > why it's not!-). I'll be happy to propose a patch if the BDFL > blesses this, but I don't even think it's worth a PEP... it's an > inexplicable though long-standing omission (given the argumentative > nature of this crowd I know I'll get pushback, but I still hope the > BDFL can Pronounce about it anyway;-). -1 I agree with all the other comments about the functional asymmetry between int() and str() in the Python universe, and that therefore str() shouldn't necessarily mimic int()'s API. Propose some other mechanism; I so far haven't seen a good reasons to prefer any of the ones already proposed. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From theller at python.net Tue Jan 17 19:17:59 2006 From: theller at python.net (Thomas Heller) Date: Tue, 17 Jan 2006 19:17:59 +0100 Subject: [Python-Dev] Building on OS X 10.4 fails Message-ID: Building the readline on OS X 10.4 fails, is this known, or am I doing something wrong? Thanks, Thomas building 'readline' extension gcc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -I/Users/theller/svn/trunk/./Include -I/Users/theller/svn/trunk/./Mac/Include -I./Include -I. -I/Users/theller/svn/trunk/Include -I/Users/theller/svn/trunk -c /Users/theller/svn/trunk/Modules/readline.c -o build/temp.darwin-8.4.0-Power_Macintosh-2.5/readline.o /Users/theller/svn/trunk/Modules/readline.c: In function 'write_history_file': /Users/theller/svn/trunk/Modules/readline.c:112: warning: implicit declaration of function 'history_truncate_file' /Users/theller/svn/trunk/Modules/readline.c: In function 'py_remove_history': /Users/theller/svn/trunk/Modules/readline.c:301: warning: implicit declaration of function 'remove_history' /Users/theller/svn/trunk/Modules/readline.c:301: warning: assignment makes pointer from integer without a cast /Users/theller/svn/trunk/Modules/readline.c:310: warning: passing argument 1 of 'free' discards qualifiers from pointer target type /Users/theller/svn/trunk/Modules/readline.c:312: warning: passing argument 1 of 'free' discards qualifiers from pointer target type /Users/theller/svn/trunk/Modules/readline.c: In function 'py_replace_history': /Users/theller/svn/trunk/Modules/readline.c:338: warning: implicit declaration of function 'replace_history_entry' /Users/theller/svn/trunk/Modules/readline.c:338: warning: assignment makes pointer from integer without a cast /Users/theller/svn/trunk/Modules/readline.c:347: warning: passing argument 1 of 'free' discards qualifiers from pointer target type /Users/theller/svn/trunk/Modules/readline.c:349: warning: passing argument 1 of 'free' discards qualifiers from pointer target type /Users/theller/svn/trunk/Modules/readline.c: In function 'get_current_history_length': /Users/theller/svn/trunk/Modules/readline.c:453: error: 'HISTORY_STATE' undeclared (first use in this function) /Users/theller/svn/trunk/Modules/readline.c:453: error: (Each undeclared identifier is reported only once /Users/theller/svn/trunk/Modules/readline.c:453: error: for each function it appears in.) /Users/theller/svn/trunk/Modules/readline.c:453: error: 'hist_st' undeclared (first use in this function) /Users/theller/svn/trunk/Modules/readline.c:455: warning: implicit declaration of function 'history_get_history_state' /Users/theller/svn/trunk/Modules/readline.c: In function 'insert_text': /Users/theller/svn/trunk/Modules/readline.c:503: warning: implicit declaration of function 'rl_insert_text' /Users/theller/svn/trunk/Modules/readline.c: In function 'on_completion': /Users/theller/svn/trunk/Modules/readline.c:637: error: 'rl_attempted_completion_over' undeclared (first use in this function) /Users/theller/svn/trunk/Modules/readline.c: In function 'flex_complete': /Users/theller/svn/trunk/Modules/readline.c:675: warning: passing argument 2 of 'completion_matches' from incompatible pointer type /Users/theller/svn/trunk/Modules/readline.c: In function 'setup_readline': /Users/theller/svn/trunk/Modules/readline.c:700: warning: passing argument 2 of 'rl_bind_key_in_map' from incompatible pointer type /Users/theller/svn/trunk/Modules/readline.c:701: warning: passing argument 2 of 'rl_bind_key_in_map' from incompatible pointer type /Users/theller/svn/trunk/Modules/readline.c: In function 'readline_until_enter_or_signal': /Users/theller/svn/trunk/Modules/readline.c:758: warning: passing argument 2 of 'rl_callback_handler_install' from incompatible pointer type /Users/theller/svn/trunk/Modules/readline.c:788: warning: implicit declaration of function 'rl_free_line_state' /Users/theller/svn/trunk/Modules/readline.c:789: warning: implicit declaration of function 'rl_cleanup_after_signal' /Users/theller/svn/trunk/Modules/readline.c: In function 'call_readline': /Users/theller/svn/trunk/Modules/readline.c:883: error: 'HISTORY_STATE' undeclared (first use in this function) /Users/theller/svn/trunk/Modules/readline.c:883: error: 'state' undeclared (first use in this function) /Users/theller/svn/trunk/Modules/readline.c:885: warning: assignment discards qualifiers from pointer target type From g.brandl-nospam at gmx.net Tue Jan 17 19:35:18 2006 From: g.brandl-nospam at gmx.net (Georg Brandl) Date: Tue, 17 Jan 2006 19:35:18 +0100 Subject: [Python-Dev] Python icon In-Reply-To: <17354.20722.560136.55434@montanaro.dyndns.org> References: <17354.20722.560136.55434@montanaro.dyndns.org> Message-ID: skip at pobox.com wrote: > >> does Python have an official icon? > > Ping> i found some images at http://www.pythonology.com/logos... > > It appears the yin/yang Python's on that page are being used in the new site > (beta.python.org). I don't know if that makes it official or not though. Interesting, didn't even know a new page was in the making... Do you know who is responsible for the new page? regards, Georg From skip at pobox.com Tue Jan 17 19:45:58 2006 From: skip at pobox.com (skip at pobox.com) Date: Tue, 17 Jan 2006 12:45:58 -0600 Subject: [Python-Dev] Python icon In-Reply-To: References: <17354.20722.560136.55434@montanaro.dyndns.org> Message-ID: <17357.15206.570762.861782@montanaro.dyndns.org> Georg> Interesting, didn't even know a new page was in the making... Do Georg> you know who is responsible for the new page? Tim Parkin is heading things up. Look here: http://beta.python.org/ Skip From arigo at tunes.org Tue Jan 17 19:09:05 2006 From: arigo at tunes.org (Armin Rigo) Date: Tue, 17 Jan 2006 19:09:05 +0100 Subject: [Python-Dev] str with base Message-ID: <20060117180905.GA15740@code1.codespeak.net> Hi Alex, > Is it finally time in Python 2.5 to allow the "obvious" use of, say, > str(5,2) to give '101', just the converse of the way int('101',1) > gives 5? +1. That's obvious enough for me. Clearly it should only work with int and long arguments, just like int(x,base) only works if x is a str or unicode argument (and not something that can be converted to a string, or even a stringish type for some definition of stringish). Armin From bob at redivi.com Tue Jan 17 20:17:38 2006 From: bob at redivi.com (Bob Ippolito) Date: Tue, 17 Jan 2006 11:17:38 -0800 Subject: [Python-Dev] Building on OS X 10.4 fails In-Reply-To: References: Message-ID: <3B20C03B-93A2-4C24-9D6C-67ADBD10A801@redivi.com> On Jan 17, 2006, at 10:17 AM, Thomas Heller wrote: > Building the readline on OS X 10.4 fails, is this known, or am I doing > something wrong? Mac OS X doesn't ship with readline. It ships with BSD libedit symlinked to readline. Not good enough for Python. You need a third party copy. I made an egg for it a while ago: http://python.org/pypi/readline -bob From barry at python.org Tue Jan 17 20:19:47 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 17 Jan 2006 14:19:47 -0500 Subject: [Python-Dev] Building on OS X 10.4 fails In-Reply-To: References: Message-ID: <1137525587.9528.19.camel@geddy.wooz.org> On Tue, 2006-01-17 at 19:17 +0100, Thomas Heller wrote: > Building the readline on OS X 10.4 fails, is this known, or am I doing > something wrong? There are definitely serious issues with readline on OS X 10.4. I've hit them too but haven't had time to post about it yet. I'm far from an expert on them (Bob, Jack and others can definitely speak with more knowledge on this) but my understanding is that the default readline that comes with Tiger is not sufficient to build the Python extension library. However, I use DarwinPorts and IWBNI Python's build process could search /opt/local for the necessary readline support, using that if found. I know that DP's Python 2.4.2 build does provide readline, so that should definitely work. I've glanced at the setup.py file, but haven't had the time to dig in in enough detail to actually fix and commit. (There was something else that bugged me about the OS X 10.4 Python from-source build but now I can't remember.) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060117/01f4dc80/attachment.pgp From theller at python.net Tue Jan 17 20:32:59 2006 From: theller at python.net (Thomas Heller) Date: Tue, 17 Jan 2006 20:32:59 +0100 Subject: [Python-Dev] Include ctypes into core Python? References: <43C58200.20803@v.loewis.de> <43CC17FB.5020904@v.loewis.de> Message-ID: "Martin v. L?wis" writes: > Guido van Rossum wrote: >> On 1/16/06, Thomas Heller wrote: >> >>>It looks like we need a pronouncement now. >> >> >> Sorry. It appeared to me that there was general agreement to using a >> strongly worded warning in the docs, so I tuned out of the rest of the >> discussion. So for the record, I'm fine with that. > > Ok. If Thomas checks in the code and the documentation, I'll do the > Windows/setup part of it. Great. Now, I'm trying to build ctypes on OS X as the first non-windows platform. As I said, ctypes currently has its own source copy of libffi (since there are no separate releases of libffi, and the only platform that comes with libffi by default that I know is cygwin). It is built into a static non-shared library with a call to configure like this: cd /configure --disable-shared --enable-static \ --enable-multilib=no --prefix= make install then libffi.a is linked into the _ctypes extension module. I know near to nothing about 'configure', can/should a call to os.system with the commands above be added to setup.py, or how could this be approached? Thomas From brett at python.org Tue Jan 17 21:47:53 2006 From: brett at python.org (Brett Cannon) Date: Tue, 17 Jan 2006 12:47:53 -0800 Subject: [Python-Dev] Building on OS X 10.4 fails In-Reply-To: <1137525587.9528.19.camel@geddy.wooz.org> References: <1137525587.9528.19.camel@geddy.wooz.org> Message-ID: On 1/17/06, Barry Warsaw wrote: > On Tue, 2006-01-17 at 19:17 +0100, Thomas Heller wrote: > > Building the readline on OS X 10.4 fails, is this known, or am I doing > > something wrong? > > There are definitely serious issues with readline on OS X 10.4. I've > hit them too but haven't had time to post about it yet. I'm far from an > expert on them (Bob, Jack and others can definitely speak with more > knowledge on this) but my understanding is that the default readline > that comes with Tiger is not sufficient to build the Python extension > library. > > However, I use DarwinPorts and IWBNI Python's build process could > search /opt/local for the necessary readline support, using that if > found. I know that DP's Python 2.4.2 build does provide readline, so > that should definitely work. I've glanced at the setup.py file, but > haven't had the time to dig in in enough detail to actually fix and > commit. > OS X-specific package support has been removed from setup.py for the more generic support of LDFLAGS and CPPFLAGS. Set those properly for /opt/local/(lib|include) and it will find readline installed by DP fine (it's how I build my checkout with readline support). -Brett From aleaxit at gmail.com Tue Jan 17 23:30:34 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Tue, 17 Jan 2006 14:30:34 -0800 Subject: [Python-Dev] str with base In-Reply-To: References: Message-ID: On 1/16/06, Guido van Rossum wrote: > On 1/16/06, Alex Martelli wrote: > > Is it finally time in Python 2.5 to allow the "obvious" use of, say, > > str(5,2) to give '101', just the converse of the way int('101',1) > [I'm sure you meant int('101', 2) here] Yep. > > gives 5? I'm not sure why str has never allowed this obvious use -- > > any bright beginner assumes it's there and it's awkward to explain > > why it's not!-). I'll be happy to propose a patch if the BDFL > > blesses this, but I don't even think it's worth a PEP... it's an > > inexplicable though long-standing omission (given the argumentative > > nature of this crowd I know I'll get pushback, but I still hope the > > BDFL can Pronounce about it anyway;-). > > I wish you had an argument better than "every bright beginner assumes > it exists". :-) What about "it should obviously be there"?-) > But (unlike for some other things that bright beginners might assume) > I don't think there's a deep reason why this couldn't exist. > > The only reasons I can come up with is "because input and output are > notoriously asymmetric in Python" and "because nobody submitted a > patch". :-) OK, so, should I just submit a patch? > There are some corner cases to consider though. > > - Should repr() support the same convention? I think not. > - Should str(3.1, n) be allowed? I think not. > - Should str(x, n) call x.__str__(n)? Neither. > - Should bases other than 2..36 be considered? I don't think so. Agreed on all scores. > Unfortunately this doesn't obsolete oct() and hex() -- oct(10) returns > '012', while str(10, 8) soulc return '12'; hex(10) returns '0xa' while > str(10, 16) would return 'a'. Sure. hex(x) is like '0x'+str(x,16) and oct(x) is like '0'+str(x,8) but hex and oct are minutely more concise. > I do think that before we add this end-user nicety, it's more > important to implement __index__() in Python 2.5. This behaves like More important, sure, but also proportionally more work, so I don't see the two issues as "competing" against each other. > __int__() for integral types, but is not defined for float or Decimal. > Operations that intrinsically require an integral argument, like > indexing and slicing, should call __index__() on their argument rather > than __int__(), so as to support non-built-in integral arguments while > still complaining about float arguments. Hear, hear. Multiplication of sequences, too. > This is currently implemented > by explicitly checking for float in a few places, which I find > repulsing. __index__() won't be requested by bright beginners, but it You might be surprised by just how bright some (Python) beginners with deep maths background can be, though they might prefer to spell it differently (__int_and_i_really_mean_it__ for example'-) because it may not be obvious (if they're not Dutch) that multiplying a sequence has to do with 'indexing';-). > is important e.g. to the Numeric Python folks, who like to implement > their own integral types but are suffering from that their integers > aren't usable everywhere. As the author and maintainer of gmpy I entirely agree -- I never liked the fact that instances of gmpy.mpq are "second class citizens" and i need to plaster int(...) around them to use them as list indices or to multiply sequences (I vaguely mentioned having in mind a 'baseinteger' check, but a special method returning the integer value, such as the __index__ you're talking about, is obviously better). Alex From aleaxit at gmail.com Tue Jan 17 23:53:51 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Tue, 17 Jan 2006 14:53:51 -0800 Subject: [Python-Dev] basenumber redux In-Reply-To: <43CCC0AE.5070800@egenix.com> References: <17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com> <43CC17B9.3060200@v.loewis.de> <37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com> <43CCC0AE.5070800@egenix.com> Message-ID: On 1/17/06, M.-A. Lemburg wrote: > Alex, I think you're missing a point here: what you are looking > for is an interface, not a base class - simply because the I expect numbers to support arithmetic operators, &c -- no need for basenumber to "spell this out", i.e., "be an itnerface". > If you look at the Python C API, you'll find that "a number" > is actually never tested. There being no way to generically test for "a number", that's unsurprising. > The tests always ask for either > integers or floats. But this doesn't apply to the Python Standard Library, for example see line 1348 of imaplib.py: "if isinstance(date_time, (int, float)):". > The addition of a basenumber base class won't make these any > simpler. Being able to change imaplib to use basenumber instead of (int, float) won't make it SIMPLER, but it will surely make it BETTER -- why should a long be rejected, or a Decimal, for that matter? Similarly, on line 1352 it should use the existing basestring, though it now uses str (this function IS weird -- if it finds date_time to be of an unknown TYPE it raises a *ValueError* rather than a *TypeError* -- ah well). Alex From guido at python.org Tue Jan 17 23:54:31 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 17 Jan 2006 14:54:31 -0800 Subject: [Python-Dev] str with base In-Reply-To: References: Message-ID: On 1/17/06, Alex Martelli wrote: > OK, so, should I just submit a patch? Hmm, there are quite a few people who strongly dislike the particular API you're proposing. The problem is, bright newbies might be led to wanting str(i, base) as an analogy to int(s, base) only because they see str() and int() as each other's opposites, not having seen other uses of either (especially str()). Given the amount of disagreement on this issue and my own lackluster interest I don't want to pronounce str(i, base) to be the right solution. Sorry! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Wed Jan 18 00:21:13 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 18 Jan 2006 00:21:13 +0100 Subject: [Python-Dev] basenumber redux In-Reply-To: References: <17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com> <43CC17B9.3060200@v.loewis.de> <37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com> <43CCC0AE.5070800@egenix.com> Message-ID: <43CD7BE9.6060705@v.loewis.de> Alex Martelli wrote: > But this doesn't apply to the Python Standard Library, for example see > line 1348 of imaplib.py: "if isinstance(date_time, (int, float)):". [...] > Being able to change imaplib to use basenumber instead of (int, float) > won't make it SIMPLER, but it will surely make it BETTER -- why should > a long be rejected, or a Decimal, for that matter? Right. I think this function should read if isinstance(date_time, str) and \ (date_time[0],date_time[-1]) == ('"','"'): return date_time # Assume in correct format if isinstance(date_time, (tuple, time.struct_time)): tt = date_time else: tt = time.localtime(date_time) If this is something that time.localtime can't handle, it will give a TypeError. This is much better than raise ValueError("date_time not of a known type") # (why does it raise a ValueError if it says "type"?) Regards, Martin From thomas at xs4all.net Wed Jan 18 00:28:43 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Wed, 18 Jan 2006 00:28:43 +0100 Subject: [Python-Dev] str with base In-Reply-To: References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> <20060117100544.GC1419@craig-wood.com> <1137498898.4342.68.camel@warna.dub.corp.google.com> <2mek37oyh7.fsf@starship.python.net> Message-ID: <20060117232843.GV18916@xs4all.nl> On Tue, Jan 17, 2006 at 09:23:29AM -0500, Jason Orendorff wrote: > I think a method 5664400.to_base(13) sounds nice. [And others suggested int-methods too] I would like to point out that this is almost, but not quite, entirely as inapropriate as using str(). Integers don't have a base. String representations of integers -- and indeed, numbers in general, as the Python tutorial explains in Appendix B -- have a base. Adding such a method to integers (and, I presume, longs) would beg the question why floats, Decimals and complex numbers don't have them. In-favour-of-%2b-ly y'rs, -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From rhamph at gmail.com Wed Jan 18 00:29:35 2006 From: rhamph at gmail.com (Adam Olsen) Date: Tue, 17 Jan 2006 16:29:35 -0700 Subject: [Python-Dev] basenumber redux In-Reply-To: References: <17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com> <43CC17B9.3060200@v.loewis.de> <37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com> <43CCC0AE.5070800@egenix.com> Message-ID: On 1/17/06, Alex Martelli wrote: > Being able to change imaplib to use basenumber instead of (int, float) > won't make it SIMPLER, but it will surely make it BETTER -- why should > a long be rejected, or a Decimal, > for that matter? Because there's no guarantee that they'll produce correct results? All number types are approximations of true numbers, and they all behave wrong in creative ways. For example: def xrange(start, stop, step): i = start while i < stop: yield i i += step This works fine so long as you only give it int as input, and has no maximum value. >>> for i in xrange(2**53, 2**53+3, 1): print i ... 9007199254740992 9007199254740993 9007199254740994 Float inputs also work so long as you don't get large enough to provoke rounding. However, as soon as you do... >>> for i in xrange(2**53, 2**53+3, 1.0): print i ... 9007199254740992 9.00719925474e+15 9.00719925474e+15 9.00719925474e+15 9.00719925474e+15 9.00719925474e+15 9.00719925474e+15 974e+15 Traceback (most recent call last): File "", line 1, in ? KeyboardInterrupt The function fails. Floating point, despite being a "number" and supporting the "number interface", does not behave close enough to what the programmer desires to work for all values. There might be a way to handle floats specially that a mathematician may understand, but the only way I know of is to convert to int at the start of the function. def xrange(start, stop, step): start, stop, step = int(start), int(stop), int(step) i = start while i < stop: yield i i += step >>> for i in xrange(2**53, 2**53+3, 1.0): print i ... 9007199254740992 9007199254740993 9007199254740994 That works so long as the floats are all integral values. Unfortunately a non-integral value will get truncated silently. An explicit check for equality after the conversion would have to be added, or Guido's __index__ could be used, but __index__ seems misnamed for this usage. Another approach would be changing operations involving floats to return intervals instead. The high end of the interval would continue to count up when rounding is provoked, and would raise an exception when the i < stop is executed (due to being ambiguous). Once float uses intervals you could state that all number types are expected to use intervals in the face of inexactness (and those who don't behave as expected would have unexpected results.) -- Adam Olsen, aka Rhamphoryncus From rhamph at gmail.com Wed Jan 18 00:38:30 2006 From: rhamph at gmail.com (Adam Olsen) Date: Tue, 17 Jan 2006 16:38:30 -0700 Subject: [Python-Dev] str with base In-Reply-To: <20060117232843.GV18916@xs4all.nl> References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> <20060117100544.GC1419@craig-wood.com> <1137498898.4342.68.camel@warna.dub.corp.google.com> <2mek37oyh7.fsf@starship.python.net> <20060117232843.GV18916@xs4all.nl> Message-ID: On 1/17/06, Thomas Wouters wrote: > On Tue, Jan 17, 2006 at 09:23:29AM -0500, Jason Orendorff wrote: > > > I think a method 5664400.to_base(13) sounds nice. > [And others suggested int-methods too] > > I would like to point out that this is almost, but not quite, entirely as > inapropriate as using str(). Integers don't have a base. String > representations of integers -- and indeed, numbers in general, as the Python > tutorial explains in Appendix B -- have a base. Adding such a method to > integers (and, I presume, longs) would beg the question why floats, Decimals > and complex numbers don't have them. I dream of a day when str(3.25, base=2) == '11.01'. That is the number a float really represents. It would be so much easier to understand why floats behave the way they do if it were possible to print them in binary. To be fair, it's not str(x, base=n) I'm after here (although it seems like a clean way to do it.) Rather, I just want SOME way of printing ints and floats in binary. > In-favour-of-%2b-ly y'rs, My only opposition to this is that the byte type may want to use it. I'd rather wait until byte is fully defined, implemented, and released in a python version before that option is taken away. -- Adam Olsen, aka Rhamphoryncus From guido at python.org Wed Jan 18 01:02:43 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 17 Jan 2006 16:02:43 -0800 Subject: [Python-Dev] str with base In-Reply-To: References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> <20060117100544.GC1419@craig-wood.com> <1137498898.4342.68.camel@warna.dub.corp.google.com> <2mek37oyh7.fsf@starship.python.net> <20060117232843.GV18916@xs4all.nl> Message-ID: On 1/17/06, Adam Olsen wrote: > > In-favour-of-%2b-ly y'rs, > > My only opposition to this is that the byte type may want to use it. > I'd rather wait until byte is fully defined, implemented, and released > in a python version before that option is taken away. Has this been proposed? What would %b print? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From rhamph at gmail.com Wed Jan 18 01:09:41 2006 From: rhamph at gmail.com (Adam Olsen) Date: Tue, 17 Jan 2006 17:09:41 -0700 Subject: [Python-Dev] str with base In-Reply-To: References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> <20060117100544.GC1419@craig-wood.com> <1137498898.4342.68.camel@warna.dub.corp.google.com> <2mek37oyh7.fsf@starship.python.net> <20060117232843.GV18916@xs4all.nl> Message-ID: On 1/17/06, Guido van Rossum wrote: > On 1/17/06, Adam Olsen wrote: > > > In-favour-of-%2b-ly y'rs, > > > > My only opposition to this is that the byte type may want to use it. > > I'd rather wait until byte is fully defined, implemented, and released > > in a python version before that option is taken away. > > Has this been proposed? What would %b print? I don't believe it's been proposed and I don't know what it'd print. Perhaps it indicates the bytes should be passed through without conversion. In any case I only advocate waiting until it's clear that bytes have no need for it before we use it for binary conversions. -- Adam Olsen, aka Rhamphoryncus From anthony at interlink.com.au Wed Jan 18 01:17:32 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed, 18 Jan 2006 11:17:32 +1100 Subject: [Python-Dev] Building on OS X 10.4 fails In-Reply-To: <1137525587.9528.19.camel@geddy.wooz.org> References: <1137525587.9528.19.camel@geddy.wooz.org> Message-ID: <200601181117.34549.anthony@interlink.com.au> On Wednesday 18 January 2006 06:19, Barry Warsaw wrote: > On Tue, 2006-01-17 at 19:17 +0100, Thomas Heller wrote: > > Building the readline on OS X 10.4 fails, is this known, or am I > > doing something wrong? > > There are definitely serious issues with readline on OS X 10.4. > I've hit them too but haven't had time to post about it yet. I'm > far from an expert on them (Bob, Jack and others can definitely > speak with more knowledge on this) but my understanding is that the > default readline that comes with Tiger is not sufficient to build > the Python extension library. It sounds like configure needs to grow a test to detect that a "libreadline" it finds is actually the crackful "libedit" and refuse to use it if so. Anthony From bob at redivi.com Wed Jan 18 02:45:07 2006 From: bob at redivi.com (Bob Ippolito) Date: Tue, 17 Jan 2006 17:45:07 -0800 Subject: [Python-Dev] str with base In-Reply-To: References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> <20060117100544.GC1419@craig-wood.com> <1137498898.4342.68.camel@warna.dub.corp.google.com> <2mek37oyh7.fsf@starship.python.net> <20060117232843.GV18916@xs4all.nl> Message-ID: <98102A23-B77F-4D39-B7E8-CE4893CCF291@redivi.com> On Jan 17, 2006, at 4:09 PM, Adam Olsen wrote: > On 1/17/06, Guido van Rossum wrote: >> On 1/17/06, Adam Olsen wrote: >>>> In-favour-of-%2b-ly y'rs, >>> >>> My only opposition to this is that the byte type may want to use it. >>> I'd rather wait until byte is fully defined, implemented, and >>> released >>> in a python version before that option is taken away. >> >> Has this been proposed? What would %b print? > > I don't believe it's been proposed and I don't know what it'd print. > Perhaps it indicates the bytes should be passed through without > conversion. That doesn't make any sense. What is "without conversion"? Does that mean UTF-8, UCS-2, UCS-4, latin-1, Shift-JIS? You can't have unicode without some kind of conversion. > In any case I only advocate waiting until it's clear that bytes have > no need for it before we use it for binary conversions. I don't see what business a byte type has mingling with string formatters other than the normal str and repr coercions via %s and %r respectively. -bob From jack at performancedrivers.com Wed Jan 18 02:01:27 2006 From: jack at performancedrivers.com (Jack Diederich) Date: Tue, 17 Jan 2006 20:01:27 -0500 Subject: [Python-Dev] str with base In-Reply-To: References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> <20060117100544.GC1419@craig-wood.com> <1137498898.4342.68.camel@warna.dub.corp.google.com> <2mek37oyh7.fsf@starship.python.net> <20060117232843.GV18916@xs4all.nl> Message-ID: <20060118010127.GB6152@performancedrivers.com> On Tue, Jan 17, 2006 at 04:02:43PM -0800, Guido van Rossum wrote: > On 1/17/06, Adam Olsen wrote: > > > In-favour-of-%2b-ly y'rs, > > > > My only opposition to this is that the byte type may want to use it. > > I'd rather wait until byte is fully defined, implemented, and released > > in a python version before that option is taken away. > > Has this been proposed? What would %b print? > It was proposed in this or another thread about the same in the last few days (gmane search doesn't like the % in '%b'). The suggestion is to add 'b' as a sprintf-like format string %[][.]b Where the optional is the base to print in and is the optional minimum length of chars to print (as I recall). Default is base 2. Me? I like it. -Jack From bob at redivi.com Wed Jan 18 02:56:48 2006 From: bob at redivi.com (Bob Ippolito) Date: Tue, 17 Jan 2006 17:56:48 -0800 Subject: [Python-Dev] str with base In-Reply-To: References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> <20060117100544.GC1419@craig-wood.com> <1137498898.4342.68.camel@warna.dub.corp.google.com> <2mek37oyh7.fsf@starship.python.net> <20060117232843.GV18916@xs4all.nl> Message-ID: <1A1FD1BA-E80E-4F18-9EE2-CD73FF41B2DA@redivi.com> On Jan 17, 2006, at 3:38 PM, Adam Olsen wrote: > On 1/17/06, Thomas Wouters wrote: >> On Tue, Jan 17, 2006 at 09:23:29AM -0500, Jason Orendorff wrote: >> >>> I think a method 5664400.to_base(13) sounds nice. >> [And others suggested int-methods too] >> >> I would like to point out that this is almost, but not quite, >> entirely as >> inapropriate as using str(). Integers don't have a base. String >> representations of integers -- and indeed, numbers in general, as >> the Python >> tutorial explains in Appendix B -- have a base. Adding such a >> method to >> integers (and, I presume, longs) would beg the question why >> floats, Decimals >> and complex numbers don't have them. > > I dream of a day when str(3.25, base=2) == '11.01'. That is the > number a float really represents. It would be so much easier to > understand why floats behave the way they do if it were possible to > print them in binary. Actually if you wanted something that closely represents what a floating point number is then you would want to see this:: >>> str(3.25, base=2) '1.101e1' >>> str(0.25, base=2) '1.0e-10' Printing the bits without an exponent is nearly as misleading as printing them in decimal. -bob From bob at redivi.com Wed Jan 18 02:59:01 2006 From: bob at redivi.com (Bob Ippolito) Date: Tue, 17 Jan 2006 17:59:01 -0800 Subject: [Python-Dev] Building on OS X 10.4 fails In-Reply-To: <200601181117.34549.anthony@interlink.com.au> References: <1137525587.9528.19.camel@geddy.wooz.org> <200601181117.34549.anthony@interlink.com.au> Message-ID: On Jan 17, 2006, at 4:17 PM, Anthony Baxter wrote: > On Wednesday 18 January 2006 06:19, Barry Warsaw wrote: >> On Tue, 2006-01-17 at 19:17 +0100, Thomas Heller wrote: >>> Building the readline on OS X 10.4 fails, is this known, or am I >>> doing something wrong? >> >> There are definitely serious issues with readline on OS X 10.4. >> I've hit them too but haven't had time to post about it yet. I'm >> far from an expert on them (Bob, Jack and others can definitely >> speak with more knowledge on this) but my understanding is that the >> default readline that comes with Tiger is not sufficient to build >> the Python extension library. > > It sounds like configure needs to grow a test to detect that a > "libreadline" it finds is actually the crackful "libedit" and refuse > to use it if so. Unless something has changed since 2.4.1, that is the current behavior. Thomas' title should've probably been "Building readline on OS X 10.4 fails", rather than "Building on OS X 10.4 fails" (which reads as "Building Python on OS X 10.4 fails" given the list). -bob From bob at redivi.com Wed Jan 18 03:11:36 2006 From: bob at redivi.com (Bob Ippolito) Date: Tue, 17 Jan 2006 18:11:36 -0800 Subject: [Python-Dev] str with base In-Reply-To: <20060118010127.GB6152@performancedrivers.com> References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> <20060117100544.GC1419@craig-wood.com> <1137498898.4342.68.camel@warna.dub.corp.google.com> <2mek37oyh7.fsf@starship.python.net> <20060117232843.GV18916@xs4all.nl> <20060118010127.GB6152@performancedrivers.com> Message-ID: <8BEAD4AB-6F49-4901-BBE7-77870E953AD5@redivi.com> On Jan 17, 2006, at 5:01 PM, Jack Diederich wrote: > On Tue, Jan 17, 2006 at 04:02:43PM -0800, Guido van Rossum wrote: >> On 1/17/06, Adam Olsen wrote: >>>> In-favour-of-%2b-ly y'rs, >>> >>> My only opposition to this is that the byte type may want to use it. >>> I'd rather wait until byte is fully defined, implemented, and >>> released >>> in a python version before that option is taken away. >> >> Has this been proposed? What would %b print? >> > It was proposed in this or another thread about the same in the > last few > days (gmane search doesn't like the % in '%b'). > > The suggestion is to add 'b' as a sprintf-like format string > %[][.]b > > Where the optional is the base to print in and is the > optional > minimum length of chars to print (as I recall). Default is base 2. > > Me? I like it. Personally I would prefer the "b" format code to behave similarly to "o", "d", and "d", except for binary instead of octal, decimal, and hexadecimal. Something that needs to account for three factors (zero pad, space pad, base) should probably be a function (maybe a builtin). Hell, maybe it could take a fourth argument to specify how a negative number should be printed (e.g. a number of bits to use for the 2's complement). However... if %b were to represent arbitrary bases, I think that's backwards. It should be %[][.]b, which would do this: >>> '%08b %08o %08d %08x' % 12 '00001100 00000014 00000012 0000000C' Where your suggestion would have this behavior (or something close to it): >>> '%08b %08o %08d %08x' % 12 '14 00000014 00000012 0000000C' -bob From rhamph at gmail.com Wed Jan 18 03:17:27 2006 From: rhamph at gmail.com (Adam Olsen) Date: Tue, 17 Jan 2006 19:17:27 -0700 Subject: [Python-Dev] str with base In-Reply-To: <1A1FD1BA-E80E-4F18-9EE2-CD73FF41B2DA@redivi.com> References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> <20060117100544.GC1419@craig-wood.com> <1137498898.4342.68.camel@warna.dub.corp.google.com> <2mek37oyh7.fsf@starship.python.net> <20060117232843.GV18916@xs4all.nl> <1A1FD1BA-E80E-4F18-9EE2-CD73FF41B2DA@redivi.com> Message-ID: On 1/17/06, Bob Ippolito wrote: > > On Jan 17, 2006, at 3:38 PM, Adam Olsen wrote: > > > I dream of a day when str(3.25, base=2) == '11.01'. That is the > > number a float really represents. It would be so much easier to > > understand why floats behave the way they do if it were possible to > > print them in binary. > > Actually if you wanted something that closely represents what a > floating point number is then you would want to see this:: > > >>> str(3.25, base=2) > '1.101e1' > >>> str(0.25, base=2) > '1.0e-10' > > Printing the bits without an exponent is nearly as misleading as > printing them in decimal. I disagree. The exponent is involved in rounding to fit in compact storage but once that is complete the value can be represented exactly without it. -- Adam Olsen, aka Rhamphoryncus From rhamph at gmail.com Wed Jan 18 03:26:53 2006 From: rhamph at gmail.com (Adam Olsen) Date: Tue, 17 Jan 2006 19:26:53 -0700 Subject: [Python-Dev] str with base In-Reply-To: <98102A23-B77F-4D39-B7E8-CE4893CCF291@redivi.com> References: <20060117100544.GC1419@craig-wood.com> <1137498898.4342.68.camel@warna.dub.corp.google.com> <2mek37oyh7.fsf@starship.python.net> <20060117232843.GV18916@xs4all.nl> <98102A23-B77F-4D39-B7E8-CE4893CCF291@redivi.com> Message-ID: On 1/17/06, Bob Ippolito wrote: > > On Jan 17, 2006, at 4:09 PM, Adam Olsen wrote: > > > On 1/17/06, Guido van Rossum wrote: > >> On 1/17/06, Adam Olsen wrote: > >>>> In-favour-of-%2b-ly y'rs, > >>> > >>> My only opposition to this is that the byte type may want to use it. > >>> I'd rather wait until byte is fully defined, implemented, and > >>> released > >>> in a python version before that option is taken away. > >> > >> Has this been proposed? What would %b print? > > > > I don't believe it's been proposed and I don't know what it'd print. > > Perhaps it indicates the bytes should be passed through without > > conversion. > > That doesn't make any sense. What is "without conversion"? Does > that mean UTF-8, UCS-2, UCS-4, latin-1, Shift-JIS? You can't have > unicode without some kind of conversion. > > > In any case I only advocate waiting until it's clear that bytes have > > no need for it before we use it for binary conversions. > > I don't see what business a byte type has mingling with string > formatters other than the normal str and repr coercions via %s and %r > respectively. Is the byte type intended to be involved in string formatters at all? Does byte("%i") % 3 have the obvious effect, or is it an error? Although upon further consideration I don't see any case where %s and %b would have different effects.. *shrug* I never said it did have a purpose, just that it *might* be given a purpose when byte was spec'd out. -- Adam Olsen, aka Rhamphoryncus From jack at performancedrivers.com Wed Jan 18 04:12:38 2006 From: jack at performancedrivers.com (Jack Diederich) Date: Tue, 17 Jan 2006 22:12:38 -0500 Subject: [Python-Dev] str with base In-Reply-To: <8BEAD4AB-6F49-4901-BBE7-77870E953AD5@redivi.com> References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> <20060117100544.GC1419@craig-wood.com> <1137498898.4342.68.camel@warna.dub.corp.google.com> <2mek37oyh7.fsf@starship.python.net> <20060117232843.GV18916@xs4all.nl> <20060118010127.GB6152@performancedrivers.com> <8BEAD4AB-6F49-4901-BBE7-77870E953AD5@redivi.com> Message-ID: <20060118031238.GC6152@performancedrivers.com> On Tue, Jan 17, 2006 at 06:11:36PM -0800, Bob Ippolito wrote: > > On Jan 17, 2006, at 5:01 PM, Jack Diederich wrote: > > >On Tue, Jan 17, 2006 at 04:02:43PM -0800, Guido van Rossum wrote: > >>On 1/17/06, Adam Olsen wrote: > >>>>In-favour-of-%2b-ly y'rs, > >>> > >>>My only opposition to this is that the byte type may want to use it. > >>>I'd rather wait until byte is fully defined, implemented, and > >>>released > >>>in a python version before that option is taken away. > >> > >>Has this been proposed? What would %b print? > >> > >It was proposed in this or another thread about the same in the > >last few > >days (gmane search doesn't like the % in '%b'). > > > >The suggestion is to add 'b' as a sprintf-like format string > > %[][.]b > > > >Where the optional is the base to print in and is the > >optional > >minimum length of chars to print (as I recall). Default is base 2. > > > >Me? I like it. > > Personally I would prefer the "b" format code to behave similarly to > "o", "d", and "d", except for binary instead of octal, decimal, and > hexadecimal. Something that needs to account for three factors (zero > pad, space pad, base) should probably be a function (maybe a > builtin). Hell, maybe it could take a fourth argument to specify how > a negative number should be printed (e.g. a number of bits to use for > the 2's complement). > > However... if %b were to represent arbitrary bases, I think that's > backwards. It should be %[][.]b, which would do this: > > >>> '%08b %08o %08d %08x' % 12 > '00001100 00000014 00000012 0000000C' > Were I BDFAD (not to be confused with BD-FOAD) I'd add %b, %B and, binary() to match %x, %X, and hex(). The arbitrary base case isn't even academic or we would see homework questions about it on c.l.py. No one asks about hex or octal because they are there. No one asks about base seven formatting because everyone knows numerologists prefer Perl. -Jack nb, that's "For A Day." From bob at redivi.com Wed Jan 18 04:30:10 2006 From: bob at redivi.com (Bob Ippolito) Date: Tue, 17 Jan 2006 19:30:10 -0800 Subject: [Python-Dev] str with base In-Reply-To: <20060118031238.GC6152@performancedrivers.com> References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> <20060117100544.GC1419@craig-wood.com> <1137498898.4342.68.camel@warna.dub.corp.google.com> <2mek37oyh7.fsf@starship.python.net> <20060117232843.GV18916@xs4all.nl> <20060118010127.GB6152@performancedrivers.com> <8BEAD4AB-6F49-4901-BBE7-77870E953AD5@redivi.com> <20060118031238.GC6152@performancedrivers.com> Message-ID: <4B72B1E6-EEA1-4DE1-A096-5E6D478340C2@redivi.com> On Jan 17, 2006, at 7:12 PM, Jack Diederich wrote: > On Tue, Jan 17, 2006 at 06:11:36PM -0800, Bob Ippolito wrote: >> >> On Jan 17, 2006, at 5:01 PM, Jack Diederich wrote: >> >>> On Tue, Jan 17, 2006 at 04:02:43PM -0800, Guido van Rossum wrote: >>>> On 1/17/06, Adam Olsen wrote: >>>>>> In-favour-of-%2b-ly y'rs, >>>>> >>>>> My only opposition to this is that the byte type may want to >>>>> use it. >>>>> I'd rather wait until byte is fully defined, implemented, and >>>>> released >>>>> in a python version before that option is taken away. >>>> >>>> Has this been proposed? What would %b print? >>>> >>> It was proposed in this or another thread about the same in the >>> last few >>> days (gmane search doesn't like the % in '%b'). >>> >>> The suggestion is to add 'b' as a sprintf-like format string >>> %[][.]b >>> >>> Where the optional is the base to print in and is the >>> optional >>> minimum length of chars to print (as I recall). Default is base 2. >>> >>> Me? I like it. >> >> Personally I would prefer the "b" format code to behave similarly to >> "o", "d", and "d", except for binary instead of octal, decimal, and >> hexadecimal. Something that needs to account for three factors (zero >> pad, space pad, base) should probably be a function (maybe a >> builtin). Hell, maybe it could take a fourth argument to specify how >> a negative number should be printed (e.g. a number of bits to use for >> the 2's complement). >> >> However... if %b were to represent arbitrary bases, I think that's >> backwards. It should be %[][.]b, which would do this: >> >> >>> '%08b %08o %08d %08x' % 12 >> '00001100 00000014 00000012 0000000C' >> > > Were I BDFAD (not to be confused with BD-FOAD) I'd add %b, %B and, > binary() > to match %x, %X, and hex(). The arbitrary base case isn't even > academic > or we would see homework questions about it on c.l.py. No one asks > about > hex or octal because they are there. No one asks about base seven > formatting because everyone knows numerologists prefer Perl. There shouldn't be a %B for the same reason there isn't an %O or %D -- they're all just digits, so there's not a need for an uppercase variant. The difference between hex() and oct() and the proposed binary() is that hex() and oct() return valid Python expressions in that base. In order for it to make sense, Python would need to grow some syntax. If Python were to have syntax for binary literals, I'd propose a trailing b: "1100b". It would be convenient at times to represent bit flags, but I'm not sure it's worth the syntax change. binarydigit ::= ("0" | "1") binaryinteger ::= binarydigit+ "b" integer ::= decimalinteger | octinteger | hexinteger | binaryinteger -bob From simon at arrowtheory.com Wed Jan 18 04:56:12 2006 From: simon at arrowtheory.com (Simon Burton) Date: Wed, 18 Jan 2006 14:56:12 +1100 Subject: [Python-Dev] computed goto's in ceval loop Message-ID: <20060118145612.2d4009ab.simon@arrowtheory.com> I have been experimenting with replacing the big switch in ceval.c by a computed goto construct [1]. It uses #define's to make it optional. This work was inspired by Mono's MINT interpreter code, and Neil Norwitz's attempt to use a function pointer table [2]. Result: about 1% slower on the pystone benchmark. It also seems to have broken the interpreter; at least one test fails (test_StringIO). Whoops. Not sure if the other switch-speedup hacks (eg. PREDICT(op)) conflict at all with this patch (ie. make it slower than it could be). Mono uses a much larger opcode set, with 2-byte opcodes, that includes type info in each opcode. This means that the actual case statements are much faster. My initial thought about using computed goto's (January 2003) was that the python opcode cases were much fatter than mono's (often involving a function call) so that the overhead of branching on opcodes would be insignificant. It seems that this is true indeed. The patch id is 1408710. [1] http://developer.apple.com/documentation/DeveloperTools/gcc-4.0.1/gcc/Labels-as-Values.html [2] http://mail.python.org/pipermail/python-dev/2003-February/033705.html Simon. -- Simon Burton, B.Sc. Licensed PO Box 8066 ANU Canberra 2601 Australia Ph. 61 02 6249 6940 http://arrowtheory.com From guido at python.org Wed Jan 18 05:25:49 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 17 Jan 2006 20:25:49 -0800 Subject: [Python-Dev] str with base In-Reply-To: <4B72B1E6-EEA1-4DE1-A096-5E6D478340C2@redivi.com> References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> <2mek37oyh7.fsf@starship.python.net> <20060117232843.GV18916@xs4all.nl> <20060118010127.GB6152@performancedrivers.com> <8BEAD4AB-6F49-4901-BBE7-77870E953AD5@redivi.com> <20060118031238.GC6152@performancedrivers.com> <4B72B1E6-EEA1-4DE1-A096-5E6D478340C2@redivi.com> Message-ID: On 1/17/06, Bob Ippolito wrote: > There shouldn't be a %B for the same reason there isn't an %O or %D > -- they're all just digits, so there's not a need for an uppercase > variant. Right. > The difference between hex() and oct() and the proposed binary() is I'd propose bin() to stay in line with the short abbreviated names. > that hex() and oct() return valid Python expressions in that base. > In order for it to make sense, Python would need to grow some syntax. Fair enough. So let's define it. > If Python were to have syntax for binary literals, I'd propose a > trailing b: "1100b". It would be convenient at times to represent > bit flags, but I'm not sure it's worth the syntax change. Typically, suffixes are used to indicated *types*: 12L, 12j, and even 12e0 in some sense. The binary type should have a 0b prefix. Perhaps this could be implemented at the PyCon sprint? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Wed Jan 18 05:53:38 2006 From: brett at python.org (Brett Cannon) Date: Tue, 17 Jan 2006 20:53:38 -0800 Subject: [Python-Dev] str with base In-Reply-To: References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> <20060117232843.GV18916@xs4all.nl> <20060118010127.GB6152@performancedrivers.com> <8BEAD4AB-6F49-4901-BBE7-77870E953AD5@redivi.com> <20060118031238.GC6152@performancedrivers.com> <4B72B1E6-EEA1-4DE1-A096-5E6D478340C2@redivi.com> Message-ID: On 1/17/06, Guido van Rossum wrote: > On 1/17/06, Bob Ippolito wrote: > > There shouldn't be a %B for the same reason there isn't an %O or %D > > -- they're all just digits, so there's not a need for an uppercase > > variant. > > Right. > > > The difference between hex() and oct() and the proposed binary() is > > I'd propose bin() to stay in line with the short abbreviated names. > > > that hex() and oct() return valid Python expressions in that base. > > In order for it to make sense, Python would need to grow some syntax. > > Fair enough. So let's define it. > > > If Python were to have syntax for binary literals, I'd propose a > > trailing b: "1100b". It would be convenient at times to represent > > bit flags, but I'm not sure it's worth the syntax change. > > Typically, suffixes are used to indicated *types*: 12L, 12j, and even > 12e0 in some sense. > > The binary type should have a 0b prefix. > 0b101 for 5? > Perhaps this could be implemented at the PyCon sprint? > Added to the wiki along with possibly hashing out the bytes type. -Brett From stephen at xemacs.org Wed Jan 18 06:25:03 2006 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 18 Jan 2006 14:25:03 +0900 Subject: [Python-Dev] Building on OS X 10.4 fails In-Reply-To: <200601181117.34549.anthony@interlink.com.au> (Anthony Baxter's message of "Wed, 18 Jan 2006 11:17:32 +1100") References: <1137525587.9528.19.camel@geddy.wooz.org> <200601181117.34549.anthony@interlink.com.au> Message-ID: <87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Anthony" == Anthony Baxter writes: Anthony> It sounds like configure needs to grow a test to detect Anthony> that a "libreadline" it finds is actually the crackful Anthony> "libedit" and refuse to use it if so. FYI: Real libreadline is GPL, and rms made a point of forcing (Aladdin-licensed) Ghostscript to remove stanzas from the Makefile that allowed linking to it as a user option. Ie, this particular pain in the neck is deliberate FSF policy, to encourage use of the GPL. Unless rms has changed his position on this, or there has been relevant legislation or a court decision in the meantime, explicitly requiring or checking for "real" libreadline, even as a user option, risks rms's wrath. (Of course, Python could change its license to GPL, which would undoubtedly flood Cambridge with tears of joy). As long as the link to fake libreadline succeeds and the resulting program works identically to one linked to real libreadline, he has no complaint. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From nnorwitz at gmail.com Wed Jan 18 07:08:06 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 17 Jan 2006 22:08:06 -0800 Subject: [Python-Dev] str with base In-Reply-To: References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> <20060117232843.GV18916@xs4all.nl> <20060118010127.GB6152@performancedrivers.com> <8BEAD4AB-6F49-4901-BBE7-77870E953AD5@redivi.com> <20060118031238.GC6152@performancedrivers.com> <4B72B1E6-EEA1-4DE1-A096-5E6D478340C2@redivi.com> Message-ID: On 1/17/06, Guido van Rossum wrote: > > > The difference between hex() and oct() and the proposed binary() is > > I'd propose bin() to stay in line with the short abbreviated names. Are these features used enough to have 3 builtins? Would format(number, base) suffice? format(5, base=2) == '101' format(5, base=8) == '5' format(5, base=8, prefix=True) == '05' format(5, base=16) == '5' format(5, base=16, prefix=True) == '0x5' Or something like that. Then there can be symmetry with int() (arbitrary bases) and we get rid of 2 other builtins eventually. Not sure if there are/should be uses other than number formating. > > that hex() and oct() return valid Python expressions in that base. > > In order for it to make sense, Python would need to grow some syntax. > > Fair enough. So let's define it. > > > If Python were to have syntax for binary literals, I'd propose a > > trailing b: "1100b". It would be convenient at times to represent > > bit flags, but I'm not sure it's worth the syntax change. > > Typically, suffixes are used to indicated *types*: 12L, 12j, and even > 12e0 in some sense. > > The binary type should have a 0b prefix. -0. Is this common enough to add (even in 3k)? For the instances I could have used this, it would have been completely impractical since the hex strings were generally over 80 characters. n From andrew-pythondev at puzzling.org Wed Jan 18 07:22:27 2006 From: andrew-pythondev at puzzling.org (Andrew Bennetts) Date: Wed, 18 Jan 2006 17:22:27 +1100 Subject: [Python-Dev] str with base In-Reply-To: References: <2mek37oyh7.fsf@starship.python.net> <20060117232843.GV18916@xs4all.nl> <20060118010127.GB6152@performancedrivers.com> <8BEAD4AB-6F49-4901-BBE7-77870E953AD5@redivi.com> <20060118031238.GC6152@performancedrivers.com> <4B72B1E6-EEA1-4DE1-A096-5E6D478340C2@redivi.com> Message-ID: <20060118062227.GA28517@home.puzzling.org> Guido van Rossum wrote: [...] > > I'd propose bin() to stay in line with the short abbreviated names. > [...] > > The binary type should have a 0b prefix. It seems odd to me to add both a builtin *and* new syntax for something that's occasionally handy, but only occasionally. If we're going to clutter a module with this function, why not e.g. the math module instead of builtins? I thought the consensus was that we had too many builtins already. Similarly, the need for 0b101 syntax seems pretty low to me when you can already do int("101", 2). -Andrew. From gisle at ActiveState.com Wed Jan 18 08:35:26 2006 From: gisle at ActiveState.com (Gisle Aas) Date: 17 Jan 2006 23:35:26 -0800 Subject: [Python-Dev] str with base In-Reply-To: <20060118031238.GC6152@performancedrivers.com> References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> <20060117100544.GC1419@craig-wood.com> <1137498898.4342.68.camel@warna.dub.corp.google.com> <2mek37oyh7.fsf@starship.python.net> <20060117232843.GV18916@xs4all.nl> <20060118010127.GB6152@performancedrivers.com> <8BEAD4AB-6F49-4901-BBE7-77870E953AD5@redivi.com> <20060118031238.GC6152@performancedrivers.com> Message-ID: Jack Diederich writes: > > However... if %b were to represent arbitrary bases, I think that's > > backwards. It should be %[][.]b, which would do this: > > > > >>> '%08b %08o %08d %08x' % 12 > > '00001100 00000014 00000012 0000000C' > > > > Were I BDFAD (not to be confused with BD-FOAD) I'd add %b, %B and, binary() > to match %x, %X, and hex(). The arbitrary base case isn't even academic > or we would see homework questions about it on c.l.py. No one asks about > hex or octal because they are there. No one asks about base seven > formatting because everyone knows numerologists prefer Perl. BTW, Perl already do binary literals and %b formatting so there is some precedence for it: $ perl -e '$a = 0b1100; printf "%08b %08o %08d %08x\n", $a, $a, $a, $a' 00001100 00000014 00000012 0000000c --Gisle From steve at holdenweb.com Wed Jan 18 09:31:18 2006 From: steve at holdenweb.com (Steve Holden) Date: Wed, 18 Jan 2006 08:31:18 +0000 Subject: [Python-Dev] str with base In-Reply-To: References: <20060117100544.GC1419@craig-wood.com> <1137498898.4342.68.camel@warna.dub.corp.google.com> <2mek37oyh7.fsf@starship.python.net> <20060117232843.GV18916@xs4all.nl> <98102A23-B77F-4D39-B7E8-CE4893CCF291@redivi.com> Message-ID: Adam Olsen wrote: > On 1/17/06, Bob Ippolito wrote: > >>On Jan 17, 2006, at 4:09 PM, Adam Olsen wrote: >> >> >>>On 1/17/06, Guido van Rossum wrote: >>> >>>>On 1/17/06, Adam Olsen wrote: >>>> >>>>>>In-favour-of-%2b-ly y'rs, >>>>> >>>>>My only opposition to this is that the byte type may want to use it. >>>>>I'd rather wait until byte is fully defined, implemented, and >>>>>released >>>>>in a python version before that option is taken away. >>>> >>>>Has this been proposed? What would %b print? >>> >>>I don't believe it's been proposed and I don't know what it'd print. >>>Perhaps it indicates the bytes should be passed through without >>>conversion. >> >>That doesn't make any sense. What is "without conversion"? Does >>that mean UTF-8, UCS-2, UCS-4, latin-1, Shift-JIS? You can't have >>unicode without some kind of conversion. >> >> >>>In any case I only advocate waiting until it's clear that bytes have >>>no need for it before we use it for binary conversions. >> >>I don't see what business a byte type has mingling with string >>formatters other than the normal str and repr coercions via %s and %r >>respectively. > > > Is the byte type intended to be involved in string formatters at all? > Does byte("%i") % 3 have the obvious effect, or is it an error? > > Although upon further consideration I don't see any case where %s and > %b would have different effects.. *shrug* I never said it did have a > purpose, just that it *might* be given a purpose when byte was spec'd > out. > I suppose we'd better reserve "%q" for 'quirky types we just invented', too? ;-) regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/ From steve at holdenweb.com Wed Jan 18 09:41:09 2006 From: steve at holdenweb.com (Steve Holden) Date: Wed, 18 Jan 2006 08:41:09 +0000 Subject: [Python-Dev] str with base In-Reply-To: References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> <20060117100544.GC1419@craig-wood.com> <1137498898.4342.68.camel@warna.dub.corp.google.com> <2mek37oyh7.fsf@starship.python.net> <20060117232843.GV18916@xs4all.nl> <1A1FD1BA-E80E-4F18-9EE2-CD73FF41B2DA@redivi.com> Message-ID: Adam Olsen wrote: > On 1/17/06, Bob Ippolito wrote: > >>On Jan 17, 2006, at 3:38 PM, Adam Olsen wrote: >> >> >>>I dream of a day when str(3.25, base=2) == '11.01'. That is the >>>number a float really represents. It would be so much easier to >>>understand why floats behave the way they do if it were possible to >>>print them in binary. >> >>Actually if you wanted something that closely represents what a >>floating point number is then you would want to see this:: >> >> >>> str(3.25, base=2) >> '1.101e1' >> >>> str(0.25, base=2) >> '1.0e-10' >> >>Printing the bits without an exponent is nearly as misleading as >>printing them in decimal. > > > I disagree. The exponent is involved in rounding to fit in compact > storage but once that is complete the value can be represented exactly > without it. > Albeit with excessively long representations for the larger values one sometimes sees represented in float form. Personally I wouldn't even be interested in seeing 1.3407807929942597e+154 written in fixed point form *in decimal*, let alone in binary where the representation, though unambiguous, would have over 500 bits, most of them zeros. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/ From thomas at xs4all.net Wed Jan 18 10:17:24 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Wed, 18 Jan 2006 10:17:24 +0100 Subject: [Python-Dev] Building on OS X 10.4 fails In-Reply-To: <87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp> References: <1137525587.9528.19.camel@geddy.wooz.org> <200601181117.34549.anthony@interlink.com.au> <87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <20060118091724.GW18916@xs4all.nl> On Wed, Jan 18, 2006 at 02:25:03PM +0900, Stephen J. Turnbull wrote: > Anthony> It sounds like configure needs to grow a test to detect > Anthony> that a "libreadline" it finds is actually the crackful > Anthony> "libedit" and refuse to use it if so. > FYI: Real libreadline is GPL, and rms made a point of forcing > (Aladdin-licensed) Ghostscript to remove stanzas from the Makefile > that allowed linking to it as a user option. Ie, this particular pain > in the neck is deliberate FSF policy, to encourage use of the GPL. [...] > As long as the link to fake libreadline succeeds and the resulting > program works identically to one linked to real libreadline, he has no > complaint. I don't think this applies to Python. The Aladdin license isn't GPL-compatible, but the current PSF license is (according to rms himself.) (Only, IIRC, 1.5.2-and-earlier, 2.0.1 and 2.1.1-and-later, not 1.6, 1.6.1[*], 2.0 or 2.1.) The Ghostscript check-for-readline is a case of "you are still linking with readline, even when you aren't actually linking" -- but this isn't a problem for (most versions of) Python, because rms said it isnt. As long as the resulting Python binary is only covered by the GPL-compatible PSF license, the GPL and no GPL-incompatible licenses, any form of linking is fine, even configure-time-not-building linking. [*] 1.6.1 is the release that contained the license change rms and his laywers wanted, but that itself doesn't make the license GPL-compatible. It apparently allows derived works from being GPL-compatible, though. 2.0 and 2.1 derive from 1.6, but 2.0.1 and 2.1.1 derive from 1.6.1. I'm sure it makes sense to someone. Go-figure'ly y'rs, -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From martin at v.loewis.de Wed Jan 18 10:19:58 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 18 Jan 2006 10:19:58 +0100 Subject: [Python-Dev] subwcrev.exe Message-ID: <43CE083E.5030205@v.loewis.de> I just added svnversion support to the VC project files; subwcrev.exe is expected to be found in the Tortoise installation, if it cannot find tortoise in the registry, it falls back to not doing svnversion. If you find problems, please let me know. Regards, Martin From steve at holdenweb.com Wed Jan 18 10:28:34 2006 From: steve at holdenweb.com (Steve Holden) Date: Wed, 18 Jan 2006 09:28:34 +0000 Subject: [Python-Dev] str with base In-Reply-To: References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> <20060117100544.GC1419@craig-wood.com> <1137498898.4342.68.camel@warna.dub.corp.google.com> <2mek37oyh7.fsf@starship.python.net> <20060117232843.GV18916@xs4all.nl> <1A1FD1BA-E80E-4F18-9EE2-CD73FF41B2DA@redivi.com> Message-ID: <43CE0A42.3060608@holdenweb.com> Steve Holden wrote: [...] > Personally I wouldn't even be interested in seeing > 1.3407807929942597e+154 written in fixed point form *in decimal*, let > alone in binary where the representation, though unambiguous, would have > over 500 bits, most of them zeros. > Well, shot myself in the foot there of course, since the number I meant was actually 2.0 ** 512 (or 13407807929942597099574024998205846127479365820592393377723561443721764030073546 976801874298166903427690031858186486050853753882811946569946433649006084096.0) rather than the decimal approximation above. But I'm sure you get the point that fixed-point representations aren't always appropriate. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/ From anthony at interlink.com.au Wed Jan 18 10:31:49 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed, 18 Jan 2006 20:31:49 +1100 Subject: [Python-Dev] Building on OS X 10.4 fails In-Reply-To: <87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp> References: <200601181117.34549.anthony@interlink.com.au> <87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <200601182031.54111.anthony@interlink.com.au> On Wednesday 18 January 2006 16:25, Stephen J. Turnbull wrote: > Unless rms has changed his position on this, or there has been > relevant legislation or a court decision in the meantime, > explicitly requiring or checking for "real" libreadline, even as a > user option, risks rms's wrath. (Of course, Python could change > its license to GPL, which would undoubtedly flood Cambridge with > tears of joy). Python's license is GPL-compatible, so this isn't an issue. > As long as the link to fake libreadline succeeds and the resulting > program works identically to one linked to real libreadline, he has > no complaint. My "complaint" is that libedit _calls_ itself libreadline, when it's pretty clear that it's not actually a drop-in replacement (or the readline module would build). Hence my use of the word "crackful" Anthony -- Anthony Baxter It's never too late to have a happy childhood. From mal at egenix.com Wed Jan 18 11:03:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 18 Jan 2006 11:03:38 +0100 Subject: [Python-Dev] basenumber redux In-Reply-To: References: <17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com> <43CC17B9.3060200@v.loewis.de> <37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com> <43CCC0AE.5070800@egenix.com> Message-ID: <43CE127A.2050700@egenix.com> Alex Martelli wrote: > On 1/17/06, M.-A. Lemburg wrote: >> Alex, I think you're missing a point here: what you are looking >> for is an interface, not a base class - simply because the > > I expect numbers to support arithmetic operators, &c -- no need for > basenumber to "spell this out", i.e., "be an itnerface". If at all, basenumber would be an abstract class. However, unlike for basestring, the interface (which methods it supports, including operator methods) would not be well- defined. >> If you look at the Python C API, you'll find that "a number" >> is actually never tested. > > There being no way to generically test for "a number", that's unsurprising. Hmm, I lost you there. If it's unsurprising that there's no check for "a number", then why would you want a basenumber ? >> The tests always ask for either >> integers or floats. > > But this doesn't apply to the Python Standard Library, for example see > line 1348 of imaplib.py: "if isinstance(date_time, (int, float)):". Why not use the functions I added to my previous mail ? >> The addition of a basenumber base class won't make these any >> simpler. > > Being able to change imaplib to use basenumber instead of (int, float) > won't make it SIMPLER, but it will surely make it BETTER -- why should > a long be rejected, or a Decimal, > for that matter? Similarly, on line 1352 it should use the existing > basestring, though it now uses str (this function IS weird -- if it > finds date_time to be of an unknown TYPE it raises a *ValueError* > rather than a *TypeError* -- ah well). Again, why not use floatnumber() instead, which takes care of all the details behind finding out whether an object should be considered a number and even converts it to a float for you ? Why try to introduce a low-level feature when a higher level solution is readily available and more usable. You will rarely really care for the type of an object if all you're interested in is the float value of an object (or the integer value). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From abo at minkirri.apana.org.au Wed Jan 18 12:48:11 2006 From: abo at minkirri.apana.org.au (Donovan Baarda) Date: Wed, 18 Jan 2006 11:48:11 +0000 Subject: [Python-Dev] str with base In-Reply-To: References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> <20060117100544.GC1419@craig-wood.com> <1137498898.4342.68.camel@warna.dub.corp.google.com> <2mek37oyh7.fsf@starship.python.net> <20060117232843.GV18916@xs4all.nl> Message-ID: <1137584891.16309.19.camel@warna.dub.corp.google.com> On Tue, 2006-01-17 at 16:38 -0700, Adam Olsen wrote: > On 1/17/06, Thomas Wouters wrote: > > On Tue, Jan 17, 2006 at 09:23:29AM -0500, Jason Orendorff wrote: [...] > I dream of a day when str(3.25, base=2) == '11.01'. That is the > number a float really represents. It would be so much easier to > understand why floats behave the way they do if it were possible to > print them in binary. [...] Heh... that's pretty much why I used base16 float notation when doing fixed point stuff in assembler... uses less digits than binary, but easily visualised as bits. However, I do think that we could go overboard here... I don't know that we really need arbitrary base string formatting for all numeric types. I think this is a case of "very little gained for too much added complexity". If we really do, and someone is prepared to implement it, then I think adding "@base" is the best way to do it (see my half joking post earlier). If we only want arbitrary bases for integer types, the best way would be to leverage off the existing ".precision" so that it means ".base" for "%d". > > In-favour-of-%2b-ly y'rs, > > My only opposition to this is that the byte type may want to use it. > I'd rather wait until byte is fully defined, implemented, and released > in a python version before that option is taken away. There's always "B" for bytes and "b" for bits... though I can't imagine why byte would need it's own conversion type. I'm not entirely sure everyone is on the same page for "%b" here... it would only be a shorthand for "binary" in the same way that "%x" is for "hexidecimal". It would not support arbitrary bases, and thus "%2b" would mean a binary string with minimum length of 2 characters. -- Donovan Baarda http://minkirri.apana.org.au/~abo/ From abo at minkirri.apana.org.au Wed Jan 18 13:16:20 2006 From: abo at minkirri.apana.org.au (Donovan Baarda) Date: Wed, 18 Jan 2006 12:16:20 +0000 Subject: [Python-Dev] str with base In-Reply-To: References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> <2mek37oyh7.fsf@starship.python.net> <20060117232843.GV18916@xs4all.nl> <20060118010127.GB6152@performancedrivers.com> <8BEAD4AB-6F49-4901-BBE7-77870E953AD5@redivi.com> <20060118031238.GC6152@performancedrivers.com> <4B72B1E6-EEA1-4DE1-A096-5E6D478340C2@redivi.com> Message-ID: <1137586580.16309.28.camel@warna.dub.corp.google.com> On Tue, 2006-01-17 at 20:25 -0800, Guido van Rossum wrote: > On 1/17/06, Bob Ippolito wrote: > > There shouldn't be a %B for the same reason there isn't an %O or %D > > -- they're all just digits, so there's not a need for an uppercase [...] so %b is "binary", +1 > > The difference between hex() and oct() and the proposed binary() is > > I'd propose bin() to stay in line with the short abbreviated names. [...] +1 > The binary type should have a 0b prefix. [...] +1 For those who argue "who would ever use it?", I would :-) Note that this does not support and is independent of supporting arbitrary bases. I don't think we need to support arbitrary bases, but if we did I would vote for ".precision" to mean ".base" for "%d"... ie; "%3.3d" % 5 == " 12" I think supporting arbitrary bases for floats is way overkill and not worth considering. -- Donovan Baarda http://minkirri.apana.org.au/~abo/ From stephen at xemacs.org Wed Jan 18 14:24:52 2006 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 18 Jan 2006 22:24:52 +0900 Subject: [Python-Dev] Building on OS X 10.4 fails In-Reply-To: <200601182031.54111.anthony@interlink.com.au> (Anthony Baxter's message of "Wed, 18 Jan 2006 20:31:49 +1100") References: <200601181117.34549.anthony@interlink.com.au> <87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp> <200601182031.54111.anthony@interlink.com.au> Message-ID: <871wz57job.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Anthony" == Anthony Baxter writes: Anthony> Python's license is GPL-compatible, so this isn't an Anthony> issue. I'm sorry, but you seem to misunderstand what "GPL compatibility" means. It is a _one-way_ street. A license is GPL-compatible if its terms permit the code it covers to be redistributed under GPL. The GPL, however, is "compatible" in this sense only with itself: _the whole of any work_ derived from a GPL work must be licensed under the GPL. Not under a GPL-compatible license, under the GPL. The import of the Ghostscript case is that the FSF considers a Makefile stanza clearly intended to cause linkage to a GPL library, even if optional and supplied by the user, to create a work derived from that library. A "GNU readline"-enabled Python is derived from GNU readline, and must be distributed under the GPL or not distributed at all. I assume that the former is not acceptable to Python, and the latter is clearly undesirable. I assure you, Peter Deutsch was just as unbelieving as you are, but facing a potentially expensive lawsuit, he caved in. I can think of many reasons why the FSF might not come after Python, but the risk is real. Please ask a lawyer. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From ncoghlan at gmail.com Wed Jan 18 15:50:57 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 19 Jan 2006 00:50:57 +1000 Subject: [Python-Dev] Python icon In-Reply-To: <17357.15206.570762.861782@montanaro.dyndns.org> References: <17354.20722.560136.55434@montanaro.dyndns.org> <17357.15206.570762.861782@montanaro.dyndns.org> Message-ID: <43CE55D1.5030107@gmail.com> skip at pobox.com wrote: > Georg> Interesting, didn't even know a new page was in the making... Do > Georg> you know who is responsible for the new page? > > Tim Parkin is heading things up. Look here: > > http://beta.python.org/ I like the look of the new page, but it took a bit of digging to find anything about *how* it has been developed, or what needs to be done before the main URL can be switched to the new page (e.g. integrating the documentation properly). While it's in beta having that info prominent on the home page might be a good idea. Something like: "This is a beta of the redesigned www.python.org. If you have specific suggestions or display problems, submit them to the tracker . General questions or comments should be posted to comp.lang.python" I thought for a moment that this week's python-url was going to help me out, but it only linked the beta site without actually linking to any of the discussion it referenced in the relevant blurb. . . Appropriate enlightenment was eventually attained through Google: http://mail.python.org/pipermail/python-list/2006-January/319862.html Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From tim.peters at gmail.com Wed Jan 18 16:21:49 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 18 Jan 2006 10:21:49 -0500 Subject: [Python-Dev] [Python-checkins] r42090 - in python/trunk: Modules/getbuildinfo.c PCbuild/make_buildinfo.vcproj PCbuild/pcbuild.sln PCbuild/pythoncore.vcproj In-Reply-To: <20060118091354.A31B91E4002@bag.python.org> References: <20060118091354.A31B91E4002@bag.python.org> Message-ID: <1f7befae0601180721vbc67e95u54bfafb0569357fe@mail.gmail.com> [martin.v.loewis ] > Date: Wed Jan 18 10:13:51 2006 > New Revision: 42090 > > Added: > python/trunk/PCbuild/make_buildinfo.vcproj > Modified: > python/trunk/Modules/getbuildinfo.c > python/trunk/PCbuild/pcbuild.sln > python/trunk/PCbuild/pythoncore.vcproj > Log: > Generate getbuildinfo.o each time the linker is invoked; > try to generate SVNVERSION information if subwcrev.exe > can be found. Umm ... looks like I don't know how to build Python anymore. A batch build crapped out with: """ ------ Build started: Project: pythoncore, Configuration: Debug Win32 ------ generate buildinfo 'make_buildinfo.exe' is not recognized as an internal or external command, operable program or batch file. """ Adding the `make_buildinfo` project to the batch-build set craps out earlier: the `make_buildinfo` project lists make_buildinfo.c as an input file, but there is no such file: """ ------ Build started: Project: make_buildinfo, Configuration: Debug Win32 ------ Compiling... make_buildinfo.c c1 : fatal error C1083: Cannot open source file: '.\make_buildinfo.c': No such file or directory """ Did this checkin perhaps forget to add that file? From aahz at pythoncraft.com Wed Jan 18 16:23:56 2006 From: aahz at pythoncraft.com (Aahz) Date: Wed, 18 Jan 2006 07:23:56 -0800 Subject: [Python-Dev] str with base In-Reply-To: References: <2mek37oyh7.fsf@starship.python.net> <20060117232843.GV18916@xs4all.nl> <20060118010127.GB6152@performancedrivers.com> <8BEAD4AB-6F49-4901-BBE7-77870E953AD5@redivi.com> <20060118031238.GC6152@performancedrivers.com> <4B72B1E6-EEA1-4DE1-A096-5E6D478340C2@redivi.com> Message-ID: <20060118152356.GA20772@panix.com> On Tue, Jan 17, 2006, Guido van Rossum wrote: > On 1/17/06, Bob Ippolito wrote: >> >> The difference between hex() and oct() and the proposed binary() is > > I'd propose bin() to stay in line with the short abbreviated names. There has been some previous discussion about removing hex()/oct() from builtins for Python 3.0, IIRC. I sure don't think bin() belongs there. > The binary type should have a 0b prefix. -0 on adding a new prefix; +1 on this syntax if we do. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From steve at holdenweb.com Wed Jan 18 16:37:20 2006 From: steve at holdenweb.com (Steve Holden) Date: Wed, 18 Jan 2006 15:37:20 +0000 Subject: [Python-Dev] Python icon In-Reply-To: <43CE55D1.5030107@gmail.com> References: <17354.20722.560136.55434@montanaro.dyndns.org> <17357.15206.570762.861782@montanaro.dyndns.org> <43CE55D1.5030107@gmail.com> Message-ID: <43CE60B0.1070806@holdenweb.com> Nick Coghlan wrote: > skip at pobox.com wrote: > >> Georg> Interesting, didn't even know a new page was in the making... Do >> Georg> you know who is responsible for the new page? >> >>Tim Parkin is heading things up. Look here: >> >> http://beta.python.org/ > > > I like the look of the new page, but it took a bit of digging to find anything > about *how* it has been developed, or what needs to be done before the main > URL can be switched to the new page (e.g. integrating the documentation > properly). While it's in beta having that info prominent on the home page > might be a good idea. Something like: > > "This is a beta of the redesigned www.python.org. If you have specific > suggestions or display problems, submit them to the tracker . General > questions or comments should be posted to comp.lang.python" > > I thought for a moment that this week's python-url was going to help me out, > but it only linked the beta site without actually linking to any of the > discussion it referenced in the relevant blurb. . . > > Appropriate enlightenment was eventually attained through Google: > http://mail.python.org/pipermail/python-list/2006-January/319862.html > That's probably not a bad idea. We should also add script explaining how to download the beta site data and the generation software so people can play with it and get ready to be webmasters :-) regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/ From tim at pollenation.net Wed Jan 18 16:47:42 2006 From: tim at pollenation.net (Tim Parkin) Date: Wed, 18 Jan 2006 15:47:42 +0000 Subject: [Python-Dev] Python icon In-Reply-To: <43CE60B0.1070806@holdenweb.com> References: <17354.20722.560136.55434@montanaro.dyndns.org> <17357.15206.570762.861782@montanaro.dyndns.org> <43CE55D1.5030107@gmail.com> <43CE60B0.1070806@holdenweb.com> Message-ID: <43CE631E.1080708@pollenation.net> Steve Holden wrote: >Nick Coghlan wrote: > > >>skip at pobox.com wrote: >> >> >> >>> Georg> Interesting, didn't even know a new page was in the making... Do >>> Georg> you know who is responsible for the new page? >>> >>>Tim Parkin is heading things up. Look here: >>> >>> http://beta.python.org/ >>> >>> >>I like the look of the new page, but it took a bit of digging to find anything >>about *how* it has been developed, or what needs to be done before the main >>URL can be switched to the new page (e.g. integrating the documentation >>properly). While it's in beta having that info prominent on the home page >>might be a good idea. Something like: >> >> "This is a beta of the redesigned www.python.org. If you have specific >>suggestions or display problems, submit them to the tracker . General >>questions or comments should be posted to comp.lang.python" >> >>I thought for a moment that this week's python-url was going to help me out, >>but it only linked the beta site without actually linking to any of the >>discussion it referenced in the relevant blurb. . . >> >>Appropriate enlightenment was eventually attained through Google: >>http://mail.python.org/pipermail/python-list/2006-January/319862.html >> >> >> >That's probably not a bad idea. We should also add script explaining how >to download the beta site data and the generation software so people can >play with it and get ready to be webmasters :-) > > > I'll try to change the home page content to have a comment about how the site is in beta and what to do to get the content and build software. I've also got to add a fix in on the current css.. If anybody needs the graphics, they can still be found on psf.pollenation.net in the resources folder (browser source > trunk > ...). Tim From raymond.hettinger at verizon.net Wed Jan 18 17:00:03 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Wed, 18 Jan 2006 11:00:03 -0500 Subject: [Python-Dev] str with base In-Reply-To: <20060118152356.GA20772@panix.com> Message-ID: <001501c61c48$3ed669a0$a3f9a244@oemcomputer> > > I'd propose bin() to stay in line with the short abbreviated names. > > There has been some previous discussion about removing hex()/oct() from > builtins for Python 3.0, IIRC. I sure don't think bin() belongs there. Perhaps introduce a single function, base(val, radix=10, prefix=''), as a universal base converter that could replace bin(), hex(), oct(), etc. That would give us fewer builtins and provide an inverse for all the int() conversions (i.e. arbitrary bases). Also, it would allow an unprefixed output which is what I usually need. Raymond From skip at pobox.com Wed Jan 18 17:07:57 2006 From: skip at pobox.com (skip at pobox.com) Date: Wed, 18 Jan 2006 10:07:57 -0600 Subject: [Python-Dev] str with base In-Reply-To: <001501c61c48$3ed669a0$a3f9a244@oemcomputer> References: <20060118152356.GA20772@panix.com> <001501c61c48$3ed669a0$a3f9a244@oemcomputer> Message-ID: <17358.26589.750736.623827@montanaro.dyndns.org> Raymond> Perhaps introduce a single function, base(val, radix=10, Raymond> prefix=''), as a universal base converter that could replace Raymond> bin(), hex(), oct(), etc. Would it (should it) work with floats, decimals, complexes? I presume it would work with ints and longs. Skip From jason.orendorff at gmail.com Wed Jan 18 17:39:38 2006 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Wed, 18 Jan 2006 11:39:38 -0500 Subject: [Python-Dev] str with base In-Reply-To: <1137586580.16309.28.camel@warna.dub.corp.google.com> References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> <20060117232843.GV18916@xs4all.nl> <20060118010127.GB6152@performancedrivers.com> <8BEAD4AB-6F49-4901-BBE7-77870E953AD5@redivi.com> <20060118031238.GC6152@performancedrivers.com> <4B72B1E6-EEA1-4DE1-A096-5E6D478340C2@redivi.com> <1137586580.16309.28.camel@warna.dub.corp.google.com> Message-ID: On 1/18/06, Donovan Baarda wrote: > I think supporting arbitrary bases for floats is way overkill and not > worth considering. If you mean actual base-3 floating-point arithmetic, I agree. That's outlandish. But if there were a stdlib function to format floats losslessly in hex or binary, Tim Peters would use it at least once every six weeks to illustrate the finer points of floating point arithmetic. <0.00390625 wink> +1.0 -j From steven.bethard at gmail.com Wed Jan 18 17:55:49 2006 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed, 18 Jan 2006 09:55:49 -0700 Subject: [Python-Dev] str with base In-Reply-To: <17358.26589.750736.623827@montanaro.dyndns.org> References: <20060118152356.GA20772@panix.com> <001501c61c48$3ed669a0$a3f9a244@oemcomputer> <17358.26589.750736.623827@montanaro.dyndns.org> Message-ID: [Raymond] > Perhaps introduce a single function, base(val, radix=10, > prefix=''), as a universal base converter that could replace > bin(), hex(), oct(), etc. +1 on introducing base() [Skip] > Would it (should it) work with floats, decimals, complexes? I presume it > would work with ints and longs. While support for floats, decimals, etc. might be nice (and I certainly wouldn't complain if someone wanted to supply the patch) I don't think those features should be necessary for base()'s initial introduction. If they're there, great, but if not, I don't think that should hold up the patch... STeVe -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From guido at python.org Wed Jan 18 18:52:59 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 18 Jan 2006 09:52:59 -0800 Subject: [Python-Dev] Building on OS X 10.4 fails In-Reply-To: <87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp> References: <1137525587.9528.19.camel@geddy.wooz.org> <200601181117.34549.anthony@interlink.com.au> <87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: On 1/17/06, Stephen J. Turnbull wrote: > >>>>> "Anthony" == Anthony Baxter writes: > > Anthony> It sounds like configure needs to grow a test to detect > Anthony> that a "libreadline" it finds is actually the crackful > Anthony> "libedit" and refuse to use it if so. > > FYI: Real libreadline is GPL, and rms made a point of forcing > (Aladdin-licensed) Ghostscript to remove stanzas from the Makefile > that allowed linking to it as a user option. Ie, this particular pain > in the neck is deliberate FSF policy, to encourage use of the GPL. Can we just all agree that RMS is an asshole now? Bah. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steve at holdenweb.com Wed Jan 18 18:59:25 2006 From: steve at holdenweb.com (Steve Holden) Date: Wed, 18 Jan 2006 17:59:25 +0000 Subject: [Python-Dev] str with base In-Reply-To: References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer> <20060117232843.GV18916@xs4all.nl> <20060118010127.GB6152@performancedrivers.com> <8BEAD4AB-6F49-4901-BBE7-77870E953AD5@redivi.com> <20060118031238.GC6152@performancedrivers.com> <4B72B1E6-EEA1-4DE1-A096-5E6D478340C2@redivi.com> <1137586580.16309.28.camel@warna.dub.corp.google.com> Message-ID: Jason Orendorff wrote: > On 1/18/06, Donovan Baarda wrote: > >>I think supporting arbitrary bases for floats is way overkill and not >>worth considering. > > > If you mean actual base-3 floating-point arithmetic, I agree. That's > outlandish. > > But if there were a stdlib function to format floats losslessly in hex > or binary, Tim Peters would use it at least once every six weeks to > illustrate the finer points of floating point arithmetic. <0.00390625 > wink> > > +1.0 > Nah, Tim's got the chops to use the struct model to get his point across. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/ From skip at pobox.com Wed Jan 18 19:34:16 2006 From: skip at pobox.com (skip at pobox.com) Date: Wed, 18 Jan 2006 12:34:16 -0600 Subject: [Python-Dev] Python icon In-Reply-To: <43CE60B0.1070806@holdenweb.com> References: <17354.20722.560136.55434@montanaro.dyndns.org> <17357.15206.570762.861782@montanaro.dyndns.org> <43CE55D1.5030107@gmail.com> <43CE60B0.1070806@holdenweb.com> Message-ID: <17358.35368.931097.508092@montanaro.dyndns.org> Steve> We should also add script explaining how to download the beta Steve> site data and the generation software so people can play with it Steve> and get ready to be webmasters :-) My first attempt ended almost immediately. Too much software to download and install for anything like casual use. Skip From skip at pobox.com Wed Jan 18 19:37:39 2006 From: skip at pobox.com (skip at pobox.com) Date: Wed, 18 Jan 2006 12:37:39 -0600 Subject: [Python-Dev] Building on OS X 10.4 fails In-Reply-To: References: <1137525587.9528.19.camel@geddy.wooz.org> <200601181117.34549.anthony@interlink.com.au> <87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <17358.35571.540508.624539@montanaro.dyndns.org> Anthony> It sounds like configure needs to grow a test to detect that a Anthony> "libreadline" it finds is actually the crackful "libedit" and Anthony> refuse to use it if so. >> FYI: Real libreadline is GPL, ... Didn't Python's readline module work with libedit once upon a time? I believe that's the readline semi-clone that Rich Salz quite awhile ago. I've forgotten the details of the difference (missing history capabilities perhaps). Is there some way we can make libedit palatable again? Skip From brett at python.org Wed Jan 18 20:09:00 2006 From: brett at python.org (Brett Cannon) Date: Wed, 18 Jan 2006 11:09:00 -0800 Subject: [Python-Dev] str with base In-Reply-To: <001501c61c48$3ed669a0$a3f9a244@oemcomputer> References: <20060118152356.GA20772@panix.com> <001501c61c48$3ed669a0$a3f9a244@oemcomputer> Message-ID: On 1/18/06, Raymond Hettinger wrote: > > > I'd propose bin() to stay in line with the short abbreviated names. > > > > There has been some previous discussion about removing hex()/oct() > from > > builtins for Python 3.0, IIRC. I sure don't think bin() belongs > there. > > Perhaps introduce a single function, base(val, radix=10, prefix=''), as > a universal base converter that could replace bin(), hex(), oct(), etc. > > That would give us fewer builtins and provide an inverse for all the > int() conversions (i.e. arbitrary bases). Also, it would allow an > unprefixed output which is what I usually need. > +1. Differs from Neal's format() function by not magically determining the prefix from the radix which I like. -Brett From bob at redivi.com Wed Jan 18 20:24:23 2006 From: bob at redivi.com (Bob Ippolito) Date: Wed, 18 Jan 2006 11:24:23 -0800 Subject: [Python-Dev] Building on OS X 10.4 fails In-Reply-To: <200601182031.54111.anthony@interlink.com.au> References: <200601181117.34549.anthony@interlink.com.au> <87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp> <200601182031.54111.anthony@interlink.com.au> Message-ID: <807E9CF0-1742-41A2-8DB4-96D9C9424BDD@redivi.com> On Jan 18, 2006, at 1:31 AM, Anthony Baxter wrote: > On Wednesday 18 January 2006 16:25, Stephen J. Turnbull wrote: >> Unless rms has changed his position on this, or there has been >> relevant legislation or a court decision in the meantime, >> explicitly requiring or checking for "real" libreadline, even as a >> user option, risks rms's wrath. (Of course, Python could change >> its license to GPL, which would undoubtedly flood Cambridge with >> tears of joy). > > Python's license is GPL-compatible, so this isn't an issue. > >> As long as the link to fake libreadline succeeds and the resulting >> program works identically to one linked to real libreadline, he has >> no complaint. > > My "complaint" is that libedit _calls_ itself libreadline, when it's > pretty clear that it's not actually a drop-in replacement (or the > readline module would build). Hence my use of the word "crackful" > That's just something stupid Apple did. Not libedit's fault. -bob From martin at v.loewis.de Wed Jan 18 20:32:42 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 18 Jan 2006 20:32:42 +0100 Subject: [Python-Dev] [Python-checkins] r42090 - in python/trunk: Modules/getbuildinfo.c PCbuild/make_buildinfo.vcproj PCbuild/pcbuild.sln PCbuild/pythoncore.vcproj In-Reply-To: <1f7befae0601180721vbc67e95u54bfafb0569357fe@mail.gmail.com> References: <20060118091354.A31B91E4002@bag.python.org> <1f7befae0601180721vbc67e95u54bfafb0569357fe@mail.gmail.com> Message-ID: <43CE97DA.5080302@v.loewis.de> Tim Peters wrote: > Did this checkin perhaps forget to add that file? Oops, indeed - please try again. Regards, Martin From jason.orendorff at gmail.com Wed Jan 18 21:02:22 2006 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Wed, 18 Jan 2006 15:02:22 -0500 Subject: [Python-Dev] basenumber redux In-Reply-To: <43CD7BE9.6060705@v.loewis.de> References: <17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com> <43CC17B9.3060200@v.loewis.de> <37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com> <43CCC0AE.5070800@egenix.com> <43CD7BE9.6060705@v.loewis.de> Message-ID: On 1/17/06, "Martin v. L?wis" wrote: > Alex Martelli wrote: > > But this doesn't apply to the Python Standard Library, for example see > > line 1348 of imaplib.py: "if isinstance(date_time, (int, float)):". > [...] > > Being able to change imaplib to use basenumber instead of (int, float) > > won't make it SIMPLER, but it will surely make it BETTER -- why should > > a long be rejected, or a Decimal, for that matter? > > Right. I think this function should read > > if isinstance(date_time, str) and \ > (date_time[0],date_time[-1]) == ('"','"'): > return date_time # Assume in correct format > > if isinstance(date_time, (tuple, time.struct_time)): > tt = date_time > else: > tt = time.localtime(date_time) So... arbitrary number-like objects should work, but arbitrary sequence-like objects should fail? Hmmm. Maybe that "if isinstance()" line should say "if hasattr(date_time, '__getitem__'):". Am I sure? No. The original author of imaplib apparently got it wrong, and Martin got it wrong, and they're both smarter than me. Really this is just further proof that type-checking is a royal pain in Python. Or rather, it's not hard to cover the builtin and stdlib types, but it's hard to support "duck typing" too. Are we going about this the right way? Options: 1. Redesign the API so each parameter has a clearly defined set of operations it must support, thus eliminating the need for type-checking. Drawback: An annoying API might be worse than the problem we're trying to solve. 2. Write a set of imprecise, general-purpose type-checking functions (is_real_number(v), is_sequence(v), ...) and use those. (They are imprecise because the requirements are vague and because it's not really possible to pin them down.) Drawback: Error-prone, compounded by deceptively clean appearance. 3. Write very specific custom type-checking code each time you need it (the imaplib approach). Drawbacks: Error-prone (as we've seen), precarious, tedious, unreadable. 4. Use the "better-to-ask-forgiveness-than-permission" idiom. Drawback: Potential bad behavior on error, again potentially worse than the original problem. Yuck. Does anyone have the answer to this one? Or is the problem not as bad as it looks? -j From tim.peters at gmail.com Wed Jan 18 21:04:15 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 18 Jan 2006 15:04:15 -0500 Subject: [Python-Dev] [Python-checkins] r42090 - in python/trunk: Modules/getbuildinfo.c PCbuild/make_buildinfo.vcproj PCbuild/pcbuild.sln PCbuild/pythoncore.vcproj In-Reply-To: <43CE97DA.5080302@v.loewis.de> References: <20060118091354.A31B91E4002@bag.python.org> <1f7befae0601180721vbc67e95u54bfafb0569357fe@mail.gmail.com> <43CE97DA.5080302@v.loewis.de> Message-ID: <1f7befae0601181204pdf33b70me0109b0c747a7d71@mail.gmail.com> [Tim] >> Did this checkin perhaps forget to add that file? [Martin] > Oops, indeed - please try again. It may ;-) be better now. Looks like the release build finished, but the debug build died: """ ------ Build started: Project: pythoncore, Configuration: Debug Win32 ------ generate buildinfo C:\Program Files\TortoiseSVN\bin\subwcrev.exe .. ..\Modules\getbuildinfo.c getbuildinfo2.c 'C:\Program' is not recognized as an internal or external command, operable program or batch file. cl.exe -c -D_WIN32 -DUSE_DL_EXPORT -D_WINDOWS -DWIN32 -D_WINDLL -D_DEBUG -MDd getbuildinfo2.c -DSUBWCREV -Fogetbuildinfo.o -I..\Include -I..\PC Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077 for 80x86 Copyright (C) Microsoft Corporation 1984-2002. All rights reserved. getbuildinfo2.c c1 : fatal error C1083: Cannot open source file: 'getbuildinfo2.c': No such file or directory Linking... LINK : fatal error LNK1181: cannot open input file 'getbuildinfo.o' """ OK, I added code to getbuildinfo.c to quote the path to the executable before invoking system(), and that seemed to fix it. On my box. From jimjjewett at gmail.com Wed Jan 18 21:22:37 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed, 18 Jan 2006 15:22:37 -0500 Subject: [Python-Dev] str with base Message-ID: Jack wrote: > The arbitrary base case isn't even academic > or we would see homework questions about it > on c.l.py. No one asks about > hex or octal because they are there. I have wanted base-36 far more often than I've wanted base-8. I haven't needed any base (except 10) often enough to justify putting it in builtins rather than a stdlib module. I do like the idea of adding Raymond's def base(number, base, prefix) to the stdlib (possibly in math). -jJ From martin at v.loewis.de Wed Jan 18 21:36:20 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 18 Jan 2006 21:36:20 +0100 Subject: [Python-Dev] basenumber redux In-Reply-To: References: <17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com> <43CC17B9.3060200@v.loewis.de> <37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com> <43CCC0AE.5070800@egenix.com> <43CD7BE9.6060705@v.loewis.de> Message-ID: <43CEA6C4.5060201@v.loewis.de> Jason Orendorff wrote: > Really this is just further proof that type-checking is a royal pain > in Python. Or rather, it's not hard to cover the builtin and stdlib > types, but it's hard to support "duck typing" too. Are we going about > this the right way? It's not as bad. There is nothing wrong with restricting the set of acceptable types if callers would have no problems to convert their input into one of the acceptable types. In the imaplib example, requesting that a broken-down time is passed as a tuple or a time.struct_time is not too hard for a caller. It will be formatted as dt = time.strftime("%d-%b-%Y %H:%M:%S", tt) so it needs to have the right number of fields. Callers having other kinds of sequence can easily use tuple(L) to convert their data into what the function accepts. Regards, Martin From tim at pollenation.net Wed Jan 18 21:40:44 2006 From: tim at pollenation.net (Tim Parkin) Date: Wed, 18 Jan 2006 20:40:44 +0000 Subject: [Python-Dev] Python icon In-Reply-To: <17358.35368.931097.508092@montanaro.dyndns.org> References: <17354.20722.560136.55434@montanaro.dyndns.org> <17357.15206.570762.861782@montanaro.dyndns.org> <43CE55D1.5030107@gmail.com> <43CE60B0.1070806@holdenweb.com> <17358.35368.931097.508092@montanaro.dyndns.org> Message-ID: <43CEA7CC.4090602@pollenation.net> skip at pobox.com wrote: > Steve> We should also add script explaining how to download the beta > Steve> site data and the generation software so people can play with it > Steve> and get ready to be webmasters :-) > >My first attempt ended almost immediately. Too much software to download >and install for anything like casual use. > > > For casual use why not just edit the rest file? That was one of main points about creating a file based admin system... Most users can just edit the rest files or data files (one of the library dependences could have been avoided by using xml for the data files but I'm not sure this would have gone down well). Creating new pages is also mostly just writing rest content (if you want to do this but can't be bothered to install the software, send your work over to me and I'll add it). You only need to have all of the software if you are creating whole new sections, in which case it wouldn't really be casual use. I may be wrong in saying this, but I thought the main idea of having software libraries was to use them? I avoided writing specific modules because the software was out there that did things a lot better than I could rewrite. Is there a level of library use that is acceptable beyond which we should rewrite components or bundle libraries? I would have thought most applications that involved web site creation would have at least three or four external library dependencies (templating, data interface, web server/http module). Tim Parkin From skip at pobox.com Wed Jan 18 22:53:26 2006 From: skip at pobox.com (skip at pobox.com) Date: Wed, 18 Jan 2006 15:53:26 -0600 Subject: [Python-Dev] Python icon In-Reply-To: <43CEA7CC.4090602@pollenation.net> References: <17354.20722.560136.55434@montanaro.dyndns.org> <17357.15206.570762.861782@montanaro.dyndns.org> <43CE55D1.5030107@gmail.com> <43CE60B0.1070806@holdenweb.com> <17358.35368.931097.508092@montanaro.dyndns.org> <43CEA7CC.4090602@pollenation.net> Message-ID: <17358.47318.720591.658564@montanaro.dyndns.org> >> My first attempt ended almost immediately. Too much software to download >> and install for anything like casual use. Tim> For casual use why not just edit the rest file? Maybe I misread the directions. I thought I had to install some new library I'd never heard of (syck), Python bindings for the same, and maybe some other stuff. It clearly wasn't just "svn co ..." and start editing. In any case, I couldn't tell what needed doing from the trac site. I'll take another look when I have a chance. I would be happy to simple edit rest files if I knew what to edit. Skip From tim at pollenation.net Wed Jan 18 23:26:40 2006 From: tim at pollenation.net (Tim Parkin) Date: Wed, 18 Jan 2006 22:26:40 +0000 Subject: [Python-Dev] Python icon In-Reply-To: <17358.47318.720591.658564@montanaro.dyndns.org> References: <17354.20722.560136.55434@montanaro.dyndns.org> <17357.15206.570762.861782@montanaro.dyndns.org> <43CE55D1.5030107@gmail.com> <43CE60B0.1070806@holdenweb.com> <17358.35368.931097.508092@montanaro.dyndns.org> <43CEA7CC.4090602@pollenation.net> <17358.47318.720591.658564@montanaro.dyndns.org> Message-ID: <43CEC0A0.20805@pollenation.net> skip at pobox.com wrote: > >> My first attempt ended almost immediately. Too much software to download > >> and install for anything like casual use. > > Tim> For casual use why not just edit the rest file? > > Maybe I misread the directions. I thought I had to install some new library > I'd never heard of (syck), Python bindings for the same, and maybe some > other stuff. It clearly wasn't just "svn co ..." and start editing. In any > case, I couldn't tell what needed doing from the trac site. > > I'll take another look when I have a chance. I would be happy to simple > edit rest files if I knew what to edit. To build all of the web pages, you need to have a yaml parser installed. As much as I'd like a pure python yaml parser, the syck c parser is currently the definitive implementation (and the pysyck wrapper at xitology the definitive python wrapper). The syck files are used to manage the data for the templates but 95% of the pages I've migrated have just been a simple nav yaml file with a rest file for the body of the content. The beta site is being rebuilt on commit so if you see any content missing, it needs generating. I've added a couple of tickets highlighting large areas that need work (the tickets on psf.pollenation.net with "CONTENT" as a prefix). If you want to proof content you can just go to the appropriate data directory (matching the url of the page in question) and edit the rest file. If you want to add new pages, it would be good to be able to build the site to check what you have added is working. However, as there aren't many people working on the site, I'd be more than happy to take whatever you can supply and integrate it into the site. At the moment I'm torn between making the software easier to use or making sure content is migrated. It should only take about 10 or 20 minutes to get the software installed (if it takes longer then please tell me what the problems are and I'll try to document any issues and streamline the process - I'd rather not optimise the install/docs until there are specific problems - I think my time is better spent getting some of the more complex content migrated, and ironing out the last problems with the css, and getting new images for the home page and writing new content for the 'for ...' sections and getting the success stories copied over, and getting wiki content integrated and getting the docs integrated into the site better, and ..well .. other stuff). I really want to make the website easier for people to work on and fully intend to provide some form of simple web based content management when enough content has been migrated to allow the site to go live. I hope that wasn't too much of a rant.. :-) Drop me an email if I can help in any way. Tim From ncoghlan at gmail.com Thu Jan 19 01:08:41 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 19 Jan 2006 10:08:41 +1000 Subject: [Python-Dev] str with base In-Reply-To: References: <20060118152356.GA20772@panix.com> <001501c61c48$3ed669a0$a3f9a244@oemcomputer> Message-ID: <43CED889.3010306@gmail.com> Brett Cannon wrote: > On 1/18/06, Raymond Hettinger wrote: >>>> I'd propose bin() to stay in line with the short abbreviated names. >>> There has been some previous discussion about removing hex()/oct() >> from >>> builtins for Python 3.0, IIRC. I sure don't think bin() belongs >> there. >> >> Perhaps introduce a single function, base(val, radix=10, prefix=''), as >> a universal base converter that could replace bin(), hex(), oct(), etc. >> >> That would give us fewer builtins and provide an inverse for all the >> int() conversions (i.e. arbitrary bases). Also, it would allow an >> unprefixed output which is what I usually need. >> > > +1. Differs from Neal's format() function by not magically > determining the prefix from the radix which I like. +1 here, too, particularly if hex/oct acquire Deprecation (or even just PendingDeprecation) warnings at the same time. I have my own reason for wanting to avoid the name format() - I'd still like to see it used one day to provide a builtin way to use string.Template syntax for arbitrary string formatting. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From anthony at interlink.com.au Thu Jan 19 02:36:36 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu, 19 Jan 2006 12:36:36 +1100 Subject: [Python-Dev] Python icon In-Reply-To: <17358.47318.720591.658564@montanaro.dyndns.org> References: <43CEA7CC.4090602@pollenation.net> <17358.47318.720591.658564@montanaro.dyndns.org> Message-ID: <200601191236.41719.anthony@interlink.com.au> On Thursday 19 January 2006 08:53, skip at pobox.com wrote: > Maybe I misread the directions. I thought I had to install some > new library I'd never heard of (syck), Python bindings for the > same, and maybe some other stuff. It clearly wasn't just "svn co > ..." and start editing. In any case, I couldn't tell what needed > doing from the trac site. We obviously need to get everything that's needed to use setuptools and eggs so people can just have things "just work". Yay for PJE! -- Anthony Baxter It's never too late to have a happy childhood. From aleaxit at gmail.com Thu Jan 19 03:16:03 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Wed, 18 Jan 2006 18:16:03 -0800 Subject: [Python-Dev] str with base In-Reply-To: References: <20060118152356.GA20772@panix.com> <001501c61c48$3ed669a0$a3f9a244@oemcomputer> Message-ID: <050D3FEE-E91C-4015-ADD9-228422DC39FA@gmail.com> On Jan 18, 2006, at 11:09 AM, Brett Cannon wrote: > On 1/18/06, Raymond Hettinger wrote: >>>> I'd propose bin() to stay in line with the short abbreviated names. >>> >>> There has been some previous discussion about removing hex()/oct() >> from >>> builtins for Python 3.0, IIRC. I sure don't think bin() belongs >> there. >> >> Perhaps introduce a single function, base(val, radix=10, >> prefix=''), as >> a universal base converter that could replace bin(), hex(), oct(), >> etc. >> >> That would give us fewer builtins and provide an inverse for all the >> int() conversions (i.e. arbitrary bases). Also, it would allow an >> unprefixed output which is what I usually need. > > +1. Differs from Neal's format() function by not magically > determining the prefix from the radix which I like. I'm not sure I see the advantage of, say, print base(x, radix=2, prefix='0b') versus print '0b'+base(x, radix=2) IOW, if the prefix needs to be explicitly specified anyway, what's the advantage of specifying it as an argument to base, rather than just string-concatenating it? Apart from that quibble, the base function appears to cover all the use cases for my proposed str-with-base, so, since it appears to attract less arguments, I'm definitely +1 on it. Alex From brett at python.org Thu Jan 19 03:50:53 2006 From: brett at python.org (Brett Cannon) Date: Wed, 18 Jan 2006 18:50:53 -0800 Subject: [Python-Dev] str with base In-Reply-To: <050D3FEE-E91C-4015-ADD9-228422DC39FA@gmail.com> References: <20060118152356.GA20772@panix.com> <001501c61c48$3ed669a0$a3f9a244@oemcomputer> <050D3FEE-E91C-4015-ADD9-228422DC39FA@gmail.com> Message-ID: On 1/18/06, Alex Martelli wrote: > On Jan 18, 2006, at 11:09 AM, Brett Cannon wrote: > > > On 1/18/06, Raymond Hettinger wrote: > >>>> I'd propose bin() to stay in line with the short abbreviated names. > >>> > >>> There has been some previous discussion about removing hex()/oct() > >> from > >>> builtins for Python 3.0, IIRC. I sure don't think bin() belongs > >> there. > >> > >> Perhaps introduce a single function, base(val, radix=10, > >> prefix=''), as > >> a universal base converter that could replace bin(), hex(), oct(), > >> etc. > >> > >> That would give us fewer builtins and provide an inverse for all the > >> int() conversions (i.e. arbitrary bases). Also, it would allow an > >> unprefixed output which is what I usually need. > > > > +1. Differs from Neal's format() function by not magically > > determining the prefix from the radix which I like. > > I'm not sure I see the advantage of, say, > > print base(x, radix=2, prefix='0b') > > versus > > print '0b'+base(x, radix=2) > > IOW, if the prefix needs to be explicitly specified anyway, what's > the advantage of specifying it as an argument to base, rather than > just string-concatenating it? It collects the data that is expected to be used in the common case in a single location/operation. This would allow you to do something like ``base(x, **radix_and_prefix_dict)`` and have everythihng in a nice, neat package. Plus the operation would be faster if base() is written in C. =) The other option is to go with Neal's solution for automatically including the prefix for known prefix types, but instead of it being a boolean, let it be a string argument. That means if you want no prefix you would just set the argument to the empty string. Not setting it will just use the most sensible prefix or none if one is not known for the specified radix. Could have something somewhere, like string or math, where more radix/prefix pairs can be added by the user and have base() reference that for its prefix values. IOW I am +0 on prefix in one of these forms. -Brett From aahz at pythoncraft.com Thu Jan 19 05:40:17 2006 From: aahz at pythoncraft.com (Aahz) Date: Wed, 18 Jan 2006 20:40:17 -0800 Subject: [Python-Dev] Building on OS X 10.4 fails In-Reply-To: References: <1137525587.9528.19.camel@geddy.wooz.org> <200601181117.34549.anthony@interlink.com.au> <87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <20060119044017.GB11147@panix.com> On Wed, Jan 18, 2006, Guido van Rossum wrote: > > Can we just all agree that RMS is an asshole now? Bah. "Citing RMS's insanity is a great way to get my blood steaming." --GvR -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From foom at fuhm.net Thu Jan 19 05:47:59 2006 From: foom at fuhm.net (James Y Knight) Date: Wed, 18 Jan 2006 23:47:59 -0500 Subject: [Python-Dev] Building on OS X 10.4 fails In-Reply-To: <20060119044017.GB11147@panix.com> References: <1137525587.9528.19.camel@geddy.wooz.org> <200601181117.34549.anthony@interlink.com.au> <87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp> <20060119044017.GB11147@panix.com> Message-ID: <188C313F-D2C6-4315-9710-7FACA3D87799@fuhm.net> On Jan 18, 2006, at 11:40 PM, Aahz wrote: > On Wed, Jan 18, 2006, Guido van Rossum wrote: >> >> Can we just all agree that RMS is an asshole now? Bah. > > "Citing RMS's insanity is a great way to get my blood steaming." -- > GvR Ya know, you don't *have* to use his software. For example, python could stop supporting readline and support libedit instead. James From bob at redivi.com Thu Jan 19 05:54:06 2006 From: bob at redivi.com (Bob Ippolito) Date: Wed, 18 Jan 2006 20:54:06 -0800 Subject: [Python-Dev] Building on OS X 10.4 fails In-Reply-To: <188C313F-D2C6-4315-9710-7FACA3D87799@fuhm.net> References: <1137525587.9528.19.camel@geddy.wooz.org> <200601181117.34549.anthony@interlink.com.au> <87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp> <20060119044017.GB11147@panix.com> <188C313F-D2C6-4315-9710-7FACA3D87799@fuhm.net> Message-ID: <3A4F7AED-0D0B-4D31-806B-C3F8886C05CA@redivi.com> On Jan 18, 2006, at 8:47 PM, James Y Knight wrote: > > On Jan 18, 2006, at 11:40 PM, Aahz wrote: > >> On Wed, Jan 18, 2006, Guido van Rossum wrote: >>> >>> Can we just all agree that RMS is an asshole now? Bah. >> >> "Citing RMS's insanity is a great way to get my blood steaming." -- >> GvR > > Ya know, you don't *have* to use his software. For example, python > could stop supporting readline and support libedit instead. It'd probably be better to just use something like pyrepl , which builds on top of curses and termios. More Python code, less GPL mess. -bob From aleaxit at gmail.com Thu Jan 19 05:57:45 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Wed, 18 Jan 2006 20:57:45 -0800 Subject: [Python-Dev] Building on OS X 10.4 fails In-Reply-To: <188C313F-D2C6-4315-9710-7FACA3D87799@fuhm.net> References: <1137525587.9528.19.camel@geddy.wooz.org> <200601181117.34549.anthony@interlink.com.au> <87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp> <20060119044017.GB11147@panix.com> <188C313F-D2C6-4315-9710-7FACA3D87799@fuhm.net> Message-ID: On Jan 18, 2006, at 8:47 PM, James Y Knight wrote: > On Jan 18, 2006, at 11:40 PM, Aahz wrote: > >> On Wed, Jan 18, 2006, Guido van Rossum wrote: >>> >>> Can we just all agree that RMS is an asshole now? Bah. >> >> "Citing RMS's insanity is a great way to get my blood steaming." -- >> GvR > > Ya know, you don't *have* to use his software. For example, python > could stop supporting readline and support libedit instead. In Py3k this should indeed be considered; I don't think it can be within Python 2.*, since it would be a major and never fully backwards-compatible change. Alex From nnorwitz at gmail.com Thu Jan 19 07:29:50 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 18 Jan 2006 22:29:50 -0800 Subject: [Python-Dev] New PEP: Using ssize_t as the index type In-Reply-To: <43C37C02.8030005@egenix.com> References: <43B3ECEE.9060300@v.loewis.de> <20051229173041.GA19575@code1.codespeak.net> <43B50B64.20904@v.loewis.de> <20060105194538.GB11736@code1.codespeak.net> <43BD8464.4060801@v.loewis.de> <43C37C02.8030005@egenix.com> Message-ID: On 1/10/06, M.-A. Lemburg wrote: > > We'd also have to make sure that old extensions don't > just import with a warning, since the change will introduce > buffer overflows and seg faults in extensions that are not > aware of the change. I agree that on 64-bit platforms we should prevent the import. In the past we only provided a warning and the users were on their own. This is different. If you read my massive checkin to check the return results of Py_InitModule*(), you'll realize this isn't as simple as just failing in Py_InitMethod*(). I was hoping to just modify Py_InitModule4() in Python/modsupport.c to fail and return NULL. That doesn't seem practical given that we didn't check return results. We will just crash the interpreter with standard python 2.4 modules. ISTM we need to modify _PyImport_LoadDynamicModule() in Python/importdl.c before calling the init function (line 56, (*p)()) to check for some magic symbol that is defined only when compiling 2.5 and above. For example we could add a static int _64_bit_clean = 1; in modsupport.h. Without some trickery we will get this defined in every .o file though, not just modules. Other ideas? n From python at rcn.com Thu Jan 19 07:55:42 2006 From: python at rcn.com (Raymond Hettinger) Date: Thu, 19 Jan 2006 01:55:42 -0500 Subject: [Python-Dev] basenumber redux In-Reply-To: <43CEA6C4.5060201@v.loewis.de> Message-ID: <001c01c61cc5$6084f8a0$50b02c81@oemcomputer> > -----Original Message----- > From: python-dev-bounces+python=rcn.com at python.org [mailto:python-dev- > bounces+python=rcn.com at python.org] On Behalf Of Martin v. L?wis > Sent: Wednesday, January 18, 2006 3:36 PM > To: Jason Orendorff > Cc: python-dev at python.org > Subject: Re: [Python-Dev] basenumber redux > > Jason Orendorff wrote: > > Really this is just further proof that type-checking is a royal pain > > in Python. Or rather, it's not hard to cover the builtin and stdlib > > types, but it's hard to support "duck typing" too. Are we going about > > this the right way? > > It's not as bad. There is nothing wrong with restricting the set of > acceptable types if callers would have no problems to convert their > input into one of the acceptable types. Somehow my earlier post on this thread didn't seem to take. There are problems for callers converting their inputs: * currently existing number-like objects would need be retro-fitted with a new base class and possibly change their behavior from old-style to new-style. * some useful classes (such as a symbolic type) already inherit from str. That prevents them from also being able to inherit from basenumber. I'm -1 on the proposal because the benefits are dubious (at best it simplifies just a handful of code fragments); it adds yet another API to learn and remember; it is darned inconvenient for existing code seeking to emulate number-like behavior; and it precludes number emulation for classes that already have a built-in base class. For the most part, functions that enforce type checking are up to no good and make life harder for their callers. If Python ultimately grows interfaces, I hope they remain optional; as soon as functions start insisting on interface checking, then it will spread like cancer. The basenumber proposal is essentially a step down that slippery slope. Raymond Raymond From raymond.hettinger at verizon.net Thu Jan 19 08:26:12 2006 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Thu, 19 Jan 2006 02:26:12 -0500 Subject: [Python-Dev] str with base In-Reply-To: Message-ID: <003801c61cc9$a0f73e80$50b02c81@oemcomputer> Guido, we may be converging on a consensus for my proposal: base(value, radix=2) So far no one has shot at it, and it has gathered +1's from Steven, Alex, Brett, and Nick. To keep it simple, the proposal is for the value to be any int or long. With an underlying __base__ method call, it wouldn't be hard for someone to build it out to support other numeric types if the need arises. The output would have no prefixes. As Alex pointed out, it is easier and more direct to add those after the fact if needed. Care to pronounce on it? Raymond From nnorwitz at gmail.com Thu Jan 19 08:37:16 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 18 Jan 2006 23:37:16 -0800 Subject: [Python-Dev] str with base In-Reply-To: <003801c61cc9$a0f73e80$50b02c81@oemcomputer> References: <003801c61cc9$a0f73e80$50b02c81@oemcomputer> Message-ID: On 1/18/06, Raymond Hettinger wrote: > Guido, we may be converging on a consensus for my proposal: > > base(value, radix=2) > > So far no one has shot at it, and it has gathered +1's from Steven, > Alex, Brett, and Nick. +1 for me too, but I'd also like to deprecate hex() and oct() and slate them for removal in 3k. To expand, valid radix values would be 2..36 (ie, same as for int). It was discussed putting base() in some module. Was there consensus about builtin vs a module? I'd prefer a module, but builtin is ok with me. > To keep it simple, the proposal is for the value to be any int or long. > With an underlying __base__ method call, it wouldn't be hard for someone > to build it out to support other numeric types if the need arises. > > The output would have no prefixes. As Alex pointed out, it is easier > and more direct to add those after the fact if needed. +1 > Care to pronounce on it? > > > Raymond From bob at redivi.com Thu Jan 19 08:58:01 2006 From: bob at redivi.com (Bob Ippolito) Date: Wed, 18 Jan 2006 23:58:01 -0800 Subject: [Python-Dev] str with base In-Reply-To: References: <003801c61cc9$a0f73e80$50b02c81@oemcomputer> Message-ID: <71F6D09D-93E0-44E1-8B58-EC3A9A37D1C6@redivi.com> On Jan 18, 2006, at 11:37 PM, Neal Norwitz wrote: > On 1/18/06, Raymond Hettinger wrote: >> Guido, we may be converging on a consensus for my proposal: >> >> base(value, radix=2) >> >> So far no one has shot at it, and it has gathered +1's from Steven, >> Alex, Brett, and Nick. > > +1 for me too, but I'd also like to deprecate hex() and oct() and > slate them for removal in 3k. > > To expand, valid radix values would be 2..36 (ie, same as for int). > It was discussed putting base() in some module. Was there consensus > about builtin vs a module? I'd prefer a module, but builtin is ok > with me. I'd drop the default radix, or make it something common like 16... especially if hex and oct are to be py3k deprecated. +1 for: base(value, radix) +1 for: "%b" % (integer,) +0 for binary literals: 0b01101 -bob From mal at egenix.com Thu Jan 19 10:18:58 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 19 Jan 2006 10:18:58 +0100 Subject: [Python-Dev] New PEP: Using ssize_t as the index type In-Reply-To: References: <43B3ECEE.9060300@v.loewis.de> <20051229173041.GA19575@code1.codespeak.net> <43B50B64.20904@v.loewis.de> <20060105194538.GB11736@code1.codespeak.net> <43BD8464.4060801@v.loewis.de> <43C37C02.8030005@egenix.com> Message-ID: <43CF5982.5050802@egenix.com> Neal Norwitz wrote: > On 1/10/06, M.-A. Lemburg wrote: >> We'd also have to make sure that old extensions don't >> just import with a warning, since the change will introduce >> buffer overflows and seg faults in extensions that are not >> aware of the change. > > I agree that on 64-bit platforms we should prevent the import. In the > past we only provided a warning and the users were on their own. This > is different. > > If you read my massive checkin to check the return results of > Py_InitModule*(), you'll realize this isn't as simple as just failing > in Py_InitMethod*(). I was hoping to just modify Py_InitModule4() in > Python/modsupport.c to fail and return NULL. That doesn't seem > practical given that we didn't check return results. We will just > crash the interpreter with standard python 2.4 modules. > > ISTM we need to modify _PyImport_LoadDynamicModule() in > Python/importdl.c before calling the init function (line 56, (*p)()) > to check for some magic symbol that is defined only when compiling 2.5 > and above. For example we could add a static int _64_bit_clean = 1; > in modsupport.h. Without some trickery we will get this defined in > every .o file though, not just modules. > > Other ideas? We could explicitly break binary compatibility for Python 2.5 on 64-bit platforms, by changing the name of an often used API, e.g. the Py_InitModule*() APIs. This is how Unicode does it - we map the various APIs to either ...UCS2 or ...UCS4, so that you cannot import an extension compiled for e.g. UCS2 into a Python interpreter compiled for UCS4. If we didn't, you'd get seg faults and buffer overflows the same way you would with the ssize_t change on 64-bit platforms. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 19 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From tim at pollenation.net Thu Jan 19 10:46:32 2006 From: tim at pollenation.net (Tim Parkin) Date: Thu, 19 Jan 2006 09:46:32 +0000 Subject: [Python-Dev] Python icon In-Reply-To: <200601191236.41719.anthony@interlink.com.au> References: <43CEA7CC.4090602@pollenation.net> <17358.47318.720591.658564@montanaro.dyndns.org> <200601191236.41719.anthony@interlink.com.au> Message-ID: <43CF5FF8.2000803@pollenation.net> Anthony Baxter wrote: > On Thursday 19 January 2006 08:53, skip at pobox.com wrote: > >>Maybe I misread the directions. I thought I had to install some >>new library I'd never heard of (syck), Python bindings for the >>same, and maybe some other stuff. It clearly wasn't just "svn co >>..." and start editing. In any case, I couldn't tell what needed >>doing from the trac site. > > > We obviously need to get everything that's needed to use setuptools > and eggs so people can just have things "just work". Yay for PJE! > > :-D thats the plan!! only problem is getting zope interface to build as an egg :-( any ideas? Tim From gmccaughan at synaptics-uk.com Thu Jan 19 11:23:30 2006 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Thu, 19 Jan 2006 10:23:30 +0000 Subject: [Python-Dev] str with base In-Reply-To: References: <20060118152356.GA20772@panix.com> <17358.26589.750736.623827@montanaro.dyndns.org> Message-ID: <200601191023.30902.gmccaughan@synaptics-uk.com> On Wednesday 2006-01-18 16:55, Steven Bethard wrote: > [Raymond] > > Perhaps introduce a single function, base(val, radix=10, > > prefix=''), as a universal base converter that could replace > > bin(), hex(), oct(), etc. > > +1 on introducing base() Introducing a new builtin with a name that's a common, short English word is a bit disagreeable. The other thing about the name "base" is that it's not entirely obvious which way it converts: do you say base(123,5) to get a string representing 123 in base 5, or base("123",5) to get the integer whose base 5 representation is "123"? Well, one option would be to have both of those work :-). (Some people may need to do some deep breathing while reciting the mantra "practicality beats purity" in order to contemplate that with equanimity.) Alternatively, a name like "to_base" that clarifies the intent and is less likely to clash with variable names might be an improvement. Or there's always %b, whether that ends up standing for "binary" or "base". Or %b for binary and %r for radix, not forgetting the modifiers to get numbers formatted as Roman numerals. -- Gareth McCaughan From thomas at xs4all.net Thu Jan 19 12:15:50 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Thu, 19 Jan 2006 12:15:50 +0100 Subject: [Python-Dev] str with base In-Reply-To: <200601191023.30902.gmccaughan@synaptics-uk.com> References: <20060118152356.GA20772@panix.com> <17358.26589.750736.623827@montanaro.dyndns.org> <200601191023.30902.gmccaughan@synaptics-uk.com> Message-ID: <20060119111550.GX18916@xs4all.nl> On Thu, Jan 19, 2006 at 10:23:30AM +0000, Gareth McCaughan wrote: > > +1 on introducing base() > Introducing a new builtin with a name that's a common, short > English word is a bit disagreeable. While I don't particularly mind the new function in either the builtin module or another, like math, I don't understand the problem with the name. Most builtin names are short and english-ish words. I like that, I'm glad they are that way, and the two names I dislike most are 'isinstance' and 'issubclass'. > The other thing about the name "base" is that it's not entirely obvious > which way it converts: do you say > > base(123,5) > > to get a string representing 123 in base 5, or > > base("123",5) > > to get the integer whose base 5 representation is "123"? This is an argument for 'str(123, 5)', but I don't agree. Not _everything_ has to be obvious at first glance. The very same could be said about hex(), oct(), dir(), even names like list() (what does it list?), str() (stripping something?), etc. Having int() do it one way and base() the other makes fine sense to me, and I don't see it as any harder to explain than, say, why hex("123") doesn't return 291. I've personally never had to explain hex/oct's behaviour. While I think 'str' would be a slightly better name than 'base' (despite the specialcasing of numbers,) I don't mind either. I do mind names like 'to_base' or 'as_str' or 'shouldbestr' or other names that make me turn on autocompletion in my editor. > Alternatively, a name like "to_base" that clarifies the intent and is less > likely to clash with variable names might be an improvement. Builtins aren't reserved names, so the clash is minimal. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From gmccaughan at synaptics-uk.com Thu Jan 19 13:20:52 2006 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Thu, 19 Jan 2006 12:20:52 +0000 Subject: [Python-Dev] str with base In-Reply-To: <20060119111550.GX18916@xs4all.nl> References: <20060118152356.GA20772@panix.com> <200601191023.30902.gmccaughan@synaptics-uk.com> <20060119111550.GX18916@xs4all.nl> Message-ID: <200601191220.53551.gmccaughan@synaptics-uk.com> On Thursday 2006-01-19 11:15, Thomas Wouters wrote: > On Thu, Jan 19, 2006 at 10:23:30AM +0000, Gareth McCaughan wrote: ... > > Introducing a new builtin with a name that's a common, short > > English word is a bit disagreeable. > > While I don't particularly mind the new function in either the builtin > module or another, like math, I don't understand the problem with the name. > Most builtin names are short and english-ish words. I like that, I'm glad > they are that way, and the two names I dislike most are 'isinstance' and > 'issubclass'. "issubclass" is horrible because wordsjammedtogetherlikethisarehardtoread, especially when they're misleading as to pronunciation ("iss-ubclass"). Short English words are nice because they're easy to type and (at least sometimes) their meanings are immediately obvious. For the same reason, they're useful as variable names. Of course the world doesn't end if a builtin name is the same as a variable name you'd like to use, but it's ... well, "a bit disagreeable" probably expresses about the right degree of nuisance. > > The other thing about the name "base" is that it's not entirely obvious > > which way it converts: do you say > > > > base(123,5) > > > > to get a string representing 123 in base 5, or > > > > base("123",5) > > > > to get the integer whose base 5 representation is "123"? > > This is an argument for 'str(123, 5)', but I don't agree. It's not (intended as) an argument *for* any particular form. > Not _everything_ > has to be obvious at first glance. The very same could be said about hex(), > oct(), dir(), even names like list() (what does it list?), str() (stripping > something?), etc. Actually, I happen to dislike hex() slightly -- I never use or see oct(), so don't much care about that -- for exactly that reason. > Having int() do it one way and base() the other makes fine > sense to me, and I don't see it as any harder to explain than, say, why > hex("123") doesn't return 291. I've personally never had to explain > hex/oct's behaviour. To me, base() is less obvious than hex(), which itself is just ambiguous enough to cost me maybe one second per month. Not a big deal at all, but not zero. > While I think 'str' would be a slightly better name than 'base' (despite the > specialcasing of numbers,) I don't mind either. I do mind names like > 'to_base' or 'as_str' or 'shouldbestr' or other names that make me turn on > autocompletion in my editor. You need to turn off the Python Mind Control feature. :-) I think math.base (apart from sounding like it ought to be a variable that controls the base in which numbers are represented, or something of the sort) is about as much typing as to_base, so I'm not sure how the latter can be much worse in this respect. > > Alternatively, a name like "to_base" that clarifies the intent and is less > > likely to clash with variable names might be an improvement. > > Builtins aren't reserved names, so the clash is minimal. Sure, it's not disabling. But in practice it's nice to be able to avoid builtin names, and "base" is a word I'd rather not have to take measures to avoid: too many meanings, some of them quite common. (I don't care much about this, and if base() gets introduced I shan't complain.) -- g From guido at python.org Thu Jan 19 18:31:30 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 19 Jan 2006 09:31:30 -0800 Subject: [Python-Dev] str with base In-Reply-To: <003801c61cc9$a0f73e80$50b02c81@oemcomputer> References: <003801c61cc9$a0f73e80$50b02c81@oemcomputer> Message-ID: On 1/18/06, Raymond Hettinger wrote: > Guido, we may be converging on a consensus for my proposal: > > base(value, radix=2) > > So far no one has shot at it, and it has gathered +1's from Steven, > Alex, Brett, and Nick. I think we ought to let this sit for a while and come back to it in a few week's time. Is 'base' really the right name? It could just as well be considered a conversion in the other direction. In common usage, 'base' and 'radix' are about synonymous (except no-one uses radix). Pethaps the 2nd argument should not be a keyword argument? Also, this discussion made me wonder if conversions using other bases than 10 should even be built-ins. A library module seems a more appropriate place. The prevalence here of people who actually use hex numbers on a regular basis is probably easily explained by a combination of old-timers, language implementers, and super-geeks; hardly the typical Python user. The desire of (bright) beginners to do any kind of non-decimal conversion probably stems from a misguided meme (dating back to the invention of computers) that in order to learn about computers you ought to begin by learning about Boolean algebra and binary numbers. That might be true long ago, but today, binary, octal and hexadecimal numbers are mostly a curiosity used in obscure low-level APIs like ioctl(). > To keep it simple, the proposal is for the value to be any int or long. > With an underlying __base__ method call, it wouldn't be hard for someone > to build it out to support other numeric types if the need arises. Let's not. What would 3.14 be expressed in base 3? > The output would have no prefixes. As Alex pointed out, it is easier > and more direct to add those after the fact if needed. Agreed. > Care to pronounce on it? Rather not yet. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy at alum.mit.edu Thu Jan 19 19:24:00 2006 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Thu, 19 Jan 2006 10:24:00 -0800 Subject: [Python-Dev] str with base In-Reply-To: References: <003801c61cc9$a0f73e80$50b02c81@oemcomputer> Message-ID: I'm not sure I believe this should be a builtin. I think the threshold for new builtins ought to be nearly as high as the threshold for new keywords. Or the proposer ought to make an argument about what the function should not go in a module. Jeremy On 1/19/06, Guido van Rossum wrote: > On 1/18/06, Raymond Hettinger wrote: > > Guido, we may be converging on a consensus for my proposal: > > > > base(value, radix=2) > > > > So far no one has shot at it, and it has gathered +1's from Steven, > > Alex, Brett, and Nick. > > I think we ought to let this sit for a while and come back to it in a > few week's time. Is 'base' really the right name? It could just as > well be considered a conversion in the other direction. In common > usage, 'base' and 'radix' are about synonymous (except no-one uses > radix). Pethaps the 2nd argument should not be a keyword argument? > > Also, this discussion made me wonder if conversions using other bases > than 10 should even be built-ins. A library module seems a more > appropriate place. The prevalence here of people who actually use hex > numbers on a regular basis is probably easily explained by a > combination of old-timers, language implementers, and super-geeks; > hardly the typical Python user. The desire of (bright) beginners to do > any kind of non-decimal conversion probably stems from a misguided > meme (dating back to the invention of computers) that in order to > learn about computers you ought to begin by learning about Boolean > algebra and binary numbers. That might be true long ago, but today, > binary, octal and hexadecimal numbers are mostly a curiosity used in > obscure low-level APIs like ioctl(). > > > To keep it simple, the proposal is for the value to be any int or long. > > With an underlying __base__ method call, it wouldn't be hard for someone > > to build it out to support other numeric types if the need arises. > > Let's not. What would 3.14 be expressed in base 3? > > > The output would have no prefixes. As Alex pointed out, it is easier > > and more direct to add those after the fact if needed. > > Agreed. > > > Care to pronounce on it? > > Rather not yet. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu > From shane.holloway at ieee.org Thu Jan 19 19:28:23 2006 From: shane.holloway at ieee.org (Shane Holloway (IEEE)) Date: Thu, 19 Jan 2006 11:28:23 -0700 Subject: [Python-Dev] str with base In-Reply-To: References: <003801c61cc9$a0f73e80$50b02c81@oemcomputer> Message-ID: On Jan 19, 2006, at 10:31, Guido van Rossum wrote: >> To keep it simple, the proposal is for the value to be any int or >> long. >> With an underlying __base__ method call, it wouldn't be hard for >> someone >> to build it out to support other numeric types if the need arises. > > Let's not. What would 3.14 be expressed in base 3? 10.010210001 It turned out to be a fun aside, and I've attached my quick and dirty script as a strawman. For what it's worth, I don't like the name base() because it sounds like something I would call on a class to get it's bases. Perhaps nbase? And maybe fbase for the floating point one... Thanks, -Shane Holloway -------------- next part -------------- A non-text attachment was scrubbed... Name: bases.py Type: text/x-python-script Size: 1385 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20060119/a9b6a3d8/attachment.bin From collinw at gmail.com Thu Jan 19 19:58:00 2006 From: collinw at gmail.com (Collin Winter) Date: Thu, 19 Jan 2006 19:58:00 +0100 Subject: [Python-Dev] Additions to the functional module Message-ID: <43aa6ff70601191058y27b1e50ard658f1a103d52b67@mail.gmail.com> Hello, I just submitted patch #1410119, adding foldl and foldr functions to Modules/functionalmodule.c (plus docs and tests in the appropriate places). If there's interest from the higher-ups, I'm interested in adding numerous other functional programming tools to this module. In addition to C implementations, I would also provide documentation and further additions to test/test_functional.py to cover the new functions. Also, I've started a project to offer a pure-python implementation of the C-language functional module, which I plan to keep in sync with the module in the standard library. If there's any interest in including something like this in Demo/, I'd be keen to contribute it as well. Thanks, Collin Winter From fredrik at pythonware.com Thu Jan 19 19:27:14 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 19 Jan 2006 19:27:14 +0100 Subject: [Python-Dev] str with base References: <003801c61cc9$a0f73e80$50b02c81@oemcomputer> Message-ID: Guido van Rossum wrote: > I think we ought to let this sit for a while and come back to it in a > few week's time. Is 'base' really the right name? It could just as > well be considered a conversion in the other direction. the same applies to hex and oct, of course. as for base itself, I'm more concerned about the google product place- ment here. what's next? a froogle builtin? From guido at python.org Thu Jan 19 20:12:38 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 19 Jan 2006 11:12:38 -0800 Subject: [Python-Dev] str with base In-Reply-To: References: <003801c61cc9$a0f73e80$50b02c81@oemcomputer> Message-ID: On 1/19/06, Fredrik Lundh wrote: > Guido van Rossum wrote: > > > I think we ought to let this sit for a while and come back to it in a > > few week's time. Is 'base' really the right name? It could just as > > well be considered a conversion in the other direction. > > the same applies to hex and oct, of course. Right. And this is not a hypothetical issue either -- in Perl, hex and oct *do* work the other way I believe. More reasons to get rid of these in Python 3000. Perhaps we should also get rid of hex/oct lterals? > as for base itself, I'm more concerned about the google product place- > ment here. what's next? a froogle builtin? The default __import__ will use Google Code to locate an appropriate module to import instead of restricting itself to the boring and predictable sys.path. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz at pythoncraft.com Thu Jan 19 20:13:13 2006 From: aahz at pythoncraft.com (Aahz) Date: Thu, 19 Jan 2006 11:13:13 -0800 Subject: [Python-Dev] str with base In-Reply-To: References: <003801c61cc9$a0f73e80$50b02c81@oemcomputer> Message-ID: <20060119191313.GA25544@panix.com> On Thu, Jan 19, 2006, Jeremy Hylton wrote: > > I'm not sure I believe this should be a builtin. I think the > threshold for new builtins ought to be nearly as high as the threshold > for new keywords. Or the proposer ought to make an argument about > what the function should not go in a module. The way I'd put it, any function that wants to go in builtins should require a formal PEP. And in case it isn't clear, I'm +1 on deprecating oct()/hex() (or moving them into another module as convenience functions for base() -- just to make conversion easier). -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From martin at v.loewis.de Thu Jan 19 20:14:05 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 19 Jan 2006 20:14:05 +0100 Subject: [Python-Dev] New PEP: Using ssize_t as the index type In-Reply-To: References: <43B3ECEE.9060300@v.loewis.de> <20051229173041.GA19575@code1.codespeak.net> <43B50B64.20904@v.loewis.de> <20060105194538.GB11736@code1.codespeak.net> <43BD8464.4060801@v.loewis.de> <43C37C02.8030005@egenix.com> Message-ID: <43CFE4FD.7050108@v.loewis.de> Neal Norwitz wrote: > Other ideas? We could rename essential function symbols, like PyInitModule4 (to, say, PyInitModule4b). Regards, Martin From Scott.Daniels at Acm.Org Thu Jan 19 20:59:52 2006 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Thu, 19 Jan 2006 11:59:52 -0800 Subject: [Python-Dev] str with base In-Reply-To: <003b01c61b22$0bbfe040$e221cb97@oemcomputer> References: <003b01c61b22$0bbfe040$e221cb97@oemcomputer> Message-ID: Raymond Hettinger wrote: > That suggests that it would be better to simply add an int method: > x.convert_to_base(7) I'd suggest allowing: x.convert_to_base('0123456') Where the (str or unicode) argument is the list of digits in order. This would allow easy converting to base-64 and other weird formats, as well as providing decimal conversion into some unicode number ranges outside the ASCII group. --Scott David Daniels Scott.Daniels at Acm.Org From jeremy at alum.mit.edu Thu Jan 19 21:13:33 2006 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Thu, 19 Jan 2006 12:13:33 -0800 Subject: [Python-Dev] str with base In-Reply-To: <20060119191313.GA25544@panix.com> References: <003801c61cc9$a0f73e80$50b02c81@oemcomputer> <20060119191313.GA25544@panix.com> Message-ID: On 1/19/06, Aahz wrote: > On Thu, Jan 19, 2006, Jeremy Hylton wrote: > > > > I'm not sure I believe this should be a builtin. I think the > > threshold for new builtins ought to be nearly as high as the threshold > > for new keywords. Or the proposer ought to make an argument about > > what the function should not go in a module. > > The way I'd put it, any function that wants to go in builtins should > require a formal PEP. I'm suggesting a criterion for evaluating the choice of builtin vs. module, not making a suggestion about the process. Jeremy From bob at redivi.com Thu Jan 19 21:44:09 2006 From: bob at redivi.com (Bob Ippolito) Date: Thu, 19 Jan 2006 12:44:09 -0800 Subject: [Python-Dev] str with base In-Reply-To: References: <003801c61cc9$a0f73e80$50b02c81@oemcomputer> Message-ID: <6E62A903-3CFF-4CCA-8459-F0A966E2FF73@redivi.com> On Jan 19, 2006, at 11:12 AM, Guido van Rossum wrote: > On 1/19/06, Fredrik Lundh wrote: >> Guido van Rossum wrote: >> >>> I think we ought to let this sit for a while and come back to it >>> in a >>> few week's time. Is 'base' really the right name? It could just as >>> well be considered a conversion in the other direction. >> >> the same applies to hex and oct, of course. > > Right. And this is not a hypothetical issue either -- in Perl, hex and > oct *do* work the other way I believe. More reasons to get rid of > these in Python 3000. Perhaps we should also get rid of hex/oct > lterals? In Perl, hex(n) is like int(n, 16) and oct(n) is like int(n, 8) -- but they "try very hard" to make sense out of the given scalar (e.g. more like int(n, 0) with a suggestion for base). $ perl -e 'print hex("12") . " " . oct("12") . " " . oct("0x12") . " " . hex("fubar")' 18 10 18 15 If you notice, oct("0x12") gives the hexadecimal result, and the functions will try and make a value even out of garbage data. In Ruby, you have the optional radix argument to_s and to_i, where to_i will just return 0 for invalid values. They will take any radix from 2..36. $ irb irb(main):001:0> 12.to_s => "12" irb(main):002:0> 12.to_s(16) => "c" irb(main):003:0> 12.to_s(8) => "14" irb(main):004:0> "12".to_i(8) => 10 irb(main):005:0> "12".to_i(16) => 18 irb(main):006:0> "0x12".to_i(16) => 18 irb(main):007:0> "0x12".to_i(8) => 0 irb(main):008:0> "0x12".to_i => 0 irb(main):009:0> "fubar".to_i => 0 irb(main):010:0> "fubar".to_i(36) => 26608563 -bob From ncoghlan at gmail.com Thu Jan 19 21:56:23 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 20 Jan 2006 06:56:23 +1000 Subject: [Python-Dev] str with base In-Reply-To: References: <003801c61cc9$a0f73e80$50b02c81@oemcomputer> Message-ID: <43CFFCF7.1070302@gmail.com> Guido van Rossum wrote: > On 1/19/06, Fredrik Lundh wrote: >> Guido van Rossum wrote: >> >>> I think we ought to let this sit for a while and come back to it in a >>> few week's time. Is 'base' really the right name? It could just as >>> well be considered a conversion in the other direction. >> the same applies to hex and oct, of course. > > Right. And this is not a hypothetical issue either -- in Perl, hex and > oct *do* work the other way I believe. More reasons to get rid of > these in Python 3000. Perhaps we should also get rid of hex/oct > lterals? I'm not aware of anyone that would miss octal literals, but there are plenty of hardware weenies like me that would find "int("DEAD", 16)" less convenient than "0xDEAD". Python is a bit too heavyweight for a lot of embedded work, but its *great* for writing host-based test harnesses. I quite like the suggestion of using 'math.base' rather than a builtin, but there are still issues to be figured out there: - the math module is currently a thin wrapper around C's "math.h". Do we really want to change that by adding more methods? - is 'base' the right name? - should we allow a "digits" argument, or just the radix argument? Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From steve at holdenweb.com Thu Jan 19 22:18:08 2006 From: steve at holdenweb.com (Steve Holden) Date: Thu, 19 Jan 2006 21:18:08 +0000 Subject: [Python-Dev] str with base In-Reply-To: <43CFFCF7.1070302@gmail.com> References: <003801c61cc9$a0f73e80$50b02c81@oemcomputer> <43CFFCF7.1070302@gmail.com> Message-ID: Nick Coghlan wrote: > Guido van Rossum wrote: > >>On 1/19/06, Fredrik Lundh wrote: >> >>>Guido van Rossum wrote: >>> >>> >>>>I think we ought to let this sit for a while and come back to it in a >>>>few week's time. Is 'base' really the right name? It could just as >>>>well be considered a conversion in the other direction. >>> >>>the same applies to hex and oct, of course. >> >>Right. And this is not a hypothetical issue either -- in Perl, hex and >>oct *do* work the other way I believe. More reasons to get rid of >>these in Python 3000. Perhaps we should also get rid of hex/oct >>lterals? > > > I'm not aware of anyone that would miss octal literals, but there are plenty > of hardware weenies like me that would find "int("DEAD", 16)" less convenient > than "0xDEAD". Python is a bit too heavyweight for a lot of embedded work, but > its *great* for writing host-based test harnesses. > > I quite like the suggestion of using 'math.base' rather than a builtin, but > there are still issues to be figured out there: > - the math module is currently a thin wrapper around C's "math.h". Do we > really want to change that by adding more methods? > - is 'base' the right name? > - should we allow a "digits" argument, or just the radix argument? > Another possibility, since Python 3 can break backward compatibility: we could take a page out of Icon's book and use an "rN" suffix for non-decimal literals. 23 == 27r8 == 17r16 regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/ From theller at python.net Thu Jan 19 22:24:08 2006 From: theller at python.net (Thomas Heller) Date: Thu, 19 Jan 2006 22:24:08 +0100 Subject: [Python-Dev] Include ctypes into core Python? References: <43C58200.20803@v.loewis.de> <43CC17FB.5020904@v.loewis.de> Message-ID: "Martin v. L?wis" writes: > Guido van Rossum wrote: >> On 1/16/06, Thomas Heller wrote: >> >>>It looks like we need a pronouncement now. >> >> >> Sorry. It appeared to me that there was general agreement to using a >> strongly worded warning in the docs, so I tuned out of the rest of the >> discussion. So for the record, I'm fine with that. > ctypes includes libffi. libffi is build with GNU tools, which would have to be committed in the python svn repository. Several of these files are licensed under GPL: aclocal.m4 config-ml.in config.guess config.sub depcomp ltcf-c.sh ltconfig missing AFAICT, these files are only used to *build* libffi, and are not linked into or have to be distributed with the binaries. Is it a problem to have these files in the Python source code? Maybe it would be possible to reimplement the functionality in the setup.py script, but this is way beyond my expertise. Thanks, Thomas From martin at v.loewis.de Thu Jan 19 23:11:52 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 19 Jan 2006 23:11:52 +0100 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: References: <43C58200.20803@v.loewis.de> <43CC17FB.5020904@v.loewis.de> Message-ID: <43D00EA8.2070904@v.loewis.de> Thomas Heller wrote: > Is it a problem to have these files in the Python source code? I would think so, yes. Including them in the Python distribution, without licensing the entire Python distribution under GPL, would be a GPL violation. The actual libffi appears to have a fairly liberal license (also it still requires to include the LICENSE file, atleast in the source distribution). Importing the original source in /external would be no problem (AFAICT), as long as you then delete them during/after the svn cp. > Maybe it would be possible to reimplement the functionality in the > setup.py script, but this is way beyond my expertise. Right. Given the large number of people that want to see this happen (not necessarily on python-dev, but on comp.lang.python), I'm sure there is somebody who can come up with a build process. If nobody volunteers to do the actual work, I have no problems with dropping the idea for now. I wouldn't have any problems with just including a Windows port, either. Regards, Martin From foom at fuhm.net Thu Jan 19 23:31:25 2006 From: foom at fuhm.net (James Y Knight) Date: Thu, 19 Jan 2006 17:31:25 -0500 Subject: [Python-Dev] Include ctypes into core Python? In-Reply-To: References: <43C58200.20803@v.loewis.de> <43CC17FB.5020904@v.loewis.de> Message-ID: <03C7403A-6B16-4727-B2C6-5FE133E69734@fuhm.net> On Jan 19, 2006, at 4:24 PM, Thomas Heller wrote: > > Several of these files are licensed under GPL: > > aclocal.m4 config-ml.in config.guess config.sub depcomp ltcf-c.sh > ltconfig missing > Are you sure? The copies of aclocal.m4 and config-ml.in both disagree with you. aclocal seems to have a completely liberal license, and config-ml has a "whatever the license of the program it's building" license. James aclocal.m4:: # generated automatically by aclocal 1.7.3 -*- Autoconf -*- # Copyright (C) 1996, 1997, 1998, 1999, 2000, 2001, 2002 # Free Software Foundation, Inc. # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY, to the extent permitted by law; without # even the implied warranty of MERCHANTABILITY or FITNESS FOR A # PARTICULAR PURPOSE. # libtool.m4 - Configure libtool for the host system. -*-Autoconf-*- config-ml.in:: # Configure fragment invoked in the post-target section for subdirs # wanting multilib support. # # Copyright (C) 1995, 1997, 1998, 1999, 2000, 2001, 2002, 2003 # Free Software Foundation, Inc. # # This file is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place - Suite 330, # Boston, MA 02111-1307, USA. # # As a special exception to the GNU General Public License, if you # distribute this file as part of a program that contains a # configuration script generated by Autoconf, you may include it under # the same distribution terms that you use for the rest of that program. From tim.peters at gmail.com Fri Jan 20 01:07:19 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 19 Jan 2006 19:07:19 -0500 Subject: [Python-Dev] str with base In-Reply-To: <43CFFCF7.1070302@gmail.com> References: <003801c61cc9$a0f73e80$50b02c81@oemcomputer> <43CFFCF7.1070302@gmail.com> Message-ID: <1f7befae0601191607i36bee147i34e96cd764d12a0e@mail.gmail.com> [Nick Coghlan] > ... > I quite like the suggestion of using 'math.base' rather than a builtin, but > there are still issues to be figured out there: > - the math module is currently a thin wrapper around C's "math.h". Do we > really want to change that by adding more methods? That's not an issue. Some math functions go beyond C's (like 2-argument log(), and all flavors of log() returning sensible results for inputs larger than the largest C double), and some functions aren't part of C at all (like math.radians()). A stronger reason to keep it out of `math` is that all functions there operate on, and return, floats: it's a bizarre place to put an integer->string function. > - is 'base' the right name? > - should we allow a "digits" argument, or just the radix argument? Add to_base(self, radix=16) as a new numeric method. End of problem ;-) From thomas at xs4all.net Fri Jan 20 01:17:14 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Fri, 20 Jan 2006 01:17:14 +0100 Subject: [Python-Dev] str with base In-Reply-To: <43CFFCF7.1070302@gmail.com> References: <003801c61cc9$a0f73e80$50b02c81@oemcomputer> <43CFFCF7.1070302@gmail.com> Message-ID: <20060120001714.GY18916@xs4all.nl> On Fri, Jan 20, 2006 at 06:56:23AM +1000, Nick Coghlan wrote: > I'm not aware of anyone that would miss octal literals, Except anyone who uses os.chmod. I would be mighty sad if we removed octal and hexadecimal literals for 'cleanliness' reasons alone. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From tim.peters at gmail.com Fri Jan 20 02:07:43 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 19 Jan 2006 20:07:43 -0500 Subject: [Python-Dev] pystate.c changes for Python 2.4.2 In-Reply-To: <43CBC95F.3020907@core-sdi.com> References: <43C68823.3000105@corest.com> <2mwth4qw3q.fsf@starship.python.net> <43C7BB1F.3000904@corest.com> <43CBC95F.3020907@core-sdi.com> Message-ID: <1f7befae0601191707q57552dc8lb75c90aa30e2924a@mail.gmail.com> [Gabriel Becedillas] > Can anybody tell me if the patch I suggested is ok ? > That will be to add the following code at the end of PyThreadState_Delete: > > if (autoTLSkey && PyThread_get_key_value(autoTLSkey) == tstate) > PyThread_delete_key_value(autoTLSkey); It needs a little work, but I think it should be fine, and people tend to listen to me on issues like this ;-) The reason it can't work as-is is shallow: autoTLSkey doesn't exist unless Python is compiled with WITH_THREAD defined. So you need to wrap that inside an "#ifdef WITH_THREAD" block; or add it similarly to tstate_delete_common(), and remove it from PyThreadState_DeleteCurrent(). The primary reason people find this so confusing is because it is :-). In the dim mists of history, neither multiple interpreters nor PyThreadState_DeleteCurrent() existed. While multiple interpreters do exist now, they're poorly understood, and the PEP that introduced the GIL state API explicitly disavowed any responsibility for the new API working at all with multiple interpreters: http://www.python.org/peps/pep-0311.html This proposal identifies a solution for extension authors with complex multi-threaded requirements, but that only require a single "PyInterpreterState". There is no attempt to cater for extensions that require multiple interpreter states. At the time of writing, no extension has been identified that requires multiple PyInterpreterStates, and indeed it is not clear if that facility works correctly in Python itself. In a parallel development, thread races during Python shutdown were discovered that could lead to segfaults, and PyThreadState_DeleteCurrent() was introduced to repair those. As a result, it so happens that core Python never uses the original PyThreadState_Delete() anymore, except when Py_NewInterpreter() has to throw away the brand new thread state it created because it turns out it can't create a new interpreter. Since core Python never calls Py_NewInterpreter() or PyThreadState_Delete(), you're in an intersection of areas no current Python developer even thinks about, let alone tests. But by the same token, _because_ it's unused, there's nothing you can do to PyThreadState_Delete() that can hurt core Python either. That's why people should be more encouraging here ;-) It certainly makes sense to me that if a thread state is going away, any record of it in the auto-GIL-state machinery must also go away. > Thank you very much. Thank you! If you put a patch on SourceForge, I'll probably be happy to check it in. From bob at redivi.com Fri Jan 20 03:11:19 2006 From: bob at redivi.com (Bob Ippolito) Date: Thu, 19 Jan 2006 18:11:19 -0800 Subject: [Python-Dev] str with base In-Reply-To: <20060120001714.GY18916@xs4all.nl> References: <003801c61cc9$a0f73e80$50b02c81@oemcomputer> <43CFFCF7.1070302@gmail.com> <20060120001714.GY18916@xs4all.nl> Message-ID: On Jan 19, 2006, at 4:17 PM, Thomas Wouters wrote: > On Fri, Jan 20, 2006 at 06:56:23AM +1000, Nick Coghlan wrote: > >> I'm not aware of anyone that would miss octal literals, > > Except anyone who uses os.chmod. I would be mighty sad if we > removed octal > and hexadecimal literals for 'cleanliness' reasons alone. I have a LOT of code that has hex literals, and a little code with oct literals (as you say, just os.chmod). I'm -1 on removing hex and oct, and +0 on adding binary. As a point of reference, both Perl and Ruby support 0b110 binary literal syntax $ ruby -e 'print 0b110, "\n"' 6 $ perl -e 'print 0b110 . "\n"' 6 -bob From jason.orendorff at gmail.com Fri Jan 20 04:41:23 2006 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Thu, 19 Jan 2006 22:41:23 -0500 Subject: [Python-Dev] PEP 343 and __context__() Message-ID: I just noticed that my name is in PEP 343 attached to the idea of the __context__() method, and I'm slightly queasy over it. The rationale was to help e.g. decimal.DecimalContext support 'with'. Maybe that's a bad idea. DecimalContext has a few problems. In code where it matters, every function you write has to worry about it. (That is, you can't just write __decimal_context__ = ... at the top of the file and be done with it, the way you can with, say, __metaclass__.) And DecimalContext doesn't fit in with generators. sys.stdout has similar problems. It feels like PEP 343 will make people want to follow this model. That is, we'll see more global behavior-controlling variables in the future. There are grizzlier fates; I just wondered if anyone had thought of this. Cheers, -j From barry at python.org Fri Jan 20 05:29:11 2006 From: barry at python.org (Barry Warsaw) Date: Thu, 19 Jan 2006 23:29:11 -0500 Subject: [Python-Dev] str with base In-Reply-To: <43CFFCF7.1070302@gmail.com> References: <003801c61cc9$a0f73e80$50b02c81@oemcomputer> <43CFFCF7.1070302@gmail.com> Message-ID: <1137731351.8802.9.camel@geddy.wooz.org> On Fri, 2006-01-20 at 06:56 +1000, Nick Coghlan wrote: > I'm not aware of anyone that would miss octal literals, but there are plenty > of hardware weenies like me that would find "int("DEAD", 16)" less convenient > than "0xDEAD". Although octal literals is handy for things like os.chmod(). Unix weenies shouldn't be totally forgotten in P3K. I'm also for keeping hex() and oct() although if you want to move them out of builtins, that's fine. +0b1 for binary literals and %b. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060119/2063725b/attachment.pgp From ncoghlan at gmail.com Fri Jan 20 10:21:43 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 20 Jan 2006 19:21:43 +1000 Subject: [Python-Dev] PEP 343 and __context__() In-Reply-To: References: Message-ID: <43D0ABA7.4080203@gmail.com> Jason Orendorff wrote: > I just noticed that my name is in PEP 343 attached to the idea of the > __context__() method, and I'm slightly queasy over it. > > The rationale was to help e.g. decimal.DecimalContext support 'with'. > Maybe that's a bad idea. > > DecimalContext has a few problems. In code where it matters, every > function you write has to worry about it. (That is, you can't just > write __decimal_context__ = ... at the top of the file and be done > with it, the way you can with, say, __metaclass__.) No, you write "decimal.setcontext(...)" instead. You then only need to worry if you have multiple threads, and you can even fix that by modifying decimal.DefaultContext while the program is still single threaded. The latter approach actually works even for a single-threaded program, but should only be done in an application script, never in an imported module. > And > DecimalContext doesn't fit in with generators. It does fit actually - you simply have to remember to restore the original context around any invocations of yield. > sys.stdout has similar problems. > > It feels like PEP 343 will make people want to follow this model. > That is, we'll see more global behavior-controlling variables in the > future. There are grizzlier fates; I just wondered if anyone had > thought of this. Yeah, it came up in response to PJE's suggestion of task-local variables for generators. The basic concept we came up with is that if you're writing a generator that uses a specific context, remember to save the original and restore it around any calls to yield (at least, I came up with the idea and no-one objected to the approach [1]). In the case of decimal.Context: def iter_sin(iterable): orig_ctx = decimal.getcontext() with orig_ctx as ctx: ctx.prec += 10 for r in iterable: y = sin(r) # Very high precision during calculation with orig_ctx: yield +y # Yielded results have normal precision # We get "ctx" back here # We get "orig_ctx" back here Similarly for sys.stdout (only not thread-safe due to the global state): def log_yielded_items(iterable, logfile): orig_stdout = sys.stdout with redirected_stdout(logfile): for r in iterable: print "Yielding:", r with redirected_stdout(orig_stdout): yield r # stdout goes back to normal here # stdout is going to the logfile again here # And stdout is back to normal at the end The key properties of a "well-behaved" context are: 1. Use thread-local state for contexts in order to be multithreading safe decimal.Context obeys this, sys.stdout doesn't 2. Provide a way to retrieve the current state for explicit restoration decimal.getcontext() and sys.stdout allow this as shown above This kind of thing is always going to be a pain, but PEP 343 makes it significantly less so. Cheers, Nick. [1] http://mail.python.org/pipermail/python-dev/2005-October/057493.html -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From mwh at python.net Fri Jan 20 10:48:00 2006 From: mwh at python.net (Michael Hudson) Date: Fri, 20 Jan 2006 09:48:00 +0000 Subject: [Python-Dev] pystate.c changes for Python 2.4.2 In-Reply-To: <1f7befae0601191707q57552dc8lb75c90aa30e2924a@mail.gmail.com> (Tim Peters's message of "Thu, 19 Jan 2006 20:07:43 -0500") References: <43C68823.3000105@corest.com> <2mwth4qw3q.fsf@starship.python.net> <43C7BB1F.3000904@corest.com> <43CBC95F.3020907@core-sdi.com> <1f7befae0601191707q57552dc8lb75c90aa30e2924a@mail.gmail.com> Message-ID: <2mirsfnsbz.fsf@starship.python.net> Tim Peters writes: > [Gabriel Becedillas] >> Can anybody tell me if the patch I suggested is ok ? >> That will be to add the following code at the end of PyThreadState_Delete: >> >> if (autoTLSkey && PyThread_get_key_value(autoTLSkey) == tstate) >> PyThread_delete_key_value(autoTLSkey); > > It needs a little work, but I think it should be fine, and people tend > to listen to me on issues like this ;-) Oh good, I've been meaning to scrape together the brain cells to think about this but PyPy is pulping them as fast as I find them... > The reason it can't work as-is is shallow: autoTLSkey doesn't exist > unless Python is compiled with WITH_THREAD defined. So you need to > wrap that inside an "#ifdef WITH_THREAD" block; or add it similarly to > tstate_delete_common(), and remove it from > PyThreadState_DeleteCurrent(). > > The primary reason people find this so confusing is because it is :-). > In the dim mists of history, neither multiple interpreters nor > PyThreadState_DeleteCurrent() existed. While multiple interpreters do > exist now, they're poorly understood, and the PEP that introduced the > GIL state API explicitly disavowed any responsibility for the new API > working at all with multiple interpreters: > > http://www.python.org/peps/pep-0311.html > > This proposal identifies a solution for extension authors with > complex multi-threaded requirements, but that only require a > single "PyInterpreterState". There is no attempt to cater for > extensions that require multiple interpreter states. At the time > of writing, no extension has been identified that requires > multiple PyInterpreterStates, and indeed it is not clear if that > facility works correctly in Python itself. > > In a parallel development, thread races during Python shutdown were > discovered that could lead to segfaults, and > PyThreadState_DeleteCurrent() was introduced to repair those. As a > result, it so happens that core Python never uses the original > PyThreadState_Delete() anymore, except when Py_NewInterpreter() has to > throw away the brand new thread state it created because it turns out > it can't create a new interpreter. Um, PyThreadState_Delete() is called from zapthreads() is called from PyInterpreterState_Delete() is called from Py_Finalize()... so I don't entirely believe you here :) > Since core Python never calls Py_NewInterpreter() or > PyThreadState_Delete(), you're in an intersection of areas no current > Python developer even thinks about, let alone tests. But by the same > token, _because_ it's unused, there's nothing you can do to > PyThreadState_Delete() that can hurt core Python either. That's why > people should be more encouraging here ;-) It certainly makes sense > to me that if a thread state is going away, any record of it in the > auto-GIL-state machinery must also go away. I guess if the patch fixes the original problem, it should go in -- my uneasiness stems not from worrying that it might do harm, but rather from the fact that I think it's incomplete. Maybe I just haven't thought it through enough. Also, every time I look at pystate.c, I think of things that could be done differently or better -- for example, I think it would be sane and possibly even sensible to only call PyThread_set_key_value() in _PyGILState_NoteThreadState() if "tstate->interp == autoInterpreterState". >> Thank you very much. > > Thank you! If you put a patch on SourceForge, I'll probably be happy > to check it in. That would get me off the "svn blame" hook :) Cheers, mwh -- My first thought was someone should offer Mr. Bush elocution lessons. But he'd probably say he knew how to work them chairs already. -- Internet Oracularity #1294-01 From thomas.mangin at exa-networks.co.uk Tue Jan 17 16:07:27 2006 From: thomas.mangin at exa-networks.co.uk (Thomas Mangin) Date: Tue, 17 Jan 2006 15:07:27 +0000 Subject: [Python-Dev] site triggering a bug in urllib2 Message-ID: <43CD082F.1090606@exa-networks.co.uk> Hello, I am contacting the list in the hope that someone will be able to understand what I am seeing. I have hit a bug with python 2.4.2 (on Mandriva 2006) using urllib2. The code which trigger the bug is as follow.. import urllib2 req = urllib2.Request("http://66.117.37.13/") # makes no difference .. req.add_header('Connection', 'close') handle = urllib2.urlopen(req) data = handle.read() print data using a timeout on the socket does not work neither. The page is well sent and received (verified with tcpdump) but the read call is never returning. strace shows lots of activity on the socket: select constantly returning a file descriptor with nothing to read on. I would like to avoid to start to use timers to kill the connection manually, any advise on how to deal with this would be welcome. Thank you for your time and help. Thomas -- Exa Networks Limited - UK - AS30740 - www.exa-networks.co.uk nic-handle : MANG-RIPE website : thomas.mangin.me.uk GPG key ID : 0xFB8B81A1 PGP key : /pgp.html Inoc-DBA # : 30740*TOM Office # : +44 (0) 845 145 1234 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: OpenPGP digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20060117/d513da05/attachment-0001.pgp From connellybarnes at yahoo.com Tue Jan 17 23:08:36 2006 From: connellybarnes at yahoo.com (Connelly Barnes) Date: Tue, 17 Jan 2006 14:08:36 -0800 (PST) Subject: [Python-Dev] timeit module Message-ID: <20060117220836.52102.qmail@web54303.mail.yahoo.com> Connelly Barnes wrote: >> Hi, >> >> Perhaps I am the only one bothered by the timeit >> module, but it seems poorly designed to me. >> >> First of all, it should use a geometric series with a >> timeout value to detect how many iterations it should >> perform. Currently, the user is required to manually >> specify the number of iterations (the default is 1 > >The provision of a default value generally seems to chop the legs from >your argument that the number of iterations is required. Suppose we wish to time two functions: f1 and f2. Function f1 takes 1e-5 seconds to run, and function f2 takes 0.1 seconds to run. If these functions are defined in interactive mode, we can time them with timeit via: >>> timeit.Timer('f1()', 'from __main__ import f1').timeit(1000000)/1000000 >>> timeit.Timer('f2()', 'from __main__ import f2').timeit(10)/10 The first line gives the approximate time to run f1 and the second line gives the approximate time to run f2. The number parameter to timeit() must be supplied for the second line, since the default 1 million iterations would take too long to be practical. The number parameter to timeit() is also supplied for the first line, since it makes the code more robust in case the default argument value in timeit() is changed. My first observation is that this is a lot of contortion to get the time of functions f1() and f2(). Something simpler would be nice. For example, using Recipe 440657 this can be simplified to: >>> pytime.pytime(f1, ()) >>> pytime.pytime(f2, ()) Here the timing routine calculates the number of iterations to make, and the timing routine divides by the number of iterations of the supplied callable. This is also convenient if you move from a computer to one that is faster or slower. If you move to a computer that is 10 times slower, then the timeit.Timer() code above will take 10 times as long to run. The usual solution is for the programmer to adjust the number of iterations as he changes computers (so he doesn't wait around all day for his benchmarks to run). Likewise, if one optimizes f2() so that it runs 100 times faster, one would again have to adjust the number of iterations in order to gain an accurate timing result (if the total time spent in the timing routine is less than 1 msec, then the resulting time for f2() is off by more than 10% on my Windows machine, which is undesirable, so the number of iterations cannot be made too small). Summary: Module timeit is complicated to use on an existing function, it does not time the number of seconds to execute 1 iteration of the function (this is the value one usually wants in benchmarking, or its reciprocal), and it requires that the end user tweak the number of iterations as he or she changes between computers with different CPU speed, or as he or she optimizes code. It is easy to make a cleaner module that fixes these problems. It might be good to incorporate such a module into Python, due to its general usefulness. >> million). If the user optimizes his or her code, then >> the number of iterations must be changed. If the user >> moves to a slower or faster computer, then the number >> of iterations must be changed again. This is >> annoying. >> >What? The purpose of timeit is to give an approximation to the length of >time to run a specific piece ofcode. > >Why must the number of iterations be changed when moving to a slower or >faster computer? > >> Secondly, there should be a way to time a callable >> directly. That is, without finding the string name of >> the callable and using a string "import X" statement. >> These contortions violate rules #1 and #3 of the Zen >> of Python. >> >Presumably for benchmarking purposes the function call overhead would be >present for all compaered techniques. Do you mean rules #0 and #2? I mean "Beautiful is better than ugly" and "Simple is better than complex." > > [...] >regards > Steve >-- >Steve Holden +44 150 684 7255 +1 800 494 3119 >Holden Web LLC www.holdenweb.com >PyCon TX 2006 www.python.org/pycon/ __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From t-meyer at ihug.co.nz Fri Jan 20 12:19:54 2006 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Sat, 21 Jan 2006 00:19:54 +1300 Subject: [Python-Dev] ConfigParser to save with order In-Reply-To: <56E4F12D-9477-4F58-9D85-3D159B3F1B71@ihug.co.nz> References: <089A37AE-E802-4903-98A5-33BBA0D6E635@ihug.co.nz> <56E4F12D-9477-4F58-9D85-3D159B3F1B71@ihug.co.nz> Message-ID: [Tony Meyer] > Allowing 'surgical' editing of configuration files, as has been > proposed many times both here and c.l.p would not require > ConfigParser to be entirely rewritten (just more extensive > modification of the write() method). After writing the summary of this thread, I figured I might as well prove this by submitting a patch, so did so: [ 1410680 ] Add 'surgical editing' to ConfigParser So there are now at least three open ConfigParser patches, all addressing similar things, although not in the same way, and all three could actually be applied. My patch (which includes new unittests and a marked-up doc patch) runs through a file and updates it to match the in-memory configuration object. Comments, blank lines, and order are preserved. By default (this can be turned off with a keyword arg) new sections/options are added to the file as well (new options in existing sections are added at the end of the last instance of that section, new sections are added at the end of the file). These new options are added in alphabetical (well, list.sort()) order, so ordering is consistent, and if an empty file is 'updated' one ends up with the output of write() but sorted. This is essentially a tidied-up version of the SpamBayes code I wrote a few years back. If anyone is still interested in adding this, feel free to take a look at the tracker :) =Tony.Meyer From stephen at xemacs.org Fri Jan 20 13:07:41 2006 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 20 Jan 2006 21:07:41 +0900 Subject: [Python-Dev] str with base In-Reply-To: <1137731351.8802.9.camel@geddy.wooz.org> (Barry Warsaw's message of "Thu, 19 Jan 2006 23:29:11 -0500") References: <003801c61cc9$a0f73e80$50b02c81@oemcomputer> <43CFFCF7.1070302@gmail.com> <1137731351.8802.9.camel@geddy.wooz.org> Message-ID: <87y81bhzle.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "BAW" == Barry Warsaw writes: BAW> Unix weenies shouldn't be totally forgotten in P3K. Great idea! Put all this stuff in a "weenie" module. You can have weenie.unix and weenie.vms and weenie.unicode, besides the weenie.math that got all this started. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From param at cs.wisc.edu Fri Jan 20 15:23:25 2006 From: param at cs.wisc.edu (Paramjit Oberoi) Date: Fri, 20 Jan 2006 06:23:25 -0800 Subject: [Python-Dev] ConfigParser to save with order In-Reply-To: References: Message-ID: > I think it's moot unless you also preserve comments. Ideally would be > something that prserved everything (ordering, blank lines, comments > etc.) from how it was read in. Modifying a value should keep its > position. Adding a value should add it to the end of the section it's > in (unless there are cases where ordering is important to the > semantics -- are there?). I wrote some code to do just that sometime back: http://www.cs.wisc.edu/~param/software/cfgparse/ It would probably need some cleanup and restructuring to be used, but it may be a good starting point if configparser is to be rewritten. -param From ncoghlan at gmail.com Fri Jan 20 16:26:19 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 21 Jan 2006 01:26:19 +1000 Subject: [Python-Dev] timeit module In-Reply-To: References: <20060116052723.31287.qmail@web54314.mail.yahoo.com> Message-ID: <43D1011B.20103@gmail.com> Steve Holden wrote: > Connelly Barnes wrote: >> Hi, >> >> Perhaps I am the only one bothered by the timeit >> module, but it seems poorly designed to me. >> >> First of all, it should use a geometric series with a >> timeout value to detect how many iterations it should >> perform. Currently, the user is required to manually >> specify the number of iterations (the default is 1 > > The provision of a default value generally seems to chop the legs from > your argument that the number of iterations is required. From the command line, timeit autocalibrates to use the smallest power of ten that results in taking at least 0.2 seconds to execute all iterations. IMO, making this the default behaviour of the timer objects themselves would be *much* more useful behaviour than the current default to 10e6 iterations. I've been caught a few times hammering on Ctrl-C when I realised that I'd just told the interpreter to go do a million iterations of something that took a few seconds for each attempt. >> million). If the user optimizes his or her code, then >> the number of iterations must be changed. If the user >> moves to a slower or faster computer, then the number >> of iterations must be changed again. This is >> annoying. >> > What? The purpose of timeit is to give an approximation to the length of > time to run a specific piece ofcode. > > Why must the number of iterations be changed when moving to a slower or > faster computer? Because the default of 10e6 may be entirely inappropriate depending on how long an individual iteration is. If a single iteration takes a microsecond, great, but if it takes a second, better send out for pizza (or maybe go for a trip around the world in the 4 months that sucker is going to take to execute!) If Timer was changed as follows, it would "just work", even when each iteration took a few seconds (6 seconds per iteration would give 1 minute for each execution of the timeit method): class Timer: def __init__(self, stmt="pass", setup="pass", timer=default_timer): """Constructor. See class doc string.""" # ... existing constructor self.default_number = None def timeit(self, number=None): # Note changed default if number is None: if self.default_number is None: self.default_number = self.calibrate() number = self.default_number # ... existing timeit method def calibrate(self): # New method, taken from script code at end of module # find smallest power of 10 >= 10 so that 0.2 seconds <= total time # capped at 10 billion (10e10) iterations, as reaching that limit # implies fewer than 20 picoseconds (2e-11) per iteration (i.e. Fast!) for i in range(1, 10): number = 10**i x = self.timeit(number) if x >= 0.2: break self.default_number = number >> Secondly, there should be a way to time a callable >> directly. That is, without finding the string name of >> the callable and using a string "import X" statement. >> These contortions violate rules #1 and #3 of the Zen >> of Python. >> > Presumably for benchmarking purposes the function call overhead would be > present for all compaered techniques. Do you mean rules #0 and #2? Timing an existing function really is a pain - timeit expects source code it can plug into a code template, so timing an existing function from an interactive session isn't as easy as it could be (you have to figure out a way to give strings to Timer that it can plug into its string template, rather than just giving it the callable directly). A subclass would deal with that quite handily: class CallTimer(Timer): # Use lambda or functional.partial to pass arguments to the callable def __init__(self, callable, timer=default_timer): def inner(_it, _timer): _t0 = _timer() for _i in _it: callable() _t1 = _timer() return _t1 - _t0 self.timer = timer self.inner = inner self.src = None # Using a real function, so leave linecache alone when printing exceptions def print_exc(self, file=None): import traceback traceback.print_exc(file=file) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From tim.peters at gmail.com Fri Jan 20 17:08:08 2006 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 20 Jan 2006 11:08:08 -0500 Subject: [Python-Dev] pystate.c changes for Python 2.4.2 In-Reply-To: <2mirsfnsbz.fsf@starship.python.net> References: <43C68823.3000105@corest.com> <2mwth4qw3q.fsf@starship.python.net> <43C7BB1F.3000904@corest.com> <43CBC95F.3020907@core-sdi.com> <1f7befae0601191707q57552dc8lb75c90aa30e2924a@mail.gmail.com> <2mirsfnsbz.fsf@starship.python.net> Message-ID: <1f7befae0601200808x7ba224dft44e8e4e64feab9f2@mail.gmail.com> [Tim] >> ... >> As a result, it so happens that core Python never uses the original >> PyThreadState_Delete() anymore, except when Py_NewInterpreter() has >> to throw away the brand new thread state it created because it turns out >> it can't create a new interpreter. [Michael] > Um, PyThreadState_Delete() is called from zapthreads() is called from > PyInterpreterState_Delete() is called from Py_Finalize()... so I don't > entirely believe you here :) That's only because I was wrong ;-) Thanks! That use was in fact half the cause of Python's segfaulting shutdown races, so it was exceptionally stupid of me to miss that one. >> Since core Python never calls Py_NewInterpreter() or >> PyThreadState_Delete(), you're in an intersection of areas no current >> Python developer even thinks about, let alone tests. But by the same >> token, _because_ it's unused, there's nothing you can do to >> PyThreadState_Delete() that can hurt core Python either. That's why >> people should be more encouraging here ;-) It certainly makes sense >> to me that if a thread state is going away, any record of it in the >> auto-GIL-state machinery must also go away. > I guess if the patch fixes the original problem, it should go in -- my > uneasiness stems not from worrying that it might do harm, but rather > from the fact that I think it's incomplete. Maybe I just haven't > thought it through enough. Oh, I don't know whether it's "complete". I don't really understand multiple interpreters, have no idea why the OP is calling PyThreadState_Delete(), and fully expect that nobody is going to add tests to Python to check any of this. But Gabriel showed real skill at tracking down problems in this area, so if he creates new problems for himself by doing this (who else plays in this intersection? nobody I know of), I'm sure he'll figure them out ;-) For example, it seems possible that he'll hit the same kinds of segfaulting shutdown races the Python core used to suffer. If a thread T releases the GIL, and goes on to delete its own thread state via PyThreadState_Delete(T's_thread_state), that _is_ a race with Python shutdown's zapthreads trying to delete the same thing. That's why PyThreadState_DeleteCurrent(void) was introduced, and any call to PyThreadState_Delete() is suspect on this count. > Also, every time I look at pystate.c, I think of things that could be > done differently or better -- for example, I think it would be sane > and possibly even sensible to only call PyThread_set_key_value() in > _PyGILState_NoteThreadState() if "tstate->interp == > autoInterpreterState". I joined Mark Hammond in ignoring the possibility of multiple interpreters when this stuff went in. The best thing to do with untested gimmicks without an audience willing to work on them is to remove them. If you want to "rehabilitate" multiple interpreters here, the best first step would be to write an addendum to PEP 311 documenting an intended design. The GIL state API is documented well enough on its own, but the _intended_ semantics when mixing threads with multiple interpreters is clear as mud. PEP 311 asked "but who cares?", and nobody answered. From pje at telecommunity.com Fri Jan 20 18:48:19 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 20 Jan 2006 12:48:19 -0500 Subject: [Python-Dev] PEP 343 and __context__() In-Reply-To: <43D0ABA7.4080203@gmail.com> References: Message-ID: <5.1.1.6.0.20060120123739.02521d00@mail.telecommunity.com> At 07:21 PM 01/20/2006 +1000, Nick Coghlan wrote: >Yeah, it came up in response to PJE's suggestion of task-local variables for >generators. The basic concept we came up with is that if you're writing a >generator that uses a specific context, remember to save the original and >restore it around any calls to yield (at least, I came up with the idea and >no-one objected to the approach [1]). That's only because Guido shut down the thread. :) For the use cases I had in mind, your approach simply doesn't scale, because it requires every yield-point to have global knowledge of all "changes to global state" that may have occurred somewhere above it in the coroutine call stack. The only reason I didn't point this out is that Guido had asked for a postponement of the discussion. Since then, I've begun implementing a library that accomodates such use cases, as well as a wide variety of other context-management use cases. Source code is at http://svn.eby-sarna.com/Contextual/ and the docs are at http://svn.eby-sarna.com/Contextual/context.txt?view=markup The library uses the current draft of PEP 343 as the basis for implementation, and includes a few helper functions that can be used to emulate the "with:" statement in Python 2.4 so that it can be used and tested. So far, my impression is that the current 343 spec is at least adequate, and maybe even elegant. From guido at python.org Fri Jan 20 19:17:46 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 20 Jan 2006 10:17:46 -0800 Subject: [Python-Dev] yield back-and-forth? Message-ID: The discussion about PEP 343 reminds me of the following. Bram Cohen pointed out in private email that, before PEP 342, there wasn't a big need for a shortcut to pass control to a "sub-generator" because the following for-loop works well enough: def main_generator(): ... for value in sub_generator(): yield value but now that yield can return a value, that value might have to be passed into sub_generator(), not to mention of exceptions. I'm sure there's a way to write that (although I haven't found the time to figure it out) but I expect it to be cumbersome and subtle. I don't think a helper function can be created to solve this problem, because the yield syntax is essential. Bram suggested the following syntax: def main_generator(): ... yieldthrough sub_generator() I'm not keen on that particular keyword, but I do believe a syntactic solution is needed, if the problem is important enough to be solved. Thoughts? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz at pythoncraft.com Fri Jan 20 19:46:35 2006 From: aahz at pythoncraft.com (Aahz) Date: Fri, 20 Jan 2006 10:46:35 -0800 Subject: [Python-Dev] site triggering a bug in urllib2 In-Reply-To: <43CD082F.1090606@exa-networks.co.uk> References: <43CD082F.1090606@exa-networks.co.uk> Message-ID: <20060120184635.GA12559@panix.com> On Tue, Jan 17, 2006, Thomas Mangin wrote: > > I am contacting the list in the hope that someone will be able to > understand what I am seeing. You'll probably get more help by subscribing and posting to comp.lang.python. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From tanzer at swing.co.at Fri Jan 20 19:25:10 2006 From: tanzer at swing.co.at (Christian Tanzer) Date: Fri, 20 Jan 2006 19:25:10 +0100 Subject: [Python-Dev] yield back-and-forth? In-Reply-To: Your message of "Fri, 20 Jan 2006 10:17:46 PST." Message-ID: Guido van Rossum wrote: > The discussion about PEP 343 reminds me of the following. Bram Cohen > pointed out in private email that, before PEP 342, there wasn't a big > need for a shortcut to pass control to a "sub-generator" because the > following for-loop works well enough: > > def main_generator(): > ... > for value in sub_generator(): > yield value For small values of `well enough`. I sometimes override a generator in a subclass and it sucks to add the loop just because the derived generator wants to add a value at the beginning or end. > but now that yield can return a value, that value might have to be > passed into sub_generator(), not to mention of exceptions. I'm sure > there's a way to write that (although I haven't found the time to > figure it out) but I expect it to be cumbersome and subtle. I don't > think a helper function can be created to solve this problem, because > the yield syntax is essential. > > Bram suggested the following syntax: > > def main_generator(): > ... > yieldthrough sub_generator() > > I'm not keen on that particular keyword, but I do believe a syntactic > solution is needed, if the problem is important enough to be solved. How about: def main_generator(): ... yield * sub_generator() Ducking-ly yrs, -- Christian Tanzer http://www.c-tanzer.at/ From pje at telecommunity.com Fri Jan 20 20:05:30 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 20 Jan 2006 14:05:30 -0500 Subject: [Python-Dev] yield back-and-forth? In-Reply-To: Message-ID: <5.1.1.6.0.20060120140021.02565a70@mail.telecommunity.com> At 10:17 AM 01/20/2006 -0800, Guido van Rossum wrote: >The discussion about PEP 343 reminds me of the following. Bram Cohen >pointed out in private email that, before PEP 342, there wasn't a big >need for a shortcut to pass control to a "sub-generator" because the >following for-loop works well enough: > > def main_generator(): > ... > for value in sub_generator(): > yield value > >but now that yield can return a value, that value might have to be >passed into sub_generator(), not to mention of exceptions. I'm sure >there's a way to write that (although I haven't found the time to >figure it out) but I expect it to be cumbersome and subtle. I don't >think a helper function can be created to solve this problem, because >the yield syntax is essential. > >Bram suggested the following syntax: > > def main_generator(): > ... > yieldthrough sub_generator() > >I'm not keen on that particular keyword, but I do believe a syntactic >solution is needed, if the problem is important enough to be solved. What's the use case for this? In the coroutine use case, the PEP 342 sample trampoline takes care of this. If you yield a sub-generator (i.e. 'yield sub_generator()'), the trampoline effectively replaces the parent with the child until the child is complete. So, that leaves only non-coroutine use cases, and I'm having a hard time thinking of any where there would be bidirectional communication. >Thoughts? If we have to have a syntax, "yield from sub_generator()" seems clearer than "yieldthrough", and doesn't require a new keyword. From janssen at parc.com Fri Jan 20 20:08:12 2006 From: janssen at parc.com (Bill Janssen) Date: Fri, 20 Jan 2006 11:08:12 PST Subject: [Python-Dev] site triggering a bug in urllib2 In-Reply-To: Your message of "Fri, 20 Jan 2006 10:46:35 PST." <20060120184635.GA12559@panix.com> Message-ID: <06Jan20.110813pst."58633"@synergy1.parc.xerox.com> Or the Web-SIG mailing list. Bill From guido at python.org Fri Jan 20 20:19:30 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 20 Jan 2006 11:19:30 -0800 Subject: [Python-Dev] yield back-and-forth? In-Reply-To: <5.1.1.6.0.20060120140021.02565a70@mail.telecommunity.com> References: <5.1.1.6.0.20060120140021.02565a70@mail.telecommunity.com> Message-ID: On 1/20/06, Phillip J. Eby wrote: > At 10:17 AM 01/20/2006 -0800, Guido van Rossum wrote: > >The discussion about PEP 343 reminds me of the following. Bram Cohen > >pointed out in private email that, before PEP 342, there wasn't a big > >need for a shortcut to pass control to a "sub-generator" because the > >following for-loop works well enough: > > > > def main_generator(): > > ... > > for value in sub_generator(): > > yield value > > > >but now that yield can return a value, that value might have to be > >passed into sub_generator(), not to mention of exceptions. I'm sure > >there's a way to write that (although I haven't found the time to > >figure it out) but I expect it to be cumbersome and subtle. I don't > >think a helper function can be created to solve this problem, because > >the yield syntax is essential. > > > >Bram suggested the following syntax: > > > > def main_generator(): > > ... > > yieldthrough sub_generator() > > > >I'm not keen on that particular keyword, but I do believe a syntactic > >solution is needed, if the problem is important enough to be solved. > > What's the use case for this? In the coroutine use case, the PEP 342 > sample trampoline takes care of this. If you yield a sub-generator (i.e. > 'yield sub_generator()'), the trampoline effectively replaces the parent > with the child until the child is complete. That's a rather specialized, subtle and elaborate framework though, and at this time I believe it isn't planned to be part of Python 2.5 (right?). > So, that leaves only non-coroutine use cases, and I'm having a hard time > thinking of any where there would be bidirectional communication. Any other use of generators where the return value of yield is used; as soon as you do this you may later want to refactor the code to to include sub-iterators. (There *are*other uses besides the trampoline, right? :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From edcjones at comcast.net Fri Jan 20 20:51:39 2006 From: edcjones at comcast.net (Edward C. Jones) Date: Fri, 20 Jan 2006 14:51:39 -0500 Subject: [Python-Dev] yield back-and-forth? Message-ID: <43D13F4B.2030406@comcast.net> Guido van Rossum wrote: The discussion about PEP 343 reminds me of the following. Bram Cohen pointed out in private email that, before PEP 342, there wasn't a big need for a shortcut to pass control to a "sub-generator" because the following for-loop works well enough: def main_generator(): ... for value in sub_generator(): yield value This is an important programming trick. It allows recursive use of generators for walking trees, etc. See the source code for os.walk for an example. From jjl at pobox.com Fri Jan 20 21:28:23 2006 From: jjl at pobox.com (John J Lee) Date: Fri, 20 Jan 2006 20:28:23 +0000 (GMT Standard Time) Subject: [Python-Dev] site triggering a bug in urllib2 In-Reply-To: <43CD082F.1090606@exa-networks.co.uk> References: <43CD082F.1090606@exa-networks.co.uk> Message-ID: On Tue, 17 Jan 2006, Thomas Mangin wrote: [...] > I have hit a bug with python 2.4.2 (on Mandriva 2006) using urllib2. > The code which trigger the bug is as follow.. > > import urllib2 > req = urllib2.Request("http://66.117.37.13/") > > # makes no difference .. > req.add_header('Connection', 'close') > > handle = urllib2.urlopen(req) > data = handle.read() > print data > > using a timeout on the socket does not work neither. This is a real bug, I think. I filed a report on the SF bug tracker: http://python.org/sf/1411097 The problem seems to be the (ab)use of socket._fileobject in urllib2 (I believe this was introduced when urllib2 switched to using httplib.HTTPConnection). The purpose of the hack (as commented in AbstractHTTPHandler.do_open()) is to provide .readline() and .readlines() methods on the response object returned by urllib2.urlopen(). Workaround if you're not using .readline() or .readlines() (against 2.4.2, but should apply against current SVN): --- urllib2.py.orig Fri Jan 20 20:10:56 2006 +++ urllib2.py Fri Jan 20 20:12:07 2006 @@ -1006,8 +1006,7 @@ # XXX It might be better to extract the read buffering code # out of socket._fileobject() and into a base class. - r.recv = r.read - fp = socket._fileobject(r) + fp = r.fp resp = addinfourl(fp, r.msg, req.get_full_url()) resp.code = r.status Not sure yet what the actual problem/cure is... John From jason.orendorff at gmail.com Fri Jan 20 21:34:52 2006 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Fri, 20 Jan 2006 15:34:52 -0500 Subject: [Python-Dev] PEP 343 and __context__() In-Reply-To: <43D0ABA7.4080203@gmail.com> References: <43D0ABA7.4080203@gmail.com> Message-ID: On 1/20/06, Nick Coghlan wrote: > Jason Orendorff wrote: > > DecimalContext has a few problems. In code where it matters, every > > function you write has to worry about it. (That is, you can't just > > write __decimal_context__ = ... at the top of the file and be done > > with it, the way you can with, say, __metaclass__.) > > No, you write "decimal.setcontext(...)" instead. You seem to be implying these are roughly equal in convenience; I disagree. Suppose I have banking.py, in which it's important to use a particular precision and rounding. Now I have to put context-munging code in every single function that banking.py exposes. And around every 'yield'. Even with 'with', that's a lot of extra lines of code. I'd much prefer to put a one-liner at the top of the file, if it were possible (...but I don't see how, yet). Again, none of this is likely to matter--unless you're interleaving banking and heavy scientific calculations, which I try to avoid. So, not a big deal. Thanks for the response. > > And > > DecimalContext doesn't fit in with generators. > > It does fit actually - you simply have to remember to restore the original > context around any invocations of yield. Feh! "Fit" is to "can be made to work with a bit of effort, just don't forget to follow the rules" as Python is to C++. -j From thomas at xs4all.net Fri Jan 20 21:43:29 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Fri, 20 Jan 2006 21:43:29 +0100 Subject: [Python-Dev] [Python-checkins] r42116 - python/branches/release24-maint/Lib/unittest.py In-Reply-To: <20060120175503.4C0C71E4002@bag.python.org> References: <20060120175503.4C0C71E4002@bag.python.org> Message-ID: <20060120204329.GZ18916@xs4all.nl> On Fri, Jan 20, 2006 at 06:55:03PM +0100, georg.brandl wrote: > Author: georg.brandl > Date: Fri Jan 20 18:55:02 2006 > New Revision: 42116 > > Modified: > python/branches/release24-maint/Lib/unittest.py > Log: > Patch #1388073: Make unittest.TestCase easier to subclass I don't believe this belongs in 2.4, since it can, actually, break code. Code that depends on the current situation, _TestCase__attributename. Fragile code, to be sure, but still. If there were a compelling reason to backport, I guess it could be hacked to work right-ish, but subclassing TestCase in this way, while convenient, isn't important enough to warrant this (IMHO). -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From guido at python.org Fri Jan 20 21:50:16 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 20 Jan 2006 12:50:16 -0800 Subject: [Python-Dev] [Python-checkins] r42116 - python/branches/release24-maint/Lib/unittest.py In-Reply-To: <20060120204329.GZ18916@xs4all.nl> References: <20060120175503.4C0C71E4002@bag.python.org> <20060120204329.GZ18916@xs4all.nl> Message-ID: Right. This *definitely* is a feature, not a bug. On 1/20/06, Thomas Wouters wrote: > On Fri, Jan 20, 2006 at 06:55:03PM +0100, georg.brandl wrote: > > Author: georg.brandl > > Date: Fri Jan 20 18:55:02 2006 > > New Revision: 42116 > > > > Modified: > > python/branches/release24-maint/Lib/unittest.py > > Log: > > Patch #1388073: Make unittest.TestCase easier to subclass > > I don't believe this belongs in 2.4, since it can, actually, break code. > Code that depends on the current situation, _TestCase__attributename. > Fragile code, to be sure, but still. If there were a compelling reason to > backport, I guess it could be hacked to work right-ish, but subclassing > TestCase in this way, while convenient, isn't important enough to warrant > this (IMHO). > > -- > Thomas Wouters > > Hi! I'm a .signature virus! copy me into your .signature file to help me spread! > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Fri Jan 20 21:53:06 2006 From: barry at python.org (Barry Warsaw) Date: Fri, 20 Jan 2006 15:53:06 -0500 Subject: [Python-Dev] [Python-checkins] r42116 - python/branches/release24-maint/Lib/unittest.py In-Reply-To: <20060120204329.GZ18916@xs4all.nl> References: <20060120175503.4C0C71E4002@bag.python.org> <20060120204329.GZ18916@xs4all.nl> Message-ID: <1137790386.8802.38.camel@geddy.wooz.org> On Fri, 2006-01-20 at 21:43 +0100, Thomas Wouters wrote: > I don't believe this belongs in 2.4, since it can, actually, break code. > Code that depends on the current situation, _TestCase__attributename. > Fragile code, to be sure, but still. If there were a compelling reason to > backport, I guess it could be hacked to work right-ish, but subclassing > TestCase in this way, while convenient, isn't important enough to warrant > this (IMHO). Exactly right. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060120/5c18d196/attachment.pgp From g.brandl-nospam at gmx.net Fri Jan 20 22:18:20 2006 From: g.brandl-nospam at gmx.net (Georg Brandl) Date: Fri, 20 Jan 2006 22:18:20 +0100 Subject: [Python-Dev] [Python-checkins] r42116 - python/branches/release24-maint/Lib/unittest.py In-Reply-To: <1137790386.8802.38.camel@geddy.wooz.org> References: <20060120175503.4C0C71E4002@bag.python.org> <20060120204329.GZ18916@xs4all.nl> <1137790386.8802.38.camel@geddy.wooz.org> Message-ID: Barry Warsaw wrote: > On Fri, 2006-01-20 at 21:43 +0100, Thomas Wouters wrote: > >> I don't believe this belongs in 2.4, since it can, actually, break code. >> Code that depends on the current situation, _TestCase__attributename. >> Fragile code, to be sure, but still. If there were a compelling reason to >> backport, I guess it could be hacked to work right-ish, but subclassing >> TestCase in this way, while convenient, isn't important enough to warrant >> this (IMHO). > > Exactly right. You're right. Next time I'm going to decide in favor of not backporting. I reverted both checkins, this and the urlparse one. For 2.5, this would be a good opportunity to add additional schemes to urlparse. Candidates? Georg From pje at telecommunity.com Fri Jan 20 22:39:13 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 20 Jan 2006 16:39:13 -0500 Subject: [Python-Dev] yield back-and-forth? In-Reply-To: References: <5.1.1.6.0.20060120140021.02565a70@mail.telecommunity.com> <5.1.1.6.0.20060120140021.02565a70@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060120143818.00a981e0@mail.telecommunity.com> At 11:19 AM 01/20/2006 -0800, Guido van Rossum wrote: >(There *are*other uses besides the trampoline, >right? :-) It's easy to come up with use cases where you feed data *into* a generator (parsers and pipelines, for example). I just don't know of any "simultaneous bidirectional" uses other than in association with a coroutine scheduler, or a dedicated pipelining tool. In both the coroutine and pipelining cases, generator composition is an easy job for whatever coroutine or pipelining library is in use; it just happens external to the generator, perhaps by yielding a special value. From ncoghlan at gmail.com Sat Jan 21 03:07:45 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 21 Jan 2006 12:07:45 +1000 Subject: [Python-Dev] PEP 343 and __context__() In-Reply-To: References: <43D0ABA7.4080203@gmail.com> Message-ID: <43D19771.1040900@gmail.com> Jason Orendorff wrote: > On 1/20/06, Nick Coghlan wrote: >> Jason Orendorff wrote: >>> DecimalContext has a few problems. In code where it matters, every >>> function you write has to worry about it. (That is, you can't just >>> write __decimal_context__ = ... at the top of the file and be done >>> with it, the way you can with, say, __metaclass__.) >> No, you write "decimal.setcontext(...)" instead. > > You seem to be implying these are roughly equal in convenience; I > disagree. Suppose I have banking.py, in which it's important to use a > particular precision and rounding. Now I have to put context-munging > code in every single function that banking.py exposes. And around > every 'yield'. Even with 'with', that's a lot of extra lines of code. Ah, I understand your point better now. > I'd much prefer to put a one-liner at the top of the file, if it were > possible (...but I don't see how, yet). Me neither. The best I can come up with is a "precise" decorator that takes care of the context munging. Which would at least reduce the boilerplate to "@precise" on functions and "@precise_gen" on generators. > Again, none of this is likely to matter--unless you're interleaving > banking and heavy scientific calculations, which I try to avoid. So, > not a big deal. Thanks for the response. No worries. I agree there's still some awkwardness, but I do think it's an improvement on the status quo (where we have the same problems with interactions between modules and generators and thread-state, but dealing with them can't really be factored out at all). >>> And >>> DecimalContext doesn't fit in with generators. >> It does fit actually - you simply have to remember to restore the original >> context around any invocations of yield. > > Feh! "Fit" is to "can be made to work with a bit of effort, just > don't forget to follow the rules" as Python is to C++. I see what you mean. I'm sure we can rely on PJE and others to push the limits of with statements and provide feedback on improving their interaction with generators in Python 2.6 :) (although before that we need to bring mwh's patch up to speed with the svn trunk. . .) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Sat Jan 21 05:40:29 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 21 Jan 2006 14:40:29 +1000 Subject: [Python-Dev] yield back-and-forth? In-Reply-To: <5.1.1.6.0.20060120140021.02565a70@mail.telecommunity.com> References: <5.1.1.6.0.20060120140021.02565a70@mail.telecommunity.com> Message-ID: <43D1BB3D.8040005@gmail.com> Phillip J. Eby wrote: >> Thoughts? > > If we have to have a syntax, "yield from sub_generator()" seems clearer > than "yieldthrough", and doesn't require a new keyword. Andrew Koenig suggested the same phrasing last year [1], and I liked it then. I don't like it any more, though, as I think it is too inflexible, and we have a better option available (specifically, stealing "continue with an argument" from PEP 340). The following ramblings (try to) explain my reasoning :) Guido does raise an interesting point. The introduction of "send" and "throw" means that the current simple loop approach does not easily allow values to be passed down to nested generators, nor does not it correctly terminate nested generators in response to an invocation of "throw". Because of the explicit for loop, the nested generator only gets cleaned up in response to GC - it never sees the exception that occurs in the body of the for loop (at the point of the yield expression). The "yield from iterable" concept could be translated roughly as follows: itr = iter(iterable) try: send_input = itr.send # Can input values be passed down? except AttributeError: send_input = None try: next = itr.next() # Get the first output except StopIteration: pass else: while 1: try: input = yield next # yield and get input except: try: throw_exc = itr.throw # Can exception be passed down? except AttributeError: raise # Nope, reraise else: throw_exc(sys.exc_info()) # Yep, pass it down else: try: if send_input is None: if input is not None: raise TypeError("Cannot send input!") next = itr.next() else: next = send_input(input) # Pass input down except StopIteration: break I'm not particularly happy with this, though, as not only is it horribly implicit and magical, it's trivial to accidentally break the chain - consider what happens if you naively do: yield from (x*x for x in sub_generator()) The chain has been broken - the sub generator no longer sees either passed in values or thrown exceptions, as the generator expression intercepts them without passing them down. Even worse, IMO, is that the syntax is entirely inflexible - we have no easy way to manipulate either the results sent from the generator, or the input values passed to it. However, an idea from Guido's PEP 340 helps with the "send" part of the story, involving passing an argument to continue: def main_generator(): ... for value in sub_generator(): continue yield value Here, sub_generator's "send" method would be invoked with the result of the call to yield value. Manipulation in either direction (input or output) is trivial: def main_generator(): ... for value in sub_generator(): input = yield value*value # Square the output values continue input*input # Square the input values, too You could even do odd things like yield each value twice, and then pass down pairs of inputs: def main_generator(): ... for value in sub_generator(): continue (yield value), (yield value) The need to use a "continue" statement eliminates the temptation to use a generator expression, and makes it far less likely the downwards connection between the main generator and the sub generator will be accidentally broken. Exception propagation is a different story. What do you want to propagate? All exceptions from the body of the for loop? Or just those from the yield statement? Well, isn't factoring out exception processing part of what PEP 343 is for? # Simply make sure the generator is closed promptly def main_generator(): ... with closing(sub_generator()) as gen: for value in gen: continue yield value # Or throw the real exception to the nested generator class throw_to(object): def __init__(self, gen): self.gen = gen def __enter__(self): return self.gen def __exit__(self, exc_type, *exc_details): if exc_type is not None: try: self.gen.throw(exc_type, *exc_details) except StopIteration: pass def main_generator(): ... with throw_to(sub_generator()) as gen: for value in gen: continue yield value # We can even limit the propagated exceptions to those # from the outside world and leave the rest alone def main_generator(): ... gen = sub_generator() for value in gen: with throw_to(gen): input = yield value continue input Thanks to Jason's other thread, I even have a hypothetical use case for all this. Specifically, "generator decorators", that manipulate the state that holds while executing the generator. Suppose I have a number of generators that I want to work in high precision decimal, but yield results with normal precision. Doing this dance manually in every generator would be a pain, so let's use a decorator: def precise_gen(gen_func): def wrapper(*args, **kwds): orig_ctx = decimal.getcontext() with orig_ctx as ctx: ctx.prec = HIGH_PRECISION gen = gen_func(*args, **kwds) for val in gen: with orig_ctx: val = +val try: yield val except: gen.throw() # make it default to sys.exc_info() wrapper.__name__ = gen_func.__name__ wrapper.__dict__ = gen_func.__dict__ wrapper.__doc__ = gen_func.__doc__ return wrapper So far, so good (although currently, unlike a raise statement, gen.throw() doesn't automatically default to sys.exc_info()). Tough luck, however, if you want to use this on a generator that accepts input values - those inputs will get dropped on the floor by the wrapper, and the wrapped generator will never see them. "yield from" wouldn't help in the least, because there are other things going on in the for loop containing the yield statement. 'continue with argument', however, would work just fine: def precise_gen(gen_func): def wrapper(*args, **kwds): orig_ctx = decimal.getcontext() with orig_ctx as ctx: ctx.prec = HIGH_PRECISION gen = gen_func(*args, **kwds) for val in gen: with orig_ctx: val = +val try: continue yield val except: gen.throw() # make it default to sys.exc_info() wrapper.__name__ = gen_func.__name__ wrapper.__dict__ = gen_func.__dict__ wrapper.__doc__ = gen_func.__doc__ return wrapper Now an arbitrary generator can be decorated with "@precise_gen", and it's internal operations will be carried out with high precision, while any interim results will be yielded using the caller's precision and the caller's rounding rules. Cheers, Nick. [1] http://mail.python.org/pipermail/python-dev/2005-October/057410.html -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Sat Jan 21 05:55:19 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 21 Jan 2006 14:55:19 +1000 Subject: [Python-Dev] yield back-and-forth? In-Reply-To: <43D1BB3D.8040005@gmail.com> References: <5.1.1.6.0.20060120140021.02565a70@mail.telecommunity.com> <43D1BB3D.8040005@gmail.com> Message-ID: <43D1BEB7.10202@gmail.com> Nick Coghlan wrote: > Exception propagation is a different story. What do you want to propagate? All > exceptions from the body of the for loop? Or just those from the yield statement? > > Well, isn't factoring out exception processing part of what PEP 343 is for? > # We can even limit the propagated exceptions to those > # from the outside world and leave the rest alone > def main_generator(): > ... > gen = sub_generator() > for value in gen: > with throw_to(gen): > input = yield value > continue input A point I should have made here (but didn't, despite using it in a later example) is that the "with" versions don't allow the nested generator to suppress the exception. A small refinement to gen.throw() that makes the first argument optional (similar to the raise statement itself) would make it straightforward to allow the generator to suppress passed in exceptions: def main_generator(): ... gen = sub_generator() for value in gen: try: input = yield value except: gen.throw() # Default to sys.exc_info(), just like raise continue input Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ark at acm.org Sat Jan 21 06:55:05 2006 From: ark at acm.org (Andrew Koenig) Date: Sat, 21 Jan 2006 00:55:05 -0500 Subject: [Python-Dev] yield back-and-forth? In-Reply-To: Message-ID: <000201c61e4f$3dafaa10$6402a8c0@arkdesktop> > The discussion about PEP 343 reminds me of the following. Bram Cohen > pointed out in private email that, before PEP 342, there wasn't a big > need for a shortcut to pass control to a "sub-generator" because the > following for-loop works well enough: > def main_generator(): > ... > for value in sub_generator(): > yield value > but now that yield can return a value, that value might have to be > passed into sub_generator(), not to mention of exceptions. I'm sure > there's a way to write that (although I haven't found the time to > figure it out) but I expect it to be cumbersome and subtle. It looks to me like continuations are starting to rear their heads... From aleaxit at gmail.com Sat Jan 21 06:57:00 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Fri, 20 Jan 2006 21:57:00 -0800 Subject: [Python-Dev] yield back-and-forth? In-Reply-To: <5.1.1.6.0.20060120143818.00a981e0@mail.telecommunity.com> References: <5.1.1.6.0.20060120140021.02565a70@mail.telecommunity.com> <5.1.1.6.0.20060120140021.02565a70@mail.telecommunity.com> <5.1.1.6.0.20060120143818.00a981e0@mail.telecommunity.com> Message-ID: <048D2824-99D0-4E79-8C87-4361CA3EF3C0@gmail.com> On Jan 20, 2006, at 1:39 PM, Phillip J. Eby wrote: > At 11:19 AM 01/20/2006 -0800, Guido van Rossum wrote: >> (There *are*other uses besides the trampoline, >> right? :-) > > It's easy to come up with use cases where you feed data *into* a > generator > (parsers and pipelines, for example). I just don't know of any > "simultaneous bidirectional" uses other than in association with a > coroutine scheduler, or a dedicated pipelining tool. In both the > coroutine Hmm, what about an iterator of iterators, where the sub-iterators need to be advanced until some condition is satisfied, then processed from that point onwards (only those subiterators for which some item did satisfy the condition). E.g.: goodfiles = [] for afile in manyfiles: for aline in afile: if aline.startswith('*starthere*'): goodfiles.append(afile) break for afile in goodfiles: ... I'm sure we've all met this kind of issue (indeed, standard library module fileinput does support this case directly, by wrapping the double loop into one yielding lines, and still providing ways to get the current file and skipping to the next file). It might be nice to wrap the general logic (double nested looping &c) in a generator leaving the specifics (exactly when to consider a subiterator 'good' and what to do in that case) out of it, sort of like fileinput does but with more generality than just files and lines thereof. In Python 2.5 it seems that a .send call with a non- None argument might be a natural convention for the loop body to tell the generator "abandon this subiterator and yield it, then move to the next one". E.g.: goodfiles = [] allinput = inputfrom(manyfiles) for aline in allinput: if aline.startswith('*starthere*'): goodfiles.append(allinput.send(True)) for afile in goodfiles: ... Perhaps not the greatest of simplifications, but it might be generalized to (e.g.) recursive walks with the same client-logic as above, while the original (nested for-loops) is pretty rigid. And the inputfrom generator doesn't look too complicated either: def inputfrom(iterators): for iterator in iterators: for item in iterator: if (yield item) is not None: yield iterator break Wouldn't this be considered a reasonable use for the new generator features? Alex From barry at python.org Sat Jan 21 17:19:11 2006 From: barry at python.org (Barry Warsaw) Date: Sat, 21 Jan 2006 11:19:11 -0500 Subject: [Python-Dev] Building on OS X 10.4 fails In-Reply-To: <871wz57job.fsf@tleepslib.sk.tsukuba.ac.jp> References: <200601181117.34549.anthony@interlink.com.au> <87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp> <200601182031.54111.anthony@interlink.com.au> <871wz57job.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: On Jan 18, 2006, at 8:24 AM, Stephen J. Turnbull wrote: > The import of the Ghostscript case is that the FSF considers a > Makefile stanza clearly intended to cause linkage to a GPL library, > even if optional and supplied by the user, to create a work derived > from that library. A "GNU readline"-enabled Python is derived from > GNU readline, and must be distributed under the GPL or not distributed > at all. I assume that the former is not acceptable to Python, and the > latter is clearly undesirable. In that case, what I think we ought to do is not add the DP paths (i.e. /opt/local) to setup.py specifically to get its readline, but instead to pick up any libraries that happen to be in DP in preference to those in OSX by default. If that /happens/ to cause a different version of readline to be used as a side-effect, then oops! sorry, but so be it. -Barry From barry at python.org Sat Jan 21 17:41:23 2006 From: barry at python.org (Barry Warsaw) Date: Sat, 21 Jan 2006 11:41:23 -0500 Subject: [Python-Dev] Building on OS X 10.4 fails In-Reply-To: References: <200601181117.34549.anthony@interlink.com.au> <87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp> <200601182031.54111.anthony@interlink.com.au> <871wz57job.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <8E8FF935-4D7F-4542-A8EF-70279810099C@python.org> On Jan 21, 2006, at 11:19 AM, Barry Warsaw wrote: > In that case, what I think we ought to do is not add the DP paths > (i.e. /opt/local) to setup.py specifically to get its readline, but > instead to pick up any libraries that happen to be in DP in > preference to those in OSX by default. If that /happens/ to cause a > different version of readline to be used as a side-effect, then oops! > sorry, but so be it. This patch (currently unassigned) does the trick for me: https://sourceforge.net/tracker/index.php? func=detail&aid=1411588&group_id=5470&atid=305470 -Barry From martin at v.loewis.de Sat Jan 21 19:37:08 2006 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 21 Jan 2006 19:37:08 +0100 Subject: [Python-Dev] [Python-checkins] r42116 - python/branches/release24-maint/Lib/unittest.py In-Reply-To: References: <20060120175503.4C0C71E4002@bag.python.org> <20060120204329.GZ18916@xs4all.nl> <1137790386.8802.38.camel@geddy.wooz.org> Message-ID: <43D27F54.70004@v.loewis.de> Georg Brandl wrote: > For 2.5, this would be a good opportunity to add additional schemes > to urlparse. Candidates? The registered ones: http://www.iana.org/assignments/uri-schemes Not sure whether urn parsing would also be desirable: http://www.iana.org/assignments/urn-namespaces Regards, Martin From fdrake at acm.org Sat Jan 21 20:23:32 2006 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Sat, 21 Jan 2006 14:23:32 -0500 Subject: [Python-Dev] =?iso-8859-1?q?=5BPython-checkins=5D_r42116_-=09pyth?= =?iso-8859-1?q?on/branches/release24-maint/Lib/unittest=2Epy?= In-Reply-To: <43D27F54.70004@v.loewis.de> References: <20060120175503.4C0C71E4002@bag.python.org> <43D27F54.70004@v.loewis.de> Message-ID: <200601211423.33402.fdrake@acm.org> On Saturday 21 January 2006 13:37, Martin v. L?wis wrote: > The registered ones: > > http://www.iana.org/assignments/uri-schemes I think that these should be supported. > Not sure whether urn parsing would also be desirable: > > http://www.iana.org/assignments/urn-namespaces The "hdl" scheme already included in the urlparse module is a URN scheme from CNRI. Either we should support URNs, or we should consider deprecating support for that one. -Fred -- Fred L. Drake, Jr. From g.brandl-nospam at gmx.net Sat Jan 21 21:11:53 2006 From: g.brandl-nospam at gmx.net (Georg Brandl) Date: Sat, 21 Jan 2006 21:11:53 +0100 Subject: [Python-Dev] New Pythondoc by effbot Message-ID: What Fredrik hacks together there (http://www.effbot.org/lib) is very impressive. I especially like the "permalinks" in this style: http://effbot.org/lib/os.path.join What I would suggest (for any new doc system) is a "split" view: on the left, the normal text, on the right, an area with only the headings, functions, example and "see also" links (which preferably stays in sight). This way, you always keep the outline in view. Of course, I wouldn't say no to a user-commenting system, but this would have to be moderated. What I'm also curious about regarding the current docs, why are optional arguments in function declarations not written in Python style? regards, Georg PS: Fredrik, if I can help you, please tell me! From g.brandl-nospam at gmx.net Sat Jan 21 21:23:11 2006 From: g.brandl-nospam at gmx.net (Georg Brandl) Date: Sat, 21 Jan 2006 21:23:11 +0100 Subject: [Python-Dev] [Python-checkins] r42116 - python/branches/release24-maint/Lib/unittest.py In-Reply-To: <200601211423.33402.fdrake@acm.org> References: <20060120175503.4C0C71E4002@bag.python.org> <43D27F54.70004@v.loewis.de> <200601211423.33402.fdrake@acm.org> Message-ID: Fred L. Drake, Jr. wrote: > On Saturday 21 January 2006 13:37, Martin v. L?wis wrote: > > The registered ones: > > > > http://www.iana.org/assignments/uri-schemes > > I think that these should be supported. That's okay, but it may be much work to find out which of them use relative paths, fragments, queries and parameters. Is there a list of these features somewhere? Georg From martin at v.loewis.de Sat Jan 21 21:59:11 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 21 Jan 2006 21:59:11 +0100 Subject: [Python-Dev] [Python-checkins] r42116 - python/branches/release24-maint/Lib/unittest.py In-Reply-To: References: <20060120175503.4C0C71E4002@bag.python.org> <43D27F54.70004@v.loewis.de> <200601211423.33402.fdrake@acm.org> Message-ID: <43D2A09F.5060101@v.loewis.de> Georg Brandl wrote: > That's okay, but it may be much work to find out which of them use relative > paths, fragments, queries and parameters. Is there a list of these features > somewhere? Not that I know of. Collecting a list of missing schemes in the file might be a good start already. Regards, Martin From guido at python.org Sun Jan 22 00:20:26 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 21 Jan 2006 15:20:26 -0800 Subject: [Python-Dev] New Pythondoc by effbot In-Reply-To: References: Message-ID: On 1/21/06, Georg Brandl wrote: > What Fredrik hacks together there (http://www.effbot.org/lib) is very > impressive. I especially like the "permalinks" in this style: > > http://effbot.org/lib/os.path.join Which (despite having "perma" in its name) evaporates and leaves behind a link to os.path.html#join. > What I would suggest (for any new doc system) is a "split" view: on the left, > the normal text, on the right, an area with only the headings, functions, > example and "see also" links (which preferably stays in sight). This way, you > always keep the outline in view. Can you mock that up a bit? I'm somewhat confused about what you're requesting, and also worried that it would take up too much horizontal space. (Despite that monitors are wider than tall, horizontal real estate feels more scarce than vertical, because horizontal scrolling is such a pain.) > Of course, I wouldn't say no to a user-commenting system, but this would have to > be moderated. Why? If wikipedia can do without moderation (for most pages) then why couldn't the Python docs? > What I'm also curious about regarding the current docs, why are optional > arguments in function declarations not written in Python style? I'm assuming you're asking why we use foo(a[, b[, c]]) instead of foo(a, b=1, c=2) ? I can see several reasons; arguments with default values aren't necessarily usable keyword arguments (at least not if they function/method is implemented in C); the default value isn't always relevant (or is dynamic, or is a huge expression); and square brackets are the standard way in computer manuals to indicate optional parts of syntax. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tjreedy at udel.edu Sun Jan 22 01:15:14 2006 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 21 Jan 2006 19:15:14 -0500 Subject: [Python-Dev] New Pythondoc by effbot References: Message-ID: >> http://effbot.org/lib/os.path.join On this page, 8 of 30 entries have a 'new in' comment. For anyone with no interest in the past, these constitute noise. I wonder if for 3.0, the timer can be reset and the docs start clean again. To keep them backwards compatible, they would also have to be littered with 'changed in 3.0' and 'deleted in 3.0' entries. Better, I think, to refer people to the last 2.x docs and a separate 2.x/3.0 changes doc. From guido at python.org Sun Jan 22 02:44:35 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 21 Jan 2006 17:44:35 -0800 Subject: [Python-Dev] New Pythondoc by effbot In-Reply-To: References: Message-ID: On 1/21/06, Terry Reedy wrote: > On this page, 8 of 30 entries have a 'new in' comment. For anyone with no > interest in the past, these constitute noise. I wonder if for 3.0, the > timer can be reset and the docs start clean again. To keep them backwards > compatible, they would also have to be littered with 'changed in 3.0' and > 'deleted in 3.0' entries. Better, I think, to refer people to the last 2.x > docs and a separate 2.x/3.0 changes doc. (That has nothing to do with Fredrik's efforts of course -- he's just faithfully copying the annotations from the existing docs.) If Python 3.0 is indeed the major incompatible release that we currently expect it will be, I agree that "new in 3.0" and "new in 2.x" annotations make little sense, since so much will be "new in 3.0" (or "deleted in 3.0"). It's probably better to have a separate document explaining the differences between 2.x and 3.0. For the 2.x line, however (and again post-3.0) I think the "new in X" annotations are important -- lots of people have to deal with different Python versions, and it's a lot easier to depend on "new in X" notations than to manually compare two or more versions of the docs. (Although you still need to do thorough testing -- like the rest of the docs, these annotations can't be 100% perfect.) As far as noise goes, "new in X" is minor compared to all the stuff that's documented that the average user never needs... :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From janssen at parc.com Sun Jan 22 02:58:14 2006 From: janssen at parc.com (Bill Janssen) Date: Sat, 21 Jan 2006 17:58:14 PST Subject: [Python-Dev] [Python-checkins] r42116 - python/branches/release24-maint/Lib/unittest.py In-Reply-To: Your message of "Fri, 20 Jan 2006 13:18:20 PST." Message-ID: <06Jan21.175820pst."58633"@synergy1.parc.xerox.com> The "data:" scheme would be a good one. Bill > Barry Warsaw wrote: > > On Fri, 2006-01-20 at 21:43 +0100, Thomas Wouters wrote: > > > >> I don't believe this belongs in 2.4, since it can, actually, break code. > >> Code that depends on the current situation, _TestCase__attributename. > >> Fragile code, to be sure, but still. If there were a compelling reason to > >> backport, I guess it could be hacked to work right-ish, but subclassing > >> TestCase in this way, while convenient, isn't important enough to warrant > >> this (IMHO). > > > > Exactly right. > > You're right. Next time I'm going to decide in favor of not backporting. > I reverted both checkins, this and the urlparse one. > > > For 2.5, this would be a good opportunity to add additional schemes > to urlparse. Candidates? > > Georg > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/janssen%40parc.com From aahz at pythoncraft.com Sun Jan 22 05:41:31 2006 From: aahz at pythoncraft.com (Aahz) Date: Sat, 21 Jan 2006 20:41:31 -0800 Subject: [Python-Dev] New Pythondoc by effbot In-Reply-To: References: Message-ID: <20060122044131.GA18077@panix.com> On Sat, Jan 21, 2006, Guido van Rossum wrote: > > Why? If wikipedia can do without moderation (for most pages) then why > couldn't the Python docs? If we're strictly talking about user comments, I won't disagree, but the main docs do need to be "authoritative" IMO. Aside to Georg: your messages are all getting /dev/null'd because you use "spam" in your From: line. I get too much spam to change that, and I'll bet other people use that heuristic. Please change your From: line for this mailing list. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From g.brandl-nospam at gmx.net Sun Jan 22 08:24:09 2006 From: g.brandl-nospam at gmx.net (Georg Brandl) Date: Sun, 22 Jan 2006 08:24:09 +0100 Subject: [Python-Dev] New Pythondoc by effbot In-Reply-To: References: Message-ID: Guido van Rossum wrote: > On 1/21/06, Terry Reedy wrote: >> On this page, 8 of 30 entries have a 'new in' comment. For anyone with no >> interest in the past, these constitute noise. I wonder if for 3.0, the >> timer can be reset and the docs start clean again. To keep them backwards >> compatible, they would also have to be littered with 'changed in 3.0' and >> 'deleted in 3.0' entries. Better, I think, to refer people to the last 2.x >> docs and a separate 2.x/3.0 changes doc. > > (That has nothing to do with Fredrik's efforts of course -- he's just > faithfully copying the annotations from the existing docs.) > > If Python 3.0 is indeed the major incompatible release that we > currently expect it will be, I agree that "new in 3.0" and "new in > 2.x" annotations make little sense, since so much will be "new in 3.0" > (or "deleted in 3.0"). It's probably better to have a separate > document explaining the differences between 2.x and 3.0. > > For the 2.x line, however (and again post-3.0) I think the "new in X" > annotations are important -- lots of people have to deal with > different Python versions, and it's a lot easier to depend on "new in > X" notations than to manually compare two or more versions of the > docs. (Although you still need to do thorough testing -- like the rest > of the docs, these annotations can't be 100% perfect.) > > As far as noise goes, "new in X" is minor compared to all the stuff > that's documented that the average user never needs... :-) And, of course, the "new in 2.x" could be formatted less space-consuming, perhaps to the right of the method name. Georg From ncoghlan at gmail.com Sun Jan 22 08:32:40 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 22 Jan 2006 17:32:40 +1000 Subject: [Python-Dev] New Pythondoc by effbot In-Reply-To: References: Message-ID: <43D33518.1090800@gmail.com> Georg Brandl wrote: > Guido van Rossum wrote: >> As far as noise goes, "new in X" is minor compared to all the stuff >> that's documented that the average user never needs... :-) > > And, of course, the "new in 2.x" could be formatted less space-consuming, > perhaps to the right of the method name. That makes it worse by giving the information a prominence it doesn't really deserve. It also doesn't help when the change being noted was a signature or semantics change, rather than the addition of the function. As with many other things, being able to clean out some of this history will be one of the benefits of Py3k. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From g.brandl at gmx.net Sun Jan 22 08:27:18 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 22 Jan 2006 08:27:18 +0100 Subject: [Python-Dev] New Pythondoc by effbot In-Reply-To: <20060122044131.GA18077@panix.com> References: <20060122044131.GA18077@panix.com> Message-ID: Aahz wrote: > On Sat, Jan 21, 2006, Guido van Rossum wrote: >> >> Why? If wikipedia can do without moderation (for most pages) then why >> couldn't the Python docs? > > If we're strictly talking about user comments, I won't disagree, but the > main docs do need to be "authoritative" IMO. > > Aside to Georg: your messages are all getting /dev/null'd because you use > "spam" in your From: line. I get too much spam to change that, and I'll > bet other people use that heuristic. Please change your From: line for > this mailing list. I don't quite understand that. Why would a spam email have "spam" in the From? For me, the "-nospam" suffix works relatively good to avoid spam, as most harvesting programs will think this is a false address. Of course, that may be more useful in Usenet, not in mailing lists. Georg From t-meyer at ihug.co.nz Sun Jan 22 08:56:44 2006 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Sun, 22 Jan 2006 20:56:44 +1300 Subject: [Python-Dev] New Pythondoc by effbot In-Reply-To: References: <20060122044131.GA18077@panix.com> Message-ID: <17BD8931-BF5F-45B0-BD2C-6FE1FABA365B@ihug.co.nz> > For me, the "-nospam" suffix works relatively good to avoid spam, > as most > harvesting programs will think this is a false address. http://spambayes.org works, too, without bothering others <0.5 wink> =Tony.Meyer From g.brandl at gmx.net Sun Jan 22 10:23:24 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 22 Jan 2006 10:23:24 +0100 Subject: [Python-Dev] New Pythondoc by effbot In-Reply-To: References: Message-ID: Guido van Rossum wrote: > On 1/21/06, Georg Brandl wrote: >> What Fredrik hacks together there (http://www.effbot.org/lib) is very >> impressive. I especially like the "permalinks" in this style: >> >> http://effbot.org/lib/os.path.join > > Which (despite having "perma" in its name) evaporates and leaves > behind a link to os.path.html#join. There may be other uses (e.g. marking a certain location in the docs with a "permalink" marker so that one can point the user to /lib/marker. Especially useful for the tutorial and recurring c.l.py questions ;) >> What I would suggest (for any new doc system) is a "split" view: on the left, >> the normal text, on the right, an area with only the headings, functions, >> example and "see also" links (which preferably stays in sight). This way, you >> always keep the outline in view. > > Can you mock that up a bit? I'm somewhat confused about what you're > requesting, and also worried that it would take up too much horizontal > space. (Despite that monitors are wider than tall, horizontal real > estate feels more scarce than vertical, because horizontal scrolling > is such a pain.) Something like http://home.in.tum.de/~brandlg/zipfile.html (It works this way (position: fixed) in most browsers, for IE one would have to apply compatibility hacks.) >> Of course, I wouldn't say no to a user-commenting system, but this would have to >> be moderated. > > Why? If wikipedia can do without moderation (for most pages) then why > couldn't the Python docs? Well, why not... it's surely worth a try. Perhaps using a spam filter like most modern weblogs would suffice. >> What I'm also curious about regarding the current docs, why are optional >> arguments in function declarations not written in Python style? > > I'm assuming you're asking why we use > > foo(a[, b[, c]]) > > instead of > > foo(a, b=1, c=2) > > ? Yep. > I can see several reasons; arguments with default values aren't > necessarily usable keyword arguments (at least not if they > function/method is implemented in C); the default value isn't always > relevant (or is dynamic, or is a huge expression); and square brackets > are the standard way in computer manuals to indicate optional parts of > syntax. Thanks. Georg From walter at livinglogic.de Sun Jan 22 11:51:00 2006 From: walter at livinglogic.de (=?iso-8859-1?Q?Walter_D=F6rwald?=) Date: Sun, 22 Jan 2006 11:51:00 +0100 (CET) Subject: [Python-Dev] New Pythondoc by effbot In-Reply-To: References: Message-ID: <61073.85.74.64.105.1137927060.squirrel@isar.livinglogic.de> Georg Brandl wrote: >> [...] >> Can you mock that up a bit? I'm somewhat confused about what you're requesting, and also worried that it would take up too >> much horizontal space. (Despite that monitors are wider than tall, horizontal real estate feels more scarce than vertical, >> because horizontal scrolling is such a pain.) > > Something like > > http://home.in.tum.de/~brandlg/zipfile.html > > (It works this way (position: fixed) in most browsers, for IE one would have to apply compatibility hacks.) Vertical spacing is IMHO pretty bad on this page. Everything has the same distance from everything else. This makes understanding the structure of the page pretty hard. Bye, Walter D?rwald From g.brandl at gmx.net Sun Jan 22 12:07:16 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 22 Jan 2006 12:07:16 +0100 Subject: [Python-Dev] New Pythondoc by effbot In-Reply-To: <61073.85.74.64.105.1137927060.squirrel@isar.livinglogic.de> References: <61073.85.74.64.105.1137927060.squirrel@isar.livinglogic.de> Message-ID: Walter D?rwald wrote: > Georg Brandl wrote: > >>> [...] >>> Can you mock that up a bit? I'm somewhat confused about what you're requesting, and also worried that it would take up too >>> much horizontal space. (Despite that monitors are wider than tall, horizontal real estate feels more scarce than vertical, >>> because horizontal scrolling is such a pain.) >> >> Something like >> >> http://home.in.tum.de/~brandlg/zipfile.html >> >> (It works this way (position: fixed) in most browsers, for IE one would have to apply compatibility hacks.) > > Vertical spacing is IMHO pretty bad on this page. Everything has the same distance from everything else. This makes > understanding the structure of the page pretty hard. The main content is the original zipfile.html from http://www.effbot.org/lib/zipfile.html. Only the sidebar is my addition. Georg From fredrik at pythonware.com Sun Jan 22 12:58:14 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sun, 22 Jan 2006 12:58:14 +0100 Subject: [Python-Dev] New Pythondoc by effbot References: Message-ID: Guido wrote: > > What Fredrik hacks together there (http://www.effbot.org/lib) is very > > impressive. I especially like the "permalinks" in this style: > > > > http://effbot.org/lib/os.path.join > > Which (despite having "perma" in its name) evaporates and leaves > behind a link to os.path.html#join. "permalink" was george's term; I'm just calling them "link URLs" http://online.effbot.org/2006_01_01_archive.htm#link-doc to make them "real permalinks", we could: - generate individual pages for all targets (on demand), with a "see this in context" - return the same page and use some javascript trickery to emulate anchors or use some other approach that I cannot think of right now however, at this point, I'm more interested in getting the information model to a point where this can be done. > > Of course, I wouldn't say no to a user-commenting system, but this would have to > > be moderated. > > Why? If wikipedia can do without moderation (for most pages) then why > couldn't the Python docs? wikipedia has a large numbers of volunteers that constantly monitor all updates to the site. I'm not we have enough volunteers in python land, and even if we have, I'm not sure site monitoring is the best way to use the volunteer resources. (this is the main motivator behind my documentation and site efforts; we > > What I'm also curious about regarding the current docs, why are optional > > arguments in function declarations not written in Python style? > > I'm assuming you're asking why we use > > foo(a[, b[, c]]) > > instead of > > foo(a, b=1, c=2) > > ? I can see several reasons; arguments with default values aren't > necessarily usable keyword arguments (at least not if they > function/method is implemented in C); the default value isn't always > relevant (or is dynamic, or is a huge expression); and square brackets > are the standard way in computer manuals to indicate optional parts of > syntax. the [] syntax has its issues, including that it makes keyword arguments look like positional arguments, and that descriptions that have lots of optional arguments tend to look like either foo([a[,b[,c[,d[,e[,f]]]]]]) foo([a][,b][,c][,d][,e][,f]) where the first one is ugly and hard to read, and the latter is, strictly speaking, not correct. there are ways to address all these problems with a more pythonic syntax, but I'll have to get back to that later. From fredrik at pythonware.com Sun Jan 22 13:14:14 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sun, 22 Jan 2006 13:14:14 +0100 Subject: [Python-Dev] New Pythondoc by effbot References: <61073.85.74.64.105.1137927060.squirrel@isar.livinglogic.de> Message-ID: Georg Brandl wrote: > > Vertical spacing is IMHO pretty bad on this page. Everything > > has the same distance from everything else. This makes under- > > standing the structure of the page pretty hard. > > The main content is the original zipfile.html from > http://www.effbot.org/lib/zipfile.html. > > Only the sidebar is my addition. the effbot.org/lib material is raw HTML; at this stage, I'm working on getting the translation and the information model right, not polishing the formatting. formatting ideas are of course welcome (especially in CSS form), but comments on the information structure are more useful at this time. From fredrik at pythonware.com Sun Jan 22 13:24:59 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sun, 22 Jan 2006 13:24:59 +0100 Subject: [Python-Dev] New Pythondoc by effbot References: Message-ID: > (this is the main motivator behind my documentation and site efforts; we should learn not to press control-enter when we mean enter. anyway, what I intended to say was that we should work harder on lowering the threshold for drive-by contributions; it should be possible for anyone to notice a problem, add a comment or correction, preview the correction properly rendered, and submit it to a moderation queue, in less than a minute. (I still think moderation is needed before corrections are made visible, but that should be no harder, nor require any more tools, than making the correction in the first place. I'm sure we could get some turbogears/django hackers to build us a nice moderation queue webapp in no time at all...) From fredrik at pythonware.com Sun Jan 22 13:30:41 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sun, 22 Jan 2006 13:30:41 +0100 Subject: [Python-Dev] New Pythondoc by effbot References: Message-ID: Guido van Rossum wrote: > > On this page, 8 of 30 entries have a 'new in' comment. For anyone with no > > interest in the past, these constitute noise. I wonder if for 3.0, the > > timer can be reset and the docs start clean again. To keep them backwards > > compatible, they would also have to be littered with 'changed in 3.0' and > > 'deleted in 3.0' entries. Better, I think, to refer people to the last 2.x > > docs and a separate 2.x/3.0 changes doc. > > (That has nothing to do with Fredrik's efforts of course -- he's just > faithfully copying the annotations from the existing docs.) > > If Python 3.0 is indeed the major incompatible release that we > currently expect it will be, I agree that "new in 3.0" and "new in > 2.x" annotations make little sense, since so much will be "new in 3.0" > (or "deleted in 3.0"). It's probably better to have a separate > document explaining the differences between 2.x and 3.0. absolutely. > For the 2.x line, however (and again post-3.0) I think the "new in X" > annotations are important -- lots of people have to deal with > different Python versions, and it's a lot easier to depend on "new in > X" notations than to manually compare two or more versions of the > docs. (Although you still need to do thorough testing -- like the rest > of the docs, these annotations can't be 100% perfect.) if we move over to a web-oriented (wiki-ish) maintenance model, we probably have to accept that "X" will, sometimes, be a future release. From ncoghlan at gmail.com Sun Jan 22 14:24:42 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 22 Jan 2006 23:24:42 +1000 Subject: [Python-Dev] New Pythondoc by effbot In-Reply-To: References: Message-ID: <43D3879A.8090101@gmail.com> Fredrik Lundh wrote: >> (this is the main motivator behind my documentation and site efforts; we > > should learn not to press control-enter when we mean enter. > > anyway, what I intended to say was that we should work harder on lowering the > threshold for drive-by contributions; it should be possible for anyone to notice a > problem, add a comment or correction, preview the correction properly rendered, > and submit it to a moderation queue, in less than a minute. > > (I still think moderation is needed before corrections are made visible, but that > should be no harder, nor require any more tools, than making the correction in > the first place. I'm sure we could get some turbogears/django hackers to build > us a nice moderation queue webapp in no time at all...) Or go low-tech and have a 'mailto' link that goes to a documentation discussion list ;) Cheers, Nick. P.S. I do like the way this is coming together - particularly if we can find some cross-platform CSS like Georg's to keep the handy list of links on the right. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From guido at python.org Sun Jan 22 16:19:13 2006 From: guido at python.org (Guido van Rossum) Date: Sun, 22 Jan 2006 07:19:13 -0800 Subject: [Python-Dev] New Pythondoc by effbot In-Reply-To: References: Message-ID: On 1/21/06, Georg Brandl wrote: > And, of course, the "new in 2.x" could be formatted less space-consuming, > perhaps to the right of the method name. No, that would give it too much attention. It's getting way too outstanding in effbot's sample, probably because he didn't implement the formatter for @since yet. It should simply be an addition to the text paragraph that precedes it, like in the regular docs. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sun Jan 22 17:16:30 2006 From: guido at python.org (Guido van Rossum) Date: Sun, 22 Jan 2006 08:16:30 -0800 Subject: [Python-Dev] New Pythondoc by effbot In-Reply-To: References: Message-ID: (This is a bulk reply to several messages in this thread from Georg and Fredrik.) [Georg] > >> http://effbot.org/lib/os.path.join [Guido] > > Which (despite having "perma" in its name) evaporates and leaves > > behind a link to os.path.html#join. [Georg] > There may be other uses (e.g. marking a certain location in the docs with > a "permalink" marker so that one can point the user to /lib/marker. > Especially useful for the tutorial and recurring c.l.py questions ;) Rather than allowing users to add new markers I'd like the document to be sprinked generously with pregenerated markers to which one can easily linked. I've seen sample docs (not for Python) with such a feature. [Fredrik, in a different message] > "permalink" was george's term; I'm just calling them "link URLs" > > http://online.effbot.org/2006_01_01_archive.htm#link-doc > > to make them "real permalinks", we could: > > - generate individual pages for all targets (on demand), with > a "see this in context" > - return the same page and use some javascript trickery to > emulate anchors > > or use some other approach that I cannot think of right now That sounds overkill. But if you could drop the redundant ".html", the links in the source could look like like .../os.path#join which is just fine with me (and solves the problem of what part is the module, on which you spend altogether much time explaining how to compute in the current version :-). > however, at this point, I'm more interested in getting the information > model to a point where this can be done. Understood. What's still missing from the info model? It looks pretty complete to me from a quick scan. [Georg again] > >> What I would suggest (for any new doc system) is a "split" view: on the left, > >> the normal text, on the right, an area with only the headings, functions, > >> example and "see also" links (which preferably stays in sight). This way, you > >> always keep the outline in view. [Guido] > > Can you mock that up a bit? I'm somewhat confused about what you're > > requesting, and also worried that it would take up too much horizontal > > space. (Despite that monitors are wider than tall, horizontal real > > estate feels more scarce than vertical, because horizontal scrolling > > is such a pain.) [Georg] > Something like > > http://home.in.tum.de/~brandlg/zipfile.html > > (It works this way (position: fixed) in most browsers, for IE one would have to > apply compatibility hacks.) I believe there's a CSS trick (most often used for images) that can makes the summary window "float" to the right so that below it the main text resumes the full breadth of the window. If you can pull that off I think this is a good addition! > >> Of course, I wouldn't say no to a user-commenting system, but this would have to > >> be moderated. > > > > Why? If wikipedia can do without moderation (for most pages) then why > > couldn't the Python docs? > > Well, why not... it's surely worth a try. Perhaps using a spam filter like most > modern weblogs would suffice. Have you studied wikipedia's approach? It's multi-layered and worth learning from (start with their FAQ on editing). (And by the way, I am *not* advocating writing the docs as one big wikipedia -- only the user commentary.) [Fredrik again] > the [] syntax has its issues, including that it makes keyword arguments > look like positional arguments, (I think that is actually Python's fault for not distinguishing them in the function definition. :-) > and that descriptions that have lots of > optional arguments tend to look like either > > foo([a[,b[,c[,d[,e[,f]]]]]]) > > foo([a][,b][,c][,d][,e][,f]) > > where the first one is ugly and hard to read, and the latter is, strictly > speaking, not correct. Or (my favorite, but still wrong) foo([a, b, c, d, e, f]) > there are ways to address all these problems with a more pythonic syntax, > but I'll have to get back to that later. Please do. [Fredrik, in yet another message] > anyway, what I intended to say was that we should work harder on lowering the > threshold for drive-by contributions; it should be possible for anyone to notice a > problem, add a comment or correction, preview the correction properly rendered, > and submit it to a moderation queue, in less than a minute. Given that it will take significantly longer to *process* the queue I don't necessarily want to make the contribution process *too* easy -- we need to balance the quality of the contributions with the ease to make them. For many simple corrections (spelling, grammar, inversions, some factual mistakes) the need to preview the correction properly rendered is quite unnecessary; the important part is being able to *point* at any word or words in the text and attach a comment. > (I still think moderation is needed before corrections are made visible, but that > should be no harder, nor require any more tools, than making the correction in > the first place. I'm sure we could get some turbogears/django hackers to build > us a nice moderation queue webapp in no time at all...) Agreed. But as I said above, I think there should also be a way to simply add comments, without moderation (but with wikipedia-style editing / spam-removal / moderation after-the-fact). And, probably, if we don't get to the moderation queue etc., the comments wiki could serve as a moderation queue as well. Being public, it would also reduce the submission of duplicate corrections (especially when the moderator is out of town for a while). [Fredrik, later] > if we move over to a web-oriented (wiki-ish) maintenance model, we > probably have to accept that "X" will, sometimes, be a future release. This makes me worry that you're thinking of abandoning having separate docs per version. That would be a shame; I think the docs ought to be maintained well enough to be authorative for a particular version (modulo bugs in the docs). Perhaps we could add links that let you navigate between versions quickly, and hopefully we can somehow (optionally?) merge the wiki-style user comments across versions. (Although I can also see the point of starting afresh for each release -- at some point the sheer amount of (possibly outdated) user comments reduces their usefulness. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From facundobatista at gmail.com Sun Jan 22 17:32:39 2006 From: facundobatista at gmail.com (Facundo Batista) Date: Sun, 22 Jan 2006 13:32:39 -0300 Subject: [Python-Dev] New Pythondoc by effbot In-Reply-To: References: Message-ID: 2006/1/22, Georg Brandl : > Guido van Rossum wrote: > > ... > > Why? If wikipedia can do without moderation (for most pages) then why > > couldn't the Python docs? > > Well, why not... it's surely worth a try. Perhaps using a spam filter like most > modern weblogs would suffice. I can see three levels of permissiveness. Everybody (with or without registration?) will be able to post whatever, but the annotation... - will be posted always - will go through an antispam - will go through a posting-revision-comitee (that is, a human antispam, ;)). I don't like the first one, I'm +1 with the second and +0 with the third. In the last two, specially in the third one, you'll need human workforce. I'm stepping forward for that, if you need it. . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From g.brandl at gmx.net Sun Jan 22 17:34:49 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 22 Jan 2006 17:34:49 +0100 Subject: [Python-Dev] New Pythondoc by effbot In-Reply-To: References: Message-ID: Guido van Rossum wrote: >> > Which (despite having "perma" in its name) evaporates and leaves >> > behind a link to os.path.html#join. > >> There may be other uses (e.g. marking a certain location in the docs with >> a "permalink" marker so that one can point the user to /lib/marker. >> Especially useful for the tutorial and recurring c.l.py questions ;) > > Rather than allowing users to add new markers I'd like the document to > be sprinked generously with pregenerated markers to which one can > easily linked. I've seen sample docs (not for Python) with such a > feature. Exactly what I had in mind. And we can add markers by request of the c.l.py "support" regulars. >> Something like >> >> http://home.in.tum.de/~brandlg/zipfile.html >> >> (It works this way (position: fixed) in most browsers, for IE one would have to >> apply compatibility hacks.) > > I believe there's a CSS trick (most often used for images) that can > makes the summary window "float" to the right so that below it the > main text resumes the full breadth of the window. If you can pull that > off I think this is a good addition! Well, floating the summary is no problem, but it wouldn't stay at the side if you scroll down then. I updated the mockup a bit: http://home.in.tum.de/~brandlg/zipfile.html An alternative with floating summary: http://home.in.tum.de/~brandlg/zipfile-float.html >> >> Of course, I wouldn't say no to a user-commenting system, but this would have to >> >> be moderated. >> > >> > Why? If wikipedia can do without moderation (for most pages) then why >> > couldn't the Python docs? >> >> Well, why not... it's surely worth a try. Perhaps using a spam filter like most >> modern weblogs would suffice. > > Have you studied wikipedia's approach? It's multi-layered and worth > learning from (start with their FAQ on editing). > > (And by the way, I am *not* advocating writing the docs as one big > wikipedia -- only the user commentary.) Agreed. Something like a "discussion page" could be fruitful. >> (I still think moderation is needed before corrections are made visible, but that >> should be no harder, nor require any more tools, than making the correction in >> the first place. I'm sure we could get some turbogears/django hackers to build >> us a nice moderation queue webapp in no time at all...) > > Agreed. But as I said above, I think there should also be a way to > simply add comments, without moderation (but with wikipedia-style > editing / spam-removal / moderation after-the-fact). And, probably, if > we don't get to the moderation queue etc., the comments wiki could > serve as a moderation queue as well. Being public, it would also > reduce the submission of duplicate corrections (especially when the > moderator is out of town for a while). +1. > [Fredrik, later] >> if we move over to a web-oriented (wiki-ish) maintenance model, we >> probably have to accept that "X" will, sometimes, be a future release. > > This makes me worry that you're thinking of abandoning having separate > docs per version. That would be a shame; I think the docs ought to be > maintained well enough to be authorative for a particular version > (modulo bugs in the docs). Perhaps we could add links that let you > navigate between versions quickly, and hopefully we can somehow > (optionally?) merge the wiki-style user comments across versions. +1. > (Although I can also see the point of starting afresh for each release > -- at some point the sheer amount of (possibly outdated) user comments > reduces their usefulness. That would have to be decided in situ. regards, Georg From tim.peters at gmail.com Sun Jan 22 19:16:56 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 22 Jan 2006 13:16:56 -0500 Subject: [Python-Dev] New Pythondoc by effbot In-Reply-To: References: <20060122044131.GA18077@panix.com> Message-ID: <1f7befae0601221016t24d1bb5dt1cde577cd2b75a82@mail.gmail.com> [Aahz] >> Aside to Georg: your messages are all getting /dev/null'd because you use >> "spam" in your From: line. I get too much spam to change that, and I'll >> bet other people use that heuristic. Please change your From: line for >> this mailing list. [Georg Brandl] > I don't quite understand that. Why would a spam email have "spam" in > the From? In general it wouldn't. Aahz didn't tell us everything, or he's simply not making sense. Because I still like ;-) the SpamBayes project, I don't do anything to try to stop receiving spam, and have several email addresses that have been visible for years. Of the 4456 spam I received via Outlook in the last 30 days, onlly 3 had "spam" in the From header: From: alison at charlotte.dontspam.wsrcc.com From: ".?.??? .?????" From: spam It was much more common among ham, including email from you, from the spambayes mailing lists, from SORBS, .... If: The `From:` header contains "spam" as a substring, case-insensitively. were a feature SpamBayes scored, it would be a strong ham clue in my training database. > For me, the "-nospam" suffix works relatively good to avoid spam, as most > harvesting programs will think this is a false address. > > Of course, that may be more useful in Usenet, not in mailing lists. It's OK by me if you keep using it. From tim at pollenation.net Sun Jan 22 20:10:04 2006 From: tim at pollenation.net (Tim Parkin) Date: Sun, 22 Jan 2006 19:10:04 +0000 Subject: [Python-Dev] New Pythondoc by effbot In-Reply-To: References: Message-ID: <43D3D88C.6070001@pollenation.net> Guido van Rossum wrote: > I believe there's a CSS trick (most often used for images) that can > makes the summary window "float" to the right so that below it the > main text resumes the full breadth of the window. If you can pull that > off I think this is a good addition! > > Something like this... http://beta.python.org/download/releases/2.4.1/ Tim From tim at pollenation.net Sun Jan 22 20:18:41 2006 From: tim at pollenation.net (Tim Parkin) Date: Sun, 22 Jan 2006 19:18:41 +0000 Subject: [Python-Dev] New Pythondoc by effbot In-Reply-To: <43D3D88C.6070001@pollenation.net> References: <43D3D88C.6070001@pollenation.net> Message-ID: <43D3DA91.9070708@pollenation.net> Tim Parkin wrote: > Guido van Rossum wrote: > >>I believe there's a CSS trick (most often used for images) that can >>makes the summary window "float" to the right so that below it the >>main text resumes the full breadth of the window. If you can pull that >>off I think this is a good addition! >> >> > > > Something like this... > > http://beta.python.org/download/releases/2.4.1/ > It would also be simple to add an 'alternate stylesheet' that can hide the left hand navigation. A small link could be added to switch styles (which would include javascript) but all good browsers allow you to choose which stylesheet you wish to use without the aid of javascript (which I beleive should only be used for 'optional enhancements' to a website). The layout also positions the main navigation at the bottom of the html source so the first items you see using a 'text browser' are the breadcrumb and document level navigation. Tim Parkin From g.brandl at gmx.net Sun Jan 22 20:40:58 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 22 Jan 2006 20:40:58 +0100 Subject: [Python-Dev] New Pythondoc by effbot In-Reply-To: References: Message-ID: Facundo Batista wrote: > 2006/1/22, Georg Brandl : > >> Guido van Rossum wrote: >> > ... >> > Why? If wikipedia can do without moderation (for most pages) then why >> > couldn't the Python docs? >> >> Well, why not... it's surely worth a try. Perhaps using a spam filter like most >> modern weblogs would suffice. > > I can see three levels of permissiveness. Everybody (with or without > registration?) will be able to post whatever, but the annotation... > > - will be posted always > - will go through an antispam > - will go through a posting-revision-comitee (that is, a human antispam, ;)). > > I don't like the first one, I'm +1 with the second and +0 with the third. > > In the last two, specially in the third one, you'll need human > workforce. I'm stepping forward for that, if you need it. Metoo, of course. Georg From fredrik at pythonware.com Sun Jan 22 20:50:31 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sun, 22 Jan 2006 20:50:31 +0100 Subject: [Python-Dev] New Pythondoc by effbot References: Message-ID: Guido wrote: > Have you studied wikipedia's approach? It's multi-layered and worth > learning from (start with their FAQ on editing). > > (And by the way, I am *not* advocating writing the docs as one big > wikipedia -- only the user commentary.) to clarify, I'm advocating maintaining the docs via a restricted wiki-like system, and using a blog-style discussion board or a wiki to collect non- specific user commentary. but it's the combination that really interests me: the ability for users to propose specific corrections (as patches, at least under the cover), and for volunteers to apply such patches with a minimum of effort. more later. From g.brandl at gmx.net Sun Jan 22 20:50:41 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 22 Jan 2006 20:50:41 +0100 Subject: [Python-Dev] New Pythondoc by effbot In-Reply-To: <43D3DA91.9070708@pollenation.net> References: <43D3D88C.6070001@pollenation.net> <43D3DA91.9070708@pollenation.net> Message-ID: Tim Parkin wrote: > Tim Parkin wrote: >> Guido van Rossum wrote: >> >>>I believe there's a CSS trick (most often used for images) that can >>>makes the summary window "float" to the right so that below it the >>>main text resumes the full breadth of the window. If you can pull that >>>off I think this is a good addition! >>> >> Something like this... >> >> http://beta.python.org/download/releases/2.4.1/ That's an ordinary float, I presume? > It would also be simple to add an 'alternate stylesheet' that can hide > the left hand navigation. A small link could be added to switch styles > (which would include javascript) but all good browsers allow you to > choose which stylesheet you wish to use without the aid of javascript > (which I beleive should only be used for 'optional enhancements' to a > website). Good idea! Dito for options like "enlarge text" or such. Georg From tim at pollenation.net Sun Jan 22 23:05:17 2006 From: tim at pollenation.net (Tim Parkin) Date: Sun, 22 Jan 2006 22:05:17 +0000 Subject: [Python-Dev] New Pythondoc by effbot In-Reply-To: References: <43D3D88C.6070001@pollenation.net> <43D3DA91.9070708@pollenation.net> Message-ID: <43D4019D.8020707@pollenation.net> Georg Brandl wrote: >Tim Parkin wrote: > >>>Something like this... >>> >>>http://beta.python.org/download/releases/2.4.1/ >>> >>> > >That's an ordinary float, I presume? > > indeed it is. > > >>It would also be simple to add an 'alternate stylesheet' that can hide >>the left hand navigation. A small link could be added to switch styles >>(which would include javascript) but all good browsers allow you to >>choose which stylesheet you wish to use without the aid of javascript >>(which I beleive should only be used for 'optional enhancements' to a >>website). >> >> > >Good idea! Dito for options like "enlarge text" or such. > > It already has a 'large text' alternate style sheet, however it needs a couple of tweaks since I've made some big style sheet changes. The style sheet switching is well documented here with example code etc... http://www.brothercake.com/site/resources/scripts/iotbs/ Tim From brett at python.org Mon Jan 23 01:48:42 2006 From: brett at python.org (Brett Cannon) Date: Sun, 22 Jan 2006 16:48:42 -0800 Subject: [Python-Dev] New Pythondoc by effbot In-Reply-To: References: Message-ID: > [Fredrik, later] > > if we move over to a web-oriented (wiki-ish) maintenance model, we > > probably have to accept that "X" will, sometimes, be a future release. > > This makes me worry that you're thinking of abandoning having separate > docs per version. That would be a shame; I think the docs ought to be > maintained well enough to be authorative for a particular version > (modulo bugs in the docs). Perhaps we could add links that let you > navigate between versions quickly, and hopefully we can somehow > (optionally?) merge the wiki-style user comments across versions. > (Although I can also see the point of starting afresh for each release > -- at some point the sheer amount of (possibly outdated) user comments > reduces their usefulness. It would probably be best to clear out the comments for each minor release. Otherwise they will just organically grow. Problem is that useful examples might disappear. This might suggest that having possible example usage tagged on to a method or such could be saved and, by default, made folded (ala folding in Ian Bicking's Python doc comment system: http://blog.ianbicking.org/python-doc-comments.html). These examples could also be verified using something like doctest so that invalidated examples do not persist or submitted. And to /F, kudos from me. I have been randomly thinking about it and I understand your desire for semantic markup now. Hopefully something can get hammered out so that at least the Python 3 docs can premiere having been developed on by the whole community. -Brett From bjourne at gmail.com Mon Jan 23 10:32:43 2006 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Mon, 23 Jan 2006 10:32:43 +0100 Subject: [Python-Dev] New Pythondoc by effbot In-Reply-To: References: Message-ID: <740c3aec0601230132n2ff31a22n1cd49f9517330cb1@mail.gmail.com> > > Have you studied wikipedia's approach? It's multi-layered and worth > > learning from (start with their FAQ on editing). > > > > (And by the way, I am *not* advocating writing the docs as one big > > wikipedia -- only the user commentary.) > > to clarify, I'm advocating maintaining the docs via a restricted wiki-like > system, and using a blog-style discussion board or a wiki to collect non- > specific user commentary. Why does it have to be "wiki-like"? Why can't it be a wiki? MediaWiki seem to work pretty well for a lot of software projects that have put their documentation in a wiki. Talk pages for commentary and primary pages for reviewed content. -- mvh Bj?rn From fredrik at pythonware.com Mon Jan 23 16:42:24 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Mon, 23 Jan 2006 16:42:24 +0100 Subject: [Python-Dev] New Pythondoc by effbot References: <740c3aec0601230132n2ff31a22n1cd49f9517330cb1@mail.gmail.com> Message-ID: BJörn Lindqvist wrote: > > > Have you studied wikipedia's approach? It's multi-layered and worth > > > learning from (start with their FAQ on editing). > > > > > > (And by the way, I am *not* advocating writing the docs as one big > > > wikipedia -- only the user commentary.) > > > > to clarify, I'm advocating maintaining the docs via a restricted wiki-like > > system, and using a blog-style discussion board or a wiki to collect non- > > specific user commentary. > > Why does it have to be "wiki-like"? Why can't it be a wiki? MediaWiki > seem to work pretty well for a lot of software projects that have put > their documentation in a wiki. Talk pages for commentary and primary > pages for reviewed content. semantics, semantics, semantics (see earlier posts on this topic). but sure, a wiki engine with support for custom source syntax (to support python-oriented semantic markup) and section editing could work. (moinmoin has the former, but not the latter, afaik). From nnorwitz at gmail.com Mon Jan 23 08:01:36 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 22 Jan 2006 23:01:36 -0800 Subject: [Python-Dev] stabilizing builds Message-ID: I would really like to stabilize a bunch of the current rough edges in 2.5. We are likely to have enough instability with the PEP 353 changes, so it would be better to fix some of these issues sooner rather than later. There are a bunch of tests that are not stable. It would really help to get people knowledgeable about a particular subdomain to provide input into bugs/patches and produce patches too! The areas that are known to have problems are listed here: http://wiki.python.org/moin/BuildBot It would be nice to clean those issues up. Does anyone have some time to spend to resolve the issues with Berkeley DB? That's the one that annoys me most right now. I have a patch that fixes at least one of the problems, but no idea if it's the correct solution or not. http://python.org/sf/1407992 n -- PS. In case you are interested in all the problems: * BSD DB 4.1 and 3.2 fail * test_pty is brittle on solaris 10, sometimes it works, sometimes not * getaddrinfo() configure check sometimes fails on OS X 10.3 From nnorwitz at gmail.com Mon Jan 23 09:36:36 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 23 Jan 2006 00:36:36 -0800 Subject: [Python-Dev] building a module catalogue with buildbot In-Reply-To: <20060111223039.GC6642@activestate.com> References: <20060111223039.GC6642@activestate.com> Message-ID: On 1/11/06, Trent Mick wrote: > > > Does that make sense? We would just need /f's script in SVN. > > > > in python/Tools/something or sandbox/something ? > > python/Doc/tools/something? Fredrik were you still working on that? I can make the changes to the bb master. I thought Trent's suggested placement was good. n From thomas at xs4all.net Mon Jan 23 22:22:10 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Mon, 23 Jan 2006 22:22:10 +0100 Subject: [Python-Dev] stabilizing builds In-Reply-To: References: Message-ID: <20060123212210.GA18916@xs4all.nl> On Sun, Jan 22, 2006 at 11:01:36PM -0800, Neal Norwitz wrote: > * test_pty is brittle on solaris 10, sometimes it works, sometimes not Do we have a Solaris 10 box to test on? I think I wrote most of test_pty, and I can see if I can pin down the problem, but I don't have a Solaris 10 box myself. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From kbk at shore.net Tue Jan 24 03:01:08 2006 From: kbk at shore.net (Kurt B. Kaiser) Date: Mon, 23 Jan 2006 21:01:08 -0500 (EST) Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200601240201.k0O218GE007855@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 391 open ( +7) / 3028 closed (+12) / 3419 total (+19) Bugs : 905 open ( -4) / 5519 closed (+19) / 6424 total (+15) RFE : 207 open ( -1) / 197 closed ( +1) / 404 total ( +0) New / Reopened Patches ______________________ Patch for #1371247 (2006-01-14) CLOSED http://python.org/sf/1406159 opened by Peter van Kampen Doc patch for bug #1396471 (2006-01-16) CLOSED http://python.org/sf/1407021 opened by Peter van Kampen anonymous mmap (2006-01-16) http://python.org/sf/1407135 opened by Ralf Schmitt ParenMatch: workaround for misinterpreting of closing parens (2006-01-16) http://python.org/sf/1407280 opened by Tal Einat Patches Closed ______________ AF_NETLINK sockets basic support (2005-01-15) http://python.org/sf/1103116 closed by loewis Further .vcproj cleanups (2006-01-03) http://python.org/sf/1396093 closed by loewis bind() for netlink sockets (2005-12-15) http://python.org/sf/1381398 closed by loewis Patch for #1371247 (2006-01-14) http://python.org/sf/1406159 closed by birkenfeld Doc patch for bug #1396471 (2006-01-16) http://python.org/sf/1407021 closed by birkenfeld New / Reopened Bugs ___________________ bz2module with no long long type support (2006-01-16) CLOSED http://python.org/sf/1407069 opened by kbob_ru urlparse does not know about sftp: urls (2006-01-17) CLOSED http://python.org/sf/1407902 opened by Stuart Bishop Bugs Closed ___________ segfault when using mmap(-1,...) [+PATCH] (2006-01-10) http://python.org/sf/1402308 closed by nnorwitz 2.3.5 RPM provides incompatibility (2006-01-11) http://python.org/sf/1403225 closed by jafo locale.windows_locale (2005-12-01) http://python.org/sf/1371247 closed by birkenfeld file.tell() returns illegal value on Windows (2006-01-04) http://python.org/sf/1396471 closed by birkenfeld bz2module with no long long type support (2006-01-15) http://python.org/sf/1407069 closed by nnorwitz TimedRotatingFileHandler at midnight rolls over at 01:00 (2006-01-03) http://python.org/sf/1396030 closed by vsajip TimedRotatingFileHandler does not recover from open error (2006-01-03) http://python.org/sf/1396040 closed by vsajip urllib2 should be more robust for sloppy proxy URLs (2004-02-22) http://python.org/sf/902075 closed by birkenfeld urlparse does not know about sftp: urls (2006-01-17) http://python.org/sf/1407902 closed by birkenfeld in email can't get attachments' filenames using get_filename (2006-01-11) http://python.org/sf/1403349 closed by bwarsaw RFE Closed __________ python.org - add RFE tracker (2006-01-05) http://python.org/sf/1397806 closed by kbk From kbk at shore.net Tue Jan 24 03:30:08 2006 From: kbk at shore.net (Kurt B. Kaiser) Date: Mon, 23 Jan 2006 21:30:08 -0500 (EST) Subject: [Python-Dev] Weekly Python Patch/Bug Summary (Revised) Message-ID: <200601240230.k0O2U8GE022475@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 391 open ( +7) / 3028 closed (+12) / 3419 total (+19) Bugs : 906 open ( -3) / 5519 closed (+19) / 6425 total (+16) RFE : 207 open ( -1) / 197 closed ( +1) / 404 total ( +0) New / Reopened Patches ______________________ Patch for #1371247 (2006-01-14) CLOSED http://python.org/sf/1406159 opened by Peter van Kampen Doc patch for bug #1396471 (2006-01-16) CLOSED http://python.org/sf/1407021 opened by Peter van Kampen anonymous mmap (2006-01-16) http://python.org/sf/1407135 opened by Ralf Schmitt ParenMatch: workaround for misinterpreting of closing parens (2006-01-16) http://python.org/sf/1407280 opened by Tal Einat fix bsddb test associate problems w/bsddb 4.1 (2006-01-16) http://python.org/sf/1407992 opened by Neal Norwitz Corrupt Berkeley DB using Modify bsddb.dbtables (2006-01-17) http://python.org/sf/1408584 opened by jross use computed goto's in ceval loop (2006-01-18) http://python.org/sf/1408710 opened by Simon David Burton Add foldr and foldl to functional module (2006-01-19) CLOSED http://python.org/sf/1410119 opened by Collin Winter Add 'surgical editing' to ConfigParser (2006-01-21) http://python.org/sf/1410680 opened by Tony Meyer Add notes to the manual about `is` and methods (2006-01-20) http://python.org/sf/1410739 opened by Collin Winter Doc patch for classmethod and staticmethod (2006-01-20) CLOSED http://python.org/sf/1410783 opened by Collin Winter Remove mention of DOS from os.py's docstring (2006-01-20) CLOSED http://python.org/sf/1410998 opened by Hamish Lawson Add DarwinPorts directories to setup.py (2006-01-21) CLOSED http://python.org/sf/1411588 opened by Barry A. Warsaw More easily extensible logging module (2006-01-22) http://python.org/sf/1412054 opened by Daniele Varrazzo Compile under mingw properly (2006-01-23) http://python.org/sf/1412448 opened by Alexandre Gir?o Bezerra de Mene Fill out the functional module (2006-01-22) http://python.org/sf/1412451 opened by Collin Winter Proper locking with asynchronous callbacks. (2006-01-23) http://python.org/sf/1412632 opened by uiltje zipfile: use correct system type on unixy systems (2006-01-23) http://python.org/sf/1412872 opened by Ronald Oussoren PyThreadState_Delete doesnt' clean autoTLSkey (2006-01-23) http://python.org/sf/1413181 opened by Gabriel Becedillas Patches Closed ______________ AF_NETLINK sockets basic support (2005-01-15) http://python.org/sf/1103116 closed by loewis Further .vcproj cleanups (2006-01-03) http://python.org/sf/1396093 closed by loewis bind() for netlink sockets (2005-12-15) http://python.org/sf/1381398 closed by loewis Patch for #1371247 (2006-01-14) http://python.org/sf/1406159 closed by birkenfeld Doc patch for bug #1396471 (2006-01-16) http://python.org/sf/1407021 closed by birkenfeld Add foldr and foldl to functional module (2006-01-19) http://python.org/sf/1410119 closed by collinwinter Need warning that `dl` module can cause segfaults/crashes (2006-01-11) http://python.org/sf/1402224 closed by birkenfeld Doc patch for classmethod and staticmethod (2006-01-20) http://python.org/sf/1410783 closed by birkenfeld Make unittest.TestCase easier to subclass (2005-12-22) http://python.org/sf/1388073 closed by birkenfeld Remove mention of DOS from os.py's docstring (2006-01-20) http://python.org/sf/1410998 closed by birkenfeld fix UnixBrowswer failure when no browser running (2005-12-03) http://python.org/sf/1372125 closed by birkenfeld Add DarwinPorts directories to setup.py (2006-01-21) http://python.org/sf/1411588 closed by bwarsaw New / Reopened Bugs ___________________ bz2module with no long long type support (2006-01-16) CLOSED http://python.org/sf/1407069 opened by kbob_ru urlparse does not know about sftp: urls (2006-01-17) CLOSED http://python.org/sf/1407902 opened by Stuart Bishop email.Message should supress warning from uu.decode (2006-01-18) http://python.org/sf/1409403 opened by Mark Sapiro frame->f_lasti not always correct (2006-01-18) http://python.org/sf/1409443 opened by John Ehresman email.Message.set_payload followed by bad result get_payload (2006-01-18) http://python.org/sf/1409455 opened by Mark Sapiro email.Utils.parseaddr() gives arcane result (2006-01-18) http://python.org/sf/1409460 opened by Mark Sapiro email.Charset.py CODEC_MAP no longer requires 'japanese' (2006-01-19) http://python.org/sf/1409538 opened by Tokio Kikuchi urllib2.urlopen() hangs due to use of socket._fileobject? (2006-01-20) http://python.org/sf/1411097 opened by John J Lee "print statement" in libref index broken (2006-01-21) http://python.org/sf/1411674 opened by Skip Montanaro XML.sax.saxutils.escape -- always escapes <, >, &, (2006-01-21) http://python.org/sf/1411695 opened by Mark Sandler minor doc issues in os (2006-01-22) CLOSED http://python.org/sf/1412227 opened by tiissa locale.format gives wrong exception on some erroneous input (2006-01-23) http://python.org/sf/1412580 opened by Tim Diggins __self - Watcom compiler reserved word (2006-01-23) http://python.org/sf/1412837 opened by kbob_ru function usage not in #ifdef WITH_THREAD #endif (2006-01-23) CLOSED http://python.org/sf/1412845 opened by kbob_ru bsddb: segfault on db.associate call with Txn and large data (2006-01-23) http://python.org/sf/1413192 opened by Alex Roitman Bugs Closed ___________ segfault when using mmap(-1,...) [+PATCH] (2006-01-10) http://python.org/sf/1402308 closed by nnorwitz 2.3.5 RPM provides incompatibility (2006-01-11) http://python.org/sf/1403225 closed by jafo locale.windows_locale (2005-12-01) http://python.org/sf/1371247 closed by birkenfeld file.tell() returns illegal value on Windows (2006-01-04) http://python.org/sf/1396471 closed by birkenfeld bz2module with no long long type support (2006-01-15) http://python.org/sf/1407069 closed by nnorwitz TimedRotatingFileHandler at midnight rolls over at 01:00 (2006-01-03) http://python.org/sf/1396030 closed by vsajip TimedRotatingFileHandler does not recover from open error (2006-01-03) http://python.org/sf/1396040 closed by vsajip urllib2 should be more robust for sloppy proxy URLs (2004-02-22) http://python.org/sf/902075 closed by birkenfeld urlparse does not know about sftp: urls (2006-01-17) http://python.org/sf/1407902 closed by birkenfeld in email can't get attachments' filenames using get_filename (2006-01-11) http://python.org/sf/1403349 closed by bwarsaw email.Generators does not separates headers with "\r\n" (2005-11-05) http://python.org/sf/1349106 closed by bwarsaw FeedParser does not comply with RFC2822 (2005-11-03) http://python.org/sf/1347874 closed by bwarsaw Odd warning behaviors (2004-02-03) http://python.org/sf/890010 closed by birkenfeld Doc error on super(cls,self) (2004-06-16) http://python.org/sf/973579 closed by birkenfeld CVS webbrowser.py (1.40) bugs (2005-10-27) http://python.org/sf/1338995 closed by birkenfeld urllib2.HTTPBasicAuthHandler fails on non-default port (2005-12-12) http://python.org/sf/1378679 closed by birkenfeld minor doc issues in os (2006-01-22) http://python.org/sf/1412227 closed by birkenfeld function usage not in #ifdef WITH_THREAD #endif (2006-01-23) http://python.org/sf/1412845 closed by mwh RFE Closed __________ python.org - add RFE tracker (2006-01-05) http://python.org/sf/1397806 closed by kbk From fredrik at pythonware.com Tue Jan 24 10:09:12 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue, 24 Jan 2006 10:09:12 +0100 Subject: [Python-Dev] building a module catalogue with buildbot References: <20060111223039.GC6642@activestate.com> Message-ID: Neal Norwitz wrote: > > > > Does that make sense? We would just need /f's script in SVN. > > > > > > in python/Tools/something or sandbox/something ? > > > > python/Doc/tools/something? > > Fredrik were you still working on that? I can make the changes to the > bb master. I thought Trent's suggested placement was good. iirc, the script needed some minor tweaking (using os.path.splitext to test for the module.so extension isn't a good idea), and I don't recall if I've fixed that or not... (probably not, since I never checked it in). I'll take a look asap. From fredrik at pythonware.com Tue Jan 24 12:08:51 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue, 24 Jan 2006 12:08:51 +0100 Subject: [Python-Dev] New Pythondoc by effbot References: Message-ID: Brett Cannon wrote: > And to /F, kudos from me. I have been randomly thinking about it and > I understand your desire for semantic markup now. thanks. > Hopefully something can get hammered out so that at least the Python > 3 docs can premiere having been developed on by the whole community. why wait for Python 3 ? what's the current release plan for Python 2.5, btw? I cannot find a relevant PEP, and the "what's new" says "late 2005": http://www.python.org/dev/doc/devel/whatsnew/contents.html From thomas at xs4all.net Tue Jan 24 12:15:29 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Tue, 24 Jan 2006 12:15:29 +0100 Subject: [Python-Dev] stabilizing builds In-Reply-To: References: Message-ID: <20060124111528.GB18916@xs4all.nl> On Sun, Jan 22, 2006 at 11:01:36PM -0800, Neal Norwitz wrote: > * test_pty is brittle on solaris 10, sometimes it works, sometimes not FWIW, it's brittle on Solaris 9, too, and the SF compilefarm has two of those. I don't know if it's the same problem, but on Solaris 9, the slave part of the tty/pty pair sometimes isn't a TTY (according to os.isatty.) The buildbot's log doesn't list the solaris 10 test_pty failure though, just the test_logging failure. It looks like a timing issue; the first run succeeds, all subsequent runs fail, for a while, anyway. I'll do some googling and browsing other tty/pty-using code to see if there's anything we're not doing we should be doing, but it looks like a platform bug that we can't fix... Not without re-implementing os.isatty anyway ;P -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From abo at minkirri.apana.org.au Tue Jan 24 14:05:34 2006 From: abo at minkirri.apana.org.au (Donovan Baarda) Date: Tue, 24 Jan 2006 13:05:34 +0000 Subject: [Python-Dev] New Pythondoc by effbot In-Reply-To: References: Message-ID: <1138107934.32075.47.camel@warna.dub.corp.google.com> On Sat, 2006-01-21 at 19:15 -0500, Terry Reedy wrote: > >> http://effbot.org/lib/os.path.join > > On this page, 8 of 30 entries have a 'new in' comment. For anyone with no > interest in the past, these constitute noise. I wonder if for 3.0, the Even the past is relative... I find the "new in" doco absolutely essential, because my "present" depends on what system I'm on, and some of them are stuck in a serious time-warp. I do not have a time-machine big enough to transport whole companies. > timer can be reset and the docs start clean again. To keep them backwards > compatible, they would also have to be littered with 'changed in 3.0' and > 'deleted in 3.0' entries. Better, I think, to refer people to the last 2.x > docs and a separate 2.x/3.0 changes doc. I also find "changed in" essential, but I don't mind not having "deleted in"... it encourages developers stuck in those time-warps to avoid features that get deleted in the future :-) -- Donovan Baarda http://minkirri.apana.org.au/~abo/ From fredrik at pythonware.com Tue Jan 24 17:20:26 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue, 24 Jan 2006 17:20:26 +0100 Subject: [Python-Dev] building a module catalogue with buildbot References: <20060111223039.GC6642@activestate.com> Message-ID: I wrote: > Neal Norwitz wrote: > > > > > > Does that make sense? We would just need /f's script in SVN. > > > > > > > > in python/Tools/something or sandbox/something ? > > > > > > python/Doc/tools/something? > > > > Fredrik were you still working on that? I can make the changes to the > > bb master. I thought Trent's suggested placement was good. > > iirc, the script needed some minor tweaking (using os.path.splitext to > test for the module.so extension isn't a good idea), and I don't recall > if I've fixed that or not... > > (probably not, since I never checked it in). > > I'll take a look asap. alright, I just checked in a Doc/tools/listmodules.py which attempts to produce a sorted list of all modules available in a given python build. by default, it prints the list to stdout, which should be good enough for a "catalog" buildbot step. From skip at pobox.com Tue Jan 24 19:27:43 2006 From: skip at pobox.com (skip at pobox.com) Date: Tue, 24 Jan 2006 12:27:43 -0600 Subject: [Python-Dev] When will regex really go away? Message-ID: <17366.29087.747411.639080@montanaro.dyndns.org> I ran Fredrik's listmodules script in my current sandbox and got a deprecation warning for the regex module. According to PEP 4 it is already obsolete. I saw nothing there about the timeframe for actual removal. Will it ever go away? Skip From guido at python.org Tue Jan 24 19:38:04 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 24 Jan 2006 10:38:04 -0800 Subject: [Python-Dev] When will regex really go away? In-Reply-To: <17366.29087.747411.639080@montanaro.dyndns.org> References: <17366.29087.747411.639080@montanaro.dyndns.org> Message-ID: On 1/24/06, skip at pobox.com wrote: > > I ran Fredrik's listmodules script in my current sandbox and got a > deprecation warning for the regex module. According to PEP 4 it is already > obsolete. I saw nothing there about the timeframe for actual removal. Will > it ever go away? ASAP please! PEP 4 lists these that were already obsolete in 2.0: addpack, cmp, cmpcache, codehack, dircmp, dump, fmt, lockfile, newdir, Para, poly, regex, regsub, tb, timing, util, whatsound, tzmod, find, grep, packmail, ni, rand, soundex, cl, sv of these, regex, regsub, and timing are still importable in 2.4 (I don't have a 2.5 handy here at Google, and my home machine seems inaccessible). ISTM these last three can safely be dropped now. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From crutcher at gmail.com Tue Jan 24 20:24:46 2006 From: crutcher at gmail.com (Crutcher Dunnavant) Date: Tue, 24 Jan 2006 11:24:46 -0800 Subject: [Python-Dev] [PATCH] Fix dictionary subclass semantics whenused as global dictionaries In-Reply-To: <43C6E25F.2040606@v.loewis.de> References: <001201c61756$cc2d8780$5916c797@oemcomputer> <43C6E25F.2040606@v.loewis.de> Message-ID: Okay, but is there any reason not to include this in 2.5? There doesn't seem to be any noticeable performance impact, and it does add consistancy (and opens some really, really cool options up). Does anyone have objections to 1402289? On 1/12/06, "Martin v. L?wis" wrote: > Crutcher Dunnavant wrote: > > I sorta disagree about it not being broken. Adding a feature which > > works for eval but not for exec seems pretty broken. > > You "seem" to have a different notion of "to be broken", then. > > Python is broken, if and only if > - the interpreter crashes, for any Python input > - the implementation does not do what the documentation says > it would do > - the BDFL pronounces it is broken > > In this specific change, the change did precisely what the requestor > of the feature wanted it to do (that eval could accept non-exact > dicts was a new feature back then also) > > > It's difficult to > > reason about what will happen in the exec context, so I can't see what > > fixing it would endanger; but I'd deffinately like to see it for 2.5. > > It would make Python code run which is currently rejected with an > exception. Therefore, it is a new feature (a behaviour change). > > Applications relying on that feature would have to specify that > they require "2.4.3"; people would find that code that runs fine > in 2.4.3 fails in 2.4.2. This is not acceptable. > > Regards, > Martin > -- Crutcher Dunnavant monket.samedi-studios.com From skip at pobox.com Tue Jan 24 20:33:08 2006 From: skip at pobox.com (skip at pobox.com) Date: Tue, 24 Jan 2006 13:33:08 -0600 Subject: [Python-Dev] When will regex really go away? In-Reply-To: References: <17366.29087.747411.639080@montanaro.dyndns.org> Message-ID: <17366.33012.687343.786721@montanaro.dyndns.org> Guido> PEP 4 lists these that were already obsolete in 2.0: Guido> addpack, cmp, cmpcache, codehack, dircmp, dump, fmt, Guido> lockfile, newdir, Para, poly, regex, regsub, tb, timing, Guido> util, whatsound, tzmod, find, grep, packmail, ni, rand, Guido> soundex, cl, sv Guido> of these, regex, regsub, and timing are still importable in 2.4. And in 2.5. Regex and regsub both import with deprecation warnings. Timing imports without a peep. None of the others import. Skip From bjourne at gmail.com Tue Jan 24 21:22:01 2006 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Tue, 24 Jan 2006 21:22:01 +0100 Subject: [Python-Dev] The path module PEP Message-ID: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> The last time this was discussed six months ago it seemed like most of python-dev fancied Jason Orendorff's path module. But Guido wanted a PEP and noone created one. So I decided to claim the fame and write one since I also love the path module. :) Much of it is copy-pasted from Peter Astrand's PEP 324, many thanks to him. ################################################# PEP: XXX Title: path - Object oriented handling of filesystem paths Version: $Revision: XXXX $ Last-Modified: $Date: 2006-01-24 19:24:59 +0100 (Sat, 29 Jan 2005) $ Author: Bj?rn Lindqvist Status: Dummo Type: Standards Track (library) Created: 24-Jan-2006 Content-Type: text/plain Python-Version: 2.4 Abstract This PEP describes a new class for handling paths in an object oriented fashion. Motivation Dealing with filesystem paths is a common task in any programming language, and very common in a high-level language like Python. Good support for this task is needed, because: - Almost every program uses paths to access files. It makes sense that a task, that is so often performed, should be as intuitive and as easy to perform as possible. - It makes Python an even better replacement language for over-complicated shell scripts. Currently, Python has a large number of different functions scattered over half a dozen modules for handling paths. This makes it hard for newbies and experienced developers to to choose the right method. The Path class provides the following enhancements over the current common practice: - One "unified" object provides all functionality from previous functions. - Subclassability - the Path object can be extended to support paths other than filesystem paths. The programmer does not need to learn a new API, but can reuse his or her knowledge of Path to deal with the extended class. - With all related functionality in one place, the right approach is easier to learn as one does not have to hunt through many different modules for the right functions. - Python is an object oriented language. Just like files, datetimes and sockets are objects so are paths, they are not merely strings to be passed to functions. Path objects are inherently a pythonic idea. - Path takes advantage of properties. Properties make for more readable code. if imgpath.ext == 'jpg': jpegdecode(imgpath) Is better than: if os.path.splitexit(imgpath)[1] == 'jpg': jpegdecode(imgpath) Rationale The following points summarizes the design: - Path extends from string, therefore all code which expects string pathnames need not be modified and no existing code will break. - A Path object can be created either by using the classmethod Path.cwd, by instantiating the class with a string representing a path or by using the default constructor which is equivalent with Path("."). - The factory functions in popen2 have been removed, because I consider the class constructor equally easy to work with. - Path provides for common pathname manipulation, pattern expansion, pattern matching and other high-level file operations including copying. Basically everything path related except for manipulation of file contents which file objects are better suited for. - Platform incompatibilites are dealt with by not instantiating system specific methods. Specification This class defines the following public methods: # Special Python methods. def __new__(cls, init = os.curdir): ... def __repr__(self): ... def __add__(self, more): ... def __radd__(self, other): ... def __div__(self, rel): ... def __truediv__(self, rel): ... # Alternative constructor. def cwd(cls): ... # Operations on path strings. def abspath(sef): ... def normcase(self): ... def normpath(self): ... def realpath(self): ... def expanduser(self): ... def expandvars(self): ... def dirname(self): ... def basename(self): ... def expand(self): ... def splitpath(self): ... def splitdrive(self): ... def splitext(self): ... def stripext(self): ... def splitunc(self): ... [1] def joinpath(self, *args): ... def splitall(self): ... def relpath(self): ... def relpathto(self, dest): ... # Properties about the path. parent, name, namebase, ext, drive, uncshare[1] # Operations that return lists of paths. def listdir(self, pattern = None): ... def dirs(self, pattern = None): ... def files(self, pattern = None): ... def walk(self, pattern = None): ... def walkdirs(self, pattern = None): ... def walkfiles(self, pattern = None): ... def match(self, pattern): def matchcase(self, pattern): def glob(self, pattern): # Methods for retrieving information about the filesystem # path. def exists(self): ... def isabs(self): ... def isdir(self): ... def isfile(self): ... def islink(self): ... def ismount(self): ... def samefile(self, other): ... [1] def getatime(self): ... def getmtime(self): ... def getctime(self): ... def getsize(self): ... def access(self, mode): ... [1] def stat(self): ... def lstat(self): ... def statvfs(self): ... [1] def pathconf(self, name): ... [1] def utime(self, times): ... def chmod(self, mode): ... def chown(self, uid, gid): ... [1] def rename(self, new): ... def renames(self, new): # Filesystem properties for path. atime, getmtime, getctime, size # Methods for manipulating information about the filesystem # path. def utime(self, times): ... def chmod(self, mode): ... def chown(self, uid, gid): ... [1] def rename(self, new): ... def renames(self, new): ... # Create/delete operations on directories def mkdir(self, mode = 0777): ... def makedirs(self, mode = 0777): ... def rmdir(self): ... def removedirs(self): ... # Modifying operations on files def touch(self): ... def remove(self): ... def unlink(self): ... # Modifying operations on links def link(self, newpath): ... def symlink(self, newlink): ... def readlink(self): ... def readlinkabs(self): ... # High-level functions from shutil def copyfile(self, dst): ... def copymode(self, dst): ... def copystat(self, dst): ... def copy(self, dst): ... def copy2(self, dst): ... def copytree(self, dst, symlinks = True): ... def move(self, dst): ... def rmtree(self, ignore_errors = False, onerror = None): ... # Special stuff from os def chroot(self): ... [1] def startfile(self): ... [1] [1] - Method is not availible on all platforms. Replacing older functions with the Path class In this section, "a ==> b" means that b can be used as a replacement for a. In the following examples, we assume that the Path class is imported with "from path import Path". Replacing os.path.join ---------------------- os.path.join(os.getcwd(), "foobar") ==> Path.cwd() / "foobar" Replacing os.path.splitext -------------------------- os.path.splitext("Python2.4.tar.gz")[1] ==> Path("Python2.4.tar.gz").ext Replacing glob.glob ------------------- glob.glob("/lib/*.so") ==> Path("/lib").glob("*.so") Deprecations Introducing this module to the standard library introduces the need to deprecate a number of existing modules and functions. The table below explains which existing functionality that must be deprecated. PATH METHOD DEPRECATES FUNCTION normcase() os.path.normcase() normpath() os.path.normpath() realpath() os.path.realpath() expanduser() os.path.expanduser() expandvars() os.path.expandvars() dirname() os.path.dirname() basename() os.path.basename() splitpath() os.path.split() splitdrive() os.path.splitdrive() splitext() os.path.splitext() splitunc() os.path.splitunc() joinpath() os.path.join() listdir() os.listdir() [fnmatch.filter()] match() fnmatch.fnmatch() matchcase() fnmatch.fnmatchcase() glob() glob.glob() exists() os.path.exists() isabs() os.path.isabs() isdir() os.path.isdir() isfile() os.path.isfile() islink() os.path.islink() ismount() os.path.ismount() samefile() os.path.samefile() getatime() os.path.getatime() getmtime() os.path.getmtime() getsize() os.path.getsize() cwd() os.getcwd() access() os.access() stat() os.stat() lstat() os.lstat() statvfs() os.statvfs() pathconf() os.pathconf() utime() os.utime() chmod() os.chmod() chown() os.chown() rename() os.rename() renames() os.renames() mkdir() os.mkdir() makedirs() os.makedirs() rmdir() os.rmdir() removedirs() os.removedirs() remove() os.remove() unlink() os.unlink() link() os.link() symlink() os.symlink() readlink() os.readlink() chroot() os.chroot() startfile() os.startfile() copyfile() shutil.copyfile() copymode() shutil.copymode() copystat() shutil.copystat() copy() shutil.copy() copy2() shutil.copy2() copytree() shutil.copytree() move() shutil.move() rmtree() shutil.rmtree() The Path class deprecates the whole of os.path, shutil, fnmatch and glob. A big chunk of os is also deprecated. Open Issues Some functionality of Jason Orendorff's path module have been omitted: * Function for opening a path - better handled by the builtin open(). * Functions for reading and writing a whole file - better handled by file objects read() and write() methods. * A chdir() function may be a worthy inclusion. * A deprecation schedule needs to be setup. How much functionality should Path implement? How much of existing functionality should it deprecate and when? * Where should the class be placed and what should it be called? The functions and modules that this new module is trying to replace (os.path, shutil, fnmatch, glob and parts of os are expected to be available in future Python versions for a long time, to preserve backwards compatibility. Reference Implementation Currently, the Path class is implemented as a thin wrapper around the standard library modules sys, os, fnmatch, glob and shutil. The intention of this PEP is to move functionality from the aforementioned modules to Path while they are being deprecated. For more detail, and diverging implementations see: * http://www.jorendorff.com/articles/python/path/path.py * http://svn.python.org/projects/sandbox/trunk/path/path.py * http://cafepy.com/quixote_extras/rex/path/enhpath.py Examples In this section, "a ==> b" means that b can be used as a replacement for a. 1. Make all python files in the a directory executable: DIR = '/usr/home/guido/bin' for f in os.listdir(DIR): if f.endswith('.py'): path = os.path.join(DIR, f) os.chmod(path, 0755) ==> for f in Path('/usr/home/guido/bin'): f.chmod(0755) 2. Delete emacs backup files: def delete_backups(arg, dirname, names): for name in names: if name.endswith('~'): os.remove(os.path.join(dirname, name)) ==> d = Path(os.environ['HOME']) for f in d.walkfiles('*~'): f.remove() 3. Finding the relative path to a file: b = Path('/users/peter/') a = Path('/users/peter/synergy/tiki.txt') a.relpathto(b) 4. Splitting a path into directory and filename: os.path.split("/path/to/foo/bar.txt") ==> Path("/path/to/foo/bar.txt").splitpath() 5. List all Python scripts in the current directory tree: list(Path().walkfiles("*.py")) 6. Create directory paths: os.path.join("foo", "bar", "baz") ==> Path("foo") / "bar" / "baz" Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: -- mvh Bj?rn From greg at electricrain.com Tue Jan 24 21:22:38 2006 From: greg at electricrain.com (Gregory P. Smith) Date: Tue, 24 Jan 2006 12:22:38 -0800 Subject: [Python-Dev] stabilizing builds In-Reply-To: References: Message-ID: <20060124202238.GM17240@zot.electricrain.com> On Sun, Jan 22, 2006 at 11:01:36PM -0800, Neal Norwitz wrote: > rather than later. There are a bunch of tests that are not stable. > It would really help to get people knowledgeable about a particular > subdomain to provide input into bugs/patches and produce patches too! > > The areas that are known to have problems are listed here: > http://wiki.python.org/moin/BuildBot > > It would be nice to clean those issues up. Does anyone have some time > to spend to resolve the issues with Berkeley DB? That's the one that > annoys me most right now. I have a patch that fixes at least one of > the problems, but no idea if it's the correct solution or not. > http://python.org/sf/1407992 Nice! It does fix the associate test problem on BerkeleyDB 3.3-4.1 and 4.2-4.4 continue to pass. I committed it. Good to have tests passing again when using "old" but still common BerkeleyDBs. > * BSD DB 4.1 and 3.2 fail Using BerkeleyDB 3.2 often segfaults for me; using 3.3 often hangs in the test suite. Both are so old I don't see much motivation to track the issues down. From python at discworld.dyndns.org Tue Jan 24 21:56:51 2006 From: python at discworld.dyndns.org (Charles Cazabon) Date: Tue, 24 Jan 2006 14:56:51 -0600 Subject: [Python-Dev] The path module PEP In-Reply-To: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> Message-ID: <20060124205651.GA28435@discworld.dyndns.org> BJ?rn Lindqvist wrote: > > 1. Make all python files in the a directory executable: [...] > ==> > for f in Path('/usr/home/guido/bin'): > f.chmod(0755) Iterating over a path string to read the contents of the directory possibly pointed to by that string seems like "magic implicit" behaviour. Perhaps making it a method explicitly returning an iterator would by more Pythonic? for f in Path(...).readDir(): > 4. Splitting a path into directory and filename: [...] > Path("/path/to/foo/bar.txt").splitpath() Good. But the opposite isn't done similarly: > 6. Create directory paths: [...] > Path("foo") / "bar" / "baz" Using "/" as "path concatenation operator" seems like un-Pythonic magic as well (while "+" would be an improvement, it's still not a large one). I would think Path('foo').appendparts('bar', 'baz') or similar would be more readable and obvious. Charles -- ----------------------------------------------------------------------- Charles Cazabon GPL'ed software available at: http://pyropus.ca/software/ ----------------------------------------------------------------------- From eric.nieuwland at xs4all.nl Tue Jan 24 22:18:52 2006 From: eric.nieuwland at xs4all.nl (Eric Nieuwland) Date: Tue, 24 Jan 2006 22:18:52 +0100 Subject: [Python-Dev] The path module PEP In-Reply-To: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> Message-ID: <22e4d10c08d514bbe2f8db0dd186e015@xs4all.nl> Good stuff. Some suggestions: > def joinpath(self, *args): ... I suggest append() or extend() as join*() sort of suggest join() as provided by strings, which does something quite different > def splitall(self): ... and this may renamed split(), as it is quite similar to split() as provided by strings > # Properties about the path. > parent, name, namebase, ext, drive, uncshare[1] so we can drop basename(), dirname(), splitdrive(), and splitext() > def dirs(self, pattern = None): ... > def files(self, pattern = None): ... can we add others()? (sockets, pipes, block and character devices) --eric From jason.orendorff at gmail.com Tue Jan 24 22:18:30 2006 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Tue, 24 Jan 2006 16:18:30 -0500 Subject: [Python-Dev] The path module PEP In-Reply-To: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> Message-ID: Thanks for doing this. I'm not sure anyone that matters here is actually keen on path, but I guess we'll see. A few comments: On 1/24/06, BJ?rn Lindqvist wrote: > The following points summarizes the design: > > - Path extends from string, therefore all code which expects > string pathnames need not be modified and no existing code will > break. Actually, I would prefer a Path that *didn't* subclass string, and a new "%p" format-thingy in PyArg_ParseTuple(). %p would expect either a Path object or a string. Stdlib C functions that take paths would be changed from using %s or %u to %p. This would pretty much eliminate the need for path objects to act like strings (except where __cmp__, __hash__, and common sense dictate). The only reason I didn't do this in path.py is that I don't have the required write access to the Python source tree. ;) Subclassing str/unicode seemed like the next best thing. > [...]omitted: > * Function for opening a path - better handled by the builtin > open(). Aside: I added this to support a few people who liked the idea of "openable objects", meaning anything that has .open(), analogous to "writeable objects" being anything with .write(). I don't use it personally. Examples 1 and 2 have errors. In example 1, the "after" code should be: d = path('/usr/home/guido/bin') for f in d.files('*.py'): f.chmod(0755) In example 2, the "before" code is missing a line -- the call to os.path.walk(). (Actually it should probably use os.walk(), which looks much nicer.) I suspect you'll be asked to change the PEP to remove __div__ for starters, in which case I propose using the Path constructor as the replacement for os.path.join(). In that case, Path.joinpath can be dropped. -j From ianb at colorstudy.com Tue Jan 24 22:28:05 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 24 Jan 2006 15:28:05 -0600 Subject: [Python-Dev] The path module PEP In-Reply-To: <20060124205651.GA28435@discworld.dyndns.org> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <20060124205651.GA28435@discworld.dyndns.org> Message-ID: <43D69BE5.2030809@colorstudy.com> Charles Cazabon wrote: > BJ?rn Lindqvist wrote: > >> 1. Make all python files in the a directory executable: > > [...] > >> ==> >> for f in Path('/usr/home/guido/bin'): >> f.chmod(0755) > > > Iterating over a path string to read the contents of the directory possibly > pointed to by that string seems like "magic implicit" behaviour. Perhaps > making it a method explicitly returning an iterator would by more Pythonic? > > for f in Path(...).readDir(): I believe .listdir() exists already as a method alternative. I'm -0 on iteration as listdir. Doing iteration like strings (over the characters) would be evil. >> 4. Splitting a path into directory and filename: > > [...] > >> Path("/path/to/foo/bar.txt").splitpath() > > > Good. But the opposite isn't done similarly: > > >> 6. Create directory paths: > > [...] > >> Path("foo") / "bar" / "baz" > > > Using "/" as "path concatenation operator" seems like un-Pythonic magic as > well (while "+" would be an improvement, it's still not a large one). I would > think > > Path('foo').appendparts('bar', 'baz') > > or similar would be more readable and obvious. .joinpath() does this. Though .join() does something else entirely that it inherits from strings, something evil to do with a path, and I think that method should raise NotImplementedError. + should not be overridden, because strings define that. Some other str methods are harmless but pointless: center, expandtabs, ljust, zfill; title, capitalize, and istitle are iffy. Also, are there any particular cases where string methods on a path produce an actual str, not another Path object? -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From ianb at colorstudy.com Tue Jan 24 22:37:46 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 24 Jan 2006 15:37:46 -0600 Subject: [Python-Dev] The path module PEP In-Reply-To: References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> Message-ID: <43D69E2A.6050205@colorstudy.com> Jason Orendorff wrote: >> [...]omitted: >> * Function for opening a path - better handled by the builtin >> open(). > > > Aside: I added this to support a few people who liked the idea of > "openable objects", meaning anything that has .open(), analogous to > "writeable objects" being anything with .write(). I don't use it > personally. Losing .open() would make it much harder for anyone wanting to write, say, a URI library that implements the Path API. > I suspect you'll be asked to change the PEP to remove __div__ for > starters, in which case I propose using the Path constructor as the > replacement for os.path.join(). In that case, Path.joinpath can be > dropped. I'm -1 on this too. This means people will be hardcoding the specific class they expect, so you can't pass in other classes. E.g., this will fail: def read_config(home_dir): f = open(Path(home_dir, '.config_file')) c = f.read() f.close() return c read_config(URI('http://localhost/foo')) I'm personally +1 on /. I think people who think it is confusing are giving a knee-jerk reaction. It's very easy to tell the difference between file-related code and math-related code, and / is currently only used for math. In contrast, + is used for concatenation and addition, and these are far more ambiguous from context -- but still it doesn't cause that many problems. But barring /, .joinpath() should remain. Too bad there isn't a shorter name, except .join() which is taken. I would also, though, note that there are some security issues. When you use open() or other path-related functions, you *know* you are doing filesystem operations. You can't be getting some weird object that does who-knows-what. For the most part if you can get an arbitrary object into the system then the system is just broken, but I can imagine cases where this encourages people to do bad things. I only bring this up because it reminds me of PHP allowed over-the-net includes, which always seemed like a bad idea. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From ianb at colorstudy.com Tue Jan 24 22:58:36 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 24 Jan 2006 15:58:36 -0600 Subject: [Python-Dev] The path module PEP In-Reply-To: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> Message-ID: <43D6A30C.30205@colorstudy.com> BJ?rn Lindqvist wrote: > * Functions for reading and writing a whole file - better handled > by file objects read() and write() methods. I would be disappointed to see this left out, because I get really tired of this little dance: f = open(filename) c = f.read() f.close() return c Then you can put a try:finally: in there too for extra correctness. Anyway, I write this code all the time, and it's really tiresome. open(filename).read() also works, but relies on Python's reference counting to be really reliable; maybe I shouldn't worry about that and use just that form. But that bothers me too. The same occurs during writing. The original Path object has a bytes and text method (I think), which nicely distinguishes between the two cases. This helps suggest better and more explicit handling of unicode, something that Python should work harder at making clear. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From ncoghlan at gmail.com Tue Jan 24 23:10:15 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 25 Jan 2006 08:10:15 +1000 Subject: [Python-Dev] The path module PEP In-Reply-To: <43D6A30C.30205@colorstudy.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6A30C.30205@colorstudy.com> Message-ID: <43D6A5C7.6080008@gmail.com> Ian Bicking wrote: > BJ?rn Lindqvist wrote: >> * Functions for reading and writing a whole file - better handled >> by file objects read() and write() methods. > > I would be disappointed to see this left out, because I get really tired > of this little dance: > > f = open(filename) > c = f.read() > f.close() > return c Python 2.5 (well, once someone finds time to update mwh's patch): with open(filename) as f: return f.read() Behaviour guaranteed by language definition ;) Cheers, Nick. P.S. I too would really like to see this happen for 2.5. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From tim.peters at gmail.com Tue Jan 24 23:22:21 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 24 Jan 2006 17:22:21 -0500 Subject: [Python-Dev] Long-time shy failure in test_socket_ssl Message-ID: <1f7befae0601241422sd783c37u5d676893de2f6055@mail.gmail.com> Has anyone else noticed this? For a long time (possibly years), I see an infrequent error in test_socket_ssl, like so (this is on WinXP Pro): test_socket_ssl test test_socket_ssl crashed -- exceptions.TypeError: 'NoneType' object is not callable I haven't been able to provoke it by running test_socket_ssl in a loop, and I don't have a guess about what's going wrong by eyeballing the code. test_rude_shutdown() is dicey, relying on a sleep() instead of proper synchronization to make it probable that the `listener` thread goes away before the main thread tries to connect, but while that race may account for bogus TestFailed deaths, it doesn't seem possible that it could account for the kind of failure above. From martin at v.loewis.de Tue Jan 24 23:52:48 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 24 Jan 2006 23:52:48 +0100 Subject: [Python-Dev] [PATCH] Fix dictionary subclass semantics whenused as global dictionaries In-Reply-To: References: <001201c61756$cc2d8780$5916c797@oemcomputer> <43C6E25F.2040606@v.loewis.de> Message-ID: <43D6AFC0.5080602@v.loewis.de> Crutcher Dunnavant wrote: > Okay, but is there any reason not to include this in 2.5? There > doesn't seem to be any noticeable performance impact, and it does add > consistancy (and opens some really, really cool options up). I see no reason, except perhaps the lack of volunteers to actually patch the repository (along with the accompanying work). Regards, Martin From martin at v.loewis.de Tue Jan 24 23:52:52 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 24 Jan 2006 23:52:52 +0100 Subject: [Python-Dev] stabilizing builds In-Reply-To: <20060124111528.GB18916@xs4all.nl> References: <20060124111528.GB18916@xs4all.nl> Message-ID: <43D6AFC4.3030200@v.loewis.de> Thomas Wouters wrote: > FWIW, it's brittle on Solaris 9, too, and the SF compilefarm has two of > those. I don't know if it's the same problem, but on Solaris 9, the slave > part of the tty/pty pair sometimes isn't a TTY (according to os.isatty.) The > buildbot's log doesn't list the solaris 10 test_pty failure though, just the > test_logging failure. It occasionally failed in test_pty, too, with the very same error message (that /dev/tty isn't a tty). > It looks like a timing issue; the first run succeeds, all subsequent runs > fail, for a while, anyway. I'll do some googling and browsing other > tty/pty-using code to see if there's anything we're not doing we should be > doing, but it looks like a platform bug that we can't fix... Not without > re-implementing os.isatty anyway ;P Couldn't there be a bug in openpty instead? Perhaps it is trying to allocate the same device again, but fails to do so correctly, and fails to recognize that it should use a different one instead. Anyway, if you still think you need an OS 10 account, please let me know, and I can give you an account to the machine the buildbot runs on. Regards, Martin From phd at mail2.phd.pp.ru Tue Jan 24 23:24:18 2006 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Wed, 25 Jan 2006 01:24:18 +0300 Subject: [Python-Dev] The path module PEP In-Reply-To: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> Message-ID: <20060124222418.GA1409@phd.pp.ru> On Tue, Jan 24, 2006 at 09:22:01PM +0100, BJ?rn Lindqvist wrote: > Path("foo") / "bar" / "baz" I really love this! But I am afraid it's too much a Unixism. (-: Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From gjc at inescporto.pt Tue Jan 24 22:34:28 2006 From: gjc at inescporto.pt (Gustavo J. A. M. Carneiro) Date: Tue, 24 Jan 2006 21:34:28 +0000 Subject: [Python-Dev] The path module PEP In-Reply-To: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> Message-ID: <1138138468.6383.3.camel@localhost.localdomain> On Tue, 2006-01-24 at 21:22 +0100, BJ?rn Lindqvist wrote: [...] > # Operations on path strings. > def abspath(sef): ... > def normcase(self): ... > def normpath(self): ... > def realpath(self): ... > def expanduser(self): ... > def expandvars(self): ... > def dirname(self): ... > def basename(self): ... > def expand(self): ... > def splitpath(self): ... > def splitdrive(self): ... > def splitext(self): ... > def stripext(self): ... > def splitunc(self): ... [1] > def joinpath(self, *args): ... > def splitall(self): ... > def relpath(self): ... > def relpathto(self, dest): ... [...etc...] If we wanted to take PEP 8 seriously, those method names should be changed to words_separated_by_underscores. And BTW, what does splitunc do? It really should have a more descriptive name. Regards. -- Gustavo J. A. M. Carneiro The universe is always one step beyond logic From ianb at colorstudy.com Wed Jan 25 00:03:37 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 24 Jan 2006 17:03:37 -0600 Subject: [Python-Dev] The path module PEP In-Reply-To: <43D69E2A.6050205@colorstudy.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D69E2A.6050205@colorstudy.com> Message-ID: <43D6B249.8090603@colorstudy.com> Ian Bicking wrote: > I'm -1 on this too. This means people will be hardcoding the specific > class they expect, so you can't pass in other classes. E.g., this will > fail: > > def read_config(home_dir): > f = open(Path(home_dir, '.config_file')) > c = f.read() > f.close() > return c > > read_config(URI('http://localhost/foo')) It occurs to me that it might be hopeless to expect substitution to work generally (at least without a specific thought on the matter) because I expect this form will be typical: def read_config(path): # convert string input to a path (has no effect on Path objects): path = Path(path) content = path.text() Since people will be passing strings in to file-related functions for the forseeable future, so people will coerce that input to paths explicitly whenever they accept a path from a public function. Now, if there were a way to make sure that "Path(x) is x" is true when x is already a Path, and maybe a way to coerce strings to a Path without coercing Path-like objects into Path objects, that would help resolve the problem. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From trentm at ActiveState.com Wed Jan 25 00:03:59 2006 From: trentm at ActiveState.com (Trent Mick) Date: Tue, 24 Jan 2006 15:03:59 -0800 Subject: [Python-Dev] The path module PEP In-Reply-To: <1138138468.6383.3.camel@localhost.localdomain> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <1138138468.6383.3.camel@localhost.localdomain> Message-ID: <20060124230359.GB32313@activestate.com> [Gustavo J. A. M. Carneiro wrote] > And BTW, what does splitunc do? http://en.wikipedia.org/wiki/Path_%28computing%29#Universal_Naming_Convention > It really should have a more descriptive name. No more that should "urllib" or "splitext". Trent -- Trent Mick TrentM at ActiveState.com From t-meyer at ihug.co.nz Tue Jan 24 23:59:50 2006 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Wed, 25 Jan 2006 11:59:50 +1300 Subject: [Python-Dev] The path module PEP In-Reply-To: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> Message-ID: > The last time this was discussed six months ago it seemed like most of > python-dev fancied Jason Orendorff's path module. But Guido wanted a > PEP and noone created one. So I decided to claim the fame and write > one since I also love the path module. :) Much of it is copy-pasted > from Peter Astrand's PEP 324, many thanks to him. It would be great (and save a lot of rehashing) if you could go over the python-dev discussion and add the relevant parts (for example, whether to include the __div__ hack) to the PEP: =Tony.Meyer From fredrik at pythonware.com Wed Jan 25 00:25:21 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 25 Jan 2006 00:25:21 +0100 Subject: [Python-Dev] The path module PEP References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <1138138468.6383.3.camel@localhost.localdomain> Message-ID: Gustavo J. A. M. Carneiro wrote: > > # Operations on path strings. > > def abspath(sef): ... > > def normcase(self): ... > > def normpath(self): ... > > def realpath(self): ... > > def expanduser(self): ... > > def expandvars(self): ... > > def dirname(self): ... > > def basename(self): ... > > def expand(self): ... > > def splitpath(self): ... > > def splitdrive(self): ... > > def splitext(self): ... > > def stripext(self): ... > > def splitunc(self): ... [1] > > def joinpath(self, *args): ... > > def splitall(self): ... > > def relpath(self): ... > > def relpathto(self, dest): ... > [...etc...] > > If we wanted to take PEP 8 seriously, those method names should be > changed to words_separated_by_underscores. > And BTW, what does splitunc do? It really should have a more > descriptive name. when did you last use the os.path module ? From ianb at colorstudy.com Wed Jan 25 00:36:36 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 24 Jan 2006 17:36:36 -0600 Subject: [Python-Dev] The path module PEP In-Reply-To: <1138138468.6383.3.camel@localhost.localdomain> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <1138138468.6383.3.camel@localhost.localdomain> Message-ID: <43D6BA04.8010409@colorstudy.com> Gustavo J. A. M. Carneiro wrote:. >> def splitall(self): ... >> def relpath(self): ... >> def relpathto(self, dest): ... > > [...etc...] > > If we wanted to take PEP 8 seriously, those method names should be > changed to words_separated_by_underscores. There's a (unspecified?) convention that many standard/core objects or objects in the standard library use squishedwords for methods. has_key is an anomoly, not the norm. Also, many of these are direct translations of methods from os.path, and so the names offer familiarity. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From ianb at colorstudy.com Wed Jan 25 00:38:45 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 24 Jan 2006 17:38:45 -0600 Subject: [Python-Dev] The path module PEP In-Reply-To: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> Message-ID: <43D6BA85.8070007@colorstudy.com> There's kind of a lot of methods in here, which is a little bothersome. It also points towards the motivation for the class -- too many options in too many places in the stdlib. But throwing them *all* in one class consolidates but doesn't simplify, especially with duplicate functionality. I'm not strongly advocating any of the methods below be removed, but at least it seems like it should be discussed. By not removing any it is easier to translate the os.path (and other) forms. I imagine it will be a long time before those forms go away, though, so I don't know how useful it is to plan for a speedy and seamless transition. BJ?rn Lindqvist wrote: > This class defines the following public methods: > > # Special Python methods. > def __new__(cls, init = os.curdir): ... > def __repr__(self): ... > def __add__(self, more): ... > def __radd__(self, other): ... > def __div__(self, rel): ... > def __truediv__(self, rel): ... > > # Alternative constructor. > def cwd(cls): ... > > # Operations on path strings. > def abspath(sef): ... > def normcase(self): ... > def normpath(self): ... > def realpath(self): ... > def expanduser(self): ... > def expandvars(self): ... This is equivalent to p.__class__(string.Template(p).safe_substitute(os.environ)). Obviously that form is a lot longer, but maybe usefully more explicit. Well, it is a *lot* longer. But if string.Template's functionality becomes a method on str (is that the plan?) then this won't be so bad. Also if string.Template returns an object of the same class as is passed in. Then maybe it'd just be p.safe_substitute(os.environ), which isn't bad at all. Maybe if this used Windows conventions on that platform -- of %VAR% -- it would seem more useful. Though I think $VAR should still work on both platforms regardless (just like / does). > def dirname(self): ... > def basename(self): ... These are duplicated as properties. basename and namebase are confusing alternatives to each other. > def expand(self): ... I see this is a combination of normpath, expanduser, and expandvars. Useful, certainly. But there's also a lot of forms, and no one applies these operations consistently it seems. > def splitpath(self): ... > def splitdrive(self): ... > def splitext(self): ... > def stripext(self): ... This is another new method, equivalent to .splitext()[0]. I'm not sure it's that important. > def splitunc(self): ... [1] > def joinpath(self, *args): ... > def splitall(self): ... And there's just so many splitting functions. Could these somehow be combined? Maybe returning a tuple/struct? Or maybe just relying on properties. > def relpath(self): ... > def relpathto(self, dest): ... These don't feel compellingly different according to the name. I find the cwd fragile too, so maybe the first form offends me from that perspective too. Just the explicit form feels sufficient to me, and unambiguous as both a reader and writer of code. > # Properties about the path. > parent, name, namebase, ext, drive, uncshare[1] Actually, I see namebase is actually the name without an extension. It seems ambiguous to me just from the names, and I'd rather both weren't there. Though ext somehow seems useful and unambiguous in a way namebase isn't. Not sure why. It's unclear which of these should be Paths. Of course parent should. None of the others? When methods return paths and when they return strings is an important part of the spec. > # Operations that return lists of paths. > def listdir(self, pattern = None): ... > def dirs(self, pattern = None): ... > def files(self, pattern = None): ... > def walk(self, pattern = None): ... > def walkdirs(self, pattern = None): ... > def walkfiles(self, pattern = None): ... Notably these aren't like os.path.walk, I assume. Which is fine by me. > def match(self, pattern): > def matchcase(self, pattern): I don't see these methods in the path class, and I'm not sure what they'd do. > def glob(self, pattern): > > # Methods for retrieving information about the filesystem > # path. > def exists(self): ... > def isabs(self): ... > def isdir(self): ... > def isfile(self): ... > def islink(self): ... > def ismount(self): ... > def samefile(self, other): ... [1] > def getatime(self): ... > def getmtime(self): ... > def getctime(self): ... > def getsize(self): ... > def access(self, mode): ... [1] > def stat(self): ... > def lstat(self): ... > def statvfs(self): ... [1] The stat and get* functions overlap too. I.e., p.getmtime() and p.stat().st_mtime are the same. Too bad about the st_* names on stat objects, otherwise I don't see any problem with using that directly. It still seems clearer. > def pathconf(self, name): ... [1] I can't figure out what this does, even from the docs. Some of these seem obscure enough they could be left in os. > def utime(self, times): ... > def chmod(self, mode): ... > def chown(self, uid, gid): ... [1] > def rename(self, new): ... > def renames(self, new): > > # Filesystem properties for path. > atime, getmtime, getctime, size Mmm... then these show up yet again. > # Methods for manipulating information about the filesystem > # path. > def utime(self, times): ... > def chmod(self, mode): ... > def chown(self, uid, gid): ... [1] > def rename(self, new): ... > def renames(self, new): ... Dups in the spec. > # Create/delete operations on directories > def mkdir(self, mode = 0777): ... > def makedirs(self, mode = 0777): ... > def rmdir(self): ... > def removedirs(self): ... > > # Modifying operations on files > def touch(self): ... > def remove(self): ... > def unlink(self): ... > > # Modifying operations on links > def link(self, newpath): ... > def symlink(self, newlink): ... > def readlink(self): ... > def readlinkabs(self): ... > > # High-level functions from shutil > def copyfile(self, dst): ... > def copymode(self, dst): ... > def copystat(self, dst): ... > def copy(self, dst): ... > def copy2(self, dst): ... > def copytree(self, dst, symlinks = True): ... > def move(self, dst): ... > def rmtree(self, ignore_errors = False, onerror = None): ... > > # Special stuff from os > def chroot(self): ... [1] > def startfile(self): ... [1] Like pathconf, maybe these don't need to be moved into the module, and can be left in os. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From tim.peters at gmail.com Wed Jan 25 00:53:36 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 24 Jan 2006 18:53:36 -0500 Subject: [Python-Dev] Long-time shy failure in test_socket_ssl In-Reply-To: <1f7befae0601241422sd783c37u5d676893de2f6055@mail.gmail.com> References: <1f7befae0601241422sd783c37u5d676893de2f6055@mail.gmail.com> Message-ID: <1f7befae0601241553v10f7dd41i364e92b98f9de4a2@mail.gmail.com> [Tim Peters] > ... > test_rude_shutdown() is dicey, relying on a sleep() instead of proper > synchronization to make it probable that the `listener` thread goes > away before the main thread tries to connect, but while that race may > account for bogus TestFailed deaths, it doesn't seem possible that it > could account for the kind of failure above. Well, since it's silly to try to guess about one weird failure when a clear cause for another kind of weird failure is known, I checked in changes to do "proper" thread synchronization and termination in that test. Hasn't failed here since, but that's not surprising (it was always a "once in a light blue moon" kind of thing). I'm not sure how/whether this test is supposed to work with Jython -- perhaps the `thread.exit()` I removed could be important there. The test relies on that a socket gets closed when a socket object becomes trash & is reclaimed; in CPython that's easy to control; I don't know why the test didn't/doesn't simply do an explicit s.close() instead. From guido at python.org Wed Jan 25 01:01:29 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 24 Jan 2006 16:01:29 -0800 Subject: [Python-Dev] Know anyone interested in a Google internship? Message-ID: Sorry for the plug. Google is looking to fill an unprecedented number of student intern positions this summer, at several US locations (Mountain View, Santa Monica, Kirkland (Wash.), and New York). If you're interested or know someone interested, please see http://www.google.com/jobs/intern.html. Candidates from outside the US are welcome. The perks are incredible (just ask Brett Cannon, who's coming back for the second year this summer :-). We're not just looking for software development interns -- there are also product management positions, and UI design and usability analyst positions. I can't guarantee that I'll be mentoring you, but if you end up using Python in Mountain View, I'm sure we'll be interacting quite a bit! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From thomas at xs4all.net Wed Jan 25 01:59:18 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Wed, 25 Jan 2006 01:59:18 +0100 Subject: [Python-Dev] stabilizing builds In-Reply-To: <43D6AFC4.3030200@v.loewis.de> References: <20060124111528.GB18916@xs4all.nl> <43D6AFC4.3030200@v.loewis.de> Message-ID: <20060125005918.GC18916@xs4all.nl> On Tue, Jan 24, 2006 at 11:52:52PM +0100, "Martin v. L?wis" wrote: > > It looks like a timing issue; the first run succeeds, all subsequent runs > > fail, for a while, anyway. I'll do some googling and browsing other > > tty/pty-using code to see if there's anything we're not doing we should be > > doing, but it looks like a platform bug that we can't fix... Not without > > re-implementing os.isatty anyway ;P > Couldn't there be a bug in openpty instead? Perhaps it is trying to > allocate the same device again, but fails to do so correctly, and fails > to recognize that it should use a different one instead. Well, a bug in openpty is what I started with. Python's posix.openpty() that is, as Solaris doesn't have openpty. Openpty is emulated using code taken almost verbatim from Solaris' pts(7D) manpage: fdm = open("/dev/ptmx", O_RDWR); /* open master */ grantpt(fdm); /* change permission of slave */ unlockpt(fdm); /* unlock slave */ slavename = ptsname(fdm); /* get name of slave */ fds = open(slavename, O_RDWR); /* open slave */ ioctl(fds, I_PUSH, "ptem"); /* push ptem */ ioctl(fds, I_PUSH, "ldterm"); /* push ldterm*/ (That's the manpage code.) This is also what openssh does (as far as I can tell). Screen does it slightly differently; it does the ptsname() call before the grantpt/unlockpt calls, but the open(slavename) after. If I make posixmodule do that, it still fails on Solaris. Mucking with it more just breaks it more. The thing is, disabling the check that fails, whether the slave-tty returned by os.openpty() is a tty, shows that the later test for the same thing succeeds. The later test is done in a child created by pty.fork(). Disabling the ptmx code on Solaris is probably the most reliable way to fix the failing test; the pty module will fall back to old-style BSD pty code and that works fine. But it's a bit of a shame not to use /dev/ptmx just because the slave fd, when used directly (rather than in a child process) sometimes doesn't seem to be a tty. They're still connected, it still seems to work fine. > Anyway, if you still think you need an OS 10 account, please let me > know, and I can give you an account to the machine the buildbot > runs on. I think I've seen enough confusing situations for a while... I'm sure the bug is the same on Solaris 10 ;P -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From mike at skew.org Wed Jan 25 04:25:10 2006 From: mike at skew.org (Mike Brown) Date: Tue, 24 Jan 2006 20:25:10 -0700 (MST) Subject: [Python-Dev] New Pythondoc by effbot In-Reply-To: <740c3aec0601230132n2ff31a22n1cd49f9517330cb1@mail.gmail.com> Message-ID: <200601250325.k0P3PAWO020866@chilled.skew.org> BJ> Why does it have to be "wiki-like"? Why can't it be a wiki? MediaWiki > seem to work pretty well for a lot of software projects that have put > their documentation in a wiki. Talk pages for commentary and primary > pages for reviewed content. And inconsistent formatting from article to article, limitations in indexing options, no way to browse a set of documentation specific to a particular Python release... I personally like the PHP docs - http://www.php.net/manual/en/ They're not versioned, and users can't modify the indexes or API docs, but they do get to add annotations. Annotations that reflect errors or major omissions from the docs can be reviewed by editors and folded into the docs as needed. *shrug* From jeremy at alum.mit.edu Wed Jan 25 04:46:17 2006 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Tue, 24 Jan 2006 22:46:17 -0500 Subject: [Python-Dev] Know anyone interested in a Google internship? In-Reply-To: References: Message-ID: On 1/24/06, Guido van Rossum wrote: > Sorry for the plug. > > Google is looking to fill an unprecedented number of student intern > positions this summer, at several US locations (Mountain View, Santa > Monica, Kirkland (Wash.), and New York). If you're interested or know > someone interested, please see > http://www.google.com/jobs/intern.html. Candidates from outside the US > are welcome. The perks are incredible (just ask Brett Cannon, who's > coming back for the second year this summer :-). > > We're not just looking for software development interns -- there are > also product management positions, and UI design and usability analyst > positions. > > I can't guarantee that I'll be mentoring you, but if you end up using > Python in Mountain View, I'm sure we'll be interacting quite a bit! And I'd be happy to work with folks in New York! Jeremy From mwh at python.net Wed Jan 25 10:34:45 2006 From: mwh at python.net (Michael Hudson) Date: Wed, 25 Jan 2006 09:34:45 +0000 Subject: [Python-Dev] stabilizing builds In-Reply-To: <20060124202238.GM17240@zot.electricrain.com> (Gregory P. Smith's message of "Tue, 24 Jan 2006 12:22:38 -0800") References: <20060124202238.GM17240@zot.electricrain.com> Message-ID: <2mlkx4lkga.fsf@starship.python.net> "Gregory P. Smith" writes: > Using BerkeleyDB 3.2 often segfaults for me; using 3.3 often hangs in > the test suite. Both are so old I don't see much motivation to track > the issues down. My goal is to not have http://www.python.org/dev/buildbot/ go red randomly because of erratic bsddb tests, so I'd definitely prefer it if we didn't build bsddb3 when a not-supported bsddb is found. Things seem much better today after your recent changes though, so thanks for that :) Cheers, mwh -- at any rate, I'm satisfied that not only do they know which end of the pointy thing to hold, but where to poke it for maximum effect. -- Eric The Read, asr, on google.com From hoffman at ebi.ac.uk Wed Jan 25 12:36:15 2006 From: hoffman at ebi.ac.uk (Michael Hoffman) Date: Wed, 25 Jan 2006 11:36:15 +0000 Subject: [Python-Dev] The path module PEP In-Reply-To: <43D69E2A.6050205@colorstudy.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D69E2A.6050205@colorstudy.com> Message-ID: [Ian Bickling] > I'm personally +1 on /. I think people who think it is confusing are > giving a knee-jerk reaction. It's very easy to tell the difference > between file-related code and math-related code, and / is currently only > used for math. In contrast, + is used for concatenation and addition, > and these are far more ambiguous from context -- but still it doesn't > cause that many problems. I was initially -0 on / but I have found it quite useful and a lot simpler than a lot of joinpath()s over time. So +1 on / for me. -- Michael Hoffman European Bioinformatics Institute From hoffman at ebi.ac.uk Wed Jan 25 12:39:46 2006 From: hoffman at ebi.ac.uk (Michael Hoffman) Date: Wed, 25 Jan 2006 11:39:46 +0000 Subject: [Python-Dev] The path module PEP In-Reply-To: References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> Message-ID: [Tony Meyer] > It would be great (and save a lot of rehashing) if you could go over > the python-dev discussion and add the relevant parts (for example, > whether to include the __div__ hack) to the PEP: > > In particular the points about Path being able to be a drop-in replacement for str/unicode are useful ones, and explain the use of joinpath() etc. It is really useful that I can use a Path anywhere I might have used an str and not have to worry about the conversions. -- Michael Hoffman European Bioinformatics Institute From skip at pobox.com Wed Jan 25 13:24:55 2006 From: skip at pobox.com (skip at pobox.com) Date: Wed, 25 Jan 2006 06:24:55 -0600 Subject: [Python-Dev] Know anyone interested in a Google internship? In-Reply-To: References: Message-ID: <17367.28183.39730.537523@montanaro.dyndns.org> Guido> Google is looking to fill an unprecedented number of student Guido> intern positions this summer, at several US locations .... Maybe a Python Job Board post would be in order... Skip From tdickenson at devmail.geminidataloggers.co.uk Wed Jan 25 13:30:06 2006 From: tdickenson at devmail.geminidataloggers.co.uk (Toby Dickenson) Date: Wed, 25 Jan 2006 12:30:06 +0000 Subject: [Python-Dev] The path module PEP In-Reply-To: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> Message-ID: <200601251230.06230.tdickenson@devmail.geminidataloggers.co.uk> On Tuesday 24 January 2006 20:22, BJ?rn Lindqvist wrote: > Replacing glob.glob > ------------------- > > glob.glob("/lib/*.so") > ==> > Path("/lib").glob("*.so") This definition seems confusing because it splits the glob pattern string in two ('/lib', and '*.so'). Unless there is an intention to change the behavior of the current glob module, why not make the glob method parameterless: glob.glob("/lib/*.so") ==> Path("/lib/*.so").glob() Possible confusion with the one parameter version: Does glob matching happen on the first half too? That is, does Path('*').glob('*.so') match files in any directory, or only directories whose name is an asterisk. What behavior can I expect from Path('/foo/').glob(bar), where bar is some arbitrary string? It could be reasonable to expect that it would only match filenames inside the foo directory. However it could also be reasonable to expect to use bar=='/etc/*' -- Toby Dickenson From jason.orendorff at gmail.com Wed Jan 25 17:12:00 2006 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Wed, 25 Jan 2006 11:12:00 -0500 Subject: [Python-Dev] The path module PEP In-Reply-To: <200601251230.06230.tdickenson@devmail.geminidataloggers.co.uk> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <200601251230.06230.tdickenson@devmail.geminidataloggers.co.uk> Message-ID: On 1/25/06, Toby Dickenson wrote: > On Tuesday 24 January 2006 20:22, BJ?rn Lindqvist wrote: > > #Replacing glob.glob > > glob.glob("/lib/*.so") > > ==> > > Path("/lib").glob("*.so") > > This definition seems confusing because it splits the glob pattern string in > two ('/lib', and '*.so'). [...] Well, let's make this look more like real code: #line 1 LIB_DIR = "/lib" ==> LIB_DIR = Path("/lib") #line 296 libs = glob.glob(os.path.join(LIB_DIR, "*.so")) ==> libs = LIB_DIR.files("*.so") Clearer? In d.files(pattern), d is simply the root directory for the search. The same is true of all the searching methods: dirs(), walkfiles(), walkdirs(), etc. I actually never use path.glob(). For example, here files() is actually more accurate, and the word "files" is surely clearer than "glob". Given files(), dirs(), and listdir(), I have never found a real use case for glob(). -j From jason.orendorff at gmail.com Wed Jan 25 17:39:03 2006 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Wed, 25 Jan 2006 11:39:03 -0500 Subject: [Python-Dev] The path module PEP In-Reply-To: <43D6BA85.8070007@colorstudy.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> Message-ID: On 1/24/06, Ian Bicking wrote: > There's kind of a lot of methods in here, which is a little bothersome. > It also points towards the motivation for the class -- too many > options in too many places in the stdlib. But throwing them *all* in > one class consolidates but doesn't simplify, especially with duplicate > functionality. I agree. Let me explain why there's so much cruft in path.py. The design is heavily skewed toward people already very familiar with the existing stdlib equivalents, because that is the market for third-party modules. I think my users want to type p.methodname() for whatever method name they already know, and have it *just work*. A sloppy API you've already learned is easier to pick up than a clean API you've never seen before. Ergo, cruft. A stdlib Path should have different design goals. It should have less redundancy, fewer methods overall, and PEP-8-compliant names. -j From bjourne at gmail.com Wed Jan 25 21:37:04 2006 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Wed, 25 Jan 2006 21:37:04 +0100 Subject: [Python-Dev] The path module PEP In-Reply-To: References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> Message-ID: <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> My comments on the issues. It was easier this way than trying to reply on every message individually. Inheritance from string (Jason) This issue has been brought up before when people were discussing the path module. I think the consensus is that, while the inheritance isn't pure, practicality beats purity. It would require to big changes to Python and would break to much existing code to not extend string. I'll add this to Resolved Issues if nobody minds. * http://mail.python.org/pipermail/python-dev/2001-August/016663.html * http://mail.python.org/pipermail/python-dev/2001-August/016665.html * http://mail.python.org/pipermail/python-list/2005-July/291091.html * http://mail.python.org/pipermail/python-list/2005-July/291152.html Remove __div__ (Ian, Jason, Michael, Oleg) This is one of those where everyone (me too) says "I don't care either way." If that is so, then I see no reason to change it unless someone can show a scenario in which it hurts readability. Plus, a few people have said that they like the shortcut. * http://mail.python.org/pipermail/python-list/2005-July/292251.html * http://mail.python.org/pipermail/python-dev/2005-June/054496.html * http://mail.python.org/pipermail/python-list/2005-July/291628.html * http://mail.python.org/pipermail/python-list/2005-July/291621.html Remove redundant methods (Eric, Ian, Jason, Ronald, Toby) I think it is a good idea to take out some of Path's methods. Especially the ones that exist as both methods and properties. I have updated the pep and dumped basename(), dirname(), splitdrive() and splitext(). I think that glob() should also go away because I can't of the top of my head think of a scenario where it would be suitable over listdir(), dirs() or files(). Plus, for contrived examples; like glob.glob("/*bin/*foo*") the Path way doesn't look so good: Path("/").glob("*bin/*foo*"). Renaming methods because of PEP 8 (Gustavo, Ian, Jason) I'm personally not keen on that. I like most of the names as they are. abspath(), joinpath(), realpath() and splitall() looks so much better than abs_path(), join_path(), real_path() and split_all() in my eyes. If someone like the underscores I'll add it to Open Issues. Removing open() and methods that manipulate the whole file at once (Ian, Jason) I think this is settled with the addition of the with statement? My idea when scrubbing these methods was that it would make it easier to get the PEP accepted. However, even with with, these methods save some typing. * http://mail.python.org/pipermail/python-dev/2005-June/054439.html * http://mail.python.org/pipermail/python-list/2005-July/291435.html ?time properties and get?time() methods Clearly, Path has either the properties or the methods, not both at once. Yet another "I don't care either way." * http://mail.python.org/pipermail/python-dev/2005-June/054439.html * http://mail.python.org/pipermail/python-list/2005-July/291424.html * http://mail.python.org/pipermail/python-list/2005-July/291460.html I have also the corrected the copy-paste errors I inadvertedly introduced. Path should *not* have an __iter__. :) * match() and matchcase() wraps the fnmatch.fnmatch() and fnmatch.fnmatchcase() functions. I believe that the renaming is uncontroversial and that the introduction of matchcase() makes it so the whole fnmatch module can be deprecated. I have created an implementation of Path that corresponds to the specification in the PEP (which I hope will appear on www.python.org/peps soon). It is based on Reinhold's (Georg Brandl) implementation from pre-PEP threads in c.l.p last summer. But I have no place to upload it. I would also like if some people wants to co-author this PEP with me - it's really neither my PEP nor my module. -- mvh Bj?rn From phd at mail2.phd.pp.ru Wed Jan 25 21:59:26 2006 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Wed, 25 Jan 2006 23:59:26 +0300 Subject: [Python-Dev] The path module PEP In-Reply-To: <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> Message-ID: <20060125205926.GA25426@phd.pp.ru> On Wed, Jan 25, 2006 at 09:37:04PM +0100, BJ?rn Lindqvist wrote: > Remove __div__ (Ian, Jason, Michael, Oleg) I didn't say "remove". Exactly opposite - I am enamoured by the beauty of the syntax! (-: Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From ianb at colorstudy.com Wed Jan 25 22:03:44 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 25 Jan 2006 15:03:44 -0600 Subject: [Python-Dev] The path module PEP In-Reply-To: <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> Message-ID: <43D7E7B0.8070209@colorstudy.com> BJ?rn Lindqvist wrote: > * match() and matchcase() wraps the fnmatch.fnmatch() and > fnmatch.fnmatchcase() functions. I believe that the renaming is > uncontroversial and that the introduction of matchcase() makes it so > the whole fnmatch module can be deprecated. The renaming is fine with me. I generally use the fnmatch module for wildcard matching, not necessarily against path names. Path.match doesn't replace that functionality. Though fnmatch.translate isn't even publically documented, which is the function I actually tend to use. Though it seems a little confusing to me that glob treats separators specially, and that's not implemented at the fnmatch level. So Path('/a/b/d/c').match('a/*/d') is true, but Path('/').walk('a/*/d') won't return Path('/a/b/c/d'). I think .match() should be fixed. But I don't think fnmatch should be changed. I'm actually finding myself a little confused by the glob arguments (if the glob contains '/'), now that I really think about them. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From ianb at colorstudy.com Wed Jan 25 22:05:33 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 25 Jan 2006 15:05:33 -0600 Subject: [Python-Dev] The path module PEP In-Reply-To: References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D69E2A.6050205@colorstudy.com> Message-ID: <43D7E81D.8090007@colorstudy.com> John J Lee wrote: > On Tue, 24 Jan 2006, Ian Bicking wrote: > [...] > >>Losing .open() would make it much harder for anyone wanting to write, >>say, a URI library that implements the Path API. > > [...] > > Why? Could you expand a bit? > > What's wrong with urlopen(filesystem_path_instance) ? My example shows this more clearly I think: def read_config(path): text = path.open().read() ... do something ... If I implement a URI object with an .open() method, then I can use it with this function, even though read_config() was written with file paths in mind. But without it that won't work: def read_config(path): text = open(path).read() ... -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From jjl at pobox.com Wed Jan 25 22:30:02 2006 From: jjl at pobox.com (John J Lee) Date: Wed, 25 Jan 2006 21:30:02 +0000 (UTC) Subject: [Python-Dev] The path module PEP In-Reply-To: <43D7E81D.8090007@colorstudy.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D69E2A.6050205@colorstudy.com> <43D7E81D.8090007@colorstudy.com> Message-ID: [Ian Bicking] >Losing .open() would make it much harder for anyone wanting to write, >say, a URI library that implements the Path API. [John] > Why? Could you expand a bit? > > What's wrong with urlopen(filesystem_path_instance) ? [Ian] > def read_config(path): > text = path.open().read() > ... do something ... I should have expected that answer, but couldn't believe that you think it's a good idea to implement that obese filesystem path API for URLs ;-) Shouldn't we instead have: a) .open()-able objects blessed in the stdlib & stdlib docs, as a separate interface from the path interface (I guess that would be an argument in favour of path implementing that one-method interface, as long as it's not tied too tightly to the fat path interface) b) a path object with a thinner interface (I know you've already expressed that preference yourself...)? John From jjl at pobox.com Wed Jan 25 22:01:21 2006 From: jjl at pobox.com (John J Lee) Date: Wed, 25 Jan 2006 21:01:21 +0000 (UTC) Subject: [Python-Dev] The path module PEP In-Reply-To: <43D69E2A.6050205@colorstudy.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D69E2A.6050205@colorstudy.com> Message-ID: On Tue, 24 Jan 2006, Ian Bicking wrote: [...] > Losing .open() would make it much harder for anyone wanting to write, > say, a URI library that implements the Path API. [...] Why? Could you expand a bit? What's wrong with urlopen(filesystem_path_instance) ? John From t-meyer at ihug.co.nz Wed Jan 25 23:25:57 2006 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Thu, 26 Jan 2006 11:25:57 +1300 Subject: [Python-Dev] The path module PEP In-Reply-To: <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> Message-ID: <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> > Remove __div__ (Ian, Jason, Michael, Oleg) > > This is one of those where everyone (me too) says "I don't care either > way." If that is so, then I see no reason to change it unless someone > can show a scenario in which it hurts readability. Plus, a few people > have said that they like the shortcut. > > * http://mail.python.org/pipermail/python-list/2005-July/292251.html > * http://mail.python.org/pipermail/python-dev/2005-June/054496.html > * http://mail.python.org/pipermail/python-list/2005-July/291628.html > * http://mail.python.org/pipermail/python-list/2005-July/291621.html Well, if you include the much larger discussion on python-list, people (including me) have said that removing __div__ is a good idea. If it's included in the PEP, please at least include a justification and cover the problems with it. The vast majority of people (at least at the time) were either +0 or -0, not +1. +0's are not justification for including something. Against it: * Zen: Beautiful is better than ugly. Explicit is better than implicit. Readability counts. There should be one-- and preferably only one --obvious way to do it. * Not every platform that Python supports has '/' as the path separator. Windows, a pretty major one, has '\'. I have no idea what various portable devices use, but there's a reasonable chance it's not '/'. * It's being used to mean "join", which is the exact opposite of /'s other meaning ("divide"). * Python's not Perl. We like using functions and not symbols. > Renaming methods because of PEP 8 (Gustavo, Ian, Jason) > > I'm personally not keen on that. I like most of the names as they > are. abspath(), joinpath(), realpath() and splitall() looks so much > better than abs_path(), join_path(), real_path() and split_all() in my > eyes. If someone like the underscores I'll add it to Open Issues. +1 to following PEP 8. These aren't built-ins, it's a library module. In addition to the PEP, underscores make it much easier to read, especially for those for whom English is not their first language. =Tony.Meyer From pje at telecommunity.com Wed Jan 25 23:54:04 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 25 Jan 2006 17:54:04 -0500 Subject: [Python-Dev] The path module PEP In-Reply-To: <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> References: <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> Message-ID: <5.1.1.6.0.20060125174748.04df9540@mail.telecommunity.com> At 11:25 AM 1/26/2006 +1300, Tony Meyer wrote: >Against it: > > * Zen: Beautiful is better than ugly. Explicit is better than >implicit. Readability counts. There should be one-- and preferably >only one --obvious way to do it. > > * Not every platform that Python supports has '/' as the path >separator. Windows, a pretty major one, has '\'. "/" also works on Windows, and the Python distutils already set the precedent of requiring /-separated paths on *all* platforms, converting them to os.sep behind the scenes. I'd also note that using the / operator seems to me to be a big win on "beautiful is better than ugly". Path-joining code is mighty ugly without it, and / is more readable as well. It'd be nice to see the urllib modules grow a URL type supporting this operator, among other path operators. I would also suggest that as with the individual posixpath, ntpath, etc. libraries today, we should be able to import NTPath and PosixPath classes directly from those modules, for code that needs to manipulate a path for some system other than the one it's running on. From ianb at colorstudy.com Thu Jan 26 00:13:37 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 25 Jan 2006 17:13:37 -0600 Subject: [Python-Dev] The path module PEP In-Reply-To: <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> Message-ID: <43D80621.1080300@colorstudy.com> Tony Meyer wrote: >>Remove __div__ (Ian, Jason, Michael, Oleg) >> >>This is one of those where everyone (me too) says "I don't care either >>way." If that is so, then I see no reason to change it unless someone >>can show a scenario in which it hurts readability. Plus, a few people >>have said that they like the shortcut. >> >>* http://mail.python.org/pipermail/python-list/2005-July/292251.html >>* http://mail.python.org/pipermail/python-dev/2005-June/054496.html >>* http://mail.python.org/pipermail/python-list/2005-July/291628.html >>* http://mail.python.org/pipermail/python-list/2005-July/291621.html > > > Well, if you include the much larger discussion on python-list, > people (including me) have said that removing __div__ is a good > idea. If it's included in the PEP, please at least include a > justification and cover the problems with it. The vast majority of > people (at least at the time) were either +0 or -0, not +1. +0's are > not justification for including something. If it were possible to use .join() for joining paths, I think I wouldn't mind so much. But reusing a string method for something very different seems like a bad idea. So we're left with .joinpath(). Still better than os.path.join() I guess, but only a little. I guess that's why I'm +1 on /. > Against it: > > * Zen: Beautiful is better than ugly. Explicit is better than > implicit. Readability counts. There should be one-- and preferably > only one --obvious way to do it. I think / is pretty. I think it reads well. There's already some inevitable redundancy in this interface. I use os.path.join so much that I know anything I use will feel readable quickly, but I also think I'll find / more appealing. > * Not every platform that Python supports has '/' as the path > separator. Windows, a pretty major one, has '\'. I have no idea > what various portable devices use, but there's a reasonable chance > it's not '/'. I believe all platforms support /; at least Windows and Mac do, in addition to their native separators. I assume any platform that supports filesystem access will support / in Python. If anything, a good shortcut for .joinpath() will at least encourage people to use it, thus discouraging hardcoding of path separators. I expect it would encourage portable paths. Though Path('/foo') / '/bar' == Path('/bar'), which is *not* intuitive, though in the context of "join" it's not as surprising. So that is a problem. If / meant "under this path" then that could be a useful operator (in that I'd really like such an operator or method). Either paths would be forced to be under the original path, or it would be an error if they somehow escaped. Currently there's no quick-and-easy way to ensure this, except to join the paths, do abspath(), then confirm that the new path starts with the old path. > * It's being used to mean "join", which is the exact opposite > of /'s other meaning ("divide"). > > * Python's not Perl. We like using functions and not symbols. A little too heavy on the truisms. Python isn't the anti-Perl. >>Renaming methods because of PEP 8 (Gustavo, Ian, Jason) >> >>I'm personally not keen on that. I like most of the names as they >>are. abspath(), joinpath(), realpath() and splitall() looks so much >>better than abs_path(), join_path(), real_path() and split_all() in my >>eyes. If someone like the underscores I'll add it to Open Issues. > > > +1 to following PEP 8. These aren't built-ins, it's a library > module. In addition to the PEP, underscores make it much easier to > read, especially for those for whom English is not their first language. I don't find abs_path() much easier to read than abspath() -- neither is a full name. absolute_path() perhaps, but that is somewhat redundant; absolute()...? Eh. Precedence in naming means something, and in this case all the names have existed for a very long time (as long as Python?) PEP 8 encourages following naming precedence. While I don't see a need to match every existing function with a method, to the degree they do match I see no reason why we shouldn't keep the names. And I see reasons why the names shouldn't be changed. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From python at discworld.dyndns.org Thu Jan 26 00:34:35 2006 From: python at discworld.dyndns.org (Charles Cazabon) Date: Wed, 25 Jan 2006 17:34:35 -0600 Subject: [Python-Dev] The path module PEP In-Reply-To: <5.1.1.6.0.20060125174748.04df9540@mail.telecommunity.com> References: <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <5.1.1.6.0.20060125174748.04df9540@mail.telecommunity.com> Message-ID: <20060125233435.GD742@discworld.dyndns.org> Phillip J. Eby wrote: > > I'd also note that using the / operator seems to me to be a big win on > "beautiful is better than ugly". It screams "magic" in a very un-Pythonic (and possibly very Perl-like) way. I'm not aware of any other part of the standard library grossly abusing standard operators in this way. As others have noted, "/" is being used here to mean precisely the opposite of what it means in every other use in Python, which alone should be justification for getting rid of it. Charles -- ----------------------------------------------------------------------- Charles Cazabon GPL'ed software available at: http://pyropus.ca/software/ ----------------------------------------------------------------------- From jjl at pobox.com Thu Jan 26 00:48:12 2006 From: jjl at pobox.com (John J Lee) Date: Wed, 25 Jan 2006 23:48:12 +0000 (UTC) Subject: [Python-Dev] / as path join operator (was: Re: The path module PEP) In-Reply-To: <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> Message-ID: On Thu, 26 Jan 2006, Tony Meyer wrote: [...] > Well, if you include the much larger discussion on python-list, > people (including me) have said that removing __div__ is a good > idea. If it's included in the PEP, please at least include a > justification and cover the problems with it. The vast majority of > people (at least at the time) were either +0 or -0, not +1. +0's are > not justification for including something. FWLIW, I'm definitely +1 on using / as a path join operator. > * It's being used to mean "join", which is the exact opposite > of /'s other meaning ("divide"). But it's a very readable way to write a common operation. Perhaps one reason the discrepancy you point out doesn't bother me is that division is the least-used of the +-*/ arithmetic operations. Also, &, | and ^ seem like some sort of precedent, to my brain (if they don't to yours, that's fine and I believe you ;-). > * Python's not Perl. We like using functions and not symbols. I think this is a tasteful, if not parsimonious, use of a symbol. John From ianb at colorstudy.com Thu Jan 26 00:49:51 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 25 Jan 2006 17:49:51 -0600 Subject: [Python-Dev] The path module PEP In-Reply-To: <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> Message-ID: <43D80E9F.4070009@colorstudy.com> BJ?rn Lindqvist wrote: > Remove __div__ (Ian, Jason, Michael, Oleg) > > This is one of those where everyone (me too) says "I don't care either > way." If that is so, then I see no reason to change it unless someone > can show a scenario in which it hurts readability. Plus, a few people > have said that they like the shortcut. > > * http://mail.python.org/pipermail/python-list/2005-July/292251.html > * http://mail.python.org/pipermail/python-dev/2005-June/054496.html > * http://mail.python.org/pipermail/python-list/2005-July/291628.html > * http://mail.python.org/pipermail/python-list/2005-July/291621.html Curious how often I use os.path.join and division, I searched a project of mine, and in 12k lines there were 34 uses of join, and 1 use of division. In smaller scripts os.path.join tends to show up a lot more (per line). I'm sure there's people who use division far more than I, and os.path.join less, but I'm guessing the majority of users are more like me. That's not necessarily a justification of / for paths, but at least this use for "/" wouldn't be obscure or mysterious after you get a little experience seeing code that uses it. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From t-meyer at ihug.co.nz Thu Jan 26 00:42:13 2006 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Thu, 26 Jan 2006 12:42:13 +1300 Subject: [Python-Dev] The path module PEP In-Reply-To: <43D80621.1080300@colorstudy.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> <43D80621.1080300@colorstudy.com> Message-ID: [Ian Bicking] > If it were possible to use .join() for joining paths, I think I > wouldn't mind so much. But reusing a string method for something > very different seems like a bad idea. So we're left with .joinpath > (). Still better than os.path.join() I guess, but only a little. > I guess that's why I'm +1 on /. Why does reusing a string method for something very different seem like a bad idea, but reusing a mathematical operator for something very different seem like a good idea? Path's aren't strings, so join () seems the logical choice. (There are also alternatives to "joinpath" if the name is the thing: add(), for example). [Tony Meyer] >> Against it: >> * Zen: Beautiful is better than ugly. Explicit is better than >> implicit. Readability counts. There should be one-- and >> preferably only one --obvious way to do it. > > I think / is pretty. I think it reads well. I suppose that the only beholder's eye that matters here is Guido's. (It still violates explicit/implicit and only-one-way. Not rules, of course, but good guidelines). > There's already some inevitable redundancy in this interface. That's hardly a reason to encourage *more*. If anything, it's a reason to try for less, where possible. >> * Not every platform that Python supports has '/' as the path >> separator. Windows, a pretty major one, has '\'. I have no idea >> what various portable devices use, but there's a reasonable >> chance it's not '/'. > > I believe all platforms support /; at least Windows and Mac do, in > addition to their native separators. This is not strictly true. Using '/' can lead to strange results with Windows, where it gets interpreted as a flag instead. It's not reliable, it's not the path separator that Windows users/developers understand, and it's not the correct (i.e. according to Microsoft) path separator. If by Mac you mean OS X, then that's just another *nix based OS. I'm pretty sure that pre OS X (which isn't supported any more anyway, right?) '/' was not, in fact, supported, and that ":" was required. I also believe it's important to remember that Windows and *nix descendants are not "all platforms". > If anything, a good shortcut for .joinpath() will at least > encourage people to use it, thus discouraging hardcoding of path > separators. I expect it would encourage portable paths. I'm not sure that I believe that the reason that people don't type "os.path.join('a', 'b')" is because they are too lazy to type it. However, I don't have any evidence either way, so it could be true. [re: PEP8 following] > Precedence in naming means something, and in this case all the > names have existed for a very long time (as long as Python?) PEP 8 > encourages following naming precedence. While I don't see a need > to match every existing function with a method, to the degree they > do match I see no reason why we shouldn't keep the names. And I > see reasons why the names shouldn't be changed. PEP 8 encourages following naming precedence within a module, doesn't it? Guido has said that he'd like to have the standard library tidied up, at least somewhat (e.g. StringIO.StringIO -> stringio.StringIO) for Python 3000. It would make it less painful if new additions already followed the plan. =Tony.Meyer From t-meyer at ihug.co.nz Thu Jan 26 00:51:04 2006 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Thu, 26 Jan 2006 12:51:04 +1300 Subject: [Python-Dev] / as path join operator (was: Re: The path module PEP) In-Reply-To: References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> Message-ID: <034C8957-94EF-4F36-906D-D4EC57221FAF@ihug.co.nz> [John J Lee] > But it's a very readable way to write a common operation. Perhaps one > reason the discrepancy you point out doesn't bother me is that > division is > the least-used of the +-*/ arithmetic operations. Do you have evidence to back that up? It seems a strange claim. Outside of doing 'maths-y' work, I would think I'd use + most (but for strings), then / (for percentages). > Also, &, | and ^ seem like some sort of precedent, to my brain (if > they > don't to yours, that's fine and I believe you ;-). I don't follow this, sorry. You're referring to the bitwise operations? [Ian Bicking] > Curious how often I use os.path.join and division, I searched a > project > of mine, and in 12k lines there were 34 uses of join, and 1 use of > division. In smaller scripts os.path.join tends to show up a lot more > (per line). I'm sure there's people who use division far more than I, > and os.path.join less, but I'm guessing the majority of users are more > like me. The problem with these sorts of guesses is that there's no evidence. (Maybe the suggestion that Brett's PhD should collect a corpus of Python scripts was a good one ). Are mathematicians that under represented? Is file processing that highly represented? I have no idea. =Tony.Meyer From bob at redivi.com Thu Jan 26 01:02:20 2006 From: bob at redivi.com (Bob Ippolito) Date: Wed, 25 Jan 2006 16:02:20 -0800 Subject: [Python-Dev] The path module PEP In-Reply-To: References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> <43D80621.1080300@colorstudy.com> Message-ID: <8B530061-63C5-46F6-8081-F56ECF2EEC06@redivi.com> On Jan 25, 2006, at 3:42 PM, Tony Meyer wrote: > [Ian Bicking] >> If it were possible to use .join() for joining paths, I think I >> wouldn't mind so much. But reusing a string method for something >> very different seems like a bad idea. So we're left with .joinpath >> (). Still better than os.path.join() I guess, but only a little. >> I guess that's why I'm +1 on /. > > Why does reusing a string method for something very different seem > like a bad idea, but reusing a mathematical operator for something > very different seem like a good idea? Path's aren't strings, so join > () seems the logical choice. (There are also alternatives to > "joinpath" if the name is the thing: add(), for example). join() is already defined for strings, division is not. Different namespace... just like + is concatenation for list+list, tuple+tuple, basestring+basestring, but it's addition for numbers... >>> * Not every platform that Python supports has '/' as the path >>> separator. Windows, a pretty major one, has '\'. I have no idea >>> what various portable devices use, but there's a reasonable >>> chance it's not '/'. >> >> I believe all platforms support /; at least Windows and Mac do, in >> addition to their native separators. > > This is not strictly true. Using '/' can lead to strange results > with Windows, where it gets interpreted as a flag instead. It's not > reliable, it's not the path separator that Windows users/developers > understand, and it's not the correct (i.e. according to Microsoft) > path separator. If by Mac you mean OS X, then that's just another > *nix based OS. I'm pretty sure that pre OS X (which isn't supported > any more anyway, right?) '/' was not, in fact, supported, and that > ":" was required. I also believe it's important to remember that > Windows and *nix descendants are not "all platforms". Mac OS X understands '/' as the separator at the POSIX layer, but ':' as the path separator at the Carbon API (which is only used in obscure places from Python). Earlier versions of Mac OS are no longer supported, and you're right -- they only understood ':' as a path separator. >> If anything, a good shortcut for .joinpath() will at least >> encourage people to use it, thus discouraging hardcoding of path >> separators. I expect it would encourage portable paths. > > I'm not sure that I believe that the reason that people don't type > "os.path.join('a', 'b')" is because they are too lazy to type it. > However, I don't have any evidence either way, so it could be true. In many cases, when I know I only care about *nix, I will use 'a/b' instead of os.path.join because it's just so much more concise and more obvious. The only times I use os.path.join are when I don't know if there will be a trailing slash or not, or if I'm actively trying to make something cross-platform. -bob From jjl at pobox.com Thu Jan 26 01:02:33 2006 From: jjl at pobox.com (John J Lee) Date: Thu, 26 Jan 2006 00:02:33 +0000 (UTC) Subject: [Python-Dev] The path module PEP In-Reply-To: References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> <43D80621.1080300@colorstudy.com> Message-ID: On Thu, 26 Jan 2006, Tony Meyer wrote: [...] > Why does reusing a string method for something very different seem > like a bad idea, but reusing a mathematical operator for something > very different seem like a good idea? [...] That's easy -- it's because, if you're going to use a name, people expect (with some level of trust) that you'll pick a good one. But people understand that there are only a few operators to use, so the meaning of operators is naturally more overloaded than that of method names. John From ianb at colorstudy.com Thu Jan 26 01:10:27 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 25 Jan 2006 18:10:27 -0600 Subject: [Python-Dev] The path module PEP In-Reply-To: References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> <43D80621.1080300@colorstudy.com> Message-ID: <43D81373.8030201@colorstudy.com> Tony Meyer wrote: > [Ian Bicking] > >> If it were possible to use .join() for joining paths, I think I >> wouldn't mind so much. But reusing a string method for something >> very different seems like a bad idea. So we're left with .joinpath >> (). Still better than os.path.join() I guess, but only a little. I >> guess that's why I'm +1 on /. > > > Why does reusing a string method for something very different seem like > a bad idea, but reusing a mathematical operator for something very > different seem like a good idea? Path's aren't strings, so join () > seems the logical choice. (There are also alternatives to "joinpath" > if the name is the thing: add(), for example). Paths are strings, that's in the PEP. As an aside, I think it should be specified what (if any) string methods won't be inherited by Path (or will be specifically disabled by making them throw some exception). I think .join() and __iter__ at least should be disabled. >> Precedence in naming means something, and in this case all the names >> have existed for a very long time (as long as Python?) PEP 8 >> encourages following naming precedence. While I don't see a need to >> match every existing function with a method, to the degree they do >> match I see no reason why we shouldn't keep the names. And I see >> reasons why the names shouldn't be changed. > > > PEP 8 encourages following naming precedence within a module, doesn't > it? Guido has said that he'd like to have the standard library tidied > up, at least somewhat (e.g. StringIO.StringIO -> stringio.StringIO) for > Python 3000. It would make it less painful if new additions already > followed the plan. I think the use of underscores or squished words isn't as bit a deal as the case of modules. It's often rather ambiguous what a "word" really is. At least in English word combinations slowly and ambiguously float towards being combined. So abspath and abs_path both feel sufficiently inside the scope of PEP 8 that precedence is worth maintaining. rfc822's getallmatchingheaders method was going too far, but a little squishing doesn't bother me, if it is consistent (and it's actually easier to be consistent about squishing than underscores). -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From hoffman at ebi.ac.uk Thu Jan 26 01:22:36 2006 From: hoffman at ebi.ac.uk (Michael Hoffman) Date: Thu, 26 Jan 2006 00:22:36 +0000 Subject: [Python-Dev] The path module PEP In-Reply-To: <20060125233435.GD742@discworld.dyndns.org> References: <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <5.1.1.6.0.20060125174748.04df9540@mail.telecommunity.com> <20060125233435.GD742@discworld.dyndns.org> Message-ID: [Charles Cazabon] > It screams "magic" in a very un-Pythonic (and possibly very > Perl-like) way. I'm not aware of any other part of the standard > library grossly abusing standard operators in this way. I think the use of the modulo operator for string substitution is pretty comparable, despite it being in the interpreter rather than in the stdlib. And some of us have come to love that, too. -- Michael Hoffman European Bioinformatics Institute From gjc at inescporto.pt Thu Jan 26 01:30:58 2006 From: gjc at inescporto.pt (Gustavo J. A. M. Carneiro) Date: Thu, 26 Jan 2006 00:30:58 +0000 Subject: [Python-Dev] The path module PEP In-Reply-To: <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> Message-ID: <1138235459.6668.14.camel@localhost.localdomain> On Wed, 2006-01-25 at 21:37 +0100, BJ?rn Lindqvist wrote: > Renaming methods because of PEP 8 (Gustavo, Ian, Jason) > > I'm personally not keen on that. I like most of the names as they > are. abspath(), joinpath(), realpath() and splitall() looks so much > better than abs_path(), join_path(), real_path() and split_all() in my > eyes. If someone like the underscores I'll add it to Open Issues. Function Names Function names should be lowercase, with words separated by underscores as necessary to improve readability. mixedCase is allowed only in contexts where that's already the prevailing style (e.g. threading.py), to retain backwards compatibility. Method Names and Instance Variables Use the function naming rules: lowercase with words separated by underscores as necessary to improve readability. It is very clear. Whether you agree with PEP 8 or not is not relevant to this discussion. Since this is a completely new module, it should be correctly named from the start. The "familiarity with os.path argument" is a very weak one, IMHO. Plus, the names are full of redundancy. Why abspath(), joinpath(), realpath(), splitall()? Why not instead: absolute(), join(), real(), split() ? Remember that they are all methods of a Path class, you don't need to keep repeating 'path' all over the place[1]. On a slightly different subject, regarding path / path, I think it feels much more natural path + path. Path.join is really just a string concatenation, except that it adds a path separator in the middle if necessary, if I'm not mistaken. Best regards. [1] Yes, I'm the kind of guy who hates struct timeval having tv_sec and tv_usec field members instead of sec and usec. -- Gustavo J. A. M. Carneiro The universe is always one step beyond logic From thomas at xs4all.net Thu Jan 26 01:41:13 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Thu, 26 Jan 2006 01:41:13 +0100 Subject: [Python-Dev] The path module PEP In-Reply-To: <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> Message-ID: <20060126004113.GD18916@xs4all.nl> On Wed, Jan 25, 2006 at 09:37:04PM +0100, BJ?rn Lindqvist wrote: > Inheritance from string (Jason) > This issue has been brought up before when people were discussing the > path module. I think the consensus is that, while the inheritance > isn't pure, practicality beats purity. It would require to big changes > to Python and would break to much existing code to not extend string. This is my only problem with the PEP. It's all very nice that subclassing from string makes it easier not to break things, but subclassing implies a certain relationship. That relationship doesn't exist, in this case. Having the string subclass behave differently than strings (consider the __iter__ and join methods) is a bad idea. I can dish up a half dozen contrived problem cases, but the main reason I don't like it is that it feels wrong. If the reason to subclass string is that it's too hard to make an object 'string-like' at a low enough level for the C API, I suggest fixing that, instead. If that means Path has to wait until Python 2.6, then that's too bad. The inability to feed C functions/types open() non-string objects has troubled me before, and though I haven't invested a lot of time in it, I don't quite understand why it isn't possible. Fixing it in a backward-compatible manner may require a new __hook__, although I think using __str__ or __unicode__ shouldn't be too problematic. Even if fixing the "%es"/"%et" etc formats to the arg-parsing methods is infeasible, I would prefer a new format code for 'string-alike as C string' over the unpythonic inappropriate subclassing. Although, even if the subclassing was completely appropriate, I would much rather improve builtin support for ducktyping than showcase subclassing ;) But perhaps I've missed that single, all-convincing argument that it has use subclassing... In that case, could it be added to the PEP? :) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From jjl at pobox.com Thu Jan 26 01:43:29 2006 From: jjl at pobox.com (John J Lee) Date: Thu, 26 Jan 2006 00:43:29 +0000 (UTC) Subject: [Python-Dev] / as path join operator (was: Re: The path module PEP) In-Reply-To: <034C8957-94EF-4F36-906D-D4EC57221FAF@ihug.co.nz> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> <034C8957-94EF-4F36-906D-D4EC57221FAF@ihug.co.nz> Message-ID: [John J Lee] > But it's a very readable way to write a common operation. Perhaps one > reason the discrepancy you point out doesn't bother me is that > division is the least-used of the +-*/ arithmetic operations. [Tony Meyer] > > Do you have evidence to back that up? No. :) [Ian Bicking] > of mine, and in 12k lines there were 34 uses of join, and 1 use of > division. In smaller scripts os.path.join tends to show up a lot more [Tony] > The problem with these sorts of guesses is that there's no evidence. > (Maybe the suggestion that Brett's PhD should collect a corpus of > Python scripts was a good one ). Are mathematicians that under > represented? Is file processing that highly represented? I have no > idea. A second data point: I looked at ~10k lines of physical data analysis code I have lying around -- presumably a relatively rare and extreme example as the Python-world in general goes. Result: 140 occurences of os.path.join 170 physical lines (as opposed to syntactical lines) containing / as a division operator (very few lines contained > 1 use of '/', so you can multiply 170 by 1.25 to get an upper bound of 213 uses in total) (To get the second number, I used find and grep heavily but very cautiously, and final manual count of stubborn lines of grep output with no use of '/' as division operator) The fact that even in this extreme case os.path.join is close on the tail of '/' strongly backs up Ian's guess that, in most Python code, / as division is rare compared to path joining. Should we deprecate use of '/' and '//' for division in Python 3.0? is-he-joking?-ly y'rs John From steven.bethard at gmail.com Thu Jan 26 01:45:20 2006 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed, 25 Jan 2006 17:45:20 -0700 Subject: [Python-Dev] / as path join operator (was: Re: The path module PEP) In-Reply-To: References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> Message-ID: John J Lee wrote: > On Thu, 26 Jan 2006, Tony Meyer wrote: > [...] > > Well, if you include the much larger discussion on python-list, > > people (including me) have said that removing __div__ is a good > > idea. If it's included in the PEP, please at least include a > > justification and cover the problems with it. The vast majority of > > people (at least at the time) were either +0 or -0, not +1. +0's are > > not justification for including something. > > > > FWLIW, I'm definitely +1 on using / as a path join operator. My only fear with the / operator is that we'll end up with the same problems we have for using % in string formatting -- the order of operations might not be what users expect. Since join is conceptually an addition-like operator, I would expect: Path('home') / 'a' * 5 to give me: home/aaaaa If I understand it right, it would actually give me something like: home/ahome/ahome/ahome/ahome/a I don't want to claim this is the most common use case, but I've certainly seen auto-generated paths that look like 'a' * 20, and it would be a pity if using the / operator for Path objects did the wrong thing by default here... STeVe -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From t-meyer at ihug.co.nz Thu Jan 26 02:22:16 2006 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Thu, 26 Jan 2006 14:22:16 +1300 Subject: [Python-Dev] The path module PEP In-Reply-To: <43D81373.8030201@colorstudy.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> <43D80621.1080300@colorstudy.com> <43D81373.8030201@colorstudy.com> Message-ID: [Ian Bicking] > Paths are strings, that's in the PEP. No, the PEP says that Path is a *subclass* of string ("Path extends from string"). In addition, it's a disputed part of the PEP (see elsewhere in this thread). =Tony.Meyer From ncoghlan at gmail.com Thu Jan 26 02:40:15 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 26 Jan 2006 11:40:15 +1000 Subject: [Python-Dev] / as path join operator In-Reply-To: References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> Message-ID: <43D8287F.70209@gmail.com> Steven Bethard wrote: > John J Lee wrote: >> On Thu, 26 Jan 2006, Tony Meyer wrote: >> [...] >>> Well, if you include the much larger discussion on python-list, >>> people (including me) have said that removing __div__ is a good >>> idea. If it's included in the PEP, please at least include a >>> justification and cover the problems with it. The vast majority of >>> people (at least at the time) were either +0 or -0, not +1. +0's are >>> not justification for including something. >> >> >> FWLIW, I'm definitely +1 on using / as a path join operator. > > My only fear with the / operator is that we'll end up with the same > problems we have for using % in string formatting -- the order of > operations might not be what users expect. Since join is conceptually > an addition-like operator, I would expect: > > Path('home') / 'a' * 5 > > to give me: > > home/aaaaa > > If I understand it right, it would actually give me something like: > > home/ahome/ahome/ahome/ahome/a > > I don't want to claim this is the most common use case, but I've > certainly seen auto-generated paths that look like 'a' * 20, and it > would be a pity if using the / operator for Path objects did the wrong > thing by default here... What if we used "subpath" as the name instead of joinpath? The main appeal to me of the division operation is that it allows multiple path elements to be joined on a single line, but the joining method accepts an arbitrary number of arguments, which helps with that just as much, and doesn't raise precedence and readability questions. The above example would be: Path('home').subpath('a'*5) An example of retrieving a config file's full name: Current: os.path.join(HOME_DIR, APP_DIR, CONFIG_FILE) Division: HOME_DIR / APP_DIR / CONFIG_FILE Subpath: HOME_DIR.subpath(APP_DIR, CONFIG_FILE) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From tim.peters at gmail.com Thu Jan 26 02:35:17 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 25 Jan 2006 20:35:17 -0500 Subject: [Python-Dev] [Python-checkins] r42185 - python/trunk/Lib/test/test_socket_ssl.py In-Reply-To: <20060125083940.E45321E4002@bag.python.org> References: <20060125083940.E45321E4002@bag.python.org> Message-ID: <1f7befae0601251735k3b000b97u6ba0f9a03af80ad4@mail.gmail.com> [neal.norwitz] > Modified: > python/trunk/Lib/test/test_socket_ssl.py > Log: > There was a race condition where the connector would try to connect > before the listener was ready (on gentoo x86 buildslave). This > caused the listener to not exit normally since nobody connected to it > (waited in accept()). The exception was raised in the other thread > and the test failed. Good catch! Thank you. > This fix doesn't completely eliminate the race, but should make it > near impossible to trigger. Hopefully it's good enough. Which race do you have in mind? The server socket doesn't need to do .accept() before a client socket can connect -- the server socket only needs to have done .listen() for a connection to succeed. > + listener_ready = threading.Event() ... [in the server] > s = socket.socket() > s.bind(('', PORT)) > s.listen(5) > + listener_ready.set() > s.accept() ... [in the client] > def connector(): > + listener_ready.wait() > s = socket.socket() > s.connect(('localhost', PORT)) Because the server doesn't set listener_ready until after the server has done listen(), and the client waits for that event, it "should be" 100% reliable that the client's connect() succeeds. Or do you have some other race in mind? From nnorwitz at gmail.com Thu Jan 26 02:54:05 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 25 Jan 2006 17:54:05 -0800 Subject: [Python-Dev] [Python-checkins] r42185 - python/trunk/Lib/test/test_socket_ssl.py In-Reply-To: <1f7befae0601251735k3b000b97u6ba0f9a03af80ad4@mail.gmail.com> References: <20060125083940.E45321E4002@bag.python.org> <1f7befae0601251735k3b000b97u6ba0f9a03af80ad4@mail.gmail.com> Message-ID: On 1/25/06, Tim Peters wrote: > > Because the server doesn't set listener_ready until after the server > has done listen(), and the client waits for that event, it "should > be" 100% reliable that the client's connect() succeeds. > > Or do you have some other race in mind? That's what I was thinking of. I thought you had to be accept()ing prior to connect() working. I thought listen() only sets the # of outstanding connections allowed (basically internal buffer). But if the listen() is sufficient, I agree there is no race. n From tim.peters at gmail.com Thu Jan 26 03:11:42 2006 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 25 Jan 2006 21:11:42 -0500 Subject: [Python-Dev] [Python-checkins] r42185 - python/trunk/Lib/test/test_socket_ssl.py In-Reply-To: References: <20060125083940.E45321E4002@bag.python.org> <1f7befae0601251735k3b000b97u6ba0f9a03af80ad4@mail.gmail.com> Message-ID: <1f7befae0601251811r5d76e908ge0441f9313e3ce1b@mail.gmail.com> [Neal Norwitz] > That's what I was thinking of. I thought you had to be accept()ing > prior to connect() working. I thought listen() only sets the # of > outstanding connections allowed (basically internal buffer). But if > the listen() is sufficient, I agree there is no race. listen() "should be" sufficient -- it allocates space for a connection queue, and marks the socket as accepting connections. accept() merely does a blocking "pop" on the queue listen() created. For fun ;-), run it in slow motion, by hand, on that box: open Python in one window, and create a server socket without doing accept(). In another window, try to connect to the first socket's address. The connect should succeed at once. Now socket docs are typically clear as mud, and it's possible that box has a bad implementation. "Thought experiments" can help: if an accept() were necessary before a connection could succeed, why would listen() have a backlog argument at all? Well, if everything else were sane, it wouldn't ;-) From trentm at ActiveState.com Thu Jan 26 03:39:44 2006 From: trentm at ActiveState.com (Trent Mick) Date: Wed, 25 Jan 2006 18:39:44 -0800 Subject: [Python-Dev] / as path join operator In-Reply-To: <43D8287F.70209@gmail.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> <43D8287F.70209@gmail.com> Message-ID: <20060126023944.GC11993@activestate.com> [Nick Coghlan wrote] > What if we used "subpath" as the name instead of joinpath? "append"? not-a-big-fan-of-joinpath-either-ly yours, Trent -- Trent Mick TrentM at ActiveState.com From xavier.morel at masklinn.net Fri Jan 20 20:14:22 2006 From: xavier.morel at masklinn.net (Morel Xavier) Date: Fri, 20 Jan 2006 20:14:22 +0100 Subject: [Python-Dev] yield back-and-forth? In-Reply-To: References: Message-ID: <43D1368E.1090001@masklinn.net> Christian Tanzer wrote: > How about: > > def main_generator(): > ... > yield * sub_generator() > > Ducking-ly yrs, > I like that one, but I'd stick the star to the generator (e.g. yield *sub_generator), the meaning being to "unpack" the generator into the yield, same as unpacking a sequence passed as function parameters. As an extension, the syntax may even (though I'm not sure it'd be a good idea to do so) work with any iterable, behaving the same way (yielding the successive values of the iterable object) From gabriel.becedillas at corest.com Tue Jan 24 17:15:23 2006 From: gabriel.becedillas at corest.com (Gabriel Becedillas) Date: Tue, 24 Jan 2006 13:15:23 -0300 Subject: [Python-Dev] pystate.c changes for Python 2.4.2 In-Reply-To: <43C68823.3000105@corest.com> References: <43C68823.3000105@corest.com> Message-ID: <43D6529B.8040909@corest.com> Thanks to everyone for the great support. I've submitted a patch for this: http://sourceforge.net/tracker/index.php?func=detail&aid=1413181&group_id=5470&atid=305470. From ianb at colorstudy.com Thu Jan 26 04:02:07 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 25 Jan 2006 21:02:07 -0600 Subject: [Python-Dev] / as path join operator In-Reply-To: References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> Message-ID: <43D83BAF.20401@colorstudy.com> Steven Bethard wrote: > My only fear with the / operator is that we'll end up with the same > problems we have for using % in string formatting -- the order of > operations might not be what users expect. Since join is conceptually > an addition-like operator, I would expect: > > Path('home') / 'a' * 5 > > to give me: > > home/aaaaa > > If I understand it right, it would actually give me something like: > > home/ahome/ahome/ahome/ahome/a Both of these examples are rather silly, of course ;) There's two operators currently used commonly with strings (that I assume Path would inherit): + and %. Both actually make sense with paths too. filename_template = '%(USER)s.conf' p = Path('/conf') / filename_template % os.environ which means: p = (Path('/conf') / filename_template) % os.environ But probably the opposite is intended. Still, it will usually be harmless. Which is sometimes worse than usually harmful. + seems completely innocuous, though: ext = '.jpg' name = fields['name'] image = Path('/images') / name + ext It doesn't really matter what order it happens in there. Assuming concatenation results in a new Path object, not a str. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From barry at python.org Thu Jan 26 04:51:08 2006 From: barry at python.org (Barry Warsaw) Date: Wed, 25 Jan 2006 22:51:08 -0500 Subject: [Python-Dev] / as path join operator (was: Re: The path module PEP) In-Reply-To: <034C8957-94EF-4F36-906D-D4EC57221FAF@ihug.co.nz> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> <034C8957-94EF-4F36-906D-D4EC57221FAF@ihug.co.nz> Message-ID: <1138247468.30252.11.camel@geddy.wooz.org> On Thu, 2006-01-26 at 12:51 +1300, Tony Meyer wrote: > [John J Lee] > > But it's a very readable way to write a common operation. Perhaps one > > reason the discrepancy you point out doesn't bother me is that > > division is > > the least-used of the +-*/ arithmetic operations. > > Do you have evidence to back that up? It seems a strange claim. > Outside of doing 'maths-y' work, I would think I'd use + most (but > for strings), then / (for percentages). I haven't followed the entire thread (I'll try to find time to catch up) but while I think using __div__ to mean path concatenation is cute, I'm not sure I'd like to see it all over the place. It does seem awfully "FAST" ("facinating and stomach turning" to use a term from years ago). What I don't like about os.path.join() having to import os and having to type all those characters over and over again. What I /like/ about os.path.join is that you can give it a bunch of path components and have it return the correctly joined path, e.g. os.path.join('a, 'b', 'c'). That seems more efficient than having to create a bunch of intermediate objects. All in all, I'd have to say I'm -0 on __div__ for path concatenation. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060125/763847fb/attachment.pgp From t-meyer at ihug.co.nz Thu Jan 26 05:03:00 2006 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Thu, 26 Jan 2006 17:03:00 +1300 Subject: [Python-Dev] The path module PEP In-Reply-To: <1138235459.6668.14.camel@localhost.localdomain> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <1138235459.6668.14.camel@localhost.localdomain> Message-ID: <79484C59-6A15-40F4-998B-0DFBA742D272@ihug.co.nz> [Gustavo J. A. M. Carneiro] > Plus, the names are full of redundancy. Why abspath(), joinpath(), > realpath(), splitall()? Why not instead: absolute(), join(), real(), > split() ? Remember that they are all methods of a Path class, you > don't > need to keep repeating 'path' all over the place. +1 for all of those that aren't also string methods. +0.9 for those that are string methods, although I suppose this depends on how much like a string a Path ends up like. Other than join() (covered in the __div__ discussion), split() is an interesting case, since the default split-on-whitespace (str.split) doesn't make a whole lot of sense with a Path, but split-on-pathsep (os.path.split) does. Does it make sense to be able to split a path on something else (like str.split), or should people just convert to/ from a string? Should there be a maxsplit argument? =Tony.Meyer From barry at python.org Thu Jan 26 05:17:43 2006 From: barry at python.org (Barry Warsaw) Date: Wed, 25 Jan 2006 23:17:43 -0500 Subject: [Python-Dev] The path module PEP In-Reply-To: <43D81373.8030201@colorstudy.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> <43D80621.1080300@colorstudy.com> <43D81373.8030201@colorstudy.com> Message-ID: <1138249063.30246.24.camel@geddy.wooz.org> On Wed, 2006-01-25 at 18:10 -0600, Ian Bicking wrote: > Paths are strings, that's in the PEP. > > As an aside, I think it should be specified what (if any) string methods > won't be inherited by Path (or will be specifically disabled by making > them throw some exception). I think .join() and __iter__ at least > should be disabled. Whenever I see derived classes deliberately disabling base class methods, I see red flags that something in the design of the hierarchy isn't right. While I understand that you want to be able to use Path instances anywhere a string is currently used, I'm not sure that deriving from str is the right thing. Maybe deriving from basestring would be better, but even then I'm not sure. Is it possible that we don't need Path objects to interchangeable with strings, but just that we can get away with expanding a few critical functions (such as open())? > I think the use of underscores or squished words isn't as bit a deal as > the case of modules. It's often rather ambiguous what a "word" really > is. At least in English word combinations slowly and ambiguously float > towards being combined. So abspath and abs_path both feel sufficiently > inside the scope of PEP 8 that precedence is worth maintaining. > rfc822's getallmatchingheaders method was going too far, but a little > squishing doesn't bother me, if it is consistent (and it's actually > easier to be consistent about squishing than underscores). For something like "abspath" which is really an abbreviation + word, I suppose squishing them isn't so bad. The alternative is absolute_path() which is certainly more readable if a bit of a pain to write. It's a trade-off that should be made for practical purposes. I've definitely come to prefer spellings like is_absolute over isabsolute, and in general dislike squish words. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060125/00fb0c1a/attachment.pgp From stephen at xemacs.org Thu Jan 26 05:21:10 2006 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 26 Jan 2006 13:21:10 +0900 Subject: [Python-Dev] / as path join operator In-Reply-To: (Steven Bethard's message of "Wed, 25 Jan 2006 17:45:20 -0700") References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> Message-ID: <87k6cnaabt.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Steven" == Steven Bethard writes: Steven> My only fear with the / operator is that we'll end up with Steven> the same problems we have for using % in string formatting Steven> -- the order of operations might not be what users expect. Besides STeVe's example, (1) I think it's logical to expect that Path('home') / 'and/or' points to a file named "and/or" in directory "home", not to a file named "or" in directory "home/and". (2) Note that '/' is also the path separator used by URIs, which RFC 2396 gives different semantics from Unix. Most of my Python usage to date has been heavily web-oriented, and I'd have little use for / unless it follows RFC 2396. By that, I mean that I would want the / operator to treat its rhs as a relative reference, so the result would be computed by the algorithm in section 5.2 of that RFC. But this is not correct (realpath) semantics on Unix. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From barry at python.org Thu Jan 26 05:25:03 2006 From: barry at python.org (Barry Warsaw) Date: Wed, 25 Jan 2006 23:25:03 -0500 Subject: [Python-Dev] / as path join operator In-Reply-To: <43D8287F.70209@gmail.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> <43D8287F.70209@gmail.com> Message-ID: <1138249503.30245.26.camel@geddy.wooz.org> On Thu, 2006-01-26 at 11:40 +1000, Nick Coghlan wrote: > The main appeal to me of the division operation is that it allows multiple > path elements to be joined on a single line, but the joining method accepts an > arbitrary number of arguments, which helps with that just as much, and doesn't > raise precedence and readability questions. Or the need to employ ugliness like backslashes when you have to split the join across multiple lines. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060125/c251763e/attachment.pgp From ianb at colorstudy.com Thu Jan 26 05:30:40 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 25 Jan 2006 22:30:40 -0600 Subject: [Python-Dev] The path module PEP In-Reply-To: <1138249063.30246.24.camel@geddy.wooz.org> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> <43D80621.1080300@colorstudy.com> <43D81373.8030201@colorstudy.com> <1138249063.30246.24.camel@geddy.wooz.org> Message-ID: <43D85070.1010809@colorstudy.com> Barry Warsaw wrote: > On Wed, 2006-01-25 at 18:10 -0600, Ian Bicking wrote: > > >>Paths are strings, that's in the PEP. >> >>As an aside, I think it should be specified what (if any) string methods >>won't be inherited by Path (or will be specifically disabled by making >>them throw some exception). I think .join() and __iter__ at least >>should be disabled. > > > Whenever I see derived classes deliberately disabling base class > methods, I see red flags that something in the design of the hierarchy > isn't right. IMHO the hierarchy problem is a misdesign of strings; iterating over strings is usually a bug, not a deliberately used feature. And it's a particularly annoying bug, leading to weird results. In this case a Path is not a container for characters. Strings aren't containers for characters either -- apparently they are containers for smaller strings, which in turn contain themselves. Paths might be seen as a container for other subpaths, but I think everyone agrees this is too ambigous and implicit. So there's nothing sensible that __iter__ can do, and having it do something not sensible (just to fill it in with something) does not seem very Pythonic. join is also a funny method that most people wouldn't expect on strings anyway. But putting that aside, the real issue I see is that it is a miscognate for os.path.join, to which it has no relation. And I can't possibly imagine what you'd use it for in the context of a path. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From ianb at colorstudy.com Thu Jan 26 05:35:23 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 25 Jan 2006 22:35:23 -0600 Subject: [Python-Dev] The path module PEP In-Reply-To: <1138235459.6668.14.camel@localhost.localdomain> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <1138235459.6668.14.camel@localhost.localdomain> Message-ID: <43D8518B.70900@colorstudy.com> Gustavo J. A. M. Carneiro wrote: > On a slightly different subject, regarding path / path, I think it > feels much more natural path + path. Path.join is really just a string > concatenation, except that it adds a path separator in the middle if > necessary, if I'm not mistaken. No, it isn't, which maybe is why / is bad. os.path.join(a, b) basically returns the path as though b is interpreted to be relative to a. I.e., os.path.join('/foo', '/bar') == '/bar'. Not much like concatenation at all. Plus string concatenation is quite useful with paths, e.g., to add an extension. If a URI class implemented the same methods, it would be something of a question whether uri.joinpath('/foo/bar', 'baz') would return '/foo/baz' (and urlparse.urljoin would) or '/foo/bar/baz' (as os.path.join does). I assume it would be be the latter, and urljoin would be a different method, maybe something novel like "urljoin". -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From barry at python.org Thu Jan 26 05:41:23 2006 From: barry at python.org (Barry Warsaw) Date: Wed, 25 Jan 2006 23:41:23 -0500 Subject: [Python-Dev] / as path join operator In-Reply-To: <43D83BAF.20401@colorstudy.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> <43D83BAF.20401@colorstudy.com> Message-ID: <1138250483.30252.38.camel@geddy.wooz.org> On Wed, 2006-01-25 at 21:02 -0600, Ian Bicking wrote: > ext = '.jpg' > name = fields['name'] > image = Path('/images') / name + ext Here's a good example of why I ultimately don't like __div__. The last line seems quite non-obvious to me. It's actually jarring enough that I have to stop and think about what it means because it /looks/ like there's math going on. OTOH, something like: image = Path('', 'images', name) + ext or even better image = Path.join('', 'images', name) + ext where .join is a staticmethod, seems much clearer. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060125/4244e51a/attachment.pgp From barry at python.org Thu Jan 26 05:48:52 2006 From: barry at python.org (Barry Warsaw) Date: Wed, 25 Jan 2006 23:48:52 -0500 Subject: [Python-Dev] The path module PEP In-Reply-To: <43D85070.1010809@colorstudy.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> <43D80621.1080300@colorstudy.com> <43D81373.8030201@colorstudy.com> <1138249063.30246.24.camel@geddy.wooz.org> <43D85070.1010809@colorstudy.com> Message-ID: <1138250932.30252.45.camel@geddy.wooz.org> On Wed, 2006-01-25 at 22:30 -0600, Ian Bicking wrote: > IMHO the hierarchy problem is a misdesign of strings; iterating over > strings is usually a bug, not a deliberately used feature. And it's a > particularly annoying bug, leading to weird results. Agreed. I've written iteration code that has to special case basestrings before and that's particularly ugly. > In this case a Path is not a container for characters. Strings aren't > containers for characters either -- apparently they are containers for > smaller strings, which in turn contain themselves. Paths might be seen > as a container for other subpaths, but I think everyone agrees this is > too ambigous and implicit. So there's nothing sensible that __iter__ > can do, and having it do something not sensible (just to fill it in with > something) does not seem very Pythonic. > > join is also a funny method that most people wouldn't expect on strings > anyway. But putting that aside, the real issue I see is that it is a > miscognate for os.path.join, to which it has no relation. And I can't > possibly imagine what you'd use it for in the context of a path. Good points, but to me that argues against having any inheritance relationship between strings and Paths rather than having such a relationship and disabling certain methods. Thomas's post seemed more on-target for me and I'd like to see that idea fleshed out in more detail. If it's proved to be an impossible (or merely an extremely infeasible) task, then I think we can discuss the shortcut of deriving from strings. It just seems gross so I'd like to be sure there's no better way. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060125/ce83f8f2/attachment-0001.pgp From martin at v.loewis.de Thu Jan 26 06:57:13 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 26 Jan 2006 06:57:13 +0100 Subject: [Python-Dev] [Python-checkins] r42185 - python/trunk/Lib/test/test_socket_ssl.py In-Reply-To: References: <20060125083940.E45321E4002@bag.python.org> <1f7befae0601251735k3b000b97u6ba0f9a03af80ad4@mail.gmail.com> Message-ID: <43D864B9.8000302@v.loewis.de> Neal Norwitz wrote: > That's what I was thinking of. I thought you had to be accept()ing > prior to connect() working. I thought listen() only sets the # of > outstanding connections allowed (basically internal buffer). But if > the listen() is sufficient, I agree there is no race. Actually, that's the whole point of listen(). Even if you initially do accept() quickly, it might be that another connection request (SYN) arrives between return of accept() and the next call to accept(); this is essentially the same as connection requests arriving before the first accept() call. The kernel will acknowledge the SYN packet immediately, and do that for at most "number-of-backlog" incoming connections. Then, subsequent accept() calls will immediately return. Regards, Martin From tony.meyer at gmail.com Thu Jan 26 05:31:26 2006 From: tony.meyer at gmail.com (Tony Meyer) Date: Thu, 26 Jan 2006 17:31:26 +1300 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 Message-ID: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> Here's the draft for the first January summary. If anyone has time to send me/Steve comments/suggestions/corrections/additions, those will be most welcome. The Py_ssize_t threads were particularly over my head. As always, many thanks :) ============= Announcements ============= ---------------------------- QOTF: Quote of the Fortnight ---------------------------- Guido, on the crashability of Python: I'm not saying it's uncrashable. I'm saying that if you crash it, it's a bug unless proven harebrained. Contributing thread: - `Include ctypes into core Python? `_ [SJB] ------------------------------------ Brett Cannon looking for Ph.D. ideas ------------------------------------ Brett Cannon is looking for Python-related ideas to form the subject of his PhD dissertation, under Eric Wohlstadter at the University of British Columbia. He has three areas in which he has possible funding (XML integration, game scripting support, and AOP); he is open to grants from anyone else interested in particular Python development. If anyone has suggestions for topics, Brett is listening! So far: * Phillip J. Eby mentioned that he has been doing some research on implementing a non-traditional form of AOP in Python. * Bill Janssen suggested a system to compile Python to AJAX. * Steve Holden suggested a Python learning system. * Ian Bicking suggested Boxer_ implemented for Python. * Armin Rigo suggested PyPy related work. * Jason Orendorff suggested collecting a downloadable corpus of Python source code, and performing various analyses on it. * Dennis Allison suggested interpreter structures for Python-like languages that do not use a global interpreter lock, as well as various other topics. .. _Boxer: http://dewey.soe.berkeley.edu/boxer.html Contributing thread: - `Ph.D. dissertation ideas? `__ [TAM] --------------------------------- 2.4 Documentation with Commentary --------------------------------- Ian Bicking put an instance_ of his commenting system, Commentary_, up against the full 2.4 documentation. If you'd like to see how the Python docs look with user-added comments, now's your chance! .. _instance: http://pythonpaste.org/comment/python24/ .. _Commentary: http://pythonpaste.org/commentary/ Contributing thread: - `[Doc-SIG] that library reference, again `__ [SJB] --------------------------------------- Builds of the Development Documentation --------------------------------------- Be sure to keep yourself up to date with the newest Python documentation - Neal Norwitz has arranged it so that the current Python development documentation is now automatically generated every 12 hours and placed at http://docs.python.org/dev/. Contributing thread: - `automated builds and tests `__ [SJB] -------------------------------- PEPs now automatically installed -------------------------------- Martin v. L?wis has arranged for an automatic installation of PEPs_: they will now get published in the subversion post-commit. Contributing thread: - `Installing PEPs `__ .. _PEPs: http://www.python.org/peps [TAM] ========= Summaries ========= --------------------------------------------------- Using buildbot to automate testing of Python builds --------------------------------------------------- A buildbot_ has been setup to facilitate `automated testing of the Python SVN`_. There are currently five slaves setup: * Linux x86 (gentoo 2.6, glibc 2.3) * Linux AMD64 (gentoo 2.6, glibc 2.3) * Solaris10 sparc * OS X 10.3 G5 * OS X 10.4 G4 Ideally, all supported platforms will be added (including OS, version, hardware, and various configurations); the slaves are testing the trunk and the 2.4 branch. Note that Window buildbot slaves have to have an appropriate compiler (MS VC 7.1), otherwise it can't compile the code; the Windows Python would also like to build several other packages (like bz2 and Tcl/Tk). A periodic "clean build" may also be added; there was discussion whether this should be done via a patch to the buildbot master, via an IRC command, or with a separate scheduler. Martin v. L?wis would rather buildbot produced static pages than act as a web server (for security reasons). Brian Warner explained that it once did, but was changed to make it easier to implement features like the "Force Build" button. An explanation of the security was given, and it seems satisfactory at the moment. The buildbot testing has already produced results: a compiler warning on OS X 10.3 was noticed. A discussion took place as to whether this should be fixed (it should) and who could do it (Ronald Oussoren volunteered, with help from Skip Montanaro). Fredrik Lundh had been playing with "automatic module discovery" in an attempt to figure out how complete the library reference really is, tracking down missing reference pages and generating stubs for pages that "should exist". He suggested that the buildbot could also run this script; at the time of this summary this was in progress. .. _automated testing of the Python SVN: http://www.python.org/dev/buildbot/ .. _buildbot: http://buildbot.sourceforge.net/ Contributing threads: - `buildbot `__ - `Buildbot questions `__ - `[Buildbot-devel] Re: buildbot `__ - `Automated Python testing `__ - `building a module catalogue with buildbot `__ - `Buildbot: doing occasional full builds `__ [TAM] ---------------------- A "Rejected Ideas" PEP ---------------------- Alexander Kozlovsky submitted a pre-PEP for the oft-repeated request that Python do away with its explicit "self" parameter. Guido responded with a definitive "This won't change", but didn't want to allow the PEP to be created just to be rejected. A number of people felt that the mailing lists weren't a sufficiently visible place for this response, especially considering the huge number of times the request has been presented. Thomas Wouters proposed a "Rejected Ideas" PEP that would list some of the more commonly raised proposals that weren't really even worthy of a PEP, with references to the appropriate python-list or python-dev discussions. Tim Peters nominated the alternate title "Don't Bother: PEPs Rejected Before Being Written" which perhaps better summarizes the intent. People seemed generally in favor of the idea, and Thomas Wouters offered to write up a first draft indicating three major rejected ideas: * removing the explicit "self" parameter * upgrading the Python parser beyond a simple LL(1) * allowing parentheses to be omitted in function calls At the time of this summary, no PEP was yet available. Contributing threads: - `Draft proposal: Implicit self in Python 3.0 `__ - `Rejected ideas PEP (was re: Draft proposal: Implicit self in Python 3.0) `__ [SJB] ------------------------------- Using ssize_t as the index type ------------------------------- Continuing from `last week`_, more process was made towards `PEP 353`_. Tim Peters, Neal Norwitz, and Martin v. L?wis discussed replacing '#' as a format character, and how this could be down in a backwards compatible way; they also discussed how Py_ssize_t could be output correctly in a printf-style statement. Martin suggested using %zd, but this isn't in VC 7.1; Tim suggested that a macro be added to pyport.h, which would use "z" where possible, and define something else where not (e.g. Windows). Tim also suggested that PyErr_Format() and PyString_Format() should be taught to recognize the "z" qualifier, with the macros hidden inside the implementation of those functions. M.-A. Lemburg pointed out other changes of output parameter types which will cause problems in extensions, and worried that authors wouldn't be able to get these converted in time for Python 2.5. Martin disagreed. While code will continue to work with 32-bit systems with little work, more needs to be done for 64-bit systems. Martin suggested that users could continue to use 32-bit Python (since nothing will be gained by using 64-bit anyway) until the required changes are made. For more details, see the "Open Issues" section of the PEP. .. _last week: http://www.python.org/dev/summary/2005-12-16_2005-12-31.html#using-ssize-t-as-the-index-type .. _PEP 353: http://www.python.org/peps/pep-0353.html Contributing threads: - `[Python-checkins] commit of r41906 - python/branches/ssize_t/Objects/obmalloc.c `__ - `New PEP: Using ssize_t as the index type `__ - `[Python-checkins] r41972 - python/branches/ssize_t/Objects/funcobject.c `__ - `Py_ssize_t output parameters (Was: [Python-checkins] r41971 ...) `__ [TAM] ------------------------------------- Adding ctypes to the standard library ------------------------------------- Thomas Heller suggested that ctypes be included in core Python (starting with 2.5). The common response was that while ctypes is a useful, popular, mature module, it does make it very easy to get a core dump, which violates the guideline that if you get a core dump it's a bug in Python or in a third party extension or you're doing something harebrained. On the other hand, it was pointed out that the dl module suffers from the same problem, and is included without any warnings (the documentation was later fixed to include warnings). Martin v. L?wis suggested making ctypes a dynamically loaded module (ctypes.pyd), so administrators could remove it, and I could also make it a separate option in the Windows installer, so administrators could opt out of installing it. Everyone seemed happy with prominent warnings in the documentation, and so this is how it was checked in. Contributing thread: - `Include ctypes into core Python? `__ [TAM] ----------------------------------------- Adding tests when no fix is yet available ----------------------------------------- After Georg Brandl checked in a test for the compiler module without a corresponding fix, Tim Peters reverted the change, indicating that no check-in should ever leave the repository in such a state that ``regrtest.py -uall`` fails. The ensuing thread discussed a number of ways of checking in a test for a bug when code to fix the bug was not yet available. Neal Norwitz proposed adding two files to Lib/test, outstanding_bugs.py and outstanding_crashes.py, which would include the appropriate tests, but which would not be run under ``regrtest.py -uall``. Some people were concerned about the maintainability of keeping failing tests for a module separate from the other tests of that module, so Neal Norwitz and Scott David Daniels presented a patch_ and a recipe_ for decorators that mark individual test functions as failing. No official pronouncement on how failing tests should be added to the Python test suite had been made at the time of this summary. .. _patch: http://python.org/sf/1399935 .. _recipe: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/466288 Contributing thread: - `Checking in a broken test was: Re: [Python-checkins] r41940 - python/trunk/Lib/test/test_compiler.py `__ - `r42003 - python/trunk/Lib/test/outstanding_bugs.pypython/trunk/Lib/test/outstanding_crashes.py `__ [SJB] ------------------------------------------- Running the test suites in multiple locales ------------------------------------------- Georg Brandl noted that unicode format modifiers were (in SVN) incorrectly using the locale (one should always use locale.format to get locale-aware formatting). Fredrik Lundh suggested that all test should be run in a different locale than C to detect these bugs, perhaps adding a "use setlocale" flag to the test runner, to simplify this. Neal Norwitz made a patch to do this, finding a failure in test_email and two failures in test_re. Contributing thread: - `locale and LC_NUMERIC `__ [TAM] ----------------------------------------------------------- Replacing the docutils directory with an svn:externals link ----------------------------------------------------------- David Goodger replaced the static copy of docutils in the "peps" repository with a svn:externals link to the docutils repository. There was some concern about svn querying servers other than svn.python.org (the docutils repository is hosted on svn.berlios.de) and about linking to a potentially broken repository. David assured python-dev that the svn.berlios.de server was hardly ever unreachable, and changed the svn:externals link to select the 0.4 revision of docutils instead of the trunk. That guaranteed a non-broken copy of docutils, and made updating to a new revision just a simple propedit. Contributing thread: - `r42015 - peps/trunk `__ [SJB] -------------- Tkinter status -------------- A brief message from Joseph Rodrigues asking about the status of Tkinter in Python started a short thread about how Tkinter is maintained in Python. Martin v. L?wis is the primary maintainer, and he mainly focuses on integrating user contributions and making sure that Python is compatible with the latest release of Tcl/Tk. New widgets don't get wrapped automatically - if you want one supported, you should submit a patch. Contributing thread: - `Tkinter `__ [SJB] ------------------------------------ Roadmap for ConfigParser development ------------------------------------ Once again, the output capability of ConfigParser was brought up; this time by Facundo Bastista, who `submitted a patch`_ that orders the output so that doctest output can be easily reliable. He pointed out that there is `another open patch`_ that allows the user to specify the order through an "ordered dictionary". Guido explained that he didn't feel that it mattered, as long as the patch also allows comments to be preserved; his preference was for everything (ordering, blank lines, and comments) to be preserved. Tony Meyer point out some examples of previous_ `python-dev`_ discussion_ about how ConfigParser should be modified, and that SpamBayes_ includes a ConfigParser subclass that allows 'surgical' editing of files. .. _submitted a patch: http://python.org/sf/1399309 .. _another open patch: http://python.org/sf/1371075 .. _previous: http://mail.python.org/pipermail/python-dev/2004-October/049454.html .. _python-dev: http://mail.python.org/pipermail/python-dev/2004-October/049527.html .. _discussion: http://aspn.activestate.com/ASPN/Mail/Message/python-dev/1483518 .. _SpamBayes: http://spambayes.org Contributing thread: - `ConfigParser to save with order `__ [TAM] ------------------------------- Patches for the logging package ------------------------------- Shane Hathaway asked about the process for submitting patches for the logging package. Vinay Sajip is the primary maintainer, and accepts patches (compatible with Python 1.5.2) through sourceforge as normal, though personal emails are appropriate if you are worried about spending time on a patch that is not accepted. There was a brief followup discussion about how to get the logging package to interact nicely with external log rotation tools, including the possible creation of a ``reopen()`` function for files, but the thread trailed off without any solid conclusion on this issue. Contributing thread: - `Logging enhancements `__ [SJB] ---------------------------------- os.path.getmtime() bugs on Windows ---------------------------------- Christian Tismer noted that if, in a single interactive session on Windows, create a file, get its os.path.getmtime(), change your time zone, and get its os.path.getmtime() again, the time stamps are different. He asked whether there was a way to circumvent the problem, or if a patch could be created. Tim Peters gave him a `reference to a CodeProject page` that discusses the problem. Martin v. L?wis has meant to work on a patch for several years; his plan is to drop usage of msvcrt's stat(3), and instead implement os.stat in terms of Windows API directly - this would also have the advantage that subsecond time-stamps can be exposed. This patch would need to be implemented twice; once for Win9X (ANSI file names) and once for NT+ (unicode file names). .. _reference to a CodeProject page: http://www.codeproject.com/datetime/dstbugs.asp Contributing thread: - `os.path.getmtime on Windows `__ [TAM] ---------------------------------------- Generating Python-ast.c and Python-ast.h ---------------------------------------- Python-ast.c and Python-ast.h are generated automatically from Parser/asdl_c.py and then checked into Subversion. However, a ``make distclean`` removes these files, and if you do not have a 2.2+ version of Python available, they cannot be regenerated. The best solution seemed to be to alter ``make distclean`` to not remove Python-ast.c and Python-ast.h as they only need to be regenerated when the AST definition changes, which is rather infrequent. Contributing thread: - `[Python-checkins] commit of r41880 - python/trunk/Python/Python-ast.c `__ [SJB] ----------------------------------------- Allowing dict subclasses for exec globals ----------------------------------------- Currently, the ``exec`` statement allows the locals expression (the second expression after ``in``) to be any mapping type. The globals expression (the first expression after ``in``) should be a dict object, but Python does not currently raise an exception if you use a dict subclass, though it will not call any overridden methods correctly. `Crutcher Dunnavant presented a patch`_ that allows the globals expression to be a dictionary subclass such that overridden methods are called properly. Because it gives Python new capabilities, if the patch is accepted, it will not become part of Python until 2.5. .. _Crutcher Dunnavant presented a patch: http://www.python.org/sf/1402289 Contributing thread: - `[PATCH] Fix dictionary subclass semantics when used as global dictionaries `__ [SJB] ================== Previous Summaries ================== - `When do sets shrink? `__ - `Including zlib... `__ - `a quit that actually quits `__ - `Real time behaviour of Pythons memory management; WAS: RE: Documentation about Python's GC,python-dev list messages referenced in Modules/gcmodule.c notreachable anymore `__ - `slight inconsistency in svn checkin email subject lines `__ - `buildno (Was: [Python-checkins] commit of r41907- python/trunk/Makefile.pre.in) `__ =============== Skipped Threads =============== - `Weekly Python Patch/Bug Summary `__ - `current test problems `__ - `mac memory leaks `__ - `API changes `__ - `file.next() vs. file.readline() `__ - `Sharing between multiple interpreters and restricted mode `__ - `TAR problems under Solaris `__ - `bsddb broken `__ - `Jython and CPython `__ - `Compiler warnings for 64-bit portability problems `__ - `some interesting readings `__ - `Birkenfeld's gone `__ - `Python-Dev Digest, Vol 29, Issue 111 `__ - `PyCon TX 2006: Early-bird registration ends Jan. 15! `__ - `test_curses `__ - `sudo security hole w/ potential Python connection `__ - `Python-Dev Digest, Vol 30, Issue 32 `__ - `PyThreadState_Delete vs PyThreadState_DeleteCurrent `__ - `Hello `__ - `Limiting the recursion limit `__ - `PEP and stdlib `__ - `DRAFT: python-dev Summary for 2005-12-01 through 2005-12-15 `__ - `Unicode print/stdoutput exceptions are not nice `__ - `pystate.c changes for Python 2.4.2 `__ - `DRAFT: python-dev Summary for 2005-12-16 through 2005-12-31 `__ - `DRAFT: python-dev Summary for 2005-12-16 through2005-12-31 `__ - `Python icon `__ From theller at python.net Thu Jan 26 09:54:51 2006 From: theller at python.net (Thomas Heller) Date: Thu, 26 Jan 2006 09:54:51 +0100 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> Message-ID: Tony Meyer writes: > ------------------------------------- > Adding ctypes to the standard library > ------------------------------------- > > Thomas Heller suggested that ctypes be included in core Python > (starting with 2.5). The common response was that while ctypes is a > useful, popular, mature module, it does make it very easy to get a > core dump, which violates the guideline that if you get a core dump > it's a bug in Python or in a third party extension or you're doing > something harebrained. On the other hand, it was pointed out that the > dl module suffers from the same problem, and is included without any > warnings (the documentation was later fixed to include warnings). > > Martin v. L?wis suggested making ctypes a dynamically loaded module > (ctypes.pyd), so administrators could remove it, and > I could also make it a separate option in the Windows installer, so > administrators could opt out of installing it. Everyone seemed happy > with prominent warnings in the documentation, and so this is how it > was checked in. Well, it is *not* yet checked in. The current state is that ctypes uses GPL'd tools to build libffi, and those can't be committed into Python SVN. http://mail.python.org/pipermail/python-dev/2006-January/059937.html Currently I tend to agree with Martin and drop the idea for now, but this probably doesn't belong into your summary ;-). Thomas From thomas at xs4all.net Thu Jan 26 10:08:22 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Thu, 26 Jan 2006 10:08:22 +0100 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 In-Reply-To: References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> Message-ID: <20060126090822.GF18916@xs4all.nl> On Thu, Jan 26, 2006 at 09:54:51AM +0100, Thomas Heller wrote: > The current state is that ctypes uses GPL'd tools to build libffi, and > those can't be committed into Python SVN. > http://mail.python.org/pipermail/python-dev/2006-January/059937.html But http://mail.python.org/pipermail/python-dev/2006-January/059938.html was never responded to. And licenses are fluid, it may be a piece of cake to get one of those 'tools' un-GPL'ed, even if they are. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From hoffman at ebi.ac.uk Thu Jan 26 10:26:27 2006 From: hoffman at ebi.ac.uk (Michael Hoffman) Date: Thu, 26 Jan 2006 09:26:27 +0000 Subject: [Python-Dev] The path module PEP In-Reply-To: <20060126004113.GD18916@xs4all.nl> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <20060126004113.GD18916@xs4all.nl> Message-ID: [Thomas Wouters] >> This issue has been brought up before when people were discussing the >> path module. I think the consensus is that, while the inheritance >> isn't pure, practicality beats purity. It would require to big changes >> to Python and would break to much existing code to not extend string. > > This is my only problem with the PEP. It's all very nice that subclassing > from string makes it easier not to break things, but subclassing implies a > certain relationship. This is the soul of arguing for purity's sake when practicality would dictate something else. If you remove the basestring superclass, then you remove the ability to use path objects as a drop-in replacement for any path string right now. You will either have to use str(pathobj) or carefully check that the function/framework you are passing the path to does not use isinstance() or any of the string methods that are now gone. http://groups.google.com/group/comp.lang.python/browse_thread/thread/1f5bcb67c4c73f15 -- Michael Hoffman European Bioinformatics Institute From gjc at inescporto.pt Thu Jan 26 13:03:28 2006 From: gjc at inescporto.pt (Gustavo J. A. M. Carneiro) Date: Thu, 26 Jan 2006 12:03:28 +0000 Subject: [Python-Dev] The path module PEP In-Reply-To: <43D8518B.70900@colorstudy.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <1138235459.6668.14.camel@localhost.localdomain> <43D8518B.70900@colorstudy.com> Message-ID: <1138277008.8256.26.camel@localhost> On Wed, 2006-01-25 at 22:35 -0600, Ian Bicking wrote: > Gustavo J. A. M. Carneiro wrote: > > On a slightly different subject, regarding path / path, I think it > > feels much more natural path + path. Path.join is really just a string > > concatenation, except that it adds a path separator in the middle if > > necessary, if I'm not mistaken. > > No, it isn't, which maybe is why / is bad. os.path.join(a, b) basically > returns the path as though b is interpreted to be relative to a. I.e., > os.path.join('/foo', '/bar') == '/bar'. Not much like concatenation at > all. Really? This is not like the unix command line. At least in Linux, /foo/bar is the same as /foo//bar and /foo///bar, etc. But I admit it can be useful in some cases. > Plus string concatenation is quite useful with paths, e.g., to add > an extension. I see your point. Although perhaps adding an extension to a file should be the exception and not the rule, since adding extensions is rarely used compared to joining paths? Maybe Path.add_extension() ? BTW, regarding Path subclassing basestr, there exists "prior art" for this Path thing in SCons. In SCons, we (users, I'm not a scons dev) have to constantly deal with Node instances. Most scons functions that accept Nodes also accept strings, but a Node is not a string. When calling an os function with Nodes, one has to convert it to string first, using str(). IMHO, having to decorate Node instances with str() sometimes is no big deal, really. And, given enough time, perhaps most of the python standard library could be enhanced to accept Path instances in addition to C strings. > If a URI class implemented the same methods, it would be something of a > question whether uri.joinpath('/foo/bar', 'baz') would return '/foo/baz' > (and urlparse.urljoin would) or '/foo/bar/baz' (as os.path.join does). > I assume it would be be the latter, and urljoin would be a different > method, maybe something novel like "urljoin". I honestly don't understand the usefulness of join('/foo/bar', 'baz') ever returning '/foo/baz' instead of '/foo/bar/baz'. How would the former be of any use? If it has no use, then please don't complicate things even more :) Regards. -- Gustavo J. A. M. Carneiro The universe is always one step beyond logic. From p.f.moore at gmail.com Thu Jan 26 13:51:00 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 26 Jan 2006 12:51:00 +0000 Subject: [Python-Dev] The path module PEP In-Reply-To: <20060126004113.GD18916@xs4all.nl> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <20060126004113.GD18916@xs4all.nl> Message-ID: <79990c6b0601260451q3015bbf9j5d3d539234c49f07@mail.gmail.com> On 1/26/06, Thomas Wouters wrote: > On Wed, Jan 25, 2006 at 09:37:04PM +0100, BJ?rn Lindqvist wrote: > > > Inheritance from string (Jason) > > > This issue has been brought up before when people were discussing the > > path module. I think the consensus is that, while the inheritance > > isn't pure, practicality beats purity. It would require to big changes > > to Python and would break to much existing code to not extend string. I don't think there is consensus at all. I've seen plenty of arguments, either directly against inheritance from string, or against features which exist *because* of the inheritance (e.g., we can't use join() because it's a string method). > This is my only problem with the PEP. It's all very nice that subclassing > from string makes it easier not to break things, but subclassing implies a > certain relationship. That relationship doesn't exist, in this case. Having > the string subclass behave differently than strings (consider the __iter__ > and join methods) is a bad idea. I can dish up a half dozen contrived > problem cases, but the main reason I don't like it is that it feels wrong. Agreed. Path objects don't feel like strings to me, either. It's certainly *arguable* that "paths are strings" in some ideal sense, but in a very practical sense they are not. Operations like split, join, justification, trimming, all of which are part of the Python string type (and hence constitute part of what it means to "be a string" in Python) do not have any sensible meaning on paths. The only justification for making Path a string subtype seems to me to avoid a few conversions - open(path) rather than open(str(path)), for example. I'd rather have to explicitly convert, to be honest. (But I'd happily accept changes to open etc to take path objects directly). > If the reason to subclass string is that it's too hard to make an object > 'string-like' at a low enough level for the C API, I suggest fixing that, > instead. If that means Path has to wait until Python 2.6, then that's too > bad. The inability to feed C functions/types open() non-string objects has > troubled me before, and though I haven't invested a lot of time in it, I > don't quite understand why it isn't possible. Fixing it in a > backward-compatible manner may require a new __hook__, although I think > using __str__ or __unicode__ shouldn't be too problematic. > > Even if fixing the "%es"/"%et" etc formats to the arg-parsing methods is > infeasible, I would prefer a new format code for 'string-alike as C string' > over the unpythonic inappropriate subclassing. Although, even if the > subclassing was completely appropriate, I would much rather improve builtin > support for ducktyping than showcase subclassing ;) Adaptation (PEP 246) would let paths be *adaptable* to strings, without *being* strings... :-) Paul. From theller at python.net Thu Jan 26 13:29:55 2006 From: theller at python.net (Thomas Heller) Date: Thu, 26 Jan 2006 13:29:55 +0100 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <20060126090822.GF18916@xs4all.nl> Message-ID: Thomas Wouters writes: > On Thu, Jan 26, 2006 at 09:54:51AM +0100, Thomas Heller wrote: > >> The current state is that ctypes uses GPL'd tools to build libffi, and >> those can't be committed into Python SVN. > >> http://mail.python.org/pipermail/python-dev/2006-January/059937.html > > But http://mail.python.org/pipermail/python-dev/2006-January/059938.html > was never responded to. Lack of time - sorry. > And licenses are fluid, it may be a piece of cake to > get one of those 'tools' un-GPL'ed, even if they are. I wouldn't even know whom to ask. Thomas From p.f.moore at gmail.com Thu Jan 26 14:15:08 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 26 Jan 2006 13:15:08 +0000 Subject: [Python-Dev] The path module PEP In-Reply-To: <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> Message-ID: <79990c6b0601260515w22ae9e2dy22d265acc4bce7c3@mail.gmail.com> On 1/25/06, BJ?rn Lindqvist wrote: > My comments on the issues. It was easier this way than trying to reply > on every message individually. > > Inheritance from string (Jason) > > This issue has been brought up before when people were discussing the > path module. I think the consensus is that, while the inheritance > isn't pure, practicality beats purity. It would require to big changes > to Python and would break to much existing code to not extend string. > > I'll add this to Resolved Issues if nobody minds. I mind (see my previous post)... > Remove __div__ (Ian, Jason, Michael, Oleg) > > This is one of those where everyone (me too) says "I don't care either > way." If that is so, then I see no reason to change it unless someone > can show a scenario in which it hurts readability. Plus, a few people > have said that they like the shortcut. Hardly. I've seen some pretty strong arguments (both for and against) - not what I'd describe as everyone saying they don't care. FWIW, I find the / operator ugly. Also, multiple concatenation (path / "a" / "b" / "c") results in building lots of intermediates, where path.join("a", "b", "c") need not. Arguing that you can't reuse string methods is bogus, IMHO, as the requirement to subclass from string is far from clear. Actually, reading that, I'd suggest: - an append() method to add extra components to a path - a multi-arg Path() constructor So, we have - path.append("a", "b", "c") - Path("C:", "Windows", "System32") Quick question - how do Path objects constructed from relative paths behave? Are there such things as relative path objects? Consider p1 = Path("a") p2 = Path("b") Is p1.append(p2) (or p1/p2) legal? What does it mean? I'd have to assume it's the same as Path("a", "b"), but I'm not sure I like that... What about Path("/a").append(Path("/b")) ??? Also note that my example Path("C:", "Windows", "System32") above is an *absolute* path on Windows. But a relative (albeit stupidly-named :-)) path on Unix. How would that be handled? Not that os.path gets it perfect: Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import os >>> os.path.join("C:", "Windows", "System32") 'C:Windows\\System32' >>> os.path.join(".", os.path.join("C:", "Windows", "System32")) '.\\C:Windows\\System32' >>> But os.path manipulates strings representing pathnames (and I can forgive oddities like this by noting that some rules about pathnames are pretty subtle...). I'd have higher standards for a dedicated Path object. Arguably, Path objects should always maintain an absolute path - there should be no such thing as a relative Path. So you would have str(Path("whatever")) === os.path.abspath("whatever") It also allows Path("C:", "Windows") to do different things on Windows and Unix (absolute on Windows, relative to os.curdir on Unix). This would imply that Path("a", a_path) or a_path.append(another_path) is an error. And of course, for this to work, Path objects *can't* be a subclass of string... :-) Paul. From thomas at xs4all.net Thu Jan 26 14:26:41 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Thu, 26 Jan 2006 14:26:41 +0100 Subject: [Python-Dev] The path module PEP In-Reply-To: References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <20060126004113.GD18916@xs4all.nl> Message-ID: <20060126132641.GG18916@xs4all.nl> On Thu, Jan 26, 2006 at 09:26:27AM +0000, Michael Hoffman wrote: > [Thomas Wouters] > > [Subclassing string] is my only problem with the PEP. It's all very nice > > that subclassing from string makes it easier not to break things, but > > subclassing implies a certain relationship. > This is the soul of arguing for purity's sake when practicality would > dictate something else. If we're going to argue that point, I don't believe this is the practicality that the 'zen of python' talks about. Practicality is the argument for 'print', and for requiring the ':' before suites, and for some of the special cases in the Python syntax and module behaviour. It isn't about the implementation. The argument to subclass string is, as far as I can tell, only the ease of implementation and the ease of transition. Nothing in the old thread convinced me otherwise, either. I've never seen Guido go for an implementation-practical solution just because he couldn't be arsed to do the work to get a conceptually-practical solution. And subclassing string isn't conceptually-practical at all. > If you remove the basestring superclass, then you remove the ability > to use path objects as a drop-in replacement for any path string right > now. You will either have to use str(pathobj) or carefully check that > the function/framework you are passing the path to does not use > isinstance() or any of the string methods that are now gone. More to the point, you will have to carefully check whether the function/framework will use the Path object in a way the Path object can handle. There's already discussion about whether certain methods should be 'disabled', in Path objects, or whether they should be doing something conceptually different. And subclassing string is not going to solve all these issues anyway. Even in the standard library there's a scary amount of 'type(x) == type("")' checks, most noteably in exactly the type of function that takes both a filename and a file-like object. I don't believe going out of your way to cater to these kind of typechecks is the right way to solve the problem. I believe the problem is more that there isn't a unified, obvious, 'one-way' to do the typechecks -- or rather, to avoid the typechecks. Parts of Python do duck-typing and nothing else; this usually works fine, and is quite flexible. Other parts, especially (but not exclusively) the parts written in C and Python code that directly deals with those C parts, need to care more about actual types. Python offers no standard or accepted or even vaguely preferred way of doing that. The standard library doesn't even do this uniformly, so how can we expect anyone to ever get this 'right'? Especially since the 'right' way depends on what you want to do with the result. This problem pops up most visibly in treating-as-int (indexing) and in treating-as-string (open() and friends), but I'm sure there are more. You could see this as an argument for formal interfaces. Perhaps it is. It could also be an argument for just a few more __hooks__. Guido already suggested __index__, which would mean 'treat me as this int for indexing and other such operations' -- which is not the same thing as __int__. Likewise, treating-as-string would require a different hook than __str__ or __repr__, and should explicitly not be defined for most objects, only those that actually *want* to be treated as a string. And there should probably be more. Or, maybe, we could do it all in one __hook__. Say, __equivalent__. obj.__equivalent__(cls) would return the instance of 'cls' that represents 'obj', or raise an appropriate error. I say 'instance of 'cls'', which means it could be an instance of a subclass, too. It isn't duck-typing and it doesn't replace interfaces, not by far, since it actually returns a new type (and thus can't be a proxy-type; I don't think changing the equivalent should change the original, anyway.) Rather, it'd be the standard way to glue duck-typing (and formal interfaces, if you use those) with strict typing mechanisms (like C's.) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From hoffman at ebi.ac.uk Thu Jan 26 15:20:04 2006 From: hoffman at ebi.ac.uk (Michael Hoffman) Date: Thu, 26 Jan 2006 14:20:04 +0000 Subject: [Python-Dev] The path module PEP In-Reply-To: <20060126132641.GG18916@xs4all.nl> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <20060126004113.GD18916@xs4all.nl> <20060126132641.GG18916@xs4all.nl> Message-ID: [Thomas Wouters] >>> [Subclassing string] is my only problem with the PEP. It's all very nice >>> that subclassing from string makes it easier not to break things, but >>> subclassing implies a certain relationship. [Michael Hoffman] >> This is the soul of arguing for purity's sake when practicality would >> dictate something else. [Thomas Wouters] > If we're going to argue that point, I don't believe this is the practicality > that the 'zen of python' talks about. Practicality is the argument for > 'print', and for requiring the ':' before suites, and for some of the > special cases in the Python syntax and module behaviour. It isn't about the > implementation. The argument to subclass string is, as far as I can tell, > only the ease of implementation and the ease of transition. Nothing in the > old thread convinced me otherwise, either. I've never seen Guido go for an > implementation-practical solution just because he couldn't be arsed to do > the work to get a conceptually-practical solution. And subclassing string > isn't conceptually-practical at all. I don't understand what "conceptually-practical" is or how it differs from "conceptually pure" which is what it seems that you're looking for. It's not hard to give Path a has-a relationship to basestring instead of an is-a relationship, so it really doesn't save much in terms of implementation. >> If you remove the basestring superclass, then you remove the ability >> to use path objects as a drop-in replacement for any path string right >> now. You will either have to use str(pathobj) or carefully check that >> the function/framework you are passing the path to does not use >> isinstance() or any of the string methods that are now gone. > > More to the point, you will have to carefully check whether the > function/framework will use the Path object in a way the Path object can > handle. There's already discussion about whether certain methods should be > 'disabled', in Path objects, or whether they should be doing something > conceptually different. Yes, and I think all of this discussion is focused on conceptual purity and misses one of the big wins of the Path module for current users--it can be trivially used anywhere where a str is expected today. If you're going to start deciding that certain str methods should be disabled for some reason, then it shouldn't be a str subclass, because it will no longer behave like string-plus. In previous discussions, string methods were identified that one programmer thought would not be useful on a path, but others disagreed. Disabling methods serves no useful purpose, except to shorten dir(). I've been using path.py for some time, and I can tell you that it would be a lot less useful if it no longer behaved like string-plus. -- Michael Hoffman European Bioinformatics Institute From ronaldoussoren at mac.com Thu Jan 26 15:22:17 2006 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Thu, 26 Jan 2006 15:22:17 +0100 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 In-Reply-To: References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <20060126090822.GF18916@xs4all.nl> Message-ID: <864123BC-0DCD-4AD3-8BDA-EC9625090FF4@mac.com> On 26-jan-2006, at 13:29, Thomas Heller wrote: > Thomas Wouters writes: > >> On Thu, Jan 26, 2006 at 09:54:51AM +0100, Thomas Heller wrote: >> >>> The current state is that ctypes uses GPL'd tools to build >>> libffi, and >>> those can't be committed into Python SVN. >> >>> http://mail.python.org/pipermail/python-dev/2006-January/059937.html >> >> But http://mail.python.org/pipermail/python-dev/2006-January/ >> 059938.html >> was never responded to. > > Lack of time - sorry. > >> And licenses are fluid, it may be a piece of cake to >> get one of those 'tools' un-GPL'ed, even if they are. > > I wouldn't even know whom to ask. It shouldn't be too hard to use Python's main configure script to calculate the information necessary to build libffi. A lot of it is already calculated anyway (sizeof various type, endianness), some can be hardcoded (FFI_NO_RAW_API). In PyObjC I just compile the files I need from my setup.py. But I have an easy task, I just need to support two CPU architectures on one OS. Ronald > > Thomas > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ > ronaldoussoren%40mac.com -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2157 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20060126/8f8b38de/attachment-0001.bin From stefan.rank at ofai.at Thu Jan 26 15:47:05 2006 From: stefan.rank at ofai.at (Stefan Rank) Date: Thu, 26 Jan 2006 15:47:05 +0100 Subject: [Python-Dev] The path module PEP In-Reply-To: <79990c6b0601260515w22ae9e2dy22d265acc4bce7c3@mail.gmail.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <79990c6b0601260515w22ae9e2dy22d265acc4bce7c3@mail.gmail.com> Message-ID: <43D8E0E9.9080202@ofai.at> on 26.01.2006 14:15 Paul Moore said the following: [snip] > > Also note that my example Path("C:", "Windows", "System32") above is > an *absolute* path on Windows. But a relative (albeit stupidly-named > :-)) path on Unix. How would that be handled? wrong, Path("C:", "Windows", "System32") is a relative path on windows. see below. > Not that os.path gets it perfect: > > Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. >>>> import os >>>> os.path.join("C:", "Windows", "System32") > 'C:Windows\\System32' >>>> os.path.join(".", os.path.join("C:", "Windows", "System32")) > '.\\C:Windows\\System32' > this is misleading. observe:: In [1]: import os In [2]: os.path.join(".", os.path.join("C:", "Windows", "System32")) Out[2]: '.\\C:Windows\\System32' but:: In [3]: os.path.join(".", os.path.join("C:\\", "Windows", "System32")) Out[3]: 'C:\\Windows\\System32' The second example uses an absolute path as second argument, and as os.path.join should do, the first argument is discarded. The first case is arguably a bug, since, on windows, C:Windows\System32 is a path relative to the *current directory on disk C:* If the cwd on C: would be C:\temp then C:Windows\System32 would point to C:\temp\Windows\System32 The problem is that Windows has a cwd per partition... (I cannot even guess why ;-) For the sake of illustration, the following is a WinXP cmd session:: Microsoft Windows XP [Version 5.1.2600] (C) Copyright 1985-2001 Microsoft Corp. C:\temp>d: D:\>cd HOME D:\HOME>c: C:\temp>d: D:\HOME>c: C:\temp>cd d:bin C:\temp>d: D:\HOME\bin> [snip] > > Arguably, Path objects should always maintain an absolute path - there > should be no such thing as a relative Path. So you would have you realise that one might need and/or want to represent a relative path? From fredrik at pythonware.com Thu Jan 26 16:17:32 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 26 Jan 2006 16:17:32 +0100 Subject: [Python-Dev] The path module PEP References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com><43D6BA85.8070007@colorstudy.com><740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com><1138235459.6668.14.camel@localhost.localdomain><43D8518B.70900@colorstudy.com> <1138277008.8256.26.camel@localhost> Message-ID: Gustavo J. A. M. Carneiro wrote: > > If a URI class implemented the same methods, it would be something of a > > question whether uri.joinpath('/foo/bar', 'baz') would return '/foo/baz' > > (and urlparse.urljoin would) or '/foo/bar/baz' (as os.path.join does). > > I assume it would be be the latter, and urljoin would be a different > > method, maybe something novel like "urljoin". > > I honestly don't understand the usefulness of join('/foo/bar', 'baz') > ever returning '/foo/baz' instead of '/foo/bar/baz'. How would the > former be of any use? it's how URL:s are joined, as noted in the paragraph you replied to (a "baz" link on the page "/foo/bar" refers to "/foo/baz", not "/foo/bar/baz") From theller at python.net Thu Jan 26 16:33:08 2006 From: theller at python.net (Thomas Heller) Date: Thu, 26 Jan 2006 16:33:08 +0100 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <20060126090822.GF18916@xs4all.nl> <864123BC-0DCD-4AD3-8BDA-EC9625090FF4@mac.com> Message-ID: Ronald Oussoren writes: > On 26-jan-2006, at 13:29, Thomas Heller wrote: > >> Thomas Wouters writes: >> >>> On Thu, Jan 26, 2006 at 09:54:51AM +0100, Thomas Heller wrote: >>> >>>> The current state is that ctypes uses GPL'd tools to build libffi, >>>> and those can't be committed into Python SVN. >>> >>>> http://mail.python.org/pipermail/python-dev/2006-January/059937.html > > It shouldn't be too hard to use Python's main configure script to > calculate the information necessary to build libffi. A lot of it is > already calculated anyway (sizeof various type, endianness), some can > be hardcoded (FFI_NO_RAW_API). > > In PyObjC I just compile the files I need from my setup.py. But I have > an easy task, I just need to support two CPU architectures on one OS. Thanks for the encouragement - Martin suggested a similar approach. >From my understanding (which goes not very far) the configuration does two things: determine the set of source files that needs to go in depending on the cpu architecture, and to determine some information and make them available in #defines. I have to check if this is possible without patching the libffi sources themselves. I guess I could look into the PyObjC setuop script. Personally I only have access to machines running windows, linux (x86 and x86_64), and OS X (plus possibly a Mac running ubuntu), so I could only do it for those. Maybe support for other architectures can be added by volunteers? Besides: James Y Knight seems to be correct that all the scripts needed to build libffi seems to have this special exception from the GPL. Thomas From theller at python.net Thu Jan 26 16:43:58 2006 From: theller at python.net (Thomas Heller) Date: Thu, 26 Jan 2006 16:43:58 +0100 Subject: [Python-Dev] Include ctypes into core Python? References: <43C58200.20803@v.loewis.de> <43CC17FB.5020904@v.loewis.de> <03C7403A-6B16-4727-B2C6-5FE133E69734@fuhm.net> Message-ID: <7j8nata9.fsf@python.net> James Y Knight writes: > On Jan 19, 2006, at 4:24 PM, Thomas Heller wrote: > >> >> Several of these files are licensed under GPL: >> >> aclocal.m4 config-ml.in config.guess config.sub depcomp ltcf-c.sh >> ltconfig missing >> > > Are you sure? The copies of aclocal.m4 and config-ml.in both disagree > with you. aclocal seems to have a completely liberal license, and > config-ml has a "whatever the license of the program it's building" > license. > It seems you are right: config-ml.in: GPL with special exception. config.guess: GPL with special exception. config.sub: GPL with special exception. configure: no limitation depcomp: GPL with special exception. install-sh: X-11 license ltcf-c.sh: GPL with special exception. ltconfig: GPL with special exception. ltmain.sh: GPL with special exception. aclocal.m4: see below Is aclocal.m4 an exception? It has several copyright notices. The first one gives unlimited permissions to copy and/or distribute, but sections after that have no exception clause. I'm unsure what this means. The files that ctypes uses are in CVS here: http://cvs.sourceforge.net/viewcvs.py/ctypes/ctypes/source/gcc/libffi/?only_with_tag=branch_1_0 Thomas From bingham at cenix-bioscience.com Thu Jan 26 16:34:32 2006 From: bingham at cenix-bioscience.com (Aaron Bingham) Date: Thu, 26 Jan 2006 16:34:32 +0100 Subject: [Python-Dev] The path module PEP In-Reply-To: <43D8E0E9.9080202@ofai.at> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <79990c6b0601260515w22ae9e2dy22d265acc4bce7c3@mail.gmail.com> <43D8E0E9.9080202@ofai.at> Message-ID: <43D8EC08.5040702@cenix-bioscience.com> Stefan Rank wrote: >on 26.01.2006 14:15 Paul Moore said the following: >[snip] > > >>Arguably, Path objects should always maintain an absolute path - there >>should be no such thing as a relative Path. So you would have >> >> > >you realise that one might need and/or want to represent a relative path? > > Of course, but it seems to me a relative path is a different type from an absolute path, in the same way that a timedelta is different from a datetime. For example: * You can't open a relative path without reference to some absolute path (possibly the cwd). * You can't join two absolute paths, but you can join a relative path to another relative path, or to an absolute path. Cheers, Aaron -------------------------------------------------------------------- Aaron Bingham Senior Software Engineer Cenix BioScience GmbH -------------------------------------------------------------------- From bjourne at gmail.com Thu Jan 26 17:16:43 2006 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Thu, 26 Jan 2006 17:16:43 +0100 Subject: [Python-Dev] Path inherits from string Message-ID: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com> This seems to be the only really major issue with the PEP. Everything else is negotiable, IMHO. But the string inheritance seem to be such a critical issue it deserves its own thread. I have tried to address all criticism of it here: Really, it is the same arguments that have been rehashed over and over again. I also think that they have been suitably countered already - Practicality beats Purity. The fact that you can plug Path into already existing code is a major benefit. It makes it possible to use and integrate the Path class *now* and not wait until the mythical Python 3000 when every single flaw in Python have been fixed. Remember that the Path module have existed for three years in the wild and is extra-ordinarily well-liked. If the fact that Path inherits from string is such a big deal, then *why* is the path module so hugely popular?? :) The inheritance is a design trade-off. An implementation detail, an insignificant wart in an otherwise very practical design. Over the years I have used the path module this wart has never ever hurt me. Not even once. But if someone can construct a Path class that does *not* inherit from string and that works equally well as Path then ofcourse, that is preferable. However, because of this (and similar issues) I don't think it is possible: class Foo: def __str__(self): return "foo" open(Foo()) TypeError: coercing to Unicode: need string or buffer, instance found However, I might be wrong because according to [1] it should work. And having to wrap the Path object in str() (open(str(somepath))) each and every time the called function expects a string is not a practical solution. Also note that because compatibility with existing code is important, Path can not override or change any methods at all from str - Liskov Substitution Principle. As Path is, it only violates LSP once: >>> type(Path()) == type('') False And there is absolutely nothing that can be done about that. As far as I can tell, the string inheritance is either livable with or is a showstopper. If it is the latter, then: 1. Someone has to make the required modifications to the Python core. 2. Create a Path class (or adapt the existing one so) that does not inherit from string. 3. Release it and wait a few years hoping for it to gain widespread acceptance in the Python community. 4. Make a PEP (or adapt this PEP) that gets accepted. This scenario makes me sad because it basically means that there will never be a Path module in Python, atleast not during my lifetime. :( Why? Because Jason Orendorff's path module already exists and works. But I may be overestimating the problems and doing what Jason suggests in [2] may be enough. Maybe someone who knows what he is talking about can expand on it further? 1 http://mail.python.org/pipermail/python-dev/2001-August/016674.html 2 http://mail.python.org/pipermail/python-dev/2006-January/060030.html -- mvh Bj?rn From fredrik at pythonware.com Thu Jan 26 17:35:05 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 26 Jan 2006 17:35:05 +0100 Subject: [Python-Dev] Path inherits from string References: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com> Message-ID: BJörn Lindqvist wrote: > However, I might be wrong because according to [1] it should work. And > having to wrap the Path object in str() (open(str(somepath))) each and > every time the called function expects a string is not a practical > solution. in Python, the usual way to access an attribute of an object is to access the attribute; e.g. f = open(p.name) From mal at egenix.com Thu Jan 26 17:51:48 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 26 Jan 2006 17:51:48 +0100 Subject: [Python-Dev] Path inherits from string In-Reply-To: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com> References: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com> Message-ID: <43D8FE24.1030500@egenix.com> BJ?rn Lindqvist wrote: > This seems to be the only really major issue with the PEP. Everything > else is negotiable, IMHO. But the string inheritance seem to be such a > critical issue it deserves its own thread. I have tried to address all > criticism of it here: I don't see why this is critical for the success of the Path object. I agree with Thomas that interfaces should be made compatible to Path object. Please note that inheritance from string will cause the C type checks of the form PyString_Check(obj) to return true. C code will then assume that it has an object which is compatible to string C API which instances aren't. If the C code then uses the C API string macros, you get segfaults - and lot's of old code does, since there was no way to inherit from a string type at the time. In fact, you're lucky that open() doesn't give you segfaults, since the code used to fetch the string argument does exactly that... >... > > And there is absolutely nothing that can be done about that. As far as > I can tell, the string inheritance is either livable with or is a > showstopper. If it is the latter, then: > > 1. Someone has to make the required modifications to the Python > core. Right. Plus convert a few PyString_Check()s to PyString_CheckExact()... > 2. Create a Path class (or adapt the existing one so) that does > not inherit from string. > 3. Release it and wait a few years hoping for it to gain > widespread acceptance in the Python community. > 4. Make a PEP (or adapt this PEP) that gets accepted. > > This scenario makes me sad because it basically means that there will > never be a Path module in Python, atleast not during my lifetime. :( Why not ? We've added Unicode support to at least some file I/O APIs - adding support for instances which support the string interface shouldn't be all that difficult :-) BTW, if you're fine with this API: class File: def __unicode__(self): return u"test.txt" then the required change is minimal: we'd just need to use PyObject_Unicode() in getargs.c:837 and you should be set. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 26 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From stefan.rank at ofai.at Thu Jan 26 17:55:24 2006 From: stefan.rank at ofai.at (Stefan Rank) Date: Thu, 26 Jan 2006 17:55:24 +0100 Subject: [Python-Dev] The path module PEP In-Reply-To: <43D8EC08.5040702@cenix-bioscience.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <79990c6b0601260515w22ae9e2dy22d265acc4bce7c3@mail.gmail.com> <43D8E0E9.9080202@ofai.at> <43D8EC08.5040702@cenix-bioscience.com> Message-ID: <43D8FEFC.40708@ofai.at> on 26.01.2006 16:34 Aaron Bingham said the following: > Stefan Rank wrote: >> on 26.01.2006 14:15 Paul Moore said the following: >> [snip] >> >>> Arguably, Path objects should always maintain an absolute path - there >>> should be no such thing as a relative Path. >>> >> you realise that one might need and/or want to represent a relative path? >> > Of course, but it seems to me a relative path is a different type from > an absolute path, in the same way that a timedelta is different from a > datetime. > > For example: > > * You can't open a relative path without reference to some absolute > path (possibly the cwd). > * You can't join two absolute paths, but you can join a relative path > to another relative path, or to an absolute path. I think the datetime/timedelta analogy is not bad: A datetime is actually also a time delta - relative to some given start-time, internally this is normally the "epoch". For human-readable representations it is the birth of your chosen deity, or the creation of the universe, ... The start time for datetime is implicit. Python has chosen some absolute reference. For paths that absolute reference (the root) is very much context dependent (platform dependent). You *can* open a relative path - because processes always have an implicit cwd as part of their context. But you might also want to represent paths that are relative on another host than the one your program is running on. I don't think it makes sense to design a Path class that captures the abstract concept of a path - because of the semantic differences between unix paths, windows paths, URL paths, ... I see the Path object as a special kind of string, that has helpful methods for relating it to the workings of filesystems in general and the local filesystem in particular. But it is still just an ordinary string that happens to be used as a special kind of address. I try to separate the concept of the 'object in the filesystem' (which is the domain of Python's file objects) from the 'hierarchical address to an object' (which is what the Path objects make easier). (Java has these two conflated in one.) So, to get to the point, a `file` is a thing that should always have an absolute path. (and it does. it should probably grow a .path attribute in addition to .name ? This could return None for files without paths.) A Path should be able to contain absolute, relative, valid, as well as invalid (on a given OS) paths. In case that future systems manage to reduce the importance of the legacy crutch that is the hierarchical filesystem ;-) we might get query-like addresses: '/tmp/[author=me;type=text/html]' and Path might grow to deal with it. Sorry I digress. +1 on Path as an enhanced string that bundles file-system-address related methods. stefan From ianb at colorstudy.com Thu Jan 26 18:00:04 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 26 Jan 2006 11:00:04 -0600 Subject: [Python-Dev] Path inherits from string In-Reply-To: References: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com> Message-ID: <43D90014.10807@colorstudy.com> Fredrik Lundh wrote: >>However, I might be wrong because according to [1] it should work. And >>having to wrap the Path object in str() (open(str(somepath))) each and >>every time the called function expects a string is not a practical >>solution. > > > in Python, the usual way to access an attribute of an object is to > access the attribute; e.g. > > f = open(p.name) You mean "f = open(Path(p).name)", because it is likely that people will also have to accept strings for the nearterm (and probably longeterm) future. And the error message without will be inscrutable (and will still be inscrutable in many cases when you try to access other methods, sadly). And currently .name is taken for something else in the API. And the string path is not really an attribute because the string path *is* the object, it is not *part* of the object. OTOH, str(path) will break unicode filenames. And unicode() breaks anything that simply desires to pass data through without effecting its encoding. An open method on paths simplifies many of these issues, but doesn't do anything for passing a path to legacy code. Changing open() and all the functions that Path replaces (e.g., os.path.join) to accept Path objects may resolve issues with a substantial portion of code. But any code that does a typecheck on arguments will be broken -- which in the case of paths is quite common since many functions take both filename and file object arguments. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From foom at fuhm.net Thu Jan 26 18:04:06 2006 From: foom at fuhm.net (James Y Knight) Date: Thu, 26 Jan 2006 12:04:06 -0500 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 In-Reply-To: References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <20060126090822.GF18916@xs4all.nl> Message-ID: On Jan 26, 2006, at 7:29 AM, Thomas Heller wrote: >> And licenses are fluid, it may be a piece of cake to >> get one of those 'tools' un-GPL'ed, even if they are. > > I wouldn't even know whom to ask. On Jan 26, 2006, at 9:22 AM, Ronald Oussoren wrote: > It shouldn't be too hard to use Python's main configure script to > calculate > the information necessary to build libffi. A lot of it is already > calculated > anyway (sizeof various type, endianness), some can be hardcoded > (FFI_NO_RAW_API). > > In PyObjC I just compile the files I need from my setup.py. But I > have an easy task, > I just need to support two CPU architectures on one OS. Let's please be sure the license isn't fine _as is_ before thinking about asking people to change them or rewriting the build system! I did not look at *all* the files listed as being GPL, but the first two in the list were not. James From gjc at inescporto.pt Thu Jan 26 18:22:58 2006 From: gjc at inescporto.pt (Gustavo J. A. M. Carneiro) Date: Thu, 26 Jan 2006 17:22:58 +0000 Subject: [Python-Dev] The path module PEP In-Reply-To: References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <1138235459.6668.14.camel@localhost.localdomain> <43D8518B.70900@colorstudy.com> <1138277008.8256.26.camel@localhost> Message-ID: <1138296178.8256.37.camel@localhost> On Thu, 2006-01-26 at 16:17 +0100, Fredrik Lundh wrote: > Gustavo J. A. M. Carneiro wrote: > > > > If a URI class implemented the same methods, it would be something of a > > > question whether uri.joinpath('/foo/bar', 'baz') would return '/foo/baz' > > > (and urlparse.urljoin would) or '/foo/bar/baz' (as os.path.join does). > > > I assume it would be be the latter, and urljoin would be a different > > > method, maybe something novel like "urljoin". > > > > I honestly don't understand the usefulness of join('/foo/bar', 'baz') > > ever returning '/foo/baz' instead of '/foo/bar/baz'. How would the > > former be of any use? > > it's how URL:s are joined, as noted in the paragraph you replied to > > (a "baz" link on the page "/foo/bar" refers to "/foo/baz", not "/foo/bar/baz") That's not how I see it. A web browser, in order to resolve the link 'baz' in the page '/foo/bar', should do: join(basename('/foo/bar'), 'baz') == join('/foo', 'baz') == '/foo/baz'. Regards. -- Gustavo J. A. M. Carneiro The universe is always one step beyond logic. From martin at v.loewis.de Thu Jan 26 19:41:36 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 26 Jan 2006 19:41:36 +0100 Subject: [Python-Dev] Path inherits from string In-Reply-To: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com> References: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com> Message-ID: <43D917E0.4050606@v.loewis.de> BJ?rn Lindqvist wrote: > This seems to be the only really major issue with the PEP. I'd like to call for order here. What PEP? I can't find it on http://www.python.org/peps/ Also, if this is a major issue, then the PEP owner should not start a thread discussing it, but instead edit the PEP should summarize the discussion, pointing out the most common arguments, and either declare them as open, or explain why they aren't issues. People can then discuss these explanations. Regards, Martin From bjourne at gmail.com Thu Jan 26 19:45:09 2006 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Thu, 26 Jan 2006 19:45:09 +0100 Subject: [Python-Dev] / as path join operator (was: Re: The path module PEP) In-Reply-To: <1138247468.30252.11.camel@geddy.wooz.org> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> <034C8957-94EF-4F36-906D-D4EC57221FAF@ihug.co.nz> <1138247468.30252.11.camel@geddy.wooz.org> Message-ID: <740c3aec0601261045l211c1e0fp54584f72259ae76c@mail.gmail.com> I think that everything that can be said aboud __div__() has already been said. But this argument was really convincing: [Tony Meyer] > The vast majority of people (at least at the time) were either +0 or > -0, not +1. +0's are not justification for including something. There is no clear consensus either way. Ultimately, Guido will decide if he thinks it is clever or not. Meanwhile I'll remove it from the PEP and keep it as an "optional extension." Also, like Jason said, the removal of __div__() leads to the ultimate demise of joinpath(), woho! [Jason Orendorff] > in which case I propose using the Path constructor as the > replacement for os.path.join(). In that case, Path.joinpath can be > dropped. Path.cwd() / "foobar" ==> Path(Path.cwd(), "foobar") Path("foo") / "bar" / "baz" ==> Path("foo", "bar", "baz") Still, in the simpler cases, __div__() looks really handy: os.chdir(pkgdir / "include") ==> os.chdir(Path(pkgdir, "include")) Oh well. You can't have everything, can you? The updated PEP and an implementation is availible from http://wiki.python.org/moin/PathClass. -- mvh Bj?rn From martin at v.loewis.de Thu Jan 26 19:54:03 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 26 Jan 2006 19:54:03 +0100 Subject: [Python-Dev] Path inherits from string In-Reply-To: <43D8FE24.1030500@egenix.com> References: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com> <43D8FE24.1030500@egenix.com> Message-ID: <43D91ACB.3090401@v.loewis.de> M.-A. Lemburg wrote: > Please note that inheritance from string will cause the C type > checks of the form PyString_Check(obj) to return true. > C code will then assume that it has an object which is > compatible to string C API which instances aren't. Oh, sure they are. Types inheriting from str have the same layout as str, and C code assuming that layout will work fine with them. Inheritance works (saying "inheritance *just* works would deny the many fine details that have been engineered to actually make it work"). Regards, Martin From news-nospam at northportal.net Thu Jan 26 20:23:30 2006 From: news-nospam at northportal.net (VanL) Date: Thu, 26 Jan 2006 12:23:30 -0700 Subject: [Python-Dev] The path module PEP In-Reply-To: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> Message-ID: The path module has a great idea, but it does too much -- it conflates too many ideas into a single module. It has methods for dealing with files (open, bytes, read, etc) as well as methods for dealing with a filesystem tree as a whole (relpath, abspath, etc). Both of these ideas are tangentially related to paths, but really aren't in the same conceptual box. Not too long ago, I had to build something loosely based upon the path module. Instead of using it as-is, I broke it up into three modules: Tree (filesystem interfaces) Path (*just* path interfaces) File (a generic filelike object) Doing it this way had two benefits: First, it put different concepts into different modules. I note that some other virtual filesystem modules also maintedned a similar separation - probably for similar reasons. Second, I was able to define an interface which could be used across remote systems -- e.g. I was able to have an FTPTree (built on the standard library ftplib) and SSHTree (built upon paramiko) as well as FileTree (a standard filesystem). This isn't full-fledged interfaces - I just implemented common functionality in a class and then delegated to a ._ops class which passed through the necessary operations. However, I was able to use the various trees and file-like objects interchangeably. From steve at holdenweb.com Thu Jan 26 20:55:08 2006 From: steve at holdenweb.com (Steve Holden) Date: Thu, 26 Jan 2006 19:55:08 +0000 Subject: [Python-Dev] Path inherits from string In-Reply-To: <43D90014.10807@colorstudy.com> References: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com> <43D90014.10807@colorstudy.com> Message-ID: <43D9291C.9040009@holdenweb.com> Ian Bicking wrote: > Fredrik Lundh wrote: > >>>However, I might be wrong because according to [1] it should work. And >>>having to wrap the Path object in str() (open(str(somepath))) each and >>>every time the called function expects a string is not a practical >>>solution. >> >> >>in Python, the usual way to access an attribute of an object is to >>access the attribute; e.g. >> >> f = open(p.name) > > > You mean "f = open(Path(p).name)", because it is likely that people will > also have to accept strings for the nearterm (and probably longeterm) > future. And the error message without will be inscrutable (and will > still be inscrutable in many cases when you try to access other methods, > sadly). And currently .name is taken for something else in the API. > And the string path is not really an attribute because the string path > *is* the object, it is not *part* of the object. > > OTOH, str(path) will break unicode filenames. And unicode() breaks > anything that simply desires to pass data through without effecting its > encoding. > > An open method on paths simplifies many of these issues, but doesn't do > anything for passing a path to legacy code. Changing open() and all the > functions that Path replaces (e.g., os.path.join) to accept Path objects > may resolve issues with a substantial portion of code. But any code > that does a typecheck on arguments will be broken -- which in the case > of paths is quite common since many functions take both filename and > file object arguments. > Would it help to redefine file/open so they called an __open__() method on the argument were one defined, otherwise reverting to current behaviour? That way we could just just define an __open__(self) method for path objects. I doubt performance is a huge issue here. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/ From ronaldoussoren at mac.com Thu Jan 26 20:57:30 2006 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Thu, 26 Jan 2006 20:57:30 +0100 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 In-Reply-To: References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <20060126090822.GF18916@xs4all.nl> Message-ID: <99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com> On 26-jan-2006, at 18:04, James Y Knight wrote: > > On Jan 26, 2006, at 7:29 AM, Thomas Heller wrote: >>> And licenses are fluid, it may be a piece of cake to >>> get one of those 'tools' un-GPL'ed, even if they are. >> >> I wouldn't even know whom to ask. > > > On Jan 26, 2006, at 9:22 AM, Ronald Oussoren wrote: >> It shouldn't be too hard to use Python's main configure script to >> calculate >> the information necessary to build libffi. A lot of it is already >> calculated >> anyway (sizeof various type, endianness), some can be hardcoded >> (FFI_NO_RAW_API). >> >> In PyObjC I just compile the files I need from my setup.py. But I >> have an easy task, >> I just need to support two CPU architectures on one OS. > > Let's please be sure the license isn't fine _as is_ before thinking > about asking people to change them or rewriting the build system! I > did not look at *all* the files listed as being GPL, but the first > two in the list were not. Merging the two configure files might be a good idea anyway, that would take away the need to run configure from setup.py. IANAL, but I don't quite get how a GPL'd support script, if there is such a thing, in the build machinery of an extension library would require that Python itself is GPL'd. Anyhow, in my copy of libffi (a snapshot where the last entry in the Changelog is from 2004-04-26) the only the following files at the toplevel op libffi are GPL licensed: config.guess, config.sub, config-ml.in, ltcf-c.sh, ltconfig, ltmain.sh, missing. All of these contain an exception clause like this one in config.guess: # As a special exception to the GNU General Public License, if you # distribute this file as part of a program that contains a # configuration script generated by Autoconf, you may include it under # the same distribution terms that you use for the rest of that program. I'd say that it should be save to include these in the Python distribution. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2157 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20060126/2301384a/attachment.bin From ronaldoussoren at mac.com Thu Jan 26 21:02:49 2006 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Thu, 26 Jan 2006 21:02:49 +0100 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 In-Reply-To: References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <20060126090822.GF18916@xs4all.nl> <864123BC-0DCD-4AD3-8BDA-EC9625090FF4@mac.com> Message-ID: <5B31C78A-E34F-4751-847A-CB35B3E623A8@mac.com> On 26-jan-2006, at 16:33, Thomas Heller wrote: > Ronald Oussoren writes: > >> On 26-jan-2006, at 13:29, Thomas Heller wrote: >> >>> Thomas Wouters writes: >>> >>>> On Thu, Jan 26, 2006 at 09:54:51AM +0100, Thomas Heller wrote: >>>> >>>>> The current state is that ctypes uses GPL'd tools to build libffi, >>>>> and those can't be committed into Python SVN. >>>> >>>>> http://mail.python.org/pipermail/python-dev/2006-January/ >>>>> 059937.html >> >> It shouldn't be too hard to use Python's main configure script to >> calculate the information necessary to build libffi. A lot of it is >> already calculated anyway (sizeof various type, endianness), some can >> be hardcoded (FFI_NO_RAW_API). >> >> In PyObjC I just compile the files I need from my setup.py. But I >> have >> an easy task, I just need to support two CPU architectures on one OS. > > Thanks for the encouragement - Martin suggested a similar approach. > >> From my understanding (which goes not very far) the configuration >> does > two things: determine the set of source files that needs to go in > depending on the cpu architecture, and to determine some > information and > make them available in #defines. I have to check if this is possible > without patching the libffi sources themselves. I guess I could look > into the PyObjC setuop script. PyObjC's solution is kind of a hack: I always compile all files needed for i386 and PPC support and use #ifdef statements to make sure only the files for the current platform are actually used. This is a hack to make it easier to build a universal (aka fat) binary of PyObjC. > > Personally I only have access to machines running windows, linux (x86 > and x86_64), and OS X (plus possibly a Mac running ubuntu), so I could > only do it for those. Maybe support for other architectures can be > added by volunteers? > > Besides: James Y Knight seems to be correct that all the scripts > needed > to build libffi seems to have this special exception from the GPL. I should catch up on python-dev before looking into this. I just noted the same thing :-) Ronald > > Thomas > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ > ronaldoussoren%40mac.com -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2157 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20060126/c38ddc0a/attachment.bin From news-nospam at northportal.net Thu Jan 26 21:07:57 2006 From: news-nospam at northportal.net (VanL) Date: Thu, 26 Jan 2006 13:07:57 -0700 Subject: [Python-Dev] The path module PEP In-Reply-To: References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> Message-ID: One other benefit that I neglected to put into the previous post - I was able to maintain separate cwd's for each tree. An example of use: Each tree has its own context, independent of the context of python: >>> local, local2 = fs.LocalTree(), fs.LocalTree() >>> local.pwd '/home/targoz' >>> local2.pwd '/home/targoz' >>> os.getcwd() '/home/targoz' >>> local.chdir('..') >>> local2.chdir('www') >>> local.pwd '/home' >>> local2.pwd '/home/targoz/www' >>> os.getcwd() '/home/targoz' Remote trees have the same interface: >>> remote_login_data = {'username': 'targoz', 'password': 'my_pass', host': 'hostname.com'} >>> remote = fs.SSHTree(access=remote_login_data) >>> remote.pwd() '/home/nportal' Trees can interact, regardless of whether they are local or remote: >>> local2.listdir('files') ['myfile', 'otherfile.txt'] >>> remote.listdir() [] >>> localfile = local2.open(('targoz/myfile') # Opens a file-like object >>> remote.savefile(localfile, 'remote_name') >>> remote.listdir() ['myfile'] From thomas at xs4all.net Thu Jan 26 21:09:56 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Thu, 26 Jan 2006 21:09:56 +0100 Subject: [Python-Dev] Path inherits from string In-Reply-To: <43D9291C.9040009@holdenweb.com> References: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com> <43D90014.10807@colorstudy.com> <43D9291C.9040009@holdenweb.com> Message-ID: <20060126200956.GI18916@xs4all.nl> On Thu, Jan 26, 2006 at 07:55:08PM +0000, Steve Holden wrote: > Would it help to redefine file/open so they called an __open__() method > on the argument were one defined, otherwise reverting to current behaviour? Not really, open() is (by far!) not the only function that breaks. Most posixmodule functions, by the looks of it, as well as a few in _tkinter and socket. And that's just a quick grep for PyArg_Parse* with '"et' as an argument. Perhaps if __open__ was generalized into __equivalent_string__... but we've been there already. ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas at xs4all.net Thu Jan 26 21:24:56 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Thu, 26 Jan 2006 21:24:56 +0100 Subject: [Python-Dev] stabilizing builds In-Reply-To: <20060125005918.GC18916@xs4all.nl> References: <20060124111528.GB18916@xs4all.nl> <43D6AFC4.3030200@v.loewis.de> <20060125005918.GC18916@xs4all.nl> Message-ID: <20060126202456.GJ18916@xs4all.nl> On Wed, Jan 25, 2006 at 01:59:18AM +0100, Thomas Wouters wrote: > [ iffy isatty behaviour on Solaris ] Considering that: - the approach for opening pty's, while not the only one, is the preferred way of doing it on Solaris, - the actual pty's seem to be completely functional, - the usual way to use pty's is not like the test does (inside a single process), and I'd say using pty's like the test does is extremely unlikely to happen in real life, - testing inside the tty-creating process is quite possibly the reason for the fickleness of the test, since its behaviour isn't guaranteed anywhere, - the test inside a subprocess, the normal way of using pty's, is not failing (as far as I can tell), I'd like to check in the attached patch. It doesn't fix anything, it just removes this one test failure on Solaris. I don't want to skip the entire test, because it still tests whether everything else works as expected, and I don't want spurious failures as they can mask a real failure later in the test. I'd need developer access back to check it in, though. Unless anyone objects, of course :) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- Index: Lib/test/test_pty.py =================================================================== --- Lib/test/test_pty.py (revision 42187) +++ Lib/test/test_pty.py (working copy) @@ -4,6 +4,13 @@ TEST_STRING_1 = "I wish to buy a fish license.\n" TEST_STRING_2 = "For my pet fish, Eric.\n" +# Solaris (at least 2.9 and 2.10) seem to have a ficke isatty(). The first +# test below, testing the result of os.openpty() for tty-ness, sometimes +# (but not always) fails. The second isatty test, in the sub-process, always +# works. Allow that fickle first test to fail on these platforms, since it +# doesn't actually affect functionality. +fickle_isatty = ["sunos5"] + if verbose: def debug(msg): print msg @@ -26,7 +33,7 @@ # " An optional feature could not be imported " ... ? raise TestSkipped, "Pseudo-terminals (seemingly) not functional." - if not os.isatty(slave_fd): + if not os.isatty(slave_fd) and sys.platform not in fickle_isatty: raise TestFailed, "slave_fd is not a tty" # IRIX apparently turns \n into \r\n. Allow that, but avoid allowing other From fredrik at pythonware.com Thu Jan 26 21:33:04 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 26 Jan 2006 21:33:04 +0100 Subject: [Python-Dev] building a module catalogue with buildbot References: <20060111223039.GC6642@activestate.com> Message-ID: any progress ? does the script work in the buildbot setting, or does it need tweaking ? > > Neal Norwitz wrote: > > > > > > > > Does that make sense? We would just need /f's script in SVN. > > > > > > > > > > in python/Tools/something or sandbox/something ? > > > > > > > > python/Doc/tools/something? > > > > > > Fredrik were you still working on that? I can make the changes to the > > > bb master. I thought Trent's suggested placement was good. > > > > iirc, the script needed some minor tweaking (using os.path.splitext to > > test for the module.so extension isn't a good idea), and I don't recall > > if I've fixed that or not... > > > > (probably not, since I never checked it in). > > > > I'll take a look asap. > > alright, I just checked in a > > Doc/tools/listmodules.py > > which attempts to produce a sorted list of all modules available in > a given python build. by default, it prints the list to stdout, which > should be good enough for a "catalog" buildbot step. From theller at python.net Thu Jan 26 21:33:13 2006 From: theller at python.net (Thomas Heller) Date: Thu, 26 Jan 2006 21:33:13 +0100 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <20060126090822.GF18916@xs4all.nl> <99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com> Message-ID: Ronald Oussoren writes: >> On Jan 26, 2006, at 9:22 AM, Ronald Oussoren wrote: >>> It shouldn't be too hard to use Python's main configure script to >>> calculate the information necessary to build libffi. A lot of it is >>> already calculated anyway (sizeof various type, endianness), some >>> can be hardcoded (FFI_NO_RAW_API). >>> >>> In PyObjC I just compile the files I need from my setup.py. But I >>> have an easy task, I just need to support two CPU architectures on >>> one OS. >> > Merging the two configure files might be a good idea anyway, that > would take away the need to run configure from setup.py. IANAL, but I > don't quite get how a GPL'd support script, if there is such a thing, > in the build machinery of an extension library would require that > Python itself is GPL'd. > > Anyhow, in my copy of libffi (a snapshot where the last entry in the > Changelog is from 2004-04-26) the only the following files at the > toplevel op libffi are GPL licensed: config.guess, config.sub, > config-ml.in, ltcf-c.sh, ltconfig, ltmain.sh, missing. All of these > contain an exception clause like this one in config.guess: ctypes libffi is somewhat newer, around 2005-05-09. > # As a special exception to the GNU General Public License, if you > # distribute this file as part of a program that contains a > # configuration script generated by Autoconf, you may include it under > # the same distribution terms that you use for the rest of that program. > > I'd say that it should be save to include these in the Python > distribution. As I said in the other thread (where the discussion should probably be continued anyway): http://mail.python.org/pipermail/python-dev/2006-January/060113.html only aclocal.m4 isn't clear to me about the license. Anyway, it could be that this file isn't needed after all - I don't know enough about the GNU toolchain to be sure. Can anyone comment on this? Neither do I know enough to merge the configure scripts. Contributions would really, really gratefully be accepted. Thomas From mal at egenix.com Thu Jan 26 21:48:16 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 26 Jan 2006 21:48:16 +0100 Subject: [Python-Dev] Path inherits from string In-Reply-To: <43D91ACB.3090401@v.loewis.de> References: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com> <43D8FE24.1030500@egenix.com> <43D91ACB.3090401@v.loewis.de> Message-ID: <43D93590.7040009@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg wrote: >> Please note that inheritance from string will cause the C type >> checks of the form PyString_Check(obj) to return true. >> C code will then assume that it has an object which is >> compatible to string C API which instances aren't. > > Oh, sure they are. Types inheriting from str have the same > layout as str, and C code assuming that layout will work fine > with them. Inheritance works (saying "inheritance *just* works > would deny the many fine details that have been engineered to > actually make it work"). You're right, I forgot about how the .__new__() works on new-style classes and that extra space is allocated appended to the base type object for the extra instance features. >From PEP 253: """ class C(B): pass ... In any case, the work for creating C is done by M's tp_new() slot. It allocates space for an "extended" type structure, containing: the type object; the auxiliary structures (as_sequence etc.); the string object containing the type name (to ensure that this object isn't deallocated while the type object is still referencing it);and some auxiliary storage (to be described later). It initializes this storage to zeros except for a few crucial slots (for example,tp_name is set to point to the type name) and then sets the tp_base slot to point to B. """ Sorry for the FUD, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 26 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From jjl at pobox.com Thu Jan 26 22:21:28 2006 From: jjl at pobox.com (John J Lee) Date: Thu, 26 Jan 2006 21:21:28 +0000 (UTC) Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 In-Reply-To: References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <20060126090822.GF18916@xs4all.nl> <99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com> Message-ID: On Thu, 26 Jan 2006, Thomas Heller wrote: [...] > As I said in the other thread (where the discussion should probably be > continued anyway): > > http://mail.python.org/pipermail/python-dev/2006-January/060113.html > > only aclocal.m4 isn't clear to me about the license. Anyway, it could > be that this file isn't needed after all - I don't know enough about the > GNU toolchain to be sure. Can anyone comment on this? >From 'info autoconf': | The Autoconf macros are defined in several files. Some of the files | are distributed with Autoconf; `autoconf' reads them first. Then it | looks for the optional file `acsite.m4' in the directory that contains | the distributed Autoconf macro files, and for the optional file | `aclocal.m4' in the current directory. Those files can contain your | site's or the package's own Autoconf macro definitions (*note Writing [...] So, I assume aclocal.m4 is under the same license as the rest of the libffi you're using. John From nnorwitz at gmail.com Thu Jan 26 22:31:06 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 26 Jan 2006 13:31:06 -0800 Subject: [Python-Dev] building a module catalogue with buildbot In-Reply-To: References: <20060111223039.GC6642@activestate.com> Message-ID: On 1/26/06, Fredrik Lundh wrote: > any progress ? does the script work in the buildbot setting, or > does it need tweaking ? I haven't gotten to it and won't be able to in the next week+. If no one beats me to it, I will get to it in a few weeks. I've got most of the buildbot machines green and working towards stable. n From ncoghlan at gmail.com Thu Jan 26 22:31:51 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 27 Jan 2006 07:31:51 +1000 Subject: [Python-Dev] The path module PEP In-Reply-To: References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <20060126004113.GD18916@xs4all.nl> <20060126132641.GG18916@xs4all.nl> Message-ID: <43D93FC7.3020401@gmail.com> Michael Hoffman wrote: > I've been using path.py for some time, and I can tell you that it > would be a lot less useful if it no longer behaved like string-plus. As Jason pointed out elsewhere, the strict typechecks that exist *in* the Python core, and the fact that path.py is *outside* that core makes the workaround of subclassing string necessary. Since the PEP process has the power to alter the *core*, then we have other options than "this is a string, only not". Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From fuzzyman at voidspace.org.uk Thu Jan 26 21:28:16 2006 From: fuzzyman at voidspace.org.uk (Fuzzyman) Date: Thu, 26 Jan 2006 20:28:16 +0000 Subject: [Python-Dev] Extension to ConfigParser Message-ID: <43D930E0.6070805@voidspace.org.uk> Hello all, In the past there has been some discussion about a new module to replace ConfigParser. Most notably at http://wiki.python.org/moin/ConfigParserShootout Specific issues that could be addressed include : * Simpler API * Nested subsections * List values * Storing/converting datatypes * Config file schema * Keeps track of order of values Plus other issues. I'm the (co-)author of ConfigObj - http://www.voidspace.org.uk/python/configobj.html This is a reasonably mature project (now in it's fourth incarnation), and is being used in projects like `Bazaar `_ and `PlanetPlus `_. It occurs to me that the ConfigObj API and syntax is *almost* fully compatible with ConfigParser. It would be possible to extend to the ConfigObj API to be backwards compatible with ConfigParser. This would bring the added benefits of ConfigObj, without needing to add an extra module to the standard library. Well nearly. ConfigObj supports config file schema with (optional) type conversion, through a companion module called validate. This could be included or left as an added option. Anyway. If this stands a *chance* of acceptance, I'll write the PEP (and if accepted, do the work - which is not inconsiderable). Summary of ConfigObj ================ ConfigObj is a Python 2.2 compatible config file parser. It's major feature is simplicity of use. It reads (and writes) INI file like config files and presents the members using a dictionary interface. The order of keys/sections is preserved, and it allows nested subsections to any level : e.g. :: key = value [section] key = value [[sub-section]] key = value It is fully documented with a barrage of doctests. All comments in the config file are also preserved as attributes of the object, and will be written back out. This can be useful for including comments in programatically generated config files. It is integrated with a powerful validation system. Difficulties & Differences ================= A ConfigObj instance is a sub-class of the dictionary datatpe. This means that the ``get`` method of ConfigParser clashes. ConfigObj allows values in the root section (why not ?). ConfigObj doesn't support line continuations (it does all multi-line values through the use of triple quoted strings). ConfigObj currently only allows '=' as a valid divider. Creating ConfigParser (and related classes) compatibility is a big job. Solution ===== All of these problems (if deemed necessary) can be resolved. Either through options, or just by extending the ConfigObj API. I'm happy to put the work in. Comments ? All the best, Fuzzyman http://www.voidspace.org.uk/python/index.shtml From jimjjewett at gmail.com Thu Jan 26 22:48:41 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu, 26 Jan 2006 16:48:41 -0500 Subject: [Python-Dev] The path module (class) PEP Message-ID: (1) What is the advantage of module Path vs just turning os.path into a class? (except that people already import from os.path, so I suppose it would need to be os.Path) (2) It sounds like quite a few stdlib calls will be both deprecated and wrapped. Having a new stdlib module rely on deprecated features leaves a bad taste. Perhaps move the functionality to the Path class and then forward from the current (about to be deprecated) modules? -jJ From tim.peters at gmail.com Fri Jan 27 03:41:53 2006 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 26 Jan 2006 21:41:53 -0500 Subject: [Python-Dev] stabilizing builds In-Reply-To: <20060126202456.GJ18916@xs4all.nl> References: <20060124111528.GB18916@xs4all.nl> <43D6AFC4.3030200@v.loewis.de> <20060125005918.GC18916@xs4all.nl> <20060126202456.GJ18916@xs4all.nl> Message-ID: <1f7befae0601261841g30a2994ak34fa7742d2971091@mail.gmail.com> [Thomas Wouters] > ... > I'd need developer access back to check it in, though. AFAICT, twouters has developer access to the Python project -- although maybe someone else re-enabled that w/o mentioning it here. > Unless anyone objects, of course :) Of course I object! I bow to the group will, though. From robey at lag.net Fri Jan 27 05:42:16 2006 From: robey at lag.net (Robey Pointer) Date: Thu, 26 Jan 2006 20:42:16 -0800 Subject: [Python-Dev] [Doc-SIG] that library reference, again In-Reply-To: References: <4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com> <43B4A238.2030101@python.org> <51A7717B-3832-477E-BED5-3C36DCA20336@lag.net> Message-ID: <05663E18-CDFB-4B18-A515-BAA21C48C1FD@lag.net> On 29 Dec 2005, at 23:13, Fredrik Lundh wrote: > Robey Pointer wrote: >>> [Fredrik Lundh] >>>> Really? >>> >>> Yes, really. >> >> Just out of curiosity (really -- not trying to jump into the flames) >> why not just use epydoc? If it's good enough for 3rd-party python >> libraries, isn't that just a small step from being good enough for >> the builtin libraries? > > but epydoc is a docstring-based format, right? > > I'm trying to put together a light-weight alternative to the markup > used for, primarily, the current library reference. > > moving all of (or parts of) the reference documentation in to the > library source code would be an alternative, of course [1], but that > would basically mean starting over from scratch. I think that would be a good thing to do no matter what markup is used. It always irritates me when I do 'help(sys.something)' and get one line of ASCII art that's probably useful to the module author but nobody else. robey From robey at lag.net Fri Jan 27 05:45:12 2006 From: robey at lag.net (Robey Pointer) Date: Thu, 26 Jan 2006 20:45:12 -0800 Subject: [Python-Dev] [Doc-SIG] that library reference, again In-Reply-To: <60ed19d40512301829q7a0df01bv3ed782fe28cbcf7a@mail.gmail.com> References: <4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com> <43B4A238.2030101@python.org> <51A7717B-3832-477E-BED5-3C36DCA20336@lag.net> <60ed19d40512301829q7a0df01bv3ed782fe28cbcf7a@mail.gmail.com> Message-ID: On 30 Dec 2005, at 18:29, Christopher Armstrong wrote: > On 12/30/05, Robey Pointer wrote: >> >> Just out of curiosity (really -- not trying to jump into the flames) >> why not just use epydoc? If it's good enough for 3rd-party python >> libraries, isn't that just a small step from being good enough for >> the builtin libraries? > > It's not really even "good enough" for a lot of my usage without some > seriously evil hacks. The fundamental design decision of epydoc to > import code, plus some other design decisions on the way it figures > types and identity seriously hinder its utility. Ever notice how > trying to document your third-party-library-using application will > *also* document that third party library, for example? Or how it'll > blow up when you're trying to document your gtk-using application on a > remote server without an X server running? Or how it just plain blows > right up with most Interface systems? etc. Err... no, I haven't. But I may be using a more recent version than you were (I'm using 2.1). It's not perfect, and occasionally annoying, but much better than anything else I've found. Sounds like there's some political reason it's hated, but thought I would bring it up anyway. robey From p.f.moore at gmail.com Fri Jan 27 11:16:09 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 27 Jan 2006 10:16:09 +0000 Subject: [Python-Dev] The path module PEP In-Reply-To: <43D8E0E9.9080202@ofai.at> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <79990c6b0601260515w22ae9e2dy22d265acc4bce7c3@mail.gmail.com> <43D8E0E9.9080202@ofai.at> Message-ID: <79990c6b0601270216x2c43449cj534427f7a8f5234c@mail.gmail.com> On 1/26/06, Stefan Rank wrote: > on 26.01.2006 14:15 Paul Moore said the following: > [snip] > > > > Also note that my example Path("C:", "Windows", "System32") above is > > an *absolute* path on Windows. But a relative (albeit stupidly-named > > :-)) path on Unix. How would that be handled? > > wrong, Path("C:", "Windows", "System32") is a relative path on windows. > see below. Hmm, relative to the CWD on C: is a valid concept, and that is a potential meaning. I hadn't thought of that. > > Not that os.path gets it perfect: > > > > Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on win32 > > Type "help", "copyright", "credits" or "license" for more information. > >>>> import os > >>>> os.path.join("C:", "Windows", "System32") > > 'C:Windows\\System32' > >>>> os.path.join(".", os.path.join("C:", "Windows", "System32")) > > '.\\C:Windows\\System32' > > > > this is misleading. observe:: > > In [1]: import os > > In [2]: os.path.join(".", os.path.join("C:", "Windows", "System32")) > Out[2]: '.\\C:Windows\\System32' > > but:: > > In [3]: os.path.join(".", os.path.join("C:\\", "Windows", "System32")) > Out[3]: 'C:\\Windows\\System32' > > > The second example uses an absolute path as second argument, and as > os.path.join should do, the first argument is discarded. > > The first case is arguably a bug, since, on windows, C:Windows\System32 > is a path relative to the *current directory on disk C:* > If the cwd on C: would be C:\temp then C:Windows\System32 would point to > C:\temp\Windows\System32 > > The problem is that Windows has a cwd per partition... > (I cannot even guess why ;-) Thanks for the clarification, you are right in your analysis. However, it doesn't really affect my main point, which was that there should be no such thing as a relative Path (please note - I say "Path" here, to refer to the new Path object, as opposed to the general concept of an OS file path). [...] > > Arguably, Path objects should always maintain an absolute path - there > > should be no such thing as a relative Path. So you would have > > you realise that one might need and/or want to represent a relative path? Absolutely. But not a Path (see distinction above). Aaron Bingham's analogy with time/timedelta applies well here. Relative paths, like relative times, have their own special semantics, which deserve to be addressed in a separate class. You argue that time is "merely" a timedelta with a fixed start point. I'd disagree - the key point with timedeltas is that they need careful handling (DST issues, for example) _depending upon precisely what they are added to_ - these issues are avoided by the time type exactly because it has a constant base. In exactly the same way, absolute paths have simpler semantics precisely because they are absolute. Paul. From steve at holdenweb.com Fri Jan 27 11:42:22 2006 From: steve at holdenweb.com (Steve Holden) Date: Fri, 27 Jan 2006 10:42:22 +0000 Subject: [Python-Dev] SourceForge Download Page, Subversion Home Page Message-ID: <43D9F90E.1040703@holdenweb.com> You may be aware that Tim Parkin's work on our "next-generation" web presence has borne fruit in the shape of beta.python.org. While there's still a lot to be done Tim has given us a great start by creating a framework that makes it rather easier to manage content. I'm just starting to work on the beta site content, and this is causing some examination of the existing web content. Is there any way to affect the target of the very prominent "Download Python" link on http://sourceforge.net/projects/python/ ? I ask because the link currently takes you to a page whose title is "Exiting with Error" and whose content is largely "No File Packages". While it's not where you arrive from a Google search for "python download" it is nevertheless a legitimate way (just now) to approach the project, and should therefore really be live. Is there any chance some SF admin could point it to http://www.python.org/download/ There's the further issue that the entire HTML body of http://svn.python.org/ currently reads It would seem like the logical place to have pointers to Subversion software (downloads, documentation, operating instructions) together with an *annotated* summary of http://svn.python.org/projects/ and links back to the main developer site at the very least. I'm not even sure where that web content is under SVN control at the moment. The web revision is going to lead to several of these kinds of discussions. I'll try to only spill on to the python-dev list what impinges on developers. Your opinions on these specific issues are probably the most significant. [pydotorgers: let's try not to spam python-dev with any discussions] regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/ From stefan.rank at ofai.at Fri Jan 27 11:47:21 2006 From: stefan.rank at ofai.at (Stefan Rank) Date: Fri, 27 Jan 2006 11:47:21 +0100 Subject: [Python-Dev] The path module PEP In-Reply-To: <79990c6b0601270216x2c43449cj534427f7a8f5234c@mail.gmail.com> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <79990c6b0601260515w22ae9e2dy22d265acc4bce7c3@mail.gmail.com> <43D8E0E9.9080202@ofai.at> <79990c6b0601270216x2c43449cj534427f7a8f5234c@mail.gmail.com> Message-ID: <43D9FA39.6060407@ofai.at> on 27.01.2006 11:16 Paul Moore said the following: > [...] >>> Arguably, Path objects should always maintain an absolute path - there >>> should be no such thing as a relative Path. So you would have >> you realise that one might need and/or want to represent a relative path? > > Absolutely. But not a Path (see distinction above). > > Aaron Bingham's analogy with time/timedelta applies well here. > Relative paths, like relative times, have their own special semantics, > which deserve to be addressed in a separate class. > > You argue that time is "merely" a timedelta with a fixed start point. > I'd disagree - the key point with timedeltas is that they need careful > handling (DST issues, for example) _depending upon precisely what they > are added to_ - these issues are avoided by the time type exactly > because it has a constant base. In exactly the same way, absolute > paths have simpler semantics precisely because they are absolute. > > Paul. I see your point. I guess there are two options: - `Path` as an enhanced string type that bundles methods related to file system addressing - `Path`s that accurately reflect the possible (abstract) paths. There would be a Path and a PathDelta (with appropriate combining semantics), and probably a UnixPath, a WindowsPath, an URLPath maybe. And there need to be appropriate methods for combining them with/creating them from strings. I actually think the latter would be very neat and somewhere else in this thread someone talks about his `Tree` - `Path` - `File` classes with specialised subclasses. The first option, however, has the benefit of simplicity and there is a working implementation. Well - I'm not the one to decide. And I think anything that bundles path related stuff (using a little object-orientation) and cleans up the standard library is a step forward. cheers, s From thomas at xs4all.net Fri Jan 27 12:00:53 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Fri, 27 Jan 2006 12:00:53 +0100 Subject: [Python-Dev] stabilizing builds In-Reply-To: <1f7befae0601261841g30a2994ak34fa7742d2971091@mail.gmail.com> References: <20060124111528.GB18916@xs4all.nl> <43D6AFC4.3030200@v.loewis.de> <20060125005918.GC18916@xs4all.nl> <20060126202456.GJ18916@xs4all.nl> <1f7befae0601261841g30a2994ak34fa7742d2971091@mail.gmail.com> Message-ID: <20060127110053.GO5045@xs4all.nl> On Thu, Jan 26, 2006 at 09:41:53PM -0500, Tim Peters wrote: > [Thomas Wouters] > > ... > > I'd need developer access back to check it in, though. > AFAICT, twouters has developer access to the Python project -- > although maybe someone else re-enabled that w/o mentioning it here. I meant svn-checkin-access (it got disabled for disuse a while back.) If I weren't schizofrenic in sysadmin jobs I could add it myself, but I can't do that. Can I? I guess I can't. Who am I again? Someone-else'ly y'rs, -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From edloper at gradient.cis.upenn.edu Fri Jan 27 13:55:01 2006 From: edloper at gradient.cis.upenn.edu (Edward Loper) Date: Fri, 27 Jan 2006 07:55:01 -0500 Subject: [Python-Dev] [Doc-SIG] that library reference, again In-Reply-To: References: <4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com> <43B4A238.2030101@python.org> <51A7717B-3832-477E-BED5-3C36DCA20336@lag.net> <60ed19d40512301829q7a0df01bv3ed782fe28cbcf7a@mail.gmail.com> Message-ID: <43DA1825.8060608@gradient.cis.upenn.edu> Robey Pointer wrote: On 30 Dec 2005, at 18:29, Christopher Armstrong wrote: >> [epydoc] is not really even "good enough" for a lot of my usage without some >> seriously evil hacks. The fundamental design decision of epydoc to >> import code, plus some other design decisions on the way it figures >> types and identity seriously hinder its utility. [...] As a side note, I've done a significant amount of work on a newer version of epydoc that supports both inspection & source-code parsing (and can merge the info from both sources, if they're available). Hopefully this new version will address some of these concerns (and some other issues I've heard raised). But I haven't had time to work on it in a while -- I need to concentrate on getting my phd thesis done. If anyone is interested in working on this new version, just let me know. :) -Edward From tim.peters at gmail.com Fri Jan 27 16:49:59 2006 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 27 Jan 2006 10:49:59 -0500 Subject: [Python-Dev] SourceForge Download Page, Subversion Home Page In-Reply-To: <43D9F90E.1040703@holdenweb.com> References: <43D9F90E.1040703@holdenweb.com> Message-ID: <1f7befae0601270749w1dbb25f5lb785e7294c81fcff@mail.gmail.com> [Steve Holden] > Is there any way to affect the target of the very prominent "Download > Python" link on > http://sourceforge.net/projects/python/ ? > > I ask because the link currently takes you to a page whose title is > "Exiting with Error" and whose content is largely "No File Packages". > While it's not where you arrive from a Google search for "python > download" it is nevertheless a legitimate way (just now) to approach the > project, and should therefore really be live. > > Is there any chance some SF admin could point it to > > http://www.python.org/download/ Sorry, looks like SF auto-creates the download "button", hardwired to the project's SF "Files" section. The Python project doesn't have any files on SF anymore. If you go to http://sourceforge.net/projects/zodb/ you'll see the equally prominent "Download ZODB and ZEO" button, pointing to that project's equally ZODB-free Files area. But, in that case, you'll see that there _is_ a package, named README, with a "release" named "How to get the source". If you click on that and fight your way through the download process, you eventually end up with file "zodb.txt", which contains a real download URL for ZODB . Overall I'm not sure that's an improvement, but it was the best I could come up with 2 years ago when ZODB stopped using SF for downloads. From thomas at xs4all.net Fri Jan 27 16:59:22 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Fri, 27 Jan 2006 16:59:22 +0100 Subject: [Python-Dev] SourceForge Download Page, Subversion Home Page In-Reply-To: <1f7befae0601270749w1dbb25f5lb785e7294c81fcff@mail.gmail.com> References: <43D9F90E.1040703@holdenweb.com> <1f7befae0601270749w1dbb25f5lb785e7294c81fcff@mail.gmail.com> Message-ID: <20060127155922.GL18916@xs4all.nl> On Fri, Jan 27, 2006 at 10:49:59AM -0500, Tim Peters wrote: > If you go to > > http://sourceforge.net/projects/zodb/ > > you'll see the equally prominent "Download ZODB and ZEO" button, > pointing to that project's equally ZODB-free Files area. But, in that > case, you'll see that there _is_ a package, named README, with a > "release" named "How to get the source". If you click on that and > fight your way through the download process, you eventually end up > with file "zodb.txt", which contains a real download URL for ZODB > . > > Overall I'm not sure that's an improvement, but it was the best I > could come up with 2 years ago when ZODB stopped using SF for > downloads. What if the package wasn't named README, but rather 'www.python.org' with a release named "visit www.python.org/download instead"? I'm not sure if that package/release name is allowed ;P -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From tim.peters at gmail.com Fri Jan 27 16:59:45 2006 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 27 Jan 2006 10:59:45 -0500 Subject: [Python-Dev] stabilizing builds In-Reply-To: <20060127110053.GO5045@xs4all.nl> References: <20060124111528.GB18916@xs4all.nl> <43D6AFC4.3030200@v.loewis.de> <20060125005918.GC18916@xs4all.nl> <20060126202456.GJ18916@xs4all.nl> <1f7befae0601261841g30a2994ak34fa7742d2971091@mail.gmail.com> <20060127110053.GO5045@xs4all.nl> Message-ID: <1f7befae0601270759o7bdaac5j1b6e630de4844d75@mail.gmail.com> [Thomas] >>> I'd need developer access back to check it in, though. [Tim] >> AFAICT, twouters has developer access to the Python project -- >> although maybe someone else re-enabled that w/o mentioning it here. [Thomas] > I meant svn-checkin-access (it got disabled for disuse a while back.) I know. AFAICT, you (twouters) already have it. There's a "Yes" in the twouters row under the "CVS Access" column on the Python project's Members admin page. Have you tried checking in? What happens when you do? If it "doesn't work", one possibility is that you did a read-only SVN _checkout_, and it's actually SVN griping at you. > If I weren't schizofrenic in sysadmin jobs I could add it myself, but I can't do > that. Can I? I guess I can't. You shouldn't be able to add yourself -- you're not listed as a godly Python project admin, just as a mortal Python developer. Many people who do checkins routinely (like Neal Norwitz) are in the same boat wrt that. > Who am I again? Thomas Wouters ;-) Your SourceForge name in the Python project is twouters. From tim.peters at gmail.com Fri Jan 27 17:04:18 2006 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 27 Jan 2006 11:04:18 -0500 Subject: [Python-Dev] stabilizing builds In-Reply-To: <1f7befae0601270759o7bdaac5j1b6e630de4844d75@mail.gmail.com> References: <20060124111528.GB18916@xs4all.nl> <43D6AFC4.3030200@v.loewis.de> <20060125005918.GC18916@xs4all.nl> <20060126202456.GJ18916@xs4all.nl> <1f7befae0601261841g30a2994ak34fa7742d2971091@mail.gmail.com> <20060127110053.GO5045@xs4all.nl> <1f7befae0601270759o7bdaac5j1b6e630de4844d75@mail.gmail.com> Message-ID: <1f7befae0601270804m3615014eyfbdfedafabf9bc75@mail.gmail.com> I suppose another possibility for why twouters couldn't check in is because someone added him to the project's cvs_acls script. If so, I don't know anything about how to get that changed. From tim.peters at gmail.com Fri Jan 27 17:10:33 2006 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 27 Jan 2006 11:10:33 -0500 Subject: [Python-Dev] stabilizing builds In-Reply-To: <1f7befae0601270759o7bdaac5j1b6e630de4844d75@mail.gmail.com> References: <20060124111528.GB18916@xs4all.nl> <43D6AFC4.3030200@v.loewis.de> <20060125005918.GC18916@xs4all.nl> <20060126202456.GJ18916@xs4all.nl> <1f7befae0601261841g30a2994ak34fa7742d2971091@mail.gmail.com> <20060127110053.GO5045@xs4all.nl> <1f7befae0601270759o7bdaac5j1b6e630de4844d75@mail.gmail.com> Message-ID: <1f7befae0601270810v1e0d5fb7v41b12c2c03769c89@mail.gmail.com> [Tim] ... > AFAICT, you (twouters) already have it. There's a "Yes" in > the twouters row under the "CVS Access" column on the Python project's > Members admin page. Have you tried checking in? What happens when > you do? ... LOL -- what a bubblehead I am! Whether you can check in has nothing to do with SourceForge CVS access anymore -- never mind, and sorry. I don't have the power to admin svn.python.org either. Martin? From theller at python.net Fri Jan 27 17:14:31 2006 From: theller at python.net (Thomas Heller) Date: Fri, 27 Jan 2006 17:14:31 +0100 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <20060126090822.GF18916@xs4all.nl> <99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com> Message-ID: John J Lee writes: > On Thu, 26 Jan 2006, Thomas Heller wrote: > [...] >> As I said in the other thread (where the discussion should probably be >> continued anyway): >> >> http://mail.python.org/pipermail/python-dev/2006-January/060113.html >> >> only aclocal.m4 isn't clear to me about the license. Anyway, it could >> be that this file isn't needed after all - I don't know enough about the >> GNU toolchain to be sure. Can anyone comment on this? > >>From 'info autoconf': > > | The Autoconf macros are defined in several files. Some of the files > | are distributed with Autoconf; `autoconf' reads them first. Then it > | looks for the optional file `acsite.m4' in the directory that contains > | the distributed Autoconf macro files, and for the optional file > | `aclocal.m4' in the current directory. Those files can contain your > | site's or the package's own Autoconf macro definitions (*note Writing > [...] > > So, I assume aclocal.m4 is under the same license as the rest of the > libffi you're using. I cannot uinderstand your reasoning. How can 'info autoconf' incluence the license of the aclocal.m4 file? Or do I misunderstand something? Given that all kind of *nix experts are here on this list - can someone tell if aclocal.m4 is needed for building libffi at all or not? Thomas From theller at python.net Fri Jan 27 18:03:58 2006 From: theller at python.net (Thomas Heller) Date: Fri, 27 Jan 2006 18:03:58 +0100 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: <1138378937.3090.255.camel@localhost.localdomain> (Anthony Green's message of "Fri, 27 Jan 2006 08:22:17 -0800") References: <1137071518.12975.6.camel@devel002> <1137186878.3098.87.camel@localhost.localdomain> <1138378937.3090.255.camel@localhost.localdomain> Message-ID: [I've added python-dev to cc:] Anthony Green writes: > On Fri, 2006-01-27 at 17:08 +0100, Thomas Heller wrote: >> Anyway, another question is: Is aclocal.m4 needed at all for building >> (or maybe for regenerating the configure scripts), or is it optional? > > aclocal.m4 is required, but is only used as a build-time tool. The fact > that aclocal.m4 is distributed under the GPL should have no impact on > the licensing terms used for software built using aclocal.m4. If I understand correctly this means that the Python source distribution would have to be GPL licensed, while the built programs would be able to use another license. I'm pretty sure this kills the whole idea (to include libffi in python). Thanks, Thomas From ronaldoussoren at mac.com Fri Jan 27 18:04:03 2006 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Fri, 27 Jan 2006 18:04:03 +0100 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 In-Reply-To: References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <20060126090822.GF18916@xs4all.nl> <99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com> Message-ID: <8A7F07B4-8847-4D37-B96F-E9D86AC429C6@mac.com> On 27-jan-2006, at 17:14, Thomas Heller wrote: > John J Lee writes: > >> On Thu, 26 Jan 2006, Thomas Heller wrote: >> [...] >>> As I said in the other thread (where the discussion should >>> probably be >>> continued anyway): >>> >>> http://mail.python.org/pipermail/python-dev/2006-January/060113.html >>> >>> only aclocal.m4 isn't clear to me about the license. Anyway, it >>> could >>> be that this file isn't needed after all - I don't know enough >>> about the >>> GNU toolchain to be sure. Can anyone comment on this? >> >>> From 'info autoconf': >> >> | The Autoconf macros are defined in several files. Some of the >> files >> | are distributed with Autoconf; `autoconf' reads them first. >> Then it >> | looks for the optional file `acsite.m4' in the directory that >> contains >> | the distributed Autoconf macro files, and for the optional file >> | `aclocal.m4' in the current directory. Those files can contain >> your >> | site's or the package's own Autoconf macro definitions (*note >> Writing >> [...] >> >> So, I assume aclocal.m4 is under the same license as the rest of the >> libffi you're using. > > I cannot uinderstand your reasoning. How can 'info autoconf' > incluence > the license of the aclocal.m4 file? Or do I misunderstand something? > > Given that all kind of *nix experts are here on this list - can > someone > tell if aclocal.m4 is needed for building libffi at all or not? aclocal.m4 is needed to build configure, it's a library of configure fragments. I try to stay away from configure as far as possible so cannot say if those fragments are really needed. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2157 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20060127/73605721/attachment-0001.bin From theller at python.net Fri Jan 27 19:32:54 2006 From: theller at python.net (Thomas Heller) Date: Fri, 27 Jan 2006 19:32:54 +0100 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: <1138381908.3090.259.camel@localhost.localdomain> (Anthony Green's message of "Fri, 27 Jan 2006 09:11:48 -0800") References: <1137071518.12975.6.camel@devel002> <1137186878.3098.87.camel@localhost.localdomain> <1138378937.3090.255.camel@localhost.localdomain> <1138381908.3090.259.camel@localhost.localdomain> Message-ID: Anthony Green writes: > On Fri, 2006-01-27 at 18:03 +0100, Thomas Heller wrote: >> [I've added python-dev to cc:] >> >> Anthony Green writes: >> >> > On Fri, 2006-01-27 at 17:08 +0100, Thomas Heller wrote: >> >> Anyway, another question is: Is aclocal.m4 needed at all for building >> >> (or maybe for regenerating the configure scripts), or is it optional? >> > >> > aclocal.m4 is required, but is only used as a build-time tool. The fact >> > that aclocal.m4 is distributed under the GPL should have no impact on >> > the licensing terms used for software built using aclocal.m4. >> >> If I understand correctly this means that the Python source distribution >> would have to be GPL licensed, while the built programs would be able to >> use another license. >> >> I'm pretty sure this kills the whole idea (to include libffi in python). > > I guess I wasn't clear. aclocal.m4 is just a tool used to build libffi. > Like your C compiler. Bundling it with the Python source distribution > should have no impact on the licensing of Python itself, since it isn't > really part of the resulting Python binary - just like your C compiler > isn't. I guess I understood this already. The difference to the C compiler is that the compiler is not 'bundled' with Python, it is installed separately. Can anyone of the python-dev core team comment: can we live with the GPL licensed aclocal.m4 file, in the source distribution and in SVN? Thomas From rasky at develer.com Fri Jan 27 20:00:00 2006 From: rasky at develer.com (Giovanni Bajo) Date: Fri, 27 Jan 2006 20:00:00 +0100 Subject: [Python-Dev] (libffi) Re: Copyright issue References: <1137071518.12975.6.camel@devel002><1137186878.3098.87.camel@localhost.localdomain><1138378937.3090.255.camel@localhost.localdomain><1138381908.3090.259.camel@localhost.localdomain> Message-ID: <074501c62373$df526cd0$bf03030a@trilan> Thomas Heller wrote: >>>> On Fri, 2006-01-27 at 17:08 +0100, Thomas Heller wrote: >>>>> Anyway, another question is: Is aclocal.m4 needed at all for building >>>>> (or maybe for regenerating the configure scripts), or is it optional? >>>> >>>> aclocal.m4 is required, but is only used as a build-time tool. The fact >>>> that aclocal.m4 is distributed under the GPL should have no impact on >>>> the licensing terms used for software built using aclocal.m4. >>> >>> If I understand correctly this means that the Python source distribution >>> would have to be GPL licensed, while the built programs would be able to >>> use another license. >>> >>> I'm pretty sure this kills the whole idea (to include libffi in python). >> >> I guess I wasn't clear. aclocal.m4 is just a tool used to build libffi. >> Like your C compiler. Bundling it with the Python source distribution >> should have no impact on the licensing of Python itself, since it isn't >> really part of the resulting Python binary - just like your C compiler >> isn't. > > I guess I understood this already. The difference to the C compiler is > that the compiler is not 'bundled' with Python, it is installed > separately. That's no different. If you burn a CD containing a copy of the GCC and a copy of a commercial software you are not violating any license. If you distribute an .ISO file containing a copy of the GCC and a copy of a commercial software, you are not violating any license. If you distribute a .zip file containing a copy of GCC and a copy of a commercial software, you are not violating any license. There is an important difference between aggreggation of different programs, and static/dynamic linking of parts. Autoconf is a build tool, and it does not impose any license on the software you use it with. Plus some files have the special exception from GPL so that even the files *generated* by autoconf (and partly made of pieces of autoconf) do not require to be GPL. This is exactly like GCC's runtime library (libgcc, and also libstdc++) which are GPL with the special exception, and you can use them also for commercial products. Also, do not understimate previous history. There are zillions of programs out there using Autconf and *not* being GPL. -- Giovanni Bajo From jjl at pobox.com Fri Jan 27 21:43:20 2006 From: jjl at pobox.com (John J Lee) Date: Fri, 27 Jan 2006 20:43:20 +0000 (UTC) Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 In-Reply-To: References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <20060126090822.GF18916@xs4all.nl> <99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com> Message-ID: On Fri, 27 Jan 2006, Thomas Heller wrote: > John J Lee writes: > > > On Thu, 26 Jan 2006, Thomas Heller wrote: > >> only aclocal.m4 isn't clear to me about the license. Anyway, it could > >> be that this file isn't needed after all - I don't know enough about the > >> GNU toolchain to be sure. Can anyone comment on this? > > > >>From 'info autoconf': > > > > | The Autoconf macros are defined in several files. Some of the files > > | are distributed with Autoconf; `autoconf' reads them first. Then it > > | looks for the optional file `acsite.m4' in the directory that contains > > | the distributed Autoconf macro files, and for the optional file > > | `aclocal.m4' in the current directory. Those files can contain your > > | site's or the package's own Autoconf macro definitions (*note Writing > > [...] > > > > So, I assume aclocal.m4 is under the same license as the rest of the > > libffi you're using. > > I cannot uinderstand your reasoning. How can 'info autoconf' incluence > the license of the aclocal.m4 file? Or do I misunderstand something? OK. I now notice I was confused as to why the license issue arose in the first place, but FWIW: My point was just that the autoconf info pages explain (if I read them right) that one keeps one's project-specific autoconf m4 macros in a file named 'aclocal.m4'. It's not a file distributed with autoconf, it's just a file naming convention, so I made the assumption that since libffi is apparently OK to include in Python, so must be its aclocal.m4 (even if some of the macros in the libffi aclocal.m4 originally derived from some other project). But I'm afraid this would fail with an AssertionError if it weren't pseudocode: import legally_compatible as compat from autoconf import acfiles from ctypes import libffi if compat(acfiles, libffi) and compat(libffi, python): assert compat(acfiles, python), "John is not legally competent" :-( John From theller at python.net Fri Jan 27 21:48:02 2006 From: theller at python.net (Thomas Heller) Date: Fri, 27 Jan 2006 21:48:02 +0100 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: (Andrew Pinski's message of "Fri, 27 Jan 2006 13:37:48 -0500") References: <1137071518.12975.6.camel@devel002> <1137186878.3098.87.camel@localhost.localdomain> <1138378937.3090.255.camel@localhost.localdomain> <1138381908.3090.259.camel@localhost.localdomain> Message-ID: Andrew Pinski writes: > Does phython already use autoconf? I think it does, if so then there > should be no issues. [Anthony Green] >>> I guess I wasn't clear. aclocal.m4 is just a tool used to build >>> libffi. Like your C compiler. Bundling it with the Python source >>> distribution should have no impact on the licensing of Python >>> itself, since it isn't really part of the resulting Python binary - >>> just like your C compiler isn't. [Thomas Heller] >> I guess I understood this already. The difference to the C compiler is >> that the compiler is not 'bundled' with Python, it is installed >> separately. > "Giovanni Bajo" writes: > That's no different. If you burn a CD containing a copy of the GCC and a > copy of a commercial software you are not violating any license. If you > distribute an .ISO file containing a copy of the GCC and a copy of a > commercial software, you are not violating any license. If you distribute a > .zip file containing a copy of GCC and a copy of a commercial software, you > are not violating any license. > > There is an important difference between aggreggation of different programs, > and static/dynamic linking of parts. Autoconf is a build tool, and it does > not impose any license on the software you use it with. Plus some files have > the special exception from GPL so that even the files *generated* by > autoconf (and partly made of pieces of autoconf) do not require to be GPL. > This is exactly like GCC's runtime library (libgcc, and also libstdc++) > which are GPL with the special exception, and you can use them also for > commercial products. > > Also, do not understimate previous history. There are zillions of programs > out there using Autconf and *not* being GPL. Ok, understood - there is no problem. Hopefully the rest of the pythondev team is also convinced. The only question I have (maybe this is too off-topic, but since we're here already): Nearly all the tools that autoconf and automake use are careful to include an exception clause to the GPL license. Why is aclocal.m4 different? Thomas From stephen at xemacs.org Fri Jan 27 21:50:04 2006 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 28 Jan 2006 05:50:04 +0900 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 In-Reply-To: (Thomas Heller's message of "Fri, 27 Jan 2006 17:14:31 +0100") References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <20060126090822.GF18916@xs4all.nl> <99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com> Message-ID: <871wyt75vn.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Thomas" == Thomas Heller writes: Thomas> I cannot uinderstand your reasoning. How can 'info Thomas> autoconf' incluence the license of the aclocal.m4 file? It doesn't. The point is the documentation explains that all of the other files are _part of autoconf_, and come under the license of autoconf. They come with special additional rights for users so that the GPL does *not* apply to a project simply because the project uses a configure script generated by autoconf. The aclocal file is not part of autoconf and so doesn't come under that license, but rather the license its author gives it. Presumably that is the same as the rest of the application (here libffi, since an aclocal for libffi was almost certainly written by a libffi developer). Thomas> Given that all kind of *nix experts are here on this list Thomas> - can someone tell if aclocal.m4 is needed for building Thomas> libffi at all or not? Not if libffi is distributed with a prebuilt configure script. Otherwise, not using the distributed aclocal.m4 may be possible, but it's a bad idea. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From jason.orendorff at gmail.com Sat Jan 28 00:19:52 2006 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Fri, 27 Jan 2006 18:19:52 -0500 Subject: [Python-Dev] / as path join operator In-Reply-To: <87k6cnaabt.fsf@tleepslib.sk.tsukuba.ac.jp> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> <87k6cnaabt.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: It's controversial that Path subclasses str. Some people think it's flat-out wrong. Even Bjorn argues that it's a practicality-vs-purity tradeoff. But a strong argument can be made that Path *should* be a string subclass, practicality be damned. Proof follows. I. Here's an example of the sort of thing you might say if you did *not* think of paths as strings: On 1/25/06, Stephen J. Turnbull wrote: > I think it's logical to expect that > Path('home') / 'and/or' > points to a file named "and/or" in directory "home", not to a file > named "or" in directory "home/and". This makes no sense whatsoever. Ergo, by reductio ad absurdum, paths are strings. II. And here is the sort of thing you'd say if you thought of paths *solely* as strings: > (2) Note that '/' is also the path separator used by URIs, which RFC > 2396 gives different semantics from Unix. Most of my Python usage to > date has been heavily web-oriented, and I'd have little use for / > unless it follows RFC 2396. The quandary is resolved by pointing out that URIs are not paths (in the sense of os.path and generally this whole horrible thread). Thus not all strings are paths. Hence the paths are a proper subset of the strings. By the existence of os.path, they have their own commonly-used operations. By definition, then, Path is a subclass of string, QED. Do I really buy all this? I dunno. To say "paths aren't strings" is all very well, and in a very abstract sense I almost agree--but you have to admit it sort of flies in the face of, you know, reality. Filesystem paths are in fact strings on all operating systems I'm aware of. And it's no accident or performance optimization. It's good design. -j From martin at v.loewis.de Sat Jan 28 00:25:12 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 28 Jan 2006 00:25:12 +0100 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: References: <1137071518.12975.6.camel@devel002> <1137186878.3098.87.camel@localhost.localdomain> <1138378937.3090.255.camel@localhost.localdomain> <1138381908.3090.259.camel@localhost.localdomain> Message-ID: <43DAABD8.5040404@v.loewis.de> Thomas Heller wrote: > Can anyone of the python-dev core team comment: can we live with the GPL > licensed aclocal.m4 file, in the source distribution and in SVN? My understanding that doing so would be in violation of section 2b) of the GPL. However, I still think it is possible to include libffi - we just have to discard the build process, and do our own. Regards, Martin From thomas at xs4all.net Sat Jan 28 00:34:03 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Sat, 28 Jan 2006 00:34:03 +0100 Subject: [Python-Dev] / as path join operator In-Reply-To: References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> <87k6cnaabt.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <20060127233403.GM18916@xs4all.nl> On Fri, Jan 27, 2006 at 06:19:52PM -0500, Jason Orendorff wrote: > To say "paths aren't strings" is all very well, and in a very abstract > sense I almost agree--but you have to admit it sort of flies in the face > of, you know, reality. Filesystem paths are in fact strings on all > operating systems I'm aware of. And it's no accident or performance > optimization. It's good design. The question isn't whether Path objects should *act* like strings. I haven't seen anyone argue that they shouldn't, except for a few specific aspects, like iteration, and those are argued on both sides of the subclassing camp. The question is whether they should be actual subclasses of the Python string type. As for what platforms do, if we want to stick to the platform handling of paths, we change nothing. That's apparently not what people want ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From martin at v.loewis.de Sat Jan 28 00:41:59 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 28 Jan 2006 00:41:59 +0100 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: <074501c62373$df526cd0$bf03030a@trilan> References: <1137071518.12975.6.camel@devel002><1137186878.3098.87.camel@localhost.localdomain><1138378937.3090.255.camel@localhost.localdomain><1138381908.3090.259.camel@localhost.localdomain> <074501c62373$df526cd0$bf03030a@trilan> Message-ID: <43DAAFC7.2060603@v.loewis.de> Giovanni Bajo wrote: > That's no different. If you burn a CD containing a copy of the GCC and a > copy of a commercial software you are not violating any license. If you > distribute an .ISO file containing a copy of the GCC and a copy of a > commercial software, you are not violating any license. If you distribute a > .zip file containing a copy of GCC and a copy of a commercial software, you > are not violating any license. You misunderstand the GPL. Section 2b) is pretty clear that any application that contains GPL-licensed code must be, itself, distributed under the terms ofthe GPL b) You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License. It further elaborates (in 2, below c) that you could distribute "identifiable sections of [your] work [which] are not derived from the Program" separately, as long as you omit the Program. But if you distribute them as a whole, the whole must be licensed under the GPL. > There is an important difference between aggreggation of different programs, > and static/dynamic linking of parts. There is an exception for "mere aggregation": # In addition, mere aggregation of another work not based on the Program # with the Program (or with a work based on the Program) on a volume of # a storage or distribution medium does not bring the other work under # the scope of this License. However, inclusion of aclocal.m4 isn't "mere aggregation [...] on a volume of storage". aclocal.m4 is an essential part of the software as a whole, so 2b) applies. > Also, do not understimate previous history. There are zillions of programs > out there using Autconf and *not* being GPL. Yes, and SCO is (rightly, IMO) complaining that zillions of open source people do not honor IP properly. That doesn't mean Python should follow. Instead, it means we need a build process for libffi which is independent of autoconf (or convince the authors of aclocal.m4 to relicense it, but that is likely futile). As a matter of fact, Python itself uses autoconf, but without aclocal.m4. autoconf generates configure for us, but configure is not GPL-licensed, even though it is generated through autoconf: # Copyright (C) 2003 Free Software Foundation, Inc. # This configure script is free software; the Free Software Foundation # gives unlimited permission to copy, distribute and modify it. Regards, Martin From martin at v.loewis.de Sat Jan 28 00:50:16 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 28 Jan 2006 00:50:16 +0100 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 In-Reply-To: <871wyt75vn.fsf@tleepslib.sk.tsukuba.ac.jp> References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <20060126090822.GF18916@xs4all.nl> <99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com> <871wyt75vn.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <43DAB1B8.9060200@v.loewis.de> Stephen J. Turnbull wrote: > Otherwise, not using the distributed aclocal.m4 may be possible, but > it's a bad idea. That may not be so bad, actually. It looks like libffi's aclocal.m4 is not hand-written, but generated through aclocal(1). Not sure why this is done, but this seems to be the cause of the explicit mentionings of the GPL (the snippets that aclocal.m4 is generated from are each GPL-licensed, make the whole aclocal.m4 GPL-licensed). That said, I'm uncertain as to what the purpose of aclocal(1) is, so I might be wrong. Regards, Martin From martin at v.loewis.de Sat Jan 28 00:53:39 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 28 Jan 2006 00:53:39 +0100 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 In-Reply-To: <99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com> References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <20060126090822.GF18916@xs4all.nl> <99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com> Message-ID: <43DAB283.70207@v.loewis.de> Ronald Oussoren wrote: > Merging the two configure files might be a good idea anyway, that would > take away the need to run configure from setup.py. IANAL, but I don't > quite get how a GPL'd support script, if there is such a thing, in the > build machinery of an extension library would require that Python > itself is GPL'd. Section 2b) of the GPL. If a part of the program is GPL, the entire program must be. Also, you must include the entire source of the program, including all build scripts (section 3). So just including the generated configure, and omitting some of its input, would also be a license violation. Regards, Martin From rasky at develer.com Sat Jan 28 01:01:57 2006 From: rasky at develer.com (Giovanni Bajo) Date: Sat, 28 Jan 2006 01:01:57 +0100 Subject: [Python-Dev] (libffi) Re: Copyright issue References: <1137071518.12975.6.camel@devel002><1137186878.3098.87.camel@localhost.localdomain><1138378937.3090.255.camel@localhost.localdomain><1138381908.3090.259.camel@localhost.localdomain> Message-ID: <0b7a01c6239e$0e464910$12b52997@bagio> Thomas Heller wrote: > > Andrew Pinski writes: > >> Does phython already use autoconf? I think it does, if so then there >> should be no issues. > > [Anthony Green] >>>> I guess I wasn't clear. aclocal.m4 is just a tool used to build >>>> libffi. Like your C compiler. Bundling it with the Python source >>>> distribution should have no impact on the licensing of Python >>>> itself, since it isn't really part of the resulting Python binary - >>>> just like your C compiler isn't. > > [Thomas Heller] >>> I guess I understood this already. The difference to the C >>> compiler is >>> that the compiler is not 'bundled' with Python, it is installed >>> separately. >> > > "Giovanni Bajo" writes: >> That's no different. If you burn a CD containing a copy of the GCC >> and a >> copy of a commercial software you are not violating any license. If >> you >> distribute an .ISO file containing a copy of the GCC and a copy of a >> commercial software, you are not violating any license. If you >> distribute a .zip file containing a copy of GCC and a copy of a >> commercial software, you >> are not violating any license. >> >> There is an important difference between aggreggation of different >> programs, >> and static/dynamic linking of parts. Autoconf is a build tool, and >> it does >> not impose any license on the software you use it with. Plus some >> files have >> the special exception from GPL so that even the files *generated* by >> autoconf (and partly made of pieces of autoconf) do not require to >> be GPL. >> This is exactly like GCC's runtime library (libgcc, and also >> libstdc++) >> which are GPL with the special exception, and you can use them also >> for >> commercial products. >> >> Also, do not understimate previous history. There are zillions of >> programs >> out there using Autconf and *not* being GPL. > > Ok, understood - there is no problem. Hopefully the rest of the > pythondev team is also convinced. > > The only question I have (maybe this is too off-topic, but since we're > here already): Nearly all the tools that autoconf and automake use are > careful to include an exception clause to the GPL license. Why is > aclocal.m4 different? Is aclocal.m4 even GPL in the first place? I don't see such a notice in my libffi copy. Giovanni Bajo From rasky at develer.com Sat Jan 28 00:56:03 2006 From: rasky at develer.com (Giovanni Bajo) Date: Sat, 28 Jan 2006 00:56:03 +0100 Subject: [Python-Dev] (libffi) Re: Copyright issue References: <1137071518.12975.6.camel@devel002> <1137186878.3098.87.camel@localhost.localdomain> <1138378937.3090.255.camel@localhost.localdomain> <1138381908.3090.259.camel@localhost.localdomain> <43DAABD8.5040404@v.loewis.de> Message-ID: <0b6d01c6239d$3b92c570$12b52997@bagio> Martin v. L?wis wrote: >> Can anyone of the python-dev core team comment: can we live with the >> GPL >> licensed aclocal.m4 file, in the source distribution and in SVN? > > My understanding that doing so would be in violation of section 2b) of > the GPL. This would be a new interpretation of the license. The whole autotools chain is GPL and it is used on way too many programs which are not GPL. They're so many I won't even mention one. Anyway, IANAL, so if you're really concerned you can mail the FSF and ask clarifications. BTW, by your reading, libffi itself would be a GPL violation. Giovanni Bajo From martin at v.loewis.de Sat Jan 28 01:06:11 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 28 Jan 2006 01:06:11 +0100 Subject: [Python-Dev] SourceForge Download Page, Subversion Home Page In-Reply-To: <1f7befae0601270749w1dbb25f5lb785e7294c81fcff@mail.gmail.com> References: <43D9F90E.1040703@holdenweb.com> <1f7befae0601270749w1dbb25f5lb785e7294c81fcff@mail.gmail.com> Message-ID: <43DAB573.3040209@v.loewis.de> Tim Peters wrote: > Overall I'm not sure that's an improvement, but it was the best I > could come up with 2 years ago when ZODB stopped using SF for > downloads. Please take a look at my attempt to solve that: putting a text just above the button. Regards, Martin From martin at v.loewis.de Sat Jan 28 01:18:03 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 28 Jan 2006 01:18:03 +0100 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: <0b6d01c6239d$3b92c570$12b52997@bagio> References: <1137071518.12975.6.camel@devel002> <1137186878.3098.87.camel@localhost.localdomain> <1138378937.3090.255.camel@localhost.localdomain> <1138381908.3090.259.camel@localhost.localdomain> <43DAABD8.5040404@v.loewis.de> <0b6d01c6239d$3b92c570$12b52997@bagio> Message-ID: <43DAB83B.2040808@v.loewis.de> Giovanni Bajo wrote: > This would be a new interpretation of the license. The whole autotools chain is > GPL and it is used on way too many programs which are not GPL. They're so many > I won't even mention one. Anyway, IANAL, so if you're really concerned you can > mail the FSF and ask clarifications. No. As Tom Tromey explains, significant parts of autoconf have a special exception, allowing parts of autoconf to be used without any restrictions when they get copied into, say, configure. If you only use those parts of autoconf, you don't violate the GPL. However, it appears that (at least parts of) aclocal.m4 originate from code which doesn't have such a liberal license. BTW, this interpretation isn't new at all. I first heard it in 1992. People tend to reinterpret it because they can't believe the original words, and then those interpretations become urban myth. > BTW, by your reading, libffi itself would be a GPL violation. Well, no. I interpret it to be dual-licensed: Identifiable parts of it are independent work, and can be licensed independently of "the Program (i.e. autoconf)". They meet all requirements of the GPL (providing all code they need to provide) - it's just difficult to find out what the license is. Regards, Martin From bjourne at gmail.com Sat Jan 28 01:29:39 2006 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Sat, 28 Jan 2006 01:29:39 +0100 Subject: [Python-Dev] Path inherits from string In-Reply-To: <43D8FE24.1030500@egenix.com> References: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com> <43D8FE24.1030500@egenix.com> Message-ID: <740c3aec0601271629k238e90a4jed3fc189d75bea24@mail.gmail.com> [M.-A. Lemburg] > I don't see why this is critical for the success of the Path > object. I agree with Thomas that interfaces should be made > compatible to Path object. See the steps I mentioned. Unless step #1 is completed there is no way to make the following code work: open(Path("foobar")) Well, there is one alternative which is: open(Path("foobar").tostring()) And that is a Java-esque workaraound that I think noone would be happy with. > Why not ? We've added Unicode support to at least some > file I/O APIs - adding support for instances which > support the string interface shouldn't be all that > difficult :-) > BTW, if you're fine with this API: > class File: > def __unicode__(self): > return u"test.txt" > then the required change is minimal: we'd just need to > use PyObject_Unicode() in getargs.c:837 and you should > be set. Alright! If that is true then maybe step #1 isn't as hard as I thought. First problem is the "string interface." What exactly is the string interface? It needs to be specified. I would have guessed that it was equal to the stuff in the builtin str class.. :) Then you need to modify Python's C code so that it accepts all objects implementing the string interface. That may not be a trivial task [1]. Also don't forget that Path probably also should work with: isinstance(Path('.'), basestring) Because one of the big selling points of Path is that it acts as a drop-in replacement for strings [2]. And you don't want to mess with the current canonical way of testing if an object implements the "string interface." But you aren't supposed to subclass basestring [3]. All I'm saying is that I don't have the technical knowledge required to modify Python to make Path work without subclassing string. If you, or someone else, do then pretty please with sugar on top make a patch so that Path doesn't have to subclass string. Then you could have a much more pure Path without the "dead" methods. But even then, one could argue that Jason Orendorffs original Path module is still more practical (atleast for compatibility reasons). def save_data(somedir, fpath, data): open(somedir + "/" + fpath.lower(), "w").write(data) The above code isn't going to work unless fpath is a string (or an object that inherits from string). It isn't good code, but it isn't extremely uncommon either. 1 http://mail.python.org/pipermail/python-dev/2006-January/059780.html 2 http://mail.python.org/pipermail/python-list/2005-July/291213.html 3 http://mail.python.org/pipermail/python-dev/2006-January/059782.html -- mvh Bj?rn From t-meyer at ihug.co.nz Sat Jan 28 01:40:39 2006 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Sat, 28 Jan 2006 13:40:39 +1300 Subject: [Python-Dev] / as path join operator In-Reply-To: References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> <87k6cnaabt.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: [Jason Orendorff] > Filesystem paths are in fact strings on all operating systems I'm > aware of. And it's no accident or performance optimization. It's > good design. Isn't that simply because filesystems aren't object orientated? I can't call methods of a path through the filesystem. There's a difference between a path, which is, yes, always (?) a string, and a Path object that provides convenient methods/properties. (Maybe one of the experimental object-orientated file systems has non- string paths. I have no idea). =Tony.Meyer From ncoghlan at gmail.com Sat Jan 28 03:39:02 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 28 Jan 2006 12:39:02 +1000 Subject: [Python-Dev] SourceForge Download Page, Subversion Home Page In-Reply-To: <43DAB573.3040209@v.loewis.de> References: <43D9F90E.1040703@holdenweb.com> <1f7befae0601270749w1dbb25f5lb785e7294c81fcff@mail.gmail.com> <43DAB573.3040209@v.loewis.de> Message-ID: <43DAD946.3060107@gmail.com> Martin v. L?wis wrote: > Tim Peters wrote: >> Overall I'm not sure that's an improvement, but it was the best I >> could come up with 2 years ago when ZODB stopped using SF for >> downloads. > > Please take a look at my attempt to solve that: putting a text just > above the button. Is it possible to make that URL a hyperlink? It's also worth considering ways to make the alternate URL stand out more, as the green button is dominant enough that I'd expect a lot of people to miss the preceding text. Something like: " The Python programming language, an object-oriented scripting and rapid application development language. Despite what the green button below says, you can NOT download it directly from Sourceforge, as SF is used only for bug tracking. You can get releases from the main Python website: Download from python.org " Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From nas at arctrix.com Sat Jan 28 03:52:26 2006 From: nas at arctrix.com (Neil Schemenauer) Date: Sat, 28 Jan 2006 02:52:26 +0000 (UTC) Subject: [Python-Dev] Path inherits from string References: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com> <43D90014.10807@colorstudy.com> Message-ID: Ian Bicking wrote: > OTOH, str(path) will break unicode filenames. And unicode() > breaks anything that simply desires to pass data through without > effecting its encoding. That general problem was the motivation for PEP 349. Originally I suggested adding a new built-in. However, Guido's suggestion of allowing str() to return unicode objects works okay too and that's the way the current PEP is written. With Python 2.5 you can do it the ugly way using a str format (e.g. '%s' % obj). Neil From rhamph at gmail.com Sat Jan 28 06:49:50 2006 From: rhamph at gmail.com (Adam Olsen) Date: Fri, 27 Jan 2006 22:49:50 -0700 Subject: [Python-Dev] / as path join operator In-Reply-To: References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> <87k6cnaabt.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: On 1/27/06, Jason Orendorff wrote: > On 1/25/06, Stephen J. Turnbull wrote: > > I think it's logical to expect that > > Path('home') / 'and/or' > > points to a file named "and/or" in directory "home", not to a file > > named "or" in directory "home/and". > > This makes no sense whatsoever. Ergo, by reductio ad absurdum, paths > are strings. It makes perfect sense to me. However, since posix doesn't permit '/' in file names I would expect it to emit an error: Errors should never pass silently. Unless explicitly silenced. In the face of ambiguity, refuse the temptation to guess. However, I'm not sure if the error be emitted when the Path is created, or when it's passed to open(). The former implies a set of OS-specific Path classes, and would allow subclassing from str. The latter allows (but does not require) a single universal Path class, and that would prohibit subclassing from str (because you need a higher-level representation to store path segments before converting them to a platform-specific format.) I'm -0 on subclassing str in the shortterm and -1 on it in the longterm. It's cruft and not something we want to be stuck with. -- Adam Olsen, aka Rhamphoryncus From stephen at xemacs.org Sat Jan 28 07:56:27 2006 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 28 Jan 2006 15:56:27 +0900 Subject: [Python-Dev] / as path join operator In-Reply-To: (Jason Orendorff's message of "Fri, 27 Jan 2006 18:19:52 -0500") References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> <87k6cnaabt.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <87vew44z8k.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Jason" == Jason Orendorff writes: Jason> I. Here's an example of the sort of thing you might say if Jason> you did *not* think of paths as strings: [...] Jason> II. And here is the sort of thing you'd say if you thought Jason> of paths *solely* as strings: Please note that my point was entirely different from trying to decide whether to subclass strings. My point was precisely that because of this schizophrenia in the use of / as a path join operator in various string representations of paths, it's a bad choice. People are naturally going to write buggy code because they don't have the implemented spec in mind. Jason> Filesystem paths are in fact strings on all operating Jason> systems I'm aware of. I have no idea what you could mean by that. The data structure used to represent a filesystem on all OS filesystems I've used is a graph of directories and files. A filesystem object is located by traversing a path in that graph. Of course there's a string representation, especially for human use, but manipulating that representation as a string in programs is a regular source of bugs. In most cases, the graph is sufficiently constrained that string manipulations are mostly accurate representations of graph traversal, but not always, and you get defects. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From martin at v.loewis.de Sat Jan 28 10:04:01 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 28 Jan 2006 10:04:01 +0100 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: <4509FB68-6882-409E-9CFF-D5ADDA6ED210@internode.on.net> References: <1137071518.12975.6.camel@devel002><1137186878.3098.87.camel@localhost.localdomain><1138378937.3090.255.camel@localhost.localdomain><1138381908.3090.259.camel@localhost.localdomain> <074501c62373$df526cd0$bf03030a@trilan> <43DAAFC7.2060603@v.loewis.de> <4509FB68-6882-409E-9CFF-D5ADDA6ED210@internode.on.net> Message-ID: <43DB3381.7040002@v.loewis.de> Bill Northcott wrote: > Quite so, but using the autotools does NOT include any GPL code in the > resulting program. Hmm. Please take a look at http://cvs.sourceforge.net/viewcvs.py/*checkout*/ctypes/ctypes/source/gcc/libffi/aclocal.m4?rev=1.1.4.1 This file contains a large number of licensing text blocks, many of which read # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2, or (at your option) # any later version. So it seems to me that this specific generated aclocal.m4 *does* include GPL code. > So this does not apply. All that is needed is to > include in the source distribution a copy of GPL, note that GPL applies > to some files in the sources and ensure that copyright notices at the > heads of GPL files are intact. If nothing in the generated files is licensed under the terms of the GPL, why would it be necessary to include a copy of the GPL? > The compiler needs specific exemptions because parts of the GPLed > runtime libraries are included in all compiled code. No part of the > autotools ends up in the finished code. If it did, you would need m4 > to run Python and you don't. It doesn't matter whether it ends up in the finished code: if the aclocal.m4 is indeed GPL-licensed, then the entire Python source distribution must be GPL-licensed, because it "contains or is derived from the Program or any part thereof". Regards, Martin From martin at v.loewis.de Sat Jan 28 11:05:12 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 28 Jan 2006 11:05:12 +0100 Subject: [Python-Dev] SourceForge Download Page, Subversion Home Page In-Reply-To: <43DAD946.3060107@gmail.com> References: <43D9F90E.1040703@holdenweb.com> <1f7befae0601270749w1dbb25f5lb785e7294c81fcff@mail.gmail.com> <43DAB573.3040209@v.loewis.de> <43DAD946.3060107@gmail.com> Message-ID: <43DB41D8.4090300@v.loewis.de> Nick Coghlan wrote: > Is it possible to make that URL a hyperlink? No, all HTML gets stripped/quoted as text. > " The Python programming language, an object-oriented scripting and rapid > application development language. > Despite what the green button below says, you can NOT download it > directly from Sourceforge, as SF is used only for bug tracking. You can get > releases from the main Python website: Sorry, no HTML allowed there. I created a support request to change/remove the button at http://sourceforge.net/tracker/index.php?func=detail&aid=1417298&group_id=1&atid=200001 Regards, Martin From fredrik at pythonware.com Sat Jan 28 11:07:30 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sat, 28 Jan 2006 11:07:30 +0100 Subject: [Python-Dev] / as path join operator References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com><43D6BA85.8070007@colorstudy.com><740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com><63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz><87k6cnaabt.fsf@tleepslib.sk.tsukuba.ac.jp> <87vew44z8k.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: Stephen J. Turnbull wrote: > Jason> Filesystem paths are in fact strings on all operating > Jason> systems I'm aware of. > > I have no idea what you could mean by that. The data structure used > to represent a filesystem on all OS filesystems I've used is a graph > of directories and files. A filesystem object is located by > traversing a path in that graph. > > Of course there's a string representation, especially for human use if you define everything that can be identified by one or more well-defined strings as a string, everything is a string. From ronaldoussoren at mac.com Sat Jan 28 11:06:01 2006 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Sat, 28 Jan 2006 11:06:01 +0100 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 In-Reply-To: <43DAB283.70207@v.loewis.de> References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <20060126090822.GF18916@xs4all.nl> <99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com> <43DAB283.70207@v.loewis.de> Message-ID: <90733CAD-8AB7-4749-9BF8-2386B12F0FAB@mac.com> On 28-jan-2006, at 0:53, Martin v. L?wis wrote: > Ronald Oussoren wrote: >> Merging the two configure files might be a good idea anyway, that >> would >> take away the need to run configure from setup.py. IANAL, but I >> don't >> quite get how a GPL'd support script, if there is such a thing, >> in the >> build machinery of an extension library would require that Python >> itself is GPL'd. > > Section 2b) of the GPL. If a part of the program is GPL, the entire > program must be. Also, you must include the entire source of the > program, including all build scripts (section 3). So just including > the > generated configure, and omitting some of its input, would also be a > license violation. You have a point there. I'm not entirely convinced though, the argument that Python would be a derived work of libffi's aclocal.m4 when libffi were included in the Python repository seems very weak to me. It is a good argument to just drop libffi's configure script and integrate the functionality of it in the main python configure script. That would remove all possible doubt and shouldn't be too much work. BTW. The argument that the readline module should be GPL licensed seems rather stronger, it's designed to work with a GPL-ed library and doesn't work with a BSD licensed work-alike of that library. Ronald > > Regards, > Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2157 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20060128/c1a198bd/attachment-0001.bin From martin at v.loewis.de Sat Jan 28 11:10:54 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 28 Jan 2006 11:10:54 +0100 Subject: [Python-Dev] stabilizing builds In-Reply-To: <20060126202456.GJ18916@xs4all.nl> References: <20060124111528.GB18916@xs4all.nl> <43D6AFC4.3030200@v.loewis.de> <20060125005918.GC18916@xs4all.nl> <20060126202456.GJ18916@xs4all.nl> Message-ID: <43DB432E.20201@v.loewis.de> Thomas Wouters wrote: > I'd need developer access back to check it in, though. Unless anyone > objects, of course :) I copied ~/thomas/authorized_keys to ~pythondev/keys/thomas.wouters, changed ownership/permissions, and ran make_authorized_keys in the pythondev account. So you should have access now. Regards, Martin From fredrik at pythonware.com Sat Jan 28 11:36:22 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sat, 28 Jan 2006 11:36:22 +0100 Subject: [Python-Dev] SourceForge Download Page, Subversion Home Page References: <43D9F90E.1040703@holdenweb.com> <1f7befae0601270749w1dbb25f5lb785e7294c81fcff@mail.gmail.com> <43DAB573.3040209@v.loewis.de><43DAD946.3060107@gmail.com> <43DB41D8.4090300@v.loewis.de> Message-ID: Martin v. Löwis wrote: > Sorry, no HTML allowed there. > > I created a support request to change/remove the button at > > http://sourceforge.net/tracker/index.php?func=detail&aid=1417298&group_id=1&atid=200001 instead of spending more time and creativity on a sourceforge account that's only used for tracking, how about moving the trackers to python.org ? there are at least two "powered-by-python" alternatives that are vastly better than source-forge's tracker: roundup, which just went 1.0: http://cheeseshop.python.org/pypi/roundup/1.0 and trac, http://www.edgewall.com/trac/ which almost everyone is using these days... From martin at v.loewis.de Sat Jan 28 11:42:02 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 28 Jan 2006 11:42:02 +0100 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 In-Reply-To: <90733CAD-8AB7-4749-9BF8-2386B12F0FAB@mac.com> References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <20060126090822.GF18916@xs4all.nl> <99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com> <43DAB283.70207@v.loewis.de> <90733CAD-8AB7-4749-9BF8-2386B12F0FAB@mac.com> Message-ID: <43DB4A7A.8070606@v.loewis.de> Ronald Oussoren wrote: > You have a point there. I'm not entirely convinced though, the argument > that Python would be a derived work of libffi's aclocal.m4 when libffi > were included in the Python repository seems very weak to me. The GPL says "contains or is derived from". Clearly, "identifiable parts" of Python are not derived from aclocal.m4. However, the work as a whole (i.e. the entire Python distribution) then becomes derived work of libffi, and aclocal.m4. > It is a good argument to just drop libffi's configure script and integrate > the functionality of it in the main python configure script. That would > remove all possible doubt and shouldn't be too much work. Several autoconf people have argued that this aclocal.m4 is a mistake, and that the intention was that it shouldn't be GPL-licensed. So if we cannot find a volunteer to rewrite the build process of libffi for use in Python, that would also be a strategy (but one that probably takes many months to complete). I would personally drop the use of automake (and just rely on autoconf), and then the need for aclocal would go away. > BTW. The argument that the readline module should be GPL licensed seems > rather stronger, it's designed to work with a GPL-ed library and doesn't > work with a BSD licensed work-alike of that library. This is the question what constitutes derivative work, and different lawyers have said different things in the past. If we really want to find out, we should ask a lawyer. IANAL, and my understanding is that a) we don't include readline in the Python distribution, so the Python source code cannot be derivative work. In U.S. copyright law, the term is apparently defined as "any . . . work [that] may be recast, transformed, or adapted. A work consisting of editorial revisions, annotations, elaborations, or other modifications which, as a whole, represents an original work of authorship, is a 'derivative work.'" http://www.chillingeffects.org/derivative/ I would argue that Modules/readline.c does *not* represent the original work of authorship (i.e. libreadline). Of course, the GPL says "derived", not "derivative", and doesn't define the term, so you should ask your lawyer; ultimately, a court might decide what it means. b) if we were to distribute binaries of Python, this issue would yet again be different: it seems that a binary readline module (builtin or not) is derivative work of the readline library, whether it is dynamically linked with that library or not. So anybody distributing such a binary might have to distribute it under the terms of the GPL. I say "might" because there is an exception in the GPL for major components normally distributed with the operating system. On those systems where Python's readline module is available, the readline library could be considered a part of the operating system. To be sure, ask your lawyer; ultimately, a court might decide whether this clause is relevant here. Regards, Martin From mwh at python.net Sat Jan 28 12:54:55 2006 From: mwh at python.net (Michael Hudson) Date: Sat, 28 Jan 2006 11:54:55 +0000 Subject: [Python-Dev] Path inherits from string In-Reply-To: <740c3aec0601271629k238e90a4jed3fc189d75bea24@mail.gmail.com> ( =?iso-8859-1?q?BJ=F6rn_Lindqvist's_message_of?= "Sat, 28 Jan 2006 01:29:39 +0100") References: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com> <43D8FE24.1030500@egenix.com> <740c3aec0601271629k238e90a4jed3fc189d75bea24@mail.gmail.com> Message-ID: <2mfyn8in3k.fsf@starship.python.net> BJ?rn Lindqvist writes: > [M.-A. Lemburg] >> I don't see why this is critical for the success of the Path >> object. I agree with Thomas that interfaces should be made >> compatible to Path object. > > See the steps I mentioned. Unless step #1 is completed there is no way > to make the following code work: > > open(Path("foobar")) > > Well, there is one alternative which is: > > open(Path("foobar").tostring()) > > And that is a Java-esque workaraound that I think noone would be happy > with. Now maybe I'm missing context here but: what on earth are you talking about? Of course there's a way to make "open(Path("foobar"))" work -- you change how the builtin open() works. This post is not intended as arguing for or against anything, except technical accuracy. Cheers, mwh -- Richard Gabriel was wrong: worse is not better, lying is better. Languages and systems succeed in the marketplace to the extent that their proponents lie about what they can do. -- Tim Bradshaw, comp.lang.lisp From green at redhat.com Fri Jan 27 18:11:48 2006 From: green at redhat.com (Anthony Green) Date: Fri, 27 Jan 2006 09:11:48 -0800 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: References: <1137071518.12975.6.camel@devel002> <1137186878.3098.87.camel@localhost.localdomain> <1138378937.3090.255.camel@localhost.localdomain> Message-ID: <1138381908.3090.259.camel@localhost.localdomain> On Fri, 2006-01-27 at 18:03 +0100, Thomas Heller wrote: > [I've added python-dev to cc:] > > Anthony Green writes: > > > On Fri, 2006-01-27 at 17:08 +0100, Thomas Heller wrote: > >> Anyway, another question is: Is aclocal.m4 needed at all for building > >> (or maybe for regenerating the configure scripts), or is it optional? > > > > aclocal.m4 is required, but is only used as a build-time tool. The fact > > that aclocal.m4 is distributed under the GPL should have no impact on > > the licensing terms used for software built using aclocal.m4. > > If I understand correctly this means that the Python source distribution > would have to be GPL licensed, while the built programs would be able to > use another license. > > I'm pretty sure this kills the whole idea (to include libffi in python). I guess I wasn't clear. aclocal.m4 is just a tool used to build libffi. Like your C compiler. Bundling it with the Python source distribution should have no impact on the licensing of Python itself, since it isn't really part of the resulting Python binary - just like your C compiler isn't. AG From pinskia at physics.uc.edu Fri Jan 27 19:37:48 2006 From: pinskia at physics.uc.edu (Andrew Pinski) Date: Fri, 27 Jan 2006 13:37:48 -0500 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: References: <1137071518.12975.6.camel@devel002> <1137186878.3098.87.camel@localhost.localdomain> <1138378937.3090.255.camel@localhost.localdomain> <1138381908.3090.259.camel@localhost.localdomain> Message-ID: On Jan 27, 2006, at 1:32 PM, Thomas Heller wrote: > > I guess I understood this already. The difference to the C > compiler is > that the compiler is not 'bundled' with Python, it is installed > separately. > > Can anyone of the python-dev core team comment: can we live with > the GPL > licensed aclocal.m4 file, in the source distribution and in SVN? Does phython already use autoconf? I think it does, if so then there should be no issues. -- Pinski From tromey at redhat.com Sat Jan 28 01:04:25 2006 From: tromey at redhat.com (Tom Tromey) Date: 27 Jan 2006 17:04:25 -0700 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: <0b6d01c6239d$3b92c570$12b52997@bagio> References: <1137071518.12975.6.camel@devel002> <1137186878.3098.87.camel@localhost.localdomain> <1138378937.3090.255.camel@localhost.localdomain> <1138381908.3090.259.camel@localhost.localdomain> <43DAABD8.5040404@v.loewis.de> <0b6d01c6239d$3b92c570$12b52997@bagio> Message-ID: >>>>> "Giovanni" == Giovanni Bajo writes: Giovanni> This would be a new interpretation of the license. The whole Giovanni> autotools chain is GPL and it is used on way too many Giovanni> programs which are not GPL. They're so many I won't even Giovanni> mention one. Anyway, IANAL, so if you're really concerned Giovanni> you can mail the FSF and ask clarifications. No, Martin is correct about this. The output of autoconf is explicitly *not* under the GPL, by design. This is also true for the m4 macros from automake -- again, an explicit decision. The problem is, some other projects have not been so careful. Tom From tromey at redhat.com Sat Jan 28 00:50:53 2006 From: tromey at redhat.com (Tom Tromey) Date: 27 Jan 2006 16:50:53 -0700 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: <43DAAFC7.2060603@v.loewis.de> References: <1137071518.12975.6.camel@devel002> <1137186878.3098.87.camel@localhost.localdomain> <1138378937.3090.255.camel@localhost.localdomain> <1138381908.3090.259.camel@localhost.localdomain> <074501c62373$df526cd0$bf03030a@trilan> <43DAAFC7.2060603@v.loewis.de> Message-ID: >>>>> "Martin" == Martin v L?wis writes: Martin> Instead, it means we need a build process for libffi which is Martin> independent of autoconf (or convince the authors of aclocal.m4 to Martin> relicense it, but that is likely futile). Martin> As a matter of fact, Python itself uses autoconf, but without Martin> aclocal.m4. autoconf generates configure for us, but configure is Martin> not GPL-licensed, even though it is generated through autoconf: Martin> # Copyright (C) 2003 Free Software Foundation, Inc. Martin> # This configure script is free software; the Free Software Foundation Martin> # gives unlimited permission to copy, distribute and modify it. libffi's aclocal.m4 is created by the aclocal tool, which stitches it together from different .m4 files it finds. For m4 files that are part of the automake distribution, the intent was explicitly to have the same more liberal permissions that apply to 'configure'. If you can't find a statement saying this somewhere, then I believe that to be a bug (I see one at the top of aclocal.m4 FWIW -- this applies to all the automake-shipped m4 files). I know I explicitly brought this up with RMS at some point in the distant past and got the needed permissions to make this so. However, libffi probably also uses macros from other places -- at least GCC and libtool. I don't know the legal status of these. The GCC macros are probably not really necessary for your situation. The libffi configury in the GCC tree is set up to build libffi as a target library. Most likely, you don't need this. libffi/acinclude.m4 needs some license clarification. It isn't clear who owns this code :-( I think a real fix is probably not out of reach, should you wish to pursue it. Tom From w.northcott at internode.on.net Sat Jan 28 02:04:50 2006 From: w.northcott at internode.on.net (Bill Northcott) Date: Sat, 28 Jan 2006 12:04:50 +1100 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: <43DAAFC7.2060603@v.loewis.de> References: <1137071518.12975.6.camel@devel002><1137186878.3098.87.camel@localhost.localdomain><1138378937.3090.255.camel@localhost.localdomain><1138381908.3090.259.camel@localhost.localdomain> <074501c62373$df526cd0$bf03030a@trilan> <43DAAFC7.2060603@v.loewis.de> Message-ID: <4509FB68-6882-409E-9CFF-D5ADDA6ED210@internode.on.net> On 28/01/2006, at 10:41 AM, Martin v. L?wis wrote: > You misunderstand the GPL. Section 2b) is pretty clear that any > application that contains GPL-licensed code must be, itself, > distributed > under the terms ofthe GPL Quite so, but using the autotools does NOT include any GPL code in the resulting program. So this does not apply. All that is needed is to include in the source distribution a copy of GPL, note that GPL applies to some files in the sources and ensure that copyright notices at the heads of GPL files are intact. The compiler needs specific exemptions because parts of the GPLed runtime libraries are included in all compiled code. No part of the autotools ends up in the finished code. If it did, you would need m4 to run Python and you don't. Cheers Bill Northcott From martin at v.loewis.de Sat Jan 28 19:46:40 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 28 Jan 2006 19:46:40 +0100 Subject: [Python-Dev] SourceForge Download Page, Subversion Home Page In-Reply-To: References: <43D9F90E.1040703@holdenweb.com> <1f7befae0601270749w1dbb25f5lb785e7294c81fcff@mail.gmail.com> <43DAB573.3040209@v.loewis.de><43DAD946.3060107@gmail.com> <43DB41D8.4090300@v.loewis.de> Message-ID: <43DBBC10.6000601@v.loewis.de> Fredrik Lundh wrote: > instead of spending more time and creativity on a sourceforge account > that's only used for tracking, how about moving the trackers to python.org ? That's an old plan. It failed so far because no volunteer ever appeared to make it happen (actually, some did appear, but didn't actually make it happen). Notice that any such moving would have to carry over the existing tracker items; any volunteer should be available for hand-holding after the move occurs. Groups of people would be better than individual volunteers. > there are at least two "powered-by-python" alternatives that are vastly > better than source-forge's tracker: roundup, which just went 1.0: > > http://cheeseshop.python.org/pypi/roundup/1.0 Guido's preference is on roundup. > and trac, > > http://www.edgewall.com/trac/ > > which almost everyone is using these days... Barry Warsaw favours Jira as a tracker. Regards, Martin From martin at v.loewis.de Sat Jan 28 19:48:51 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 28 Jan 2006 19:48:51 +0100 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: References: <1137071518.12975.6.camel@devel002> <1137186878.3098.87.camel@localhost.localdomain> <1138378937.3090.255.camel@localhost.localdomain> <1138381908.3090.259.camel@localhost.localdomain> Message-ID: <43DBBC93.7010703@v.loewis.de> Andrew Pinski wrote: > Does phython already use autoconf? I think it does, if so then there > should be no issues. Yes, but your conclusion is wrong. Python uses autoconf, but not aclocal/automake. The generated configure is explicitly not covered by the GPL; the status of the generated aclocal.m4 is unclear (it says it is GPL'ed; it also says it isn't). Regards, Martin From nyamatongwe at gmail.com Sat Jan 28 22:42:41 2006 From: nyamatongwe at gmail.com (Neil Hodgson) Date: Sun, 29 Jan 2006 08:42:41 +1100 Subject: [Python-Dev] / as path join operator In-Reply-To: <87vew44z8k.fsf@tleepslib.sk.tsukuba.ac.jp> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> <87k6cnaabt.fsf@tleepslib.sk.tsukuba.ac.jp> <87vew44z8k.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <50862ebd0601281342r75566d1cke18b4b65554ff713@mail.gmail.com> Stephen J. Turnbull: > Jason> Filesystem paths are in fact strings on all operating > Jason> systems I'm aware of. > > I have no idea what you could mean by that. The data structure used > to represent a filesystem on all OS filesystems I've used is a graph > of directories and files. A filesystem object is located by > traversing a path in that graph. > > Of course there's a string representation, especially for human use, Not always. IIRC very old MacOS used an integer directory ID and a string file name. The directory ID was a cookie that you received from the UI and passed through to the file system and there was little support for manipulating the directory ID. Textualized paths were never supposed to be shown to users. Neil From unknown_kev_cat at hotmail.com Sat Jan 28 22:55:14 2006 From: unknown_kev_cat at hotmail.com (Joe Smith) Date: Sat, 28 Jan 2006 16:55:14 -0500 Subject: [Python-Dev] SourceForge Download Page, Subversion Home Page References: <43D9F90E.1040703@holdenweb.com> Message-ID: "Steve Holden" wrote in message news:43D9F90E.1040703 at holdenweb.com... > You may be aware that Tim Parkin's work on our "next-generation" web > presence has borne fruit in the shape of beta.python.org. While there's > still a lot to be done Tim has given us a great start by creating a > framework that makes it rather easier to manage content. > For the record, I'm just a user of the language. However, I tend to prefer the current site design over the new one. But I also felt the same way when the Mozilla.org site was updated to be more 'friendly' (quite a while ago), and have learned to live with it, so it is not a major problem. > > There's the further issue that the entire HTML body of > http://svn.python.org/ currently reads > > > > It would seem like the logical place to have pointers to Subversion > software (downloads, documentation, operating instructions) together > with an *annotated* summary of http://svn.python.org/projects/ and links > back to the main developer site at the very least. I'm not even sure > where that web content is under SVN control at the moment. Also If there is any sort of webcvs software running, it may be ice to link to it from that page. From bjourne at gmail.com Sun Jan 29 08:13:38 2006 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Sun, 29 Jan 2006 08:13:38 +0100 Subject: [Python-Dev] Path inherits from string In-Reply-To: <2mfyn8in3k.fsf@starship.python.net> References: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com> <43D8FE24.1030500@egenix.com> <740c3aec0601271629k238e90a4jed3fc189d75bea24@mail.gmail.com> <2mfyn8in3k.fsf@starship.python.net> Message-ID: <740c3aec0601282313m3f671c0al142f350cdb586087@mail.gmail.com> > > See the steps I mentioned. Unless step #1 is completed there is no way > > to make the following code work: > > > > open(Path("foobar")) > > > > Well, there is one alternative which is: > > > > open(Path("foobar").tostring()) > > > > And that is a Java-esque workaraound that I think noone would be happy > > with. > > Now maybe I'm missing context here but: what on earth are you talking > about? Of course there's a way to make "open(Path("foobar"))" work -- > you change how the builtin open() works. That's what I said: Someone has to make the required modifications to the Python core. Changing how open() works would AFAIK be a modification to the Python core. open() was just an example and changing only how open() works would not be sufficient I think. __import__(), execfile() and third party extensions implemented in C would also have to be modified so that they treat Path("foobar") equivalent to "foobar." -- mvh Bj?rn From martin at v.loewis.de Sun Jan 29 09:00:13 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 29 Jan 2006 09:00:13 +0100 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: References: <1137071518.12975.6.camel@devel002><1137186878.3098.87.camel@localhost.localdomain><1138378937.3090.255.camel@localhost.localdomain><1138381908.3090.259.camel@localhost.localdomain> <074501c62373$df526cd0$bf03030a@trilan> <43DAAFC7.2060603@v.loewis.de> <4509FB68-6882-409E-9CFF-D5ADDA6ED210@internode.on.net> <43DB3381.7040002@v.loewis.de> Message-ID: <43DC760D.2020308@v.loewis.de> Bill Northcott wrote: > The build tools: m4 scripts, the configure shell script and the > Makefiles all contain GPL code and are under GPL. > > However, none of this ends up in the 'finished program' which is the > executable versions of Python and its associated libraries. The build > tools are just tools not part of the program. The program is not > 'derived' from the build tools. Again: What matters is what ends up in the source distribution, http://www.python.org/ftp/python/2.4/Python-2.4.tgz The source distribution would contain aclocal.m4; it would not contain the autoconf/autoheader tools themselves. The configure script is *NOT* under GPL, it explicitly says # Copyright (C) 2003 Free Software Foundation, Inc. # This configure script is free software; the Free Software Foundation # gives unlimited permission to copy, distribute and modify it. Likewise, the Makefile.in is not under GPL, either: Makefile.in says # Copyright (C) 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, # 2003, 2004 Free Software Foundation, Inc. # This Makefile.in is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. > Of course the computer on which I wrote the book has an operating > system which is copyright. The word processing software I used to do > the writing is also copyright. However none of either program ends up > in my novel. So the novel is not derived from them, and their > copyright holders have no rights over the novel. See, and this is precisely the difference. The word processor doesn't end up in the distributed book. However, aclocal.m4 ends up in the source distribution. > I can happily package appropriately licensed copies of the word > processing software with the finished book. Right: "appropriately licensed". Whether you *actually* can do this depends on the license of the word processor. For example, if the license says "not for resale", you cannot do this - even if you are, yourself, licensed to use the copy you got. So do I have permission to distribute copies of aclocal.m4? Yes, the GPL allows this. Does it put additional obligations in doing so? Yes, it does: b) You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License. Is (a future) Python-2.5.tgz "work that I distribute or publish"? Yes, it is. Does it "in whole or in part contain" "the Program or any part thereof"? Yes, it does. So I must "cause" Python-2.5.tgz "to be licensed as whole at no charge to all third parties under the terms of" the GPL. > A Python binary is no more derived from the autotools than the book is > derived from the word processing software. No, the Python binary isn't. The Python source is. Regards, Martin From martin at v.loewis.de Sun Jan 29 09:04:56 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 29 Jan 2006 09:04:56 +0100 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: References: <1137071518.12975.6.camel@devel002> <1137186878.3098.87.camel@localhost.localdomain> <1138378937.3090.255.camel@localhost.localdomain> <1138381908.3090.259.camel@localhost.localdomain> <43DBBC93.7010703@v.loewis.de> Message-ID: <43DC7728.3040801@v.loewis.de> Bill Northcott wrote: > What makes you think that? I can see no such concession in the > autoconf source distribution. A configure script is built up from lots > of code fragments out of the autoconf and automake M4 files, and would > clearly be covered by GPL. No. As I just said in the other mail: The generated configure contains the explicit permission: # Copyright (C) 2003 Free Software Foundation, Inc. # This configure script is free software; the Free Software Foundation # gives unlimited permission to copy, distribute and modify it. If you look at the autoconf sources, you will find various such blocks (e.g. in lib/autoconf/general.m4): # As a special exception, the Free Software Foundation gives unlimited # permission to copy, distribute and modify the configure scripts that # are the output of Autoconf. You need not follow the terms of the GNU # General Public License when using or distributing such scripts, even # though portions of the text of Autoconf appear in them. The GNU # General Public License (GPL) does govern all other use of the material # that constitutes the Autoconf program. Regards, Martin From mwh at python.net Sun Jan 29 10:11:38 2006 From: mwh at python.net (Michael Hudson) Date: Sun, 29 Jan 2006 09:11:38 +0000 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: <43DC760D.2020308@v.loewis.de> ( =?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Sun, 29 Jan 2006 09:00:13 +0100") References: <1137071518.12975.6.camel@devel002> <1137186878.3098.87.camel@localhost.localdomain> <1138378937.3090.255.camel@localhost.localdomain> <1138381908.3090.259.camel@localhost.localdomain> <074501c62373$df526cd0$bf03030a@trilan> <43DAAFC7.2060603@v.loewis.de> <4509FB68-6882-409E-9CFF-D5ADDA6ED210@internode.on.net> <43DB3381.7040002@v.loewis.de> <43DC760D.2020308@v.loewis.de> Message-ID: <2m3bj7iek5.fsf@starship.python.net> "Martin v. L?wis" writes: > The source distribution would contain aclocal.m4; it would not > contain the autoconf/autoheader tools themselves. To a rather different point, do we need aclocal.m4 at all? This is the log for aclocal.m4: ------------------------------------------------------------------------ r34284 | anthonybaxter | 2003-09-27 11:12:27 +0200 (Sat, 27 Sep 2003) | 5 lines fix for bug #811160 - autoconf vs. hp/ux system header files. also applied to release23-maint. Note that aclocal.m4 can go away when autoconf 2.58 is out. ------------------------------------------------------------------------ I think 2.58 actually had a brown-paper-bag release style bug, but 2.59 has been out for ages now. If we were prepared to AC_PREREQ(2.59), I think this whole issue could go away. Cheers, mwh -- Finding a needle in a haystack is a lot easier if you burn down the haystack and scan the ashes with a metal detector. -- the Silicon Valley Tarot (another one nicked from David Rush) From martin at v.loewis.de Sun Jan 29 10:54:56 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 29 Jan 2006 10:54:56 +0100 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: <2m3bj7iek5.fsf@starship.python.net> References: <1137071518.12975.6.camel@devel002> <1137186878.3098.87.camel@localhost.localdomain> <1138378937.3090.255.camel@localhost.localdomain> <1138381908.3090.259.camel@localhost.localdomain> <074501c62373$df526cd0$bf03030a@trilan> <43DAAFC7.2060603@v.loewis.de> <4509FB68-6882-409E-9CFF-D5ADDA6ED210@internode.on.net> <43DB3381.7040002@v.loewis.de> <43DC760D.2020308@v.loewis.de> <2m3bj7iek5.fsf@starship.python.net> Message-ID: <43DC90F0.8060105@v.loewis.de> Michael Hudson wrote: > I think 2.58 actually had a brown-paper-bag release style bug, but > 2.59 has been out for ages now. If we were prepared to > AC_PREREQ(2.59), I think this whole issue could go away. It seems you are right, so I removed the file, and require ac 2.59. Regards, Martin From hyeshik at gmail.com Sun Jan 29 14:50:59 2006 From: hyeshik at gmail.com (Hye-Shik Chang) Date: Sun, 29 Jan 2006 22:50:59 +0900 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: <43DAABD8.5040404@v.loewis.de> References: <1137071518.12975.6.camel@devel002> <1137186878.3098.87.camel@localhost.localdomain> <1138378937.3090.255.camel@localhost.localdomain> <1138381908.3090.259.camel@localhost.localdomain> <43DAABD8.5040404@v.loewis.de> Message-ID: <4f0b69dc0601290550s4543d2c7tad7c3374b96ac4f1@mail.gmail.com> On 1/28/06, "Martin v. L?wis" wrote: > Thomas Heller wrote: > > Can anyone of the python-dev core team comment: can we live with the GPL > > licensed aclocal.m4 file, in the source distribution and in SVN? > > My understanding that doing so would be in violation of section 2b) of > the GPL. > > However, I still think it is possible to include libffi - we just have > to discard the build process, and do our own. > I did some work to make ctypes+libffi compacter and liberal. http://openlook.org/svnpublic/ctypes-compactffi/ (svn) I removed sources/gcc and put sources/libffi copied from gcc 4.0.2. And removed all automake-related build processes and integrated them into setup.py. There's still aclocal.m4 in sources/libffi. But it is just identical to libffi's acinclude.m4 which looks liberal. Hye-Shik From stephen at xemacs.org Sun Jan 29 17:55:36 2006 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 30 Jan 2006 01:55:36 +0900 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 In-Reply-To: =?iso-8859-1?q?=3C43DB4A7A=2E8070606=40v=2Eloewis=2Ede=3E_?= =?iso-8859-1?q?=28Martin_v=2E_L=F6wis's_message_of_=22Sat=2C_28_Jan_200?= =?iso-8859-1?q?6_11=3A42=3A02_+0100=22=29?= References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <20060126090822.GF18916@xs4all.nl> <99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com> <43DAB283.70207@v.loewis.de> <90733CAD-8AB7-4749-9BF8-2386B12F0FAB@mac.com> <43DB4A7A.8070606@v.loewis.de> Message-ID: <87oe1v2ctz.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Martin" == Martin v L?wis writes: >> BTW. The argument that the readline module should be GPL >> licensed seems rather stronger, it's designed to work with a >> GPL-ed library and doesn't work with a BSD licensed work-alike >> of that library. Martin> This is the question what constitutes derivative work, and Martin> different lawyers have said different things in the Martin> past. If we really want to find out, we should ask a Martin> lawyer. You also need to ask about the cost of defending against a lawsuit by the FSF, which is both the copyright holder of the library and the primary advocate of the interpretation that a work which is intended to be linked with another work is a derivative. I think the FSF pretty much would have to fight any claims that contest its interpretation of the concept of "derived work", because any interpretation that requires a direct source-to-source copy will make the GPL irrelevant. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From martin at v.loewis.de Sun Jan 29 19:09:24 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 29 Jan 2006 19:09:24 +0100 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: <4f0b69dc0601290550s4543d2c7tad7c3374b96ac4f1@mail.gmail.com> References: <1137071518.12975.6.camel@devel002> <1137186878.3098.87.camel@localhost.localdomain> <1138378937.3090.255.camel@localhost.localdomain> <1138381908.3090.259.camel@localhost.localdomain> <43DAABD8.5040404@v.loewis.de> <4f0b69dc0601290550s4543d2c7tad7c3374b96ac4f1@mail.gmail.com> Message-ID: <43DD04D4.8010601@v.loewis.de> Hye-Shik Chang wrote: > I did some work to make ctypes+libffi compacter and liberal. > http://openlook.org/svnpublic/ctypes-compactffi/ (svn) > > I removed sources/gcc and put sources/libffi copied from gcc 4.0.2. > And removed all automake-related build processes and integrated > them into setup.py. There's still aclocal.m4 in sources/libffi. But > it is just identical to libffi's acinclude.m4 which looks liberal. Well done! Would you like to derive a Python patch from that? Don't worry about MSVC, yet, I will do that once the sources are in the subversion. (Of course, for due process, it would be better if this code gets integrated into the official ctypes first, and then we incorporate some named/versioned snapshot into /external, and svn cp it into python/trunk from there). Regards, Martin From martin at v.loewis.de Sun Jan 29 19:20:31 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 29 Jan 2006 19:20:31 +0100 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: References: <1137071518.12975.6.camel@devel002> <1137186878.3098.87.camel@localhost.localdomain> <1138378937.3090.255.camel@localhost.localdomain> <1138381908.3090.259.camel@localhost.localdomain> <074501c62373$df526cd0$bf03030a@trilan> <43DAAFC7.2060603@v.loewis.de> <4509FB68-6882-409E-9CFF-D5ADDA6ED210@internode.on.net> <43DB3381.7040002@v.loewis.de> <43DC760D.2020308@v.loewis.de><2m3bj7iek5.fsf@starship.python.net> <43DC90F0.8060105@v.loewis.de> Message-ID: <43DD076F.5030003@v.loewis.de> Terry Reedy wrote: >>>I think 2.58 actually had a brown-paper-bag release style bug, but >>>2.59 has been out for ages now. If we were prepared to >>>AC_PREREQ(2.59), I think this whole issue could go away. >> >>It seems you are right, so I removed the file, and require ac 2.59. > > > Does this mean that ctypes can and will be included with 2.5? No; as Michael Hudson said: this is an entirely unrelated issue, with Python's aclocal.m4 (which I deleted). It just occurred to him that this had TODOs as well (from a technological view, not from a licensing view). However, Hye-Shik Chan is to praise for looking into the libffi build process. So there is still a good chance that ctypes will be in 2.5 (if he or somebody else follows up). Regards, Martin From martin at v.loewis.de Sun Jan 29 19:39:30 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 29 Jan 2006 19:39:30 +0100 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 In-Reply-To: <87oe1v2ctz.fsf@tleepslib.sk.tsukuba.ac.jp> References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <20060126090822.GF18916@xs4all.nl> <99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com> <43DAB283.70207@v.loewis.de> <90733CAD-8AB7-4749-9BF8-2386B12F0FAB@mac.com> <43DB4A7A.8070606@v.loewis.de> <87oe1v2ctz.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <43DD0BE2.4050306@v.loewis.de> Stephen J. Turnbull wrote: > You also need to ask about the cost of defending against a lawsuit by > the FSF, which is both the copyright holder of the library and the > primary advocate of the interpretation that a work which is intended > to be linked with another work is a derivative. I think the FSF > pretty much would have to fight any claims that contest its > interpretation of the concept of "derived work", because any > interpretation that requires a direct source-to-source copy will make > the GPL irrelevant. So would you just like to see the readline module to be removed from the Python distribution? I personally don't, because I don't believe that the status quo conflicts with FSF's interpretation of the GPL, atleast not wrt. to anything the PSF does (i.e. source and Windows distribution). Also, I firmly believe that the FSF would *not* sue the PSF, but instead first ask that the status is corrected. Notice that the LGPL 2.1 somewhat elaborates on what the FSF considers "derived" wrt. linking: # When a program is linked with a library, whether statically or using a # shared library, the combination of the two is legally speaking a # combined work, a derivative of the original library. The ordinary # General Public License therefore permits such linking only if the # entire combination fits its criteria of freedom. So it is the act of linking (and probably to some extent, the act of compiling) that creates the derivative work. Regards, Martin From tim.peters at gmail.com Sun Jan 29 20:31:20 2006 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 29 Jan 2006 14:31:20 -0500 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 In-Reply-To: <43DD0BE2.4050306@v.loewis.de> References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <20060126090822.GF18916@xs4all.nl> <99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com> <43DAB283.70207@v.loewis.de> <90733CAD-8AB7-4749-9BF8-2386B12F0FAB@mac.com> <43DB4A7A.8070606@v.loewis.de> <87oe1v2ctz.fsf@tleepslib.sk.tsukuba.ac.jp> <43DD0BE2.4050306@v.loewis.de> Message-ID: <1f7befae0601291131jd2f7c52j4356efd889b070ff@mail.gmail.com> [Martin v. L?wis] > ... > Also, I firmly believe that the FSF would *not* sue the PSF, but > instead first ask that the status is corrected. I'd say that's almost certain. Like any organization with something fuzzy to protect, the FSF has far more to lose than to gain by daring a court to rule on their beliefs. Of course the PSF is in a similar boat: both parties would view a lawsuit as a rock-bottom last resort. I'm one PSF Director who would vote to capitulate in an instant to avoid fighting the FSF over readline, and not because of legal arguments, but because I respect their wishes about how _their_ work can be used. OTOH, in the absence of a statement from the FSF that they're unhappy about Python's readline module, I wouldn't yank it just to avoid a theoretical possibility that the FSF might complain someday. From vinay_sajip at yahoo.co.uk Mon Jan 30 00:53:14 2006 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Sun, 29 Jan 2006 23:53:14 +0000 (UTC) Subject: [Python-Dev] Extension to ConfigParser References: <43D930E0.6070805@voidspace.org.uk> Message-ID: Fuzzyman voidspace.org.uk> writes: > In the past there has been some discussion about a new module to replace > ConfigParser. Most notably at > http://wiki.python.org/moin/ConfigParserShootout [snip] > It would be possible to extend to the ConfigObj API to be backwards > compatible with ConfigParser. This would bring the added benefits of > ConfigObj, without needing to add an extra module to the standard library. > > Well nearly. ConfigObj supports config file schema with (optional) type > conversion, through a companion module called validate. This could be > included or left as an added option. > > Anyway. If this stands a *chance* of acceptance, I'll write the PEP (and > if accepted, do the work - which is not inconsiderable). Personally, I'd prefer to have the different candidates in the Shootout be evaluated and a "winner" picked (I'm not sure who would do this, or when it would be done). I'll readily declare an interest - I've implemented an alternative hierarchical config module (which is in no way backward compatible with ConfigParser), see http://www.red-dove.com/python_config.html Regards, Vinay Sajip From barry at python.org Mon Jan 30 00:57:07 2006 From: barry at python.org (Barry Warsaw) Date: Sun, 29 Jan 2006 18:57:07 -0500 Subject: [Python-Dev] SourceForge Download Page, Subversion Home Page In-Reply-To: <43DBBC10.6000601@v.loewis.de> References: <43D9F90E.1040703@holdenweb.com> <1f7befae0601270749w1dbb25f5lb785e7294c81fcff@mail.gmail.com> <43DAB573.3040209@v.loewis.de><43DAD946.3060107@gmail.com> <43DB41D8.4090300@v.loewis.de> <43DBBC10.6000601@v.loewis.de> Message-ID: <1138579027.11097.13.camel@geddy.wooz.org> On Sat, 2006-01-28 at 19:46 +0100, "Martin v. L?wis" wrote: > Barry Warsaw favours Jira as a tracker. Still do! At at one time the Atlassian folks offered to help us import the SF tracker data into Jira if we could get a machine readable (hopefully XML-ish) dump of the current SF tracker data. I don't know if that offer still stands, but if Jira were in the running I would contact them again about it. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060129/895a13fe/attachment-0001.pgp From stephen at xemacs.org Mon Jan 30 07:39:06 2006 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 30 Jan 2006 15:39:06 +0900 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 In-Reply-To: =?iso-8859-1?q?=3C43DD0BE2=2E4050306=40v=2Eloewis=2Ede=3E_?= =?iso-8859-1?q?=28Martin_v=2E_L=F6wis's_message_of_=22Sun=2C_29_Jan_200?= =?iso-8859-1?q?6_19=3A39=3A30_+0100=22=29?= References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <20060126090822.GF18916@xs4all.nl> <99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com> <43DAB283.70207@v.loewis.de> <90733CAD-8AB7-4749-9BF8-2386B12F0FAB@mac.com> <43DB4A7A.8070606@v.loewis.de> <87oe1v2ctz.fsf@tleepslib.sk.tsukuba.ac.jp> <43DD0BE2.4050306@v.loewis.de> Message-ID: <87r76qz0c5.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Martin" == Martin v L?wis writes: Martin> So would you just like to see the readline module to be Martin> removed from the Python distribution? No. I would much prefer that the readline module be made compatible with libedit (or whatever the pseudo-readline library is called) and that is what I recommend in the short term. Alternatively, libedit should be made more completely compatible with libreadline. In the long term, I would like to see your interpretation established in law (either by legislation or by precedent), but I certainly don't want any precedent set by having the PSF and the FSF at odds with each other. Martin> So it is the act of linking (and probably to some extent, Martin> the act of compiling) that creates the derivative work. The FSF did not accept that position when Aladdin advanced it in the case of Ghostscript. Cf. my reply to Tim Peters. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From theller at python.net Mon Jan 30 08:16:01 2006 From: theller at python.net (Thomas Heller) Date: Mon, 30 Jan 2006 08:16:01 +0100 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: <4f0b69dc0601290550s4543d2c7tad7c3374b96ac4f1@mail.gmail.com> (Hye-Shik Chang's message of "Sun, 29 Jan 2006 22:50:59 +0900") References: <1137071518.12975.6.camel@devel002> <1137186878.3098.87.camel@localhost.localdomain> <1138378937.3090.255.camel@localhost.localdomain> <1138381908.3090.259.camel@localhost.localdomain> <43DAABD8.5040404@v.loewis.de> <4f0b69dc0601290550s4543d2c7tad7c3374b96ac4f1@mail.gmail.com> Message-ID: Hye-Shik Chang writes: > On 1/28/06, "Martin v. L?wis" wrote: >> Thomas Heller wrote: >> > Can anyone of the python-dev core team comment: can we live with the GPL >> > licensed aclocal.m4 file, in the source distribution and in SVN? >> >> My understanding that doing so would be in violation of section 2b) of >> the GPL. >> >> However, I still think it is possible to include libffi - we just have >> to discard the build process, and do our own. >> > > I did some work to make ctypes+libffi compacter and liberal. > http://openlook.org/svnpublic/ctypes-compactffi/ (svn) > > I removed sources/gcc and put sources/libffi copied from gcc 4.0.2. > And removed all automake-related build processes and integrated > them into setup.py. There's still aclocal.m4 in sources/libffi. But > it is just identical to libffi's acinclude.m4 which looks liberal. That's great! Would you like to integrate these changes into to ctypes CVS repository yourself? I indend to do a few releases still from there, before the first Python 2.5. If so, please tell me your SF username and I'll add you as developer. Also, please note that all work should be done on the 'branch_1_0' CVS branch - the HEAD is only experimental code (and not currently used). Thomas From stephen at xemacs.org Mon Jan 30 09:51:11 2006 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 30 Jan 2006 17:51:11 +0900 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 In-Reply-To: <1f7befae0601291131jd2f7c52j4356efd889b070ff@mail.gmail.com> (Tim Peters's message of "Sun, 29 Jan 2006 14:31:20 -0500") References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <20060126090822.GF18916@xs4all.nl> <99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com> <43DAB283.70207@v.loewis.de> <90733CAD-8AB7-4749-9BF8-2386B12F0FAB@mac.com> <43DB4A7A.8070606@v.loewis.de> <87oe1v2ctz.fsf@tleepslib.sk.tsukuba.ac.jp> <43DD0BE2.4050306@v.loewis.de> <1f7befae0601291131jd2f7c52j4356efd889b070ff@mail.gmail.com> Message-ID: <873bj6yu80.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Tim" == Tim Peters writes: Tim> [Martin v. L?wis] >> Also, I firmly believe that the FSF would *not* sue the PSF, >> but instead first ask that the status is corrected. They would ask first. That's what they did in the case of Aladdin Ghostscript's use of readline. Tim> I'd say that's almost certain. Like any organization with Tim> something fuzzy to protect, the FSF has far more to lose than Tim> to gain by daring a court to rule on their beliefs. Of Tim> course the PSF is in a similar boat: both parties would view Tim> a lawsuit as a rock-bottom last resort. Aladdin took a position similar to Martin's, and only yanked the offending Makefile stanza when the FSF called them and said "we're ready to go to court; are you?" Tim> I wouldn't yank it just to avoid a theoretical possibility Tim> that the FSF might complain someday. It's not theoretical; it's almost identical to the Aladdin case. Legally the PSF is, if anything, in a weaker position than Aladdin (which did not distribute the module that interfaced to libreadline in Ghostscript, but merely a makefile stanza that used it if it were found). -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From fuzzyman at voidspace.org.uk Mon Jan 30 11:11:45 2006 From: fuzzyman at voidspace.org.uk (Fuzzyman) Date: Mon, 30 Jan 2006 10:11:45 +0000 Subject: [Python-Dev] Extension to ConfigParser In-Reply-To: References: <43D930E0.6070805@voidspace.org.uk> Message-ID: <43DDE661.4020808@voidspace.org.uk> Vinay Sajip wrote: > Fuzzyman voidspace.org.uk> writes: > > Hello Vinjay, >> In the past there has been some discussion about a new module to replace >> ConfigParser. Most notably at >> http://wiki.python.org/moin/ConfigParserShootout >> > [snip] > >> It would be possible to extend to the ConfigObj API to be backwards >> compatible with ConfigParser. This would bring the added benefits of >> ConfigObj, without needing to add an extra module to the standard library. >> >> Well nearly. ConfigObj supports config file schema with (optional) type >> conversion, through a companion module called validate. This could be >> included or left as an added option. >> >> Anyway. If this stands a *chance* of acceptance, I'll write the PEP (and >> if accepted, do the work - which is not inconsiderable). >> > > Personally, I'd prefer to have the different candidates in the Shootout be > evaluated and a "winner" picked (I'm not sure who would do this, or when it > would be done). Quite. I'm suggesting an alternative that bypasses that tortuous and unlikely process. ;-) > I'll readily declare an interest - I've implemented an > alternative hierarchical config module (which is in no way backward compatible > with ConfigParser), see > > http://www.red-dove.com/python_config.html > > I realise that there are several alternative modules available . Obviously my personal preference is ConfigObj (I'm not unbiased of course). :-) Lack of complexity is the major feature I would tout here - I guess other people would have different priorities. However, this not the only issue. Adding a new module, with a different API and possibly a different syntax for files, is a recipe for (some) confusion. Not to mention the difficulty of achieving a consensus on python-dev. (Again - ;-) The resolution I'm suggesting means that people can continue to use ConfigParser, with major feature enhancements. *Or* they can migrate to a slightly different API that is easier to use - without needing to switch between incompatible modules. I'm currently adding the ``ConfigParser.get*`` methods to ConfigObj (user request) and also adding full (straightforward) unicode support for reading and writing. These changes will be available in a beta release in the next few days. Anyway, debate on the issue is welcomed. All the best, Fuzzyman http://www.voidspace.org.uk/python/configobj.html > Regards, > > Vinay Sajip > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060130/e709ee5b/attachment.html From hyeshik at gmail.com Mon Jan 30 11:38:19 2006 From: hyeshik at gmail.com (Hye-Shik Chang) Date: Mon, 30 Jan 2006 19:38:19 +0900 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: References: <1137071518.12975.6.camel@devel002> <1137186878.3098.87.camel@localhost.localdomain> <1138378937.3090.255.camel@localhost.localdomain> <1138381908.3090.259.camel@localhost.localdomain> <43DAABD8.5040404@v.loewis.de> <4f0b69dc0601290550s4543d2c7tad7c3374b96ac4f1@mail.gmail.com> Message-ID: <4f0b69dc0601300238l635c2c35v7a1e1bf88f394bda@mail.gmail.com> On 1/30/06, "Martin v. L?wis" wrote: > Well done! Would you like to derive a Python patch from that? Yup. I'll do. On 1/30/06, Thomas Heller wrote: > That's great! Would you like to integrate these changes into to ctypes > CVS repository yourself? I indend to do a few releases still from > there, before the first Python 2.5. If so, please tell me your SF > username and I'll add you as developer. > > Also, please note that all work should be done on the 'branch_1_0' CVS > branch - the HEAD is only experimental code (and not currently used). My SF username is perky. I'll try to make it portable but it'll lose some platforms through compilers because few of libffi sources are in preprocessed assembly (.S) which isn't supported by distutils yet. So, we'll need tests on various compiler/platforms before the release. Hye-Shik From theller at python.net Mon Jan 30 13:18:16 2006 From: theller at python.net (Thomas Heller) Date: Mon, 30 Jan 2006 13:18:16 +0100 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: <4f0b69dc0601300238l635c2c35v7a1e1bf88f394bda@mail.gmail.com> (Hye-Shik Chang's message of "Mon, 30 Jan 2006 19:38:19 +0900") References: <1137071518.12975.6.camel@devel002> <1137186878.3098.87.camel@localhost.localdomain> <1138378937.3090.255.camel@localhost.localdomain> <1138381908.3090.259.camel@localhost.localdomain> <43DAABD8.5040404@v.loewis.de> <4f0b69dc0601290550s4543d2c7tad7c3374b96ac4f1@mail.gmail.com> <4f0b69dc0601300238l635c2c35v7a1e1bf88f394bda@mail.gmail.com> Message-ID: Hye-Shik Chang writes: > On 1/30/06, "Martin v. L?wis" wrote: >> Well done! Would you like to derive a Python patch from that? > > Yup. I'll do. > > On 1/30/06, Thomas Heller wrote: >> That's great! Would you like to integrate these changes into to ctypes >> CVS repository yourself? I indend to do a few releases still from >> there, before the first Python 2.5. If so, please tell me your SF >> username and I'll add you as developer. >> >> Also, please note that all work should be done on the 'branch_1_0' CVS >> branch - the HEAD is only experimental code (and not currently used). > > My SF username is perky. Added. > I'll try to make it portable but it'll > lose some platforms through compilers because few of libffi > sources are in preprocessed assembly (.S) which isn't supported > by distutils yet. So, we'll need tests on various compiler/platforms > before the release. Isn't it sufficient to append ".S" to self.compiler.src_extensions? Thanks, Thomas From gmccaughan at synaptics-uk.com Mon Jan 30 14:56:44 2006 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Mon, 30 Jan 2006 13:56:44 +0000 Subject: [Python-Dev] / as path join operator (was: Re: The path module PEP) In-Reply-To: References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> Message-ID: <200601301356.44616.gmccaughan@synaptics-uk.com> Steven Bethard wrote: > My only fear with the / operator is that we'll end up with the same > problems we have for using % in string formatting -- the order of > operations might not be what users expect. Since join is conceptually > an addition-like operator, I would expect: > > Path('home') / 'a' * 5 > > to give me: > > home/aaaaa > > If I understand it right, it would actually give me something like: > > home/ahome/ahome/ahome/ahome/a Is there any very deep magic that says / is the One True Operator to use for this, given that there's to be an operator for it? For instance, & has lower correct precedence (so that Path('home') & 'a'*5 does something less unexpected), and doesn't look quite so much as if it denotes arithmetic, and avoids semantic interference from the idea that "/" should divide things or make them smaller. (Though, for what it's worth, I think sticking another subdirectory onto a path *is* dividing and making smaller: think of a path as representing a subtree.) You do lose the pun on the Unix path separator, which is a shame. -- g From anthony at interlink.com.au Mon Jan 30 16:56:29 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue, 31 Jan 2006 02:56:29 +1100 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 In-Reply-To: <87r76qz0c5.fsf@tleepslib.sk.tsukuba.ac.jp> References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <43DD0BE2.4050306@v.loewis.de> <87r76qz0c5.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <200601310256.31337.anthony@interlink.com.au> Rather than the back-n-forth about what the FSF might or might not do, can we just ask them for an official opinion and settle the matter? The Aladdin case makes me think we should do this, probably before 2.5 comes out - because if we do have to yank readline, I'd really not like to see this happen in a bugfix release, for fairly obvious reasons. Who would we need to talk to for a definitive answer? I'm sure there's various FSF mailing lists where we could get 157 different potential answers, but I'm talking about an actual, official FSF statement <0.5 wink> Anthony From jeff at taupro.com Mon Jan 30 18:17:44 2006 From: jeff at taupro.com (Jeff Rush) Date: Mon, 30 Jan 2006 11:17:44 -0600 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 In-Reply-To: <200601310256.31337.anthony@interlink.com.au> References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <43DD0BE2.4050306@v.loewis.de> <87r76qz0c5.fsf@tleepslib.sk.tsukuba.ac.jp> <200601310256.31337.anthony@interlink.com.au> Message-ID: <43DE4A38.1020904@taupro.com> Anthony Baxter wrote: > Rather than the back-n-forth about what the FSF might or might not do, > can we just ask them for an official opinion and settle the matter? > > Who would we need to talk to for a definitive answer? I'm sure there's > various FSF mailing lists where we could get 157 different potential > answers, but I'm talking about an actual, official FSF statement <0.5 > wink> From http://www.fsf.org/licensing/licenses/index_html: "If you want help choosing a license, evaluating a license, or have any other questions about licenses, you can email us at ." -Jeff From ianb at colorstudy.com Mon Jan 30 18:50:47 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 30 Jan 2006 11:50:47 -0600 Subject: [Python-Dev] Extension to ConfigParser In-Reply-To: <43DDE661.4020808@voidspace.org.uk> References: <43D930E0.6070805@voidspace.org.uk> <43DDE661.4020808@voidspace.org.uk> Message-ID: <43DE51F7.4010007@colorstudy.com> Fuzzyman wrote: > The resolution I'm suggesting means that people can continue to use > ConfigParser, with major feature enhancements. *Or* they can migrate to > a slightly different API that is easier to use - without needing to > switch between incompatible modules. I don't think enhancing ConfigParser significantly is a good way forward. Because of ConfigParser's problems people have made all sorts of workarounds, and so I don't think there's any public interface that we can maintain while changing the internals without breaking lots of code. In practice, everything is a public interface. So I think the implementation as it stands should stay in place, and if anything it should be deprecated instead of being enhanced in-place. Another class or module could be added that fulfills the documented interface to ConfigParser. This would provide an easy upgrade path, without calling it a backward-compatible interface. I personally would like if any new config system included a parser, and then an interface to the configuration that was read (ConfigParser is only the latter). Then people who want to do their own thing can work with just the parser, without crudely extending and working around the configuration interface. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From jason.orendorff at gmail.com Mon Jan 30 19:06:16 2006 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Mon, 30 Jan 2006 13:06:16 -0500 Subject: [Python-Dev] / as path join operator In-Reply-To: <87vew44z8k.fsf@tleepslib.sk.tsukuba.ac.jp> References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> <87k6cnaabt.fsf@tleepslib.sk.tsukuba.ac.jp> <87vew44z8k.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: On 1/28/06, Stephen J. Turnbull wrote: > Please note that my point was entirely different from trying to decide > whether to subclass strings. Noted -- sorry I took you out of context there; that was careless. > Jason> Filesystem paths are in fact strings on all operating > Jason> systems I'm aware of. > > I have no idea what you could mean by that. The data structure used > to represent a filesystem on all OS filesystems I've used is a graph > of directories and files. A filesystem object is located by > traversing a path in that graph. You seem to think that because I said "operating systems", I'm talking about kernel algorithms and such. I'm not. By "on all operating systems" I really mean systems, not kernels: system APIs, standard tools, documentation, the conventions everyone follows--that sort of thing. Userspace. Thought experiment: How are filesystem paths used? Well, programs pass them into system calls like open() and chmod(). Programs use them to communicate with other programs. Users pass them to programs. Compare this to how you'd answer the question "How are integers used?": I think paths are used more for communication, less for computation. Their utility for communication is tightly bound to their string-nature. Essentially all APIs involving filesystem paths treat them as strings. It's not just that they take string parameters. The way they're designed, they encourage users to think of paths as strings, not graph-paths. Java's stdlib is the only API that even comes close to distinguishing paths from strings. The .NET class library doesn't bother. Many many people much smarter than I have thought about creating non-string-oriented filesystem APIs. Somehow it hasn't caught on. Essentially all users expect to see a filesystem path as a string of characters in the conventional format. Display it any other way (say, as a sequence of edge-names) and you risk baffling them. My position is (a) the convention that paths are strings really does exist, embedded in the design and culture of the dominant operating systems--in fact it's overwhelming, and I'm surprised anyone can miss it; (b) there might be a reason for it, even aside from momentum. -j From w.northcott at internode.on.net Sun Jan 29 01:57:37 2006 From: w.northcott at internode.on.net (Bill Northcott) Date: Sun, 29 Jan 2006 11:57:37 +1100 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: <43DB3381.7040002@v.loewis.de> References: <1137071518.12975.6.camel@devel002><1137186878.3098.87.camel@localhost.localdomain><1138378937.3090.255.camel@localhost.localdomain><1138381908.3090.259.camel@localhost.localdomain> <074501c62373$df526cd0$bf03030a@trilan> <43DAAFC7.2060603@v.loewis.de> <4509FB68-6882-409E-9CFF-D5ADDA6ED210@internode.on.net> <43DB3381.7040002@v.loewis.de> Message-ID: On 28/01/2006, at 8:04 PM, Martin v. L?wis wrote: >> The compiler needs specific exemptions because parts of the GPLed >> runtime libraries are included in all compiled code. No part of the >> autotools ends up in the finished code. If it did, you would need m4 >> to run Python and you don't. > > It doesn't matter whether it ends up in the finished code: if the > aclocal.m4 is indeed GPL-licensed, then the entire Python source > distribution must be GPL-licensed, because it "contains or > is derived from the Program or any part thereof". The build tools: m4 scripts, the configure shell script and the Makefiles all contain GPL code and are under GPL. However, none of this ends up in the 'finished program' which is the executable versions of Python and its associated libraries. The build tools are just tools not part of the program. The program is not 'derived' from the build tools. Maybe it would help to put it in a another domain: Say I decide to write the great Australian novel. I sit at my computer and type in the words. Those words are my copyright and I am able to restrict their use as provided in copyright legislation. These rights are inherent and exist regardless of whether or not I add a copyright notice. The notice is just a courtesy and a convenience which tells anyone who would like to use the work they need to get my permission. A license is just a set of standard permissions passed on with the work. Now I print out my novel. That printed version is 'derived' from the representation in the computer. So it is covered by the same copyright. If I compress the computer files, those are still derived work. If some American decides to write the great American novel by boiler-plating large chunks of my work with a little of his, then that work is derived and copyright to both of us. His actions are legal unless he tries to publish it, which would require my permission. This is analogous to linking a program with a library provided by another party. That is why one needs the specific concessions in the gcc compiler to cover the linked run time libraries. Of course the computer on which I wrote the book has an operating system which is copyright. The word processing software I used to do the writing is also copyright. However none of either program ends up in my novel. So the novel is not derived from them, and their copyright holders have no rights over the novel. I can happily package appropriately licensed copies of the word processing software with the finished book. So that others can try their hand at the same thing. In no way does such an operation give the software copyright holders any rights over the book. This is exactly analogous to including the GPL tools with your source distribution. You must comply with the GPL in respect of the GPL code. So you must include copyright notices and and make any modifications or improvements freely available. However, if you build a binary from the sources which does not include or link GPL code then the FSF have no rights over it and you are not obliged to acknowledge them or include their copyright notice. A Python binary is no more derived from the autotools than the book is derived from the word processing software. Bill Northcott From w.northcott at unsw.edu.au Sun Jan 29 02:04:58 2006 From: w.northcott at unsw.edu.au (Bill Northcott) Date: Sun, 29 Jan 2006 12:04:58 +1100 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: <43DBBC93.7010703@v.loewis.de> References: <1137071518.12975.6.camel@devel002> <1137186878.3098.87.camel@localhost.localdomain> <1138378937.3090.255.camel@localhost.localdomain> <1138381908.3090.259.camel@localhost.localdomain> <43DBBC93.7010703@v.loewis.de> Message-ID: On 29/01/2006, at 5:48 AM, Martin v. L?wis wrote: > Yes, but your conclusion is wrong. Python uses autoconf, but not > aclocal/automake. The generated configure is explicitly not covered by > the GPL; What makes you think that? I can see no such concession in the autoconf source distribution. A configure script is built up from lots of code fragments out of the autoconf and automake M4 files, and would clearly be covered by GPL. Bill Northcott From w.northcott at unsw.edu.au Sun Jan 29 02:14:10 2006 From: w.northcott at unsw.edu.au (Bill Northcott) Date: Sun, 29 Jan 2006 12:14:10 +1100 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: <43DBBC93.7010703@v.loewis.de> References: <1137071518.12975.6.camel@devel002> <1137186878.3098.87.camel@localhost.localdomain> <1138378937.3090.255.camel@localhost.localdomain> <1138381908.3090.259.camel@localhost.localdomain> <43DBBC93.7010703@v.loewis.de> Message-ID: OTOH this (from python-sig-mac) is a worry if it is correct: > s> Apparently the readline library in MacOSX isn't really > readline. > s> It's a renamed libedit. Not having encountered this deception > s> before, Python's build procedure doesn't know to test the > capability > s> of readline. I suggest you just comment out the readline > checks in > s> setup.py. > > Piet> libedit (editline) is a different library with a similar > function > Piet> as readline, but not compatible with it. It has a BSD > license. It > Piet> is often used instead of readline for non-GPL software. > > True. I believe it was written by Rich Salz several years ago. > Python used > to work with it, but eventually started using some readline > functionality > which libedit doesn't provide. I don't recall what that is. If you distribute Python sources that link the GNU libreadline then all your code will be covered by GPL, because libreadline uses the full GPL not LGPL. That is why Apple and other commercial vendors do not include it in their OS. Bill Northcott From w.northcott at unsw.edu.au Sun Jan 29 02:19:14 2006 From: w.northcott at unsw.edu.au (Bill Northcott) Date: Sun, 29 Jan 2006 12:19:14 +1100 Subject: [Python-Dev] (libffi) Re: Copyright issue References: Message-ID: <99D8275D-F7A2-4863-82C3-9AFB8E3D9055@unsw.edu.au> On 28/01/2006, at 8:04 PM, Martin v. L?wis wrote: >> The compiler needs specific exemptions because parts of the GPLed >> runtime libraries are included in all compiled code. No part of the >> autotools ends up in the finished code. If it did, you would need m4 >> to run Python and you don't. > > It doesn't matter whether it ends up in the finished code: if the > aclocal.m4 is indeed GPL-licensed, then the entire Python source > distribution must be GPL-licensed, because it "contains or > is derived from the Program or any part thereof". The build tools: m4 scripts, the configure shell script and the Makefiles all contain GPL code and are under GPL. However, none of this ends up in the 'finished program' which is the executable versions of Python and its associated libraries. The build tools are just tools not part of the program. The program is not 'derived' from the build tools. Maybe it would help to put it in a another domain: Say I decide to write the great Australian novel. I sit at my computer and type in the words. Those words are my copyright and I am able to restrict their use as provided in copyright legislation. These rights are inherent and exist regardless of whether or not I add a copyright notice. The notice is just a courtesy and a convenience which tells anyone who would like to use the work they need to get my permission. A license is just a set of standard permissions passed on with the work. Now I print out my novel. That printed version is 'derived' from the representation in the computer. So it is covered by the same copyright. If I compress the computer files, those are still derived work. If some American decides to write the great American novel by boiler-plating large chunks of my work with a little of his, then that work is derived and copyright to both of us. His actions are legal unless he tries to publish it, which would require my permission. This is analogous to linking a program with a library provided by another party. That is why one needs the specific concessions in the gcc compiler to cover the linked run time libraries. Of course the computer on which I wrote the book has an operating system which is copyright. The word processing software I used to do the writing is also copyright. However none of either program ends up in my novel. So the novel is not derived from them, and their copyright holders have no rights over the novel. I can happily package appropriately licensed copies of the word processing software with the finished book. So that others can try their hand at the same thing. In no way does such an operation give the software copyright holders any rights over the book. This is exactly analogous to including the GPL tools with your source distribution. You must comply with the GPL in respect of the GPL code. So you must include copyright notices and and make any modifications or improvements freely available. However, if you build a binary from the sources which does not include or link GPL code then the FSF have no rights over it and you are not obliged to acknowledge them or include their copyright notice. A Python binary is no more derived from the autotools than the book is derived from the word processing software. Bill Northcott From w.northcott at unsw.edu.au Sun Jan 29 03:04:54 2006 From: w.northcott at unsw.edu.au (Bill Northcott) Date: Sun, 29 Jan 2006 13:04:54 +1100 Subject: [Python-Dev] (libffi) Re: Copyright issue References: Message-ID: <7F4CBD32-9093-4F9C-AC4B-BFD2675C2885@unsw.edu.au> On 29/01/2006, at 5:48 AM, Martin v. L?wis wrote: > Yes, but your conclusion is wrong. Python uses autoconf, but not > aclocal/automake. The generated configure is explicitly not covered by > the GPL; What makes you think that? I can see no such concession in the autoconf source distribution. A configure script is built up from lots of code fragments out of the autoconf and automake M4 files, and would clearly be covered by GPL. Bill Northcott From w.northcott at unsw.edu.au Sun Jan 29 12:15:18 2006 From: w.northcott at unsw.edu.au (Bill Northcott) Date: Sun, 29 Jan 2006 22:15:18 +1100 Subject: [Python-Dev] (libffi) Re: Copyright issue In-Reply-To: <43DC760D.2020308@v.loewis.de> References: <1137071518.12975.6.camel@devel002><1137186878.3098.87.camel@localhost.localdomain><1138378937.3090.255.camel@localhost.localdomain><1138381908.3090.259.camel@localhost.localdomain> <074501c62373$df526cd0$bf03030a@trilan> <43DAAFC7.2060603@v.loewis.de> <4509FB68-6882-409E-9CFF-D5ADDA6ED210@internode.on.net> <43DB3381.7040002@v.loewis.de> <43DC760D.2020308@v.loewis.de> Message-ID: On 29/01/2006, at 7:00 PM, Martin v. L?wis wrote: > Again: What matters is what ends up in the source distribution, > http://www.python.org/ftp/python/2.4/Python-2.4.tgz No really that is wrong. What matters is what is in the Python executables, but you don't want to know that. So I will bow out of the discussion. Cheers Bill PS Linking with GNU libreadline really is a problem. From guido at python.org Mon Jan 30 20:20:18 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 30 Jan 2006 11:20:18 -0800 Subject: [Python-Dev] Extension to ConfigParser In-Reply-To: <43DE51F7.4010007@colorstudy.com> References: <43D930E0.6070805@voidspace.org.uk> <43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com> Message-ID: On 1/30/06, Ian Bicking wrote: > I don't think enhancing ConfigParser significantly is a good way > forward. Because of ConfigParser's problems people have made all sorts > of workarounds, and so I don't think there's any public interface that > we can maintain while changing the internals without breaking lots of > code. In practice, everything is a public interface. So I think the > implementation as it stands should stay in place, and if anything it > should be deprecated instead of being enhanced in-place. Somehow that's not my experience. What's so bad about ConfigParser? What would break if we rewrote the save functionality to produce a predictable order? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fuzzyman at voidspace.org.uk Mon Jan 30 21:19:49 2006 From: fuzzyman at voidspace.org.uk (Fuzzyman) Date: Mon, 30 Jan 2006 20:19:49 +0000 Subject: [Python-Dev] Extension to ConfigParser In-Reply-To: References: <43D930E0.6070805@voidspace.org.uk> <43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com> Message-ID: <43DE74E5.7000001@voidspace.org.uk> Guido van Rossum wrote: > On 1/30/06, Ian Bicking wrote: > >> I don't think enhancing ConfigParser significantly is a good way >> forward. Because of ConfigParser's problems people have made all sorts >> of workarounds, and so I don't think there's any public interface that >> we can maintain while changing the internals without breaking lots of >> code. In practice, everything is a public interface. So I think the >> implementation as it stands should stay in place, and if anything it >> should be deprecated instead of being enhanced in-place. >> > > Somehow that's not my experience. What's so bad about ConfigParser? > What would break if we rewrote the save functionality to produce a > predictable order? > Well, what I'm suggesting is not merely enhancing ConfigParser in place - but replacing it with the ConfigObj and a compatibility layer to support legacy code. As I mentioned, it would require *some* changes to the parser (ConfigObj) to be fully compatible with the existing syntax, but that is well achievable. The aim would be that no existing code breaks, and few (if any) config files break. I could flesh this out in a PEP if it was likely that it would be accepted. Issues with ConfigParser that ConfigObj *immediately* fixes : * Simpler API - uses dictionary syntax/methods for adding/deleting/accessing keys and sections (Each section - including the ConfigObj itself - is a dictionary subclass with all methods available) * Nested sub-sections to any depth. * Fully integrated (extensible) validation schema system, with type conversion. No complexity overhead for not using it. * List values parsed by default (handling quotes sanely) * Retains order of keys/section for iteration and writing out config file. * Retains *all* comments and allows inline comments The next enhancement will add full unicode support, which for a text based format really makes sense and I should have implemented it earlier. Additionally ConfigObj allows values in the root section - meaning that for simple config files you aren't *forced* to have an arbitrary 'section'. All the best, Fuzzyman http://www.voidspace.org.uk/python/configobj.html > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > > From ianb at colorstudy.com Mon Jan 30 21:23:16 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 30 Jan 2006 14:23:16 -0600 Subject: [Python-Dev] Extension to ConfigParser In-Reply-To: References: <43D930E0.6070805@voidspace.org.uk> <43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com> Message-ID: <43DE75B4.7000502@colorstudy.com> Guido van Rossum wrote: >>I don't think enhancing ConfigParser significantly is a good way >>forward. Because of ConfigParser's problems people have made all sorts >>of workarounds, and so I don't think there's any public interface that >>we can maintain while changing the internals without breaking lots of >>code. In practice, everything is a public interface. So I think the >>implementation as it stands should stay in place, and if anything it >>should be deprecated instead of being enhanced in-place. > > > Somehow that's not my experience. What's so bad about ConfigParser? > What would break if we rewrote the save functionality to produce a > predictable order? That's a fairly minor improvement, and I can't see how that would break anything. But Michael (aka Fuzzyman -- sorry, I just can't refer to you as Fuzzyman without feeling absurd ;) was proposing ConfigObj specifically (http://www.voidspace.org.uk/python/configobj.html). I assume the internals of ConfigObj bear no particular resemblence to ConfigParser, even if ConfigObj can parse the same syntax (plus some, and with different failure cases) and provide the same public API. While ConfigParser is okay for simple configuration, it is (IMHO) not a very good basis for anyone who wants to build better systems, like config files that can be changed programmatically, or error messages that point to file and line numbers. Those aren't necessarily features we need to expose in the standard library, but it'd be nice if you could implement that kind of feature without having to ignore the standard library entirely. That said, I'm not particularly enthused about a highly featureful config file *format* in the standard library, even if I would like a much more robust implementation. From my light reading on ConfigObj, it looks like it satisfies my personal goals (though I haven't used it), but maybe has too many features, like nested sections. And it seems like maybe the API can be reduced in size with a little high-level refactoring -- APIs generally grow over time so as to preserve backward compatibility, but I think if it was introduced into the standard library that might be an opportunity to trim the API back again before it enters the long-term API freeze that the standard library demands. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From fuzzyman at voidspace.org.uk Mon Jan 30 21:50:53 2006 From: fuzzyman at voidspace.org.uk (Fuzzyman) Date: Mon, 30 Jan 2006 20:50:53 +0000 Subject: [Python-Dev] Extension to ConfigParser In-Reply-To: <43DE51F7.4010007@colorstudy.com> References: <43D930E0.6070805@voidspace.org.uk> <43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com> Message-ID: <43DE7C2D.5060804@voidspace.org.uk> Ian Bicking wrote: > Fuzzyman wrote: >> The resolution I'm suggesting means that people can continue to use >> ConfigParser, with major feature enhancements. *Or* they can migrate >> to a slightly different API that is easier to use - without needing >> to switch between incompatible modules. > > I don't think enhancing ConfigParser significantly is a good way > forward. Because of ConfigParser's problems people have made all > sorts of workarounds, and so I don't think there's any public > interface that we can maintain while changing the internals without > breaking lots of code. In practice, everything is a public > interface. So I think the implementation as it stands should stay in > place, and if anything it should be deprecated instead of being > enhanced in-place. > > Another class or module could be added that fulfills the documented > interface to ConfigParser. This would provide an easy upgrade path, > without calling it a backward-compatible interface. I personally > would like if any new config system included a parser, and then an > interface to the configuration that was read (ConfigParser is only the > latter). Then people who want to do their own thing can work with just > the parser, without crudely extending and working around the > configuration interface. > But to implement a parser you still need to agree a basic syntax. For example ConfigObj supports list values, handles quotes sanely, uses triple quotes for multiline values, allows inline comments, and has a syntax for nested sections. Once you have agreed these and a *basic* API spec for accessing the data then you've implemented most of your config file handler. In order to implement a write (save) method you also have to agree on how to handle non string values (like lists). There's not an *awful* lot left to do. In ConfigObj if you switch off list handling when parsing and interpolation when fetching values, you have access to the raw values and are free to extend value handling any way you choose. In my opinion dictionary syntax is the most natural way to represent a config file - it is a data structure with named members. This means the API for accessing the data - whether from a subclass that does additional value processing or from code that just accesses the data. I may be misunderstanding you of course (or more likely not fully understanding). :-) All the best, Fuzzyman http://www.voidspace.org.uk/python/index.shtml From fuzzyman at voidspace.org.uk Mon Jan 30 22:02:59 2006 From: fuzzyman at voidspace.org.uk (Fuzzyman) Date: Mon, 30 Jan 2006 21:02:59 +0000 Subject: [Python-Dev] Extension to ConfigParser In-Reply-To: <43DE75B4.7000502@colorstudy.com> References: <43D930E0.6070805@voidspace.org.uk> <43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com> <43DE75B4.7000502@colorstudy.com> Message-ID: <43DE7F03.1080202@voidspace.org.uk> Ian Bicking wrote: > Guido van Rossum wrote: > >>> I don't think enhancing ConfigParser significantly is a good way >>> forward. Because of ConfigParser's problems people have made all sorts >>> of workarounds, and so I don't think there's any public interface that >>> we can maintain while changing the internals without breaking lots of >>> code. In practice, everything is a public interface. So I think the >>> implementation as it stands should stay in place, and if anything it >>> should be deprecated instead of being enhanced in-place. >>> >> Somehow that's not my experience. What's so bad about ConfigParser? >> What would break if we rewrote the save functionality to produce a >> predictable order? >> > > That's a fairly minor improvement, and I can't see how that would break > anything. But Michael (aka Fuzzyman -- sorry, I just can't refer to you > as Fuzzyman without feeling absurd ;) Don't worry, absurdity is a common reaction I inspire in people. Actually Fuzzyman is more of a description than a nickname. I understand that I may not be the only one worthy of such a title of course. ;-) > was proposing ConfigObj > specifically (http://www.voidspace.org.uk/python/configobj.html). I > assume the internals of ConfigObj bear no particular resemblence to > ConfigParser, even if ConfigObj can parse the same syntax (plus some, > and with different failure cases) and provide the same public API. > > While ConfigParser is okay for simple configuration, it is (IMHO) not a > very good basis for anyone who wants to build better systems, like > config files that can be changed programmatically, or error messages > that point to file and line numbers. Those aren't necessarily features > we need to expose in the standard library, but it'd be nice if you could > implement that kind of feature without having to ignore the standard > library entirely. > > Can you elaborate on what kinds of programattic changes you envisage ? I'm just wondering if there are classes of usage not covered by ConfigObj. Of course you can pretty much do anything to a ConfigObj instance programattically, but even so... > That said, I'm not particularly enthused about a highly featureful > config file *format* in the standard library, even if I would like a > much more robust implementation. > > I don't see how you can easily separate the format from the parser - unless you just leave raw values. (As I said in the other email, I don't think I fully understand you.) If accessing raw values suits your purposes, why not subclass ConfigParser and do magic in the get* methods ? > From my light reading on ConfigObj, it looks like it satisfies my > personal goals (though I haven't used it), but maybe has too many > features, like nested sections. And it seems like maybe the API can be > I personally think nested sections are very useful and would be sad to not see them included. Grouping additional configuration options as a sub-section can be *very* handy. > reduced in size with a little high-level refactoring -- APIs generally > grow over time so as to preserve backward compatibility, but I think if > it was introduced into the standard library that might be an opportunity > to trim the API back again before it enters the long-term API freeze > that the standard library demands. Hmmm.... possibly. :-) walk, decode and encode are the obvious extraneous methods. walk can be re-implemented as a recursive function and encode/decode will be unnecessary when unicode support is complete. Additional complexity comes from retaining all comments and supporting validation. I think retaining comments is worthwhile, because it allows a program to modify a config file (e.g. from a GUI) without losing user comments in a handwritten config file. The validation support has several levels to it - but it can be completely ignored with no complexity for the user. Some validation support is integrated into instantiation, but this could *probably* be broken out into a subclass. Anyway, ho hum - possibly. All for future hypothetical discussion I guess... All the best, Fuzzyman http://www.voidspace.org.uk/python/configobj.html From tim.peters at gmail.com Mon Jan 30 22:44:47 2006 From: tim.peters at gmail.com (Tim Peters) Date: Mon, 30 Jan 2006 16:44:47 -0500 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 In-Reply-To: <873bj6yu80.fsf@tleepslib.sk.tsukuba.ac.jp> References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com> <43DAB283.70207@v.loewis.de> <90733CAD-8AB7-4749-9BF8-2386B12F0FAB@mac.com> <43DB4A7A.8070606@v.loewis.de> <87oe1v2ctz.fsf@tleepslib.sk.tsukuba.ac.jp> <43DD0BE2.4050306@v.loewis.de> <1f7befae0601291131jd2f7c52j4356efd889b070ff@mail.gmail.com> <873bj6yu80.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <1f7befae0601301344u14547b7fw44d5d64eb841c1b5@mail.gmail.com> [Stephen J. Turnbull] > ... > Aladdin took a position similar to Martin's, and only yanked the > offending Makefile stanza when the FSF called them and said "we're > ready to go to court; are you?" > ... > It's not theoretical; it's almost identical to the Aladdin case. > Legally the PSF is, if anything, in a weaker position than Aladdin > (which did not distribute the module that interfaced to libreadline in > Ghostscript, but merely a makefile stanza that used it if it were > found). I'm not making myself clear. The FSF (like most organizations, including the PSF) has agendas that are more aptly described as political than as legal -- legalities are more of a club to try to force what they want. If the FSF merely _says_ they want to make it difficult, or even impossible, to link Python and/or Python apps with GNU readline, that's fine by me, and they don't have to make up creative license readings or threats to get that. I'm happy to accede to their wishes in the matter regardless of what the license says, or is claimed to say. OTOH, I have no reason to _presume_ that this is their hoped-for outcome wrt Python, neither to presume that the politics shaping their tussle with Aladdin are relevant to the PSF. "The law" is rarely applied uniformly, in large part because it usually is means rather than end. Python is a high-profile project that hasn't been hiding its readline module, and if I presume anything here it's that the FSF would have complained by now if they really didn't want this. In the meantime, I'm satisfied that the people involved with Python's readline module acted on good-faith readings of the licenses involved. If someone cares enough to rip it out, and/or to supply an alternative, fine by me too (but it won't _be_ me). From skip at pobox.com Mon Jan 30 22:17:28 2006 From: skip at pobox.com (skip at pobox.com) Date: Mon, 30 Jan 2006 15:17:28 -0600 Subject: [Python-Dev] Extension to ConfigParser In-Reply-To: References: <43D930E0.6070805@voidspace.org.uk> <43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com> Message-ID: <17374.33384.775422.175195@montanaro.dyndns.org> Guido> What's so bad about ConfigParser? My guess is that people find its limited nesting, well, limiting. Python containers being the arbitrary nesting little devils that they are, I can understand the desire to push the envelope. It's my opinion that ConfigParser should stay pretty much as it is other than perhaps adding round trip capability. It is pretty good at reading and writing Windows INI files, which was what it was designed to do. My guess is there are a fair number of people who wouldn't want to lose that. Mixing INI file handling with something more powerful seems doomed. If we want more sophisticated functionality a new module should be written, or one of the existing shootout candidates on the Wiki should be chosen and perhaps enhanced. I have a horse in that race (actually, it's more like a pony). Skip From guido at python.org Mon Jan 30 22:56:37 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 30 Jan 2006 13:56:37 -0800 Subject: [Python-Dev] Extension to ConfigParser In-Reply-To: <17374.33384.775422.175195@montanaro.dyndns.org> References: <43D930E0.6070805@voidspace.org.uk> <43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com> <17374.33384.775422.175195@montanaro.dyndns.org> Message-ID: On 1/30/06, skip at pobox.com wrote: > > Guido> What's so bad about ConfigParser? > > My guess is that people find its limited nesting, well, limiting. Python > containers being the arbitrary nesting little devils that they are, I can > understand the desire to push the envelope. To the contrary. Config file syntax should be limited by design so as to avoid confusing the people who have to edit them without knowing Python, and if an app is successful, that's most of the app's users. (If the app is only targeted at Python users, why bother with a config file? You could just have one designated .py file where all the configuration code is concentrated, like faqwizard's faqconf.py.) > It's my opinion that ConfigParser should stay pretty much as it is other > than perhaps adding round trip capability. It is pretty good at reading and > writing Windows INI files, which was what it was designed to do. My guess > is there are a fair number of people who wouldn't want to lose that. Mixing > INI file handling with something more powerful seems doomed. If we want > more sophisticated functionality a new module should be written, or one of > the existing shootout candidates on the Wiki should be chosen and perhaps > enhanced. I have a horse in that race (actually, it's more like a pony). Aha. I am beginning to understand. When people say "ConfigParser is hopeless" they mean ".INI files are hopeless". I happen to disagree. (There's also a meme that says that every aspect of an app should be configurable. I disagree with that too. As Joel Spolski points out in his book on UI design, most configuration options are signs of indecisive developers, and most users never change the defaults.) Regarding the claim that dicts are the best kind of API for configuration, I disagree -- the dict API doesn't maintain order, and doesn't provide access to comments and other metadata. And it has many operations that make little sense for configuration option values. (And would Michael please start signing with his full name, since he doesn't seem to care much for true anonymity? It's okay if your email says From: Michael So-and-so It's suboptimal if it says From: Fuzzyman You might also start signing your posts "Michael So-and-so" rather than the alias. I don't sign my email "BDFL" either. :-) Since there are many Michaels, signing just "Michael" doesn't really help. I'm too old fogey to get used to using IRC handles to refer to people; I can never keep the mapping in my head.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fuzzyman at voidspace.org.uk Mon Jan 30 23:17:10 2006 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 30 Jan 2006 22:17:10 +0000 Subject: [Python-Dev] Extension to ConfigParser In-Reply-To: References: <43D930E0.6070805@voidspace.org.uk> <43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com> <17374.33384.775422.175195@montanaro.dyndns.org> Message-ID: <43DE9066.2040100@voidspace.org.uk> Guido van Rossum wrote: > On 1/30/06, skip at pobox.com wrote: > >> Guido> What's so bad about ConfigParser? >> >> My guess is that people find its limited nesting, well, limiting. Python >> containers being the arbitrary nesting little devils that they are, I can >> understand the desire to push the envelope. >> > > To the contrary. Config file syntax should be limited by design so as > to avoid confusing the people who have to edit them without knowing > Python, and if an app is successful, that's most of the app's users. > (If the app is only targeted at Python users, why bother with a config > file? You could just have one designated .py file where all the > configuration code is concentrated, like faqwizard's faqconf.py.) > > >> It's my opinion that ConfigParser should stay pretty much as it is other >> than perhaps adding round trip capability. It is pretty good at reading and >> writing Windows INI files, which was what it was designed to do. My guess >> is there are a fair number of people who wouldn't want to lose that. Mixing >> INI file handling with something more powerful seems doomed. If we want >> more sophisticated functionality a new module should be written, or one of >> the existing shootout candidates on the Wiki should be chosen and perhaps >> enhanced. I have a horse in that race (actually, it's more like a pony). >> > > Aha. I am beginning to understand. When people say "ConfigParser is > hopeless" they mean ".INI files are hopeless". I happen to disagree. > (There's also a meme that says that every aspect of an app should be > configurable. I disagree with that too. As Joel Spolski points out in > his book on UI design, most configuration options are signs of > indecisive developers, and most users never change the defaults.) > > But like it or not, configuration files are often used to store data about what a program does - not just the UI options. Storing this in a human readable and editable format is of great advantage. Yes, a lot of the entries will never be modified by a user - but many will need to be edited and read by a developer. In a production environment importing from python modules is not likely to be safe. I don't dislike the INI format. ``key = value`` is intuitive. Nesting of sections is a logical extension that provides additional flexibility without breaking backwards compatibility. It needn't be added complexity, because you don't need to use it. There are plenty of people who will though. > Regarding the claim that dicts are the best kind of API for > configuration, I disagree -- the dict API doesn't maintain order, and > doesn't provide access to comments and other metadata. And it has many > operations that make little sense for configuration option values. > > An ordered dictionary is a fairly trivial modification of the dictionary (if you don't mind the performance hit), and yes there is other metadata also needs to be stored. Hey, since Python 2.2 some folk put a lot of work in allowing us to subclass dict. ;-) Perhaps a better way of saying this is that the dictionary API is an intuitive way to access config file data (including modifying and creating). You are mapping values to names, it's a logical match. As soon as you generate/manipulate config files programattically a lot of the dictionary methods become useful. Perhaps not all though. ``get``, ``update``, ``has_key`` are all useful. > (And would Michael please start signing with his full name, since he > doesn't seem to care much for true anonymity? It's okay if your email > says > > From: Michael So-and-so > > Michael So-and-so ??? Hey, that's fighting talk. ;-) > It's suboptimal if it says > > From: Fuzzyman > > You might also start signing your posts "Michael So-and-so" rather > than the alias. I don't sign my email "BDFL" either. :-) Since there > are many Michaels, signing just "Michael" doesn't really help. I'm too > old fogey to get used to using IRC handles to refer to people; I can > never keep the mapping in my head.) > Wow, my first rebuke from the BDFL. May it be the first of many. ;-) I've changed this client, but I use this account from multiple locations. Forgive me if I forget to change others (but feel free to remind me). Michael Foord (Chastened but Fuzzy nonetheless.) > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > > From guido at python.org Mon Jan 30 23:18:25 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 30 Jan 2006 14:18:25 -0800 Subject: [Python-Dev] [Doc-SIG] that library reference, again In-Reply-To: <05663E18-CDFB-4B18-A515-BAA21C48C1FD@lag.net> References: <4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com> <43B4A238.2030101@python.org> <51A7717B-3832-477E-BED5-3C36DCA20336@lag.net> <05663E18-CDFB-4B18-A515-BAA21C48C1FD@lag.net> Message-ID: On 1/26/06, Robey Pointer wrote: [quoting /F] > > moving all of (or parts of) the reference documentation in to the > > library source code would be an alternative, of course [1], but that > > would basically mean starting over from scratch. Bad idea. Putting the full docs in the source adds more coupling between the files and makes it harder instead of easier to edit the docs, especially for C code. Also, the needs for interactive help are different from those for off-line docs. Also note that several object types (e.g. lists and dicts, like used for sys.path and sys.modules) don't have __doc__ properties so documenting them in the source is problematic. The list goes on; e.g. the source code ordering may not be the same as the ideal documentation ordering; where do you put subsection introductions, etc. > I think that would be a good thing to do no matter what markup is > used. It always irritates me when I do 'help(sys.something)' and get > one line of ASCII art that's probably useful to the module author but > nobody else. Wrong guess. That string was put there by the author specifically so that it shows up in help(sys.whatever), and the author probably (mistakenly) though that whatever he wrote was sufficient. If you ever find a docstring that is IYO insufficient, please submit a patch or bug. Until we have a better mechanism, simply submitting the new text of the docstring, without bothering to make it into a context-diff or to use the proper markup, is fine. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From thomas at xs4all.net Mon Jan 30 23:23:21 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Mon, 30 Jan 2006 23:23:21 +0100 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 In-Reply-To: <1f7befae0601301344u14547b7fw44d5d64eb841c1b5@mail.gmail.com> References: <99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com> <43DAB283.70207@v.loewis.de> <90733CAD-8AB7-4749-9BF8-2386B12F0FAB@mac.com> <43DB4A7A.8070606@v.loewis.de> <87oe1v2ctz.fsf@tleepslib.sk.tsukuba.ac.jp> <43DD0BE2.4050306@v.loewis.de> <1f7befae0601291131jd2f7c52j4356efd889b070ff@mail.gmail.com> <873bj6yu80.fsf@tleepslib.sk.tsukuba.ac.jp> <1f7befae0601301344u14547b7fw44d5d64eb841c1b5@mail.gmail.com> Message-ID: <20060130222321.GP18916@xs4all.nl> On Mon, Jan 30, 2006 at 04:44:47PM -0500, Tim Peters wrote: > Python is a high-profile project that hasn't been hiding its readline > module, and if I presume anything here it's that the FSF would have > complained by now if they really didn't want this. In fact, we can be absolutely certain the FSF knows (or people in charge have known, rather) of the readline module, since it was an important reason to get Python's license GPL-compatible. Had they, at that time, considered code-that-wraps-libreadline a problem, they would have made that known. As it is, it was clear that the resulting binary (of non-GPL-compatible source and a GPL library) was a GPL violation, but the source itself was not. Not that they can't change their minds, of course. But I wouldn't go and ask them if they changed their minds yet; it might make them ;P -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From guido at python.org Mon Jan 30 23:40:04 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 30 Jan 2006 14:40:04 -0800 Subject: [Python-Dev] Extension to ConfigParser In-Reply-To: <43DE9066.2040100@voidspace.org.uk> References: <43D930E0.6070805@voidspace.org.uk> <43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com> <17374.33384.775422.175195@montanaro.dyndns.org> <43DE9066.2040100@voidspace.org.uk> Message-ID: > Guido van Rossum wrote: > > Aha. I am beginning to understand. When people say "ConfigParser is > > hopeless" they mean ".INI files are hopeless". I happen to disagree. > > (There's also a meme that says that every aspect of an app should be > > configurable. I disagree with that too. As Joel Spolski points out in > > his book on UI design, most configuration options are signs of > > indecisive developers, and most users never change the defaults.) On 1/30/06, Michael Foord wrote: > But like it or not, configuration files are often used to store data > about what a program does - not just the UI options. Storing this in a > human readable and editable format is of great advantage. > > Yes, a lot of the entries will never be modified by a user - but many > will need to be edited and read by a developer. In a production > environment importing from python modules is not likely to be safe. Ah. This definitely isn't what ConfigParser was meant to do. I'd think for this you should use some kind of XML pickle though. That's horrible if end users must edit it, but great for saving near-arbitrary persistent data in a readable and occasionally editable (for the developer) form. You should probably not mix the end user config data with the program's persistent store. > I don't dislike the INI format. ``key = value`` is intuitive. Nesting of > sections is a logical extension that provides additional flexibility > without breaking backwards compatibility. It needn't be added > complexity, because you don't need to use it. There are plenty of people > who will though. And I'm trying to say that (for config files) they probably shouldn't, because most of their users won't understand it. Of course this doesn't apply to your use case, so maybe we're just violently agreeing -- except you seem to be confusing matters by talking about "configuration" (which suggests end-user-editable) when you really mean "persistent state" (which is under full program control). > > Regarding the claim that dicts are the best kind of API for > > configuration, I disagree -- the dict API doesn't maintain order, and > > doesn't provide access to comments and other metadata. And it has many > > operations that make little sense for configuration option values. > > > An ordered dictionary is a fairly trivial modification of the dictionary > (if you don't mind the performance hit), and yes there is other metadata > also needs to be stored. Hey, since Python 2.2 some folk put a lot of > work in allowing us to subclass dict. ;-) Yeah, but that was more as a matter of principle than to encourage that use... Narrow APIs are typically better than wide APIs, and the dict API is the ultimate wide API. (I guess strings are just as bad; I haven't contributed to that thread yet but I think the new path module should not inherit from str or unicode either.) > Perhaps a better way of saying this is that the dictionary API is an > intuitive way to access config file data (including modifying and > creating). You are mapping values to names, it's a logical match. > > As soon as you generate/manipulate config files programattically a lot > of the dictionary methods become useful. Perhaps not all though. > ``get``, ``update``, ``has_key`` are all useful. has_key is obsolete though -- use 'in'. Anyway, we have a name for this -- we call it the "mapping protocol". In most cases it's better to implement a protocol than inherit a standard object. (This is perhaps unique to Python -- other OO languages often encourage inheritance, so I can understand the confusion.) > Wow, my first rebuke from the BDFL. May it be the first of many. ;-) > > I've changed this client, but I use this account from multiple > locations. Forgive me if I forget to change others (but feel free to > remind me). > > Michael Foord > (Chastened but Fuzzy nonetheless.) Thanks much! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik at pythonware.com Mon Jan 30 23:50:20 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Mon, 30 Jan 2006 23:50:20 +0100 Subject: [Python-Dev] Extension to ConfigParser References: <43D930E0.6070805@voidspace.org.uk><43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com><17374.33384.775422.175195@montanaro.dyndns.org><43DE9066.2040100@voidspace.org.uk> Message-ID: Guido van Rossum wrote: > Ah. This definitely isn't what ConfigParser was meant to do. I'd think > for this you should use some kind of XML pickle though. That's > horrible if end users must edit it, but great for saving > near-arbitrary persistent data in a readable and occasionally editable > (for the developer) form. fwiw, I've *never* used INI files to store program state, and I've never used the save support in ConfigParser. on the other hand, earlier today, Firefox messed up my user profile, and I'm not sure I had managed to sort that out if they hadn't used an INI file to keep track of available profiles... (especially not if they'd used Mork ;-) From fuzzyman at voidspace.org.uk Tue Jan 31 00:05:13 2006 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 30 Jan 2006 23:05:13 +0000 Subject: [Python-Dev] Extension to ConfigParser In-Reply-To: References: <43D930E0.6070805@voidspace.org.uk> <43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com> <17374.33384.775422.175195@montanaro.dyndns.org> <43DE9066.2040100@voidspace.org.uk> Message-ID: <43DE9BA9.3040807@voidspace.org.uk> Guido van Rossum wrote: > > On 1/30/06, Michael Foord wrote: > >> But like it or not, configuration files are often used to store data >> about what a program does - not just the UI options. Storing this in a >> human readable and editable format is of great advantage. >> >> Yes, a lot of the entries will never be modified by a user - but many >> will need to be edited and read by a developer. In a production >> environment importing from python modules is not likely to be safe. >> > > Ah. This definitely isn't what ConfigParser was meant to do. I'd think > for this you should use some kind of XML pickle though. That's > horrible if end users must edit it, but great for saving > near-arbitrary persistent data in a readable and occasionally editable > (for the developer) form. > > You should probably not mix the end user config data with the > program's persistent store. > > I have a feeling that for many complex applications the developer is the user. Agreed that giving complex configuration data along with program data to the end-user is not good. Despite this, data storage that needs to be edited by developers and system administration is a very common usage. I'm saying that for this purpose (up to a point) an INI file like syntax is much *better* than XML. >> I don't dislike the INI format. ``key = value`` is intuitive. Nesting of >> sections is a logical extension that provides additional flexibility >> without breaking backwards compatibility. It needn't be added >> complexity, because you don't need to use it. There are plenty of people >> who will though. >> > > And I'm trying to say that (for config files) they probably shouldn't, > because most of their users won't understand it. Of course this > doesn't apply to your use case, so maybe we're just violently agreeing > -- except you seem to be confusing matters by talking about > "configuration" (which suggests end-user-editable) when you really > mean "persistent state" (which is under full program control). > > I'm not really sure there is such a clear distinction. Especially for programs without a GUI, things like IP addresses, network details, paths to data files, plugged in components, device names, (etc) seem to be both. >>> Regarding the claim that dicts are the best kind of API for >>> configuration, I disagree [snip..] >> Hey, since Python 2.2 some folk put a lot of >> work in allowing us to subclass dict. ;-) >> > > Yeah, but that was more as a matter of principle than to encourage that use... > > Narrow APIs are typically better than wide APIs, and the dict API is > the ultimate wide API. (I guess strings are just as bad; I haven't > contributed to that thread yet but I think the new path module should > not inherit from str or unicode either.) > > I am a firm fan of the new path module but agnostic on whether it should inherit from basestring in some way. Wrong thread though. [snip...] > > Anyway, we have a name for this -- we call it the "mapping protocol". > In most cases it's better to implement a protocol than inherit a > standard object. (This is perhaps unique to Python -- other OO > languages often encourage inheritance, so I can understand the > confusion.) > > Mapping protocol is fine. Subclassing dict has the advantage that some aspects of the API are already implemented in C. Anyway - I modify my statement accordingly. For accessing configuration type data, the mapping protocol is an intuitive match. Named members mapping to values. Whether this is done via subclassing dict is an implementation detail. (Although I guess it implies that the full dictionary API will be available and you seem to be saying that you would rather that *only* relevant methods are available). All the best, Uhm... Michael Foord I guess From t-meyer at ihug.co.nz Mon Jan 30 23:56:09 2006 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Tue, 31 Jan 2006 11:56:09 +1300 Subject: [Python-Dev] Extension to ConfigParser In-Reply-To: <17374.33384.775422.175195@montanaro.dyndns.org> References: <43D930E0.6070805@voidspace.org.uk> <43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com> <17374.33384.775422.175195@montanaro.dyndns.org> Message-ID: <37371F19-FA7B-470F-ABB4-32B9E7E65A85@ihug.co.nz> [Guido] >> What's so bad about ConfigParser? [Skip Montanaro] > It's my opinion that ConfigParser should stay pretty much as it is > other > than perhaps adding round trip capability. [...] > If we want more sophisticated functionality a new module should be > written, > or one of the existing shootout candidates on the Wiki should be > chosen and perhaps > enhanced. +1. =Tony.Meyer From guido at python.org Tue Jan 31 00:15:45 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 30 Jan 2006 15:15:45 -0800 Subject: [Python-Dev] Extension to ConfigParser In-Reply-To: <43DE9BA9.3040807@voidspace.org.uk> References: <43D930E0.6070805@voidspace.org.uk> <43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com> <17374.33384.775422.175195@montanaro.dyndns.org> <43DE9066.2040100@voidspace.org.uk> <43DE9BA9.3040807@voidspace.org.uk> Message-ID: On 1/30/06, Michael Foord wrote: > I have a feeling that for many complex applications the developer is the > user. Agreed that giving complex configuration data along with program > data to the end-user is not good. > > Despite this, data storage that needs to be edited by developers and > system administration is a very common usage. > > I'm saying that for this purpose (up to a point) an INI file like syntax > is much *better* than XML. Sure. But for the same reasons, arbitrary nesting is probably bad. > I'm not really sure there is such a clear distinction. Especially for > programs without a GUI, things like IP addresses, network details, paths > to data files, plugged in components, device names, (etc) seem to be both. Just try to keep it simple. I still think it's a typical programmer's mistake to assume that there's no clear distinction. For a user who's looking to change one of the three simple properties that make sense for him to configure, having to hunt these down in a file containing hundreds of stuff she shouldn't tuch is a pain, plus there's the risk that an occasional slip of the finger causes damage to stuff they don't understand. > [snip...] > Mapping protocol is fine. Subclassing dict has the advantage that some > aspects of the API are already implemented in C. And many you don't need. You'll never be able to change the API because it's constrained by all existing uses, which means every dict API you've never heard of. If you're using this so much that speed's an issue perhaps you're overusing it. > Anyway - I modify my statement accordingly. For accessing configuration > type data, the mapping protocol is an intuitive match. Named members > mapping to values. Whether this is done via subclassing dict is an > implementation detail. (Although I guess it implies that the full > dictionary API will be available and you seem to be saying that you > would rather that *only* relevant methods are available). I can see several improvements on the dict protocol. For example, a handy API would be soemthing that produces a default value that is specific to the key you are requesting. Using dict.get(key, default) has the problem that if you ever decide to change the default you'll have to modify every place that references it. If would be nice if you could use dict.get(key) or even dict[key] and it gave you a default that was specified once when the the file was read. Another idea: have a list of multiple dict objects representing different config files (e.g. ./.appconfig, ../appconfig, ../../appconfig, ..., ~/.appconfig, /etc/appconfig, /opt/app/config). The dict semantics get in the way when making changes -- which file should be updated? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From t-meyer at ihug.co.nz Tue Jan 31 00:13:03 2006 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Tue, 31 Jan 2006 12:13:03 +1300 Subject: [Python-Dev] Extension to ConfigParser In-Reply-To: References: <43D930E0.6070805@voidspace.org.uk> <43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com> Message-ID: <6E8262A4-17F8-404E-8EA4-773BA2639100@ihug.co.nz> [Guido van Rossum] > What would break if we rewrote the save functionality to produce a > predictable order? As a reminder to anyone interested, there are three patches on SF that provide this (each in a different way): ConfigParser to accept a custom dict to allow ordering http://python.org/sf/1371075 Adds an optional argument to the constructor, the value of which will be the class used to store the configuration data (so that a third- party order-preserving dictionary class can be used). ConfigParser to save with order http://python.org/sf/1399309 Does a sort() of the sections and options before writing them. Add 'surgical editing' to ConfigParser http://python.org/sf/1410680 Adds an update_file method (no other changes) that updates the given file to match the current configuration, preserving blank lines, comments, and order. [Disclaimer: this is my patch] =Tony.Meyer From fuzzyman at voidspace.org.uk Tue Jan 31 00:31:35 2006 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 30 Jan 2006 23:31:35 +0000 Subject: [Python-Dev] Extension to ConfigParser In-Reply-To: References: <43D930E0.6070805@voidspace.org.uk> <43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com> <17374.33384.775422.175195@montanaro.dyndns.org> <43DE9066.2040100@voidspace.org.uk> <43DE9BA9.3040807@voidspace.org.uk> Message-ID: <43DEA1D7.6070208@voidspace.org.uk> Guido van Rossum wrote: > On 1/30/06, Michael Foord wrote: > >> I have a feeling that for many complex applications the developer is the >> user. Agreed that giving complex configuration data along with program >> data to the end-user is not good. >> >> Despite this, data storage that needs to be edited by developers and >> system administration is a very common usage. >> >> I'm saying that for this purpose (up to a point) an INI file like syntax >> is much *better* than XML. >> > > Sure. But for the same reasons, arbitrary nesting is probably bad. > > Interesting. It's one of the most frequent complaints (lack of nesting) I hear about ConfigParser. I guess I hang out in different circles. I still think it's a convenient way of grouping related options. Breaking them out into separate files is another way. >> I'm not really sure there is such a clear distinction. Especially for >> programs without a GUI, things like IP addresses, network details, paths >> to data files, plugged in components, device names, (etc) seem to be both. >> > > Just try to keep it simple. > > I still think it's a typical programmer's mistake to assume that > there's no clear distinction. For a user who's looking to change one > of the three simple properties that make sense for him to configure, > having to hunt these down in a file containing hundreds of stuff she > shouldn't tuch is a pain, plus there's the risk that an occasional > slip of the finger causes damage to stuff they don't understand. > > Right, I guess I'm still thinking more of the developer than user. For end-user simpler is definitely better. I still see use cases for ConfigObj where complex editable data (relatively - not database record sets) needs to be stored, and COnfigObj syntax is preferable to XML. > [snip..] >> Anyway - I modify my statement accordingly. For accessing configuration >> type data, the mapping protocol is an intuitive match. Named members >> mapping to values. Whether this is done via subclassing dict is an >> implementation detail. (Although I guess it implies that the full >> dictionary API will be available and you seem to be saying that you >> would rather that *only* relevant methods are available). >> > > I can see several improvements on the dict protocol. For example, a > handy API would be soemthing that produces a default value that is > specific to the key you are requesting. Using dict.get(key, default) > has the problem that if you ever decide to change the default you'll > have to modify every place that references it. If would be nice if you > could use dict.get(key) or even dict[key] and it gave you a default > that was specified once when the the file was read. We do this through the schema. You specify a default value for keys (optionally). This is supplied if the value is missing, but isn't written out again unless modified. For a *simpler* case (no possibility of a write) you can just merge in two config files, user data over the defaults. It doesn't keep a track of which file values come from though (your use case below). > Another idea: have > a list of multiple dict objects representing different config files > (e.g. ./.appconfig, ../appconfig, ../../appconfig, ..., ~/.appconfig, > /etc/appconfig, /opt/app/config). The dict semantics get in the way > when making changes -- which file should be updated? > > Hmmm.... all in all, I guess this is no. ConfigObj is fine as a separate module, and it's features are well used - so to that extent the status quo is fine. Have we got anywhere on an updated spec for ConfigParser replacement/enhancement ? Suggested features seem to include : * Round tripping (preserving order and possibly comments) (http://python.org/sf/1410680 maybe) * Mapping protocol for access is not a bad idea * List values seem a no brainer good idea to me * Better support for default values * Keeping track of values from multiple files, and writing changed values to the correct file We haven't discussed unicode of course, seems like a good idea for a text format. (It was one of the motivations for an alternative implementation Dict4ini.) I'd also like to see values allowed without a section. Flat config files with a few values are appropriate for many applications. (Maybe section=None to access, this was the approach for ConfigObj 3). Maybe the shooutout needs an update. All the best, Michael Foord > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > > From martin at v.loewis.de Tue Jan 31 01:07:58 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 31 Jan 2006 01:07:58 +0100 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 In-Reply-To: <200601310256.31337.anthony@interlink.com.au> References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <43DD0BE2.4050306@v.loewis.de> <87r76qz0c5.fsf@tleepslib.sk.tsukuba.ac.jp> <200601310256.31337.anthony@interlink.com.au> Message-ID: <43DEAA5E.2080300@v.loewis.de> Anthony Baxter wrote: > Rather than the back-n-forth about what the FSF might or might not do, > can we just ask them for an official opinion and settle the matter? We can ask, sure. Whether this settles the matter depends on the answer. Regards, Martin From martin at v.loewis.de Tue Jan 31 01:50:46 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 31 Jan 2006 01:50:46 +0100 Subject: [Python-Dev] "DOS" error codes, WindowsError, and errno Message-ID: <43DEB466.8030206@v.loewis.de> I have a new implementation of stat/fstat/wstat which directly uses Win32 API, rather than using msvcrt. This works fine so far, except that error handling turns out to be tricky. mscvcrt maps errors (GetLastError()) into errno.h values, and also preserves the original error code in _doserrno. Currently, stat() will raise OSError, with errno set to the errno.h value. Because the Win32 error codes are much more fine-grained, this conversion loses information. Python raises WindowsError in some cases (e.g. os.listdir); WindowsError inherits from OSError, but the errno attribute now must be interpreted as a Win32 error. This is unfortunate, because the values overlap, and somebody handling OSError might confuse the error codes for errno.h (msvcrt) values. So what should I do in the new stat implementation? Try to map error codes also? Raise WindowsErrors instead? Do something else entirely? Comments appreciated. Regards, Martin From guido at python.org Tue Jan 31 02:09:20 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 30 Jan 2006 17:09:20 -0800 Subject: [Python-Dev] "DOS" error codes, WindowsError, and errno In-Reply-To: <43DEB466.8030206@v.loewis.de> References: <43DEB466.8030206@v.loewis.de> Message-ID: What a mess. :-( WindowsError should have used a different name for the Windows-native error code, so we could have defined both separately without confusion. Is it too late to change WindowsError in that way? Unhelpfully, --Guido On 1/30/06, "Martin v. L?wis" wrote: > I have a new implementation of stat/fstat/wstat which directly uses > Win32 API, rather than using msvcrt. This works fine so far, except > that error handling turns out to be tricky. > > mscvcrt maps errors (GetLastError()) into errno.h values, and also > preserves the original error code in _doserrno. Currently, stat() > will raise OSError, with errno set to the errno.h value. > > Because the Win32 error codes are much more fine-grained, this > conversion loses information. Python raises WindowsError in > some cases (e.g. os.listdir); WindowsError inherits from OSError, > but the errno attribute now must be interpreted as a Win32 error. > This is unfortunate, because the values overlap, and somebody > handling OSError might confuse the error codes for errno.h > (msvcrt) values. > > So what should I do in the new stat implementation? Try to map > error codes also? Raise WindowsErrors instead? Do something else > entirely? > > Comments appreciated. > > Regards, > Martin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Jan 31 05:54:22 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 30 Jan 2006 20:54:22 -0800 Subject: [Python-Dev] from __future__ syntax changed Message-ID: It looks like the syntax for "from __future__ import ..." changes in Python 2.5. I was playing around with Cheetah, and it stumbled over a file with the following structure: # comments """ docstring """ __author__ = "..." __version__ = "..." from __future__ import generators Python 2.2 throug 2.4 accept this fine. Perhaps the AST tree enforces stricter syntax for what can precede a future statement? I think this should be fixed. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From mhammond at skippinet.com.au Tue Jan 31 06:03:01 2006 From: mhammond at skippinet.com.au (Mark Hammond) Date: Tue, 31 Jan 2006 16:03:01 +1100 Subject: [Python-Dev] "DOS" error codes, WindowsError, and errno In-Reply-To: Message-ID: Guido: > What a mess. :-( > > WindowsError should have used a different name for the Windows-native > error code, so we could have defined both separately without > confusion. > > Is it too late to change WindowsError in that way? I guess "too late" is purely a judgement call about breaking existing code. One thing to our advantage is that I believe the most common errno explicitly checked for will be ENOENT, which happily has the same value as ERROR_FILE_NOT_FOUND. [Actually, checking 2 *or* 3 (ERROR_PATH_NOT_FOUND) is also common - but ESRCH==3, which sounds reasonable] Another way forward may be to issue a warning whenever '.errno' or '[0]' is referenced from WindowsError, noting that the semantics are soon to change, and to use the new attribute if they really do want a win32 error code. I'm not sure how practical that is though... Mark > > Unhelpfully, > > --Guido > > On 1/30/06, "Martin v. L?wis" wrote: > > I have a new implementation of stat/fstat/wstat which directly uses > > Win32 API, rather than using msvcrt. This works fine so far, except > > that error handling turns out to be tricky. > > > > mscvcrt maps errors (GetLastError()) into errno.h values, and also > > preserves the original error code in _doserrno. Currently, stat() > > will raise OSError, with errno set to the errno.h value. > > > > Because the Win32 error codes are much more fine-grained, this > > conversion loses information. Python raises WindowsError in > > some cases (e.g. os.listdir); WindowsError inherits from OSError, > > but the errno attribute now must be interpreted as a Win32 error. > > This is unfortunate, because the values overlap, and somebody > > handling OSError might confuse the error codes for errno.h > > (msvcrt) values. > > > > So what should I do in the new stat implementation? Try to map > > error codes also? Raise WindowsErrors instead? Do something else > > entirely? > > From tim.peters at gmail.com Tue Jan 31 06:53:26 2006 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 31 Jan 2006 00:53:26 -0500 Subject: [Python-Dev] from __future__ syntax changed In-Reply-To: References: Message-ID: <1f7befae0601302153n701d732ek79307c509962d115@mail.gmail.com> [Guido van Rossum] > It looks like the syntax for "from __future__ import ..." changes in > Python 2.5. I was playing around with Cheetah, and it stumbled over a > file with the following structure: > > # comments > """ docstring """ > __author__ = "..." > __version__ = "..." > from __future__ import generators > > Python 2.2 throug 2.4 accept this fine. > > Perhaps the AST tree enforces stricter syntax for what can precede a > future statement? I think this should be fixed. Peculiar. It wasn't intended that be allowed; PEP 236 spelled this out: """" In addition, all future_statments must appear near the top of the module. The only lines that can appear before a future_statement are: + The module docstring (if any). + Comments. + Blank lines. + Other future_statements. """" That isn't frivolous, since a __future__ gimmick may affect the legality of binding statements (as in your example) or other import statements, etc -- it was deliberate that only blank lines, comments and (maybe) a docstring could appear before the first future statment. I'd call this a bug in 2.2-2.4. Note too that Tools/scripts/cleanfuture.py in 2.4 doesn't change that example, because it doesn't believe it contains any __future__ statements (cleanfuture.py follows the PEP's syntax). From martin at v.loewis.de Tue Jan 31 07:46:26 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 31 Jan 2006 07:46:26 +0100 Subject: [Python-Dev] "DOS" error codes, WindowsError, and errno In-Reply-To: References: <43DEB466.8030206@v.loewis.de> Message-ID: <43DF07C2.5080302@v.loewis.de> Guido van Rossum wrote: > WindowsError should have used a different name for the Windows-native > error code, so we could have defined both separately without > confusion. > > Is it too late to change WindowsError in that way? We could define a different exception, say, Win32Error which inherits from OSError, and has a POSIX errno in the errno attribute, and also an additional attribute win32_error. We could then stop raising WindowsError. So code that expects WindowsError will never see it, but will see OSError instead (which it also should handle if it was written portable). Or we could just break the errno value in WindowsError, which would make code that catches WindowsError continue to work, but break code that looks for specific errno values in WindowsError. Code aiming for backwards compatibility should then write except WindowsError, e: try: windows_error = e.windows_error except NameError: # Python 2.4 and earlier windows_error = e.errno As for naming, I would like to avoid naming new things Win32: Microsoft just changed the name of the API to "Windows API" (pointing out that it isn't necessarily 32 bits anymore). Regards, Martin From martin at v.loewis.de Tue Jan 31 07:58:25 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 31 Jan 2006 07:58:25 +0100 Subject: [Python-Dev] "DOS" error codes, WindowsError, and errno In-Reply-To: References: Message-ID: <43DF0A91.9020407@v.loewis.de> Mark Hammond wrote: > I guess "too late" is purely a judgement call about breaking existing code. > One thing to our advantage is that I believe the most common errno > explicitly checked for will be ENOENT, which happily has the same value as > ERROR_FILE_NOT_FOUND. [Actually, checking 2 *or* 3 (ERROR_PATH_NOT_FOUND) > is also common - but ESRCH==3, which sounds reasonable] The strerror for ESRCH is "No such process", which isn't really that reasonable. Breakage would occur, because people might currently check for ENOENT (or any other specific error code); if they don't see that specific error code, they will assume it is a "real" error. That said: I would actually expect that it is infrequent that people do look at the errno of a WindowsError, so breakage might be small. > Another way forward may be to issue a warning whenever '.errno' or '[0]' is > referenced from WindowsError, noting that the semantics are soon to change, > and to use the new attribute if they really do want a win32 error code. I'm > not sure how practical that is though... It would need to get suppressed when the value is merely displayed, atleast in the default __str__ implementation. Regards, Martin From stephen at xemacs.org Tue Jan 31 10:51:12 2006 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 31 Jan 2006 18:51:12 +0900 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 In-Reply-To: <1f7befae0601301344u14547b7fw44d5d64eb841c1b5@mail.gmail.com> (Tim Peters's message of "Mon, 30 Jan 2006 16:44:47 -0500") References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com> <43DAB283.70207@v.loewis.de> <90733CAD-8AB7-4749-9BF8-2386B12F0FAB@mac.com> <43DB4A7A.8070606@v.loewis.de> <87oe1v2ctz.fsf@tleepslib.sk.tsukuba.ac.jp> <43DD0BE2.4050306@v.loewis.de> <1f7befae0601291131jd2f7c52j4356efd889b070ff@mail.gmail.com> <873bj6yu80.fsf@tleepslib.sk.tsukuba.ac.jp> <1f7befae0601301344u14547b7fw44d5d64eb841c1b5@mail.gmail.com> Message-ID: <87vew0ohdb.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Tim" == Tim Peters writes: Tim> I'm not making myself clear. Whatever makes you think that? In fact, everything you've said about your criteria for behavior was quite clear from the first, and it was fairly easy to infer your beliefs about the implications of history. I just wanted to state that there is a clear precedent of an FSF complaint about exactly what Python is doing. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From stephen at xemacs.org Tue Jan 31 11:09:26 2006 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 31 Jan 2006 19:09:26 +0900 Subject: [Python-Dev] / as path join operator In-Reply-To: (Jason Orendorff's message of "Mon, 30 Jan 2006 13:06:16 -0500") References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com> <43D6BA85.8070007@colorstudy.com> <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com> <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz> <87k6cnaabt.fsf@tleepslib.sk.tsukuba.ac.jp> <87vew44z8k.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <87r76oogix.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Jason" == Jason Orendorff writes: Jason> You seem to think that because I said "operating systems", Jason> I'm talking about kernel algorithms and such. I can see how you'd get that impression, but it's not true. My reason for mentioning OS-level filesystem was to show that even in that limited domain, treating paths as strings leads to bugs. Jason> I think paths are used more for communication, less for Jason> computation. True. For that purpose it is absolutely essential to have a string represention. However, I was discussing the use of "/" to invoke path composition, which is a computation. Nobody will use that for communication (except to describe a path expression in graph theoretic terms), and I don't think it's a good idea to use that particular symbol for that operation. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From p.f.moore at gmail.com Tue Jan 31 11:52:00 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 31 Jan 2006 10:52:00 +0000 Subject: [Python-Dev] Extension to ConfigParser In-Reply-To: References: <43D930E0.6070805@voidspace.org.uk> <43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com> <17374.33384.775422.175195@montanaro.dyndns.org> Message-ID: <79990c6b0601310252t2bff85cfkc11273ea646bcc31@mail.gmail.com> On 1/30/06, Guido van Rossum wrote: > Aha. I am beginning to understand. When people say "ConfigParser is > hopeless" they mean ".INI files are hopeless". I happen to disagree. > (There's also a meme that says that every aspect of an app should be > configurable. I disagree with that too. As Joel Spolski points out in > his book on UI design, most configuration options are signs of > indecisive developers, and most users never change the defaults.) While you're right that a lot of the comments people make about ConfigParser are actually about INI files (e.g., no nested sections, no "top-level" values), I do find that when I use ConfigParser, I often need to write a wrapper around it to make it more usable. A couple of examples, from a current application: * No way to merge files or sections. Usually to provide default values. I have a suite of applications, all using the same framework. I have a hardcoded DEFAULT_CONFIG in the code, overriden by a .ini, overridden again by a .ini. OK, maybe it's overengineered, but I do use the various levels, so it's also useful... (The defaults parameter to the constructor is misnamed, AFAICT, as it's for interpolation defaults, not value defaults). * A dictionary interface, or even attribute access (where the key names are Python identifiers, which is usually the case), is usually far cleaner to work with in code. For example, conf.log.level vs conf.get('log', 'level'). But that may just be me - it's certainly a style issue. Nothing major, but worth considering. I agree entirely that once you go beyond INI format, you should have a new module. Of course, ConfigParser already allows more than basic INI format ("key: value" lines, continuations, and substitution strings) but we'll gloss over that... :-) Paul. From thomas at xs4all.net Tue Jan 31 11:59:20 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Tue, 31 Jan 2006 11:59:20 +0100 Subject: [Python-Dev] Compiler warnings Message-ID: <20060131105920.GQ18916@xs4all.nl> I noticed a few compiler warnings, when I compile Python on my amd64 with gcc 4.0.3: Objects/longobject.c: In function ?PyLong_AsDouble?: Objects/longobject.c:655: warning: ?e? may be used uninitialized in this function Objects/longobject.c: In function ?long_true_divide?: Objects/longobject.c:2263: warning: ?aexp? may be used uninitialized in this function Objects/longobject.c:2263: warning: ?bexp? may be used uninitialized in this function Modules/linuxaudiodev.c: In function ?lad_obuffree?: Modules/linuxaudiodev.c:392: warning: ?ssize? may be used uninitialized in this function Modules/linuxaudiodev.c: In function ?lad_bufsize?: Modules/linuxaudiodev.c:348: warning: ?ssize? may be used uninitialized in this function Modules/linuxaudiodev.c: In function ?lad_obufcount?: Modules/linuxaudiodev.c:369: warning: ?ssize? may be used uninitialized in this function Those are all fairly harmless, just the compiler can't figure out that they are never actually used when they aren't explicitly initialized, because the initialization happens a few functioncalls deeper into the callstack. This one: Python/marshal.c: In function ?PyMarshal_ReadShortFromFile?: Python/marshal.c:410: warning: ?rf.end? is used uninitialized in this function Python/marshal.c:410: warning: ?rf.ptr? is used uninitialized in this function (The linenumber is off, it should be 884) is more of the same, except you can't tell from the function that it is a "can never happen" situation: if PyMarshal_ReadShortFromFile() was called with NULL as fp, it would actually use the uninitialized 'end' and 'ptr' struct members. An assert() is probably in place here. Should these warnings be fixed? I know Tim has always argued to fix them, in the past (and I agree,) and it doesn't look like doing so, by initializing the variables, wouldn't be too big a performance hit. I also noticed test_logging is spuriously failing, and not just on my machine (according to buildbot logs.) Is anyone (Vinay?) looking at that yet? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From fuzzyman at voidspace.org.uk Tue Jan 31 13:27:25 2006 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 31 Jan 2006 12:27:25 +0000 Subject: [Python-Dev] Extension to ConfigParser In-Reply-To: <79990c6b0601310252t2bff85cfkc11273ea646bcc31@mail.gmail.com> References: <43D930E0.6070805@voidspace.org.uk> <43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com> <17374.33384.775422.175195@montanaro.dyndns.org> <79990c6b0601310252t2bff85cfkc11273ea646bcc31@mail.gmail.com> Message-ID: <43DF57AD.5010007@voidspace.org.uk> Paul Moore wrote: > On 1/30/06, Guido van Rossum wrote: > >> Aha. I am beginning to understand. When people say "ConfigParser is >> hopeless" they mean ".INI files are hopeless". I happen to disagree. >> (There's also a meme that says that every aspect of an app should be >> configurable. I disagree with that too. As Joel Spolski points out in >> his book on UI design, most configuration options are signs of >> indecisive developers, and most users never change the defaults.) >> > > While you're right that a lot of the comments people make about > ConfigParser are actually about INI files (e.g., no nested sections, > no "top-level" values), I do find that when I use ConfigParser, I > often need to write a wrapper around it to make it more usable. > > A couple of examples, from a current application: > > * No way to merge files or sections. Usually to provide default > values. I have a suite of applications, all using the same framework. > I have a hardcoded DEFAULT_CONFIG in the code, overriden by a > .ini, overridden again by a .ini. OK, maybe it's > overengineered, but I do use the various levels, so it's also > useful... (The defaults parameter to the constructor is misnamed, > AFAICT, as it's for interpolation defaults, not value defaults). > > * A dictionary interface, or even attribute access (where the key > names are Python identifiers, which is usually the case), is usually > far cleaner to work with in code. For example, conf.log.level vs > conf.get('log', 'level'). But that may just be me - it's certainly a > style issue. > > Nothing major, but worth considering. > > I agree entirely that once you go beyond INI format, you should have a > new module. Of course, ConfigParser already allows more than basic INI > format ("key: value" lines, continuations, and substitution strings) > but we'll gloss over that... :-) > Well, ConfigObj is listed as an XML alternative at : http://www.pault.com/pault/pxml/xmlalternatives.html For complex configuration situations, many people feel that handcrafting XML is not a fun alternative. One of the driving forces behind ConfigObj was a firm that *used* to have their system administrators using ZConfig, but wanted something more friendly. They have configuration files nested a few levels deep configuring services, plugins, network details, etc. This is all properly 'application configuration'. (Not data persistence.) The file format (and config structure) is easily visible from the files. The config files are automatically validated from the schema (configspecs) which also facilitate the automatic creation of new sections. I *personally* think that this is a common use case and the format is a natural extension of the 'INI-like' format. I'm happy for it to remain a separate module (as indeed ConfigObj will). There are several valid complaints about ConfigParser - and it is *possible* (although possibly not desirable) to largely solve them all without breaking backwards compatibility with ConfigParser. Is it likely that an *additional* module could be added ? The suggestion to narrow the ConfigObj API and not subclass dict (but still use the mapping protocol) are interesting, and certainly not out of the question. Nested sections however, are a significant part of the reason for it's existence. All the best, Fuzzyman http://www.voidspace.org.uk/python/index.shtml > Paul. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > > From fuzzyman at voidspace.org.uk Tue Jan 31 13:38:19 2006 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 31 Jan 2006 12:38:19 +0000 Subject: [Python-Dev] Extension to ConfigParser In-Reply-To: <6E8262A4-17F8-404E-8EA4-773BA2639100@ihug.co.nz> References: <43D930E0.6070805@voidspace.org.uk> <43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com> <6E8262A4-17F8-404E-8EA4-773BA2639100@ihug.co.nz> Message-ID: <43DF5A3B.7010705@voidspace.org.uk> Tony Meyer wrote: > [Guido van Rossum] > >> What would break if we rewrote the save functionality to produce a >> predictable order? >> > > As a reminder to anyone interested, there are three patches on SF > that provide this (each in a different way): > > ConfigParser to accept a custom dict to allow ordering > http://python.org/sf/1371075 > Adds an optional argument to the constructor, the value of which will > be the class used to store the configuration data (so that a third- > party order-preserving dictionary class can be used). > > ConfigParser to save with order > http://python.org/sf/1399309 > Does a sort() of the sections and options before writing them. > > These first two patches don't seem to address the main issue, which is preserving the *original* order of the file. > Add 'surgical editing' to ConfigParser > http://python.org/sf/1410680 > Adds an update_file method (no other changes) that updates the given > file to match the current configuration, preserving blank lines, > comments, and order. [Disclaimer: this is my patch] > > This one however does, and if it does what it says on the box - is definitely worth accepting. All the best, Michael Foord http://www.voidspace.org.uk/python/index.shtml > =Tony.Meyer > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > > From vinay_sajip at yahoo.co.uk Tue Jan 31 17:13:19 2006 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Tue, 31 Jan 2006 16:13:19 +0000 (UTC) Subject: [Python-Dev] Extension to ConfigParser References: <43D930E0.6070805@voidspace.org.uk> <43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com> <43DE7C2D.5060804@voidspace.org.uk> Message-ID: Fuzzyman voidspace.org.uk> writes: > In my opinion dictionary syntax is the most natural way to represent a > config file - it is a data structure with named members. This means the > API for accessing the data - whether from a subclass that does > additional value processing or from code that just accesses the data. I agree with this, and also with Michael's comments that configuration is not just for the end user. (Alternatively, you could say that developers are end users too!) As a developer who works for many different clients, I find it saves me a fair amount of time when I have configurable software modules which I can put together to meet client requirements. The client doesn't need to see this "internal" configuration stuff - it's for developers and maintainers of the software, but very useful nonetheless! When I implemented my own config module (see http://www.red-dove.com/python_config.html), I decided not to use an INI-like format, as I found it wanting. The syntax I used is like Python and like JSON, but it is not executable (i.e. not Python) and allows bits of the configuration to refer to other bits of the configuration, provides some restricted evaluation of values such as "sys.stderr" but in a general way, readily allows defaults at different levels (e.g. user, program, suite), handles repeating values and mappings, uses dict or attribute syntax etc. - and all in a single 1500-line module, since I didn't have to implement INI support. What I'd like to know is, what happens how? There are a whole lot of alternatives put forward in the ConfigParserShootout wiki page, but how do we move things forward from here? Regards, Vinay Sajip From martin at v.loewis.de Tue Jan 31 18:04:45 2006 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Tue, 31 Jan 2006 18:04:45 +0100 Subject: [Python-Dev] Compiler warnings In-Reply-To: <20060131105920.GQ18916@xs4all.nl> References: <20060131105920.GQ18916@xs4all.nl> Message-ID: <43DF98AD.3050602@v.loewis.de> Thomas Wouters wrote: > I also noticed test_logging is spuriously failing, and not just on my > machine (according to buildbot logs.) Is anyone (Vinay?) looking at that > yet? I looked at it, and I couldn't figure it out. It appears that the last line of communication is somehow lost, but I could not find out why that might happen. Regards, Martin From barry at python.org Mon Jan 30 23:23:18 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 30 Jan 2006 17:23:18 -0500 Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through 2006-01-15 In-Reply-To: <1f7befae0601301344u14547b7fw44d5d64eb841c1b5@mail.gmail.com> References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com> <99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com> <43DAB283.70207@v.loewis.de> <90733CAD-8AB7-4749-9BF8-2386B12F0FAB@mac.com> <43DB4A7A.8070606@v.loewis.de> <87oe1v2ctz.fsf@tleepslib.sk.tsukuba.ac.jp> <43DD0BE2.4050306@v.loewis.de> <1f7befae0601291131jd2f7c52j4356efd889b070ff@mail.gmail.com> <873bj6yu80.fsf@tleepslib.sk.tsukuba.ac.jp> <1f7befae0601301344u14547b7fw44d5d64eb841c1b5@mail.gmail.com> Message-ID: <1138659798.10363.24.camel@geddy.wooz.org> On Mon, 2006-01-30 at 16:44 -0500, Tim Peters wrote: > OTOH, I have no reason to _presume_ that this is their hoped-for > outcome wrt Python, neither to presume that the politics shaping their > tussle with Aladdin are relevant to the PSF. "The law" is rarely > applied uniformly, in large part because it usually is means rather > than end. Python is a high-profile project that hasn't been hiding > its readline module, and if I presume anything here it's that the FSF > would have complained by now if they really didn't want this. I agree, and suggest we do nothing unless the FSF asks us to. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060130/9230701f/attachment.pgp From evdo.hsdpa at gmail.com Tue Jan 31 19:45:09 2006 From: evdo.hsdpa at gmail.com (Robert Kim Wireless Internet Advisor) Date: Tue, 31 Jan 2006 10:45:09 -0800 Subject: [Python-Dev] (no subject) Message-ID: <1ec620e90601311045g6fd0d206x29b998a83b6f28d4@mail.gmail.com> anybody here? bob 2611 s highway 101 san diego, ca 92007 209 984 0880 http://evdo-coverage.com/cell-phone-antenna-booster.html -- Robert Q Kim, Wireless Internet Advisor http://evdo-coverage.com/cellular-repeater.html http://hsdpa-coverage.com 2611 S. Pacific Coast Highway 101 Suite 102 Cardiff by the Sea, CA 92007 206 984 0880 From thomas at xs4all.net Tue Jan 31 20:16:20 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Tue, 31 Jan 2006 20:16:20 +0100 Subject: [Python-Dev] Compiler warnings In-Reply-To: <43DF98AD.3050602@v.loewis.de> References: <20060131105920.GQ18916@xs4all.nl> <43DF98AD.3050602@v.loewis.de> Message-ID: <20060131191620.GR18916@xs4all.nl> On Tue, Jan 31, 2006 at 06:04:45PM +0100, "Martin v. L?wis" wrote: > Thomas Wouters wrote: > > I also noticed test_logging is spuriously failing, and not just on my > > machine (according to buildbot logs.) Is anyone (Vinay?) looking at that > > yet? > I looked at it, and I couldn't figure it out. It appears that the last > line of communication is somehow lost, but I could not find out why that > might happen. Not just the last line, see below. It never happens, as far as I can tell, when the test_logging test is run alone; it never once broke in several hundred solo tests. When I ran all combinations of a non-logging test followed by test_logging, it seems to break fairly randomly with almost all tests. Certainly not just tests that deal with signals, subprocesses, threads or sockets, also with tests like test_dircache, test_cookielib, test_coding, test_filecmp, and others. Just about the only correlation that makes sense is the duration of the first test. Not all of the other tests trigger test_logging's failure in any degree of reliability, although I get the feeling tests that take longer trigger the bug more often -- except that a test consisting of 'time.sleep(30)' never triggered it. All in all, it seems like a timing issue -- the timing is different because the previous test already imported some modules. The test_logging test uses threads, *ugh*. I tried adding a few Event-waits and time.sleep()'s at strategic places, but it doesn't seem to matter. Regardless of my pinpointing efforts, much more than just the last line is sometimes missing, though: $ ./python -E -tt Lib/test/regrtest.py -uall test_dircache test_logging test_dircache test_logging test test_logging produced unexpected output: ********************************************************************** *** line 515 of expected output missing: - DEB -> WARNING: Message 16 (via logrecv.tcp.DEB) *** line 521 of expected output missing: - UNDEF -> INFO: Message 22 (via logrecv.tcp.UNDEF) *** line 524 of expected output missing: - INF -> INFO: Finish up, it's closing time. Messages should bear numbers 0 through 24. (via logrecv.tcp.INF) ********************************************************************** I've 'seen' at least all messages between 16 and 22 disappear, although not more than 3 or 4 in total. There's only ever messages missing, never out of order or garbled. I'm starting to think there might actually be a bug in the logging module ;P -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From scott+python-dev at scottdial.com Tue Jan 31 20:43:00 2006 From: scott+python-dev at scottdial.com (Scott Dial) Date: Tue, 31 Jan 2006 14:43:00 -0500 Subject: [Python-Dev] Extension to ConfigParser In-Reply-To: <43DF5A3B.7010705@voidspace.org.uk> References: <43D930E0.6070805@voidspace.org.uk> <43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com> <6E8262A4-17F8-404E-8EA4-773BA2639100@ihug.co.nz> <43DF5A3B.7010705@voidspace.org.uk> Message-ID: <43DFBDC4.6020804@scottdial.com> Michael Foord wrote: > Tony Meyer wrote: >> [Guido van Rossum] >> Add 'surgical editing' to ConfigParser >> http://python.org/sf/1410680 >> Adds an update_file method (no other changes) that updates the given >> file to match the current configuration, preserving blank lines, >> comments, and order. [Disclaimer: this is my patch] >> >> > This one however does, and if it does what it says on the box - is > definitely worth accepting. > I've spent a small amount of time playing with this patch, and the intent is there, but it appears to have some obvious bugs with adding blank lines and (at least) making an empty [DEFAULT] section appear and disappear. I'm not sure that this is the place to make these comments, so I will stop with that. In case it wasn't clear, I am new here.. Hi. -- Scott Dial scott at scottdial.com dialsa at rose-hulman.edu From mattheww at chiark.greenend.org.uk Tue Jan 31 20:46:39 2006 From: mattheww at chiark.greenend.org.uk (mattheww at chiark.greenend.org.uk) Date: Tue, 31 Jan 2006 19:46:39 +0000 (UTC) Subject: [Python-Dev] Octal literals References: Message-ID: Guido van Rossum wrote: >Right. And this is not a hypothetical issue either -- in Perl, hex and >oct *do* work the other way I believe. More reasons to get rid of >these in Python 3000. Perhaps we should also get rid of hex/oct >lterals? I would like to argue for removing octal literals. This feature has a very bad property: it can cause obscure problems for people who do not know or care about it. I have seen people try to use leading zeroes to make integer literals line up in a table. If they are lucky, they will get a syntax error. If they are unlucky, their program will silently do the wrong thing. It would be rather offputting to have to warn about this in the tutorial. But at present, a learner who isn't familiar with another language using this convention would have no reason to suspect it exists. As far as I can tell, it's documented only in the BNF. I think the safe thing in Python 3000 would be for literals with leading 0 to be syntax errors. Possibly os.chmod and os.umask could be extended to take a string argument so we could write chmod(path, "0640"). -M- From ark at acm.org Tue Jan 31 20:55:33 2006 From: ark at acm.org (Andrew Koenig) Date: Tue, 31 Jan 2006 14:55:33 -0500 Subject: [Python-Dev] Octal literals In-Reply-To: Message-ID: <00ec01c626a0$5afdc080$6402a8c0@arkdesktop> > Possibly os.chmod and os.umask could be extended to take a string > argument so we could write chmod(path, "0640"). -1. Would you really want chmod(path, 0640) and chmod(path, "0640") to have different meanings? From g.brandl at gmx.net Tue Jan 31 20:57:18 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 31 Jan 2006 20:57:18 +0100 Subject: [Python-Dev] YAML (was Re: Extension to ConfigParser) In-Reply-To: References: <43D930E0.6070805@voidspace.org.uk> <43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com> <17374.33384.775422.175195@montanaro.dyndns.org> <43DE9066.2040100@voidspace.org.uk> Message-ID: Guido van Rossum wrote: >> But like it or not, configuration files are often used to store data >> about what a program does - not just the UI options. Storing this in a >> human readable and editable format is of great advantage. >> >> Yes, a lot of the entries will never be modified by a user - but many >> will need to be edited and read by a developer. In a production >> environment importing from python modules is not likely to be safe. > > Ah. This definitely isn't what ConfigParser was meant to do. I'd think > for this you should use some kind of XML pickle though. That's > horrible if end users must edit it, but great for saving > near-arbitrary persistent data in a readable and occasionally editable > (for the developer) form. While we're at it, is the Python library going to incorporate some YAML parser in the future? YAML seems like a perfectly matching data format for Python. Georg From t-meyer at ihug.co.nz Tue Jan 31 20:49:02 2006 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Wed, 1 Feb 2006 08:49:02 +1300 Subject: [Python-Dev] Extension to ConfigParser In-Reply-To: <43DFBDC4.6020804@scottdial.com> References: <43D930E0.6070805@voidspace.org.uk> <43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com> <6E8262A4-17F8-404E-8EA4-773BA2639100@ihug.co.nz> <43DF5A3B.7010705@voidspace.org.uk> <43DFBDC4.6020804@scottdial.com> Message-ID: <4127C17B-01A0-4E47-93C4-71B1C138F8E7@ihug.co.nz> [Scott Dial] [Re: http://python.org/sf/1410680] > I've spent a small amount of time playing with this patch, and the > intent is there, but it appears to have some obvious bugs with adding > blank lines and (at least) making an empty [DEFAULT] section appear > and > disappear. I'm not sure that this is the place to make these comments, > so I will stop with that. Could you please add whatever information you have to the tracker? If you don't have a sf account and don't want to get one, I suppose you could just email it to me and I'll add it. =Tony.Meyer From t-meyer at ihug.co.nz Tue Jan 31 20:54:42 2006 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Wed, 1 Feb 2006 08:54:42 +1300 Subject: [Python-Dev] Extension to ConfigParser In-Reply-To: <79990c6b0601310252t2bff85cfkc11273ea646bcc31@mail.gmail.com> References: <43D930E0.6070805@voidspace.org.uk> <43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com> <17374.33384.775422.175195@montanaro.dyndns.org> <79990c6b0601310252t2bff85cfkc11273ea646bcc31@mail.gmail.com> Message-ID: [Paul Moore] > * No way to merge files or sections. Usually to provide default > values. I have a suite of applications, all using the same framework. > I have a hardcoded DEFAULT_CONFIG in the code, overriden by a > .ini, overridden again by a .ini. OK, maybe it's > overengineered, but I do use the various levels, so it's also > useful... (The defaults parameter to the constructor is misnamed, > AFAICT, as it's for interpolation defaults, not value defaults). Why doesn't this work? It does here: $ cat suite.ini [sect] opt1 = 1 opt2 = 2 $ cat app.ini [sect] opt1 = 3 opt4 = 5 $ python Python 2.4.1 (#2, Mar 31 2005, 00:05:10) [GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import ConfigParser >>> c = ConfigParser.ConfigParser() >>> c.read(("suite.ini", "app.ini")) ['suite.ini', 'app.ini'] >>> c.sections() ['sect'] >>> c.options("sect") ['opt4', 'opt2', 'opt1'] >>> c.get("sect", "opt1") '3' Or do you mean something else? =Tony.Meyer From mattheww at chiark.greenend.org.uk Tue Jan 31 21:03:36 2006 From: mattheww at chiark.greenend.org.uk (mattheww at chiark.greenend.org.uk) Date: Tue, 31 Jan 2006 20:03:36 +0000 (UTC) Subject: [Python-Dev] Octal literals References: <00ec01c626a0$5afdc080$6402a8c0@arkdesktop> Message-ID: Andrew Koenig wrote: >> Possibly os.chmod and os.umask could be extended to take a string >> argument so we could write chmod(path, "0640"). > >-1. > >Would you really want chmod(path, 0640) and chmod(path, "0640") to have >different meanings? I want the former to be a syntax error, as I said in the preceding paragraph. -M- From martin at v.loewis.de Tue Jan 31 21:04:39 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 31 Jan 2006 21:04:39 +0100 Subject: [Python-Dev] Compiler warnings In-Reply-To: <20060131191620.GR18916@xs4all.nl> References: <20060131105920.GQ18916@xs4all.nl> <43DF98AD.3050602@v.loewis.de> <20060131191620.GR18916@xs4all.nl> Message-ID: <43DFC2D7.5080506@v.loewis.de> Thomas Wouters wrote: > All in all, it seems like a timing issue -- the timing is different because > the previous test already imported some modules. The test_logging test uses > threads, *ugh*. I tried adding a few Event-waits and time.sleep()'s at > strategic places, but it doesn't seem to matter. Regardless of my > pinpointing efforts, much more than just the last line is sometimes missing, > though: Can you create a truss/strace dump of a failed run (I tried, but it always succeeded for me when run under strace). Regards, Martin From phd at mail2.phd.pp.ru Tue Jan 31 21:07:48 2006 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Tue, 31 Jan 2006 23:07:48 +0300 Subject: [Python-Dev] JSON (was: YAML) In-Reply-To: References: <43D930E0.6070805@voidspace.org.uk> <43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com> <17374.33384.775422.175195@montanaro.dyndns.org> <43DE9066.2040100@voidspace.org.uk> Message-ID: <20060131200748.GA19860@phd.pp.ru> On Tue, Jan 31, 2006 at 08:57:18PM +0100, Georg Brandl wrote: > While we're at it, is the Python library going to incorporate some YAML > parser in the future? YAML seems like a perfectly matching data format > for Python. JSON is even better! Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From fdrake at acm.org Tue Jan 31 21:06:29 2006 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Tue, 31 Jan 2006 15:06:29 -0500 Subject: [Python-Dev] Octal literals In-Reply-To: <00ec01c626a0$5afdc080$6402a8c0@arkdesktop> References: <00ec01c626a0$5afdc080$6402a8c0@arkdesktop> Message-ID: <200601311506.29633.fdrake@acm.org> On Tuesday 31 January 2006 14:55, Andrew Koenig wrote: > Would you really want chmod(path, 0640) and chmod(path, "0640") to have > different meanings? Actually, the proposal that suggested this also proposed that 0640 would raise a SyntaxError, since it was all about getting rid of octal literals. -Fred -- Fred L. Drake, Jr. From p.f.moore at gmail.com Tue Jan 31 21:29:04 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 31 Jan 2006 20:29:04 +0000 Subject: [Python-Dev] Extension to ConfigParser In-Reply-To: References: <43D930E0.6070805@voidspace.org.uk> <43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com> <17374.33384.775422.175195@montanaro.dyndns.org> <79990c6b0601310252t2bff85cfkc11273ea646bcc31@mail.gmail.com> Message-ID: <79990c6b0601311229o4a88bd1evf279c7bfcaa7db4a@mail.gmail.com> On 1/31/06, Tony Meyer wrote: > Why doesn't this work? It does here: > > $ cat suite.ini > [sect] > opt1 = 1 > opt2 = 2 > $ cat app.ini > [sect] > opt1 = 3 > opt4 = 5 > $ python > Python 2.4.1 (#2, Mar 31 2005, 00:05:10) > [GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin > Type "help", "copyright", "credits" or "license" for more information. > >>> import ConfigParser > >>> c = ConfigParser.ConfigParser() > >>> c.read(("suite.ini", "app.ini")) > ['suite.ini', 'app.ini'] > >>> c.sections() > ['sect'] > >>> c.options("sect") > ['opt4', 'opt2', 'opt1'] > >>> c.get("sect", "opt1") > '3' > > Or do you mean something else? Err. Because I missed the fact that read() method takes multiple filenames? There's even a specific explanation of how to load defaults and then override them with optional files. I don't know how I missed that. Thanks for pointing it out. (The whole day's been like that - I'm not sure why I get out of bed sometimes....:-) Paul. From thomas at xs4all.net Tue Jan 31 21:42:57 2006 From: thomas at xs4all.net (Thomas Wouters) Date: Tue, 31 Jan 2006 21:42:57 +0100 Subject: [Python-Dev] Compiler warnings In-Reply-To: <43DFC2D7.5080506@v.loewis.de> References: <20060131105920.GQ18916@xs4all.nl> <43DF98AD.3050602@v.loewis.de> <20060131191620.GR18916@xs4all.nl> <43DFC2D7.5080506@v.loewis.de> Message-ID: <20060131204257.GS18916@xs4all.nl> On Tue, Jan 31, 2006 at 09:04:39PM +0100, "Martin v. L?wis" wrote: > Thomas Wouters wrote: > > All in all, it seems like a timing issue -- the timing is different because > > the previous test already imported some modules. The test_logging test uses > > threads, *ugh*. I tried adding a few Event-waits and time.sleep()'s at > > strategic places, but it doesn't seem to matter. Regardless of my > > pinpointing efforts, much more than just the last line is sometimes missing, > > though: > Can you create a truss/strace dump of a failed run (I tried, but it > always succeeded for me when run under strace). Nope, they always succeed when run under strace. I haven't been able to capture the supposedly-TCP traffic either, not even when binding and connecting to my actual IP address. I wonder if glibc is doing trickery because it sees both endpoints of the socket are in the same process. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From guido at python.org Tue Jan 31 21:47:12 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 31 Jan 2006 12:47:12 -0800 Subject: [Python-Dev] Octal literals In-Reply-To: <00ec01c626a0$5afdc080$6402a8c0@arkdesktop> References: <00ec01c626a0$5afdc080$6402a8c0@arkdesktop> Message-ID: On 1/31/06, Andrew Koenig wrote: > > Possibly os.chmod and os.umask could be extended to take a string > > argument so we could write chmod(path, "0640"). > > -1. > > Would you really want chmod(path, 0640) and chmod(path, "0640") to have > different meanings? Apart from making 0640 a syntax error (which I think is wrong too), could this be solved by *requiring* the argument to be a string? (Or some other data type, but that's probably overkill.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ark at acm.org Tue Jan 31 23:17:22 2006 From: ark at acm.org (Andrew Koenig) Date: Tue, 31 Jan 2006 17:17:22 -0500 Subject: [Python-Dev] Octal literals In-Reply-To: Message-ID: <011c01c626b4$2d6a0750$6402a8c0@arkdesktop> > Apart from making 0640 a syntax error (which I think is wrong too), > could this be solved by *requiring* the argument to be a string? (Or > some other data type, but that's probably overkill.) That solves the problem only in that particular context. I would think that if it is deemed undesirable for a leading 0 to imply octal, then it would be best to decide on a different syntax for octal literals and use that syntax consistently everywhere. I am personally partial to allowing an optional radix (in decimal) followed by the letter r at the beginning of a literal, so 19, 8r23, and 16r13 would all represent the same value.