From ehuss at ironport.com Sun Apr 1 01:44:16 2007 From: ehuss at ironport.com (Eric Huss) Date: Sat, 31 Mar 2007 16:44:16 -0700 (PDT) Subject: [Python-Dev] BaseException pickle issue Message-ID: Some subclasses of Exception are no longer pickleable in Python 2.5. An example: class A(Exception): def __init__(self, foo): self.foo = foo The key problem here is if you do not somehow set self.args to the correct arguments (via Exception.__init__ or setting self.args directly), you will get a TypeError when you try to unpickle it: TypeError: __init__() takes exactly 2 arguments (1 given) I do not think this is unusual. I found a few examples in Python's standard library that have this problem: subprocess.CalledProcessError HTMLParser.HTMLParseError httplib.UnknownProtocol, httplib.IncompleteRead, httplib.BadStatusLine optparse.OptParseError pickle._Stop and on and on... Does anyone have any thoughts about this? Is it a bug? I can imagine one could argue that exceptions should call the base __init__ method to properly set args, but there are so many exceptions out there that do not do this that it would be very difficult to track them all down. I removed the __reduce__ and __setstate__ methods from BaseException and everything seems to just work. Pickling/unpickling works for all protocols whether or not you set self.args. Is this an appropriate solution? I'm not sure what the motivation for having these methods is. -Eric From status at bugs.python.org Sun Apr 1 02:00:55 2007 From: status at bugs.python.org (Tracker) Date: Sun, 1 Apr 2007 00:00:55 +0000 (UTC) Subject: [Python-Dev] Summary of Tracker Issues Message-ID: <20070401000055.55AFE78078@psf.localdomain> ACTIVITY SUMMARY (03/25/07 - 04/01/07) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1647 open ( +3) / 8584 closed ( +2) / 10231 total ( +5) Average duration of open issues: 751 days. Median duration of open issues: 700 days. Open Issues Breakdown open 1647 ( +3) pending 0 ( +0) Issues Created Or Reopened (5) ______________________________ Buy Tramadol online 03/25/07 CLOSED http://bugs.python.org/issue1023 created conor2k Testing bugs.python.org domain 03/26/07 CLOSED http://bugs.python.org/issue1024 created izak Test issue 03/26/07 http://bugs.python.org/issue1025 created loewis Test issue, to report@ 03/27/07 http://bugs.python.org/issue1026 created forsberg base64.urlsafe_b64encode() shouldn't use the = character 03/28/07 http://bugs.python.org/issue1027 created onlinestocktrading Issues Now Closed (2) _____________________ Buy Tramadol online 3 days http://bugs.python.org/issue1023 amk Testing bugs.python.org domain 0 days http://bugs.python.org/issue1024 izak -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070401/e66e986d/attachment.html From g.brandl at gmx.net Sun Apr 1 12:14:26 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 01 Apr 2007 12:14:26 +0200 Subject: [Python-Dev] Python 3000 PEP: Postfix type declarations Message-ID: PEP: XXX Title: Postfix type declarations Version: $Revision: $ Last-Modified: $Date: $ Author: Georg Brandl Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 01-Apr-2007 Python-Version: 3.0 Abstract ======== This PEP proposes the addition of a postfix type declaration syntax to Python. It also specifies a new ``typedef`` statement which is used to create new mappings between types and declarators. Its acceptance will greatly enhance the Python user experience as well as eliminate one of the warts that deter users of other programming languages from switching to Python. Rationale ========= Python has long suffered from the lack of explicit type declarations. Being one of the few aspects in which the language deviates from its Zen, this wart has sparked many a discussion between Python heretics and members of the PSU (for a few examples, see [EX1]_, [EX2]_ or [EX3]_), and it also made it a large-scale enterprise success unlikely. However, if one wants to put an end to this misery, a decent Pythonic syntax must be found. In almost all languages that have them, type declarations lack this quality: they are verbose, often needing *multiple words* for a single type, or they are hard to comprehend (e.g., a certain language uses completely unrelated [#]_ adjectives like ``dim`` for type declaration). Therefore, this PEP combines the move to type declarations with another bold move that will once again prove that Python is not only future-proof but future-embracing: the introduction of Unicode characters as an integral constituent of source code. Unicode makes it possible to express much more with much less characters, which is in accordance with the Zen ("Readability counts.") [ZEN]_. Additionally, it eliminates the need for a separate type declaration statement, and last but not least, it makes Python measure up to Perl 6, which already uses Unicode for its operators. [#]_ Specification ============= When the type declaration mode is in operation, the grammar is changed so that each ``NAME`` must consist of two parts: a name and a type declarator, which is exactly one Unicode character. The declarator uniquely specifies the type of the name, and if it occurs on the left hand side of an expression, this type is enforced: an ``InquisitionError`` exception is raised if the returned type doesn't match the declared type. [#]_ Also, function call result types have to be specified. If the result of the call does not have the declared type, an ``InquisitionError`` is raised. Caution: the declarator for the result should not be confused with the declarator for the function object (see the example below). Type declarators after names that are only read, not assigned to, are not strictly necessary but enforced anyway (see the Python Zen: "Explicit is better than implicit."). The mapping between types and declarators is not static. It can be completely customized by the programmer, but for convenience there are some predefined mappings for some built-in types: ========================= =================================================== Type Declarator ========================= =================================================== ``object`` ? (REPLACEMENT CHARACTER) ``int`` ? (DOUBLE-STRUCK CAPITAL N) ``float`` ? (ESTIMATED SYMBOL) ``bool`` ? (CHECK MARK) ``complex`` ? (DOUBLE-STRUCK CAPITAL C) ``str`` ? (LOWER RIGHT PENCIL) ``unicode`` ? (BLACK NIB) ``tuple`` ? (PARENTHESIZED LATIN SMALL LETTER T) ``list`` ? (HOT SPRINGS) ``dict`` ? (DOUBLE-ENDED MULTIMAP) ``set`` ? (EMPTY SET) (*Note:* this is also for full sets) ``frozenset`` ? (SNOWMAN) ``datetime`` ? (WATCH) ``function`` ? (LATIN SMALL LETTER LAMBDA WITH STROKE) ``generator`` ? (ATOM SYMBOL) ``Exception`` ? (ELECTRIC ARROW) ========================= =================================================== The declarator for the ``None`` type is a zero-width space. These characters should be obvious and easy to remember and type for every programmer. Unicode replacement units ========================= Since even in our modern, globalized world there are still some old-fashioned rebels who can't or don't want to use Unicode in their source code, and since Python is a forgiving language, a fallback is provided for those: Instead of the single Unicode character, they can type ``name${UNICODE NAME OF THE DECLARATOR}$``. For example, these two function definitions are equivalent:: def foo?(x?): return None and :: def foo${LATIN SMALL LETTER LAMBDA WITH STROKE}$(x${DOUBLE-STRUCK CAPITAL C}$): return None${ZERO WIDTH NO-BREAK SPACE}$ This is still easy to read and makes the full power of type-annotated Python available to ASCII believers. The ``typedef`` statement ========================= The mapping between types and declarators can be extended with this new statement. The syntax is as follows:: typedef_stmt ::= "typedef" expr DECLARATOR where ``expr`` resolves to a type object. For convenience, the ``typedef`` statement can also be mixed with the ``class`` statement for new classes, like so:: typedef class Foo?(object?): pass Example ======= This is the standard ``os.path.normpath`` function, converted to type declaration syntax:: def normpath?(path?)?: """Normalize path, eliminating double slashes, etc.""" if path? == '': return '.' initial_slashes? = path?.startswith?('/')? # POSIX allows one or two initial slashes, but treats three or more # as single slash. if (initial_slashes? and path?.startswith?('//')? and not path?.startswith?('///')?)?: initial_slashes? = 2 comps? = path?.split?('/')? new_comps? = []? for comp? in comps?: if comp? in ('', '.')?: continue if (comp? != '..' or (not initial_slashes? and not new_comps?)? or (new_comps? and new_comps?[-1]? == '..')?)?: new_comps?.append?(comp?) elif new_comps?: new_comps?.pop?()? comps? = new_comps? path? = '/'.join(comps?)? if initial_slashes?: path? = '/'*initial_slashes? + path? return path? or '.' As you can clearly see, the type declarations add expressiveness, while at the same time they make the code look much more professional. Compatibility issues ==================== To enable type declaration mode, one has to write:: from __future__ import type_declarations which enables Unicode parsing of the source [#]_, makes ``typedef`` a keyword and enforces correct types for all assignments and function calls. References ========== .. [EX1] http://mail.python.org/pipermail/python-list/2003-June/210588.html .. [EX2] http://mail.python.org/pipermail/python-list/2000-May/034685.html .. [EX3] http://groups.google.com/group/comp.lang.python/browse_frm/thread/6ae8c6add913635a/de40d4ffe9bd4304?lnk=gst&q=type+declarations&rnum=6 .. [#] Though, if you know the language in question, it may not be *that* unrelated. .. [ZEN] http://www.python.org/dev/peps/pep-0020/ .. [#] Well, it would, if there was a Perl 6. .. [#] Since the name ``TypeError`` is already in use, this name has been chosen for obvious reasons. .. [#] The encoding in which the code is written is read from a standard coding cookie. There will also be an autodetection mechanism, invoked by ``from __future__ import encoding_hell``. Acknowledgements ================ Many thanks go to Armin Ronacher, Alexander Schremmer and Marek Kubica who helped find the most suitable and mnemonic declarator for built-in types. Thanks also to the Unicode Consortium for including all those useful characters in the Unicode standard. Copyright ========= This document has been placed in the public domain. .. Local Variables: coding: utf-8 mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: From johann at rocholl.net Sun Apr 1 12:52:09 2007 From: johann at rocholl.net (Johann C. Rocholl) Date: Sun, 1 Apr 2007 12:52:09 +0200 Subject: [Python-Dev] Python 3000 PEP: Postfix type declarations In-Reply-To: References: Message-ID: <8233478f0704010352q342e365eg467a1daa44c89766@mail.gmail.com> Brilliant! On 4/1/07, Georg Brandl wrote: > def foo${LATIN SMALL LETTER LAMBDA WITH STROKE}$(x${DOUBLE-STRUCK CAPITAL C}$): > return None${ZERO WIDTH NO-BREAK SPACE}$ > > This is still easy to read and makes the full power of type-annotated Python > available to ASCII believers. +1 J From gjcarneiro at gmail.com Sun Apr 1 14:41:56 2007 From: gjcarneiro at gmail.com (Gustavo Carneiro) Date: Sun, 1 Apr 2007 13:41:56 +0100 Subject: [Python-Dev] Python 3000 PEP: Postfix type declarations In-Reply-To: References: Message-ID: On 4/1/07, Georg Brandl wrote: [...] > Example > ======= > > This is the standard ``os.path.normpath`` function, converted to type > declaration > syntax:: > > def normpath?(path?)?: > """Normalize path, eliminating double slashes, etc.""" > if path? == '': > return '.' > initial_slashes? = path?.startswith?('/')? > # POSIX allows one or two initial slashes, but treats three or > more > # as single slash. > if (initial_slashes? and > path?.startswith?('//')? and not path?.startswith?('///')?)?: > initial_slashes? = 2 > comps? = path?.split?('/')? > new_comps? = []? > for comp? in comps?: > if comp? in ('', '.')?: > continue > if (comp? != '..' or (not initial_slashes? and not > new_comps?)? or > (new_comps? and new_comps?[-1]? == '..')?)?: > new_comps?.append?(comp?) > elif new_comps?: > new_comps?.pop?()? > comps? = new_comps? > path? = '/'.join(comps?)? > if initial_slashes?: > path? = '/'*initial_slashes? + path? > return path? or '.' > > As you can clearly see, the type declarations add expressiveness, while at > the > same time they make the code look much more professional. Is this supposed to be a joke? Please tell me this isn't a real PEP. While I'm all for allowing unicode identifiers in Python, postfix type annotations make Python look like Perl. And how can you claim this code is more readable? It certainly is _less_ readable, not more. I agree that Python should support type annotations, but they should be optional and only present at the function interfaces, i.e. specify the type in the function parameter lists, like in plain old C. +1 from me for allowing unicode identifiers. -MAXVOTE for type annotations in identifiers. -- Gustavo J. A. M. Carneiro "The universe is always one step beyond logic." -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070401/e35ebbfb/attachment.html From gjcarneiro at gmail.com Sun Apr 1 14:47:39 2007 From: gjcarneiro at gmail.com (Gustavo Carneiro) Date: Sun, 1 Apr 2007 13:47:39 +0100 Subject: [Python-Dev] Python 3000 PEP: Postfix type declarations In-Reply-To: References: Message-ID: On 4/1/07, Gustavo Carneiro wrote: > > On 4/1/07, Georg Brandl wrote: > [...] > > > Example > > ======= > > > > This is the standard ``os.path.normpath`` function, converted to type > > declaration > > syntax:: > > > > def normpath?(path?)?: > > """Normalize path, eliminating double slashes, etc.""" > > if path? == '': > > return '.' > > initial_slashes? = path?.startswith?('/')? > > # POSIX allows one or two initial slashes, but treats three or > > more > > # as single slash. > > if (initial_slashes? and > > path?.startswith?('//')? and not > > path?.startswith?('///')?)?: > > initial_slashes? = 2 > > comps? = path?.split?('/')? > > new_comps? = []? > > for comp? in comps?: > > if comp? in ('', '.')?: > > continue > > if (comp? != '..' or (not initial_slashes? and not > > new_comps?)? or > > (new_comps? and new_comps?[-1]? == '..')?)?: > > new_comps?.append?(comp?) > > elif new_comps?: > > new_comps?.pop?()? > > comps? = new_comps? > > path? = '/'.join(comps?)? > > if initial_slashes?: > > path? = '/'*initial_slashes? + path? > > return path? or '.' > > > > As you can clearly see, the type declarations add expressiveness, while > > at the > > same time they make the code look much more professional. > > > Is this supposed to be a joke? /me ashamed for not having noticed the date of this PEP... :P -- Gustavo J. A. M. Carneiro "The universe is always one step beyond logic." -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070401/52f10a0c/attachment.html From rrr at ronadam.com Sun Apr 1 15:27:00 2007 From: rrr at ronadam.com (Ron Adam) Date: Sun, 01 Apr 2007 08:27:00 -0500 Subject: [Python-Dev] Python 3000 PEP: Postfix type declarations In-Reply-To: <8233478f0704010352q342e365eg467a1daa44c89766@mail.gmail.com> References: <8233478f0704010352q342e365eg467a1daa44c89766@mail.gmail.com> Message-ID: <460FB324.5050601@ronadam.com> Johann C. Rocholl wrote: > Brilliant! > > On 4/1/07, Georg Brandl wrote: >> def foo${LATIN SMALL LETTER LAMBDA WITH STROKE}$(x${DOUBLE-STRUCK CAPITAL C}$): >> return None${ZERO WIDTH NO-BREAK SPACE}$ >> >> This is still easy to read and makes the full power of type-annotated Python >> available to ASCII believers. > > +1 > > J This also has the advantage that any good editor with auto correct and completion can convert from one form to the other as you type. A feature I love in my word processor because it *always* does *exactly* what I want and saves me much typing. +1? ;-) From list-ener at strank.info Sun Apr 1 15:39:00 2007 From: list-ener at strank.info (Stefan Rank) Date: Sun, 01 Apr 2007 15:39:00 +0200 Subject: [Python-Dev] proposed which.py replacement In-Reply-To: References: <460EC367.8090007@ncee.net> Message-ID: <460FB5F4.6020502@strank.info> on 31.03.2007 22:39 Guido van Rossum said the following: > If you ask me, having it hosted by Trent is probably more helpful for > its popularity than putting it in the Python source distro; the Tools > directory is mostly a poorly-maintained collection of trivia I wrote > many years ago that is now quietly gathering dust. Some time ago, I posted a `feature request`_ about which.py including the proposal to put it into the std-lib as ``which`` or ``os.which`` to allow programmatic use and:: python -m which ... This should take care of the visibility problem. ;-) However, there are several todos_, including tests and docs, before this can even be considered. I am afraid I did not have any time to work on it yet. cheers, stefan .. _feature request: http://sourceforge.net/tracker/index.php?func=detail&aid=1509798&group_id=5470&atid=355470 .. _todos: http://trentm.com/projects/which/TODO.txt > (Not all of it, of course; there's some useful stuff there that I > *didn't* write, which ended up there because it is either *used* by > the distro (e.g. the compiler package support) or because the author > needed a channel that guaranteed open source status (e.g. world and > pynche). But Trent's which.py doesn't seem to fall in either > category.) > > --Guido > > On 3/31/07, Shane Geiger wrote: >> Trent Mick has a module called which.py that might make a nice >> platform-independent replacement for python2.5/Tools/scripts/which.py. >> >> http://www.trentm.com/projects/which/ >> >> Why which.py? >> >> |which.py| is a small GNU-which replacement. It has the following features: >> >> * it is portable (Windows, Linux, Mac OS X, Un*x); >> * it understands PATHEXT and "App Paths" registration on Windows >> (i.e. it will find everything that |start| does from the command >> shell); >> * it can print all matches on the PATH; >> * it can note "near misses" on the PATH (e.g. files that match but >> may not, say, have execute permissions); and >> * it can be used as a Python module. >> >> I also would be happy to have this be a replacement for the |which.py| >> in the Python CVS tree at |dist/src/Tools/scripts/which.py| which is >> Unix-specific and not usable as a module; and perhaps for inclusion in >> the stdlib. From ziga.seilnacht at gmail.com Sun Apr 1 15:51:51 2007 From: ziga.seilnacht at gmail.com (=?windows-1252?Q?=8Eiga_Seilnacht?=) Date: Sun, 01 Apr 2007 15:51:51 +0200 Subject: [Python-Dev] BaseException pickle issue In-Reply-To: References: Message-ID: <460FB8F7.1010004@gmail.com> Eric Huss wrote: > Some subclasses of Exception are no longer pickleable in Python 2.5. An > example: > [snip] > > Does anyone have any thoughts about this? Is it a bug? > > I can imagine one could argue that exceptions should call the base > __init__ method to properly set args, but there are so many exceptions out > there that do not do this that it would be very difficult to track them > all down. > > I removed the __reduce__ and __setstate__ methods from BaseException and > everything seems to just work. Pickling/unpickling works for all > protocols whether or not you set self.args. Is this an appropriate > solution? I'm not sure what the motivation for having these methods is. > I think that this is a bug, but removing those methods is not the right solution. The __reduce__ method is needed because builtin exceptions don't store their attributes in the __dict__ anymore; if you remove it, then those attributes will be lost during pickling. The __setstate__ method was added to help with unpickling old exceptions, which did store all their attributes in the __dict__. See this patch for details: http://www.python.org/sf/1498571 A better solution would be to move the initial args attribute assignment to BaseException.__new__. See this patch: http://www.python.org/sf/1692335 Could you check if that fixes your problem? Ziga From guido at python.org Sun Apr 1 17:23:24 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 1 Apr 2007 08:23:24 -0700 Subject: [Python-Dev] proposed which.py replacement In-Reply-To: <460FB5F4.6020502@strank.info> References: <460EC367.8090007@ncee.net> <460FB5F4.6020502@strank.info> Message-ID: It's out of character for the standard library, since (regardless of whether it's implemented in Python or part of the standard library) it's a stand-alone utility. I don't see much use for this as a library module. On 4/1/07, Stefan Rank wrote: > on 31.03.2007 22:39 Guido van Rossum said the following: > > If you ask me, having it hosted by Trent is probably more helpful for > > its popularity than putting it in the Python source distro; the Tools > > directory is mostly a poorly-maintained collection of trivia I wrote > > many years ago that is now quietly gathering dust. > > Some time ago, I posted a `feature request`_ about which.py including > the proposal to put it into the std-lib as > ``which`` or ``os.which`` to allow programmatic use and:: > > python -m which ... > > This should take care of the visibility problem. ;-) > However, there are several todos_, including tests and docs, before this > can even be considered. > > I am afraid I did not have any time to work on it yet. > > cheers, > stefan > > > .. _feature request: > http://sourceforge.net/tracker/index.php?func=detail&aid=1509798&group_id=5470&atid=355470 > .. _todos: http://trentm.com/projects/which/TODO.txt > > > > (Not all of it, of course; there's some useful stuff there that I > > *didn't* write, which ended up there because it is either *used* by > > the distro (e.g. the compiler package support) or because the author > > needed a channel that guaranteed open source status (e.g. world and > > pynche). But Trent's which.py doesn't seem to fall in either > > category.) > > > > --Guido > > > > On 3/31/07, Shane Geiger wrote: > >> Trent Mick has a module called which.py that might make a nice > >> platform-independent replacement for python2.5/Tools/scripts/which.py. > >> > >> http://www.trentm.com/projects/which/ > >> > >> Why which.py? > >> > >> |which.py| is a small GNU-which replacement. It has the following features: > >> > >> * it is portable (Windows, Linux, Mac OS X, Un*x); > >> * it understands PATHEXT and "App Paths" registration on Windows > >> (i.e. it will find everything that |start| does from the command > >> shell); > >> * it can print all matches on the PATH; > >> * it can note "near misses" on the PATH (e.g. files that match but > >> may not, say, have execute permissions); and > >> * it can be used as a Python module. > >> > >> I also would be happy to have this be a replacement for the |which.py| > >> in the Python CVS tree at |dist/src/Tools/scripts/which.py| which is > >> Unix-specific and not usable as a module; and perhaps for inclusion in > >> the stdlib. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From xah at xahlee.org Sun Apr 1 17:39:04 2007 From: xah at xahlee.org (xah lee) Date: Sun, 1 Apr 2007 08:39:04 -0700 Subject: [Python-Dev] Python Documentation Problem Example Message-ID: <7CA42E72-A937-4002-88C6-FA4ECF8931EC@xahlee.org> Python Doc Problem Example: os.path.split() Xah Lee, 20050918 Quote from: http://www.python.org/doc/2.4.1/lib/module-os.path.html ? split(path) Split the pathname path into a pair, (head, tail) where tail is the last pathname component and head is everything leading up to that. The tail part will never contain a slash; if path ends in a slash, tail will be empty. If there is no slash in path, head will be empty. If path is empty, both head and tail are empty. Trailing slashes are stripped from head unless it is the root (one or more slashes only). In nearly all cases, join(head, tail) equals path (the only exception being when there were multiple slashes separating head from tail). ? This confusive verbiage is a result of the author's pretention in a austere style and his failure to think clearly before writing. Suggested rewrite: ? split(path) returns a pair (dirname,filename), where dirname is the part of path up to the last slash, and filename is the rest of the string after the last slash. Exceptional cases are: ? if path is a single slash (or repeated), then path == dirname and filename is empty. ? If the last slash is repeated, they are treated as one single slash. ? I was working on a program where i needed to split a path into dirname, corename, and suffix. I came to this page and took me a while to understand what split() is about. There are other path related functions splitext(), splitdrive (), basename(), dirname(). User has to scan the whole page and read painfully each one to fully understand how to choose and use them for the task at hand. As i have explained before (see references at bottom), documentation should be organized oriented towards programer's tasks, not alphabetically, compiler view, or computer sciency scheme. On this os.path module, split(), splittext(), dirname(), basename() should all be under one section. This way, their usefulness and each's fitness becomes clearer, and also easier to document as a collective. Other functions that test files or get info about files should be grouped together. Don't be afraid of having functions that won't fit into some grouping scheme. For exapmle, the walk() and supports_unicode_filenames() can be lumped at the bottom as Other. The need to present materials in some aloof, computer sciency, academic, technically precise way is a major usability problem of the Python doc. (the Pythoners's need to present materials in a formal style is a backlash of the happy-go-lucky sloppiness of unix/perl community. However, they being pretty much the same crowd without significant critical thinking and writing skills, cannot do better by hiding in formality.) Also, at the top we see: Warning: On Windows, many of these functions do not properly support UNC pathnames. splitunc() and ismount() do handle them correctly. As indicated before, this is a exhibition of tech geeking and jargonizing. If this warning is necessary, place it at the bottom of the page as a footnote. Also, spell out UNC, and provide a link to its proper spec. Tech geekers are very pretentious and cryptic in their tech docs. They are afraid, as if spelling out UNC would make them unprofessional, that their peers would deem them inferior. There are a myriad of technical standards that any programer could only be familiar with a fraction, confined to his area of expertise. Standards and its acronyms come and go, and each with varying degrees of precision, actual relevance, and they are intermingled with de facto practices in the commercial world that may not even have official names. The tech geekers are clouded by their tech-expertise. The purpose of documentation is not some cold academic presentation. Vast majority who came to use os.path wouldn't know what UNC is nor do they need to know. Spell things out when in doubt. UNC here, isn't really a significant ?standard?. This warning should be left out. Suggestions: * Regroup path manipulating functions together, file info functions together, etc. * After regrouping the functions, much of the writing can be simplified since these functions are doing similar tasks. Writing can focus on their differences. Rewrite them with a focus on what users needs to do. Add examples in intricate situations. Avoid the computer science thesis writing style. * Remove the UNC warning at the top. ----- Here are a few other Python doc problems i've written in the past years: Python doc problem example: os.system() http://xahlee.org/perl-python/python_doc_os.html Python Doc Problem Example: os.path.split() http://xahlee.org/perl-python/python_doc_os_path_split.html Python Doc Problem Example: sort() http://xahlee.org/perl-python/python_doc_sort.html Python Doc Problem Example: gzip module http://xahlee.org/perl-python/python_doc_gzip.html Xah xah at xahlee.org ? http://xahlee.org/ ? From collinw at gmail.com Sun Apr 1 18:49:51 2007 From: collinw at gmail.com (Collin Winter) Date: Sun, 1 Apr 2007 11:49:51 -0500 Subject: [Python-Dev] Python 3000 PEP: Postfix type declarations In-Reply-To: References: Message-ID: <43aa6ff70704010949v5d38ccb2pa2c183594124267c@mail.gmail.com> On 4/1/07, Georg Brandl wrote: [snip several pages of excellent ideas] > > The mapping between types and declarators is not static. It can be completely > customized by the programmer, but for convenience there are some predefined > mappings for some built-in types: > > ========================= =================================================== > Type Declarator > ========================= =================================================== > ``object`` ? (REPLACEMENT CHARACTER) > ``int`` ? (DOUBLE-STRUCK CAPITAL N) > ``float`` ? (ESTIMATED SYMBOL) > ``bool`` ? (CHECK MARK) > ``complex`` ? (DOUBLE-STRUCK CAPITAL C) > ``str`` ? (LOWER RIGHT PENCIL) > ``unicode`` ? (BLACK NIB) > ``tuple`` ? (PARENTHESIZED LATIN SMALL LETTER T) > ``list`` ? (HOT SPRINGS) > ``dict`` ? (DOUBLE-ENDED MULTIMAP) > ``set`` ? (EMPTY SET) (*Note:* this is also for full sets) > ``frozenset`` ? (SNOWMAN) > ``datetime`` ? (WATCH) > ``function`` ? (LATIN SMALL LETTER LAMBDA WITH STROKE) > ``generator`` ? (ATOM SYMBOL) > ``Exception`` ? (ELECTRIC ARROW) > ========================= =================================================== > > The declarator for the ``None`` type is a zero-width space. > > These characters should be obvious and easy to remember and type for every > programmer. > [snip] > > Example > ======= > > This is the standard ``os.path.normpath`` function, converted to type declaration > syntax:: > > def normpath?(path?)?: > """Normalize path, eliminating double slashes, etc.""" > if path? == '': > return '.' > initial_slashes? = path?.startswith?('/')? > # POSIX allows one or two initial slashes, but treats three or more > # as single slash. > if (initial_slashes? and > path?.startswith?('//')? and not path?.startswith?('///')?)?: > initial_slashes? = 2 > comps? = path?.split?('/')? > new_comps? = []? > for comp? in comps?: > if comp? in ('', '.')?: > continue > if (comp? != '..' or (not initial_slashes? and not new_comps?)? or > (new_comps? and new_comps?[-1]? == '..')?)?: > new_comps?.append?(comp?) > elif new_comps?: > new_comps?.pop?()? > comps? = new_comps? > path? = '/'.join(comps?)? > if initial_slashes?: > path? = '/'*initial_slashes? + path? > return path? or '.' > > As you can clearly see, the type declarations add expressiveness, while at the > same time they make the code look much more professional. My only concern is that this doesn't go far enough. While knowing that some object is a ? is a good start, it would be so much more helpful to know that it's a ? of ?s. I think something like ???3? to indicate a 3-? of ?s would be nice. This would change the line in the above from "if comp? in ('', '.')?:" to "if comp? in ('', '.')??2?:", which I think is a nice win in terms of readability, EIBTI and all that. (Sidebar: I think the PEP should feature a section on how these new type declarations will cut down on mailing list volume and documentation size.) In light of this PEP, PEP 3107's function annotations should be rejected. All that hippie feel-good crap about "user-defined annotations" and "open-ended semantics" and "no rules, man" was just going to get us into trouble. This PEP's more modern conception of type annotations give the language a power and expressiveness that my PEP could never hope to match. This is clearly a move in the right direction. +4 billion. Collin Winter From martin at v.loewis.de Sun Apr 1 19:27:28 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 01 Apr 2007 19:27:28 +0200 Subject: [Python-Dev] Get 2.5 changes in now, branch will be frozen soon In-Reply-To: <007901c77408$19f697e0$6601a8c0@RaymondLaptop1> References: <7a9c25c20703310204gcd9e061m19971a743133fc38@mail.gmail.com> <007901c77408$19f697e0$6601a8c0@RaymondLaptop1> Message-ID: <460FEB80.5070800@v.loewis.de> Raymond Hettinger schrieb: > [Stephen Hansen= >> I just wanted to offer a gentle prod to see if a decision can be made; >> if any decision requires an adjustment to patches, tests and documentation, >> I'm willing to do them. > > We should get a pronouncement on this or else whatever goes out in Py2.5.1 > becomes the de-facto decision and becomes much harder to change. Changing 2.5.x was not ever proposed in the entire thread (up to now); the existing change is for 2.6 only. Regards, Martin From list-ener at strank.info Sun Apr 1 18:41:54 2007 From: list-ener at strank.info (Stefan Rank) Date: Sun, 01 Apr 2007 18:41:54 +0200 Subject: [Python-Dev] proposed which.py replacement In-Reply-To: References: <460EC367.8090007@ncee.net> <460FB5F4.6020502@strank.info> Message-ID: <460FE0D2.1090507@strank.info> on 01.04.2007 17:23 Guido van Rossum said the following: > It's out of character for the standard library, since (regardless of > whether it's implemented in Python or part of the standard library) > it's a stand-alone utility. I don't see much use for this as a library > module. I use it as a library, because it encodes knowledge about locating executables on different platforms, especially Windows. Unixoids have which and the search is relatively straightforward. Windows searches paths in PATH and in the registry, and uses PATHEXT, so, for me, the main benefit of which.py is that it provides a which replacement on Windows that takes these quirks into account. A small use case, but a use case nevertheless. I never use which.py as a stand-alone utility as I have cygwin. (But I think it would be helpful when deploying on somebody else's Windows computer.) Of course it is your call if it fits in the stdlib or not. cheers, stefan From skip at pobox.com Sun Apr 1 19:56:50 2007 From: skip at pobox.com (skip at pobox.com) Date: Sun, 1 Apr 2007 12:56:50 -0500 Subject: [Python-Dev] proposed which.py replacement In-Reply-To: <460FE0D2.1090507@strank.info> References: <460EC367.8090007@ncee.net> <460FB5F4.6020502@strank.info> <460FE0D2.1090507@strank.info> Message-ID: <17935.62050.140610.64935@montanaro.dyndns.org> Stefan> I use it as a library, because it encodes knowledge about Stefan> locating executables on different platforms, especially Windows. Stefan> Unixoids have which and the search is relatively Stefan> straightforward. Windows searches paths in PATH and in the Stefan> registry, and uses PATHEXT, so, for me, the main benefit of Stefan> which.py is that it provides a which replacement on Windows that Stefan> takes these quirks into account. I'll second that. In SpamBayes we not so long ago added the ability to run OCR software over images to try and identify image-based spam. Needless to say, we had to write fairly different bits of code on Unix v. Windows to locate the gocr or ocrad executables. Having something like a platform-independent which available in Python would have made this aspect of the code easier to write. Skip From ehuss at ironport.com Sun Apr 1 21:07:23 2007 From: ehuss at ironport.com (Eric Huss) Date: Sun, 1 Apr 2007 12:07:23 -0700 (PDT) Subject: [Python-Dev] BaseException pickle issue In-Reply-To: <460FB8F7.1010004@gmail.com> References: <460FB8F7.1010004@gmail.com> Message-ID: > I think that this is a bug, but removing those methods is not the right > solution. The __reduce__ method is needed because builtin exceptions > don't store their attributes in the __dict__ anymore; if you remove it, > then those attributes will be lost during pickling. The __setstate__ > method was added to help with unpickling old exceptions, which did store > all their attributes in the __dict__. See this patch for details: > > http://www.python.org/sf/1498571 > > A better solution would be to move the initial args attribute assignment > to BaseException.__new__. See this patch: > > http://www.python.org/sf/1692335 > > Could you check if that fixes your problem? Thanks, it certainly does! I wrote a very extensive unittest to try out various permutations. I'm not sure if there's an expectation that pickled strings of user-defined exceptions from previous versions work in newer versions. If you define your own __init__ method, and try to load a pickled string from 2.4, I still get a TypeError when trying to unpickle it. I will send you my test file directly. This isn't a problem for me since we don't store pickled exceptions. -Eric From guido at python.org Sun Apr 1 22:00:45 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 1 Apr 2007 13:00:45 -0700 Subject: [Python-Dev] proposed which.py replacement In-Reply-To: <460FE0D2.1090507@strank.info> References: <460EC367.8090007@ncee.net> <460FB5F4.6020502@strank.info> <460FE0D2.1090507@strank.info> Message-ID: Ok, sorry, that *is* a valid use case. Since Trent on his webpage suggests his code as a stdlib replacement I support this now. (But I'm not going to take further action -- I hope you can get one of the regular 2.6 dev guys to adopt this case. Also, the PSF needs to have a contribution form on file from Trent Mick.) --Guido On 4/1/07, Stefan Rank wrote: > on 01.04.2007 17:23 Guido van Rossum said the following: > > It's out of character for the standard library, since (regardless of > > whether it's implemented in Python or part of the standard library) > > it's a stand-alone utility. I don't see much use for this as a library > > module. > > I use it as a library, because it encodes knowledge about locating > executables on different platforms, especially Windows. > Unixoids have which and the search is relatively straightforward. > Windows searches paths in PATH and in the registry, and uses PATHEXT, > so, for me, the main benefit of which.py is that it provides a which > replacement on Windows that takes these quirks into account. > > A small use case, but a use case nevertheless. > > I never use which.py as a stand-alone utility as I have cygwin. > (But I think it would be helpful when deploying on somebody else's > Windows computer.) > > Of course it is your call if it fits in the stdlib or not. > > cheers, > stefan > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sun Apr 1 22:14:57 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 1 Apr 2007 13:14:57 -0700 Subject: [Python-Dev] Python 3000 PEP: Postfix type declarations In-Reply-To: <43aa6ff70704010949v5d38ccb2pa2c183594124267c@mail.gmail.com> References: <43aa6ff70704010949v5d38ccb2pa2c183594124267c@mail.gmail.com> Message-ID: +18446744073709551616 from me too. This also fits nicely in with my plan to abandon the python-dev and python-3000 mailing lists. Mailing lists are so 20th century! I propose that from now on, all Python development should be carried out on blogs, so that readers can use customized RSS feeds to read only those contributions they are interested in. I note that all the key developers already have a blog, e.g.: Aahz - http://www.artima.com/weblogs/index.jsp?blogger=aahz Neal Norwitz - http://nnorwitz.blogspot.com/ Fredrik Lundh - http://effbot.org/pyref/blog.htm Jeremy Hylton - http://www.python.org/~jeremy/weblog/ Anthony Baxter - http://codingweasel.blogspot.com/ Phillip Eby - http://dirtsimple.org/programming/index.html Talin - http://www.advogato.org/person/Talin/diary.html David Ascher - http://ascher.ca/blog/ Fred Drake - http://www.advogato.org/person/fdrake/diary.html (and myself, of course - http://www.artima.com/weblogs/index.jsp?blogger=guido) --Guido On 4/1/07, Collin Winter wrote: > On 4/1/07, Georg Brandl wrote: > [snip several pages of excellent ideas] > > > > The mapping between types and declarators is not static. It can be completely > > customized by the programmer, but for convenience there are some predefined > > mappings for some built-in types: > > > > ========================= =================================================== > > Type Declarator > > ========================= =================================================== > > ``object`` ? (REPLACEMENT CHARACTER) > > ``int`` ? (DOUBLE-STRUCK CAPITAL N) > > ``float`` ? (ESTIMATED SYMBOL) > > ``bool`` ? (CHECK MARK) > > ``complex`` ? (DOUBLE-STRUCK CAPITAL C) > > ``str`` ? (LOWER RIGHT PENCIL) > > ``unicode`` ? (BLACK NIB) > > ``tuple`` ? (PARENTHESIZED LATIN SMALL LETTER T) > > ``list`` ? (HOT SPRINGS) > > ``dict`` ? (DOUBLE-ENDED MULTIMAP) > > ``set`` ? (EMPTY SET) (*Note:* this is also for full sets) > > ``frozenset`` ? (SNOWMAN) > > ``datetime`` ? (WATCH) > > ``function`` ? (LATIN SMALL LETTER LAMBDA WITH STROKE) > > ``generator`` ? (ATOM SYMBOL) > > ``Exception`` ? (ELECTRIC ARROW) > > ========================= =================================================== > > > > The declarator for the ``None`` type is a zero-width space. > > > > These characters should be obvious and easy to remember and type for every > > programmer. > > > [snip] > > > > Example > > ======= > > > > This is the standard ``os.path.normpath`` function, converted to type declaration > > syntax:: > > > > def normpath?(path?)?: > > """Normalize path, eliminating double slashes, etc.""" > > if path? == '': > > return '.' > > initial_slashes? = path?.startswith?('/')? > > # POSIX allows one or two initial slashes, but treats three or more > > # as single slash. > > if (initial_slashes? and > > path?.startswith?('//')? and not path?.startswith?('///')?)?: > > initial_slashes? = 2 > > comps? = path?.split?('/')? > > new_comps? = []? > > for comp? in comps?: > > if comp? in ('', '.')?: > > continue > > if (comp? != '..' or (not initial_slashes? and not new_comps?)? or > > (new_comps? and new_comps?[-1]? == '..')?)?: > > new_comps?.append?(comp?) > > elif new_comps?: > > new_comps?.pop?()? > > comps? = new_comps? > > path? = '/'.join(comps?)? > > if initial_slashes?: > > path? = '/'*initial_slashes? + path? > > return path? or '.' > > > > As you can clearly see, the type declarations add expressiveness, while at the > > same time they make the code look much more professional. > > My only concern is that this doesn't go far enough. While knowing that > some object is a ? is a good start, it would be so much more helpful > to know that it's a ? of ?s. I think something like ???3? to indicate > a 3-? of ?s would be nice. This would change the line in the above > from "if comp? in ('', '.')?:" to "if comp? in ('', '.')??2?:", which > I think is a nice win in terms of readability, EIBTI and all that. > > (Sidebar: I think the PEP should feature a section on how these new > type declarations will cut down on mailing list volume and > documentation size.) > > In light of this PEP, PEP 3107's function annotations should be > rejected. All that hippie feel-good crap about "user-defined > annotations" and "open-ended semantics" and "no rules, man" was just > going to get us into trouble. This PEP's more modern conception of > type annotations give the language a power and expressiveness that my > PEP could never hope to match. > > This is clearly a move in the right direction. +4 billion. > > Collin Winter > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From sgeiger at ncee.net Sun Apr 1 23:30:04 2007 From: sgeiger at ncee.net (Shane Geiger) Date: Sun, 01 Apr 2007 16:30:04 -0500 Subject: [Python-Dev] Python+XUL Message-ID: <4610245C.4080001@ncee.net> Has anyone heard the Python+XUL community cry "I'm not dead yet!" or are they really dead? I haven't seen mentions of new work in these areas lately. XUL in general seems to have died (so many broken links on XUL pages). Was this just a fad? If not, and if there's some really useful of it (especially if in Python) please point it out. -- Shane Geiger IT Director National Council on Economic Education sgeiger at ncee.net | 402-438-8958 | http://www.ncee.net Leading the Campaign for Economic and Financial Literacy -------------- next part -------------- A non-text attachment was scrubbed... Name: sgeiger.vcf Type: text/x-vcard Size: 297 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20070401/43357d78/attachment.vcf From mhammond at skippinet.com.au Mon Apr 2 01:37:08 2007 From: mhammond at skippinet.com.au (Mark Hammond) Date: Mon, 2 Apr 2007 09:37:08 +1000 Subject: [Python-Dev] Python+XUL In-Reply-To: <4610245C.4080001@ncee.net> Message-ID: <05e701c774b6$aa669d30$0200a8c0@enfoldsystems.local> > Has anyone heard the Python+XUL community cry "I'm not dead > yet!" or are > they really dead? I haven't seen mentions of new work in these areas > lately. XUL in general seems to have died (so many broken > links on XUL > pages). Was this just a fad? If not, and if there's some > really useful > of it (especially if in Python) please point it out. This work is still ongoing. The primary docs are at http://developer.mozilla.org/en/docs/PyXPCOM, and the #pyxpcom channel on irc.mozilla.org does sometimes see some action. The mozilla trunk has Python in XUL working, and is being used by at least a few people. The ongoing work is fairly sporadic, but is currently aimed at closer integration of Javascript and Python with the DOM - notably enabling "native" attributes on DOM objects to be accessed by other languages. Mark From trentm at activestate.com Mon Apr 2 18:40:03 2007 From: trentm at activestate.com (Trent Mick) Date: Mon, 02 Apr 2007 09:40:03 -0700 Subject: [Python-Dev] proposed which.py replacement In-Reply-To: References: <460EC367.8090007@ncee.net> Message-ID: <461131E3.8090305@activestate.com> Guido van Rossum wrote: > If you ask me, having it hosted by Trent is probably more helpful for > its popularity than putting it in the Python source distro; the Tools > directory is mostly a poorly-maintained collection of trivia I wrote > many years ago that is now quietly gathering dust. > > (Not all of it, of course; there's some useful stuff there that I > *didn't* write, which ended up there because it is either *used* by > the distro (e.g. the compiler package support) or because the author > needed a channel that guaranteed open source status (e.g. world and > pynche). But Trent's which.py doesn't seem to fall in either > category.) Agreed. However, I think my which.py might be a good candidate for the stdlib (for Py2.6) as per python.org/sf/1509798 and could put together a patch (along with test suite integrate and docs) if others think it worthwhile. Trent -- Trent Mick trentm at activestate.com From guido at python.org Mon Apr 2 19:18:18 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 2 Apr 2007 10:18:18 -0700 Subject: [Python-Dev] proposed which.py replacement In-Reply-To: <461131E3.8090305@activestate.com> References: <460EC367.8090007@ncee.net> <461131E3.8090305@activestate.com> Message-ID: I think it's worthwhile. On 4/2/07, Trent Mick wrote: > Guido van Rossum wrote: > > If you ask me, having it hosted by Trent is probably more helpful for > > its popularity than putting it in the Python source distro; the Tools > > directory is mostly a poorly-maintained collection of trivia I wrote > > many years ago that is now quietly gathering dust. > > > > (Not all of it, of course; there's some useful stuff there that I > > *didn't* write, which ended up there because it is either *used* by > > the distro (e.g. the compiler package support) or because the author > > needed a channel that guaranteed open source status (e.g. world and > > pynche). But Trent's which.py doesn't seem to fall in either > > category.) > > Agreed. However, I think my which.py might be a good candidate for the > stdlib (for Py2.6) as per python.org/sf/1509798 and could put together a > patch (along with test suite integrate and docs) if others think it > worthwhile. > > Trent > > -- > Trent Mick > trentm at activestate.com > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From skip at pobox.com Mon Apr 2 19:31:39 2007 From: skip at pobox.com (skip at pobox.com) Date: Mon, 2 Apr 2007 12:31:39 -0500 Subject: [Python-Dev] proposed which.py replacement In-Reply-To: <461131E3.8090305@activestate.com> References: <460EC367.8090007@ncee.net> <461131E3.8090305@activestate.com> Message-ID: <17937.15867.331700.294327@montanaro.dyndns.org> Trent> Agreed. However, I think my which.py might be a good candidate Trent> for the stdlib (for Py2.6) as per python.org/sf/1509798 and could Trent> put together a patch (along with test suite integrate and docs) Trent> if others think it worthwhile. +1. Since I run with 2.6 at home and run SpamBayes from source I can tweak my SpamBayes code to use it as well. I'm now monitoring the above tracker id and will try to massage things once there's a suitable patch uploaded. Skip From greg.ewing at canterbury.ac.nz Mon Apr 2 23:19:34 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 03 Apr 2007 09:19:34 +1200 Subject: [Python-Dev] Quoting netiquette reminder [Re: proposed which.py replacement] In-Reply-To: References: <460EC367.8090007@ncee.net> <461131E3.8090305@activestate.com> Message-ID: <46117366.4090005@canterbury.ac.nz> Guido van Rossum wrote: > I think it's worthwhile. (followed by a whole page of quoted text) Folks, could you please try to remember to quote minimally from messages that you are replying to? -- Greg From guido at python.org Tue Apr 3 02:11:17 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 2 Apr 2007 17:11:17 -0700 Subject: [Python-Dev] test_socketserver flakey? Message-ID: The test_socketserver unittest seems to be failing occasionally for me. (Svn HEAD, Ubuntu dapper.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From facundo at taniquetil.com.ar Tue Apr 3 03:21:20 2007 From: facundo at taniquetil.com.ar (Facundo Batista) Date: Tue, 3 Apr 2007 01:21:20 +0000 (UTC) Subject: [Python-Dev] test_socketserver flakey? References: Message-ID: Guido van Rossum wrote: > The test_socketserver unittest seems to be failing occasionally for > me. (Svn HEAD, Ubuntu dapper.) I have Ubuntu Edgy, will take a look at it... Regards, -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From facundo at taniquetil.com.ar Tue Apr 3 03:28:10 2007 From: facundo at taniquetil.com.ar (Facundo Batista) Date: Tue, 3 Apr 2007 01:28:10 +0000 (UTC) Subject: [Python-Dev] About SSL tests References: <20070329004057.18920.331373336.divmod.quotient.6356@ohm> Message-ID: Jean-Paul Calderone wrote: > If the openssl binary is available, when the test starts, launch it in > a child process, talk to it for the test, then kill it when the test is > done. Ok, I have a demo of this. Right now, I face this problem. I launch openssl through subprocess, but I do *not* find a way to tell him to quit serving, so all I can do is to kill the process (through the pid from the Popen object). The problem is that os.kill only works in Unix and Macintosh. So, there's a better way to do this? Or I shall check if I'm in one of those both platforms and only execute the tests there? Another question, this one more operative. For openssl I'll need both cert.pem and key.pem files. I have them, but in which directory I shall commit them? ./Lib/test? Or create another directory inside that? Regards, -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From sgeiger at ncee.net Tue Apr 3 04:55:49 2007 From: sgeiger at ncee.net (Shane Geiger) Date: Mon, 02 Apr 2007 21:55:49 -0500 Subject: [Python-Dev] possible urllib bug on Windows XP Message-ID: <4611C235.4030606@ncee.net> I have a curious situation which might be a bug in urllib (Python 2.5 installed from a .msi file gotten from python.org) when running on Windows XP. (This is running inside a Parallels virtual machine, but I don't see how that matters, because, as you will see in a second, I'm able to access the Web server both by its IP address and name. Here's a screenshot of a console session: http://www.ncee.net/shane/urllib.png You can see from the screenshot: 1. This machine is able to ping the Web server's IP address. 2. I can even access the Web server via the DNS name assigned (www.nce.net) using wget.exe (from UnxUtils). 3. However, when I try to simulate wget.exe by using urllib, I get an error. Here's the urllib command (which, incidentally works fine from my MacBook Pro): python -c "import urllib ; urllib.urlretrieve(\"http://www.ncee.net/shane/test.py\",\"test.py\")" I tested this on a Windows 2000 Server, and urllib behaved normally. Is this possibly a bug on Windows XP? -- Shane Geiger IT Director National Council on Economic Education sgeiger at ncee.net | 402-438-8958 | http://www.ncee.net Leading the Campaign for Economic and Financial Literacy -------------- next part -------------- A non-text attachment was scrubbed... Name: sgeiger.vcf Type: text/x-vcard Size: 297 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20070402/667dd264/attachment.vcf From scott+python-dev at scottdial.com Tue Apr 3 03:59:20 2007 From: scott+python-dev at scottdial.com (Scott Dial) Date: Mon, 02 Apr 2007 21:59:20 -0400 Subject: [Python-Dev] possible urllib bug on Windows XP In-Reply-To: <4611C235.4030606@ncee.net> References: <4611C235.4030606@ncee.net> Message-ID: <4611B4F8.3050906@scottdial.com> Shane Geiger wrote: > Is this possibly a bug on Windows XP? I can't reproduce this bug on Windows XP. -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From mike.klaas at gmail.com Tue Apr 3 04:23:43 2007 From: mike.klaas at gmail.com (Mike Klaas) Date: Mon, 2 Apr 2007 19:23:43 -0700 Subject: [Python-Dev] possible urllib bug on Windows XP In-Reply-To: <4611C235.4030606@ncee.net> References: <4611C235.4030606@ncee.net> Message-ID: <3d2ce8cb0704021923i5546742dm4d5456dd6c3c2fd4@mail.gmail.com> On 4/2/07, Shane Geiger wrote: > I have a curious situation which might be a bug in urllib (Python 2.5 > installed from a .msi file gotten from python.org) when running on > Windows XP. (This is running inside a Parallels virtual machine, but I > don't see how that matters, because, as you will see in a second, I'm > able to access the Web server both by its IP address and name. > > Here's a screenshot of a console session: > > http://www.ncee.net/shane/urllib.png > > You can see from the screenshot: > > 1. This machine is able to ping the Web server's IP address. > 2. I can even access the Web server via the DNS name assigned > (www.nce.net) using wget.exe (from UnxUtils). > 3. However, when I try to simulate wget.exe by using urllib, I get an > error. > > Here's the urllib command (which, incidentally works fine from my > MacBook Pro): > python -c "import urllib ; > urllib.urlretrieve(\"http://www.ncee.net/shane/test.py\",\"test.py\")" In the screen shot, you are retrieving a different filename with python: python -c "import urllib ; urllib.urlretrieve(\"http://www.ncee.net/shane/login_test.py\")" -Mike From jcarlson at uci.edu Tue Apr 3 04:33:57 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Mon, 02 Apr 2007 19:33:57 -0700 Subject: [Python-Dev] About SSL tests In-Reply-To: References: <20070329004057.18920.331373336.divmod.quotient.6356@ohm> Message-ID: <20070402193214.04BA.JCARLSON@uci.edu> Facundo Batista wrote: > Jean-Paul Calderone wrote: > > If the openssl binary is available, when the test starts, launch it in > > a child process, talk to it for the test, then kill it when the test is > > done. > > Ok, I have a demo of this. > > Right now, I face this problem. > > I launch openssl through subprocess, but I do *not* find a way to tell > him to quit serving, so all I can do is to kill the process (through the > pid from the Popen object). > > The problem is that os.kill only works in Unix and Macintosh. So, > there's a better way to do this? Or I shall check if I'm in one of those > both platforms and only execute the tests there? If you have a compilation of pywin32 (isn't it shipped by default in Python 2.5+?), you can kill the process with win32process.TerminateProcess() . - Josiah From martin at v.loewis.de Tue Apr 3 06:17:08 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 03 Apr 2007 06:17:08 +0200 Subject: [Python-Dev] About SSL tests In-Reply-To: <20070402193214.04BA.JCARLSON@uci.edu> References: <20070329004057.18920.331373336.divmod.quotient.6356@ohm> <20070402193214.04BA.JCARLSON@uci.edu> Message-ID: <4611D544.70304@v.loewis.de> >> The problem is that os.kill only works in Unix and Macintosh. So, >> there's a better way to do this? Or I shall check if I'm in one of those >> both platforms and only execute the tests there? > > If you have a compilation of pywin32 (isn't it shipped by default in > Python 2.5+?), you can kill the process with > win32process.TerminateProcess() . On Win32, you also have subprocess.TerminateProcess, if you have the subprocess module in the first place. Regards, Martin From jcarlson at uci.edu Tue Apr 3 08:44:15 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Mon, 02 Apr 2007 23:44:15 -0700 Subject: [Python-Dev] About SSL tests In-Reply-To: <4611D544.70304@v.loewis.de> References: <20070402193214.04BA.JCARLSON@uci.edu> <4611D544.70304@v.loewis.de> Message-ID: <20070402234328.04BF.JCARLSON@uci.edu> "Martin v. L?wis" wrote: > > >> The problem is that os.kill only works in Unix and Macintosh. So, > >> there's a better way to do this? Or I shall check if I'm in one of those > >> both platforms and only execute the tests there? > > > > If you have a compilation of pywin32 (isn't it shipped by default in > > Python 2.5+?), you can kill the process with > > win32process.TerminateProcess() . > > On Win32, you also have subprocess.TerminateProcess, if you have the > subprocess module in the first place. Indeed! Or even ctypes. Ok, 3 different possible ways of killing a process on Windows. I think that's enough for now. :) - Josiah From hasan.diwan at gmail.com Tue Apr 3 09:48:50 2007 From: hasan.diwan at gmail.com (Hasan Diwan) Date: Tue, 3 Apr 2007 00:48:50 -0700 Subject: [Python-Dev] test_socketserver flakey? In-Reply-To: References: Message-ID: <2cda2fc90704030048u709763a9w19713d86cd83ec6a@mail.gmail.com> > Guido van Rossum wrote: > > > The test_socketserver unittest seems to be failing occasionally for > > me. (Svn HEAD, Ubuntu dapper.) Specifically, which test is failing? -- Cheers, Hasan Diwan -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070403/c86122ed/attachment.htm From ronaldoussoren at mac.com Tue Apr 3 14:27:12 2007 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Tue, 3 Apr 2007 14:27:12 +0200 Subject: [Python-Dev] Distutils and -framework on MacOSX In-Reply-To: <460C9FA8.7060307@canterbury.ac.nz> References: <460B88E4.4060204@canterbury.ac.nz> <32AD8D7B-249A-4940-A316-8B4228FACAB0@mac.com> <460C9FA8.7060307@canterbury.ac.nz> Message-ID: On 30 Mar, 2007, at 7:27, Greg Ewing wrote: > Ronald Oussoren wrote: > >> What's wrong with adding -framework flags to the end? I do this >> all the time and have yet to run into problems. > > I don't know what's wrong. Sometimes it works for me too, > but often it doesn't, and when it doesn't, putting them > at the front seems to fix it. Could you create an example where adding -framework to the end of the command-line doesn't work? BTW. I'm asking around for a more definitive statement on how - framework arguments are interpreted. Ronald From facundo at taniquetil.com.ar Tue Apr 3 14:51:19 2007 From: facundo at taniquetil.com.ar (Facundo Batista) Date: Tue, 3 Apr 2007 12:51:19 +0000 (UTC) Subject: [Python-Dev] About SSL tests References: <20070329004057.18920.331373336.divmod.quotient.6356@ohm> <20070402193214.04BA.JCARLSON@uci.edu> <4611D544.70304@v.loewis.de> Message-ID: Martin v. L?wis wrote: > On Win32, you also have subprocess.TerminateProcess, if you have the > subprocess module in the first place. The problem of TerminateProcess is that I need the handle of the process. I don't like the idea of rely on the private _handle and do: process = subprocess.Popen(...) ... subprocess.TerminateProcess(int(process._handle), -1) so, I'll end doing this: -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From facundo at taniquetil.com.ar Tue Apr 3 14:53:20 2007 From: facundo at taniquetil.com.ar (Facundo Batista) Date: Tue, 3 Apr 2007 12:53:20 +0000 (UTC) Subject: [Python-Dev] About SSL tests References: <20070329004057.18920.331373336.divmod.quotient.6356@ohm> <20070402193214.04BA.JCARLSON@uci.edu> <4611D544.70304@v.loewis.de> Message-ID: Martin v. L?wis wrote: > On Win32, you also have subprocess.TerminateProcess, if you have the > subprocess module in the first place. The problem of TerminateProcess is that I need the handle of the process. I don't like the idea of rely on the private _handle and do: process = subprocess.Popen(...) ... subprocess.TerminateProcess(int(process._handle), -1) so, I'll end doing this: process = subprocess.Popen(...) ... handle =ctypes.windll.kernel32.OpenProcess(1, False, process.pid) ctypes.windll.kernel32.TerminateProcess(handle, -1) ctypes.windll.kernel32.CloseHandle(handle) Is this ok? Thank you!! -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From eric+python-dev at trueblade.com Tue Apr 3 17:34:32 2007 From: eric+python-dev at trueblade.com (Eric V. Smith) Date: Tue, 03 Apr 2007 11:34:32 -0400 Subject: [Python-Dev] About SSL tests In-Reply-To: References: <20070329004057.18920.331373336.divmod.quotient.6356@ohm> <20070402193214.04BA.JCARLSON@uci.edu> <4611D544.70304@v.loewis.de> Message-ID: <46127408.2040107@trueblade.com> [sorry if you see 2 copies of this] Facundo Batista wrote: > The problem of TerminateProcess is that I need the handle of the > process. > > I don't like the idea of rely on the private _handle and do: > > process = subprocess.Popen(...) > ... > subprocess.TerminateProcess(int(process._handle), -1) > > so, I'll end doing this: > > process = subprocess.Popen(...) > ... > handle =ctypes.windll.kernel32.OpenProcess(1, False, process.pid) > ctypes.windll.kernel32.TerminateProcess(handle, -1) > ctypes.windll.kernel32.CloseHandle(handle) > > Is this ok? Would it not be better to put a platform-independent version of this into subprocess, so that this code doesn't have to be duplicated all over the place? Maybe a method on a Popen object called terminate()? That way you wouldn't even need ctypes, since you could look at the _handle without feeling bad about it! Eric. From facundo at taniquetil.com.ar Tue Apr 3 18:25:57 2007 From: facundo at taniquetil.com.ar (Facundo Batista) Date: Tue, 3 Apr 2007 16:25:57 +0000 (UTC) Subject: [Python-Dev] About SSL tests References: <20070329004057.18920.331373336.divmod.quotient.6356@ohm> <20070402193214.04BA.JCARLSON@uci.edu> <4611D544.70304@v.loewis.de> <46127408.2040107@trueblade.com> Message-ID: Eric V. Smith wrote: > Would it not be better to put a platform-independent version of this > into subprocess, so that this code doesn't have to be duplicated all > over the place? Maybe a method on a Popen object called terminate()? Yes. But I'm not up to that task. Really don't know how to kill processes in *all* the platforms where Python's subprocess is present. My problem is smaller, I'm making a new tests in test_socket_ssl.py, and I just wanted to support win32 (as well as Unix and Macintosh). Regards, -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From guido at python.org Tue Apr 3 18:36:01 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 3 Apr 2007 09:36:01 -0700 Subject: [Python-Dev] test_socketserver flakey? In-Reply-To: <2cda2fc90704030048u709763a9w19713d86cd83ec6a@mail.gmail.com> References: <2cda2fc90704030048u709763a9w19713d86cd83ec6a@mail.gmail.com> Message-ID: On 4/3/07, Hasan Diwan wrote: > > Guido van Rossum wrote: > > > The test_socketserver unittest seems to be failing occasionally for > > > me. (Svn HEAD, Ubuntu dapper.) > > Specifically, which test is failing? Hard to say -- so far, I've just seen the blanket "test failed" report from regrtest.py, occasionally. The few times I ran the test directly it passed. I'll try running it in a loop to see what happens. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Apr 3 19:15:54 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 3 Apr 2007 10:15:54 -0700 Subject: [Python-Dev] test_socketserver flakey? In-Reply-To: References: <2cda2fc90704030048u709763a9w19713d86cd83ec6a@mail.gmail.com> Message-ID: On 4/3/07, Guido van Rossum wrote: > On 4/3/07, Hasan Diwan wrote: > > > Guido van Rossum wrote: > > > > The test_socketserver unittest seems to be failing occasionally for > > > > me. (Svn HEAD, Ubuntu dapper.) > > > > Specifically, which test is failing? > > Hard to say -- so far, I've just seen the blanket "test failed" report > from regrtest.py, occasionally. The few times I ran the test directly > it passed. I'll try running it in a loop to see what happens. OK, did that, after a bunch of iterations it failed like this: ADDR = @test17637 CLASS = SocketServer.UnixStreamServer server created server running test client 0 test test_socketserver crashed -- : (2, 'No such file or directory') Traceback (most recent call last): File "../Lib/test/regrtest.py", line 557, in runtest_inner indirect_test() File "/usr/local/google/home/guido/python/trunk/Lib/test/test_socketserver.py", line 212, in test_main testall() File "/usr/local/google/home/guido/python/trunk/Lib/test/test_socketserver.py", line 200, in testall testloop(socket.AF_UNIX, streamservers, MyStreamHandler, teststream) File "/usr/local/google/home/guido/python/trunk/Lib/test/test_socketserver.py", line 144, in testloop testfunc(proto, addr) File "/usr/local/google/home/guido/python/trunk/Lib/test/test_socketserver.py", line 62, in teststream s.connect(addr) File "", line 1, in connect error: (2, 'No such file or directory') -- --Guido van Rossum (home page: http://www.python.org/~guido/) From eric+python-dev at trueblade.com Tue Apr 3 20:39:04 2007 From: eric+python-dev at trueblade.com (Eric V. Smith) Date: Tue, 03 Apr 2007 14:39:04 -0400 Subject: [Python-Dev] About SSL tests In-Reply-To: References: <20070329004057.18920.331373336.divmod.quotient.6356@ohm> <20070402193214.04BA.JCARLSON@uci.edu> <4611D544.70304@v.loewis.de> <46127408.2040107@trueblade.com> Message-ID: <46129F48.8030504@trueblade.com> Facundo Batista wrote: > Eric V. Smith wrote: > >> Would it not be better to put a platform-independent version of this >> into subprocess, so that this code doesn't have to be duplicated all >> over the place? Maybe a method on a Popen object called terminate()? > > Yes. But I'm not up to that task. Really don't know how to kill > processes in *all* the platforms where Python's subprocess is present. I'd be willing to look at adding it, if the group thinks it's the right thing to do. Eric. From lists at cheimes.de Tue Apr 3 21:28:27 2007 From: lists at cheimes.de (Christian Heimes) Date: Tue, 03 Apr 2007 21:28:27 +0200 Subject: [Python-Dev] About SSL tests In-Reply-To: <46129F48.8030504@trueblade.com> References: <20070329004057.18920.331373336.divmod.quotient.6356@ohm> <20070402193214.04BA.JCARLSON@uci.edu> <4611D544.70304@v.loewis.de> <46127408.2040107@trueblade.com> <46129F48.8030504@trueblade.com> Message-ID: > I'd be willing to look at adding it, if the group thinks it's the right > thing to do. I like the idea and I'm proposing to add two more methods to subprocess Popen. class Popen(...): ... def signal(self, signal): """Send a signal to the process (UNIX only) signal is constant from the signal module """ def terminate(self, force=False): """Terminate the process On UNIX terminate(False) is equivalent to signal(SIGTERM) and terminate(True) to signal(SIGKILL). On Windows ... (does Windows support a forced terminate?) """ Christian From facundo at taniquetil.com.ar Tue Apr 3 21:40:01 2007 From: facundo at taniquetil.com.ar (Facundo Batista) Date: Tue, 3 Apr 2007 19:40:01 +0000 (UTC) Subject: [Python-Dev] About SSL tests References: <20070329004057.18920.331373336.divmod.quotient.6356@ohm> <20070402193214.04BA.JCARLSON@uci.edu> <4611D544.70304@v.loewis.de> <46127408.2040107@trueblade.com> <46129F48.8030504@trueblade.com> Message-ID: Eric V. Smith wrote: > I'd be willing to look at adding it, if the group thinks it's the right > thing to do. +1 to have the functionality of "kill the process you started" in subprocess. -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From steven.bethard at gmail.com Tue Apr 3 21:45:27 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue, 3 Apr 2007 13:45:27 -0600 Subject: [Python-Dev] About SSL tests In-Reply-To: References: <20070329004057.18920.331373336.divmod.quotient.6356@ohm> <20070402193214.04BA.JCARLSON@uci.edu> <4611D544.70304@v.loewis.de> <46127408.2040107@trueblade.com> <46129F48.8030504@trueblade.com> Message-ID: On 4/3/07, Christian Heimes wrote: > > I'd be willing to look at adding it, if the group thinks it's the right > > thing to do. > > I like the idea and I'm proposing to add two more methods to subprocess > Popen. > > class Popen(...): > ... > def signal(self, signal): > """Send a signal to the process (UNIX only) > > signal is constant from the signal module > """ > > def terminate(self, force=False): > """Terminate the process > > On UNIX terminate(False) is equivalent to signal(SIGTERM) and > terminate(True) to signal(SIGKILL). > > On Windows ... (does Windows support a forced terminate?) > """ Another difference I believe is that TerminateProcess on Windows doesn't kill the tree of processes like kill would. It would be nice if Popen.terminate() did the same thing on both Unix and Windows. (I assume that would mean making *all* the appropriate TerminateProcess calls.) Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From draghuram at gmail.com Tue Apr 3 22:17:15 2007 From: draghuram at gmail.com (Raghuram Devarakonda) Date: Tue, 3 Apr 2007 16:17:15 -0400 Subject: [Python-Dev] About SSL tests In-Reply-To: References: <20070329004057.18920.331373336.divmod.quotient.6356@ohm> Message-ID: <2c51ecee0704031317u27bbf264hde92bc1f580c05fa@mail.gmail.com> On 4/2/07, Facundo Batista wrote: > I launch openssl through subprocess, but I do *not* find a way to tell > him to quit serving, so all I can do is to kill the process (through the > pid from the Popen object). > > The problem is that os.kill only works in Unix and Macintosh. So, > there's a better way to do this? Or I shall check if I'm in one of those > both platforms and only execute the tests there? I just checked the man page for s_server (on linux) and it mentions something about exiting. "Certain single letter commands are also recognized which perform special operations: these are listed below. q end the current SSL connection but still accept new connections. Q end the current SSL connection and exit." Can a command "Q" be sent to the server once testing is complete? Thanks, Raghu. From jcarlson at uci.edu Tue Apr 3 22:37:52 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 03 Apr 2007 13:37:52 -0700 Subject: [Python-Dev] About SSL tests In-Reply-To: References: Message-ID: <20070403131404.04D5.JCARLSON@uci.edu> "Steven Bethard" wrote: > On 4/3/07, Christian Heimes wrote: > > > I'd be willing to look at adding it, if the group thinks it's the right > > > thing to do. > > > > I like the idea and I'm proposing to add two more methods to subprocess > > Popen. > > > > class Popen(...): > > ... > > def signal(self, signal): > > """Send a signal to the process (UNIX only) > > > > signal is constant from the signal module > > """ > > > > def terminate(self, force=False): > > """Terminate the process > > > > On UNIX terminate(False) is equivalent to signal(SIGTERM) and > > terminate(True) to signal(SIGKILL). > > > > On Windows ... (does Windows support a forced terminate?) > > """ > > Another difference I believe is that TerminateProcess on Windows > doesn't kill the tree of processes like kill would. It would be nice > if Popen.terminate() did the same thing on both Unix and Windows. (I > assume that would mean making *all* the appropriate TerminateProcess > calls.) To kill child processes on Windows, one must walk the tree and kill those as well. wxWidgets has an implementation in src/msw/utils.cpp: http://cvs.wxwidgets.org/viewcvs.cgi/wxWidgets/src/msw/utils.cpp See the differences between wxKill and wxKillAllChildren . I would also mention that wxWidgets implements SIGKILL as TerminateProcess(), and SIGTERM as sending a QUIT message to the process. - Josiah From guido at python.org Wed Apr 4 00:17:11 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 3 Apr 2007 15:17:11 -0700 Subject: [Python-Dev] test_socketserver flakey? In-Reply-To: <2cda2fc90704031457x639fc10v65c7143ebd01763@mail.gmail.com> References: <2cda2fc90704030048u709763a9w19713d86cd83ec6a@mail.gmail.com> <2cda2fc90704031424p15091052n5ae9e77e3878080@mail.gmail.com> <2cda2fc90704031457x639fc10v65c7143ebd01763@mail.gmail.com> Message-ID: I'm not sure I agree with your analysis -- the test that fails is using unix domain sockets, not TCP. On 4/3/07, Hasan Diwan wrote: > Guido: > > On 03/04/07, Guido van Rossum wrote: > > But I saw this failure during a regular regrtest run as well. How > > would you suggest to fix this?i > > > > You'll need to make the connect/close lines into a critical section and have > execution suspended for the TCP wait time. It defaults to 60 seconds on BSD > sockets, but I'm not aware of the constant for it (a quick survey of the > header files doesn't show anything promising). On the other hand, I don't > think you're going to be running this particular test a number of times > quickly (or will you?). In the final examination, I suspect it's a race > condition. > -- > Cheers, > > Hasan Diwan > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From trentm at activestate.com Wed Apr 4 00:31:22 2007 From: trentm at activestate.com (Trent Mick) Date: Tue, 03 Apr 2007 15:31:22 -0700 Subject: [Python-Dev] About SSL tests In-Reply-To: References: <20070329004057.18920.331373336.divmod.quotient.6356@ohm> <20070402193214.04BA.JCARLSON@uci.edu> <4611D544.70304@v.loewis.de> <46127408.2040107@trueblade.com> <46129F48.8030504@trueblade.com> Message-ID: <4612D5BA.8090209@activestate.com> Christian Heimes wrote: >> I'd be willing to look at adding it, if the group thinks it's the right >> thing to do. > > I like the idea and I'm proposing to add two more methods to subprocess > Popen. > > class Popen(...): > ... > def signal(self, signal): > """Send a signal to the process (UNIX only) > > signal is constant from the signal module > """ > > def terminate(self, force=False): > """Terminate the process > > On UNIX terminate(False) is equivalent to signal(SIGTERM) and > terminate(True) to signal(SIGKILL). > > On Windows ... (does Windows support a forced terminate?) > """ Here is what my process.py [1] does for termination (not suggesting you follow the API here, just showing how it handles termination on Windows). Some of the Windows logic is borrowed from PyWin32's winprocess.py [2] > def kill(self, exitCode=0, gracePeriod=1.0, sig=None): > """Kill process. > > "exitCode" [deprecated, not supported] (Windows only) is the > code the terminated process should exit with. > "gracePeriod" (Windows only) is a number of seconds the process is > allowed to shutdown with a WM_CLOSE signal before a hard > terminate is called. > "sig" (Unix only) is the signal to use to kill the process. Defaults > to signal.SIGKILL. See os.kill() for more information. > > Windows: > Try for an orderly shutdown via WM_CLOSE. If still running > after gracePeriod (1 sec. default), terminate. > """ > if sys.platform.startswith("win"): > import win32gui > # Send WM_CLOSE to windows in this process group. > win32gui.EnumWindows(self._close_, 0) > > # Send Ctrl-Break signal to all processes attached to this > # console. This is supposed to trigger shutdown handlers in > # each of the processes. > try: > win32api.GenerateConsoleCtrlEvent(CTRL_BREAK_EVENT, > self._processId) > except AttributeError: > log.warn("The win32api module does not have "\ > "GenerateConsoleCtrlEvent(). This may mean that "\ > "parts of this process group have NOT been killed.") > except win32api.error, ex: > if ex.args[0] not in (6, 87): > # Ignore the following: > # api_error: (87, 'GenerateConsoleCtrlEvent', 'The parameter is incorrect.') > # api_error: (6, 'GenerateConsoleCtrlEvent', 'The handle is invalid.') > # Get error 6 if there is no console. > raise > > # Last resort: call TerminateProcess if it has not yet. > retval = 0 > try: > self.wait(gracePeriod) > except ProcessError, ex: > log.info("[%s] Process.kill: calling TerminateProcess", id(self)) > win32process.TerminateProcess(self._hProcess, -1) > win32api.Sleep(100) # wait for resources to be released > > else: > if sig is None: > sig = signal.SIGKILL > try: > os.kill(self._pid, sig) > except OSError, ex: > if ex.errno != 3: > # Ignore: OSError: [Errno 3] No such process > raise Links: [1] http://trentm.com/projects/process/ [2] http://pywin32.cvs.sourceforge.net/pywin32/pywin32/win32/Demos/winprocess.py?revision=1.2&view=markup#l_121 -- Trent Mick trentm at activestate.com From trentm at activestate.com Wed Apr 4 00:38:31 2007 From: trentm at activestate.com (Trent Mick) Date: Tue, 03 Apr 2007 15:38:31 -0700 Subject: [Python-Dev] About SSL tests In-Reply-To: References: <20070329004057.18920.331373336.divmod.quotient.6356@ohm> <20070402193214.04BA.JCARLSON@uci.edu> <4611D544.70304@v.loewis.de> <46127408.2040107@trueblade.com> <46129F48.8030504@trueblade.com> Message-ID: <4612D767.4070002@activestate.com> Steven Bethard wrote: > Another difference I believe is that TerminateProcess on Windows > doesn't kill the tree of processes like kill would. It would be nice > if Popen.terminate() did the same thing on both Unix and Windows. (I > assume that would mean making *all* the appropriate TerminateProcess > calls.) Possibly that should be kept separate and handled via eventual handling for process jobs: http://msdn2.microsoft.com/en-us/library/ms684161.aspx > A job object allows groups of processes to be managed as a unit. Job objects are namable, securable, sharable objects that control attributes of the processes associated with them. Operations performed on the job object affect all processes associated with the job object. > > ... > To terminate all processes currently associated with a job object, use the TerminateJobObject function. Trent -- Trent Mick trentm at activestate.com From facundo at taniquetil.com.ar Wed Apr 4 01:48:23 2007 From: facundo at taniquetil.com.ar (Facundo Batista) Date: Tue, 3 Apr 2007 23:48:23 +0000 (UTC) Subject: [Python-Dev] About SSL tests References: <20070329004057.18920.331373336.divmod.quotient.6356@ohm> <2c51ecee0704031317u27bbf264hde92bc1f580c05fa@mail.gmail.com> Message-ID: Raghuram Devarakonda wrote: > Q end the current SSL connection and exit." > > Can a command "Q" be sent to the server once testing is complete? For openssl to recognize your "Q", you need to have a connection active. So, we need a better way to kill the external openssl in the tests (don't killing it sometimes when the connection can not be stablished is not an option). Regards, -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From martin at v.loewis.de Wed Apr 4 07:06:37 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 04 Apr 2007 07:06:37 +0200 Subject: [Python-Dev] About SSL tests In-Reply-To: References: <20070329004057.18920.331373336.divmod.quotient.6356@ohm> <20070402193214.04BA.JCARLSON@uci.edu> <4611D544.70304@v.loewis.de> Message-ID: <4613325D.70706@v.loewis.de> > I don't like the idea of rely on the private _handle and do: > > process = subprocess.Popen(...) > ... > subprocess.TerminateProcess(int(process._handle), -1) > > so, I'll end doing this: > > process = subprocess.Popen(...) > ... > handle =ctypes.windll.kernel32.OpenProcess(1, False, process.pid) > ctypes.windll.kernel32.TerminateProcess(handle, -1) > ctypes.windll.kernel32.CloseHandle(handle) > > Is this ok? I don't like it. I would rather rely on the private _handle member. If that ever gets changed, the test fails. Regards, Martin From martin at v.loewis.de Wed Apr 4 07:10:02 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 04 Apr 2007 07:10:02 +0200 Subject: [Python-Dev] About SSL tests In-Reply-To: References: <20070329004057.18920.331373336.divmod.quotient.6356@ohm> <20070402193214.04BA.JCARLSON@uci.edu> <4611D544.70304@v.loewis.de> <46127408.2040107@trueblade.com> <46129F48.8030504@trueblade.com> Message-ID: <4613332A.8070804@v.loewis.de> > Another difference I believe is that TerminateProcess on Windows > doesn't kill the tree of processes like kill would. I believe you are wrong here: kill on Unix would *not* kill the tree of processes. Killing the parent process just does that: kill the parent process. Killing process groups is an entirely different issue. It would be nice if subprocess also supported process groups (through the same-named POSIX concept on POSIX, and job objects on Win32), but that again is a different story. Regards, Martin From anugupta at pu.ac.in Wed Apr 4 06:44:05 2007 From: anugupta at pu.ac.in (Anu Gupta DCSA) Date: Wed, 4 Apr 2007 11:14:05 +0630 Subject: [Python-Dev] A Survey on Defect Management Practices in Free/Open Source Software Message-ID: <20070404044339.M83476@pu.ac.in> Dear Python Contributors, I seek help from designers, developers, testers,defect fixers,project managers or playing any other key role in Free/Open Source software development or maintenence in carrying out a study on practices and problems of defect management in various Free/Open Source Software projects. The insights gained from the study can further help us to extract publicly accessible defect data and determine impact of defect management practices on software quality. Please spend a few minutes of your precious time to fill up the Questionnaire. The most of the questions follow multiple choice formats and are quite easy to answer. To have the Online Questionnaire, please visit: http://anu.puchd.ac.in/phpESP/public/survey.php?name=FOSS_Defect_Survey (You can also copy and paste this link into your browser, and hit the 'Return' key.) I hope you will find all the questions interesting and thought-provoking. Your answers will be kept anonymous.The data thus collected will only be used for research purpose.It would be nice if you may further refer this mail to others actively engaged with Free/Open Source Software development. If you have any query or suggestions then feel free to contact. Thank You With regards, Anu Gupta Senior Lecturer Department of Computer Science and Applications, Panjab University, Chandigarh. INDIA In case of any problem in accessing/using the above mentioned link please contact: E-mail: anugupta at pu.ac.in anugupta98 at gmail.com From anthony at interlink.com.au Wed Apr 4 08:07:18 2007 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed, 4 Apr 2007 16:07:18 +1000 Subject: [Python-Dev] A Survey on Defect Management Practices in Free/Open Source Software In-Reply-To: <20070404044339.M83476@pu.ac.in> References: <20070404044339.M83476@pu.ac.in> Message-ID: <200704041607.18737.anthony@interlink.com.au> On Wednesday 04 April 2007 14:44, Anu Gupta DCSA wrote: > Dear Python Contributors, > > I seek help from designers, developers, testers,defect > fixers,project managers or playing any other key role in > Free/Open Source software development or maintenence > in carrying out a study on practices and problems of defect > management in various Free/Open Source Software projects. The Just a random aside - is anyone else getting increasingly annoyed by these mass-mailed out survey requests from students? I must get a several a week now. This one was at least personally addressed (well, to "Python Contributors"), which is a step ahead of most of them. -- Anthony Baxter It's never too late to have a happy childhood. From g.brandl at gmx.net Wed Apr 4 08:36:24 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 04 Apr 2007 08:36:24 +0200 Subject: [Python-Dev] A Survey on Defect Management Practices in Free/Open Source Software In-Reply-To: <200704041607.18737.anthony@interlink.com.au> References: <20070404044339.M83476@pu.ac.in> <200704041607.18737.anthony@interlink.com.au> Message-ID: Anthony Baxter schrieb: > On Wednesday 04 April 2007 14:44, Anu Gupta DCSA wrote: >> Dear Python Contributors, >> >> I seek help from designers, developers, testers,defect >> fixers,project managers or playing any other key role in >> Free/Open Source software development or maintenence >> in carrying out a study on practices and problems of defect >> management in various Free/Open Source Software projects. The > > Just a random aside - is anyone else getting increasingly annoyed by > these mass-mailed out survey requests from students? I must get a > several a week now. This one was at least personally addressed > (well, to "Python Contributors"), which is a step ahead of most of > them. > Though, in an email targeted at developers, this looks ridiculous to me: | (You can also copy and paste this link into your browser, and hit the | 'Return' key.) cheers, Georg From stephen at xemacs.org Wed Apr 4 10:45:22 2007 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 04 Apr 2007 17:45:22 +0900 Subject: [Python-Dev] A Survey on Defect Management Practices in Free/Open Source Software In-Reply-To: <200704041607.18737.anthony@interlink.com.au> References: <20070404044339.M83476@pu.ac.in> <200704041607.18737.anthony@interlink.com.au> Message-ID: <87tzvwttnh.fsf@uwakimon.sk.tsukuba.ac.jp> Anthony Baxter writes: > Just a random aside - is anyone else getting increasingly annoyed by > these mass-mailed out survey requests from students? Annoyed, not particularly. Scared, yes: it's long been known that a field=FIELD is moribund when people start getting "PhDs in FIELD" for dissertations in "The History of FIELD". I suspect that that same concept applies to theses on "ADJECTIVE Management Practices in FIELD". This guy, however, has been at it for over a month, this is his fourth (or so) mailing. I think an entry in the Mailman persona-non-grata list may be appropriate. From amk at amk.ca Wed Apr 4 15:27:50 2007 From: amk at amk.ca (A.M. Kuchling) Date: Wed, 4 Apr 2007 09:27:50 -0400 Subject: [Python-Dev] A Survey on Defect Management Practices in Free/Open Source Software In-Reply-To: <200704041607.18737.anthony@interlink.com.au> References: <20070404044339.M83476@pu.ac.in> <200704041607.18737.anthony@interlink.com.au> Message-ID: <20070404132750.GA11225@localhost.localdomain> On Wed, Apr 04, 2007 at 04:07:18PM +1000, Anthony Baxter wrote: > This one was at least personally addressed > (well, to "Python Contributors"), which is a step ahead of most of > them. What gets me is that such surveys are invalid because the respondents aren't randomly selected -- they're the people who care enough to answer. It would be more efficient and reliable to just go look at a sampling of project web sites and look at their bug trackers, but that requires effort from the student. Recently I was interviewing a student who worked on a project that had instrumented an IDE to record every operation (editing, saving, etc.) in a database. I asked "so what did you learn from this data?", and was told that no one has analyzed it; they're just *accumulating* the data. It's stamp collecting as computer science. --amk, who save his files every 87.4 seconds and types a character every 2.3sec. From skip at pobox.com Wed Apr 4 15:39:19 2007 From: skip at pobox.com (skip at pobox.com) Date: Wed, 4 Apr 2007 08:39:19 -0500 Subject: [Python-Dev] A Survey on Defect Management Practices in Free/Open Source Software In-Reply-To: <20070404132750.GA11225@localhost.localdomain> References: <20070404044339.M83476@pu.ac.in> <200704041607.18737.anthony@interlink.com.au> <20070404132750.GA11225@localhost.localdomain> Message-ID: <17939.43655.321828.507209@montanaro.dyndns.org> amk> ... no one has analyzed it; they're just *accumulating* the data. amk> It's stamp collecting as computer science. +1 QOTF. Skip From steve at holdenweb.com Wed Apr 4 15:57:52 2007 From: steve at holdenweb.com (Steve Holden) Date: Wed, 04 Apr 2007 09:57:52 -0400 Subject: [Python-Dev] A Survey on Defect Management Practices in Free/Open Source Software In-Reply-To: <20070404132750.GA11225@localhost.localdomain> References: <20070404044339.M83476@pu.ac.in> <200704041607.18737.anthony@interlink.com.au> <20070404132750.GA11225@localhost.localdomain> Message-ID: A.M. Kuchling wrote: > On Wed, Apr 04, 2007 at 04:07:18PM +1000, Anthony Baxter wrote: >> This one was at least personally addressed >> (well, to "Python Contributors"), which is a step ahead of most of >> them. > > What gets me is that such surveys are invalid because the respondents > aren't randomly selected -- they're the people who care enough to > answer. Or the people who are getting so pissed off with these surveys that they are willing to spend the time to mess up the results. It would be more efficient and reliable to just go look at a > sampling of project web sites and look at their bug trackers, but that > requires effort from the student. > I remember those days ... > Recently I was interviewing a student who worked on a project that had > instrumented an IDE to record every operation (editing, saving, etc.) > in a database. I asked "so what did you learn from this data?", and > was told that no one has analyzed it; they're just *accumulating* the > data. It's stamp collecting as computer science. > > --amk, who save his files every 87.4 seconds and types a character > every 2.3sec. > -- steve, who hits backspace every 1.6 seconds and still makes loads of typos -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://del.icio.us/steve.holden Recent Ramblings http://holdenweb.blogspot.com From ncoghlan at gmail.com Wed Apr 4 17:23:35 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 05 Apr 2007 01:23:35 +1000 Subject: [Python-Dev] A Survey on Defect Management Practices in Free/Open Source Software In-Reply-To: <20070404132750.GA11225@localhost.localdomain> References: <20070404044339.M83476@pu.ac.in> <200704041607.18737.anthony@interlink.com.au> <20070404132750.GA11225@localhost.localdomain> Message-ID: <4613C2F7.4060109@gmail.com> A.M. Kuchling wrote: > On Wed, Apr 04, 2007 at 04:07:18PM +1000, Anthony Baxter wrote: >> This one was at least personally addressed >> (well, to "Python Contributors"), which is a step ahead of most of >> them. > > What gets me is that such surveys are invalid because the respondents > aren't randomly selected -- they're the people who care enough to > answer. Not only that, the few I have even bothered trying to answer I've given up on after my reaction to the first half a dozen questions involved variations on the phrase "Well, it depends on exactly what you mean by term ". Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From facundo at taniquetil.com.ar Wed Apr 4 18:10:35 2007 From: facundo at taniquetil.com.ar (Facundo Batista) Date: Wed, 4 Apr 2007 16:10:35 +0000 (UTC) Subject: [Python-Dev] About SSL tests References: <20070329004057.18920.331373336.divmod.quotient.6356@ohm> <20070402193214.04BA.JCARLSON@uci.edu> <4611D544.70304@v.loewis.de> <4613325D.70706@v.loewis.de> Message-ID: Martin v. L?wis wrote: > I don't like it. I would rather rely on the private _handle member. > If that ever gets changed, the test fails. I made it using _handle. Right now, we have test_socket_ssl.py using a local openssl and passing all the tests in all the buildbots, :D Thanks for your (you as in y'all, ;) help! Regards, -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From tjreedy at udel.edu Wed Apr 4 18:27:52 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 4 Apr 2007 12:27:52 -0400 Subject: [Python-Dev] A Survey on Defect Management Practicesin Free/Open Source Software References: <20070404044339.M83476@pu.ac.in><200704041607.18737.anthony@interlink.com.au> <87tzvwttnh.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: "Stephen J. Turnbull" wrote in message news:87tzvwttnh.fsf at uwakimon.sk.tsukuba.ac.jp... | This guy, however, has been at it for over a month, this is his fourth | (or so) mailing. I think an entry in the Mailman persona-non-grata | list may be appropriate. So would a letter/email to the chair of his/her department. I tried via the postmaster. I have also noticed that I have never seen one of the promised followups or reports. tjr From guido at python.org Wed Apr 4 19:45:17 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 4 Apr 2007 10:45:17 -0700 Subject: [Python-Dev] test_socketserver flakey? In-Reply-To: References: <2cda2fc90704030048u709763a9w19713d86cd83ec6a@mail.gmail.com> <2cda2fc90704031424p15091052n5ae9e77e3878080@mail.gmail.com> <2cda2fc90704031457x639fc10v65c7143ebd01763@mail.gmail.com> Message-ID: FWIW, I've checked a fix into the trunk. The code was waiting half a second and assumign that the server thread was ready. This occasionally failed of course. I fixed it by using an event variable instead of a sleep call. Not sure if it's worth backporting, as it is only a race condition in the unit test. (But I also don't see anything wrong with a backport. :) --Guido On 4/3/07, Guido van Rossum wrote: > I'm not sure I agree with your analysis -- the test that fails is > using unix domain sockets, not TCP. > > On 4/3/07, Hasan Diwan wrote: > > Guido: > > > > On 03/04/07, Guido van Rossum wrote: > > > But I saw this failure during a regular regrtest run as well. How > > > would you suggest to fix this?i > > > > > > > You'll need to make the connect/close lines into a critical section and have > > execution suspended for the TCP wait time. It defaults to 60 seconds on BSD > > sockets, but I'm not aware of the constant for it (a quick survey of the > > header files doesn't show anything promising). On the other hand, I don't > > think you're going to be running this particular test a number of times > > quickly (or will you?). In the final examination, I suspect it's a race > > condition. > > -- > > Cheers, > > > > Hasan Diwan > > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Wed Apr 4 21:34:54 2007 From: brett at python.org (Brett Cannon) Date: Wed, 4 Apr 2007 12:34:54 -0700 Subject: [Python-Dev] A Survey on Defect Management Practices in Free/Open Source Software In-Reply-To: <20070404132750.GA11225@localhost.localdomain> References: <20070404044339.M83476@pu.ac.in> <200704041607.18737.anthony@interlink.com.au> <20070404132750.GA11225@localhost.localdomain> Message-ID: On 4/4/07, A.M. Kuchling wrote: > > On Wed, Apr 04, 2007 at 04:07:18PM +1000, Anthony Baxter wrote: > > This one was at least personally addressed > > (well, to "Python Contributors"), which is a step ahead of most of > > them. > > What gets me is that such surveys are invalid because the respondents > aren't randomly selected -- they're the people who care enough to > answer. It would be more efficient and reliable to just go look at a > sampling of project web sites and look at their bug trackers, but that > requires effort from the student. > > Recently I was interviewing a student who worked on a project that had > instrumented an IDE to record every operation (editing, saving, etc.) > in a database. I asked "so what did you learn from this data?", and > was told that no one has analyzed it; they're just *accumulating* the > data. It's stamp collecting as computer science. I am putting way too much effort into my PhD if this kind of stuff can get me a degree. =) -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070404/0f0e2b6d/attachment.htm From greg.ewing at canterbury.ac.nz Wed Apr 4 22:39:49 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 05 Apr 2007 08:39:49 +1200 Subject: [Python-Dev] Distutils and -framework on MacOSX In-Reply-To: References: <460B88E4.4060204@canterbury.ac.nz> <32AD8D7B-249A-4940-A316-8B4228FACAB0@mac.com> <460C9FA8.7060307@canterbury.ac.nz> Message-ID: <46140D15.2060708@canterbury.ac.nz> Ronald Oussoren wrote: > Could you create an example where adding -framework to the end of the > command-line doesn't work? I'll see what I can do. -- Greg From anthony at interlink.com.au Thu Apr 5 06:56:24 2007 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu, 5 Apr 2007 14:56:24 +1000 Subject: [Python-Dev] branch is frozen for release of 2.5.1c1 (and 2.5.1) Message-ID: <200704051456.28567.anthony@interlink.com.au> Please stay out of the 2.5 branch unless you're one of the release team - I'm cutting 2.5.1c1 at the moment. There will be a 2.5.1 final next week, assuming all goes well. If you have an urgent bugfix you need in, please post here and get someone to approve it before the checkin! Thanks, Anthony -- Anthony Baxter It's never too late to have a happy childhood. From bob.sidebotham at gmail.com Thu Apr 5 21:54:22 2007 From: bob.sidebotham at gmail.com (Bob Sidebotham) Date: Thu, 5 Apr 2007 15:54:22 -0400 Subject: [Python-Dev] context manager - generator interaction? Message-ID: <1f8770e0704051254g7bd0296coe27de33bfec0a762@mail.gmail.com> The interaction shown below feels like a bug, or at least very much a trap for the unwary. It's some sort of interaction between a context manager and a generator, both of which can raise StopIteration. The code is excised from a real application, so it's a bit artificial looking. Nevertheless, it represents an entirely natural evolution of some code (whereby the call to a context manager was added deep within the code). Since it seems to work with open as the context manager, but not with my own null context manager defined with context_manager(), it feels like this is probably a bug in contextmanager(). I did look in the archives, and it's possible this has been raised before, but I wasn't clear on that. So apologies in advance if this is already understood. This is using the stock distribution of python 2.5 on Linux 2.4. Comments? Bob Sidebotham bash-2.05b$ cat test.py from __future__ import with_statement def gen(): return# Should cause stop iteration yield # Just to make this a generator, for this example # This works as expected. try: gen().next() # Should immediately raise StopIteration except StopIteration: print "Part 1: should reach here!" else: print "Part 1: shouldn't reach here!" # Now try the same thing, but with a context manager wrapping the generator call. # This also works as expected. try: with open("/etc/passwd") as pw: gen().next() except StopIteration: print "Part 2: should reach here!" else: print "Part 2: shouldn't reach here!" # Finally, try this with our own home grown null context manager. # This does not work as expected. from contextlib import contextmanager @contextmanager def ctxt(): yield try: with ctxt(): gen().next() except StopIteration: print "Part 3: should reach here!" else: print "Part 3: shouldn't reach here!" bash-2.05b$ python test.py Part 1: should reach here! Part 2: should reach here! Part 3: shouldn't reach here! bash-2.05b$ From aahz at pythoncraft.com Thu Apr 5 22:07:54 2007 From: aahz at pythoncraft.com (Aahz) Date: Thu, 5 Apr 2007 13:07:54 -0700 Subject: [Python-Dev] context manager - generator interaction? In-Reply-To: <1f8770e0704051254g7bd0296coe27de33bfec0a762@mail.gmail.com> References: <1f8770e0704051254g7bd0296coe27de33bfec0a762@mail.gmail.com> Message-ID: <20070405200753.GA10431@panix.com> On Thu, Apr 05, 2007, Bob Sidebotham wrote: > > The interaction shown below feels like a bug, or at least very much a > trap for the unwary. Please go ahead and file a bug report on SourceForge -- that way there is a record. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Why is this newsgroup different from all other newsgroups? From guido at python.org Thu Apr 5 22:10:06 2007 From: guido at python.org (Guido van Rossum) Date: Thu, 5 Apr 2007 13:10:06 -0700 Subject: [Python-Dev] context manager - generator interaction? In-Reply-To: <1f8770e0704051254g7bd0296coe27de33bfec0a762@mail.gmail.com> References: <1f8770e0704051254g7bd0296coe27de33bfec0a762@mail.gmail.com> Message-ID: Isn't this violating the rule that a try/except should only enclose the smallest expression where the exception is expected? BTW this is probably more appropriate for c.l.py -- even if there's a change to the language to come out of the discussion, it should be held there until the use case is well established. But my suspicion is that you get what you ask for with an overly-wide try block. --Guido On 4/5/07, Bob Sidebotham wrote: > The interaction shown below feels like a bug, or at least very much a > trap for the unwary. It's some sort of interaction between a context > manager and a generator, both of which can raise StopIteration. The > code is excised from a real application, so it's a bit artificial > looking. Nevertheless, it represents an entirely natural evolution of > some code (whereby the call to a context manager was added deep within > the code). Since it seems to work with open as the context manager, > but not with my own null context manager defined with > context_manager(), it feels like this is probably a bug in > contextmanager(). > > I did look in the archives, and it's possible this has been raised > before, but I wasn't clear on that. So apologies in advance if this is > already understood. > > This is using the stock distribution of python 2.5 on Linux 2.4. > > Comments? > > Bob Sidebotham > > bash-2.05b$ cat test.py > from __future__ import with_statement > > def gen(): > return# Should cause stop iteration > yield # Just to make this a generator, for this example > > # This works as expected. > try: > gen().next() # Should immediately raise StopIteration > except StopIteration: > print "Part 1: should reach here!" > else: > print "Part 1: shouldn't reach here!" > > # Now try the same thing, but with a context manager wrapping the > generator call. > # This also works as expected. > try: > with open("/etc/passwd") as pw: > gen().next() > except StopIteration: > print "Part 2: should reach here!" > else: > print "Part 2: shouldn't reach here!" > > # Finally, try this with our own home grown null context manager. > # This does not work as expected. > from contextlib import contextmanager > > @contextmanager > def ctxt(): > yield > > try: > with ctxt(): > gen().next() > except StopIteration: > print "Part 3: should reach here!" > else: > print "Part 3: shouldn't reach here!" > > bash-2.05b$ python test.py > Part 1: should reach here! > Part 2: should reach here! > Part 3: shouldn't reach here! > bash-2.05b$ > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Thu Apr 5 22:38:49 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 05 Apr 2007 16:38:49 -0400 Subject: [Python-Dev] context manager - generator interaction? In-Reply-To: <1f8770e0704051254g7bd0296coe27de33bfec0a762@mail.gmail.com > Message-ID: <5.1.1.6.0.20070405163124.029a2738@sparrow.telecommunity.com> At 03:54 PM 4/5/2007 -0400, Bob Sidebotham wrote: >The interaction shown below feels like a bug, or at least very much a >trap for the unwary. It's some sort of interaction between a context >manager and a generator, both of which can raise StopIteration. The >code is excised from a real application, so it's a bit artificial >looking. Nevertheless, it represents an entirely natural evolution of >some code (whereby the call to a context manager was added deep within >the code). Since it seems to work with open as the context manager, >but not with my own null context manager defined with >context_manager(), it feels like this is probably a bug in >contextmanager(). It actually appears to be a quirk in the "with" machinery itself, that only affects generator context managers because of their need to handle StopIteration specially. If you rewrite your context manager as a class with __enter__ and __exit__, the problem will go away. Checking what was happening via pdb, I found that the contextmanager's __exit__ is being called with a "value" of None, rather than with an exception *instance*. This fouls up GeneratorContextManager.__exit__'s special handling for StopIteration -- which was apparently never tested by any unit tests. I can see a few different ways to work around the problem in GeneratorContextManager itself, the simplest of which is to add these two lines right before the self.gen.throw() call: if type is StopIteration and value is None: value = type() It seems to me that this is a bit of a kludge, but trying to make it so that __exit__ always gets called with an instance is much more of a pain since it appears that the None a performance enhancement for simple errors like StopIteration, that avoids creating an instance. From fumanchu at amor.org Thu Apr 5 22:58:22 2007 From: fumanchu at amor.org (Robert Brewer) Date: Thu, 5 Apr 2007 13:58:22 -0700 Subject: [Python-Dev] context manager - generator interaction? In-Reply-To: Message-ID: <435DF58A933BA74397B42CDEB8145A860AC9BA84@ex9.hostedexchange.local> Guido van Rossum wrote: > Isn't this violating the rule that a try/except should only enclose > the smallest expression where the exception is expected? Yeah, and I keep finding myself wanting to hyperlink to that "rule" in the official docs, but it only seems to be written down in developer's heads. Can we get that into the Language Ref somewhere? Maybe on the http://docs.python.org/ref/try.html page? Robert Brewer System Architect Amor Ministries fumanchu at amor.org From tjreedy at udel.edu Fri Apr 6 02:26:37 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 5 Apr 2007 20:26:37 -0400 Subject: [Python-Dev] context manager - generator interaction? References: <435DF58A933BA74397B42CDEB8145A860AC9BA84@ex9.hostedexchange.local> Message-ID: "Robert Brewer" wrote in message news:435DF58A933BA74397B42CDEB8145A860AC9BA84 at ex9.hostedexchange.local... | Guido van Rossum wrote: | > Isn't this violating the rule that a try/except should only enclose | > the smallest expression where the exception is expected? | | Yeah, and I keep finding myself wanting to hyperlink to that "rule" in | the official docs, but it only seems to be written down in developer's | heads. Can we get that into the Language Ref somewhere? Maybe on the | http://docs.python.org/ref/try.html page? Submit a doc patch (normal text without Latex markup is fine) to SF with the sentence you would like, the location, and reason. From python at rcn.com Fri Apr 6 07:18:23 2007 From: python at rcn.com (Raymond Hettinger) Date: Thu, 5 Apr 2007 22:18:23 -0700 Subject: [Python-Dev] context manager - generator interaction? References: <435DF58A933BA74397B42CDEB8145A860AC9BA84@ex9.hostedexchange.local> Message-ID: <009401c7780b$038bb790$f001a8c0@RaymondLaptop1> > Guido van Rossum wrote: >> Isn't this violating the rule that a try/except should only enclose >> the smallest expression where the exception is expected? > > Yeah, and I keep finding myself wanting to hyperlink to that "rule" in > the official docs, but it only seems to be written down in developer's > heads. Can we get that into the Language Ref somewhere? Maybe on the > http://docs.python.org/ref/try.html page? I'll add a snippet to the tutorial with advice on try/except. I don't think it should go in the main documentation because it is really more of a suggestion or technique than an actual rule. Also, there are plenty of other situations where the "rule" doesn't apply. Raymond From guido at python.org Fri Apr 6 11:39:33 2007 From: guido at python.org (Guido van Rossum) Date: Fri, 6 Apr 2007 02:39:33 -0700 Subject: [Python-Dev] context manager - generator interaction? In-Reply-To: <009401c7780b$038bb790$f001a8c0@RaymondLaptop1> References: <435DF58A933BA74397B42CDEB8145A860AC9BA84@ex9.hostedexchange.local> <009401c7780b$038bb790$f001a8c0@RaymondLaptop1> Message-ID: On 4/5/07, Raymond Hettinger wrote: > > Guido van Rossum wrote: > >> Isn't this violating the rule that a try/except should only enclose > >> the smallest expression where the exception is expected? > > > > Yeah, and I keep finding myself wanting to hyperlink to that "rule" in > > the official docs, but it only seems to be written down in developer's > > heads. Can we get that into the Language Ref somewhere? Maybe on the > > http://docs.python.org/ref/try.html page? > > I'll add a snippet to the tutorial with advice on try/except. > > I don't think it should go in the main documentation because it is > really more of a suggestion or technique than an actual rule. > Also, there are plenty of other situations where the "rule" doesn't apply. I see a lot of code written by Googlers who are not (yet) experienced Python users. Overly wide try/except clauses are a common mistake, and it is nearly always easy to apply this rule. I would argue, as a style issue, for making this rule the norm, and require deviations to be thought through very carefully. This rule has no place in a pure language *reference* manual. But it certainly deserves mention in any form of more practical documentation, be it a tutorial or a more advanced programming manual. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steve at holdenweb.com Fri Apr 6 13:18:26 2007 From: steve at holdenweb.com (Steve Holden) Date: Fri, 06 Apr 2007 07:18:26 -0400 Subject: [Python-Dev] context manager - generator interaction? In-Reply-To: References: <435DF58A933BA74397B42CDEB8145A860AC9BA84@ex9.hostedexchange.local> <009401c7780b$038bb790$f001a8c0@RaymondLaptop1> Message-ID: <46162C82.2010300@holdenweb.com> Guido van Rossum wrote: > On 4/5/07, Raymond Hettinger wrote: >>> Guido van Rossum wrote: >>>> Isn't this violating the rule that a try/except should only enclose >>>> the smallest expression where the exception is expected? >>> Yeah, and I keep finding myself wanting to hyperlink to that "rule" in >>> the official docs, but it only seems to be written down in developer's >>> heads. Can we get that into the Language Ref somewhere? Maybe on the >>> http://docs.python.org/ref/try.html page? >> I'll add a snippet to the tutorial with advice on try/except. >> >> I don't think it should go in the main documentation because it is >> really more of a suggestion or technique than an actual rule. >> Also, there are plenty of other situations where the "rule" doesn't apply. > > I see a lot of code written by Googlers who are not (yet) experienced > Python users. Overly wide try/except clauses are a common mistake, and > it is nearly always easy to apply this rule. I would argue, as a style > issue, for making this rule the norm, and require deviations to be > thought through very carefully. > > This rule has no place in a pure language *reference* manual. But it > certainly deserves mention in any form of more practical > documentation, be it a tutorial or a more advanced programming manual. > PEP 8 would (also? I don't see this being relevant to the reference manual, but that's not a biggie) seem a logical place for this information to go, since it deals with try-except. I'd suggest something along the lines of: A frequent beginner's error is to make the scope of a try clause too broad, covering a larger span of code that necessary so that the except clause is triggered by unanticipated exceptions. Try to ensure that a try clause applies to the fewest possible statements to avoid these issues. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://del.icio.us/steve.holden Recent Ramblings http://holdenweb.blogspot.com From skip at pobox.com Fri Apr 6 16:31:53 2007 From: skip at pobox.com (skip at pobox.com) Date: Fri, 6 Apr 2007 09:31:53 -0500 Subject: [Python-Dev] context manager - generator interaction? In-Reply-To: References: <435DF58A933BA74397B42CDEB8145A860AC9BA84@ex9.hostedexchange.local> <009401c7780b$038bb790$f001a8c0@RaymondLaptop1> Message-ID: <17942.23001.94261.696709@montanaro.dyndns.org> Guido> This rule has no place in a pure language *reference* manual. But Guido> it certainly deserves mention in any form of more practical Guido> documentation, be it a tutorial or a more advanced programming Guido> manual. PEP 8 anyone? Skip From fdrake at acm.org Fri Apr 6 16:40:44 2007 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Fri, 6 Apr 2007 10:40:44 -0400 Subject: [Python-Dev] context manager - generator interaction? In-Reply-To: <17942.23001.94261.696709@montanaro.dyndns.org> References: <435DF58A933BA74397B42CDEB8145A860AC9BA84@ex9.hostedexchange.local> <17942.23001.94261.696709@montanaro.dyndns.org> Message-ID: <200704061040.46653.fdrake@acm.org> On where to write guidelines about exception handling scope: Guido wrote: > This rule has no place in a pure language *reference* manual. But > it certainly deserves mention in any form of more practical > documentation, be it a tutorial or a more advanced programming > manual. On Friday 06 April 2007 10:31, skip at pobox.com wrote: > PEP 8 anyone? New users should be exposed sooner than this; most will never read any PEP. The tutorial seems like a good place. This is general good programming practice we're talking about here, not style. -Fred -- Fred L. Drake, Jr. From barry at python.org Fri Apr 6 16:58:41 2007 From: barry at python.org (Barry Warsaw) Date: Fri, 6 Apr 2007 10:58:41 -0400 Subject: [Python-Dev] context manager - generator interaction? In-Reply-To: <200704061040.46653.fdrake@acm.org> References: <435DF58A933BA74397B42CDEB8145A860AC9BA84@ex9.hostedexchange.local> <17942.23001.94261.696709@montanaro.dyndns.org> <200704061040.46653.fdrake@acm.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Apr 6, 2007, at 10:40 AM, Fred L. Drake, Jr. wrote: > On Friday 06 April 2007 10:31, skip at pobox.com wrote: >> PEP 8 anyone? > > New users should be exposed sooner than this; most will never read > any PEP. > The tutorial seems like a good place. This is general good > programming > practice we're talking about here, not style. I'm not so sure that's true about PEP 8. Several organizations that I know of link their own "Python Coding Standards" directly to that PEP, and expect their devs to read it and adhere to it, possibly with a few local exceptions. It might even be worthwhile to give PEP 8 more exposure in the tutorial or on the documentation ToC page. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (Darwin) iD8DBQFGFmAl2YZpQepbvXERAs7XAJ9IAjcpiyeh6x1c6YDdpj8jmQoKRwCgse+a R/I6zIMjvBKmIqfqbrPrtFo= =5x7r -----END PGP SIGNATURE----- From phd at phd.pp.ru Fri Apr 6 17:06:44 2007 From: phd at phd.pp.ru (Oleg Broytmann) Date: Fri, 6 Apr 2007 19:06:44 +0400 Subject: [Python-Dev] More exposure for PEP8 (was: context manager - generator interaction?) In-Reply-To: References: <435DF58A933BA74397B42CDEB8145A860AC9BA84@ex9.hostedexchange.local> <17942.23001.94261.696709@montanaro.dyndns.org> <200704061040.46653.fdrake@acm.org> Message-ID: <20070406150644.GF17830@phd.pp.ru> On Fri, Apr 06, 2007 at 10:58:41AM -0400, Barry Warsaw wrote: > It might even be worthwhile to give PEP 8 > more exposure in the tutorial or on the documentation ToC page. And bundle the PEP with the distributed documentation. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From guido at python.org Fri Apr 6 17:09:45 2007 From: guido at python.org (Guido van Rossum) Date: Fri, 6 Apr 2007 08:09:45 -0700 Subject: [Python-Dev] context manager - generator interaction? In-Reply-To: References: <435DF58A933BA74397B42CDEB8145A860AC9BA84@ex9.hostedexchange.local> <17942.23001.94261.696709@montanaro.dyndns.org> <200704061040.46653.fdrake@acm.org> Message-ID: I've added a paragraph to the PEP, with examples. On 4/6/07, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Apr 6, 2007, at 10:40 AM, Fred L. Drake, Jr. wrote: > > > On Friday 06 April 2007 10:31, skip at pobox.com wrote: > >> PEP 8 anyone? > > > > New users should be exposed sooner than this; most will never read > > any PEP. > > The tutorial seems like a good place. This is general good > > programming > > practice we're talking about here, not style. > > I'm not so sure that's true about PEP 8. Several organizations that > I know of link their own "Python Coding Standards" directly to that > PEP, and expect their devs to read it and adhere to it, possibly with > a few local exceptions. It might even be worthwhile to give PEP 8 > more exposure in the tutorial or on the documentation ToC page. > > - -Barry > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.6 (Darwin) > > iD8DBQFGFmAl2YZpQepbvXERAs7XAJ9IAjcpiyeh6x1c6YDdpj8jmQoKRwCgse+a > R/I6zIMjvBKmIqfqbrPrtFo= > =5x7r > -----END PGP SIGNATURE----- > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Fri Apr 6 21:16:20 2007 From: python at rcn.com (Raymond Hettinger) Date: Fri, 6 Apr 2007 12:16:20 -0700 Subject: [Python-Dev] [Python-checkins] Python Regression Test Failures opt (1) References: <20070406081355.GA24121@python.psfb.org> Message-ID: <00d201c77880$10cd8580$f001a8c0@RaymondLaptop1> I believe the failures started with Facundo Batista's checkin on April 4th: ------------------------------------- Author: facundo.batista Date: Wed Apr 4 16:10:40 2007 New Revision: 54681 Modified: python/trunk/Lib/test/test_socket_ssl.py Log: Fixed the way that the .pem files are looked for, and changed how to kill the process in win32 to use the _handle attribute. Modified: python/trunk/Lib/test/test_socket_ssl.py ----- Original Message ----- From: Brett Cannon To: nnorwitz at gmail.com Cc: python-checkins at python.org Sent: Friday, April 06, 2007 11:27 AM Subject: Re: [Python-checkins] Python Regression Test Failures opt (1) On 4/6/07, Neal Norwitz wrote: [SNIP] test_socket_ssl test test_socket_ssl failed -- Traceback (most recent call last): File "/tmp/python-test/local/lib/python2.6/test/test_socket_ssl.py", line 117, in testBasic self.assertEqual (i, "Foo\n") AssertionError: 'ERRO' != 'Foo\n' test_socketserver test_socketserver skipped -- Use of the `network' resource not enabled test_softspace test_sort Exception in thread Thread-43: Traceback (most recent call last): File "/tmp/python-test/local/lib/python2.6/threading.py", line 460, in __bootstrap self.run() File "/tmp/python-test/local/lib/python2.6/test/test_socket_ssl.py", line 163, in run self.s.stdin.write(l) IOError: [Errno 32] Broken pipe Anyone know what caused these two tests to start failing? -Brett ------------------------------------------------------------------------------ _______________________________________________ Python-checkins mailing list Python-checkins at python.org http://mail.python.org/mailman/listinfo/python-checkins -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070406/e5116302/attachment.htm From and-dev at doxdesk.com Sat Apr 7 17:02:13 2007 From: and-dev at doxdesk.com (Andrew Clover) Date: Sat, 07 Apr 2007 16:02:13 +0100 Subject: [Python-Dev] minidom and DOM level 2 In-Reply-To: References: <4603942D.4060508@v.loewis.de> Message-ID: <4617B275.4030406@doxdesk.com> Jason Orendorff wrote: > OK, I think this is worthwhile then. :) I'll read the spec and submit > a patch. You're planning to implement EntityReference in minidom? That'll be fun! :-) One of the nastier corners of DOM and XML in general. > I'd be happy to do some diffing between the implementation, > documentation, tests, and the Recommendation I hacked up an experimental test harness for the W3 DOM Test Suite in order to test my own imp; you might find it useful: http://doxdesk.com/software/py/domts.html The TS is far from definitive, but its tests for Level 1 and 2 are to the best of my knowledge accurate. Incidentally minidom falls far short of passing even Level 1 Core for more reasons than omission of EntityReference. I noted the main known problems with it here: http://pyxml.sourceforge.net/topics/compliance.html Good luck! -- And Clover mailto:and at doxdesk.com http://www.doxdesk.com/ From pythondev at aerojockey.com Sat Apr 7 21:26:28 2007 From: pythondev at aerojockey.com (Carl Banks) Date: Sat, 07 Apr 2007 15:26:28 -0400 Subject: [Python-Dev] Extended buffer PEP In-Reply-To: <4612DC7C.5090706@ee.byu.edu> References: <4612DC7C.5090706@ee.byu.edu> Message-ID: <4617F064.6020605@aerojockey.com> Only one concern: > typedef int (*getbufferproc)(PyObject *obj, struct bufferinfo *view) I'd like to see it accept a flags argument over what kind of buffer it's allowed to return. I'd rather not burden the user to check all the entries in bufferinfo to make sure it doesn't get something unexpected. I imagine most uses of buffer protocol would be for direct, one-dimensional arrays of bytes with no striding. It's not clear whether read-only or read-write should be the least common denominator, so require at least one of these flags: Py_BUF_READONLY Py_PUF_READWRITE Then allow any of these flags to allow more complex access: Py_BUF_MULTIDIM - allows strided and multidimensional arrays Py_BUF_INDIRECT - allows indirect buffers (implies Py_BUF_MULTIDIM) An object is allowed to return a simpler array than requested, but not more complex. If you allow indirect buffers, you might still get a one-dimensional array of bytes. Other than that, I would add a note about the other things considered and rejected (the old prototype for getbufferproc, the delegated buffer object). List whether to backport the buffer protocol to 2.6 as an open question. Then submit it as a real PEP. I believe this idea has run its course as PEP XXX and needs a real number. (I was intending to start making patches for the Py3K library modules as soon as that happened.) Carl Banks Travis Oliphant wrote: > > Here is my "final" draft of the extended buffer interface PEP. > For those who have been following the discussion, I eliminated the > releaser object and the lock-buffer function. I decided that there is > enough to explain with the new striding and sub-offsets without the > added confusion of releasing buffers, especially when it is not clear > what is to be gained by such complexity except a few saved lines of code. > > The striding and sub-offsets, however, allow extension module writers to > write code (say video and image processing code or scientific computing > code or data-base processing code) that works on any object exposing the > buffer interface. I think this will be of great benefit and so is worth > the complexity. > > This will take some work to get implemented for Python 3k. I could use > some help with this in order to speed up the process. I'm working right > now on the extensions to the struct module until the rest is approved. > > Thank you for any and all comments: > > -Travis From jason.orendorff at gmail.com Sat Apr 7 21:43:28 2007 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Sat, 7 Apr 2007 15:43:28 -0400 Subject: [Python-Dev] minidom and DOM level 2 In-Reply-To: <4617B275.4030406@doxdesk.com> References: <4603942D.4060508@v.loewis.de> <4617B275.4030406@doxdesk.com> Message-ID: On 4/7/07, Andrew Clover wrote: > Jason Orendorff wrote: > > OK, I think this is worthwhile then. :) I'll read the spec and submit > > a patch. > > You're planning to implement EntityReference in minidom? That'll be fun! > :-) One of the nastier corners of DOM and XML in general. Mmm. So I'm finding. EntityReferences seem to force detailed knowledge of entity handling into the DOM implementation. expat doesn't expose the information in a particularly helpful way. In a word, blaargh. I'd be happy to set this aside and work on Level 1 compliance: > Incidentally minidom falls far short of passing even Level 1 Core for > more reasons than omission of EntityReference. I noted the main known > problems with it here: > http://pyxml.sourceforge.net/topics/compliance.html Very nice. Thanks for posting this. I don't suppose you'd be willing to update it for Python 2.5, would you? Martin, have you looked at this? Some of these might be hard to fix, given expat. -j From steven.bethard at gmail.com Sat Apr 7 23:53:08 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Sat, 07 Apr 2007 15:53:08 -0600 Subject: [Python-Dev] function for counting items in a sequence In-Reply-To: <1hw2ea3.1mmluqy1an8gx4N%aleax@mac.com> References: <46146F75.9050904@gmail.com> <1hw2ea3.1mmluqy1an8gx4N%aleax@mac.com> Message-ID: <461812C4.9080904@gmail.com> Alex Martelli wrote: > Steven Bethard wrote: > >> Alex Martelli wrote: >> > If we had a "turn sequence into bag" function somewhere >> > (and it might be worth having it for other reasons): >> > >> > def bagit(seq): >> > import collections >> > d = collections.defaultdict(int) >> > for x in seq: d[x] += 1 >> > return d >> >> I use this function all the time -- I call it dicttools.count(). I'd >> love to see this appear in the collections module or somewhere else in >> the stdlib. > > Maybe you should propose it in python-dev -- it does seem a reasonable > utility addition to the collections module, after all. Here's a patch implementing collections.counts() as suggested above: http://bugs.python.org/1696199 Example usage, from the docstring:: >>> items = 'acabbacba' >>> item_counts = counts(items) >>> for item in 'abcd': ... print item, item_counts[item] ... a 4 b 3 c 2 d 0 The documentation is a little terse, but the function was so simple, I wasn't really sure what more to say. STeVe From martin at v.loewis.de Sun Apr 8 00:10:08 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 08 Apr 2007 00:10:08 +0200 Subject: [Python-Dev] minidom and DOM level 2 In-Reply-To: References: <4603942D.4060508@v.loewis.de> <4617B275.4030406@doxdesk.com> Message-ID: <461816C0.1010204@v.loewis.de> > Martin, have you looked at this? Only just now. I assume Andrew is right on these, although one would have to verify each and every one, reading the spec, reading the documentation, reading the source, testing, fixing. Very time-consuming. In any case, the *claim* certainly is that minidom supports level 2 core. Any proof to the contrary indicates a bug; patches are welcome. Regards, Martin From jason.orendorff at gmail.com Sun Apr 8 01:00:35 2007 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Sat, 7 Apr 2007 19:00:35 -0400 Subject: [Python-Dev] minidom and DOM level 2 In-Reply-To: <461816C0.1010204@v.loewis.de> References: <4603942D.4060508@v.loewis.de> <4617B275.4030406@doxdesk.com> <461816C0.1010204@v.loewis.de> Message-ID: On 4/7/07, "Martin v. L?wis" wrote: > In any case, the *claim* certainly is that minidom supports > level 2 core. Any proof to the contrary indicates a bug; > patches are welcome. OK-- I'll work on this. I can fix the easy ones, anyway. -j From status at bugs.python.org Sun Apr 8 02:00:50 2007 From: status at bugs.python.org (Tracker) Date: Sun, 8 Apr 2007 00:00:50 +0000 (UTC) Subject: [Python-Dev] Summary of Tracker Issues Message-ID: <20070408000050.7A4AE78043@psf.localdomain> ACTIVITY SUMMARY (04/01/07 - 04/08/07) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1647 open ( +0) / 8584 closed ( +0) / 10231 total ( +0) Average duration of open issues: 758 days. Median duration of open issues: 707 days. Open Issues Breakdown open 1647 ( +0) pending 0 ( +0) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070408/137bb922/attachment.html From steven.bethard at gmail.com Sun Apr 8 03:01:10 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Sat, 7 Apr 2007 19:01:10 -0600 Subject: [Python-Dev] function for counting items in a sequence In-Reply-To: <461812C4.9080904@gmail.com> References: <46146F75.9050904@gmail.com> <1hw2ea3.1mmluqy1an8gx4N%aleax@mac.com> <461812C4.9080904@gmail.com> Message-ID: On 4/7/07, Steven Bethard wrote: > Here's a patch implementing collections.counts() as suggested above: > > http://bugs.python.org/1696199 > > Example usage, from the docstring:: > > >>> items = 'acabbacba' > >>> item_counts = counts(items) > >>> for item in 'abcd': > ... print item, item_counts[item] > ... > a 4 > b 3 > c 2 > d 0 Guido commented in the tracker that it would be worth discussing whether that last item (``item_counts['d']``) should return 0 (as a defaultdict would) or raise KeyError (as a dict would). Anyone have a good motivation for one approach or the other? STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From rhamph at gmail.com Sun Apr 8 03:04:08 2007 From: rhamph at gmail.com (Adam Olsen) Date: Sat, 7 Apr 2007 19:04:08 -0600 Subject: [Python-Dev] function for counting items in a sequence In-Reply-To: <461812C4.9080904@gmail.com> References: <46146F75.9050904@gmail.com> <1hw2ea3.1mmluqy1an8gx4N%aleax@mac.com> <461812C4.9080904@gmail.com> Message-ID: On 4/7/07, Steven Bethard wrote: > Here's a patch implementing collections.counts() as suggested above: The name doesn't make it obvious to me what's going on. Maybe countunique()? Some other options are countdistinct() and countduplicates(). > >>> items = 'acabbacba' > >>> item_counts = counts(items) > >>> for item in 'abcd': > ... print item, item_counts[item] > ... > a 4 > b 3 > c 2 > d 0 Would become: >>> items = 'acabbacba' >>> counts = countunique(items) >>> for item in 'abcd': ... print item, counts[item] ... a 4 b 3 c 2 d 0 -- Adam Olsen, aka Rhamphoryncus From brett at python.org Sun Apr 8 03:31:31 2007 From: brett at python.org (Brett Cannon) Date: Sat, 7 Apr 2007 18:31:31 -0700 Subject: [Python-Dev] function for counting items in a sequence In-Reply-To: References: <46146F75.9050904@gmail.com> <1hw2ea3.1mmluqy1an8gx4N%aleax@mac.com> <461812C4.9080904@gmail.com> Message-ID: On 4/7/07, Steven Bethard wrote: > > On 4/7/07, Steven Bethard wrote: > > Here's a patch implementing collections.counts() as suggested above: > > > > http://bugs.python.org/1696199 > > > > Example usage, from the docstring:: > > > > >>> items = 'acabbacba' > > >>> item_counts = counts(items) > > >>> for item in 'abcd': > > ... print item, item_counts[item] > > ... > > a 4 > > b 3 > > c 2 > > d 0 > > Guido commented in the tracker that it would be worth discussing > whether that last item (``item_counts['d']``) should return 0 (as a > defaultdict would) or raise KeyError (as a dict would). > > Anyone have a good motivation for one approach or the other? I say 0 if it is really meant to represent a count as the absense of something is 0. The object returned is being viewed as an object representing the count of items, not specifically as a dict. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070407/a14cfdf9/attachment-0001.html From greg.ewing at canterbury.ac.nz Sun Apr 8 03:29:35 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 08 Apr 2007 13:29:35 +1200 Subject: [Python-Dev] function for counting items in a sequence In-Reply-To: References: <46146F75.9050904@gmail.com> <1hw2ea3.1mmluqy1an8gx4N%aleax@mac.com> <461812C4.9080904@gmail.com> Message-ID: <4618457F.6050703@canterbury.ac.nz> Steven Bethard wrote: > Guido commented in the tracker that it would be worth discussing > whether that last item (``item_counts['d']``) should return 0 (as a > defaultdict would) or raise KeyError (as a dict would). If you've asked for a count of 'd's, and there aren't any 'd's, the most sensible answer would seem to be 0, not an exception. Also it would be a useful property if you were guaranteed that all items in the input sequence are usable as keys in the result without causing an exception. -- Greg From greg.ewing at canterbury.ac.nz Sun Apr 8 03:31:26 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 08 Apr 2007 13:31:26 +1200 Subject: [Python-Dev] function for counting items in a sequence In-Reply-To: References: <46146F75.9050904@gmail.com> <1hw2ea3.1mmluqy1an8gx4N%aleax@mac.com> <461812C4.9080904@gmail.com> Message-ID: <461845EE.2040908@canterbury.ac.nz> Adam Olsen wrote: > The name doesn't make it obvious to me what's going on. Maybe > countunique()? That doesn't sound any more obvious to me. counteach? countall? -- Greg From guido at python.org Sun Apr 8 04:00:42 2007 From: guido at python.org (Guido van Rossum) Date: Sat, 7 Apr 2007 19:00:42 -0700 Subject: [Python-Dev] function for counting items in a sequence In-Reply-To: References: <46146F75.9050904@gmail.com> <1hw2ea3.1mmluqy1an8gx4N%aleax@mac.com> <461812C4.9080904@gmail.com> Message-ID: On 4/7/07, Adam Olsen wrote: > On 4/7/07, Steven Bethard wrote: > > Here's a patch implementing collections.counts() as suggested above: > > The name doesn't make it obvious to me what's going on. Maybe > countunique()? Some other options are countdistinct() and > countduplicates(). -1. I like the name that Steve proposed just fine; "counts" perfectly explain that it produces counts of items. If there couldn't be duplicates then "counts" wouldn't make sense since the only outcomes would be True or False (present or not). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From bioinformed at gmail.com Sun Apr 8 05:12:21 2007 From: bioinformed at gmail.com (Kevin Jacobs ) Date: Sat, 7 Apr 2007 23:12:21 -0400 Subject: [Python-Dev] function for counting items in a sequence In-Reply-To: References: <46146F75.9050904@gmail.com> <1hw2ea3.1mmluqy1an8gx4N%aleax@mac.com> <461812C4.9080904@gmail.com> Message-ID: <2e1434c10704072012n41be5de3w641a19611227406c@mail.gmail.com> On 4/7/07, Guido van Rossum wrote: > > On 4/7/07, Adam Olsen wrote: > > On 4/7/07, Steven Bethard wrote: > > > Here's a patch implementing collections.counts() as suggested above: > > > > The name doesn't make it obvious to me what's going on. Maybe > > countunique()? Some other options are countdistinct() and > > countduplicates(). > > -1. I like the name that Steve proposed just fine; "counts" perfectly > explain that it produces counts of items. If there couldn't be > duplicates then "counts" wouldn't make sense since the only outcomes > would be True or False (present or not). > I call that function 'tally'. -Kevin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070407/f1ee686a/attachment.htm From python at rcn.com Sun Apr 8 09:56:07 2007 From: python at rcn.com (Raymond Hettinger) Date: Sun, 8 Apr 2007 00:56:07 -0700 Subject: [Python-Dev] function for counting items in a sequence Message-ID: <00ca01c779b3$5e2469a0$f201a8c0@RaymondLaptop1> Instead of a separate function in collections, it may be sensible to stick with the familiar and provide a dictionary contructor for the commonly encounter use case of counting things: @classmethod def fromcount(cls, iterable): d = cls() for elem in iterable: d[elem] = d.get(elem, 0) + 1 return d Raymond From pogonyshev at gmx.net Sun Apr 8 14:10:32 2007 From: pogonyshev at gmx.net (Paul Pogonyshev) Date: Sun, 8 Apr 2007 15:10:32 +0300 Subject: [Python-Dev] concerns regarding callable() method Message-ID: <200704081510.32382.pogonyshev@gmx.net> Hi, I have seen in PEP 3100 that callable() function is planned to be removed in Python 3000 with this replacement: "just call the object and catch the exception???". For one, the object (if it is callable) can raise exception itself, so you need to somehow to differentiate between exception raised inside its __call__ and exception raised if object is not callable to begin with. Additionally consider something like something.set_callback (x) Assume that set_callback() wants to check if `x' is callable at all, to raise exception early and make error tracking easier. Currently, you can assert callable (x) But if callable() is removed, there is no apparent replacement. Of course, you cannot call `x' since it might have side-effects or be slow etc. Please reconsider removal of callable() or provide an adequate replacement. Paul From ark at acm.org Sun Apr 8 15:58:12 2007 From: ark at acm.org (Andrew Koenig) Date: Sun, 8 Apr 2007 09:58:12 -0400 Subject: [Python-Dev] concerns regarding callable() method In-Reply-To: <200704081510.32382.pogonyshev@gmx.net> Message-ID: <000001c779e5$f57efb80$6402a8c0@arkdesktop> > I have seen in PEP 3100 that callable() function is planned to be > removed in Python 3000 with this replacement: "just call the object > and catch the exception???". For one, the object (if it is > callable) can raise exception itself, so you need to somehow to > differentiate between exception raised inside its __call__ and > exception raised if object is not callable to begin with. I seem to recall bringing up the same issue a while ago; at the time, the answer was that if you need it, you can write your own: def callable(x): return hasattr(x, '__call__') My own preference would be for such queries to be defined abstractly as a built-in part of the language, but apparently my preference is out of sync with the community in this particular respect. From guido at python.org Sun Apr 8 17:39:30 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 8 Apr 2007 08:39:30 -0700 Subject: [Python-Dev] concerns regarding callable() method In-Reply-To: <200704081510.32382.pogonyshev@gmx.net> References: <200704081510.32382.pogonyshev@gmx.net> Message-ID: On 4/8/07, Paul Pogonyshev wrote: > I have seen in PEP 3100 that callable() function is planned to be > removed in Python 3000 with this replacement: "just call the object > and catch the exception???". For one, the object (if it is > callable) can raise exception itself, so you need to somehow to > differentiate between exception raised inside its __call__ and > exception raised if object is not callable to begin with. Why? > Additionally consider something like > > something.set_callback (x) > > Assume that set_callback() wants to check if `x' is callable at > all, to raise exception early and make error tracking easier. > Currently, you can > > assert callable (x) > > But if callable() is removed, there is no apparent replacement. Of > course, you cannot call `x' since it might have side-effects or be > slow etc. assert hasattr(x, '__call__') I note that callable() was introduced before all callable objects had a __call__ attribute. This is no longer the case, so it's not needed. > Please reconsider removal of callable() or provide an adequate > replacement. What if someone passes a callable that doesn't have the expected signature? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pogonyshev at gmx.net Sun Apr 8 17:56:42 2007 From: pogonyshev at gmx.net (Paul Pogonyshev) Date: Sun, 8 Apr 2007 18:56:42 +0300 Subject: [Python-Dev] concerns regarding callable() method In-Reply-To: References: <200704081510.32382.pogonyshev@gmx.net> Message-ID: <200704081856.42362.pogonyshev@gmx.net> Guido van Rossum wrote: > On 4/8/07, Paul Pogonyshev wrote: > > Additionally consider something like > > > > something.set_callback (x) > > > > Assume that set_callback() wants to check if `x' is callable at > > all, to raise exception early and make error tracking easier. > > Currently, you can > > > > assert callable (x) > > > > But if callable() is removed, there is no apparent replacement. Of > > course, you cannot call `x' since it might have side-effects or be > > slow etc. > > assert hasattr(x, '__call__') > > I note that callable() was introduced before all callable objects had > a __call__ attribute. This is no longer the case, so it's not needed. I just didn't think about that possibility. If that works the same way, callable() is just a sugar and not something unimplementable in other ways. Therefore, my objection is discarded. (But PEP 3100 should probably be update to mention this, otherwise you may get this complaint again ;) > > Please reconsider removal of callable() or provide an adequate > > replacement. > > What if someone passes a callable that doesn't have the expected signature? Well, I don't know a way to catch such situations now, so removing callable() will not make it worse (even if you don't know about hasattr trick above.) Paul From guido at python.org Sun Apr 8 17:55:26 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 8 Apr 2007 08:55:26 -0700 Subject: [Python-Dev] concerns regarding callable() method In-Reply-To: <200704081856.42362.pogonyshev@gmx.net> References: <200704081510.32382.pogonyshev@gmx.net> <200704081856.42362.pogonyshev@gmx.net> Message-ID: On 4/8/07, Paul Pogonyshev wrote: > Guido van Rossum wrote: > > What if someone passes a callable that doesn't have the expected signature? > > Well, I don't know a way to catch such situations now, so removing > callable() will not make it worse (even if you don't know about hasattr > trick above.) My point is that it's futile to use callable() -- even if it passes, you have no assurance that you actually have a valid callback. So why bother with it at all? It's counter to the spirit of Python. If someone passes you a bad callback, they will see a traceback when you call it. Then they fix their program. That's how it's supposed to work. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From snaury at gmail.com Sun Apr 8 17:57:47 2007 From: snaury at gmail.com (Alexey Borzenkov) Date: Sun, 8 Apr 2007 19:57:47 +0400 Subject: [Python-Dev] concerns regarding callable() method In-Reply-To: <200704081856.42362.pogonyshev@gmx.net> References: <200704081510.32382.pogonyshev@gmx.net> <200704081856.42362.pogonyshev@gmx.net> Message-ID: On 4/8/07, Paul Pogonyshev wrote: > > assert hasattr(x, '__call__') > > > > I note that callable() was introduced before all callable objects had > > a __call__ attribute. This is no longer the case, so it's not needed. > I just didn't think about that possibility. If that works the same way, > callable() is just a sugar and not something unimplementable in other > ways. Therefore, my objection is discarded. (But PEP 3100 should probably > be update to mention this, otherwise you may get this complaint again ;) I whole-heartedly agree here, because people who start learning python (like me some time ago) usually learn that it's bad to test for __call__ (can't remember where I read about that though), and that we should always use callable() to be on a safe side. When I first heard that callable() was going to be removed I kinda panicked myself, it's good to know that at least there's still a way to be sure that object is callable or not... From snaury at gmail.com Sun Apr 8 18:01:38 2007 From: snaury at gmail.com (Alexey Borzenkov) Date: Sun, 8 Apr 2007 20:01:38 +0400 Subject: [Python-Dev] concerns regarding callable() method In-Reply-To: References: <200704081510.32382.pogonyshev@gmx.net> <200704081856.42362.pogonyshev@gmx.net> Message-ID: On 4/8/07, Guido van Rossum wrote: > On 4/8/07, Paul Pogonyshev wrote: > > Guido van Rossum wrote: > > > What if someone passes a callable that doesn't have the expected signature? > > Well, I don't know a way to catch such situations now, so removing > > callable() will not make it worse (even if you don't know about hasattr > > trick above.) > My point is that it's futile to use callable() -- even if it passes, > you have no assurance that you actually have a valid callback. So why > bother with it at all? It's counter to the spirit of Python. If > someone passes you a bad callback, they will see a traceback when you > call it. Then they fix their program. That's how it's supposed to > work. But what if you need to do different things based on argument is callable or not? Take for example "Dependency Injection" recipe: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/413268 It uses callable to differentiate whether it needs to use object as singleton or to instantiate it on each request. I'm sure there might be other uses for callable when it's really useful. From pogonyshev at gmx.net Sun Apr 8 18:22:57 2007 From: pogonyshev at gmx.net (Paul Pogonyshev) Date: Sun, 8 Apr 2007 19:22:57 +0300 Subject: [Python-Dev] concerns regarding callable() method In-Reply-To: References: <200704081510.32382.pogonyshev@gmx.net> <200704081856.42362.pogonyshev@gmx.net> Message-ID: <200704081922.57275.pogonyshev@gmx.net> Guido van Rossum wrote: > On 4/8/07, Paul Pogonyshev wrote: > > Guido van Rossum wrote: > > > What if someone passes a callable that doesn't have the expected signature? > > > > Well, I don't know a way to catch such situations now, so removing > > callable() will not make it worse (even if you don't know about hasattr > > trick above.) > > My point is that it's futile to use callable() -- even if it passes, > you have no assurance that you actually have a valid callback. So why > bother with it at all? It's counter to the spirit of Python. If > someone passes you a bad callback, they will see a traceback when you > call it. Then they fix their program. That's how it's supposed to > work. I have no problems with Python being untyped. But I want that error stack traces provide some useful information as possible with reasonable effort and that errors happen as early as possible. In particular, stack trace should mention that error occured when you passed something wrong to set_callback() call and not in some obscure place 200 lines later, because otherwise it will only obfuscate error reason. Yes, assert will not catch all errors, but at least it will some. I consider it perfectly acceptable that you cannot test signature, because (since Python is untyped) you could only test number of arguments and even that would probably involve dumb syntax. So, I understand such assert will not catch all errors. But I don't want to remove it, since I find catching at least some errors (e.g. like passing None) an improvement over catching no errors. Paul From oliphant.travis at ieee.org Sun Apr 8 19:17:12 2007 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Sun, 08 Apr 2007 11:17:12 -0600 Subject: [Python-Dev] Extended buffer PEP In-Reply-To: <4617F064.6020605@aerojockey.com> References: <4612DC7C.5090706@ee.byu.edu> <4617F064.6020605@aerojockey.com> Message-ID: <46192398.9000001@ieee.org> Carl Banks wrote: > Only one concern: > > > typedef int (*getbufferproc)(PyObject *obj, struct bufferinfo > *view) > > > I'd like to see it accept a flags argument over what kind of buffer > it's allowed to return. I'd rather not burden the user to check all > the entries in bufferinfo to make sure it doesn't get something > unexpected. Yes, I agree. We had something like that at one point. > > I imagine most uses of buffer protocol would be for direct, > one-dimensional arrays of bytes with no striding. It's not clear > whether read-only or read-write should be the least common > denominator, so require at least one of these flags: > > Py_BUF_READONLY > Py_PUF_READWRITE > > Then allow any of these flags to allow more complex access: > > Py_BUF_MULTIDIM - allows strided and multidimensional arrays > Py_BUF_INDIRECT - allows indirect buffers (implies Py_BUF_MULTIDIM) > > An object is allowed to return a simpler array than requested, but not > more complex. If you allow indirect buffers, you might still get a > one-dimensional array of bytes. > > > Other than that, I would add a note about the other things considered > and rejected (the old prototype for getbufferproc, the delegated > buffer object). List whether to backport the buffer protocol to 2.6 > as an open question. Thanks for the suggestions. > > Then submit it as a real PEP. I believe this idea has run its course > as PEP XXX and needs a real number. How does one do that. Who assigns the number? I thought I "had" submitted it as a real PEP. -Travis From jcarlson at uci.edu Sun Apr 8 19:52:56 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Sun, 08 Apr 2007 10:52:56 -0700 Subject: [Python-Dev] concerns regarding callable() method In-Reply-To: References: <200704081856.42362.pogonyshev@gmx.net> Message-ID: <20070408100150.6273.JCARLSON@uci.edu> "Guido van Rossum" wrote: > On 4/8/07, Paul Pogonyshev wrote: > > Guido van Rossum wrote: > > > What if someone passes a callable that doesn't have the expected signature? > > > > Well, I don't know a way to catch such situations now, so removing > > callable() will not make it worse (even if you don't know about hasattr > > trick above.) > > My point is that it's futile to use callable() -- even if it passes, > you have no assurance that you actually have a valid callback. So why > bother with it at all? It's counter to the spirit of Python. If > someone passes you a bad callback, they will see a traceback when you > call it. Then they fix their program. That's how it's supposed to > work. The point of using callable(x), or it's equivalent now of hasattr(x, '__call__') is to reduce reduce the time/lines of code between when an error occurs and when it is reported. Errors should never pass silently. While we currently cannot verify that some callable takes the proper arguments, number of arguments, etc., we *can* verify that it is callable. I think this is a good thing, as allowing the assignment of a non-callable to a name that is supposed to be callable is the silent passing of an error. With relatively minimal effort in Python 3, one could use a function signature object (PEPs 3107 and 362) to verify that a callable takes the proper number of arguments, expected keyword arguments, etc., which while still not allowing one to verify that the implementation of a callback is correct (technically impossible), it does get us closer to being able to know whether a callable "is" or "may not be" crap when it is assigned. If you still think that these two operations are undesireable (testing the callability, and that a callable takes certain arguments), that's fine (I disagree completely). But unless we mangle callables to not support these operations, people are probably going to do them anyways; especially those who are using annotations, function signature objects, etc., in their various frameworks. But maybe I'm misreading or reading to much into your statement of "If someone passes you a bad callback, they will see a traceback when you call it. Then they fix their program. That's how it's supposed to work." - Josiah From steven.bethard at gmail.com Sun Apr 8 20:11:06 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Sun, 8 Apr 2007 12:11:06 -0600 Subject: [Python-Dev] function for counting items in a sequence In-Reply-To: <00ca01c779b3$5e2469a0$f201a8c0@RaymondLaptop1> References: <00ca01c779b3$5e2469a0$f201a8c0@RaymondLaptop1> Message-ID: A summary response to the issues raised so far... On what the name should be: * Adam Olsen - countunique(), countdistinct(), countduplicates() * Greg Ewing - counteach(), countall() * Kevin Jacobs - tally() * Guido - counts() is fine So I guess I'll stick with counts(). On whether the count of a missing item should be 0 or a KeyError: * Brett Cannon - 0 * Greg Ewing - 0 Because the number of times an unseen item was seen is 0. So I'll stick with returning 0. That's the normal behavior for a defaultdict(int). Raymond Hettinger suggested that the function should be called dict.fromcount() instead. I lean towards the collections module instead of a dict classmethod because the feedback above suggests that the count of an unseen item should be 0. This means returning a defaultdict(int), which might be a bit confusing from a classmethod of *dict*. Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From g.brandl at gmx.net Sun Apr 8 20:23:15 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 08 Apr 2007 20:23:15 +0200 Subject: [Python-Dev] function for counting items in a sequence In-Reply-To: References: <00ca01c779b3$5e2469a0$f201a8c0@RaymondLaptop1> Message-ID: Steven Bethard schrieb: > A summary response to the issues raised so far... > > On what the name should be: > * Adam Olsen - countunique(), countdistinct(), countduplicates() > * Greg Ewing - counteach(), countall() > * Kevin Jacobs - tally() > * Guido - counts() is fine > So I guess I'll stick with counts(). I can live with counts(), though I always ask myself "who counts?" ;) > On whether the count of a missing item should be 0 or a KeyError: > * Brett Cannon - 0 > * Greg Ewing - 0 > Because the number of times an unseen item was seen is 0. So I'll > stick with returning 0. That's the normal behavior for a > defaultdict(int). +1 from me too for 0. > Raymond Hettinger suggested that the function should be called > dict.fromcount() instead. I lean towards the collections module > instead of a dict classmethod because the feedback above suggests that > the count of an unseen item should be 0. This means returning a > defaultdict(int), which might be a bit confusing from a classmethod of > *dict*. defaultdict.fromcount? Too obscure, I guess. Georg From guido at python.org Sun Apr 8 23:34:05 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 8 Apr 2007 14:34:05 -0700 Subject: [Python-Dev] function for counting items in a sequence In-Reply-To: References: <00ca01c779b3$5e2469a0$f201a8c0@RaymondLaptop1> Message-ID: On 4/8/07, Georg Brandl wrote: > I can live with counts(), though I always ask myself "who counts?" ;) The Count, of course. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Mon Apr 9 00:18:54 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 08 Apr 2007 18:18:54 -0400 Subject: [Python-Dev] concerns regarding callable() method In-Reply-To: References: <200704081510.32382.pogonyshev@gmx.net> <200704081856.42362.pogonyshev@gmx.net> Message-ID: <5.1.1.6.0.20070408181346.03de05b0@sparrow.telecommunity.com> At 08:01 PM 4/8/2007 +0400, Alexey Borzenkov wrote: >On 4/8/07, Guido van Rossum wrote: > > On 4/8/07, Paul Pogonyshev wrote: > > > Guido van Rossum wrote: > > > > What if someone passes a callable that doesn't have the expected > signature? > > > Well, I don't know a way to catch such situations now, so removing > > > callable() will not make it worse (even if you don't know about hasattr > > > trick above.) > > My point is that it's futile to use callable() -- even if it passes, > > you have no assurance that you actually have a valid callback. So why > > bother with it at all? It's counter to the spirit of Python. If > > someone passes you a bad callback, they will see a traceback when you > > call it. Then they fix their program. That's how it's supposed to > > work. > >But what if you need to do different things based on argument is >callable or not? Then delegate that behavior to the object using adaptation or a generic function. That is, use *table-driven* code, rather than if-then based code. >Take for example "Dependency Injection" recipe: > > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/413268 > >It uses callable to differentiate whether it needs to use object as >singleton or to instantiate it on each request. And that's a *bad* thing. What if I want to have a callable singleton? Or what if I want to have a singleton that gets notified of each request? With table-driven code (adaptation, generic functions) I can support the first case by reregistering an existing handler, and the second by registering my own handler. If-then based introspection is harmful here, because it's an attempt to *guess* what should be done with the object. Explicit is better than implicit, and it's better not to guess. callable(), like hasattr() or any other "interface probing" is essentially an attempt to guess at the callers intent, instead of providing a reasonable default that they can override. From guido at python.org Mon Apr 9 00:52:54 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 8 Apr 2007 15:52:54 -0700 Subject: [Python-Dev] concerns regarding callable() method In-Reply-To: <20070408100150.6273.JCARLSON@uci.edu> References: <200704081856.42362.pogonyshev@gmx.net> <20070408100150.6273.JCARLSON@uci.edu> Message-ID: On 4/8/07, Josiah Carlson wrote: > "Guido van Rossum" wrote: > > On 4/8/07, Paul Pogonyshev wrote: > > > Guido van Rossum wrote: > > > > What if someone passes a callable that doesn't have the expected signature? > > > > > > Well, I don't know a way to catch such situations now, so removing > > > callable() will not make it worse (even if you don't know about hasattr > > > trick above.) > > > > My point is that it's futile to use callable() -- even if it passes, > > you have no assurance that you actually have a valid callback. So why > > bother with it at all? It's counter to the spirit of Python. If > > someone passes you a bad callback, they will see a traceback when you > > call it. Then they fix their program. That's how it's supposed to > > work. > > The point of using callable(x), or it's equivalent now of hasattr(x, > '__call__') is to reduce reduce the time/lines of code between when an > error occurs and when it is reported. > Errors should never pass silently. I'm not sure if that argument weighs much (taken a bit farther it would require static typing :-). Two arguments made earlier are stronger IMO: - The traceback can be much clearer if it comes from setcallback(x) rather than from the actual call, much later. - The pragmatic ability (which often occurs in "DWIM"-ish behavior in template systems) to do different things depending on whether something is callable or not. (While in theory I agree with Phillip's objection that this should be dealt with in a more systematic way, in practice, at least for Python 2.x, I think it's an "okay" thing to do.) But the same thing can be said for properties like iterable, or hashable, and other primitive operations. FWIW, I haven't given up on doing something with abstract base classes here. I think they (or interfaces or generic functions, for that matter :-) provide a more systematic approach than either callable() or hasattr(x, "__call__"). TBC, --Guido > While we currently cannot verify that some callable takes the proper > arguments, number of arguments, etc., we *can* verify that it is > callable. I think this is a good thing, as allowing the assignment of a > non-callable to a name that is supposed to be callable is the silent > passing of an error. > > With relatively minimal effort in Python 3, one could use a function > signature object (PEPs 3107 and 362) to verify that a callable takes the > proper number of arguments, expected keyword arguments, etc., which > while still not allowing one to verify that the implementation of a > callback is correct (technically impossible), it does get us closer to > being able to know whether a callable "is" or "may not be" crap when it > is assigned. > > If you still think that these two operations are undesireable (testing > the callability, and that a callable takes certain arguments), that's > fine (I disagree completely). But unless we mangle callables to not > support these operations, people are probably going to do them anyways; > especially those who are using annotations, function signature objects, > etc., in their various frameworks. > > But maybe I'm misreading or reading to much into your statement of "If > someone passes you a bad callback, they will see a traceback when you > call it. Then they fix their program. That's how it's supposed to work." > > > - Josiah > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Mon Apr 9 02:15:20 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 09 Apr 2007 12:15:20 +1200 Subject: [Python-Dev] function for counting items in a sequence In-Reply-To: <00ca01c779b3$5e2469a0$f201a8c0@RaymondLaptop1> References: <00ca01c779b3$5e2469a0$f201a8c0@RaymondLaptop1> Message-ID: <46198598.8010009@canterbury.ac.nz> Raymond Hettinger wrote: > Instead of a separate function in collections, it may be sensible to stick with > the familiar and provide a dictionary contructor for the commonly encounter use > case of counting things: I don't think that's a good idea. The dict class should restrict itself to being a general-purpose mapping object, and not become a catchall for utility functions that happen to return a dict. -- Greg From bjourne at gmail.com Mon Apr 9 03:01:48 2007 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Mon, 9 Apr 2007 01:01:48 +0000 Subject: [Python-Dev] concerns regarding callable() method In-Reply-To: <200704081922.57275.pogonyshev@gmx.net> References: <200704081510.32382.pogonyshev@gmx.net> <200704081856.42362.pogonyshev@gmx.net> <200704081922.57275.pogonyshev@gmx.net> Message-ID: <740c3aec0704081801o66a5c085n788b1e53b84262a0@mail.gmail.com> On 4/8/07, Paul Pogonyshev wrote: > I have no problems with Python being untyped. But I want that error > stack traces provide some useful information as possible with reasonable > effort and that errors happen as early as possible. In particular, stack > trace should mention that error occured when you passed something wrong > to set_callback() call and not in some obscure place 200 lines later, > because otherwise it will only obfuscate error reason. Using the duck typing philosophy; "if it quacks like a duck and walks like a duck, then it probably is a duck." But how can you be so sure it is NOT a duck if you have never seen it walk or heard it quack? What if you are passing in an object that is not callable but later on becomes callable? Is it really an error? I think the plan is that in py3k, you will be able to do type-checking using function annotations (bleach). Like this: def set_callback(self, callback : CallableType): self.callback = callback You probably also need to add some more gunk to make it work. I believe it should be able to replace most uses of callable(). -- mvh Bj?rn From greg.ewing at canterbury.ac.nz Mon Apr 9 03:00:05 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 09 Apr 2007 13:00:05 +1200 Subject: [Python-Dev] Extended buffer PEP In-Reply-To: <46192398.9000001@ieee.org> References: <4612DC7C.5090706@ee.byu.edu> <4617F064.6020605@aerojockey.com> <46192398.9000001@ieee.org> Message-ID: <46199015.9020307@canterbury.ac.nz> Travis Oliphant wrote: > Carl Banks wrote: > > I'd like to see it accept a flags argument over what kind of buffer > > it's allowed to return. I'd rather not burden the user to check all > > the entries in bufferinfo to make sure it doesn't get something > > unexpected. > Yes, I agree. We had something like that at one point. Maybe this could be handled in an intermediate layer between the user and implementor of the interface, i.e. the user calls PyBuffer_GetBuffer(obj, &info, flags); the object's tp_getbufferinfo just gets called as getbufferinfo(self, &info) and PyBuffer_GetBuffer then checks that the result conforms to the requested feature set. This would relieve users of the interface from having to check that themselves, while not requiring implementors to be burdened with it either. Possibly you might want to pass the flags on to the object as well, in case it can reduce the amount of work it has to do if not all the features are being requested. But the object would be free to ignore them if it wants. -- Greg From greg.ewing at canterbury.ac.nz Mon Apr 9 03:04:27 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 09 Apr 2007 13:04:27 +1200 Subject: [Python-Dev] function for counting items in a sequence In-Reply-To: References: <00ca01c779b3$5e2469a0$f201a8c0@RaymondLaptop1> Message-ID: <4619911B.1010205@canterbury.ac.nz> Steven Bethard wrote: > * Greg Ewing - counteach(), countall() > * Guido - counts() is fine I'm happy with counts() too -- I only suggested the others in case counts() wasn't acceptable for some reason. If Guido likes it, that's good enough for me. -- Greg From skip at pobox.com Mon Apr 9 04:53:09 2007 From: skip at pobox.com (skip at pobox.com) Date: Sun, 8 Apr 2007 21:53:09 -0500 Subject: [Python-Dev] concerns regarding callable() method In-Reply-To: References: <200704081510.32382.pogonyshev@gmx.net> <200704081856.42362.pogonyshev@gmx.net> Message-ID: <17945.43669.943883.77109@montanaro.dyndns.org> Guido> My point is that it's futile to use callable() -- even if it Guido> passes, you have no assurance that you actually have a valid Guido> callback. So why bother with it at all? It's counter to the Guido> spirit of Python. If someone passes you a bad callback, they will Guido> see a traceback when you call it. Then they fix their Guido> program. That's how it's supposed to work. There's one place where I find the traceback somewhat unhelpful. Consider calling a method of a class with incorrect arguments: >>> class C: ... def __init__(self): ... pass ... >>> C(1) Traceback (most recent call last): File "", line 1, in TypeError: __init__() takes exactly 1 argument (2 given) While in this example it's clear what method wasn't called correctly, a callback called with the wrong number of arguments yields a fairly useless stack trace. I'm thinking in particular of callbacks called from C code (e.g. Gtk signal handlers). I think it would be helpful to check to see if the function being called had an "im_class" attribute. If so, then resolve the class name and include it in the TypeError message: TypeError: C.__init__() takes exactly 1 argument (2 given) Skip From kbk at shore.net Mon Apr 9 07:38:59 2007 From: kbk at shore.net (Kurt B. Kaiser) Date: Mon, 9 Apr 2007 01:38:59 -0400 (EDT) Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200704090538.l395cxKe013626@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 349 open ( +7) / 3737 closed (+25) / 4086 total (+32) Bugs : 939 open (-12) / 6648 closed (+60) / 7587 total (+48) RFE : 249 open ( -8) / 278 closed (+12) / 527 total ( +4) New / Reopened Patches ______________________ Python 2.5 "What's New" document contains socket errors (2007-03-17) CLOSED http://python.org/sf/1682878 opened by Jp Calderone ConfigParser support for alt delimiters (2007-03-18) http://python.org/sf/1682942 opened by Stephen Hansen Demo/parser/unparse.py fixes and cleanups (2007-03-19) CLOSED http://python.org/sf/1683328 opened by Piet Delport Refactor test_minidom.py to use unittest. (2007-03-18) CLOSED http://python.org/sf/1683397 opened by Jerry Seutter PEP 361 Warnings (2007-03-19) http://python.org/sf/1683908 opened by Adam Olsen add os.chflags() and os.lchflags() where available (2006-05-17) http://python.org/sf/1490190 reopened by nnorwitz Documentation for C-API string functions (2007-03-21) CLOSED http://python.org/sf/1684834 opened by Bj?rn Lindqvist cgi.py invalid REQUEST_METHOD set (2005-03-08) CLOSED http://python.org/sf/1159139 reopened by joesalmeri new function: os.path.relpath (2005-10-27) CLOSED http://python.org/sf/1339796 reopened by loewis MSVCCompiler creates redundant and long PATH strings (2007-03-21) CLOSED http://python.org/sf/1685563 opened by Scott Dial Add IllegalStateError (2007-03-22) http://python.org/sf/1685642 opened by Bj?rn Lindqvist Method cache (2007-03-22) http://python.org/sf/1685986 opened by Armin Rigo More PEP 3116 classes (2007-03-22) CLOSED http://python.org/sf/1686273 opened by Mark Russell int to Py_ssize_t changes (2007-03-22) CLOSED http://python.org/sf/1686451 opened by Alexander Belopolsky Allow any mapping after ** in calls (2007-03-22) http://python.org/sf/1686487 opened by Alexander Belopolsky extending os.walk to support following symlinks (2005-08-26) CLOSED http://python.org/sf/1273829 reopened by loewis Replace time_t by Py_time_t (2007-03-27) http://python.org/sf/1689402 opened by Christian Heimes Refactor test_sax.py to use unittest. (2007-03-28) CLOSED http://python.org/sf/1690164 opened by Jerry Seutter Refactor test_pyexpat.py to use unittest. (2007-03-28) CLOSED http://python.org/sf/1690169 opened by Jerry Seutter Added support for custom readline functions (2007-03-28) http://python.org/sf/1690201 opened by Ben Timby Don't block on Queue get/put when time is moved back (2007-03-29) http://python.org/sf/1690578 opened by xiaowen Migrate test_minidom.py to unittest (2007-03-30) http://python.org/sf/1691032 opened by Jason Orendorff Fix for bug #1283289 (2007-03-30) http://python.org/sf/1691070 opened by Roger Upole Move initial args assignment to BaseException.__new__ (2007-04-01) http://python.org/sf/1692335 opened by ?iga Seilnacht warnings.py gets filename wrong for eval/exec (2007-04-01) http://python.org/sf/1692664 opened by Adam Olsen trace.py --ignore-module should accept module name list. (2007-04-02) http://python.org/sf/1693149 opened by Raghuram Devarakonda Fix for duplicate "preferences" menu-OS X (2007-04-02) http://python.org/sf/1693258 opened by Kevin Walzer tarfile bug when opening a file directly (2007-04-05) http://python.org/sf/1695229 opened by Arve Knudsen Fix test_urllib on Windows buildbots (2007-04-07) http://python.org/sf/1695862 opened by Ziga Seilnacht Remove redundant code in ntpath.walk() (2007-04-08) http://python.org/sf/1696393 opened by Michael Haggerty Adding an index method to tuples (2007-04-08) http://python.org/sf/1696444 opened by Paul Boddie Patches Closed ______________ TypeError swallowing in UNPACK_SEQUENCE opcode (2007-03-16) http://python.org/sf/1682205 closed by gbrandl Make PyComplex_AsCComplex use __complex__ (2007-03-07) http://python.org/sf/1675423 closed by gbrandl remove sys.exitfunc, rewrite atexit in C (2007-03-14) http://python.org/sf/1680961 closed by collinwinter telnetlib option subnegotiation fix (2003-01-07) http://python.org/sf/664020 closed by gbrandl telnetlib.py change to ease option handling. (2006-07-10) http://python.org/sf/1520081 closed by gbrandl Python 2.5 "What's New" document contains socket errors (2007-03-17) http://python.org/sf/1682878 closed by gbrandl PEP 3115 patch (2007-03-14) http://python.org/sf/1681101 closed by gvanrossum Adding a testcase for the bug in find_longest_match (2007-03-11) http://python.org/sf/1678339 closed by gbrandl Patch to add tempfile.SpooledTemporaryFile (for #415692) (2007-01-07) http://python.org/sf/1630118 closed by collinwinter Demo/parser/unparse.py fixes and cleanups (2007-03-19) http://python.org/sf/1683328 closed by gbrandl Refactor test_minidom.py to use unittest. (2007-03-19) http://python.org/sf/1683397 closed by collinwinter fix Makefile.pre to use config env (2002-12-29) http://python.org/sf/659809 closed by bcannon Documentation for C-API string functions (2007-03-21) http://python.org/sf/1684834 closed by gbrandl cgi.py invalid REQUEST_METHOD set (2005-03-08) http://python.org/sf/1159139 closed by sf-robot Replacing and deleting files in a zipfile archive. (2003-03-10) http://python.org/sf/700858 closed by sf-robot desktop module (providing startfile as open, all platforms) (2005-09-23) http://python.org/sf/1301512 closed by pboddie new function: os.path.relpath (2005-10-27) http://python.org/sf/1339796 closed by collinwinter Adding timeout to socket.py and httplib.py (2007-03-08) http://python.org/sf/1676823 closed by facundobatista New block cipher API (2003-01-27) http://python.org/sf/675698 closed by collinwinter Updates to syntax rules in reference manual (2006-05-16) http://python.org/sf/1489771 closed by zseil MSVCCompiler creates redundant and long PATH strings (2007-03-21) http://python.org/sf/1685563 closed by nnorwitz More PEP 3116 classes (2007-03-22) http://python.org/sf/1686273 closed by gvanrossum int to Py_ssize_t changes (2007-03-23) http://python.org/sf/1686451 closed by loewis extending os.walk to support following symlinks (2005-08-26) http://python.org/sf/1273829 closed by zseil Support CREATE_SUSPENDED flag in subprocess.py for Win32 (2007-03-09) http://python.org/sf/1677688 closed by sf-robot Refactor test_sax.py to use unittest. (2007-03-28) http://python.org/sf/1690164 closed by collinwinter Refactor test_pyexpat.py to use unittest. (2007-03-28) http://python.org/sf/1690169 closed by collinwinter test_timeout updates (2003-04-28) http://python.org/sf/728815 closed by collinwinter ability to pass a timeout to underlying socket (2003-04-17) http://python.org/sf/723312 closed by facundobatista Cygwin case-sensitive import patch (2005-05-07) http://python.org/sf/1197318 closed by zseil Improvements to ceval.c (2006-05-22) http://python.org/sf/1492828 closed by rhettinger httplib : SSL fake sockets to close correctly (2005-10-01) http://python.org/sf/1310381 closed by loewis New / Reopened Bugs ___________________ Documentation error (section 3.4.1) (2007-03-17) CLOSED http://python.org/sf/1682729 opened by emlyn next method assignment is ignored in new-style classes (2007-03-17) http://python.org/sf/1682749 reopened by albert2611 next method assignment is ignored in new-style classes (2007-03-17) http://python.org/sf/1682749 opened by Albert Weichselbraun os.walk should traverse outward symlinks (2007-03-18) CLOSED http://python.org/sf/1682940 opened by Tasci Synx xreload.py won't update class docstrings (2007-03-18) http://python.org/sf/1683288 opened by Jim Jewett select.select() injecting spurious text in stdout (2007-03-19) CLOSED http://python.org/sf/1683316 opened by Peter Williams object.__init__ shouldn't allow args/kwds (2007-03-18) http://python.org/sf/1683368 opened by Blake Ross module-ctypes should show platform availability (2007-03-19) CLOSED http://python.org/sf/1683872 opened by Noah Spurrier regression in webbrowser command line parsing (2007-03-20) CLOSED http://python.org/sf/1684254 opened by Lukas Lalinsky modindex.html builds incorrectly in Python 3 (2007-03-20) CLOSED http://python.org/sf/1684811 opened by Collin Winter Keyword arguments to __init__() of a subclass of set (2007-03-21) CLOSED http://python.org/sf/1684943 opened by J?rgen Fr?jk Kj?rsgaard Explain __method__ lookup semantics for new-style classes (2007-03-21) http://python.org/sf/1684991 opened by Georg Brandl DoS asyncore vulnerability (2007-03-21) http://python.org/sf/1685000 opened by billiejoex email package should work better with unicode (2007-03-21) http://python.org/sf/1685453 opened by Barry A. Warsaw Command line interface to timeit.py (2007-03-21) CLOSED http://python.org/sf/1685704 opened by Tim Heaney tarfile file names under win32 (2007-03-22) CLOSED http://python.org/sf/1685773 opened by ralf msvccompiler.py doesn't recongnize VS2005 (2007-03-22) CLOSED http://python.org/sf/1685775 opened by Minmin Gong socket.getaddrinfo() should take an address tuple. (2007-03-22) http://python.org/sf/1685962 opened by Alan Kennedy logging package failure for NTEventLogHandler (2007-03-22) CLOSED http://python.org/sf/1686200 opened by j vickroy Python SEGFAULT on invalid superclass access (2007-03-22) http://python.org/sf/1686386 opened by toxik os.stat() WindowsError 13 when file in use (2007-03-22) CLOSED http://python.org/sf/1686475 opened by d roberts descrintro: error describing __new__ behavior (2007-03-23) http://python.org/sf/1686597 opened by Gabriel Genellina cannot catch KeyboardInterrupt when using curses getkey() (2007-03-23) http://python.org/sf/1687125 opened by al65536 Inconsistent Exceptions for Readonly Attributes (2007-03-23) CLOSED http://python.org/sf/1687163 opened by Raymond Hettinger Round funcion (2007-03-24) CLOSED http://python.org/sf/1687314 opened by Gust1 Python/C PyClass_* are not documented (2007-03-26) CLOSED http://python.org/sf/1688274 opened by Tommi Vainikainen sock.recvfrom(-24) crashes (2007-03-27) CLOSED http://python.org/sf/1688393 opened by Andrew Bennetts os.path.join.__doc__ should mention absolute paths (2007-03-26) http://python.org/sf/1688564 opened by Eugene Kramer pdb unable to jump to first statement (2007-03-27) http://python.org/sf/1689458 opened by gotgenes strptime(): can't switch locales more than once (2005-09-13) CLOSED http://python.org/sf/1290505 reopened by bcannon Intel icc build fails with optimizations -O2 (2007-03-28) http://python.org/sf/1689617 opened by Michael Forbes trace module borks __file__ (2007-03-28) http://python.org/sf/1690103 opened by Skip Montanaro Typographical Error (2007-03-29) CLOSED http://python.org/sf/1690324 opened by Mervyn Yeo email.utils.formataddr() should be rfc2047 aware (2007-03-29) http://python.org/sf/1690608 opened by Barry A. Warsaw xmlrpclib methods submit call on __str__, __repr__ (2007-03-29) http://python.org/sf/1690840 opened by Greg Hazel struct.pack of floats in non-native endian order (2003-03-18) http://python.org/sf/705836 reopened by collinwinter struct.pack raises TypeError where it used to convert (2006-07-28) http://python.org/sf/1530559 reopened by collinwinter Duplicate "preferences" menu item/Tk Aqua 8.4.14 (2007-03-30) http://python.org/sf/1691411 opened by Kevin Walzer lambda function segfault (2007-04-01) CLOSED http://python.org/sf/1692280 opened by Dasn webbrowser module cannot open the right URL (2007-04-02) CLOSED http://python.org/sf/1692695 opened by Pan Yongzhi \w not helpful for non-Roman scripts (2007-04-02) http://python.org/sf/1693050 opened by nlmiles Can't save empty array in shelve (2007-04-02) CLOSED http://python.org/sf/1693079 opened by IWL email.Message set_param rfc2231 encoding incorrect (2007-04-03) http://python.org/sf/1693546 opened by Anders Hammarquist Portability issie: os.rename behaves differently on win32 (2007-04-03) http://python.org/sf/1693753 opened by cs_ Puzzled over list append bvehavior (2007-04-03) CLOSED http://python.org/sf/1693941 opened by elgordo Python 2.5 installer ended prematurely (2007-04-04) http://python.org/sf/1694155 opened by Miclav Failure to build on AIX 5.3 (2007-04-04) http://python.org/sf/1694442 opened by Jaman Swearingen Overloading int.__pow__ does not work (2007-04-04) http://python.org/sf/1694663 opened by Terry J. Reedy Bad documentation for existing imp methods (2007-04-05) http://python.org/sf/1694833 opened by Christoph Zwerschke Bad documentation of SMTPException (2007-04-05) http://python.org/sf/1694950 opened by Christoph Zwerschke Failure to build on AIX 5.3 (2007-04-05) CLOSED http://python.org/sf/1695179 opened by Jaman Swearingen generator expression produces incomplete list (2007-04-05) CLOSED http://python.org/sf/1695272 opened by wplapper Unexpected feature vs. integer and True values (2007-04-06) CLOSED http://python.org/sf/1695688 opened by dev at bacademy PEP 302 broken (2007-04-06) CLOSED http://python.org/sf/1695718 opened by phil logging.handlers.SocketHandler.makeSocket() blocks app (2007-04-07) http://python.org/sf/1695948 opened by jtdeng codecs.EncodedFile() - the same data and file encodings (2007-04-07) CLOSED http://python.org/sf/1696025 opened by Alexandr V. Demeshko Add collections.counts() (2007-04-07) http://python.org/sf/1696199 opened by Steven Bethard Strange performance with unicode and lists (2007-04-08) CLOSED http://python.org/sf/1696390 opened by Vsevolod Bugs Closed ___________ Python and Indeterminate Forms (Math) (2007-03-15) http://python.org/sf/1681671 closed by tjreedy Documentation error (section 3.4.1) (2007-03-17) http://python.org/sf/1682729 closed by zseil next method assignment is ignored in new-style classes (2007-03-17) http://python.org/sf/1682749 closed by gbrandl os.walk should traverse outward symlinks (2007-03-18) http://python.org/sf/1682940 closed by gbrandl Lib/io.py open uses unset "bs" (2007-03-12) http://python.org/sf/1679498 closed by gvanrossum clarify 80-char limit (2007-01-05) http://python.org/sf/1628906 closed by gvanrossum PyFloat_FromString deprecated form (2007-02-02) http://python.org/sf/1650903 closed by gbrandl Incorrect file path reported by inspect.getabsfile() (2007-02-23) http://python.org/sf/1666807 closed by zseil select.select() injecting spurious text in stdout (2007-03-18) http://python.org/sf/1683316 closed by sf-robot Import value 1e400 from pyc fails (2005-12-11) http://python.org/sf/1378305 closed by gbrandl Python2.5 can't read a (complex) pickle build by python2.4 (2007-03-07) http://python.org/sf/1675967 closed by zseil date-datetime comparison doesn't work (2007-03-04) http://python.org/sf/1673403 closed by collinwinter module-ctypes should show platform availability (2007-03-19) http://python.org/sf/1683872 closed by gbrandl too long string causes segmentation fault (2007-03-15) http://python.org/sf/1681762 closed by gbrandl regression in webbrowser command line parsing (2007-03-20) http://python.org/sf/1684254 closed by gbrandl __cdecl / __stdcall unspecified in *.h (2001-10-15) http://python.org/sf/471432 closed by bcannon modindex.html builds incorrectly in Python 3 (2007-03-20) http://python.org/sf/1684811 closed by collinwinter Explain how to subclass Exception (2002-07-06) http://python.org/sf/578180 closed by bcannon extend namespace docs to cover LOAD_NAME (2002-04-19) http://python.org/sf/546196 closed by bcannon -S hides standard dynamic modules (2002-07-25) http://python.org/sf/586680 closed by bcannon IBCarbon module (2002-07-26) http://python.org/sf/587011 closed by bcannon Using .next() on file open in write mode writes junk to file (2006-10-01) http://python.org/sf/1569057 closed by bcannon Keyword arguments to __init__() of a subclass of set (2007-03-20) http://python.org/sf/1684943 closed by nnorwitz python always searches python23.zip (2003-02-25) http://python.org/sf/692884 closed by collinwinter Move modules out of Carbon (2003-03-02) http://python.org/sf/696206 closed by bcannon socket's makefile file object doesn't work with timeouts. (2004-01-22) http://python.org/sf/882297 closed by facundobatista Command line interface to timeit.py (2007-03-22) http://python.org/sf/1685704 closed by gbrandl tarfile file names under win32 (2007-03-22) http://python.org/sf/1685773 closed by indi4source msvccompiler.py doesn't recongnize VS2005 (2007-03-22) http://python.org/sf/1685775 closed by loewis logging package failure for NTEventLogHandler (2007-03-22) http://python.org/sf/1686200 closed by vsajip os.stat() WindowsError 13 when file in use (2007-03-23) http://python.org/sf/1686475 closed by loewis Inconsistent Exceptions for Readonly Attributes (2007-03-23) http://python.org/sf/1687163 closed by collinwinter Round funcion (2007-03-24) http://python.org/sf/1687314 closed by gbrandl format error in 2.5 ref.pdf in section 5.3.4 Calls (2006-12-20) http://python.org/sf/1619641 closed by zseil texinfo library documentation fails to build (2006-11-12) http://python.org/sf/1595164 closed by loewis broken info files generation (2006-09-30) http://python.org/sf/1568429 closed by zseil Python/C PyClass_* are not documented (2007-03-26) http://python.org/sf/1688274 closed by collinwinter sock.recvfrom(-24) crashes (2007-03-26) http://python.org/sf/1688393 closed by nnorwitz [2.5 regression?] failure to import the ORBit extension (2007-02-17) http://python.org/sf/1662529 closed by nnorwitz strptime(): can't switch locales more than once (2005-09-13) http://python.org/sf/1290505 closed by bcannon .eml attachments in email (2006-10-06) http://python.org/sf/1572084 closed by sf-robot Typographical Error (2007-03-29) http://python.org/sf/1690324 closed by quiver language reference index links are broken (2006-12-27) http://python.org/sf/1622664 closed by gbrandl preferred diff format should be mentioned as "unified". (2006-12-27) http://python.org/sf/1623153 closed by rhettinger thirdparty extensions, --enable-shared, static linking (2007-02-08) http://python.org/sf/1655392 closed by gbrandl lambda function segfault (2007-04-01) http://python.org/sf/1692280 closed by gbrandl webbrowser module cannot open the right URL (2007-04-02) http://python.org/sf/1692695 closed by gbrandl Can't save empty array in shelve (2007-04-02) http://python.org/sf/1693079 closed by rhettinger slice obj with no start index is 0 instead of None sometimes (2007-02-28) http://python.org/sf/1671137 closed by jyzude SystemError with re.match(array) (2007-01-29) http://python.org/sf/1647541 closed by rhettinger Incorrect docs for bisect_left (2006-11-24) http://python.org/sf/1602378 closed by rhettinger Tutorial section 9.5.1 ignores MRO for new-style classes (2005-12-16) http://python.org/sf/1382213 closed by rhettinger incorrect traceback filename from pyc (2004-10-21) http://python.org/sf/1051638 closed by zseil 'Ok' key in options dialog does nothing (2007-02-11) http://python.org/sf/1657034 closed by torhu Puzzled over list append bvehavior (2007-04-03) http://python.org/sf/1693941 closed by rhettinger struct.unpack doens't support buffer protocol objects (2006-09-22) http://python.org/sf/1563759 closed by rhettinger clearing up dictionary keys/set member docs (2005-11-28) http://python.org/sf/1368768 closed by rhettinger zlib.crc32() not cross-platform (2007-03-10) http://python.org/sf/1678102 closed by tjreedy Failure to build on AIX 5.3 (2007-04-05) http://python.org/sf/1695179 deleted by tiriss generator expression produces incomplete list (2007-04-05) http://python.org/sf/1695272 closed by gbrandl Unexpected feature vs. integer and True values (2007-04-06) http://python.org/sf/1695688 closed by gbrandl PEP 302 broken (2007-04-06) http://python.org/sf/1695718 closed by gbrandl codecs.EncodedFile() - the same data and file encodings (2007-04-07) http://python.org/sf/1696025 closed by doerwalter Strange performance with unicode and lists (2007-04-08) http://python.org/sf/1696390 closed by gbrandl New / Reopened RFE __________________ Add tail recursion (2007-03-20) CLOSED http://python.org/sf/1684603 opened by Tommy Nordgren csv sniff example (2007-03-22) http://python.org/sf/1686390 opened by Thomas Guettler Add "methodcaller" to the operator module (2006-06-14) CLOSED http://python.org/sf/1506171 reopened by rhettinger Call sys.except_hook if exception occurs in __del__ (2007-03-30) http://python.org/sf/1691387 opened by John Ehresman Stripping debugging symbols from compiled C extensions (2007-04-02) http://python.org/sf/1692592 opened by Viktor Ferenczi RFE Closed __________ optparse should support arbitrary number of arguments (2006-07-24) http://python.org/sf/1527705 closed by gbrandl smarter temporary file object (2001-04-12) http://python.org/sf/415692 closed by collinwinter Add tail recursion (2007-03-20) http://python.org/sf/1684603 closed by rhettinger dict(key,values) initializer (2007-02-09) http://python.org/sf/1656538 closed by rhettinger Allowing the definition of constants (2006-04-05) http://python.org/sf/1465406 closed by collinwinter itemgetter built-in? (2005-10-10) http://python.org/sf/1322308 closed by collinwinter Add os.path.relpath (2005-09-30) http://python.org/sf/1309676 closed by collinwinter Add a settimeout to ftplib.FTP object (2005-04-06) http://python.org/sf/1177998 closed by facundobatista 'attrmap' function, attrmap(x)['attname'] == x.attname (2005-01-26) http://python.org/sf/1110010 closed by collinwinter Add "methodcaller" to the operator module (2006-06-14) http://python.org/sf/1506171 closed by collinwinter Py_DEBUG (2006-12-09) http://python.org/sf/1612190 closed by collinwinter bug with ill-formed rfc822 attachments (2003-09-30) http://python.org/sf/815563 closed by collinwinter Need Windows os.link() support (2001-11-05) http://python.org/sf/478407 closed by collinwinter From p.f.moore at gmail.com Mon Apr 9 12:25:15 2007 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 9 Apr 2007 11:25:15 +0100 Subject: [Python-Dev] Extended Buffer Protocol - simple use examples Message-ID: <79990c6b0704090325n11721793r133e700f60c0e086@mail.gmail.com> Hi, I'll admit right off that I haven't followed all of the extended buffer protocol discussions - I have no real need for anything much beyond the existing "here's a blob of memory" level of functionality. I have skimmed (briefly, I'll admit!) the pre-PEP, but I've found it extremely difficult to find a simple example of the basic (in my view) use case of an undifferentiated block of bytes. Can I suggest that the following uses would be the significant majority of buffer protocol uses - they are certainly going to be the basic transitional case from the existing protocol - and deserve to be singled out in the PEP with some description or (better) pseudo-code examples. If they are already there, I apologise, but I did look, and got bogged down in the more sophisticated stuff - so they probably need making more visible. 1. (Producer) I have a block of memory in my C extension and I want to expose it as a simple contiguous block of bytes to Python. 2. (Consumer) I want to get at a block of memory exposed as a buffer. I am only interested in, and only support, viewing a buffer as a block of contiguous bytes. I expect most if not all extensions to be able to provide such a view. I'd hope these 2 cases are pretty simple to code, but I'd like to see that made explicit in the PEP. unsophisticated-ly y'rs Paul. From oliphant.travis at ieee.org Mon Apr 9 22:26:16 2007 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 09 Apr 2007 14:26:16 -0600 Subject: [Python-Dev] Extended Buffer Protocol - simple use examples In-Reply-To: <79990c6b0704090325n11721793r133e700f60c0e086@mail.gmail.com> References: <79990c6b0704090325n11721793r133e700f60c0e086@mail.gmail.com> Message-ID: Paul Moore wrote: > Hi, > I'll admit right off that I haven't followed all of the extended > buffer protocol discussions - I have no real need for anything much > beyond the existing "here's a blob of memory" level of functionality. > > I have skimmed (briefly, I'll admit!) the pre-PEP, but I've found it > extremely difficult to find a simple example of the basic (in my view) > use case of an undifferentiated block of bytes. > This is a great suggestion and it was on my to-do list. I've included some examples of this use-case in the new PEP. > > 1. (Producer) I have a block of memory in my C extension and I want to > expose it as a simple contiguous block of bytes to Python. This is now Ex. 2 in the PEP. > > 2. (Consumer) I want to get at a block of memory exposed as a buffer. > I am only interested in, and only support, viewing a buffer as a block > of contiguous bytes. I expect most if not all extensions to be able to > provide such a view. > This is now Ex. 3 Thanks for the suggestions. -Travis From oliphant.travis at ieee.org Mon Apr 9 22:28:07 2007 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 09 Apr 2007 14:28:07 -0600 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) Message-ID: Changes: * added the "flags" variable to allow simpler calling for getbuffer. * added some explanation of ideas that were discussed and abandoned. * added examples for simple use cases. * added more C-API calls to allow easier usage. Thanks for all feedback. -Travis -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: pep-3118_new.txt Url: http://mail.python.org/pipermail/python-dev/attachments/20070409/870c8d99/attachment-0001.txt From nyamatongwe at gmail.com Tue Apr 10 01:16:45 2007 From: nyamatongwe at gmail.com (Neil Hodgson) Date: Tue, 10 Apr 2007 09:16:45 +1000 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) In-Reply-To: References: Message-ID: <50862ebd0704091616v5ec35d2at88a867b7a5f4220c@mail.gmail.com> Travis Oliphant: > PEP: 3118 > ... I'd like to see the PEP include discussion of what to do when an incompatible request is received while locked. Should there be a standard "Can't do that: my buffer has been got" exception? Neil From tdelaney at avaya.com Tue Apr 10 01:17:24 2007 From: tdelaney at avaya.com (Delaney, Timothy (Tim)) Date: Tue, 10 Apr 2007 09:17:24 +1000 Subject: [Python-Dev] function for counting items in a sequence Message-ID: <2773CAC687FD5F4689F526998C7E4E5FF1ED53@au3010avexu1.global.avaya.com> Greg Ewing wrote: > Steven Bethard wrote: > >> * Greg Ewing - counteach(), countall() >> * Guido - counts() is fine > > I'm happy with counts() too -- I only suggested the > others in case counts() wasn't acceptable for some > reason. If Guido likes it, that's good enough for > me. I think I'd prefer "tally" as less ambiguous, but "counts" works too. Tim Delaney From pythondev at aerojockey.com Tue Apr 10 01:59:41 2007 From: pythondev at aerojockey.com (Carl Banks) Date: Mon, 09 Apr 2007 19:59:41 -0400 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) In-Reply-To: References: Message-ID: <461AD36D.4080608@aerojockey.com> Travis Oliphant wrote: > Py_BUF_READONLY > The returned buffer must be readonly and the underlying object should make > its memory readonly if that is possible. I don't like the "if possible" thing. If it makes no guarantees, it pretty much useless over Py_BUF_SIMPLE. > Py_BUF_FORMAT > The consumer will be using the format string information so make sure that > member is filled correctly. Is the idea to throw an exception if there's some other data format besides "b", and this flag isn't set? It seems superfluous otherwise. > Py_BUF_SHAPE > The consumer can (and might) make use of using the ndims and shape members of the structure > so make sure they are filled in correctly. > > Py_BUF_STRIDES (implies SHAPE) > The consumer can (and might) make use of the strides member of the structure (as well > as ndims and shape) Is there any reasonable benefit for allowing Py_BUF_SHAPE without Py_BUF_STRIDES? Would the array be C- or Fortran-like? Another little mistake I made: looking at the Python source, it seems that most C defines do not use the Py_ prefix, so probably we shouldn't here. Sorry. Carl From oliphant.travis at ieee.org Tue Apr 10 02:58:06 2007 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 09 Apr 2007 18:58:06 -0600 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) In-Reply-To: <461AD36D.4080608@aerojockey.com> References: <461AD36D.4080608@aerojockey.com> Message-ID: <461AE11E.8040901@ieee.org> Carl Banks wrote: > > > Travis Oliphant wrote: > > Py_BUF_READONLY > > The returned buffer must be readonly and the underlying object > should make > > its memory readonly if that is possible. > > I don't like the "if possible" thing. If it makes no guarantees, it > pretty much useless over Py_BUF_SIMPLE. O.K. Let's make it raise an error if it can't set it read-only. >> Py_BUF_FORMAT >> The consumer will be using the format string information so make >> sure that member is filled correctly. > > Is the idea to throw an exception if there's some other data format > besides "b", and this flag isn't set? It seems superfluous otherwise. The idea is that a consumer may not care about the format and the exporter may want to know that to simplify the interface. In other words the flag is a way for the consumer to communicate that it wants format information (or not). If the exporter wants to raise an exception if the format is not requested is up to the exporter. >> Py_BUF_SHAPE >> The consumer can (and might) make use of using the ndims and shape >> members of the structure >> so make sure they are filled in correctly. Py_BUF_STRIDES >> (implies SHAPE) >> The consumer can (and might) make use of the strides member of the >> structure (as well >> as ndims and shape) > > Is there any reasonable benefit for allowing Py_BUF_SHAPE without > Py_BUF_STRIDES? Would the array be C- or Fortran-like? Yes, I could see a consumer not being able to handle simple striding but could handle shape information. Many users of NumPy arrays like to think of the array as an N-d array but want to ignore striding. I've made the changes in numpy's SVN. Hopefully they will get mirrored over to the python PEP directory eventually. -Travis From brett at python.org Tue Apr 10 06:33:36 2007 From: brett at python.org (Brett Cannon) Date: Mon, 9 Apr 2007 21:33:36 -0700 Subject: [Python-Dev] deprecating BaseException.message Message-ID: During the PyCon sprint I tried to make BaseException accept only a single argument and bind it to BaseException.message . I was successful (see the p3yk_no_args_on_exc branch), but it was very painful to pull off as anyone who sat around me the last three days of the sprint will tell you as they had to listen to me curse incessantly. Because of the pain that I went through in the transition and thus the lessons learned, Guido and I discussed it and we think it would be best to give up on forcing BaseException to accept only a single argument. I think it is still doable, but requires a multi-release transition period and not the one that 2.6 -> 3.0 is offering. And so Guido and I plan on deprecating BaseException.message as its entire point in existence was to help transition to what we are not going to have happen. =) Now that means BaseException.message might hold the record for shortest lived feature as it was only introduced in 2.5 and is now to be deprecated in 2.6 and removed in 2.7/3.0. =) Below is PEP 352, revised to reflect the removal of BaseException.messageand for letting the 'args' attribute stay (along with suggesting one should only pass a single argument to BaseException). Basically the interface for exceptions doesn't really change in 3.0 except for the removal of __getitem__. -------------------------------------------------------------------------- PEP: 352 Title: Required Superclass for Exceptions Version: $Revision: 53592 $ Last-Modified: $Date: 2007-01-28 21:54:11 -0800 (Sun, 28 Jan 2007) $ Author: Brett Cannon Guido van Rossum Status: Final Type: Standards Track Content-Type: text/x-rst Created: 27-Oct-2005 Post-History: Abstract ======== In Python 2.4 and before, any (classic) class can be raised as an exception. The plan for 2.5 was to allow new-style classes, but this makes the problem worse -- it would mean *any* class (or instance) can be raised! This is a problem as it prevents any guarantees from being made about the interface of exceptions. This PEP proposes introducing a new superclass that all raised objects must inherit from. Imposing the restriction will allow a standard interface for exceptions to exist that can be relied upon. It also leads to a known hierarchy for all exceptions to adhere to. One might counter that requiring a specific base class for a particular interface is unPythonic. However, in the specific case of exceptions there's a good reason (which has generally been agreed to on python-dev): requiring hierarchy helps code that wants to *catch* exceptions by making it possible to catch *all* exceptions explicitly by writing ``except BaseException:`` instead of ``except *:``. [#hierarchy-good]_ Introducing a new superclass for exceptions also gives us the chance to rearrange the exception hierarchy slightly for the better. As it currently stands, all exceptions in the built-in namespace inherit from Exception. This is a problem since this includes two exceptions (KeyboardInterrupt and SystemExit) that often need to be excepted from the application's exception handling: the default behavior of shutting the interpreter down without a traceback is usually more desirable than whatever the application might do (with the possible exception of applications that emulate Python's interactive command loop with ``>>>`` prompt). Changing it so that these two exceptions inherit from the common superclass instead of Exception will make it easy for people to write ``except`` clauses that are not overreaching and not catch exceptions that should propagate up. This PEP is based on previous work done for PEP 348 [#pep348]_. Requiring a Common Superclass ============================= This PEP proposes introducing a new exception named BaseException that is a new-style class and has a single attribute, ``args``. Below is the code as the exception will work in Python 3.0 (how it will work in Python 2.x is covered in the `Transition Plan`_ section):: class BaseException(object): """Superclass representing the base of the exception hierarchy. Provides a 'message' attribute that contains either the single argument to the constructor or the empty string. This attribute is used in the string representation for the exception. This is so that it provides the extra details in the traceback. """ def __init__(self, *args): """Set the 'message' attribute'""" self.args = args def __str__(self): """Return the str of 'message'""" if len(self.args) == 1: return str(self.args[0]) else: return str(self.args) def __repr__(self): return "%s(*%s)" % (self.__class__.__name__, repr(self.args)) No restriction is placed upon what may be passed in for ``args`` for backwards-compatibility reasons. In practice, though, only a single string argument should be used. This keeps the string representation of the exception to be a useful message about the exception that is human-readable; this is why the ``__str__`` method special-cases on length-1 ``args`` value. Including programmatic information (e.g., an error code number) should be stored as a separate attribute in a subclass. The ``raise`` statement will be changed to require that any object passed to it must inherit from BaseException. This will make sure that all exceptions fall within a single hierarchy that is anchored at BaseException [#hierarchy-good]_. This also guarantees a basic interface that is inherited from BaseException. The change to ``raise`` will be enforced starting in Python 3.0 (see the `Transition Plan`_ below). With BaseException being the root of the exception hierarchy, Exception will now inherit from it. Exception Hierarchy Changes =========================== With the exception hierarchy now even more important since it has a basic root, a change to the existing hierarchy is called for. As it stands now, if one wants to catch all exceptions that signal an error *and* do not mean the interpreter should be allowed to exit, you must specify all but two exceptions specifically in an ``except`` clause or catch the two exceptions separately and then re-raise them and have all other exceptions fall through to a bare ``except`` clause:: except (KeyboardInterrupt, SystemExit): raise except: ... That is needlessly explicit. This PEP proposes moving KeyboardInterrupt and SystemExit to inherit directly from BaseException. :: - BaseException |- KeyboardInterrupt |- SystemExit |- Exception |- (all other current built-in exceptions) Doing this makes catching Exception more reasonable. It would catch only exceptions that signify errors. Exceptions that signal that the interpreter should exit will not be caught and thus be allowed to propagate up and allow the interpreter to terminate. KeyboardInterrupt has been moved since users typically expect an application to exit when the press the interrupt key (usually Ctrl-C). If people have overly broad ``except`` clauses the expected behaviour does not occur. SystemExit has been moved for similar reasons. Since the exception is raised when ``sys.exit()`` is called the interpreter should normally be allowed to terminate. Unfortunately overly broad ``except`` clauses can prevent the explicitly requested exit from occurring. To make sure that people catch Exception most of the time, various parts of the documentation and tutorials will need to be updated to strongly suggest that Exception be what programmers want to use. Bare ``except`` clauses or catching BaseException directly should be discouraged based on the fact that KeyboardInterrupt and SystemExit almost always should be allowed to propagate up. Transition Plan =============== Since semantic changes to Python are being proposed, a transition plan is needed. The goal is to end up with the new semantics being used in Python 3.0 while providing a smooth transition for 2.x code. All deprecations mentioned in the plan will lead to the removal of the semantics starting in the version following the initial deprecation. Here is BaseException as implemented in the 2.x series:: class BaseException(object): """Superclass representing the base of the exception hierarchy. The __getitem__ method is provided for backwards-compatibility and will be deprecated at some point. """ def __init__(self, *args): """Set the 'args' attribute.""" self.args = args def __str__(self): """Return the str of args[0] or args, depending on length.""" return str(self.args[0] if len(self.args) <= 1 else self.args) def __repr__(self): func_args = repr(self.args) if self.args else "()" return self.__class__.__name__ + func_args def __getitem__(self, index): """Index into arguments passed in during instantiation. Provided for backwards-compatibility and will be deprecated. """ return self.args[index] Deprecation of features in Python 2.9 is optional. This is because it is not known at this time if Python 2.9 (which is slated to be the last version in the 2.x series) will actively deprecate features that will not be in 3.0 . It is conceivable that no deprecation warnings will be used in 2.9 since there could be such a difference between 2.9 and 3.0 that it would make 2.9 too "noisy" in terms of warnings. Thus the proposed deprecation warnings for Python 2.9 will be revisited when development of that version begins to determine if they are still desired. * Python 2.5 [done] - all standard exceptions become new-style classes - introduce BaseException - Exception, KeyboardInterrupt, and SystemExit inherit from BaseException - deprecate raising string exceptions * Python 2.6 - deprecate catching string exceptions - deprecate ``message`` attribute (see `Retracted Ideas`_) * Python 2.7 - deprecate raising exceptions that do not inherit from BaseException - remove ``message`` attribute * Python 2.8 - deprecate catching exceptions that do not inherit from BaseException * Python 2.9 - deprecate ``__getitem__`` (optional) * Python 3.0 [done] - drop everything that was deprecated above: + string exceptions (both raising and catching) + all exceptions must inherit from BaseException + drop ``__getitem__`` Retracted Ideas =============== A previous version of this PEP that was implemented in Python 2.5 included a 'message' attribute on BaseException. Its purpose was to begin a transition to BaseException accepting only a single argument. This was to tighten the interface and to force people to use attributes in subclasses to carry arbitrary information with an exception instead of cramming it all into ``args``. Unfortunately, while implementing the removal of the ``args`` attribute in Python 3.0 at the PyCon 2007 sprint [#pycon2007-sprint-email]_, it was discovered that the transition was very painful, especially for C extension modules. It was decided that it would be better to deprecate the ``message`` attribute in Python 2.6 (and remove in Python 2.7 and Python 3.0) and consider a more long-term transition strategy in Python 3.0 to remove multiple-argument support in BaseException in preference of accepting only a single argument. Thus the introduction of ``message`` and the original deprecation of ``args`` has been retracted. References ========== .. [#pep348] PEP 348 (Exception Reorganization for Python 3.0) http://www.python.org/peps/pep-0348.html .. [#hierarchy-good] python-dev Summary for 2004-08-01 through 2004-08-15 http://www.python.org/dev/summary/2004-08-01_2004-08-15.html#an-exception-is-an-exception-unless-it-doesn-t-inherit-from-exception .. [#SF_1104669] SF patch #1104669 (new-style exceptions) http://www.python.org/sf/1104669 .. [#pycon2007-sprint-email] python-3000 email ("How far to go with cleaning up exceptions") http://mail.python.org/pipermail/python-3000/2007-March/005911.html Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070409/940e9b4b/attachment.html From p.f.moore at gmail.com Tue Apr 10 11:28:20 2007 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 10 Apr 2007 10:28:20 +0100 Subject: [Python-Dev] Extended Buffer Protocol - simple use examples In-Reply-To: References: <79990c6b0704090325n11721793r133e700f60c0e086@mail.gmail.com> Message-ID: <79990c6b0704100228s7382e013p7f0685d1a7e3ed6@mail.gmail.com> On 09/04/07, Travis Oliphant wrote: > > I have skimmed (briefly, I'll admit!) the pre-PEP, but I've found it > > extremely difficult to find a simple example of the basic (in my view) > > use case of an undifferentiated block of bytes. > > > > This is a great suggestion and it was on my to-do list. I've included > some examples of this use-case in the new PEP. Nice - those look clear to me. One question - if a producer is generating a more complex data format (for example, the RGBA example in the PEP) then would the "simple consumer" code (Ex. 3) still get a pointer (or would the RGBA code need to go through extra effort to allow this)? Sorry, again this is probably clear from reading the PEP details, but my eyes glaze over when I read about strides, shapes, etc... My motivation here is that it would be a shame if "old-style" code that was prepared to guess the format of a memory block stopped working when the producer of the memory added shape information (that the consumer didn't care about, except to validate its guess). Paul. From ncoghlan at gmail.com Tue Apr 10 11:58:02 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 10 Apr 2007 19:58:02 +1000 Subject: [Python-Dev] Checking PEP autobuild results Message-ID: <461B5FAA.1010403@gmail.com> The PEP I checked in for Travis hasn't turned up on the website, despite building without warnings when I run pep2html.py. The changes I made to PEP 0 haven't turned up either. I thought the subversion commit hook for the PEPs folder caused the website to update automatically. Have I forgotten a step in the process somewhere, or is something broken? Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Tue Apr 10 12:02:46 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 10 Apr 2007 20:02:46 +1000 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) In-Reply-To: <461AD36D.4080608@aerojockey.com> References: <461AD36D.4080608@aerojockey.com> Message-ID: <461B60C6.8030108@gmail.com> Carl Banks wrote: > Another little mistake I made: looking at the Python source, it seems > that most C defines do not use the Py_ prefix, so probably we shouldn't > here. Sorry. Most of the #define's aren't exposed via Python.h and aren't part of the public C API. The public ones are meant to use the prefix. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From amk at amk.ca Tue Apr 10 13:30:13 2007 From: amk at amk.ca (A.M. Kuchling) Date: Tue, 10 Apr 2007 07:30:13 -0400 Subject: [Python-Dev] Checking PEP autobuild results In-Reply-To: <461B5FAA.1010403@gmail.com> References: <461B5FAA.1010403@gmail.com> Message-ID: <20070410113013.GA16198@andrew-kuchlings-computer.local> On Tue, Apr 10, 2007 at 07:58:02PM +1000, Nick Coghlan wrote: > Have I forgotten a step in the process somewhere, or is something broken? Something's broken; I'm looking into it. --amk From facundo at taniquetil.com.ar Tue Apr 10 15:12:13 2007 From: facundo at taniquetil.com.ar (Facundo Batista) Date: Tue, 10 Apr 2007 13:12:13 +0000 (UTC) Subject: [Python-Dev] HTTP responses and errors References: <46092EA2.3030804@v.loewis.de> <46095087.9050002@v.loewis.de> Message-ID: Facundo Batista wrote: > Martin v. L?wis wrote: > ... > >> think it should treat all 2xx responses as success. Callers can >> then still check the response code themselves if they need to. > > The same I think. If nobody has a conflic with this decission, I'll fix > this. Nobody raised any objection, I'll fix this these days. Regards, -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From python at rcn.com Tue Apr 10 20:26:41 2007 From: python at rcn.com (Raymond Hettinger) Date: Tue, 10 Apr 2007 14:26:41 -0400 (EDT) Subject: [Python-Dev] Changes to decimal.py Message-ID: <20070410142641.BEU75896@ms09.lnh.mail.rcn.net> Now that the checkin is done, I don't think it needs to be reverted. But, in general, we should probably abstain from making wholesale revisions that add zero value for the users. The stylistic change from: ValueError, 'foo' ValueError('foo') is fine. Changing MockThreading to subclass from object though borders on being a semantic change and should be done with care. In this particular case, I see no harm in it, but then I haven't tested it on a Py2.3 build with threading disabled. As promised in the decimal.py header, the spec updates should all be considered as bugs and backported at some point after they are fully tested and we're happy with them all around. Also, as promised, the module should continue to run on Py2.3. For the most part, many of the new operations can be implemented in terms of the existing ops or in terms of the support functions that we already use internally. Ideally, you can refactor common code while leaving almost all of the exisiting algorithm implementation code untouched. The spec's choice of new method names is unfortunate. You'll have to come-up with something better than copy() and class(). FWIW, I think the new decimal development should probably be done in a branch off of the current head. That way, you can check-in at will and get feedback from everyone without risking the integrity of the head. If you want to discuss anything during development, I'm usually available on AOL instant messaging with the screename: raymondewt Likewise, consider soliciting Tim's input on how to implement the ln() operation. That one will be tricky to get done efficiently and correctly. Raymond -------------- next part -------------- An embedded message was scrubbed... From: "Facundo Batista" Subject: Re: [Python-checkins] r54732 - python/trunk/Lib/decimal.py Date: Tue, 10 Apr 2007 14:31:53 -0300 Size: 3893 Url: http://mail.python.org/pipermail/python-dev/attachments/20070410/06783483/attachment.eml From brett at python.org Tue Apr 10 21:20:34 2007 From: brett at python.org (Brett Cannon) Date: Tue, 10 Apr 2007 12:20:34 -0700 Subject: [Python-Dev] Checking PEP autobuild results In-Reply-To: <461B5FAA.1010403@gmail.com> References: <461B5FAA.1010403@gmail.com> Message-ID: On 4/10/07, Nick Coghlan wrote: > > The PEP I checked in for Travis hasn't turned up on the website, despite > building without warnings when I run pep2html.py. The changes I made to > PEP 0 haven't turned up either. > > I thought the subversion commit hook for the PEPs folder caused the > website to update automatically. > > Have I forgotten a step in the process somewhere, or is something broken? Had this happen to me. Usually means that another PEP is invalid and is breaking things. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070410/4257835b/attachment.htm From python at rcn.com Tue Apr 10 22:14:10 2007 From: python at rcn.com (Raymond Hettinger) Date: Tue, 10 Apr 2007 16:14:10 -0400 (EDT) Subject: [Python-Dev] Py2.5.1 release candidate Message-ID: <20070410161410.BEV15978@ms09.lnh.mail.rcn.net> It looks like the release candidate has been held-up for a bit. If it is going to stay held-up for a few days, can we unfreeze it so some bugfixes can go in (fixing the +0/-0 problem, eliminating some segfaults, and fixing some exception code)? Raymond From python at rcn.com Tue Apr 10 22:29:27 2007 From: python at rcn.com (Raymond Hettinger) Date: Tue, 10 Apr 2007 16:29:27 -0400 (EDT) Subject: [Python-Dev] USE_FAST code in stringobject.c Message-ID: <20070410162927.BEW01278@ms09.lnh.mail.rcn.net> Do any of the Iceland sprinters (Fredrik or Andrew perhaps) know why the USE_FAST section is segregated and whether it is supposed to be on or off by default? Raymond From martin at v.loewis.de Tue Apr 10 23:07:24 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 10 Apr 2007 23:07:24 +0200 Subject: [Python-Dev] Py2.5.1 release candidate In-Reply-To: <20070410161410.BEV15978@ms09.lnh.mail.rcn.net> References: <20070410161410.BEV15978@ms09.lnh.mail.rcn.net> Message-ID: <461BFC8C.5020003@v.loewis.de> Raymond Hettinger schrieb: > It looks like the release candidate has been held-up for a bit. If > it is going to stay held-up for a few days, can we unfreeze it so > some bugfixes can go in (fixing the +0/-0 problem, eliminating some > segfaults, and fixing some exception code)? No, the release binaries are all produced, and just await upload. If it's an urgent issue, we need another RC. If it isn't urgent (e.g. not a regression relative to 2.5.0), I think it should wait for 2.5.2. (IMHO all, of course) Regards, Martin From python at rcn.com Tue Apr 10 23:14:45 2007 From: python at rcn.com (Raymond Hettinger) Date: Tue, 10 Apr 2007 17:14:45 -0400 (EDT) Subject: [Python-Dev] Fwd: Re: Py2.5.1 release candidate Message-ID: <20070410171445.BEW17248@ms09.lnh.mail.rcn.net> > No, the release binaries are all produced, and just await upload. Woohoo! > If it's an urgent issue, we need another RC. > If it isn't urgent (e.g. not a regression relative to 2.5.0), > I think it should wait for 2.5.2. (IMHO all, of course) These bug fixes will have to wait. Raymond -------------- next part -------------- An embedded message was scrubbed... From: =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?= Subject: Re: [Python-Dev] Py2.5.1 release candidate Date: Tue, 10 Apr 2007 23:07:24 +0200 Size: 2170 Url: http://mail.python.org/pipermail/python-dev/attachments/20070410/63f3f86c/attachment-0001.eml From amk at amk.ca Wed Apr 11 01:07:32 2007 From: amk at amk.ca (A.M. Kuchling) Date: Tue, 10 Apr 2007 19:07:32 -0400 Subject: [Python-Dev] Checking PEP autobuild results In-Reply-To: <20070410113013.GA16198@andrew-kuchlings-computer.local> References: <461B5FAA.1010403@gmail.com> <20070410113013.GA16198@andrew-kuchlings-computer.local> Message-ID: <20070410230732.GA12644@Siri.local> On Tue, Apr 10, 2007 at 07:30:13AM -0400, A.M. Kuchling wrote: > Something's broken; I'm looking into it. David Goodger found the problem and repaired it. It was nothing to do with the PEPs; another directory was causing the build process to stop with an error. --amk From facundo at taniquetil.com.ar Wed Apr 11 02:37:02 2007 From: facundo at taniquetil.com.ar (Facundo Batista) Date: Wed, 11 Apr 2007 00:37:02 +0000 (UTC) Subject: [Python-Dev] Changes to decimal.py References: <20070410142641.BEU75896@ms09.lnh.mail.rcn.net> Message-ID: Raymond Hettinger wrote: > As promised in the decimal.py header, the spec > updates should all be considered as bugs and backported at some point > after they are fully tested and we're happy with them all around. > Also, as promised, the module should continue to run on Py2.3. Ok. So far, I'm dealing just with this. decimal.py passes, for example, the old quantize.decTest, but not the new one. My first step in this journey is to get the new test cases pass ok. > For the most part, many of the new operations can be implemented in terms > of the existing ops or in terms of the support functions that we > already use internally. Ideally, you can refactor common code while > leaving almost all of the exisiting algorithm implementation code > untouched. Yes. Some of the existing code will be touched, but mostly for bug fixing. > The spec's choice of new method names is unfortunate. You'll have to > come-up with something better than copy() and class(). The names, as the new functions will be discussed here in the second step. For example, I'm not absolute sure that something like... >>> Decimal("1100").xor(Decimal("0110") Decimal("1010") ...is actually needed. > FWIW, I think the new decimal development should probably be done in a > branch off of the current head. That way, you can check-in at will > and get feedback from everyone without risking the integrity of the head. This is a very good idea. > If you want to discuss anything during development, I'm usually available on > AOL instant messaging with the screename: raymondewt > > Likewise, consider soliciting Tim's input on how to implement the ln() operation. > That one will be tricky to get done efficiently and correctly. Great, thank you!! -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From python at rcn.com Wed Apr 11 03:16:40 2007 From: python at rcn.com (Raymond Hettinger) Date: Tue, 10 Apr 2007 21:16:40 -0400 (EDT) Subject: [Python-Dev] Fwd: Re: Changes to decimal.py Message-ID: <20070410211640.BEX02323@ms09.lnh.mail.rcn.net> [Facundo Batista] >The names, as the new functions will be discussed here in the second >step. For example, I'm not absolute sure that something like... > >>>> Decimal("1100").xor(Decimal("0110") >Decimal("1010") > >...is actually needed. > It doesn't matter. We promise to offer a full impleme Raymond From anthony at interlink.com.au Wed Apr 11 03:55:18 2007 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed, 11 Apr 2007 11:55:18 +1000 Subject: [Python-Dev] Py2.5.1 release candidate In-Reply-To: <461BFC8C.5020003@v.loewis.de> References: <20070410161410.BEV15978@ms09.lnh.mail.rcn.net> <461BFC8C.5020003@v.loewis.de> Message-ID: <200704111155.29576.anthony@interlink.com.au> On Wednesday 11 April 2007 07:07, Martin v. L?wis wrote: > Raymond Hettinger schrieb: > > It looks like the release candidate has been held-up for a bit. > > If it is going to stay held-up for a few days, can we unfreeze > > it so some bugfixes can go in (fixing the +0/-0 problem, > > eliminating some segfaults, and fixing some exception code)? > > No, the release binaries are all produced, and just await upload. Apologies for the delay in the uploading - some stuff came up over the Easter break, and then the website wouldn't rebuild (David and Andrew fixed the latter, yay!) Anthony -- Anthony Baxter It's never too late to have a happy childhood. From anthony at python.org Wed Apr 11 06:16:09 2007 From: anthony at python.org (Anthony Baxter) Date: Wed, 11 Apr 2007 14:16:09 +1000 Subject: [Python-Dev] RELEASED Python 2.5.1, release candidate 1 Message-ID: <200704111416.09853.anthony@python.org> On behalf of the Python development team and the Python community, I'm happy to announce the release of Python 2.5.1 (release candidate 1). This is the first bugfix release of Python 2.5. Python 2.5 is now in bugfix-only mode; no new features are being added. According to the release notes, over 150 bugs and patches have been addressed since Python 2.5, including a fair number in the new AST compiler (an internal implementation detail of the Python interpreter). For more information on Python 2.5.1, including download links for various platforms, release notes, and known issues, please see: http://www.python.org/2.5.1/ Highlights of this new release include: Bug fixes. According to the release notes, at least 150 have been fixed. Highlights of the previous major Python release (2.5) are available from the Python 2.5 page, at http://www.python.org/2.5/highlights.html Enjoy this release, Anthony Anthony Baxter anthony at python.org Python Release Manager (on behalf of the entire python-dev team) From barry at python.org Wed Apr 11 16:34:28 2007 From: barry at python.org (Barry Warsaw) Date: Wed, 11 Apr 2007 10:34:28 -0400 Subject: [Python-Dev] [Python-checkins] svn dead? In-Reply-To: <4E9372E6B2234D4F859320D896059A9508BF48EBAF@exchis.ccp.ad.local> References: <4E9372E6B2234D4F859320D896059A9508BF48EBAF@exchis.ccp.ad.local> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Apr 11, 2007, at 9:15 AM, Kristj?n Valur J?nsson wrote: > The SVN repository hasn?t answered http requests since this > morning. Anyone know what is up with that? Known breakage. We're working on it. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRhzx9HEjvBPtnXfVAQJwEQQApPQumTtLJPhlRU9a44dOuRI9DFGMQePD 6Hbsux0HcKvcLpR5QQlsUrKEAyGOt2qDm1YpoWM3uwSN68JnnlAc0iiYUQe1s8/U pRBy18eIhd9mGR2F2k9ZdDX0tqDapyVlc5bDb2jAaWSMwMO0AAUXyz4gtIP7thDW b9vZ18YXnvQ= =9riM -----END PGP SIGNATURE----- From barry at python.org Wed Apr 11 17:10:52 2007 From: barry at python.org (Barry Warsaw) Date: Wed, 11 Apr 2007 11:10:52 -0400 Subject: [Python-Dev] [Python-checkins] svn dead? In-Reply-To: References: <4E9372E6B2234D4F859320D896059A9508BF48EBAF@exchis.ccp.ad.local> Message-ID: <022AFF34-4BAE-4B26-BA3E-9823D9253B94@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Apr 11, 2007, at 10:34 AM, Barry Warsaw wrote: > On Apr 11, 2007, at 9:15 AM, Kristj?n Valur J?nsson wrote: >> The SVN repository hasn?t answered http requests since this >> morning. Anyone know what is up with that? > Known breakage. We're working on it. svn.python.org is back online now (both http and https). I'll take this opportunity to mention that we've ordered a new ssl cert for our expired one on https. It hasn't arrived yet, but should within a few days. I'll make another announcement when the new cert is installed. Cheers, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRhz6fXEjvBPtnXfVAQJKYAQAhtF4kA/5HeEvSZhktR3zAHkCZMjDcs1M sPeaAOOF55Pej0HagUi6haYcxoGeFOiGYq0pVzhu2FnuuUIi5LIOumn65w1s7xci z14OhYquTOMTyY2leyZdIj8eywg9ZMtKPXHS5spCm912/9gyuqYI6X9pkPakSRZ/ 0kD1/DmVvhE= =h93N -----END PGP SIGNATURE----- From barry at python.org Wed Apr 11 20:05:17 2007 From: barry at python.org (Barry Warsaw) Date: Wed, 11 Apr 2007 14:05:17 -0400 Subject: [Python-Dev] svn.python.org In-Reply-To: <022AFF34-4BAE-4B26-BA3E-9823D9253B94@python.org> References: <4E9372E6B2234D4F859320D896059A9508BF48EBAF@exchis.ccp.ad.local> <022AFF34-4BAE-4B26-BA3E-9823D9253B94@python.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Apr 11, 2007, at 11:10 AM, Barry Warsaw wrote: > I'll take this opportunity to mention that we've ordered a new ssl > cert for our expired one on https. It hasn't arrived yet, but > should within a few days. I'll make another announcement when the > new cert is installed. The new certificate has been installed. It's good until 2010. Please let me know if you have any problems with it. Cheers, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRh0jXnEjvBPtnXfVAQKuQwP/QZr/mP6bppUDw4Lz1CoKeWlICTSk0Qg/ g+RdFJFXgpwEwWUPRc8w6Cg7yxQZkpaWzoI1+wQRf10G6sw0JUWzN6A2wgBc0lNy 7W2sFFghjQ55gFqoYIGDs3dhLlKcNHhyhTWPFKTw5cnQ41GV9fVLXVtuMqGzeylg dFGUcXqazdU= =jv2x -----END PGP SIGNATURE----- From theller at ctypes.org Wed Apr 11 20:36:19 2007 From: theller at ctypes.org (Thomas Heller) Date: Wed, 11 Apr 2007 20:36:19 +0200 Subject: [Python-Dev] RELEASED Python 2.5.1, release candidate 1 In-Reply-To: <200704111416.09853.anthony@python.org> References: <200704111416.09853.anthony@python.org> Message-ID: Anthony Baxter schrieb: > On behalf of the Python development team and the Python community, > I'm happy to announce the release of Python 2.5.1 (release > candidate 1). > On the 2.5.1 page http://www.python.org/download/releases/2.5.1/ several things are wrong: - The download link for the AMD64 windows installer is missing; probably http://www.python.org/ftp/python/2.5.1/python-2.5.1c1.amd64.msi - The links in the last section (Files, MD5 checksums, signatures and sizes) are all wrong. Example: http://www.python.org/ftp/python/python-2.5.1c1.amd64.msi/Python-2.5.1c1.tgz Thomas From nnorwitz at gmail.com Wed Apr 11 20:48:02 2007 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 11 Apr 2007 11:48:02 -0700 Subject: [Python-Dev] RELEASED Python 2.5.1, release candidate 1 In-Reply-To: References: <200704111416.09853.anthony@python.org> Message-ID: On 4/11/07, Thomas Heller wrote: > > On the 2.5.1 page http://www.python.org/download/releases/2.5.1/ > several things are wrong: Can someone help fix this? I think Anthony is sleeping right now and generally pretty busy. Thanks, n From gustavotabares at gmail.com Wed Apr 11 21:03:47 2007 From: gustavotabares at gmail.com (Gustavo Tabares) Date: Wed, 11 Apr 2007 15:03:47 -0400 Subject: [Python-Dev] RELEASED Python 2.5.1, release candidate 1 In-Reply-To: References: <200704111416.09853.anthony@python.org> Message-ID: <1871c95a0704111203l75646f4bq3f9605db197fa92b@mail.gmail.com> The title for the highlights page also says Python 2.4 Gustavo On 4/11/07, Thomas Heller wrote: > > Anthony Baxter schrieb: > > On behalf of the Python development team and the Python community, > > I'm happy to announce the release of Python 2.5.1 (release > > candidate 1). > > > > On the 2.5.1 page http://www.python.org/download/releases/2.5.1/ > several things are wrong: > > - The download link for the AMD64 windows installer is missing; probably > > http://www.python.org/ftp/python/2.5.1/python-2.5.1c1.amd64.msi > > - The links in the last section (Files, MD5 checksums, signatures and > sizes) are all wrong. > Example: > > > http://www.python.org/ftp/python/python-2.5.1c1.amd64.msi/Python-2.5.1c1.tgz > > Thomas > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/gustavotabares%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070411/fd0ab395/attachment.htm From rwgk at yahoo.com Thu Apr 12 01:20:26 2007 From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve) Date: Wed, 11 Apr 2007 16:20:26 -0700 (PDT) Subject: [Python-Dev] Python 2.5.1c1 pickle problem Message-ID: <203406.61679.qm@web31101.mail.mud.yahoo.com> When pickling instances of an object derived from dict, the pickle in Python 2.5.1c1 calls the object's __getitem__() method. In contrast, earlier versions of Python incl. 2.5 don't call that method. Below is a minimal example with outputs. Is the difference in behavior an oversight or new feature? I couldn't find anything directly related in the release notes. The difference in behavior breaks some of our code. Thanks! Ralf class user(dict): def __getitem__(self, key): print "GETITEM", key if (__name__ == "__main__"): import sys print sys.version u = user() u[1] = 2 import pickle pickle.dumps(u) print "Done." 2.5.1c1 (r251c1:54692, Apr 11 2007, 16:15:52) [GCC 4.1.0 20060304 (Red Hat 4.1.0-3)] GETITEM 1 Done. 2.5 (r25:51908, Apr 11 2007, 16:11:19) [GCC 4.1.0 20060304 (Red Hat 4.1.0-3)] Done. 2.4.2 (#1, Feb 12 2006, 03:45:41) [GCC 4.1.0 20060210 (Red Hat 4.1.0-0.24)] Done. 2.3.4 (#1, Oct 26 2004, 16:45:38) [GCC 3.4.2 20041017 (Red Hat 3.4.2-6.fc3)] Done. 2.2.1 (#1, Aug 30 2002, 12:15:30) [GCC 3.2 20020822 (Red Hat Linux Rawhide 3.2-4)] Done. ____________________________________________________________________________________ No need to miss a message. Get email on-the-go with Yahoo! Mail for Mobile. Get started. http://mobile.yahoo.com/mail -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070411/1a27efc9/attachment.html From dalcinl at gmail.com Thu Apr 12 02:21:57 2007 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Wed, 11 Apr 2007 21:21:57 -0300 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) In-Reply-To: References: Message-ID: On 4/9/07, Travis Oliphant wrote: Travis, all this is far better and simpler than previous approaches... Just a few comments > The bufferinfo structure is:: > > struct bufferinfo { > void *buf; > Py_ssize_t len; > int readonly; > char *format; > int ndims; > Py_ssize_t *shape; > Py_ssize_t *strides; > Py_ssize_t *suboffsets; > void *internal; > }; 1) I assume that 'bufferinfo' structure will be part of public Python API. In such a case, I think it should be renamed and prefixed. Something like 'PyBufferInfo' sounds good? 2) I also think 'bufferinfo' could also have an 'itemsize' field filled if Py_BUF_ITEMSIZE flag is passed. What do you think? Exporters can possibly fill this field more efficiently than next parsing 'format' string, it can also save consumers from an API call. 3) It does make sense to make 'format' be 'const char *' ? 4) I am not sure about this, but perhaps 'buferingo' should save the flags passed to 'getbuffer' in a 'flags' field. This can be possibly needed at 'releasebuffer' call. > typedef struct { > PyObject_HEAD > PyObject *base; > struct bufferinfo view; > int itemsize; > int flags; > } PyMemoryViewObject; 5) If my previous comments go in, so 'PyMemoryViewObject' will not need 'itemsize' and 'flags' fields (they are in 'bufferinfo' structure). 6) Perhaps 'view' field can be renamed to 'info'. > int PyObject_SizeFromFormat(char *) 7) Why not 'const char *' here? > int PyObject_GetContiguous(PyObject *obj, void **buf, Py_ssize_t *len, > int fortran) > > Return a contiguous chunk of memory representing the buffer. If a > copy is made then return 1. If no copy was needed return 0. 8) If a copy was made, What should consumers call to free memory? > If the object is multi-dimensional, then if > fortran is 1, the first dimension of the underlying array will vary > the fastest in the buffer. If fortran is 0, then the last dimension > will vary the fastest (C-style contiguous). If fortran is -1, then it > does not matter and you will get whatever the object decides is more > efficient. 9) What about using a char, like 'c' or 'C', and 'f' or 'F', and 0 or 'a' or 'A' (any) ? > int PyObject_CopyToObject(PyObject *obj, void *buf, Py_ssize_t len, > int fortran) 10) Better name? Perhaps PyObject_CopyBuffer or PyObject_CopyMemory? > int PyObject_SizeFromFormat(char *) > > int PyObject_IsContiguous(struct bufferinfo *view, int fortran); > > void PyObject_FillContiguousStrides(int *ndims, Py_ssize_t *shape, > int itemsize, > Py_ssize_t *strides, int fortran) > > int PyObject_FillBufferInfo(struct bufferinfo *view, void *buf, Py_ssize_t len, > int readonly, int infoflags) > 11) Perhaps the 'PyObject_' prefix is wrong, as those functions does not operate with Python objects. Regards, -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From greg.ewing at canterbury.ac.nz Thu Apr 12 03:29:03 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 12 Apr 2007 13:29:03 +1200 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) In-Reply-To: References: Message-ID: <461D8B5F.2020208@canterbury.ac.nz> Lisandro Dalcin wrote: > 4) I am not sure about this, but perhaps 'buferingo' should save the > flags passed to 'getbuffer' in a 'flags' field. This can be possibly > needed at 'releasebuffer' call. The object isn't necessarily providing all the things that were requested in the flags, so it's going to have to keep its own track of what has been allocated. > 11) Perhaps the 'PyObject_' prefix is wrong, as those functions does > not operate with Python objects. Yes, PyBuffer_ would be a better prefix for these, I think. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Thu Apr 12 03:35:42 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 12 Apr 2007 13:35:42 +1200 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) In-Reply-To: References: Message-ID: <461D8CEE.2040503@canterbury.ac.nz> From PEP 3118: A memory-view object is an extended buffer object that should replace the buffer object in Python 3K. typedef struct { PyObject_HEAD PyObject *base; struct bufferinfo view; int itemsize; int flags; } PyMemoryViewObject; If the purpose is to provide Python-level access to an object via its buffer interface, then keeping a bufferinfo struct in it is the wrong implementation strategy, since it implies keeping the base object's memory locked as long as the view object exists. That was the mistake made by the original buffer object, and the solution is not to hold onto the info returned by the base object's buffer interface, but to make a new buffer request for each Python-level access. If that's not the purpose of this object, you need to explain what its purpose actually is. Also some of what you say about this object is rather unclear, e.g. "It exports a view using the base object." I don't know what that is supposed to mean. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From oliphant.travis at ieee.org Thu Apr 12 04:06:58 2007 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 11 Apr 2007 20:06:58 -0600 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) In-Reply-To: <461D8CEE.2040503@canterbury.ac.nz> References: <461D8CEE.2040503@canterbury.ac.nz> Message-ID: <461D9442.7030109@ieee.org> Greg Ewing wrote: > From PEP 3118: > > A memory-view object is an extended buffer object that > should replace the buffer object in Python 3K. > > typedef struct { > PyObject_HEAD > PyObject *base; > struct bufferinfo view; > int itemsize; > int flags; > } PyMemoryViewObject; > > If the purpose is to provide Python-level access to an > object via its buffer interface, then keeping a bufferinfo > struct in it is the wrong implementation strategy, since it > implies keeping the base object's memory locked as long as > the view object exists. Yes, but that was the intention. The MemoryView Object is basically an N-d array object. > > That was the mistake made by the original buffer object, > and the solution is not to hold onto the info returned by > the base object's buffer interface, but to make a new > buffer request for each Python-level access. I could see this approach also, but if we went this way then the memory view object should hold "slice" information so that it can be a "sliced" view of a memory area. Because slicing NumPy array's already does it by holding on to a view, I guess having an object that doesn't hold on to a view in Python but "re-gets" it every time it is needed, would be useful. In that case: typedef struct { PyObject_HEAD PyObject *base; int ndims; PyObject **slices; /* or 3 Py_ssize_t arrays */ int flags; } PyMemoryViewObject; would be enough to store, I suppose. -Travis From pythondev at aerojockey.com Thu Apr 12 04:17:35 2007 From: pythondev at aerojockey.com (Carl Banks) Date: Wed, 11 Apr 2007 22:17:35 -0400 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) In-Reply-To: <461AE11E.8040901@ieee.org> References: <461AD36D.4080608@aerojockey.com> <461AE11E.8040901@ieee.org> Message-ID: <461D96BF.2080702@aerojockey.com> Travis Oliphant wrote: > Carl Banks wrote: >> >> >> Travis Oliphant wrote: >> > Py_BUF_READONLY >> > The returned buffer must be readonly and the underlying object >> should make >> > its memory readonly if that is possible. >> >> I don't like the "if possible" thing. If it makes no guarantees, it >> pretty much useless over Py_BUF_SIMPLE. > O.K. Let's make it raise an error if it can't set it read-only. The thing that bothers me about this whole flags setup is that different flags can do opposite things. Some of the flags RESTRICT the kind of buffers that can be exported (Py_BUF_WRITABLE); other flags EXPAND the kind of buffers that can be exported (Py_BUF_INDIRECT). That is highly confusing and I'm -1 on any proposal that includes both behaviors. (Mutually exclusive sets of flags are a minor exception: they can be thought of as either RESTICTING or EXPANDING, so they could be mixed with either.) I originally suggested a small set of flags that expand the set of allowed buffers. Here's a little Venn diagram of buffers to illustrate what I was thinking: http://www.aerojockey.com/temp/venn.png With no flags, the only buffers allowed to be returned are in the "All" circle but no others. Add Py_BUF_WRITABLE and now you can export writable buffers as well. Add Py_BUF_STRIDED and the strided circle is opened to you, and so on. My recommendation is, any flag should turn on some circle in the Venn diagram (it could be a circle I didn't draw--shaped arrays, for example--but it should be *some* circle). >>> Py_BUF_FORMAT >>> The consumer will be using the format string information so make >>> sure that member is filled correctly. >> >> Is the idea to throw an exception if there's some other data format >> besides "b", and this flag isn't set? It seems superfluous otherwise. > > The idea is that a consumer may not care about the format and the > exporter may want to know that to simplify the interface. In other > words the flag is a way for the consumer to communicate that it wants > format information (or not). I'm -1 on using the flags for this. It's completely out of character compared to the rest of the flags. All other flags are there for the benefit of the consumer; this flag is useless to the consumer. More concretely, all the rest of the flags are there to tell the exporter what kind of buffer they're prepared to accept. This flag, alone, does not do that. Even the benefits to the exporter are dubious. This flag can't reduce code complexity, since all buffer objects have to be prepared to furnish type information. At best, this flag is a rare optimization. In fact, most buffers are going to point format to a constant string, regardless of whether this flag was passed or not: bufinfo->format = "b"; > If the exporter wants to raise an exception if the format is not > requested is up to the exporter. That seems like a bad idea. Suppose I have a contiguous numpy array of floats and I want to view it as a sequence of bytes. If the exporter's allowed to raise an exception for this, any consumer that wanted a data-neutral view of the data would still have to pass Py_BUF_FORMAT to guard against this. Wouldn't that be ironic? >>> Py_BUF_SHAPE >>> The consumer can (and might) make use of using the ndims and shape >>> members of the structure >>> so make sure they are filled in correctly. Py_BUF_STRIDES >>> (implies SHAPE) >>> The consumer can (and might) make use of the strides member of the >>> structure (as well >>> as ndims and shape) >> >> Is there any reasonable benefit for allowing Py_BUF_SHAPE without >> Py_BUF_STRIDES? Would the array be C- or Fortran-like? > > Yes, I could see a consumer not being able to handle simple striding > but could handle shape information. Many users of NumPy arrays like to > think of the array as an N-d array but want to ignore striding. Ok, but is the indexing row-major or column-major? That has to be decided. Carl Banks From python at rcn.com Thu Apr 12 04:19:19 2007 From: python at rcn.com (Raymond Hettinger) Date: Wed, 11 Apr 2007 22:19:19 -0400 (EDT) Subject: [Python-Dev] Python 2.5.1c1 pickle problem Message-ID: <20070411221919.BFB04685@ms09.lnh.mail.rcn.net> The pickle issue may be related to revision 53655 fixing a psuedo-bug (it was arguable whether current or prior behavior was most desirable). Will look at this more and will report back. Raymond From rkelley at cs.uri.edu Thu Apr 12 05:31:15 2007 From: rkelley at cs.uri.edu (Ryan Kelley) Date: Wed, 11 Apr 2007 23:31:15 -0400 Subject: [Python-Dev] Just saying hello Message-ID: <20070411233115.08fewb0wg8gooos0@www.cs.uri.edu> Hi I just got accepted on the google summer of code to work on a project for python. I just wanted to say hello to everyone out there as i know i will end up asking a few questions before the summers over. -- Ryan Kelley NETS - University of Rhode Island Network Security http://www.uri.edu/security http://www.uri.edu/security/sentinel http://www.uri.edu/virus From python at rcn.com Thu Apr 12 09:14:50 2007 From: python at rcn.com (Raymond Hettinger) Date: Thu, 12 Apr 2007 00:14:50 -0700 Subject: [Python-Dev] Python 2.5.1c1 pickle problem References: <20070411221919.BFB04685@ms09.lnh.mail.rcn.net> Message-ID: <005201c77cd2$4c64cf80$f001a8c0@RaymondLaptop1> Ralf, your issue is arising because of revision 53655 which fixes SF 1615701. Subclasses of builtins are pickled using obj.__reduce_ex__() which returns a tuple with a _reconstructor function and a tuple of arguments to that function. That tuple of arguments include the subclass name, the base class, and a state which is computed as state=base(obj). So, in your case, state=dict(yourobject). Formerly, casting a dict subclass to a dict would use a fast internal copying method which duplicated the hash table directly. The OP for SF 1615701 felt strongly that this was buggy behavior because it would bypass the custom __getitem__ () method implemented by his subclass. An argument in favor of the bugfix was that dict(m) or dict.update(m) shouldn't behave differently depending on whether m was a dict-subclass or another mapping-like object. For the latter, writing dict(m) is the semantic equivalent for writing dict((k,m[k]) for k in m.keys()). This is where your __getitem__() call comes from. At first, I did not agree with the proposed bugfix because 1) it would unnecessarily slowdown several operations on dict subclasses and make them less attractive; 2) it violated the OpenClosedPrinciple where the subclass has no business knowing how the internals of the base class are implemented (in this case, the subclass should not depend on whether dict(m) is implemented in terms of keys/getitem, in terms of iteritems, or through direct access to the hash table); and 3) because there were clearer and more reliable ways to implement the use cases suggested by the OP. The matter was briefly discussed on SF and python-dev where the OP found other proponents. It became clear that if left alone, the existing implementation would continue to defy the expectations of some folks subclassing builtins. Those expectations arise naturally from a mental model of builtins behaving just like a fast version of the most natural pure-Python equivalents. I'm not sure what your code was doing where the bugfix would cause breakage. If its __getitem__() override returned a meaningful value for each element in obj.keys(), then it should have worked fine. Of course, if it was raising an exception or triggering a side-effect, then one could argue that the bugfix was working as intended by allowing the subclasser to affect how the base class goes about its business. Am leaving this open for others to discuss and decide. The old behavior was surprising to some, but the revised behavior also appears to have some unforeseen consequences. Raymond P.S. In addition to rev 53655, a number of similar changes were made to sets. From P at draigBrady.com Thu Apr 12 13:47:25 2007 From: P at draigBrady.com (=?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?=) Date: Thu, 12 Apr 2007 12:47:25 +0100 Subject: [Python-Dev] [PATCH] pep 0324 URL update Message-ID: <461E1C4D.80007@draigBrady.com> The old URL still works, but will probably not in the future. thanks, P?draig. -------------- next part -------------- A non-text attachment was scrubbed... Name: pep-0324.pixelbeat.diff Type: text/x-patch Size: 363 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20070412/12ab5d57/attachment.bin From amk at amk.ca Thu Apr 12 14:47:47 2007 From: amk at amk.ca (A.M. Kuchling) Date: Thu, 12 Apr 2007 08:47:47 -0400 Subject: [Python-Dev] RELEASED Python 2.5.1, release candidate 1 In-Reply-To: <1871c95a0704111203l75646f4bq3f9605db197fa92b@mail.gmail.com> References: <200704111416.09853.anthony@python.org> <1871c95a0704111203l75646f4bq3f9605db197fa92b@mail.gmail.com> Message-ID: <20070412124747.GA4854@localhost.localdomain> On Wed, Apr 11, 2007 at 03:03:47PM -0400, Gustavo Tabares wrote: > The title for the highlights page also says Python 2.4 I've fixed the broken download links that Thomas reported. I can't find the above error, though the bugs and license pages did have a 'Python 2.4.4' heading. --amk From gustavotabares at gmail.com Thu Apr 12 15:30:16 2007 From: gustavotabares at gmail.com (Gustavo Tabares) Date: Thu, 12 Apr 2007 09:30:16 -0400 Subject: [Python-Dev] RELEASED Python 2.5.1, release candidate 1 In-Reply-To: <20070412124747.GA4854@localhost.localdomain> References: <200704111416.09853.anthony@python.org> <1871c95a0704111203l75646f4bq3f9605db197fa92b@mail.gmail.com> <20070412124747.GA4854@localhost.localdomain> Message-ID: <1871c95a0704120630s5391926dtc9750b1ee5c88171@mail.gmail.com> Open http://www.python.org/2.5/highlights.html The title of the page says "Highlights: Python 2.4" On 4/12/07, A.M. Kuchling wrote: > > On Wed, Apr 11, 2007 at 03:03:47PM -0400, Gustavo Tabares wrote: > > The title for the highlights page also says Python 2.4 > > I've fixed the broken download links that Thomas reported. I can't > find the above error, though the bugs and license pages did have a > 'Python 2.4.4' heading. > > --amk > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/gustavotabares%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070412/d53a6a9c/attachment.htm From jeremy at alum.mit.edu Thu Apr 12 16:35:50 2007 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Thu, 12 Apr 2007 10:35:50 -0400 Subject: [Python-Dev] HTTP responses and errors In-Reply-To: References: <46092EA2.3030804@v.loewis.de> <46095087.9050002@v.loewis.de> Message-ID: On 4/10/07, Facundo Batista wrote: > Facundo Batista wrote: > > > Martin v. L?wis wrote: > > ... > > > >> think it should treat all 2xx responses as success. Callers can > >> then still check the response code themselves if they need to. > > > > The same I think. If nobody has a conflic with this decission, I'll fix > > this. > > Nobody raised any objection, I'll fix this these days. +1 Jeremy From aahz at pythoncraft.com Thu Apr 12 17:22:52 2007 From: aahz at pythoncraft.com (Aahz) Date: Thu, 12 Apr 2007 08:22:52 -0700 Subject: [Python-Dev] Just saying hello In-Reply-To: <20070411233115.08fewb0wg8gooos0@www.cs.uri.edu> References: <20070411233115.08fewb0wg8gooos0@www.cs.uri.edu> Message-ID: <20070412152252.GA29240@panix.com> On Wed, Apr 11, 2007, Ryan Kelley wrote: > > I just got accepted on the google summer of code to work on a project > for python. I just wanted to say hello to everyone out there as i know > i will end up asking a few questions before the summers over. Congrats! -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Help a hearing-impaired person: http://rule6.info/hearing.html From tjreedy at udel.edu Thu Apr 12 18:12:48 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 12 Apr 2007 12:12:48 -0400 Subject: [Python-Dev] RELEASED Python 2.5.1, release candidate 1 References: <200704111416.09853.anthony@python.org><1871c95a0704111203l75646f4bq3f9605db197fa92b@mail.gmail.com><20070412124747.GA4854@localhost.localdomain> <1871c95a0704120630s5391926dtc9750b1ee5c88171@mail.gmail.com> Message-ID: "Gustavo Tabares" wrote in message news:1871c95a0704120630s5391926dtc9750b1ee5c88171 at mail.gmail.com... | Open http://www.python.org/2.5/highlights.html | | The title of the page says "Highlights: Python 2.4" To be clear, it is not the heading/title *on* the page, which was changed, but the title *of* the page displayed on the browser's title bar (and the Windows task bar). From draghuram at gmail.com Thu Apr 12 23:59:26 2007 From: draghuram at gmail.com (Raghuram Devarakonda) Date: Thu, 12 Apr 2007 17:59:26 -0400 Subject: [Python-Dev] build problem on windows: unable to find getbuildinfo2.c Message-ID: <2c51ecee0704121459n71514e6dtaa3b12dd6dd0828@mail.gmail.com> Hi, I am trying to build latest python on Windows XP and ran into the following error: "c1 : fatal error C1083: Cannot open source file: '.\getbuildinfo2.c': No such file or directory" I am using Visual C++ 2005 Express Edition to build. I opened "PCbuilld8/pcbuild.sln" and did a "release" build of "pythoncore". I didn't find getbuildinfo2.c in the source. Can some one tell me if I am missing some thing here? Are there any additional steps need to follow on windows? Thanks, Raghu. From brett at python.org Fri Apr 13 00:27:47 2007 From: brett at python.org (Brett Cannon) Date: Thu, 12 Apr 2007 15:27:47 -0700 Subject: [Python-Dev] build problem on windows: unable to find getbuildinfo2.c In-Reply-To: <2c51ecee0704121459n71514e6dtaa3b12dd6dd0828@mail.gmail.com> References: <2c51ecee0704121459n71514e6dtaa3b12dd6dd0828@mail.gmail.com> Message-ID: This is not the right list to ask for technical support. Please either ask some place like comp.lang.python or file a bug report. -Brett On 4/12/07, Raghuram Devarakonda wrote: > > Hi, > > I am trying to build latest python on Windows XP and ran into the > following error: > > "c1 : fatal error C1083: Cannot open source file: '.\getbuildinfo2.c': > No such file or directory" > > I am using Visual C++ 2005 Express Edition to build. I opened > "PCbuilld8/pcbuild.sln" and did a "release" build of "pythoncore". > > I didn't find getbuildinfo2.c in the source. Can some one tell me if > I am missing some thing here? Are there any additional steps need to > follow on windows? > > Thanks, > Raghu. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/brett%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070412/3701b24c/attachment.html From greg.ewing at canterbury.ac.nz Fri Apr 13 01:34:59 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 13 Apr 2007 11:34:59 +1200 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) In-Reply-To: <461D9442.7030109@ieee.org> References: <461D8CEE.2040503@canterbury.ac.nz> <461D9442.7030109@ieee.org> Message-ID: <461EC223.5000002@canterbury.ac.nz> Travis Oliphant wrote: > Because slicing NumPy array's already does it by holding on to a view, I > guess having an object that doesn't hold on to a view in Python but > "re-gets" it every time it is needed, would be useful. I guess this problem doesn't arise in NumPy, because the size of the memory block doesn't change once the array is created, so locking isn't an issue. But in the more general case, I think it would be surprising if one object refused to carry out some operation just because of the existence of some other object. Also, keep in mind that dropping references to an object isn't a guarantee that it will be deallocated immediately, so there would need to be some kind of explicit closing operation on the view object, making things more messy. -- Greg From oliphant.travis at ieee.org Fri Apr 13 06:09:06 2007 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 12 Apr 2007 22:09:06 -0600 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) In-Reply-To: <50862ebd0704091616v5ec35d2at88a867b7a5f4220c@mail.gmail.com> References: <50862ebd0704091616v5ec35d2at88a867b7a5f4220c@mail.gmail.com> Message-ID: <461F0262.5030300@ieee.org> Neil Hodgson wrote: > Travis Oliphant: > >> PEP: 3118 >> ... > > I'd like to see the PEP include discussion of what to do when an > incompatible request is received while locked. Should there be a > standard "Can't do that: my buffer has been got" exception? I'm not sure what standard to make a decision about that by. Sure, why not? It's not something I'd considered. -Travis From oliphant.travis at ieee.org Fri Apr 13 06:19:20 2007 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 12 Apr 2007 22:19:20 -0600 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) In-Reply-To: References: Message-ID: <461F04C8.6060108@ieee.org> Lisandro Dalcin wrote: > On 4/9/07, Travis Oliphant wrote: > > Travis, all this is far better and simpler than previous approaches... > Just a few comments Thanks for your wonderful suggestions. I've incorporated many of them. > > 1) I assume that 'bufferinfo' structure will be part of public Python > API. In such a case, I think it should be renamed and prefixed. > Something like 'PyBufferInfo' sounds good? I prefer that as well. > > 2) I also think 'bufferinfo' could also have an 'itemsize' field > filled if Py_BUF_ITEMSIZE flag is passed. What do you think? Exporters > can possibly fill this field more efficiently than next parsing > 'format' string, it can also save consumers from an API call. I think the itemsize member is a good idea. I'm re-visiting what the flags should be after suggestions by Carl. > > 3) It does make sense to make 'format' be 'const char *' ? Yes, > > 4) I am not sure about this, but perhaps 'buferingo' should save the > flags passed to 'getbuffer' in a 'flags' field. This can be possibly > needed at 'releasebuffer' call. > I think this is un-necessary. > >> typedef struct { >> PyObject_HEAD >> PyObject *base; >> struct bufferinfo view; >> int itemsize; >> int flags; >> } PyMemoryViewObject; > > 5) If my previous comments go in, so 'PyMemoryViewObject' will not > need 'itemsize' and 'flags' fields (they are in 'bufferinfo' > structure). > After suggestions by Greg, I like the idea of the PyMemoryViewObject holding a pointer to another object (from which it gets memory on request) as well as information about a slice of that memory. Thus, the memory view object is something like: typedef struct { PyObject_HEAD PyObject *base; int ndims; Py_ssize_t *offsets; /* slice starts */ Py_ssize_t *lengths; /* slice stops */ Py_ssize_t *skips; /* slice steps */ } PyMemoryViewObject; It is more convenient to store any slicing information (so a memory view object could store an arbitrary slice of another object) as offsets, lengths, and skips which can be used to adjust the memory buffer returned by base. >> int PyObject_GetContiguous(PyObject *obj, void **buf, Py_ssize_t >> *len, >> int fortran) >> >> Return a contiguous chunk of memory representing the buffer. If a >> copy is made then return 1. If no copy was needed return 0. > > 8) If a copy was made, What should consumers call to free memory? You are right. We need a free function. > 9) What about using a char, like 'c' or 'C', and 'f' or 'F', and 0 or > 'a' or 'A' (any) ? I'm happy with that too. > >> int PyObject_CopyToObject(PyObject *obj, void *buf, Py_ssize_t len, >> int fortran) > > 10) Better name? Perhaps PyObject_CopyBuffer or PyObject_CopyMemory? I'm not sure why those are better names. The current name reflects the idea of copying the data into the object. > >> int PyObject_SizeFromFormat(char *) >> >> int PyObject_IsContiguous(struct bufferinfo *view, int fortran); >> >> void PyObject_FillContiguousStrides(int *ndims, Py_ssize_t *shape, >> int itemsize, >> Py_ssize_t *strides, int >> fortran) >> >> int PyObject_FillBufferInfo(struct bufferinfo *view, void *buf, >> Py_ssize_t len, >> int readonly, int infoflags) >> > > 11) Perhaps the 'PyObject_' prefix is wrong, as those functions does > not operate with Python objects. Agreed. -Travis From martin at v.loewis.de Fri Apr 13 07:48:17 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 13 Apr 2007 07:48:17 +0200 Subject: [Python-Dev] build problem on windows: unable to find getbuildinfo2.c In-Reply-To: <2c51ecee0704121459n71514e6dtaa3b12dd6dd0828@mail.gmail.com> References: <2c51ecee0704121459n71514e6dtaa3b12dd6dd0828@mail.gmail.com> Message-ID: <461F19A1.4020907@v.loewis.de> > I didn't find getbuildinfo2.c in the source. Can some one tell me if > I am missing some thing here? Are there any additional steps need to > follow on windows? It's a generated file. Search all build description files for that file name to find out how it is generated, and then research why generating it fails on your machine. Regards, Martin From oliphant.travis at ieee.org Fri Apr 13 09:03:04 2007 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Fri, 13 Apr 2007 01:03:04 -0600 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) In-Reply-To: <461D96BF.2080702@aerojockey.com> References: <461AD36D.4080608@aerojockey.com> <461AE11E.8040901@ieee.org> <461D96BF.2080702@aerojockey.com> Message-ID: <461F2B28.6030508@ieee.org> Carl Banks wrote: > > The thing that bothers me about this whole flags setup is that > different flags can do opposite things. > > Some of the flags RESTRICT the kind of buffers that can be > exported (Py_BUF_WRITABLE); other flags EXPAND the kind of buffers that > can be exported (Py_BUF_INDIRECT). That is highly confusing and I'm -1 > on any proposal that includes both behaviors. (Mutually exclusive sets > of flags are a minor exception: they can be thought of as either > RESTICTING or EXPANDING, so they could be mixed with either.) The mutually exclusive set is the one example of the restriction that you gave. I think the flags setup I've described is much closer to your Venn diagram concept than you give it credit for. I've re-worded some of the discussion (see http://projects.scipy.org/scipy/numpy/browser/trunk/numpy/numpy/doc/pep_buffer.txt ) so that it is more clear that each flag is a description what kind of buffer the consumer is prepared to deal with. For example, if the consumer cares about what's 'in' the array, it uses Py_BUF_FORMAT. Exporters are free to do what they want with this information. I agree that NumPy would not force you to use it's buffer only as a region of some specific type, but some other object may want to be much more restrictive and only export to consumers who will recognize the data stored for what it is. I think it's up to the exporters to decide whether or not to raise an error when a certain kind of buffer is requested. Basically, every flag corresponds to a different property of the buffer that the consumer is requesting: Py_BUF_SIMPLE --- you are requesting the simplest possible (0x00) Py_BUF_WRITEABLE -- get a writeable buffer (0x01) Py_BUF_READONLY -- get a read-only buffer (0x02) Py_BUF_FORMAT -- get a "formatted" buffer. (0x04) Py_BUF_SHAPE -- get a buffer with shape information (0x08) Py_BUF_STRIDES -- get a buffer with stride information (and shape) (0x18) Py_BUF_OFFSET -- get a buffer with suboffsets (and strides and shape) (0x38) This is a logical sequence. There is progression. Each flag is a bit that indicates something about how the consumer can use the buffer. In other words, the consumer states what kind of buffer is being requested. The exporter obliges (and can save possibly significant time if the consumer is not requesting the information it must otherwise produce). > I originally suggested a small set of flags that expand the set of > allowed buffers. Here's a little Venn diagram of buffers to > illustrate what I was thinking: > > http://www.aerojockey.com/temp/venn.png > > With no flags, the only buffers allowed to be returned are in the "All" > circle but no others. Add Py_BUF_WRITABLE and now you can export > writable buffers as well. Add Py_BUF_STRIDED and the strided circle is > opened to you, and so on. > > My recommendation is, any flag should turn on some circle in the Venn > diagram (it could be a circle I didn't draw--shaped arrays, for > example--but it should be *some* circle). I don't think your Venn diagram is broad enough and it un-necessarily limits the use of flags to communicate between consumer and exporter. We don't have to ram these flags down that point-of-view for them to be productive. If you have a specific alternative proposal, or specific criticisms, then I'm very willing to hear them. I've thought through the flags again, and I'm not sure how I would change them. They make sense to me. Especially in light of past usages of the buffer protocol (where most people request read-or-write buffers i.e. Py_BUF_SIMPLE. I'm also not sure our mental diagrams are both oriented the same. For me, the most restrictive requests are PY_BUF_WRITEABLE | Py_BUF_FORMAT and Py_BUF_READONLY | Py_BUF_FORMAT The most un-restrictive request (the largest circle in my mental Venn diagram) is Py_BUF_OFFSETS followed by Py_BUF_STRIDES followed by Py_BUF_SHAPE adding Py_BUF_FORMATS, Py_BUF_WRITEABLE, or Py_BUF_READONLY serves to restrict any of the other circles Is this dual use of flags what bothers you? (i.e. use of some flags for restricting circles in your Venn diagram that are turned on by other flags? --- you say Py_BUF_OFFSETS | Py_BUF_WRITEABLE to get the intersection of the Py_BUF_OFFSETS largest circle with the WRITEABLE subset?) Such concerns are not convincing to me. Just don't think of the flags in that way. Think of them as turning "on" members of the bufferinfo structure. > > >>>> Py_BUF_FORMAT >>>> The consumer will be using the format string information so make >>>> sure that member is filled correctly. >>> >>> Is the idea to throw an exception if there's some other data format >>> besides "b", and this flag isn't set? It seems superfluous otherwise. >> >> The idea is that a consumer may not care about the format and the >> exporter may want to know that to simplify the interface. In other >> words the flag is a way for the consumer to communicate that it wants >> format information (or not). > > I'm -1 on using the flags for this. It's completely out of character > compared to the rest of the flags. All other flags are there for the > benefit of the consumer; this flag is useless to the consumer. > > More concretely, all the rest of the flags are there to tell the > exporter what kind of buffer they're prepared to accept. This flag, > alone, does not do that. I agree. This flag is used by the consumer to state that it wants, will be making note of, and is prepared to deal with a "formatted" buffer. I think it's short-sighted to have flags to control providing the other members of the PyBuffer structure and not this one. Actually, the "rare" optimization to the exporter can still be significant if most consumers don't care about it's format (which perhaps it has to construct at request time). >> If the exporter wants to raise an exception if the format is not >> requested is up to the exporter. > > That seems like a bad idea. Suppose I have a contiguous numpy array of > floats and I want to view it as a sequence of bytes. If the exporter's > allowed to raise an exception for this, any consumer that wanted a > data-neutral view of the data would still have to pass Py_BUF_FORMAT to > guard against this. Wouldn't that be ironic? I agree that NumPy would not do this as it would allow un-formatted views. In fact, most exporters would probably choose not to raise an error. But, an exporter that really only wants it's data viewed as (e.g. complex numbers) would raise an error to force a consumer to be explicit (by providing the Py_BUF_FORMAT flag) about by-passing that desire. > > Ok, but is the indexing row-major or column-major? That has to be > decided. I think it's called row-major, but I don't like that term because what do you mean for an N-D array? I use 'last-index varies the fastest' if I want to be explicity and C-contiguous if we know what we are talking about. Yes this is assumed in such cases. -Travis From and-dev at doxdesk.com Fri Apr 13 12:47:56 2007 From: and-dev at doxdesk.com (Andrew Clover) Date: Fri, 13 Apr 2007 11:47:56 +0100 Subject: [Python-Dev] minidom and DOM level 2 In-Reply-To: References: <4603942D.4060508@v.loewis.de> <4617B275.4030406@doxdesk.com> Message-ID: <461F5FDC.4070609@doxdesk.com> Jason Orendorff wrote: > I don't suppose you'd be willing to update it for Python 2.5, would you? Can do, but at this point I'm not aware of any work having been done on the issues listed there between the 2.3 and 2.5 releases. The danger is people may be used to the "wrong" minidom behaviours, given they have been static for so long and are in many cases central to how minidom works. -- And Clover mailto:and at doxdesk.com http://www.doxdesk.com/ From amk at amk.ca Fri Apr 13 13:07:08 2007 From: amk at amk.ca (A.M. Kuchling) Date: Fri, 13 Apr 2007 07:07:08 -0400 Subject: [Python-Dev] RELEASED Python 2.5.1, release candidate 1 In-Reply-To: References: <1871c95a0704120630s5391926dtc9750b1ee5c88171@mail.gmail.com> Message-ID: <20070413110708.GA13817@Siri.local> On Thu, Apr 12, 2007 at 12:12:48PM -0400, Terry Reedy wrote: > | Open http://www.python.org/2.5/highlights.html > | > | The title of the page says "Highlights: Python 2.4" Ah, so it's a 2.5 page, not a 2.5.1 page; that's why my grep didn't find it. Fixed now; thanks! --amk From jason.orendorff at gmail.com Fri Apr 13 15:28:38 2007 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Fri, 13 Apr 2007 09:28:38 -0400 Subject: [Python-Dev] minidom and DOM level 2 In-Reply-To: <461F5FDC.4070609@doxdesk.com> References: <4603942D.4060508@v.loewis.de> <4617B275.4030406@doxdesk.com> <461F5FDC.4070609@doxdesk.com> Message-ID: On 4/13/07, Andrew Clover wrote: > Jason Orendorff wrote: > > I don't suppose you'd be willing to update it for Python 2.5, would you? > > Can do, but at this point I'm not aware of any work having been done on > the issues listed there between the 2.3 and 2.5 releases. I've been running the DOM test suite against trunk, using your test harness. It's kind of alarming at first that over 100 tests fail. :) But many of the failures involve entity references. An even larger portion involve error cases: we accept things we should check and reject. For example, doc.createElement('\t') should fail. These are certainly bugs, and they're easy to fix. I'm working through them. > The danger is people may be used to the "wrong" minidom behaviours, > given they have been static for so long and are in many cases central to > how minidom works. When I get to these, I'll post about it. -j From draghuram at gmail.com Fri Apr 13 15:59:59 2007 From: draghuram at gmail.com (Raghuram Devarakonda) Date: Fri, 13 Apr 2007 09:59:59 -0400 Subject: [Python-Dev] build problem on windows: unable to find getbuildinfo2.c In-Reply-To: <461F19A1.4020907@v.loewis.de> References: <2c51ecee0704121459n71514e6dtaa3b12dd6dd0828@mail.gmail.com> <461F19A1.4020907@v.loewis.de> Message-ID: <2c51ecee0704130659r48d7ac42v9e1a0ca494be309e@mail.gmail.com> On 4/13/07, "Martin v. L?wis" wrote: > > I didn't find getbuildinfo2.c in the source. Can some one tell me if > > I am missing some thing here? Are there any additional steps need to > > follow on windows? > > It's a generated file. Search all build description files for that > file name to find out how it is generated, and then research why > generating it fails on your machine. Thanks. I found that make_buildinfo project failed with some unresolved symbol errors causing the missing getbuildinfo2.c. Once I fixed it, python got built just fine. From dalcinl at gmail.com Fri Apr 13 16:05:41 2007 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Fri, 13 Apr 2007 11:05:41 -0300 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) In-Reply-To: <461F04C8.6060108@ieee.org> References: <461F04C8.6060108@ieee.org> Message-ID: On 4/13/07, Travis Oliphant wrote: > >> int PyObject_GetContiguous(PyObject *obj, void **buf, Py_ssize_t > >> *len, > >> int fortran) > >> > >> Return a contiguous chunk of memory representing the buffer. If a > >> copy is made then return 1. If no copy was needed return 0. > > > > 8) If a copy was made, What should consumers call to free memory? > > You are right. We need a free function. > I think now the memory perhaps should be allocated with PyMem_New and deallocated with PyMem_Free. Additionally, the return should perhaps indicate success or failure, and a new argument should be passed in order to know if a copy was made, something like int PyObject_GetContiguous(PyObject *obj, void **buf, Py_ssize_t *len, int *copy, char layout) -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From barry at python.org Fri Apr 13 16:32:28 2007 From: barry at python.org (Barry Warsaw) Date: Fri, 13 Apr 2007 10:32:28 -0400 Subject: [Python-Dev] test_pty.py hangs in verbose mode on Mac OS X? Message-ID: <905BBADA-70C7-4A80-978F-82646AF54CE8@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I've been getting some test failures in Python 2.5 svn head on Mac OS X 10.4.9 which I'm not getting on Linux (Ubuntu feisty beta). test_sqlite and test_zipimport both fail, however, when run in verbose mode (e.g. ./python.exe Lib/test/test_sqlite.py) both pass. But that's not exactly why I'm writing this email . In the course of trying to debug this, I ran the following on my Mac: make TESTOPTS=-v test This runs the entire test suite in verbose mode, and you do get a lot of output. However the test suite hangs on test_pty.py. In fact, if you run that test alone: ./python.exe Lib/test/test_pty.py it too hangs for me. The reason is that in verbose mode, debug() actually prints stuff to stdout and on the Mac, when the child of the pty.fork() writes to its stdout, it blocks and so the parent's waitpid () never returns. This doesn't happen on Linux though; the child's stdout prints don't block, it exits, and the parent continues after the waitpid(). Here's a very simple program that reproduces the problem: - -----snip snip----- import os, pty, sys pid, fd = pty.fork() print >> sys.stderr, pid, fd if pid: os.waitpid(pid, 0) else: os._exit(0) - -----snip snip----- stderr, stdout, doesn't matter. This hangs on the Mac but completes successfully on Linux. Of course, in neither case do you see the child's output. I don't know if this is caused by a bug in the Mac's pty implementation or something we're doing wrong on that platform. I played around with several modifications to pty.fork() on the Mac, including letting it drop down to the openpty()/os.fork() code, even adding an explicit ioctl(slave_fd, TIOCSCTTY) call which Stevens chapter 19 recommends for 4.3+BSD. I can't get it to not block. Barring a fix to pty.fork() (or possibly os.forkpty()) for the Mac, then I would like to at least make test_pty.py not block when run in verbose mode. A very simple hack would add something like this to the "if pid == pty.CHILD" stanza: "def debug(msg): pass", possibly protected by a "if verbose:". A less icky hack would be to read the output from the master_fd in the parent, though you have to be careful with that on Linux else the read can throw an input/output error. Disabling debug output is band-aid yes, and any application on the Mac like the above snippet will still fail. If anybody has any suggestions, I'm all ears, but I've reached the limit of my pty-fu. mr-t-says-i-pty-the-fu-ly y'rs, - -Barry P.S. chime on on the test_sqlite and test_zipimport oddities if you want. :) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRh+Ug3EjvBPtnXfVAQJ4HgQAkm3XUq27GPBDKJj1NaIHuYijUerSE2oW 9YWiCYNQ+hBaYJJ16gdqZ2f7t/ENYVBzj3r9fNj+uyI1a6OFFMlE1tOdDFscQWUE e04XGwxo1fEPelJveIw8/dLHCPCpmCqvzLrT8fAtr7HL3thW3rk8s69i9i+CtJ1L LP7AY/SYXxg= =I+pF -----END PGP SIGNATURE----- From exarkun at divmod.com Fri Apr 13 16:57:31 2007 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Fri, 13 Apr 2007 10:57:31 -0400 Subject: [Python-Dev] test_pty.py hangs in verbose mode on Mac OS X? In-Reply-To: <905BBADA-70C7-4A80-978F-82646AF54CE8@python.org> Message-ID: <20070413145731.19381.1455034657.divmod.quotient.498@ohm> On Fri, 13 Apr 2007 10:32:28 -0400, Barry Warsaw wrote: >-----BEGIN PGP SIGNED MESSAGE----- >Hash: SHA1 > >I've been getting some test failures in Python 2.5 svn head on Mac OS >X 10.4.9 which I'm not getting on Linux (Ubuntu feisty beta). >test_sqlite and test_zipimport both fail, however, when run in >verbose mode (e.g. ./python.exe Lib/test/test_sqlite.py) both pass. > >But that's not exactly why I'm writing this email . In the >course of trying to debug this, I ran the following on my Mac: > >make TESTOPTS=-v test > >This runs the entire test suite in verbose mode, and you do get a lot >of output. However the test suite hangs on test_pty.py. In fact, if >you run that test alone: > >./python.exe Lib/test/test_pty.py > >it too hangs for me. The reason is that in verbose mode, debug() >actually prints stuff to stdout and on the Mac, when the child of the >pty.fork() writes to its stdout, it blocks and so the parent's waitpid >() never returns. This doesn't happen on Linux though; the child's >stdout prints don't block, it exits, and the parent continues after >the waitpid(). > >Here's a very simple program that reproduces the problem: > >- -----snip snip----- >import os, pty, sys > >pid, fd = pty.fork() >print >> sys.stderr, pid, fd >if pid: > os.waitpid(pid, 0) >else: > os._exit(0) >- -----snip snip----- > >stderr, stdout, doesn't matter. This hangs on the Mac but completes >successfully on Linux. Of course, in neither case do you see the >child's output. > >I don't know if this is caused by a bug in the Mac's pty >implementation or something we're doing wrong on that platform. I >played around with several modifications to pty.fork() on the Mac, >including letting it drop down to the openpty()/os.fork() code, even >adding an explicit ioctl(slave_fd, TIOCSCTTY) call which Stevens >chapter 19 recommends for 4.3+BSD. I can't get it to not block. What about reading from the child in the parent before calling waitpid? > >Barring a fix to pty.fork() (or possibly os.forkpty()) for the Mac, >then I would like to at least make test_pty.py not block when run in >verbose mode. A very simple hack would add something like this to >the "if pid == pty.CHILD" stanza: "def debug(msg): pass", possibly >protected by a "if verbose:". A less icky hack would be to read the >output from the master_fd in the parent, though you have to be >careful with that on Linux else the read can throw an input/output >error. > >Disabling debug output is band-aid yes, and any application on the >Mac like the above snippet will still fail. If anybody has any >suggestions, I'm all ears, but I've reached the limit of my pty-fu. > I don't think this is an OS X PTY bug. Writing to a blocking file descriptor can block. Programs that do this need to account for the possibility. Jean-Paul From barry at python.org Fri Apr 13 17:02:01 2007 From: barry at python.org (Barry Warsaw) Date: Fri, 13 Apr 2007 11:02:01 -0400 Subject: [Python-Dev] test_pty.py hangs in verbose mode on Mac OS X? In-Reply-To: <20070413145731.19381.1455034657.divmod.quotient.498@ohm> References: <20070413145731.19381.1455034657.divmod.quotient.498@ohm> Message-ID: <05B1B1C2-F8F3-47B1-A62E-D1294156A95B@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Apr 13, 2007, at 10:57 AM, Jean-Paul Calderone wrote: >> I don't know if this is caused by a bug in the Mac's pty >> implementation or something we're doing wrong on that platform. I >> played around with several modifications to pty.fork() on the Mac, >> including letting it drop down to the openpty()/os.fork() code, even >> adding an explicit ioctl(slave_fd, TIOCSCTTY) call which Stevens >> chapter 19 recommends for 4.3+BSD. I can't get it to not block. > > What about reading from the child in the parent before calling > waitpid? Yep, this is what I suggested below. Porting the same change over to Linux produced an OSError, but that's probably just because I wasn't as careful as I should have been late last night. >> Barring a fix to pty.fork() (or possibly os.forkpty()) for the Mac, >> then I would like to at least make test_pty.py not block when run in >> verbose mode. A very simple hack would add something like this to >> the "if pid == pty.CHILD" stanza: "def debug(msg): pass", possibly >> protected by a "if verbose:". A less icky hack would be to read the >> output from the master_fd in the parent, though you have to be >> careful with that on Linux else the read can throw an input/output >> error. >> >> Disabling debug output is band-aid yes, and any application on the >> Mac like the above snippet will still fail. If anybody has any >> suggestions, I'm all ears, but I've reached the limit of my pty-fu. >> > > I don't think this is an OS X PTY bug. Writing to a blocking file > descriptor can block. Programs that do this need to account for the > possibility. Why doesn't it block on Linux then? - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRh+banEjvBPtnXfVAQJxFQP+LRAdGVHuqquvGNT9fncGeJFuCjGm+dsS VnE8cirvfoSEdJ0nZ11wHMoMH2vtbXyfjLJJ4G/sROMmHKWJ2t5ieNIxmVutiiLa OPyls2bzuXL8IoyYh+c8tKRyBd76O5GN2EZABxGm2Tie5nt72ezVSEfiDovc6qEu 4lYERTD1YKA= =fZaZ -----END PGP SIGNATURE----- From exarkun at divmod.com Fri Apr 13 17:07:04 2007 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Fri, 13 Apr 2007 11:07:04 -0400 Subject: [Python-Dev] test_pty.py hangs in verbose mode on Mac OS X? In-Reply-To: <05B1B1C2-F8F3-47B1-A62E-D1294156A95B@python.org> Message-ID: <20070413150704.19381.1048766101.divmod.quotient.502@ohm> On Fri, 13 Apr 2007 11:02:01 -0400, Barry Warsaw wrote: >-----BEGIN PGP SIGNED MESSAGE----- >Hash: SHA1 > >On Apr 13, 2007, at 10:57 AM, Jean-Paul Calderone wrote: >>>I don't know if this is caused by a bug in the Mac's pty >>>implementation or something we're doing wrong on that platform. I >>>played around with several modifications to pty.fork() on the Mac, >>>including letting it drop down to the openpty()/os.fork() code, even >>>adding an explicit ioctl(slave_fd, TIOCSCTTY) call which Stevens >>>chapter 19 recommends for 4.3+BSD. I can't get it to not block. >> >>What about reading from the child in the parent before calling waitpid? > >Yep, this is what I suggested below. Porting the same change over to Linux >produced an OSError, but that's probably just because I wasn't as careful >as I should have been late last night. >>>Barring a fix to pty.fork() (or possibly os.forkpty()) for the Mac, >>>then I would like to at least make test_pty.py not block when run in >>>verbose mode. A very simple hack would add something like this to >>>the "if pid == pty.CHILD" stanza: "def debug(msg): pass", possibly >>>protected by a "if verbose:". A less icky hack would be to read the >>>output from the master_fd in the parent, though you have to be >>>careful with that on Linux else the read can throw an input/output >>>error. >>> >>>Disabling debug output is band-aid yes, and any application on the >>>Mac like the above snippet will still fail. If anybody has any >>>suggestions, I'm all ears, but I've reached the limit of my pty-fu. >> >>I don't think this is an OS X PTY bug. Writing to a blocking file >>descriptor can block. Programs that do this need to account for the >>possibility. > >Why doesn't it block on Linux then? > Likely differing buffering behavior. Prior to Linux 2.6, the pipe implementation allowed only a single buffer (that is, the bytes from a single write call) in a pipe at a time, and blocked subsequent writes until that buffer was read. Recently this has changed to allow multiple buffers up to 4k total length, so multiple short writes won't block anymore. OS X may have some other buffering behavior which is causing writes to block where they don't on Linux. All these details are left to the platform, and there are a variety of behaviors which can be considered valid. Of course, I don't actually /know/ the cause of the problem here, but this explanation seems plausible to me, and I'd investigate it before looking for platform-specific pty bugs (although OS X is a good platform on which to go looking for those ;). Jean-Paul From rrr at ronadam.com Fri Apr 13 17:15:40 2007 From: rrr at ronadam.com (Ron Adam) Date: Fri, 13 Apr 2007 10:15:40 -0500 Subject: [Python-Dev] Pydoc Rewrite Discussion at doc-sig list. Message-ID: <461F9E9C.2070901@ronadam.com> If anyone is interested in participating in discussing the details of the PyDoc rewrite/refactoring I've been working on, a discussion is being started on the doc-sig list. Doc-Sig at python.org The goal of this discussion will be to get it to a final finished form so a patch can be submitted and a final discussion can take place on the python-dev list at a later date. Thanks and Regards, Ron Adam From barry at python.org Fri Apr 13 17:22:34 2007 From: barry at python.org (Barry Warsaw) Date: Fri, 13 Apr 2007 11:22:34 -0400 Subject: [Python-Dev] test_pty.py hangs in verbose mode on Mac OS X? In-Reply-To: <20070413150704.19381.1048766101.divmod.quotient.502@ohm> References: <20070413150704.19381.1048766101.divmod.quotient.502@ohm> Message-ID: <8F66EA72-4412-4EA1-8F46-FD34BEB65811@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Apr 13, 2007, at 11:07 AM, Jean-Paul Calderone wrote: > Likely differing buffering behavior. Prior to Linux 2.6, the pipe > implementation allowed only a single buffer (that is, the bytes from > a single write call) in a pipe at a time, and blocked subsequent > writes until that buffer was read. Recently this has changed to allow > multiple buffers up to 4k total length, so multiple short writes won't > block anymore. OS X may have some other buffering behavior which is > causing writes to block where they don't on Linux. All these details > are left to the platform, and there are a variety of behaviors which > can be considered valid. > > Of course, I don't actually /know/ the cause of the problem here, but > this explanation seems plausible to me, and I'd investigate it before > looking for platform-specific pty bugs (although OS X is a good > platform > on which to go looking for those ;). Seems plausible to me too. If I change the child writes to > 4000 bytes, Linux w/2.6 kernel will block too. I'm going to change the test_pty.py code to read from the master_fd. You have to wrap the read in a try/except to catch the OSError input/ output error you'll get on Linux when you read past the end of the buffer and the child process has already exited. Thanks, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRh+gO3EjvBPtnXfVAQJkQAP9E2Jp/EeF3ChIeNZEPVMDeG4Kd+PVslSs blNFO2oO6eO8Yn9X3hlnLwCe3W6+89bS9u7MyN1C2CCZTBz6N1QAkfDaAnxuhbNC 3hfvytMguTY0v3RiZhEVY+y/h2PfbWe6fJGXeBIfcth1HWhP0KWIgAoDA+odDm1B vZSvcxBRN1Y= =fDrh -----END PGP SIGNATURE----- From oliphant.travis at ieee.org Fri Apr 13 23:19:26 2007 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Fri, 13 Apr 2007 15:19:26 -0600 Subject: [Python-Dev] Extended Buffer Protocol - simple use examples In-Reply-To: <79990c6b0704100228s7382e013p7f0685d1a7e3ed6@mail.gmail.com> References: <79990c6b0704090325n11721793r133e700f60c0e086@mail.gmail.com> <79990c6b0704100228s7382e013p7f0685d1a7e3ed6@mail.gmail.com> Message-ID: Paul Moore wrote: > On 09/04/07, Travis Oliphant wrote: > >>>I have skimmed (briefly, I'll admit!) the pre-PEP, but I've found it >>>extremely difficult to find a simple example of the basic (in my view) >>>use case of an undifferentiated block of bytes. >>> >> >>This is a great suggestion and it was on my to-do list. I've included >>some examples of this use-case in the new PEP. > > > Nice - those look clear to me. One question - if a producer is > generating a more complex data format (for example, the RGBA example > in the PEP) then would the "simple consumer" code (Ex. 3) still get a > pointer (or would the RGBA code need to go through extra effort to > allow this)? Sorry, again this is probably clear from reading the PEP > details, but my eyes glaze over when I read about strides, shapes, > etc... Unless, the exporter took some measures to create a contiguous copy of its buffer in situations like that, the exporter would have to raise an error if a consumer asked for a simple contiguous buffer. > > My motivation here is that it would be a shame if "old-style" code > that was prepared to guess the format of a memory block stopped > working when the producer of the memory added shape information (that > the consumer didn't care about, except to validate its guess). No, it wouldn't stop working, because right now, an error would get raised anyway. For example, in NumPy, you can get an "old-style" buffer from an ndarray only as long as it is contiguous. If it is discontiguous you will get an error. Example: >>> a = numpy.array([[1,2,3],[4,5,6]]) >>> b = a[::2,::2] >>> buffer(a)[:] >>> buffer(b)[:] # the last statement will raise an error in current Python. Part of the intent of the extended buffer protocol is to allow you to share even discontiguous memory with those that know how to handle it. In addition, the two C-API calls that allow copying data to and from a contiguous buffer is to create a standard way to work with "any" object for routines that only know how to deal with contiguous memory. -Travis From tim.peters at gmail.com Fri Apr 13 23:27:50 2007 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 13 Apr 2007 17:27:50 -0400 Subject: [Python-Dev] Changes to decimal.py In-Reply-To: <20070410142641.BEU75896@ms09.lnh.mail.rcn.net> References: <20070410142641.BEU75896@ms09.lnh.mail.rcn.net> Message-ID: <1f7befae0704131427h2c7e707dj122c2ca02d73789e@mail.gmail.com> [Raymond Hettinger] > ... > Likewise, consider soliciting Tim's input on how to implement the ln() > operation. That one will be tricky to get done efficiently and correctly. One low-effort approach is to use a general root-finding algorithm and build ln(x) on top of exp() via (numerically) solving the equation exp(ln(x)) == x for ln(x). That appears to be what Don Peterson did in his implementation of transcendental functions for Decimal: http://cheeseshop.python.org/pypi/decimalfuncs/1.4 In a bit of disguised form, that appears to be what Brian Beck and Christopher Hesse also did: http://cheeseshop.python.org/pypi/dmath/0.9 The former is GPL-licensed and the latter MIT, so the latter would be easier to start from for core (re)distribution. However, the IBM spec requires < 1 ULP worst-case error, and that may be unreasonably hard to meet with a root-finding approach. If this can wait a couple months, I'd be happy to own it. A possible saving grace for ln() is that while the mathematical function is one-to-one, in any fixed precision it's necessarily many-to-one (e.g., log10 of the representable numbers between 10 and 1e100 must be a representable number between 1 and 100, and there are a lot more of the former than of the latter -- many distinct representable numbers must map to the same representable log). From trentm at activestate.com Fri Apr 13 23:44:18 2007 From: trentm at activestate.com (Trent Mick) Date: Fri, 13 Apr 2007 14:44:18 -0700 Subject: [Python-Dev] ok to just checkin minor obvious fixes? Message-ID: <461FF9B2.6000502@activestate.com> Is it generally accepted to just checkin obviously non-controversial fixes (*) (as long as there is no code freeze), or is it still preferred that I go through adding a patch to the SF tracker and getting review? Trent (*) In my case a tweak to the old VC6 Windows build files to ensure a clean build works (_ctypes project must depend on pythoncore proj). -- Trent Mick trentm at activestate.com From martin at v.loewis.de Sat Apr 14 00:42:44 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 14 Apr 2007 00:42:44 +0200 Subject: [Python-Dev] ok to just checkin minor obvious fixes? In-Reply-To: <461FF9B2.6000502@activestate.com> References: <461FF9B2.6000502@activestate.com> Message-ID: <46200764.7000205@v.loewis.de> Trent Mick schrieb: > Is it generally accepted to just checkin obviously non-controversial > fixes (*) (as long as there is no code freeze), or is it still preferred > that I go through adding a patch to the SF tracker and getting review? > > Trent > > (*) In my case a tweak to the old VC6 Windows build files to ensure a > clean build works (_ctypes project must depend on pythoncore proj). Feel free to check in anything you consider appropriate, and use SF for cases where you want review (in particular, if you want the review recorded). Fixing the VC6 project files is certainly uncontroversial. Notice that the 2.5 branch is currently frozen (for 2.5.1); any commits to the branch need approval on python-dev. Regards, Martin From fredrik.johansson at gmail.com Sat Apr 14 02:48:13 2007 From: fredrik.johansson at gmail.com (Fredrik Johansson) Date: Sat, 14 Apr 2007 02:48:13 +0200 Subject: [Python-Dev] Changes to decimal.py In-Reply-To: <1f7befae0704131427h2c7e707dj122c2ca02d73789e@mail.gmail.com> References: <20070410142641.BEU75896@ms09.lnh.mail.rcn.net> <1f7befae0704131427h2c7e707dj122c2ca02d73789e@mail.gmail.com> Message-ID: <3d0cebfb0704131748g7664d7b0re72855f77ea7d842@mail.gmail.com> On 4/13/07, Tim Peters wrote: > One low-effort approach is to use a general root-finding algorithm and > build ln(x) on top of exp() via (numerically) solving the equation > exp(ln(x)) == x for ln(x). That appears to be what Don Peterson did > in his implementation of transcendental functions for Decimal: > > http://cheeseshop.python.org/pypi/decimalfuncs/1.4 > > In a bit of disguised form, that appears to be what Brian Beck and > Christopher Hesse also did: > > http://cheeseshop.python.org/pypi/dmath/0.9 > Whatever they do, they are very slow :-) I get the following timings with decimalfuncs (dmath is even slower): exp(Decimal(1)): 0.02 seconds at 28 digits, 0.2 seconds at 100 digits ln(Decimal(2)): 0.1 seconds at 28 digits, 0.6 seconds at 100 digits A while back I implemented exp and ln using fixed-point integer arithmetic, which unsurprisingly turned out to be much faster than working with Decimals. If I convert a decimal input to fixed-point, calculate exp or log, and convert the output back to decimal, I get the following timings: exp(1): 0.0008 seconds at 28 digits, 0.001 seconds at 100 digits ln(2): 0.001 seconds at 28 digits, 0.007 seconds at 100 digits [Note: I didn't use the exact numbers 1 and 2 for these timings; I added noise digits to make sure the decimal conversion code wasn't taking any shortcuts.] I use the Taylor series for exp(x) but first divide x by 2^32 to improve the convergence rate and finally square the sum 32 times. (32 could be replaced by any number; 32 just happens to be fast for a large range of precisions on my system.) Looking more closely at the 28-digit exp calculation: 0.0005 s is spent converting the Decimal to fixed-point 0.0001 s is spent summing the Taylor series (integer arithmetic) 0.0002 s is spent converting the fixed-point number back to Decimal For comparison, multiplying two 28-digit decimals takes 0.0003 seconds. So, computing exp this way is only about twice as expensive as multiplication at 28-digit precision; at 100 digit precision, I find that they are equally expensive. This approach assumes that the input has magnitude close to 1. For very large or very small numbers, you have to use the functional properties of exp to get a number close to 1, then convert back, all probably best (at least most easily) done entirely using Decimal arithmetic. For ln, I use Newton's method to invert exp, with an initial estimate from math.log, doubling the working precision at each iteration so that only one full-precision evaluation of exp is necessary. Using fixed-point internally during the Newton iteration, ln should be at most twice as slow as exp. Of course, this could all be irrelevant if the decimal module ever gets replaced by a C implementation... Fredrik From greg.ewing at canterbury.ac.nz Sat Apr 14 03:00:01 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 14 Apr 2007 13:00:01 +1200 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) In-Reply-To: <461F04C8.6060108@ieee.org> References: <461F04C8.6060108@ieee.org> Message-ID: <46202791.2030703@canterbury.ac.nz> Travis Oliphant wrote: > It is more convenient to store any slicing information (so a memory view > object could store an arbitrary slice of another object) as offsets, > lengths, and skips which can be used to adjust the memory buffer > returned by base. What happens if the base object changes its memory layout in such a way that the stored offsets, lengths and skips are no longer correct for the slice that was requested? -- Greg From kristjan at ccpgames.com Sat Apr 14 02:07:35 2007 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Sat, 14 Apr 2007 00:07:35 +0000 Subject: [Python-Dev] my 2.5 checkins Message-ID: <4E9372E6B2234D4F859320D896059A9508C00CD97B@exchis.ccp.ad.local> Hello all. I made two checkins to the 25 maintainance branch before Martin kindly pointed out to me that it is frozen. These are quite simple fixes to real crashes I have experienced. The fix in frameobject.c will be necessary if you work with opcodes > 128, which we routinely do at CCP J. Security through opcode randomization.... Anyway, just let me know if you like me to roll them back. Cheers, Kristj?n -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070414/a35b83b6/attachment.htm From greg.ewing at canterbury.ac.nz Sat Apr 14 03:41:11 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 14 Apr 2007 13:41:11 +1200 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) In-Reply-To: <461F2B28.6030508@ieee.org> References: <461AD36D.4080608@aerojockey.com> <461AE11E.8040901@ieee.org> <461D96BF.2080702@aerojockey.com> <461F2B28.6030508@ieee.org> Message-ID: <46203137.9000208@canterbury.ac.nz> Travis Oliphant wrote: > Py_BUF_SIMPLE --- you are requesting the simplest possible (0x00) > > Py_BUF_WRITEABLE -- get a writeable buffer (0x01) > > Py_BUF_READONLY -- get a read-only buffer (0x02) I don't see how these three form a progression. From the other things you say, it appears you mean Py_BUF_SIMPLE to mean both readable and writeable. But then Py_BUF_READONLY is turning off a capability (being able to write to the buffer) rather than turning one on. Seems to me the "simplest possible" buffer is a read-only one (assuming we don't want to allow for write-only buffers -- or do we?), in which case a more logical arrangement to my mind would be Py_BUF_READONLY = 0x00 # simplest possible PY_BUF_READWRITE = 0x01 If we do want write-only buffers, then there isn't a single simplest possible buffer, except for one that's neither readable nor writable, which doesn't seem very useful. So we would have Py_BUF_READABLE = 0x01 Py_BUF_WRITABLE = 0x02 Py_BUF_READWRITE = 0x03 -- Greg From oliphant.travis at ieee.org Sat Apr 14 08:33:05 2007 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Sat, 14 Apr 2007 00:33:05 -0600 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) In-Reply-To: <46202791.2030703@canterbury.ac.nz> References: <461F04C8.6060108@ieee.org> <46202791.2030703@canterbury.ac.nz> Message-ID: <462075A1.7090102@ieee.org> Greg Ewing wrote: > Travis Oliphant wrote: > >> It is more convenient to store any slicing information (so a memory >> view object could store an arbitrary slice of another object) as >> offsets, lengths, and skips which can be used to adjust the memory >> buffer returned by base. > > What happens if the base object changes its memory > layout in such a way that the stored offsets, lengths > and skips are no longer correct for the slice that > was requested? When the memory view object gets the buffer info again from the base object, it will be able to figure this out and raise an error. -Travis From nmm1 at cus.cam.ac.uk Sat Apr 14 09:25:28 2007 From: nmm1 at cus.cam.ac.uk (Nick Maclaren) Date: Sat, 14 Apr 2007 08:25:28 +0100 Subject: [Python-Dev] Changes to decimal.py Message-ID: "Tim Peters" wrote: > > One low-effort approach is to use a general root-finding algorithm and > build ln(x) on top of exp() via (numerically) solving the equation > exp(ln(x)) == x for ln(x). ... Not a general one, please! At least use one that assumes the continuity of second derivatives :-) > However, the IBM spec requires < 1 ULP worst-case error, and that may > be unreasonably hard to meet with a root-finding approach. ... It's a doddle in a software implementation for civilised functions like log. All you do is to do the calculation in a sufficiently extended precision - and you can work that out in advance for very little extra effort. Then you round to the result - and, hey! Bob's your uncle! 0.501 ULPs is as easy as 1, if a bit slower. Now, doing that for the power function is a right royal pain in the neck, and you really DON'T want to even contemplate a generic gamma function, let alone one of the beta functions :-( But none of this is specific to a decimal base, of course. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1 at cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679 From kristjan at ccpgames.com Sat Apr 14 11:14:08 2007 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Sat, 14 Apr 2007 09:14:08 +0000 Subject: [Python-Dev] __lltrace__ Message-ID: <4E9372E6B2234D4F859320D896059A9508C00CD982@exchis.ccp.ad.local> Hello again. So, using __lltrace__, I stumbled over a crashbug (to which I have already submitted a fix).i To trigger the crash, I modified lib/test/test_trace.py, lines 609 onwards: def test_16_no_jump_into_finally_block(self): globals()["__lltrace__"] = 1 self.run_test(no_jump_into_finally_block) del globals()["__lltrace__"] This would crash in all versions I tried. Now, I wonder, is it worth it adding a regression test for the __lltrace__ functionality? I realize it is a debug feature, only availible in debug builds, for debugging python itself, on the other hand, it had been broken for a long time and no one had noticed, hence perhaps the need for a test? What do you think? Kristj?n -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070414/9a05c088/attachment.htm From guido at python.org Sat Apr 14 16:58:01 2007 From: guido at python.org (Guido van Rossum) Date: Sat, 14 Apr 2007 07:58:01 -0700 Subject: [Python-Dev] __lltrace__ In-Reply-To: <4E9372E6B2234D4F859320D896059A9508C00CD982@exchis.ccp.ad.local> References: <4E9372E6B2234D4F859320D896059A9508C00CD982@exchis.ccp.ad.local> Message-ID: The __lltrace__ feature costs in maintenance; it should either be fixed or ripped out. On 4/14/07, Kristj?n Valur J?nsson wrote: > > > > > Hello again. > > So, using __lltrace__, I stumbled over a crashbug (to which I have already > submitted a fix).i > > To trigger the crash, I modified lib/test/test_trace.py, lines 609 onwards: > > def test_16_no_jump_into_finally_block(self): > > globals()["__lltrace__"] = 1 > > self.run_test(no_jump_into_finally_block) > > del globals()["__lltrace__"] > > > > This would crash in all versions I tried. > > Now, I wonder, is it worth it adding a regression test for the __lltrace__ > functionality? I realize it is a debug feature, only availible in > > debug builds, for debugging python itself, on the other hand, it had been > broken for a long time and no one had noticed, hence perhaps the need for a > test? > > > > What do you think? > > > > Kristj?n > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From selliott4 at austin.rr.com Sat Apr 14 18:23:21 2007 From: selliott4 at austin.rr.com (Steven Elliott) Date: Sat, 14 Apr 2007 11:23:21 -0500 Subject: [Python-Dev] Making builtins more efficient In-Reply-To: References: <1141879691.11091.78.camel@grey> <440FF9CB.5030407@gmail.com> <79990c6b0603090400h25dd2c7ev3d5c379f6529f3c2@mail.gmail.com> <1141915806.11091.127.camel@grey> <4410BE69.3080004@canterbury.ac.nz> <1171984065.22648.47.camel@grey> Message-ID: <1176567802.16345.16.camel@grey> ;`On Thu, 2007-02-22 at 01:26 +0100, Giovanni Bajo wrote: > On 20/02/2007 16.07, Steven Elliott wrote: > > > I'm finally getting back into this. I'd like to take one more shot at > > it with a revised version of what I proposed before. > > > > For those of you that did not see the original thread it was about ways > > that accessing builtins could be more efficient. It's a bit much to > > summarize again now, but you should be able to find it in the archive > > with this subject and a date of 2006-03-08. > > Are you aware of this patch, which is still awaiting review? > https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1616125&group_id=5470 I was not aware of your patch. I've since downloaded it, applied it, and played with it a bit. I find the cached module lookups (cached lookups when loading attributes in modules via LOAD_ATTR) to be particularly interesting since it addresses a case where PEP 280 leaves off. Your idea is to have an indexable array of objects that is only used when the hash table has not been changed, which can be determined by the timestamps you added. That may be the best way of handling attributes in modules (LOAD_ATTR). For global variables (LOAD_GLOBAL) I'm curious how it compares to PEP 280 and or Greg Ewing's idea. -- ----------------------------------------------------------------------- | Steven Elliott | selliott4 at austin.rr.com | ----------------------------------------------------------------------- From kristjan at ccpgames.com Sat Apr 14 18:29:50 2007 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Sat, 14 Apr 2007 16:29:50 +0000 Subject: [Python-Dev] fdopen mode needs sanitizing Message-ID: <4E9372E6B2234D4F859320D896059A9508C00CD98C@exchis.ccp.ad.local> In fileobject the mode string is sanitized using a function, sanitize_the_mode(). The same thing needs to happen for posixmodule.fdopen(). Otherwise, the universal_newlines regression test in test_subprocess.py will fail for python compiled with Visual Studio 2005. I have made a local patch by copying this static function over, and it fixes the issue, but I wonder if we would want to share this code? Also, are there other places that spring to mind where we have to sanitize mode strings before handing them to fopen-like functions? Kristj?n -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070414/ac246b81/attachment.html From spmion at gmail.com Sat Apr 14 22:08:29 2007 From: spmion at gmail.com (Sergio Mion) Date: Sat, 14 Apr 2007 15:08:29 -0500 Subject: [Python-Dev] new subscriber looking for grunt work Message-ID: Hello All, I'll be as brief as possible. The welcome message to this list suggested that I post a brief introduction of myself, so here it goes. I've been programming for about 10 years now (7 professionally). I would rank myself as a moderate programmer always looking to improve, and would like to help on this project in whatever way I can. I'll be trolling for grunt work and any documentation that needs to be done while I get the hang of things around here. Thanks for your patience. Sergio Mion -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070414/a4ead05b/attachment.html From status at bugs.python.org Sun Apr 15 02:00:44 2007 From: status at bugs.python.org (Tracker) Date: Sun, 15 Apr 2007 00:00:44 +0000 (UTC) Subject: [Python-Dev] Summary of Tracker Issues Message-ID: <20070415000044.E9D607818E@psf.upfronthosting.co.za> ACTIVITY SUMMARY (04/08/07 - 04/15/07) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1649 open ( +2) / 8584 closed ( +0) / 10233 total ( +2) Average duration of open issues: 764 days. Median duration of open issues: 715 days. Open Issues Breakdown open 1649 ( +2) pending 0 ( +0) Issues Created Or Reopened (2) ______________________________ Art 04/12/07 http://bugs.python.org/issue1028 created artleafoll session error 04/13/07 http://bugs.python.org/issue1029 created artleafoll -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070415/b6141c1e/attachment.html From adamadamadamamiadam at gmail.com Sun Apr 15 12:20:28 2007 From: adamadamadamamiadam at gmail.com (SevenInchBread) Date: Sun, 15 Apr 2007 06:20:28 -0400 Subject: [Python-Dev] Some new additions to functools Message-ID: <453f59300704150320ib1eb372p8541f3daf8be556a@mail.gmail.com> So I've cooked up some very simple functions to add to functools - to expand it into a more general-purpose module. def cat(x): return x class nullfunc(object): def __call__(self, *args, **kargs): return self def __getattr__(self, name): return getattr(None, name) def multimap(func, seq, n=2): assert n > 0, "n must be positive" if n == 1: return map(func, seq) else: return map(lambda x: multimap(func, x, n-1), seq) def multifilter(func, seq, n=2): return multimap(lambda x: filter(func, x), seq, n-1) def multireduce(func, seq, n=2): return multimap(lambda x: reduce(func, x), seq, n-1) In an expression, cat achieves the effect of doing nothing - which makes it a nice default value for some filter-like functions. It's not a huge improvement - lambda x: x is almost as trivial to define, but I think it would be nice to have a standard identity function that all of Python could recognize. nullfunc is a function that *chomps* away at its input while otherwise retaining the properties of None - much like the recently proposed callable None - except not as disasterous to existing practices and potentially more useful as an optional behavior. This is something that cannot be as quickly implemented as the cat function. multimap is a multi-dimensional mapping function that recursively decends a sequence and applies a function to the data within. multifilter is a multi-dimensional filter function that recursively descends a sequence and applies a function to the data within multireduce - well... you get the idea A better place to put some of these functions might be __builtin__, so you don't have to waste time importing something as basic as cat. -- "What's money? A man is a success if he gets up in the morning and goes to bed at night and in between does what he wants to do." ~ Bob Dylan -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070415/0bf125d1/attachment.htm From adamadamadamamiadam at gmail.com Sun Apr 15 12:38:43 2007 From: adamadamadamamiadam at gmail.com (SevenInchBread) Date: Sun, 15 Apr 2007 06:38:43 -0400 Subject: [Python-Dev] functools additions Message-ID: <453f59300704150338q76108940i2238ed58815eee2c@mail.gmail.com> So.... if it's alright with the privledged folk - I'd like to commit these minor (and probably non-controversial) additions to the functools module. def cat(x): return x def multimap(func, s, n=2): assert n > 0, "n must be positive" return (map(func, seq) if n == 1 else map(lambda x: multimap(func, x, n-1), seq)) def multifilter(func, s, n=2): return multimap(lambda x: filter(func, x), s, n-1) def multireduce(func, s, n=2): return multimap(lambda x: reduce(func, x), s, n-1) class nullfunc(object): def __call__(self, *a, **k): return self def __getattr(self, name): return getattr(None, name) cat is a generic identity function - useful for some higher-order functions to specify a function that "does nothing". multimap, multifilter, and multireduce, are all multi-dimensional versions of map, filter, and reduce. nullfunc is a special callable object that emulates the failed callable None proposal - which wasn't really a good idea, but would have been more useful as a different object apart from None. you could probably even put cat in __builtins__ - so you don't need to waste effort importing such a trivial function. --. "What's money? A man is a success if he gets up in the morning and goes to bed at night and in between does what he wants to do." ~ Bob Dylan -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070415/bc896418/attachment.html From martin at v.loewis.de Sun Apr 15 12:39:54 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 15 Apr 2007 12:39:54 +0200 Subject: [Python-Dev] new subscriber looking for grunt work In-Reply-To: References: Message-ID: <462200FA.7030807@v.loewis.de> > I'll be as brief as possible. The welcome message to this list suggested > that I post a brief introduction of myself, so here it goes. I've been > programming for about 10 years now (7 professionally). I would rank > myself as a moderate programmer always looking to improve, and > would like to help on this project in whatever way I can. I'll be trolling > for grunt work and any documentation that needs to be done while > I get the hang of things around here. Hi Sergio, an activity that is always worthwhile is bug and patch review. Pick a patch or a bug report that hasn't seen any feedback (there are, unfortunately, plenty of them), and try to analyse it. See http://mail.python.org/pipermail/python-dev/2002-November/029831.html for an older article of what these activites involve. Regards, Martin From oliphant at ee.byu.edu Wed Apr 4 01:00:12 2007 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 03 Apr 2007 17:00:12 -0600 Subject: [Python-Dev] Extended buffer PEP Message-ID: <4612DC7C.5090706@ee.byu.edu> Here is my "final" draft of the extended buffer interface PEP. For those who have been following the discussion, I eliminated the releaser object and the lock-buffer function. I decided that there is enough to explain with the new striding and sub-offsets without the added confusion of releasing buffers, especially when it is not clear what is to be gained by such complexity except a few saved lines of code. The striding and sub-offsets, however, allow extension module writers to write code (say video and image processing code or scientific computing code or data-base processing code) that works on any object exposing the buffer interface. I think this will be of great benefit and so is worth the complexity. This will take some work to get implemented for Python 3k. I could use some help with this in order to speed up the process. I'm working right now on the extensions to the struct module until the rest is approved. Thank you for any and all comments: -Travis -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: pep_buffer.txt Url: http://mail.python.org/pipermail/python-dev/attachments/20070403/7d11e443/attachment-0001.txt From oliphant at ee.byu.edu Mon Apr 9 18:59:10 2007 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Mon, 09 Apr 2007 10:59:10 -0600 Subject: [Python-Dev] Extended buffer PEP In-Reply-To: <46199015.9020307@canterbury.ac.nz> References: <4612DC7C.5090706@ee.byu.edu> <4617F064.6020605@aerojockey.com> <46192398.9000001@ieee.org> <46199015.9020307@canterbury.ac.nz> Message-ID: <461A70DE.1060609@ee.byu.edu> Greg Ewing wrote: > Travis Oliphant wrote: > >> Carl Banks wrote: >> > I'd like to see it accept a flags argument over what kind of buffer >> > it's allowed to return. I'd rather not burden the user to check >> all > the entries in bufferinfo to make sure it doesn't get something >> > unexpected. >> Yes, I agree. We had something like that at one point. > > > Maybe this could be handled in an intermediate layer > between the user and implementor of the interface, > i.e. the user calls > > PyBuffer_GetBuffer(obj, &info, flags); > > the object's tp_getbufferinfo just gets called as > > getbufferinfo(self, &info) > > and PyBuffer_GetBuffer then checks that the result > conforms to the requested feature set. This would > relieve users of the interface from having to check > that themselves, while not requiring implementors > to be burdened with it either. I like this strategy. Then, any intermediate buffering (that prompted the killed release-buffer object in the protocol) could be handled in this layer as well. I also like the idea of passing something to the getbuffer call so that exporters can do less work if some things are not being requested, but that the exporter should be free to ignore the flag and always produce everything. -Travis From jcarlson at uci.edu Sun Apr 15 20:10:29 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Sun, 15 Apr 2007 11:10:29 -0700 Subject: [Python-Dev] Some new additions to functools In-Reply-To: <453f59300704150320ib1eb372p8541f3daf8be556a@mail.gmail.com> References: <453f59300704150320ib1eb372p8541f3daf8be556a@mail.gmail.com> Message-ID: <20070415110143.6304.JCARLSON@uci.edu> SevenInchBread wrote: > So I've cooked up some very simple functions to add to functools - to expand > it into a more general-purpose module. [snip] I don't use a functional approach very often, but I haven't ever had a case where I would want or need to use any of the functions you provide. Does anyone else think that any or all of these would be useful? > A better place to put some of these functions might be __builtin__, so you > don't have to waste time importing something as basic as cat. We do not arbitrarily add functions to builtins. The general approach that most people have is to *reduce* the number of builtins, not increase them. Generally, -.5 on the functions/objects you provide (I haven't used them, but I'm reserving a -1 or -0 for if people have other opinions). -1000 on adding them to the builtins. - Josiah From martin at v.loewis.de Sun Apr 15 21:27:21 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 15 Apr 2007 21:27:21 +0200 Subject: [Python-Dev] functools additions In-Reply-To: <453f59300704150338q76108940i2238ed58815eee2c@mail.gmail.com> References: <453f59300704150338q76108940i2238ed58815eee2c@mail.gmail.com> Message-ID: <46227C99.9000508@v.loewis.de> > So.... if it's alright with the privledged folk - I'd like to commit > these minor (and probably non-controversial) additions to the functools > module. Do you have commit access? What's your real name? -1 on these additions. If lambda x:x would be added, it should be named "identity", not "cat" (if that name is meant to raise associations with the Unix cat(1) utility, please understand that "cat" is short for "concatenate"). Regards, Martin From paul.hankin at pobox.com Sun Apr 15 22:11:23 2007 From: paul.hankin at pobox.com (Paul Hankin) Date: Sun, 15 Apr 2007 21:11:23 +0100 Subject: [Python-Dev] Some new additions to functools In-Reply-To: <453f59300704150320ib1eb372p8541f3daf8be556a@mail.gmail.com> References: <453f59300704150320ib1eb372p8541f3daf8be556a@mail.gmail.com> Message-ID: On 4/15/07, SevenInchBread wrote: > So I've cooked up some very simple functions to add to functools - to expand > it into a more general-purpose module. > > def cat(x): return x > > class nullfunc(object): > def __call__(self, *args, **kargs): return self > def __getattr__(self, name): return getattr(None, name) > > def multimap(func, seq, n=2): > assert n > 0, "n must be positive" > if n == 1: return map(func, seq) > else: return map(lambda x: multimap(func, x, n-1), seq) > > def multifilter(func, seq, n=2): > return multimap(lambda x: filter(func, x), seq, n-1) > > def multireduce(func, seq, n=2): > return multimap(lambda x: reduce(func, x), seq, n-1) -0 for cat (renamed to 'identity'), -1 for the rest. I've occasionally wanted multimap but only in the default case where it is simpler to be explicit. -- Paul Hankin From adamadamadamamiadam at gmail.com Sun Apr 15 23:49:58 2007 From: adamadamadamamiadam at gmail.com (SevenInchBread) Date: Sun, 15 Apr 2007 17:49:58 -0400 Subject: [Python-Dev] Py3: function signatures, type checking, and related crap Message-ID: <453f59300704151449w3c766ceu34530e4035341840@mail.gmail.com> People seem to be pushing for a consistent method for checking the "x-ness" of objects (that is, interfaces that the object implements). So I present an idea for a simple and straightforward type that provides a way to construct descriptions of object structures, and I'd like some help expanding it into a useful extension to Python's standard set of utilities. Its a basic constructor that produces callable interface-checking predicates (which can be use in things such as list comprehensions, filter, if statements, or even a new syntax for function signatures that allows for automatic interface-checking). These predicates check that an object matches the behavior described by the constructor . Since I can't think of a name for this constructor, and because I've never liked the term "interface", I'll just call it "can". Can takes an arbitrary number of keyword arguments and produces a callable object. The keys represent object attributes, while the values are behavior-checking predicates like the ones produced by can. Since the can constructor produces an object that can in turn be used in other can constructors, using previously defined interfaces in new constructions is fairly straight-forward callable = can(__call__ = object)#Returns an object that describes objects with a __call__ attribute readable = can(read = callable) # ...with a callable read attribute writable = can(write = callable) # ...with a callable write attribute #a join operator can be used to combine can objects... #...for now I'll just use "and" and "or" to represent them. isfilelike = readable and writable #returns an object that matches any type that is described #by both readable and writable IOable = readable or writable #any type that is readable or writable objects that are constructed with can, when called, return True or False based on whether or not the passed object matches the behavior described. callable(hash) #returns True - as it would in the current version of Python. Here's some more nifty examples: iterable = can(__iter__=callable) or can(next=callable) hashable = can(__hash__=callable) completefile = isfilelike and iterable and can(fileno=callable, close=callable) def outlines(f, seq): """Outputs a sequence of lines to a file-like object""" assert isfilelike(f), "%r is not a file-like object." % f assert isiterable(seq), "%r must be iterable" % seq f.write("\n".join(seq)) #a trivial example... you'd get similar error messages from Python builtins even without the assertions. As it stands, I don't think that this deserves to be in Python - but I think the basic premise could be used as a foundation for better things. -- "What's money? A man is a success if he gets up in the morning and goes to bed at night and in between does what he wants to do." ~ Bob Dylan -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070415/fbece2de/attachment.htm From rwgk at yahoo.com Sun Apr 15 23:53:03 2007 From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve) Date: Sun, 15 Apr 2007 14:53:03 -0700 (PDT) Subject: [Python-Dev] Python 2.5.1c1 pickle problem Message-ID: <120024.88520.qm@web31108.mail.mud.yahoo.com> Hi Raymond, Thanks for the detailed explanation! > I'm not sure what your code was doing where the bugfix would cause > breakage. If its __getitem__() override returned a meaningful value > for each element in obj.keys(), then it should have worked fine. Of > course, if it was raising an exception or triggering a side-effect, > then one could argue that the bugfix was working as intended by > allowing the subclasser to affect how the base class goes about its > business. It wasn't actually my own code, but that of a colleague. His __getitem__() override is below. It fails because __getitem__() is called with the wrong type of argument (plain int instead of a list of atoms). I'm not sure it presents a convincing use case. I would have used composition instead of inheritance in the same situation. I don't feel competent to join the discussion about the changes in the pickling behavior, but I feel quite strongly that such a change should not be introduced in a bug-fix release. Ralf def __getitem__(self, atoms): LIST = [] for atom in atoms: attr = 'index' if hasattr(atom, 'unique_index'): attr = 'unique_index' LIST.append(getattr(atom, attr)) key1 = tuple(LIST) LIST.reverse() key2 = tuple(LIST) rc = None try: rc = dict.__getitem__(self, key1) except KeyError, e: pass try: rc = dict.__getitem__(self, key2) except KeyError, e: pass return rc __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070415/d4def50d/attachment.html From adamadamadamamiadam at gmail.com Mon Apr 16 00:18:16 2007 From: adamadamadamamiadam at gmail.com (SevenInchBread) Date: Sun, 15 Apr 2007 18:18:16 -0400 Subject: [Python-Dev] functools additions Message-ID: <453f59300704151518p77be74c5i1fb06d7ae21000ad@mail.gmail.com> >> Do you have commit access? What's your real name? I prefer to remain pseudonymous, and I don't have commit access. Yeah... they're not terribly useful - more or less there for the sake of being there. Batteries included and all that ...but now I've got a more useful idea for a function wrapper around predicate functions to make them a little more functionally inclined. Something like....... @predicate def hasnext(x): return hasattr(x, "next") @predicate def hasiter(x): return hasattr(x, "__iter__") isiterable = hasnext or hasiter #or/and/not operators construct new predicate functions isEmptyIterable = isiterable and not predicate(bool) isgenerator = isiterable and (lambda x: hasattr(x, "send") and hasattr(x, "throw")) filter(isgenerator or callable, SomeSequence) -- "What's money? A man is a success if he gets up in the morning and goes to bed at night and in between does what he wants to do." ~ Bob Dylan -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070415/2f20f155/attachment.htm From bjourne at gmail.com Mon Apr 16 00:36:42 2007 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Mon, 16 Apr 2007 00:36:42 +0200 Subject: [Python-Dev] functools additions In-Reply-To: <453f59300704150338q76108940i2238ed58815eee2c@mail.gmail.com> References: <453f59300704150338q76108940i2238ed58815eee2c@mail.gmail.com> Message-ID: <740c3aec0704151536u74f46e92t6e34dbd123013ef5@mail.gmail.com> > def cat(x): return x > > def multimap(func, s, n=2): > assert n > 0, "n must be positive" > return (map(func, seq) > if n == 1 else > map(lambda x: multimap(func, x, n-1), > seq)) > > def multifilter(func, s, n=2): > return multimap(lambda x: filter(func, x), s, n-1) > > def multireduce(func, s, n=2): > return multimap(lambda x: reduce(func, x), s, n-1) > > class nullfunc(object): > def __call__(self, *a, **k): return self > def __getattr(self, name): return getattr(None, name) Could you describe what these functions do? Preferably with examples that demonstrates that they are useful. -- mvh Bj?rn From exarkun at divmod.com Mon Apr 16 01:09:12 2007 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Sun, 15 Apr 2007 19:09:12 -0400 Subject: [Python-Dev] functools additions In-Reply-To: <453f59300704151518p77be74c5i1fb06d7ae21000ad@mail.gmail.com> Message-ID: <20070415230912.19381.1236772864.divmod.quotient.1797@ohm> On Sun, 15 Apr 2007 18:18:16 -0400, SevenInchBread wrote: >>>Do you have commit access? What's your real name? >I prefer to remain pseudonymous, and I don't have commit access. > >Yeah... they're not terribly useful - more or less there for the sake of >being there. Batteries included and all that > Please discuss this on the python-ideas list before bringing it up on python-dev. Jean-Paul From guido at python.org Mon Apr 16 02:07:19 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 15 Apr 2007 17:07:19 -0700 Subject: [Python-Dev] Some new additions to functools In-Reply-To: <453f59300704150320ib1eb372p8541f3daf8be556a@mail.gmail.com> References: <453f59300704150320ib1eb372p8541f3daf8be556a@mail.gmail.com> Message-ID: On 4/15/07, SevenInchBread wrote: [...] > -- > "What's money? A man is a success if he gets up in the morning and goes to > bed at night and in between does what he wants to do." ~ Bob Dylan If you ask me, SevenInchBread (not her real name) is a Troll. Let her prove she isn't. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From anthony at interlink.com.au Mon Apr 16 05:42:31 2007 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon, 16 Apr 2007 13:42:31 +1000 Subject: [Python-Dev] my 2.5 checkins In-Reply-To: <4E9372E6B2234D4F859320D896059A9508C00CD97B@exchis.ccp.ad.local> References: <4E9372E6B2234D4F859320D896059A9508C00CD97B@exchis.ccp.ad.local> Message-ID: <200704161342.32708.anthony@interlink.com.au> On Saturday 14 April 2007 10:07, Kristj?n Valur J?nsson wrote: > Hello all. > I made two checkins to the 25 maintainance branch before Martin > kindly pointed out to me that it is frozen. These are quite > simple fixes to real crashes I have experienced. The fix in > frameobject.c will be necessary if you work with opcodes > 128, > which we routinely do at CCP J. Security through opcode > randomization.... Anyway, just let me know if you like me to roll > them back. I really, really, really don't want to cut another release candidate. These fixes don't strike me as critical enough to need that - and I'm not happy to do the release and just hope. I'll roll them all back. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From nnorwitz at gmail.com Mon Apr 16 08:37:01 2007 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 15 Apr 2007 23:37:01 -0700 Subject: [Python-Dev] my 2.5 checkins In-Reply-To: <200704161342.32708.anthony@interlink.com.au> References: <4E9372E6B2234D4F859320D896059A9508C00CD97B@exchis.ccp.ad.local> <200704161342.32708.anthony@interlink.com.au> Message-ID: I reverted both changes. They can be re-applied after the 2.5 is unfrozen. -- n On 4/15/07, Anthony Baxter wrote: > On Saturday 14 April 2007 10:07, Kristj?n Valur J?nsson wrote: > > Hello all. > > I made two checkins to the 25 maintainance branch before Martin > > kindly pointed out to me that it is frozen. These are quite > > simple fixes to real crashes I have experienced. The fix in > > frameobject.c will be necessary if you work with opcodes > 128, > > which we routinely do at CCP J. Security through opcode > > randomization.... Anyway, just let me know if you like me to roll > > them back. > > I really, really, really don't want to cut another release > candidate. These fixes don't strike me as critical enough to need > that - and I'm not happy to do the release and just hope. I'll roll > them all back. > > Anthony > -- > Anthony Baxter > It's never too late to have a happy childhood. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com > From nnorwitz at gmail.com Mon Apr 16 09:03:05 2007 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 16 Apr 2007 00:03:05 -0700 Subject: [Python-Dev] Python 2.5.1c1 pickle problem In-Reply-To: <005201c77cd2$4c64cf80$f001a8c0@RaymondLaptop1> References: <20070411221919.BFB04685@ms09.lnh.mail.rcn.net> <005201c77cd2$4c64cf80$f001a8c0@RaymondLaptop1> Message-ID: On 4/12/07, Raymond Hettinger wrote: > Ralf, your issue is arising because of revision 53655 which fixes SF 1615701. I have backed out this change for 2.5.1. > Am leaving this open for others to discuss and decide. The old behavior was > surprising to some, but the revised behavior also appears to have some > unforeseen consequences. Anthony and I talked about this a bit. The current behavior seems less than optimal. However, changing it in a bug fix release seems particularly risky. > P.S. In addition to rev 53655, a number of similar changes were made to sets. Raymond, can you revert the changes? I looked in Objects/setobject.c and saw 53667. Is that the revision you are referring to? Are there any other revs? Thanks, n From nnorwitz at gmail.com Mon Apr 16 09:13:09 2007 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 16 Apr 2007 00:13:09 -0700 Subject: [Python-Dev] SystemErrors in generator (just happened in 2.5) Message-ID: There were some SystemErrors on one of the Windows build slaves. Does anyone have any ideas what might be the cause? I looked through about 5 previous logs on the same slave and didn't see the same problem. I haven't seen this myself and I don't know if it's WIndows specific. I don't know if this has happened on the trunk or just 2.5. For the full log see: http://python.org/dev/buildbot/2.5/x86%20XP%202.5/builds/197/step-test/0 n -- *** Previous context (I don't think this really matters): Re-running test 'test_profilehooks' in verbose mode test_caught_exception (test.test_profilehooks.ProfileHookTestCase) ... ok test_caught_nested_exception (test.test_profilehooks.ProfileHookTestCase) ... ok test_distant_exception (test.test_profilehooks.ProfileHookTestCase) ... ok test_exception (test.test_profilehooks.ProfileHookTestCase) ... ok test_exception_in_except_clause (test.test_profilehooks.ProfileHookTestCase) ... ok test_exception_propogation (test.test_profilehooks.ProfileHookTestCase) ... ok test_generator (test.test_profilehooks.ProfileHookTestCase) ... ok test_nested_exception (test.test_profilehooks.ProfileHookTestCase) ... ok test_raise (test.test_profilehooks.ProfileHookTestCase) ... ok test_raise_reraise (test.test_profilehooks.ProfileHookTestCase) ... ok test_raise_twice (test.test_profilehooks.ProfileHookTestCase) ... ok test_simple (test.test_profilehooks.ProfileHookTestCase) ... ok test_stop_iteration (test.test_profilehooks.ProfileHookTestCase) ... ok test_basic_exception (test.test_profilehooks.ProfileSimulatorTestCase) ... ok test_caught_exception (test.test_profilehooks.ProfileSimulatorTestCase) ... ok test_distant_exception (test.test_profilehooks.ProfileSimulatorTestCase) ... ok test_simple (test.test_profilehooks.ProfileSimulatorTestCase) ... ok ---------------------------------------------------------------------- Ran 17 tests in 0.016s OK *** Here is the problem: Exception exceptions.SystemError: 'error return without exception set' in ignored Exception exceptions.SystemError: 'error return without exception set' in ignored Exception exceptions.SystemError: 'error return without exception set' in ignored Exception exceptions.SystemError: 'error return without exception set' in ignored Exception exceptions.SystemError: 'error return without exception set' in ignored *** The following messages occur in other successful tests too: a DOS box should flash briefly ... find_library('c') -> None find_library('m') -> None C:\buildbot_py25\2.5.mcintyre-windows\build\lib\test\test_unicode_file.py:103: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal filename1==filename2 C:\buildbot_py25\2.5.mcintyre-windows\build\lib\shutil.py:36: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal os.path.normcase(os.path.abspath(dst))) warning: DBTxn aborted in destructor. No prior commit() or abort(). From barry at python.org Mon Apr 16 13:54:00 2007 From: barry at python.org (Barry Warsaw) Date: Mon, 16 Apr 2007 07:54:00 -0400 Subject: [Python-Dev] [Python-checkins] r54805 - python/branches/release25-maint/Lib/test/test_pty.py In-Reply-To: <461FEF11.4020100@v.loewis.de> References: <20070413124313.BFJ16741@ms09.lnh.mail.rcn.net> <461FEF11.4020100@v.loewis.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Apr 13, 2007, at 4:58 PM, Martin v. L?wis wrote: >> I don't know, maybe it is. This one's extremely low risk as it only >> affects certain platforms when test_pty is run verbosely. But if it >> ain't cool, I'll back it out and re-apply after 2.5.1 final. > > Please back it out. If there are changes to the branch, we would have > to create another release candidate (IMO). I see Neal did already... thanks Neal. I was away for most of the weekend. Cheers, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRiNj3XEjvBPtnXfVAQLgfQP+KVKtGkEzZpo8Bjhlo6w7zII5oQdo9pDy L18n2OjcJyAsz+Hgo4hkel6W+O39o/lR0EtjohCUlfG92DTgyoUxGXc2g88LyT83 G4Gob2UBoqiWnsHpYjWFP6/HlAcr6nUXa6CGJqkyhzud6+c9ZsdfOIhDpZfyxh2x ZzH/D5f8D7M= =aI0k -----END PGP SIGNATURE----- From grig.gheorghiu at gmail.com Mon Apr 16 17:59:42 2007 From: grig.gheorghiu at gmail.com (Grig Gheorghiu) Date: Mon, 16 Apr 2007 08:59:42 -0700 Subject: [Python-Dev] test_normalization failures across community buildbots Message-ID: <3f09d5a00704160859l49f35f66g707d3da252f97fd7@mail.gmail.com> Almost all community buildbots have failed the test step due to a failure in test_normalization. Here's a link to the community farm for the trunk: http://www.python.org/dev/buildbot/community/trunk/ And here's an example of a failure: http://www.python.org/dev/buildbot/community/trunk/x86%20OSX%20trunk/builds/567/step-test/0 test test_normalization failed -- Traceback (most recent call last): File "/Users/builder/pybots/pybot/trunk.osaf-x86/build/Lib/test/test_normalization.py", line 36, in test_main for line in open_urlresource(TESTDATAURL): File "/Users/builder/pybots/pybot/trunk.osaf-x86/build/Lib/test/test_support.py", line 271, in open_urlresource requires('urlfetch') File "/Users/builder/pybots/pybot/trunk.osaf-x86/build/Lib/test/test_support.py", line 94, in requires raise ResourceDenied(msg) ResourceDenied: Use of the `urlfetch' resource not enabled Grig -- http://agiletesting.blogspot.com From jimjjewett at gmail.com Mon Apr 16 19:14:59 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 16 Apr 2007 13:14:59 -0400 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) Message-ID: Reading this message without the entire PEP in front of me showed some confusing usage. (Details below) Most (but not all) I could resolve from the PEP itself, but they could be clarified with different constant names. Counter Proposal at bottom, and specific questions in between. Travis Oliphant wrote: > Carl Banks wrote: >> Some of the flags RESTRICT the kind of buffers that can be >> exported (Py_BUF_WRITABLE); other flags EXPAND the >> kind of buffers that can be exported (Py_BUF_INDIRECT). >> That is highly confusing and I'm -1 >> on any proposal that includes both behaviors. > Basically, every flag corresponds to a different property of > the buffer that the consumer is requesting: I had trouble seeing it like that. Could you at least rename them something like SHAPE_VALID? But I would prefer the change suggested at the bottom. > Py_BUF_SIMPLE --- you are requesting the simplest possible (0x00) ??? Does this automatically assume a readable buffer? (No hardware write buffers?) (This one I couldn't tell from the PEP) > Py_BUF_WRITEABLE -- get a writeable buffer (0x01) > Py_BUF_READONLY -- get a read-only buffer (0x02) Do Py_BUF_READONLY promise that *this* consumer won't try to change it, or request that it be immutable (and no one else will change it either)? (From the PEP itself, I think you wanted IMMUTABLE.) When it comes back, does it mean "you, the consumer, promised not to change this", or "I, the exporter, won't authorize anyone to change it, including myself." If it is really a request for immutability, and the exporter can't make that promise, then should the buffer protocol itself make and return an immutable copy? For this grouping to make sense, I have to assume that you really mean Py_BUF_READABLE (0 - no way to mark it write-only) Py_BUF_WRITABLE (1 - meaning *this* consumer can write) Py_BUF_IMMUTABLE (2 - meaning no one can change it.) so 3 => compile-time error but that still doesn't mesh with your later statement that: > ... most people request read-or-write buffers > i.e. Py_BUF_SIMPLE. > Py_BUF_FORMAT -- get a "formatted" buffer. (0x04) Is this saying "I the consumer will respect your formatting" or "I the consumer will fail if you don't tell me the formatting"? If this flag comes back, does that mean that understanding the formatting is mandatory, or is it just informational? ??? To make this concrete, there are libraries that sniff and guess a format. Should they pass this flag or not? > Py_BUF_SHAPE -- get a buffer with shape information (0x08) Wait -- what is the difference between these two again? Is "format" the internal format of a single element, and shape the dimensions of an array? Should Py_BUF_FORMAT be Py_BUF_ELT_FORMAT? Do you pass Py_BUF_SHAPE to indicate that you'll accept N-Dim arrays, or to say that you prefer them or somehow need them? ??? If Py_BUF_SHAPE was requested, but the buffer really is 1-dimensional, should this still be set on the way back? (Presumably setting the "shape" variable to point to a (one-element) array containing len/itemsize? > Py_BUF_STRIDES -- get a buffer with stride information (and shape) (0x18) > Py_BUF_OFFSET -- get a buffer with suboffsets (and strides and shape) (0x38) I assume that Py_BUF_OFFSET doesn't make sense without Py_BUF_STRIDES, and Py_BUF_STRIDES doesn't make sense without Py_BUF_SHAPE. What happens if someone *does* pass 0x20? If you want to avoid that, it might make sense to treat this as 4 enumerated values (1-D, n-D, n-D with strides, n-D with strides and offsets) instead of 3 flags. If Py_BUF_OFFSET was requested, but the return value is a single continuous 1D array, should Py_BUF_OFFSET still come back but have the zeros filled in? Also, why are all negative offsets invalid? It seems like they might be useful for some re-orderings, and using an offset of 0 has the same effect as marking the offset unused. > For me, the most restrictive requests are > PY_BUF_WRITEABLE | Py_BUF_FORMAT and > Py_BUF_READONLY | Py_BUF_FORMAT > The most un-restrictive request (the largest circle in my mental Venn > diagram) is > Py_BUF_OFFSETS followed by Py_BUF_STRIDES followed by Py_BUF_SHAPE I think this is what he meant when he said that the flags had opposite meanings. Ideally, 0 and 0xFF should be shortcuts to either the most restrictive or the least restrictive. > Think of them as turning "on" members of the bufferinfo structure. If the senses can't be reversed, could you at least rename them to indicate this? Something like Py_BUF_VALID_SUBOFFSETS? ================================ Counter Proposal: 0x00: Py_BUF_RWLOCK: "Consumer is a new owner. It can read and write; no other code can." (most restrictive) 0x01: Py_BUF_READONLY: "Consumer doesn't need to (cannot) write." 0x02: Py_BUF_MUTABLE: "Other code can write" (implies) 0x03 = PY_BUF_UR: Py_BUF_READONLY | Py_BUF_MUTABLE : "Unconfirmed Read: Consumer won't write, but other code might." (skipping 0x04 just to get the shapes all in the second digit) 0x08: Py_BUF_ELT_FORMAT: "If you send the element format, I'll look at it." ??? What does this mean in a response? Just that the *format isn't NULL? # Note that these are four shapes alternatives, # rather than two orthogonal flags 0x0n: Py_BUF_1D: "No shape information." (in response, means none required, -- should the shape pointer still be set?) 0x1n: Py_BUF_SHAPE: "continous n-D arrays acceptable (to consumer) or used (by exporter)" 0x2n: Py_BUF_STRIDE: " ? How to best describe strides? Srides acceptable (to consumer) or used (by exporter)" 0x3n: Py_BUF_OFFSET: " ? How to best describe offsets? offests acceptable (to consumer) or used (by exporter)" -jJ From brett at python.org Mon Apr 16 21:36:43 2007 From: brett at python.org (Brett Cannon) Date: Mon, 16 Apr 2007 12:36:43 -0700 Subject: [Python-Dev] test_normalization failures across community buildbots In-Reply-To: <3f09d5a00704160859l49f35f66g707d3da252f97fd7@mail.gmail.com> References: <3f09d5a00704160859l49f35f66g707d3da252f97fd7@mail.gmail.com> Message-ID: On 4/16/07, Grig Gheorghiu wrote: > > Almost all community buildbots have failed the test step due to a > failure in test_normalization. Here's a link to the community farm for > the trunk: > > http://www.python.org/dev/buildbot/community/trunk/ > > And here's an example of a failure: > > > http://www.python.org/dev/buildbot/community/trunk/x86%20OSX%20trunk/builds/567/step-test/0 > > test test_normalization failed -- Traceback (most recent call last): > File > "/Users/builder/pybots/pybot/trunk.osaf-x86/build/Lib/test/test_normalization.py", > line 36, in test_main > for line in open_urlresource(TESTDATAURL): > File > "/Users/builder/pybots/pybot/trunk.osaf-x86/build/Lib/test/test_support.py", > line 271, in open_urlresource > requires('urlfetch') > File > "/Users/builder/pybots/pybot/trunk.osaf-x86/build/Lib/test/test_support.py", > line 94, in requires > raise ResourceDenied(msg) > ResourceDenied: Use of the `urlfetch' resource not enabled Don't know what suddenly triggered this (nothing I did), but the code basically looks correct. What should be happening is regrtest should be catching that exception and just saying the test was skipped. The last commit on regrtest was March 12, test_support was touched April 3, and test_normalization on April 5 according to ``svn log -r COMMITTED``. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070416/9e7cefcd/attachment.htm From tim.peters at gmail.com Mon Apr 16 21:57:31 2007 From: tim.peters at gmail.com (Tim Peters) Date: Mon, 16 Apr 2007 15:57:31 -0400 Subject: [Python-Dev] SystemErrors in generator (just happened in 2.5) In-Reply-To: References: Message-ID: <1f7befae0704161257q6d14088frfbda1c5978742fd6@mail.gmail.com> [Neal Norwitz] > There were some SystemErrors on one of the Windows build slaves. Not always, though -- looks to be rare. > Does anyone have any ideas what might be the cause? I looked through about > 5 previous logs on the same slave and didn't see the same problem. I'm home today and fired up my slave, and no problem there either. > I haven't seen this myself and I don't know if it's WIndows specific. I > don't know if this has happened on the trunk or just 2.5. Me neither. > For the full log see: > http://python.org/dev/buildbot/2.5/x86%20XP%202.5/builds/197/step-test/0 > > n > -- > > *** Previous context (I don't think this really matters): > > Re-running test 'test_profilehooks' in verbose mode > test_caught_exception (test.test_profilehooks.ProfileHookTestCase) ... ok > test_caught_nested_exception (test.test_profilehooks.ProfileHookTestCase) ... ok > test_distant_exception (test.test_profilehooks.ProfileHookTestCase) ... ok > test_exception (test.test_profilehooks.ProfileHookTestCase) ... ok > test_exception_in_except_clause > (test.test_profilehooks.ProfileHookTestCase) ... ok > test_exception_propogation (test.test_profilehooks.ProfileHookTestCase) ... ok > test_generator (test.test_profilehooks.ProfileHookTestCase) ... ok > test_nested_exception (test.test_profilehooks.ProfileHookTestCase) ... ok > test_raise (test.test_profilehooks.ProfileHookTestCase) ... ok > test_raise_reraise (test.test_profilehooks.ProfileHookTestCase) ... ok > test_raise_twice (test.test_profilehooks.ProfileHookTestCase) ... ok > test_simple (test.test_profilehooks.ProfileHookTestCase) ... ok > test_stop_iteration (test.test_profilehooks.ProfileHookTestCase) ... ok > test_basic_exception (test.test_profilehooks.ProfileSimulatorTestCase) ... ok > test_caught_exception (test.test_profilehooks.ProfileSimulatorTestCase) ... ok > test_distant_exception (test.test_profilehooks.ProfileSimulatorTestCase) ... ok > test_simple (test.test_profilehooks.ProfileSimulatorTestCase) ... ok > > ---------------------------------------------------------------------- > Ran 17 tests in 0.016s > > OK > > *** Here is the problem: > > Exception exceptions.SystemError: 'error return without exception set' in ignored > Exception exceptions.SystemError: 'error return without exception set' in ignored > Exception exceptions.SystemError: 'error return without exception set' in ignored > Exception exceptions.SystemError: 'error return without exception set' in ignored > Exception exceptions.SystemError: 'error return without exception set' in ignored > *** The following messages occur in other successful tests too: > a DOS box should flash briefly ... Always happens in test_subprocess, during the Windows-specific test_creationflags. This is expected. When you /watch/ the tests running on Windows, it's intended to prevent panic when a mysterious DOS box appears ;-) > find_library('c') -> None > find_library('m') -> None Mysterious. Looks like debug/trace(!) output while running Lib/ctypes/test/test_loading.py's test_find(). > C:\buildbot_py25\2.5.mcintyre-windows\build\lib\test\test_unicode_file.py:103: > UnicodeWarning: Unicode equal comparison failed to convert both > arguments to Unicode - interpreting them as being unequal > filename1==filename2 > C:\buildbot_py25\2.5.mcintyre-windows\build\lib\shutil.py:36: > UnicodeWarning: Unicode equal comparison failed to convert both > arguments to Unicode - interpreting them as being unequal > os.path.normcase(os.path.abspath(dst))) Those started showing up months ago. > warning: DBTxn aborted in destructor. No prior commit() or abort(). Likewise, from the bsddb test, and we've been seeing this one on Unixish boxes too. From grig.gheorghiu at gmail.com Mon Apr 16 22:00:06 2007 From: grig.gheorghiu at gmail.com (Grig Gheorghiu) Date: Mon, 16 Apr 2007 13:00:06 -0700 Subject: [Python-Dev] test_normalization failures across community buildbots In-Reply-To: References: <3f09d5a00704160859l49f35f66g707d3da252f97fd7@mail.gmail.com> Message-ID: <3f09d5a00704161300i67a015fdocfdc928b81b10ddd@mail.gmail.com> On 4/16/07, Brett Cannon wrote: > Don't know what suddenly triggered this (nothing I did), but the code > basically looks correct. What should be happening is regrtest should be > catching that exception and just saying the test was skipped. > > The last commit on regrtest was March 12, test_support was touched April 3, > and test_normalization on April 5 according to ``svn log -r COMMITTED``. > There's a check-in related to test_normalization.py on April 5th that seems to have triggered the failures: http://www.python.org/dev/buildbot/community/trunk/changes/1292 Sorry I haven't reported this sooner, I've been swamped the last couple weeks. Grig From brett at python.org Mon Apr 16 23:10:53 2007 From: brett at python.org (Brett Cannon) Date: Mon, 16 Apr 2007 14:10:53 -0700 Subject: [Python-Dev] test_normalization failures across community buildbots In-Reply-To: <3f09d5a00704161300i67a015fdocfdc928b81b10ddd@mail.gmail.com> References: <3f09d5a00704160859l49f35f66g707d3da252f97fd7@mail.gmail.com> <3f09d5a00704161300i67a015fdocfdc928b81b10ddd@mail.gmail.com> Message-ID: On 4/16/07, Grig Gheorghiu wrote: > > On 4/16/07, Brett Cannon wrote: > > Don't know what suddenly triggered this (nothing I did), but the code > > basically looks correct. What should be happening is regrtest should be > > catching that exception and just saying the test was skipped. > > > > The last commit on regrtest was March 12, test_support was touched April > 3, > > and test_normalization on April 5 according to ``svn log -r COMMITTED``. > > > > There's a check-in related to test_normalization.py on April 5th that > seems to have triggered the failures: > > http://www.python.org/dev/buildbot/community/trunk/changes/1292 > > Sorry I haven't reported this sooner, I've been swamped the last couple > weeks. That's Collin's checkin when he updated the test to use unittest. Any ideas, Collin? -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070416/0ee4b59f/attachment.htm From collinw at gmail.com Tue Apr 17 00:14:42 2007 From: collinw at gmail.com (Collin Winter) Date: Mon, 16 Apr 2007 15:14:42 -0700 Subject: [Python-Dev] test_normalization failures across community buildbots In-Reply-To: References: <3f09d5a00704160859l49f35f66g707d3da252f97fd7@mail.gmail.com> <3f09d5a00704161300i67a015fdocfdc928b81b10ddd@mail.gmail.com> Message-ID: <43aa6ff70704161514i2b5cbd32k9109441f2b13dcc4@mail.gmail.com> On 4/16/07, Brett Cannon wrote: > On 4/16/07, Grig Gheorghiu wrote: > > On 4/16/07, Brett Cannon wrote: > > > Don't know what suddenly triggered this (nothing I did), but the code > > > basically looks correct. What should be happening is regrtest should be > > > catching that exception and just saying the test was skipped. > > > > > > The last commit on regrtest was March 12, test_support was touched April > 3, > > > and test_normalization on April 5 according to ``svn log -r COMMITTED``. > > > > > > > There's a check-in related to test_normalization.py on April 5th that > > seems to have triggered the failures: > > > > > http://www.python.org/dev/buildbot/community/trunk/changes/1292 > > > > Sorry I haven't reported this sooner, I've been swamped the last couple > weeks. > > > That's Collin's checkin when he updated the test to use unittest. Any > ideas, Collin? This should be fixed in r54844. The problem was that the availability of the urlfetch resource wasn't being checked early enough and so test_support.run_suite() was converting the ResourceDenied exception into a TestError instance. This wasn't showing up on other machines since the urlfetch'd files weren't being cleaned out between test runs. Collin Winter From exarkun at divmod.com Tue Apr 17 00:53:20 2007 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Mon, 16 Apr 2007 18:53:20 -0400 Subject: [Python-Dev] new metaclass error condition checks In-Reply-To: 0 Message-ID: <20070416225320.19381.297920264.divmod.quotient.2282@ohm> I just noticed r53997 (from some unit tests it broke), which disallowed things like this: class X(object): def __repr__(self): return "blah" class Y(X, type): pass class Z: __metaclass__ = Y Making X classic eliminates the TypeError, and is probably an acceptable fix in a lot of cases (at least as long as classic classes are available). I wonder if the ability to override type special methods like this was considered when the change was made, though? Jean-Paul From oliphant.travis at ieee.org Tue Apr 17 01:47:47 2007 From: oliphant.travis at ieee.org (Travis E. Oliphant) Date: Mon, 16 Apr 2007 17:47:47 -0600 Subject: [Python-Dev] Access to Python SVN Message-ID: <46240B23.5050702@ieee.org> I'd like to ask for access to Python SVN so that I can keep the PEP 3118 up to date as well as to eventually make the changes needed for implementing the extended buffer protocol. I have quite a bit of experience with the Python C-API and understand many parts of the code base fairly well (though I would not claim to be an expert on all of it). I promise to only adjust the PEP until such time as patches to implement the extended buffer protocol are approved. I will email my public SSH key to the appropriate place. Thank you very much, -Travis Oliphant From guido at python.org Tue Apr 17 01:49:54 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 16 Apr 2007 16:49:54 -0700 Subject: [Python-Dev] Access to Python SVN In-Reply-To: <46240B23.5050702@ieee.org> References: <46240B23.5050702@ieee.org> Message-ID: +1 On 4/16/07, Travis E. Oliphant wrote: > I'd like to ask for access to Python SVN so that I can keep the PEP 3118 > up to date as well as to eventually make the changes needed for > implementing the extended buffer protocol. > > I have quite a bit of experience with the Python C-API and understand > many parts of the code base fairly well (though I would not claim to be > an expert on all of it). > > I promise to only adjust the PEP until such time as patches to implement > the extended buffer protocol are approved. > > I will email my public SSH key to the appropriate place. > > Thank you very much, > > -Travis Oliphant > > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Tue Apr 17 05:21:33 2007 From: python at rcn.com (Raymond Hettinger) Date: Mon, 16 Apr 2007 20:21:33 -0700 Subject: [Python-Dev] test_normalization failures across communitybuildbots References: <3f09d5a00704160859l49f35f66g707d3da252f97fd7@mail.gmail.com><3f09d5a00704161300i67a015fdocfdc928b81b10ddd@mail.gmail.com> <43aa6ff70704161514i2b5cbd32k9109441f2b13dcc4@mail.gmail.com> Message-ID: <00b601c7809f$f85e3f30$f301a8c0@RaymondLaptop1> [Collin Winter] > This should be fixed in r54844. The problem was that the availability > of the urlfetch resource wasn't being checked early enough and so > test_support.run_suite() was converting the ResourceDenied exception > into a TestError instance. This wasn't showing up on other machines > since the urlfetch'd files weren't being cleaned out between test > runs. Can you add code to clean them out between runs? Raymond From martin at v.loewis.de Tue Apr 17 05:25:44 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 17 Apr 2007 05:25:44 +0200 Subject: [Python-Dev] Access to Python SVN In-Reply-To: <46240B23.5050702@ieee.org> References: <46240B23.5050702@ieee.org> Message-ID: <46243E38.8040303@v.loewis.de> Travis E. Oliphant schrieb: > I'd like to ask for access to Python SVN so that I can keep the PEP 3118 > up to date as well as to eventually make the changes needed for > implementing the extended buffer protocol. > > I will email my public SSH key to the appropriate place. Please send it to me. Martin From pythondev at aerojockey.com Tue Apr 17 08:36:39 2007 From: pythondev at aerojockey.com (Carl Banks) Date: Tue, 17 Apr 2007 02:36:39 -0400 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) In-Reply-To: <461F2B28.6030508@ieee.org> References: <461AD36D.4080608@aerojockey.com> <461AE11E.8040901@ieee.org> <461D96BF.2080702@aerojockey.com> <461F2B28.6030508@ieee.org> Message-ID: <46246AF7.5000304@aerojockey.com> Travis Oliphant wrote: > Carl Banks wrote: >> My recommendation is, any flag should turn on some circle in the Venn >> diagram (it could be a circle I didn't draw--shaped arrays, for >> example--but it should be *some* circle). > I don't think your Venn diagram is broad enough and it un-necessarily > limits the use of flags to communicate between consumer and exporter. > We don't have to ram these flags down that point-of-view for them to be > productive. If you have a specific alternative proposal, or specific > criticisms, then I'm very willing to hear them. Ok, I've thought quite a bit about this, and I have an idea that I think will be ok with you, and I'll be able to drop my main objection. It's not a big change, either. The key is to explicitly say whether the flag allows or requires. But I made a few other changes as well. First of all, let me define how I'm using the word "contiguous": it's a single buffer with no gaps. So, if you were to do this: "memset(bufinfo->buf,0,bufinfo->len)", you would not touch any data that isn't being exported. Without further ado, here is my proposal: ------ With no flags, the PyObject_GetBuffer will raise an exception if the buffer is not direct, contiguous, and one-dimensional. Here are the flags and how they affect that: Py_BUF_REQUIRE_WRITABLE - Raise exception if the buffer isn't writable. Py_BUF_REQUIRE_READONLY - Raise excpetion if the buffer is writable. Py_BUF_ALLOW_NONCONTIGUOUS - Allow noncontiguous buffers. (This turns on "shape" and "strides".) Py_BUF_ALLOW_MULTIDIMENSIONAL - Allow multidimensional buffers. (Also turns on "shape" and "strides".) (Neither of the above two flags implies the other.) Py_BUF_ALLOW_INDIRECT - Allow indirect buffers. Implies Py_BUF_ALLOW_NONCONTIGUOUS and Py_BUF_ALLOW_MULTIDIMENSIONAL. (Turns on "shape", "strides", and "suboffsets".) Py_BUF_REQUIRE_CONTIGUOUS_C_ARRAY or Py_BUF_REQUIRE_ROW_MAJOR - Raise an exception if the array isn't a contiguous array with in C (row-major) format. Py_BUF_REQUIRE_CONTIGUOUS_FORTRAN_ARRAY or Py_BUF_REQUIRE_COLUMN_MAJOR - Raise an exception if the array isn't a contiguous array with in Fortran (column-major) format. Py_BUF_ALLOW_NONCONTIGUOUS, Py_BUF_REQUIRE_CONTIGUOUS_C_ARRAY, and Py_BUF_REQUIRE_CONTIGUOUS_FORTRAN_ARRAY all conflict with each other, and an exception should be raised if more than one are set. (I would go with ROW_MAJOR and COLUMN_MAJOR: even though the terms only make sense for 2D arrays, I believe the terms are commonly generalized to other dimensions.) Possible pseudo-flags: Py_BUF_SIMPLE = 0; Py_BUF_ALLOW_STRIDED = Py_BUF_ALLOW_NONCONTIGUOUS | Py_BUF_ALLOW_MULTIDIMENSIONAL; ------ Now, for each flag, there should be an associated function to test the condition, given a bufferinfo struct. (Though I suppose they don't necessarily have to map one-to-one, I'll do that here.) int PyBufferInfo_IsReadonly(struct bufferinfo*); int PyBufferInfo_IsWritable(struct bufferinfo*); int PyBufferInfo_IsContiguous(struct bufferinfo*); int PyBufferInfo_IsMultidimensional(struct bufferinfo*); int PyBufferInfo_IsIndirect(struct bufferinfo*); int PyBufferInfo_IsRowMajor(struct bufferinfo*); int PyBufferInfo_IsColumnMajor(struct bufferinfo*); The function PyObject_GetBuffer then has a pretty obvious implementation. Here is an except: if ((flags & Py_BUF_REQUIRE_READONLY) && !PyBufferInfo_IsReadonly(&bufinfo)) { PyExc_SetString(PyErr_BufferError,"buffer not read-only"); return 0; } Pretty straightforward, no? Now, here is a key point: for these functions to work (indeed, for PyObject_GetBuffer to work at all), you need enough information in bufinfo to figure it out. The bufferinfo struct should be self-contained; you should not need to know what flags were passed to PyObject_GetBuffer in order to know exactly what data you're looking at. Therefore, format must always be supplied by getbuffer. You cannot tell if an array is contiguous without the format string. (But see below.) And even if the consumer isn't asking for a contiguous buffer, it has to know the item size so it knows what data not to step on. (This is true even in your own proposal, BTW. If a consumer asks for a non-strided array in your proposal, PyObject_GetBuffer would have to know the item size to determine if the array is contiguous.) ------ FAQ: Q. Why ALLOW_NONCONTIGUOUS and ALLOW_MULTIDIMENSIONAL instead of ALLOW_STRIDED and ALLOW_SHAPED? A. It's more useful to the consumer that way. With ALLOW_STRIDED and ALLOW_SHAPED, there's no way for a consumer to request a general one-dimensional array (it can only request a non-strided one-dimensional array), and requesting a SHAPED array but not a STRIDED one can only return a C-like (row-major) array, although a consumer might reasonably want a Fortran-like (column-major) array. This approach maps more directly to the consumer's needs, is more flexible, and still maintains the same functionality of ALLOW_SHAPED and ALLOW_STRIDED. Q. Why call it ALLOW_INDIRECT instead of ALLOW_OFFSETS? A. It's just a name, and not too important to me, but I wanted to emphasize the consumer's usage, rather than the benefit to the exporter. The consumers, after all, are the ones setting the flags. Q. Why ALLOW_NONCONTIGUOUS instead of REQUIRE_CONTIGUOUS? Two reasons: 1. Contiguous arrays are "simpler", so it's better to make the people who want more complex arrays to work harder, and 2. ALLOW_NONCONTIGUOUS is closely tied to ALLOW_MULTIDIMENSIONAL. If the negative is a problem, perhaps a name like ALLOW_DISCONTINUOUS or ALLOW_GAPS would be better? Q. What about Py_BUF_FORMAT? A. Ok, fine, if it's that imporant to you. I think it's totally superfluous, but it's not evil. But consider these things: 1. Require that it does not throw an exception. It's not the exporter's business to tell the consumer to how to use its data. 2. Even if you don't supply the format string, you need to supply an itemsize in struct bufferinfo, otherwise there is no way for a consumer to determine if the array is contiguous, and or to know (in general) what data is being exported. The itemsize must ALWAYS be available. 3. Invert Py_BUF_FORMAT. Use Py_BUF_DONT_NEED_FORMAT instead. Make the consumer that cares about performance ask for the optimization. (You admit yourself that Py_BUF_FORMAT is part of the least common denominator, so invert it.) I would be -0 on it if all three of these conditions are met. ------ Conclusion: My main objection, that the flags are confusing because some allow and others restrict, would be remedied just by using ALLOW and REQUIRE in the constant. Even if you still want to go with ALLOW_STRIDED and ALLOW_SHAPE, I'd still be -0 as long as the ALLOW is there. I still think Py_BUF_FORMAT is superfluous, but I can live with it if some other things happen. Carl Banks From greg.ewing at canterbury.ac.nz Tue Apr 17 08:49:49 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 17 Apr 2007 18:49:49 +1200 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) In-Reply-To: <46246AF7.5000304@aerojockey.com> References: <461AD36D.4080608@aerojockey.com> <461AE11E.8040901@ieee.org> <461D96BF.2080702@aerojockey.com> <461F2B28.6030508@ieee.org> <46246AF7.5000304@aerojockey.com> Message-ID: <46246E0D.7060506@canterbury.ac.nz> Carl Banks wrote: > Py_BUF_REQUIRE_READONLY - Raise excpetion if the buffer is writable. Is there a use case for this? -- Greg From facundo at taniquetil.com.ar Tue Apr 17 18:28:51 2007 From: facundo at taniquetil.com.ar (Facundo Batista) Date: Tue, 17 Apr 2007 16:28:51 +0000 (UTC) Subject: [Python-Dev] Changes to decimal.py References: <20070410142641.BEU75896@ms09.lnh.mail.rcn.net> <1f7befae0704131427h2c7e707dj122c2ca02d73789e@mail.gmail.com> Message-ID: Tim Peters wrote: > can wait a couple months, I'd be happy to own it. A possible saving > grace for ln() is that while the mathematical function is one-to-one, I'm working right now in making the old operation to pass the new tests ok. Soon I'll cut a branch to work publicly on this (good idea from Raymond), and I'll be pleased to get your help here. Two months is ok: the "updated decimal" won't be finished before that. Thank you! -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From facundo at taniquetil.com.ar Tue Apr 17 18:45:59 2007 From: facundo at taniquetil.com.ar (Facundo Batista) Date: Tue, 17 Apr 2007 16:45:59 +0000 (UTC) Subject: [Python-Dev] new subscriber looking for grunt work References: <462200FA.7030807@v.loewis.de> Message-ID: Martin v. L?wis wrote: > an activity that is always worthwhile is bug and patch review. Pick a > patch or a bug report that hasn't seen any feedback (there are, > unfortunately, plenty of them), and try to analyse it. Sergio, welcome. As Martin said, bugs and patch revision is a fruitful activity, for Python, and for you. You can use these pages to have a grand vision of which bugs and patchs we have still open and start choosing your first: http://www.taniquetil.com.ar/facundo/py_bugs_00.html http://www.taniquetil.com.ar/facundo/py_patchs_00.html Regards, -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From collinw at gmail.com Tue Apr 17 19:43:22 2007 From: collinw at gmail.com (Collin Winter) Date: Tue, 17 Apr 2007 10:43:22 -0700 Subject: [Python-Dev] test_normalization failures across communitybuildbots In-Reply-To: <00b601c7809f$f85e3f30$f301a8c0@RaymondLaptop1> References: <3f09d5a00704160859l49f35f66g707d3da252f97fd7@mail.gmail.com> <3f09d5a00704161300i67a015fdocfdc928b81b10ddd@mail.gmail.com> <43aa6ff70704161514i2b5cbd32k9109441f2b13dcc4@mail.gmail.com> <00b601c7809f$f85e3f30$f301a8c0@RaymondLaptop1> Message-ID: <43aa6ff70704171043iba353we3f0c0d211df74f@mail.gmail.com> On 4/16/07, Raymond Hettinger wrote: > [Collin Winter] > > This should be fixed in r54844. The problem was that the availability > > of the urlfetch resource wasn't being checked early enough and so > > test_support.run_suite() was converting the ResourceDenied exception > > into a TestError instance. This wasn't showing up on other machines > > since the urlfetch'd files weren't being cleaned out between test > > runs. > > Can you add code to clean them out between runs? I'll see what I can do. Collin From alexander.belopolsky at gmail.com Wed Apr 18 03:10:43 2007 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 17 Apr 2007 21:10:43 -0400 Subject: [Python-Dev] distutils mixes 32/64bit binaries Message-ID: On the platforms that can run both 32 and 64bit python, such as x86_64, distutils builds both 32 and 64 bit libraries in the same directory such as build/lib.linux-x86_64-2.5. This can be easily fixed by placing 64 bit libraries in build/lib64.linux-x86_64-2.5 instead. On the other hand it may be time to revisit use of os.uname in distutils' get_platform to determine the target directory. In the environments that can run python interpreters compiled for different architectures, distutils' get_platform should return the platform for which the interpreter was built, not the platform on which it runs. Similar problem exists with the default install directory $prefix/lib/python2.5/lib-dynload. 64-bit libraries should probably go to $prefix/lib/python2.5/lib64-dynload instead. I've noticed that there was a similar proposal to use $prefix/lib64 for 64bit binary components, but it was quickly rejected: http://mail.python.org/pipermail/python-dev/2006-November/070044.html In my view, following LSB spirit would suggest splitting pythonX.Y tree into platform dependent part (*.so) that would go under $prefix/lib or $prefix/lib64 and a platform independent part (*.py{,c,o}) that would go under $prefix/share. This, of course, would be a much bigger change than the lib64-dynload band aid that I am proposing here. PS: As a data point, the R project has made the following changes to their program to deal with this issue: """ o LDFLAGS now defaults to -L/usr/local/lib64 on most Linux 64-bit OSes (but not ia64). The use of lib/lib64 can be overridden by the new variable LIBnn. o The default installation directory is now ${prefix}/${LIBnn}/R, /usr/local/lib64/R on most 64-bit Linux OSes and /usr/local/lib/R elsewhere. """ From jason.orendorff at gmail.com Wed Apr 18 04:37:24 2007 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Tue, 17 Apr 2007 22:37:24 -0400 Subject: [Python-Dev] minidom -> new-style classes? Message-ID: I'm working on minidom's DOM Level 1 compliance, targeting Python 2.6. We have some bugs involving DOM property behavior. For example, setting the nodeValue attribute of an Element is supposed to have no effect. We don't implement this. The right way to implement these quirks is using new-style classes and properties. Right now minidom uses old-style classes and lots of hackery, and it's pretty broken. (Another example--there is an Attr._set_prefix method, but it is *not* called from __setattr__.) Surely nobody is subclassing these classes. You don't subclass DOM interfaces--the DOM doesn't work that way. So this change should be OK. Right? -j From collinw at gmail.com Wed Apr 18 04:51:34 2007 From: collinw at gmail.com (Collin Winter) Date: Tue, 17 Apr 2007 19:51:34 -0700 Subject: [Python-Dev] minidom -> new-style classes? In-Reply-To: References: Message-ID: <43aa6ff70704171951p2d609afetba6bd5cf34c826bd@mail.gmail.com> On 4/17/07, Jason Orendorff wrote: > Surely nobody is subclassing these classes. You don't subclass DOM > interfaces--the DOM doesn't work that way. So this change should be > OK. Right? People are definitely subclassing those classes: http://www.google.com/codesearch?hl=en&lr=&q=class%5Cs%2B.%2B%5C%28.*minidom%5C.&btnG=Search Whether any of those uses will break if switched to new-style class, I can't say. Collin Winter From fdrake at acm.org Wed Apr 18 04:56:52 2007 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Tue, 17 Apr 2007 22:56:52 -0400 Subject: [Python-Dev] minidom -> new-style classes? In-Reply-To: References: Message-ID: <200704172256.52949.fdrake@acm.org> On Tuesday 17 April 2007 22:37, Jason Orendorff wrote: > The right way to implement these quirks is using new-style classes and > properties. Right now minidom uses old-style classes and lots of > hackery, and it's pretty broken. (Another example--there is an > Attr._set_prefix method, but it is *not* called from __setattr__.) Yes, it's truly vile. Better than it used to be, but.... There's also some vague attempt at supporting the Python CORBA IDL mapping, since the W3C DOM specifications use that. I expect the support is incomplete at best. > Surely nobody is subclassing these classes. You don't subclass DOM > interfaces--the DOM doesn't work that way. So this change should be > OK. Right? There are people who've tried building new DOM implementations by subclassing the minidom classes to get most of the behavior. I'm don't know the status of any of these implementations, but changes to these classes have proved difficult due to this and the possibility of breaking pickles (amazed me, that one did!). I'd love to see a sane implementation, using new-style classes and properties, but as long as we've got to support existing applications, we're pretty well hosed as far as xml.dom.minidom is concerned. A new DOM implementation conforming to the W3C recommendations would be nice, but I'd really rather see an XML object model that doesn't suck, but that supports as much of the information found in the W3C DOM as possible. Something based more on the ElementTree API, perhaps. The value of the W3C-approved API has certainly turned out to be more decoy than anything. -Fred -- Fred L. Drake, Jr. From guido at python.org Wed Apr 18 05:40:00 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 17 Apr 2007 20:40:00 -0700 Subject: [Python-Dev] minidom -> new-style classes? In-Reply-To: <200704172256.52949.fdrake@acm.org> References: <200704172256.52949.fdrake@acm.org> Message-ID: Perhaps a rewrite could target 3.0 and 2.6 could use a backported version of this *if* py3k compatibility mode is enabled? I'd love to see at least the 3.0 version cleaned up. --Guido On 4/17/07, Fred L. Drake, Jr. wrote: > On Tuesday 17 April 2007 22:37, Jason Orendorff wrote: > > The right way to implement these quirks is using new-style classes and > > properties. Right now minidom uses old-style classes and lots of > > hackery, and it's pretty broken. (Another example--there is an > > Attr._set_prefix method, but it is *not* called from __setattr__.) > > Yes, it's truly vile. Better than it used to be, but.... > > There's also some vague attempt at supporting the Python CORBA IDL mapping, > since the W3C DOM specifications use that. I expect the support is > incomplete at best. > > > Surely nobody is subclassing these classes. You don't subclass DOM > > interfaces--the DOM doesn't work that way. So this change should be > > OK. Right? > > There are people who've tried building new DOM implementations by subclassing > the minidom classes to get most of the behavior. I'm don't know the status > of any of these implementations, but changes to these classes have proved > difficult due to this and the possibility of breaking pickles (amazed me, > that one did!). > > I'd love to see a sane implementation, using new-style classes and properties, > but as long as we've got to support existing applications, we're pretty well > hosed as far as xml.dom.minidom is concerned. > > A new DOM implementation conforming to the W3C recommendations would be nice, > but I'd really rather see an XML object model that doesn't suck, but that > supports as much of the information found in the W3C DOM as possible. > Something based more on the ElementTree API, perhaps. The value of the > W3C-approved API has certainly turned out to be more decoy than anything. > > > -Fred > > -- > Fred L. Drake, Jr. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jason.orendorff at gmail.com Wed Apr 18 14:26:23 2007 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Wed, 18 Apr 2007 08:26:23 -0400 Subject: [Python-Dev] minidom -> new-style classes? In-Reply-To: References: <200704172256.52949.fdrake@acm.org> Message-ID: On 4/17/07, Guido van Rossum wrote: > Perhaps a rewrite could target 3.0 and 2.6 could use a backported > version of this *if* py3k compatibility mode is enabled? I'd love to > see at least the 3.0 version cleaned up. A lot of these bugs can be fixed without forking. I've been conservative so far. I looked at a diff this morning. Even pickled documents shouldn't break. If this patch lands, I can look at further cleanup after that. Thanks everyone, -j From guido at python.org Wed Apr 18 16:18:42 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 18 Apr 2007 07:18:42 -0700 Subject: [Python-Dev] minidom -> new-style classes? In-Reply-To: References: <200704172256.52949.fdrake@acm.org> Message-ID: Great -- if you target 2.6, it'll automatically be merged into 3.0 the next time somebody runs svnmerge. (Thomas?) --Guido On 4/18/07, Jason Orendorff wrote: > On 4/17/07, Guido van Rossum wrote: > > Perhaps a rewrite could target 3.0 and 2.6 could use a backported > > version of this *if* py3k compatibility mode is enabled? I'd love to > > see at least the 3.0 version cleaned up. > > A lot of these bugs can be fixed without forking. I've been > conservative so far. I looked at a diff this morning. Even > pickled documents shouldn't break. > > If this patch lands, I can look at further cleanup after that. > > Thanks everyone, > -j > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From g.brandl at gmx.net Wed Apr 18 18:34:36 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 18 Apr 2007 18:34:36 +0200 Subject: [Python-Dev] Proposal: Allow any mapping after ** in calls In-Reply-To: References: Message-ID: Alexander Belopolsky schrieb: > Python allows arbitrary sequences after * in calls, but an expression > following ** must be a (subclass of) dict. I believe * and ** should > be treated similarly and since f(*UserList(..)) is valid, > f(**UserDict(..)) should be valid as well. > > Of course, I can work around this limitation by writing > f(**dict(UserDict(..))), but this is unnatural given that I don't have > to do f(*tuple(UserList(..))). > > I have submitted a patch on SF that makes Python accept arbitrary > mappings after **. Ping! Nobody interested in this? Georg From guido at python.org Wed Apr 18 19:07:20 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 18 Apr 2007 10:07:20 -0700 Subject: [Python-Dev] Proposal: Allow any mapping after ** in calls In-Reply-To: References: Message-ID: I'm +1 on the idea, but have no time to review the change. On 4/18/07, Georg Brandl wrote: > Alexander Belopolsky schrieb: > > Python allows arbitrary sequences after * in calls, but an expression > > following ** must be a (subclass of) dict. I believe * and ** should > > be treated similarly and since f(*UserList(..)) is valid, > > f(**UserDict(..)) should be valid as well. > > > > Of course, I can work around this limitation by writing > > f(**dict(UserDict(..))), but this is unnatural given that I don't have > > to do f(*tuple(UserList(..))). > > > > I have submitted a patch on SF that makes Python accept arbitrary > > mappings after **. > > Ping! Nobody interested in this? > > Georg > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From theller at ctypes.org Wed Apr 18 21:25:36 2007 From: theller at ctypes.org (Thomas Heller) Date: Wed, 18 Apr 2007 21:25:36 +0200 Subject: [Python-Dev] SystemErrors in generator (just happened in 2.5) In-Reply-To: <1f7befae0704161257q6d14088frfbda1c5978742fd6@mail.gmail.com> References: <1f7befae0704161257q6d14088frfbda1c5978742fd6@mail.gmail.com> Message-ID: >> *** The following messages occur in other successful tests too: >> a DOS box should flash briefly ... > > Always happens in test_subprocess, during the Windows-specific > test_creationflags. This is expected. When you /watch/ the tests > running on Windows, it's intended to prevent panic when a mysterious > DOS box appears ;-) > >> find_library('c') -> None >> find_library('m') -> None > > Mysterious. Looks like debug/trace(!) output while running > Lib/ctypes/test/test_loading.py's test_find(). I enabled these prints to have a way to examine what find_library() returns on the buildbots, without checking for a certain result in a test. Should these prints be removed again? Thomas From oliphant.travis at ieee.org Thu Apr 19 06:03:40 2007 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 18 Apr 2007 22:03:40 -0600 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) In-Reply-To: <46246E0D.7060506@canterbury.ac.nz> References: <461AD36D.4080608@aerojockey.com> <461AE11E.8040901@ieee.org> <461D96BF.2080702@aerojockey.com> <461F2B28.6030508@ieee.org> <46246AF7.5000304@aerojockey.com> <46246E0D.7060506@canterbury.ac.nz> Message-ID: <4626EA1C.9040407@ieee.org> Greg Ewing wrote: > Carl Banks wrote: > >> Py_BUF_REQUIRE_READONLY - Raise excpetion if the buffer is writable. > > Is there a use case for this? Yes. The idea is used in NumPy all the time. Suppose you want to write to an array but only have an algorithm that works with contiguous data. Then you need to make a copy of the data into a contiguous buffer but you would like to make the original memory read-only until you are done with the algorithm and have copied the data back into memory. That way when you release the GIL, the memory area will now be read-only and so other instances won't write to it (because any writes will be eradicated by the copy back when the algorithm is done). NumPy uses this idea all the time in its UPDATE_IF_COPY flag. -Travis From oliphant.travis at ieee.org Thu Apr 19 06:12:47 2007 From: oliphant.travis at ieee.org (Travis E. Oliphant) Date: Wed, 18 Apr 2007 22:12:47 -0600 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) In-Reply-To: References: Message-ID: <4626EC3F.4080105@ieee.org> Jim Jewett wrote: > Reading this message without the entire PEP in front of me showed some > confusing usage. (Details below) Most (but not all) I could resolve > from the PEP itself, but they could be clarified with different > constant names. > I'm going to adapt some suggestions made by you and Carl Banks. Look for an updated flags section of the PEP shortly. -Travis From oliphant.travis at ieee.org Thu Apr 19 06:12:47 2007 From: oliphant.travis at ieee.org (Travis E. Oliphant) Date: Wed, 18 Apr 2007 22:12:47 -0600 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) In-Reply-To: References: Message-ID: <4626EC3F.4080105@ieee.org> Jim Jewett wrote: > Reading this message without the entire PEP in front of me showed some > confusing usage. (Details below) Most (but not all) I could resolve > from the PEP itself, but they could be clarified with different > constant names. > I'm going to adapt some suggestions made by you and Carl Banks. Look for an updated flags section of the PEP shortly. -Travis From kbk at shore.net Thu Apr 19 06:23:05 2007 From: kbk at shore.net (Kurt B. Kaiser) Date: Thu, 19 Apr 2007 00:23:05 -0400 (EDT) Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200704190423.l3J4N5t2029975@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 357 open ( +8) / 3745 closed ( +8) / 4102 total (+16) Bugs : 958 open (+19) / 6657 closed ( +9) / 7615 total (+28) RFE : 251 open ( +2) / 280 closed ( +2) / 531 total ( +4) New / Reopened Patches ______________________ Help with Python codebase (2007-04-11) CLOSED http://python.org/sf/1698723 opened by Munawar Update to Macintosh library documentation (2007-04-11) http://python.org/sf/1698768 opened by Kevin Walzer ZipFile.printdir fix (2.5) (2007-04-11) http://python.org/sf/1698915 opened by Alan McIntyre ZipFile.printdir fix (2.6) (2007-04-11) http://python.org/sf/1698917 opened by Alan McIntyre Bastion and rexec message out-of-date (2007-04-12) http://python.org/sf/1698951 opened by Gabriel Genellina getstate/setstate for incremental codecs (2007-04-12) CLOSED http://python.org/sf/1698994 opened by Walter D?rwald replacing char* with const char* in sysmodule.c/.h (2007-04-12) http://python.org/sf/1699259 opened by sebastinas Armin's method cache optimization updated for Python 2.6 (2007-04-13) http://python.org/sf/1700288 opened by Kevin Jacobs VC6 build patch for trunk (2007-04-14) http://python.org/sf/1700463 opened by Hirokazu Yamamoto stack size of python_d.exe on VC6 (2007-04-14) http://python.org/sf/1700467 opened by Hirokazu Yamamoto link_objects in setup.cfg crashes build (2007-04-18) http://python.org/sf/1703178 opened by Markus Schatten silenced a compiler warning (2007-04-18) http://python.org/sf/1703268 opened by Alexander Belopolsky missing declaration in readline.c (2007-04-18) http://python.org/sf/1703270 opened by Alexander Belopolsky bug fix: ctypes truncates 64-bit pointers (2007-04-18) http://python.org/sf/1703286 opened by Alexander Belopolsky fixes non ansi c declarations in libffi (2007-04-18) http://python.org/sf/1703300 opened by Alexander Belopolsky Refactor test_frozen.py to use unittest. (2007-04-18) http://python.org/sf/1703379 opened by Jerry Seutter Patches Closed ______________ Fix test_urllib on Windows buildbots (2007-04-07) http://python.org/sf/1695862 closed by zseil Don't block on Queue get/put when time is moved back (2007-03-29) http://python.org/sf/1690578 closed by loewis struct.pack() on 64bit architectures (2004-03-30) http://python.org/sf/925932 closed by zseil bug# 1607041: Condition.wait timeout fails on clock change (2006-12-01) http://python.org/sf/1607149 closed by loewis Add IllegalStateError (2007-03-21) http://python.org/sf/1685642 closed by gvanrossum Help with Python codebase (2007-04-11) http://python.org/sf/1698723 closed by gbrandl getstate/setstate for incremental codecs (2007-04-12) http://python.org/sf/1698994 closed by doerwalter Change *args from a tuple to list (2006-05-31) http://python.org/sf/1498441 closed by collinwinter New / Reopened Bugs ___________________ README is referencing obsolete? http://starship.python.net (2007-04-09) http://python.org/sf/1696740 opened by Jiri Navratil package.pth file name not used as described. (2007-04-09) http://python.org/sf/1697169 opened by cfk winreg module for cygwin? (2007-04-09) http://python.org/sf/1697175 opened by Zooko O'Whielacronx Property with -> annotation triggers assert (2007-04-09) http://python.org/sf/1697248 opened by Guido van Rossum types.InstanceType is missing but used by pydoc (2007-04-10) http://python.org/sf/1697782 reopened by gbrandl types.InstanceType is missing but used by pydoc (2007-04-10) http://python.org/sf/1697782 opened by Christian Heimes __getslice__ still used in built-in types (2007-04-10) http://python.org/sf/1697820 opened by Torsten Bronger Segfaults on memory error (2007-04-10) http://python.org/sf/1697916 reopened by gbrandl Segfaults on memory error (2007-04-10) http://python.org/sf/1697916 opened by STINNER Victor msgfmt cannot cope with BOM (2007-04-10) http://python.org/sf/1697943 opened by Christoph Zwerschke xml.etree document element.tag (2007-04-11) http://python.org/sf/1698167 opened by paul rubin wrong % of params for format string in ZipFile.printdir() (2007-04-11) http://python.org/sf/1698398 opened by Szabolcs Berecz dtdparser discards last line (2007-04-11) CLOSED http://python.org/sf/1698944 opened by L. Peter Deutsch shlex fails to parse strings correctly (2007-04-12) http://python.org/sf/1699594 opened by Collin Winter pickle example contains errors (2007-04-13) http://python.org/sf/1699759 opened by Mark Edgington locale.getlocale() output fails as setlocale() input (2007-04-13) http://python.org/sf/1699853 opened by Bernhard Reiter import and capital letters (2007-04-13) http://python.org/sf/1700132 opened by omsynge pydoc.help samples sys.stdout and sys.stdin at import time (2007-04-13) http://python.org/sf/1700304 opened by James Lingard ctypes Fundamental data types (2007-04-14) http://python.org/sf/1700455 opened by hg6980 Carbon.Scrap.PutScrapFlavor (2007-04-14) http://python.org/sf/1700507 opened by dingus9 PCbuild8/pcbuild.sln is missing "_socket" sub-project (2007-04-14) http://python.org/sf/1700865 opened by Raghuram Devarakonda questionable results from IDLE, Windows, & several built-in (2007-04-15) http://python.org/sf/1700912 opened by bavantgarde314 symbol conflict for 'readahead' (2007-04-15) http://python.org/sf/1701192 opened by ajrh svnversion_init assertion failed (2007-04-15) CLOSED http://python.org/sf/1701207 opened by ajrh utf-16 codec problems with multiple file append (2007-04-16) http://python.org/sf/1701389 opened by Iceberg Luo Segfault in c_char_p of ctypes (2007-04-16) http://python.org/sf/1701409 opened by sebastien Martini documentation bug in profile module (2007-04-16) CLOSED http://python.org/sf/1701449 opened by Keith Briggs Feature request: Comparing regexps (2007-04-16) http://python.org/sf/1701452 opened by Thomas Dybdahl Ahle Turtle isn't thread-safe (crashes) (2007-04-17) http://python.org/sf/1702036 opened by lomm distutils sdist does not exclude SVN/CVS files on Windows (2007-04-18) http://python.org/sf/1702551 opened by Guy Dalberto Incorrect example for add_password() (2007-04-18) http://python.org/sf/1703110 opened by Bob Kline Bugs Closed ___________ logging.handlers.SocketHandler.makeSocket() blocks app (2007-04-07) http://python.org/sf/1695948 closed by vsajip asyncore DoS vulnerability (2007-03-21) http://python.org/sf/1685000 closed by josiahcarlson compiler package needs better documentation. (2000-11-27) http://python.org/sf/223616 closed by facundobatista types.InstanceType is missing but used by pydoc (2007-04-10) http://python.org/sf/1697782 closed by gbrandl Segfaults on memory error (2007-04-10) http://python.org/sf/1697916 closed by haypo dead link in tkinter documentation (2007-03-11) http://python.org/sf/1678710 closed by fdrake Condition.wait timeout fails when system clock changes (2006-12-01) http://python.org/sf/1607041 closed by rhettinger dtdparser discards last line (2007-04-12) http://python.org/sf/1698944 closed by loewis svnversion_init assertion failed (2007-04-15) http://python.org/sf/1701207 closed by nnorwitz documentation bug in profile module (2007-04-16) http://python.org/sf/1701449 closed by quiver New / Reopened RFE __________________ audioop module - request to add a note (2007-04-15) http://python.org/sf/1700821 opened by ncryptor Prevent textwrap from breaking words at hyphens (2007-04-17) http://python.org/sf/1702681 opened by Matt Kraai RFE Closed __________ make slices pickable (2005-04-28) http://python.org/sf/1191699 closed by rhettinger 'str'.translate(None) => identity translation (2005-04-30) http://python.org/sf/1193128 closed by rhettinger From oliphant.travis at ieee.org Thu Apr 19 06:40:28 2007 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 18 Apr 2007 22:40:28 -0600 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) In-Reply-To: <46246AF7.5000304@aerojockey.com> References: <461AD36D.4080608@aerojockey.com> <461AE11E.8040901@ieee.org> <461D96BF.2080702@aerojockey.com> <461F2B28.6030508@ieee.org> <46246AF7.5000304@aerojockey.com> Message-ID: <4626F2BC.8070608@ieee.org> Carl Banks wrote: > Ok, I've thought quite a bit about this, and I have an idea that I > think will be ok with you, and I'll be able to drop my main > objection. It's not a big change, either. The key is to explicitly > say whether the flag allows or requires. But I made a few other > changes as well. I'm good with using an identifier to differentiate between an "allowed" flag and a "require" flag. I'm not a big fan of VERY_LONG_IDENTIFIER_NAMES though. Just enough to understand what it means but not so much that it takes forever to type and uses up horizontal real-estate. We use flags in NumPy quite a bit, and I'm obviously trying to adapt some of this to the general case here, but I'm biased by my 10 years of experience with the way I think about NumPy arrays. Thanks for helping out and offering your fresh approach. I like a lot of what you've come up with. There are a few modifications I would make, though. > > First of all, let me define how I'm using the word "contiguous": it's > a single buffer with no gaps. So, if you were to do this: > "memset(bufinfo->buf,0,bufinfo->len)", you would not touch any data > that isn't being exported. Sure, we call this NPY_ONESEGMENT in NumPy-speak, though, because contiguous could be NPY_C_CONTIGUOUS or NPY_F_CONTIGUOUS. We also don't use the terms ROW_MAJOR and COLUMN_MAJOR and so I'm not a big fan of bringing them up in the Python space because the NumPy community has already learned the C_ and F_ terminology which also generalizes to multiple-dimensions more clearly without using 2-d concepts. > > Without further ado, here is my proposal: > > > ------ > > With no flags, the PyObject_GetBuffer will raise an exception if the > buffer is not direct, contiguous, and one-dimensional. Here are the > flags and how they affect that: I'm not sure what you mean by "direct" here. But, this looks like the Py_BUF_SIMPLE case (which was a named-constant for 0) in my proposal. The exporter receiving no flags would need to return a simple buffer (and it wouldn't need to fill in the format character either --- valuable information for the exporter to know). > > Py_BUF_REQUIRE_WRITABLE - Raise exception if the buffer isn't writable. WRITEABLE is an alternative spelling and the one that NumPy uses. So, either include both of these as alternatives or just use WRITEABLE. > > Py_BUF_REQUIRE_READONLY - Raise excpetion if the buffer is writable. Or if the object memory can't be made read-only if it is writeable. > > Py_BUF_ALLOW_NONCONTIGUOUS - Allow noncontiguous buffers. (This turns > on "shape" and "strides".) > Fine. > Py_BUF_ALLOW_MULTIDIMENSIONAL - Allow multidimensional buffers. (Also > turns on "shape" and "strides".) Just use ND instead of MULTIDIMENSIONAL and only turn on shape if it is present. > > (Neither of the above two flags implies the other.) > > Py_BUF_ALLOW_INDIRECT - Allow indirect buffers. Implies > Py_BUF_ALLOW_NONCONTIGUOUS and Py_BUF_ALLOW_MULTIDIMENSIONAL. (Turns > on "shape", "strides", and "suboffsets".) If we go with this consumer-oriented naming scheme, I like indirect also. > > Py_BUF_REQUIRE_CONTIGUOUS_C_ARRAY or Py_BUF_REQUIRE_ROW_MAJOR - Raise > an exception if the array isn't a contiguous array with in C > (row-major) format. > > Py_BUF_REQUIRE_CONTIGUOUS_FORTRAN_ARRAY or Py_BUF_REQUIRE_COLUMN_MAJOR > - Raise an exception if the array isn't a contiguous array with in > Fortran (column-major) format. Just name them C_CONTIGUOUS and F_CONTIGUOUS like in NumPy. > > Py_BUF_ALLOW_NONCONTIGUOUS, Py_BUF_REQUIRE_CONTIGUOUS_C_ARRAY, and > Py_BUF_REQUIRE_CONTIGUOUS_FORTRAN_ARRAY all conflict with each other, > and an exception should be raised if more than one are set. > > (I would go with ROW_MAJOR and COLUMN_MAJOR: even though the terms > only make sense for 2D arrays, I believe the terms are commonly > generalized to other dimensions.) As I mentioned there is already a well-established history with NumPy. We've dealt with this issue already. > > Possible pseudo-flags: > > Py_BUF_SIMPLE = 0; > Py_BUF_ALLOW_STRIDED = Py_BUF_ALLOW_NONCONTIGUOUS > | Py_BUF_ALLOW_MULTIDIMENSIONAL; > > ------ > > Now, for each flag, there should be an associated function to test the > condition, given a bufferinfo struct. (Though I suppose they don't > necessarily have to map one-to-one, I'll do that here.) > > int PyBufferInfo_IsReadonly(struct bufferinfo*); > int PyBufferInfo_IsWritable(struct bufferinfo*); > int PyBufferInfo_IsContiguous(struct bufferinfo*); > int PyBufferInfo_IsMultidimensional(struct bufferinfo*); > int PyBufferInfo_IsIndirect(struct bufferinfo*); > int PyBufferInfo_IsRowMajor(struct bufferinfo*); > int PyBufferInfo_IsColumnMajor(struct bufferinfo*); > > The function PyObject_GetBuffer then has a pretty obvious > implementation. Here is an except: > > if ((flags & Py_BUF_REQUIRE_READONLY) && > !PyBufferInfo_IsReadonly(&bufinfo)) { > PyExc_SetString(PyErr_BufferError,"buffer not read-only"); > return 0; > } > > Pretty straightforward, no? > > Now, here is a key point: for these functions to work (indeed, for > PyObject_GetBuffer to work at all), you need enough information in > bufinfo to figure it out. The bufferinfo struct should be > self-contained; you should not need to know what flags were passed to > PyObject_GetBuffer in order to know exactly what data you're looking at. Naturally. > > > Therefore, format must always be supplied by getbuffer. You cannot > tell if an array is contiguous without the format string. (But see > below.) No, I don't think this is quite true. You don't need to know what "kind" of data you are looking at if you don't get strides. If you use the SIMPLE interface, then both consumer and exporter know the object is looking at "bytes" which always has an itemsize of 1. > > And even if the consumer isn't asking for a contiguous buffer, it has > to know the item size so it knows what data not to step on. > > (This is true even in your own proposal, BTW. If a consumer asks for > a non-strided array in your proposal, PyObject_GetBuffer would have to > know the item size to determine if the array is contiguous.) Yes, it is true, that getting strides requires that the format be specified as well. That was an oversight of the original proposal. But, if strides are not needed, then format is also not needed. > > > ------ > > FAQ: > > Q. Why ALLOW_NONCONTIGUOUS and ALLOW_MULTIDIMENSIONAL instead of > ALLOW_STRIDED and ALLOW_SHAPED? > > A. It's more useful to the consumer that way. With ALLOW_STRIDED and > ALLOW_SHAPED, there's no way for a consumer to request a general > one-dimensional array (it can only request a non-strided > one-dimensional array), and requesting a SHAPED array but not a > STRIDED one can only return a C-like (row-major) array, although a > consumer might reasonably want a Fortran-like (column-major) array. > This approach maps more directly to the consumer's needs, is more > flexible, and still maintains the same functionality of ALLOW_SHAPED > and ALLOW_STRIDED. > > > Q. Why call it ALLOW_INDIRECT instead of ALLOW_OFFSETS? > > A. It's just a name, and not too important to me, but I wanted to > emphasize the consumer's usage, rather than the benefit to the > exporter. The consumers, after all, are the ones setting the flags. > > > Q. Why ALLOW_NONCONTIGUOUS instead of REQUIRE_CONTIGUOUS? > > Two reasons: 1. Contiguous arrays are "simpler", so it's better to > make the people who want more complex arrays to work harder, and 2. > ALLOW_NONCONTIGUOUS is closely tied to ALLOW_MULTIDIMENSIONAL. If the > negative is a problem, perhaps a name like ALLOW_DISCONTINUOUS or > ALLOW_GAPS would be better? > > > Q. What about Py_BUF_FORMAT? > > A. Ok, fine, if it's that imporant to you. I think it's totally > superfluous, but it's not evil. But consider these things: > > 1. Require that it does not throw an exception. It's not the > exporter's business to tell the consumer to how to use its data. Look, consumers that want to be "in-charge" can just ask for format data and ignore it. If an exporter wants to be persnickety about how its data is viewed, then it should be allowed to be. Perhaps it has good reason. It's just a matter of how much "work" it is to get the "wrong" view of the data. > > 2. Even if you don't supply the format string, you need to supply an > itemsize in struct bufferinfo, otherwise there is no way for a > consumer to determine if the array is contiguous, and or to know (in > general) what data is being exported. The itemsize must ALWAYS be > available. Only if strides is provided and format isn't is itemsize actually needed. But, we've added the itemsize field anyway. > > 3. Invert Py_BUF_FORMAT. Use Py_BUF_DONT_NEED_FORMAT instead. Make > the consumer that cares about performance ask for the optimization. > (You admit yourself that Py_BUF_FORMAT is part of the least common > denominator, so invert it.) Either way. I think the Py_BUF_FORMAT is easier because then Py_BUF_SIMPLE is just a numerical value of 0. I'll update the PEP with my adaptation of your suggestions in a little while. -Travis From pythondev at aerojockey.com Thu Apr 19 08:53:50 2007 From: pythondev at aerojockey.com (Carl Banks) Date: Thu, 19 Apr 2007 02:53:50 -0400 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) Message-ID: <462711FE.6080102@aerojockey.com> Travis Oliphant wrote: > Carl Banks wrote: >> Ok, I've thought quite a bit about this, and I have an idea that I >> think will be ok with you, and I'll be able to drop my main >> objection. It's not a big change, either. The key is to explicitly >> say whether the flag allows or requires. But I made a few other >> changes as well. > I'm good with using an identifier to differentiate between an "allowed" > flag and a "require" flag. I'm not a big fan of > VERY_LONG_IDENTIFIER_NAMES though. Just enough to understand what it > means but not so much that it takes forever to type and uses up > horizontal real-estate. That's fine with me. I'm not very particular about spellings, as long as they're not misleading. >> Now, here is a key point: for these functions to work (indeed, for >> PyObject_GetBuffer to work at all), you need enough information in >> bufinfo to figure it out. The bufferinfo struct should be >> self-contained; you should not need to know what flags were passed to >> PyObject_GetBuffer in order to know exactly what data you're looking at. > Naturally. > >> Therefore, format must always be supplied by getbuffer. You cannot >> tell if an array is contiguous without the format string. (But see >> below.) > > No, I don't think this is quite true. You don't need to know what > "kind" of data you are looking at if you don't get strides. If you use > the SIMPLE interface, then both consumer and exporter know the object is > looking at "bytes" which always has an itemsize of 1. But doesn't this violate the above maxim? Suppose these are the contents of bufinfo: ndim = 1 len = 20 shape = (10,) strides = (2,) format = NULL How does it know whether it's looking at contiguous array of 10 two-byte objects, or a discontiguous array of 10 one-byte objects, without having at least an item size? Since item size is now in the mix, it's moot, of course. The idea that Py_BUF_SIMPLE implies bytes is news to me. What if you want a contiguous, one-dimensional array of an arbitrary type? I was thinking this would be acceptable with Py_BUF_SIMPLE. It seems you want to require Py_BUF_FORMAT for that, which would suggest to me that Py_BUF_ALLOW_ND amd Py_BUF_ALLOW_NONCONTIGUOUS, etc., would imply Py_BUF_FORMAT? IOW, pretty much anything that's not SIMPLE implies FORMAT? If that's the case, then most of the issues I brought up about item size don't apply. Also, if that's the case, you're right that Py_BUF_FORMAT makes more sense than Py_BUF_DONT_NEED_FORAMT. But it now it seems even more unnecessary than it did before. Wouldn't any consumer that just wants to look at a chunk of bytes always use Py_BUF_FORMAT, especially if there's danger of a presumptuous exporter raising an exception? > I'll update the PEP with my adaptation of your suggestions in a little > while. Ok. Thanks for taking the lead, and for putting up with my verbiose nitpicking. :) Carl Banks From oliphant.travis at ieee.org Thu Apr 19 09:47:10 2007 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 19 Apr 2007 01:47:10 -0600 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) In-Reply-To: <46270F94.8000304@aerojockey.com> References: <461AD36D.4080608@aerojockey.com> <461AE11E.8040901@ieee.org> <461D96BF.2080702@aerojockey.com> <461F2B28.6030508@ieee.org> <46246AF7.5000304@aerojockey.com> <4626F2BC.8070608@ieee.org> <46270F94.8000304@aerojockey.com> Message-ID: <46271E7E.3030409@ieee.org> Carl Banks wrote: > Travis Oliphant wrote: >> Carl Banks wrote: >>> Ok, I've thought quite a bit about this, and I have an idea that I >>> think will be ok with you, and I'll be able to drop my main >>> objection. It's not a big change, either. The key is to explicitly >>> say whether the flag allows or requires. But I made a few other >>> changes as well. >> I'm good with using an identifier to differentiate between an >> "allowed" flag and a "require" flag. I'm not a big fan of >> VERY_LONG_IDENTIFIER_NAMES though. Just enough to understand what it >> means but not so much that it takes forever to type and uses up >> horizontal real-estate. > > That's fine with me. I'm not very particular about spellings, as long > as they're not misleading. > >>> Now, here is a key point: for these functions to work (indeed, for >>> PyObject_GetBuffer to work at all), you need enough information in >>> bufinfo to figure it out. The bufferinfo struct should be >>> self-contained; you should not need to know what flags were passed >>> to PyObject_GetBuffer in order to know exactly what data you're >>> looking at. >> Naturally. >> >>> Therefore, format must always be supplied by getbuffer. You cannot >>> tell if an array is contiguous without the format string. (But see >>> below.) >> >> No, I don't think this is quite true. You don't need to know what >> "kind" of data you are looking at if you don't get strides. If you >> use the SIMPLE interface, then both consumer and exporter know the >> object is looking at "bytes" which always has an itemsize of 1. > > But doesn't this violate the above maxim? Suppose these are the > contents of bufinfo: > > ndim = 1 > len = 20 > shape = (10,) > strides = (2,) > format = NULL In my thinking, format/itemsize is necessary if you have strides (as you do here) but not needed if you don't have strides information (i.e. you are assuming a C_CONTIGUOUS memory-chunk). The intent of the simple interface is to basically allow consumers to mimic the old buffer protocol, very easily. > > How does it know whether it's looking at contiguous array of 10 > two-byte objects, or a discontiguous array of 10 one-byte objects, > without having at least an item size? Since item size is now in the > mix, it's moot, of course. My only real concern is to have some way to tell the exporter that it doesn't need to "figure out" the format if the consumer doesn't care about it. Given the open-ended nature of the format string, it is possible that a costly format-string construction step could be under-taken even when the consumer doesn't care about it. I can see you are considering the buffer structure as a self-introspecting structure where I was considering it only in terms of how the consumer would be using its members (which implied it knew what it was asking for and wouldn't touch anything else). How about we assume FORMAT will always be filled in but we add a Py_BUF_REQUIRE_PRIMITIVE flag that will only return "primitive" format strings (i.e. basic c-types)? An exporter receiving this flag will have to return complicated data-types as 'bytes'. I would add this to the Py_BUF_SIMPLE default. > > The idea that Py_BUF_SIMPLE implies bytes is news to me. What if you > want a contiguous, one-dimensional array of an arbitrary type? I was > thinking this would be acceptable with Py_BUF_SIMPLE. Unsigned bytes are just the lowest common denominator. They represent the old way of sharing memory. Doesn't an "arbitrary type" mean bytes? Or did you mean what if you wanted a contiguous, one-dimensional array of a *specific* type? > It seems you want to require Py_BUF_FORMAT for that, which would > suggest to me that > But it now it seems even more unnecessary than it did before. > Wouldn't any consumer that just wants to look at a chunk of bytes > always use Py_BUF_FORMAT, especially if there's danger of a > presumptuous exporter raising an exception? > I'll put in the REQUIRE_PRIMITIVE_FORMAT idea in the next update to the PEP. I can just check in my changes to SVN, so it should show up by Friday. Thanks again, -Travis From anthony at python.org Thu Apr 19 10:45:59 2007 From: anthony at python.org (Anthony Baxter) Date: Thu, 19 Apr 2007 18:45:59 +1000 Subject: [Python-Dev] RELEASED Python 2.5.1, FINAL Message-ID: <200704191846.06857.anthony@python.org> On behalf of the Python development team and the Python community, I'm happy to announce the release of Python 2.5.1 (FINAL) This is the first bugfix release of Python 2.5. Python 2.5 is now in bugfix-only mode; no new features are being added. According to the release notes, over 150 bugs and patches have been addressed since Python 2.5, including a fair number in the new AST compiler (an internal implementation detail of the Python interpreter). This is a production release of Python, and should be a painless upgrade from 2.5. Since the release candidate, we have backed out a couple of small changes that caused 2.5.1 to behave differently to 2.5. See the release notes for more. For more information on Python 2.5.1, including download links for various platforms, release notes, and known issues, please see: http://www.python.org/2.5.1/ Highlights of this new release include: Bug fixes. According to the release notes, at least 150 have been fixed. Highlights of the previous major Python release (2.5) are available from the Python 2.5 page, at http://www.python.org/2.5/highlights.html Enjoy this release, Anthony Anthony Baxter anthony at python.org Python Release Manager (on behalf of the entire python-dev team) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20070419/b35f5586/attachment.pgp From ndbecker2 at gmail.com Thu Apr 19 19:33:31 2007 From: ndbecker2 at gmail.com (Neal Becker) Date: Thu, 19 Apr 2007 13:33:31 -0400 Subject: [Python-Dev] python3k change to slicing Message-ID: There is one thing I'd like to see changed in a future python. I always found it surprising, that >>> x = [1,2,3,4,5] >>> x[1:10] [2, 3, 4, 5] is not an error. This is perhaps the only case (but a fundamental one!) where an error is silently ignored. I really can't think of a good justification for it. If I really meant x[1:] I would have said so. From guido at python.org Thu Apr 19 19:40:41 2007 From: guido at python.org (Guido van Rossum) Date: Thu, 19 Apr 2007 10:40:41 -0700 Subject: [Python-Dev] python3k change to slicing In-Reply-To: References: Message-ID: [+python-3000; followups please remove python-dev] -1 While this may be theoretically preferable, I believe that in practice changing this would be a major pain for very little gain. I don't recall ever finding a bug related to this feature, and I believe it's occasionally useful. Here's something that would be much more cumbersome with your proposed change: suppose I have a string of unknown length and I want to get the first three characters, or less if it's not that long. Today I can write s[:3]. With your proposal I would have to write s[:min(3, len(s))]. --Guido On 4/19/07, Neal Becker wrote: > There is one thing I'd like to see changed in a future python. I always > found it surprising, that > >>> x = [1,2,3,4,5] > >>> x[1:10] > [2, 3, 4, 5] > > is not an error. This is perhaps the only case (but a fundamental one!) > where an error is silently ignored. > > I really can't think of a good justification for it. If I really meant > x[1:] > I would have said so. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jon+python-dev at unequivocal.co.uk Thu Apr 19 20:03:56 2007 From: jon+python-dev at unequivocal.co.uk (Jon Ribbens) Date: Thu, 19 Apr 2007 19:03:56 +0100 Subject: [Python-Dev] python3k change to slicing In-Reply-To: References: Message-ID: <20070419180356.GA23226@snowy.squish.net> Guido van Rossum wrote: > -1 Me too. > While this may be theoretically preferable, I believe that in practice > changing this would be a major pain for very little gain. I don't > recall ever finding a bug related to this feature, and I believe it's > occasionally useful. I find it quite frequently useful. > Here's something that would be much more cumbersome with your proposed > change: suppose I have a string of unknown length and I want to get > the first three characters, or less if it's not that long. Today I can > write s[:3]. With your proposal I would have to write s[:min(3, > len(s))]. ... and that's exactly the sort of situation I find it useful ;-) I certainly think it's easier to check that your result is the length you wanted on the occasions you need to, than it would be to make the replacement you show above everywhere else. From tjreedy at udel.edu Thu Apr 19 20:15:45 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 19 Apr 2007 14:15:45 -0400 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) References: <461AD36D.4080608@aerojockey.com><461AE11E.8040901@ieee.org> <461D96BF.2080702@aerojockey.com><461F2B28.6030508@ieee.org> <46246AF7.5000304@aerojockey.com> <4626F2BC.8070608@ieee.org> Message-ID: "Travis Oliphant" wrote in message news:4626F2BC.8070608 at ieee.org... | I'm good with using an identifier to differentiate between an "allowed" | flag and a "require" flag. I'm not a big fan of | VERY_LONG_IDENTIFIER_NAMES though. Just enough to understand what it | means but not so much that it takes forever to type and uses up | horizontal real-estate. To save fingers and real-estate, adopt the following convention: by default, adjectives like writable and contiguous are 'required' unless tagged with 'OK', as in WRITABLE_OK. Explain that in the flag doc just before the flags themselves. And yes, ND for N_DIMENSIONAL or MULTIDIMENSIONAL is also a great win that can also be explained in the same intro. Terry Jan Reedy From greg.ewing at canterbury.ac.nz Fri Apr 20 00:51:49 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 20 Apr 2007 10:51:49 +1200 Subject: [Python-Dev] PEP 3118: Extended buffer protocol (new version) In-Reply-To: <4626EA1C.9040407@ieee.org> References: <461AD36D.4080608@aerojockey.com> <461AE11E.8040901@ieee.org> <461D96BF.2080702@aerojockey.com> <461F2B28.6030508@ieee.org> <46246AF7.5000304@aerojockey.com> <46246E0D.7060506@canterbury.ac.nz> <4626EA1C.9040407@ieee.org> Message-ID: <4627F285.6070001@canterbury.ac.nz> Travis Oliphant wrote: > you would like to make the original memory > read-only until you are done with the algorithm and have copied the data > back into memory. Okay, I see what you mean now. Maybe this should be called Py_BUF_LOCK_CONTENTS or something to make the semantics clearer. Otherwise it could mean either that *you* don't intend to write to it, or that you require nobody ever to write to it. -- Greg From tds333+pydev at gmail.com Fri Apr 20 10:04:56 2007 From: tds333+pydev at gmail.com (Wolfgang Langner) Date: Fri, 20 Apr 2007 10:04:56 +0200 Subject: [Python-Dev] Win64, 64 Bit Version and 32 Bit version parallel install not possible Message-ID: <4c45c1530704200104i2d26f69es9c8e2c8cd4d6b85a@mail.gmail.com> Hello, I tried to install the 64 Bit python version (2.5.1) and the 32 Bit version on my computer. But it is not possible because the installer complains that this version of python is already installed. Is it in general not possible to install both versions (in separate directories) ? Or is it a bug ? Wolfgang From kristjan at ccpgames.com Fri Apr 20 15:54:43 2007 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Fri, 20 Apr 2007 13:54:43 +0000 Subject: [Python-Dev] static analysis of python source Message-ID: <4E9372E6B2234D4F859320D896059A9508C00CE094@exchis.ccp.ad.local> I just ran some static analysis of the python core 2.5 with Visual Studio team system. There was the stray error discovered. I will update the most obvious ones. Some questions, though: This code in binascii.c, line 1168 (and again 1238) is wrong: while (in < datalen) { if ((data[in] > 126) || (data[in] == '=') || (header && data[in] == '_') || ((data[in] == '.') && (linelen == 1)) || (!istext && ((data[in] == '\r') || (data[in] == '\n'))) || ((data[in] == '\t' || data[in] == ' ') && (in + 1 == datalen)) || ((data[in] < 33) && (data[in] != '\r') && (data[in] != '\n') && (quotetabs && ((data[in] != '\t') || (data[in] != ' '))))) The final ((data[in] != '\t') || (data[in] != ' ')) is always true. What is the right form? ((data[in] == '\t') || (data[in] == ' ')) ? 1) 2) There is a lot of code that goes like this: -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070420/6f3d8751/attachment.htm From thomas at python.org Fri Apr 20 15:56:02 2007 From: thomas at python.org (Thomas Wouters) Date: Fri, 20 Apr 2007 15:56:02 +0200 Subject: [Python-Dev] new metaclass error condition checks In-Reply-To: <20070416225320.19381.297920264.divmod.quotient.2282@ohm> References: <20070416225320.19381.297920264.divmod.quotient.2282@ohm> Message-ID: <9e804ac0704200656k323d0633i106fdc5daf612f8a@mail.gmail.com> On 4/17/07, Jean-Paul Calderone wrote: > > I just noticed r53997 (from some unit tests it broke), which disallowed > things > like this: > > class X(object): > def __repr__(self): > return "blah" > > class Y(X, type): > pass > > class Z: > __metaclass__ = Y > > Making X classic eliminates the TypeError, and is probably an acceptable > fix > in a lot of cases (at least as long as classic classes are available). I > wonder if the ability to override type special methods like this was > considered > when the change was made, though? Probably not, or at least not consciously. I think the point is that 'type' and its subclasses are special enough that this warrants a separate 'X' that inherits from 'type' rather than 'object'. Reduced reusability, but how badly does this affect you in the real world, really? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070420/ed0c6bef/attachment.html From kristjan at ccpgames.com Fri Apr 20 15:57:55 2007 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Fri, 20 Apr 2007 13:57:55 +0000 Subject: [Python-Dev] FW: static analysis of python source Message-ID: <4E9372E6B2234D4F859320D896059A9508C00CE095@exchis.ccp.ad.local> I just ran some static analysis of the python core 2.5 with Visual Studio team system. There was the stray error discovered. I will update the most obvious ones. Some questions, though: This code in binascii.c, line 1168 (and again 1238) is wrong: while (in < datalen) { if ((data[in] > 126) || (data[in] == '=') || (header && data[in] == '_') || ((data[in] == '.') && (linelen == 1)) || (!istext && ((data[in] == '\r') || (data[in] == '\n'))) || ((data[in] == '\t' || data[in] == ' ') && (in + 1 == datalen)) || ((data[in] < 33) && (data[in] != '\r') && (data[in] != '\n') && (quotetabs && ((data[in] != '\t') || (data[in] != ' '))))) The final ((data[in] != '\t') || (data[in] != ' ')) is always true. What is the right form? ((data[in] == '\t') || (data[in] == ' ')) ? 1) 2) There is a lot of code that goes like this: f->buf = PyMem_Realloc(f->buf, newsize); if (!f->buf) return PyErr_NoMemory(), 0; Now, this if Realloc fails, this causes a memory leak. Is there any interest to fix this flawed pattern wholesale? Cheers, Kristjan -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070420/82e29ac3/attachment-0001.htm From kumar.mcmillan at gmail.com Fri Apr 20 17:37:33 2007 From: kumar.mcmillan at gmail.com (Kumar McMillan) Date: Fri, 20 Apr 2007 10:37:33 -0500 Subject: [Python-Dev] Deallocation of a pointer not malloced, any tips? Message-ID: I get this warning from my test suite when I introduced a segment of code: python(18603,0xa000d000) malloc: *** Deallocation of a pointer not malloced: 0x310caa3; This could be a double free(), or free() called with the middle of an allocated block; Try setting environment variable MallocHelp to see tools to help debug Always the same warning; sometimes it even segfaults python, but rarely. I only see it on Mac OS X (two separate machines), python 2.4.3 both built from source via Darwin Ports. The same suite gives no such warning on a 2.4.3 production machine running Free BSD. I know what segment of code causes it but can't for the life of me think of writing it a different way in this specific application. In a nutshell, it's two sets of two nested try blocks (try, try, except, except * 2) where the inner except can re-raise a new exception if a rescue function wants to. it sounds convoluted because it is! :( I tried really hard recreating it in a test case outside of my application but of course there is something too deeply buried that I can't catch. I turned on MallocHelp but just stared blankly at heaps and stack options. Is there anything I can switch on that would be helpful to anyone who might be interested in this problem? :) I also wanted to put out some feelers in case it sounds like something that is fixed elsewhere (I'm having trouble running my existing suite in 2.5 due to setuptools not working and me not having enough time to fiddle with it). Sorry if this is all very vague, I'm just a bit stumped and wanted to see if anyone had some suggestions. -Kumar From jcarlson at uci.edu Fri Apr 20 17:59:15 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Fri, 20 Apr 2007 08:59:15 -0700 Subject: [Python-Dev] Deallocation of a pointer not malloced, any tips? In-Reply-To: References: Message-ID: <20070420085606.637A.JCARLSON@uci.edu> "Kumar McMillan" wrote: > I get this warning from my test suite when I introduced a segment of code: > > python(18603,0xa000d000) malloc: *** Deallocation of a pointer not > malloced: 0x310caa3; This could be a double free(), or free() called > with the middle of an allocated block; Try setting environment > variable MallocHelp to see tools to help debug [snip] > Sorry if this is all very vague, I'm just a bit stumped and wanted to > see if anyone had some suggestions. You may want the python-list mailing list or the equivalent comp.lang.python newsgroup, unless this is a bug with Python itself (you may try running Python 2.4.4, which is the next bugfix of the Python 2.4 series). Regardless, you can help those who want to help you by providing the code that you have written that causes the error. A doubly nested try/except isn't all that exciting, so I would guess many would be surprised that you are getting double free errors. - Josiah From jcarlson at uci.edu Fri Apr 20 17:59:55 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Fri, 20 Apr 2007 08:59:55 -0700 Subject: [Python-Dev] FW: static analysis of python source In-Reply-To: <4E9372E6B2234D4F859320D896059A9508C00CE095@exchis.ccp.ad.local> References: <4E9372E6B2234D4F859320D896059A9508C00CE095@exchis.ccp.ad.local> Message-ID: <20070420084951.6377.JCARLSON@uci.edu> Kristj?n Valur J?nsson wrote: > > I just ran some static analysis of the python core 2.5 with Visual Studio team system. > There was the stray error discovered. I will update the most obvious ones. [snip] > 2) There is a lot of code that goes like this: > f->buf = PyMem_Realloc(f->buf, newsize); > if (!f->buf) > return PyErr_NoMemory(), 0; > Now, this if Realloc fails, this causes a memory leak. Is there any > interest to fix this flawed pattern wholesale? I believe the idea is that if you run into a MemoryError, in particular on linux (whose allocator will give you a nonzero pointer well beyond what was actually available), you can't really trust the state of the interpreter, so it is expected that Python will be ending shortly. Forcing the leak (leaving the code as-is) basically encourages the interpreter to have more and more errors before the expected, possibly inevitable (and perhaps desireable) quitting of the Python interpreter. - Josiah From amk at amk.ca Fri Apr 20 19:06:57 2007 From: amk at amk.ca (A.M. Kuchling) Date: Fri, 20 Apr 2007 13:06:57 -0400 Subject: [Python-Dev] Fwd: AMD64 version of Python Message-ID: <20070420170657.GA8785@localhost.localdomain> >From Walter Beck to webmaster: >I tried to install the AMD64 version of python on my new laptop, Vista >system. The installer said the processor was not correct. My processor >is a Turion64x2. 32 bit Python installed and ran fine. Is there a bug >in the installer or doesn't it like Vista or is this the wrong ADM64? Can anyone help him out? --amk From skip at pobox.com Fri Apr 20 19:30:32 2007 From: skip at pobox.com (skip at pobox.com) Date: Fri, 20 Apr 2007 12:30:32 -0500 Subject: [Python-Dev] Deallocation of a pointer not malloced, any tips? In-Reply-To: References: Message-ID: <17960.63672.30897.925610@montanaro.dyndns.org> Kumar> Always the same warning; sometimes it even segfaults python, but Kumar> rarely. I only see it on Mac OS X (two separate machines), python Kumar> 2.4.3 both built from source via Darwin Ports. The same suite gives no Kumar> such warning on a 2.4.3 production machine running Free BSD. In addition to Josiah's response I will note that the warning you get is specific to the Mac's malloc implementation (or maybe more generally to Darwin or *BSD). It's not necessarily that the problem doesn't occur on other platforms, just that the malloc implementation doesn't include that sort of check by default. Kumar> I turned on MallocHelp but just stared blankly at heaps and stack Kumar> options. I've used that stuff but it's been awhile. Try "man malloc" in a terminal window. If that doesn't get you going post your questions pythonmac-sig at python.org mailing list. That's were most of the Mac-head Python developers hang out. Skip From kristjan at ccpgames.com Fri Apr 20 20:19:56 2007 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Fri, 20 Apr 2007 18:19:56 +0000 Subject: [Python-Dev] .dll import Message-ID: <4E9372E6B2234D4F859320D896059A9508C00CE125@exchis.ccp.ad.local> I was surprised when 2.5 stopped importing our custom modules. So, I found this in dyload_win.c: /* Temporarily disable .dll, to avoid conflicts between sqlite3.dll and the sqlite3 package. If this needs to be reverted for 2.5, some other solution for the naming conflict must be found. This temporary fix seems to have been forgotten. Isn't it time it was fixed back? K -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070420/a2018524/attachment.html From martin at v.loewis.de Fri Apr 20 22:11:46 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 20 Apr 2007 22:11:46 +0200 Subject: [Python-Dev] Win64, 64 Bit Version and 32 Bit version parallel install not possible In-Reply-To: <4c45c1530704200104i2d26f69es9c8e2c8cd4d6b85a@mail.gmail.com> References: <4c45c1530704200104i2d26f69es9c8e2c8cd4d6b85a@mail.gmail.com> Message-ID: <46291E82.4090103@v.loewis.de> > I tried to install the 64 Bit python version (2.5.1) and the 32 Bit > version on my computer. > But it is not possible because the installer complains that this > version of python is already installed. > > Is it in general not possible to install both versions (in separate > directories) ? > > Or is it a bug ? It's not possible, because they use the same program GUID. You can use an MSI editor to change the program GUID, then you should be able to install them simultaneously (you may have to change the UpgradeCode as well, as otherwise installing one might silently uninstall the other). Of course, both installations will stomp on each other's registry keys, so while this MSI conflict is not deliberate (I never considered that setup), it may be a good thing that it already breaks at installation time. Regards, Martin From martin at v.loewis.de Fri Apr 20 22:16:16 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 20 Apr 2007 22:16:16 +0200 Subject: [Python-Dev] FW: static analysis of python source In-Reply-To: <20070420084951.6377.JCARLSON@uci.edu> References: <4E9372E6B2234D4F859320D896059A9508C00CE095@exchis.ccp.ad.local> <20070420084951.6377.JCARLSON@uci.edu> Message-ID: <46291F90.4000701@v.loewis.de> > I believe the idea is that if you run into a MemoryError, in particular > on linux (whose allocator will give you a nonzero pointer well beyond > what was actually available), you can't really trust the state of the > interpreter, so it is expected that Python will be ending shortly. > Forcing the leak (leaving the code as-is) basically encourages the > interpreter to have more and more errors before the expected, possibly > inevitable (and perhaps desireable) quitting of the Python interpreter. I don't think that this is the intention (and if it is, I think this intention is flawed). If you really need to shut down ASAP, you should exit(), not raise an exception. Raising MemoryError will exit soon enough, anyway. So I don't think the leak is a good thing - but I don't think it is a bad thing, either, since the code is essentially dead (i.e. it is fairly unlikely that it ever triggers). Regards, Martin From martin at v.loewis.de Fri Apr 20 22:18:53 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 20 Apr 2007 22:18:53 +0200 Subject: [Python-Dev] Fwd: AMD64 version of Python In-Reply-To: <20070420170657.GA8785@localhost.localdomain> References: <20070420170657.GA8785@localhost.localdomain> Message-ID: <4629202D.9040600@v.loewis.de> >>From Walter Beck to webmaster: > >> I tried to install the AMD64 version of python on my new laptop, Vista >> system. The installer said the processor was not correct. My processor >> is a Turion64x2. 32 bit Python installed and ran fine. Is there a bug >> in the installer or doesn't it like Vista or is this the wrong ADM64? Walter, Most likely, there is indeed a problem with your system installation. You likely have the 32-bit (x86) version of Vista installed, not the x64 version. On the 32-bit windows, the AMD64 mode of the processor is not available, and 64-bit windows binaries cannot run on that system. HTH, Martin From martin at v.loewis.de Fri Apr 20 22:20:57 2007 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Fri, 20 Apr 2007 22:20:57 +0200 Subject: [Python-Dev] .dll import In-Reply-To: <4E9372E6B2234D4F859320D896059A9508C00CE125@exchis.ccp.ad.local> References: <4E9372E6B2234D4F859320D896059A9508C00CE125@exchis.ccp.ad.local> Message-ID: <462920A9.3040304@v.loewis.de> > /* Temporarily disable .dll, to avoid conflicts between sqlite3.dll > > and the sqlite3 package. If this needs to be reverted for 2.5, > > some other solution for the naming conflict must be found. > > > > This temporary fix seems to have been forgotten. Isn?t it time it was > fixed back? It's time to remove the comment. The removal of .dll extensions for extension modules is now permanent. It hasn't caused too many problems, and no alternative solution to the sqlite3.dll problem has been implemented. Regards, Martin From facundo at taniquetil.com.ar Fri Apr 20 23:15:46 2007 From: facundo at taniquetil.com.ar (Facundo Batista) Date: Fri, 20 Apr 2007 21:15:46 +0000 (UTC) Subject: [Python-Dev] New decimal branch - news and status Message-ID: Ok, I cut a branch in svn to work with decimal.py (decimal-branch). I commited the work I made during the last weeks. Right now, the state is: - All the operations that already existed pass ok the new tests (except ``power``). For this, I fixed some bugs, reordered some code (without changing functionality), and made only one important change (see below). - The ``power`` operation has been redefined to allow raising a number to a non-integral power, that's why the old code does not pass the new tests. - These are the new operations (see the spec for info about them, but basically you have the logarithmic ones, and a lot that work with Decimal like a binary number): and, class, compare-total, compare-total-magnitude, copy, copy-abs, copy-negate, copy-sign, exp, fused-multiply-add, invert, ln, log10, logb, max-magnitude, min-magnitude, next-minus, next-plus, next-toward, or, rotate, scaleb, shift, to-integral-exact, trim, xor. Note that these names are from the spec, not the definitive ones. - Worked a bit in test_decimal, to make it a bit more debug-friendly: now, you can call it with the name of the test(s) you want to execute, and it will only execute that(those) test(s), and will *not* execute the doctests (normally, I have "print"s in the code, so doctests will always fail). Also, you can call it with --debug, and it will show the test number, and context, before executing each test. *Important change:* Added a "P" value to the valid exponents. See this test (with a precision of 5 in the context), for example:: maxx670 max 11 -sNaN12345678901 -> -NaN78901 Invalid_operation This is translated to something like:: Decimal('11').max(Decimal('-sNaN12345678901')) Doing ``Decimal('-sNaN12345678901')`` must signal InvalidOperation, because the payload has more digits that the context, and we can signal it through ConversionSyntax or InvalidOperation itself. ConversionSyntax must return a quiet NaN. In this case, the max operation will return the number, and it's not the desired result in this case. InvalidOperation must return a quiet NaN also, with the original sign, and an optional diagnostic information. The ``max`` operation, so, will always receive a quiet NaN, but it *must* know that it was before signaled. And it must know it from the diagnostic information. So far, for diagnostic information we used the Decimal._int digits, because that was all that was needed. I think it's more clear to leave there the payload digits, and not make that structure more complex, and create the new exponent. "P" is from "phantom" signal, because for everybody asking self._isnan(), it will return 1 (quiet NaN). But if you want to look at it in more detail, you will now that in a past life, it was signaled... Any help is greatly appreciated. Thank you very much! Regards, -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From anthony at interlink.com.au Sat Apr 21 08:45:37 2007 From: anthony at interlink.com.au (Anthony Baxter) Date: Sat, 21 Apr 2007 16:45:37 +1000 Subject: [Python-Dev] 2.5 branch unfrozen Message-ID: <200704211645.39875.anthony@interlink.com.au> Ok, things seem to be OK. So the release25-maint branch is unfrozen. Go crazy. Well, a little bit crazy. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From g.brandl at gmx.net Sat Apr 21 09:08:06 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 21 Apr 2007 09:08:06 +0200 Subject: [Python-Dev] static analysis of python source In-Reply-To: <4E9372E6B2234D4F859320D896059A9508C00CE094@exchis.ccp.ad.local> References: <4E9372E6B2234D4F859320D896059A9508C00CE094@exchis.ccp.ad.local> Message-ID: Kristj?n Valur J?nsson schrieb: > I just ran some static analysis of the python core 2.5 with Visual > Studio team system. > > There was the stray error discovered. I will update the most obvious ones. > > > > Some questions, though: > > > > This code in binascii.c, line 1168 (and again 1238) is wrong: > while (in < datalen) { > > if ((data[in] > 126) || > > (data[in] == '=') || > > (header && data[in] == '_') || > > ((data[in] == '.') && (linelen == 1)) || > > (!istext && ((data[in] == '\r') || (data[in] == '\n'))) || > > ((data[in] == '\t' || data[in] == ' ') && (in + 1 == > datalen)) || > > ((data[in] < 33) && > > (data[in] != '\r') && (data[in] != '\n') && > > (quotetabs && ((data[in] != '\t') || (data[in] != ' '))))) > > The final ((data[in] != '\t') || (data[in] != ' ')) is always > true. What is the right form? ((data[in] == '\t') || (data[in] == ' ')) ? FYI, this has already been fixed in the heads. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From kristjan at ccpgames.com Sat Apr 21 12:04:08 2007 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Sat, 21 Apr 2007 10:04:08 +0000 Subject: [Python-Dev] static analysis of python source In-Reply-To: References: <4E9372E6B2234D4F859320D896059A9508C00CE094@exchis.ccp.ad.local> Message-ID: <4E9372E6B2234D4F859320D896059A9508C00CE13D@exchis.ccp.ad.local> I see. There appear to be two fixes, one for the dots, and another for the quotetabs. Shouldn't this be backported? Kristj?n -----Original Message----- From: python-dev-bounces+kristjan=ccpgames.com at python.org [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Georg Brandl Sent: Saturday, April 21, 2007 07:08 To: python-dev at python.org Subject: Re: [Python-Dev] static analysis of python source > The final ((data[in] != '\t') || (data[in] != ' ')) is always > true. What is the right form? ((data[in] == '\t') || (data[in] == ' ')) ? FYI, this has already been fixed in the heads. From lars at gustaebel.de Sat Apr 21 13:57:09 2007 From: lars at gustaebel.de (Lars =?iso-8859-15?Q?Gust=E4bel?=) Date: Sat, 21 Apr 2007 13:57:09 +0200 Subject: [Python-Dev] 2.5 branch unfrozen In-Reply-To: <200704211645.39875.anthony@interlink.com.au> References: <200704211645.39875.anthony@interlink.com.au> Message-ID: <20070421115709.GA25234@axis.g33x.de> On Sat, Apr 21, 2007 at 04:45:37PM +1000, Anthony Baxter wrote: > Ok, things seem to be OK. So the release25-maint branch is unfrozen. > Go crazy. Well, a little bit crazy. I'm afraid that I went crazy a little too early. Sorry for that. Won't happen again. -- Lars Gust?bel lars at gustaebel.de The truth is rarely pure and never simple. (Oscar Wilde) From g.brandl at gmx.net Sat Apr 21 17:26:26 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 21 Apr 2007 17:26:26 +0200 Subject: [Python-Dev] static analysis of python source In-Reply-To: <4E9372E6B2234D4F859320D896059A9508C00CE13D@exchis.ccp.ad.local> References: <4E9372E6B2234D4F859320D896059A9508C00CE094@exchis.ccp.ad.local> <4E9372E6B2234D4F859320D896059A9508C00CE13D@exchis.ccp.ad.local> Message-ID: Kristj?n Valur J?nsson schrieb: > I see. There appear to be two fixes, one for the dots, and another for the quotetabs. > Shouldn't this be backported? Apparently I wasn't sure at the time. You now checked in one half of the fix, and put an exclamation mark at the wrong point (I assume that's why the 2.5 quopri tests are failing). I think that the fix should applied completely or not at all. Do you intend to apply the other fixes to the HEAD also? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From status at bugs.python.org Sun Apr 22 02:00:49 2007 From: status at bugs.python.org (Tracker) Date: Sun, 22 Apr 2007 00:00:49 +0000 (UTC) Subject: [Python-Dev] Summary of Tracker Issues Message-ID: <20070422000049.7A42C7821A@psf.upfronthosting.co.za> ACTIVITY SUMMARY (04/15/07 - 04/22/07) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1649 open ( +0) / 8584 closed ( +0) / 10233 total ( +0) Average duration of open issues: 771 days. Median duration of open issues: 722 days. Open Issues Breakdown open 1649 ( +0) pending 0 ( +0) Issues Now Closed (1) _____________________ Buy Tramadol online 21 days http://bugs.python.org/issue1023 tjreedy -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070422/04cbf263/attachment.htm From kristjan at ccpgames.com Sun Apr 22 03:17:42 2007 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Sun, 22 Apr 2007 01:17:42 +0000 Subject: [Python-Dev] static analysis of python source In-Reply-To: References: <4E9372E6B2234D4F859320D896059A9508C00CE094@exchis.ccp.ad.local> <4E9372E6B2234D4F859320D896059A9508C00CE13D@exchis.ccp.ad.local> Message-ID: <4E9372E6B2234D4F859320D896059A9508C00CE166@exchis.ccp.ad.local> Hello Georg. I must have messed up because I'm purdy sure I ran the test suite. Anyway, I'll do your fix tomorrow early, I promise. K -----Original Message----- From: python-dev-bounces+kristjan=ccpgames.com at python.org [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Georg Brandl Sent: Saturday, April 21, 2007 15:26 To: python-dev at python.org Subject: Re: [Python-Dev] static analysis of python source Kristj?n Valur J?nsson schrieb: > I see. There appear to be two fixes, one for the dots, and another for the quotetabs. > Shouldn't this be backported? Apparently I wasn't sure at the time. You now checked in one half of the fix, and put an exclamation mark at the wrong point (I assume that's why the 2.5 quopri tests are failing). I think that the fix should applied completely or not at all. Do you intend to apply the other fixes to the HEAD also? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com From ironfroggy at gmail.com Sun Apr 22 06:18:25 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sun, 22 Apr 2007 00:18:25 -0400 Subject: [Python-Dev] Python-Dev Summary Draft (April 1-15, 2007) Message-ID: <76fd5acf0704212118o6e50f97bn7ceadbf158f7e092@mail.gmail.com> There were a good number of skipped threads, but I've been out of the loop with a lot of busy things keeping me away from such things. Now that I'm getting back into it, I'll keep up with things again. So, let me know if any of the skipped threads should be brought in. Corrections and comments welcome as always. ============= Announcements ============= ========= Summaries ========= --------------- About SSL tests --------------- An open bug about missing SSL tests (#451607) was brought up to close or keep open. Jean-Paul Calderone mentioned an improved testing method by spawning a 'openssl s_server' for testing purposes. This sparked some talk about the terminating of subprocesses in a cross-platform manner (See `Cross-Platform Subprocess Termination`). Contributing thread: - `About SSL tests `__ ------------------------------------- Cross-Platform Subprocess Termination ------------------------------------- Because os.kill only works on *nix and OS X platforms, leaving Windows platforms uncovered. Although subprocess.TerminateProcess exists for Windows platforms, it requires the use of handles and additional overhead for use. Support was overall given for adding a UNIX-only signal() method and a cross- platform terminate() method to Popen instances. Nothing was said of actually incorporating these changes into the subprocess module, although code was given. Contributing thread: - (This was mixed in with the `About SSL tests` thread.) ------------------------ Extended buffer protocol ------------------------ Updates to the buffer protocol are discussed, along with the proposal pre-PEP documenting the updates for Python 3000, but brought to Python-Dev, due so the plans of backporting to 2.6 afterwards. (Note: I couldn't summarize this well enough to cover much of it do any real degree, but it is currently a Py3k issue, so maybe that is OK. If no one wants to add to the summary, it will stay short.) Contributing threads: - `Extended buffer PEP `__ - `PEP 3118: Extended buffer protocol (new version) `__ - `Extended Buffer Protocol - simple use examples `__ ----------------------------------------- function for counting items in a sequence ----------------------------------------- A patch was submitted by Steven Bethard (http://bugs.python.org/1696199), implemented a discussed collections.counts() function to provide a mapping between items in an iterable and the number of times they appear. There were suggested names, but none overthrew the original 'counts()' and a question of items not appearing being counted as 0 or raising a KeyError, with 0 winning due to a just-makes-sense factor. Contributing thread: - `function for counting items in a sequence `__ ---------------------------------------- context manager - generator interaction? ---------------------------------------- A problem was brought up with iterator context managers and iteration inside the with-block raising its StopIteration, but being caught by the context manager mechanics. It was also responded that the problem would not exist without the use of overly broad try blocks, and this lead to the addition of a formal note in PEP 8 about keeping narrow try blocks. Contributing thread: - `context manager - generator interaction? `__ ----------------------------- proposed which.py replacement ----------------------------- Suggestion of replacing the which.py script in the Tools directory migrated to its proposal for inclusion into the standard library. A patch and tests have yet to be provided. Contributing thread: - `proposed which.py replacement `_ ----------------------- minidom and DOM level 2 ----------------------- What is missing for DOM Level 2 support in minidom was highlighted and some work jumpstarted. Contributing thread: - `minidom and DOM level 2 `__ ---------------------------------------------- test_pty.py hangs in verbose mode on Mac OS X? ---------------------------------------------- Differing buffering behavior was causing test_pty to block only in verbose mode. Solutions may include reading to clear the buffer of child processes before a waitpid() call. Contributing thread: - `test_pty.py hangs in verbose mode on Mac OS X? `__ ================ Deferred Threads ================ ================== Previous Summaries ================== =============== Skipped Threads =============== - `Checking PEP autobuild results `__ - `Python 2.5.1c1 pickle problem `__ - `build problem on windows: unable to find getbuildinfo2.c `__ - `BaseException pickle issue `__ - `Summary of Tracker Issues `__ - `possible urllib bug on Windows XP `__ - `Py2.5.1 release candidate `__ - `Some new additions to functools `__ - `Python+XUL `__ - `Distutils and -framework on MacOSX `__ - `HTTP responses and errors `__ - `[Python-checkins] svn dead? `__ - `Just saying hello `__ - `ok to just checkin minor obvious fixes? `__ - `__lltrace__ `__ - `new subscriber looking for grunt work `__ - `functools additions `__ - `Python Documentation Problem Example `__ - `Get 2.5 changes in now, branch will be frozen soon `__ - `Quoting netiquette reminder [Re: proposed which.py replacement] `__ - `branch is frozen for release of 2.5.1c1 (and 2.5.1) `__ - `More exposure for PEP8 (was: context manager - generator interaction?) `__ - `[Python-checkins] Python Regression Test Failures opt (1) `__ - `Weekly Python Patch/Bug Summary `__ - `deprecating BaseException.message `__ - `USE_FAST code in stringobject.c `__ - `Fwd: Re: Py2.5.1 release candidate `__ - `Fwd: Re: Changes to decimal.py `__ - `svn.python.org `__ - `[PATCH] pep 0324 URL update `__ - `Pydoc Rewrite Discussion at doc-sig list. `__ - `my 2.5 checkins `__ - `Making builtins more efficient `__ - `fdopen mode needs sanitizing `__ - `Py3: function signatures, type checking, and related crap `__ - `concerns regarding callable() method `__ - `A Survey on Defect Management Practices in Free/Open Source Software `__ - `RELEASED Python 2.5.1, release candidate 1 `__ - `Python 3000 PEP: Postfix type declarations `__ - `test_socketserver flakey? `__ - `Changes to decimal.py `__ -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From jimjjewett at gmail.com Mon Apr 23 05:05:14 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Sun, 22 Apr 2007 23:05:14 -0400 Subject: [Python-Dev] PEP 30xx: Access to Module/Class/Function Currently Being Defined (this) Message-ID: (Please note that several groups were Cc'd. For now, please limit followups to python-3000. This would *probably* be backported to 2.6, but that wouldn't be decided until the implementation strategy was settled.) PEP: 30XX Title: Access to Module/Class/Function Currently Being Defined (this) Version: $Revision$ Last-Modified: $Date$ Author: Jim J. Jewett Status: Draft Type: Standards Track Content-Type: text/plain Created: 22-Apr-2007 Python-Version: 3.0 Post-History: 22-Apr-2007 Abstract It is common to need a reference to the current module, class, or function, but there is currently no entirely correct way to do this. This PEP proposes adding the keywords __module__, __class__, and __function__. Rationale Many modules export various functions, classes, and other objects, but will perform additional activities (such as running unit tests) when run as a script. The current idiom is to test whether the module's name has been set to magic value. if __name__ == "__main__": ... More complicated introspection requires a module to (attempt to) import itself. If importing the expected name actually produces a different module, there is no good workaround. Proposal: Add a __module__ keyword which refers to the module currently being defined (executed). (But see open issues.) if __module__ is sys.main: ... # assuming PEP 3020, Cannon Class methods are passed the current instance; from this they can determine self.__class__ (or cls, for classmethods). Unfortunately, this reference is to the object's actual class, which may be a subclass of the defining class. The current workaround is to repeat the name of the class, and assume that the name will not be rebound. class C(B): def meth(self): super(C, self).meth() # Hope C is never rebound. class D(C): def meth(self): super(C, self).meth() # ?!? issubclass(D,C), so it "works" Proposal: Add a __class__ keyword which refers to the class currently being defined (executed). (But see open issues.) class C(B): def meth(self): super(__class__, self).meth() Note that super calls may be further simplified by PEP 30XX, Jewett. The __class__ (or __this_class__) attribute came up in attempts to simplify the explanation and/or implementation of that PEP, but was separated out as an independent decision. Note that __class__ (or __this_class__) is not quite the same as the __thisclass__ property on bound super objects. The existing super.__thisclass__ property refers to the class from which the Method Resolution Order search begins. In the above class D, it would refer to (the current reference of name) C. Functions (including methods) often want access to themselves, usually for a private storage location. While there are several workarounds, all have their drawbacks. def counter(_total=[0]): # _total shouldn't really appear in the _total[0] += 1 # signature at all; the list wrapping and return _total[0] # [0] unwrapping obscure the code @annotate(total=0) def counter(): counter.total += 1 # Assume name counter is never rebound return counter.total class _wrap(object): # class exists only to provide storage __total=0 def f(self): self.__total += 1 return self.__total accum=_wrap().f # set module attribute to a bound method Proposal: Add a __function__ keyword which refers to the function (or method) currently being defined (executed). (But see open issues.) @annotate(total=0) def counter(): __function__.total += 1 # Always refers to this function obj return __function__.total Backwards Compatibility While a user could be using these names already, __anything__ names are explicitly reserved to the interpreter. It is therefore acceptable to introduce special meaning to these names within a single feature release. Implementation Ideally, these names would be keywords treated specially by the bytecode compiler. Guido has suggested [1] using a cell variable filled in by the metaclass. Michele Simionato has provided a prototype using bytecode hacks [2]. Open Issues - Are __module__, __class__, and __function__ the right names? In particular, should the names include the word "this", either as __this_module__, __this_class__, and __this_function__, (format discussed on the python-3000 and python-ideas lists) or as __thismodule__, __thisclass__, and __thisfunction__ (inspired by, but conflicting with, current usage of super.__thisclass__). - Are all three keywords needed, or should this enhancement be limited to a subset of the objects? Should methods be treated separately from other functions? References [1] Fixing super anyone? Guido van Rossum http://mail.python.org/pipermail/python-3000/2007-April/006671.html [2] Descriptor/Decorator challenge, Michele Simionato http://groups.google.com/group/comp.lang.python/browse_frm/thread/a6010c7494871bb1/62a2da68961caeb6?lnk=gst&q=simionato+challenge&rnum=1&hl=en#62a2da68961caeb6 Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: From jason.orendorff at gmail.com Mon Apr 23 16:02:08 2007 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Mon, 23 Apr 2007 10:02:08 -0400 Subject: [Python-Dev] Patch reviews and request Message-ID: OK, here's the patch I'd like to direct attention to: http://python.org/sf/1704134 [ 1704134 ] minidom Level 1 DOM compliance This is only the first step toward DOM Level 1 compliance. It fixes the stuff that's easy to fix. Here are the patch reviews. I put more detailed comments in the SF tracker. http://python.org/sf/1704547 [ 1704547 ] Use MoveFileEx() to implement os.rename() on windows -1. This changes the documented behavior of a commonly used function. http://python.org/sf/1678345 [ 1678345 ] A fix for the bug #1528074 [warning: quite slow] This can be rejected. http://python.org/sf/1673007 [ 1673007 ] urllib2 requests history + HEAD support urllib2.urlopen() object seems like the wrong place for history to be attached. These objects are pretty ephemeral, in my code anyway. Patch is extremely rough. -1. http://python.org/sf/1665292 [ 1665292 ] Datetime enhancements The patch here adds __int__ and __float__ to datetime.timedelta. I'm -1 on that. It also implements > < == comparison between timedelta objects and numbers, which is right out--they don't have compatible hash codes. http://python.org/sf/1652328 [ 1652328 ] stream writing support in wave.py This can be rejected. (Update: After my comments on this one, Neal closed it.) The following patches look good, but I didn't attempt to run them. I just read the source code. http://python.org/sf/1669481 [ 1669481 ] subprocess: Support close_fds on Win32 Looks good, and a definite +1. http://python.org/sf/1704621 [ 1704621 ] interpreter crash when multiplying large lists Yep, it crashes. Patch looks good. +1. http://python.org/sf/1692664 [ 1692664 ] warnings.py gets filename wrong for eval/exec Nice to have. +1. http://python.org/sf/1676135 [ 1676135 ] Remove trailing slash from --prefix Also nice to have. +0. -j From dd55 at cornell.edu Mon Apr 23 17:15:34 2007 From: dd55 at cornell.edu (Darren Dale) Date: Mon, 23 Apr 2007 11:15:34 -0400 Subject: [Python-Dev] styleguide inconsistency Message-ID: <200704231115.34245.dd55@cornell.edu> I was recently searching for some guidance on how to name packages and modules, and discovered an inconsistency in the style guides published at www.python.org. http://www.python.org/doc/essays/styleguide.html says "Module names can be either MixedCase or lowercase." That page also refers to PEP 8 at http://www.python.org/dev/peps/pep-0008/, which says "Modules should have short, all-lowercase names. ... Python packages should also have short, all-lowercase names ...". Some discussion on dev.lang.python has so far turned up the following points of view: 1) There isn't technically a contradiction because "can be" is not the same as "should be". However, since this is a style guide and not a syntax guide, I still think the documents are contradictory. 2) There isn't any confusion because the styleguide refers to the PEPs, so they have priority. However, styleguide.html does not explain that the PEPs are more up-to-date. We shouldn't expect someone to go to the PEPs after finding an answer to their question in the styleguide. Perhaps one of these documents could be revised to make the situation more clear? Thanks, Darren Dale From brett at python.org Mon Apr 23 18:25:48 2007 From: brett at python.org (Brett Cannon) Date: Mon, 23 Apr 2007 09:25:48 -0700 Subject: [Python-Dev] styleguide inconsistency In-Reply-To: <200704231115.34245.dd55@cornell.edu> References: <200704231115.34245.dd55@cornell.edu> Message-ID: On 4/23/07, Darren Dale wrote: > I was recently searching for some guidance on how to name packages and > modules, and discovered an inconsistency in the style guides published at > www.python.org. http://www.python.org/doc/essays/styleguide.html > says "Module names can be either MixedCase or lowercase." That > page also refers to PEP 8 at > http://www.python.org/dev/peps/pep-0008/, which says "Modules > should have short, all-lowercase names. ... Python packages should > also have short, all-lowercase names ...". > > Some discussion on dev.lang.python has so far turned up the following points > of view: > > 1) There isn't technically a contradiction because "can be" is not the same > as "should be". > > However, since this is a style guide and not a syntax guide, I still think the > documents are contradictory. > > 2) There isn't any confusion because the styleguide refers to the PEPs, so > they have priority. > Yes, the PEPs take priority. > However, styleguide.html does not explain that the PEPs are more up-to-date. The PEPs are more up-to-date. > We shouldn't expect someone to go to the PEPs after finding an answer to > their question in the styleguide. > > Perhaps one of these documents could be revised to make the situation more > clear? I personally think the style guide should just go and/or redirect to PEP 8. I didn't even know it existed until this email. And I only know of people updated PEP 8. -Brett From dustin at v.igoro.us Mon Apr 23 18:30:28 2007 From: dustin at v.igoro.us (dustin at v.igoro.us) Date: Mon, 23 Apr 2007 11:30:28 -0500 Subject: [Python-Dev] styleguide inconsistency In-Reply-To: References: <200704231115.34245.dd55@cornell.edu> Message-ID: <20070423163028.GQ10109@v.igoro.us> On Mon, Apr 23, 2007 at 09:25:48AM -0700, Brett Cannon wrote: > I personally think the style guide should just go and/or redirect to > PEP 8. I didn't even know it existed until this email. And I only > know of people updated PEP 8. The top of the style guide basically does that: This style guide has been converted to several PEPs (Python Enhancement Proposals): PEP 8 for the main text, PEP 257 for docstring conventions. See the PEP index. Perhaps that warning could be strengthened to suggest that the style guide is outdated and (apparently) unfinished? Beyond that, I don't see any contradiction here, and I don't see any reason to spend time updating the style guide. Dustin From guido at python.org Mon Apr 23 19:24:31 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 23 Apr 2007 10:24:31 -0700 Subject: [Python-Dev] styleguide inconsistency In-Reply-To: <20070423163028.GQ10109@v.igoro.us> References: <200704231115.34245.dd55@cornell.edu> <20070423163028.GQ10109@v.igoro.us> Message-ID: Someone should just take down that styleguide, or *replace* it with a link (and only a link) to PEP 8. I can't do that myself, it's in the hands of the webmasters. On 4/23/07, dustin at v.igoro.us wrote: > On Mon, Apr 23, 2007 at 09:25:48AM -0700, Brett Cannon wrote: > > I personally think the style guide should just go and/or redirect to > > PEP 8. I didn't even know it existed until this email. And I only > > know of people updated PEP 8. > > The top of the style guide basically does that: > > This style guide has been converted to several PEPs (Python > Enhancement Proposals): PEP 8 for the main text, PEP 257 for docstring > conventions. See the PEP index. > > Perhaps that warning could be strengthened to suggest that the style > guide is outdated and (apparently) unfinished? Beyond that, I don't see > any contradiction here, and I don't see any reason to spend time > updating the style guide. > > Dustin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Mon Apr 23 23:01:28 2007 From: brett at python.org (Brett Cannon) Date: Mon, 23 Apr 2007 14:01:28 -0700 Subject: [Python-Dev] Clarifications for import PEPs (302 and 328) Message-ID: After my import rewrite, I discovered some things that were left out of both PEP 302 (import hooks) and PEP 328 (absolute/relative imports). I wanted to run them by everyone before I made the subsequent changes to the PEPs. For PEP 302, a loader should raise ImportError if load_module fails w/o an explicit exception being raised. Also, __name__ needs to be set along with __file__ (if applicable as 'sys' does not define __file__) before the execution of code proceeds. For PEP 328, None redirection (if 'pkg.string' is set to None the import machinery is supposed to automatically return the 'string' module) should be removed when absolute/relative imports become the default. The redirection was a hack to optimize for import's old semantics of trying a relative import first and then an absolute import. The new semantics do away with the fallback and thus this mechanism is worthless. -Brett From pje at telecommunity.com Mon Apr 23 23:34:01 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 23 Apr 2007 17:34:01 -0400 Subject: [Python-Dev] Clarifications for import PEPs (302 and 328) In-Reply-To: Message-ID: <5.1.1.6.0.20070423173231.04d56e00@sparrow.telecommunity.com> At 02:01 PM 4/23/2007 -0700, Brett Cannon wrote: >For PEP 302, a loader should raise ImportError if load_module fails >w/o an explicit exception being raised. I'm not sure I understand this; could you point to the relevant part of the PEP and what you would be adding? Thanks. (The rest of your comments sounded fine, btw.) From brett at python.org Tue Apr 24 00:16:44 2007 From: brett at python.org (Brett Cannon) Date: Mon, 23 Apr 2007 15:16:44 -0700 Subject: [Python-Dev] Clarifications for import PEPs (302 and 328) In-Reply-To: <5.1.1.6.0.20070423173231.04d56e00@sparrow.telecommunity.com> References: <5.1.1.6.0.20070423173231.04d56e00@sparrow.telecommunity.com> Message-ID: On 4/23/07, Phillip J. Eby wrote: > At 02:01 PM 4/23/2007 -0700, Brett Cannon wrote: > >For PEP 302, a loader should raise ImportError if load_module fails > >w/o an explicit exception being raised. > > I'm not sure I understand this; could you point to the relevant part of the > PEP and what you would be adding? Thanks. > The PEP does not explicitly state how to signal that a loader cannot load a module it is asked to. This could happen if someone called a loader without consulting its respective importer. I would want to add something like: """ If the loader is unable to load the specified module and a specific exception is not raised in determining this, ImportError is raised. This may occur if a loader is called without first consulting an importer as to if the loader can load the specified module but the loader is aware of the fact it cannot fulfill the request made. """ > (The rest of your comments sounded fine, btw.) Great! -Brett From pje at telecommunity.com Tue Apr 24 01:21:00 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 23 Apr 2007 19:21:00 -0400 Subject: [Python-Dev] Clarifications for import PEPs (302 and 328) In-Reply-To: References: <5.1.1.6.0.20070423173231.04d56e00@sparrow.telecommunity.com> <5.1.1.6.0.20070423173231.04d56e00@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070423191509.045d8f68@sparrow.telecommunity.com> At 03:16 PM 4/23/2007 -0700, Brett Cannon wrote: >The PEP does not explicitly state how to signal that a loader cannot >load a module it is asked to. This could happen if someone called a >loader without consulting its respective importer. I would want to >add something like: >""" >If the loader is unable to load the specified module and a specific >exception is not raised in determining this, ImportError is raised. >This may occur if a loader is called without first consulting an >importer as to if the loader can load the specified module but the >loader is aware of the fact it cannot fulfill the request made. >""" Okay, now I understand what you're trying to say, but I still don't understand what the purpose is. I mean, what else would you do except raise an error? I.e., isn't the actual contract "load the specified module or raise an exception of some kind"? I mean, how about something like: "If for any reason the loader can't load the requested module, it must raise an exception (such as an ImportError), or allow an existing exception to propagate out of the load_module() call." From brett at python.org Tue Apr 24 01:23:22 2007 From: brett at python.org (Brett Cannon) Date: Mon, 23 Apr 2007 16:23:22 -0700 Subject: [Python-Dev] Clarifications for import PEPs (302 and 328) In-Reply-To: <5.1.1.6.0.20070423191509.045d8f68@sparrow.telecommunity.com> References: <5.1.1.6.0.20070423173231.04d56e00@sparrow.telecommunity.com> <5.1.1.6.0.20070423191509.045d8f68@sparrow.telecommunity.com> Message-ID: On 4/23/07, Phillip J. Eby wrote: > At 03:16 PM 4/23/2007 -0700, Brett Cannon wrote: > >The PEP does not explicitly state how to signal that a loader cannot > >load a module it is asked to. This could happen if someone called a > >loader without consulting its respective importer. I would want to > >add something like: > >""" > >If the loader is unable to load the specified module and a specific > >exception is not raised in determining this, ImportError is raised. > >This may occur if a loader is called without first consulting an > >importer as to if the loader can load the specified module but the > >loader is aware of the fact it cannot fulfill the request made. > >""" > > Okay, now I understand what you're trying to say, but I still don't > understand what the purpose is. I mean, what else would you do except > raise an error? I.e., isn't the actual contract "load the specified module > or raise an exception of some kind"? Not if you read the PEP. It doesn't explicitly say what should happen. I want to standardize on raising ImportError. The PEP itself has two ways of signaling an error; find_module() returns None instead of raising an error to signal it can't do something while path hooks raise ImportError. I just want to be as explicit as possible. -Brett From collinw at gmail.com Tue Apr 24 02:39:42 2007 From: collinw at gmail.com (Collin Winter) Date: Mon, 23 Apr 2007 17:39:42 -0700 Subject: [Python-Dev] Tweaking the stdlib test infrastructure Message-ID: <43aa6ff70704231739i383fc9e2m55129d1df8369a61@mail.gmail.com> Fast and simple: I want all stdlib test cases to stop subclassing unittest.TestCase and start subclassing test_support.TestCase. Why: With a single parent class common to the entire stdlib, adding new functionality/testing instruments to *every single test at once* becomes trivial. One example would be to move "regrtest -R"'s focus from module-level refleaks to test case-level refleaks (that is, looped execution of individual test cases, rather than test modules). I intend to implement this new refleak checking soon, if this base change goes ahead. How: At first, test_support.TestCase could be as simple as "TestCase = unittest.TestCase". Once we want to extend the test harness's capabilities, we change it to "class TestCase(unittest.TestCase): blah blah blah". All code under Lib/test/ is changed to subclass the proper class. test_support.run_unittest() can be modified to make sure that incoming classes inherit as they're supposed to. But that sounds like a lot of work: yes, but that's what we have regexes for. Most of this work can be done with a few global search-and-replace operations. So: any objections to making this change? Collin Winter From pje at telecommunity.com Tue Apr 24 03:21:46 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 23 Apr 2007 21:21:46 -0400 Subject: [Python-Dev] Clarifications for import PEPs (302 and 328) In-Reply-To: References: <5.1.1.6.0.20070423191509.045d8f68@sparrow.telecommunity.com> <5.1.1.6.0.20070423173231.04d56e00@sparrow.telecommunity.com> <5.1.1.6.0.20070423191509.045d8f68@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070423211402.04c2d158@sparrow.telecommunity.com> At 04:23 PM 4/23/2007 -0700, Brett Cannon wrote: >On 4/23/07, Phillip J. Eby wrote: > > At 03:16 PM 4/23/2007 -0700, Brett Cannon wrote: > > >The PEP does not explicitly state how to signal that a loader cannot > > >load a module it is asked to. This could happen if someone called a > > >loader without consulting its respective importer. I would want to > > >add something like: > > >""" > > >If the loader is unable to load the specified module and a specific > > >exception is not raised in determining this, ImportError is raised. > > >This may occur if a loader is called without first consulting an > > >importer as to if the loader can load the specified module but the > > >loader is aware of the fact it cannot fulfill the request made. > > >""" > > > > Okay, now I understand what you're trying to say, but I still don't > > understand what the purpose is. I mean, what else would you do except > > raise an error? I.e., isn't the actual contract "load the specified module > > or raise an exception of some kind"? > >Not if you read the PEP. It doesn't explicitly say what should happen. > >I want to standardize on raising ImportError. The PEP itself has two >ways of signaling an error; find_module() returns None instead of >raising an error to signal it can't do something while path hooks >raise ImportError. I just want to be as explicit as possible. Ok, so this should at least lose the passive voice ("ImportError is raised") -- it should say that it's the loader's job to raise ImportError. Otherwise, it's not actually removing any ambiguity. I also don't find the "This may occur" sentence to be helpful; it seems we should just say, "load_module() must return a loaded module or raise an error, preferably an ImportError unless an existing exception is being propagated." Meanwhile, if you also mean to say that load_module() is explicitly allowed to fail with an ImportError when it's asked for any module other than the one it was obtained for (via find_module()), then let's say that, too. Notice, by the way, that this is a stronger condition than the one you proposed, yet it doesn't imply that there is or should be an ongoing relationship between the loader and the importer. It also doesn't imply that perhaps it's okay for a loader to not succeed in reloading the *same* module! (i.e., as I understand it, a module can currently invoke __loader__.load_module(__name__) as a rough synonym for reload().) From nnorwitz at gmail.com Tue Apr 24 04:13:53 2007 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 23 Apr 2007 19:13:53 -0700 Subject: [Python-Dev] Tweaking the stdlib test infrastructure In-Reply-To: <43aa6ff70704231739i383fc9e2m55129d1df8369a61@mail.gmail.com> References: <43aa6ff70704231739i383fc9e2m55129d1df8369a61@mail.gmail.com> Message-ID: On 4/23/07, Collin Winter wrote: > Fast and simple: I want all stdlib test cases to stop subclassing > unittest.TestCase and start subclassing test_support.TestCase. Go for it! n From loupgaroublond at gmail.com Tue Apr 24 05:13:34 2007 From: loupgaroublond at gmail.com (Yaakov Nemoy) Date: Mon, 23 Apr 2007 23:13:34 -0400 Subject: [Python-Dev] Attachment Policy? Message-ID: <7f692fec0704232013l1980d694ubf5c74cfdcb596a9@mail.gmail.com> Hi List, i want to post a little bit of something I've been working on. Before I post it, i want to know if I'm allowed to post it as an attachment. Coming up, some work on ASTVisitor in the compiler module. Cheers, Yaakov From barry at python.org Tue Apr 24 05:34:17 2007 From: barry at python.org (Barry Warsaw) Date: Mon, 23 Apr 2007 23:34:17 -0400 Subject: [Python-Dev] Tweaking the stdlib test infrastructure In-Reply-To: <43aa6ff70704231739i383fc9e2m55129d1df8369a61@mail.gmail.com> References: <43aa6ff70704231739i383fc9e2m55129d1df8369a61@mail.gmail.com> Message-ID: <14F9B4CC-6FE7-4829-8804-663099F05A8E@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Apr 23, 2007, at 8:39 PM, Collin Winter wrote: > Fast and simple: I want all stdlib test cases to stop subclassing > unittest.TestCase and start subclassing test_support.TestCase. > > So: any objections to making this change? Please use the absolute package name test.test_support.TestCase. Remember that some of the code in the stdlib is released externally to Python (e.g. the email package), so their test suites have to be runnable in that deployment mode. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRi16v3EjvBPtnXfVAQKS4wQAiAaoIrhLhnnKxsrJL8RHZzPB9J3u98LJ EWZD7h9z0rXBEElOSDCQqZ3zlSVJNO5PGcogXPivrKE3Hg5t/pK88FCAMFNjv8iY sLK2fwpMlsGLLnIfGRbSOTxVvyEmuUEPQ5CuLUniPEMocE42bUi+7hpp5pLc2PDA U7dzYzGwBxA= =g+lH -----END PGP SIGNATURE----- From ironfroggy at gmail.com Tue Apr 24 05:51:45 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Mon, 23 Apr 2007 23:51:45 -0400 Subject: [Python-Dev] Patch to skip positional arguments with partials Message-ID: <76fd5acf0704232051l95de0ah90a959afb069319c@mail.gmail.com> I posted about this on python-ideas, and didn't get any objections about the idea itself, so I took the opportunity to dive into the C API and get my hands dirty. I posted the idea and patch as a report on SF (1706256). For anyone interested at least to look over, I'm also just including the small patch at the end of this email. Everything is passing for me with these changes, except the inability to subclass the partial type. I've found the current end of my abilities in understanding how I broke that, so I'd appreciate any reaction to the situation. Index: Modules/_functoolsmodule.c =================================================================== --- Modules/_functoolsmodule.c (revision 54922) +++ Modules/_functoolsmodule.c (working copy) @@ -21,6 +21,7 @@ } partialobject; static PyTypeObject partial_type; +static PyObject *partial_skip; static PyObject * partial_new(PyTypeObject *type, PyObject *args, PyObject *kw) @@ -89,6 +90,10 @@ { PyObject *ret; PyObject *argappl = NULL, *kwappl = NULL; + PyObject *ptoargscopy, *arg; + Py_ssize_t skip_index = 0; + Py_ssize_t pull_index = 0; + Py_ssize_t i; assert (PyCallable_Check(pto->fn)); assert (PyTuple_Check(pto->args)); @@ -101,7 +106,25 @@ argappl = pto->args; Py_INCREF(pto->args); } else { - argappl = PySequence_Concat(pto->args, args); + + // For each partial_skip in the pto args, replace it with a new arg + ptoargscopy = PyTuple_New(PyTuple_GET_SIZE(pto->args)); + for (i = 0; iargs); ++i) { + arg = PyTuple_GetItem(pto->args, i); + Py_XINCREF(arg); + PyTuple_SetItem(ptoargscopy, i, arg); + if (arg == NULL) { + break; + } else if (arg == partial_skip) { + arg = PyTuple_GetItem(args, pull_index); + Py_XINCREF(arg); + PyTuple_SetItem(ptoargscopy, i, arg); + pull_index += 1; + } + } + + arg = PyTuple_GetSlice(args, pull_index, PySequence_Length(args)); + argappl = PySequence_Concat(ptoargscopy, arg); if (argappl == NULL) return NULL; } @@ -142,7 +165,8 @@ PyDoc_STRVAR(partial_doc, "partial(func, *args, **keywords) - new function with partial application\n\ - of the given arguments and keywords.\n"); + of the given arguments and keywords. Pass partial.skip to any positional\n\ + argument to be bound to later."); #define OFF(x) offsetof(partialobject, x) static PyMemberDef partial_memberlist[] = { @@ -255,7 +280,7 @@ init_functools(void) { int i; - PyObject *m; + PyObject *m, *builtins, *skip, *object; char *name; PyTypeObject *typelist[] = { &partial_type, @@ -274,4 +299,16 @@ Py_INCREF(typelist[i]); PyModule_AddObject(m, name+1, (PyObject *)typelist[i]); } + + builtins = PyImport_ImportModule("__builtin__"); + object = PyObject_GetAttrString(builtins, "object"); + Py_DECREF(builtins); + skip = PyObject_CallObject(object, NULL, NULL); + Py_DECREF(object); + + partial_type.tp_dict = PyDict_New(); + PyDict_SetItemString(partial_type.tp_dict, "skip", skip); + partial_skip = skip; + + // skip is not decref'ed because a reference is always held by the static pointer. } -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From nnorwitz at gmail.com Tue Apr 24 06:39:28 2007 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 23 Apr 2007 21:39:28 -0700 Subject: [Python-Dev] Attachment Policy? In-Reply-To: <7f692fec0704232013l1980d694ubf5c74cfdcb596a9@mail.gmail.com> References: <7f692fec0704232013l1980d694ubf5c74cfdcb596a9@mail.gmail.com> Message-ID: On 4/23/07, Yaakov Nemoy wrote: > Hi List, > > i want to post a little bit of something I've been working on. Before > I post it, i want to know if I'm allowed to post it as an attachment. > Coming up, some work on ASTVisitor in the compiler module. Hi Yaakov. If you'd like to post patches, the appropriate place to do that is here: http://sourceforge.net/tracker/?atid=305470&group_id=5470&func=browse This way everyone can look at it, comment, and we don't lose track of your patch. Cheers, n From loupgaroublond at gmail.com Tue Apr 24 07:36:39 2007 From: loupgaroublond at gmail.com (Yaakov Nemoy) Date: Tue, 24 Apr 2007 01:36:39 -0400 Subject: [Python-Dev] Attachment Policy? In-Reply-To: References: <7f692fec0704232013l1980d694ubf5c74cfdcb596a9@mail.gmail.com> Message-ID: <7f692fec0704232236k540cb4dejcd17052c35804f4c@mail.gmail.com> On 4/24/07, Neal Norwitz wrote: > On 4/23/07, Yaakov Nemoy wrote: > > Hi List, > > > > i want to post a little bit of something I've been working on. Before > > I post it, i want to know if I'm allowed to post it as an attachment. > > Coming up, some work on ASTVisitor in the compiler module. > > Hi Yaakov. > > If you'd like to post patches, the appropriate place to do that is here: > > http://sourceforge.net/tracker/?atid=305470&group_id=5470&func=browse > > This way everyone can look at it, comment, and we don't lose track of > your patch. Done. :) http://sourceforge.net/tracker/index.php?func=detail&aid=1706323&group_id=5470&atid=305470 From walter at livinglogic.de Tue Apr 24 09:19:37 2007 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Tue, 24 Apr 2007 09:19:37 +0200 Subject: [Python-Dev] Tweaking the stdlib test infrastructure In-Reply-To: <43aa6ff70704231739i383fc9e2m55129d1df8369a61@mail.gmail.com> References: <43aa6ff70704231739i383fc9e2m55129d1df8369a61@mail.gmail.com> Message-ID: <462DAF89.9010104@livinglogic.de> Collin Winter wrote: > Fast and simple: I want all stdlib test cases to stop subclassing > unittest.TestCase and start subclassing test_support.TestCase. > > Why: With a single parent class common to the entire stdlib, adding > new functionality/testing instruments to *every single test at once* > becomes trivial. One example would be to move "regrtest -R"'s focus > from module-level refleaks to test case-level refleaks (that is, > looped execution of individual test cases, rather than test modules). > I intend to implement this new refleak checking soon, if this base > change goes ahead. > > How: At first, test_support.TestCase could be as simple as "TestCase = > unittest.TestCase". Once we want to extend the test harness's > capabilities, we change it to "class TestCase(unittest.TestCase): blah > blah blah". All code under Lib/test/ is changed to subclass the proper > class. test_support.run_unittest() can be modified to make sure that > incoming classes inherit as they're supposed to. +1 > [...] Servus, Walter From p.f.moore at gmail.com Tue Apr 24 10:24:50 2007 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 24 Apr 2007 09:24:50 +0100 Subject: [Python-Dev] Clarifications for import PEPs (302 and 328) In-Reply-To: References: <5.1.1.6.0.20070423173231.04d56e00@sparrow.telecommunity.com> <5.1.1.6.0.20070423191509.045d8f68@sparrow.telecommunity.com> Message-ID: <79990c6b0704240124q4b467cf7t9c3cec66836356af@mail.gmail.com> On 24/04/07, Brett Cannon wrote: > Not if you read the PEP. It doesn't explicitly say what should happen. > > I want to standardize on raising ImportError. The PEP itself has two > ways of signaling an error; find_module() returns None instead of > raising an error to signal it can't do something while path hooks > raise ImportError. I just want to be as explicit as possible. I'm happy with that. As far as I remember, that was the original intention - and you're right, it should be made explicit. Paul. From ironfroggy at gmail.com Tue Apr 24 16:13:00 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Tue, 24 Apr 2007 10:13:00 -0400 Subject: [Python-Dev] Python-Dev Summary Draft (April 1-15, 2007) In-Reply-To: <76fd5acf0704212118o6e50f97bn7ceadbf158f7e092@mail.gmail.com> References: <76fd5acf0704212118o6e50f97bn7ceadbf158f7e092@mail.gmail.com> Message-ID: <76fd5acf0704240713s2633c62dq1048ca62bc69b6b@mail.gmail.com> I have not gotten any replies about this. No comments, suggestions for not skipping any missed threads, or corrections. Is everyone good with this or should I give it another day or two? On 4/22/07, Calvin Spealman wrote: > There were a good number of skipped threads, but I've been out of the > loop with a lot of busy things keeping me away from such things. Now > that I'm getting back into it, I'll keep up with things again. So, let > me know if any of the skipped threads should be brought in. > Corrections and comments welcome as always. > > ============= > Announcements > ============= > > ========= > Summaries > ========= > > --------------- > About SSL tests > --------------- > > An open bug about missing SSL tests (#451607) was brought up to close or keep > open. Jean-Paul Calderone mentioned an improved testing method by spawning a > 'openssl s_server' for testing purposes. This sparked some talk about the > terminating of subprocesses in a cross-platform manner (See `Cross-Platform > Subprocess Termination`). > > Contributing thread: > > - `About SSL tests > `__ > > ------------------------------------- > Cross-Platform Subprocess Termination > ------------------------------------- > > Because os.kill only works on *nix and OS X platforms, leaving Windows > platforms uncovered. Although subprocess.TerminateProcess exists for Windows > platforms, it requires the use of handles and additional overhead for use. > Support was overall given for adding a UNIX-only signal() method and a cross- > platform terminate() method to Popen instances. Nothing was said of actually > incorporating these changes into the subprocess module, although code was > given. > > Contributing thread: > > - (This was mixed in with the `About SSL tests` thread.) > > ------------------------ > Extended buffer protocol > ------------------------ > > Updates to the buffer protocol are discussed, along with the proposal pre-PEP > documenting the updates for Python 3000, but brought to Python-Dev, due so the > plans of backporting to 2.6 afterwards. > > (Note: I couldn't summarize this well enough to cover much of it do any real > degree, but it is currently a Py3k issue, so maybe that is OK. If no one wants > to add to the summary, it will stay short.) > > Contributing threads: > > - `Extended buffer PEP > `__ > - `PEP 3118: Extended buffer protocol (new version) > `__ > - `Extended Buffer Protocol - simple use examples > `__ > > ----------------------------------------- > function for counting items in a sequence > ----------------------------------------- > > A patch was submitted by Steven Bethard (http://bugs.python.org/1696199), > implemented a discussed collections.counts() function to provide a mapping > between items in an iterable and the number of times they appear. There were > suggested names, but none overthrew the original 'counts()' and a question of > items not appearing being counted as 0 or raising a KeyError, with 0 winning > due to a just-makes-sense factor. > > Contributing thread: > > - `function for counting items in a sequence > `__ > > ---------------------------------------- > context manager - generator interaction? > ---------------------------------------- > > A problem was brought up with iterator context managers and iteration inside > the with-block raising its StopIteration, but being caught by the context > manager mechanics. It was also responded that the problem would not exist > without the use of overly broad try blocks, and this lead to the addition of > a formal note in PEP 8 about keeping narrow try blocks. > > Contributing thread: > > - `context manager - generator interaction? > `__ > > ----------------------------- > proposed which.py replacement > ----------------------------- > > Suggestion of replacing the which.py script in the Tools directory migrated to > its proposal for inclusion into the standard library. A patch and tests have > yet to be provided. > > Contributing thread: > > - `proposed which.py replacement > `_ > > ----------------------- > minidom and DOM level 2 > ----------------------- > > What is missing for DOM Level 2 support in minidom was highlighted and > some work jumpstarted. > > Contributing thread: > > - `minidom and DOM level 2 > `__ > > ---------------------------------------------- > test_pty.py hangs in verbose mode on Mac OS X? > ---------------------------------------------- > > Differing buffering behavior was causing test_pty to block only in verbose > mode. Solutions may include reading to clear the buffer of child processes > before a waitpid() call. > > Contributing thread: > > - `test_pty.py hangs in verbose mode on Mac OS X? > `__ > > > > ================ > Deferred Threads > ================ > > ================== > Previous Summaries > ================== > > =============== > Skipped Threads > =============== > > - `Checking PEP autobuild results > `__ > - `Python 2.5.1c1 pickle problem > `__ > - `build problem on windows: unable to find getbuildinfo2.c > `__ > - `BaseException pickle issue > `__ > - `Summary of Tracker Issues > `__ > - `possible urllib bug on Windows XP > `__ > - `Py2.5.1 release candidate > `__ > - `Some new additions to functools > `__ > - `Python+XUL `__ > - `Distutils and -framework on MacOSX > `__ > - `HTTP responses and errors > `__ > - `[Python-checkins] svn dead? > `__ > - `Just saying hello > `__ > - `ok to just checkin minor obvious fixes? > `__ > - `__lltrace__ `__ > - `new subscriber looking for grunt work > `__ > - `functools additions > `__ > - `Python Documentation Problem Example > `__ > - `Get 2.5 changes in now, branch will be frozen soon > `__ > - `Quoting netiquette reminder [Re: proposed which.py replacement] > `__ > - `branch is frozen for release of 2.5.1c1 (and 2.5.1) > `__ > - `More exposure for PEP8 (was: context manager - generator > interaction?) `__ > - `[Python-checkins] Python Regression Test Failures opt (1) > `__ > - `Weekly Python Patch/Bug Summary > `__ > - `deprecating BaseException.message > `__ > - `USE_FAST code in stringobject.c > `__ > - `Fwd: Re: Py2.5.1 release candidate > `__ > - `Fwd: Re: Changes to decimal.py > `__ > - `svn.python.org > `__ > - `[PATCH] pep 0324 URL update > `__ > - `Pydoc Rewrite Discussion at doc-sig list. > `__ > - `my 2.5 checkins > `__ > - `Making builtins more efficient > `__ > - `fdopen mode needs sanitizing > `__ > - `Py3: function signatures, type checking, and related crap > `__ > - `concerns regarding callable() method > `__ > - `A Survey on Defect Management Practices in Free/Open Source > Software `__ > - `RELEASED Python 2.5.1, release candidate 1 > `__ > - `Python 3000 PEP: Postfix type declarations > `__ > - `test_socketserver flakey? > `__ > - `Changes to decimal.py > `__ > > > -- > Read my blog! I depend on your acceptance of my opinion! I am interesting! > http://ironfroggy-code.blogspot.com/ > -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From glyph at divmod.com Tue Apr 24 17:31:14 2007 From: glyph at divmod.com (glyph at divmod.com) Date: Tue, 24 Apr 2007 15:31:14 -0000 Subject: [Python-Dev] Tweaking the stdlib test infrastructure In-Reply-To: <43aa6ff70704231739i383fc9e2m55129d1df8369a61@mail.gmail.com> References: <43aa6ff70704231739i383fc9e2m55129d1df8369a61@mail.gmail.com> Message-ID: <20070424153114.7769.1705000136.divmod.xquotient.2900@joule.divmod.com> On 12:39 am, collinw at gmail.com wrote: >Fast and simple: I want all stdlib test cases to stop subclassing >unittest.TestCase and start subclassing test_support.TestCase. >So: any objections to making this change? Not an objection so much as a question - if these feature additions are generally interesting (and the ones you mentioned sounded like they are) why not simply add them to unittest.TestCase itself? After all, that is in the stdlib itself already. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070424/04b3b024/attachment.html From talin at acm.org Tue Apr 24 17:53:52 2007 From: talin at acm.org (Talin) Date: Tue, 24 Apr 2007 08:53:52 -0700 Subject: [Python-Dev] Python-Dev Summary Draft (April 1-15, 2007) In-Reply-To: <76fd5acf0704240713s2633c62dq1048ca62bc69b6b@mail.gmail.com> References: <76fd5acf0704212118o6e50f97bn7ceadbf158f7e092@mail.gmail.com> <76fd5acf0704240713s2633c62dq1048ca62bc69b6b@mail.gmail.com> Message-ID: <462E2810.5030806@acm.org> Calvin Spealman wrote: > I have not gotten any replies about this. No comments, suggestions for > not skipping any missed threads, or corrections. Is everyone good with > this or should I give it another day or two? Part of the issue, for me anyway, is that many of the really "interesting" conversations have moved to Python-3000 and Python-ideas. That being said: There are a few threads in the "skipped" section that I would have liked to understand better, without having to read through all those messages, such as the various "decimal" threads. Other than that, the summaries remain very valuable. Thank you :) -- Talin From collinw at gmail.com Tue Apr 24 18:36:29 2007 From: collinw at gmail.com (Collin Winter) Date: Tue, 24 Apr 2007 09:36:29 -0700 Subject: [Python-Dev] Tweaking the stdlib test infrastructure In-Reply-To: <20070424153114.7769.1705000136.divmod.xquotient.2900@joule.divmod.com> References: <43aa6ff70704231739i383fc9e2m55129d1df8369a61@mail.gmail.com> <20070424153114.7769.1705000136.divmod.xquotient.2900@joule.divmod.com> Message-ID: <43aa6ff70704240936w50a8632bt4a9332983c74ecc1@mail.gmail.com> On 4/24/07, glyph at divmod.com wrote: > On 12:39 am, collinw at gmail.com wrote: > >Fast and simple: I want all stdlib test cases to stop subclassing > >unittest.TestCase and start subclassing test_support.TestCase. > > >So: any objections to making this change? > > Not an objection so much as a question - if these feature additions are > generally interesting (and the ones you mentioned sounded like they are) why > not simply add them to unittest.TestCase itself? After all, that is in the > stdlib itself already. Because something like per-test refleak checking is completely useless when testing pure-python code. More generally, because not everything that will be useful for the stdlib's test suite will be useful to every single third-party test suite. It's also an issue of support: I want the freedom to add functionality to the stdlib's test suite without worrying about the impact on third-party unittest code. Collin Winter From glyph at divmod.com Tue Apr 24 18:45:04 2007 From: glyph at divmod.com (glyph at divmod.com) Date: Tue, 24 Apr 2007 16:45:04 -0000 Subject: [Python-Dev] Tweaking the stdlib test infrastructure In-Reply-To: <43aa6ff70704240936w50a8632bt4a9332983c74ecc1@mail.gmail.com> References: <43aa6ff70704231739i383fc9e2m55129d1df8369a61@mail.gmail.com> <20070424153114.7769.1705000136.divmod.xquotient.2900@joule.divmod.com> <43aa6ff70704240936w50a8632bt4a9332983c74ecc1@mail.gmail.com> Message-ID: <20070424164504.7769.1225990021.divmod.xquotient.2962@joule.divmod.com> On 04:36 pm, collinw at gmail.com wrote: >On 4/24/07, glyph at divmod.com wrote: >>On 12:39 am, collinw at gmail.com wrote: >> >Fast and simple: I want all stdlib test cases to stop subclassing >> >unittest.TestCase and start subclassing test_support.TestCase. >> >> >So: any objections to making this change? >> >>Not an objection so much as a question - if these feature additions >>are >>generally interesting (and the ones you mentioned sounded like they >>are) why >>not simply add them to unittest.TestCase itself? After all, that is >>in the >>stdlib itself already. > >Because something like per-test refleak checking is completely useless >when testing pure-python code. Not everybody outside the stdlib is using the python unittest module to test pure-python code. >More generally, because not everything >that will be useful for the stdlib's test suite will be useful to >every single third-party test suite. I don't think that every single third-party test suite is using every single feature of the existing unittest module either, though. >It's also an issue of support: I >want the freedom to add functionality to the stdlib's test suite >without worrying about the impact on third-party unittest code. This makes sense to me, and I won't belabor the point. I just wanted to give a gentle reminder that if this functionality is useful to the standard library, it's likely that there are other libraries it would be useful to. Not putting it into the supported library right away makes sense, but if this functionality does prove useful, please consider making it standard. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070424/37971f35/attachment.html From brett at python.org Tue Apr 24 19:46:09 2007 From: brett at python.org (Brett Cannon) Date: Tue, 24 Apr 2007 10:46:09 -0700 Subject: [Python-Dev] Clarifications for import PEPs (302 and 328) In-Reply-To: <5.1.1.6.0.20070423211402.04c2d158@sparrow.telecommunity.com> References: <5.1.1.6.0.20070423173231.04d56e00@sparrow.telecommunity.com> <5.1.1.6.0.20070423191509.045d8f68@sparrow.telecommunity.com> <5.1.1.6.0.20070423211402.04c2d158@sparrow.telecommunity.com> Message-ID: On 4/23/07, Phillip J. Eby wrote: > At 04:23 PM 4/23/2007 -0700, Brett Cannon wrote: > >On 4/23/07, Phillip J. Eby wrote: > > > At 03:16 PM 4/23/2007 -0700, Brett Cannon wrote: > > > >The PEP does not explicitly state how to signal that a loader cannot > > > >load a module it is asked to. This could happen if someone called a > > > >loader without consulting its respective importer. I would want to > > > >add something like: > > > >""" > > > >If the loader is unable to load the specified module and a specific > > > >exception is not raised in determining this, ImportError is raised. > > > >This may occur if a loader is called without first consulting an > > > >importer as to if the loader can load the specified module but the > > > >loader is aware of the fact it cannot fulfill the request made. > > > >""" > > > > > > Okay, now I understand what you're trying to say, but I still don't > > > understand what the purpose is. I mean, what else would you do except > > > raise an error? I.e., isn't the actual contract "load the specified module > > > or raise an exception of some kind"? > > > >Not if you read the PEP. It doesn't explicitly say what should happen. > > > >I want to standardize on raising ImportError. The PEP itself has two > >ways of signaling an error; find_module() returns None instead of > >raising an error to signal it can't do something while path hooks > >raise ImportError. I just want to be as explicit as possible. > > Ok, so this should at least lose the passive voice ("ImportError is > raised") -- it should say that it's the loader's job to raise > ImportError. Otherwise, it's not actually removing any ambiguity. > Yeah, the paragraph was from the cuff. > I also don't find the "This may occur" sentence to be helpful; it seems we > should just say, "load_module() must return a loaded module or raise an > error, preferably an ImportError unless an existing exception is being > propagated." > > Meanwhile, if you also mean to say that load_module() is explicitly allowed > to fail with an ImportError when it's asked for any module other than the > one it was obtained for (via find_module()), then let's say that, too. > Yes, I do mean that. > Notice, by the way, that this is a stronger condition than the one you > proposed, yet it doesn't imply that there is or should be an ongoing > relationship between the loader and the importer. It also doesn't imply > that perhaps it's okay for a loader to not succeed in reloading the *same* > module! (i.e., as I understand it, a module can currently invoke > __loader__.load_module(__name__) as a rough synonym for reload().) Makes sense. I will make the changes when I have a working laptop again. Thanks for the input, Phillip! -Brett From grig.gheorghiu at gmail.com Tue Apr 24 19:47:04 2007 From: grig.gheorghiu at gmail.com (Grig Gheorghiu) Date: Tue, 24 Apr 2007 10:47:04 -0700 Subject: [Python-Dev] test_pty fails on Sparc Solaris 10 for trunk Message-ID: <3f09d5a00704241047w6912798asc7fb7388a909f84e@mail.gmail.com> This is happening both in the Python buildbot farm, and in the community buildbot farm. See: http://www.python.org/dev/buildbot/trunk/sparc%20solaris10%20gcc%20trunk/builds/1960/step-test/0 http://www.python.org/dev/buildbot/community/trunk/sparc%20Solaris%2010%20trunk/builds/484/step-test/0 Grig -- http://agiletesting.blogspot.com From brett at python.org Tue Apr 24 19:47:36 2007 From: brett at python.org (Brett Cannon) Date: Tue, 24 Apr 2007 10:47:36 -0700 Subject: [Python-Dev] Python-Dev Summary Draft (April 1-15, 2007) In-Reply-To: <76fd5acf0704240713s2633c62dq1048ca62bc69b6b@mail.gmail.com> References: <76fd5acf0704212118o6e50f97bn7ceadbf158f7e092@mail.gmail.com> <76fd5acf0704240713s2633c62dq1048ca62bc69b6b@mail.gmail.com> Message-ID: On 4/24/07, Calvin Spealman wrote: > I have not gotten any replies about this. No comments, suggestions for > not skipping any missed threads, or corrections. Is everyone good with > this or should I give it another day or two? > Up to you. Usually if anyone is going to reply without prodding it happens within a 24 hours (if you send out the draft on the weekend I tended to wait until Monday evening). -Brett From brett at python.org Tue Apr 24 19:51:25 2007 From: brett at python.org (Brett Cannon) Date: Tue, 24 Apr 2007 10:51:25 -0700 Subject: [Python-Dev] Tweaking the stdlib test infrastructure In-Reply-To: <43aa6ff70704231739i383fc9e2m55129d1df8369a61@mail.gmail.com> References: <43aa6ff70704231739i383fc9e2m55129d1df8369a61@mail.gmail.com> Message-ID: On 4/23/07, Collin Winter wrote: [SNIP - Adding test.test_support.TestCase] > > So: any objections to making this change? God no! I am dying for you to write your replacement for unittest and get it into the stdlib! I am tired of camelCase method names in all of my test code. -Brett From ironfroggy at gmail.com Tue Apr 24 20:00:19 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Tue, 24 Apr 2007 14:00:19 -0400 Subject: [Python-Dev] Python-Dev Summary Draft (April 1-15, 2007) In-Reply-To: <462E2810.5030806@acm.org> References: <76fd5acf0704212118o6e50f97bn7ceadbf158f7e092@mail.gmail.com> <76fd5acf0704240713s2633c62dq1048ca62bc69b6b@mail.gmail.com> <462E2810.5030806@acm.org> Message-ID: <76fd5acf0704241100i698baa09r15f5b3fadcaa3f0e@mail.gmail.com> On 4/24/07, Talin wrote: > Calvin Spealman wrote: > > I have not gotten any replies about this. No comments, suggestions for > > not skipping any missed threads, or corrections. Is everyone good with > > this or should I give it another day or two? > > Part of the issue, for me anyway, is that many of the really > "interesting" conversations have moved to Python-3000 and Python-ideas. > > That being said: There are a few threads in the "skipped" section that I > would have liked to understand better, without having to read through > all those messages, such as the various "decimal" threads. > > Other than that, the summaries remain very valuable. Thank you :) > > -- Talin > Yes, I plan to solve this by pulling them for summaries, as well. I'm not sure at the moment if its better to pull them into one summary or three. I would think one, with all the cross over that happens these days. Any opinions? -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From aahz at pythoncraft.com Tue Apr 24 20:05:52 2007 From: aahz at pythoncraft.com (Aahz) Date: Tue, 24 Apr 2007 11:05:52 -0700 Subject: [Python-Dev] Python-Dev Summary Draft (April 1-15, 2007) In-Reply-To: References: <76fd5acf0704212118o6e50f97bn7ceadbf158f7e092@mail.gmail.com> <76fd5acf0704240713s2633c62dq1048ca62bc69b6b@mail.gmail.com> Message-ID: <20070424180552.GA15845@panix.com> On Tue, Apr 24, 2007, Brett Cannon wrote: > On 4/24/07, Calvin Spealman wrote: >> >> I have not gotten any replies about this. No comments, suggestions for >> not skipping any missed threads, or corrections. Is everyone good with >> this or should I give it another day or two? > > Up to you. Usually if anyone is going to reply without prodding it > happens within a 24 hours (if you send out the draft on the weekend I > tended to wait until Monday evening). Actually, my schedule is such that I usually don't read it UNTIL the weekend, so waiting until the Monday after sending out a draft probably doesn't hurt. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "...string iteration isn't about treating strings as sequences of strings, it's about treating strings as sequences of characters. The fact that characters are also strings is the reason we have problems, but characters are strings for other good reasons." --Aahz From arigo at tunes.org Tue Apr 24 20:47:01 2007 From: arigo at tunes.org (arigo at tunes.org) Date: Tue, 24 Apr 2007 11:47:01 -0700 Subject: [Python-Dev] EuroPython 2007 - call for refereed papers Message-ID: <20070424184701.GA2267@bespin.org> Hi all, He're a reminder to submit a talk at EuroPython! Like each year, we have both the regular conference (see call at http://indico.cern.ch/conferenceCFA.py?confId=13919) and a somewhat separated Refereed Papers section. Here is the call for the latter. The deadline for both is the 18th of May. ======================================================================== EuroPython 2007 Vilnius, Lithuania 9-11 July Call for Refereed Papers http://www.europython.org/ ======================================================================== EuroPython is the only conference in the Python world that has a properly prestigious peer-reviewed forum for presenting technical and scientific papers. Such papers, with advanced and highly innovative contents, can equally well stem from academic research or industrial research. We think this is an important function for EuroPython, so we are even making some grants available to help people with travel costs. We will be happy to consider papers in subject areas including, but not necessarily limited to, the following: * Python language and implementations * Python modules (in the broadest sense) * Python extensions * Interoperation between Python and other languages / subsystems * Scientific applications of Python * Python in Education * Games * Agile Methodologies and Testing * Social Skills We are looking for Python-related scientific and technical papers of advanced, highly innovative content that present the results of original research (be it of the academic or "industrial research" kind), with proper attention to "state of the art" and previous relevant literature/results (whether such relevant previous literature is itself directly related to Python or not). We do not intend to let the specific subject area block a paper's acceptance, as long as the paper satisfies other requirements: innovative, Python-related, reflecting original research, with proper attention to previous literature. Abstracts ========= Please submit abstracts of no more than 200 words to the refereeing committee. You can send submissions no later than 18 May 2007. We shall inform you whether your paper has been selected and announce the conference schedule on the 25 May 2007. For all details regarding the submission of abstracts, please see the EuroPython website (http://www.europython.org). WARNING: Independently of their topic, all abstracts must be submitted *in the Refereed Papers track* in order to be considered by the refereeing committee! If your abstract is accepted, you must submit your corresponding paper before 29 June 2006. Last-minute changes will be accepted until the start of the conference. You should submit the paper as a PDF file, in A4 format, complete, "stand-alone", and readable on any standards-compliant PDF reader (basically, the paper must include all fonts and figures it uses, rather than using external pointers to them; by default, most PDF-preparation programs typically produce such valid "stand-alone" PDF documents). There are no strict typesetting rules. Refereeing ========== The refereeing committee, selected by Armin Rigo, will examine all abstracts and papers. The committee may consult external experts as it deems fit. Referees may suggest or require certain changes and editing in submissions, and make acceptance conditional on such changes being performed. We expect all papers to reflect the abstract as approved and reserve the right, at our discretion, to reject a paper, despite having accepted the corresponding abstract, if the paper does not substantially correspond to the approved abstract. Presentation ============ The paper must be presented at EuroPython by one or more of the authors. Presentation time will be between half an hour and an hour, including time for questions and answers, depending on each paper's details, and also on the total number of talks approved for presentation. Proceedings =========== We will publish the conference's proceedings in purely electronic form. By presenting a paper, authors agree to give the EuroPython conference non-exclusive rights to publish the paper in electronic forms (including, but not limited to, partial and total publication on web sites and/or such media as CDROM and DVD-ROM), and warrant that the papers are not infringing on the rights of any third parties. Authors retain all other intellectual property rights on their submitted abstracts and papers excepting only this non-exclusive license. Subsidised travel ================= We have funds available to subsidise travel costs for some presenters who would otherwise not be able to attend EuroPython. When submitting your abstract, please indicate if you would need such a subsidy as a precondition of being able to come and present your paper. (Yes, this possibility does exist even if you are coming from outside of Europe. Papers from people in New Zealand who can only come if their travel is subsidised, for example, would be just fine with us...). -+- Armin Rigo From ironfroggy at gmail.com Tue Apr 24 22:37:14 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Tue, 24 Apr 2007 16:37:14 -0400 Subject: [Python-Dev] Summaries and the New Lists Message-ID: <76fd5acf0704241337r3062f50o166a14875fb9faca@mail.gmail.com> Now that I should be able to actually keep up with my summary duties, I need to figure out how to tackle the changing landscape of the development lists. The old summaries were no problem, before my time. When the python-3000 list was created, nearly everything was just conceptual, floaty talk that didn't have place in the concrete world of real development conversation in python-dev. The day recently came when python-3000 got to the point of being "real" enough to warrant a third list, python-ideas, for real floaty ideas and now conversations routinely cross all three. Something might be brought up in ideas, move to 3000 to be solidified, and then to dev to discuss backporting to 2.6 or so. Obviously, we're missing out on a lot for the summaries. So, the question I pose is how would everyone like to see this resolved? As I see it, there are two things I can do. I can either summaries each list separately, and try to sort out the cross overs. Or, I can start pulling in all three development lists into all the summaries. I prefer the second option, but I want to clear with everyone else. I hope no one has a problem with getting more with the summaries from now on? If not, I'll begin with the second half of April. Along with that, I need to know if my svn rights for submitting the summaries extends to the actual summary scripts? I'll need to change them to pull in extra lists. I'm actually thinking lots of projects could use such a script, so while I'm at it I want to just generalize it a bit and put it somewhere for anyone else to use, if no one has issues with that. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From phd at phd.pp.ru Tue Apr 24 22:45:48 2007 From: phd at phd.pp.ru (Oleg Broytmann) Date: Wed, 25 Apr 2007 00:45:48 +0400 Subject: [Python-Dev] Summaries and the New Lists In-Reply-To: <76fd5acf0704241337r3062f50o166a14875fb9faca@mail.gmail.com> References: <76fd5acf0704241337r3062f50o166a14875fb9faca@mail.gmail.com> Message-ID: <20070424204548.GA14806@phd.pp.ru> On Tue, Apr 24, 2007 at 04:37:14PM -0400, Calvin Spealman wrote: > So, the question I pose is how would everyone like to see this > resolved? As I see it, there are two things I can do. I can either > summaries each list separately, and try to sort out the cross overs. > Or, I can start pulling in all three development lists into all the > summaries. I prefer the second option I prefer the second, too. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From ironfroggy at gmail.com Tue Apr 24 17:36:13 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Tue, 24 Apr 2007 11:36:13 -0400 Subject: [Python-Dev] Automatic Scope Macros Message-ID: <76fd5acf0704240836y638ad56br624016eab7e7dc32@mail.gmail.com> Could we use and add this macro to object.h? It could be a much cleaner and safer way of dealing with new references you want to clean up in the same scope. The first one will make sure to decref your new reference when you are done with it. The second one will make sure to set a borrowed reference to NULL afterward, so you can't access it after you are done with it. The second might be useless, but it seems like it might be useful for debugging and being sure you stopped using a reference when you thought you did. Anyone, please let me know if this is a dumb idea. /* Obtain a New Reference and clean it up at the end of the scope. */ #define Py_CLEANREF(_refname, _newref, _scopecode)\ (PyObject *) _refname = _newref;\ _scopecode;\ Py_XDECREF(_refname); /* Obtain a Borrowed Reference and forget about it at the end of the scope. */ #define Py_FORGETREF(_refname, _newref, _scopecode)\ (PyObject *) _refname = _newref; _scopecode;\ _refname = NULL; -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From steven.bethard at gmail.com Tue Apr 24 22:58:12 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue, 24 Apr 2007 14:58:12 -0600 Subject: [Python-Dev] Summaries and the New Lists In-Reply-To: <76fd5acf0704241337r3062f50o166a14875fb9faca@mail.gmail.com> References: <76fd5acf0704241337r3062f50o166a14875fb9faca@mail.gmail.com> Message-ID: On 4/24/07, Calvin Spealman wrote: > So, the question I pose is how would everyone like to see this > resolved? As I see it, there are two things I can do. I can either > summaries each list separately, and try to sort out the cross overs. > Or, I can start pulling in all three development lists into all the > summaries. I think putting all three into the same summaries is fine. > Along with that, I need to know if my svn rights for submitting the > summaries extends to the actual summary scripts? Feel free to modify those as you see fit -- they were really just my scripts, so I'd be amazed if changing them bothered anyone else. If you think you can make them more generally useful, that'd be great. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From g.brandl at gmx.net Tue Apr 24 22:59:35 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 24 Apr 2007 22:59:35 +0200 Subject: [Python-Dev] Automatic Scope Macros In-Reply-To: <76fd5acf0704240836y638ad56br624016eab7e7dc32@mail.gmail.com> References: <76fd5acf0704240836y638ad56br624016eab7e7dc32@mail.gmail.com> Message-ID: Calvin Spealman schrieb: > Could we use and add this macro to object.h? It could be a much > cleaner and safer way of dealing with new references you want to clean > up in the same scope. The first one will make sure to decref your new > reference when you are done with it. The second one will make sure to > set a borrowed reference to NULL afterward, so you can't access it > after you are done with it. The second might be useless, but it seems > like it might be useful for debugging and being sure you stopped using > a reference when you thought you did. > > Anyone, please let me know if this is a dumb idea. > > /* Obtain a New Reference and clean it up at the end of the scope. */ > #define Py_CLEANREF(_refname, _newref, _scopecode)\ > (PyObject *) _refname = _newref;\ > _scopecode;\ > Py_XDECREF(_refname); > > /* Obtain a Borrowed Reference and forget about it at the end of the scope. */ > #define Py_FORGETREF(_refname, _newref, _scopecode)\ > (PyObject *) _refname = _newref; > _scopecode;\ > _refname = NULL; In most cases, you will have error checking in _scopecode which DECREFs the variable and returns, or jumps to a "cleanup" label. I can't see how this would be possible with these macros. Georg From brett at python.org Tue Apr 24 23:04:40 2007 From: brett at python.org (Brett Cannon) Date: Tue, 24 Apr 2007 14:04:40 -0700 Subject: [Python-Dev] Summaries and the New Lists In-Reply-To: <76fd5acf0704241337r3062f50o166a14875fb9faca@mail.gmail.com> References: <76fd5acf0704241337r3062f50o166a14875fb9faca@mail.gmail.com> Message-ID: On 4/24/07, Calvin Spealman wrote: > Now that I should be able to actually keep up with my summary duties, > I need to figure out how to tackle the changing landscape of the > development lists. The old summaries were no problem, before my time. > When the python-3000 list was created, nearly everything was just > conceptual, floaty talk that didn't have place in the concrete world > of real development conversation in python-dev. The day recently came > when python-3000 got to the point of being "real" enough to warrant a > third list, python-ideas, for real floaty ideas and now conversations > routinely cross all three. Something might be brought up in ideas, > move to 3000 to be solidified, and then to dev to discuss backporting > to 2.6 or so. Obviously, we're missing out on a lot for the summaries. > > So, the question I pose is how would everyone like to see this > resolved? As I see it, there are two things I can do. I can either > summaries each list separately, and try to sort out the cross overs. > Or, I can start pulling in all three development lists into all the > summaries. I prefer the second option, but I want to clear with > everyone else. I hope no one has a problem with getting more with the > summaries from now on? If not, I'll begin with the second half of > April. > All in one is fine. Just be *very* wary of getting burned out. I especially would watch out for python-ideas as any random idea can end up there and just go on and on with no resolution. I think only worrying about python-dev is fine, and if you want to pull in python-3000 that's great. But I personally consider python-ideas too wild to worry about summarizing. -Brett From kristjan at ccpgames.com Tue Apr 24 23:11:53 2007 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Tue, 24 Apr 2007 21:11:53 +0000 Subject: [Python-Dev] Automatic Scope Macros In-Reply-To: References: <76fd5acf0704240836y638ad56br624016eab7e7dc32@mail.gmail.com> Message-ID: <4E9372E6B2234D4F859320D896059A9508CD57CF76@exchis.ccp.ad.local> What you really want is to use C++. However, that is a brash and immature language that is still awaiting wider acceptance and is therefore not supported. K -----Original Message----- From: python-dev-bounces+kristjan=ccpgames.com at python.org [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Georg Brandl Sent: Tuesday, April 24, 2007 21:00 To: python-dev at python.org Subject: Re: [Python-Dev] Automatic Scope Macros Calvin Spealman schrieb: > Could we use and add this macro to object.h? It could be a much > cleaner and safer way of dealing with new references you want to clean > up in the same scope. In most cases, you will have error checking in _scopecode which DECREFs the variable and returns, or jumps to a "cleanup" label. I can't see how this would be possible with these macros. From kumar.mcmillan at gmail.com Wed Apr 25 01:12:16 2007 From: kumar.mcmillan at gmail.com (Kumar McMillan) Date: Tue, 24 Apr 2007 18:12:16 -0500 Subject: [Python-Dev] Deallocation of a pointer not malloced, any tips? In-Reply-To: <20070420085606.637A.JCARLSON@uci.edu> References: <20070420085606.637A.JCARLSON@uci.edu> Message-ID: On 4/20/07, Josiah Carlson wrote: > You may want the python-list mailing list or the equivalent > comp.lang.python newsgroup, unless this is a bug with Python itself (you > may try running Python 2.4.4, which is the next bugfix of the Python 2.4 > series). yes, seems like it's either a bug in python or the mac os x malloc library. > > Regardless, you can help those who want to help you by providing the > code that you have written that causes the error. A doubly nested > try/except isn't all that exciting, so I would guess many would be > surprised that you are getting double free errors. again, sorry for being vague. i tried my best at reproducing it in a test case but couldn't. However, I finally experimented with trying to force deallocation within the block of code and got something that worked! Here is a very small portion of it, with the fix, which was putting a ref to the exc_info vars outside the tries and specifically del'ing them in the finally block: def load_or_rescue(self, load): etype, val, tb = None, None, None try: try: try: load() except Exception, exc: etype, val, tb = sys.exc_info() if not self.rescue_load((etype, val, tb), exc): log.critical("EXCEPTION %s while processing %s", etype, self.ext_event) raise exc, None, tb except NonFatalError, e: self.handle_non_fatal_error(e) finally: del etype, val, tb to put this in perspective a little more, the warning/crash happens when the code is getting called more or less like this... def load(): self.start_file(fileobj) for row in self.file_reader(fileobj): self.load_or_rescue(lambda: self.load_row(row)) self.load_or_rescue(load) From ffesti at redhat.com Mon Apr 23 14:08:22 2007 From: ffesti at redhat.com (Florian Festi) Date: Mon, 23 Apr 2007 14:08:22 +0200 Subject: [Python-Dev] gzip Patch Message-ID: <462CA1B6.5040102@redhat.com> Hi! I posted patch 1675951 a while ago that fixes a performance problem for small reads in the gzip stdlib module. It also removes the necessity for seeking while reading gzip files (allows reading from stdin now). Is there anything I can/have to do to get the patch in? tia Florian Festi From turnbull at sk.tsukuba.ac.jp Wed Apr 25 04:16:37 2007 From: turnbull at sk.tsukuba.ac.jp (Stephen J. Turnbull) Date: Wed, 25 Apr 2007 11:16:37 +0900 Subject: [Python-Dev] Summaries and the New Lists In-Reply-To: References: <76fd5acf0704241337r3062f50o166a14875fb9faca@mail.gmail.com> Message-ID: <87vefljj0a.fsf@uwakimon.sk.tsukuba.ac.jp> Brett Cannon writes: > All in one is fine. Just be *very* wary of getting burned out. I > especially would watch out for python-ideas as any random idea can end > up there and just go on and on with no resolution. As basically a lurker, I second that -- these summaries (and the weekly tracker summary) are great, and anything that keeps the volunteers happy and healthy is really important. Similarly, as an irregular reader, I would prefer all-in-one to the extent that you do summarize more than one list. However, it would be nice if the thread summaries included a list of thread subjects occurring on any lists that don't get a verbose summary. For example, something automatic based on Subject header would be fine, any extra checking for topic changes that don't get a subject change or subject changes that don't correspond to a thread break is above and beyond the call of the volunteer spirit. Oh, and as part of the boilerplate call for comments on the summaries, suggest that people who care about a particular thread that isn't represented due to topic-Subject skew to suggest an entry for it (one line maximum). From bioinformed at gmail.com Wed Apr 25 05:32:57 2007 From: bioinformed at gmail.com (Kevin Jacobs ) Date: Tue, 24 Apr 2007 23:32:57 -0400 Subject: [Python-Dev] gzip Patch In-Reply-To: <462CA1B6.5040102@redhat.com> References: <462CA1B6.5040102@redhat.com> Message-ID: <2e1434c10704242032r7a881d9fj62dc9592e13f80b7@mail.gmail.com> I've tested the patch and it works as advertised for me. On 4/23/07, Florian Festi wrote: > > Hi! > > I posted patch 1675951 a while ago that fixes a performance problem for > small reads in the gzip stdlib module. It also removes the necessity for > seeking while reading gzip files (allows reading from stdin now). > > Is there anything I can/have to do to get the patch in? > > tia > > Florian Festi > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/jacobs%40bioinformed.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070424/0a80bb13/attachment.htm From nnorwitz at gmail.com Wed Apr 25 06:58:35 2007 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 24 Apr 2007 21:58:35 -0700 Subject: [Python-Dev] Deallocation of a pointer not malloced, any tips? In-Reply-To: References: <20070420085606.637A.JCARLSON@uci.edu> Message-ID: On 4/24/07, Kumar McMillan wrote: > > Here is a very small portion of it, with the fix, which was putting a > ref to the exc_info vars outside the tries and specifically del'ing > them in the finally block: Kumar, Can you create a completely self-contained code example that triggers the problem and post a bug to Source Forge? http://sourceforge.net/projects/python In the bug, please note the operating system version, how long this takes to trigger, the error message you see and all the steps necessary to reproduce it. Attach the code as a file, so we can download and test our the problem. Thanks, n From martin at v.loewis.de Wed Apr 25 07:06:38 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 25 Apr 2007 07:06:38 +0200 Subject: [Python-Dev] gzip Patch In-Reply-To: <462CA1B6.5040102@redhat.com> References: <462CA1B6.5040102@redhat.com> Message-ID: <462EE1DE.40703@v.loewis.de> > I posted patch 1675951 a while ago that fixes a performance problem for > small reads in the gzip stdlib module. It also removes the necessity for > seeking while reading gzip files (allows reading from stdin now). > > Is there anything I can/have to do to get the patch in? If you want to prioritize processing over other patches, please review 5 other patches, then post the list of those patches to python-dev, and I will give your patch priority. Regards, Martin From nnorwitz at gmail.com Wed Apr 25 08:52:55 2007 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 24 Apr 2007 23:52:55 -0700 Subject: [Python-Dev] whitespace normalization Message-ID: I just checked in a whitespace normalization change that was way too big. Should this task be automated? Assuming the answer is yes, these are the questions should be answered: 1) Which branches should this occur on: trunk, 2.5 (last release), 3k 2) Should there be a special user for these checkins 3) How often should this be done 4) Are there any other tasks that can be automated My preferences are: 1) all active branches (ie, trunk and currently 2.5 and p3yk) 2) Yes, there should be a special user 3) Once per week 4) Not that I know of The way to fix the files is to run: python ./Tools/scripts/reindent.py -r Lib At least that's what I did. Hopefully I didn't screw anything up. :-) n From g.brandl at gmx.net Wed Apr 25 09:16:18 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 25 Apr 2007 09:16:18 +0200 Subject: [Python-Dev] whitespace normalization In-Reply-To: References: Message-ID: Neal Norwitz schrieb: > I just checked in a whitespace normalization change that was way too > big. Should this task be automated? > > Assuming the answer is yes, these are the questions should be answered: > 1) Which branches should this occur on: trunk, 2.5 (last release), 3k > 2) Should there be a special user for these checkins Absolutely, since that will be very helpful when doing "svn praise". > 3) How often should this be done Weekly makes sense. Of course, we could also setup a svn pre-commit hook that rejects trailing whitespace >:-> > 4) Are there any other tasks that can be automated > > My preferences are: > 1) all active branches (ie, trunk and currently 2.5 and p3yk) > 2) Yes, there should be a special user > 3) Once per week > 4) Not that I know of > > The way to fix the files is to run: python ./Tools/scripts/reindent.py -r Lib > At least that's what I did. Hopefully I didn't screw anything up. :-) I looked over it, and everything seems alright ;) Though I was impressed that we had some tab- and 2 space-indents lying around. Georg From duncan.booth at suttoncourtenay.org.uk Wed Apr 25 09:40:22 2007 From: duncan.booth at suttoncourtenay.org.uk (Duncan Booth) Date: Wed, 25 Apr 2007 08:40:22 +0100 Subject: [Python-Dev] whitespace normalization References: Message-ID: "Neal Norwitz" wrote in news:ee2a432c0704242352w5a489a98sd1520427737dda6c at mail.gmail.com: > I just checked in a whitespace normalization change that was way too > big. Should this task be automated? IMHO, changing whitespace retrospectively in a version control system is a bad idea. How much overhead would it be to have a checkin hook which compares each modified file against the output of running reindent.py over the same file and rejects the checkin if it changes anything? (With of course an appropriate message suggesting the use of Reindent.py before reatttempting the checkin). That way the whitespace ought to stay normalized so you shouldn't need a separate cleanup step and you won't be breaking diff and blame for the sources (and if the reindent does ever break anything it should be more tracable). From arigo at tunes.org Tue Apr 24 11:32:39 2007 From: arigo at tunes.org (Armin Rigo) Date: Tue, 24 Apr 2007 11:32:39 +0200 Subject: [Python-Dev] new metaclass error condition checks In-Reply-To: <20070416225320.19381.297920264.divmod.quotient.2282@ohm> References: <20070416225320.19381.297920264.divmod.quotient.2282@ohm> Message-ID: <20070424093239.GA23574@code0.codespeak.net> Hi Jean-Paul, On Mon, Apr 16, 2007 at 06:53:20PM -0400, Jean-Paul Calderone wrote: > I just noticed r53997 (from some unit tests it broke), which disallowed things > like this: > > class X(object): > def __repr__(self): > return "blah" > > class Y(X, type): > pass > > class Z: > __metaclass__ = Y This is now valid again: I reverted r53997, as I got no answer to http://mail.python.org/pipermail/python-dev/2007-March/071796.html . A bientot, Armin. From steve at holdenweb.com Wed Apr 25 15:39:26 2007 From: steve at holdenweb.com (Steve Holden) Date: Wed, 25 Apr 2007 09:39:26 -0400 Subject: [Python-Dev] whitespace normalization In-Reply-To: References: Message-ID: Duncan Booth wrote: > "Neal Norwitz" wrote in > news:ee2a432c0704242352w5a489a98sd1520427737dda6c at mail.gmail.com: > >> I just checked in a whitespace normalization change that was way too >> big. Should this task be automated? > > IMHO, changing whitespace retrospectively in a version control system is a > bad idea. > > How much overhead would it be to have a checkin hook which compares each > modified file against the output of running reindent.py over the same file > and rejects the checkin if it changes anything? (With of course an > appropriate message suggesting the use of Reindent.py before reatttempting > the checkin). > > That way the whitespace ought to stay normalized so you shouldn't need a > separate cleanup step and you won't be breaking diff and blame for the > sources (and if the reindent does ever break anything it should be more > tracable). +1 Enforcing whitespace correctness on checkin has the added advantage that we will be able to screw another 1% out of uncle Timmy, who will no longer have to make his repeated whitespace correction checkins and will therefore have time for more productive tasks. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://del.icio.us/steve.holden Recent Ramblings http://holdenweb.blogspot.com From fdrake at acm.org Wed Apr 25 16:16:13 2007 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Wed, 25 Apr 2007 10:16:13 -0400 Subject: [Python-Dev] whitespace normalization In-Reply-To: References: Message-ID: <200704251016.13354.fdrake@acm.org> On Wednesday 25 April 2007, Steve Holden wrote: > Duncan Booth wrote: > > That way the whitespace ought to stay normalized so you shouldn't need a > > separate cleanup step and you won't be breaking diff and blame for the > > sources (and if the reindent does ever break anything it should be more > > tracable). > > +1 +1 here as well; there's no need to let things like this get out-of-sync from what we want. -Fred -- Fred L. Drake, Jr. From dustin at v.igoro.us Wed Apr 25 16:54:43 2007 From: dustin at v.igoro.us (dustin at v.igoro.us) Date: Wed, 25 Apr 2007 09:54:43 -0500 Subject: [Python-Dev] whitespace normalization In-Reply-To: References: Message-ID: <20070425145443.GD2332@v.igoro.us> On Wed, Apr 25, 2007 at 08:40:22AM +0100, Duncan Booth wrote: > IMHO, changing whitespace retrospectively in a version control system is a > bad idea. In my experience Duncan's assertion is absolutely true, and all the more so in a project that maintains a large body of pending patches, as Python does. Spurious whitespaces changes in the repo will generate patch rejections that will drive both maintainers (as they check in a patch) and submitters (as they try to refresh patches against head) mad. I'm very much interested in VC systems *not* driving developers mad! Dustin From facundo at taniquetil.com.ar Wed Apr 25 18:53:54 2007 From: facundo at taniquetil.com.ar (Facundo Batista) Date: Wed, 25 Apr 2007 16:53:54 +0000 (UTC) Subject: [Python-Dev] whitespace normalization References: Message-ID: Georg Brandl wrote: > Of course, we could also setup a svn pre-commit hook that rejects trailing > whitespace >:-> I was about to suggest a mail to python-checkins when a bad whitespace is detected, but had no idea that a pre-commit check existed in SVN. It'd be great if the system won't let you commit if whitespace is bad. Regards, -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From draghuram at gmail.com Wed Apr 25 20:11:13 2007 From: draghuram at gmail.com (Raghuram Devarakonda) Date: Wed, 25 Apr 2007 14:11:13 -0400 Subject: [Python-Dev] some bug reviews Message-ID: <2c51ecee0704251111r12f15a99lf69d1986da5d8dfa@mail.gmail.com> Hi, The following two bugs can be closed with possible document update. I have put my suggestions for doc changes in the SF tracker. ---------- http://python.org/sf/1615275 - tempile.TemporaryFile differences between linux and windows http://python.org/sf/1633941 - for line in sys.stdin: doesn't notice EOF the first time I opened 1643712 to request doc change for this bug. ----------- http://python.org/sf/1668596 - distutils chops the first character of filenames This one requires no action and can be closed, IMO. Thanks, Raghu From g.brandl at gmx.net Wed Apr 25 20:17:40 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 25 Apr 2007 20:17:40 +0200 Subject: [Python-Dev] whitespace normalization In-Reply-To: References: Message-ID: Facundo Batista schrieb: > Georg Brandl wrote: > >> Of course, we could also setup a svn pre-commit hook that rejects trailing >> whitespace >:-> > > I was about to suggest a mail to python-checkins when a bad whitespace > is detected, but had no idea that a pre-commit check existed in SVN. > > It'd be great if the system won't let you commit if whitespace is bad. Well, there are editors that don't intelligently strip whitespace, so that people using them would be constantly pained by such a hook. Things like two-space indents or tabs, however, should really be rejected IMO. Stripping the trailing whitespace automatically on checkin would also be fine, but I don't think that kind of thing is possible with SVN (also, it would mean that your WC is out of date after checkin). Geog -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Wed Apr 25 20:19:26 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 25 Apr 2007 20:19:26 +0200 Subject: [Python-Dev] whitespace normalization In-Reply-To: References: Message-ID: Steve Holden schrieb: > Duncan Booth wrote: >> "Neal Norwitz" wrote in >> news:ee2a432c0704242352w5a489a98sd1520427737dda6c at mail.gmail.com: >> >>> I just checked in a whitespace normalization change that was way too >>> big. Should this task be automated? >> >> IMHO, changing whitespace retrospectively in a version control system is a >> bad idea. >> >> How much overhead would it be to have a checkin hook which compares each >> modified file against the output of running reindent.py over the same file >> and rejects the checkin if it changes anything? (With of course an >> appropriate message suggesting the use of Reindent.py before reatttempting >> the checkin). >> >> That way the whitespace ought to stay normalized so you shouldn't need a >> separate cleanup step and you won't be breaking diff and blame for the >> sources (and if the reindent does ever break anything it should be more >> tracable). > > +1 > > Enforcing whitespace correctness on checkin has the added advantage that > we will be able to screw another 1% out of uncle Timmy, who will no > longer have to make his repeated whitespace correction checkins and will > therefore have time for more productive tasks. As I said, this might be a pain for users of editors which don't strip indentation whitespace intelligently; e.g. if you write def foo(): print 1 print 2 there might be 4 spaces left on line 3. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From guido at python.org Wed Apr 25 20:24:37 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Apr 2007 11:24:37 -0700 Subject: [Python-Dev] whitespace normalization In-Reply-To: References: Message-ID: On 4/25/07, Georg Brandl wrote: > Well, there are editors that don't intelligently strip whitespace, so that > people using them would be constantly pained by such a hook. And they should. There really is no excuse for letting one developer's poor choice of tools cause later grief for all other developers. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.peters at gmail.com Wed Apr 25 20:50:26 2007 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 25 Apr 2007 14:50:26 -0400 Subject: [Python-Dev] whitespace normalization In-Reply-To: References: Message-ID: <1f7befae0704251150q5122b45dg3add99620663054@mail.gmail.com> [Neal Norwitz] > ... > The way to fix the files is to run: python ./Tools/scripts/reindent.py -r Lib I apply it to everything in the checkout. That is, I run reindent.py from the root of my sandbox, and use "." instead of "Lib". The goal is that every .py file (not just under Lib/) that eventually shows up in a release use whitespace consistently. > At least that's what I did. Hopefully I didn't screw anything up. :-) reindent.py has never been blamed for a "legitimate" screwup. On rare occasions it has broken tests, but only when the code mysteriously relied on significant trailing whitespace in the .py file. Such invisible dependence is considered (by me :-)) to be a bug in the code, not in reindent.py. The other no-brainer is to run Tools/scripts/svneol.py regularly. That finds text files that don't have the svn:eol-style property set, and sets it. From martin at v.loewis.de Wed Apr 25 21:03:41 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 25 Apr 2007 21:03:41 +0200 Subject: [Python-Dev] whitespace normalization In-Reply-To: References: Message-ID: <462FA60D.9010408@v.loewis.de> Neal Norwitz schrieb: > I just checked in a whitespace normalization change that was way too > big. Should this task be automated? > > Assuming the answer is yes, these are the questions should be answered: > 1) Which branches should this occur on: trunk, 2.5 (last release), 3k > 2) Should there be a special user for these checkins What *really* should be there is a pre-commit hook that rejects a checkin if the violates the formatting style in force. Regards, Martin From martin at v.loewis.de Wed Apr 25 21:05:42 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 25 Apr 2007 21:05:42 +0200 Subject: [Python-Dev] whitespace normalization In-Reply-To: References: Message-ID: <462FA686.20102@v.loewis.de> > Well, there are editors that don't intelligently strip whitespace, so that > people using them would be constantly pained by such a hook. Those users can run reindent.py before checking in, or switch editors. Regards, Martin From exarkun at divmod.com Wed Apr 25 21:07:09 2007 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Wed, 25 Apr 2007 15:07:09 -0400 Subject: [Python-Dev] object.__init__ In-Reply-To: 0 Message-ID: <20070425190709.19381.57831227.divmod.quotient.5321@ohm> As a data point, I thought I'd point out that the recent object.__init__ change broke a handful of Twisted unit tests. The fix for this was simple, and I've already implemented it, but it would have been nice if the old behavior had been deprecated first and then removed, instead of just disappearing. Jean-Paul From g.brandl at gmx.net Wed Apr 25 21:19:45 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 25 Apr 2007 21:19:45 +0200 Subject: [Python-Dev] whitespace normalization In-Reply-To: References: Message-ID: Guido van Rossum schrieb: > On 4/25/07, Georg Brandl wrote: >> Well, there are editors that don't intelligently strip whitespace, so that >> people using them would be constantly pained by such a hook. > > And they should. There really is no excuse for letting one developer's > poor choice of tools cause later grief for all other developers. Okay, attached is a pre-commit hook script for that purpose. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: pre-commit Url: http://mail.python.org/pipermail/python-dev/attachments/20070425/eefcc9b7/attachment.asc From g.brandl at gmx.net Wed Apr 25 21:28:10 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 25 Apr 2007 21:28:10 +0200 Subject: [Python-Dev] 2.5 branch unfrozen In-Reply-To: <20070421115709.GA25234@axis.g33x.de> References: <200704211645.39875.anthony@interlink.com.au> <20070421115709.GA25234@axis.g33x.de> Message-ID: Lars Gust?bel schrieb: > On Sat, Apr 21, 2007 at 04:45:37PM +1000, Anthony Baxter wrote: >> Ok, things seem to be OK. So the release25-maint branch is unfrozen. >> Go crazy. Well, a little bit crazy. > > I'm afraid that I went crazy a little too early. Sorry for that. > Won't happen again. BTW, svn provides a "lock" mechanism by which the branch freezing could be enforced more strictly... Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From exarkun at divmod.com Wed Apr 25 21:35:17 2007 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Wed, 25 Apr 2007 15:35:17 -0400 Subject: [Python-Dev] 2.5 branch unfrozen In-Reply-To: Message-ID: <20070425193517.19381.1555166373.divmod.quotient.5329@ohm> On Wed, 25 Apr 2007 21:28:10 +0200, Georg Brandl wrote: >Lars Gust?bel schrieb: >> On Sat, Apr 21, 2007 at 04:45:37PM +1000, Anthony Baxter wrote: >>> Ok, things seem to be OK. So the release25-maint branch is unfrozen. >>> Go crazy. Well, a little bit crazy. >> >> I'm afraid that I went crazy a little too early. Sorry for that. >> Won't happen again. > >BTW, svn provides a "lock" mechanism by which the branch freezing could >be enforced more strictly... It doesn't work on directories, though. Jean-Paul From guido at python.org Wed Apr 25 21:43:23 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Apr 2007 12:43:23 -0700 Subject: [Python-Dev] object.__init__ In-Reply-To: <20070425190709.19381.57831227.divmod.quotient.5321@ohm> References: <20070425190709.19381.57831227.divmod.quotient.5321@ohm> Message-ID: 2.6 isn't even in alpha. It's not too late to submit a patch that adds the deprecation; I'd be totally fine with that. On 4/25/07, Jean-Paul Calderone wrote: > > As a data point, I thought I'd point out that the recent object.__init__ change > broke a handful of Twisted unit tests. The fix for this was simple, and I've > already implemented it, but it would have been nice if the old behavior had been > deprecated first and then removed, instead of just disappearing. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From g.brandl at gmx.net Wed Apr 25 21:43:37 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 25 Apr 2007 21:43:37 +0200 Subject: [Python-Dev] 2.5 branch unfrozen In-Reply-To: <20070425193517.19381.1555166373.divmod.quotient.5329@ohm> References: <20070425193517.19381.1555166373.divmod.quotient.5329@ohm> Message-ID: Jean-Paul Calderone schrieb: > On Wed, 25 Apr 2007 21:28:10 +0200, Georg Brandl wrote: >>Lars Gust?bel schrieb: >>> On Sat, Apr 21, 2007 at 04:45:37PM +1000, Anthony Baxter wrote: >>>> Ok, things seem to be OK. So the release25-maint branch is unfrozen. >>>> Go crazy. Well, a little bit crazy. >>> >>> I'm afraid that I went crazy a little too early. Sorry for that. >>> Won't happen again. >> >>BTW, svn provides a "lock" mechanism by which the branch freezing could >>be enforced more strictly... > > It doesn't work on directories, though. Oh, didn't realize that. Thanks. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From skip at pobox.com Wed Apr 25 20:37:41 2007 From: skip at pobox.com (skip at pobox.com) Date: Wed, 25 Apr 2007 13:37:41 -0500 Subject: [Python-Dev] whitespace normalization In-Reply-To: References: Message-ID: <17967.40949.778426.208213@montanaro.dyndns.org> >> Well, there are editors that don't intelligently strip whitespace, so >> that people using them would be constantly pained by such a hook. Guido> And they should. There really is no excuse for letting one Guido> developer's poor choice of tools cause later grief for all other Guido> developers. Just a little FYI, python-mode (the one Barry and I manage - dunno about the one distributed w/ GNU Emacs these days) is one of those tools that leaves trailing whitespace behind when advancing to the next line.. Skip From skip at pobox.com Wed Apr 25 21:14:13 2007 From: skip at pobox.com (skip at pobox.com) Date: Wed, 25 Apr 2007 14:14:13 -0500 Subject: [Python-Dev] whitespace normalization In-Reply-To: <1f7befae0704251150q5122b45dg3add99620663054@mail.gmail.com> References: <1f7befae0704251150q5122b45dg3add99620663054@mail.gmail.com> Message-ID: <17967.43141.628718.803881@montanaro.dyndns.org> Tim> ... but only when the code mysteriously Tim> relied on significant trailing whitespace in the .py file. Maybe \s should expand to a single space by the lexer so people who want to rely on trailing spaces can be explicit about it. There already exists non-whitespace escape sequences for tabs and newlines. Skip From skip at pobox.com Tue Apr 24 05:26:55 2007 From: skip at pobox.com (skip at pobox.com) Date: Mon, 23 Apr 2007 22:26:55 -0500 Subject: [Python-Dev] Attachment Policy? In-Reply-To: <7f692fec0704232013l1980d694ubf5c74cfdcb596a9@mail.gmail.com> References: <7f692fec0704232013l1980d694ubf5c74cfdcb596a9@mail.gmail.com> Message-ID: <17965.30975.493972.176877@montanaro.dyndns.org> Yaakov> i want to post a little bit of something I've been working on. Yaakov> Before I post it, i want to know if I'm allowed to post it as an Yaakov> attachment. Coming up, some work on ASTVisitor in the compiler Yaakov> module. If you're proposing that your change be included in Python please create a patch at SourceForge so it doesn't get lost: http://sourceforge.net/tracker/?group_id=5470&atid=305470 Skip From tim.peters at gmail.com Wed Apr 25 22:00:04 2007 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 25 Apr 2007 16:00:04 -0400 Subject: [Python-Dev] whitespace normalization In-Reply-To: <17967.43141.628718.803881@montanaro.dyndns.org> References: <1f7befae0704251150q5122b45dg3add99620663054@mail.gmail.com> <17967.43141.628718.803881@montanaro.dyndns.org> Message-ID: <1f7befae0704251300m98b5538x19465565c8511d7c@mail.gmail.com> [Skip] > Maybe \s should expand to a single space by the lexer so people who want to > rely on trailing spaces can be explicit about it. There already exists > non-whitespace escape sequences for tabs and newlines. Trailing whitspace is never significant on a code line, only inside a multiline string literal. In the extremely rare cases where one of those requires trailing spaces, \x20 works fine. From guido at python.org Wed Apr 25 22:10:22 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Apr 2007 13:10:22 -0700 Subject: [Python-Dev] whitespace normalization In-Reply-To: <17967.40949.778426.208213@montanaro.dyndns.org> References: <17967.40949.778426.208213@montanaro.dyndns.org> Message-ID: On 4/25/07, skip at pobox.com wrote: > >> Well, there are editors that don't intelligently strip whitespace, so > >> that people using them would be constantly pained by such a hook. > > Guido> And they should. There really is no excuse for letting one > Guido> developer's poor choice of tools cause later grief for all other > Guido> developers. > > Just a little FYI, python-mode (the one Barry and I manage - dunno about the > one distributed w/ GNU Emacs these days) is one of those tools that leaves > trailing whitespace behind when advancing to the next line.. Funny -- that's not my experience. Perhaps this depends on a separate Emacs setting? That's what I always assumed. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From skip at pobox.com Wed Apr 25 22:11:07 2007 From: skip at pobox.com (skip at pobox.com) Date: Wed, 25 Apr 2007 15:11:07 -0500 Subject: [Python-Dev] whitespace normalization In-Reply-To: <17967.40949.778426.208213@montanaro.dyndns.org> References: <17967.40949.778426.208213@montanaro.dyndns.org> Message-ID: <17967.46555.279633.320817@montanaro.dyndns.org> skip> Just a little FYI, python-mode (the one Barry and I manage - dunno skip> about the one distributed w/ GNU Emacs these days) is one of those skip> tools that leaves trailing whitespace behind when advancing to the skip> next line.. At least so I thiought. I know I've seen these turds before, but a quick check wouldn't reproduce it... Skip From g.brandl at gmx.net Wed Apr 25 22:14:50 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 25 Apr 2007 22:14:50 +0200 Subject: [Python-Dev] whitespace normalization In-Reply-To: <17967.46555.279633.320817@montanaro.dyndns.org> References: <17967.40949.778426.208213@montanaro.dyndns.org> <17967.46555.279633.320817@montanaro.dyndns.org> Message-ID: skip at pobox.com schrieb: > skip> Just a little FYI, python-mode (the one Barry and I manage - dunno > skip> about the one distributed w/ GNU Emacs these days) is one of those > skip> tools that leaves trailing whitespace behind when advancing to the > skip> next line.. > > At least so I thiought. I know I've seen these turds before, but a quick > check wouldn't reproduce it... This isn't my experience either. Also, there's always show-trailing-whitespace and trailing-whitespace-face... Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From barry at python.org Wed Apr 25 22:15:07 2007 From: barry at python.org (Barry Warsaw) Date: Wed, 25 Apr 2007 16:15:07 -0400 Subject: [Python-Dev] whitespace normalization In-Reply-To: <17967.40949.778426.208213@montanaro.dyndns.org> References: <17967.40949.778426.208213@montanaro.dyndns.org> Message-ID: <84585667-F74D-4001-92B4-D57FD1D76159@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Apr 25, 2007, at 2:37 PM, skip at pobox.com wrote: >>> Well, there are editors that don't intelligently strip >>> whitespace, so >>> that people using them would be constantly pained by such a hook. > > Guido> And they should. There really is no excuse for letting one > Guido> developer's poor choice of tools cause later grief for > all other > Guido> developers. > > Just a little FYI, python-mode (the one Barry and I manage - dunno > about the > one distributed w/ GNU Emacs these days) is one of those tools that > leaves > trailing whitespace behind when advancing to the next line.. Maybe we should include this function in the mode: ;; untabify and clean up lines with just whitespace (defun baw-whitespace-normalization () "Like untabify, but also cleans up lines with trailing whitespace." (interactive) (save-excursion (save-restriction (untabify (point-min) (point-max)) (goto-char (point-min)) (while (re-search-forward "[ \t]+$" nil t) (let ((bol (save-excursion (beginning-of-line) (point))) (eol (save-excursion (end-of-line) (point)))) (goto-char (match-beginning 0)) (if (and (bolp) (eq (char-after) ?\ )) (forward-char 1)) (skip-chars-backward " \t" bol) (delete-region (point) eol) )) ;; Now clean up any trailing blank lines (goto-char (point-max)) (skip-chars-backward " \t\n") (if (not (bolp)) (forward-char 1)) (delete-region (point) (point-max)) ;; But make sure the files ends in a newline (if (not (bolp)) (newline)) ))) (defalias 'baw-normalize-whitespace 'baw-whitespace-normalization) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRi+2y3EjvBPtnXfVAQKyCAP5AUwHxTgOCRkkgCRfAP/oWCxf1WT/Y1lk PUrb4eRaR2s5B8XP417V3O/uMPHfiQr9Ci1txoIuq22PRbPvOFT4bfSYXFxPrrGS FOlvivpE8rnXSxv4bJC/3B11GNDoyQJds8D4/13XIZXnFE4W2WtyA0fG1kxQXBEc gvjk/FXuRS8= =3Wll -----END PGP SIGNATURE----- From tjreedy at udel.edu Wed Apr 25 23:00:42 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 25 Apr 2007 17:00:42 -0400 Subject: [Python-Dev] 2.5 branch unfrozen References: <20070425193517.19381.1555166373.divmod.quotient.5329@ohm> Message-ID: "Georg Brandl" wrote in message news:f0ob15$dlb$1 at sea.gmane.org... | Jean-Paul Calderone schrieb: | > On Wed, 25 Apr 2007 21:28:10 +0200, Georg Brandl wrote: | >>Lars Gust?bel schrieb: | >>> On Sat, Apr 21, 2007 at 04:45:37PM +1000, Anthony Baxter wrote: | >>>> Ok, things seem to be OK. So the release25-maint branch is unfrozen. | >>>> Go crazy. Well, a little bit crazy. | >>> | >>> I'm afraid that I went crazy a little too early. Sorry for that. | >>> Won't happen again. | >> | >>BTW, svn provides a "lock" mechanism by which the branch freezing could | >>be enforced more strictly... | > | > It doesn't work on directories, though. | | Oh, didn't realize that. Thanks. Does the SVN tracker (presuming there is one) take RFEs? From dustin at v.igoro.us Wed Apr 25 23:11:00 2007 From: dustin at v.igoro.us (dustin at v.igoro.us) Date: Wed, 25 Apr 2007 16:11:00 -0500 Subject: [Python-Dev] 2.5 branch unfrozen In-Reply-To: References: Message-ID: <20070425211100.GI8265@v.igoro.us> On Wed, Apr 25, 2007 at 05:00:42PM -0400, Terry Reedy wrote: > Does the SVN tracker (presuming there is one) take RFEs? http://subversion.tigris.org/project_issues.html But I would expect that this proposal will not pass the "buddy system" (which is a cute idea IMHO), as locking is counter to the Subversion Way. Also, you could accomplish what you want (a locked branch) with a pre-commit hook that just scans for paths in that branch. Dustin From tjreedy at udel.edu Wed Apr 25 23:13:42 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 25 Apr 2007 17:13:42 -0400 Subject: [Python-Dev] whitespace normalization References: Message-ID: "Guido van Rossum" wrote in message news:ca471dc20704251124v3b80b33ala02167305f41e191 at mail.gmail.com... | On 4/25/07, Georg Brandl wrote: | > Well, there are editors that don't intelligently strip whitespace, so that | > people using them would be constantly pained by such a hook. | | And they should. There really is no excuse for letting one developer's | poor choice of tools cause later grief for all other developers. In 2.4.3, IDLE leaves trailing whitespace as is, even where senseless :-( I'll check the 2.5.1 version when I install it. tjr From glyph at divmod.com Thu Apr 26 00:14:41 2007 From: glyph at divmod.com (glyph at divmod.com) Date: Wed, 25 Apr 2007 22:14:41 -0000 Subject: [Python-Dev] whitespace normalization In-Reply-To: <84585667-F74D-4001-92B4-D57FD1D76159@python.org> References: <17967.40949.778426.208213@montanaro.dyndns.org> <84585667-F74D-4001-92B4-D57FD1D76159@python.org> Message-ID: <20070425221441.7769.1552989567.divmod.xquotient.3147@joule.divmod.com> On 08:15 pm, barry at python.org wrote: >;; untabify and clean up lines with just whitespace >(defun baw-whitespace-normalization () Looks a lot like (whitespace-cleanup), if you've got the right configuration. A function I've used many times :). -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070425/f3c505fb/attachment.htm From martin at v.loewis.de Thu Apr 26 00:29:10 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 26 Apr 2007 00:29:10 +0200 Subject: [Python-Dev] 2.5 branch unfrozen In-Reply-To: <20070425211100.GI8265@v.igoro.us> References: <20070425211100.GI8265@v.igoro.us> Message-ID: <462FD636.5020202@v.loewis.de> > But I would expect that this proposal will not pass the "buddy system" > (which is a cute idea IMHO), as locking is counter to the Subversion > Way. > > Also, you could accomplish what you want (a locked branch) with a > pre-commit hook that just scans for paths in that branch. I think "social engineering" is the appropriate way of dealing with frozen code bases. IMHO, committers do need to consider release mechanics in each and every commit (anticipating the demands of release engineering whether or not a release is immanent); it's not a bad thing if commits get rolled back because the committer violated some constraints of the release manager. The 2.5.1 release happened to reveal some misunderstandings as to how subversion is used in the release process; having these issues cleared up was well worth the trouble. Regards, Martin From tim.peters at gmail.com Thu Apr 26 00:34:46 2007 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 25 Apr 2007 18:34:46 -0400 Subject: [Python-Dev] whitespace normalization In-Reply-To: <17967.40949.778426.208213@montanaro.dyndns.org> References: <17967.40949.778426.208213@montanaro.dyndns.org> Message-ID: <1f7befae0704251534v605993e9l2767273d30d6a8d5@mail.gmail.com> [Skip] > Just a little FYI, python-mode (the one Barry and I manage - dunno about the > one distributed w/ GNU Emacs these days) is one of those tools that leaves > trailing whitespace behind when advancing to the next line.. Shouldn't be -- unless the behavior of the Emacs newline-and-indent has changed. The pymode version (py-newline-and-indent) relies in part on newline-and-indent; the intent is documented in the py-newline-and-indent docstring: (defun py-newline-and-indent () "... In general, deletes the whitespace before point, inserts a newline, and takes an educated guess as to how you want the new line indented." IIRC, pymode leaves C-j bound to plain old newline, so if you're in the habit of starting a new line via C-j no changes of any kind are made to whitespace. But that's not the intended way to use pymode. From greg.ewing at canterbury.ac.nz Thu Apr 26 03:22:37 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 26 Apr 2007 13:22:37 +1200 Subject: [Python-Dev] whitespace normalization In-Reply-To: <17967.40949.778426.208213@montanaro.dyndns.org> References: <17967.40949.778426.208213@montanaro.dyndns.org> Message-ID: <462FFEDD.20109@canterbury.ac.nz> skip at pobox.com wrote: > Just a little FYI, python-mode (the one Barry and I manage - dunno about the > one distributed w/ GNU Emacs these days) is one of those tools that leaves > trailing whitespace behind when advancing to the next line.. I get extremely annoyed with editors that *don't* leave whitespace at the end of lines alone, at least while you're working on the file. You type part of a line, leave a space, go to copy something from somewhere else to put on the end, and when you get back, the space you deliberately left there is gone. Stripping trailing whitespace on saving wouldn't be so bad, though. Is there an option for that in python-mode? -- Greg From skip at pobox.com Thu Apr 26 04:53:01 2007 From: skip at pobox.com (skip at pobox.com) Date: Wed, 25 Apr 2007 21:53:01 -0500 Subject: [Python-Dev] whitespace normalization In-Reply-To: <462FFEDD.20109@canterbury.ac.nz> References: <17967.40949.778426.208213@montanaro.dyndns.org> <462FFEDD.20109@canterbury.ac.nz> Message-ID: <17968.5133.551371.284656@montanaro.dyndns.org> >> Just a little FYI, python-mode (the one Barry and I manage - dunno >> about the one distributed w/ GNU Emacs these days) is one of those >> tools that leaves trailing whitespace behind when advancing to the >> next line.. (See my earlier retraction...) Greg> I get extremely annoyed with editors that *don't* leave whitespace Greg> at the end of lines alone, at least while you're working on the Greg> file. You type part of a line, leave a space, go to copy something Greg> from somewhere else to put on the end, and when you get back, the Greg> space you deliberately left there is gone. That's not the way python-mode works. If I enter this: def f(a): b = a * a @ % then hit LF or RET after the statement, it inserts a newline and inserts enough spaces to leave the cursor where I placed the @ sign. If I hit another LF or RET it removes the "trailing" whitespace on the current line, enters a newline and inserts enough spaces to leave the cursor where I placed the % sign. Greg> Stripping trailing whitespace on saving wouldn't be Greg> so bad, though. Is there an option for that in python-mode? Not by default. You could add a hook to write-file-hooks though. Skip From martin at v.loewis.de Thu Apr 26 07:59:31 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 26 Apr 2007 07:59:31 +0200 Subject: [Python-Dev] whitespace normalization In-Reply-To: References: Message-ID: <46303FC3.2070708@v.loewis.de> > Okay, attached is a pre-commit hook script for that purpose. How does that deal with deletions? What do you think about the attached alternative? Regards, Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: checkindent.py Type: text/x-python Size: 995 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20070426/632ec5a1/attachment.py From g.brandl at gmx.net Thu Apr 26 08:15:05 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 26 Apr 2007 08:15:05 +0200 Subject: [Python-Dev] whitespace normalization In-Reply-To: <46303FC3.2070708@v.loewis.de> References: <46303FC3.2070708@v.loewis.de> Message-ID: Martin v. L?wis schrieb: >> Okay, attached is a pre-commit hook script for that purpose. > > How does that deal with deletions? Ah, you'd have to look at the first two chars of every lines that I cut away. > What do you think about > the attached alternative? I'd say it's better since it uses the Python SVN bindings instead of calling external programs... Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From tanzer at swing.co.at Thu Apr 26 09:28:10 2007 From: tanzer at swing.co.at (Christian Tanzer) Date: Thu, 26 Apr 2007 09:28:10 +0200 Subject: [Python-Dev] whitespace normalization In-Reply-To: Your message of "Thu, 26 Apr 2007 13:22:37 +1200." <462FFEDD.20109@canterbury.ac.nz> Message-ID: Greg Ewing wrote: > Stripping trailing whitespace on saving wouldn't be > so bad, though. Is there an option for that in python-mode? I've got thje following in my ~/.emacs: (if (fboundp 'delete-trailing-whitespace) (add-hook 'write-file-hooks 'delete-trailing-whitespace) ) In modern Emacsen, the `if` should be unnecessary. -- Christian Tanzer http://www.c-tanzer.at/ From martin at v.loewis.de Thu Apr 26 12:41:16 2007 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 26 Apr 2007 12:41:16 +0200 Subject: [Python-Dev] Pre-commit hook for normalized white-space Message-ID: <463081CC.3010903@v.loewis.de> I have now installed a pre-commit hook that verifies that the code being committed follows the formatting of reindent.py, for all files under /python in the projects repository. Please let me know if this causes any problems. Regards, Martin From loupgaroublond at gmail.com Thu Apr 26 15:25:09 2007 From: loupgaroublond at gmail.com (Yaakov Nemoy) Date: Thu, 26 Apr 2007 09:25:09 -0400 Subject: [Python-Dev] Attachment Policy? In-Reply-To: <17965.30975.493972.176877@montanaro.dyndns.org> References: <7f692fec0704232013l1980d694ubf5c74cfdcb596a9@mail.gmail.com> <17965.30975.493972.176877@montanaro.dyndns.org> Message-ID: <7f692fec0704260625y46228ca9v16a653ab96019bb6@mail.gmail.com> On 4/23/07, skip at pobox.com wrote: > > Yaakov> i want to post a little bit of something I've been working on. > Yaakov> Before I post it, i want to know if I'm allowed to post it as an > Yaakov> attachment. Coming up, some work on ASTVisitor in the compiler > Yaakov> module. > > If you're proposing that your change be included in Python please create a > patch at SourceForge so it doesn't get lost: > > http://sourceforge.net/tracker/?group_id=5470&atid=305470 > > Skip Already done :) http://python.org/sf/1706323 From goodger at python.org Thu Apr 26 16:39:47 2007 From: goodger at python.org (David Goodger) Date: Thu, 26 Apr 2007 10:39:47 -0400 Subject: [Python-Dev] call for PEPs Message-ID: <4335d2c40704260739k1b9fc46fs1701ecabd9eb7d4e@mail.gmail.com> I'm one of the PEP editors at the end of peps-at-python-dot-org; Barry Warsaw is the other. Since before PyCon, I haven't had time for PEP number assignments, editing, and checkins. I don't follow python-dev or python-3000 closely, so I only recently learned of the April 30 deadline for PEPs. I'd like to get all the outstanding PEPs checked in, but I need help. There are a bunch of draft PEPs in my inbox, but I can't trust it because some have already been checked in, and others have since been updated or become obsolete. Please re-send the latest drafts (as attachments!) of any outstanding PEPs to peps-at-python-dot-org. Include the dates of any postings of the PEP to python-dev, python-3000, or python-list (in the Post-History header). Be sure to read and follow PEP 1 (PEP Purpose and Guidelines; http://www.python.org/dev/peps/pep-0001/) for details on what to include. Posting a draft PEP to a python-* mailing list IS NOT SUFFICIENT. It must be sent to peps-at-python-dot-org. Committers (those with commit access to Python's SVN) are welcome to add PEPs themselves. Just take the next available number from PEP 0 (which must be updated). BTW, with PSF and PyCon duties (I'm chairing PyCon 2008), I don't see my availability for PEP editing increasing any time soon. A new PEP Editor would be welcome! -- David Goodger From skip at pobox.com Thu Apr 26 19:18:44 2007 From: skip at pobox.com (skip at pobox.com) Date: Thu, 26 Apr 2007 12:18:44 -0500 Subject: [Python-Dev] call for PEPs In-Reply-To: <4335d2c40704260739k1b9fc46fs1701ecabd9eb7d4e@mail.gmail.com> References: <4335d2c40704260739k1b9fc46fs1701ecabd9eb7d4e@mail.gmail.com> Message-ID: <17968.57076.311577.775834@montanaro.dyndns.org> David> ... I only recently learned of the April 30 deadline for David> PEPs. This is the first I heard of any sort of deadline... Skip From guido at python.org Thu Apr 26 19:22:01 2007 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Apr 2007 10:22:01 -0700 Subject: [Python-Dev] call for PEPs In-Reply-To: <17968.57076.311577.775834@montanaro.dyndns.org> References: <4335d2c40704260739k1b9fc46fs1701ecabd9eb7d4e@mail.gmail.com> <17968.57076.311577.775834@montanaro.dyndns.org> Message-ID: On 4/26/07, skip at pobox.com wrote: > David> ... I only recently learned of the April 30 deadline for > David> PEPs. > > This is the first I heard of any sort of deadline... I posted to python-3000 a while ago. It never occurred to me that people interested in writing PEPs for Py3k would not be subscribed to that list. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From g.brandl at gmx.net Thu Apr 26 19:23:42 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 26 Apr 2007 19:23:42 +0200 Subject: [Python-Dev] call for PEPs In-Reply-To: <17968.57076.311577.775834@montanaro.dyndns.org> References: <4335d2c40704260739k1b9fc46fs1701ecabd9eb7d4e@mail.gmail.com> <17968.57076.311577.775834@montanaro.dyndns.org> Message-ID: skip at pobox.com schrieb: > David> ... I only recently learned of the April 30 deadline for > David> PEPs. > > This is the first I heard of any sort of deadline... Guido has stated a few times now that PEPs to be considered for 3.0a1 should (must?) be presented in April, lastly in a thread titled "Reminder: Py3k PEPs due by April"... ;) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From skip at pobox.com Thu Apr 26 19:46:24 2007 From: skip at pobox.com (skip at pobox.com) Date: Thu, 26 Apr 2007 12:46:24 -0500 Subject: [Python-Dev] call for PEPs In-Reply-To: References: <4335d2c40704260739k1b9fc46fs1701ecabd9eb7d4e@mail.gmail.com> <17968.57076.311577.775834@montanaro.dyndns.org> Message-ID: <17968.58736.342346.797763@montanaro.dyndns.org> Guido> On 4/26/07, skip at pobox.com wrote: David> ... I only recently learned of the April 30 deadline for David> PEPs. >> >> This is the first I heard of any sort of deadline... Guido> I posted to python-3000 a while ago. It never occurred to me that Guido> people interested in writing PEPs for Py3k would not be Guido> subscribed to that list. I'm not on the python-3000 list anymore (it was way too high-traffic for me). David's message was cross-posted to python-dev and didn't indicate that the deadline had anything to do with Python 3.0. Skip From khabkr at yahoo.com Fri Apr 27 16:01:16 2007 From: khabkr at yahoo.com (Khalid A. Bakr) Date: Fri, 27 Apr 2007 07:01:16 -0700 (PDT) Subject: [Python-Dev] Python 2.5.1 Message-ID: <811027.24175.qm@web56608.mail.re3.yahoo.com> Hello I downloaded Python's 2.5.1 (final) bz2 source archive the other day to try to compile it in MinGW. I have noted down the following observations that might be of interest. 1. The bz2 archive ships with \Modules\collectionsmodule.c instead of the \Modules\_collectionsmodule.c used in the 2.5 SVN branch. In fact the collectionsmodule.c was removed some time ago. 2. If _collectionsmodule.c is the one to be used in branch and source then it follows that \PC\config.c needs an update. 3. test_1686475 of test_os appears to rely on the existence of "c:\pagefile.sys" like so: def test_1686475(self): # Verify that an open file can be stat'ed try: os.stat(r"c:\pagefile.sys") except WindowsError, e: if e == 2: # file does not exist; cannot run test return self.fail("Could not stat pagefile.sys") But since that file does not appear to be in my C drive and since the Windows error returned is not a numeric, but rather a string of the sort: "[Error 5] Access is denied: 'c:\\pagefile.sys'" then that test fails for me both in the MinGW compiled Python and in the officially distributed one. 4. Also test_1565150 of test_os which reads as follows: # Restrict test to Win32, since there is no guarantee other # systems support centiseconds if sys.platform == 'win32': def test_1565150(self): t1 = 1159195039.25 os.utime(self.fname, (t1, t1)) self.assertEquals(os.stat(self.fname).st_mtime, t1) fails in the MinGW compiled Python with the following message: ====================================================================== FAIL: test_1565150 (test.test_os.StatAttributeTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "G:\projs\py25\python\r25\lib\test\test_os.py", line 241, in test_1565150 self.assertEquals(os.stat(self.fname).st_mtime, t1) AssertionError: 1159195040 != 1159195039.25 If the same test passes in the official CPython on the same machine (and it does), can it then be deduced that this is not a system's issue but a compiler one? Thanks Khalid __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From skip at pobox.com Fri Apr 27 17:29:14 2007 From: skip at pobox.com (skip at pobox.com) Date: Fri, 27 Apr 2007 10:29:14 -0500 Subject: [Python-Dev] whitespace normalization In-Reply-To: References: Message-ID: <17970.5834.844667.865807@montanaro.dyndns.org> > Just a little FYI, python-mode (the one Barry and I manage - dunno > about the one distributed w/ GNU Emacs these days) is one of those > tools that leaves trailing whitespace behind when advancing to the > next line.. Okay, I figured this out. The dangling whitespace turds are caused by LF or RET followed by a regular cursor motion keypress (e.g. C-n or the arrow keys) which move the cursor away from that line. Those commands operate outside python-mode's control there so it doesn't get the opportunity to remove the whitespace before moving the cursor. XEmacs (at least) doesn't have a cursor-motion-hook so there's no clean way to do this. I've been using Emacs for so many years that my central control program long ago pushed out most of the basic cursor motion control code to ganglia which reside in my wrists. The channel is full duplex but highly assymetric. The upload speed is very slow. It thus takes a couple days to realize in my frontal cortex how this low level stuff works... Skip From facundo at taniquetil.com.ar Fri Apr 27 20:39:50 2007 From: facundo at taniquetil.com.ar (Facundo Batista) Date: Fri, 27 Apr 2007 18:39:50 +0000 (UTC) Subject: [Python-Dev] New operations in Decimal Message-ID: The following are the new operations in the decimal module that we'll be available according to the last published specification. I wrote here the proposed name by me, the original name between parenthesis, where it will be located between square brackets (C for context and D for the decimal object itself), and a small description of the operation (I specially trimmed the verbose special cases from the description, if you want an exact explanation of what does each, read the spec). Note that always it says how many operands it takes. The number implies you're calling it through the context. If called through Decimal, you have to substract one from it (as the first operand will be the number itself in this case). Enjoy it, and any feedback is very appreciated, :) - compare_signal (compare?signal) [CD]: Takes two operands and compares their values numerically (pretty much like compare(), but all NaNs signal, with signaling NaNs taking precedence over quiet NaNs. - fma (fused-multiply-add) [C]: Takes three operands; the first two are multiplied together, using multiply, the third operand is then added to the result of that multiplication, using add, all with only one final rounding. - ln (ln) [CD]: Takes one operand, and the result is the natural (base e) logarithm of the operand. - log10 (log10) [CD]: Takes one operand, and the result is the base 10 logarithm of the operand. - max_mag (max?magnitude), and min_mag (min-magnitude) [CD]: Takes two operands and compares their values numerically with their sign ignored. - next_minus (next?minus) [CD]: Takes one operand and the result is the largest representable number that is smaller than the operand. - next_plus (next?plus) [CD]: Takes one operand and the result is the smallest representable number that is larger than the operand. - next_toward (next?toward) [CD]: Takes two operands and the result is the representable number closest to the first operand (but not the first operand) that is in the direction towards the second operand, unless the operands have the same value. - to_integral_exact (round-to-integral-exact) [C]: Takes one operand. It is similar to the round?to?integral?value (the old to_integral), with the difference the now Inexact and Rounded flags are allowed in the result. The following operations appear in a new section of the specification called "Miscelaneous Operations". Between these are logical operations that take logical operands, which are finite positive non-exponent numbers with a coefficient whose digits must all be either 0 or 1. - and (and), or (or), xor (xor) [CD]: Takes two logical operands, the result is the logical operation applied between each digit. - canonical (canonical) [CD]: Canonical takes one operand, returns the same Decimal object, as we do not have different encodings for the same number. - number_class (class) [CD]: Takes one operando, returns an indication of the class of the operand, where the class is one of the following: "sNaN", "NaN", "?Infinity", "?Normal", "?Subnormal", "?Zero", "+Zero", "+Subnormal", "+Normal" or "+Infinity". - compare_total (compare?total) [CD]: Takes two operands and compares them using their abstract representation rather than their numerical value (a total ordering is defined for all possible abstract representations). - compare_total_mag (compare?total?magnitude) [CD]: Takes two operands and compares them using their abstract representation rather than their numerical value, with their sign ignored and assumed to be 0. - copy_abs (copy-abs) [CD]: Takes one operand, returns a copy of it with the sign set to 0. - copy_negate (copy-negate) [CD]: Takes one operand, returns a copy of it with the sign inverted. - copy_sign (copy?sign) [CD]: Takes two operands, returns a copy of the first operand with the sign equal to the sign of the second operand. - invert (invert) [CD]: Takes one logical operand, the result is the digit-wise inversion of the operand. - is-canonical (is?canonical) [CD]: Takes one operand, returns 1 if the operand is canonical; otherwise returns 0. - is_finite (is?finite) [CD]: Takes one operand, returns 1 if the operand is neither infinite nor a NaN, otherwise returns 0. - is_infinite (is?infinite) [CD]: Takes one operand, returns 1 if the operand is an Infinite, otherwise returns 0. - is_nan (is?NaN) [CD]: Takes one operand, returns 1 if the operand is a quiet or signaling NaN, otherwise returns 0. - is_normal (is?normal) [CD]: Takes one operand, returns 1 if the operand is a positive or negative normal number, otherwise returns 0. - is_qnan (is?qNaN) [CD]: Takes one operand, returns 1 if the operand is a quiet NaN, otherwise returns 0. - is_signed (is?signed) [CD]: Takes one operand, returns 1 if the sign of the operand is 1, otherwise returns 0. - is_snan (is?sNaN) [CD]: Takes one operand, returns 1 if the operand is a signaling NaN, otherwise returns 0. - is_subnormal (is?subnormal) [CD]: Takes one operand, returns 1 if the operand is a positive or negative subnormal number, otherwise returns 0. - is_zero (is?zero) [CD]: Takes one operand, returns 1 if the operand is a zero, otherwise returns 0. - logb (logb) [CD]: Takes one operand, returns the integer which is the exponent of the magnitude of the most significant digit of the operand (as though the operand were truncated to a single digit while maintaining the value of that digit and without limiting the resulting exponent). - radix (radix) [CD]: Takes no operands, returns 10. - rotate (rotate) [CD]: Takes two operands. The coefficient of the result is a rotated copy of the digits in the coefficient of the first operand. The number of places of rotation is taken from the absolute value of the second operand, with the rotation being to the left if the second operand is positive or to the right otherwise. - scaleb (scaleb) [CD]: Takes two operands. The result is the first operand modified by adding the value of the second operand to its exponent. - shift (shift) [CD]: Takes two operands. The coefficient of the result is a shifted copy of the digits in the coefficient of the first operand. The number of places to shift is taken from the absolute value of the second operand, with the shift being to the left if the second operand is positive or to the right otherwise. Digits shifted into the coefficient are zeros. Regards, -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From martin at v.loewis.de Sat Apr 28 01:10:13 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 28 Apr 2007 01:10:13 +0200 Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <811027.24175.qm@web56608.mail.re3.yahoo.com> References: <811027.24175.qm@web56608.mail.re3.yahoo.com> Message-ID: <463282D5.4040008@v.loewis.de> Khalid A. Bakr schrieb: > 1. The bz2 archive ships with > \Modules\collectionsmodule.c instead of the > \Modules\_collectionsmodule.c used in the 2.5 SVN > branch. In fact the collectionsmodule.c was removed > some time ago. Why do you say that? http://svn.python.org/projects/python/branches/release25-maint/Modules/collectionsmodule.c is still present AFAICT. > 2. If _collectionsmodule.c is the one to be used in > branch and source then it follows that \PC\config.c > needs an update. But it isn't the one to be used. > But since that file does not appear to be in my C > drive and since the Windows error returned is not a > numeric, but rather a string of the sort: "[Error 5] > Access is denied: 'c:\\pagefile.sys'" then that test > fails for me both in the MinGW compiled Python and in > the officially distributed one. That's true. Can you come up with a patch? Looking at the error code of the exception should be sufficient; it should be 5 (as the message shows). The exception is *not* a string, but an object. > 4. Also test_1565150 of test_os which reads as > follows: [...] > fails in the MinGW compiled Python with the following > message: > > ====================================================================== > FAIL: test_1565150 (test.test_os.StatAttributeTests) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "G:\projs\py25\python\r25\lib\test\test_os.py", > line 241, in test_1565150 > > self.assertEquals(os.stat(self.fname).st_mtime, > t1) > AssertionError: 1159195040 != 1159195039.25 That would indicate a bug in the MingW port. > If the same test passes in the official CPython on the > same machine (and it does), can it then be deduced > that this is not a system's issue but a compiler one? Likely, neither nor. My guess is that the MingW port, for some reason, decides not to use the Win32 API to perform stat, but the C library. That is incorrect, as the C library will perform truncation of subsecond time stamps. The compiler itself should have no effect (other than defining different compiler recognition macros). Regards, Martin From kbk at shore.net Sat Apr 28 05:18:38 2007 From: kbk at shore.net (Kurt B. Kaiser) Date: Fri, 27 Apr 2007 23:18:38 -0400 (EDT) Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200704280318.l3S3Ict2023404@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 356 open ( -1) / 3756 closed (+11) / 4112 total (+10) Bugs : 968 open (+10) / 6673 closed (+16) / 7641 total (+26) RFE : 254 open ( +3) / 282 closed ( +2) / 536 total ( +5) New / Reopened Patches ______________________ fixes bug #1703952 - ctypes: problem with large integers (2007-04-19) CLOSED http://python.org/sf/1703972 opened by Alexander Belopolsky minidom Level 1 DOM compliance (2007-04-20) http://python.org/sf/1704134 opened by Jason Orendorff textwrap.py mod. for jython (2007-04-20) CLOSED http://python.org/sf/1704473 opened by Tim Couper test_optparse.py mod. for jython (2007-04-20) http://python.org/sf/1704474 opened by Tim Couper Use MoveFileEx() to implement os.rename() on windows (2007-04-20) http://python.org/sf/1704547 opened by Raghuram Devarakonda interpreter crash when multiplying large lists (2007-04-20) http://python.org/sf/1704621 opened by Daniel Stutzbach Added clearerr() to clear EOF state (2007-04-23) http://python.org/sf/1706039 opened by jos Updated ASTVisitor Classes (2007-04-24) http://python.org/sf/1706323 opened by Yaakov Nemoy Implementation of @abstractmethod for PEP 3119 (2007-04-24) http://python.org/sf/1706989 opened by Guido van Rossum get status output fix for Win32 (2007-04-25) http://python.org/sf/1707753 opened by Ken Phillis Jr. Make isinstance/issubclass overloadable (2007-04-26) http://python.org/sf/1708353 opened by Guido van Rossum Patches Closed ______________ silenced a compiler warning (2007-04-18) http://python.org/sf/1703268 closed by nnorwitz missing declaration in readline.c (2007-04-18) http://python.org/sf/1703270 closed by nnorwitz ZipFile.printdir fix (2.5) (2007-04-11) http://python.org/sf/1698915 closed by nnorwitz Migrate test_minidom.py to unittest (2007-03-30) http://python.org/sf/1691032 closed by jorend fixes bug #1703952 - ctypes: problem with large integers (2007-04-20) http://python.org/sf/1703972 closed by theller Update to Macintosh library documentation (2007-04-11) http://python.org/sf/1698768 closed by gbrandl textwrap.py mod. for jython (2007-04-20) http://python.org/sf/1704473 closed by gbrandl tarfile bug when opening a file directly (2007-04-05) http://python.org/sf/1695229 closed by gustaebel stream writing support in wave.py (2007-02-05) http://python.org/sf/1652328 closed by nnorwitz PEP 3114 -- next() -> __next__() (2007-03-07) http://python.org/sf/1675363 closed by gbrandl warnings.py gets filename wrong for eval/exec (2007-04-01) http://python.org/sf/1692664 closed by nnorwitz New / Reopened Bugs ___________________ "t.join(); assert t not in threading.enumerate()" fails (2007-04-19) http://python.org/sf/1703448 opened by Andrew Bennetts have a way to ignore nonexisting locales in locale.setlocale (2007-04-19) http://python.org/sf/1703592 opened by Matthias Klose ctypes: problem with large integers (2007-04-19) http://python.org/sf/1703952 reopened by belopolsky ctypes: problem with large integers (2007-04-19) http://python.org/sf/1703952 opened by Alexander Belopolsky TarFile.addfile() throws a struct.error (2007-04-20) CLOSED http://python.org/sf/1704156 opened by K. C. Wong must run "make" before "make install" (2007-04-20) http://python.org/sf/1704287 opened by Joseph VanAndel Exception message when using subprocess.POpen (2007-04-21) CLOSED http://python.org/sf/1704790 opened by Sam Ruby incorrect return value of unicodedata.lookup() - beoynd BMP (2007-04-21) http://python.org/sf/1704793 opened by vbr contextmanager eats StopIteration (2007-04-22) http://python.org/sf/1705170 opened by Adam Olsen cannot change cursor color in IDLE (2007-04-22) http://python.org/sf/1705362 opened by zorkin 'nonlocal x' at top level crashes interpreter (2007-04-22) CLOSED http://python.org/sf/1705365 opened by John Reese Select() failure (race condition) (2007-04-22) http://python.org/sf/1705393 opened by Ron Garret Error in sys.argv documenation (2007-04-23) CLOSED http://python.org/sf/1705717 opened by Kamil Kisiel pydoc.py has typo. (2007-04-23) CLOSED http://python.org/sf/1705997 opened by charlesmchen SWIG options not being followed in all cases (2007-04-24) CLOSED http://python.org/sf/1706381 opened by Morten Lied Johansen socket.error exceptions not subclass of StandardError (2007-04-24) http://python.org/sf/1706815 opened by John Nagle tarfile fails in 2.5.1 (NoneType has no ...) (2007-04-24) CLOSED http://python.org/sf/1706850 opened by Thomas Leonard Failed to build Python 2.5.1 with sqlite3 (2007-04-24) http://python.org/sf/1706863 opened by Vitaliy Yermolenko lost global variables in main memory intensive execution (2007-04-25) http://python.org/sf/1707255 opened by Jordi Pujol Ahull? generation errors in PDF-A4 tags for lib.pdf (2007-04-25) http://python.org/sf/1707497 opened by Antoine LECA raw text can't end with backslash (2007-04-25) CLOSED http://python.org/sf/1707607 opened by lomm os.path.normpath changes path (chops of trailing slash) (2007-04-26) http://python.org/sf/1707768 opened by Robert Siemer doctest work with Windows PyReadline (2007-04-26) http://python.org/sf/1708316 opened by manuelg find_module doc ambiguity (2007-04-26) http://python.org/sf/1708326 opened by Andrew McNabb Why not sequential? (2007-04-26) CLOSED http://python.org/sf/1708362 opened by Lucas Malor TimedRotatingFileHandler's doRollover opens file in "w" mode (2007-04-27) http://python.org/sf/1708538 opened by Florian Schmidt Exact matching (2007-04-27) http://python.org/sf/1708652 opened by Tom Lynn Tutorial error (2007-04-27) CLOSED http://python.org/sf/1708710 opened by john hite Bugs Closed ___________ questionable results from IDLE, Windows, & several built-in (2007-04-15) http://python.org/sf/1700912 closed by bavantgarde314 utf-16 codec problems with multiple file append (2007-04-16) http://python.org/sf/1701389 closed by doerwalter ctypes: problem with large integers (2007-04-19) http://python.org/sf/1703952 closed by theller README is referencing obsolete? http://starship.python.net (2007-04-09) http://python.org/sf/1696740 closed by tjreedy TarFile.addfile() throws a struct.error (2007-04-20) http://python.org/sf/1704156 closed by gustaebel Exception message when using subprocess.Popen (2007-04-21) http://python.org/sf/1704790 closed by gbrandl Incorrect example for add_password() (2007-04-18) http://python.org/sf/1703110 closed by nnorwitz SSL support in socket module needs tests (2001-08-16) http://python.org/sf/451607 closed by facundobatista 'nonlocal x' at top level crashes interpreter (2007-04-23) http://python.org/sf/1705365 closed by ncoghlan Error in sys.argv documenation (2007-04-23) http://python.org/sf/1705717 closed by gbrandl Property with -> annotation triggers assert (2007-04-10) http://python.org/sf/1697248 closed by ncoghlan pydoc.py has typo. (2007-04-23) http://python.org/sf/1705997 closed by gbrandl SWIG options not being followed in all cases (2007-04-24) http://python.org/sf/1706381 closed by gbrandl Tix is not included in 2.5 for Windows (2006-09-30) http://python.org/sf/1568240 closed by loewis tarfile fails in 2.5.1 (NoneType has no ...) (2007-04-24) http://python.org/sf/1706850 closed by gbrandl raw text can't end with backslash (2007-04-25) http://python.org/sf/1707607 closed by gbrandl Why not sequential? (2007-04-26) http://python.org/sf/1708362 deleted by lucas_malor Why not sequential? (2007-04-26) http://python.org/sf/1708362 closed by lucas_malor Tutorial error (2007-04-27) http://python.org/sf/1708710 closed by gbrandl New / Reopened RFE __________________ pyunit should allow __unittest in locals to trim stackframes (2007-04-23) http://python.org/sf/1705520 opened by Robert Collins Give Partial the ability to skip positionals (2007-04-23) CLOSED http://python.org/sf/1706256 opened by Calvin Spealman access to unicodedata (via codepoints or 2-char surrogates) (2007-04-24) http://python.org/sf/1706460 opened by vbr Please add .bz2 in encodings_map (in the module mimetypes) (2007-04-25) http://python.org/sf/1707693 opened by elghinn RFE Closed __________ Feature request: Comparing regexps (2007-04-16) http://python.org/sf/1701452 closed by rhettinger Give Partial the ability to skip positionals (2007-04-24) http://python.org/sf/1706256 closed by loewis From khabkr at yahoo.com Sat Apr 28 05:31:05 2007 From: khabkr at yahoo.com (Khalid A. Bakr) Date: Fri, 27 Apr 2007 20:31:05 -0700 (PDT) Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <463282D5.4040008@v.loewis.de> Message-ID: <208553.28313.qm@web56614.mail.re3.yahoo.com> --- "Martin v. L?wis" wrote: > Khalid A. Bakr schrieb: > > 1. The bz2 archive ships with > > \Modules\collectionsmodule.c instead of the > > \Modules\_collectionsmodule.c used in the 2.5 SVN > > branch. In fact the collectionsmodule.c was > removed > > some time ago. > > Why do you say that? > > http://svn.python.org/projects/python/branches/release25-maint/Modules/collectionsmodule.c > > is still present AFAICT. > > > 2. If _collectionsmodule.c is the one to be used > in > > branch and source then it follows that > \PC\config.c > > needs an update. > > But it isn't the one to be used. I am sorry. Me repository seems corrupted. Doing a clean check out of the 2.5 branch as I write this. > That's true. Can you come up with a patch? Looking > at the error code of the exception should be > sufficient; > it should be 5 (as the message shows). The exception > is *not* a string, but an object. Okay. It seems I mixed up WindowsError with the exception e in my post; at least it is now known that e is not a number. The patch is short and is as follows: Index: Lib/test/test_os.py =================================================================== --- Lib/test/test_os.py (revision 55014) +++ Lib/test/test_os.py (working copy) @@ -245,7 +245,8 @@ try: os.stat(r"c:\pagefile.sys") except WindowsError, e: - if e == 2: # file does not exist; cannot run test + # file may not exist, or access is denied; cannot run test + if e.winerror == 2 or e.winerror == 5: return self.fail("Could not stat pagefile.sys") Or do I need to submit this through sourceforge? > That would indicate a bug in the MingW port. > > > If the same test passes in the official CPython on > the > > same machine (and it does), can it then be deduced > > that this is not a system's issue but a compiler > one? > > Likely, neither nor. My guess is that the MingW > port, > for some reason, decides not to use the Win32 API to > perform stat, but the C library. That is incorrect, > as the C library will perform truncation of > subsecond > time stamps. The compiler itself should have no > effect > (other than defining different compiler recognition > macros). > > Regards, > Martin > I will try to check what can be done about this. Regards, Khalid __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From martin at v.loewis.de Sat Apr 28 07:28:05 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 28 Apr 2007 07:28:05 +0200 Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <208553.28313.qm@web56614.mail.re3.yahoo.com> References: <208553.28313.qm@web56614.mail.re3.yahoo.com> Message-ID: <4632DB65.9040209@v.loewis.de> > Or do I need to submit this through sourceforge? Please do. Why are you checking for error 2, though, if the error that occurs is 5? Regards, Martin From khabkr at yahoo.com Sat Apr 28 08:47:26 2007 From: khabkr at yahoo.com (Khalid A. Bakr) Date: Fri, 27 Apr 2007 23:47:26 -0700 (PDT) Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <4632DB65.9040209@v.loewis.de> Message-ID: <427687.97884.qm@web56613.mail.re3.yahoo.com> --- "Martin v. L?wis" wrote: > > Or do I need to submit this through sourceforge? > > Please do. Why are you checking for error 2, though, > if the error that occurs is 5? > > Regards, > Martin > > Done. I was just correcting the previous posted check for error 2; I thought it must have been there for a reason I don't know. Anyway, this part is now removed from patch, which is at: http://python.org/sf/1709112 Regards, Khalid __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From ironfroggy at gmail.com Sat Apr 28 14:43:16 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sat, 28 Apr 2007 08:43:16 -0400 Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <208553.28313.qm@web56614.mail.re3.yahoo.com> References: <463282D5.4040008@v.loewis.de> <208553.28313.qm@web56614.mail.re3.yahoo.com> Message-ID: <76fd5acf0704280543s75e08073m485f80404db09fe2@mail.gmail.com> On 4/27/07, Khalid A. Bakr wrote: > Okay. It seems I mixed up WindowsError with the > exception e in my post; at least it is now known that > e is not a number. The patch is short and is as > follows: > > Index: Lib/test/test_os.py > =================================================================== > --- Lib/test/test_os.py (revision 55014) > +++ Lib/test/test_os.py (working copy) > @@ -245,7 +245,8 @@ > try: > os.stat(r"c:\pagefile.sys") > except WindowsError, e: > - if e == 2: # file does not exist; > cannot run test > + # file may not exist, or access is > denied; cannot run test > + if e.winerror == 2 or e.winerror == > 5: > return > self.fail("Could not stat > pagefile.sys") I have a patch myself that creates an open file and uses that as the test. My reasoning is that pagefile.sys was chosen as a file that should always exist and be open, so its safe to test against, so we should just be testing against a fixture, instead. It is here, and if someone would reference a SF bug report, I'll attach to it, as well. I should also point out that I got the time errors as well, but with different incorrect results. However, I can't seem to reproduce it this morning, but I can say that I did have the test failing for me on VC yesterday. Index: test_os.py =================================================================== --- test_os.py (revision 54982) +++ test_os.py (working copy) @@ -6,6 +6,7 @@ import unittest import warnings import sys +import tempfile from test import test_support warnings.filterwarnings("ignore", "tempnam", RuntimeWarning, __name__) @@ -241,13 +242,18 @@ self.assertEquals(os.stat(self.fname).st_mtime, t1) def test_1686475(self): + fn = tempfile.mktemp() + openfile = open(fn, 'w') # Verify that an open file can be stat'ed try: - os.stat(r"c:\pagefile.sys") + os.stat(fn) except WindowsError, e: if e == 2: # file does not exist; cannot run test return self.fail("Could not stat pagefile.sys") + finally: + openfile.close() + os.remove(fn) from test import mapping_tests -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From draghuram at gmail.com Sat Apr 28 15:32:57 2007 From: draghuram at gmail.com (Raghuram Devarakonda) Date: Sat, 28 Apr 2007 09:32:57 -0400 Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <76fd5acf0704280543s75e08073m485f80404db09fe2@mail.gmail.com> References: <463282D5.4040008@v.loewis.de> <208553.28313.qm@web56614.mail.re3.yahoo.com> <76fd5acf0704280543s75e08073m485f80404db09fe2@mail.gmail.com> Message-ID: <2c51ecee0704280632x1d06d5cau1e19f7269c6ff635@mail.gmail.com> On 4/28/07, Calvin Spealman wrote: > Index: test_os.py > =================================================================== > --- test_os.py (revision 54982) > +++ test_os.py (working copy) > @@ -6,6 +6,7 @@ > import unittest > import warnings > import sys > +import tempfile > from test import test_support > > warnings.filterwarnings("ignore", "tempnam", RuntimeWarning, __name__) > @@ -241,13 +242,18 @@ > self.assertEquals(os.stat(self.fname).st_mtime, t1) > > def test_1686475(self): > + fn = tempfile.mktemp() > + openfile = open(fn, 'w') > # Verify that an open file can be stat'ed > try: > - os.stat(r"c:\pagefile.sys") > + os.stat(fn) > except WindowsError, e: > if e == 2: # file does not exist; cannot run test > return > self.fail("Could not stat pagefile.sys") > + finally: > + openfile.close() > + os.remove(fn) > > from test import mapping_tests mktemp() is deprecated. You may want to use mkstemp(). There will be no need for explicit open as well as mkstemp() also returns open descriptor. Thanks, Raghu. From ironfroggy at gmail.com Sat Apr 28 16:25:08 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sat, 28 Apr 2007 10:25:08 -0400 Subject: [Python-Dev] April 1-15, 2007 Summaries Final Draft Message-ID: <76fd5acf0704280725yc86d4acif9bed2d9fa0dc23a@mail.gmail.com> If no one has any further comments over the weekend and Monday, I'll post it as the final summary that evening/night. ============= Announcements ============= ========= Summaries ========= --------------- About SSL tests --------------- An open bug about missing SSL tests (#451607) was brought up to close or keep open. Jean-Paul Calderone mentioned an improved testing method by spawning a 'openssl s_server' for testing purposes. This sparked some talk about the terminating of subprocesses in a cross-platform manner (See `Cross-Platform Subprocess Termination`). Contributing thread: - `About SSL tests `__ ------------------------------------- Cross-Platform Subprocess Termination ------------------------------------- Because os.kill only works on *nix and OS X platforms, leaving Windows platforms uncovered. Although subprocess.TerminateProcess exists for Windows platforms, it requires the use of handles and additional overhead for use. Support was overall given for adding a UNIX-only signal() method and a cross- platform terminate() method to Popen instances. Nothing was said of actually incorporating these changes into the subprocess module, although code was given. Contributing thread: - (This was mixed in with the `About SSL tests` thread.) ------------------------ Extended buffer protocol ------------------------ Updates to the buffer protocol are discussed, along with the proposal pre-PEP documenting the updates for Python 3000, but brought to Python-Dev, due so the plans of backporting to 2.6 afterwards. (Note: I couldn't summarize this well enough to cover much of it do any real degree, but it is currently a Py3k issue, so maybe that is OK. If no one wants to add to the summary, it will stay short.) Contributing threads: - `Extended buffer PEP `__ - `PEP 3118: Extended buffer protocol (new version) `__ - `Extended Buffer Protocol - simple use examples `__ ----------------------------------------- function for counting items in a sequence ----------------------------------------- A patch was submitted by Steven Bethard (http://bugs.python.org/1696199), implemented a discussed collections.counts() function to provide a mapping between items in an iterable and the number of times they appear. There were suggested names, but none overthrew the original 'counts()' and a question of items not appearing being counted as 0 or raising a KeyError, with 0 winning due to a just-makes-sense factor. Contributing thread: - `function for counting items in a sequence `__ ---------------------------------------- context manager - generator interaction? ---------------------------------------- A problem was brought up with iterator context managers and iteration inside the with-block raising its StopIteration, but being caught by the context manager mechanics. It was also responded that the problem would not exist without the use of overly broad try blocks, and this lead to the addition of a formal note in PEP 8 about keeping narrow try blocks. Contributing thread: - `context manager - generator interaction? `__ ----------------------------- proposed which.py replacement ----------------------------- Suggestion of replacing the which.py script in the Tools directory migrated to its proposal for inclusion into the standard library. A patch and tests have yet to be provided. Contributing thread: - `proposed which.py replacement `_ ----------------------- minidom and DOM level 2 ----------------------- What is missing for DOM Level 2 support in minidom was highlighted and some work jumpstarted. Contributing thread: - `minidom and DOM level 2 `__ ---------------------------------------------- test_pty.py hangs in verbose mode on Mac OS X? ---------------------------------------------- Differing buffering behavior was causing test_pty to block only in verbose mode. Solutions may include reading to clear the buffer of child processes before a waitpid() call. Contributing thread: - `test_pty.py hangs in verbose mode on Mac OS X? `__ ------------------------- HTTP Responses and Errors ------------------------- In 2xx HTTP responses mean that the request was handled OK. The existing library was special-casing the most common responses and treating others as errors. After verifying that there wasn't a good reason for the old behavior, Facundo Batista fixed it. Contributing thread: - `HTTP responses and errors `__ ------------------------ Build Problem on Windows ------------------------ It is a generated file; the actual problem is in make_buildinfo Contributing thread: - `build problem on windows: unable to find getbuildinfo2.c `__ -------------------------- BaseException Pickle Issue -------------------------- Exceptions are now new-style classes; this caused some obscure problems with picking and unpickling. http://www.python.org/sf/1498571 and later http://www.python.org/sf/1692335 should resolve the issue. Contributing thread: - `BaseException pickle issue `__ --------------------------------- Deprecating BaseException.message --------------------------------- Plans changed; BaseException will still accept an args tuple, so don't bother with the .message attribute. Probably the shorted leaved Python feature ever. See PEP 352. Contributing thread: - `deprecating BaseException.message `__ --------------------- Changes to decimal.py --------------------- The external standard has been updated; python's implementation will be updated to match. Contributing threads: - `Changes to decimal.py `__ - `Fwd: Re: Changes to decimal.py `__ ---------------------------------------- Pydoc Rewrite Discussion at doc-sig list ---------------------------------------- An announcement that the (normally quiet) doc-sig mailing list would be discussing a rewrite of pydoc. http://mail.python.org/pipermail/doc-sig/ Contributing thread: - `Pydoc Rewrite Discussion at doc-sig list. `__ ------------------------------ Making builtins more efficient ------------------------------ Andrea Griffini posted a patch at sourceforge that makes builtin lookups almost as fast as locals. https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1616125&group_id=5470 Contributing thread: - `Making builtins more efficient `__ ================ Deferred Threads ================ ================== Previous Summaries ================== =============== Skipped Threads =============== - `Checking PEP autobuild results `__ - `Python 2.5.1c1 pickle problem `__ - `Summary of Tracker Issues `__ - `possible urllib bug on Windows XP `__ - `Py2.5.1 release candidate `__ - `Some new additions to functools `__ - `Python+XUL `__ - `Distutils and -framework on MacOSX `__ - `[Python-checkins] svn dead? `__ - `Just saying hello `__ - `ok to just checkin minor obvious fixes? `__ - `__lltrace__ `__ - `new subscriber looking for grunt work `__ - `functools additions `__ - `Python Documentation Problem Example `__ - `Get 2.5 changes in now, branch will be frozen soon `__ - `Quoting netiquette reminder [Re: proposed which.py replacement] `__ - `branch is frozen for release of 2.5.1c1 (and 2.5.1) `__ - `More exposure for PEP8 (was: context manager - generator interaction?) `__ - `[Python-checkins] Python Regression Test Failures opt (1) `__ - `Weekly Python Patch/Bug Summary `__ - `USE_FAST code in stringobject.c `__ - `Fwd: Re: Py2.5.1 release candidate `__ - `svn.python.org `__ - `[PATCH] pep 0324 URL update `__ - `my 2.5 checkins `__ - `fdopen mode needs sanitizing `__ - `Py3: function signatures, type checking, and related crap `__ - `concerns regarding callable() method `__ - `A Survey on Defect Management Practices in Free/Open Source Software `__ - `RELEASED Python 2.5.1, release candidate 1 `__ - `Python 3000 PEP: Postfix type declarations `__ - `test_socketserver flakey? `__ -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From exarkun at divmod.com Sat Apr 28 18:17:31 2007 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Sat, 28 Apr 2007 12:17:31 -0400 Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <2c51ecee0704280632x1d06d5cau1e19f7269c6ff635@mail.gmail.com> Message-ID: <20070428161731.19381.676206279.divmod.quotient.6052@ohm> On Sat, 28 Apr 2007 09:32:57 -0400, Raghuram Devarakonda wrote: >On 4/28/07, Calvin Spealman wrote: >> Index: test_os.py >> =================================================================== >> --- test_os.py (revision 54982) >> +++ test_os.py (working copy) >> @@ -6,6 +6,7 @@ >> import unittest >> import warnings >> import sys >> +import tempfile >> from test import test_support >> >> warnings.filterwarnings("ignore", "tempnam", RuntimeWarning, __name__) >> @@ -241,13 +242,18 @@ >> self.assertEquals(os.stat(self.fname).st_mtime, t1) >> >> def test_1686475(self): >> + fn = tempfile.mktemp() >> + openfile = open(fn, 'w') >> # Verify that an open file can be stat'ed >> try: >> - os.stat(r"c:\pagefile.sys") >> + os.stat(fn) >> except WindowsError, e: >> if e == 2: # file does not exist; cannot run test >> return >> self.fail("Could not stat pagefile.sys") >> + finally: >> + openfile.close() >> + os.remove(fn) >> >> from test import mapping_tests > >mktemp() is deprecated. You may want to use mkstemp(). There will be >no need for explicit open as well as mkstemp() also returns open >descriptor. You still need fdopen() though, since os.stat() won't take a file descriptor. The patch is incomplete though, since it should remove the ENOENT handling and the remaining reference to pagefile.sys. As for mktemp() being deprecated - the docstring warns users away, but actually calling it emits no warning. Sure, using it can lead to insecurities, but there's hardly any worry of that here. If the function were actually deprecated (that is, if calling it emitted a DeprecationWarning), that would be a good reason to avoid calling it, though. Jean-Paul From ironfroggy at gmail.com Sat Apr 28 20:21:12 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sat, 28 Apr 2007 14:21:12 -0400 Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <20070428161731.19381.676206279.divmod.quotient.6052@ohm> References: <2c51ecee0704280632x1d06d5cau1e19f7269c6ff635@mail.gmail.com> <20070428161731.19381.676206279.divmod.quotient.6052@ohm> Message-ID: <76fd5acf0704281121l31ead120j3fb441dbd7d1098@mail.gmail.com> On 4/28/07, Jean-Paul Calderone wrote: > On Sat, 28 Apr 2007 09:32:57 -0400, Raghuram Devarakonda wrote: > >On 4/28/07, Calvin Spealman wrote: > >> Index: test_os.py > >> =================================================================== > >> --- test_os.py (revision 54982) > >> +++ test_os.py (working copy) > >> @@ -6,6 +6,7 @@ > >> import unittest > >> import warnings > >> import sys > >> +import tempfile > >> from test import test_support > >> > >> warnings.filterwarnings("ignore", "tempnam", RuntimeWarning, __name__) > >> @@ -241,13 +242,18 @@ > >> self.assertEquals(os.stat(self.fname).st_mtime, t1) > >> > >> def test_1686475(self): > >> + fn = tempfile.mktemp() > >> + openfile = open(fn, 'w') > >> # Verify that an open file can be stat'ed > >> try: > >> - os.stat(r"c:\pagefile.sys") > >> + os.stat(fn) > >> except WindowsError, e: > >> if e == 2: # file does not exist; cannot run test > >> return > >> self.fail("Could not stat pagefile.sys") > >> + finally: > >> + openfile.close() > >> + os.remove(fn) > >> > >> from test import mapping_tests > > > >mktemp() is deprecated. You may want to use mkstemp(). There will be > >no need for explicit open as well as mkstemp() also returns open > >descriptor. > > You still need fdopen() though, since os.stat() won't take a file > descriptor. > > The patch is incomplete though, since it should remove the ENOENT > handling and the remaining reference to pagefile.sys. > > As for mktemp() being deprecated - the docstring warns users away, > but actually calling it emits no warning. Sure, using it can lead > to insecurities, but there's hardly any worry of that here. If the > function were actually deprecated (that is, if calling it emitted a > DeprecationWarning), that would be a good reason to avoid calling it, > though. > > Jean-Paul Yes, duh, I changed the pagefile.sys in the message, which was silly to have left. As for os.fdopen(), it is not needed. tempfile.mkstemp() returns both the file descriptor and the path, so we simply ignore the file descriptor and pass the path to os.stat(). I'll report the patch with the message fixed. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From status at bugs.python.org Sun Apr 29 02:00:52 2007 From: status at bugs.python.org (Tracker) Date: Sun, 29 Apr 2007 00:00:52 +0000 (UTC) Subject: [Python-Dev] Summary of Tracker Issues Message-ID: <20070429000052.1DF16780B4@psf.upfronthosting.co.za> ACTIVITY SUMMARY (04/22/07 - 04/29/07) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1649 open ( +0) / 8584 closed ( +0) / 10233 total ( +0) Average duration of open issues: 778 days. Median duration of open issues: 729 days. Open Issues Breakdown open 1649 ( +0) pending 0 ( +0) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070429/c9859bf9/attachment-0001.html From martin at v.loewis.de Sun Apr 29 03:23:22 2007 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 29 Apr 2007 03:23:22 +0200 Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <76fd5acf0704280543s75e08073m485f80404db09fe2@mail.gmail.com> References: <463282D5.4040008@v.loewis.de> <208553.28313.qm@web56614.mail.re3.yahoo.com> <76fd5acf0704280543s75e08073m485f80404db09fe2@mail.gmail.com> Message-ID: <4633F38A.4040605@v.loewis.de> > I have a patch myself that creates an open file and uses that as the > test. My reasoning is that pagefile.sys was chosen as a file that > should always exist and be open, so its safe to test against, so we > should just be testing against a fixture, instead. It is here, and if > someone would reference a SF bug report, I'll attach to it, as well. There must be more to the problem than just an open file. Please undo the change that triggered the addition of the test, and see whether you can reproduce the original problem with an arbitrary open file (I could trigger the problem with pagefile.sys at the time). Regards, Martin From ironfroggy at gmail.com Sun Apr 29 04:43:04 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sat, 28 Apr 2007 22:43:04 -0400 Subject: [Python-Dev] New Super PEP Message-ID: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> Comments welcome, of course. Bare with my first attempt at crafting a PEP. PEP: XXX Title: Super As A Keyword Version: $Revision$ Last-Modified: $Date$ Author: Calvin Spealman Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 30-Apr-2007 Python-Version: 2.6 Post-History: Abstract ======== The PEP defines the proposal to enhance the super builtin to work implicitly upon the class within which it is used and upon the instance the current function was called on. The premise of the new super usage suggested is as follows: super.foo(1, 2) to replace the old: super(Foo, self).foo(1, 2) Rationale ========= The current usage of super requires an explicit passing of both the class and instance it must operate from, requiring a breaking of the DRY (Don't Repeat Yourself) rule. This hinders any change in class name, and is often considered a wart by many. Specification ============= Replacing the old usage of super, calls to the next class in the MRO (method resolution order) will be made without an explicit super object creation, by simply accessing an attribute on the super type directly, which will automatically apply the class and instance to perform the proper lookup. The following example demonstrates the use of this. :: class A(object): def f(self): return 'A' class B(A): def f(self): return 'B' + super.f() class C(A): def f(self): return 'C' + super.f() class D(B, C): def f(self): return 'D' + super.f() assert D().f() == 'DBCA' The proposal adds a dynamic attribute lookup to the super type, which will automatically determine the proper class and instance parameters. Each super attribute lookup identifies these parameters and performs the super lookup on the instance, as the current super implementation does with the explicit invokation of a super object upon a class and instance. The enhancements to the super type will define a new __getattr__ classmethod of the super type, which must look backwards to the previous frame and locate the instance object. This can be naively determined by located the local named by the first argument to the function. Using super outside of a function where this is a valid lookup for the instance can be considered undocumented in its behavior. Every class will gain a new special attribute, __super__, which is a super object instansiated only with the class it is an attribute of. In this capacity, the new super also acts as its own descriptor, create an instance- specific super upon lookup. Much of this was discussed in the thread of the python-dev list, "Fixing super anyone?" [1]_. Open Issues ----------- __call__ methods '''''''''''''''' Backward compatability of the super type API raises some issues. Names, the lookup of the __call__ of the super type itself, which means a conflict with doing an actual super lookup of the __call__ attribute. Namely, the following is ambiguous in the current proposal: :: super.__call__(arg) Which means the backward compatible API, which involves instansiating the super type, will either not be possible, because it will actually do a super lookup on the __call__ attribute, or there will be no way to perform a super lookup on the __call__ attribute. Both seem unacceptable, so any suggestions are welcome. super type's new getattr '''''''''''''''''''''''' To give the behavior needed, the super type either needs a way to do dynamic lookup of attributes on the super type object itself or define a metaclass for the builtin type. This author is unsure which, if either, is possible with C- defined types. When should we create __super__ attributes? ''''''''''''''''''''''''''''''''''''''''''' They either need to be created on class creation or on __super__ attribute lookup. For the second, they could be cached, of course, which seems like it may be the best idea, if implicit creation of a super object for every class is considered too much overhead. References ========== .. [1] Fixing super anyone? (http://mail.python.org/pipermail/python-3000/2007-April/006667.html) Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From ironfroggy at gmail.com Sun Apr 29 04:49:32 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sat, 28 Apr 2007 22:49:32 -0400 Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <4633F38A.4040605@v.loewis.de> References: <463282D5.4040008@v.loewis.de> <208553.28313.qm@web56614.mail.re3.yahoo.com> <76fd5acf0704280543s75e08073m485f80404db09fe2@mail.gmail.com> <4633F38A.4040605@v.loewis.de> Message-ID: <76fd5acf0704281949j52f09dd8ibbb14c474de1bb16@mail.gmail.com> On 4/28/07, "Martin v. L?wis" wrote: > > I have a patch myself that creates an open file and uses that as the > > test. My reasoning is that pagefile.sys was chosen as a file that > > should always exist and be open, so its safe to test against, so we > > should just be testing against a fixture, instead. It is here, and if > > someone would reference a SF bug report, I'll attach to it, as well. > > There must be more to the problem than just an open file. Please undo > the change that triggered the addition of the test, and see whether you > can reproduce the original problem with an arbitrary open file (I > could trigger the problem with pagefile.sys at the time). > > Regards, > Martin I'm sorry, but somehow I could not parse this. My understanding was that the unittest was meant to make sure an os.stat call would be successful on an open file, and that pagefile.sys was simply used as a known open file, which is no longer correct. If that is the case, I am unsure what problem there is with my fix of a temporary open file to test upon. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From rhamph at gmail.com Sun Apr 29 06:22:37 2007 From: rhamph at gmail.com (Adam Olsen) Date: Sat, 28 Apr 2007 22:22:37 -0600 Subject: [Python-Dev] New Super PEP In-Reply-To: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> Message-ID: On 4/28/07, Calvin Spealman wrote: > Comments welcome, of course. Bare with my first attempt at crafting a PEP. > > PEP: XXX > Title: Super As A Keyword > Version: $Revision$ > Last-Modified: $Date$ > Author: Calvin Spealman > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 30-Apr-2007 > Python-Version: 2.6 > Post-History: You need a section on alternate proposals. -- Adam Olsen, aka Rhamphoryncus From collinw at gmail.com Sun Apr 29 06:38:29 2007 From: collinw at gmail.com (Collin Winter) Date: Sat, 28 Apr 2007 21:38:29 -0700 Subject: [Python-Dev] New Super PEP In-Reply-To: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> Message-ID: <43aa6ff70704282138h6b9c1b43o6b9f79d090062f0a@mail.gmail.com> On 4/28/07, Calvin Spealman wrote: > Comments welcome, of course. Bare with my first attempt at crafting a PEP. > > PEP: XXX > Title: Super As A Keyword > Version: $Revision$ > Last-Modified: $Date$ > Author: Calvin Spealman > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 30-Apr-2007 > Python-Version: 2.6 [snip] > Which means the backward compatible API, which involves instansiating the super > type, will either not be possible, because it will actually do a super lookup > on the __call__ attribute, or there will be no way to perform a super lookup on > the __call__ attribute. Both seem unacceptable, so any suggestions are welcome. You're offering absolutely zero backwards compatibility and you're targeting 2.6? Um, no; absolutely not. Even if you intend this for 3.0, you'll still need to define either a backwards compatibility solution or a migration strategy (e.g., a 2to3 fixer). Without a clear-cut way of addressing existing code, this idea is toast. Collin Winter From khabkr at yahoo.com Sun Apr 29 06:40:32 2007 From: khabkr at yahoo.com (Khalid A. Bakr) Date: Sat, 28 Apr 2007 21:40:32 -0700 (PDT) Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <4633F38A.4040605@v.loewis.de> Message-ID: <242497.34958.qm@web56609.mail.re3.yahoo.com> --- "Martin v. L?wis" wrote: > There must be more to the problem than just an open > file. Please undo the change that triggered the > addition of the test, and see whether you > can reproduce the original problem with an arbitrary > open file (I > could trigger the problem with pagefile.sys at the > time). > > Regards, > Martin --- "Calvin Spealman" wrote: > I'm sorry, but somehow I could not parse this. My > understanding was that the unittest was meant to > make sure an os.stat call would be successful on an > open file, and that pagefile.sys was >simply used as > a known open file, which is no longer correct. If > that is the case, I am unsure what problem there is > with >my fix of a temporary open file to test upon I think the point is that the problem should be solved (stat of open file) for any arbitrary open file including pagefile.sys. After booting into Win98, I can see that I have a pagefile.sys indeed in my C drive which WinXP hides from view. While in Win98 and with the file not in use, the test passes on Win98 when looking for pagefile.sys in C drive (no complaint about file not found or access denied, even though I know that the file is not open). And the test passes for the running python.exe that is processing the test. After some googling it seems to me that this could likely be a User Rights Assignment issue of a systems file not an open file stat one, hence the Access denied error message (winerror 5) that I got in WinXP, as opposed to the File not found windows error (winerror 2) which one might expect if the pagefile.sys did not exist. And therefore if the point of the test is just to test stating an open file then the temporary file approach makes sense. If modifying a systems file with or without User Rights Assignment is a requirement then we may need a new test altogether. Regards, Khalid __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From khabkr at yahoo.com Sun Apr 29 06:49:59 2007 From: khabkr at yahoo.com (Khalid A. Bakr) Date: Sat, 28 Apr 2007 21:49:59 -0700 (PDT) Subject: [Python-Dev] Python 2.5.1 Message-ID: <873884.30122.qm@web56613.mail.re3.yahoo.com> For reference, this is the result of running the regression tests of the official Python 2.5.1 (final) on Win98. I think I saw it in the installtion screen that Python 2.5 is the last release to support Win98. Even though the unicode tests failing might be expected, what interested me was the fact that test_1565150 of test_os failed. Details follow. 259 tests OK. 8 tests failed: test_anydbm test_bsddb test_mailbox test_os test_shelve test_unicode_file test_uuid test_whichdb 54 tests skipped: test__locale test_aepack test_al test_applesingle test_bsddb185 test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_commands test_crypt test_curses test_dbm test_dl test_fcntl test_fork1 test_gdbm test_gl test_grp test_imgfile test_ioctl test_largefile test_linuxaudiodev test_macfs test_macostools test_mhlib test_nis test_normalization test_openpty test_ossaudiodev test_pep277 test_plistlib test_poll test_posix test_pty test_pwd test_resource test_scriptpackages test_signal test_socket_ssl test_socketserver test_sunaudiodev test_tcl test_threadsignals test_timeout test_urllib2net test_urllibnet test_wait3 test_wait4 test_zipfile64 1 skip unexpected on win32: test_tcl find_library('c') -> None find_library('m') -> None a DOS box should flash briefly ... $ python -i Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> from test import test_os as t >>> t.test_main() test_access (test.test_os.FileTests) ... ok test_tempnam (test.test_os.TemporaryFileTests) ... ok test_tmpfile (test.test_os.TemporaryFileTests) ... ok test_tmpnam (test.test_os.TemporaryFileTests) ... ok test_1565150 (test.test_os.StatAttributeTests) ... FAIL test_1686475 (test.test_os.StatAttributeTests) ... ok test_stat_attributes (test.test_os.StatAttributeTests) ... ok test_statvfs_attributes (test.test_os.StatAttributeTests) ... ok test_bool (test.test_os.EnvironTests) ... ok test_constructor (test.test_os.EnvironTests) ... ok test_get (test.test_os.EnvironTests) ... ok test_getitem (test.test_os.EnvironTests) ... ok test_items (test.test_os.EnvironTests) ... ok test_keys (test.test_os.EnvironTests) ... ok test_len (test.test_os.EnvironTests) ... ok test_pop (test.test_os.EnvironTests) ... ok test_popitem (test.test_os.EnvironTests) ... ok test_read (test.test_os.EnvironTests) ... ok test_setdefault (test.test_os.EnvironTests) ... ok test_update (test.test_os.EnvironTests) ... ok test_update2 (test.test_os.EnvironTests) ... ok test_values (test.test_os.EnvironTests) ... ok test_write (test.test_os.EnvironTests) ... ok test_traversal (test.test_os.WalkTests) ... ok test_makedir (test.test_os.MakedirTests) ... ok test_devnull (test.test_os.DevNullTests) ... ok test_urandom (test.test_os.URandomTests) ... ok test_access (test.test_os.Win32ErrorTests) ... ok test_chdir (test.test_os.Win32ErrorTests) ... ok test_chmod (test.test_os.Win32ErrorTests) ... ok test_mkdir (test.test_os.Win32ErrorTests) ... ok test_remove (test.test_os.Win32ErrorTests) ... ok test_rename (test.test_os.Win32ErrorTests) ... ok test_utime (test.test_os.Win32ErrorTests) ... ok ====================================================================== FAIL: test_1565150 (test.test_os.StatAttributeTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "G:\GMISC\PY25\lib\test\test_os.py", line 232, in test_1565150 self.assertEquals(os.stat(self.fname).st_mtime, t1) AssertionError: 1159195040.0 != 1159195039.25 ---------------------------------------------------------------------- Ran 34 tests in 0.440s FAILED (failures=1) Traceback (most recent call last): File "", line 1, in File "G:\GMISC\PY25\lib\test\test_os.py", line 434, in test_main Win32ErrorTests File "G:\GMISC\PY25\lib\test\test_support.py", line 441, in run_unittest run_suite(suite, testclass) File "G:\GMISC\PY25\lib\test\test_support.py", line 426, in run_suite raise TestFailed(err) test.test_support.TestFailed: Traceback (most recent call last): File "G:\GMISC\PY25\lib\test\test_os.py", line 232, in test_1565150 self.assertEquals(os.stat(self.fname).st_mtime, t1) AssertionError: 1159195040.0 != 1159195039.25 >>> >>> >>> Regards, Khalid __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From foom at fuhm.net Sun Apr 29 06:12:12 2007 From: foom at fuhm.net (James Y Knight) Date: Sun, 29 Apr 2007 00:12:12 -0400 Subject: [Python-Dev] New Super PEP In-Reply-To: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> Message-ID: On Apr 28, 2007, at 10:43 PM, Calvin Spealman wrote: > Abstract > ======== > > The PEP defines the proposal to enhance the super builtin to work > implicitly > upon the class within which it is used and upon the instance the > current > function was called on. The premise of the new super usage > suggested is as > follows: > > super.foo(1, 2) > > to replace the old: > > super(Foo, self).foo(1, 2) > > > Rationale > ========= > > The current usage of super requires an explicit passing of both the > class and > instance it must operate from, requiring a breaking of the DRY > (Don't Repeat > Yourself) rule. This hinders any change in class name, and is often > considered > a wart by many. This is only a halfway fix to DRY, and it really only fixes the less important half. The important problem with super is that it encourages people to write incorrect code by requiring that you explicitly specify an argument list. Since calling super with any arguments other than the exact same arguments you have received is nearly always wrong, requiring that the arglist be specified is an attractive nuisance. Now, I'm no syntax designer, but, just being able to say "super()" seems nice to me. (but don't get too hung up on that spelling, the concept of not having to repeat the arglist is the important point.) James From jcarlson at uci.edu Sun Apr 29 07:04:59 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat, 28 Apr 2007 22:04:59 -0700 Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <242497.34958.qm@web56609.mail.re3.yahoo.com> References: <4633F38A.4040605@v.loewis.de> <242497.34958.qm@web56609.mail.re3.yahoo.com> Message-ID: <20070428220126.643C.JCARLSON@uci.edu> "Khalid A. Bakr" wrote: > > --- "Martin v. L?wis" wrote: > > There must be more to the problem than just an open > > file. Please undo the change that triggered the > > addition of the test, and see whether you > > can reproduce the original problem with an arbitrary > > open file (I > > could trigger the problem with pagefile.sys at the > > time). > > > > Regards, > > Martin > > --- "Calvin Spealman" wrote: > > I'm sorry, but somehow I could not parse this. My > > understanding was that the unittest was meant to > > make sure an os.stat call would be successful on an > > open file, and that pagefile.sys was >simply used as > > a known open file, which is no longer correct. If > > that is the case, I am unsure what problem there is > > with >my fix of a temporary open file to test upon > > > I think the point is that the problem should be solved > (stat of open file) for any arbitrary open file > including pagefile.sys. On Windows there is no guarantee that there will be a pagefile.sys on the C drive, or even that there exists a C drive. The test checking for the result of os.stat('C:\\pagefile.sys') is broken. Create a temporary file, open it with Python, then stat it (like you later suggest). Either that or use sys.executable . Either one would likely be fine. - Josiah From talin at acm.org Sun Apr 29 09:16:06 2007 From: talin at acm.org (Talin) Date: Sun, 29 Apr 2007 00:16:06 -0700 Subject: [Python-Dev] New Super PEP In-Reply-To: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> Message-ID: <46344636.8000403@acm.org> Calvin Spealman wrote: > Comments welcome, of course. Bare with my first attempt at crafting a PEP. See below for comments; In general, I'm having problems understanding some of the terms used. I don't have any comments on the technical merits of the PEP yet, since I don't completely understand what is being said. > PEP: XXX > Title: Super As A Keyword > Version: $Revision$ > Last-Modified: $Date$ > Author: Calvin Spealman > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 30-Apr-2007 > Python-Version: 2.6 > Post-History: > > > Abstract > ======== > > The PEP defines the proposal to enhance the super builtin to work implicitly > upon the class within which it is used and upon the instance the current > function was called on. The premise of the new super usage suggested is as > follows: > > super.foo(1, 2) > > to replace the old: > > super(Foo, self).foo(1, 2) > > > Rationale > ========= > > The current usage of super requires an explicit passing of both the class and > instance it must operate from, requiring a breaking of the DRY (Don't Repeat > Yourself) rule. This hinders any change in class name, and is often considered > a wart by many. > > > Specification > ============= > > Replacing the old usage of super, calls to the next class in the MRO (method > resolution order) will be made without an explicit super object creation, > by simply accessing an attribute on the super type directly, which will > automatically apply the class and instance to perform the proper lookup. The > following example demonstrates the use of this. I don't understand the phrase 'by simply accessing an attribute on the super type directly'. See below for for more detail. > :: > > class A(object): > def f(self): > return 'A' > > class B(A): > def f(self): > return 'B' + super.f() > > class C(A): > def f(self): > return 'C' + super.f() > > class D(B, C): > def f(self): > return 'D' + super.f() > > assert D().f() == 'DBCA' > The example is clear enough. > The proposal adds a dynamic attribute lookup to the super type, which will > automatically determine the proper class and instance parameters. Each super > attribute lookup identifies these parameters and performs the super lookup on > the instance, as the current super implementation does with the explicit > invokation of a super object upon a class and instance. When you say 'the super type' I'm not sure what you mean. Do you mean the next class in the MRO, or the base class in which the super method is defined? Or something else? What defines the 'proper' class? Can we have a definition of what a "super object" is? > The enhancements to the super type will define a new __getattr__ classmethod > of the super type, which must look backwards to the previous frame and locate > the instance object. This can be naively determined by located the local named > by the first argument to the function. Using super outside of a function where > this is a valid lookup for the instance can be considered undocumented in its > behavior. As I am reading this I get the impression that the phrase 'the super type' is actually referring to the 'super' keyword itself - for example, you say that the super type has a new __getattr__ classmethod, which I read as saying that you can now say "super.x". > Every class will gain a new special attribute, __super__, which is a super > object instansiated only with the class it is an attribute of. In this > capacity, the new super also acts as its own descriptor, create an instance- > specific super upon lookup. I'm trying to parse that first sentence. How about "Every class will gain a new special attribute, __super__, which refers to an instance of the associated super object for that class". What does the phrase 'the new super' refer to - they keyword 'super', the super type, or the super object? > Much of this was discussed in the thread of the python-dev list, "Fixing super > anyone?" [1]_. > > Open Issues > ----------- > > __call__ methods > '''''''''''''''' > > Backward compatability of the super type API raises some issues. Names, the > lookup of the __call__ of the super type itself, which means a conflict with > doing an actual super lookup of the __call__ attribute. Namely, the following > is ambiguous in the current proposal: > > :: > > super.__call__(arg) > > Which means the backward compatible API, which involves instansiating the super > type, will either not be possible, because it will actually do a super lookup > on the __call__ attribute, or there will be no way to perform a super lookup on > the __call__ attribute. Both seem unacceptable, so any suggestions are welcome. > > super type's new getattr > '''''''''''''''''''''''' > > To give the behavior needed, the super type either needs a way to do dynamic > lookup of attributes on the super type object itself or define a metaclass for > the builtin type. This author is unsure which, if either, is possible with C- > defined types. > > When should we create __super__ attributes? > ''''''''''''''''''''''''''''''''''''''''''' > > They either need to be created on class creation or on __super__ attribute > lookup. For the second, they could be cached, of course, which seems like it > may be the best idea, if implicit creation of a super object for every class is > considered too much overhead. > > > References > ========== > > .. [1] Fixing super anyone? > (http://mail.python.org/pipermail/python-3000/2007-April/006667.html) > > > Copyright > ========= > > This document has been placed in the public domain. > > > > .. > Local Variables: > mode: indented-text > indent-tabs-mode: nil > sentence-end-double-space: t > fill-column: 70 > coding: utf-8 > End: > > From martin at v.loewis.de Sun Apr 29 09:44:57 2007 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 29 Apr 2007 09:44:57 +0200 Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <76fd5acf0704281949j52f09dd8ibbb14c474de1bb16@mail.gmail.com> References: <463282D5.4040008@v.loewis.de> <208553.28313.qm@web56614.mail.re3.yahoo.com> <76fd5acf0704280543s75e08073m485f80404db09fe2@mail.gmail.com> <4633F38A.4040605@v.loewis.de> <76fd5acf0704281949j52f09dd8ibbb14c474de1bb16@mail.gmail.com> Message-ID: <46344CF9.6090500@v.loewis.de> > I'm sorry, but somehow I could not parse this. My understanding was > that the unittest was meant to make sure an os.stat call would be > successful on an open file, and that pagefile.sys was simply used as a > known open file, which is no longer correct. No. The unit test was meant to test that os.stat is successful on an open file on which 2.5 reported ERROR_SHARING_VIOLATION. I believe it was not the case that it would report ERROR_SHARING_VIOLATION when stat'ing arbitrary open files, but just open files where some additional conditions must be met. I might be misremembering - I *think* I tried to write the test case just like you did, and it could not trigger the bug. I then concluded that I could not figure out what these additional conditions were, and that it *had* to be the pagefile. So please verify that your new test indeed breaks on 2.5.0 (or undo r54686). Regards, Martin From martin at v.loewis.de Sun Apr 29 09:53:07 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 29 Apr 2007 09:53:07 +0200 Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <242497.34958.qm@web56609.mail.re3.yahoo.com> References: <242497.34958.qm@web56609.mail.re3.yahoo.com> Message-ID: <46344EE3.9080003@v.loewis.de> > After some googling it seems to me that this could > likely be a User Rights Assignment issue of a systems > file not an open file stat one, hence the Access > denied error message (winerror 5) that I got in WinXP, > as opposed to the File not found windows error > (winerror 2) which one might expect if the > pagefile.sys did not exist. Ah - I didn't check what the error number is. If you can get ERROR_ACCESS_DENIED, then most likely we need to check for ERROR_FILE_NOT_FOUND as well (as your original patch did) - it would be good if somebody confirmed that the modified test indeed passes/gets skipped when the pagefile is not on C: > And therefore if the point of the test is just to test > stating an open file then the temporary file approach > makes sense. If modifying a systems file with or > without User Rights Assignment is a requirement then > we may need a new test altogether. The test should ideally verify that you can stat all open files in 2.5 that you could also stat in 2.4. If you get ERROR_ACCESS_DENIED when stat'ing c:\pagefile.sys in 2.5, I would *hope* that you get a similar error from 2.4. If 2.4 could stat c:\pagefile.sys and 2.5 gives ERROR_ACCESS_DENIED, then the bug still isn't fixed. Regards, Martin From martin at v.loewis.de Sun Apr 29 09:59:47 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 29 Apr 2007 09:59:47 +0200 Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <873884.30122.qm@web56613.mail.re3.yahoo.com> References: <873884.30122.qm@web56613.mail.re3.yahoo.com> Message-ID: <46345073.9020504@v.loewis.de> Khalid A. Bakr schrieb: > For reference, this is the result of running the > regression tests of the official Python 2.5.1 (final) > on Win98. I think I saw it in the installtion screen > that Python 2.5 is the last release to support Win98. > > Even though the unicode tests failing might be > expected, what interested me was the fact that > test_1565150 of test_os failed. Details follow. Ah, ok. This means that the test is bogus. On Win95 (and, strictly speaking, NT+ installed on FAT), the test will certainly fail, because FAT has no subsecond resolution for file modification times (it doesn't even have second resolution - the granularity is 2s). This failure does not bother me much - os.stat itself works just fine on this system. If you want to contribute a patch to skip the test on W9x, please go ahead - but IMO, there is no harm done having this test fail on W9x for the rest of 2.5.x. Regards, Martin From martin at v.loewis.de Sun Apr 29 10:02:47 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 29 Apr 2007 10:02:47 +0200 Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <20070428220126.643C.JCARLSON@uci.edu> References: <4633F38A.4040605@v.loewis.de> <242497.34958.qm@web56609.mail.re3.yahoo.com> <20070428220126.643C.JCARLSON@uci.edu> Message-ID: <46345127.1020504@v.loewis.de> > On Windows there is no guarantee that there will be a pagefile.sys on > the C drive, or even that there exists a C drive. The test checking for > the result of os.stat('C:\\pagefile.sys') is broken. Create a temporary > file, open it with Python, then stat it (like you later suggest). > Either that or use sys.executable . Either one would likely be fine. As I said - I'm not convinced that is indeed correct. Before accepting a replacement test I would like confirmation that this test will fail on 2.5.0. You might not get ERROR_SHARING_VIOLATION in all cases of open files with 2.5.0. Regards, Martin From gjcarneiro at gmail.com Sun Apr 29 13:02:23 2007 From: gjcarneiro at gmail.com (Gustavo Carneiro) Date: Sun, 29 Apr 2007 12:02:23 +0100 Subject: [Python-Dev] New Super PEP In-Reply-To: References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> Message-ID: On 29/04/07, James Y Knight wrote: > > > On Apr 28, 2007, at 10:43 PM, Calvin Spealman wrote: > > Abstract > > ======== > > > > The PEP defines the proposal to enhance the super builtin to work > > implicitly > > upon the class within which it is used and upon the instance the > > current > > function was called on. The premise of the new super usage > > suggested is as > > follows: > > > > super.foo(1, 2) > > > > to replace the old: > > > > super(Foo, self).foo(1, 2) > > > > > > Rationale > > ========= > > > > The current usage of super requires an explicit passing of both the > > class and > > instance it must operate from, requiring a breaking of the DRY > > (Don't Repeat > > Yourself) rule. This hinders any change in class name, and is often > > considered > > a wart by many. > > This is only a halfway fix to DRY, and it really only fixes the less > important half. The important problem with super is that it > encourages people to write incorrect code by requiring that you > explicitly specify an argument list. Since calling super with any > arguments other than the exact same arguments you have received is > nearly always wrong, Erm. Excuse me, but are you saying this code is wrong? class Rectangle: def __init__(self, width, height): self.width = width self.height = height class Square: def __init__(self, side): Rectangle.__init__(self, side, side) Or are you even saying this type of code is rare? I would disagree with both statements, therefore I also disagree with your recommendation. -- Gustavo J. A. M. Carneiro "The universe is always one step beyond logic." -- Frank Herbert -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070429/5419126b/attachment.html From l.mastrodomenico at gmail.com Sun Apr 29 13:26:20 2007 From: l.mastrodomenico at gmail.com (Lino Mastrodomenico) Date: Sun, 29 Apr 2007 13:26:20 +0200 Subject: [Python-Dev] New Super PEP In-Reply-To: References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> Message-ID: 2007/4/29, Gustavo Carneiro : > On 29/04/07, James Y Knight wrote: > > Since calling super with any > > arguments other than the exact same arguments you have received is > > nearly always wrong, > > Erm. Excuse me, but are you saying this code is wrong? > > class Rectangle: > def __init__(self, width, height): > self.width = width > self.height = height > > class Square: > def __init__(self, side): > Rectangle.__init__(self, side, side) You probably mean "class Square(Rectangle):". Anyway it's not wrong, but it isn't multiple-inheritance-friendly either. -- Lino Mastrodomenico E-mail: l.mastrodomenico at gmail.com From skip at pobox.com Sun Apr 29 13:39:09 2007 From: skip at pobox.com (skip at pobox.com) Date: Sun, 29 Apr 2007 06:39:09 -0500 Subject: [Python-Dev] New Super PEP In-Reply-To: References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> Message-ID: <17972.33757.278659.977298@montanaro.dyndns.org> James> This is only a halfway fix to DRY, and it really only fixes the James> less important half. The important problem with super is that it James> encourages people to write incorrect code by requiring that you James> explicitly specify an argument list. Since calling super with any James> arguments other than the exact same arguments you have received James> is nearly always wrong, requiring that the arglist be specified James> is an attractive nuisance. Since the language doesn't require that a subclassed method take the same parameters as the base class method, you can't assume that it does. super() should simply mean "call with no arguments". From ironfroggy at gmail.com Sun Apr 29 14:20:26 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sun, 29 Apr 2007 08:20:26 -0400 Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <46344CF9.6090500@v.loewis.de> References: <463282D5.4040008@v.loewis.de> <208553.28313.qm@web56614.mail.re3.yahoo.com> <76fd5acf0704280543s75e08073m485f80404db09fe2@mail.gmail.com> <4633F38A.4040605@v.loewis.de> <76fd5acf0704281949j52f09dd8ibbb14c474de1bb16@mail.gmail.com> <46344CF9.6090500@v.loewis.de> Message-ID: <76fd5acf0704290520h50dd01f9y750d23acc1f1840@mail.gmail.com> On 4/29/07, "Martin v. L?wis" wrote: > > I'm sorry, but somehow I could not parse this. My understanding was > > that the unittest was meant to make sure an os.stat call would be > > successful on an open file, and that pagefile.sys was simply used as a > > known open file, which is no longer correct. > > No. The unit test was meant to test that os.stat is successful on an > open file on which 2.5 reported ERROR_SHARING_VIOLATION. I believe > it was not the case that it would report ERROR_SHARING_VIOLATION > when stat'ing arbitrary open files, but just open files where some > additional conditions must be met. I might be misremembering - I > *think* I tried to write the test case just like you did, and it > could not trigger the bug. I then concluded that I could not figure > out what these additional conditions were, and that it *had* to > be the pagefile. > > So please verify that your new test indeed breaks on 2.5.0 (or > undo r54686). > > Regards, > Martin Some record of this or documentation of just what conditions the tests are expecting to test against would probably be a good idea. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From ironfroggy at gmail.com Sun Apr 29 14:22:25 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sun, 29 Apr 2007 08:22:25 -0400 Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <46345127.1020504@v.loewis.de> References: <4633F38A.4040605@v.loewis.de> <242497.34958.qm@web56609.mail.re3.yahoo.com> <20070428220126.643C.JCARLSON@uci.edu> <46345127.1020504@v.loewis.de> Message-ID: <76fd5acf0704290522u674118eai97c4a8fe778243b4@mail.gmail.com> On 4/29/07, "Martin v. L?wis" wrote: > > On Windows there is no guarantee that there will be a pagefile.sys on > > the C drive, or even that there exists a C drive. The test checking for > > the result of os.stat('C:\\pagefile.sys') is broken. Create a temporary > > file, open it with Python, then stat it (like you later suggest). > > Either that or use sys.executable . Either one would likely be fine. > > As I said - I'm not convinced that is indeed correct. Before accepting > a replacement test I would like confirmation that this test will fail > on 2.5.0. You might not get ERROR_SHARING_VIOLATION in all cases of > open files with 2.5.0. But i am running 2.5.0 during my entire writing of this patch. I've not upgraded my laptop to 2.5.1 yet, although I verified against another installation of 2.5.1 before publishing. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From martin at v.loewis.de Sun Apr 29 14:49:20 2007 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 29 Apr 2007 14:49:20 +0200 Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <76fd5acf0704290520h50dd01f9y750d23acc1f1840@mail.gmail.com> References: <463282D5.4040008@v.loewis.de> <208553.28313.qm@web56614.mail.re3.yahoo.com> <76fd5acf0704280543s75e08073m485f80404db09fe2@mail.gmail.com> <4633F38A.4040605@v.loewis.de> <76fd5acf0704281949j52f09dd8ibbb14c474de1bb16@mail.gmail.com> <46344CF9.6090500@v.loewis.de> <76fd5acf0704290520h50dd01f9y750d23acc1f1840@mail.gmail.com> Message-ID: <46349450.1040200@v.loewis.de> > Some record of this or documentation of just what conditions the tests > are expecting to test against would probably be a good idea. There is the sourceforge tracker item. If that is insufficient, feel free to add more information. Regards, Martin From martin at v.loewis.de Sun Apr 29 14:50:52 2007 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 29 Apr 2007 14:50:52 +0200 Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <76fd5acf0704290522u674118eai97c4a8fe778243b4@mail.gmail.com> References: <4633F38A.4040605@v.loewis.de> <242497.34958.qm@web56609.mail.re3.yahoo.com> <20070428220126.643C.JCARLSON@uci.edu> <46345127.1020504@v.loewis.de> <76fd5acf0704290522u674118eai97c4a8fe778243b4@mail.gmail.com> Message-ID: <463494AC.1080608@v.loewis.de> >> As I said - I'm not convinced that is indeed correct. Before accepting >> a replacement test I would like confirmation that this test will fail >> on 2.5.0. You might not get ERROR_SHARING_VIOLATION in all cases of >> open files with 2.5.0. > > But i am running 2.5.0 during my entire writing of this patch. I've > not upgraded my laptop to 2.5.1 yet, although I verified against > another installation of 2.5.1 before publishing. And you saw your test pass? Then it is not a valid test case for the bug being test, because the bug is present in 2.5.0, so your test case should fail there. Regards, Martin From martin at v.loewis.de Sun Apr 29 14:53:57 2007 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 29 Apr 2007 14:53:57 +0200 Subject: [Python-Dev] New Super PEP In-Reply-To: References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> Message-ID: <46349565.2010602@v.loewis.de> > Since calling super with any > arguments other than the exact same arguments you have received is > nearly always wrong, > > > Erm. Excuse me, but are you saying this code is wrong? > > class Rectangle: > def __init__(self, width, height): > self.width = width > self.height = height > > class Square: > def __init__(self, side): > Rectangle.__init__(self, side, side) That's not what he said. Your code does not call super(), so the observation that it normally should pass the exact same arguments does not apply. Regards, Martin From ironfroggy at gmail.com Sun Apr 29 15:29:43 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sun, 29 Apr 2007 09:29:43 -0400 Subject: [Python-Dev] New Super PEP In-Reply-To: <46344636.8000403@acm.org> References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> <46344636.8000403@acm.org> Message-ID: <76fd5acf0704290629t1d669a9an46640bd491bd118d@mail.gmail.com> Yes, I bad wordly things did. Fix will I. On 4/29/07, Talin wrote: > Calvin Spealman wrote: > > Comments welcome, of course. Bare with my first attempt at crafting a PEP. > > See below for comments; In general, I'm having problems understanding > some of the terms used. I don't have any comments on the technical > merits of the PEP yet, since I don't completely understand what is being > said. > > > PEP: XXX > > Title: Super As A Keyword > > Version: $Revision$ > > Last-Modified: $Date$ > > Author: Calvin Spealman > > Status: Draft > > Type: Standards Track > > Content-Type: text/x-rst > > Created: 30-Apr-2007 > > Python-Version: 2.6 > > Post-History: > > > > > > Abstract > > ======== > > > > The PEP defines the proposal to enhance the super builtin to work implicitly > > upon the class within which it is used and upon the instance the current > > function was called on. The premise of the new super usage suggested is as > > follows: > > > > super.foo(1, 2) > > > > to replace the old: > > > > super(Foo, self).foo(1, 2) > > > > > > Rationale > > ========= > > > > The current usage of super requires an explicit passing of both the class and > > instance it must operate from, requiring a breaking of the DRY (Don't Repeat > > Yourself) rule. This hinders any change in class name, and is often considered > > a wart by many. > > > > > > Specification > > ============= > > > > Replacing the old usage of super, calls to the next class in the MRO (method > > resolution order) will be made without an explicit super object creation, > > by simply accessing an attribute on the super type directly, which will > > automatically apply the class and instance to perform the proper lookup. The > > following example demonstrates the use of this. > > I don't understand the phrase 'by simply accessing an attribute on the > super type directly'. See below for for more detail. > > > :: > > > > class A(object): > > def f(self): > > return 'A' > > > > class B(A): > > def f(self): > > return 'B' + super.f() > > > > class C(A): > > def f(self): > > return 'C' + super.f() > > > > class D(B, C): > > def f(self): > > return 'D' + super.f() > > > > assert D().f() == 'DBCA' > > > > The example is clear enough. > > > The proposal adds a dynamic attribute lookup to the super type, which will > > automatically determine the proper class and instance parameters. Each super > > attribute lookup identifies these parameters and performs the super lookup on > > the instance, as the current super implementation does with the explicit > > invokation of a super object upon a class and instance. > > When you say 'the super type' I'm not sure what you mean. Do you mean > the next class in the MRO, or the base class in which the super method > is defined? Or something else? What defines the 'proper' class? There is some ambiguous nature there that I missed. I think I used the same term for both the super type, as in the actual builtin type named "super" and to refer to the next type in the MRO order after the current type. What proper terminology would differentiate? > Can we have a definition of what a "super object" is? An instance of the builtin type named "super". Something for which isinstance(o, super) is True. > > The enhancements to the super type will define a new __getattr__ classmethod > > of the super type, which must look backwards to the previous frame and locate > > the instance object. This can be naively determined by located the local named > > by the first argument to the function. Using super outside of a function where > > this is a valid lookup for the instance can be considered undocumented in its > > behavior. > > As I am reading this I get the impression that the phrase 'the super > type' is actually referring to the 'super' keyword itself - for example, > you say that the super type has a new __getattr__ classmethod, which I > read as saying that you can now say "super.x". Yes. > > Every class will gain a new special attribute, __super__, which is a super > > object instansiated only with the class it is an attribute of. In this > > capacity, the new super also acts as its own descriptor, create an instance- > > specific super upon lookup. > > I'm trying to parse that first sentence. How about "Every class will > gain a new special attribute, __super__, which refers to an instance of > the associated super object for that class". Check. > What does the phrase 'the new super' refer to - they keyword 'super', > the super type, or the super object? "the new super" refers to the updated super type this PEP proposes. Instances of the new super type will act as their own descriptors, which is how they are bound to specific instances of the classes they are associated with. > > Much of this was discussed in the thread of the python-dev list, "Fixing super > > anyone?" [1]_. > > > > Open Issues > > ----------- > > > > __call__ methods > > '''''''''''''''' > > > > Backward compatability of the super type API raises some issues. Names, the > > lookup of the __call__ of the super type itself, which means a conflict with > > doing an actual super lookup of the __call__ attribute. Namely, the following > > is ambiguous in the current proposal: > > > > :: > > > > super.__call__(arg) > > > > Which means the backward compatible API, which involves instansiating the super > > type, will either not be possible, because it will actually do a super lookup > > on the __call__ attribute, or there will be no way to perform a super lookup on > > the __call__ attribute. Both seem unacceptable, so any suggestions are welcome. > > > > super type's new getattr > > '''''''''''''''''''''''' > > > > To give the behavior needed, the super type either needs a way to do dynamic > > lookup of attributes on the super type object itself or define a metaclass for > > the builtin type. This author is unsure which, if either, is possible with C- > > defined types. > > > > When should we create __super__ attributes? > > ''''''''''''''''''''''''''''''''''''''''''' > > > > They either need to be created on class creation or on __super__ attribute > > lookup. For the second, they could be cached, of course, which seems like it > > may be the best idea, if implicit creation of a super object for every class is > > considered too much overhead. > > > > > > References > > ========== > > > > .. [1] Fixing super anyone? > > (http://mail.python.org/pipermail/python-3000/2007-April/006667.html) > > > > > > Copyright > > ========= > > > > This document has been placed in the public domain. > > > > > > > > .. > > Local Variables: > > mode: indented-text > > indent-tabs-mode: nil > > sentence-end-double-space: t > > fill-column: 70 > > coding: utf-8 > > End: > > > > > I'll also add a section on alternate proposals. I'll look through the "Fixing super anyone?" thread for them, but are there any others I might not be aware of? If anyone has a suggestion for an alternative proposal that deserves documenting, let me know. I'll make the changes needed, and try to clarify the language more. Maybe It was too late when I wrote this. On 4/29/07, Collin Winter wrote: > On 4/28/07, Calvin Spealman wrote: > > Comments welcome, of course. Bare with my first attempt at crafting a PEP. > > Which means the backward compatible API, which involves instansiating the super > > type, will either not be possible, because it will actually do a super lookup > > on the __call__ attribute, or there will be no way to perform a super lookup on > > the __call__ attribute. Both seem unacceptable, so any suggestions are welcome. > > You're offering absolutely zero backwards compatibility and you're > targeting 2.6? Um, no; absolutely not. Even if you intend this for > 3.0, you'll still need to define either a backwards compatibility > solution or a migration strategy (e.g., a 2to3 fixer). Without a > clear-cut way of addressing existing code, this idea is toast. Obviously its probably the biggest problem. I'm not positive this should target 2.6, no one really seemed to mention a version in the previous discussions. This is specifically syntax that Guido said he wanted a way to do, and with backward compatibility. I would absolutely love for someone to figure out how to solve the __call__ problem, but I don't know how to do it without weakening either strategy. The super type itself needs changed for this to work, so maybe we can keep the old one around in 2.6, add a future import to bind the super name to the new version, and make the new version default in 3.0? That would let us have backwards compatibility without the two types interfering with each other, and simplify the implementation greatly. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From guido at python.org Sun Apr 29 17:04:06 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 29 Apr 2007 08:04:06 -0700 Subject: [Python-Dev] New Super PEP In-Reply-To: References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> Message-ID: On 4/28/07, James Y Knight wrote: > This is only a halfway fix to DRY, and it really only fixes the less > important half. The important problem with super is that it > encourages people to write incorrect code by requiring that you > explicitly specify an argument list. Since calling super with any > arguments other than the exact same arguments you have received is > nearly always wrong, requiring that the arglist be specified is an > attractive nuisance. Nearly always wrong? You must be kidding. There are tons of reasons to call your super method with modified arguments. E.g. clipping, transforming, ... -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ironfroggy at gmail.com Sun Apr 29 17:37:55 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sun, 29 Apr 2007 11:37:55 -0400 Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <463494AC.1080608@v.loewis.de> References: <4633F38A.4040605@v.loewis.de> <242497.34958.qm@web56609.mail.re3.yahoo.com> <20070428220126.643C.JCARLSON@uci.edu> <46345127.1020504@v.loewis.de> <76fd5acf0704290522u674118eai97c4a8fe778243b4@mail.gmail.com> <463494AC.1080608@v.loewis.de> Message-ID: <76fd5acf0704290837t1c57ede8wdb6e45ab6c6f3b95@mail.gmail.com> On 4/29/07, "Martin v. L?wis" wrote: > >> As I said - I'm not convinced that is indeed correct. Before accepting > >> a replacement test I would like confirmation that this test will fail > >> on 2.5.0. You might not get ERROR_SHARING_VIOLATION in all cases of > >> open files with 2.5.0. > > > > But i am running 2.5.0 during my entire writing of this patch. I've > > not upgraded my laptop to 2.5.1 yet, although I verified against > > another installation of 2.5.1 before publishing. > > And you saw your test pass? Then it is not a valid test case for the > bug being test, because the bug is present in 2.5.0, so your test > case should fail there. I think I'm a little confused. Are you saying the original test should fail for me or that the test I changed it to should fail for me? The original test failed, my new one does not. As for documentating the intent of these tests, I don't think tracker items are visible enough. When I'm looking at the unittest itself, am I to always search the entire tracker for any bugs still relevent and pertaining to each test I look at? That seems contorted, and easy to miss. I'll check the tracker, and I'd like to add any information to the test itself. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From martin at v.loewis.de Sun Apr 29 18:04:12 2007 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 29 Apr 2007 18:04:12 +0200 Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <76fd5acf0704290837t1c57ede8wdb6e45ab6c6f3b95@mail.gmail.com> References: <4633F38A.4040605@v.loewis.de> <242497.34958.qm@web56609.mail.re3.yahoo.com> <20070428220126.643C.JCARLSON@uci.edu> <46345127.1020504@v.loewis.de> <76fd5acf0704290522u674118eai97c4a8fe778243b4@mail.gmail.com> <463494AC.1080608@v.loewis.de> <76fd5acf0704290837t1c57ede8wdb6e45ab6c6f3b95@mail.gmail.com> Message-ID: <4634C1FC.7090408@v.loewis.de> >> And you saw your test pass? Then it is not a valid test case for >> the >> bug being test, because the bug is present in 2.5.0, so your >> test case should fail there. >> > > I think I'm a little confused. Are you saying the original test > should fail for me or that the test I changed it to should fail for > me? If "for me" means "in 2.5.0", then yes: both the original test and the one you modified should have failed. > The original test failed, my new one does not. Then this change is incorrect: the test should fail in 2.5.0. > As for documentating the intent of these tests, I don't think > tracker items are visible enough. Hmm. Is it asked too much to go to python.org/sf/1686475 when editing a test case named 'test_1686475'? When researching the intent of some piece of code, you actually have more information available: the set of changes that was committed together (which would include a Misc/NEWS change, and the actual change to posixmodule.c), and the log message. > When I'm looking at the unittest itself, am I to always search the > entire tracker for any bugs still relevent and pertaining to each > test I look at? That seems contorted, and easy to miss. I'll check > the tracker, and I'd like to add any information to the test > itself. Clearly, if you think some relevant information is missing in a comment, submit a patch to add that information. I was unable to add anything, because I did not know it was missing. Regards, Martin From collinw at gmail.com Sun Apr 29 18:06:27 2007 From: collinw at gmail.com (Collin Winter) Date: Sun, 29 Apr 2007 09:06:27 -0700 Subject: [Python-Dev] New Super PEP In-Reply-To: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> Message-ID: <43aa6ff70704290906p43b59ccdkaa2292ae615174bd@mail.gmail.com> On 4/28/07, Calvin Spealman wrote: [snip] > The PEP defines the proposal to enhance the super builtin to work implicitly > upon the class within which it is used and upon the instance the current > function was called on. The premise of the new super usage suggested is as > follows: > > super.foo(1, 2) > > to replace the old: > > super(Foo, self).foo(1, 2) [snip] > The enhancements to the super type will define a new __getattr__ classmethod > of the super type, which must look backwards to the previous frame and locate > the instance object. This can be naively determined by located the local named > by the first argument to the function. Using super outside of a function where > this is a valid lookup for the instance can be considered undocumented in its > behavior. What if the instance isn't called "self"? PEP 3099 states that "self will not become implicit"; it's talking about method signatures, but I think that dictum applies equally well in this case. Also, it's my understanding that not all Python implementations have an easy analogue to CPython's frames; have you given any thought to whether and how PyPy, IronPython, Jython, etc, will implement this? Collin Winter From ironfroggy at gmail.com Sun Apr 29 18:47:54 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sun, 29 Apr 2007 12:47:54 -0400 Subject: [Python-Dev] New Super PEP In-Reply-To: <43aa6ff70704290906p43b59ccdkaa2292ae615174bd@mail.gmail.com> References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> <43aa6ff70704290906p43b59ccdkaa2292ae615174bd@mail.gmail.com> Message-ID: <76fd5acf0704290947o79cb9722k66c8ba37fa0b6826@mail.gmail.com> On 4/29/07, Collin Winter wrote: > What if the instance isn't called "self"? PEP 3099 states that "self > will not become implicit"; it's talking about method signatures, but I > think that dictum applies equally well in this case. I don't use the name self. I use whatever the first argument name is, found by this line of python code: instance_name = calling_frame.f_code.co_varnames[0] > Also, it's my understanding that not all Python implementations have > an easy analogue to CPython's frames; have you given any thought to > whether and how PyPy, IronPython, Jython, etc, will implement this? I'll bring this up for input from PyPy and IronPython people, but I don't know any Jython people. Are we yet letting the alternative implementations influence so strongly what we do in CPython? I'm not saying "screw them", just pointing out that there is always a way to implement anything, and if its some trouble for them, well, 2.6 or 3.0 targetting is far down the road for any of them yet. I'll add a reference implementation I have to the second draft of the PEP. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From ironfroggy at gmail.com Sun Apr 29 18:53:40 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sun, 29 Apr 2007 12:53:40 -0400 Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <4634C1FC.7090408@v.loewis.de> References: <4633F38A.4040605@v.loewis.de> <242497.34958.qm@web56609.mail.re3.yahoo.com> <20070428220126.643C.JCARLSON@uci.edu> <46345127.1020504@v.loewis.de> <76fd5acf0704290522u674118eai97c4a8fe778243b4@mail.gmail.com> <463494AC.1080608@v.loewis.de> <76fd5acf0704290837t1c57ede8wdb6e45ab6c6f3b95@mail.gmail.com> <4634C1FC.7090408@v.loewis.de> Message-ID: <76fd5acf0704290953i6dcc0bffk98d49619ef922e1b@mail.gmail.com> On 4/29/07, "Martin v. L?wis" wrote: > > The > > original test failed, my new one does not. > > Then this change is incorrect: the test should fail in 2.5.0. I think I don't get why the test _must_ fail. If it fails, I assumed something was broken. If it failed because it was testing against a non-existant file, I assumed the test itself was broken. > > As for documentating the > > intent of these tests, I don't think tracker items are visible enough. > > Hmm. Is it asked too much to go to python.org/sf/1686475 when editing > a test case named 'test_1686475'? Maybe this is my flag for "I'm dumb sometimes", but I did wonder what the number was for and completely neglected to consider it being a ticket number! > When researching the intent of some piece of code, you actually have > more information available: the set of changes that was committed > together (which would include a Misc/NEWS change, and the actual > change to posixmodule.c), and the log message. > > > When I'm looking at the unittest itself, am I to always search the > > entire tracker for any bugs still relevent and pertaining to each test > > I look at? That seems contorted, and easy to miss. I'll check the > > tracker, and I'd like to add any information to the test itself. > > Clearly, if you think some relevant information is missing in a comment, > submit a patch to add that information. I was unable to add anything, > because I did not know it was missing. I will do so. Maybe even just a link to the tracker, because of the likelihood of me not being the only person to complete miss what the number in the test name is for. ... I've read the bug report now. I see what I was missing all along. I think maybe you thought I knew of the bug report, and thus we were both confused talking on different frequencies and completely missing each other, Martin. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From ironfroggy at gmail.com Sun Apr 29 19:19:10 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sun, 29 Apr 2007 13:19:10 -0400 Subject: [Python-Dev] New Super PEP In-Reply-To: <76fd5acf0704290947o79cb9722k66c8ba37fa0b6826@mail.gmail.com> References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> <43aa6ff70704290906p43b59ccdkaa2292ae615174bd@mail.gmail.com> <76fd5acf0704290947o79cb9722k66c8ba37fa0b6826@mail.gmail.com> Message-ID: <76fd5acf0704291019y3b30e3ebn2fff564bb71bc462@mail.gmail.com> Draft Attempt Number Duo: PEP: XXX Title: New Super Version: $Revision$ Last-Modified: $Date$ Author: Calvin Spealman Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 28-Apr-2007 Python-Version: 2.6 Post-History: 28-Apr-2007, 29-Apr-2007 Abstract ======== The PEP defines the proposal to enhance the super builtin to work implicitly upon the class within which it is used and upon the instance the current function was called on. The premise of the new super usage suggested is as follows: super.foo(1, 2) to replace the old: super(Foo, self).foo(1, 2) Rationale ========= The current usage of super requires an explicit passing of both the class and instance it must operate from, requiring a breaking of the DRY (Don't Repeat Yourself) rule. This hinders any change in class name, and is often considered a wart by many. Specification ============= Within the specification section, some special terminology will be used to distinguish similar and closely related concepts. "Super type" will refer to the actual builtin type named "super". "Next Class/Type in the MRO" will refer to the class where attribute lookups will be performed by super, for example, in the following, A is the "Next class in the MRO" for the use of super. :: class A(object): def f(self): return 'A' class B(A): def f(self): super(B, self).f() # Here, A would be out "Next class in the # MRO", of course. A "super object" is simply an instance of the super type, which is associated with a class and possibly with an instance of that class. Finally, "new super" refers to the new super type, which will replace the original. Replacing the old usage of super, calls to the next class in the MRO (method resolution order) will be made without an explicit super object creation, by simply accessing an attribute on the super type directly, which will automatically apply the class and instance to perform the proper lookup. The following example demonstrates the use of this. :: class A(object): def f(self): return 'A' class B(A): def f(self): return 'B' + super.f() class C(A): def f(self): return 'C' + super.f() class D(B, C): def f(self): return 'D' + super.f() assert D().f() == 'DBCA' The proposal adds a dynamic attribute lookup to the super type, which will automatically determine the proper class and instance parameters. Each super attribute lookup identifies these parameters and performs the super lookup on the instance, as the current super implementation does with the explicit invokation of a super object upon a class and instance. The enhancements to the super type will define a new __getattr__ classmethod of the super type, which must look backwards to the previous frame and locate the instance object. This can be naively determined by located the local named by the first argument to the function. Using super outside of a function where this is a valid lookup for the instance can be considered undocumented in its behavior. This special method will actually be invoked on attribute lookups to the super type itself, as opposed to super objects, as the current implementation works. This may pose open issues, which are detailed below. "Every class will gain a new special attribute, __super__, which refers to an instance of the associated super object for that class" In this capacity, the new super also acts as its own descriptor, create an instance-specific super upon lookup. Much of this was discussed in the thread of the python-dev list, "Fixing super anyone?" [1]_. Open Issues ----------- __call__ methods '''''''''''''''' Backward compatability of the super type API raises some issues. Names, the lookup of the __call__ of the super type itself, which means a conflict with doing an actual super lookup of the __call__ attribute. Namely, the following is ambiguous in the current proposal: :: super.__call__(arg) Which means the backward compatible API, which involves instansiating the super type, will either not be possible, because it will actually do a super lookup on the __call__ attribute, or there will be no way to perform a super lookup on the __call__ attribute. Both seem unacceptable, so any suggestions are welcome. Actually keeping the old super around in 2.x and creating a completely new super type seperately may be the best option. A future import or even a simple import in 2.x of the new super type from some builtin module may offer a way to choose which each module uses, even mixing uses by binding to different names. Such a builtin module might be called 'newsuper'. This module is also the reference implementation, which I will present below. super type's new getattr '''''''''''''''''''''''' To give the behavior needed, the super type either needs a way to do dynamic lookup of attributes on the super type object itself or define a metaclass for the builtin type. This author is unsure which, if either, is possible with C- defined types. When should we create __super__ attributes? ''''''''''''''''''''''''''''''''''''''''''' They either need to be created on class creation or on __super__ attribute lookup. For the second, they could be cached, of course, which seems like it may be the best idea, if implicit creation of a super object for every class is considered too much overhead. Reference Implementation ======================== This implementation was a cooperative contribution in the original thread [1]_. :: #!/usr/bin/env python # # newsuper.py import sys class SuperMetaclass(type): def __getattr__(cls, attr): calling_frame = sys._getframe().f_back instance_name = calling_frame.f_code.co_varnames[0] instance = calling_frame.f_locals[instance_name] return getattr(instance.__super__, attr) class Super(object): __metaclass__ = SuperMetaclass def __init__(self, type, obj=None): if isinstance(obj, Super): obj = obj.__obj__ self.__type__ = type self.__obj__ = obj def __get__(self, obj, cls=None): if obj is None: raise Exception('only supports instances') else: return Super(self.__type__, obj) def __getattr__(self, attr): mro = iter(self.__obj__.__class__.__mro__) for cls in mro: if cls is self.__type__: break for cls in mro: if attr in cls.__dict__: x = cls.__dict__[attr] if hasattr(x, '__get__'): x = x.__get__(self, cls) return x raise AttributeError, attr class autosuper(type): def __init__(cls, name, bases, clsdict): cls.__super__ = Super(cls) if __name__ == '__main__': class A(object): __metaclass__ = autosuper def f(self): return 'A' class B(A): def f(self): return 'B' + Super.f() class C(A): def f(self): return 'C' + Super.f() class D(B, C): def f(self, arg=None): var = None return 'D' + Super.f() assert D().f() == 'DBCA' History ======= 29-Apr-2007 - Changed title from "Super As A Keyword" to "New Super" - Updated much of the language and added a terminology section for clarification in confusing places. - Added reference implementation and history sections. References ========== .. [1] Fixing super anyone? (http://mail.python.org/pipermail/python-3000/2007-April/006667.html) Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: From ironfroggy at gmail.com Sun Apr 29 19:30:25 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sun, 29 Apr 2007 13:30:25 -0400 Subject: [Python-Dev] New Super PEP In-Reply-To: <76fd5acf0704291019y3b30e3ebn2fff564bb71bc462@mail.gmail.com> References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> <43aa6ff70704290906p43b59ccdkaa2292ae615174bd@mail.gmail.com> <76fd5acf0704290947o79cb9722k66c8ba37fa0b6826@mail.gmail.com> <76fd5acf0704291019y3b30e3ebn2fff564bb71bc462@mail.gmail.com> Message-ID: <76fd5acf0704291030u1447ddbcv691f1071d2d466ae@mail.gmail.com> On 4/29/07, Calvin Spealman wrote: > Draft Attempt Number Duo: > > PEP: XXX > Title: New Super > Version: $Revision$ > Last-Modified: $Date$ > Author: Calvin Spealman > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 28-Apr-2007 > Python-Version: 2.6 > Post-History: 28-Apr-2007, 29-Apr-2007 > > > Abstract > ======== > > The PEP defines the proposal to enhance the super builtin to work implicitly > upon the class within which it is used and upon the instance the current > function was called on. The premise of the new super usage suggested is as > follows: > > super.foo(1, 2) > > to replace the old: > > super(Foo, self).foo(1, 2) > > > Rationale > ========= > > The current usage of super requires an explicit passing of both the class and > instance it must operate from, requiring a breaking of the DRY (Don't Repeat > Yourself) rule. This hinders any change in class name, and is often considered > a wart by many. > > > Specification > ============= > > Within the specification section, some special terminology will be used to > distinguish similar and closely related concepts. "Super type" will refer to > the actual builtin type named "super". "Next Class/Type in the MRO" will refer > to the class where attribute lookups will be performed by super, for example, > in the following, A is the "Next class in the MRO" for the use of super. > > :: > > class A(object): > def f(self): > return 'A' > > class B(A): > def f(self): > super(B, self).f() # Here, A would be out "Next class in the > # MRO", of course. > > A "super object" is simply an instance of the super type, which is associated > with a class and possibly with an instance of that class. Finally, "new super" > refers to the new super type, which will replace the original. > > Replacing the old usage of super, calls to the next class in the MRO (method > resolution order) will be made without an explicit super object creation, > by simply accessing an attribute on the super type directly, which will > automatically apply the class and instance to perform the proper lookup. The > following example demonstrates the use of this. > > :: > > class A(object): > def f(self): > return 'A' > > class B(A): > def f(self): > return 'B' + super.f() > > class C(A): > def f(self): > return 'C' + super.f() > > class D(B, C): > def f(self): > return 'D' + super.f() > > assert D().f() == 'DBCA' > > The proposal adds a dynamic attribute lookup to the super type, which will > automatically determine the proper class and instance parameters. Each super > attribute lookup identifies these parameters and performs the super lookup on > the instance, as the current super implementation does with the explicit > invokation of a super object upon a class and instance. > > The enhancements to the super type will define a new __getattr__ classmethod > of the super type, which must look backwards to the previous frame and locate > the instance object. This can be naively determined by located the local named > by the first argument to the function. Using super outside of a function where > this is a valid lookup for the instance can be considered undocumented in its > behavior. This special method will actually be invoked on attribute lookups to > the super type itself, as opposed to super objects, as the current > implementation works. This may pose open issues, which are detailed below. > > "Every class will gain a new special attribute, __super__, which refers to an > instance of the associated super object for that class" In this capacity, the > new super also acts as its own descriptor, create an instance-specific super > upon lookup. > > Much of this was discussed in the thread of the python-dev list, "Fixing super > anyone?" [1]_. > > Open Issues > ----------- > > __call__ methods > '''''''''''''''' > > Backward compatability of the super type API raises some issues. Names, the > lookup of the __call__ of the super type itself, which means a conflict with > doing an actual super lookup of the __call__ attribute. Namely, the following > is ambiguous in the current proposal: > > :: > > super.__call__(arg) > > Which means the backward compatible API, which involves instansiating the super > type, will either not be possible, because it will actually do a super lookup > on the __call__ attribute, or there will be no way to perform a super lookup on > the __call__ attribute. Both seem unacceptable, so any suggestions are welcome. > > Actually keeping the old super around in 2.x and creating a completely new super > type seperately may be the best option. A future import or even a simple import > in 2.x of the new super type from some builtin module may offer a way to choose > which each module uses, even mixing uses by binding to different names. Such a > builtin module might be called 'newsuper'. This module is also the reference > implementation, which I will present below. > > super type's new getattr > '''''''''''''''''''''''' > > To give the behavior needed, the super type either needs a way to do dynamic > lookup of attributes on the super type object itself or define a metaclass for > the builtin type. This author is unsure which, if either, is possible with C- > defined types. > > When should we create __super__ attributes? > ''''''''''''''''''''''''''''''''''''''''''' > > They either need to be created on class creation or on __super__ attribute > lookup. For the second, they could be cached, of course, which seems like it > may be the best idea, if implicit creation of a super object for every class is > considered too much overhead. > > > Reference Implementation > ======================== > > This implementation was a cooperative contribution in the original thread [1]_. > > :: > > #!/usr/bin/env python > # > # newsuper.py > > import sys > > class SuperMetaclass(type): > def __getattr__(cls, attr): > calling_frame = sys._getframe().f_back > instance_name = calling_frame.f_code.co_varnames[0] > instance = calling_frame.f_locals[instance_name] > return getattr(instance.__super__, attr) > > class Super(object): > __metaclass__ = SuperMetaclass > def __init__(self, type, obj=None): > if isinstance(obj, Super): > obj = obj.__obj__ > self.__type__ = type > self.__obj__ = obj > def __get__(self, obj, cls=None): > if obj is None: > raise Exception('only supports instances') > else: > return Super(self.__type__, obj) > def __getattr__(self, attr): > mro = iter(self.__obj__.__class__.__mro__) > for cls in mro: > if cls is self.__type__: > break > for cls in mro: > if attr in cls.__dict__: > x = cls.__dict__[attr] > if hasattr(x, '__get__'): > x = x.__get__(self, cls) > return x > raise AttributeError, attr > > class autosuper(type): > def __init__(cls, name, bases, clsdict): > cls.__super__ = Super(cls) > > if __name__ == '__main__': > class A(object): > __metaclass__ = autosuper > def f(self): > return 'A' > > class B(A): > def f(self): > return 'B' + Super.f() > > class C(A): > def f(self): > return 'C' + Super.f() > > class D(B, C): > def f(self, arg=None): > var = None > return 'D' + Super.f() > > assert D().f() == 'DBCA' > > > History > ======= > 29-Apr-2007 - Changed title from "Super As A Keyword" to "New Super" > - Updated much of the language and added a terminology section > for clarification in confusing places. > - Added reference implementation and history sections. > > References > ========== > > .. [1] Fixing super anyone? > (http://mail.python.org/pipermail/python-3000/2007-April/006667.html) > > > Copyright > ========= > > This document has been placed in the public domain. > > > > .. > Local Variables: > mode: indented-text > indent-tabs-mode: nil > sentence-end-double-space: t > fill-column: 70 > coding: utf-8 > End: > I just realized I could change a small part of this. If we do the two super types thing, where we keep the original unchanged and make a completely different, new super type, and you can pick which you use, that simplifies tons of issues with the new super. The builtin name 'super' can refer to an instance of some NewSuper type, making its __getattr__ way simpler, etc. I still wasn't really aware of any alternative suggestions that need to be included in this. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From martin at v.loewis.de Sun Apr 29 19:51:46 2007 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 29 Apr 2007 19:51:46 +0200 Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <76fd5acf0704290953i6dcc0bffk98d49619ef922e1b@mail.gmail.com> References: <4633F38A.4040605@v.loewis.de> <242497.34958.qm@web56609.mail.re3.yahoo.com> <20070428220126.643C.JCARLSON@uci.edu> <46345127.1020504@v.loewis.de> <76fd5acf0704290522u674118eai97c4a8fe778243b4@mail.gmail.com> <463494AC.1080608@v.loewis.de> <76fd5acf0704290837t1c57ede8wdb6e45ab6c6f3b95@mail.gmail.com> <4634C1FC.7090408@v.loewis.de> <76fd5acf0704290953i6dcc0bffk98d49619ef922e1b@mail.gmail.com> Message-ID: <4634DB32.1020009@v.loewis.de> >> > The >> > original test failed, my new one does not. >> >> Then this change is incorrect: the test should fail in 2.5.0. > > I think I don't get why the test _must_ fail. If it fails, I assumed > something was broken. Correct. That is the whole point of this patch: It fixes a bug in 2.5.0, and provides a test case to show that the bug was fixed. The interesting change here is *not* the test case, but the change to posixmodule.c. > If it failed because it was testing against a > non-existant file, I assumed the test itself was broken. Right. It shouldn't fail if the file is absent (it shouldn't pass in that case, either, but regrtest has no support for INCONCLUSIVE test outcomes). > I will do so. Maybe even just a link to the tracker, because of the > likelihood of me not being the only person to complete miss what the > number in the test name is for. Ok. However, this pattern is quite common in the Python test suite (62 test cases, with prefixes such as test_, test_bug_, test_sf_, test_bug, test_patch_), so adding it just to this single test case may be a drop in the ocean for people unfamiliar with that convention. > I've read the bug report now. I see what I was missing all along. I > think maybe you thought I knew of the bug report, and thus we were > both confused talking on different frequencies and completely missing > each other, Martin. Ok! When you come up with a way to test this problem "stand-alone" (i.e. without relying on the pagefile), please submit a patch. I'll let this sit for some time, and if nothing happens, I go for Khalid's patch before 2.5.2 is released (which is still months ahead). Regards, Martin From ironfroggy at gmail.com Sun Apr 29 19:57:10 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sun, 29 Apr 2007 13:57:10 -0400 Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <4634DB32.1020009@v.loewis.de> References: <4633F38A.4040605@v.loewis.de> <242497.34958.qm@web56609.mail.re3.yahoo.com> <20070428220126.643C.JCARLSON@uci.edu> <46345127.1020504@v.loewis.de> <76fd5acf0704290522u674118eai97c4a8fe778243b4@mail.gmail.com> <463494AC.1080608@v.loewis.de> <76fd5acf0704290837t1c57ede8wdb6e45ab6c6f3b95@mail.gmail.com> <4634C1FC.7090408@v.loewis.de> <76fd5acf0704290953i6dcc0bffk98d49619ef922e1b@mail.gmail.com> <4634DB32.1020009@v.loewis.de> Message-ID: <76fd5acf0704291057g590b909emc9ef3dfa6343450d@mail.gmail.com> On 4/29/07, "Martin v. L?wis" wrote: > Right. It shouldn't fail if the file is absent (it shouldn't > pass in that case, either, but regrtest has no support for INCONCLUSIVE > test outcomes). Perhaps that could become part of the improvements made through test.test_support.TestCase? > Ok. However, this pattern is quite common in the Python test suite > (62 test cases, with prefixes such as test_, test_bug_, test_sf_, > test_bug, test_patch_), so adding it just to this single test case > may be a drop in the ocean for people unfamiliar with that convention. Very true, but maybe more tests could have the more descriptive names, then. For example, I would have known what it meant if the test name prefix was test_sf_ instead of just test_. Changing the names shouldn't interfere with anything else, so if I rename them in an effort to help the next guy, would that be accepted? > Ok! When you come up with a way to test this problem "stand-alone" > (i.e. without relying on the pagefile), please submit a patch. I'll > let this sit for some time, and if nothing happens, I go for > Khalid's patch before 2.5.2 is released (which is still months > ahead). Now that I have the full picture, I have less motivation about it. Although, I am curious what is different about the situation where pagefile.sys could not be stat'ed in 2.5.0 but other open files could. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From collinw at gmail.com Sun Apr 29 20:18:56 2007 From: collinw at gmail.com (Collin Winter) Date: Sun, 29 Apr 2007 11:18:56 -0700 Subject: [Python-Dev] New Super PEP In-Reply-To: <76fd5acf0704291030u1447ddbcv691f1071d2d466ae@mail.gmail.com> References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> <43aa6ff70704290906p43b59ccdkaa2292ae615174bd@mail.gmail.com> <76fd5acf0704290947o79cb9722k66c8ba37fa0b6826@mail.gmail.com> <76fd5acf0704291019y3b30e3ebn2fff564bb71bc462@mail.gmail.com> <76fd5acf0704291030u1447ddbcv691f1071d2d466ae@mail.gmail.com> Message-ID: <43aa6ff70704291118x7ebdf1bu87a3bebc2ca7189d@mail.gmail.com> On 4/29/07, Calvin Spealman wrote: [snip] > I still wasn't really aware of any alternative suggestions that need > to be included in this. Here are two off the top of my head: http://mail.python.org/pipermail/python-3000/2007-April/006805.html http://mail.python.org/pipermail/python-3000/2007-April/006811.html More generally, you're ignoring all the proposals along the line of "let's fix the super type without making it a keyword". Collin Winter From martin at v.loewis.de Sun Apr 29 20:22:36 2007 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 29 Apr 2007 20:22:36 +0200 Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <76fd5acf0704291057g590b909emc9ef3dfa6343450d@mail.gmail.com> References: <4633F38A.4040605@v.loewis.de> <242497.34958.qm@web56609.mail.re3.yahoo.com> <20070428220126.643C.JCARLSON@uci.edu> <46345127.1020504@v.loewis.de> <76fd5acf0704290522u674118eai97c4a8fe778243b4@mail.gmail.com> <463494AC.1080608@v.loewis.de> <76fd5acf0704290837t1c57ede8wdb6e45ab6c6f3b95@mail.gmail.com> <4634C1FC.7090408@v.loewis.de> <76fd5acf0704290953i6dcc0bffk98d49619ef922e1b@mail.gmail.com> <4634DB32.1020009@v.loewis.de> <76fd5acf0704291057g590b909emc9ef3dfa6343450d@mail.gmail.com> Message-ID: <4634E26C.5030208@v.loewis.de> Calvin Spealman schrieb: > On 4/29/07, "Martin v. L?wis" wrote: >> Right. It shouldn't fail if the file is absent (it shouldn't >> pass in that case, either, but regrtest has no support for INCONCLUSIVE >> test outcomes). > > Perhaps that could become part of the improvements made through > test.test_support.TestCase? Sure. I think this is PEP material - I would like to declare "expected failure" as well. > Very true, but maybe more tests could have the more descriptive names, > then. For example, I would have known what it meant if the test name > prefix was test_sf_ instead of just test_. Changing the names > shouldn't interfere with anything else, so if I rename them in an > effort to help the next guy, would that be accepted? That would be fine (of course, we move away from SF, so these method names, at some point, will trigger synapses only for old-timers that still remember sourceforge; the bug IDs will remain constant in the next tracker). > Now that I have the full picture, I have less motivation about it. > Although, I am curious what is different about the situation where > pagefile.sys could not be stat'ed in 2.5.0 but other open files could. The error Windows reports is ERROR_SHARING_VIOLATION. I never understood sharing fully, but it may be that if the file is opened in "exclusive sharing", stat'ing it may fail. I personally consider it a bug in Windows that you cannot get file attributes if some other process has opened it. Exclusive access should only restrict access to file contents, but not file attributes. Regards, Martin From ironfroggy at gmail.com Sun Apr 29 20:47:06 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sun, 29 Apr 2007 14:47:06 -0400 Subject: [Python-Dev] New Super PEP In-Reply-To: <43aa6ff70704291118x7ebdf1bu87a3bebc2ca7189d@mail.gmail.com> References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> <43aa6ff70704290906p43b59ccdkaa2292ae615174bd@mail.gmail.com> <76fd5acf0704290947o79cb9722k66c8ba37fa0b6826@mail.gmail.com> <76fd5acf0704291019y3b30e3ebn2fff564bb71bc462@mail.gmail.com> <76fd5acf0704291030u1447ddbcv691f1071d2d466ae@mail.gmail.com> <43aa6ff70704291118x7ebdf1bu87a3bebc2ca7189d@mail.gmail.com> Message-ID: <76fd5acf0704291147x5ec0aac9wa3dc2d416096d4f8@mail.gmail.com> On 4/29/07, Collin Winter wrote: > On 4/29/07, Calvin Spealman wrote: > [snip] > > I still wasn't really aware of any alternative suggestions that need > > to be included in this. > > Here are two off the top of my head: > http://mail.python.org/pipermail/python-3000/2007-April/006805.html > http://mail.python.org/pipermail/python-3000/2007-April/006811.html > > More generally, you're ignoring all the proposals along the line of > "let's fix the super type without making it a keyword". > > Collin Winter > I'll add both of these, but neither had much support in the original thread. Also, I don't see that I'm ignoring anything along the line of "let's fix the super type without making it a keyword", because I am not advocating it become an actual keyword. I was always under the impression that was never meant literally. We have no where else where a keyword looks like an object. At the absolutely most I could almost see how super may become a constant, a'la None, in 3.0, but never a keyword. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From collinw at gmail.com Sun Apr 29 20:48:15 2007 From: collinw at gmail.com (Collin Winter) Date: Sun, 29 Apr 2007 11:48:15 -0700 Subject: [Python-Dev] New Super PEP In-Reply-To: <76fd5acf0704291147x5ec0aac9wa3dc2d416096d4f8@mail.gmail.com> References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> <43aa6ff70704290906p43b59ccdkaa2292ae615174bd@mail.gmail.com> <76fd5acf0704290947o79cb9722k66c8ba37fa0b6826@mail.gmail.com> <76fd5acf0704291019y3b30e3ebn2fff564bb71bc462@mail.gmail.com> <76fd5acf0704291030u1447ddbcv691f1071d2d466ae@mail.gmail.com> <43aa6ff70704291118x7ebdf1bu87a3bebc2ca7189d@mail.gmail.com> <76fd5acf0704291147x5ec0aac9wa3dc2d416096d4f8@mail.gmail.com> Message-ID: <43aa6ff70704291148q581bc29dt2f8e279519391c3e@mail.gmail.com> On 4/29/07, Calvin Spealman wrote: > On 4/29/07, Collin Winter wrote: > > On 4/29/07, Calvin Spealman wrote: > > [snip] > > > I still wasn't really aware of any alternative suggestions that need > > > to be included in this. > > > > Here are two off the top of my head: > > http://mail.python.org/pipermail/python-3000/2007-April/006805.html > > http://mail.python.org/pipermail/python-3000/2007-April/006811.html > > > > More generally, you're ignoring all the proposals along the line of > > "let's fix the super type without making it a keyword". > > > > Collin Winter > > > > I'll add both of these, but neither had much support in the original > thread. Also, I don't see that I'm ignoring anything along the line of > "let's fix the super type without making it a keyword", because I am > not advocating it become an actual keyword. Sorry, I was thrown off by the original title of your PEP. From collinw at gmail.com Sun Apr 29 20:49:33 2007 From: collinw at gmail.com (Collin Winter) Date: Sun, 29 Apr 2007 11:49:33 -0700 Subject: [Python-Dev] New Super PEP In-Reply-To: <76fd5acf0704291019y3b30e3ebn2fff564bb71bc462@mail.gmail.com> References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> <43aa6ff70704290906p43b59ccdkaa2292ae615174bd@mail.gmail.com> <76fd5acf0704290947o79cb9722k66c8ba37fa0b6826@mail.gmail.com> <76fd5acf0704291019y3b30e3ebn2fff564bb71bc462@mail.gmail.com> Message-ID: <43aa6ff70704291149i4c2380cdt986de3de6e45bed4@mail.gmail.com> On 4/29/07, Calvin Spealman wrote: > The PEP defines the proposal to enhance the super builtin to work implicitly > upon the class within which it is used and upon the instance the current > function was called on. The premise of the new super usage suggested is as > follows: > > super.foo(1, 2) > > to replace the old: > > super(Foo, self).foo(1, 2) Now that I think about it, your proposal seems to address only one of super()'s three forms (http://docs.python.org/lib/built-in-funcs.html#l2h-72): 1. super(type) 2. super(type, instance) 3. super(type, type) If your intention is to remove the first and third forms from the language, please justify their removal in your PEP, including your proposed work-around for their use-cases. Collin Winter From arigo at tunes.org Sun Apr 29 20:50:12 2007 From: arigo at tunes.org (Armin Rigo) Date: Sun, 29 Apr 2007 20:50:12 +0200 Subject: [Python-Dev] New Super PEP In-Reply-To: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> Message-ID: <20070429185012.GA32077@code0.codespeak.net> Hi Calvin, On Sat, Apr 28, 2007 at 10:43:04PM -0400, Calvin Spealman wrote: > The proposal adds a dynamic attribute lookup to the super type, which will > automatically determine the proper class and instance parameters. Can you describe how you intend the dynamic attribute lookup to find the proper class? Is it related to your proposal to add a new attribute '__super__' to each class? If so, I don't see how... and if not, can you explain why you need '__super__' then? A bientot, Armin. From ironfroggy at gmail.com Sun Apr 29 20:52:12 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sun, 29 Apr 2007 14:52:12 -0400 Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <4634E26C.5030208@v.loewis.de> References: <4633F38A.4040605@v.loewis.de> <46345127.1020504@v.loewis.de> <76fd5acf0704290522u674118eai97c4a8fe778243b4@mail.gmail.com> <463494AC.1080608@v.loewis.de> <76fd5acf0704290837t1c57ede8wdb6e45ab6c6f3b95@mail.gmail.com> <4634C1FC.7090408@v.loewis.de> <76fd5acf0704290953i6dcc0bffk98d49619ef922e1b@mail.gmail.com> <4634DB32.1020009@v.loewis.de> <76fd5acf0704291057g590b909emc9ef3dfa6343450d@mail.gmail.com> <4634E26C.5030208@v.loewis.de> Message-ID: <76fd5acf0704291152haa4fbc6x5662d40f5617e6ef@mail.gmail.com> On 4/29/07, "Martin v. L?wis" wrote: > Calvin Spealman schrieb: > > On 4/29/07, "Martin v. L?wis" wrote: > >> Right. It shouldn't fail if the file is absent (it shouldn't > >> pass in that case, either, but regrtest has no support for INCONCLUSIVE > >> test outcomes). > > > > Perhaps that could become part of the improvements made through > > test.test_support.TestCase? > > Sure. I think this is PEP material - I would like to declare > "expected failure" as well. I would second that. Twisted's trial system does a lot of this, already. I suppose that this conversation would end inside this thread now; it is officially off topic of the thread. But, yes, I would love to see it. test_support could definately act as a testbed for new things before moving to unittest or the unittest replacement. > > Very true, but maybe more tests could have the more descriptive names, > > then. For example, I would have known what it meant if the test name > > prefix was test_sf_ instead of just test_. Changing the names > > shouldn't interfere with anything else, so if I rename them in an > > effort to help the next guy, would that be accepted? > > That would be fine (of course, we move away from SF, so these method > names, at some point, will trigger synapses only for old-timers > that still remember sourceforge; the bug IDs will remain constant in > the next tracker). So test_bug_ then. > > Now that I have the full picture, I have less motivation about it. > > Although, I am curious what is different about the situation where > > pagefile.sys could not be stat'ed in 2.5.0 but other open files could. > > The error Windows reports is ERROR_SHARING_VIOLATION. I never > understood sharing fully, but it may be that if the file is opened > in "exclusive sharing", stat'ing it may fail. > > I personally consider it a bug in Windows that you cannot get file > attributes if some other process has opened it. Exclusive access > should only restrict access to file contents, but not file attributes. Perhaps there are things about the implementation of file operations in "exclusive sharing" mode of files that can not guarantee consistant, reliable, or correct results of a stat call during that time, and the decision was simply "If it can't be correct, it can't be at all." -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From ironfroggy at gmail.com Sun Apr 29 20:53:58 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sun, 29 Apr 2007 14:53:58 -0400 Subject: [Python-Dev] New Super PEP In-Reply-To: <20070429185012.GA32077@code0.codespeak.net> References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> <20070429185012.GA32077@code0.codespeak.net> Message-ID: <76fd5acf0704291153y5d20f453u66d7c8bbd64d62ad@mail.gmail.com> On 4/29/07, Armin Rigo wrote: > Hi Calvin, > > On Sat, Apr 28, 2007 at 10:43:04PM -0400, Calvin Spealman wrote: > > The proposal adds a dynamic attribute lookup to the super type, which will > > automatically determine the proper class and instance parameters. > > Can you describe how you intend the dynamic attribute lookup to find the > proper class? Is it related to your proposal to add a new attribute > '__super__' to each class? If so, I don't see how... and if not, can > you explain why you need '__super__' then? Yes, it is highly dependent on the new __super__ attribute. See the reference implementation for the how. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From collinw at gmail.com Sun Apr 29 21:15:38 2007 From: collinw at gmail.com (Collin Winter) Date: Sun, 29 Apr 2007 12:15:38 -0700 Subject: [Python-Dev] New Super PEP In-Reply-To: <76fd5acf0704290947o79cb9722k66c8ba37fa0b6826@mail.gmail.com> References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> <43aa6ff70704290906p43b59ccdkaa2292ae615174bd@mail.gmail.com> <76fd5acf0704290947o79cb9722k66c8ba37fa0b6826@mail.gmail.com> Message-ID: <43aa6ff70704291215g1e6be3ffq60a3c3954d88fb19@mail.gmail.com> On 4/29/07, Calvin Spealman wrote: > On 4/29/07, Collin Winter wrote: > > What if the instance isn't called "self"? PEP 3099 states that "self > > will not become implicit"; it's talking about method signatures, but I > > think that dictum applies equally well in this case. > > I don't use the name self. I use whatever the first argument name is, > found by this line of python code: > > instance_name = calling_frame.f_code.co_varnames[0] So I can't use super with anything but the method's invocant? That seems arbitrary. > > Also, it's my understanding that not all Python implementations have > > an easy analogue to CPython's frames; have you given any thought to > > whether and how PyPy, IronPython, Jython, etc, will implement this? > > I'll bring this up for input from PyPy and IronPython people, but I > don't know any Jython people. Are we yet letting the alternative > implementations influence so strongly what we do in CPython? I'm not > saying "screw them", just pointing out that there is always a way to > implement anything, and if its some trouble for them, well, 2.6 or 3.0 > targetting is far down the road for any of them yet. It's a smell test: if a given proposal is unduly difficult for anything but CPython to implement, it's probably a bad idea. The language shouldn't go down the Perl 5 road, where python (the C interpreter) becomes the only thing that can implement Python (the language). Collin Winter From ironfroggy at gmail.com Sun Apr 29 21:29:10 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sun, 29 Apr 2007 15:29:10 -0400 Subject: [Python-Dev] New Super PEP In-Reply-To: <43aa6ff70704291149i4c2380cdt986de3de6e45bed4@mail.gmail.com> References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> <43aa6ff70704290906p43b59ccdkaa2292ae615174bd@mail.gmail.com> <76fd5acf0704290947o79cb9722k66c8ba37fa0b6826@mail.gmail.com> <76fd5acf0704291019y3b30e3ebn2fff564bb71bc462@mail.gmail.com> <43aa6ff70704291149i4c2380cdt986de3de6e45bed4@mail.gmail.com> Message-ID: <76fd5acf0704291229g2e36bfecg83211452b527b8df@mail.gmail.com> On 4/29/07, Collin Winter wrote: > On 4/29/07, Calvin Spealman wrote: > > The PEP defines the proposal to enhance the super builtin to work implicitly > > upon the class within which it is used and upon the instance the current > > function was called on. The premise of the new super usage suggested is as > > follows: > > > > super.foo(1, 2) > > > > to replace the old: > > > > super(Foo, self).foo(1, 2) > > Now that I think about it, your proposal seems to address only one of > super()'s three forms > (http://docs.python.org/lib/built-in-funcs.html#l2h-72): > > 1. super(type) > 2. super(type, instance) > 3. super(type, type) > > If your intention is to remove the first and third forms from the > language, please justify their removal in your PEP, including your > proposed work-around for their use-cases. The first is not removed, but actually utilized by the proposal itself. That is what the __super__ attributes are: super objects associated only with a type, but no instance. As for the third form, I wasn't even aware of it, I thought. I didn't consider the cases like how super is used in __new__ methods, but I tested it with the reference implementation, and it works just fine. If there are any cases I'm missing, there is no reason not to support it. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From ironfroggy at gmail.com Sun Apr 29 21:31:27 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sun, 29 Apr 2007 15:31:27 -0400 Subject: [Python-Dev] New Super PEP In-Reply-To: <43aa6ff70704291215g1e6be3ffq60a3c3954d88fb19@mail.gmail.com> References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> <43aa6ff70704290906p43b59ccdkaa2292ae615174bd@mail.gmail.com> <76fd5acf0704290947o79cb9722k66c8ba37fa0b6826@mail.gmail.com> <43aa6ff70704291215g1e6be3ffq60a3c3954d88fb19@mail.gmail.com> Message-ID: <76fd5acf0704291231u79bb85ffrff20e517db517fdd@mail.gmail.com> On 4/29/07, Collin Winter wrote: > On 4/29/07, Calvin Spealman wrote: > > On 4/29/07, Collin Winter wrote: > > > What if the instance isn't called "self"? PEP 3099 states that "self > > > will not become implicit"; it's talking about method signatures, but I > > > think that dictum applies equally well in this case. > > > > I don't use the name self. I use whatever the first argument name is, > > found by this line of python code: > > > > instance_name = calling_frame.f_code.co_varnames[0] > > So I can't use super with anything but the method's invocant? That > seems arbitrary. This will be added to the open issues, but it comes back to the problem with allow the exact same super implementation both operate in the super(Class, Object).foo() form and also the super.__call__() form in the new version. Any suggestions are welcome for how to solve this. > > > Also, it's my understanding that not all Python implementations have > > > an easy analogue to CPython's frames; have you given any thought to > > > whether and how PyPy, IronPython, Jython, etc, will implement this? > > > > I'll bring this up for input from PyPy and IronPython people, but I > > don't know any Jython people. Are we yet letting the alternative > > implementations influence so strongly what we do in CPython? I'm not > > saying "screw them", just pointing out that there is always a way to > > implement anything, and if its some trouble for them, well, 2.6 or 3.0 > > targetting is far down the road for any of them yet. > > It's a smell test: if a given proposal is unduly difficult for > anything but CPython to implement, it's probably a bad idea. The > language shouldn't go down the Perl 5 road, where python (the C > interpreter) becomes the only thing that can implement Python (the > language). Understandable. I still haven't contacted anyone about it on in the PyPy or IronPython worlds, and anyone familiar with Jython who can comment would be appreciated. Ditto for PyPy and IronPython, even though I should be able to find some information there myself. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From pje at telecommunity.com Sun Apr 29 21:40:10 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 29 Apr 2007 15:40:10 -0400 Subject: [Python-Dev] New Super PEP In-Reply-To: <76fd5acf0704291019y3b30e3ebn2fff564bb71bc462@mail.gmail.co m> References: <76fd5acf0704290947o79cb9722k66c8ba37fa0b6826@mail.gmail.com> <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> <43aa6ff70704290906p43b59ccdkaa2292ae615174bd@mail.gmail.com> <76fd5acf0704290947o79cb9722k66c8ba37fa0b6826@mail.gmail.com> Message-ID: <5.1.1.6.0.20070429153725.04aadda0@sparrow.telecommunity.com> At 01:19 PM 4/29/2007 -0400, Calvin Spealman wrote: >Backward compatability of the super type API raises some issues. Names, the >lookup of the __call__ of the super type itself, which means a conflict with >doing an actual super lookup of the __call__ attribute. Namely, the following >is ambiguous in the current proposal: > > :: > > super.__call__(arg) > >Which means the backward compatible API, which involves instansiating the >super >type, will either not be possible, because it will actually do a super lookup >on the __call__ attribute, or there will be no way to perform a super >lookup on >the __call__ attribute. Both seem unacceptable, so any suggestions are >welcome. Note that if you have a class with a __call__ method, it will still be called, even if you override __getattribute__ to return something else when asked for the __call__ attribute, e.g.: >>> class DoubleCall(object): ... def __call__(self): ... return "called!" ... def __getattribute__(self, attr): ... if attr=='__call__': ... return lambda: "attribute" >>> dc = DoubleCall() >>> dc() 'called!' >>> dc.__call__() 'attribute' From jimjjewett at gmail.com Sun Apr 29 22:31:33 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Sun, 29 Apr 2007 16:31:33 -0400 Subject: [Python-Dev] New Super PEP Message-ID: Phillip J. Eby wrote: > a __call__ method, it will still be > called, even if you override __getattribute__ to return something else >>> dc = DoubleCall() >>> dc() 'called!' >>> dc.__call__() 'attribute' That is sufficiently fragile that I would be uncomfortable with a feature that couldn't be supplied otherwise. -jJ From arigo at tunes.org Sun Apr 29 22:42:22 2007 From: arigo at tunes.org (Armin Rigo) Date: Sun, 29 Apr 2007 22:42:22 +0200 Subject: [Python-Dev] New Super PEP In-Reply-To: <76fd5acf0704291153y5d20f453u66d7c8bbd64d62ad@mail.gmail.com> References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> <20070429185012.GA32077@code0.codespeak.net> <76fd5acf0704291153y5d20f453u66d7c8bbd64d62ad@mail.gmail.com> Message-ID: <20070429204222.GA7987@code0.codespeak.net> Hi Calvin, On Sun, Apr 29, 2007 at 02:53:58PM -0400, Calvin Spealman wrote: > Yes, it is highly dependent on the new __super__ attribute. See the > reference implementation for the how. Ah, there is a reference implementation. There is no link to it in the PEP you posted, hence my confusion. Where is it? A bientot, Armin. From jimjjewett at gmail.com Sun Apr 29 23:15:33 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Sun, 29 Apr 2007 17:15:33 -0400 Subject: [Python-Dev] New Super PEP - draft 2 Message-ID: (Adding python-3000 to the Cc, since that is where much of the recent discussion occurred) Calvin's full proposal is archived at: http://mail.python.org/pipermail/python-dev/2007-April/072835.html > Abstract > ======== > The PEP defines the proposal to enhance the super builtin to work implicitly > upon the class within which it is used and upon the instance the current > function was called on. The premise of the new super usage suggested is as > follows: > super.foo(1, 2) > to replace the old: > super(Foo, self).foo(1, 2) The alternatives section needs to explain why "super.foo(*args)" is the right level of simplification. Alternatives include (1) No changes Con: fragility in name binding (2) super(__this_class__, self).foo(*args) Con: The "(__this_class__, self)" is boilerplate. Note that the __this_class__ PEP should be referenced, if only as a possible explanation for what is happening under the covers. (3) self.__super__.foo(*args) # or super.foo(*args) Con: Shouldn't need a __double_underscore_name__ in normal functions. Con: __super__ (even if renamed super) is not a simple attribute; it is a property representing a partially applied function (that gets further applied by the "self") (4) super(self, *args) # ? or __super__(self, *args) Actually, I sort of like this one, as it allows the upcall signature to exactly match the method definition signature. Con: super is still not a simple attribute. Con: changing the method name becomes a hassle. (5) super.foo(self, *args) # ? or __super__.foo(self, *args) Con: self isn't really an argument just to super.foo -- it is an argument to super which is used to find/instantiate foo in the first place. (Plus the objections to partially applied function attributes.) (6) super # ? or super() Pro: matches java; doesn't require a repeat of *args Con: doesn't match anything else in python, needs to be a keyword (and would still need object support). In the first example: class A(object): def f(self): return 'A' class B(A): def f(self): return 'B' + super.f() class C(A): def f(self): return 'C' + super.f() class D(B, C): def f(self): return 'D' + super.f() assert D().f() == 'DBCA' You should probably include tests that fail with some of the more obvious (but wrong) solutions, such as class A(object): def f(self): return 'A' class Bempty(A): # Make sure it doesn't call the A or B method twice pass class B(Bempty): def f(self): return 'B' + super.f() class C(A): def f(self): return 'C' + super.f() class D(B, C): def f(self): return 'D' + super.f() assert D().f() == 'DBCA' class E(C, B): # Show that B can point to C as next or vice versa def f(self): return 'E' + super.f() assert E().f() == 'ECBA' class F(D): # show that the instance may not be a direct instance pass assert D().f() == 'DBCA' > The enhancements to the super type will define a new __getattr__ classmethod > of the super type, Did you really mean, it gets the class of builtin.super, but no info on which class is using super? If not, it isn't a classmethod of the super type. It may be a classmethod of the currently-being-defined-type (__this_class__). > which must look backwards to the previous frame and locate > the instance object. frame manipulation is fragile. Even if it didn't cause problems for other implementations, it causes problems for nested functions and callbacks. def f(self, button, *args): def callback(*args): super(__this_class__, self).f(*args) button.callback=callback When this gets called, the appropriate self won't be in the frame, except as a lexically scoped variable (which might have a different name). Also, note that this fails on most of Thomas Wouters' advanced usages. While you say that super shouldn't be called outside a method, you can't really keep the super object itself from getting returned. > "Every class will gain a new special attribute, __super__, which refers to an > instance of the associated super object for that class" In this capacity, the > new super also acts as its own descriptor, create an instance-specific super > upon lookup. If you didn't say instance-specific, this would be equivalent to a class decorator, translating class A(object): ... into class A(object): ... A.__super__=super(A) As is, it gets a bit trickier, since you need to I think the translation is closer to class A(object):... @property def __super__(self): return __this_class__.__supermaker() A.__supermaker=super(A) I don't see a good non-magical way to pass both the instance and the class-at-time-of-writing (as opposed to class of calling instance), which is why the other solutions used bytecode hacks or name mangling. > Much of this was discussed in the thread of the python-dev list, "Fixing super > anyone?" [1]_. I assume you meant the python-3000 list. -jJ From jimjjewett at gmail.com Sun Apr 29 23:25:53 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Sun, 29 Apr 2007 17:25:53 -0400 Subject: [Python-Dev] [Python-3000] Pre-pre PEP for 'super' keyword In-Reply-To: <00b601c78a9f$38ec9390$0201a8c0@ryoko> References: <76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com> <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> <002401c78778$75fb7eb0$0201a8c0@ryoko> <00b601c78a9f$38ec9390$0201a8c0@ryoko> Message-ID: On 4/29/07, Tim Delaney wrote: > I've been intending to write up a PEP for fixing super, but I haven't had > time to get to it. Calvin Spealman has the most recent draft. I hope he will incorporate this into his draft. > 1. 'super' becomes a keyword, that returns a super object for the instance > method currently being executed. So it is a "keyword" in the sense that None is a keyword; not in the stronger sense that "if" is a keyword? > 4. super objects are callable, and calling them will execute the super > method with the same name as the instance method currently being executed. > Lookup of this method occurs when the instance method is entered. > > class A(object): > def f(self): > pass > > class B(A): > def f(self): > super() # Calls A.f(self) > If you want name lookup to occur at the time of the call, you can explicitly > specify the method name (just like with any other super attribute): > > class A(object): > def f(self): > pass > > class B(A): > def f(self): > super.f() # Calls A.f(self) As long as you can be explicit, should the shortcut be a full shortcut? That is, def f(self, a, b=c, *args, **kwargs): super() # passes the exact arglist that f got vs def __init__(self, myvar, passed_var): super.__init__(self, passed_var) # flags that you are changing the args -jJ From ironfroggy at gmail.com Mon Apr 30 00:23:52 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sun, 29 Apr 2007 18:23:52 -0400 Subject: [Python-Dev] [Python-3000] Pre-pre PEP for 'super' keyword In-Reply-To: References: <76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com> <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> <002401c78778$75fb7eb0$0201a8c0@ryoko> <00b601c78a9f$38ec9390$0201a8c0@ryoko> Message-ID: <76fd5acf0704291523q78a14a26lb421d927252495a8@mail.gmail.com> On 4/29/07, Jim Jewett wrote: > On 4/29/07, Tim Delaney wrote: > > I've been intending to write up a PEP for fixing super, but I haven't had > > time to get to it. > > Calvin Spealman has the most recent draft. I hope he will incorporate > this into his draft. I will incorporate this into my draft, particularly taking care of the issue with inner functions. > > 1. 'super' becomes a keyword, that returns a super object for the instance > > method currently being executed. > > So it is a "keyword" in the sense that None is a keyword; not in the > stronger sense that "if" is a keyword? I would like to say super becomes a constant in the way that None is a constant, and if there is some reason the implementation today or tomorrow can benefit from actually making it a keyword, that won't break anything if its already constant. One problem with an actual keyword, is there is no other part of Python where an actual keyword evaluates to something. > > 4. super objects are callable, and calling them will execute the super > > method with the same name as the instance method currently being executed. > > Lookup of this method occurs when the instance method is entered. > > > > class A(object): > > def f(self): > > pass > > > > class B(A): > > def f(self): > > super() # Calls A.f(self) This might run into the same issue I had to cover, where you get an ambiguous situation trying to distinguish between calling super and calling the __call__ method of the next class in the MRO. We should absolutely avoid a situation in python now where X() differs from X.__call__() > > If you want name lookup to occur at the time of the call, you can explicitly > > specify the method name (just like with any other super attribute): > > > > class A(object): > > def f(self): > > pass > > > > class B(A): > > def f(self): > > super.f() # Calls A.f(self) > > As long as you can be explicit, should the shortcut be a full > shortcut? That is, > > def f(self, a, b=c, *args, **kwargs): > super() # passes the exact arglist that f got I sure wish my previous complaints didn't hinder this, because I really love the idea of being able to this, which would really encourage more compatible method signatures, so you can use the shortcut! I'm desperate for a solution that satisfies all the sides of the equation. > vs > > def __init__(self, myvar, passed_var): > super.__init__(self, passed_var) # flags that you are > changing the args > > -jJ > -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From guido at python.org Mon Apr 30 01:30:18 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 29 Apr 2007 16:30:18 -0700 Subject: [Python-Dev] [Python-3000] Pre-pre PEP for 'super' keyword In-Reply-To: References: <76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com> <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> <002401c78778$75fb7eb0$0201a8c0@ryoko> <00b601c78a9f$38ec9390$0201a8c0@ryoko> Message-ID: On 4/29/07, Jim Jewett wrote: > So it is a "keyword" in the sense that None is a keyword; not in the > stronger sense that "if" is a keyword? Um, how do you see those two differ? Is 'if' a keyword in the same sense as 'or', or in a different sense? I realize that in Python 2.5, None is not a full-fledged keyword but cannot be used as an assignment target. But that's only transitional. In 3.0 I imagine it becoming a keyword in the grammar (whose only appearance would be as one of the alternatives for 'atom'). And we're talking 3.0 here. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Mon Apr 30 02:58:40 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 30 Apr 2007 12:58:40 +1200 Subject: [Python-Dev] New Super PEP In-Reply-To: References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> Message-ID: <46353F40.60309@canterbury.ac.nz> Gustavo Carneiro wrote: > Erm. Excuse me, but are you saying this code is wrong? > > class Rectangle: > def __init__(self, width, height): > self.width = width > self.height = height > > class Square: > def __init__(self, side): > Rectangle.__init__(self, side, side) The PEP is all about cooperative super calls, so it doesn't apply to this. I'd still rather see syntactic support for ordinary inherited calls before cooperative ones, as cooperative ones are used much more rarely, in my experience. -- Greg From ironfroggy at gmail.com Mon Apr 30 03:01:42 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sun, 29 Apr 2007 21:01:42 -0400 Subject: [Python-Dev] [Python-3000] Pre-pre PEP for 'super' keyword In-Reply-To: References: <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> <002401c78778$75fb7eb0$0201a8c0@ryoko> <00b601c78a9f$38ec9390$0201a8c0@ryoko> Message-ID: <76fd5acf0704291801o47733e29u634ffa317d32a0a7@mail.gmail.com> On 4/29/07, Guido van Rossum wrote: > On 4/29/07, Jim Jewett wrote: > > So it is a "keyword" in the sense that None is a keyword; not in the > > stronger sense that "if" is a keyword? > > Um, how do you see those two differ? Is 'if' a keyword in the same > sense as 'or', or in a different sense? > > I realize that in Python 2.5, None is not a full-fledged keyword but > cannot be used as an assignment target. But that's only transitional. > In 3.0 I imagine it becoming a keyword in the grammar (whose only > appearance would be as one of the alternatives for 'atom'). And we're > talking 3.0 here. I think any concerns about it not being fit as a keyword would fall under two catagories or varying validity: 1) Too many keywords is a valid concern, because it complicates the language. 2) It just doesn't "feel" like a keyword. Less valid, unless it _really_ doesn't feel like a keyword. It doesn't feel like a keyword. But it doesn't feel too much not like a keyword. Anyway, I tried to address the concerns laid out, and I'm more than happy to alter the PEP to actually say "Lets implement this as a keyword", and I actually meant to keep more agnostic on that point in the proposal itself. I was more interested in covering the interface, at least to begin, than the actual implementation. Although, being able to have a solid, working reference implementation based on the frame lookups and such is nice, so we can see how it will actually work in real code, and even use it to backport code using the new super to just about any recent Python version. I also checked and PyPy does implement a sys._getframe() and a IronPython currently doesn't, but seems to plan on it (there is a placeholder, at present). I am not sure if notes on this belongs in the PEP or not. Draft Three follows for all. I think I'm turning off e-mail for the rest of this evening, so I'll catch up tomorrow. ------------------------------------------------------- PEP: XXX Title: New Super Version: $Revision$ Last-Modified: $Date$ Author: Calvin Spealman Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 28-Apr-2007 Python-Version: 2.6 Post-History: 28-Apr-2007, 29-Apr-2007 (1), 29-Apr-2007 (2) Abstract ======== The PEP defines the proposal to enhance the super builtin to work implicitly upon the class within which it is used and upon the instance the current function was called on. The premise of the new super usage suggested is as follows: super.foo(1, 2) to replace the old: super(Foo, self).foo(1, 2) Rationale ========= The current usage of super requires an explicit passing of both the class and instance it must operate from, requiring a breaking of the DRY (Don't Repeat Yourself) rule. This hinders any change in class name, and is often considered a wart by many. Specification ============= Within the specification section, some special terminology will be used to distinguish similar and closely related concepts. "Super type" will refer to the actual builtin type named "super". "Next Class/Type in the MRO" will refer to the class where attribute lookups will be performed by super, for example, in the following, A is the "Next class in the MRO" for the use of super. :: class A(object): def f(self): return 'A' class B(A): def f(self): super(B, self).f() # Here, A would be out "Next class in the # MRO", of course. A "super object" is simply an instance of the super type, which is associated with a class and possibly with an instance of that class. Finally, "new super" refers to the new super type, which will replace the original. Replacing the old usage of super, calls to the next class in the MRO (method resolution order) will be made without an explicit super object creation, by simply accessing an attribute on the super type directly, which will automatically apply the class and instance to perform the proper lookup. The following example demonstrates the use of this. :: class A(object): def f(self): return 'A' class B(A): def f(self): return 'B' + super.f() class C(A): def f(self): return 'C' + super.f() class D(B, C): def f(self): return 'D' + super.f() assert D().f() == 'DBCA' The proposal adds a dynamic attribute lookup to the super type, which will automatically determine the proper class and instance parameters. Each super attribute lookup identifies these parameters and performs the super lookup on the instance, as the current super implementation does with the explicit invokation of a super object upon a class and instance. The enhancements to the super type will define a new __getattr__ classmethod of the super type, which must look backwards to the previous frame and locate the instance object. This can be naively determined by located the local named by the first argument to the function. Using super outside of a function where this is a valid lookup for the instance can be considered undocumented in its behavior. This special method will actually be invoked on attribute lookups to the super type itself, as opposed to super objects, as the current implementation works. This may pose open issues, which are detailed below. "Every class will gain a new special attribute, __super__, which refers to an instance of the associated super object for that class" In this capacity, the new super also acts as its own descriptor, create an instance-specific super upon lookup. Much of this was discussed in the thread of the python-dev list, "Fixing super anyone?" [1]_. Open Issues ----------- __call__ methods '''''''''''''''' Backward compatability of the super type API raises some issues. Names, the lookup of the __call__ of the super type itself, which means a conflict with doing an actual super lookup of the __call__ attribute. Namely, the following is ambiguous in the current proposal: :: super.__call__(arg) Which means the backward compatible API, which involves instansiating the super type, will either not be possible, because it will actually do a super lookup on the __call__ attribute, or there will be no way to perform a super lookup on the __call__ attribute. Both seem unacceptable, so any suggestions are welcome. Actually keeping the old super around in 2.x and creating a completely new super type seperately may be the best option. A future import or even a simple import in 2.x of the new super type from some builtin module may offer a way to choose which each module uses, even mixing uses by binding to different names. Such a builtin module might be called 'newsuper'. This module is also the reference implementation, which I will present below. super type's new getattr '''''''''''''''''''''''' To give the behavior needed, the super type either needs a way to do dynamic lookup of attributes on the super type object itself or define a metaclass for the builtin type. This author is unsure which, if either, is possible with C- defined types. When should we create __super__ attributes? ''''''''''''''''''''''''''''''''''''''''''' They either need to be created on class creation or on __super__ attribute lookup. For the second, they could be cached, of course, which seems like it may be the best idea, if implicit creation of a super object for every class is considered too much overhead. How does it work in inner functions? '''''''''''''''''''''''''''''''''''' If a method defines a function and super is used inside of it, how does this work? The frame looking and instance detection breaks here. However, if there can be some unambiguous way to use both the new super form and still be able to explicitly name the type and instance, I think its an acceptable tradeoff to simply be explicit in these cases, rather than add weird super-specific lookup rules in these cases. An example of such a problematic bit of code is: :: class B(A): def f(self): def g(): return super.f() return g() Should super actually become a keyword? ''''''''''''''''''''''''''''''''''''''' This would solve many of the problems and allow more direct implementation of super into the language proper. However, some are against the actual keyword- ization of super. The simplest solution is often the correct solution and the simplest solution may well not be adding additional keywords to the language when they are not needed. Still, it may solve many of the other open issues. Can we also allow super()? '''''''''''''''''''''''''' There is strong sentiment for and against this, but implementation and style concerns are obvious. Particularly, that its "magical" and that super() would differ from super.__call__(), being very unpythonic. Reference Implementation ======================== This implementation was a cooperative contribution in the original thread [1]_. :: #!/usr/bin/env python # # newsuper.py import sys class SuperMetaclass(type): def __getattr__(cls, attr): calling_frame = sys._getframe().f_back instance_name = calling_frame.f_code.co_varnames[0] instance = calling_frame.f_locals[instance_name] return getattr(instance.__super__, attr) class Super(object): __metaclass__ = SuperMetaclass def __init__(self, type, obj=None): if isinstance(obj, Super): obj = obj.__obj__ self.__type__ = type self.__obj__ = obj def __get__(self, obj, cls=None): if obj is None: raise Exception('only supports instances') else: return Super(self.__type__, obj) def __getattr__(self, attr): mro = iter(self.__obj__.__class__.__mro__) for cls in mro: if cls is self.__type__: break for cls in mro: if attr in cls.__dict__: x = cls.__dict__[attr] if hasattr(x, '__get__'): x = x.__get__(self, cls) return x raise AttributeError, attr class autosuper(type): def __init__(cls, name, bases, clsdict): cls.__super__ = Super(cls) if __name__ == '__main__': class A(object): __metaclass__ = autosuper def f(self): return 'A' class B(A): def f(self): return 'B' + Super.f() class C(A): def f(self): return 'C' + Super.f() class D(B, C): def f(self, arg=None): var = None return 'D' + Super.f() assert D().f() == 'DBCA' Alternative Proposals ===================== No Changes ---------- Although its always attractive to just keep things how they are, people have sought a change in the usage of super calling for some time, and for good reason, all mentioned previously. * Decoupling from the class name (which might not even be bound to the right class anymore! [2]_) * Simpler looking, cleaner super calls would be better super(__this_class__, self) --------------------------- This is nearly an anti-proposal, as it basically relies on the acceptance of the __this_class__ PEP, which proposes a special name that would always be bound to the class within which it is used. If that is accepted, __this_class__ could simply be used instead of the class' name explicitly, solving the name binding issues [2]_. self.__super__.foo(*args) ------------------------- The __super__ attribute is mentioned in this PEP in several places, and could be a candidate for the completel solution, actually using it explicitly instead of any super usage directly. However, double-underscore names are usually an internal detail, and attempted to be kept out of everyday code. super(self, *args) or __super__(self, *args) -------------------------------------------- This solution only solves the problem of the type indication, does not handle differently named super methods, and is explicit about the name of the instance. It is less flexable without being able to enacted on other method names, in cases where that is needed. One use case this fails is where a base- class has a factory classmethod and a subclass has two factory classmethods, both of which needing to properly make super calls to the one in the base- class. super.foo(self, *args) ---------------------- This variation actually eliminates the problems with locating the proper instance, and if any of the alternatives were pushed into the spotlight, I would want it to be this one. super or super() ---------------- This proposal leaves no room for different names, signatures, or application to other classes, or instances. A way to allow some similar use alongside the normal proposal would be favorable, encouraging good design of multiple inheritence trees and compatible methods. History ======= 29-Apr-2007 - Changed title from "Super As A Keyword" to "New Super" - Updated much of the language and added a terminology section for clarification in confusing places. - Added reference implementation and history sections. References ========== .. [1] Fixing super anyone? (http://mail.python.org/pipermail/python-3000/2007-April/006667.html) Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: From greg.ewing at canterbury.ac.nz Mon Apr 30 03:03:16 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 30 Apr 2007 13:03:16 +1200 Subject: [Python-Dev] New Super PEP In-Reply-To: <17972.33757.278659.977298@montanaro.dyndns.org> References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> <17972.33757.278659.977298@montanaro.dyndns.org> Message-ID: <46354054.7070403@canterbury.ac.nz> skip at pobox.com wrote: > Since the language doesn't require that a subclassed method take the same > parameters as the base class method, you can't assume that it does. The argument is that in the special case of a cooperative super call, it doesn't make sense for the parameter list to be different, because you don't know which method you're calling, so the signature of all methods along the super chain has to be standardised. -- Greg From greg.ewing at canterbury.ac.nz Mon Apr 30 03:07:01 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 30 Apr 2007 13:07:01 +1200 Subject: [Python-Dev] New Super PEP In-Reply-To: References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> Message-ID: <46354135.6080801@canterbury.ac.nz> Guido van Rossum wrote: > Nearly always wrong? You must be kidding. There are tons of reasons to > call your super method with modified arguments. E.g. clipping, > transforming, ... That's a good point. Just because the *signature* is the same doesn't mean the *values* of the parameters need to be the same. -- Greg From mithrandi-python-dev at mithrandi.za.net Mon Apr 30 02:11:37 2007 From: mithrandi-python-dev at mithrandi.za.net (Tristan Seligmann) Date: Mon, 30 Apr 2007 02:11:37 +0200 Subject: [Python-Dev] [Python-3000] Pre-pre PEP for 'super' keyword In-Reply-To: References: <76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com> <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> <002401c78778$75fb7eb0$0201a8c0@ryoko> <00b601c78a9f$38ec9390$0201a8c0@ryoko> Message-ID: <20070430001137.GA29084@mithrandi.za.net> * Guido van Rossum [2007-04-29 16:30:18 -0700]: > On 4/29/07, Jim Jewett wrote: > > So it is a "keyword" in the sense that None is a keyword; not in the > > stronger sense that "if" is a keyword? > > Um, how do you see those two differ? Is 'if' a keyword in the same > sense as 'or', or in a different sense? In my mind, 'if' and 'or' are "syntax", whereas things like 'None' or 'True' are "values"; even if None becomes an actual keyword, rather than a builtin. -- mithrandi, i Ainil en-Balandor, a faer Ambar -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: Digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20070430/c2005210/attachment.pgp From guido at python.org Mon Apr 30 03:19:20 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 29 Apr 2007 18:19:20 -0700 Subject: [Python-Dev] [Python-3000] Pre-pre PEP for 'super' keyword In-Reply-To: <20070430001137.GA29084@mithrandi.za.net> References: <76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com> <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> <002401c78778$75fb7eb0$0201a8c0@ryoko> <00b601c78a9f$38ec9390$0201a8c0@ryoko> <20070430001137.GA29084@mithrandi.za.net> Message-ID: On 4/29/07, Tristan Seligmann wrote: > * Guido van Rossum [2007-04-29 16:30:18 -0700]: > > > On 4/29/07, Jim Jewett wrote: > > > So it is a "keyword" in the sense that None is a keyword; not in the > > > stronger sense that "if" is a keyword? > > > > Um, how do you see those two differ? Is 'if' a keyword in the same > > sense as 'or', or in a different sense? > > In my mind, 'if' and 'or' are "syntax", whereas things like 'None' or > 'True' are "values"; even if None becomes an actual keyword, rather than > a builtin. I'm sorry, but that is such an incredibly subjective difference that I can't do anything with it. String literals and numeric literals are syntax too, even though they are values. A keyword, or reserved word, is simply something that looks like an identifier but is converted into a different token (by the lexer or by something sitting between the lexer and the parse) before the parser sees it. Also note that null is a keyword in Java. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From khabkr at yahoo.com Mon Apr 30 03:57:39 2007 From: khabkr at yahoo.com (Khalid A. Bakr) Date: Sun, 29 Apr 2007 18:57:39 -0700 (PDT) Subject: [Python-Dev] Python 2.5.1 Message-ID: <17623.55650.qm@web56611.mail.re3.yahoo.com> All test_os tests pass for me now on WinXP under MinGW. Please see http://wwww.python.org/sf/1709112 for details and patches. Regards, Khalid __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From tdelaney at avaya.com Mon Apr 30 04:48:26 2007 From: tdelaney at avaya.com (Delaney, Timothy (Tim)) Date: Mon, 30 Apr 2007 12:48:26 +1000 Subject: [Python-Dev] [Python-3000] Pre-pre PEP for 'super' keyword Message-ID: <2773CAC687FD5F4689F526998C7E4E5FF1ED71@au3010avexu1.global.avaya.com> Jim Jewett wrote: > On 4/29/07, Tim Delaney wrote: >> I've been intending to write up a PEP for fixing super, but I >> haven't had time to get to it. > > Calvin Spealman has the most recent draft. I hope he will incorporate > this into his draft. Sorry about this - wasn't receiving python-dev at home, so didn't realise Calvin had released the PEP. I think the current PEP draft is way too complicated - I don't think there's any need for descriptors, etc. I think we can make things work in the following way: 1. When a method is defined, the class is bound to it via an attribute (which in my version is called func_class). 2. Every non-static method has an implicit cell variable called 'super'. This would preferably not be able to be rebound. I also think it would be beneficial if the first parameter to the method couldn't be rebound (point 7 in my original email in this thread). 3. When a method is entered, the 'super' cell variable is populated by a call equivalent to: super = __builtin__.super(func_class, first_parameter) This would result in 'super' being a constant object, within the scope of the currently-executing method. 'keyword' was perhaps too strong - I was thinking this would only need to be done if 'super' were actually used, which would be easier to determine if 'super' actually were a keyword. This could still be done by not emitting the above call unless the 'super' cell variable were ever actually used. I've done bytecode-hacky stuff to do the equivalent of the above (as much as I've been able to), but a real implementation would just emit the correct bytecode (or java bytecode, or whatever) in the compiled code object. The issue of super() vs. super.__call__() ambiguity - I'll need to look at that when I get home. I'm a strong -1 against super() automatically passing the parameters that were passed to the currently-executing method. Tim Delaney From jimjjewett at gmail.com Mon Apr 30 05:29:25 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Sun, 29 Apr 2007 23:29:25 -0400 Subject: [Python-Dev] PEP 30XZ: Simplified Parsing Message-ID: PEP: 30xz Title: Simplified Parsing Version: $Revision$ Last-Modified: $Date$ Author: Jim J. Jewett Status: Draft Type: Standards Track Content-Type: text/plain Created: 29-Apr-2007 Post-History: 29-Apr-2007 Abstract Python initially inherited its parsing from C. While this has been generally useful, there are some remnants which have been less useful for python, and should be eliminated. + Implicit String concatenation + Line continuation with "\" + 034 as an octal number (== decimal 28). Note that this is listed only for completeness; the decision to raise an Exception for leading zeros has already been made in the context of PEP XXX, about adding a binary literal. Rationale for Removing Implicit String Concatenation Implicit String concatentation can lead to confusing, or even silent, errors. [1] def f(arg1, arg2=None): pass f("abc" "def") # forgot the comma, no warning ... # silently becomes f("abcdef", None) or, using the scons build framework, sourceFiles = [ 'foo.c', 'bar.c', #...many lines omitted... 'q1000x.c'] It's a common mistake to leave off a comma, and then scons complains that it can't find 'foo.cbar.c'. This is pretty bewildering behavior even if you *are* a Python programmer, and not everyone here is. Note that in C, the implicit concatenation is more justified; there is no other way to join strings without (at least) a function call. In Python, strings are objects which support the __add__ operator; it is possible to write: "abc" + "def" Because these are literals, this addition can still be optimized away by the compiler. Guido indicated [2] that this change should be handled by PEP, because there were a few edge cases with other string operators, such as the %. The resolution is to treat them the same as today. ("abc %s def" + "ghi" % var) # fails like today. # raises TypeError because of # precedence. (% before +) ("abc" + "def %s ghi" % var) # works like today; precedence makes # the optimization more difficult to # recognize, but does not change the # semantics. ("abc %s def" + "ghi") % var # works like today, because of # precedence: () before % # CPython compiler can already # add the literals at compile-time. Rationale for Removing Explicit Line Continuation A terminal "\" indicates that the logical line is continued on the following physical line (after whitespace). Note that a non-terminal "\" does not have this meaning, even if the only additional characters are invisible whitespace. (Python depends heavily on *visible* whitespace at the beginning of a line; it does not otherwise depend on *invisible* terminal whitespace.) Adding whitespace after a "\" will typically cause a syntax error rather than a silent bug, but it still isn't desirable. The reason to keep "\" is that occasionally code looks better with a "\" than with a () pair. assert True, ( "This Paren is goofy") But realistically, that paren is no worse than a "\". The only advantage of "\" is that it is slightly more familiar to users of C-based languages. These same languages all also support line continuation with (), so reading code will not be a problem, and there will be one less rule to learn for people entirely new to programming. Rationale for Removing Implicit Octal Literals This decision should be covered by PEP ???, on numeric literals. It is mentioned here only for completeness. C treats integers beginning with "0" as octal, rather than decimal. Historically, Python has inherited this usage. This has caused quite a few annoying bugs for people who forgot the rule, and tried to line up their constants. a = 123 b = 024 # really only 20, because octal c = 245 In Python 3.0, the second line will instead raise a SyntaxError, because of the ambiguity. Instead, the line should be written as in one of the following ways: b = 24 # PEP 8 b = 24 # columns line up, for quick scanning b = 0t24 # really did want an Octal! References [1] Implicit String Concatenation, Jewett, Orendorff http://mail.python.org/pipermail/python-ideas/2007-April/000397.html [2] PEP 12, Sample reStructuredText PEP Template, Goodger, Warsaw http://www.python.org/peps/pep-0012 [3] http://www.opencontent.org/openpub/ Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: From mithrandi-python-dev at mithrandi.za.net Mon Apr 30 05:36:19 2007 From: mithrandi-python-dev at mithrandi.za.net (Tristan Seligmann) Date: Mon, 30 Apr 2007 05:36:19 +0200 Subject: [Python-Dev] [Python-3000] Pre-pre PEP for 'super' keyword In-Reply-To: References: <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> <002401c78778$75fb7eb0$0201a8c0@ryoko> <00b601c78a9f$38ec9390$0201a8c0@ryoko> <20070430001137.GA29084@mithrandi.za.net> Message-ID: <20070430033619.GA1854@mithrandi.za.net> * Guido van Rossum [2007-04-29 18:19:20 -0700]: > > In my mind, 'if' and 'or' are "syntax", whereas things like 'None' or > > 'True' are "values"; even if None becomes an actual keyword, rather than > > a builtin. > > I'm sorry, but that is such an incredibly subjective difference that I > can't do anything with it. String literals and numeric literals are > syntax too, even though they are values. A keyword, or reserved word, > is simply something that looks like an identifier but is converted > into a different token (by the lexer or by something sitting between > the lexer and the parse) before the parser sees it. Let me try a less subjective description. Things like None, 2.3, 'foo', True are values or "expressions"; I'm not certain exactly what the term for these is in Python's grammar, but I basically mean something that can be on the RHS of an assignment.. However, something like 'for' or 'if' is part of some other grammatical construct, generally a statement or operator of some kind, so I tend to think of those differently. -- mithrandi, i Ainil en-Balandor, a faer Ambar -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: Digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20070430/c9f727ce/attachment-0001.pgp From foom at fuhm.net Mon Apr 30 05:41:56 2007 From: foom at fuhm.net (James Y Knight) Date: Sun, 29 Apr 2007 23:41:56 -0400 Subject: [Python-Dev] New Super PEP In-Reply-To: <46354135.6080801@canterbury.ac.nz> References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> <46354135.6080801@canterbury.ac.nz> Message-ID: <0890E5B0-8451-4C04-9F3F-B88BD130B9CE@fuhm.net> On Apr 29, 2007, at 9:07 PM, Greg Ewing wrote: > Guido van Rossum wrote: > >> Nearly always wrong? You must be kidding. There are tons of >> reasons to >> call your super method with modified arguments. E.g. clipping, >> transforming, ... > > That's a good point. Just because the *signature* is the > same doesn't mean the *values* of the parameters need > to be the same. This is quite true, and is indeed one case where it can be right to pass different arguments. Just in case there was confusion, I did not intend to imply that it should be impossible to pass different arguments, only that passing the same ones should be foolproof and not require repeating yourself. James From guido at python.org Mon Apr 30 05:58:28 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 29 Apr 2007 20:58:28 -0700 Subject: [Python-Dev] [Python-3000] Pre-pre PEP for 'super' keyword In-Reply-To: <2773CAC687FD5F4689F526998C7E4E5FF1ED71@au3010avexu1.global.avaya.com> References: <2773CAC687FD5F4689F526998C7E4E5FF1ED71@au3010avexu1.global.avaya.com> Message-ID: On 4/29/07, Delaney, Timothy (Tim) wrote: > I think the current PEP draft is way too complicated - I don't think > there's any need for descriptors, etc. I think we can make things work > in the following way: > > 1. When a method is defined, the class is bound to it via an attribute > (which in my version is called func_class). In Py3k all the func_XXX attrs are renamed __XXX__, so this would be __class__; but that's a name reserved for something else, so it would need to be something else. E.g. __containing_class__. Also, this would have to be done when the class is defined; when the method is being defined the class doesn't exist yet. > 2. Every non-static method has an implicit cell variable called 'super'. I think you're using 'cell' in a different sense than it is normally used in Python's implementation. What you are looking for is called a local variable (I deduce this from your initialization of the "cell" with something computed from the first argument). A 'cell' (as I introduced in my original back-of-the-envelop proposal; apparently you totally missed this reference) is a special kind of object containing a reference to another object and used to implement nonlocal variable references (the Python 2 read-only nonlocals as well as the Python 3 writable nonlocals). The cell is initialized in the surrounding scope and attached to the function object (as func_closure in 2.x, or __closure__ in 3.0). When the function is called, a reference to the cell is stored in the call frame. If anything in your proposal resembles a cell (in the sense that I and other Pythonistas mean it), it's the func_class attribute. In fact, in this sense our proposals are equivalent modulo slight implementation details; I proposed using a cell because there's an existing mechanism to get these mapped into a local variable, and because cells are cheaper than function attributes -- the first function attribute requires creating a dict, which is one of the larger built-in objects. > This would preferably not be able to be rebound. I also think it would > be beneficial if the first parameter to the method couldn't be rebound > (point 7 in my original email in this thread). The latter part sounds irrelevant to me. > 3. When a method is entered, the 'super' cell variable is populated by a > call equivalent to: > > super = __builtin__.super(func_class, first_parameter) > > This would result in 'super' being a constant object, within the scope > of the currently-executing method. 'keyword' was perhaps too strong - I > was thinking this would only need to be done if 'super' were actually > used, which would be easier to determine if 'super' actually were a > keyword. This could still be done by not emitting the above call unless > the 'super' cell variable were ever actually used. This sounds about right to me, with the strong preference that super *should* be made a keyword, and the built-in of the same name renamed to __super__, as it has a somewhat similar relationship to the super keyword as the built-in __import__ has to the import keyword. > I've done bytecode-hacky stuff to do the equivalent of the above (as > much as I've been able to), but a real implementation would just emit > the correct bytecode (or java bytecode, or whatever) in the compiled > code object. Shouldn't be too hard for someone who's hacked Python/compile.c before. > The issue of super() vs. super.__call__() ambiguity - I'll need to look > at that when I get home. Sounds irrelevant -- if super is equivalent to __builtin__.__super__(, ) then super never gets called; the only thing ever done to it (in the normal course of things) is to request an attribute of it. > I'm a strong -1 against super() automatically passing the parameters > that were passed to the currently-executing method. Same here. There seems to be the remaining controversy that there are three forms of super calls currently in use: super(ThisClass, self).method(...) # in a regular method super(ThisClass, cls).method(...) # in a class method super(ThisClass).method(...) # ??? The first two are handled by your "use the first argument" approach. The third exists so that you can create an "unbound" super instance which is useful for the oft-misunderstood autosuper example in my "descrintro" essay: http://www.python.org/download/releases/2.2.3/descrintro/#metaclass_examples . But since the latter is the hack that we're trying to replace with a proper implementation here, I suspect we can get away with only supporting the first two forms (and the third form is still supported using __builtin__.__super__). IOW I think you're on to something. Keep up the good word. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jseutter at gmail.com Mon Apr 30 06:21:12 2007 From: jseutter at gmail.com (Jerry Seutter) Date: Sun, 29 Apr 2007 22:21:12 -0600 Subject: [Python-Dev] test_pty fails on Sparc Solaris 10 for trunk In-Reply-To: <3f09d5a00704241047w6912798asc7fb7388a909f84e@mail.gmail.com> References: <3f09d5a00704241047w6912798asc7fb7388a909f84e@mail.gmail.com> Message-ID: <2c8d48d70704292121k6c97599fpd5bbf049e0a4727c@mail.gmail.com> I was mucking about in test_pty.py not too long ago. I'll take a look. Jerry On 4/24/07, Grig Gheorghiu wrote: > > This is happening both in the Python buildbot farm, and in the > community buildbot farm. > > See: > > > http://www.python.org/dev/buildbot/trunk/sparc%20solaris10%20gcc%20trunk/builds/1960/step-test/0 > > http://www.python.org/dev/buildbot/community/trunk/sparc%20Solaris%2010%20trunk/builds/484/step-test/0 > > Grig > > -- > http://agiletesting.blogspot.com > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/jseutter%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070429/3c04bb0e/attachment.htm From nnorwitz at gmail.com Mon Apr 30 06:27:39 2007 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 29 Apr 2007 21:27:39 -0700 Subject: [Python-Dev] test_pty fails on Sparc Solaris 10 for trunk In-Reply-To: <2c8d48d70704292121k6c97599fpd5bbf049e0a4727c@mail.gmail.com> References: <3f09d5a00704241047w6912798asc7fb7388a909f84e@mail.gmail.com> <2c8d48d70704292121k6c97599fpd5bbf049e0a4727c@mail.gmail.com> Message-ID: No need to worry about it, I fixed the failing test. Although it would be good to review the comments and make them more accurate. Take a look at the last change to test_pty.py. n -- On 4/29/07, Jerry Seutter wrote: > I was mucking about in test_pty.py not too long ago. I'll take a look. > > Jerry > > > On 4/24/07, Grig Gheorghiu < grig.gheorghiu at gmail.com> wrote: > > > > This is happening both in the Python buildbot farm, and in the > > community buildbot farm. > > > > See: > > > > > http://www.python.org/dev/buildbot/trunk/sparc%20solaris10%20gcc%20trunk/builds/1960/step-test/0 > > > http://www.python.org/dev/buildbot/community/trunk/sparc%20Solaris%2010%20trunk/builds/484/step-test/0 > > > > Grig > > > > -- > > http://agiletesting.blogspot.com > > _______________________________________________ > > Python-Dev mailing list > > Python-Dev at python.org > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/jseutter%40gmail.com > > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com > > From nnorwitz at gmail.com Mon Apr 30 06:28:26 2007 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 29 Apr 2007 21:28:26 -0700 Subject: [Python-Dev] deprecating macpath and macurl2path Message-ID: PEP 11 notes that Mac OS 9 support was unsupported in 2.4. There are still quite a few places that we check for sys.platform == 'mac'. There are also (at least) 2 modules (macpath and macurl2path) that look specific to Mac OS. (OS X has sys.platform == 'darwin'.) Shall I add a deprecation warning on import to these modules or remove them? I plan to remove other mac specific code (checking sys.platform) in the stdlib unless someone beat me to it. n From tdelaney at avaya.com Mon Apr 30 06:56:20 2007 From: tdelaney at avaya.com (Delaney, Timothy (Tim)) Date: Mon, 30 Apr 2007 14:56:20 +1000 Subject: [Python-Dev] [Python-3000] Pre-pre PEP for 'super' keyword Message-ID: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com> Guido van Rossum wrote: >> 1. When a method is defined, the class is bound to it via an attribute >> (which in my version is called func_class). > In Py3k all the func_XXX attrs are renamed __XXX__, so this would be > __class__; but that's a name reserved for something else, so it would > need to be something else. E.g. __containing_class__. Yep - I've just used a placeholder name. > Also, this would have to be done when the class is defined; when the > method is being defined the class doesn't exist yet. Good point. Change that to "at the first opportunity" ;) >> 2. Every non-static method has an implicit cell variable called >> 'super'. > > I think you're using 'cell' in a different sense than it is normally > used in Python's implementation. What you are looking for is called a > local variable (I deduce this from your initialization of the "cell" > with something computed from the first argument). Actually, I think I'm using the terminology correctly - I'm talking about an entry in co_cellvars. Given the following class: class A(object): def f(self): super = super_factory() def inner(): return 'A' + super.f() print inner() the use of 'super' in both A.f and A.f.inner will produce an entry in A.f.func_code.co_cellvars and A.f.inner.func_code.co_freevars. What I'm proposing is that the `super = super_factory()` line be implicit in this case, resulting in the following code behaving identically: class A(object): def f(self): def inner(): return 'A' + super.f() print inner() >> The issue of super() vs. super.__call__() ambiguity - I'll need to >> look at that when I get home. > > Sounds irrelevant -- if super is equivalent to > __builtin__.__super__(, ) then super never gets > called; the only thing ever done to it (in the normal course of > things) is to request an attribute of it. Sorry - this is related to my proposal that the following two bits of code behave the same: class A(object): def f(self, *p, **kw): super.f(*p, **kw) class A(object): def f(self, *p, **kw): super(*p, **kw) But as has been pointed out, this creates an ambiguity with: class A(object): def f(self, *p, **kw): super.__call__(*p, **kw) so I want to see if I can resolve it. > super(ThisClass).method(...) # ??? > > The third exists so that you can create an "unbound" super instance > which is useful for the oft-misunderstood autosuper example in my > "descrintro" essay: > http://www.python.org/download/releases/2.2.3/descrintro/#metaclass_exam ples > . > > But since the latter is the hack that we're trying to replace with a > proper implementation here, I suspect we can get away with only > supporting the first two forms (and the third form is still supported > using __builtin__.__super__). Yep - that's my thought as well. I think it would be very rare to need super(ThisClass), although it makes some sense that that would be what super means in a static method ... Tim Delaney From snaury at gmail.com Mon Apr 30 09:47:55 2007 From: snaury at gmail.com (Alexey Borzenkov) Date: Mon, 30 Apr 2007 11:47:55 +0400 Subject: [Python-Dev] Python 2.5.1 In-Reply-To: <4634E26C.5030208@v.loewis.de> References: <4633F38A.4040605@v.loewis.de> <46345127.1020504@v.loewis.de> <76fd5acf0704290522u674118eai97c4a8fe778243b4@mail.gmail.com> <463494AC.1080608@v.loewis.de> <76fd5acf0704290837t1c57ede8wdb6e45ab6c6f3b95@mail.gmail.com> <4634C1FC.7090408@v.loewis.de> <76fd5acf0704290953i6dcc0bffk98d49619ef922e1b@mail.gmail.com> <4634DB32.1020009@v.loewis.de> <76fd5acf0704291057g590b909emc9ef3dfa6343450d@mail.gmail.com> <4634E26C.5030208@v.loewis.de> Message-ID: Hi Martin, On 4/29/07, "Martin v. L?wis" wrote: > The error Windows reports is ERROR_SHARING_VIOLATION. I never > understood sharing fully, but it may be that if the file is opened > in "exclusive sharing", stat'ing it may fail. Sharing is actually very easy. If you didn't specify SHARE_READ/SHARE_WRITE/SHARE_DELETE when opening the file and you succeeded, then any process can't read/write/delete the file until you close the handle. > I personally consider it a bug in Windows that you cannot get file > attributes if some other process has opened it. Exclusive access > should only restrict access to file contents, but not file attributes. After doing some research I found that it seems to be impossible to use CreateFile for a file that doesn't have SHARE_READ. I played with different combinations and with FLAG_BACKUP_SEMANTICS and nothing helped. However on Windows there's still a possibility to read attributes: use FindFirstFile. _WIN32_FIND_DATA structure seems to have all the necessary fields (attributes, file times, size and full/short filename), and FindFirstFile doesn't care about sharing at all... From skip at pobox.com Mon Apr 30 11:41:47 2007 From: skip at pobox.com (skip at pobox.com) Date: Mon, 30 Apr 2007 04:41:47 -0500 Subject: [Python-Dev] New Super PEP In-Reply-To: <0890E5B0-8451-4C04-9F3F-B88BD130B9CE@fuhm.net> References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> <46354135.6080801@canterbury.ac.nz> <0890E5B0-8451-4C04-9F3F-B88BD130B9CE@fuhm.net> Message-ID: <17973.47579.138026.658402@montanaro.dyndns.org> James> I did not intend to imply that it should be impossible to pass James> different arguments, only that passing the same ones should be James> foolproof and not require repeating yourself. Then some other argument syntax than "()" is needed which clearly means, "pass no arguments". Skip From rrr at ronadam.com Mon Apr 30 11:43:16 2007 From: rrr at ronadam.com (Ron Adam) Date: Mon, 30 Apr 2007 04:43:16 -0500 Subject: [Python-Dev] [Python-3000] Pre-pre PEP for 'super' keyword In-Reply-To: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com> References: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com> Message-ID: <4635BA34.6060404@ronadam.com> Delaney, Timothy (Tim) wrote: > What I'm proposing is that the `super = super_factory()` line be > implicit in this case, resulting in the following code behaving > identically: > > class A(object): > def f(self): > def inner(): > return 'A' + super.f() > > print inner() As Guido pointed out it has some resemblance to how import works, but I also think there is resemblance to the context of how global is used. So if it is made into a keyword, could it work like the global keyword? class A(object): def f(self): def inner(): super f return 'A' + f() print inner() Cheers, Ron From tcdelaney at optusnet.com.au Mon Apr 30 13:38:30 2007 From: tcdelaney at optusnet.com.au (Tim Delaney) Date: Mon, 30 Apr 2007 21:38:30 +1000 Subject: [Python-Dev] [Python-3000] Pre-pre PEP for 'super' keyword References: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com> Message-ID: <00d701c78b1c$143b3c40$0201a8c0@ryoko> From: "Delaney, Timothy (Tim)" > Sorry - this is related to my proposal that the following two bits of > code behave the same: > > class A(object): > def f(self, *p, **kw): > super.f(*p, **kw) > > class A(object): > def f(self, *p, **kw): > super(*p, **kw) > > But as has been pointed out, this creates an ambiguity with: > > class A(object): > def f(self, *p, **kw): > super.__call__(*p, **kw) > > so I want to see if I can resolve it. A 'super' instance would be callable, without being able to access it's __call__ method (because super.__call__ would refer to the base class method of that name). But I find I really don't care. The only place where that would really matter IMO is if you want to find out if a 'super' instance is callable. Calling a base class __call__ method would not be ambiguous - the following two classes would work the same: class A(object): def __call__(self, *p, **kw): return super.__call__(*p, **kw) class A(object): def __call__(self, *p, **kw): return super(*p, **kw) So, I guess my question is whether the most common case of calling the base class method with the same name is worth having some further syntactic sugar to avoid repetition? I think it is, but that would be your call Guido. Cheers, Tim Delaney From dynamobigjosh at hotmail.com Mon Apr 30 13:42:48 2007 From: dynamobigjosh at hotmail.com (JOSHUA ABRAHAM) Date: Mon, 30 Apr 2007 11:42:48 +0000 Subject: [Python-Dev] (no subject) Message-ID: I was hoping you guys would consider creating function in os.path or otherwise that would find the full path of a file when given only it's base name and nothing else.I have been made to understand that this is not currently possible.If it is in any can you please inform me of it[i know this is not a tutorial] but please i need thiis badly if it's possible.Thank you _________________________________________________________________ FREE pop-up blocking with the new MSN Toolbar - get it now! http://toolbar.msn.click-url.com/go/onm00200415ave/direct/01/ From skip at pobox.com Mon Apr 30 14:17:21 2007 From: skip at pobox.com (skip at pobox.com) Date: Mon, 30 Apr 2007 07:17:21 -0500 Subject: [Python-Dev] Find file given basename In-Reply-To: References: Message-ID: <17973.56913.622871.463262@montanaro.dyndns.org> Joshua> I was hoping you guys would consider creating function in Joshua> os.path or otherwise that would find the full path of a file Joshua> when given only it's base name and nothing else.I have been made Joshua> to understand that this is not currently possible.If it is in Joshua> any can you please inform me of it[i know this is not a Joshua> tutorial] but please i need thiis badly if it's possible.Thank Joshua> you Joshua, This topic should be discussed (at least until there is a concrete proposal) on the Usenet newgroup comp.lang.python (a.k.a. python-list at python.org). It would be helpful if you presented a concrete example. As stated though there is no unique file that matches a given basename. You might also want to check out the glob module to see if it addresses your needs. Skip From ironfroggy at gmail.com Mon Apr 30 14:29:22 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Mon, 30 Apr 2007 08:29:22 -0400 Subject: [Python-Dev] [Python-3000] Pre-pre PEP for 'super' keyword In-Reply-To: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com> References: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com> Message-ID: <76fd5acf0704300529r1801bc6at9a2f3d5329d1ea0d@mail.gmail.com> On 4/30/07, Delaney, Timothy (Tim) wrote: > Guido van Rossum wrote: > > >> 1. When a method is defined, the class is bound to it via an > attribute > >> (which in my version is called func_class). > > > In Py3k all the func_XXX attrs are renamed __XXX__, so this would be > > __class__; but that's a name reserved for something else, so it would > > need to be something else. E.g. __containing_class__. > > Yep - I've just used a placeholder name. > > > Also, this would have to be done when the class is defined; when the > > method is being defined the class doesn't exist yet. > > Good point. Change that to "at the first opportunity" ;) > > >> 2. Every non-static method has an implicit cell variable called > >> 'super'. > > > > I think you're using 'cell' in a different sense than it is normally > > used in Python's implementation. What you are looking for is called a > > local variable (I deduce this from your initialization of the "cell" > > with something computed from the first argument). > > Actually, I think I'm using the terminology correctly - I'm talking > about an entry in co_cellvars. Given the following class: > > class A(object): > def f(self): > super = super_factory() > > def inner(): > return 'A' + super.f() > > print inner() > > the use of 'super' in both A.f and A.f.inner will produce an entry in > A.f.func_code.co_cellvars and A.f.inner.func_code.co_freevars. What I'm > proposing is that the `super = super_factory()` line be implicit in this > case, resulting in the following code behaving identically: > > class A(object): > def f(self): > def inner(): > return 'A' + super.f() > > print inner() I believe the direction my PEP took with all this is a good bit primitive compared to this approach, although I still find value in it because at least a prototype came out of it that can be used to test the waters, regardless of if a more direct-in-the-language approach would be superior. However, I seem to think that if the __this_class__ PEP goes through, your version can be simplified as well. No tricky stuffy things in cells would be needed, but we can just expand the super 'keyword' to __super__(__this_class__, self), which has been suggested at least once. It seems this would be much simpler to implement, and it also brings up a second point. Also, I like that the super object is created at the beginning of the function, which my proposal couldn't even do. It is more efficient if you have multiple super calls, and gets around a problem I completely missed: what happens if the instance name were rebound before the implicit lookup of the instance object at the time of the super call? > >> The issue of super() vs. super.__call__() ambiguity - I'll need to > >> look at that when I get home. > > > > Sounds irrelevant -- if super is equivalent to > > __builtin__.__super__(, ) then super never gets > > called; the only thing ever done to it (in the normal course of > > things) is to request an attribute of it. > > Sorry - this is related to my proposal that the following two bits of > code behave the same: > > class A(object): > def f(self, *p, **kw): > super.f(*p, **kw) > > class A(object): > def f(self, *p, **kw): > super(*p, **kw) > > But as has been pointed out, this creates an ambiguity with: > > class A(object): > def f(self, *p, **kw): > super.__call__(*p, **kw) > > so I want to see if I can resolve it. Turns out, it doesn't. A lot of people expect it, but it turns out to be simple. super(*p, **kw) is not equivalent to super.__call__(*p, **kw) but to type(super).__call__.__get__(super, type(super))(*p, **kw), due to operator methods being looked up on the type directly and bound by the descriptor, without any lookup on the super object itself. Thus, no attribute is ever done on the super object itself, type(super).__getattribute__ is never invoked, and there is no problem at all. This also means we can still invoke super the old, explicit way. I've already got my copy of the PEP updated to reflect this. > > super(ThisClass).method(...) # ??? > > > > The third exists so that you can create an "unbound" super instance > > which is useful for the oft-misunderstood autosuper example in my > > "descrintro" essay: > > > http://www.python.org/download/releases/2.2.3/descrintro/#metaclass_exam > ples > > . > > > > But since the latter is the hack that we're trying to replace with a > > proper implementation here, I suspect we can get away with only > > supporting the first two forms (and the third form is still supported > > using __builtin__.__super__). > > Yep - that's my thought as well. I think it would be very rare to need > super(ThisClass), although it makes some sense that that would be what > super means in a static method ... Does super mean anything in a static method today? -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From ironfroggy at gmail.com Mon Apr 30 14:33:43 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Mon, 30 Apr 2007 08:33:43 -0400 Subject: [Python-Dev] New Super PEP In-Reply-To: <17973.47579.138026.658402@montanaro.dyndns.org> References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> <46354135.6080801@canterbury.ac.nz> <0890E5B0-8451-4C04-9F3F-B88BD130B9CE@fuhm.net> <17973.47579.138026.658402@montanaro.dyndns.org> Message-ID: <76fd5acf0704300533n2f70d403g5871275c22426c5c@mail.gmail.com> On 4/30/07, skip at pobox.com wrote: > > James> I did not intend to imply that it should be impossible to pass > James> different arguments, only that passing the same ones should be > James> foolproof and not require repeating yourself. > > Then some other argument syntax than "()" is needed which clearly means, > "pass no arguments". I second, because I would like an actually good way to do this. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From eric+python-dev at trueblade.com Mon Apr 30 14:39:51 2007 From: eric+python-dev at trueblade.com (Eric V. Smith) Date: Mon, 30 Apr 2007 08:39:51 -0400 Subject: [Python-Dev] [Python-3000] PEP 30XZ: Simplified Parsing In-Reply-To: References: Message-ID: <4635E397.9000403@trueblade.com> Jim Jewett wrote: > Rationale for Removing Implicit String Concatenation > > Implicit String concatentation can lead to confusing, or even > silent, errors. [1] > > def f(arg1, arg2=None): pass > > f("abc" "def") # forgot the comma, no warning ... > # silently becomes f("abcdef", None) > > or, using the scons build framework, > > sourceFiles = [ > 'foo.c', > 'bar.c', > #...many lines omitted... > 'q1000x.c'] > Since your first example omits the comma, I think this one should, too. sourceFiles = [ 'foo.c' 'bar.c', #...many lines omitted... 'q1000x.c'] That is, either both examples should show an error, or both examples should work, but point out how easy it is to make an error. Eric. From tcdelaney at optusnet.com.au Mon Apr 30 14:47:41 2007 From: tcdelaney at optusnet.com.au (Tim Delaney) Date: Mon, 30 Apr 2007 22:47:41 +1000 Subject: [Python-Dev] [Python-3000] Pre-pre PEP for 'super' keyword References: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com> <76fd5acf0704300529r1801bc6at9a2f3d5329d1ea0d@mail.gmail.com> Message-ID: <00e701c78b25$be456c20$0201a8c0@ryoko> From: "Calvin Spealman" > I believe the direction my PEP took with all this is a good bit > primitive compared to this approach, although I still find value in it > because at least a prototype came out of it that can be used to test > the waters, regardless of if a more direct-in-the-language approach > would be superior. I've been working on improved super syntax for quite a while now - my original approach was 'self.super' which used _getframe() and mro crawling too. I hit on using bytecode hacking to instantiate a super object at the start of the method to gain performance, which required storing the class in co_consts, etc. It turns out that using a metaclass then makes this a lot cleaner. > However, I seem to think that if the __this_class__ PEP goes through, > your version can be simplified as well. No tricky stuffy things in > cells would be needed, but we can just expand the super 'keyword' to > __super__(__this_class__, self), which has been suggested at least > once. It seems this would be much simpler to implement, and it also > brings up a second point. > > Also, I like that the super object is created at the beginning of the > function, which my proposal couldn't even do. It is more efficient if > you have multiple super calls, and gets around a problem I completely > missed: what happens if the instance name were rebound before the > implicit lookup of the instance object at the time of the super call? You could expand it inline, but I think your second point is a strong argument against it. Also, sticking the super instance into a cell means that inner classes get access to it for free. Otherwise each inner class would *also* need to instantiate a super instance, and __this_class__ (or whatever it's called) would need to be in a cell for them to get access to it instead. BTW, one of my test cases involves multiple super calls in the same method - there is a *very* large performance improvement by instantiating it once. >> I think it would be very rare to need >> super(ThisClass), although it makes some sense that that would be what >> super means in a static method ... > > Does super mean anything in a static method today? Well, since all super instantiations are explicit currently, it can mean whatever you want it to. class A(object): @staticmethod def f(): print super(A) print super(A, A) Cheers, Tim Delaney From ironfroggy at gmail.com Mon Apr 30 15:09:35 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Mon, 30 Apr 2007 09:09:35 -0400 Subject: [Python-Dev] [Python-3000] Pre-pre PEP for 'super' keyword In-Reply-To: <00e701c78b25$be456c20$0201a8c0@ryoko> References: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com> <76fd5acf0704300529r1801bc6at9a2f3d5329d1ea0d@mail.gmail.com> <00e701c78b25$be456c20$0201a8c0@ryoko> Message-ID: <76fd5acf0704300609o4285b9d6p4895a891420aa88d@mail.gmail.com> On 4/30/07, Tim Delaney wrote: > I've been working on improved super syntax for quite a while now - my > original approach was 'self.super' which used _getframe() and mro crawling > too. I hit on using bytecode hacking to instantiate a super object at the > start of the method to gain performance, which required storing the class in > co_consts, etc. It turns out that using a metaclass then makes this a lot > cleaner. > > > However, I seem to think that if the __this_class__ PEP goes through, > > your version can be simplified as well. No tricky stuffy things in > > cells would be needed, but we can just expand the super 'keyword' to > > __super__(__this_class__, self), which has been suggested at least > > once. It seems this would be much simpler to implement, and it also > > brings up a second point. > > > > Also, I like that the super object is created at the beginning of the > > function, which my proposal couldn't even do. It is more efficient if > > you have multiple super calls, and gets around a problem I completely > > missed: what happens if the instance name were rebound before the > > implicit lookup of the instance object at the time of the super call? > > You could expand it inline, but I think your second point is a strong > argument against it. Also, sticking the super instance into a cell means > that inner classes get access to it for free. Otherwise each inner class > would *also* need to instantiate a super instance, and __this_class__ (or > whatever it's called) would need to be in a cell for them to get access to > it instead. > > BTW, one of my test cases involves multiple super calls in the same method - > there is a *very* large performance improvement by instantiating it once. Note that I would now advocate the creation of the super object at the beginning of function, and when I talk about expanding super to super(__this_class__, self) I do mean doing it at the beginning of the function, just like you are saying, so we're in agreement here. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From arigo at tunes.org Mon Apr 30 15:14:27 2007 From: arigo at tunes.org (Armin Rigo) Date: Mon, 30 Apr 2007 15:14:27 +0200 Subject: [Python-Dev] New Super PEP In-Reply-To: <76fd5acf0704300534i48fb29c6lc3fe873c375b6e9d@mail.gmail.com> References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> <20070429185012.GA32077@code0.codespeak.net> <76fd5acf0704291153y5d20f453u66d7c8bbd64d62ad@mail.gmail.com> <20070429204222.GA7987@code0.codespeak.net> <76fd5acf0704291504j5687e2a0sfd8158303b180534@mail.gmail.com> <20070430095232.GA23558@code0.codespeak.net> <76fd5acf0704300534i48fb29c6lc3fe873c375b6e9d@mail.gmail.com> Message-ID: <20070430131427.GA5518@code0.codespeak.net> Hi Calvin, On Mon, Apr 30, 2007 at 08:34:56AM -0400, Calvin Spealman wrote: > If you want, you can also grab the reference > implementation from my blog: http://ironfroggy-code.blogspot.com/ This reference implementation is broken. It doesn't pass the following test, for example: class A(object): __metaclass__ = autosuper def f(self): if type(self) is B: return "we've got a B" else: return "oups" class B(A): def f(self): return Super.f() assert B().f() == "we've got a B" I don't honestly think that you can easily fix the implementation to pass this test, but I'm ready to check proposed fixes and write tests for them. If there is really no working implementation yet then it shouldn't have been a PEP posted on python-dev in the first place, IMHO (and the discussion shouldn't go on on python-dev either). A bientot, Armin. From ironfroggy at gmail.com Mon Apr 30 15:18:48 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Mon, 30 Apr 2007 09:18:48 -0400 Subject: [Python-Dev] New Super PEP In-Reply-To: <20070430131427.GA5518@code0.codespeak.net> References: <76fd5acf0704281943i6ea9162by4448cb7ce5b646bf@mail.gmail.com> <20070429185012.GA32077@code0.codespeak.net> <76fd5acf0704291153y5d20f453u66d7c8bbd64d62ad@mail.gmail.com> <20070429204222.GA7987@code0.codespeak.net> <76fd5acf0704291504j5687e2a0sfd8158303b180534@mail.gmail.com> <20070430095232.GA23558@code0.codespeak.net> <76fd5acf0704300534i48fb29c6lc3fe873c375b6e9d@mail.gmail.com> <20070430131427.GA5518@code0.codespeak.net> Message-ID: <76fd5acf0704300618v4edd9816odecad4be95cec738@mail.gmail.com> On 4/30/07, Armin Rigo wrote: > Hi Calvin, > > On Mon, Apr 30, 2007 at 08:34:56AM -0400, Calvin Spealman wrote: > > If you want, you can also grab the reference > > implementation from my blog: http://ironfroggy-code.blogspot.com/ > > This reference implementation is broken. It doesn't pass the following > test, for example: > > class A(object): > __metaclass__ = autosuper > def f(self): > if type(self) is B: > return "we've got a B" > else: > return "oups" > > class B(A): > def f(self): > return Super.f() > > assert B().f() == "we've got a B" > > I don't honestly think that you can easily fix the implementation to > pass this test, but I'm ready to check proposed fixes and write tests > for them. If there is really no working implementation yet then it > shouldn't have been a PEP posted on python-dev in the first place, IMHO > (and the discussion shouldn't go on on python-dev either). You are absolutely correct, it should not have gone to python-dev. It was supposed to go to python-3000, where the initial discussion took place, where Guido asked for someone to write the PEP right off the bat. However, as much as I love GMail, it makes it less than easy to always know which mailing list I'm on, so I actually thought I was in that discussion at dev, not 3000. I apologize to everyone for mixing the threads between lists! This is exactly why I need to start pulling in all the dev lists to the summaries. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From ronaldoussoren at mac.com Mon Apr 30 15:45:34 2007 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Mon, 30 Apr 2007 06:45:34 -0700 Subject: [Python-Dev] deprecating macpath and macurl2path In-Reply-To: References: Message-ID: <58BA292E-0112-1000-C0F5-0AF6D7A7DBCD-Webmail-10014@mac.com> On Monday, April 30, 2007, at 06:28AM, "Neal Norwitz" wrote: >PEP 11 notes that Mac OS 9 support was unsupported in 2.4. There are >still quite a few places that we check for sys.platform == 'mac'. >There are also (at least) 2 modules (macpath and macurl2path) that >look specific to Mac OS. (OS X has sys.platform == 'darwin'.) > >Shall I add a deprecation warning on import to these modules or remove them? > >I plan to remove other mac specific code (checking sys.platform) in >the stdlib unless someone beat me to it. I don't know if anyone uses macpath for it, but OS9-style paths are still used in a number of APIs on OSX. macurl2path seems OS9 specific. Ronald From draghuram at gmail.com Mon Apr 30 16:49:09 2007 From: draghuram at gmail.com (Raghuram Devarakonda) Date: Mon, 30 Apr 2007 10:49:09 -0400 Subject: [Python-Dev] os.rename on windows Message-ID: <2c51ecee0704300749m63620406j4c8d9d6bff6aee6a@mail.gmail.com> Hi, I have submitted a patch (http://www.python.org/sf/1704547) that allows os.rename to replace the destination file if it exists, on windows. As part of discussion in the tracker, Martin suggested that python-dev should discuss the change. Currently, os.rename() on windows uses the API MoveFile() which fails if the destination file exists. The patch replaces this API with MoveFileEx() and uses the flag MOVEFILE_REPLACE_EXISTING which causes the destination file to be replaced if it exists. However, this change is subtle and if there is any existing code that depends on current os.rename behaviour on windows, their code is silently broken with (any) destination file being overwritten. But the functionality of replacing is important and I would like to know the best of way of supporting it. If it is deemed that this change is not good to go in as-is, how about having an optional parameter to os.rename (say, win_replace) that can be used by callers to explicitly request replacing? I must also point out that the patch uses another flag MOVEFILE_COPY_ALLOWED effectively allowing renamed files to be on separate file systems. The renaming in this case is not atomic and I used this flag only to support current functionality. It is not a bad idea to disallow such renames which brings it in line with the behaviour on many unix flavors. This also has the potential to break code but not silently. Lastly, I found an old discussion about the same topic by this list. http://mail.python.org/pipermail/python-dev/2001-May/014957.html Even though Guido indicated that he doesn't support API change in this thread, I am posting again as I did not see any one mention MoveFileEx() in that thread. Thanks, Raghu. From guido at python.org Mon Apr 30 19:43:18 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 30 Apr 2007 10:43:18 -0700 Subject: [Python-Dev] [Python-3000] Pre-pre PEP for 'super' keyword In-Reply-To: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com> References: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com> Message-ID: On 4/29/07, Delaney, Timothy (Tim) wrote: > Guido van Rossum wrote: > >> 2. Every non-static method has an implicit cell variable called > >> 'super'. > > > > I think you're using 'cell' in a different sense than it is normally > > used in Python's implementation. What you are looking for is called a > > local variable (I deduce this from your initialization of the "cell" > > with something computed from the first argument). > > Actually, I think I'm using the terminology correctly - I'm talking > about an entry in co_cellvars. Given the following class: > > class A(object): > def f(self): > super = super_factory() > > def inner(): > return 'A' + super.f() > > print inner() > > the use of 'super' in both A.f and A.f.inner will produce an entry in > A.f.func_code.co_cellvars and A.f.inner.func_code.co_freevars. What I'm > proposing is that the `super = super_factory()` line be implicit in this > case, resulting in the following code behaving identically: > > class A(object): > def f(self): > def inner(): > return 'A' + super.f() > > print inner() OK, I see now. I thought you meant for the cell-ness to be related to the basic implementation; but you meant it so that it woks correctly inside nested functions. That's fine of course. You might clarify this somewhere. > >> The issue of super() vs. super.__call__() ambiguity - I'll need to > >> look at that when I get home. > > > > Sounds irrelevant -- if super is equivalent to > > __builtin__.__super__(, ) then super never gets > > called; the only thing ever done to it (in the normal course of > > things) is to request an attribute of it. > > Sorry - this is related to my proposal that the following two bits of > code behave the same: > > class A(object): > def f(self, *p, **kw): > super.f(*p, **kw) > > class A(object): > def f(self, *p, **kw): > super(*p, **kw) > > But as has been pointed out, this creates an ambiguity with: > > class A(object): > def f(self, *p, **kw): > super.__call__(*p, **kw) > > so I want to see if I can resolve it. I think the shortcut is more confusing than helpful; I propose to drop it for the sake of focusing on the essential. > > super(ThisClass).method(...) # ??? > > > > The third exists so that you can create an "unbound" super instance > > which is useful for the oft-misunderstood autosuper example in my > > "descrintro" essay: > > > > http://www.python.org/download/releases/2.2.3/descrintro/#metaclass_exampples > > . > > > > But since the latter is the hack that we're trying to replace with a > > proper implementation here, I suspect we can get away with only > > supporting the first two forms (and the third form is still supported > > using __builtin__.__super__). > > Yep - that's my thought as well. I think it would be very rare to need > super(ThisClass), although it makes some sense that that would be what > super means in a static method ... But that doesn't work with the current (2.x) super, and hasn't been requested AFAIK, so I'd suggest dropping the use case -- KISS again. -- --Guido van Rossum (home page: http://www.python.org/~guido/)