From andymac at bullseye.apana.org.au Tue Jan 1 00:59:36 2008 From: andymac at bullseye.apana.org.au (Andrew MacIntyre) Date: Tue, 01 Jan 2008 09:59:36 +1000 Subject: [Python-Dev] Fate of Windows build directories In-Reply-To: <4778E8B3.3090004@cheimes.de> References: <4778E8B3.3090004@cheimes.de> Message-ID: <47798268.7020809@bullseye.andymac.org> Christian Heimes wrote: {...} > PC/os2*/ > OS2 build dirs. The files are no longer maintained. While I haven't been visible much lately, especially in the 3.0 space, I do have plans to continue supporting the OS/2 EMX environment (PC/os2emx). For 3.x, the PC/os2vacpp subdirectory can go, as I don't expect to be able to support the VAC++ toolchain. -- ------------------------------------------------------------------------- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac at bullseye.apana.org.au (pref) | Snail: PO Box 370 andymac at pcug.org.au (alt) | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From martin at v.loewis.de Tue Jan 1 22:18:51 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 01 Jan 2008 22:18:51 +0100 Subject: [Python-Dev] Tkinter problems with Tcl/Tk 8.5 In-Reply-To: <20071215153445.szf1gwdlw4ssk484@webmail.xmission.com> References: <20071215153445.szf1gwdlw4ssk484@webmail.xmission.com> Message-ID: <477AAE3B.5060008@v.loewis.de> > It seems very peculiar how the text widget's bbox is returning a > Python-like list and therefore breaking the Tcl callback. I haven't > thus far been able to determine which python method is causing that, > or if it's something related to the hooks you have added. The same > problem doesn't occur with 8.4. I have now studied this in detail, and fixed it in Python's trunk; see the tracker item for details. In short: - IDLE redirects all widget sub-commands, either calling an overridden definition, or the original Tk command. - in the redirection of "bbox", Tk 8.4 would return a string, whereas 8.5 now returns a list. - _tkinter converts the string to a Python string, and the list to a Python tuple - IDLE returns the Python value to _tkinter, which in turn converts it to a string. This leaves the 8.4 string unchanged, but converts the 8.5 tuple into something like "(10, 10, 20, 20)". - Tk interprets it as a list of numbers, choking as "(10," is not a meaningful number. The fix is to return an ObjResult to Tcl from a Python callback. I'm skeptical about back-porting this to 2.5, as it may affect behavior. So for 2.5, we probably have to recommend not using Tk 8.5. There are a number of additional incompatible changes. For example, text::index returns textindex objects now, where it used to return strings. I have fixed that in Tkinter, which converts the textindex back to a string. I'm sure there are other places where Tk 8.5 will break existing Tkinter applications. Regards, Martin From python at rcn.com Thu Jan 3 02:19:27 2008 From: python at rcn.com (Raymond Hettinger) Date: Wed, 2 Jan 2008 20:19:27 -0500 (EST) Subject: [Python-Dev] Syntax suggestion for imports Message-ID: <20080102201927.AEJ96366@ms19.lnh.mail.rcn.net> The standard library, my personal code, third-party packages, and my employer's code base are filled with examples of the following pattern: try: import threading except ImportError: import dummy_threading as threading try: import xml.etree.cElementTree as ET except ImportError: try: import cElementTree as ET except ImportError: import elementtree.ElementTree as ET try: from cStringIO import StringIO except ImportError: from StringIO import StringIO try: import readline except ImportError: pass How about a new, simpler syntax: * import threading or dummy_threading as threading * import xml.etree.CElementTree or cElementTree or elementree.ElementTree as ET * from cStringIO or StringIO import StringIO * import readline or emptymodule From jbarham at gmail.com Thu Jan 3 02:59:19 2008 From: jbarham at gmail.com (John Barham) Date: Wed, 2 Jan 2008 17:59:19 -0800 Subject: [Python-Dev] Syntax suggestion for imports In-Reply-To: <20080102201927.AEJ96366@ms19.lnh.mail.rcn.net> References: <20080102201927.AEJ96366@ms19.lnh.mail.rcn.net> Message-ID: <4f34febc0801021759m47a9ce03nde70da0dba762873@mail.gmail.com> Raymond Hettinger wrote: > How about a new, simpler syntax: > > * import threading or dummy_threading as threading > * import xml.etree.CElementTree or cElementTree or elementree.ElementTree as ET > * from cStringIO or StringIO import StringIO These all look good to me. The "short circuit" import syntax and semantics are compact and intuitive. > * import readline or emptymodule This I find more problematic as "emptymodule" seems too magical. Even now any code that wants to use a module that might not have been successfully imported needs to check if that's the case. E.g., a fuller current use-case would be: try: readline = None import readline except ImportError: pass if readline is not None: readline.foo() ... Conceivably emptymodule could act as a Null object but that could create more problems than it solves. John From guido at python.org Thu Jan 3 03:05:13 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 2 Jan 2008 18:05:13 -0800 Subject: [Python-Dev] Syntax suggestion for imports In-Reply-To: <20080102201927.AEJ96366@ms19.lnh.mail.rcn.net> References: <20080102201927.AEJ96366@ms19.lnh.mail.rcn.net> Message-ID: I wonder if your perceived need for this isn't skewed by your working within the core? Also, in 3.0 many of the use cases should go away -- e.g. cStringIO vs, StringIO, etc., as we switch the stdlib to having a single "public" name for an API which automatically replaces or augments itself with the accelerated C version if available. --Guido On Jan 2, 2008 5:19 PM, Raymond Hettinger wrote: > The standard library, my personal code, third-party packages, and my employer's code base are filled with examples of the following pattern: > > try: > import threading > except ImportError: > import dummy_threading as threading > > try: > import xml.etree.cElementTree as ET > except ImportError: > try: > import cElementTree as ET > except ImportError: > import elementtree.ElementTree as ET > > try: > from cStringIO import StringIO > except ImportError: > from StringIO import StringIO > > try: > import readline > except ImportError: > pass > > > How about a new, simpler syntax: > > * import threading or dummy_threading as threading > > * import xml.etree.CElementTree or cElementTree or elementree.ElementTree as ET > > * from cStringIO or StringIO import StringIO > > * import readline or emptymodule > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz at pythoncraft.com Thu Jan 3 03:08:25 2008 From: aahz at pythoncraft.com (Aahz) Date: Wed, 2 Jan 2008 18:08:25 -0800 Subject: [Python-Dev] Syntax suggestion for imports In-Reply-To: <20080102201927.AEJ96366@ms19.lnh.mail.rcn.net> References: <20080102201927.AEJ96366@ms19.lnh.mail.rcn.net> Message-ID: <20080103020825.GA21494@panix.com> On Wed, Jan 02, 2008, Raymond Hettinger wrote: > > The standard library, my personal code, third-party packages, and > my employer's code base are filled with examples of the following > pattern: > > try: > import threading > except ImportError: > import dummy_threading as threading > > How about a new, simpler syntax: > > * import threading or dummy_threading as threading My gut reaction is -0. For starters, many of these should go away with Python 3.0 (e.g. cStringIO). Also, annoying as the try/except is, I think the fact that it signals the special import is helpful; your suggestion is too light-weight IMO. If you could devise something just a bit heavier, that would be much better. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From terry at jon.es Thu Jan 3 03:23:17 2008 From: terry at jon.es (Terry Jones) Date: Thu, 3 Jan 2008 03:23:17 +0100 Subject: [Python-Dev] Syntax suggestion for imports In-Reply-To: Your message at 17:59:19 on Wednesday, 2 January 2008 References: <20080102201927.AEJ96366@ms19.lnh.mail.rcn.net> <4f34febc0801021759m47a9ce03nde70da0dba762873@mail.gmail.com> Message-ID: <18300.18197.558942.248252@terry.local> >>>>> "John" == John Barham writes: >> * import readline or emptymodule John> This I find more problematic as "emptymodule" seems too magical. John> Even now any code that wants to use a module that might not have been John> successfully imported needs to check if that's the case. E.g., a John> fuller current use-case would be: John> try: John> readline = None John> import readline John> except ImportError: John> pass John> if readline is not None: John> readline.foo() John> Conceivably emptymodule could act as a Null object but that could John> create more problems than it solves. How about import readline or None as readline This is just for cases where you don't want to trigger ImportException on import and do want the symbol set to None for later testing. A standalone "import None" could have no effect. Terry From barry at python.org Thu Jan 3 04:18:14 2008 From: barry at python.org (Barry Warsaw) Date: Wed, 2 Jan 2008 22:18:14 -0500 Subject: [Python-Dev] Syntax suggestion for imports In-Reply-To: <20080103020825.GA21494@panix.com> References: <20080102201927.AEJ96366@ms19.lnh.mail.rcn.net> <20080103020825.GA21494@panix.com> Message-ID: <84B8D885-C7CA-4DF3-997A-ED2B21D6B828@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 2, 2008, at 9:08 PM, Aahz wrote: > On Wed, Jan 02, 2008, Raymond Hettinger wrote: >> >> The standard library, my personal code, third-party packages, and >> my employer's code base are filled with examples of the following >> pattern: >> >> try: >> import threading >> except ImportError: >> import dummy_threading as threading >> >> How about a new, simpler syntax: >> >> * import threading or dummy_threading as threading > > My gut reaction is -0. For starters, many of these should go away > with > Python 3.0 (e.g. cStringIO). Also, annoying as the try/except is, I > think the fact that it signals the special import is helpful; your > suggestion is too light-weight IMO. If you could devise something > just > a bit heavier, that would be much better. I tend to agree. The little bit of syntactic sugar doesn't really seem worth it. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBR3xT9nEjvBPtnXfVAQIX/QQAmg9RczTAxHUe3qsry3F9DcUQZX32C9HS VMETbSnoVS0Xrdm5J7cCYqjpHlPXLKRoFCuYGWJ03ivjws/DzAsTXhpFwSmISZId 43W2UCC6mX8izr3E+bC4uEagw7EiVFsDEIX52FoUx6vIig0piZq3XolpQUqk4tP2 EhxQzTZwx6c= =U9qz -----END PGP SIGNATURE----- From python at rcn.com Thu Jan 3 04:29:28 2008 From: python at rcn.com (Raymond Hettinger) Date: Wed, 2 Jan 2008 22:29:28 -0500 (EST) Subject: [Python-Dev] Syntax suggestion for imports Message-ID: <20080102222928.AEK07274@ms19.lnh.mail.rcn.net> [GvR] > I wonder if your perceived need for this isn't skewed by your > working within the core? The need was perceived by a colleague who does not work on the core. My own skew was in the opposite direction -- I've seen the pattern so often that I'm oblivious to it. Before posting, I ran some scans of our code base at work and found plenty of examples (mostly third-party cmodules vs python equivalents and a few that searched for similar functionality in different packages). It might be helpful if others were to also search their own code bases and post their findings: find . -name "*py" | xargs grep -C2 ImportError *py Also, Google's codesearch gives some examples (and a lot of cases that really do need the try/except form): http://www.google.com/codesearch?q=+lang:python+%22except+ImportError%22 I was surprised to see many examples in the form of: try: import xyz except ImportError: xyz = None I was also surprised to find plenty of code that is likely to be buggy because the two alternative loaded different names: try: from Products.OpenPT.OpenPTFile import OpenPTFile as ptFile except ImportError: from Products.PageTemplates.PageTemplateFile import PageTemplateFile I was not surprised to see searches for similar functionality across different packages like kjbuckets vs kjbuckets0 , Zope vs Zope2, or HTMLParser vs SGMLParser, or attempts to load any of several packages compliant with the DBAPI. Surely, Py3.0's automatic vectoring to C equivalent modules will help with the cases like cStringIO, cPickle. I don't think it will help with the general case of searching for a best available package (like gdbm vs dbm vs dumbdbm or threading vs dummythreading) or a best available implementation of a single function (like twisted.protocols._c_urlarg.unquote vs urllib.unquote or one of the various implementations of date utilities or encryption functions). Am curious to see what everyone else finds in their own code searches. [John Barham] > This I find more problematic as "emptymodule" seems too magical. > . . . > try: > readline = None > import readline > except ImportError: > pass Perhaps "import readline or None" would have been a better way to capture that pattern as well as the "except pass" pattern. Raymond From asmodai at in-nomine.org Thu Jan 3 07:38:52 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Thu, 3 Jan 2008 07:38:52 +0100 Subject: [Python-Dev] Syntax suggestion for imports In-Reply-To: <20080102222928.AEK07274@ms19.lnh.mail.rcn.net> References: <20080102222928.AEK07274@ms19.lnh.mail.rcn.net> Message-ID: <20080103063852.GJ67953@nexus.in-nomine.org> -On [20080103 04:29], Raymond Hettinger (python at rcn.com) wrote: >Am curious to see what everyone else finds in their own code searches. On the Trac project using your grep gives me 203 lines, if we take ~2 lines for and after in consideration, it still means 203/5 ~= 40 occurences. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ In every stone sleeps a crystal... From murman at gmail.com Thu Jan 3 07:56:51 2008 From: murman at gmail.com (Michael Urman) Date: Thu, 3 Jan 2008 00:56:51 -0600 Subject: [Python-Dev] Syntax suggestion for imports In-Reply-To: <20080102201927.AEJ96366@ms19.lnh.mail.rcn.net> References: <20080102201927.AEJ96366@ms19.lnh.mail.rcn.net> Message-ID: On Jan 2, 2008 7:19 PM, Raymond Hettinger wrote: > How about a new, simpler syntax: > > * import threading or dummy_threading as threading > > * import xml.etree.CElementTree or cElementTree or elementree.ElementTree as ET > > * from cStringIO or StringIO import StringIO > > * import readline or emptymodule I'm sympathetic to this syntax proposal, as I find the repetition and extra lines for a simple idea to be a little unpleasant. This would also allow us to decide whether the import 'or' handling would be triggered equivalently to the except ImportError case you described, or only for missing imports. The latter would stop errors in existing modules from being silenced (and may give reason to allow emptymodule or None), but I'm unsure if that's a good thing. Of the above I find the ElementTree migration to be the most compelling, yet it seems ill-timed to bring syntax into Python 2.x you'd be unable to use until you no longer needed it. However your later example of the PageTemplateFile, which appears to be due to a third-party module reorganization and could certainly happen during the lifetime of late 2.x, helps swing me back the other way. -- Michael Urman From python at rcn.com Thu Jan 3 08:25:23 2008 From: python at rcn.com (Raymond Hettinger) Date: Wed, 2 Jan 2008 23:25:23 -0800 Subject: [Python-Dev] Syntax suggestion for imports References: <20080102222928.AEK07274@ms19.lnh.mail.rcn.net> <20080103063852.GJ67953@nexus.in-nomine.org> Message-ID: <004901c84dd9$cececb00$6800a8c0@RaymondLaptop1> [Jeroen Ruigrok van der Werven] > On the Trac project using your grep gives me 203 lines, if we take ~2 lines > for and after in consideration, it still means 203/5 ~= 40 occurences. Thanks. I'm more curious about the content of those lines. Does the proposed syntax help, does the need go away in Py3.0, what is the typical pattern? Raymond From jyasskin at gmail.com Thu Jan 3 08:52:46 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Wed, 2 Jan 2008 23:52:46 -0800 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 Message-ID: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> I've been backporting pep 3141 to the trunk, and ran into the issue that round, math.floor, and math.ceil, which it specifies to return Integrals, currently return floats. Guido suggested privately that, to make sure that 2.6 is compatible with 2.5, they should keep returning floats for float arguments. Probably this implies that they should also keep returning float for int and long arguments. For other types, we're probably free to do whatever. Consistency across all Real implementations suggests that those 3 functions should always return their argument type. Consistency and compatibility with 3.0 suggest that they should return long for every new type we add them to. What does the list think? Thanks, Jeffrey Yasskin From python at rcn.com Thu Jan 3 09:33:22 2008 From: python at rcn.com (Raymond Hettinger) Date: Thu, 3 Jan 2008 00:33:22 -0800 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> Message-ID: <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> > Consistency and compatibility with > 3.0 suggest that they should return long for every new type we add > them to. What does the list think? I think Py2.6 and Py2.5 should be treated with more respect. Will backporting this change can only cause relief or create headaches?. By definition, the Py3.0 release was supposed to be the one big incompatible set of changes. Backporting with a goal of "consistency and compatibility with 3.0" suggests that the backporters may have lost their compass. FWIW, Py2.6 hasn't been released yet and no one *has* to upgrade to it. So, if it is to have any chance of success, it needs to be a better Python than 2.5. IMO, jamming 3.0 junk into 2.x just provides an incentive to skip the upgrade altogether. In the press release for 2.6, we need to be able to say something stronger than: filled with deprecations, two ways to do everything, dozens of tiny incompatibilities all done in the name of 3.0, and runs slower. I think there ought to be a much more agressive standard for 3.0 backports:, "does the proposed backport make 2.6 more attractive?" Remember, for large code bases, upgrading is a PITA (I think Google is still running tons of code on 2.2, 2.3, and 2.4). There needs to be a good incentive for upgrading; otherwise, Py2.6 *will* fail. The specific change suggested here is possibly (and perhaps probably) harmless; however, it is part of a larger trend that is not healthy for the 2.x series. Raymond From asmodai at in-nomine.org Thu Jan 3 10:08:42 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Thu, 3 Jan 2008 10:08:42 +0100 Subject: [Python-Dev] Syntax suggestion for imports In-Reply-To: <004901c84dd9$cececb00$6800a8c0@RaymondLaptop1> References: <20080102222928.AEK07274@ms19.lnh.mail.rcn.net> <20080103063852.GJ67953@nexus.in-nomine.org> <004901c84dd9$cececb00$6800a8c0@RaymondLaptop1> Message-ID: <20080103090842.GK67953@nexus.in-nomine.org> -On [20080103 08:53], Raymond Hettinger (python at rcn.com) wrote: >Thanks. I'm more curious about the content of those lines. Does the >proposed syntax help, does the need go away in Py3.0, what is the typical >pattern? These are some of the examples, I've tried to reduce them to specific use patterns: To determine if some feature is available: try: pygments = __import__('pygments', {}, {}, []) have_pygments = True except ImportError: have_pygments = False try: from docutils import nodes from docutils.core import publish_parts from docutils.parsers import rst from docutils import __version__ except ImportError: raise TracError(_('Docutils not found')) if StrictVersion(__version__) < StrictVersion('0.3.9'): raise TracError(_('Docutils version >= %(version)s required, ' '%(found)s found', version='0.3.9', found=__version__)) To specify which specific version we have of a given module: try: import pysqlite2.dbapi2 as sqlite have_pysqlite = 2 except ImportError: try: import sqlite3 as sqlite have_pysqlite = 2 except ImportError: try: import sqlite have_pysqlite = 1 except ImportError: have_pysqlite = 0 This gets repeated a lot in order to see if we have threading: try: import threading except ImportError: import dummy_threading as threading threading._get_ident = lambda: 0 Functions we do use and need, fall-back to support code if not present: try: from operator import attrgetter, itemgetter except ImportError: def attrgetter(name): def _getattr(obj): return getattr(obj, name) return _getattr def itemgetter(name): def _getitem(obj): return obj[name] return _getitem Masking functions if a particular function is not found: try: from base64 import b64decode except ImportError: from base64 import decodestring as b64decode -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ Only I can change my life. No one can do it for me... From facundobatista at gmail.com Thu Jan 3 13:30:30 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Thu, 3 Jan 2008 10:30:30 -0200 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> Message-ID: 2008/1/3, Raymond Hettinger : > I think Py2.6 and Py2.5 should be treated with more respect. Will backporting this change can only cause relief or create > headaches?. By definition, the Py3.0 release was supposed to be the one big incompatible set of changes. Backporting with a goal Well, as issue 1689 states, the backporting was commited by Jeffrey on rev 5967 [2], so this is the time to understand if we want this or not. Personally, I'm -0. I was involved in this because of Decimal, but I can grow some __methods__ in it that can be in the trunk, unused, and when ported to 3k fit ok in the new infrastructure. Regards, [1] http://bugs.python.org/issue1689 [2] http://svn.python.org/view?rev=59671&view=rev -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From lists at cheimes.de Thu Jan 3 15:20:53 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 03 Jan 2008 15:20:53 +0100 Subject: [Python-Dev] Syntax suggestion for imports In-Reply-To: <20080102201927.AEJ96366@ms19.lnh.mail.rcn.net> References: <20080102201927.AEJ96366@ms19.lnh.mail.rcn.net> Message-ID: <477CEF45.5030500@cheimes.de> Raymond Hettinger wrote: > How about a new, simpler syntax: > > * import threading or dummy_threading as threading > > * import xml.etree.CElementTree or cElementTree or elementree.ElementTree as ET > > * from cStringIO or StringIO import StringIO > > * import readline or emptymodule The syntax idea has a nice ring to it, except for the last idea. As others have already said, the name emptymodule is too magic. The readline example becomes more readable when you change the import to import readline or None as readline In my opinion the import or as syntax definition is easy to understand if you force the user to always have an "as" statement. The None name is optional but must be the last name: import name[, or name2[, or name3 ...] [, or None] as target This translates into: try: import name as target except ImportError: try: import name2 as target except ImportError: target = None from name[, or name2 ...] [, or None] import target translates into try: from name import target except ImportError: try: from name2 import target except ImportError: target = None Christian From percivall at gmail.com Thu Jan 3 15:28:54 2008 From: percivall at gmail.com (Simon Percivall) Date: Thu, 3 Jan 2008 15:28:54 +0100 Subject: [Python-Dev] Syntax suggestion for imports In-Reply-To: <20080102201927.AEJ96366@ms19.lnh.mail.rcn.net> References: <20080102201927.AEJ96366@ms19.lnh.mail.rcn.net> Message-ID: <51D8CC0C-1F9B-4409-B67A-C49671742C3D@gmail.com> On 3 jan 2008, at 02.19, Raymond Hettinger wrote: > How about a new, simpler syntax: > > * import threading or dummy_threading as threading > > * import xml.etree.CElementTree or cElementTree or > elementree.ElementTree as ET > > * from cStringIO or StringIO import StringIO > > * import readline or emptymodule Wouldn't a (stdlib) function suffice in the cases where this is needed? ET = import_with_alternative("xml.etree.CElementTree", "cElementTree", "elementtree.ElementTree") It's not as elegant, but it's easier than status quo. //Simon From rrr at ronadam.com Thu Jan 3 18:48:28 2008 From: rrr at ronadam.com (Ron Adam) Date: Thu, 03 Jan 2008 11:48:28 -0600 Subject: [Python-Dev] Syntax suggestion for imports In-Reply-To: <20080102201927.AEJ96366@ms19.lnh.mail.rcn.net> References: <20080102201927.AEJ96366@ms19.lnh.mail.rcn.net> Message-ID: <477D1FEC.1000202@ronadam.com> Raymond Hettinger wrote: > The standard library, my personal code, third-party packages, and my employer's code base are filled with examples of the following pattern: > > try: > import threading > except ImportError: > import dummy_threading as threading > > try: > import xml.etree.cElementTree as ET > except ImportError: > try: > import cElementTree as ET > except ImportError: > import elementtree.ElementTree as ET > > try: > from cStringIO import StringIO > except ImportError: > from StringIO import StringIO > > try: > import readline > except ImportError: > pass > > > How about a new, simpler syntax: > > * import threading or dummy_threading as threading > > * import xml.etree.CElementTree or cElementTree or elementree.ElementTree as ET > > * from cStringIO or StringIO import StringIO > > * import readline or emptymodule I don't think it's as visually clear, but that may be I'm just not used to it. An alternative possibility might be, rather than "or", reuse "else" before import. import threading else import dummy_threading as threading import xml.etree.CElementTree as ET else import cElementTree as ET else import elementtree.ElementTree as ET The readline example above should be in a try except as it allows a failure to pass. For example if you wanted to allow the above elementtree example to pass instead of raising an exception you would write.. try: import xml.etree.CElementTree as ET else import cElementTree as ET else import elementtree.ElementTree as ET except ImportError: pass # or handle failed import. This still improves readability and flattens out the multiple nested structure which I believe is what makes the current way unappealing. I think multiple possible imports in "from - import"s statements should not be allowed. When you consider multiple imports from possibly multiple sources, it seems like that could get a bit messy when debugging. Regards, Ron From aahz at pythoncraft.com Thu Jan 3 19:11:58 2008 From: aahz at pythoncraft.com (Aahz) Date: Thu, 3 Jan 2008 10:11:58 -0800 Subject: [Python-Dev] Syntax suggestion for imports In-Reply-To: <20080102222928.AEK07274@ms19.lnh.mail.rcn.net> References: <20080102222928.AEK07274@ms19.lnh.mail.rcn.net> Message-ID: <20080103181157.GA12666@panix.com> On Wed, Jan 02, 2008, Raymond Hettinger wrote: > > Before posting, I ran some scans of our code base at work and found > plenty of examples (mostly third-party cmodules vs python equivalents > and a few that searched for similar functionality in different > packages). It might be helpful if others were to also search their > own code bases and post their findings: > > find . -name "*py" | xargs grep -C2 ImportError *py Most of my company's examples fall into cases like this: try: klass = load_class(foo) except ImportError: klass = NullClass -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From janssen at parc.com Thu Jan 3 19:18:22 2008 From: janssen at parc.com (Bill Janssen) Date: Thu, 3 Jan 2008 10:18:22 PST Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> Message-ID: <08Jan3.101829pst."58696"@synergy1.parc.xerox.com> > I think there ought to be a much more agressive standard for 3.0 backports:, "does the proposed backport make 2.6 more attractive?" > Remember, for large code bases, upgrading is a PITA (I think Google is still running tons of code on 2.2, 2.3, and 2.4). There > needs to be a good incentive for upgrading; otherwise, Py2.6 *will* fail. Very well put, Raymond. I wonder, though, whether the appearance of a viable 3.0 will essentially kill the demand for 2.6? Bill From guido at python.org Thu Jan 3 19:37:27 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 3 Jan 2008 10:37:27 -0800 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> Message-ID: On Jan 3, 2008 12:33 AM, Raymond Hettinger wrote: > > Consistency and compatibility with > > 3.0 suggest that they should return long for every new type we add > > them to. What does the list think? > > I think Py2.6 and Py2.5 should be treated with more respect. Will backporting this change can only cause relief or create > headaches?. By definition, the Py3.0 release was supposed to be the one big incompatible set of changes. Backporting with a goal > of "consistency and compatibility with 3.0" suggests that the backporters may have lost their compass. Whoa. Jeffrey is mostly backporting new stuff (such as numbers.py) which doesn't introduce incompatibilities. The backporting is intended to make 2.6 more attractive by allowing developers to do some of the painful work of porting to 3.0 while staying on 2.6. > FWIW, Py2.6 hasn't been released yet and no one *has* to upgrade to it. So, if it is to have any chance of success, it needs to be > a better Python than 2.5. IMO, jamming 3.0 junk into 2.x just provides an incentive to skip the upgrade altogether. In the press > release for 2.6, we need to be able to say something stronger than: filled with deprecations, two ways to do everything, dozens of > tiny incompatibilities all done in the name of 3.0, and runs slower. Watch your words, Raymond. I don't mind personally, but you run the risk of discouraging contributors by slamming down on them with with words like "jamming 3.0 junk into 2.x", and that's the last thing we want to happen. We're thin on contributors as it is (have you noticed how few people are submitting anything at all lately?). 2.6 should be extremely compatible with 2.5 by default. Its main attraction should be that it is an important stepping stone in the upgrade path from 2.{2,3,4,5} to 3.0 -- going straight from 2.5 to 3.0 will be much harder than going from 2.5 to 2.6, doing a bunch of work while on 2.6, and then going to 3.0. The initial step from 2.5 to 2.6 should be very simple and painless. There will still be plenty of good stuff in 2.6 to encourage folks to upgrade who have no need for 3.0 (yet) -- this is the backported 3.0 stuff that doesn't create incompatibilities (like abc.py and numbers.py). > I think there ought to be a much more agressive standard for 3.0 backports:, "does the proposed backport make 2.6 more attractive?" > Remember, for large code bases, upgrading is a PITA (I think Google is still running tons of code on 2.2, 2.3, and 2.4). There > needs to be a good incentive for upgrading; otherwise, Py2.6 *will* fail. The incentive for upgrading will be "you can reach 3.0 easier via 2.6" and perhaps "the latest version of 3rd party software X runs best on 2.6". > The specific change suggested here is possibly (and perhaps probably) harmless; however, it is part of a larger trend that is not > healthy for the 2.x series. Where do you see evidence for a larger trend? I agree that a trend towards making 2.6 less compatible with 2.5 would be a mistake, but I don't think I see it happening. Also, Facundo wrote: > Well, as issue 1689 states, the backporting was commited by Jeffrey on > rev 5967 [2], so this is the time to understand if we want this or > not. This is a problem. Right now, in the trunk, math.float(1) returns 1, where it should return 1.0 for compatibility with 2.5. Jeffrey, can you fix this and similar incompatibilities you introduced? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From asmodai at in-nomine.org Thu Jan 3 20:05:56 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Thu, 3 Jan 2008 20:05:56 +0100 Subject: [Python-Dev] Contributing to Python In-Reply-To: References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> Message-ID: <20080103190556.GO67953@nexus.in-nomine.org> Guido, -On [20080103 19:38], Guido van Rossum (guido at python.org) wrote: >We're thin on contributors as it is (have you noticed how few people are >submitting anything at all lately?). When you say this are you talking about code or contributions all over the project, e.g. documentation? Is there a reason that it thinned out? I'm asking since I never really dove into the Python project much aside from using Python here and there, but I seem to get a career using Python almost full-time so perhaps there are things I, and I am sure others, can step up to help out. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ A frightened mental vortex we will be, a Sun we seek, a Sun we flee... From guido at python.org Thu Jan 3 20:15:04 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 3 Jan 2008 11:15:04 -0800 Subject: [Python-Dev] Contributing to Python In-Reply-To: <20080103190556.GO67953@nexus.in-nomine.org> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> Message-ID: We're getting a fair number of doc contributions, especially since the docs were converted from Latex to ReST, and especially since the start of the GHOP project. My main gripe is with code contributions to Py3k and 2.6; Py3k is mostly done by a handful of people, and almost nobody is working much on 2.6. --Guido On Jan 3, 2008 11:05 AM, Jeroen Ruigrok van der Werven wrote: > Guido, > > -On [20080103 19:38], Guido van Rossum (guido at python.org) wrote: > >We're thin on contributors as it is (have you noticed how few people are > >submitting anything at all lately?). > > When you say this are you talking about code or contributions all over the > project, e.g. documentation? > > Is there a reason that it thinned out? > > I'm asking since I never really dove into the Python project much aside from > using Python here and there, but I seem to get a career using Python almost > full-time so perhaps there are things I, and I am sure others, can step up to > help out. > > -- > Jeroen Ruigrok van der Werven / asmodai > イェルーン ラウフロック ヴァン デル ウェルヴェン > http://www.in-nomine.org/ | http://www.rangaku.org/ > A frightened mental vortex we will be, a Sun we seek, a Sun we flee... > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From facundobatista at gmail.com Thu Jan 3 20:19:08 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Thu, 3 Jan 2008 17:19:08 -0200 Subject: [Python-Dev] Syntax suggestion for imports In-Reply-To: <20080102201927.AEJ96366@ms19.lnh.mail.rcn.net> References: <20080102201927.AEJ96366@ms19.lnh.mail.rcn.net> Message-ID: 2008/1/2, Raymond Hettinger : > How about a new, simpler syntax: > > * import threading or dummy_threading as threading > > * import xml.etree.CElementTree or cElementTree or elementree.ElementTree as ET > > * from cStringIO or StringIO import StringIO > > * import readline or emptymodule With the minor modification of changing emptymodule by a None in the last line, I'm +0 on it. I don't use this that much, but I find this syntax clear and understable. It does not bring additional complexity when not used, and is very helpful when needed. Don't know how much effort for the parser it is, though. Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From tonynelson at georgeanelson.com Thu Jan 3 20:20:00 2008 From: tonynelson at georgeanelson.com (Tony Nelson) Date: Thu, 3 Jan 2008 14:20:00 -0500 Subject: [Python-Dev] Syntax suggestion for imports In-Reply-To: <477CEF45.5030500@cheimes.de> References: <20080102201927.AEJ96366@ms19.lnh.mail.rcn.net> <477CEF45.5030500@cheimes.de> Message-ID: At 3:20 PM +0100 1/3/08, Christian Heimes wrote: >Raymond Hettinger wrote: >> How about a new, simpler syntax: ... >> * import readline or emptymodule > >The syntax idea has a nice ring to it, except for the last idea. As >others have already said, the name emptymodule is too magic. > >The readline example becomes more readable when you change the import to > >import readline or None as readline > > >In my opinion the import or as syntax definition is easy to understand >if you force the user to always have an "as" statement. The None name is >optional but must be the last name: > >import name[, or name2[, or name3 ...] [, or None] as target ... At 11:48 AM -0600 1/3/08, Ron Adam wrote: ... >An alternative possibility might be, rather than "or", reuse "else" before >import. ... I prefer "else" to "or" but with the original single-statement syntax. If the last clause could be an expression as well as a module name, what I've done (used with and copied from BeautifulSoup): try: from htmlentitydefs import name2codepoint except ImportError: name2codepoint = {} could become: from htmlentitydefs else ({}) import name2codepoint as name2codepoint Also: import foo or (None) as foo -- ____________________________________________________________________ TonyN.:' ' From titus at caltech.edu Thu Jan 3 20:40:24 2008 From: titus at caltech.edu (Titus Brown) Date: Thu, 3 Jan 2008 11:40:24 -0800 Subject: [Python-Dev] Contributing to Python In-Reply-To: References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> Message-ID: <20080103194024.GA15580@caltech.edu> On Thu, Jan 03, 2008 at 11:15:04AM -0800, Guido van Rossum wrote: -> We're getting a fair number of doc contributions, especially since the -> docs were converted from Latex to ReST, and especially since the start -> of the GHOP project. -> -> My main gripe is with code contributions to Py3k and 2.6; Py3k is -> mostly done by a handful of people, and almost nobody is working much -> on 2.6. What needs to be done with 2.6? I'm happy to review patches, although even were commit access on offer I'm too scatterbrained to do a good job of it. Incidentally, I'm planning to set up an SVK repos containing the GHOP doc patches; that way they can stay sync'ed with 2.6 work. I'd be happy to do the same thing with reviewed-and-probably-OK patches, although I don't know if repository proliferation is a generally good idea ;). --titus From facundobatista at gmail.com Thu Jan 3 20:48:25 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Thu, 3 Jan 2008 17:48:25 -0200 Subject: [Python-Dev] Contributing to Python In-Reply-To: <20080103194024.GA15580@caltech.edu> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> <20080103194024.GA15580@caltech.edu> Message-ID: 2008/1/3, Titus Brown : > What needs to be done with 2.6? I'm happy to review patches, although > even were commit access on offer I'm too scatterbrained to do a good job > of it. We have 109 patches open for 2.5 [1], and 118 patches open for 2.6 [2]. Note that the added number is less than the sum, as some patchs are marked for both versions, and even some are marked as 3.0 also. And of course I'm not counting bugs, here, just issues marked as patch. I deliberately work only on the trunk (except for some issues regarding Decimal), because of the balance Guido talked about. Regards, [1] http://www.taniquetil.com.ar/cgi-bin/pytickets.py?nropag=0&priority=0&severity=0&component=0&version=2&keyword=2 [2] http://www.taniquetil.com.ar/cgi-bin/pytickets.py?nropag=0&priority=0&severity=0&component=0&version=1&keyword=2 -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From fdrake at acm.org Thu Jan 3 20:49:27 2008 From: fdrake at acm.org (Fred Drake) Date: Thu, 3 Jan 2008 14:49:27 -0500 Subject: [Python-Dev] Contributing to Python In-Reply-To: References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> Message-ID: <2CAFFA1B-601E-4C27-90C0-6032438EF8F8@acm.org> On Jan 3, 2008, at 2:15 PM, Guido van Rossum wrote: > My main gripe is with code contributions to Py3k and 2.6; Py3k is > mostly done by a handful of people, and almost nobody is working much > on 2.6. For those of us still using Python 2.4 and earlier, it's hard to be motivated to worry about Python 3.0, no matter how wonderful it looks. (It doesn't help that my own available time appears to decrease daily with the kids and all.) Python 2.6 seems to be entirely targeted at people who really want to be on Python 3, but have code that will need to be ported. I certainly don't view it as interesting in it's own right. -Fred -- Fred Drake From titus at caltech.edu Thu Jan 3 20:51:44 2008 From: titus at caltech.edu (Titus Brown) Date: Thu, 3 Jan 2008 11:51:44 -0800 Subject: [Python-Dev] Contributing to Python In-Reply-To: References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> <20080103194024.GA15580@caltech.edu> Message-ID: <20080103195144.GA13654@caltech.edu> On Thu, Jan 03, 2008 at 05:48:25PM -0200, Facundo Batista wrote: -> 2008/1/3, Titus Brown : -> -> > What needs to be done with 2.6? I'm happy to review patches, although -> > even were commit access on offer I'm too scatterbrained to do a good job -> > of it. -> -> We have 109 patches open for 2.5 [1], and 118 patches open for 2.6 [2]. The question is, is reviewing patches a good place to contribute? Also, if I (and others) could have a "core mentor" with commit access, that might streamline things. As it is, I am worried that patch reviews will pass through the ether without leaving a visible trace; that's OK and understandable, but it's demotivating. (I've had a very good experience with Georg and documentation commits for GHOP.) cheers, --titus From titus at caltech.edu Thu Jan 3 20:53:04 2008 From: titus at caltech.edu (Titus Brown) Date: Thu, 3 Jan 2008 11:53:04 -0800 Subject: [Python-Dev] Contributing to Python In-Reply-To: <2CAFFA1B-601E-4C27-90C0-6032438EF8F8@acm.org> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> <2CAFFA1B-601E-4C27-90C0-6032438EF8F8@acm.org> Message-ID: <20080103195303.GB13654@caltech.edu> On Thu, Jan 03, 2008 at 02:49:27PM -0500, Fred Drake wrote: -> On Jan 3, 2008, at 2:15 PM, Guido van Rossum wrote: -> > My main gripe is with code contributions to Py3k and 2.6; Py3k is -> > mostly done by a handful of people, and almost nobody is working much -> > on 2.6. -> -> For those of us still using Python 2.4 and earlier, it's hard to be -> motivated to worry about Python 3.0, no matter how wonderful it -> looks. (It doesn't help that my own available time appears to -> decrease daily with the kids and all.) -> -> Python 2.6 seems to be entirely targeted at people who really want to -> be on Python 3, but have code that will need to be ported. I -> certainly don't view it as interesting in it's own right. 3k and 26 are, however, the only place where we can propose new features -- which makes it the place for cleanup and additional testing, as well as backwards-incompatible bug fixes... --titus From asmodai at in-nomine.org Thu Jan 3 21:17:31 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Thu, 3 Jan 2008 21:17:31 +0100 Subject: [Python-Dev] Contributing to Python In-Reply-To: References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> Message-ID: <20080103201731.GA82115@nexus.in-nomine.org> -On [20080103 20:39], Guido van Rossum (guido at python.org) wrote: >My main gripe is with code contributions to Py3k and 2.6; Py3k is >mostly done by a handful of people, and almost nobody is working much >on 2.6. You don't put the bar high for newbies on the Python project eh? :) I am assumign that most of those contributions code-wise need a fair amount of knowledge of Python's internals? -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ Mayoi nagara demo ii aruki dashite... From janssen at parc.com Thu Jan 3 21:17:52 2008 From: janssen at parc.com (Bill Janssen) Date: Thu, 3 Jan 2008 12:17:52 PST Subject: [Python-Dev] Contributing to Python In-Reply-To: References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> Message-ID: <08Jan3.121800pst."58696"@synergy1.parc.xerox.com> > My main gripe is with code contributions to Py3k and 2.6; Py3k is > mostly done by a handful of people, and almost nobody is working much > on 2.6. There's a great Duke Ellington quote: ``Without a deadline, baby, I wouldn't do nothing.'' The SSL code in 2.6 is out-of-date (buggy) compared to the code in the 3.x branch, for instance. I just haven't prioritized backporting the 3.x fixes, because there's no schedule for 2.6. Bill From josepharmbruster at gmail.com Thu Jan 3 21:24:16 2008 From: josepharmbruster at gmail.com (Joseph Armbruster) Date: Thu, 3 Jan 2008 15:24:16 -0500 Subject: [Python-Dev] Contributing to Python In-Reply-To: <20080103195303.GB13654@caltech.edu> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> <2CAFFA1B-601E-4C27-90C0-6032438EF8F8@acm.org> <20080103195303.GB13654@caltech.edu> Message-ID: <938f42d70801031224w76527b25ya60f8988677c3e84@mail.gmail.com> Titus, Having a "core mentor" would be great but do they really have time for that? I've been lucky at finding people in #python / #python-dev) that can answer development inquiries (or at least verify something is or is not a bug). With respects to the bug tracker, when I select Search and Python 2.6, I retrieved 208 open bugs. At a quick glance, I found two that were windows, but not tagged appropriately. If it's worthwhile, I can spend some time this evening browsing the list of current 2.6 bugs to see if there are any duplicates, collisions, etc. Joseph Armbruster On Jan 3, 2008 2:53 PM, Titus Brown wrote: > On Thu, Jan 03, 2008 at 02:49:27PM -0500, Fred Drake wrote: > -> On Jan 3, 2008, at 2:15 PM, Guido van Rossum wrote: > -> > My main gripe is with code contributions to Py3k and 2.6; Py3k is > -> > mostly done by a handful of people, and almost nobody is working much > -> > on 2.6. > -> > -> For those of us still using Python 2.4 and earlier, it's hard to be > -> motivated to worry about Python 3.0, no matter how wonderful it > -> looks. (It doesn't help that my own available time appears to > -> decrease daily with the kids and all.) > -> > -> Python 2.6 seems to be entirely targeted at people who really want to > -> be on Python 3, but have code that will need to be ported. I > -> certainly don't view it as interesting in it's own right. > > 3k and 26 are, however, the only place where we can propose new features > -- which makes it the place for cleanup and additional testing, as well > as backwards-incompatible bug fixes... > > --titus > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/josepharmbruster%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080103/5486bc5d/attachment-0001.htm From lists at cheimes.de Thu Jan 3 21:55:44 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 03 Jan 2008 21:55:44 +0100 Subject: [Python-Dev] Contributing to Python In-Reply-To: <20080103201731.GA82115@nexus.in-nomine.org> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> <20080103201731.GA82115@nexus.in-nomine.org> Message-ID: <477D4BD0.1040204@cheimes.de> Jeroen Ruigrok van der Werven wrote: > You don't put the bar high for newbies on the Python project eh? :) > > I am assumign that most of those contributions code-wise need a fair amount of > knowledge of Python's internals? It's neither impossible nor too hard to get involved. I got from "haven't done serious C coding in years" to "Python core developer with full svn access" in less than 9 months. OK, I've more than 5 years of Python experience but you don't need it to contribute. You can start by updating or enhancing the existing documentation, writing new docs and tutorials or updating the unit test suite. New tests need to be written and existing test should be ported to the new unit test module. Large parts of Python are written *in* Python. You don't need to be a C coder to contribute. For example you can give a C coder a hand by writing documentation and unit tests for the C coder. This way the C coder can concentrate on the C code and you can enhance your unit testing Fu. :) Christian From guido at python.org Thu Jan 3 22:02:27 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 3 Jan 2008 13:02:27 -0800 Subject: [Python-Dev] Contributing to Python In-Reply-To: <20080103194024.GA15580@caltech.edu> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> <20080103194024.GA15580@caltech.edu> Message-ID: On Jan 3, 2008 11:40 AM, Titus Brown wrote: > On Thu, Jan 03, 2008 at 11:15:04AM -0800, Guido van Rossum wrote: > -> We're getting a fair number of doc contributions, especially since the > -> docs were converted from Latex to ReST, and especially since the start > -> of the GHOP project. > -> > -> My main gripe is with code contributions to Py3k and 2.6; Py3k is > -> mostly done by a handful of people, and almost nobody is working much > -> on 2.6. > > What needs to be done with 2.6? I'm happy to review patches, although > even were commit access on offer I'm too scatterbrained to do a good job > of it. IMO the main issue with 2.6 is not handling bugs and patches but backporting 3.0 stuff. There's a spreadsheet somewhere that shows there's a huge amount of work to be done: http://spreadsheets.google.com/ccc?key=pCKY4oaXnT81FrGo3ShGHGg (if you can't access it, try the published version: http://spreadsheets.google.com/pub?key=pCKY4oaXnT81FrGo3ShGHGg ) > Incidentally, I'm planning to set up an SVK repos containing the GHOP > doc patches; that way they can stay sync'ed with 2.6 work. I'd be happy > to do the same thing with reviewed-and-probably-OK patches, although I > don't know if repository proliferation is a generally good idea ;). IMO patches should just be applied to the trunk ASAP. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From amk at amk.ca Thu Jan 3 22:02:24 2008 From: amk at amk.ca (A.M. Kuchling) Date: Thu, 3 Jan 2008 16:02:24 -0500 Subject: [Python-Dev] Contributing to Python In-Reply-To: <2CAFFA1B-601E-4C27-90C0-6032438EF8F8@acm.org> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> <2CAFFA1B-601E-4C27-90C0-6032438EF8F8@acm.org> Message-ID: <20080103210224.GA17239@amk-desktop.matrixgroup.net> On Thu, Jan 03, 2008 at 02:49:27PM -0500, Fred Drake wrote: > Python 2.6 seems to be entirely targeted at people who really want to > be on Python 3, but have code that will need to be ported. I > certainly don't view it as interesting in its own right. The bulk of the *language* changes in 2.6 are driven by 3.0, but the abstract base class support is fairly significant even if you don't plan on going to 3.0. There are a fair number of new features in the library: Bill's new SSL code, collections.namedtuple, the signal handling/event loop fix, the new floating point features dealing with infinities and NaNs. None are earth-shattering to me personally, but for the right audience they might be very compelling. So far I view 2.6 as a relatively cautious release, like 2.3. (That assessment may change once I research the numeric changes that just went in.) Most of the action has been in the surrounding tools, like the new documentation format and the adoption of Roundup. --amk From lists at cheimes.de Thu Jan 3 22:06:16 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 03 Jan 2008 22:06:16 +0100 Subject: [Python-Dev] Contributing to Python In-Reply-To: <938f42d70801031224w76527b25ya60f8988677c3e84@mail.gmail.com> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> <2CAFFA1B-601E-4C27-90C0-6032438EF8F8@acm.org> <20080103195303.GB13654@caltech.edu> <938f42d70801031224w76527b25ya60f8988677c3e84@mail.gmail.com> Message-ID: <477D4E48.4030905@cheimes.de> Joseph Armbruster wrote: > With respects to the bug tracker, when I select Search and Python 2.6, I > retrieved 208 open bugs. At a quick glance, I found two that were windows, > but not tagged appropriately. If it's worthwhile, I can spend some time > this evening browsing the list of current 2.6 bugs to see if there are any > duplicates, collisions, etc. The bug tracker is an issue and in my opinion a development hindrance, too. It contains almost 1,400 bugs, about 450 w/o a target version and about 400 more bugs with a target version <= 2.4. The bug tracker was discussed two threads "Bug tracker: meaning of resolution keywords" and "1324 bugs in the tracker" lately. I don't want to repeat the arguments here but Brett's answer http://permalink.gmane.org/gmane.comp.python.devel/90137 is worth reading. Christian From guido at python.org Thu Jan 3 22:07:36 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 3 Jan 2008 13:07:36 -0800 Subject: [Python-Dev] Contributing to Python In-Reply-To: <2CAFFA1B-601E-4C27-90C0-6032438EF8F8@acm.org> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> <2CAFFA1B-601E-4C27-90C0-6032438EF8F8@acm.org> Message-ID: On Jan 3, 2008 11:49 AM, Fred Drake wrote: > For those of us still using Python 2.4 and earlier, it's hard to be > motivated to worry about Python 3.0, no matter how wonderful it > looks. (It doesn't help that my own available time appears to > decrease daily with the kids and all.) Oh, just get rid of the kids. :-) > Python 2.6 seems to be entirely targeted at people who really want to > be on Python 3, but have code that will need to be ported. I > certainly don't view it as interesting in it's own right. It will be though -- it will have genuine new features -- yes, backported from 3.0, but new features nevertheless, and in a compatible fashion. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Thu Jan 3 22:10:25 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 03 Jan 2008 22:10:25 +0100 Subject: [Python-Dev] Contributing to Python In-Reply-To: <20080103195144.GA13654@caltech.edu> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> <20080103194024.GA15580@caltech.edu> <20080103195144.GA13654@caltech.edu> Message-ID: <477D4F41.50902@v.loewis.de> > The question is, is reviewing patches a good place to contribute? Also, > if I (and others) could have a "core mentor" with commit access, that > might streamline things. As it is, I am worried that patch reviews will > pass through the ether without leaving a visible trace; that's OK and > understandable, but it's demotivating. Keep posting to this list, if you think a patch can be accepted. If you think a patch should be rejected, it would probably be easiest if you just close it. > (I've had a very good experience with Georg and documentation commits > for GHOP.) You can only find out what feedback you get on code reviews when you actually start reviewing :-) I would like to promise guaranteed responses to you, but I feel that recently, I found that I cannot keep up with such promises. But I would hope that some of the active committers will "quickly" commit patches when they find that your review leaves nothing to be added. Regards, Martin From guido at python.org Thu Jan 3 22:12:51 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 3 Jan 2008 13:12:51 -0800 Subject: [Python-Dev] Contributing to Python In-Reply-To: <20080103201731.GA82115@nexus.in-nomine.org> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> <20080103201731.GA82115@nexus.in-nomine.org> Message-ID: On Jan 3, 2008 12:17 PM, Jeroen Ruigrok van der Werven wrote: > -On [20080103 20:39], Guido van Rossum (guido at python.org) wrote: > >My main gripe is with code contributions to Py3k and 2.6; Py3k is > >mostly done by a handful of people, and almost nobody is working much > >on 2.6. > > You don't put the bar high for newbies on the Python project eh? :) > > I am assuming that most of those contributions code-wise need a fair amount of > knowledge of Python's internals? Actually, it goes all over the place. Some things (like doing "-3" warnings for uses of .keys() that assume the result is a list) required wizard level knowledge; other things are fairly simple. For example, abc.py and _abcoll.py were backported successfully by someone who was learning on the job. Backporting pure Python code often isn't that hard. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Thu Jan 3 22:13:04 2008 From: python at rcn.com (Raymond Hettinger) Date: Thu, 3 Jan 2008 16:13:04 -0500 (EST) Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 Message-ID: <20080103161304.AEL30212@ms19.lnh.mail.rcn.net> [GvR] > We're thin on contributors as it is (have you noticed > how few people are submitting anything at all lately?). The people who are contributing are doing a nice job. Also, it was nice that the change was discussed on the list. > 2.6 should be extremely compatible with 2.5 by default. Good to hear that is still a primary goal. Along the way, I worried that that sentiment had been lost and that little incompatibilities were sneaking in (iirc, the proposed transition plan for leading zeroes was headed down this path). > The incentive for upgrading will be "you can reach 3.0 > easier via 2.6" and perhaps "the latest version of 3rd > party software X runs best on 2.6". Does the 2to3 tool work from 2.5 or from 2.6 or does it make difference? If it works from 2.5, I'm thinking my company will make the jump all at once (after the 3.x series stabilizes, gets optimized, and key third-party packages have been migrated). I'm also expecting that some chuck of users will be left in the 2.x world and that they would like highest version to be as clean as possible (with migration features going into the category of things that don't provide them any benefit). > Right now, in the trunk, math.float(1) returns 1, > where it should return 1.0 for compatibility with 2.5. > Jeffrey, can you fix this and similar incompatibilities > you introduced? Thanks for zapping this. Raymond From guido at python.org Thu Jan 3 22:14:53 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 3 Jan 2008 13:14:53 -0800 Subject: [Python-Dev] Contributing to Python In-Reply-To: <1295843670530617266@unknownmsgid> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> <1295843670530617266@unknownmsgid> Message-ID: On Jan 3, 2008 12:17 PM, Bill Janssen wrote: > > My main gripe is with code contributions to Py3k and 2.6; Py3k is > > mostly done by a handful of people, and almost nobody is working much > > on 2.6. > > There's a great Duke Ellington quote: ``Without a deadline, baby, I > wouldn't do nothing.'' > > The SSL code in 2.6 is out-of-date (buggy) compared to the code in the > 3.x branch, for instance. I just haven't prioritized backporting the > 3.x fixes, because there's no schedule for 2.6. Eh? PEP 3000 has a schedule that includes 2.6: * August 2007: release 3.0a1. * December 2007: release 3.0a2. * Early 2007 (pre-PyCon): release 2.6a1. * May 2008 (post-PyCon): full feature freeze for 3.0 and 2.6. * July 2008: release 2.6 (final). * August 2008: release 3.0 (final). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From lists at cheimes.de Thu Jan 3 22:19:33 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 03 Jan 2008 22:19:33 +0100 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: <20080103161304.AEL30212@ms19.lnh.mail.rcn.net> References: <20080103161304.AEL30212@ms19.lnh.mail.rcn.net> Message-ID: Raymond Hettinger wrote: > Does the 2to3 tool work from 2.5 or from 2.6 or does it make difference? If it works from 2.5, I'm thinking my company will make the jump all at once (after the 3.x series stabilizes, gets optimized, and key third-party packages have been migrated). It's not guaranteed that it will work from 2.5. The transition plan for 2to3 is: * Get your code working under python2.6 -3 without Python 3.0 deprecation warnings * Use 2to3 to migrate the code from 2.6 to 3.0 Christian From guido at python.org Thu Jan 3 22:27:19 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 3 Jan 2008 13:27:19 -0800 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: <20080103161304.AEL30212@ms19.lnh.mail.rcn.net> References: <20080103161304.AEL30212@ms19.lnh.mail.rcn.net> Message-ID: On Jan 3, 2008 1:13 PM, Raymond Hettinger wrote: > [GvR] > > We're thin on contributors as it is (have you noticed > > how few people are submitting anything at all lately?). > > The people who are contributing are doing a nice job. Also, it was nice that the change was discussed on the list. We can always use more contributors! (See separate thread.) > > 2.6 should be extremely compatible with 2.5 by default. > > Good to hear that is still a primary goal. Along the way, I worried that that sentiment had been lost and that little incompatibilities were sneaking in (iirc, the proposed transition plan for leading zeroes was headed down this path). I don't recall vetting (or even seeing) that plan! Without the "-3" option, 2.6 should accept the same syntax (or a superset) as 2.5. With the "-3" flag, however, you may get warnings about stuff that changes in 3.0. I don't like that "-3" would change the accepted syntax; for that, we should use "from __future__ import ...". A hypothetical example for octal numbers: I propose that bare 2.6 accepts either 0777 and 0o777 as octal literals. If you use "python -3", you might get a warning about using 0777, although this is borderline useless since 2to3 will take care of it just fine. If people think it's worth it (I doubt it for this example) we could add "from __future__ import octal_literals" which would prohibit 0777 altogether. > > The incentive for upgrading will be "you can reach 3.0 > > easier via 2.6" and perhaps "the latest version of 3rd > > party software X runs best on 2.6". > > Does the 2to3 tool work from 2.5 or from 2.6 or does it make difference? If it works from 2.5, I'm thinking my company will make the jump all at once (after the 3.x series stabilizes, gets optimized, and key third-party packages have been migrated). The 2to3 tool works from either baseline, but there is lots of stuff it can't deal with (basically because 2to3 doesn't do type inference, and it's unlikely that it ever will). In 2.6, the "-3" warnings will help you find stuff that you should transform yourself. (Usually there is a transformation that will do the right thing in 2.6 and which will, after conversion by 2to3, continue to do the right thing in 3.0.) > I'm also expecting that some chuck of users will be left in the 2.x world and that they would like highest version to be as clean as possible (with migration features going into the category of things that don't provide them any benefit). They could either stick with 2.5 (or at least insist on compatibility with 2.5) or use 2.6 without the "-3" flag. They might benefit from the migration features 5 years from now, and until then they shouldn't hurt. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From janssen at parc.com Thu Jan 3 23:05:27 2008 From: janssen at parc.com (Bill Janssen) Date: Thu, 3 Jan 2008 14:05:27 PST Subject: [Python-Dev] Contributing to Python In-Reply-To: References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> <1295843670530617266@unknownmsgid> Message-ID: <08Jan3.140531pst."58696"@synergy1.parc.xerox.com> > > 3.x fixes, because there's no schedule for 2.6. > > Eh? PEP 3000 has a schedule that includes 2.6: OK, no schedule that I knew about :-). I'll get back to work on it. Bill From titus at caltech.edu Fri Jan 4 01:49:37 2008 From: titus at caltech.edu (Titus Brown) Date: Thu, 3 Jan 2008 16:49:37 -0800 Subject: [Python-Dev] Contributing to Python In-Reply-To: References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> <20080103194024.GA15580@caltech.edu> Message-ID: <20080104004937.GD3689@caltech.edu> -> > Incidentally, I'm planning to set up an SVK repos containing the GHOP -> > doc patches; that way they can stay sync'ed with 2.6 work. I'd be happy -> > to do the same thing with reviewed-and-probably-OK patches, although I -> > don't know if repository proliferation is a generally good idea ;). -> -> IMO patches should just be applied to the trunk ASAP. (We're waiting on contributor forms for the GHOP students. That's on my TODO list, too.) --titus From titus at caltech.edu Fri Jan 4 01:59:35 2008 From: titus at caltech.edu (Titus Brown) Date: Thu, 3 Jan 2008 16:59:35 -0800 Subject: [Python-Dev] Contributing to Python In-Reply-To: <477D4BD0.1040204@cheimes.de> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> <20080103201731.GA82115@nexus.in-nomine.org> <477D4BD0.1040204@cheimes.de> Message-ID: <20080104005935.GE3689@caltech.edu> On Thu, Jan 03, 2008 at 09:55:44PM +0100, Christian Heimes wrote: -> Jeroen Ruigrok van der Werven wrote: -> > You don't put the bar high for newbies on the Python project eh? :) -> > -> > I am assumign that most of those contributions code-wise need a fair amount of -> > knowledge of Python's internals? -> -> It's neither impossible nor too hard to get involved. I got from -> "haven't done serious C coding in years" to "Python core developer with -> full svn access" in less than 9 months. OK, I've more than 5 years of -> Python experience but you don't need it to contribute. -> -> You can start by updating or enhancing the existing documentation, -> writing new docs and tutorials or updating the unit test suite. New -> tests need to be written and existing test should be ported to the new -> unit test module. As far as I am aware, we/GHOP have patches for all of the tests in 2.6 => unittest or doctest. They have not all been applied yet, but that will come soon. Speaking as someone with a few years experience in Python and quite a bit of C knowledge, it is not at all easy to come up with a way to dive into the core code. For GHOP, it was a bit of a struggle to come up with tasks; I ended up with three main sources of tasks: - a blog post asking people about their favorite poorly documented module; - the 3.0 spreadsheet and some grepping, to figure out which tests still needed to be ported over to unittest/doctest; - a test coverage analysis to figure out which modules were largely untested; A parent of a GHOP student asked us if we could somehow come up with a suggested task list for new contributors (not just GHOP students). While this could be a lot of work, I think the contributions from GHOP to core in the areas of test coverage, test porting, and module documentation indicate that this kind of outreach may be very effective. If you have ideas on how to keep such a task list fresh and up-to-date as well as populated with good, simple tasks, I'm interested in hearing from you! cheers, --titus From titus at caltech.edu Fri Jan 4 02:24:24 2008 From: titus at caltech.edu (Titus Brown) Date: Thu, 3 Jan 2008 17:24:24 -0800 Subject: [Python-Dev] Contributing to Python In-Reply-To: <938f42d70801031224w76527b25ya60f8988677c3e84@mail.gmail.com> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> <2CAFFA1B-601E-4C27-90C0-6032438EF8F8@acm.org> <20080103195303.GB13654@caltech.edu> <938f42d70801031224w76527b25ya60f8988677c3e84@mail.gmail.com> Message-ID: <20080104012424.GK3689@caltech.edu> On Thu, Jan 03, 2008 at 03:24:16PM -0500, Joseph Armbruster wrote: -> Having a "core mentor" would be great but do they really have time for -> that? I've been lucky at finding people in #python / #python-dev) that can -> answer development inquiries (or at least verify something is or is not a -> bug). Again, IMO as someone on the lunatic fringe of core development (i.e. I'm a happy spectator, but I'm too busy to actually get much done): Mentoring coders may not be a traditional route for hard-core OSS developers, but it sure can be effective, as I've found with GHOP. For example, many core Python developers can save an outsider hours of effort by simply and quickly outlining the issues involved in a particular patch or coding effort. Having actual committers involved is especially good, because they can evaluate whether or not a patch is likely to be accepted, potentially cutting out more hours of effort; and they can directly commit patches, leading to the very important gratification of an actual commit. >From another angle, there are a lot of "easy" fixes/patches/updates to be done to Python, but I'll be damned if I can figure out which ones are easy meat, or complex, or likely to touch a nerve. Having someone experienced to quickly give an opinion is invaluable. (I'm an overconfident loudmouth, so I don't mind posting to this list, but I think python-dev is pretty intimidating for people new to the hurly burly of OSS development.) As I've said in other responses in this thread, I'm not sure how to make it happen, but I'm leaning towards asking the active GHOP mentors to try to extend the GHOP mentoring effort into a general python-love effort. We've got a good group of experienced people, and it's been a pretty friendly list IMO. cheers, --titus From guido at python.org Fri Jan 4 02:46:21 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 3 Jan 2008 17:46:21 -0800 Subject: [Python-Dev] Backport threading.py fix to 2.5.2? Message-ID: See http://bugs.python.org/issue1731. Should we consider it safe to backport r57216 to 2.5.2? This is Thomas Wouters's code to disable spurious tracebacks when daemon threads die. We're running some 2.4 apps with (a variant of) this at Google that get many 1000s of invocations a day, so I'm pretty confident that it works. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From asmodai at in-nomine.org Fri Jan 4 07:44:59 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Fri, 4 Jan 2008 07:44:59 +0100 Subject: [Python-Dev] Backport threading.py fix to 2.5.2? In-Reply-To: References: Message-ID: <20080104064459.GC82115@nexus.in-nomine.org> -On [20080104 02:46], Guido van Rossum (guido at python.org) wrote: >See http://bugs.python.org/issue1731. Should we consider it safe to >backport r57216 to 2.5.2? This is Thomas Wouters's code to disable >spurious tracebacks when daemon threads die. We're running some 2.4 >apps with (a variant of) this at Google that get many 1000s of >invocations a day, so I'm pretty confident that it works. If it fixes a bunch of unnecessary tracebacks which only add confusion whether or not something is working I'm +1 on it. Being on the grunt-side of handling bug reports anything that gets rid of unnecessary bug reports is a good change to backport in my opinion. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ Light-in-Darkness, lift me up from here... From drewp at bigasterisk.com Fri Jan 4 07:51:00 2008 From: drewp at bigasterisk.com (Drew Perttula) Date: Thu, 03 Jan 2008 22:51:00 -0800 Subject: [Python-Dev] Syntax suggestion for imports In-Reply-To: <51D8CC0C-1F9B-4409-B67A-C49671742C3D@gmail.com> References: <20080102201927.AEJ96366@ms19.lnh.mail.rcn.net> <51D8CC0C-1F9B-4409-B67A-C49671742C3D@gmail.com> Message-ID: <477DD754.409@bigasterisk.com> Simon Percivall wrote: > Wouldn't a (stdlib) function suffice in the cases where this is needed? > > ET = import_with_alternative("xml.etree.CElementTree", "cElementTree", > "elementtree.ElementTree") > > It's not as elegant, but it's easier than status quo. > I like that direction a lot better than the syntax proposals. This isn't the kind of thing that needs to get composed into bigger expressions, which is where clever uses of operators really shine. Here, I think the operators were subtracting clarity and your named function was adding clarity. When I saw the OP, I actually wondered why people whose codebases are "filled" with the same try/except block over and over hadn't just written their own import_with_alternative function in the first place. If I wanted half my lines of code to be devoted to control flow and block structure, I'd write in one of those curly-brace languages :) From jyasskin at gmail.com Fri Jan 4 09:13:46 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Fri, 4 Jan 2008 00:13:46 -0800 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> Message-ID: <5d44f72f0801040013o6fc8e455r254f430c415d14d8@mail.gmail.com> On Jan 3, 2008 10:37 AM, Guido van Rossum wrote: > > Well, as issue 1689 states, the backporting was commited by Jeffrey on > > rev 5967 [2], so this is the time to understand if we want this or > > not. > > This is a problem. Right now, in the trunk, math.float(1) returns 1, > where it should return 1.0 for compatibility with 2.5. Jeffrey, can > you fix this and similar incompatibilities you introduced? Whoops! I've committed r59707 to fix math.{floor,ceil}. Let me know if you find any other problems. ... Hmm. I've also changed the behavior of round(2.5). I'll change that back tomorrow morning. I haven't seen any answers to the original question. It looks like Decimal is decided by 2.5 too: return a float from everything. Rational, being a completely new type, is up to you guys, but because new support for the conversion routines seems to be rare, it's probably best to have it return floats too. -- Namast?, Jeffrey Yasskin From brett at python.org Fri Jan 4 09:27:11 2008 From: brett at python.org (Brett Cannon) Date: Fri, 4 Jan 2008 00:27:11 -0800 Subject: [Python-Dev] Contributing to Python In-Reply-To: <20080104012424.GK3689@caltech.edu> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> <2CAFFA1B-601E-4C27-90C0-6032438EF8F8@acm.org> <20080103195303.GB13654@caltech.edu> <938f42d70801031224w76527b25ya60f8988677c3e84@mail.gmail.com> <20080104012424.GK3689@caltech.edu> Message-ID: On Jan 3, 2008 5:24 PM, Titus Brown wrote: > On Thu, Jan 03, 2008 at 03:24:16PM -0500, Joseph Armbruster wrote: > -> Having a "core mentor" would be great but do they really have time for > -> that? I've been lucky at finding people in #python / #python-dev) that can > -> answer development inquiries (or at least verify something is or is not a > -> bug). > > Again, IMO as someone on the lunatic fringe of core development (i.e. > I'm a happy spectator, but I'm too busy to actually get much done): > > Mentoring coders may not be a traditional route for hard-core OSS > developers, but it sure can be effective, as I've found with GHOP. > > For example, many core Python developers can save an outsider hours of > effort by simply and quickly outlining the issues involved in a > particular patch or coding effort. Having actual committers involved is > especially good, because they can evaluate whether or not a patch is > likely to be accepted, potentially cutting out more hours of effort; and > they can directly commit patches, leading to the very important > gratification of an actual commit. > I know I am happy to do stuff that way when I have time. I know I am just currently drowning under work for the first half of 2008 (PyCon, my own Py3K stuff, and thesis). > >From another angle, there are a lot of "easy" fixes/patches/updates to > be done to Python, but I'll be damned if I can figure out which ones are > easy meat, or complex, or likely to touch a nerve. Having someone > experienced to quickly give an opinion is invaluable. (I'm an > overconfident loudmouth, so I don't mind posting to this list, but I > think python-dev is pretty intimidating for people new to the hurly > burly of OSS development.) > I hope that when it comes time to change the issue tracker schema we can have a reasonable difficulty rating. That should also help for Python bug days since we can say "look at easy bugs if you don't really know C stuff, look at medium if you know C, and tackle hard if you want to dive into the nitty-gritty". > As I've said in other responses in this thread, I'm not sure how to make > it happen, but I'm leaning towards asking the active GHOP mentors to try > to extend the GHOP mentoring effort into a general python-love effort. > We've got a good group of experienced people, and it's been a pretty > friendly list IMO. Could work. Don't know if another list is really needed; couldn't python-dev handle general questions? But then again, that is what the tracker is for. -Brett From asmodai at in-nomine.org Fri Jan 4 09:27:47 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Fri, 4 Jan 2008 09:27:47 +0100 Subject: [Python-Dev] Syntax suggestion for imports In-Reply-To: <477DD754.409@bigasterisk.com> References: <20080102201927.AEJ96366@ms19.lnh.mail.rcn.net> <51D8CC0C-1F9B-4409-B67A-C49671742C3D@gmail.com> <477DD754.409@bigasterisk.com> Message-ID: <20080104082747.GE82115@nexus.in-nomine.org> [Half tongue-in-cheek] -On [20080104 08:04], Drew Perttula (drewp at bigasterisk.com) wrote: >When I saw the OP, I actually wondered why people whose codebases are >"filled" with the same try/except block over and over hadn't just >written their own import_with_alternative function in the first place. Because you would have to import it as well in order to use it? ;) -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ Taxation without representation is tyranny... From scott+python-dev at scottdial.com Fri Jan 4 09:48:38 2008 From: scott+python-dev at scottdial.com (Scott Dial) Date: Fri, 04 Jan 2008 03:48:38 -0500 Subject: [Python-Dev] Syntax suggestion for imports In-Reply-To: <20080104082747.GE82115@nexus.in-nomine.org> References: <20080102201927.AEJ96366@ms19.lnh.mail.rcn.net> <51D8CC0C-1F9B-4409-B67A-C49671742C3D@gmail.com> <477DD754.409@bigasterisk.com> <20080104082747.GE82115@nexus.in-nomine.org> Message-ID: <477DF2E6.3090003@scottdial.com> Jeroen Ruigrok van der Werven wrote: > [Half tongue-in-cheek] > > -On [20080104 08:04], Drew Perttula (drewp at bigasterisk.com) wrote: >> When I saw the OP, I actually wondered why people whose codebases are >> "filled" with the same try/except block over and over hadn't just >> written their own import_with_alternative function in the first place. > > Because you would have to import it as well in order to use it? ;) > The keyphrase was "people whose codebases are 'filled' with ..." If they are maintaining a codebase, then I am sure adding one utility function to their library would be trivial. Or if you have enough of them in one file.. dedicating 7 lines of code for: def import_any(*names): for name in names: try: return __import__(name) except StandardError: pass raise ImportError('no importable names') , seems like a bargain for readability. Or to include the "None" case that people have brought up, we can spare 2 more lines: def import_any(*names, **kwargs): required = kwargs.get('required', True) for name in names: try: return __import__(name) except StandardError: pass if required: raise ImportError('no importable names') If nothing else, writing this email will remind me to avoid this try/except pattern in future codebases, if I have more than 1 instance of it (right about the break even point). -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From tdickenson at geminidataloggers.com Fri Jan 4 11:27:04 2008 From: tdickenson at geminidataloggers.com (Toby Dickenson) Date: Fri, 04 Jan 2008 10:27:04 +0000 Subject: [Python-Dev] Syntax suggestion for imports References: <20080102222928.AEK07274@ms19.lnh.mail.rcn.net> <20080103063852.GJ67953@nexus.in-nomine.org> <004901c84dd9$cececb00$6800a8c0@RaymondLaptop1> Message-ID: Raymond Hettinger wrote: > [Jeroen Ruigrok van der Werven] >> On the Trac project using your grep gives me 203 lines, if we take ~2 >> lines for and after in consideration, it still means 203/5 ~= 40 >> occurences. > > Thanks. I'm more curious about the content of those lines. Does the > proposed syntax help, does the need go away in Py3.0, what is the typical > pattern? I dont think I would use this based on a survey of 16 cases over 35k lines here..... The proposed syntax could be easily used 11 times. 10 of those are the 'or None' variant and account for platform-specific modules. 2 of those currently issue a warning about the missing module in the exception branch, but I guess I could live without that. Two other cases uses a different fallback value other than None; one a string literal, and the other 'object'. The remaining three cases do not really fit the proposed syntax. They are currently in the form: try: import xxx except ImportError: recovery actions else: do something with xxx From facundobatista at gmail.com Fri Jan 4 13:34:22 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Fri, 4 Jan 2008 10:34:22 -0200 Subject: [Python-Dev] Contributing to Python In-Reply-To: <20080103195144.GA13654@caltech.edu> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> <20080103194024.GA15580@caltech.edu> <20080103195144.GA13654@caltech.edu> Message-ID: 2008/1/3, Titus Brown : > The question is, is reviewing patches a good place to contribute? Also, > if I (and others) could have a "core mentor" with commit access, that > might streamline things. As it is, I am worried that patch reviews will For a core_mentor/padawan (wink) relationship to work ok, you need that both parties get interested on working on the same stuff. For example, Mark has been working a lot on Decimal, and I have been a merely supervisor and committer of his changes, and we could generate successfully a good patch flow. I don't think that this can be successful when both parties tries to handle wide portions of Python, but I'd love to see a lot of this small sparks of productivity. Now thinking of how to produce this relationships, I think that I will change my approach to the issues. I'll start to be more aggressive when reviewing a patch or bug. Aggressive in the sense of asking/commenting/proposing even if I don't get the full grasp of the issue. This could lead to a better interaction on the mid/long term, even if in the short it appears to increase the noise a little. -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From facundobatista at gmail.com Fri Jan 4 13:40:51 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Fri, 4 Jan 2008 10:40:51 -0200 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: <5d44f72f0801040013o6fc8e455r254f430c415d14d8@mail.gmail.com> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <5d44f72f0801040013o6fc8e455r254f430c415d14d8@mail.gmail.com> Message-ID: 2008/1/4, Jeffrey Yasskin : > I haven't seen any answers to the original question. It looks like > Decimal is decided by 2.5 too: return a float from everything. > Rational, being a completely new type, is up to you guys, but because > new support for the conversion routines seems to be rare, it's > probably best to have it return floats too. Sorry, I didn't understand this parragraph. Do you mean that the response in the following case is of type "float"?: >>> round(decimal.Decimal("2.5")) 3.0 Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From lists at cheimes.de Fri Jan 4 14:07:14 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 04 Jan 2008 14:07:14 +0100 Subject: [Python-Dev] long(float('nan')) conversion Message-ID: Hello! Bug http://bugs.python.org/issue1481296 describes a problem where long(float('nan')) causes a seg fault on Mac OS X. On other platforms it returns 0L, e.g. Python 2.5.1 (r251:54863, Oct 5 2007, 13:36:32) [GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> float('nan') nan >>> long(float('nan')) 0L My patch to Python 2.6 adds an explicit check for NaNs to always return 0L. It did feel a bit strange but it's the default on some platforms. Today an user pointed out that he doesn't like the patch, too. How should the problem be solved? In my humble opinion long(float('nan')) should return 0L in Python 2.5.2 for b/w compatibility and raise a ValueError or OverflowError in Python 2.6+. Christian From amk at amk.ca Fri Jan 4 14:51:43 2008 From: amk at amk.ca (A.M. Kuchling) Date: Fri, 4 Jan 2008 08:51:43 -0500 Subject: [Python-Dev] Bug day proposal: January 19th Message-ID: <20080104135143.GA6291@amk-desktop.matrixgroup.net> A while ago Georg suggested holding a bug day some time soon. I suggest Saturday January 19th. It's a weekend day; it gives us two weeks to prepare by drawing up bug lists; there's a PyPy sprint January 12-19 (http://codespeak.net/pypy/dist/pypy/doc/news.html), so it may be helpful to have the PyPy participants and the python-dev participants available via IM/IRC all at one time. January 12th or 13th would be an alternative date: less time to prepare, but still overlapping with PyPy (they'll be starting instead of winding down). Does anyone feel strongly about the dates, or have other reasons to choose one date or the other? If not, I'll go with the 19th and update the wiki page and send some announcements. --amk From ijmorlan at cs.uwaterloo.ca Fri Jan 4 14:58:16 2008 From: ijmorlan at cs.uwaterloo.ca (Isaac Morland) Date: Fri, 4 Jan 2008 08:58:16 -0500 (EST) Subject: [Python-Dev] Extracting variables from string.Template objects Message-ID: I found myself wanting to obtain the variables from a string.Template object. The situation is that part of input consists of a filename template, and I want to make sure that all the parameter values I have are actually represented in the filename template. So: def templateVariables (template): """Extract the variable names from a string.Template. Returns a tuple of all variable names found in the template, in the order in which they occur. If an invalid escape sequence occurs, the same error will be raised as if an attempt was made to expand the template. """ result = [] for match in template.pattern.finditer (template.template): if match.group ('invalid') is not None: # Raises ValueError template._invalid (match) if match.group ('escaped') is not None: continue # The "or None" should be moot. It is there to ensure equivalent # treatment for an empty 'named' and an empty 'braced'. result.append (match.group ('named') or match.group ('braced') or None) return tuple (result) Note that almost all the work is actually done by calling in to parts of the template object, so I think this should work properly with subclasses and other less common cases. I would like to add this as a method of string.Template, which I think amounts to changing "template" to "self" and putting it in the Template class in string.py rather than on its own. If the general idea is approved I would be happy to make a patch. Also, on a related issue, does it make sense to scan the template string for invalid escape sequences in Template.__init__? For the applications I can imagine of string.Template, I would prefer to get an error upon creating the Template object rather than arbitrarily later when I try to .substitute with it. Isaac Morland CSCF Web Guru DC 2554C, x36650 WWW Software Specialist From ironfroggy at gmail.com Fri Jan 4 15:02:08 2008 From: ironfroggy at gmail.com (Calvin Spealman) Date: Fri, 4 Jan 2008 09:02:08 -0500 Subject: [Python-Dev] long(float('nan')) conversion In-Reply-To: References: Message-ID: <76fd5acf0801040602j3b8ad336xc8f6a77eaaf7d730@mail.gmail.com> On Jan 4, 2008 8:07 AM, Christian Heimes wrote: > Hello! > > Bug http://bugs.python.org/issue1481296 describes a problem where > long(float('nan')) causes a seg fault on Mac OS X. On other platforms it > returns 0L, e.g. > > Python 2.5.1 (r251:54863, Oct 5 2007, 13:36:32) > [GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> float('nan') > nan > >>> long(float('nan')) > 0L I get even more fun results! Python 2.4.4 (#1, Oct 18 2006, 10:34:39) [GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> l = long(float('nan')) 000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000L >>> int(l) 0 >>> long(int(l)) 0L > My patch to Python 2.6 adds an explicit check for NaNs to always return > 0L. It did feel a bit strange but it's the default on some platforms. > Today an user pointed out that he doesn't like the patch, too. > > How should the problem be solved? In my humble opinion > long(float('nan')) should return 0L in Python 2.5.2 for b/w > compatibility and raise a ValueError or OverflowError in Python 2.6+. > > Christian > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ironfroggy%40gmail.com > -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From ironfroggy at gmail.com Fri Jan 4 15:06:32 2008 From: ironfroggy at gmail.com (Calvin Spealman) Date: Fri, 4 Jan 2008 09:06:32 -0500 Subject: [Python-Dev] long(float('nan')) conversion In-Reply-To: <76fd5acf0801040602j3b8ad336xc8f6a77eaaf7d730@mail.gmail.com> References: <76fd5acf0801040602j3b8ad336xc8f6a77eaaf7d730@mail.gmail.com> Message-ID: <76fd5acf0801040606s11bef8fdte3f4846ea1dcb375@mail.gmail.com> Yes, I realize now that I was on the wrong box running the wrong version, so ignore me if I'm stupid and its irrelevant! On Jan 4, 2008 9:02 AM, Calvin Spealman wrote: > On Jan 4, 2008 8:07 AM, Christian Heimes wrote: > > Hello! > > > > Bug http://bugs.python.org/issue1481296 describes a problem where > > long(float('nan')) causes a seg fault on Mac OS X. On other platforms it > > returns 0L, e.g. > > > > Python 2.5.1 (r251:54863, Oct 5 2007, 13:36:32) > > [GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2 > > Type "help", "copyright", "credits" or "license" for more information. > > >>> float('nan') > > nan > > >>> long(float('nan')) > > 0L > > I get even more fun results! > > Python 2.4.4 (#1, Oct 18 2006, 10:34:39) > [GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin > Type "help", "copyright", "credits" or "license" for more information. > >>> l = long(float('nan')) > 000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000L > >>> int(l) > 0 > >>> long(int(l)) > > 0L > > > My patch to Python 2.6 adds an explicit check for NaNs to always return > > 0L. It did feel a bit strange but it's the default on some platforms. > > Today an user pointed out that he doesn't like the patch, too. > > > > How should the problem be solved? In my humble opinion > > long(float('nan')) should return 0L in Python 2.5.2 for b/w > > compatibility and raise a ValueError or OverflowError in Python 2.6+. > > > > Christian > > > > _______________________________________________ > > Python-Dev mailing list > > Python-Dev at python.org > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ironfroggy%40gmail.com > > > > > > -- > Read my blog! I depend on your acceptance of my opinion! I am interesting! > http://ironfroggy-code.blogspot.com/ > -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From schmir at gmail.com Fri Jan 4 16:05:22 2008 From: schmir at gmail.com (Ralf Schmitt) Date: Fri, 4 Jan 2008 16:05:22 +0100 Subject: [Python-Dev] siginterrupt on linux (issue1089358) Message-ID: <932f8baf0801040705m611fa02dt3afbde332a88f8c7@mail.gmail.com> Hi all, I've added a patch implementing a wrapper for siginterrupt on linux/unix like platforms and attached it to http://bugs.python.org/issue1089358 (which mentions a need for siginterrupt). Any chance to get this in? Regards, - Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080104/1b06ba1a/attachment.htm From schmir at gmail.com Fri Jan 4 16:15:06 2008 From: schmir at gmail.com (Ralf Schmitt) Date: Fri, 4 Jan 2008 16:15:06 +0100 Subject: [Python-Dev] Check for signals during regular expression matches (issue846388) Message-ID: <932f8baf0801040715saa29902udfcdb4d70ad81f2d@mail.gmail.com> Hi all, Currently the re module does not handle signals. I've been once a victim of such a case in a real world setup (i.e. the python interpreter seems to hang at 100% CPU and is not interuptible with control c) http://bugs.python.org/issue846388 opened 4 years ago contained a patch for this issue (which ofcourse didn't apply). I've updated that patch and added an example regular expression, which 'hangs' the python interpreter. I think this issue should be handled. Regards, - Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080104/083ad53d/attachment-0001.htm From amk at amk.ca Fri Jan 4 16:23:54 2008 From: amk at amk.ca (A.M. Kuchling) Date: Fri, 4 Jan 2008 10:23:54 -0500 Subject: [Python-Dev] Bug day tasks Message-ID: <20080104152354.GA25839@amk-desktop.matrixgroup.net> I've updated the bug day pages in the wiki: http://wiki.python.org/moin/PythonBugDay http://wiki.python.org/moin/PythonBugDayStatus How do we want to flag good candidate bugs? Should we add a keyword to Roundup, or just list them on the PythonBugDayStatus wiki page? Another task is to get logging set up for the #python-dev IRC channel. Searching didn't find any existing archive; we could run it on python.org somewhere, but does anyone here already run an IRC logging bot? Maybe someone could just add #python-dev to their existing setup. --amk From facundobatista at gmail.com Fri Jan 4 16:48:31 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Fri, 4 Jan 2008 13:48:31 -0200 Subject: [Python-Dev] Bug day tasks In-Reply-To: <20080104152354.GA25839@amk-desktop.matrixgroup.net> References: <20080104152354.GA25839@amk-desktop.matrixgroup.net> Message-ID: 2008/1/4, A.M. Kuchling : > I've updated the bug day pages in the wiki: This one should be also updated: http://wiki.python.org/moin/MissingFromDocumentation All the issues pointed by it are already closed (or don't exist (!)). > http://wiki.python.org/moin/PythonBugDay The URL... http://moin.pocoo.org/Python25a ..., from the Question section, appears to be dead. > How do we want to flag good candidate bugs? Should we add a keyword > to Roundup, or just list them on the PythonBugDayStatus wiki page? As this bug days are spurious, I think that just list them in the wiki page is ok. Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From aahz at pythoncraft.com Fri Jan 4 16:52:56 2008 From: aahz at pythoncraft.com (Aahz) Date: Fri, 4 Jan 2008 07:52:56 -0800 Subject: [Python-Dev] Extracting variables from string.Template objects In-Reply-To: References: Message-ID: <20080104155256.GA27695@panix.com> On Fri, Jan 04, 2008, Isaac Morland wrote: > > I would like to add this as a method of string.Template, which I think > amounts to changing "template" to "self" and putting it in the Template > class in string.py rather than on its own. If the general idea is > approved I would be happy to make a patch. Make a patch regardless of whether you get any approval -- someone may later have a different opinion, and writing a patch records your work. > Also, on a related issue, does it make sense to scan the template > string for invalid escape sequences in Template.__init__? For the > applications I can imagine of string.Template, I would prefer to get > an error upon creating the Template object rather than arbitrarily > later when I try to .substitute with it. No, create an is_valid() method at best. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From lists at cheimes.de Fri Jan 4 16:53:46 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 04 Jan 2008 16:53:46 +0100 Subject: [Python-Dev] Bug day tasks In-Reply-To: <20080104152354.GA25839@amk-desktop.matrixgroup.net> References: <20080104152354.GA25839@amk-desktop.matrixgroup.net> Message-ID: <477E568A.4060608@cheimes.de> A.M. Kuchling wrote: > Another task is to get logging set up for the #python-dev IRC channel. > Searching didn't find any existing archive; we could run it on > python.org somewhere, but does anyone here already run an IRC logging > bot? Maybe someone could just add #python-dev to their existing > setup. It'd be nice if we can also get a bot into #python-dev to broadcast svn commits and bug tracker changes. The Twisted guys have good bot with decent msg coloring but IIRC it's tight into TRAC. For svn we could probably use CIA bot and tie it into a svn post commit hook. Christian From jyasskin at gmail.com Fri Jan 4 17:14:31 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Fri, 4 Jan 2008 08:14:31 -0800 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <5d44f72f0801040013o6fc8e455r254f430c415d14d8@mail.gmail.com> Message-ID: <5d44f72f0801040814n29efa003tee1dc01efc0224c2@mail.gmail.com> On Jan 4, 2008 4:40 AM, Facundo Batista wrote: > 2008/1/4, Jeffrey Yasskin : > > > I haven't seen any answers to the original question. It looks like > > Decimal is decided by 2.5 too: return a float from everything. > > Rational, being a completely new type, is up to you guys, but because > > new support for the conversion routines seems to be rare, it's > > probably best to have it return floats too. > > Sorry, I didn't understand this parragraph. > > Do you mean that the response in the following case is of type "float"?: > > >>> round(decimal.Decimal("2.5")) > 3.0 Yes. -- Namast?, Jeffrey Yasskin From dickinsm at gmail.com Fri Jan 4 17:24:52 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Fri, 4 Jan 2008 11:24:52 -0500 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: <5d44f72f0801040814n29efa003tee1dc01efc0224c2@mail.gmail.com> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <5d44f72f0801040013o6fc8e455r254f430c415d14d8@mail.gmail.com> <5d44f72f0801040814n29efa003tee1dc01efc0224c2@mail.gmail.com> Message-ID: <5c6f2a5d0801040824v38667036ud82ddf440983a437@mail.gmail.com> On Jan 4, 2008 11:14 AM, Jeffrey Yasskin wrote: > On Jan 4, 2008 4:40 AM, Facundo Batista wrote: > > Do you mean that the response in the following case is of type "float"?: > > > > >>> round(decimal.Decimal("2.5")) > > 3.0 > > Yes. > That seems a little peculiar to me: wouldn't it be more natural to have round(Decimal_instance) return another Decimal? Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080104/addaff87/attachment.htm From facundobatista at gmail.com Fri Jan 4 17:37:50 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Fri, 4 Jan 2008 14:37:50 -0200 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: <5c6f2a5d0801040824v38667036ud82ddf440983a437@mail.gmail.com> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <5d44f72f0801040013o6fc8e455r254f430c415d14d8@mail.gmail.com> <5d44f72f0801040814n29efa003tee1dc01efc0224c2@mail.gmail.com> <5c6f2a5d0801040824v38667036ud82ddf440983a437@mail.gmail.com> Message-ID: 2008/1/4, Mark Dickinson : > That seems a little peculiar to me: wouldn't it be more natural to have > round(Decimal_instance) return another Decimal? Yes. Now I find that now round() delegates its work to __round__: http://docs.python.org/dev/library/functions.html#round This doc says that round() "Return the floating point value x rounded to n digits after the decimal point.". It don't specify if its binary or decimal floating point, ;) Feel free to create an issue and assign to me if you think that this should be done. Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From exarkun at divmod.com Fri Jan 4 17:45:37 2008 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Fri, 4 Jan 2008 11:45:37 -0500 Subject: [Python-Dev] Bug day tasks In-Reply-To: <477E568A.4060608@cheimes.de> Message-ID: <20080104164537.31425.522882183.divmod.quotient.695@ohm> On Fri, 04 Jan 2008 16:53:46 +0100, Christian Heimes wrote: >A.M. Kuchling wrote: >> Another task is to get logging set up for the #python-dev IRC channel. >> Searching didn't find any existing archive; we could run it on >> python.org somewhere, but does anyone here already run an IRC logging >> bot? Maybe someone could just add #python-dev to their existing >> setup. > >It'd be nice if we can also get a bot into #python-dev to broadcast svn >commits and bug tracker changes. The Twisted guys have good bot with >decent msg coloring but IIRC it's tight into TRAC. For svn we could >probably use CIA bot and tie it into a svn post commit hook. The trac integration is entirely optional, so don't let that discourage you. If anyone wants to investigate setting this up, svn://divmod.org/svn/Divmod/sandbox/exarkun/commit-bot The code has no unit tests and there is no documentation. Also notice "sandbox" in the SVN URL. The only real advantage that it has over CIA that I can point out is that you don't have to write an XML or have a SQL server running in order to use it. Jean-Paul From guido at python.org Fri Jan 4 17:45:46 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 4 Jan 2008 08:45:46 -0800 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <5d44f72f0801040013o6fc8e455r254f430c415d14d8@mail.gmail.com> <5d44f72f0801040814n29efa003tee1dc01efc0224c2@mail.gmail.com> <5c6f2a5d0801040824v38667036ud82ddf440983a437@mail.gmail.com> Message-ID: On Jan 4, 2008 8:37 AM, Facundo Batista wrote: > 2008/1/4, Mark Dickinson : > > > That seems a little peculiar to me: wouldn't it be more natural to have > > round(Decimal_instance) return another Decimal? > > Yes. > > Now I find that now round() delegates its work to __round__: > > http://docs.python.org/dev/library/functions.html#round > > This doc says that round() "Return the floating point value x rounded > to n digits after the decimal point.". > > It don't specify if its binary or decimal floating point, ;) > > Feel free to create an issue and assign to me if you think that this > should be done. In 3.0, round() of a Decimal should return an int if invoked with 1 arg, a Decimal if invoked with 2. Similar for round() of a float. round() of an int can always return an int. In 2.6, round() should always return a float, regardless of the type of the first arg or the arg count, for best compatibility with 2.5. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Jan 4 17:50:03 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 4 Jan 2008 08:50:03 -0800 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: <5d44f72f0801040013o6fc8e455r254f430c415d14d8@mail.gmail.com> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <5d44f72f0801040013o6fc8e455r254f430c415d14d8@mail.gmail.com> Message-ID: On Jan 4, 2008 12:13 AM, Jeffrey Yasskin wrote: > On Jan 3, 2008 10:37 AM, Guido van Rossum wrote: > > > Well, as issue 1689 states, the backporting was commited by Jeffrey on > > > rev 5967 [2], so this is the time to understand if we want this or > > > not. > > > > This is a problem. Right now, in the trunk, math.float(1) returns 1, > > where it should return 1.0 for compatibility with 2.5. Jeffrey, can > > you fix this and similar incompatibilities you introduced? > > Whoops! I've committed r59707 to fix math.{floor,ceil}. Let me know if > you find any other problems. ... Hmm. I've also changed the behavior > of round(2.5). I'll change that back tomorrow morning. Looks like in Python 2.6, round(0.5) right now returns 0.0, whereas in 2.5, it returns 1.0. I think it should return 1.0, for best compatibility with Python 2.5. > I haven't seen any answers to the original question. It looks like > Decimal is decided by 2.5 too: return a float from everything. Right. > Rational, being a completely new type, is up to you guys, but because > new support for the conversion routines seems to be rare, it's > probably best to have it return floats too. I think the pre-3.0 rule should be: round(), math.floor(), math.ceil() *always* return a float. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From skip at pobox.com Fri Jan 4 17:58:33 2008 From: skip at pobox.com (skip at pobox.com) Date: Fri, 4 Jan 2008 10:58:33 -0600 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <5d44f72f0801040013o6fc8e455r254f430c415d14d8@mail.gmail.com> Message-ID: <18302.26041.596158.499906@montanaro.dyndns.org> Guido> Looks like in Python 2.6, round(0.5) right now returns 0.0, Guido> whereas in 2.5, it returns 1.0. Guido> I think it should return 1.0, for best compatibility with Python Guido> 2.5. And for best compatibility with everything else! <0.5 wink> Skip From guido at python.org Fri Jan 4 18:05:43 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 4 Jan 2008 09:05:43 -0800 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: <18302.26041.596158.499906@montanaro.dyndns.org> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <5d44f72f0801040013o6fc8e455r254f430c415d14d8@mail.gmail.com> <18302.26041.596158.499906@montanaro.dyndns.org> Message-ID: On Jan 4, 2008 8:58 AM, wrote: > > Guido> Looks like in Python 2.6, round(0.5) right now returns 0.0, > Guido> whereas in 2.5, it returns 1.0. > > Guido> I think it should return 1.0, for best compatibility with Python > Guido> 2.5. > > And for best compatibility with everything else! <0.5 wink> Should I round that to 0 whole winks or 1 whole wink? Rounding 0.5 to 0 is a new fad, called round-to-even, or statistician's rounding. Read http://en.wikipedia.org/wiki/Rounding . It's a standard, and more correct when doing statistical calculations. That's why we've chosen it for Python 3.0. But it should not bleed into Python 2.6. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Jan 4 18:11:25 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 4 Jan 2008 09:11:25 -0800 Subject: [Python-Dev] Backport threading.py fix to 2.5.2? In-Reply-To: References: Message-ID: OK, I'll backport it. On Jan 4, 2008 4:49 AM, Thomas Wouters wrote: > > > > On Jan 4, 2008 2:46 AM, Guido van Rossum wrote: > > See http://bugs.python.org/issue1731. Should we consider it safe to > > backport r57216 to 2.5.2? This is Thomas Wouters's code to disable > > spurious tracebacks when daemon threads die. We're running some 2.4 > > apps with (a variant of) this at Google that get many 1000s of > > invocations a day, so I'm pretty confident that it works. > > > > I'm also pretty confident it works, although it isn't really guaranteed to > catch *all* such situations. No reason not to backport it, just a remark > about how it checks to see if Python is shutting down. It is, however, > incredibly unlikely '_sys' won't be gone when we check for it if the > exception is indeed a spurious one. What could happen is that a legitimate > exception happens right before interpreter shutdown, then a thread switch > occurs before Python tries to report the exception, the interpreter exits, > and then the daemonic thread starts to report its exception. The only way to > catch that is to do the 'are we exiting' check in the C code that does the > actual thread-exception reporting. I'm not sure if it's worth it, as the > timing has to be pretty exact for that to happen, and you wouldn't want to > introduce a bug there; could be years before someone figures it out :P > > -- > Thomas Wouters > > Hi! I'm a .signature virus! copy me into your .signature file to help me > spread! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Jan 4 18:19:30 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 4 Jan 2008 09:19:30 -0800 Subject: [Python-Dev] long(float('nan')) conversion In-Reply-To: References: Message-ID: On Jan 4, 2008 5:07 AM, Christian Heimes wrote: > Bug http://bugs.python.org/issue1481296 describes a problem where > long(float('nan')) causes a seg fault on Mac OS X. On other platforms it > returns 0L, e.g. > > Python 2.5.1 (r251:54863, Oct 5 2007, 13:36:32) > [GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> float('nan') > nan > >>> long(float('nan')) > 0L > > My patch to Python 2.6 adds an explicit check for NaNs to always return > 0L. It did feel a bit strange but it's the default on some platforms. > Today an user pointed out that he doesn't like the patch, too. > > How should the problem be solved? In my humble opinion > long(float('nan')) should return 0L in Python 2.5.2 for b/w > compatibility and raise a ValueError or OverflowError in Python 2.6+. If long(nan) or int(nan) returns 0 on most platforms in 2.5, we should fix them to always return 0 in 2.5 *and* 2.6. In 3.0 they should raise ValueError. It looks like long(inf) and int(inf) already raise OverflowError and that should stay. We should make sure inf and nan are treated correctly by the new round(), floor() and ceil() in 3.0 -- it looks like right now round(nan) returns 0, but it should raise ValueError. (Also, Jeffrey, I thought math.floor() was to return an int? Or do I misremember?) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From skip at pobox.com Fri Jan 4 18:25:32 2008 From: skip at pobox.com (skip at pobox.com) Date: Fri, 4 Jan 2008 11:25:32 -0600 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <5d44f72f0801040013o6fc8e455r254f430c415d14d8@mail.gmail.com> <18302.26041.596158.499906@montanaro.dyndns.org> Message-ID: <18302.27660.899384.319382@montanaro.dyndns.org> Guido> Rounding 0.5 to 0 is a new fad, called round-to-even, or Guido> statistician's rounding. Read Guido> http://en.wikipedia.org/wiki/Rounding . It's a standard, and Guido> more correct when doing statistical calculations. That's why Guido> we've chosen it for Python 3.0. But it should not bleed into Guido> Python 2.6. Thanks for the pointer. Given that it's been an ASTM standard since 1940 and apparently in fairly common use since the early 1900s, I wonder why it's not been more widely used in the past in programming languages. Skip From jyasskin at gmail.com Fri Jan 4 18:48:26 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Fri, 4 Jan 2008 09:48:26 -0800 Subject: [Python-Dev] long(float('nan')) conversion In-Reply-To: References: Message-ID: <5d44f72f0801040948u7fa8fa47hed414ef657350f21@mail.gmail.com> On Jan 4, 2008 9:19 AM, Guido van Rossum wrote: > We should make sure inf and nan are treated correctly by the new > round(), floor() and ceil() in 3.0 -- it looks like right now > round(nan) returns 0, but it should raise ValueError. (Also, Jeffrey, > I thought math.floor() was to return an int? Or do I misremember?) That's correct. It's currently broken, but I was waiting to push on http://bugs.python.org/issue1656 until the backport was figured out. -- Namast?, Jeffrey Yasskin From status at bugs.python.org Fri Jan 4 19:06:08 2008 From: status at bugs.python.org (Tracker) Date: Fri, 4 Jan 2008 18:06:08 +0000 (UTC) Subject: [Python-Dev] Summary of Tracker Issues Message-ID: <20080104180608.19308781B1@psf.upfronthosting.co.za> ACTIVITY SUMMARY (12/28/07 - 01/04/08) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1371 open (+15) / 11849 closed (+16) / 13220 total (+31) Open issues with patches: 430 Average duration of open issues: 687 days. Median duration of open issues: 947 days. Open Issues Breakdown open 1359 (+15) pending 12 ( +0) Issues Created Or Reopened (32) _______________________________ urllib* 20x responses not OK? 01/02/08 CLOSED http://bugs.python.org/issue1177 reopened kbk patch probable bug in code.py with Python 3.0a2 12/29/07 CLOSED http://bugs.python.org/issue1707 created aroberge improvements for linecache 12/30/07 http://bugs.python.org/issue1708 created umaxx patch logging: log actual function name 12/30/07 CLOSED http://bugs.python.org/issue1709 created umaxx Pypi's "score" column is not explained 12/30/07 CLOSED http://bugs.python.org/issue1710 created giampaolo.rodola socket functions that should return unsigned int return signed i 12/30/07 http://bugs.python.org/issue1711 created mthibaut Small errors in new-style classes doc 12/30/07 http://bugs.python.org/issue1712 created salty-horse posixpath.ismount() claims symlink to a mountpoint is a mountpoi 12/31/07 CLOSED http://bugs.python.org/issue1713 created drougge ConfigParser.py do not allow leading (and trailing) space in val 12/31/07 http://bugs.python.org/issue1714 created msuchy Make pydoc list submodules 12/31/07 http://bugs.python.org/issue1715 created gustavo patch String format operator '%i' fails for large floats 01/01/08 http://bugs.python.org/issue1716 created ctl Get rid of more refercenes to __cmp__ 01/01/08 http://bugs.python.org/issue1717 created gvanrossum patch Tarfile fails to fully extract tar.bz2/tar.gz package 01/01/08 CLOSED http://bugs.python.org/issue1718 created cartman Cosmetic patch for doc/code mismatch (msilib.UuidCreate) 01/02/08 http://bugs.python.org/issue1719 created ocean-city VC6 build patch for release-maint25 01/02/08 http://bugs.python.org/issue1720 created ocean-city patch _tkinter.c:903: AsObj: Assertion `size < size * sizeof(Tcl_UniCh 01/02/08 CLOSED http://bugs.python.org/issue1721 created tiran py3k Undocumented urllib functions 01/02/08 http://bugs.python.org/issue1722 created barry Respuesta autom??tica de Yahoo! 01/02/08 CLOSED http://bugs.python.org/issue1723 created gagenellina Py_SIZE() macro used as an lvalue 01/02/08 CLOSED http://bugs.python.org/issue1724 created rhettinger -1e-1000 converted incorrectly 01/03/08 http://bugs.python.org/issue1725 created gvanrossum Remove Python/atof.c from PCBuild/pythoncore.vcproj 01/03/08 CLOSED http://bugs.python.org/issue1726 created ocean-city py3k, patch VC6 build patch for python3000 01/03/08 CLOSED http://bugs.python.org/issue1727 created ocean-city py3k, patch distutils.cmd breaks inspect 01/03/08 CLOSED http://bugs.python.org/issue1728 created astronouth7303 Allow float('infinity') as well as float('inf') 01/03/08 CLOSED http://bugs.python.org/issue1729 created marketdickinson Documentation corrections for os module 01/03/08 http://bugs.python.org/issue1730 created robin.stocker patch Random errors on interpreter shutdown 01/03/08 http://bugs.python.org/issue1731 created lobais Doc build fails with a KeyError 01/04/08 CLOSED http://bugs.python.org/issue1732 created rhettinger Maybe PC/VS7.1/pythoncore.vcproj is missing Modules/md5module.c 01/04/08 CLOSED http://bugs.python.org/issue1733 created ocean-city documentation on metaclasses is incomplete and misleading 01/04/08 CLOSED http://bugs.python.org/issue1734 created lpd tarfile.TarFile.extractall not setting directory permissions cor 01/04/08 CLOSED http://bugs.python.org/issue1735 created eandres patch Three bugs of FCICreate (PC/_msi.c) 01/04/08 http://bugs.python.org/issue1736 created ocean-city py3k, patch Windows installer issue (ObjectBrowser.py) 01/04/08 http://bugs.python.org/issue1737 created dsuch Issues Now Closed (43) ______________________ urllib* 20x responses not OK? 0 days http://bugs.python.org/issue1177 kbk patch urllib fail to read URL contents, urllib2 crash Python 99 days http://bugs.python.org/issue1205 gvanrossum os.system() oddity under Windows XP SP2 80 days http://bugs.python.org/issue1279 tiran Bad assertion in _tkinter.c 77 days http://bugs.python.org/issue1301 thyrsus patch VS2008, quick hack for distutils.msvccompiler 44 days http://bugs.python.org/issue1455 loewis py3k, patch Problem with static libs on Windows 35 days http://bugs.python.org/issue1527 tiran py3k, patch Patch for new API method _PyImport_ImportModuleNoLock(char *name 28 days http://bugs.python.org/issue1567 tiran Crash on cancellation of windows install 8 days http://bugs.python.org/issue1690 tiran Please check merge from trunk 8 days http://bugs.python.org/issue1693 gvanrossum py3k Distutils ignore dry-run flag at clean-up of distutils.command.b 9 days http://bugs.python.org/issue1696 gvanrossum patch unconditional definiton of _BSD_SOURCE breaks builds with g++-4. 4 days http://bugs.python.org/issue1699 loewis Regular Expression inline flags not handled correctly for some u 9 days http://bugs.python.org/issue1700 gvanrossum getpass broken in Py3k: must flush() 5 days http://bugs.python.org/issue1703 gvanrossum possible bug in randint 2 days http://bugs.python.org/issue1704 rhettinger trace module does not annotate global statement 5 days http://bugs.python.org/issue1705 gvanrossum probable bug in code.py with Python 3.0a2 4 days http://bugs.python.org/issue1707 gvanrossum logging: log actual function name 3 days http://bugs.python.org/issue1709 vsajip Pypi's "score" column is not explained 0 days http://bugs.python.org/issue1710 loewis posixpath.ismount() claims symlink to a mountpoint is a mountpoi 4 days http://bugs.python.org/issue1713 tiran Tarfile fails to fully extract tar.bz2/tar.gz package 0 days http://bugs.python.org/issue1718 gvanrossum _tkinter.c:903: AsObj: Assertion `size < size * sizeof(Tcl_UniCh 0 days http://bugs.python.org/issue1721 loewis py3k Respuesta autom??tica de Yahoo! 0 days http://bugs.python.org/issue1723 gvanrossum Py_SIZE() macro used as an lvalue 0 days http://bugs.python.org/issue1724 loewis Remove Python/atof.c from PCBuild/pythoncore.vcproj 0 days http://bugs.python.org/issue1726 tiran py3k, patch VC6 build patch for python3000 1 days http://bugs.python.org/issue1727 tiran py3k, patch distutils.cmd breaks inspect 0 days http://bugs.python.org/issue1728 gvanrossum Allow float('infinity') as well as float('inf') 0 days http://bugs.python.org/issue1729 gvanrossum Doc build fails with a KeyError 0 days http://bugs.python.org/issue1732 tiran Maybe PC/VS7.1/pythoncore.vcproj is missing Modules/md5module.c 0 days http://bugs.python.org/issue1733 tiran documentation on metaclasses is incomplete and misleading 0 days http://bugs.python.org/issue1734 lpd tarfile.TarFile.extractall not setting directory permissions cor 0 days http://bugs.python.org/issue1735 lars.gustaebel patch Rewrite _reduce and _reconstructor in C 1926 days http://bugs.python.org/issue614555 gvanrossum smtpd.py move setuid, allow dashes 1793 days http://bugs.python.org/issue681515 tiran patch readline module that checks signals 1651 days http://bugs.python.org/issue761863 tiran patch slow socket binding & netinfo lookups 1628 days http://bugs.python.org/issue774751 tiran 2.3c2 test_pwd fails on Mac OS X 10.2.6 1620 days http://bugs.python.org/issue779218 tiran socketmodule.c does not pickup all symbols with MS visC++ 1481 days http://bugs.python.org/issue860134 tiran win32 raw socket support 1431 days http://bugs.python.org/issue889544 tiran configure argument --libdir is ignored 1427 days http://bugs.python.org/issue891930 tiran Build fails on XP w/mingw 1406 days http://bugs.python.org/issue906405 tiran dir(object) does not list __name__ 1343 days http://bugs.python.org/issue945861 tiran Problem linking on windows using mingw32 and C++ 1205 days http://bugs.python.org/issue1028697 tiran xml.etree document element.tag 267 days http://bugs.python.org/issue1698167 effbot Top Issues Most Discussed (10) ______________________________ 11 distutils.cmd breaks inspect 0 days closed http://bugs.python.org/issue1728 11 Enhancements for mathmodule 18 days open http://bugs.python.org/issue1640 8 Bad assertion in _tkinter.c 77 days closed http://bugs.python.org/issue1301 7 socket functions that should return unsigned int return signed 5 days open http://bugs.python.org/issue1711 7 VC6 build patch for trunk 266 days open http://bugs.python.org/issue1700463 7 Distutils ignore dry-run flag at clean-up of distutils.command. 9 days closed http://bugs.python.org/issue1696 6 -1e-1000 converted incorrectly 2 days open http://bugs.python.org/issue1725 6 Incorrectly displayed non ascii characters in prompt using "inp 13 days open http://bugs.python.org/issue1688 5 floating point number round failures under Linux 11 days open http://bugs.python.org/issue1694 5 option.dest not set when callback called with optparse 90 days open http://bugs.python.org/issue1243 From amk at amk.ca Fri Jan 4 19:32:39 2008 From: amk at amk.ca (A.M. Kuchling) Date: Fri, 4 Jan 2008 13:32:39 -0500 Subject: [Python-Dev] Bug day tasks In-Reply-To: <477E568A.4060608@cheimes.de> References: <20080104152354.GA25839@amk-desktop.matrixgroup.net> <477E568A.4060608@cheimes.de> Message-ID: <20080104183239.GA28629@amk-desktop.matrixgroup.net> On Fri, Jan 04, 2008 at 04:53:46PM +0100, Christian Heimes wrote: > It'd be nice if we can also get a bot into #python-dev to broadcast svn > commits and bug tracker changes. The Twisted guys have good bot with > decent msg coloring but IIRC it's tight into TRAC. For svn we could > probably use CIA bot and tie it into a svn post commit hook. I'm thinking more about something to log all messages, so that participants can say "go see Raymond's explanation around 9AM". For whoever's set up the CIA bot on python-dev: it seems to broadcast messages twice. (01:26:10 PM) amk_: I've just made a commit to the Python repository; we'll see if the CIA bot does anything. (01:26:36 PM) CIA-35: andrew.kuchling * r59722 Doc/whatsnew/2.6.rst: Fix markup (01:26:36 PM) CIA-35: andrew.kuchling * r59723 Doc/library/math.rst: Fix markup (01:26:43 PM) amk_: Aha! (01:26:45 PM) CIA-35: andrew.kuchling * r59722 Doc/whatsnew/2.6.rst: Fix markup (01:26:46 PM) CIA-35: andrew.kuchling * r59723 Doc/library/math.rst: Fix markup The commits also showed up twice in #commits: (01:26:35 PM) CIA-9: merb: vox at exdolo.com * r1164 /trunk/app_generators/merb/templates/Rakefile: core_ext/string is not loaded at this point, and there for the String#/ alias to File.join is not available. Rather than load that largish dep at this point, just change to File.join. resolves #395 (01:26:36 PM) CIA-9: python: andrew.kuchling * r59722 Doc/whatsnew/2.6.rst: Fix markup (01:26:36 PM) CIA-9: python: andrew.kuchling * r59723 Doc/library/math.rst: Fix markup (01:26:45 PM) CIA-9: python: andrew.kuchling * r59722 Doc/whatsnew/2.6.rst: Fix markup (01:26:46 PM) CIA-9: python: andrew.kuchling * r59723 Doc/library/math.rst: Fix markup (01:27:02 PM) CIA-9: jmol: nicove * r8906 /trunk/Jmol-FAH/projects/ (p3060.xyz.gz p3061.xyz.gz p3062.xyz.gz p3906.xyz.gz): Folding at Home --amk From guido at python.org Fri Jan 4 20:08:43 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 4 Jan 2008 11:08:43 -0800 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: <18302.27660.899384.319382@montanaro.dyndns.org> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <5d44f72f0801040013o6fc8e455r254f430c415d14d8@mail.gmail.com> <18302.26041.596158.499906@montanaro.dyndns.org> <18302.27660.899384.319382@montanaro.dyndns.org> Message-ID: On Jan 4, 2008 9:25 AM, wrote: > > Guido> Rounding 0.5 to 0 is a new fad, called round-to-even, or > Guido> statistician's rounding. Read > Guido> http://en.wikipedia.org/wiki/Rounding . It's a standard, and > Guido> more correct when doing statistical calculations. That's why > Guido> we've chosen it for Python 3.0. But it should not bleed into > Guido> Python 2.6. > > Thanks for the pointer. Given that it's been an ASTM standard since 1940 > and apparently in fairly common use since the early 1900s, I wonder why it's > not been more widely used in the past in programming languages. Because programming language designers rarely are statisticians? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From lists at cheimes.de Fri Jan 4 20:17:39 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 04 Jan 2008 20:17:39 +0100 Subject: [Python-Dev] Bug day tasks In-Reply-To: <477E568A.4060608@cheimes.de> References: <20080104152354.GA25839@amk-desktop.matrixgroup.net> <477E568A.4060608@cheimes.de> Message-ID: <477E8653.8090301@cheimes.de> Christian Heimes wrote: > It'd be nice if we can also get a bot into #python-dev to broadcast svn > commits and bug tracker changes. The Twisted guys have good bot with > decent msg coloring but IIRC it's tight into TRAC. For svn we could > probably use CIA bot and tie it into a svn post commit hook. I've written a CIA notification extension for Roundup: http://pastebin.ca/841568 It works standalone but I haven't been able to enable email and CIA notifications in my local demo of Roundup. The output could probably use some adjustments, too. Can somebody please test it and integrate it into the bug tracker? Christian From guido at python.org Fri Jan 4 20:18:27 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 4 Jan 2008 11:18:27 -0800 Subject: [Python-Dev] Need closure on __cmp__ removal Message-ID: In the past some folks have been pushing for the resurrection of (some form of) __cmp__, which is currently removed from Py3k (except for some remnants which we'll clean up in due time). I'd like to get closure on this issue. If someone volunteers within a week to write a PEP, I'll give them a month to write the PEP, and then I'll review it. The PEP better come with a patch implementing (roughly) the desired behavior as well, relative to the 3.0 branch. If I don't hear from a committed volunteer within a week, I'll drop this and start removing __cmp__ references aggressively (starting with issue #1717). Saying "if no-one else volunteers, I can give it a shot" is not sufficient commitment. Saying "I will give it a shot" is. If someone commits but no PEP+patch is in my possession by February 4 (and no attenuating circumstances have been brought to my attention), I will assume the PEP won't happen and will start removing __cmp__ references. Once a PEP and patch are presented, I'll review them and make a decision. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.peters at gmail.com Fri Jan 4 20:31:10 2008 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 4 Jan 2008 14:31:10 -0500 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <5d44f72f0801040013o6fc8e455r254f430c415d14d8@mail.gmail.com> <18302.26041.596158.499906@montanaro.dyndns.org> <18302.27660.899384.319382@montanaro.dyndns.org> Message-ID: <1f7befae0801041131v75af2a2dtb451391376c202fa@mail.gmail.com> [skip at pobox.com] > Thanks for the pointer. Given that it's [round-to-even[ been an ASTM > standard since 1940 and apparently in fairly common use since the > early 1900s, I wonder why it's not been more widely used in the past > in programming languages. Because "add a half and chop" was also in wide use for even longer, is also (Wikipedia notwithstanding) part of many standards (for example, the US IRS requires it if you do your taxes under the "round to whole dollars" option), and-- probably the real driver --is a little cheaper to implement for "normal sized" floats. Curiously, round-to-nearest can be unboundedly more expensive to implement in some obscure contexts when floats can have very large exponents (as they can in Python's "decimal" module -- this is why the proposed decimal standard allows operations like "remainder-near" to fail if applied to inputs that are "too far apart": http://www2.hursley.ibm.com/decimal/daops.html#footnote.8 The "secret reason" is that it can be unboundedly more expensive to determine the last bit of the quotient (to see whether it's even or odd) than to determine an exact remainder). From rhamph at gmail.com Fri Jan 4 20:35:23 2008 From: rhamph at gmail.com (Adam Olsen) Date: Fri, 4 Jan 2008 12:35:23 -0700 Subject: [Python-Dev] [Python-3000] Need closure on __cmp__ removal In-Reply-To: References: Message-ID: On Jan 4, 2008 12:18 PM, Guido van Rossum wrote: > In the past some folks have been pushing for the resurrection of (some > form of) __cmp__, which is currently removed from Py3k (except for > some remnants which we'll clean up in due time). > > I'd like to get closure on this issue. If someone volunteers within a > week to write a PEP, I'll give them a month to write the PEP, and then > I'll review it. The PEP better come with a patch implementing > (roughly) the desired behavior as well, relative to the 3.0 branch. > > If I don't hear from a committed volunteer within a week, I'll drop > this and start removing __cmp__ references aggressively (starting with > issue #1717). Saying "if no-one else volunteers, I can give it a shot" > is not sufficient commitment. Saying "I will give it a shot" is. If > someone commits but no PEP+patch is in my possession by February 4 > (and no attenuating circumstances have been brought to my attention), > I will assume the PEP won't happen and will start removing __cmp__ > references. Once a PEP and patch are presented, I'll review them and > make a decision. I can't speak for the others, but I know I've decided not to pursue it. -- Adam Olsen, aka Rhamphoryncus -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080104/f11e5fea/attachment.htm From guido at python.org Fri Jan 4 20:48:29 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 4 Jan 2008 11:48:29 -0800 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: <1f7befae0801041131v75af2a2dtb451391376c202fa@mail.gmail.com> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <5d44f72f0801040013o6fc8e455r254f430c415d14d8@mail.gmail.com> <18302.26041.596158.499906@montanaro.dyndns.org> <18302.27660.899384.319382@montanaro.dyndns.org> <1f7befae0801041131v75af2a2dtb451391376c202fa@mail.gmail.com> Message-ID: On Jan 4, 2008 11:31 AM, Tim Peters wrote: > [skip at pobox.com] > > Thanks for the pointer. Given that it's [round-to-even[ been an ASTM > > standard since 1940 and apparently in fairly common use since the > > early 1900s, I wonder why it's not been more widely used in the past > > in programming languages. > > Because "add a half and chop" was also in wide use for even longer, is > also (Wikipedia notwithstanding) part of many standards (for example, > the US IRS requires it if you do your taxes under the "round to whole > dollars" option), and-- probably the real driver --is a little cheaper > to implement for "normal sized" floats. Curiously, round-to-nearest > can be unboundedly more expensive to implement in some obscure > contexts when floats can have very large exponents (as they can in > Python's "decimal" module -- this is why the proposed decimal standard > allows operations like "remainder-near" to fail if applied to inputs > that are "too far apart": > > http://www2.hursley.ibm.com/decimal/daops.html#footnote.8 > > The "secret reason" is that it can be unboundedly more expensive to > determine the last bit of the quotient (to see whether it's even or > odd) than to determine an exact remainder). Wow. Do you have an opinion as to whether we should adopt round-to-even at all (as a default)? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From amk at amk.ca Fri Jan 4 20:50:23 2008 From: amk at amk.ca (A.M. Kuchling) Date: Fri, 4 Jan 2008 14:50:23 -0500 Subject: [Python-Dev] Repeatability of looping over dicts Message-ID: <20080104195023.GA29828@amk-desktop.matrixgroup.net> This post describes work aimed at getting Django to run on Jython: http://zyasoft.com/pythoneering/2008/01/django-on-jython-minding-gap.html One outstanding issue is whether to use Java's ConcurrentHashMap type to underly Jython's dict type. See . ConcurrentHashMap scales better in the face of threading because it doesn't lock the whole table when updating it, but iterating over the map can return elements in a different order each time. This would mean that list(dict_var) doesn't return values in the same order as a later call to list(dict_var), even if dict_var hasn't been modified. Why? Under the hood, there are 32 different locks, each guarding a subset of the hash buckets, so if there are multiple threads iterating over the dictionary, they may not go through the buckets in order. discusses the implementation, at least in 2003. So, do Python implementations need to guarantee that list(dict_var) == a later result from list(dict_var)? --amk From timothy.c.delaney at gmail.com Fri Jan 4 21:05:55 2008 From: timothy.c.delaney at gmail.com (Tim Delaney) Date: Sat, 5 Jan 2008 07:05:55 +1100 Subject: [Python-Dev] Repeatability of looping over dicts References: <20080104195023.GA29828@amk-desktop.matrixgroup.net> Message-ID: <00b001c84f0d$38eceec0$0301a8c0@ryoko> A.M. Kuchling wrote: > So, do Python implementations need to guarantee that list(dict_var) == > a later result from list(dict_var)? As I just posted to the blog, yes. Look at section 3.8 of the reference manual (Mapping Types), specifically footnote 3: http://docs.python.org/lib/typesmapping.html (3) Keys and values are listed in an arbitrary order which is non-random, varies across Python implementations, and depends on the dictionary's history of insertions and deletions. If items(), keys(), values(), iteritems(), iterkeys(), and itervalues() are called with no intervening modifications to the dictionary, the lists will directly correspond. This allows the creation of (value, key) pairs using zip(): "pairs = zip(a.values(), a.keys())". The same relationship holds for the iterkeys() and itervalues() methods: "pairs = zip(a.itervalues(), a.iterkeys())" provides the same value for pairs. Another way to create the same list is "pairs = [(v, k) for (k, v) in a.iteritems()]". Tim Delaney From timothy.c.delaney at gmail.com Fri Jan 4 21:05:55 2008 From: timothy.c.delaney at gmail.com (Tim Delaney) Date: Sat, 5 Jan 2008 07:05:55 +1100 Subject: [Python-Dev] Repeatability of looping over dicts References: <20080104195023.GA29828@amk-desktop.matrixgroup.net> Message-ID: <00b101c84f0d$3c367470$0301a8c0@ryoko> A.M. Kuchling wrote: > So, do Python implementations need to guarantee that list(dict_var) == > a later result from list(dict_var)? As I just posted to the blog, yes. Look at section 3.8 of the reference manual (Mapping Types), specifically footnote 3: http://docs.python.org/lib/typesmapping.html (3) Keys and values are listed in an arbitrary order which is non-random, varies across Python implementations, and depends on the dictionary's history of insertions and deletions. If items(), keys(), values(), iteritems(), iterkeys(), and itervalues() are called with no intervening modifications to the dictionary, the lists will directly correspond. This allows the creation of (value, key) pairs using zip(): "pairs = zip(a.values(), a.keys())". The same relationship holds for the iterkeys() and itervalues() methods: "pairs = zip(a.itervalues(), a.iterkeys())" provides the same value for pairs. Another way to create the same list is "pairs = [(v, k) for (k, v) in a.iteritems()]". Tim Delaney From fdrake at acm.org Fri Jan 4 21:11:52 2008 From: fdrake at acm.org (Fred Drake) Date: Fri, 4 Jan 2008 15:11:52 -0500 Subject: [Python-Dev] Repeatability of looping over dicts In-Reply-To: <00b101c84f0d$3c367470$0301a8c0@ryoko> References: <20080104195023.GA29828@amk-desktop.matrixgroup.net> <00b101c84f0d$3c367470$0301a8c0@ryoko> Message-ID: <73F50605-9172-4C35-B7E2-14859938CBBE@acm.org> On Jan 4, 2008, at 3:05 PM, Tim Delaney wrote: > As I just posted to the blog, yes. Look at section 3.8 of the > reference > manual > (Mapping Types), specifically footnote 3: You type faster than I do. :-) This guarantee has been in place for about a decade, as I recall. -Fred -- Fred Drake From amk at amk.ca Fri Jan 4 21:41:55 2008 From: amk at amk.ca (A.M. Kuchling) Date: Fri, 4 Jan 2008 15:41:55 -0500 Subject: [Python-Dev] Repeatability of looping over dicts In-Reply-To: <00b001c84f0d$38eceec0$0301a8c0@ryoko> References: <20080104195023.GA29828@amk-desktop.matrixgroup.net> <00b001c84f0d$38eceec0$0301a8c0@ryoko> Message-ID: <20080104204155.GA30997@amk-desktop.matrixgroup.net> On Sat, Jan 05, 2008 at 07:05:55AM +1100, Tim Delaney wrote: > >So, do Python implementations need to guarantee that list(dict_var) == > >a later result from list(dict_var)? > > As I just posted to the blog, yes. Look at section 3.8 of the reference > manual > (Mapping Types), specifically footnote 3: Ah, thanks! I guess that rules out using ConcurrentHashMap, short of materializing the entire list and sorting it in some way. Oh well. --amk From josepharmbruster at gmail.com Fri Jan 4 22:25:57 2008 From: josepharmbruster at gmail.com (Joseph Armbruster) Date: Fri, 4 Jan 2008 16:25:57 -0500 Subject: [Python-Dev] function call syntax oddity Message-ID: <938f42d70801041325t1c943c4cu47e23a6b2012c219@mail.gmail.com> All, I was showing a co-worker some python today (using the trunk) when I stumbled upon this. I was not sure what to think since I have never really experimented with using spaces like this. So here tis. Be careful to notice the spaces as they are significant here. >>> 1.3.__str__() '1.3' >>> 1.3 .__str__() '1.3' >>> a = " somestring " >>> a .split() ['somestring'] >>> a. split() ['somestring'] Cool I suppose, except here's an odd man out: >>> 1.__str__() File "", line 1 1.__str__() ^ SyntaxError: invalid syntax >>> 1 .__str__() '1' Joseph Armbruster -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080104/b40ae1b2/attachment.htm From p.f.moore at gmail.com Fri Jan 4 22:32:44 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 4 Jan 2008 21:32:44 +0000 Subject: [Python-Dev] function call syntax oddity In-Reply-To: <938f42d70801041325t1c943c4cu47e23a6b2012c219@mail.gmail.com> References: <938f42d70801041325t1c943c4cu47e23a6b2012c219@mail.gmail.com> Message-ID: <79990c6b0801041332k2bc77edas858dc05d897a71e1@mail.gmail.com> On 04/01/2008, Joseph Armbruster wrote: > Cool I suppose, except here's an odd man out: > > >>> 1.__str__() > File "", line 1 > 1.__str__() > ^ > SyntaxError: invalid syntax It's parsed a floating point number - "1." - followed by the keyword "__str__". That's not valid. > >>> 1 .__str__() > '1' This one is a number "1" followed by the operator "." followed by "__str__". The lexer reads the longest valid token each time. Paul. From python at rcn.com Fri Jan 4 23:35:06 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 4 Jan 2008 17:35:06 -0500 (EST) Subject: [Python-Dev] function call syntax oddity Message-ID: <20080104173506.AEN05688@ms19.lnh.mail.rcn.net> [Paul Moore] > 1 .__str__() > This one is a number "1" followed by > the operator "." followed by "__str__". FWIW, I would avoid the odd spacing and write this as: (1).__str__() Raymond From python at rcn.com Fri Jan 4 23:45:34 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 4 Jan 2008 17:45:34 -0500 (EST) Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 Message-ID: <20080104174534.AEN07247@ms19.lnh.mail.rcn.net> [GvR to Tim] > Do you have an opinion as to whether we should > adopt round-to-even at all (as a default)? For the sake of other implementations (Jython, etc) and for ease of reproducing the results with other tools (Excel, etc), the simplest choice is int(x+0.5). That works everywhere, it is easy to explain, it runs fast, and it is not hard to get right. my two cents, Raymond From guido at python.org Fri Jan 4 23:54:49 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 4 Jan 2008 14:54:49 -0800 Subject: [Python-Dev] Repeatability of looping over dicts In-Reply-To: <20080104195023.GA29828@amk-desktop.matrixgroup.net> References: <20080104195023.GA29828@amk-desktop.matrixgroup.net> Message-ID: On Jan 4, 2008 11:50 AM, A.M. Kuchling wrote: > This post describes work aimed at getting Django to run on Jython: > http://zyasoft.com/pythoneering/2008/01/django-on-jython-minding-gap.html > > One outstanding issue is whether to use Java's ConcurrentHashMap type > to underly Jython's dict type. See > . > > ConcurrentHashMap scales better in the face of threading because it > doesn't lock the whole table when updating it, but iterating over the > map can return elements in a different order each time. This would > mean that list(dict_var) doesn't return values in the same order as a > later call to list(dict_var), even if dict_var hasn't been modified. > > Why? Under the hood, there are 32 different locks, each guarding a > subset of the hash buckets, so if there are multiple threads iterating > over the dictionary, they may not go through the buckets in order. > discusses > the implementation, at least in 2003. > > So, do Python implementations need to guarantee that list(dict_var) == > a later result from list(dict_var)? What code would break if we loosened this restriction? I guess defining d.items() as zip(d.keys(), d.values()) would no longer fly, but does anyone actually depend on this? Just like we changed how we think about auto-closing files once Jython came along, I think this is at least worth considering. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Sat Jan 5 00:02:32 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 05 Jan 2008 00:02:32 +0100 Subject: [Python-Dev] Contributing to Python In-Reply-To: References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> <20080103194024.GA15580@caltech.edu> <20080103195144.GA13654@caltech.edu> Message-ID: <477EBB08.1060207@v.loewis.de> > Now thinking of how to produce this relationships, I think that I will > change my approach to the issues. I'll start to be more aggressive > when reviewing a patch or bug. Aggressive in the sense of > asking/commenting/proposing even if I don't get the full grasp of the > issue. This could lead to a better interaction on the mid/long term, > even if in the short it appears to increase the noise a little. I think this is exactly right. If you don't understand a certain aspect, don't hesitate to ask. Maybe the submitter doesn't understand either, and it was a flaw in the patch, or maybe there is a real issue, and the submitter forgot to comment it properly - it's always the submitter's fault if you don't understand :-) Regards, Martin From fdrake at acm.org Sat Jan 5 00:02:40 2008 From: fdrake at acm.org (Fred Drake) Date: Fri, 4 Jan 2008 18:02:40 -0500 Subject: [Python-Dev] Repeatability of looping over dicts In-Reply-To: References: <20080104195023.GA29828@amk-desktop.matrixgroup.net> Message-ID: <202753D0-BC75-443F-9D2C-690721950DBB@acm.org> On Jan 4, 2008, at 5:54 PM, Guido van Rossum wrote: > What code would break if we loosened this restriction? I guess > defining d.items() as zip(d.keys(), d.values()) would no longer fly, > but does anyone actually depend on this? I don't know what code would break today; this was initially added to the set of promises made by the dict methods a decode ago, but it was in response to a need in the software I was working on at the time (possibly Grail, for which 2.6/3.0 isn't an issue). That question should probably be addressed to a fairly wide audience (comp.lang.python) since the promise has been there for so long. -Fred -- Fred Drake From ijmorlan at cs.uwaterloo.ca Sat Jan 5 00:07:00 2008 From: ijmorlan at cs.uwaterloo.ca (Isaac Morland) Date: Fri, 4 Jan 2008 18:07:00 -0500 (EST) Subject: [Python-Dev] Extracting variables from string.Template objects In-Reply-To: <20080104155256.GA27695@panix.com> References: <20080104155256.GA27695@panix.com> Message-ID: On Fri, 4 Jan 2008, Aahz wrote: >> Also, on a related issue, does it make sense to scan the template >> string for invalid escape sequences in Template.__init__? For the >> applications I can imagine of string.Template, I would prefer to get >> an error upon creating the Template object rather than arbitrarily >> later when I try to .substitute with it. > > No, create an is_valid() method at best. I'm curious as to why. Is it to avoid changing the behaviour of existing code (i.e., backwards compatibility), or do you see a design problem with having the Template constructor reject invalid template strings? Isaac Morland CSCF Web Guru DC 2554C, x36650 WWW Software Specialist From amk at amk.ca Sat Jan 5 01:20:57 2008 From: amk at amk.ca (A.M. Kuchling) Date: Fri, 4 Jan 2008 19:20:57 -0500 Subject: [Python-Dev] Repeatability of looping over dicts In-Reply-To: References: <20080104195023.GA29828@amk-desktop.matrixgroup.net> Message-ID: <20080105002057.GA11525@amk.local> On Fri, Jan 04, 2008 at 02:54:49PM -0800, Guido van Rossum wrote: > What code would break if we loosened this restriction? I guess > defining d.items() as zip(d.keys(), d.values()) would no longer fly, > but does anyone actually depend on this? Just like we changed how we http://www.google.com/codesearch?hl=en&q=+lang:python+zip+keys&start=10&sa=N turns up a few pieces of code that would break: trac-0.10.3/trac/versioncontrol/cache.py Twisted-2.2.0/twisted/pb/test/test_banana.py Mattricks-0.7/Mattricks/MatchPrediction.py FlickrUploadr-1.0.0/src/xmltramp.py Projects trying to stay compatible with Python versions that don't have .items() may be more likely to use this idiom. Some classes may do this to implement .items(); openbabel-2.1.1/scripts/python/pybel.py does this. So some code will break, and I don't see a way to warn about this problem before breaking compatibility. --amk From python at rcn.com Sat Jan 5 01:30:26 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 4 Jan 2008 19:30:26 -0500 (EST) Subject: [Python-Dev] Repeatability of looping over dicts Message-ID: <20080104193026.AEN17860@ms19.lnh.mail.rcn.net> >> ConcurrentHashMap scales better in the face of threading .. . >> So, do Python implementations need to guarantee that list(dict_var) == >> a later result from list(dict_var)? > What code would break if we loosened this restriction? I can imagine someone has code like this: for k in d: print '%5s' % k for k in d: print '-----' for v in d.values(): print '%5s' % v It seems likely to me that a lot of code using d.values() would care about the order of the values and the only order that matters is corresponding to some set of keys. There are probably a lot of functions that take keys and values separately, so it would not be uncommon to call those with a pattern like: save_config(configfile, d.keys(), d.values()). In the OP's context where multiple threads are running, it may be fair to say that all bets are off. Raymond From glyph at divmod.com Sat Jan 5 07:16:09 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Sat, 05 Jan 2008 06:16:09 -0000 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: <20080104174534.AEN07247@ms19.lnh.mail.rcn.net> References: <20080104174534.AEN07247@ms19.lnh.mail.rcn.net> Message-ID: <20080105061609.21558.1204703501.divmod.xquotient.4318@joule.divmod.com> On 4 Jan, 10:45 pm, python at rcn.com wrote: >[GvR to Tim] >>Do you have an opinion as to whether we should >>adopt round-to-even at all (as a default)? > >For the sake of other implementations (Jython, etc) and for ease of >reproducing the results with other tools (Excel, etc), the simplest >choice is int(x+0.5). That works everywhere, it is easy to explain, it >runs fast, and it is not hard to get right. I agree for the default. Except the part where Excel, Jython, and Python's current round, actually use sign(x) * int(abs(x)+0.5), so maybe it's not *completely* easy to get right ;-). Having other rounding methods *available*, though, would be neat. The only application I've ever worked on where I cared about the difference, the user had to select it (since accounting requirements differ by jurisdiction and, apparently, by bank preference). Having a standard way to express this (especially if it worked across different numeric types, but perhaps I digress) would be pleasant. Implementing stochastic rounding and banker's rounding oneself, while not exactly hard, is a drag. From greg at krypto.org Sat Jan 5 07:17:35 2008 From: greg at krypto.org (Gregory P. Smith) Date: Fri, 4 Jan 2008 22:17:35 -0800 Subject: [Python-Dev] Repeatability of looping over dicts In-Reply-To: <20080104195023.GA29828@amk-desktop.matrixgroup.net> References: <20080104195023.GA29828@amk-desktop.matrixgroup.net> Message-ID: <52dc1c820801042217n1e96278fj2cfc2e55e9f265f2@mail.gmail.com> On 1/4/08, A.M. Kuchling wrote: > This post describes work aimed at getting Django to run on Jython: > http://zyasoft.com/pythoneering/2008/01/django-on-jython-minding-gap.html > > One outstanding issue is whether to use Java's ConcurrentHashMap type > to underly Jython's dict type. See > . > As a side note, you may also want to consider using a NonBlockingHashMap as implemented by Cliff Click instead of ConcurrentHashMap: http://blogs.azulsystems.com/cliff/2007/03/a_nonblocking_h.html http://blogs.azulsystems.com/cliff/2007/03/talking_with_go.html http://video.google.com/videoplay?docid=2139967204534450862 http://sourceforge.net/projects/high-scale-lib I haven't looked to see if it has the same issues repeatability issues but it does scale even better. Anyways, could you not use something with repeatability issues with a wrapper that caches lists of keys to satisfy the repeatability? (don't lock and maintain the list, just regenerate the list on demand when a caller needs it and discard the cached list anytime a key is added or deleted) Also, should we drop this guarantee in py3k to give future implementations more flexibility? -gps > ConcurrentHashMap scales better in the face of threading because it > doesn't lock the whole table when updating it, but iterating over the > map can return elements in a different order each time. This would > mean that list(dict_var) doesn't return values in the same order as a > later call to list(dict_var), even if dict_var hasn't been modified. > > Why? Under the hood, there are 32 different locks, each guarding a > subset of the hash buckets, so if there are multiple threads iterating > over the dictionary, they may not go through the buckets in order. > discusses > the implementation, at least in 2003. > > So, do Python implementations need to guarantee that list(dict_var) == > a later result from list(dict_var)? > > --amk From mike.klaas at gmail.com Sat Jan 5 08:45:34 2008 From: mike.klaas at gmail.com (Mike Klaas) Date: Fri, 4 Jan 2008 23:45:34 -0800 Subject: [Python-Dev] Contributing to Python In-Reply-To: References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> <2CAFFA1B-601E-4C27-90C0-6032438EF8F8@acm.org> Message-ID: <4F61EE50-A329-4A7C-8813-9CAE29704E9A@gmail.com> On 3-Jan-08, at 1:07 PM, Guido van Rossum wrote: > On Jan 3, 2008 11:49 AM, Fred Drake wrote: >> >> Python 2.6 seems to be entirely targeted at people who really want to >> be on Python 3, but have code that will need to be ported. I >> certainly don't view it as interesting in it's own right. > > It will be though -- it will have genuine new features -- yes, > backported from 3.0, but new features nevertheless, and in a > compatible fashion. I think that there are still tons of people like me for whom 3.0 is still a future concern that is too big to devote cycles to at the moment, but are still very much interested in improving the 2.x series (which improves 3.0) at the same time. I've been inspired by this thread to start working on a few 2.6 items that I had in mind, starting with http://bugs.python.org/ issue1663329 , which mostly just needed documentation and cleanup (now done). Question: should patches include edits to whatsnew.rst, or is the committer responsible for adding a note? -Mike From jyasskin at gmail.com Sat Jan 5 09:56:23 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Sat, 5 Jan 2008 00:56:23 -0800 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <5d44f72f0801040013o6fc8e455r254f430c415d14d8@mail.gmail.com> Message-ID: <5d44f72f0801050056i2a7c6086m22ef44915d68c212@mail.gmail.com> On Jan 4, 2008 8:50 AM, Guido van Rossum wrote: > On Jan 4, 2008 12:13 AM, Jeffrey Yasskin wrote: > > On Jan 3, 2008 10:37 AM, Guido van Rossum wrote: > > > > Well, as issue 1689 states, the backporting was commited by Jeffrey on > > > > rev 5967 [2], so this is the time to understand if we want this or > > > > not. > > > > > > This is a problem. Right now, in the trunk, math.float(1) returns 1, > > > where it should return 1.0 for compatibility with 2.5. Jeffrey, can > > > you fix this and similar incompatibilities you introduced? > > > > Whoops! I've committed r59707 to fix math.{floor,ceil}. Let me know if > > you find any other problems. ... Hmm. I've also changed the behavior > > of round(2.5). I'll change that back tomorrow morning. > > Looks like in Python 2.6, round(0.5) right now returns 0.0, whereas in > 2.5, it returns 1.0. > > I think it should return 1.0, for best compatibility with Python 2.5. > > > I haven't seen any answers to the original question. It looks like > > Decimal is decided by 2.5 too: return a float from everything. > > Right. > > > Rational, being a completely new type, is up to you guys, but because > > new support for the conversion routines seems to be rare, it's > > probably best to have it return floats too. > > I think the pre-3.0 rule should be: round(), math.floor(), math.ceil() > *always* return a float. These rollbacks, and that of pow(-1,.5)==1j are submitted as r59731. Keep letting me know what else I broke. -- Namast?, Jeffrey Yasskin From skip at pobox.com Sat Jan 5 15:58:22 2008 From: skip at pobox.com (skip at pobox.com) Date: Sat, 5 Jan 2008 08:58:22 -0600 Subject: [Python-Dev] Repeatability of looping over dicts In-Reply-To: References: <20080104195023.GA29828@amk-desktop.matrixgroup.net> Message-ID: <18303.39694.313826.221607@montanaro.dyndns.org> Guido> What code would break if we loosened this restriction? I don't know how much, but I do know I've relied on this behavior before. (In fact, I've asked about it before.) I guess the counter question to yours would be, "What would be gained by loosening this restriction"? If the answer is, "not much", then I don't see why this is even an idle thought. Skip From guido at python.org Sat Jan 5 17:54:44 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 5 Jan 2008 08:54:44 -0800 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: <20080105061609.21558.1204703501.divmod.xquotient.4318@joule.divmod.com> References: <20080104174534.AEN07247@ms19.lnh.mail.rcn.net> <20080105061609.21558.1204703501.divmod.xquotient.4318@joule.divmod.com> Message-ID: On Jan 4, 2008 10:16 PM, wrote: > On 4 Jan, 10:45 pm, python at rcn.com wrote: > >[GvR to Tim] > >>Do you have an opinion as to whether we should > >>adopt round-to-even at all (as a default)? > > > >For the sake of other implementations (Jython, etc) and for ease of > >reproducing the results with other tools (Excel, etc), the simplest > >choice is int(x+0.5). That works everywhere, it is easy to explain, it > >runs fast, and it is not hard to get right. > > I agree for the default. Except the part where Excel, Jython, and > Python's current round, actually use sign(x) * int(abs(x)+0.5), so maybe > it's not *completely* easy to get right ;-). > > Having other rounding methods *available*, though, would be neat. The > only application I've ever worked on where I cared about the difference, > the user had to select it (since accounting requirements differ by > jurisdiction and, apparently, by bank preference). Having a standard > way to express this (especially if it worked across different numeric > types, but perhaps I digress) would be pleasant. Implementing > stochastic rounding and banker's rounding oneself, while not exactly > hard, is a drag. The decimal module already supports rounding modes in its context. For other types, perhaps converting to decimal might be good enough? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Jan 5 17:56:02 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 5 Jan 2008 08:56:02 -0800 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: <5d44f72f0801050056i2a7c6086m22ef44915d68c212@mail.gmail.com> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <5d44f72f0801040013o6fc8e455r254f430c415d14d8@mail.gmail.com> <5d44f72f0801050056i2a7c6086m22ef44915d68c212@mail.gmail.com> Message-ID: On Jan 5, 2008 12:56 AM, Jeffrey Yasskin wrote: > > On Jan 4, 2008 8:50 AM, Guido van Rossum wrote: > > On Jan 4, 2008 12:13 AM, Jeffrey Yasskin wrote: > > > On Jan 3, 2008 10:37 AM, Guido van Rossum wrote: > > > > > Well, as issue 1689 states, the backporting was commited by Jeffrey on > > > > > rev 5967 [2], so this is the time to understand if we want this or > > > > > not. > > > > > > > > This is a problem. Right now, in the trunk, math.float(1) returns 1, > > > > where it should return 1.0 for compatibility with 2.5. Jeffrey, can > > > > you fix this and similar incompatibilities you introduced? > > > > > > Whoops! I've committed r59707 to fix math.{floor,ceil}. Let me know if > > > you find any other problems. ... Hmm. I've also changed the behavior > > > of round(2.5). I'll change that back tomorrow morning. > > > > Looks like in Python 2.6, round(0.5) right now returns 0.0, whereas in > > 2.5, it returns 1.0. > > > > I think it should return 1.0, for best compatibility with Python 2.5. > > > > > I haven't seen any answers to the original question. It looks like > > > Decimal is decided by 2.5 too: return a float from everything. > > > > Right. > > > > > Rational, being a completely new type, is up to you guys, but because > > > new support for the conversion routines seems to be rare, it's > > > probably best to have it return floats too. > > > > I think the pre-3.0 rule should be: round(), math.floor(), math.ceil() > > *always* return a float. > > These rollbacks, and that of pow(-1,.5)==1j are submitted as r59731. > Keep letting me know what else I broke. I think the consensus is against round-to-even in 3.0 -- this requires a PEP update as well as code changes. (Sorry for having caused so much extra work, I should have flagged this earlier.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Jan 5 18:00:38 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 5 Jan 2008 09:00:38 -0800 Subject: [Python-Dev] Repeatability of looping over dicts In-Reply-To: <18303.39694.313826.221607@montanaro.dyndns.org> References: <20080104195023.GA29828@amk-desktop.matrixgroup.net> <18303.39694.313826.221607@montanaro.dyndns.org> Message-ID: On Jan 5, 2008 6:58 AM, wrote: > > Guido> What code would break if we loosened this restriction? > > I don't know how much, but I do know I've relied on this behavior before. > (In fact, I've asked about it before.) I guess the counter question to > yours would be, "What would be gained by loosening this restriction"? If > the answer is, "not much", then I don't see why this is even an idle > thought. It sounds like loosening the restriction would allow Jython to use an implementation that is more efficient when used concurrently. That may not be sufficient reason; Jython apps that need a more efficient concurrent dict could import the ConcurrentHashMap class directly, and CPython apps are out of luck anyway. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From amk at amk.ca Sat Jan 5 18:22:23 2008 From: amk at amk.ca (A.M. Kuchling) Date: Sat, 5 Jan 2008 12:22:23 -0500 Subject: [Python-Dev] Contributing to Python In-Reply-To: <4F61EE50-A329-4A7C-8813-9CAE29704E9A@gmail.com> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> <2CAFFA1B-601E-4C27-90C0-6032438EF8F8@acm.org> <4F61EE50-A329-4A7C-8813-9CAE29704E9A@gmail.com> Message-ID: <20080105172223.GA12056@amk.local> On Fri, Jan 04, 2008 at 11:45:34PM -0800, Mike Klaas wrote: > Question: should patches include edits to whatsnew.rst, or is the > committer responsible for adding a note? It's OK to submit or commit patches that don't update whatsnew.rst; I'll notice the checkin and decide whether to include the change. --amk From techdir at emailbins.com Sat Jan 5 19:01:35 2008 From: techdir at emailbins.com (Art Rasmussen) Date: Sat, 05 Jan 2008 12:01:35 -0600 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 Message-ID: <20080105120135.s98yth1l44w8sgcw@webmail.opentransfer.com> Added Python to the referenced article (because I believe Python should be seen everywhere C#, PHP, Visual Basic, etc., are seen). Please let me know if the article needs updating/fixing. http://en.wikipedia.org/wiki/Rounding --- Art Rasmussen From jyasskin at gmail.com Sat Jan 5 19:56:49 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Sat, 5 Jan 2008 10:56:49 -0800 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <5d44f72f0801040013o6fc8e455r254f430c415d14d8@mail.gmail.com> <5d44f72f0801050056i2a7c6086m22ef44915d68c212@mail.gmail.com> Message-ID: <5d44f72f0801051056o5e343872ia9cbeff646d53a54@mail.gmail.com> On Jan 5, 2008 8:56 AM, Guido van Rossum wrote: > I think the consensus is against round-to-even in 3.0 -- this requires > a PEP update as well as code changes. (Sorry for having caused so much > extra work, I should have flagged this earlier.) I'm not convinced that speed is a real issue in this case, since this is only the explicit round() operation and not rounding for arithmetic operations. But consistency with other languages is important. Does the following patch to the PEP represent the consensus? If so, I'll check it in, and update the py3k branch and wikipedia article to match. I've allowed each type to define its own half-rounding behavior so that Decimal can follow the current context, and float can, once there's a portable way to set it (like C99's fesetround), follow the current rounding mode (if people want that at the time). Index: pep-3141.txt =================================================================== --- pep-3141.txt (revision 59739) +++ pep-3141.txt (working copy) @@ -206,7 +206,10 @@ """Rounds self to ndigits decimal places, defaulting to 0. If ndigits is omitted or None, returns an Integral, otherwise - returns a Real. Rounds half toward even. + returns a Real. Types may choose which direction to round half. For + example, float rounds half away from 0, and Decimal rounds it + according to the active context. + """ raise NotImplementedError @@ -428,14 +431,15 @@ least Integral ``>= x``. 4. ``__round__(self)``, called from ``round(x)``, which returns the - Integral closest to ``x``, rounding half toward even. There is also - a 2-argument version, ``__round__(self, other)``, called from - ``round(x, y)``, which should return a Real. + Integral closest to ``x``, rounding half as the type chooses. There + is also a 2-argument version, ``__round__(self, other)``, called + from ``round(x, y)``, which should return a Real. Because the ``int()`` conversion implemented by ``float`` (and by ``decimal.Decimal``) is equivalent to but less explicit than ``trunc()``, let's remove it. (Or, if that breaks too much, just add a -deprecation warning.) +deprecation warning.) In 2.6, ``math.floor``, ``math.ceil``, and +``round`` will continue to return floats. ``complex.__{divmod,mod,floordiv,int,float}__`` also go away. It would be nice to provide a nice error message to help confused porters, but -- Namast?, Jeffrey Yasskin From facundobatista at gmail.com Sat Jan 5 20:35:45 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Sat, 5 Jan 2008 17:35:45 -0200 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: <20080105120135.s98yth1l44w8sgcw@webmail.opentransfer.com> References: <20080105120135.s98yth1l44w8sgcw@webmail.opentransfer.com> Message-ID: 2008/1/5, Art Rasmussen : > Added Python to the referenced article (because I believe Python > should be seen everywhere C#, PHP, Visual Basic, etc., are seen). > Please let me know if the article needs updating/fixing. Well, don't know. It talks about the rounding in Python, but mentioning only the binary floating point. In Decimal you have a lot of different roundings available... it's worth to mention them? Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From jyasskin at gmail.com Sat Jan 5 20:40:44 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Sat, 5 Jan 2008 11:40:44 -0800 Subject: [Python-Dev] Repeatability of looping over dicts In-Reply-To: <00b001c84f0d$38eceec0$0301a8c0@ryoko> References: <20080104195023.GA29828@amk-desktop.matrixgroup.net> <00b001c84f0d$38eceec0$0301a8c0@ryoko> Message-ID: <5d44f72f0801051140g1a74557ey2c0187829afab466@mail.gmail.com> On Jan 4, 2008 12:05 PM, Tim Delaney wrote: > history of insertions and deletions. If items(), keys(), values(), > iteritems(), iterkeys(), and itervalues() are called with no intervening > modifications to the dictionary, the lists will directly correspond. I looked over the Java code briefly (http://gee.cs.oswego.edu/dl/classes/EDU/oswego/cs/dl/util/concurrent/ConcurrentHashMap.java:HashIterator) and I don't see anything that would cause it to violate this condition. In particular, the locks don't affect the order. It's a complicated class though, so I could have missed it. Do you see a reason for it to change its iteration order spontaneously? If another thread is concurrently modifying the dict, that's an intervening modification, and changing the iteration order is fine. There's the second question of whether using ConcurrentHashMap for python dicts is a good idea. I can't find any mention of thread-safety in the Jython docs, so I assume it follows Java's rules, which are that most variables must be locked in order to be accessed and modified concurrently. Making dict a ConcurrentHashMap would provide a stronger guarantee for some built-in types but not others, which is likely to confuse people. Further, not all synchronized maps can be replaced by ConcurrentHashMap because you may need groups of operations to be atomic, while CHM only provides atomicity across single method calls. I think a new ConcurrentDict class would probably be a better way to integrate ConcurrentHashMap. -- Namast?, Jeffrey Yasskin From phd at phd.pp.ru Sat Jan 5 20:58:11 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Sat, 5 Jan 2008 22:58:11 +0300 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: References: <20080105120135.s98yth1l44w8sgcw@webmail.opentransfer.com> Message-ID: <20080105195811.GB8463@phd.pp.ru> On Sat, Jan 05, 2008 at 05:35:45PM -0200, Facundo Batista wrote: > 2008/1/5, Art Rasmussen : > > Added Python to the referenced article (because I believe Python > > should be seen everywhere C#, PHP, Visual Basic, etc., are seen). > > Please let me know if the article needs updating/fixing. > > Well, don't know. > > It talks about the rounding in Python, but mentioning only the binary > floating point. In Decimal you have a lot of different roundings > available... it's worth to mention them? IMO it's worth to mention the existing of them, briefly. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From jquinn at cs.oberlin.edu Sat Jan 5 22:43:37 2008 From: jquinn at cs.oberlin.edu (Jameson "Chema" Quinn) Date: Sat, 5 Jan 2008 15:43:37 -0600 Subject: [Python-Dev] Extend reST spec to allow automatic recognition of identifiers in comments? In-Reply-To: References: Message-ID: This is a VERY VERY rough draft of a PEP. The idea is that there should be some formal way that reST parsers can differentiate (in docstrings) between variable/function names and identical English words, within comments. PEP: XXX Title: Catching unmarked identifiers in docstrings Version: 0.0.0.0.1 Last-Modified: 23-Aug-2007 Author: Jameson Quinn Status: Draft Type: Informational Content-Type: text/x-rst Created: 23-Aug-2007 Post-History: 30-Aug-2002 Abstract ======== This PEP makes explicit some additional ways to parse docstrings and comments for python identifiers. These are intended to be implementable on their own or as extensions to reST, and to make as many existing docstrings as possible usable by tools that change the visible representation of identifiers, such as translating (non-english) code editors or visual programming environments. Docstrings in widely-used modules are encouraged to use \`explicit backquotes\` to mark identifiers which are not caught by these cases. THIS IS AN EARLY DRAFT OF THIS PEP FOR DISCUSSION PURPOSES ONLY. ALL LOGIC IS INTENTIONALLY DEFINED ONLY BY EXAMPLES AND THERE IS NO REFERENCE IMPLEMENTATION UNTIL A THERE ARE AT LEAST GLIMMERINGS OF CONSENSUS ON THE RULE SET. Rationale ========= Python, like most computer languages, is based on English. This can represent a hurdle to those who do not speak English. Work is underway on Bityi_, a code viewer/editor which translates code to another language on load and save. Among the many design issues in Bityi is that of identifiers in docstrings. A view which translates the identifiers in code, but leaves the untranslated identifier in the docstrings, makes the docstrings worse than useless, even if the programmer has a rudimentary grasp of English. Yet if all identifiers in docstrings are translated, there is the problem of overtranslation in either direction. It is necessary to distinguish between the variable named "variable", which should be translated, and the comment that something is "highly variable", which should not. .. _Bityi: http://wiki.laptop.org/go/Bityi Note that this is just one use-case; syntax coloring and docstring hyperlinks are another one. This PEP is not the place for a discussion of all the pros and cons of a translating viewer. PEP 287 standardizes reST as an optional way to markup docstrings. This includes the possibility of using \`backquotes\` to flag Python identifiers. However, as this PEP is purely optional, there are many cases of identifiers in docstrings which are not flagged as such. Moreover, many of these unflagged cases could be caught programatically. This would reduce the task of making a module internationally-viewable, or hyperlinkable, considerably. This syntax is kept relatively open to allow for reuse with other programming languages. Common cases of identifiers in docstrings ========================================= The most common case is that of lists of argument or method names. We call these "identifier lists":: def register(func, *targs, **kargs): """register a function to be executed someday func - function to be called targs - optional arguments to pass kargs - optional keyword arguments to pass """ #func, targs, and kargs would be recognized as identifiers in the above. class MyClass(object): """Just a silly demonstration, with some methods: thisword : is a class method and you can call it - it may even return a value. As with reST, the associated text can have several paragraphs. BUT - you can't nest this construct, so BUT isn't counted. anothermethod: is another method. eventhis -- is counted as a method. anynumber --- of dashes are allowed in this syntax But consider: two words are NOT counted as an identifier. things(that,look,like,functions): are functions (see below) Also, the docstring may have explanatory text, below or by itself: so we have to deal with that. Thus, any paragraph which is NOT preceded by an empty line or another identifier list - like "itself" above - does not count as an identifier. """ #thisword, anothermethod, eventhis, anynumber, and things would be #recognized as identifiers in the above. Another case is things which look like functions, lists, indexes, or dicts:: """ afunction(is,a,word,with,parentheses) [a,list,is,a,bunch,of,words,in,brackets] anindex[is, like, a, cross, between, the, above] {adict:is,just:words,in:curly, brackets: likethis} """ #all of the above would be recogniszed as identifiers. The "syntax" of what goes inside these is very loose. identifier_list ::= [] { } , with no whitespace after initial_word, and where separator_symbol is the set of symbols ".,<>{}[]+-*^%=|/()[]{}" MINUS closing_symbol. content_word could maybe be a quoted string, too. In the "function name", no whitespace is allowed, but the symbols ".,*^=><-" are. Thus:: """ this.long=>function.*name(counts, and: so |do| these {so-called] arguments) {but,you - cant|use[nested]brackets{so,these,are.identifiers }but,these,arent} {heres.an.example.of."a string, no identifiers in here",but.out.here.yes } { even.one.pair.of.words.with.no symbols.means.nothing.here.is.an.identifier} Any of these structures that open on one line {but.close.on. the.next} are NOT counted as identifiers. """ #in the above: lines 1,2,and the parts of 3 outside the quotes #would be recognized as identifiers The above flexibility is intended to cover the various possibilities for argument lists in a fair subset of other languages. Languages which use only whitespace for argument separation are not covered by these rules. The final case is words that are in some_kind of mixedCase. These are only optionally counted as identifiers if they are also present as an identifier OUTSIDE the comments somewhere in the same file. Doctest and preformatted reST sections should be considered as 100% python code and treated as identifiers (or keywords). Recommended use =============== The rules above are designed to catch the large majority of identifiers already present in docstrings, while applying only extremely rarely to words that should properly be considered as natural language. However, they are inevitably imperfect. All docstrings of modules intended for wide use should manually fix all cases in which these rules fail. If the rules underapply, you can use either \`back quotes\` or parentheses() to mark words as identifiers; if they overapply and reformatting tricks don't fix the problem, Optional use inside comments or non-docstring strings ===================================================== Comments -------- Comments or blocks of comments alone on consecutive lines should be able, optionally, to use these same tricks to spotlight identifiers. Other strings ------------- I'm not sure yet what the rules should be here. One option I'm considering is to be able to turn on all the above logic with some evil hack such as '' 'a string like this, concatenated with an empty string'. Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080105/9bb61d35/attachment.htm From lists at cheimes.de Sat Jan 5 23:36:23 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 05 Jan 2008 23:36:23 +0100 Subject: [Python-Dev] Contributing to Python In-Reply-To: <4F61EE50-A329-4A7C-8813-9CAE29704E9A@gmail.com> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> <2CAFFA1B-601E-4C27-90C0-6032438EF8F8@acm.org> <4F61EE50-A329-4A7C-8813-9CAE29704E9A@gmail.com> Message-ID: Mike Klaas wrote: > Question: should patches include edits to whatsnew.rst, or is the > committer responsible for adding a note? A patch should contain edits for Misc/NEWS. Patches without documentation and NEWS updates costs the committer more time and reduces the likelihood of a commit. Even a perfect patch costs several minutes of our life time. The patch must be reviewed, applied, compiled, the entire unit test suite must pass and finally it must be committed and the issues needs to be closed, too. Christian From glyph at divmod.com Sat Jan 5 23:54:15 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Sat, 05 Jan 2008 22:54:15 -0000 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: References: <20080104174534.AEN07247@ms19.lnh.mail.rcn.net> <20080105061609.21558.1204703501.divmod.xquotient.4318@joule.divmod.com> Message-ID: <20080105225415.21558.671830484.divmod.xquotient.4351@joule.divmod.com> On 04:54 pm, guido at python.org wrote: >On Jan 4, 2008 10:16 PM, wrote: >>Having other rounding methods *available*, though, would be neat. The >>only application I've ever worked on where I cared about the >>difference, >>the user had to select it (since accounting requirements differ by >>jurisdiction and, apparently, by bank preference). Having a standard >>way to express this (especially if it worked across different numeric >>types, but perhaps I digress) would be pleasant. Implementing >>stochastic rounding and banker's rounding oneself, while not exactly >>hard, is a drag. > >The decimal module already supports rounding modes in its context. For >other types, perhaps converting to decimal might be good enough? Yes, that's the right thing to do. I had missed it. After all it is decimal rounding I want, and any financial applications I'm going to write these days are using decimals already for all the usual reasons. At first I didn't realize why I'd missed this feature. While the rounding *modes* are well documented, though, after 20 minutes of reading documentation I still haven't found a method or function that simply rounds a decimal to a given significant digit. Is there one, should there be one, or is the user simply meant to use Context.quantize with appropriate arguments? From guido at python.org Sun Jan 6 00:31:09 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 5 Jan 2008 15:31:09 -0800 Subject: [Python-Dev] Contributing to Python In-Reply-To: References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> <2CAFFA1B-601E-4C27-90C0-6032438EF8F8@acm.org> <4F61EE50-A329-4A7C-8813-9CAE29704E9A@gmail.com> Message-ID: On Jan 5, 2008 2:36 PM, Christian Heimes wrote: > Mike Klaas wrote: > > Question: should patches include edits to whatsnew.rst, or is the > > committer responsible for adding a note? > > A patch should contain edits for Misc/NEWS. Patches without > documentation and NEWS updates costs the committer more time and reduces > the likelihood of a commit. > > Even a perfect patch costs several minutes of our life time. The patch > must be reviewed, applied, compiled, the entire unit test suite must > pass and finally it must be committed and the issues needs to be closed, > too. Several minutes?! The average "perfect" patch costs me more like between half an hour and an hour. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From dickinsm at gmail.com Sun Jan 6 00:34:05 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Sat, 5 Jan 2008 18:34:05 -0500 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: <20080105225415.21558.671830484.divmod.xquotient.4351@joule.divmod.com> References: <20080104174534.AEN07247@ms19.lnh.mail.rcn.net> <20080105061609.21558.1204703501.divmod.xquotient.4318@joule.divmod.com> <20080105225415.21558.671830484.divmod.xquotient.4351@joule.divmod.com> Message-ID: <5c6f2a5d0801051534l8974dd5n8630a9ad46f95d34@mail.gmail.com> On Jan 5, 2008 5:54 PM, wrote: > At first I didn't realize why I'd missed this feature. While the > rounding *modes* are well documented, though, after 20 minutes of > reading documentation I still haven't found a method or function that > simply rounds a decimal to a given significant digit. Is there one, > should there be one, or is the user simply meant to use Context.quantize > with appropriate arguments? > quantize is about as close as it gets. Note that it's a Decimal method as well as a Context method, so you can invoke it directly on a given decimal: >>> Decimal("2.34567").quantize(Decimal("0.01")) Decimal("2.35") I've also occasionally felt a need for a simple rounding function that isn't affected by context. Would others be interested in such a function being added to Decimal? I guess there are two possibly useful operations: (1) round to a particular decimal place ( e.g. nearest ten, nearest hundredth, ..) and (2) to round to a particular number of significant digits; in both cases, the user should be able to specify the desired rounding mode. And for each operation, it might also be useful to specify whether the result should be padded with zeros to the desired length or not. (i.e. when rounding 3.399 to 3 significant places, should it produce 3.4 or 3.40?) Any thoughts? Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080105/773cbef5/attachment.htm From tim.peters at gmail.com Sun Jan 6 01:20:22 2008 From: tim.peters at gmail.com (Tim Peters) Date: Sat, 5 Jan 2008 19:20:22 -0500 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <5d44f72f0801040013o6fc8e455r254f430c415d14d8@mail.gmail.com> <18302.26041.596158.499906@montanaro.dyndns.org> <18302.27660.899384.319382@montanaro.dyndns.org> <1f7befae0801041131v75af2a2dtb451391376c202fa@mail.gmail.com> Message-ID: <1f7befae0801051620y6108d7frfa9f824187453460@mail.gmail.com> [Tim] >> Because "add a half and chop" was also in wide use for even longer, is >> also (Wikipedia notwithstanding) part of many standards (for example, >> the US IRS requires it if you do your taxes under the "round to whole >> dollars" option), and-- probably the real driver --is a little cheaper >> to implement for "normal sized" floats. Curiously, round-to-nearest >> can be unboundedly more expensive to implement in some obscure >> contexts when floats can have very large exponents (as they can in >> Python's "decimal" module -- this is why the proposed decimal standard >> allows operations like "remainder-near" to fail if applied to inputs >> that are "too far apart": >> >> http://www2.hursley.ibm.com/decimal/daops.html#footnote.8 >> >> The "secret reason" is that it can be unboundedly more expensive to >> determine the last bit of the quotient (to see whether it's even or >> odd) than to determine an exact remainder). [Guido] > Wow. Do you have an opinion as to whether we should adopt > round-to-even at all (as a default)? Yes: yes :-) There's no need to be unduly influenced by "some obscure contexts when floats can have very large exponents", and the decimal standard already weasels its way out of the bad consequences then. I should clarify that the standard "allows" relevant operations to fail then in the same sense the IRS "allows" you to pay your taxes: it's not just allowed, failure is required. Nearest/even is without doubt the rounding method experts want most often, which is half of what makes it the best default. The other half is that, while newbies don't understand why experts would want it, the underlying reasons nevertheless act to spare newbies from subtle numeric problems. As to what the numerically innocent /expect/, "(at least) all of the above" is my only guess. For example (and here I'm making up a very simple one to show the essence), under the Windows native Python "%.0f" % 2.5 produces "3", while under glibc-based implementations (including Cygwin's Python) it produces "2". Over the years I've seen "bug reports" filed against both outcomes. According to the 754 standard, the glibc-based result (nearest/even rounding) is correct, and the Microsoft result is wrong. Why fight it? All the HW float operations do nearest/even rounding now too (by default), ditto the decimal module, and I'm secretly grateful the people who decided on those were downright eager to oppose Excel's numerically innocent implementors ;-) From tim.peters at gmail.com Sun Jan 6 01:32:53 2008 From: tim.peters at gmail.com (Tim Peters) Date: Sat, 5 Jan 2008 19:32:53 -0500 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: <5c6f2a5d0801051534l8974dd5n8630a9ad46f95d34@mail.gmail.com> References: <20080104174534.AEN07247@ms19.lnh.mail.rcn.net> <20080105061609.21558.1204703501.divmod.xquotient.4318@joule.divmod.com> <20080105225415.21558.671830484.divmod.xquotient.4351@joule.divmod.com> <5c6f2a5d0801051534l8974dd5n8630a9ad46f95d34@mail.gmail.com> Message-ID: <1f7befae0801051632mc84aa98gdd14edc48cd4f7c6@mail.gmail.com> [Mark Dickinson] > quantize is about as close as it gets. Note that it's a Decimal method as > well as a Context method, so you can invoke it directly on a given decimal: > > > >>> Decimal("2.34567").quantize(Decimal("0.01")) > Decimal("2.35") This "reads better" in many cases if you define a constant first, like: PENNIES = Decimal("0.01") ... [lots of code] ... rounded = some_decimal.quantize(PENNIES) > I've also occasionally felt a need for a simple rounding function that isn't > affected by context. Would others be interested in such a function being > added to Decimal? I guess there are two possibly useful operations: (1) > round to a particular decimal place ( e.g. nearest ten, nearest hundredth, > ..) and (2) to round to a particular number of significant digits; in both > cases, the user should be able to specify the desired rounding mode. And > for each operation, it might also be useful to specify whether the result > should be padded with zeros to the desired length or not. ( i.e. when > rounding 3.399 to 3 significant places, should it produce 3.4 or 3.40?) > > Any thoughts? +1 from me. Like the 754 standard, the decimal std is trying to mandate a more-or-less minimal set of core functionality, with no concern for user interface. "Convenience functions" can be valuable additions in such cases, & I agree it's far from obvious to most how to accomplish rounding using the decimal facilities. I think it's obvious ;-) that rounding 3.399 to 3 sig. dig. should produce 3.40. From aahz at pythoncraft.com Sun Jan 6 02:04:09 2008 From: aahz at pythoncraft.com (Aahz) Date: Sat, 5 Jan 2008 17:04:09 -0800 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: <20080105225415.21558.671830484.divmod.xquotient.4351@joule.divmod.com> References: <20080104174534.AEN07247@ms19.lnh.mail.rcn.net> <20080105061609.21558.1204703501.divmod.xquotient.4318@joule.divmod.com> <20080105225415.21558.671830484.divmod.xquotient.4351@joule.divmod.com> Message-ID: <20080106010409.GA18018@panix.com> On Sat, Jan 05, 2008, glyph at divmod.com wrote: > > At first I didn't realize why I'd missed this feature. While the > rounding *modes* are well documented, though, after 20 minutes of > reading documentation I still haven't found a method or function > that simply rounds a decimal to a given significant digit. Is > there one, should there be one, or is the user simply meant to use > Context.quantize with appropriate arguments? Rephrasing Uncle Timmy: Decimal has so far focused on adhering to the decimal standard, not on providing convenience to users. As long as the core continues to adhere to the standard, there's no reason not to add convenience. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From daniel at stutzbachenterprises.com Sun Jan 6 02:22:20 2008 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Sat, 5 Jan 2008 19:22:20 -0600 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: <1f7befae0801041131v75af2a2dtb451391376c202fa@mail.gmail.com> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <5d44f72f0801040013o6fc8e455r254f430c415d14d8@mail.gmail.com> <18302.26041.596158.499906@montanaro.dyndns.org> <18302.27660.899384.319382@montanaro.dyndns.org> <1f7befae0801041131v75af2a2dtb451391376c202fa@mail.gmail.com> Message-ID: On Jan 4, 2008 1:31 PM, Tim Peters wrote: > Curiously, round-to-nearest > can be unboundedly more expensive to implement in some obscure > contexts when floats can have very large exponents (as they can in > Python's "decimal" module -- this is why the proposed decimal standard > allows operations like "remainder-near" to fail if applied to inputs > that are "too far apart": Just to be clear, this problem doesn't come up in round(), right? Because in round(), you just test the evenness of the last digit computed. There is never a need to compute extra digits just to perform the test. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From jyasskin at gmail.com Sun Jan 6 03:57:42 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Sat, 5 Jan 2008 18:57:42 -0800 Subject: [Python-Dev] Rounding Decimals Message-ID: <5d44f72f0801051857i3b13f1b0q8a9e485831dcd603@mail.gmail.com> On Jan 5, 2008 3:34 PM, Mark Dickinson wrote: > > On Jan 5, 2008 5:54 PM, wrote: > > > > At first I didn't realize why I'd missed this feature. While the > > rounding *modes* are well documented, though, after 20 minutes of > > reading documentation I still haven't found a method or function that > > simply rounds a decimal to a given significant digit. Is there one, > > should there be one, or is the user simply meant to use Context.quantize > > with appropriate arguments? > > > > > > > > > > quantize is about as close as it gets. Note that it's a Decimal method as > well as a Context method, so you can invoke it directly on a given decimal: > > > >>> Decimal("2.34567").quantize(Decimal("0.01")) > Decimal("2.35") > > > I've also occasionally felt a need for a simple rounding function that isn't > affected by context. Would others be interested in such a function being > added to Decimal? I guess there are two possibly useful operations: (1) > round to a particular decimal place ( e.g. nearest ten, nearest hundredth, > ..) and (2) to round to a particular number of significant digits; in both > cases, the user should be able to specify the desired rounding mode. And > for each operation, it might also be useful to specify whether the result > should be padded with zeros to the desired length or not. ( i.e. when > rounding 3.399 to 3 significant places, should it produce 3.4 or 3.40?) > > Any thoughts? I think pep 3141's round(x, ndigits) does (1). The only thing it doesn't support yet is specifying the rounding mode. Perhaps the pep should say that round() passes any extra named arguments on to the __round__() method so that users can specify a rounding mode for types that support it? The Real interface doesn't say anything about significant digits, so Decimal can do whatever we want. My first guess would be that round should remove significant digits but not add them. (i.e. round("3.199", 2) == "3.20" but round("3", 1) == "3".) As you know, I'm working on a patch to implement 3141 for Decimal at http://bugs.python.org/issue1623, which I'll update as soon as this thread comes to a conclusion. Other people who are interested in getting this right should add themselves to its nosy list so they can object before I check something dumb in. :) I currently plan to keep __round__ in 2.6's Decimal with 3.0's behavior because it no longer affects the round() builtin in 2.6. Users who want it can call it directly ? or we might provide, in 2.6, a module that provides 3.0 versions of the core library functions that change behavior so that they can get the new round() explicitly. -- Namast?, Jeffrey Yasskin From python at rcn.com Sun Jan 6 04:11:49 2008 From: python at rcn.com (Raymond Hettinger) Date: Sat, 5 Jan 2008 19:11:49 -0800 Subject: [Python-Dev] Rounding Decimals References: <5d44f72f0801051857i3b13f1b0q8a9e485831dcd603@mail.gmail.com> Message-ID: <003101c85011$e1572530$6800a8c0@RaymondLaptop1> > I think pep 3141's round(x, ndigits) does (1). The only thing it > doesn't support yet is specifying the rounding mode. Perhaps the pep > should say that round() passes any extra named arguments on to the > __round__() method so that users can specify a rounding mode for types > that support it? That would clutter the interface. Just specify a universal rounding mode for __round__ and have Decimal's implementation of that method comply. If someone wants more control than that, they can manipulate the decimal object directly. Raymond From tim.peters at gmail.com Sun Jan 6 06:12:51 2008 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 6 Jan 2008 00:12:51 -0500 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <5d44f72f0801040013o6fc8e455r254f430c415d14d8@mail.gmail.com> <18302.26041.596158.499906@montanaro.dyndns.org> <18302.27660.899384.319382@montanaro.dyndns.org> <1f7befae0801041131v75af2a2dtb451391376c202fa@mail.gmail.com> Message-ID: <1f7befae0801052112h3c04343bi2380769cd66d8e10@mail.gmail.com> [Tim] >> Curiously, round-to-nearest >> can be unboundedly more expensive to implement in some obscure >> contexts when floats can have very large exponents (as they can in >> Python's "decimal" module -- this is why the proposed decimal standard >> allows operations like "remainder-near" to fail if applied to inputs >> that are "too far apart": [Daniel Stutzbach] > Just to be clear, this problem doesn't come up in round(), right? Right! It's unique to 2-argument mod-like functions. > Because in round(), you just test the evenness of the last digit > computed. There is never a need to compute extra digits just to > perform the test. Right, round has to compute the last (retained) digit in any case. For mod(x, y) (for various definitions of mod), the result is x - n*y (for various ways of defining an integer n), and there exist efficient ways to determine the final result without learning anything about the value of n in the process. For example, consider Python's pow(10, 100000000, 136). It goes very fast to compute the answer 120, but internally Python never develops any idea about the value of n such that 10**100000000 - 136*n = 120. Is n even or odd? "remainder-near" can care, but there's no efficient way I know of to find out (dividing a 100-million digit integer by 136 to find out isn't efficient ;-)). From python at rcn.com Sun Jan 6 08:41:58 2008 From: python at rcn.com (Raymond Hettinger) Date: Sat, 5 Jan 2008 23:41:58 -0800 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 References: <20080104174534.AEN07247@ms19.lnh.mail.rcn.net><20080105061609.21558.1204703501.divmod.xquotient.4318@joule.divmod.com><20080105225415.21558.671830484.divmod.xquotient.4351@joule.divmod.com><5c6f2a5d0801051534l8974dd5n8630a9ad46f95d34@mail.gmail.com> <1f7befae0801051632mc84aa98gdd14edc48cd4f7c6@mail.gmail.com> Message-ID: <007401c85037$9e977ad0$6800a8c0@RaymondLaptop1> [Tim] > I agree it's far from obvious to most how > to accomplish rounding using the decimal facilities. FWIW, there is an entry for this in the Decimal FAQ: http://docs.python.org/lib/decimal-faq.html Raymond From g.brandl at gmx.net Sun Jan 6 09:37:16 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 06 Jan 2008 09:37:16 +0100 Subject: [Python-Dev] Contributing to Python In-Reply-To: References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> <2CAFFA1B-601E-4C27-90C0-6032438EF8F8@acm.org> <4F61EE50-A329-4A7C-8813-9CAE29704E9A@gmail.com> Message-ID: Christian Heimes schrieb: > Mike Klaas wrote: >> Question: should patches include edits to whatsnew.rst, or is the >> committer responsible for adding a note? > > A patch should contain edits for Misc/NEWS. And documentation changes should include proper "versionchanged" or "versionadded" tags if it's not just a bug fix. > Patches without > documentation and NEWS updates costs the committer more time and reduces > the likelihood of a commit. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From asmodai at in-nomine.org Sun Jan 6 09:55:27 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Sun, 6 Jan 2008 09:55:27 +0100 Subject: [Python-Dev] Extend reST spec to allow automatic recognition of identifiers in comments? In-Reply-To: References: Message-ID: <20080106085527.GX82115@nexus.in-nomine.org> -On [20080105 22:44], Jameson Chema Quinn (jquinn at cs.oberlin.edu) wrote: >The "syntax" of what goes inside these is very loose. >identifier_list ::= [] >{ } >, with no whitespace after initial_word, and where separator_symbol is the set >of symbols ".,<>{}[]+-*^%=|/()[]{}" MINUS closing_symbol. content_word could >maybe be a quoted string, too. >In the "function name", no whitespace >is allowed, but the symbols ".,*^=><-" are. Thus:: Given the fact Python 3k will use unicode natively for strings and, IIRC, UTF-8 as standard encoding for Python files, have you thought about the half-width and full-width characters like the ones you describe above? ?? are, for example, very common in Japanese where English would use quotes. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ Possession is nine points of the law... From jyasskin at gmail.com Sun Jan 6 10:21:11 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Sun, 6 Jan 2008 01:21:11 -0800 Subject: [Python-Dev] Rounding Decimals In-Reply-To: <003101c85011$e1572530$6800a8c0@RaymondLaptop1> References: <5d44f72f0801051857i3b13f1b0q8a9e485831dcd603@mail.gmail.com> <003101c85011$e1572530$6800a8c0@RaymondLaptop1> Message-ID: <5d44f72f0801060121l39c4d2a0jfdea69f644bc1822@mail.gmail.com> On Jan 5, 2008 7:11 PM, Raymond Hettinger wrote: > > I think pep 3141's round(x, ndigits) does (1). The only thing it > > doesn't support yet is specifying the rounding mode. Perhaps the pep > > should say that round() passes any extra named arguments on to the > > __round__() method so that users can specify a rounding mode for types > > that support it? > > That would clutter the interface. Just specify a universal rounding mode for __round__ and have Decimal's implementation of that > method comply. Yeah, thinking about this more, a .round() method on Context objects seems like a better way to handle the part of Mark's request that the round builtin doesn't cover. Of course, there may be even better alternatives that I haven't thought of. I'll post a patch to http://bugs.python.org/issue1623 tomorrow so we have something concrete to talk about. -- Namast?, Jeffrey Yasskin From jyasskin at gmail.com Sun Jan 6 10:50:42 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Sun, 6 Jan 2008 01:50:42 -0800 Subject: [Python-Dev] Return type of round, floor, and ceil in 2.6 In-Reply-To: <1f7befae0801051620y6108d7frfa9f824187453460@mail.gmail.com> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <5d44f72f0801040013o6fc8e455r254f430c415d14d8@mail.gmail.com> <18302.26041.596158.499906@montanaro.dyndns.org> <18302.27660.899384.319382@montanaro.dyndns.org> <1f7befae0801041131v75af2a2dtb451391376c202fa@mail.gmail.com> <1f7befae0801051620y6108d7frfa9f824187453460@mail.gmail.com> Message-ID: <5d44f72f0801060150o7b18f79dxaf2f6f473015d6e5@mail.gmail.com> On Jan 5, 2008 4:20 PM, Tim Peters wrote: > [Guido] > > Wow. Do you have an opinion as to whether we should adopt > > round-to-even at all (as a default)? > > Yes: yes :-) Thanks for the suggestion, Tim. Here's a new proposed patch to the pep. It still allows type-defined half-rounding behavior so that Decimal can follow the current context. I'll submit it tomorrow unless there are significant objections. Index: pep-3141.txt =================================================================== --- pep-3141.txt (revision 59739) +++ pep-3141.txt (working copy) @@ -205,8 +205,12 @@ def __round__(self, ndigits:Integral=None): """Rounds self to ndigits decimal places, defaulting to 0. - If ndigits is omitted or None, returns an Integral, otherwise - returns a Real. Rounds half toward even. + If ndigits is omitted or None, returns an Integral, + otherwise returns a Real, preferably of the same type as + self. Types may choose which direction to round half. For + example, float rounds half toward even, and Decimal rounds + it according to the current context. + """ raise NotImplementedError @@ -428,10 +432,14 @@ least Integral ``>= x``. 4. ``__round__(self)``, called from ``round(x)``, which returns the - Integral closest to ``x``, rounding half toward even. There is also - a 2-argument version, ``__round__(self, other)``, called from - ``round(x, y)``, which should return a Real. + Integral closest to ``x``, rounding half as the type chooses. + ``float`` will change in 3.0 to round half toward even. There is + also a 2-argument version, ``__round__(self, ndigits)``, called + from ``round(x, ndigits)``, which should return a Real. +In 2.6, ``math.floor``, ``math.ceil``, and ``round`` will continue to +return floats. + Because the ``int()`` conversion implemented by ``float`` (and by ``decimal.Decimal``) is equivalent to but less explicit than ``trunc()``, let's remove it. (Or, if that breaks too much, just add a -- Namast?, Jeffrey Yasskin From lists at cheimes.de Sun Jan 6 16:33:58 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 06 Jan 2008 16:33:58 +0100 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages Message-ID: <4780F4E6.6040208@cheimes.de> Hello! We are discussing name space packages on the stdlib reorg list. For Python 3.0 we plan to organize the packages by purpose, e.g. put all database related packages like sqlite and shelve in a 'databases' name space. Of course we want to have the name spaces extensible by 3rd party software. The Python world as currently two ways to create extensible name space packages: pkgutil and pkg_resource. pkgutil is part of Python 2.5. pkg_resource is scheduled for Python 2.6 and 3.0 in PEP 365 [1]. The PEP hasn't been accepted yet. Questions: * PJE: Is pkg_resource ready for Python 2.6 and 3.0? * GvR: Are you going to accept Phillip's PEP? * PJE: Does pkg_resource have an easy way to overwrite a package in a name space package? E.g. an user wants to overwrite Python's databases.sqlite with a newer version of sqlite. Can he simply do it by inserting a package into sys.path before Lib/ ? Christian [1] http://www.python.org/dev/peps/pep-0365/ From skip at pobox.com Sun Jan 6 17:57:28 2008 From: skip at pobox.com (skip at pobox.com) Date: Sun, 6 Jan 2008 10:57:28 -0600 Subject: [Python-Dev] Do we need both Trac and Roundup? Message-ID: <18305.2168.79863.686868@montanaro.dyndns.org> A ticket I opened on Roundup about a website issue some time ago was closed today. It was related to a different topic, but in the discussion I wrote: Is there some reason at this point that we need to maintain two separate trackers? A ton of work went into making our Roundup instance what the key players wanted. Why not use it to track website issues as well? With them separate you have two communities of maintainers and triage folks who are mostly disjoint. I assume there would be some trac-to-roundup conversion necessary. Perhaps something like that already exists. Do people think it would be worthwhile to merge the Trac tracker content (the issue tracker that holds tickets related to the python.org website) into the Roundup tracker (the issue tracker that holds tickets related to Python the language)? While they are nominally distinct activities, it seems to me that we must waste some precious resources (mostly people) maintaining two mostly disjoint trackers. There is also some functionality overlap, mostly in the documentation area. Skip From steven.bethard at gmail.com Sun Jan 6 19:00:47 2008 From: steven.bethard at gmail.com (Steven Bethard) Date: Sun, 6 Jan 2008 11:00:47 -0700 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <4780F4E6.6040208@cheimes.de> References: <4780F4E6.6040208@cheimes.de> Message-ID: On Jan 6, 2008 8:33 AM, Christian Heimes wrote: > * PJE: Does pkg_resource have an easy way to overwrite a package in a > name space package? E.g. an user wants to overwrite Python's > databases.sqlite with a newer version of sqlite. Can he simply do it by > inserting a package into sys.path before Lib/ ? Do we really want to encourage this? Wouldn't that just introduce more pyxml-like nightmares? I've been bitten way too many times by pyxml overwriting the regular xml package and causing version incompatibilities. I'd hate for this kind of thing to become common practice. Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From Scott.Daniels at Acm.Org Sun Jan 6 19:18:03 2008 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Sun, 06 Jan 2008 10:18:03 -0800 Subject: [Python-Dev] Contributing to Python In-Reply-To: References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> <2CAFFA1B-601E-4C27-90C0-6032438EF8F8@acm.org> <4F61EE50-A329-4A7C-8813-9CAE29704E9A@gmail.com> Message-ID: Guido van Rossum wrote: > On Jan 5, 2008 2:36 PM, Christian Heimes wrote: >> A patch should contain edits for Misc/NEWS. Patches without >> documentation and NEWS updates costs the committer more time >> and reduces the likelihood of a commit. >> >> Even a perfect patch costs several minutes of our life time. >> The patch must be reviewed, applied, compiled, the entire unit >> test suite must pass and finally it must be committed and the >> issues needs to be closed, too. > > Several minutes?! The average "perfect" patch costs me more like > between half an hour and an hour. > QOTW. I think this excerpt should show up in the new developer area. --Scott David Daniels Scott.Daniels at Acm.Org From Scott.Daniels at Acm.Org Sun Jan 6 19:37:30 2008 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Sun, 06 Jan 2008 10:37:30 -0800 Subject: [Python-Dev] Do we need both Trac and Roundup? In-Reply-To: <18305.2168.79863.686868@montanaro.dyndns.org> References: <18305.2168.79863.686868@montanaro.dyndns.org> Message-ID: skip at pobox.com wrote: > Do people think it would be worthwhile to merge the Trac tracker content > (the issue tracker that holds tickets related to the python.org website) > into the Roundup tracker (the issue tracker that holds tickets related to > Python the language)? While they are nominally distinct activities, it > seems to me that we must waste some precious resources (mostly people) > maintaining two mostly disjoint trackers. I think it would be a great idea. As I try to get back up on a new, "privacy=paranoid" machine, I went to mention that somewhere in the logging in process, there should be an indication of how much your box must be opened up before you can "log on," at least in the screen you get to when you log on. However, each time I went to enter such a note, I needed to "log on." I just gave up. -Scott David Daniels Scott.Daniels at Acm.Org From lists at cheimes.de Sun Jan 6 19:34:37 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 06 Jan 2008 19:34:37 +0100 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: References: <4780F4E6.6040208@cheimes.de> Message-ID: <47811F3D.3060403@cheimes.de> Steven Bethard wrote: > Do we really want to encourage this? Wouldn't that just introduce > more pyxml-like nightmares? I've been bitten way too many times by > pyxml overwriting the regular xml package and causing version > incompatibilities. I'd hate for this kind of thing to become common > practice. I like to give 3rd party software a chance to *extend* a name space package like xml rather then to overwrite it. As far as I understand your problem pyxml is overwriting the name space and claiming it for itself rather than extending it. Christian From lists at cheimes.de Sun Jan 6 19:34:37 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 06 Jan 2008 19:34:37 +0100 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: References: <4780F4E6.6040208@cheimes.de> Message-ID: <47811F3D.3060403@cheimes.de> Steven Bethard wrote: > Do we really want to encourage this? Wouldn't that just introduce > more pyxml-like nightmares? I've been bitten way too many times by > pyxml overwriting the regular xml package and causing version > incompatibilities. I'd hate for this kind of thing to become common > practice. I like to give 3rd party software a chance to *extend* a name space package like xml rather then to overwrite it. As far as I understand your problem pyxml is overwriting the name space and claiming it for itself rather than extending it. Christian From steven.bethard at gmail.com Sun Jan 6 20:03:31 2008 From: steven.bethard at gmail.com (Steven Bethard) Date: Sun, 6 Jan 2008 12:03:31 -0700 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <47811F3D.3060403@cheimes.de> References: <4780F4E6.6040208@cheimes.de> <47811F3D.3060403@cheimes.de> Message-ID: On Jan 6, 2008 11:34 AM, Christian Heimes wrote: > Steven Bethard wrote: > > Do we really want to encourage this? Wouldn't that just introduce > > more pyxml-like nightmares? I've been bitten way too many times by > > pyxml overwriting the regular xml package and causing version > > incompatibilities. I'd hate for this kind of thing to become common > > practice. > > I like to give 3rd party software a chance to *extend* a name space > package like xml rather then to overwrite it. As far as I understand > your problem pyxml is overwriting the name space and claiming it for > itself rather than extending it. The most recent problem was that pyxml installs a different version of pyexpat so that ``xml.parsers.pyexpat`` != ``pyexpat``. This causes problems with mod_python: http://www.dscpl.com.au/wiki/ModPython/Articles/ExpatCausingApacheCrash What concerned me was your comment: E.g. an user wants to overwrite Python's databases.sqlite with a newer version of sqlite Maybe the situation is different here, but having someone installing a different version of sqlite behind my back makes me nervous. Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From pje at telecommunity.com Sun Jan 6 20:23:27 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 06 Jan 2008 14:23:27 -0500 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <47811F3D.3060403@cheimes.de> References: <4780F4E6.6040208@cheimes.de> <47811F3D.3060403@cheimes.de> Message-ID: <20080106192311.B9D7E3A406C@sparrow.telecommunity.com> At 07:34 PM 1/6/2008 +0100, Christian Heimes wrote: >Steven Bethard wrote: > > Do we really want to encourage this? Wouldn't that just introduce > > more pyxml-like nightmares? I've been bitten way too many times by > > pyxml overwriting the regular xml package and causing version > > incompatibilities. I'd hate for this kind of thing to become common > > practice. > >I like to give 3rd party software a chance to *extend* a name space >package like xml rather then to overwrite it. As far as I understand >your problem pyxml is overwriting the name space and claiming it for >itself rather than extending it. Indeed. It should also be noted that namespace packages are actually a very mature technology at this point. Before setuptools, pkgutil already supported them, from the time of 2.3's release. From pje at telecommunity.com Sun Jan 6 20:32:50 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 06 Jan 2008 14:32:50 -0500 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <4780F4E6.6040208@cheimes.de> References: <4780F4E6.6040208@cheimes.de> Message-ID: <20080106193233.5BBD63A406C@sparrow.telecommunity.com> At 04:33 PM 1/6/2008 +0100, Christian Heimes wrote: >Hello! > >We are discussing name space packages on the stdlib reorg list. For >Python 3.0 we plan to organize the packages by purpose, e.g. put all >database related packages like sqlite and shelve in a 'databases' name >space. > >Of course we want to have the name spaces extensible by 3rd party >software. The Python world as currently two ways to create extensible >name space packages: pkgutil and pkg_resource. > >pkgutil is part of Python 2.5. pkg_resource is scheduled for Python 2.6 >and 3.0 in PEP 365 [1]. The PEP hasn't been accepted yet. > >Questions: > >* PJE: Is pkg_resource ready for Python 2.6 and 3.0? The extra feature proposed in PEP 365 isn't done yet. Without the PEP acceptance, I didn't feel the need to rush on finishing it. Apart from that, the pkg_resources module has been pretty darn stable. >* GvR: Are you going to accept Phillip's PEP? > >* PJE: Does pkg_resource have an easy way to overwrite a package in a >name space package? OverRIDE, yes; overWRITE, no. >E.g. an user wants to overwrite Python's >databases.sqlite with a newer version of sqlite. Can he simply do it by >inserting a package into sys.path before Lib/ ? As long as the 'databases' package hasn't been imported or had its namespace declared yet, yes. (I'm making the assumption, of course, that all of the namespace package requirements have been met: i.e., each package has a database/__init__.py that contains a namespace declaration and nothing else.) From p.f.moore at gmail.com Sun Jan 6 20:46:28 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 6 Jan 2008 19:46:28 +0000 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: References: <4780F4E6.6040208@cheimes.de> <47811F3D.3060403@cheimes.de> Message-ID: <79990c6b0801061146t16afc5eet4883ba4554262f23@mail.gmail.com> On 06/01/2008, Steven Bethard wrote: > What concerned me was your comment: > > E.g. an user wants to overwrite Python's > databases.sqlite with a newer version of sqlite > > Maybe the situation is different here, but having someone installing a > different version of sqlite behind my back makes me nervous. Yes, that concerned me. Whether possible or not, I see this as bad practice in general. On the other hand, allowing cx_Oracle to register itself as databases.cx_Oracle - which is the basic "namespace" functionality - is something I feel is important. (I have no idea if the cx_Oracle author would want to do this, but I think the option should be available). Paul. From pje at telecommunity.com Sun Jan 6 21:07:56 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 06 Jan 2008 15:07:56 -0500 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: References: <4780F4E6.6040208@cheimes.de> <47811F3D.3060403@cheimes.de> Message-ID: <20080106200740.014833A406C@sparrow.telecommunity.com> At 12:03 PM 1/6/2008 -0700, Steven Bethard wrote: >Maybe the situation is different here, but having someone installing a >different version of sqlite behind my back makes me nervous. Er, someone who? Behind whose back? I'm quite confused by what it is that's making you nervous. Do you worry about people bundling newer versions of say, the optparse module or wsgiref with their applications? If so, why? Or if not, what's different? From steven.bethard at gmail.com Sun Jan 6 23:01:57 2008 From: steven.bethard at gmail.com (Steven Bethard) Date: Sun, 6 Jan 2008 15:01:57 -0700 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <20080106200740.014833A406C@sparrow.telecommunity.com> References: <4780F4E6.6040208@cheimes.de> <47811F3D.3060403@cheimes.de> <20080106200740.014833A406C@sparrow.telecommunity.com> Message-ID: On Jan 6, 2008 1:07 PM, Phillip J. Eby wrote: > At 12:03 PM 1/6/2008 -0700, Steven Bethard wrote: > >Maybe the situation is different here, but having someone installing a > >different version of sqlite behind my back makes me nervous. > > Er, someone who? Behind whose back? I'm quite confused by what it > is that's making you nervous. > > Do you worry about people bundling newer versions of say, the > optparse module or wsgiref with their applications? If so, why? Or > if not, what's different? I worry about exactly the pyxml problem. Someone installs pyxml on my system, pyxml replaces xml.parsers.expat with a different version of pyexpat than the rest of Python, and then programs like mod_python crash because the two components were compiled against different versions of a common library. Here's the link again that I posted earlier: http://www.dscpl.com.au/wiki/ModPython/Articles/ExpatCausingApacheCrash Note that this all happens "behind my back" because I didn't know that pyxml would be replacing pyexpat in such a way that would cause this crash. In fact, I didn't even know that pyxml was installing pyexpat. Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From brett at python.org Sun Jan 6 23:02:12 2008 From: brett at python.org (Brett Cannon) Date: Sun, 6 Jan 2008 14:02:12 -0800 Subject: [Python-Dev] [Pydotorg] Do we need both Trac and Roundup? In-Reply-To: <18305.2168.79863.686868@montanaro.dyndns.org> References: <18305.2168.79863.686868@montanaro.dyndns.org> Message-ID: On Jan 6, 2008 8:57 AM, wrote: > > A ticket I opened on Roundup about a website issue some time ago was closed > today. It was related to a different topic, but in the discussion I wrote: > > Is there some reason at this point that we need to maintain two separate > trackers? A ton of work went into making our Roundup instance what the > key players wanted. Why not use it to track website issues as well? > With them separate you have two communities of maintainers and triage > folks who are mostly disjoint. > > I assume there would be some trac-to-roundup conversion necessary. > Perhaps something like that already exists. > > Do people think it would be worthwhile to merge the Trac tracker content > (the issue tracker that holds tickets related to the python.org website) > into the Roundup tracker (the issue tracker that holds tickets related to > Python the language)? While they are nominally distinct activities, it > seems to me that we must waste some precious resources (mostly people) > maintaining two mostly disjoint trackers. There is also some functionality > overlap, mostly in the documentation area. Yes, we should consolidate. But first we need more manpower to manage the tracker as it is now. Some time this month I will be sending out an announcement asking for more volunteers to help maintain the tracker. If we can get enough people to come forward to also help out we can include moving the web site tracker over as well. -Brett From brett at python.org Sun Jan 6 23:10:46 2008 From: brett at python.org (Brett Cannon) Date: Sun, 6 Jan 2008 14:10:46 -0800 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: References: <4780F4E6.6040208@cheimes.de> <47811F3D.3060403@cheimes.de> <20080106200740.014833A406C@sparrow.telecommunity.com> Message-ID: On Jan 6, 2008 2:01 PM, Steven Bethard wrote: > On Jan 6, 2008 1:07 PM, Phillip J. Eby wrote: > > At 12:03 PM 1/6/2008 -0700, Steven Bethard wrote: > > >Maybe the situation is different here, but having someone installing a > > >different version of sqlite behind my back makes me nervous. > > > > Er, someone who? Behind whose back? I'm quite confused by what it > > is that's making you nervous. > > > > Do you worry about people bundling newer versions of say, the > > optparse module or wsgiref with their applications? If so, why? Or > > if not, what's different? > > I worry about exactly the pyxml problem. Someone installs pyxml on my > system, pyxml replaces xml.parsers.expat with a different version of > pyexpat than the rest of Python, and then programs like mod_python > crash because the two components were compiled against different > versions of a common library. Here's the link again that I posted > earlier: > > http://www.dscpl.com.au/wiki/ModPython/Articles/ExpatCausingApacheCrash > > Note that this all happens "behind my back" because I didn't know that > pyxml would be replacing pyexpat in such a way that would cause this > crash. In fact, I didn't even know that pyxml was installing pyexpat. This is my worry as well (and why I was going to phrase the situation in a more "should this be considered" tone than the way Christian stated it. =) . But then again none of this is impossible even if we do leave out extending the namespace. Tweaking a package's __path__ value after the fact is not exactly hard. So even if we don't do this we are not preventing anyone from extending the namespace, just discouraging. My question becomes whether we want to allow something like this even if we explicitly state people should not use this mechanism to override pre-existing modules. Do we want people tossing stuff into the 'databases' package, or should the packages in the stdlib be considered sacred? I am leaning towards not, but that's because I personally like knowing that if I import something from a stdlib namespace it is from the stdlib. I don't want some bug report from a naive user of cx_Oracle ending up in the stdlib because the import came from the 'databases' package. And I would guess that if you don't consider this a stdlib thing but for any third-party package that other third-party packages would not love other packages mucking with their namespace. -Brett From p.f.moore at gmail.com Mon Jan 7 00:12:43 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 6 Jan 2008 23:12:43 +0000 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: References: <4780F4E6.6040208@cheimes.de> <47811F3D.3060403@cheimes.de> <20080106200740.014833A406C@sparrow.telecommunity.com> Message-ID: <79990c6b0801061512u544fd4adwd463352a86b74529@mail.gmail.com> On 06/01/2008, Brett Cannon wrote: > My question becomes whether we want to allow something like this even > if we explicitly state people should not use this mechanism to > override pre-existing modules. Do we want people tossing stuff into > the 'databases' package, or should the packages in the stdlib be > considered sacred? I am leaning towards not, but that's because I > personally like knowing that if I import something from a stdlib > namespace it is from the stdlib. I don't want some bug report from a > naive user of cx_Oracle ending up in the stdlib because the import > came from the 'databases' package. And I would guess that if you don't > consider this a stdlib thing but for any third-party package that > other third-party packages would not love other packages mucking with > their namespace. I see the point, but I'm against reserving generic package names like "databases" for the stdlib, unless 3rd parties can add modules in there. Another example might be "gui.tkinter" - why shouldn't "gui.wx" be allowed? If we want a "guaranteed-stdlib" package form, we should probably have a top-level package, "std" or whatever. That notion has, I believe, been shot down before (no time to look up references now). Paul. From jyasskin at gmail.com Mon Jan 7 00:26:28 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Sun, 6 Jan 2008 15:26:28 -0800 Subject: [Python-Dev] Rounding Decimals In-Reply-To: <5d44f72f0801060121l39c4d2a0jfdea69f644bc1822@mail.gmail.com> References: <5d44f72f0801051857i3b13f1b0q8a9e485831dcd603@mail.gmail.com> <003101c85011$e1572530$6800a8c0@RaymondLaptop1> <5d44f72f0801060121l39c4d2a0jfdea69f644bc1822@mail.gmail.com> Message-ID: <5d44f72f0801061526x788b0a46lbbc17ec5dde44ceb@mail.gmail.com> On Jan 6, 2008 1:21 AM, Jeffrey Yasskin wrote: > On Jan 5, 2008 7:11 PM, Raymond Hettinger wrote: > > > I think pep 3141's round(x, ndigits) does (1). The only thing it > > > doesn't support yet is specifying the rounding mode. Perhaps the pep > > > should say that round() passes any extra named arguments on to the > > > __round__() method so that users can specify a rounding mode for types > > > that support it? > > > > That would clutter the interface. Just specify a universal rounding mode for __round__ and have Decimal's implementation of that > > method comply. > > Yeah, thinking about this more, a .round() method on Context objects > seems like a better way to handle the part of Mark's request that the > round builtin doesn't cover. Of course, there may be even better > alternatives that I haven't thought of. I'll post a patch to > http://bugs.python.org/issue1623 tomorrow so we have something > concrete to talk about. The new patch is posted: http://bugs.python.org/file9080/decimal-3141.patch I'll implement Context.round() in a separate patch. Comment away. :) -- Namast?, Jeffrey Yasskin From phd at phd.pp.ru Mon Jan 7 00:28:38 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Mon, 7 Jan 2008 02:28:38 +0300 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <79990c6b0801061512u544fd4adwd463352a86b74529@mail.gmail.com> References: <4780F4E6.6040208@cheimes.de> <47811F3D.3060403@cheimes.de> <20080106200740.014833A406C@sparrow.telecommunity.com> <79990c6b0801061512u544fd4adwd463352a86b74529@mail.gmail.com> Message-ID: <20080106232838.GA24155@phd.pp.ru> On Sun, Jan 06, 2008 at 11:12:43PM +0000, Paul Moore wrote: > If we want a "guaranteed-stdlib" package form, we should probably have > a top-level package, "std" or whatever. py. > That notion has, I believe, > been shot down before (no time to look up references now). Mr Van Rossum has spoken against it many times. Now I think - if we don't want a separate Python's top-level namespace may be we should think about a separate top-level non-Python's (3rd parties') namespace? With it we could have database.sqlite (Python's sqlite) and user.database.sqlite (a newer version); and by doing import database.sqlite you know exactly what version you are importing. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From pje at telecommunity.com Mon Jan 7 00:31:33 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 06 Jan 2008 18:31:33 -0500 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: References: <4780F4E6.6040208@cheimes.de> <47811F3D.3060403@cheimes.de> <20080106200740.014833A406C@sparrow.telecommunity.com> Message-ID: <20080106233118.370253A406C@sparrow.telecommunity.com> At 03:01 PM 1/6/2008 -0700, Steven Bethard wrote: >Note that this all happens "behind my back" because I didn't know that >pyxml would be replacing pyexpat in such a way that would cause this >crash. In fact, I didn't even know that pyxml was installing pyexpat. Ah -- so this is 100% orthogonal to namespace packages, since this is something that can happen even without __path__ munging. Namespace packages don't actually make this any easier, either, so I don't see how this reflects on the proposal. Without a namespace package, packages earlier on sys.path *completely* override those that are installed later. With a namespace package, only the specific submodules/subpackages that exist can override the ones that appear later. IOW, without namespace packages, if you have two 'foo' packages, one containing 'bar' and the other both 'bar' and 'baz', then with namespace packages you'll always see a foo.bar and a foo.baz, with the contents of foo.bar depending on path order. *Without* namespace packages, the exact same thing is true of foo.bar, but foo.baz will also be either visible or invisible depending on the path order. In other words, the status quo actually has *more* variability of what happens. So, while it might be a good idea to advise people against replacing packages they don't "own" or at least making it prominently visible that a package replaces something in the stdlib, it doesn't (so far as I can tell) have anything to do with the merits of namespace packages one way or the ohter. Unless there is something else I'm missing? From jyasskin at gmail.com Mon Jan 7 00:35:06 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Sun, 6 Jan 2008 15:35:06 -0800 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <20080106232838.GA24155@phd.pp.ru> References: <4780F4E6.6040208@cheimes.de> <47811F3D.3060403@cheimes.de> <20080106200740.014833A406C@sparrow.telecommunity.com> <79990c6b0801061512u544fd4adwd463352a86b74529@mail.gmail.com> <20080106232838.GA24155@phd.pp.ru> Message-ID: <5d44f72f0801061535l7c02b7cbw59ae10dc0f87795@mail.gmail.com> On Jan 6, 2008 3:28 PM, Oleg Broytmann wrote: > Now I think - if we don't want a separate Python's top-level namespace > may be we should think about a separate top-level non-Python's (3rd > parties') namespace? With it we could have database.sqlite (Python's > sqlite) and user.database.sqlite (a newer version); and by doing import > database.sqlite you know exactly what version you are importing. Bleh. I'm +1 on allowing third-party additions to the same 'database' namespace that the stdlib puts packages in. People will just have to get used to the package, and not the namespace, determining who to complain to. In theory, it might make sense to allow libraries to "close" some namespaces to deal with Brett's worry, but I think the consenting adults rule says not to bother. -- Namast?, Jeffrey Yasskin From pje at telecommunity.com Mon Jan 7 00:41:00 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 06 Jan 2008 18:41:00 -0500 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: References: <4780F4E6.6040208@cheimes.de> <47811F3D.3060403@cheimes.de> <20080106200740.014833A406C@sparrow.telecommunity.com> Message-ID: <20080106234043.C0BE83A406C@sparrow.telecommunity.com> At 02:10 PM 1/6/2008 -0800, Brett Cannon wrote: >My question becomes whether we want to allow something like this even >if we explicitly state people should not use this mechanism to >override pre-existing modules. Do we want people tossing stuff into >the 'databases' package, or should the packages in the stdlib be >considered sacred? This is actually an excellent point, given that the actual intended use of namespace packages is to allow an *organization* to control a namespace: e.g. zope.* and zc.* packages, osaf.* packages, etc. Using names that have meaning (like "email" or "databases") sort of goes against the whole point of namespace packages in the first place. For some reason, I wasn't thinking about that when the original post came through. So, now that I've thought about it, I'm -1 on the stdlib using *top-level* namespace packages to sort out its contents (e.g. "databases" as a top-level package name) If we want to allow separately-distributed *upgrades* or bugfix releases of projects (such as an updated sqlite module), then using 2nd-level namespace packages like "std.databases.*" would allow that. Note, by the way, that this implies that somebody creating their own Oracle driver would *not* be expected to put it into std.databases. Again, the whole point of a namespace package is to reserve that namespace for packages produced by a particular organization, similar to the way e.g. the 'org.apache.projectname' packages in Java work. From guido at python.org Mon Jan 7 01:03:44 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 6 Jan 2008 16:03:44 -0800 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <5d44f72f0801061535l7c02b7cbw59ae10dc0f87795@mail.gmail.com> References: <4780F4E6.6040208@cheimes.de> <47811F3D.3060403@cheimes.de> <20080106200740.014833A406C@sparrow.telecommunity.com> <79990c6b0801061512u544fd4adwd463352a86b74529@mail.gmail.com> <20080106232838.GA24155@phd.pp.ru> <5d44f72f0801061535l7c02b7cbw59ae10dc0f87795@mail.gmail.com> Message-ID: On Jan 6, 2008 3:35 PM, Jeffrey Yasskin wrote: > On Jan 6, 2008 3:28 PM, Oleg Broytmann wrote: > > Now I think - if we don't want a separate Python's top-level namespace > > may be we should think about a separate top-level non-Python's (3rd > > parties') namespace? With it we could have database.sqlite (Python's > > sqlite) and user.database.sqlite (a newer version); and by doing import > > database.sqlite you know exactly what version you are importing. > > Bleh. > > I'm +1 on allowing third-party additions to the same 'database' > namespace that the stdlib puts packages in. People will just have to > get used to the package, and not the namespace, determining who to > complain to. > > In theory, it might make sense to allow libraries to "close" some > namespaces to deal with Brett's worry, but I think the consenting > adults rule says not to bother. There seems to be a misunderstanding. This is *not* going to happen for standard library package names. I'm fine with inventing mechanisms to allow 3rd party packages to beo cobbled together from multiple contributions (it would seem to make sense for e.g. Twisted or Zope). But I am absolutely squarely against 3rd party installs adding to (or worse, overriding) standard library package names. This is (to me) simple common sense, the reason being "knowing who to blame". We get enough bug reports in the Python tracker about 3rd party software as it is. (In general I'm not keen on attempts to create a single unified ontology for library modules. It's like expecting all doctors in town to live on the same street. But my feelings on keeping a clean demarcation between standard and 3rd party are much stronger.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From lists at cheimes.de Mon Jan 7 01:10:25 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 07 Jan 2008 01:10:25 +0100 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <20080106234043.C0BE83A406C@sparrow.telecommunity.com> References: <4780F4E6.6040208@cheimes.de> <47811F3D.3060403@cheimes.de> <20080106200740.014833A406C@sparrow.telecommunity.com> <20080106234043.C0BE83A406C@sparrow.telecommunity.com> Message-ID: <47816DF1.6040604@cheimes.de> Phillip J. Eby wrote: > This is actually an excellent point, given that the actual intended > use of namespace packages is to allow an *organization* to control a > namespace: e.g. zope.* and zc.* packages, osaf.* packages, > etc. Using names that have meaning (like "email" or "databases") > sort of goes against the whole point of namespace packages in the first place. > > For some reason, I wasn't thinking about that when the original post > came through. > > So, now that I've thought about it, I'm -1 on the stdlib using > *top-level* namespace packages to sort out its contents (e.g. > "databases" as a top-level package name) I don't have a strong opinion as PJE but tend to agree with his point of view. We must not reserve a complete set of top level names and close them for 3rd party software. *Either* we create subject area packages (e.g. databases, web) and open them for 3rd party software *or* we create a top level name space for Python's stdlib and reserve it for our own purpose. For the initial example of "databases.sqlite" I think py.db.sqlite has a nice ring to it. > If we want to allow separately-distributed *upgrades* or bugfix > releases of projects (such as an updated sqlite module), then using > 2nd-level namespace packages like "std.databases.*" would allow that. I like to see a possibility to easily upgrade, bugfix or extend a stdlib package with 3rd party extensions. PEP 297 (http://www.python.org/dev/peps/pep-0297/) contains some useful ideas. > Note, by the way, that this implies that somebody creating their own > Oracle driver would *not* be expected to put it into > std.databases. Again, the whole point of a namespace package is to > reserve that namespace for packages produced by a particular > organization, similar to the way e.g. the 'org.apache.projectname' > packages in Java work. The initial idea behind the new packages for Python 3.0 weren't really based on name space packages. It was more about grouping related modules by subject. Christian From lists at cheimes.de Mon Jan 7 01:10:25 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 07 Jan 2008 01:10:25 +0100 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <20080106234043.C0BE83A406C@sparrow.telecommunity.com> References: <4780F4E6.6040208@cheimes.de> <47811F3D.3060403@cheimes.de> <20080106200740.014833A406C@sparrow.telecommunity.com> <20080106234043.C0BE83A406C@sparrow.telecommunity.com> Message-ID: <47816DF1.6040604@cheimes.de> Phillip J. Eby wrote: > This is actually an excellent point, given that the actual intended > use of namespace packages is to allow an *organization* to control a > namespace: e.g. zope.* and zc.* packages, osaf.* packages, > etc. Using names that have meaning (like "email" or "databases") > sort of goes against the whole point of namespace packages in the first place. > > For some reason, I wasn't thinking about that when the original post > came through. > > So, now that I've thought about it, I'm -1 on the stdlib using > *top-level* namespace packages to sort out its contents (e.g. > "databases" as a top-level package name) I don't have a strong opinion as PJE but tend to agree with his point of view. We must not reserve a complete set of top level names and close them for 3rd party software. *Either* we create subject area packages (e.g. databases, web) and open them for 3rd party software *or* we create a top level name space for Python's stdlib and reserve it for our own purpose. For the initial example of "databases.sqlite" I think py.db.sqlite has a nice ring to it. > If we want to allow separately-distributed *upgrades* or bugfix > releases of projects (such as an updated sqlite module), then using > 2nd-level namespace packages like "std.databases.*" would allow that. I like to see a possibility to easily upgrade, bugfix or extend a stdlib package with 3rd party extensions. PEP 297 (http://www.python.org/dev/peps/pep-0297/) contains some useful ideas. > Note, by the way, that this implies that somebody creating their own > Oracle driver would *not* be expected to put it into > std.databases. Again, the whole point of a namespace package is to > reserve that namespace for packages produced by a particular > organization, similar to the way e.g. the 'org.apache.projectname' > packages in Java work. The initial idea behind the new packages for Python 3.0 weren't really based on name space packages. It was more about grouping related modules by subject. Christian From guido at python.org Mon Jan 7 01:23:59 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 6 Jan 2008 16:23:59 -0800 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <47816DF1.6040604@cheimes.de> References: <4780F4E6.6040208@cheimes.de> <47811F3D.3060403@cheimes.de> <20080106200740.014833A406C@sparrow.telecommunity.com> <20080106234043.C0BE83A406C@sparrow.telecommunity.com> <47816DF1.6040604@cheimes.de> Message-ID: On Jan 6, 2008 4:10 PM, Christian Heimes wrote: > Phillip J. Eby wrote: > > This is actually an excellent point, given that the actual intended > > use of namespace packages is to allow an *organization* to control a > > namespace: e.g. zope.* and zc.* packages, osaf.* packages, > > etc. Using names that have meaning (like "email" or "databases") > > sort of goes against the whole point of namespace packages in the first place. > > > > For some reason, I wasn't thinking about that when the original post > > came through. > > > > So, now that I've thought about it, I'm -1 on the stdlib using > > *top-level* namespace packages to sort out its contents (e.g. > > "databases" as a top-level package name) > > I don't have a strong opinion as PJE but tend to agree with his point of > view. We must not reserve a complete set of top level names and close > them for 3rd party software. Why not? 3rd party software can copy the same hierarchy under its own toplevel package, e.g. zope.db, zope.net, etc. Regarding using common words, either the stdlib grabs these, or *nobody* gets to use them (for fear of conflicting with some other 3rd party package grabbing the same). > *Either* we create subject area packages (e.g. databases, web) and open > them for 3rd party software *or* we create a top level name space for > Python's stdlib and reserve it for our own purpose. > > For the initial example of "databases.sqlite" I think py.db.sqlite has a > nice ring to it. Java's example notwithstanding, I don't want "py" or "python" to be part of the stdlib package namespace. *If* (part of) the stdlib has to live under a single distinguished name, pick something like "std" or "core". When I'm using Python I already know I'm using Python, I don't want to be reminded of that fact on every import line. > > If we want to allow separately-distributed *upgrades* or bugfix > > releases of projects (such as an updated sqlite module), then using > > 2nd-level namespace packages like "std.databases.*" would allow that. > > I like to see a possibility to easily upgrade, bugfix or extend a stdlib > package with 3rd party extensions. PEP 297 > (http://www.python.org/dev/peps/pep-0297/) contains some useful ideas. I see mostly possibilities for abuse here. Linux vendors can install bugfixes or using their own packaging system. > > Note, by the way, that this implies that somebody creating their own > > Oracle driver would *not* be expected to put it into > > std.databases. Again, the whole point of a namespace package is to > > reserve that namespace for packages produced by a particular > > organization, similar to the way e.g. the 'org.apache.projectname' > > packages in Java work. Right. > The initial idea behind the new packages for Python 3.0 weren't really > based on name space packages. It was more about grouping related modules > by subject. See my previous post. This is only mildly useful. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Mon Jan 7 02:13:21 2008 From: python at rcn.com (Raymond Hettinger) Date: Sun, 6 Jan 2008 17:13:21 -0800 Subject: [Python-Dev] Rounding Decimals References: <5d44f72f0801051857i3b13f1b0q8a9e485831dcd603@mail.gmail.com> <003101c85011$e1572530$6800a8c0@RaymondLaptop1> <5d44f72f0801060121l39c4d2a0jfdea69f644bc1822@mail.gmail.com> <5d44f72f0801061526x788b0a46lbbc17ec5dde44ceb@mail.gmail.com> Message-ID: <007601c850ca$7f2ab3e0$6800a8c0@RaymondLaptop1> FWIW, I don't think all of these patches are helpful. The implementations are so easy and the effects are so obvious, that it is simply better to decide what to do first, then do it afterwards. My own preference is to leave the decimal module alone (except for a __round__ method to be called by builtin.round). Adding more methods to that module does not make it easier to use -- there are already a mind-numbing range of choices in the current, minimal implementation of the spec. I would support having a separate module of convenience functions or a separate module that uses a subset, but growth of the core module will only make it harder to use and more difficult to maintain. Another idea for simplifcation is to write-up a good tutorial to the basics of using the module. Please show restraint when proposing API changes to decimal -- I think it would be easy to do more harm than good. Raymond ----- Original Message ----- From: "Jeffrey Yasskin" To: "Raymond Hettinger" Cc: "Mark Dickinson" ; "Python 3000" ; Sent: Sunday, January 06, 2008 3:26 PM Subject: Re: [Python-Dev] Rounding Decimals > On Jan 6, 2008 1:21 AM, Jeffrey Yasskin wrote: >> On Jan 5, 2008 7:11 PM, Raymond Hettinger wrote: >> > > I think pep 3141's round(x, ndigits) does (1). The only thing it >> > > doesn't support yet is specifying the rounding mode. Perhaps the pep >> > > should say that round() passes any extra named arguments on to the >> > > __round__() method so that users can specify a rounding mode for types >> > > that support it? >> > >> > That would clutter the interface. Just specify a universal rounding mode for __round__ and have Decimal's implementation of >> > that >> > method comply. >> >> Yeah, thinking about this more, a .round() method on Context objects >> seems like a better way to handle the part of Mark's request that the >> round builtin doesn't cover. Of course, there may be even better >> alternatives that I haven't thought of. I'll post a patch to >> http://bugs.python.org/issue1623 tomorrow so we have something >> concrete to talk about. > > The new patch is posted: http://bugs.python.org/file9080/decimal-3141.patch > I'll implement Context.round() in a separate patch. Comment away. :) > > -- > Namast?, > Jeffrey Yasskin > From jyasskin at gmail.com Mon Jan 7 03:03:01 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Sun, 6 Jan 2008 18:03:01 -0800 Subject: [Python-Dev] Rounding Decimals In-Reply-To: <007601c850ca$7f2ab3e0$6800a8c0@RaymondLaptop1> References: <5d44f72f0801051857i3b13f1b0q8a9e485831dcd603@mail.gmail.com> <003101c85011$e1572530$6800a8c0@RaymondLaptop1> <5d44f72f0801060121l39c4d2a0jfdea69f644bc1822@mail.gmail.com> <5d44f72f0801061526x788b0a46lbbc17ec5dde44ceb@mail.gmail.com> <007601c850ca$7f2ab3e0$6800a8c0@RaymondLaptop1> Message-ID: <5d44f72f0801061803m2fc7ca45gc6a08f415d5a4901@mail.gmail.com> On Jan 6, 2008 5:13 PM, Raymond Hettinger wrote: > FWIW, I don't think all of these patches are helpful. The implementations are so easy and the effects are so obvious, that it is > simply better to decide what to do first, then do it afterwards. > > My own preference is to leave the decimal module alone (except for a __round__ method to be called by builtin.round). You must mean __trunc__ to support builtin.trunc(), since builtin.round no longer calls __round__ in 2.6. The other 3 methods specified by PEP 3141 aren't strictly necessary for 2.6, but they will be needed for 3.0. I'd rather not make the two versions of Decimal gratuitously different, so this patch puts them in the 2.6 version too. That's basically all of the changes in this patch, so I'm not sure exactly what you're objecting to. Could you be more precise? -- Namast?, Jeffrey Yasskin From reed at reedobrien.com Mon Jan 7 03:07:07 2008 From: reed at reedobrien.com (Reed O'Brien) Date: Sun, 6 Jan 2008 21:07:07 -0500 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <20080106232838.GA24155@phd.pp.ru> References: <4780F4E6.6040208@cheimes.de> <47811F3D.3060403@cheimes.de> <20080106200740.014833A406C@sparrow.telecommunity.com> <79990c6b0801061512u544fd4adwd463352a86b74529@mail.gmail.com> <20080106232838.GA24155@phd.pp.ru> Message-ID: On Jan 6, 2008, at 6:28 PM, Oleg Broytmann wrote: > On Sun, Jan 06, 2008 at 11:12:43PM +0000, Paul Moore wrote: >> If we want a "guaranteed-stdlib" package form, we should probably >> have >> a top-level package, "std" or whatever. > > py. > >> That notion has, I believe, >> been shot down before (no time to look up references now). > > Mr Van Rossum has spoken against it many times. > > Now I think - if we don't want a separate Python's top-level > namespace > may be we should think about a separate top-level non-Python's (3rd > parties') namespace? I think some things make sense to move some things into a common namespace: hashlib, email, xml et al... I also think there are probably other candidates for similar grouping and cleanup, but I don't have a well thought out set. (db, url??, zip??); and I think it is happening, albeit slowly. > With it we could have database.sqlite (Python's > sqlite) and user.database.sqlite (a newer version); and by doing > import > database.sqlite you know exactly what version you are importing. At first blush I am +1 for a third party or non stdlib namespace; user is already a module though. Other ideas: sitepkg, addon, extra, ext But then what of packages that are already namespaced? It would be tedious to; from sitepkg.zope.app.interface import SomeThing So while I like the idea, I think I am against wedging everything into namespaces just for the sake of it; at the end of the day I think I am -1 on *requiring* it. Additionally, I have only been reading this list for a week or so and feel a little like I may be injecting a novice and poorly formed opinion into a discussion that has been ongoing for years. Please pardon my ignorance if this is the case. ~ro > Oleg. > -- > Oleg Broytmann http://phd.pp.ru/ > phd at phd.pp.ru > Programmers don't die, they just GOSUB without RETURN. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/reed% > 40reedobrien.com From pje at telecommunity.com Mon Jan 7 04:23:10 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 06 Jan 2008 22:23:10 -0500 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: References: <4780F4E6.6040208@cheimes.de> <47811F3D.3060403@cheimes.de> <20080106200740.014833A406C@sparrow.telecommunity.com> <20080106234043.C0BE83A406C@sparrow.telecommunity.com> <47816DF1.6040604@cheimes.de> Message-ID: <20080107032255.B7EFB3A406C@sparrow.telecommunity.com> At 04:23 PM 1/6/2008 -0800, Guido van Rossum wrote: >Regarding using common words, either the stdlib grabs these, or >*nobody* gets to use them (for fear of conflicting with some other 3rd >party package grabbing the same). This isn't quite true; a standalone Python application that isn't extensible doesn't need to worry about this. And it's standalone apps that are most likely to claim these common words. (For example, until recently, Chandler's database library packages were in 'repository.*'.) But of course this is still a pretty minor point overall. If the stdlib does go for deeper nestings, I have a slight preference for seeing them under std.* or some such rather than top level. But I don't really see a whole lot of point to doing a major re-org of the stdlib space to begin with, for the simple reason that package names are not really categories -- they're *names*. IMO 'from databases import sqlite' doesn't add any value over 'import pysqlite3' to begin with. Worse, it will likely be an attractive nuisance for people saying "why don't we have databases.Oracle?" and suchlike. And you still have to remember the names, only now they're longer. And was it database or databases? Greater uniqueness of names is just another reason flat is better than nested. :) From python at rcn.com Mon Jan 7 04:40:31 2008 From: python at rcn.com (Raymond Hettinger) Date: Sun, 6 Jan 2008 19:40:31 -0800 Subject: [Python-Dev] Rounding Decimals References: <5d44f72f0801051857i3b13f1b0q8a9e485831dcd603@mail.gmail.com> <003101c85011$e1572530$6800a8c0@RaymondLaptop1> <5d44f72f0801060121l39c4d2a0jfdea69f644bc1822@mail.gmail.com> <5d44f72f0801061526x788b0a46lbbc17ec5dde44ceb@mail.gmail.com> <007601c850ca$7f2ab3e0$6800a8c0@RaymondLaptop1> <5d44f72f0801061803m2fc7ca45gc6a08f415d5a4901@mail.gmail.com> Message-ID: <00a801c850df$0e953e10$6800a8c0@RaymondLaptop1> [Jeffrey Yasskin] > The other 3 methods > specified by PEP 3141 aren't strictly necessary for 2.6, but they will > be needed for 3.0. I'd rather not make the two versions of Decimal > gratuitously different, so this patch puts them in the 2.6 version > too. If I understand you correctly, then the patch backports magic methods that do not have corresponding invocation methods in Py2.6. So, they are basically useless. If that's true, then the patch is clutter -- it makes 2.6 less desirable. It is not obvious to me how this will help someone transition to Py3.0. I'm curious to hear how Guido makes the call on this. Also, the goal of keeping versions identical across 2.6 and 3.0 is at odds with previous discussions where I believe we said that that is *not* the goal and will likely not even be possible in many cases. Besides, if the invocation of the magic methods is different in 3.0, how are you going to keep the test suite code the same across versions? There should probably be a PEP sets clearer guidelines about what should be backported from Py3.0. Perhaps something like this: * If there is a new feature that can be implemented in both and will make both more attractive, then it should be in both. * If something is going away or changing in 3.0, then the 2.6 conversion tool mode should warn about it if possible. * If neither of the above apply, then leave 2.6 alone. > I'm not > sure exactly what you're objecting to. Could you be more precise? You note said: "I'll implement Context.round() in a separate patch. Comment away." Unless I misread what you're trying to do, that is a gratuitous API build-out (whether talking about Py2.6 or Py3.0) which adds a second-way-to-do-it to a module that already has a huge number of methods. Our goal is to keep that module as a minimal implementation of the spec. Fattening the API will make the module harder to use, not simpler. Raymond From guido at python.org Mon Jan 7 05:28:01 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 6 Jan 2008 20:28:01 -0800 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <20080107032255.B7EFB3A406C@sparrow.telecommunity.com> References: <4780F4E6.6040208@cheimes.de> <47811F3D.3060403@cheimes.de> <20080106200740.014833A406C@sparrow.telecommunity.com> <20080106234043.C0BE83A406C@sparrow.telecommunity.com> <47816DF1.6040604@cheimes.de> <20080107032255.B7EFB3A406C@sparrow.telecommunity.com> Message-ID: On Jan 6, 2008 7:23 PM, Phillip J. Eby wrote: > At 04:23 PM 1/6/2008 -0800, Guido van Rossum wrote: > >Regarding using common words, either the stdlib grabs these, or > >*nobody* gets to use them (for fear of conflicting with some other 3rd > >party package grabbing the same). > > This isn't quite true; a standalone Python application that isn't > extensible doesn't need to worry about this. And it's standalone > apps that are most likely to claim these common words. (For example, > until recently, Chandler's database library packages were in 'repository.*'.) > > But of course this is still a pretty minor point overall. If the > stdlib does go for deeper nestings, I have a slight preference for > seeing them under std.* or some such rather than top level. But I > don't really see a whole lot of point to doing a major re-org of the > stdlib space to begin with, for the simple reason that package names > are not really categories -- they're *names*. IMO 'from databases > import sqlite' doesn't add any value over 'import pysqlite3' to begin with. > > Worse, it will likely be an attractive nuisance for people saying > "why don't we have databases.Oracle?" and suchlike. And you still > have to remember the names, only now they're longer. And was it > database or databases? Greater uniqueness of names is just another > reason flat is better than nested. :) Right. Packages are useful if it helps make the sub-names shorter. The email package is a good example: it uses lots of generic names like errors, charset, encoders, message, parser, utils, iterators. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Jan 7 05:31:36 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 6 Jan 2008 20:31:36 -0800 Subject: [Python-Dev] [Python-3000] Rounding Decimals In-Reply-To: <00a801c850df$0e953e10$6800a8c0@RaymondLaptop1> References: <5d44f72f0801051857i3b13f1b0q8a9e485831dcd603@mail.gmail.com> <003101c85011$e1572530$6800a8c0@RaymondLaptop1> <5d44f72f0801060121l39c4d2a0jfdea69f644bc1822@mail.gmail.com> <5d44f72f0801061526x788b0a46lbbc17ec5dde44ceb@mail.gmail.com> <007601c850ca$7f2ab3e0$6800a8c0@RaymondLaptop1> <5d44f72f0801061803m2fc7ca45gc6a08f415d5a4901@mail.gmail.com> <00a801c850df$0e953e10$6800a8c0@RaymondLaptop1> Message-ID: On Jan 6, 2008 7:40 PM, Raymond Hettinger wrote: > [Jeffrey Yasskin] > > The other 3 methods > > specified by PEP 3141 aren't strictly necessary for 2.6, but they will > > be needed for 3.0. I'd rather not make the two versions of Decimal > > gratuitously different, so this patch puts them in the 2.6 version > > too. > > If I understand you correctly, then the patch backports magic methods that do not have corresponding invocation methods in Py2.6. > So, they are basically useless. If that's true, then the patch is clutter -- it makes 2.6 less desirable. It is not obvious to me > how this will help someone transition to Py3.0. I'm curious to hear how Guido makes the call on this. > > Also, the goal of keeping versions identical across 2.6 and 3.0 is at odds with previous discussions where I believe we said that > that is *not* the goal and will likely not even be possible in many cases. Besides, if the invocation of the magic methods is > different in 3.0, how are you going to keep the test suite code the same across versions? I tend to agree. 2.6 and 3.0 decimal already differ plenty. > There should probably be a PEP sets clearer guidelines about what should be backported from Py3.0. Can you volunteer a draft? It could be PEP 3003. I'm probably too close to 3.0 to be able to objectively write this, which really represents end users' desire for stability in 2.6. > Perhaps something like this: > * If there is a new feature that can be implemented in both and will make both more attractive, then it should be in both. > * If something is going away or changing in 3.0, then the 2.6 conversion tool mode should warn about it if possible. > * If neither of the above apply, then leave 2.6 alone. > > > I'm not > > sure exactly what you're objecting to. Could you be more precise? > > You note said: "I'll implement Context.round() in a separate patch. Comment away." > > Unless I misread what you're trying to do, that is a gratuitous API build-out (whether talking about Py2.6 or Py3.0) which adds a > second-way-to-do-it to a module that already has a huge number of methods. Our goal is to keep that module as a minimal > implementation of the spec. Fattening the API will make the module harder to use, not simpler. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Mon Jan 7 06:08:21 2008 From: python at rcn.com (Raymond Hettinger) Date: Sun, 6 Jan 2008 21:08:21 -0800 Subject: [Python-Dev] [Python-3000] Rounding Decimals References: <5d44f72f0801051857i3b13f1b0q8a9e485831dcd603@mail.gmail.com> <003101c85011$e1572530$6800a8c0@RaymondLaptop1> <5d44f72f0801060121l39c4d2a0jfdea69f644bc1822@mail.gmail.com> <5d44f72f0801061526x788b0a46lbbc17ec5dde44ceb@mail.gmail.com> <007601c850ca$7f2ab3e0$6800a8c0@RaymondLaptop1> <5d44f72f0801061803m2fc7ca45gc6a08f415d5a4901@mail.gmail.com> <00a801c850df$0e953e10$6800a8c0@RaymondLaptop1> Message-ID: <00bf01c850eb$53b84120$6800a8c0@RaymondLaptop1> [Raymond] >> There should probably be a PEP sets clearer guidelines about what should be backported from Py3.0. >> >> Perhaps something like this: >> * If there is a new feature that can be implemented in both and will make both more attractive, then it should be in both. >> * If something is going away or changing in 3.0, then the 2.6 conversion tool mode should warn about it if possible. >> * If neither of the above apply, then leave 2.6 alone. [Guido van Rossum] > Can you volunteer a draft? It could be PEP 3003. I'm probably too > close to 3.0 to be able to objectively write this, which really > represents end users' desire for stability in 2.6. Yes. Will do. Raymond From jyasskin at gmail.com Mon Jan 7 06:54:30 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Sun, 6 Jan 2008 21:54:30 -0800 Subject: [Python-Dev] Rounding Decimals In-Reply-To: <00a801c850df$0e953e10$6800a8c0@RaymondLaptop1> References: <5d44f72f0801051857i3b13f1b0q8a9e485831dcd603@mail.gmail.com> <003101c85011$e1572530$6800a8c0@RaymondLaptop1> <5d44f72f0801060121l39c4d2a0jfdea69f644bc1822@mail.gmail.com> <5d44f72f0801061526x788b0a46lbbc17ec5dde44ceb@mail.gmail.com> <007601c850ca$7f2ab3e0$6800a8c0@RaymondLaptop1> <5d44f72f0801061803m2fc7ca45gc6a08f415d5a4901@mail.gmail.com> <00a801c850df$0e953e10$6800a8c0@RaymondLaptop1> Message-ID: <5d44f72f0801062154n4a1dab9fu60abb3bb4b0d02dc@mail.gmail.com> On Jan 6, 2008 7:40 PM, Raymond Hettinger wrote: > [Jeffrey Yasskin] > > The other 3 methods > > specified by PEP 3141 aren't strictly necessary for 2.6, but they will > > be needed for 3.0. I'd rather not make the two versions of Decimal > > gratuitously different, so this patch puts them in the 2.6 version > > too. > > If I understand you correctly, then the patch backports magic methods that do not have corresponding invocation methods in Py2.6. > So, they are basically useless. If that's true, then the patch is clutter -- it makes 2.6 less desirable. It is not obvious to me > how this will help someone transition to Py3.0. I'm curious to hear how Guido makes the call on this. > > Also, the goal of keeping versions identical across 2.6 and 3.0 is at odds with previous discussions where I believe we said that > that is *not* the goal and will likely not even be possible in many cases. Besides, if the invocation of the magic methods is > different in 3.0, how are you going to keep the test suite code the same across versions? Given Guido's agreement, expect another version of this patch with only __trunc__. > There should probably be a PEP sets clearer guidelines about what should be backported from Py3.0. Perhaps something like this: > * If there is a new feature that can be implemented in both and will make both more attractive, then it should be in both. > * If something is going away or changing in 3.0, then the 2.6 conversion tool mode should warn about it if possible. > * If neither of the above apply, then leave 2.6 alone. > > > I'm not > > sure exactly what you're objecting to. Could you be more precise? > > You note said: "I'll implement Context.round() in a separate patch. Comment away." Oh, sorry for not being clear then. I don't intend to write or discuss that separate patch until this one's approved and submitted. I think it's worth discussing eventually, but this change is more important. I mentioned this sentiment at http://bugs.python.org/msg59417 too, but clearly wasn't explicit enough in either case. -- Namast?, Jeffrey Yasskin From python at rcn.com Mon Jan 7 07:51:32 2008 From: python at rcn.com (Raymond Hettinger) Date: Sun, 6 Jan 2008 22:51:32 -0800 Subject: [Python-Dev] Rounding Decimals References: <5d44f72f0801051857i3b13f1b0q8a9e485831dcd603@mail.gmail.com> <003101c85011$e1572530$6800a8c0@RaymondLaptop1> <5d44f72f0801060121l39c4d2a0jfdea69f644bc1822@mail.gmail.com> <5d44f72f0801061526x788b0a46lbbc17ec5dde44ceb@mail.gmail.com> <007601c850ca$7f2ab3e0$6800a8c0@RaymondLaptop1> <5d44f72f0801061803m2fc7ca45gc6a08f415d5a4901@mail.gmail.com> <00a801c850df$0e953e10$6800a8c0@RaymondLaptop1> <5d44f72f0801062154n4a1dab9fu60abb3bb4b0d02dc@mail.gmail.com> Message-ID: <00d001c850f9$bd67e2c0$6800a8c0@RaymondLaptop1> [Jeffrey Yasskin] >> > I'm not >> > sure exactly what you're objecting to. Could you be more precise? >> >> You note said: "I'll implement Context.round() in a separate patch. Comment away." > > Oh, sorry for not being clear then. I don't intend to write or discuss > that separate patch until this one's approved and submitted. I think > it's worth discussing eventually, but this change is more important. I > mentioned this sentiment at http://bugs.python.org/msg59417 too, but > clearly wasn't explicit enough in either case. It's hard to keep up with all of these patches and I'm growing concerned that some will sneak by. In the case of complex proposals, it is often helpful to have a patch that we can discuss concretely, but in the case of simple ones like this, they just clutter to patch tracker. There's no need to create the patch until the idea has been discussed. In the case of Context.round(), I've already advised against it. So, I'm not sure why you still intend write a separate patch and bring it back up for discussion. Ideally, it should die right now. The sentiment is unchanged -- please do not build-out the API for the decimal module. It is intended to be minimal and needs to stay that way. Further additions will only make it harder to use, harder to maintain, and will complicate efforts to rewrite the module in C. Raymond From python at rcn.com Mon Jan 7 08:28:57 2008 From: python at rcn.com (Raymond Hettinger) Date: Sun, 6 Jan 2008 23:28:57 -0800 Subject: [Python-Dev] Rounding Decimals References: <5d44f72f0801051857i3b13f1b0q8a9e485831dcd603@mail.gmail.com> <003101c85011$e1572530$6800a8c0@RaymondLaptop1> <5d44f72f0801060121l39c4d2a0jfdea69f644bc1822@mail.gmail.com> <5d44f72f0801061526x788b0a46lbbc17ec5dde44ceb@mail.gmail.com> <007601c850ca$7f2ab3e0$6800a8c0@RaymondLaptop1> <5d44f72f0801061803m2fc7ca45gc6a08f415d5a4901@mail.gmail.com> <00a801c850df$0e953e10$6800a8c0@RaymondLaptop1> <5d44f72f0801062154n4a1dab9fu60abb3bb4b0d02dc@mail.gmail.com> Message-ID: <00ea01c850fe$f98aaf30$6800a8c0@RaymondLaptop1> [Jeffrey Yasskin] > Given Guido's agreement, expect another version of this patch with > only __trunc__. Why is __trunc__ being backported? Is a trunc() builtin being backported? What is the point for a synonym for int() and __int__ in Py2.6. Unless I'm missing something, this doesn't improve Py2.6 in the least. Raymond From jyasskin at gmail.com Mon Jan 7 10:08:26 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Mon, 7 Jan 2008 01:08:26 -0800 Subject: [Python-Dev] Rounding Decimals In-Reply-To: <00d001c850f9$bd67e2c0$6800a8c0@RaymondLaptop1> References: <5d44f72f0801051857i3b13f1b0q8a9e485831dcd603@mail.gmail.com> <003101c85011$e1572530$6800a8c0@RaymondLaptop1> <5d44f72f0801060121l39c4d2a0jfdea69f644bc1822@mail.gmail.com> <5d44f72f0801061526x788b0a46lbbc17ec5dde44ceb@mail.gmail.com> <007601c850ca$7f2ab3e0$6800a8c0@RaymondLaptop1> <5d44f72f0801061803m2fc7ca45gc6a08f415d5a4901@mail.gmail.com> <00a801c850df$0e953e10$6800a8c0@RaymondLaptop1> <5d44f72f0801062154n4a1dab9fu60abb3bb4b0d02dc@mail.gmail.com> <00d001c850f9$bd67e2c0$6800a8c0@RaymondLaptop1> Message-ID: <5d44f72f0801070108j896bb34td7fd82daf696d4d0@mail.gmail.com> On Jan 6, 2008 10:51 PM, Raymond Hettinger wrote: > [Jeffrey Yasskin] > >> > I'm not > >> > sure exactly what you're objecting to. Could you be more precise? > >> > >> You note said: "I'll implement Context.round() in a separate patch. Comment away." > > > > Oh, sorry for not being clear then. I don't intend to write or discuss > > that separate patch until this one's approved and submitted. I think > > it's worth discussing eventually, but this change is more important. I > > mentioned this sentiment at http://bugs.python.org/msg59417 too, but > > clearly wasn't explicit enough in either case. > > It's hard to keep up with all of these patches and I'm growing concerned > that some will sneak by. You're certainly right to be concerned. The most significant patch, the original backport of 3141, _did_ sneak through. The only email I've gotten about it has been in response to my question about a detail. I don't know how I could have made it more obvious though: nobody who suggested the backport (http://bugs.python.org/issue1623) seemed to think it would be contentious enough to suggest emailing this list first. Since then, I've tried to keep things transparent by suggesting that you add yourself to the nosy list of http://bugs.python.org/issue1623, which is where I'll post any patches before thinking about committing them. > In the case of complex proposals, it is often > helpful to have a patch that we can discuss concretely, but in the case of > simple ones like this, they just clutter to patch tracker. There's no need > to create the patch until the idea has been discussed. I always like to have a patch around because abstract discussions, even (especially?) on simple topics, have a tendency to run off into the weeds. A patch keeps things focused and moving forward. > In the case of Context.round(), I've already advised against it. > So, I'm not sure why you still intend write a separate patch > and bring it back up for discussion. Ideally, it should die right now. You advised against it. Tim, Mark, and glyph advised for something like it. I think it's consistent with the fact that most operations on Decimal instances are mirrored on Context instances, and __round__ will be an operation on Decimal instances. That doesn't sound like enough agreement to simply drop the discussion. > The sentiment is unchanged -- please do not build-out the API for the > decimal module. It is intended to be minimal and needs to stay that > way. Further additions will only make it harder to use, harder to > maintain, and will complicate efforts to rewrite the module in C. I am not building out the decimal API. I am adding enough methods to maintain the comment that "Decimal floating point objects share many properties with the other builtin numeric types such as float and int. All of the usual math operations and special methods apply." in both 2.6 and 3.0. After that's done, I am interested in talking about adding one method that 3 people on this list have been interested in. I understand that you need to be vigilant against extraneous methods, but I'm really not just adding these on a whim. On Jan 6, 2008 11:28 PM, Raymond Hettinger wrote: > [Jeffrey Yasskin] > > Given Guido's agreement, expect another version of this patch with > > only __trunc__. > > Why is __trunc__ being backported? Is a trunc() builtin being backported? What is the point for a synonym for int() and __int__ in > Py2.6. > > Unless I'm missing something, this doesn't improve Py2.6 in the least. The trunc() builtin was backported in http://svn.python.org/view?rev=59671&view=rev and hasn't been rolled back. All of the reasons to include trunc() in 3.0 apply to 2.6. A couple that I like are: if trunc had been around in the beginning, __index__ would have been unnecessary; and the name says what it's doing, unlike int(). I don't know what Guido's reasoning was for accepting it into the PEP. -- Namast?, Jeffrey Yasskin From python at rcn.com Mon Jan 7 10:36:41 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 7 Jan 2008 01:36:41 -0800 Subject: [Python-Dev] Rounding Decimals References: <5d44f72f0801051857i3b13f1b0q8a9e485831dcd603@mail.gmail.com> <003101c85011$e1572530$6800a8c0@RaymondLaptop1> <5d44f72f0801060121l39c4d2a0jfdea69f644bc1822@mail.gmail.com> <5d44f72f0801061526x788b0a46lbbc17ec5dde44ceb@mail.gmail.com> <007601c850ca$7f2ab3e0$6800a8c0@RaymondLaptop1> <5d44f72f0801061803m2fc7ca45gc6a08f415d5a4901@mail.gmail.com> <00a801c850df$0e953e10$6800a8c0@RaymondLaptop1> <5d44f72f0801062154n4a1dab9fu60abb3bb4b0d02dc@mail.gmail.com> <00d001c850f9$bd67e2c0$6800a8c0@RaymondLaptop1> <5d44f72f0801070108j896bb34td7fd82daf696d4d0@mail.gmail.com> Message-ID: <011301c85110$cf9c9050$6800a8c0@RaymondLaptop1> [Jeffrey Yasskin] > I always like to have a patch around because abstract discussions, > even (especially?) on simple topics, have a tendency to run off into > the weeds. A patch keeps things focused and moving forward. Please recognize that our little system of patches and newsgroup discussions is easily overwhelmed by someone who is so prolific. Your work will not receive quality review and discussion if the volume blows beyond the bandwidth of the other participants. >> The sentiment is unchanged -- please do not build-out the API for the >> decimal module. It is intended to be minimal and needs to stay that >> way. Further additions will only make it harder to use, harder to >> maintain, and will complicate efforts to rewrite the module in C. > > I am not building out the decimal API. I am adding enough methods to > maintain the comment that "Decimal floating point objects share many > properties with the other builtin numeric types such as float and int. > All of the usual math operations and special methods apply." in both > 2.6 and 3.0. After that's done, I am interested in talking about > adding one method that 3 people on this list have been interested in. > I understand that you need to be vigilant against extraneous methods, > but I'm really not just adding these on a whim. Arghh! You seem hell-bent on jamming this in. Please leave the decimal module alone. It does *not* need both a round() method and a quantize() method. I read Glyph's note and agree that it wasn't obvious how to round with the existing API. That is a documentation problem, not a call to add duplicate methods. When one of the module authors asks you to refrain for expanding the API, please pay special attention. In this case, you are working directly against a core philosophy for the design and maintenance of the module. I admire your enthusiasm, but please hold it in check. Raymond From p.f.moore at gmail.com Mon Jan 7 12:37:11 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 7 Jan 2008 11:37:11 +0000 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: References: <4780F4E6.6040208@cheimes.de> <47811F3D.3060403@cheimes.de> <20080106200740.014833A406C@sparrow.telecommunity.com> <79990c6b0801061512u544fd4adwd463352a86b74529@mail.gmail.com> <20080106232838.GA24155@phd.pp.ru> <5d44f72f0801061535l7c02b7cbw59ae10dc0f87795@mail.gmail.com> Message-ID: <79990c6b0801070337h7207c8fdjf89d764377c2ec97@mail.gmail.com> On 07/01/2008, Guido van Rossum wrote: > There seems to be a misunderstanding. This is *not* going to happen > for standard library package names. I'm fine with inventing mechanisms > to allow 3rd party packages to beo cobbled together from multiple > contributions (it would seem to make sense for e.g. Twisted or Zope). > But I am absolutely squarely against 3rd party installs adding to (or > worse, overriding) standard library package names. This is (to me) > simple common sense, the reason being "knowing who to blame". We get > enough bug reports in the Python tracker about 3rd party software as > it is. OK, that seems pretty clear. In which case, I'm -1 on having a "databases" package in the stdlib (this discussion should probably move back to the stdlib reorg list now). Keeping things like sqlite at the top level seems fine to me. I don't have a general rule I'm applying here - I think each suggestion should be considered on its own merits. Paul. From phd at phd.pp.ru Mon Jan 7 13:42:02 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Mon, 7 Jan 2008 15:42:02 +0300 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: References: <4780F4E6.6040208@cheimes.de> <47811F3D.3060403@cheimes.de> <20080106200740.014833A406C@sparrow.telecommunity.com> <20080106234043.C0BE83A406C@sparrow.telecommunity.com> <47816DF1.6040604@cheimes.de> Message-ID: <20080107124202.GC444@phd.pp.ru> On Sun, Jan 06, 2008 at 04:23:59PM -0800, Guido van Rossum wrote: > I don't want "py" or "python" to be > part of the stdlib package namespace. *If* (part of) the stdlib has to > live under a single distinguished name, pick something like "std" or > "core". When I'm using Python I already know I'm using Python, I don't > want to be reminded of that fact on every import line. When I'm using Jython - am I using Python of Java? After from java.lang import Class should it be from py import exceptions or from core import exceptions ? Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From mal at egenix.com Mon Jan 7 13:48:06 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 07 Jan 2008 13:48:06 +0100 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <4780F4E6.6040208@cheimes.de> References: <4780F4E6.6040208@cheimes.de> Message-ID: <47821F86.2030802@egenix.com> On 2008-01-06 16:33, Christian Heimes wrote: > Hello! > > We are discussing name space packages on the stdlib reorg list. For > Python 3.0 we plan to organize the packages by purpose, e.g. put all > database related packages like sqlite and shelve in a 'databases' name > space. Regardless of whether this would really be helpful or not (I used to be in favor of this some time ago, but then realized that this doesn't really buy you anything much), please be aware that by using new generic top-level names you are likely going to get into conflict with existing applications. 'databases' will likely work since most apps will use 'database' as module or package name. > Of course we want to have the name spaces extensible by 3rd party > software. This is not a good idea. The standard lib should not be extensible by 3rd party packages, instead they should use their own top-level package and deal with whatever namespace extension mechanism is used in that package. > The Python world as currently two ways to create extensible > name space packages: pkgutil and pkg_resource. I don't think pkg_resources should go into Python 2.6 as is - for the reasons I've stated last year when this question came up. I also don't like the import mechanism hackery that's being used in setuptools to get namespace packages working. IMHO, it would be a lot cleaner to extend the existing import mechanism to support search continuation, ie. have the import find mechanism continue the search for a sub-package if it doesn't find the module in the first instance of a top-level package directory. E.g. say you want to import 'a.c' and you have directories on you sys.path: libs1/a/ libs1/a/__init__.py libs1/a/b.py libs2/a/ libs2/a/__init__.py libs2/a/c.py the import mechanism would look in libs1/a/, see that c.py is not available, then continue the search and find libs2/a/c.py. This is a sketch, but I think you get the idea. Next, we add a per-user site-packages directory to the standard sys.path, and then we could get rid of most of the setuptools import and sys.path hackery, making it a lot cleaner. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 07 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From mal at egenix.com Mon Jan 7 15:01:45 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 07 Jan 2008 15:01:45 +0100 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <91E6C1F7-3C26-4B0B-96D1-A769A9FFE67D@acm.org> References: <4780F4E6.6040208@cheimes.de> <47821F86.2030802@egenix.com> <91E6C1F7-3C26-4B0B-96D1-A769A9FFE67D@acm.org> Message-ID: <478230C9.5020602@egenix.com> On 2008-01-07 14:57, Fred Drake wrote: > On Jan 7, 2008, at 7:48 AM, M.-A. Lemburg wrote: >> Next, we add a per-user site-packages directory to the standard >> sys.path, and then we could get rid of most of the setuptools >> import and sys.path hackery, making it a lot cleaner. > > > PYTHONPATH already provides this functionality. I see no need to > duplicate that. Agreed, but one of the main arguments for all the .pth file hackery in setuptools is that having to change PYTHONPATH in order to enable user installations of packages is too hard for the typical user. We could easily resolve that issue, if we add a per-user site-packages dir to sys.path in site.py (this is already done for Macs). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 07 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From fdrake at acm.org Mon Jan 7 14:57:20 2008 From: fdrake at acm.org (Fred Drake) Date: Mon, 7 Jan 2008 08:57:20 -0500 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <47821F86.2030802@egenix.com> References: <4780F4E6.6040208@cheimes.de> <47821F86.2030802@egenix.com> Message-ID: <91E6C1F7-3C26-4B0B-96D1-A769A9FFE67D@acm.org> On Jan 7, 2008, at 7:48 AM, M.-A. Lemburg wrote: > Next, we add a per-user site-packages directory to the standard > sys.path, and then we could get rid of most of the setuptools > import and sys.path hackery, making it a lot cleaner. PYTHONPATH already provides this functionality. I see no need to duplicate that. -Fred -- Fred Drake From barry at python.org Mon Jan 7 15:32:48 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 7 Jan 2008 09:32:48 -0500 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <478230C9.5020602@egenix.com> References: <4780F4E6.6040208@cheimes.de> <47821F86.2030802@egenix.com> <91E6C1F7-3C26-4B0B-96D1-A769A9FFE67D@acm.org> <478230C9.5020602@egenix.com> Message-ID: <65501915-2A55-49E5-8711-B347BDD25887@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 7, 2008, at 9:01 AM, M.-A. Lemburg wrote: > We could easily resolve that issue, if we add a per-user site-packages > dir to sys.path in site.py (this is already done for Macs). +1. I've advocated that for years. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBR4I4EHEjvBPtnXfVAQLpPwQAizIj/FBCc9tTNVoPRTghv37WVL0wAk8Z WtmDVpag8H/j7uNKKwMd7Ld0HdVMAwpDetdGPklDnhMDqD/jY5Da2551uKgawFnZ 57WfY5C/VvYbI8jofDEJxb2bEJyG2QILnqut+D8/9zU+z/G1ubL+jgTY03F7a71O JQAaGa6DxNU= =S5C/ -----END PGP SIGNATURE----- From facundobatista at gmail.com Mon Jan 7 15:47:58 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Mon, 7 Jan 2008 12:47:58 -0200 Subject: [Python-Dev] [Python-3000] Rounding Decimals In-Reply-To: <011301c85110$cf9c9050$6800a8c0@RaymondLaptop1> References: <5d44f72f0801051857i3b13f1b0q8a9e485831dcd603@mail.gmail.com> <5d44f72f0801060121l39c4d2a0jfdea69f644bc1822@mail.gmail.com> <5d44f72f0801061526x788b0a46lbbc17ec5dde44ceb@mail.gmail.com> <007601c850ca$7f2ab3e0$6800a8c0@RaymondLaptop1> <5d44f72f0801061803m2fc7ca45gc6a08f415d5a4901@mail.gmail.com> <00a801c850df$0e953e10$6800a8c0@RaymondLaptop1> <5d44f72f0801062154n4a1dab9fu60abb3bb4b0d02dc@mail.gmail.com> <00d001c850f9$bd67e2c0$6800a8c0@RaymondLaptop1> <5d44f72f0801070108j896bb34td7fd82daf696d4d0@mail.gmail.com> <011301c85110$cf9c9050$6800a8c0@RaymondLaptop1> Message-ID: 2008/1/7, Raymond Hettinger : > Arghh! You seem hell-bent on jamming this in. Please leave the > decimal module alone. It does *not* need both a round() method > and a quantize() method. Question. I'm so used to quantize that I don't care. And I'm, in general, -0 to adding new methods to Decimal and/or Context. But in Py3, when you make round(x), x.__round__ will be executed. Same with trunc, ceil and floor. And I think that Decimal should grow these __xxx__ methods. Having said that, the only thing that is not clear to me is if we should: - Diverge 2.6 and 3.0 decimal.py code - Add these __xxx__ to 2.6, and have still one code of decimal. Guido voted for this last one, in the issue 1623: """ If there aren't too many differences between the 2.6 and 3.0 version of decimal.py and your patch, do 2.6 first, then the next time we merge stuff into 3.0 from the trunk it'll be forward-ported automatically. """ But, in the light of this thread, and the new PEP that you will be writing, I think that now it's time to decide this. Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From sdeibel at wingware.com Mon Jan 7 16:10:45 2008 From: sdeibel at wingware.com (Stephan Deibel) Date: Mon, 07 Jan 2008 10:10:45 -0500 Subject: [Python-Dev] [Pydotorg] Do we need both Trac and Roundup? In-Reply-To: <18305.2168.79863.686868@montanaro.dyndns.org> References: <18305.2168.79863.686868@montanaro.dyndns.org> Message-ID: <478240F5.3070101@wingware.com> skip at pobox.com wrote: > Do people think it would be worthwhile to merge the Trac tracker content > (the issue tracker that holds tickets related to the python.org website) > into the Roundup tracker (the issue tracker that holds tickets related to > Python the language)? While they are nominally distinct activities, it > seems to me that we must waste some precious resources (mostly people) > maintaining two mostly disjoint trackers. There is also some functionality > overlap, mostly in the documentation area. It would be great to merge into Roundup. The Trac instance is ancient and I've not had the time to try to upgrade it, which would indeed now be a waste of effort. When Roundup is ready, please let me know and I can at least make a pass through the pydotorg tickets to close those that aren't worth moving over to Roundup. There are quite a few stale and questionable items. In fact, it wouldn't be the end of the world to discard almost all of them, but still worth a review. - Stephan From guido at python.org Mon Jan 7 16:12:44 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 7 Jan 2008 07:12:44 -0800 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <65501915-2A55-49E5-8711-B347BDD25887@python.org> References: <4780F4E6.6040208@cheimes.de> <47821F86.2030802@egenix.com> <91E6C1F7-3C26-4B0B-96D1-A769A9FFE67D@acm.org> <478230C9.5020602@egenix.com> <65501915-2A55-49E5-8711-B347BDD25887@python.org> Message-ID: On Jan 7, 2008 6:32 AM, Barry Warsaw wrote: > On Jan 7, 2008, at 9:01 AM, M.-A. Lemburg wrote: > > We could easily resolve that issue, if we add a per-user site-packages > > dir to sys.path in site.py (this is already done for Macs). > > +1. I've advocated that for years. I'm not sure what this buys given that you can do this using PYTHONPATH anyway, but because of that I also can't be against it. +0 from me. Patches for 2.6 gratefully accepted. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz at pythoncraft.com Mon Jan 7 16:53:26 2008 From: aahz at pythoncraft.com (Aahz) Date: Mon, 7 Jan 2008 07:53:26 -0800 Subject: [Python-Dev] [Python-3000] Rounding Decimals In-Reply-To: <011301c85110$cf9c9050$6800a8c0@RaymondLaptop1> References: <003101c85011$e1572530$6800a8c0@RaymondLaptop1> <5d44f72f0801060121l39c4d2a0jfdea69f644bc1822@mail.gmail.com> <5d44f72f0801061526x788b0a46lbbc17ec5dde44ceb@mail.gmail.com> <007601c850ca$7f2ab3e0$6800a8c0@RaymondLaptop1> <5d44f72f0801061803m2fc7ca45gc6a08f415d5a4901@mail.gmail.com> <00a801c850df$0e953e10$6800a8c0@RaymondLaptop1> <5d44f72f0801062154n4a1dab9fu60abb3bb4b0d02dc@mail.gmail.com> <00d001c850f9$bd67e2c0$6800a8c0@RaymondLaptop1> <5d44f72f0801070108j896bb34td7fd82daf696d4d0@mail.gmail.com> <011301c85110$cf9c9050$6800a8c0@RaymondLaptop1> Message-ID: <20080107155326.GA12886@panix.com> On Mon, Jan 07, 2008, Raymond Hettinger wrote: > [Jeffrey Yasskin] >> >> I am not building out the decimal API. I am adding enough methods to >> maintain the comment that "Decimal floating point objects share many >> properties with the other builtin numeric types such as float and int. >> All of the usual math operations and special methods apply." in both >> 2.6 and 3.0. After that's done, I am interested in talking about >> adding one method that 3 people on this list have been interested in. >> I understand that you need to be vigilant against extraneous methods, >> but I'm really not just adding these on a whim. > > Arghh! You seem hell-bent on jamming this in. Please leave the > decimal module alone. It does *not* need both a round() method > and a quantize() method. > > I read Glyph's note and agree that it wasn't obvious how to round > with the existing API. That is a documentation problem, not a call > to add duplicate methods. Excuse me. I have always thought that "quantize()" makes Decimal confusing in the context of the other mechanisms that Python makes available for other kinds of numbers. I have refrained from making an issue of it because I understand and (mostly) support the idea of keeping Decimal restricted to matching the decimal standard, but at this point, it seems that Tim Peters likes the idea, too. While I have a lot of respect for you, I think Uncle Timmy deserves some special respect, too. > When one of the module authors asks you to refrain for expanding > the API, please pay special attention. In this case, you are working > directly against a core philosophy for the design and maintenance > of the module. I'm not a current maintainer, but I was one of the instigators for the existence of Decimal. I understand that you're feeling frustrated about changes getting shoved in, but I don't think that discussion should be shut down because you have a particular point of view: the decimal module does not represent only your work. I suggest that the patches get dropped for the moment and the discussion rebooted. I haven't been following closely, but it seems that the main issue is that Python 3.0 is introducing a __round__() method. How do you think the decimal module should handle that? (Although I still like the idea of a .round() method for decimals, I think that round(Decimal()) is probably sufficient enough that I won't insist on it.) -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From barry at python.org Mon Jan 7 17:24:06 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 7 Jan 2008 11:24:06 -0500 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: References: <4780F4E6.6040208@cheimes.de> <47821F86.2030802@egenix.com> <91E6C1F7-3C26-4B0B-96D1-A769A9FFE67D@acm.org> <478230C9.5020602@egenix.com> <65501915-2A55-49E5-8711-B347BDD25887@python.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 7, 2008, at 10:12 AM, Guido van Rossum wrote: > On Jan 7, 2008 6:32 AM, Barry Warsaw wrote: >> On Jan 7, 2008, at 9:01 AM, M.-A. Lemburg wrote: >>> We could easily resolve that issue, if we add a per-user site- >>> packages >>> dir to sys.path in site.py (this is already done for Macs). >> >> +1. I've advocated that for years. > > I'm not sure what this buys given that you can do this using > PYTHONPATH anyway, but because of that I also can't be against it. +0 > from me. Patches for 2.6 gratefully accepted. I think it's PEP-worthy too, just so that the semantics get nailed down. Here's a strawman proto-quasi-pre-PEP. Python automatically adds ~/.python/site-packages to sys.path; this is added /before/ the system site-packages file. An open question is whether it needs to go at the front of the list. It should definitely be searched before the system site-packages. Python treats ~/.python/site-packages the same as the system site- packages, w.r.t. .pth files, etc. Open question: should we add yet another environment variable to control this? It's pretty typical for apps to expose such a thing so that the base directory (e.g. ~/.python) can be moved. I think that's all that's needed. It would make playing with easy_install/setuptools nicer to have this. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBR4JSJ3EjvBPtnXfVAQIXfgP9HCi8PNwUbPTbeaf7JPHLTkguYwUAyLH2 vD31w76fnh3pChIZSY9CtP51qRmB22DChSQxaagLmnl9FXjdS1dmXOu7oT7lfj2z avTptyJ2MB8kFuGLK2In/mjbWxPUgAd19sbm4jtE5b3nauOBVyZh23TxFvH5uRdD bUaqRNib3vE= =P3p4 -----END PGP SIGNATURE----- From mal at egenix.com Mon Jan 7 17:30:45 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 07 Jan 2008 17:30:45 +0100 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: References: <4780F4E6.6040208@cheimes.de> <47821F86.2030802@egenix.com> <91E6C1F7-3C26-4B0B-96D1-A769A9FFE67D@acm.org> <478230C9.5020602@egenix.com> <65501915-2A55-49E5-8711-B347BDD25887@python.org> Message-ID: <478253B5.7080605@egenix.com> On 2008-01-07 17:24, Barry Warsaw wrote: > On Jan 7, 2008, at 10:12 AM, Guido van Rossum wrote: > >> On Jan 7, 2008 6:32 AM, Barry Warsaw wrote: >>> On Jan 7, 2008, at 9:01 AM, M.-A. Lemburg wrote: >>>> We could easily resolve that issue, if we add a per-user site-packages >>>> dir to sys.path in site.py (this is already done for Macs). >>> >>> +1. I've advocated that for years. > >> I'm not sure what this buys given that you can do this using >> PYTHONPATH anyway, but because of that I also can't be against it. +0 >> from me. Patches for 2.6 gratefully accepted. > > I think it's PEP-worthy too, just so that the semantics get nailed > down. Here's a strawman proto-quasi-pre-PEP. > > Python automatically adds ~/.python/site-packages to sys.path; this is > added /before/ the system site-packages file. An open question is > whether it needs to go at the front of the list. It should definitely > be searched before the system site-packages. > > Python treats ~/.python/site-packages the same as the system > site-packages, w.r.t. .pth files, etc. > > Open question: should we add yet another environment variable to control > this? It's pretty typical for apps to expose such a thing so that the > base directory (e.g. ~/.python) can be moved. I'd suggest to make the "~/.python" part configurable by an env var, e.g. PYTHONRESOURCES. Perhaps we could use that directory for other Python-related resources as well, e.g. an optional sys.path lookup cache (pickled dictionary of known package/module file locations to reduces Python startup time). > I think that's all that's needed. It would make playing with > easy_install/setuptools nicer to have this. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 07 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From pje at telecommunity.com Mon Jan 7 17:32:13 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 07 Jan 2008 11:32:13 -0500 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <47821F86.2030802@egenix.com> References: <4780F4E6.6040208@cheimes.de> <47821F86.2030802@egenix.com> Message-ID: <20080107163158.10AA93A40D9@sparrow.telecommunity.com> At 01:48 PM 1/7/2008 +0100, M.-A. Lemburg wrote: >I also don't like the import mechanism hackery that's being >used in setuptools to get namespace packages working. I believe you're mistaken: there is no import mechanism "hackery" in pkg_resources. (__path__ is a documented *hook*, not a hack, and it's the only import-related hook that pkg_resources uses). And, if you don't like namespace packages, perhaps you should be campaigning for this to be removed: http://python.org/doc/2.4.1/lib/module-pkgutil.html pkg_resources only updates the routine provided in pkgutil to be a bit more automatic, and to better support PEP 302 and zipfile importing. From pje at telecommunity.com Mon Jan 7 17:34:03 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 07 Jan 2008 11:34:03 -0500 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <478230C9.5020602@egenix.com> References: <4780F4E6.6040208@cheimes.de> <47821F86.2030802@egenix.com> <91E6C1F7-3C26-4B0B-96D1-A769A9FFE67D@acm.org> <478230C9.5020602@egenix.com> Message-ID: <20080107163346.3459A3A40B0@sparrow.telecommunity.com> At 03:01 PM 1/7/2008 +0100, M.-A. Lemburg wrote: >On 2008-01-07 14:57, Fred Drake wrote: > > On Jan 7, 2008, at 7:48 AM, M.-A. Lemburg wrote: > >> Next, we add a per-user site-packages directory to the standard > >> sys.path, and then we could get rid of most of the setuptools > >> import and sys.path hackery, making it a lot cleaner. > > > > > > PYTHONPATH already provides this functionality. I see no need to > > duplicate that. > >Agreed, but one of the main arguments for all the .pth file hackery in >setuptools is that having to change PYTHONPATH in order to enable >user installations of packages is too hard for the typical user. > >We could easily resolve that issue, if we add a per-user site-packages >dir to sys.path in site.py (this is already done for Macs). Actually, neither PYTHONPATH nor your proposal solve all of the problems that .pth files do. To date, nobody has proposed any real substitute for them. From pje at telecommunity.com Mon Jan 7 17:37:19 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 07 Jan 2008 11:37:19 -0500 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: References: <4780F4E6.6040208@cheimes.de> <47821F86.2030802@egenix.com> <91E6C1F7-3C26-4B0B-96D1-A769A9FFE67D@acm.org> <478230C9.5020602@egenix.com> <65501915-2A55-49E5-8711-B347BDD25887@python.org> Message-ID: <20080107163702.ACF603A40B0@sparrow.telecommunity.com> At 11:24 AM 1/7/2008 -0500, Barry Warsaw wrote: >-----BEGIN PGP SIGNED MESSAGE----- >Hash: SHA1 > >On Jan 7, 2008, at 10:12 AM, Guido van Rossum wrote: > > > On Jan 7, 2008 6:32 AM, Barry Warsaw wrote: > >> On Jan 7, 2008, at 9:01 AM, M.-A. Lemburg wrote: > >>> We could easily resolve that issue, if we add a per-user site- > >>> packages > >>> dir to sys.path in site.py (this is already done for Macs). > >> > >> +1. I've advocated that for years. > > > > I'm not sure what this buys given that you can do this using > > PYTHONPATH anyway, but because of that I also can't be against it. +0 > > from me. Patches for 2.6 gratefully accepted. > >I think it's PEP-worthy too, just so that the semantics get nailed >down. Here's a strawman proto-quasi-pre-PEP. > >Python automatically adds ~/.python/site-packages to sys.path; this is >added /before/ the system site-packages file. An open question is >whether it needs to go at the front of the list. It should definitely >be searched before the system site-packages. What about including the Python version in the directory name? C Extensions may not work correctly across versions, and bytecode will get recompiled a lot if you're using multiple versions. Also, if this is a 2.6/3.0 change, it's likely that the *source* won't be compatible across versions either. :) >Python treats ~/.python/site-packages the same as the system site- >packages, w.r.t. .pth files, etc. > >Open question: should we add yet another environment variable to >control this? It's pretty typical for apps to expose such a thing so >that the base directory (e.g. ~/.python) can be moved. > >I think that's all that's needed. It would make playing with >easy_install/setuptools nicer to have this. Assuming that this is a true "site" directory (i.e., .pth files are recognized), then yes. From python at rcn.com Mon Jan 7 18:29:18 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 7 Jan 2008 09:29:18 -0800 Subject: [Python-Dev] [Python-3000] Rounding Decimals References: <003101c85011$e1572530$6800a8c0@RaymondLaptop1><5d44f72f0801060121l39c4d2a0jfdea69f644bc1822@mail.gmail.com><5d44f72f0801061526x788b0a46lbbc17ec5dde44ceb@mail.gmail.com><007601c850ca$7f2ab3e0$6800a8c0@RaymondLaptop1><5d44f72f0801061803m2fc7ca45gc6a08f415d5a4901@mail.gmail.com><00a801c850df$0e953e10$6800a8c0@RaymondLaptop1><5d44f72f0801062154n4a1dab9fu60abb3bb4b0d02dc@mail.gmail.com><00d001c850f9$bd67e2c0$6800a8c0@RaymondLaptop1><5d44f72f0801070108j896bb34td7fd82daf696d4d0@mail.gmail.com><011301c85110$cf9c9050$6800a8c0@RaymondLaptop1> <20080107155326.GA12886@panix.com> Message-ID: <017e01c85152$d5bf47b0$6800a8c0@RaymondLaptop1> [Aahz] > I have always thought that "quantize()" makes Decimal > confusing in the context of the other mechanisms that Python makes > available for other kinds of numbers. No doubt, the spec made a number of choices that are obvious only if you work at IBM. And, there is no doubt, the module has a high level of complexity. Jeffrey is proposing to add a Context.round() method with the purpose of making the module easier to use. I contend that this is self-defeating on the face of it. Please count the number of existing Context methods ( http://docs.python.org/dev/library/decimal.html#context-objects ) and tell me that adding yet another method will reduce complexity or shorten the learning curve. In fact, it will just further confuse which method should be used. Also, I thought we were committed to a minimal implementation of the spec. One reason for this was that programs could be ported back and forth with other implementations of the spec. Another reason is that the core spec has extensive test cases while build-outs have none. And another reason is that we are trying to make life easier for someone (perhaps me) to rewrite the module in C. There is also the principle of having only one way to do it. However, we already have Decimal.quantize and Context.quantize and are about to introduce a number of magic methods callable from builtin functions. Do we really need even more ways to round? The rounding operation is so fundamental to the module that user must grapple with it at some point. Having multiple ways to do it will only make that task more difficult. I spent about a month of my life cleaning, documenting, and testing the decimal module to get it ready for prime time. I'm not at all pleased at the current proposal to add a duplicate method. If someone really cared about making the module easier to use, I would fully support efforts to write a tutorial on the fundamentals or an effort to write a client module implementing only a subset of the spec. Raymond From brett at python.org Mon Jan 7 21:19:44 2008 From: brett at python.org (Brett Cannon) Date: Mon, 7 Jan 2008 12:19:44 -0800 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: References: <4780F4E6.6040208@cheimes.de> <20080106200740.014833A406C@sparrow.telecommunity.com> <20080106234043.C0BE83A406C@sparrow.telecommunity.com> <47816DF1.6040604@cheimes.de> <20080107032255.B7EFB3A406C@sparrow.telecommunity.com> Message-ID: On Jan 6, 2008 8:28 PM, Guido van Rossum wrote: > On Jan 6, 2008 7:23 PM, Phillip J. Eby wrote: > > At 04:23 PM 1/6/2008 -0800, Guido van Rossum wrote: > > >Regarding using common words, either the stdlib grabs these, or > > >*nobody* gets to use them (for fear of conflicting with some other 3rd > > >party package grabbing the same). > > > > This isn't quite true; a standalone Python application that isn't > > extensible doesn't need to worry about this. And it's standalone > > apps that are most likely to claim these common words. (For example, > > until recently, Chandler's database library packages were in 'repository.*'.) > > > > But of course this is still a pretty minor point overall. If the > > stdlib does go for deeper nestings, I have a slight preference for > > seeing them under std.* or some such rather than top level. But I > > don't really see a whole lot of point to doing a major re-org of the > > stdlib space to begin with, for the simple reason that package names > > are not really categories -- they're *names*. IMO 'from databases > > import sqlite' doesn't add any value over 'import pysqlite3' to begin with. > > > > Worse, it will likely be an attractive nuisance for people saying > > "why don't we have databases.Oracle?" and suchlike. And you still > > have to remember the names, only now they're longer. And was it > > database or databases? Greater uniqueness of names is just another > > reason flat is better than nested. :) > > Right. Packages are useful if it helps make the sub-names shorter. The > email package is a good example: it uses lots of generic names like > errors, charset, encoders, message, parser, utils, iterators. So only do the 'databases' package if we can change the modules names to make it worth it? So whichdb becomes databases.which, dumbdbm becomes databases.dumb, etc.? And then extend this to any other package that we consider creating? Otherwise leave it out? How would that follow for sqlite since that is not going to get any shorter thanks to a package? Should it still go into the package for organizational purposes? In other words, should the stdlib reorg only introduce new packages if the majority of modules that go into the package end up with a shorter name? -Brett From guido at python.org Mon Jan 7 21:40:52 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 7 Jan 2008 12:40:52 -0800 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: References: <4780F4E6.6040208@cheimes.de> <20080106200740.014833A406C@sparrow.telecommunity.com> <20080106234043.C0BE83A406C@sparrow.telecommunity.com> <47816DF1.6040604@cheimes.de> <20080107032255.B7EFB3A406C@sparrow.telecommunity.com> Message-ID: On Jan 7, 2008 12:19 PM, Brett Cannon wrote: > On Jan 6, 2008 8:28 PM, Guido van Rossum wrote: > > On Jan 6, 2008 7:23 PM, Phillip J. Eby wrote: > > > At 04:23 PM 1/6/2008 -0800, Guido van Rossum wrote: > > > >Regarding using common words, either the stdlib grabs these, or > > > >*nobody* gets to use them (for fear of conflicting with some other 3rd > > > >party package grabbing the same). > > > > > > This isn't quite true; a standalone Python application that isn't > > > extensible doesn't need to worry about this. And it's standalone > > > apps that are most likely to claim these common words. (For example, > > > until recently, Chandler's database library packages were in 'repository.*'.) > > > > > > But of course this is still a pretty minor point overall. If the > > > stdlib does go for deeper nestings, I have a slight preference for > > > seeing them under std.* or some such rather than top level. But I > > > don't really see a whole lot of point to doing a major re-org of the > > > stdlib space to begin with, for the simple reason that package names > > > are not really categories -- they're *names*. IMO 'from databases > > > import sqlite' doesn't add any value over 'import pysqlite3' to begin with. > > > > > > Worse, it will likely be an attractive nuisance for people saying > > > "why don't we have databases.Oracle?" and suchlike. And you still > > > have to remember the names, only now they're longer. And was it > > > database or databases? Greater uniqueness of names is just another > > > reason flat is better than nested. :) > > > > Right. Packages are useful if it helps make the sub-names shorter. The > > email package is a good example: it uses lots of generic names like > > errors, charset, encoders, message, parser, utils, iterators. > > So only do the 'databases' package if we can change the modules names > to make it worth it? So whichdb becomes databases.which, dumbdbm > becomes databases.dumb, etc.? Bad example IMO; these are all about relatively simple "dict-on-disk" APIs, not about (relational) databases. I'd be +0 things like dbm.gnu, dbm.any, dbm.dumb, dbm.which. > And then extend this to any other > package that we consider creating? Otherwise leave it out? How would > that follow for sqlite since that is not going to get any shorter > thanks to a package? Should it still go into the package for > organizational purposes? If you're asking me, the "organizational purpose" is 100% misguided. > In other words, should the stdlib reorg only introduce new packages if > the majority of modules that go into the package end up with a shorter > name? See what others say. Another reason to have a top-level package would be if there are a lot of subpackages/submodules. This applies to the xml package for example. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Mon Jan 7 21:56:23 2008 From: brett at python.org (Brett Cannon) Date: Mon, 7 Jan 2008 12:56:23 -0800 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: References: <4780F4E6.6040208@cheimes.de> <20080106234043.C0BE83A406C@sparrow.telecommunity.com> <47816DF1.6040604@cheimes.de> <20080107032255.B7EFB3A406C@sparrow.telecommunity.com> Message-ID: On Jan 7, 2008 12:40 PM, Guido van Rossum wrote: > > On Jan 7, 2008 12:19 PM, Brett Cannon wrote: > > On Jan 6, 2008 8:28 PM, Guido van Rossum wrote: > > > On Jan 6, 2008 7:23 PM, Phillip J. Eby wrote: > > > > At 04:23 PM 1/6/2008 -0800, Guido van Rossum wrote: > > > > >Regarding using common words, either the stdlib grabs these, or > > > > >*nobody* gets to use them (for fear of conflicting with some other 3rd > > > > >party package grabbing the same). > > > > > > > > This isn't quite true; a standalone Python application that isn't > > > > extensible doesn't need to worry about this. And it's standalone > > > > apps that are most likely to claim these common words. (For example, > > > > until recently, Chandler's database library packages were in 'repository.*'.) > > > > > > > > But of course this is still a pretty minor point overall. If the > > > > stdlib does go for deeper nestings, I have a slight preference for > > > > seeing them under std.* or some such rather than top level. But I > > > > don't really see a whole lot of point to doing a major re-org of the > > > > stdlib space to begin with, for the simple reason that package names > > > > are not really categories -- they're *names*. IMO 'from databases > > > > import sqlite' doesn't add any value over 'import pysqlite3' to begin with. > > > > > > > > Worse, it will likely be an attractive nuisance for people saying > > > > "why don't we have databases.Oracle?" and suchlike. And you still > > > > have to remember the names, only now they're longer. And was it > > > > database or databases? Greater uniqueness of names is just another > > > > reason flat is better than nested. :) > > > > > > Right. Packages are useful if it helps make the sub-names shorter. The > > > email package is a good example: it uses lots of generic names like > > > errors, charset, encoders, message, parser, utils, iterators. > > > > So only do the 'databases' package if we can change the modules names > > to make it worth it? So whichdb becomes databases.which, dumbdbm > > becomes databases.dumb, etc.? > > Bad example IMO; these are all about relatively simple "dict-on-disk" > APIs, not about (relational) databases. I'd be +0 things like dbm.gnu, > dbm.any, dbm.dumb, dbm.which. > OK. So an html package could have htmllib for its __init__ (or html.lib), and then have html.entities and html.parser for htmlentitydefs and HTMLParser, respectively. Another example is http: http.lib, http.server.cgi, http.server.base, http.server.simple. Both examples are grouped because they make sense, but primarily to make the tail module name simpler. > > And then extend this to any other > > package that we consider creating? Otherwise leave it out? How would > > that follow for sqlite since that is not going to get any shorter > > thanks to a package? Should it still go into the package for > > organizational purposes? > > If you're asking me, the "organizational purpose" is 100% misguided. > Well that will make the reorg simpler. =) > > In other words, should the stdlib reorg only introduce new packages if > > the majority of modules that go into the package end up with a shorter > > name? > > See what others say. This will be interesting. > > Another reason to have a top-level package would be if there are a lot > of subpackages/submodules. This applies to the xml package for > example. The only place I can see that coming into play is if there is any desire to group OS-specific modules together. Otherwise I think Tk-specific stuff is the only place where this has not been done before. -Brett From barry at python.org Mon Jan 7 22:01:42 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 7 Jan 2008 16:01:42 -0500 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <20080107163702.ACF603A40B0@sparrow.telecommunity.com> References: <4780F4E6.6040208@cheimes.de> <47821F86.2030802@egenix.com> <91E6C1F7-3C26-4B0B-96D1-A769A9FFE67D@acm.org> <478230C9.5020602@egenix.com> <65501915-2A55-49E5-8711-B347BDD25887@python.org> <20080107163702.ACF603A40B0@sparrow.telecommunity.com> Message-ID: <2A83DC81-BC55-4282-803D-6E835AF6F5C9@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 7, 2008, at 11:37 AM, Phillip J. Eby wrote: >> Python automatically adds ~/.python/site-packages to sys.path; this >> is >> added /before/ the system site-packages file. An open question is >> whether it needs to go at the front of the list. It should >> definitely >> be searched before the system site-packages. > > What about including the Python version in the directory name? C > Extensions may not work correctly across versions, and bytecode will > get recompiled a lot if you're using multiple versions. Also, if > this is a 2.6/3.0 change, it's likely that the *source* won't be > compatible across versions either. :) D'oh, yes of course. So make that: ~/.python/lib/pythonX.Y/site-packages >> Python treats ~/.python/site-packages the same as the system site- >> packages, w.r.t. .pth files, etc. >> >> Open question: should we add yet another environment variable to >> control this? It's pretty typical for apps to expose such a thing so >> that the base directory (e.g. ~/.python) can be moved. >> >> I think that's all that's needed. It would make playing with >> easy_install/setuptools nicer to have this. > > Assuming that this is a true "site" directory (i.e., .pth files are > recognized), then yes. IMO, it should be a true site directory. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBR4KTN3EjvBPtnXfVAQI6twP9HUQ23I0KCGZy4CC9sA6vvM5xYfmEgKQb 7H9AP84nSqaaX0iyONmRqlPahzHaEkjkoMW68EA3AIkXagf72sNbyBMO1p7I7ZQ6 5X6dR78o8w+NywO6sgdgxqQq3ikXNEEAy2EICLamT94Um1QxR7OV7SlihdpDs20w MujSDYAr9CQ= =el6S -----END PGP SIGNATURE----- From barry at python.org Mon Jan 7 22:03:27 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 7 Jan 2008 16:03:27 -0500 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <478253B5.7080605@egenix.com> References: <4780F4E6.6040208@cheimes.de> <47821F86.2030802@egenix.com> <91E6C1F7-3C26-4B0B-96D1-A769A9FFE67D@acm.org> <478230C9.5020602@egenix.com> <65501915-2A55-49E5-8711-B347BDD25887@python.org> <478253B5.7080605@egenix.com> Message-ID: <8A836983-A4A5-4DFE-A718-6B36573F6D3D@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 7, 2008, at 11:30 AM, M.-A. Lemburg wrote: >> Open question: should we add yet another environment variable to >> control >> this? It's pretty typical for apps to expose such a thing so that >> the >> base directory (e.g. ~/.python) can be moved. > > I'd suggest to make the "~/.python" part configurable by an > env var, e.g. PYTHONRESOURCES. Since we can't use PYTHONOHOME, this works for me. > Perhaps we could use that directory for other Python-related > resources as well, e.g. an optional sys.path lookup cache (pickled > dictionary of known package/module file locations to reduces Python > startup time). Sure, why not? - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBR4KTn3EjvBPtnXfVAQLQlAP/R887WX0k3VNPoiyVs3qerYBj0XQyP4DT leyFWInEamtgk1+5hL+Vu180k+EFcjEQ8d2yet3sMMVUZ4piFHEd6SdNJantVKrl YBVTHkhEBNX2qMFxYmyTwzvjaMttbIn5w9TuEG4PnNiYQv4E4HlZ9HOkRY9YuNDh UMr4e4DCZtw= =PI+B -----END PGP SIGNATURE----- From fdrake at acm.org Mon Jan 7 22:09:38 2008 From: fdrake at acm.org (Fred Drake) Date: Mon, 7 Jan 2008 16:09:38 -0500 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: References: <4780F4E6.6040208@cheimes.de> <20080106234043.C0BE83A406C@sparrow.telecommunity.com> <47816DF1.6040604@cheimes.de> <20080107032255.B7EFB3A406C@sparrow.telecommunity.com> Message-ID: <6DA6752E-2CD8-4303-9418-FAB2A4C48C26@acm.org> On Jan 7, 2008, at 3:56 PM, Brett Cannon wrote: > OK. So an html package could have htmllib for its __init__ (or > html.lib), and then have html.entities and html.parser for > htmlentitydefs and HTMLParser, respectively. Actually, I'd be inclined not to have both HTMLParser and htmllib (regardless of names); a single capable interface should be provided. But that's a separate discussion. -Fred -- Fred Drake From python at rcn.com Mon Jan 7 23:24:23 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 7 Jan 2008 17:24:23 -0500 (EST) Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages Message-ID: <20080107172423.AEQ13728@ms19.lnh.mail.rcn.net> >> And then extend this to any other >> package that we consider creating? Otherwise leave it out? How would >> that follow for sqlite since that is not going to get any shorter >> thanks to a package? Should it still go into the package for >> organizational purposes? > If you're asking me, the "organizational purpose" is 100% misguided. It is my hope that there will be a great deal of restraint in the effort to group modules into packages in Py3.0. For the most part, putting modules in packages only makes them more difficult to access and introduces an additional burden of remembering which package to invoke. The best existing indicator we have is the organization of the docs for the standard library. I, for one, have a hell of a difficult time finding modules via the "organized" table of contents in the Library Reference. Instead, I always go the the Global Module Index where the somewhat flat namespace makes it easy to go directly to the module of interest. I'm curious whether the other developers have had the same experience -- if so, then it is a bad omen for over-organizing the standard library. Another indicator of what lies ahead is the current organization of os vs os.path. While that split-out was well done and necessary, I routinely have difficulty remembering which one contains a function of interest. There are handful of groupings that are obvious (i.e. html and socket modules going into an internet package). Outside of those, I think it best to leave the rest of the modules in a flat namespace (I don't want to have to quess where to find queue, random, pickle, pprint, decimal, and collections. I think most of the concerns with the standard library revolve around module quality, duplication, and obsolence. In contrast, I do not think that introducing a hierarchy of namespaces is of interest to most users. That exercise may well make the language harder to use/learn, not easier. my-two-cents-ly, Raymond From p.f.moore at gmail.com Mon Jan 7 23:38:04 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 7 Jan 2008 22:38:04 +0000 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <20080107172423.AEQ13728@ms19.lnh.mail.rcn.net> References: <20080107172423.AEQ13728@ms19.lnh.mail.rcn.net> Message-ID: <79990c6b0801071438m658804c2rb8c0b5cc00d7df4a@mail.gmail.com> On 07/01/2008, Raymond Hettinger wrote: > It is my hope that there will be a great deal of restraint in the effort to group modules into > packages in Py3.0. +1 > The best existing indicator we have is the organization of the docs for the standard library. > I, for one, have a hell of a difficult time finding modules via the "organized" table of > contents in the Library Reference. Instead, I always go the the Global Module Index > where the somewhat flat namespace makes it easy to go directly to the module of > interest. I'm curious whether the other developers have had the same experience -- if so, > then it is a bad omen for over-organizing the standard library. Yes, I have the same problem. I had not considered this, but I agree that it's the best indication available of how a hierarchical organisation might end up, and what issues there might be. >From the Zen of Python: "Flat is better than nested". > There are handful of groupings that are obvious (i.e. html and socket modules going into > an internet package). One man's obvious is another man's confusing. I'd stick to Guido's principle, that packages should only be used where they simplify sub-names. And even there, use restraint. I know I was earlier tending more towards the side of having more packages. I've been convinced otherwise. Paul. From pje at telecommunity.com Mon Jan 7 23:47:22 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 07 Jan 2008 17:47:22 -0500 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <20080107172423.AEQ13728@ms19.lnh.mail.rcn.net> References: <20080107172423.AEQ13728@ms19.lnh.mail.rcn.net> Message-ID: <20080107224708.2122F3A40AE@sparrow.telecommunity.com> At 05:24 PM 1/7/2008 -0500, Raymond Hettinger wrote: >The best existing indicator we have is the organization of the docs >for the standard library. I, for one, have a hell of a difficult >time finding modules via the "organized" table of contents in the >Library Reference. Instead, I always go the the Global Module Index >where the somewhat flat namespace makes it easy to go directly to >the module of interest. I'm curious whether the other developers >have had the same experience -- if so, then it is a bad omen for >over-organizing the standard library. I do this too. From mithrandi-python-dev at mithrandi.za.net Mon Jan 7 23:49:46 2008 From: mithrandi-python-dev at mithrandi.za.net (Tristan Seligmann) Date: Tue, 8 Jan 2008 00:49:46 +0200 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <2A83DC81-BC55-4282-803D-6E835AF6F5C9@python.org> References: <4780F4E6.6040208@cheimes.de> <47821F86.2030802@egenix.com> <91E6C1F7-3C26-4B0B-96D1-A769A9FFE67D@acm.org> <478230C9.5020602@egenix.com> <65501915-2A55-49E5-8711-B347BDD25887@python.org> <20080107163702.ACF603A40B0@sparrow.telecommunity.com> <2A83DC81-BC55-4282-803D-6E835AF6F5C9@python.org> Message-ID: <20080107224946.GA10908@mithrandi.za.net> * Barry Warsaw [2008-01-07 16:01:42 -0500]: > On Jan 7, 2008, at 11:37 AM, Phillip J. Eby wrote: > > >> Python automatically adds ~/.python/site-packages to sys.path; this > >> is > >> added /before/ the system site-packages file. An open question is > >> whether it needs to go at the front of the list. It should > >> definitely > >> be searched before the system site-packages. > > > > What about including the Python version in the directory name? C > > Extensions may not work correctly across versions, and bytecode will > > get recompiled a lot if you're using multiple versions. Also, if > > this is a 2.6/3.0 change, it's likely that the *source* won't be > > compatible across versions either. :) > > D'oh, yes of course. So make that: > > ~/.python/lib/pythonX.Y/site-packages In that case how about: ~/.local/lib/pythonX.Y/site-packages or: ~/local/lib/pythonX.Y/site-packages I believe both of these locations are already in use by various systems / people, so it would fit in better with existing practice. > IMO, it should be a true site directory. This would be ideal. -- mithrandi, i Ainil en-Balandor, a faer Ambar -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: Digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20080108/9140e1c1/attachment-0001.pgp From p.f.moore at gmail.com Mon Jan 7 23:56:34 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 7 Jan 2008 22:56:34 +0000 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <20080107224946.GA10908@mithrandi.za.net> References: <4780F4E6.6040208@cheimes.de> <47821F86.2030802@egenix.com> <91E6C1F7-3C26-4B0B-96D1-A769A9FFE67D@acm.org> <478230C9.5020602@egenix.com> <65501915-2A55-49E5-8711-B347BDD25887@python.org> <20080107163702.ACF603A40B0@sparrow.telecommunity.com> <2A83DC81-BC55-4282-803D-6E835AF6F5C9@python.org> <20080107224946.GA10908@mithrandi.za.net> Message-ID: <79990c6b0801071456t41a27986y4fb32898b604b8d6@mail.gmail.com> On 07/01/2008, Tristan Seligmann wrote: > > D'oh, yes of course. So make that: > > > > ~/.python/lib/pythonX.Y/site-packages > > In that case how about: > > ~/.local/lib/pythonX.Y/site-packages > > or: > > ~/local/lib/pythonX.Y/site-packages What would be used on Windows? It's likely to be of marginal use on Windows, but an appropriate equivalent should be defined. Possibly just replace ~ with %USERPROFILE%. I'd argue against anything under %APPDATA% as that directory is hidden. Also, should bdist_wininst/bdist_msi installers be updated to offer the option of installing to this directory? What about python setup.py install (add a --user flag, for example)? Paul. From andrewm at object-craft.com.au Tue Jan 8 00:38:21 2008 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Tue, 08 Jan 2008 10:38:21 +1100 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <20080107172423.AEQ13728@ms19.lnh.mail.rcn.net> References: <20080107172423.AEQ13728@ms19.lnh.mail.rcn.net> Message-ID: <20080107233821.4758B68C06B@longblack.object-craft.com.au> >The best existing indicator we have is the organization of the docs for >the standard library. I, for one, have a hell of a difficult time finding >modules via the "organized" table of contents in the Library Reference. >Instead, I always go the the Global Module Index where the somewhat flat >namespace makes it easy to go directly to the module of interest. I'm >curious whether the other developers have had the same experience -- if >so, then it is a bad omen for over-organizing the standard library. I nearly always use my browser's search function to find the module of interest, so yes, I'm effectively using a flat namespace. >Another indicator of what lies ahead is the current organization of os vs >os.path. While that split-out was well done and necessary, I routinely >have difficulty remembering which one contains a function of interest. I mostly remember, but there are some notable exceptions: exists (posix system call, expect to find it in os), walk (which is the old deprecated one? have to check doc). -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From guido at python.org Tue Jan 8 00:47:11 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 7 Jan 2008 15:47:11 -0800 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: References: <4780F4E6.6040208@cheimes.de> <20080106234043.C0BE83A406C@sparrow.telecommunity.com> <47816DF1.6040604@cheimes.de> <20080107032255.B7EFB3A406C@sparrow.telecommunity.com> Message-ID: On Jan 7, 2008 12:56 PM, Brett Cannon wrote: > OK. So an html package could have htmllib for its __init__ (or > html.lib), and then have html.entities and html.parser for > htmlentitydefs and HTMLParser, respectively. I'd be very reluctant to have more "asymmetric" packages like os where the package contains functionality at the top level as well as submodules, because it means that anyone using one of the submodules will pay the price of importing the top-level package. In this example, I can easily see someone using htmlentitydefs without needing htmllib. > Another example is http: > http.lib, http.server.cgi, http.server.base, http.server.simple. That sounds like a good one. > Both examples are grouped because they make sense, but primarily to > make the tail module name simpler. [...] > > Another reason to have a top-level package would be if there are a lot > > of subpackages/submodules. This applies to the xml package for > > example. > > The only place I can see that coming into play is if there is any > desire to group OS-specific modules together. Otherwise I think > Tk-specific stuff is the only place where this has not been done > before. Well, that's a little different -- plat-* and lib-tk are not subpackages but subdirectories. For the plat-* subdirs, this is done so that the same logical module name can have different implementations per platform. For lib-tk it was done to make it easier to create a distribution that didn't include any Tk stuff. But the test package structure doesn't follow this lead, and I'm not sure if it still makes sense for lib-tk. OTOH maybe lib-tk could be promoted to being a real package (named tkinter?), with the core tkinter functionality in __init__ and the rest in submodules with names conforming to PEP 8; this is one example where that type of organization actually makes sense. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fuzzyman at voidspace.org.uk Tue Jan 8 00:29:16 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 07 Jan 2008 23:29:16 +0000 Subject: [Python-Dev] [python] Re: pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <79990c6b0801071456t41a27986y4fb32898b604b8d6@mail.gmail.com> References: <4780F4E6.6040208@cheimes.de> <47821F86.2030802@egenix.com> <91E6C1F7-3C26-4B0B-96D1-A769A9FFE67D@acm.org> <478230C9.5020602@egenix.com> <65501915-2A55-49E5-8711-B347BDD25887@python.org> <20080107163702.ACF603A40B0@sparrow.telecommunity.com> <2A83DC81-BC55-4282-803D-6E835AF6F5C9@python.org> <20080107224946.GA10908@mithrandi.za.net> <79990c6b0801071456t41a27986y4fb32898b604b8d6@mail.gmail.com> Message-ID: <4782B5CC.3040707@voidspace.org.uk> Paul Moore wrote: > On 07/01/2008, Tristan Seligmann wrote: > >>> D'oh, yes of course. So make that: >>> >>> ~/.python/lib/pythonX.Y/site-packages >>> >> In that case how about: >> >> ~/.local/lib/pythonX.Y/site-packages >> >> or: >> >> ~/local/lib/pythonX.Y/site-packages >> > > What would be used on Windows? It's likely to be of marginal use on > Windows, Could be very useful. A lot of machines are used with multiple users. > but an appropriate equivalent should be defined. Possibly > just replace ~ with %USERPROFILE%. I'd argue against anything under > %APPDATA% as that directory is hidden. > Replacing ~ with %USERPROFILE% sounds like the right thing to do. > Also, should bdist_wininst/bdist_msi installers be updated to offer > the option of installing to this directory? What about python setup.py > install (add a --user flag, for example)? > The installers should give the user the choice at install time (preferably). A '--user' flag would also be useful (IMHO). Michael Foord > Paul. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > > From python at rcn.com Tue Jan 8 02:38:03 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 7 Jan 2008 20:38:03 -0500 (EST) Subject: [Python-Dev] Backport PEP 3129: Class Decorators Message-ID: <20080107203803.AEQ34625@ms19.lnh.mail.rcn.net> This seems like something that could reasonably be added to Py2.6. Raymond From lists at cheimes.de Tue Jan 8 03:05:23 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 08 Jan 2008 03:05:23 +0100 Subject: [Python-Dev] Backport PEP 3129: Class Decorators In-Reply-To: <20080107203803.AEQ34625@ms19.lnh.mail.rcn.net> References: <20080107203803.AEQ34625@ms19.lnh.mail.rcn.net> Message-ID: Raymond Hettinger wrote: > This seems like something that could reasonably be added to Py2.6. +1 from me PEP 3107 (function annotation), PEP 3104 (nonlocal) and PEP 3132 (extended iterable unpacking: a, *b = 1,2,3) are IMHO other useful feature for 2.6. nonlocal would require a __future__ import. Christian From lists at cheimes.de Tue Jan 8 03:29:49 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 08 Jan 2008 03:29:49 +0100 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <79990c6b0801071456t41a27986y4fb32898b604b8d6@mail.gmail.com> References: <4780F4E6.6040208@cheimes.de> <47821F86.2030802@egenix.com> <91E6C1F7-3C26-4B0B-96D1-A769A9FFE67D@acm.org> <478230C9.5020602@egenix.com> <65501915-2A55-49E5-8711-B347BDD25887@python.org> <20080107163702.ACF603A40B0@sparrow.telecommunity.com> <2A83DC81-BC55-4282-803D-6E835AF6F5C9@python.org> <20080107224946.GA10908@mithrandi.za.net> <79990c6b0801071456t41a27986y4fb32898b604b8d6@mail.gmail.com> Message-ID: <4782E01D.1040401@cheimes.de> Paul Moore wrote: > What would be used on Windows? It's likely to be of marginal use on > Windows, but an appropriate equivalent should be defined. Possibly > just replace ~ with %USERPROFILE%. I'd argue against anything under > %APPDATA% as that directory is hidden. No, we shouldn't mess with the profile root directory on Windows. The data should either be installed under "Application Data" or under "My Documents". You are right, the appdata directory is hidden by the installer could add a link to Application Data\Python\python2.x\ to the start menu. I also don't like the idea to replace ~ with another string. Every OS is probably using a different path (~/.python/ on Linux, ~/Library/Python on Mac and APPDATA\Python on Windows). The distutils package should gain two methods: global_sitedir() and local_sitedir([username]). > Also, should bdist_wininst/bdist_msi installers be updated to offer > the option of installing to this directory? What about python setup.py > install (add a --user flag, for example)? Sounds good to me. Apropos My Documents and other special directories on Windows. Python doesn't have an API to get the directories from the registry. Is somebody interested in having a module for the task? I've some code for the job on disk. Christian From lists at cheimes.de Tue Jan 8 03:29:49 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 08 Jan 2008 03:29:49 +0100 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <79990c6b0801071456t41a27986y4fb32898b604b8d6@mail.gmail.com> References: <4780F4E6.6040208@cheimes.de> <47821F86.2030802@egenix.com> <91E6C1F7-3C26-4B0B-96D1-A769A9FFE67D@acm.org> <478230C9.5020602@egenix.com> <65501915-2A55-49E5-8711-B347BDD25887@python.org> <20080107163702.ACF603A40B0@sparrow.telecommunity.com> <2A83DC81-BC55-4282-803D-6E835AF6F5C9@python.org> <20080107224946.GA10908@mithrandi.za.net> <79990c6b0801071456t41a27986y4fb32898b604b8d6@mail.gmail.com> Message-ID: <4782E01D.1040401@cheimes.de> Paul Moore wrote: > What would be used on Windows? It's likely to be of marginal use on > Windows, but an appropriate equivalent should be defined. Possibly > just replace ~ with %USERPROFILE%. I'd argue against anything under > %APPDATA% as that directory is hidden. No, we shouldn't mess with the profile root directory on Windows. The data should either be installed under "Application Data" or under "My Documents". You are right, the appdata directory is hidden by the installer could add a link to Application Data\Python\python2.x\ to the start menu. I also don't like the idea to replace ~ with another string. Every OS is probably using a different path (~/.python/ on Linux, ~/Library/Python on Mac and APPDATA\Python on Windows). The distutils package should gain two methods: global_sitedir() and local_sitedir([username]). > Also, should bdist_wininst/bdist_msi installers be updated to offer > the option of installing to this directory? What about python setup.py > install (add a --user flag, for example)? Sounds good to me. Apropos My Documents and other special directories on Windows. Python doesn't have an API to get the directories from the registry. Is somebody interested in having a module for the task? I've some code for the job on disk. Christian From brett at python.org Tue Jan 8 03:43:07 2008 From: brett at python.org (Brett Cannon) Date: Mon, 7 Jan 2008 18:43:07 -0800 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: References: <4780F4E6.6040208@cheimes.de> <20080106234043.C0BE83A406C@sparrow.telecommunity.com> <47816DF1.6040604@cheimes.de> <20080107032255.B7EFB3A406C@sparrow.telecommunity.com> Message-ID: On Jan 7, 2008 3:47 PM, Guido van Rossum wrote: > On Jan 7, 2008 12:56 PM, Brett Cannon wrote: > > OK. So an html package could have htmllib for its __init__ (or > > html.lib), and then have html.entities and html.parser for > > htmlentitydefs and HTMLParser, respectively. > > I'd be very reluctant to have more "asymmetric" packages like os where > the package contains functionality at the top level as well as > submodules, because it means that anyone using one of the submodules > will pay the price of importing the top-level package. In this > example, I can easily see someone using htmlentitydefs without needing > htmllib. > Fair enough. Then something like html.lib or html.tools could be had for miscellaneous code. > > Another example is http: > > http.lib, http.server.cgi, http.server.base, http.server.simple. > > That sounds like a good one. Great! I think I get what you are after then for the reorg in terms of any new packages coming about. > > > Both examples are grouped because they make sense, but primarily to > > make the tail module name simpler. > > [...] > > > > Another reason to have a top-level package would be if there are a lot > > > of subpackages/submodules. This applies to the xml package for > > > example. > > > > The only place I can see that coming into play is if there is any > > desire to group OS-specific modules together. Otherwise I think > > Tk-specific stuff is the only place where this has not been done > > before. > > Well, that's a little different -- plat-* and lib-tk are not > subpackages but subdirectories. For the plat-* subdirs, this is done > so that the same logical module name can have different > implementations per platform. For lib-tk it was done to make it easier > to create a distribution that didn't include any Tk stuff. But the > test package structure doesn't follow this lead, and I'm not sure if > it still makes sense for lib-tk. OTOH maybe lib-tk could be promoted > to being a real package (named tkinter?), with the core tkinter > functionality in __init__ and the rest in submodules with names > conforming to PEP 8; this is one example where that type of > organization actually makes sense. If the platform-specific stuff is made into their own packages (e.g., unix, mac, win, tkinter, etc.) then this can apply generically across the stdlib (sans Modules, of course, unless we eventually change how we handle building extension modules such that they are kept in /Lib as well). I think that would be nice in terms of organization of the code and the documentation as it makes it more obvious which modules are platform-specific. Is applying this to OS-specific modules work for you like it does for tkinter stuff? -Brett From brett at python.org Tue Jan 8 04:08:12 2008 From: brett at python.org (Brett Cannon) Date: Mon, 7 Jan 2008 19:08:12 -0800 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <20080107172423.AEQ13728@ms19.lnh.mail.rcn.net> References: <20080107172423.AEQ13728@ms19.lnh.mail.rcn.net> Message-ID: On Jan 7, 2008 2:24 PM, Raymond Hettinger wrote: > >> And then extend this to any other > >> package that we consider creating? Otherwise leave it out? How would > >> that follow for sqlite since that is not going to get any shorter > >> thanks to a package? Should it still go into the package for > >> organizational purposes? > > > If you're asking me, the "organizational purpose" is 100% misguided. > > It is my hope that there will be a great deal of restraint in the effort to group modules into packages in Py3.0. > There will be. There is a reason why I am willing to let a committee get involved with this as that will make almost any added package difficult to pull off. > For the most part, putting modules in packages only makes them more difficult to access and introduces an additional burden of remembering which package to invoke. > > The best existing indicator we have is the organization of the docs for the standard library. I, for one, have a hell of a difficult time finding modules via the "organized" table of contents in the Library Reference. Instead, I always go the the Global Module Index where the somewhat flat namespace makes it easy to go directly to the module of interest. I'm curious whether the other developers have had the same experience -- if so, then it is a bad omen for over-organizing the standard library. > > Another indicator of what lies ahead is the current organization of os vs os.path. While that split-out was well done and necessary, I routinely have difficulty remembering which one contains a function of interest. > Yeah, 'os' could stand a reorganization, but I am not touching that one. > There are handful of groupings that are obvious (i.e. html and socket modules going into an internet package). Outside of those, I think it best to leave the rest of the modules in a flat namespace (I don't want to have to quess where to find queue, random, pickle, pprint, decimal, and collections. > > I think most of the concerns with the standard library revolve around module quality, duplication, and obsolence. That is definitely a concern of mine and the removal of obsolete code is still the primary motivation for me for this reorg. But I also realize that if we can group reasonable modules together (as you and Guido have now both suggested based more on similarity and coming up with shorter names) then I think we should take the chance now. > In contrast, I do not think that introducing a hierarchy of namespaces is of interest to most users. That exercise may well make the language harder to use/learn, not easier. I was never planning on organizing the entire stdlib into a complete, one-level deep hierarchy. -Brett From dickinsm at gmail.com Tue Jan 8 04:19:17 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 7 Jan 2008 22:19:17 -0500 Subject: [Python-Dev] New Developer Message-ID: <5c6f2a5d0801071919v3a03ef75n3b9574232b1241ff@mail.gmail.com> Hello all, I've recently been granted commit privileges; so, following the usual protocol, here's a quick introduction. I'm a mathematician by day; my degree is in number theory, but five summers of Fortran 77 programming and two semesters of teaching numerical analysis have given me a taste for numerics as well. I discovered Python around twelve years ago and found that it fit my brain nicely (even more so after nested namespaces were introduced) and now use it almost daily for a wide variety of tasks. I've been lurking on python-dev for longer than I care to admit to. I also dabble in Haskell and O'Caml. I'd like to say a huge thank you to Facundo Batista: I've been helping him update Decimal to the most recent specification over the last few months, and without his support and encouragement I certainly wouldn't be writing this now---he's been a fantastic guide and mentor. Please don't hesitate to let me know if there's anything I can help out with; I'm primarily interested in mathematical, numerical, and algorithmic aspects of Python at the moment, but am looking to broaden my knowledge of the internals. I've got so much out of Python over the years, and I hope I can now give something back. Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080107/db0fe6df/attachment.htm From guido at python.org Tue Jan 8 05:44:19 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 7 Jan 2008 20:44:19 -0800 Subject: [Python-Dev] test_sys failures In-Reply-To: <476B73D8.5010400@gmail.com> References: <476B73D8.5010400@gmail.com> Message-ID: On Dec 21, 2007 12:05 AM, Nick Coghlan wrote: > Guido van Rossum wrote: > > When I build from scratch and run most tests (regrtest.py -uall) I get > > some strange failures with test_sys.py: > > > > test test_sys failed -- Traceback (most recent call last): > > File "/usr/local/google/home/guido/python/py3kd/Lib/test/test_sys.py", > > line 302, in test_43581 > > self.assertEqual(sys.__stdout__.encoding, sys.__stderr__.encoding) > > AssertionError: 'ascii' != 'ISO-8859-1' > > > > The same test doesn't fail when run in isolation. > > > > Interestingly, I saw this with 2.5 as well as 3.0, but not with 2.6! > > > > Any ideas? > > It looks like the chunk of code in TextIOWrapper might be getting > different answers when trying to work out the encoding for stdin and > stderr (not that I can see how that would happen...). Or maybe there is > some way that test_sys could be seeing the temporary '__stderr__' entry > that is put in place until the io module is available? > > What do you get for stdin/stdout/stderr.encoding at the interactive > prompt? On Ubuntu, I get UTF-8 for all of them in both 3.0a2 and 2.5.1, > but I'm not seeing the problem, either. > > Other values of possible interest: > os.device_encoding(1) > os.device_encoding(2) > locale.getpreferredencoding() > > Another possibility would be to throw some debugging code into io.py to > print out the encoding name to stderr when file is 1 or 2. No answers yet, but a clue, and anyone on OSX should now be able to reproduce this (with 2.5, 2.6 or 3.0) as follows: ./python ./Lib/test/test_sys.py | cat That is, the problem happens when stdout is redirected to a pipe (or a file). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From p.f.moore at gmail.com Tue Jan 8 10:47:40 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 8 Jan 2008 09:47:40 +0000 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <4782E01D.1040401@cheimes.de> References: <4780F4E6.6040208@cheimes.de> <478230C9.5020602@egenix.com> <65501915-2A55-49E5-8711-B347BDD25887@python.org> <20080107163702.ACF603A40B0@sparrow.telecommunity.com> <2A83DC81-BC55-4282-803D-6E835AF6F5C9@python.org> <20080107224946.GA10908@mithrandi.za.net> <79990c6b0801071456t41a27986y4fb32898b604b8d6@mail.gmail.com> <4782E01D.1040401@cheimes.de> Message-ID: <79990c6b0801080147x692ca5ek838b16e889011779@mail.gmail.com> On 08/01/2008, Christian Heimes wrote: > Paul Moore wrote: > > What would be used on Windows? It's likely to be of marginal use on > > Windows, but an appropriate equivalent should be defined. Possibly > > just replace ~ with %USERPROFILE%. I'd argue against anything under > > %APPDATA% as that directory is hidden. > > No, we shouldn't mess with the profile root directory on Windows. The > data should either be installed under "Application Data" or under "My > Documents". You are right, the appdata directory is hidden by the > installer could add a link to Application Data\Python\python2.x\ to the > start menu. Not My Documents, please! That's for documents, not configuration. %USERPROFILE% is actually where most other applications put stuff. The alternative would be %HOMEDRIVE%%HOMEPATH% which is what os.path.expanduser uses. > Apropos My Documents and other special directories on Windows. Python > doesn't have an API to get the directories from the registry. Is > somebody interested in having a module for the task? I've some code for > the job on disk. It would probably be a good idea to have it in the core, although I'm sure it's in pywin32, and anyone coding Python on Windows will have that. Personally, I've no code that would benefit from this, so I'd be +0 on theoretical grounds only. Paul. From p.f.moore at gmail.com Tue Jan 8 10:50:16 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 8 Jan 2008 09:50:16 +0000 Subject: [Python-Dev] New Developer In-Reply-To: <5c6f2a5d0801071919v3a03ef75n3b9574232b1241ff@mail.gmail.com> References: <5c6f2a5d0801071919v3a03ef75n3b9574232b1241ff@mail.gmail.com> Message-ID: <79990c6b0801080150g7b6db1dbuf873b0d395828ce9@mail.gmail.com> On 08/01/2008, Mark Dickinson wrote: > Hello all, > > I've recently been granted commit privileges; so, following the usual > protocol, here's a quick introduction. Welcome, congratulations and thanks for your work so far! Paul. From ncoghlan at gmail.com Tue Jan 8 12:09:30 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 08 Jan 2008 21:09:30 +1000 Subject: [Python-Dev] test_sys failures In-Reply-To: References: <476B73D8.5010400@gmail.com> Message-ID: <478359EA.3090709@gmail.com> Guido van Rossum wrote: > No answers yet, but a clue, and anyone on OSX should now be able to > reproduce this (with 2.5, 2.6 or 3.0) as follows: > > ./python ./Lib/test/test_sys.py | cat > > That is, the problem happens when stdout is redirected to a pipe (or a file). Redirecting stdout also fails for both the trunk and the py3k branch for me on Ubuntu. If I redirected stderr as well then the tests worked again. Given that a pipe/file and the console very likely *do* have different encodings, maybe the test is just wrong? Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Tue Jan 8 12:30:26 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 08 Jan 2008 21:30:26 +1000 Subject: [Python-Dev] New Developer In-Reply-To: <5c6f2a5d0801071919v3a03ef75n3b9574232b1241ff@mail.gmail.com> References: <5c6f2a5d0801071919v3a03ef75n3b9574232b1241ff@mail.gmail.com> Message-ID: <47835ED2.3010601@gmail.com> Mark Dickinson wrote: > Hello all, > > I've recently been granted commit privileges; so, following the usual > protocol, here's a quick introduction. Welcome, and as Paul said, thanks for everything you've done already. Cheers, Nick. P.S. Does this mean Tim now has another person he can talk numerical correctness with without having to slow right down and use really small words? ;) -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From asmodai at in-nomine.org Tue Jan 8 12:46:09 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Tue, 8 Jan 2008 12:46:09 +0100 Subject: [Python-Dev] test_sys failures In-Reply-To: <478359EA.3090709@gmail.com> References: <476B73D8.5010400@gmail.com> <478359EA.3090709@gmail.com> Message-ID: <20080108114609.GJ75977@nexus.in-nomine.org> -On [20080108 12:09], Nick Coghlan (ncoghlan at gmail.com) wrote: >Redirecting stdout also fails for both the trunk and the py3k branch for >me on Ubuntu. If I redirected stderr as well then the tests worked again. > >Given that a pipe/file and the console very likely *do* have different >encodings, maybe the test is just wrong? This sounds like a problem I recently blogged about (verbatim copy): When you use Python with sys.stdout you might run into a problem where sys.stdout.encoding suddenly becomes None. This happens due to the fact that upon using a pipe or redirection, at least under Unix, it falls back to not knowing anything about the target. In order to work around this you can add a fallback to use locale.getpreferredencoding(). So if you use encode() on a string you can do something like: from locale import getpreferredencoding text = u"Something special" print text.encode(sys.stdout.encoding or getpreferredencoding() or 'ascii', 'replace') This is how we currently use it within Babel as well for printing the locale list. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ Angel to some, Daemon to others... From lists at cheimes.de Tue Jan 8 17:07:27 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 08 Jan 2008 17:07:27 +0100 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <79990c6b0801080147x692ca5ek838b16e889011779@mail.gmail.com> References: <4780F4E6.6040208@cheimes.de> <478230C9.5020602@egenix.com> <65501915-2A55-49E5-8711-B347BDD25887@python.org> <20080107163702.ACF603A40B0@sparrow.telecommunity.com> <2A83DC81-BC55-4282-803D-6E835AF6F5C9@python.org> <20080107224946.GA10908@mithrandi.za.net> <79990c6b0801071456t41a27986y4fb32898b604b8d6@mail.gmail.com> <4782E01D.1040401@cheimes.de> <79990c6b0801080147x692ca5ek838b16e889011779@mail.gmail.com> Message-ID: <47839FBF.10301@cheimes.de> Paul Moore wrote: > Not My Documents, please! That's for documents, not configuration. > %USERPROFILE% is actually where most other applications put stuff. The > alternative would be %HOMEDRIVE%%HOMEPATH% which is what > os.path.expanduser uses. On mys system only one application has put configuration data directly into USERPROFILE. It's Python's IDLE and I don't like it. It should store its configuration under APPDATA. I also don't agree that Python extensions are configuration data. They are code, maybe plugins and the files are user editable content. Ms products like Visual Studio store files like them in My Documents. > It would probably be a good idea to have it in the core, although I'm > sure it's in pywin32, and anyone coding Python on Windows will have > that. Personally, I've no code that would benefit from this, so I'd be > +0 on theoretical grounds only. Python's _winreg module and pywin32 expose several functions to get the paths from the registry but I don't think it has a simple function like get_mydocuments(). Christian From lists at cheimes.de Tue Jan 8 17:23:11 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 08 Jan 2008 17:23:11 +0100 Subject: [Python-Dev] Backport PEP 3129: Class Decorators In-Reply-To: <4783A235.7030003@nibor.org> References: <20080107203803.AEQ34625@ms19.lnh.mail.rcn.net> <4783A235.7030003@nibor.org> Message-ID: <4783A36F.4090908@cheimes.de> Robin Stocker wrote: > I'm planning to work on PEP 3107 (function annotations) after I have > finished backporting PEP 3102 (keyword-only arguments) (issue1745). Thanks! I've backported class decorators: http://bugs.python.org/issue1759 Could somebody with more knowledge about grammer and ASDL check it out, please? Christian From robin at nibor.org Tue Jan 8 17:17:57 2008 From: robin at nibor.org (Robin Stocker) Date: Tue, 08 Jan 2008 17:17:57 +0100 Subject: [Python-Dev] Backport PEP 3129: Class Decorators In-Reply-To: References: <20080107203803.AEQ34625@ms19.lnh.mail.rcn.net> Message-ID: <4783A235.7030003@nibor.org> Christian Heimes schrieb: > PEP 3107 (function annotation), PEP 3104 (nonlocal) and PEP 3132 > (extended iterable unpacking: a, *b = 1,2,3) are IMHO other useful > feature for 2.6. nonlocal would require a __future__ import. I'm planning to work on PEP 3107 (function annotations) after I have finished backporting PEP 3102 (keyword-only arguments) (issue1745). Could someone with access rights update the spreadsheet so there won't be duplicated efforts? http://spreadsheets.google.com/pub?key=pCKY4oaXnT81FrGo3ShGHGg&gid=2 Robin Stocker From eli at courtwright.org Tue Jan 8 17:42:07 2008 From: eli at courtwright.org (Eli Courtwright) Date: Tue, 8 Jan 2008 11:42:07 -0500 Subject: [Python-Dev] Backport PEP 3129: Class Decorators In-Reply-To: <4783A36F.4090908@cheimes.de> References: <20080107203803.AEQ34625@ms19.lnh.mail.rcn.net> <4783A235.7030003@nibor.org> <4783A36F.4090908@cheimes.de> Message-ID: <3f6c86f50801080842v33960d2fh24d7220c07d09fa3@mail.gmail.com> Since we're talking about class decorators, I have a question about function and instancemethod objects. The following code works class Root(object): def index(self): return "Hello World!" index.exposed = True but this code class Root(object): def index(self): return "Hello World!" index.exposed = True gives the following exception Traceback (most recent call last): File "", line 1, in AttributeError: 'instancemethod' object has no attribute 'exposed' I can tell that instancemethods can't have attributes added to them outside of their class definition. Is this part of the Python language spec, or just an implementation detail of CPython? I bring this up here because it makes writing certain class decorators much more annoying. For example, if I want to write a class decorator that will set "exposed=True" for every method of a class, I must resort to shenanigans. - Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080108/aafbc326/attachment.htm From lists at cheimes.de Tue Jan 8 17:49:09 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 08 Jan 2008 17:49:09 +0100 Subject: [Python-Dev] Backport PEP 3129: Class Decorators In-Reply-To: <3f6c86f50801080842v33960d2fh24d7220c07d09fa3@mail.gmail.com> References: <20080107203803.AEQ34625@ms19.lnh.mail.rcn.net> <4783A235.7030003@nibor.org> <4783A36F.4090908@cheimes.de> <3f6c86f50801080842v33960d2fh24d7220c07d09fa3@mail.gmail.com> Message-ID: <4783A985.7030906@cheimes.de> Eli Courtwright wrote: > I can tell that instancemethods can't have attributes added to them outside > of their class definition. Is this part of the Python language spec, or > just an implementation detail of CPython? You can't modify the attributes of an instance method. You have to modify the attribute of the function object. > I bring this up here because it makes writing certain class decorators much > more annoying. For example, if I want to write a class decorator that will > set "exposed=True" for every method of a class, I must resort to > shenanigans. No, you don't. You have to retrieve the function object from the instance method object. The example should shed some light on the problem: >>> class Root(object): ... def index(self): ... return "Hello World!" ... print type(index) ... index.exposed = True ... >>> type(Root.index) >>> Root.index.exposed True >>> Root.index.exposed = False Traceback (most recent call last): File "", line 1, in AttributeError: 'instancemethod' object has no attribute 'exposed' >>> Root.index.im_func >>> Root.index.im_func.exposed = False >>> Root.index.exposed False Christian From lists at cheimes.de Tue Jan 8 17:49:09 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 08 Jan 2008 17:49:09 +0100 Subject: [Python-Dev] Backport PEP 3129: Class Decorators In-Reply-To: <3f6c86f50801080842v33960d2fh24d7220c07d09fa3@mail.gmail.com> References: <20080107203803.AEQ34625@ms19.lnh.mail.rcn.net> <4783A235.7030003@nibor.org> <4783A36F.4090908@cheimes.de> <3f6c86f50801080842v33960d2fh24d7220c07d09fa3@mail.gmail.com> Message-ID: <4783A985.7030906@cheimes.de> Eli Courtwright wrote: > I can tell that instancemethods can't have attributes added to them outside > of their class definition. Is this part of the Python language spec, or > just an implementation detail of CPython? You can't modify the attributes of an instance method. You have to modify the attribute of the function object. > I bring this up here because it makes writing certain class decorators much > more annoying. For example, if I want to write a class decorator that will > set "exposed=True" for every method of a class, I must resort to > shenanigans. No, you don't. You have to retrieve the function object from the instance method object. The example should shed some light on the problem: >>> class Root(object): ... def index(self): ... return "Hello World!" ... print type(index) ... index.exposed = True ... >>> type(Root.index) >>> Root.index.exposed True >>> Root.index.exposed = False Traceback (most recent call last): File "", line 1, in AttributeError: 'instancemethod' object has no attribute 'exposed' >>> Root.index.im_func >>> Root.index.im_func.exposed = False >>> Root.index.exposed False Christian From eli at courtwright.org Tue Jan 8 18:05:12 2008 From: eli at courtwright.org (Eli Courtwright) Date: Tue, 8 Jan 2008 12:05:12 -0500 Subject: [Python-Dev] Backport PEP 3129: Class Decorators In-Reply-To: <4783A985.7030906@cheimes.de> References: <20080107203803.AEQ34625@ms19.lnh.mail.rcn.net> <4783A235.7030003@nibor.org> <4783A36F.4090908@cheimes.de> <3f6c86f50801080842v33960d2fh24d7220c07d09fa3@mail.gmail.com> <4783A985.7030906@cheimes.de> Message-ID: <3f6c86f50801080905v644104aflc5b6e69816ed3b89@mail.gmail.com> Christian, Thanks for the example; I'm sorry that I didn't read the docs carefully enough to realize that I could extract the original function and set the attribute on that. - Eli On Jan 8, 2008 11:49 AM, Christian Heimes wrote: > The example should shed some light on the problem: > > ... > > >>> Root.index.im_func.exposed = False > >>> Root.index.exposed > False > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080108/6aac9f2b/attachment-0001.htm From lists at cheimes.de Tue Jan 8 23:30:25 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 08 Jan 2008 23:30:25 +0100 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <79990c6b0801080147x692ca5ek838b16e889011779@mail.gmail.com> References: <4780F4E6.6040208@cheimes.de> <478230C9.5020602@egenix.com> <65501915-2A55-49E5-8711-B347BDD25887@python.org> <20080107163702.ACF603A40B0@sparrow.telecommunity.com> <2A83DC81-BC55-4282-803D-6E835AF6F5C9@python.org> <20080107224946.GA10908@mithrandi.za.net> <79990c6b0801071456t41a27986y4fb32898b604b8d6@mail.gmail.com> <4782E01D.1040401@cheimes.de> <79990c6b0801080147x692ca5ek838b16e889011779@mail.gmail.com> Message-ID: <4783F981.3040900@cheimes.de> Paul Moore wrote: > Not My Documents, please! That's for documents, not configuration. > %USERPROFILE% is actually where most other applications put stuff. The > alternative would be %HOMEDRIVE%%HOMEPATH% which is what > os.path.expanduser uses. http://msdn2.microsoft.com/en-us/library/bb762494(VS.85).aspx CSIDL_PROFILE (FOLDERID_Profile) Version 5.0. The user's profile folder. A typical path is C:\Documents and Settings\username. Applications should not create files or folders at this level; they should put their data under the locations referred to by CSIDL_APPDATA or CSIDL_LOCAL_APPDATA. Christian From lists at cheimes.de Wed Jan 9 00:51:26 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 09 Jan 2008 00:51:26 +0100 Subject: [Python-Dev] PEP: Lazy module imports and post import hook Message-ID: <47840C7E.3040809@cheimes.de> I've attached the first public draft of my first PEP. A working patch against the py3k branch is available at http://bugs.python.org/issue1576 Christian -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: pep-0369.txt Url: http://mail.python.org/pipermail/python-dev/attachments/20080109/c9c29316/attachment.txt From ndbecker2 at gmail.com Wed Jan 9 01:53:35 2008 From: ndbecker2 at gmail.com (Neal Becker) Date: Tue, 08 Jan 2008 19:53:35 -0500 Subject: [Python-Dev] PEP: Lazy module imports and post import hook References: <47840C7E.3040809@cheimes.de> Message-ID: Christian Heimes wrote: > I've attached the first public draft of my first PEP. A working patch > against the py3k branch is available at http://bugs.python.org/issue1576 > > Christian Note also that mercurial has demandimport http://www.selenic.com/mercurial/wiki/ From andrew-pythondev at puzzling.org Wed Jan 9 02:09:09 2008 From: andrew-pythondev at puzzling.org (Andrew Bennetts) Date: Wed, 9 Jan 2008 12:09:09 +1100 Subject: [Python-Dev] PEP: Lazy module imports and post import hook In-Reply-To: References: <47840C7E.3040809@cheimes.de> Message-ID: <20080109010909.GB24740@steerpike.home.puzzling.org> Neal Becker wrote: > Christian Heimes wrote: > > > I've attached the first public draft of my first PEP. A working patch > > against the py3k branch is available at http://bugs.python.org/issue1576 > > > > Christian > > Note also that mercurial has demandimport > http://www.selenic.com/mercurial/wiki/ And that bzr has lazy_import (inspired by mercurial's demandimport): http://codebrowse.launchpad.net/~bzr/bzr/trunk/annotate/3170?file_id=lazy_import.py-20060910203832-f77c54gf3n232za0-1 http://bazaar-vcs.org/ -Andrew. From aahz at pythoncraft.com Wed Jan 9 03:09:42 2008 From: aahz at pythoncraft.com (Aahz) Date: Tue, 8 Jan 2008 18:09:42 -0800 Subject: [Python-Dev] Extracting variables from string.Template objects In-Reply-To: References: <20080104155256.GA27695@panix.com> Message-ID: <20080109020942.GA2555@panix.com> On Fri, Jan 04, 2008, Isaac Morland wrote: > On Fri, 4 Jan 2008, Aahz wrote: > >>>Also, on a related issue, does it make sense to scan the template >>>string for invalid escape sequences in Template.__init__? For the >>>applications I can imagine of string.Template, I would prefer to get >>>an error upon creating the Template object rather than arbitrarily >>>later when I try to .substitute with it. >> >>No, create an is_valid() method at best. > > I'm curious as to why. Is it to avoid changing the behaviour of existing > code (i.e., backwards compatibility), or do you see a design problem with > having the Template constructor reject invalid template strings? Mostly the former, though I'm not sure about the performance implications of scanning on instance creation. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From fdrake at acm.org Wed Jan 9 04:31:54 2008 From: fdrake at acm.org (Fred Drake) Date: Tue, 8 Jan 2008 22:31:54 -0500 Subject: [Python-Dev] PEP: Lazy module imports and post import hook In-Reply-To: References: <47840C7E.3040809@cheimes.de> Message-ID: <6D66421D-770B-4016-9A97-285FADD3C00A@acm.org> On Jan 8, 2008, at 7:53 PM, Neal Becker wrote: > Note also that mercurial has demandimport > http://www.selenic.com/mercurial/wiki/ Let's not forget zope.deferredimport: http://pypi.python.org/pypi/zope.deferredimport -Fred -- Fred Drake From glyph at divmod.com Wed Jan 9 04:50:03 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Wed, 09 Jan 2008 03:50:03 -0000 Subject: [Python-Dev] PEP: Lazy module imports and post import hook In-Reply-To: <20080109010909.GB24740@steerpike.home.puzzling.org> References: <47840C7E.3040809@cheimes.de> <20080109010909.GB24740@steerpike.home.puzzling.org> Message-ID: <20080109035003.21558.743227508.divmod.xquotient.4494@joule.divmod.com> On 01:09 am, andrew-pythondev at puzzling.org wrote: >Neal Becker wrote: >>Christian Heimes wrote: >> >> > I've attached the first public draft of my first PEP. A working >>patch >> > against the py3k branch is available at >>http://bugs.python.org/issue1576 >> > >> > Christian >> >>Note also that mercurial has demandimport >>http://www.selenic.com/mercurial/wiki/ > >And that bzr has lazy_import (inspired by mercurial's demandimport): and very recently, I implemented similar functionality myself (though it isn't in use in Twisted yet): http://codebrowse.launchpad.net/~glyph/+junk/pyexport/files Something that I notice about every other implementation of this functionality is that they're all in Python. But the proposed implementation here is written in C, and therefore more prone to crashing bugs. Looking at the roundup log, I can see that several refcounting bugs have already been found and fixed. Perhaps the post- import hooks, being a modification to the import mechanism itself, needs to be in C, but given that lazy imports *have* been implemented before without using C code (and without post import hooks), I can't see why they need to be in the same PEP. Also, as is my custom, I'd like to suggest that, rather than designing something new, one of the third-party implementations be adopted (or at least its interface) since these have been demonstrated to work in real application code already. Happily, I can escape the charge of bias this time since I don't think my own implementation should be taken seriously in this case :). From lists at cheimes.de Wed Jan 9 05:03:24 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 09 Jan 2008 05:03:24 +0100 Subject: [Python-Dev] PEP: Lazy module imports and post import hook In-Reply-To: <6D66421D-770B-4016-9A97-285FADD3C00A@acm.org> References: <47840C7E.3040809@cheimes.de> <6D66421D-770B-4016-9A97-285FADD3C00A@acm.org> Message-ID: <4784478C.6090906@cheimes.de> Fred Drake wrote: > On Jan 8, 2008, at 7:53 PM, Neal Becker wrote: >> Note also that mercurial has demandimport >> http://www.selenic.com/mercurial/wiki/ > > > Let's not forget zope.deferredimport: > > http://pypi.python.org/pypi/zope.deferredimport I've mentioned zope.deferredimport in my PEP. :) Christian From lists at cheimes.de Wed Jan 9 05:03:24 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 09 Jan 2008 05:03:24 +0100 Subject: [Python-Dev] PEP: Lazy module imports and post import hook In-Reply-To: <6D66421D-770B-4016-9A97-285FADD3C00A@acm.org> References: <47840C7E.3040809@cheimes.de> <6D66421D-770B-4016-9A97-285FADD3C00A@acm.org> Message-ID: <4784478C.6090906@cheimes.de> Fred Drake wrote: > On Jan 8, 2008, at 7:53 PM, Neal Becker wrote: >> Note also that mercurial has demandimport >> http://www.selenic.com/mercurial/wiki/ > > > Let's not forget zope.deferredimport: > > http://pypi.python.org/pypi/zope.deferredimport I've mentioned zope.deferredimport in my PEP. :) Christian From fdrake at acm.org Wed Jan 9 05:39:57 2008 From: fdrake at acm.org (Fred Drake) Date: Tue, 8 Jan 2008 23:39:57 -0500 Subject: [Python-Dev] PEP: Lazy module imports and post import hook In-Reply-To: <4784478C.6090906@cheimes.de> References: <47840C7E.3040809@cheimes.de> <6D66421D-770B-4016-9A97-285FADD3C00A@acm.org> <4784478C.6090906@cheimes.de> Message-ID: <3EE7C30E-F8D9-4D29-BF7B-2B062B2D4288@acm.org> I foolishly wrote: > Let's not forget zope.deferredimport: > > http://pypi.python.org/pypi/zope.deferredimport On Jan 8, 2008, at 11:03 PM, Christian Heimes wrote: > I've mentioned zope.deferredimport in my PEP. :) Indeed; that's what I get for scanning the text too quickly. Sorry! The URL I posted may be somewhat better for most users, though. -Fred -- Fred Drake From lists at cheimes.de Wed Jan 9 06:01:01 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 09 Jan 2008 06:01:01 +0100 Subject: [Python-Dev] PEP: Lazy module imports and post import hook In-Reply-To: <20080109035003.21558.743227508.divmod.xquotient.4494@joule.divmod.com> References: <47840C7E.3040809@cheimes.de> <20080109010909.GB24740@steerpike.home.puzzling.org> <20080109035003.21558.743227508.divmod.xquotient.4494@joule.divmod.com> Message-ID: <4784550D.40904@cheimes.de> glyph at divmod.com wrote: > and very recently, I implemented similar functionality myself (though it > isn't in use in Twisted yet): > > http://codebrowse.launchpad.net/~glyph/+junk/pyexport/files I'm going to study your implementation tomorrow. > Something that I notice about every other implementation of this > functionality is that they're all in Python. But the proposed > implementation here is written in C, and therefore more prone to > crashing bugs. Looking at the roundup log, I can see that several > refcounting bugs have already been found and fixed. Perhaps the post- > import hooks, being a modification to the import mechanism itself, needs > to be in C, but given that lazy imports *have* been implemented before > without using C code (and without post import hooks), I can't see why > they need to be in the same PEP. Correct, the first few patches were rough implementation sketches. I wanted to discuss implementation details with PJE and it was the easiest way to exchange code. A distributed VCS would have made the task easier but that's a different story. Eventually I focused on the reference counter and I straightened out some ref count bugs. Deferred loading of modules can be implemented without post import hooks and vice versa. But PJE urged me to - at least - consider lazy modules / deferred importing in the post import hook implementation. 3rd party tools need a way to interact with the hooks and notify the system when their own lazy module implementation loads a real module. I figured out I could write a PEP for both features in one PEP and implement them in a combined patch. I could separate them and create two PEPs. > Also, as is my custom, I'd like to suggest that, rather than designing > something new, one of the third-party implementations be adopted (or at > least its interface) since these have been demonstrated to work in real > application code already. Happily, I can escape the charge of bias this > time since I don't think my own implementation should be taken seriously > in this case :). :) I *did* mention that my implementation of lazy imports is based on PJE's peak.util.import: """ PJE's peak.util.imports [3] supports lazy modules an post load hooks. My implementation shares a lot with his and it's partly based on his ideas. """ Finally I like to say something about my choice to implement the lazy loading in C. Most implementation like zope.deferredimport, Hg's demandimport and bzr's lazy_import are using the proxy design pattern. They are storing a wrapper object in the module and sys.modules instead of the real module and they are loading the real module when an attribute is accessed. Some of some (brz) are even using some strange scope replacing. I've to study the implementation more carefully to understand what's going on. peak.util.imports' way is slightly different. It return the instance of a module subclass. When the real module is loaded it redirect name lookup of the lazy module to the real module by replacing the __getattribute__ and __setattr__ methods of the lazy module with the methods of the real module. My implementation uses a similar approach. But I've the luxury that I can modify the C implementation of the module class. It makes the load-real-module-when-attribute-is-accessed dance much easier. I don't have to write a proxy or deal with name spaces. However the PEP covers only the basic infrastructure for lazy imports. For example imp.lazy_import("a.b.c") doesn't put "a" and "a.b" in sys.modules. It neither doesn't cover the replacement __import__ hook provided by Hg nor the deprecation feature of zope.deferredimport. Such infrastructure functions can be implemented in Python. Please remember, the PEP is a draft. I'm waiting for comments and suggestions from users and developers. Christian From lists at cheimes.de Wed Jan 9 06:01:01 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 09 Jan 2008 06:01:01 +0100 Subject: [Python-Dev] PEP: Lazy module imports and post import hook In-Reply-To: <20080109035003.21558.743227508.divmod.xquotient.4494@joule.divmod.com> References: <47840C7E.3040809@cheimes.de> <20080109010909.GB24740@steerpike.home.puzzling.org> <20080109035003.21558.743227508.divmod.xquotient.4494@joule.divmod.com> Message-ID: <4784550D.40904@cheimes.de> glyph at divmod.com wrote: > and very recently, I implemented similar functionality myself (though it > isn't in use in Twisted yet): > > http://codebrowse.launchpad.net/~glyph/+junk/pyexport/files I'm going to study your implementation tomorrow. > Something that I notice about every other implementation of this > functionality is that they're all in Python. But the proposed > implementation here is written in C, and therefore more prone to > crashing bugs. Looking at the roundup log, I can see that several > refcounting bugs have already been found and fixed. Perhaps the post- > import hooks, being a modification to the import mechanism itself, needs > to be in C, but given that lazy imports *have* been implemented before > without using C code (and without post import hooks), I can't see why > they need to be in the same PEP. Correct, the first few patches were rough implementation sketches. I wanted to discuss implementation details with PJE and it was the easiest way to exchange code. A distributed VCS would have made the task easier but that's a different story. Eventually I focused on the reference counter and I straightened out some ref count bugs. Deferred loading of modules can be implemented without post import hooks and vice versa. But PJE urged me to - at least - consider lazy modules / deferred importing in the post import hook implementation. 3rd party tools need a way to interact with the hooks and notify the system when their own lazy module implementation loads a real module. I figured out I could write a PEP for both features in one PEP and implement them in a combined patch. I could separate them and create two PEPs. > Also, as is my custom, I'd like to suggest that, rather than designing > something new, one of the third-party implementations be adopted (or at > least its interface) since these have been demonstrated to work in real > application code already. Happily, I can escape the charge of bias this > time since I don't think my own implementation should be taken seriously > in this case :). :) I *did* mention that my implementation of lazy imports is based on PJE's peak.util.import: """ PJE's peak.util.imports [3] supports lazy modules an post load hooks. My implementation shares a lot with his and it's partly based on his ideas. """ Finally I like to say something about my choice to implement the lazy loading in C. Most implementation like zope.deferredimport, Hg's demandimport and bzr's lazy_import are using the proxy design pattern. They are storing a wrapper object in the module and sys.modules instead of the real module and they are loading the real module when an attribute is accessed. Some of some (brz) are even using some strange scope replacing. I've to study the implementation more carefully to understand what's going on. peak.util.imports' way is slightly different. It return the instance of a module subclass. When the real module is loaded it redirect name lookup of the lazy module to the real module by replacing the __getattribute__ and __setattr__ methods of the lazy module with the methods of the real module. My implementation uses a similar approach. But I've the luxury that I can modify the C implementation of the module class. It makes the load-real-module-when-attribute-is-accessed dance much easier. I don't have to write a proxy or deal with name spaces. However the PEP covers only the basic infrastructure for lazy imports. For example imp.lazy_import("a.b.c") doesn't put "a" and "a.b" in sys.modules. It neither doesn't cover the replacement __import__ hook provided by Hg nor the deprecation feature of zope.deferredimport. Such infrastructure functions can be implemented in Python. Please remember, the PEP is a draft. I'm waiting for comments and suggestions from users and developers. Christian From lists at cheimes.de Wed Jan 9 06:09:05 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 09 Jan 2008 06:09:05 +0100 Subject: [Python-Dev] Import by filename with __import__ ? Message-ID: Today I stumbled about an unknown and undocumented (?) feature. At least it's not documented in our docs. __import__ can import a module by file name: >>> open("/tmp/example.py", "w").write("test = 23\n") >>> mod = __import__("/tmp/example") >>> mod >>> mod.__name__ '/tmp/example' >>> mod.__file__ '/tmp/example.py' >>> mod.test 23 Is it just a coincidence? Is it a desired feature? Why isn't it documented? Christian From pje at telecommunity.com Wed Jan 9 06:09:41 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 09 Jan 2008 00:09:41 -0500 Subject: [Python-Dev] PEP: Lazy module imports and post import hook In-Reply-To: <4784550D.40904@cheimes.de> References: <47840C7E.3040809@cheimes.de> <20080109010909.GB24740@steerpike.home.puzzling.org> <20080109035003.21558.743227508.divmod.xquotient.4494@joule.divmod.com> <4784550D.40904@cheimes.de> Message-ID: <20080109050925.0685D3A406C@sparrow.telecommunity.com> At 06:01 AM 1/9/2008 +0100, Christian Heimes wrote: >However the PEP covers only the basic infrastructure for lazy imports. >For example imp.lazy_import("a.b.c") doesn't put "a" and "a.b" in >sys.modules. That's probably a mistake. In the early days of peak.util.imports (which is now available as the "Importing" package on PyPI), I went through a lot of hairy edge conditions regarding such matters as when parent packages must be present, and ensuring the correct runtime ordering of post-import callbacks. Although I glanced at some of your earlier patch versions and had comments, you should be aware that I have *not* validated it for these sorts of trickier timing considerations, mostly because I'm not 100% sure what they are any more, only that I made sure peak.util.imports could handle them. From guido at python.org Wed Jan 9 06:25:37 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 8 Jan 2008 21:25:37 -0800 Subject: [Python-Dev] Import by filename with __import__ ? In-Reply-To: References: Message-ID: Sounds like a coincidence. Looks like it's using os.path.join(X, name + ".py") where X is a member of sys.path, e.g. the initial ".". This gives good results for valid module names (which never contain slashes) but in this example, os.path.join() ignores the first component if the second starts with '/'. Feel free to add a check that the module name doesn't contain '/', '\\' or '.'. (I think that silently accepting other non-identifier characters is fine, since it doesn't interfere with parsing either the module name or the filename.) On Jan 8, 2008 9:09 PM, Christian Heimes wrote: > Today I stumbled about an unknown and undocumented (?) feature. At least > it's not documented in our docs. __import__ can import a module by file > name: > > >>> open("/tmp/example.py", "w").write("test = 23\n") > >>> mod = __import__("/tmp/example") > >>> mod > > >>> mod.__name__ > '/tmp/example' > >>> mod.__file__ > '/tmp/example.py' > >>> mod.test > 23 > > Is it just a coincidence? Is it a desired feature? Why isn't it documented? > > Christian > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From asmodai at in-nomine.org Wed Jan 9 07:48:39 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Wed, 9 Jan 2008 07:48:39 +0100 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <47839FBF.10301@cheimes.de> References: <65501915-2A55-49E5-8711-B347BDD25887@python.org> <20080107163702.ACF603A40B0@sparrow.telecommunity.com> <2A83DC81-BC55-4282-803D-6E835AF6F5C9@python.org> <20080107224946.GA10908@mithrandi.za.net> <79990c6b0801071456t41a27986y4fb32898b604b8d6@mail.gmail.com> <4782E01D.1040401@cheimes.de> <79990c6b0801080147x692ca5ek838b16e889011779@mail.gmail.com> <47839FBF.10301@cheimes.de> Message-ID: <20080109064839.GM75977@nexus.in-nomine.org> -On [20080108 17:07], Christian Heimes (lists at cheimes.de) wrote: >Python's _winreg module and pywin32 expose several functions to get the >paths from the registry but I don't think it has a simple function like >get_mydocuments(). Careful with the name though. Microsoft Windows Vista did away with 'My Documents & Settings'. It is now C:\Users. So you get: C:\Users\\AppData\Local\ (former Local Settings\Application Data) C:\Users\\AppData\Roaming\ (former Application Data) C:\Users\\Documents (former My Documents) C:\Users\\Music (former My Music) C:\Users\\Pictures (former My Pictures) C:\Users\\Videos (former My Videos) -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ Vae victis! From ncoghlan at gmail.com Wed Jan 9 10:33:34 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 09 Jan 2008 19:33:34 +1000 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <20080109064839.GM75977@nexus.in-nomine.org> References: <65501915-2A55-49E5-8711-B347BDD25887@python.org> <20080107163702.ACF603A40B0@sparrow.telecommunity.com> <2A83DC81-BC55-4282-803D-6E835AF6F5C9@python.org> <20080107224946.GA10908@mithrandi.za.net> <79990c6b0801071456t41a27986y4fb32898b604b8d6@mail.gmail.com> <4782E01D.1040401@cheimes.de> <79990c6b0801080147x692ca5ek838b16e889011779@mail.gmail.com> <47839FBF.10301@cheimes.de> <20080109064839.GM75977@nexus.in-nomine.org> Message-ID: <478494EE.8040003@gmail.com> Jeroen Ruigrok van der Werven wrote: > -On [20080108 17:07], Christian Heimes (lists at cheimes.de) wrote: >> Python's _winreg module and pywin32 expose several functions to get the >> paths from the registry but I don't think it has a simple function like >> get_mydocuments(). > > Careful with the name though. Microsoft Windows Vista did away with 'My > Documents & Settings'. It is now C:\Users. > > So you get: > > C:\Users\\AppData\Local\ (former Local Settings\Application Data) > C:\Users\\AppData\Roaming\ (former Application Data) > C:\Users\\Documents (former My Documents) > C:\Users\\Music (former My Music) > C:\Users\\Pictures (former My Pictures) > C:\Users\\Videos (former My Videos) > Somewhat off topic, but hooray, it looks like someone at MS rediscovered the command line and why "Long and Wordy Names with lots of spaces" are really annoying. Easier access to the application data directory is a good thing too. I guess moving to Vista wouldn't be all bad then ;) To get marginally back on topic, I would actually prefer to have a function like os.gethomedir() that takes on optional user name. (I don't want it enough to write a patch to make it happen though) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From fuzzyman at voidspace.org.uk Wed Jan 9 11:02:44 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 09 Jan 2008 10:02:44 +0000 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <478494EE.8040003@gmail.com> References: <65501915-2A55-49E5-8711-B347BDD25887@python.org> <20080107163702.ACF603A40B0@sparrow.telecommunity.com> <2A83DC81-BC55-4282-803D-6E835AF6F5C9@python.org> <20080107224946.GA10908@mithrandi.za.net> <79990c6b0801071456t41a27986y4fb32898b604b8d6@mail.gmail.com> <4782E01D.1040401@cheimes.de> <79990c6b0801080147x692ca5ek838b16e889011779@mail.gmail.com> <47839FBF.10301@cheimes.de> <20080109064839.GM75977@nexus.in-nomine.org> <478494EE.8040003@gmail.com> Message-ID: <47849BC4.6020800@voidspace.org.uk> Nick Coghlan wrote: > Jeroen Ruigrok van der Werven wrote: > >> -On [20080108 17:07], Christian Heimes (lists at cheimes.de) wrote: >> >>> Python's _winreg module and pywin32 expose several functions to get the >>> paths from the registry but I don't think it has a simple function like >>> get_mydocuments(). >>> >> Careful with the name though. Microsoft Windows Vista did away with 'My >> Documents & Settings'. It is now C:\Users. >> >> So you get: >> >> C:\Users\\AppData\Local\ (former Local Settings\Application Data) >> C:\Users\\AppData\Roaming\ (former Application Data) >> C:\Users\\Documents (former My Documents) >> C:\Users\\Music (former My Music) >> C:\Users\\Pictures (former My Pictures) >> C:\Users\\Videos (former My Videos) >> >> > > Note today's Coding Horror blog entry: "Don't Pollute User Space" http://www.codinghorror.com/blog/archives/001032.html Keep your dirty, filthy paws out of my personal user space! Take a look in your Documents folder right now. Go ahead. Look. Do you see any files or folders in there that you personally did not create? If so, you've been victimized. Applications should never create or modify anything in your documents folder without your permission. {snip...] If applications need to store shared files, that's what the \AppData and \Application Data folders are for. Sentiments I agree with... Michael http://www.manning.com/foord > Somewhat off topic, but hooray, it looks like someone at MS rediscovered > the command line and why "Long and Wordy Names with lots of spaces" are > really annoying. Easier access to the application data directory is a > good thing too. > > I guess moving to Vista wouldn't be all bad then ;) > > To get marginally back on topic, I would actually prefer to have a > function like os.gethomedir() that takes on optional user name. (I don't > want it enough to write a patch to make it happen though) > > Cheers, > Nick. > > From p.f.moore at gmail.com Wed Jan 9 12:58:14 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 9 Jan 2008 11:58:14 +0000 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <47849BC4.6020800@voidspace.org.uk> References: <65501915-2A55-49E5-8711-B347BDD25887@python.org> <2A83DC81-BC55-4282-803D-6E835AF6F5C9@python.org> <20080107224946.GA10908@mithrandi.za.net> <79990c6b0801071456t41a27986y4fb32898b604b8d6@mail.gmail.com> <4782E01D.1040401@cheimes.de> <79990c6b0801080147x692ca5ek838b16e889011779@mail.gmail.com> <47839FBF.10301@cheimes.de> <20080109064839.GM75977@nexus.in-nomine.org> <478494EE.8040003@gmail.com> <47849BC4.6020800@voidspace.org.uk> Message-ID: <79990c6b0801090358h4b92868ev98cb4b742a4e8f5d@mail.gmail.com> On 09/01/2008, Michael Foord wrote: > Note today's Coding Horror blog entry: "Don't Pollute User Space" > > http://www.codinghorror.com/blog/archives/001032.html > > Keep your dirty, filthy paws out of my personal user space! :-) Absolutely [...] > If applications need to store shared files, that's what the \AppData and > \Application Data folders are for. > > Sentiments I agree with... Yes, with the one proviso that Windows (XP, at least, maybe Vista is actually better in this regard) makes it extremely difficult for the user to manually do anything in these directories, so that I'd say this is only appropriate for fully application-maintained files. As far as I know, Windows lacks any really sensible place to store application configuration data which is meant to be user edited on occasion (e.g. rc/ini files, startup scripts, etc). That's often what I see ending up in %USERPROFILE% - even though as Christian points out, it's "officially" wrong. Paul. From reed at reedobrien.com Wed Jan 9 13:51:14 2008 From: reed at reedobrien.com (Reed O'Brien) Date: Wed, 9 Jan 2008 07:51:14 -0500 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <20080109064839.GM75977@nexus.in-nomine.org> References: <65501915-2A55-49E5-8711-B347BDD25887@python.org> <20080107163702.ACF603A40B0@sparrow.telecommunity.com> <2A83DC81-BC55-4282-803D-6E835AF6F5C9@python.org> <20080107224946.GA10908@mithrandi.za.net> <79990c6b0801071456t41a27986y4fb32898b604b8d6@mail.gmail.com> <4782E01D.1040401@cheimes.de> <79990c6b0801080147x692ca5ek838b16e889011779@mail.gmail.com> <47839FBF.10301@cheimes.de> <20080109064839.GM75977@nexus.in-nomine.org> Message-ID: <333D85CA-FFBB-4957-B3B3-36C9D8B4F4CC@reedobrien.com> On Jan 9, 2008, at 1:48 AM, Jeroen Ruigrok van der Werven wrote: > -On [20080108 17:07], Christian Heimes (lists at cheimes.de) wrote: >> Python's _winreg module and pywin32 expose several functions to >> get the >> paths from the registry but I don't think it has a simple function >> like >> get_mydocuments(). > > Careful with the name though. Microsoft Windows Vista did away with > 'My > Documents & Settings'. It is now C:\Users. > > So you get: > > C:\Users\\AppData\Local\ (former Local Settings > \Application Data) > C:\Users\\AppData\Roaming\ (former Application Data) > C:\Users\\Documents (former My Documents) > C:\Users\\Music (former My Music) > C:\Users\\Pictures (former My Pictures) > C:\Users\\Videos (former My Videos) > yay, next up posix support.... *dreams* ~ro > -- > Jeroen Ruigrok van der Werven / asmodai > ????? ?????? ??? ?? ?????? > http://www.in-nomine.org/ | http://www.rangaku.org/ > Vae victis! > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/reed% > 40reedobrien.com From nick at craig-wood.com Wed Jan 9 14:02:08 2008 From: nick at craig-wood.com (Nick Craig-Wood) Date: Wed, 9 Jan 2008 13:02:08 +0000 Subject: [Python-Dev] PEP: Lazy module imports and post import hook In-Reply-To: <47840C7E.3040809@cheimes.de> References: <47840C7E.3040809@cheimes.de> Message-ID: <20080109130208.EB85214C5EA@irishsea.home.craig-wood.com> Christian Heimes wrote: > I've attached the first public draft of my first PEP. Some brief thoughts from me on the PEP... Post import hooks sound great and a good idea. Lazy importing sounds like a recipe for hard to find bugs and rather too magic for my taste. Perhaps you should separate these two things into two PEPs and implementations? -- Nick Craig-Wood -- http://www.craig-wood.com/nick From fuzzyman at voidspace.org.uk Wed Jan 9 14:47:17 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 09 Jan 2008 13:47:17 +0000 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <79990c6b0801090358h4b92868ev98cb4b742a4e8f5d@mail.gmail.com> References: <65501915-2A55-49E5-8711-B347BDD25887@python.org> <2A83DC81-BC55-4282-803D-6E835AF6F5C9@python.org> <20080107224946.GA10908@mithrandi.za.net> <79990c6b0801071456t41a27986y4fb32898b604b8d6@mail.gmail.com> <4782E01D.1040401@cheimes.de> <79990c6b0801080147x692ca5ek838b16e889011779@mail.gmail.com> <47839FBF.10301@cheimes.de> <20080109064839.GM75977@nexus.in-nomine.org> <478494EE.8040003@gmail.com> <47849BC4.6020800@voidspace.org.uk> <79990c6b0801090358h4b92868ev98cb4b742a4e8f5d@mail.gmail.com> Message-ID: <4784D065.5040604@voidspace.org.uk> Paul Moore wrote: > On 09/01/2008, Michael Foord wrote: > >> Note today's Coding Horror blog entry: "Don't Pollute User Space" >> >> http://www.codinghorror.com/blog/archives/001032.html >> >> Keep your dirty, filthy paws out of my personal user space! >> > > :-) Absolutely > > [...] > >> If applications need to store shared files, that's what the \AppData and >> \Application Data folders are for. >> >> Sentiments I agree with... >> > > Yes, with the one proviso that Windows (XP, at least, maybe Vista is > actually better in this regard) makes it extremely difficult for the > user to manually do anything in these directories, Only because Windows XP uses a stupidly long path with spaces in it. It's not actually *hard* to navigate manually to these directories. If the windows installers created by the 'bdist' commands allow you to automatically put stuff there then it shouldn't be too much of a problem. > so that I'd say > this is only appropriate for fully application-maintained files. As > far as I know, Windows lacks any really sensible place to store > application configuration data which is meant to be user edited on > occasion (e.g. rc/ini files, startup scripts, etc). > What user editing is *meant* to be done with extension modules installed into a site directory. Programmer editing maybe... :-) Michael Foord > That's often what I see ending up in %USERPROFILE% - even though as > Christian points out, it's "officially" wrong. > > Paul. > > From p.f.moore at gmail.com Wed Jan 9 15:01:04 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 9 Jan 2008 14:01:04 +0000 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <4784D065.5040604@voidspace.org.uk> References: <65501915-2A55-49E5-8711-B347BDD25887@python.org> <79990c6b0801071456t41a27986y4fb32898b604b8d6@mail.gmail.com> <4782E01D.1040401@cheimes.de> <79990c6b0801080147x692ca5ek838b16e889011779@mail.gmail.com> <47839FBF.10301@cheimes.de> <20080109064839.GM75977@nexus.in-nomine.org> <478494EE.8040003@gmail.com> <47849BC4.6020800@voidspace.org.uk> <79990c6b0801090358h4b92868ev98cb4b742a4e8f5d@mail.gmail.com> <4784D065.5040604@voidspace.org.uk> Message-ID: <79990c6b0801090601s705f539cw3acf2e02ccc7107f@mail.gmail.com> On 09/01/2008, Michael Foord wrote: > Only because Windows XP uses a stupidly long path with spaces in it. > It's not actually *hard* to navigate manually to these directories. The directories are also hidden. That does make it hard to navigate there. I know you can un-hide hidden files, but I view the hidden attribute as useful - just badly misused in this case, unless you assume that these directories are intended to be left alone by the user. > What user editing is *meant* to be done with extension modules installed > into a site directory. Programmer editing maybe... :-) Sorry, I had drifted slightly off topic here. I don't have a problem with user-specific extensions going in appdata, on the presumption that we're talking code only (and excluding something obscure like distutils' distutils.cfg file). Paul. From lists at cheimes.de Wed Jan 9 15:05:56 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 09 Jan 2008 15:05:56 +0100 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <20080109064839.GM75977@nexus.in-nomine.org> References: <65501915-2A55-49E5-8711-B347BDD25887@python.org> <20080107163702.ACF603A40B0@sparrow.telecommunity.com> <2A83DC81-BC55-4282-803D-6E835AF6F5C9@python.org> <20080107224946.GA10908@mithrandi.za.net> <79990c6b0801071456t41a27986y4fb32898b604b8d6@mail.gmail.com> <4782E01D.1040401@cheimes.de> <79990c6b0801080147x692ca5ek838b16e889011779@mail.gmail.com> <47839FBF.10301@cheimes.de> <20080109064839.GM75977@nexus.in-nomine.org> Message-ID: <4784D4C4.1020406@cheimes.de> Jeroen Ruigrok van der Werven wrote: > Careful with the name though. Microsoft Windows Vista did away with 'My > Documents & Settings'. It is now C:\Users. > > So you get: > > C:\Users\\AppData\Local\ (former Local Settings\Application Data) > C:\Users\\AppData\Roaming\ (former Application Data) > C:\Users\\Documents (former My Documents) > C:\Users\\Music (former My Music) > C:\Users\\Pictures (former My Pictures) > C:\Users\\Videos (former My Videos) My latest version uses the SHGetFolderPathW() function from ShlObj.h. It's the preferred way to get shell folder paths like CLSID_PERSONAL (my documents). It's compatible with 2000 and newer (maybe even older, but we don't support ME, NT4 or older) and works on Vista, too. Christian From lists at cheimes.de Wed Jan 9 15:11:42 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 09 Jan 2008 15:11:42 +0100 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <79990c6b0801090601s705f539cw3acf2e02ccc7107f@mail.gmail.com> References: <65501915-2A55-49E5-8711-B347BDD25887@python.org> <79990c6b0801071456t41a27986y4fb32898b604b8d6@mail.gmail.com> <4782E01D.1040401@cheimes.de> <79990c6b0801080147x692ca5ek838b16e889011779@mail.gmail.com> <47839FBF.10301@cheimes.de> <20080109064839.GM75977@nexus.in-nomine.org> <478494EE.8040003@gmail.com> <47849BC4.6020800@voidspace.org.uk> <79990c6b0801090358h4b92868ev98cb4b742a4e8f5d@mail.gmail.com> <4784D065.5040604@voidspace.org.uk> <79990c6b0801090601s705f539cw3acf2e02ccc7107f@mail.gmail.com> Message-ID: <4784D61E.8040307@cheimes.de> Paul Moore wrote: > The directories are also hidden. That does make it hard to navigate > there. I know you can un-hide hidden files, but I view the hidden > attribute as useful - just badly misused in this case, unless you > assume that these directories are intended to be left alone by the > user. It's not an issue for experienced users. For the rest we can put a link in the start menu under Python 2.5 which opens a new explorer with the user package directory. Christian From lists at cheimes.de Wed Jan 9 15:40:45 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 09 Jan 2008 15:40:45 +0100 Subject: [Python-Dev] Coverity Scan, Python upgraded to rung 2 Message-ID: I read the announcement of the Python Users list and figured out that some of the other core developers might be interested in the news, too. Among other projects Python was upgraded to Rung 2 on the Coverity Scan list: http://scan.coverity.com/ Christian From p.f.moore at gmail.com Wed Jan 9 16:18:34 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 9 Jan 2008 15:18:34 +0000 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <4784D61E.8040307@cheimes.de> References: <65501915-2A55-49E5-8711-B347BDD25887@python.org> <79990c6b0801080147x692ca5ek838b16e889011779@mail.gmail.com> <47839FBF.10301@cheimes.de> <20080109064839.GM75977@nexus.in-nomine.org> <478494EE.8040003@gmail.com> <47849BC4.6020800@voidspace.org.uk> <79990c6b0801090358h4b92868ev98cb4b742a4e8f5d@mail.gmail.com> <4784D065.5040604@voidspace.org.uk> <79990c6b0801090601s705f539cw3acf2e02ccc7107f@mail.gmail.com> <4784D61E.8040307@cheimes.de> Message-ID: <79990c6b0801090718v4a272fc9nae2b6c732fe0aa6c@mail.gmail.com> On 09/01/2008, Christian Heimes wrote: > It's not an issue for experienced users. For the rest we can put a link > in the start menu under Python 2.5 which opens a new explorer with the > user package directory. Um, I'm an experienced user and it's an issue for me... The problem isn't so much knowing where the directory is, as that tools (quite rightly) ignore (ie, hide!) hidden directories. For example, 4NT (my command shell) doesn't auto-complete hidden directory names by default, many tools' wildcard matching ignores them (always, not just by default). Blast, I said I wasn't going to start a big flamewar over Windows behaviour. I haven't seen anyone propose that something intended to be edited by a user be stored in a hidden directory (appdata) on Windows. As long as that is the case, I don't care. If you are suggesting that a file intended to be viewed/edited by a user manually should go in AppData, then please be explicit. We can then argue the concrete issues, rather than just theoretical principles. Paul. From lists at cheimes.de Wed Jan 9 16:22:27 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 09 Jan 2008 16:22:27 +0100 Subject: [Python-Dev] Coverity Scan, Python upgraded to rung 2 In-Reply-To: <938f42d70801090715r4ce22df5w50f562d98bb09691@mail.gmail.com> References: <938f42d70801090715r4ce22df5w50f562d98bb09691@mail.gmail.com> Message-ID: <4784E6B3.5000506@cheimes.de> Joseph Armbruster wrote: > Christian, > > Is there any way you (or someone else) could post up the results? It > looks like you need a log in to check them out. I haven't figured out how to access the results. Who has a login and access to the site? Christian From ncoghlan at gmail.com Wed Jan 9 16:38:04 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 10 Jan 2008 01:38:04 +1000 Subject: [Python-Dev] PEP: Lazy module imports and post import hook In-Reply-To: <20080109130208.EB85214C5EA@irishsea.home.craig-wood.com> References: <47840C7E.3040809@cheimes.de> <20080109130208.EB85214C5EA@irishsea.home.craig-wood.com> Message-ID: <4784EA5C.8020109@gmail.com> Nick Craig-Wood wrote: > Christian Heimes wrote: >> I've attached the first public draft of my first PEP. > > Some brief thoughts from me on the PEP... > > Post import hooks sound great and a good idea. > > Lazy importing sounds like a recipe for hard to find bugs and rather > too magic for my taste. > > Perhaps you should separate these two things into two PEPs and > implementations? > And now that I think about it some more, so long as the post-import hook implementation recognises that lazy import libraries *exist*, all it really needs to do is provide a way for the externally implemented lazy import to trigger whatever import hooks have been registered when the module is finally loaded. So count me in with the people that think it is worth separating the two ideas - just focus in the PEP on the API needed to make it possible for 3rd party implementation of lazy imports to interoperate correctly with the post-import hooks, without proposing to add such an implementation directly to the standard library. In the hands of power users lazy imports can definitely be a useful trick to speed up script start times. Those kinds of users will know where to find one of the existing solutions, or will write their own (as the responses to this thread seem to suggest). But I still have deep reservations about making them available as a standard library feature. My main concern is that the import lock is something Python users typically don't have to worry about (aside from the 'don't spawn a thread as a side effect of importing' guideline), but making the acquisition of the import lock implicit in accessing a module attribute is going to lead to some very nasty surprises for users that aren't as intimately familiar with the underlying mechanics of the import system. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Wed Jan 9 16:51:46 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 10 Jan 2008 01:51:46 +1000 Subject: [Python-Dev] Coverity Scan, Python upgraded to rung 2 In-Reply-To: <4784E6B3.5000506@cheimes.de> References: <938f42d70801090715r4ce22df5w50f562d98bb09691@mail.gmail.com> <4784E6B3.5000506@cheimes.de> Message-ID: <4784ED92.800@gmail.com> Christian Heimes wrote: > Joseph Armbruster wrote: >> Christian, >> >> Is there any way you (or someone else) could post up the results? It >> looks like you need a log in to check them out. > > I haven't figured out how to access the results. > > Who has a login and access to the site? I know Neal has access (if I'm recalling the various checkin message correctly, he did the lion's share of the work in addressing the problems Coverity reported). I think some of the other folks on the security list may have one also. Searching the SVN version history for references to Coverity may provide an interesting list. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From Jukka.P.Laurila at nokia.com Wed Jan 9 17:26:08 2008 From: Jukka.P.Laurila at nokia.com (Jukka.P.Laurila at nokia.com) Date: Wed, 9 Jan 2008 18:26:08 +0200 Subject: [Python-Dev] Changing PyAPI_DATA(type) into PyAPI_DATA(type, variablename)? Message-ID: <191CF299661D6E4EA39DC5DB2D4EB73E05388447@esebe105.NOE.Nokia.com> Hello, python-dev! One-liner summary: Would it be possible to change PyAPI_DATA(type) into PyAPI_DATA(type, variablename) to help portability to funny platforms? We've been working on porting the Python 2.5.1 core to the Symbian S60 smartphone platform. Unlike the previous 2.2.2 port, this time we're trying really hard to keep our changes clean so that they could some day be accepted into the main Python codebase. One problem we've encountered is that Symbian doesn't properly support exporting data from a DLL, and instead we must export a function that returns a pointer to the variable in question, and hide this from client applications using a macro: In a generated header, dataexports.h: #define Py_Verbose (*__DATAEXPORT_Py_Verbose()) extern __declspec(dllimport) int *__DATAEXPORT_Py_Verbose(); In a generated C file, dataexport.c: __declspec(dllexport) int *__DATAEXPORT_Py_Verbose() { return &Py_Verbose; } dataexports.h is included into Python.h when Py_BUILD_CORE isn't defined. Now, the problem is that we haven't figured out any way to define the PyAPI_DATA macro in its one-argument form so that this scheme would work, since when compiling client applications we'd like all the PyAPI_DATA declarations to expand to nothing, and get the declarations from dataexport.h instead. The best solution we've come up with is to change PyAPI_DATA into a two-argument form and change all declarations accordingly. Whether to export data with this mechanism or directly could be a pyconfig.h option, DATA_EXPORT_AS_FUNCTIONS (or HAVE_DATA_EXPORT?). Briefly the changes would touch: pyport.h: #ifdef Py_BUILD_CORE #define PyAPI_DATA(RTYPE,NAME) extern RTYPE NAME // data item not exported from the Python DLL #else #define PyAPI_DATA(RTYPE,NAME) // PyAPI_DATA can be blank since then the real declarations come from dataexports.h #endif pydebug.h and all other files that declare exported data: PyAPI_DATA(int) Py_VerboseFlag; --> PyAPI_DATA(int, Py_VerboseFlag) In addition, complex types such as function pointers and arrays would need to be typedef'ed, e.g.: PyAPI_DATA(char) *(*PyOS_ReadlineFunctionPointer)(FILE *, FILE *, char *); --> typedef char *(*t_PyOS_ReadlineFunctionPointer)(FILE *, FILE *, char *); PyAPI_DATA(t_PyOS_ReadlineFunctionPointer, PyOS_ReadlineFunctionPointer) and multiple PyAPI_DATA declarations on a line would need to be split to several PyAPI_DATA declarations. If this change isn't acceptable then we'd pretty much have to fork this aspect of the interpreter - either by just maintaining a patch that's applied to the headers or by making a preprocessor that fixes the headers on the fly. Does this sound reasonable? Any better ideas? -Jukka From skip at pobox.com Wed Jan 9 18:08:21 2008 From: skip at pobox.com (skip at pobox.com) Date: Wed, 9 Jan 2008 11:08:21 -0600 Subject: [Python-Dev] Coverity Scan, Python upgraded to rung 2 In-Reply-To: References: Message-ID: <18308.65413.671943.346078@montanaro.dyndns.org> Christian> I read the announcement of the Python Users list and figured Christian> out that some of the other core developers might be Christian> interested in the news, too. Christian> Among other projects Python was upgraded to Rung 2 on the Christian> Coverity Scan list: http://scan.coverity.com/ I went to the run2 page: http://scan.coverity.com/rung2.html I shows 6 uninspected defects for Python. How do we see what they are? What is an uninspected defect? Any idea how the Coverity folks compute Defects/KLOC? For example, how does tcl manage to get a 0.0 score? Skip From ntoronto at cs.byu.edu Wed Jan 9 18:14:47 2008 From: ntoronto at cs.byu.edu (Neil Toronto) Date: Wed, 09 Jan 2008 10:14:47 -0700 Subject: [Python-Dev] PEP: Lazy module imports and post import hook In-Reply-To: <4784EA5C.8020109@gmail.com> References: <47840C7E.3040809@cheimes.de> <20080109130208.EB85214C5EA@irishsea.home.craig-wood.com> <4784EA5C.8020109@gmail.com> Message-ID: <47850107.20004@cs.byu.edu> Aside: Nick Coghlan wrote: > My main concern is that the import lock is something Python users > typically don't have to worry about (aside from the 'don't spawn a > thread as a side effect of importing' guideline)... I've never heard of this and I can't get Google to find it. Is it a deadlock problem? Neil From mail at timgolden.me.uk Wed Jan 9 18:20:32 2008 From: mail at timgolden.me.uk (Tim Golden) Date: Wed, 09 Jan 2008 17:20:32 +0000 Subject: [Python-Dev] PEP: Lazy module imports and post import hook In-Reply-To: <47850107.20004@cs.byu.edu> References: <47840C7E.3040809@cheimes.de> <20080109130208.EB85214C5EA@irishsea.home.craig-wood.com> <4784EA5C.8020109@gmail.com> <47850107.20004@cs.byu.edu> Message-ID: <47850260.4020608@timgolden.me.uk> Neil Toronto wrote: > Aside: > > Nick Coghlan wrote: >> My main concern is that the import lock is something Python users >> typically don't have to worry about (aside from the 'don't spawn a >> thread as a side effect of importing' guideline)... > > I've never heard of this and I can't get Google to find it. Is it a > deadlock problem? Here's one reference: http://mail.python.org/pipermail/python-list/2004-December/295970.html TJG From lists at cheimes.de Wed Jan 9 18:47:28 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 09 Jan 2008 18:47:28 +0100 Subject: [Python-Dev] Coverity Scan, Python upgraded to rung 2 In-Reply-To: <18308.65413.671943.346078@montanaro.dyndns.org> References: <18308.65413.671943.346078@montanaro.dyndns.org> Message-ID: <478508B0.1010101@cheimes.de> skip at pobox.com wrote: I shows 6 uninspected defects for Python. How do we see what they are? > What is an uninspected defect? Any idea how the Coverity folks compute > Defects/KLOC? For example, how does tcl manage to get a 0.0 score? I can't answer your question. I don't have access to the Python project on their site and the project is currently under maintenance. Maybe Neal can sheds some light on the Coverity Scan project. Christian From martin at v.loewis.de Wed Jan 9 20:14:30 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 09 Jan 2008 20:14:30 +0100 Subject: [Python-Dev] Changing PyAPI_DATA(type) into PyAPI_DATA(type, variablename)? In-Reply-To: <191CF299661D6E4EA39DC5DB2D4EB73E05388447@esebe105.NOE.Nokia.com> References: <191CF299661D6E4EA39DC5DB2D4EB73E05388447@esebe105.NOE.Nokia.com> Message-ID: <47851D16.3030500@v.loewis.de> > One-liner summary: Would it be possible to change PyAPI_DATA(type) into > PyAPI_DATA(type, variablename) to help portability to funny platforms? Quick response: this sounds fine to me. Of course, changing it in 2.5.x is not possible, but making the change in 2.6 should be possible. I would like to see some explicit rationale being recorded in the source so that people know what this was meant for and won't change it back. Regards, Martin From guido at python.org Wed Jan 9 20:15:00 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 9 Jan 2008 11:15:00 -0800 Subject: [Python-Dev] Coverity Scan, Python upgraded to rung 2 In-Reply-To: <478508B0.1010101@cheimes.de> References: <18308.65413.671943.346078@montanaro.dyndns.org> <478508B0.1010101@cheimes.de> Message-ID: On Jan 9, 2008 9:47 AM, Christian Heimes wrote: > skip at pobox.com wrote: > I shows 6 uninspected defects for Python. How do we see what they are? > > What is an uninspected defect? Any idea how the Coverity folks compute > > Defects/KLOC? For example, how does tcl manage to get a 0.0 score? > > I can't answer your question. I don't have access to the Python project > on their site and the project is currently under maintenance. Maybe Neal > can sheds some light on the Coverity Scan project. I'm pretty sure I have an account and I can't get in either. I have contacted coverity asking if they can give accounts to other core developers besides Neal and myself. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Wed Jan 9 21:12:07 2008 From: brett at python.org (Brett Cannon) Date: Wed, 9 Jan 2008 12:12:07 -0800 Subject: [Python-Dev] PEP: Lazy module imports and post import hook In-Reply-To: <4784EA5C.8020109@gmail.com> References: <47840C7E.3040809@cheimes.de> <20080109130208.EB85214C5EA@irishsea.home.craig-wood.com> <4784EA5C.8020109@gmail.com> Message-ID: On Jan 9, 2008 7:38 AM, Nick Coghlan wrote: > Nick Craig-Wood wrote: > > Christian Heimes wrote: > >> I've attached the first public draft of my first PEP. > > > > Some brief thoughts from me on the PEP... > > > > Post import hooks sound great and a good idea. > > > > Lazy importing sounds like a recipe for hard to find bugs and rather > > too magic for my taste. > > > > Perhaps you should separate these two things into two PEPs and > > implementations? > > > > And now that I think about it some more, so long as the post-import hook > implementation recognises that lazy import libraries *exist*, all it > really needs to do is provide a way for the externally implemented lazy > import to trigger whatever import hooks have been registered when the > module is finally loaded. > > So count me in with the people that think it is worth separating the two > ideas - just focus in the PEP on the API needed to make it possible for > 3rd party implementation of lazy imports to interoperate correctly with > the post-import hooks, without proposing to add such an implementation > directly to the standard library. > I agree with Nick and Nick. This should really be two separate PEPs. -Brett From lists at cheimes.de Wed Jan 9 21:13:35 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 09 Jan 2008 21:13:35 +0100 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <79990c6b0801090718v4a272fc9nae2b6c732fe0aa6c@mail.gmail.com> References: <65501915-2A55-49E5-8711-B347BDD25887@python.org> <79990c6b0801080147x692ca5ek838b16e889011779@mail.gmail.com> <47839FBF.10301@cheimes.de> <20080109064839.GM75977@nexus.in-nomine.org> <478494EE.8040003@gmail.com> <47849BC4.6020800@voidspace.org.uk> <79990c6b0801090358h4b92868ev98cb4b742a4e8f5d@mail.gmail.com> <4784D065.5040604@voidspace.org.uk> <79990c6b0801090601s705f539cw3acf2e02ccc7107f@mail.gmail.com> <4784D61E.8040307@cheimes.de> <79990c6b0801090718v4a272fc9nae2b6c732fe0aa6c@mail.gmail.com> Message-ID: <47852AEF.9060400@cheimes.de> Paul Moore wrote: > If you are suggesting that a file intended to be viewed/edited by a > user manually should go in AppData, then please be explicit. We can > then argue the concrete issues, rather than just theoretical > principles. I'm frustrated as well. Neither AppData nor MyDocuments fulfill our requirements. I don't argue for AppData, I only argue against UserProfile as the base directory for a user package directory. I'm open for any suggestion which doesn't violate MS' style guides. (I don't want to imply that I like the style guide but we don't make the rules :( ) Christian From theller at ctypes.org Wed Jan 9 21:14:10 2008 From: theller at ctypes.org (Thomas Heller) Date: Wed, 09 Jan 2008 21:14:10 +0100 Subject: [Python-Dev] Coverity Scan, Python upgraded to rung 2 In-Reply-To: <18308.65413.671943.346078@montanaro.dyndns.org> References: <18308.65413.671943.346078@montanaro.dyndns.org> Message-ID: skip at pobox.com schrieb: > Christian> I read the announcement of the Python Users list and figured > Christian> out that some of the other core developers might be > Christian> interested in the news, too. > > Christian> Among other projects Python was upgraded to Rung 2 on the > Christian> Coverity Scan list: http://scan.coverity.com/ > > I went to the run2 page: > > http://scan.coverity.com/rung2.html On this page, when I click the 'sign in' link, I see the page http://scan.coverity.com/maintenance.html which says: """ Scan administrators are performing maintenance on the selected project. Normally, project members will have received notification in advance of any lengthy unavailability of their project. Please try again later, or contact scan-admin at coverity.com with any questions. Return to Main Page """ Could it be that they were a little bit early with the press release, and the rung2 scanner is not yet active? > I shows 6 uninspected defects for Python. How do we see what they are? > What is an uninspected defect? Any idea how the Coverity folks compute > Defects/KLOC? For example, how does tcl manage to get a 0.0 score? Seems they are referring to the results of the rung 1 run (what ever 'rung' means ;-). With the account Neal made me some months ago, I can login on this page: http://scan.coverity.com:7475/ and see the scan results for Python. Last run at 2007-12-27: 11 Outstanding Defects, 6 of them marked "uninspected", 3 marked "pending", and 2 marked "bug". Thomas From theller at ctypes.org Wed Jan 9 21:19:34 2008 From: theller at ctypes.org (Thomas Heller) Date: Wed, 09 Jan 2008 21:19:34 +0100 Subject: [Python-Dev] Coverity Scan, Python upgraded to rung 2 In-Reply-To: References: <18308.65413.671943.346078@montanaro.dyndns.org> <478508B0.1010101@cheimes.de> Message-ID: Guido van Rossum schrieb: > On Jan 9, 2008 9:47 AM, Christian Heimes wrote: >> skip at pobox.com wrote: >> I shows 6 uninspected defects for Python. How do we see what they are? >> > What is an uninspected defect? Any idea how the Coverity folks compute >> > Defects/KLOC? For example, how does tcl manage to get a 0.0 score? >> >> I can't answer your question. I don't have access to the Python project >> on their site and the project is currently under maintenance. Maybe Neal >> can sheds some light on the Coverity Scan project. > > I'm pretty sure I have an account and I can't get in either. I have > contacted coverity asking if they can give accounts to other core > developers besides Neal and myself. > As I said in the other reply, I can still login on this page: http://scan.coverity.com:7475/ It looks like about 20 users are registered; if wanted I can post the list here. Thomas From lists at cheimes.de Wed Jan 9 21:20:26 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 09 Jan 2008 21:20:26 +0100 Subject: [Python-Dev] PEP: Lazy module imports and post import hook In-Reply-To: References: <47840C7E.3040809@cheimes.de> <20080109130208.EB85214C5EA@irishsea.home.craig-wood.com> <4784EA5C.8020109@gmail.com> Message-ID: <47852C8A.3070007@cheimes.de> Brett Cannon wrote: > I agree with Nick and Nick. This should really be two separate PEPs. I'm fine with the proposal and I'm going to chop the PEP in two parts tonight. Somehow I suspect that the lazy import PEP will be postponed or reject. Christian From lists at cheimes.de Wed Jan 9 22:12:52 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 09 Jan 2008 22:12:52 +0100 Subject: [Python-Dev] Coverity Scan, Python upgraded to rung 2 In-Reply-To: References: <18308.65413.671943.346078@montanaro.dyndns.org> Message-ID: <478538D4.1050103@cheimes.de> Thomas Heller wrote: > Seems they are referring to the results of the rung 1 run (what ever 'rung' means ;-). > With the account Neal made me some months ago, I can login on this page: > > http://scan.coverity.com:7475/ > > and see the scan results for Python. > > Last run at 2007-12-27: 11 Outstanding Defects, 6 of them marked "uninspected", > 3 marked "pending", and 2 marked "bug". My dict says: rung (of a ladder)- Leitersprossen Python has climbed up one step (or rung) of the ladder. Do you have the required permission to add more users to the site? Christian From lists at cheimes.de Wed Jan 9 22:12:52 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 09 Jan 2008 22:12:52 +0100 Subject: [Python-Dev] Coverity Scan, Python upgraded to rung 2 In-Reply-To: References: <18308.65413.671943.346078@montanaro.dyndns.org> Message-ID: <478538D4.1050103@cheimes.de> Thomas Heller wrote: > Seems they are referring to the results of the rung 1 run (what ever 'rung' means ;-). > With the account Neal made me some months ago, I can login on this page: > > http://scan.coverity.com:7475/ > > and see the scan results for Python. > > Last run at 2007-12-27: 11 Outstanding Defects, 6 of them marked "uninspected", > 3 marked "pending", and 2 marked "bug". My dict says: rung (of a ladder)- Leitersprossen Python has climbed up one step (or rung) of the ladder. Do you have the required permission to add more users to the site? Christian From theller at ctypes.org Wed Jan 9 22:17:54 2008 From: theller at ctypes.org (Thomas Heller) Date: Wed, 09 Jan 2008 22:17:54 +0100 Subject: [Python-Dev] Coverity Scan, Python upgraded to rung 2 In-Reply-To: <478538D4.1050103@cheimes.de> References: <18308.65413.671943.346078@montanaro.dyndns.org> <478538D4.1050103@cheimes.de> Message-ID: Christian Heimes schrieb: > Thomas Heller wrote: >> Seems they are referring to the results of the rung 1 run (what ever 'rung' means ;-). >> With the account Neal made me some months ago, I can login on this page: >> >> http://scan.coverity.com:7475/ >> >> and see the scan results for Python. >> >> Last run at 2007-12-27: 11 Outstanding Defects, 6 of them marked "uninspected", >> 3 marked "pending", and 2 marked "bug". > > My dict says: rung (of a ladder)- Leitersprossen > > Python has climbed up one step (or rung) of the ladder. Thanks. I was too lazy to fire up dict.leo.org ;-) > Do you have the required permission to add more users to the site? No, I can only view the results (and add comments or so...). Thomas From pje at telecommunity.com Wed Jan 9 22:56:55 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 09 Jan 2008 16:56:55 -0500 Subject: [Python-Dev] PEP: Lazy module imports and post import hook In-Reply-To: <47852C8A.3070007@cheimes.de> References: <47840C7E.3040809@cheimes.de> <20080109130208.EB85214C5EA@irishsea.home.craig-wood.com> <4784EA5C.8020109@gmail.com> <47852C8A.3070007@cheimes.de> Message-ID: <20080109215639.533CD3A40A2@sparrow.telecommunity.com> At 09:20 PM 1/9/2008 +0100, Christian Heimes wrote: >Brett Cannon wrote: > > I agree with Nick and Nick. This should really be two separate PEPs. > >I'm fine with the proposal and I'm going to chop the PEP in two parts >tonight. > >Somehow I suspect that the lazy import PEP will be postponed or reject. Probably. After the split, I'll review things again, with a closer eye on the initialization order issues, especially with respect to ensuring that lazy imports set the corresponding attribute in the parent package at the right point in time. (It should happen *before* the submodule can actually be imported.) The big advantage to a stdlib implementation of lazy modules would be that it could be more vetted and "blessed" -- the downside is that it's a new and nontrivial implementation. :( From p.f.moore at gmail.com Thu Jan 10 00:12:59 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 9 Jan 2008 23:12:59 +0000 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <47852AEF.9060400@cheimes.de> References: <65501915-2A55-49E5-8711-B347BDD25887@python.org> <20080109064839.GM75977@nexus.in-nomine.org> <478494EE.8040003@gmail.com> <47849BC4.6020800@voidspace.org.uk> <79990c6b0801090358h4b92868ev98cb4b742a4e8f5d@mail.gmail.com> <4784D065.5040604@voidspace.org.uk> <79990c6b0801090601s705f539cw3acf2e02ccc7107f@mail.gmail.com> <4784D61E.8040307@cheimes.de> <79990c6b0801090718v4a272fc9nae2b6c732fe0aa6c@mail.gmail.com> <47852AEF.9060400@cheimes.de> Message-ID: <79990c6b0801091512m776fa278sc4cfd020d06d06fc@mail.gmail.com> On 09/01/2008, Christian Heimes wrote: > Paul Moore wrote: > > If you are suggesting that a file intended to be viewed/edited by a > > user manually should go in AppData, then please be explicit. We can > > then argue the concrete issues, rather than just theoretical > > principles. > > I'm frustrated as well. Neither AppData nor MyDocuments fulfill our > requirements. For what? If you're talking about a per-user site-packages directory, I have no problem with AppData (as Michael said, there's no intention that users be able to edit the files). Of course, modules/packages/extensions don't really qualify as application *data*, so it is a somewhat odd usage certainly, but it's not disastrous. Windows doesn't really have a concept of a per-user application code directory. Come to that, if I use a bdist_wininst installer to install a package in my personal site packages, then I log off and you log on, would you expect to see my package when you look in add/remove programs? Would you expect to be able to uninstall it? Would you expect to be able to install a personal copy of your own? What would we see in add/remove programs *then*?? No matter how you cut it, Windows isn't designed for per-user installable programs. Maybe a per-user site-packages just isn't appropriate on Windows. Paul. PS This is mostly theoretical for me, as I don't have a personal need for a per-user site packages directory on any of the machines I use. From steve at holdenweb.com Thu Jan 10 00:52:58 2008 From: steve at holdenweb.com (Steve Holden) Date: Wed, 09 Jan 2008 18:52:58 -0500 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <79990c6b0801091512m776fa278sc4cfd020d06d06fc@mail.gmail.com> References: <65501915-2A55-49E5-8711-B347BDD25887@python.org> <20080109064839.GM75977@nexus.in-nomine.org> <478494EE.8040003@gmail.com> <47849BC4.6020800@voidspace.org.uk> <79990c6b0801090358h4b92868ev98cb4b742a4e8f5d@mail.gmail.com> <4784D065.5040604@voidspace.org.uk> <79990c6b0801090601s705f539cw3acf2e02ccc7107f@mail.gmail.com> <4784D61E.8040307@cheimes.de> <79990c6b0801090718v4a272fc9nae2b6c732fe0aa6c@mail.gmail.com> <47852AEF.9060400@cheimes.de> <79990c6b0801091512m776fa278sc4cfd020d06d06fc@mail.gmail.com> Message-ID: Paul Moore wrote: [...] > No matter how you cut it, Windows isn't designed for per-user > installable programs. Maybe a per-user site-packages just isn't > appropriate on Windows. > This reminds me of the early days of Microsoft Terminal Service (read: "X Window done wrong fifteen years later"), when various applications had to be modified to stop saving per-user data in global locations. It underlines the fact that essentially Windows is still betrayed by its original conception as a single-user system. The idea that users would /program their own computers/ was totally alien to the Windows mindset. looking-at-Vista-and-thinking-very-hard-ly y'rs - steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From barry at python.org Thu Jan 10 01:34:46 2008 From: barry at python.org (Barry Warsaw) Date: Wed, 9 Jan 2008 19:34:46 -0500 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <20080107224946.GA10908@mithrandi.za.net> References: <4780F4E6.6040208@cheimes.de> <47821F86.2030802@egenix.com> <91E6C1F7-3C26-4B0B-96D1-A769A9FFE67D@acm.org> <478230C9.5020602@egenix.com> <65501915-2A55-49E5-8711-B347BDD25887@python.org> <20080107163702.ACF603A40B0@sparrow.telecommunity.com> <2A83DC81-BC55-4282-803D-6E835AF6F5C9@python.org> <20080107224946.GA10908@mithrandi.za.net> Message-ID: <4BFB2541-DAF7-4A20-ADAE-A539286C1102@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 7, 2008, at 5:49 PM, Tristan Seligmann wrote: > > In that case how about: > > ~/.local/lib/pythonX.Y/site-packages > > or: > > ~/local/lib/pythonX.Y/site-packages > > I believe both of these locations are already in use by various > systems > / people, so it would fit in better with existing practice. I like the former path. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBR4VoJ3EjvBPtnXfVAQK9zQP/dup8RHQha0v9ZkYee+L5/8/fhvIHBEoP MGkBxZUv79SmtnY4WWovS66NYAuKHlChFN0VSZCUO8o7WE8b52lXhgeG2xGX/pU/ MZ6D6zjxZmFhQjLLNf5gRcvchYHM+18AYQFapnuCpV048cgmJ7DQcGqznpyYRhnm SvqTeBtstNw= =HYwR -----END PGP SIGNATURE----- From schmir at gmail.com Wed Jan 9 22:58:46 2008 From: schmir at gmail.com (Ralf Schmitt) Date: Wed, 9 Jan 2008 22:58:46 +0100 Subject: [Python-Dev] some bugs that need attention Message-ID: <932f8baf0801091358p4185fcb4r2c4350c0beaca83b@mail.gmail.com> Hi, I've taken a look at some bugs in the bugtracker. I think these should be closed: http://bugs.python.org/issue1720992 is about automatic imports. http://bugs.python.org/issue1448325 and http://bugs.python.org/issue1566086 is about the regular expression engine "hanging". These are duplicates of http://bugs.python.org/issue1662581 and should be closed. Regards, - Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080109/2c5fb211/attachment.htm From lists at cheimes.de Thu Jan 10 03:20:00 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 10 Jan 2008 03:20:00 +0100 Subject: [Python-Dev] PEP: Post import hooks Message-ID: I've parted the former PEP as promised. Here is the post import hook. I'll tackle the other half of the PEP later. PEP: 369 Title: Post import hooks Version: $Revision$ Last-Modified: $Date$ Author: Christian Heimes Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 02-Jan-2008 Python-Version: 2.6, 3.0 Post-History: Abstract ======== This PEP proposes enhancements for the import machinery to add post import hooks. It is intended primarily to support the wider use of abstract base classes that is expected in Python 3.0. The PEP originally started as a combined PEP for lazy imports and post import hooks. After some discussion on the python-dev mailing list the PEP was parted in two separate PEPs. [1]_ Rationale ========= Python has no API to hook into the import machinery and execute code *after* a module is successfully loaded. The import hooks of PEP 302 are about finding modules and loading modules but they were not designed to as post import hooks. Use cases ========= A use case for a post import hook is mentioned in Nick Coghlan's initial posting [2]_. about callbacks on module import. It was found during the development of Python 3.0 and its ABCs. We wanted to register classes like decimal.Decimal with an ABC but the module should not be imported on every interpreter startup. Nick came up with this example:: @imp.when_imported('decimal') def register(decimal): Inexact.register(decimal.Decimal) The function ``register`` is registered as callback for the module named 'decimal'. When decimal is imported the function is called with the module object as argument. While this particular example isn't necessary in practice, (as decimal.Decimal will inherit from the appropriate abstract Number base class in 2.6 and 3.0), it still illustrates the principle. Existing implementations ======================== PJE's peak.util.imports [3]_ implements post load hooks. My implementation shares a lot with his and it's partly based on his ideas. Post import hook implementation =============================== Post import hooks are called after a module has been loaded. The hooks are callable which take one argument, the module instance. They are registered by the dotted name of the module, e.g. 'os' or 'os.path'. The callable are stored in the dict ``sys.post_import_hooks`` which is a mapping from names (as string) to a list of callables or None. States ------ No hook was registered '''''''''''''''''''''' sys.post_import_hooks contains no entry for the module A hook is registered and the module is not loaded yet ''''''''''''''''''''''''''''''''''''''''''''''''''''' The import hook registry contains an entry sys.post_import_hooks["name"] = [hook1] A module is successfully loaded ''''''''''''''''''''''''''''''' The import machinery checks if sys.post_import_hooks contains post import hooks for the newly loaded module. If hooks are found then the hooks are called in the order they were registered with the module instance as first argument. The processing of the hooks is stopped when a method raises an exception. At the end the entry for the module name is removed from sys.post_import_hooks, even when an error has occured. A module can't be loaded '''''''''''''''''''''''' The import hooks are neither called nor removed from the registry. It may be possible to load the module later. A hook is registered but the module is already loaded ''''''''''''''''''''''''''''''''''''''''''''''''''''' The hook is fired immediately. C API ----- New PyImport_* API functions '''''''''''''''''''''''''''' PyObject* PyImport_GetPostImportHooks(void) Returns the dict sys.post_import_hooks or NULL PyObject* PyImport_NotifyModuleLoaded(PyObject *module) Notify the post import system that a module was requested. Returns the module or NULL if an error has occured. PyObject* PyImport_RegisterPostImportHook(PyObject *callable, PyObject *mod_name) Register a new hook ``callable`` for the module ``mod_name`` The PyImport_PostImportNotify() method is called by PyImport_ImportModuleLevel():: PyImport_ImportModuleLevel(...) { ... result = import_module_level(name, globals, locals, fromlist, level); result = PyImport_PostImportNotify(result); ... } Python API ---------- The import hook registry and two new API methods are exposed through the ``sys`` and ``imp`` module. sys.post_import_hooks The dict contains the post import hooks: {"name" : [hook1, hook2], ...} imp.register_post_import_hook(hook: "callable", name: str) Register a new hook *hook* for the module *name* imp.notify_module_loaded(module: "module instance") -> module Notify the system that a module has been loaded. The method is provided for compatibility with existing lazy / deferred import extensions. The when_imported function decorator is also in the imp module, which is equivalent to: def when_imported(name): def register(hook): register_post_import_hook(hook, name) return register imp.when_imported(name) -> decorator function for @when_imported(name) def hook(module): pass Open issues =========== The when_imported decorator hasn't been written. The code contains several XXX comments. They are mostly about error handling in edge cases. Backwards Compatibility ======================= The new features and API don't conflict with old import system of Python and don't cause any backward compatibility issues for most software. However systems like PEAK and Zope which implement their own lazy import magic need to follow some rules. The post import hooks carefully designed to cooperate with existing deferred and lazy import systems. It's the suggestion of the PEP author to replace own on-load-hooks with the new hook API. The alternative lazy or deferred imports will still work but the implementations must call the ``imp.notify_module_loaded`` function. Reference Implementation ======================== A reference implementation is already written and is available in the *py3k-importhook* branch. [4]_ It still requires some cleanups, documentation updates and additional unit tests. Acknowledgments =============== Nick Coghlan, for proof reading and the initial discussion Phillip J. Eby, for his implementation in PEAK and help with my own implementation Copyright ========= This document has been placed in the public domain. References ========== .. [1] PEP: Lazy module imports and post import hook http://permalink.gmane.org/gmane.comp.python.devel/90949 .. [2] Interest in PEP for callbacks on module import http://permalink.gmane.org/gmane.comp.python.python-3000.devel/11126 .. [3] peak.utils.imports http://svn.eby-sarna.com/Importing/peak/util/imports.py?view=markup .. [4] py3k-importhook branch http://svn.python.org/view/python/branches/py3k-importhook/ .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: From pje at telecommunity.com Thu Jan 10 05:29:41 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 09 Jan 2008 23:29:41 -0500 Subject: [Python-Dev] PEP: Post import hooks In-Reply-To: References: Message-ID: <20080110043258.BEBAB3A40A9@sparrow.telecommunity.com> At 03:20 AM 1/10/2008 +0100, Christian Heimes wrote: >PyObject* PyImport_NotifyModuleLoaded(PyObject *module) > Notify the post import system that a module was requested. Returns the > module or NULL if an error has occured. The big problem here is that the interaction with laziness is actually pretty complex, when it comes to re-entrancy and initialization order. A package can actually import a submodule, and not yet be finished importing, for example. So you can actually have child hooks executing before parent hooks in this case. The "Importing" package prevents this by not registering child hooks until a parent is actually imported, thus guaranteeing a sane hook execution order. Relative order for hooks targeting the same module is maintained, but parent module hooks are guaranteed to execute before child hooks, even if the child finishes importing before the parent. This would be a bit trickier to implement with your C API, since "Importing" does this by registering a lot of lambdas. But, now that I've reviewed my own code and pieced back together the rationale for it doing things in this seemingly-odd way, it makes sense. There's also one twist that I haven't sorted out yet: "Importing" guarantees that a parent module 'foo' will have a 'bar' attribute for the 'foo.bar' module, if 'foo.bar' is lazy. It does this by registering a callback, ideally *before* any other callback is registered for 'foo' or 'foo.bar' that would look at 'foo.bar'. I don't see how to maintain this condition in a world where import callbacks can be registered independently. Bleah. All of the above isn't really a good explanation of the problem. Let me try to simplify it: * Lazy importing needs to guarantee that foo.bar = sys.modules['foo.bar'], when callbacks for 'foo.bar' execute (in case they refer to foo.bar) * To do this, it registers a callback that sets foo.bar = sys.modules['foo.bar'], and does not actually register any foo.bar callbacks until 'foo' is really imported (and thus foo.bar gets set by that callback) In the case of the PEP, it's harder for me to figure out what happens, because you might not have any lazy modules around, and the foo.bar issue would then not come up. You also have the possibility of a problem where a lazy import callback occurs in 3rd party code, while callbacks are occurring from the import machinery. (Which means that the notification API should probably set the hooks entry to None while it's running, so that if it's called from inside a hook, it will not double-run the hooks, and new hooks registered while hooks are running will get run immediately as they are encountered, instead of getting added to the list.) From nnorwitz at gmail.com Thu Jan 10 06:03:12 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 9 Jan 2008 21:03:12 -0800 Subject: [Python-Dev] Coverity Scan, Python upgraded to rung 2 In-Reply-To: <478538D4.1050103@cheimes.de> References: <18308.65413.671943.346078@montanaro.dyndns.org> <478538D4.1050103@cheimes.de> Message-ID: On Jan 9, 2008 1:12 PM, Christian Heimes wrote: > Thomas Heller wrote: > > Seems they are referring to the results of the rung 1 run (what ever 'rung' means ;-). > > With the account Neal made me some months ago, I can login on this page: > > > > http://scan.coverity.com:7475/ > > > > and see the scan results for Python. > > > > Last run at 2007-12-27: 11 Outstanding Defects, 6 of them marked "uninspected", > > 3 marked "pending", and 2 marked "bug". > > My dict says: rung (of a ladder)- Leitersprossen > > Python has climbed up one step (or rung) of the ladder. They botched the link where it says Sign in. Use the link Thomas posted, ie: http://scan.coverity.com:7475/ That will show you the results from the latest coverity checker. > Do you have the required permission to add more users to the site? I think only Coverity can add people. You can send them a message if you would like to be added: scan-admin at coverity.com. Or you can send mail to me and I can forward along all the people that would like to be added. I'll wait a few days to collect names so I can batch up the request. n From nnorwitz at gmail.com Thu Jan 10 06:11:21 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 9 Jan 2008 21:11:21 -0800 Subject: [Python-Dev] Coverity Scan, Python upgraded to rung 2 In-Reply-To: <18308.65413.671943.346078@montanaro.dyndns.org> References: <18308.65413.671943.346078@montanaro.dyndns.org> Message-ID: On Jan 9, 2008 9:08 AM, wrote: > > Christian> I read the announcement of the Python Users list and figured > Christian> out that some of the other core developers might be > Christian> interested in the news, too. > > Christian> Among other projects Python was upgraded to Rung 2 on the > Christian> Coverity Scan list: http://scan.coverity.com/ > > I went to the run2 page: > > http://scan.coverity.com/rung2.html > > I shows 6 uninspected defects for Python. How do we see what they are? > What is an uninspected defect? Any idea how the Coverity folks compute > Defects/KLOC? For example, how does tcl manage to get a 0.0 score? The 6 have been inspected by me and I never came to a conclusion of whether they were a problem or not. There are 3 things which should be fixed and I haven't gotten around to them. They are not a big deal: Python/traceback.c line 177 Objects/codeobject.c line 322 Modules/mmapmodule.c line 1080 For traceback.c, namebuf defined on line 155 should be moved out one block since filename is an alias to namebuf and it is used outside the current scope. I think this is unlikely to be a problem in practice, but is technically wrong and should be fixed. For codeobject.c, line 327 should not be reachable. I kinda like the code as it is even though it is currently dead. I never decided if I wanted to change that or suppress the warning. For mmapmodule.c, fd should be checked for -1 before calling stat on line 1064. The rest were not obvious problems to me, and I never returned to them. n From nnorwitz at gmail.com Thu Jan 10 06:19:01 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 9 Jan 2008 21:19:01 -0800 Subject: [Python-Dev] Coverity Scan, Python upgraded to rung 2 In-Reply-To: <18308.65413.671943.346078@montanaro.dyndns.org> References: <18308.65413.671943.346078@montanaro.dyndns.org> Message-ID: On Jan 9, 2008 9:08 AM, wrote: > > I went to the run2 page: > > http://scan.coverity.com/rung2.html > > I shows 6 uninspected defects for Python. How do we see what they are? > What is an uninspected defect? Any idea how the Coverity folks compute > Defects/KLOC? For example, how does tcl manage to get a 0.0 score? Sorry, I forgot to answer the second part of your question. I have no idea how they compute Defects/KLOC. But the data is very old so I wouldn't worry about what that says. The most recent run has 286622 lines in 602 files. I already mentioned the 3 defects that should be fixed. Not sure what to do about the rest. n From ncoghlan at gmail.com Thu Jan 10 10:16:53 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 10 Jan 2008 19:16:53 +1000 Subject: [Python-Dev] some bugs that need attention In-Reply-To: <932f8baf0801091358p4185fcb4r2c4350c0beaca83b@mail.gmail.com> References: <932f8baf0801091358p4185fcb4r2c4350c0beaca83b@mail.gmail.com> Message-ID: <4785E285.2050708@gmail.com> Ralf Schmitt wrote: > Hi, > > I've taken a look at some bugs in the bugtracker. I think these should > be closed: > > http://bugs.python.org/issue1720992 is about automatic imports. > > http://bugs.python.org/issue1448325 > and > http://bugs.python.org/issue1566086 > > is about the regular expression engine "hanging". These are duplicates of > http://bugs.python.org/issue1662581 and should be closed. To save anyone else needing to go and look: Christian and Georg have now taken care of closing these. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Thu Jan 10 10:22:58 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 10 Jan 2008 19:22:58 +1000 Subject: [Python-Dev] PEP: Post import hooks In-Reply-To: References: Message-ID: <4785E3F2.8030509@gmail.com> Christian Heimes wrote: > A module is successfully loaded > ''''''''''''''''''''''''''''''' > > The import machinery checks if sys.post_import_hooks contains post import > hooks for the newly loaded module. If hooks are found then the hooks are > called in the order they were registered with the module instance as first > argument. The processing of the hooks is stopped when a method raises an > exception. At the end the entry for the module name is removed from > sys.post_import_hooks, even when an error has occured. Doesn't the module remain in post_import_hooks, only mapped to None to indicate that any hooks should be run immediately? (Finding an object in sys.modules isn't enough, due to the possibility of it being a 3rd party lazy module rather than the actual module). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From p.f.moore at gmail.com Thu Jan 10 11:47:41 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 10 Jan 2008 10:47:41 +0000 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: References: <65501915-2A55-49E5-8711-B347BDD25887@python.org> <47849BC4.6020800@voidspace.org.uk> <79990c6b0801090358h4b92868ev98cb4b742a4e8f5d@mail.gmail.com> <4784D065.5040604@voidspace.org.uk> <79990c6b0801090601s705f539cw3acf2e02ccc7107f@mail.gmail.com> <4784D61E.8040307@cheimes.de> <79990c6b0801090718v4a272fc9nae2b6c732fe0aa6c@mail.gmail.com> <47852AEF.9060400@cheimes.de> <79990c6b0801091512m776fa278sc4cfd020d06d06fc@mail.gmail.com> Message-ID: <79990c6b0801100247x6ea6f1ccy8875c854af33d80@mail.gmail.com> On 09/01/2008, Steve Holden wrote: > The idea that users would /program their own computers/ was totally > alien to the Windows mindset. Actually, the alien idea is that more than one person would use the same (Windows) computer. Not surprising as these were *personal* computers. It's Windows as a server OS that's the bizarre idea... Paul. From amk at amk.ca Thu Jan 10 12:49:33 2008 From: amk at amk.ca (A.M. Kuchling) Date: Thu, 10 Jan 2008 06:49:33 -0500 Subject: [Python-Dev] Coverity Scan, Python upgraded to rung 2 In-Reply-To: References: <18308.65413.671943.346078@montanaro.dyndns.org> Message-ID: <20080110114933.GA1812@amk.local> On Wed, Jan 09, 2008 at 09:11:21PM -0800, Neal Norwitz wrote: > For mmapmodule.c, fd should be checked for -1 before calling stat on line 1064. I'll fix the mmap problem. --amk From amk at amk.ca Thu Jan 10 14:38:15 2008 From: amk at amk.ca (A.M. Kuchling) Date: Thu, 10 Jan 2008 08:38:15 -0500 Subject: [Python-Dev] Coverity Scan, Python upgraded to rung 2 In-Reply-To: References: <18308.65413.671943.346078@montanaro.dyndns.org> Message-ID: <20080110133815.GA6026@amk-desktop.matrixgroup.net> On Wed, Jan 09, 2008 at 09:11:21PM -0800, Neal Norwitz wrote: > For mmapmodule.c, fd should be checked for -1 before calling stat on line 1064. On looking at this, it doesn't seem like an actual problem. fstat(-1, ...) returns a -1 and errno is set to EBADF, 'bad file descriptor'. /* on OpenVMS we must ensure that all bytes are written to the file */ fsync(fd); # endif if (fstat(fd, &st) == 0 && S_ISREG(st.st_mode)) { ... In rev. 59888, I've added 'fd != -1' to the 'if' just to save a pointless fstat() call, and made the OpenVMS fsync() call similarly conditional, but I don't think this item is a bug, much less a security bug. I won't bother backporting this to 25-maint, unless asked to do so by the 2.5 maintainer. --amk From eric+python-dev at trueblade.com Thu Jan 10 14:31:54 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Thu, 10 Jan 2008 08:31:54 -0500 Subject: [Python-Dev] Backporting PEP 3101 to 2.6 Message-ID: <47861E4A.4020006@trueblade.com> (I'm posting to python-dev, because this isn't strictly 3.0 related. Hopefully most people read it in addition to python-3000). I'm working on backporting the changes I made for PEP 3101 (Advanced String Formatting) to the trunk, in order to meet the pre-PyCon release date for 2.6a1. I have a few questions about how I should handle str/unicode. 3.0 was pretty easy, because everything was unicode. 1: How should the builtin format() work? It takes 2 parameters, an object o and a string s, and returns o.__format__(s). If s is None, it returns o.__format__(empty_string). In 3.0, the empty string is of course unicode. For 2.6, should I use u'' or ''? 2: In 3.0, object.__format__() is essentially this: class object: def __format__(self, format_spec): return format(str(self), format_spec) In 2.6, I assume it should be the equivalent of: class object: def __format__(self, format_spec): if isinstance(format_spec, str): return format(str(self), format_spec) elif isinstance(format_spec, unicode): return format(unicode(self), format_spec) else: error Does that seem right? 3: Every overridden __format__() method is going to have to check for string or unicode, just like object.__format() does, and return either a string or unicode object, appropriately. I don't see any way around this, but I'd like to hear any thoughts. I guess there aren't all that many __format__ methods that will be implemented, so this might not be a big burden. I'll of course implement the built in ones. Thanks in advance for any insights. Eric. From mal at egenix.com Thu Jan 10 15:07:07 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 10 Jan 2008 15:07:07 +0100 Subject: [Python-Dev] Backporting PEP 3101 to 2.6 In-Reply-To: <47861E4A.4020006@trueblade.com> References: <47861E4A.4020006@trueblade.com> Message-ID: <4786268B.5010905@egenix.com> On 2008-01-10 14:31, Eric Smith wrote: > (I'm posting to python-dev, because this isn't strictly 3.0 related. > Hopefully most people read it in addition to python-3000). > > I'm working on backporting the changes I made for PEP 3101 (Advanced > String Formatting) to the trunk, in order to meet the pre-PyCon release > date for 2.6a1. > > I have a few questions about how I should handle str/unicode. 3.0 was > pretty easy, because everything was unicode. Since this is a new feature, why bother with strings at all (even in 2.6) ? Use Unicode throughout and be done with it. > 1: How should the builtin format() work? It takes 2 parameters, an > object o and a string s, and returns o.__format__(s). If s is None, it > returns o.__format__(empty_string). In 3.0, the empty string is of > course unicode. For 2.6, should I use u'' or ''? > > > 2: In 3.0, object.__format__() is essentially this: > > class object: > def __format__(self, format_spec): > return format(str(self), format_spec) > > In 2.6, I assume it should be the equivalent of: > > class object: > def __format__(self, format_spec): > if isinstance(format_spec, str): > return format(str(self), format_spec) > elif isinstance(format_spec, unicode): > return format(unicode(self), format_spec) > else: > error > > Does that seem right? > > > 3: Every overridden __format__() method is going to have to check for > string or unicode, just like object.__format() does, and return either a > string or unicode object, appropriately. I don't see any way around > this, but I'd like to hear any thoughts. I guess there aren't all that > many __format__ methods that will be implemented, so this might not be a > big burden. I'll of course implement the built in ones. > > Thanks in advance for any insights. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 10 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From barry at python.org Thu Jan 10 15:28:52 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 10 Jan 2008 09:28:52 -0500 Subject: [Python-Dev] Backporting PEP 3101 to 2.6 In-Reply-To: <4786268B.5010905@egenix.com> References: <47861E4A.4020006@trueblade.com> <4786268B.5010905@egenix.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 10, 2008, at 9:07 AM, M.-A. Lemburg wrote: > On 2008-01-10 14:31, Eric Smith wrote: >> (I'm posting to python-dev, because this isn't strictly 3.0 related. >> Hopefully most people read it in addition to python-3000). >> >> I'm working on backporting the changes I made for PEP 3101 (Advanced >> String Formatting) to the trunk, in order to meet the pre-PyCon >> release >> date for 2.6a1. >> >> I have a few questions about how I should handle str/unicode. 3.0 >> was >> pretty easy, because everything was unicode. > > Since this is a new feature, why bother with strings at all > (even in 2.6) ? > > Use Unicode throughout and be done with it. +1 - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBR4YrpHEjvBPtnXfVAQJcgwP+PV+XsqtZZ2aFA4yxIYRzkVVCyk+rwFSN H58DygPu4AQvhb1Dzuudag1OkfdpUHeRkvTyjSkUTWbK/03Y4R5A8X8iDkkQozQd m92DynvSEIOtX3WJZT4SOvGj+QavQC4FmkTPlEPNwqBkIl4GkjfOnwMsKx2lwKN+ rOXUf7Mtvd8= =1ME/ -----END PGP SIGNATURE----- From eric+python-dev at trueblade.com Thu Jan 10 15:13:28 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Thu, 10 Jan 2008 09:13:28 -0500 Subject: [Python-Dev] Backporting PEP 3101 to 2.6 In-Reply-To: <4786268B.5010905@egenix.com> References: <47861E4A.4020006@trueblade.com> <4786268B.5010905@egenix.com> Message-ID: <47862808.5060801@trueblade.com> M.-A. Lemburg wrote: > On 2008-01-10 14:31, Eric Smith wrote: >> (I'm posting to python-dev, because this isn't strictly 3.0 related. >> Hopefully most people read it in addition to python-3000). >> >> I'm working on backporting the changes I made for PEP 3101 (Advanced >> String Formatting) to the trunk, in order to meet the pre-PyCon release >> date for 2.6a1. >> >> I have a few questions about how I should handle str/unicode. 3.0 was >> pretty easy, because everything was unicode. > > Since this is a new feature, why bother with strings at all > (even in 2.6) ? > > Use Unicode throughout and be done with it. I was hoping someone would say that! It would certainly make things much easier. But for my own selfish reasons, I'd like to have str.format() work in 2.6. Other than the issues I raised here, I've already done the vast majority of the work for the code to support either string or unicode. For example, I put most of the implementation in Objects/stringlib, so I can include it either as string or unicode. But I can live with unicode only if that's the consensus. Eric. From lists at cheimes.de Thu Jan 10 16:54:24 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 10 Jan 2008 16:54:24 +0100 Subject: [Python-Dev] Coverity Scan, Python upgraded to rung 2 In-Reply-To: References: <18308.65413.671943.346078@montanaro.dyndns.org> Message-ID: <47863FB0.8040002@cheimes.de> Neal Norwitz wrote: > For traceback.c, namebuf defined on line 155 should be moved out one > block since filename is an alias to namebuf and it is used outside the > current scope. I think this is unlikely to be a problem in practice, > but is technically wrong and should be fixed. Agreed, the early allocation of a few hundreds bytes on the stack won't kill us. > For codeobject.c, line 327 should not be reachable. I kinda like the > code as it is even though it is currently dead. I never decided if I > wanted to change that or suppress the warning. Please suppress the warning. I removed the last two lines and GCC complained "control reaches end of non-void function". It's not clever enough to understand that cmp can never be 0. > For mmapmodule.c, fd should be checked for -1 before calling stat on line 1064. if (fd != -1 && fstat(fd, &st) == 0 && S_ISREG(st.st_mode)) Christian From lists at cheimes.de Thu Jan 10 16:57:40 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 10 Jan 2008 16:57:40 +0100 Subject: [Python-Dev] Coverity Scan, Python upgraded to rung 2 In-Reply-To: References: <18308.65413.671943.346078@montanaro.dyndns.org> <478538D4.1050103@cheimes.de> Message-ID: <47864074.2050301@cheimes.de> Neal Norwitz wrote: > I think only Coverity can add people. You can send them a message if > you would like to be added: scan-admin at coverity.com. Or you can send > mail to me and I can forward along all the people that would like to > be added. > > I'll wait a few days to collect names so I can batch up the request. Count me in! Christian From lists at cheimes.de Thu Jan 10 16:57:40 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 10 Jan 2008 16:57:40 +0100 Subject: [Python-Dev] Coverity Scan, Python upgraded to rung 2 In-Reply-To: References: <18308.65413.671943.346078@montanaro.dyndns.org> <478538D4.1050103@cheimes.de> Message-ID: <47864074.2050301@cheimes.de> Neal Norwitz wrote: > I think only Coverity can add people. You can send them a message if > you would like to be added: scan-admin at coverity.com. Or you can send > mail to me and I can forward along all the people that would like to > be added. > > I'll wait a few days to collect names so I can batch up the request. Count me in! Christian From josepharmbruster at gmail.com Thu Jan 10 17:01:01 2008 From: josepharmbruster at gmail.com (Joseph Armbruster) Date: Thu, 10 Jan 2008 11:01:01 -0500 Subject: [Python-Dev] Coverity Scan, Python upgraded to rung 2 In-Reply-To: <47864074.2050301@cheimes.de> References: <18308.65413.671943.346078@montanaro.dyndns.org> <478538D4.1050103@cheimes.de> <47864074.2050301@cheimes.de> Message-ID: <938f42d70801100801mdbd01d5l454b3c3ec7c7fb6c@mail.gmail.com> I am not a developer but i'm interested in browsing it. Is it possible to be added? On Jan 10, 2008 10:57 AM, Christian Heimes wrote: > Neal Norwitz wrote: > > I think only Coverity can add people. You can send them a message if > > you would like to be added: scan-admin at coverity.com. Or you can send > > mail to me and I can forward along all the people that would like to > > be added. > > > > I'll wait a few days to collect names so I can batch up the request. > > Count me in! > > Christian > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/josepharmbruster%40gmail.com > From pje at telecommunity.com Thu Jan 10 18:30:04 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 10 Jan 2008 12:30:04 -0500 Subject: [Python-Dev] PEP: Post import hooks In-Reply-To: <4785E3F2.8030509@gmail.com> References: <4785E3F2.8030509@gmail.com> Message-ID: <20080110173007.760C73A406C@sparrow.telecommunity.com> At 07:22 PM 1/10/2008 +1000, Nick Coghlan wrote: >Christian Heimes wrote: > > A module is successfully loaded > > ''''''''''''''''''''''''''''''' > > > > The import machinery checks if sys.post_import_hooks contains post import > > hooks for the newly loaded module. If hooks are found then the hooks are > > called in the order they were registered with the module instance as first > > argument. The processing of the hooks is stopped when a method raises an > > exception. At the end the entry for the module name is removed from > > sys.post_import_hooks, even when an error has occured. > >Doesn't the module remain in post_import_hooks, only mapped to None to >indicate that any hooks should be run immediately? It should be, yes. From pje at telecommunity.com Thu Jan 10 18:33:02 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 10 Jan 2008 12:33:02 -0500 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <79990c6b0801100247x6ea6f1ccy8875c854af33d80@mail.gmail.com > References: <65501915-2A55-49E5-8711-B347BDD25887@python.org> <47849BC4.6020800@voidspace.org.uk> <79990c6b0801090358h4b92868ev98cb4b742a4e8f5d@mail.gmail.com> <4784D065.5040604@voidspace.org.uk> <79990c6b0801090601s705f539cw3acf2e02ccc7107f@mail.gmail.com> <4784D61E.8040307@cheimes.de> <79990c6b0801090718v4a272fc9nae2b6c732fe0aa6c@mail.gmail.com> <47852AEF.9060400@cheimes.de> <79990c6b0801091512m776fa278sc4cfd020d06d06fc@mail.gmail.com> <79990c6b0801100247x6ea6f1ccy8875c854af33d80@mail.gmail.com> Message-ID: <20080110173304.04EA63A406C@sparrow.telecommunity.com> At 10:47 AM 1/10/2008 +0000, Paul Moore wrote: >On 09/01/2008, Steve Holden wrote: > > The idea that users would /program their own computers/ was totally > > alien to the Windows mindset. > >Actually, the alien idea is that more than one person would use the >same (Windows) computer. Not surprising as these were *personal* >computers. It's Windows as a server OS that's the bizarre idea... multiuser != server I've been in a couple of organizations where PCs were shared and thus had different users logging into them. Cash registers and call centers, for example. From eric+python-dev at trueblade.com Thu Jan 10 18:57:58 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Thu, 10 Jan 2008 12:57:58 -0500 Subject: [Python-Dev] Backporting PEP 3101 to 2.6 In-Reply-To: <47861E4A.4020006@trueblade.com> References: <47861E4A.4020006@trueblade.com> Message-ID: <47865CA6.2070400@trueblade.com> Eric Smith wrote: > (I'm posting to python-dev, because this isn't strictly 3.0 related. > Hopefully most people read it in addition to python-3000). > > I'm working on backporting the changes I made for PEP 3101 (Advanced > String Formatting) to the trunk, in order to meet the pre-PyCon release > date for 2.6a1. > > I have a few questions about how I should handle str/unicode. 3.0 was > pretty easy, because everything was unicode. > > 1: How should the builtin format() work? It takes 2 parameters, an > object o and a string s, and returns o.__format__(s). If s is None, it > returns o.__format__(empty_string). In 3.0, the empty string is of > course unicode. For 2.6, should I use u'' or ''? I just re-read PEP 3101, and it doesn't mention this behavior with None. The way the code actually works is that the specifier is optional, and if it isn't present then it defaults to an empty string. This behavior isn't mentioned in the PEP, either. This feature came from a request from Talin[0]. We should either add this to the PEP (and docs), or remove it. If we document it, it should mention the 2.x behavior (as other places in the PEP do). If we removed it, it would remove the one place in the backport that's not just hard, but ambiguous. I'd just as soon see the feature go away, myself. > 3: Every overridden __format__() method is going to have to check for > string or unicode, just like object.__format() does, and return either a > string or unicode object, appropriately. I don't see any way around > this, but I'd like to hear any thoughts. I guess there aren't all that > many __format__ methods that will be implemented, so this might not be a > big burden. I'll of course implement the built in ones. The PEP actually mentions that this is how 2.x will have to work. So I'll go ahead and implement it that way, on the assumption that getting string support into 2.6 is desirable. Eric. [0] http://mail.python.org/pipermail/python-3000/2007-August/010089.html From guido at python.org Thu Jan 10 19:08:58 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 10 Jan 2008 10:08:58 -0800 Subject: [Python-Dev] Backporting PEP 3101 to 2.6 In-Reply-To: <47865CA6.2070400@trueblade.com> References: <47861E4A.4020006@trueblade.com> <47865CA6.2070400@trueblade.com> Message-ID: On Jan 10, 2008 9:57 AM, Eric Smith wrote: > Eric Smith wrote: > > (I'm posting to python-dev, because this isn't strictly 3.0 related. > > Hopefully most people read it in addition to python-3000). > > > > I'm working on backporting the changes I made for PEP 3101 (Advanced > > String Formatting) to the trunk, in order to meet the pre-PyCon release > > date for 2.6a1. > > > > I have a few questions about how I should handle str/unicode. 3.0 was > > pretty easy, because everything was unicode. > > > > 1: How should the builtin format() work? It takes 2 parameters, an > > object o and a string s, and returns o.__format__(s). If s is None, it > > returns o.__format__(empty_string). In 3.0, the empty string is of > > course unicode. For 2.6, should I use u'' or ''? > > I just re-read PEP 3101, and it doesn't mention this behavior with None. > The way the code actually works is that the specifier is optional, and > if it isn't present then it defaults to an empty string. This behavior > isn't mentioned in the PEP, either. > > This feature came from a request from Talin[0]. We should either add > this to the PEP (and docs), or remove it. If we document it, it should > mention the 2.x behavior (as other places in the PEP do). If we removed > it, it would remove the one place in the backport that's not just hard, > but ambiguous. I'd just as soon see the feature go away, myself. IIUC, the 's' argument is the format specifier. Format specifiers are written in a very conservative character set, so I'm not sure it matters. Or are you assuming that the *type* of 's' also determines the type of the output? I may be in the minority here, but I think I like having a default for 's' (as implemented -- the PEP ought to be updated) and I also think it should default to an 8-bit string, assuming you support 8-bit strings at all -- after all in 2.x 8-bit strings are the default string type (as reflected by their name, 'str'). > > 3: Every overridden __format__() method is going to have to check for > > string or unicode, just like object.__format() does, and return either a > > string or unicode object, appropriately. I don't see any way around > > this, but I'd like to hear any thoughts. I guess there aren't all that > > many __format__ methods that will be implemented, so this might not be a > > big burden. I'll of course implement the built in ones. > > The PEP actually mentions that this is how 2.x will have to work. So > I'll go ahead and implement it that way, on the assumption that getting > string support into 2.6 is desirable. I think it is. (But then I still live in a predominantly ASCII world. :-) For data types whose output uses only ASCII, would it be acceptable if they always returned an 8-bit string and left it up to the caller to convert it to Unicode? This would apply to all numeric types. (The date/time types have a strftime() style API which means the user must be able to specifiy Unicode.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From eric+python-dev at trueblade.com Thu Jan 10 20:12:14 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Thu, 10 Jan 2008 14:12:14 -0500 Subject: [Python-Dev] Backporting PEP 3101 to 2.6 In-Reply-To: References: <47861E4A.4020006@trueblade.com> <47865CA6.2070400@trueblade.com> Message-ID: <47866E0E.8090605@trueblade.com> Guido van Rossum wrote: > On Jan 10, 2008 9:57 AM, Eric Smith wrote: >> Eric Smith wrote: >>> 1: How should the builtin format() work? It takes 2 parameters, an >>> object o and a string s, and returns o.__format__(s). If s is None, it >>> returns o.__format__(empty_string). In 3.0, the empty string is of >>> course unicode. For 2.6, should I use u'' or ''? >> I just re-read PEP 3101, and it doesn't mention this behavior with None. >> The way the code actually works is that the specifier is optional, and >> if it isn't present then it defaults to an empty string. This behavior >> isn't mentioned in the PEP, either. >> >> This feature came from a request from Talin[0]. We should either add >> this to the PEP (and docs), or remove it. If we document it, it should >> mention the 2.x behavior (as other places in the PEP do). If we removed >> it, it would remove the one place in the backport that's not just hard, >> but ambiguous. I'd just as soon see the feature go away, myself. > > IIUC, the 's' argument is the format specifier. Format specifiers are > written in a very conservative character set, so I'm not sure it > matters. Or are you assuming that the *type* of 's' also determines > the type of the output? Yes, 's' is the format specifier. I should have used its actual name. I'm am saying that the type of 's' determines the type of the output. Maybe that's a needless assumption for the builtin format(), since it doesn't inspect the value of 's' (other than to verify its type). But for ''.format() and u''.format(), I was thinking it will be true (but see below). It just seems weird to me that the result of format(3, u'd') would be a '3', not u'3'. > I may be in the minority here, but I think I like having a default for > 's' (as implemented -- the PEP ought to be updated) and I also think > it should default to an 8-bit string, assuming you support 8-bit > strings at all -- after all in 2.x 8-bit strings are the default > string type (as reflected by their name, 'str'). As long as it's defined, I'm okay with it. I think making the 2.6 default be an empty str is reasonable. >>> 3: Every overridden __format__() method is going to have to check for >>> string or unicode, just like object.__format() does, and return either a >>> string or unicode object, appropriately. I don't see any way around >>> this, but I'd like to hear any thoughts. I guess there aren't all that >>> many __format__ methods that will be implemented, so this might not be a >>> big burden. I'll of course implement the built in ones. >> The PEP actually mentions that this is how 2.x will have to work. So >> I'll go ahead and implement it that way, on the assumption that getting >> string support into 2.6 is desirable. > > I think it is. (But then I still live in a predominantly ASCII world. :-) I live in that same world, which is why I started implementing this to begin with! I've always been more interested in the ascii version for 2.6 than for the 3.0 unicode version. Doing it first in 3.0 was my way of getting it into 2.6. > For data types whose output uses only ASCII, would it be acceptable if > they always returned an 8-bit string and left it up to the caller to > convert it to Unicode? This would apply to all numeric types. (The > date/time types have a strftime() style API which means the user must > be able to specifiy Unicode.) I guess in str.format() I could require the result of format(obj, format_spec) to be a str, and in unicode.format() I could convert it to be unicode, which would either succeed or fail. I think all I need to do is have the numeric formatters work with both unicode and str format specifiers, and always return str results. That should be doable. As you say, the format specifiers for the numerics are restricted to 8-bit strings, anyway. Now that I think about it, the str .__format__() will also need to accept unicode and produce a str, for this to work: u"{0}{1}{2}".format('a', u'b', 3) I'll give these ideas a shot and see how far I get. Thanks for the feedback! Eric. From lists at cheimes.de Thu Jan 10 21:40:05 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 10 Jan 2008 21:40:05 +0100 Subject: [Python-Dev] PEP: Post import hooks In-Reply-To: <20080110043258.BEBAB3A40A9@sparrow.telecommunity.com> References: <20080110043258.BEBAB3A40A9@sparrow.telecommunity.com> Message-ID: Phillip J. Eby wrote: [...] > There's also one twist that I haven't sorted out yet: "Importing" > guarantees that a parent module 'foo' will have a 'bar' attribute for > the 'foo.bar' module, if 'foo.bar' is lazy. It does this by > registering a callback, ideally *before* any other callback is > registered for 'foo' or 'foo.bar' that would look at 'foo.bar'. I > don't see how to maintain this condition in a world where import > callbacks can be registered independently. I've moved the PyImport_NotifyModuleLoaded() call to import_submodule(). It (should) guarantee that the hooks for a parent package is called before the hooks for its children are called. I've analyzed the code carefully enough to be sure but all unit test results are on my side. On other words "import a.b.c" fires the hooks for "a", then "a.b" and at last "a.b.c". I could also modify imp.notify_module_loaded to accepts the module name as string ("a.b.c."). If the module is provided by name (e.g. "a.b.c.") rather than by object it makes sure that the hooks for "a", "a.b" and "a.b.c" are called in the right order. > Bleah. All of the above isn't really a good explanation of the > problem. Let me try to simplify it: > > * Lazy importing needs to guarantee that foo.bar = > sys.modules['foo.bar'], when callbacks for 'foo.bar' execute (in case > they refer to foo.bar) > > * To do this, it registers a callback that sets foo.bar = > sys.modules['foo.bar'], and does not actually register any foo.bar > callbacks until 'foo' is really imported (and thus foo.bar gets set > by that callback) Would the modification fulfill your needs if imp.notify_module_loaded("foo.bar.baz") call the hooks for "foo", "foo.bar" and "foo.bar.baz" in that order? > In the case of the PEP, it's harder for me to figure out what > happens, because you might not have any lazy modules around, and the > foo.bar issue would then not come up. You also have the possibility > of a problem where a lazy import callback occurs in 3rd party code, > while callbacks are occurring from the import machinery. (Which > means that the notification API should probably set the hooks entry > to None while it's running, so that if it's called from inside a > hook, it will not double-run the hooks, and new hooks registered > while hooks are running will get run immediately as they are > encountered, instead of getting added to the list.) The initial design used to set the hooks to None *after* the hooks were called. I removed code yesterday because I thought it's not required. Today I've re-added the checks for Py_None. I'm not setting the hooks to Py_None before the hook are called. Christian From pje at telecommunity.com Thu Jan 10 23:32:28 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 10 Jan 2008 17:32:28 -0500 Subject: [Python-Dev] PEP: Post import hooks In-Reply-To: References: <20080110043258.BEBAB3A40A9@sparrow.telecommunity.com> Message-ID: <20080110223235.13D5B3A406C@sparrow.telecommunity.com> At 09:40 PM 1/10/2008 +0100, Christian Heimes wrote: >Phillip J. Eby wrote: >[...] > > > There's also one twist that I haven't sorted out yet: "Importing" > > guarantees that a parent module 'foo' will have a 'bar' attribute for > > the 'foo.bar' module, if 'foo.bar' is lazy. It does this by > > registering a callback, ideally *before* any other callback is > > registered for 'foo' or 'foo.bar' that would look at 'foo.bar'. I > > don't see how to maintain this condition in a world where import > > callbacks can be registered independently. > >I've moved the PyImport_NotifyModuleLoaded() call to import_submodule(). >It (should) guarantee that the hooks for a parent package is called >before the hooks for its children are called. I've analyzed the code >carefully enough to be sure but all unit test results are on my side. > >On other words "import a.b.c" fires the hooks for "a", then "a.b" and at >last "a.b.c". Yes, that's the general idea. But what happens if you are in the middle of firing hooks for 'a', and a new hook for 'a.b.c' is added? What about a new hook for 'a'? >I could also modify imp.notify_module_loaded to accepts the module name >as string ("a.b.c."). If the module is provided by name (e.g. "a.b.c.") >rather than by object it makes sure that the hooks for "a", "a.b" and >"a.b.c" are called in the right order. Well, it certainly can (and should) do the same if a module object is provided, since the module has a __name__. >Would the modification fulfill your needs if >imp.notify_module_loaded("foo.bar.baz") call the hooks for "foo", >"foo.bar" and "foo.bar.baz" in that order? Only if you can guarantee that no hook for a submodule is run until all the parent hooks are finished being called *and* that adding new callbacks while callbacks are being run will still call them... *after* any already-added callbacks. >The initial design used to set the hooks to None *after* the hooks were >called. I removed code yesterday because I thought it's not required. >Today I've re-added the checks for Py_None. In general, if you think something in peak.util.imports isn't required, you're probably wrong. ;) >I'm not setting the hooks to Py_None before the hook are called. That's fine, but here's a different catch: are you iterating over the hooks by taking the length before you start? If so, then hooks that are added *while* the hooks are being called back, will never get called, because they'll be added to the end of the list (and you'll never reach the end). Make sure there's a test for that case. peak.util.imports sets to None after callbacks, but it uses regular list iteration, so hooks can be added to the end of the list while the hooks are still being called. An error while running the hooks should also set the hook list to None and discard all the hooks. There isn't any sane way to recover from an error in a post-import hook. From snaury at gmail.com Thu Jan 10 23:33:51 2008 From: snaury at gmail.com (Alexey Borzenkov) Date: Fri, 11 Jan 2008 01:33:51 +0300 Subject: [Python-Dev] distutils.cygwinccompiler: invalid executable for interpreters Message-ID: Hi everyone, Some time ago I've stumbled upon a problem with compiling py2exe with mingw: it resulted in invalid executable run.exe. At first I dismissed it as some very strange problem, but recently I decided to look into it again and found that the reason for this is a .def file, supplied during compilation, that causes linker to produce executable with IMAGE_FILE_DLL in its PE Header Characteristics field, and operating system refuses to execute it. I traced this problem to the following block in distutils/cygwinccompiler.py: # handle export symbols by creating a def-file # with executables this only works with gcc/ld as linker if ((export_symbols is not None) and (target_desc != self.EXECUTABLE or self.linker_dll == "gcc")): Removing 'or self.linker_dll == "gcc"' obviously helps. To me this sounds like a bug (introduced in revision 17747 on trunk, more that 7 years old!), but I was wondering if there's any logic behind generating a .def file for the executable (sidenote: look at distutils/emxccompiler.py where .def file is generated for executables only). To me it's very unlikely that anyone *ever* needs to export symbols from an executable (I'm not even sure if it can work as intended on Windows). Even then, if there's anyone at all who needs such a machinery (again, what for?), at the very least it shouldn't produce "LIBRARY" line. As minimal change as this: # Generate .def file contents = [ "LIBRARY %s" % os.path.basename(output_filename), "EXPORTS"] if target_desc == self.EXECUTABLE: contents.pop(0) Did the trick (generated executable runs fine). Is anyone interested in a bug report for this? Also, is there any chance that if there is interest it could end up in release25-maint? Best regards, Alexey. From lists at cheimes.de Thu Jan 10 23:45:41 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 10 Jan 2008 23:45:41 +0100 Subject: [Python-Dev] PEP: Post import hooks In-Reply-To: <20080110223235.13D5B3A406C@sparrow.telecommunity.com> References: <20080110043258.BEBAB3A40A9@sparrow.telecommunity.com> <20080110223235.13D5B3A406C@sparrow.telecommunity.com> Message-ID: <4786A015.1000306@cheimes.de> Phillip J. Eby wrote: >> I'm not setting the hooks to Py_None before the hook are called. Err, make that NOW, not NOT ... stupid typo. I'm NOW setting the hooks to Py_None before the hooks are called. > That's fine, but here's a different catch: are you iterating over the > hooks by taking the length before you start? If so, then hooks that are > added *while* the hooks are being called back, will never get called, > because they'll be added to the end of the list (and you'll never reach > the end). Make sure there's a test for that case. it = iter(self.post_import_registry[name]) self.post_import_registry[name] = None for hook in it: hook(module) > peak.util.imports sets to None after callbacks, but it uses regular list > iteration, so hooks can be added to the end of the list while the hooks > are still being called. In my version a hook is immediately called when the the registry value is set to None. When a hook is registered for a module during the execution of the callback then the hook is fired directly and not after the existing hooks are called. Is this a problem for you? module "foo" is loaded: hook1 hook2 -> registers hookX for "foo" hookX is called directly hook3 hook4 > An error while running the hooks should also set the hook list to None > and discard all the hooks. There isn't any sane way to recover from an > error in a post-import hook. The hooks are set to None even when an exception is raised by a hook. Christian From lists at cheimes.de Fri Jan 11 00:08:41 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 11 Jan 2008 00:08:41 +0100 Subject: [Python-Dev] PEP: Post import hooks In-Reply-To: <20080110223235.13D5B3A406C@sparrow.telecommunity.com> References: <20080110043258.BEBAB3A40A9@sparrow.telecommunity.com> <20080110223235.13D5B3A406C@sparrow.telecommunity.com> Message-ID: <4786A579.2070605@cheimes.de> Phillip J. Eby wrote: > Yes, that's the general idea. But what happens if you are in the middle > of firing hooks for 'a', and a new hook for 'a.b.c' is added? What > about a new hook for 'a'? If 'a' registers a new hook for a child of 'a' (e.g. 'a.b.c' or 'a.f') then the new hooks are called with the remaining hooks for 'a.b.c': import a.b.c * hook_a1 * hook_a1 -> registers the hook_ab2 for 'a.b' * hook_ab1 -> registers a hook_aX for 'a' hook_aX is fired immediately * hook_ab2 -- the hook registered by hook_a1 > Well, it certainly can (and should) do the same if a module object is > provided, since the module has a __name__. Maybe I should add two methods to imp. One that calls the parent hooks of a module automatically but relies on the existence of the parents and the module in sys.modules. And a second method which calls the hooks for a module object w/o inspecting sys.modules. > Only if you can guarantee that no hook for a submodule is run until all > the parent hooks are finished being called *and* that adding new > callbacks while callbacks are being run will still call them... *after* > any already-added callbacks. Uhm, now it starts to become a mind bending problem. I may have to queue and delay the registration of new hooks while other hooks are called by the system. If an user registers a callback for 'a' while the callbacks for 'a' are called than the registration is queued and the called after the remaining hooks for 'a' are called. This could probably be implemented by *not* setting the entry to None after I get hold of the iterator but after the end of the iteration. iter() notices when a new element is appended. In the above sample can hook_aX be called after hook_ab1 or should it be called after hook_ab2? > In general, if you think something in peak.util.imports isn't required, > you're probably wrong. ;) *g* ok. I'll check your implementation again. Christian From pje at telecommunity.com Fri Jan 11 00:10:08 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 10 Jan 2008 18:10:08 -0500 Subject: [Python-Dev] PEP: Post import hooks In-Reply-To: <4786A015.1000306@cheimes.de> References: <20080110043258.BEBAB3A40A9@sparrow.telecommunity.com> <20080110223235.13D5B3A406C@sparrow.telecommunity.com> <4786A015.1000306@cheimes.de> Message-ID: <20080110231011.1F0883A406C@sparrow.telecommunity.com> At 11:45 PM 1/10/2008 +0100, Christian Heimes wrote: >In my version a hook is immediately called when the the registry value >is set to None. When a hook is registered for a module during the >execution of the callback then the hook is fired directly and not after >the existing hooks are called. Is this a problem for you? Yes, because it violates the invariant that hooks for a given module are called in the same order that they were registered in. In case you're wondering why it's a problem, it's because if a hook imports something that registers a hook, when previously that thing wasn't imported until later, all of a sudden your callback order is very different than what it was before. More succinctly: if making small changes to your program can cause large differences in the result, it's hard to debug. (Especially if you have no idea what 3rd party module changed its import order and messed things up.) Believe me, it's a lot easier to debug if there is a globally understandable hook order, even if it's still a partial ordering. From lists at cheimes.de Fri Jan 11 01:47:38 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 11 Jan 2008 01:47:38 +0100 Subject: [Python-Dev] PEP: Post import hooks In-Reply-To: <20080110231011.1F0883A406C@sparrow.telecommunity.com> References: <20080110043258.BEBAB3A40A9@sparrow.telecommunity.com> <20080110223235.13D5B3A406C@sparrow.telecommunity.com> <4786A015.1000306@cheimes.de> <20080110231011.1F0883A406C@sparrow.telecommunity.com> Message-ID: <4786BCAA.6080501@cheimes.de> Phillip J. Eby wrote: > At 11:45 PM 1/10/2008 +0100, Christian Heimes wrote: >> In my version a hook is immediately called when the the registry value >> is set to None. When a hook is registered for a module during the >> execution of the callback then the hook is fired directly and not after >> the existing hooks are called. Is this a problem for you? > > Yes, because it violates the invariant that hooks for a given module > are called in the same order that they were registered in. Please check the changes and the new unit test in r59902 ( http://svn.python.org/view?rev=59902&view=rev ). Are you satisfied with the ordering or do you think I should queue the hooks for already loaded modules? Christian From pje at telecommunity.com Fri Jan 11 01:54:25 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 10 Jan 2008 19:54:25 -0500 Subject: [Python-Dev] PEP: Post import hooks In-Reply-To: <4786BCAA.6080501@cheimes.de> References: <20080110043258.BEBAB3A40A9@sparrow.telecommunity.com> <20080110223235.13D5B3A406C@sparrow.telecommunity.com> <4786A015.1000306@cheimes.de> <20080110231011.1F0883A406C@sparrow.telecommunity.com> <4786BCAA.6080501@cheimes.de> Message-ID: <20080111005429.0518B3A406C@sparrow.telecommunity.com> At 01:47 AM 1/11/2008 +0100, Christian Heimes wrote: >Phillip J. Eby wrote: > > At 11:45 PM 1/10/2008 +0100, Christian Heimes wrote: > >> In my version a hook is immediately called when the the registry value > >> is set to None. When a hook is registered for a module during the > >> execution of the callback then the hook is fired directly and not after > >> the existing hooks are called. Is this a problem for you? > > > > Yes, because it violates the invariant that hooks for a given module > > are called in the same order that they were registered in. > >Please check the changes and the new unit test in r59902 ( >http://svn.python.org/view?rev=59902&view=rev ). Are you satisfied with >the ordering or do you think I should queue the hooks for already loaded >modules? As I said above, calling the callbacks immediately is a problem, because it leads to significantly less predictability (and therefore control) of callback order. From pje at telecommunity.com Fri Jan 11 02:05:44 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 10 Jan 2008 20:05:44 -0500 Subject: [Python-Dev] PEP: Post import hooks In-Reply-To: <4786A579.2070605@cheimes.de> References: <20080110043258.BEBAB3A40A9@sparrow.telecommunity.com> <20080110223235.13D5B3A406C@sparrow.telecommunity.com> <4786A579.2070605@cheimes.de> Message-ID: <20080111010557.63A2B3A406C@sparrow.telecommunity.com> At 12:08 AM 1/11/2008 +0100, Christian Heimes wrote: >Phillip J. Eby wrote: > > Yes, that's the general idea. But what happens if you are in the middle > > of firing hooks for 'a', and a new hook for 'a.b.c' is added? What > > about a new hook for 'a'? > >If 'a' registers a new hook for a child of 'a' (e.g. 'a.b.c' or 'a.f') >then the new hooks are called with the remaining hooks for 'a.b.c': > >import a.b.c >* hook_a1 >* hook_a1 -> registers the hook_ab2 for 'a.b' >* hook_ab1 -> registers a hook_aX for 'a' > hook_aX is fired immediately >* hook_ab2 -- the hook registered by hook_a1 This scenario isn't specific enough to produce/deduce the problem. > > Well, it certainly can (and should) do the same if a module object is > > provided, since the module has a __name__. > >Maybe I should add two methods to imp. One that calls the parent hooks >of a module automatically but relies on the existence of the parents and >the module in sys.modules. >And a second method which calls the hooks for a module object w/o >inspecting sys.modules. *sigh*. We seem to be getting further and further off course, here. The more different you make the semantics of the system, the harder it will be to verify that it's doing the right thing without having real field experience with the new approach. > > Only if you can guarantee that no hook for a submodule is run until all > > the parent hooks are finished being called *and* that adding new > > callbacks while callbacks are being run will still call them... *after* > > any already-added callbacks. > >Uhm, now it starts to become a mind bending problem. I may have to queue >and delay the registration of new hooks while other hooks are called by >the system. If an user registers a callback for 'a' while the callbacks >for 'a' are called than the registration is queued and the called after >the remaining hooks for 'a' are called. This could probably be >implemented by *not* setting the entry to None after I get hold of the >iterator but after the end of the iteration. iter() notices when a new >element is appended. Yep - that's precisely what peak.util.imports does, and is why it does it that way. However, it has other ways of guaranteeing that the notification callback occurs only once, because the module object keeps track of that. >In the above sample can hook_aX be called after hook_ab1 or should it be >called after hook_ab2? That question can't be answered from the limited information you supplied about the scenario. What must not happen is that hook_aX must not be called before any *already-registered* hooks for 'a'. From Scott.Daniels at Acm.Org Fri Jan 11 03:16:55 2008 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Thu, 10 Jan 2008 18:16:55 -0800 Subject: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages In-Reply-To: <333D85CA-FFBB-4957-B3B3-36C9D8B4F4CC@reedobrien.com> References: <65501915-2A55-49E5-8711-B347BDD25887@python.org> <20080107163702.ACF603A40B0@sparrow.telecommunity.com> <2A83DC81-BC55-4282-803D-6E835AF6F5C9@python.org> <20080107224946.GA10908@mithrandi.za.net> <79990c6b0801071456t41a27986y4fb32898b604b8d6@mail.gmail.com> <4782E01D.1040401@cheimes.de> <79990c6b0801080147x692ca5ek838b16e889011779@mail.gmail.com> <47839FBF.10301@cheimes.de> <20080109064839.GM75977@nexus.in-nomine.org> <333D85CA-FFBB-4957-B3B3-36C9D8B4F4CC@reedobrien.com> Message-ID: Reed O'Brien wrote: > On Jan 9, 2008, at 1:48 AM, Jeroen Ruigrok van der Werven wrote: >> -On [20080108 17:07], Christian Heimes (lists at cheimes.de) wrote: >>> Python's _winreg module and pywin32 expose several functions to >>> get the paths from the registry but I don't think it has a simple >>> function like get_mydocuments(). >> Careful with the name though. Microsoft Windows Vista did away with >> 'My Documents & Settings'. It is now C:\Users. >> >> So you get: >> C:\Users\\AppData\Local\ (former Local Settings\Application Data) >> C:\Users\\AppData\Roaming\ (former Application Data) >> C:\Users\\Documents (former My Documents) >> C:\Users\\Music (former My Music) >> C:\Users\\Pictures (former My Pictures) >> C:\Users\\Videos (former My Videos) > yay, next up posix support.... I suspect that the whole thing was done to make sure that developers of applications could: A: cope with stupidly long path names. V: cope with spaces in path names. I bet they never intended to keep the huge names, just to make you cope with them. --Scott David Daniels Scott.Daniels at Acm.Org From nnorwitz at gmail.com Fri Jan 11 05:00:47 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 10 Jan 2008 20:00:47 -0800 Subject: [Python-Dev] Coverity Scan, Python upgraded to rung 2 In-Reply-To: <938f42d70801100801mdbd01d5l454b3c3ec7c7fb6c@mail.gmail.com> References: <18308.65413.671943.346078@montanaro.dyndns.org> <478538D4.1050103@cheimes.de> <47864074.2050301@cheimes.de> <938f42d70801100801mdbd01d5l454b3c3ec7c7fb6c@mail.gmail.com> Message-ID: On Jan 10, 2008 8:01 AM, Joseph Armbruster wrote: > I am not a developer but i'm interested in browsing it. Is it > possible to be added? Yes, I've added you to the list. I'll probably send the list off tomorrow, so let me know if you would like to be added. n -- > > > On Jan 10, 2008 10:57 AM, Christian Heimes wrote: > > Neal Norwitz wrote: > > > I think only Coverity can add people. You can send them a message if > > > you would like to be added: scan-admin at coverity.com. Or you can send > > > mail to me and I can forward along all the people that would like to > > > be added. > > > > > > I'll wait a few days to collect names so I can batch up the request. > > > > Count me in! > > > > Christian > > > > _______________________________________________ > > Python-Dev mailing list > > Python-Dev at python.org > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: http://mail.python.org/mailman/options/python-dev/josepharmbruster%40gmail.com > > > From vinay_sajip at red-dove.com Fri Jan 11 11:44:45 2008 From: vinay_sajip at red-dove.com (Vinay Sajip at Red Dove) Date: Fri, 11 Jan 2008 10:44:45 -0000 Subject: [Python-Dev] Passing contextual information when logging Message-ID: <001101c8543e$fffa04e0$0800a8c0@DELTA> I recently posted the following to c.l.py, and didn't get much feedback. I'm planning to add a new class to the logging package, and before I do so I'd like to get feedback from python-dev. I know many of the devs may not be monitoring c.l.py, so I'm reposting here for your feedback. Some users of the logging package have raised an issue regarding the difficulty of passing additional contextual information when logging. For example, the developer of a networked application may want to log, in addition to specifics related to to the network service being provided, information about the IP address of the remote machine and the username of the person logged into and using the service. Python 2.4 introduced an 'extra' keyword argument which was intended to hold a dict-like object containing additional information to be added to a LogRecord. The additional information would then be printed via placeholders in a format string. While this works, it is a little unwieldy to use in practice, because users need to provide the 'extra' parameter in every logging call. This has led people in some instances to create context-specific Logger instances (e.g. one logger for every connection). This has a drawback in that a logger name can only provide limited contextual information, and moreover, if the number of connections is effectively unbounded over time, the number of Logger instances will also grow in an unbounded way. (Logger instances are never garbage collected, because references to them are always held in the logging package. This alleviates a burden on users in that they never have to pass loggers around, but means that creating a lot of Logger instances will lead to a long-lived memory burden.) One solution is to create a generic wrapper around loggers to which a logger name and contextual information can be passed. The wrapper would delegate logging calls to the logger with the specified name, but would manipulate the arguments passed to the logging call to insert the contextual information. I have created such a wrapper class, called LoggerAdapter, which is in the example script located at http://dpaste.com/30613/ I would welcome your views on whether the LoggerAdapter class is suitable for adding to the logging package in Python 2.6/3.0. Does it do what might reasonably be expected out of the box? LoggerAdapters are, of course, garbage collected in the normal way and so impose no particular memory burden. I'll wait a few days for any comments/suggestions from this list, then (assuming no adverse comments) go ahead and add the LoggerAdapter class and update the docs. Best regards, Vinay Sajip From ncoghlan at gmail.com Fri Jan 11 16:03:53 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 12 Jan 2008 01:03:53 +1000 Subject: [Python-Dev] Backporting PEP 3101 to 2.6 In-Reply-To: References: <47861E4A.4020006@trueblade.com> <47865CA6.2070400@trueblade.com> Message-ID: <47878559.4090401@gmail.com> Guido van Rossum wrote: > For data types whose output uses only ASCII, would it be acceptable if > they always returned an 8-bit string and left it up to the caller to > convert it to Unicode? This would apply to all numeric types. (The > date/time types have a strftime() style API which means the user must > be able to specifiy Unicode.) To elaborate on this a bit (and handwaving a lot of important details out of the way) do you mean something like the following for the builtin format?: def format(obj, fmt_spec=None): if fmt_spec is None: fmt_spec='' result = obj.__format__(fmt_spec) if isinstance(fmt_spec, unicode): if isinstance(result, str): result = unicode(result) return result Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Fri Jan 11 16:18:57 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 12 Jan 2008 01:18:57 +1000 Subject: [Python-Dev] Passing contextual information when logging In-Reply-To: <001101c8543e$fffa04e0$0800a8c0@DELTA> References: <001101c8543e$fffa04e0$0800a8c0@DELTA> Message-ID: <478788E1.208@gmail.com> Vinay Sajip at Red Dove wrote: > One solution is to create a generic wrapper around loggers to which a logger > name and contextual information can be passed. The wrapper would delegate > logging calls to the logger with the specified name, but would manipulate > the arguments passed to the logging call to insert the contextual > information. I have created such a wrapper class, called LoggerAdapter, > which is in the example script located at > > http://dpaste.com/30613/ > > I would welcome your views on whether the LoggerAdapter class is suitable > for adding to the logging package in Python 2.6/3.0. Does it do what might > reasonably be expected out of the box? LoggerAdapters are, of course, > garbage collected in the normal way and so impose no particular memory > burden. It looks pretty good (and useful) to me, but I have a couple of questions. When you talk about stacking in the constructor docstring, do you mean something like having LoggerAdapterA delegating to LoggerAdapterB which in turn delegates to the real logger? If that's what you mean, then I think the process() method will need to be changed to handle the case where there is already an "extras" entry in the keyword arguments. If that's not what you mean... then please explain because I don't have any other ideas :) I was also going to suggest modifying the constructor to accept keyword arguments, but realised that case can be adequately handled just by using a dict: logger_with_extra = LoggerAdapter(logger, dict(attr1=5, attr2=True)) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From vinay_sajip at yahoo.co.uk Fri Jan 11 16:57:14 2008 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Fri, 11 Jan 2008 15:57:14 +0000 (UTC) Subject: [Python-Dev] Passing contextual information when logging References: <001101c8543e$fffa04e0$0800a8c0@DELTA> <478788E1.208@gmail.com> Message-ID: Nick Coghlan gmail.com> writes: > It looks pretty good (and useful) to me, but I have a couple of questions. > > When you talk about stacking in the constructor docstring, do you mean > something like having LoggerAdapterA delegating to LoggerAdapterB which > in turn delegates to the real logger? > > If that's what you mean, then I think the process() method will need to > be changed to handle the case where there is already an "extras" entry > in the keyword arguments. If that's not what you mean... then please > explain because I don't have any other ideas :) > > I was also going to suggest modifying the constructor to accept keyword > arguments, but realised that case can be adequately handled just by > using a dict: > > logger_with_extra = LoggerAdapter(logger, dict(attr1=5, attr2=True)) > Hi Nick, Thanks for the feedback. The stacking is a bit of a corner case, so maybe I should take that comment out of the docstring - but that's what I meant by stacking. A set of stacked adapters could work by e.g. appending or prepending their contextual info to the message, or by manipulating the "extra" keyword argument to the base logger. The example implementation uses the "extra" method, but if another adapter was to be stacked on top of that one, its process method would need to check the passed in kwargs for key "extra" and replace it with some merged version of the two sets of contexts - its own and that of the underlying logger. Another point about the "extra" method is that it lends itself to working with a Formatter whose format string can hold user-defined placeholders like %(ip)s in the example. This will work well in some scenarios, but in other scenarios (where it may be desired to share formatters across various handlers, or because stacking is desired and there's no way for one formatter to know all the placeholders) it makes sense to just manipulate msg in the process method, for example: def process(self, msg, kwargs): prefix = "IP: %(ip)-15s User: %(user)-8s" % self.extra return "%s %s" % (prefix, msg), kwargs When I check it in I will elaborate on the class docstring to explain the usage in a bit more detail, as well as of course detailing it in the docs in a separate section entitled "Passing contextual information to the log" (or similar). Best regards, Vinay Sajip From ncoghlan at gmail.com Fri Jan 11 17:15:58 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 12 Jan 2008 02:15:58 +1000 Subject: [Python-Dev] Passing contextual information when logging In-Reply-To: References: <001101c8543e$fffa04e0$0800a8c0@DELTA> <478788E1.208@gmail.com> Message-ID: <4787963E.2000405@gmail.com> Vinay Sajip wrote: > When I check it in I will elaborate on the class docstring to explain the usage > in a bit more detail, as well as of course detailing it in the docs in a > separate section entitled "Passing contextual information to the log" (or > similar). Thanks, I understand what you meant quite a bit better now. The comment didn't make a great deal of sense looking at the LoggingAdapter class in isolation, but is significantly more meaningful once you start discussing other possibilities for what can be done in the process() method. I also like that the design of the class allows users to easily create their own adapters simply by subclassing this one and overriding the process() method. I imagine that's why you wrote it that way :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From steve at holdenweb.com Fri Jan 11 17:19:02 2008 From: steve at holdenweb.com (Steve Holden) Date: Fri, 11 Jan 2008 11:19:02 -0500 Subject: [Python-Dev] Backporting PEP 3101 to 2.6 In-Reply-To: <47878559.4090401@gmail.com> References: <47861E4A.4020006@trueblade.com> <47865CA6.2070400@trueblade.com> <47878559.4090401@gmail.com> Message-ID: Nick Coghlan wrote: > Guido van Rossum wrote: >> For data types whose output uses only ASCII, would it be acceptable if >> they always returned an 8-bit string and left it up to the caller to >> convert it to Unicode? This would apply to all numeric types. (The >> date/time types have a strftime() style API which means the user must >> be able to specifiy Unicode.) > > To elaborate on this a bit (and handwaving a lot of important details > out of the way) do you mean something like the following for the builtin > format?: > > def format(obj, fmt_spec=None): > if fmt_spec is None: fmt_spec='' > result = obj.__format__(fmt_spec) > if isinstance(fmt_spec, unicode): > if isinstance(result, str): > result = unicode(result) > return result > Isn't unicode idempotent? Couldn't if isinstance(result, str): result = unicode(result) avoid repeating in Python a test already made in C by re-spelling it as result = unicode(result) or have you hand-waved away important details that mean the test really is required? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From eric+python-dev at trueblade.com Fri Jan 11 16:39:04 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Fri, 11 Jan 2008 10:39:04 -0500 Subject: [Python-Dev] Backporting PEP 3101 to 2.6 In-Reply-To: <47878559.4090401@gmail.com> References: <47861E4A.4020006@trueblade.com> <47865CA6.2070400@trueblade.com> <47878559.4090401@gmail.com> Message-ID: <47878D98.1020601@trueblade.com> Nick Coghlan wrote: > Guido van Rossum wrote: >> For data types whose output uses only ASCII, would it be acceptable if >> they always returned an 8-bit string and left it up to the caller to >> convert it to Unicode? This would apply to all numeric types. (The >> date/time types have a strftime() style API which means the user must >> be able to specifiy Unicode.) > > To elaborate on this a bit (and handwaving a lot of important details > out of the way) do you mean something like the following for the builtin > format?: > > def format(obj, fmt_spec=None): > if fmt_spec is None: fmt_spec='' > result = obj.__format__(fmt_spec) > if isinstance(fmt_spec, unicode): > if isinstance(result, str): > result = unicode(result) > return result That's the approach I'm taking. The builtin format is the only caller of __format__ that I know of, so it's the only place this would need to be done. From eric+python-dev at trueblade.com Fri Jan 11 17:43:40 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Fri, 11 Jan 2008 11:43:40 -0500 Subject: [Python-Dev] Backporting PEP 3101 to 2.6 In-Reply-To: References: <47861E4A.4020006@trueblade.com> <47865CA6.2070400@trueblade.com> <47878559.4090401@gmail.com> Message-ID: <47879CBC.6000900@trueblade.com> Steve Holden wrote: > Nick Coghlan wrote: >> To elaborate on this a bit (and handwaving a lot of important details >> out of the way) do you mean something like the following for the builtin >> format?: >> >> def format(obj, fmt_spec=None): >> if fmt_spec is None: fmt_spec='' >> result = obj.__format__(fmt_spec) >> if isinstance(fmt_spec, unicode): >> if isinstance(result, str): >> result = unicode(result) >> return result >> > Isn't unicode idempotent? Couldn't > > if isinstance(result, str): > result = unicode(result) > > > avoid repeating in Python a test already made in C by re-spelling it as > > result = unicode(result) > > or have you hand-waved away important details that mean the test really > is required? This code is written in C. It already has a check to verify that the return from __format__ is either str or unicode, and another check that fmt_spec is str or unicode. So doing the conversion only if result is str and fmt_spec is unicode would be a cheap decision. Good catch, though. I wouldn't have thought of it, and there are parts that are written in Python, so maybe I can leverage this elsewhere. Thanks! Eric. From amk at amk.ca Fri Jan 11 21:13:39 2008 From: amk at amk.ca (A.M. Kuchling) Date: Fri, 11 Jan 2008 15:13:39 -0500 Subject: [Python-Dev] Bug day preparations: tagging bugs? Message-ID: <20080111201339.GA11352@amk-desktop.matrixgroup.net> No one pushed back on the 19th as a bug day, so I'm going to go ahead and send out announcements this evening. We should mark issues in the tracker that are good candidates for new developers. How do we want to do this? We could add a new keyword ('beginner', 'bugday', ???) or add a prefix to the subject line. Does Roundup search also look at the text of the change notes added to a bug? If yes, then just adding a comment with the magic keyword would be sufficient. --amk From guido at python.org Fri Jan 11 21:30:12 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 11 Jan 2008 12:30:12 -0800 Subject: [Python-Dev] Bug day preparations: tagging bugs? In-Reply-To: <20080111201339.GA11352@amk-desktop.matrixgroup.net> References: <20080111201339.GA11352@amk-desktop.matrixgroup.net> Message-ID: On Jan 11, 2008 12:13 PM, A.M. Kuchling wrote: > No one pushed back on the 19th as a bug day, so I'm going to go ahead > and send out announcements this evening. > > We should mark issues in the tracker that are good candidates for new > developers. How do we want to do this? We could add a new keyword > ('beginner', 'bugday', ???) or add a prefix to the subject line. > > Does Roundup search also look at the text of the change notes added to > a bug? If yes, then just adding a comment with the magic keyword > would be sufficient. Yes, it does. Unfortunately, I've been using "bug day" in some comments and that doesn't seem to work well in the search. I think a keyword is easier, I can add it if you want me to. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From lists at cheimes.de Fri Jan 11 21:45:14 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 11 Jan 2008 21:45:14 +0100 Subject: [Python-Dev] Bug day preparations: tagging bugs? In-Reply-To: References: <20080111201339.GA11352@amk-desktop.matrixgroup.net> Message-ID: <4787D55A.5030705@cheimes.de> Guido van Rossum wrote: > Yes, it does. Unfortunately, I've been using "bug day" in some > comments and that doesn't seem to work well in the search. > > I think a keyword is easier, I can add it if you want me to. > Please add two keywords, one for bug day and one for easy tasks. May "beginner task", "easy" or "novice level" makes a good keyword. We could also mark complex/hard tasks, tasks which require C programming and documentation only tasks, even though I'm not sure how useful the additional tags would be. Christian From amk at amk.ca Fri Jan 11 22:10:27 2008 From: amk at amk.ca (A.M. Kuchling) Date: Fri, 11 Jan 2008 16:10:27 -0500 Subject: [Python-Dev] Bug day preparations: tagging bugs? In-Reply-To: <4787D55A.5030705@cheimes.de> References: <20080111201339.GA11352@amk-desktop.matrixgroup.net> <4787D55A.5030705@cheimes.de> Message-ID: <20080111211027.GA13180@amk-desktop.matrixgroup.net> On Fri, Jan 11, 2008 at 09:45:14PM +0100, Christian Heimes wrote: > Please add two keywords, one for bug day and one for easy tasks. May > "beginner task", "easy" or "novice level" makes a good keyword. I think marking easy tasks is all we need. That would certainly be useful during ongoing non-bugday development, but marking issues as 'bug-day' doesn't seem useful to me. What would it mean? > We could also mark complex/hard tasks, tasks which require C programming > and documentation only tasks, even though I'm not sure how useful the > additional tags would be. Shouldn't documentation-only bugs be marked as being for the 'Documentation' component? The C vs. Python requirement is also discoverable from the component field, sort of; Library is for issues in Lib/, which is all Python, and C extensions are the 'Extension Modules' component. --amk From lists at cheimes.de Fri Jan 11 22:10:48 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 11 Jan 2008 22:10:48 +0100 Subject: [Python-Dev] PEP: Post import hooks In-Reply-To: <20080111010557.63A2B3A406C@sparrow.telecommunity.com> References: <20080110043258.BEBAB3A40A9@sparrow.telecommunity.com> <20080110223235.13D5B3A406C@sparrow.telecommunity.com> <4786A579.2070605@cheimes.de> <20080111010557.63A2B3A406C@sparrow.telecommunity.com> Message-ID: <4787DB58.5010900@cheimes.de> Phillip J. Eby wrote: > *sigh*. We seem to be getting further and further off course, > here. The more different you make the semantics of the system, the > harder it will be to verify that it's doing the right thing without > having real field experience with the new approach. *sigh*, too. :/ This discussion has neither helping me nor you. Could you please write an unit test or two to show me exactly what my implementation is doing wrong and how you think the callbacks should be called. I know a simple test won't proof the correctness of the implementation but a failing test would at least show the incorrectness. I'm still not sure which way is the correct way in your opinion and I hate guessing what you are trying to explain to me. Christian PS: I've a pending patch that queues new registrations while hooks are processed in PyImport_NotifyModuleLoaded(). From jflatow at northwestern.edu Sat Jan 12 00:01:10 2008 From: jflatow at northwestern.edu (Jared Flatow) Date: Fri, 11 Jan 2008 17:01:10 -0600 Subject: [Python-Dev] PySequence_Concat for dicts Message-ID: <3A374A5D-6019-4D32-A379-6636820DC009@northwestern.edu> Hi all, I am fairly new to the Python community so please forgive me (and correct me) if I am going about this wrong. I think it would be convenient and pythonic if dict objects implemented the PySequence_Concat method. I see there was once a short-lived discussion about this here: http://mail.python.org/pipermail/patches/2004-March/014323.html I have also been following the discussion about contributing to Python. It seems to me that this would be a fairly easy feature to implement (perhaps naively?), and I would be glad to try writing a patch for this if there is at least some chance of it making it into one of the branches. Can someone please advise me on what the correct order for going about this would be? Do I need to first write a PEP justifying why I think it would be an improvement? Which version of Python (if any), should a patch be targeted at? Otherwise, is there a good reason dicts do not already implement this method? I somewhat understand the complaint about commutativity, but as mentioned in the previous discussion, list concatenation is not commutative either. Seeing as update is the only builtin method for concatenation of dicts in the first place, it doesn't seem all that confusing that 'summing' two dicts should conveniently return a new instance that is the (only form of) concatenation of the two dicts. regards, jared From lists at cheimes.de Sat Jan 12 00:27:43 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 12 Jan 2008 00:27:43 +0100 Subject: [Python-Dev] PEP: per user site-packages directory Message-ID: MA Lemburg has suggested a per user site-packages directory in the "pkgutil, pkg_resource and Python 3.0 name space packages" thread. I've written a short PEP about it for Python 2.6 and 3.0. PEP: XXX Title: Per user site-packages directory Version: $Revision$ Last-Modified: $Date$ Author: Christian Heimes Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 11-Jan-2008 Python-Version: 2.6, 3.0 Post-History: Abstract ======== This PEP proposes a new a per user site-packages directory to allow users the local installation of Python packages in their home directory. Rationale ========= Current Python versions don't have an unified way to install packages into the home directory of an user (except for Mac Framework builds). Users are either forced to ask the system administrator to install or update a package for them or to use one of the many workaround like Virtual Python [1]_, Working Env [2]_ or Virtual Env [3]_. It's not the goal of the PEP to replace the tools or to implement isolated installations of Python. It only implements the most common use case of an additional site-packages directory for each user. The feature can't be implemented using the environment variable *PYTHONPATH*. The env var just inserts a new directory to the beginning of *sys.path* but it doesn't parse the pth files in the directory. A full blown site-packages path is required for several applications and Python eggs. Specification ============= site directory (site-packages) A directory in sys.path. In contrast to ordinary directories the pth files in the directory are processed, too. user site directory A site directory inside the users' home directory. An user site directory is specific to a Python version. The path contains the version number (major and minor only). Windows: %APPDATA%/Python/Python26/site-packages Mac: ~/Library/Python/2.6/site-packages Unix: ~/.local/lib/python2.6/site-packages user configuration directory Usually the parent directory of the user site directory. It's meant for Python version specific data like config files. Windows: %APPDATA%/Python/Python26 Mac: ~/Library/Python/2.6 Unix: ~/.local/lib/python2.6 user base directory It's located inside the user's home directory. The user site and use config directory are inside the base directory. On some systems the directory may be shared with 3rd party apps. Windows: %APPDATA%/Python Mac: ~/Library/Python Unix: ~/.local On Windows APPDATA was chosen because it is the most logical place for application data. Microsoft recommands that software doesn't write to USERPROFILE [5]_ and My Documents is not suited for application data, too. [8]_ On Linux ~/.local/ was chosen in favor over ./~python/ because the directory is already used by several other programs in analogy to /usr/local. [7]_ Implementation ============== The site module gets a new method adduserpackage() which adds the appropriate directory to the search path. The directory is not added if it doesn't exist when Python is started. However the location of the user site directory and user base directory is stored in an internal variable for distutils. The user site directory is added before the system site directories but after Python's search paths and PYTHONPATH. This setup allows the user to install a different version of a package than the system administrator but it prevents the user from accidently overwriting a stdlib module. Stdlib modules can still be overwritten with PYTHONPATH. distutils.command.install (setup.py install) gets a new argument --user to install packages in the user site directory. The required directories are created by install. distutils.sysconfig will get methods to access the private variables of site. (not yet implemented) The Windows updater needs to be updated, too. It should create an menu item which opens the user site directory in a new explorer windows. Backwards Compatibility ======================= TBD Open Questions ============== * Are the directories for Windows, Mac and Unix fine? * Mac: Should framework and non-framework builds of Python use the same directories? * The patch also adds a usecustomize hook to site. Is it useful and should it stay? Reference Implementation ======================== A reference implementation is available in the bug tracker. [4]_ Acknowledgments =============== The implementation of this PEP, if accepted, is sponsored by Google through the Google Summer of Code program. Copyright ========= This document has been placed in the public domain. References ========== .. [1] Virtual Python http://peak.telecommunity.com/DevCenter/EasyInstall#creating-a-virtual-python .. [2] Working Env http://pypi.python.org/pypi/workingenv.py http://blog.ianbicking.org/workingenv-revisited.html .. [3] Virtual Env http://pypi.python.org/pypi/virtualenv .. [4] reference implementation http://bugs.python.org/issue1799 .. [5] MSDN: CSIDL http://msdn2.microsoft.com/en-us/library/bb762494.aspx .. [6] Initial suggestion for a per user site-packages directory http://permalink.gmane.org/gmane.comp.python.devel/90902 .. [7] Suggestion of ~/.local/ http://permalink.gmane.org/gmane.comp.python.devel/90925 .. [8] APPDATA discussion http://permalink.gmane.org/gmane.comp.python.devel/90932 .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: From jimjjewett at gmail.com Sat Jan 12 00:34:28 2008 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri, 11 Jan 2008 18:34:28 -0500 Subject: [Python-Dev] Coverity Scan, Python upgraded to rung 2 Message-ID: Neal Norwitz wrote: >> For codeobject.c, line 327 should not be reachable. ... Christian Heimes wrote: > Please suppress the warning. I removed the last > two lines and GCC complained ... Either way, it would be worth adding a comment to the source code so this doesn't come up again. -jJ From guido at python.org Sat Jan 12 00:58:35 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 11 Jan 2008 15:58:35 -0800 Subject: [Python-Dev] Bug day preparations: tagging bugs? In-Reply-To: <20080111211027.GA13180@amk-desktop.matrixgroup.net> References: <20080111201339.GA11352@amk-desktop.matrixgroup.net> <4787D55A.5030705@cheimes.de> <20080111211027.GA13180@amk-desktop.matrixgroup.net> Message-ID: I've added an "easy" keyword, with description "This is an easy task (e.g. suitable for GHOP or bug day beginners)". Let me know if more is needed, Andrew. (I have no idea if anyone can add keywords or if I have magical powers. On the left side bar there's a section "Administration" which contains a link "Edit Keywords".) --Guido On Jan 11, 2008 1:10 PM, A.M. Kuchling wrote: > On Fri, Jan 11, 2008 at 09:45:14PM +0100, Christian Heimes wrote: > > Please add two keywords, one for bug day and one for easy tasks. May > > "beginner task", "easy" or "novice level" makes a good keyword. > > I think marking easy tasks is all we need. That would certainly be > useful during ongoing non-bugday development, but marking issues as > 'bug-day' doesn't seem useful to me. What would it mean? > > > We could also mark complex/hard tasks, tasks which require C programming > > and documentation only tasks, even though I'm not sure how useful the > > additional tags would be. > > Shouldn't documentation-only bugs be marked as being for the > 'Documentation' component? The C vs. Python requirement is also > discoverable from the component field, sort of; Library is for issues > in Lib/, which is all Python, and C extensions are the 'Extension > Modules' component. > > --amk > _______________________________________________ > > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From lists at cheimes.de Sat Jan 12 01:02:27 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 12 Jan 2008 01:02:27 +0100 Subject: [Python-Dev] Bug day preparations: tagging bugs? In-Reply-To: References: <20080111201339.GA11352@amk-desktop.matrixgroup.net> <4787D55A.5030705@cheimes.de> <20080111211027.GA13180@amk-desktop.matrixgroup.net> Message-ID: <47880393.5080209@cheimes.de> Guido van Rossum wrote: > (I have no idea if anyone can add keywords or if I have magical > powers. On the left side bar there's a section "Administration" which > contains a link "Edit Keywords".) I'm unable to add new keywords although I've the developer role. I guess one needs to be a coordinator to add new keywords. Christian From lists at cheimes.de Sat Jan 12 01:02:27 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 12 Jan 2008 01:02:27 +0100 Subject: [Python-Dev] Bug day preparations: tagging bugs? In-Reply-To: References: <20080111201339.GA11352@amk-desktop.matrixgroup.net> <4787D55A.5030705@cheimes.de> <20080111211027.GA13180@amk-desktop.matrixgroup.net> Message-ID: <47880393.5080209@cheimes.de> Guido van Rossum wrote: > (I have no idea if anyone can add keywords or if I have magical > powers. On the left side bar there's a section "Administration" which > contains a link "Edit Keywords".) I'm unable to add new keywords although I've the developer role. I guess one needs to be a coordinator to add new keywords. Christian From python at rcn.com Sat Jan 12 01:45:20 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 11 Jan 2008 19:45:20 -0500 (EST) Subject: [Python-Dev] PySequence_Concat for dicts Message-ID: <20080111194520.AEX17855@ms19.lnh.mail.rcn.net> [Jared] > I think it would be convenient and pythonic if dict > objects implemented the PySequence_Concat method. IMO, the chainmap() recipe on ASPN is a much better solution since it doesn't create a third dictionary with the all the attendant allocation and copying effort. It isn't a common use case to need to sum two dictionaries while keeping both of the inputs unaltered. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/305268. Raymond From jflatow at northwestern.edu Sat Jan 12 02:07:37 2008 From: jflatow at northwestern.edu (Jared Flatow) Date: Fri, 11 Jan 2008 19:07:37 -0600 Subject: [Python-Dev] PySequence_Concat for dicts In-Reply-To: <20080111194520.AEX17855@ms19.lnh.mail.rcn.net> References: <20080111194520.AEX17855@ms19.lnh.mail.rcn.net> Message-ID: On Jan 11, 2008, at 6:45 PM, Raymond Hettinger wrote: > IMO, the chainmap() recipe on ASPN is a much better solution since > it doesn't create a third dictionary with the all the attendant > allocation and copying effort. I wasn't suggesting that the result of concatenation would be a chained table, rather that it would perform the equivalent of an update and return the new dict (the same way extend works for lists). > It isn't a common use case to need to sum two dictionaries while > keeping both of the inputs unaltered. Inplace concatenation could be implemented more efficiently but would be exactly the same as calling update. jared From python at rcn.com Sat Jan 12 02:21:16 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 11 Jan 2008 20:21:16 -0500 (EST) Subject: [Python-Dev] PySequence_Concat for dicts Message-ID: <20080111202116.AEX21061@ms19.lnh.mail.rcn.net> > I wasn't suggesting that the result of concatenation would > be a chained table, rather that it would perform the > equivalent of an update and return the new dict >(the same way extend works for lists) When does it come-up that you want a third summed dict while keeping the two originals around unchanged? Does it matter that the addition is non-commutative? Would a + b + c produce an intermediate a/b combo and then another new object for a/b/c so that the entries in a get copied twice and memory usage has to hold a, b, a/b, c, and a/b/c in memory all at the same time? What are the use cases? FWIW, the Py3.0 API for dicts will support some set-like operations. Perhaps, that fits the bill. Raymond From fperez.net at gmail.com Sat Jan 12 02:57:12 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 11 Jan 2008 18:57:12 -0700 Subject: [Python-Dev] New Developer References: <5c6f2a5d0801071919v3a03ef75n3b9574232b1241ff@mail.gmail.com> Message-ID: Mark Dickinson wrote: > Hello all, > I've recently been granted commit privileges; so, following the usual > protocol, here's a quick introduction. I'm a mathematician by day; my > degree is in number theory, but five summers of Fortran 77 programming and > two semesters of teaching numerical analysis have given me a taste for > numerics as well. I discovered Python around twelve years ago and found > that it fit my brain nicely (even more so after nested namespaces were > introduced) and now use it almost daily for a wide variety of tasks. I've > been lurking on python-dev for longer than I care to admit to. I also > dabble in Haskell and O'Caml. Very interesting! Are you aware of Sage? http://sagemath.org. All Python-based, developed originally by a number theorist (http://wstein.org), and with a rapidly growing team of developers (including John Cremona, who's contributed a lot of his code to Sage). The Python-dev team should be proud of the impact Python is having in scientific computing: python is without a doubt the leading tool for open source, high-level scientific codes (i.e. not Fortran/C), and growing. Thanks! I normally wouldn't announce this here, but please forgive the mini-spam (and let's continue off list if you are interested): http://wiki.sagemath.org/days8 Just contact me off list at Fernando.Perez at colorado.edu if you think you'd like to attend. Cheers, f From jgarber at ionzoft.com Sat Jan 12 00:04:33 2008 From: jgarber at ionzoft.com (Jason Garber) Date: Fri, 11 Jan 2008 17:04:33 -0600 Subject: [Python-Dev] How to change path at compile time? Message-ID: Hello, Is there any reasonable way to change the default sys.path at compile time? (ie. add a directory). (I am aware of $PYTHONPATH for runtime) -- Best Regards, Jason Garber Senior Systems Engineer IonZoft, Inc. (814) 941-2390 jgarber at ionzoft.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080111/0d989f9a/attachment.htm From alexandre at peadrop.com Sat Jan 12 04:18:34 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Fri, 11 Jan 2008 22:18:34 -0500 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: References: Message-ID: I can't comment on the implementation details, but +1 for the idea. I think this feature will be very useful in a shared hosting environment. -- Alexandre On Jan 11, 2008 6:27 PM, Christian Heimes wrote: > PEP: XXX > Title: Per user site-packages directory > Version: $Revision$ > Last-Modified: $Date$ > Author: Christian Heimes > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 11-Jan-2008 > Python-Version: 2.6, 3.0 > Post-History: > From guido at python.org Sat Jan 12 04:22:19 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 11 Jan 2008 19:22:19 -0800 Subject: [Python-Dev] PySequence_Concat for dicts In-Reply-To: <20080111202116.AEX21061@ms19.lnh.mail.rcn.net> References: <20080111202116.AEX21061@ms19.lnh.mail.rcn.net> Message-ID: On Jan 11, 2008 5:21 PM, Raymond Hettinger wrote: > > I wasn't suggesting that the result of concatenation would > > be a chained table, rather that it would perform the > > equivalent of an update and return the new dict > >(the same way extend works for lists) > > When does it come-up that you want a third summed dict > while keeping the two originals around unchanged? Does > it matter that the addition is non-commutative? Would > a + b + c produce an intermediate a/b combo and then > another new object for a/b/c so that the entries in > a get copied twice and memory usage has to hold a, b, > a/b, c, and a/b/c in memory all at the same time? > What are the use cases? > > FWIW, the Py3.0 API for dicts will support some set-like > operations. Perhaps, that fits the bill. While that was at one point proposed, It's been retracted. It does suggest that we have two choices for the proposed operation: d1+d2 or d1|d2. I think the latter may be more appropriate: len(seq1+seq2) == len(seq1) ++len(seq2), but no such invariant exists for set union. I'd like to take an "all or nothing" approach to this: either we implement the same 4 operations as for sets (|, &, ^ and -) or we implement none of them. We can assign semantics to these such that if the values are all the same, these work the same as set operations on the keys. E.g.: {1:1, 2:2} | {2:2, 3:3} == {1:1, 2:2, 3:3} {1:1, 2:2} & {2:2, 3:3} == {2:2} {1:1, 2:2} ^ {2:2, 3:3} == {1:1, 3:3} {1:1, 2:2} - {2:2, 3:3} == {1:1} If the values differ, we have to decide what happens. For ^ and -, we only need to take the keys into account. For | I propose that the RHS wins, matching dict.update(); this makes sense since set.update() is equivalent to |= for sets, so |= should match dict.update() for dict, and then | follows. For & I'm on the fence, but I think it's easier to say that the RHS wins in all cases. So we get: {1:1, 2:2} | {2:0, 3:0} = {1:1, 2:0, 3:0} {1:1, 2:2} & {2:0, 3:0} = {2:0} {1:1, 2:2} ^ {2:0, 3:0} = {1:1, 3:0} {1:1, 2:2} - {2:0, 3:0} = {1:1} (The |=, &=, ^= and -= operators don't pose new problems, they update the LHS in place.) I'm not sure where I stand on this proposal -- I guess a +0, if someone else does the work. (The abc.py module needs to be updated too.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From amk at amk.ca Sat Jan 12 04:28:54 2008 From: amk at amk.ca (A.M. Kuchling) Date: Fri, 11 Jan 2008 22:28:54 -0500 Subject: [Python-Dev] Bug day preparations: tagging bugs? In-Reply-To: References: <20080111201339.GA11352@amk-desktop.matrixgroup.net> <4787D55A.5030705@cheimes.de> <20080111211027.GA13180@amk-desktop.matrixgroup.net> Message-ID: <20080112032854.GA2104@amk.local> On Fri, Jan 11, 2008 at 03:58:35PM -0800, Guido van Rossum wrote: > Let me know if more is needed, Andrew. Thanks! I've run searches for a few different components and marked a few bugs with the 'easy' keyword; I see that Tiran is doing this, too. > (I have no idea if anyone can add keywords or if I have magical > powers. On the left side bar there's a section "Administration" which > contains a link "Edit Keywords".) I have that link, too. --amk From python at rcn.com Sat Jan 12 05:25:45 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 11 Jan 2008 20:25:45 -0800 Subject: [Python-Dev] PySequence_Concat for dicts References: <20080111202116.AEX21061@ms19.lnh.mail.rcn.net> Message-ID: <005c01c854d3$3409f6c0$6800a8c0@RaymondLaptop1> > I'd like to take an "all or nothing" approach to this: either we > implement the same 4 operations as for sets (|, &, ^ and -) or we > implement none of them. . . . . > I'm not sure where I stand on this proposal -- I guess a +0, if > someone else does the work. (The abc.py module needs to be updated > too.) I think this would end-up being clutter, something added just because it could be done, not because we have compelling use cases that can't be met with existing dicts, sets, and lists. Raymond From jflatow at northwestern.edu Sat Jan 12 17:51:28 2008 From: jflatow at northwestern.edu (Jared Flatow) Date: Sat, 12 Jan 2008 10:51:28 -0600 Subject: [Python-Dev] PySequence_Concat for dicts In-Reply-To: References: <20080111202116.AEX21061@ms19.lnh.mail.rcn.net> Message-ID: <25D06FB4-2906-4531-A4D4-BF1B1396F9FA@northwestern.edu> > On Jan 11, 2008 5:21 PM, Raymond Hettinger wrote: >> When does it come-up that you want a third summed dict >> while keeping the two originals around unchanged? Does >> it matter that the addition is non-commutative? Would >> a + b + c produce an intermediate a/b combo and then >> another new object for a/b/c so that the entries in >> a get copied twice and memory usage has to hold a, b, >> a/b, c, and a/b/c in memory all at the same time? There is no way around it, this will be less efficient than the inplace operation. If there were a precedent for calling a method like update and returning itself instead of None, I would suggest that, but since this is the way that has already been established for lists, it seems a natural extension. On Jan 11, 2008, at 9:22 PM, Guido van Rossum wrote: > It does suggest that we have two choices for the proposed operation: > d1+d2 or d1|d2. I think the latter may be more appropriate: > len(seq1+seq2) == len(seq1) ++len(seq2), but no such invariant exists > for set union. This might be way out there but I suppose you could consider that a dict is actually two different things, depending on what you are doing, and that these operations might actually do different things. Interpreted as a sequence, it is a sequence of key mappings. As confirmed in another recent discussion, Python guarantees consistent (read: repeatable) ordering of iteration through a dict, so in this sense it really is a sequence. (On a side note, I frequently rely on the repeatability of ordering when interacting with the Python shell). The notable sequence operation being + for concatenation, would perform an update of the keys (thus for the sequence op the mappings aren't guaranteed to be preserved, only the keys). The other interpretation of dict is obviously as a set of (key, value) pairs. For sets, the four major operations could behave exactly as any other set of (key, value) tuples (i.e. if you transform it to a list and then apply the same ops you should get the same result). jared p.s. If I were to get approval to implement some version of this, which version of Python would be appropriate to work with? From lists at cheimes.de Sat Jan 12 18:18:13 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 12 Jan 2008 18:18:13 +0100 Subject: [Python-Dev] PySequence_Concat for dicts In-Reply-To: <25D06FB4-2906-4531-A4D4-BF1B1396F9FA@northwestern.edu> References: <20080111202116.AEX21061@ms19.lnh.mail.rcn.net> <25D06FB4-2906-4531-A4D4-BF1B1396F9FA@northwestern.edu> Message-ID: Jared Flatow wrote: > p.s. If I were to get approval to implement some version of this, > which version of Python would be appropriate to work with? The minimum version is 2.6. 2.5 is in maintenance mode. Changes made to 2.6 are merged into 3.0 so you don't have to create two patches. It'd stick to 2.6 because it's easy to port a 2.6 patch to 3.0, if the new feature won't make it into 2.6. Christian From jim at zope.com Sat Jan 12 18:25:00 2008 From: jim at zope.com (Jim Fulton) Date: Sat, 12 Jan 2008 12:25:00 -0500 Subject: [Python-Dev] inst_persistent_id Message-ID: Recently, I reviewed the documentation source for pickle and came across the following comment: BAW: Both pickle and cPickle support something called inst_persistent_id() which appears to give unknown types a second shot at producing a persistent id. Since Jim Fulton can't remember why it was added or what it's for, I'm leaving it undocumented. I couldn't remember this and decided to dig into it and realized that this was a very useful experimental feature I added way back when cPickle was in its infancy. This is a fairly useful optimization that I never got around to evaluating. (Yeah, I know.) It is a hook, like the persistent_id hook that is called with objects to determine if they should be pickled by persistent reference. Unlike persistent_id, it isn't called for built-in types (really types for which pickle has specific handlers), like strings, numbers, lists, tuples and so on. It is only called for "instances" (types for which pickle doesn't have specific handlers). This vastly reduces the number of calls to the hook. Some tests with ZODB indicated significant improvements in pickling speed when a hook is used. If there are no objections, I'll update the Python 2 documentation to describe this and add a test. The comment above suggests that this hook is in pickle and cPickle. It is in cPickle, but was removed from pickle. I propose to add it back to pickle -- mainly for consistency with cPickle. I'll need to double check how this interacts with the type dispatching in pickle protocol 2. Any objections? Jim -- Jim Fulton Zope Corporation From guido at python.org Sat Jan 12 18:32:29 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 12 Jan 2008 09:32:29 -0800 Subject: [Python-Dev] PySequence_Concat for dicts In-Reply-To: <25D06FB4-2906-4531-A4D4-BF1B1396F9FA@northwestern.edu> References: <20080111202116.AEX21061@ms19.lnh.mail.rcn.net> <25D06FB4-2906-4531-A4D4-BF1B1396F9FA@northwestern.edu> Message-ID: On Jan 12, 2008 8:51 AM, Jared Flatow wrote: > > On Jan 11, 2008 5:21 PM, Raymond Hettinger wrote: > >> When does it come-up that you want a third summed dict > >> while keeping the two originals around unchanged? Does > >> it matter that the addition is non-commutative? Would > >> a + b + c produce an intermediate a/b combo and then > >> another new object for a/b/c so that the entries in > >> a get copied twice and memory usage has to hold a, b, > >> a/b, c, and a/b/c in memory all at the same time? > > There is no way around it, this will be less efficient than the > inplace operation. If there were a precedent for calling a method > like update and returning itself instead of None, I would suggest > that, but since this is the way that has already been established for > lists, it seems a natural extension. > > On Jan 11, 2008, at 9:22 PM, Guido van Rossum wrote: > > It does suggest that we have two choices for the proposed operation: > > d1+d2 or d1|d2. I think the latter may be more appropriate: > > len(seq1+seq2) == len(seq1) ++len(seq2), but no such invariant exists > > for set union. > > This might be way out there but I suppose you could consider that a > dict is actually two different things, depending on what you are > doing, and that these operations might actually do different things. > Interpreted as a sequence, it is a sequence of key mappings. As > confirmed in another recent discussion, Python guarantees consistent > (read: repeatable) ordering of iteration through a dict, so in this > sense it really is a sequence. (On a side note, I frequently rely on > the repeatability of ordering when interacting with the Python shell). > > The notable sequence operation being + for concatenation, would > perform an update of the keys (thus for the sequence op the mappings > aren't guaranteed to be preserved, only the keys). > > The other interpretation of dict is obviously as a set of (key, > value) pairs. For sets, the four major operations could behave > exactly as any other set of (key, value) tuples (i.e. if you > transform it to a list and then apply the same ops you should get the > same result). No, a dict is not a set of (key, value) pairs. The keys form a set, you cannot have duplicate keys. > jared > > p.s. If I were to get approval to implement some version of this, > which version of Python would be appropriate to work with? > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Sat Jan 12 22:41:51 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 12 Jan 2008 22:41:51 +0100 Subject: [Python-Dev] How to change path at compile time? In-Reply-To: References: Message-ID: <4789341F.1080409@v.loewis.de> > Is there any reasonable way to change the default sys.path at compile > time? (ie. add a directory). [this is off-topic for python-dev] Edit PYTHONPATH in Modules/Setup, e.g. by setting SITEPATH. Regards, Martin From lists at cheimes.de Sat Jan 12 23:38:41 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 12 Jan 2008 23:38:41 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: References: Message-ID: <47894171.7030004@cheimes.de> Christian Heimes wrote: > MA Lemburg has suggested a per user site-packages directory in the > "pkgutil, pkg_resource and Python 3.0 name space packages" thread. I've > written a short PEP about it for Python 2.6 and 3.0. Addition: An user has requested a new option to suppress the user site packages directory: -s : don't add user site directory to sys.path; also PYTHONNOUSERSITE Christian From python at rcn.com Sun Jan 13 00:32:31 2008 From: python at rcn.com (Raymond Hettinger) Date: Sat, 12 Jan 2008 15:32:31 -0800 Subject: [Python-Dev] PySequence_Concat for dicts References: <20080111202116.AEX21061@ms19.lnh.mail.rcn.net> <25D06FB4-2906-4531-A4D4-BF1B1396F9FA@northwestern.edu> Message-ID: <004901c85573$67c9af50$6800a8c0@RaymondLaptop1> [Raymond] >>> When does it come-up that you want a third summed dict >>> while keeping the two originals around unchanged? Does >>> it matter that the addition is non-commutative? Would >>> a + b + c produce an intermediate a/b combo and then >>> another new object for a/b/c so that the entries in >>> a get copied twice and memory usage has to hold a, b, >>> a/b, c, and a/b/c in memory all at the same time? [Jared] > There is no way around it, this will be less efficient than the > inplace operation. If there were a precedent for calling a method > like update and returning itself instead of None, I would suggest > that, but since this is the way that has already been established for > lists, it seems a natural extension. Not natural, just inefficient and cute. Also, there was no answer to the question about use cases. AFAICT, this feature has never been requested. The closest was a feature request for a variant of update() that avoided overwrites when a duplicate key was encountered -- Guido rejected that one a long time ago. Your previous note suggests that there are alternative interpretations of what the syntax could mean and that's not good a good thing. That sort of ambiguity damages the language. It is not even clear where the appropriate operators would be +-* or the set operators &|^-. How about we keep sets for set operations and dict for mapping operations and not foolishly conflate the two just because we can. The mapping API is central to the language. Altering it should be approached with a great deal of care. Also, the argument that we used + for lists so now we have to do it for dicts is a weak one -- they are completely different animals. Operators are not the solution to all problems. In this case, we don't even have a problem to be solved; there is just an urge to hypergeneralize what was done for other datatypes where it was appropriate. The .update() method we have now is explicit, clear about its intent, and efficient. IMO, the only thing this proposal has going for it is that it is cute. Raymond From alexandre at peadrop.com Sun Jan 13 01:33:38 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Sat, 12 Jan 2008 19:33:38 -0500 Subject: [Python-Dev] [Python-3000] inst_persistent_id In-Reply-To: References: Message-ID: On Jan 12, 2008 12:25 PM, Jim Fulton wrote: > If there are no objections, I'll update the Python 2 documentation to > describe this and add a test. The comment above suggests that this > hook is in pickle and cPickle. It is in cPickle, but was removed from > pickle. I propose to add it back to pickle -- mainly for consistency > with cPickle. I'll need to double check how this interacts with the > type dispatching in pickle protocol 2. > > Any objections? > Well, in Python 3K, inst_persistent_id() won't be usable, since PyInstance_Type was removed. Adding (actually supporting) this feature in Python 2.x will make it slightly harder to port code. So, I think it would probably best to leave it as it is right now -- i.e., undocumented and unsupported. By the way, you might be interested to look at my work on pickle [1] for Python 3K. As part of last year Summer of Code, I removed the differences between the Python and C implementation of pickle, and thus allowing the C version to be transparently used, instead of the Python one, when it is available. Currently, I am polishing the code for inclusion into the main branch. I also started to work on the next version of the pickle protocol, that will make it more suitable for Python 3K. If you are interested to help out, just send me an email and I will explain you what needs to be done. -- Alexandre .. [1] http://peadrop.com/alex-py3k/?file/91639e5487dc/Modules/_picklemodule.c From greg at krypto.org Sun Jan 13 02:08:05 2008 From: greg at krypto.org (Gregory P. Smith) Date: Sat, 12 Jan 2008 17:08:05 -0800 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <47894171.7030004@cheimes.de> References: <47894171.7030004@cheimes.de> Message-ID: <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> On 1/12/08, Christian Heimes wrote: > > Christian Heimes wrote: > > MA Lemburg has suggested a per user site-packages directory in the > > "pkgutil, pkg_resource and Python 3.0 name space packages" thread. I've > > written a short PEP about it for Python 2.6 and 3.0. > > Addition: > An user has requested a new option to suppress the user site packages > directory: > > -s : don't add user site directory to sys.path; also PYTHONNOUSERSITE +0.5 Thanks for writing this up as a PEP. My main suggestion was going to be the ability to turn it off as you already mentioned. However, please consider leaving it off by default to avoid problems for installed python scripts importing user supplied code. For shared hosting environments where this becomes really useful users can easily add the -s (or whatever flag is chosen) to their programs themselves. I don't know what that'd mean on windows where #! lines don't exist. Yet another file extension to imply the flag (yuck)? A .cmd wrapper script to run python with the flag (ugh)? For security reasons we also need it disabled when the getuid() != geteuid() to avoid user supplied code being executed as another user. Defaulting to disabled would mean that security could be left up to the end user to mess up. (many systems do not allow setuid #! scripts but this issue would still apply to things run under sudo) -gps -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080112/5b094d5f/attachment-0001.htm From jyasskin at gmail.com Sun Jan 13 02:09:00 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Sat, 12 Jan 2008 17:09:00 -0800 Subject: [Python-Dev] Rounding Decimals In-Reply-To: <007601c850ca$7f2ab3e0$6800a8c0@RaymondLaptop1> References: <5d44f72f0801051857i3b13f1b0q8a9e485831dcd603@mail.gmail.com> <003101c85011$e1572530$6800a8c0@RaymondLaptop1> <5d44f72f0801060121l39c4d2a0jfdea69f644bc1822@mail.gmail.com> <5d44f72f0801061526x788b0a46lbbc17ec5dde44ceb@mail.gmail.com> <007601c850ca$7f2ab3e0$6800a8c0@RaymondLaptop1> Message-ID: <5d44f72f0801121709x7c25efbcvfa13e29864b5b577@mail.gmail.com> During the discussion about the new Rational implementation (http://bugs.python.org/issue1682), Guido and Raymond decided that Decimal should not implement the new Real ABC from PEP 3141. So I've closed http://bugs.python.org/issue1623 and won't be pursuing any of the extra rounding methods mentioned on this thread. -- Namast?, Jeffrey Yasskin From fuzzyman at voidspace.org.uk Sun Jan 13 02:24:12 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sun, 13 Jan 2008 01:24:12 +0000 Subject: [Python-Dev] [python] Re: PEP: per user site-packages directory In-Reply-To: <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> References: <47894171.7030004@cheimes.de> <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> Message-ID: <4789683C.2020005@voidspace.org.uk> Gregory P. Smith wrote: > > On 1/12/08, *Christian Heimes* > wrote: > > Christian Heimes wrote: > > MA Lemburg has suggested a per user site-packages directory in the > > "pkgutil, pkg_resource and Python 3.0 name space packages" > thread. I've > > written a short PEP about it for Python 2.6 and 3.0. > > Addition: > An user has requested a new option to suppress the user site packages > directory: > > -s : don't add user site directory to sys.path; also > PYTHONNOUSERSITE > > > +0.5 Thanks for writing this up as a PEP. > > My main suggestion was going to be the ability to turn it off as you > already mentioned. However, please consider leaving it off by default > to avoid problems for installed python scripts importing user supplied > code. For shared hosting environments where this becomes really > useful users can easily add the -s (or whatever flag is chosen) to > their programs themselves. I don't know what that'd mean on windows > where #! lines don't exist. Yet another file extension to imply the > flag (yuck)? A .cmd wrapper script to run python with the flag (ugh)? +1 from me on implementing it and having it on by default for Windows. Why do you think the user namespace overriding the system namespace be more of a problem for Windows than other platforms? This would be a really useful feature for me and it would be a shame for it not to be on by default on Windows (and another set of complexities for setuptools I suspect). Michael Foord > > For security reasons we also need it disabled when the getuid() != > geteuid() to avoid user supplied code being executed as another user. > Defaulting to disabled would mean that security could be left up to > the end user to mess up. (many systems do not allow setuid #! scripts > but this issue would still apply to things run under sudo) > > -gps > > ------------------------------------------------------------------------ > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > From guido at python.org Sun Jan 13 05:21:40 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 12 Jan 2008 20:21:40 -0800 Subject: [Python-Dev] Rounding Decimals In-Reply-To: <5d44f72f0801121709x7c25efbcvfa13e29864b5b577@mail.gmail.com> References: <5d44f72f0801051857i3b13f1b0q8a9e485831dcd603@mail.gmail.com> <003101c85011$e1572530$6800a8c0@RaymondLaptop1> <5d44f72f0801060121l39c4d2a0jfdea69f644bc1822@mail.gmail.com> <5d44f72f0801061526x788b0a46lbbc17ec5dde44ceb@mail.gmail.com> <007601c850ca$7f2ab3e0$6800a8c0@RaymondLaptop1> <5d44f72f0801121709x7c25efbcvfa13e29864b5b577@mail.gmail.com> Message-ID: On Jan 12, 2008 5:09 PM, Jeffrey Yasskin wrote: > During the discussion about the new Rational implementation > (http://bugs.python.org/issue1682), Guido and Raymond decided that > Decimal should not implement the new Real ABC from PEP 3141. So I've > closed http://bugs.python.org/issue1623 and won't be pursuing any of > the extra rounding methods mentioned on this thread. Well, I didn't really decide anything. I suggested that if the developers of Decimal preferred it, it might be better if Decimal did not implement the Real ABC, and Raymond said he indeed preferred it. Since this reduces the usefulness of numeric.Real, I'm somewhat disappointed in this outcome (as I imagine you are). However, Decimal has very little of my own sweat in it, and a lot of Raymond, Facundo (and others, including Mark), and the ABC thing is still somewhat experimental, so it wouldn't make sense for me to impose my ideas onto Decimal unilaterally, and I'm sure Raymond etc. have their reasons. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz at pythoncraft.com Sun Jan 13 05:58:02 2008 From: aahz at pythoncraft.com (Aahz) Date: Sat, 12 Jan 2008 20:58:02 -0800 Subject: [Python-Dev] OSCON 2008 Call for Proposals Message-ID: <20080113045802.GA19791@panix.com> The O'Reilly Open Source Convention (OSCON) is accepting proposals for tutorials and presentations. The submission period ends Feb 4. OSCON 2008 will be in Portland, Oregon July 21-25. For more information and to submit a proposal, see http://conferences.oreilly.com/oscon/ -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From lists at cheimes.de Sun Jan 13 10:45:04 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 13 Jan 2008 10:45:04 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> References: <47894171.7030004@cheimes.de> <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> Message-ID: <4789DDA0.4020900@cheimes.de> Gregory P. Smith wrote: > My main suggestion was going to be the ability to turn it off as you already > mentioned. However, please consider leaving it off by default to avoid > problems for installed python scripts importing user supplied code. For > shared hosting environments where this becomes really useful users can > easily add the -s (or whatever flag is chosen) to their programs > themselves. I don't know what that'd mean on windows where #! lines don't > exist. Yet another file extension to imply the flag (yuck)? A .cmd wrapper > script to run python with the flag (ugh)? So you prefer to make the per use site-package directory an opt-in option? I prefer it as an opt-out option. It's enabled by default, unless the user disables the feature with -s. I'm not sure how to solve the problem on Windows. IMHO the feature should be enabled on Windows at least but I like to keep it enabled on all systems. The PEP doesn't add a new attack vector. The problem also exist with PYTHONPATH. Paranoid programs should start with -E -s anyway and paranoid system administrators can switch a flag in site.py: # Enable per user site-packages directory # set it to False to disable the feature or True to force the feature ENABLE_USER_SITE = None If we disable the feature by default it won't be available for a lot of users. > For security reasons we also need it disabled when the getuid() != geteuid() > to avoid user supplied code being executed as another user. Defaulting to > disabled would mean that security could be left up to the end user to mess > up. (many systems do not allow setuid #! scripts but this issue would still > apply to things run under sudo) It sounds like a reasonable and easy implementable idea, at least on Unix. Windows doesn't have getuid() and geteuid(). On the other hand Windows doesn't have the suid bit, too. I also tried to check if os.stat(__main__.__file__).st_uid == os.getuid() but the real __main__ is not available in site.py. It's loaded and assigned much later. Christian From arash at getdropbox.com Fri Jan 4 00:03:02 2008 From: arash at getdropbox.com (Arash Ferdowsi) Date: Thu, 3 Jan 2008 15:03:02 -0800 Subject: [Python-Dev] Contributing to Python In-Reply-To: <3142721346143487750@unknownmsgid> References: <5d44f72f0801022352x1845d0e2yce0ff3edba1985b2@mail.gmail.com> <008401c84de3$4da25ba0$6800a8c0@RaymondLaptop1> <20080103190556.GO67953@nexus.in-nomine.org> <1295843670530617266@unknownmsgid> <3142721346143487750@unknownmsgid> Message-ID: how about that py 2.5.2 release. anybody? =D On Jan 3, 2008 2:05 PM, Bill Janssen wrote: > > > 3.x fixes, because there's no schedule for 2.6. > > > > Eh? PEP 3000 has a schedule that includes 2.6: > > OK, no schedule that I knew about :-). I'll get back to work on it. > > Bill > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/arashf%40mit.edu > -- Arash Ferdowsi CTO, Dropbox 913.707.5875 (m) From twouters at google.com Fri Jan 4 13:49:36 2008 From: twouters at google.com (Thomas Wouters) Date: Fri, 4 Jan 2008 13:49:36 +0100 Subject: [Python-Dev] Backport threading.py fix to 2.5.2? In-Reply-To: References: Message-ID: On Jan 4, 2008 2:46 AM, Guido van Rossum wrote: > See http://bugs.python.org/issue1731. Should we consider it safe to > backport r57216 to 2.5.2? This is Thomas Wouters's code to disable > spurious tracebacks when daemon threads die. We're running some 2.4 > apps with (a variant of) this at Google that get many 1000s of > invocations a day, so I'm pretty confident that it works. > I'm also pretty confident it works, although it isn't really guaranteed to catch *all* such situations. No reason not to backport it, just a remark about how it checks to see if Python is shutting down. It is, however, incredibly unlikely '_sys' won't be gone when we check for it if the exception is indeed a spurious one. What could happen is that a legitimate exception happens right before interpreter shutdown, then a thread switch occurs before Python tries to report the exception, the interpreter exits, and then the daemonic thread starts to report its exception. The only way to catch that is to do the 'are we exiting' check in the C code that does the actual thread-exception reporting. I'm not sure if it's worth it, as the timing has to be pretty exact for that to happen, and you wouldn't want to introduce a bug there; could be years before someone figures it out :P -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080104/caed8baa/attachment-0001.htm From status at bugs.python.org Fri Jan 11 19:06:08 2008 From: status at bugs.python.org (Tracker) Date: Fri, 11 Jan 2008 18:06:08 +0000 (UTC) Subject: [Python-Dev] Summary of Tracker Issues Message-ID: <20080111180608.3782E783A1@psf.upfronthosting.co.za> ACTIVITY SUMMARY (01/04/08 - 01/11/08) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1323 open (+31) / 11946 closed (+18) / 13269 total (+49) Open issues with patches: 436 Average duration of open issues: 674 days. Median duration of open issues: 951 days. Open Issues Breakdown open 1309 (+31) pending 14 ( +0) Issues Created Or Reopened (51) _______________________________ documentation on metaclasses is incomplete and misleading 01/05/08 CLOSED http://bugs.python.org/issue1734 reopened georg.brandl filecmp.dircmp does exact match only 01/04/08 http://bugs.python.org/issue1738 created flxkid Testing CIA.vc 01/05/08 CLOSED http://bugs.python.org/issue1739 created loewis use unittest for test_logging 01/06/08 http://bugs.python.org/issue1740 created pitrou patch .pypirc not found on windows 01/06/08 http://bugs.python.org/issue1741 created gerdus os.path.relpath fails when path == start 01/06/08 CLOSED http://bugs.python.org/issue1742 created townerj IDLE fails to launch 01/06/08 http://bugs.python.org/issue1743 created richjtd readline module - set/get quote delimiters 01/06/08 http://bugs.python.org/issue1744 created loic patch Backport of PEP 3102 "keyword-only arguments" to 2.6 01/06/08 http://bugs.python.org/issue1745 created robin.stocker patch ZIP files with archive comments longer than 4k not recognized as 01/06/08 http://bugs.python.org/issue1746 created alanmcintyre issubclass(NotSubclassOfObject, InstanceOfAbcMeta) fails instead 01/06/08 CLOSED http://bugs.python.org/issue1747 created jyasskin patch contextlib.contextmanager does not use functools.wraps 01/06/08 http://bugs.python.org/issue1748 created pitrou Test: This is title 01/06/08 CLOSED http://bugs.python.org/issue1750 created loewis Fast BytesIO implementation + misc changes 01/07/08 http://bugs.python.org/issue1751 created alexandre.vassalotti patch logging.basicConfig misleading behaviour 01/07/08 CLOSED http://bugs.python.org/issue1752 created imax TextIOWrapper.write writes utf BOM for every string 01/07/08 CLOSED http://bugs.python.org/issue1753 created erickt WindowsError messages are not properly encoded 01/07/08 http://bugs.python.org/issue1754 created Romulo A. Ceccon Misspelling in future.c in 2.5.1 source (handl should be handle) 01/07/08 CLOSED http://bugs.python.org/issue1755 created brad -m broken in trunk 01/07/08 CLOSED http://bugs.python.org/issue1756 created gvanrossum Decimal hash depends on current context 01/07/08 CLOSED http://bugs.python.org/issue1757 created marketdickinson Wrong link in documentation 01/07/08 CLOSED http://bugs.python.org/issue1758 created martin.marcher Backport of PEP 3129 "class decorators" 01/08/08 http://bugs.python.org/issue1759 created tiran patch PEP 341 is not reflected in the documentation 01/08/08 http://bugs.python.org/issue1760 created Yinon Bug in re.sub() 01/08/08 CLOSED http://bugs.python.org/issue1761 created jmravon Inheriting from ABC slows Decimal down. 01/08/08 http://bugs.python.org/issue1762 created jyasskin Winpath module - easy access to Windows directories like My Docu 01/08/08 http://bugs.python.org/issue1763 created tiran patch Remove cmp parameter to list.sort() and builtin.sorted() 01/09/08 http://bugs.python.org/issue1771 created gvanrossum popen fails when there are two or more pathnames/parameters with 01/09/08 http://bugs.python.org/issue1772 created shoermann Reference to Python issue tracker incorrect 01/09/08 CLOSED http://bugs.python.org/issue1773 created aioryi Reference to New style classes documentation is incorrect 01/09/08 CLOSED http://bugs.python.org/issue1774 created aioryi filehandle.write() does not return None (Python 3.0a2) 01/09/08 http://bugs.python.org/issue1775 created aroberge __import__ must not accept filenames 01/09/08 CLOSED http://bugs.python.org/issue1776 created tiran ElementTree/cElementTree findtext inconsistency 01/09/08 http://bugs.python.org/issue1777 created gmonroe SyntaxError.offset sometimes wrong 01/09/08 http://bugs.python.org/issue1778 created AchimGaedke int("- 1") is valud, but float("- 1") isn't. Which is right? 01/10/08 http://bugs.python.org/issue1779 created gvanrossum Decimal constructor accepts newline terminated strings 01/10/08 http://bugs.python.org/issue1780 created marketdickinson ConfigParser: add_section('DEFAULT') causes duplicate sections. 01/10/08 http://bugs.python.org/issue1781 created tlesher PyModule_AddIntConstant and PyModule_AddStringConstant can leak 01/10/08 http://bugs.python.org/issue1782 created hniksic nonexistent data items declared as exports in sysmodule.h 01/10/08 http://bugs.python.org/issue1783 created jlaurila Error with OptionParser.parse_args() 01/10/08 CLOSED http://bugs.python.org/issue1784 created draghuram "inspect" gets broken by some descriptors 01/10/08 http://bugs.python.org/issue1785 created dmaurer pdb should set stdin+stdout around exec call 01/11/08 http://bugs.python.org/issue1786 created gvanrossum patch segfault in obmalloc.c 01/11/08 CLOSED http://bugs.python.org/issue1787 created ctheune Outdated link in the tutorial 01/11/08 CLOSED http://bugs.python.org/issue1788 created spe-anatol incorrect assumption about unsigned long byte size 01/11/08 http://bugs.python.org/issue1789 created spe-anatol xmlrpclib ServerProxy page has out-of-date content 01/11/08 http://bugs.python.org/issue1790 created vilya The Library Reference still refers to the old bug tracker. 01/11/08 CLOSED http://bugs.python.org/issue1791 created vilya o(n*n) marshal.dumps performance for largish objects with patch 01/11/08 http://bugs.python.org/issue1792 created aaron_watters patch ctypes.util.find_msvcrt() function 01/11/08 http://bugs.python.org/issue1793 created theller patch Hot keys must work in any keyboard layout 01/11/08 http://bugs.python.org/issue1794 created Nashev logging: need a way to discard Logger objects 01/06/08 http://bugs.python.org/issue932563 reopened georg.brandl Issues Now Closed (349) _______________________ 2to3 crashes on input files with no trailing newlines 136 days http://bugs.python.org/issue1001 admin pyexpat.XMParserType broken (was: pydoc doesn't work on pyexpat) 134 days http://bugs.python.org/issue1020 admin Tkinter binding involving Control-spacebar raises unicode error 133 days http://bugs.python.org/issue1028 admin patch py3k _bsddb.c patch to use the new buffer API 133 days http://bugs.python.org/issue1036 admin patch Ill-coded identifier crashes python when coding spec is utf-8 132 days http://bugs.python.org/issue1037 admin [py3k] pdb does not work in python 3000 132 days http://bugs.python.org/issue1038 admin ssl.py shouldn't change class names from 2.6 to 3.x 129 days http://bugs.python.org/issue1065 admin Implement PEPs 3109, 3134 129 days http://bugs.python.org/issue1066 admin patch test_smtplib failures (caused by asyncore) 128 days http://bugs.python.org/issue1067 admin patch invalid file encoding results in "SyntaxError: None" 129 days http://bugs.python.org/issue1069 admin unicode identifiers in error messages 129 days http://bugs.python.org/issue1070 admin file.seek allows float arguments 127 days http://bugs.python.org/issue1081 admin Warning required when calling register() on an ABCMeta subclass 37 days http://bugs.python.org/issue1109 admin patch _curses issues on 64-bit big-endian (e.g, AIX) 125 days http://bugs.python.org/issue1114 akuchling patch Document inspect.getfullargspec() 122 days http://bugs.python.org/issue1121 admin No tests for inspect.getfullargspec() 122 days http://bugs.python.org/issue1127 admin patch Idle - Save (buffer) - closes IDLE and does not save file (Windo 121 days http://bugs.python.org/issue1130 admin Parsing a simple script eats all of your memory 120 days http://bugs.python.org/issue1134 admin patch pyexpat patch for changing buffer_size 121 days http://bugs.python.org/issue1137 AchimGaedke patch Fixer needed for __future__ imports 119 days http://bugs.python.org/issue1138 admin test_urllib2net fails on test_ftp 116 days http://bugs.python.org/issue1157 admin Should itertools.count work for arbitrary integers? 113 days http://bugs.python.org/issue1165 admin allow subclassing of bytes type 111 days http://bugs.python.org/issue1171 admin patch Paticular decimal mod operation wrongly output NaN. 110 days http://bugs.python.org/issue1182 marketdickinson test fixes for immutable bytes change 108 days http://bugs.python.org/issue1184 admin patch os.system() encoding bug on Windows 105 days http://bugs.python.org/issue1193 admin Restrict Google search to docs when in the docs subtree? 101 days http://bugs.python.org/issue1218 georg.brandl 3.0 library/stdtypes.rst 96 days http://bugs.python.org/issue1229 admin option.dest not set when callback called with optparse 90 days http://bugs.python.org/issue1243 gvanrossum subprocess Popen wait() function hangs when stdout is > 20480 88 days http://bugs.python.org/issue1256 tiran PEP 3137: make bytesobject.c methods 88 days http://bugs.python.org/issue1261 admin patch PEP 3137 patch - str8/str comparison should return false 87 days http://bugs.python.org/issue1263 admin patch pdb bug with "with" statement 87 days http://bugs.python.org/issue1265 admin Py3K cannot run as ``python -S`` 77 days http://bugs.python.org/issue1267 admin patch PyBytes (buffer) .extend method needs to accept any iterable of 83 days http://bugs.python.org/issue1283 admin patch Fixes for profile/cprofile 79 days http://bugs.python.org/issue1302 admin patch adapt str8 constructor to bytes constructor 79 days http://bugs.python.org/issue1303 admin patch fix a few PyInt_Check vs PyLong_Check problems 75 days http://bugs.python.org/issue1316 admin patch zipimport.zipimporter.archive undocumented and untested 73 days http://bugs.python.org/issue1325 georg.brandl Fix truncate on Windows, this time for real 73 days http://bugs.python.org/issue1330 admin patch More fixes for Windows 73 days http://bugs.python.org/issue1331 admin patch Fix for test_netrc on Windows 71 days http://bugs.python.org/issue1345 admin patch BaseHTTPServer writing strings to bytes interface 71 days http://bugs.python.org/issue1347 admin patch more uses of ord() in plat-mac/ic.py 71 days http://bugs.python.org/issue1349 admin patch Add getsize() to io instances 70 days http://bugs.python.org/issue1351 admin patch Two bsddb tests temporarily commented out in py3k branch 66 days http://bugs.python.org/issue1371 admin turn off socket timeout in test_xmlrpc 65 days http://bugs.python.org/issue1373 admin patch test_import breaks on Linux 59 days http://bugs.python.org/issue1377 admin reloading imported modules sometimes fail with 'parent not in sy 64 days http://bugs.python.org/issue1379 admin patch fix for test_asynchat and test_asyncore on pep3137 branch 64 days http://bugs.python.org/issue1380 admin patch py3k-pep3137: patch for test_ctypes 63 days http://bugs.python.org/issue1382 admin patch Windows fix for inspect tests 63 days http://bugs.python.org/issue1384 admin patch hmac module violates RFC for some hash functions, e.g. sha512 63 days http://bugs.python.org/issue1385 admin py3k-pep3137: patch to ensure that all codecs return bytes 63 days http://bugs.python.org/issue1386 admin patch py3k-pep3137: patch for hashlib on Windows 63 days http://bugs.python.org/issue1387 admin patch py3k-pep3137: possible ref leak in ctypes 63 days http://bugs.python.org/issue1388 admin py3k-pep3137: struct module is leaking references 63 days http://bugs.python.org/issue1389 admin py3k-pep3137: issue warnings / errors on str(bytes()) and simila 62 days http://bugs.python.org/issue1392 admin patch function comparing lacks NotImplemented error 61 days http://bugs.python.org/issue1393 gvanrossum py3k: duplicated line endings when using read(1) 62 days http://bugs.python.org/issue1395 admin py3k-pep3137: patch for mailbox 61 days http://bugs.python.org/issue1396 admin patch Py3k's print() flushing problem 60 days http://bugs.python.org/issue1400 admin py_compile and compileall need unit tests 59 days http://bugs.python.org/issue1403 admin Use widechar api for os.environ 59 days http://bugs.python.org/issue1406 admin patch new keyword-only function parameters interact badly with nested 59 days http://bugs.python.org/issue1409 admin Fix for refleak tests 58 days http://bugs.python.org/issue1414 admin patch py3k: pythonw.exe fails because std streams a missing 58 days http://bugs.python.org/issue1415 admin Unicode literals in tokenize.py and tests. 56 days http://bugs.python.org/issue1420 admin patch Writing to an invalid fd doesn't raise an exception 56 days http://bugs.python.org/issue1422 admin py3k: readline and rlcompleter doesn't list choices 56 days http://bugs.python.org/issue1424 admin readline: no display matches hook set 56 days http://bugs.python.org/issue1425 admin readline module needs a review 56 days http://bugs.python.org/issue1426 admin Calling base class methods is slow due to __instancecheck__ over 54 days http://bugs.python.org/issue1438 admin proposed 3000 patch for socket.py - "socket GC worries" 54 days http://bugs.python.org/issue1439 admin patch Checks for PySys_GetObject("std???") == Py_None 54 days http://bugs.python.org/issue1440 admin patch SSL patch for Python 3000 52 days http://bugs.python.org/issue1451 admin patch VS2008, quick hack for distutils.msvccompiler 50 days http://bugs.python.org/issue1455 admin patch inet_pton redefined while building with windows SDK 6.0 48 days http://bugs.python.org/issue1464 admin patch SSL tests leak memory 48 days http://bugs.python.org/issue1469 admin py3k unit tests are removing %TEMP% dir on Windows 48 days http://bugs.python.org/issue1470 admin Drop _EXPORTS macros in PCbuild9 48 days http://bugs.python.org/issue1473 admin pythoncore.vcproj fails to generate buildinfo (when spaces in pa 47 days http://bugs.python.org/issue1478 admin patch csv module is leaking references 47 days http://bugs.python.org/issue1479 admin sqlite module is leaking lots of references 47 days http://bugs.python.org/issue1480 admin PCBuild9 _ssl.vcproj improperly launches build 45 days http://bugs.python.org/issue1488 admin patch Patch to remove unbound methods 44 days http://bugs.python.org/issue1493 admin patch add str.maketrans() 43 days http://bugs.python.org/issue1496 admin patch Patch to remove API to create new unbound methods 42 days http://bugs.python.org/issue1497 admin patch Rename __builtins__ to __root_namespace__? 41 days http://bugs.python.org/issue1498 admin failure behaviour of compile() is missing 41 days http://bugs.python.org/issue1499 georg.brandl 0 ** 0 documentation 41 days http://bugs.python.org/issue1501 georg.brandl Changes to PyMethod_New breaks ctypes on Windows 40 days http://bugs.python.org/issue1505 admin Removal of stale code in _csv.c / pyexpat.c 40 days http://bugs.python.org/issue1508 admin Problem with static libs on Windows 37 days http://bugs.python.org/issue1527 admin patch sys.maxfloat patch 37 days http://bugs.python.org/issue1534 admin patch Rename __builtin__ to builtins 36 days http://bugs.python.org/issue1535 admin patch test_collections: failing refleak test 35 days http://bugs.python.org/issue1539 admin Port Python/import.c to py3k branch 34 days http://bugs.python.org/issue1551 admin import distutils.msvccompiler hangs when the environment is too 32 days http://bugs.python.org/issue1557 admin x64 platform doesn't define _WIN64 32 days http://bugs.python.org/issue1558 admin 64bit Decimal can't be subclassed useful 32 days http://bugs.python.org/issue1562 facundobatista The set implementation should special-case PyUnicode instead of 31 days http://bugs.python.org/issue1564 admin Better description of 'L' repr removal in "What's New" 29 days http://bugs.python.org/issue1571 admin 404 report of SimpleXMLRPCServer is broken 29 days http://bugs.python.org/issue1572 admin Improper use of the keyword-only syntax makes the parser crash 29 days http://bugs.python.org/issue1573 admin Problems in win_getpass 28 days http://bugs.python.org/issue1578 admin Documentation patch for reversed() and __reversed__() 27 days http://bugs.python.org/issue1582 georg.brandl IDLE uses non-existent xrange() function (Py30a2) 27 days http://bugs.python.org/issue1585 admin IDLE no longer shows colour syntax highlighting in the Shell (Py 26 days http://bugs.python.org/issue1586 admin instancemethod wrapper for PyCFunctions 26 days http://bugs.python.org/issue1587 admin patch popen2.Popen3 class (Unix) documentation misleading / confusing 26 days http://bugs.python.org/issue1591 georg.brandl spacing of the builtin_format function is inconsistent 26 days http://bugs.python.org/issue1593 admin patch IDLE not working correctly on Windows (Py30a2/IDLE30a1) 26 days http://bugs.python.org/issue1601 admin Patch for TCL 8.5 support 25 days http://bugs.python.org/issue1607 admin patch test_str.py crashes 27 days http://bugs.python.org/issue1608 gvanrossum Makefile's VPATH feature is broken 24 days http://bugs.python.org/issue1613 admin New @spam.getter property syntax modifies the property in place 24 days http://bugs.python.org/issue1620 admin patch test_distutils unit test is failing rev:59499 23 days http://bugs.python.org/issue1628 admin sys.maxint is documented but should not be 23 days http://bugs.python.org/issue1630 admin Float patch for inf and nan on Windows (and other platforms) 23 days http://bugs.python.org/issue1635 admin patch urlparse.urlparse misparses URLs with query but no path 20 days http://bugs.python.org/issue1637 gvanrossum Make socket support TIPC. 21 days http://bugs.python.org/issue1646 tiran patch Make math.{floor,ceil}(float) return ints per PEP 3141 19 days http://bugs.python.org/issue1656 admin patch -E command line parameter intent 18 days http://bugs.python.org/issue1668 georg.brandl patch "World" tool ported to py3k 17 days http://bugs.python.org/issue1671 admin patch what is decimal.Context.get_manager()? 16 days http://bugs.python.org/issue1680 georg.brandl parameter name for optparse parse_args incorrect 16 days http://bugs.python.org/issue1681 georg.brandl Syntax Error exception dosen't print string; not informative 14 days http://bugs.python.org/issue1692 admin patch Please check merge from trunk 14 days http://bugs.python.org/issue1693 admin urlparse and usernames containing @ 14 days http://bugs.python.org/issue1698 orsenthil patch Word "alias" used in confusing way to compare open() and file() 10 days http://bugs.python.org/issue1702 georg.brandl Small errors in new-style classes doc 6 days http://bugs.python.org/issue1712 georg.brandl Cosmetic patch for doc/code mismatch (msilib.UuidCreate) 4 days http://bugs.python.org/issue1719 georg.brandl _tkinter.c:903: AsObj: Assertion `size < size * sizeof(Tcl_UniCh 4 days http://bugs.python.org/issue1721 admin -1e-1000 converted incorrectly 2 days http://bugs.python.org/issue1725 gvanrossum patch Remove Python/atof.c from PCBuild/pythoncore.vcproj 3 days http://bugs.python.org/issue1726 admin patch VC6 build patch for python3000 3 days http://bugs.python.org/issue1727 admin patch Documentation corrections for os module 2 days http://bugs.python.org/issue1730 georg.brandl patch Random errors on interpreter shutdown 6 days http://bugs.python.org/issue1731 gvanrossum documentation on metaclasses is incomplete and misleading 2 days http://bugs.python.org/issue1734 georg.brandl Windows installer issue (ObjectBrowser.py) 4 days http://bugs.python.org/issue1737 loewis Testing CIA.vc 0 days http://bugs.python.org/issue1739 loewis os.path.relpath fails when path == start 0 days http://bugs.python.org/issue1742 georg.brandl issubclass(NotSubclassOfObject, InstanceOfAbcMeta) fails instead 0 days http://bugs.python.org/issue1747 jyasskin patch Test: This is title 3 days http://bugs.python.org/issue1750 georg.brandl logging.basicConfig misleading behaviour 0 days http://bugs.python.org/issue1752 vsajip TextIOWrapper.write writes utf BOM for every string 0 days http://bugs.python.org/issue1753 alexandre.vassalotti Misspelling in future.c in 2.5.1 source (handl should be handle) 0 days http://bugs.python.org/issue1755 georg.brandl -m broken in trunk 0 days http://bugs.python.org/issue1756 gvanrossum Decimal hash depends on current context 1 days http://bugs.python.org/issue1757 facundobatista Wrong link in documentation 0 days http://bugs.python.org/issue1758 loewis Bug in re.sub() 3 days http://bugs.python.org/issue1761 amaury.forgeotdarc Reference to Python issue tracker incorrect 0 days http://bugs.python.org/issue1773 akuchling Reference to New style classes documentation is incorrect 0 days http://bugs.python.org/issue1774 facundobatista __import__ must not accept filenames 0 days http://bugs.python.org/issue1776 tiran Error with OptionParser.parse_args() 0 days http://bugs.python.org/issue1784 gvanrossum segfault in obmalloc.c 0 days http://bugs.python.org/issue1787 gvanrossum Outdated link in the tutorial 0 days http://bugs.python.org/issue1788 facundobatista The Library Reference still refers to the old bug tracker. 0 days http://bugs.python.org/issue1791 facundobatista How to install some stuff to /usr/sbin 2331 days http://bugs.python.org/issue452144 tiran Tcl event loop callback woes 2330 days http://bugs.python.org/issue452973 tiran Improve the ZipFile Interface 2286 days http://bugs.python.org/issue467924 georg.brandl python directory not added to PATH 2240 days http://bugs.python.org/issue482531 tiran Portable compiler option specification 2219 days http://bugs.python.org/issue490264 tiran Print line number of string if at EOF 2011 days http://bugs.python.org/issue577295 tiran option for not writing .py[co] files 1956 days http://bugs.python.org/issue602345 georg.brandl patch need easy way of decoding a literal 1920 days http://bugs.python.org/issue617979 tiran Windows binary missing IPv6 support 1918 days http://bugs.python.org/issue618593 tiran refcount problem involving debugger 1764 days http://bugs.python.org/issue699594 tiran Easy tutorial printing should be possible 1739 days http://bugs.python.org/issue714469 georg.brandl event handling support 1674 days http://bugs.python.org/issue750423 tiran Thread in threading.py can only be started once 1671 days http://bugs.python.org/issue751260 tiran inspect.getmembers broken (?) 1658 days http://bugs.python.org/issue759525 georg.brandl Zero Configuration Networking? 1559 days http://bugs.python.org/issue813922 tiran Digital Unix build fails to create ccpython.o 1512 days http://bugs.python.org/issue842171 tiran Check for signals during regular expression matches 1510 days http://bugs.python.org/issue846388 facundobatista patch PyErrr_Display and traceback.format_exception_only behaviour 1483 days http://bugs.python.org/issue860326 admin patch distutils doesn't see user-directories set via VisStudio 7.1 1442 days http://bugs.python.org/issue883969 tiran Python configured with --disable-unicode fails tests, more 1389 days http://bugs.python.org/issue919614 georg.brandl There ought to be a way to uninstall 1314 days http://bugs.python.org/issue963845 tiran Ctrl-C causes odd behaviour 1312 days http://bugs.python.org/issue964949 tiran getattr(object,name) accepts only strings 1280 days http://bugs.python.org/issue985094 tiran os constants missing 1269 days http://bugs.python.org/issue990749 tiran PyErr_Clear() vs. AsynchronousException 1236 days http://bugs.python.org/issue1009929 tiran patch Seting defaultencoding through an environment variable 1170 days http://bugs.python.org/issue1052098 tiran Memory leak in socket.py on Mac OS X 10.3 1103 days http://bugs.python.org/issue1092502 tiran subprocess fails on GetStdHandle in interactive GUI 1055 days http://bugs.python.org/issue1124861 dserodio No os.statvfs on FreeBSD 1048 days http://bugs.python.org/issue1145231 tiran No os.spawn*p* on Windows 1023 days http://bugs.python.org/issue1166378 tiran Time module is missing inverse of gmtime() 894 days http://bugs.python.org/issue1245224 georg.brandl patch cross compile and mingw support 800 days http://bugs.python.org/issue1339673 tiran patch PythonD new file: python2.4/plat-ms-dos5/djstat.py 788 days http://bugs.python.org/issue1351036 tiran patch socket.recv(OOB) raises exception on closed socket 757 days http://bugs.python.org/issue1379209 facundobatista re search infinite loop 668 days http://bugs.python.org/issue1448325 tiran Add .format() method to str and unicode 644 days http://bugs.python.org/issue1463370 admin patch Make -tt the default 634 days http://bugs.python.org/issue1469190 admin patch Fix test_augassign in p3yk 632 days http://bugs.python.org/issue1470424 admin patch fix test_exceptions in p3yk 632 days http://bugs.python.org/issue1470459 admin patch fix test_bisect in p3yk 632 days http://bugs.python.org/issue1470460 admin patch Mention coercion removal in Misc/NEWS 632 days http://bugs.python.org/issue1470502 admin partially fix test_class in p3yk 632 days http://bugs.python.org/issue1470504 admin patch fix test_descrtut in p3yk 632 days http://bugs.python.org/issue1470515 admin patch fix test_enumerate in p3yk 632 days http://bugs.python.org/issue1470522 admin patch Fix test_descr in p3yk 632 days http://bugs.python.org/issue1470536 admin patch fix test_getargs2 in p3yk 632 days http://bugs.python.org/issue1470543 admin patch fix test_inspect in p3yk 632 days http://bugs.python.org/issue1470560 admin patch fix test_itertools in p3yk 632 days http://bugs.python.org/issue1470566 admin patch Forbid iteration over strings 630 days http://bugs.python.org/issue1471291 admin patch make range be xrange 628 days http://bugs.python.org/issue1472639 admin patch Remove types.InstanceType and new.instance 590 days http://bugs.python.org/issue1495675 admin patch Fix test_exceptions.py 589 days http://bugs.python.org/issue1496135 admin patch deprecate METH_OLDARGS 587 days http://bugs.python.org/issue1496957 admin patch Change *args from a tuple to list 585 days http://bugs.python.org/issue1498441 admin patch (py3k) Remove the sets module 581 days http://bugs.python.org/issue1500611 admin patch Remove the repr()-backticks 581 days http://bugs.python.org/issue1500623 admin patch There should be a Python build using Visual Studio 2005 572 days http://bugs.python.org/issue1504947 tiran Add some dicts to datetime module 570 days http://bugs.python.org/issue1506296 tiran Remove reduce() 559 days http://bugs.python.org/issue1513249 admin patch Move reduce() to functools 558 days http://bugs.python.org/issue1513870 admin patch Under OS X, compiling against local readline fails 552 days http://bugs.python.org/issue1516068 tiran Remove sys.exitfunc 552 days http://bugs.python.org/issue1516375 admin patch Add some explication to PEP 3100 542 days http://bugs.python.org/issue1522038 admin patch Remove operator.truth() and operator.abs() 542 days http://bugs.python.org/issue1522059 admin patch tarfile chokes on ipython archive on Windows 532 days http://bugs.python.org/issue1527974 lars.gustaebel tarfile __slots__ addition 511 days http://bugs.python.org/issue1540385 lars.gustaebel patch xrange that supports longs, etc 500 days http://bugs.python.org/issue1546078 admin patch Create a real zip iterator object; not using itertools.izip 500 days http://bugs.python.org/issue1546297 admin patch set literals 497 days http://bugs.python.org/issue1547796 admin patch "if x in setliteral" peepholer optimization 496 days http://bugs.python.org/issue1548082 admin patch set comprehensions 496 days http://bugs.python.org/issue1548388 admin patch Implementation of PEP 3102 Keyword Only Argument 494 days http://bugs.python.org/issue1549670 admin patch Ellipsis literal "..." 492 days http://bugs.python.org/issue1550786 admin patch make exec a function 492 days http://bugs.python.org/issue1550800 admin patch shutil.copyfile incomplete on NTFS 477 days http://bugs.python.org/issue1559684 georg.brandl copy() method of dictionaries is not "deep" 481 days http://bugs.python.org/issue1560327 akuchling RE (regular expression) matching stuck in loop 470 days http://bugs.python.org/issue1566086 georg.brandl help(x) for keywords too 456 days http://bugs.python.org/issue1572210 georg.brandl modulefinder changes for py3k 436 days http://bugs.python.org/issue1585966 admin patch __bool__ instead of __nonzero__ 411 days http://bugs.python.org/issue1600346 admin patch sys.intern() 409 days http://bugs.python.org/issue1601678 admin patch Optional Argument Syntax 400 days http://bugs.python.org/issue1607548 admin patch tempfile.TemporaryFile differences between linux and windows 389 days http://bugs.python.org/issue1615275 georg.brandl extended slicing for bytes objects 385 days http://bugs.python.org/issue1617678 admin patch sys.intern() 2to3 fixer 383 days http://bugs.python.org/issue1619049 admin patch Bug fixes for int unification branch 382 days http://bugs.python.org/issue1619846 admin patch update to PEP 344 - exception attributes 369 days http://bugs.python.org/issue1626538 admin patch backticks will not be used at all 368 days http://bugs.python.org/issue1627052 admin patch Win32: Fix build when you have TortoiseSVN but no .svn/* 367 days http://bugs.python.org/issue1628061 admin patch Win32: Add bytesobject.c to pythoncore.vcproj 367 days http://bugs.python.org/issue1628062 admin patch clarify 80-char limit 366 days http://bugs.python.org/issue1628906 admin The Unicode "lazy strings" patches 366 days http://bugs.python.org/issue1629305 admin patch Implement named exception cleanup 364 days http://bugs.python.org/issue1630248 admin patch New exception syntax 362 days http://bugs.python.org/issue1631942 admin patch Py3k: Fix pybench so it runs 359 days http://bugs.python.org/issue1634499 admin patch PyFloat_FromString deprecated form 338 days http://bugs.python.org/issue1650903 admin Hide iteration variable in list comprehensions 325 days http://bugs.python.org/issue1660500 admin patch inspect.getargspec() fails with keyword-only arguments 315 days http://bugs.python.org/issue1668565 admin methods for bytes 314 days http://bugs.python.org/issue1669633 admin patch Remove Py_PROTO from socket in py3k 313 days http://bugs.python.org/issue1670209 admin patch Class Decorators 312 days http://bugs.python.org/issue1671208 admin patch New File I/O type for Python 3000, plus .h and unit tests 312 days http://bugs.python.org/issue1671314 admin patch PEP 3114 -- next() -> __next__() 306 days http://bugs.python.org/issue1675363 admin patch Removal of Tuple Parameter Unpacking [PEP3113] 302 days http://bugs.python.org/issue1678060 admin patch Lib/io.py open uses unset "bs" 300 days http://bugs.python.org/issue1679498 admin remove sys.exitfunc, rewrite atexit in C 298 days http://bugs.python.org/issue1680961 admin patch PEP 3115 patch 298 days http://bugs.python.org/issue1681101 admin patch subprocess.Popen fails with socket._fileobject on Windows 297 days http://bugs.python.org/issue1681674 tiran xreload.py won't update class docstrings 294 days http://bugs.python.org/issue1683288 admin modindex.html builds incorrectly in Python 3 292 days http://bugs.python.org/issue1684811 admin Add IllegalStateError 291 days http://bugs.python.org/issue1685642 admin patch More PEP 3116 classes 290 days http://bugs.python.org/issue1686273 admin patch csv sniff example 290 days http://bugs.python.org/issue1686390 georg.brandl Inconsistent Exceptions for Readonly Attributes 289 days http://bugs.python.org/issue1687163 admin Remove redundant code in ntpath.walk() 273 days http://bugs.python.org/issue1696393 georg.brandl patch Property with -> annotation triggers assert 272 days http://bugs.python.org/issue1697248 admin types.InstanceType is missing but used by pydoc 271 days http://bugs.python.org/issue1697782 admin getstate/setstate for incremental codecs 270 days http://bugs.python.org/issue1698994 admin patch 'nonlocal x' at top level crashes interpreter 259 days http://bugs.python.org/issue1705365 admin Implementation of @abstractmethod for PEP 3119 257 days http://bugs.python.org/issue1706989 admin patch Make isinstance/issubclass overloadable for PEP 3119 255 days http://bugs.python.org/issue1708353 admin patch PEP 3132: extended unpacking 249 days http://bugs.python.org/issue1711529 admin patch Patch for PEP 3109 245 days http://bugs.python.org/issue1713889 admin patch PEP 3123 implementation 238 days http://bugs.python.org/issue1718153 admin patch Remove backslash escapes from tokenize.c. 235 days http://bugs.python.org/issue1720390 admin patch automatic imports 237 days http://bugs.python.org/issue1720992 juanmabc3 dict size changes during iter 227 days http://bugs.python.org/issue1724999 admin patch subprocess: unreliability of returncode not clear from docs 223 days http://bugs.python.org/issue1727024 georg.brandl First steps towards new super (PEP 3135) 223 days http://bugs.python.org/issue1727209 admin patch os._execvpe raises assignment error in python 3000 svn 217 days http://bugs.python.org/issue1730441 admin Kill StandardError 209 days http://bugs.python.org/issue1735485 admin patch Interactive help raise exception while listing modules 201 days http://bugs.python.org/issue1739659 admin struni: fix for test_cmd_line 180 days http://bugs.python.org/issue1751493 admin patch struni: Replace assert_(==) with failUnlessEqual 180 days http://bugs.python.org/issue1751515 admin patch struni: str() doesn't call __str__() of subclasses of str 180 days http://bugs.python.org/issue1751598 admin Dict comprehensions 179 days http://bugs.python.org/issue1751800 admin patch Patch for Windows build 179 days http://bugs.python.org/issue1751801 admin patch struni: Three ctypes tests are SEGFAULTing 179 days http://bugs.python.org/issue1751885 admin struni: help() is broken 179 days http://bugs.python.org/issue1751932 admin struni: gettext fixes 179 days http://bugs.python.org/issue1751958 admin patch Typo in Lib/lib-tk/ScrolledText.py 179 days http://bugs.python.org/issue1751965 admin patch Use the bytes type in asynchat 179 days http://bugs.python.org/issue1752173 admin patch PyHeapTypeObject fix 179 days http://bugs.python.org/issue1752184 admin patch struni: for for poll, pep263, bigmem and a bit pickletool 179 days http://bugs.python.org/issue1752195 admin patch struni: _fileio fixes for Windows 179 days http://bugs.python.org/issue1752225 admin patch Use the Unicode API in dlmodule.c 179 days http://bugs.python.org/issue1752229 admin patch reference leak in _PyUnicode_AsDefaultEncodedString 179 days http://bugs.python.org/issue1752317 admin patch fix failing unit tests in mmap in struni branch 178 days http://bugs.python.org/issue1752647 admin patch Lib/regrtest.py -x 'test_xxx' does not work 178 days http://bugs.python.org/issue1753310 admin patch struni: assertion in Windows debug build 177 days http://bugs.python.org/issue1753395 admin test_urllib2.test_fie passes 177 days http://bugs.python.org/issue1753889 admin patch struni: Bytes support for TextIOWrapper.write() 175 days http://bugs.python.org/issue1754339 admin patch struni: Various patches for windows 175 days http://bugs.python.org/issue1754484 admin patch document default values for sort parameters 174 days http://bugs.python.org/issue1755097 georg.brandl urllib2 tests pass 174 days http://bugs.python.org/issue1755133 admin patch struni: corrections for test_cProfile 174 days http://bugs.python.org/issue1755176 admin patch Deadlocks with fork() and multithreading 175 days http://bugs.python.org/issue1755179 facundobatista struni: corrections in ftplib 174 days http://bugs.python.org/issue1755206 admin patch struni: correction for sockets on win32 174 days http://bugs.python.org/issue1755214 admin patch struni: correction for _winreg module 174 days http://bugs.python.org/issue1755229 admin patch Add support for seeking/writing beyond EOF to io.BytesIO 170 days http://bugs.python.org/issue1757683 admin patch struni: make test_ucn pass 170 days http://bugs.python.org/issue1757758 admin patch struni: fix str/bytes errors for test_oldmailbox 170 days http://bugs.python.org/issue1757774 admin patch struni: make test_mailbox and test_old_mailbox pass 170 days http://bugs.python.org/issue1757839 admin patch struni: Fix test_macostools 168 days http://bugs.python.org/issue1758570 admin patch pyexpat unit tests - str/uni branch 167 days http://bugs.python.org/issue1759016 admin patch struni pulldom: Don't use 'types' to check strings 166 days http://bugs.python.org/issue1759922 admin patch struni: Fix test_aepack by converting 4cc's to bytes 164 days http://bugs.python.org/issue1761465 admin patch struni: test_xml_etree.py 163 days http://bugs.python.org/issue1762412 admin patch struni: test_urllib2, test_cookielib 162 days http://bugs.python.org/issue1762940 admin patch Fix for test_socketserver for Py3k 159 days http://bugs.python.org/issue1764815 admin patch Fix for test_zipimport 157 days http://bugs.python.org/issue1766592 admin patch test_csv struni fixes + unicode support in _csv 156 days http://bugs.python.org/issue1767398 admin patch test_asyncore fix 155 days http://bugs.python.org/issue1767834 admin patch Fix for failing test_scriptpackages in py3k-struni 153 days http://bugs.python.org/issue1768976 admin patch Fix for failing test_plistlib in py3k-struni 153 days http://bugs.python.org/issue1769016 admin patch struni: test_xml_etree_c 152 days http://bugs.python.org/issue1769767 admin patch Remove cStringIO usage 151 days http://bugs.python.org/issue1770008 admin patch ctypes: c_char now uses bytes and not str (unicode) 151 days http://bugs.python.org/issue1770355 admin patch Errors in site.py not reported properly 150 days http://bugs.python.org/issue1771260 admin Misc improvements for the io module 150 days http://bugs.python.org/issue1771364 admin patch dir() on traceback objects returns an empty list 148 days http://bugs.python.org/issue1772489 admin exec() doesn't take an open file 147 days http://bugs.python.org/issue1772686 admin Unify __builtins__ -> __builtin__ 145 days http://bugs.python.org/issue1774369 admin patch Make it possible to use SVK to develop Python 145 days http://bugs.python.org/issue1774414 admin patch Binding fails 144 days http://bugs.python.org/issue1774736 admin Override flags set by IOBase in io.StringIO 144 days http://bugs.python.org/issue1774828 admin patch Convert str to bytes in io.BytesIO.__init__ 144 days http://bugs.python.org/issue1774833 admin patch utf-32 codecs 143 days http://bugs.python.org/issue1775604 admin patch memoryview('test') is causing a segfault 141 days http://bugs.python.org/issue1777057 admin Top Issues Most Discussed (10) ______________________________ 19 Move Demo/classes/Rat.py to Lib/rational.py and fix it up. 21 days open http://bugs.python.org/issue1682 13 Decimal constructor accepts newline terminated strings 2 days open http://bugs.python.org/issue1780 13 Bug in re.sub() 3 days closed http://bugs.python.org/issue1761 12 logging: need a way to discard Logger objects 5 days open http://bugs.python.org/issue932563 11 IDLE fails to launch 5 days open http://bugs.python.org/issue1743 10 option for not writing .py[co] files 1956 days closed http://bugs.python.org/issue602345 10 WindowsError messages are not properly encoded 4 days open http://bugs.python.org/issue1754 9 Winpath module - easy access to Windows directories like My Doc 3 days open http://bugs.python.org/issue1763 9 Do not assume signed integer overflow behavior 29 days open http://bugs.python.org/issue1621 9 Paticular decimal mod operation wrongly output NaN. 110 days closed http://bugs.python.org/issue1182 From g.brandl at gmx.net Sun Jan 13 18:28:31 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 13 Jan 2008 17:28:31 +0000 Subject: [Python-Dev] complaints.. In-Reply-To: References: Message-ID: Jay schrieb: Only addressing the easier points here: > #---------------------------------------------------------- > # flaw #2 > # > # In functions, reads are scoped. Writes are not. > # > > A = "1" > > def F1(): > A = "2" # not an error > > def F2(): > #B = A # error > A = "3" > > F1() > F2() > print(A) "Writes" creates a new local. If you want to modify the enclosing binding, the new "nonlocal" statement in 2.6/3.0 will enable that. > #---------------------------------------------------------- > # flaw #3: > # lambda is neutered > # It allows just one expression, and no statements Lambda can't be altered to allow statements *and* have a consistent indentation-based syntax. > #---------------------------------------------------------- > # flaw #5 > # > # for loops suck > # > # It should be easy to iterate from 0 to n, not 0 to n - 1, > # thereby knowing why the loop terminated > # > > #This should work: > > # print the first even number, if there are any > > A = [1, 3] > for i in range(0, len(A)): > if ((A[i] % 2) == 0): > print("even number found") > break; > if (i == len(A)): > print("no even numbers found") The for loop has an else clause. for x in A: if x % 2 == 0: print "even number found" break else: print "no even numbers found" > Flaw #6 > > The docs are very good. > > However the reference doesn't give much in the way > of semantic description. > > It is surprising that an experienced programmer MUST > depend on the tutorial or any examples or semantics. > Light on examples, ok for reference. > > The language reference is little more than the grammar in parts. > > There needs to be links from the reference back to the tutorial. > > Perhaps merge the indices for Tutorial, Language Reference, Library > Reference. > Or maybe that's what search is for. Look at the new docs for 2.6 -- there's many more references in it. http://docs.python.org/dev > #---------------------------------------------------------- > # Flaw #7 > # > # This is a compatibility issue. > # print is both a statement and a function, or something.. > # > > print() # should print just a newline, but prints two parens > > # workaround: > > print("") print is a statement, and no function. () is an empty tuple, so "print ()" prints an empty tuple. (""), on the other hand, is the same as "". > #---------------------------------------------------------- > # flaw #8 > # > # Having to eval expressions but exec statements feels wrong. > # There should just be eval. > # > > # eval("print(1)") # error > exec("print(1)") # not an error > > exec("1 + 2") # not an error? > eval("1 + 2") # not an error There's a clear distinction between expressions and statements, so it makes sense here too. cheers, Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From skip at pobox.com Sun Jan 13 18:29:14 2008 From: skip at pobox.com (skip at pobox.com) Date: Sun, 13 Jan 2008 11:29:14 -0600 Subject: [Python-Dev] Some tardy mailman checking Message-ID: <18314.19050.553337.493066@montanaro.dyndns.org> The previous four posts were dredged out of the holding pen in Mailman. Sorry for the delay. Skip From guido at python.org Sun Jan 13 18:31:58 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 13 Jan 2008 09:31:58 -0800 Subject: [Python-Dev] complaints.. In-Reply-To: References: Message-ID: It would make more sense to redirect "criticism" out of beginners' ignorance to comp.lang.python rather than spend time discussing their misunderstandings here. On Jan 13, 2008 9:28 AM, Georg Brandl wrote: > Jay schrieb: > > Only addressing the easier points here: > > > #---------------------------------------------------------- > > # flaw #2 > > # > > # In functions, reads are scoped. Writes are not. > > # > > > > A = "1" > > > > def F1(): > > A = "2" # not an error > > > > def F2(): > > #B = A # error > > A = "3" > > > > F1() > > F2() > > print(A) > > "Writes" creates a new local. If you want to modify the enclosing > binding, the new "nonlocal" statement in 2.6/3.0 will enable that. > > > #---------------------------------------------------------- > > # flaw #3: > > # lambda is neutered > > # It allows just one expression, and no statements > > Lambda can't be altered to allow statements *and* have a consistent > indentation-based syntax. > > > #---------------------------------------------------------- > > # flaw #5 > > # > > # for loops suck > > # > > # It should be easy to iterate from 0 to n, not 0 to n - 1, > > # thereby knowing why the loop terminated > > # > > > > #This should work: > > > > # print the first even number, if there are any > > > > A = [1, 3] > > for i in range(0, len(A)): > > if ((A[i] % 2) == 0): > > print("even number found") > > break; > > if (i == len(A)): > > print("no even numbers found") > > The for loop has an else clause. > > for x in A: > if x % 2 == 0: > print "even number found" > break > else: > print "no even numbers found" > > > Flaw #6 > > > > The docs are very good. > > > > However the reference doesn't give much in the way > > of semantic description. > > > > It is surprising that an experienced programmer MUST > > depend on the tutorial or any examples or semantics. > > Light on examples, ok for reference. > > > > The language reference is little more than the grammar in parts. > > > > There needs to be links from the reference back to the tutorial. > > > > Perhaps merge the indices for Tutorial, Language Reference, Library > > Reference. > > Or maybe that's what search is for. > > Look at the new docs for 2.6 -- there's many more references in it. > http://docs.python.org/dev > > > #---------------------------------------------------------- > > # Flaw #7 > > # > > # This is a compatibility issue. > > # print is both a statement and a function, or something.. > > # > > > > print() # should print just a newline, but prints two parens > > > > # workaround: > > > > print("") > > print is a statement, and no function. () is an empty tuple, so > "print ()" prints an empty tuple. > (""), on the other hand, is the same as "". > > > #---------------------------------------------------------- > > # flaw #8 > > # > > # Having to eval expressions but exec statements feels wrong. > > # There should just be eval. > > # > > > > # eval("print(1)") # error > > exec("print(1)") # not an error > > > > exec("1 + 2") # not an error? > > eval("1 + 2") # not an error > > There's a clear distinction between expressions and statements, > so it makes sense here too. > > cheers, > Georg > > -- > Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. > Four shall be the number of spaces thou shalt indent, and the number of thy > indenting shall be four. Eight shalt thou not indent, nor either indent thou > two, excepting that thou then proceed to four. Tabs are right out. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg at krypto.org Sun Jan 13 20:05:54 2008 From: greg at krypto.org (Gregory P. Smith) Date: Sun, 13 Jan 2008 11:05:54 -0800 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <4789DDA0.4020900@cheimes.de> References: <47894171.7030004@cheimes.de> <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> <4789DDA0.4020900@cheimes.de> Message-ID: <52dc1c820801131105l86c192dqe445f05ba518e121@mail.gmail.com> On 1/13/08, Christian Heimes wrote: > > Gregory P. Smith wrote: > > My main suggestion was going to be the ability to turn it off as you > already > > mentioned. However, please consider leaving it off by default to avoid > > problems for installed python scripts importing user supplied code. For > > shared hosting environments where this becomes really useful users can > > easily add the -s (or whatever flag is chosen) to their programs > > themselves. I don't know what that'd mean on windows where #! lines > don't > > exist. Yet another file extension to imply the flag (yuck)? A .cmd > wrapper > > script to run python with the flag (ugh)? > > So you prefer to make the per use site-package directory an opt-in > option? I prefer it as an opt-out option. It's enabled by default, > unless the user disables the feature with -s. > > I'm not sure how to solve the problem on Windows. IMHO the feature > should be enabled on Windows at least but I like to keep it enabled on > all systems. The PEP doesn't add a new attack vector. The problem also > exist with PYTHONPATH. Paranoid programs should start with -E -s anyway > and paranoid system administrators can switch a flag in site.py: Good point, leave it on by default. # Enable per user site-packages directory > # set it to False to disable the feature or True to force the feature > ENABLE_USER_SITE = None > > If we disable the feature by default it won't be available for a lot of > users. > > > For security reasons we also need it disabled when the getuid() != > geteuid() > > to avoid user supplied code being executed as another user. Defaulting > to > > disabled would mean that security could be left up to the end user to > mess > > up. (many systems do not allow setuid #! scripts but this issue would > still > > apply to things run under sudo) > > It sounds like a reasonable and easy implementable idea, at least on > Unix. Windows doesn't have getuid() and geteuid(). On the other hand > Windows doesn't have the suid bit, too. > > I also tried to check if os.stat(__main__.__file__).st_uid == > os.getuid() but the real __main__ is not available in site.py. It's > loaded and assigned much later. Is sys.argv[0] available at that point? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080113/a6f7f6d8/attachment.htm From lists at cheimes.de Sun Jan 13 20:12:43 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 13 Jan 2008 20:12:43 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <52dc1c820801131105l86c192dqe445f05ba518e121@mail.gmail.com> References: <47894171.7030004@cheimes.de> <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> <4789DDA0.4020900@cheimes.de> <52dc1c820801131105l86c192dqe445f05ba518e121@mail.gmail.com> Message-ID: <478A62AB.8010409@cheimes.de> Gregory P. Smith wrote: >> I also tried to check if os.stat(__main__.__file__).st_uid == >> os.getuid() but the real __main__ is not available in site.py. It's >> loaded and assigned much later. > > Is sys.argv[0] available at that point? No, it's not available, too. The 'site' module is imported by Py_Initialize(). sys.argv and the real __main__ module are set much later in Modules/main.c. Christian From jflatow at northwestern.edu Sun Jan 13 20:13:57 2008 From: jflatow at northwestern.edu (Jared Flatow) Date: Sun, 13 Jan 2008 13:13:57 -0600 Subject: [Python-Dev] PySequence_Concat for dicts In-Reply-To: <004901c85573$67c9af50$6800a8c0@RaymondLaptop1> References: <20080111202116.AEX21061@ms19.lnh.mail.rcn.net> <25D06FB4-2906-4531-A4D4-BF1B1396F9FA@northwestern.edu> <004901c85573$67c9af50$6800a8c0@RaymondLaptop1> Message-ID: <2BA83E6A-D871-472B-8E3F-491C14A2AF75@northwestern.edu> On Jan 12, 2008, at 5:32 PM, Raymond Hettinger wrote: > Not natural, just inefficient and cute. Also, there was no answer > to the question about use cases. Fair enough. I will present some use cases below. > AFAICT, this feature has never been requested. The closest was a > feature request for a > variant of update() that avoided overwrites when a duplicate > key was encountered -- Guido rejected that one a long time ago. What about the patch I initially presented (and which you originally dealt with)? http://mail.python.org/pipermail/patches/2004-March/014323.html It seems the original request just never discussed the issue of duplicate keys (for some odd reason). > Your previous note suggests that there are alternative interpretations > of what the syntax could mean and that's not good a good thing. > That sort of ambiguity damages the language. It is not even > clear where the appropriate operators would be +-* or the > set operators &|^-. How about we keep sets for set operations and > dict for mapping operations and not foolishly conflate the two > just because we can. The mapping API is central to the language. > Altering it should be approached with a great deal of care. It was foolish of me to make those comments, you're right, and I should have known better. Guido has made it clear that the correct interpretation is that the keys of a dict form a set, which gets rid of any ambiguity. The set operators are most appropriate, though I am not exactly clear on whether this is already going to be implemented in a future version of Python, or if its just that noone will object if it appears in a future version. If it is the latter, I would still like to take a stab at implementing this as a first contribution. Would you please advise? > Also, the argument that we used + for lists so now we have > to do it for dicts is a weak one -- they are completely different > animals. > Operators are not the solution to all problems. In this case, we > don't even have a problem to be solved; there is just an urge > to hypergeneralize what was done for other datatypes where > it was appropriate. The .update() method we have now is explicit, > clear about its intent, and efficient. I agree operators are not the solution to all problems (need they be the solution to any?). My argument about + for lists was merely based on the precedent it established for sometimes sacrificing efficiency for clarity. Sometimes you may not want to alter the original lists (just as sometimes you may not want to alter the original dicts), but even when it does not matter if you do, you might still write: def prepend_with_a_b(list_a): return ['a', 'b'] + list_a instead of def prepend_with_a_b(list_a): list_b = ['a', 'b'] list_b.extend(list_a) return list_b Even though I suspect the latter will often be more efficient. The | operation for dict will not do anything that you could not otherwise do with update. I suspect most usage will be to simplify code as above. As for use cases when you actually want a new dict, I am guessing you do not want to know specifically why I don't want to alter dicts, but a more general use case, in which event the most generalized example is any case where you simply do not want to modify the original dicts. Since it seems that you might not actually need convincing that having the 4 set operations supported would be a reasonable thing to do, I will stop here for now. > IMO, the only thing this proposal has going for it is that it is cute. I suppose I should be glad for that? I might have thought you to put that to its discredit. Anyhow, I am not sure if we are now in agreement or not, but if so would you please advise on how to proceed? jared From jyasskin at gmail.com Mon Jan 14 00:12:56 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Sun, 13 Jan 2008 15:12:56 -0800 Subject: [Python-Dev] Rounding Decimals In-Reply-To: References: <5d44f72f0801051857i3b13f1b0q8a9e485831dcd603@mail.gmail.com> <003101c85011$e1572530$6800a8c0@RaymondLaptop1> <5d44f72f0801060121l39c4d2a0jfdea69f644bc1822@mail.gmail.com> <5d44f72f0801061526x788b0a46lbbc17ec5dde44ceb@mail.gmail.com> <007601c850ca$7f2ab3e0$6800a8c0@RaymondLaptop1> <5d44f72f0801121709x7c25efbcvfa13e29864b5b577@mail.gmail.com> Message-ID: <5d44f72f0801131512r7273c014ta4cf2583b8f73f6f@mail.gmail.com> On Jan 12, 2008 8:21 PM, Guido van Rossum wrote: > > On Jan 12, 2008 5:09 PM, Jeffrey Yasskin wrote: > > During the discussion about the new Rational implementation > > (http://bugs.python.org/issue1682), Guido and Raymond decided that > > Decimal should not implement the new Real ABC from PEP 3141. So I've > > closed http://bugs.python.org/issue1623 and won't be pursuing any of > > the extra rounding methods mentioned on this thread. > > Well, I didn't really decide anything. I suggested that if the > developers of Decimal preferred it, it might be better if Decimal did > not implement the Real ABC, and Raymond said he indeed preferred it. > > Since this reduces the usefulness of numeric.Real, I'm somewhat > disappointed in this outcome (as I imagine you are). However, Decimal > has very little of my own sweat in it, and a lot of Raymond, Facundo > (and others, including Mark), and the ABC thing is still somewhat > experimental, so it wouldn't make sense for me to impose my ideas onto > Decimal unilaterally, and I'm sure Raymond etc. have their reasons. Sorry for misrepresenting you. I am a little disappointed, but if we're right that numbers.Real is a good idea, then users will ask for Decimal to implement it, and the Decimal developers can change their minds in 3.1. Since I'm not one of those users, I'm not the right person to argue for this anyway. :) -- Namast?, Jeffrey Yasskin From lists at cheimes.de Mon Jan 14 00:14:53 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 14 Jan 2008 00:14:53 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: References: Message-ID: <478A9B6D.5070400@cheimes.de> I've uploaded a new patch: http://bugs.python.org/issue1799 Christian From adlaiff6 at gmail.com Mon Jan 14 00:41:50 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Sun, 13 Jan 2008 18:41:50 -0500 Subject: [Python-Dev] Rounding Decimals In-Reply-To: <5d44f72f0801131512r7273c014ta4cf2583b8f73f6f@mail.gmail.com> References: <5d44f72f0801051857i3b13f1b0q8a9e485831dcd603@mail.gmail.com> <003101c85011$e1572530$6800a8c0@RaymondLaptop1> <5d44f72f0801060121l39c4d2a0jfdea69f644bc1822@mail.gmail.com> <5d44f72f0801061526x788b0a46lbbc17ec5dde44ceb@mail.gmail.com> <007601c850ca$7f2ab3e0$6800a8c0@RaymondLaptop1> <5d44f72f0801121709x7c25efbcvfa13e29864b5b577@mail.gmail.com> <5d44f72f0801131512r7273c014ta4cf2583b8f73f6f@mail.gmail.com> Message-ID: <11b6d2f60801131541t5c0853f1l39da084a733c03a@mail.gmail.com> On Jan 13, 2008 6:12 PM, Jeffrey Yasskin wrote: > On Jan 12, 2008 8:21 PM, Guido van Rossum wrote: > > > > On Jan 12, 2008 5:09 PM, Jeffrey Yasskin wrote: > > > During the discussion about the new Rational implementation > > > (http://bugs.python.org/issue1682), Guido and Raymond decided that > > > Decimal should not implement the new Real ABC from PEP 3141. So I've > > > closed http://bugs.python.org/issue1623 and won't be pursuing any of > > > the extra rounding methods mentioned on this thread. > > > > Well, I didn't really decide anything. I suggested that if the > > developers of Decimal preferred it, it might be better if Decimal did > > not implement the Real ABC, and Raymond said he indeed preferred it. > > > > Since this reduces the usefulness of numeric.Real, I'm somewhat > > disappointed in this outcome (as I imagine you are). However, Decimal > > has very little of my own sweat in it, and a lot of Raymond, Facundo > > (and others, including Mark), and the ABC thing is still somewhat > > experimental, so it wouldn't make sense for me to impose my ideas onto > > Decimal unilaterally, and I'm sure Raymond etc. have their reasons. > > Sorry for misrepresenting you. I am a little disappointed, but if > we're right that numbers.Real is a good idea, then users will ask for > Decimal to implement it, and the Decimal developers can change their > minds in 3.1. Since I'm not one of those users, I'm not the right > person to argue for this anyway. :) I haven't been watching the python-dev list for very long, so maybe this has already been discussed ad nauseam (in which case, sorry), but, from the devil's advocate-ish mathematics side of things, unless numbers.Decimal is planned to be implemented with infinite precision (which I doubt, but could not confirm with an admittedly small amount of research), shouldn't it implement numbers.Rational instead? Was this ever discussed, or was it just assumed that numbers.Real was the right decision? Cheers, Leif From jyasskin at gmail.com Mon Jan 14 01:26:34 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Sun, 13 Jan 2008 16:26:34 -0800 Subject: [Python-Dev] Rounding Decimals In-Reply-To: <11b6d2f60801131541t5c0853f1l39da084a733c03a@mail.gmail.com> References: <5d44f72f0801051857i3b13f1b0q8a9e485831dcd603@mail.gmail.com> <003101c85011$e1572530$6800a8c0@RaymondLaptop1> <5d44f72f0801060121l39c4d2a0jfdea69f644bc1822@mail.gmail.com> <5d44f72f0801061526x788b0a46lbbc17ec5dde44ceb@mail.gmail.com> <007601c850ca$7f2ab3e0$6800a8c0@RaymondLaptop1> <5d44f72f0801121709x7c25efbcvfa13e29864b5b577@mail.gmail.com> <5d44f72f0801131512r7273c014ta4cf2583b8f73f6f@mail.gmail.com> <11b6d2f60801131541t5c0853f1l39da084a733c03a@mail.gmail.com> Message-ID: <5d44f72f0801131626w780aac5axd505ce1335961377@mail.gmail.com> On Jan 13, 2008 3:41 PM, Leif Walsh wrote: > I haven't been watching the python-dev list for very long, so maybe > this has already been discussed ad nauseam (in which case, sorry), > but, from the devil's advocate-ish mathematics side of things, unless > numbers.Decimal is planned to be implemented with infinite precision > (which I doubt, but could not confirm with an admittedly small amount > of research), shouldn't it implement numbers.Rational instead? Was > this ever discussed, or was it just assumed that numbers.Real was the > right decision? Guido mentioned the possibility briefly at http://mail.python.org/pipermail/python-3000/2007-April/007015.html ("One could argue that float and Decimal are <:Q, but I'm not sure if that makes things better pragmatically") but never really discussed as far as I can find (although I did think about it :). I prefer having float not implement Rational (Decimal's off the table) because developers usually think of them as approximations to arbitrary, possibly-irrational, real numbers, rather than as rational numbers with some odd constraints on their denominators. That is, to me, the ABCs a type implements are more about how developers should think about the type than the implementation of the type. [ A new thread is probably appropriate if anyone wants to discuss the philosophy, but I probably won't participate... ] -- Namast?, Jeffrey Yasskin From adlaiff6 at gmail.com Mon Jan 14 01:37:46 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Sun, 13 Jan 2008 19:37:46 -0500 Subject: [Python-Dev] Rounding Decimals In-Reply-To: <5d44f72f0801131626w780aac5axd505ce1335961377@mail.gmail.com> References: <5d44f72f0801051857i3b13f1b0q8a9e485831dcd603@mail.gmail.com> <003101c85011$e1572530$6800a8c0@RaymondLaptop1> <5d44f72f0801060121l39c4d2a0jfdea69f644bc1822@mail.gmail.com> <5d44f72f0801061526x788b0a46lbbc17ec5dde44ceb@mail.gmail.com> <007601c850ca$7f2ab3e0$6800a8c0@RaymondLaptop1> <5d44f72f0801121709x7c25efbcvfa13e29864b5b577@mail.gmail.com> <5d44f72f0801131512r7273c014ta4cf2583b8f73f6f@mail.gmail.com> <11b6d2f60801131541t5c0853f1l39da084a733c03a@mail.gmail.com> <5d44f72f0801131626w780aac5axd505ce1335961377@mail.gmail.com> Message-ID: <11b6d2f60801131637n50ed3b31s5fe151c66b5f23b6@mail.gmail.com> On Jan 13, 2008 7:26 PM, Jeffrey Yasskin wrote: > Guido mentioned the possibility briefly at > http://mail.python.org/pipermail/python-3000/2007-April/007015.html > ("One could argue that float and Decimal are <:Q, but I'm not sure if > that makes things better pragmatically") but never really discussed as > far as I can find (although I did think about it :). Well, if nothing else, I'm certainly satisfied that my concerns about accuracy are being addressed. I will have to go read more of that discussion. > I prefer having > float not implement Rational (Decimal's off the table) because > developers usually think of them as approximations to arbitrary, > possibly-irrational, real numbers, rather than as rational numbers > with some odd constraints on their denominators. That is, to me, the > ABCs a type implements are more about how developers should think > about the type than the implementation of the type. I see where you're coming from. My personal stance is that developers _shouldn't_ think that way, but I know that's an unpopular opinion, and isn't really worth discussion here. > [ A new thread is probably appropriate if anyone wants to discuss the > philosophy, but I probably won't participate... ] I don't feel any pressing need to continue this here and now. Thanks! -- Cheers, Leif From python at rcn.com Mon Jan 14 04:53:57 2008 From: python at rcn.com (Raymond Hettinger) Date: Sun, 13 Jan 2008 19:53:57 -0800 Subject: [Python-Dev] r59947 - in python/trunk:Lib/test/test_structseq.py Misc/NEWS References: <20080114033353.2B9241E401A@bag.python.org> Message-ID: <001201c85661$181a4390$6800a8c0@RaymondLaptop1> [Christian Heimes] > Log: > Added new an better structseq representation. E.g. repr(time.gmtime(0)) now returns 'time.struct_time(tm_year=1970, tm_mon=1, > tm_mday=1, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=3, tm_yday=1, tm_isdst=0)' instead of '(1970, 1, 1, 0, 0, 0, 3, 1, 0)'. The > feature is part of #1816: sys.flags FWIW, I was looking into something similar but didn't proceed because it would break eval(repr(s)) == s as the constructor signature wants all the args in a tuple and won't accept keywords. Still, I think what you did is a nice improvement. Here's a couple more if you want to proceed down that path: 1. Have structseq subclass from PyTupleObject so that isinstance(s, tuple) returns True. This makes the object usable whenever tuples are needed. 2. Add _fields, _asdict, and _replace to match the API in collections.namedtuple(). The _fields tuple should only include the visible positional fields while _asdict() and _replace() should include all of the fields whether visible or accessible only by attribute access. Thanks for the nice work. Raymond From guido at python.org Mon Jan 14 05:15:57 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 13 Jan 2008 20:15:57 -0800 Subject: [Python-Dev] r59947 - in python/trunk:Lib/test/test_structseq.py Misc/NEWS In-Reply-To: <001201c85661$181a4390$6800a8c0@RaymondLaptop1> References: <20080114033353.2B9241E401A@bag.python.org> <001201c85661$181a4390$6800a8c0@RaymondLaptop1> Message-ID: On Jan 13, 2008 7:53 PM, Raymond Hettinger wrote: > [Christian Heimes] > > Log: > > Added new an better structseq representation. E.g. repr(time.gmtime(0)) now returns 'time.struct_time(tm_year=1970, tm_mon=1, > > tm_mday=1, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=3, tm_yday=1, tm_isdst=0)' instead of '(1970, 1, 1, 0, 0, 0, 3, 1, 0)'. The > > feature is part of #1816: sys.flags > > FWIW, I was looking into something similar but didn't proceed because it would break eval(repr(s)) == s as the constructor signature > wants all the args in a tuple and won't accept keywords. Still, I think what you did is a nice improvement. > > Here's a couple more if you want to proceed down that path: > > 1. Have structseq subclass from PyTupleObject so that isinstance(s, tuple) returns True. This makes the object usable whenever > tuples are needed. Hmm, is that really necessary? structseq has been in use for quite a while and this need hasn't come up -- it's been designed to be quite compatible with tuple *except* for isinstance checks, and that has worked well. > 2. Add _fields, _asdict, and _replace to match the API in collections.namedtuple(). The _fields tuple should only include the > visible positional fields while _asdict() and _replace() should include all of the fields whether visible or accessible only by > attribute access. > > Thanks for the nice work. > > > Raymond > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From lists at cheimes.de Mon Jan 14 05:25:44 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 14 Jan 2008 05:25:44 +0100 Subject: [Python-Dev] r59947 - in python/trunk:Lib/test/test_structseq.py Misc/NEWS In-Reply-To: <001201c85661$181a4390$6800a8c0@RaymondLaptop1> References: <20080114033353.2B9241E401A@bag.python.org> <001201c85661$181a4390$6800a8c0@RaymondLaptop1> Message-ID: <478AE448.10704@cheimes.de> Raymond Hettinger wrote: > FWIW, I was looking into something similar but didn't proceed because it would break eval(repr(s)) == s as the constructor signature > wants all the args in a tuple and won't accept keywords. Still, I think what you did is a nice improvement. I agree that eval(repr(s)) == s is nice to have but most complex object don't support it. In this very case the readability of the repr() is more important than eval(repr(s)). I'm sure lots of users still think that os.stat or time.time() return an ordinary tuple. I've turned your request into an issue: http://bugs.python.org/issue1820 It's a feasible yet challenging task for a new CPython coder. Christian From python at rcn.com Mon Jan 14 05:26:45 2008 From: python at rcn.com (Raymond Hettinger) Date: Sun, 13 Jan 2008 20:26:45 -0800 Subject: [Python-Dev] r59947 - in python/trunk:Lib/test/test_structseq.py Misc/NEWS References: <20080114033353.2B9241E401A@bag.python.org> <001201c85661$181a4390$6800a8c0@RaymondLaptop1> Message-ID: <000a01c85665$ad028bd0$6800a8c0@RaymondLaptop1> >> 1. Have structseq subclass from PyTupleObject so that isinstance(s, tuple) returns True. This makes the object usable whenever >> tuples are needed. > > Hmm, is that really necessary? structseq has been in use for quite a > while and this need hasn't come up -- it's been designed to be quite > compatible with tuple *except* for isinstance checks, and that has > worked well. It looks like that is the only difference, so subclassing from tuple won't cause any other behavioral changes. It looks like making it a subtype involves only changing a few lines. I think it can only help, especially if we start to use structseq for more things in the future. Also, I would like the API to match collections.namedtuple() as closely as possible so that there is just a single useful concept that gets applied whereever needed. The idea is basically that they *are* tuples with the added nicity of attribute access and a self-documenting repr. Raymond From martin at v.loewis.de Mon Jan 14 08:22:09 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 14 Jan 2008 08:22:09 +0100 Subject: [Python-Dev] [Python-checkins] r59947 - in python/trunk:Lib/test/test_structseq.py Misc/NEWS In-Reply-To: References: <20080114033353.2B9241E401A@bag.python.org> <001201c85661$181a4390$6800a8c0@RaymondLaptop1> Message-ID: <478B0DA1.3030405@v.loewis.de> > Hmm, is that really necessary? structseq has been in use for quite a > while and this need hasn't come up -- it's been designed to be quite > compatible with tuple *except* for isinstance checks, and that has > worked well. In addition, I would like to suggest that the current structseq usage is changed to regular classes (perhaps with slots) in Python 3. structseq was a compatibility mechanism for existing code that assumes the things returned are tuples, when they are really objects with named fields. So for struct stat and struct tm in particular, I think the indexed access should be removed in Python 3. Whether then structseq needs to be preserved at all, I don't know. Regards, Martin From asmodai at in-nomine.org Mon Jan 14 10:20:05 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Mon, 14 Jan 2008 10:20:05 +0100 Subject: [Python-Dev] docs.python.org Message-ID: <20080114092005.GR75977@nexus.in-nomine.org> Will docs.python.org be redirected to www.python.org/doc? I think I am not the only person on the planet that had been using docs.python.org for a long time now to get to the documentation. (And some links are now broken @ docs.python.org, so it seems all the more logical to make such a redirect. docs.python.org is the 3rd result from Google too by the way.) -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ Want ik kan niet leven zonder alles of niets... From walter at livinglogic.de Mon Jan 14 13:08:20 2008 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Mon, 14 Jan 2008 13:08:20 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: References: Message-ID: <478B50B4.3040809@livinglogic.de> Christian Heimes wrote: > [...] > PEP: XXX > Title: Per user site-packages directory > Version: $Revision$ > Last-Modified: $Date$ > Author: Christian Heimes > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 11-Jan-2008 > Python-Version: 2.6, 3.0 > Post-History: > [...] > user site directory > > A site directory inside the users' home directory. An user site > directory is specific to a Python version. The path contains > the version number (major and minor only). > > Windows: %APPDATA%/Python/Python26/site-packages > Mac: ~/Library/Python/2.6/site-packages > Unix: ~/.local/lib/python2.6/site-packages > > > user configuration directory > > Usually the parent directory of the user site directory. It's meant > for Python version specific data like config files. > > Windows: %APPDATA%/Python/Python26 > Mac: ~/Library/Python/2.6 > Unix: ~/.local/lib/python2.6 So if I'm using the --user option, where would scripts be installed? Would this be: Windows: %APPDATA%/Python/Python26/bin Mac: ~/Library/Python/2.6/bin Unix: ~/.local/lib/python2.6/bin I'd like to be able to switch between several versions of my user installation simply by changing a link. (On the Mac I'm doing this by relinking ~/Library/Python to different directories.) Servus, Walter From amk at amk.ca Mon Jan 14 14:28:09 2008 From: amk at amk.ca (A.M. Kuchling) Date: Mon, 14 Jan 2008 08:28:09 -0500 Subject: [Python-Dev] docs.python.org In-Reply-To: <20080114092005.GR75977@nexus.in-nomine.org> References: <20080114092005.GR75977@nexus.in-nomine.org> Message-ID: <20080114132809.GA5808@amk-desktop.matrixgroup.net> On Mon, Jan 14, 2008 at 10:20:05AM +0100, Jeroen Ruigrok van der Werven wrote: > Will docs.python.org be redirected to www.python.org/doc? > > I think I am not the only person on the planet that had been using > docs.python.org for a long time now to get to the documentation. (And some > links are now broken @ docs.python.org, so it seems all the more logical to > make such a redirect. docs.python.org is the 3rd result from Google too by the I doubt it; instead I think that, once Python 2.6 is released, the new documentation system that's now at http://docs.python.org/dev/ will move up to become the top page of docs.python.org, and /dev/ will remain the trunk version, updated daily. Various links on the top page of docs.python.org have become outdated, though; I've gone through the page and updated or removed them. --amk From jimjjewett at gmail.com Mon Jan 14 14:48:06 2008 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 14 Jan 2008 08:48:06 -0500 Subject: [Python-Dev] [Python-3000] Rounding Decimals In-Reply-To: References: <5d44f72f0801051857i3b13f1b0q8a9e485831dcd603@mail.gmail.com> <003101c85011$e1572530$6800a8c0@RaymondLaptop1> <5d44f72f0801060121l39c4d2a0jfdea69f644bc1822@mail.gmail.com> <5d44f72f0801061526x788b0a46lbbc17ec5dde44ceb@mail.gmail.com> <007601c850ca$7f2ab3e0$6800a8c0@RaymondLaptop1> <5d44f72f0801121709x7c25efbcvfa13e29864b5b577@mail.gmail.com> Message-ID: On 1/12/08, Guido van Rossum wrote: > On Jan 12, 2008 5:09 PM, Jeffrey Yasskin wrote: > > During the discussion about the new Rational implementation > > (http://bugs.python.org/issue1682), Guido and Raymond decided that > > Decimal should not implement the new Real ABC from PEP 3141. So I've > > closed http://bugs.python.org/issue1623 and won't be pursuing any of > > the extra rounding methods mentioned on this thread. > Well, I didn't really decide anything. I suggested that if the > developers of Decimal preferred it, it might be better if Decimal did > not implement the Real ABC, and Raymond said he indeed preferred it. I read his objections slightly differently. He is very clear that Decimal itself should be restricted to the standard, and therefore should not natively implement the extensions. But he also said that it might be reasonable for another package to subset or superset it in a friendlier way. numbers.py is a different module, which must be explicitly imported. If the objection is that >>> decimal.Decimal("43.2").imag would work (instead of throwing an exception) only when numbers.py has already been imported, then ... well, that confusion is inherent in the abstract classes. Or is the problem that it *still* wouldn't work, without help from the decimal module itself? In that case, 3rd party registration is fairly limited, and this might be a good candidate for trying to figure out ABCs and adapters *should* work together. -jJ From asmodai at in-nomine.org Mon Jan 14 15:15:33 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Mon, 14 Jan 2008 15:15:33 +0100 Subject: [Python-Dev] docs.python.org In-Reply-To: <20080114132809.GA5808@amk-desktop.matrixgroup.net> References: <20080114092005.GR75977@nexus.in-nomine.org> <20080114132809.GA5808@amk-desktop.matrixgroup.net> Message-ID: <20080114141533.GZ75977@nexus.in-nomine.org> -On [20080114 14:54], A.M. Kuchling (amk at amk.ca) wrote: >I doubt it; instead I think that, once Python 2.6 is released, the new >documentation system that's now at http://docs.python.org/dev/ will >move up to become the top page of docs.python.org, and /dev/ will remain >the trunk version, updated daily. The thing is: http://www.python.org/ has a link Documentation, which links to http://www.python.org/doc/ and from there has a link with Browse Current Documentation to http://docs.python.org/ - it's kind of confusing with the Quick Links content at the bottom. You have two choices in the left-hand side menu that both read Documentation. That's ambiguous. Add to that that the stylesheet between www.python.org and docs.python.org are different and the menu structure is also different and it makes for a bad user experience. And to make matters even more confusing docs.python.org/dev/ has its own stylesheet as well. Great work, all in all, but this is bad style (pun intended). I wonder if a mix between www.python.org's stylesheet and docs.python.org/dev/'s can be made. The colours of www.python.org's headers and the links are too similar in colour, plus the dotted line beneath the URL disappears against the white background. In that aspect docs.python.org/dev/ is a step up. >Various links on the top page of docs.python.org have become outdated, >though; I've gone through the page and updated or removed them. Thanks, much appreciated. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ What is the short meaning of this long speech..? From glyph at divmod.com Mon Jan 14 15:15:39 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Mon, 14 Jan 2008 14:15:39 -0000 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <478B50B4.3040809@livinglogic.de> References: <478B50B4.3040809@livinglogic.de> Message-ID: <20080114141539.21558.782173562.divmod.xquotient.4580@joule.divmod.com> On 12:08 pm, walter at livinglogic.de wrote: >So if I'm using the --user option, where would scripts be installed? >Would this be: > >Windows: %APPDATA%/Python/Python26/bin >Mac: ~/Library/Python/2.6/bin >Unix: ~/.local/lib/python2.6/bin > >I'd like to be able to switch between several versions of my user >installation simply by changing a link. (On the Mac I'm doing this by >relinking ~/Library/Python to different directories.) I think the relevant link to change here would be ~/.local. I have personally been using the ~/.local convention for a while, and I believe ~/.local/bin is where scripts should go. Python is not the only thing that can be locally installed, and the fact that it's ~/.local/lib/python2.6/site-packages suggests that ~/.local has the same layout as /usr (or /usr/local, for those who use that convention). From guido at python.org Mon Jan 14 16:37:45 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 14 Jan 2008 07:37:45 -0800 Subject: [Python-Dev] [Python-checkins] r59947 - in python/trunk:Lib/test/test_structseq.py Misc/NEWS In-Reply-To: <478B0DA1.3030405@v.loewis.de> References: <20080114033353.2B9241E401A@bag.python.org> <001201c85661$181a4390$6800a8c0@RaymondLaptop1> <478B0DA1.3030405@v.loewis.de> Message-ID: On Jan 13, 2008 11:22 PM, "Martin v. L?wis" wrote: > > Hmm, is that really necessary? structseq has been in use for quite a > > while and this need hasn't come up -- it's been designed to be quite > > compatible with tuple *except* for isinstance checks, and that has > > worked well. > > In addition, I would like to suggest that the current structseq usage > is changed to regular classes (perhaps with slots) in Python 3. > structseq was a compatibility mechanism for existing code that assumes > the things returned are tuples, when they are really objects with named > fields. > > So for struct stat and struct tm in particular, I think the indexed > access should be removed in Python 3. Whether then structseq needs > to be preserved at all, I don't know. There seems to be a use case for it still, otherwise collections.namedtuple wouldn't be added to 2.6, right? I agree that it's time to clean up the tuple-ness of the stat and tm structures (especially since they both may have "hidden" fields that are not accessible via the tuple structure). And I see no need for the new sys.flags object to be a tuple at all. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From arigo at tunes.org Mon Jan 14 18:59:43 2008 From: arigo at tunes.org (Armin Rigo) Date: Mon, 14 Jan 2008 18:59:43 +0100 Subject: [Python-Dev] [Python-3000] inst_persistent_id In-Reply-To: References: Message-ID: <20080114175943.GA26431@code0.codespeak.net> Hi, On Sat, Jan 12, 2008 at 07:33:38PM -0500, Alexandre Vassalotti wrote: > Well, in Python 3K, inst_persistent_id() won't be usable, since > PyInstance_Type was removed. Looking at the code, inst_persistent_id() is just a confusing name. It has got nothing to do with PyInstance_Type; it's called for any object type that cPickle.c doesn't know how to handle. In fact, it seems that cPickle.c never calls inst_persistent_id() for objects of type PyInstance_Type... A bientot, Armin. From gnewsg at gmail.com Mon Jan 14 19:00:43 2008 From: gnewsg at gmail.com (Giampaolo Rodola') Date: Mon, 14 Jan 2008 10:00:43 -0800 (PST) Subject: [Python-Dev] test module availability on different Python implementations Message-ID: <397297f9-1bb0-48d1-8912-4473519860af@q77g2000hsh.googlegroups.com> Hi, Today I received a report from a guy who tried to run the test suite of a module of mine. The test suite uses the test module for running tests, for temporary files support and for doing a bunch of other things. He tested it on CentOS 5 and looks like python 2.5.1 includes the test module but python 2.4.3 does not: [root at dell test]# python2.4 test_ftpd.py Traceback (most recent call last): File "test_ftpd.py", line 37, in ? from test import test_support ImportError: No module named test Another guy reported the same thing by having tried to run the test suite against a Windows mobile OS equipped with PythonCE 2.4 where the test module seems to be not available too. I was wondering if there was someone aware of this problem. From lists at cheimes.de Mon Jan 14 19:37:13 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 14 Jan 2008 19:37:13 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <20080114141539.21558.782173562.divmod.xquotient.4580@joule.divmod.com> References: <478B50B4.3040809@livinglogic.de> <20080114141539.21558.782173562.divmod.xquotient.4580@joule.divmod.com> Message-ID: <478BABD9.6070206@cheimes.de> glyph at divmod.com wrote: > I think the relevant link to change here would be ~/.local. > > I have personally been using the ~/.local convention for a while, and I > believe ~/.local/bin is where scripts should go. Python is not the only > thing that can be locally installed, and the fact that it's > ~/.local/lib/python2.6/site-packages suggests that ~/.local has the > same layout as /usr (or /usr/local, for those who use that convention). ~/.local/bin or ~/bin ? ~/bin is the standard directory for user binaries. I can see how ~/.local/bin follows the idea of other directories. But ~/bin is more convenient than ~/.local/bin. I don't want to teach users how to change their .bashrc or .profile file for ~/.local/bin. A line like if [ -d ~/bin ]; then PATH=~/bin:$PATH fi is in most .profile files (often commented out). I'm proposing ~/bin for Unix (including Mac) and %APPDATA%/Python/Scripts for Windows. The Windows installer could then add the directory to the PATH environment and install a minimal bat file "@C:\Python26\python.exe %*" as python26.bat, too. Christian From lists at cheimes.de Mon Jan 14 19:37:13 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 14 Jan 2008 19:37:13 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <20080114141539.21558.782173562.divmod.xquotient.4580@joule.divmod.com> References: <478B50B4.3040809@livinglogic.de> <20080114141539.21558.782173562.divmod.xquotient.4580@joule.divmod.com> Message-ID: <478BABD9.6070206@cheimes.de> glyph at divmod.com wrote: > I think the relevant link to change here would be ~/.local. > > I have personally been using the ~/.local convention for a while, and I > believe ~/.local/bin is where scripts should go. Python is not the only > thing that can be locally installed, and the fact that it's > ~/.local/lib/python2.6/site-packages suggests that ~/.local has the > same layout as /usr (or /usr/local, for those who use that convention). ~/.local/bin or ~/bin ? ~/bin is the standard directory for user binaries. I can see how ~/.local/bin follows the idea of other directories. But ~/bin is more convenient than ~/.local/bin. I don't want to teach users how to change their .bashrc or .profile file for ~/.local/bin. A line like if [ -d ~/bin ]; then PATH=~/bin:$PATH fi is in most .profile files (often commented out). I'm proposing ~/bin for Unix (including Mac) and %APPDATA%/Python/Scripts for Windows. The Windows installer could then add the directory to the PATH environment and install a minimal bat file "@C:\Python26\python.exe %*" as python26.bat, too. Christian From guido at python.org Mon Jan 14 19:39:00 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 14 Jan 2008 10:39:00 -0800 Subject: [Python-Dev] [Python-3000] Rounding Decimals In-Reply-To: References: <5d44f72f0801051857i3b13f1b0q8a9e485831dcd603@mail.gmail.com> <003101c85011$e1572530$6800a8c0@RaymondLaptop1> <5d44f72f0801060121l39c4d2a0jfdea69f644bc1822@mail.gmail.com> <5d44f72f0801061526x788b0a46lbbc17ec5dde44ceb@mail.gmail.com> <007601c850ca$7f2ab3e0$6800a8c0@RaymondLaptop1> <5d44f72f0801121709x7c25efbcvfa13e29864b5b577@mail.gmail.com> Message-ID: On Jan 14, 2008 5:48 AM, Jim Jewett wrote: > On 1/12/08, Guido van Rossum wrote: > > On Jan 12, 2008 5:09 PM, Jeffrey Yasskin wrote: > > > During the discussion about the new Rational implementation > > > (http://bugs.python.org/issue1682), Guido and Raymond decided that > > > Decimal should not implement the new Real ABC from PEP 3141. So I've > > > closed http://bugs.python.org/issue1623 and won't be pursuing any of > > > the extra rounding methods mentioned on this thread. > > > Well, I didn't really decide anything. I suggested that if the > > developers of Decimal preferred it, it might be better if Decimal did > > not implement the Real ABC, and Raymond said he indeed preferred it. > > I read his objections slightly differently. > > He is very clear that Decimal itself should be restricted to the > standard, and therefore should not natively implement the extensions. Well, to me that sounds like idolizing the standard. The standard exists so as to ensure that decimal numbers have well-understood semantics that are implemented correctly. I don't think the standard was ever meant to exclude optimal interaction with the rest of a language. I'd like to see if ReXX implements numeric features over and above the standard. > But he also said that it might be reasonable for another package to > subset or superset it in a friendlier way. >From a user's perspective, it might make more sense to move the strictly standard-conforming code into a module "decimal_standard" and make the friendlier-named "decimal" module be that friendly superset. > numbers.py is a different module, which must be explicitly imported. > > If the objection is that > > >>> decimal.Decimal("43.2").imag > > would work (instead of throwing an exception) only when numbers.py has > already been imported, then ... well, that confusion is inherent in > the abstract classes. Fortunately, the confusion is in your mind only, because that's not how ABCs work according to PEP 3119. While as a side effect of importing the numbers module, the Decimal class might become a virtual subclass of numbers.Real, that doesn't mean that implementation features are automatically added. The registration that might take place inside numbers.py *only* affects the outcome of isinstance() and issubclass() checks -- it doesn't magically add missing attributes or methods to the class. > Or is the problem that it *still* wouldn't work, without help from the > decimal module itself? Right. > In that case, 3rd party registration is fairly > limited, and this might be a good candidate for trying to figure out > ABCs and adapters *should* work together. Of course 3rd party registration is limited -- there's no way that you could automatically define the right semantics, since the ABCs don't have a way to express semantics. PEP 3119 is meant as a *first step* towards adopting ABCs as a core feature. The changes added to collections.py and numbers.py are best considered as minimal examples. The main achievement of the PEP is probably the hooks that allow you to override isinstance() and issubclass(), plus the decision (which was discussed at length) to use ABCs (with virtual inheritance), not interfaces, to model similarities between class APIs. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From lists at cheimes.de Mon Jan 14 19:41:55 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 14 Jan 2008 19:41:55 +0100 Subject: [Python-Dev] [Python-checkins] r59947 - in python/trunk:Lib/test/test_structseq.py Misc/NEWS In-Reply-To: References: <20080114033353.2B9241E401A@bag.python.org> <001201c85661$181a4390$6800a8c0@RaymondLaptop1> <478B0DA1.3030405@v.loewis.de> Message-ID: <478BACF3.8010406@cheimes.de> Guido van Rossum wrote: > And I see no need for the new sys.flags object to be a tuple at all. Correct. We don't need item access anymore. However the struct seq should still be slice-able for functions like time.mktime(). Christian From guido at python.org Mon Jan 14 19:46:40 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 14 Jan 2008 10:46:40 -0800 Subject: [Python-Dev] test module availability on different Python implementations In-Reply-To: <397297f9-1bb0-48d1-8912-4473519860af@q77g2000hsh.googlegroups.com> References: <397297f9-1bb0-48d1-8912-4473519860af@q77g2000hsh.googlegroups.com> Message-ID: The test *package* is much older than Python 2.4, but many Python distros exclude it from their default configuration. Perhaps there is a way to add it back; on many Linux-based systems running apt-get or yum with the appropriate arguments is all it takes. Sorry, I can't help you with the appropriate parameters, since every distro does it differently. --Guido On Jan 14, 2008 10:00 AM, Giampaolo Rodola' wrote: > Hi, > Today I received a report from a guy who tried to run the test suite > of a module of mine. > The test suite uses the test module for running tests, for temporary > files support and for doing a bunch of other things. > He tested it on CentOS 5 and looks like python 2.5.1 includes the test > module but python 2.4.3 does not: > > [root at dell test]# python2.4 test_ftpd.py > Traceback (most recent call last): > File "test_ftpd.py", line 37, in ? > from test import test_support > ImportError: No module named test > > Another guy reported the same thing by having tried to run the test > suite against a Windows mobile OS equipped with PythonCE 2.4 where the > test module seems to be not available too. > > I was wondering if there was someone aware of this problem. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Jan 14 19:50:58 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 14 Jan 2008 10:50:58 -0800 Subject: [Python-Dev] [Python-checkins] r59947 - in python/trunk:Lib/test/test_structseq.py Misc/NEWS In-Reply-To: <478BACF3.8010406@cheimes.de> References: <20080114033353.2B9241E401A@bag.python.org> <001201c85661$181a4390$6800a8c0@RaymondLaptop1> <478B0DA1.3030405@v.loewis.de> <478BACF3.8010406@cheimes.de> Message-ID: On Jan 14, 2008 10:41 AM, Christian Heimes wrote: > Guido van Rossum wrote: > > And I see no need for the new sys.flags object to be a tuple at all. > > Correct. We don't need item access anymore. However the struct seq > should still be slice-able for functions like time.mktime(). Oh, that's a good point. I think it actually doesn't make sense for it to be sliceable without behaving like a sequence in all (or most) aspects. I don't think this applies to struct stat though; I can never remember the order of the values there, unlike the first 6 of a time tuple, which have an obvious order. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From gnewsg at gmail.com Mon Jan 14 20:10:55 2008 From: gnewsg at gmail.com (Giampaolo Rodola') Date: Mon, 14 Jan 2008 11:10:55 -0800 (PST) Subject: [Python-Dev] test module availability on different Python implementations In-Reply-To: References: <397297f9-1bb0-48d1-8912-4473519860af@q77g2000hsh.googlegroups.com> Message-ID: <8a25fcc4-846d-49de-bbca-ff03d727f273@f47g2000hsd.googlegroups.com> On 14 Gen, 19:46, "Guido van Rossum" wrote: > The test *package* is much older than Python 2.4, but many Python > distros exclude it from their default configuration. Perhaps there is > a way to add it back; on many Linux-based systems running apt-get or > yum with the appropriate arguments is all it takes. Sorry, I can't > help you with the appropriate parameters, since every distro does it > differently. > > --Guido Good to know, thanks for the information. I didn't know that python-tests were treated as a separate package on certain platforms. As replacement I'll use unittest.main() for running tests and a static string defined by me instead of test.test_support.TESTFN. From glyph at divmod.com Mon Jan 14 20:13:26 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Mon, 14 Jan 2008 19:13:26 -0000 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <478BABD9.6070206@cheimes.de> References: <478B50B4.3040809@livinglogic.de> <20080114141539.21558.782173562.divmod.xquotient.4580@joule.divmod.com> <478BABD9.6070206@cheimes.de> Message-ID: <20080114191326.21558.1896823221.divmod.xquotient.4614@joule.divmod.com> On 06:37 pm, lists at cheimes.de wrote: >glyph at divmod.com wrote: >>I think the relevant link to change here would be ~/.local. >> >>I have personally been using the ~/.local convention for a while, and >>I >>believe ~/.local/bin is where scripts should go. Python is not the >>only >>thing that can be locally installed, and the fact that it's >>~/.local/lib/python2.6/site-packages suggests that ~/.local has the >>same layout as /usr (or /usr/local, for those who use that >>convention). > >~/.local/bin or ~/bin ? > >~/bin is the standard directory for user binaries. "the" standard directory? according to what? commented-out examples in some linux distribution? >I can see how ~/.local/bin follows the idea of other directories. I think consistency is important here. Python is not the only tool that could follow this convention, and it would be nice to have a consistent convention that it would be easy for other tools and environments to adopt. >But ~/bin is more >convenient than ~/.local/bin. I don't want to teach users how to change >their .bashrc or .profile file for ~/.local/bin. A line like as you say, it is often commented out. You have to teach users this anyway, even if sometimes it will work by accident. ~/bin comes from the convention of "./configure --prefix=$HOME", as autoconf suggests. This means users must have visible directories in their home folder named (among other things) "bin", "share", "lib", "src", "sbin", "man", and "include". I find this ugly. I only find it slightly less ugly that Python will now hide its locally-installed library files from me but not its locally-installed executable scripts. "./configure --prefix=$HOME/.local" will instead put things into ~/.local/bin, ~/.local/lib, etc. Now, I am aware that setup.py already has a special "--home" option, and it can be incompatibly changed, but I don't see a reason for this. Note that --home $HOME will currently put files into ~/lib, not ~/.local/lib. (and --home $HOME/local will put scripts into ~/.local/bin, not ~/bin). But, now that I've told you what I think in more detail, unless you like my ideas and have specific questions, I will try to refrain from commenting further, lest I dig my own bike shed here :). From lists at cheimes.de Mon Jan 14 20:46:55 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 14 Jan 2008 20:46:55 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <20080114191326.21558.1896823221.divmod.xquotient.4614@joule.divmod.com> References: <478B50B4.3040809@livinglogic.de> <20080114141539.21558.782173562.divmod.xquotient.4580@joule.divmod.com> <478BABD9.6070206@cheimes.de> <20080114191326.21558.1896823221.divmod.xquotient.4614@joule.divmod.com> Message-ID: <478BBC2F.3090807@cheimes.de> glyph at divmod.com wrote: > "the" standard directory? according to what? commented-out examples in > some linux distribution? Yes ... :/ I should be more carefully when I use the word "standard". > But, now that I've told you what I think in more detail, unless you like > my ideas and have specific questions, I will try to refrain from > commenting further, lest I dig my own bike shed here :). I have no strong opinion on ~/bin or ~/.local/bin. I understand your point of view and I concur with some of your arguments. I'm making this discussion point easy for me. I let the others decide! :) I don't feel like wasting my time on a bike shed discussion unless it's painted in a different color than blue. I like blue ... *g* Christian From barry at python.org Mon Jan 14 21:07:20 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 14 Jan 2008 15:07:20 -0500 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <20080114191326.21558.1896823221.divmod.xquotient.4614@joule.divmod.com> References: <478B50B4.3040809@livinglogic.de> <20080114141539.21558.782173562.divmod.xquotient.4580@joule.divmod.com> <478BABD9.6070206@cheimes.de> <20080114191326.21558.1896823221.divmod.xquotient.4614@joule.divmod.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 14, 2008, at 2:13 PM, glyph at divmod.com wrote: > ~/bin comes from the convention of "./configure --prefix=$HOME", as > autoconf suggests. This means users must have visible directories in > their home folder named (among other things) "bin", "share", "lib", > "src", "sbin", "man", and "include". I find this ugly. I only find > it > slightly less ugly that Python will now hide its locally-installed > library files from me but not its locally-installed executable > scripts. > > "./configure --prefix=$HOME/.local" will instead put things into > ~/.local/bin, ~/.local/lib, etc. > > Now, I am aware that setup.py already has a special "--home" option, > and > it can be incompatibly changed, but I don't see a reason for this. > Note > that --home $HOME will currently put files into ~/lib, not ~/.local/ > lib. > (and --home $HOME/local will put scripts into ~/.local/bin, not ~/ > bin). > > But, now that I've told you what I think in more detail, unless you > like > my ideas and have specific questions, I will try to refrain from > commenting further, lest I dig my own bike shed here :). I feel pretty strongly that ~/bin should *not* be used. It makes sense to me that ~/.local would mirror /usr/local. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBR4vA+XEjvBPtnXfVAQICEgP8CAuFVzZld1769uQKDWj2h4Y7x+besq5o 9bujNYJ6SayNac4u1jWLWmCIdBSuQU6/xNF6+ljpn5Pz4H/yRBl/HK2ibF2ksZwg quv23PHLvMnLju77FNKE5VclVJk3rBKpkpjmS/yXMcyfBwAccIDEJY+QUshtQzql 8mRZ4cEUP0I= =gepD -----END PGP SIGNATURE----- From lists at cheimes.de Mon Jan 14 21:27:30 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 14 Jan 2008 21:27:30 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: References: <478B50B4.3040809@livinglogic.de> <20080114141539.21558.782173562.divmod.xquotient.4580@joule.divmod.com> <478BABD9.6070206@cheimes.de> <20080114191326.21558.1896823221.divmod.xquotient.4614@joule.divmod.com> Message-ID: <478BC5B2.6020908@cheimes.de> Barry Warsaw wrote: > I feel pretty strongly that ~/bin should *not* be used. It makes > sense to me that ~/.local would mirror /usr/local. Two votes for ~/.local/bin and one undecided PEP author ... I'm changing the code to ~/.local/bin and I'm adding a new section to the PEP. Can I just submit the PEP or do I have to follow a procedure before I can add it to the PEP list? Christian From guido at python.org Mon Jan 14 21:38:52 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 14 Jan 2008 12:38:52 -0800 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <478BC5B2.6020908@cheimes.de> References: <478B50B4.3040809@livinglogic.de> <20080114141539.21558.782173562.divmod.xquotient.4580@joule.divmod.com> <478BABD9.6070206@cheimes.de> <20080114191326.21558.1896823221.divmod.xquotient.4614@joule.divmod.com> <478BC5B2.6020908@cheimes.de> Message-ID: On Jan 14, 2008 12:27 PM, Christian Heimes wrote: > Barry Warsaw wrote: > > I feel pretty strongly that ~/bin should *not* be used. It makes > > sense to me that ~/.local would mirror /usr/local. > > Two votes for ~/.local/bin and one undecided PEP author ... I'm changing > the code to ~/.local/bin and I'm adding a new section to the PEP. > > Can I just submit the PEP or do I have to follow a procedure before I > can add it to the PEP list? You can submit the PEP and update PEP 0 (note that each PEP occurs twice!) but you can't mark it "accepted" :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From lists at cheimes.de Mon Jan 14 22:23:00 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 14 Jan 2008 22:23:00 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: References: Message-ID: <478BD2B4.2010706@cheimes.de> The PEP is now available at http://www.python.org/dev/peps/pep-0370/. The reference implementation is in svn, too: svn+ssh://pythondev at svn.python.org/sandbox/trunk/pep370 Christian From martin at v.loewis.de Mon Jan 14 22:41:00 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 14 Jan 2008 22:41:00 +0100 Subject: [Python-Dev] [Python-checkins] r59947 - in python/trunk:Lib/test/test_structseq.py Misc/NEWS In-Reply-To: References: <20080114033353.2B9241E401A@bag.python.org> <001201c85661$181a4390$6800a8c0@RaymondLaptop1> <478B0DA1.3030405@v.loewis.de> Message-ID: <478BD6EC.3080600@v.loewis.de> > There seems to be a use case for it still, otherwise > collections.namedtuple wouldn't be added to 2.6, right? Perhaps; I'm not strongly opposed to keeping structseq. I'll work on making stat_result and struct_time regular fixed-size types, for 3.0. Regards, Martin From mal at egenix.com Mon Jan 14 22:44:56 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 14 Jan 2008 22:44:56 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <478BD2B4.2010706@cheimes.de> References: <478BD2B4.2010706@cheimes.de> Message-ID: <478BD7D8.7080806@egenix.com> On 2008-01-14 22:23, Christian Heimes wrote: > The PEP is now available at http://www.python.org/dev/peps/pep-0370/. > The reference implementation is in svn, too: > svn+ssh://pythondev at svn.python.org/sandbox/trunk/pep370 Thanks for the effort, Christian. Much appreciated. Regarding the recent ~/bin vs. ~/.local/bin discussion: I usually maintain my ~/bin directories by hand and wouldn't want any application to install things in there automatically (and so far I haven't been using any application that does), so I'd be in favor of the ~/.local/bin dir. Note that users typically don't know which scripts are made available by a Python application and it's not always clear what functionality they provide, whether they can be trusted, include bugs, need to be run with extra care, etc, so IMHO making it a little harder to run them by accident is well warranted. Thanks again, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 14 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From martin at v.loewis.de Mon Jan 14 23:19:56 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 14 Jan 2008 23:19:56 +0100 Subject: [Python-Dev] [Python-checkins] r59947 - in python/trunk:Lib/test/test_structseq.py Misc/NEWS In-Reply-To: <478BACF3.8010406@cheimes.de> References: <20080114033353.2B9241E401A@bag.python.org> <001201c85661$181a4390$6800a8c0@RaymondLaptop1> <478B0DA1.3030405@v.loewis.de> <478BACF3.8010406@cheimes.de> Message-ID: <478BE00C.3030500@v.loewis.de> > Correct. We don't need item access anymore. However the struct seq > should still be slice-able for functions like time.mktime(). Can you please explain that? What application do you have in mind? Regards, Martin From guido at python.org Mon Jan 14 23:49:37 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 14 Jan 2008 14:49:37 -0800 Subject: [Python-Dev] [Python-checkins] r59947 - in python/trunk:Lib/test/test_structseq.py Misc/NEWS In-Reply-To: <478BE00C.3030500@v.loewis.de> References: <20080114033353.2B9241E401A@bag.python.org> <001201c85661$181a4390$6800a8c0@RaymondLaptop1> <478B0DA1.3030405@v.loewis.de> <478BACF3.8010406@cheimes.de> <478BE00C.3030500@v.loewis.de> Message-ID: On Jan 14, 2008 2:19 PM, "Martin v. L?wis" wrote: > > Correct. We don't need item access anymore. However the struct seq > > should still be slice-able for functions like time.mktime(). > > Can you please explain that? What application do you have in mind? Well, mktime() assumes its argument to be a tuple, and there are plenty of places that either emulate its API (like calendar.timegm()) or provide a tuple for it. I wouldn't want to lose the ability to manually construct a tuple to go into mktime() and friends. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From alexandre at peadrop.com Tue Jan 15 00:04:06 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Mon, 14 Jan 2008 18:04:06 -0500 Subject: [Python-Dev] [Python-3000] inst_persistent_id In-Reply-To: <20080114175943.GA26431@code0.codespeak.net> References: <20080114175943.GA26431@code0.codespeak.net> Message-ID: Oh, you are right. I thought that save_inst() used inst_persistent_id, but that isn't the case. Now, I have checked more thoroughly and found the relevant piece of code: if (!pers_save && self->inst_pers_func) { if ((tmp = save_pers(self, args, self->inst_pers_func)) != 0) { res = tmp; goto finally; } } which is indeed called only when the object is not "supported" by pickle. I guess my original argument doesn't hold anymore, thus I don't have anything against supporting this feature officially. Thanks for correcting me! -- Alexandre On Jan 14, 2008 12:59 PM, Armin Rigo wrote: > Hi, > > On Sat, Jan 12, 2008 at 07:33:38PM -0500, Alexandre Vassalotti wrote: > > Well, in Python 3K, inst_persistent_id() won't be usable, since > > PyInstance_Type was removed. > > Looking at the code, inst_persistent_id() is just a confusing name. It > has got nothing to do with PyInstance_Type; it's called for any object > type that cPickle.c doesn't know how to handle. In fact, it seems that > cPickle.c never calls inst_persistent_id() for objects of type > PyInstance_Type... > > > A bientot, > > Armin. > From martin at v.loewis.de Tue Jan 15 00:28:27 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 15 Jan 2008 00:28:27 +0100 Subject: [Python-Dev] [Python-checkins] r59947 - in python/trunk:Lib/test/test_structseq.py Misc/NEWS In-Reply-To: References: <20080114033353.2B9241E401A@bag.python.org> <001201c85661$181a4390$6800a8c0@RaymondLaptop1> <478B0DA1.3030405@v.loewis.de> <478BACF3.8010406@cheimes.de> <478BE00C.3030500@v.loewis.de> Message-ID: <478BF01B.3080505@v.loewis.de> Guido van Rossum wrote: > On Jan 14, 2008 2:19 PM, "Martin v. L?wis" wrote: >>> Correct. We don't need item access anymore. However the struct seq >>> should still be slice-able for functions like time.mktime(). >> Can you please explain that? What application do you have in mind? > > Well, mktime() assumes its argument to be a tuple, and there are > plenty of places that either emulate its API (like calendar.timegm()) > or provide a tuple for it. I wouldn't want to lose the ability to > manually construct a tuple to go into mktime() and friends. But what about the slicing? AFAICT, mktime doesn't support "short" tuples. mktime could continue to support tuples (including manually created ones), yet struct_time could still be a proper class, as long as mktime accepts that as well. Regards, Martin From guido at python.org Tue Jan 15 00:37:13 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 14 Jan 2008 15:37:13 -0800 Subject: [Python-Dev] [Python-checkins] r59947 - in python/trunk:Lib/test/test_structseq.py Misc/NEWS In-Reply-To: <478BF01B.3080505@v.loewis.de> References: <20080114033353.2B9241E401A@bag.python.org> <001201c85661$181a4390$6800a8c0@RaymondLaptop1> <478B0DA1.3030405@v.loewis.de> <478BACF3.8010406@cheimes.de> <478BE00C.3030500@v.loewis.de> <478BF01B.3080505@v.loewis.de> Message-ID: A use case for slicing was just found in zipfile.py: date = "%d-%02d-%02d %02d:%02d:%02d" % zinfo.date_time[:6] On Jan 14, 2008 3:28 PM, "Martin v. L?wis" wrote: > Guido van Rossum wrote: > > On Jan 14, 2008 2:19 PM, "Martin v. L?wis" wrote: > >>> Correct. We don't need item access anymore. However the struct seq > >>> should still be slice-able for functions like time.mktime(). > >> Can you please explain that? What application do you have in mind? > > > > Well, mktime() assumes its argument to be a tuple, and there are > > plenty of places that either emulate its API (like calendar.timegm()) > > or provide a tuple for it. I wouldn't want to lose the ability to > > manually construct a tuple to go into mktime() and friends. > > But what about the slicing? AFAICT, mktime doesn't support "short" > tuples. > > mktime could continue to support tuples (including manually created > ones), yet struct_time could still be a proper class, as long as mktime > accepts that as well. > > Regards, > Martin > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jon+python-dev at unequivocal.co.uk Tue Jan 15 00:41:47 2008 From: jon+python-dev at unequivocal.co.uk (Jon Ribbens) Date: Mon, 14 Jan 2008 23:41:47 +0000 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: References: <478B50B4.3040809@livinglogic.de> <20080114141539.21558.782173562.divmod.xquotient.4580@joule.divmod.com> <478BABD9.6070206@cheimes.de> <20080114191326.21558.1896823221.divmod.xquotient.4614@joule.divmod.com> Message-ID: <20080114234147.GO30947@snowy.squish.net> On Mon, Jan 14, 2008 at 03:07:20PM -0500, Barry Warsaw wrote: > I feel pretty strongly that ~/bin should *not* be used. It makes > sense to me that ~/.local would mirror /usr/local. It makes sense, but personally I have never heard before of ~/.local. Whereas ~/bin is something I am quite familiar with. From adlaiff6 at gmail.com Tue Jan 15 01:03:34 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Mon, 14 Jan 2008 19:03:34 -0500 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <20080114234147.GO30947@snowy.squish.net> References: <478B50B4.3040809@livinglogic.de> <20080114141539.21558.782173562.divmod.xquotient.4580@joule.divmod.com> <478BABD9.6070206@cheimes.de> <20080114191326.21558.1896823221.divmod.xquotient.4614@joule.divmod.com> <20080114234147.GO30947@snowy.squish.net> Message-ID: <11b6d2f60801141603r58974d59u51a3c43bd84090e9@mail.gmail.com> On Jan 14, 2008 6:41 PM, Jon Ribbens wrote: > It makes sense, but personally I have never heard before of ~/.local. > Whereas ~/bin is something I am quite familiar with. *raises hand* I have one, fwiw. -- Cheers, Leif From asmodai at in-nomine.org Tue Jan 15 08:37:18 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Tue, 15 Jan 2008 08:37:18 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: References: Message-ID: <20080115073718.GB75977@nexus.in-nomine.org> One thing I miss in this PEP and discussion is the point of view from a webhosting company and webhosting user. If the webhoster upgrades his hosting software and Python get updated by a revision (say 2.5 to 2.6) this would in the current case mean that the webhosting user using per-user site-packages is out of luck due to the hard-wired (and apparently needed) version identifier. Or is that a wrong assumption on my side? Right now Python faces a lot of problems in the webhosting world because it's tedious to set up and maintain for the webhosting user since they often have to compile and install their own Python in their home directory. The per-user site-packages will help in that aspect a lot, I think, but we do need to detail some more use-cases and scenarios (patterns if you like) on how such deployments would work. I think this is extremely important due to the proliferation of Python now more and more as a choice for webapp development. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ The greatest of faults, I should say, is to be conscious of none... From lists at cheimes.de Tue Jan 15 09:54:35 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 15 Jan 2008 09:54:35 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <20080115073718.GB75977@nexus.in-nomine.org> References: <20080115073718.GB75977@nexus.in-nomine.org> Message-ID: <478C74CB.5020109@cheimes.de> Jeroen Ruigrok van der Werven wrote: > One thing I miss in this PEP and discussion is the point of view from a > webhosting company and webhosting user. > > If the webhoster upgrades his hosting software and Python get updated by a > revision (say 2.5 to 2.6) this would in the current case mean that the > webhosting user using per-user site-packages is out of luck due to the > hard-wired (and apparently needed) version identifier. Or is that a wrong > assumption on my side? The problem exists for almost every Python extension. 3rd party packages with C modules have to be compiled for every Python version. Even pure Python packages don't always work on a new version. You can't expect software to run under every version of FOO, not even the next future version of FOO where FOO is Python, Java, C#, Perl, PHP ... you name it. It's not the intention of the PEP to solve the problem. A web hoster can support multiple versions of Python at once. You can install every minor version of Python from 2.0 to 3.0 (8 versions) without major conflicts. A *good* web hoster should give an user several months up to a year to migrate from - say - 2.5 to 2.6. Most Linux distributions (except Redhat) have at least 2.4 and 2.5 in their repositories. > Right now Python faces a lot of problems in the webhosting world because it's > tedious to set up and maintain for the webhosting user since they often have > to compile and install their own Python in their home directory. The per-user > site-packages will help in that aspect a lot, I think, but we do need to > detail some more use-cases and scenarios (patterns if you like) on how such > deployments would work. > I think this is extremely important due to the proliferation of Python now > more and more as a choice for webapp development. Can you write up some use cases for us? I'm especially interested in use cases that are not covered by PEP 370. So far only a few problems come into my mind: * Installation extensions for mod_python, requires root permissions or full control over the Apache process anyway * Multiple versions of an extension. The problem is mostly covered by pkg_resources and setup tools. * Installing a newer version of an extension, covered by PEP 370. * Registering more search paths. Drop a pth file into the user site directory or use a usercustumize.py. * easy_install packages to the user site directory. Setuptools are not part of the Python stdlib. I'm confident that Phil is going to add the feature as soon as the feature is implemented for Python 2.6 What's missing? Christian From phd at phd.pp.ru Tue Jan 15 12:24:13 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Tue, 15 Jan 2008 14:24:13 +0300 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <20080114234147.GO30947@snowy.squish.net> References: <478B50B4.3040809@livinglogic.de> <20080114141539.21558.782173562.divmod.xquotient.4580@joule.divmod.com> <478BABD9.6070206@cheimes.de> <20080114191326.21558.1896823221.divmod.xquotient.4614@joule.divmod.com> <20080114234147.GO30947@snowy.squish.net> Message-ID: <20080115112413.GA13674@phd.pp.ru> On Mon, Jan 14, 2008 at 11:41:47PM +0000, Jon Ribbens wrote: > It makes sense, but personally I have never heard before of ~/.local. > Whereas ~/bin is something I am quite familiar with. Me too. python-dev is the only place I have heard of ~/.local. I have been using Linux (different distributions), Solaris and FreeBSD for quite a long time (though I have never used GNOME/KDE/etc.) Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From bioinformed at gmail.com Tue Jan 15 12:57:02 2008 From: bioinformed at gmail.com (Kevin Jacobs ) Date: Tue, 15 Jan 2008 06:57:02 -0500 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <20080115112413.GA13674@phd.pp.ru> References: <478B50B4.3040809@livinglogic.de> <20080114141539.21558.782173562.divmod.xquotient.4580@joule.divmod.com> <478BABD9.6070206@cheimes.de> <20080114191326.21558.1896823221.divmod.xquotient.4614@joule.divmod.com> <20080114234147.GO30947@snowy.squish.net> <20080115112413.GA13674@phd.pp.ru> Message-ID: <2e1434c10801150357i247cf03dje874a36351c89c22@mail.gmail.com> On Jan 15, 2008 6:24 AM, Oleg Broytmann wrote: > On Mon, Jan 14, 2008 at 11:41:47PM +0000, Jon Ribbens wrote: > > It makes sense, but personally I have never heard before of ~/.local. > > Whereas ~/bin is something I am quite familiar with. > > Me too. python-dev is the only place I have heard of ~/.local. I have > been using Linux (different distributions), Solaris and FreeBSD for quite > a long time (though I have never used GNOME/KDE/etc.) > Never heard of it either, would be completely baffled if caught unawares by it in the wild. Has anyone consulted with the LSB or a cross-platform filesystem layout guide to see what the recommended best-practice is? -Kevin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080115/4af6b235/attachment.htm From lists at cheimes.de Tue Jan 15 13:21:46 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 15 Jan 2008 13:21:46 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <2e1434c10801150357i247cf03dje874a36351c89c22@mail.gmail.com> References: <478B50B4.3040809@livinglogic.de> <20080114141539.21558.782173562.divmod.xquotient.4580@joule.divmod.com> <478BABD9.6070206@cheimes.de> <20080114191326.21558.1896823221.divmod.xquotient.4614@joule.divmod.com> <20080114234147.GO30947@snowy.squish.net> <20080115112413.GA13674@phd.pp.ru> <2e1434c10801150357i247cf03dje874a36351c89c22@mail.gmail.com> Message-ID: <478CA55A.9010606@cheimes.de> Kevin Jacobs wrote: > Never heard of it either, would be completely baffled if caught unawares by > it in the wild. Has anyone consulted with the LSB or a cross-platform > filesystem layout guide to see what the recommended best-practice is? It took me a while to find a reference to .local. It's part of the FreeDesktop.Org standards: http://standards.freedesktop.org/basedir-spec/basedir-spec-0.6.html Christian From phd at phd.pp.ru Tue Jan 15 13:28:28 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Tue, 15 Jan 2008 15:28:28 +0300 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <478CA55A.9010606@cheimes.de> References: <478B50B4.3040809@livinglogic.de> <20080114141539.21558.782173562.divmod.xquotient.4580@joule.divmod.com> <478BABD9.6070206@cheimes.de> <20080114191326.21558.1896823221.divmod.xquotient.4614@joule.divmod.com> <20080114234147.GO30947@snowy.squish.net> <20080115112413.GA13674@phd.pp.ru> <2e1434c10801150357i247cf03dje874a36351c89c22@mail.gmail.com> <478CA55A.9010606@cheimes.de> Message-ID: <20080115122828.GD14063@phd.pp.ru> On Tue, Jan 15, 2008 at 01:21:46PM +0100, Christian Heimes wrote: > It took me a while to find a reference to .local. It's part of the > FreeDesktop.Org standards: > http://standards.freedesktop.org/basedir-spec/basedir-spec-0.6.html The site only mentions $HOME/.local/share, there is no $HOME/.local/bin at the site. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From guido at python.org Tue Jan 15 16:37:50 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 15 Jan 2008 07:37:50 -0800 Subject: [Python-Dev] Monkeypatching idioms -- elegant or ugly? Message-ID: I ran into the need of monkeypatching a large number of classes (for what I think are very good reasons :-) and invented two new recipes. These don't depend on Py3k, and the second one actually works all the way back to Python 2.2. Neither of these allows monkeypatching built-in types like list. If you don't know what monkeypatching is, see see http://en.wikipedia.org/wiki/Monkey_patch. I think it's useful to share these recipes, if only to to establish whether they have been discovered before, or to decide whether they are worthy of a place in the standard library. I didn't find any relevant hits on the ASPN Python cookbook. First, a decorator to add a single method to an existing class: def monkeypatch_method(cls): def decorator(func): setattr(cls, func.__name__, func) return func return decorator To use: from import @monkeypatch_method() def (self, args): return This adds to Second, a "metaclass" to add a number of methods (or other attributes) to an existing class, using a convenient class notation: def monkeypatch_class(name, bases, namespace): assert len(bases) == 1, "Exactly one base class required" base = bases[0] for name, value in namespace.iteritems(): if name != "__metaclass__": setattr(base, name, value) return base To use: from import class (): __metaclass__ = monkeypatch_class def (...): ... def (...): ... ... This adds , , etc. to , and makes a local alias for . -- --Guido van Rossum (home page: http://www.python.org/~guido/) From glyph at divmod.com Tue Jan 15 16:59:53 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Tue, 15 Jan 2008 15:59:53 -0000 Subject: [Python-Dev] Monkeypatching idioms -- elegant or ugly? In-Reply-To: References: Message-ID: <20080115155953.21558.61307424.divmod.xquotient.4672@joule.divmod.com> On 03:37 pm, guido at python.org wrote: >I think it's useful to share these recipes, if only to to establish >whether they have been discovered before, or to decide whether they >are worthy of a place in the standard library. I didn't find any >relevant hits on the ASPN Python cookbook. >from import > >class (): > __metaclass__ = monkeypatch_class > def (...): ... > def (...): ... > ... I've expressed this one before as "class someclass(reopen(someclass)):", but have thankfully never needed to actually use that in a real program. It's been a helpful tool in explaining to overzealous Ruby-ists that "reopenable" classes are not as unique as they think. My feelings on monkeypatching is that it *should* feel a little gross when you have to do it, so the code I've written that does monkeypatching for real is generally a bit ugly. From steve at holdenweb.com Tue Jan 15 17:17:02 2008 From: steve at holdenweb.com (Steve Holden) Date: Tue, 15 Jan 2008 11:17:02 -0500 Subject: [Python-Dev] Monkeypatching idioms -- elegant or ugly? In-Reply-To: <20080115155953.21558.61307424.divmod.xquotient.4672@joule.divmod.com> References: <20080115155953.21558.61307424.divmod.xquotient.4672@joule.divmod.com> Message-ID: glyph at divmod.com wrote: > On 03:37 pm, guido at python.org wrote: >> I think it's useful to share these recipes, if only to to establish >> whether they have been discovered before, or to decide whether they >> are worthy of a place in the standard library. I didn't find any >> relevant hits on the ASPN Python cookbook. > >>from import >> class (): >> __metaclass__ = monkeypatch_class >> def (...): ... >> def (...): ... >> ... > > I've expressed this one before as "class someclass(reopen(someclass)):", > but have thankfully never needed to actually use that in a real program. > It's been a helpful tool in explaining to overzealous Ruby-ists that > "reopenable" classes are not as unique as they think. > > My feelings on monkeypatching is that it *should* feel a little gross > when you have to do it, so the code I've written that does > monkeypatching for real is generally a bit ugly. Yes, monkeypatching should never be formalized to the point where novices see it as other than a tool of last resort. Otherwise a user base will grow that uses monkeypatching instead of subclassing, for example (shudder). regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From pje at telecommunity.com Tue Jan 15 18:33:11 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 15 Jan 2008 12:33:11 -0500 Subject: [Python-Dev] PEP: Post import hooks In-Reply-To: <4787DB58.5010900@cheimes.de> References: <20080110043258.BEBAB3A40A9@sparrow.telecommunity.com> <20080110223235.13D5B3A406C@sparrow.telecommunity.com> <4786A579.2070605@cheimes.de> <20080111010557.63A2B3A406C@sparrow.telecommunity.com> <4787DB58.5010900@cheimes.de> Message-ID: <20080115173311.6FA9F3A4077@sparrow.telecommunity.com> At 10:10 PM 1/11/2008 +0100, Christian Heimes wrote: >Phillip J. Eby wrote: > > *sigh*. We seem to be getting further and further off course, > > here. The more different you make the semantics of the system, the > > harder it will be to verify that it's doing the right thing without > > having real field experience with the new approach. > >*sigh*, too. :/ > >This discussion has neither helping me nor you. Could you please write >an unit test or two to show me exactly what my implementation is doing >wrong and how you think the callbacks should be called. I know a simple >test won't proof the correctness of the implementation but a failing >test would at least show the incorrectness. when_imported('a.b')(func_ab1) when_imported('a.b')(func_ab2) @when_imported('a') def func_a1(module_a): when_imported('a.b')(func_ab3) notify_module('a.b') # <- this is here to foil trivial implementations when_imported('a')(func_a2) notify_module('a.b') This should produce the calling sequence: func_a1, func_a2, func_ab1, func_ab2, func_ab3. >I'm still not sure which way is the correct way in your opinion and I >hate guessing what you are trying to explain to me. The invariants to ensure are: 1. notification is only done once for a given module, ever, even if the notification function is called more than once, even if it's called during notifications for that module 2. notifications for a child module/package may not begin until the notifications for the parent package have begun 3. two registrations for the same module must always be invoked in the same order as they were registered, even if some of the registrations are done during notification. In order to implement these invariants, you will have to have a way to know whether notifications have been begun for a given module. In peak.util.imports, the module objects effectively keep track of this, although they don't have a specific flag. For the Python implementation, you could add a __notified__ field to module objects, and implement the notify function thus: def notify(name): try: module = sys.modules[name] except KeyError: raise ImportError("Module %s has not been imported" % (name,)) if module.__notified__: return if '.' in name: notify(name[:name.rfind('.')]) try: module.__notified__ = True for callback in post_import_hooks[name]: callback(module) finally: post_import_hooks[name] = None Of course, __notified__ would actually be a structure slot, rather than an attribute, so as to avoid any attribute lookup issues with module subtypes (e.g. lazy modules). The register function would simply append a hook to the entry in post_import_hooks if it's not None, or call the hook otherwise. With this implementation, I could make a version of peak.util.imports that did its own lazy modules, but used the base system for all the hooks. From fumanchu at aminus.org Tue Jan 15 19:13:25 2008 From: fumanchu at aminus.org (Robert Brewer) Date: Tue, 15 Jan 2008 10:13:25 -0800 Subject: [Python-Dev] Monkeypatching idioms -- elegant or ugly? In-Reply-To: References: Message-ID: Guido van Rossum: > I ran into the need of monkeypatching a large number of classes (for > what I think are very good reasons :-) and invented two new recipes. > These don't depend on Py3k, and the second one actually works all the > way back to Python 2.2. Neither of these allows monkeypatching > built-in types like list. If you don't know what monkeypatching is, > see see http://en.wikipedia.org/wiki/Monkey_patch. > > I think it's useful to share these recipes, if only to to establish > whether they have been discovered before, or to decide whether they > are worthy of a place in the standard library. I didn't find any > relevant hits on the ASPN Python cookbook. > > First, a decorator to add a single method to an existing class: > > def monkeypatch_method(cls): > def decorator(func): > setattr(cls, func.__name__, func) > return func > return decorator > > To use: > > from import > > @monkeypatch_method() > def (self, args): > return > > This adds to I like it, but my first thought was, "and if that method already exists?" I'd extend monkeypatch_method to store a reference to the old method(s): def monkeypatch_method(cls): """Add the decorated method to the given class; replace as needed. If the named method already exists on the given class, it will be replaced, and a reference to the old method appended to a list at cls._old_. If the "_old_" attribute already exists and is not a list, KeyError is raised. """ def decorator(func): fname = func.__name__ old_func = getattr(cls, fname, None) if old_func is not None: # Add the old func to a list of old funcs. old_ref = "_old_%s" % fname old_funcs = getattr(cls, old_ref, None) if old_funcs is None: setattr(cls, old_ref, []) elif not isinstance(old_funcs, list): raise KeyError("%s.%s already exists." % (cls.__name__, old_ref)) getattr(cls, old_ref).append(old_func) setattr(cls, fname, func) return func return decorator I chose a renaming scheme somewhat at random. The list allows you (or someone else ;) to call monkeypatch repeatedly on the same cls.method (but it's not thread-safe). And although it might seem to be making monkeypatches easier to perform, at least it's very explicit about what's going on as long as you keep "monkeypatch" in the name. Robert Brewer fumanchu at aminus.org From solipsis at pitrou.net Tue Jan 15 17:22:25 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 15 Jan 2008 16:22:25 +0000 (UTC) Subject: [Python-Dev] Monkeypatching idioms -- elegant or ugly? References: Message-ID: Guido van Rossum python.org> writes: > > I ran into the need of monkeypatching a large number of classes (for > what I think are very good reasons and invented two new recipes. > These don't depend on Py3k, and the second one actually works all the > way back to Python 2.2. Neither of these allows monkeypatching > built-in types like list. If you don't know what monkeypatching is, > see see http://en.wikipedia.org/wiki/Monkey_patch. > > I think it's useful to share these recipes, if only to to establish > whether they have been discovered before, or to decide whether they > are worthy of a place in the standard library. I didn't find any > relevant hits on the ASPN Python cookbook. pypy has something that requires the base class to use a specific metaclass: https://codespeak.net/viewvc/pypy/dist/pypy/tool/pairtype.py?view=markup From guido at python.org Tue Jan 15 19:31:09 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 15 Jan 2008 10:31:09 -0800 Subject: [Python-Dev] Monkeypatching idioms -- elegant or ugly? In-Reply-To: References: Message-ID: On Jan 15, 2008 8:22 AM, Antoine Pitrou wrote: > pypy has something that requires the base class to use a specific metaclass: > https://codespeak.net/viewvc/pypy/dist/pypy/tool/pairtype.py?view=markup That's no good for my particular use case. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From djarb at highenergymagic.org Tue Jan 15 21:11:37 2008 From: djarb at highenergymagic.org (Daniel Arbuckle) Date: Tue, 15 Jan 2008 12:11:37 -0800 Subject: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57 In-Reply-To: References: Message-ID: On Tue, 15 Jan 2008 06:57:02 -0500, Kevin Jacobs wrote: > On Jan 15, 2008 6:24 AM, Oleg Broytmann wrote: > > > On Mon, Jan 14, 2008 at 11:41:47PM +0000, Jon Ribbens wrote: > > > It makes sense, but personally I have never heard before of ~/.local. > > > Whereas ~/bin is something I am quite familiar with. > > > > Me too. python-dev is the only place I have heard of ~/.local. I have > > been using Linux (different distributions), Solaris and FreeBSD for quite > > a long time (though I have never used GNOME/KDE/etc.) > > > Never heard of it either, would be completely baffled if caught unawares by > it in the wild. Has anyone consulted with the LSB or a cross-platform > filesystem layout guide to see what the recommended best-practice is? I use ~/local, with a layout analogous to /usr, all the time. It's not a standard, but in my experience it is by far the best solution to installing things in the home directory. It doesn't matter much whether you call it local or .local or .pythonlocal (although that last would limit the utility somewhat, by implying that other things should be installed there). It does matter that it be a _subdirectory_ of ~, and that it be structured like /usr. To those folks who favor creating ~/bin, ~/lib, ~/share, ad nauseum, I point out that non-hidden, non-user-created files in ~ ought to be kept to a minimum. It is, after all, the user's default workspace. From phd at phd.pp.ru Tue Jan 15 21:18:13 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Tue, 15 Jan 2008 23:18:13 +0300 Subject: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57 In-Reply-To: References: Message-ID: <20080115201812.GA25987@phd.pp.ru> On Tue, Jan 15, 2008 at 12:11:37PM -0800, Daniel Arbuckle wrote: > I use ~/local, with a layout analogous to /usr, all the time. It's not > a standard, but in my experience it is by far the best solution to > installing things in the home directory. It doesn't matter much > whether you call it local or .local or .pythonlocal (although that > last would limit the utility somewhat, by implying that other things > should be installed there). It does matter that it be a _subdirectory_ > of ~, and that it be structured like /usr. ~/.python ~/.python/bin ~/.python/lib ~/.python/lib/python2.5 Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From skip at pobox.com Tue Jan 15 21:29:16 2008 From: skip at pobox.com (skip at pobox.com) Date: Tue, 15 Jan 2008 14:29:16 -0600 Subject: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57 In-Reply-To: References: Message-ID: <18317.6044.192089.113683@montanaro.dyndns.org> Daniel> I use ~/local, with a layout analogous to /usr, ... Ditto. Makes things nice and clear. I install stuff without becoming root or polluting central directories. Daniel> To those folks who favor creating ~/bin, ~/lib, ~/share, ad Daniel> nauseum, I point out that non-hidden, non-user-created files in Daniel> ~ ought to be kept to a minimum. It is, after all, the user's Daniel> default workspace. Agreed. You're just asking for trouble if you set --prefix=$HOME. Eventually you will encounter some package which would have scribbled all over /usr/local and only disrupted other external packages (which you can always reinstall) but instead scribbles all over your home directory, wiping out your almost finished Ph.D. thesis in Library Science which you had saved to ~/lib. Skip From skip at pobox.com Tue Jan 15 21:34:02 2008 From: skip at pobox.com (skip at pobox.com) Date: Tue, 15 Jan 2008 14:34:02 -0600 Subject: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57 In-Reply-To: <20080115201812.GA25987@phd.pp.ru> References: <20080115201812.GA25987@phd.pp.ru> Message-ID: <18317.6330.621953.789068@montanaro.dyndns.org> Oleg> ~/.python Oleg> ~/.python/bin Oleg> ~/.python/lib Oleg> ~/.python/lib/python2.5 The drawback of this approach is that it implies that Perl, Tcl, IPython, etc. belong in their own .whatever directory. The IT folks here at work do things that way (though not in home directories). If I want to build a package which relies on zlib, libpng, libtiff, libjpeg, etc., imagine what my CPPFLAGS, CFLAGS and LDFLAGS arguments look like. :barf: Skip From stephen at xemacs.org Tue Jan 15 21:43:33 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 16 Jan 2008 05:43:33 +0900 Subject: [Python-Dev] Monkeypatching idioms -- elegant or ugly? In-Reply-To: References: Message-ID: <87prw23jm2.fsf@uwakimon.sk.tsukuba.ac.jp> Guido van Rossum writes: > I think it's useful to share these recipes, But only to people who have demonstrated that they already know when, why and how to monkeypatch on their own. Lisp's `defadvice' plays a starring role in a number of my nightmares. Most recently, 15 minutes ago. Come to think of it, `defadvice' does have the advantage that it also automatically monkeypatches the docstring to say "This function has advice", and with a little cooperation from the advisor can also provide documentation of the changed semantics. From phd at phd.pp.ru Tue Jan 15 21:45:11 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Tue, 15 Jan 2008 23:45:11 +0300 Subject: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57 In-Reply-To: <18317.6330.621953.789068@montanaro.dyndns.org> References: <20080115201812.GA25987@phd.pp.ru> <18317.6330.621953.789068@montanaro.dyndns.org> Message-ID: <20080115204511.GA26905@phd.pp.ru> On Tue, Jan 15, 2008 at 02:34:02PM -0600, skip at pobox.com wrote: > Oleg> ~/.python > Oleg> ~/.python/bin > Oleg> ~/.python/lib > Oleg> ~/.python/lib/python2.5 > > The drawback of this approach is that it implies that Perl, Tcl, IPython, > etc. belong in their own .whatever directory. How many users install (parts of) all of these into their homes? > The IT folks here at work do > things that way (though not in home directories). If I want to build a > package which relies on zlib, libpng, libtiff, libjpeg, etc., imagine what > my CPPFLAGS, CFLAGS and LDFLAGS arguments look like. :barf: Why not use GNU stow? Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From lists at cheimes.de Tue Jan 15 22:14:20 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 15 Jan 2008 22:14:20 +0100 Subject: [Python-Dev] PEP: Post import hooks In-Reply-To: <20080115173311.6FA9F3A4077@sparrow.telecommunity.com> References: <20080110043258.BEBAB3A40A9@sparrow.telecommunity.com> <20080110223235.13D5B3A406C@sparrow.telecommunity.com> <4786A579.2070605@cheimes.de> <20080111010557.63A2B3A406C@sparrow.telecommunity.com> <4787DB58.5010900@cheimes.de> <20080115173311.6FA9F3A4077@sparrow.telecommunity.com> Message-ID: <478D222C.2050904@cheimes.de> Phillip J. Eby wrote: > when_imported('a.b')(func_ab1) > when_imported('a.b')(func_ab2) > > @when_imported('a') > def func_a1(module_a): > when_imported('a.b')(func_ab3) > notify_module('a.b') # <- this is here to foil trivial > implementations > > when_imported('a')(func_a2) > notify_module('a.b') > > This should produce the calling sequence: > > func_a1, func_a2, func_ab1, func_ab2, func_ab3. My implementation calls the hooks in the right order but I had to insert fake modules into sys.path. # insert the modules into sys.modules to fake a 3rd party import a = imp.new_module('a') ab = imp.new_module('a.b') a.b = ab sys.modules["a"] = a sys.modules["a.b"] = ab # notify imp.notify_module_loaded('a.b') Otherwise the code fails to fetch the module objects from sys.path and an exception is raised. > 1. notification is only done once for a given module, ever, even if the > notification function is called more than once, even if it's called > during notifications for that module The latter is not yet implemented. notify_module('a.b') currently fails with a recursion limit exceeded. It's on my todo list. > 2. notifications for a child module/package may not begin until the > notifications for the parent package have begun This is already implemented for notification by name but not for notification by module object. It's also on my todo list. > 3. two registrations for the same module must always be invoked in the > same order as they were registered, even if some of the registrations > are done during notification. I'm pretty but not yet absolute sure that my implementation guarantees the last invariant. I've to go through my code several times and play through use cases. > In order to implement these invariants, you will have to have a way to > know whether notifications have been begun for a given module. In > peak.util.imports, the module objects effectively keep track of this, > although they don't have a specific flag. For the Python > implementation, you could add a __notified__ field to module objects, > and implement the notify function thus: That's a nice idea to fix the recursion issue with nested notifications. > Of course, __notified__ would actually be a structure slot, rather than > an attribute, so as to avoid any attribute lookup issues with module > subtypes (e.g. lazy modules). Sure, I can use a slot for PyModule and its subclasses. Unfortunately other implementations of lazy imports are adding a proxy object which is not a subclass of PyModule. I've to use an attribute if not isinstance(mod, moduletype). > The register function would simply append a hook to the entry in > post_import_hooks if it's not None, or call the hook otherwise. My code queues up new hooks while a sequence of hooks is processed. It makes sure that hooks for a parent aren't called in the middle of a child's hook chain. notification_in_progress = False queue = [] def register_hook(hook, name): if notification_in_progress: queue.append((hook, name)) return hooks = sys.post_import_hook_register(name, None) if hooks is None: module = sys.modules.get(name, None) if modules: sys.post_import_hook_register[name] = None hook(module) else: hooks = [] sys.post_import_hook_register[name] = hooks hooks.append(hook) def notify_loaded(mod_or_name): notification_in_progress = True try: ... finally: notification_in_progress = False while queue: hook, name = queue.pop(0) hook(sys.modules[name]) > With this implementation, I could make a version of peak.util.imports > that did its own lazy modules, but used the base system for all the hooks. Very good! Christian From martin at v.loewis.de Tue Jan 15 22:23:24 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Tue, 15 Jan 2008 22:23:24 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <20080115073718.GB75977@nexus.in-nomine.org> References: <20080115073718.GB75977@nexus.in-nomine.org> Message-ID: <478D244C.8070408@v.loewis.de> > Right now Python faces a lot of problems in the webhosting world because it's > tedious to set up and maintain for the webhosting user since they often have > to compile and install their own Python in their home directory. I don't understand why they *have* to do that. I can believe they do that as they don't know better - but why can't they use the Python interpreter already available on the system, and just install additional packages in their home directory? > I think this is extremely important due to the proliferation of Python now > more and more as a choice for webapp development. For that, I think the requirements need to be much more explicit. Regards, Martin From stephen at xemacs.org Tue Jan 15 22:31:42 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 16 Jan 2008 06:31:42 +0900 Subject: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57 In-Reply-To: <20080115201812.GA25987@phd.pp.ru> References: <20080115201812.GA25987@phd.pp.ru> Message-ID: <87lk6q3hdt.fsf@uwakimon.sk.tsukuba.ac.jp> Oleg Broytmann writes: > ~/.python To me, this strongly suggests user configuration files, not a place where an app can store user-specific packages. True, there are apps that store their stuff in such places, like most GNOME apps. But they have no user-servicable parts (including config files) anyway. I think both for UI reasons (given above) and for API reasons (given by others) there should be a separate ~/SOMETHING/{bin,etc,lib,share} hierarchy for user-specific packaged contents. I like ~/.local a little better than ~/local, but both work for me. From phd at phd.pp.ru Tue Jan 15 22:38:31 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Wed, 16 Jan 2008 00:38:31 +0300 Subject: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57 In-Reply-To: <87lk6q3hdt.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20080115201812.GA25987@phd.pp.ru> <87lk6q3hdt.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20080115213831.GA28333@phd.pp.ru> On Wed, Jan 16, 2008 at 06:31:42AM +0900, Stephen J. Turnbull wrote: > I think both for UI reasons (given above) and for API reasons (given > by others) there should be a separate ~/SOMETHING/{bin,etc,lib,share} > hierarchy for user-specific packaged contents. I like ~/.local a > little better than ~/local, but both work for me. Having ~/.python allows me remove it with just one command. It's harder to clear ~/.local/{bin,lib} without affecting whatever I want to preserve there. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From guido at python.org Tue Jan 15 22:51:25 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 15 Jan 2008 13:51:25 -0800 Subject: [Python-Dev] Monkeypatching idioms -- elegant or ugly? In-Reply-To: <478D2537.4090503@v.loewis.de> References: <478D2537.4090503@v.loewis.de> Message-ID: On Jan 15, 2008 1:27 PM, "Martin v. L?wis" wrote: > > Second, a "metaclass" to add a number of methods (or other attributes) > > to an existing class, using a convenient class notation: > > I think this is similar to my "partial" classes: > > http://pypi.python.org/pypi/partial Indeed it is. I guess my only innovation is realizing that you don't have to create a real metaclass -- you can set __metaclass__ to a function that does the magic. I like your feature of refusing overrides unless flagged with @replace. I think that despite the objection that monkeypatching shoudn't be made too easy, it's worth at looking into a unification of the API, features, and implementation. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Tue Jan 15 23:04:42 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 15 Jan 2008 17:04:42 -0500 Subject: [Python-Dev] PEP: Post import hooks In-Reply-To: <478D222C.2050904@cheimes.de> References: <20080110043258.BEBAB3A40A9@sparrow.telecommunity.com> <20080110223235.13D5B3A406C@sparrow.telecommunity.com> <4786A579.2070605@cheimes.de> <20080111010557.63A2B3A406C@sparrow.telecommunity.com> <4787DB58.5010900@cheimes.de> <20080115173311.6FA9F3A4077@sparrow.telecommunity.com> <478D222C.2050904@cheimes.de> Message-ID: <20080115220443.AE18B3A4077@sparrow.telecommunity.com> At 10:14 PM 1/15/2008 +0100, Christian Heimes wrote: >My code queues up new hooks while a sequence of hooks is processed. It >makes sure that hooks for a parent aren't called in the middle of a >child's hook chain. Notice that that's not necessary with the notification algorithm I gave, since the list in post_import_hooks suffices as a queue. So, just as in peak.util.imports, the registration code doesn't need to know whether callbacks are being run; it only needs to know whether they're *finished*. Of course, both the notification and registration functions must hold the import lock to prevent a race condition where one thread adds a hook to the list after another thread has just finished iterating over it and is about to replace the list with None. At least, they have to if they're executing any Python code that might cause the GIL to be released. The callbacks will release the GIL, of course, but the registration code probably doesn't... well, it will if it calls the hook, and ISTM that the hooks should always execute with the import lock held, even if they're fired at registration. From pje at telecommunity.com Tue Jan 15 23:08:37 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 15 Jan 2008 17:08:37 -0500 Subject: [Python-Dev] Monkeypatching idioms -- elegant or ugly? In-Reply-To: References: <478D2537.4090503@v.loewis.de> Message-ID: <20080115220838.CC6373A4077@sparrow.telecommunity.com> At 01:51 PM 1/15/2008 -0800, Guido van Rossum wrote: >On Jan 15, 2008 1:27 PM, "Martin v. L?wis" wrote: > > > Second, a "metaclass" to add a number of methods (or other attributes) > > > to an existing class, using a convenient class notation: > > > > I think this is similar to my "partial" classes: > > > > http://pypi.python.org/pypi/partial > >Indeed it is. I guess my only innovation is realizing that you don't >have to create a real metaclass -- you can set __metaclass__ to a >function that does the magic. I like your feature of refusing >overrides unless flagged with @replace. > >I think that despite the objection that monkeypatching shoudn't be >made too easy, it's worth at looking into a unification of the API, >features, and implementation. I'm curious: has this affected your thoughts re: overloading existing functions? Note that overloading-in-place would provide the next_method idiom for calling the original function. (I'm assuming you still don't like the idea of changing a function's code to do it, just wondering about the non-implementation aspect. :) ) From guido at python.org Tue Jan 15 23:19:36 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 15 Jan 2008 14:19:36 -0800 Subject: [Python-Dev] Monkeypatching idioms -- elegant or ugly? In-Reply-To: <20080115220838.CC6373A4077@sparrow.telecommunity.com> References: <478D2537.4090503@v.loewis.de> <20080115220838.CC6373A4077@sparrow.telecommunity.com> Message-ID: On Jan 15, 2008 2:08 PM, Phillip J. Eby wrote: > At 01:51 PM 1/15/2008 -0800, Guido van Rossum wrote: > >On Jan 15, 2008 1:27 PM, "Martin v. L?wis" wrote: > > > > Second, a "metaclass" to add a number of methods (or other attributes) > > > > to an existing class, using a convenient class notation: > > > > > > I think this is similar to my "partial" classes: > > > > > > http://pypi.python.org/pypi/partial > > > >Indeed it is. I guess my only innovation is realizing that you don't > >have to create a real metaclass -- you can set __metaclass__ to a > >function that does the magic. I like your feature of refusing > >overrides unless flagged with @replace. > > > >I think that despite the objection that monkeypatching shoudn't be > >made too easy, it's worth at looking into a unification of the API, > >features, and implementation. > > I'm curious: has this affected your thoughts re: overloading existing > functions? Note that overloading-in-place would provide the > next_method idiom for calling the original function. > > (I'm assuming you still don't like the idea of changing a function's > code to do it, just wondering about the non-implementation aspect. :) ) Not really -- I'm still thinking of a class as something with public internal structure while I think of a function as something with private internal structure (except for function attributes settable in its __dict__). While I have you, I've come across a need that I don't know how to do with GFs. Suppose I have a GF that implements some recursive function over container types, e.g. serialization or flattening. Now suppose I'd like to create *another* GF that implements the same algorithm except it does something different for one particular type; as a concrete example, suppose we want to treat tuples atomically when flattening. Is there a way to reuse the work of the first GF? It doesn't work to create a new GF that calls on the first GF for types it doesn't understand; e.g. a list could contain a tuple. Does your GF machinery let me do this in a relatively clean way? (I think of it as "subclassing" a GF, where I think of the GF as a class that has a method for each supported type. If GFs were implemented this way, I could simply subclass it and override the method for tuples.) You might want to change the subject if you care to respond. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Tue Jan 15 22:27:19 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 15 Jan 2008 22:27:19 +0100 Subject: [Python-Dev] Monkeypatching idioms -- elegant or ugly? In-Reply-To: References: Message-ID: <478D2537.4090503@v.loewis.de> > Second, a "metaclass" to add a number of methods (or other attributes) > to an existing class, using a convenient class notation: I think this is similar to my "partial" classes: http://pypi.python.org/pypi/partial Regards, Martin From stephen at xemacs.org Wed Jan 16 00:17:11 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 16 Jan 2008 08:17:11 +0900 Subject: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57 In-Reply-To: <20080115213831.GA28333@phd.pp.ru> References: <20080115201812.GA25987@phd.pp.ru> <87lk6q3hdt.fsf@uwakimon.sk.tsukuba.ac.jp> <20080115213831.GA28333@phd.pp.ru> Message-ID: <87ejci3ci0.fsf@uwakimon.sk.tsukuba.ac.jp> Oleg Broytmann writes: > On Wed, Jan 16, 2008 at 06:31:42AM +0900, Stephen J. Turnbull wrote: > > I think both for UI reasons (given above) and for API reasons (given > > by others) there should be a separate ~/SOMETHING/{bin,etc,lib,share} > > hierarchy for user-specific packaged contents. I like ~/.local a > > little better than ~/local, but both work for me. > > Having ~/.python allows me remove it with just one command. It's harder > to clear ~/.local/{bin,lib} without affecting whatever I want to preserve > there. No, it shouldn't be. Part of the implementation should be a way for user-specific packages to uninstall themselves. It's also not clear to me how you can "just remove" ~/.python if there is more than one package installed there. From pje at telecommunity.com Wed Jan 16 00:40:00 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 15 Jan 2008 18:40:00 -0500 Subject: [Python-Dev] Extending generic functions In-Reply-To: References: <478D2537.4090503@v.loewis.de> <20080115220838.CC6373A4077@sparrow.telecommunity.com> Message-ID: <20080115234000.84CF63A4077@sparrow.telecommunity.com> At 02:19 PM 1/15/2008 -0800, Guido van Rossum wrote: >While I have you, I've come across a need that I don't know how to do >with GFs. Suppose I have a GF that implements some recursive function >over container types, e.g. serialization or flattening. Now suppose >I'd like to create *another* GF that implements the same algorithm >except it does something different for one particular type; as a >concrete example, suppose we want to treat tuples atomically when >flattening. Is there a way to reuse the work of the first GF? Yes. RuleDispatch actually has a 'clone()' feature for single-dispatch generics that does exactly what you're looking for: http://peak.telecommunity.com/DevCenter/VisitorRevisited (see the heading "Extension and Reuse"). It's probably not a bad idea to put a cloning feature on my extended to-do list for PEAK-Rules. In PEAK-Rules (the system after which PEP 3124 was modelled), a generic function has a RuleSet that contains its rules, and RuleSets can be subscribed to. So, you could create a listener that automatically takes the rules added to one function and adds them to others. It's not packaged as a convenient decorator or anything, but one certainly could make one. It'd also need to have some way to ensure that the rules from the original function were treated at a lower combination precedence than anything else, but that could be handled by with a custom method type pretty easily, I think. All in all, a cloning feature might be somewhere around 20-50 lines of code to add in -- and a third party could probably roll their own without changing PEAK-Rules' source. > It doesn't work to create a new GF that calls on the first GF for types >it doesn't understand; e.g. a list could contain a tuple. Does your GF >machinery let me do this in a relatively clean way? It's relatively clean. One of my goals for the changed architecture in PEAK-Rules vs. RuleDispatch was to make it possible to do all sorts of things like this, by opening up the whole thing to extensibility. Btw, a lot of the credit for PEAK-Rules' design goes to you, in a roundabout way. Your tuple-of-types prototype made me see that it could be practical to implement generic functions using generic functions as a base -- getting rid of interfaces and adaptation altogether. I just needed to come up with a design that allowed separating the genericness of a function (e.g. the rules to be applied) from the implementation of genericness (the "engine" that turns rules into executability. In this way, a generic function can start out using just tuples of types, but then graduate to a full expression-based system, just by changing the engine. From skip at pobox.com Wed Jan 16 01:31:53 2008 From: skip at pobox.com (skip at pobox.com) Date: Tue, 15 Jan 2008 18:31:53 -0600 Subject: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57 In-Reply-To: <20080115204511.GA26905@phd.pp.ru> References: <20080115201812.GA25987@phd.pp.ru> <18317.6330.621953.789068@montanaro.dyndns.org> <20080115204511.GA26905@phd.pp.ru> Message-ID: <18317.20601.712124.726412@montanaro.dyndns.org> Oleg> Why not use GNU stow? Thanks for the reference. I'd never heard of it before. I suspect our IT folks may not have as well. Skip From janssen at parc.com Wed Jan 16 02:23:50 2008 From: janssen at parc.com (Bill Janssen) Date: Tue, 15 Jan 2008 17:23:50 PST Subject: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57 In-Reply-To: <20080115213831.GA28333@phd.pp.ru> References: <20080115201812.GA25987@phd.pp.ru> <87lk6q3hdt.fsf@uwakimon.sk.tsukuba.ac.jp> <20080115213831.GA28333@phd.pp.ru> Message-ID: <08Jan15.172353pst."58696"@synergy1.parc.xerox.com> Oleg Broytmann writes: > On Wed, Jan 16, 2008 at 06:31:42AM +0900, Stephen J. Turnbull wrote: > > I think both for UI reasons (given above) and for API reasons (given > > by others) there should be a separate ~/SOMETHING/{bin,etc,lib,share} > > hierarchy for user-specific packaged contents. I like ~/.local a > > little better than ~/local, but both work for me. > > Having ~/.python allows me remove it with just one command. It's harder > to clear ~/.local/{bin,lib} without affecting whatever I want to preserve > there. Good point, but I prefer ~/Library/Python to either of these. Bill From lists at cheimes.de Wed Jan 16 02:28:15 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 16 Jan 2008 02:28:15 +0100 Subject: [Python-Dev] PEP: Post import hooks In-Reply-To: <20080115220443.AE18B3A4077@sparrow.telecommunity.com> References: <20080110043258.BEBAB3A40A9@sparrow.telecommunity.com> <20080110223235.13D5B3A406C@sparrow.telecommunity.com> <4786A579.2070605@cheimes.de> <20080111010557.63A2B3A406C@sparrow.telecommunity.com> <4787DB58.5010900@cheimes.de> <20080115173311.6FA9F3A4077@sparrow.telecommunity.com> <478D222C.2050904@cheimes.de> <20080115220443.AE18B3A4077@sparrow.telecommunity.com> Message-ID: <478D5DAF.8060709@cheimes.de> Phillip J. Eby wrote: > At 10:14 PM 1/15/2008 +0100, Christian Heimes wrote: >> My code queues up new hooks while a sequence of hooks is processed. It >> makes sure that hooks for a parent aren't called in the middle of a >> child's hook chain. > > Notice that that's not necessary with the notification algorithm I gave, > since the list in post_import_hooks suffices as a queue. So, just as in > peak.util.imports, the registration code doesn't need to know whether > callbacks are being run; it only needs to know whether they're *finished*. Are you sure your proposed algorithm and output match for the test case? I'm confident I got it right in C but I'm getting a different output. Without the extra imp.notify_module_loaded('a.b') in func_a1(mod):: ['func_a1', 'func_a2', 'func_ab1', 'func_ab2', 'func_ab3'] With the extra imp.notify_module_loaded('a.b') in func_a1(mod):: ['func_a1', 'func_ab1', 'func_ab2', 'func_ab3', 'func_a2'] I can't see how your implementation results in the first output when func_a1() calls the notification method. > Of course, both the notification and registration functions must hold > the import lock to prevent a race condition where one thread adds a hook > to the list after another thread has just finished iterating over it and > is about to replace the list with None. At least, they have to if > they're executing any Python code that might cause the GIL to be > released. The callbacks will release the GIL, of course, but the > registration code probably doesn't... well, it will if it calls the > hook, and ISTM that the hooks should always execute with the import lock > held, even if they're fired at registration. I'm aware of the implications and my code already uses the lock. The PyImport_NotifyLoaded() method excepts to be called with the importer lock acquired. So I'm locking the importer lock in imp_notify_module_loaded(). The PyImport_RegisterPostImportHook() method does the locking before it accesses sys.modules and sys.post_import_hooks. Christian From lists at cheimes.de Wed Jan 16 02:33:08 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 16 Jan 2008 02:33:08 +0100 Subject: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57 In-Reply-To: <08Jan15.172353pst."58696"@synergy1.parc.xerox.com> References: <20080115201812.GA25987@phd.pp.ru> <87lk6q3hdt.fsf@uwakimon.sk.tsukuba.ac.jp> <20080115213831.GA28333@phd.pp.ru> <08Jan15.172353pst."58696"@synergy1.parc.xerox.com> Message-ID: <478D5ED4.4060208@cheimes.de> Bill Janssen wrote: > Good point, but I prefer ~/Library/Python to either of these. ~/Library/ is a Mac OS X thing. I haven't seen it on other Unix systems. I *could* add yet another environment variable PYTHONUSERHOME to set the base path but I prefer not. Christian From lists at cheimes.de Wed Jan 16 02:33:08 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 16 Jan 2008 02:33:08 +0100 Subject: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57 In-Reply-To: <08Jan15.172353pst."58696"@synergy1.parc.xerox.com> References: <20080115201812.GA25987@phd.pp.ru> <87lk6q3hdt.fsf@uwakimon.sk.tsukuba.ac.jp> <20080115213831.GA28333@phd.pp.ru> <08Jan15.172353pst."58696"@synergy1.parc.xerox.com> Message-ID: <478D5ED4.4060208@cheimes.de> Bill Janssen wrote: > Good point, but I prefer ~/Library/Python to either of these. ~/Library/ is a Mac OS X thing. I haven't seen it on other Unix systems. I *could* add yet another environment variable PYTHONUSERHOME to set the base path but I prefer not. Christian From pje at telecommunity.com Wed Jan 16 02:42:56 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 15 Jan 2008 20:42:56 -0500 Subject: [Python-Dev] PEP: Post import hooks In-Reply-To: <478D5DAF.8060709@cheimes.de> References: <20080110043258.BEBAB3A40A9@sparrow.telecommunity.com> <20080110223235.13D5B3A406C@sparrow.telecommunity.com> <4786A579.2070605@cheimes.de> <20080111010557.63A2B3A406C@sparrow.telecommunity.com> <4787DB58.5010900@cheimes.de> <20080115173311.6FA9F3A4077@sparrow.telecommunity.com> <478D222C.2050904@cheimes.de> <20080115220443.AE18B3A4077@sparrow.telecommunity.com> <478D5DAF.8060709@cheimes.de> Message-ID: <20080116014256.C33DF3A4077@sparrow.telecommunity.com> At 02:28 AM 1/16/2008 +0100, Christian Heimes wrote: >Phillip J. Eby wrote: > > At 10:14 PM 1/15/2008 +0100, Christian Heimes wrote: > >> My code queues up new hooks while a sequence of hooks is processed. It > >> makes sure that hooks for a parent aren't called in the middle of a > >> child's hook chain. > > > > Notice that that's not necessary with the notification algorithm I gave, > > since the list in post_import_hooks suffices as a queue. So, just as in > > peak.util.imports, the registration code doesn't need to know whether > > callbacks are being run; it only needs to know whether they're *finished*. > >Are you sure your proposed algorithm and output match for the test case? >I'm confident I got it right in C but I'm getting a different output. I guess it's not right then. ;-) Though I shouldn't make fun, since it turns out that my code sketch was not a correct translation of peak.util.imports. (See below.) >Without the extra imp.notify_module_loaded('a.b') in func_a1(mod):: > > ['func_a1', 'func_a2', 'func_ab1', 'func_ab2', 'func_ab3'] > > >With the extra imp.notify_module_loaded('a.b') in func_a1(mod):: > > ['func_a1', 'func_ab1', 'func_ab2', 'func_ab3', 'func_a2'] Right - that's why I put it in there, to foil trivial implementations that don't really satisfy the invariant. >I can't see how your implementation results in the first output when >func_a1() calls the notification method. Hm, you're right, my implementation sketch waits too long to set the __notified__ flag. It should have read: def notify(name): try: module = sys.modules[name] except KeyError: raise ImportError("Module %s has not been imported" % (name,)) if module.__notified__: return try: module.__notified__ = True if '.' in name: notify(name[:name.rfind('.')]) for callback in post_import_hooks[name]: callback(module) finally: post_import_hooks[name] = None That is, module.__notified__ has to be set *before* the recursive notification call. This effectively happens in peak.util.imports now, except that __notified__ isn't an explicit attribute, just a side effect of other module state changes. >I'm aware of the implications and my code already uses the lock. The >PyImport_NotifyLoaded() method excepts to be called with the importer >lock acquired. So I'm locking the importer lock in >imp_notify_module_loaded(). The PyImport_RegisterPostImportHook() method >does the locking before it accesses sys.modules and sys.post_import_hooks. Great! From janssen at parc.com Wed Jan 16 03:24:41 2008 From: janssen at parc.com (Bill Janssen) Date: Tue, 15 Jan 2008 18:24:41 PST Subject: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57 In-Reply-To: <478D5ED4.4060208@cheimes.de> References: <20080115201812.GA25987@phd.pp.ru> <87lk6q3hdt.fsf@uwakimon.sk.tsukuba.ac.jp> <20080115213831.GA28333@phd.pp.ru> <08Jan15.172353pst."58696"@synergy1.parc.xerox.com> <478D5ED4.4060208@cheimes.de> Message-ID: <08Jan15.182444pst."58696"@synergy1.parc.xerox.com> > Bill Janssen wrote: > > Good point, but I prefer ~/Library/Python to either of these. > > ~/Library/ is a Mac OS X thing. I haven't seen it on other Unix systems. > I *could* add yet another environment variable PYTHONUSERHOME to set the > base path but I prefer not. > > Christian Sure, but it's clearly where this should be on an OS X system, by default. And I'm sure there's a different "best place" on Windows (for instance, all of our accounts are network roaming accounts, and you don't want to put anything in ~). And there are probably various right places for various flavors of Linux. Any PEP on this that doesn't take these OS-specific differences into account probably isn't worth reading, IMO. Bill From lists at cheimes.de Wed Jan 16 04:35:25 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 16 Jan 2008 04:35:25 +0100 Subject: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57 In-Reply-To: <08Jan15.182444pst."58696"@synergy1.parc.xerox.com> References: <20080115201812.GA25987@phd.pp.ru> <87lk6q3hdt.fsf@uwakimon.sk.tsukuba.ac.jp> <20080115213831.GA28333@phd.pp.ru> <08Jan15.172353pst."58696"@synergy1.parc.xerox.com> <478D5ED4.4060208@cheimes.de> <08Jan15.182444pst."58696"@synergy1.parc.xerox.com> Message-ID: <478D7B7D.2060701@cheimes.de> Bill Janssen wrote: > Sure, but it's clearly where this should be on an OS X system, by > default. And I'm sure there's a different "best place" on Windows > (for instance, all of our accounts are network roaming accounts, and > you don't want to put anything in ~). And there are probably various > right places for various flavors of Linux. > > Any PEP on this that doesn't take these OS-specific differences into > account probably isn't worth reading, IMO. Your comment doesn't help at all. In fact my proposal uses different "best places" for Windows (%APPDATA%), Mac (~/Library/Python) and Unix (~/.local). I can't comment on OS2, Ricos and VAX because I've zero experience with the platforms. The right place for a platform is the place we decide is right. As long as the directory is somewhere inside the home directory of a user and the directory doesn't show up in the user data folder (hence APPDATA on Windows and a dot directory on Unix) I consider any sensible directory as the right one. Do you think APPDATA. ~/Library/Python or ~/.local are not sensible? Please stop bitching and bring in some useful, constructive criticism and use cases not covered by my PEP. Have you read the PEP at all? Christian From lists at cheimes.de Wed Jan 16 04:40:21 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 16 Jan 2008 04:40:21 +0100 Subject: [Python-Dev] PEP: Post import hooks In-Reply-To: <20080116014256.C33DF3A4077@sparrow.telecommunity.com> References: <20080110043258.BEBAB3A40A9@sparrow.telecommunity.com> <20080110223235.13D5B3A406C@sparrow.telecommunity.com> <4786A579.2070605@cheimes.de> <20080111010557.63A2B3A406C@sparrow.telecommunity.com> <4787DB58.5010900@cheimes.de> <20080115173311.6FA9F3A4077@sparrow.telecommunity.com> <478D222C.2050904@cheimes.de> <20080115220443.AE18B3A4077@sparrow.telecommunity.com> <478D5DAF.8060709@cheimes.de> <20080116014256.C33DF3A4077@sparrow.telecommunity.com> Message-ID: <478D7CA5.9030200@cheimes.de> Phillip J. Eby wrote: > I guess it's not right then. ;-) Though I shouldn't make fun, since it > turns out that my code sketch was not a correct translation of > peak.util.imports. (See below.) *grrrrr* I spent more than hour to find my error ... > That is, module.__notified__ has to be set *before* the recursive > notification call. This effectively happens in peak.util.imports now, > except that __notified__ isn't an explicit attribute, just a side effect > of other module state changes. It's done. Your proposed test cases passes together with my tests. The ref leak tests don't show a single missing reference. Christian From pje at telecommunity.com Wed Jan 16 05:21:20 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 15 Jan 2008 23:21:20 -0500 Subject: [Python-Dev] PEP: Post import hooks In-Reply-To: <478D7CA5.9030200@cheimes.de> References: <20080110043258.BEBAB3A40A9@sparrow.telecommunity.com> <20080110223235.13D5B3A406C@sparrow.telecommunity.com> <4786A579.2070605@cheimes.de> <20080111010557.63A2B3A406C@sparrow.telecommunity.com> <4787DB58.5010900@cheimes.de> <20080115173311.6FA9F3A4077@sparrow.telecommunity.com> <478D222C.2050904@cheimes.de> <20080115220443.AE18B3A4077@sparrow.telecommunity.com> <478D5DAF.8060709@cheimes.de> <20080116014256.C33DF3A4077@sparrow.telecommunity.com> <478D7CA5.9030200@cheimes.de> Message-ID: <20080116042120.C52EC3A4077@sparrow.telecommunity.com> At 04:40 AM 1/16/2008 +0100, Christian Heimes wrote: >Phillip J. Eby wrote: > > I guess it's not right then. ;-) Though I shouldn't make fun, since it > > turns out that my code sketch was not a correct translation of > > peak.util.imports. (See below.) > >*grrrrr* I spent more than hour to find my error ... Sorry about that - as I said, __notified__ is very much an implicit thing in peak.util.imports. And I believe I've also mentioned a lot of times how hard it is to get this stuff right... :) > > That is, module.__notified__ has to be set *before* the recursive > > notification call. This effectively happens in peak.util.imports now, > > except that __notified__ isn't an explicit attribute, just a side effect > > of other module state changes. > >It's done. Your proposed test cases passes together with my tests. The >ref leak tests don't show a single missing reference. Congrats! Now all we need to do is get the authors of other lazy import/export/whatever systems to chime in with whatever additional invariants *they* might need... ;-) From lists at cheimes.de Wed Jan 16 06:24:02 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 16 Jan 2008 06:24:02 +0100 Subject: [Python-Dev] PEP: Post import hooks In-Reply-To: <20080116042120.C52EC3A4077@sparrow.telecommunity.com> References: <20080110043258.BEBAB3A40A9@sparrow.telecommunity.com> <20080110223235.13D5B3A406C@sparrow.telecommunity.com> <4786A579.2070605@cheimes.de> <20080111010557.63A2B3A406C@sparrow.telecommunity.com> <4787DB58.5010900@cheimes.de> <20080115173311.6FA9F3A4077@sparrow.telecommunity.com> <478D222C.2050904@cheimes.de> <20080115220443.AE18B3A4077@sparrow.telecommunity.com> <478D5DAF.8060709@cheimes.de> <20080116014256.C33DF3A4077@sparrow.telecommunity.com> <478D7CA5.9030200@cheimes.de> <20080116042120.C52EC3A4077@sparrow.telecommunity.com> Message-ID: <478D94F2.1030608@cheimes.de> Phillip J. Eby wrote: > Sorry about that - as I said, __notified__ is very much an implicit > thing in peak.util.imports. > > And I believe I've also mentioned a lot of times how hard it is to get > this stuff right... :) *hehe* Indeed, you did! I should have been warned. :) > Congrats! Now all we need to do is get the authors of other lazy > import/export/whatever systems to chime in with whatever additional > invariants *they* might need... ;-) Oh, no! You are kidding me ... But yes that's "all" we need to do. And I've to update the patch and explain why the implementation does all the crazy stuff. Christian From skip at pobox.com Wed Jan 16 18:42:53 2008 From: skip at pobox.com (skip at pobox.com) Date: Wed, 16 Jan 2008 11:42:53 -0600 Subject: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57 In-Reply-To: <08Jan15.182444pst."58696"@synergy1.parc.xerox.com> References: <20080115201812.GA25987@phd.pp.ru> <87lk6q3hdt.fsf@uwakimon.sk.tsukuba.ac.jp> <20080115213831.GA28333@phd.pp.ru> <08Jan15.172353pst."58696"@synergy1.parc.xerox.com> <478D5ED4.4060208@cheimes.de> <08Jan15.182444pst."58696"@synergy1.parc.xerox.com> Message-ID: <18318.16925.294977.983999@montanaro.dyndns.org> >> ~/Library/ is a Mac OS X thing. Bill> Sure, but it's clearly where this should be on an OS X system, by Bill> default. I think only for stuff that is a Mac-ish GUI app type of thing and/or that plays with Mac's distinct APIs (Carbon, Cocoa, whatever). Would you install, for example, a personal version of gcc there? I wouldn't, but that's just me. Skip From janssen at parc.com Wed Jan 16 19:24:11 2008 From: janssen at parc.com (Bill Janssen) Date: Wed, 16 Jan 2008 10:24:11 PST Subject: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57 In-Reply-To: <18318.16925.294977.983999@montanaro.dyndns.org> References: <20080115201812.GA25987@phd.pp.ru> <87lk6q3hdt.fsf@uwakimon.sk.tsukuba.ac.jp> <20080115213831.GA28333@phd.pp.ru> <08Jan15.172353pst."58696"@synergy1.parc.xerox.com> <478D5ED4.4060208@cheimes.de> <08Jan15.182444pst."58696"@synergy1.parc.xerox.com> <18318.16925.294977.983999@montanaro.dyndns.org> Message-ID: <08Jan16.102415pst."58696"@synergy1.parc.xerox.com> > >> ~/Library/ is a Mac OS X thing. > > Bill> Sure, but it's clearly where this should be on an OS X system, by > Bill> default. > > I think only for stuff that is a Mac-ish GUI app type of thing and/or that > plays with Mac's distinct APIs (Carbon, Cocoa, whatever). Would you > install, for example, a personal version of gcc there? I wouldn't, but > that's just me. Is that really the question? I'd agree with you there. But if you wanted a standard per-user place where gcc would look for custom filters of some sort, on OS X, I'd think it would be appropriate. Which makes more sense, to me, as an analogue of a per-user site-packages directory. Bill From george.sakkis at gmail.com Wed Jan 16 19:50:47 2008 From: george.sakkis at gmail.com (George Sakkis) Date: Wed, 16 Jan 2008 13:50:47 -0500 Subject: [Python-Dev] SyntaxError: 'import *' not allowed with 'from .' Message-ID: <91ad5bf80801161050x16d7765ar7ee518bab52931c0@mail.gmail.com> I posted this on c.l.py but I didn't get a definite answer so I am asking again here: is it documented anywhere that "from .relative.module import *' is syntax error? Unless I missed it, PEP 328 doesn't mention anything about it. Also, while I understand the general caveats and warnings against "import *", at first glance it seems inconsistent to disallow it for relative imports while it is valid for absolute. Are there any particular reasons that allowing relative '*' imports is harder to implement, more fragile or generally worse in some way ? George From stephen at xemacs.org Wed Jan 16 22:12:52 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 17 Jan 2008 06:12:52 +0900 Subject: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57 In-Reply-To: <08Jan16.102415pst."58696"@synergy1.parc.xerox.com> References: <20080115201812.GA25987@phd.pp.ru> <87lk6q3hdt.fsf@uwakimon.sk.tsukuba.ac.jp> <20080115213831.GA28333@phd.pp.ru> <08Jan15.172353pst."58696"@synergy1.parc.xerox.com> <478D5ED4.4060208@cheimes.de> <08Jan15.182444pst."58696"@synergy1.parc.xerox.com> <18318.16925.294977.983999@montanaro.dyndns.org> <08Jan16.102415pst."58696"@synergy1.parc.xerox.com> Message-ID: <87r6gh325n.fsf@uwakimon.sk.tsukuba.ac.jp> Bill Janssen writes: > > >> ~/Library/ is a Mac OS X thing. > > > > Bill> Sure, but it's clearly where this should be on an OS X system, by > > Bill> default. > > [etc.] > [tocatta and fugue ad lib] Doesn't Apple publish standards for this? They do for everything else, it seems. From titus at caltech.edu Wed Jan 16 22:16:53 2008 From: titus at caltech.edu (Titus Brown) Date: Wed, 16 Jan 2008 13:16:53 -0800 Subject: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57 In-Reply-To: <87r6gh325n.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20080115201812.GA25987@phd.pp.ru> <87lk6q3hdt.fsf@uwakimon.sk.tsukuba.ac.jp> <20080115213831.GA28333@phd.pp.ru> <08Jan15.172353pst."58696"@synergy1.parc.xerox.com> <478D5ED4.4060208@cheimes.de> <08Jan15.182444pst."58696"@synergy1.parc.xerox.com> <18318.16925.294977.983999@montanaro.dyndns.org> <08Jan16.102415pst."58696"@synergy1.parc.xerox.com> <87r6gh325n.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20080116211653.GA9862@caltech.edu> On Thu, Jan 17, 2008 at 06:12:52AM +0900, Stephen J. Turnbull wrote: -> Bill Janssen writes: -> > > >> ~/Library/ is a Mac OS X thing. -> > > -> > > Bill> Sure, but it's clearly where this should be on an OS X system, by -> > > Bill> default. -> -> > > [etc.] -> -> > [tocatta and fugue ad lib] -> -> Doesn't Apple publish standards for this? They do for everything -> else, it seems. Yep -- finally found it: http://www.codinghorror.com/blog/archives/001032.html http://developer.apple.com/documentation/MacOSX/Conceptual/BPFileSystem/Articles/WhereToPutFiles.html --titus From lists at cheimes.de Thu Jan 17 08:55:51 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 17 Jan 2008 08:55:51 +0100 Subject: [Python-Dev] PEP 370, open questions Message-ID: <478F0A07.5040406@cheimes.de> The PEP 370 (http://www.python.org/dev/peps/pep-0370) "per user site packages directory" has several open questions: * Are the directories for Windows, Mac and Unix fine? * Mac: Should framework and non-framework builds of Python use the same directories? * The patch also adds a usecustomize hook to site. Is it useful and should it stay? * Should the site package directory also be ignored if process gid != effective gid? * Should the Windows installer add %APPDATA%/Python/Scripts to PATH? * Should the base directory be configurable with an environment variable like PYTHONUSERHOME? A CC of the mail goes to the authors of setuptools, virtual python, working env and virtual env. What's your opinion on the PEP? Do you have some input and advice for me? Christian From p.f.moore at gmail.com Thu Jan 17 09:26:29 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 17 Jan 2008 08:26:29 +0000 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <478F0A07.5040406@cheimes.de> References: <478F0A07.5040406@cheimes.de> Message-ID: <79990c6b0801170026x3386aa71w76092f06240bf5b0@mail.gmail.com> On 17/01/2008, Christian Heimes wrote: > * Should the Windows installer add %APPDATA%/Python/Scripts to PATH? The Windows installers currently do not add the main Python\Scripts directory to PATH, so they shouldn't add this one either. Paul From glyph at divmod.com Thu Jan 17 09:40:04 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Thu, 17 Jan 2008 08:40:04 -0000 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <478F0A07.5040406@cheimes.de> References: <478F0A07.5040406@cheimes.de> Message-ID: <20080117084004.21558.1005254566.divmod.xquotient.4836@joule.divmod.com> On 07:55 am, lists at cheimes.de wrote: >A CC of the mail goes to the authors of setuptools, virtual python, >working env and virtual env. What's your opinion on the PEP? Do you >have >some input and advice for me? I wrote a similar tool called Combinator (http://divmod.org/trac/wiki/DivmodCombinator) which Divmod uses quite intensively. I think a significant portion of the Twisted community also uses it for development. Luckily you don't need to CC me specially as I'm here already ;-). One of the features it provides is a user site-packages directory much in the same style that the PEP proposes, so I'll comment on my experiences maintaining it. >The PEP 370 (http://www.python.org/dev/peps/pep-0370) "per user site >packages directory" has several open questions: > >* Are the directories for Windows, Mac and Unix fine? >* Mac: Should framework and non-framework builds of Python use > the same directories? One of the problems we've encountered over and over again with Combinator is that MacOS has a dual personality. Its personality is not just an issue of framework vs. non-framework build, but a fundamental question of "is this platform a UNIX or is it not a UNIX". An example: Let's say you have a GNU autotools project in C, which we'll call "Xxx", and a set of Python bindings, PyXxx. Combinator deals with this by using ~/.local, and providing scripts to set up PATH and LD_LIBRARY_PATH to point to ~/.local/bin and ~/.local/lib, respectively. I'm not suggesting that Python come with such a tool (although it might be handy, at least until this convention catches on with distributors), but it should at least make it clear how one would achieve the following straightforwardly: cd xxx-0.2.4 ./configure --prefix ~/.local make make install cd ../pyxxx-0.0.1 python setup.py install --prefix ~/.local Using Combinator, the user is now fully set up to "import xxx". But only if they're using Linux, because I made the same mistake (which we will probably be correcting at some point soon) of using Python's *existing* per-user site-packages directory of ~/Library/Python on the mac, and not adding ~/.local. On the Mac, our user now has a problem: given that ~/Library/Python/ doesn't follow the /usr or /usr/local style filesystem layout, autotools-based projects will not build their libraries in the right places using a single command-line option. (I guess maybe you could do something with --bindir and --sbindir and --exec-prefix or whatever, I don't know. I tend to get bored and wander off when I have to pass more than 3 options to a configure script.) A user familiar with the Mac alone might know what to do at this point (to be honest, I don't!), but I do know that people familiar with both platforms are confused by this apparently gratuitous inconsistency. They follow familiar procedure on a Linux box, it works great, then they do the same thing on a Mac (with the same shell, an apparently similar version of Python) and it doesn't. Keep in mind that "./configure --prefix" is familiar procedure from a lot more places than Combinator. For example, on shared hosting where I didn't have root, I've used this same trick without Combinator, building Python itself with --prefix ~/.local and editing .bashrc to modify the appropriate env vars. My suggestion would be to have *both* ~/.local *and* ~/Library/Python be honored on the Mac, because there really isn't much harm in doing so. Perhaps it would make sense for non-framework builds to not honor ~/Library/Python, but I am pretty certain, based on experience fielding requests from confused users, that framework builds should still honor ~/.local/lib/python.../site-packages. Windows has this problem less because everything has to be done so completely differently. In any event, the really important thing from my perspective is that the PEP should explain how this very common use-case for per-user installation of libraries can be accomplished on each of the "big three" platforms. This explanation should be put somewhere that users can easily find it when they are building libraries. I don't know what the "right way" to do this on Windows is though, so I can't offer much help there. Something to do with MinGW and intense suffering, I would guess. >* The patch also adds a usecustomize hook to site. Is it useful and > should it stay? Should this be "usercustomize"? I thought it was a typo but I see the same typo in the PEP. I have often wished for something like this for debugging, and it might have other uses, but there should be a caution against abuse :). From ronaldoussoren at mac.com Thu Jan 17 09:58:45 2008 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Thu, 17 Jan 2008 09:58:45 +0100 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <478F0A07.5040406@cheimes.de> References: <478F0A07.5040406@cheimes.de> Message-ID: <71CF2CC0-1D3C-4C02-9B1F-DEB299FA2145@mac.com> On 17 Jan, 2008, at 8:55, Christian Heimes wrote: > The PEP 370 (http://www.python.org/dev/peps/pep-0370) "per user site > packages directory" has several open questions: > > * Are the directories for Windows, Mac and Unix fine? The Mac directories look fine to me. Is it worthwhile to note in the PEP that the directories for the Mac are already used in Python 2.5? > > * Mac: Should framework and non-framework builds of Python use > the same directories? Yes, because that makes supporting users easier and keeps things nice and simple (no need to ask which kind of Python they installed to determine where their user directory is). Distutils should already link extensions in a way that allows you to have a python2.6 unix- build and framework-build that share the same site-packages directory. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20080117/0fdd4064/attachment-0001.bin From lists at cheimes.de Thu Jan 17 10:41:35 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 17 Jan 2008 10:41:35 +0100 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <71CF2CC0-1D3C-4C02-9B1F-DEB299FA2145@mac.com> References: <478F0A07.5040406@cheimes.de> <71CF2CC0-1D3C-4C02-9B1F-DEB299FA2145@mac.com> Message-ID: <478F22CF.8090403@cheimes.de> Ronald Oussoren wrote: > The Mac directories look fine to me. > > Is it worthwhile to note in the PEP that the directories for the Mac are > already used in Python 2.5? Good point! >> * Mac: Should framework and non-framework builds of Python use >> the same directories? > > Yes, because that makes supporting users easier and keeps things nice > and simple (no need to ask which kind of Python they installed to > determine where their user directory is). Distutils should already link > extensions in a way that allows you to have a python2.6 unix-build and > framework-build that share the same site-packages directory. Have you read glyph's posting? He brought up some good points why Python on Mac should use ~/.local. Framework builds have an additional import directory in ~/Library/Python Christian From mhammond at skippinet.com.au Thu Jan 17 11:16:41 2008 From: mhammond at skippinet.com.au (Mark Hammond) Date: Thu, 17 Jan 2008 21:16:41 +1100 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <478F0A07.5040406@cheimes.de> References: <478F0A07.5040406@cheimes.de> Message-ID: <11de01c858f2$15e0c0c0$41a24240$@com.au> > * Are the directories for Windows, Mac and Unix fine? Regarding Windows, I personally think its OK (but I also personally think the status-quo is OK too). As has already been mentioned in this thread, Windows actually provides an API you can call to determine various "well known" folders. I assume the PEP literally means "the environment variable APPDATA" instead of the result of SHGetSpecialFolderPath(CSIDL_APPDATA); it might be worth mentioning that (and ideally *why* the env-var is preferred). Similarly, a justification for why CSIDL_APPDATA/%APPDATA% is preferred over CSIDL_LOCAL_APPDATA - ie, why we are suggesting Python is a "roaming" application (eg, this implies that in certain Unix environments, it is likely that ~/.local/ will follow users as they log into different machines on the network - I've no idea how true that is) > * Should the Windows installer add %APPDATA%/Python/Scripts to PATH? I'd recommend not; IIUC, Python's main binary directory isn't added to the path, but even if it was, it will only be useful if .py is also on PATHEXT. Either way though, I'd suggest that particular decision is outside the scope of the PEP and a question for the windows installer assuming the PEP be accepted. Maybe some general hand-waving along the lines of "env vars on Windows are left to the installer, where such global issues are considered in a wider context" would do :) Cheers, Mark From lists at cheimes.de Thu Jan 17 12:08:26 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 17 Jan 2008 12:08:26 +0100 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <11de01c858f2$15e0c0c0$41a24240$@com.au> References: <478F0A07.5040406@cheimes.de> <11de01c858f2$15e0c0c0$41a24240$@com.au> Message-ID: <478F372A.5010402@cheimes.de> Mark Hammond wrote: >> * Are the directories for Windows, Mac and Unix fine? > > Regarding Windows, I personally think its OK (but I also personally think > the status-quo is OK too). As has already been mentioned in this thread, > Windows actually provides an API you can call to determine various "well > known" folders. I assume the PEP literally means "the environment variable > APPDATA" instead of the result of SHGetSpecialFolderPath(CSIDL_APPDATA); it > might be worth mentioning that (and ideally *why* the env-var is preferred). > Similarly, a justification for why CSIDL_APPDATA/%APPDATA% is preferred over > CSIDL_LOCAL_APPDATA - ie, why we are suggesting Python is a "roaming" > application (eg, this implies that in certain Unix environments, it is > likely that ~/.local/ will follow users as they log into different machines > on the network - I've no idea how true that is) Isn't SHGetSpecialFolderPath() XP and newer only? Python 2.6 is going to support 2000 and newer. I've written a patch for the ntpath module which adds a getshellfolders() function. The function uses SHGetFolderPathW to build a dict of id to path mappings. The patch is available at http://bugs.python.org/issue1763. Maybe you have some time to review it? As far as I understand and remember the Windows API SHGetFolderPath() and friends are retrieving the information from the registry. The shell folder keys are (IIRC) REG_EXPAND_SZ keys which are expanded with the function ExpandEnvironmentStrings(). So at the end CSIDL_APPDATA uses the %APPDATA% env var -- unless I'm totally mistaken. Or do you suggest that I'm using the win32 API calls instead of the env var? Point taken! I'll justify why I view Python as a roaming app. All company and university Linux boxes I've used in the past had exported $HOME via NFS. So ~/.local is roamed. > I'd recommend not; IIUC, Python's main binary directory isn't added to the > path, but even if it was, it will only be useful if .py is also on PATHEXT. > Either way though, I'd suggest that particular decision is outside the scope > of the PEP and a question for the windows installer assuming the PEP be > accepted. Maybe some general hand-waving along the lines of "env vars on > Windows are left to the installer, where such global issues are considered > in a wider context" would do :) *hehe* I love your idea. It should be trivial to write a view lines of Python code which modify PATH in HKCU. I'm going to write a simple example and drop it into Tools/scripts/. Christian PS: Have you tested the new PCbuild directory for VS 2008? Martin said that you know how to create a new set of base addresses for pre-linking. dllbase_nt.txt is probably outdated. From mail at timgolden.me.uk Thu Jan 17 12:30:36 2008 From: mail at timgolden.me.uk (Tim Golden) Date: Thu, 17 Jan 2008 11:30:36 +0000 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <478F372A.5010402@cheimes.de> References: <478F0A07.5040406@cheimes.de> <11de01c858f2$15e0c0c0$41a24240$@com.au> <478F372A.5010402@cheimes.de> Message-ID: <478F3C5C.1040009@timgolden.me.uk> Christian Heimes wrote: > I'll justify why I view Python as a roaming app. All > company and university Linux boxes I've used in the past had exported > $HOME via NFS. So ~/.local is roamed. I think there is a slight subtlety here: the exported NFS $HOME is more equivalent to the HOMEDRIVE/HOMEPATH which comes from the HOME directory in NT/AD. ie it is simply a share pointed to by a drive letter available wherever the user logs on. Roaming profiles actually *copy* the data from your network versions of USERPROFILE to the local machine [need to check they still do this; a while since I've administered this kind of setup]. The difference therefore is that installing large quantities of Python modules into a roaming profile path will involve their being copied to-and-fro on logon/logoff which, historically at least, was a known cause of slow startup/shutdown. I'll try to confirm if this is still the case. My own feeling was to use ~/.local on Windows as well (or whatever is agreed on for *nix) and let os.path.expanduser handle it. But... from r54364, USERPROFILE takes priority over HOMEDRIVE/HOMEPATH. Obviously Georg had some rationale there although a quick search of python-dev doesn't throw anything up. If we decide anything else here, though, we would seem to be somewhat in conflict with that interpretation of home/~ as USERPROFILE. TJG From mhammond at skippinet.com.au Thu Jan 17 12:31:26 2008 From: mhammond at skippinet.com.au (Mark Hammond) Date: Thu, 17 Jan 2008 22:31:26 +1100 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <478F372A.5010402@cheimes.de> References: <478F0A07.5040406@cheimes.de> <11de01c858f2$15e0c0c0$41a24240$@com.au> <478F372A.5010402@cheimes.de> Message-ID: <11fa01c858fc$8248ef80$86dace80$@com.au> > Isn't SHGetSpecialFolderPath() XP and newer only? MSDN documents: Minimum operating systems Windows 2000, Windows NT 4.0 with Internet Explorer 4.0, Windows 98, Windows 95 with Internet Explorer 4.0 > The patch is available at > http://bugs.python.org/issue1763. Maybe you have some time to review > it? Sure - I'll do my best to do that in the next 24 hours - feel free to prod if I don't :) > As far as I understand and remember the Windows API SHGetFolderPath() > and friends are retrieving the information from the registry. The shell > folder keys are (IIRC) REG_EXPAND_SZ keys which are expanded with the > function ExpandEnvironmentStrings(). So at the end CSIDL_APPDATA uses > the %APPDATA% env var -- unless I'm totally mistaken. I'm not sure exactly how that is implemented, and I'm not necessarily arguing you must call that function (eg, it's not hard to argue the env-var may be preferred, as it offers an easy override) - it's just that the PEP shouldn't be silent about the issue. Something I meant to mention was that supporting multiple "per-user" directories could be considered - that way CSIDL_LOCAL_APPDATA could be added later. YAGNI is easy to justify here, but now is the time to consider such issues. > PS: Have you tested the new PCbuild directory for VS 2008? Martin said > that you know how to create a new set of base addresses for pre- > linking. > dllbase_nt.txt is probably outdated. I'm afaid I haven't. The base-addresses are fairly easily tested - just import each and every module listed while running under the debugger - if you see messages in the output window telling you about relocation, you have a problem :) pywin32 is screaming for a new release (it didn't manage a single one in 2007!) and that is likely to happen before I get to play with the new-fangled stuff Mark From exarkun at divmod.com Thu Jan 17 12:35:37 2008 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Thu, 17 Jan 2008 06:35:37 -0500 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <478F0A07.5040406@cheimes.de> Message-ID: <20080117113537.31425.290988528.divmod.quotient.5897@ohm> On Thu, 17 Jan 2008 08:55:51 +0100, Christian Heimes wrote: > >* Should the site package directory also be ignored if process > gid != effective gid? If it should, I think the PEP should explain the attack this defends against in more detail. The current brief mention of "security issues" is a bit hand-wavey. For example, what is the relationship between security, this feature, and the PYTHONPATH environment variable? Isn't the attack of putting malicious code into a user site-packages directory the same as the attack of putting it into a directory in PYTHONPATH? Jean-Paul From lists at cheimes.de Thu Jan 17 13:09:34 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 17 Jan 2008 13:09:34 +0100 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <20080117113537.31425.290988528.divmod.quotient.5897@ohm> References: <478F0A07.5040406@cheimes.de> <20080117113537.31425.290988528.divmod.quotient.5897@ohm> Message-ID: Jean-Paul Calderone wrote: > If it should, I think the PEP should explain the attack this defends > against in more detail. The current brief mention of "security issues" > is a bit hand-wavey. For example, what is the relationship between > security, this feature, and the PYTHONPATH environment variable? Isn't > the attack of putting malicious code into a user site-packages directory > the same as the attack of putting it into a directory in PYTHONPATH? The PYTHONPATH env var has the same security implications. However a user has multiple ways to avoid problems. For example the user can use the -E flag or set up sudo to ignore the environment. The uid and gid tests aren't really required. They just provide an extra safety net if a user forgets to add the -s flag to a suid app. Christian From lists at cheimes.de Thu Jan 17 13:15:58 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 17 Jan 2008 13:15:58 +0100 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <478F3C5C.1040009@timgolden.me.uk> References: <478F0A07.5040406@cheimes.de> <11de01c858f2$15e0c0c0$41a24240$@com.au> <478F372A.5010402@cheimes.de> <478F3C5C.1040009@timgolden.me.uk> Message-ID: <478F46FE.5060401@cheimes.de> Tim Golden wrote: > Christian Heimes wrote: >> I'll justify why I view Python as a roaming app. All >> company and university Linux boxes I've used in the past had exported >> $HOME via NFS. So ~/.local is roamed. > > I think there is a slight subtlety here: the exported NFS > $HOME is more equivalent to the HOMEDRIVE/HOMEPATH which > comes from the HOME directory in NT/AD. ie it is simply > a share pointed to by a drive letter available wherever > the user logs on. Roaming profiles actually *copy* the > data from your network versions of USERPROFILE to the > local machine [need to check they still do this; a while > since I've administered this kind of setup]. > > The difference therefore is that installing large quantities > of Python modules into a roaming profile path will involve > their being copied to-and-fro on logon/logoff which, historically > at least, was a known cause of slow startup/shutdown. I'll > try to confirm if this is still the case. I can't comment on the matter. I've not used roaming user profiles on Windows for more than five years. Can someone with more experience shed some like on the matter? > My own feeling was to use ~/.local on Windows as well > (or whatever is agreed on for *nix) and let os.path.expanduser > handle it. But... from r54364, USERPROFILE takes priority > over HOMEDRIVE/HOMEPATH. Obviously Georg had some rationale > there although a quick search of python-dev doesn't throw > anything up. If we decide anything else here, though, we > would seem to be somewhat in conflict with that interpretation > of home/~ as USERPROFILE. The PEP already explains why I don't want to write to ~/ on Windows. I had considered it shortly but MSDN advices against it. Is %USERPROFILE% not equal to %HOMEDRIVE%%HOMEPATH%? Maybe Gregor did a mistake because he thought that both point to the same location. Christian From ronaldoussoren at mac.com Thu Jan 17 13:02:20 2008 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Thu, 17 Jan 2008 13:02:20 +0100 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <20080117084004.21558.1005254566.divmod.xquotient.4836@joule.divmod.com> References: <478F0A07.5040406@cheimes.de> <20080117084004.21558.1005254566.divmod.xquotient.4836@joule.divmod.com> Message-ID: <64254CC6-8412-4343-903D-D117DCFB0117@mac.com> On 17 Jan, 2008, at 9:40, glyph at divmod.com wrote: > > On 07:55 am, lists at cheimes.de wrote: >> > >> The PEP 370 (http://www.python.org/dev/peps/pep-0370) "per user site >> packages directory" has several open questions: >> >> * Are the directories for Windows, Mac and Unix fine? >> * Mac: Should framework and non-framework builds of Python use >> the same directories? > > One of the problems we've encountered over and over again with > Combinator is that MacOS has a dual personality. Its personality is > not > just an issue of framework vs. non-framework build, but a fundamental > question of "is this platform a UNIX or is it not a UNIX". IMHO the question is wrong. MacOS X is a UNIX platform (and Apple has the paperwork to prove it), but at the same time it has a non- traditional filesystem layout. The "darwin" system at the foundation of OSX is a classic unix system, with the usual filesystem layout, but the bits that make OSX OSX use a completely diffent filesystem layout. The framework build of Python definitly targets the upper layer of the OSX stack, not just the Unix core. This sometimes causes confusion, but mostly from developers wandering over from a Linux system that complain that OSX isn't Linux. Note that even Darwin is not Linux, there are several differences that cause problems for naive porters. Two of those: Darwin uses different binary formats for shared libraries and plugins; the darwin linker hardcodes the full path to shared libraries into executables (without a runtime search path). And that's only the start of fun, universal (fat) binaries make live even more interesting. > > > An example: Let's say you have a GNU autotools project in C, which > we'll > call "Xxx", and a set of Python bindings, PyXxx. Combinator deals > with > this by using ~/.local, and providing scripts to set up PATH and > LD_LIBRARY_PATH to point to ~/.local/bin and ~/.local/lib, > respectively. ~/Library/Combinator would be a better installation root on OSX, that location fits better into guidelines from Apple and also avoids completely hiding the Combinator data from the user. > > I'm not suggesting that Python come with such a tool (although it > might > be handy, at least until this convention catches on with > distributors), > but it should at least make it clear how one would achieve the > following > straightforwardly: This is probably off-topic for python-dev, but how is combinator different from zc.buildout and virtualenv? I'm particularly liking virtualenv at the moment as it allows me to have seperate trees for projects without having several full python installations. > > > cd xxx-0.2.4 > ./configure --prefix ~/.local > make > make install > cd ../pyxxx-0.0.1 > python setup.py install --prefix ~/.local > > Using Combinator, the user is now fully set up to "import xxx". But > only if they're using Linux, because I made the same mistake (which we > will probably be correcting at some point soon) of using Python's > *existing* per-user site-packages directory of ~/Library/Python on the > mac, and not adding ~/.local. Why? Just because users can't remember on which platform they are developing ;-)? That said, there's a libpython.a file in the framework build for basicly that reason: enough users complained about there not being a python library they could link to that it was easier to add a symlink then trying to educate everyone. > > > On the Mac, our user now has a problem: given that ~/Library/Python/ > doesn't follow the /usr or /usr/local style filesystem layout, > autotools-based projects will not build their libraries in the right > places using a single command-line option. (I guess maybe you could > do > something with --bindir and --sbindir and --exec-prefix or whatever, I > don't know. I tend to get bored and wander off when I have to pass > more > than 3 options to a configure script.) You could teach Combinator about running configure scripts ;-). > A user familiar with the Mac alone might know what to do at this point > (to be honest, I don't!), but I do know that people familiar with both > platforms are confused by this apparently gratuitous inconsistency. This is not a gratuitous inconsistency. The Mac developers (mostly Jack Jansen, but I was involved as well) decided that following the platform conventions is a good thing. > > They follow familiar procedure on a Linux box, it works great, then > they > do the same thing on a Mac (with the same shell, an apparently similar > version of Python) and it doesn't. Keep in mind that "./configure > --prefix" is familiar procedure from a lot more places than > Combinator. > For example, on shared hosting where I didn't have root, I've used > this > same trick without Combinator, building Python itself with --prefix > ~/.local and editing .bashrc to modify the appropriate env vars. > > My suggestion would be to have *both* ~/.local *and* ~/Library/ > Python be > honored on the Mac, because there really isn't much harm in doing so. > Perhaps it would make sense for non-framework builds to not honor > ~/Library/Python, but I am pretty certain, based on experience > fielding > requests from confused users, that framework builds should still honor > ~/.local/lib/python.../site-packages. As long as the default installation location is still ~/Library/Python/ X.Y for framework builds I wouldn't mind too much. > > Windows has this problem less because everything has to be done so > completely differently. > > In any event, the really important thing from my perspective is that > the > PEP should explain how this very common use-case for per-user > installation of libraries can be accomplished on each of the "big > three" > platforms. This explanation should be put somewhere that users can > easily find it when they are building libraries. I don't know if the PEP should mention this, but I do agree that this should be in the Python documentation. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20080117/b3bcf94f/attachment-0001.bin From fuzzyman at voidspace.org.uk Thu Jan 17 13:21:22 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 17 Jan 2008 12:21:22 +0000 Subject: [Python-Dev] [python] Re: PEP 370, open questions In-Reply-To: <478F46FE.5060401@cheimes.de> References: <478F0A07.5040406@cheimes.de> <11de01c858f2$15e0c0c0$41a24240$@com.au> <478F372A.5010402@cheimes.de> <478F3C5C.1040009@timgolden.me.uk> <478F46FE.5060401@cheimes.de> Message-ID: <478F4842.90502@voidspace.org.uk> Christian Heimes wrote: > Tim Golden wrote: > >> Christian Heimes wrote: >> >>> I'll justify why I view Python as a roaming app. All >>> company and university Linux boxes I've used in the past had exported >>> $HOME via NFS. So ~/.local is roamed. >>> >> I think there is a slight subtlety here: the exported NFS >> $HOME is more equivalent to the HOMEDRIVE/HOMEPATH which >> comes from the HOME directory in NT/AD. ie it is simply >> a share pointed to by a drive letter available wherever >> the user logs on. Roaming profiles actually *copy* the >> data from your network versions of USERPROFILE to the >> local machine [need to check they still do this; a while >> since I've administered this kind of setup]. >> >> The difference therefore is that installing large quantities >> of Python modules into a roaming profile path will involve >> their being copied to-and-fro on logon/logoff which, historically >> at least, was a known cause of slow startup/shutdown. I'll >> try to confirm if this is still the case. >> > > I can't comment on the matter. I've not used roaming user profiles on > Windows for more than five years. Can someone with more experience shed > some like on the matter? > Roaming profiles still load and save the profile on logon/logoff. I think using the home directory (or subdirectory thereof) is preferable as that can *either* be on a shared drive *or* part of the roaming profile. Michael Foord From exarkun at divmod.com Thu Jan 17 13:26:09 2008 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Thu, 17 Jan 2008 07:26:09 -0500 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: Message-ID: <20080117122609.31425.1820567953.divmod.quotient.5912@ohm> On Thu, 17 Jan 2008 13:09:34 +0100, Christian Heimes wrote: >Jean-Paul Calderone wrote: >> If it should, I think the PEP should explain the attack this defends >> against in more detail. The current brief mention of "security issues" >> is a bit hand-wavey. For example, what is the relationship between >> security, this feature, and the PYTHONPATH environment variable? Isn't >> the attack of putting malicious code into a user site-packages directory >> the same as the attack of putting it into a directory in PYTHONPATH? > >The PYTHONPATH env var has the same security implications. However a >user has multiple ways to avoid problems. For example the user can use >the -E flag or set up sudo to ignore the environment. I'm not sure how sudo gets involved. sudo doesn't set the euid, it sets the uid. This is about programs with the setuid bit set. (I assume this doesn't also apply to Python programs that explicitly make use of the seteuid() call, since this will probably only be checked at interpreter startup before any Python application code has run.) > >The uid and gid tests aren't really required. They just provide an extra >safety net if a user forgets to add the -s flag to a suid app. It's not much of a safety net if PYTHONPATH still allows injection of arbitrary code. It's just needless additional complexity for no benefit. On the other hand, if all of the other mechanisms for modifying how imports work is also made to behave this way, then maybe there's a point. Jean-Paul From mail at timgolden.me.uk Thu Jan 17 13:30:53 2008 From: mail at timgolden.me.uk (Tim Golden) Date: Thu, 17 Jan 2008 12:30:53 +0000 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <478F46FE.5060401@cheimes.de> References: <478F0A07.5040406@cheimes.de> <11de01c858f2$15e0c0c0$41a24240$@com.au> <478F372A.5010402@cheimes.de> <478F3C5C.1040009@timgolden.me.uk> <478F46FE.5060401@cheimes.de> Message-ID: <478F4A7D.8090101@timgolden.me.uk> Christian Heimes wrote: > Tim Golden wrote: >> The difference therefore is that installing large quantities >> of Python modules into a roaming profile path will involve >> their being copied to-and-fro on logon/logoff which, historically >> at least, was a known cause of slow startup/shutdown. I'll >> try to confirm if this is still the case. > > I can't comment on the matter. I've not used roaming user profiles on > Windows for more than five years. Can someone with more experience shed > some like on the matter? I'll try to confirm with our support guys here. >> My own feeling was to use ~/.local on Windows as well >> (or whatever is agreed on for *nix) and let os.path.expanduser >> handle it. But... from r54364, USERPROFILE takes priority >> over HOMEDRIVE/HOMEPATH. Obviously Georg had some rationale >> there although a quick search of python-dev doesn't throw >> anything up. If we decide anything else here, though, we >> would seem to be somewhat in conflict with that interpretation >> of home/~ as USERPROFILE. > > The PEP already explains why I don't want to write to ~/ on Windows. I > had considered it shortly but MSDN advices against it. Entirely understood, but then... > Is %USERPROFILE% not equal to %HOMEDRIVE%%HOMEPATH%? No. On my machine, for example: HOMEDRIVE=H: HOMEPATH=\ HOMESHARE=\\vogbs022\it\goldent USERPROFILE=C:\Documents and Settings\goldent However, using an account without a mapped home drive then yes, they are the same. TJG From eric+python-dev at trueblade.com Thu Jan 17 13:19:57 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Thu, 17 Jan 2008 07:19:57 -0500 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <20080117084004.21558.1005254566.divmod.xquotient.4836@joule.divmod.com> References: <478F0A07.5040406@cheimes.de> <20080117084004.21558.1005254566.divmod.xquotient.4836@joule.divmod.com> Message-ID: <478F47ED.9000706@trueblade.com> glyph at divmod.com wrote: > One of the problems we've encountered over and over again with > Combinator is that MacOS has a dual personality. Its personality is not > just an issue of framework vs. non-framework build, but a fundamental > question of "is this platform a UNIX or is it not a UNIX". MacOS isn't the only platform that has this problem. I use cygwin under Windows, and I wish Python (whether or not a cygwin build) would also use ~/.local. > On the Mac, our user now has a problem: given that ~/Library/Python/ > doesn't follow the /usr or /usr/local style filesystem layout, > autotools-based projects will not build their libraries in the right > places using a single command-line option. (I guess maybe you could do > something with --bindir and --sbindir and --exec-prefix or whatever, I > don't know. I tend to get bored and wander off when I have to pass more > than 3 options to a configure script.) I guess Windows isn't as bad as MacOS in this case, because there's no standard to follow and conflict with. Everything under whatever directory we choose won't likely match other packages, so we can do whatever we want. Still, I'd prefer something under $HOME. > Windows has this problem less because everything has to be done so > completely differently. True, except that cygwin tries hard to make it look like Unix. I'd rather Python under Windows really look like Unix, but I'm probably in a minority. And yes, I do share computers (physically and through Terminal Server), so per-user local libraries would be nice. Eric. From glyph at divmod.com Thu Jan 17 14:22:12 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Thu, 17 Jan 2008 13:22:12 -0000 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <64254CC6-8412-4343-903D-D117DCFB0117@mac.com> References: <478F0A07.5040406@cheimes.de> <20080117084004.21558.1005254566.divmod.xquotient.4836@joule.divmod.com> <64254CC6-8412-4343-903D-D117DCFB0117@mac.com> Message-ID: <20080117132212.21558.1891189134.divmod.xquotient.4917@joule.divmod.com> On 12:02 pm, ronaldoussoren at mac.com wrote: >On 17 Jan, 2008, at 9:40, glyph at divmod.com wrote: >>On 07:55 am, lists at cheimes.de wrote: >The framework build of Python definitly targets the upper layer of the >OSX stack, not just the Unix core. This sometimes causes confusion, >but mostly from developers wandering over from a Linux system that >complain that OSX isn't Linux. The framework build targets both layers, as I understand it - and it sounds like you're saying it too, since it's not "just" the UNIX core. Solaris isn't Linux either, by the way. These conventions hold across far more than one operating system :). >Note that even Darwin is not Linux, there are several differences that >cause problems for naive porters. Two of those: Darwin uses different >binary formats for shared libraries and plugins; the darwin linker >hardcodes the full path to shared libraries into executables (without >a runtime search path). Distutils should take care of this distinction in Python's case, no? Xxx's autotools generate a shared library, PyXxx's setup.py generates a plugin (or "dylib", is that still what they're called these days?). >>An example: Let's say you have a GNU autotools project in C, which >>we'll >>call "Xxx", and a set of Python bindings, PyXxx. Combinator deals >>with >>this by using ~/.local, and providing scripts to set up PATH and >>LD_LIBRARY_PATH to point to ~/.local/bin and ~/.local/lib, >>respectively. > >~/Library/Combinator would be a better installation root on OSX, that >location fits better into guidelines from Apple and also avoids >completely hiding the Combinator data from the user. I disagree, but Combinator's a complex beast and has a lot of other moving parts which need to live in specific places. Its main purpose is to manage your import path to easily switch between different development branches of multiple projects, and so most of its data is already in locations that the user has specified. A key thing about ~/.local in this case is that it isn't specific to Combinator. It's any per-user installed dependency libraries for development purposes, not necessarily on Combinator-managed projects, and not even necessarily Python projects. >This is probably off-topic for python-dev, but how is combinator >different from zc.buildout and virtualenv? We are definitely veering in that direction, but it probably bears a brief description just so folks here can see how it does and doesn't apply to the PEP. zc.buildout and virtualenv are primarily heterogeneous deployment tools, with development being just a different type of deployment. They're ways of installing Python packages into an isolated, configured environment. Combinator is primarily a development tool. Although I've used it as a deployment tool (as you can use zc.buildout as a development tool) that's not really its focus. Combinator has no "installation" step for most projects. ~/.local is a special case, reserved for common unchanging dependencies that require building; most code managed by Combinator lives in ~/Projects/YourProject/trunk or ~/Projects/YourProject/branches. (I *used* to be a Mac guy. Can you tell? :-)) The idea with zc.buildout is you are installing application A which requires library X version Q, and installing application B which requires library X version M; you want to keep those separated. The idea with combinator is that you are *developing* application A, and you want to make sure that it continues working with both version Q and M, so you can easily do chbranch X releases/Q # the most common combinator command trial a.test chbranch X releases/M trial a.test It also has specific integration with subversion for creating and merging branches. chbranch will try to check out releases/Q if there isn't already such a directory present, based on svn metadata. When you create a branch to start working on a feature, your environment is automatically reconfigured to import code from that branch. When you merge a branch to trunk, it is adjusted to load code from the merged trunk again. Hopefully some day soon it will also have integration with bazaar too. >Why? Just because users can't remember on which platform they are >developing ;-)? That said, there's a libpython.a file in the framework >build for basicly that reason: enough users complained about there not >being a python library they could link to that it was easier to add a >symlink then trying to educate everyone. The system installation of Python on the mac has a nod in this direction as well. /usr/bin/python is also present, as is /usr/lib/pythonX.Y?0Cas symlinks between the two locations. >You could teach Combinator about running configure scripts ;-). Better yet, perhaps somebody should teach configure about MacOS, and about per-user installation ;). But the real question here is not what Combinator should do, but what Python should do. >This is not a gratuitous inconsistency. The Mac developers (mostly >Jack Jansen, but I was involved as well) decided that following the >platform conventions is a good thing. I agree. I'm not saying Python shouldn't follow the platform conventions of NeXTSTEP^WOpenStep^WYellowBox^WCocoa - is it still called Cococa? - but to follow both those conventions and the conventions of its other pedigree, at least until autotools knows how to properly populate ~/Library/Frameworks. >As long as the default installation location is still ~/Library/Python/ >X.Y for framework builds I wouldn't mind too much. Is there a default "installation" location? When we started this part of the discussion, it was just which paths should be on sys.path. I agree, though. I would like to propose that if the user does not have write access to the system installation location, distutils should select ~/.local or ~/Library (or %APPDATA%) as appropriate by default. The only problem with this proposal is that distutils' --prefix ~/.local will pick up on the autotools-friendly ~/.local/include and ~/.local/lib for building C extensions, whereas it's not clear where to look for where autotools might have put those things in ~/Library. That is arguably a separate issue though, and I think we should close one can of worms before opening another, if possible. >I don't know if the PEP should mention this, but I do agree that this >should be in the Python documentation. The PEP interacts with it, so it should make reference to it, but the explanation needs to go somewhere users will see it. From glyph at divmod.com Thu Jan 17 14:27:23 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Thu, 17 Jan 2008 13:27:23 -0000 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <478F47ED.9000706@trueblade.com> References: <478F0A07.5040406@cheimes.de> <20080117084004.21558.1005254566.divmod.xquotient.4836@joule.divmod.com> <478F47ED.9000706@trueblade.com> Message-ID: <20080117132723.21558.1601258577.divmod.xquotient.4926@joule.divmod.com> On 12:19 pm, eric+python-dev at trueblade.com wrote: >glyph at divmod.com wrote: >MacOS isn't the only platform that has this problem. I use cygwin >under >Windows, and I wish Python (whether or not a cygwin build) would also >use ~/.local. I would like to agree. MacOS has an advantage here though. Windows, even without cygwin, but doubly so once cygwin gets involved, has a schizophrenic idea of what "~" means. (I believe other messages in this thread have made reference to issues with expanduser.) You definitely are going to have a hard time getting non-cygwin and cygwin python to agree on the location of your home directory. I rather brutally modified my last Windows installation to get those two values to line up, and it required surgery for each user created, and caused a number of "interesting issues" with various tools writing fake cygwin UNIX paths and others thinking they were Windows drive-letter paths... >>Windows has this problem less because everything has to be done so >>completely differently. > >True, except that cygwin tries hard to make it look like Unix. I'd >rather Python under Windows really look like Unix, but I'm probably in >a >minority. And yes, I do share computers (physically and through >Terminal Server), so per-user local libraries would be nice. If you can figure out how to make this work without raising any of the bogeymen I just made reference to, I would be strongly in favor. From glyph at divmod.com Thu Jan 17 14:49:01 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Thu, 17 Jan 2008 13:49:01 -0000 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <20080117122609.31425.1820567953.divmod.quotient.5912@ohm> References: <20080117122609.31425.1820567953.divmod.quotient.5912@ohm> Message-ID: <20080117134901.21558.1528703291.divmod.xquotient.4968@joule.divmod.com> On 12:26 pm, exarkun at divmod.com wrote: >On Thu, 17 Jan 2008 13:09:34 +0100, Christian Heimes >wrote: >>The uid and gid tests aren't really required. They just provide an >>extra >>safety net if a user forgets to add the -s flag to a suid app. >It's not much of a safety net if PYTHONPATH still allows injection of >arbitrary code. It's just needless additional complexity for no >benefit. By confusing users' expectations, it may actually be *worse* to add this "safety net" than to do nothing. It should be obvious right now that tightly controlling the environment is a requirement of any suid Python code. However, talking about different behavior in the case of differing euid and uid might confuse some developers and/or administrators into thinking that Python was doing all it needed to. There's also the confusion that the value of $HOME is actually the relevant thing for controlling "user-installed" imports, not the (E)UID. I think it would be good to have a look at the security implications of this and other environment-dependent execution, including $PYTHONPATH and $PYTHONSTARTUP, in a separate PEP. It might be good to change the way some of these things work, but in either case it would be good to have an unambiguous declaration of the *expected* security properties and potential attack vectors against the Python interpreter, for both developers and system administrators. From p.f.moore at gmail.com Thu Jan 17 16:01:58 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 17 Jan 2008 15:01:58 +0000 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <478F4A7D.8090101@timgolden.me.uk> References: <478F0A07.5040406@cheimes.de> <11de01c858f2$15e0c0c0$41a24240$@com.au> <478F372A.5010402@cheimes.de> <478F3C5C.1040009@timgolden.me.uk> <478F46FE.5060401@cheimes.de> <478F4A7D.8090101@timgolden.me.uk> Message-ID: <79990c6b0801170701y10ddcf93qd3f23fd0b5c36ea@mail.gmail.com> On 17/01/2008, Tim Golden wrote: > > Is %USERPROFILE% not equal to %HOMEDRIVE%%HOMEPATH%? > > No. On my machine, for example: > > HOMEDRIVE=H: > HOMEPATH=\ > HOMESHARE=\\vogbs022\it\goldent > > USERPROFILE=C:\Documents and Settings\goldent > > However, using an account without a mapped home > drive then yes, they are the same. Given this information (and the earlier comments about roaming profiles) I would suggest that Python should use %HOMEDRIVE%HOMEPATH%. This matches os.path.expanduser('~'). My previous comments were based on my experience on a machine without a mapped home drive, so I'd missed the distinction. Yes, this does contradict the Microsoft guideline that you shouldn't write to USERPROFILE, but Microsoft themselves set HOMEDRIVE and HOMEPATH, so they vioated the guidelines, not us :-). The point here is that we want "the user's home". This is clearly %HOMEDRIVE%%HOMEPATH% on Windows, guidelines notwithstanding. Paul. From ronaldoussoren at mac.com Thu Jan 17 16:33:16 2008 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Thu, 17 Jan 2008 16:33:16 +0100 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <20080117132212.21558.1891189134.divmod.xquotient.4917@joule.divmod.com> References: <478F0A07.5040406@cheimes.de> <20080117084004.21558.1005254566.divmod.xquotient.4836@joule.divmod.com> <64254CC6-8412-4343-903D-D117DCFB0117@mac.com> <20080117132212.21558.1891189134.divmod.xquotient.4917@joule.divmod.com> Message-ID: On 17 Jan, 2008, at 14:22, glyph at divmod.com wrote: > >> As long as the default installation location is still ~/Library/ >> Python/ X.Y for framework builds I wouldn't mind too much. > > Is there a default "installation" location? When we started this > part of the discussion, it was just which paths should be on sys.path. I'm pretty sure I've seen a suggestion for a distutils options for installing into your home location (something like "python setup.py install --user"). Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20080117/58941e1b/attachment.bin From mail at timgolden.me.uk Thu Jan 17 16:36:00 2008 From: mail at timgolden.me.uk (Tim Golden) Date: Thu, 17 Jan 2008 15:36:00 +0000 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <79990c6b0801170701y10ddcf93qd3f23fd0b5c36ea@mail.gmail.com> References: <478F0A07.5040406@cheimes.de> <11de01c858f2$15e0c0c0$41a24240$@com.au> <478F372A.5010402@cheimes.de> <478F3C5C.1040009@timgolden.me.uk> <478F46FE.5060401@cheimes.de> <478F4A7D.8090101@timgolden.me.uk> <79990c6b0801170701y10ddcf93qd3f23fd0b5c36ea@mail.gmail.com> Message-ID: <478F75E0.9020002@timgolden.me.uk> Paul Moore wrote: > On 17/01/2008, Tim Golden wrote: >>> Is %USERPROFILE% not equal to %HOMEDRIVE%%HOMEPATH%? >> No. On my machine, for example: >> >> HOMEDRIVE=H: >> HOMEPATH=\ >> HOMESHARE=\\vogbs022\it\goldent >> >> USERPROFILE=C:\Documents and Settings\goldent >> >> However, using an account without a mapped home >> drive then yes, they are the same. > > Given this information (and the earlier comments about roaming > profiles) I would suggest that Python should use %HOMEDRIVE%HOMEPATH%. > This matches os.path.expanduser('~'). My previous comments were based > on my experience on a machine without a mapped home drive, so I'd > missed the distinction. > > Yes, this does contradict the Microsoft guideline that you shouldn't > write to USERPROFILE, but Microsoft themselves set HOMEDRIVE and > HOMEPATH, so they vioated the guidelines, not us :-). > > The point here is that we want "the user's home". This is clearly > %HOMEDRIVE%%HOMEPATH% on Windows, guidelines notwithstanding. A few datapoints here from WinXP SP2. I'll try Win2k when I get home this evening if no-one's done it before then. -------------- My support guys have temporarily switched me over to a roaming profile so I could check what happens. As expected, all the non-Local... folders are copied to and from the network share specified as ProfilePath on my AD entry. (In fact, our policy forbids profiles bigger than a few tens of Mb so I had to hive off my existing profile and create a new one to do this test). Ignoring Terminal Services -- which have their own profiles -- and Cygwin -- which does its own thing -- we have four situations: Users with and without mapped HOME drives; and with and without network (roaming) profiles. USERPROFILE points to c:\docu..\goldent regardless of whether it's a network-based profile or not. The difference is what happens when you log on/off. If the home drive is mapped, HOMEDRIVE/PATH point to that mapping If the home drive isn't mapped, HOMEDRIVE/PATH point to USERPROFILE. APPDATA is a non-local part of othe profile and as such is copied to and from the network share if the profile is roaming. -------------- There doesn't seem to be a clear solution. If we point to APPDATA, HOMEDRIVE/PATH or USERPROFILE we might be bulking up the content a user has to copy to and fro. And local space policies might hinder things. If we point to local settings\application data then the packages won't roam with the user. The best we can hope for is to point to HOMEDRIVE/PATH as Paul suggested (*possibly* rewiring os.path.expanduser to try that first, depending on Win2K stuff) and make sure the user is aware of what might happen if the HOMEDRIVE/PATH is really a pointer to a network-based profile. TJG From ronaldoussoren at mac.com Thu Jan 17 16:47:38 2008 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Thu, 17 Jan 2008 16:47:38 +0100 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <20080117132212.21558.1891189134.divmod.xquotient.4917@joule.divmod.com> References: <478F0A07.5040406@cheimes.de> <20080117084004.21558.1005254566.divmod.xquotient.4836@joule.divmod.com> <64254CC6-8412-4343-903D-D117DCFB0117@mac.com> <20080117132212.21558.1891189134.divmod.xquotient.4917@joule.divmod.com> Message-ID: <5808E387-56FD-428C-A8AA-5B4176C373F5@mac.com> On 17 Jan, 2008, at 14:22, glyph at divmod.com wrote: > On 12:02 pm, ronaldoussoren at mac.com wrote: >> On 17 Jan, 2008, at 9:40, glyph at divmod.com wrote: >>> On 07:55 am, lists at cheimes.de wrote: > >> The framework build of Python definitly targets the upper layer of >> the OSX stack, not just the Unix core. This sometimes causes >> confusion, but mostly from developers wandering over from a Linux >> system that complain that OSX isn't Linux. > > The framework build targets both layers, as I understand it - and it > sounds like you're saying it too, since it's not "just" the UNIX > core. Solaris isn't Linux either, by the way. These conventions > hold across far more than one operating system :). I know, but somehow most people I run into that even know that there are more unix flavors than Linux also know that while all unices are simular they are not exactly the same. > >> Note that even Darwin is not Linux, there are several differences >> that cause problems for naive porters. Two of those: Darwin uses >> different binary formats for shared libraries and plugins; the >> darwin linker hardcodes the full path to shared libraries into >> executables (without a runtime search path). > > Distutils should take care of this distinction in Python's case, no? > Xxx's autotools generate a shared library, PyXxx's setup.py > generates a plugin (or "dylib", is that still what they're called > these days?). Distutils does take care of it, but that doesn't stop users from complaining that 'cc -shared' doesn't result in a workable python extension ;-). BTW. dylibs are shared libraries, there is no common suffix for bundles. Python uses '.so' for its bundle, adding to the confusion for folks used to systems using ELF libraries ;-) > >>> An example: Let's say you have a GNU autotools project in C, which >>> we'll >>> call "Xxx", and a set of Python bindings, PyXxx. Combinator deals >>> with >>> this by using ~/.local, and providing scripts to set up PATH and >>> LD_LIBRARY_PATH to point to ~/.local/bin and ~/.local/lib, >>> respectively. >> >> ~/Library/Combinator would be a better installation root on OSX, >> that location fits better into guidelines from Apple and also >> avoids completely hiding the Combinator data from the user. > > I disagree, but Combinator's a complex beast and has a lot of other > moving parts which need to live in specific places. Its main > purpose is to manage your import path to easily switch between > different development branches of multiple projects, and so most of > its data is already in locations that the user has specified. > > A key thing about ~/.local in this case is that it isn't specific to > Combinator. It's any per-user installed dependency libraries for > development purposes, not necessarily on Combinator-managed > projects, and not even necessarily Python projects. Ah, so ~/.local is "just" a per-user variation on /usr/local? I didn't even know of that convention before this thread started, I tend to use ~/local (without dot) instead. > >> Why? Just because users can't remember on which platform they are >> developing ;-)? That said, there's a libpython.a file in the >> framework build for basicly that reason: enough users complained >> about there not being a python library they could link to that it >> was easier to add a symlink then trying to educate everyone. > > The system installation of Python on the mac has a nod in this > direction as well. /usr/bin/python is also present, as is /usr/lib/ > pythonX.Y?0Cas symlinks between the two locations. >> You could teach Combinator about running configure scripts ;-). > > Better yet, perhaps somebody should teach configure about MacOS, and > about per-user installation ;). I value my life to much to even think about developing this feature for auto*. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20080117/e6f94a44/attachment.bin From p.f.moore at gmail.com Thu Jan 17 17:27:07 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 17 Jan 2008 16:27:07 +0000 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <478F75E0.9020002@timgolden.me.uk> References: <478F0A07.5040406@cheimes.de> <11de01c858f2$15e0c0c0$41a24240$@com.au> <478F372A.5010402@cheimes.de> <478F3C5C.1040009@timgolden.me.uk> <478F46FE.5060401@cheimes.de> <478F4A7D.8090101@timgolden.me.uk> <79990c6b0801170701y10ddcf93qd3f23fd0b5c36ea@mail.gmail.com> <478F75E0.9020002@timgolden.me.uk> Message-ID: <79990c6b0801170827p5c1c286eh124cb8369519368f@mail.gmail.com> > The best we can hope for is to point to HOMEDRIVE/PATH as > Paul suggested (*possibly* rewiring os.path.expanduser to > try that first, depending on Win2K stuff) os.path.expanduser uses HOMEDRIVE/HOMEPATH. It checks for an explicit HOME (set manually by the user, this isn't a Windows standard name) first. (At least in 2.5, see ntpath.py) I think consistency with os.path.expanduser('~') is a significant benefit. > and make sure > the user is aware of what might happen if the HOMEDRIVE/PATH > is really a pointer to a network-based profile. Given that os.path.expanduser allows HOME to override HOMEDRIVE/HOMEPATH, users in this position could set an explicit HOME. The one downside of following expanduser is that Christian's code is in C, where ntpath.expanduser is in Python, so there are 2 versions to keep in sync. Maybe Christian could expose his C implementation, which ntpath.expanduser could then reuse? Paul. From mail at timgolden.me.uk Thu Jan 17 17:38:48 2008 From: mail at timgolden.me.uk (Tim Golden) Date: Thu, 17 Jan 2008 16:38:48 +0000 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <79990c6b0801170827p5c1c286eh124cb8369519368f@mail.gmail.com> References: <478F0A07.5040406@cheimes.de> <11de01c858f2$15e0c0c0$41a24240$@com.au> <478F372A.5010402@cheimes.de> <478F3C5C.1040009@timgolden.me.uk> <478F46FE.5060401@cheimes.de> <478F4A7D.8090101@timgolden.me.uk> <79990c6b0801170701y10ddcf93qd3f23fd0b5c36ea@mail.gmail.com> <478F75E0.9020002@timgolden.me.uk> <79990c6b0801170827p5c1c286eh124cb8369519368f@mail.gmail.com> Message-ID: <478F8498.2080705@timgolden.me.uk> Paul Moore wrote: >> The best we can hope for is to point to HOMEDRIVE/PATH as >> Paul suggested (*possibly* rewiring os.path.expanduser to >> try that first, depending on Win2K stuff) > > os.path.expanduser uses HOMEDRIVE/HOMEPATH. It checks for an explicit > HOME (set manually by the user, this isn't a Windows standard name) > first. (At least in 2.5, see ntpath.py) Ummm... see my earlier point a few posts back which refers to r54364 which gave priority to USERPROFILE over HOMEDRIVE/HOMEPATH > I think consistency with os.path.expanduser('~') is a significant benefit. Agreed entirely. > Given that os.path.expanduser allows HOME to override > HOMEDRIVE/HOMEPATH, users in this position could set an explicit HOME. You're right of course (and that's what I do now, in fact). It's just that it doesn't feel quite right relying on a nonstandard env var. It means if I log on to another machine and want to take advantage of these theoretically roaming Python modules I'd have to remember to set my HOME var to something. > The one downside of following expanduser is that Christian's code is > in C, where ntpath.expanduser is in Python, so there are 2 versions to > keep in sync. Maybe Christian could expose his C implementation, which > ntpath.expanduser could then reuse? If it had its priorities switched around, Christian's patch could simply call os.path.expanduser. I'll hold off now. The situation's clear enough and I'm not really offering anything here. There hard part's going to be making a decision. I think my proposal amounts to: 1) Switch the priority of env var checks in os.path.expanduser so that HOMEDRIVE/PATH comes before USERPROFILE. 2) Have the new code use os.path.expanduser under Windows 3) Document the fact that installing in a per-user way might be writing considerable amounts to a network-based profile. TJG From p.f.moore at gmail.com Thu Jan 17 18:32:18 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 17 Jan 2008 17:32:18 +0000 Subject: [Python-Dev] ntpath r54364 (was: PEP 370, open questions) Message-ID: <79990c6b0801170932y336dc800h6110cc8766ff0b2b@mail.gmail.com> On 17/01/2008, Tim Golden wrote: > Ummm... see my earlier point a few posts back which refers > to r54364 which gave priority to USERPROFILE over HOMEDRIVE/HOMEPATH Sorry. I'd not realised this was a post-2.5 revision (ie, not in released code yet...) Looking at the change, it appears that the point is to support ~user, which it does by getting the current user's home, and replacing the last part, on the assumption that it is the user name. This works for USERPROFILE (in my experience, I don't know if there are corner cases where it isn't true) but does *not* work if HOME is set, and only works otherwise by preferring USERPROFILE over HOMEDRIVE/HOMEPATH (and I believe that preference is misleading, as USERPROFILE is always set if HOMEDRIVE/PATH is, so there's no point checking for HD/HP if USERDIR fails). I'd recommend that this change be reverted. To correctly get another user's home directory would involve reading (via the registry, or maybe some utility function I couldn't find at a quick glance) the value of HOMEDRIVE/HOMEPATH which is set for the other user. (In fact, given that I guess these could be set in a login script, it's entirely possible that the information simply isn't available). I don't believe that ~user is a useful concept on Windows, and I'd rather that code using it fail than give what are quite probably unreliable answers. If someone wants to support ~user where user is the name of the current user, then that could be covered by an explicit check that the name matches, and if so use the code for '~', but personally I don't think that's worth it. Paul. From lists at cheimes.de Thu Jan 17 18:57:14 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 17 Jan 2008 18:57:14 +0100 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: References: <478F0A07.5040406@cheimes.de> <20080117084004.21558.1005254566.divmod.xquotient.4836@joule.divmod.com> <64254CC6-8412-4343-903D-D117DCFB0117@mac.com> <20080117132212.21558.1891189134.divmod.xquotient.4917@joule.divmod.com> Message-ID: <478F96FA.3050903@cheimes.de> Ronald Oussoren wrote: > I'm pretty sure I've seen a suggestion for a distutils options for > installing into your home location (something like "python setup.py > install --user"). If you have read the PEP then I'm pretty sure that you have read about the python setup.py install --user thing. ;) Christian From lists at cheimes.de Thu Jan 17 19:32:22 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 17 Jan 2008 19:32:22 +0100 Subject: [Python-Dev] ntpath r54364 (was: PEP 370, open questions) In-Reply-To: <79990c6b0801170932y336dc800h6110cc8766ff0b2b@mail.gmail.com> References: <79990c6b0801170932y336dc800h6110cc8766ff0b2b@mail.gmail.com> Message-ID: <478F9F36.7080705@cheimes.de> Paul Moore wrote: > I'd recommend that this change be reverted. To correctly get another > user's home directory would involve reading (via the registry, or > maybe some utility function I couldn't find at a quick glance) the > value of HOMEDRIVE/HOMEPATH which is set for the other user. (In fact, > given that I guess these could be set in a login script, it's entirely > possible that the information simply isn't available). I concur! The changes should be reverted. The implementation is most likely to fail in more elaborate cases, e.g. c:\users\a\alpha or c:\users\groupa\name. The proper way for a ~user implementation is a bunch of Win32 API calls. Christian From lists at cheimes.de Thu Jan 17 19:36:41 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 17 Jan 2008 19:36:41 +0100 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <79990c6b0801170827p5c1c286eh124cb8369519368f@mail.gmail.com> References: <478F0A07.5040406@cheimes.de> <11de01c858f2$15e0c0c0$41a24240$@com.au> <478F372A.5010402@cheimes.de> <478F3C5C.1040009@timgolden.me.uk> <478F46FE.5060401@cheimes.de> <478F4A7D.8090101@timgolden.me.uk> <79990c6b0801170701y10ddcf93qd3f23fd0b5c36ea@mail.gmail.com> <478F75E0.9020002@timgolden.me.uk> <79990c6b0801170827p5c1c286eh124cb8369519368f@mail.gmail.com> Message-ID: <478FA039.9030001@cheimes.de> Paul Moore wrote: > The one downside of following expanduser is that Christian's code is > in C, where ntpath.expanduser is in Python, so there are 2 versions to > keep in sync. Maybe Christian could expose his C implementation, which > ntpath.expanduser could then reuse? What code are you talking about? Except for the -s option PEP 370 is implemented in Python. Or are you talking about my os.path.getshellfolder() patch? Christian From lists at cheimes.de Thu Jan 17 19:36:41 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 17 Jan 2008 19:36:41 +0100 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <79990c6b0801170827p5c1c286eh124cb8369519368f@mail.gmail.com> References: <478F0A07.5040406@cheimes.de> <11de01c858f2$15e0c0c0$41a24240$@com.au> <478F372A.5010402@cheimes.de> <478F3C5C.1040009@timgolden.me.uk> <478F46FE.5060401@cheimes.de> <478F4A7D.8090101@timgolden.me.uk> <79990c6b0801170701y10ddcf93qd3f23fd0b5c36ea@mail.gmail.com> <478F75E0.9020002@timgolden.me.uk> <79990c6b0801170827p5c1c286eh124cb8369519368f@mail.gmail.com> Message-ID: <478FA039.9030001@cheimes.de> Paul Moore wrote: > The one downside of following expanduser is that Christian's code is > in C, where ntpath.expanduser is in Python, so there are 2 versions to > keep in sync. Maybe Christian could expose his C implementation, which > ntpath.expanduser could then reuse? What code are you talking about? Except for the -s option PEP 370 is implemented in Python. Or are you talking about my os.path.getshellfolder() patch? Christian From lists at cheimes.de Thu Jan 17 19:42:17 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 17 Jan 2008 19:42:17 +0100 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <478F8498.2080705@timgolden.me.uk> References: <478F0A07.5040406@cheimes.de> <11de01c858f2$15e0c0c0$41a24240$@com.au> <478F372A.5010402@cheimes.de> <478F3C5C.1040009@timgolden.me.uk> <478F46FE.5060401@cheimes.de> <478F4A7D.8090101@timgolden.me.uk> <79990c6b0801170701y10ddcf93qd3f23fd0b5c36ea@mail.gmail.com> <478F75E0.9020002@timgolden.me.uk> <79990c6b0801170827p5c1c286eh124cb8369519368f@mail.gmail.com> <478F8498.2080705@timgolden.me.uk> Message-ID: <478FA189.80709@cheimes.de> Tim Golden wrote: > If it had its priorities switched around, Christian's patch could simply > call os.path.expanduser. No, I can't use expanduser("~") because there is no sensible way to get from the path to APPDATA. In it's infinite wisdom MS has decided to localize the name of its shell folders. On my German WinXP box the APPDATA folder is called "Anwendungsdaten". > I'll hold off now. The situation's clear enough and I'm not really > offering anything here. There hard part's going to be making a > decision. I think my proposal amounts to: > > 1) Switch the priority of env var checks in os.path.expanduser > so that HOMEDRIVE/PATH comes before USERPROFILE. > 2) Have the new code use os.path.expanduser under Windows > 3) Document the fact that installing in a per-user way might > be writing considerable amounts to a network-based profile. 3) is sensible but I'd prefer 4): 4) Default to the %APPDATA% env var but give the user a chance to overwrite the policy with the env var PYTHONUSERBASE, or whatever is a good name for it. Did I mention that I'm not good in naming things? *g* Christian From lists at cheimes.de Thu Jan 17 19:42:17 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 17 Jan 2008 19:42:17 +0100 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <478F8498.2080705@timgolden.me.uk> References: <478F0A07.5040406@cheimes.de> <11de01c858f2$15e0c0c0$41a24240$@com.au> <478F372A.5010402@cheimes.de> <478F3C5C.1040009@timgolden.me.uk> <478F46FE.5060401@cheimes.de> <478F4A7D.8090101@timgolden.me.uk> <79990c6b0801170701y10ddcf93qd3f23fd0b5c36ea@mail.gmail.com> <478F75E0.9020002@timgolden.me.uk> <79990c6b0801170827p5c1c286eh124cb8369519368f@mail.gmail.com> <478F8498.2080705@timgolden.me.uk> Message-ID: <478FA189.80709@cheimes.de> Tim Golden wrote: > If it had its priorities switched around, Christian's patch could simply > call os.path.expanduser. No, I can't use expanduser("~") because there is no sensible way to get from the path to APPDATA. In it's infinite wisdom MS has decided to localize the name of its shell folders. On my German WinXP box the APPDATA folder is called "Anwendungsdaten". > I'll hold off now. The situation's clear enough and I'm not really > offering anything here. There hard part's going to be making a > decision. I think my proposal amounts to: > > 1) Switch the priority of env var checks in os.path.expanduser > so that HOMEDRIVE/PATH comes before USERPROFILE. > 2) Have the new code use os.path.expanduser under Windows > 3) Document the fact that installing in a per-user way might > be writing considerable amounts to a network-based profile. 3) is sensible but I'd prefer 4): 4) Default to the %APPDATA% env var but give the user a chance to overwrite the policy with the env var PYTHONUSERBASE, or whatever is a good name for it. Did I mention that I'm not good in naming things? *g* Christian From mail at timgolden.me.uk Thu Jan 17 20:51:08 2008 From: mail at timgolden.me.uk (Tim Golden) Date: Thu, 17 Jan 2008 19:51:08 +0000 Subject: [Python-Dev] ntpath r54364 In-Reply-To: <478F9F36.7080705@cheimes.de> References: <79990c6b0801170932y336dc800h6110cc8766ff0b2b@mail.gmail.com> <478F9F36.7080705@cheimes.de> Message-ID: <478FB1AC.8080900@timgolden.me.uk> Christian Heimes wrote: > Paul Moore wrote: >> I'd recommend that this change be reverted. To correctly get another >> user's home directory would involve reading (via the registry, or >> maybe some utility function I couldn't find at a quick glance) the >> value of HOMEDRIVE/HOMEPATH which is set for the other user. (In fact, >> given that I guess these could be set in a login script, it's entirely >> possible that the information simply isn't available). > > I concur! The changes should be reverted. The implementation is most > likely to fail in more elaborate cases, e.g. c:\users\a\alpha or > c:\users\groupa\name. The proper way for a ~user implementation is a > bunch of Win32 API calls. Frustratingly, I don't believe there's *any* canonical way to find ~user without actually going through the whole process of getting a token and impersonating them. If they've logged onto this machine already you can have a good go by following the code posted the other day on c.l.py [1] (although I'm now not sure about the l10n aspects which Christian mentioned). Any technique of going up one (from your own profile) and then down one is no better than assuming that all users are in /home on a *nix box. As to a logged-on user's *own* home path, I'd prefer HOMEDRIVE/PATH over USERPROFILE since the latter is at least explictly named HOMEsomething, although the latter can be the fallback. Both are present on Win2k & on WinXP. Can't check WinNT or Win9x but I don't know if we're looking to support those or not. TJG [1] http://groups.google.com/group/comp.lang.python/msg/71b20a67a9ca76dd From lists at cheimes.de Thu Jan 17 21:43:39 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 17 Jan 2008 21:43:39 +0100 Subject: [Python-Dev] ntpath r54364 In-Reply-To: <478FB1AC.8080900@timgolden.me.uk> References: <79990c6b0801170932y336dc800h6110cc8766ff0b2b@mail.gmail.com> <478F9F36.7080705@cheimes.de> <478FB1AC.8080900@timgolden.me.uk> Message-ID: <478FBDFB.4080307@cheimes.de> Tim Golden wrote: > Frustratingly, I don't believe there's *any* canonical > way to find ~user without actually going through the whole > process of getting a token and impersonating them. If > they've logged onto this machine already you can have > a good go by following the code posted the other day > on c.l.py [1] (although I'm now not sure about the l10n > aspects which Christian mentioned). Any technique of > going up one (from your own profile) and then down one > is no better than assuming that all users are in /home > on a *nix box. A while ago I've seen a variant of ExpandEnvironmentStrings that takes a user token, too. It's called ExpandEnvironmentStringsForUser(). http://msdn2.microsoft.com/en-us/library/aa373507(VS.85).aspx Before anybody gets exciting about the good news: Here is the bad news. A user token can't be obtained easily. In fact it requires a login + password or a process handler. :( > As to a logged-on user's *own* home path, I'd prefer > HOMEDRIVE/PATH over USERPROFILE since the latter is at > least explictly named HOMEsomething, although the latter > can be the fallback. Both are present on Win2k & on WinXP. > Can't check WinNT or Win9x but I don't know if we're looking > to support those or not. Python 2.6 targets Windows 2000 and newer. So no, we don't have to support NT and the 9x series any more. Christian From lists at cheimes.de Thu Jan 17 21:43:39 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 17 Jan 2008 21:43:39 +0100 Subject: [Python-Dev] ntpath r54364 In-Reply-To: <478FB1AC.8080900@timgolden.me.uk> References: <79990c6b0801170932y336dc800h6110cc8766ff0b2b@mail.gmail.com> <478F9F36.7080705@cheimes.de> <478FB1AC.8080900@timgolden.me.uk> Message-ID: <478FBDFB.4080307@cheimes.de> Tim Golden wrote: > Frustratingly, I don't believe there's *any* canonical > way to find ~user without actually going through the whole > process of getting a token and impersonating them. If > they've logged onto this machine already you can have > a good go by following the code posted the other day > on c.l.py [1] (although I'm now not sure about the l10n > aspects which Christian mentioned). Any technique of > going up one (from your own profile) and then down one > is no better than assuming that all users are in /home > on a *nix box. A while ago I've seen a variant of ExpandEnvironmentStrings that takes a user token, too. It's called ExpandEnvironmentStringsForUser(). http://msdn2.microsoft.com/en-us/library/aa373507(VS.85).aspx Before anybody gets exciting about the good news: Here is the bad news. A user token can't be obtained easily. In fact it requires a login + password or a process handler. :( > As to a logged-on user's *own* home path, I'd prefer > HOMEDRIVE/PATH over USERPROFILE since the latter is at > least explictly named HOMEsomething, although the latter > can be the fallback. Both are present on Win2k & on WinXP. > Can't check WinNT or Win9x but I don't know if we're looking > to support those or not. Python 2.6 targets Windows 2000 and newer. So no, we don't have to support NT and the 9x series any more. Christian From lists at cheimes.de Thu Jan 17 21:53:31 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 17 Jan 2008 21:53:31 +0100 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <20080117084004.21558.1005254566.divmod.xquotient.4836@joule.divmod.com> References: <478F0A07.5040406@cheimes.de> <20080117084004.21558.1005254566.divmod.xquotient.4836@joule.divmod.com> Message-ID: <478FC04B.10802@cheimes.de> glyph at divmod.com wrote: > My suggestion would be to have *both* ~/.local *and* ~/Library/Python be > honored on the Mac, because there really isn't much harm in doing so. > Perhaps it would make sense for non-framework builds to not honor > ~/Library/Python, but I am pretty certain, based on experience fielding > requests from confused users, that framework builds should still honor > ~/.local/lib/python.../site-packages. I'm taking your suggestion under consideration: ~/.local for standard and framework builds and ~/Library/Python for framework builds only. > In any event, the really important thing from my perspective is that the > PEP should explain how this very common use-case for per-user > installation of libraries can be accomplished on each of the "big three" > platforms. This explanation should be put somewhere that users can > easily find it when they are building libraries. > > I don't know what the "right way" to do this on Windows is though, so I > can't offer much help there. Something to do with MinGW and intense > suffering, I would guess. Good point, but is it the job of the PEP to explain such problems in detail? The PEP should consider the problem but the explanation belongs to a tutorial - IMHO. >> * The patch also adds a usecustomize hook to site. Is it useful and >> should it stay? > > Should this be "usercustomize"? I thought it was a typo but I see the > same typo in the PEP. I have often wished for something like this for > debugging, and it might have other uses, but there should be a caution > against abuse :). Yeah! I've a tendency to drag typos through my code because I heavily use copy 'n paste. A while ago I copied a typo from C code into the Python unit tests and later into the docs, too Christian From lists at cheimes.de Thu Jan 17 21:53:31 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 17 Jan 2008 21:53:31 +0100 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <20080117084004.21558.1005254566.divmod.xquotient.4836@joule.divmod.com> References: <478F0A07.5040406@cheimes.de> <20080117084004.21558.1005254566.divmod.xquotient.4836@joule.divmod.com> Message-ID: <478FC04B.10802@cheimes.de> glyph at divmod.com wrote: > My suggestion would be to have *both* ~/.local *and* ~/Library/Python be > honored on the Mac, because there really isn't much harm in doing so. > Perhaps it would make sense for non-framework builds to not honor > ~/Library/Python, but I am pretty certain, based on experience fielding > requests from confused users, that framework builds should still honor > ~/.local/lib/python.../site-packages. I'm taking your suggestion under consideration: ~/.local for standard and framework builds and ~/Library/Python for framework builds only. > In any event, the really important thing from my perspective is that the > PEP should explain how this very common use-case for per-user > installation of libraries can be accomplished on each of the "big three" > platforms. This explanation should be put somewhere that users can > easily find it when they are building libraries. > > I don't know what the "right way" to do this on Windows is though, so I > can't offer much help there. Something to do with MinGW and intense > suffering, I would guess. Good point, but is it the job of the PEP to explain such problems in detail? The PEP should consider the problem but the explanation belongs to a tutorial - IMHO. >> * The patch also adds a usecustomize hook to site. Is it useful and >> should it stay? > > Should this be "usercustomize"? I thought it was a typo but I see the > same typo in the PEP. I have often wished for something like this for > debugging, and it might have other uses, but there should be a caution > against abuse :). Yeah! I've a tendency to drag typos through my code because I heavily use copy 'n paste. A while ago I copied a typo from C code into the Python unit tests and later into the docs, too Christian From lists at cheimes.de Thu Jan 17 22:05:02 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 17 Jan 2008 22:05:02 +0100 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <5808E387-56FD-428C-A8AA-5B4176C373F5@mac.com> References: <478F0A07.5040406@cheimes.de> <20080117084004.21558.1005254566.divmod.xquotient.4836@joule.divmod.com> <64254CC6-8412-4343-903D-D117DCFB0117@mac.com> <20080117132212.21558.1891189134.divmod.xquotient.4917@joule.divmod.com> <5808E387-56FD-428C-A8AA-5B4176C373F5@mac.com> Message-ID: <478FC2FE.2060203@cheimes.de> Ronald Oussoren wrote: > Ah, so ~/.local is "just" a per-user variation on /usr/local? I didn't > even know of that convention before this thread started, I tend to use > ~/local (without dot) instead. I assume ~/.local was first introduced by the freedesktop.org people. On my box it's only used for some desktop related apps like ~/.local/share/Trash or XFC4. http://www.freedesktop.org/wiki/Specifications/basedir-spec Contrary to ~/local, the dot local directory doesn't show up, unless the user unhides dot files. Christian From lists at cheimes.de Thu Jan 17 22:05:02 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 17 Jan 2008 22:05:02 +0100 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <5808E387-56FD-428C-A8AA-5B4176C373F5@mac.com> References: <478F0A07.5040406@cheimes.de> <20080117084004.21558.1005254566.divmod.xquotient.4836@joule.divmod.com> <64254CC6-8412-4343-903D-D117DCFB0117@mac.com> <20080117132212.21558.1891189134.divmod.xquotient.4917@joule.divmod.com> <5808E387-56FD-428C-A8AA-5B4176C373F5@mac.com> Message-ID: <478FC2FE.2060203@cheimes.de> Ronald Oussoren wrote: > Ah, so ~/.local is "just" a per-user variation on /usr/local? I didn't > even know of that convention before this thread started, I tend to use > ~/local (without dot) instead. I assume ~/.local was first introduced by the freedesktop.org people. On my box it's only used for some desktop related apps like ~/.local/share/Trash or XFC4. http://www.freedesktop.org/wiki/Specifications/basedir-spec Contrary to ~/local, the dot local directory doesn't show up, unless the user unhides dot files. Christian From fdrake at acm.org Thu Jan 17 22:14:18 2008 From: fdrake at acm.org (Fred Drake) Date: Thu, 17 Jan 2008 16:14:18 -0500 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: <478FC2FE.2060203@cheimes.de> References: <478F0A07.5040406@cheimes.de> <20080117084004.21558.1005254566.divmod.xquotient.4836@joule.divmod.com> <64254CC6-8412-4343-903D-D117DCFB0117@mac.com> <20080117132212.21558.1891189134.divmod.xquotient.4917@joule.divmod.com> <5808E387-56FD-428C-A8AA-5B4176C373F5@mac.com> <478FC2FE.2060203@cheimes.de> Message-ID: On Jan 17, 2008, at 4:05 PM, Christian Heimes wrote: > I assume ~/.local was first introduced by the freedesktop.org > people. On > my box it's only used for some desktop related apps like > ~/.local/share/Trash or XFC4. I've only seen ~/.local/ defined there, and only ~/.local/share/ in that case. Anything beyond that is grass-roots, and still pretty obscure. That said, I think extending usage of ~/.local/ is fine for things that aren't supposed to be edited or directly used by users. ~/.local/ bin/ seems suspect. -Fred -- Fred Drake From suraj at barkale.com Thu Jan 17 23:11:31 2008 From: suraj at barkale.com (Suraj Barkale) Date: Thu, 17 Jan 2008 22:11:31 +0000 (UTC) Subject: [Python-Dev] PEP 370, open questions References: <478F0A07.5040406@cheimes.de> <11de01c858f2$15e0c0c0$41a24240$@com.au> <478F372A.5010402@cheimes.de> <478F3C5C.1040009@timgolden.me.uk> <478F46FE.5060401@cheimes.de> <478F4A7D.8090101@timgolden.me.uk> <79990c6b0801170701y10ddcf93qd3f23fd0b5c36ea@mail.gmail.com> Message-ID: Paul Moore gmail.com> writes: > > Yes, this does contradict the Microsoft guideline that you shouldn't > write to USERPROFILE, but Microsoft themselves set HOMEDRIVE and > HOMEPATH, so they vioated the guidelines, not us . > > The point here is that we want "the user's home". This is clearly > %HOMEDRIVE%%HOMEPATH% on Windows, guidelines notwithstanding. > To add another twist to this the HOMEDRIVE mapping is not guarantied at all on windows startup (http://www.google.com/search?q=windows+homedrive+mapping). This has caused a lot of pain for me in the past. Can this be changed into using %APPDATA% by default and changing to %HOMEDRIVE%%HOMEPATH% based on some MSI switch? Regards, Suraj From guido at python.org Thu Jan 17 23:19:16 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 17 Jan 2008 14:19:16 -0800 Subject: [Python-Dev] building _ctypes in trunk fails first time around Message-ID: If I run "make clean" and then "make", builting _ctypes fails with this message: *** WARNING: renaming "_ctypes" since importing it failed: No module named _weakref Typing "make" a second time fixes this -- it seems a simple matter of _ctypes being built before _weakref. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Jan 18 02:09:46 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 17 Jan 2008 17:09:46 -0800 Subject: [Python-Dev] What to do for bytes in 2.6? Message-ID: I believe the issue of whether and how to backport bytes (and bytearray?) from 3.0 to 2.6 has come up before, but I don't think we've come to any kind of conclusion. It's quite subtle. In a private email, Thomas Wouters and I discussed this: [Guido] > > Perhaps the biggest question in my mind is what to do with bytes in > > 2.6. Make it an alias for str? Or a new type that doesn't mix with > > either str or unicode? But then how do we get read() calls and the > > like on binary files or sockets to return bytes instances? Perhaps a > > trivial str subclass? Or return bytes when -3 is specified? I need to > > bring this up on the list. [Thomas] > I don't know yet. I'm not sure yet what to do with strings and unicode in > 2.6 in -3 mode (or perhaps through a future-import, with some hackery.) > Perhaps a *set* of str-subclasses makes sense: a will-be-unicode subclass, > and a will-be-bytes subclass. String literals and text-mode input would > produce will-be-unicode, binary input and byte literals will-be-bytes. > Switching between -3 and normal mode for that would be changing a few > pointers in the core, making all operations return normal strings in normal > mode. And the would-be types can produce whatever warning they wish -- they > need not worry about speed. But I'm not sure if that'll work. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Fri Jan 18 03:00:41 2008 From: python at rcn.com (Raymond Hettinger) Date: Thu, 17 Jan 2008 21:00:41 -0500 (EST) Subject: [Python-Dev] What to do for bytes in 2.6? Message-ID: <20080117210041.AFG05602@ms19.lnh.mail.rcn.net> [GvR] > I believe the issue of whether and how to backport bytes > (and bytearray?) from 3.0 to 2.6 has come up before, but > I don't think we've come to any kind of conclusion. My recommendation is to leave it out of 2.6. Not every 3.0 concept has to be backported. This particular one doesn't have a straight-forward integration. It duplicates some existing functionality and in general doesn't make life better for the 2.6 coder. The only benefit I can see is that it lets you write code that is a step closer to 3.0; however, it has not been our goal to write one piece of code that runs under both major releases. I think both 2.x and 3.0 are better served if 2.x cleanly stays with the str/uncode model and 3.0 sticks with the bytes/text model. Commingling the two muddies the waters and conflates the distinctions. I think it best to make that jump all at once. Multiple-ways-to-do-it should not be the motto for 2.6. Instead, let's provide the best transition tools possible and keep both 2.6 and 3.0 as clean as possible (no duplicative or half-intergrated functionality). Raymond From guido at python.org Fri Jan 18 03:30:42 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 17 Jan 2008 18:30:42 -0800 Subject: [Python-Dev] What to do for bytes in 2.6? In-Reply-To: <20080117210041.AFG05602@ms19.lnh.mail.rcn.net> References: <20080117210041.AFG05602@ms19.lnh.mail.rcn.net> Message-ID: On Jan 17, 2008 6:00 PM, Raymond Hettinger wrote: > [GvR] > > I believe the issue of whether and how to backport bytes > > (and bytearray?) from 3.0 to 2.6 has come up before, but > > I don't think we've come to any kind of conclusion. > > My recommendation is to leave it out of 2.6. > > Not every 3.0 concept has to be backported. This particular one doesn't have a straight-forward integration. It duplicates some existing functionality and in general doesn't make life better for the 2.6 coder. The only benefit I can see is that it lets you write code that is a step closer to 3.0; however, it has not been our goal to write one piece of code that runs under both major releases. Indeed. However, it *is* one of our stated goals that 2.6 will provide a variety of features that will make source-to-source conversion more likely to succeed. I'm currently expecting that the str/unicode/bytes issue is by far the biggest issue where automated conversion is unlikely to be effective. *If* we provide some kind of "backport" of bytes (even if it's just an alias for or trivial subclass of str), it should be part of a strategy that makes it easier to write code that runs under 2.6 and can be automatically translated to run under 3.0 with the same semantics. I realize that this is a lofty goal and that I haven't thought through how to get there -- maybe it won't be possible anyway, but I think it's worth a try. (I'll think about it more later.) > I think both 2.x and 3.0 are better served if 2.x cleanly stays with the str/uncode model and 3.0 sticks with the bytes/text model. Commingling the two muddies the waters and conflates the distinctions. I think it best to make that jump all at once. > > Multiple-ways-to-do-it should not be the motto for 2.6. Instead, let's provide the best transition tools possible and keep both 2.6 and 3.0 as clean as possible (no duplicative or half-intergrated functionality). Well, 2.6 *is* one of the transition tools. We should probably do a case study on the back of an envelope on how to take existing code that deals with encoded text, decoded text, and binary data, and how to restructure it for automatic translation to correct 3.0 code. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Fri Jan 18 04:11:10 2008 From: python at rcn.com (Raymond Hettinger) Date: Thu, 17 Jan 2008 22:11:10 -0500 (EST) Subject: [Python-Dev] What to do for bytes in 2.6? Message-ID: <20080117221110.AFG13320@ms19.lnh.mail.rcn.net> > *If* we provide some kind of "backport" of > bytes (even if it's just an alias for or trivial > subclass of str), it should be part of a strategy > that makes it easier to write code that > runs under 2.6 and can be automatically translated > to run under 3.0 with the same semantics. If it's just an alias or trivial subclass, then we haven't added anything that can't be done trivially by the 2-to-3 tool. I'm thinking that this is a deeper change. It doesn't serve either 2.6 or 3.0 to conflate str/unicode model with the bytes/text model. Mixing the two in one place just creates a mess in that one place. I'm sure we're thinking that this is just an optional transition tool, but the reality is that once people write 2.6 tools that use the new model, then 2.6 users are forced to deal with that model. It stops being optional or something in the future, it becomes a mental jump that needs to be made now (while still retaining the previous model in mind for all the rest of the code). I don't think you need a case study to forsee that it will be unpleasant to work with a code base that commingles the two world views. One other thought. I'm guessing that apps that would care about the distinction are already using unicode and are already treating text as distinct from arrays of bytes. Instead, it's backwards thinking 20th-century neanderthal ascii-bound folks like myself who are going to have transition issues. It would be nice for us knuckle-draggers to not have to face the issue until 3.0. Raymond From Ole.Nielsen at ga.gov.au Fri Jan 18 05:30:10 2008 From: Ole.Nielsen at ga.gov.au (Ole.Nielsen at ga.gov.au) Date: Fri, 18 Jan 2008 15:30:10 +1100 Subject: [Python-Dev] Rationale for NamedTemporaryFile revisited [SEC=UNCLASSIFIED] Message-ID: <8BD19F29B0E16E4F88277A997CD872C203542DEA@mail.agso.gov.au> I found this posting, and those following it, as I too am baffled that NamedTemporaryFile doesn't let you read the generated file - even within the same script. For unit testing it is very commonplace to generate a test file on the fly and then use it as input the function being tested. NamedTemporaryFile would be the perfect candidate for this except that the file seems to disappear immediately after it has been closed. This is contrary to Dustin's comment which states that the lifetime extends until the object is garbage collected. The following script illustrates this isn't the case: from tempfile import NamedTemporaryFile fid = NamedTemporaryFile(mode='w', suffix='.tmp', dir='.') fid.write('My temp file') fid.close() # Read it (fid hasn't been garbage collected yet) print fid.name # But the file itself has gone fid2 = open(fid.name) print fid2.readlines() Can someone please help point me to the best way of creating temporary files in Python that can be read in immediately after their creation. Thanks Ole Nielsen, Geoscience Australia On Sun, Mar 18, 2007 at 11:49:55AM +1200, Greg Ewing wrote: > I've just discovered the hard way that NamedTemporaryFile > automatically deletes the file when you close it. That > doesn't seem very useful to me, since surely the reason > you're using NamedTemporaryFile instead of TemporaryFile > is that you want to do something else with it afterwards? > What's the rationale for this behaviour? For both TemporaryFile and NamedTemporaryFile, the rationale is that "temporary" extends until the object is garbage collected. TemporaryFile is deleted immediately (to prevent other applications from modifying your temporary file). NamedTemporaryFile is inteded for use when you need access to the file by filename during the lifetime of the file. In either case, when the object goes, the file should too. Dustin From guido at python.org Fri Jan 18 05:43:47 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 17 Jan 2008 20:43:47 -0800 Subject: [Python-Dev] What to do for bytes in 2.6? In-Reply-To: <20080117221110.AFG13320@ms19.lnh.mail.rcn.net> References: <20080117221110.AFG13320@ms19.lnh.mail.rcn.net> Message-ID: On Jan 17, 2008 7:11 PM, Raymond Hettinger wrote: > > *If* we provide some kind of "backport" of > > bytes (even if it's just an alias for or trivial > > subclass of str), it should be part of a strategy > > that makes it easier to write code that > > runs under 2.6 and can be automatically translated > > to run under 3.0 with the same semantics. > > If it's just an alias or trivial subclass, then we > haven't added anything that can't be done trivially > by the 2-to-3 tool. I suggest you study how the 2to3 tool actually works before asserting this. Consider the following function. def stuff(blah): foo = "" while True: bar = blah.read(1024) if bar == "": break foo += bar return foo Is it reading text or binary data from stream blah? We can't tell. If it's meant to be reading text, 2to3 should leave it alone. But if it's meant to be reading binary data, 2to3 should change the string literals to bytes literals (b"" in this case). (If it's used for both, there's no hope.) As it stands, 2to3 hasn't a chance to decide what to do, so it will leave it alone -- but the "translated" code will be wrong if it was meant to be reading bytes. However, if the two empty string literals were changed to b"", we would know it was reading bytes. 2to3 could leave it alone, but at least the untranslated code would be correct for 2.6 and the translated code would be correct for 3.0. This may seem trivial (because we do all the work, and 2to3 just leaves stuff alone), but having b"" and bytes as aliases for "" and str in 2.6 would mean that we could write 2.6 code that correctly expresses the use of binary data -- and we could use u"" and unicode for code using text, and 2to3 would translate those to "" and str and the code would be correct 3.0 text processing code. Note that we really can't make 2to3 assume that all uses of str and "" are referring to binary data -- that would mistranslate the vast majority of code that does non-Unicode-aware text processing, which I estimate is the majority of small and mid-size programs. > I'm thinking that this is a deeper change. > It doesn't serve either 2.6 or 3.0 to conflate > str/unicode model with the bytes/text model. > Mixing the two in one place just creates a mess > in that one place. > > I'm sure we're thinking that this is just an optional > transition tool, but the reality is that once people > write 2.6 tools that use the new model, > then 2.6 users are forced to deal with that model. > It stops being optional or something in the future, > it becomes a mental jump that needs to be made now > (while still retaining the previous model in mind > for all the rest of the code). This may be true. But still, 2.6 *will* run 2.5 code without any effort, so we will be able to mix modules using the 2.5 style and modules using the 3.0 style (or at least some aspects of 3.0 style) in one interpreter. Neither 2.5 nor 3.0 will support this combination. That's why 2.6 is so important it's a stepping stone. > I don't think you need a case study to forsee that > it will be unpleasant to work with a code base > that commingles the two world views. Well, you shouldn't commingle the two world view in a single module or package. But that would just be bad style -- you shouldn't use competing style rules within a package either (like using words_with_underscores and camelCaseWords for method names). > One other thought. I'm guessing that apps that would > care about the distinction are already using unicode > and are already treating text as distinct from arrays > of bytes. Yes, but 99% of these still accept str instances in positions where they require text. The problem is that the str type and its literals are ambiguous -- their use is not enough to be able to guess whether text or data is meant. Just being able to (voluntarily! on a per-module basis!) use a different type name and literal style for data could help forward-looking programmers get started on making the distinction clear, thus getting ready for 3.0 without making the jump just yet (or maintaining a 2.6 and a 3.0 version of the same package easily, using 2to3 to automatically generate the 3.0 version from the 2.6 code base). > Instead, it's backwards thinking 20th-century > neanderthal ascii-bound folks like myself who are going > to have transition issues. It would be nice for us > knuckle-draggers to not have to face the issue until 3.0. Oh, you won't. Just don't use the -3 command-line flag and don't put "from __future__ import " at the top of your modules, and you won't have to change your ways at all. You can continue to distribute your packages in 2.5 syntax that will also work with 2.6, and your users will be happy (as long as they don't want to use your code on 3.0 -- but if you want to give them that, *that* is when you will finally be forced to face the issue. :-) Note that I believe that the -3 flag should not change semantics -- it should only add warnings. Semantic changes must either be backwards compatible or be requested explicitly with a __forward__ import (which 2to3 can remove). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Jan 18 05:45:27 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 17 Jan 2008 20:45:27 -0800 Subject: [Python-Dev] Rationale for NamedTemporaryFile revisited [SEC=UNCLASSIFIED] In-Reply-To: <8BD19F29B0E16E4F88277A997CD872C203542DEA@mail.agso.gov.au> References: <8BD19F29B0E16E4F88277A997CD872C203542DEA@mail.agso.gov.au> Message-ID: Don't close it until you're done with it. Isn't that obvious? On Jan 17, 2008 8:30 PM, wrote: > I found this posting, and those following it, as I too am baffled that > NamedTemporaryFile doesn't let you read the generated file - even within the > same script. > For unit testing it is very commonplace to generate a test file on the fly > and then use it as input the function being tested. NamedTemporaryFile would > be the perfect candidate for this except that the file seems to disappear > immediately after it has been closed. This is contrary to Dustin's comment > which states that the lifetime extends until the object is garbage collected. > The following script illustrates this isn't the case: > > from tempfile import NamedTemporaryFile > fid = NamedTemporaryFile(mode='w', > suffix='.tmp', > dir='.') > > fid.write('My temp file') > fid.close() > > # Read it (fid hasn't been garbage collected yet) > print fid.name > > # But the file itself has gone > fid2 = open(fid.name) > print fid2.readlines() > > > Can someone please help point me to the best way of creating temporary files > in Python that can be read in immediately after their creation. > > Thanks > Ole Nielsen, Geoscience Australia > > > > On Sun, Mar 18, 2007 at 11:49:55AM +1200, Greg Ewing wrote: > > I've just discovered the hard way that NamedTemporaryFile > > automatically deletes the file when you close it. That > > doesn't seem very useful to me, since surely the reason > > you're using NamedTemporaryFile instead of TemporaryFile > > is that you want to do something else with it afterwards? > > What's the rationale for this behaviour? > > For both TemporaryFile and NamedTemporaryFile, the rationale is that > "temporary" extends until the object is garbage collected. > TemporaryFile is deleted immediately (to prevent other applications from > modifying your temporary file). NamedTemporaryFile is inteded for use > when you need access to the file by filename during the lifetime of the > file. In either case, when the object goes, the file should too. > > Dustin > > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From stephen at xemacs.org Fri Jan 18 05:57:22 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 18 Jan 2008 13:57:22 +0900 Subject: [Python-Dev] What to do for bytes in 2.6? In-Reply-To: <20080117221110.AFG13320@ms19.lnh.mail.rcn.net> References: <20080117221110.AFG13320@ms19.lnh.mail.rcn.net> Message-ID: <87ir1rloi5.fsf@uwakimon.sk.tsukuba.ac.jp> Raymond Hettinger writes: > One other thought. I'm guessing that apps that would > care about the distinction are already using unicode > and are already treating text as distinct from arrays > of bytes. Indeed. Mailman, for instance. Yet Mailman still has problems with (broken) wire protocol that sneaks past the gate, and causes some exception that is only handled by the top-level "no matter what goes wrong, we're not going to lose this post" handler (which literally shunts it into a queue that only human admins look it -- it's not Mailman's problem any more.) However, I am not sure it would help Mailman to catch such bugs to move from the str/unicode paradigm to the bytes/text paradigm. The problem Mailman faces is that there is no (single) Japanese foyer where the characters have to exchange their muddy "bytes" shoes for nice clean "unicode" slippers. Instead, there are a number of ways to get in, and the translation takes place (and sometimes not) at different stages. But this is not a Python issue; it has to do with Mailman's design. So I don't think this would be improved if we changed the paradigm forcibly. I don't see a benefit to apps like Mailman from changing over in 2.x. From Ole.Nielsen at ga.gov.au Fri Jan 18 06:05:09 2008 From: Ole.Nielsen at ga.gov.au (Ole.Nielsen at ga.gov.au) Date: Fri, 18 Jan 2008 16:05:09 +1100 Subject: [Python-Dev] Rationale for NamedTemporaryFile revisited [SEC=UNCLASSIFIED] Message-ID: <8BD19F29B0E16E4F88277A997CD872C20358E669@mail.agso.gov.au> Thank you very much for the quick reply. I believe we have to close the file in order be able to read it in - in this case to feed a unittest. I actually tried to read it in before closing it, but (as I suspected) the data wasn't available. We use unit testing for pretty much all functions and classes in our system. Some functions read files and do something with them. We therefore need a standardised and robust way to generate temporary files that exist on the filesystem long enough for the function being tested to read them. We are happy to write a wrapper ourselves that will do just that - but before doing so we had a look at the standard functionality. Hence the question about NamedTemporaryFile: What is the purpose of creating a name if it disappears after the file object is closed? I agree with Dustin that it would make sense for the file to live on the disk for as long as the File object is available to the script, closed or not. But I don't think that is the way NamedTemporaryFile currently works as illustrated in the test script in my first posting. Any help or comments, please? Cheers Ole Nielsen, Geoscience Australia -----Original Message----- From: gvanrossum at gmail.com [mailto:gvanrossum at gmail.com] On Behalf Of Guido van Rossum Sent: Friday, 18 January 2008 3:45 To: Nielsen Ole Cc: python-dev at python.org Subject: Re: [Python-Dev] Rationale for NamedTemporaryFile revisited [SEC=UNCLASSIFIED] Don't close it until you're done with it. Isn't that obvious? On Jan 17, 2008 8:30 PM, wrote: > I found this posting, and those following it, as I too am baffled that > NamedTemporaryFile doesn't let you read the generated file - even within the > same script. > For unit testing it is very commonplace to generate a test file on the fly > and then use it as input the function being tested. NamedTemporaryFile would > be the perfect candidate for this except that the file seems to disappear > immediately after it has been closed. This is contrary to Dustin's comment > which states that the lifetime extends until the object is garbage collected. > The following script illustrates this isn't the case: > > from tempfile import NamedTemporaryFile > fid = NamedTemporaryFile(mode='w', > suffix='.tmp', > dir='.') > > fid.write('My temp file') > fid.close() > > # Read it (fid hasn't been garbage collected yet) > print fid.name > > # But the file itself has gone > fid2 = open(fid.name) > print fid2.readlines() > > > Can someone please help point me to the best way of creating temporary files > in Python that can be read in immediately after their creation. > > Thanks > Ole Nielsen, Geoscience Australia > > > > On Sun, Mar 18, 2007 at 11:49:55AM +1200, Greg Ewing wrote: > > I've just discovered the hard way that NamedTemporaryFile > > automatically deletes the file when you close it. That > > doesn't seem very useful to me, since surely the reason > > you're using NamedTemporaryFile instead of TemporaryFile > > is that you want to do something else with it afterwards? > > What's the rationale for this behaviour? > > For both TemporaryFile and NamedTemporaryFile, the rationale is that > "temporary" extends until the object is garbage collected. > TemporaryFile is deleted immediately (to prevent other applications from > modifying your temporary file). NamedTemporaryFile is inteded for use > when you need access to the file by filename during the lifetime of the > file. In either case, when the object goes, the file should too. > > Dustin > > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tjreedy at udel.edu Fri Jan 18 06:30:37 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 18 Jan 2008 00:30:37 -0500 Subject: [Python-Dev] What to do for bytes in 2.6? References: <20080117221110.AFG13320@ms19.lnh.mail.rcn.net> Message-ID: "Guido van Rossum" wrote in message news:ca471dc20801172043l3356e04et9b8e807177230c6f at mail.gmail.com... | Is it reading text or binary data from stream blah? We can't tell. If | it's meant to be reading text, 2to3 should leave it alone. But if it's | meant to be reading binary data, 2to3 should change the string | literals to bytes literals (b"" in this case). (If it's used for both, | there's no hope.) As it stands, 2to3 hasn't a chance to decide what to | do, so it will leave it alone -- but the "translated" code will be | wrong if it was meant to be reading bytes. It seems that the main purpose of adding bytes (as more or less a synonym for str when used as bytes) is to aid 2to3 translation. So I think I would favor it being part of a future import. | Note that I believe that the -3 flag should not change semantics -- it | should only add warnings. Semantic changes must either be backwards | compatible or be requested explicitly with a __forward__ import (which | 2to3 can remove). Were you planning to make bytes a __future__ (or __forward__?) import? I think making it so should satisfy Raymond's concerns. Even if whatever you eventually do is technically backwards compatible, he is suggesting that conceptually, it is not. I see some validity to that view. tjr From python at rcn.com Fri Jan 18 06:44:52 2008 From: python at rcn.com (Raymond Hettinger) Date: Thu, 17 Jan 2008 21:44:52 -0800 Subject: [Python-Dev] What to do for bytes in 2.6? References: <20080117221110.AFG13320@ms19.lnh.mail.rcn.net> Message-ID: <004e01c85995$403abe90$6800a8c0@RaymondLaptop1> > having b"" and bytes as aliases for "" and > str in 2.6 would mean that we could write 2.6 code that correctly > expresses the use of binary data -- and we could use u"" and unicode > for code using text, and 2to3 would translate those to "" and str and > the code would be correct 3.0 text processing code. I see. There's a healthy benefit for 2to3 translation that cannot be accomplished in any other way. This may potentially more than offset the negative impact to the 2.x world where it complexifies the language without any immediate payoff. FWIW, I'm sold on the rationale. Hopefully, this can be confined to just annotation and aliasing but not porting back any C API changes or code that depends on the bytes/text distinction. I worry that as soon as the annotation is made available, it will ripple throughout the code and pervade the language so that it cannot be ignored by Py2.6 coders. It's a bit of a Pandora's Box. > Just being able to (voluntarily! on a > per-module basis!) use a different type name and literal style for > data could help forward-looking programmers get started on making the > distinction clear, thus getting ready for 3.0 without making the jump > just yet (or maintaining a 2.6 and a 3.0 version of the same package > easily, using 2to3 to automatically generate the 3.0 version from the > 2.6 code base). Are we going to be "forward-looking" and change all of the standard library modules? Is this going to pop-up everywhere and become something you have to know about whether or not you're a volunteering forward-looking programmer? I hope not. Raymond From glyph at divmod.com Fri Jan 18 07:11:17 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Fri, 18 Jan 2008 06:11:17 -0000 Subject: [Python-Dev] What to do for bytes in 2.6? In-Reply-To: References: <20080117221110.AFG13320@ms19.lnh.mail.rcn.net> Message-ID: <20080118061117.21558.1970259221.divmod.xquotient.5019@joule.divmod.com> On 04:43 am, guido at python.org wrote: >Just being able to (voluntarily! on a >per-module basis!) use a different type name and literal style for >data could help forward-looking programmers get started on making the >distinction clear, thus getting ready for 3.0 without making the jump >just yet (or maintaining a 2.6 and a 3.0 version of the same package >easily, using 2to3 to automatically generate the 3.0 version from the >2.6 code base). Yes! Yes! Yes! A thousand times yes! :-D This is *the* crucial feature which will make porting large libraries like Twisted to 3.0 even possible. Thank you, Guido. To the critics of this idea: libraries which process text, if they are meant to be correct, will need to deal explicitly with the issue of what data-types they believe to be text, what methods they will call on them, and how they deal with them. You cannot get away from this. It is not an issue reserved for the "pure" future of 3.0; if your code doesn't handle these types correctly now, it has bugs in it *now*. (In fact I am fixing some code with just such a bug in it right now :).) It is definitely possible to make your library code do the right thing for different data types, continue to support str literals in 2.6, and eventually require text / unicode input (after an appropriate deprecation period, of course). And it will be a lot easier if the translations imposed by 2to3 are as minimal as possible. >Note that I believe that the -3 flag should not change semantics -- it >should only add warnings. Semantic changes must either be backwards >compatible or be requested explicitly with a __forward__ import (which >2to3 can remove). This also sounds great. From steve at holdenweb.com Fri Jan 18 07:42:26 2008 From: steve at holdenweb.com (Steve Holden) Date: Fri, 18 Jan 2008 01:42:26 -0500 Subject: [Python-Dev] Rationale for NamedTemporaryFile revisited [SEC=UNCLASSIFIED] In-Reply-To: <8BD19F29B0E16E4F88277A997CD872C20358E669@mail.agso.gov.au> References: <8BD19F29B0E16E4F88277A997CD872C20358E669@mail.agso.gov.au> Message-ID: Ole.Nielsen at ga.gov.au wrote: > Thank you very much for the quick reply. > > I believe we have to close the file in order be able to read it in - in this > case to feed a unittest. I actually tried to read it in before closing it, > but (as I suspected) the data wasn't available. > Did you try seeking back to the beginning before attempting your read? > We use unit testing for pretty much all functions and classes in our system. > Some functions read files and do something with them. We therefore need a > standardised and robust way to generate temporary files that exist on the > filesystem long enough for the function being tested to read them. > Well you can hardly believe this requirement is so novel that its absence hasn't been missed in Python, then. > We are happy to write a wrapper ourselves that will do just that - but before > doing so we had a look at the standard functionality. Hence the question > about NamedTemporaryFile: What is the purpose of creating a name if it > disappears after the file object is closed? I agree with Dustin that it would > make sense for the file to live on the disk for as long as the File object is > available to the script, closed or not. But I don't think that is the way > NamedTemporaryFile currently works as illustrated in the test script in my > first posting. > > Any help or comments, please? > The mistake you made was to change the mode from the default "w+b". The following code works on both Windows and Linux: from tempfile import NamedTemporaryFile fid = NamedTemporaryFile(mode='w+b', suffix='.tmp', dir='.') fid.write('My temp file') fid.seek(0) data = fid.read() print data fid.close() By the way, this is the sort of topic normally reserved for comp.lang.python, whose denizens would have straightened you out in no time flat. regards Steve > Cheers > Ole Nielsen, Geoscience Australia > > > -----Original Message----- > From: gvanrossum at gmail.com [mailto:gvanrossum at gmail.com] On Behalf Of Guido > van Rossum > Sent: Friday, 18 January 2008 3:45 > To: Nielsen Ole > Cc: python-dev at python.org > Subject: Re: [Python-Dev] Rationale for NamedTemporaryFile revisited > [SEC=UNCLASSIFIED] > > Don't close it until you're done with it. Isn't that obvious? > > On Jan 17, 2008 8:30 PM, wrote: >> I found this posting, and those following it, as I too am baffled that >> NamedTemporaryFile doesn't let you read the generated file - even within > the >> same script. >> For unit testing it is very commonplace to generate a test file on the fly >> and then use it as input the function being tested. NamedTemporaryFile > would >> be the perfect candidate for this except that the file seems to disappear >> immediately after it has been closed. This is contrary to Dustin's comment >> which states that the lifetime extends until the object is garbage > collected. >> The following script illustrates this isn't the case: >> >> from tempfile import NamedTemporaryFile >> fid = NamedTemporaryFile(mode='w', >> suffix='.tmp', >> dir='.') >> >> fid.write('My temp file') >> fid.close() >> >> # Read it (fid hasn't been garbage collected yet) >> print fid.name >> >> # But the file itself has gone >> fid2 = open(fid.name) >> print fid2.readlines() >> >> >> Can someone please help point me to the best way of creating temporary > files >> in Python that can be read in immediately after their creation. >> >> Thanks >> Ole Nielsen, Geoscience Australia >> >> >> >> On Sun, Mar 18, 2007 at 11:49:55AM +1200, Greg Ewing wrote: >>> I've just discovered the hard way that NamedTemporaryFile >>> automatically deletes the file when you close it. That >>> doesn't seem very useful to me, since surely the reason >>> you're using NamedTemporaryFile instead of TemporaryFile >>> is that you want to do something else with it afterwards? >>> What's the rationale for this behaviour? >> For both TemporaryFile and NamedTemporaryFile, the rationale is that >> "temporary" extends until the object is garbage collected. >> TemporaryFile is deleted immediately (to prevent other applications from >> modifying your temporary file). NamedTemporaryFile is inteded for use >> when you need access to the file by filename during the lifetime of the >> file. In either case, when the object goes, the file should too. >> >> Dustin >> >> >> >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > > > -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From theller at ctypes.org Fri Jan 18 08:22:15 2008 From: theller at ctypes.org (Thomas Heller) Date: Fri, 18 Jan 2008 08:22:15 +0100 Subject: [Python-Dev] building _ctypes in trunk fails first time around In-Reply-To: References: Message-ID: Guido van Rossum schrieb: > If I run "make clean" and then "make", builting _ctypes fails with this message: > > *** WARNING: renaming "_ctypes" since importing it failed: No module > named _weakref > > Typing "make" a second time fixes this -- it seems a simple matter of > _ctypes being built before _weakref. > What a pity. Can the order be changed in which the extensions are built, or should the importing be tried after all the extensions are ready? Or should _weakref be made a builtin module? Thomas From theller at ctypes.org Fri Jan 18 08:24:21 2008 From: theller at ctypes.org (Thomas Heller) Date: Fri, 18 Jan 2008 08:24:21 +0100 Subject: [Python-Dev] What to do for bytes in 2.6? In-Reply-To: References: Message-ID: Guido van Rossum schrieb: > I believe the issue of whether and how to backport bytes (and > bytearray?) from 3.0 to 2.6 has come up before, but I don't think > we've come to any kind of conclusion. It's quite subtle. In a private > email, Thomas Wouters and I discussed this: > > [Guido] >> > Perhaps the biggest question in my mind is what to do with bytes in >> > 2.6. Make it an alias for str? Or a new type that doesn't mix with >> > either str or unicode? But then how do we get read() calls and the >> > like on binary files or sockets to return bytes instances? Perhaps a >> > trivial str subclass? Or return bytes when -3 is specified? I need to >> > bring this up on the list. > > [Thomas] >> I don't know yet. I'm not sure yet what to do with strings and unicode in >> 2.6 in -3 mode (or perhaps through a future-import, with some hackery.) >> Perhaps a *set* of str-subclasses makes sense: a will-be-unicode subclass, >> and a will-be-bytes subclass. String literals and text-mode input would >> produce will-be-unicode, binary input and byte literals will-be-bytes. >> Switching between -3 and normal mode for that would be changing a few >> pointers in the core, making all operations return normal strings in normal >> mode. And the would-be types can produce whatever warning they wish -- they >> need not worry about speed. But I'm not sure if that'll work. > Is the bytes type required for PEP3118 'Revising the buffer protocol'? I just started to work on implementing this PEP for ctypes, in the hope that these changes can be merged into trunk. Thomas From theller at ctypes.org Fri Jan 18 09:05:02 2008 From: theller at ctypes.org (Thomas Heller) Date: Fri, 18 Jan 2008 09:05:02 +0100 Subject: [Python-Dev] Error in PEP3118? Message-ID: <47905DAE.9020802@ctypes.org> Hi Travis, The pep contains this sample: """ Nested array :: struct { int ival; double data[16*4]; } """i:ival: (16,4)d:data: """ """ I think it is wrong and must be changed to the following; is this correct? """ Nested array :: struct { int ival; double data[16][4]; } """i:ival: (16,4)d:data: """ """ Thomas From lists at cheimes.de Fri Jan 18 09:27:26 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 18 Jan 2008 09:27:26 +0100 Subject: [Python-Dev] What to do for bytes in 2.6? In-Reply-To: References: <20080117221110.AFG13320@ms19.lnh.mail.rcn.net> Message-ID: <479062EE.4020805@cheimes.de> Guido van Rossum wrote: > Oh, you won't. Just don't use the -3 command-line flag and don't put > "from __future__ import " at the top of your modules, and > you won't have to change your ways at all. You can continue to > distribute your packages in 2.5 syntax that will also work with 2.6, > and your users will be happy (as long as they don't want to use your > code on 3.0 -- but if you want to give them that, *that* is when you > will finally be forced to face the issue. :-) from __future__ import bytes ? What should the __future__ statement change? * "" returns unicode instead of str * b"" is allowed and returns bytes (a new subclass of str) * u"" is still allowed and returns unicode, too. * The str built-in is not available. > Note that I believe that the -3 flag should not change semantics -- it > should only add warnings. Semantic changes must either be backwards > compatible or be requested explicitly with a __forward__ import (which > 2to3 can remove). Agreed! Should the -3 flag also warn about deprecated features which can easily migrated with 2to3? In the past a warning for `` (back ticks) were added. The feature is deprecated but the migration is very simple. A 2to26 fixer may be worth implementing, too. It could migrate minor changes like `` -> repr and "except egg, spam" -> "except egg as spam". Christian From lists at cheimes.de Fri Jan 18 09:27:26 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 18 Jan 2008 09:27:26 +0100 Subject: [Python-Dev] What to do for bytes in 2.6? In-Reply-To: References: <20080117221110.AFG13320@ms19.lnh.mail.rcn.net> Message-ID: <479062EE.4020805@cheimes.de> Guido van Rossum wrote: > Oh, you won't. Just don't use the -3 command-line flag and don't put > "from __future__ import " at the top of your modules, and > you won't have to change your ways at all. You can continue to > distribute your packages in 2.5 syntax that will also work with 2.6, > and your users will be happy (as long as they don't want to use your > code on 3.0 -- but if you want to give them that, *that* is when you > will finally be forced to face the issue. :-) from __future__ import bytes ? What should the __future__ statement change? * "" returns unicode instead of str * b"" is allowed and returns bytes (a new subclass of str) * u"" is still allowed and returns unicode, too. * The str built-in is not available. > Note that I believe that the -3 flag should not change semantics -- it > should only add warnings. Semantic changes must either be backwards > compatible or be requested explicitly with a __forward__ import (which > 2to3 can remove). Agreed! Should the -3 flag also warn about deprecated features which can easily migrated with 2to3? In the past a warning for `` (back ticks) were added. The feature is deprecated but the migration is very simple. A 2to26 fixer may be worth implementing, too. It could migrate minor changes like `` -> repr and "except egg, spam" -> "except egg as spam". Christian From lists at cheimes.de Fri Jan 18 10:57:26 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 18 Jan 2008 10:57:26 +0100 Subject: [Python-Dev] building _ctypes in trunk fails first time around In-Reply-To: References: Message-ID: Guido van Rossum wrote: > If I run "make clean" and then "make", builting _ctypes fails with this message: > > *** WARNING: renaming "_ctypes" since importing it failed: No module > named _weakref > > Typing "make" a second time fixes this -- it seems a simple matter of > _ctypes being built before _weakref. I applied a quick fix in r60043. Christian From solipsis at pitrou.net Fri Jan 18 10:57:30 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 18 Jan 2008 09:57:30 +0000 (UTC) Subject: [Python-Dev] PEP 370, open questions References: <478F0A07.5040406@cheimes.de> <20080117084004.21558.1005254566.divmod.xquotient.4836@joule.divmod.com> <64254CC6-8412-4343-903D-D117DCFB0117@mac.com> <20080117132212.21558.1891189134.divmod.xquotient.4917@joule.divmod.com> <5808E387-56FD-428C-A8AA-5B4176C373F5@mac.com> <478FC2FE.2060203@cheimes.de> Message-ID: Christian Heimes cheimes.de> writes: > > I assume ~/.local was first introduced by the freedesktop.org people. On > my box it's only used for some desktop related apps like > ~/.local/share/Trash or XFC4. Same here, only ~/.local/share is used (on two boxes, one Mandriva and one Ubuntu). From lists at cheimes.de Fri Jan 18 12:09:24 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 18 Jan 2008 12:09:24 +0100 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: References: <478F0A07.5040406@cheimes.de> <11de01c858f2$15e0c0c0$41a24240$@com.au> <478F372A.5010402@cheimes.de> <478F3C5C.1040009@timgolden.me.uk> <478F46FE.5060401@cheimes.de> <478F4A7D.8090101@timgolden.me.uk> <79990c6b0801170701y10ddcf93qd3f23fd0b5c36ea@mail.gmail.com> Message-ID: <479088E4.7090802@cheimes.de> Suraj Barkale wrote: > Can this be changed into using %APPDATA% by default and changing to > %HOMEDRIVE%%HOMEPATH% based on some MSI switch? I'll stick to APPDATA but I'm going to introduce a new env var PYTHONUSERBASE. It overwrites the path to the base directory and allows users (and companies) to adjust the user directory for all Python versions. Christian From lists at cheimes.de Fri Jan 18 12:09:24 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 18 Jan 2008 12:09:24 +0100 Subject: [Python-Dev] PEP 370, open questions In-Reply-To: References: <478F0A07.5040406@cheimes.de> <11de01c858f2$15e0c0c0$41a24240$@com.au> <478F372A.5010402@cheimes.de> <478F3C5C.1040009@timgolden.me.uk> <478F46FE.5060401@cheimes.de> <478F4A7D.8090101@timgolden.me.uk> <79990c6b0801170701y10ddcf93qd3f23fd0b5c36ea@mail.gmail.com> Message-ID: <479088E4.7090802@cheimes.de> Suraj Barkale wrote: > Can this be changed into using %APPDATA% by default and changing to > %HOMEDRIVE%%HOMEPATH% based on some MSI switch? I'll stick to APPDATA but I'm going to introduce a new env var PYTHONUSERBASE. It overwrites the path to the base directory and allows users (and companies) to adjust the user directory for all Python versions. Christian From guido at python.org Fri Jan 18 15:52:25 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 18 Jan 2008 06:52:25 -0800 Subject: [Python-Dev] What to do for bytes in 2.6? In-Reply-To: References: <20080117221110.AFG13320@ms19.lnh.mail.rcn.net> Message-ID: On Jan 17, 2008 9:30 PM, Terry Reedy wrote: > "Guido van Rossum" wrote in message > news:ca471dc20801172043l3356e04et9b8e807177230c6f at mail.gmail.com... > | Is it reading text or binary data from stream blah? We can't tell. If > | it's meant to be reading text, 2to3 should leave it alone. But if it's > | meant to be reading binary data, 2to3 should change the string > | literals to bytes literals (b"" in this case). (If it's used for both, > | there's no hope.) As it stands, 2to3 hasn't a chance to decide what to > | do, so it will leave it alone -- but the "translated" code will be > | wrong if it was meant to be reading bytes. > > It seems that the main purpose of adding bytes (as more or less a > synonym for str when used as bytes) is to aid 2to3 translation. > So I think I would favor it being part of a future import. > > | Note that I believe that the -3 flag should not change semantics -- it > | should only add warnings. Semantic changes must either be backwards > | compatible or be requested explicitly with a __forward__ import (which > | 2to3 can remove). > > Were you planning to make bytes a __future__ (or __forward__?) import? > I think making it so should satisfy Raymond's concerns. Even if whatever > you eventually do is technically backwards compatible, he is suggesting > that conceptually, it is not. I see some validity to that view. While it *could* be made conditional on a __future__ import, the cost of those (both in terms of implementing them and using them) is rather high so I'd prefer it to be always available. Given Raymond's later response, I'm not sure it's worth the effort. On Jan 17, 2008 9:44 PM, Raymond Hettinger wrote: > > having b"" and bytes as aliases for "" and > > str in 2.6 would mean that we could write 2.6 code that correctly > > expresses the use of binary data -- and we could use u"" and unicode > > for code using text, and 2to3 would translate those to "" and str and > > the code would be correct 3.0 text processing code. > > I see. There's a healthy benefit for 2to3 translation that cannot be > accomplished in any other way. This may potentially more than offset the > negative impact to the 2.x world where it complexifies the language > without any immediate payoff. > > FWIW, I'm sold on the rationale. Hopefully, this can be confined > to just annotation and aliasing but not porting back any C API > changes or code that depends on the bytes/text distinction. I'm fine with only making this a superficial veneer: b"" and bytes as aliases for "" and str. However Thomas Heller's response requires more thought. > I worry > that as soon as the annotation is made available, it will ripple > throughout the code and pervade the language so that it cannot > be ignored by Py2.6 coders. It's a bit of a Pandora's Box. Well, that's one opinion against another. And I presume by now you have read Glyph's enthusiastic response. Getting Twisted on 3.0 is a big enabler for getting other 3rd party packages an apps on board! > > Just being able to (voluntarily! on a > > per-module basis!) use a different type name and literal style for > > data could help forward-looking programmers get started on making the > > distinction clear, thus getting ready for 3.0 without making the jump > > just yet (or maintaining a 2.6 and a 3.0 version of the same package > > easily, using 2to3 to automatically generate the 3.0 version from the > > 2.6 code base). > > Are we going to be "forward-looking" and change all of the > standard library modules? Is this going to pop-up everywhere > and become something you have to know about whether or > not you're a volunteering forward-looking programmer? I hope not. I have no desire to fix up the standard library cosmetically, and I don't see the need -- the standard library has already been forward ported to 3.0, so the rationale just doesn't apply. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Jan 18 15:56:26 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 18 Jan 2008 06:56:26 -0800 Subject: [Python-Dev] What to do for bytes in 2.6? In-Reply-To: References: Message-ID: On Jan 17, 2008 11:24 PM, Thomas Heller wrote: > Is the bytes type required for PEP3118 'Revising the buffer protocol'? I don't think so. I would like to see this PEP backported (but keep the old 'buffer' of course for b/w compatibility). Whenever this PEP talks about bytes we can just use str/PyString in the backport. > I just started to work on implementing this PEP for ctypes, in the > hope that these changes can be merged into trunk. Remember that the trunk ought to remain backwards compatible with 2.5 -- 2.6 should not break code that worked under 2.5, since we want as many users as possible to feel comfortable with upgrading to 2.6. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Jan 18 16:00:57 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 18 Jan 2008 07:00:57 -0800 Subject: [Python-Dev] What to do for bytes in 2.6? In-Reply-To: <479062EE.4020805@cheimes.de> References: <20080117221110.AFG13320@ms19.lnh.mail.rcn.net> <479062EE.4020805@cheimes.de> Message-ID: On Jan 18, 2008 12:27 AM, Christian Heimes wrote: > Guido van Rossum wrote: > > Oh, you won't. Just don't use the -3 command-line flag and don't put > > "from __future__ import " at the top of your modules, and > > you won't have to change your ways at all. You can continue to > > distribute your packages in 2.5 syntax that will also work with 2.6, > > and your users will be happy (as long as they don't want to use your > > code on 3.0 -- but if you want to give them that, *that* is when you > > will finally be forced to face the issue. :-) > > from __future__ import bytes ? What should the __future__ statement change? > > * "" returns unicode instead of str > * b"" is allowed and returns bytes (a new subclass of str) > * u"" is still allowed and returns unicode, too. > * The str built-in is not available. I don't think any of that is necessary. I would rather have the following two in the language by default (see my response to Terry and Raymond): bytes is an alias for str (not even a subclass) b"" is an alias for "" > > Note that I believe that the -3 flag should not change semantics -- it > > should only add warnings. Semantic changes must either be backwards > > compatible or be requested explicitly with a __forward__ import (which > > 2to3 can remove). > > Agreed! Should the -3 flag also warn about deprecated features which can > easily migrated with 2to3? In the past a warning for `` (back ticks) > were added. The feature is deprecated but the migration is very simple. Personally, I think such warnings are misguided. -3 should augment 2to3, not duplicate its work. > A 2to26 fixer may be worth implementing, too. It could migrate minor > changes like `` -> repr and "except egg, spam" -> "except egg as spam". 2to3 with selected options can do that already. But again, I don't see the point -- 2.6 code can just use 2.5 syntax and 2to3 will take care of the migration. There is only one thing I'd like -3 to warn about regarding except clauses: if the 'err' variable is used past the end of the except clause. That is a true semantic change that 2to3 cannot easily fix or flag. Consider this an example of how 2to3 and -3 should augment each other. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From lists at cheimes.de Fri Jan 18 17:00:17 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 18 Jan 2008 17:00:17 +0100 Subject: [Python-Dev] What to do for bytes in 2.6? In-Reply-To: References: <20080117221110.AFG13320@ms19.lnh.mail.rcn.net> <479062EE.4020805@cheimes.de> Message-ID: <4790CD11.7040803@cheimes.de> Guido van Rossum wrote: > I don't think any of that is necessary. I would rather have the > following two in the language by default (see my response to Terry and > Raymond): > > bytes is an alias for str (not even a subclass) > b"" is an alias for "" Ah, you like to keep it simple. The aliases are easily to implement. Give me twenty minutes to implement it and write some unit tests. > There is only one thing I'd like -3 to warn about regarding except > clauses: if the 'err' variable is used past the end of the except > clause. That is a true semantic change that 2to3 cannot easily fix or > flag. Consider this an example of how 2to3 and -3 should augment each > other. Several warnings have to be removed: ``, callable and maybe more. Christian From facundobatista at gmail.com Fri Jan 18 17:15:48 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Fri, 18 Jan 2008 14:15:48 -0200 Subject: [Python-Dev] What to do for bytes in 2.6? In-Reply-To: References: <20080117221110.AFG13320@ms19.lnh.mail.rcn.net> <479062EE.4020805@cheimes.de> Message-ID: 2008/1/18, Guido van Rossum : > I don't think any of that is necessary. I would rather have the > following two in the language by default (see my response to Terry and > Raymond): > > bytes is an alias for str (not even a subclass) > b"" is an alias for "" +1 -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From lists at cheimes.de Fri Jan 18 17:24:00 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 18 Jan 2008 17:24:00 +0100 Subject: [Python-Dev] What to do for bytes in 2.6? In-Reply-To: <4790CD11.7040803@cheimes.de> References: <20080117221110.AFG13320@ms19.lnh.mail.rcn.net> <479062EE.4020805@cheimes.de> <4790CD11.7040803@cheimes.de> Message-ID: <4790D2A0.1060808@cheimes.de> Christian Heimes wrote: > Ah, you like to keep it simple. The aliases are easily to implement. > Give me twenty minutes to implement it and write some unit tests. http://bugs.python.org/issue1865 From status at bugs.python.org Fri Jan 18 19:06:10 2008 From: status at bugs.python.org (Tracker) Date: Fri, 18 Jan 2008 18:06:10 +0000 (UTC) Subject: [Python-Dev] Summary of Tracker Issues Message-ID: <20080118180610.4880278173@psf.upfronthosting.co.za> ACTIVITY SUMMARY (01/11/08 - 01/18/08) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1329 open (+31) / 11986 closed (+15) / 13315 total (+46) Open issues with patches: 437 Average duration of open issues: 673 days. Median duration of open issues: 989 days. Open Issues Breakdown open 1310 (+31) pending 19 ( +0) Issues Created Or Reopened (47) _______________________________ PEP 754 update 01/11/08 http://bugs.python.org/issue1795 created tiran ctypes should allow a tuple when an Array is expected 01/11/08 CLOSED http://bugs.python.org/issue1796 created theller patch ctypes function pointer enhancements 01/11/08 http://bugs.python.org/issue1797 created theller patch Add ctypes calling convention that allows safe access of errno 01/11/08 http://bugs.python.org/issue1798 created theller patch Per user site-packages and setup.py install --user patch 01/11/08 http://bugs.python.org/issue1799 created tiran patch ctypes callback fails when called in Python with array argument 01/11/08 http://bugs.python.org/issue1800 created kermode docs for os.symlink(src, dst) doesn't mention exceptions 01/11/08 CLOSED http://bugs.python.org/issue1809 created dgardner AST compile() patch 01/11/08 http://bugs.python.org/issue1810 created thomas.lee patch True division of integers could be more accurate 01/12/08 http://bugs.python.org/issue1811 created marketdickinson doctest _load_testfile function -- newline handling seems incorr 01/12/08 http://bugs.python.org/issue1812 created pdonis easy Codec lookup failing under turkish locale 01/12/08 http://bugs.python.org/issue1813 created arnimar Victor Stinner's GMP patch for longs 01/12/08 http://bugs.python.org/issue1814 created tiran patch Distutils add ability to skip build [Feature Request] 01/12/08 http://bugs.python.org/issue1815 created Eloff sys.cmd_flags patch 01/12/08 CLOSED http://bugs.python.org/issue1816 created tiran patch module-cgi: handling GET and POST together 01/13/08 http://bugs.python.org/issue1817 created alef13 easy Add named tuple reader to CSV module 01/13/08 http://bugs.python.org/issue1818 created rhettinger Speed hack for function calls with named parameters 01/14/08 http://bugs.python.org/issue1819 created pitrou patch Enhance Object/structseq.c to match namedtuple and tuple api 01/14/08 http://bugs.python.org/issue1820 created tiran easy configure.ac change for FreeBSD 01/14/08 CLOSED http://bugs.python.org/issue1821 created asmodai email.mime.multipart.MIMEMultipart.is_multipart() returns false 01/14/08 http://bugs.python.org/issue1822 created Sharebear Possible to set invalid Content-Transfer-Encoding on email.mime. 01/14/08 http://bugs.python.org/issue1823 created Sharebear UUIDCreate()->UuidCreate() in msilib docs. 01/14/08 CLOSED http://bugs.python.org/issue1824 created Jimbo easy msilib.add_data() takes exactly three parameters 01/14/08 http://bugs.python.org/issue1825 created Jimbo operator.attrgetter() should accept dotted attribute paths 01/14/08 http://bugs.python.org/issue1826 created barry easy svnversion_init() doesn't support svn urls in sandbox/trunk 01/14/08 http://bugs.python.org/issue1827 created tiran Renaming platform path modules 01/15/08 CLOSED http://bugs.python.org/issue1828 created gutworth Renaming platform path modules 01/15/08 CLOSED http://bugs.python.org/issue1829 created gutworth pygettext.py py3k errors 01/15/08 CLOSED http://bugs.python.org/issue1830 created scav ctypes.Structure constructor arguments 01/15/08 CLOSED http://bugs.python.org/issue1831 created arigo easy Update version number in __init__.py 01/15/08 http://bugs.python.org/issue1835 created jlp easy 'weekly' rotating logging file rotation incorrect 01/15/08 http://bugs.python.org/issue1836 created kmk Add Queue.LifoQueue and Queue.PriorityQueue 01/15/08 CLOSED http://bugs.python.org/issue1837 created rhettinger patch Ctypes C-level infinite recursion 01/16/08 http://bugs.python.org/issue1838 created fijal Tools/i18n/msgfmt.py fixes for Python 3.0 01/16/08 http://bugs.python.org/issue1840 created scav patch Broken link in msilib docs 01/16/08 CLOSED http://bugs.python.org/issue1854 created Jimbo Codepage unset in msilib.init_database() 01/16/08 http://bugs.python.org/issue1855 created Jimbo shutdown (exit) can hang or segfault with daemon threads running 01/17/08 http://bugs.python.org/issue1856 created gregory.p.smith subprocess.Popen.poll/__del__ API breakage 01/17/08 http://bugs.python.org/issue1857 created rene.st Make .pypirc handle multiple servers 01/17/08 http://bugs.python.org/issue1858 created tarek patch, easy textwrap doesn't linebreak on "\n" 01/17/08 CLOSED http://bugs.python.org/issue1859 created palfrey traceback.print_last fails 01/17/08 CLOSED http://bugs.python.org/issue1860 created tzot sched scheduler.queue class member is unordered 01/17/08 CLOSED http://bugs.python.org/issue1861 created pbureau Error on "Save As" in IDLE (Vista 32-bit) 01/17/08 http://bugs.python.org/issue1862 created richjtd NameError with genexp in class scope 01/17/08 CLOSED http://bugs.python.org/issue1863 created cptnwillard test_locale doesn't use unittest 01/18/08 http://bugs.python.org/issue1864 created pitrou Bytes alias for 2.6 01/18/08 http://bugs.python.org/issue1865 created tiran patch qtsupport.py mistake leads to bad _Qt module 01/12/08 http://bugs.python.org/issue1570672 reopened georg.brandl patch Issues Now Closed (46) ______________________ Performance regression in 2.5 139 days http://bugs.python.org/issue1045 rhettinger email.Utils.parseaddr("a(WRONG)@b") 103 days http://bugs.python.org/issue1221 tiran openpty does not give bidirectional pipe 99 days http://bugs.python.org/issue1239 tiran IDLE - patch Delegator to support callables 96 days http://bugs.python.org/issue1252 kbk patch pyconfig.h not compatible with MS VC++ Express Edition 85 days http://bugs.python.org/issue1297 tiran smtplib starttls() should ehlo() if it needs to 82 days http://bugs.python.org/issue1339 gregory.p.smith easy Small bat files to build docs on Windows 59 days http://bugs.python.org/issue1472 tiran patch help('modules') broken by several 3rd party libraries (svn patch 41 days http://bugs.python.org/issue1550 ping patch, easy Implement PEP-3141 for Decimal 30 days http://bugs.python.org/issue1623 jyasskin patch Random errors on interpreter shutdown 12 days http://bugs.python.org/issue1731 jcea Decimal constructor accepts newline terminated strings 2 days http://bugs.python.org/issue1780 marketdickinson patch pdb should set stdin+stdout around exec call 5 days http://bugs.python.org/issue1786 gvanrossum patch, easy xmlrpclib ServerProxy page has out-of-date content 0 days http://bugs.python.org/issue1790 akuchling ctypes should allow a tuple when an Array is expected 5 days http://bugs.python.org/issue1796 theller patch docs for os.symlink(src, dst) doesn't mention exceptions 0 days http://bugs.python.org/issue1809 georg.brandl sys.cmd_flags patch 1 days http://bugs.python.org/issue1816 tiran patch configure.ac change for FreeBSD 0 days http://bugs.python.org/issue1821 theller UUIDCreate()->UuidCreate() in msilib docs. 0 days http://bugs.python.org/issue1824 georg.brandl easy Renaming platform path modules 0 days http://bugs.python.org/issue1828 gvanrossum Renaming platform path modules 0 days http://bugs.python.org/issue1829 tiran pygettext.py py3k errors 0 days http://bugs.python.org/issue1830 gvanrossum ctypes.Structure constructor arguments 1 days http://bugs.python.org/issue1831 theller easy Add Queue.LifoQueue and Queue.PriorityQueue 1 days http://bugs.python.org/issue1837 rhettinger patch Broken link in msilib docs 0 days http://bugs.python.org/issue1854 georg.brandl textwrap doesn't linebreak on "\n" 0 days http://bugs.python.org/issue1859 gvanrossum traceback.print_last fails 0 days http://bugs.python.org/issue1860 amaury.forgeotdarc sched scheduler.queue class member is unordered 0 days http://bugs.python.org/issue1861 rhettinger NameError with genexp in class scope 1 days http://bugs.python.org/issue1863 georg.brandl Parser crashes for deeply nested list displays 2668 days http://bugs.python.org/issue215555 gvanrossum plistlib should be moved out of plat-mac 1630 days http://bugs.python.org/issue779460 surekap Fixes smtplib starttls HELO errors 1545 days http://bugs.python.org/issue829951 gregory.p.smith patch logging: need a way to discard Logger objects 12 days http://bugs.python.org/issue932563 vsajip modulefinder misses modules 894 days http://bugs.python.org/issue1252550 theller patch Elemental Security contribution - parsexml.py 808 days http://bugs.python.org/issue1337648 gvanrossum patch Decimal should allow leading or trailing spaces. 557 days http://bugs.python.org/issue1516613 marketdickinson RPM build fails for Py2.5b2 547 days http://bugs.python.org/issue1522046 tiran array.array borks on deepcopy 510 days http://bugs.python.org/issue1545837 mephinet Some Compiler Warnings on VC6 319 days http://bugs.python.org/issue1669637 tiran Building python 2.5 for AMD64 (windows) and VS2005 316 days http://bugs.python.org/issue1672336 tiran Method cache 296 days http://bugs.python.org/issue1685986 georg.brandl patch Python 2.5 installer ended prematurely 289 days http://bugs.python.org/issue1694155 techtonik wrong % of params for format string in ZipFile.printdir() 278 days http://bugs.python.org/issue1698398 rhettinger ZipFile.printdir fix (2.5) 278 days http://bugs.python.org/issue1698915 rhettinger patch ZipFile.printdir fix (2.6) 278 days http://bugs.python.org/issue1698917 rhettinger patch 0.0 and -0.0 end up referring to the same object 226 days http://bugs.python.org/issue1729014 tiran add new bytecodes: JUMP_IF_{FALSE|TRUE}_AND_POP 167 days http://bugs.python.org/issue1764638 rhettinger Top Issues Most Discussed (10) ______________________________ 13 shutdown (exit) can hang or segfault with daemon threads runnin 2 days open http://bugs.python.org/issue1856 12 operator.attrgetter() should accept dotted attribute paths 4 days open http://bugs.python.org/issue1826 11 textwrap doesn't linebreak on "\n" 0 days closed http://bugs.python.org/issue1859 11 Move Demo/classes/Rat.py to Lib/rational.py and fix it up. 28 days open http://bugs.python.org/issue1682 10 ctypes.Structure constructor arguments 1 days closed http://bugs.python.org/issue1831 10 sys.cmd_flags patch 1 days closed http://bugs.python.org/issue1816 8 Armin's method cache optimization updated for Python 2.6 280 days open http://bugs.python.org/issue1700288 6 NameError with genexp in class scope 1 days closed http://bugs.python.org/issue1863 6 sched scheduler.queue class member is unordered 0 days closed http://bugs.python.org/issue1861 6 Renaming platform path modules 0 days closed http://bugs.python.org/issue1828 From graham.horler at gmail.com Sat Jan 19 08:57:34 2008 From: graham.horler at gmail.com (Graham Horler) Date: Sat, 19 Jan 2008 07:57:34 +0000 Subject: [Python-Dev] Rationale for NamedTemporaryFile revisited [SEC=UNCLASSIFIED] References: <8BD19F29B0E16E4F88277A997CD872C20358E669@mail.agso.gov.au> Message-ID: <6fv3lh$6d377c@venus.eclipse.kcom.com> On 18 Jan 2008, 06:42:26, Steve Holden wrote: > Ole.Nielsen at ga.gov.au wrote: > > Thank you very much for the quick reply. > > > > I believe we have to close the file in order be able to read it in - in this > > case to feed a unittest. I actually tried to read it in before closing it, > > but (as I suspected) the data wasn't available. > The mistake you made was to change the mode from the default "w+b". The > following code works on both Windows and Linux: > > from tempfile import NamedTemporaryFile > fid = NamedTemporaryFile(mode='w+b', > suffix='.tmp', > dir='.') > > fid.write('My temp file') > fid.seek(0) > data = fid.read() > print data > fid.close() If you need to read the data by opening the file again (from unittest code for example), then you need to call fid.flush() (no need for seeking). Again, comp.lang.python, but I could not resist. regards. From lstoakes at gmail.com Sat Jan 19 12:26:23 2008 From: lstoakes at gmail.com (Lorenzo Stoakes) Date: Sat, 19 Jan 2008 11:26:23 +0000 Subject: [Python-Dev] Initial Development Message-ID: <40ec110b0801190326l5425ab68jaedc520ca852b34e@mail.gmail.com> Hi, Apologies if it's not appropriate to send this email to the mailing list, however I am really really eager to contribute to the Python project (as a developer), and wondered whether anyone could advise me as to some entry-level tasks I could start out with. Thanks, Lorenzo Stoakes From lists at cheimes.de Sat Jan 19 12:32:58 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 19 Jan 2008 12:32:58 +0100 Subject: [Python-Dev] Initial Development In-Reply-To: <40ec110b0801190326l5425ab68jaedc520ca852b34e@mail.gmail.com> References: <40ec110b0801190326l5425ab68jaedc520ca852b34e@mail.gmail.com> Message-ID: Lorenzo Stoakes wrote: > Apologies if it's not appropriate to send this email to the mailing > list, however I am really really eager to contribute to the Python > project (as a developer), and wondered whether anyone could advise me > as to some entry-level tasks I could start out with. We are having a bug day today. Please join us in #python-dev on irc.freenode.net Christian From g.brandl at gmx.net Sat Jan 19 18:40:13 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 19 Jan 2008 18:40:13 +0100 Subject: [Python-Dev] Is "0x" a valid token? Message-ID: Python accepts 0x as an integer literal. Is this intended (the docs say otherwise, but int() and tokenize.py concur)? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From guido at python.org Sat Jan 19 18:43:41 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 19 Jan 2008 09:43:41 -0800 Subject: [Python-Dev] Is "0x" a valid token? In-Reply-To: References: Message-ID: On Jan 19, 2008 9:40 AM, Georg Brandl wrote: > Python accepts 0x as an integer literal. Is this intended (the docs > say otherwise, but int() and tokenize.py concur)? Definitely a bug. I see no use case for this. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From nicko at nicko.org Sat Jan 19 18:19:43 2008 From: nicko at nicko.org (Nicko van Someren) Date: Sat, 19 Jan 2008 17:19:43 +0000 Subject: [Python-Dev] Monkeypatching idioms -- elegant or ugly? In-Reply-To: References: Message-ID: <4BBC2D11-EE64-4C00-BA17-5E3398D1044C@nicko.org> On 15 Jan 2008, at 15:37, Guido van Rossum wrote: > Second, a "metaclass" to add a number of methods (or other attributes) > to an existing class, using a convenient class notation: ... > class (): > __metaclass__ = monkeypatch_class > def (...): ... > def (...): ... > ... In Objective-C it's perfectly common to extend existing classes using 'categories' and I have often found this idiom very useful. What is described here is basically categories for Python. I've implemented something like this before and I would be happy to see this added to the standard library and formalised. Nicko From nas at arctrix.com Sat Jan 19 19:46:07 2008 From: nas at arctrix.com (Neil Schemenauer) Date: Sat, 19 Jan 2008 18:46:07 +0000 (UTC) Subject: [Python-Dev] What to do for bytes in 2.6? References: <20080117221110.AFG13320@ms19.lnh.mail.rcn.net> Message-ID: Guido van Rossum wrote: > This may seem trivial (because we do all the work, and 2to3 just > leaves stuff alone), but having b"" and bytes as aliases for "" and > str in 2.6 would mean that we could write 2.6 code that correctly > expresses the use of binary data -- and we could use u"" and unicode > for code using text, and 2to3 would translate those to "" and str and > the code would be correct 3.0 text processing code. I like this solution because of its simplicity. Neil From lists at cheimes.de Sat Jan 19 19:51:57 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 19 Jan 2008 19:51:57 +0100 Subject: [Python-Dev] What to do for bytes in 2.6? In-Reply-To: References: <20080117221110.AFG13320@ms19.lnh.mail.rcn.net> Message-ID: <479246CD.5030108@cheimes.de> Neil Schemenauer wrote: > I like this solution because of its simplicity. I've implemented and submitted the feature yesterday: Python 2.6a0 (trunk:60048M, Jan 18 2008, 19:08:16) [GCC 4.2.1 (Ubuntu 4.2.1-5ubuntu4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> b'a' 'a' >>> br'\a' '\\a' >>> bytes Christian From lists at cheimes.de Sat Jan 19 19:51:57 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 19 Jan 2008 19:51:57 +0100 Subject: [Python-Dev] What to do for bytes in 2.6? In-Reply-To: References: <20080117221110.AFG13320@ms19.lnh.mail.rcn.net> Message-ID: <479246CD.5030108@cheimes.de> Neil Schemenauer wrote: > I like this solution because of its simplicity. I've implemented and submitted the feature yesterday: Python 2.6a0 (trunk:60048M, Jan 18 2008, 19:08:16) [GCC 4.2.1 (Ubuntu 4.2.1-5ubuntu4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> b'a' 'a' >>> br'\a' '\\a' >>> bytes Christian From martin at v.loewis.de Sat Jan 19 19:53:42 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 19 Jan 2008 19:53:42 +0100 Subject: [Python-Dev] Monkeypatching idioms -- elegant or ugly? In-Reply-To: <4BBC2D11-EE64-4C00-BA17-5E3398D1044C@nicko.org> References: <4BBC2D11-EE64-4C00-BA17-5E3398D1044C@nicko.org> Message-ID: <47924736.4040202@v.loewis.de> > In Objective-C it's perfectly common to extend existing classes using > 'categories' and I have often found this idiom very useful. What is > described here is basically categories for Python. I've implemented > something like this before and I would be happy to see this added to > the standard library and formalised. If they are like Smalltalk categories, then they are semantically different. For method categories, you group the methods by "functionality" or "purpose" or "intended audience", as the developer of a class. It is important that the category has a name, and the categories are part of the class' documentation. For monkey-patching classes, the grouping is "by application". The additional/changed methods are not written by the author of the class, but by the author of the application using the class. The name of the group is irrelevant, and the documentation (if any exists) documents it as an application-specific thing. In-between are the partial classes of C# and the way C classes get implemented: it is the implementor of the class who fragments the class into separate pieces of code, but not for the sake of the class clients, but for maintainability of the class itself (so that you can spread a large class over several source files). They use same implementation strategy; for integrating it into the standard library, I think the usage scenarios need to be considered more. I like to see all three scenarios supported; to support categories properly, you need to also have support in the reflection, in pydoc, and perhaps also in the online documentation (if standard library classes get split into categories). Regards, Martin From nas at arctrix.com Sat Jan 19 19:53:42 2008 From: nas at arctrix.com (Neil Schemenauer) Date: Sat, 19 Jan 2008 18:53:42 +0000 (UTC) Subject: [Python-Dev] What to do for bytes in 2.6? References: <20080117221110.AFG13320@ms19.lnh.mail.rcn.net> <479062EE.4020805@cheimes.de> Message-ID: Guido van Rossum wrote: > bytes is an alias for str (not even a subclass) > b"" is an alias for "" One advantage of a subclass is that there could be a flag that warns about combining bytes and unicode data. For example, b"x" + u"y" would produce a warning. As someone who writes internationalized software, I would happly use both the byte designating syntax and the warning flag, even if I wasn't planning to move to Python 3. Neil From guido at python.org Sat Jan 19 20:32:46 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 19 Jan 2008 11:32:46 -0800 Subject: [Python-Dev] What to do for bytes in 2.6? In-Reply-To: References: <20080117221110.AFG13320@ms19.lnh.mail.rcn.net> <479062EE.4020805@cheimes.de> Message-ID: On Jan 19, 2008 10:53 AM, Neil Schemenauer wrote: > Guido van Rossum wrote: > > bytes is an alias for str (not even a subclass) > > b"" is an alias for "" > > One advantage of a subclass is that there could be a flag that warns > about combining bytes and unicode data. For example, b"x" + u"y" > would produce a warning. As someone who writes internationalized > software, I would happly use both the byte designating syntax and > the warning flag, even if I wasn't planning to move to Python 3. Yeah, that's what everybody thinks at first -- but the problem is that most of the time (in 2.6 anyway) the "bytes" object is something read from a socket, for example, not something created from a b"" literal. There is no way to know whether that return value means text or data (plenty of apps legitimately read text straight off a socket in 2.x), so the socket object can't return a bytes instance, and hence the warning won't trigger. Really, the pure aliasing solution is just about optimal in terms of bang per buck. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jyasskin at gmail.com Sat Jan 19 21:06:56 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Sat, 19 Jan 2008 12:06:56 -0800 Subject: [Python-Dev] Rational approximation methods Message-ID: <5d44f72f0801191206x1027a49do62376026727fbb@mail.gmail.com> In the Rational class that I've recently checked into Python 2.6 (http://bugs.python.org/issue1682), it might be nice to provide a method that, given a particular rational number, returns a nearby number that's nicer in some way. I know of two reasonable behaviors for this operation. Since I don't know which is likely to be more useful, or even if either is useful enough to include in the first version of the module, I'm coming to you guys for advice. Both were implemented a while ago in http://svn.python.org/view/sandbox/trunk/rational/Rational.py?rev=40988&view=markup and are based on the continued fraction representation of the number (http://en.wikipedia.org/wiki/Continued_fraction#Best_rational_approximations). The first returns the closest rational whose denominator is less than a given integer. For discussion, we might call it .limit_denominator(), although I'm also looking for good names. This seems useful for computations in which you want to sacrifice some accuracy for speed. On the other hand, Decimal may fill the niche for fast-ish computation that surprises people less than float. The second returns the simplest rational within some distance. For instance, it'll prefer 22/7 over 333/106 if both are close enough. We might call it .simplest_within() for now. This seems useful for converting from float and displaying results to users, where we prefer readability over accuracy or have reason to believe that a simpler fraction might be more correct. What does the list think? Jeffrey From Scott.Daniels at Acm.Org Sat Jan 19 23:35:50 2008 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Sat, 19 Jan 2008 14:35:50 -0800 Subject: [Python-Dev] Rational approximation methods In-Reply-To: <5d44f72f0801191206x1027a49do62376026727fbb@mail.gmail.com> References: <5d44f72f0801191206x1027a49do62376026727fbb@mail.gmail.com> Message-ID: Jeffrey Yasskin wrote: > The second returns the simplest rational within some distance. For > instance, it'll prefer 22/7 over 333/106 if both are close enough. We > might call it .simplest_within() for now. This seems useful for > converting from float and displaying results to users, where we prefer > readability over accuracy or have reason to believe that a simpler > fraction might be more correct. You can use a mediant walk to get to two surrounding fractions w/ some limit, and then return the pair to let the caller choose. See http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/52317 for an imperfect implementation. From amk at amk.ca Sun Jan 20 02:14:36 2008 From: amk at amk.ca (A.M. Kuchling) Date: Sat, 19 Jan 2008 20:14:36 -0500 Subject: [Python-Dev] Bug Day outcome Message-ID: <20080120011436.GA26802@amk.local> Today's bug day was a great success. Experienced people like Georg, Facundo, and Gregory P. Smith participated, and we also had people who submitted their first patches, some of which got applied today, too. Hopefully we'll see those people again. As of this writing, 37 issues were closed today, leaving 1300 open. (I saved a search in Roundup to list the bugs closed today.) --amk From dickinsm at gmail.com Sun Jan 20 02:22:40 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Sat, 19 Jan 2008 20:22:40 -0500 Subject: [Python-Dev] Rational approximation methods In-Reply-To: <5d44f72f0801191206x1027a49do62376026727fbb@mail.gmail.com> References: <5d44f72f0801191206x1027a49do62376026727fbb@mail.gmail.com> Message-ID: <5c6f2a5d0801191722w6394e933y4da0b6ca6786c8d5@mail.gmail.com> On Jan 19, 2008 3:06 PM, Jeffrey Yasskin wrote: > In the Rational class that I've recently checked into Python 2.6 > (http://bugs.python.org/issue1682), it might be nice to provide a > method that, given a particular rational number, returns a nearby > number that's nicer in some way. I know of two reasonable behaviors > [...] > What does the list think? > I'm having trouble imagining a real use case for either of these methods. When I'm using Rational, it's almost certainly because I want exact arithmetic; if I'm getting results with huge denominators and numerators then either that's what I really want, or I probably shouldn't have been using Rational in the first place... The only semi-use case I see is providing some way for users to turn the float 1.1 into the rational 11/10, instead of Rational(2476979795053773L,2251799813685248L). So I'd say leave both these methods out for now---it would be easy enough to add them later if it turns out there's a real need. Just my 200/391 UK pennies :) Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080119/1ab61ba1/attachment.htm From glyph at divmod.com Sun Jan 20 02:54:48 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Sun, 20 Jan 2008 01:54:48 -0000 Subject: [Python-Dev] What to do for bytes in 2.6? In-Reply-To: References: <20080117221110.AFG13320@ms19.lnh.mail.rcn.net> <479062EE.4020805@cheimes.de> Message-ID: <20080120015448.21558.742223331.divmod.xquotient.5046@joule.divmod.com> On 19 Jan, 07:32 pm, guido at python.org wrote: >There is no way to know whether that return value means text or data >(plenty of apps legitimately read text straight off a socket in 2.x), IMHO, this is a stretch of the word "legitimately" ;-). If you're reading from a socket, what you're getting are bytes, whether they're represented by str() or bytes(); correct code in 2.x must currently do a .decode("ascii") or .decode("charmap") to "legitimately" identify the result as text of some kind. Now, ad-hoc code with a fast and loose definition of "text" can still read arrays of bytes off a socket without specifying an encoding and get away with it, but that's because Python's unicode implementation has thus far been very forgiving, not because the data is cleanly text yet. Why can't we get that warning in -3 mode just the same from something read from a socket and a b"" literal? I've written lots of code that aggressively rejects str() instances as text, as well as unicode instances as bytes, and that's in code that still supports 2.3 ;). >Really, the pure aliasing solution is just about optimal in terms of >bang per buck. :-) Not that I'm particularly opposed to the aliasing solution, either. It would still allow writing code that was perfectly useful in 2.6 as well as 3.0, and it would avoid disturbing code that did checks of type(""). It would just remove an opportunity to get one potentially helpful warning. From guido at python.org Sun Jan 20 05:26:43 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 19 Jan 2008 20:26:43 -0800 Subject: [Python-Dev] What to do for bytes in 2.6? In-Reply-To: <20080120015448.21558.742223331.divmod.xquotient.5046@joule.divmod.com> References: <20080117221110.AFG13320@ms19.lnh.mail.rcn.net> <479062EE.4020805@cheimes.de> <20080120015448.21558.742223331.divmod.xquotient.5046@joule.divmod.com> Message-ID: On Jan 19, 2008 5:54 PM, wrote: > On 19 Jan, 07:32 pm, guido at python.org wrote: > >There is no way to know whether that return value means text or data > >(plenty of apps legitimately read text straight off a socket in 2.x), > > IMHO, this is a stretch of the word "legitimately" ;-). If you're > reading from a socket, what you're getting are bytes, whether they're > represented by str() or bytes(); correct code in 2.x must currently do a > .decode("ascii") or .decode("charmap") to "legitimately" identify the > result as text of some kind. > > Now, ad-hoc code with a fast and loose definition of "text" can still > read arrays of bytes off a socket without specifying an encoding and get > away with it, but that's because Python's unicode implementation has > thus far been very forgiving, not because the data is cleanly text yet. I would say that depends on the application, and on arrangements that client and server may have made off-line about the encoding. In 2.x, text can legitimately be represented as str -- there's even the locale module to further specify how it is to be interpreted as characters. Sure, this doesn't work for full unicode, and it doesn't work for all protocols used with sockets, but claiming that only fast and loose code ever uses str to represent text is quite far from reality -- this would be saying that the locale module is only for quick and dirty code, which just ain't so. > Why can't we get that warning in -3 mode just the same from something > read from a socket and a b"" literal? If you really want this, please think through all the consequences, and report back here. While I have a hunch that it'll end up giving too many false positives and at the same time too many false negatives, perhaps I haven't thought it through enough. But if you really think this'll be important for you, I hope you'll be willing to do at least some of the thinking. I believe that a constraint should be that by default (without -3 or a __future__ import) str and bytes should be the same thing. Or, another way of looking at this, reads from binary files and reads from sockets (and other similar things, like ctypes and mmap and the struct module, for example) should return str instances, not instances of a str subclass by default -- IMO returning a subclass is bound to break too much code. (Remember that there is still *lots* of code out there that uses "type(x) is types.StringType)" rather than "isinstance(x, str)", and while I'd be happy to warn about that in -3 mode if we could, I think it's unacceptable to break that in the default environment -- let it break in 3.0 instead.) > I've written lots of code that > aggressively rejects str() instances as text, as well as unicode > instances as bytes, and that's in code that still supports 2.3 ;). Yeah, well, but remember, while keeping you happy is high on my list of priorities, it's not the only priority. :-) > >Really, the pure aliasing solution is just about optimal in terms of > >bang per buck. :-) > > Not that I'm particularly opposed to the aliasing solution, either. It > would still allow writing code that was perfectly useful in 2.6 as well > as 3.0, and it would avoid disturbing code that did checks of type(""). Right. > It would just remove an opportunity to get one potentially helpful > warning. I worry that the warning wouldn't come often enough, and that too often it would be unhelpful. There will inevitably be some stuff where you just have to try to convert the code using 2to3 and try to run it under 3.0 in order to see if it works. And there's also the concern of those who want to use 2.6 because it offers 2.5 compatibility plus a fair number of new features, but who aren't interested (yet) in moving up to 3.0. I expect that Google will initially be in this category too. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sun Jan 20 05:33:58 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 19 Jan 2008 20:33:58 -0800 Subject: [Python-Dev] Bug Day outcome In-Reply-To: <20080120011436.GA26802@amk.local> References: <20080120011436.GA26802@amk.local> Message-ID: On Jan 19, 2008 5:14 PM, A.M. Kuchling wrote: > Today's bug day was a great success. Experienced people like Georg, > Facundo, and Gregory P. Smith participated, and we also had people who > submitted their first patches, some of which got applied today, too. > Hopefully we'll see those people again. > > As of this writing, 37 issues were closed today, leaving 1300 open. > (I saved a search in Roundup to list the bugs closed today.) Thanks Andrew, and everyone else who participated! I've been doing other things, but have seen the checkins go by and am very happy with the success of the day. To everyone who participated for the first time (raise your hand if you're not subscribed to python-dev yet :-) -- welcome, and I hope you'll be back for more! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From glyph at divmod.com Sun Jan 20 08:49:56 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Sun, 20 Jan 2008 07:49:56 -0000 Subject: [Python-Dev] What to do for bytes in 2.6? In-Reply-To: References: <20080117221110.AFG13320@ms19.lnh.mail.rcn.net> <479062EE.4020805@cheimes.de> <20080120015448.21558.742223331.divmod.xquotient.5046@joule.divmod.com> Message-ID: <20080120074956.21558.1110044057.divmod.xquotient.5336@joule.divmod.com> On 04:26 am, guido at python.org wrote: >On Jan 19, 2008 5:54 PM, wrote: >>On 19 Jan, 07:32 pm, guido at python.org wrote: Starting with the most relevant bit before getting off into digressions that may not interest most people: >>Why can't we get that warning in -3 mode just the same from something >>read from a socket and a b"" literal? >If you really want this, please think through all the consequences, >and report back here. While I have a hunch that it'll end up giving >too many false positives and at the same time too many false >negatives, perhaps I haven't thought it through enough. But if you >really think this'll be important for you, I hope you'll be willing to >do at least some of the thinking. While I stand by my statement that unicode is the Right Way to do text in python, this particular feature isn't really that important, and I can see there are cases where it might cause problems or make life more difficult. I suspect that I won't really know whether I want the warning anyway before I've actually tried to port any nuanced, real text-processing code to 3.0, and it looks like it's going to be a little while before that happens. I suspect that if I do want the warning, it would be a feature for 2.7, not 2.6, so I don't want to waste a lot of everyone's time advocating for it. Now for a nearly irrelevant digression (please feel free to stop reading here): >>Now, ad-hoc code with a fast and loose definition of "text" can still >>read arrays of bytes off a socket without specifying an encoding and >>get >>away with it, but that's because Python's unicode implementation has >>thus far been very forgiving, not because the data is cleanly text >>yet. > >I would say that depends on the application, and on arrangements that >client and server may have made off-line about the encoding. I can see your point. I think it probably holds better on files and streams than on sockets, though - please forgive me if I don't think that server applications which require environment-dependent out-of-band arrangements about locale are correct :). >In 2.x, text can legitimately be represented as str -- there's even >the locale module to further specify how it is to be interpreted as >characters. I'm aware that this specific example is kind of a ridiculous stretch, but it's the first one that came to mind. Consider len(u'?'.encode('utf-8').rjust(5).decode('utf-8')). Of course unicode.rjust() won't do the right thing in the case of surrogate pairs, not to mention RTL text, but it still handles a lot more cases than str.rjust(), since code points behave a lot more like characters than code units do. >Sure, this doesn't work for full unicode, and it doesn't work for all >protocols used with sockets, but claiming that only fast and loose >code ever uses str to represent text is quite far from reality -- this >would be saying that the locale module is only for quick and dirty >code, which just ain't so. It would definitely be overreaching to say all code that uses str is quick and dirty. But I do think that it fits into one of two categories: quick and dirty, or legacy. locale is an example of a legacy case for which there is no replacement (that I'm aware of). Even if I were writing a totally unicode-clean application, as far as I'm aware, there's no common replacement for i.e. locale.currency(). Still, locale is limiting. It's ... uncomfortable to call locale.currency() in a multi-user server process. It would be nice if there were a replacement that completely separated encoding issues from localization issues. >I believe that a constraint should be that by default (without -3 or a >__future__ import) str and bytes should be the same thing. Or, another >way of looking at this, reads from binary files and reads from sockets >(and other similar things, like ctypes and mmap and the struct module, >for example) should return str instances, not instances of a str >subclass by default -- IMO returning a subclass is bound to break too >much code. (Remember that there is still *lots* of code out there that >uses "type(x) is types.StringType)" rather than "isinstance(x, str)", >and while I'd be happy to warn about that in -3 mode if we could, I >think it's unacceptable to break that in the default environment -- >let it break in 3.0 instead.) I agree. But, it's precisely because this is so subtle that it would be nice to have tools which would report warnings to help fix it. *Certainly* by default, everywhere that's "str" in 2.5 should be "str" in 2.6. Probably even in -3 mode, if the goal there is "warnings only". However, the feature still strikes me as potentially useful while porting. If I were going to advocate for it, though, it would be as a separate option, e.g. "--separate-bytes-type". I say this as separate from just trying to run the code on 3.0 to see what happens because it seems like the most subtle and difficult aspect of the port to get right; it would be nice to be able to tweak it individually, without the other issues related to 3.0. For example, some of the code I work on has a big stack of dependencies. Some of those are in C, most of them don't process text at all. However, most of them aren't going to port to 3.0 very early, but it would be good to start running in as 3.0-like of an environment as possible earlier than that so that the hard stuff is done by the time the full stack has been migrated. >>I've written lots of code that >>aggressively rejects str() instances as text, as well as unicode >>instances as bytes, and that's in code that still supports 2.3 ;). > >Yeah, well, but remember, while keeping you happy is high on my list Thanks, good to hear :) >of priorities, it's not the only priority. :-) I don't think it's even my fianc?e's *only* priority, and I think it should stay higher on her list than yours ;-). From qgallet at gmail.com Sun Jan 20 11:23:38 2008 From: qgallet at gmail.com (Quentin Gallet-Gilles) Date: Sun, 20 Jan 2008 11:23:38 +0100 Subject: [Python-Dev] Bug Day outcome In-Reply-To: <20080120011436.GA26802@amk.local> References: <20080120011436.GA26802@amk.local> Message-ID: <8b943f2b0801200223g3b619eb1w10d0543e44447fe@mail.gmail.com> Excellent ! When will be the next one ? :-) On Jan 20, 2008 2:14 AM, A.M. Kuchling wrote: > Today's bug day was a great success. Experienced people like Georg, > Facundo, and Gregory P. Smith participated, and we also had people who > submitted their first patches, some of which got applied today, too. > Hopefully we'll see those people again. > > As of this writing, 37 issues were closed today, leaving 1300 open. > (I saved a search in Roundup to list the bugs closed today.) > > --amk > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/qgallet%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080120/72ee261e/attachment.htm From lists at cheimes.de Sun Jan 20 12:18:53 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 20 Jan 2008 12:18:53 +0100 Subject: [Python-Dev] Bug Day outcome In-Reply-To: <8b943f2b0801200223g3b619eb1w10d0543e44447fe@mail.gmail.com> References: <20080120011436.GA26802@amk.local> <8b943f2b0801200223g3b619eb1w10d0543e44447fe@mail.gmail.com> Message-ID: <47932E1D.2050902@cheimes.de> Quentin Gallet-Gilles wrote: > Excellent ! When will be the next one ? :-) Everyday can be a bug day. :) Nobody is going to stop you from squalling through the bug tracker. Christian From lists at cheimes.de Sun Jan 20 12:18:53 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 20 Jan 2008 12:18:53 +0100 Subject: [Python-Dev] Bug Day outcome In-Reply-To: <8b943f2b0801200223g3b619eb1w10d0543e44447fe@mail.gmail.com> References: <20080120011436.GA26802@amk.local> <8b943f2b0801200223g3b619eb1w10d0543e44447fe@mail.gmail.com> Message-ID: <47932E1D.2050902@cheimes.de> Quentin Gallet-Gilles wrote: > Excellent ! When will be the next one ? :-) Everyday can be a bug day. :) Nobody is going to stop you from squalling through the bug tracker. Christian From p.f.moore at gmail.com Sun Jan 20 14:28:27 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 20 Jan 2008 13:28:27 +0000 Subject: [Python-Dev] Rational approximation methods In-Reply-To: <5d44f72f0801191206x1027a49do62376026727fbb@mail.gmail.com> References: <5d44f72f0801191206x1027a49do62376026727fbb@mail.gmail.com> Message-ID: <79990c6b0801200528i3b25cfaal80cd574b5669099@mail.gmail.com> On 19/01/2008, Jeffrey Yasskin wrote: > The first returns the closest rational whose denominator is less than > a given integer. [...] > The second returns the simplest rational within some distance. Both of these are likely to be of limited use. The most common usage I know of is to make a "sensible" rational from a float (i.e., a DWIM style conversion 0.1 -> 1/10) or to provide readable output. On the other hand, both are subtle to implement, so having a standard implementation saves people having to code (and debug) their own. I suspect that simplest within a particular distance is the form that is most often wanted, but I don't have any good evidence for that feeling. Putting both in might help people to realise that there *is* a choice. Given that you have implemented them, I'd say leave them in. And I like the names trim and approximate, as given in the code you referenced. Paul. From aahz at pythoncraft.com Sun Jan 20 15:56:59 2008 From: aahz at pythoncraft.com (Aahz) Date: Sun, 20 Jan 2008 06:56:59 -0800 Subject: [Python-Dev] What to do for bytes in 2.6? In-Reply-To: References: <20080117221110.AFG13320@ms19.lnh.mail.rcn.net> <479062EE.4020805@cheimes.de> <20080120015448.21558.742223331.divmod.xquotient.5046@joule.divmod.com> Message-ID: <20080120145659.GB4656@panix.com> On Sat, Jan 19, 2008, Guido van Rossum wrote: > > I believe that a constraint should be that by default (without -3 or a > __future__ import) str and bytes should be the same thing. Or, another > way of looking at this, reads from binary files and reads from sockets > (and other similar things, like ctypes and mmap and the struct module, > for example) should return str instances, not instances of a str > subclass by default -- IMO returning a subclass is bound to break too > much code. This makes perfect sense to me. And yet, I also like the idea that b""+u"" raises an exception. I have a suggestion, then: for 2.6, let's make bytes a subclass of string whose only difference is that it contains a flag. I don't care whether b""+u"" raises an exception. This should be enough for people on an accelerated 3.0 conversion schedule, and they can write their own test for the flag if they care. For 2.7, we can start tightening up. b""+u"" can raise an exception, and the socket module might use a RETURN_BYTES variable. To put it another way, I don't think we should consider 2.6 the stopping point for 2.x/3.x conversion help. There's nothing wrong with planning for features to go into 2.7 -- just as several PEPs in the past have had multi-version planning. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "All problems in computer science can be solved by another level of indirection." --Butler Lampson From helmert at informatik.uni-freiburg.de Sun Jan 20 13:09:46 2008 From: helmert at informatik.uni-freiburg.de (Malte Helmert) Date: Sun, 20 Jan 2008 13:09:46 +0100 Subject: [Python-Dev] Bug Day outcome In-Reply-To: <47932E1D.2050902@cheimes.de> References: <20080120011436.GA26802@amk.local> <8b943f2b0801200223g3b619eb1w10d0543e44447fe@mail.gmail.com> <47932E1D.2050902@cheimes.de> Message-ID: Christian Heimes wrote: > Quentin Gallet-Gilles wrote: >> Excellent ! When will be the next one ? :-) > > Everyday can be a bug day. :) > Nobody is going to stop you from squalling through the bug tracker. I enjoyed the bug day a lot and will certainly come back for more when I can spare a little time. One question there: Will the "easy" keyword in roundup be maintained further, i.e. will new easy bugs be marked in the future? That would be very useful for neophyte contributors. Malte From lists at cheimes.de Sun Jan 20 17:18:07 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 20 Jan 2008 17:18:07 +0100 Subject: [Python-Dev] Bug Day outcome In-Reply-To: References: <20080120011436.GA26802@amk.local> <8b943f2b0801200223g3b619eb1w10d0543e44447fe@mail.gmail.com> <47932E1D.2050902@cheimes.de> Message-ID: Malte Helmert wrote: > One question there: Will the "easy" keyword in roundup be maintained > further, i.e. will new easy bugs be marked in the future? That would be > very useful for neophyte contributors. Yes, we will keep marking easy bugs with the "easy" keyword. Christian From lists at janc.be Sun Jan 20 17:38:40 2008 From: lists at janc.be (Jan Claeys) Date: Sun, 20 Jan 2008 17:38:40 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: References: Message-ID: <1200847120.24656.83.camel@localhost> Op zaterdag 12-01-2008 om 00:27 uur [tijdzone +0100], schreef Christian Heimes: > Specification > ============= [...] > user configuration directory > > Usually the parent directory of the user site directory. It's meant > for Python version specific data like config files. > > Windows: %APPDATA%/Python/Python26 > Mac: ~/Library/Python/2.6 > Unix: ~/.local/lib/python2.6 What do you mean by "configuration directory"? IMHO configuration files on linux/unix should go into ~/.python2.6 or ~/.config/python2.6 or something like that? (I would never look in a directory named 'lib' for configuration files.) -- Jan Claeys From lists at janc.be Sun Jan 20 17:57:55 2008 From: lists at janc.be (Jan Claeys) Date: Sun, 20 Jan 2008 17:57:55 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <4789DDA0.4020900@cheimes.de> References: <47894171.7030004@cheimes.de> <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> <4789DDA0.4020900@cheimes.de> Message-ID: <1200848275.24656.95.camel@localhost> Op zondag 13-01-2008 om 10:45 uur [tijdzone +0100], schreef Christian Heimes: > Gregory P. Smith wrote: > > My main suggestion was going to be the ability to turn it off as you already > > mentioned. However, please consider leaving it off by default to avoid > > problems for installed python scripts importing user supplied code. For > > shared hosting environments where this becomes really useful users can > > easily add the -s (or whatever flag is chosen) to their programs > > themselves. I don't know what that'd mean on windows where #! lines don't > > exist. Yet another file extension to imply the flag (yuck)? A .cmd wrapper > > script to run python with the flag (ugh)? > > So you prefer to make the per use site-package directory an opt-in > option? I prefer it as an opt-out option. It's enabled by default, > unless the user disables the feature with -s. One possible issue with enabling it by default could be that less-experienced users can accidentally break the system configuration tools written in python used by several linux distros (Ubuntu, Gentoo, Fedora/Red Hat, ...). There should be a way for distro developers to make sure the users local 'site-packages' is *not* used when running those tools. I'd rather have to set/uncomment an environment variable on my system than having 100 "normal" users break their systems accidentally... ;-) -- Jan Claeys From lists at cheimes.de Sun Jan 20 18:00:31 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 20 Jan 2008 18:00:31 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <1200848275.24656.95.camel@localhost> References: <47894171.7030004@cheimes.de> <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> <4789DDA0.4020900@cheimes.de> <1200848275.24656.95.camel@localhost> Message-ID: Jan Claeys wrote: > There should be a way for distro developers to make sure the users local > 'site-packages' is *not* used when running those tools. There is an option. Those tools should use the -E and -s argument: #!/usr/bin/env python -E -s Christian From lists at cheimes.de Sun Jan 20 18:01:30 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 20 Jan 2008 18:01:30 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <1200847120.24656.83.camel@localhost> References: <1200847120.24656.83.camel@localhost> Message-ID: Jan Claeys wrote: > What do you mean by "configuration directory"? IMHO configuration files > on linux/unix should go into ~/.python2.6 or ~/.config/python2.6 or > something like that? It's already renamed in the PEP: http://www.python.org/dev/peps/pep-0370/#specification Christian From g.brandl at gmx.net Sun Jan 20 18:09:08 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 20 Jan 2008 18:09:08 +0100 Subject: [Python-Dev] Bug Day outcome In-Reply-To: <20080120011436.GA26802@amk.local> References: <20080120011436.GA26802@amk.local> Message-ID: A.M. Kuchling schrieb: > Today's bug day was a great success. Experienced people like Georg, > Facundo, and Gregory P. Smith participated, and we also had people who > submitted their first patches, some of which got applied today, too. > Hopefully we'll see those people again. > > As of this writing, 37 issues were closed today, leaving 1300 open. > (I saved a search in Roundup to list the bugs closed today.) Since I was a bit incapacitated yesterday, I continued the bug day on my own and closed 17 more issues today. Sadly, the number of open issues is now 1715 since Martin imported those from SF.net which weren't in the export in the first round. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From asmodai at in-nomine.org Sun Jan 20 18:10:03 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Sun, 20 Jan 2008 18:10:03 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <20080115122828.GD14063@phd.pp.ru> References: <478B50B4.3040809@livinglogic.de> <20080114141539.21558.782173562.divmod.xquotient.4580@joule.divmod.com> <478BABD9.6070206@cheimes.de> <20080114191326.21558.1896823221.divmod.xquotient.4614@joule.divmod.com> <20080114234147.GO30947@snowy.squish.net> <20080115112413.GA13674@phd.pp.ru> <2e1434c10801150357i247cf03dje874a36351c89c22@mail.gmail.com> <478CA55A.9010606@cheimes.de> <20080115122828.GD14063@phd.pp.ru> Message-ID: <20080120171003.GF61556@nexus.in-nomine.org> -On [20080116 07:15], Oleg Broytmann (phd at phd.pp.ru) wrote: > The site only mentions $HOME/.local/share, there is no $HOME/.local/bin >at the site. As was mentioned earlier in the thread. Only $HOME/.local/share was in the Free Desktop specification. The bin is something that got introduced in this thread. It makes sense to stick to hier(7) for this sort of thing. Although the entire concept of a $HOME/.local was new to me as well and is not mandated at all in POSIX or other circles aside from, apparently, the Free Desktop folks. Pendantic note: ~ is an expansion character, the correct variable to talk about is HOME (see IEEE Std 1003.1, 2004 section 2.5.3 and 2.6.1). -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ Peace comes from within. Do not seek it without. - Buddha From phd at phd.pp.ru Sun Jan 20 18:12:38 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Sun, 20 Jan 2008 20:12:38 +0300 Subject: [Python-Dev] #! magic (was PEP: per user site-packages directory) In-Reply-To: References: <47894171.7030004@cheimes.de> <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> <4789DDA0.4020900@cheimes.de> <1200848275.24656.95.camel@localhost> Message-ID: <20080120171238.GA16055@phd.pp.ru> On Sun, Jan 20, 2008 at 06:00:31PM +0100, Christian Heimes wrote: > #!/usr/bin/env python -E -s On most Unicies #! magic may have only one parameter after the program; the program here is env, the parameter is python, and that's all. Adding python options will result in different errors - some platforms silently ignores the options, some reports an error, some tries to find "python -E -s" in the PATH and report "Bad command or file name". Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From asmodai at in-nomine.org Sun Jan 20 18:25:57 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Sun, 20 Jan 2008 18:25:57 +0100 Subject: [Python-Dev] #! magic (was PEP: per user site-packages directory) In-Reply-To: <20080120171238.GA16055@phd.pp.ru> References: <47894171.7030004@cheimes.de> <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> <4789DDA0.4020900@cheimes.de> <1200848275.24656.95.camel@localhost> <20080120171238.GA16055@phd.pp.ru> Message-ID: <20080120172557.GH61556@nexus.in-nomine.org> -On [20080120 18:12], Oleg Broytmann (phd at phd.pp.ru) wrote: > On most Unicies #! magic may have only one parameter after the program; >the program here is env, the parameter is python, and that's all. Adding >python options will result in different errors - some platforms silently >ignores the options, some reports an error, some tries to find "python -E -s" >in the PATH and report "Bad command or file name". IEEE Std 1003.1, 2004: The shell reads its input from a file (see sh), from the -c option or from the system() and popen() functions defined in the System Interfaces volume of IEEE Std 1003.1-2001. If the first line of a file of shell commands starts with the characters "#!", the results are unspecified. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ I dream of Love as Time runs through my hand... From phd at phd.pp.ru Sun Jan 20 18:38:28 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Sun, 20 Jan 2008 20:38:28 +0300 Subject: [Python-Dev] #! magic In-Reply-To: <20080120172557.GH61556@nexus.in-nomine.org> References: <47894171.7030004@cheimes.de> <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> <4789DDA0.4020900@cheimes.de> <1200848275.24656.95.camel@localhost> <20080120171238.GA16055@phd.pp.ru> <20080120172557.GH61556@nexus.in-nomine.org> Message-ID: <20080120173828.GB16055@phd.pp.ru> On Sun, Jan 20, 2008 at 06:25:57PM +0100, Jeroen Ruigrok van der Werven wrote: > -On [20080120 18:12], Oleg Broytmann (phd at phd.pp.ru) wrote: > > On most Unicies #! magic may have only one parameter after the program; > >the program here is env, the parameter is python, and that's all. Adding > >python options will result in different errors - some platforms silently > >ignores the options, some reports an error, some tries to find "python -E -s" > >in the PATH and report "Bad command or file name". > > IEEE Std 1003.1, 2004: > > The shell reads its input A shell has nothing to do with it as it is the OS (exec system call) that upon reading the magic of the file sees #! and executes the program (up to the first space) and pass to the program the first (and the only) parameter. #! /usr/bin/env python -O [trying to execute the script on Linux] /usr/bin/env: python -O: No such file or directory Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From lists at janc.be Sun Jan 20 18:46:31 2008 From: lists at janc.be (Jan Claeys) Date: Sun, 20 Jan 2008 18:46:31 +0100 Subject: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57 In-Reply-To: <478D5ED4.4060208@cheimes.de> References: <20080115201812.GA25987@phd.pp.ru> <87lk6q3hdt.fsf@uwakimon.sk.tsukuba.ac.jp> <20080115213831.GA28333@phd.pp.ru> <08Jan15.172353pst."58696"@synergy1.parc.xerox.com> <478D5ED4.4060208@cheimes.de> Message-ID: <1200851191.28915.4.camel@localhost> Op woensdag 16-01-2008 om 02:33 uur [tijdzone +0100], schreef Christian Heimes: > Bill Janssen wrote: > > Good point, but I prefer ~/Library/Python to either of these. > > ~/Library/ is a Mac OS X thing. I haven't seen it on other Unix systems. There is (at least) one linux distro using it, but it's not very well-known. > I *could* add yet another environment variable PYTHONUSERHOME to set the > base path but I prefer not. A solution might be a way for python distributors to set the default location , and then provide PYTHONUSERHOME for stubborn users to override that. ;) -- Jan Claeys From phd at phd.pp.ru Sun Jan 20 18:46:57 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Sun, 20 Jan 2008 20:46:57 +0300 Subject: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57 In-Reply-To: <1200851191.28915.4.camel@localhost> References: <20080115201812.GA25987@phd.pp.ru> <87lk6q3hdt.fsf@uwakimon.sk.tsukuba.ac.jp> <20080115213831.GA28333@phd.pp.ru> <08Jan15.172353pst."58696"@synergy1.parc.xerox.com> <478D5ED4.4060208@cheimes.de> <1200851191.28915.4.camel@localhost> Message-ID: <20080120174657.GC16055@phd.pp.ru> On Sun, Jan 20, 2008 at 06:46:31PM +0100, Jan Claeys wrote: > Op woensdag 16-01-2008 om 02:33 uur [tijdzone +0100], schreef Christian > Heimes: > > Bill Janssen wrote: > > > Good point, but I prefer ~/Library/Python to either of these. > > > > ~/Library/ is a Mac OS X thing. I haven't seen it on other Unix systems. > > There is (at least) one linux distro using it, but it's not very > well-known. Gobo Linux? Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From lists at janc.be Sun Jan 20 19:23:46 2008 From: lists at janc.be (Jan Claeys) Date: Sun, 20 Jan 2008 19:23:46 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: References: <1200847120.24656.83.camel@localhost> Message-ID: <1200853426.28915.7.camel@localhost> Op zondag 20-01-2008 om 18:01 uur [tijdzone +0100], schreef Christian Heimes: > Jan Claeys wrote: > > What do you mean by "configuration directory"? IMHO configuration files > > on linux/unix should go into ~/.python2.6 or ~/.config/python2.6 or > > something like that? > > It's already renamed in the PEP: > http://www.python.org/dev/peps/pep-0370/#specification So this is stuff that should never be changed by the user? -- Jan Claeys From lists at cheimes.de Sun Jan 20 19:30:03 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 20 Jan 2008 19:30:03 +0100 Subject: [Python-Dev] #! magic In-Reply-To: <20080120173828.GB16055@phd.pp.ru> References: <47894171.7030004@cheimes.de> <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> <4789DDA0.4020900@cheimes.de> <1200848275.24656.95.camel@localhost> <20080120171238.GA16055@phd.pp.ru> <20080120172557.GH61556@nexus.in-nomine.org> <20080120173828.GB16055@phd.pp.ru> Message-ID: <4793932B.9010106@cheimes.de> Oleg Broytmann wrote: > A shell has nothing to do with it as it is the OS (exec system call) > that upon reading the magic of the file sees #! and executes the program > (up to the first space) and pass to the program the first (and the only) > parameter. > > #! /usr/bin/env python -O > > [trying to execute the script on Linux] > > /usr/bin/env: python -O: No such file or directory > > Oleg. Oh right. I was sure that I've seen a shebang with options somewhere. It might have been a shell script or a perl script. Yet another python executable could solve the issue, named "pythons" as python secure. /* gcc -DNDEBUG -g -O2 -Wall -Wstrict-prototypes -IInclude -I. -pthread -Xlinker -lpthread -ldl -lutil -lm -export-dynamic -o pythons2.6 pythons.c libpython2.6.a */ #include "Python.h" int main(int argc, char **argv) { /* disable some possible harmful features */ Py_IgnoreEnvironmentFlag++; Py_NoUserSiteDirectory++; Py_InteractiveFlag -= INT_MAX; Py_InspectFlag -= INT_MAX; return Py_Main(argc, argv); } $ ./pythons2.6 Python 2.6a0 (:59956M, Jan 14 2008, 22:09:17) [GCC 4.2.1 (Ubuntu 4.2.1-5ubuntu4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.flags sys.flags(debug=0, py3k_warning=0, division_warning=0, division_new=0, inspect=-2147483647, interactive=-2147483647, optimize=0, dont_write_bytecode=0, no_user_site=1, no_site=0, ingnore_environment=1, tabcheck=0, verbose=0, unicode=0) Christian From lists at janc.be Sun Jan 20 19:34:02 2008 From: lists at janc.be (Jan Claeys) Date: Sun, 20 Jan 2008 19:34:02 +0100 Subject: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57 In-Reply-To: <20080120174657.GC16055@phd.pp.ru> References: <20080115201812.GA25987@phd.pp.ru> <87lk6q3hdt.fsf@uwakimon.sk.tsukuba.ac.jp> <20080115213831.GA28333@phd.pp.ru> <08Jan15.172353pst."58696"@synergy1.parc.xerox.com> <478D5ED4.4060208@cheimes.de> <1200851191.28915.4.camel@localhost> <20080120174657.GC16055@phd.pp.ru> Message-ID: <1200854042.28915.12.camel@localhost> Op zondag 20-01-2008 om 20:46 uur [tijdzone +0300], schreef Oleg Broytmann: > On Sun, Jan 20, 2008 at 06:46:31PM +0100, Jan Claeys wrote: > > Op woensdag 16-01-2008 om 02:33 uur [tijdzone +0100], schreef Christian > > Heimes: > > > ~/Library/ is a Mac OS X thing. I haven't seen it on other Unix systems. > > > > There is (at least) one linux distro using it, but it's not very > > well-known. > > Gobo Linux? Now that I think about it, I'm not sure they use it under $HOME too, but I was thinking about that distro, yes, as they use such a scheme for the rest of their system (and it seems natural to do so throughout then). -- Jan Claeys From lists at cheimes.de Sun Jan 20 19:33:43 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 20 Jan 2008 19:33:43 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <20080120171003.GF61556@nexus.in-nomine.org> References: <478B50B4.3040809@livinglogic.de> <20080114141539.21558.782173562.divmod.xquotient.4580@joule.divmod.com> <478BABD9.6070206@cheimes.de> <20080114191326.21558.1896823221.divmod.xquotient.4614@joule.divmod.com> <20080114234147.GO30947@snowy.squish.net> <20080115112413.GA13674@phd.pp.ru> <2e1434c10801150357i247cf03dje874a36351c89c22@mail.gmail.com> <478CA55A.9010606@cheimes.de> <20080115122828.GD14063@phd.pp.ru> <20080120171003.GF61556@nexus.in-nomine.org> Message-ID: Jeroen Ruigrok van der Werven wrote: > Pendantic note: ~ is an expansion character, the correct variable to talk > about is HOME (see IEEE Std 1003.1, 2004 section 2.5.3 and 2.6.1). Most Python developers should the meaning of "~". Should I replace ~ with $HOME for those who don't have as much experience with Unix as we? Christian From lists at cheimes.de Sun Jan 20 19:37:01 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 20 Jan 2008 19:37:01 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <1200853426.28915.7.camel@localhost> References: <1200847120.24656.83.camel@localhost> <1200853426.28915.7.camel@localhost> Message-ID: Jan Claeys wrote: > So this is stuff that should never be changed by the user? ~/.local/lib/python2.6 has the same semantics as /usr/local/lib/python2.6 except it's a per user directory and not per machine. Christian From phd at phd.pp.ru Sun Jan 20 19:40:37 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Sun, 20 Jan 2008 21:40:37 +0300 Subject: [Python-Dev] #! magic In-Reply-To: <4793932B.9010106@cheimes.de> References: <47894171.7030004@cheimes.de> <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> <4789DDA0.4020900@cheimes.de> <1200848275.24656.95.camel@localhost> <20080120171238.GA16055@phd.pp.ru> <20080120172557.GH61556@nexus.in-nomine.org> <20080120173828.GB16055@phd.pp.ru> <4793932B.9010106@cheimes.de> Message-ID: <20080120184037.GA17310@phd.pp.ru> On Sun, Jan 20, 2008 at 07:30:03PM +0100, Christian Heimes wrote: > Oleg Broytmann wrote: > > #! /usr/bin/env python -O > > > > [trying to execute the script on Linux] > > > > /usr/bin/env: python -O: No such file or directory > > > > Oleg. > > Oh right. I was sure that I've seen a shebang with options somewhere. It > might have been a shell script or a perl script. Yes, one can write #! /usr/bin/python -O This works. The OS (exec system call) splits the string into a program (python) and a single parameter (-O), and python knows how to handle -O. The problem with #! /usr/bin/env python -O is that exec splits it into a program (env) and a single parameter (python -O) and env doesn't know how to run "python -O". > Yet another python executable could solve the issue I doubt it. It is env that doesn't know how to run "python -O" in my example, not python's problem. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From phd at phd.pp.ru Sun Jan 20 19:41:28 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Sun, 20 Jan 2008 21:41:28 +0300 Subject: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57 In-Reply-To: <1200854042.28915.12.camel@localhost> References: <20080115201812.GA25987@phd.pp.ru> <87lk6q3hdt.fsf@uwakimon.sk.tsukuba.ac.jp> <20080115213831.GA28333@phd.pp.ru> <08Jan15.172353pst."58696"@synergy1.parc.xerox.com> <478D5ED4.4060208@cheimes.de> <1200851191.28915.4.camel@localhost> <20080120174657.GC16055@phd.pp.ru> <1200854042.28915.12.camel@localhost> Message-ID: <20080120184128.GB17310@phd.pp.ru> On Sun, Jan 20, 2008 at 07:34:02PM +0100, Jan Claeys wrote: > Op zondag 20-01-2008 om 20:46 uur [tijdzone +0300], schreef Oleg > Broytmann: > > On Sun, Jan 20, 2008 at 06:46:31PM +0100, Jan Claeys wrote: > > > Op woensdag 16-01-2008 om 02:33 uur [tijdzone +0100], schreef Christian > > > Heimes: > > > > ~/Library/ is a Mac OS X thing. I haven't seen it on other Unix systems. > > > > > > There is (at least) one linux distro using it, but it's not very > > > well-known. > > > > Gobo Linux? > > Now that I think about it, I'm not sure they use it under $HOME too, but > I was thinking about that distro, yes, as they use such a scheme for the > rest of their system (and it seems natural to do so throughout then). I don't know if they have a scheme for $HOME. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From martin at v.loewis.de Sun Jan 20 19:42:35 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 20 Jan 2008 19:42:35 +0100 Subject: [Python-Dev] Priorities in the tracker Message-ID: <4793961B.6060603@v.loewis.de> After some months of tracker operation, I'd like to discuss one aspect of the tracker schema: priorities. Each issue has a severity and a priority. The severity is assigned by the submitter, defaults to normal, and indicates how serious the issue impacts him and the community. The priority is assigned by a developer (and cannot be set by a "mere" user), and indicates how quickly this issue must be processed. The priority is initially unset, requiring a developer to perform screening. It appears that developers rarely set the priority, leaving the issues formally unscreened. So what should we do? Leave things as-is? Drop the notion of priority? Change our process to make sure priorities do get set (and honored)? Regards, Martin From steve at holdenweb.com Sun Jan 20 19:53:03 2008 From: steve at holdenweb.com (Steve Holden) Date: Sun, 20 Jan 2008 13:53:03 -0500 Subject: [Python-Dev] Priorities in the tracker In-Reply-To: <4793961B.6060603@v.loewis.de> References: <4793961B.6060603@v.loewis.de> Message-ID: <4793988F.4070705@holdenweb.com> Martin v. L?wis wrote: > After some months of tracker operation, I'd like to discuss one aspect > of the tracker schema: priorities. > > Each issue has a severity and a priority. The severity is assigned by > the submitter, defaults to normal, and indicates how serious the issue > impacts him and the community. > > The priority is assigned by a developer (and cannot be set by > a "mere" user), and indicates how quickly this issue must be processed. > The priority is initially unset, requiring a developer to perform screening. > > It appears that developers rarely set the priority, leaving the issues > formally unscreened. > > So what should we do? Leave things as-is? Drop the notion of priority? > Change our process to make sure priorities do get set (and honored)? > It would appear useful to define some workflow aspects of issue processing, for example to ensure that priority gets set by a "responsible" reviewer. However whether such workflow should be imposed by automation or simply a discipline of development I'm not in a position to say. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From adlaiff6 at gmail.com Sun Jan 20 20:25:29 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Sun, 20 Jan 2008 14:25:29 -0500 Subject: [Python-Dev] Rational approximation methods In-Reply-To: <79990c6b0801200528i3b25cfaal80cd574b5669099@mail.gmail.com> References: <5d44f72f0801191206x1027a49do62376026727fbb@mail.gmail.com> <79990c6b0801200528i3b25cfaal80cd574b5669099@mail.gmail.com> Message-ID: <11b6d2f60801201125t2491be32n8523360b81d4f795@mail.gmail.com> On 1/20/08, Paul Moore wrote: > Both of these are likely to be of limited use. The most common usage I > know of is to make a "sensible" rational from a float (i.e., a DWIM > style conversion 0.1 -> 1/10) or to provide readable output. On the > other hand, both are subtle to implement, so having a standard > implementation saves people having to code (and debug) their own. > > I suspect that simplest within a particular distance is the form that > is most often wanted, but I don't have any good evidence for that > feeling. Putting both in might help people to realise that there *is* > a choice. > > Given that you have implemented them, I'd say leave them in. And I > like the names trim and approximate, as given in the code you > referenced. I can't think of a use case for these either, but the first (limit_denominator) seems like a generalized round, and I suppose might be useful for something. -- Cheers, Leif From brett at python.org Sun Jan 20 20:26:44 2008 From: brett at python.org (Brett Cannon) Date: Sun, 20 Jan 2008 11:26:44 -0800 Subject: [Python-Dev] Priorities in the tracker In-Reply-To: <4793961B.6060603@v.loewis.de> References: <4793961B.6060603@v.loewis.de> Message-ID: On Jan 20, 2008 10:42 AM, "Martin v. L?wis" wrote: > After some months of tracker operation, I'd like to discuss one aspect > of the tracker schema: priorities. > > Each issue has a severity and a priority. The severity is assigned by > the submitter, defaults to normal, and indicates how serious the issue > impacts him and the community. > > The priority is assigned by a developer (and cannot be set by > a "mere" user), and indicates how quickly this issue must be processed. > The priority is initially unset, requiring a developer to perform screening. > > It appears that developers rarely set the priority, leaving the issues > formally unscreened. > > So what should we do? Leave things as-is? Drop the notion of priority? > Change our process to make sure priorities do get set (and honored)? In my dream schema, severity becomes difficulty (e.g., easy, medium, hard, expert), type goes away, and priority changes to be release-based (e.g., RFE since those can occur whenever, next minor release, next micro release, immediately, etc.). I think priority should more be tied to our release cycles than this generic notion of importance. If it is important, we should try to get it done by a certain release. Let's have that fact directly reflected by the tracker. -Brett From g.brandl at gmx.net Sun Jan 20 20:26:55 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 20 Jan 2008 20:26:55 +0100 Subject: [Python-Dev] Priorities in the tracker In-Reply-To: <4793961B.6060603@v.loewis.de> References: <4793961B.6060603@v.loewis.de> Message-ID: Martin v. L?wis schrieb: > After some months of tracker operation, I'd like to discuss one aspect > of the tracker schema: priorities. > > Each issue has a severity and a priority. The severity is assigned by > the submitter, defaults to normal, and indicates how serious the issue > impacts him and the community. > > The priority is assigned by a developer (and cannot be set by > a "mere" user), and indicates how quickly this issue must be processed. > The priority is initially unset, requiring a developer to perform screening. > > It appears that developers rarely set the priority, leaving the issues > formally unscreened. > > So what should we do? Leave things as-is? Drop the notion of priority? > Change our process to make sure priorities do get set (and honored)? Christian currently does a good job of assigning the correct properties to new bugs. In any case, I'd prefer to keep a way to mark a bug as "high-priority" (meaning that it should be fixed before the next release) even if most of the bugs don't have an assigned priority. As to whether we need both severity and priority, I have no opinion. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From lists at cheimes.de Sun Jan 20 20:46:38 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 20 Jan 2008 20:46:38 +0100 Subject: [Python-Dev] Priorities in the tracker In-Reply-To: References: <4793961B.6060603@v.loewis.de> Message-ID: Georg Brandl wrote: > Christian currently does a good job of assigning the correct properties > to new bugs. In any case, I'd prefer to keep a way to mark a bug as > "high-priority" (meaning that it should be fixed before the next release) > even if most of the bugs don't have an assigned priority. I've defined the priorities for me: low = nice to have feature normal = well ... pretty normal :) high = important, investigate the bug before the next release gets out urgent = bug breaks development or is a show stopper immediate = security issue or major development stopper Christian From guido at python.org Sun Jan 20 22:04:44 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 20 Jan 2008 13:04:44 -0800 Subject: [Python-Dev] What to do for bytes in 2.6? In-Reply-To: <20080120145659.GB4656@panix.com> References: <20080117221110.AFG13320@ms19.lnh.mail.rcn.net> <479062EE.4020805@cheimes.de> <20080120015448.21558.742223331.divmod.xquotient.5046@joule.divmod.com> <20080120145659.GB4656@panix.com> Message-ID: On Jan 20, 2008 6:56 AM, Aahz wrote: > On Sat, Jan 19, 2008, Guido van Rossum wrote: > > > > I believe that a constraint should be that by default (without -3 or a > > __future__ import) str and bytes should be the same thing. Or, another > > way of looking at this, reads from binary files and reads from sockets > > (and other similar things, like ctypes and mmap and the struct module, > > for example) should return str instances, not instances of a str > > subclass by default -- IMO returning a subclass is bound to break too > > much code. > > This makes perfect sense to me. And yet, I also like the idea that > b""+u"" raises an exception. I have a suggestion, then: for 2.6, let's > make bytes a subclass of string whose only difference is that it contains > a flag. This still begs the question which standard APIs should return bytes. > I don't care whether b""+u"" raises an exception. This should > be enough for people on an accelerated 3.0 conversion schedule, and they > can write their own test for the flag if they care. Well, it being a subclass, it doesn't even need to have a flag, right? The class itself acts as a flag. But still, without support from built-in and standard library APIs I fear it's not going to be very useful. And fixing all the standard APIs to make the correct distinction is going to create exactly the ripple effect that Raymond so desperately wants to avoid -- and I agree, to the extent that rippling this through the stdlib is a waste of time from the stdlib's POV -- it's already been 3.0-ified. > For 2.7, we can start tightening up. b""+u"" can raise an exception, > and the socket module might use a RETURN_BYTES variable. > > To put it another way, I don't think we should consider 2.6 the stopping > point for 2.x/3.x conversion help. There's nothing wrong with planning > for features to go into 2.7 -- just as several PEPs in the past have had > multi-version planning. Personally, I very much doubt there will *be* a 2.7. I certainly don't expect to participate in its development the same way I am trying to steer 2.6. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jim at zope.com Sun Jan 20 22:10:24 2008 From: jim at zope.com (Jim Fulton) Date: Sun, 20 Jan 2008 16:10:24 -0500 Subject: [Python-Dev] [Python-3000] inst_persistent_id In-Reply-To: <20080114175943.GA26431@code0.codespeak.net> References: <20080114175943.GA26431@code0.codespeak.net> Message-ID: <0E953DBC-50C7-4234-9389-D2AE0515C1C4@zope.com> I took Python-3000 out of the cc list as I originally just wanted to make them aware of this issue. On Jan 14, 2008, at 12:59 PM, Armin Rigo wrote: > Hi, > > On Sat, Jan 12, 2008 at 07:33:38PM -0500, Alexandre Vassalotti wrote: >> Well, in Python 3K, inst_persistent_id() won't be usable, since >> PyInstance_Type was removed. > > Looking at the code, inst_persistent_id() is just a confusing name. > It > has got nothing to do with PyInstance_Type; it's called for any object > type that cPickle.c doesn't know how to handle. It has to do with instances in a more general sense. > In fact, it seems that > cPickle.c never calls inst_persistent_id() for objects of type > PyInstance_Type... Hm, good point. At the time, all I cared about were ExtensionClass instances, which were akin to modern-day new-style instances. It doesn't make sense to not call this function for classic instances. This optimization is fairly useful. I propose to update the logic to call this function for classic instances too. I'm also open to renaming it if that would make people happier. :) Thoughts? Jim -- Jim Fulton Zope Corporation From g.brandl at gmx.net Sun Jan 20 22:13:44 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 20 Jan 2008 22:13:44 +0100 Subject: [Python-Dev] Bug Day outcome In-Reply-To: <20080120011436.GA26802@amk.local> References: <20080120011436.GA26802@amk.local> Message-ID: A.M. Kuchling schrieb: > Today's bug day was a great success. Experienced people like Georg, > Facundo, and Gregory P. Smith participated, and we also had people who > submitted their first patches, some of which got applied today, too. > Hopefully we'll see those people again. > > As of this writing, 37 issues were closed today, leaving 1300 open. > (I saved a search in Roundup to list the bugs closed today.) Since this bug day was a relative success, I suggest to introduce a more-or-less regular schedule. Feb 23 would make a nice second bug day in 2008, wouldn't it? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From mithrandi-python-dev at mithrandi.za.net Sun Jan 20 23:17:20 2008 From: mithrandi-python-dev at mithrandi.za.net (Tristan Seligmann) Date: Mon, 21 Jan 2008 00:17:20 +0200 Subject: [Python-Dev] #! magic (was PEP: per user site-packages directory) In-Reply-To: <20080120171238.GA16055@phd.pp.ru> References: <47894171.7030004@cheimes.de> <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> <4789DDA0.4020900@cheimes.de> <1200848275.24656.95.camel@localhost> <20080120171238.GA16055@phd.pp.ru> Message-ID: <20080120221720.GB10699@mithrandi.za.net> * Oleg Broytmann [2008-01-20 20:12:38 +0300]: > On Sun, Jan 20, 2008 at 06:00:31PM +0100, Christian Heimes wrote: > > #!/usr/bin/env python -E -s > > On most Unicies #! magic may have only one parameter after the program; > the program here is env, the parameter is python, and that's all. Adding > python options will result in different errors - some platforms silently > ignores the options, some reports an error, some tries to find "python -E -s" > in the PATH and report "Bad command or file name". Distro tools shouldn't be using /usr/bin/env in any case. Does #!/usr/bin/python -E -s work? -- mithrandi, i Ainil en-Balandor, a faer Ambar -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: Digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20080121/f41a823b/attachment.pgp From stephen at xemacs.org Sun Jan 20 23:23:37 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 21 Jan 2008 07:23:37 +0900 Subject: [Python-Dev] Priorities in the tracker In-Reply-To: References: <4793961B.6060603@v.loewis.de> Message-ID: <87fxwsdtli.fsf@uwakimon.sk.tsukuba.ac.jp> Brett Cannon writes: > In my dream schema, severity becomes Then how does the user indicate how important it is to him? My severities (in an experimental roundup tracker I'm implementing) are 'inelegant', 'inconvenient', 'some work obstructed', 'much work obstructed', 'security', 'data loss', 'sudden death'. > difficulty (e.g., easy, medium, hard, expert), This is hard to classify with confidence, IMO. It also means different things: You could associate it with the set of developers qualified to handle it, or with the amount of time it would take an experienced developer to deal with it. These points of view are correlated, but I wouldn't be surprised if they have different effects on the workflow. > type goes away, I'm leaning to this myself, except that I think it is useful to inform non-developers that some things are organizationally hard. Eg, a "big RFE" will require a PEP, while a "big bug" just gets fixed ASAP. > and priority changes to be release-based (e.g., RFE since those can > occur whenever, next minor release, next micro release, > immediately, etc.). What you're talking about here is what a lot of projects call milestones. However, priority is a separate notion, especially for features. It indicates which features scheduled for the next milestone are most likely to get delayed to a later one. > I think priority should more be tied to our release cycles than this > generic notion of importance. If it is important, we should try to get > it done by a certain release. Let's have that fact directly reflected > by the tracker. Sure. Note that if you take everything I said literally and implement it, you get an unwieldy mess. Something's gotta go, I expect. But I think there are aspects to the "dream schema" you present that are not going to work as well as you would hope. From g.brandl at gmx.net Sun Jan 20 23:34:02 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 20 Jan 2008 23:34:02 +0100 Subject: [Python-Dev] #! magic (was PEP: per user site-packages directory) In-Reply-To: <20080120221720.GB10699@mithrandi.za.net> References: <47894171.7030004@cheimes.de> <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> <4789DDA0.4020900@cheimes.de> <1200848275.24656.95.camel@localhost> <20080120171238.GA16055@phd.pp.ru> <20080120221720.GB10699@mithrandi.za.net> Message-ID: Tristan Seligmann schrieb: > * Oleg Broytmann [2008-01-20 20:12:38 +0300]: > >> On Sun, Jan 20, 2008 at 06:00:31PM +0100, Christian Heimes wrote: >> > #!/usr/bin/env python -E -s >> >> On most Unicies #! magic may have only one parameter after the program; >> the program here is env, the parameter is python, and that's all. Adding >> python options will result in different errors - some platforms silently >> ignores the options, some reports an error, some tries to find "python -E -s" >> in the PATH and report "Bad command or file name". > > Distro tools shouldn't be using /usr/bin/env in any case. Does > > #!/usr/bin/python -E -s > > work? No, but #!/usr/bin/python -Es would. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From phd at phd.pp.ru Sun Jan 20 23:36:57 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Mon, 21 Jan 2008 01:36:57 +0300 Subject: [Python-Dev] #! magic In-Reply-To: <20080120221720.GB10699@mithrandi.za.net> References: <47894171.7030004@cheimes.de> <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> <4789DDA0.4020900@cheimes.de> <1200848275.24656.95.camel@localhost> <20080120171238.GA16055@phd.pp.ru> <20080120221720.GB10699@mithrandi.za.net> Message-ID: <20080120223657.GA19754@phd.pp.ru> On Mon, Jan 21, 2008 at 12:17:20AM +0200, Tristan Seligmann wrote: > * Oleg Broytmann [2008-01-20 20:12:38 +0300]: > > > On Sun, Jan 20, 2008 at 06:00:31PM +0100, Christian Heimes wrote: > > > #!/usr/bin/env python -E -s > > > > On most Unicies #! magic may have only one parameter after the program; > > the program here is env, the parameter is python, and that's all. Adding > > python options will result in different errors - some platforms silently > > ignores the options, some reports an error, some tries to find "python -E -s" > > in the PATH and report "Bad command or file name". > > Distro tools shouldn't be using /usr/bin/env in any case. Does > > #!/usr/bin/python -E -s > > work? No: Unknown option: - usage: /usr/bin/python [option] ... [-c cmd | -m mod | file | -] [arg] ... Try `python -h' for more information. -Es together work. I think this is why Unix has the tradition of combining options. PS. My python doesn't understand -s, so I tested a different options, but the result is the same. There are Unix variants that understand many options (I believe FreeBSD allows them) but most allow no more than one parameter in #!. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From amk at amk.ca Mon Jan 21 00:53:24 2008 From: amk at amk.ca (A.M. Kuchling) Date: Sun, 20 Jan 2008 18:53:24 -0500 Subject: [Python-Dev] Bug Day outcome In-Reply-To: <8b943f2b0801200223g3b619eb1w10d0543e44447fe@mail.gmail.com> References: <20080120011436.GA26802@amk.local> <8b943f2b0801200223g3b619eb1w10d0543e44447fe@mail.gmail.com> Message-ID: <20080120235324.GA27435@amk.local> On Sun, Jan 20, 2008 at 11:23:38AM +0100, Quentin Gallet-Gilles wrote: > Excellent ! When will be the next one ? :-) We could certainly try to hold one in February. Maybe on the 16th? For March the best date is probably the Monday of the sprints at PyCon, because there will be a Python core sprint going. --amk From goodger at python.org Mon Jan 21 02:35:22 2008 From: goodger at python.org (David Goodger) Date: Sun, 20 Jan 2008 20:35:22 -0500 Subject: [Python-Dev] PyCon 2008 Registration Open! Message-ID: <4793F6DA.5060802@python.org> I am pleased to announce that PyCon 2008 registration is now open! http://us.pycon.org/2008/registration/ Early-bird registration is open until February 20, so there's one month to register at the low rates. Regular on-line registration will be available from February 21 through March 7. PyCon 2008 will be held from March 13 through 20 in Chicago: * Tutorials: Thursday, March 13 * Conference: Friday, March 14, through Sunday, March 16 * Sprints: Monday, March 17 through Thursday, March 20 Register early, especially if you'd like to attend a tutorial or three, as tutorials fill up early. And yes, you heard that correctly -- this year, we have three tutorial sessions: morning, afternoon, and evening. There are 29 great tutorials to choose from. More about the tutorials soon, but for now, you can check out the descriptions yourself here: http://us.pycon.org/2008/tutorials/schedule/ The talk schedule is also up (in a preliminary form), although some changes are expected: http://us.pycon.org/2008/conference/schedule/ Hotel reservations are also open: https://www.cteusa.com/pycon1/ The rate is $99/night plus tax ($112/night net), also until February 20. More hotel information is here: http://us.pycon.org/2008/registration/hotel/ See the PyCon web site for complete details: http://us.pycon.org/2008/ Thanks to the PyCon organizers for developing the software, providing the website content, testing & debugging. PyCon is a community conference, of, by, and for the Python community. There is still much to do, and you can help! http://us.pycon.org/2008/helping/ See you in Chicago! David Goodger PyCon 2008 Chair -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 251 bytes Desc: OpenPGP digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20080120/4f3c377d/attachment.pgp From tim.peters at gmail.com Mon Jan 21 02:54:33 2008 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 20 Jan 2008 20:54:33 -0500 Subject: [Python-Dev] Rational approximation methods In-Reply-To: <5d44f72f0801191206x1027a49do62376026727fbb@mail.gmail.com> References: <5d44f72f0801191206x1027a49do62376026727fbb@mail.gmail.com> Message-ID: <1f7befae0801201754j505a9a59t8cacf90c279fc8b0@mail.gmail.com> What would be useful is a method that generates (i.e., a generator in the Python sense) the (continued fraction) convergents to a rational. People wanting specific constraints on a rational approximation (including, but not limited to, the two you identified) can easily build them on top of such a generator. By "useful" I don't mean lots of people will use it ;-) I mean /some/ people will use it -- a way to generate the sequence of convergents is a fundamental tool that can be used for all sorts of stuff, by those with advanced applications. From andrewm at object-craft.com.au Mon Jan 21 06:50:55 2008 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Mon, 21 Jan 2008 16:50:55 +1100 Subject: [Python-Dev] Monkeypatching idioms -- elegant or ugly? In-Reply-To: References: <478D2537.4090503@v.loewis.de> Message-ID: <20080121055055.EA5C26007A7@longblack.object-craft.com.au> >I think that despite the objection that monkeypatching shoudn't be >made too easy, it's worth at looking into a unification of the API, >features, and implementation. I agree. The other virtue of having it in the standard library is that it's immediately recognisable for what it is. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From asmodai at in-nomine.org Mon Jan 21 09:00:39 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Mon, 21 Jan 2008 09:00:39 +0100 Subject: [Python-Dev] #! magic In-Reply-To: <20080120223657.GA19754@phd.pp.ru> References: <47894171.7030004@cheimes.de> <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> <4789DDA0.4020900@cheimes.de> <1200848275.24656.95.camel@localhost> <20080120171238.GA16055@phd.pp.ru> <20080120221720.GB10699@mithrandi.za.net> <20080120223657.GA19754@phd.pp.ru> Message-ID: <20080121080039.GL61556@nexus.in-nomine.org> -On [20080120 23:36], Oleg Broytmann (phd at phd.pp.ru) wrote: >PS. My python doesn't understand -s, so I tested a different options, but >the result is the same. There are Unix variants that understand many >options (I believe FreeBSD allows them) but most allow no more than one >parameter in #!. FreeBSD 6.3-STABLE: % ./test.py Unknown option: - usage: /usr/local/bin/python [option] ... [-c cmd | -m mod | file | -] [arg] ... Try `python -h' for more information. Contracting to -Es works, aside from -s being unknown to my Python. And also, /usr/bin won't work, FreeBSD uses /usr/local/bin since Python is installed through ports and is not a system binary. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ Every revolution was first a thought in one man's mind... From asmodai at in-nomine.org Mon Jan 21 09:11:39 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Mon, 21 Jan 2008 09:11:39 +0100 Subject: [Python-Dev] #! magic In-Reply-To: <20080120173828.GB16055@phd.pp.ru> References: <47894171.7030004@cheimes.de> <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> <4789DDA0.4020900@cheimes.de> <1200848275.24656.95.camel@localhost> <20080120171238.GA16055@phd.pp.ru> <20080120172557.GH61556@nexus.in-nomine.org> <20080120173828.GB16055@phd.pp.ru> Message-ID: <20080121081139.GM61556@nexus.in-nomine.org> -On [20080120 18:38], Oleg Broytmann (phd at phd.pp.ru) wrote: > A shell has nothing to do with it as it is the OS (exec system call) >that upon reading the magic of the file sees #! and executes the program >(up to the first space) and pass to the program the first (and the only) >parameter. Yes, you are right. *sigh* It's been too long since I dug around the kernel on a daily basis. I had totally forgotten about imgact_shell.c's parsing of the hashbang. As execve(2) documents on FreeBSD: An interpreter file begins with a line of the form: #! interpreter [arg] When an interpreter file is execve'd, the system actually execve's the specified interpreter. If the optional arg is specified, it becomes the first argument to the interpreter, and the name of the originally execve'd file becomes the second argument; otherwise, the name of the originally execve'd file becomes the first argument. The original argu- ments are shifted over to become the subsequent arguments. The zeroth argument is set to the specified interpreter. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ The greatness of a nation and its moral progress can be judged by the way its animals are treated. From asmodai at in-nomine.org Mon Jan 21 09:33:48 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Mon, 21 Jan 2008 09:33:48 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: References: <478BABD9.6070206@cheimes.de> <20080114191326.21558.1896823221.divmod.xquotient.4614@joule.divmod.com> <20080114234147.GO30947@snowy.squish.net> <20080115112413.GA13674@phd.pp.ru> <2e1434c10801150357i247cf03dje874a36351c89c22@mail.gmail.com> <478CA55A.9010606@cheimes.de> <20080115122828.GD14063@phd.pp.ru> <20080120171003.GF61556@nexus.in-nomine.org> Message-ID: <20080121083348.GN61556@nexus.in-nomine.org> -On [20080120 19:34], Christian Heimes (lists at cheimes.de) wrote: >Most Python developers should the meaning of "~". Should I replace ~ >with $HOME for those who don't have as much experience with Unix as we? The problem is that ~ is an expansion character. It expands the contents of $HOME. If HOME is set to /usr/local, using ~ will expand to /usr/local. This behaviour is undefined by POSIX, if HOME is null/empty than expanding ~ is undefined. If it is just ~ then it will expand to the contents of HOME, if ~ is followed by a string then it will do a getpwnam() call on the string and expand that to the initial working directory of said login name string. HOME is POSIX-mandated to be '[t]he pathname of the user's home directory'. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ Hope is the last refuge for mad men and dreamers... From p.f.moore at gmail.com Mon Jan 21 09:44:25 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 21 Jan 2008 08:44:25 +0000 Subject: [Python-Dev] Rational approximation methods In-Reply-To: <1f7befae0801201754j505a9a59t8cacf90c279fc8b0@mail.gmail.com> References: <5d44f72f0801191206x1027a49do62376026727fbb@mail.gmail.com> <1f7befae0801201754j505a9a59t8cacf90c279fc8b0@mail.gmail.com> Message-ID: <79990c6b0801210044n7ef9cc15p2bee33e063193a10@mail.gmail.com> On 21/01/2008, Tim Peters wrote: > What would be useful is a method that generates (i.e., a generator in > the Python sense) the (continued fraction) convergents to a rational. > People wanting specific constraints on a rational approximation > (including, but not limited to, the two you identified) can easily > build them on top of such a generator. Interesting. I thought of suggesting something like this, but my potential use cases are so rare that I couldn't justify it. But as you say, it's one general building block from which any other desired functionality can be produced. > By "useful" I don't mean lots of people will use it ;-) I mean /some/ > people will use it -- a way to generate the sequence of convergents is > a fundamental tool that can be used for all sorts of stuff, by those > with advanced applications. Good point. Fundamental tools belong in the library. +1 Paul. From lists at cheimes.de Mon Jan 21 12:01:29 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 21 Jan 2008 12:01:29 +0100 Subject: [Python-Dev] #! magic In-Reply-To: <20080121080039.GL61556@nexus.in-nomine.org> References: <47894171.7030004@cheimes.de> <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> <4789DDA0.4020900@cheimes.de> <1200848275.24656.95.camel@localhost> <20080120171238.GA16055@phd.pp.ru> <20080120221720.GB10699@mithrandi.za.net> <20080120223657.GA19754@phd.pp.ru> <20080121080039.GL61556@nexus.in-nomine.org> Message-ID: <47947B89.3090402@cheimes.de> Jeroen Ruigrok van der Werven wrote: > % ./test.py > Unknown option: - > usage: /usr/local/bin/python [option] ... [-c cmd | -m mod | file | -] [arg] > ... > Try `python -h' for more information. > > Contracting to -Es works, aside from -s being unknown to my Python. > > And also, /usr/bin won't work, FreeBSD uses /usr/local/bin since Python is > installed through ports and is not a system binary. The -s option is part of my PEP and is not available in the trunk yet. The arg -Es may work because Python's arg parser doesn't recognize it as two args -E -s but as the arg -E. Christian From lists at cheimes.de Mon Jan 21 12:01:29 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 21 Jan 2008 12:01:29 +0100 Subject: [Python-Dev] #! magic In-Reply-To: <20080121080039.GL61556@nexus.in-nomine.org> References: <47894171.7030004@cheimes.de> <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> <4789DDA0.4020900@cheimes.de> <1200848275.24656.95.camel@localhost> <20080120171238.GA16055@phd.pp.ru> <20080120221720.GB10699@mithrandi.za.net> <20080120223657.GA19754@phd.pp.ru> <20080121080039.GL61556@nexus.in-nomine.org> Message-ID: <47947B89.3090402@cheimes.de> Jeroen Ruigrok van der Werven wrote: > % ./test.py > Unknown option: - > usage: /usr/local/bin/python [option] ... [-c cmd | -m mod | file | -] [arg] > ... > Try `python -h' for more information. > > Contracting to -Es works, aside from -s being unknown to my Python. > > And also, /usr/bin won't work, FreeBSD uses /usr/local/bin since Python is > installed through ports and is not a system binary. The -s option is part of my PEP and is not available in the trunk yet. The arg -Es may work because Python's arg parser doesn't recognize it as two args -E -s but as the arg -E. Christian From phd at phd.pp.ru Mon Jan 21 12:17:01 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Mon, 21 Jan 2008 14:17:01 +0300 Subject: [Python-Dev] #! magic In-Reply-To: <47947B89.3090402@cheimes.de> References: <47894171.7030004@cheimes.de> <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> <4789DDA0.4020900@cheimes.de> <1200848275.24656.95.camel@localhost> <20080120171238.GA16055@phd.pp.ru> <20080120221720.GB10699@mithrandi.za.net> <20080120223657.GA19754@phd.pp.ru> <20080121080039.GL61556@nexus.in-nomine.org> <47947B89.3090402@cheimes.de> Message-ID: <20080121111701.GC2327@phd.pp.ru> On Mon, Jan 21, 2008 at 12:01:29PM +0100, Christian Heimes wrote: > The arg -Es may work because Python's arg parser doesn't recognize it as > two args -E -s but as the arg -E. Thank goodness python is better than that: $ python -Es Unknown option: -s usage: python [option] ... [-c cmd | -m mod | file | -] [arg] ... Try `python -h' for more information. $ python -E -s Unknown option: -s usage: python [option] ... [-c cmd | -m mod | file | -] [arg] ... Try `python -h' for more information. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From lists at cheimes.de Mon Jan 21 13:09:51 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 21 Jan 2008 13:09:51 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <1200848275.24656.95.camel@localhost> References: <47894171.7030004@cheimes.de> <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> <4789DDA0.4020900@cheimes.de> <1200848275.24656.95.camel@localhost> Message-ID: <47948B8F.6060707@cheimes.de> Jan Claeys wrote: > There should be a way for distro developers to make sure the users local > 'site-packages' is *not* used when running those tools. > > I'd rather have to set/uncomment an environment variable on my system > than having 100 "normal" users break their systems accidentally... ;-) "#!/usr/bin/env python -E -s" doesn't work on most Unices. [1] I came up with two possible solutions. Both depend on a new 'paranoid' flag -P which disables several features like PYTHON* env vars, inspect interactively, user site directory and the '' in sys.path. * Create a new, minimal Python executable which sets Py_ParanoidFlag to a true value and calls Py_Main(). The new executable is to be named pythons2.x (python secure). * Add a new source flag "# -*- py-paranoid -*-" which must be in the second or third line of a script. Modules/main.c:Py_Main() checks for the flag around line 430. A rough Python version of the C code could look like: def find_paranoid(fname): if not os.path.isfile(fname): return data = open(fname).read(4096) if not data.startswith("#!"): return for i in (1, 2): data = data[data.find('\n'):] if data.startswith("# -*- py-paranoid -*-"): return True return False Christian [1] Cygwin discussion thread about #! env http://www.cygwin.com/ml/cygwin/2002-02/msg00657.html/ From lists at cheimes.de Mon Jan 21 13:09:51 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 21 Jan 2008 13:09:51 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <1200848275.24656.95.camel@localhost> References: <47894171.7030004@cheimes.de> <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> <4789DDA0.4020900@cheimes.de> <1200848275.24656.95.camel@localhost> Message-ID: <47948B8F.6060707@cheimes.de> Jan Claeys wrote: > There should be a way for distro developers to make sure the users local > 'site-packages' is *not* used when running those tools. > > I'd rather have to set/uncomment an environment variable on my system > than having 100 "normal" users break their systems accidentally... ;-) "#!/usr/bin/env python -E -s" doesn't work on most Unices. [1] I came up with two possible solutions. Both depend on a new 'paranoid' flag -P which disables several features like PYTHON* env vars, inspect interactively, user site directory and the '' in sys.path. * Create a new, minimal Python executable which sets Py_ParanoidFlag to a true value and calls Py_Main(). The new executable is to be named pythons2.x (python secure). * Add a new source flag "# -*- py-paranoid -*-" which must be in the second or third line of a script. Modules/main.c:Py_Main() checks for the flag around line 430. A rough Python version of the C code could look like: def find_paranoid(fname): if not os.path.isfile(fname): return data = open(fname).read(4096) if not data.startswith("#!"): return for i in (1, 2): data = data[data.find('\n'):] if data.startswith("# -*- py-paranoid -*-"): return True return False Christian [1] Cygwin discussion thread about #! env http://www.cygwin.com/ml/cygwin/2002-02/msg00657.html/ From asmodai at in-nomine.org Mon Jan 21 14:03:44 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Mon, 21 Jan 2008 14:03:44 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <478D244C.8070408@v.loewis.de> References: <20080115073718.GB75977@nexus.in-nomine.org> <478D244C.8070408@v.loewis.de> Message-ID: <20080121130344.GO61556@nexus.in-nomine.org> -On [20080116 07:15], "Martin v. L?wis" (martin at v.loewis.de) wrote: >I don't understand why they *have* to do that. I can believe they do >that as they don't know better - but why can't they use the Python >interpreter already available on the system, and just install additional >packages in their home directory? Good question. I guess there just isn't a clear and precise document available to point them at. I'll be glad to (help) write such an article. >For that, I think the requirements need to be much more explicit. Yeah, sorry, have not had the time available to verify all cases Christian reported versus what I can think of. Impending surgery tends to distract the mind a bit. ;) -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ Where does the world end if the enternal optimist loses faith..? From steve at holdenweb.com Mon Jan 21 15:16:46 2008 From: steve at holdenweb.com (Steve Holden) Date: Mon, 21 Jan 2008 09:16:46 -0500 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <478D244C.8070408@v.loewis.de> References: <20080115073718.GB75977@nexus.in-nomine.org> <478D244C.8070408@v.loewis.de> Message-ID: <4794A94E.1080304@holdenweb.com> Martin v. L?wis wrote: >> Right now Python faces a lot of problems in the webhosting world because it's >> tedious to set up and maintain for the webhosting user since they often have >> to compile and install their own Python in their home directory. > > I don't understand why they *have* to do that. I can believe they do > that as they don't know better - but why can't they use the Python > interpreter already available on the system, and just install additional > packages in their home directory? > One possible reason is that the hosting vendor doesn't provide a sufficiently up-to-date Python release for certain site requirements. >> I think this is extremely important due to the proliferation of Python now >> more and more as a choice for webapp development. > > For that, I think the requirements need to be much more explicit. > True. But IMHO Python is still seen as a "difficult" language to package. We haven't reached CPAN levels of ease-of-use ye. Yes, I *do* know that CPAN isn't perfect, and I think easy_install is a great boon to Python users, but CPAN does a great job 98% of the time. Maybe once we get easy_install as a part of the core (so there's no need to find and run ez_setup.py to start with) things will start to improve. This is an issue the whole developer community needs to take seriously if we are interested in increasing take-up. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Mon Jan 21 15:16:46 2008 From: steve at holdenweb.com (Steve Holden) Date: Mon, 21 Jan 2008 09:16:46 -0500 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <478D244C.8070408@v.loewis.de> References: <20080115073718.GB75977@nexus.in-nomine.org> <478D244C.8070408@v.loewis.de> Message-ID: <4794A94E.1080304@holdenweb.com> Martin v. L?wis wrote: >> Right now Python faces a lot of problems in the webhosting world because it's >> tedious to set up and maintain for the webhosting user since they often have >> to compile and install their own Python in their home directory. > > I don't understand why they *have* to do that. I can believe they do > that as they don't know better - but why can't they use the Python > interpreter already available on the system, and just install additional > packages in their home directory? > One possible reason is that the hosting vendor doesn't provide a sufficiently up-to-date Python release for certain site requirements. >> I think this is extremely important due to the proliferation of Python now >> more and more as a choice for webapp development. > > For that, I think the requirements need to be much more explicit. > True. But IMHO Python is still seen as a "difficult" language to package. We haven't reached CPAN levels of ease-of-use ye. Yes, I *do* know that CPAN isn't perfect, and I think easy_install is a great boon to Python users, but CPAN does a great job 98% of the time. Maybe once we get easy_install as a part of the core (so there's no need to find and run ez_setup.py to start with) things will start to improve. This is an issue the whole developer community needs to take seriously if we are interested in increasing take-up. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From lists at cheimes.de Mon Jan 21 15:25:07 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 21 Jan 2008 15:25:07 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <4794A94E.1080304@holdenweb.com> References: <20080115073718.GB75977@nexus.in-nomine.org> <478D244C.8070408@v.loewis.de> <4794A94E.1080304@holdenweb.com> Message-ID: <4794AB43.10301@cheimes.de> Steve Holden wrote: > Maybe once we get easy_install as a part of the core (so there's no need > to find and run ez_setup.py to start with) things will start to improve. > This is an issue the whole developer community needs to take seriously > if we are interested in increasing take-up. setuptools and easy_install won't be included in Python 2.6 and 3.0: http://www.python.org/dev/peps/pep-0365/ Christian From amk at amk.ca Mon Jan 21 15:40:00 2008 From: amk at amk.ca (A.M. Kuchling) Date: Mon, 21 Jan 2008 09:40:00 -0500 Subject: [Python-Dev] Bug Day outcome In-Reply-To: References: <20080120011436.GA26802@amk.local> Message-ID: <20080121144000.GA15906@amk-desktop.matrixgroup.net> On Sun, Jan 20, 2008 at 10:13:44PM +0100, Georg Brandl wrote: > Since this bug day was a relative success, I suggest to introduce a > more-or-less regular schedule. > > Feb 23 would make a nice second bug day in 2008, wouldn't it? That works for me. I've updated the wiki page to give Feb. 23 as the date, and will update pydotorg this evening. --amk From fuzzyman at voidspace.org.uk Mon Jan 21 15:41:35 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 21 Jan 2008 14:41:35 +0000 Subject: [Python-Dev] [python] Re: PEP: per user site-packages directory In-Reply-To: <4794AB43.10301@cheimes.de> References: <20080115073718.GB75977@nexus.in-nomine.org> <478D244C.8070408@v.loewis.de> <4794A94E.1080304@holdenweb.com> <4794AB43.10301@cheimes.de> Message-ID: <4794AF1F.9090007@voidspace.org.uk> Christian Heimes wrote: > Steve Holden wrote: > >> Maybe once we get easy_install as a part of the core (so there's no need >> to find and run ez_setup.py to start with) things will start to improve. >> This is an issue the whole developer community needs to take seriously >> if we are interested in increasing take-up. >> > > setuptools and easy_install won't be included in Python 2.6 and 3.0: > http://www.python.org/dev/peps/pep-0365/ > Which is a shame. I agree with Steve on this - although I realise that Phillip is basically the only person able to do the work. Michael Foord > Christian > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > > From steve at holdenweb.com Mon Jan 21 15:48:41 2008 From: steve at holdenweb.com (Steve Holden) Date: Mon, 21 Jan 2008 09:48:41 -0500 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <4794AB43.10301@cheimes.de> References: <20080115073718.GB75977@nexus.in-nomine.org> <478D244C.8070408@v.loewis.de> <4794A94E.1080304@holdenweb.com> <4794AB43.10301@cheimes.de> Message-ID: Christian Heimes wrote: > Steve Holden wrote: >> Maybe once we get easy_install as a part of the core (so there's no need >> to find and run ez_setup.py to start with) things will start to improve. >> This is an issue the whole developer community needs to take seriously >> if we are interested in increasing take-up. > > setuptools and easy_install won't be included in Python 2.6 and 3.0: > http://www.python.org/dev/peps/pep-0365/ > Yes, and yet another release (two releases) will go out without easy access to the functionality in Pypi. PEP 365 is a good start, but Pypi loses much of its point until new Python users get access to it "out of the box". I also appreciate that resource limitations are standing in the way of setuptools' inclusion (is there something I can do about that?) Just to hammer the point home, however ... The easy_install utility from setuptools is probably the biggest potential recruiter for Python. It makes the language accessible to the majority of users who are interested only in acquiring new functionality, and not at all in which language that functionality is implemented in. Installation of the latest/most appropriate version of something then becomes a simple recipe without command switches regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From ncoghlan at gmail.com Mon Jan 21 16:06:16 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 22 Jan 2008 01:06:16 +1000 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: References: <20080115073718.GB75977@nexus.in-nomine.org> <478D244C.8070408@v.loewis.de> <4794A94E.1080304@holdenweb.com> <4794AB43.10301@cheimes.de> Message-ID: <4794B4E8.9090105@gmail.com> Steve Holden wrote: > Christian Heimes wrote: >> Steve Holden wrote: >>> Maybe once we get easy_install as a part of the core (so there's no need >>> to find and run ez_setup.py to start with) things will start to improve. >>> This is an issue the whole developer community needs to take seriously >>> if we are interested in increasing take-up. >> setuptools and easy_install won't be included in Python 2.6 and 3.0: >> http://www.python.org/dev/peps/pep-0365/ >> > Yes, and yet another release (two releases) will go out without easy > access to the functionality in Pypi. PEP 365 is a good start, but Pypi > loses much of its point until new Python users get access to it "out of > the box". I also appreciate that resource limitations are standing in > the way of setuptools' inclusion (is there something I can do about > that?) Just to hammer the point home, however ... Have another look at the rationale given in PEP 365 - it isn't the resourcing to do the work that's a problem, but the relatively slow release cycle of the core. By including pkg_resources in the core (with the addition of access to pure Python modules and packages on PyPI), we would get a simple, stable base for Python packaging to work from, and put users a single standard command away from the more advanced (but also more volatile) features of easy_install and friends. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From pje at telecommunity.com Mon Jan 21 16:23:54 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 21 Jan 2008 10:23:54 -0500 Subject: [Python-Dev] PEP 365 (was Re: PEP: per user site-packages directory) In-Reply-To: <4794B4E8.9090105@gmail.com> References: <20080115073718.GB75977@nexus.in-nomine.org> <478D244C.8070408@v.loewis.de> <4794A94E.1080304@holdenweb.com> <4794AB43.10301@cheimes.de> <4794B4E8.9090105@gmail.com> Message-ID: <20080121152353.B8CB13A40A9@sparrow.telecommunity.com> At 01:06 AM 1/22/2008 +1000, Nick Coghlan wrote: >Steve Holden wrote: > > Christian Heimes wrote: > >> Steve Holden wrote: > >>> Maybe once we get easy_install as a part of the core (so there's no need > >>> to find and run ez_setup.py to start with) things will start to improve. > >>> This is an issue the whole developer community needs to take seriously > >>> if we are interested in increasing take-up. > >> setuptools and easy_install won't be included in Python 2.6 and 3.0: > >> http://www.python.org/dev/peps/pep-0365/ > >> > > Yes, and yet another release (two releases) will go out without easy > > access to the functionality in Pypi. PEP 365 is a good start, but Pypi > > loses much of its point until new Python users get access to it "out of > > the box". I also appreciate that resource limitations are standing in > > the way of setuptools' inclusion (is there something I can do about > > that?) Just to hammer the point home, however ... > >Have another look at the rationale given in PEP 365 - it isn't the >resourcing to do the work that's a problem, but the relatively slow >release cycle of the core. > >By including pkg_resources in the core (with the addition of access to >pure Python modules and packages on PyPI), we would get a simple, stable >base for Python packaging to work from, and put users a single standard >command away from the more advanced (but also more volatile) features of >easy_install and friends. By the way, if we're actually going to get that into 2.6, it would be good for the PEP to actually be approved before then. :) With respect to Steve's comments about out-of-the-box usability, it should be noted that when you bootstrap a package with pkg_resources, it should be possible to include other command-line arguments after the package specifier. So for example: python -m pkg_resources setuptools SomePackage==1.2 would download and install setuptools, and run its "bootstrap script" with "SomePackage==1.2" as a command-line argument. And setuptools' bootstrap script is basically easy_install with some extra code to make sure the setuptools egg gets installed too. In other words, with PEP 365 in place, "python -m pkg_resources setuptools" is basically a way to say "easy_install" without needing setuptools installed. (Heck, if what you really want is to have easy_install support in 2.6, we could just as easily bundle an easy_install.py that asks for an install of setuptools if it's not already present.) From steve at holdenweb.com Mon Jan 21 16:48:02 2008 From: steve at holdenweb.com (Steve Holden) Date: Mon, 21 Jan 2008 10:48:02 -0500 Subject: [Python-Dev] PEP 365 (was Re: PEP: per user site-packages directory) In-Reply-To: <20080121152353.B8CB13A40A9@sparrow.telecommunity.com> References: <20080115073718.GB75977@nexus.in-nomine.org> <478D244C.8070408@v.loewis.de> <4794A94E.1080304@holdenweb.com> <4794AB43.10301@cheimes.de> <4794B4E8.9090105@gmail.com> <20080121152353.B8CB13A40A9@sparrow.telecommunity.com> Message-ID: <4794BEB2.4090004@holdenweb.com> Phillip J. Eby wrote: > At 01:06 AM 1/22/2008 +1000, Nick Coghlan wrote: >> Steve Holden wrote: >>> Christian Heimes wrote: >>>> Steve Holden wrote: >>>>> Maybe once we get easy_install as a part of the core (so there's no need >>>>> to find and run ez_setup.py to start with) things will start to improve. >>>>> This is an issue the whole developer community needs to take seriously >>>>> if we are interested in increasing take-up. >>>> setuptools and easy_install won't be included in Python 2.6 and 3.0: >>>> http://www.python.org/dev/peps/pep-0365/ >>>> >>> Yes, and yet another release (two releases) will go out without easy >>> access to the functionality in Pypi. PEP 365 is a good start, but Pypi >>> loses much of its point until new Python users get access to it "out of >>> the box". I also appreciate that resource limitations are standing in >>> the way of setuptools' inclusion (is there something I can do about >>> that?) Just to hammer the point home, however ... >> Have another look at the rationale given in PEP 365 - it isn't the >> resourcing to do the work that's a problem, but the relatively slow >> release cycle of the core. >> >> By including pkg_resources in the core (with the addition of access to >> pure Python modules and packages on PyPI), we would get a simple, stable >> base for Python packaging to work from, and put users a single standard >> command away from the more advanced (but also more volatile) features of >> easy_install and friends. > > By the way, if we're actually going to get that into 2.6, it would be > good for the PEP to actually be approved before then. :) > Yes, I noticed it wasn't yet approved and wondered whether this was simply an oversight or whether there is still work to do before it can be approved. > With respect to Steve's comments about out-of-the-box usability, it > should be noted that when you bootstrap a package with pkg_resources, > it should be possible to include other command-line arguments after > the package specifier. So for example: > > python -m pkg_resources setuptools SomePackage==1.2 > > would download and install setuptools, and run its "bootstrap script" > with "SomePackage==1.2" as a command-line argument. And setuptools' > bootstrap script is basically easy_install with some extra code to > make sure the setuptools egg gets installed too. > > In other words, with PEP 365 in place, "python -m pkg_resources > setuptools" is basically a way to say "easy_install" without needing > setuptools installed. > Well that's probably a simple enough recipe to include in the distribution, wouldn't you say? > (Heck, if what you really want is to have easy_install support in > 2.6, we could just as easily bundle an easy_install.py that asks for > an install of setuptools if it's not already present.) > Would the easiest way to do this be to insert a default dependency on setuptools? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Mon Jan 21 16:48:02 2008 From: steve at holdenweb.com (Steve Holden) Date: Mon, 21 Jan 2008 10:48:02 -0500 Subject: [Python-Dev] PEP 365 (was Re: PEP: per user site-packages directory) In-Reply-To: <20080121152353.B8CB13A40A9@sparrow.telecommunity.com> References: <20080115073718.GB75977@nexus.in-nomine.org> <478D244C.8070408@v.loewis.de> <4794A94E.1080304@holdenweb.com> <4794AB43.10301@cheimes.de> <4794B4E8.9090105@gmail.com> <20080121152353.B8CB13A40A9@sparrow.telecommunity.com> Message-ID: <4794BEB2.4090004@holdenweb.com> Phillip J. Eby wrote: > At 01:06 AM 1/22/2008 +1000, Nick Coghlan wrote: >> Steve Holden wrote: >>> Christian Heimes wrote: >>>> Steve Holden wrote: >>>>> Maybe once we get easy_install as a part of the core (so there's no need >>>>> to find and run ez_setup.py to start with) things will start to improve. >>>>> This is an issue the whole developer community needs to take seriously >>>>> if we are interested in increasing take-up. >>>> setuptools and easy_install won't be included in Python 2.6 and 3.0: >>>> http://www.python.org/dev/peps/pep-0365/ >>>> >>> Yes, and yet another release (two releases) will go out without easy >>> access to the functionality in Pypi. PEP 365 is a good start, but Pypi >>> loses much of its point until new Python users get access to it "out of >>> the box". I also appreciate that resource limitations are standing in >>> the way of setuptools' inclusion (is there something I can do about >>> that?) Just to hammer the point home, however ... >> Have another look at the rationale given in PEP 365 - it isn't the >> resourcing to do the work that's a problem, but the relatively slow >> release cycle of the core. >> >> By including pkg_resources in the core (with the addition of access to >> pure Python modules and packages on PyPI), we would get a simple, stable >> base for Python packaging to work from, and put users a single standard >> command away from the more advanced (but also more volatile) features of >> easy_install and friends. > > By the way, if we're actually going to get that into 2.6, it would be > good for the PEP to actually be approved before then. :) > Yes, I noticed it wasn't yet approved and wondered whether this was simply an oversight or whether there is still work to do before it can be approved. > With respect to Steve's comments about out-of-the-box usability, it > should be noted that when you bootstrap a package with pkg_resources, > it should be possible to include other command-line arguments after > the package specifier. So for example: > > python -m pkg_resources setuptools SomePackage==1.2 > > would download and install setuptools, and run its "bootstrap script" > with "SomePackage==1.2" as a command-line argument. And setuptools' > bootstrap script is basically easy_install with some extra code to > make sure the setuptools egg gets installed too. > > In other words, with PEP 365 in place, "python -m pkg_resources > setuptools" is basically a way to say "easy_install" without needing > setuptools installed. > Well that's probably a simple enough recipe to include in the distribution, wouldn't you say? > (Heck, if what you really want is to have easy_install support in > 2.6, we could just as easily bundle an easy_install.py that asks for > an install of setuptools if it's not already present.) > Would the easiest way to do this be to insert a default dependency on setuptools? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From dickinsm at gmail.com Mon Jan 21 16:51:16 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 21 Jan 2008 10:51:16 -0500 Subject: [Python-Dev] Rational approximation methods In-Reply-To: <79990c6b0801210044n7ef9cc15p2bee33e063193a10@mail.gmail.com> References: <5d44f72f0801191206x1027a49do62376026727fbb@mail.gmail.com> <1f7befae0801201754j505a9a59t8cacf90c279fc8b0@mail.gmail.com> <79990c6b0801210044n7ef9cc15p2bee33e063193a10@mail.gmail.com> Message-ID: <5c6f2a5d0801210751n350fa0ffobd518e10518d629b@mail.gmail.com> On Jan 21, 2008 3:44 AM, Paul Moore wrote: > On 21/01/2008, Tim Peters wrote:> By "useful" I > don't mean lots of people will use it ;-) I mean /some/ > > people will use it -- a way to generate the sequence of convergents is > > a fundamental tool that can be used for all sorts of stuff, by those > > with advanced applications. > > Good point. Fundamental tools belong in the library. +1 How about a generator for the convergents in the rational module together with an example in the documentation showing how to use the convergents to implement something like trim? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080121/24159339/attachment-0001.htm From pje at telecommunity.com Mon Jan 21 17:25:14 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 21 Jan 2008 11:25:14 -0500 Subject: [Python-Dev] PEP 365 (was Re: PEP: per user site-packages directory) In-Reply-To: <4794BEB2.4090004@holdenweb.com> References: <20080115073718.GB75977@nexus.in-nomine.org> <478D244C.8070408@v.loewis.de> <4794A94E.1080304@holdenweb.com> <4794AB43.10301@cheimes.de> <4794B4E8.9090105@gmail.com> <20080121152353.B8CB13A40A9@sparrow.telecommunity.com> <4794BEB2.4090004@holdenweb.com> Message-ID: <20080121162520.D75B23A40A9@sparrow.telecommunity.com> At 10:48 AM 1/21/2008 -0500, Steve Holden wrote: >Phillip J. Eby wrote: > > (Heck, if what you really want is to have easy_install support in > > 2.6, we could just as easily bundle an easy_install.py that asks for > > an install of setuptools if it's not already present.) > > >Would the easiest way to do this be to insert a default dependency on >setuptools? Sorry, I don't understand the question. What do you mean by "default dependency" and to what are you proposing it be inserted? :) What I meant was that we could include an easy_install.py whose sole function is to ensure that setuptools is installed and then invoke the "real" easy_install. Thus, the first time you ran easy_install, a current version would be downloaded, and thereafter the real one would be runnable. From martin at v.loewis.de Mon Jan 21 22:39:43 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 21 Jan 2008 22:39:43 +0100 Subject: [Python-Dev] [python] Re: PEP: per user site-packages directory In-Reply-To: <4794AF1F.9090007@voidspace.org.uk> References: <20080115073718.GB75977@nexus.in-nomine.org> <478D244C.8070408@v.loewis.de> <4794A94E.1080304@holdenweb.com> <4794AB43.10301@cheimes.de> <4794AF1F.9090007@voidspace.org.uk> Message-ID: <4795111F.5090000@v.loewis.de> > Which is a shame. I agree with Steve on this - although I realise that > Phillip is basically the only person able to do the work. Not the only one able - the only one willing to. People shouldn't complain about the state of things if they aren't willing to do anything about it (except complaining). Regards, Martin From martin at v.loewis.de Mon Jan 21 22:46:22 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 21 Jan 2008 22:46:22 +0100 Subject: [Python-Dev] PEP 365 (was Re: PEP: per user site-packages directory) In-Reply-To: <20080121162520.D75B23A40A9@sparrow.telecommunity.com> References: <20080115073718.GB75977@nexus.in-nomine.org> <478D244C.8070408@v.loewis.de> <4794A94E.1080304@holdenweb.com> <4794AB43.10301@cheimes.de> <4794B4E8.9090105@gmail.com> <20080121152353.B8CB13A40A9@sparrow.telecommunity.com> <4794BEB2.4090004@holdenweb.com> <20080121162520.D75B23A40A9@sparrow.telecommunity.com> Message-ID: <479512AE.4000204@v.loewis.de> > What I meant was that we could include an easy_install.py whose sole > function is to ensure that setuptools is installed and then invoke > the "real" easy_install. Thus, the first time you ran easy_install, > a current version would be downloaded, and thereafter the real one > would be runnable. Despite my (well-known) objections to setuptools as the foundation for the .egg format (and everything that comes with it), I think that would be a desirable and easy thing to do. Including a setuptools bootstrapper into the distribution still leaves the choice of actually using setuptools to the end user. Regards, Martin From brett at python.org Tue Jan 22 00:25:09 2008 From: brett at python.org (Brett Cannon) Date: Mon, 21 Jan 2008 15:25:09 -0800 Subject: [Python-Dev] PEP 365 (was Re: PEP: per user site-packages directory) In-Reply-To: <479512AE.4000204@v.loewis.de> References: <478D244C.8070408@v.loewis.de> <4794A94E.1080304@holdenweb.com> <4794AB43.10301@cheimes.de> <4794B4E8.9090105@gmail.com> <20080121152353.B8CB13A40A9@sparrow.telecommunity.com> <4794BEB2.4090004@holdenweb.com> <20080121162520.D75B23A40A9@sparrow.telecommunity.com> <479512AE.4000204@v.loewis.de> Message-ID: On Jan 21, 2008 1:46 PM, "Martin v. L?wis" wrote: > > What I meant was that we could include an easy_install.py whose sole > > function is to ensure that setuptools is installed and then invoke > > the "real" easy_install. Thus, the first time you ran easy_install, > > a current version would be downloaded, and thereafter the real one > > would be runnable. > > Despite my (well-known) objections to setuptools as the foundation for > the .egg format (and everything that comes with it), I think that would > be a desirable and easy thing to do. Including a setuptools bootstrapper > into the distribution still leaves the choice of actually using > setuptools to the end user. What Martin said, almost verbatim. =) -Brett From mhammond at skippinet.com.au Tue Jan 22 11:05:57 2008 From: mhammond at skippinet.com.au (Mark Hammond) Date: Tue, 22 Jan 2008 21:05:57 +1100 Subject: [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <46512B09.1080304@v.loewis.de> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> Message-ID: <022401c85cde$678bcf10$36a36d30$@com.au> Hi Martin, Way back on Monday, 21 May 2007, you wrote: > This is an issue to be discussed for Python 2.6. I'm personally > hesitant to have the "official" build infrastructure deviate > from the layout that has been in-use for so many years, as a lot > of things depend on it. > > I don't find the need to have separate object directories convincing: > For building the Win32/Win64 binaries, I have separate checkouts > *anyway*, since all the add-on libraries would have to support > multi-arch builds, but I think they don't. ... > > * Re the x64 directory used by the PCBuild8 process. IMO, it makes > > sense to generate x64 binaries to their own directory - my expectation > > is that cross-compiling between platforms is a reasonable use-case, > > and we should support multiple achitectures for the same compiler > > version. > > See above; I disagree. [For additional context; the question was regarding where the output binaries were created for each platform, and specifically if x64 build should use something like 'PCBuild/x64'] I'm bringing this topic up again as I noticed the AMD64 builds for Python 2.6 create their output in the PCBuild/x64 directory, not directly into the PCBuild directory. When I raised this with Christian, he also expressed a desire to be able to cross-compile in the same directory structure and was unaware of this discussion (but I made him aware of it :) You made excellent points about how tools currently recognize the PCBuild directory, and we can't break them, and I agree. I'd like to propose a compromise that might be able to keep everyone happy. The main problem I see with all platforms using 'PCBuild' is that in a cross-compilation scenario, it is impossible for the distutils to determine the location of the correct Python libraries to link with (stuff in its own PYTHONHOME is no good; it's for the wrong platform). Obviously, we can change the distutils so that the location of the correct libraries can be specified, but that implies all other build systems that currently assume PCBuild must also change to work in a cross-compilation scenario. While those external tools *will* work today in a native build on x64, eventually they may need to be upgraded to deal with cross-compilation - and this means that all such tools will also be unable to automatically determine the location of the libraries. They will all need to take the library dir as a user-specified option, which seems a pain (eg, buildbots would seem to need site-specific configuration information to make this work). If eventually all such tools are likely to upgrade, it would be ideal if we can offer them a way to upgrade so the correct libraries can be determined automatically, as they can now for native builds. My compromise proposal is this: Each platform builds in its own directory (ie, PCBuild/x86_64, PCBuild/x86). Every single build configuration has a post-build step that copies the result of the build to the PCBuild directory. We then add an option to the distutils so that a cross-compile platform can be specified and when it is, to use 'PCBuild/{platform}" instead of PCBuild, and we encourage other tools to use the same directory should they want to support "configure-less" cross-compilation (if distutils on the trunk is not still trying to maintain b/w compat, it should just *always* look in PCBuild/{platform}, and I'd suggest py3k not bother with PCBuild at all; build systems will be touched to upgrade to py3k, so that change isn't painful. But I digress :) The main advantage of the compromise is that it allows for the status quo to remain in place - just continue to use separate source trees for each platform, and only ever build a single platform in each tree, and tools are free to have ways of specifying which directory should be used. The official build process need not change. However, it also supports a way to do cross-compilation in a way that build tools can *automatically* locate the correct platform's libraries, and this will be particularly useful for extension authors. Would something like that be acceptable? Thanks, Mark From mal at egenix.com Tue Jan 22 16:42:34 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 22 Jan 2008 16:42:34 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <4794B4E8.9090105@gmail.com> References: <20080115073718.GB75977@nexus.in-nomine.org> <478D244C.8070408@v.loewis.de> <4794A94E.1080304@holdenweb.com> <4794AB43.10301@cheimes.de> <4794B4E8.9090105@gmail.com> Message-ID: <47960EEA.3010609@egenix.com> I don't really understand what all this has to do with per user site-packages. Note that the motivation for having per user site-packages was to: * address a common request by Python extension package users, * get rid off the hackery done by setuptools in order to provide this. As such the PEP can also be seen as an effort to enable code cleanup *before* adding e.g. pkg_resources to the stdlib. Cheers, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 22 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 On 2008-01-21 16:06, Nick Coghlan wrote: > Steve Holden wrote: >> Christian Heimes wrote: >>> Steve Holden wrote: >>>> Maybe once we get easy_install as a part of the core (so there's no need >>>> to find and run ez_setup.py to start with) things will start to improve. >>>> This is an issue the whole developer community needs to take seriously >>>> if we are interested in increasing take-up. >>> setuptools and easy_install won't be included in Python 2.6 and 3.0: >>> http://www.python.org/dev/peps/pep-0365/ >>> >> Yes, and yet another release (two releases) will go out without easy >> access to the functionality in Pypi. PEP 365 is a good start, but Pypi >> loses much of its point until new Python users get access to it "out of >> the box". I also appreciate that resource limitations are standing in >> the way of setuptools' inclusion (is there something I can do about >> that?) Just to hammer the point home, however ... > > Have another look at the rationale given in PEP 365 - it isn't the > resourcing to do the work that's a problem, but the relatively slow > release cycle of the core. > > By including pkg_resources in the core (with the addition of access to > pure Python modules and packages on PyPI), we would get a simple, stable > base for Python packaging to work from, and put users a single standard > command away from the more advanced (but also more volatile) features of > easy_install and friends. > > Cheers, > Nick. > From lists at cheimes.de Tue Jan 22 18:03:19 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 22 Jan 2008 18:03:19 +0100 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <47960EEA.3010609@egenix.com> References: <20080115073718.GB75977@nexus.in-nomine.org> <478D244C.8070408@v.loewis.de> <4794A94E.1080304@holdenweb.com> <4794AB43.10301@cheimes.de> <4794B4E8.9090105@gmail.com> <47960EEA.3010609@egenix.com> Message-ID: M.-A. Lemburg wrote: > I don't really understand what all this has to do with per user > site-packages. > > Note that the motivation for having per user site-packages > was to: > > * address a common request by Python extension package users, > > * get rid off the hackery done by setuptools in order > to provide this. > > As such the PEP can also be seen as an effort to enable code > cleanup *before* adding e.g. pkg_resources to the stdlib. I guess it's a typical case of "while you on it, can you fix the rest of the world, too?". People see that I invest a fair share of time into the PEP. So they hope I'm going to solve loosely connected issues, too. Personally I'm not going to solve every security issue with my PEP. I suggest that all the people, who were interested in security, come together and write another PEP. Christian From pje at telecommunity.com Tue Jan 22 18:24:20 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 22 Jan 2008 12:24:20 -0500 Subject: [Python-Dev] PEP: per user site-packages directory In-Reply-To: <47960EEA.3010609@egenix.com> References: <20080115073718.GB75977@nexus.in-nomine.org> <478D244C.8070408@v.loewis.de> <4794A94E.1080304@holdenweb.com> <4794AB43.10301@cheimes.de> <4794B4E8.9090105@gmail.com> <47960EEA.3010609@egenix.com> Message-ID: <20080122172419.538B13A4076@sparrow.telecommunity.com> At 04:42 PM 1/22/2008 +0100, M.-A. Lemburg wrote: >I don't really understand what all this has to do with per user >site-packages. > >Note that the motivation for having per user site-packages >was to: > > * address a common request by Python extension package users, > > * get rid off the hackery done by setuptools in order > to provide this. Setuptools doesn't do any hackery for per-user site-packages, although its documentation does explain how to set up such a thing if you want it: http://peak.telecommunity.com/DevCenter/EasyInstall#administrator-installation http://peak.telecommunity.com/DevCenter/EasyInstall#mac-os-x-user-installation Meanwhile, note that having per-user site-packages directories doesn't eliminate the need to be able to have PYTHONPATH directories treated as "site" directories, which is hasn't been discussed at all. >As such the PEP can also be seen as an effort to enable code >cleanup *before* adding e.g. pkg_resources to the stdlib. Code cleanup of what? There's nothing in pkg_resources that would change for per-user site package directories, since pkg_resources doesn't do any installation work. From tomerfiliba at gmail.com Tue Jan 22 20:26:59 2008 From: tomerfiliba at gmail.com (tomer filiba) Date: Tue, 22 Jan 2008 11:26:59 -0800 (PST) Subject: [Python-Dev] misbehaving __contains__ Message-ID: <9834f887-7e99-4c3f-9ec1-b28f798b36b4@s13g2000prd.googlegroups.com> i'm using python to create expression objects, for more intuitive usage as predicates, for instance: x = (Arg(0) > 17) & (Arg(1).foo == "bar") instead of x = And(Gt(Arg(0), 17), Eq(GetAttr(Arg(1), "foo"), "bar")) so now i can use x.eval(18, "spam") and get the result of the expression. i'm doing it by overriding all the operators to return expression objects, instead of evaluating the result immediately. it works fine, but i encountered a problem with making __contains__ behave so. it turns out __contains__ coerces the return value into a bool. this might seem logical at first, but is not consistent with the result of the language. consider the following code: >>> class Foo(object): ... def __contains__(self, key): ... return 17 ... def __eq__(self, other): ... return 19 ... >>> >>> f=Foo() >>> f == 8 19 >>> 8 in f True as you can see, __eq__ does not coerces the result to bool, so why should __contains__ do that? i've looked at PySequence_Contains in objects/abstract.c, but i can't quite understand where the coercion is made. is because the function is typed as int? if so, i believe it should either be changed to PyObject, or have cmp_outcome (in ceval.c) not use this API directly, and rather return the result as it is returned from __contains__. i see no reason it should behave differently than __eq__ and the rest of comparison operators. -tomer From cmason at vcentertainment.com Tue Jan 22 20:35:12 2008 From: cmason at vcentertainment.com (Chuck Mason (Visual Concepts)) Date: Tue, 22 Jan 2008 11:35:12 -0800 Subject: [Python-Dev] misbehaving __contains__ In-Reply-To: <9834f887-7e99-4c3f-9ec1-b28f798b36b4@s13g2000prd.googlegroups.com> Message-ID: <0637A91E30020C4AB137E582438A14B20C549A@2KGNOVEXG01.2kgames.t2.corp> My guess is that results from the bottom of cmp_outcome (ceval.c), The default case handles the PyCmp_EQ case via PyObject_RichCompare, which might not return Py_True or Py_False. But 'in' is handled inside the switch and is subject to the final statements: v = res ? Py_True : Py_False; Py_INCREF(v); return v Though, IANAPydev. C -----Original Message----- From: python-dev-bounces+cmason=vcentertainment.com at python.org [mailto:python-dev-bounces+cmason=vcentertainment.com at python.org] On Behalf Of tomer filiba Sent: Tuesday, January 22, 2008 11:27 AM To: python-dev at python.org Subject: [Python-Dev] misbehaving __contains__ i'm using python to create expression objects, for more intuitive usage as predicates, for instance: x = (Arg(0) > 17) & (Arg(1).foo == "bar") instead of x = And(Gt(Arg(0), 17), Eq(GetAttr(Arg(1), "foo"), "bar")) so now i can use x.eval(18, "spam") and get the result of the expression. i'm doing it by overriding all the operators to return expression objects, instead of evaluating the result immediately. it works fine, but i encountered a problem with making __contains__ behave so. it turns out __contains__ coerces the return value into a bool. this might seem logical at first, but is not consistent with the result of the language. consider the following code: >>> class Foo(object): ... def __contains__(self, key): ... return 17 ... def __eq__(self, other): ... return 19 ... >>> >>> f=Foo() >>> f == 8 19 >>> 8 in f True as you can see, __eq__ does not coerces the result to bool, so why should __contains__ do that? i've looked at PySequence_Contains in objects/abstract.c, but i can't quite understand where the coercion is made. is because the function is typed as int? if so, i believe it should either be changed to PyObject, or have cmp_outcome (in ceval.c) not use this API directly, and rather return the result as it is returned from __contains__. i see no reason it should behave differently than __eq__ and the rest of comparison operators. -tomer _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/cmason%40vcentertainme nt.com From guido at python.org Tue Jan 22 20:55:19 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 22 Jan 2008 11:55:19 -0800 Subject: [Python-Dev] misbehaving __contains__ In-Reply-To: <9834f887-7e99-4c3f-9ec1-b28f798b36b4@s13g2000prd.googlegroups.com> References: <9834f887-7e99-4c3f-9ec1-b28f798b36b4@s13g2000prd.googlegroups.com> Message-ID: The issue is that the sq_contains slot is defined as returning an int, while the tp_richcompare slot is defined as returning a PyObject *. Your first opportunity for changing this will be Py3k. Please submit a patch! On Jan 22, 2008 11:26 AM, tomer filiba wrote: > i'm using python to create expression objects, for more intuitive > usage as > predicates, for instance: > x = (Arg(0) > 17) & (Arg(1).foo == "bar") > instead of > x = And(Gt(Arg(0), 17), Eq(GetAttr(Arg(1), "foo"), "bar")) > > so now i can use x.eval(18, "spam") and get the result of the > expression. > > i'm doing it by overriding all the operators to return expression > objects, > instead of evaluating the result immediately. it works fine, but i > encountered > a problem with making __contains__ behave so. > > it turns out __contains__ coerces the return value into a bool. this > might > seem logical at first, but is not consistent with the result of the > language. > > consider the following code: > > >>> class Foo(object): > ... def __contains__(self, key): > ... return 17 > ... def __eq__(self, other): > ... return 19 > ... > >>> > >>> f=Foo() > >>> f == 8 > 19 > >>> 8 in f > True > > as you can see, __eq__ does not coerces the result to bool, so why > should > __contains__ do that? > > i've looked at PySequence_Contains in objects/abstract.c, but i can't > quite > understand where the coercion is made. is because the function is > typed > as int? if so, i believe it should either be changed to PyObject, or > have > cmp_outcome (in ceval.c) not use this API directly, and rather return > the > result as it is returned from __contains__. > > i see no reason it should behave differently than __eq__ and the rest > of comparison operators. > > > -tomer > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From mal at egenix.com Tue Jan 22 22:24:40 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 22 Jan 2008 22:24:40 +0100 Subject: [Python-Dev] #! magic In-Reply-To: <4793932B.9010106@cheimes.de> References: <47894171.7030004@cheimes.de> <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> <4789DDA0.4020900@cheimes.de> <1200848275.24656.95.camel@localhost> <20080120171238.GA16055@phd.pp.ru> <20080120172557.GH61556@nexus.in-nomine.org> <20080120173828.GB16055@phd.pp.ru> <4793932B.9010106@cheimes.de> Message-ID: <47965F18.50907@egenix.com> On 2008-01-20 19:30, Christian Heimes wrote: > Yet another python executable could solve the issue, named "pythons" as > python secure. > > /* > gcc -DNDEBUG -g -O2 -Wall -Wstrict-prototypes -IInclude -I. -pthread > -Xlinker -lpthread -ldl -lutil -lm -export-dynamic -o pythons2.6 > > pythons.c libpython2.6.a > */ > > #include "Python.h" > > int main(int argc, char **argv) { > /* disable some possible harmful features */ > Py_IgnoreEnvironmentFlag++; > Py_NoUserSiteDirectory++; > Py_InteractiveFlag -= INT_MAX; > Py_InspectFlag -= INT_MAX; > > return Py_Main(argc, argv); > } > > $ ./pythons2.6 > Python 2.6a0 (:59956M, Jan 14 2008, 22:09:17) > [GCC 4.2.1 (Ubuntu 4.2.1-5ubuntu4)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> import sys >>>> sys.flags > sys.flags(debug=0, py3k_warning=0, division_warning=0, division_new=0, > inspect=-2147483647, interactive=-2147483647, optimize=0, > dont_write_bytecode=0, no_user_site=1, no_site=0, ingnore_environment=1, Is this a copy&paste error or a typo in the code ----^ ? > tabcheck=0, verbose=0, unicode=0) To make this even more secure, you'd have to package this up together with a copy of the stdlib, but like mxCGIPython does (or did... I have to revive that project at some point :-): http://www.egenix.com/www2002/python/mxCGIPython.html -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 22 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From daniel at stutzbachenterprises.com Wed Jan 23 00:46:48 2008 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Tue, 22 Jan 2008 17:46:48 -0600 Subject: [Python-Dev] misbehaving __contains__ In-Reply-To: <9834f887-7e99-4c3f-9ec1-b28f798b36b4@s13g2000prd.googlegroups.com> References: <9834f887-7e99-4c3f-9ec1-b28f798b36b4@s13g2000prd.googlegroups.com> Message-ID: On Jan 22, 2008 1:26 PM, tomer filiba wrote: > >>> class Foo(object): > ... def __contains__(self, key): > ... return 17 > ... def __eq__(self, other): > ... return 19 > ... > >>> > >>> f=Foo() > >>> f == 8 > 19 > >>> 8 in f > True There are many places in the C implementation where a slot returns an int rather than a PyObject. There other replies in this thread seem to support altering the signature of the slot to return a PyObject. Is this setting a precedent that _all_ slots should return a PyObject? Consider the following third behavior: >>> class foo(object): ... def __len__(self): ... return 'foo' ... >>> x = foo() >>> len(x) Traceback (most recent call last): File "", line 1, in TypeError: an integer is required -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From guido at python.org Wed Jan 23 00:51:12 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 22 Jan 2008 15:51:12 -0800 Subject: [Python-Dev] misbehaving __contains__ In-Reply-To: References: <9834f887-7e99-4c3f-9ec1-b28f798b36b4@s13g2000prd.googlegroups.com> Message-ID: On Jan 22, 2008 3:46 PM, Daniel Stutzbach wrote: > On Jan 22, 2008 1:26 PM, tomer filiba wrote: > > >>> class Foo(object): > > ... def __contains__(self, key): > > ... return 17 > > ... def __eq__(self, other): > > ... return 19 > > ... > > >>> > > >>> f=Foo() > > >>> f == 8 > > 19 > > >>> 8 in f > > True > > There are many places in the C implementation where a slot returns an > int rather than a PyObject. There other replies in this thread seem > to support altering the signature of the slot to return a PyObject. > Is this setting a precedent that _all_ slots should return a PyObject? > > Consider the following third behavior: > > >>> class foo(object): > ... def __len__(self): > ... return 'foo' > ... > >>> x = foo() > >>> len(x) > Traceback (most recent call last): > File "", line 1, in > TypeError: an integer is required Possibly. I guess it depends on the use case. It would be nice to research this more, e.g. figure out how much code needs to be changed to make each of these possible changes, and how likely there will be a use case. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Wed Jan 23 01:40:16 2008 From: python at rcn.com (Raymond Hettinger) Date: Tue, 22 Jan 2008 19:40:16 -0500 (EST) Subject: [Python-Dev] misbehaving __contains__ Message-ID: <20080122194016.AFN25681@ms19.lnh.mail.rcn.net> [Daniel Stutzbach] > There are many places in the C implementation where a slot > returns an int rather than a PyObject. There other replies > in this thread seem to support altering the signature of the > slot to return a PyObject. Is this setting a precedent that > _all_ slots should return a PyObject? I hope not. One cost of these hypergeneralizations is that it makes Python slower. Also, it makes it harder to write code when you can no longer count on simple predicates to return either True or False. [tomer filiba] > i see no reason it should behave differently than > __eq__ and the rest of comparison operators The rest of the equality operators were a special case to allow rich comparisons which were needed to implement some vector arithmetic idioms. It was thought that those use case had enough benefit to pay for the negative impacts: * making the language more complex * slowing everything down a bit. * complicating the underlying implementation * making the language more difficult to learn (rich operators are harder than cmp) Does making __contains__ return a PyObject have use cases that are worth those costs? It is reminiscent of the ill-fated proposals to overrride 'and', 'or', 'is', and 'is not'. A couple more questions about the proposal: If __contains__ returns 17 in Tomer's example, what would "8 not in f" return? And, would we lose the nice relationship expressed by: for elem in container: assert elem in container It looks like changing the semantics would introduce some weirdness. Raymond From steven.bethard at gmail.com Wed Jan 23 02:03:57 2008 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue, 22 Jan 2008 18:03:57 -0700 Subject: [Python-Dev] misbehaving __contains__ In-Reply-To: <20080122194016.AFN25681@ms19.lnh.mail.rcn.net> References: <20080122194016.AFN25681@ms19.lnh.mail.rcn.net> Message-ID: On Jan 22, 2008 5:40 PM, Raymond Hettinger wrote: > [Daniel Stutzbach] > > There are many places in the C implementation where a slot > > returns an int rather than a PyObject. There other replies > > in this thread seem to support altering the signature of the > > slot to return a PyObject. Is this setting a precedent that > > _all_ slots should return a PyObject? > > I hope not. [snip] > And, would we lose the nice relationship expressed by: > > for elem in container: > assert elem in container We've already lost this if anyone really wants to break it:: >>> class C(object): ... def __iter__(self): ... return iter(xrange(3)) ... def __contains__(self, item): ... return False ... >>> c = C() >>> for item in c: ... print item in c ... False False False Of course, anyone who decides to break their container classes in that way is asking for trouble. ;-) STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From python at rcn.com Wed Jan 23 02:19:08 2008 From: python at rcn.com (Raymond Hettinger) Date: Tue, 22 Jan 2008 20:19:08 -0500 (EST) Subject: [Python-Dev] misbehaving __contains__ Message-ID: <20080122201908.AFN30890@ms19.lnh.mail.rcn.net> >> And, would we lose the nice relationship expressed by: >> >> for elem in container: >> assert elem in container [Steven Bethard] >We've already lost this if anyone really wants to break it:: > > >>> class C(object): > ... def __iter__(self): > ... return iter(xrange(3)) > ... def __contains__(self, item): > ... return False I think this is in the same category as writing a __hash__ that doens't match __eq__. It is explicitly shooting yourself in the foot. I'm more concerned about introducing an API where well-meaning attempts to code a __contains__ override will implicitly shoot the toes off of client code that innocently assumes a somewhat sensible invariant relation between looping over elements in a container and those elements being in the container. The code for sets and frozensets makes that assumption, for example. And, when the code does break, the errors will be darned hard to find. Raymond From guido at python.org Wed Jan 23 05:47:02 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 22 Jan 2008 20:47:02 -0800 Subject: [Python-Dev] 2.5.2 release coming up Message-ID: While the exact release schedule for 2.5.2 is still up in the air, I expect that it will be within a few weeks. This means that we need to make sure that anything that should go into 2.5.2 goes in ASAP, preferably this week. It also means that we should be very careful what goes in though -- and we should be paying particular attention to stability on all platforms! Fortunately it looks like quite a few 2.5 buildbots are green: http://python.org/dev/buildbot/2.5/ I propose that anything that ought to go into 2.5.2 (or should be reviewed for suitability to go into it) should be marked "urgent" in the tracker, *and* have its version set to (or include) "Python 2.5". Also, *nothing* should go into the 2.4 branch any more *except* important security patches. If we're doing a security release, it'll most likely be a source-only release, and it will differ from 2.4.4 only in that it will have some new security patches defined. A reminder: 2.5.2 should only get bugfixes, new features. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Wed Jan 23 05:57:02 2008 From: brett at python.org (Brett Cannon) Date: Tue, 22 Jan 2008 20:57:02 -0800 Subject: [Python-Dev] 2.5.2 release coming up In-Reply-To: References: Message-ID: On Jan 22, 2008 8:47 PM, Guido van Rossum wrote: > While the exact release schedule for 2.5.2 is still up in the air, I > expect that it will be within a few weeks. This means that we need to > make sure that anything that should go into 2.5.2 goes in ASAP, > preferably this week. It also means that we should be very careful > what goes in though -- and we should be paying particular attention to > stability on all platforms! Fortunately it looks like quite a few 2.5 > buildbots are green: http://python.org/dev/buildbot/2.5/ > > I propose that anything that ought to go into 2.5.2 (or should be > reviewed for suitability to go into it) should be marked "urgent" in > the tracker, *and* have its version set to (or include) "Python 2.5". > > Also, *nothing* should go into the 2.4 branch any more *except* > important security patches. If we're doing a security release, it'll > most likely be a source-only release, and it will differ from 2.4.4 > only in that it will have some new security patches defined. > > A reminder: 2.5.2 should only get bugfixes, new features. If Guido felt like dragging the time machine out he would catch his mistake and have that say "NO new features". -Brett From jyasskin at gmail.com Wed Jan 23 06:20:48 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Tue, 22 Jan 2008 21:20:48 -0800 Subject: [Python-Dev] Rational approximation methods In-Reply-To: <1f7befae0801201754j505a9a59t8cacf90c279fc8b0@mail.gmail.com> References: <5d44f72f0801191206x1027a49do62376026727fbb@mail.gmail.com> <1f7befae0801201754j505a9a59t8cacf90c279fc8b0@mail.gmail.com> Message-ID: <5d44f72f0801222120y6597243bo79d8c14ccd934c00@mail.gmail.com> On Jan 20, 2008 5:54 PM, Tim Peters wrote: > What would be useful is a method that generates (i.e., a generator in > the Python sense) the (continued fraction) convergents to a rational. > People wanting specific constraints on a rational approximation > (including, but not limited to, the two you identified) can easily > build them on top of such a generator. I think that's significantly better than my two proposals. (Looking back at the issue, I see that Mark mentioned this possibility too and I just didn't understand him.) I'll plan on implementing that, along with the documentation Mark suggested, unless there are objections. Thanks! Jeffrey From stephen at xemacs.org Wed Jan 23 07:24:02 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 23 Jan 2008 15:24:02 +0900 Subject: [Python-Dev] 2.5.2 release coming up In-Reply-To: References: Message-ID: <87lk6hoy9p.fsf@uwakimon.sk.tsukuba.ac.jp> Brett Cannon writes: > On Jan 22, 2008 8:47 PM, Guido van Rossum wrote: > > A reminder: 2.5.2 should only get bugfixes, new features. > > If Guido felt like dragging the time machine out he would catch his > mistake and have that say "NO new features". What, nothing about "Read my lips"? From lists at cheimes.de Wed Jan 23 07:57:28 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 23 Jan 2008 07:57:28 +0100 Subject: [Python-Dev] #! magic In-Reply-To: <47965F18.50907@egenix.com> References: <47894171.7030004@cheimes.de> <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> <4789DDA0.4020900@cheimes.de> <1200848275.24656.95.camel@localhost> <20080120171238.GA16055@phd.pp.ru> <20080120172557.GH61556@nexus.in-nomine.org> <20080120173828.GB16055@phd.pp.ru> <4793932B.9010106@cheimes.de> <47965F18.50907@egenix.com> Message-ID: <4796E558.4040704@cheimes.de> M.-A. Lemburg wrote: >> sys.flags(debug=0, py3k_warning=0, division_warning=0, division_new=0, >> inspect=-2147483647, interactive=-2147483647, optimize=0, >> dont_write_bytecode=0, no_user_site=1, no_site=0, ingnore_environment=1, > > Is this a copy&paste error or a typo in the code ----^ ? It's a typo in the code. It's already fixed in the trunk but it was not fixed in the 370 branch. Christian From lists at cheimes.de Wed Jan 23 07:57:28 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 23 Jan 2008 07:57:28 +0100 Subject: [Python-Dev] #! magic In-Reply-To: <47965F18.50907@egenix.com> References: <47894171.7030004@cheimes.de> <52dc1c820801121708h5ed25b7evc597306dca2058e4@mail.gmail.com> <4789DDA0.4020900@cheimes.de> <1200848275.24656.95.camel@localhost> <20080120171238.GA16055@phd.pp.ru> <20080120172557.GH61556@nexus.in-nomine.org> <20080120173828.GB16055@phd.pp.ru> <4793932B.9010106@cheimes.de> <47965F18.50907@egenix.com> Message-ID: <4796E558.4040704@cheimes.de> M.-A. Lemburg wrote: >> sys.flags(debug=0, py3k_warning=0, division_warning=0, division_new=0, >> inspect=-2147483647, interactive=-2147483647, optimize=0, >> dont_write_bytecode=0, no_user_site=1, no_site=0, ingnore_environment=1, > > Is this a copy&paste error or a typo in the code ----^ ? It's a typo in the code. It's already fixed in the trunk but it was not fixed in the 370 branch. Christian From schmir at gmail.com Wed Jan 23 08:53:56 2008 From: schmir at gmail.com (Ralf Schmitt) Date: Wed, 23 Jan 2008 08:53:56 +0100 Subject: [Python-Dev] why is mmap a builtin module on windows? Message-ID: <932f8baf0801222353v2593e6edq556a6fe5a3e25e60@mail.gmail.com> Hi all, I want to use the mmap module from python trunk with python 2.5. On Linux I can easily replace it, as it is a dynamically loaded module. On windows it is a builtin module and I fear that I must compile python on windows (or resort to some other ugly hack) What is the reason for mmap being a builtin module? Regards, - Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080123/3bb2f00f/attachment-0001.htm From lists at cheimes.de Wed Jan 23 09:01:32 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 23 Jan 2008 09:01:32 +0100 Subject: [Python-Dev] why is mmap a builtin module on windows? In-Reply-To: <932f8baf0801222353v2593e6edq556a6fe5a3e25e60@mail.gmail.com> References: <932f8baf0801222353v2593e6edq556a6fe5a3e25e60@mail.gmail.com> Message-ID: Ralf Schmitt wrote: > Hi all, > > I want to use the mmap module from python trunk with python 2.5. > On Linux I can easily replace it, as it is a dynamically loaded module. On > windows > it is a builtin module and I fear that I must compile python on windows (or > resort to some other ugly hack) > > What is the reason for mmap being a builtin module? On Windows lots of modules are linked into the python main dll. The file PC/config.c contains a list of all modules. From the point of the maintainer it's much easier to link the modules into the main dll instead of creating standalone dlls. I also suspect that it's much faster because relocation is slow (see PC/dllbase_nt.txt). Martin or Mark can give you a better answer. Why do you want to overwrite the existing module instead of using a different name like ralfmmap? import ralfmmap as mmap sys.modules]'mmap'] = mmap Christian From schmir at gmail.com Wed Jan 23 09:29:59 2008 From: schmir at gmail.com (Ralf Schmitt) Date: Wed, 23 Jan 2008 09:29:59 +0100 Subject: [Python-Dev] why is mmap a builtin module on windows? In-Reply-To: References: <932f8baf0801222353v2593e6edq556a6fe5a3e25e60@mail.gmail.com> Message-ID: <932f8baf0801230029j75432701od05c31a3fbb7329a@mail.gmail.com> On Jan 23, 2008 9:01 AM, Christian Heimes wrote: > > > > What is the reason for mmap being a builtin module? > > On Windows lots of modules are linked into the python main dll. The file > PC/config.c contains a list of all modules. From the point of the > maintainer it's much easier to link the modules into the main dll > instead of creating standalone dlls. I also suspect that it's much > faster because relocation is slow (see PC/dllbase_nt.txt). Martin or > Mark can give you a better answer. > ok > > Why do you want to overwrite the existing module instead of using a > different name like ralfmmap? > > import ralfmmap as mmap > sys.modules]'mmap'] = mmap > I thought about that (ugly hack) too. I would have to change the imports at a lot of places (and revert the changes when we switch to python 2.6). (whereas on Unix I would only have to do install the new mmap module). - Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080123/4fdc6b4a/attachment.htm From lists at cheimes.de Wed Jan 23 09:35:50 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 23 Jan 2008 09:35:50 +0100 Subject: [Python-Dev] why is mmap a builtin module on windows? In-Reply-To: <932f8baf0801230029j75432701od05c31a3fbb7329a@mail.gmail.com> References: <932f8baf0801222353v2593e6edq556a6fe5a3e25e60@mail.gmail.com> <932f8baf0801230029j75432701od05c31a3fbb7329a@mail.gmail.com> Message-ID: <4796FC66.4090908@cheimes.de> Ralf Schmitt wrote: >> Why do you want to overwrite the existing module instead of using a >> different name like ralfmmap? >> >> import ralfmmap as mmap >> sys.modules]'mmap'] = mmap >> > > I thought about that (ugly hack) too. I would have to change the imports at > a lot of places (and revert the changes when we switch to python 2.6). > (whereas on Unix I would only have to do install the new mmap module). It's not an ugly hack, it's a well known feature. Add you don't have to change a lot of places, too. It's sufficient to add the alias at the entry point of your application (the script that starts your app). Once the alias sys.modules]'mmap'] = ralfmmap is set, every import mmap gets your ralfmmap. Christian From schmir at gmail.com Wed Jan 23 09:46:15 2008 From: schmir at gmail.com (Ralf Schmitt) Date: Wed, 23 Jan 2008 09:46:15 +0100 Subject: [Python-Dev] why is mmap a builtin module on windows? In-Reply-To: <4796FC66.4090908@cheimes.de> References: <932f8baf0801222353v2593e6edq556a6fe5a3e25e60@mail.gmail.com> <932f8baf0801230029j75432701od05c31a3fbb7329a@mail.gmail.com> <4796FC66.4090908@cheimes.de> Message-ID: <932f8baf0801230046n2adb6726l869ce9063a6da876@mail.gmail.com> On Jan 23, 2008 9:35 AM, Christian Heimes wrote: > > It's not an ugly hack, it's a well known feature. Add you don't have to > change a lot of places, too. It's sufficient to add the alias at the > entry point of your application (the script that starts your app). Once > the alias sys.modules]'mmap'] = ralfmmap is set, every import mmap gets > your ralfmmap. Well, I have multiple scripts using multiple libraries using the new mmap objects. So, I would have to change those scripts (if I don't change the libraries), or change the libraries. And when I want to distribute my apps with bbfreeze (similar to py2exe), I also need to handle that import hackery. This is what I call ugly :) (at least in comparison to linux). Regards, - Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080123/41bbb693/attachment.htm From amauryfa at gmail.com Wed Jan 23 11:28:43 2008 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Wed, 23 Jan 2008 11:28:43 +0100 Subject: [Python-Dev] why is mmap a builtin module on windows? In-Reply-To: <932f8baf0801230046n2adb6726l869ce9063a6da876@mail.gmail.com> References: <932f8baf0801222353v2593e6edq556a6fe5a3e25e60@mail.gmail.com> <932f8baf0801230029j75432701od05c31a3fbb7329a@mail.gmail.com> <4796FC66.4090908@cheimes.de> <932f8baf0801230046n2adb6726l869ce9063a6da876@mail.gmail.com> Message-ID: Hello, Ralf Schmitt: > > It's not an ugly hack, it's a well known feature. Add you don't have to > > change a lot of places, too. It's sufficient to add the alias at the > > entry point of your application (the script that starts your app). Once > > the alias sys.modules]'mmap'] = ralfmmap is set, every import mmap gets > > your ralfmmap. > > Well, I have multiple scripts using multiple libraries using the new mmap > objects. > So, I would have to change those scripts (if I don't change the libraries), > or change the libraries. And when I want to distribute my apps with bbfreeze > (similar to py2exe), I also need to handle that import hackery. This is > what I call ugly :) (at least in comparison to linux). You can also modify site.py and put the hack there. -- Amaury Forgeot d'Arc From tomerfiliba at gmail.com Wed Jan 23 14:21:44 2008 From: tomerfiliba at gmail.com (tomer filiba) Date: Wed, 23 Jan 2008 15:21:44 +0200 Subject: [Python-Dev] misbehaving __contains__ In-Reply-To: <20080122201908.AFN30890@ms19.lnh.mail.rcn.net> References: <20080122201908.AFN30890@ms19.lnh.mail.rcn.net> Message-ID: <1d85506f0801230521sca37768l97c0a3108c6a8616@mail.gmail.com> On Jan 23, 2008 3:19 AM, Raymond Hettinger wrote: > [Steven Bethard] > > >We've already lost this if anyone really wants to break it:: > > > > >>> class C(object): > > ... def __iter__(self): > > ... return iter(xrange(3)) > > ... def __contains__(self, item): > > ... return False [snip] > I'm more concerned about introducing an API where well-meaning attempts to code a __contains__ override will implicitly shoot the toes off of client code that innocently assumes a somewhat sensible invariant relation between looping over elements in a container and those elements being in the container. The code for sets and frozensets makes that assumption, for example. And, when the code does break, the errors will be darned hard to find. well, i don't see how limiting __contains__ to returning booleans solves the problem you mentioned. what you're talking about is *semantics*, and that's always up to the programmer to get right. allowing __contains__ to return a FooBar object won't change that a bit -- but it will allow me to construct expression objects more easily. and, statements like if x in y: pass will work as expected, assuming the object returned by y.__contains__(x) has a proper __bool__/__nonzero__ method. we're just deferring the type-cast to the point it's actually needed. as per performance, i don't believe changing a slot method to return PyObject* instead of int would change anything... am i wrong? -tomer From oliphant.travis at ieee.org Wed Jan 23 17:17:01 2008 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 23 Jan 2008 10:17:01 -0600 Subject: [Python-Dev] Error in PEP3118? In-Reply-To: <47905DAE.9020802@ctypes.org> References: <47905DAE.9020802@ctypes.org> Message-ID: Thomas Heller wrote: > Hi Travis, > > The pep contains this sample: > > """ > Nested array > :: > > struct { > int ival; > double data[16*4]; > } > """i:ival: > (16,4)d:data: > """ > """ > > I think it is wrong and must be changed to the following; is this correct? > > """ > Nested array > :: > > struct { > int ival; > double data[16][4]; > } > """i:ival: > (16,4)d:data: > """ > """ I responded off list to this email and wanted to summarize my response for others to peruse. Basically, the answer is that the struct syntax proposed for multi-dimensional arrays is not intended to mimic how the C-compiler handles statically defined C-arrays (i.e. the pointer-to-pointers style of multi-dimensional arrays). It is intended to handle the contiguous-block-of-data style of multi-dimensional arrays that NumPy uses. I wanted to avoid 2-d static arrays in the examples because it gets confusing and AFAIK the layout of the memory for a double data[16][4] is the same as data[16*4]. The only difference is how the C-compiler translates data[4][3] and data[4]. The intent of the struct syntax is to handle describing memory. The point is not to replicate how the C-compiler deals with statically defined N-D arrays. Thus, even though the struct syntax allows *communicating* the intent of a contiguous block of memory inside a structure as an N-d array, the fundamental memory block is the equivalent of a 1-d array in C. So, I think the example is correct (and intentional). -Travis O. From stephenemslie at gmail.com Wed Jan 23 17:38:12 2008 From: stephenemslie at gmail.com (stephen emslie) Date: Wed, 23 Jan 2008 16:38:12 +0000 Subject: [Python-Dev] rfc822_escape doing the right thing? Message-ID: <51f97e530801230838t472a0d1fv24487c94ee42ca4f@mail.gmail.com> I've been working on a project that renders PKG-INFO metadata in a number of ways. I have noticed that fields with any indentation were flattened out, which is being done in distutils.util.rfc822_escape. This unfortunately means that you cant use reStructuredText formatting in your long description (suggested in PEP345), or are limited to a set that doesn't require indentation (no block quotes, etc.). It looks like this behavior was intentionally added in rev 20099, but that was about 7 years ago - before reStructuredText and eggs. I wonder if it makes sense to re-think that implementation with this sort of metadata in mind, assuming this behavior isn't required to be rfc822 compliant. I think it would certainly be a shame to miss out on a good thing like proper (renderable) reST in our metadata. A quick example of what I mean: >>> rest = """ ... a literal python block:: ... >>> import this ... """ >>> print distutils.util.rfc822_escape(rest) a literal python block:: >>> import this should look something like: a literal python block:: >>> import this Is distutils being over-cautious in flattening out all whitespace? A w3c discussion on multiple lines in rfc822 [1] seems to suggest that whitespace can be 'unfolded' safely, so it seems a shame to be throwing it away when it can have important meaning. [1] http://www.w3.org/Protocols/rfc822/3_Lexical.html Thanks for any comments Stephen Emslie From guido at python.org Wed Jan 23 17:45:23 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 23 Jan 2008 08:45:23 -0800 Subject: [Python-Dev] Error in PEP3118? In-Reply-To: References: <47905DAE.9020802@ctypes.org> Message-ID: Travis, Perhaps you can add this rationale to the PEP? It seems helpful and might stave off future confusion. --Guido On Jan 23, 2008 8:17 AM, Travis Oliphant wrote: > Thomas Heller wrote: > > Hi Travis, > > > > The pep contains this sample: > > > > """ > > Nested array > > :: > > > > struct { > > int ival; > > double data[16*4]; > > } > > """i:ival: > > (16,4)d:data: > > """ > > """ > > > > I think it is wrong and must be changed to the following; is this correct? > > > > """ > > Nested array > > :: > > > > struct { > > int ival; > > double data[16][4]; > > } > > """i:ival: > > (16,4)d:data: > > """ > > """ > > I responded off list to this email and wanted to summarize my response > for others to peruse. > > Basically, the answer is that the struct syntax proposed for > multi-dimensional arrays is not intended to mimic how the C-compiler > handles statically defined C-arrays (i.e. the pointer-to-pointers style > of multi-dimensional arrays). It is intended to handle the > contiguous-block-of-data style of multi-dimensional arrays that NumPy uses. > > I wanted to avoid 2-d static arrays in the examples because it gets > confusing and AFAIK the layout of the memory for a double data[16][4] is > the same as data[16*4]. The only difference is how the C-compiler > translates data[4][3] and data[4]. > > The intent of the struct syntax is to handle describing memory. The > point is not to replicate how the C-compiler deals with statically > defined N-D arrays. Thus, even though the struct syntax allows > *communicating* the intent of a contiguous block of memory inside a > structure as an N-d array, the fundamental memory block is the > equivalent of a 1-d array in C. > > So, I think the example is correct (and intentional). > > -Travis O. > > > > > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From eric+python-dev at trueblade.com Wed Jan 23 16:40:57 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Wed, 23 Jan 2008 10:40:57 -0500 Subject: [Python-Dev] Is anyone porting PyNumber_ToBase to trunk? Message-ID: <47976009.5040107@trueblade.com> In 3.0, the code to implement long.__format__() calls PyNumber_ToBase to do the heavy lifting. If someone is planning on implementing PyNumber_ToBase in 2.6, I'll wait for them to do so. If not, I guess I'll either do it myself, or hack around it. Is this on anyone's To Do list? I don't see it on Brett's spreadsheet at http://spreadsheets.google.com/pub?key=pCKY4oaXnT81FrGo3ShGHGg From guido at python.org Wed Jan 23 17:56:38 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 23 Jan 2008 08:56:38 -0800 Subject: [Python-Dev] Is anyone porting PyNumber_ToBase to trunk? In-Reply-To: <47976009.5040107@trueblade.com> References: <47976009.5040107@trueblade.com> Message-ID: On Jan 23, 2008 7:40 AM, Eric Smith wrote: > In 3.0, the code to implement long.__format__() calls PyNumber_ToBase to > do the heavy lifting. If someone is planning on implementing > PyNumber_ToBase in 2.6, I'll wait for them to do so. If not, I guess > I'll either do it myself, or hack around it. > > Is this on anyone's To Do list? I don't see it on Brett's spreadsheet > at http://spreadsheets.google.com/pub?key=pCKY4oaXnT81FrGo3ShGHGg I'm sure this is because nobody had thought of this detail until now. Just get started on it. Feel free to add it to the spreadsheet. (Anyone who needs an invite for editing the spreadsheet, let me know the email to which I should send it to -- either a gmail address or an email associated with a Google account. If you don't have a Google account yet, go here: https://www.google.com/accounts/NewAccount -- no gmail address needed!) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From theller at ctypes.org Wed Jan 23 20:03:55 2008 From: theller at ctypes.org (Thomas Heller) Date: Wed, 23 Jan 2008 20:03:55 +0100 Subject: [Python-Dev] Error in PEP3118? In-Reply-To: References: <47905DAE.9020802@ctypes.org> Message-ID: Travis Oliphant schrieb: [...] > I responded off list to this email and wanted to summarize my response > for others to peruse. > > Basically, the answer is that the struct syntax proposed for > multi-dimensional arrays is not intended to mimic how the C-compiler > handles statically defined C-arrays (i.e. the pointer-to-pointers style > of multi-dimensional arrays). It is intended to handle the > contiguous-block-of-data style of multi-dimensional arrays that NumPy uses. > > I wanted to avoid 2-d static arrays in the examples because it gets > confusing and AFAIK the layout of the memory for a double data[16][4] is > the same as data[16*4]. The only difference is how the C-compiler > translates data[4][3] and data[4]. > > The intent of the struct syntax is to handle describing memory. The > point is not to replicate how the C-compiler deals with statically > defined N-D arrays. Thus, even though the struct syntax allows > *communicating* the intent of a contiguous block of memory inside a > structure as an N-d array, the fundamental memory block is the > equivalent of a 1-d array in C. > > So, I think the example is correct (and intentional). Sorry, I do not think so. If you use a 2-d array in the example, you must describe it correctly. The difference between this pep and the old buffer interface is that the pep allows to describe both how the compiler sees the memory block plus the size and layout of the memory block, while the old buffer interface only describes single-segment memory blocks. And 'double data[16][4]' *is* a single memory block containing a 2-d array, and *not* an array of pointers. --- Here is another typo (?) in the pep; I think it should be changed: Index: pep-3118.txt =================================================================== --- pep-3118.txt (revision 60037) +++ pep-3118.txt (working copy) @@ -338,7 +338,7 @@ ``len`` the total bytes of memory the object uses. This should be the - same as the product of the shape array multiplied by the number of + same as the length of the shape array multiplied by the number of bytes per item of memory. ``readonly`` After all, imo there's a lot to do to fully implement the pep for python 2.6. Thomas From gnewsg at gmail.com Wed Jan 23 20:21:37 2008 From: gnewsg at gmail.com (Giampaolo Rodola') Date: Wed, 23 Jan 2008 11:21:37 -0800 (PST) Subject: [Python-Dev] 2.5.2 release coming up In-Reply-To: References: Message-ID: > Also, *nothing* should go into the 2.4 branch any more *except* > important security patches. http://bugs.python.org/issue1745035 I guess this one should concern both 2.4 and 2.5 branches. On 23 Gen, 05:47, "Guido van Rossum" wrote: > While the exact release schedule for 2.5.2 is still up in the air, I > expect that it will be within a few weeks. This means that we need to > make sure that anything that should go into 2.5.2 goes in ASAP, > preferably this week. It also means that we should be very careful > what goes in though -- and we should be paying particular attention to > stability on all platforms! Fortunately it looks like quite a few 2.5 > buildbots are green:http://python.org/dev/buildbot/2.5/ > > I propose that anything that ought to go into 2.5.2 (or should be > reviewed for suitability to go into it) should be marked "urgent" in > the tracker, *and* have its version set to (or include) "Python 2.5". > > Also, *nothing* should go into the 2.4 branch any more *except* > important security patches. If we're doing a security release, it'll > most likely be a source-only release, and it will differ from 2.4.4 > only in that it will have some new security patches defined. > > A reminder: 2.5.2 should only get bugfixes, new features. > > -- > --Guido van Rossum (home page:http://www.python.org/~guido/) > _______________________________________________ > Python-Dev mailing list > Python-... at python.orghttp://mail.python.org/mailman/listinfo/python-dev > Unsubscribe:http://mail.python.org/mailman/options/python-dev/python-dev2-garchiv... From robert.kern at gmail.com Wed Jan 23 20:32:50 2008 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 23 Jan 2008 13:32:50 -0600 Subject: [Python-Dev] Error in PEP3118? In-Reply-To: References: <47905DAE.9020802@ctypes.org> Message-ID: Thomas Heller wrote: > Here is another typo (?) in the pep; I think it should be changed: > > Index: pep-3118.txt > =================================================================== > --- pep-3118.txt (revision 60037) > +++ pep-3118.txt (working copy) > @@ -338,7 +338,7 @@ > > ``len`` > the total bytes of memory the object uses. This should be the > - same as the product of the shape array multiplied by the number of > + same as the length of the shape array multiplied by the number of > bytes per item of memory. > > ``readonly`` While the original could be reworded ("product of the elements of the shape array"), the amendment is incorrect. For a shape array like {4*5*6}, the number of bytes is (4*5*6)*bytes_per_item, not 3*bytes_per_item. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From lists at cheimes.de Wed Jan 23 20:32:54 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 23 Jan 2008 20:32:54 +0100 Subject: [Python-Dev] math and numerical fixes (was: When is min(a, b) != min(b, a)?) In-Reply-To: References: Message-ID: <47979666.2020900@cheimes.de> Jason wrote: > Please note that NaN's are very funky and platform dependent. They > depend on their underlying platform's C library for creation and > display. On windows, "float('nan')" will cause an exception, as there > are no valid string representations of NAN that can be converted to > the special floating point value. Also, if you manage to create a nan > under Windows, it displays as "1.#QNAN". > > Infinite values are also problematic. In almost all cases, it is far > better to avoid infinite and NaN values. CC to Python Dev I've fixed that and enhanced the support for NaN and inf for 2.6 and 3.0. I'm working together with Mark on more NaN and inf related fixes and he has fixed some numerical issues in the cmath module. We both hope to get Python's math more sound and stable across platforms. So far I got float('nan'), float('inf') and float('-inf') working on all platforms. The math module got three new methods: isinf, isnan, copysign. Additionally the trunk-math branch contains code for inverse hyberbolic functions (acosh), log1p, Mark's fixes for complex math and more stuff. For example operations on NaNs now return a NaN on all platforms (except 1**NAN which is defined as 1 and 0**NAN which is defined as 0). In 2.5 it depends on the platform whether a function raises an exception or returns NaN. Mark had the nice idea to introduce a thread local or global flag for NaN support. Depending on a flag Python turns a NaN into an exception. The feature needs a proper PEP. Maybe Mark has time to write a PEP in time. Christian From steve at holdenweb.com Wed Jan 23 21:25:46 2008 From: steve at holdenweb.com (Steve Holden) Date: Wed, 23 Jan 2008 15:25:46 -0500 Subject: [Python-Dev] 2.5.2 release coming up In-Reply-To: References: Message-ID: <4797A2CA.6060403@holdenweb.com> Giampaolo Rodola' wrote: >> Also, *nothing* should go into the 2.4 branch any more *except* >> important security patches. ^^^^^^^^^ > > http://bugs.python.org/issue1745035 > I guess this one should concern both 2.4 and 2.5 branches. > Egregious though the error may be I can't myself see that a complete new release is justified simply to include a four-line patch in a single (not often-used?) module. If it were a buffer overflow it might be different (but that would pretty much have to involve a C component). Couldn't we just publicize the patch? I can't bring myself to believe that 1745035 is really "important" enough. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Wed Jan 23 21:25:46 2008 From: steve at holdenweb.com (Steve Holden) Date: Wed, 23 Jan 2008 15:25:46 -0500 Subject: [Python-Dev] 2.5.2 release coming up In-Reply-To: References: Message-ID: <4797A2CA.6060403@holdenweb.com> Giampaolo Rodola' wrote: >> Also, *nothing* should go into the 2.4 branch any more *except* >> important security patches. ^^^^^^^^^ > > http://bugs.python.org/issue1745035 > I guess this one should concern both 2.4 and 2.5 branches. > Egregious though the error may be I can't myself see that a complete new release is justified simply to include a four-line patch in a single (not often-used?) module. If it were a buffer overflow it might be different (but that would pretty much have to involve a C component). Couldn't we just publicize the patch? I can't bring myself to believe that 1745035 is really "important" enough. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From greg at krypto.org Wed Jan 23 21:37:37 2008 From: greg at krypto.org (Gregory P. Smith) Date: Wed, 23 Jan 2008 12:37:37 -0800 Subject: [Python-Dev] struct module docs vs reality Message-ID: <52dc1c820801231237y7ab70cc4y85a2338fc77498bd@mail.gmail.com> The documentation for the struct module says: http://docs.python.org/dev/library/struct.html#module-struct "short is 2 bytes; int and long are 4 bytes; long long (__int64 on Windows) is 8 bytes" and lists 'l' and 'L' as the pack code for a C long. As its implemented today, the documentation is incorrect. On an LP64 host (pretty much any 64-bit linux, bsd or unixish thing) a long is 8 bytes. I assume this means there is existing code out there that expects the current not-as-documented behavior. There is also code out there that expects the documented behavior but behaves wrong when a 64bit Python is used. I assume I should just fix the documentation and anything in Lib that uses the struct module incorrectly (zipfile for example) rather than change the behavior? This is for http://bugs.python.org/issue1789 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080123/e7f95a36/attachment.htm From guido at python.org Wed Jan 23 21:40:34 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 23 Jan 2008 12:40:34 -0800 Subject: [Python-Dev] 2.5.2 release coming up In-Reply-To: <4797A2CA.6060403@holdenweb.com> References: <4797A2CA.6060403@holdenweb.com> Message-ID: On Jan 23, 2008 12:25 PM, Steve Holden wrote: > Giampaolo Rodola' wrote: > >> Also, *nothing* should go into the 2.4 branch any more *except* > >> important security patches. > ^^^^^^^^^ > > > > http://bugs.python.org/issue1745035 > > I guess this one should concern both 2.4 and 2.5 branches. > > > > Egregious though the error may be I can't myself see that a complete new > release is justified simply to include a four-line patch in a single > (not often-used?) module. If it were a buffer overflow it might be > different (but that would pretty much have to involve a C component). > > Couldn't we just publicize the patch? I can't bring myself to believe > that 1745035 is really "important" enough. It should go into 2.5 for sure. It should go into 2.4 at the discretion of the release manager. We *are* considering a pure-security-fixes source-only release of 2.4 (I wasn't 100% clear on that in my first mail in this thread). IMO DoS vulnerabilities are rarely worth getting excited about, unless they have the potential of bringing down a significant portion of the internet. This one doesn't. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From theller at ctypes.org Wed Jan 23 22:00:07 2008 From: theller at ctypes.org (Thomas Heller) Date: Wed, 23 Jan 2008 22:00:07 +0100 Subject: [Python-Dev] struct module docs vs reality In-Reply-To: <52dc1c820801231237y7ab70cc4y85a2338fc77498bd@mail.gmail.com> References: <52dc1c820801231237y7ab70cc4y85a2338fc77498bd@mail.gmail.com> Message-ID: Gregory P. Smith schrieb: > The documentation for the struct module says: > > http://docs.python.org/dev/library/struct.html#module-struct > > "short is 2 bytes; int and long are 4 bytes; long long (__int64 on Windows) > is 8 bytes" > > and lists 'l' and 'L' as the pack code for a C long. > > As its implemented today, the documentation is incorrect. On an LP64 host > (pretty much any 64-bit linux, bsd or unixish thing) a long is 8 bytes. > > I assume this means there is existing code out there that expects the > current not-as-documented behavior. There is also code out there that > expects the documented behavior but behaves wrong when a 64bit Python is > used. > > I assume I should just fix the documentation and anything in Lib that uses > the struct module incorrectly (zipfile for example) rather than change the > behavior? +1 (actually +100) from me. Thomas From martin at v.loewis.de Wed Jan 23 22:12:31 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 23 Jan 2008 22:12:31 +0100 Subject: [Python-Dev] why is mmap a builtin module on windows? In-Reply-To: References: <932f8baf0801222353v2593e6edq556a6fe5a3e25e60@mail.gmail.com> Message-ID: <4797ADBF.5050201@v.loewis.de> > On Windows lots of modules are linked into the python main dll. The file > PC/config.c contains a list of all modules. From the point of the > maintainer it's much easier to link the modules into the main dll > instead of creating standalone dlls. I also suspect that it's much > faster because relocation is slow (see PC/dllbase_nt.txt). Martin or > Mark can give you a better answer. Actually, that *is* the current answer. That plus a remark "Contributions are welcome, as long as they a) come with a clear, objective policy on what should go into pythonxy.dll and what not, and b) automate all aspects of adding modules that should not go into pythonxy.dll according to the policy." Regards, Martin From steve at holdenweb.com Wed Jan 23 21:49:51 2008 From: steve at holdenweb.com (Steve Holden) Date: Wed, 23 Jan 2008 15:49:51 -0500 Subject: [Python-Dev] 2.5.2 release coming up In-Reply-To: References: <4797A2CA.6060403@holdenweb.com> Message-ID: <4797A86F.5020404@holdenweb.com> Guido van Rossum wrote: > On Jan 23, 2008 12:25 PM, Steve Holden wrote: >> Giampaolo Rodola' wrote: >>>> Also, *nothing* should go into the 2.4 branch any more *except* >>>> important security patches. >> ^^^^^^^^^ >>> http://bugs.python.org/issue1745035 >>> I guess this one should concern both 2.4 and 2.5 branches. >>> >> Egregious though the error may be I can't myself see that a complete new >> release is justified simply to include a four-line patch in a single >> (not often-used?) module. If it were a buffer overflow it might be >> different (but that would pretty much have to involve a C component). >> >> Couldn't we just publicize the patch? I can't bring myself to believe >> that 1745035 is really "important" enough. > > It should go into 2.5 for sure. It should go into 2.4 at the > discretion of the release manager. We *are* considering a > pure-security-fixes source-only release of 2.4 (I wasn't 100% clear on > that in my first mail in this thread). > > IMO DoS vulnerabilities are rarely worth getting excited about, unless > they have the potential of bringing down a significant portion of the > internet. This one doesn't. > Yes. There has to be a 2.5.2 release and there's no reason to exclude it from that. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From greg at krypto.org Wed Jan 23 22:53:53 2008 From: greg at krypto.org (Gregory P. Smith) Date: Wed, 23 Jan 2008 13:53:53 -0800 Subject: [Python-Dev] struct module docs vs reality In-Reply-To: References: <52dc1c820801231237y7ab70cc4y85a2338fc77498bd@mail.gmail.com> Message-ID: <52dc1c820801231353i782cf47ftf63dc15a7fb4b123@mail.gmail.com> On 1/23/08, Thomas Heller wrote: > > Gregory P. Smith schrieb: > > The documentation for the struct module says: > > > > http://docs.python.org/dev/library/struct.html#module-struct > > > > "short is 2 bytes; int and long are 4 bytes; long long (__int64 on > Windows) > > is 8 bytes" > > > > and lists 'l' and 'L' as the pack code for a C long. > > > > As its implemented today, the documentation is incorrect. On an LP64 > host > > (pretty much any 64-bit linux, bsd or unixish thing) a long is 8 bytes. > > > > I assume this means there is existing code out there that expects the > > current not-as-documented behavior. There is also code out there that > > expects the documented behavior but behaves wrong when a 64bit Python is > > used. > > > > I assume I should just fix the documentation and anything in Lib that > uses > > the struct module incorrectly (zipfile for example) rather than change > the > > behavior? > > +1 (actually +100) from me. > > Thomas Ok, its a pretty big diff. Much of the standard library is completely broken when used on LP64 platforms. I added it to the bug. I'm running unit tests now. These fixes should go into 2.5.2. The diff could use some eyes on it, especially the Mac API stuff that I know nothing about but am assuming relies on 32bit values. http://bugs.python.org/issue1789 Greg -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080123/d52b5990/attachment-0001.htm From aahz at pythoncraft.com Thu Jan 24 01:49:39 2008 From: aahz at pythoncraft.com (Aahz) Date: Wed, 23 Jan 2008 16:49:39 -0800 Subject: [Python-Dev] REMINDER: OSCON 2008 Call for Proposals Message-ID: <20080124004939.GA6387@panix.com> The O'Reilly Open Source Convention (OSCON) is accepting proposals for tutorials and presentations. The submission period ends Feb 4. OSCON 2008 will be in Portland, Oregon July 21-25. For more information and to submit a proposal, see http://conferences.oreilly.com/oscon/ -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "All problems in computer science can be solved by another level of indirection." --Butler Lampson From ocean at m2.ccsnet.ne.jp Thu Jan 24 04:28:26 2008 From: ocean at m2.ccsnet.ne.jp (ocean) Date: Thu, 24 Jan 2008 12:28:26 +0900 Subject: [Python-Dev] 2.5.2 release coming up References: Message-ID: <000701c85e39$2e6dbe30$0300a8c0@whiterabc2znlh> Is threre any chance to fix this bug before releasing 2.5.2? http://bugs.python.org/issue1736 It contains potential buffer overrun, I think this is somewhat important. If multibyte support (CharNext) is not needed, I 'll rewrite the patch gracefully. From guido at python.org Thu Jan 24 05:09:39 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 23 Jan 2008 20:09:39 -0800 Subject: [Python-Dev] 2.5.2 release coming up In-Reply-To: <000701c85e39$2e6dbe30$0300a8c0@whiterabc2znlh> References: <000701c85e39$2e6dbe30$0300a8c0@whiterabc2znlh> Message-ID: On Jan 23, 2008 7:28 PM, ocean wrote: > Is threre any chance to fix this bug before releasing 2.5.2? > http://bugs.python.org/issue1736 > It contains potential buffer overrun, I think this is somewhat important. > If multibyte support (CharNext) is not needed, I 'll rewrite the patch > gracefully. I'll leave that to MvL to decide; given that AFAIK msilib is only used to build the Python installer I'm not sure it's worth defending against malicious code -- it would be easier to simply remove it from an installation if you have reason to believe you might be executing malicious Python code. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Thu Jan 24 05:40:32 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 24 Jan 2008 05:40:32 +0100 Subject: [Python-Dev] 2.5.2 release coming up In-Reply-To: References: <000701c85e39$2e6dbe30$0300a8c0@whiterabc2znlh> Message-ID: <479816C0.1020609@v.loewis.de> >> Is threre any chance to fix this bug before releasing 2.5.2? >> http://bugs.python.org/issue1736 >> It contains potential buffer overrun, I think this is somewhat important. >> If multibyte support (CharNext) is not needed, I 'll rewrite the patch >> gracefully. > > I'll leave that to MvL to decide; given that AFAIK msilib is only used > to build the Python installer I'm not sure it's worth defending > against malicious code -- it would be easier to simply remove it from > an installation if you have reason to believe you might be executing > malicious Python code. > I'll look into it. msilib is used in distutils (for bdist_msi), so it should get fixed. Regards, Martin From nnorwitz at gmail.com Thu Jan 24 09:00:33 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 24 Jan 2008 00:00:33 -0800 Subject: [Python-Dev] 2.5.2 release coming up In-Reply-To: <479816C0.1020609@v.loewis.de> References: <000701c85e39$2e6dbe30$0300a8c0@whiterabc2znlh> <479816C0.1020609@v.loewis.de> Message-ID: We need to fix the Windows buildbots. 2 tests are currently failing for 2.5.2: test_mailbox test_winreg From: http://www.python.org/dev/buildbot/all/x86%20XP-4%202.5/builds/107/step-test/0 The errors are: File "E:\cygwin\home\db3l\buildarea\2.5.bolen-windows\build\lib\test\test_mailbox.py", line 522, in test_consistent_factory msg2 = box.get_message(key) File "E:\cygwin\home\db3l\buildarea\2.5.bolen-windows\build\lib\mailbox.py", line 315, in get_message subpath = self._lookup(key) File "E:\cygwin\home\db3l\buildarea\2.5.bolen-windows\build\lib\mailbox.py", line 484, in _lookup raise KeyError('No message with key: %s' % key) KeyError: 'No message with key: 1201127998.M194999P232Q203.buildbot' and: test test_winreg failed -- Not the correct number of sub keys There are several warnings from the tests that we should try to clean up: lib\test\test_generators.py:1: DeprecationWarning: raising string exceptions is deprecated tutorial_tests = """ lib\zipfile.py:714: DeprecationWarning: struct integer overflow masking is deprecated 0, 0, count, count, pos2 - pos1, -1, 0) lib\zipfile.py:691: DeprecationWarning: struct integer overflow masking is deprecated header_offset) lib\test\test_unicode_file.py:103: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal filename1==filename2 lib\shutil.py:36: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal os.path.normcase(os.path.abspath(dst))) n From ajm at flonidan.dk Thu Jan 24 09:10:31 2008 From: ajm at flonidan.dk (Anders J. Munch) Date: Thu, 24 Jan 2008 09:10:31 +0100 Subject: [Python-Dev] struct module docs vs reality In-Reply-To: <52dc1c820801231237y7ab70cc4y85a2338fc77498bd@mail.gmail.com> Message-ID: <9B1795C95533CA46A83BA1EAD4B01030031FE5@flonidanmail.flonidan.net> Gregory P. Smith wrote: > > The documentation for the struct module says: > > http://docs.python.org/dev/library/struct.html#module-struct > > " short is 2 bytes; int and long are 4 bytes; long long ( __int64 on Windows) is 8 bytes" > > and lists 'l' and 'L' as the pack code for a C long. > > As its implemented today, the documentation is incorrect. On an LP64 > host (pretty much any 64-bit linux, bsd or unixish thing) a long is 8 > bytes. You overlooked the words "Standard size and alignment are as follows" that start the quoted paragraph. It's a little confusing because standard size is not the default. The default is platform-specific sizes. Only if you start the format string with >, <, ! or = do you get standard sizes. The reference documentation is correct as it stands, and, I suspect, so is the LP64 implementation. Doesn't struct.pack('>l',42) produce a 4-byte string on LP64? The tutorial at http://docs.python.org/tut/node13.html#SECTION0013300000000000000000%3E has a bug though: the format string should use '<'. I believe zipfile.py correctly uses '<' throughout. regards, Anders From greg at krypto.org Thu Jan 24 09:39:30 2008 From: greg at krypto.org (Gregory P. Smith) Date: Thu, 24 Jan 2008 00:39:30 -0800 Subject: [Python-Dev] struct module docs vs reality In-Reply-To: <9B1795C95533CA46A83BA1EAD4B01030031FE5@flonidanmail.flonidan.net> References: <52dc1c820801231237y7ab70cc4y85a2338fc77498bd@mail.gmail.com> <9B1795C95533CA46A83BA1EAD4B01030031FE5@flonidanmail.flonidan.net> Message-ID: <52dc1c820801240039sc6411em1e166245949a536e@mail.gmail.com> Oh good. Reading the Modules/_struct.c code I see that is indeed what happens. There are still several instances of misused struct pack and unpack strings in Lib but the problem is less serious, I'll make a new patch that just addresses those. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080124/2399fd18/attachment.htm From theller at ctypes.org Thu Jan 24 11:44:01 2008 From: theller at ctypes.org (Thomas Heller) Date: Thu, 24 Jan 2008 11:44:01 +0100 Subject: [Python-Dev] 2.5.2 release coming up In-Reply-To: References: <000701c85e39$2e6dbe30$0300a8c0@whiterabc2znlh> <479816C0.1020609@v.loewis.de> Message-ID: Neal Norwitz schrieb: > We need to fix the Windows buildbots. 2 tests are currently failing > for 2.5.2: test_mailbox test_winreg The test_winreg failure is a funny one: The py3k test_winreg fails because of a bug in the test itself; I fixed this in rev. 60236. The failing test leaves a key in the registry that make the trunk and release25-maint tests also fail. I have manually removed the offending key in the registry on the x86 XP-3 buildbot; we'll see if it works now. The real solution, however, would be to change all the test_winreg tests to remove the test-key completely before and after their test. But I'll leave this to someone else. Thomas From ocean at m2.ccsnet.ne.jp Thu Jan 24 12:07:54 2008 From: ocean at m2.ccsnet.ne.jp (ocean) Date: Thu, 24 Jan 2008 20:07:54 +0900 Subject: [Python-Dev] 2.5.2 release coming up References: <000701c85e39$2e6dbe30$0300a8c0@whiterabc2znlh><479816C0.1020609@v.loewis.de> Message-ID: <001f01c85e79$5e8557f0$0300a8c0@whiterabc2znlh> > From: > http://www.python.org/dev/buildbot/all/x86%20XP-4%202.5/builds/107/step-test/0 > > The errors are: > > File "E:\cygwin\home\db3l\buildarea\2.5.bolen-windows\build\lib\test\test_mailbox .py", > line 522, in test_consistent_factory > msg2 = box.get_message(key) > File "E:\cygwin\home\db3l\buildarea\2.5.bolen-windows\build\lib\mailbox.py", > line 315, in get_message > subpath = self._lookup(key) > File "E:\cygwin\home\db3l\buildarea\2.5.bolen-windows\build\lib\mailbox.py", > line 484, in _lookup > raise KeyError('No message with key: %s' % key) > KeyError: 'No message with key: 1201127998.M194999P232Q203.buildbot' I did quick investigation on this error. After self._refresh() (line 480 in _loopkup - Lib/mailbox.py) runs, self._toc contains key like this. 1201171711.M848000P1176Q16.whiterab-c2znlh!2,FR Please look at exclamation mark. Probably this is not intended on most platforms like Unix. It should be ":" colon. But on windows, ":" is special letter after drive letter. (ex: "C:/Winnt/foo/bar") So I imagine open() or something converts ":" to "!" (valid character as file name). After I applied following experimental patch, test_mailbox.py run successfully on windows. Index: Lib/mailbox.py =================================================================== --- Lib/mailbox.py (revision 60233) +++ Lib/mailbox.py (working copy) @@ -223,7 +223,8 @@ class Maildir(Mailbox): """A qmail-style Maildir mailbox.""" - colon = ':' +# colon = ':' + colon = "!" def __init__(self, dirname, factory=rfc822.Message, create=True): """Initialize a Maildir instance.""" From ocean at m2.ccsnet.ne.jp Thu Jan 24 12:49:44 2008 From: ocean at m2.ccsnet.ne.jp (ocean) Date: Thu, 24 Jan 2008 20:49:44 +0900 Subject: [Python-Dev] 2.5.2 release coming up References: <000701c85e39$2e6dbe30$0300a8c0@whiterabc2znlh><479816C0.1020609@v.loewis.de> <001f01c85e79$5e8557f0$0300a8c0@whiterabc2znlh> Message-ID: <003301c85e7f$36d16810$0300a8c0@whiterabc2znlh> > But on windows, ":" is special letter after drive letter. (ex: > "C:/Winnt/foo/bar") > So I imagine open() or something converts ":" to "!" (valid character as > file name). Sorry, I lied. :-( open() didn't change ":", test_mailbox.py (TestMaildir.setUp) changed self._box.colon to "!" Otherwise, newly created mail box in test_consistent_factory 's "colon" is ":". This mismatch causes error. Following patch solves error, but I don't know if this is good solution. Index: Lib/test/test_mailbox.py =================================================================== --- Lib/test/test_mailbox.py (revision 60233) +++ Lib/test/test_mailbox.py (working copy) @@ -458,12 +458,11 @@ class TestMaildir(TestMailbox): - _factory = lambda self, path, factory=None: mailbox.Maildir(path, factory) - - def setUp(self): - TestMailbox.setUp(self) + def _factory(self, path, factory=None): + box = mailbox.Maildir(path, factory) if os.name in ('nt', 'os2') or sys.platform == 'cygwin': - self._box.colon = '!' + box.colon = '!' + return box def test_add_MM(self): # Add a MaildirMessage instance @@ -518,7 +517,7 @@ # Create new mailbox with class FakeMessage(mailbox.MaildirMessage): pass - box = mailbox.Maildir(self._path, factory=FakeMessage) + box = self._factory(self._path, factory=FakeMessage) msg2 = box.get_message(key) self.assert_(isinstance(msg2, FakeMessage)) From mike.kent at sage.com Thu Jan 24 16:33:47 2008 From: mike.kent at sage.com (Mike Kent) Date: Thu, 24 Jan 2008 15:33:47 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?Incorrect_documentation_of_the_raw=5Finput?= =?utf-8?q?_built-in_function?= Message-ID: Recently I was trying to debug an old python program who's maintenance I inherited. I was using the quick-and-dirty method of putting some 'print >>sys.stderr' statements in the code, and then running the command with '2>filename' appended to the end of the command line. Imagine my surprise to see that all of the prompt text from the program's raw_input calls were also disappearing from the screen output, and appearing in the stderr output routed to the file. The latest documentation for raw_input states "If the prompt argument is present, it is written to standard output without a trailing newline." I posted a question regarding the observed behavior to comp.lang.python and Gabriel Genellina (thanks Gabriel!) pointed out that despite the documentation, raw_input was hard-coded to always output its prompt text to stderr. This raises two questions: 1. Shouldn't the current documentation be corrected to state that raw_input writes its prompt to standard error? 2. Is this really the hard-coded behavior we want? I don't think my use-case is that odd; in fact, what I find very odd is that the prompt output is send to stderr. I mean, I'm printing the prompt for a question, not some error message. Can there not at least be an optional parameter to indicate that you want the output sent to stdout rather than stderr? From stephenemslie at gmail.com Thu Jan 24 16:48:05 2008 From: stephenemslie at gmail.com (stephen emslie) Date: Thu, 24 Jan 2008 15:48:05 +0000 Subject: [Python-Dev] rfc822_escape doing the right thing? In-Reply-To: <52dc1c820801231000h7503494fi397d940a0f44a44d@mail.gmail.com> References: <51f97e530801230838t472a0d1fv24487c94ee42ca4f@mail.gmail.com> <52dc1c820801231000h7503494fi397d940a0f44a44d@mail.gmail.com> Message-ID: <51f97e530801240748g7a32f524x971c54cdc0c5b15@mail.gmail.com> I have created issue #1923 to keep track of this. Stephen Emslie On Jan 23, 2008 6:00 PM, Gregory P. Smith wrote: > could you put this on bugs.python.org and follow up with a reference to the > issue # for better tracking? > > > > On 1/23/08, stephen emslie wrote: > > > > > > > > I've been working on a project that renders PKG-INFO metadata in a > > number of ways. I have noticed that fields with any indentation were > > flattened out, which is being done in distutils.util.rfc822_escape. > > This unfortunately means that you cant use reStructuredText formatting > > in your long description (suggested in PEP345), or are limited to a > > set that doesn't require indentation (no block quotes, etc.). > > > > It looks like this behavior was intentionally added in rev 20099, but > > that was about 7 years ago - before reStructuredText and eggs. I > > wonder if it makes sense to re-think that implementation with this > > sort of metadata in mind, assuming this behavior isn't required to be > > rfc822 compliant. I think it would certainly be a shame to miss out on > > a good thing like proper (renderable) reST in our metadata. > > > > A quick example of what I mean: > > > > >>> rest = """ > > ... a literal python block:: > > ... >>> import this > > ... """ > > >>> print distutils.util.rfc822_escape(rest) > > > > a literal python block:: > > >>> import this > > > > should look something like: > > > > a literal python block:: > > >>> import this > > > > > > Is distutils being over-cautious in flattening out all whitespace? A > > w3c discussion on multiple lines in rfc822 [1] seems to suggest that > > whitespace can be 'unfolded' safely, so it seems a shame to be > > throwing it away when it can have important meaning. > > > > [1] http://www.w3.org/Protocols/rfc822/3_Lexical.html > > > > Thanks for any comments > > > > Stephen Emslie > > _______________________________________________ > > Python-Dev mailing list > > Python-Dev at python.org > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/greg%40krypto.org > > > > From skip at pobox.com Thu Jan 24 17:51:15 2008 From: skip at pobox.com (skip at pobox.com) Date: Thu, 24 Jan 2008 10:51:15 -0600 Subject: [Python-Dev] =?utf-8?q?Incorrect_documentation_of_the_raw=5Finput?= =?utf-8?q?_built-in_function?= In-Reply-To: References: Message-ID: <18328.49667.162531.446063@montanaro.dyndns.org> Mike> 2. Is this really the hard-coded behavior we want? I don't think Mike> my use-case is that odd; in fact, what I find very odd is that Mike> the prompt output is send to stderr. I mean, I'm printing the Mike> prompt for a question, not some error message. Can there not at Mike> least be an optional parameter to indicate that you want the Mike> output sent to stdout rather than stderr? I can think of situations where you don't want the output to go to stdout either (suppose it's the regular output of the file you want to save to a file). Having a choice seems the best route. From ijmorlan at cs.uwaterloo.ca Thu Jan 24 18:36:51 2008 From: ijmorlan at cs.uwaterloo.ca (Isaac Morland) Date: Thu, 24 Jan 2008 12:36:51 -0500 (EST) Subject: [Python-Dev] =?utf-8?q?Incorrect_documentation_of_the_raw=5Finput?= =?utf-8?q?_built-in_function?= In-Reply-To: <18328.49667.162531.446063@montanaro.dyndns.org> References: <18328.49667.162531.446063@montanaro.dyndns.org> Message-ID: On Thu, 24 Jan 2008, skip at pobox.com wrote: > Mike> 2. Is this really the hard-coded behavior we want? I don't think > Mike> my use-case is that odd; in fact, what I find very odd is that > Mike> the prompt output is send to stderr. I mean, I'm printing the > Mike> prompt for a question, not some error message. Can there not at > Mike> least be an optional parameter to indicate that you want the > Mike> output sent to stdout rather than stderr? > > I can think of situations where you don't want the output to go to stdout > either (suppose it's the regular output of the file you want to save to a > file). Having a choice seems the best route. What about an option (maybe even a default) to send the prompt to stdin? The Postgres command line interface psql appears to do this: $ psql 2>&1 >/dev/null Password: $ (I typed my password and then I quit by typing ^D; if I type the wrong password, it looks the same on screen but it quits right away without waiting for ^D) I think ssh also does this when it needs to prompt for a password. Really the prompt is part of the input, not part of the output, in a certain sense. Have people actually verified that the prompt is really sent to stderr right now by using 2>/dev/null to attempt to suppress it? Isaac Morland CSCF Web Guru DC 2554C, x36650 WWW Software Specialist From phd at phd.pp.ru Thu Jan 24 18:44:26 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Thu, 24 Jan 2008 20:44:26 +0300 Subject: [Python-Dev] Incorrect documentation of the raw_input built-in function In-Reply-To: References: <18328.49667.162531.446063@montanaro.dyndns.org> Message-ID: <20080124174426.GA18735@phd.pp.ru> On Thu, Jan 24, 2008 at 12:36:51PM -0500, Isaac Morland wrote: > What about an option (maybe even a default) to send the prompt to stdin? > > The Postgres command line interface psql appears to do this: > > $ psql 2>&1 >/dev/null > Password: > $ > > (I typed my password and then I quit by typing ^D; if I type the wrong > password, it looks the same on screen but it quits right away without > waiting for ^D) > > I think ssh also does this when it needs to prompt for a password. One cannot write to stdin: >>> sys.stdin.write("1\n") Traceback (most recent call last): File "", line 1, in IOError: [Errno 9] Bad file descriptor But it is possible to explicitly open the current console for reading and writing and do I/O regardless of stdin/stdout/stderr redirects: >>> tty = open("/dev/tty", "r+") >>> tty.write("1\n") 1 >>> line = tty.readline() DDD >>> line 'DDD\n' Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From skip at pobox.com Thu Jan 24 18:47:53 2008 From: skip at pobox.com (skip at pobox.com) Date: Thu, 24 Jan 2008 11:47:53 -0600 Subject: [Python-Dev] =?utf-8?q?Incorrect_documentation_of_the_raw=5Finput?= =?utf-8?q?_built-in_function?= In-Reply-To: References: <18328.49667.162531.446063@montanaro.dyndns.org> Message-ID: <18328.53065.28351.914802@montanaro.dyndns.org> Isaac> Have people actually verified that the prompt is really sent to Isaac> stderr right now by using 2>/dev/null to attempt to suppress it? Good point. On my machine at work (Solaris), Python 2.4 seems to send its raw_input prompt to stdout, not stderr: % python Python 2.4.2 (#1, Feb 23 2006, 12:48:31) [GCC 3.4.1] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>> raw_input("?") ?z 'z' >>> ink% python 2>/dev/null >>> raw_input("?") ?sdf 'sdf' >>> ink% python >/dev/null Python 2.4.2 (#1, Feb 23 2006, 12:48:31) [GCC 3.4.1] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>> raw_input("?") wer >>> ^D Same for 2.6a0 on my Mac at home. Mike, are you sure about it prompting to stderr? If so, what's your setup? Skip From python at rcn.com Thu Jan 24 19:36:14 2008 From: python at rcn.com (Raymond Hettinger) Date: Thu, 24 Jan 2008 13:36:14 -0500 (EST) Subject: [Python-Dev] trunc() Message-ID: <20080124133614.AFQ48572@ms19.lnh.mail.rcn.net> Can anyone explain to me why we need both trunc() and int()? We used to be very resistant to adding new built-ins and magic method protocols. In days not long past, this would have encountered fierce opposition. ISTM that numbers.py has taken on a life of its own and is causing non-essential offspring to sprout-up everywhere. Also, it seems that ABC is starting to evolve from an optional tool into something embedded in the language in a way that you can't escape it. I would prefer that it not be on the list of concepts that a beginner *must* know in order to be functional in the language. There are a handful of needs met by the numeric tower but they only warrant minimal changes to the language. Raymond P.S. The docstring for trunc() makes it sound like an imprecisely specified version of round(). ---- trunc() looks like int() but it isn't --------------------------- >>> v = [-4.9, -4.5, -4.1, -4.0, -3.5, -0.0, 0.0, 3.5, 4.0, 4.1, 4.5, 4.9] >>> map(int, v) [-4, -4, -4, -4, -3, 0, 0, 3, 4, 4, 4, 4] >>> map(trunc, v) [-4, -4, -4, -4, -3, 0, 0, 3, 4, 4, 4, 4] >>> trunc is int False >>> print trunc.__doc__ trunc(Real) -> Integral returns the integral closest to x between 0 and x. From guido at python.org Thu Jan 24 19:46:09 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 24 Jan 2008 10:46:09 -0800 Subject: [Python-Dev] trunc() In-Reply-To: <20080124133614.AFQ48572@ms19.lnh.mail.rcn.net> References: <20080124133614.AFQ48572@ms19.lnh.mail.rcn.net> Message-ID: On Jan 24, 2008 10:36 AM, Raymond Hettinger wrote: > Can anyone explain to me why we need both trunc() and int()? trunc() has well-defined semantics -- it takes a Real instance and converts it to an Integer instance using round-towards-zero semantics. int() has undefined semantics -- it takes any object and converts it to an int (a concrete type!) using whatever rules it likes -- the definition of __int__ is up to whatever the source type likes to do. For float this has been defined the same as trunc() above, but for other types, who knows! int() of a string does something completely different. Perhaps worse is that sometimes int() is lossy (e.g. with a float input) and sometimes it is not (e.g. with a string input, or with a non-standard representation of integers). There are still some places where a float is accepted incorrectly (silently truncating) due to the use of the __int__ slot. If trunc() had been part of the language from the beginning we wouldn't have needed to introduce __index__. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Jan 24 19:47:46 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 24 Jan 2008 10:47:46 -0800 Subject: [Python-Dev] Incorrect documentation of the raw_input built-in function In-Reply-To: <18328.53065.28351.914802@montanaro.dyndns.org> References: <18328.49667.162531.446063@montanaro.dyndns.org> <18328.53065.28351.914802@montanaro.dyndns.org> Message-ID: Beware: this may depend on whether GNU readline is enabled or not -- under many circumstances, raw_input() calls GNU readline instead of using sys.stdin/stdout. I do agree that if there are any conditions where it uses stderr instead of stdout those are mistakes. On Jan 24, 2008 9:47 AM, wrote: > > Isaac> Have people actually verified that the prompt is really sent to > Isaac> stderr right now by using 2>/dev/null to attempt to suppress it? > > Good point. On my machine at work (Solaris), Python 2.4 seems to send its > raw_input prompt to stdout, not stderr: > > % python > Python 2.4.2 (#1, Feb 23 2006, 12:48:31) > [GCC 3.4.1] on sunos5 > Type "help", "copyright", "credits" or "license" for more information. > >>> raw_input("?") > ?z > 'z' > >>> > ink% python 2>/dev/null > >>> raw_input("?") > ?sdf > 'sdf' > >>> ink% python >/dev/null > Python 2.4.2 (#1, Feb 23 2006, 12:48:31) > [GCC 3.4.1] on sunos5 > Type "help", "copyright", "credits" or "license" for more information. > >>> raw_input("?") > wer > >>> ^D > > Same for 2.6a0 on my Mac at home. Mike, are you sure about it prompting to > stderr? If so, what's your setup? > > Skip > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Thu Jan 24 20:36:18 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 24 Jan 2008 20:36:18 +0100 Subject: [Python-Dev] trunc() In-Reply-To: References: <20080124133614.AFQ48572@ms19.lnh.mail.rcn.net> Message-ID: <4798E8B2.2080602@v.loewis.de> > trunc() has well-defined semantics -- it takes a Real instance and > converts it to an Integer instance using round-towards-zero semantics. No. trunc calls __trunc__, which does whatever it pleases to do. >>> class A: ... def __trunc__(self): ... return 0 ... >>> a=A() >>> trunc(a) 0 > > int() has undefined semantics -- it takes any object and converts it > to an int (a concrete type!) using whatever rules it likes -- the > definition of __int__ is up to whatever the source type likes to do. > For float this has been defined the same as trunc() above, but for > other types, who knows! int() of a string does something completely > different. But why is that a reason to keep trunc()? If you only ever want to convert floats to ints, you can use either one, with no difference. int() does *not* have undefined semantics, for floats, it converts it to an integer using round-towards-zero semantics. Regards, Martin From mike.kent at sage.com Thu Jan 24 20:41:46 2008 From: mike.kent at sage.com (Mike Kent) Date: Thu, 24 Jan 2008 19:41:46 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?Incorrect_documentation_of_the_raw=5Finput?= =?utf-8?q?_built-in_function?= References: <18328.49667.162531.446063@montanaro.dyndns.org> <18328.53065.28351.914802@montanaro.dyndns.org> Message-ID: pobox.com> writes: > > > Isaac> Have people actually verified that the prompt is really sent to > Isaac> stderr right now by using 2>/dev/null to attempt to suppress it? > > Good point. On my machine at work (Solaris), Python 2.4 seems to send its > raw_input prompt to stdout, not stderr: > > > Same for 2.6a0 on my Mac at home. Mike, are you sure about it prompting to > stderr? If so, what's your setup? > > Skip Skip, Guido and others: Interesting point about whether GNU readline is installed. My setup is RedHat Linux, with Python 2.5 that I built and installed myself. GNU readline is not, in fact, installed. If you look at Python2.5/Parser/myreadline.c, function PyOS_StdioReadline, line 125, you will see that prompt output is being sent to stderr. As best as my Python-fu can determine, this is the code used to output a raw_input prompt (thanks again to Gabriel Genellina for pointing me in the right direction.) It's entirely likely that the difference in what I am seeing and what you guys are seeing is caused by my not having GNU readline installed. Nevertheless, the behavior without it seems wrong, and is certainly different from the documentation. From guido at python.org Thu Jan 24 21:08:09 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 24 Jan 2008 12:08:09 -0800 Subject: [Python-Dev] trunc() In-Reply-To: <4798E8B2.2080602@v.loewis.de> References: <20080124133614.AFQ48572@ms19.lnh.mail.rcn.net> <4798E8B2.2080602@v.loewis.de> Message-ID: On Jan 24, 2008 11:36 AM, "Martin v. L?wis" wrote: > > trunc() has well-defined semantics -- it takes a Real instance and > > converts it to an Integer instance using round-towards-zero semantics. > > No. trunc calls __trunc__, which does whatever it pleases to do. > > >>> class A: > ... def __trunc__(self): > ... return 0 > ... > >>> a=A() > >>> trunc(a) > 0 However, PEP 3141 specifies what it should do for Numbers. > > int() has undefined semantics -- it takes any object and converts it > > to an int (a concrete type!) using whatever rules it likes -- the > > definition of __int__ is up to whatever the source type likes to do. > > For float this has been defined the same as trunc() above, but for > > other types, who knows! int() of a string does something completely > > different. > > But why is that a reason to keep trunc()? If you only ever want to > convert floats to ints, you can use either one, with no difference. > int() does *not* have undefined semantics, for floats, it converts > it to an integer using round-towards-zero semantics. Yes (although it wasn't always specified this way). But I don't like that, because of the ambiguity of int(x). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Jan 24 21:09:12 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 24 Jan 2008 12:09:12 -0800 Subject: [Python-Dev] Incorrect documentation of the raw_input built-in function In-Reply-To: References: <18328.49667.162531.446063@montanaro.dyndns.org> <18328.53065.28351.914802@montanaro.dyndns.org> Message-ID: On Jan 24, 2008 11:41 AM, Mike Kent wrote: > pobox.com> writes: > > > > > > > Isaac> Have people actually verified that the prompt is really sent to > > Isaac> stderr right now by using 2>/dev/null to attempt to suppress it? > > > > Good point. On my machine at work (Solaris), Python 2.4 seems to send its > > raw_input prompt to stdout, not stderr: > > > > > > > Same for 2.6a0 on my Mac at home. Mike, are you sure about it prompting to > > stderr? If so, what's your setup? > > > > Skip > > Skip, Guido and others: > > Interesting point about whether GNU readline is installed. My setup is RedHat > Linux, with Python 2.5 that I built and installed myself. GNU readline is not, > in fact, installed. If you look at Python2.5/Parser/myreadline.c, function > PyOS_StdioReadline, line 125, you will see that prompt output is being sent to > stderr. As best as my Python-fu can determine, this is the code used to output > a raw_input prompt (thanks again to Gabriel Genellina for pointing me in the > right direction.) > > It's entirely likely that the difference in what I am seeing and what you guys > are seeing is caused by my not having GNU readline installed. Nevertheless, > the behavior without it seems wrong, and is certainly different from the > documentation. Agreed. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From mike.kent at sage.com Thu Jan 24 21:14:18 2008 From: mike.kent at sage.com (Mike Kent) Date: Thu, 24 Jan 2008 20:14:18 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?Incorrect_documentation_of_the_raw=5Finput?= =?utf-8?q?built-in=09function?= References: <18328.49667.162531.446063@montanaro.dyndns.org> <18328.53065.28351.914802@montanaro.dyndns.org> Message-ID: Guido van Rossum python.org> writes: > > > It's entirely likely that the difference in what I am seeing and what you > > guys > > are seeing is caused by my not having GNU readline installed. Nevertheless, > > the behavior without it seems wrong, and is certainly different from the > > documentation. > > Agreed. > Being that Guido agrees, should I put a bug report into the system for it, and if so, which should it report as the bug: the actual behavior of the raw_input prompt when GNU readline is not installed, or the documentation? From guido at python.org Thu Jan 24 21:17:47 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 24 Jan 2008 12:17:47 -0800 Subject: [Python-Dev] Incorrect documentation of the raw_inputbuilt-in function In-Reply-To: References: <18328.49667.162531.446063@montanaro.dyndns.org> <18328.53065.28351.914802@montanaro.dyndns.org> Message-ID: IMO the actual behavior is incorrect. On Jan 24, 2008 12:14 PM, Mike Kent wrote: > Guido van Rossum python.org> writes: > > > > > > It's entirely likely that the difference in what I am seeing and what you > > > guys > > > are seeing is caused by my not having GNU readline installed. Nevertheless, > > > the behavior without it seems wrong, and is certainly different from the > > > documentation. > > > > Agreed. > > > > Being that Guido agrees, should I put a bug report into the system for it, and > if so, which should it report as the bug: the actual behavior of the raw_input > prompt when GNU readline is not installed, or the documentation? > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From skip at pobox.com Thu Jan 24 21:22:47 2008 From: skip at pobox.com (skip at pobox.com) Date: Thu, 24 Jan 2008 14:22:47 -0600 Subject: [Python-Dev] Incorrect documentation of the raw_input built-in function In-Reply-To: References: <18328.49667.162531.446063@montanaro.dyndns.org> <18328.53065.28351.914802@montanaro.dyndns.org> Message-ID: <18328.62359.814869.183102@montanaro.dyndns.org> >>>>> "Guido" == Guido van Rossum writes: >> It's entirely likely that the difference in what I am seeing and what >> you guys are seeing is caused by my not having GNU readline >> installed. Nevertheless, the behavior without it seems wrong, and is >> certainly different from the documentation. Guido> Agreed. http://bugs.python.org/issue1927 Skip From python at rcn.com Thu Jan 24 21:34:44 2008 From: python at rcn.com (Raymond Hettinger) Date: Thu, 24 Jan 2008 15:34:44 -0500 (EST) Subject: [Python-Dev] trunc() Message-ID: <20080124153444.AFQ72482@ms19.lnh.mail.rcn.net> >> Can anyone explain to me why we need both trunc() and int()? > trunc() has well-defined semantics -- it takes a Real instance > and converts it to an Integer instance using round-towards-zero > semantics. Since something similar is happening to math.ceil and math.floor, I'm curious why trunc() ended-up in builtins instead of the math module. Doesn't it make sense to collect similar functions with similar signatures in the same place? Raymond From eric+python-dev at trueblade.com Thu Jan 24 21:40:39 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Thu, 24 Jan 2008 15:40:39 -0500 Subject: [Python-Dev] Is anyone porting PyNumber_ToBase to trunk? In-Reply-To: References: <47976009.5040107@trueblade.com> Message-ID: <4798F7C7.6010002@trueblade.com> Guido van Rossum wrote: > On Jan 23, 2008 7:40 AM, Eric Smith wrote: >> In 3.0, the code to implement long.__format__() calls PyNumber_ToBase to >> do the heavy lifting. If someone is planning on implementing >> PyNumber_ToBase in 2.6, I'll wait for them to do so. If not, I guess >> I'll either do it myself, or hack around it. >> >> Is this on anyone's To Do list? I don't see it on Brett's spreadsheet >> at http://spreadsheets.google.com/pub?key=pCKY4oaXnT81FrGo3ShGHGg > > I'm sure this is because nobody had thought of this detail until now. > Just get started on it. Feel free to add it to the spreadsheet. It's an implementation detail of PEP 3127 (oct() and bin()), which is on the spreadsheet. I guess I'll have to work on oct() and bin() first. The spreadsheet notes a possible __future__ statement requirement for oct(). The PEP doesn't mention this, and I'm not sure I understand the issue. Any clues? From lists at cheimes.de Thu Jan 24 21:45:14 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 24 Jan 2008 21:45:14 +0100 Subject: [Python-Dev] trunc() In-Reply-To: <20080124153444.AFQ72482@ms19.lnh.mail.rcn.net> References: <20080124153444.AFQ72482@ms19.lnh.mail.rcn.net> Message-ID: Raymond Hettinger wrote: > Since something similar is happening to math.ceil and math.floor, > I'm curious why trunc() ended-up in builtins instead of the math > module. Doesn't it make sense to collect similar functions > with similar signatures in the same place? Traditionally the math module is a tiny wrapper around the system's libm. Functions for magic hooks like __trunc__ usually end up in builtins. In this particular case I don't mind where the function is going to live. Christian From daniel at stutzbachenterprises.com Thu Jan 24 22:09:54 2008 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Thu, 24 Jan 2008 15:09:54 -0600 Subject: [Python-Dev] trunc() In-Reply-To: References: <20080124133614.AFQ48572@ms19.lnh.mail.rcn.net> Message-ID: On Jan 24, 2008 12:46 PM, Guido van Rossum wrote: > trunc() has well-defined semantics -- it takes a Real instance and > converts it to an Integer instance using round-towards-zero semantics. > > int() has undefined semantics -- it takes any object and converts it > to an int (a concrete type!) using whatever rules it likes -- the > definition of __int__ is up to whatever the source type likes to do. What are the use-cases for when trunc() vs int() should be used, and for when a class should define __trunc__ vs __int__? This might help clear up whether both deserve to be a built-in, as well provide a starting point for 3.0 best practices. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From python at rcn.com Thu Jan 24 22:11:54 2008 From: python at rcn.com (Raymond Hettinger) Date: Thu, 24 Jan 2008 16:11:54 -0500 (EST) Subject: [Python-Dev] trunc() Message-ID: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> [Raymond Hettinger] > Since something similar is happening to math.ceil and math.floor, > I'm curious why trunc() ended-up in builtins instead of the math > module. Doesn't it make sense to collect similar functions > with similar signatures in the same place? [Christian Heimes] > Traditionally the math module is a tiny wrapper around the > system's libm. Functions for magic hooks like __trunc__ > usually end up in builtins. In this particular case I don't > mind where the function is going to live. Traditions have gone out the window. ceil() and floor() are going to have their signatures changed (Real --> Integral) and are going to have their own magic methods. They cannot be characterized as a thin wrapper around libm. So the question stands, why is trunc() different? Can anything good come from having trunc() and int() in the same namespace? Raymond From guido at python.org Thu Jan 24 23:11:39 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 24 Jan 2008 14:11:39 -0800 Subject: [Python-Dev] Is anyone porting PyNumber_ToBase to trunk? In-Reply-To: <4798F7C7.6010002@trueblade.com> References: <47976009.5040107@trueblade.com> <4798F7C7.6010002@trueblade.com> Message-ID: On Jan 24, 2008 12:40 PM, Eric Smith wrote: > The spreadsheet notes a possible __future__ statement requirement for > oct(). The PEP doesn't mention this, and I'm not sure I understand the > issue. Any clues? You can't change the default oct() to return '0o123'; but perhaps you could import a different version that returned that instead of the default '0123'. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Fri Jan 25 00:10:52 2008 From: python at rcn.com (Raymond Hettinger) Date: Thu, 24 Jan 2008 18:10:52 -0500 (EST) Subject: [Python-Dev] Float --> Integer Ratio Message-ID: <20080124181052.AFR00184@ms19.lnh.mail.rcn.net> rational.py contains code for turning a float into an exact integer ratio. I've needed something like this in other situations as well. The output is more convenient than the mantissa/exponent pair returned by math.frexp(). I propose C-coding this function and either putting it in the math module or making it a method for floats. That would simplify and speed-up rational.py while making the tool available for other applications. Also, the tool is currently in the wrong place (while the output is clearly useful for rational.py, the internals of the function are entirely about floats and ints and has no internal references to the rational module). Raymond ------------------------------------------------------- def _binary_float_to_ratio(x): """x -> (top, bot), a pair of ints s.t. x = top/bot. The conversion is done exactly, without rounding. bot > 0 guaranteed. Some form of binary fp is assumed. Pass NaNs or infinities at your own risk. >>> _binary_float_to_ratio(10.0) (10, 1) >>> _binary_float_to_ratio(0.0) (0, 1) >>> _binary_float_to_ratio(-.25) (-1, 4) """ From jyasskin at gmail.com Fri Jan 25 00:14:56 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Thu, 24 Jan 2008 15:14:56 -0800 Subject: [Python-Dev] trunc() In-Reply-To: References: <20080124133614.AFQ48572@ms19.lnh.mail.rcn.net> Message-ID: <5d44f72f0801241514y223ae3b2n5c18015288a36346@mail.gmail.com> On Jan 24, 2008 1:09 PM, Daniel Stutzbach wrote: > On Jan 24, 2008 12:46 PM, Guido van Rossum wrote: > > trunc() has well-defined semantics -- it takes a Real instance and > > converts it to an Integer instance using round-towards-zero semantics. > > > > int() has undefined semantics -- it takes any object and converts it > > to an int (a concrete type!) using whatever rules it likes -- the > > definition of __int__ is up to whatever the source type likes to do. > > What are the use-cases for when trunc() vs int() should be used, and > for when a class should define __trunc__ vs __int__? If you intend to convert a real number (usually float, since Decimal has decided not to support it) to an Integral (usually int), use whichever of trunc(), round(), math.floor(), or math.ceil() you intend. In 2.6, that list only includes trunc(). If you absolutely need an int (the concrete, not duck type) from an Integral or you want to parse a string, use int(). Real numbers should define __trunc__. Integrals and, perhaps, some string-like types (maybe an MD5 class?) should define __int__. At least, that's my suggestion. -- Namast?, Jeffrey Yasskin From jyasskin at gmail.com Fri Jan 25 00:24:08 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Thu, 24 Jan 2008 15:24:08 -0800 Subject: [Python-Dev] trunc() In-Reply-To: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> Message-ID: <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> On Jan 24, 2008 1:11 PM, Raymond Hettinger wrote: > [Raymond Hettinger] > > Since something similar is happening to math.ceil and math.floor, > > I'm curious why trunc() ended-up in builtins instead of the math > > module. Doesn't it make sense to collect similar functions > > with similar signatures in the same place? > > [Christian Heimes] > > Traditionally the math module is a tiny wrapper around the > > system's libm. Functions for magic hooks like __trunc__ > > usually end up in builtins. In this particular case I don't > > mind where the function is going to live. > > Traditions have gone out the window. ceil() and floor() > are going to have their signatures changed (Real --> Integral) > and are going to have their own magic methods. They cannot > be characterized as a thin wrapper around libm. > > So the question stands, why is trunc() different? Can anything > good come from having trunc() and int() in the same namespace? One of my goals for trunc() is to replace the from-float use of int(), even though we can't remove it for backward-compatibility reasons. PEP 3141 says: "Because the int() conversion implemented by float (and by decimal.Decimal) is equivalent to but less explicit than trunc(), let's remove it. (Or, if that breaks too much, just add a deprecation warning.)" That needs to be updated and implemented. I think the decision was that removing float.__int__() would break too much, so it needs a deprecation warning in 3.0. int has to be a builtin because it's a fundamental type. trunc() followed round() into the builtins. I have no opinion on whether ceil and floor should move there; it probably depends on how often they're used. -- Namast?, Jeffrey Yasskin From fuzzyman at voidspace.org.uk Fri Jan 25 00:32:07 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 24 Jan 2008 23:32:07 +0000 Subject: [Python-Dev] [python] Re: trunc() In-Reply-To: <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> Message-ID: <47991FF7.5050202@voidspace.org.uk> Jeffrey Yasskin wrote: > On Jan 24, 2008 1:11 PM, Raymond Hettinger wrote: > >> [Raymond Hettinger] >> >>> Since something similar is happening to math.ceil and math.floor, >>> I'm curious why trunc() ended-up in builtins instead of the math >>> module. Doesn't it make sense to collect similar functions >>> with similar signatures in the same place? >>> >> [Christian Heimes] >> >>> Traditionally the math module is a tiny wrapper around the >>> system's libm. Functions for magic hooks like __trunc__ >>> usually end up in builtins. In this particular case I don't >>> mind where the function is going to live. >>> >> Traditions have gone out the window. ceil() and floor() >> are going to have their signatures changed (Real --> Integral) >> and are going to have their own magic methods. They cannot >> be characterized as a thin wrapper around libm. >> >> So the question stands, why is trunc() different? Can anything >> good come from having trunc() and int() in the same namespace? >> > > One of my goals for trunc() is to replace the from-float use of int(), > even though we can't remove it for backward-compatibility reasons. PEP > 3141 says: > > "Because the int() conversion implemented by float (and by > decimal.Decimal) is equivalent to but less explicit than trunc(), > let's remove it. (Or, if that breaks too much, just add a deprecation > warning.)" > > That needs to be updated and implemented. I think the decision was > that removing float.__int__() would break too much, so it needs a > deprecation warning in 3.0. > > int has to be a builtin because it's a fundamental type. trunc() > followed round() into the builtins. I have no opinion on whether ceil > and floor should move there; it probably depends on how often they're > used. > > So you won't be able to construct an int from a float? That sucks (and is unintuitive). Michael From fuzzyman at voidspace.org.uk Fri Jan 25 00:23:06 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 24 Jan 2008 23:23:06 +0000 Subject: [Python-Dev] [python] trunc() In-Reply-To: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> Message-ID: <47991DDA.8090604@voidspace.org.uk> Raymond Hettinger wrote: > [Raymond Hettinger] > >> Since something similar is happening to math.ceil and math.floor, >> I'm curious why trunc() ended-up in builtins instead of the math >> module. Doesn't it make sense to collect similar functions >> with similar signatures in the same place? >> > > [Christian Heimes] > >> Traditionally the math module is a tiny wrapper around the >> system's libm. Functions for magic hooks like __trunc__ >> usually end up in builtins. In this particular case I don't >> mind where the function is going to live. >> > > Traditions have gone out the window. ceil() and floor() > are going to have their signatures changed (Real --> Integral) > and are going to have their own magic methods. They cannot > be characterized as a thin wrapper around libm. > > So the question stands, why is trunc() different? Can anything > good come from having trunc() and int() in the same namespace? > If the ambiguity is that 'int' behaviour is unspecified for floats - is it naive to suggest we specify the behaviour? Michael Foord > > Raymond > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > > From guido at python.org Fri Jan 25 00:44:58 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 24 Jan 2008 15:44:58 -0800 Subject: [Python-Dev] [python] Re: trunc() In-Reply-To: <47991FF7.5050202@voidspace.org.uk> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <47991FF7.5050202@voidspace.org.uk> Message-ID: On Jan 24, 2008 3:32 PM, Michael Foord wrote: > > Jeffrey Yasskin wrote: > > On Jan 24, 2008 1:11 PM, Raymond Hettinger wrote: > > > >> [Raymond Hettinger] > >> > >>> Since something similar is happening to math.ceil and math.floor, > >>> I'm curious why trunc() ended-up in builtins instead of the math > >>> module. Doesn't it make sense to collect similar functions > >>> with similar signatures in the same place? > >>> > >> [Christian Heimes] > >> > >>> Traditionally the math module is a tiny wrapper around the > >>> system's libm. Functions for magic hooks like __trunc__ > >>> usually end up in builtins. In this particular case I don't > >>> mind where the function is going to live. > >>> > >> Traditions have gone out the window. ceil() and floor() > >> are going to have their signatures changed (Real --> Integral) > >> and are going to have their own magic methods. They cannot > >> be characterized as a thin wrapper around libm. > >> > >> So the question stands, why is trunc() different? Can anything > >> good come from having trunc() and int() in the same namespace? > >> > > > > One of my goals for trunc() is to replace the from-float use of int(), > > even though we can't remove it for backward-compatibility reasons. PEP > > 3141 says: > > > > "Because the int() conversion implemented by float (and by > > decimal.Decimal) is equivalent to but less explicit than trunc(), > > let's remove it. (Or, if that breaks too much, just add a deprecation > > warning.)" > > > > That needs to be updated and implemented. I think the decision was > > that removing float.__int__() would break too much, so it needs a > > deprecation warning in 3.0. > > > > int has to be a builtin because it's a fundamental type. trunc() > > followed round() into the builtins. I have no opinion on whether ceil > > and floor should move there; it probably depends on how often they're > > used. > > > > > So you won't be able to construct an int from a float? That sucks (and > is unintuitive). Yes, you can, but you have to specify how you want it done by using trunc() or round() or ceil() or floor(). (In 3.0, round(x) will return an int, not a float.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Fri Jan 25 00:46:43 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 25 Jan 2008 00:46:43 +0100 Subject: [Python-Dev] [python] trunc() In-Reply-To: <47991DDA.8090604@voidspace.org.uk> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <47991DDA.8090604@voidspace.org.uk> Message-ID: <47992363.3010402@v.loewis.de> > If the ambiguity is that 'int' behaviour is unspecified for floats - is > it naive to suggest we specify the behaviour? The concern is that whatever gets specified is arbitrary. There are many ways how an int can be constructed from a float, so why is any such way better than the others, and deserves to be the meaning of int()? Regards, Martin From jyasskin at gmail.com Fri Jan 25 01:09:53 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Thu, 24 Jan 2008 16:09:53 -0800 Subject: [Python-Dev] Float --> Integer Ratio In-Reply-To: <20080124181052.AFR00184@ms19.lnh.mail.rcn.net> References: <20080124181052.AFR00184@ms19.lnh.mail.rcn.net> Message-ID: <5d44f72f0801241609p2c02b786r7b7a9fbe3874a45a@mail.gmail.com> +1. Where do people want it, and what should its name be? I can implement it if you like. Another plausible output would be similar to frexp but with an integer for the mantissa instead of a float. So: A) math.frexp(3.2) == (0.80000000000000004, 2) B) integral_frexp(3.2) == (3602879701896397, -50) C) exact_integral_ratio(3.2) == (3602879701896397, 1125899906842624) (B) is easier to implement in C (not that that's a deciding factor); (C) is probably more useful. So I'd vote for (C), your suggestion, but just wanted to make the choice explicit. Also, to make things explicit, the implementation in rational.py doesn't guarantee that the fraction is in lowest terms. Should the library version? On Jan 24, 2008 3:10 PM, Raymond Hettinger wrote: > rational.py contains code for turning a float into an > exact integer ratio. I've needed something like this in > other situations as well. The output is more convenient > than the mantissa/exponent pair returned by math.frexp(). > > I propose C-coding this function and either putting it in > the math module or making it a method for floats. That > would simplify and speed-up rational.py while making > the tool available for other applications. Also, the > tool is currently in the wrong place (while the output > is clearly useful for rational.py, the internals of > the function are entirely about floats and ints and > has no internal references to the rational module). > > Raymond > > > > ------------------------------------------------------- > > def _binary_float_to_ratio(x): > """x -> (top, bot), a pair of ints s.t. x = top/bot. > > The conversion is done exactly, without rounding. > bot > 0 guaranteed. > Some form of binary fp is assumed. > Pass NaNs or infinities at your own risk. > > >>> _binary_float_to_ratio(10.0) > (10, 1) > >>> _binary_float_to_ratio(0.0) > (0, 1) > >>> _binary_float_to_ratio(-.25) > (-1, 4) > """ > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/jyasskin%40gmail.com > -- Namast?, Jeffrey Yasskin http://jeffrey.yasskin.info/ "Religion is an improper response to the Divine." ? "Skinny Legs and All", by Tom Robbins From schmir at gmail.com Fri Jan 25 10:56:30 2008 From: schmir at gmail.com (Ralf Schmitt) Date: Fri, 25 Jan 2008 10:56:30 +0100 Subject: [Python-Dev] why is mmap a builtin module on windows? In-Reply-To: <4797ADBF.5050201@v.loewis.de> References: <932f8baf0801222353v2593e6edq556a6fe5a3e25e60@mail.gmail.com> <4797ADBF.5050201@v.loewis.de> Message-ID: <932f8baf0801250156l6c978ae5v7aad7f03c70a4685@mail.gmail.com> On Jan 23, 2008 10:12 PM, "Martin v. L?wis" wrote: > > On Windows lots of modules are linked into the python main dll. The file > > PC/config.c contains a list of all modules. From the point of the > > maintainer it's much easier to link the modules into the main dll > > instead of creating standalone dlls. I also suspect that it's much > > faster because relocation is slow (see PC/dllbase_nt.txt). Martin or > > Mark can give you a better answer. > > Actually, that *is* the current answer. That plus a remark > "Contributions are welcome, as long as they > a) come with a clear, objective policy on what should go into > pythonxy.dll and what not, and I'd say anything that is needed by "import sys, os" is a candidate for being included. Currently even the _csv module is a builtin module on windows. But then I don't know how much slower importing from a .pyd file is.. > > b) automate all aspects of adding modules that should not go > into pythonxy.dll according to the policy." > i.e. create visual studio project files for those modules? and make them built automatically? Regards, - Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080125/fef7573e/attachment.htm From lists at cheimes.de Fri Jan 25 11:42:52 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 25 Jan 2008 11:42:52 +0100 Subject: [Python-Dev] why is mmap a builtin module on windows? In-Reply-To: <932f8baf0801250156l6c978ae5v7aad7f03c70a4685@mail.gmail.com> References: <932f8baf0801222353v2593e6edq556a6fe5a3e25e60@mail.gmail.com> <4797ADBF.5050201@v.loewis.de> <932f8baf0801250156l6c978ae5v7aad7f03c70a4685@mail.gmail.com> Message-ID: Ralf Schmitt wrote: > i.e. create visual studio project files for those modules? and make them > built automatically? It's not that simple. You also have to integrate it into the build and carefully arrange the dependencies. You also have to find a free and sensible base address for the dll. At last the new modules must be integrated into the MSI installer script. Christian From facundobatista at gmail.com Fri Jan 25 13:28:36 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Fri, 25 Jan 2008 10:28:36 -0200 Subject: [Python-Dev] trunc() In-Reply-To: <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> Message-ID: 2008/1/24, Guido van Rossum : > > So you won't be able to construct an int from a float? That sucks (and > > is unintuitive). > > Yes, you can, but you have to specify how you want it done by using > trunc() or round() or ceil() or floor(). (In 3.0, round(x) will return > an int, not a float.) 2008/1/24, Jeffrey Yasskin : > That needs to be updated and implemented. I think the decision was > that removing float.__int__() would break too much, so it needs a > deprecation warning in 3.0. What I understand here is as int() is "ambiguous", in the future if you want to specify how you want to convert a float to int. But ceil and floor returns a float. And round and trunc will return an int. So, how I could convert a float to its upper int? Like this?: >>> trunc(math.ceil(.3)) 1 BTW, int is not giving me a deprecation warning: >>> int(.1) 0 Thanks! -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From steve at holdenweb.com Fri Jan 25 13:47:53 2008 From: steve at holdenweb.com (Steve Holden) Date: Fri, 25 Jan 2008 07:47:53 -0500 Subject: [Python-Dev] trunc() In-Reply-To: <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> Message-ID: <4799DA79.3010005@holdenweb.com> Jeffrey Yasskin wrote: > On Jan 24, 2008 1:11 PM, Raymond Hettinger wrote: [...] > > One of my goals for trunc() is to replace the from-float use of int(), > even though we can't remove it for backward-compatibility reasons. PEP > 3141 says: > > "Because the int() conversion implemented by float (and by > decimal.Decimal) is equivalent to but less explicit than trunc(), > let's remove it. (Or, if that breaks too much, just add a deprecation > warning.)" > > That needs to be updated and implemented. I think the decision was > that removing float.__int__() would break too much, so it needs a > deprecation warning in 3.0. > It seems strange to me that a release that has the avowed intention of producing a "more rational" language by breaking backwards compatibility should start out with deprecation warnings of this type. What's next: "This isn't Perl" when someone tries to add a number and a string? Surely the place to raise warnings is in 2to3. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Fri Jan 25 13:47:53 2008 From: steve at holdenweb.com (Steve Holden) Date: Fri, 25 Jan 2008 07:47:53 -0500 Subject: [Python-Dev] trunc() In-Reply-To: <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> Message-ID: <4799DA79.3010005@holdenweb.com> Jeffrey Yasskin wrote: > On Jan 24, 2008 1:11 PM, Raymond Hettinger wrote: [...] > > One of my goals for trunc() is to replace the from-float use of int(), > even though we can't remove it for backward-compatibility reasons. PEP > 3141 says: > > "Because the int() conversion implemented by float (and by > decimal.Decimal) is equivalent to but less explicit than trunc(), > let's remove it. (Or, if that breaks too much, just add a deprecation > warning.)" > > That needs to be updated and implemented. I think the decision was > that removing float.__int__() would break too much, so it needs a > deprecation warning in 3.0. > It seems strange to me that a release that has the avowed intention of producing a "more rational" language by breaking backwards compatibility should start out with deprecation warnings of this type. What's next: "This isn't Perl" when someone tries to add a number and a string? Surely the place to raise warnings is in 2to3. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From p.f.moore at gmail.com Fri Jan 25 14:53:15 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 25 Jan 2008 13:53:15 +0000 Subject: [Python-Dev] trunc() In-Reply-To: <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> Message-ID: <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> On 24/01/2008, Jeffrey Yasskin wrote: > int has to be a builtin because it's a fundamental type. trunc() > followed round() into the builtins. I have no opinion on whether ceil > and floor should move there; it probably depends on how often they're > used. Suggestion: - int() has to stay in builtins for obvious reasons. - put *all* of trunc, ceil, floor, round into math. - make int(float) an error The only fly in the ointment is that 2to3 can't handle the semantic issues around converting int(n) to math.trunc(n) because it can't detect the type of n. So why not make int(float) warn in -3 mode on 2.6, and then let 2to3 do the conversion (on the basis that 2to3 should only be run on code that is -3 warning free)? Did I miss a requirement here? Paul. From facundobatista at gmail.com Fri Jan 25 15:09:13 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Fri, 25 Jan 2008 12:09:13 -0200 Subject: [Python-Dev] trunc() In-Reply-To: <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> Message-ID: 2008/1/25, Paul Moore : > - int() has to stay in builtins for obvious reasons. +1 > - put *all* of trunc, ceil, floor, round into math. +1 > - make int(float) an error -0 (you should be able to convert between builtin datatypes without the use of a module). +1 to keep it and define exactly the behaviour, and point to math module in the documentation if you want better control over the conversion. Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From p.f.moore at gmail.com Fri Jan 25 15:10:45 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 25 Jan 2008 14:10:45 +0000 Subject: [Python-Dev] trunc() In-Reply-To: References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> Message-ID: <79990c6b0801250610o6ae90bbx9e351394671f8c89@mail.gmail.com> On 25/01/2008, Facundo Batista wrote: > > - make int(float) an error > > -0 (you should be able to convert between builtin datatypes without > the use of a module). Good point. > +1 to keep it and define exactly the behaviour, and point to math > module in the documentation if you want better control over the > conversion. Sonds good. Paul From lists at cheimes.de Fri Jan 25 15:11:22 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 25 Jan 2008 15:11:22 +0100 Subject: [Python-Dev] trunc() In-Reply-To: <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> Message-ID: Paul Moore wrote: > On 24/01/2008, Jeffrey Yasskin wrote: >> int has to be a builtin because it's a fundamental type. trunc() >> followed round() into the builtins. I have no opinion on whether ceil >> and floor should move there; it probably depends on how often they're >> used. > > Suggestion: > > - int() has to stay in builtins for obvious reasons. > - put *all* of trunc, ceil, floor, round into math. > - make int(float) an error Slightly different suggestion: - define int(float) as int(trunc(float)) In my humble opinion lots of people expect int(-2.5) == -2 and int(2.5) == 2. Or in other words: int(float) chops of the digits after the dot. As far as I can see float.__int__ already behaves like int(trunc(float)) >>> for n in (2.4, 2.6, -2.4, -2.6): ... print(n, (math.floor(n), math.ceil(n), round(n), trunc(n), int(n))) ... Python 3:0 2.4 ( 2, 3, 2, 2, 2) 2.6 ( 2, 3, 3, 2, 2) -2.4 (-3, -2, -2, -2, -2) -2.6 (-3, -2, -3, -2, -2) Python 2.6: 2.4 ( 2.0, 3.0, 2.0, 2, 2) 2.6 ( 2.0, 3.0, 3.0, 2, 2) -2.4 (-3.0, -2.0, -2.0, -2, -2) -2.6 (-3.0, -2.0, -3.0, -2, -2) Christian From ncoghlan at gmail.com Fri Jan 25 16:35:44 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 26 Jan 2008 01:35:44 +1000 Subject: [Python-Dev] trunc() In-Reply-To: References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> Message-ID: <479A01D0.7090302@gmail.com> Christian Heimes wrote: > Paul Moore wrote: >> Suggestion: >> >> - int() has to stay in builtins for obvious reasons. >> - put *all* of trunc, ceil, floor, round into math. >> - make int(float) an error > > Slightly different suggestion: > - define int(float) as int(trunc(float)) > > In my humble opinion lots of people expect int(-2.5) == -2 and int(2.5) > == 2. Or in other words: int(float) chops of the digits after the dot. FWIW, this approach gets a +1 from me (although I'm -0 on taking round() out of builtins - that seems like a pointless incompatibility to me). Yes, blessing 'trunc' as the default mechanism for converting a non-integral number to an integer is somewhat arbitrary, but it's a piece of arbitrariness with a very long history behind it. For that matter, we could even formally define int() as falling back to trunc() if there is no direct conversion (i.e. int knows about the type natively, or the type provides an __int__ method). That way non-integral types could just implement __trunc__ without having to worry about adding "__int__ = __trunc__" to the class definition. We would still have operator.index to identify types which can be losslessly converted to a builtin integer. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From jquinn at cs.oberlin.edu Fri Jan 25 17:13:12 2008 From: jquinn at cs.oberlin.edu (Jameson "Chema" Quinn) Date: Fri, 25 Jan 2008 10:13:12 -0600 Subject: [Python-Dev] Simple syntax proposal: "not is" Message-ID: I'm writing a source code editor that translates identifiers and keywords on-screen into a different natural language. This tool will do no transformations except at the reversible word level. There is one simple, avoidable case where this results in nonsense in many languages: "is not". I propose allowing "not is" as an acceptable alternative to "is not". Obviously English syntax has a deep influence on python syntax, and I would never propose deeper syntactical changes for natural-language-compatibility. This is a trivial change, one that is still easily parseable by an English-native mind (and IMO actually makes more sense logically, since it does not invite confusion with the nonsensical "is (not ...)"). The use-cases where you have to grep for "is not" are few, and the "(is not)|(not is)" pattern that would replace it is still pretty simple. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080125/c555c274/attachment.htm From steven.bethard at gmail.com Fri Jan 25 17:17:36 2008 From: steven.bethard at gmail.com (Steven Bethard) Date: Fri, 25 Jan 2008 09:17:36 -0700 Subject: [Python-Dev] Simple syntax proposal: "not is" In-Reply-To: References: Message-ID: On Jan 25, 2008 9:13 AM, Jameson Chema Quinn wrote: > I'm writing a source code editor that translates identifiers and keywords > on-screen into a different natural language. This tool will do no > transformations except at the reversible word level. There is one simple, > avoidable case where this results in nonsense in many languages: "is not". I > propose allowing "not is" as an acceptable alternative to "is not". -1. There should be one obvious way to do it. And honestly, if you're dealing with natural language, and your system is not able to change word order words between languages, you're in a lot of trouble already. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From ncoghlan at gmail.com Fri Jan 25 17:37:02 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 26 Jan 2008 02:37:02 +1000 Subject: [Python-Dev] Simple syntax proposal: "not is" In-Reply-To: References: Message-ID: <479A102E.6000803@gmail.com> Jameson "Chema" Quinn wrote: > I'm writing a source code editor that translates identifiers and > keywords on-screen into a different natural language. This tool will do > no transformations except at the reversible word level. There is one > simple, avoidable case where this results in nonsense in many languages: > "is not". I propose allowing "not is" as an acceptable alternative to > "is not". Your editor is going to have to deal with code written the normal way anyway - given that this is a pretty special case, the special handling should stay in the tool that needs it (your editor) not in the programming language. As you say, Python is heavily influenced by English, and in English "a not is b" is out-and-out nonsense (there is no way to build a coherent parse tree with that word order), while "not a is b" and "a is not b" both make sense (although you are correct in pointing out that the latter is technically ambiguous as an English phrase). "is not" and "not in" are just normal binary operators that happen to consist of two words in the concrete syntax - your editor is going to need to be able to deal with that (even if that means having to handle translations that span multiple words). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From jyasskin at gmail.com Fri Jan 25 17:57:46 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Fri, 25 Jan 2008 08:57:46 -0800 Subject: [Python-Dev] trunc() In-Reply-To: References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> Message-ID: <5d44f72f0801250857g180fc277g16c7c909d86cbb52@mail.gmail.com> On Jan 25, 2008 4:28 AM, Facundo Batista wrote: > 2008/1/24, Guido van Rossum : > > > > So you won't be able to construct an int from a float? That sucks (and > > > is unintuitive). > > > > Yes, you can, but you have to specify how you want it done by using > > trunc() or round() or ceil() or floor(). (In 3.0, round(x) will return > > an int, not a float.) > > > 2008/1/24, Jeffrey Yasskin : > > > That needs to be updated and implemented. I think the decision was > > that removing float.__int__() would break too much, so it needs a > > deprecation warning in 3.0. > > > What I understand here is as int() is "ambiguous", in the future if > you want to specify how you want to convert a float to int. > > But ceil and floor returns a float. And round and trunc will return an > int. So, how I could convert a float to its upper int? Like this?: > > >>> trunc(math.ceil(.3)) > 1 Like this, in 3.0: >>> math.ceil(2.2) 3 There was a previous thread in which we decided not to change that behavior in 2.6. > BTW, int is not giving me a deprecation warning: > > >>> int(.1) > 0 Correct; that's not implemented yet. -- Namast?, Jeffrey Yasskin http://jeffrey.yasskin.info/ From jyasskin at gmail.com Fri Jan 25 18:24:07 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Fri, 25 Jan 2008 09:24:07 -0800 Subject: [Python-Dev] trunc() In-Reply-To: <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> Message-ID: <5d44f72f0801250924m63f70047x9f3d1106d7580801@mail.gmail.com> On Jan 25, 2008 5:53 AM, Paul Moore wrote: > On 24/01/2008, Jeffrey Yasskin wrote: > > int has to be a builtin because it's a fundamental type. trunc() > > followed round() into the builtins. I have no opinion on whether ceil > > and floor should move there; it probably depends on how often they're > > used. > > Suggestion: > > - int() has to stay in builtins for obvious reasons. > - put *all* of trunc, ceil, floor, round into math. > - make int(float) an error I'd deal with Facundo's objection that you should be able to convert between builtin datatypes without the use of a module by leaving trunc in the builtins, but it looks like I'm in the minority here. If the decision comes to be that int(float) should be blessed as a correct way to truncate a float, I'd agree with Raymond that trunc() is just duplication and should be eliminated. I'd, of course, rather have a spelling that says what it means. :) > The only fly in the ointment is that 2to3 can't handle the semantic > issues around converting int(n) to math.trunc(n) because it can't > detect the type of n. So why not make int(float) warn in -3 mode on > 2.6, and then let 2to3 do the conversion (on the basis that 2to3 > should only be run on code that is -3 warning free)? I'd be happy with that too. -- Namast?, Jeffrey Yasskin From facundobatista at gmail.com Fri Jan 25 18:45:06 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Fri, 25 Jan 2008 15:45:06 -0200 Subject: [Python-Dev] trunc() In-Reply-To: <5d44f72f0801250924m63f70047x9f3d1106d7580801@mail.gmail.com> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801250924m63f70047x9f3d1106d7580801@mail.gmail.com> Message-ID: 2008/1/25, Jeffrey Yasskin : > decision comes to be that int(float) should be blessed as a correct > way to truncate a float, I'd agree with Raymond that trunc() is just > duplication and should be eliminated. I'd, of course, rather have a > spelling that says what it means. :) Mmm... no. int() is a builtin way to transform the builtin data type float into the builtin data type float. There's no "correct" way for a float to become an integer, but in the math module you have several ways to do it (floor, ceil, round, trunc, choose the one that you want, but you're "notified" that there're different ways to do it). -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From jyasskin at gmail.com Fri Jan 25 19:03:38 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Fri, 25 Jan 2008 10:03:38 -0800 Subject: [Python-Dev] trunc() In-Reply-To: References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801250924m63f70047x9f3d1106d7580801@mail.gmail.com> Message-ID: <5d44f72f0801251003y1cdde73dn78b464d037083718@mail.gmail.com> On Jan 25, 2008 9:45 AM, Facundo Batista wrote: > 2008/1/25, Jeffrey Yasskin : > > > decision comes to be that int(float) should be blessed as a correct > > way to truncate a float, I'd agree with Raymond that trunc() is just > > duplication and should be eliminated. I'd, of course, rather have a > > spelling that says what it means. :) > > Mmm... no. int() is a builtin way to transform the builtin data type > float into the builtin data type float. I assume you meant "int" instead of your second "float". There doesn't have to be a way to convert from every builtin type to every other builtin type. Take dict(float), for example. Further, if there is a conversion, no law says it has to be named the same as the target type. If there are two plausible ways to get from the source to the target, using just the target's name to pick out one of them is really a terrible idea. But, just because the designers of C made a bad decision that we've inherited, doesn't mean that it's a terrible idea to keep it. As Nick said, defining int(float) as truncation has a long history, and there are some obvious costs to breaking away from that. I think it is time to correct the mistake, but it's not an open and shut case. -- Namast?, Jeffrey Yasskin From guido at python.org Fri Jan 25 19:13:37 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 25 Jan 2008 10:13:37 -0800 Subject: [Python-Dev] Simple syntax proposal: "not is" In-Reply-To: References: Message-ID: On Jan 25, 2008 8:13 AM, Jameson Chema Quinn wrote: > I'm writing a source code editor that translates identifiers and keywords > on-screen into a different natural language. This tool will do no > transformations except at the reversible word level. There is one simple, > avoidable case where this results in nonsense in many languages: "is not". I > propose allowing "not is" as an acceptable alternative to "is not". > > Obviously English syntax has a deep influence on python syntax, and I would > never propose deeper syntactical changes for natural-language-compatibility. > This is a trivial change, one that is still easily parseable by an > English-native mind (and IMO actually makes more sense logically, since it > does not invite confusion with the nonsensical "is (not ...)"). The > use-cases where you have to grep for "is not" are few, and the "(is > not)|(not is)" pattern that would replace it is still pretty simple. Sorry, but this use case just doesn't sound strong enough to change a programming language's grammar. While I promise you I will remain -1 on the proposal simply because it doesn't serve any programmer's goals, you've piqued my curiosity -- can you give an example of what your tool does? From your description I actually have no clue. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From adlaiff6 at gmail.com Fri Jan 25 19:22:14 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Fri, 25 Jan 2008 13:22:14 -0500 Subject: [Python-Dev] trunc() In-Reply-To: References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801250924m63f70047x9f3d1106d7580801@mail.gmail.com> Message-ID: On Jan 25, 2008 12:45 PM, Facundo Batista wrote: > Mmm... no. int() is a builtin way to transform the builtin data type > float into the builtin data type float [sic]. > > There's no "correct" way for a float to become an integer, but in the > math module you have several ways to do it (floor, ceil, round, trunc, > choose the one that you want, but you're "notified" that > there're different ways to do it). In keeping with this theme, why not define int() for floats (and other real types) as def __int__(n, method=math.trunc) or something similar, so that, by default, int() provides the same functionality as before (or whatever is decided to be preferred, I'm making no judgements on that end), but has a way --- by passing a different function --- of changing the way it rounds? The other (probably preferred) option, I suppose, would be to provide a few constants (float.FLOOR_METHOD et al.) instead of passing an arbitrary function (which, of course, makes me a bit uncomfortable). -- Cheers, Leif From python at rcn.com Fri Jan 25 20:22:12 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 25 Jan 2008 14:22:12 -0500 (EST) Subject: [Python-Dev] trunc() Message-ID: <20080125142212.AFS43742@ms19.lnh.mail.rcn.net> > If the decision comes to be that int(float) should be blessed > as a correct way to truncate a float, I'd agree with Raymond > that trunc() is just duplication and should be eliminated. Yay, we've make progress! > I'd,of course, rather have a spelling that says what it means. :) I wouldn't fret about this too much. Intrepreting int(f) as meaning truncate has a *long* history in *many* programming languages. It is a specious argument int(f) is ambiguous. No one thinks it means ceil(f). Go ask a dozen people if they are surprised that int(3.7) returns 3. No one will be surprised (even folks who just use Excel or VB). It is foolhardy to be a purist and rage against the existing art: SQL: "The INT() function returns its numeric argument with any fractional digits removed and truncates all digits to the right of the decimal point." www.basis.com/onlinedocs/documentation/b3odbc/bbjds_int_function.htm VB: "Both the Int and Fix functions remove the fractional part of Number and return the resulting integer value." http://msdn2.microsoft.com/en-us/library/xh29swte.aspx Excel: "The Int function returns the integer portion of a number." http://www.techonthenet.com/excel/formulas/int.php These docs suggest where the thinking has gone wrong. Writing int(f) doesn't mean "arbritrary select one of round|ceil|floor|trunc as a way of getting to an integer"; instead, it means "return the integer portion (non-fractional component) of a number." The latter definition seems common and is entirely unambiguous. Raymond From guido at python.org Fri Jan 25 20:32:54 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 25 Jan 2008 11:32:54 -0800 Subject: [Python-Dev] trunc() In-Reply-To: <20080125142212.AFS43742@ms19.lnh.mail.rcn.net> References: <20080125142212.AFS43742@ms19.lnh.mail.rcn.net> Message-ID: Does no-one thinks it means round(f) either? That's the main confusion here (plus the fact that in C it's undefined -- or at some point was undefined). BTW the list of functions considered here should include round() in addition to ceil(), floor(), and trunc(), even if 2-arg round() doesn't quite fit. --Guido On Jan 25, 2008 11:22 AM, Raymond Hettinger wrote: > > If the decision comes to be that int(float) should be blessed > > as a correct way to truncate a float, I'd agree with Raymond > > that trunc() is just duplication and should be eliminated. > > Yay, we've make progress! > > > > I'd,of course, rather have a spelling that says what it means. :) > > I wouldn't fret about this too much. Intrepreting int(f) as > meaning truncate has a *long* history in *many* programming > languages. It is a specious argument int(f) is ambiguous. > No one thinks it means ceil(f). > > Go ask a dozen people if they are surprised that int(3.7) returns 3. > No one will be surprised (even folks who just use Excel or VB). It > is foolhardy to be a purist and rage against the existing art: > > SQL: "The INT() function returns its numeric argument with any fractional > digits removed and truncates all digits to the right of the decimal > point." > www.basis.com/onlinedocs/documentation/b3odbc/bbjds_int_function.htm > > VB: "Both the Int and Fix functions remove the fractional part of > Number and return the resulting integer value." > http://msdn2.microsoft.com/en-us/library/xh29swte.aspx > > Excel: "The Int function returns the integer portion of a number." > http://www.techonthenet.com/excel/formulas/int.php > > These docs suggest where the thinking has gone wrong. Writing int(f) > doesn't mean "arbritrary select one of round|ceil|floor|trunc as > a way of getting to an integer"; instead, it means "return the > integer portion (non-fractional component) of a number." The > latter definition seems common and is entirely unambiguous. > > > Raymond > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From asmodai at in-nomine.org Fri Jan 25 20:35:20 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Fri, 25 Jan 2008 20:35:20 +0100 Subject: [Python-Dev] trunc() In-Reply-To: References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> Message-ID: <20080125193520.GD1050@nexus.in-nomine.org> [I am still recovering, so if I say something totally misinformed I blame my recovery. :) ] -On [20080125 15:12], Christian Heimes (lists at cheimes.de) wrote: >Python 3:0 > > 2.4 ( 2, 3, 2, 2, 2) > 2.6 ( 2, 3, 3, 2, 2) >-2.4 (-3, -2, -2, -2, -2) >-2.6 (-3, -2, -3, -2, -2) > >Python 2.6: > 2.4 ( 2.0, 3.0, 2.0, 2, 2) > 2.6 ( 2.0, 3.0, 3.0, 2, 2) >-2.4 (-3.0, -2.0, -2.0, -2, -2) >-2.6 (-3.0, -2.0, -3.0, -2, -2) Am I the only one who wonders about the sudden change in decimal significance? Especially given the fact that the ISO C standard specifies floor(), for example, as returning a floating point value and the above in Python 3.0 deviates to just returning an integer. Which is also different from 2.5's behaviour. Can I assume we are all familiar with the concept of significant digits and that we agree that from this point of view 2 != 2.0? And that results such as the above would be a regression and loss in precision? -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ We have met the enemy and they are ours... From phd at phd.pp.ru Fri Jan 25 20:38:26 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Fri, 25 Jan 2008 22:38:26 +0300 Subject: [Python-Dev] trunc() In-Reply-To: References: <20080125142212.AFS43742@ms19.lnh.mail.rcn.net> Message-ID: <20080125193826.GA14626@phd.pp.ru> On Fri, Jan 25, 2008 at 11:32:54AM -0800, Guido van Rossum wrote: > Does no-one thinks it means round(f) either? I don't think so. I often emulate round(f) as int(f + 0.5). Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From solipsis at pitrou.net Fri Jan 25 20:39:17 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 25 Jan 2008 19:39:17 +0000 (UTC) Subject: [Python-Dev] trunc() References: <20080125142212.AFS43742@ms19.lnh.mail.rcn.net> Message-ID: Raymond Hettinger rcn.com> writes: > Go ask a dozen people if they are surprised that int(3.7) returns 3. > No one will be surprised (even folks who just use Excel or VB). It > is foolhardy to be a purist and rage against the existing art: > Well, for what it's worth, here are MySQL's own two cents: mysql> create table t (a int); Query OK, 0 rows affected (0.00 sec) mysql> insert t (a) values (1.4), (1.6), (-1.6), (-1.4); Query OK, 4 rows affected (0.00 sec) Records: 4 Duplicates: 0 Warnings: 0 mysql> select * from t; +------+ | a | +------+ | 1 | | 2 | | -2 | | -1 | +------+ 4 rows in set (0.00 sec) From jflatow at northwestern.edu Fri Jan 25 20:47:19 2008 From: jflatow at northwestern.edu (Jared Flatow) Date: Fri, 25 Jan 2008 13:47:19 -0600 Subject: [Python-Dev] trunc() In-Reply-To: <20080125142212.AFS43742@ms19.lnh.mail.rcn.net> References: <20080125142212.AFS43742@ms19.lnh.mail.rcn.net> Message-ID: On Jan 25, 2008, at 1:22 PM, Raymond Hettinger wrote: > I wouldn't fret about this too much. Intrepreting int(f) as > meaning truncate has a *long* history in *many* programming > languages. It is a specious argument int(f) is ambiguous. > No one thinks it means ceil(f). Not that I think my opinion will have any weight in this discussion, but I'd agree that int has a long history not likely to be misinterpreted when applied to real numbers. Quoting from Graham, Knuth and Patashnik "Concrete Mathematics...2nd edition" page 67: "We start by covering the floor (greatest integer) and ceiling (least integer) functions, which are defined for all real x... ...some pocket calculators have an INT function, defined as floor(x) when x is positive and ceil(x) when x is negative. The designers of these calculators probably wanted their INT function to satisfy the identity INT(-X) = -INT(X). But we'll stick to our floor and ceiling functions, because they have even nicer properties than this." jared From python at rcn.com Fri Jan 25 20:50:56 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 25 Jan 2008 14:50:56 -0500 (EST) Subject: [Python-Dev] trunc() Message-ID: <20080125145056.AFS48804@ms19.lnh.mail.rcn.net> >.some pocket calculators have an INT function, defined > as floor(x) when x is positive and ceil(x) when x is negative That's the mathematical definition. The way they explain it is dirt simple: return the integer portion of a number. Some of the calculators that have int() also have frac() which has the obvious interpretation. Raymond From collinw at gmail.com Fri Jan 25 20:56:27 2008 From: collinw at gmail.com (Collin Winter) Date: Fri, 25 Jan 2008 11:56:27 -0800 Subject: [Python-Dev] Simple syntax proposal: "not is" In-Reply-To: References: Message-ID: <43aa6ff70801251156g6258a046yc4b87d32136ff1a7@mail.gmail.com> On Jan 25, 2008 8:13 AM, Jameson Chema Quinn wrote: > I'm writing a source code editor that translates identifiers and keywords > on-screen into a different natural language. This tool will do no > transformations except at the reversible word level. There is one simple, > avoidable case where this results in nonsense in many languages: "is not". I > propose allowing "not is" as an acceptable alternative to "is not". > > Obviously English syntax has a deep influence on python syntax, and I would > never propose deeper syntactical changes for natural-language-compatibility. > This is a trivial change, one that is still easily parseable by an > English-native mind (and IMO actually makes more sense logically, since it > does not invite confusion with the nonsensical "is (not ...)"). The > use-cases where you have to grep for "is not" are few, and the "(is > not)|(not is)" pattern that would replace it is still pretty simple. "not is" makes no sense to -- and is not easily parsed by -- my English-native mind. Collin Winter From python at rcn.com Fri Jan 25 21:00:56 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 25 Jan 2008 15:00:56 -0500 (EST) Subject: [Python-Dev] trunc() Message-ID: <20080125150056.AFS50430@ms19.lnh.mail.rcn.net> [GvR] > Does no-one thinks it means round(f) either? Heck no. The int() and round() functions have been in Lotus and Excel for an eternity and nobody has a problem learning what those functions do. Also, the extra argument for round(f, n) makes it clear that it can return other floats rounded to n-digits. I've taught a lot of classes to spreadsheet users and have observed that people get it right away. Raymond From steve at holdenweb.com Fri Jan 25 21:26:28 2008 From: steve at holdenweb.com (Steve Holden) Date: Fri, 25 Jan 2008 15:26:28 -0500 Subject: [Python-Dev] trunc() In-Reply-To: References: <20080125142212.AFS43742@ms19.lnh.mail.rcn.net> Message-ID: <479A45F4.2040603@holdenweb.com> Antoine Pitrou wrote: > Raymond Hettinger rcn.com> writes: >> Go ask a dozen people if they are surprised that int(3.7) returns 3. >> No one will be surprised (even folks who just use Excel or VB). It >> is foolhardy to be a purist and rage against the existing art: >> > > Well, for what it's worth, here are MySQL's own two cents: > > mysql> create table t (a int); > Query OK, 0 rows affected (0.00 sec) > > mysql> insert t (a) values (1.4), (1.6), (-1.6), (-1.4); > Query OK, 4 rows affected (0.00 sec) > Records: 4 Duplicates: 0 Warnings: 0 > > mysql> select * from t; > +------+ > | a | > +------+ > | 1 | > | 2 | > | -2 | > | -1 | > +------+ > 4 rows in set (0.00 sec) > Two points. Firstly, regarding MySQL as authoritative from a standards point of view is bound to lead to trouble, since they have always played fast and loose with the standard for reasons (I suspect) of implementation convenience. Second, that example isn't making use of the INT() function. I was going to show you result of taking the INT() of a float column containing your test values. That was when I found out that MySQL (5.0.41, anyway) doesn't implement the INT() function. What was I saying about standards? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Fri Jan 25 21:28:20 2008 From: steve at holdenweb.com (Steve Holden) Date: Fri, 25 Jan 2008 15:28:20 -0500 Subject: [Python-Dev] Simple syntax proposal: "not is" In-Reply-To: References: Message-ID: Guido van Rossum wrote: > On Jan 25, 2008 8:13 AM, Jameson Chema Quinn wrote: >> I'm writing a source code editor that translates identifiers and keywords >> on-screen into a different natural language. This tool will do no >> transformations except at the reversible word level. There is one simple, >> avoidable case where this results in nonsense in many languages: "is not". I >> propose allowing "not is" as an acceptable alternative to "is not". >> >> Obviously English syntax has a deep influence on python syntax, and I would >> never propose deeper syntactical changes for natural-language-compatibility. >> This is a trivial change, one that is still easily parseable by an >> English-native mind (and IMO actually makes more sense logically, since it >> does not invite confusion with the nonsensical "is (not ...)"). The >> use-cases where you have to grep for "is not" are few, and the "(is >> not)|(not is)" pattern that would replace it is still pretty simple. > > Sorry, but this use case just doesn't sound strong enough to change a > programming language's grammar. > > While I promise you I will remain -1 on the proposal simply because it > doesn't serve any programmer's goals, you've piqued my curiosity -- > can you give an example of what your tool does? From your description > I actually have no clue. > It not does sound like a very good idea to me. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From p.f.moore at gmail.com Fri Jan 25 21:32:53 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 25 Jan 2008 20:32:53 +0000 Subject: [Python-Dev] trunc() In-Reply-To: References: <20080125142212.AFS43742@ms19.lnh.mail.rcn.net> Message-ID: <79990c6b0801251232n775baf3cv52cb59825054c50b@mail.gmail.com> On 25/01/2008, Guido van Rossum wrote: > Does no-one thinks it means round(f) either? That's the main confusion > here (plus the fact that in C it's undefined -- or at some point was > undefined). Not me. My intuition agrees with Raymond that int means "the integer part of", i.e. trunc(), when dealing with a numeric argument. (int(string) is a different matter, and I have no problem having different intuitions for that. So sue me) I'd like to keep trunc (as math.trunc) for those cases where I want to be completely explicit, but it's not a big deal to me. > BTW the list of functions considered here should include round() in > addition to ceil(), floor(), and trunc(), even if 2-arg round() > doesn't quite fit. My original message suggested that round go into math as well. It didn't get much comment, though, as everyone latched onto the more controversial suggestion that int(float) be an error - which is the bit I was least concerned about, ironically. Paul. From mal at egenix.com Fri Jan 25 22:03:44 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 25 Jan 2008 22:03:44 +0100 Subject: [Python-Dev] trunc() In-Reply-To: <479A45F4.2040603@holdenweb.com> References: <20080125142212.AFS43742@ms19.lnh.mail.rcn.net> <479A45F4.2040603@holdenweb.com> Message-ID: <479A4EB0.7020707@egenix.com> On 2008-01-25 21:26, Steve Holden wrote: > Antoine Pitrou wrote: >> Raymond Hettinger rcn.com> writes: >>> Go ask a dozen people if they are surprised that int(3.7) returns 3. >>> No one will be surprised (even folks who just use Excel or VB). It >>> is foolhardy to be a purist and rage against the existing art: >>> >> Well, for what it's worth, here are MySQL's own two cents: >> >> mysql> create table t (a int); >> Query OK, 0 rows affected (0.00 sec) >> >> mysql> insert t (a) values (1.4), (1.6), (-1.6), (-1.4); >> Query OK, 4 rows affected (0.00 sec) >> Records: 4 Duplicates: 0 Warnings: 0 >> >> mysql> select * from t; >> +------+ >> | a | >> +------+ >> | 1 | >> | 2 | >> | -2 | >> | -1 | >> +------+ >> 4 rows in set (0.00 sec) >> > Two points. Firstly, regarding MySQL as authoritative from a standards > point of view is bound to lead to trouble, since they have always played > fast and loose with the standard for reasons (I suspect) of > implementation convenience. > > Second, that example isn't making use of the INT() function. I was going > to show you result of taking the INT() of a float column containing your > test values. That was when I found out that MySQL (5.0.41, anyway) > doesn't implement the INT() function. What was I saying about standards? FWIW, here's what IBM has to say to this: http://publib.boulder.ibm.com/infocenter/db2luw/v9/index.jsp?topic=/com.ibm.db2.udb.admin.doc/doc/r0000814.htm """If the argument is a numeric-expression, the result is the same number that would occur if the argument were assigned to a large integer column or variable. If the whole part of the argument is not within the range of integers, an error occurs. The decimal part of the argument is truncated if present.""" AFAIK, the INTEGER() function is not part of the SQL standard, at least not of SQL92: http://www.contrib.andrew.cmu.edu/~shadow/sql/sql1992.txt The way to convert a value to an integer is by casting it to one, e.g. CAST (X AS INTEGER). The INT() function is basically a short-cut for this. Regards, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 25 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From solipsis at pitrou.net Fri Jan 25 23:11:21 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 25 Jan 2008 22:11:21 +0000 (UTC) Subject: [Python-Dev] trunc() References: <20080125142212.AFS43742@ms19.lnh.mail.rcn.net> <479A45F4.2040603@holdenweb.com> Message-ID: Hello, > Two points. Firstly, regarding MySQL as authoritative from a standards > point of view is bound to lead to trouble, since they have always played > fast and loose with the standard for reasons (I suspect) of > implementation convenience. To that I heartily agree. I was just pointing out that implicit conversion of float to int did not always use the moral equivalent of trunc() under all programming platforms. That said, I don't think it's that important, it's just something you have to learn. Regards Antoine. From martin at v.loewis.de Sat Jan 26 00:06:17 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 26 Jan 2008 00:06:17 +0100 Subject: [Python-Dev] why is mmap a builtin module on windows? In-Reply-To: <932f8baf0801250156l6c978ae5v7aad7f03c70a4685@mail.gmail.com> References: <932f8baf0801222353v2593e6edq556a6fe5a3e25e60@mail.gmail.com> <4797ADBF.5050201@v.loewis.de> <932f8baf0801250156l6c978ae5v7aad7f03c70a4685@mail.gmail.com> Message-ID: <479A6B69.609@v.loewis.de> > b) automate all aspects of adding modules that should not go > into pythonxy.dll according to the policy." > > > i.e. create visual studio project files for those modules? and make them > built automatically? And adjust msi.py to package them automatically. Regards, Martin From python at rcn.com Sat Jan 26 00:15:50 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 25 Jan 2008 18:15:50 -0500 (EST) Subject: [Python-Dev] Organization of ABC modules Message-ID: <20080125181550.AFS81539@ms19.lnh.mail.rcn.net> All of the abstract base classes should be collected in one place. I propose that the modules be collected into a package so that we can write: import abc.numbers import abc.collections . . . Besides collecting all the relevant code in one place, it has a nice additional benefit of clearly designating when you're working with one of the provided abstract base classes. When I see "import numbers", I don't immediately recognize this as being somehow different from "import math" or some other concrete implementation. IMO. the emerging practice of spreading modules these across the library isn't serving us well. Raymond From guido at python.org Sat Jan 26 00:26:32 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 25 Jan 2008 15:26:32 -0800 Subject: [Python-Dev] Organization of ABC modules In-Reply-To: <20080125181550.AFS81539@ms19.lnh.mail.rcn.net> References: <20080125181550.AFS81539@ms19.lnh.mail.rcn.net> Message-ID: On Jan 25, 2008 3:15 PM, Raymond Hettinger wrote: > All of the abstract base classes should be collected in one place. I propose that the modules be collected into a package so that we can write: > > import abc.numbers > import abc.collections > . . . > > Besides collecting all the relevant code in one place, it has a nice additional benefit of clearly designating when you're working with one of the provided abstract base classes. When I see "import numbers", I don't immediately recognize this as being somehow different from "import math" or some other concrete implementation. > > IMO. the emerging practice of spreading modules these across the library isn't serving us well. I don't think so. Things should be grouped by application area, not by some property like "uses metaclasses" or "defines abstract base classes" or "overrides operators". The old types module collected all "built-in types" and it was a mistake. You might as well propose that all decorators should live in the same package, or all generic functions (if we ever have them). There's a lot more synergy between the collection ABCs and other collections than between collection ABCs and number ABCs. Another angle: the io.py module also uses ABCs. Should it therefore be a subpackage of ABC? No way! If you want ABCs to be more easily recognizable as such, perhaps we could use a naming convention, though personally I think we don't need that either -- the generic names like "Sequence" or "Real" are enough of a tip-off that these are type categories and not implementation types. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Sat Jan 26 00:48:13 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 25 Jan 2008 18:48:13 -0500 (EST) Subject: [Python-Dev] Organization of ABC modules Message-ID: <20080125184813.AFS85141@ms19.lnh.mail.rcn.net> > If you want ABCs to be more easily recognizable > as such, perhaps we could use a naming convention, Essentially, that's all I was asking for. It doesn't really matter to me whether numbers.py gets called abc_numbers or abc.numbers. Either one would be an improvement. Raymond From steve at holdenweb.com Sat Jan 26 01:01:07 2008 From: steve at holdenweb.com (Steve Holden) Date: Fri, 25 Jan 2008 19:01:07 -0500 Subject: [Python-Dev] Organization of ABC modules In-Reply-To: <20080125184813.AFS85141@ms19.lnh.mail.rcn.net> References: <20080125184813.AFS85141@ms19.lnh.mail.rcn.net> Message-ID: Raymond Hettinger wrote: >> If you want ABCs to be more easily recognizable >> as such, perhaps we could use a naming convention, > > Essentially, that's all I was asking for. It doesn't > really matter to me whether numbers.py gets called > abc_numbers or abc.numbers. Either one would be an > improvement. > abstract_numbers would seem to express the essence without focusing on the mechanism unduly. Of course no sane person would suggest using a keyword, like import abstract numbers Wouldn't want to let reader know what was going on ... regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From ncoghlan at gmail.com Sat Jan 26 06:09:27 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 26 Jan 2008 15:09:27 +1000 Subject: [Python-Dev] trunc() In-Reply-To: <20080125193520.GD1050@nexus.in-nomine.org> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <20080125193520.GD1050@nexus.in-nomine.org> Message-ID: <479AC087.7080604@gmail.com> Jeroen Ruigrok van der Werven wrote: > Can I assume we are all familiar with the concept of significant digits and > that we agree that from this point of view 2 != 2.0? And that results such as > the above would be a regression and loss in precision? Not really, because if someone cares about precision to that extent they won't be using binary floats anyway (they'll be using Decimal, or something like it). Besides, for trunc, ceil and floor, it's all zeroes after the decimal point by definition (no matter how many significant digits you started with). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Sat Jan 26 06:31:49 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 26 Jan 2008 15:31:49 +1000 Subject: [Python-Dev] Organization of ABC modules In-Reply-To: <20080125184813.AFS85141@ms19.lnh.mail.rcn.net> References: <20080125184813.AFS85141@ms19.lnh.mail.rcn.net> Message-ID: <479AC5C5.9040509@gmail.com> Raymond Hettinger wrote: >> If you want ABCs to be more easily recognizable >> as such, perhaps we could use a naming convention, > > Essentially, that's all I was asking for. It doesn't > really matter to me whether numbers.py gets called > abc_numbers or abc.numbers. Either one would be an > improvement. I think the module level is the wrong granularity to be thinking about this - look at a module like collections, which contains not only ABC's related to containers, but also some useful concrete containers like deque and namedtuple. Any naming convention would have to be at the class level. For example: class NumberABC(metaclass=ABCMeta): ... class ComplexABC(NumberABC): ... class RealABC(ComplexABC): ... class RationalABC(NumberABC): ... class IntegralABC(RationalABC): ... I think adopting such a convention (at least for the standard library) would actually be useful. Normally, finding isinstance() type checks in code bothers me severely as it usually indicates that duck-typing has been broken. On the other hand, if I find a check in code that does something like: if not isinstance(x, NumberABC): raise TypeError("Not a number!") it would clearly still be extensible, as I would know just from the naming convention that a third party can register their type with NumberABC to make it usable with this function. Without a naming convention, I would always have to go check the definition of the type in the isinstance() call to see if the code was legitimate. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From jyasskin at gmail.com Sat Jan 26 06:38:14 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Fri, 25 Jan 2008 21:38:14 -0800 Subject: [Python-Dev] Organization of ABC modules In-Reply-To: <479AC5C5.9040509@gmail.com> References: <20080125184813.AFS85141@ms19.lnh.mail.rcn.net> <479AC5C5.9040509@gmail.com> Message-ID: <5d44f72f0801252138i4cf57786k7faf75a523fe71d6@mail.gmail.com> On Jan 25, 2008 9:31 PM, Nick Coghlan wrote: > Raymond Hettinger wrote: > >> If you want ABCs to be more easily recognizable > >> as such, perhaps we could use a naming convention, > > > > Essentially, that's all I was asking for. It doesn't > > really matter to me whether numbers.py gets called > > abc_numbers or abc.numbers. Either one would be an > > improvement. > > I think the module level is the wrong granularity to be thinking about > this - look at a module like collections, which contains not only ABC's > related to containers, but also some useful concrete containers like > deque and namedtuple. > > Any naming convention would have to be at the class level. For example: > > class NumberABC(metaclass=ABCMeta): > ... > > class ComplexABC(NumberABC): > ... > > class RealABC(ComplexABC): > ... > > class RationalABC(NumberABC): > ... > > class IntegralABC(RationalABC): > ... > > I think adopting such a convention (at least for the standard library) > would actually be useful. Normally, finding isinstance() type checks in > code bothers me severely as it usually indicates that duck-typing has > been broken. On the other hand, if I find a check in code that does > something like: > > if not isinstance(x, NumberABC): > raise TypeError("Not a number!") > > it would clearly still be extensible, as I would know just from the > naming convention that a third party can register their type with > NumberABC to make it usable with this function. > > Without a naming convention, I would always have to go check the > definition of the type in the isinstance() call to see if the code was > legitimate. That's a good point. Someone already convinced me to name the test for numbers.py, test_abstract_numbers.py, so renaming the module makes sense too, although I agree that collections, which contains some concrete classes, should keep its current name. If others agree, want to send a patch? -- Namast?, Jeffrey Yasskin From ncoghlan at gmail.com Sat Jan 26 06:55:08 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 26 Jan 2008 15:55:08 +1000 Subject: [Python-Dev] Organization of ABC modules In-Reply-To: <5d44f72f0801252138i4cf57786k7faf75a523fe71d6@mail.gmail.com> References: <20080125184813.AFS85141@ms19.lnh.mail.rcn.net> <479AC5C5.9040509@gmail.com> <5d44f72f0801252138i4cf57786k7faf75a523fe71d6@mail.gmail.com> Message-ID: <479ACB3C.3090404@gmail.com> Jeffrey Yasskin wrote: > That's a good point. Someone already convinced me to name the test for > numbers.py, test_abstract_numbers.py, so renaming the module makes > sense too, although I agree that collections, which contains some > concrete classes, should keep its current name. If others agree, want > to send a patch? I'm not so worried about the name of the numbers module, but if Guido likes the idea of a standard naming convention (such as the ABC suffix) for classes that use the ABCMeta metaclass, I'd certainly be happy to go through and update the affected classes and the code which refers to them. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From python at rcn.com Sat Jan 26 07:45:23 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 25 Jan 2008 22:45:23 -0800 Subject: [Python-Dev] Organization of ABC modules References: <20080125184813.AFS85141@ms19.lnh.mail.rcn.net> <479AC5C5.9040509@gmail.com> <5d44f72f0801252138i4cf57786k7faf75a523fe71d6@mail.gmail.com> <479ACB3C.3090404@gmail.com> Message-ID: <00ef01c85fe7$07cc1b00$6800a8c0@RaymondLaptop1> > but if Guido > likes the idea of a standard naming convention (such as the ABC suffix) > for classes that use the ABCMeta metaclass, I'd certainly be happy to go > through and update the affected classes and the code which refers to them. A prefix would be better. Raymond From tjreedy at udel.edu Sat Jan 26 07:50:21 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 26 Jan 2008 01:50:21 -0500 Subject: [Python-Dev] Simple syntax proposal: "not is" References: Message-ID: "Jameson "Chema" Quinn" wrote in message news:faf2c12b0801250813n5b0b38eepe71944b6016224db at mail.gmail.com... | I'm writing a source code editor that translates identifiers and keywords | on-screen into a different natural language. This tool will do no | transformations except at the reversible word level. There is one simple, | avoidable case where this results in nonsense in many languages: "is not". I | propose allowing "not is" as an acceptable alternative to "is not". I an rather sure that the tokenizer outputs "is not" as a single token. Otherwise 'a is not b' would likely be parsed as 'a is (not b)', which is something quit different. So your translater should recognize it as such also and output, for instance (en Espanol), 'no es'. | Obviously English syntax has a deep influence on python syntax, and I would | never propose deeper syntactical changes for natural-language-compatibility. | This is a trivial change, one that is still easily parseable by an | English-native mind (and IMO actually makes more sense logically, since it | does not invite confusion with the nonsensical "is (not ...)"). The | use-cases where you have to grep for "is not" are few, and the "(is | not)|(not is)" pattern that would replace it is still pretty simple. 'a not is b' is much worse in English than I believe 'a es no b' in Espanol. -1 tjr From tjreedy at udel.edu Sat Jan 26 08:03:40 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 26 Jan 2008 02:03:40 -0500 Subject: [Python-Dev] trunc() References: <20080125142212.AFS43742@ms19.lnh.mail.rcn.net> Message-ID: "Guido van Rossum" wrote in message news:ca471dc20801251132j7a758bc5uc076e859dd9f84b0 at mail.gmail.com... | Does no-one thinks it means round(f) either? Not me. Int should truncate, so trunc() not needed unless is does something different. And I would prefer the float-input-only converters be in math. There is nothing second-class, really, about functions in modules. Just more specific. tjr From tjreedy at udel.edu Sat Jan 26 08:10:47 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 26 Jan 2008 02:10:47 -0500 Subject: [Python-Dev] [python] trunc() References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net><47991DDA.8090604@voidspace.org.uk> <47992363.3010402@v.loewis.de> Message-ID: ""Martin v. L?wis"" wrote in message news:47992363.3010402 at v.loewis.de... |> If the ambiguity is that 'int' behaviour is unspecified for floats - is | > it naive to suggest we specify the behaviour? | | The concern is that whatever gets specified is arbitrary. There are many | ways how an int can be constructed from a float, so why is any such way | better than the others, and deserves to be the meaning of int()? Decades of usage, in English, with the meaning it already has in Python. From python at rcn.com Sat Jan 26 08:21:11 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 25 Jan 2008 23:21:11 -0800 Subject: [Python-Dev] trunc() References: <20080124133614.AFQ48572@ms19.lnh.mail.rcn.net> Message-ID: <010001c85fec$07f4bf60$6800a8c0@RaymondLaptop1> > trunc() has well-defined semantics -- it takes a Real instance and > converts it to an Integer instance using round-towards-zero semantics. > > int() has undefined semantics -- it takes any object and converts it > to an int (a concrete type!) So, the problem is basically this: Since int(o) must return an int, you're precluded from defining a new Real class with an __int__() method that returns an instance of some new Integral class that isn't an int. And the proposed solution is: Introduce trunc(o) which calls o.__trunc__() which can return any Integral type. And, client code must use trunc() instead of int(); otherwise, the two new numeric types won't be exercised. Is that basically what trunc() is all about -- a variant of int() that can return something other than an int type -- something that behaves just like the new math.floor() for positive numbers and math.ceil() for negative numbers? If so, I think we would be *much* better-off leaving trunc() out entirely and seeing if any use cases arise that aren't handled by math.floor() and math.ceil() with their precise semantics, unambigous naming, and ability to handle generic Reals and Integrals. The existence of trunc() clashes so severely with existence of int(), that we need to be damned sure that there are real-world, worthwhile problems that are not readily solved with floor() and ceil() for rounding Reals into Integrals or with int() for making ints from the non-fractional portion of a float. Raymond From g.brandl at gmx.net Sat Jan 26 09:43:20 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 26 Jan 2008 09:43:20 +0100 Subject: [Python-Dev] trunc() In-Reply-To: <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> Message-ID: Paul Moore schrieb: > On 24/01/2008, Jeffrey Yasskin wrote: >> int has to be a builtin because it's a fundamental type. trunc() >> followed round() into the builtins. I have no opinion on whether ceil >> and floor should move there; it probably depends on how often they're >> used. > > Suggestion: > > - int() has to stay in builtins for obvious reasons. > - put *all* of trunc, ceil, floor, round into math. That, and making int(float) == int(trunc(float)) seems like reasonable behavior to me as a person not involved in the discussion. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From python at rcn.com Sat Jan 26 10:16:55 2008 From: python at rcn.com (Raymond Hettinger) Date: Sat, 26 Jan 2008 01:16:55 -0800 Subject: [Python-Dev] [Python-checkins] r60283 - in python/trunk: Include/longintrepr.h Include/longobject.h Include/unicodeobject.h Misc/NEWS Modules/_fileio.c Objects/longobject.c Objects/unicodeobject.c setup.py References: <20080125121844.4BC4D1E400F@bag.python.org> Message-ID: <019001c85ffc$32a91c00$6800a8c0@RaymondLaptop1> [christian.heimes] >> Backport of several functions from Python 3.0 to 2.6 including PyUnicode_FromString, PyUnicode_Format and PyLong_From/AsSsize_t. >> The functions are partly required for the backport of the bytearray type and _fileio module. They should also make it easier to >> port C to 3.0. Are you planning to backport bytearrays? I thought we had clearly decided to *not* backport any of the text/bytes model so that the Py2.6 string/unicode model remained pure. Guido proposed a couple of harmless aliases for annotation purposes (to help the 2-to-3 tool do the right thing) and that was it. Your reply was: > Ah, you like to keep it simple. The aliases are easily to implement. > Give me twenty minutes to implement it and write some unit tests. How did we get from two simple aliases to plans for backporting major chucks of the new model? I'm not seeing any limits on Py3.0-->Py2.6 contamination. Raymond From asmodai at in-nomine.org Sat Jan 26 10:17:42 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Sat, 26 Jan 2008 10:17:42 +0100 Subject: [Python-Dev] trunc() In-Reply-To: References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> Message-ID: <20080126091742.GF1050@nexus.in-nomine.org> -On [20080126 09:43], Georg Brandl (g.brandl at gmx.net) wrote: >Paul Moore schrieb: >> - int() has to stay in builtins for obvious reasons. >> - put *all* of trunc, ceil, floor, round into math. > >That, and making int(float) == int(trunc(float)) seems like reasonable >behavior to me as a person not involved in the discussion. +1 from my side for the above. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ We have met the enemy and they are ours... From skip at pobox.com Sat Jan 26 12:41:33 2008 From: skip at pobox.com (skip at pobox.com) Date: Sat, 26 Jan 2008 05:41:33 -0600 Subject: [Python-Dev] TIOBE Programming Community Index Message-ID: <18331.7277.777931.189775@montanaro.dyndns.org> Sorry if this is a repeat. I don't remember seeing this in the various Python-related stuff I've received before (and no longer monitor c.l.py, only look at the weekly summary), though based on the dateline of the Slashdot item I saw (Jan 9) it should be old news. Apparently, in 2007 Python surged in the TIOBE Programming Community Index to #6, surpassing Perl: http://www.tiobe.com/tpci.htm It's got aways to go before it surpasses C++, but that's the next rung in the ladder... I'm a little suspicious of the data though. Given the huge popularity in AJAX tools I would have expected JavaScript to have increased in popularity, not decreased. Hmmm... And they have both Delphi and COBOL jumping in popularity by three rungs. At any rate, there you have it. Skip From fuzzyman at voidspace.org.uk Sat Jan 26 13:26:12 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sat, 26 Jan 2008 12:26:12 +0000 Subject: [Python-Dev] [python] TIOBE Programming Community Index In-Reply-To: <18331.7277.777931.189775@montanaro.dyndns.org> References: <18331.7277.777931.189775@montanaro.dyndns.org> Message-ID: <479B26E4.6070600@voidspace.org.uk> skip at pobox.com wrote: > Sorry if this is a repeat. I don't remember seeing this in the various > Python-related stuff I've received before (and no longer monitor c.l.py, > only look at the weekly summary), though based on the dateline of the > Slashdot item I saw (Jan 9) it should be old news. Apparently, in 2007 > Python surged in the TIOBE Programming Community Index to #6, surpassing > Perl: > > http://www.tiobe.com/tpci.htm > > It's got aways to go before it surpasses C++, but that's the next rung in > the ladder... > > I'm a little suspicious of the data though. Given the huge popularity in > AJAX tools I would have expected JavaScript to have increased in popularity, > not decreased. Hmmm... And they have both Delphi and COBOL jumping in > popularity by three rungs. > This has been fairly widely reported - and in fact is on the front page of Python.org! A site with a more interesting range of metrics (in my opinion) is 'langpop': http://www.langpop.com/ In its 'Normalized Comparison' it shows Python doing well, but not *quite* having overtaken Perl yet. All the best, Michael Foord > At any rate, there you have it. > > Skip > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > > From steve at holdenweb.com Sat Jan 26 13:38:09 2008 From: steve at holdenweb.com (Steve Holden) Date: Sat, 26 Jan 2008 07:38:09 -0500 Subject: [Python-Dev] trunc() In-Reply-To: References: <20080125142212.AFS43742@ms19.lnh.mail.rcn.net> Message-ID: Terry Reedy wrote: > "Guido van Rossum" wrote in message > news:ca471dc20801251132j7a758bc5uc076e859dd9f84b0 at mail.gmail.com... > | Does no-one thinks it means round(f) either? > > Not me. Int should truncate, so trunc() not needed unless is does > something different. > Like returning a value of the same type as the input? int(), being a call to a type, should never return a value of another type. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Sat Jan 26 13:42:11 2008 From: steve at holdenweb.com (Steve Holden) Date: Sat, 26 Jan 2008 07:42:11 -0500 Subject: [Python-Dev] TIOBE Programming Community Index In-Reply-To: <18331.7277.777931.189775@montanaro.dyndns.org> References: <18331.7277.777931.189775@montanaro.dyndns.org> Message-ID: skip at pobox.com wrote: > Sorry if this is a repeat. I don't remember seeing this in the various > Python-related stuff I've received before (and no longer monitor c.l.py, > only look at the weekly summary), though based on the dateline of the > Slashdot item I saw (Jan 9) it should be old news. Apparently, in 2007 > Python surged in the TIOBE Programming Community Index to #6, surpassing > Perl: > > http://www.tiobe.com/tpci.htm > > It's got aways to go before it surpasses C++, but that's the next rung in > the ladder... > > I'm a little suspicious of the data though. Given the huge popularity in > AJAX tools I would have expected JavaScript to have increased in popularity, > not decreased. Hmmm... And they have both Delphi and COBOL jumping in > popularity by three rungs. > > At any rate, there you have it. > """ Python is TIOBE's Language of the Year for 2007 Python moves up from 8 to 6 in TIOBE's list of popular programming languages Published: Fri, 18 Jan 2008, 10:30 -0500 """ From http://python.org/, with a link to the TIOBE index. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From ncoghlan at gmail.com Sat Jan 26 16:01:38 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 27 Jan 2008 01:01:38 +1000 Subject: [Python-Dev] Organization of ABC modules In-Reply-To: <00ef01c85fe7$07cc1b00$6800a8c0@RaymondLaptop1> References: <20080125184813.AFS85141@ms19.lnh.mail.rcn.net> <479AC5C5.9040509@gmail.com> <5d44f72f0801252138i4cf57786k7faf75a523fe71d6@mail.gmail.com> <479ACB3C.3090404@gmail.com> <00ef01c85fe7$07cc1b00$6800a8c0@RaymondLaptop1> Message-ID: <479B4B52.1060301@gmail.com> Raymond Hettinger wrote: >> but if Guido likes the idea of a standard naming convention (such as >> the ABC suffix) for classes that use the ABCMeta metaclass, I'd >> certainly be happy to go through and update the affected classes and >> the code which refers to them. > > A prefix would be better. I initially thought that, but found the suffix to be the least annoying of the ideas I had for denoting abstract base classes. To try and give some idea of how I came to that conclusion I've listed some of the other ideas I had below (again using the numeric stack as an example). INumber IComplex IReal IRational IIntegral Using a simple 'I' prefix annoys me for cases like the last one where the class name actually starts with an I, and 'interface' is a bit of a misnomer for abstract base classes anyway - they have a lot in common, but they're not the same thing. This is also an existing convention for things like zope.interface, so probably not a good idea to adopt to mean something else. ABCNumber ABCComplex ABCReal ABCRational ABCIntegral I find using the ABC acronym as a prefix ends up with too many capitals together at the start of the word and it becomes far more obtrusive than it needs to be. The main thing about these classes is what they represent - the fact that they're ABC's, while still significant enough to indicate somewhere in the name, should be of secondary importance. AbcNumber AbcComplex AbcReal AbcRational AbcIntegral This one bugs me mainly because I don't like lowercasing letters in acronyms just to try and fit in with a camel-casing scheme. It's just ugly. After those 3, I couldn't think of any other prefixes that were both short and likely to be an effective mnemonic for identifying ABC's - Abstract is too long, Abs is far too easy to associated with 'absolute' instead of "Abstract base class", etc. Which lead me to the idea I actually used in my earlier post: NumberABC ComplexABC RealABC RationalABC IntegralABC That way the normal class names are still front and centre, with a short annotation tacked on to the end to say "oh, by the way, this happens to be an abstract base class". I'm not sure any more discussion will be particularly useful though - any convention that gets adopted is going to be by Guido's fiat and will need to suit his aesthetic sense rather than mine :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From lists at cheimes.de Sat Jan 26 17:04:35 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 26 Jan 2008 17:04:35 +0100 Subject: [Python-Dev] [Python-checkins] r60283 - in python/trunk: Include/longintrepr.h Include/longobject.h Include/unicodeobject.h Misc/NEWS Modules/_fileio.c Objects/longobject.c Objects/unicodeobject.c setup.py In-Reply-To: <019001c85ffc$32a91c00$6800a8c0@RaymondLaptop1> References: <20080125121844.4BC4D1E400F@bag.python.org> <019001c85ffc$32a91c00$6800a8c0@RaymondLaptop1> Message-ID: <479B5A13.5070500@cheimes.de> Raymond Hettinger wrote: > Are you planning to backport bytearrays? I thought we had clearly decided to *not* backport any of the text/bytes model so that the > Py2.6 string/unicode model remained pure. The idea raised when I was working on a backport of the io module. The io module requires a mutable byte type. Guido answered a question in my issue http://bugs.python.org/issue1919 Backporting bytearray should be relatively clean since it's a brand new type right? My initial question was: I'm not sure how to proceed with the missing bytearray type: * backport bytearray and the new buffer interface from 3.0 * write a replacement for 2.6 * replace the bytearray code with new code bytearray is a new type that has no equivalent in Python 2.6. Python 3.0's byte type is basically a modified string type. I don't see a problem with bytearray, do you? Christian From guido at python.org Sat Jan 26 18:42:55 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 26 Jan 2008 09:42:55 -0800 Subject: [Python-Dev] Simple syntax proposal: "not is" In-Reply-To: References: Message-ID: On Jan 25, 2008 10:50 PM, Terry Reedy wrote: > "Jameson "Chema" Quinn" wrote in message > news:faf2c12b0801250813n5b0b38eepe71944b6016224db at mail.gmail.com... > | I'm writing a source code editor that translates identifiers and keywords > | on-screen into a different natural language. This tool will do no > | transformations except at the reversible word level. There is one simple, > | avoidable case where this results in nonsense in many languages: "is > not". I > | propose allowing "not is" as an acceptable alternative to "is not". > > I an rather sure that the tokenizer outputs "is not" as a single token. > Otherwise 'a is not b' would likely be parsed as 'a is (not b)', which is > something quit different. So your translater should recognize it as such > also and output, for instance (en Espanol), 'no es'. Sorry to burst your bubble, but 'is' and 'not' are two separate tokens; the grammar handles this by giving unary 'not' a priority lower than comparison operators. > | Obviously English syntax has a deep influence on python syntax, and I > would > | never propose deeper syntactical changes for > natural-language-compatibility. > | This is a trivial change, one that is still easily parseable by an > | English-native mind (and IMO actually makes more sense logically, since > it > | does not invite confusion with the nonsensical "is (not ...)"). The > | use-cases where you have to grep for "is not" are few, and the "(is > | not)|(not is)" pattern that would replace it is still pretty simple. > > 'a not is b' is much worse in English than I believe 'a es no b' in > Espanol. This proposal not is going to happen... -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Jan 26 18:50:43 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 26 Jan 2008 09:50:43 -0800 Subject: [Python-Dev] [Python-checkins] r60283 - in python/trunk: Include/longintrepr.h Include/longobject.h Include/unicodeobject.h Misc/NEWS Modules/_fileio.c Objects/longobject.c Objects/unicodeobject.c setup.py In-Reply-To: <019001c85ffc$32a91c00$6800a8c0@RaymondLaptop1> References: <20080125121844.4BC4D1E400F@bag.python.org> <019001c85ffc$32a91c00$6800a8c0@RaymondLaptop1> Message-ID: On Jan 26, 2008 1:16 AM, Raymond Hettinger wrote: > Are you planning to backport bytearrays? > I thought we had clearly decided to *not* backport any of the text/bytes model so that the > Py2.6 string/unicode model remained pure. 'bytearray' is a separate issue. It's a brand new type: a *mutable* byte array. Its status as a built-in and completely different API makes it much more convenient than using the old array module. Introducing bytearray is not going to disturb the balance between str/bytes/unicode. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fuzzyman at voidspace.org.uk Sat Jan 26 18:57:23 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sat, 26 Jan 2008 17:57:23 +0000 Subject: [Python-Dev] [python] Re: [Python-checkins] r60283 - in python/trunk: Include/longintrepr.h Include/longobject.h Include/unicodeobject.h Misc/NEWS Modules/_fileio.c Objects/longobject.c Objects/unicodeobject.c setup.py In-Reply-To: References: <20080125121844.4BC4D1E400F@bag.python.org> <019001c85ffc$32a91c00$6800a8c0@RaymondLaptop1> Message-ID: <479B7483.7030105@voidspace.org.uk> Guido van Rossum wrote: > On Jan 26, 2008 1:16 AM, Raymond Hettinger wrote: > >> Are you planning to backport bytearrays? >> I thought we had clearly decided to *not* backport any of the text/bytes model so that the >> Py2.6 string/unicode model remained pure. >> > > 'bytearray' is a separate issue. It's a brand new type: a *mutable* > byte array. Its status as a built-in and completely different API > makes it much more convenient than using the old array module. > Introducing bytearray is not going to disturb the balance between > str/bytes/unicode. > I'm looking forward to having a native mutable bytes type in Python 2.x. Michael Foord From python at rcn.com Sat Jan 26 19:45:12 2008 From: python at rcn.com (Raymond Hettinger) Date: Sat, 26 Jan 2008 10:45:12 -0800 Subject: [Python-Dev] [Python-checkins] r60283 - in python/trunk: Include/longintrepr.h Include/longobject.h Include/unicodeobject.h Misc/NEWS Modules/_fileio.c Objects/longobject.c Objects/unicodeobject.c setup.py References: <20080125121844.4BC4D1E400F@bag.python.org> <019001c85ffc$32a91c00$6800a8c0@RaymondLaptop1> Message-ID: <021d01c8604b$9628a940$6800a8c0@RaymondLaptop1> > 'bytearray' is a separate issue. It's a brand new type: a *mutable* > byte array. Its status as a built-in and completely different API > makes it much more convenient than using the old array module. That's reasonable. Raymond From shiblon at gmail.com Sat Jan 26 22:06:31 2008 From: shiblon at gmail.com (Chris Monson) Date: Sat, 26 Jan 2008 16:06:31 -0500 Subject: [Python-Dev] Incorrect documentation of the raw_input built-in function In-Reply-To: References: <18328.49667.162531.446063@montanaro.dyndns.org> Message-ID: Incodentally, I think your example is wrong. It should be either psql >/dev/null 2>&1 or psql &>/dev/null Notice the reversal of redirects in the first one. Of course, this doesn't change anything about the core discussion... - C On 1/24/08, Isaac Morland wrote: > On Thu, 24 Jan 2008, skip at pobox.com wrote: > > > Mike> 2. Is this really the hard-coded behavior we want? I don't think > > Mike> my use-case is that odd; in fact, what I find very odd is that > > Mike> the prompt output is send to stderr. I mean, I'm printing the > > Mike> prompt for a question, not some error message. Can there not > at > > Mike> least be an optional parameter to indicate that you want the > > Mike> output sent to stdout rather than stderr? > > > > I can think of situations where you don't want the output to go to stdout > > either (suppose it's the regular output of the file you want to save to a > > file). Having a choice seems the best route. > > What about an option (maybe even a default) to send the prompt to stdin? > > The Postgres command line interface psql appears to do this: > > $ psql 2>&1 >/dev/null > Password: > $ > > (I typed my password and then I quit by typing ^D; if I type the wrong > password, it looks the same on screen but it quits right away without > waiting for ^D) > > I think ssh also does this when it needs to prompt for a password. > > Really the prompt is part of the input, not part of the output, in a > certain sense. Have people actually verified that the prompt is really > sent to stderr right now by using 2>/dev/null to attempt to suppress it? > > Isaac Morland CSCF Web Guru > DC 2554C, x36650 WWW Software Specialist > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/shiblon%40gmail.com > -- Sent from Gmail for mobile | mobile.google.com From jyasskin at gmail.com Sat Jan 26 23:10:48 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Sat, 26 Jan 2008 14:10:48 -0800 Subject: [Python-Dev] trunc() In-Reply-To: <010001c85fec$07f4bf60$6800a8c0@RaymondLaptop1> References: <20080124133614.AFQ48572@ms19.lnh.mail.rcn.net> <010001c85fec$07f4bf60$6800a8c0@RaymondLaptop1> Message-ID: <5d44f72f0801261410l4fdb830v4badc5e182807f5@mail.gmail.com> On Jan 25, 2008 11:21 PM, Raymond Hettinger wrote: > ... int() for making ints from the non-fractional > portion of a float. This interpretation implies that complex should provide __float__() to return the non-imaginary portion of a complex number. Is that what you intend? -- Namast?, Jeffrey Yasskin From lists at cheimes.de Sat Jan 26 23:23:40 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 26 Jan 2008 23:23:40 +0100 Subject: [Python-Dev] trunc() In-Reply-To: <5d44f72f0801261410l4fdb830v4badc5e182807f5@mail.gmail.com> References: <20080124133614.AFQ48572@ms19.lnh.mail.rcn.net> <010001c85fec$07f4bf60$6800a8c0@RaymondLaptop1> <5d44f72f0801261410l4fdb830v4badc5e182807f5@mail.gmail.com> Message-ID: Jeffrey Yasskin wrote: > This interpretation implies that complex should provide __float__() to > return the non-imaginary portion of a complex number. Is that what you > intend? No, please don't. If somebody wants to implement __float__ for complex numbers please define it as hypot(complex) / sqrt(c.real**2 + c.img**2). In my opinion float(complex) does do the most sensible thing. It fails and points the user to abs(). >>> float(complex(1,1)) Traceback (most recent call last): File "", line 1, in TypeError: can't convert complex to float; use abs(z) >>> abs(complex(1,1)) 1.4142135623730951 From jyasskin at gmail.com Sat Jan 26 23:42:01 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Sat, 26 Jan 2008 14:42:01 -0800 Subject: [Python-Dev] trunc() In-Reply-To: References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> Message-ID: <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> On Jan 26, 2008 12:43 AM, Georg Brandl wrote: > That, and making int(float) == int(trunc(float)) seems like reasonable > behavior to me as a person not involved in the discussion. That's the only position in this discussion that I don't understand. Although int() and trunc() would still have very slightly different behaviors, they seem too close together (identical on most common types) to be compatible with "only one way to do it". I've been advocating trunc() under the assumption that int(float) would be deprecated and eliminated as soon as practical (with an error message similar to float(complex)). Could one of the people in favor of keeping both explain why they think that's a good idea? -- Namast?, Jeffrey Yasskin From python at rcn.com Sat Jan 26 23:46:22 2008 From: python at rcn.com (Raymond Hettinger) Date: Sat, 26 Jan 2008 14:46:22 -0800 Subject: [Python-Dev] trunc() References: <20080124133614.AFQ48572@ms19.lnh.mail.rcn.net> <010001c85fec$07f4bf60$6800a8c0@RaymondLaptop1><5d44f72f0801261410l4fdb830v4badc5e182807f5@mail.gmail.com> Message-ID: <005201c8606d$46d47690$6800a8c0@RaymondLaptop1> [Christian Heimes] > In my opinion float(complex) does do the most sensible thing. It fails > and points the user to abs(). Right. Raymond From python at rcn.com Sat Jan 26 23:53:49 2008 From: python at rcn.com (Raymond Hettinger) Date: Sat, 26 Jan 2008 14:53:49 -0800 Subject: [Python-Dev] trunc() References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net><5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com><79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> Message-ID: <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> > I've been > advocating trunc() under the assumption that int(float) would be > deprecated and eliminated as soon as practical And how much code would break for basically zero benefit? This position is totally nuts. There is *nothing* wrong with int() as it stands now. Nobody has problems with it. The tools is ubiquitous in other languages, spreadsheets, and calculators. Raymond From jyasskin at gmail.com Sun Jan 27 01:00:21 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Sat, 26 Jan 2008 16:00:21 -0800 Subject: [Python-Dev] trunc() In-Reply-To: <005201c8606d$46d47690$6800a8c0@RaymondLaptop1> References: <20080124133614.AFQ48572@ms19.lnh.mail.rcn.net> <010001c85fec$07f4bf60$6800a8c0@RaymondLaptop1> <5d44f72f0801261410l4fdb830v4badc5e182807f5@mail.gmail.com> <005201c8606d$46d47690$6800a8c0@RaymondLaptop1> Message-ID: <5d44f72f0801261600l4ea2d006vd58fa5999ba73197@mail.gmail.com> On Jan 26, 2008 2:46 PM, Raymond Hettinger wrote: > [Christian Heimes] > > In my opinion float(complex) does do the most sensible thing. It fails > > and points the user to abs(). > > Right. To elaborate the point I was trying to make: If float() does not mean "the float part of" and should not take a complex argument (which I completely agree with), then int() does not mean "the int part of" and should not take a float argument. Even assuming that's correct, I'm not certain that it's worth breaking (well, deprecating) backwards compatibility to achieve a more consistent set of functions in this case. I think it is, but I'm happy to leave that point up to the rest of the list (which does seem to be leaning against it). -- Namast?, Jeffrey Yasskin From nnorwitz at gmail.com Sun Jan 27 02:05:15 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sat, 26 Jan 2008 17:05:15 -0800 Subject: [Python-Dev] refleaks and caches Message-ID: Around Jan 13, the refleak hunting test that is reported on python-checkins started to report refleaks on virtually every run. I suspect this is due to r59944 (at 2008-01-13 16:29:41) which was from patch #1700288 to cache methods. With this patch it makes it much harder to spot refleaks. Does anyone have ideas how to fix it? The only one I have is to disable the cache with a special flag, env't variable, sys variable/function or the like. We could make this available only in debug mode. The cache should normally be enabled, but could be disabled solely on the refleak runs. Suggestions? n From tjreedy at udel.edu Sun Jan 27 03:13:28 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 26 Jan 2008 21:13:28 -0500 Subject: [Python-Dev] trunc() References: <20080124133614.AFQ48572@ms19.lnh.mail.rcn.net><010001c85fec$07f4bf60$6800a8c0@RaymondLaptop1><5d44f72f0801261410l4fdb830v4badc5e182807f5@mail.gmail.com><005201c8606d$46d47690$6800a8c0@RaymondLaptop1> <5d44f72f0801261600l4ea2d006vd58fa5999ba73197@mail.gmail.com> Message-ID: "Jeffrey Yasskin" wrote in message news:5d44f72f0801261600l4ea2d006vd58fa5999ba73197 at mail.gmail.com... || To elaborate the point I was trying to make: If float() does not mean | "the float part of" The 'float part' of a complex number is meaningless since both components of a complex are floats (in practice, or reals in theory). The same is true in polar representation. | and should not take a complex argument (which I | completely agree with), then int() does not mean "the int part of" and | should not take a float argument. The 'integer (int) part' of a float/rational/real is established thru decades of usage. Your consequent is false and in no way follows from your antecendent. tjr From tjreedy at udel.edu Sun Jan 27 03:25:33 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 26 Jan 2008 21:25:33 -0500 Subject: [Python-Dev] Organization of ABC modules References: <20080125184813.AFS85141@ms19.lnh.mail.rcn.net> <479AC5C5.9040509@gmail.com><5d44f72f0801252138i4cf57786k7faf75a523fe71d6@mail.gmail.com><479ACB3C.3090404@gmail.com><00ef01c85fe7$07cc1b00$6800a8c0@RaymondLaptop1> <479B4B52.1060301@gmail.com> Message-ID: "Nick Coghlan" wrote in message news:479B4B52.1060301 at gmail.com... | Raymond Hettinger wrote: | > A prefix would be better. | | I initially thought that, but found the suffix to be the least annoying | of the ideas I had for denoting abstract base classes. To try and give || INumber || ABCNumber | AbcNumber | [etc] | After those 3, I couldn't think of any other prefixes that were both | short and likely to be an effective mnemonic for identifying ABC's - | instead of "Abstract base class", etc. ... | NumberABC I *think* I would prefer to any of these either ANumber or aNumber, which one can read as either an abbreviation of Abstract Number or simply a contraction of 'a Number' (a Real, an Integral, etc) taken to mean the abstraction. | any convention that gets adopted is going to be by Guido's fiat and will | need to suit his aesthetic sense rather than mine :) One more possibility for him to consider. tjr From guido at python.org Sun Jan 27 06:18:23 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 26 Jan 2008 21:18:23 -0800 Subject: [Python-Dev] trunc() In-Reply-To: <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> Message-ID: On Jan 26, 2008 2:53 PM, Raymond Hettinger wrote: [Jeffrey] > > I've been > > advocating trunc() under the assumption that int(float) would be > > deprecated and eliminated as soon as practical > > And how much code would break for basically zero benefit? We'll see. Jeffrey did say "deprecated" and "as soon as practical". A -3 warning in 2.6 could do wonders. > This position is totally nuts. Hold it right there. You may disagree, but that doesn't make it nuts. There is actually quite an important signal to the reader that is present when you see trunc(x) but absent when you see int(x): with trunc(x), the implication is that x is a (Real) number. With int(x), you can make no such assumption -- x could be a string, or it could be a totally different type that happens to define __int__, perhaps a custom date/time type (I believe mxDateTime supports this). > There is *nothing* wrong with int() as it stands now. Nobody has > problems with it. The tools is ubiquitous in other languages, Talk to C++ experts. They have a huge beef with C's confusing use of casts for two different purposes: reinterpretation of the same bit pattern vs. value conversion. Python problem isn't quite the same, but I still really wish I had followed Pascal's lead instead of C's here: Pascal requires you to use trunc() to convert a real to an integer. (BTW Pascal also had the division operator right, unlike C, and we're finally fixing this in Py3k by following Pascal's nearly-40-year-old lead.) If we had done it that way, we wouldn't have had to introduce the index() builtin and the corresponding infrastructure (__index__ and a whole slew of C APIs). > spreadsheets, and calculators. I don't think that Excel should be held up as a shining example for Python. At least, that would be quite a change. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Sun Jan 27 08:14:15 2008 From: python at rcn.com (Raymond Hettinger) Date: Sat, 26 Jan 2008 23:14:15 -0800 Subject: [Python-Dev] trunc() References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> Message-ID: <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> >. You may disagree, but that doesn't make it nuts. Too many thoughts compressed into one adjective ;-) Deprecating int(float)-->int may not be nuts, but it is disruptive. Having both trunc() and int() in Py2.6 may not be nuts, but it is duplicative and confusing. The original impetus for facilitating a new Real type being able to trunc() into a new Integral type may not be nuts, but the use case seems far fetched (we're never had a feature request for it -- the notion was born entirely out of numeric tower considerations). The idea that programmers are confused by int(3.7)-->3 may not be nuts, but it doesn't match any experience I've had with any programmer, ever. The idea that trunc() is beneficial may not be nuts, but it is certainly questionable. In short, the idea may not be nuts, but I think it is legitimate to suggest that it is unnecessary and that it will do more harm than good. > I don't think that Excel should be held up as a shining example for > Python. It is simply a datapoint. We don't care why they chose int() for truncation. The important thing is the subsequent user experiences which indicates that people do not have a problem with it. When it comes to int() and round(), people just get it. I've taught spreadsheets to a lot people and no one has ever construed int() as meaning round(). What was your experience with 18 years of python? Was it a recurring a point of confusion? On the newsgroup, helplist, and tutorial list, I never seen this come up as problem. I looked in other places for corroboration. My HP-12C has intg and frac buttons. In BASIC, int() means to return the integer part of a floating point number. The surprise find was that int() in Matlab and Maple means integrate :-) FWIW, Perl's int() does what ours currently does. One other thought: In Python, it has been nice that we have simple type coercions using the type name: * set(p)-->q can accept a list, tuple, string, or any iterable and make a set * int(p)-->q can accept an int, long, float, or string and make an int * float(p)-->q can accept an int, long, float, or string and make an int * list(p)-->q can accept a list, tuple, string, or any iterable and make a list * unicode(p)--> can accept a str, buffer, or unicode object and make a unicode object It's a bit weird to decide that int() needs to lose that capability so we get generalized Integrals as output. What's wrong with coercion to a concrete type? Raymond From jyasskin at gmail.com Sun Jan 27 09:19:39 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Sun, 27 Jan 2008 00:19:39 -0800 Subject: [Python-Dev] trunc() In-Reply-To: <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> Message-ID: <5d44f72f0801270019od852151l494f8f329db03861@mail.gmail.com> On Jan 26, 2008 11:14 PM, Raymond Hettinger wrote: > The idea that programmers are confused by int(3.7)-->3 may not be nuts, but it doesn't match any experience I've had with any > programmer, ever. In C, I'm pretty sure I've seen people write (long)x where they'd have been better off with lround(x). They know that the cast truncates, and generally that they actually wanted to round, but the type they wanted to get to was foremost in their mind, so they didn't bother to think about it a little and write what they really wanted. It's not that they're confused; it's that they're accepting a default that shouldn't be a default. Your other points seem to have been answered already, although people will disagree on how compelling the answers are, so I won't repeat them here. -- Namast?, Jeffrey Yasskin From asmodai at in-nomine.org Sun Jan 27 09:29:36 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Sun, 27 Jan 2008 09:29:36 +0100 Subject: [Python-Dev] Organization of ABC modules In-Reply-To: References: <20080125184813.AFS85141@ms19.lnh.mail.rcn.net> <479B4B52.1060301@gmail.com> Message-ID: <20080127082936.GC55130@nexus.in-nomine.org> -On [20080127 03:25], Terry Reedy (tjreedy at udel.edu) wrote: >I *think* I would prefer to any of these either >ANumber or >aNumber, >which one can read as either an abbreviation of Abstract Number or simply a >contraction of 'a Number' (a Real, an Integral, etc) taken to mean the >abstraction. This will be a bikeshed argument until Guido speaks out his preference/decision I guess. But isn't it a more common solution to name the base class just Number and derive from it by means of using Base.Number or something similar? Looks cleaner to me rather than all these odd looking pre- or suffixes. (I am not charmed about ABC in the name at all to be honest, doesn't really give me a Python feeling.) -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ We have met the enemy and they are ours... From lists at cheimes.de Sun Jan 27 14:51:55 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 27 Jan 2008 14:51:55 +0100 Subject: [Python-Dev] trunc() In-Reply-To: <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> Message-ID: <479C8C7B.2050802@cheimes.de> Raymond Hettinger wrote: > The idea that programmers are confused by int(3.7)-->3 may not be nuts, but it doesn't match any experience I've had with any > programmer, ever. You haven't been doing newbie support in #python lately. Statements like Python is rounding wrong are common. ;) It's astonishing how many major rounding and division bugs are found by newbies. Christian From lists at cheimes.de Sun Jan 27 14:51:55 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 27 Jan 2008 14:51:55 +0100 Subject: [Python-Dev] trunc() In-Reply-To: <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> Message-ID: <479C8C7B.2050802@cheimes.de> Raymond Hettinger wrote: > The idea that programmers are confused by int(3.7)-->3 may not be nuts, but it doesn't match any experience I've had with any > programmer, ever. You haven't been doing newbie support in #python lately. Statements like Python is rounding wrong are common. ;) It's astonishing how many major rounding and division bugs are found by newbies. Christian From guido at python.org Sun Jan 27 17:43:14 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 27 Jan 2008 08:43:14 -0800 Subject: [Python-Dev] trunc() In-Reply-To: <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> Message-ID: On Jan 26, 2008 11:14 PM, Raymond Hettinger wrote: > >. You may disagree, but that doesn't make it nuts. > > Too many thoughts compressed into one adjective ;-) [snip] > > I don't think that Excel should be held up as a shining example for > > Python. > > It is simply a datapoint. [snip] You're beginning to repeat your argument; none of that was new. > One other thought: In Python, it has been nice that we have simple type coercions using the type name: > * set(p)-->q can accept a list, tuple, string, or any iterable and make a set > * int(p)-->q can accept an int, long, float, or string and make an int > * float(p)-->q can accept an int, long, float, or string and make an int > * list(p)-->q can accept a list, tuple, string, or any iterable and make a list > * unicode(p)--> can accept a str, buffer, or unicode object and make a unicode object > It's a bit weird to decide that int() needs to lose that capability so we get generalized Integrals as output. What's wrong with > coercion to a concrete type? Let me get back to you on that. I first want to point out that you snipped the core of my argument in the post you otherwise quoted. I will repeat it here because I would like your explicit reaction: [Guido] > There is actually quite an important signal to the reader that is > present when you see trunc(x) but absent when you see int(x): with > trunc(x), the implication is that x is a (Real) number. With int(x), > you can make no such assumption -- x could be a string, or it could be > a totally different type that happens to define __int__, perhaps a > custom date/time type (I believe mxDateTime supports this). Can I assume that you agree with this? That would be progress. Coming back to your argument that other types have a constructor that takes all kinds of arguments, I'd like to point out that they all have restrictions too (except for str()): list() and tuple() only accept iterables, float() only accepts strings and certain numbers (not complex), and so on. Taking the latter as an example: >>> float(0j) Traceback (most recent call last): File "", line 1, in TypeError: can't convert complex to float; use abs(z) >>> I think that (eventually) int(0.0) could do something similar: >>> int(0.0) Traceback (most recent call last): File "", line 1, in TypeError: can't convert float to int; use round(x) or trunc(x) >>> But see below. Yet another (minor) argument that has always made me uncomfortable with int() == trunc(): the % operator. I think it's a big improvement over C that Python's % operator is defined as x%y == x - y*floor(x/y) # where / is real division rather than C's division, which uses trunc() instead of floor(). In C this nicely meshes with the definition of int(): you can define x%y as x - y*(int)(x/y); but not so in Python. I don't want to use this as an argument for defining int(x) as floor(x), but I do want to point out that it has always given me a twinge of discomfort. FInally, there's the "one way" argument. That's a nice slogan, but doesn't really hold anyways in practice. To copy a list, we can write either L[:] or list(L). To get the keys of a dict, we can write either D.keys() or list(D). To convert a number to a string we can write either "%g" % X or str(X). For octal we can write "%#o" % X or oct(X). To convert a unicode string to UTF-8, we can write either U.encode("utf8") or str(U, "utf8"). And so on. In many cases, these notations aren't exactly the same in semantics (e.g. list(X) always returns a list, X[:] returns whatever sequence type X is), but nevertheless they have large areas of overlap. This is how I see trunc() and int() live together. After all that, here's my current proposal: - Deprecating int() is pretty radical, I think it would have to happen in the distant future. OR not at all. I'm at best +0 on this, more like exactly 0. I realize that in practice this kills the idea. The "purist" argument for it would have worked better if it was made 18 years ago. - trunc(), round(), floor() and ceil() should all be built-ins, corresponding to __trunc__, __round__, __floor__ and __ceil__. Then we have the four standard ways to go from Reals to Integers, which are properly extensible for folks who write their own number types. (We can't control how folks implement __round__, but we can document expected behavior -- that's how we treat __add__ and all other operators too, after all.) - In the docs (especially for beginners) we recommend that users write trunc() instead of int(), emphasizing that trunc() ensures the argument is a number, while suggesting int(x) for conversion from strings. But the implementation won't chastise users by issuing annoying warnings. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sun Jan 27 17:53:55 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 27 Jan 2008 08:53:55 -0800 Subject: [Python-Dev] Organization of ABC modules In-Reply-To: <20080127082936.GC55130@nexus.in-nomine.org> References: <20080125184813.AFS85141@ms19.lnh.mail.rcn.net> <479B4B52.1060301@gmail.com> <20080127082936.GC55130@nexus.in-nomine.org> Message-ID: On Jan 27, 2008 12:29 AM, Jeroen Ruigrok van der Werven wrote: > This will be a bikeshed argument until Guido speaks out his > preference/decision I guess. > > But isn't it a more common solution to name the base class just Number and > derive from it by means of using Base.Number or something similar? Looks > cleaner to me rather than all these odd looking pre- or suffixes. (I am not > charmed about ABC in the name at all to be honest, doesn't really give me a > Python feeling.) My preference is still *not* to use a naming convention. I just suggested it as a lesser evil compared to segregating all abstract base classes in an "abc" package ghetto. I really don't see why names like "Number" or "MutableSequence" would need any additional help to make the reader see they are abstract. I note that at least for built-in types there will be the naming convention that concrete implementation classes are all lowercase, like int, float, list, namedtuple, defaultdict, and so on, while the ABCs all have a Capitalized[Words] name: Hashable, Number, Real, MutableMapping, etc. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From agriff at tin.it Sun Jan 27 18:26:01 2008 From: agriff at tin.it (Andrea Griffini) Date: Sun, 27 Jan 2008 18:26:01 +0100 Subject: [Python-Dev] trunc() In-Reply-To: References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> Message-ID: On Jan 27, 2008 5:43 PM, Guido van Rossum wrote: > - Deprecating int() is pretty radical, I think it would have to > happen in the distant future. OR not at all. I'm at best +0 on this, > more like exactly 0. I realize that in practice this kills the idea. > The "purist" argument for it would have worked better if it was made > 18 years ago. Also what happens with "%i" % 3.14 ? We incidentally found a problem with a script using python 2.5 because apparently the "%" formatting operator doesn't use "int()" for doing the conversion (to be more specific if the float is too large for a 32-bit integer then the format operator chokes while the int() operator returns a long). Anyway I want just to say that if "implicit" conversion from float to integer goes away then what happens to formatting conversion ? Removing that too IMO would break a lot of code and it's IMO very difficult to help fixing that. Andrea From python at rcn.com Sun Jan 27 18:30:16 2008 From: python at rcn.com (Raymond Hettinger) Date: Sun, 27 Jan 2008 09:30:16 -0800 Subject: [Python-Dev] trunc() Message-ID: <001801c8610a$4891efb0$6800a8c0@RaymondLaptop1> [Raymond Hettinger] >> The idea that programmers are confused by int(3.7)-->3 may not be nuts, but it doesn't match any experience I've had with any >> programmer, ever. [Christian Heimes] > You haven't been doing newbie support in #python lately. Statements like > Python is rounding wrong are common. Of course, those floating point repr questions are common, but that wasn't the issue here. The trunc() protagonists are contending that the int(f) function will be confused with the round(f, n) function. I've never seen than come up, ever. Raymond From jyasskin at gmail.com Sun Jan 27 18:43:39 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Sun, 27 Jan 2008 09:43:39 -0800 Subject: [Python-Dev] trunc() In-Reply-To: References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> Message-ID: <5d44f72f0801270943n4e587229rafb407fb686e7be9@mail.gmail.com> On Jan 27, 2008 9:26 AM, Andrea Griffini wrote: > On Jan 27, 2008 5:43 PM, Guido van Rossum wrote: > > > - Deprecating int() is pretty radical, I think it would have to > > happen in the distant future. OR not at all. I'm at best +0 on this, > > more like exactly 0. I realize that in practice this kills the idea. > > The "purist" argument for it would have worked better if it was made > > 18 years ago. > > Also what happens with "%i" % 3.14 ? We incidentally found a problem > with a script using python 2.5 because apparently the "%" formatting > operator doesn't use "int()" for doing the conversion (to be more specific > if the float is too large for a 32-bit integer then the format operator chokes > while the int() operator returns a long). > Anyway I want just to say that if "implicit" conversion from float > to integer goes away then what happens to formatting conversion ? > Removing that too IMO would break a lot of code and it's IMO very > difficult to help fixing that. Well, it seems like it would be as easy to make some formatting conversions raise a warning on float inputs as it would be to make int() do it. But I think you're safe here; it doesn't look like either will be deprecated. -- Namast?, Jeffrey Yasskin http://jeffrey.yasskin.info/ From python at rcn.com Sun Jan 27 19:29:25 2008 From: python at rcn.com (Raymond Hettinger) Date: Sun, 27 Jan 2008 10:29:25 -0800 Subject: [Python-Dev] trunc() References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> Message-ID: <002901c86112$8c3ca770$6800a8c0@RaymondLaptop1> > [Guido] >> There is actually quite an important signal to the reader that is >> present when you see trunc(x) but absent when you see int(x): with >> trunc(x), the implication is that x is a (Real) number. With int(x), >> you can make no such assumption -- x could be a string, or it could be >> a totally different type that happens to define __int__, perhaps a >> custom date/time type (I believe mxDateTime supports this). > > Can I assume that you agree with this? That would be progress. I agree that it provides a cue that that input is Real. I don't agree that that has any value. We had a lot of tools that can accept multiple input types. For instance, float() can accept a string or number. We could do like C and split-out the atof() functionality but that just makes two functions where one would suffice. > Yet another (minor) argument that has always made me uncomfortable > with int() == trunc(): the % operator. I think it's a big improvement > over C that Python's % operator is defined as > > x%y == x - y*floor(x/y) # where / is real division > > rather than C's division, which uses trunc() instead of floor(). In C > this nicely meshes with the definition of int(): you can define x%y as > x - y*(int)(x/y); but not so in Python. I don't want to use this as an > argument for defining int(x) as floor(x), but I do want to point out > that it has always given me a twinge of discomfort. It hasn't bugged me much, but I do understand. > After all that, here's my current proposal: > > - Deprecating int() is pretty radical, I think it would have to > happen in the distant future. OR not at all. I'm at best +0 on this, > more like exactly 0. I realize that in practice this kills the idea. > The "purist" argument for it would have worked better if it was made > 18 years ago. "pretty radical" is what I meant when I said "it's nuts" ;-) This is a major piece of progress. Most of the prior proposal was predicated on int() being deprecated. > - trunc(), round(), floor() and ceil() should all be built-ins, > corresponding to __trunc__, __round__, __floor__ and __ceil__. Then we > have the four standard ways to go from Reals to Integers, which are > properly extensible for folks who write their own number types. (We > can't control how folks implement __round__, but we can document > expected behavior -- that's how we treat __add__ and all other > operators too, after all.) ISTM, you really don't need trunc() in this mix. The ceil() and floor() functions do an excellent job of convering most use cases. The overlap of trunc() and int() just isn't worth it. There's the mental cost of having to explain the difference on day one to every beginner. There's the back end cost of every class that has __int__ also needing to decide whether to provide __trunc__ which will typically be the same thing. In time, I believe it will become self-evident that having both int() and trunc() is a wart. If trunc() goes into 2.6 and 3.0, then we're going to have to live with this for a long time. Purity may suggest that you need trunc(). Practicality says it isn't worth the cost. At least we've made important progress by saving int(). Raymond From guido at python.org Sun Jan 27 19:33:06 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 27 Jan 2008 10:33:06 -0800 Subject: [Python-Dev] trunc() In-Reply-To: References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> Message-ID: On Jan 27, 2008 9:26 AM, Andrea Griffini wrote: > On Jan 27, 2008 5:43 PM, Guido van Rossum wrote: > > > - Deprecating int() is pretty radical, I think it would have to > > happen in the distant future. OR not at all. I'm at best +0 on this, > > more like exactly 0. I realize that in practice this kills the idea. > > The "purist" argument for it would have worked better if it was made > > 18 years ago. > > Also what happens with "%i" % 3.14 ? We incidentally found a problem > with a script using python 2.5 because apparently the "%" formatting > operator doesn't use "int()" for doing the conversion (to be more specific > if the float is too large for a 32-bit integer then the format operator chokes > while the int() operator returns a long). That's quite a separate issue. Please ses http://bugs.python.org/issue1742669. > Anyway I want just to say that if "implicit" conversion from float > to integer goes away then what happens to formatting conversion ? > Removing that too IMO would break a lot of code and it's IMO very > difficult to help fixing that. The formatting code could assign specific meanings. I suspect though that it was never meant to be possible to use %d with a float -- it just is one of the artifacts of using implicit conversion, and one not well-thought through. Note: >>> "%.0f" % 3.9999999999 '4' >>> "%d" % 3.9999999999 '3' >>> I think the latter is wrong and confusing. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From murman at gmail.com Sun Jan 27 19:39:35 2008 From: murman at gmail.com (Michael Urman) Date: Sun, 27 Jan 2008 12:39:35 -0600 Subject: [Python-Dev] trunc() In-Reply-To: References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> Message-ID: Is this a valid summary of the arguments so far? I see two arguments for the change: 1) The semantics of trunc() are clear: it maps R -> Z in a specific fashion 2) The semantics of int() are fuzzy; even non-numeric types (strings) are handled Yet there will be a __trunc__ that will allow any chosen mapping to be implemented, so long as it results in an integer, so (1) is only guaranteed true for the builtin types. This leaves us with (2) which seems strongly tied to string parsing (as __index__ resolved the other common X -> integer case). I see one main argument against: *) trunc() results in duplication at best, zealous deprecation at worst Given that the deprecation or removal of int(2.3) has been dropped, the argument is about pointless duplication. What problem is trunc() supposed to solve if it does to floats what int() does now? I've done some initial code searches for: lang:python "int(", and I've seen three primary uses for calling int(x): a) parsing strings, such as from a user, a config file, or other serialized format b) limiting the input to a function to an integer, such as in a calendar trying to ensure it has integer months c) truncation, such as throwing away sub-seconds from time.time(), or ensuring integer results from division It's unclear to me whether (b) could be better served by more type-specific operations that would prevent passing in strings, or whether uses like (c) often have latent bugs due to truncation instead of rounding. If trunc() is to clarify round vs. integer-portion, it's something people learn -- the general lack of comments in (c) usages indicates nobody considers it special behavior. If it's to clarify the argument's type (the parsing of strings vs. getting integers from other numeric types), would separating parsing from the int (and float) constructors also solve this? Is the aim to "clean up" the following fake example? (Real world uses of map(int, ...) seem almost uniformly related to string parsing.) >>> map(int, ("42", 6.022, 2**32)) [42, 6, 4294967296L] -- Michael Urman From guido at python.org Sun Jan 27 19:43:23 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 27 Jan 2008 10:43:23 -0800 Subject: [Python-Dev] trunc() In-Reply-To: <002901c86112$8c3ca770$6800a8c0@RaymondLaptop1> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> <002901c86112$8c3ca770$6800a8c0@RaymondLaptop1> Message-ID: On Jan 27, 2008 10:29 AM, Raymond Hettinger wrote: > > [Guido] > >> There is actually quite an important signal to the reader that is > >> present when you see trunc(x) but absent when you see int(x): with > >> trunc(x), the implication is that x is a (Real) number. With int(x), > >> you can make no such assumption -- x could be a string, or it could be > >> a totally different type that happens to define __int__, perhaps a > >> custom date/time type (I believe mxDateTime supports this). > > > > Can I assume that you agree with this? That would be progress. > > I agree that it provides a cue that that input is Real. > I don't agree that that has any value. We had a lot of tools that > can accept multiple input types. For instance, float() can accept > a string or number. We could do like C and split-out the atof() > functionality but that just makes two functions where one would suffice. All I claim is that it has *some* value for me. If it doesn't have value for you, that doesn't make it worthless. > > Yet another (minor) argument that has always made me uncomfortable > > with int() == trunc(): the % operator. I think it's a big improvement > > over C that Python's % operator is defined as > > > > x%y == x - y*floor(x/y) # where / is real division > > > > rather than C's division, which uses trunc() instead of floor(). In C > > this nicely meshes with the definition of int(): you can define x%y as > > x - y*(int)(x/y); but not so in Python. I don't want to use this as an > > argument for defining int(x) as floor(x), but I do want to point out > > that it has always given me a twinge of discomfort. > > It hasn't bugged me much, but I do understand. Progress. > > After all that, here's my current proposal: > > > > - Deprecating int() is pretty radical, I think it would have to > > happen in the distant future. OR not at all. I'm at best +0 on this, > > more like exactly 0. I realize that in practice this kills the idea. > > The "purist" argument for it would have worked better if it was made > > 18 years ago. > > "pretty radical" is what I meant when I said "it's nuts" ;-) > > This is a major piece of progress. Most of the prior proposal was > predicated on int() being deprecated. Good. > > - trunc(), round(), floor() and ceil() should all be built-ins, > > corresponding to __trunc__, __round__, __floor__ and __ceil__. Then we > > have the four standard ways to go from Reals to Integers, which are > > properly extensible for folks who write their own number types. (We > > can't control how folks implement __round__, but we can document > > expected behavior -- that's how we treat __add__ and all other > > operators too, after all.) > > ISTM, you really don't need trunc() in this mix. The ceil() and floor() > functions do an excellent job of convering most use cases. Not really. You'd have to use an if predicated on the sign to emulate trunc() using ceil() and floor(). Or something like sign(x) * floor(abs(x)) -- but we don't have sign(). > The overlap of trunc() and int() just isn't worth it. There's the mental > cost of having to explain the difference on day one to every beginner. > There's the back end cost of every class that has __int__ also needing > to decide whether to provide __trunc__ which will typically be the > same thing. > > In time, I believe it will become self-evident that having both int() > and trunc() is a wart. If trunc() goes into 2.6 and 3.0, then we're > going to have to live with this for a long time. > > Purity may suggest that you need trunc(). Practicality says it isn't > worth the cost. > > At least we've made important progress by saving int(). I don't see much cost for end users at all. It's not like trunc() represents a difficult concept. Having trunc() in addition to ceil(), floor() and round() creates a nicely complete set of float->int conversions. Then int() can be defined by deferring to trunc() -- as opposed to round(). My proposal stands (less any talk of deprecation of int()). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Sun Jan 27 19:54:10 2008 From: python at rcn.com (Raymond Hettinger) Date: Sun, 27 Jan 2008 10:54:10 -0800 Subject: [Python-Dev] trunc() References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> <002901c86112$8c3ca770$6800a8c0@RaymondLaptop1> Message-ID: <003601c86116$010f16c0$6800a8c0@RaymondLaptop1> > Then int() can be defined by deferring to trunc() > -- as opposed to round(). That part is new and represents some progress. If I understand it correctly, it means that we won't have both __int__ and __trunc__ magic methods. That's a good thing. Raymond From guido at python.org Sun Jan 27 19:53:43 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 27 Jan 2008 10:53:43 -0800 Subject: [Python-Dev] trunc() In-Reply-To: References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> Message-ID: On Jan 27, 2008 10:39 AM, Michael Urman wrote: > Is this a valid summary of the arguments so far? > > I see two arguments for the change: > > 1) The semantics of trunc() are clear: it maps R -> Z in a specific fashion > 2) The semantics of int() are fuzzy; even non-numeric types > (strings) are handled > > Yet there will be a __trunc__ that will allow any chosen mapping to be > implemented, so long as it results in an integer, so (1) is only > guaranteed true for the builtin types. We can easily add docs to the Real ABC indicating that __trunc__ *should* implement a certain semantics, just like we do (or should do) for __add__ and everything else. While this doesn't provide a hard guarantee in the presence of non-conforming implementations, it's as good as it gets anywhere in Python. Given that __int__ may be implemented for things that aren't reals (like dates), it's much harder to prescribe what it *should* do. > This leaves us with (2) which > seems strongly tied to string parsing (as __index__ resolved the other > common X -> integer case). > > I see one main argument against: > > *) trunc() results in duplication at best, zealous deprecation at worst > > Given that the deprecation or removal of int(2.3) has been dropped, > the argument is about pointless duplication. To some it's pointless. To others there's a fine point to it. > What problem is trunc() supposed to solve if it does to floats what > int() does now? I've done some initial code searches for: lang:python > "int(", and I've seen three primary uses for calling int(x): > > a) parsing strings, such as from a user, a config file, or other > serialized format > b) limiting the input to a function to an integer, such as in a > calendar trying to ensure it has integer months > c) truncation, such as throwing away sub-seconds from time.time(), > or ensuring integer results from division > > It's unclear to me whether (b) could be better served by more > type-specific operations that would prevent passing in strings, or > whether uses like (c) often have latent bugs due to truncation instead > of rounding. Case (b) should be using index() instead. Most likely the code either predates index() or needs to be compatible with Python versions that don't have it, or the programmer wasn't aware of index(), which hasn't received a lot of publicity. > If trunc() is to clarify round vs. integer-portion, it's something > people learn -- the general lack of comments in (c) usages indicates > nobody considers it special behavior. If it's to clarify the > argument's type (the parsing of strings vs. getting integers from > other numeric types), would separating parsing from the int (and > float) constructors also solve this? But there's a long-standing tradition that all numeric types in Python accept a string as argument. This was just added to decimal too. > Is the aim to "clean up" the following fake example? (Real world uses > of map(int, ...) seem almost uniformly related to string parsing.) > > >>> map(int, ("42", 6.022, 2**32)) > [42, 6, 4294967296L] That's an artificial example and hence it is impossible to derive the intent of the programmer. Heterogeneous lists are pretty rare. Let me give another artificial example. Suppose I have a need to implement floats and ints that print themselves in hex. I can make it so that this property is maintained across addition etc. without having to change the code that *uses* these numbers. But code that uses int() to convert a float to an int will lose the property. If that code used trunc() instead I can provide a __trunc__ on my hexfloat that returns a hexint. QED. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Sun Jan 27 19:54:01 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 27 Jan 2008 19:54:01 +0100 Subject: [Python-Dev] Upcoming 2.5.2 release Message-ID: <479CD349.6060604@v.loewis.de> It is my pleasure to announce that I have been appointed as the release manager for the upcoming releases. For 2.5.2, I would like to release a candidate on February 14, and the final version on February 21. As Guido already mentioned, this will be a bug fix release only; no new features allowed. Kind regards, Martin From guido at python.org Sun Jan 27 19:54:56 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 27 Jan 2008 10:54:56 -0800 Subject: [Python-Dev] trunc() In-Reply-To: <003601c86116$010f16c0$6800a8c0@RaymondLaptop1> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> <002901c86112$8c3ca770$6800a8c0@RaymondLaptop1> <003601c86116$010f16c0$6800a8c0@RaymondLaptop1> Message-ID: On Jan 27, 2008 10:54 AM, Raymond Hettinger wrote: > > Then int() can be defined by deferring to trunc() > > -- as opposed to round(). > > That part is new and represents some progress. If I understand it > correctly, it means that we won't have both __int__ and __trunc__ > magic methods. That's a good thing. A single type wouln't need both. But int() should still try both, because it doesn't make sense for e.g. a date type to have to define __trunc__ for conversion to an int. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From daniel at stutzbachenterprises.com Sun Jan 27 19:55:26 2008 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Sun, 27 Jan 2008 12:55:26 -0600 Subject: [Python-Dev] trunc() In-Reply-To: References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> Message-ID: On Jan 27, 2008 10:43 AM, Guido van Rossum wrote: > - trunc(), round(), floor() and ceil() should all be built-ins, > corresponding to __trunc__, __round__, __floor__ and __ceil__. Then we > have the four standard ways to go from Reals to Integers, which are > properly extensible for folks who write their own number types. (We > can't control how folks implement __round__, but we can document > expected behavior -- that's how we treat __add__ and all other > operators too, after all.) If I'm following this discussion properly, the advantage of trunc() is that a Real-class-that-isn't-float can define a __trunc__ that can return an Integer-class-that-isn't-int, right? In that case, why not have the Real ABC grow trunc(), ceil(), floor(), and round() methods (replacing the __ varieties), and get rid of the builtins/math-module functions? x.trunc() is just as clear as trunc(x), and doesn't require a builtin. The syntax when used on float literals is ugly ("2.56 .round()"), but there's no use case for these methods on literals (just write "3"). 2to3 could handle this conversion pretty easily in most cases, so there won't be much breakage. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From eric+python-dev at trueblade.com Sun Jan 27 20:21:23 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Sun, 27 Jan 2008 14:21:23 -0500 Subject: [Python-Dev] trunc() In-Reply-To: References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> Message-ID: <479CD9B3.4020106@trueblade.com> Guido van Rossum wrote: > On Jan 27, 2008 9:26 AM, Andrea Griffini wrote: >> Anyway I want just to say that if "implicit" conversion from float >> to integer goes away then what happens to formatting conversion ? >> Removing that too IMO would break a lot of code and it's IMO very >> difficult to help fixing that. > > The formatting code could assign specific meanings. I suspect though > that it was never meant to be possible to use %d with a float -- it > just is one of the artifacts of using implicit conversion, and one not > well-thought through. Note: > >>>> "%.0f" % 3.9999999999 > '4' >>>> "%d" % 3.9999999999 > '3' > > I think the latter is wrong and confusing. format() has the same issue, for the same reason: >>> format(3.9, 'f') '3.900000' >>> format(3.9, 'd') '3' I never noticed this before, and it's a definite error. PEP 3101 says the valid types for float are 'eEfFgGn%' and an empty string. I'll remove the integer conversion in the float formatter. Eric. From martin at v.loewis.de Sun Jan 27 20:27:27 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 27 Jan 2008 20:27:27 +0100 Subject: [Python-Dev] Upcoming 2.4.5 and 2.3.7 releases Message-ID: <479CDB1F.60809@v.loewis.de> Along with the release of 2.5.2, I would also like to release new versions of 2.3 and 2.4. These will be security-only releases, and include a few security-relevant bug fixes that are still being finalized. As we don't have the infrastructure to produce full releases of 2.3 or 2.4 anymore, this will be a source release only. As such, it will likely see less testing than other releases, and users will have more difficulties in obtaining the software for their system - the releases will be targeted primarily at system vendors who can chose to include them as security patches in their system updates. As a consequence, I would like to roll back all changes from the 2.3 and 2.4 branches which aren't security fixes. In specific cases, the nature of a change might be debatable; clear security fixes are prevention of memory corruption and interpreter crashes, clear non-security fixes are documentation and test-suite changes. For 2.3, there are only few revisions that would be rolled back: r52798, r52803, r52824, r54342. For 2.4, the list is longer; all changes on the branch since r52382 are candidate for roll-back. I would like to prepare a white-list of patches that should be preserved; if you think any of the patches committed in the 2.4 branch is a security fix, please let me know. Regards, Martin From martin at v.loewis.de Sun Jan 27 20:54:02 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 27 Jan 2008 20:54:02 +0100 Subject: [Python-Dev] trunc() In-Reply-To: References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> Message-ID: <479CE15A.2010106@v.loewis.de> > If I'm following this discussion properly, the advantage of trunc() is > that a Real-class-that-isn't-float can define a __trunc__ that can > return an Integer-class-that-isn't-int, right? Depends on what you compare to. Compared to int(), the advantage is that trunc() sends a clear message what its semantics is. > In that case, why not have the Real ABC grow trunc(), ceil(), floor(), > and round() methods (replacing the __ varieties), and get rid of the > builtins/math-module functions? > > x.trunc() is just as clear as trunc(x), and doesn't require a builtin. +1. Students just asked me why len() is not a method, and I didn't know a good answer; the same holds for many other builtins. This is a clear candidate for a method, IMO. > The syntax when used on float literals is ugly ("2.56 .round()"), but > there's no use case for these methods on literals (just write "3"). Actually, it works fine for float literals: py> 2.45.round() Traceback (most recent call last): File "", line 1, in AttributeError: 'float' object has no attribute 'round' It's only int literals which have a problem with methods, but then, you won't call any of these on an int literal, normally. Regards, Martin From pje at telecommunity.com Sun Jan 27 21:02:01 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 27 Jan 2008 15:02:01 -0500 Subject: [Python-Dev] refleaks and caches In-Reply-To: References: Message-ID: <20080127200156.0F6413A4065@sparrow.telecommunity.com> At 05:05 PM 1/26/2008 -0800, Neal Norwitz wrote: >Around Jan 13, the refleak hunting test that is reported on >python-checkins started to report refleaks on virtually every run. I >suspect this is due to r59944 (at 2008-01-13 16:29:41) which was from >patch #1700288 to cache methods. With this patch it makes it much >harder to spot refleaks. Does anyone have ideas how to fix it? The >only one I have is to disable the cache with a special flag, env't >variable, sys variable/function or the like. We could make this >available only in debug mode. The cache should normally be enabled, >but could be disabled solely on the refleak runs. > >Suggestions? Expose an API to clear the cache, and clear it at shutdown? It should probably be part of interpreter shutdown anyway. From python at rcn.com Sun Jan 27 21:12:43 2008 From: python at rcn.com (Raymond Hettinger) Date: Sun, 27 Jan 2008 12:12:43 -0800 Subject: [Python-Dev] trunc() References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> <002901c86112$8c3ca770$6800a8c0@RaymondLaptop1> Message-ID: <001101c86120$fa813a80$6800a8c0@RaymondLaptop1> [GvR] >> > - trunc(), round(), floor() and ceil() should all be built-ins, >> > corresponding to __trunc__, __round__, __floor__ and __ceil__. In Py2.6, what will the signatures be? Last I heard, ceil() and floor() were still going to be float-->float. Raymond From aahz at pythoncraft.com Sun Jan 27 21:53:09 2008 From: aahz at pythoncraft.com (Aahz) Date: Sun, 27 Jan 2008 12:53:09 -0800 Subject: [Python-Dev] functions vs methods (was Re: trunc()) In-Reply-To: <479CE15A.2010106@v.loewis.de> References: <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> <479CE15A.2010106@v.loewis.de> Message-ID: <20080127205309.GA17015@panix.com> On Sun, Jan 27, 2008, "Martin v. L?wis" wrote: > > Students just asked me why len() is not a method, and I didn't know a > good answer; the same holds for many other builtins. This is a clear > candidate for a method, IMO. This is why len() is not a method: map(len, list_of_strings) What I don't know is to what extent this argument still holds in the presence of listcomps and genexps: [s.len() for s in list_of_strings] However, you still need ``len`` as a function to pass around as a callback in other cases where you need a generic function because the type of your data is not predetermined. In any event, I consider dropping len() from builtins to be gratuitous breakage, even in 3.0. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "All problems in computer science can be solved by another level of indirection." --Butler Lampson From brett at python.org Sun Jan 27 22:06:05 2008 From: brett at python.org (Brett Cannon) Date: Sun, 27 Jan 2008 13:06:05 -0800 Subject: [Python-Dev] functions vs methods (was Re: trunc()) In-Reply-To: <20080127205309.GA17015@panix.com> References: <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> <479CE15A.2010106@v.loewis.de> <20080127205309.GA17015@panix.com> Message-ID: On Jan 27, 2008 12:53 PM, Aahz wrote: > On Sun, Jan 27, 2008, "Martin v. L?wis" wrote: > > > > Students just asked me why len() is not a method, and I didn't know a > > good answer; the same holds for many other builtins. This is a clear > > candidate for a method, IMO. > > This is why len() is not a method: > > map(len, list_of_strings) > > What I don't know is to what extent this argument still holds in the > presence of listcomps and genexps: > > [s.len() for s in list_of_strings] > > However, you still need ``len`` as a function to pass around as a > callback in other cases where you need a generic function because the > type of your data is not predetermined. > Or you just use ``lambda x: x.len()`` for the callback. And a built-in could easily be created that does what the lambda is doing. I know for me the reason I always thought the built-ins were there on top of using a method naming convention for operator overloading was so that a reasonable default could be provided by the built-in (e.g., iter() being able to work with an object that only provides __getitem__(), on top of its second argument allowing for better iteration control). > In any event, I consider dropping len() from builtins to be gratuitous > breakage, even in 3.0. I don't think anyone is proposing that. -Brett From guido at python.org Sun Jan 27 22:18:10 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 27 Jan 2008 13:18:10 -0800 Subject: [Python-Dev] trunc() In-Reply-To: <001101c86120$fa813a80$6800a8c0@RaymondLaptop1> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> <002901c86112$8c3ca770$6800a8c0@RaymondLaptop1> <001101c86120$fa813a80$6800a8c0@RaymondLaptop1> Message-ID: On Jan 27, 2008 12:12 PM, Raymond Hettinger wrote: > [GvR] > >> > - trunc(), round(), floor() and ceil() should all be built-ins, > >> > corresponding to __trunc__, __round__, __floor__ and __ceil__. > > In Py2.6, what will the signatures be? > Last I heard, ceil() and floor() were still going to be float-->float. When imported from math, yes, for 2.5 compatibility. If we add builtins, the builtins won't have this restriction. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sun Jan 27 22:24:55 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 27 Jan 2008 13:24:55 -0800 Subject: [Python-Dev] trunc() In-Reply-To: <479CE15A.2010106@v.loewis.de> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> <479CE15A.2010106@v.loewis.de> Message-ID: On Jan 27, 2008 11:54 AM, "Martin v. L?wis" wrote: > > If I'm following this discussion properly, the advantage of trunc() is > > that a Real-class-that-isn't-float can define a __trunc__ that can > > return an Integer-class-that-isn't-int, right? > > Depends on what you compare to. Compared to int(), the advantage is > that trunc() sends a clear message what its semantics is. > > > In that case, why not have the Real ABC grow trunc(), ceil(), floor(), > > and round() methods (replacing the __ varieties), and get rid of the > > builtins/math-module functions? > > > > x.trunc() is just as clear as trunc(x), and doesn't require a builtin. > > +1. Students just asked me why len() is not a method, and I didn't know > a good answer; the same holds for many other builtins. This is a clear > candidate for a method, IMO. Well, there's the generic functions line of thought, which isn't quite dead yet. And there's the idea that the built-in function can check that the result has a certain type, like len(), which insists on returning an int. But mostly it's because I find things like len(x), round(x) and cos(x) read more natural than method notation. It builds on a long tradition in math and applied math in programming language -- at least round() and cos() do, and so does trunc(). IOW it's a matter of aesthetics, and will never be explainable to everyone's satisfaction. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sun Jan 27 22:27:49 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 27 Jan 2008 13:27:49 -0800 Subject: [Python-Dev] functions vs methods (was Re: trunc()) In-Reply-To: <20080127205309.GA17015@panix.com> References: <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> <479CE15A.2010106@v.loewis.de> <20080127205309.GA17015@panix.com> Message-ID: On Jan 27, 2008 12:53 PM, Aahz wrote: > This is why len() is not a method: > > map(len, list_of_strings) I disagree with that explanation -- I couldn't know that when I made len() a function, because map() wasn't to become part of the language for another 5-6 years. My explanation is (a) I translated ABC's #x notation, and (b) I like the way it looks better. Your explanation is at best a handy side effect. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From lists at cheimes.de Mon Jan 28 00:37:00 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 28 Jan 2008 00:37:00 +0100 Subject: [Python-Dev] refleaks and caches In-Reply-To: <20080127200156.0F6413A4065@sparrow.telecommunity.com> References: <20080127200156.0F6413A4065@sparrow.telecommunity.com> Message-ID: Phillip J. Eby wrote: > Expose an API to clear the cache, and clear it at shutdown? It > should probably be part of interpreter shutdown anyway. Good point. I've implemented PyType_ClearCache and exposed it via sys._cleartypecache(). The function is called during finalization, too. Can somebody please double check the change? The results are promising and I'm sure I've implemented it correctly but you never know ;) Christian From tjreedy at udel.edu Mon Jan 28 00:37:19 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 27 Jan 2008 18:37:19 -0500 Subject: [Python-Dev] trunc() References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net><5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com><79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com><5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com><005701c8606e$5196b740$6800a8c0@RaymondLaptop1><012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> Message-ID: "Michael Urman" wrote in message news:dcbbbb410801271039s4bf2b669ob919c53e43cb0cea at mail.gmail.com... | 2) The semantics of int() are fuzzy; even non-numeric types | (strings) are handled One could just as well say that the semantics of float() are fuzzy since it also handles strings. The actual claim seems to have been that the semantics of int() itself is fuzzy. This in turn seems to be based one of two ideas (I am not sure which) a. 'int' might either mean to some people 'some int associated with the float input' rather than the more precise 'the integer part of the float input', or b. that some people might think the latter means the same as the former. Possibility a is easy taken care of by documentation, especially when the behavior is what people expect. Possibility b conflicts with what I believe to be both the plain English meaning of 'integer part' and what I believe Americans, at least, learn in elementary school. Moreover, I challenge the notion that 'truncate' is unambiguous. It simply means to cut off, without specifying how much. '3.141' is pi truncated to 3 decimal places. Indeed, it would not be unreasonable to expect trunc() to have a second parameter specifying where to cut. For one data point, I asked my bright 13-year-old for the 'integer part' of 3.1, 3.9, -3.1, and -3.9 and got the expected 3,3,-3,-3 (as with int()). But asking 'truncate' the same numbers or even 'truncate toward zero' got a blank stare, which did not surprise me too much as it is not a common (American) English word. tjr From nnorwitz at gmail.com Mon Jan 28 01:35:35 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 27 Jan 2008 16:35:35 -0800 Subject: [Python-Dev] refleaks and caches In-Reply-To: References: <20080127200156.0F6413A4065@sparrow.telecommunity.com> Message-ID: On Jan 27, 2008 3:37 PM, Christian Heimes wrote: > Phillip J. Eby wrote: > > Expose an API to clear the cache, and clear it at shutdown? It > > should probably be part of interpreter shutdown anyway. > > Good point. I've implemented PyType_ClearCache and exposed it via > sys._cleartypecache(). The function is called during finalization, too. I'm not sure we should expose an API to clear the cache, but I don't have strong opinions either way. If we keep the ability to clear the cache, should we also consider some control over the int/float freelist? These are worse than the tuple/frame free lists since int/floats are unbounded. I suspect the method free lists in Objects/methodobject.c and Objects/classobject.c don't have that many entries that could be removed. There may not be a lot we can do for the int/float caches and I'm not sure if it's worth it. We can't move objects once created, although we could scan the blocks and if there are no live objects in a block, free it. That would presumably help with this case: list(range(some_big_number)) I don't know how important these things are in practice. > Can somebody please double check the change? The results are promising > and I'm sure I've implemented it correctly but you never know ;) The biggest problem I have with the patch is the attribute name. I would prefer underscores. ie _clear_type_cache instead of _cleartypecache. Attributes in sys are currently inconsistent, but it seems that most of the newer names have underscores. (Aside: if we are going to move attrs out of sys for py3k, we should consider renaming them to be consistent too. Regardless of moving them, should we rename them.) n From brett at python.org Mon Jan 28 02:33:05 2008 From: brett at python.org (Brett Cannon) Date: Sun, 27 Jan 2008 17:33:05 -0800 Subject: [Python-Dev] refleaks and caches In-Reply-To: References: <20080127200156.0F6413A4065@sparrow.telecommunity.com> Message-ID: On Jan 27, 2008 3:37 PM, Christian Heimes wrote: > Phillip J. Eby wrote: > > Expose an API to clear the cache, and clear it at shutdown? It > > should probably be part of interpreter shutdown anyway. > > Good point. I've implemented PyType_ClearCache and exposed it via > sys._cleartypecache(). The function is called during finalization, too. > Ignoring whether this is the right thing to do, should this be in sys or in gc? -Brett From gnewsg at gmail.com Mon Jan 28 02:42:53 2008 From: gnewsg at gmail.com (Giampaolo Rodola') Date: Sun, 27 Jan 2008 17:42:53 -0800 (PST) Subject: [Python-Dev] Upcoming 2.4.5 and 2.3.7 releases In-Reply-To: <479CDB1F.60809@v.loewis.de> References: <479CDB1F.60809@v.loewis.de> Message-ID: On Jan 27, 8:27?pm, "Martin v. L?wis" wrote: > Along with the release of 2.5.2, I would also like to release > new versions of 2.3 and 2.4. These will be security-only releases, > and include a few security-relevant bug fixes that are still being > finalized. > > As we don't have the infrastructure to produce full releases of 2.3 > or 2.4 anymore, this will be a source release only. As such, it > will likely see less testing than other releases, and users will have > more difficulties in obtaining the software for their system - the > releases will be targeted primarily at system vendors who can chose > to include them as security patches in their system updates. > > As a consequence, I would like to roll back all changes from the 2.3 > and 2.4 branches which aren't security fixes. In specific cases, the > nature of a change might be debatable; clear security fixes are > prevention of memory corruption and interpreter crashes, clear > non-security fixes are documentation and test-suite changes. > > For 2.3, there are only few revisions that would be rolled back: > r52798, r52803, r52824, r54342. > > For 2.4, the list is longer; all changes on the branch since > r52382 are candidate for roll-back. I would like to prepare > a white-list of patches that should be preserved; if you think > any of the patches committed in the 2.4 branch is a security > fix, please let me know. > > Regards, > Martin Don't know if this concerns 2.3 but there was a debate whether including patch #1745035 in 2.4 branch: http://groups.google.it/group/python-dev2/browse_thread/thread/33cad7b7c1cdb19f?hl=en# From lists at cheimes.de Mon Jan 28 03:08:48 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 28 Jan 2008 03:08:48 +0100 Subject: [Python-Dev] refleaks and caches In-Reply-To: References: <20080127200156.0F6413A4065@sparrow.telecommunity.com> Message-ID: Brett Cannon wrote: > Ignoring whether this is the right thing to do, should this be in sys or in gc? Yeah, good idea. The gc module makes sense. Christian From lists at cheimes.de Mon Jan 28 03:13:32 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 28 Jan 2008 03:13:32 +0100 Subject: [Python-Dev] refleaks and caches In-Reply-To: References: <20080127200156.0F6413A4065@sparrow.telecommunity.com> Message-ID: <479D3A4C.8050205@cheimes.de> Neal Norwitz wrote: > I'm not sure we should expose an API to clear the cache, but I don't > have strong opinions either way. If we keep the ability to clear the > cache, should we also consider some control over the int/float > freelist? These are worse than the tuple/frame free lists since > int/floats are unbounded. I suspect the method free lists in > Objects/methodobject.c and Objects/classobject.c don't have that many > entries that could be removed. Do the int/float free lists cause any trouble or can they eat lots of memory? And what about the string intern list? > The biggest problem I have with the patch is the attribute name. I > would prefer underscores. ie _clear_type_cache instead of > _cleartypecache. Attributes in sys are currently inconsistent, but it > seems that most of the newer names have underscores. (Aside: if we > are going to move attrs out of sys for py3k, we should consider > renaming them to be consistent too. Regardless of moving them, should > we rename them.) The attribute name is the least problem. It's easy to fix. Brett came up with a nice idea, too. He suggested the gc module as the place for the function. Christian From ncoghlan at gmail.com Mon Jan 28 04:40:15 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 28 Jan 2008 13:40:15 +1000 Subject: [Python-Dev] Organization of ABC modules In-Reply-To: References: <20080125184813.AFS85141@ms19.lnh.mail.rcn.net> <479B4B52.1060301@gmail.com> <20080127082936.GC55130@nexus.in-nomine.org> Message-ID: <479D4E9F.7050600@gmail.com> Guido van Rossum wrote: > I note that at least for built-in types there will be the naming > convention that concrete implementation classes are all lowercase, > like int, float, list, namedtuple, defaultdict, and so on, while the > ABCs all have a Capitalized[Words] name: Hashable, Number, Real, > MutableMapping, etc. That's a very good point. I also suspect that for any actual 2.6/3.0 code base I end up working with there will only be a very limited number of abstract base classes that get tested for via isinstance - so the red flag for isinstance checks would be types I didn't already recognise as being abstract base classes. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From schmir at gmail.com Mon Jan 28 06:21:17 2008 From: schmir at gmail.com (Ralf Schmitt) Date: Mon, 28 Jan 2008 06:21:17 +0100 Subject: [Python-Dev] why is mmap a builtin module on windows? In-Reply-To: <479A6B69.609@v.loewis.de> References: <932f8baf0801222353v2593e6edq556a6fe5a3e25e60@mail.gmail.com> <4797ADBF.5050201@v.loewis.de> <932f8baf0801250156l6c978ae5v7aad7f03c70a4685@mail.gmail.com> <479A6B69.609@v.loewis.de> Message-ID: <932f8baf0801272121u61ce1737qca6e6846bb45fc21@mail.gmail.com> On Jan 26, 2008 12:06 AM, "Martin v. L?wis" wrote: > > b) automate all aspects of adding modules that should not go > > into pythonxy.dll according to the policy." > > > > > > i.e. create visual studio project files for those modules? and make them > > built automatically? > > And adjust msi.py to package them automatically. > I have a working built environment now on windows (turns out it was rather easy). If time permits, I might have a look at it. - Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080128/d006077a/attachment-0001.htm From greg.ewing at canterbury.ac.nz Mon Jan 28 07:10:13 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 28 Jan 2008 19:10:13 +1300 Subject: [Python-Dev] trunc() In-Reply-To: <5d44f72f0801261410l4fdb830v4badc5e182807f5@mail.gmail.com> References: <20080124133614.AFQ48572@ms19.lnh.mail.rcn.net> <010001c85fec$07f4bf60$6800a8c0@RaymondLaptop1> <5d44f72f0801261410l4fdb830v4badc5e182807f5@mail.gmail.com> Message-ID: <479D71C5.1020501@canterbury.ac.nz> > On Jan 25, 2008 11:21 PM, Raymond Hettinger wrote: > >>... int() for making ints from the non-fractional >>portion of a float. To me, the concept of the "integer part of a float" isn't all that well defined. It's really a property of a particular representation rather than the number itself. You think of it as a string of digits and chop off the part after the point, then turn what's left back into a number. If negative floats were represented in two's complement, for example, then chopping off the digits after the point would give a result more like floor than trunc. -- Greg From greg.ewing at canterbury.ac.nz Mon Jan 28 09:35:35 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 28 Jan 2008 21:35:35 +1300 Subject: [Python-Dev] Incorrect documentation of the raw_input built-in function In-Reply-To: References: <18328.49667.162531.446063@montanaro.dyndns.org> Message-ID: <479D93D7.2050206@canterbury.ac.nz> Isaac Morland wrote: > What about an option (maybe even a default) to send the prompt to stdin? > > The Postgres command line interface psql appears to do this: > > $ psql 2>&1 >/dev/null > Password: > $ No, it's probably using the C stdlib routine getpass(). From the man page: The getpass() function displays a prompt to, and reads in a password from, /dev/tty. If this file is not accessible, getpass() displays the prompt on the standard error output and reads from the standard input. So it appears that the official Unix Way prefers using stderr over stdout for prompting, if using the std files for it at all. Writing to stdin would be wrong, since it's usually read-only, even when connected to a terminal. -- Greg From asmodai at in-nomine.org Mon Jan 28 11:55:21 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Mon, 28 Jan 2008 11:55:21 +0100 Subject: [Python-Dev] refleaks and caches In-Reply-To: <479D3A4C.8050205@cheimes.de> References: <20080127200156.0F6413A4065@sparrow.telecommunity.com> <479D3A4C.8050205@cheimes.de> Message-ID: <20080128105521.GH55130@nexus.in-nomine.org> -On [20080128 03:13], Christian Heimes (lists at cheimes.de) wrote: >Do the int/float free lists cause any trouble or can they eat lots of >memory? I hope I am interpreting it correctly, but it seems http://evanjones.ca/memoryallocator/ explanation on that still applies: "The worst offenders are integers and floats. These two object types allocate their own blocks of memory of approximately 1kB, which are allocated with the system malloc(). These blocks are used as arrays of integers and float objects, which avoids waste from pymalloc rounding the object size up to the nearest multiple of 8. These objects are then linked on to a simple free list. When an object is needed, one is taken from the list or a new block is allocated. When an object is freed, it is returned to the free list. This scheme is very simple and very fast, however, it exhibits a significant problem: the memory that is allocated to integers can never be used for anything else. That means if you write a program which goes and allocates 1000000 integers, then frees them and allocates 1000000 floats, Python will hold on to enough memory for 2000000 numerical objects. The solution is to apply a similar approach as was described above. Pools could be requested from pymalloc, so they are properly aligned. When freeing an integer or a float, the object would be put on a free list for its specific pool. When the pool was no longer needed, it could be returned to pymalloc. The challenge is that these types of objects are used frequently, so care is required to ensure good performance. Dictionaries and lists use a different scheme. Python always keeps a maximum of 80 free lists and dictionaries, any extra are freed. This is not optimal because some applications would perform better with a larger list, while others need less. It is possible that self-tuning the list size could be more efficient." -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ We have met the enemy and they are ours... From mal at egenix.com Mon Jan 28 12:07:19 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 28 Jan 2008 12:07:19 +0100 Subject: [Python-Dev] trunc() In-Reply-To: <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> Message-ID: <479DB767.60001@egenix.com> On 2008-01-27 08:14, Raymond Hettinger wrote: >> . You may disagree, but that doesn't make it nuts. > > Too many thoughts compressed into one adjective ;-) > > Deprecating int(float)-->int may not be nuts, but it is disruptive. > > Having both trunc() and int() in Py2.6 may not be nuts, but it is duplicative and confusing. > > The original impetus for facilitating a new Real type being able to trunc() into a new Integral type may not be nuts, but the use > case seems far fetched (we're never had a feature request for it -- the notion was born entirely out of numeric tower > considerations). > > The idea that programmers are confused by int(3.7)-->3 may not be nuts, but it doesn't match any experience I've had with any > programmer, ever. > > The idea that trunc() is beneficial may not be nuts, but it is certainly questionable. > > In short, the idea may not be nuts, but I think it is legitimate to suggest that it is unnecessary and that it will do more harm > than good. All this reminds me a lot of discussions we've had when we needed a new way to spell out string.join(). In the end, we ended up adding a method to strings (thanks to Tim Peters, IIRC) instead of adding a builtin join(). Since all of the suggested builtins are only meant to work on floats, why not simply add methods for them to the float object ?! E.g. x = 3.141 print x.trunc(), x.floor(), x.ceil() etc. This approach also makes it possible to write types or classes that expose the same API without having to resort to new special methods (we have too many of those already). Please consider that type constructors have a different scope than helper functions. Helper functions should only be made builtins if they are really really useful and often needed. If they don't meet this criteria, they are better off in a separate module. I don't see any of the suggested helper functions meeting this criteria and we already have math.floor() and math.ceil(). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 28 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From barry at python.org Mon Jan 28 02:11:34 2008 From: barry at python.org (Barry Warsaw) Date: Sun, 27 Jan 2008 20:11:34 -0500 Subject: [Python-Dev] Upcoming 2.4.5 and 2.3.7 releases In-Reply-To: <479CDB1F.60809@v.loewis.de> References: <479CDB1F.60809@v.loewis.de> Message-ID: <0124660D-3CF9-48BF-8DB1-B71763B1D2EC@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 27, 2008, at 2:27 PM, Martin v. L?wis wrote: > Along with the release of 2.5.2, I would also like to release > new versions of 2.3 and 2.4. These will be security-only releases, > and include a few security-relevant bug fixes that are still being > finalized. > > As we don't have the infrastructure to produce full releases of 2.3 > or 2.4 anymore, this will be a source release only. As such, it > will likely see less testing than other releases, and users will have > more difficulties in obtaining the software for their system - the > releases will be targeted primarily at system vendors who can chose > to include them as security patches in their system updates. If the intent is really to do a source-only releases mostly for system vendors, then I don't see the harm in leaving those changes in. I mean, a vendor is going to cherry pick the ones they want anyway, so let's just make it easy for them to do this. That might mean publishing the svn logs a long with the source release, or publishing each diff and log message separately. I would be bummed to rollback the email package changes. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBR50rxnEjvBPtnXfVAQK4UQP/Xbj6YhXAbhqdZBdirgTw3O9dqocgzgT/ 7Sky4fl0WaSTxRJc3/P0E4uR69TixZ5E7z3pt4G0QeKZCjub3l6xzKiJ7dccAWbW V8otTxCyavXN8wW99orBqgB2J+H0BFkHlPJdYOOBigBuWrSn/Ew2wrZeXRnBfz// qBvWbwlA+sE= =u/23 -----END PGP SIGNATURE----- From lists at cheimes.de Mon Jan 28 13:54:32 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 28 Jan 2008 13:54:32 +0100 Subject: [Python-Dev] refleaks and caches In-Reply-To: <20080128105521.GH55130@nexus.in-nomine.org> References: <20080127200156.0F6413A4065@sparrow.telecommunity.com> <479D3A4C.8050205@cheimes.de> <20080128105521.GH55130@nexus.in-nomine.org> Message-ID: <479DD088.2090109@cheimes.de> Jeroen Ruigrok van der Werven wrote: > -On [20080128 03:13], Christian Heimes (lists at cheimes.de) wrote: >> Do the int/float free lists cause any trouble or can they eat lots of >> memory? > > I hope I am interpreting it correctly, but it seems > http://evanjones.ca/memoryallocator/ explanation on that still applies: [snip] Yes, the explanation still applies. It took me a while to understand how the free lists are working. Some genius came up with the idea to (ab)use the op_type field of the PyObject struct to link the freed objects. All the time I wondered why it assigns something complete different than a type object to the type field. In patch http://bugs.python.org/issue1953 I've moved the compact code from the PyFloat/Int_Fini functions to two new functions and exposed them as a single Python function gc.compact_freelists(). It doesn't solve the problem described in the text but at least it gives a user the chance to free some memory manually. Christian From amk at amk.ca Mon Jan 28 15:17:57 2008 From: amk at amk.ca (A.M. Kuchling) Date: Mon, 28 Jan 2008 09:17:57 -0500 Subject: [Python-Dev] Upcoming 2.4.5 and 2.3.7 releases In-Reply-To: <479CDB1F.60809@v.loewis.de> References: <479CDB1F.60809@v.loewis.de> Message-ID: <20080128141757.GA11901@amk-desktop.matrixgroup.net> On Sun, Jan 27, 2008 at 08:27:27PM +0100, "Martin v. L?wis" wrote: > For 2.3, there are only few revisions that would be rolled back: > r52798, r52803, r52824, r54342. These changes can be reverted if you like; they were added for Jython's sake, but it looks like they've now decided to skip directly to 2.5 compatibility. > For 2.4, the list is longer; all changes on the branch since > r52382 are candidate for roll-back. I would like to prepare > a white-list of patches that should be preserved; if you think > any of the patches committed in the 2.4 branch is a security > fix, please let me know. r55350 is a security fix for cgitb. --amk From guido at python.org Mon Jan 28 17:06:26 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 28 Jan 2008 08:06:26 -0800 Subject: [Python-Dev] Upcoming 2.4.5 and 2.3.7 releases In-Reply-To: <0124660D-3CF9-48BF-8DB1-B71763B1D2EC@python.org> References: <479CDB1F.60809@v.loewis.de> <0124660D-3CF9-48BF-8DB1-B71763B1D2EC@python.org> Message-ID: On Jan 27, 2008 5:11 PM, Barry Warsaw wrote: > On Jan 27, 2008, at 2:27 PM, Martin v. L?wis wrote: > > > Along with the release of 2.5.2, I would also like to release > > new versions of 2.3 and 2.4. These will be security-only releases, > > and include a few security-relevant bug fixes that are still being > > finalized. > > > > As we don't have the infrastructure to produce full releases of 2.3 > > or 2.4 anymore, this will be a source release only. As such, it > > will likely see less testing than other releases, and users will have > > more difficulties in obtaining the software for their system - the > > releases will be targeted primarily at system vendors who can chose > > to include them as security patches in their system updates. > > If the intent is really to do a source-only releases mostly for system > vendors, then I don't see the harm in leaving those changes in. I > mean, a vendor is going to cherry pick the ones they want anyway, so > let's just make it easy for them to do this. That might mean > publishing the svn logs a long with the source release, or publishing > each diff and log message separately. > > I would be bummed to rollback the email package changes. But which vendor would cherry-pick those changes for 2.3 or 2.4? I presume vendors are also in security-fixes-only mode. Are any of the email package fixes security fixes? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From rowen at cesmail.net Mon Jan 28 18:07:34 2008 From: rowen at cesmail.net (Russell E. Owen) Date: Mon, 28 Jan 2008 09:07:34 -0800 Subject: [Python-Dev] [python] trunc() References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <47991DDA.8090604@voidspace.org.uk> <47992363.3010402@v.loewis.de> Message-ID: In article <47992363.3010402 at v.loewis.de>, "Martin v. Lowis" wrote: > > If the ambiguity is that 'int' behaviour is unspecified for floats - is > > it naive to suggest we specify the behaviour? > > The concern is that whatever gets specified is arbitrary. There are many > ways how an int can be constructed from a float, so why is any such way > better than the others, and deserves to be the meaning of int()? But something should be specified. Users should be able to expect consistent behavior. Surely there must be some efficiency reason why it is not specified (e.g. it uses a low-level C call that is not specified)??? I agree with the idea of putting trunc in the math library since it seems to similar to floor. -- Russell From rowen at cesmail.net Mon Jan 28 18:13:29 2008 From: rowen at cesmail.net (Russell E. Owen) Date: Mon, 28 Jan 2008 09:13:29 -0800 Subject: [Python-Dev] trunc() References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> Message-ID: In article <79990c6b0801250553l2e6247adudf48112436dcda70 at mail.gmail.com>, "Paul Moore" wrote: > On 24/01/2008, Jeffrey Yasskin wrote: > > int has to be a builtin because it's a fundamental type. trunc() > > followed round() into the builtins. I have no opinion on whether ceil > > and floor should move there; it probably depends on how often they're > > used. > > Suggestion: > > - int() has to stay in builtins for obvious reasons. > - put *all* of trunc, ceil, floor, round into math. > - make int(float) an error I like all of your suggestions except the last one. Remember the problem with a/b depending on whether b happened to be a float or an int? I think you'll be creating a very similar problem here. In my opinion int(foo) should do its best to turn foo into an int with *predictable* behavior. The least surprising behavior for int(float) is probably trunc(float). Personally I prefer round(float), but I doubt it is worth breaking code and retraining everybody. -- Russell From guido at python.org Mon Jan 28 18:57:01 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 28 Jan 2008 09:57:01 -0800 Subject: [Python-Dev] Incorrect documentation of the raw_input built-in function In-Reply-To: <479D93D7.2050206@canterbury.ac.nz> References: <18328.49667.162531.446063@montanaro.dyndns.org> <479D93D7.2050206@canterbury.ac.nz> Message-ID: On Jan 28, 2008 12:35 AM, Greg Ewing wrote: > Isaac Morland wrote: > > > What about an option (maybe even a default) to send the prompt to stdin? > > > > The Postgres command line interface psql appears to do this: > > > > $ psql 2>&1 >/dev/null > > Password: > > $ > > No, it's probably using the C stdlib routine getpass(). From the man > page: > > The getpass() function displays a prompt to, and reads in a password > from, /dev/tty. If this file is not accessible, getpass() displays the > prompt on the standard error output and reads from the standard input. > > So it appears that the official Unix Way prefers using stderr > over stdout for prompting, if using the std files for it at all. That's a dangerous generalization from just one example. I'd prefer it if you could unearth some POSIX or Linux base document saying this. > Writing to stdin would be wrong, since it's usually read-only, even > when connected to a terminal. Nowadays, it often is writable I've found, but we still shouldn't assume that. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From asmodai at in-nomine.org Mon Jan 28 19:50:16 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Mon, 28 Jan 2008 19:50:16 +0100 Subject: [Python-Dev] Incorrect documentation of the raw_input built-in function In-Reply-To: References: <18328.49667.162531.446063@montanaro.dyndns.org> <479D93D7.2050206@canterbury.ac.nz> Message-ID: <20080128185016.GL55130@nexus.in-nomine.org> -On [20080128 18:57], Guido van Rossum (guido at python.org) wrote: >On Jan 28, 2008 12:35 AM, Greg Ewing wrote: >> So it appears that the official Unix Way prefers using stderr >> over stdout for prompting, if using the std files for it at all. > >That's a dangerous generalization from just one example. I'd prefer it >if you could unearth some POSIX or Linux base document saying this. I cannot find anything explicitly in favour of or against this in POSIX. The stuff quoted below is what I could find. I do not think there's a hard mandate either way, but from my experience of having been a committer for both FreeBSD and DragonFly BSD for a number of years stderr was not preferred for prompting. stderr was always the domain of diagnostics. "3.105 Command Language Interpreter An interface that interprets sequences of text input as commands. It may operate on an input stream or it may interactively prompt and read commands from a terminal. It is possible for applications to invoke utilities through a number of interfaces, which are collectively considered to act as command interpreters." POSIX defines per utility what stdin, stdout and stderr do. The default definition for these is: STDIN There is no additional rationale provided for this section. STDOUT The format description is intended to be sufficiently rigorous to allow post-processing of output by other programs, particularly by an awk or lex parser. STDERR This section does not describe error messages that refer to incorrect operation of the utility. Consider a utility that processes program source code as its input. This section is used to describe messages produced by a correctly operating utility that encounters an error in the program source code on which it is processing. However, a message indicating that the utility had insufficient memory in which to operate would not be described. Some utilities have traditionally produced warning messages without returning a non-zero exit status; these are specifically noted in their sections. Other utilities shall not write to standard error if they complete successfully, unless the implementation provides some sort of extension to increase the verbosity or debugging level. The format descriptions are intended to be sufficiently rigorous to allow post-processing of output by other programs. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ We have met the enemy and they are ours... From python at rcn.com Mon Jan 28 19:50:28 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 28 Jan 2008 13:50:28 -0500 (EST) Subject: [Python-Dev] Py2.6 release schedule Message-ID: <20080128135028.AFV97486@ms19.lnh.mail.rcn.net> What is the current thinking on this? Is the target still April 2008 as mentioned in PEP361? Are we going to have an alpha sometime soonish? Raymond From guido at python.org Mon Jan 28 20:02:08 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 28 Jan 2008 11:02:08 -0800 Subject: [Python-Dev] Py2.6 release schedule In-Reply-To: <20080128135028.AFV97486@ms19.lnh.mail.rcn.net> References: <20080128135028.AFV97486@ms19.lnh.mail.rcn.net> Message-ID: On Jan 28, 2008 10:50 AM, Raymond Hettinger wrote: > What is the current thinking on this? Is the target still April 2008 as mentioned in PEP361? Are we going to have an alpha sometime soonish? I would like to have an alpha before PyCon. It looks like there are two release schedules for 2.6: one in PEP 361, and another in PEP 3000. The dates in PEP 3000 look a bit more realistic: * Early 2008 (pre-PyCon): release 2.6a1. * May 2008 (post-PyCon): full feature freeze for 3.0 and 2.6. * July 2008: release 2.6 (final). * August 2008: release 3.0 (final). Though it all depends on who volunteers to be the 2.6 release manager. I'm not volunteering, although I'll continue to do 3.0 releases until someone tells me it needs to be done in a more professional manner (probably once the betas start). I do think that another (final) 3.0 alpha before PyCon would also be a good idea. This way we can gel the release some more. For 2.6 I think we'll need more alpha releases after PyCon; I doubt the backporting from 3.0 (which has only started seriously quite recently) will be done by PyCon. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steve at holdenweb.com Mon Jan 28 20:17:50 2008 From: steve at holdenweb.com (Steve Holden) Date: Mon, 28 Jan 2008 14:17:50 -0500 Subject: [Python-Dev] Incorrect documentation of the raw_input built-in function In-Reply-To: References: <18328.49667.162531.446063@montanaro.dyndns.org> <479D93D7.2050206@canterbury.ac.nz> Message-ID: <479E2A5E.9090909@holdenweb.com> Guido van Rossum wrote: > On Jan 28, 2008 12:35 AM, Greg Ewing wrote: >> Isaac Morland wrote: >> >>> What about an option (maybe even a default) to send the prompt to stdin? >>> >>> The Postgres command line interface psql appears to do this: >>> >>> $ psql 2>&1 >/dev/null >>> Password: >>> $ >> No, it's probably using the C stdlib routine getpass(). From the man >> page: >> >> The getpass() function displays a prompt to, and reads in a password >> from, /dev/tty. If this file is not accessible, getpass() displays the >> prompt on the standard error output and reads from the standard input. >> >> So it appears that the official Unix Way prefers using stderr >> over stdout for prompting, if using the std files for it at all. > > That's a dangerous generalization from just one example. I'd prefer it > if you could unearth some POSIX or Linux base document saying this. > While I couldn't locate such a document, it makes sense when you consider that if such a process is part of a pipeline you really don't want the prompts being handled as input by the downstream processes. That's why mv and similar utilities prompt on standard error. I'd be very surprised if you can find a UNIX or Linux shell, for example, that prompts on standard output. >> Writing to stdin would be wrong, since it's usually read-only, even >> when connected to a terminal. > > Nowadays, it often is writable I've found, but we still shouldn't assume that. > regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Mon Jan 28 20:17:50 2008 From: steve at holdenweb.com (Steve Holden) Date: Mon, 28 Jan 2008 14:17:50 -0500 Subject: [Python-Dev] Incorrect documentation of the raw_input built-in function In-Reply-To: References: <18328.49667.162531.446063@montanaro.dyndns.org> <479D93D7.2050206@canterbury.ac.nz> Message-ID: <479E2A5E.9090909@holdenweb.com> Guido van Rossum wrote: > On Jan 28, 2008 12:35 AM, Greg Ewing wrote: >> Isaac Morland wrote: >> >>> What about an option (maybe even a default) to send the prompt to stdin? >>> >>> The Postgres command line interface psql appears to do this: >>> >>> $ psql 2>&1 >/dev/null >>> Password: >>> $ >> No, it's probably using the C stdlib routine getpass(). From the man >> page: >> >> The getpass() function displays a prompt to, and reads in a password >> from, /dev/tty. If this file is not accessible, getpass() displays the >> prompt on the standard error output and reads from the standard input. >> >> So it appears that the official Unix Way prefers using stderr >> over stdout for prompting, if using the std files for it at all. > > That's a dangerous generalization from just one example. I'd prefer it > if you could unearth some POSIX or Linux base document saying this. > While I couldn't locate such a document, it makes sense when you consider that if such a process is part of a pipeline you really don't want the prompts being handled as input by the downstream processes. That's why mv and similar utilities prompt on standard error. I'd be very surprised if you can find a UNIX or Linux shell, for example, that prompts on standard output. >> Writing to stdin would be wrong, since it's usually read-only, even >> when connected to a terminal. > > Nowadays, it often is writable I've found, but we still shouldn't assume that. > regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From guido at python.org Mon Jan 28 20:30:31 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 28 Jan 2008 11:30:31 -0800 Subject: [Python-Dev] Incorrect documentation of the raw_input built-in function In-Reply-To: <479E2A5E.9090909@holdenweb.com> References: <18328.49667.162531.446063@montanaro.dyndns.org> <479D93D7.2050206@canterbury.ac.nz> <479E2A5E.9090909@holdenweb.com> Message-ID: On Jan 28, 2008 11:17 AM, Steve Holden wrote: > > That's a dangerous generalization from just one example. I'd prefer it > > if you could unearth some POSIX or Linux base document saying this. > > > While I couldn't locate such a document, it makes sense when you > consider that if such a process is part of a pipeline you really don't > want the prompts being handled as input by the downstream processes. > > That's why mv and similar utilities prompt on standard error. > > I'd be very surprised if you can find a UNIX or Linux shell, for > example, that prompts on standard output. Ah, indeed. It looks like Python's prompt indeed gets written to stderr, at least when stdout is redirected to a file. Oddly, when stderr is redirected to a file, the prompt gets written to stdout. This seems backwards then (and not what the shells do -- when stderr isn't interactive, they don't write a prompt at all). But fixing it is delicate -- e.g. getpass very carefully makes sure to write to stdout, not stderr (and to avoid raw_input() and hence GNU readline). And cmd.py (used by pdb.py) also assumes the prompt should go to stdout when it is not using raw_input. I'm beginning to think that, despite the example of the shells, we'd be better off if Python *always* prompted to stdout (or not at all if that's not a tty). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From lists at cheimes.de Mon Jan 28 20:33:14 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 28 Jan 2008 20:33:14 +0100 Subject: [Python-Dev] Py2.6 release schedule In-Reply-To: References: <20080128135028.AFV97486@ms19.lnh.mail.rcn.net> Message-ID: <479E2DFA.8040904@cheimes.de> Guido van Rossum wrote: > I do think that another (final) 3.0 alpha before PyCon would also be a > good idea. This way we can gel the release some more. For 2.6 I think > we'll need more alpha releases after PyCon; I doubt the backporting > from 3.0 (which has only started seriously quite recently) will be > done by PyCon. I've back ported class decorators (http://bugs.python.org/issue1759). Two tests are failing and I need some help to solve the riddle. Several back ports like the bytearray type and the new io module depend on a back port of the new buffer protocol. Travis, can you please increase your priority on the port of your PEP to 2.6? Christian From thomas at python.org Mon Jan 28 20:44:07 2008 From: thomas at python.org (Thomas Wouters) Date: Mon, 28 Jan 2008 11:44:07 -0800 Subject: [Python-Dev] refleaks and caches In-Reply-To: References: <20080127200156.0F6413A4065@sparrow.telecommunity.com> Message-ID: <9e804ac0801281144u3d1e4f4csc3d934d4de8af17e@mail.gmail.com> On Jan 27, 2008 6:08 PM, Christian Heimes wrote: > Brett Cannon wrote: > > Ignoring whether this is the right thing to do, should this be in sys or > in gc? > > Yeah, good idea. The gc module makes sense. > Does it? The gc module is specific to the cyclic-gc system. I don't see that this method is. If cyclic-gc is unavailable, should this function be unavailable too? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080128/c1ab8c26/attachment.htm From asmodai at in-nomine.org Mon Jan 28 20:55:35 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Mon, 28 Jan 2008 20:55:35 +0100 Subject: [Python-Dev] Incorrect documentation of the raw_input built-in function In-Reply-To: <20080128185016.GL55130@nexus.in-nomine.org> References: <18328.49667.162531.446063@montanaro.dyndns.org> <479D93D7.2050206@canterbury.ac.nz> <20080128185016.GL55130@nexus.in-nomine.org> Message-ID: <20080128195535.GA75774@nexus.in-nomine.org> -On [20080128 19:51], Jeroen Ruigrok van der Werven (asmodai at in-nomine.org) wrote: >I cannot find anything explicitly in favour of or against this in POSIX. The >stuff quoted below is what I could find. I do not think there's a hard >mandate either way, but from my experience of having been a committer for >both FreeBSD and DragonFly BSD for a number of years stderr was not >preferred for prompting. stderr was always the domain of diagnostics. Sorry, I made a mental typo here: but from my experience of having been a committer for both FreeBSD and DragonFly BSD for a number of years stdout was not preferred for prompting. stderr was always the domain of diagnostics. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ We have met the enemy and they are ours... From martin at v.loewis.de Mon Jan 28 22:22:00 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 28 Jan 2008 22:22:00 +0100 Subject: [Python-Dev] Upcoming 2.4.5 and 2.3.7 releases In-Reply-To: <0124660D-3CF9-48BF-8DB1-B71763B1D2EC@python.org> References: <479CDB1F.60809@v.loewis.de> <0124660D-3CF9-48BF-8DB1-B71763B1D2EC@python.org> Message-ID: <479E4778.7010500@v.loewis.de> > If the intent is really to do a source-only releases mostly for system > vendors, then I don't see the harm in leaving those changes in. I mean, > a vendor is going to cherry pick the ones they want anyway, so let's > just make it easy for them to do this. That might mean publishing the > svn logs a long with the source release, or publishing each diff and log > message separately. It's not just vendors, also end-users who are concerned about the security of their installations. > I would be bummed to rollback the email package changes. You don't have to - I will do it for you (although I don't understand fully what "to be bummed" means). Regards, Martin From barry at python.org Mon Jan 28 22:47:34 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 28 Jan 2008 16:47:34 -0500 Subject: [Python-Dev] Upcoming 2.4.5 and 2.3.7 releases In-Reply-To: <479E4778.7010500@v.loewis.de> References: <479CDB1F.60809@v.loewis.de> <0124660D-3CF9-48BF-8DB1-B71763B1D2EC@python.org> <479E4778.7010500@v.loewis.de> Message-ID: <479E4D76.1080205@python.org> Martin v. L?wis wrote: >> If the intent is really to do a source-only releases mostly for system >> vendors, then I don't see the harm in leaving those changes in. I mean, >> a vendor is going to cherry pick the ones they want anyway, so let's >> just make it easy for them to do this. That might mean publishing the >> svn logs a long with the source release, or publishing each diff and log >> message separately. > > It's not just vendors, also end-users who are concerned about the > security of their installations. > >> I would be bummed to rollback the email package changes. > > You don't have to - I will do it for you (although I don't > understand fully what "to be bummed" means). It means I'd be sad. ;) The problem is that I make separate releases of the standalone email package from these branches, so that means that email 3.0.3 or 2.5.10 will have regressions. Unless you're offering to also re-apply these changes after you make the Python releases . -Barry From martin at v.loewis.de Mon Jan 28 22:44:38 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 28 Jan 2008 22:44:38 +0100 Subject: [Python-Dev] Incorrect documentation of the raw_input built-in function In-Reply-To: <479E2A5E.9090909@holdenweb.com> References: <18328.49667.162531.446063@montanaro.dyndns.org> <479D93D7.2050206@canterbury.ac.nz> <479E2A5E.9090909@holdenweb.com> Message-ID: <479E4CC6.6070003@v.loewis.de> > While I couldn't locate such a document, it makes sense when you > consider that if such a process is part of a pipeline you really don't > want the prompts being handled as input by the downstream processes. > > That's why mv and similar utilities prompt on standard error. No, that doesn't really make sense. If you are in the middle of a pipe, what good does it do to write the prompt to stderr, yet read the user input from stdin? If you really care about processes that run in a pipe, interact with /dev/tty. Regards, Martin From guido at python.org Mon Jan 28 22:52:19 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 28 Jan 2008 13:52:19 -0800 Subject: [Python-Dev] Upcoming 2.4.5 and 2.3.7 releases In-Reply-To: <479E4D76.1080205@python.org> References: <479CDB1F.60809@v.loewis.de> <0124660D-3CF9-48BF-8DB1-B71763B1D2EC@python.org> <479E4778.7010500@v.loewis.de> <479E4D76.1080205@python.org> Message-ID: On Jan 28, 2008 1:47 PM, Barry Warsaw wrote: > Martin v. L?wis wrote: > >> If the intent is really to do a source-only releases mostly for system > >> vendors, then I don't see the harm in leaving those changes in. I mean, > >> a vendor is going to cherry pick the ones they want anyway, so let's > >> just make it easy for them to do this. That might mean publishing the > >> svn logs a long with the source release, or publishing each diff and log > >> message separately. > > > > It's not just vendors, also end-users who are concerned about the > > security of their installations. > > > >> I would be bummed to rollback the email package changes. > > > > You don't have to - I will do it for you (although I don't > > understand fully what "to be bummed" means). > > It means I'd be sad. ;) > > The problem is that I make separate releases of the standalone email > package from these branches, so that means that email 3.0.3 or 2.5.10 > will have regressions. > > Unless you're offering to also re-apply these changes after you make the > Python releases . This sounds like a special case that we might consider. Though I also wonder if it wouldn't be easiest for you to just create separate branches for the email package rather than rely on the core Python branching structure and release rules. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Mon Jan 28 22:54:35 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 28 Jan 2008 22:54:35 +0100 Subject: [Python-Dev] functions vs methods (was Re: trunc()) In-Reply-To: <20080127205309.GA17015@panix.com> References: <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> <479CE15A.2010106@v.loewis.de> <20080127205309.GA17015@panix.com> Message-ID: <479E4F1B.8060002@v.loewis.de> > This is why len() is not a method: > > map(len, list_of_strings) That's a good use case - but is that the reason? I would think that the true historical reason is that when len() was introduced, there weren't methods in the language. And even when support for methods was added, many types supporting len wouldn't easily support methods (e.g. the string type). > What I don't know is to what extent this argument still holds in the > presence of listcomps and genexps: > > [s.len() for s in list_of_strings] > > However, you still need ``len`` as a function to pass around as a > callback in other cases where you need a generic function because the > type of your data is not predetermined. You don't *need* it for that; you could also pass around lambda x:x.len() It's clear that you need functions for a functional programming style. However, I would question that typical Python code is inherently functional, and instead I believe that it could just as well or better be object-oriented - it's just that Python mandates functions at certain places. > In any event, I consider dropping len() from builtins to be gratuitous > breakage, even in 3.0. I wouldn't want to propose removal of len(), no. However, I do think that adding more builtins (trunc in particular) is bad, especially when they make perfect methods. Regards, Martin From fdrake at acm.org Mon Jan 28 22:56:59 2008 From: fdrake at acm.org (Fred Drake) Date: Mon, 28 Jan 2008 16:56:59 -0500 Subject: [Python-Dev] Upcoming 2.4.5 and 2.3.7 releases In-Reply-To: <479E4D76.1080205@python.org> References: <479CDB1F.60809@v.loewis.de> <0124660D-3CF9-48BF-8DB1-B71763B1D2EC@python.org> <479E4778.7010500@v.loewis.de> <479E4D76.1080205@python.org> Message-ID: <5E6A9FAF-F3BA-4281-9053-3BD986B79C69@acm.org> On Jan 28, 2008, at 4:47 PM, Barry Warsaw wrote: > The problem is that I make separate releases of the standalone email > package from these branches, so that means that email 3.0.3 or 2.5.10 > will have regressions. Releasing the email package from the Python maintenance branches is probably insane. This kind of thing would be less of a problem if the standard library were smaller, and the email package only available separately. It's also why some of us want a smaller standard library. -Fred -- Fred Drake From python at rcn.com Mon Jan 28 22:58:20 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 28 Jan 2008 16:58:20 -0500 (EST) Subject: [Python-Dev] functions vs methods (was Re: trunc()) Message-ID: <20080128165820.AFW31293@ms19.lnh.mail.rcn.net> [MvL] > I wouldn't want to propose removal of len(), no. However, > I do think that adding more builtins (trunc in particular) > is bad, especially when they make perfect methods. +1 Raymond From guido at python.org Mon Jan 28 23:00:33 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 28 Jan 2008 14:00:33 -0800 Subject: [Python-Dev] functions vs methods (was Re: trunc()) In-Reply-To: <479E4F1B.8060002@v.loewis.de> References: <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> <479CE15A.2010106@v.loewis.de> <20080127205309.GA17015@panix.com> <479E4F1B.8060002@v.loewis.de> Message-ID: On Jan 28, 2008 1:54 PM, "Martin v. L?wis" wrote: > > This is why len() is not a method: > > > > map(len, list_of_strings) > > That's a good use case - but is that the reason? > > I would think that the true historical reason is that when len() > was introduced, there weren't methods in the language. And even > when support for methods was added, many types supporting len wouldn't > easily support methods (e.g. the string type). Well, there were methods, but there were no "special" methods, and the length of a sequence was intended to be implemented in C. The very first version of the language (used internally at CWI for a while) didn't even have classic classes -- the only way to add a new type was to write a C extension. > > What I don't know is to what extent this argument still holds in the > > presence of listcomps and genexps: > > > > [s.len() for s in list_of_strings] > > > > However, you still need ``len`` as a function to pass around as a > > callback in other cases where you need a generic function because the > > type of your data is not predetermined. > > You don't *need* it for that; you could also pass around > lambda x:x.len() > > It's clear that you need functions for a functional programming style. > However, I would question that typical Python code is inherently > functional, and instead I believe that it could just as well or better > be object-oriented - it's just that Python mandates functions at certain > places. I think that for certain things (e.g. len()) the functional notation is just more readable than the method notation, because it provides more information to the reader: len(x) guarantees to return an int. x.len() has no such guarantee, it could be an unrelated len method on an object that has nothing in common with sequences. > > In any event, I consider dropping len() from builtins to be gratuitous > > breakage, even in 3.0. > > I wouldn't want to propose removal of len(), no. However, I do think > that adding more builtins (trunc in particular) is bad, especially > when they make perfect methods. No, using trunc(x) makes it clear that the argument and return value are numbers. Using x.trunc() doesn't. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Mon Jan 28 23:08:44 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 28 Jan 2008 23:08:44 +0100 Subject: [Python-Dev] Upcoming 2.4.5 and 2.3.7 releases In-Reply-To: <479E4D76.1080205@python.org> References: <479CDB1F.60809@v.loewis.de> <0124660D-3CF9-48BF-8DB1-B71763B1D2EC@python.org> <479E4778.7010500@v.loewis.de> <479E4D76.1080205@python.org> Message-ID: <479E526C.3020100@v.loewis.de> > The problem is that I make separate releases of the standalone email > package from these branches, so that means that email 3.0.3 or 2.5.10 > will have regressions. > > Unless you're offering to also re-apply these changes after you make the > Python releases . If they are there primarily to track any changes that you want to release in the future (i.e. they are the repository of standalone email package), then a solution must be found, I agree. Would it help if I branch branches/release23-maint/Lib/email to branches/email-2.3 ? Or should I branch the entire release23-maint to email-2.3? (branches are cheap in subversion) (*) It would mean that you have to check in future 2.3 fixes of the email package into a different branch, and if you run into security fixes, you need to commit them into two branches (email-2.3 and release23-maint). Same for 2.4. For 2.5, bug fixes could be added until that branch also goes into the security-only mode (i.e. after 2.6 is released). Please let me know what you think. Regards, Martin (*) other locations are possible as well, such as branches/email/2.3 or /email/branches/2.3 From daniel at stutzbachenterprises.com Mon Jan 28 23:28:23 2008 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Mon, 28 Jan 2008 16:28:23 -0600 Subject: [Python-Dev] functions vs methods (was Re: trunc()) In-Reply-To: References: <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> <479CE15A.2010106@v.loewis.de> <20080127205309.GA17015@panix.com> <479E4F1B.8060002@v.loewis.de> Message-ID: On Jan 28, 2008 4:00 PM, Guido van Rossum wrote: > No, using trunc(x) makes it clear that the argument and return value > are numbers. Using x.trunc() doesn't. How often do you expect someone to be looking at code where a trunc() method is being called and the variable could plausibly be both a number or something else? (a Google Code search for "def trunc(self)" lang:python returns 1 hit) How does the that additional value weigh against the cost of adding another builtin and trying to explain trunc() versus int() to new users? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From guido at python.org Mon Jan 28 23:48:24 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 28 Jan 2008 14:48:24 -0800 Subject: [Python-Dev] functions vs methods (was Re: trunc()) In-Reply-To: References: <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> <479CE15A.2010106@v.loewis.de> <20080127205309.GA17015@panix.com> <479E4F1B.8060002@v.loewis.de> Message-ID: On Jan 28, 2008 2:28 PM, Daniel Stutzbach wrote: > On Jan 28, 2008 4:00 PM, Guido van Rossum wrote: > > No, using trunc(x) makes it clear that the argument and return value > > are numbers. Using x.trunc() doesn't. > > How often do you expect someone to be looking at code where a trunc() > method is being called and the variable could plausibly be both a > number or something else? (a Google Code search for "def trunc(self)" > lang:python returns 1 hit) > > How does the that additional value weigh against the cost of adding > another builtin and trying to explain trunc() versus int() to new > users? It's all pretty hypothetical at this point. I do think that the *concept* of trunc() is very easy to understand so the cost to the user of having to learn it (only when they encounter code that uses it or feel the need to use it themselves) is negligible. One thing I'm beginning to feel more and more strongly about is that round, trunc, ceil and floor all belong in the same category, and either should all be builtins or should all be in math. I should also admit that the 2-arg version of round() was borrowed from ABC, but the use case for it there doesn't map to Python: in ABC it's the only tool you have for floating point formatting on output, while in Python the same thing would typically be done with "%.2f" % x rather than round(x, 2). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Mon Jan 28 23:59:07 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 29 Jan 2008 11:59:07 +1300 Subject: [Python-Dev] Incorrect documentation of the raw_input built-in function In-Reply-To: References: <18328.49667.162531.446063@montanaro.dyndns.org> <479D93D7.2050206@canterbury.ac.nz> Message-ID: <479E5E3B.9010400@canterbury.ac.nz> Guido van Rossum wrote: > On Jan 28, 2008 12:35 AM, Greg Ewing wrote: >>So it appears that the official Unix Way prefers using stderr >>over stdout for prompting, if using the std files for it at all. > > That's a dangerous generalization from just one example. I'd prefer it > if you could unearth some POSIX or Linux base document saying this. Well, it makes sense in the context of the general Unix philosophy that stdout is for the result that the program is computing, and therefore shouldn't be polluted by error messages or other extraneous things. Also, the Python interpreter itself appears to write its prompts to stderr, both with and without readline. -- Greg From greg.ewing at canterbury.ac.nz Tue Jan 29 00:08:04 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 29 Jan 2008 12:08:04 +1300 Subject: [Python-Dev] Incorrect documentation of the raw_input built-in function In-Reply-To: <20080128185016.GL55130@nexus.in-nomine.org> References: <18328.49667.162531.446063@montanaro.dyndns.org> <479D93D7.2050206@canterbury.ac.nz> <20080128185016.GL55130@nexus.in-nomine.org> Message-ID: <479E6054.3020204@canterbury.ac.nz> Jeroen Ruigrok van der Werven wrote: > I cannot find anything explicitly in favour of or against this in POSIX. The > stuff quoted below is what I could find. > > "3.105 Command Language Interpreter > > STDERR > > This section does not describe error messages that refer to incorrect > operation of the utility. What document did this come from? This sounds more like it's talking about what should be described in various sections of a man page, not what should be written to the various streams by a program. Otherwise, > a message indicating that the > utility had insufficient memory in which to operate would not be described. sounds like an out-of-memory message shouldn't be written to stderr, which I'm fairly sure is not the case! -- Greg From python at rcn.com Tue Jan 29 00:19:26 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 28 Jan 2008 18:19:26 -0500 (EST) Subject: [Python-Dev] functions vs methods (was Re: trunc()) Message-ID: <20080128181926.AFW43829@ms19.lnh.mail.rcn.net> > One thing I'm beginning to feel more and more strongly about > is that round, trunc, ceil and floor all belong in the same > category, and either should all be builtins or should all > be in math. > > I should also admit that the 2-arg version of round() was > borrowed from ABC, but the use case for it there doesn't map > to Python: in ABC it's the only tool you have for floating point > formatting on output, while in Python the same thing would > typically be done with "%.2f" % x rather than round(x, 2). New idea: * Drop the existing second argument for round. * Fold ceil(), floor(), trunc(), and round() into a single function: def round(x, mode=): if mode==CEIL: return math.ceil(x) elif mode==FLOOR: return math.floor(x) elif mode==TRUNC: return math.floor(x) if x>0 else math.ceil(x) elif mode==ROUND_TO_EVEN: . . . Maybe we don't need a separate function and magic method for every possible rounding mode. Also, I think there was some discussion of changing round() to use statistician's rounding. We might as well make it explicit that this is the new default. Raymond P.S. One place to attach the constants is to the function object, so we would write: x = round(3.1) # ROUND_TO_EVEN x = round(3.1, round.CEIL) . . . From fuzzyman at voidspace.org.uk Tue Jan 29 00:28:50 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 28 Jan 2008 23:28:50 +0000 Subject: [Python-Dev] [python] Re: functions vs methods (was Re: trunc()) Message-ID: <6f4025010801281528k507f4d93p47101c3cdb03fb30@mail.gmail.com> Raymond Hettinger wrote: [snip..] P.S. One place to attach the constants is to the function object, so we would write: x = round(3.1) # ROUND_TO_EVEN x = round(3.1, round.CEIL) . . . Or Python could gain a first class enumeration structure, a type which has proved useful in other languages. ;-) Michael Foord _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.manning.com/foord http://www.voidspace.org.uk http://www.ironpython.info -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080128/d0f11e64/attachment.htm From python at rcn.com Tue Jan 29 00:56:47 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 28 Jan 2008 18:56:47 -0500 (EST) Subject: [Python-Dev] functions vs methods (was Re: trunc()) Message-ID: <20080128185647.AFW48175@ms19.lnh.mail.rcn.net> Please ignore my last email. The idea for combining trunc, ceil, floor, etc was probably just a distractor. [GvR] > One thing I'm beginning to feel more and more strongly about > is that round, trunc, ceil and floor all belong in the same > category, and either should all be builtins or should all > be in math. +1 for all four going into the math module. > I should also admit that the 2-arg version of round() was > borrowed from ABC, but the use case for it there doesn't map > to Python: in ABC it's the only tool you have for floating point > formatting on output, while in Python the same thing would > typically be done with "%.2f" % x rather than round(x, 2). +1 for deprecating the second argument to round(). Raymond From barry at python.org Tue Jan 29 02:08:34 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 28 Jan 2008 20:08:34 -0500 Subject: [Python-Dev] Upcoming 2.4.5 and 2.3.7 releases In-Reply-To: <479E526C.3020100@v.loewis.de> References: <479CDB1F.60809@v.loewis.de> <0124660D-3CF9-48BF-8DB1-B71763B1D2EC@python.org> <479E4778.7010500@v.loewis.de> <479E4D76.1080205@python.org> <479E526C.3020100@v.loewis.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 28, 2008, at 5:08 PM, Martin v. L?wis wrote: > Would it help if I branch > branches/release23-maint/Lib/email to branches/email-2.3 ? > Or should I branch the entire release23-maint to email-2.3? > (branches are cheap in subversion) > (*) I just realized there's a good solution to this. The sandbox has an emailpkg directory with subdirectories for each of the email package releases. Currently, the actual code gets svn:external'd in from the Python branches, but for email 2.5 and 3.0 we should just svn copy the branches into here and remove the svn:external definitions. I'm not able to do this right now, but I should be able to do this in the next day or so if you don't beat me to it. > It would mean that you have to check in future 2.3 fixes of the > email package into a different branch, and if you run into security > fixes, you need to commit them into two branches (email-2.3 and > release23-maint). Using the above scenario, that would be fine. > Same for 2.4. For 2.5, bug fixes could be added until that branch > also goes into the security-only mode (i.e. after 2.6 is released). So for now, we'll leave email 4.0 external'd in from the Python 2.5 branch. When 2.6 is released, we'll svn copy that in like we did above. The only catch is that it would probably be best to rev the email package version number in Python 2.6 just to be clear it's a different branch. 4.1.0 would be a fine number even if it's mostly just a bookkeeping trick. > Please let me know what you think. What do you think of the above? Thanks! - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBR558knEjvBPtnXfVAQL1vQQAmjNPN27scZ2Tl/ayEnggZOym8U1MEdgP zvBvfbCrp0V4dSF/zddYjB61+9uRP4bN10oNEbwPKUifJ/me5o6qkguLeO56qzqH fs2rTWUlRXKR27g8WDV0B/KL0F5vpd+3PglQI/TfmSbcZjzAlGIMIY/k27bCL4Hy z9XE+LOmNjU= =ZCyu -----END PGP SIGNATURE----- From brett at python.org Tue Jan 29 02:20:09 2008 From: brett at python.org (Brett Cannon) Date: Mon, 28 Jan 2008 17:20:09 -0800 Subject: [Python-Dev] Announcing the Python core sprint at PyCon 2008 Message-ID: As has occurred since the inception of PyCon, there will be a sprint on the Python core at this year's conference! If you will be attending PyCon (or will be in Chicago during the dates of the sprints), attending the sprint is a great way to give back to Python. Working on Python itself tends to deepens one knowledge of the language and the standard library. Plus it is just plain fun getting to sit around some tables with fellow Python programmers for several days (the sprint will last four days, but you do not need to attend all four days to participate). The sprint is open to everyone of all skill levels. We can always use help with things from updating documentation to preparing for the next release of Python 3.0. On Sunday evening of the conference there will not only be a sprint intro session, but also a tutorial on how to develop for Python. Details will be covered from where to look in the code base for things to some best practices tips. If you are interested enough to want to sign up to attend, please go to http://wiki.python.org/moin/PyCon2008/SprintSignups/Python and add your name and email address. If you have questions you may email me. Please sign up for the sprint by the end of February as an email on what you need to do beforehand will be sent at that time based on the sprint sign-up page. And if you are not attending PyCon, we will most likely have several people in attendance on IRC, thus allowing even people not at PyCon to participate! -Brett Cannon Python core sprint coach, PyCon 2008 From python at rcn.com Tue Jan 29 04:19:38 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 28 Jan 2008 19:19:38 -0800 Subject: [Python-Dev] functions vs methods (was Re: trunc()) References: <5d44f72f0801241524x60b64046ga704d7c479080d39@mail.gmail.com><5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com><005701c8606e$5196b740$6800a8c0@RaymondLaptop1><012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1><479CE15A.2010106@v.loewis.de> <20080127205309.GA17015@panix.com><479E4F1B.8060002@v.loewis.de> Message-ID: <003501c86225$c98863d0$6800a8c0@RaymondLaptop1> > No, using trunc(x) makes it clear that the argument and return > value are numbers. Using x.trunc() doesn't. Not sure where this is notion comes from. Terry Reedy's post provides a datapoint to the contrary. Besides, there is no problem along these lines that can't be cured by a better method name: f.integer_portion() Also, if you go the method route, then the API can easily be expanded to cover all the normal rounding methods: f.round_to_even() f.round_half_up() ... These are all specific and explicit. Also, we can take advantage of the ABC mixin capabilities to automatically provide all of these given one or two of them as primitives. Raymond P.S I get no shortage of hits for searches like: http://www.google.com/search?q=truncate+string From python at rcn.com Tue Jan 29 04:40:12 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 28 Jan 2008 19:40:12 -0800 Subject: [Python-Dev] trunc() Message-ID: <005f01c86228$a7b41530$6800a8c0@RaymondLaptop1> [Terry Reedy] > For one data point, I asked my bright 13-year-old for the 'integer part' of > 3.1, 3.9, -3.1, and -3.9 and got the expected 3,3,-3,-3 (as with int()). > But asking 'truncate' the same numbers or even 'truncate toward zero' got a > blank stare, which did not surprise me too much as it is not a common > (American) English word. Even to adults, the word has other meanings: http://en.wikipedia.org/wiki/Truncation_%28disambiguation%29 Raymond From steve at holdenweb.com Tue Jan 29 05:00:29 2008 From: steve at holdenweb.com (Steve Holden) Date: Mon, 28 Jan 2008 23:00:29 -0500 Subject: [Python-Dev] trunc() In-Reply-To: <005f01c86228$a7b41530$6800a8c0@RaymondLaptop1> References: <005f01c86228$a7b41530$6800a8c0@RaymondLaptop1> Message-ID: Raymond Hettinger wrote: > [Terry Reedy] >> For one data point, I asked my bright 13-year-old for the 'integer part' of >> 3.1, 3.9, -3.1, and -3.9 and got the expected 3,3,-3,-3 (as with int()). >> But asking 'truncate' the same numbers or even 'truncate toward zero' got a >> blank stare, which did not surprise me too much as it is not a common >> (American) English word. > > > Even to adults, the word has other meanings: > > http://en.wikipedia.org/wiki/Truncation_%28disambiguation%29 > I wish this thread could be truncated. It's got me going round and round. I'm completely floored, and will doubtless end up barking mad - you know, like a ceil. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From guido at python.org Tue Jan 29 05:07:21 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 28 Jan 2008 20:07:21 -0800 Subject: [Python-Dev] trunc() In-Reply-To: References: <005f01c86228$a7b41530$6800a8c0@RaymondLaptop1> Message-ID: On Jan 28, 2008 8:00 PM, Steve Holden wrote: > I wish this thread could be truncated. It's got me going round and > round. I'm completely floored, and will doubtless end up barking mad - > you know, like a ceil. Me too. All the arguments have been repeated over and over. It really isn't worth more energy at this point; I see a use for trunc() separate from int() and I really don't care if Raymond doesn't see it. Let's move trunc() to the math module and be done with it, leaving round() a built-in with both the one- and two-argument versions intact. PS. There's something wrong with Raymond's mailer that creates a thread in gmail whenever he responds. I suspect it's not correctly adding an In-reply-to header. That makes the thread feel much more disconnected than most, because often the quoted text is in a different thread. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jyasskin at gmail.com Tue Jan 29 05:13:23 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Mon, 28 Jan 2008 20:13:23 -0800 Subject: [Python-Dev] trunc() In-Reply-To: References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> Message-ID: <5d44f72f0801282013q37e8d4aas1204fa5261e4f060@mail.gmail.com> On Jan 27, 2008 3:37 PM, Terry Reedy wrote: > The actual claim seems to have been that the semantics of int() > itself is fuzzy. This in turn seems to be based one of two ideas (I am not > sure which) > a. 'int' might either mean to some people 'some int associated with the > float input' rather than the more precise 'the integer part of the float > input', or Just like set(sequence) is the set associated with that sequence, not the set part of that sequence, and float('3.14') is the float associated with '3.14', not the float part of '3.14', etc. Type names do not normally retrieve pieces of other objects. When there's no unique or otherwise special instance of a given type associated with an instance of another type, the first should not take the second in its constructor. (This does not imply the inverse.) On the other hand, to retrieve a piece of another object, you do normally call a member on that object, so I'd have no objections to changing trunc() and round() (although the second isn't strictly retrieving a piece) to methods. > b. that some people might think the latter means the same as the former. I don't understand this sentence. I haven't seen anyone advocating trunc() say that "some int associated with" means the same as "the integer part of". > Possibility a is easy taken care of by documentation, especially when the > behavior is what people expect. I'll grant that the behavior is relatively easily learned. It'll take some more evidence to convince me that it's what people expect, given that it _looks_ like a type conversion, and many systems implement that conversion by rounding. > Possibility b conflicts with what I > believe to be both the plain English meaning of 'integer part' and what I > believe Americans, at least, learn in elementary school. You may understand the same thing from "int" and "integer part", but that's a learned association that I didn't have before either you or Raymond brought it up. Among their other differences, "int" is not an English word. > Moreover, I challenge the notion that 'truncate' is unambiguous. It simply > means to cut off, without specifying how much. '3.141' is pi truncated to > 3 decimal places. Indeed, it would not be unreasonable to expect trunc() > to have a second parameter specifying where to cut. No more unreasonable than round() having the same second parameter. And it would be reasonable to default it to "0" as round() does, since that's the only distinguished point in its range. > For one data point, I asked my bright 13-year-old for the 'integer part' of > 3.1, 3.9, -3.1, and -3.9 and got the expected 3,3,-3,-3 (as with int()). > But asking 'truncate' the same numbers or even 'truncate toward zero' got a > blank stare, which did not surprise me too much as it is not a common > (American) English word. You asked a different question than the one we're talking about. You should have asked your 13-year-old what the "int" of 3.9 was, or, even better, how to convert 3.9 to an integer. For the first, you'd likely have gotten the same blank stare. For the second, I expect you'd have gotten either 4, or an objection that it's simply not an integer and can't be "converted" to one. -- Namast?, Jeffrey Yasskin From martin at v.loewis.de Tue Jan 29 07:01:01 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 29 Jan 2008 07:01:01 +0100 Subject: [Python-Dev] Upcoming 2.4.5 and 2.3.7 releases In-Reply-To: References: <479CDB1F.60809@v.loewis.de> <0124660D-3CF9-48BF-8DB1-B71763B1D2EC@python.org> <479E4778.7010500@v.loewis.de> <479E4D76.1080205@python.org> <479E526C.3020100@v.loewis.de> Message-ID: <479EC11D.8030706@v.loewis.de> > What do you think of the above? Sounds fine to me. I won't touch this then for the moment, please let me know when you are done rearranging things. Regards, Martin From rrr at ronadam.com Tue Jan 29 07:37:33 2008 From: rrr at ronadam.com (Ron Adam) Date: Tue, 29 Jan 2008 00:37:33 -0600 Subject: [Python-Dev] functions vs methods (was Re: trunc()) In-Reply-To: <20080128185647.AFW48175@ms19.lnh.mail.rcn.net> References: <20080128185647.AFW48175@ms19.lnh.mail.rcn.net> Message-ID: <479EC9AD.7000008@ronadam.com> Raymond Hettinger wrote: > Please ignore my last email. The idea for combining trunc, ceil, > floor, etc was probably just a distractor. I was going to suggest the same round(n, mode="round_method_of choice") to both round and int(), where int is more forgiving for input. But wasn't convinced. > [GvR] >> One thing I'm beginning to feel more and more strongly about >> is that round, trunc, ceil and floor all belong in the same >> category, and either should all be builtins or should all >> be in math. > > +1 for all four going into the math module. Works for me. It seems simple enough to just do ... from math import round_even as round This gives me the specific rounding behavior I want without a lot of noise. Ron >> I should also admit that the 2-arg version of round() was >> borrowed from ABC, but the use case for it there doesn't map >> to Python: in ABC it's the only tool you have for floating point >> formatting on output, while in Python the same thing would >> typically be done with "%.2f" % x rather than round(x, 2). > > +1 for deprecating the second argument to round(). > > > Raymond > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/rrr%40ronadam.com > From asmodai at in-nomine.org Tue Jan 29 09:26:51 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Tue, 29 Jan 2008 09:26:51 +0100 Subject: [Python-Dev] Incorrect documentation of the raw_input built-in function In-Reply-To: <479E6054.3020204@canterbury.ac.nz> References: <18328.49667.162531.446063@montanaro.dyndns.org> <479D93D7.2050206@canterbury.ac.nz> <20080128185016.GL55130@nexus.in-nomine.org> <479E6054.3020204@canterbury.ac.nz> Message-ID: <20080129082650.GE75774@nexus.in-nomine.org> -On [20080129 00:13], Greg Ewing (greg.ewing at canterbury.ac.nz) wrote: >What document did this come from? This sounds more like it's >talking about what should be described in various sections of >a man page, not what should be written to the various streams >by a program. It did, it's from the C. Rationale for Shell and Utilities (XCU). >Otherwise, > > > a message indicating that the >> utility had insufficient memory in which to operate would not be described. > >sounds like an out-of-memory message shouldn't be written to >stderr, which I'm fairly sure is not the case! It's only talking about what will be described in the various sections of the utilities' documentation, yes. But it was the closest I could find to *anything* explicitly mentioning stdin, stdout, or stderr and their function. I could find nothing with regard to prompting or other such descriptions. So, don't count on POSIX to mandate anything on this (as far as I can tell). -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ We have met the enemy and they are ours... From jon+python-dev at unequivocal.co.uk Tue Jan 29 11:45:13 2008 From: jon+python-dev at unequivocal.co.uk (Jon Ribbens) Date: Tue, 29 Jan 2008 10:45:13 +0000 Subject: [Python-Dev] trunc() In-Reply-To: References: <005f01c86228$a7b41530$6800a8c0@RaymondLaptop1> Message-ID: <20080129104513.GE30325@snowy.squish.net> On Mon, Jan 28, 2008 at 08:07:21PM -0800, Guido van Rossum wrote: > PS. There's something wrong with Raymond's mailer that creates a > thread in gmail whenever he responds. I suspect it's not correctly > adding an In-reply-to header. That makes the thread feel much more > disconnected than most, because often the quoted text is in a > different thread. His mails don't have any indication they're a reply at all - not even so much as a "Re: " in the Subject. Even Outlook Express isn't *that* broken; I suspect he's not actually using the 'reply' button in his mailer. From steve at holdenweb.com Tue Jan 29 13:14:01 2008 From: steve at holdenweb.com (Steve Holden) Date: Tue, 29 Jan 2008 07:14:01 -0500 Subject: [Python-Dev] trunc() In-Reply-To: <5d44f72f0801282013q37e8d4aas1204fa5261e4f060@mail.gmail.com> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> <5d44f72f0801282013q37e8d4aas1204fa5261e4f060@mail.gmail.com> Message-ID: <479F1889.90409@holdenweb.com> Jeffrey Yasskin wrote: [...] > Just like set(sequence) is the set associated with that sequence, not > the set part of that sequence, and float('3.14') is the float > associated with '3.14', not the float part of '3.14', etc. Type names > do not normally retrieve pieces of other objects. >>> type(object) >>> list({1:2, 3:4}) [1, 3] >>> set({1:2, 3:4}) set([1, 3]) >>> [...] Surely the real issue here is that int() grew up purely as a conversion function, and metamorphosed into a type when the classic classes were moved into the background. A brief scan of the 2.4 library (the nearest to hand) shows no uses of int() without an argument in the top level modules. There's clearly no point calling int() with a literal integer argument, so its uses for conversion clearly dominate its use as a pure type constructor. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Tue Jan 29 13:14:01 2008 From: steve at holdenweb.com (Steve Holden) Date: Tue, 29 Jan 2008 07:14:01 -0500 Subject: [Python-Dev] trunc() In-Reply-To: <5d44f72f0801282013q37e8d4aas1204fa5261e4f060@mail.gmail.com> References: <20080124161154.AFQ79357@ms19.lnh.mail.rcn.net> <79990c6b0801250553l2e6247adudf48112436dcda70@mail.gmail.com> <5d44f72f0801261442h47012cb1ocac1ee2966879f2e@mail.gmail.com> <005701c8606e$5196b740$6800a8c0@RaymondLaptop1> <012401c860b4$3a30b6f0$6800a8c0@RaymondLaptop1> <5d44f72f0801282013q37e8d4aas1204fa5261e4f060@mail.gmail.com> Message-ID: <479F1889.90409@holdenweb.com> Jeffrey Yasskin wrote: [...] > Just like set(sequence) is the set associated with that sequence, not > the set part of that sequence, and float('3.14') is the float > associated with '3.14', not the float part of '3.14', etc. Type names > do not normally retrieve pieces of other objects. >>> type(object) >>> list({1:2, 3:4}) [1, 3] >>> set({1:2, 3:4}) set([1, 3]) >>> [...] Surely the real issue here is that int() grew up purely as a conversion function, and metamorphosed into a type when the classic classes were moved into the background. A brief scan of the 2.4 library (the nearest to hand) shows no uses of int() without an argument in the top level modules. There's clearly no point calling int() with a literal integer argument, so its uses for conversion clearly dominate its use as a pure type constructor. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Tue Jan 29 13:17:30 2008 From: steve at holdenweb.com (Steve Holden) Date: Tue, 29 Jan 2008 07:17:30 -0500 Subject: [Python-Dev] trunc() In-Reply-To: <20080129104513.GE30325@snowy.squish.net> References: <005f01c86228$a7b41530$6800a8c0@RaymondLaptop1> <20080129104513.GE30325@snowy.squish.net> Message-ID: <479F195A.20603@holdenweb.com> Jon Ribbens wrote: > On Mon, Jan 28, 2008 at 08:07:21PM -0800, Guido van Rossum wrote: >> PS. There's something wrong with Raymond's mailer that creates a >> thread in gmail whenever he responds. I suspect it's not correctly >> adding an In-reply-to header. That makes the thread feel much more >> disconnected than most, because often the quoted text is in a >> different thread. > > His mails don't have any indication they're a reply at all - not even > so much as a "Re: " in the Subject. Even Outlook Express isn't *that* > broken; I suspect he's not actually using the 'reply' button in his > mailer. Raymond, I don;t know why everyone's talking about you as though you weren't reading the thread and able to answer questions. Though I *have* sometimes wondered why your posts lack attributions for the quotes. I see from your headers you have, at least some of the time, been posting via gmane using "mirapoint webmail direct", whatever that is. Perhaps that's the problem? sticking-with-t'bird-despite-its-faults-ly y'rs - steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Tue Jan 29 13:17:30 2008 From: steve at holdenweb.com (Steve Holden) Date: Tue, 29 Jan 2008 07:17:30 -0500 Subject: [Python-Dev] trunc() In-Reply-To: <20080129104513.GE30325@snowy.squish.net> References: <005f01c86228$a7b41530$6800a8c0@RaymondLaptop1> <20080129104513.GE30325@snowy.squish.net> Message-ID: <479F195A.20603@holdenweb.com> Jon Ribbens wrote: > On Mon, Jan 28, 2008 at 08:07:21PM -0800, Guido van Rossum wrote: >> PS. There's something wrong with Raymond's mailer that creates a >> thread in gmail whenever he responds. I suspect it's not correctly >> adding an In-reply-to header. That makes the thread feel much more >> disconnected than most, because often the quoted text is in a >> different thread. > > His mails don't have any indication they're a reply at all - not even > so much as a "Re: " in the Subject. Even Outlook Express isn't *that* > broken; I suspect he's not actually using the 'reply' button in his > mailer. Raymond, I don;t know why everyone's talking about you as though you weren't reading the thread and able to answer questions. Though I *have* sometimes wondered why your posts lack attributions for the quotes. I see from your headers you have, at least some of the time, been posting via gmane using "mirapoint webmail direct", whatever that is. Perhaps that's the problem? sticking-with-t'bird-despite-its-faults-ly y'rs - steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From barry at python.org Tue Jan 29 13:32:58 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 29 Jan 2008 07:32:58 -0500 Subject: [Python-Dev] Upcoming 2.4.5 and 2.3.7 releases In-Reply-To: <479EC11D.8030706@v.loewis.de> References: <479CDB1F.60809@v.loewis.de> <0124660D-3CF9-48BF-8DB1-B71763B1D2EC@python.org> <479E4778.7010500@v.loewis.de> <479E4D76.1080205@python.org> <479E526C.3020100@v.loewis.de> <479EC11D.8030706@v.loewis.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 29, 2008, at 1:01 AM, Martin v. L?wis wrote: >> What do you think of the above? > > Sounds fine to me. I won't touch this then for the moment, > please let me know when you are done rearranging things. Cool, I'll try to get to this today. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBR58c+nEjvBPtnXfVAQI1XgP/S9YUtkAC/x77alsmBYXtaoGZNrQ/QvtR bziE1zzQ/Luexe9nt41JEaSbVyWb1dw0KZvpoDzqhqbLfhYAXCvJVFIlbw0U94bh pf1G42L3U1YauzzUQL1Q9g0IT4jRMc3D3G8hwX4iOECEW8uHla9uAiqSl5r/V5af K8BcRtkzz0Q= =8tA5 -----END PGP SIGNATURE----- From barry at python.org Tue Jan 29 15:15:04 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 29 Jan 2008 09:15:04 -0500 Subject: [Python-Dev] Upcoming 2.4.5 and 2.3.7 releases In-Reply-To: <479EC11D.8030706@v.loewis.de> References: <479CDB1F.60809@v.loewis.de> <0124660D-3CF9-48BF-8DB1-B71763B1D2EC@python.org> <479E4778.7010500@v.loewis.de> <479E4D76.1080205@python.org> <479E526C.3020100@v.loewis.de> <479EC11D.8030706@v.loewis.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 29, 2008, at 1:01 AM, Martin v. L?wis wrote: >> What do you think of the above? > > Sounds fine to me. I won't touch this then for the moment, > please let me know when you are done rearranging things. All done! Thanks. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBR5806HEjvBPtnXfVAQKGTgQAkPUgEssRQOYCEEhAshQcO3hYm7I47Tdk uY1TG7a7hKd0kVksWsRlJCDOHUrxay0EORxf++IJyht+xkSUo50Yq6nAM0cx1xWA mjupkNu4ChwLpvfI+2Q6Haz6Ws30e0M5YtZeH1XJlFXIaPmwF44HEsBmVGxf2EYO ihFY6Xt5M3A= =G22Z -----END PGP SIGNATURE----- From lists at cheimes.de Tue Jan 29 14:49:50 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 29 Jan 2008 14:49:50 +0100 Subject: [Python-Dev] Py2.6 release schedule In-Reply-To: <479E3AD6.5020705@enthought.com> References: <20080128135028.AFV97486@ms19.lnh.mail.rcn.net> <479E2DFA.8040904@cheimes.de> <479E3AD6.5020705@enthought.com> Message-ID: <479F2EFE.1050904@cheimes.de> Travis E. Oliphant wrote: > Yes, I will. What are your time-lines? I've been targeting first week > in March. I like to port bytearray to 2.6 as early as possible. I'd be grateful if you could port a limited subset of the new buffer protocol within the next few weeks. bytearray needs: PyBuffer_FillInfo PyObject_ReleaseBuffer PyObject_GetBuffer PyBuffer_ToContiguous PyObject_CheckBuffer PyExc_BufferError Christian From python at rcn.com Tue Jan 29 20:04:50 2008 From: python at rcn.com (Raymond Hettinger) Date: Tue, 29 Jan 2008 14:04:50 -0500 (EST) Subject: [Python-Dev] Slot for __trunc__ Message-ID: <20080129140450.AFX81853@ms19.lnh.mail.rcn.net> Should the implementation of __trunc__ have its own slot like we have for nb_index? Raymond ------------------------------------------------------- rhettinger at localhost ~/py26/Objects $ grep "__trunc__" *.c floatobject.c: {"__trunc__", (PyCFunction)float_trunc, METH_NOARGS, intobject.c: {"__trunc__", (PyCFunction)int_int, METH_NOARGS, longobject.c: {"__trunc__", (PyCFunction)long_long, METH_NOARGS, From guido at python.org Tue Jan 29 20:17:07 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 29 Jan 2008 11:17:07 -0800 Subject: [Python-Dev] Slot for __trunc__ In-Reply-To: <20080129140450.AFX81853@ms19.lnh.mail.rcn.net> References: <20080129140450.AFX81853@ms19.lnh.mail.rcn.net> Message-ID: I don't see why. __index__ has a slot because its primary use is to be called from C code, where slots add a slight performance advantage. __trunc__ doesn't get called from C AFAIK. On Jan 29, 2008 11:04 AM, Raymond Hettinger wrote: > Should the implementation of __trunc__ have its own slot like we have for nb_index? > > Raymond > > > ------------------------------------------------------- > rhettinger at localhost ~/py26/Objects $ grep "__trunc__" *.c > floatobject.c: {"__trunc__", (PyCFunction)float_trunc, METH_NOARGS, > intobject.c: {"__trunc__", (PyCFunction)int_int, METH_NOARGS, > longobject.c: {"__trunc__", (PyCFunction)long_long, METH_NOARGS, > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Tue Jan 29 20:34:15 2008 From: python at rcn.com (Raymond Hettinger) Date: Tue, 29 Jan 2008 14:34:15 -0500 (EST) Subject: [Python-Dev] Slot for __trunc__ Message-ID: <20080129143415.AFX88089@ms19.lnh.mail.rcn.net> [GvR] > I don't see why. __index__ has a slot because its > primary use is to be called from C code, where slots > add a slight performance advantage. > __trunc__ doesn't get called from C AFAIK. I thought the __trunc__ method only gets called from the C code for the trunc() function which is currently implemented with PyObject_CallMethod(number, "__trunc__", "") instead of a fast call to a slot. Raymond From eric+python-dev at trueblade.com Tue Jan 29 20:37:43 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Tue, 29 Jan 2008 14:37:43 -0500 Subject: [Python-Dev] Slot for __trunc__ In-Reply-To: <20080129143415.AFX88089@ms19.lnh.mail.rcn.net> References: <20080129143415.AFX88089@ms19.lnh.mail.rcn.net> Message-ID: <479F8087.9040303@trueblade.com> Raymond Hettinger wrote: > [GvR] >> I don't see why. __index__ has a slot because its >> primary use is to be called from C code, where slots >> add a slight performance advantage. >> __trunc__ doesn't get called from C AFAIK. > > I thought the __trunc__ method only gets called from > the C code for the trunc() function which is currently > implemented with PyObject_CallMethod(number, "__trunc__", "") > instead of a fast call to a slot. And if __trunc__ qualifies, what about __format__, which is similar? I'm not pushing for it, I just wonder how the decision is made. Eric. From guido at python.org Tue Jan 29 21:04:15 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 29 Jan 2008 12:04:15 -0800 Subject: [Python-Dev] Slot for __trunc__ In-Reply-To: <20080129143415.AFX88089@ms19.lnh.mail.rcn.net> References: <20080129143415.AFX88089@ms19.lnh.mail.rcn.net> Message-ID: On Jan 29, 2008 11:34 AM, Raymond Hettinger wrote: > [GvR] > > I don't see why. __index__ has a slot because its > > primary use is to be called from C code, where slots > > add a slight performance advantage. > > __trunc__ doesn't get called from C AFAIK. > > I thought the __trunc__ method only gets called from > the C code for the trunc() function which is currently > implemented with PyObject_CallMethod(number, "__trunc__", "") > instead of a fast call to a slot. I see. Well, it would bounce around a bit but it would never execute Python byte codes. I don't see trunc() being all that performance critical. The cost of adding a new slot is considerable -- for one, *all* type objects become 4 (or 8) bytes longer. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Tue Jan 29 21:54:16 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 29 Jan 2008 21:54:16 +0100 Subject: [Python-Dev] Upcoming 2.4.5 and 2.3.7 releases In-Reply-To: <5E6A9FAF-F3BA-4281-9053-3BD986B79C69@acm.org> References: <479CDB1F.60809@v.loewis.de> <0124660D-3CF9-48BF-8DB1-B71763B1D2EC@python.org> <479E4778.7010500@v.loewis.de> <479E4D76.1080205@python.org> <5E6A9FAF-F3BA-4281-9053-3BD986B79C69@acm.org> Message-ID: <479F9278.7020109@v.loewis.de> > Releasing the email package from the Python maintenance branches is > probably insane. People seem to use a lot of strong language lately. >From m-w.com: insane (adjective) 1. mentally disordered 3. absurd == ridiculously unreasonable > This kind of thing would be less of a problem if the standard library > were smaller, and the email package only available separately. It's > also why some of us want a smaller standard library. I see no problem with the email package being maintained in the Python trunk, per se, and I don't think it should be removed from the standard library at all. Regards, Martin From mike.klaas at gmail.com Wed Jan 30 01:08:30 2008 From: mike.klaas at gmail.com (Mike Klaas) Date: Tue, 29 Jan 2008 16:08:30 -0800 Subject: [Python-Dev] 2.5.2 release coming up In-Reply-To: References: Message-ID: On 22-Jan-08, at 8:47 PM, Guido van Rossum wrote: > While the exact release schedule for 2.5.2 is still up in the air, I > expect that it will be within a few weeks. This means that we need to > make sure that anything that should go into 2.5.2 goes in ASAP, > preferably this week. It also means that we should be very careful > what goes in though -- and we should be paying particular attention to > stability on all platforms! Fortunately it looks like quite a few 2.5 > buildbots are green: http://python.org/dev/buildbot/2.5/ > > I propose that anything that ought to go into 2.5.2 (or should be > reviewed for suitability to go into it) should be marked "urgent" in > the tracker, *and* have its version set to (or include) "Python 2.5". I'm not sure if it is particularly urgent because of the rarity of occurrence, but I discovered a bug that causes httplib to hang indefinitely given some rarely-occurring input in the wild. To reproduce: python -c 'import urllib2; urllib2.urlopen("http:// www.hunteros.com").read()' WARNING: the page was tagged by one of our users and is definitely NSFW. Again, it seems to occur very rarely, but the behaviour is quite painful and the fix trivial (see http://bugs.python.org/issue1966). Thanks, -Mike From theller at ctypes.org Wed Jan 30 09:20:17 2008 From: theller at ctypes.org (Thomas Heller) Date: Wed, 30 Jan 2008 09:20:17 +0100 Subject: [Python-Dev] pep3118, ctypes, py3k Message-ID: <47A03341.8060701@ctypes.org> Hi Travis, I have started the pep3118 implementation for ctypes. If you have time, could you please review , especially the tests in file Lib/ctypes/test/test_pep3118.py to ensure that I have understood and implemented the pep correctly? Thanks, Thomas From ndbecker2 at gmail.com Wed Jan 30 14:15:09 2008 From: ndbecker2 at gmail.com (Neal Becker) Date: Wed, 30 Jan 2008 08:15:09 -0500 Subject: [Python-Dev] ctypes issue Message-ID: In python 2.5.1: from ctypes import * c = c_int(4) print c == 4 print c.value == 4 False True This seem unreasonable to me, and a big invitation to error. From jcea at argo.es Wed Jan 30 14:39:07 2008 From: jcea at argo.es (Jesus Cea) Date: Wed, 30 Jan 2008 14:39:07 +0100 Subject: [Python-Dev] TIOBE Programming Community Index In-Reply-To: References: <18331.7277.777931.189775@montanaro.dyndns.org> Message-ID: <47A07DFB.7000902@argo.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Steve Holden wrote: |> Sorry if this is a repeat. Published also in Slashdot. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at argo.es http://www.argo.es/~jcea/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBR6B9+plgi5GaxT1NAQIWIAP+L0CZdJedSywHWR6h0+tcRQrtmgVRfYH1 WA1PoWBqcP943rYENfK/fiaEh/D+QSrDVDcWktfN+sAP+9UuBkMb0euM4NEtdbqt K4ow+N2By6quBNKey/S2Ngr0cgnCk45B1UstD63S72+dtOvJ9bIBn1XMEhu7ANzJ MPhvHZBtqGY= =201s -----END PGP SIGNATURE----- From jcea at argo.es Wed Jan 30 14:41:33 2008 From: jcea at argo.es (Jesus Cea) Date: Wed, 30 Jan 2008 14:41:33 +0100 Subject: [Python-Dev] Upcoming 2.5.2 release In-Reply-To: <479CD349.6060604@v.loewis.de> References: <479CD349.6060604@v.loewis.de> Message-ID: <47A07E8D.7000500@argo.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martin v. L?wis wrote: | It is my pleasure to announce that I have been appointed as the release | manager for the upcoming releases. For 2.5.2, I would like to release | a candidate on February 14, and the final version on February 21. As | Guido already mentioned, this will be a bug fix release only; no new | features allowed. As current bsddb module maintainer, I was wondering if 2.5.2 will support BerkeleyDB 4.6 :-?. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at argo.es http://www.argo.es/~jcea/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBR6B+jZlgi5GaxT1NAQIX+QQAl7PFJtZb64kiTpewupJppos5mAXFKB29 lrud1t6AkgFOgvE9M45ISCwTDzbSa4qLOiDX/v5F9RltcH+RK5Tlu3eTR8Djmn6q PSzjo92LZx7wudq2WaBL2+PIE/IhAOO9gKHtwNtB3iXQGTl8NKIIJUbkDUclvPPv bH97ECQvb7U= =yV47 -----END PGP SIGNATURE----- From jcea at argo.es Wed Jan 30 17:49:20 2008 From: jcea at argo.es (Jesus Cea) Date: Wed, 30 Jan 2008 17:49:20 +0100 Subject: [Python-Dev] Assigning issues Message-ID: <47A0AA90.1060701@argo.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I just trying to open a new bug in bssdb module and assign it to me :-). Seems I have no permissions to do that :-). The issue is http://bugs.python.org/issue1976 - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at argo.es http://www.argo.es/~jcea/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBR6CqkJlgi5GaxT1NAQJ4VAQAoa7FGC7OxeNiQUDKsjaZ9Q6gtOBXtuJp 0O31VllV82eINCKU2BA/XAHTik/CxvvVRPjLElumln3Lccj++8IVeSgHoPntDZuc 7kJhrD+OZXTPWkV4/g+sgepycF+wNr2ICM2o75ABz2CuyXNYmLum0fH7UPUORi+U V921rj9Kzss= =0Soy -----END PGP SIGNATURE----- From martin at v.loewis.de Wed Jan 30 21:04:48 2008 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 30 Jan 2008 21:04:48 +0100 Subject: [Python-Dev] Upcoming 2.5.2 release In-Reply-To: <47A07E8D.7000500@argo.es> References: <479CD349.6060604@v.loewis.de> <47A07E8D.7000500@argo.es> Message-ID: <47A0D860.5030108@v.loewis.de> > | It is my pleasure to announce that I have been appointed as the release > | manager for the upcoming releases. For 2.5.2, I would like to release > | a candidate on February 14, and the final version on February 21. As > | Guido already mentioned, this will be a bug fix release only; no new > | features allowed. > > As current bsddb module maintainer, I was wondering if 2.5.2 will > support BerkeleyDB 4.6 :-?. Maybe I'm misunderstanding the question - whom are asking? If me - Python 2.5.2 will essentially do what the maintenance branch does currently. Regards, Martin From brett at python.org Wed Jan 30 22:56:41 2008 From: brett at python.org (Brett Cannon) Date: Wed, 30 Jan 2008 13:56:41 -0800 Subject: [Python-Dev] Assigning issues In-Reply-To: <47A0AA90.1060701@argo.es> References: <47A0AA90.1060701@argo.es> Message-ID: On Jan 30, 2008 8:49 AM, Jesus Cea wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > I just trying to open a new bug in bssdb module and assign it to me :-). > Seems I have no permissions to do that :-). > > The issue is http://bugs.python.org/issue1976 > We have not worked out any policy on this, but I always assumed we would only assign issues to people with commit privileges. I quick check suggests that you don't have them, Jesus. We can give you the rights to modify issues and set the assignment, but I don't know if you should be able to be assigned a bug yet. What do other people think? -Brett > - -- > Jesus Cea Avion _/_/ _/_/_/ _/_/_/ > jcea at argo.es http://www.argo.es/~jcea/ _/_/ _/_/ _/_/ _/_/ _/_/ > jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ > ~ _/_/ _/_/ _/_/ _/_/ _/_/ > "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ > "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ > "El amor es poner tu felicidad en la felicidad de otro" - Leibniz > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.6 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iQCVAwUBR6CqkJlgi5GaxT1NAQJ4VAQAoa7FGC7OxeNiQUDKsjaZ9Q6gtOBXtuJp > 0O31VllV82eINCKU2BA/XAHTik/CxvvVRPjLElumln3Lccj++8IVeSgHoPntDZuc > 7kJhrD+OZXTPWkV4/g+sgepycF+wNr2ICM2o75ABz2CuyXNYmLum0fH7UPUORi+U > V921rj9Kzss= > =0Soy > -----END PGP SIGNATURE----- > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org > From ncoghlan at gmail.com Wed Jan 30 23:20:52 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 31 Jan 2008 08:20:52 +1000 Subject: [Python-Dev] Assigning issues In-Reply-To: References: <47A0AA90.1060701@argo.es> Message-ID: <47A0F844.9080504@gmail.com> Brett Cannon wrote: > On Jan 30, 2008 8:49 AM, Jesus Cea wrote: >> I just trying to open a new bug in bssdb module and assign it to me :-). >> Seems I have no permissions to do that :-). > > We have not worked out any policy on this, but I always assumed we > would only assign issues to people with commit privileges. I quick > check suggests that you don't have them, Jesus. We can give you the > rights to modify issues and set the assignment, but I don't know if > you should be able to be assigned a bug yet. > > What do other people think? Given that everyone is free to work on whichever issues are of interest to them (and post comments to the relevant tracker items), I've always seen the assignment field as a way to request that a specific committer either fix the problem themselves, or review a proposed solution and either suggest improvements or check it in. I think you're right that there isn't really an official policy written down anywhere though - PEP 3 doesn't go into that kind of detail. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From martin at v.loewis.de Thu Jan 31 06:08:46 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 31 Jan 2008 06:08:46 +0100 Subject: [Python-Dev] Assigning issues In-Reply-To: References: <47A0AA90.1060701@argo.es> Message-ID: <47A157DE.5010004@v.loewis.de> > We have not worked out any policy on this, but I always assumed we > would only assign issues to people with commit privileges. I quick > check suggests that you don't have them, Jesus. We can give you the > rights to modify issues and set the assignment, but I don't know if > you should be able to be assigned a bug yet. > > What do other people think? That has exactly been the policy that I have been executing, yes. As for giving additional rights on a per-user basis - that's tricky to implement. We have the User and Developer roles in the tracker, and you get additional rights only with additional roles, normally. I also agree with Nick as to what the purpose of assignments is. To indicate that you are working on a specific issue, a message saying so is enough (which could also include estimated completion dates, which a mere self-assignment can't). Regards, Martin From kevin at bud.ca Thu Jan 31 06:00:58 2008 From: kevin at bud.ca (Kevin Teague) Date: Wed, 30 Jan 2008 21:00:58 -0800 Subject: [Python-Dev] Monkeypatching idioms -- elegant or ugly? Message-ID: <640554BC-544A-4CB6-BB20-EF544B120954@bud.ca> +1 on having established Python idioms for these techniques. While I don't know if there has ever been a formal definition of monkey patch, the term monkey patch came from guerilla patch, which came from two or more dynamic modifications to a class interfering with each other. These modifications were usually made by extension code (Zope add-on Products) to upstream code (the Zope framework), so I would define a monkey patch only as dynamic modifications made to a class with the *intent to change or correct behaviour in upstream code*. The term has also caught on with the a second definition of referring to any dynamic modification of class, regardless of intent though. I would perhaps call these methods something like: * add_method_to_class * extend_class This gives you a better idea of what they do, rather than use a term with a somewhat ambigous definition. With monkeypatch_method under the definition of "altering existing upstream behviour", I might expect it to raise an error if the method I was replacing on a class did not exist (e.g. upstream code was refactored so my patch no longer applied). From mhammond at skippinet.com.au Thu Jan 31 07:25:18 2008 From: mhammond at skippinet.com.au (Mark Hammond) Date: Thu, 31 Jan 2008 17:25:18 +1100 Subject: [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <022401c85cde$678bcf10$36a36d30$@com.au> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <022401c85cde$678bcf10$36a36d30$@com.au> Message-ID: <083301c863d2$12dc2b40$389481c0$@com.au> I'm wondering if anyone has any comment on this issue? Its currently impossible to use distutils as documented to build x64 extensions, either natively or cross-compiled using the trunk and while I'm keen to help fix this, I'd like agreement on the direction. Can I assume silence means general agreement on the compromise proposal I outlined? Thanks, Mark > -----Original Message----- > From: python-dev-bounces+mhammond=keypoint.com.au at python.org > [mailto:python-dev-bounces+mhammond=keypoint.com.au at python.org] On > Behalf Of Mark Hammond > Sent: Tuesday, 22 January 2008 9:06 PM > To: '"Martin v. L?wis"' > Cc: distutils-sig at python.org; python-dev at python.org > Subject: Re: [Python-Dev] Adventures with x64, VS7 and VS8 on Windows > > Hi Martin, > > Way back on Monday, 21 May 2007, you wrote: > > > This is an issue to be discussed for Python 2.6. I'm personally > > hesitant to have the "official" build infrastructure deviate > > from the layout that has been in-use for so many years, as a lot > > of things depend on it. > > > > I don't find the need to have separate object directories convincing: > > For building the Win32/Win64 binaries, I have separate checkouts > > *anyway*, since all the add-on libraries would have to support > > multi-arch builds, but I think they don't. > ... > > > * Re the x64 directory used by the PCBuild8 process. IMO, it makes > > > sense to generate x64 binaries to their own directory - my > expectation > > > is that cross-compiling between platforms is a reasonable use-case, > > > and we should support multiple achitectures for the same compiler > > > version. > > > > See above; I disagree. > > [For additional context; the question was regarding where the output > binaries were created for each platform, and specifically if x64 build > should use something like 'PCBuild/x64'] > > I'm bringing this topic up again as I noticed the AMD64 builds for > Python > 2.6 create their output in the PCBuild/x64 directory, not directly into > the > PCBuild directory. When I raised this with Christian, he also > expressed a > desire to be able to cross-compile in the same directory structure and > was > unaware of this discussion (but I made him aware of it :) > > You made excellent points about how tools currently recognize the > PCBuild > directory, and we can't break them, and I agree. I'd like to propose a > compromise that might be able to keep everyone happy. > > The main problem I see with all platforms using 'PCBuild' is that in a > cross-compilation scenario, it is impossible for the distutils to > determine > the location of the correct Python libraries to link with (stuff in its > own > PYTHONHOME is no good; it's for the wrong platform). Obviously, we can > change the distutils so that the location of the correct libraries can > be > specified, but that implies all other build systems that currently > assume > PCBuild must also change to work in a cross-compilation scenario. > While > those external tools *will* work today in a native build on x64, > eventually > they may need to be upgraded to deal with cross-compilation - and this > means > that all such tools will also be unable to automatically determine the > location of the libraries. They will all need to take the library dir > as a > user-specified option, which seems a pain (eg, buildbots would seem to > need > site-specific configuration information to make this work). If > eventually > all such tools are likely to upgrade, it would be ideal if we can offer > them > a way to upgrade so the correct libraries can be determined > automatically, > as they can now for native builds. > > My compromise proposal is this: Each platform builds in its own > directory > (ie, PCBuild/x86_64, PCBuild/x86). Every single build configuration > has a > post-build step that copies the result of the build to the PCBuild > directory. We then add an option to the distutils so that a cross- > compile > platform can be specified and when it is, to use 'PCBuild/{platform}" > instead of PCBuild, and we encourage other tools to use the same > directory > should they want to support "configure-less" cross-compilation (if > distutils > on the trunk is not still trying to maintain b/w compat, it should just > *always* look in PCBuild/{platform}, and I'd suggest py3k not bother > with > PCBuild at all; build systems will be touched to upgrade to py3k, so > that > change isn't painful. But I digress :) > > The main advantage of the compromise is that it allows for the status > quo to > remain in place - just continue to use separate source trees for each > platform, and only ever build a single platform in each tree, and tools > are > free to have ways of specifying which directory should be used. The > official build process need not change. However, it also supports a > way to > do cross-compilation in a way that build tools can *automatically* > locate > the correct platform's libraries, and this will be particularly useful > for > extension authors. > > Would something like that be acceptable? > > Thanks, > > Mark > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python- > dev/mhammond%40keypoint.com.au From jcea at argo.es Thu Jan 31 10:42:21 2008 From: jcea at argo.es (Jesus Cea) Date: Thu, 31 Jan 2008 10:42:21 +0100 Subject: [Python-Dev] Upcoming 2.5.2 release In-Reply-To: <47A0D860.5030108@v.loewis.de> References: <479CD349.6060604@v.loewis.de> <47A07E8D.7000500@argo.es> <47A0D860.5030108@v.loewis.de> Message-ID: <47A197FD.9080001@argo.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martin v. L?wis wrote: |> As current bsddb module maintainer, I was wondering if 2.5.2 will |> support BerkeleyDB 4.6 :-?. | | Maybe I'm misunderstanding the question - whom are asking? | If me - Python 2.5.2 will essentially do what the maintenance branch | does currently. I beg your pardon. My role is recent (a week) and I'm still learning my way thru procedures and conventions :-). Current bsddb module in 2.5.1 supports up to BerkeleyDB 4.5. There is support for 4.6 in trunk (future 2.6, I guess) and I'm working in a private branch at the moment, since I have no commit access to python repository. That private version is intented to be merged into python 2.6 by Greg, when time comes. My guess is that 2.5 branch is still open to more patches than pure security/stability patches, so "backporting" BerkeleyDB 4.6 support seems reasonable (to me). If I'm wrong, please educate me :-). This backport would include also stability patches. For example, I just solved a memory leak. Greg, any opinion?. Beware, new kid in the block! :). - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at argo.es http://www.argo.es/~jcea/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBR6GX/Zlgi5GaxT1NAQLZfQP/Zl95IiH9wyIaVA2K2ulxnQG6Su7OKASM p4Ej3c2/1JHxprabXixDdn2i8fPZJ+02qMbJIxmWhupvW5TpHsoH4Rrs0QV/+SpD LFvvIgVruJCaVgZUFEoqOoRA07OwxRwg6tgPLPwsVcKZISTBRGRBiARply83vaDz hjjFA5abVPU= =Jti3 -----END PGP SIGNATURE----- From jcea at argo.es Thu Jan 31 10:50:54 2008 From: jcea at argo.es (Jesus Cea) Date: Thu, 31 Jan 2008 10:50:54 +0100 Subject: [Python-Dev] Assigning issues In-Reply-To: <47A157DE.5010004@v.loewis.de> References: <47A0AA90.1060701@argo.es> <47A157DE.5010004@v.loewis.de> Message-ID: <47A199FE.2010100@argo.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martin v. L?wis wrote: | I also agree with Nick as to what the purpose of assignments is. | To indicate that you are working on a specific issue, a message | saying so is enough (which could also include estimated completion | dates, which a mere self-assignment can't). So, in the concrete case of http://bugs.python.org/issue1976 , patch available, what the next step would be?. Must I contact Greg (previous bsddb maintainer, with python commit access) to review, apply patch (more coming, so no hurry) and tracker update?. Should I maintain bsddb tracking outside python one, to reduce your burden?. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at argo.es http://www.argo.es/~jcea/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBR6GZ/plgi5GaxT1NAQKf6AP/QGhRVE8mfrzFAYz6RRCjF4Z5QXieh2Gu 5aSYsJ/0uksLTitkM/ihK3aiKHsX/PNWJCXskUqNp+RA3JNgTFbCMvL9CcUOH6Vs zEXeiutoBLo2xacqCk3JscNQZKC086WdHhHJptpR1ItegiHGfuzwTPo23qMyfJDC TQ1sjdEVMik= =Ukol -----END PGP SIGNATURE----- From jcea at argo.es Thu Jan 31 11:07:24 2008 From: jcea at argo.es (Jesus Cea) Date: Thu, 31 Jan 2008 11:07:24 +0100 Subject: [Python-Dev] Tracker marks my messages as spam :-) Message-ID: <47A19DDC.7030508@argo.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 This will be my last email today, I don't want to waste (more of) your *valuable* time. http://bugs.python.org/issue1391 http://bugs.python.org/msg61892 - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at argo.es http://www.argo.es/~jcea/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBR6Gd25lgi5GaxT1NAQLDiwP/aMUOxhoRH8/ZnCtHCUzr95tIJUe1ySh6 SuDjR3OS19S8lcRVgEL0droIP44lmozpdyOW1eaPDPBMA02XCqiPWmCxBCeXsbJ/ xf/XVzl53vAQmtfqxHrNyrS+mXv5YW2CjOKWk52IKuf/Rckf9FYSP13OKW7WTjNy orjAdOYRd/8= =gSNB -----END PGP SIGNATURE----- From ncoghlan at gmail.com Thu Jan 31 12:26:20 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 31 Jan 2008 21:26:20 +1000 Subject: [Python-Dev] Tracker marks my messages as spam :-) In-Reply-To: <47A19DDC.7030508@argo.es> References: <47A19DDC.7030508@argo.es> Message-ID: <47A1B05C.3070601@gmail.com> Jesus Cea wrote: > http://bugs.python.org/msg61892 I don't know what's going on there, but it appears to require higher permissions than mine to flag the message as miscategorised (I can read it when I'm logged in, but I don't see anything that would let me mark it as a legitimate message). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From lists at cheimes.de Thu Jan 31 14:25:53 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 31 Jan 2008 14:25:53 +0100 Subject: [Python-Dev] Upcoming 2.5.2 release In-Reply-To: <47A197FD.9080001@argo.es> References: <479CD349.6060604@v.loewis.de> <47A07E8D.7000500@argo.es> <47A0D860.5030108@v.loewis.de> <47A197FD.9080001@argo.es> Message-ID: Jesus Cea wrote: > My guess is that 2.5 branch is still open to more patches than pure > security/stability patches, so "backporting" BerkeleyDB 4.6 support > seems reasonable (to me). If I'm wrong, please educate me :-). I think you are wrong, sorry pal! DB 4.6 support is a new feature. New features must land in the development version(s) of Python, that is Python 2.6 and 3.0. You must change as less code as possible in Python 2.5 to fix a severe problem. Christian From aahz at pythoncraft.com Thu Jan 31 17:07:39 2008 From: aahz at pythoncraft.com (Aahz) Date: Thu, 31 Jan 2008 08:07:39 -0800 Subject: [Python-Dev] Upcoming 2.5.2 release In-Reply-To: <47A197FD.9080001@argo.es> References: <479CD349.6060604@v.loewis.de> <47A07E8D.7000500@argo.es> <47A0D860.5030108@v.loewis.de> <47A197FD.9080001@argo.es> Message-ID: <20080131160738.GA13888@panix.com> On Thu, Jan 31, 2008, Jesus Cea wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Martin v. L?wis wrote: > |> As current bsddb module maintainer, I was wondering if 2.5.2 will > |> support BerkeleyDB 4.6 :-?. > | > | Maybe I'm misunderstanding the question - whom are asking? > | If me - Python 2.5.2 will essentially do what the maintenance branch > | does currently. > > I beg your pardon. My role is recent (a week) and I'm still learning my > way thru procedures and conventions :-). Please read PEP6: http://www.python.org/dev/peps/pep-0006/ The gist is that point releases are purely bugfix releases, because adding features lessens code quality and makes it more difficult to track changes. The big push to stick with PEP6 came from the mistake of adding True/False to Python 2.2.1. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "All problems in computer science can be solved by another level of indirection." --Butler Lampson From aahz at pythoncraft.com Thu Jan 31 17:19:00 2008 From: aahz at pythoncraft.com (Aahz) Date: Thu, 31 Jan 2008 08:19:00 -0800 Subject: [Python-Dev] DEADLINE Feb 4: OSCON 2008 Call for Proposals Message-ID: <20080131161900.GA2002@panix.com> The O'Reilly Open Source Convention (OSCON) is accepting proposals for tutorials and presentations. The submission period ends Feb 4. OSCON 2008 will be in Portland, Oregon July 21-25. For more information and to submit a proposal, see http://conferences.oreilly.com/oscon/ -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "All problems in computer science can be solved by another level of indirection." --Butler Lampson From guido at python.org Thu Jan 31 18:23:26 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 31 Jan 2008 09:23:26 -0800 Subject: [Python-Dev] Monkeypatching idioms -- elegant or ugly? In-Reply-To: <640554BC-544A-4CB6-BB20-EF544B120954@bud.ca> References: <640554BC-544A-4CB6-BB20-EF544B120954@bud.ca> Message-ID: On Jan 30, 2008 9:00 PM, Kevin Teague wrote: > +1 on having established Python idioms for these techniques. > > While I don't know if there has ever been a formal definition of > monkey patch, the term monkey patch came from guerilla patch, which > came from two or more dynamic modifications to a class interfering > with each other. These modifications were usually made by extension > code (Zope add-on Products) to upstream code (the Zope framework), so > I would define a monkey patch only as dynamic modifications made to a > class with the *intent to change or correct behaviour in upstream code*. > > The term has also caught on with the a second definition of referring > to any dynamic modification of class, regardless of intent though. Check out the wikipedia entry too. > I would perhaps call these methods something like: > > * add_method_to_class > > * extend_class I don't like extend because in Java that's how you define a subclass. > This gives you a better idea of what they do, rather than use a term > with a somewhat ambigous definition. With monkeypatch_method under the > definition of "altering existing upstream behviour", I might expect it > to raise an error if the method I was replacing on a class did not > exist (e.g. upstream code was refactored so my patch no longer applied). Funny, several examples mentioned earlier in this thread actually check that the method *doesn't* already exist... -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Jan 31 18:25:18 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 31 Jan 2008 09:25:18 -0800 Subject: [Python-Dev] Assigning issues In-Reply-To: <47A199FE.2010100@argo.es> References: <47A0AA90.1060701@argo.es> <47A157DE.5010004@v.loewis.de> <47A199FE.2010100@argo.es> Message-ID: On Jan 31, 2008 1:50 AM, Jesus Cea wrote: > Martin v. L?wis wrote: > | I also agree with Nick as to what the purpose of assignments is. > | To indicate that you are working on a specific issue, a message > | saying so is enough (which could also include estimated completion > | dates, which a mere self-assignment can't). > > So, in the concrete case of http://bugs.python.org/issue1976 , patch > available, what the next step would be?. Must I contact Greg (previous > bsddb maintainer, with python commit access) to review, apply patch > (more coming, so no hurry) and tracker update?. Yes, that's the typical approach. After a while, if your patches are generally good, Greg will trust you and stop reviewing your code in detail; eventually he will recommend you be granted commit privileges. > Should I maintain bsddb tracking outside python one, to reduce your burden?. No. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From nate at binkert.org Thu Jan 31 18:49:18 2008 From: nate at binkert.org (nathan binkert) Date: Thu, 31 Jan 2008 09:49:18 -0800 Subject: [Python-Dev] Monkeypatching idioms -- elegant or ugly? In-Reply-To: References: <640554BC-544A-4CB6-BB20-EF544B120954@bud.ca> Message-ID: <217accd40801310949h7b0fb019l5b28a4bdc86dde39@mail.gmail.com> Another thing about monkeypatching is that it seems like the best way to write an extension class where you want half to be in C/C++ and half in Python. I'm in the middle of working on such a class and there are plenty of members that just don't need to be in C++. Is there a better/preferred idiom for such a thing? I don't want to subclass my new class because I want any objects created on the C++ side to also get the python methods. Nate On Jan 31, 2008 9:23 AM, Guido van Rossum wrote: > On Jan 30, 2008 9:00 PM, Kevin Teague wrote: > > +1 on having established Python idioms for these techniques. > > > > While I don't know if there has ever been a formal definition of > > monkey patch, the term monkey patch came from guerilla patch, which > > came from two or more dynamic modifications to a class interfering > > with each other. These modifications were usually made by extension > > code (Zope add-on Products) to upstream code (the Zope framework), so > > I would define a monkey patch only as dynamic modifications made to a > > class with the *intent to change or correct behaviour in upstream code*. > > > > The term has also caught on with the a second definition of referring > > to any dynamic modification of class, regardless of intent though. > > Check out the wikipedia entry too. > > > I would perhaps call these methods something like: > > > > * add_method_to_class > > > > * extend_class > > I don't like extend because in Java that's how you define a subclass. > > > This gives you a better idea of what they do, rather than use a term > > with a somewhat ambigous definition. With monkeypatch_method under the > > definition of "altering existing upstream behviour", I might expect it > > to raise an error if the method I was replacing on a class did not > > exist (e.g. upstream code was refactored so my patch no longer applied). > > Funny, several examples mentioned earlier in this thread actually > check that the method *doesn't* already exist... > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/nate%40binkert.org > > From martin at v.loewis.de Thu Jan 31 19:25:17 2008 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 31 Jan 2008 19:25:17 +0100 Subject: [Python-Dev] Upcoming 2.5.2 release In-Reply-To: <47A197FD.9080001@argo.es> References: <479CD349.6060604@v.loewis.de> <47A07E8D.7000500@argo.es> <47A0D860.5030108@v.loewis.de> <47A197FD.9080001@argo.es> Message-ID: <47A2128D.3030003@v.loewis.de> > Current bsddb module in 2.5.1 supports up to BerkeleyDB 4.5. There is > support for 4.6 in trunk (future 2.6, I guess) and I'm working in a > private branch at the moment, since I have no commit access to python > repository. That private version is intented to be merged into python > 2.6 by Greg, when time comes. > > My guess is that 2.5 branch is still open to more patches than pure > security/stability patches, so "backporting" BerkeleyDB 4.6 support > seems reasonable (to me). If I'm wrong, please educate me :-). I think it depends on the nature of the patch. The 2.5 branch already supports BerkeleyDB 4.6 (since r58345), so I'm not sure what else needs to be done. > This backport would include also stability patches. For example, I just > solved a memory leak. > > Greg, any opinion?. > > Beware, new kid in the block! :). Being new is not a problem in itself. But do please take the extra work of double- and triple-checking any claims you make. If you want to become the new maintainer of the bsddb module, you need to learn how to check out a branch of Python and how to determine what patches have been applied to it. Regards, Martin From martin at v.loewis.de Thu Jan 31 19:29:03 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 31 Jan 2008 19:29:03 +0100 Subject: [Python-Dev] Tracker marks my messages as spam :-) In-Reply-To: <47A19DDC.7030508@argo.es> References: <47A19DDC.7030508@argo.es> Message-ID: <47A2136F.7030108@v.loewis.de> > This will be my last email today, I don't want to waste (more of) your > *valuable* time. > > http://bugs.python.org/issue1391 > http://bugs.python.org/msg61892 It does that sometimes when the text is very short. The Bayesian filter needs more training, so just keep reporting messages as misclassified here. Georg has reclassified the message as ham now. Regards, Martin From guido at python.org Thu Jan 31 20:50:51 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 31 Jan 2008 11:50:51 -0800 Subject: [Python-Dev] Monkeypatching idioms -- elegant or ugly? In-Reply-To: <217accd40801310949h7b0fb019l5b28a4bdc86dde39@mail.gmail.com> References: <640554BC-544A-4CB6-BB20-EF544B120954@bud.ca> <217accd40801310949h7b0fb019l5b28a4bdc86dde39@mail.gmail.com> Message-ID: On Jan 31, 2008 9:49 AM, nathan binkert wrote: > Another thing about monkeypatching is that it seems like the best way > to write an extension class where you want half to be in C/C++ and > half in Python. I'm in the middle of working on such a class and > there are plenty of members that just don't need to be in C++. > > Is there a better/preferred idiom for such a thing? I don't want to > subclass my new class because I want any objects created on the C++ > side to also get the python methods. Have you tried this? I believe it doesn't even work; types defined in C++ are supposed to be immutable. Try adding a new method to list or dict. However the C++ side should be able to create instances of the Python-defined subclass as long as it runs in a method, since it has a reference to the actual class. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From nate at binkert.org Thu Jan 31 21:33:23 2008 From: nate at binkert.org (nathan binkert) Date: Thu, 31 Jan 2008 12:33:23 -0800 Subject: [Python-Dev] Monkeypatching idioms -- elegant or ugly? In-Reply-To: References: <640554BC-544A-4CB6-BB20-EF544B120954@bud.ca> <217accd40801310949h7b0fb019l5b28a4bdc86dde39@mail.gmail.com> Message-ID: <217accd40801311233k37bc234dkc25155ef4699d7c8@mail.gmail.com> > Have you tried this? I believe it doesn't even work; types defined in > C++ are supposed to be immutable. Try adding a new method to list or > dict. I noticed that and I was trying to figure out if I could create a new metatype which would add a __dict__ and a method (called add_method or something like that) that could be used as a decorator for monkeypatching. I was partly using this as an exercise to learn more about the Python internals and I'm probably going down a rathole. > However the C++ side should be able to create instances of the > Python-defined subclass as long as it runs in a method, since it has > a reference to the actual class. This is what I'm currently doing and it does work, but I'm trying to build my extension type in such a way that it lazily gets initialized as a python object only if it is passed to the python side of things. For this object, 75% of the time, it is created and used in C++ only and I'd like to not pay for the python object creation overhead if I don't have to. I am putting PyObject_Head at the front of my class, but only initializing it if it is passed to python. I had intended to initialize the python bits with a C++ type, but I guess I could do it with a cached module lookup of the python derived type. Allocation is really tricky too, so this all may just not be worth the hassle. :) Anyway, if these types of issues are inappropriate for this forum, I'll keep quiet. From status at bugs.python.org Fri Jan 25 19:06:18 2008 From: status at bugs.python.org (Tracker) Date: Fri, 25 Jan 2008 18:06:18 -0000 Subject: [Python-Dev] Summary of Tracker Issues Message-ID: <20080125180615.E96CE78173@psf.upfronthosting.co.za> ACTIVITY SUMMARY (01/18/08 - 01/25/08) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1668 open (+31) / 12132 closed (+22) / 13800 total (+53) Open issues with patches: 421 Average duration of open issues: 769 days. Median duration of open issues: 1030 days. Open Issues Breakdown open 1646 (+29) pending 22 ( +2) Issues Created Or Reopened (55) _______________________________ 'weekly' rotating logging file rotation incorrect 01/21/08 CLOSED http://bugs.python.org/issue1836 reopened tiran easy const arg for PyInt_FromString 01/18/08 http://bugs.python.org/issue1866 created philipdumont patch for pydoc to work in py3k 01/18/08 CLOSED http://bugs.python.org/issue1867 created sgala patch, easy threading.local doesn't free attrs when assigning thread exits 01/18/08 http://bugs.python.org/issue1868 created pfein Builtin round function is sometimes inaccurate for floats 01/19/08 http://bugs.python.org/issue1869 created marketdickinson bug tracker exception, when searching for "creator" 01/19/08 CLOSED http://bugs.python.org/issue1870 created ThurnerRupert help fix memory usage or leak - edgewall trac 0.11 blocked ... 01/19/08 CLOSED http://bugs.python.org/issue1871 created ThurnerRupert change the bool struct format code from 't' to '?' 01/19/08 http://bugs.python.org/issue1872 created theller patch threading.Thread.join() description could be more explicit 01/19/08 CLOSED http://bugs.python.org/issue1873 created roysmith easy email parser does not register a defect for invalid Content-Tran 01/19/08 http://bugs.python.org/issue1874 created Sharebear patch, easy "if 0: return" not raising SyntaxError 01/19/08 http://bugs.python.org/issue1875 created arigo bogus attrgetter test in test_operator 01/20/08 CLOSED http://bugs.python.org/issue1876 created pitrou unhelpful error when calling "python " 01/20/08 http://bugs.python.org/issue1877 created georg.brandl easy class attribute cache failure (regression) 01/20/08 http://bugs.python.org/issue1878 created _doublep sqrt(-1) doesn't raise ValueError on OS X 01/21/08 http://bugs.python.org/issue1879 created marketdickinson patch Generalize math.hypot function 01/21/08 CLOSED http://bugs.python.org/issue1880 created LambertDW increase parser stack limit 01/21/08 http://bugs.python.org/issue1881 created schmir py_compile does not follow PEP 0263 01/21/08 CLOSED http://bugs.python.org/issue1882 created jwilk Adapt pydoc to new doc system 01/21/08 http://bugs.python.org/issue1883 created georg.brandl msilib.SetProperty(msilib.PID_CODEPAGE, '1252') raises 0x65d = t 01/21/08 http://bugs.python.org/issue1884 created Jimbo [distutils] - error when processing the "--formats=tar" option 01/21/08 CLOSED http://bugs.python.org/issue1885 created giampaolo.rodola Permit to easily use distutils "--formats=tar,gztar,bztar" on al 01/21/08 http://bugs.python.org/issue1886 created giampaolo.rodola patch distutils doesn't support out-of-source builds 01/22/08 http://bugs.python.org/issue1887 created sdirector patch strange 'global' 01/22/08 CLOSED http://bugs.python.org/issue1888 created oyster string literal documentation differs from implementation 01/22/08 CLOSED http://bugs.python.org/issue1889 created dalke easy Test 01/22/08 CLOSED http://bugs.python.org/issue1902 created tiran Add key argument to heapq functions 01/22/08 http://bugs.python.org/issue1904 created gpolo patch PythonLauncher not working correctly on OS X 10.5/Leopad 01/22/08 http://bugs.python.org/issue1905 created wordtech Distinguish between cfunction, cvar and cmacro 01/22/08 CLOSED http://bugs.python.org/issue1906 created tiran Httplib.py not supporting timeout - Propose Fix attached 01/22/08 CLOSED http://bugs.python.org/issue1907 created xlash make html fails - patchlevel is missing 01/22/08 CLOSED http://bugs.python.org/issue1908 created tiran Backport: Mixing default keyword arguments with *args 01/22/08 http://bugs.python.org/issue1909 created rhettinger Document that with is slower than try/finally 01/23/08 http://bugs.python.org/issue1910 reopened tiran easy webbrowser.open firefox 3 issues 01/23/08 http://bugs.python.org/issue1911 created elessar patch, easy Segmentation fault 01/23/08 CLOSED http://bugs.python.org/issue1912 created mpunkeri test_pep263 fails 01/23/08 CLOSED http://bugs.python.org/issue1913 created tiran test_plistlib fails 01/23/08 http://bugs.python.org/issue1914 created tiran Python 2.5.1 compile failed with configure option "--enable-unic 01/23/08 CLOSED http://bugs.python.org/issue1915 created songma Add inspect.isgenerator 01/23/08 http://bugs.python.org/issue1916 created tiran easy init_types() in Parser/asdl_c.py conflicts with init_types() in 01/23/08 CLOSED http://bugs.python.org/issue1917 created gsmecher weak references are removed before __del__ is called. 01/23/08 http://bugs.python.org/issue1918 created jforan Backport of io.py 01/24/08 http://bugs.python.org/issue1919 created tiran while else loop seems to behave incorrectly 01/24/08 CLOSED http://bugs.python.org/issue1920 created mark Confusing Descrintro example 01/24/08 http://bugs.python.org/issue1921 created arsatiki easy meaningful whitespace can be lost in rfc822_escape 01/24/08 http://bugs.python.org/issue1923 created stephenemslie easy %i string format breaks for large floats (incomplete int convers 01/24/08 CLOSED http://bugs.python.org/issue1924 created goodger TypeError in deepcopy 01/24/08 CLOSED http://bugs.python.org/issue1925 created efrerich NNTPS support in nntplib 01/24/08 http://bugs.python.org/issue1926 created chasonr patch raw_input behavior incorrect if readline not enabled 01/24/08 http://bugs.python.org/issue1927 created skip.montanaro test_urllib fails 01/24/08 http://bugs.python.org/issue1928 created gvanrossum httplib _read_chunked TypeError ||| i = line.find(";") 01/25/08 http://bugs.python.org/issue1929 created orivej easy sys.maxint not found in Python 3.0a2 01/25/08 CLOSED http://bugs.python.org/issue1930 created rclark NameError: global name 'basestring' is not defined 01/25/08 http://bugs.python.org/issue1931 created rclark Cosmetic patch to supress compiler warning 01/25/08 http://bugs.python.org/issue1932 created ocean-city Bypassing __dict__ readonlyness 01/23/08 CLOSED http://bugs.python.org/issue1303614 reopened gvanrossum Issues Now Closed (152) _______________________ zipfile password fails validation 149 days http://bugs.python.org/issue1003 gregory.p.smith bytes buffer API needs to support read locking and/or PyBUF_LOCK 145 days http://bugs.python.org/issue1035 gregory.p.smith patch zipfile cannot handle files larger than 2GB (inside archive) 143 days http://bugs.python.org/issue1060 gregory.p.smith Webchecker not parsing css "@import url" 137 days http://bugs.python.org/issue1124 gvanrossum TextWrap vs words 1-character shorter than the width 130 days http://bugs.python.org/issue1146 georg.brandl patch bsddb does not build with bsddb lib v3.1. 122 days http://bugs.python.org/issue1191 gregory.p.smith segfault in curses when calling redrawwin() before refresh() 100 days http://bugs.python.org/issue1266 akuchling Exception in pstats print_callers() 101 days http://bugs.python.org/issue1269 georg.brandl easy mailbox.Maildir: factory not used 97 days http://bugs.python.org/issue1277 akuchling patch logging records cache the result of formatException() 95 days http://bugs.python.org/issue1295 vsajip patch optparse's OptionGroup not described 93 days http://bugs.python.org/issue1296 akuchling easy subprocess.list2cmdline doesn't do pipe symbols 93 days http://bugs.python.org/issue1300 gregory.p.smith easy threading.RLock().aquire(0) fails with python-2.5.1.amd64.msi 85 days http://bugs.python.org/issue1332 tiran subprocess.Popen hangs when child writes to stderr 88 days http://bugs.python.org/issue1336 gvanrossum patch Documentation lacking for the sqlite3 module. 53 days http://bugs.python.org/issue1509 jarpa easy [Patch] Working post import hook and lazy modules 46 days http://bugs.python.org/issue1576 tiran patch Rare exception in test_urllib2net 41 days http://bugs.python.org/issue1617 nnorwitz IDLE messes around with sys.exitfunc 37 days http://bugs.python.org/issue1647 kbk add new function, sys.gettrace 33 days http://bugs.python.org/issue1648 georg.brandl patch Limit the max size of PyUnicodeObject->defenc? 35 days http://bugs.python.org/issue1651 gvanrossum shutil.rmtree fails on symlink, after deleting contents 32 days http://bugs.python.org/issue1669 draghuram tokenizer permits invalid hex integer 32 days http://bugs.python.org/issue1679 MartinRinehart easy plistlib.py restricts to Python int when writing 28 days http://bugs.python.org/issue1687 georg.brandl floating point number round failures under Linux 26 days http://bugs.python.org/issue1694 marketdickinson Make pydoc list submodules 21 days http://bugs.python.org/issue1715 georg.brandl patch IDLE fails to launch 17 days http://bugs.python.org/issue1743 kbk popen fails when there are two or more pathnames/parameters with 12 days http://bugs.python.org/issue1772 tiran int("- 1") is valud, but float("- 1") isn't. Which is right? 10 days http://bugs.python.org/issue1779 facundobatista easy PyModule_AddIntConstant and PyModule_AddStringConstant can leak 9 days http://bugs.python.org/issue1782 georg.brandl assumption about unsigned long byte size in struct module usage 13 days http://bugs.python.org/issue1789 gregory.p.smith 64bit, easy PEP 754 update 13 days http://bugs.python.org/issue1795 tiran email.mime.multipart.MIMEMultipart.is_multipart() returns false 5 days http://bugs.python.org/issue1822 facundobatista 'weekly' rotating logging file rotation incorrect 0 days http://bugs.python.org/issue1836 vsajip easy Error on "Save As" in IDLE (Vista 32-bit) 6 days http://bugs.python.org/issue1862 kbk Bytes alias for 2.6 0 days http://bugs.python.org/issue1865 tiran patch patch for pydoc to work in py3k 3 days http://bugs.python.org/issue1867 sgala patch, easy bug tracker exception, when searching for "creator" 0 days http://bugs.python.org/issue1870 georg.brandl help fix memory usage or leak - edgewall trac 0.11 blocked ... 0 days http://bugs.python.org/issue1871 tiran threading.Thread.join() description could be more explicit 0 days http://bugs.python.org/issue1873 georg.brandl easy bogus attrgetter test in test_operator 0 days http://bugs.python.org/issue1876 georg.brandl Generalize math.hypot function 0 days http://bugs.python.org/issue1880 georg.brandl py_compile does not follow PEP 0263 0 days http://bugs.python.org/issue1882 georg.brandl [distutils] - error when processing the "--formats=tar" option 0 days http://bugs.python.org/issue1885 tiran strange 'global' 0 days http://bugs.python.org/issue1888 gvanrossum string literal documentation differs from implementation 0 days http://bugs.python.org/issue1889 georg.brandl easy Test 0 days http://bugs.python.org/issue1902 tiran Distinguish between cfunction, cvar and cmacro 0 days http://bugs.python.org/issue1906 georg.brandl Httplib.py not supporting timeout - Propose Fix attached 0 days http://bugs.python.org/issue1907 facundobatista make html fails - patchlevel is missing 0 days http://bugs.python.org/issue1908 georg.brandl Segmentation fault 0 days http://bugs.python.org/issue1912 tiran test_pep263 fails 0 days http://bugs.python.org/issue1913 georg.brandl Python 2.5.1 compile failed with configure option "--enable-unic 0 days http://bugs.python.org/issue1915 tiran init_types() in Parser/asdl_c.py conflicts with init_types() in 0 days http://bugs.python.org/issue1917 gsmecher while else loop seems to behave incorrectly 1 days http://bugs.python.org/issue1920 marketdickinson %i string format breaks for large floats (incomplete int convers 0 days http://bugs.python.org/issue1924 facundobatista TypeError in deepcopy 0 days http://bugs.python.org/issue1925 gvanrossum sys.maxint not found in Python 3.0a2 0 days http://bugs.python.org/issue1930 tiran Parser crashes for deeply nested list displays 2671 days http://bugs.python.org/issue215555 tiran sys.settrace inheritance 2317 days http://bugs.python.org/issue462716 draghuram Put Demo/rpc/rpc.py into standard lib 2266 days http://bugs.python.org/issue479195 draghuram smart module import 2242 days http://bugs.python.org/issue487566 georg.brandl weaklist 2244 days http://bugs.python.org/issue487738 fdrake patch request sys.required_version 2233 days http://bugs.python.org/issue491331 draghuram Using the lib index mechanically 2118 days http://bugs.python.org/issue538961 georg.brandl Windows getpass bug 2101 days http://bugs.python.org/issue546558 georg.brandl os.popen() negative error code IOError 1968 days http://bugs.python.org/issue602245 facundobatista traceback.print_stack extension 1954 days http://bugs.python.org/issue609824 draghuram sys.execpthook not used in threads 1937 days http://bugs.python.org/issue618633 draghuram Tkinter: BitmapImage vanishes if not stored in non-local var 1905 days http://bugs.python.org/issue632323 facundobatista extra __builtin__ stuff not documented 1865 days http://bugs.python.org/issue652749 georg.brandl PyErr_Warn may cause import deadlock 1805 days http://bugs.python.org/issue683658 georg.brandl patch zipfile should have a tarfile-like API 1804 days http://bugs.python.org/issue683910 georg.brandl SocketServer timeout, zombies 1702 days http://bugs.python.org/issue742598 akuchling patch Tutorial: executable scripts on Windows 1670 days http://bugs.python.org/issue760657 georg.brandl pydoc.TextDoc raises for some kinds of objects 1650 days http://bugs.python.org/issue771334 georg.brandl 2.3c1: zipfile.py confused by garbage at the very end of zip 1645 days http://bugs.python.org/issue774221 akuchling patch plistlib and bundlebuilder not in the documentation 1637 days http://bugs.python.org/issue779825 georg.brandl easy test_repr failure on m68k-linux 1633 days http://bugs.python.org/issue784443 draghuram documentation for sys.platform is unclear 1601 days http://bugs.python.org/issue799369 georg.brandl rlcompleter incompatibility 1586 days http://bugs.python.org/issue808197 tiran ntohs on Solaris can return negative values 1580 days http://bugs.python.org/issue811295 tiran select behavior undefined for empty lists 1573 days http://bugs.python.org/issue817920 draghuram mailbox._Subfile readline() bug 1567 days http://bugs.python.org/issue818065 akuchling [2.3.2] test_socket failure on IRIX 6.5 1543 days http://bugs.python.org/issue835338 draghuram Incorrect shared library build 1531 days http://bugs.python.org/issue840065 tiran 64 bit solaris versus /usr/local/lib 1519 days http://bugs.python.org/issue847812 tiran ZipInfo shows incorrect file size for large files 1517 days http://bugs.python.org/issue849218 tiran urllib.py does not use "no_proxy"; 1504 days http://bugs.python.org/issue856047 georg.brandl EAGAIN when sys.stdin.readline() 1501 days http://bugs.python.org/issue858253 tiran Python 2.3.3 test_tempfile test_mode() fails on AIX 5.2 1474 days http://bugs.python.org/issue872686 tiran build_ssl can't find ActiveState Perl from unusual locations 1440 days http://bugs.python.org/issue894743 tiran Embedded Python Interpreter in MFC apps leaks 1418 days http://bugs.python.org/issue909308 tiran make fails using -std option 1395 days http://bugs.python.org/issue924008 tiran ihooks chokes on BioPython package 1350 days http://bugs.python.org/issue951275 tiran Python 2.3.4 on Linux: test test_grp failed 1332 days http://bugs.python.org/issue962226 tiran mmap needs a rfind method 1307 days http://bugs.python.org/issue976880 akuchling patch, easy Cannot specify compiler for 'install' on command line 1307 days http://bugs.python.org/issue977461 tiran Enclosing Scope missing from namespace in Tutorial 1274 days http://bugs.python.org/issue997912 georg.brandl easy Simple File fix for Windows Runtime incompatability 1264 days http://bugs.python.org/issue1003535 tiran patch wrong socket error returned 1236 days http://bugs.python.org/issue1019808 akuchling patch Only "Overwrite" mode possible with curses.textpad.Textbox 1189 days http://bugs.python.org/issue1048820 akuchling patch import on Windows: please call SetErrorMode first 1160 days http://bugs.python.org/issue1069410 tiran dyld: ./python.exe multiple definitions of symbol _BC 1152 days http://bugs.python.org/issue1072642 tiran Pydoc can't find browser (bug+solution!) 1139 days http://bugs.python.org/issue1081879 georg.brandl mmap instance method access bug 1128 days http://bugs.python.org/issue1087735 georg.brandl subclassable mmap 1130 days http://bugs.python.org/issue1087741 georg.brandl Memory leak in socket.py on Mac OS X 1118 days http://bugs.python.org/issue1092502 martey copy.deepcopy barfs when copying a class derived from dict 1105 days http://bugs.python.org/issue1099746 tiran urllib.urlopen should put the http-error-code in .info() 1077 days http://bugs.python.org/issue1117751 georg.brandl zip incorrectly and incompletely documented 1074 days http://bugs.python.org/issue1121416 rhettinger string interpolation breaks with %d and large float 1057 days http://bugs.python.org/issue1153226 marketdickinson urllib.py overwrite HTTPError code with 200 1019 days http://bugs.python.org/issue1178141 georg.brandl package_data chops off first char of default package 1010 days http://bugs.python.org/issue1183712 draghuram patch zipfile module and 2G boundary 1000 days http://bugs.python.org/issue1189216 gregory.p.smith patch wrong location for math lib with --prefix 975 days http://bugs.python.org/issue1205736 tiran tp_richcompare documentation wrong and incomplete 951 days http://bugs.python.org/issue1219903 georg.brandl ftplib storbinary/storlines callback function 951 days http://bugs.python.org/issue1221598 giampaolo.rodola patch garbage collection asserts failing 914 days http://bugs.python.org/issue1241545 tiran Bypassing __dict__ readonlyness 1 days http://bugs.python.org/issue1303614 gvanrossum 2.3.5 configure / make infinite loop 847 days http://bugs.python.org/issue1304179 tiran Switch to make pprint.pprint display ints and longs in hex 803 days http://bugs.python.org/issue1351692 georg.brandl Tweak pprint.PrettyPrinter.format for subclassing 776 days http://bugs.python.org/issue1373762 georg.brandl patch segfaults when using __del__ and weakrefs 775 days http://bugs.python.org/issue1377858 gvanrossum XML.sax.saxutils.escape -- always escapes <, >, &, 728 days http://bugs.python.org/issue1411695 georg.brandl patch, easy Implement preemptive threads in Python 702 days http://bugs.python.org/issue1432694 tiran Splitting CJK codecs from pythoncore dll 677 days http://bugs.python.org/issue1449471 georg.brandl patch Python 2.4.3 build issue on Cygwin 656 days http://bugs.python.org/issue1464788 akuchling dbhash __len__ not reliable 569 days http://bugs.python.org/issue1516078 gregory.p.smith distutils doesn't notice --with-pydebug 541 days http://bugs.python.org/issue1530959 georg.brandl C/API sec 10 is clipped 536 days http://bugs.python.org/issue1533491 georg.brandl bsddb can't use unicode filenames 524 days http://bugs.python.org/issue1541671 gregory.p.smith checking size of int... configure: error: cannot compute siz 517 days http://bugs.python.org/issue1544306 georg.brandl New-style classes fail to cleanup attributes 515 days http://bugs.python.org/issue1545463 georg.brandl Please include pliblist for all plattforms 499 days http://bugs.python.org/issue1555501 georg.brandl qtsupport.py mistake leads to bad _Qt module 9 days http://bugs.python.org/issue1570672 georg.brandl patch Building using Sleepycat db 4.5.20 is broken 474 days http://bugs.python.org/issue1571754 gregory.p.smith this module (Zen of Python) docs list broken URL 394 days http://bugs.python.org/issue1621660 tiran sre module has misleading docs 381 days http://bugs.python.org/issue1631394 tlynn subprocess/popen close_fds perform poor if SC_OPEN_MAX is hi 334 days http://bugs.python.org/issue1663329 georg.brandl patch Fix for urllib.ftpwrapper.retrfile() and none existing files 334 days http://bugs.python.org/issue1664522 georg.brandl patch setup.py LDFLAGS regexp is wrong 318 days http://bugs.python.org/issue1675533 facundobatista easy A fix for the bug #1528074 [warning: quite slow] 316 days http://bugs.python.org/issue1678345 georg.brandl patch fix a bug mixing up 0.0 and-0.0 316 days http://bugs.python.org/issue1678668 marketdickinson patch Replace time_t by Py_time_t 298 days http://bugs.python.org/issue1689402 tiran patch patch to make 'trace.py --ignore-module' accept module name list 292 days http://bugs.python.org/issue1693149 facundobatista patch, easy stack size of python_d.exe on VC6 281 days http://bugs.python.org/issue1700467 georg.brandl patch "t.join(); assert t not in threading.enumerate()" fails 278 days http://bugs.python.org/issue1703448 gregory.p.smith patch Allow T_BOOL in PyMemberDef definitions 249 days http://bugs.python.org/issue1720595 georg.brandl patch Line iteration readability 240 days http://bugs.python.org/issue1726198 georg.brandl patch urllib.urlretrieve/URLopener.retrieve - 'buff' argument 229 days http://bugs.python.org/issue1731720 georg.brandl patch Non-portable address length calculation for AF_UNIX sockets 192 days http://bugs.python.org/issue1754489 arigo patch logging: delay_fh option and configuration kwargs 176 days http://bugs.python.org/issue1765140 vsajip patch Top Issues Most Discussed (10) ______________________________ 34 Enhancements for mathmodule 39 days open http://bugs.python.org/issue1640 23 Do not assume signed integer overflow behavior 43 days open http://bugs.python.org/issue1621 15 class attribute cache failure (regression) 5 days open http://bugs.python.org/issue1878 12 increase parser stack limit 4 days open http://bugs.python.org/issue1881 10 "t.join(); assert t not in threading.enumerate()" fails 278 days closed http://bugs.python.org/issue1703448 10 shutil.move() does not use os.rename() if dst is a directory 47 days open http://bugs.python.org/issue1577 9 Document that with is slower than try/finally 2 days pending http://bugs.python.org/issue1910 8 tokenizer permits invalid hex integer 32 days closed http://bugs.python.org/issue1679 7 assumption about unsigned long byte size in struct module usage 13 days closed http://bugs.python.org/issue1789 7 Bypassing __dict__ readonlyness 1 days closed http://bugs.python.org/issue1303614 From oliphant at enthought.com Mon Jan 28 21:28:06 2008 From: oliphant at enthought.com (Travis E. Oliphant) Date: Mon, 28 Jan 2008 14:28:06 -0600 Subject: [Python-Dev] Py2.6 release schedule In-Reply-To: <479E2DFA.8040904@cheimes.de> References: <20080128135028.AFV97486@ms19.lnh.mail.rcn.net> <479E2DFA.8040904@cheimes.de> Message-ID: <479E3AD6.5020705@enthought.com> Christian Heimes wrote: > Guido van Rossum wrote: > >> I do think that another (final) 3.0 alpha before PyCon would also be a >> good idea. This way we can gel the release some more. For 2.6 I think >> we'll need more alpha releases after PyCon; I doubt the backporting >> from 3.0 (which has only started seriously quite recently) will be >> done by PyCon. >> > > I've back ported class decorators (http://bugs.python.org/issue1759). > Two tests are failing and I need some help to solve the riddle. > > Several back ports like the bytearray type and the new io module depend > on a back port of the new buffer protocol. Travis, can you please > increase your priority on the port of your PEP to 2.6? > > Yes, I will. What are your time-lines? I've been targeting first week in March. -Travis