From ncoghlan at gmail.com Fri Jul 1 03:50:53 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 1 Jul 2011 11:50:53 +1000 Subject: [Python-Dev] svn.python.org confusion In-Reply-To: References: <4E0B9B68.6090207@v.loewis.de> <4E0C857C.6040906@netwok.org> <20110630153251.751dddfd@snowdog> Message-ID: On Fri, Jul 1, 2011 at 1:40 AM, Tres Seaver wrote: > On 06/30/2011 10:32 AM, Barry Warsaw wrote: >> I'm not against adding this to svn, but please be sure these files don't leak >> into the tarballs should we need to cut another 2.5 or 2.6 release. ?I think >> that just means tweaking sandbox/release/release.py. > > The fact that releases might / will still be made from those branches > argues against including the file on them at all: ?they are in fact the > "canonical" repository locations, even if most of the work is done in hg > and patched into them. Indeed, 2.5 and 2.6 should be left alone. +1 on adding such a file to the more recent branches that are handled in hg, though. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From skippy.hammond at gmail.com Fri Jul 1 04:21:54 2011 From: skippy.hammond at gmail.com (Mark Hammond) Date: Fri, 01 Jul 2011 12:21:54 +1000 Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference implementation In-Reply-To: References: <4E0BFA31.8020200@gmail.com> Message-ID: <4E0D2F42.4040802@gmail.com> On 30/06/2011 10:09 PM, Vinay Sajip wrote: > There's a lot to like in the PEP, and I have some questions relating to the > latest version: > > 1. In the section on shebang line parsing, it says "If the command starts with > the definition of a customized command followed by a space character, the > customized command will be used." Sorry if I'm being dense, but what's the > significance of the trailing space character? In fact, your vpython example in > the customeised comments section doesn't show a trailing space - you've used > '#! vpython' as the shebang line. This is just clumsy wording on my part - what I'm trying to say is simply that 'vpython3' will not match a customized command 'vpython'. > 2. The section on .ini files says that "Commands specified in the [.ini file > in the] "application directory" [APPDATA] will have precedence over the one > next to the [launcher] executable." What's the benefit of this? The idea is that a user without admin permissions can customize the behaviour - so the admin can add a .ini file, but the user can override any commands in that file with their own definitions. > If you have > only one launcher executable of one type (say console, 32-bit) on a system, > then there's no point in having two locations for .ini files. The idea is that some users will not have permission to edit the one next to the launcher. I'd be open to considering this yagni if others agree. > If you have > multiple copies of the launcher and multiple .ini files next to them, then > with this precedence order, you can't specialise the behaviour of anything in > a specific launcher .ini that's also specified in the APPDATA .ini. The intention is that there only be a single launcher, as only one app can be associated with .py files. OTOH though, file associations can be configured per-user IIRC, and assuming that is the case, we could avoid my multiple-ini-file usecase above by just allowing a different launcher to be registered for a specific user. This is sounding difficult from the UI perspective though (ie, does the installer then need to ask that question, will there be a post-install technique for per-user registration, etc?) > Doesn't it > make more sense to look for a setting in any file next to the launcher, and if > not found there, look in the APPDATA? That way common things can be defined in > the APPDATA .ini and overridden for special cases in the launcher-adjacent > .ini. Or have I got completely the wrong end of the stick? I came at this from the other angle under the assumption there will only be one launcher installed - so common things can be stored in the launcher-adjacent file and per-user special cases can be in the per-user APPDATA dir. If the expectation was multiple launchers be installed, then I'd probably then prefer to KISS and only have the launcher-adjacent file supported at all. > > By the way, I've already converted the existing Python reference implementation > to C (I did it before I saw your response to my post). It basically works in > that it does what the Python version does, but doesn't include any handling of > -32 suffixes or .ini files. I can post this on BitBucket if anyone's interested, > but it's a work in progress. I'm working on some simple tests. Sure, that would be awesome! I think that will mean your impl is fairly close to the first draft of the PEP I checked into HG, which is nice and still quite useful to use :) Thanks! Mark From bernie_bill at rocketmail.com Fri Jul 1 06:56:05 2011 From: bernie_bill at rocketmail.com (Bernie Bill) Date: Thu, 30 Jun 2011 21:56:05 -0700 (PDT) Subject: [Python-Dev] commant Message-ID: <1309496165.69522.YahooMailNeo@web113320.mail.gq1.yahoo.com> and thanks!! lederbill -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulrich.eckhardt at dominolaser.com Fri Jul 1 08:51:22 2011 From: ulrich.eckhardt at dominolaser.com (Ulrich Eckhardt) Date: Fri, 1 Jul 2011 08:51:22 +0200 Subject: [Python-Dev] time.sleep(-1) behaviour In-Reply-To: <20110630192145.7C83D2506C1@webabinitio.net> References: <201106302113.47144.ulrich.eckhardt@dominolaser.com> <20110630192145.7C83D2506C1@webabinitio.net> Message-ID: <201107010851.23193.ulrich.eckhardt@dominolaser.com> On Thursday 30 June 2011, R. David Murray wrote: > Please file a bug report at bugs.python.org. Filed as bug #12459. Uli ************************************************************************************** Domino Laser GmbH, Fangdieckstra?e 75a, 22547 Hamburg, Deutschland Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Visit our website at http://www.dominolaser.com ************************************************************************************** Diese E-Mail einschlie?lich s?mtlicher Anh?nge ist nur f?r den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empf?nger sein sollten. Die E-Mail ist in diesem Fall zu l?schen und darf weder gelesen, weitergeleitet, ver?ffentlicht oder anderweitig benutzt werden. E-Mails k?nnen durch Dritte gelesen werden und Viren sowie nichtautorisierte ?nderungen enthalten. Domino Laser GmbH ist f?r diese Folgen nicht verantwortlich. ************************************************************************************** From victor.stinner at haypocalc.com Fri Jul 1 11:18:11 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Fri, 01 Jul 2011 11:18:11 +0200 Subject: [Python-Dev] [Python-checkins] cpython (3.2): test_os: add TemporaryFileTests to the testcase list In-Reply-To: References: Message-ID: <1309511891.3655.2.camel@marge> I tried to find which commit removes TemporaryFileTests from the testcase list (to see if there is a good reason to do that, or if it's just a mistake): it's somewhere between Python 2.x and Python 3.0, but I didn't find the commit. Is there a tool to detect that a testcase is never executed to ensure that there is no other forgiven testcase? Victor Le vendredi 01 juillet 2011 ? 03:06 +0200, victor.stinner a ?crit : > http://hg.python.org/cpython/rev/00a874ad4fc9 > changeset: 71104:00a874ad4fc9 > branch: 3.2 > parent: 71101:7c60c1b41da9 > user: Victor Stinner > date: Fri Jul 01 02:56:15 2011 +0200 > summary: > test_os: add TemporaryFileTests to the testcase list > > The testcase was never executed, it's now fixed. > > files: > Lib/test/test_os.py | 1 + > 1 files changed, 1 insertions(+), 0 deletions(-) > > > diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py > --- a/Lib/test/test_os.py > +++ b/Lib/test/test_os.py > @@ -1344,6 +1344,7 @@ > PidTests, > LoginTests, > LinkTests, > + TemporaryFileTests, > ) From vinay_sajip at yahoo.co.uk Fri Jul 1 11:20:18 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Fri, 1 Jul 2011 09:20:18 +0000 (UTC) Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference implementation References: <4E0BFA31.8020200@gmail.com> <4E0D2F42.4040802@gmail.com> Message-ID: Mark Hammond gmail.com> writes: > The intention is that there only be a single launcher, as only one app > can be associated with .py files. OTOH though, file associations can be > configured per-user IIRC, and assuming that is the case, we could avoid > my multiple-ini-file usecase above by just allowing a different launcher > to be registered for a specific user. This is sounding difficult from > the UI perspective though (ie, does the installer then need to ask that > question, will there be a post-install technique for per-user > registration, etc?) I don't like this, for the reasons you state. I think it would be better if the PEP was changed to say that there is intended to be just one launcher installation per machine. As the intention is for the launcher to ship with Python, and there can be multiple Pythons installed, I presume we'll have to handle this by installing the launcher in some common-to-all-Pythons location somewhere outside a Python installation location, such as "c:\Program Files\Python Launcher". This should be stated in the PEP, and we'll also need to indicate how the launcher will be cleaned up if for some strange reason someone uninstalls all Pythons from a machine. Then we can just state that there's a global .ini file (where the launcher is installed) and a local one (in the user's APPDATA). From that perspective, it makes sense to have items in the local (APPDATA) override the global (launcher installation location). > Sure, that would be awesome! I think that will mean your impl is fairly > close to the first draft of the PEP I checked into HG, which is nice and > still quite useful to use :) Okay, I'll do this soon - once I've added a few tests. I've updated the implementation to include help output. BTW I thought of another thing that perhaps needs handling: what if a customized command points to the launcher itself? It'd be turtles all the way down :-) Regards, Vinay Sajip From solipsis at pitrou.net Fri Jul 1 11:24:53 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 1 Jul 2011 11:24:53 +0200 Subject: [Python-Dev] cpython: Issue #12451: Add support.create_empty_file() References: Message-ID: <20110701112453.7ad659fa@pitrou.net> On Thu, 30 Jun 2011 23:25:59 +0200 victor.stinner wrote: > http://hg.python.org/cpython/rev/0c49260e85a0 > changeset: 71103:0c49260e85a0 > user: Victor Stinner > date: Thu Jun 30 23:25:47 2011 +0200 > summary: > Issue #12451: Add support.create_empty_file() > > We don't need to create a temporary buffered binary or text file object just to > create an empty file. Is there a reason for this? I find it quite explicit and obvious what the following does: with open("somefile", "wb"): pass so I wonder what replacing it with support.create_empty_file() brings (except from having to lookup yet another helper function). Regards Antoine. From victor.stinner at haypocalc.com Fri Jul 1 13:12:07 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Fri, 01 Jul 2011 13:12:07 +0200 Subject: [Python-Dev] cpython: Issue #12451: Add support.create_empty_file() In-Reply-To: <20110701112453.7ad659fa@pitrou.net> References: <20110701112453.7ad659fa@pitrou.net> Message-ID: <1309518727.3655.10.camel@marge> Le vendredi 01 juillet 2011 ? 11:24 +0200, Antoine Pitrou a ?crit : > On Thu, 30 Jun 2011 23:25:59 +0200 > victor.stinner wrote: > > http://hg.python.org/cpython/rev/0c49260e85a0 > > changeset: 71103:0c49260e85a0 > > user: Victor Stinner > > date: Thu Jun 30 23:25:47 2011 +0200 > > summary: > > Issue #12451: Add support.create_empty_file() > > > > We don't need to create a temporary buffered binary or text file object just to > > create an empty file. > > Is there a reason for this? The code was correct. I think that a function with an explicit name is better than the open().close() pattern (which has various flavours, see the diff). This pattern came sometimes with a comment explaining what it does (create an empty file), which let me think that it's not easy to understand what it is supposed to do (if you don't have the comment). My initial need was to make quiet a warning (of my patched Python, see #12451) if a file is opened in text mode without an explicit encoding. > I find it quite explicit and obvious what the following does: > > with open("somefile", "wb"): > pass For "wb", the only gain is to avoid the creation of temporary FileIO and BufferedWriter objects, a micro optimisation. Most of the time, "w" mode was used, so another temporary TextIOWrapper object was also created. I also saw "w+" mode (use BufferedRandom+TextIOWrapper objects). Victor From tlesher at gmail.com Fri Jul 1 14:17:48 2011 From: tlesher at gmail.com (Tim Lesher) Date: Fri, 1 Jul 2011 08:17:48 -0400 Subject: [Python-Dev] time.sleep(-1) behaviour In-Reply-To: <201106302113.47144.ulrich.eckhardt@dominolaser.com> References: <201106302113.47144.ulrich.eckhardt@dominolaser.com> Message-ID: On Thu, Jun 30, 2011 at 15:13, Ulrich Eckhardt wrote: > Hi! > > This is a request for clarification for the thread "how to call a function for > evry 10 seconds" from the user mailinglist/newsgroup. > > > The gist is this: > 1. On Linux/Python 2.6, time.sleep(-1.0) raises an IOError. > 2. On MS Windows/Python 2.5 or 2.7 this sleeps forever. It seems that > converting this to a 32-bit integer for Sleep() causes an underflow. > 3. Is the behaviour under MS Windows acceptable or a bug? On the Windows side, Sleep(-1) as "infinite" is correct and documented: http://msdn.microsoft.com/en-us/library/ms686298(v=vs.85).aspx -- Tim Lesher From ncoghlan at gmail.com Fri Jul 1 14:39:31 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 1 Jul 2011 22:39:31 +1000 Subject: [Python-Dev] [Python-checkins] cpython (3.2): test_os: add TemporaryFileTests to the testcase list In-Reply-To: <1309511891.3655.2.camel@marge> References: <1309511891.3655.2.camel@marge> Message-ID: On Fri, Jul 1, 2011 at 7:18 PM, Victor Stinner wrote: > I tried to find which commit removes TemporaryFileTests from the > testcase list (to see if there is a good reason to do that, or if it's > just a mistake): it's somewhere between Python 2.x and Python 3.0, but I > didn't find the commit. > > Is there a tool to detect that a testcase is never executed to ensure > that there is no other forgiven testcase? Coverage data for the test suite itself. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Fri Jul 1 14:46:16 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 1 Jul 2011 22:46:16 +1000 Subject: [Python-Dev] time.sleep(-1) behaviour In-Reply-To: References: <201106302113.47144.ulrich.eckhardt@dominolaser.com> Message-ID: On Fri, Jul 1, 2011 at 10:17 PM, Tim Lesher wrote: > On Thu, Jun 30, 2011 at 15:13, Ulrich Eckhardt > wrote: >> Hi! >> >> This is a request for clarification for the thread "how to call a function for >> evry 10 seconds" from the user mailinglist/newsgroup. >> >> >> The gist is this: >> 1. On Linux/Python 2.6, time.sleep(-1.0) raises an IOError. >> 2. On MS Windows/Python 2.5 or 2.7 this sleeps forever. It seems that >> converting this to a 32-bit integer for Sleep() causes an underflow. > >> 3. Is the behaviour under MS Windows acceptable or a bug? > > On the Windows side, Sleep(-1) as "infinite" is correct and documented: > > http://msdn.microsoft.com/en-us/library/ms686298(v=vs.85).aspx For Sleep, yes, but the time.sleep() docs [1] are completely silent on the behaviour of negative sleep values at the Python level. Question 3 isn't "Is there something wrong with Sleep()?", it is "Is there something wrong with the way time.sleep() *uses* Sleep()?" My personal preference would be to standardise on ValueError (since the negative sleep value is the real problem), or, failing that, at least raising an exception of some kind. [1] http://docs.python.org/dev/library/time#time.sleep Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From victor.stinner at haypocalc.com Fri Jul 1 14:48:19 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Fri, 01 Jul 2011 14:48:19 +0200 Subject: [Python-Dev] time.sleep(-1) behaviour In-Reply-To: References: <201106302113.47144.ulrich.eckhardt@dominolaser.com> Message-ID: <1309524499.7942.5.camel@marge> Le vendredi 01 juillet 2011 ? 08:17 -0400, Tim Lesher a ?crit : > On Thu, Jun 30, 2011 at 15:13, Ulrich Eckhardt > wrote: > > Hi! > > > > This is a request for clarification for the thread "how to call a function for > > evry 10 seconds" from the user mailinglist/newsgroup. > > > > > > The gist is this: > > 1. On Linux/Python 2.6, time.sleep(-1.0) raises an IOError. > > 2. On MS Windows/Python 2.5 or 2.7 this sleeps forever. It seems that > > converting this to a 32-bit integer for Sleep() causes an underflow. > > > 3. Is the behaviour under MS Windows acceptable or a bug? > > On the Windows side, Sleep(-1) as "infinite" is correct and documented: > > http://msdn.microsoft.com/en-us/library/ms686298(v=vs.85).aspx I answered on the bug tracker: http://bugs.python.org/issue12459 Victor From rosslagerwall at gmail.com Fri Jul 1 14:55:39 2011 From: rosslagerwall at gmail.com (Ross Lagerwall) Date: Fri, 01 Jul 2011 14:55:39 +0200 Subject: [Python-Dev] [Python-checkins] cpython (3.2): test_os: add TemporaryFileTests to the testcase list In-Reply-To: <1309511891.3655.2.camel@marge> References: <1309511891.3655.2.camel@marge> Message-ID: <1309524939.1473.4.camel@hobo> On Fri, 2011-07-01 at 11:18 +0200, Victor Stinner wrote: > I tried to find which commit removes TemporaryFileTests from the > testcase list (to see if there is a good reason to do that, or if it's > just a mistake): it's somewhere between Python 2.x and Python 3.0, but I > didn't find the commit. For what it's worth, $ hg grep --all TemporaryFileTests Lib/test/test_os.py Lib/test/test_os.py:45773:+:class TemporaryFileTests(unittest.TestCase): Lib/test/test_os.py:43680:-:class TemporaryFileTests(unittest.TestCase): Lib/test/test_os.py:43680:-: TemporaryFileTests, will show where TemporaryFileTests was removed and added. From tlesher at gmail.com Fri Jul 1 15:23:17 2011 From: tlesher at gmail.com (Tim Lesher) Date: Fri, 1 Jul 2011 09:23:17 -0400 Subject: [Python-Dev] time.sleep(-1) behaviour In-Reply-To: References: <201106302113.47144.ulrich.eckhardt@dominolaser.com> Message-ID: On Fri, Jul 1, 2011 at 08:46, Nick Coghlan wrote: > For Sleep, yes, but the time.sleep() docs [1] are completely silent on > the behaviour of negative sleep values at the Python level. Question 3 > isn't "Is there something wrong with Sleep()?", it is "Is there > something wrong with the way time.sleep() *uses* Sleep()?" That makes sense. Better to be consistent within the time API--I know the different semantics of time.clock() have confused people around here. -- Tim Lesher From victor.stinner at haypocalc.com Fri Jul 1 15:34:55 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Fri, 01 Jul 2011 15:34:55 +0200 Subject: [Python-Dev] [Python-checkins] cpython (3.2): test_os: add TemporaryFileTests to the testcase list In-Reply-To: <1309524939.1473.4.camel@hobo> References: <1309511891.3655.2.camel@marge> <1309524939.1473.4.camel@hobo> Message-ID: <1309527295.7942.11.camel@marge> Le vendredi 01 juillet 2011 ? 14:55 +0200, Ross Lagerwall a ?crit : > On Fri, 2011-07-01 at 11:18 +0200, Victor Stinner wrote: > > I tried to find which commit removes TemporaryFileTests from the > > testcase list (to see if there is a good reason to do that, or if it's > > just a mistake): it's somewhere between Python 2.x and Python 3.0, but I > > didn't find the commit. > > For what it's worth, > > $ hg grep --all TemporaryFileTests Lib/test/test_os.py > Lib/test/test_os.py:45773:+:class TemporaryFileTests(unittest.TestCase): > Lib/test/test_os.py:43680:-:class TemporaryFileTests(unittest.TestCase): > Lib/test/test_os.py:43680:-: TemporaryFileTests, > > > will show where TemporaryFileTests was removed and added. It was added in the middle of an horrible trunk->3.x merge: "Merged revisions 61239-61249,61252-61257,61260-61264,61269-61275,61278-61279,61285-61286,61288-61290,61298,61303-61305, svn+ssh://pythondev at svn.python.org/python/trunk" To find which commit removed a function in a file, hg bisect can be used (trick given on #mercurial): PREVREV=$(hg id); hg bisect -r; hg bisect -g 0; hg bisect -c 'grep -q tmpnam Modules/posixmodule.c'; hg update $PREVREV Even if it searchs in ~70.000 commits, it takes less than a second to find the commit which removed tmpnam in Python 3: ------------ changeset: 43680:a8818ffe24d1 parent: 43673:d66018ed3ded user: Guido van Rossum date: Thu Oct 25 23:18:51 2007 +0000 files: Doc/library/os.rst Lib/test/test_os.py Lib/test/test_posix.py Misc/NEWS Modules/posixmodule.c description: Patch 1318 by Christian Heimes: remove os.tmpnam(), os.tempnam(), and os.tmpfile(). ------------ Note: I did a new commit on test_os in Python 3.3 to remove TemporaryFileTests (again): most tests were useless because os.tmpnam(), os.tempnam() and os.tmpfile() don't exist anymore in Python 3.3. I moved remaining useful tests to another testcase. Victor From driedel at cox.net Fri Jul 1 16:01:36 2011 From: driedel at cox.net (David P. Riedel) Date: Fri, 01 Jul 2011 10:01:36 -0400 Subject: [Python-Dev] What happened to Python 3.2.1? Message-ID: Hi Python 3.2.1 was scheduled to be released on 6/19, I believe but there is no mention of it anywhere. Has it been delayed? Thanks. From brian.curtin at gmail.com Fri Jul 1 16:38:04 2011 From: brian.curtin at gmail.com (Brian Curtin) Date: Fri, 1 Jul 2011 09:38:04 -0500 Subject: [Python-Dev] What happened to Python 3.2.1? In-Reply-To: References: Message-ID: On Fri, Jul 1, 2011 at 09:01, David P. Riedel wrote: > Hi > > Python 3.2.1 was scheduled to be released on 6/19, I believe but there is > no mention of it anywhere. Has it been delayed? > > Thanks. There are two remaining blockers for the release: http://bugs.python.org/issue12346 and http://bugs.python.org/issue12291 -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Fri Jul 1 16:43:22 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 1 Jul 2011 16:43:22 +0200 Subject: [Python-Dev] What happened to Python 3.2.1? References: Message-ID: <20110701164322.72c6f105@pitrou.net> On Fri, 1 Jul 2011 09:38:04 -0500 Brian Curtin wrote: > On Fri, Jul 1, 2011 at 09:01, David P. Riedel wrote: > > > Hi > > > > Python 3.2.1 was scheduled to be released on 6/19, I believe but there is > > no mention of it anywhere. Has it been delayed? > > > > Thanks. > > > There are two remaining blockers for the release: > http://bugs.python.org/issue12346 and http://bugs.python.org/issue12291 Perhaps http://bugs.python.org/issue12213 should also be a blocker? From status at bugs.python.org Fri Jul 1 18:07:25 2011 From: status at bugs.python.org (Python tracker) Date: Fri, 1 Jul 2011 18:07:25 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20110701160725.65B1D1CCB4@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2011-06-24 - 2011-07-01) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 2850 ( +5) closed 21399 (+64) total 24249 (+69) Open issues with patches: 1233 Issues opened (50) ================== #5572: packaging should respect the LIBS configure env var http://bugs.python.org/issue5572 reopened by eric.araujo #11363: Curses - add missing functions to doc http://bugs.python.org/issue11363 reopened by eric.araujo #12401: unset PYTHON* environment variables when running tests http://bugs.python.org/issue12401 opened by henry.precheur #12403: Mention sys.displayhook in code module docs and the compile bu http://bugs.python.org/issue12403 opened by tebeka #12405: packaging does not record/remove directories it creates http://bugs.python.org/issue12405 opened by vinay.sajip #12406: msi.py needs updating for Python 3.3 http://bugs.python.org/issue12406 opened by vinay.sajip #12410: Create a new helper function that enable to test that an opera http://bugs.python.org/issue12410 opened by mouad #12411: cgi.parse_multipart is broken on 3.x http://bugs.python.org/issue12411 opened by jonas.wagner #12412: non defined representation for pwd.struct_passwd http://bugs.python.org/issue12412 opened by fgarciar #12413: make faulthandler dump traceback of child processes http://bugs.python.org/issue12413 opened by neologix #12414: getsizeof() on code objects is wrong http://bugs.python.org/issue12414 opened by benjamin.peterson #12415: Missing: How to checkout the Doc sources http://bugs.python.org/issue12415 opened by philip #12416: packaging does not have hooks callable during distribution rem http://bugs.python.org/issue12416 opened by vinay.sajip #12418: python should inherit the library search path from the compile http://bugs.python.org/issue12418 opened by vorlon #12420: distutils tests fail if PATH is not defined http://bugs.python.org/issue12420 opened by henry.precheur #12423: signal handler doesn't handle SIGABRT from os.abort http://bugs.python.org/issue12423 opened by kisielk #12424: distutils2: extension section uses bad environment marker sepa http://bugs.python.org/issue12424 opened by eli.collins #12425: gettext breaks on empty plural-forms value http://bugs.python.org/issue12425 opened by djc #12426: packaging.tests.test_command_install_dist.InstallTestCase fail http://bugs.python.org/issue12426 opened by haypo #12427: packaging register fails because "POST data should be bytes" http://bugs.python.org/issue12427 opened by vinay.sajip #12428: functools test coverage http://bugs.python.org/issue12428 opened by Thorney #12429: test_io.check_interrupted_write() sporadic failures on FreeBSD http://bugs.python.org/issue12429 opened by haypo #12432: remove a bunch of unused imports in Lib http://bugs.python.org/issue12432 opened by vincele #12434: Strengthen 2.7 io types warning http://bugs.python.org/issue12434 opened by terry.reedy #12436: Provide reference to detailed installation instructions http://bugs.python.org/issue12436 opened by ncoghlan #12437: _ctypes.dlopen does not include errno in OSError http://bugs.python.org/issue12437 opened by anacrolix #12438: IDLE problem displaying warning message http://bugs.python.org/issue12438 opened by JBernardo #12439: BaseHTTPServer's send_reponse adds extra "\r\n" when using HTT http://bugs.python.org/issue12439 opened by Yoav.Weiss #12440: test_ssl.test_options() failure on Snow Leopard: can't clear o http://bugs.python.org/issue12440 opened by haypo #12441: _GLOBAL_DEFAULT_TIMEOUT remains as an object() in HTTPConnecti http://bugs.python.org/issue12441 opened by juanjux #12445: dict view values objects are missing tp_richcmp and tp_as_numb http://bugs.python.org/issue12445 opened by Julian #12446: StreamReader Readlines behavior odd http://bugs.python.org/issue12446 opened by Thomas.Barnet-Lamb #12448: smtplib's __main__ doesn't flush when prompting http://bugs.python.org/issue12448 opened by anacrolix #12449: Add accelerator "F" to button "Finish" in all MSI installers m http://bugs.python.org/issue12449 opened by cool-RR #12451: open: avoid the locale encoding when possible http://bugs.python.org/issue12451 opened by haypo #12452: reuse plistlib in sysconfig; deprecation process in plistlib http://bugs.python.org/issue12452 opened by haypo #12453: test_import.test_failing_import_sticks() sporadic failures http://bugs.python.org/issue12453 opened by haypo #12454: mailbox: use ASCII to read/write .mh_sequences files http://bugs.python.org/issue12454 opened by haypo #12455: urllib2 forces title() on header names, breaking some requests http://bugs.python.org/issue12455 opened by Cal.Leeming #12456: Hangs in concurrent.futures http://bugs.python.org/issue12456 opened by rosslagerwall #12457: type() returns incorrect type for nested classes http://bugs.python.org/issue12457 opened by pwil3058 #12458: Tracebacks should contain the first line of continuation lines http://bugs.python.org/issue12458 opened by psss #12459: time.sleep(-1.0) behaviour http://bugs.python.org/issue12459 opened by eckhardt #12460: SocketServer.shutdown() does not have "timeout=None" parameter http://bugs.python.org/issue12460 opened by mmarkk #12461: it's not clear how the shutil.copystat() should work on symlin http://bugs.python.org/issue12461 opened by mmarkk #12463: Calling SocketServer.shutdown() when server_forever() was not http://bugs.python.org/issue12463 opened by mmarkk #12464: tempfile.TemporaryDirectory.cleanup follows symbolic links http://bugs.python.org/issue12464 opened by abacabadabacaba #12465: gc.get_referents can be used to crash Python http://bugs.python.org/issue12465 opened by abacabadabacaba #12466: test_subprocess.test_close_fds() sporadic failures on Mac OS X http://bugs.python.org/issue12466 opened by haypo #12467: test_threading.test_6_daemon_threads(): Assertion failed: PyUn http://bugs.python.org/issue12467 opened by haypo Most recent 15 issues with no replies (15) ========================================== #12467: test_threading.test_6_daemon_threads(): Assertion failed: PyUn http://bugs.python.org/issue12467 #12466: test_subprocess.test_close_fds() sporadic failures on Mac OS X http://bugs.python.org/issue12466 #12465: gc.get_referents can be used to crash Python http://bugs.python.org/issue12465 #12464: tempfile.TemporaryDirectory.cleanup follows symbolic links http://bugs.python.org/issue12464 #12463: Calling SocketServer.shutdown() when server_forever() was not http://bugs.python.org/issue12463 #12461: it's not clear how the shutil.copystat() should work on symlin http://bugs.python.org/issue12461 #12460: SocketServer.shutdown() does not have "timeout=None" parameter http://bugs.python.org/issue12460 #12458: Tracebacks should contain the first line of continuation lines http://bugs.python.org/issue12458 #12454: mailbox: use ASCII to read/write .mh_sequences files http://bugs.python.org/issue12454 #12453: test_import.test_failing_import_sticks() sporadic failures http://bugs.python.org/issue12453 #12452: reuse plistlib in sysconfig; deprecation process in plistlib http://bugs.python.org/issue12452 #12448: smtplib's __main__ doesn't flush when prompting http://bugs.python.org/issue12448 #12439: BaseHTTPServer's send_reponse adds extra "\r\n" when using HTT http://bugs.python.org/issue12439 #12436: Provide reference to detailed installation instructions http://bugs.python.org/issue12436 #12434: Strengthen 2.7 io types warning http://bugs.python.org/issue12434 Most recent 15 issues waiting for review (15) ============================================= #12459: time.sleep(-1.0) behaviour http://bugs.python.org/issue12459 #12454: mailbox: use ASCII to read/write .mh_sequences files http://bugs.python.org/issue12454 #12452: reuse plistlib in sysconfig; deprecation process in plistlib http://bugs.python.org/issue12452 #12451: open: avoid the locale encoding when possible http://bugs.python.org/issue12451 #12445: dict view values objects are missing tp_richcmp and tp_as_numb http://bugs.python.org/issue12445 #12432: remove a bunch of unused imports in Lib http://bugs.python.org/issue12432 #12429: test_io.check_interrupted_write() sporadic failures on FreeBSD http://bugs.python.org/issue12429 #12428: functools test coverage http://bugs.python.org/issue12428 #12425: gettext breaks on empty plural-forms value http://bugs.python.org/issue12425 #12424: distutils2: extension section uses bad environment marker sepa http://bugs.python.org/issue12424 #12420: distutils tests fail if PATH is not defined http://bugs.python.org/issue12420 #12411: cgi.parse_multipart is broken on 3.x http://bugs.python.org/issue12411 #12410: Create a new helper function that enable to test that an opera http://bugs.python.org/issue12410 #12406: msi.py needs updating for Python 3.3 http://bugs.python.org/issue12406 #12401: unset PYTHON* environment variables when running tests http://bugs.python.org/issue12401 Top 10 most discussed issues (10) ================================= #12398: Sending binary data with a POST request in httplib can cause U http://bugs.python.org/issue12398 14 msgs #6721: Locks in python standard library should be sanitized on fork http://bugs.python.org/issue6721 13 msgs #10181: Problems with Py_buffer management in memoryobject.c (and else http://bugs.python.org/issue10181 13 msgs #12352: multiprocessing.Value() hangs http://bugs.python.org/issue12352 12 msgs #12455: urllib2 forces title() on header names, breaking some requests http://bugs.python.org/issue12455 11 msgs #8716: test_tk/test_tkk_guionly fails on OS X if run from buildbot sl http://bugs.python.org/issue8716 10 msgs #12291: file written using marshal in 3.2 can be read by 2.7, but not http://bugs.python.org/issue12291 10 msgs #2193: Cookie Colon Name Bug http://bugs.python.org/issue2193 9 msgs #11870: test_3_join_in_forked_from_thread() of test_threading hangs 1 http://bugs.python.org/issue11870 9 msgs #12451: open: avoid the locale encoding when possible http://bugs.python.org/issue12451 9 msgs Issues closed (57) ================== #3435: 3rd party program calls trace.py on non Python files http://bugs.python.org/issue3435 closed by terry.reedy #5114: 2.7: test_threading hangs on Solaris http://bugs.python.org/issue5114 closed by neologix #5375: Unified locals/consts array + register-based instructions http://bugs.python.org/issue5375 closed by pitrou #8661: FAQ item 2.25 is unclear http://bugs.python.org/issue8661 closed by brett.cannon #8746: os.chflags() and os.lchflags() are not built when they should http://bugs.python.org/issue8746 closed by ned.deily #9955: multiprocessing.Pipe segmentation fault when recv of unpicklab http://bugs.python.org/issue9955 closed by neologix #10206: python program starting with unmatched quote spews spaces to s http://bugs.python.org/issue10206 closed by r.david.murray #10326: Can't pickle unittest.TestCase instances http://bugs.python.org/issue10326 closed by rhettinger #10510: packaging upload/register should use CRLF in HTTP requests http://bugs.python.org/issue10510 closed by eric.araujo #10736: test_ttk_guionly fails on OS X using ActiveState Tcl 8.5.9 (Co http://bugs.python.org/issue10736 closed by ned.deily #10845: test_multiprocessing failure under Windows http://bugs.python.org/issue10845 closed by ncoghlan #11163: iter() documentation code doesn't work http://bugs.python.org/issue11163 closed by rhettinger #11197: information leakage with SimpleHTTPServer http://bugs.python.org/issue11197 closed by orsenthil #11302: Add more tests to test_ast.py http://bugs.python.org/issue11302 closed by python-dev #11469: Fix resource warning in test_trailers http://bugs.python.org/issue11469 closed by sandro.tosi #11493: Add python.exe-gdb.py to .hgignore http://bugs.python.org/issue11493 closed by sandro.tosi #11568: docstring of select.epoll.register() is wrong http://bugs.python.org/issue11568 closed by python-dev #11669: Clarify Lang Ref "Compound statements" footnote http://bugs.python.org/issue11669 closed by ezio.melotti #11758: increase xml.dom.minidom test coverage http://bugs.python.org/issue11758 closed by rhettinger #11802: filecmp.cmp needs a documented way to clear cache http://bugs.python.org/issue11802 closed by rhettinger #11889: 'enumerate' 'start' parameter documentation is confusing http://bugs.python.org/issue11889 closed by rhettinger #12086: Tutorial doesn't discourage name mangling http://bugs.python.org/issue12086 closed by rhettinger #12134: json.dump much slower than dumps http://bugs.python.org/issue12134 closed by rhettinger #12139: Add CCC command support to ftplib http://bugs.python.org/issue12139 closed by giampaolo.rodola #12141: sysconfig.get_config_vars('srcdir') fails in specific cases http://bugs.python.org/issue12141 closed by ned.deily #12164: str.translate docstring doesn't mention that 'table' can be No http://bugs.python.org/issue12164 closed by mark.dickinson #12172: IDLE crashes when I use F5 to run http://bugs.python.org/issue12172 closed by ned.deily #12228: Stat doc - fix swap of UF_OPAQUE and UF_NOUNLINK description http://bugs.python.org/issue12228 closed by mark.dickinson #12303: expose sigwaitinfo() and sigtimedwait() in the signal module http://bugs.python.org/issue12303 closed by rosslagerwall #12341: Some additions to .hgignore http://bugs.python.org/issue12341 closed by ezio.melotti #12379: build outside source fail in head http://bugs.python.org/issue12379 closed by ned.deily #12385: the help for bytearray.maketrans describes bytes.maketrans http://bugs.python.org/issue12385 closed by python-dev #12392: pthread_kill() doesn't work on the main thread on FreeBSD6 http://bugs.python.org/issue12392 closed by haypo #12399: simplify cell var initialization by storing constant data on t http://bugs.python.org/issue12399 closed by python-dev #12400: regrtest: always run tests in verbose mode, but hide the outpu http://bugs.python.org/issue12400 closed by haypo #12402: Overriding code.InteractiveConsole.write does not work http://bugs.python.org/issue12402 closed by r.david.murray #12404: c99 code in mmapmodule http://bugs.python.org/issue12404 closed by rosslagerwall #12407: test_subinterps fails on Windows http://bugs.python.org/issue12407 closed by pitrou #12408: Relative import used on test_future5 http://bugs.python.org/issue12408 closed by mark.dickinson #12409: Moving "Documenting Python" to Devguide http://bugs.python.org/issue12409 closed by rhettinger #12417: Inappropriate copyright on profile files http://bugs.python.org/issue12417 closed by python-dev #12419: Add ident parameter to SysLogHandler http://bugs.python.org/issue12419 closed by python-dev #12421: Use PYTHON when calling Parser/asdl_c.py http://bugs.python.org/issue12421 closed by r.david.murray #12422: When deepcopying, don't store immutable objects in the memo di http://bugs.python.org/issue12422 closed by python-dev #12430: Pip fails to fetch from mirror if PyPi checksum times out http://bugs.python.org/issue12430 closed by skrah #12431: urllib2.Request.get_full_url() broken in newer versions of Pyt http://bugs.python.org/issue12431 closed by orsenthil #12433: make clean doesn't clean up static libraries on 2.x http://bugs.python.org/issue12433 closed by ned.deily #12435: Input function does not strip trailing '\r' from string input http://bugs.python.org/issue12435 closed by amaury.forgeotdarc #12442: shutil.disk_usage() http://bugs.python.org/issue12442 closed by giampaolo.rodola #12443: locale.setlocale(locale.LC_ALL, locale.getlocale()) fails for http://bugs.python.org/issue12443 closed by haypo #12444: accept sets or collections for str.strip/lstrip/rstrip http://bugs.python.org/issue12444 closed by rhettinger #12447: ~True is not False http://bugs.python.org/issue12447 closed by amaury.forgeotdarc #12450: Use the Grisu algorithms to convert floats to strings http://bugs.python.org/issue12450 closed by rhettinger #12462: time.sleep(1): call PyErr_CheckSignals() if the sleep was inte http://bugs.python.org/issue12462 closed by haypo #1067702: urllib fails with multiple ftp transfers http://bugs.python.org/issue1067702 closed by python-dev #5722: settimer / gettimer functionality on FreeBSD 6.3 (not 7.x) http://bugs.python.org/issue5722 closed by terry.reedy #1669349: make install fails if no previous Python installation http://bugs.python.org/issue1669349 closed by terry.reedy From vinay_sajip at yahoo.co.uk Fri Jul 1 18:17:03 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Fri, 1 Jul 2011 16:17:03 +0000 (UTC) Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference implementation References: <4E0BFA31.8020200@gmail.com> <4E0D2F42.4040802@gmail.com> Message-ID: Mark Hammond gmail.com> writes: > Sure, that would be awesome! I think that will mean your impl is fairly > close to the first draft of the PEP I checked into HG, which is nice and > still quite useful to use :) My C implementation of the launcher is now available at https://bitbucket.org/vinay.sajip/pylauncher It's been built using VS 2008, and tested on Windows 7 (32-bit) with ActivePython 2.6.2.2 and ActivePython 3.2.0.0 installed. I've added support for .ini files [commands] section; I haven't looked at implementing [defaults] yet. There's incomplete support for -32 suffix at the moment - that can be looked at in due course. A couple of points: I've also allowed for /usr/local/bin/python as a built-in command, this can always be removed later if YAGNI. I've assumed that if a customised command is provided with arguments in the shebang line, these will be ignored - if people want to run with different options they can always define more customised commands. If you agree with this, the PEP should probably explicitly state this. In a couple of cases I've implemented using fixed size arrays - for the lists of installed Pythons and customised commands. Of course these can be made dynamic, but what's there is good enough for the moment for exploration. Do have a play, and let me know what you think. Regards, Vinay Sajip From victor.stinner at haypocalc.com Fri Jul 1 18:25:33 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Fri, 01 Jul 2011 18:25:33 +0200 Subject: [Python-Dev] What happened to Python 3.2.1? In-Reply-To: <20110701164322.72c6f105@pitrou.net> References: <20110701164322.72c6f105@pitrou.net> Message-ID: <1309537533.13804.2.camel@marge> Le vendredi 01 juillet 2011 ? 16:43 +0200, Antoine Pitrou a ?crit : > On Fri, 1 Jul 2011 09:38:04 -0500 > Brian Curtin wrote: > > On Fri, Jul 1, 2011 at 09:01, David P. Riedel wrote: > > > > > Hi > > > > > > Python 3.2.1 was scheduled to be released on 6/19, I believe but there is > > > no mention of it anywhere. Has it been delayed? > > > > > > Thanks. > > > > > > There are two remaining blockers for the release: > > http://bugs.python.org/issue12346 and http://bugs.python.org/issue12291 > > Perhaps http://bugs.python.org/issue12213 should also be a blocker? If you care of interlaced read-write, you may also see http://bugs.python.org/issue12215 I don't think that #12213 and #12215 are blocker. Nobody noticed it since the introduction of the io module (ok, except me), it's not a regression. Python 3.2.1 contains fixes of regressions, users are waiting for them (e.g. "\r" in the Windows console). Can't we schedule another release later to fix #12213 and #12215? Victor From victor.stinner at haypocalc.com Fri Jul 1 20:46:10 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Fri, 01 Jul 2011 20:46:10 +0200 Subject: [Python-Dev] [Python-checkins] cpython (3.2): #11873: fix test regex so it covers windows os.sep as well. In-Reply-To: References: Message-ID: <1309545970.13804.5.camel@marge> Le vendredi 01 juillet 2011 ? 17:54 +0200, r.david.murray a ?crit : > http://hg.python.org/cpython/rev/f8ece8c93918 > changeset: 71119:f8ece8c93918 > branch: 3.2 > parent: 71117:3f30cfe51315 > user: R David Murray > date: Fri Jul 01 11:51:50 2011 -0400 > summary: > #11873: fix test regex so it covers windows os.sep as well. > > files: > Lib/test/test_compileall.py | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > > diff --git a/Lib/test/test_compileall.py b/Lib/test/test_compileall.py > --- a/Lib/test/test_compileall.py > +++ b/Lib/test/test_compileall.py > @@ -248,7 +248,7 @@ > self.assertEqual(b'', quiet) > > def test_regexp(self): > - self.assertRunOK('-q', '-x', 'ba[^\/]*$', self.pkgdir) > + self.assertRunOK('-q', '-x', r'ba[^\/]*$', self.pkgdir) > self.assertNotCompiled(self.barfn) > self.assertCompiled(self.initfn) I'm not sure that your regex is correct. You may have to use a double slash or it is interpreted by the regex :-/ >>> import re >>> re.match(r'ba[^\/]*$', 'babar') <_sre.SRE_Match object at 0x7f420559a7e8> >>> re.match(r'ba[^\/]*$', 'babar/') >>> re.match(r'ba[^\/]*$', 'babar\\a') <_sre.SRE_Match object at 0x7f420559a850> Correct regex: >>> re.match(r'ba[^\\/]*$', 'baba\\') >>> re.match(r'ba[^\\/]*$', 'baba/') Victor From skippy.hammond at gmail.com Sat Jul 2 09:16:07 2011 From: skippy.hammond at gmail.com (Mark Hammond) Date: Sat, 02 Jul 2011 17:16:07 +1000 Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference implementation In-Reply-To: References: <4E0BFA31.8020200@gmail.com> <4E0D2F42.4040802@gmail.com> Message-ID: <4E0EC5B7.9010607@gmail.com> On 1/07/2011 7:20 PM, Vinay Sajip wrote: > Mark Hammond gmail.com> writes: > >> The intention is that there only be a single launcher, as only one app >> can be associated with .py files. OTOH though, file associations can be >> configured per-user IIRC, and assuming that is the case, we could avoid >> my multiple-ini-file usecase above by just allowing a different launcher >> to be registered for a specific user. This is sounding difficult from >> the UI perspective though (ie, does the installer then need to ask that >> question, will there be a post-install technique for per-user >> registration, etc?) > > I don't like this, for the reasons you state. I think it would be better if the > PEP was changed to say that there is intended to be just one launcher > installation per machine. As the intention is for the launcher to ship with > Python, and there can be multiple Pythons installed, I presume we'll have to > handle this by installing the launcher in some common-to-all-Pythons location > somewhere outside a Python installation location, such as "c:\Program > Files\Python Launcher". This should be stated in the PEP, and we'll also need to > indicate how the launcher will be cleaned up if for some strange reason someone > uninstalls all Pythons from a machine. Then we can just state that there's a > global .ini file (where the launcher is installed) and a local one (in the > user's APPDATA). From that perspective, it makes sense to have items in the > local (APPDATA) override the global (launcher installation location). The PEP does say "if possible, should be installed somewhere likely to already be on the system PATH (eg., the Windows System32) directory." It is silent about what to do when that isn't possible, but I'd think it OK if the launcher was installed directly in the Python directory - IOW, I'd think it OK if the PEP said "should be installed next to the PythonXX.dll being installed" - but an important point in the above working is the "already be on the system PATH" - ie, I don't really want it put in a newly created directory unless the installer also adds that directory to the PATH - and what to do on uninstall then becomes an issue. One problem with all of this is uninstallation and specifically if the user is uninstalling the most recent Python installation while leaving earlier ones. I guess there are 2 general answers to this: * The installer process could remember the previous association and restore that on uninstall. * We treat this as a "wont-fix" and document a work-around of asking the user to reinstall the previous version to restore the file association. We probably need to be mindful of adding too much extra work for the installer process as that may well end up blocking us on getting this into the next appropriate release. In particular, Martin's thoughts here would be very useful. This would force the user to reinstall that older one to re-establish the associations correctly > BTW I thought of another thing that perhaps needs handling: what if a customized > command points to the launcher itself? It'd be turtles all the way down :-) Yeah - I wonder if we can leverage the "job" api here and refuse to start if there are already 2 processes in the job? OTOH, that is tricky as it would also prevent someone using os.startfile with a .py file.... From your second mail: > I've assumed that if a customised command is provided with arguments in the > shebang line, these will be ignored - if people want to run with different > options they can always define more customised commands. If you agree with this, > the PEP should probably explicitly state this. I'm not too bothered to be honest - the customized commands exist purely for alternative implementations, so my initial thoughts are that additional args would be as useful for them as they are for cpython invocations. IOW, if they don't need it, then CPython invocations don't need it either, so maybe it can be dropped completely? > In a couple of cases I've implemented using fixed size arrays - for the lists of > installed Pythons and customised commands. Of course these can be made dynamic, > but what's there is good enough for the moment for exploration. Sure - I think there is some policy that a Python version number will never be > 10, so that sounds fine to me. So long as the launcher doesn't blindly run off the end of such arrays I think it is fine - limitations can be addressed in later versions. It will be a few days until I can look at the implementation, but I'm very happy to see it started. Given it is now ahead of the Python reference impl, I wonder if we should just drop all wording about that reference impl and just treat the C impl as canonical? Cheers, Mark From vinay_sajip at yahoo.co.uk Sat Jul 2 11:08:50 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Sat, 2 Jul 2011 09:08:50 +0000 (UTC) Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference implementation References: <4E0BFA31.8020200@gmail.com> <4E0D2F42.4040802@gmail.com> <4E0EC5B7.9010607@gmail.com> Message-ID: Mark Hammond gmail.com> writes: > The PEP does say "if possible, should be installed somewhere likely to > already be on the system PATH (eg., the Windows System32) directory." > It is silent about what to do when that isn't possible, but I'd think it > OK if the launcher was installed directly in the Python directory - IOW, > I'd think it OK if the PEP said "should be installed next to the > PythonXX.dll being installed" - but an important point in the above > working is the "already be on the system PATH" - ie, I don't really want > it put in a newly created directory unless the installer also adds that > directory to the PATH - and what to do on uninstall then becomes an issue. But "next to PythonXY.dll" implies multiple copies, which we want to avoid, right? > One problem with all of this is uninstallation and specifically if the > user is uninstalling the most recent Python installation while leaving > earlier ones. I guess there are 2 general answers to this: > > * The installer process could remember the previous association and > restore that on uninstall. We'd need to do that in the case of the earlier Python not having come with a launcher, i.e. when its version is < 3.3. > * We treat this as a "wont-fix" and document a work-around of asking the > user to reinstall the previous version to restore the file association. This is not ideal from a user's perspective. > We probably need to be mindful of adding too much extra work for the > installer process as that may well end up blocking us on getting this > into the next appropriate release. In particular, Martin's thoughts > here would be very useful. > > This would force the user to reinstall that older one to re-establish > the associations correctly It sounds onerous for users to have to reinstall an older Python. I'm not that familiar with Windows Installer features, so I don't know if this is too fancy, but perhaps we could remember the last non-launcher association when we install the launcher, and either restore that when the launcher is uninstalled (if it's still pointing to an installed Python) or remove the association altogether, otherwise. If this logic is too fancy to include in the installer itself, perhaps we can provide this logic in the launcher itself or via an ancillary executable or DLL that the installer can just call into. Is there some mechanism like the SharedDLLs registry key for executables, which could be used to reference count launcher installations so that it could be uninstalled at the appropriate time? Could we perhaps used the SharedDLLs feature itself? > Yeah - I wonder if we can leverage the "job" api here and refuse to > start if there are already 2 processes in the job? OTOH, that is tricky > as it would also prevent someone using os.startfile with a .py file.... If there's only ever one launcher installed (which we could ensure by installing it in a Windows rather than a Python location) then perhaps we could perhaps check for the value of a customised command pointing at the one-and- only launcher, but this is circumventable. Anyway, perhaps we just need to handle a user error rather than someone deliberately trying to engineer recursion. Another approach might be - rather than limit the number of processes in the job, look to see if the launcher executable is already associated with an existing job. I'm not au fait with the job API, and hence unsure of how that would play with usages such as os.startfile, subprocess etc. > I'm not too bothered to be honest - the customized commands exist purely > for alternative implementations, so my initial thoughts are that > additional args would be as useful for them as they are for cpython > invocations. IOW, if they don't need it, then CPython invocations don't > need it either, so maybe it can be dropped completely? I think they would be useful, so let me check the implementation again. > It will be a few days until I can look at the implementation, but I'm > very happy to see it started. Given it is now ahead of the Python > reference impl, I wonder if we should just drop all wording about that > reference impl and just treat the C impl as canonical? Once you've taken a closer look, if you think it looks good enough, then that's fine. If you have a BitBucket account, I can add your account to the repo so you can push changes to it. Regards, Vinay Sajip From skippy.hammond at gmail.com Sun Jul 3 09:00:43 2011 From: skippy.hammond at gmail.com (Mark Hammond) Date: Sun, 03 Jul 2011 17:00:43 +1000 Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference implementation In-Reply-To: References: <4E0BFA31.8020200@gmail.com> <4E0D2F42.4040802@gmail.com> <4E0EC5B7.9010607@gmail.com> Message-ID: <4E10139B.9050700@gmail.com> On 2/07/2011 7:08 PM, Vinay Sajip wrote: > Mark Hammond gmail.com> writes: > >> The PEP does say "if possible, should be installed somewhere likely to >> already be on the system PATH (eg., the Windows System32) directory." >> It is silent about what to do when that isn't possible, but I'd think it >> OK if the launcher was installed directly in the Python directory - IOW, >> I'd think it OK if the PEP said "should be installed next to the >> PythonXX.dll being installed" - but an important point in the above >> working is the "already be on the system PATH" - ie, I don't really want >> it put in a newly created directory unless the installer also adds that >> directory to the PATH - and what to do on uninstall then becomes an issue. > > But "next to PythonXY.dll" implies multiple copies, which we want to avoid, > right? I believe this will be most useful when people select "install for all users", which will force it into system32. If people select "just for me", then there will be multiple copies on disk, but only one will be active (ie, we will not support different ones being registered for different users. It isn't ideal, but I think it good enough, avoids some complexity and should meet the needs of the majority of users. >> One problem with all of this is uninstallation and specifically if the >> user is uninstalling the most recent Python installation while leaving >> earlier ones. I guess there are 2 general answers to this: >> >> * The installer process could remember the previous association and >> restore that on uninstall. > > We'd need to do that in the case of the earlier Python not having come with a > launcher, i.e. when its version is< 3.3. OTOH, we don't do that now AFAIK. >> * We treat this as a "wont-fix" and document a work-around of asking the >> user to reinstall the previous version to restore the file association. > > This is not ideal from a user's perspective. Sure, but neither is the current situation - I don't recall enough users complaining about that to make the effort worthwhile. I'm not fundamentally opposed to doing something better here - I'm just trying to avoid this kind of stuff being a requirement for acceptance. If you are more familiar with the installer than I and feel it can be done simply, then I'm happy for you to go for it! >> We probably need to be mindful of adding too much extra work for the >> installer process as that may well end up blocking us on getting this >> into the next appropriate release. In particular, Martin's thoughts >> here would be very useful. >> >> This would force the user to reinstall that older one to re-establish >> the associations correctly > > It sounds onerous for users to have to reinstall an older Python. But this only happens when they install a later version, then uninstall the later one and continue to use the old one. I'd suggest that is (a) rare and (b) possibly already broken (ie, the old association may not be restored now). If the old association currently is correctly restored, then I'd expect things to just magically work in this new model without any effort. > I'm not that > familiar with Windows Installer features, so I don't know if this is too fancy, > but perhaps we could remember the last non-launcher association when we install > the launcher, and either restore that when the launcher is uninstalled (if it's > still pointing to an installed Python) or remove the association altogether, > otherwise. If this logic is too fancy to include in the installer itself, > perhaps we can provide this logic in the launcher itself or via an ancillary > executable or DLL that the installer can just call into. I'm still inclined to call YAGNI, but as above, I don't fundamentally oppose such bells-and-whistles. > Is there some mechanism like the SharedDLLs registry key for executables, which > could be used to reference count launcher installations so that it could be > uninstalled at the appropriate time? Could we perhaps used the SharedDLLs > feature itself? > >> Yeah - I wonder if we can leverage the "job" api here and refuse to >> start if there are already 2 processes in the job? OTOH, that is tricky >> as it would also prevent someone using os.startfile with a .py file.... > > If there's only ever one launcher installed (which we could ensure by > installing it in a Windows rather than a Python location) We probably can't ensure this while the installer supports a non-UAC installer (ie, the "just for me" option.) OTOH, I'm not sure the "just for me" option currently works without needing UAC elevation anyway. But assuming it does (or the intention is to fix things if it does not), then we can't guarantee a writable system32. > then perhaps we could > perhaps check for the value of a customised command pointing at the one-and- > only launcher, but this is circumventable. Anyway, perhaps we just need to > handle a user error rather than someone deliberately trying to engineer > recursion. Yeah, I agree. > Another approach might be - rather than limit the number of processes in the > job, look to see if the launcher executable is already associated with an > existing job. I'm not au fait with the job API, and hence unsure of how that > would play with usages such as os.startfile, subprocess etc. Exactly - code doing os.startfile on a .py file will correctly cause the same launcher to be re-executed and preventing this would break such scripts. However, I think this would be extremely rare and the more usual case would be to use sys.executable. Indeed, any script using os.startfile for a .py file is already broken, even if the author doesn't know it yet :) >> I'm not too bothered to be honest - the customized commands exist purely >> for alternative implementations, so my initial thoughts are that >> additional args would be as useful for them as they are for cpython >> invocations. IOW, if they don't need it, then CPython invocations don't >> need it either, so maybe it can be dropped completely? > > I think they would be useful, so let me check the implementation again. > >> It will be a few days until I can look at the implementation, but I'm >> very happy to see it started. Given it is now ahead of the Python >> reference impl, I wonder if we should just drop all wording about that >> reference impl and just treat the C impl as canonical? > > Once you've taken a closer look, if you think it looks good enough, then that's > fine. If you have a BitBucket account, I can add your account to the repo so > you can push changes to it. Great, thanks! Mark From vinay_sajip at yahoo.co.uk Sun Jul 3 12:13:43 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Sun, 3 Jul 2011 10:13:43 +0000 (UTC) Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference implementation References: <4E0BFA31.8020200@gmail.com> <4E0D2F42.4040802@gmail.com> <4E0EC5B7.9010607@gmail.com> <4E10139B.9050700@gmail.com> Message-ID: Mark Hammond gmail.com> writes: > I'm not fundamentally opposed to doing something better here - I'm just > trying to avoid this kind of stuff being a requirement for acceptance. > If you are more familiar with the installer than I and feel it can be > done simply, then I'm happy for you to go for it! No, you're right - I'm just throwing ideas out. I'm not especially familiar with MSI in general, though I believe you can do the relevant things with logic in custom actions. AFAIK there are already custom actions used in the Python MSI, but I wouldn't want to propose any such changes to the MSI unless Martin were to make encouraging comments :-) > But this only happens when they install a later version, then uninstall > the later one and continue to use the old one. I'd suggest that is (a) > rare and (b) possibly already broken (ie, the old association may not be > restored now). If the old association currently is correctly restored, > then I'd expect things to just magically work in this new model without > any effort. As to (a), not uncommon scenarios would be (i) later version breaks something, so go back to earlier version, or (ii) using 2.x, installing 3.x to try out, going back to 2.x. I'm not sure about (b). > I'm still inclined to call YAGNI, but as above, I don't fundamentally > oppose such bells-and-whistles. You're probably right about the cost(effort)-benefit trade-off. > We probably can't ensure this while the installer supports a non-UAC > installer (ie, the "just for me" option.) OTOH, I'm not sure the "just > for me" option currently works without needing UAC elevation anyway. > But assuming it does (or the intention is to fix things if it does not), > then we can't guarantee a writable system32. Okay. From a cursory glance at msi.py, UAC elevation appears to be requested. > Exactly - code doing os.startfile on a .py file will correctly cause the > same launcher to be re-executed and preventing this would break such > scripts. However, I think this would be extremely rare and the more > usual case would be to use sys.executable. Indeed, any script using > os.startfile for a .py file is already broken, even if the author > doesn't know it yet :) I had a closer look at the Job API, and it does seem to offer the required functionality, so we could tell whether the current process is in a job -> get all processes in that job -> get their executables and command lines -> see if recursion is occurring. However, would we class this as an implementation detail or does it need a mention in the PEP? > > I think they would be useful, so let me check the implementation again. I've made some updates so the following works: with the configuration containing [commands] shell=cmd /c with a shebang in doit.py of '#!shell python -v' the launcher will run 'cmd /c python -v doit.py'. Regards, Vinay Sajip From vinay_sajip at yahoo.co.uk Sun Jul 3 15:06:43 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Sun, 3 Jul 2011 13:06:43 +0000 (UTC) Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference implementation References: <4E0BFA31.8020200@gmail.com> <4E0D2F42.4040802@gmail.com> <4E0EC5B7.9010607@gmail.com> <4E10139B.9050700@gmail.com> Message-ID: Mark Hammond gmail.com> writes: > But this only happens when they install a later version, then uninstall > the later one and continue to use the old one. I'd suggest that is (a) > rare and (b) possibly already broken (ie, the old association may not be > restored now). If the old association currently is correctly restored, > then I'd expect things to just magically work in this new model without > any effort. I've now checked, and it appears that we don't do the right thing now anyway. On a clean Win7-x64, I installed Python 2.7 (32-bit), for all users, with registration of extensions - and Python.File etc. were added to the registry pointing at Python 2.7. I then installed Python 3.2 the same way, and rhe registry entries then pointed to 3.2. I then uninstalled 3.2, and the registry entries were gone! No magical restoration to the earlier values :-( Okay - if users have to do this now anyway, we at least wouldn't be naking things worse :-) Regards, Vinay Sajip From p.f.moore at gmail.com Sun Jul 3 17:39:07 2011 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 3 Jul 2011 16:39:07 +0100 Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference implementation In-Reply-To: References: <4E0BFA31.8020200@gmail.com> <4E0C26FB.9040704@timgolden.me.uk> <4E0C5A55.6000706@voidspace.org.uk> Message-ID: On 30 June 2011 13:50, Paul Moore wrote: > On 30 June 2011 12:13, Michael Foord wrote: >> I have that email (the update one from Mark not the silent nodding from Tim) >> still sitting in my inbox waiting for me to properly read through and >> comment on... Sorry for being useless, I'll try and move it up the priority >> list. >> >> I really like the PEP and think it will be a *huge* step forward for Windows >> users - so it's purely the details that need thrashing out (heh). > > That's my situation, too. I'll try to look through it properly in the > next day or two. OK, having looked through this, it looks pretty solid to me. I might try installing Vinay's implementation and seeing how it feels in use, as well... On restoring associations, I think it's entirely reasonable not to bother. It would be nice if py.exe remained installed as long as any (all-users) Python installation remained on the PC, but this may be complicated (given that older versions won't uninstall it) so maybe don't even bother with that. Actually, I'd question whether this shouldn't be packaged as a separate application, available as an independent download from python.org (I do think it's important enough to be hosted on python.org rather than PyPI, though). Future versions of python could bundle it as well, but that could be purely for convenience and to ensure that users don't miss it. If Python does bundle it, then I'd suggest that it remain a separate item in add/remove programs. That allows the user to choose whether to uninstall it or not. As py.exe and python live in separate directories (except in per-user installs) the installers don't have any nasty interactions like who deletes the directory to worry about. If you want to bother with an "active installation count" mechanism (whether SharedDLLs or a simple count/listing in the py.ini file, or something else) then when the count hits "No", just offer the user the choice to uninstall as part of the Python uninstall. I'd be inclined not to worry too much about per-user installations. Are per-user file associations possible? I've never used them, myself. As a simple approach, just install py.exe alongside python.exe in the user dir, don't worry about it being on PATH or having a separate add/remove programs item, and let the user do what he wants with it. Of course, if someone with more experience of per-user installs and the sorts of environment where they are needed wants to suggest something different, believe them rather than me! Basically, my feeling is that the core functionality is fine. Nothing in this will make life worse for anyone, so we can safely leave corner cases to be addressed later (with a standalone release, if it's urgent enough). Paul. From vinay_sajip at yahoo.co.uk Sun Jul 3 20:20:32 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Sun, 3 Jul 2011 18:20:32 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?PEP_397_=28Python_launcher_for_Windows=29_?= =?utf-8?q?reference=09implementation?= References: <4E0BFA31.8020200@gmail.com> <4E0C26FB.9040704@timgolden.me.uk> <4E0C5A55.6000706@voidspace.org.uk> Message-ID: Paul Moore gmail.com> writes: > OK, having looked through this, it looks pretty solid to me. I might > try installing Vinay's implementation and seeing how it feels in use, > as well... Do have a play, it would be nice to get feedback. It's only available as source, though - is that OK? > I'd be inclined not to worry too much about per-user installations. > Are per-user file associations possible? I've never used them, myself. Yes, and they can be problematic: see http://www.deadlybloodyserious.com/2007/05/no-script-arguments/ where the problem described has also bitten me. I've no idea what put that association in the registry. Regards, Vinay Sajip From p.f.moore at gmail.com Sun Jul 3 20:33:01 2011 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 3 Jul 2011 19:33:01 +0100 Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference implementation In-Reply-To: References: <4E0BFA31.8020200@gmail.com> <4E0C26FB.9040704@timgolden.me.uk> <4E0C5A55.6000706@voidspace.org.uk> Message-ID: On 3 July 2011 19:20, Vinay Sajip wrote: > Paul Moore gmail.com> writes: > >> OK, having looked through this, it looks pretty solid to me. I might >> try installing Vinay's implementation and seeing how it feels in use, >> as well... > > Do have a play, it would be nice to get feedback. It's only available as source, > though - is that OK? No problem - I can build it from source. >> I'd be inclined not to worry too much about per-user installations. >> Are per-user file associations possible? I've never used them, myself. > > Yes, and they can be problematic: see > > http://www.deadlybloodyserious.com/2007/05/no-script-arguments/ > > where the problem described has also bitten me. I've no idea what put that > association in the registry. Ugh. That's nasty. It doesn't even look like a standard file type association. Paul. From listas at alejolp.com Mon Jul 4 01:24:57 2011 From: listas at alejolp.com (Alejandro Santos) Date: Sun, 3 Jul 2011 20:24:57 -0300 Subject: [Python-Dev] Extending documentation innacuracies Message-ID: Hi, While reading through the Extending and Embedding docs for Python 3.2 I've noticed something. While the "Py_InitModule" does not exists on Py3k, it is still mentioned on the docs: http://docs.python.org/py3k/extending/extending.html#keyword-parameters-for-extension-functions http://docs.python.org/py3k/extending/windows.html#a-cookbook-approach http://docs.python.org/py3k/faq/extending.html#what-does-systemerror-pyimport-fixupextension-module-yourmodule-not-loaded-mean Should I report a new issue? -- Alejandro Santos http://alejolp.com.ar From ncoghlan at gmail.com Mon Jul 4 02:33:42 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 4 Jul 2011 10:33:42 +1000 Subject: [Python-Dev] Extending documentation innacuracies In-Reply-To: References: Message-ID: On Mon, Jul 4, 2011 at 9:24 AM, Alejandro Santos wrote: > Should I report a new issue? Yes please, tracker items are the best way to get that kind of oversight cleaned up. Thanks for noticing! Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Mon Jul 4 03:20:03 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 4 Jul 2011 11:20:03 +1000 Subject: [Python-Dev] [Python-checkins] cpython: no one passes NULL here (or should anyway) In-Reply-To: References: Message-ID: On Mon, Jul 4, 2011 at 8:02 AM, benjamin.peterson wrote: > http://hg.python.org/cpython/rev/bbbeddafeec0 > changeset: ? 71160:bbbeddafeec0 > user: ? ? ? ?Benjamin Peterson > date: ? ? ? ?Sun Jul 03 17:06:32 2011 -0500 > summary: > ?no one passes NULL here (or should anyway) > > files: > ?Python/ceval.c | ?3 --- > ?1 files changed, 0 insertions(+), 3 deletions(-) > > > diff --git a/Python/ceval.c b/Python/ceval.c > --- a/Python/ceval.c > +++ b/Python/ceval.c > @@ -1115,9 +1115,6 @@ > > ?/* Start of code */ > > - ? ?if (f == NULL) > - ? ? ? ?return NULL; > - May need to replace that with an assert(f != NULL) to keep static analysers happy. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From benjamin at python.org Mon Jul 4 04:54:23 2011 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 3 Jul 2011 21:54:23 -0500 Subject: [Python-Dev] [Python-checkins] cpython: no one passes NULL here (or should anyway) In-Reply-To: References: Message-ID: 2011/7/3 Nick Coghlan : > On Mon, Jul 4, 2011 at 8:02 AM, benjamin.peterson > wrote: >> http://hg.python.org/cpython/rev/bbbeddafeec0 >> changeset: ? 71160:bbbeddafeec0 >> user: ? ? ? ?Benjamin Peterson >> date: ? ? ? ?Sun Jul 03 17:06:32 2011 -0500 >> summary: >> ?no one passes NULL here (or should anyway) >> >> files: >> ?Python/ceval.c | ?3 --- >> ?1 files changed, 0 insertions(+), 3 deletions(-) >> >> >> diff --git a/Python/ceval.c b/Python/ceval.c >> --- a/Python/ceval.c >> +++ b/Python/ceval.c >> @@ -1115,9 +1115,6 @@ >> >> ?/* Start of code */ >> >> - ? ?if (f == NULL) >> - ? ? ? ?return NULL; >> - > > May need to replace that with an assert(f != NULL) to keep static > analysers happy. Surely static analyzers don't assume every argument passed in is NULL? -- Regards, Benjamin From ncoghlan at gmail.com Mon Jul 4 05:44:59 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 4 Jul 2011 13:44:59 +1000 Subject: [Python-Dev] [Python-checkins] cpython: no one passes NULL here (or should anyway) In-Reply-To: References: Message-ID: On Mon, Jul 4, 2011 at 12:54 PM, Benjamin Peterson wrote: > 2011/7/3 Nick Coghlan : >> May need to replace that with an assert(f != NULL) to keep static >> analysers happy. > > Surely static analyzers don't assume every argument passed in is NULL? I didn't check - was this change in a static function? For those, I think they can figure it out. For functions exposed to the linker, I think they demand an explicit check for a non-NULL pointer (which may be in the form of an assertion). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From benjamin at python.org Mon Jul 4 06:15:53 2011 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 3 Jul 2011 23:15:53 -0500 Subject: [Python-Dev] [Python-checkins] cpython: no one passes NULL here (or should anyway) In-Reply-To: References: Message-ID: 2011/7/3 Nick Coghlan : > On Mon, Jul 4, 2011 at 12:54 PM, Benjamin Peterson wrote: >> 2011/7/3 Nick Coghlan : >>> May need to replace that with an assert(f != NULL) to keep static >>> analysers happy. >> >> Surely static analyzers don't assume every argument passed in is NULL? > > I didn't check - was this change in a static function? For those, I > think they can figure it out. For functions exposed to the linker, I > think they demand an explicit check for a non-NULL pointer (which may > be in the form of an assertion). If someone's static analysis tool starts complaining about it, I'd be happy to consider adding an assert... -- Regards, Benjamin From greg.ewing at canterbury.ac.nz Mon Jul 4 07:59:14 2011 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 04 Jul 2011 17:59:14 +1200 Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference implementation In-Reply-To: <4E10139B.9050700@gmail.com> References: <4E0BFA31.8020200@gmail.com> <4E0D2F42.4040802@gmail.com> <4E0EC5B7.9010607@gmail.com> <4E10139B.9050700@gmail.com> Message-ID: <4E1156B2.6050404@canterbury.ac.nz> Mark Hammond wrote: > On 2/07/2011 7:08 PM, Vinay Sajip wrote: >> perhaps we could remember the last non-launcher association when >> we install the launcher, It might be better to look in the registry for other Python installations and ask the user which one to restore if there is more than one. Trying to restore the "last" one would be prone to breakage if the user didn't uninstall versions in precisely the reverse of the order of installation. -- Greg From skippy.hammond at gmail.com Mon Jul 4 08:27:17 2011 From: skippy.hammond at gmail.com (Mark Hammond) Date: Mon, 04 Jul 2011 16:27:17 +1000 Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference implementation In-Reply-To: <4E1156B2.6050404@canterbury.ac.nz> References: <4E0BFA31.8020200@gmail.com> <4E0D2F42.4040802@gmail.com> <4E0EC5B7.9010607@gmail.com> <4E10139B.9050700@gmail.com> <4E1156B2.6050404@canterbury.ac.nz> Message-ID: <4E115D45.8000101@gmail.com> On 4/07/2011 3:59 PM, Greg Ewing wrote: > Mark Hammond wrote: > >> On 2/07/2011 7:08 PM, Vinay Sajip wrote: > >>> perhaps we could remember the last non-launcher association when we >>> install the launcher, > > It might be better to look in the registry for other Python > installations and ask the user which one to restore if there > is more than one. Trying to restore the "last" one would be > prone to breakage if the user didn't uninstall versions in > precisely the reverse of the order of installation. While that makes alot of sense, the fact we are already "broken" in exactly the same way means I hope we can treat the restoration of associations as a separate issue - a worthwhile one, but not a pre-requisite for this PEP being approved. Cheers, Mark From aigarius at gmail.com Mon Jul 4 12:10:57 2011 From: aigarius at gmail.com (Aigars Mahinovs) Date: Mon, 4 Jul 2011 13:10:57 +0300 Subject: [Python-Dev] Threaded asynchronous return from functions Message-ID: Short version: Could we get def funct( args ): if args == 'good': return 'good' async_return "I'll think about it" if args == '123': do_x() elif args == 'abc': do_y() else: do_z() as equivalent to def do_thinking( args ): if args == '123': do_x() elif args == 'abc': do_y() else: do_z() def funct( args ): if args == 'good': return 'good' t = threading.Thread(target=do_thinking, args=args) t.start() return "I'll think about it" Longer version: I have been doing some multithreaded work lately and have found that often what I find wanting to do is to call a function, have it check it's arguments, possibly do some work and then return to the caller, but still do some extra processing right after that. Currently to accomplish such feat I need to separate the 'extra processing' bit into a separate function and call that in a separate thread. A nice convenience would be a function or statement that would allow to return a value from the current function, but still keep running its code (in a separate thread). Such approach could then be used in many places where async processing is required, such as GUI programming, XMLRPC, web applications, ... with less boilerplate and more obvious code flow. -- Best regards, ? ? Aigars Mahinovs? ? ? ? mailto:aigarius at debian.org ? #--------------------------------------------------------------# ?| .''`.? ? Debian GNU/Linux (http://www.debian.org)? ? ? ? ? ? | ?| : :' :?? Latvian Open Source Assoc. (http://www.laka.lv)? ?? | ?| `. `'? ? Linux Administration and Free Software Consulting?? | ?|?? `-? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?? (http://www.aiteki.com) | ?#--------------------------------------------------------------# From amauryfa at gmail.com Mon Jul 4 13:11:49 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Mon, 4 Jul 2011 13:11:49 +0200 Subject: [Python-Dev] Threaded asynchronous return from functions In-Reply-To: References: Message-ID: Hello, 2011/7/4 Aigars Mahinovs > I have been doing some multithreaded work lately and have found that often what I find wanting to do is to call a function, have it check > it's arguments, possibly do some work and then return to the caller, > but still do some extra processing right after that. Currently to > accomplish such feat I need to separate the 'extra processing' bit > into a separate function and call that in a separate thread. A nice > convenience would be a function or statement that would allow to > return a value from the current function, but still keep running its > code (in a separate thread). Such approach could then be used in many > places where async processing is required, such as GUI programming, > XMLRPC, web applications, ... with less boilerplate and more obvious > code flow. > This kind of topic is not suitable to python-dev. Please ask this on the python-list mailing list, or eventually on python-ideas. (where someone will probably suggest you to use a nested function) Cheers, -- Amaury Forgeot d'Arc -------------- next part -------------- An HTML attachment was scrubbed... URL: From peck at us.ibm.com Mon Jul 4 12:04:18 2011 From: peck at us.ibm.com (Jon K Peck) Date: Mon, 4 Jul 2011 04:04:18 -0600 Subject: [Python-Dev] AUTO: Jon K Peck is out of the office (returning 07/15/2011) Message-ID: I am out of the office until 07/15/2011. I am out of the office traveling Monday July 4 through Friday, July 15 I will have little or no access to email during this time, so I will be delayed in responding. Note: This is an automated response to your message "Python-Dev Digest, Vol 96, Issue 6" sent on 7/4/11 0:27:26. This is the only notification you will receive while this person is away. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jnoller at gmail.com Mon Jul 4 17:51:37 2011 From: jnoller at gmail.com (Jesse Noller) Date: Mon, 4 Jul 2011 11:51:37 -0400 Subject: [Python-Dev] Speed.Python.org Message-ID: Now that we have the machine, we need to start working on collecting/organizing the resources needed to get a shared codespeed system in place. After speaking with various people, we felt that overloading codespeed-dev, pypy-dev or python-dev with the discussions around this would be sub optimal. I've spun up a new mailing list here: http://mail.python.org/mailman/listinfo/speed Those who are interested in working on or contributing to the speed.python.org project can subscribe there. I personally can not lead the project, and so I will be looking to the current speed.pypy.org team, and python-dev contributors for leadership in this. I got you the hardware and hosting! :) jesse From solipsis at pitrou.net Mon Jul 4 18:23:44 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 4 Jul 2011 18:23:44 +0200 Subject: [Python-Dev] cpython: Issue #12469: replace assertions by explicit if+raise References: Message-ID: <20110704182344.7bad6b62@pitrou.net> On Mon, 04 Jul 2011 18:06:53 +0200 victor.stinner wrote: > http://hg.python.org/cpython/rev/7eef821ab20d > changeset: 71197:7eef821ab20d > user: Victor Stinner > date: Mon Jul 04 18:06:35 2011 +0200 > summary: > Issue #12469: replace assertions by explicit if+raise Instead of generic Exception, it would be better to use AssertionError. Regards Antoine. From g.brandl at gmx.net Mon Jul 4 19:37:47 2011 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 04 Jul 2011 19:37:47 +0200 Subject: [Python-Dev] cpython: Issue #12469: replace assertions by explicit if+raise In-Reply-To: <20110704182344.7bad6b62@pitrou.net> References: <20110704182344.7bad6b62@pitrou.net> Message-ID: Am 04.07.2011 18:23, schrieb Antoine Pitrou: > On Mon, 04 Jul 2011 18:06:53 +0200 > victor.stinner wrote: >> http://hg.python.org/cpython/rev/7eef821ab20d >> changeset: 71197:7eef821ab20d >> user: Victor Stinner >> date: Mon Jul 04 18:06:35 2011 +0200 >> summary: >> Issue #12469: replace assertions by explicit if+raise > > Instead of generic Exception, it would be better to use AssertionError. What is the reason for this change anyway -- as far as I can see this code is never run with -O. Also I don't see how it relates to #12469. Georg From georg at python.org Mon Jul 4 19:48:50 2011 From: georg at python.org (Georg Brandl) Date: Mon, 04 Jul 2011 19:48:50 +0200 Subject: [Python-Dev] [RELEASED] Python 3.2.1 rc 2 Message-ID: <4E11FD02.3040205@python.org> On behalf of the Python development team, I am pleased to announce the second release candidate of Python 3.2.1. Python 3.2.1 will the first bugfix release for Python 3.2, fixing over 120 bugs and regressions in Python 3.2. For an extensive list of changes and features in the 3.2 line, see http://docs.python.org/3.2/whatsnew/3.2.html To download Python 3.2.1 visit: http://www.python.org/download/releases/3.2.1/ This is a testing release: Please consider trying Python 3.2.1 with your code and reporting any bugs you may notice to: http://bugs.python.org/ Enjoy! -- Georg Brandl, Release Manager georg at python.org (on behalf of the entire python-dev team and 3.2's contributors) From greg at krypto.org Mon Jul 4 19:48:45 2011 From: greg at krypto.org (Gregory P. Smith) Date: Mon, 4 Jul 2011 10:48:45 -0700 Subject: [Python-Dev] cpython: Issue #12469: replace assertions by explicit if+raise In-Reply-To: <20110704182344.7bad6b62@pitrou.net> References: <20110704182344.7bad6b62@pitrou.net> Message-ID: On Mon, Jul 4, 2011 at 9:23 AM, Antoine Pitrou wrote: > On Mon, 04 Jul 2011 18:06:53 +0200 > victor.stinner wrote: > > http://hg.python.org/cpython/rev/7eef821ab20d > > changeset: 71197:7eef821ab20d > > user: Victor Stinner > > date: Mon Jul 04 18:06:35 2011 +0200 > > summary: > > Issue #12469: replace assertions by explicit if+raise > > Instead of generic Exception, it would be better to use AssertionError. > or in many cases given this was in unittests... use the self.assertFoo methods and avoid assert and if statements all together. > > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/greg%40krypto.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From victor.stinner at haypocalc.com Mon Jul 4 20:55:06 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Mon, 04 Jul 2011 20:55:06 +0200 Subject: [Python-Dev] cpython: Issue #12469: replace assertions by explicit if+raise In-Reply-To: <20110704182344.7bad6b62@pitrou.net> References: <20110704182344.7bad6b62@pitrou.net> Message-ID: <1309805706.30323.4.camel@marge> Le lundi 04 juillet 2011 ? 18:23 +0200, Antoine Pitrou a ?crit : > On Mon, 04 Jul 2011 18:06:53 +0200 > victor.stinner wrote: > > http://hg.python.org/cpython/rev/7eef821ab20d > > changeset: 71197:7eef821ab20d > > user: Victor Stinner > > date: Mon Jul 04 18:06:35 2011 +0200 > > summary: > > Issue #12469: replace assertions by explicit if+raise > Instead of generic Exception, it would be better to use AssertionError. and > or in many cases given this was in unittests... use the self.assertFoo > methods and avoid assert and if statements all together. The code is running in a subprocess (python -c ...), not in an unittest.TestCase, so I cannot use self.assertFoo and it doesn't really matter if the exception is an Exception or an AssertionError. > What is the reason for this change anyway -- as far as I can > see this code is never run with -O. I'm not sure that the code will never be running using -O, so I prefer to use an explicit if+raise. I don't like the assert statement because it doesn't provide any information about the failure (content of the variables) by default. Victor From digitalxero at gmail.com Mon Jul 4 21:04:44 2011 From: digitalxero at gmail.com (Dj Gilcrease) Date: Mon, 4 Jul 2011 15:04:44 -0400 Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference implementation In-Reply-To: <4E115D45.8000101@gmail.com> References: <4E0BFA31.8020200@gmail.com> <4E0D2F42.4040802@gmail.com> <4E0EC5B7.9010607@gmail.com> <4E10139B.9050700@gmail.com> <4E1156B2.6050404@canterbury.ac.nz> <4E115D45.8000101@gmail.com> Message-ID: On Mon, Jul 4, 2011 at 2:27 AM, Mark Hammond wrote: > While that makes alot of sense, the fact we are already "broken" in exactly > the same way means I hope we can treat the restoration of associations as a > separate issue - a worthwhile one, but not a pre-requisite for this PEP > being approved. I agree let the restoration or not of file association be an implementation detail and not a requirement. I also agree with Paul Moore that py.exe should be a separate install/uninstall It can be bundled with the python MSI but should be a standalone MSI that the python MSI executes. So uninstalling the version of python that installed it alone is not enough to remove it. This will make it seem like the file associations are just working for naive users but will still let them remove the associations. We can just put a warning in the uninstall process of py.exe that lets the user know if they still have python versions installed they will need to re-install them to get the file association again. From senthil at uthcode.com Mon Jul 4 23:30:51 2011 From: senthil at uthcode.com (Senthil Kumaran) Date: Mon, 4 Jul 2011 14:30:51 -0700 Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members in documentation In-Reply-To: References: <20110626185242.GB2710@mathmagic> <20110627102428.6fc1955b@msiwind> <4E09085D.80202@voidspace.org.uk> Message-ID: <20110704213051.GA4320@mathmagic> On Tue, Jun 28, 2011 at 10:30:14AM +1000, Nick Coghlan wrote: > Rather than fighting that convention, we should probably just confront > the ambiguity head on and update > http://docs.python.org/dev/glossary.html#term-attribute to describe > both uses of the term (and add a separate entry for "data attribute", > with a definition which basically says "attributes which are not > methods"). http://bugs.python.org/issue12491 is the issue to track it. The glossary term should give us a stance on what is meant by attributes. For the other issue (10403), I just concentrated on removing the term members and used attributes and methods appropriately focussing on clarity rather than presenting the detail on the object model. For our rescue, sphinx reST provide :attr: for attribute and :meth: for methods. :) -- Senthil From vinay_sajip at yahoo.co.uk Tue Jul 5 00:05:31 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Mon, 4 Jul 2011 22:05:31 +0000 (UTC) Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference implementation References: <4E0BFA31.8020200@gmail.com> <4E0D2F42.4040802@gmail.com> <4E0EC5B7.9010607@gmail.com> <4E10139B.9050700@gmail.com> <4E1156B2.6050404@canterbury.ac.nz> <4E115D45.8000101@gmail.com> Message-ID: Mark Hammond gmail.com> writes: > > It might be better to look in the registry for other Python > > installations and ask the user which one to restore if there > > is more than one. Trying to restore the "last" one would be > > prone to breakage if the user didn't uninstall versions in > > precisely the reverse of the order of installation. > > While that makes alot of sense, the fact we are already "broken" in > exactly the same way means I hope we can treat the restoration of > associations as a separate issue - a worthwhile one, but not a > pre-requisite for this PEP being approved. I agree, but there's one aspect of associations which is perhaps worth exploring: the installation of a pre-3.3 version of Python after Python 3.3 is installed with the launcher will, if the user selects "Register Extensions", hijack the laumcher's associations to that earlier Python. Then bye bye launcher - how do we deal with that? Or don't we? There'll be no warning for the user, and this problem will occur even if the launcher is packaged separately from Python. so I think we need to think about this a little more. What say? Regards, Vinay Sajip From jimjjewett at gmail.com Tue Jul 5 00:27:33 2011 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 4 Jul 2011 18:27:33 -0400 Subject: [Python-Dev] [Python-checkins] cpython: Remove mention of medical condition from the test suite. In-Reply-To: References: Message-ID: If you're going to get rid of the pun, you might as well change the whole sentence... On Sun, Jul 3, 2011 at 1:22 PM, georg.brandl wrote: > http://hg.python.org/cpython/rev/76452b892838 > changeset: ? 71146:76452b892838 > parent: ? ? ?71144:ce52310f61a0 > user: ? ? ? ?Georg Brandl > date: ? ? ? ?Sun Jul 03 19:22:42 2011 +0200 > summary: > ?Remove mention of medical condition from the test suite. > > files: > ?Lib/test/test_csv.py | ?8 ++++---- > ?1 files changed, 4 insertions(+), 4 deletions(-) > > > diff --git a/Lib/test/test_csv.py b/Lib/test/test_csv.py > --- a/Lib/test/test_csv.py > +++ b/Lib/test/test_csv.py > @@ -459,20 +459,20 @@ > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?'5', '6']]) > > ? ? def test_quoted_quote(self): > - ? ? ? ?self.readerAssertEqual('1,2,3,"""I see,"" said the blind man","as he picked up his hammer and saw"', > + ? ? ? ?self.readerAssertEqual('1,2,3,"""I see,"" said the happy man","as he picked up his hammer and saw"', > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[['1', '2', '3', > - ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? '"I see," said the blind man', > + ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? '"I see," said the happy man', > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?'as he picked up his hammer and saw']]) > > ? ? def test_quoted_nl(self): > ? ? ? ? input = '''\ > ?1,2,3,"""I see,"" > -said the blind man","as he picked up his > +said the happy man","as he picked up his > ?hammer and saw" > ?9,8,7,6''' > ? ? ? ? self.readerAssertEqual(input, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[['1', '2', '3', > - ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? '"I see,"\nsaid the blind man', > + ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? '"I see,"\nsaid the happy man', > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?'as he picked up his\nhammer and saw'], > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ['9','8','7','6']]) > > > -- > Repository URL: http://hg.python.org/cpython > > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > > From vinay_sajip at yahoo.co.uk Tue Jul 5 01:19:39 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Mon, 4 Jul 2011 23:19:39 +0000 (UTC) Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference implementation References: <4E0BFA31.8020200@gmail.com> <4E0D2F42.4040802@gmail.com> <4E0EC5B7.9010607@gmail.com> <4E10139B.9050700@gmail.com> <4E1156B2.6050404@canterbury.ac.nz> <4E115D45.8000101@gmail.com> Message-ID: One more thing about associations - we've got pyw.exe for Python.NoConFile and py.exe for Python.file, but how do we handle Python.CompiledFile? It doesn't really make sense to have the association not handled by the launcher. Unfortunately, of course, both pyw and py compile to pyo, so we don't know which launcher to use. It is, of course, easy for either py or pyw to determine which version of python is to be used to invoke a .pyc - just not which Windows variant. BTW just as a test I implemented .pyc support in the C implementation - it works fine apart from the "python.exe or pythonw.exe?" question. Regards, Vinay Sajip From greg.ewing at canterbury.ac.nz Tue Jul 5 03:23:20 2011 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 05 Jul 2011 13:23:20 +1200 Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference implementation In-Reply-To: References: <4E0BFA31.8020200@gmail.com> <4E0D2F42.4040802@gmail.com> <4E0EC5B7.9010607@gmail.com> <4E10139B.9050700@gmail.com> <4E1156B2.6050404@canterbury.ac.nz> <4E115D45.8000101@gmail.com> Message-ID: <4E126788.7020705@canterbury.ac.nz> Vinay Sajip wrote: > the installation of a pre-3.3 version of Python after Python 3.3 is > installed with the launcher will, if the user selects "Register Extensions", > hijack the laumcher's associations to that earlier Python. Then bye bye launcher I don't see how anything can be done about that. It also doesn't seem like too serious a problem -- it's no worse than what currently happens if you install an older version over a newer one, i.e. associations revert to the old version. Making the launcher a separate install at least offers a way of fixing that if it happens and isn't desired, i.e. reinstall the launcher. -- Greg From skippy.hammond at gmail.com Tue Jul 5 04:12:54 2011 From: skippy.hammond at gmail.com (Mark Hammond) Date: Tue, 05 Jul 2011 12:12:54 +1000 Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference implementation In-Reply-To: <4E126788.7020705@canterbury.ac.nz> References: <4E0BFA31.8020200@gmail.com> <4E0D2F42.4040802@gmail.com> <4E0EC5B7.9010607@gmail.com> <4E10139B.9050700@gmail.com> <4E1156B2.6050404@canterbury.ac.nz> <4E115D45.8000101@gmail.com> <4E126788.7020705@canterbury.ac.nz> Message-ID: <4E127326.7090202@gmail.com> On 5/07/2011 11:23 AM, Greg Ewing wrote: > Vinay Sajip wrote: >> the installation of a pre-3.3 version of Python after Python 3.3 is >> installed with the launcher will, if the user selects "Register >> Extensions", >> hijack the laumcher's associations to that earlier Python. Then bye >> bye launcher > > I don't see how anything can be done about that. It > also doesn't seem like too serious a problem -- it's > no worse than what currently happens if you install an > older version over a newer one, i.e. associations revert > to the old version. > > Making the launcher a separate install at least offers > a way of fixing that if it happens and isn't desired, > i.e. reinstall the launcher. Or an MSI installer may be able to offer a "repair" feature without too much pain. However, I'm a bit torn on the separate installer issue. I really think it should be installed with later Python versions even if it is made available as a separate installer for the benefit of earlier ones as it is one less step for people to get confused about - eg, in a few years when 3.3+ is the most common Python being downloaded and used, books or people helping on python-list could start referring to the launcher under the assumption it is installed, rather than avoiding mention of it simply to avoid the whole spiel about "this will only work if you have installed an optional package." IOW, it will be impossible to give unconditional advice on certain aspects of launching Python. If the launcher is such that we can unconditionally recommend its use, IMO we should just install it with Python. I'll go with the consensus though... Mark From ncoghlan at gmail.com Tue Jul 5 04:26:26 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 5 Jul 2011 12:26:26 +1000 Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference implementation In-Reply-To: <4E127326.7090202@gmail.com> References: <4E0BFA31.8020200@gmail.com> <4E0D2F42.4040802@gmail.com> <4E0EC5B7.9010607@gmail.com> <4E10139B.9050700@gmail.com> <4E1156B2.6050404@canterbury.ac.nz> <4E115D45.8000101@gmail.com> <4E126788.7020705@canterbury.ac.nz> <4E127326.7090202@gmail.com> Message-ID: On Tue, Jul 5, 2011 at 12:12 PM, Mark Hammond wrote: > If the launcher is such that we can unconditionally recommend its use, IMO > we should just install it with Python. ?I'll go with the consensus though... I've installed other WIndows apps that create multiple add/remove programs entries from a single installer. I believe people are suggesting a similar thing here (i.e. have the launcher installed automatically when installing python, but create a separate add/remove entry so uninstallation leaves it behind unless removal is explicitly requested) Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From stephen at xemacs.org Tue Jul 5 08:27:00 2011 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 05 Jul 2011 15:27:00 +0900 Subject: [Python-Dev] [Python-checkins] cpython: Remove mention of medical condition from the test suite. In-Reply-To: References: Message-ID: <87fwmldyyz.fsf@uwakimon.sk.tsukuba.ac.jp> Jim Jewett writes: > If you're going to get rid of the pun, you might as well change the > whole sentence... In fact, he should, since this is such a well-known pun that many people will consciously make the reverse substitution, and wonder WTF python-dev was thinking when they put the amended sentence in the test. Maybe somebody was *trying* to offend people who make such corrections habitually by demonstrating the kind of nonsense they occasionally produce. I've seen that target hit several times. Also, the log should say why this was done. -?Remove mention of medical condition from the test suite. + Change potentially offensive language in the test suite. One could also use the somewhat euphemistic "unprofessional language". That points to why such changes are justified despite an author's right to have her mode of expression respected -- the Python project aims at professionalism, and offensive language detracts from it. From skippy.hammond at gmail.com Tue Jul 5 08:59:42 2011 From: skippy.hammond at gmail.com (Mark Hammond) Date: Tue, 05 Jul 2011 16:59:42 +1000 Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference implementation In-Reply-To: <4E0EC5B7.9010607@gmail.com> References: <4E0BFA31.8020200@gmail.com> <4E0D2F42.4040802@gmail.com> <4E0EC5B7.9010607@gmail.com> Message-ID: <4E12B65E.5010504@gmail.com> On 2/07/2011 5:16 PM, I wrote: > Given [the C implementation] is now ahead of the Python > reference impl, I wonder if we should just drop all wording about that > reference impl and just treat the C impl as canonical? I'm looking to update the PEP based on this discussion - does anyone object to the above? Or to put it another way, does anyone believe dropping the Python reference implementation is to the detriment of the PEP? Thanks, Mark From vinay_sajip at yahoo.co.uk Tue Jul 5 10:01:58 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Tue, 5 Jul 2011 08:01:58 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?PEP_397_=28Python_launcher_for_Windows=29_?= =?utf-8?q?reference=09implementation?= References: <4E0BFA31.8020200@gmail.com> <4E0D2F42.4040802@gmail.com> <4E0EC5B7.9010607@gmail.com> <4E10139B.9050700@gmail.com> <4E1156B2.6050404@canterbury.ac.nz> <4E115D45.8000101@gmail.com> <4E126788.7020705@canterbury.ac.nz> <4E127326.7090202@gmail.com> Message-ID: Nick Coghlan gmail.com> writes: > I've installed other WIndows apps that create multiple add/remove > programs entries from a single installer. I believe people are > suggesting a similar thing here (i.e. have the launcher installed > automatically when installing python, but create a separate add/remove > entry so uninstallation leaves it behind unless removal is explicitly > requested) Were those other Windows apps packaged as .msi, or .exe? AFAICT, although you can embed an MSI inside another one, the practice of concurrent/nested installations is strongly discouraged by Microsoft - see http://goo.gl/FJx1S (Rule 20). Also, IIUC, each entry in Add/Remove programs would correspond to a specific MSI (since you can e.g. repair that specific entry, it would imply its own installer database). So you could package Python and the launcher as separate MSIs (this would make sense so that you could restore associations to the launcher just by repairing its installation), but since nested MSIs are a no-no, that means installing via a bootstrapping .exe. This is a bigger change to our Windows packaging than some people might be comfortable with ... Regards, Vinay Sajip From p.f.moore at gmail.com Tue Jul 5 10:18:56 2011 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 5 Jul 2011 09:18:56 +0100 Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference implementation In-Reply-To: References: <4E0BFA31.8020200@gmail.com> <4E0D2F42.4040802@gmail.com> <4E0EC5B7.9010607@gmail.com> <4E10139B.9050700@gmail.com> <4E1156B2.6050404@canterbury.ac.nz> <4E115D45.8000101@gmail.com> <4E126788.7020705@canterbury.ac.nz> <4E127326.7090202@gmail.com> Message-ID: On 5 July 2011 03:26, Nick Coghlan wrote: > On Tue, Jul 5, 2011 at 12:12 PM, Mark Hammond wrote: >> If the launcher is such that we can unconditionally recommend its use, IMO >> we should just install it with Python. ?I'll go with the consensus though... > > I've installed other WIndows apps that create multiple add/remove > programs entries from a single installer. I believe people are > suggesting a similar thing here (i.e. have the launcher installed > automatically when installing python, but create a separate add/remove > entry so uninstallation leaves it behind unless removal is explicitly > requested) That's certainly what I was meaning. I'm 100% in favour of Python 3.3 and later containing the installer as part of the core Python installer. One download, one install. (And two add/remove entries). I'd like to see a standalone installer for users of Python 2.7/3.2 and earlier. It's too useful a feature to not make it available for people who haven't installed 3.3 yet. And I'd prefer it if that standalone installer was hosted on python.org for visibility, rather than on PyPI. I'm not enough of an MSI expert to know if this can be implemented by having a standalone MSI, and then "embedding" it in the Python 3.3 MSI. That was what I'd thought of, but Vinay's later email suggests it might not be advisable: > AFAICT, although you can embed an MSI inside another one, the practice > of concurrent/nested installations is strongly discouraged by Microsoft - > see http://goo.gl/FJx1S (Rule 20). [...] > So you could package Python and the launcher as separate MSIs (this would > make sense so that you could restore associations to the launcher just by > repairing its installation), but since nested MSIs are a no-no, that means > installing via a bootstrapping .exe. This is a bigger change to our Windows > packaging than some people might be comfortable with ... Paul. From solipsis at pitrou.net Tue Jul 5 11:47:48 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 5 Jul 2011 11:47:48 +0200 Subject: [Python-Dev] [Python-checkins] cpython: Remove mention of medical condition from the test suite. References: <87fwmldyyz.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20110705114748.5c9fca6d@pitrou.net> On Tue, 05 Jul 2011 15:27:00 +0900 "Stephen J. Turnbull" wrote: > > One could also use the somewhat euphemistic "unprofessional language". > That points to why such changes are justified despite an author's > right to have her mode of expression respected -- the Python project > aims at professionalism, and offensive language detracts from it. I sincerely hope we don't start using the word "professional" to denote "careful" or "good quality". Most professional work is crap, and that's why we have open source. Regards Antoine. From solipsis at pitrou.net Tue Jul 5 11:49:03 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 5 Jul 2011 11:49:03 +0200 Subject: [Python-Dev] cpython (3.2): Issue #12497: Install test/data to prevent failures of the various codecmaps References: Message-ID: <20110705114903.4503d138@pitrou.net> On Tue, 05 Jul 2011 04:11:45 +0200 ned.deily wrote: > LIBSUBDIRS= tkinter tkinter/test tkinter/test/test_tkinter \ > tkinter/test/test_ttk site-packages test \ > - test/capath \ > + test/capath test/data \ > test/cjkencodings test/decimaltestdata test/xmltestdata test/subprocessdata \ > test/tracedmodules test/encoded_modules \ > concurrent concurrent/futures encodings \ Shouldn't we have something less dumb than a hardcoded list of directories? :) Regards Antoine. From victor.stinner at haypocalc.com Tue Jul 5 12:09:39 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Tue, 5 Jul 2011 12:09:39 +0200 Subject: [Python-Dev] cpython (3.2): Issue #12497: Install test/data to prevent failures of the various codecmaps In-Reply-To: <20110705114903.4503d138@pitrou.net> References: <20110705114903.4503d138@pitrou.net> Message-ID: <201107051209.39742.victor.stinner@haypocalc.com> Le mardi 05 juillet 2011 11:49:03, Antoine Pitrou a ?crit : > On Tue, 05 Jul 2011 04:11:45 +0200 > > ned.deily wrote: > > LIBSUBDIRS= tkinter tkinter/test tkinter/test/test_tkinter \ > > > > tkinter/test/test_ttk site-packages test \ > > > > - test/capath \ > > + test/capath test/data \ > > > > test/cjkencodings test/decimaltestdata test/xmltestdata > > test/subprocessdata \ test/tracedmodules test/encoded_modules \ > > concurrent concurrent/futures encodings \ > > Shouldn't we have something less dumb than a hardcoded list of > directories? :) Yes we should, especially because Makefile is not the only file that should be fixed: Tools/msi/msi.py I recently added Lib/test/cjkencodings directory, see issue #12057: R. David Murray: "Haypo, since you've created a new directory there are makefile (and PC build file, I think) updates that will need to be made. (This should be documented in the dev guide if it isn't already.)" Terry J. Reedy: "I presume and hope David meant the process, as I would have no idea how to add a directory. And David did not seem completely sure." Victor From vinay_sajip at yahoo.co.uk Tue Jul 5 13:24:48 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Tue, 5 Jul 2011 11:24:48 +0000 (UTC) Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference implementation References: <4E0BFA31.8020200@gmail.com> <4E0D2F42.4040802@gmail.com> <4E0EC5B7.9010607@gmail.com> <4E10139B.9050700@gmail.com> <4E1156B2.6050404@canterbury.ac.nz> <4E115D45.8000101@gmail.com> <4E126788.7020705@canterbury.ac.nz> <4E127326.7090202@gmail.com> Message-ID: Mark Hammond gmail.com> writes: > Or an MSI installer may be able to offer a "repair" feature without too > much pain. A few more observations to do with installation: 1. It's been mentioned that a standalone version should be available for use with earlier Python versions. This could be done with a merge module which can be used either with a standalone installer or the Python .msi. 2. For the standalone MSI, we will most likely need separate 32-bit and 64-bit MSIs, because the MSI system will look at the MSI type to determine whether registry stuff goes in the main registry or the Wow6432Node used for 32-bit applications. Regards, Vinay Sajip From murman at gmail.com Tue Jul 5 15:38:54 2011 From: murman at gmail.com (Michael Urman) Date: Tue, 5 Jul 2011 08:38:54 -0500 Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference implementation In-Reply-To: References: <4E0BFA31.8020200@gmail.com> <4E0D2F42.4040802@gmail.com> <4E0EC5B7.9010607@gmail.com> <4E10139B.9050700@gmail.com> <4E1156B2.6050404@canterbury.ac.nz> <4E115D45.8000101@gmail.com> <4E126788.7020705@canterbury.ac.nz> <4E127326.7090202@gmail.com> Message-ID: On Tue, Jul 5, 2011 at 03:01, Vinay Sajip wrote: > Were those other Windows apps packaged as .msi, or .exe? AFAICT, although you > can embed an MSI inside another one, the practice of concurrent/nested > installations is strongly discouraged by Microsoft - see http://goo.gl/FJx1S > (Rule 20). Right, you cannot sanely embed one .msi inside another .msi; the support for "nested" or "concurrent" installations referred to in that link is indeed something to avoid. > So you could package Python and the launcher as separate MSIs (this would make > sense so that you could restore associations to the launcher just by repairing > its installation), but since nested MSIs are a no-no, that means installing via > a bootstrapping .exe. This is a bigger change to our Windows packaging than some > people might be comfortable with ... You can certainly jump through all these hoops, but the pieces here are much more suited towards a component definition that can be shared among multiple products. If the component always installs to the same place, has the same GUID, and otherwise only changes by versions the exe, this is a completely safe correct use of one. Last I knew, msi.py allocates random GUIDs, so may or may not be suitable for generating this. If there is only rare need to update this exe, and msi.py has support for merge modules, that could be one approach. My recommendation for distributing this: each .msi which wants to include it should have a component that includes the following. Note that each .exe (py, pyw) and each architecture (x86, x64) need their own component with their own static GUID. * Defined unchanging GUID * Defined target location (perhaps SystemFolder) * msidbComponentAttributesSharedDllRefCount (cooperate with non-MSI installers), msidbComponentAttributesShared (keep highest version), and possibly msidbComponentAttributesPermanent flags set on the component * Versioned .exe (using a Windows version block) * File association information Then these components can be included in the python 3.3 installer, future releases, and even a standalone installer, and reference count correctly. Again, these can optionally be made available as merge modules for other consumers, but there's likely no need. -- Michael Urman From ncoghlan at gmail.com Tue Jul 5 15:57:24 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 5 Jul 2011 23:57:24 +1000 Subject: [Python-Dev] [Python-checkins] cpython: Remove mention of medical condition from the test suite. In-Reply-To: <87fwmldyyz.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87fwmldyyz.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Tue, Jul 5, 2011 at 4:27 PM, Stephen J. Turnbull wrote: > That points to why such changes are justified despite an author's > right to have her mode of expression respected -- the Python project > aims at professionalism, and offensive language detracts from it. Given that the contents of many test strings are quite arbitrary, I personally consider a bit of inoffensive humour or cultural references to be a fine thing to include rather than yet another instance of "foobar" (which itself has humorous *and* offensive origins). Heck, stripping just the Monty Python quotes from the test suite would probably take a while :) Avoiding offensive text has nothing to do with a desire to appear "professional" (at least for me) - it's about demonstrating common courtesy to the potentially wide variety of people that will read the Python source code in the future. (In the specific case, I thought quoting the venerable pun was fine, but I also don't have any real problem with modifying it) Cheers, Nick. P.S. 'twas a sad day when copyright concerns cost us the old test audio file :( -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From vinay_sajip at yahoo.co.uk Tue Jul 5 17:43:53 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Tue, 5 Jul 2011 15:43:53 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?PEP_397_=28Python_launcher_for_Windows=29_?= =?utf-8?q?reference=09implementation?= References: <4E0BFA31.8020200@gmail.com> <4E0D2F42.4040802@gmail.com> <4E0EC5B7.9010607@gmail.com> <4E10139B.9050700@gmail.com> <4E1156B2.6050404@canterbury.ac.nz> <4E115D45.8000101@gmail.com> <4E126788.7020705@canterbury.ac.nz> <4E127326.7090202@gmail.com> Message-ID: Michael Urman gmail.com> writes: > You can certainly jump through all these hoops, but the pieces here > are much more suited towards a component definition that can be shared > among multiple products. If the component always installs to the same > place, has the same GUID, and otherwise only changes by versions the > exe, this is a completely safe correct use of one. Last I knew, msi.py > allocates random GUIDs, so may or may not be suitable for generating > this. If there is only rare need to update this exe, and msi.py has > support for merge modules, that could be one approach. I'm currently experimenting with a merge module approach: launcher_module.msm -> launcher.msi, and x64 variants in separate .amd64.ms? files. > My recommendation for distributing this: each .msi which wants to > include it should have a component that includes the following. Note > that each .exe (py, pyw) and each architecture (x86, x64) need their > own component with their own static GUID. > * Defined unchanging GUID > * Defined target location (perhaps SystemFolder) I'm doing that. > * msidbComponentAttributesSharedDllRefCount (cooperate with non-MSI > installers), msidbComponentAttributesShared (keep highest version), > and possibly msidbComponentAttributesPermanent flags set on the > component Thanks, that's interesting information. I'll read up about these. I'm a Windows installer tyro :-) > * Versioned .exe (using a Windows version block) I'm doing that, too. > * File association information Currently I'm putting the file association information in the same component as the files, since the registry values cross reference those files. Regards, Vinay Sajip From stephen at xemacs.org Tue Jul 5 18:23:55 2011 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 06 Jul 2011 01:23:55 +0900 Subject: [Python-Dev] [Python-checkins] cpython: Remove mention of medical condition from the test suite. In-Reply-To: <20110705114748.5c9fca6d@pitrou.net> References: <87fwmldyyz.fsf@uwakimon.sk.tsukuba.ac.jp> <20110705114748.5c9fca6d@pitrou.net> Message-ID: <87aacselwk.fsf@uwakimon.sk.tsukuba.ac.jp> Antoine Pitrou writes: > I sincerely hope we don't start using the word "professional" to denote > "careful" or "good quality". No, by "professional" I mean "of a profession," which is a service that is provided by experts to laymen, and therefore demands adherence to certain standards since the clients are not able to judge the product for themselves, but in general must trust the vendor. The care and good quality proceed from the commitment of the professional. > Most professional work is crap, and that's why we have open source. Open source is not a substitute for professionalism. From solipsis at pitrou.net Tue Jul 5 18:42:29 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 5 Jul 2011 18:42:29 +0200 Subject: [Python-Dev] [Python-checkins] cpython: Remove mention of medical condition from the test suite. In-Reply-To: <87aacselwk.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87fwmldyyz.fsf@uwakimon.sk.tsukuba.ac.jp> <20110705114748.5c9fca6d@pitrou.net> <87aacselwk.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20110705184229.037a15ed@pitrou.net> On Wed, 06 Jul 2011 01:23:55 +0900 "Stephen J. Turnbull" wrote: > Antoine Pitrou writes: > > > I sincerely hope we don't start using the word "professional" to denote > > "careful" or "good quality". > > No, by "professional" I mean "of a profession," which is a service > that is provided by experts to laymen, and therefore demands adherence > to certain standards since the clients are not able to judge the > product for themselves, but in general must trust the vendor. The > care and good quality proceed from the commitment of the professional. I see. But the "experts" are not necessarily vendors (we aren't), and our users aren't "clients". Besides, we are not merely providing a service, we are building a community where everyone can take part in the shared work, thereby blurring the barrier between "experts" and "laymen". Regards Antoine. From stephen at xemacs.org Tue Jul 5 19:33:12 2011 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 06 Jul 2011 02:33:12 +0900 Subject: [Python-Dev] [Python-checkins] cpython: Remove mention of medical condition from the test suite. In-Reply-To: <20110705184229.037a15ed@pitrou.net> References: <87fwmldyyz.fsf@uwakimon.sk.tsukuba.ac.jp> <20110705114748.5c9fca6d@pitrou.net> <87aacselwk.fsf@uwakimon.sk.tsukuba.ac.jp> <20110705184229.037a15ed@pitrou.net> Message-ID: <878vsceip3.fsf@uwakimon.sk.tsukuba.ac.jp> Antoine Pitrou writes: > On Wed, 06 Jul 2011 01:23:55 +0900 > "Stephen J. Turnbull" wrote: > > Antoine Pitrou writes: > > > > > I sincerely hope we don't start using the word "professional" to denote > > > "careful" or "good quality". > > > > No, by "professional" I mean "of a profession," > I see. But the "experts" are not necessarily vendors (we aren't), and > our users aren't "clients". Besides, we are not merely providing a > service, we are building a community where everyone can take part in > the shared work, thereby blurring the barrier between "experts" and > "laymen". *sigh* And another good word bites the dust. OK, I will reserve the adjective "professional" for those who appreciate it. Nick's "common courtesy" will do for the current purpose. From drsalists at gmail.com Tue Jul 5 21:12:24 2011 From: drsalists at gmail.com (Dan Stromberg) Date: Tue, 5 Jul 2011 12:12:24 -0700 Subject: [Python-Dev] Compiling Python 3.2 on Cygwin fails In-Reply-To: References: <0a6de4fa-aa64-4485-a6da-72cf1bdad739@m9g2000yqe.googlegroups.com> <4e40b036-e232-45d5-be9d-b9019fe4b414@gh5g2000vbb.googlegroups.com> Message-ID: On Tue, Jul 5, 2011 at 7:25 AM, David Robinow wrote: > > Cygwin is not really a supported platform. ... > [Ultimately somebody with an > interest in cygwin will need to get active in python development. I've > been meaning to do this but life gets in the way.] > I was bitten by the lack of Cygwin support in 3.2 as well. IMO, python-dev needs continuous integration on a build farm that includes representative platforms. Most of the machines in the farm could be virtualboxes. I don't think the problem is so much that the right people haven't gotten involved, as that the currently-involved people don't know when they're breaking something for someone else due to the lack of continuous integration. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.curtin at gmail.com Tue Jul 5 21:18:22 2011 From: brian.curtin at gmail.com (Brian Curtin) Date: Tue, 5 Jul 2011 14:18:22 -0500 Subject: [Python-Dev] Compiling Python 3.2 on Cygwin fails In-Reply-To: References: <0a6de4fa-aa64-4485-a6da-72cf1bdad739@m9g2000yqe.googlegroups.com> <4e40b036-e232-45d5-be9d-b9019fe4b414@gh5g2000vbb.googlegroups.com> Message-ID: On Tue, Jul 5, 2011 at 14:12, Dan Stromberg wrote: > > On Tue, Jul 5, 2011 at 7:25 AM, David Robinow wrote: > >> >> Cygwin is not really a supported platform. > > ... > >> [Ultimately somebody with an >> interest in cygwin will need to get active in python development. I've >> been meaning to do this but life gets in the way.] >> > > I was bitten by the lack of Cygwin support in 3.2 as well. > > IMO, python-dev needs continuous integration on a build farm that > includes representative platforms. Most of the machines in the farm could > be virtualboxes. > > I don't think the problem is so much that the right people haven't gotten > involved, as that the currently-involved people don't know when they're > breaking something for someone else due to the lack of continuous > integration. > We've had that for some time now: http://www.python.org/dev/buildbot/ There are several issues on the bug tracker about cygwin build issues, but to my knowledge, none of them have included successful patches. -------------- next part -------------- An HTML attachment was scrubbed... URL: From drsalists at gmail.com Tue Jul 5 21:41:33 2011 From: drsalists at gmail.com (Dan Stromberg) Date: Tue, 5 Jul 2011 12:41:33 -0700 Subject: [Python-Dev] Compiling Python 3.2 on Cygwin fails In-Reply-To: References: <0a6de4fa-aa64-4485-a6da-72cf1bdad739@m9g2000yqe.googlegroups.com> <4e40b036-e232-45d5-be9d-b9019fe4b414@gh5g2000vbb.googlegroups.com> Message-ID: On Tue, Jul 5, 2011 at 12:18 PM, Brian Curtin wrote: > On Tue, Jul 5, 2011 at 14:12, Dan Stromberg wrote: > >> >> On Tue, Jul 5, 2011 at 7:25 AM, David Robinow wrote: >> >>> >>> Cygwin is not really a supported platform. >> >> ... >> >>> [Ultimately somebody with an >>> interest in cygwin will need to get active in python development. I've >>> been meaning to do this but life gets in the way.] >>> >> >> I was bitten by the lack of Cygwin support in 3.2 as well. >> >> IMO, python-dev needs continuous integration on a build farm that >> includes representative platforms. Most of the machines in the farm could >> be virtualboxes. >> >> I don't think the problem is so much that the right people haven't gotten >> involved, as that the currently-involved people don't know when they're >> breaking something for someone else due to the lack of continuous >> integration. >> > > We've had that for some time now: http://www.python.org/dev/buildbot/ > Good to know. Apologies for my incorrect assumption. Where do the e-mail notifications of build and/or test failures go? Shouldn't Cygwin be represented here? I don't see it in the list of builds. Some shops have a policy that nothing gets merged into trunk unless it's passing critical automated tests... Would that work here? There are several issues on the bug tracker about cygwin build issues, but > to my knowledge, none of them have included successful patches. > I think you'll find that most people using Cygwin would rather be working on some other OS, but are forced to use Windows for policy reasons. It's remains a rather significant need in many cases. How does the buildbot initiate builds? ssh? Happily Cygwin mostly allows sshd (as long as you don't need a CIFS share or something). Native Windows builds do appear to be represented. Is there any reason not to set up a buildbot for Cygwin on the same (virtual?) hardware? I'm inclined to believe that whoever rearranged the shared object stuff in CPython 3.x's build system would've been more careful if they'd had feedback about what it was doing to Cygwin. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.curtin at gmail.com Tue Jul 5 22:00:24 2011 From: brian.curtin at gmail.com (Brian Curtin) Date: Tue, 5 Jul 2011 15:00:24 -0500 Subject: [Python-Dev] Compiling Python 3.2 on Cygwin fails In-Reply-To: References: <0a6de4fa-aa64-4485-a6da-72cf1bdad739@m9g2000yqe.googlegroups.com> <4e40b036-e232-45d5-be9d-b9019fe4b414@gh5g2000vbb.googlegroups.com> Message-ID: On Tue, Jul 5, 2011 at 14:41, Dan Stromberg wrote: > > On Tue, Jul 5, 2011 at 12:18 PM, Brian Curtin wrote: > >> On Tue, Jul 5, 2011 at 14:12, Dan Stromberg wrote: >> >>> >>> On Tue, Jul 5, 2011 at 7:25 AM, David Robinow wrote: >>> >>>> >>>> Cygwin is not really a supported platform. >>> >>> ... >>> >>>> [Ultimately somebody with an >>>> interest in cygwin will need to get active in python development. I've >>>> been meaning to do this but life gets in the way.] >>>> >>> >>> I was bitten by the lack of Cygwin support in 3.2 as well. >>> >>> IMO, python-dev needs continuous integration on a build farm that >>> includes representative platforms. Most of the machines in the farm could >>> be virtualboxes. >>> >>> I don't think the problem is so much that the right people haven't gotten >>> involved, as that the currently-involved people don't know when they're >>> breaking something for someone else due to the lack of continuous >>> integration. >>> >> >> We've had that for some time now: http://www.python.org/dev/buildbot/ >> > > Good to know. Apologies for my incorrect assumption. Where do the e-mail > notifications of build and/or test failures go? > There might be an RSS feed or something, but I don't think there's any email notification. #python-dev on IRC receives live failure info. Other than that, you'd just have to look at one of the views of the fleet to see which build slaves are failing. > Shouldn't Cygwin be represented here? I don't see it in the list of > builds. > Probably, but it isn't represented because no one contributed a build slave for it. I know some of the other Windows build slave operators use Cygwin to some degree, but I'm not sure if anyone has looked into actually setting up a build slave for it. Some shops have a policy that nothing gets merged into trunk unless it's > passing critical automated tests... Would that work here? > We don't make much use of branching, but that would work if we did. If no one is actively contributing work on the Cygwin build then I don't see us holding up work in order to figure out any Cygwin-specific issues. There are several issues on the bug tracker about cygwin build issues, but >> to my knowledge, none of them have included successful patches. >> > > I think you'll find that most people using Cygwin would rather be working > on some other OS, but are forced to use Windows for policy reasons. It's > remains a rather significant need in many cases. > I don't disagree with that, but if there's no one contributing Cygwin patches then it will probably just die off and we'll have situations like the current one where it doesn't build. A great majority of the contributing developers are on UNIX-based systems with no access to Windows. A small handful, myself included, are Windows users, but I don't think any of us use Cygwin (I don't). Native Windows builds do appear to be represented. Is there any reason not > to set up a buildbot for Cygwin on the same (virtual?) hardware? > Besides the time and effort needed to set it up and occasionally look over it, no. We'd have to have a successfully compiling Cygwin build before we think about adding a build slave for it. I wouldn't be opposed to hosting this myself, but I need to steal some time and get my Windows 2008 build slave back to some form of a functional system - it has been up and down for a few months now. If someone else is interested, go ahead. -------------- next part -------------- An HTML attachment was scrubbed... URL: From drsalists at gmail.com Tue Jul 5 22:10:14 2011 From: drsalists at gmail.com (Dan Stromberg) Date: Tue, 5 Jul 2011 13:10:14 -0700 Subject: [Python-Dev] Compiling Python 3.2 on Cygwin fails In-Reply-To: References: <0a6de4fa-aa64-4485-a6da-72cf1bdad739@m9g2000yqe.googlegroups.com> <4e40b036-e232-45d5-be9d-b9019fe4b414@gh5g2000vbb.googlegroups.com> Message-ID: On Tue, Jul 5, 2011 at 1:00 PM, Brian Curtin wrote: > On Tue, Jul 5, 2011 at 14:41, Dan Stromberg wrote: > >> >> On Tue, Jul 5, 2011 at 12:18 PM, Brian Curtin wrote: >> >>> On Tue, Jul 5, 2011 at 14:12, Dan Stromberg wrote: >>> >>>> >>>> On Tue, Jul 5, 2011 at 7:25 AM, David Robinow wrote: >>>> >>>>> >>>>> Cygwin is not really a supported platform. >>>> >>>> ... >>>> >>>>> [Ultimately somebody with an >>>>> interest in cygwin will need to get active in python development. I've >>>>> been meaning to do this but life gets in the way.] >>>>> >>>> >>>> I was bitten by the lack of Cygwin support in 3.2 as well. >>>> >>>> IMO, python-dev needs continuous integration on a build farm that >>>> includes representative platforms. Most of the machines in the farm could >>>> be virtualboxes. >>>> >>>> I don't think the problem is so much that the right people haven't >>>> gotten involved, as that the currently-involved people don't know when >>>> they're breaking something for someone else due to the lack of continuous >>>> integration. >>>> >>> >>> We've had that for some time now: http://www.python.org/dev/buildbot/ >>> >> >> Good to know. Apologies for my incorrect assumption. Where do the e-mail >> notifications of build and/or test failures go? >> > > There might be an RSS feed or something, but I don't think there's any > email notification. #python-dev on IRC receives live failure info. Other > than that, you'd just have to look at one of the views of the fleet to see > which build slaves are failing. > Am I correct in assuming that "stable" buildbots are required to be reasonably functional before a release is tagged? > Shouldn't Cygwin be represented here? I don't see it in the list of >> builds. >> > > Probably, but it isn't represented because no one contributed a build slave > for it. I know some of the other Windows build slave operators use Cygwin to > some degree, but I'm not sure if anyone has looked into actually setting up > a build slave for it. > I see. > Some shops have a policy that nothing gets merged into trunk unless it's >> passing critical automated tests... Would that work here? >> > > We don't make much use of branching, but that would work if we did. If no > one is actively contributing work on the Cygwin build then I don't see us > holding up work in order to figure out any Cygwin-specific issues. > I might suggest that doing so (using branching, keeping trunk stable) might be of benefit, especially with a DVCS. > > There are several issues on the bug tracker about cygwin build issues, but >>> to my knowledge, none of them have included successful patches. >>> >> >> I think you'll find that most people using Cygwin would rather be working >> on some other OS, but are forced to use Windows for policy reasons. It's >> remains a rather significant need in many cases. >> > > I don't disagree with that, but if there's no one contributing Cygwin > patches then it will probably just die off and we'll have situations like > the current one where it doesn't build. A great majority of the contributing > developers are on UNIX-based systems with no access to Windows. A small > handful, myself included, are Windows users, but I don't think any of us use > Cygwin (I don't). > I see. Is there a python.org resource for setting up mailing lists - say, for a python-cygwin mailing list? > Native Windows builds do appear to be represented. Is there any reason not >> to set up a buildbot for Cygwin on the same (virtual?) hardware? >> > > Besides the time and effort needed to set it up and occasionally look over > it, no. We'd have to have a successfully compiling Cygwin build before we > think about adding a build slave for it. > That makes sense. > I wouldn't be opposed to hosting this myself, but I need to steal some time > and get my Windows 2008 build slave back to some form of a functional system > - it has been up and down for a few months now. If someone else is > interested, go ahead. > I might contribute some elbow grease if someone could get me VNC access to a suitable Windows server. BTW, is there someone available who is familiar with the meanings of the various shared object-related make symbols? I glanced at them and didn't find their naming astonishingly clear - seems like something to document, or... maybe it already is, and I just haven't seen where it is yet. -------------- next part -------------- An HTML attachment was scrubbed... URL: From victor.stinner at haypocalc.com Tue Jul 5 23:41:22 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Tue, 05 Jul 2011 23:41:22 +0200 Subject: [Python-Dev] [Python-checkins] cpython: Issue #12452: Plist and Dict are now deprecated In-Reply-To: <4E12FC8E.2070609@trueblade.com> References: <4E12FC8E.2070609@trueblade.com> Message-ID: <1309902082.24795.2.camel@marge> Le mardi 05 juillet 2011 ? 07:59 -0400, Eric Smith a ?crit : > On 7/4/2011 8:28 AM, victor.stinner wrote: > > http://hg.python.org/cpython/rev/4f14050a963f > > changeset: 71194:4f14050a963f > > user: Victor Stinner > > date: Mon Jul 04 14:28:45 2011 +0200 > > summary: > > Issue #12452: Plist and Dict are now deprecated > > > > Replace PendingDeprecationWarning warnings by DeprecationWarning. > > Shouldn't this be in MISC/News, be accompanied by documentation changes, > and have tests? Plist and Dict were never documented (in Doc/library/plistlib.rst). These classes have no test. You mean that I should add an entry to Misc/NEWS saying that these classe are now deprecated? Should I also mention the deprecation to the "What's new in Python 3.3?" document? Victor From eric at trueblade.com Tue Jul 5 23:52:21 2011 From: eric at trueblade.com (Eric Smith) Date: Tue, 05 Jul 2011 17:52:21 -0400 Subject: [Python-Dev] [Python-checkins] cpython: Issue #12452: Plist and Dict are now deprecated In-Reply-To: <1309902082.24795.2.camel@marge> References: <4E12FC8E.2070609@trueblade.com> <1309902082.24795.2.camel@marge> Message-ID: <4E138795.4090707@trueblade.com> > Plist and Dict were never documented (in Doc/library/plistlib.rst). > These classes have no test. Ouch! > You mean that I should add an entry to Misc/NEWS saying that these > classe are now deprecated? Should I also mention the deprecation to the > "What's new in Python 3.3?" document? Yes. I think this should make it into a "What's new" document, usually via Misc/NEWS. Thanks. Eric. From guido at python.org Wed Jul 6 00:29:34 2011 From: guido at python.org (Guido van Rossum) Date: Tue, 5 Jul 2011 15:29:34 -0700 Subject: [Python-Dev] cpython: Issue #12469: replace assertions by explicit if+raise In-Reply-To: <1309805706.30323.4.camel@marge> References: <20110704182344.7bad6b62@pitrou.net> <1309805706.30323.4.camel@marge> Message-ID: Exception is for catching, not raising. On Jul 4, 2011 11:57 AM, "Victor Stinner" wrote: > Le lundi 04 juillet 2011 ? 18:23 +0200, Antoine Pitrou a ?crit : >> On Mon, 04 Jul 2011 18:06:53 +0200 >> victor.stinner wrote: >> > http://hg.python.org/cpython/rev/7eef821ab20d >> > changeset: 71197:7eef821ab20d >> > user: Victor Stinner >> > date: Mon Jul 04 18:06:35 2011 +0200 >> > summary: >> > Issue #12469: replace assertions by explicit if+raise > > >> Instead of generic Exception, it would be better to use AssertionError. > > and > >> or in many cases given this was in unittests... use the self.assertFoo >> methods and avoid assert and if statements all together. > > The code is running in a subprocess (python -c ...), not in an > unittest.TestCase, so I cannot use self.assertFoo and it doesn't really > matter if the exception is an Exception or an AssertionError. > >> What is the reason for this change anyway -- as far as I can >> see this code is never run with -O. > > I'm not sure that the code will never be running using -O, so I prefer > to use an explicit if+raise. I don't like the assert statement because > it doesn't provide any information about the failure (content of the > variables) by default. > > Victor > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Wed Jul 6 00:50:09 2011 From: guido at python.org (Guido van Rossum) Date: Tue, 5 Jul 2011 15:50:09 -0700 Subject: [Python-Dev] [Python-checkins] cpython: Remove mention of medical condition from the test suite. In-Reply-To: References: Message-ID: It's not a bug and shouldn't be "fixed". We leave lots of minor infractions in the code because the code churn of fixing them all would be too distracting. On Jul 3, 2011 10:22 AM, "georg.brandl" wrote: > http://hg.python.org/cpython/rev/76452b892838 > changeset: 71146:76452b892838 > parent: 71144:ce52310f61a0 > user: Georg Brandl > date: Sun Jul 03 19:22:42 2011 +0200 > summary: > Remove mention of medical condition from the test suite. > > files: > Lib/test/test_csv.py | 8 ++++---- > 1 files changed, 4 insertions(+), 4 deletions(-) > > > diff --git a/Lib/test/test_csv.py b/Lib/test/test_csv.py > --- a/Lib/test/test_csv.py > +++ b/Lib/test/test_csv.py > @@ -459,20 +459,20 @@ > '5', '6']]) > > def test_quoted_quote(self): > - self.readerAssertEqual('1,2,3,"""I see,"" said the blind man","as he picked up his hammer and saw"', > + self.readerAssertEqual('1,2,3,"""I see,"" said the happy man","as he picked up his hammer and saw"', > [['1', '2', '3', > - '"I see," said the blind man', > + '"I see," said the happy man', > 'as he picked up his hammer and saw']]) > > def test_quoted_nl(self): > input = '''\ > 1,2,3,"""I see,"" > -said the blind man","as he picked up his > +said the happy man","as he picked up his > hammer and saw" > 9,8,7,6''' > self.readerAssertEqual(input, > [['1', '2', '3', > - '"I see,"\nsaid the blind man', > + '"I see,"\nsaid the happy man', > 'as he picked up his\nhammer and saw'], > ['9','8','7','6']]) > > > -- > Repository URL: http://hg.python.org/cpython -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Wed Jul 6 02:35:44 2011 From: guido at python.org (Guido van Rossum) Date: Tue, 5 Jul 2011 17:35:44 -0700 Subject: [Python-Dev] [Python-checkins] cpython: Remove mention of medical condition from the test suite. In-Reply-To: References: Message-ID: To clarify, now that I have access to an actual keyboard instead of just a cellphone: I think it should be rolled back, since the proper process for controversial changes was not followed. Our process (part of our culture, if you will) for anything controversial is to discuss the change first, then, if deemed necessary, fix the code. And from the size of this thread it clearly is controversial. Georg, can you please revert this change? Note that another part of our process/culture is that we try not to engage in battling commits, i.e. generally we don't unilaterally roll back a change just to make the point that it is incorrect; we ask the original committer to roll it back. As to the controversy itself, if you want to make blind people fell more at home in the Python community surely there are more useful things to do than to remove puns involving blindness; e.g. improve accessibility of python.org or some part of it. Or maybe find some blind programmers and ask them what would help them. --Guido On Tue, Jul 5, 2011 at 3:50 PM, Guido van Rossum wrote: > It's not a bug and shouldn't be "fixed". We leave lots of minor infractions > in the code because the code churn of fixing them all would be too > distracting. > > On Jul 3, 2011 10:22 AM, "georg.brandl" wrote: >> http://hg.python.org/cpython/rev/76452b892838 >> changeset: 71146:76452b892838 >> parent: 71144:ce52310f61a0 >> user: Georg Brandl >> date: Sun Jul 03 19:22:42 2011 +0200 >> summary: >> Remove mention of medical condition from the test suite. >> >> files: >> Lib/test/test_csv.py | 8 ++++---- >> 1 files changed, 4 insertions(+), 4 deletions(-) >> >> >> diff --git a/Lib/test/test_csv.py b/Lib/test/test_csv.py >> --- a/Lib/test/test_csv.py >> +++ b/Lib/test/test_csv.py >> @@ -459,20 +459,20 @@ >> '5', '6']]) >> >> def test_quoted_quote(self): >> - self.readerAssertEqual('1,2,3,"""I see,"" said the blind man","as he >> picked up his hammer and saw"', >> + self.readerAssertEqual('1,2,3,"""I see,"" said the happy man","as he >> picked up his hammer and saw"', >> [['1', '2', '3', >> - '"I see," said the blind man', >> + '"I see," said the happy man', >> 'as he picked up his hammer and saw']]) >> >> def test_quoted_nl(self): >> input = '''\ >> 1,2,3,"""I see,"" >> -said the blind man","as he picked up his >> +said the happy man","as he picked up his >> hammer and saw" >> 9,8,7,6''' >> self.readerAssertEqual(input, >> [['1', '2', '3', >> - '"I see,"\nsaid the blind man', >> + '"I see,"\nsaid the happy man', >> 'as he picked up his\nhammer and saw'], >> ['9','8','7','6']]) >> >> >> -- >> Repository URL: http://hg.python.org/cpython > -- --Guido van Rossum (python.org/~guido) From brian.curtin at gmail.com Wed Jul 6 02:38:29 2011 From: brian.curtin at gmail.com (Brian Curtin) Date: Tue, 5 Jul 2011 19:38:29 -0500 Subject: [Python-Dev] Compiling Python 3.2 on Cygwin fails In-Reply-To: References: <0a6de4fa-aa64-4485-a6da-72cf1bdad739@m9g2000yqe.googlegroups.com> <4e40b036-e232-45d5-be9d-b9019fe4b414@gh5g2000vbb.googlegroups.com> Message-ID: On Tue, Jul 5, 2011 at 15:10, Dan Stromberg wrote: > > On Tue, Jul 5, 2011 at 1:00 PM, Brian Curtin wrote: > >> On Tue, Jul 5, 2011 at 14:41, Dan Stromberg wrote: >> >>> >>> On Tue, Jul 5, 2011 at 12:18 PM, Brian Curtin wrote: >>> >>>> On Tue, Jul 5, 2011 at 14:12, Dan Stromberg wrote: >>>> >>>>> >>>>> On Tue, Jul 5, 2011 at 7:25 AM, David Robinow wrote: >>>>> >>>>>> >>>>>> Cygwin is not really a supported platform. >>>>> >>>>> ... >>>>> >>>>>> [Ultimately somebody with an >>>>>> interest in cygwin will need to get active in python development. I've >>>>>> been meaning to do this but life gets in the way.] >>>>>> >>>>> >>>>> I was bitten by the lack of Cygwin support in 3.2 as well. >>>>> >>>>> IMO, python-dev needs continuous integration on a build farm that >>>>> includes representative platforms. Most of the machines in the farm could >>>>> be virtualboxes. >>>>> >>>>> I don't think the problem is so much that the right people haven't >>>>> gotten involved, as that the currently-involved people don't know when >>>>> they're breaking something for someone else due to the lack of continuous >>>>> integration. >>>>> >>>> >>>> We've had that for some time now: http://www.python.org/dev/buildbot/ >>>> >>> >>> Good to know. Apologies for my incorrect assumption. Where do the >>> e-mail notifications of build and/or test failures go? >>> >> >> There might be an RSS feed or something, but I don't think there's any >> email notification. #python-dev on IRC receives live failure info. Other >> than that, you'd just have to look at one of the views of the fleet to see >> which build slaves are failing. >> > > Am I correct in assuming that "stable" buildbots are required to be > reasonably functional before a release is tagged? > Yep - all green is the goal. > > >> Shouldn't Cygwin be represented here? I don't see it in the list of >>> builds. >>> >> >> Probably, but it isn't represented because no one contributed a build >> slave for it. I know some of the other Windows build slave operators use >> Cygwin to some degree, but I'm not sure if anyone has looked into actually >> setting up a build slave for it. >> > > I see. > > >> Some shops have a policy that nothing gets merged into trunk unless it's >>> passing critical automated tests... Would that work here? >>> >> >> We don't make much use of branching, but that would work if we did. If no >> one is actively contributing work on the Cygwin build then I don't see us >> holding up work in order to figure out any Cygwin-specific issues. >> > > I might suggest that doing so (using branching, keeping trunk stable) might > be of benefit, especially with a DVCS. > > >> >> There are several issues on the bug tracker about cygwin build issues, but >>>> to my knowledge, none of them have included successful patches. >>>> >>> >>> I think you'll find that most people using Cygwin would rather be working >>> on some other OS, but are forced to use Windows for policy reasons. It's >>> remains a rather significant need in many cases. >>> >> >> I don't disagree with that, but if there's no one contributing Cygwin >> patches then it will probably just die off and we'll have situations like >> the current one where it doesn't build. A great majority of the contributing >> developers are on UNIX-based systems with no access to Windows. A small >> handful, myself included, are Windows users, but I don't think any of us use >> Cygwin (I don't). >> > > I see. > > Is there a python.org resource for setting up mailing lists - say, for a > python-cygwin mailing list? > You might want to suggest something like cygwin-sig as a mailing list. Check out http://www.python.org/community/sigs/guidelines/ for more info. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Wed Jul 6 06:33:35 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 6 Jul 2011 14:33:35 +1000 Subject: [Python-Dev] Compiling Python 3.2 on Cygwin fails In-Reply-To: References: <0a6de4fa-aa64-4485-a6da-72cf1bdad739@m9g2000yqe.googlegroups.com> <4e40b036-e232-45d5-be9d-b9019fe4b414@gh5g2000vbb.googlegroups.com> Message-ID: On Wed, Jul 6, 2011 at 10:38 AM, Brian Curtin wrote: > On Tue, Jul 5, 2011 at 15:10, Dan Stromberg wrote: >> Am I correct in assuming that "stable" buildbots are required to be >> reasonably functional before a release is tagged? > > Yep - all green is the goal. Indeed, that's the main difference between the stable and unstable buildbots. stable = this should work. If it doesn't, somebody broke something and the relevant branch should be fixed unstable = someone cared enough to set up this buildbot, but due to problems with either the platform in general or the specific machine it spends a lot of its time red for reasons that aren't the fault of recent changes to Python A Cygwin buildbot would start in the latter category then potentially migrate to stable if it proved itself with green results over a period of time. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Wed Jul 6 08:49:32 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 6 Jul 2011 16:49:32 +1000 Subject: [Python-Dev] [Python-checkins] cpython: Remove mention of medical condition from the test suite. In-Reply-To: References: <87fwmldyyz.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Wed, Jul 6, 2011 at 6:17 AM, Georg Brandl wrote: > For this test string, a) I'm not a native speaker and therefore don't know of > any special treatment this pun deserves It's not an especially *good* joke, just a very old one that plays on double meanings of both "see" (as in sight and understanding) and "saw" (as in sight and a tool). Still, I'd put it in the same category as the Monty Python quotes we have scattered around the test suite - if people came across them and didn't realise they were quotes they might be puzzled, but attempting to retain that Pythonesque sense of humour is itself part of what makes the Python community what it is. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From stefan_ml at behnel.de Wed Jul 6 08:51:21 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 06 Jul 2011 08:51:21 +0200 Subject: [Python-Dev] cpython: Remove mention of medical condition from the test suite. In-Reply-To: References: Message-ID: Georg Brandl, 06.07.2011 07:35: > Well, it was stated that even non-joking use of such words can offend > (the example given was "your argument is blind to (these facts)"). > I consider use in jokes to be more serious, since it's careless use. > Sorry if I overreacted here. There's a common saying that a disabled person can't be considered "normal" unless he/she can also be called an asshole from time to time. Personally, I do not consider the pun in question harmful, simply because it's so clearly just a play on words that doesn't make much sense, apart from having a double meaning. In general, however, I think it's important to make jokes with and about disabled people. That keeps them inside of our society, just like anyone else. Treating them as "untouchable" means to single them out. On that note, if anyone of you ever makes it to the beautiful village of Amersfoort (NL), don't miss the "Downey's" caf?. Nice and friendly place to be. Stefan From g.brandl at gmx.net Wed Jul 6 09:08:17 2011 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 06 Jul 2011 09:08:17 +0200 Subject: [Python-Dev] cpython: Remove mention of medical condition from the test suite. In-Reply-To: References: Message-ID: Am 06.07.2011 08:51, schrieb Stefan Behnel: > Georg Brandl, 06.07.2011 07:35: >> Well, it was stated that even non-joking use of such words can offend >> (the example given was "your argument is blind to (these facts)"). >> I consider use in jokes to be more serious, since it's careless use. >> Sorry if I overreacted here. > > There's a common saying that a disabled person can't be considered "normal" > unless he/she can also be called an asshole from time to time. > > Personally, I do not consider the pun in question harmful, simply because > it's so clearly just a play on words that doesn't make much sense, apart > from having a double meaning. > > In general, however, I think it's important to make jokes with and about > disabled people. That keeps them inside of our society, just like anyone > else. Treating them as "untouchable" means to single them out. I don't think we need to rehash this here (not meaning to pick on you, Stefan); those who are interested can sign up to the diversity list and read/post to the original thread here: http://mail.python.org/mailman/private/diversity/2011-July/002509.html Georg From stefan_ml at behnel.de Wed Jul 6 09:13:58 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 06 Jul 2011 09:13:58 +0200 Subject: [Python-Dev] cpython: Remove mention of medical condition from the test suite. In-Reply-To: References: Message-ID: Georg Brandl, 06.07.2011 09:08: > Am 06.07.2011 08:51, schrieb Stefan Behnel: >> Georg Brandl, 06.07.2011 07:35: >>> Well, it was stated that even non-joking use of such words can offend >>> (the example given was "your argument is blind to (these facts)"). >>> I consider use in jokes to be more serious, since it's careless use. >>> Sorry if I overreacted here. >> >> There's a common saying that a disabled person can't be considered "normal" >> unless he/she can also be called an asshole from time to time. >> >> Personally, I do not consider the pun in question harmful, simply because >> it's so clearly just a play on words that doesn't make much sense, apart >> from having a double meaning. >> >> In general, however, I think it's important to make jokes with and about >> disabled people. That keeps them inside of our society, just like anyone >> else. Treating them as "untouchable" means to single them out. > > I don't think we need to rehash this here (not meaning to pick on you, Stefan); Agreed. > those who are interested can sign up to the diversity list and read/post to > the original thread here: > > http://mail.python.org/mailman/private/diversity/2011-July/002509.html Speaking of "singling out", I've been wondering more than once why that list has a private archive ... Stefan From solipsis at pitrou.net Wed Jul 6 14:12:00 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 6 Jul 2011 14:12:00 +0200 Subject: [Python-Dev] cpython: Remove mention of medical condition from the test suite. References: Message-ID: <20110706141200.4e2309e9@pitrou.net> On Wed, 06 Jul 2011 09:13:58 +0200 Stefan Behnel wrote: > > > those who are interested can sign up to the diversity list and read/post to > > the original thread here: > > > > http://mail.python.org/mailman/private/diversity/2011-July/002509.html > > Speaking of "singling out", I've been wondering more than once why that > list has a private archive ... I suspect the explanation is itself given somewhere in a message inside the private archive ;) Regards Antoine. From vinay_sajip at yahoo.co.uk Wed Jul 6 20:31:55 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Wed, 6 Jul 2011 18:31:55 +0000 (UTC) Subject: [Python-Dev] Python Launcher for Windows (PEP 397) needs testing! Message-ID: The C implementation of the PEP 397-compatible Python Launcher for Windows has come along nicely in the last few days, and now reached a point where it would benefit from some testing by interested python-dev members. Points of note: 1. As well as source available on https://bitbucket.org/vinay.sajip/pylauncher there are built 32- and 64-bit msi files at https://bitbucket.org/vinay.sajip/pylauncher/downloads Please remember that this is beta software. While it appears stable, I've tested in virtual machines (WinXP 32-bit, Win7 32-bit and 64-bit) which I can readily restore from backup. There are also msm files which could be used to e.g. integrate with the Python msi files, if approved, at some later date. 2. On installation, any existing associations are saved, and restored when the launcher is uninstalled (this is done automatically by Windows Installer). 3. On uninstallation, if there are Pythons installed and no associations, a dialog pops up listing all the installed Pythons and offering the user the chance to associate one of the Pythons with the Python extensions. The user can choose to associate or not, but once they choose an association, then that association will always be there unless the launcher is reinstalled (in which case it takes over the association while it's still installed, and restores the previous one when it's uninstalled). 4. I've tried to cover all of the points in the PEP. There is a test suite - while this appears to be small (7 tests) the individual shebangs are all tested, as are the customisable commands etc. However, I'm sure some of you will break it ;-) 5. I used WiX to build the msm/msi files, but that's only because of increased familiarity over msilib. The build procedure could switch over to msilib at some later date. All the other code is just plain C and Win32 APIs (gosh - takes me back! Window procedures, anyone?). The code builds with Visual Studio and also Visual Studio Express (C++ edition). Regards, Vinay Sajip From wolfson at gmail.com Thu Jul 7 01:02:38 2011 From: wolfson at gmail.com (Ben Wolfson) Date: Wed, 6 Jul 2011 16:02:38 -0700 Subject: [Python-Dev] PEP 3101 implementation vs. documentation In-Reply-To: References: Message-ID: FWIW, new patches have been attached to the bug report (http://bugs.python.org/issue12014), one of which is intended to bring behavior in line with the documentation, and the other of which is intended to implement Greg Ewing's proposal to allow only identifiers (or integers) in the arg_name, attribute_name, and element_index sections. On Fri, Jun 10, 2011 at 2:15 PM, Ben Wolfson wrote: > Hello, > > I'm writing because discussion in a bug report I submitted > () has suggested that, insofar as > at least part of the issue revolves around the interpretation of PEP > 3101, that aspect belonged on python-dev. In particular, I was told > that the PEP, not the documentation, is authoritative. Since I'm the > one who thinks something is wrong, it seems appropriate for me to be > the one to bring it up. > > Basically, the issue is that the current behavior of str.format is at > variance at least with the documentation > , is almost > certainly at variance with PEP3101 in one respect, and is in my > opinion at variance with PEP3101 in another respect as well, regarding > what characters can be present in what the grammar given in the > documentation calls an element_index, that is, the bit between the > square brackets in "{0.attr[idx]}". > > Both discovering the current behavior and interpreting the > documentation are pretty straightforward; interpreting what the PEP > actually calls for is more vexed. I'll do the first two things first. > TOC for the remainder: > > 1. What does the current implementation do? > 2. What does the documentation say? > 3. What does the PEP say? [this part is long, but the PEP is not > clear, and I wanted to be thorough] > 4. Who cares? > > 1. What does the current implementation do? > > Suppose you have this dictionary: > > d = {"@": 0, > ? ? "!": 1, > ? ? ":": 2, > ? ? "^": 3, > ? ? "}": 4, > ? ? "{": {"}": 5}, > ? ?} > > Then the following expressions have the following results: > > (a) "{0[@]}".format(d) ? ?--> '0' > (b) "{0[!]}".format(d) ? ?--> ValueError: Missing ']' in format string > (c) "{0[:]}".format(d) ? ?--> ValueError: Missing ']' in format string > (d) "{0[^]}".format(d) ? ?--> '3' > (e) "{0[}]}".format(d) ? ?--> ValueError: Missing ']' in format string > (f) "{0[{]}".format(d) ? ?--> ValueError: unmatched '{' in format > (g) "{0[{][}]}".format(d) --> '5' > > Given (e) and (f), I think (g) should be a little surprising, though > you can probably guess what's going on and it's not hard to see why it > happens if you look at the source: (e) and (f) fail because > MarkupIterator_next (in Objects/stringlib/string_format.h) scans > through the string looking for curly braces, because it treats them as > semantically significant no matter what context they occur in. So, > according to MarkupIterator_next, the *first* right curly brace in (e) > indicates the end of the replacement field, giving "{0[}". In (f), the > second left curly brace indicates (to MarkupIterator_next) the start > of a *new* replacement field, and since there's only one right curly > brace, it complains. In (g), MarkupIterator_next treats the second > left curly brace as starting a new replacement field and the first > right curly brace as closing it. However, actually, those braces don't > define new replacement fields, as indicated by the fact that the whole > expression treats the element_index fields as just plain old strings. > (So the current implementation is somewhat schizophrenic, acting at > one point as if the braces have to be balanced because '{[]}' is a > replacement field and at another point treating the braces as > insignificant.) > > The explanation for (b) and (c) is that parse_field (same source file) > treats ':' and '!' ?as indicating the end of the field_name section of > the replacement field, regardless of whether those characters occur > within square brackets or not. > > So, that's what the current implementation does, in those cases. > > 2. What does the documentation say? > > The documentation gives a grammar for replacement fields: > > """ > replacement_field ::= ?"{" [field_name] ["!" conversion] [":" format_spec] "}" > field_name ? ? ? ?::= ?arg_name ("." attribute_name | "[" element_index "]")* > arg_name ? ? ? ? ?::= ?[identifier | integer] > attribute_name ? ?::= ?identifier > element_index ? ? ::= ?integer | index_string > index_string ? ? ?::= ? + > conversion ? ? ? ?::= ?"r" | "s" > format_spec ? ? ? ::= ? > """ > > Given this definition of index_string, all of (a)--(g) should be > legal, and the results should be the strings '0', '1', '2', '3', > "{'}': 5}", and '5'. There is no need to exclude ':', '!', '}', or '{' > from the index_string field; allowing them creates no ambiguity, > because the field is delimited by square brackets. > > Tangent: the definition of attribute_name here also does not describe > the current behavior ("{0. ?;}".format(x) works fine and will call > getattr(x, " ;")) and the PEP does not require the attribute_name to > be an identifier; in fact it explicitly states that the attribute_name > doesn't need to be a valid Python identifier. attribute_name should > read (to reflect something like actual behavior, anyway) " character except '[', '.', ':', '!', '{', or '}'> +". The same goes > for arg_name (with the integer alternation). Observe: > >>>> x = lambda: None >>>> setattr(x, ']]', 3) >>>> "{].]]}".format(**{"]":x}) ? ? # (h) > '3' > > One can also presently do this (hence "something like actual behavior"): >>>> setattr(x, 'f}', 4) >>>> "{a{s.f}}".format(**{"a{s":x}) > '4' > But not this: >>>> "{a{s.func_name}".format(**{"a{s":x}) > as it raises a ValueError, for the same reason as explains (g) above. > > > 3. What does the PEP say? > > Well... It's actually hard to tell! > > Summary: The PEP does not contain a grammar for replacement fields, > and is surprisingly nonspecific about what can appear where, at least > when talking about the part of the replacement field before the format > specifier. The most reasonable interpretation of the parts of the PEP > that seem to be relevant favors the documentation, rather than the > implementation. > > This can be separated into two sub-questions. > > A. What does the PEP say about ':' and '!'? > > It says two things that pertain to element_index fields. > > The first is this: > """ > ? ? ? ? ? ? ? ? ? The rules for parsing an item key are very simple. > ? ?If it starts with a digit, then it is treated as a number, otherwise > ? ?it is used as a string. > > ? ?Because keys are not quote-delimited, it is not possible to > ? ?specify arbitrary dictionary keys (e.g., the strings "10" or > ? ?":-]") from within a format string. > """ > > So it notes that some things can't be used: > > ?- Because anything composed entirely of digits is treated as a > number, you can't get a string composed entirely of digits. Clear > enough. > > ?- What's the explanation for the second example, though? Well, you > can't have a right square bracket in the index_string, so that would > already mean that you can't do this: "{0[:-]]}".format(...) regardless > of the whether colons are legal or not. So, although the PEP gives an > example of a string that can't in the element_index part of a > replacement field, and that string contains a colon, that string would > have been disallowed for other reasons anyway. > > The second is this: > > """ > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?The str.format() function will have > ? ?a minimalist parser which only attempts to figure out when it is > ? ?"done" with an identifier (by finding a '.' or a ']', or '}', > ? ?etc.). > """ > > This requires some interpretation. For one thing, the contents of an > element_index aren't identifiers. For another, it's not true that > you're done with an identifier (or whatever) whenever you see *any* of > those; it depends on context. When parsing this: "{0[a.b]}" the parser > should not stop at the '.'; it should keep going until it reaches the > ']', and that will give the element_index. And when parsing this: > "{0.f]oo[bar].baz}", it's done with the identifier "foo" when it > reaches the '[', not when it reaches the second '.', and not when it > reaches the ']', either (recall (h)). The "minimalist parser" is, I > take it, one that works like this: when parsing an arg_name you're > done when you reach a '[', a ':', a '!', a '.', '{', or a '}'; the > same rules apply when parsing a attribute_name; when parsing an > element_index you're done when you see a ']'. > > Now as regards the curly braces there are some other parts of the PEP > that perhaps should be taken into account (see below), but I can find > no justification for excluding ':' and '!' from the element_index > field; the bit quoted above about having a minimalist parser isn't a > good justification for that, and it's the only part of the entire PEP > that comes close to addressing the question. > > B. What does it say about '}' and '{'? > > It still doesn't say much explicitly. There's the quotation I just > gave, and then these passages: > > """ > ? ?Brace characters ('curly braces') are used to indicate a > ? ?replacement field within the string: > > ? ?[...] > > ? ?Braces can be escaped by doubling: > """ > > Taken by itself, this would suggest that wherever there's an unescaped > brace, there's a replacement field. That would mean that the current > implementation's behavior is correct in (e) and (f) but incorrect in > (g). However, I think this is a bad interpretation; unescaped braces > can indicate the presence of a replacement field without that meaning > that *within* a replacement field braces have that meaning, no matter > where within the replacement field they occur. > > Later in the PEP, talking about this example: > > ? ? ? ?"{0:{1}}".format(a, b) > > We have this: > > """ > ? ?These 'internal' replacement fields can only occur in the format > ? ?specifier part of the replacement field. ?Internal replacement fields > ? ?cannot themselves have format specifiers. ?This implies also that > ? ?replacement fields cannot be nested to arbitrary levels. > > ? ?Note that the doubled '}' at the end, which would normally be > ? ?escaped, is not escaped in this case. ?The reason is because > ? ?the '{{' and '}}' syntax for escapes is only applied when used > ? ?*outside* of a format field. ?Within a format field, the brace > ? ?characters always have their normal meaning. > """ > > The claim "within a format field, the brace characters always have > their normal meaning" might be taken to mean that within a > *replacement* field, brace characters always indicate the start (or > end) of a replacement field. But the PEP at this point is clearly > talking about the formatting section of a replacement field---the part > that follows the ':', if present. ("Format field" is nowhere defined > in the PEP, but it seems reasonable to take it to mean "the format > specifier of a replacement field".) However, it seems most reasonable > to me to take "normal meaning" to mean "just a brace character". > > Note that the present implementation only kinda sorta conforms to the > PEP in this respect: > > >>>> import datetime >>>> format(datetime.datetime.now(), "{{%Y") > '{{2011' >>>> "{0:{{%{1}}".format(datetime.datetime.now(), 'Y') # (i) > Traceback (most recent call last): > ?File "", line 1, in > ValueError: unmatched '{' in format >>>> "{0:{{%{1}}}}".format(datetime.datetime.now(), 'Y') # (j) > '{2011}' > > Here the brace characters in (i) and (j) are treated, again in > MarkupIterator_next, as indicating the start of a replacement field. > In (i), this leads the function to throw an exception; since they're > balanced in (j), processing proceeds further, and the doubled braces > aren't treated as indicating the start or end of a replacement > field---because they're escaped. ?Given that the format spec part of a > replacement field can contain further replacement fields, this is, I > think, correct behavior, but it's not easy to see how it comports with > the PEP, whose language is not very exact. > > The call to the built-in format() bypasses the mechanism that leads to > these results. > > The PEP is very, very nonspecific about the parts of the replacement > field that precede the format specifier. I don't know what kind of > discussion surrounded the drawing up of the grammar that appears in > the documentation, but I think that it, and not the implementation, > should be followed. > > The implementation only works the way it does because of parsing > shortcuts: it raises ValueErrors for (b) and (c) because it > generalizes something true of the attribute_name field (encountering a > ':' or '!' means one has moved on to the format_spec or conversion > part of the replacement field) to the element_index field. And it > raises an error for (e) and (f), but lets (g) through, for the reasons > already mentioned. It is, in that respect, inconsistent; it treats the > curly brace as having one semantic significance at one point and an > entirely different significance at another point, so that it does the > right thing in the case of (g) entirely by accident. There is, I > think, no way to construe the PEP so that it is reasonable to do what > the present implementation does in all three cases (if "{" indicates > the start of a replacement field in (f), it should do so in (g) as > well); I think it's > actually pretty difficult to construe the PEP in any way that makes > what it does in the case of (e) and (f) correct. > > 4. Who cares? > > Well, I do. (Obviously.) I even have a use case: I want to be able to > put arbitrary (or as close to arbitrary as possible) strings in the > element_index field, because I've got some objects that (should!) > enable me to do this: > > p.say("I'm warning you, {e.red.underline[don't touch that!]}") > > and have this written ("e" for "effects") to p.out: > > I'm warning you, \x1b[31m\x1b[4mdon't touch that!\x1b[0m > > I have a way around the square bracket problem, but it would be quite > burdensome to have to deal with all of !:{} as well; enough that > I would fall back on something like this: > > "I'm warning you, {0}".format(e.red.underline("don't touch that!")) > > or some similar interpolation-based strategy, which I think would be a > shame, because of the way it chops up the string. > > But I also think the present behavior is extremely counterintuitive, > unnecessarily complex, and illogical (even unpythonic!). Isn't it > bizarre that (g) should work, given what (e) and (f) do? Isn't it > strange that (b) and (c) should fail, given that there's no real > reason for them to do so---no ambiguity that has to be avoided? And > something's gotta give; the documentation and the implementation do > not correspond. > > Beyond the counterintuitiveness of the present implementation, it is > also, I think, self-inconsistent. (e) and (f) fail because the > interior brace is treated as starting or ending a replacement field, > even though interior replacement fields aren't allowed in that part of > a replacement field. (g) succeeds because the braces are balanced: > they are treated at one point as if they were part of a replacement > field, and at another (correctly) as if they are not. But this makes > the failure of (e) and (f) unaccountable. It would not have been > impossible for the PEP to allow interior replacement fields anywhere, > and not just in the format spec, in which case we might have had this: > > (g') "{0[{][}]}".format(range(10), **{'][':4}) --> '3' > or this: > (g'') "{0[{][}]}".format({'4':3}, **{'][':4}) --> '3' > or something with that general flavor. > > As far as I can tell, the only way to consistently maintain that (e) > and (f) should fail requires that one take (g') or (g'') to be > correct: either the interior braces signal replacement fields (hence > must be balanced) or they don't (or they're escaped). > > Regarding the documentation, it could of course be rendered correct by > changing *it*, rather than the implementation. But wouldn't it be > tremendously weird to have to explain that, in the part of the > replacement field preceding the conversion, you can't employ any curly > braces, unless they're balanced, and you can't employ ':' or '!' at > all, even though they have no semantic significance? So these are out: > > {0[{].foo} > {0[}{}]} > {0[a:b]} > > But these are in: > > {0[{}{}]} > {0[{{].foo.}}} ?(k) > > ((k) does work, if you give it an object with the right structure, > though it probably should not.) > > And, moreover, someone would then have to actually change the > documentation, whereas there's a patch already, attached to the bug > report linked way up at the top of this email, that makes (a)--(g) all > work, leaves (i) and (j) as they are, and has the welcome side-effect > of making (k) *not* work (if any code anywhere relies on (k) or things > like it working, I will be very surprised: anyway the fact that (k) > works is, technically, undocumented). It is also quite simple. It > doesn't effect the built-in format(), because the built-in format() is > concerned only with the format-specifier part of the replacement field > and not all the stuff that comes before that, telling str.format > *what* object to format. > > Thanks for reading, > -- > Ben Wolfson > "Human kind has used its intelligence to vary the flavour of drinks, > which may be sweet, aromatic, fermented or spirit-based. ... Family > and social life also offer numerous other occasions to consume drinks > for pleasure." [Larousse, "Drink" entry] > -- Ben Wolfson "Human kind has used its intelligence to vary the flavour of drinks, which may be sweet, aromatic, fermented or spirit-based. ... Family and social life also offer numerous other occasions to consume drinks for pleasure." [Larousse, "Drink" entry] From victor.stinner at haypocalc.com Thu Jul 7 02:51:55 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Thu, 07 Jul 2011 02:51:55 +0200 Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and codecs.StreamWriter Message-ID: <1309999915.2958.8.camel@marge> Hi, Last may, I proposed to deprecate open() function, StreamWriter and StreamReader classes of the codecs module. I accepted to keep open() after the discussion on python-dev. Here is a more complete proposition as a PEP. It is a draft and I expect a lot of comments :) Victor ----------------------- PEP: xxx Title: Deprecate codecs.StreamReader and codecs.StreamWriter Version: $Revision$ Last-Modified: $Date$ Author: Victor Stinner Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 28-May-2011 Python-Version: 3.3 Abstract ======== io.TextIOWrapper and codecs.StreamReaderWriter offer the same API [#f1]_. TextIOWrapper has more features and is faster than StreamReaderWriter. Duplicate code means that bugs should be fixed twice and that we may have subtle differences between the two implementations. The codecs modules was introduced in Python 2.0, see the PEP 100. The io module was introduced in Python 2.6 and 3.0 (see the PEP 3116), and reimplemented in C in Python 2.7 and 3.1. Motivation ========== When the Python I/O model was updated for 3.0, the concept of a "stream-with-known-encoding" was introduced in the form of io.TextIOWrapper. As this class is critical to the performance of text-based I/O in Python 3, this module has an optimised C version which is used by CPython by default. Many corner cases in handling buffering, stateful codecs and universal newlines have been dealt with since the release of Python 3.0. This new interface overlaps heavily with the legacy codecs.StreamReader, codecs.StreamWriter and codecs.StreamReaderWriter interfaces that were part of the original codec interface design in PEP 100. These interfaces are organised around the principle of an encoding with an associated stream (i.e. the reverse of arrangement in the io module), so the original PEP 100 design required that codec writers provide appropriate StreamReader and StreamWriter implementations in addition to the core codec encode() and decode() methods. This places a heavy burden on codec authors providing these specialised implementations to correctly handle many of the corner cases that have now been dealt with by io.TextIOWrapper. While deeper integration between the codec and the stream allows for additional optimisations in theory, these optimisations have in practice either not been carried out and else the associated code duplication means that the corner cases that have been fixed in io.TextIOWrapper are still not handled correctly in the various StreamReader and StreamWriter implementations. Accordingly, this PEP proposes that: * codecs.open() be updated to delegate to the builtin open() in Python 3.3; * the legacy codecs.Stream* interfaces, including the streamreader and streamwriter attributes of codecs.CodecInfo be deprecated in Python 3.3 and removed in Python 3.4. Rationale ========= StreamReader and StreamWriter issues '''''''''''''''''''''''''''''''''''' * StreamReader is unable to translate newlines. * StreamReaderWriter handles reads using StreamReader and writes using StreamWriter. These two classes may be inconsistent. To stay consistent, flush() must be called after each write which slows down interlaced read-write. * StreamWriter doesn't support "line buffering" (flush if the input text contains a newline). * StreamReader classes of the CJK encodings (e.g. GB18030) don't support universal newlines, only UNIX newlines ('\\n'). * StreamReader and StreamWriter are stateful codecs but don't expose functions to control their state (getstate() or setstate()). Each codec has to implement corner cases, see "Issue with stateful codecs". * StreamReader and StreamWriter are very similar to IncrementalReader and IncrementalEncoder, some code is duplicated for stateful codecs (e.g. UTF-16). * Each codec has to reimplement its own StreamReader and StreamWriter class, even if it's trivial (just call the encoder/decoder). * codecs.open(filename, "r") creates a io.TextIOWrapper object. * No codec implements an optimized method in StreamReader or StreamWriter based on the specificities of the codec. TextIOWrapper features '''''''''''''''''''''' * TextIOWrapper supports any kind of newline, including translating newlines (to UNIX newlines), to read and write. * TextIOWrapper reuses incremental encoders and decoders (no duplication of code). * The io module (TextIOWrapper) is faster than the codecs module (StreamReader). It is implemented in C, whereas codecs is implemented in Python. * TextIOWrapper has a readahead algorithm which speeds up small reads: read character by character or line by line (io is 10x through 25x faster than codecs on these operations). * TextIOWrapper has a write buffer. * TextIOWrapper.tell() is optimized. * TextIOWrapper supports random access (read+write) using a single class which permit to optimize interlaced read-write (but no such optimization is implemented). Possible improvements of StreamReader and StreamWriter '''''''''''''''''''''''''''''''''''''''''''''''''''''' It would be possible to add functions to StreamReader and StreamWriter to give access to the state of codec. And so it would be possible fix issues with stateful codecs in a base class instead of having to fix them is each stateful StreamReader and StreamWriter classes. It would be possible to change StreamReader and StreamWriter to make them use IncrementalDecoder and IncrementalEncoder. A codec can implement variants which are optimized for the specific encoding or intercept certain stream methods to add functionality or improve the encoding/decoding performance. TextIOWrapper cannot implement such optimization, but TextIOWrapper uses incremental encoders and decoders and uses read and write buffers, so the overhead of incomplete inputs is low or nul. A lot more could be done for other variable length encoding codecs, e.g. UTF-8, since these often have problems near the end of a read due to missing bytes. The UTF-32-BE/LE codecs could simply multiply the character position by 4 to get the byte position. Usage of StreamReader and StreamWriter '''''''''''''''''''''''''''''''''''''' These classes are rarely used directly, but indirectly using codecs.open(). They are not used in Python 3 standard library (except in the codecs module). Some projects implement their own codec with StreamReader and StreamWriter, but don't use these classes. Backwards Compatibility ======================= Keep the public API, codecs.open '''''''''''''''''''''''''''''''' codecs.open() can be replaced by the builtin open() function. open() has a similar API but has also more options. codecs.open() was the only way to open a text file in Unicode mode until Python 2.6. Many Python 2 programs uses this function. Removing codecs.open() implies more work to port programs from Python 2 to Python 3, especially projets using the same code base for the two Python versions (without using 2to3 program). codecs.open() is kept for backward compatibility with Python 2. Deprecate StreamReader and StreamWriter ''''''''''''''''''''''''''''''''''''''' Instanciate StreamReader or StreamWriter must raise a DeprecationWarning in Python 3.3. Implement a subclass don't raise a DeprecationWarning. codecs.open() will be changed to reuse the builtin open() function (TextIOWrapper). EncodedFile(), StreamRandom, StreamReader, StreamReaderWriter and StreamWriter will be removed in Python 3.4. Issue with stateful codecs ========================== It is difficult to use correctly a stateful codec with a stream. Some cases are supported by the codecs module, while io has no more known bug related to stateful codecs. The main difference between the codecs and the io module is that bugs have to be fixed in StreamReader and/or StreamWriter classes of each codec for the codecs module, whereas bugs can be fixed only once in io.TextIOWrapper. Here are some examples of issues with stateful codecs. Stateful codecs ''''''''''''''' Python supports the following stateful codecs: * cp932 * cp949 * cp950 * euc_jis_2004 * euc_jisx2003 * euc_jp * euc_kr * gb18030 * gbk * hz * iso2022_jp * iso2022_jp_1 * iso2022_jp_2 * iso2022_jp_2004 * iso2022_jp_3 * iso2022_jp_ext * iso2022_kr * shift_jis * shift_jis_2004 * shift_jisx0213 * utf_8_sig * utf_16 * utf_32 Read and seek(0) '''''''''''''''' :: with open(filename, 'w', encoding='utf_16') as f: f.write('abc') f.write('def') f.seek(0) assert f.read() == 'abcdef' f.seek(0) assert f.read() == 'abcdef' The io and codecs modules support this usecase correctly. Write, seek(0) and seek(n) '''''''''''''''''''''''''' :: with open(filename, 'w', encoding='utf_16') as f: f.write('abc') pos = f.tell() with open(filename, 'r+', encoding='utf_16') as f: f.seek(pos) f.write('def') f.seek(0) f.write('###') with open(filename, 'r', encoding='utf_16') as f: assert f.read() == '###def' The io module supports this usecase, whereas codecs fails because it writes a new BOM on the second write. Append mode ''''''''''' :: with open(filename, 'w', encoding='utf_16') as f: f.write('abc') with open(filename, 'a', encoding='utf_16') as f: f.write('def') with open(filename, 'r', encoding='utf_16') as f: assert f.read() == 'abcdef' The io module supports this usecase, whereas codecs fails because it writes a new BOM on the second write. Links ===== * `PEP 100: Python Unicode Integration `_ * `PEP 3116 `_ * `Issue #8796: Deprecate codecs.open() `_ * `[python-dev] Deprecate codecs.open() and StreamWriter/StreamReader `_ Copyright ========= This document has been placed in the public domain. Footnotes ========= .. [#f1] StreamReaderWriter has two more attributes than TextIOWrapper, reader and writer. From benjamin at python.org Thu Jul 7 03:16:39 2011 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 6 Jul 2011 20:16:39 -0500 Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and codecs.StreamWriter In-Reply-To: <1309999915.2958.8.camel@marge> References: <1309999915.2958.8.camel@marge> Message-ID: 2011/7/6 Victor Stinner : > codecs.open() will be changed to reuse the builtin open() function > (TextIOWrapper). This doesn't strike me as particularly backwards compatible, since you've just enumerated the differences between StreamWriter/Reader and TextIOWrapper. -- Regards, Benjamin From ncoghlan at gmail.com Thu Jul 7 05:26:20 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 7 Jul 2011 13:26:20 +1000 Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and codecs.StreamWriter In-Reply-To: References: <1309999915.2958.8.camel@marge> Message-ID: On Thu, Jul 7, 2011 at 11:16 AM, Benjamin Peterson wrote: > 2011/7/6 Victor Stinner : >> codecs.open() will be changed to reuse the builtin open() function >> (TextIOWrapper). > > This doesn't strike me as particularly backwards compatible, since > you've just enumerated the differences between StreamWriter/Reader and > TextIOWrapper. The API of the resulting object is the same (i.e. they're file-like objects). The behavioural differences are due to cases where the codec-specific classes are currently broken. Victor, could you please check this into the PEPs repo? It's easier to reference once it has a real number. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From benjamin at python.org Thu Jul 7 05:51:52 2011 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 6 Jul 2011 22:51:52 -0500 Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and codecs.StreamWriter In-Reply-To: References: <1309999915.2958.8.camel@marge> Message-ID: 2011/7/6 Nick Coghlan : > On Thu, Jul 7, 2011 at 11:16 AM, Benjamin Peterson wrote: >> 2011/7/6 Victor Stinner : >>> codecs.open() will be changed to reuse the builtin open() function >>> (TextIOWrapper). >> >> This doesn't strike me as particularly backwards compatible, since >> you've just enumerated the differences between StreamWriter/Reader and >> TextIOWrapper. > > The API of the resulting object is the same (i.e. they're file-like > objects). The behavioural differences are due to cases where the > codec-specific classes are currently broken. Yes, but as we all know too well, people are surely relying on whatever behavior there is, broken or not. -- Regards, Benjamin From vinay_sajip at yahoo.co.uk Thu Jul 7 08:53:50 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Thu, 7 Jul 2011 06:53:50 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?Draft_PEP=3A_Deprecate_codecs=2EStreamRead?= =?utf-8?q?er_and=09codecs=2EStreamWriter?= References: <1309999915.2958.8.camel@marge> Message-ID: Benjamin Peterson python.org> writes: > > 2011/7/6 Nick Coghlan gmail.com>: > > The API of the resulting object is the same (i.e. they're file-like > > objects). The behavioural differences are due to cases where the > > codec-specific classes are currently broken. > > Yes, but as we all know too well, people are surely relying on > whatever behavior there is, broken or not. > There's also the fact that code which currently runs under 2.x and 3.x would stop working if codecs.StreamReader/StreamWriter were to go away. Of course, if the codecs interfaces were re-implemented using io module code, the only portability issues would be because of people relying on broken aspects of the existing codecs code - which is unlikely to be all (or even most) of the people using codecs.StreamReader/StreamWriter. Regards, Vinay Sajip From mal at egenix.com Thu Jul 7 10:07:38 2011 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 07 Jul 2011 10:07:38 +0200 Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and codecs.StreamWriter In-Reply-To: <1309999915.2958.8.camel@marge> References: <1309999915.2958.8.camel@marge> Message-ID: <4E15694A.9070002@egenix.com> Victor Stinner wrote: > Hi, > > Last may, I proposed to deprecate open() function, StreamWriter and > StreamReader classes of the codecs module. I accepted to keep open() > after the discussion on python-dev. Here is a more complete proposition > as a PEP. It is a draft and I expect a lot of comments :) The PEP's arguments for deprecating two essential codec design components are very one sided, by comparing "issues" to "features". Please add all the comments I've made on the subject to the PEP. The most important one missing is the fact and major difference that TextIOWrapper does not work on a per codec basis, but only on a per stream basis. By removing the StreamReader and StreamWriter API parts of the codec design, you essentially make it impossible to add per codec variations and optimizations that require full access to the stream interface. A mentioned before, many improvements are possible and lots of those can build on TextIOWrapper and the incremental codec parts. That said, I'm not really up for a longer discussion on this. We've already had the discussion and decided against removing those parts of the codec API. Redirecting codecs.open() to open() should be investigated. For the issues you mention in the PEP, please open tickets or add ticket references to the PEP. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 07 2011) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ > Victor > > ----------------------- > > PEP: xxx > Title: Deprecate codecs.StreamReader and codecs.StreamWriter > Version: $Revision$ > Last-Modified: $Date$ > Author: Victor Stinner > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 28-May-2011 > Python-Version: 3.3 > > > Abstract > ======== > > io.TextIOWrapper and codecs.StreamReaderWriter offer the same API > [#f1]_. TextIOWrapper has more features and is faster than > StreamReaderWriter. Duplicate code means that bugs should be fixed > twice and that we may have subtle differences between the two > implementations. > > The codecs modules was introduced in Python 2.0, see the PEP 100. The > io module was introduced in Python 2.6 and 3.0 (see the PEP 3116), and > reimplemented in C in Python 2.7 and 3.1. > > > Motivation > ========== > > When the Python I/O model was updated for 3.0, the concept of a > "stream-with-known-encoding" was introduced in the form of > io.TextIOWrapper. As this class is critical to the performance of > text-based I/O in Python 3, this module has an optimised C version > which is used by CPython by default. Many corner cases in handling > buffering, stateful codecs and universal newlines have been dealt with > since the release of Python 3.0. > > This new interface overlaps heavily with the legacy > codecs.StreamReader, codecs.StreamWriter and codecs.StreamReaderWriter > interfaces that were part of the original codec interface design in > PEP 100. These interfaces are organised around the principle of an > encoding with an associated stream (i.e. the reverse of arrangement in > the io module), so the original PEP 100 design required that codec > writers provide appropriate StreamReader and StreamWriter > implementations in addition to the core codec encode() and decode() > methods. This places a heavy burden on codec authors providing these > specialised implementations to correctly handle many of the corner > cases that have now been dealt with by io.TextIOWrapper. While deeper > integration between the codec and the stream allows for additional > optimisations in theory, these optimisations have in practice either > not been carried out and else the associated code duplication means > that the corner cases that have been fixed in io.TextIOWrapper are > still not handled correctly in the various StreamReader and > StreamWriter implementations. > > Accordingly, this PEP proposes that: > > * codecs.open() be updated to delegate to the builtin open() in Python > 3.3; > * the legacy codecs.Stream* interfaces, including the streamreader and > streamwriter attributes of codecs.CodecInfo be deprecated in Python > 3.3 and removed in Python 3.4. > > > Rationale > ========= > > StreamReader and StreamWriter issues > '''''''''''''''''''''''''''''''''''' > > * StreamReader is unable to translate newlines. > * StreamReaderWriter handles reads using StreamReader and writes > using StreamWriter. These two classes may be inconsistent. To stay > consistent, flush() must be called after each write which slows > down interlaced read-write. > * StreamWriter doesn't support "line buffering" (flush if the input > text contains a newline). > * StreamReader classes of the CJK encodings (e.g. GB18030) don't > support universal newlines, only UNIX newlines ('\\n'). > * StreamReader and StreamWriter are stateful codecs but don't expose > functions to control their state (getstate() or setstate()). Each > codec has to implement corner cases, see "Issue with stateful > codecs". > * StreamReader and StreamWriter are very similar to IncrementalReader > and IncrementalEncoder, some code is duplicated for stateful codecs > (e.g. UTF-16). > * Each codec has to reimplement its own StreamReader and StreamWriter > class, even if it's trivial (just call the encoder/decoder). > * codecs.open(filename, "r") creates a io.TextIOWrapper object. > * No codec implements an optimized method in StreamReader or > StreamWriter based on the specificities of the codec. > > > TextIOWrapper features > '''''''''''''''''''''' > > * TextIOWrapper supports any kind of newline, including translating > newlines (to UNIX newlines), to read and write. > * TextIOWrapper reuses incremental encoders and decoders (no > duplication of code). > * The io module (TextIOWrapper) is faster than the codecs module > (StreamReader). It is implemented in C, whereas codecs is > implemented in Python. > * TextIOWrapper has a readahead algorithm which speeds up small > reads: read character by character or line by line (io is 10x > through 25x faster than codecs on these operations). > * TextIOWrapper has a write buffer. > * TextIOWrapper.tell() is optimized. > * TextIOWrapper supports random access (read+write) using a single > class which permit to optimize interlaced read-write (but no such > optimization is implemented). > > > Possible improvements of StreamReader and StreamWriter > '''''''''''''''''''''''''''''''''''''''''''''''''''''' > > It would be possible to add functions to StreamReader and StreamWriter > to give access to the state of codec. And so it would be possible fix > issues with stateful codecs in a base class instead of having to fix > them is each stateful StreamReader and StreamWriter classes. > > It would be possible to change StreamReader and StreamWriter to make > them use IncrementalDecoder and IncrementalEncoder. > > A codec can implement variants which are optimized for the specific > encoding or intercept certain stream methods to add functionality or > improve the encoding/decoding performance. TextIOWrapper cannot > implement such optimization, but TextIOWrapper uses incremental > encoders and decoders and uses read and write buffers, so the overhead > of incomplete inputs is low or nul. > > A lot more could be done for other variable length encoding codecs, > e.g. UTF-8, since these often have problems near the end of a read due > to missing bytes. The UTF-32-BE/LE codecs could simply multiply the > character position by 4 to get the byte position. > > > Usage of StreamReader and StreamWriter > '''''''''''''''''''''''''''''''''''''' > > These classes are rarely used directly, but indirectly using > codecs.open(). They are not used in Python 3 standard library (except > in the codecs module). > > Some projects implement their own codec with StreamReader and > StreamWriter, but don't use these classes. > > > Backwards Compatibility > ======================= > > Keep the public API, codecs.open > '''''''''''''''''''''''''''''''' > > codecs.open() can be replaced by the builtin open() function. open() > has a similar API but has also more options. > > codecs.open() was the only way to open a text file in Unicode mode > until Python 2.6. Many Python 2 programs uses this function. Removing > codecs.open() implies more work to port programs from Python 2 to > Python 3, especially projets using the same code base for the two > Python versions (without using 2to3 program). > > codecs.open() is kept for backward compatibility with Python 2. > > > Deprecate StreamReader and StreamWriter > ''''''''''''''''''''''''''''''''''''''' > > Instanciate StreamReader or StreamWriter must raise a > DeprecationWarning in Python 3.3. Implement a subclass don't raise a > DeprecationWarning. > > codecs.open() will be changed to reuse the builtin open() function > (TextIOWrapper). > > EncodedFile(), StreamRandom, StreamReader, StreamReaderWriter and > StreamWriter will be removed in Python 3.4. > > > Issue with stateful codecs > ========================== > > It is difficult to use correctly a stateful codec with a stream. Some > cases are supported by the codecs module, while io has no more known > bug related to stateful codecs. The main difference between the codecs > and the io module is that bugs have to be fixed in StreamReader and/or > StreamWriter classes of each codec for the codecs module, whereas bugs > can be fixed only once in io.TextIOWrapper. Here are some examples of > issues with stateful codecs. > > Stateful codecs > ''''''''''''''' > > Python supports the following stateful codecs: > > * cp932 > * cp949 > * cp950 > * euc_jis_2004 > * euc_jisx2003 > * euc_jp > * euc_kr > * gb18030 > * gbk > * hz > * iso2022_jp > * iso2022_jp_1 > * iso2022_jp_2 > * iso2022_jp_2004 > * iso2022_jp_3 > * iso2022_jp_ext > * iso2022_kr > * shift_jis > * shift_jis_2004 > * shift_jisx0213 > * utf_8_sig > * utf_16 > * utf_32 > > Read and seek(0) > '''''''''''''''' > > :: > > with open(filename, 'w', encoding='utf_16') as f: > f.write('abc') > f.write('def') > f.seek(0) > assert f.read() == 'abcdef' > f.seek(0) > assert f.read() == 'abcdef' > > The io and codecs modules support this usecase correctly. > > Write, seek(0) and seek(n) > '''''''''''''''''''''''''' > > :: > > with open(filename, 'w', encoding='utf_16') as f: > f.write('abc') > pos = f.tell() > with open(filename, 'r+', encoding='utf_16') as f: > f.seek(pos) > f.write('def') > f.seek(0) > f.write('###') > with open(filename, 'r', encoding='utf_16') as f: > assert f.read() == '###def' > > The io module supports this usecase, whereas codecs fails because it > writes a new BOM on the second write. > > Append mode > ''''''''''' > > :: > > with open(filename, 'w', encoding='utf_16') as f: > f.write('abc') > with open(filename, 'a', encoding='utf_16') as f: > f.write('def') > with open(filename, 'r', encoding='utf_16') as f: > assert f.read() == 'abcdef' > > The io module supports this usecase, whereas codecs fails because it > writes a new BOM on the second write. > > > Links > ===== > > * `PEP 100: Python Unicode Integration > `_ > * `PEP 3116 `_ > * `Issue #8796: Deprecate codecs.open() > `_ > * `[python-dev] Deprecate codecs.open() and StreamWriter/StreamReader > `_ > > > Copyright > ========= > > This document has been placed in the public domain. > > > Footnotes > ========= > > .. [#f1] StreamReaderWriter has two more attributes than > TextIOWrapper, reader and writer. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/mal%40egenix.com From ncoghlan at gmail.com Thu Jul 7 10:29:26 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 7 Jul 2011 18:29:26 +1000 Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and codecs.StreamWriter In-Reply-To: References: <1309999915.2958.8.camel@marge> Message-ID: On Thu, Jul 7, 2011 at 1:51 PM, Benjamin Peterson wrote: > 2011/7/6 Nick Coghlan : >> On Thu, Jul 7, 2011 at 11:16 AM, Benjamin Peterson wrote: >>> 2011/7/6 Victor Stinner : >>>> codecs.open() will be changed to reuse the builtin open() function >>>> (TextIOWrapper). >>> >>> This doesn't strike me as particularly backwards compatible, since >>> you've just enumerated the differences between StreamWriter/Reader and >>> TextIOWrapper. >> >> The API of the resulting object is the same (i.e. they're file-like >> objects). The behavioural differences are due to cases where the >> codec-specific classes are currently broken. > > Yes, but as we all know too well, people are surely relying on > whatever behavior there is, broken or not. True, but that's why changes like this are always a judgement call - is the gain in correctness worth the risk of breakage? We sometimes break workarounds when we fix bugs, too. From the discussion last time around, that particular change wasn't very controversial, which is why it is already in the 3.3 development tree. Unless somebody steps forward to fix them, the Stream* classes have to go (albeit with a suitable period of deprecation). They're *actively harmful* in their current state, so retaining the status quo is not a viable option in this case. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From vinay_sajip at yahoo.co.uk Thu Jul 7 10:49:22 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Thu, 7 Jul 2011 08:49:22 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?Draft_PEP=3A_Deprecate_codecs=2EStreamRead?= =?utf-8?q?er_and=09codecs=2EStreamWriter?= References: <1309999915.2958.8.camel@marge> Message-ID: Nick Coghlan gmail.com> writes: > Unless somebody steps forward to fix them, the Stream* classes have to > go (albeit with a suitable period of deprecation). They're *actively > harmful* in their current state, so retaining the status quo is not a > viable option in this case. I can understand that there might be specific issues with them, but isn't "actively harmful" a little strong? I don't see who is being actively harmed by them, nor how. Regards, Vinay Sajip From ncoghlan at gmail.com Thu Jul 7 11:00:48 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 7 Jul 2011 19:00:48 +1000 Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and codecs.StreamWriter In-Reply-To: References: <1309999915.2958.8.camel@marge> Message-ID: On Thu, Jul 7, 2011 at 6:49 PM, Vinay Sajip wrote: > Nick Coghlan gmail.com> writes: > >> Unless somebody steps forward to fix them, the Stream* classes have to >> go (albeit with a suitable period of deprecation). They're *actively >> harmful* in their current state, so retaining the status quo is not a >> viable option in this case. > > I can understand that there might be specific issues with them, but isn't > "actively harmful" a little strong? I don't see who is being actively harmed by > them, nor how. Anyone forward porting codecs.open based code will get subpar IO in Python 3 *because* they're trying to do the right thing in Python 2. That's actively harmful in my book. Codec writers are also told to implement utterly unnecessary functionality just because PEP 100 says so. Significantly less common, but still harmful. The bare minimum change needed is for codecs.open() to do the right thing in Py3k and be a wrapper around builtin open() and the main IO stack. Once that happens, the legacy Stream* APIs become redundant cruft that should be deprecated (although that part is significantly less important than fixing codecs.open() itself) Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From victor.stinner at haypocalc.com Thu Jul 7 12:22:51 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Thu, 07 Jul 2011 12:22:51 +0200 Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and codecs.StreamWriter In-Reply-To: References: <1309999915.2958.8.camel@marge> Message-ID: <4E1588FB.2050403@haypocalc.com> Le 07/07/2011 03:16, Benjamin Peterson a ?crit : > 2011/7/6 Victor Stinner: >> codecs.open() will be changed to reuse the builtin open() function >> (TextIOWrapper). > This doesn't strike me as particularly backwards compatible, since > you've just enumerated the differences between StreamWriter/Reader and > TextIOWrapper. Which kind of differences are you thinking about? I only listed two attributes specific to StreamReaderWriter (.reader and .writer). You mean that these attributes are used? There are maybe other subtle differences between Stream* and TextIOWrapper, but I don't think that anyone relies on them. Should I try to list all differences in the PEP? If I understood correctly the previous discussion, an important point is to be able to write code Python 2 which "just works" on Python 3 without using 2to3. If you don't rely on the subtle implementation details of Stream*, it's possible (to use codecs.open in Python 3, even if codecs.open is implemented to reuse TextIOWrapper via open). If you rely on the differences, I bet that it is easy to not use these differences (and continue to be compatible with Python 2). For example, you can replace f.reader.read() by f.read(), it's just the same. The two classical usages of codecs.open() (of text files) are: - read the whole file content - read the file line by line For this two usecases, the API is exactly the same. Using f=codecs.open() or f=open() (in Python 3, or f=io.open() in Python 2), you can use: - for line in f: ... - while True: line = f.readline(); if not line: break; ... - f.read() I'm not saying that my PEP doesn't break the compatibility, it *does* break the backward compatibility. That's why we need a PEP. That's why there is a section called "Backwards Compatibility" in the PEP. I'm trying to say that I bet that nobody will notice. The most impacting change will be (at the end) the removal of the StreamReader/StreamWriter API. If a program uses directly these classes, it will have to be updated to use TextIOWrapper (or codecs.open() or open() maybe). I wrote in a previous email: "I did search for usage of these classes on the Internet, and except projects implementing their own codecs (and so implement their StreamReader/StreamWriter classes, even if they don't use it), I only found one project using directly StreamReader: pygment (*). I searched quickly, so don't trust these results :-) StreamReader & friends are used indirectly through codecs.open(). My patch changes codecs.open() to make it reuse open (io.TextIOWrapper), so the deprecation of StreamReader would not be noticed by most users. (*) I also found Sphinx, but I was wrong: it doesn't use StreamReader, it just has a full copy of the UTF-8-SIG codec which has a StreamReader class. I don't think that the class is used." Victor From victor.stinner at haypocalc.com Thu Jul 7 12:26:06 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Thu, 07 Jul 2011 12:26:06 +0200 Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and codecs.StreamWriter In-Reply-To: References: <1309999915.2958.8.camel@marge> Message-ID: <4E1589BE.9040508@haypocalc.com> Le 07/07/2011 05:26, Nick Coghlan a ?crit : > Victor, could you please check this into the PEPs repo? It's easier to > reference once it has a real number. How do I upload it? Should I contact a PEP editor? How? Victor From victor.stinner at haypocalc.com Thu Jul 7 12:32:32 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Thu, 07 Jul 2011 12:32:32 +0200 Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and codecs.StreamWriter In-Reply-To: <4E15694A.9070002@egenix.com> References: <1309999915.2958.8.camel@marge> <4E15694A.9070002@egenix.com> Message-ID: <4E158B40.6010500@haypocalc.com> Le 07/07/2011 10:07, M.-A. Lemburg a ?crit : > The PEP's arguments for deprecating two essential codec design > components are very one sided, by comparing "issues" to "features". Yes, please help me to write an unbiased PEP. I don't know which tool is more appropriate to write a PEP with many authors. Can I upload it to the peps repository? According to the PEP 1, only a PEP editor can do that. > Please add all the comments I've made on the subject to the PEP. I tried to incorporate all of your comments, but because the discussion on the bug tracker and on python-dev was long, I missed maybe some (important) points. Sorry about that, and please tell me which points should be added to the PEP. > The most important one missing is the fact and major difference > that TextIOWrapper does not work on a per codec basis, but > only on a per stream basis. Yeah, it's not clear in the PEP, I should detail this point. > By removing the StreamReader and StreamWriter API parts of the > codec design, you essentially make it impossible to add > per codec variations and optimizations that require full access > to the stream interface. > > A mentioned before, many improvements are possible and lots of those > can build on TextIOWrapper and the incremental codec parts. I wrote that in the "Possible improvements of StreamReader and StreamWriter" section: "A codec can implement variants which are optimized for the specific encoding ..." and "It would be possible to change StreamReader and StreamWriter to make them use IncrementalDecoder and IncrementalEncoder." > For the issues you mention in the PEP, please open tickets > or add ticket references to the PEP. Ok, I will do that. There are other Stream* issues, a recent example: http://bugs.python.org/issue12508 Victor From vinay_sajip at yahoo.co.uk Thu Jul 7 12:53:56 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Thu, 7 Jul 2011 10:53:56 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?Draft_PEP=3A_Deprecate_codecs=2EStreamRead?= =?utf-8?q?er_and=09codecs=2EStreamWriter?= References: <1309999915.2958.8.camel@marge> Message-ID: Nick Coghlan gmail.com> writes: > Anyone forward porting codecs.open based code will get subpar IO in > Python 3 *because* they're trying to do the right thing in Python 2. > That's actively harmful in my book. I see. Presumably if they're doing a porting exercise, then it's easy enough for them to convert codecs.open() calls to open(), if they don't want the performance to be sub-optimal. But I thought the main thrust of this was about deprecation of the Stream* classes, not open() vs. codecs.open(). > Codec writers are also told to implement utterly unnecessary > functionality just because PEP 100 says so. Significantly less common, > but still harmful. Presumably only an issue for anyone writing new codecs for 2.x - I'm not sure how many cases that'd be. > The bare minimum change needed is for codecs.open() to do the right > thing in Py3k and be a wrapper around builtin open() and the main IO > stack. Once that happens, the legacy Stream* APIs become redundant > cruft that should be deprecated (although that part is significantly > less important than fixing codecs.open() itself) I've no issue with telling people to use open() rather than codecs.open() when moving code from 2.x to 3.x. But in 2.x, is there any other API which allows you to wrap arbitrary streams? If not, then ISTM that removing the Stream* classes would give 2.x->3.x porting projects more trouble than codecs.open() -> open(). Regards, Vinay Sajip From barry at python.org Thu Jul 7 13:12:27 2011 From: barry at python.org (Barry Warsaw) Date: Thu, 7 Jul 2011 07:12:27 -0400 Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and codecs.StreamWriter In-Reply-To: <4E1589BE.9040508@haypocalc.com> References: <1309999915.2958.8.camel@marge> <4E1589BE.9040508@haypocalc.com> Message-ID: <20110707071227.7ea79618@resist.wooz.org> On Jul 07, 2011, at 12:26 PM, Victor Stinner wrote: >Le 07/07/2011 05:26, Nick Coghlan a ?crit : >> Victor, could you please check this into the PEPs repo? It's easier to >> reference once it has a real number. >How do I upload it? Should I contact a PEP editor? How? Email peps at python.org Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From ncoghlan at gmail.com Thu Jul 7 13:43:24 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 7 Jul 2011 21:43:24 +1000 Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and codecs.StreamWriter In-Reply-To: <20110707071227.7ea79618@resist.wooz.org> References: <1309999915.2958.8.camel@marge> <4E1589BE.9040508@haypocalc.com> <20110707071227.7ea79618@resist.wooz.org> Message-ID: On Thu, Jul 7, 2011 at 9:12 PM, Barry Warsaw wrote: > On Jul 07, 2011, at 12:26 PM, Victor Stinner wrote: > >>Le 07/07/2011 05:26, Nick Coghlan a ?crit : >>> Victor, could you please check this into the PEPs repo? It's easier to >>> reference once it has a real number. >>How do I upload it? Should I contact a PEP editor? How? > > Email peps at python.org Or just check it in to hg.python.org/peps (claiming the next number in sequence - 400 at the time of writing this email). I asked if that approach was OK quite some time ago and David said yes - PEP 1 is written the way it is because not everyone that writes a PEP has commit privileges for the python.org repos. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Thu Jul 7 14:08:45 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 7 Jul 2011 22:08:45 +1000 Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and codecs.StreamWriter In-Reply-To: References: <1309999915.2958.8.camel@marge> Message-ID: On Thu, Jul 7, 2011 at 8:53 PM, Vinay Sajip wrote: > I've no issue with telling people to use open() rather than codecs.open() when > moving code from 2.x to 3.x. But in 2.x, is there any other API which allows you > to wrap arbitrary streams? If not, then ISTM that removing the Stream* classes > would give 2.x->3.x porting projects more trouble than codecs.open() -> open(). No, using the io module is a far more robust way to wrap arbitrary streams than using the codecs module. It's unfortunate that nobody pointed out the redundancy when PEP 3116 was discussed and implemented, as I expect PEP 100 would have been updated and the Stream* APIs would have been either reused or officially jettisoned as part of the Py3k migration. However, we're now in a situation where we have: 1. A robust Unicode capable IO implementation (the io module, based on PEP 3116) that is available in both 2.x and 3.x that is designed to minimise the amount of work involved in writing new codecs 2. A legacy IO implementation (the codecs module) that is available in both 2.x and 3.x, but requires additional work on the part of codec authors and isn't as robust as the PEP 3116 implementation So the options are: A. Bring the codecs module IO implementation up to the standard of the io module implementation (less the C acceleration) and maintain the requirement that codec authors provide StreamReader and StreamWriter implementations. B. Retain the full codecs module API, but reimplement relevant parts in terms of the io module. C. Deprecate the codecs.Stream* interfaces and make codecs.open() a simple wrapper around the builtin open() function. Formally drop the requirement that codec authors provide StreamReader/Writer instances (since they are not used by the core IO implementation) Currently, nobody has stepped forward to do the work of maintaining the codecs IO implementation independently of the io module, so the only two options seriously on the table are B and C. That may change if someone actually goes through and *fixes* all the error cases that are handled correctly by the io module but not by the codecs module and credibly promises to continue to do so for at least the life of 3.3. A 2to3 fixer that simply changes "codecs.open" to "open" is not viable, as the function signatures are not compatible (the buffering parameter appears in a different location): codecs.open(): open(filename, mode='rb', encoding=None, errors='strict', buffering=1) 3.x builtin open(): open(file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True) Now, the backported io module does make it possible to write correct code as far back as 2.6 that can be forward ported cleanly to 3.x without requiring code modifications. However, it would be nice to transparently upgrade code that uses codecs.open to the core IO implementation in 3.x. For people new to Python, the parallel (and currently deficient) alternative IO implementation also qualifies at the very least as an attractive nuisance. Now, it may be that this PEP runs afoul of Guido's stated preference not to introduce any more backwards incompatibilities between 2.x and 3.x that aren't absolutely essential. In that case, it may be reasonable to add an option D to the mix, where we just add documentation notes telling people not to use the affected codecs module APIs and officially declare that bug reports on those APIs will be handled with "don't use these, use the io module instead", as that would also deal with the maintenance problem. It's pretty ugly from an end user's point of view, though. Regards, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From victor.stinner at haypocalc.com Thu Jul 7 14:11:14 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Thu, 07 Jul 2011 14:11:14 +0200 Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and codecs.StreamWriter In-Reply-To: References: <1309999915.2958.8.camel@marge> <4E1589BE.9040508@haypocalc.com> <20110707071227.7ea79618@resist.wooz.org> Message-ID: <4E15A262.20100@haypocalc.com> Le 07/07/2011 13:43, Nick Coghlan a ?crit : > Or just check it in to hg.python.org/peps (claiming the next number in > sequence - 400 at the time of writing this email). I asked if that > approach was OK quite some time ago and David said yes - PEP 1 is > written the way it is because not everyone that writes a PEP has > commit privileges for the python.org repos. Ok, done: http://www.python.org/dev/peps/pep-0400/ http://hg.python.org/peps/file/tip/pep-0400.txt I started to include Marc-Andre's suggestions. Victor From gklein at xs4all.nl Thu Jul 7 14:59:21 2011 From: gklein at xs4all.nl (Gertjan Klein) Date: Thu, 07 Jul 2011 14:59:21 +0200 Subject: [Python-Dev] Python Launcher for Windows (PEP 397) needs testing! References: Message-ID: <6rab1717d1t3prr464dfcgofebp7mr59js@4ax.com> Vinay Sajip wrote: >The C implementation of the PEP 397-compatible Python Launcher for Windows has >come along nicely in the last few days, and now reached a point where it would >benefit from some testing by interested python-dev members. I've gotten the sources from: >https://bitbucket.org/vinay.sajip/pylauncher GUILauncher and CLILauncher refuse to build with Visual C++ 2008 Express Edition (using Launchers.sln): .\CLILauncher.rc(97) : error RC2135 : file not found: C:\Users\Vinay\Projects\Launchers\launcher.ico This is on Windows XP Pro, 32 bit. There are a few compilation warnings as well: .\launcher.c(59) : warning C4996: '_wgetenv': This function or variable may be unsafe. Consider using _wdupenv_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details. Associator builds (with the same warning displayed above). When checking it's dependencies, I see it depends on MSVCR90.DLL. You've mentioned that you use only "plain C and Win32 APIs"; would it be possible to remove this dependency? That would make it possible to copy the executable to a directory on the PATH, without having to worry about installing the C(++) runtime. Regards, Gertjan. From victor.stinner at haypocalc.com Thu Jul 7 15:26:11 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Thu, 07 Jul 2011 15:26:11 +0200 Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and codecs.StreamWriter In-Reply-To: References: <1309999915.2958.8.camel@marge> Message-ID: <4E15B3F3.8010506@haypocalc.com> Le 07/07/2011 12:53, Vinay Sajip a ?crit : > I've no issue with telling people to use open() rather than codecs.open() when > moving code from 2.x to 3.x. But in 2.x, is there any other API which allows you > to wrap arbitrary streams? Yes, io.TextIOWrapper. Victor From p.f.moore at gmail.com Thu Jul 7 16:10:50 2011 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 7 Jul 2011 15:10:50 +0100 Subject: [Python-Dev] Python Launcher for Windows (PEP 397) needs testing! In-Reply-To: References: Message-ID: On 6 July 2011 19:31, Vinay Sajip wrote: > The C implementation of the PEP 397-compatible Python Launcher for Windows has > come along nicely in the last few days, and now reached a point where it would > benefit from some testing by interested python-dev members. Points of note: > > 1. As well as source available on > > https://bitbucket.org/vinay.sajip/pylauncher > > there are built 32- and 64-bit msi files at > > https://bitbucket.org/vinay.sajip/pylauncher/downloads I just tried to install launcher.msi, and it seems to have done nothing. This is on Windows XP SP3 with Python 2.7 installed. It's a corporate build so there may be funny security limits hindering things, but I have admin rights and normally don't have problems. msiexec /i launcher.msi /l* launcher.log gives the following log file. I can't see any errors in there, but all those "return code 1" entries look odd (surely 0 is success and 1 failure as with OS commands?) Paul. === Logging started: 07/07/2011 15:01:42 === Action 15:01:42: INSTALL. Action start 15:01:42: INSTALL. Action 15:01:42: ProgramFilesFolder.9CA91B9B_69A1_494B_A200_44AA4D54BC92. Action start 15:01:42: ProgramFilesFolder.9CA91B9B_69A1_494B_A200_44AA4D54BC92. Action ended 15:01:42: ProgramFilesFolder.9CA91B9B_69A1_494B_A200_44AA4D54BC92. Return value 1. Action 15:01:42: ValidateProductID. Action start 15:01:42: ValidateProductID. Action ended 15:01:42: ValidateProductID. Return value 1. Action 15:01:42: CostInitialize. Computing space requirements Action start 15:01:42: CostInitialize. Action ended 15:01:42: CostInitialize. Return value 1. Action 15:01:42: FileCost. Computing space requirements Action start 15:01:42: FileCost. Action ended 15:01:42: FileCost. Return value 1. Action 15:01:42: CostFinalize. Computing space requirements Action start 15:01:42: CostFinalize. Action ended 15:01:42: CostFinalize. Return value 1. Action 15:01:42: ExecuteAction. Action start 15:01:42: ExecuteAction. Action start 15:01:42: INSTALL. Action start 15:01:42: ProgramFilesFolder.9CA91B9B_69A1_494B_A200_44AA4D54BC92. Action ended 15:01:42: ProgramFilesFolder.9CA91B9B_69A1_494B_A200_44AA4D54BC92. Return value 1. Action start 15:01:42: ValidateProductID. Action ended 15:01:42: ValidateProductID. Return value 1. Action start 15:01:42: CostInitialize. Action ended 15:01:42: CostInitialize. Return value 1. Action start 15:01:42: FileCost. Action ended 15:01:42: FileCost. Return value 1. Action start 15:01:42: CostFinalize. Action ended 15:01:42: CostFinalize. Return value 1. Action start 15:01:42: InstallValidate. Action ended 15:01:42: InstallValidate. Return value 1. Action start 15:01:42: InstallInitialize. Action ended 15:01:42: InstallInitialize. Return value 1. Action start 15:01:42: ProcessComponents. Action ended 15:01:42: ProcessComponents. Return value 1. Action start 15:01:42: UnpublishFeatures. Action ended 15:01:42: UnpublishFeatures. Return value 1. Action start 15:01:42: RemoveRegistryValues. Action ended 15:01:42: RemoveRegistryValues. Return value 1. Action start 15:01:42: RemoveFiles. Action ended 15:01:42: RemoveFiles. Return value 0. Action start 15:01:42: InstallFiles. Action ended 15:01:42: InstallFiles. Return value 1. Action start 15:01:42: WriteRegistryValues. Action ended 15:01:42: WriteRegistryValues. Return value 1. Action start 15:01:42: RegisterUser. Action ended 15:01:42: RegisterUser. Return value 0. Action start 15:01:42: RegisterProduct. Action ended 15:01:42: RegisterProduct. Return value 1. Action start 15:01:42: PublishFeatures. Action ended 15:01:42: PublishFeatures. Return value 1. Action start 15:01:42: PublishProduct. Action ended 15:01:42: PublishProduct. Return value 1. Action start 15:01:42: InstallFinalize. Action ended 15:01:43: InstallFinalize. Return value 1. Action ended 15:01:43: INSTALL. Return value 1. Property(S): TARGETDIR = D:\ Property(S): SourceDir = D:\Downloads\ Property(S): Manufacturer = Vinay Sajip Property(S): ProductCode = {298B5D62-1287-427F-B8D9-B44D605F8F6B} Property(S): ProductLanguage = 1033 Property(S): ProductName = Python Launcher Property(S): ProductVersion = 1.0.0.0 Property(S): UpgradeCode = {36B0A82E-0B4E-47CD-895B-FD4EC726B3AC} Property(S): ARPPRODUCTICON = arpicon Property(S): ProgramFilesFolder.9CA91B9B_69A1_494B_A200_44AA4D54BC92 = C:\Program Files\ Property(S): ProgDir.9CA91B9B_69A1_494B_A200_44AA4D54BC92 = C:\Program Files\Python Launcher\ Property(S): PackageCode = {C9DC0563-F67F-4F32-A0B1-1D77B10AD2AF} Property(S): ProductState = 5 Property(S): ProductToBeRegistered = 1 Property(S): CURRENTDIRECTORY = D:\Data Property(S): CLIENTUILEVEL = 0 Property(S): CLIENTPROCESSID = 5636 Property(S): PRODUCTLANGUAGE = 1033 Property(S): USERNAME = Atos Origin Property(S): COMPANYNAME = Atos Origin Property(S): ROOTDRIVE = D:\ Property(S): EXECUTEACTION = INSTALL Property(S): ACTION = INSTALL Property(S): INSTALLLEVEL = 1 Property(S): SECONDSEQUENCE = 1 Property(S): _MSI_FEATURE_SELECTION = _NONE_ Property(S): VersionDatabase = 200 Property(S): VersionMsi = 3.01 Property(S): VersionNT = 501 Property(S): WindowsBuild = 2600 Property(S): ServicePackLevel = 3 Property(S): ServicePackLevelMinor = 0 Property(S): MsiNTProductType = 1 Property(S): WindowsFolder = C:\WINDOWS\ Property(S): WindowsVolume = C:\ Property(S): SystemFolder = C:\WINDOWS\system32\ Property(S): System16Folder = C:\WINDOWS\system\ Property(S): RemoteAdminTS = 1 Property(S): TempFolder = C:\TEMP\ Property(S): ProgramFilesFolder = C:\Program Files\ Property(S): CommonFilesFolder = C:\Program Files\Common Files\ Property(S): AppDataFolder = D:\Documents and Settings\UK03306\Application Data\ Property(S): FavoritesFolder = D:\Documents and Settings\UK03306\Favorites\ Property(S): NetHoodFolder = D:\Documents and Settings\UK03306\NetHood\ Property(S): PersonalFolder = D:\Documents and Settings\UK03306\My Documents\ Property(S): PrintHoodFolder = D:\Documents and Settings\UK03306\PrintHood\ Property(S): RecentFolder = D:\Documents and Settings\UK03306\Recent\ Property(S): SendToFolder = D:\Documents and Settings\UK03306\SendTo\ Property(S): TemplateFolder = D:\Documents and Settings\UK03306\Templates\ Property(S): CommonAppDataFolder = D:\Documents and Settings\All Users\Application Data\ Property(S): LocalAppDataFolder = D:\Documents and Settings\UK03306\Local Settings\Application Data\ Property(S): MyPicturesFolder = D:\Documents and Settings\UK03306\My Documents\My Pictures\ Property(S): AdminToolsFolder = D:\Documents and Settings\UK03306\Start Menu\Programs\Administrative Tools\ Property(S): StartupFolder = D:\Documents and Settings\UK03306\Start Menu\Programs\Startup\ Property(S): ProgramMenuFolder = D:\Documents and Settings\UK03306\Start Menu\Programs\ Property(S): StartMenuFolder = D:\Documents and Settings\UK03306\Start Menu\ Property(S): DesktopFolder = D:\Documents and Settings\UK03306\Desktop\ Property(S): FontsFolder = C:\WINDOWS\Fonts\ Property(S): GPTSupport = 1 Property(S): OLEAdvtSupport = 1 Property(S): ShellAdvtSupport = 1 Property(S): Intel = 6 Property(S): PhysicalMemory = 2991 Property(S): VirtualMemory = 3325 Property(S): AdminUser = 1 Property(S): LogonUser = UK03306 Property(S): UserSID = S-1-5-21-1957994488-1604221776-515967899-4426 Property(S): UserLanguageID = 2057 Property(S): ComputerName = UKCNU1131P6N Property(S): SystemLanguageID = 2057 Property(S): ScreenX = 1280 Property(S): ScreenY = 1024 Property(S): CaptionHeight = 26 Property(S): BorderTop = 1 Property(S): BorderSide = 1 Property(S): TextHeight = 16 Property(S): ColorBits = 32 Property(S): TTCSupport = 1 Property(S): Time = 15:01:43 Property(S): Date = 07/07/2011 Property(S): MsiNetAssemblySupport = 2.0.50727.3053 Property(S): MsiWin32AssemblySupport = 5.1.2600.5512 Property(S): RedirectedDllSupport = 2 Property(S): Privileged = 1 Property(S): Installed = 00:00:00 Property(S): DATABASE = C:\WINDOWS\Installer\153760c.msi Property(S): OriginalDatabase = D:\Downloads\launcher.msi Property(S): UILevel = 5 Property(S): CostingComplete = 1 Property(S): OutOfDiskSpace = 0 Property(S): OutOfNoRbDiskSpace = 0 Property(S): PrimaryVolumeSpaceAvailable = 0 Property(S): PrimaryVolumeSpaceRequired = 0 Property(S): PrimaryVolumeSpaceRemaining = 0 Property(S): SOURCEDIR = D:\Downloads\ Property(S): SourcedirProduct = {298B5D62-1287-427F-B8D9-B44D605F8F6B} Action ended 15:01:43: ExecuteAction. Return value 1. Action ended 15:01:43: INSTALL. Return value 1. Property(C): TARGETDIR = D:\ Property(C): Manufacturer = Vinay Sajip Property(C): ProductCode = {298B5D62-1287-427F-B8D9-B44D605F8F6B} Property(C): ProductLanguage = 1033 Property(C): ProductName = Python Launcher Property(C): ProductVersion = 1.0.0.0 Property(C): UpgradeCode = {36B0A82E-0B4E-47CD-895B-FD4EC726B3AC} Property(C): ARPPRODUCTICON = arpicon Property(C): ProgramFilesFolder.9CA91B9B_69A1_494B_A200_44AA4D54BC92 = C:\Program Files\ Property(C): ProgDir.9CA91B9B_69A1_494B_A200_44AA4D54BC92 = C:\Program Files\Python Launcher\ Property(C): PackageCode = {C9DC0563-F67F-4F32-A0B1-1D77B10AD2AF} Property(C): ProductState = 5 Property(C): ProductToBeRegistered = 1 Property(C): CURRENTDIRECTORY = D:\Data Property(C): CLIENTUILEVEL = 0 Property(C): CLIENTPROCESSID = 5636 Property(C): PRODUCTLANGUAGE = 1033 Property(C): VersionDatabase = 200 Property(C): VersionMsi = 3.01 Property(C): VersionNT = 501 Property(C): WindowsBuild = 2600 Property(C): ServicePackLevel = 3 Property(C): ServicePackLevelMinor = 0 Property(C): MsiNTProductType = 1 Property(C): WindowsFolder = C:\WINDOWS\ Property(C): WindowsVolume = C:\ Property(C): SystemFolder = C:\WINDOWS\system32\ Property(C): System16Folder = C:\WINDOWS\system\ Property(C): RemoteAdminTS = 1 Property(C): TempFolder = C:\TEMP\ Property(C): ProgramFilesFolder = C:\Program Files\ Property(C): CommonFilesFolder = C:\Program Files\Common Files\ Property(C): AppDataFolder = D:\Documents and Settings\UK03306\Application Data\ Property(C): FavoritesFolder = D:\Documents and Settings\UK03306\Favorites\ Property(C): NetHoodFolder = D:\Documents and Settings\UK03306\NetHood\ Property(C): PersonalFolder = D:\Documents and Settings\UK03306\My Documents\ Property(C): PrintHoodFolder = D:\Documents and Settings\UK03306\PrintHood\ Property(C): RecentFolder = D:\Documents and Settings\UK03306\Recent\ Property(C): SendToFolder = D:\Documents and Settings\UK03306\SendTo\ Property(C): TemplateFolder = D:\Documents and Settings\UK03306\Templates\ Property(C): CommonAppDataFolder = D:\Documents and Settings\All Users\Application Data\ Property(C): LocalAppDataFolder = D:\Documents and Settings\UK03306\Local Settings\Application Data\ Property(C): MyPicturesFolder = D:\Documents and Settings\UK03306\My Documents\My Pictures\ Property(C): AdminToolsFolder = D:\Documents and Settings\UK03306\Start Menu\Programs\Administrative Tools\ Property(C): StartupFolder = D:\Documents and Settings\UK03306\Start Menu\Programs\Startup\ Property(C): ProgramMenuFolder = D:\Documents and Settings\UK03306\Start Menu\Programs\ Property(C): StartMenuFolder = D:\Documents and Settings\UK03306\Start Menu\ Property(C): DesktopFolder = D:\Documents and Settings\UK03306\Desktop\ Property(C): FontsFolder = C:\WINDOWS\Fonts\ Property(C): GPTSupport = 1 Property(C): OLEAdvtSupport = 1 Property(C): ShellAdvtSupport = 1 Property(C): Intel = 6 Property(C): PhysicalMemory = 2991 Property(C): VirtualMemory = 3325 Property(C): AdminUser = 1 Property(C): LogonUser = UK03306 Property(C): UserSID = S-1-5-21-1957994488-1604221776-515967899-4426 Property(C): UserLanguageID = 2057 Property(C): ComputerName = UKCNU1131P6N Property(C): SystemLanguageID = 2057 Property(C): ScreenX = 1280 Property(C): ScreenY = 1024 Property(C): CaptionHeight = 26 Property(C): BorderTop = 1 Property(C): BorderSide = 1 Property(C): TextHeight = 16 Property(C): ColorBits = 32 Property(C): TTCSupport = 1 Property(C): Time = 15:01:43 Property(C): Date = 07/07/2011 Property(C): MsiNetAssemblySupport = 2.0.50727.3053 Property(C): MsiWin32AssemblySupport = 5.1.2600.5512 Property(C): RedirectedDllSupport = 2 Property(C): Privileged = 1 Property(C): USERNAME = Atos Origin Property(C): COMPANYNAME = Atos Origin Property(C): Installed = 00:00:00 Property(C): DATABASE = C:\WINDOWS\Installer\153760c.msi Property(C): OriginalDatabase = D:\Downloads\launcher.msi Property(C): VersionHandler = 3.01 Property(C): ROOTDRIVE = D:\ Property(C): EXECUTEACTION = INSTALL Property(C): ACTION = INSTALL Property(C): UILevel = 5 Property(C): CostingComplete = 0 Property(C): OutOfDiskSpace = 0 Property(C): OutOfNoRbDiskSpace = 0 Property(C): PrimaryVolumeSpaceAvailable = 0 Property(C): PrimaryVolumeSpaceRequired = 0 Property(C): PrimaryVolumeSpaceRemaining = 0 Property(C): INSTALLLEVEL = 1 === Logging stopped: 07/07/2011 15:01:43 === MSI (c) (04:38) [15:01:43:087]: Product: Python Launcher -- Configuration completed successfully. From victor.stinner at haypocalc.com Thu Jul 7 16:27:14 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Thu, 07 Jul 2011 16:27:14 +0200 Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and codecs.StreamWriter In-Reply-To: References: <1309999915.2958.8.camel@marge> Message-ID: <4E15C242.20902@haypocalc.com> > B. Retain the full codecs module API, but reimplement relevant parts > in terms of the io module. This solution would not break backward compatibility, or less than my PEP. I didn't try to implement this solution. It should be possible for StreamReader (-> TextIOWrapper), StreamWriter (-> TextIOWrapper) and StreamReaderWriter (-> TextIOWrapper), but not for EncodedFile (by the why, who use this horrible class? :-)). I would prefer solution C to have only one obvious way to read-write text files in Python 3(.4). > A 2to3 fixer that simply changes "codecs.open" to "open" is not > viable, as the function signatures are not compatible (the buffering > parameter appears in a different location): > codecs.open(): open(filename, mode='rb', encoding=None, > errors='strict', buffering=1) > 3.x builtin open(): open(file, mode='r', buffering=-1, > encoding=None, errors=None, newline=None, closefd=True) What? The prototypes are very close, you just to have to invert some arguments. Why do you think that we cannot implement that? > Now, the backported io module does make it possible to write correct > code as far back as 2.6 that can be forward ported cleanly to 3.x > without requiring code modifications. However, it would be nice to > transparently upgrade code that uses codecs.open to the core IO > implementation in 3.x. For people new to Python, the parallel (and > currently deficient) alternative IO implementation also qualifies at > the very least as an attractive nuisance. Use codecs.open() if you would like to support Python 3 and Python 2 older than 2.6. If you don't care of Python 2.5, use directly the io module (you just have to know that it is slow in Python 2.6). Victor From solipsis at pitrou.net Thu Jul 7 16:28:07 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 7 Jul 2011 16:28:07 +0200 Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and codecs.StreamWriter References: <1309999915.2958.8.camel@marge> Message-ID: <20110707162807.7c5932aa@pitrou.net> On Thu, 7 Jul 2011 06:53:50 +0000 (UTC) Vinay Sajip wrote: > Benjamin Peterson python.org> writes: > > > > > 2011/7/6 Nick Coghlan gmail.com>: > > > > The API of the resulting object is the same (i.e. they're file-like > > > objects). The behavioural differences are due to cases where the > > > codec-specific classes are currently broken. > > > > Yes, but as we all know too well, people are surely relying on > > whatever behavior there is, broken or not. > > There's also the fact that code which currently runs under 2.x and 3.x would > stop working if codecs.StreamReader/StreamWriter were to go away. That's a fact of life for any deprecation. But it only stops working *after* the deprecation period has expired. And deprecated stuff can actually stay in for a long time, depending on its popularity. The main point of the PEP, IMO, is actually the deprecation itself. By deprecating, we signal that something isn't actively maintained anymore, and that a (allegedly better) alternative is available. I think that's a very reasonable thing to do, regardless of whether or not the "thing" actually gets removed in a later version. Regards Antoine. From solipsis at pitrou.net Thu Jul 7 16:31:37 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 7 Jul 2011 16:31:37 +0200 Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and codecs.StreamWriter References: <1309999915.2958.8.camel@marge> <4E15694A.9070002@egenix.com> Message-ID: <20110707163137.38d88c33@pitrou.net> On Thu, 07 Jul 2011 10:07:38 +0200 "M.-A. Lemburg" wrote: > > That said, I'm not really up for a longer discussion on this. We've > already had the discussion and decided against removing those > parts of the codec API. I don't remember any such decision. We decided against unilateraly removing them without a PEP, which is quite different. Now we have a PEP and the matter can be discussed using the appropriate procedure. Regards Antoine. From p.f.moore at gmail.com Thu Jul 7 16:41:46 2011 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 7 Jul 2011 15:41:46 +0100 Subject: [Python-Dev] Python Launcher for Windows (PEP 397) needs testing! In-Reply-To: <1310048670.36941.YahooMailRC@web25808.mail.ukl.yahoo.com> References: <1310048670.36941.YahooMailRC@web25808.mail.ukl.yahoo.com> Message-ID: On 7 July 2011 15:24, Vinay Sajip wrote: > Hi Paul, > > Thanks for trying it out. If it installs successfully, nothing will appear to > happen, Unix-style :-) > > There should be files installed in c:\Program Files\Python Launcher: > > py.exe, pyw.exe, py.ini > > and there should be registry entries in HKEY_CLASSES_ROOT\Python.File, > Python.NoConFile and Python.CompiledFile which point to those installed files. > > Can you confirm if this is the case? Ah, yes. This is what has happened. In that case, some points: 1. A "silent by default" installer like this is not very usual in the Windows world, I'd have expected a confirmation dialog at least. For silent installs, msiexec /silent is available. 2. I thought the idea from the PEP was to put py.exe/pyw.exe into a "system" location which is already on the PATH (likely windows\system32, or whatever the 64-bit equivalent is). That way, py or py -3 and similar can be used to launch the interactive interpreter. 3. Given that you're not installing in system32, you should add the install dir to PATH (and remove it on uninstall :-)) That's definitely a second-best option, though, so I'd rather you didn't, but installed in system32 instead :-) 4. If you embed both of the py.ico and pyc.ico files into the launcher exes, you wouldn't need to have them as separate files (I see py.ico is embedded already) and so the footprint becomes 2 exes and an ini file. That might be a bit more palatable for people who are uncomfortable with installing into somewhere like system32. Otherwise it looks great. > P.S. I didn't copy the list, as if there is some problem which requires going > back and forth a bit, it would be OT for the list. Agreed. I've added the list back in as the above is really design feedback, and so would benefit from comments from the list. Paul. From vinay_sajip at yahoo.co.uk Thu Jul 7 17:07:04 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Thu, 7 Jul 2011 15:07:04 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?Python_Launcher_for_Windows_=28PEP_397=29_?= =?utf-8?q?needs=09testing!?= References: <6rab1717d1t3prr464dfcgofebp7mr59js@4ax.com> Message-ID: Hi Gertjan, Thanks for trying it. > .\CLILauncher.rc(97) : error RC2135 : file not found: > C:\Users\Vinay\Projects\Launchers\launcher.ico Somewhere there's an absolute path where it should be relative - I'll get on it. > There are a few compilation warnings as well: > > .\launcher.c(59) : warning C4996: '_wgetenv': This function or variable > may be unsafe. Consider using _wdupenv_s instead. To disable > deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details. This is a security related warning which I think can be ignored - we only ever read environment values, never write them, so making a private copy via _wdupenv isn't really needed IMO. I don't disable the warning because it would disable all security-related warnings. > Associator builds (with the same warning displayed above). When checking > it's dependencies, I see it depends on MSVCR90.DLL. You've mentioned > that you use only "plain C and Win32 APIs"; would it be possible to > remove this dependency? That would make it possible to copy the > executable to a directory on the PATH, without having to worry about > installing the C(++) runtime. I noticed this too, and later builds of associator have the library linked in statically. If you get any more issues, you can post them on the BitBucket issue tracker - they are probably OT for here, unless design/PEP related. Regards, Vinay Sajip From vinay_sajip at yahoo.co.uk Thu Jul 7 17:19:26 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Thu, 7 Jul 2011 15:19:26 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?Python_Launcher_for_Windows_=28PEP_397=29_?= =?utf-8?q?needs=09testing!?= References: <1310048670.36941.YahooMailRC@web25808.mail.ukl.yahoo.com> Message-ID: Paul Moore gmail.com> writes: > In that case, some points: > > 1. A "silent by default" installer like this is not very usual in the > Windows world, I'd have expected a confirmation dialog at least. For > silent installs, msiexec /silent is available. Agreed, a "success" message is probably a good idea for the standalone launcher. The installer is a bit tentative because (a) the PEP is still under discussion, and (b) I don't know exactly how launcher installation will work as part of Python installation. > 2. I thought the idea from the PEP was to put py.exe/pyw.exe into a > "system" location which is already on the PATH (likely > windows\system32, or whatever the 64-bit equivalent is). That way, py > or py -3 and similar can be used to launch the interactive > interpreter. This can always be changed in the installer - the PEP says install in System32 "if possible", and I'm not yet sure of all the security/permissions implications of that. The current location should be usable for test purposes, even if you have to manually add to the path for now. > 3. Given that you're not installing in system32, you should add the > install dir to PATH (and remove it on uninstall ) That's definitely > a second-best option, though, so I'd rather you didn't, but installed > in system32 instead I'll look at changing the installer builds to do this. > 4. If you embed both of the py.ico and pyc.ico files into the launcher > exes, you wouldn't need to have them as separate files (I see py.ico > is embedded already) and so the footprint becomes 2 exes and an ini > file. That might be a bit more palatable for people who are > uncomfortable with installing into somewhere like system32. > > Otherwise it looks great. > Thanks for the feedback. Please log any implementation-related issues on the BitBucket tracker. Regards, Vinay Sajip From solipsis at pitrou.net Thu Jul 7 18:22:43 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 7 Jul 2011 18:22:43 +0200 Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and codecs.StreamWriter References: <1309999915.2958.8.camel@marge> Message-ID: <20110707182243.6037f5fd@pitrou.net> On Thu, 7 Jul 2011 22:08:45 +1000 Nick Coghlan wrote: > Currently, nobody has stepped forward to do the work of maintaining > the codecs IO implementation independently of the io module, so the > only two options seriously on the table are B and C. Since nobody has stepped up to implement option B either, I think the only option seriously on the table is C. > Now, it may be that this PEP runs afoul of Guido's stated preference > not to introduce any more backwards incompatibilities between 2.x and > 3.x that aren't absolutely essential. Well, a deprecation isn't an incompatibility in itself. Especially when deprecation warnings are hidden by default. Regards Antoine. From widebandwidth at gmail.com Thu Jul 7 18:57:59 2011 From: widebandwidth at gmail.com (high bandwidth) Date: Thu, 7 Jul 2011 12:57:59 -0400 Subject: [Python-Dev] making socket.getaddrinfo use cached dns Message-ID: I use cached dns lookups with pdnsd on my ubuntu machine to speed up web access as regular lookups can take 15-30 seconds. However, python's mechanize and urllib etc use socket.getaddrinfo, which seems not to be using dns cacheing or taking a long time because of ipv6 lookups. In either case, I subsequent access to the same site to be fast and not require lengthy calls to getaddrinfo. How can I get python to correctly use cached dns lookups and ipv4 only (at least in those cases where it is appropriate). As mentioned here: http://stackoverflow.com/questions/2014534/force-python-mechanize-urllib2-to-only-use-a-requests/2035686#2035686 It seems that many python libraries are hardcoded to look for both ipv6 and ipv4. Since ipv6 lookups are extremely slow at least for some users, perhaps the devs should carry over these optional arguments to higher level libraries like mechanize, urllib, httplib etc. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Thu Jul 7 19:33:17 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 07 Jul 2011 10:33:17 -0700 Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and codecs.StreamWriter In-Reply-To: <20110707162807.7c5932aa@pitrou.net> References: <1309999915.2958.8.camel@marge> <20110707162807.7c5932aa@pitrou.net> Message-ID: On 7/7/2011 7:28 AM, Antoine Pitrou wrote: > The main point of the PEP, IMO, is actually the deprecation itself. By > deprecating, we signal that something isn't actively maintained > anymore, and that a (allegedly better) alternative is available. > I think that's a very reasonable thing to do, regardless of whether or > not the "thing" actually gets removed in a later version. Yes, the final decision could be deprecate now, remove in 4.0, as happened during the 2.x series. From phd at phdru.name Thu Jul 7 19:59:01 2011 From: phd at phdru.name (Oleg Broytman) Date: Thu, 7 Jul 2011 21:59:01 +0400 Subject: [Python-Dev] making socket.getaddrinfo use cached dns In-Reply-To: References: Message-ID: <20110707175901.GA28561@iskra.aviel.ru> Hello. We are sorry but we cannot help you. This mailing list is to work on developing Python (adding new features to Python itself and fixing bugs); if you're having problems learning, understanding or using Python, please find another forum. Probably python-list/comp.lang.python mailing list/news group is the best place; there are Python developers who participate in it; you may get a faster, and probably more complete, answer there. See http://www.python.org/community/ for other lists/news groups/fora. Thank you for understanding. On Thu, Jul 07, 2011 at 12:57:59PM -0400, high bandwidth wrote: > I use cached dns lookups with pdnsd on my ubuntu machine to speed up web > access as regular lookups can take 15-30 seconds. However, python's > mechanize and urllib etc use socket.getaddrinfo, which seems not to be using > dns cacheing or taking a long time because of ipv6 lookups. In either case, > I subsequent access to the same site to be fast and not require lengthy > calls to getaddrinfo. How can I get python to correctly use cached dns > lookups and ipv4 only (at least in those cases where it is appropriate). > > As mentioned here: > http://stackoverflow.com/questions/2014534/force-python-mechanize-urllib2-to-only-use-a-requests/2035686#2035686 > It seems that many python libraries are hardcoded to look for both ipv6 and > ipv4. Since ipv6 lookups are extremely slow at least for some users, perhaps > the devs should carry over these optional arguments to higher level > libraries like mechanize, urllib, httplib etc. Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From victor.stinner at haypocalc.com Thu Jul 7 20:43:51 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Thu, 07 Jul 2011 20:43:51 +0200 Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and codecs.StreamWriter In-Reply-To: References: <1309999915.2958.8.camel@marge> <20110707162807.7c5932aa@pitrou.net> Message-ID: <4E15FE67.8020601@haypocalc.com> Le 07/07/2011 19:33, Terry Reedy a ?crit : > On 7/7/2011 7:28 AM, Antoine Pitrou wrote: > >> The main point of the PEP, IMO, is actually the deprecation itself. By >> deprecating, we signal that something isn't actively maintained >> anymore, and that a (allegedly better) alternative is available. >> I think that's a very reasonable thing to do, regardless of whether or >> not the "thing" actually gets removed in a later version. > > Yes, the final decision could be deprecate now, remove in 4.0, as > happened during the 2.x series. Python 4? Are you serious? Victor From brett at python.org Thu Jul 7 21:59:41 2011 From: brett at python.org (Brett Cannon) Date: Thu, 7 Jul 2011 12:59:41 -0700 Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and codecs.StreamWriter In-Reply-To: <4E15FE67.8020601@haypocalc.com> References: <1309999915.2958.8.camel@marge> <20110707162807.7c5932aa@pitrou.net> <4E15FE67.8020601@haypocalc.com> Message-ID: On Thu, Jul 7, 2011 at 11:43, Victor Stinner wrote: > Le 07/07/2011 19:33, Terry Reedy a ?crit : > > On 7/7/2011 7:28 AM, Antoine Pitrou wrote: >> >> The main point of the PEP, IMO, is actually the deprecation itself. By >>> deprecating, we signal that something isn't actively maintained >>> anymore, and that a (allegedly better) alternative is available. >>> I think that's a very reasonable thing to do, regardless of whether or >>> not the "thing" actually gets removed in a later version. >>> >> >> Yes, the final decision could be deprecate now, remove in 4.0, as happened >> during the 2.x series. >> > > Python 4? Are you serious? > Yes he is, as are others who would support that position (not me; I prefer two releases of pending deprecation, one release deprecation, then removal). When I was organizing the stdlib reorg, one viewpoint that came up was to never actually remove module code but simply deprecate it so that that those who care to use the module can continue to do so, but otherwise let it bit-rot so that pre-existing code does not necessarily break. -Brett > > Victor > > ______________________________**_________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/**mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/**mailman/options/python-dev/** > brett%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Fri Jul 8 03:16:46 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 8 Jul 2011 11:16:46 +1000 Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and codecs.StreamWriter In-Reply-To: <4E15FE67.8020601@haypocalc.com> References: <1309999915.2958.8.camel@marge> <20110707162807.7c5932aa@pitrou.net> <4E15FE67.8020601@haypocalc.com> Message-ID: On Fri, Jul 8, 2011 at 4:43 AM, Victor Stinner wrote: > Le 07/07/2011 19:33, Terry Reedy a ?crit : >> >> On 7/7/2011 7:28 AM, Antoine Pitrou wrote: >> >>> The main point of the PEP, IMO, is actually the deprecation itself. By >>> deprecating, we signal that something isn't actively maintained >>> anymore, and that a (allegedly better) alternative is available. >>> I think that's a very reasonable thing to do, regardless of whether or >>> not the "thing" actually gets removed in a later version. >> >> Yes, the final decision could be deprecate now, remove in 4.0, as happened >> during the 2.x series. > > Python 4? Are you serious? Py3k was a mythological "some time in the dim distant future" target for backwards incompatible changes for a long time before it became a real project that people were working on actually building. Py4k is now a similarly mythological beast :) However, like Brett, I don't think it's actually needed in this particular case. Deprecation in 3.3, removal in 3.5 is a time frame completely in line with the desire to avoid a repeat of the PyCObject/PyCapsule related incompatibility problems. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From sylvain.thenault at logilab.fr Fri Jul 8 15:51:19 2011 From: sylvain.thenault at logilab.fr (Sylvain =?utf-8?B?VGjDqW5hdWx0?=) Date: Fri, 8 Jul 2011 15:51:19 +0200 Subject: [Python-Dev] status of absolute_import w/ python 2.7 Message-ID: <20110708135119.GD2114@lupus.logilab.fr> Hi there, the documentation state that absolute_import feature is the default behaviour with python 2.7, though it seems that it behave differently with the __future__ import : $ cat package/__init__.py import subpackage $ python2.7 Python 2.7.1+ (default, Apr 20 2011, 22:33:39) [GCC 4.5.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import package >>> $ cat package/__init__.py from __future__ import absolute_import import subpackage $ python2.7 Python 2.7.1+ (default, Apr 20 2011, 22:33:39) [GCC 4.5.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import package Traceback (most recent call last): File "", line 1, in File "package/__init__.py", line 23, in import subpackage ImportError: No module named subpackage Maybe the doc should be fixed ? -- Sylvain Th?nault LOGILAB, Paris (France) Formations Python, Debian, M?th. Agiles: http://www.logilab.fr/formations D?veloppement logiciel sur mesure: http://www.logilab.fr/services CubicWeb, the semantic web framework: http://www.cubicweb.org From status at bugs.python.org Fri Jul 8 18:07:23 2011 From: status at bugs.python.org (Python tracker) Date: Fri, 8 Jul 2011 18:07:23 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20110708160723.22E3F1DEE9@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2011-07-01 - 2011-07-08) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 2857 ( +7) closed 21444 (+45) total 24301 (+52) Open issues with patches: 1243 Issues opened (33) ================== #10898: posixmodule.c redefines FSTAT http://bugs.python.org/issue10898 reopened by alanh #12472: Build failure on IRIX http://bugs.python.org/issue12472 opened by kais58 #12476: ctypes: need example how to pass raw data from Python http://bugs.python.org/issue12476 opened by techtonik #12478: Possible error in HTTPErrorProcessor documentation http://bugs.python.org/issue12478 opened by sandro.tosi #12479: Add HTTPErrorProcessor class definition http://bugs.python.org/issue12479 opened by sandro.tosi #12480: urllib2 doesn't use proxy (fieddler2), configed the proxy with http://bugs.python.org/issue12480 opened by Or.Wilder #12483: CThunkObject_dealloc should call PyObject_GC_UnTrack? http://bugs.python.org/issue12483 opened by rfk #12484: The Py_InitModule functions no longer exist, but remain in the http://bugs.python.org/issue12484 opened by alejolp #12485: textwrap.wrap: new argument for more pleasing output http://bugs.python.org/issue12485 opened by parent5446 #12486: tokenize module should have a unicode API http://bugs.python.org/issue12486 opened by Devin Jeanpierre #12488: multiprocessing.Connection does not communicate pipe closure b http://bugs.python.org/issue12488 opened by lcampagn #12489: email.errors.HeaderParseError if base64url is used http://bugs.python.org/issue12489 opened by guettli #12491: Update glossary documentation for the term 'attribute' http://bugs.python.org/issue12491 opened by orsenthil #12494: subprocess: check_output() doesn't close pipes on error http://bugs.python.org/issue12494 opened by haypo #12495: Rewrite InterProcessSignalTests http://bugs.python.org/issue12495 opened by haypo #12498: asyncore.dispatcher_with_send, disconnection problem + miss-co http://bugs.python.org/issue12498 opened by Fran??ois-Xavier.Bourlet #12499: textwrap.wrap: add control for fonts with different character http://bugs.python.org/issue12499 opened by parent5446 #12500: Skip test_ssl.test_connect_ex() on connection error http://bugs.python.org/issue12500 opened by haypo #12502: 100% cpu usage when using asyncore with UNIX socket http://bugs.python.org/issue12502 opened by Alexey.Agapitov #12503: "with" statement error message is more confusing in Py2.7 http://bugs.python.org/issue12503 opened by Ismael.Garrido #12506: NIS module cant handle multiple NIS map entries for the same G http://bugs.python.org/issue12506 opened by bjorn.lofdahl #12507: tkSimpleDialog problem http://bugs.python.org/issue12507 opened by rzn8tr #12508: Codecs Anomaly http://bugs.python.org/issue12508 opened by spatz123 #12509: test_gdb fails on debug build when builddir != srcdir http://bugs.python.org/issue12509 opened by dmalcolm #12510: IDLE get_the_calltip mishandles raw strings http://bugs.python.org/issue12510 opened by Roy.Fox #12512: codecs: StreamWriter issues with stateful codecs after a seek http://bugs.python.org/issue12512 opened by haypo #12513: codec.StreamReaderWriter: issues with interlaced read-write http://bugs.python.org/issue12513 opened by haypo #12514: timeit disables garbage collection if timed code raises an exc http://bugs.python.org/issue12514 opened by Gareth.Rees #12515: email modifies the message structure when the parsed email is http://bugs.python.org/issue12515 opened by xavierd #12516: imghdr.what should take one argument http://bugs.python.org/issue12516 opened by jfinkels #12517: Large file support on Windows: sizeof(off_t) is 32 bits http://bugs.python.org/issue12517 opened by haypo #12518: In string.Template it's impossible to transform delimiter in t http://bugs.python.org/issue12518 opened by py.user #12519: Call next version 3.3.0 http://bugs.python.org/issue12519 opened by eric.araujo Most recent 15 issues with no replies (15) ========================================== #12519: Call next version 3.3.0 http://bugs.python.org/issue12519 #12518: In string.Template it's impossible to transform delimiter in t http://bugs.python.org/issue12518 #12516: imghdr.what should take one argument http://bugs.python.org/issue12516 #12515: email modifies the message structure when the parsed email is http://bugs.python.org/issue12515 #12513: codec.StreamReaderWriter: issues with interlaced read-write http://bugs.python.org/issue12513 #12510: IDLE get_the_calltip mishandles raw strings http://bugs.python.org/issue12510 #12509: test_gdb fails on debug build when builddir != srcdir http://bugs.python.org/issue12509 #12506: NIS module cant handle multiple NIS map entries for the same G http://bugs.python.org/issue12506 #12498: asyncore.dispatcher_with_send, disconnection problem + miss-co http://bugs.python.org/issue12498 #12495: Rewrite InterProcessSignalTests http://bugs.python.org/issue12495 #12491: Update glossary documentation for the term 'attribute' http://bugs.python.org/issue12491 #12486: tokenize module should have a unicode API http://bugs.python.org/issue12486 #12483: CThunkObject_dealloc should call PyObject_GC_UnTrack? http://bugs.python.org/issue12483 #12479: Add HTTPErrorProcessor class definition http://bugs.python.org/issue12479 #12478: Possible error in HTTPErrorProcessor documentation http://bugs.python.org/issue12478 Most recent 15 issues waiting for review (15) ============================================= #12517: Large file support on Windows: sizeof(off_t) is 32 bits http://bugs.python.org/issue12517 #12516: imghdr.what should take one argument http://bugs.python.org/issue12516 #12514: timeit disables garbage collection if timed code raises an exc http://bugs.python.org/issue12514 #12509: test_gdb fails on debug build when builddir != srcdir http://bugs.python.org/issue12509 #12502: 100% cpu usage when using asyncore with UNIX socket http://bugs.python.org/issue12502 #12500: Skip test_ssl.test_connect_ex() on connection error http://bugs.python.org/issue12500 #12499: textwrap.wrap: add control for fonts with different character http://bugs.python.org/issue12499 #12494: subprocess: check_output() doesn't close pipes on error http://bugs.python.org/issue12494 #12485: textwrap.wrap: new argument for more pleasing output http://bugs.python.org/issue12485 #12483: CThunkObject_dealloc should call PyObject_GC_UnTrack? http://bugs.python.org/issue12483 #12479: Add HTTPErrorProcessor class definition http://bugs.python.org/issue12479 #12454: mailbox: use ASCII to read/write .mh_sequences files http://bugs.python.org/issue12454 #12452: reuse plistlib in sysconfig; deprecation process in plistlib http://bugs.python.org/issue12452 #12451: open: avoid the locale encoding when possible http://bugs.python.org/issue12451 #12448: smtplib's __main__ doesn't flush when prompting http://bugs.python.org/issue12448 Top 10 most discussed issues (10) ================================= #10181: Problems with Py_buffer management in memoryobject.c (and else http://bugs.python.org/issue10181 35 msgs #6721: Locks in python standard library should be sanitized on fork http://bugs.python.org/issue6721 10 msgs #8716: test_tk/test_tkk_guionly fails on OS X if run from buildbot sl http://bugs.python.org/issue8716 9 msgs #10898: posixmodule.c redefines FSTAT http://bugs.python.org/issue10898 8 msgs #12167: test_packaging reference leak http://bugs.python.org/issue12167 7 msgs #12411: cgi.parse_multipart is broken on 3.x http://bugs.python.org/issue12411 6 msgs #12488: multiprocessing.Connection does not communicate pipe closure b http://bugs.python.org/issue12488 6 msgs #10117: Tools/scripts/reindent.py fails on non-UTF-8 encodings http://bugs.python.org/issue10117 5 msgs #10883: urllib: socket is not closed explicitly http://bugs.python.org/issue10883 5 msgs #11732: Skip decorator for tests requiring manual intervention on Wind http://bugs.python.org/issue11732 5 msgs Issues closed (45) ================== #4673: Distutils should provide an uninstall command http://bugs.python.org/issue4673 closed by eric.araujo #6234: cgi.FieldStorage is broken when given POST data http://bugs.python.org/issue6234 closed by eric.araujo #7123: Multiprocess Process does not always exit when run from a thre http://bugs.python.org/issue7123 closed by neologix #9015: f.write(s) for s > 2GB hangs in win64 (and win32?) http://bugs.python.org/issue9015 closed by haypo #9611: FileIO not 64-bit safe under Windows http://bugs.python.org/issue9611 closed by haypo #9642: #ifdef and mbcs: don't check for defined(HAVE_USABLE_WCHAR_T) http://bugs.python.org/issue9642 closed by haypo #10403: Use "member" consistently http://bugs.python.org/issue10403 closed by python-dev #11512: adding test suite for cgitb http://bugs.python.org/issue11512 closed by brian.curtin #11859: test_interrupted_write_text() of test_io failed of Python 3.3 http://bugs.python.org/issue11859 closed by haypo #12016: Wrong behavior for '\xff\n'.decode('gb2312', 'ignore') http://bugs.python.org/issue12016 closed by haypo #12147: smtplib.send_message does not implement corectly rfc 2822 http://bugs.python.org/issue12147 closed by r.david.murray #12169: Factor out common code for d2 commands register, upload and up http://bugs.python.org/issue12169 closed by eric.araujo #12198: zipfile.py:1047: DeprecationWarning: 'H' format requires 0 <= http://bugs.python.org/issue12198 closed by petri.lehtinen #12291: file written using marshal in 3.2 can be read by 2.7, but not http://bugs.python.org/issue12291 closed by python-dev #12352: multiprocessing.Value() hangs http://bugs.python.org/issue12352 closed by neologix #12391: packaging install fails to clean up temp files http://bugs.python.org/issue12391 closed by python-dev #12412: non defined representation for pwd.struct_passwd http://bugs.python.org/issue12412 closed by eric.araujo #12423: signal handler doesn't handle SIGABRT from os.abort http://bugs.python.org/issue12423 closed by haypo #12426: packaging.tests.test_command_install_dist.InstallTestCase fail http://bugs.python.org/issue12426 closed by haypo #12429: test_io.check_interrupted_write() sporadic failures on FreeBSD http://bugs.python.org/issue12429 closed by haypo #12438: IDLE problem displaying warning message http://bugs.python.org/issue12438 closed by python-dev #12456: Hangs in concurrent.futures http://bugs.python.org/issue12456 closed by pitrou #12459: time.sleep(-1.0) behaviour http://bugs.python.org/issue12459 closed by haypo #12465: gc.get_referents can be used to crash Python http://bugs.python.org/issue12465 closed by georg.brandl #12467: test_threading.test_6_daemon_threads(): Assertion failed: PyUn http://bugs.python.org/issue12467 closed by haypo #12468: longjmp causes uninitialized stack frame http://bugs.python.org/issue12468 closed by neologix #12469: test_faulthandler failures on FreeBSD 6 http://bugs.python.org/issue12469 closed by haypo #12470: Fix cut&paste typo in test_shutil http://bugs.python.org/issue12470 closed by orsenthil #12471: wrong TypeError message on '%i' formatting http://bugs.python.org/issue12471 closed by python-dev #12473: factory func of collections.defaultdict should receive the "mi http://bugs.python.org/issue12473 closed by amaury.forgeotdarc #12474: Invalid read in symtable.c http://bugs.python.org/issue12474 closed by python-dev #12475: Generator bug allows you to chain arbitrary tracebacks to the http://bugs.python.org/issue12475 closed by python-dev #12477: input() does'nt strip CR http://bugs.python.org/issue12477 closed by georg.brandl #12481: test_faulthandler: popups on Windows/Debug/x64 http://bugs.python.org/issue12481 closed by haypo #12482: input() not working correctly on Mac OS X http://bugs.python.org/issue12482 closed by ned.deily #12487: urllib2.urlopen() returns object missing context manager http://bugs.python.org/issue12487 closed by orsenthil #12490: Documentation for itertools.chain.from_iterable is incorrect http://bugs.python.org/issue12490 closed by RodolphoEckhardt #12492: Inconsistent Python find() behavior http://bugs.python.org/issue12492 closed by georg.brandl #12493: subprocess: Popen.communicate() doesn't handle EINTR in some c http://bugs.python.org/issue12493 closed by haypo #12496: test_ssl test_connect_capath fails with "certificate verify fa http://bugs.python.org/issue12496 closed by ned.deily #12497: test_codecmaps_* skipped - Could not retrieve * http://bugs.python.org/issue12497 closed by ned.deily #12501: callable(): remove/amend the deprecation warning in Python 2.7 http://bugs.python.org/issue12501 closed by python-dev #12504: packaging: fix database to stop skipping uninstall tests on wi http://bugs.python.org/issue12504 closed by eric.araujo #12505: python interpreter not handle wildards properly http://bugs.python.org/issue12505 closed by brian.curtin #12511: class/instance xyz has no attribute '_abc' http://bugs.python.org/issue12511 closed by Andreas.Hasenkopf From mail at kerrickstaley.com Sun Jul 10 02:45:04 2011 From: mail at kerrickstaley.com (Kerrick Staley) Date: Sat, 9 Jul 2011 19:45:04 -0500 Subject: [Python-Dev] [PEPs] Support the /usr/bin/python2 symlink upstream In-Reply-To: <4D7E88BB.9030109@voidspace.org.uk> References: <20110303211140.1e2280fd@neurotica.wooz.org> <20110304040954.GE3275@unaka.lan> <20110304135723.F3307249609@kimball.webabinitio.net> <20110304101057.6b16622e@neurotica.wooz.org> <20110304133127.554716a0@neurotica.wooz.org> <4D7157EC.5090709@v.loewis.de> <4D718816.20304@skippinet.com.au> <4D723C90.6090307@voidspace.org.uk> <4D72F267.3050204@gmail.com> <4D743248.9060705@gmail.com> <4D754123.7070703@voidspace.org.uk> <4D756FCA.4070509@canterbury.ac.nz> <4D75718B.7050908@voidspace.org.uk> <4D759568.9000508@g.nevcal.com> <4D7E88BB.9030109@voidspace.org.uk> Message-ID: Sorry that I dropped the ball on this. I'd still like to see this get implemented, but I got distracted with school and forgot about it. Updates I have made to the PEP will be sent as a patch immediately after this email. Here's a summary of what was happenening when we left off: * The draft SVN version from March 4 was pretty much complete. * Some were concerned about addressing Windows, but it was generally agreed to leave the Windows issue to another PEP. PEP 397 was started on March 15 to address the Windows side of the issue. PEP 397 recommends that the Windows Python launcher read the shebang and use it to determine which Python version to use; this allows one syntax for both operating systems that is compatible with the current PEP 394 recommendation. * There were concerns from Ned Deily about the naming of other binaries such as idle, pydoc, and python-config; these need to be created as idle2, pydoc2, and python2-config, with links created at the locations of the original binaries. * There were concerns from Glenn Linderman that the shebang line doesn't encode enough information to flexibly handle Windows launching (or even launching in general). ==== Here are my thoughts: * For Ned's comments, I agree. Although the issue isn't as large with these programs, there's no reason we can't handle them in the same way. I updated the PEP. * For Glenn's comments, I think the method you propose adds too much complexity. Regardless, if the #@ syntax is implemented, it can be described in PEP 397; it won't have an impact on the contents of this PEP. I think, though, that having multiple syntaxes may cause many scripts to be incompatible with multiple platforms when they don't have to be, since Unix coders will rarely add a #@ line, and Windows coders will likely forget the #! line. Also, #@ really ignores the purpose of a shebang: shebangs simply indicate an interpreter that works with the script; the shebang allows users to run arbitrary scripts without worrying about which interpreter they should specify. There's no reason that a script should use one interpreter on Unix, but be incompatible with that interpreter on Windows yet compatible with another. The name of the Unix binary is enough to determine the implementation and version of the interpreter to be used on Unix, and a Windows launcher should always invoke the same implementation/version on Windows (and this won't require hard-coding anything into the launcher if done right). If you want the script to run with a specific interpreter and version, possibly contingent on which operating system you're running the script under, then you can just invoke the interpreter by name with the script as an argument (e.g. python3 myprogram.py). TL;DR: shebangs encode a default implementation/version, and if you need something special, you can just manually run python3 myprogram.py or use a .bat file. ==== Also, I updated the PEP with the clarification that commands like python3 should be hard links (because they'll be invoked from code and are more efficient; also, hard links are just as flexible as symlinks here), while commands like python should be soft links (because this makes it clear to sysadmins that they can be "switched", and it's needed for flexibility if python3 changes). This really doesn't matter, but can we keep it this way unless there are serious objections? Regards, Kerrick Staley From mail at kerrickstaley.com Sun Jul 10 02:46:09 2011 From: mail at kerrickstaley.com (Kerrick Staley) Date: Sat, 9 Jul 2011 19:46:09 -0500 Subject: [Python-Dev] [PEPs] Support the /usr/bin/python2 symlink upstream In-Reply-To: References: <20110303211140.1e2280fd@neurotica.wooz.org> <20110304040954.GE3275@unaka.lan> <20110304135723.F3307249609@kimball.webabinitio.net> <20110304101057.6b16622e@neurotica.wooz.org> <20110304133127.554716a0@neurotica.wooz.org> <4D7157EC.5090709@v.loewis.de> <4D718816.20304@skippinet.com.au> <4D723C90.6090307@voidspace.org.uk> <4D72F267.3050204@gmail.com> <4D743248.9060705@gmail.com> <4D754123.7070703@voidspace.org.uk> <4D756FCA.4070509@canterbury.ac.nz> <4D75718B.7050908@voidspace.org.uk> <4D759568.9000508@g.nevcal.com> <4D7E88BB.9030109@voidspace.org.uk> Message-ID: $ svn diff Index: pep-0394.txt =================================================================== --- pep-0394.txt (revision 88860) +++ pep-0394.txt (working copy) @@ -1,5 +1,5 @@ PEP: 394 -Title: The "python" command on Unix-Like Systems +Title: The "python" Command on Unix-Like Systems Version: $Revision$ Last-Modified: $Date$ Author: Kerrick Staley , @@ -53,10 +53,14 @@ * When reinvoking the interpreter from a Python script, querying ``sys.executable`` to avoid hardcoded assumptions regarding the interpreter location remains the preferred approach. +* The ``idle``, ``pydoc``, and ``python-config`` binaries from Python 2.0 +should likewise be available as ``idle2``, ``pydoc2``, and ``python2-config``, +with the original commands invoking these binaries by default, but possibly +invoking the Python 3.0 versions instead. These recommendations are the outcome of the relevant python-dev discussion in -March 2011 [1] (NOTE: More accurately, they will be such once that "Draft" -status disappears from the PEP header, it has been moved into the "Other +March to July 2011 [1] (NOTE: More accurately, they will be such once that +"Draft" status disappears from the PEP header, it has been moved into the "Other Informational PEP" section in PEP 0 and this note has been deleted) @@ -144,11 +148,16 @@ While technically a new feature, the ``make install`` command and the Mac OS X installer in the 2.7 version of CPython will be adjusted to create the -new ``python2`` command in addition to the existing ``python`` and -``python2.7`` commands. This feature will first appear in CPython 2.7.2. +``python2.7``, ``idle2.7``, ``pydoc2.7``, and ``python2.7-config`` binaries, +with ``python2``, ``idle2``, ``pydoc2``, and ``python2-config`` as hard links +to the respective binaries, and ``python``, ``idle``, ``pydoc``, and +``python-config`` as symbolic links to the respective hard links. This feature +will first appear in CPython 2.7.2. -The ``make install`` command in the CPython 3.x series will continue to -install only the ``python3`` symlink for the foreseeable future. +The ``make install`` command in the CPython 3.x series will similarly install +the ``python3.x``, ``idle3.x``, ``pydoc3.x``, and ``python3.x-config`` binaries +(with appropriate ``x``), and ``python3``, ``idle3``, ``pydoc3``, and +``python3-config`` as hard links. Impact on PYTHON* Environment Variables @@ -166,27 +175,12 @@ Exclusions of MS Windows ======================== -This PEP deliberately excludes any proposals relating to Microsoft Windows. -The use of parallel installs on Windows suffers from numerous issues, -including the "last installed wins" behaviour for handling of file -associations, a lack of universal robust symlink support for easy aliasing of -commands, the fact that the Python executable is not available on ``PATH`` by -default, the fact that the ``python.exe`` and ``pythonw.exe`` names are -used for both Python 2 and Python 3 binaries and the lack of distinction -between the different Python versions when the Start menu shortcuts are -divorced from their containing folders (e.g. when they appear in the -"Recently Used" list. +This PEP deliberately excludes any proposals relating to Microsoft Windows: +devising an equivalent solution for Windows was deemed too complex to handle +here. PEP 397 and the related discussion on the python-dev mailing list address +this issue. -While these questions are well worth addressing, they do not have easy -answers. The authors of this particular PEP aren't inclined to even begin -trying to answer them, but anyone that wants to tackle them should feel free -to start working on their own PEP. -Note that, while the topic has been excluded from this PEP, there is plenty of -material in the linked python-dev discussion that may be useful in the design -and implementation of a Windows-specific solution. - - References ========== From riscutiavlad at gmail.com Sun Jul 10 21:54:46 2011 From: riscutiavlad at gmail.com (Vlad Riscutia) Date: Sun, 10 Jul 2011 12:54:46 -0700 Subject: [Python-Dev] ctypes: Configurable bitfield allocation strategy In-Reply-To: References: <4E067121.9070802@canterbury.ac.nz> Message-ID: Opened http://bugs.python.org/issue12528 to track this. Thank you, Vlad On Sun, Jun 26, 2011 at 5:48 PM, Vlad Riscutia wrote: > Well it's not really layout, because alignment is handled by pack option. > It is how the field gets allocated. At this point I believe it will be more > complex to come up with custom allocation option, precisely because it's up > to each compiler to allocate the structure. Such flexibility will add a lot > of complexity and it might not payoff. This is debatable, but I would go > with a 3 option strategy at this point - native, GCC, MSVC and if we > actually find interop issues with other compilers we can look into custom > allocation. > > I will try to refactor the C code to more easily accommodate a mode option > (while leaving behavior the same for now), then we can decide how the > library interface should look like. > > Thank you, > Vlad > > On Sat, Jun 25, 2011 at 4:37 PM, Greg Ewing wrote: > >> Vlad Riscutia wrote: >> >>> Longer term though, I think it would be better to add a property on the >>> Structure class for configurable allocation strategy, for example Native >>> (default), GCC, MSVC >>> >> >> It could also be good to have a mode which lets you specify >> *exactly* how the bits are laid out, independent of any >> particular compiler. >> >> -- >> Greg >> >> ______________________________**_________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/**mailman/listinfo/python-dev >> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/** >> riscutiavlad%40gmail.com >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From georg at python.org Mon Jul 11 07:23:32 2011 From: georg at python.org (Georg Brandl) Date: Mon, 11 Jul 2011 07:23:32 +0200 Subject: [Python-Dev] [RELEASED] Python 3.2.1 Message-ID: <4E1A88D4.5070704@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On behalf of the Python development team, I am pleased to announce the final release of Python 3.2.1. Python 3.2.1 is the first bugfix release for Python 3.2, fixing over 120 bugs and regressions in Python 3.2. For an extensive list of changes and features in the 3.2 line, see http://docs.python.org/3.2/whatsnew/3.2.html To download Python 3.2.1 visit: http://www.python.org/download/releases/3.2.1/ This is a final release: Please report any bugs you may notice to: http://bugs.python.org/ Enjoy! - -- Georg Brandl, Release Manager georg at python.org (on behalf of the entire python-dev team and 3.2's contributors) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (GNU/Linux) iEYEARECAAYFAk4aiNMACgkQN9GcIYhpnLDofwCglfgDQ1/B/TxxwfqtDxK13ksz micAn0CVWmNNaYE2a6z0N7+Dz+hCZSj1 =7Mia -----END PGP SIGNATURE----- From martin at v.loewis.de Mon Jul 11 09:47:01 2011 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 11 Jul 2011 09:47:01 +0200 Subject: [Python-Dev] ctypes: Configurable bitfield allocation strategy In-Reply-To: References: Message-ID: <4E1AAA75.7010903@v.loewis.de> Am 25.06.2011 18:33, schrieb Vlad Riscutia: > I would like to hear some opinions on this. Sounds fine to me in principle. Warnings can be added to the documentation at any time; please submit a patch to the tracker. As for the API change - make sure you post your proposed API to the list before spending time to implement it. Regards, Martin From mfoord at python.org Mon Jul 11 16:36:49 2011 From: mfoord at python.org (Michael Foord) Date: Mon, 11 Jul 2011 15:36:49 +0100 Subject: [Python-Dev] Fwd: Dead link to documentation on Python main page In-Reply-To: <2EF26144-B59B-4EF1-9F65-AB623C421CE2@59bits.nl> References: <2EF26144-B59B-4EF1-9F65-AB623C421CE2@59bits.nl> Message-ID: <4E1B0A81.9040702@python.org> -------- Original Message -------- Subject: Dead link to documentation on Python main page Date: Mon, 11 Jul 2011 09:27:57 -0500 From: Ren? van Oostrum To: webmaster at python.org Hi, http://www.python.org -> Quick Links (3.2.1), Documentation points to http://http://docs.python.org/3.2.1/, which does not exist. Best regards, Ren? van Oostrum -------------- next part -------------- An HTML attachment was scrubbed... URL: From riscutiavlad at gmail.com Mon Jul 11 17:59:08 2011 From: riscutiavlad at gmail.com (Vlad Riscutia) Date: Mon, 11 Jul 2011 08:59:08 -0700 Subject: [Python-Dev] ctypes: Configurable bitfield allocation strategy In-Reply-To: <4E1AAA75.7010903@v.loewis.de> References: <4E1AAA75.7010903@v.loewis.de> Message-ID: Actually I already attached patch implementing everything to the issue (not too much time spent :)). I'm hoping someone can review. Thank you, Vlad On Mon, Jul 11, 2011 at 12:47 AM, "Martin v. L?wis" wrote: > Am 25.06.2011 18:33, schrieb Vlad Riscutia: > > I would like to hear some opinions on this. > > Sounds fine to me in principle. Warnings can be added to the > documentation at any time; please submit a patch to the tracker. > As for the API change - make sure you post your proposed API > to the list before spending time to implement it. > > Regards, > Martin > -------------- next part -------------- An HTML attachment was scrubbed... URL: From merwok at netwok.org Tue Jul 12 15:30:30 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Tue, 12 Jul 2011 15:30:30 +0200 Subject: [Python-Dev] [Python-checkins] cpython: Fix syntax in packaging docs and update suspicious ignore file. In-Reply-To: References: Message-ID: <4E1C4C76.8040504@netwok.org> > changeset: 71283:3a4b983dd70b > user: Georg Brandl > summary: > Fix syntax in packaging docs and update suspicious ignore file. > > files: > Doc/library/packaging.compiler.rst | 2 +- > Doc/packaging/builtdist.rst | 2 +- > Doc/packaging/commandref.rst | 2 +- Thanks for the fixes, Georg. I fix problems when I get warnings, but these fell through. Maybe you used a special command line to find those? > Doc/tools/sphinxext/susp-ignored.csv | 27 ++++++++++++++- I?ve always wondered about that file?s role, and I?ve finally found answers in Doc/tools/sphinxext/suspicious.py. Should we update this file when we get suspicious output? Does using ?.. block-code:: none? prevent program output from appearing as suspicious? Regards From g.brandl at gmx.net Tue Jul 12 19:59:50 2011 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 12 Jul 2011 19:59:50 +0200 Subject: [Python-Dev] [Python-checkins] cpython: Fix syntax in packaging docs and update suspicious ignore file. In-Reply-To: <4E1C4C76.8040504@netwok.org> References: <4E1C4C76.8040504@netwok.org> Message-ID: Am 12.07.2011 15:30, schrieb ?ric Araujo: >> changeset: 71283:3a4b983dd70b >> user: Georg Brandl >> summary: >> Fix syntax in packaging docs and update suspicious ignore file. >> >> files: >> Doc/library/packaging.compiler.rst | 2 +- >> Doc/packaging/builtdist.rst | 2 +- >> Doc/packaging/commandref.rst | 2 +- > > Thanks for the fixes, Georg. I fix problems when I get warnings, but > these fell through. Maybe you used a special command line to find those? Yes: "make suspicious" in the Doc directory invokes the special builder that looks for things that look like reST markup in the output, where it shouldn't occur. This is a routine step in PEP 101, so it will be done before every release. >> Doc/tools/sphinxext/susp-ignored.csv | 27 ++++++++++++++- > > I?ve always wondered about that file?s role, and I?ve finally found > answers in Doc/tools/sphinxext/suspicious.py. Should we update this > file when we get suspicious output? Does using ?.. block-code:: none? > prevent program output from appearing as suspicious? No (and neither do normal code blocks). That is one thing that should be improved in the builder: the content of code blocks ("literal" nodes in docutils) shouldn't be checked. Georg From brett at python.org Wed Jul 13 05:41:28 2011 From: brett at python.org (Brett Cannon) Date: Tue, 12 Jul 2011 20:41:28 -0700 Subject: [Python-Dev] status of absolute_import w/ python 2.7 In-Reply-To: <20110708135119.GD2114@lupus.logilab.fr> References: <20110708135119.GD2114@lupus.logilab.fr> Message-ID: On Fri, Jul 8, 2011 at 06:51, Sylvain Th?nault wrote: > Hi there, > > the documentation state that absolute_import feature is the default > behaviour with python 2.7, though it seems that it behave differently > with the __future__ import : > > $ cat package/__init__.py > > import subpackage > > $ python2.7 > Python 2.7.1+ (default, Apr 20 2011, 22:33:39) > [GCC 4.5.2] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import package > >>> > > $ cat package/__init__.py > > from __future__ import absolute_import > import subpackage > > $ python2.7 > Python 2.7.1+ (default, Apr 20 2011, 22:33:39) > [GCC 4.5.2] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import package > Traceback (most recent call last): > File "", line 1, in > File "package/__init__.py", line 23, in > import subpackage > ImportError: No module named subpackage > So are you claiming that the import of 'package' w/o the __future__ statement actually succeeds even though there is no package.subpackage module? Obviously that would be a flat-out bug, but I just double-checked my sanity and that does nto work with a CPython 2.7 checkout. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Wed Jul 13 06:40:50 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 13 Jul 2011 14:40:50 +1000 Subject: [Python-Dev] status of absolute_import w/ python 2.7 In-Reply-To: References: <20110708135119.GD2114@lupus.logilab.fr> Message-ID: On Wed, Jul 13, 2011 at 1:41 PM, Brett Cannon wrote: > So are you claiming that the import of 'package' w/o the __future__ > statement actually succeeds even though there is no package.subpackage > module? Obviously that would be a flat-out bug, but I just double-checked my > sanity and that does nto work with a CPython 2.7 checkout. No, the problem is that __future__.absolute_import claims to be the default behaviour in 2.7, but this does not appear to actually be the case - it still tries the implicit relative import. E.g, given the following setup: cwd /package /__init__.py /submodule.py /submodule2.py an "import submodule" in __init__.py or submodule2.py will succeed. Now, the what's new for 2.7 doesn't actually *say* we made that change and I can't find any evidence for it in NEWS either, so I think the bug is actually in the __future__ module (and docs: http://docs.python.org/library/__future__). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ulrich.eckhardt at dominolaser.com Wed Jul 13 10:25:46 2011 From: ulrich.eckhardt at dominolaser.com (Ulrich Eckhardt) Date: Wed, 13 Jul 2011 10:25:46 +0200 Subject: [Python-Dev] [Python-checkins] cpython: no one passes NULL here (or should anyway) In-Reply-To: References: Message-ID: <201107131025.46685.ulrich.eckhardt@dominolaser.com> On Monday 04 July 2011, Benjamin Peterson wrote: > If someone's static analysis tool starts complaining about it, I'd be > happy to consider adding an assert... Here is one! To me an assertion is what actually documents that you don't expect NULL in this context and that the missing check is not an oversight. +1 for assert() Uli ************************************************************************************** Domino Laser GmbH, Fangdieckstra?e 75a, 22547 Hamburg, Deutschland Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Visit our website at http://www.dominolaser.com ************************************************************************************** Diese E-Mail einschlie?lich s?mtlicher Anh?nge ist nur f?r den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empf?nger sein sollten. Die E-Mail ist in diesem Fall zu l?schen und darf weder gelesen, weitergeleitet, ver?ffentlicht oder anderweitig benutzt werden. E-Mails k?nnen durch Dritte gelesen werden und Viren sowie nichtautorisierte ?nderungen enthalten. Domino Laser GmbH ist f?r diese Folgen nicht verantwortlich. ************************************************************************************** From barry at python.org Wed Jul 13 15:31:25 2011 From: barry at python.org (Barry Warsaw) Date: Wed, 13 Jul 2011 09:31:25 -0400 Subject: [Python-Dev] status of absolute_import w/ python 2.7 In-Reply-To: References: <20110708135119.GD2114@lupus.logilab.fr> Message-ID: <20110713093125.247ed4ea@resist.wooz.org> On Jul 13, 2011, at 02:40 PM, Nick Coghlan wrote: >Now, the what's new for 2.7 doesn't actually *say* we made that change >and I can't find any evidence for it in NEWS either, so I think the >bug is actually in the __future__ module (and docs: >http://docs.python.org/library/__future__). I think that's right. The change was not made for 2.7. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From merwok at netwok.org Wed Jul 13 16:34:54 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Wed, 13 Jul 2011 16:34:54 +0200 Subject: [Python-Dev] status of absolute_import w/ python 2.7 In-Reply-To: References: <20110708135119.GD2114@lupus.logilab.fr> Message-ID: <4E1DAD0E.8020101@netwok.org> Le 13/07/2011 06:40, Nick Coghlan a ?crit : > Now, the what's new for 2.7 doesn't actually *say* we made that change > and I can't find any evidence for it in NEWS either, so I think the > bug is actually in the __future__ module (and docs: > http://docs.python.org/library/__future__). I seemed to recall the change was done in 2.6, but I found only that: > C API: the PyImport_Import() and PyImport_ImportModule() functions > now default to absolute imports, not relative imports. This will > affect C extensions that import other modules. http://docs.python.org/dev/whatsnew/2.6#porting-to-python-2-6 Regards From thinke365 at gmail.com Thu Jul 14 03:02:01 2011 From: thinke365 at gmail.com (smith jack) Date: Thu, 14 Jul 2011 09:02:01 +0800 Subject: [Python-Dev] Failed to install PIL on windows, may have something to do with python implementaton Message-ID: I am a programmer with more than two years of python programming experience, often the debugging process may led me to the details of some details of python implementation, this is exciting : ) Now i face a problem, that is i want to install PIL on windows, but failed, the error messages tells that the python installation cannot find python information in the windows register, lol, this is true, since i got a zipped python and extract it the a directory, not using a msi file to install python. So how should i do in order to install PIL for my python environment? I find that PIL installation file is a zipped file with execution capability (is it of 7z format?) Now i can open this exe file using 7z, and there are lots of .py files there, with directory somewhat as follows: PLATLIB PIL\*.py PIL.pth (a plaintext file with content "PIL") SCRIPTS *.py so can i just extrat the py files to my python installation directory? then will PIL work correctly? (should i extract PIL\*.py to python_24\Lib\site-packages, and extract *.py to python_24\Scripts?) Will this work? since this have something to do with the implementation of Python, i come to this maillist and ask for help..... Hope i can receive some suggestions, Thank you :) From rdmurray at bitdance.com Thu Jul 14 03:35:10 2011 From: rdmurray at bitdance.com (R. David Murray) Date: Wed, 13 Jul 2011 21:35:10 -0400 Subject: [Python-Dev] Failed to install PIL on windows, may have something to do with python implementaton In-Reply-To: References: Message-ID: <20110714013511.BDF04B14005@webabinitio.net> On Thu, 14 Jul 2011 09:02:01 +0800, smith jack wrote: > Will this work? since this have something to do with the > implementation of Python, i come to this maillist and ask for > help..... Hope i can receive some suggestions, Thank you :) Yes, but this list is about the *development* of Python. For help on topic like this, which are about using Python, you'll have better luck getting an answer if you post to python-list. There are also probably a lot more people with experience using Python on windows on that list than there are here. Since we release Python as an MSI, we're not too likely to know how to work with it on Windows if you don't install it from the MSI. -- R. David Murray http://www.bitdance.com From ethan at stoneleaf.us Thu Jul 14 07:41:05 2011 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 13 Jul 2011 22:41:05 -0700 Subject: [Python-Dev] Failed to install PIL on windows, may have something to do with python implementaton In-Reply-To: References: Message-ID: <4E1E8171.1040809@stoneleaf.us> smith jack wrote: > i want to install PIL on windows, but failed Python-Dev is for discussion of developing the next release of Python. This question should go to python-list, as your last question did. Good luck. ~Ethan~ From g.brandl at gmx.net Thu Jul 14 13:33:32 2011 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 14 Jul 2011 13:33:32 +0200 Subject: [Python-Dev] cpython: Issue #12550: regrtest displays the Python traceback on SIGALRM or SIGUSR1 In-Reply-To: References: Message-ID: Would it make sense to not propagate the signal in one case (e.g. SIGUSR1), i.e. just display the traceback in this case? Georg Am 13.07.2011 23:49, schrieb victor.stinner: > http://hg.python.org/cpython/rev/30f91fbfc8b3 > changeset: 71315:30f91fbfc8b3 > user: Victor Stinner > date: Wed Jul 13 23:47:21 2011 +0200 > summary: > Issue #12550: regrtest displays the Python traceback on SIGALRM or SIGUSR1 > > files: > Lib/test/regrtest.py | 12 +++++++++++- > 1 files changed, 11 insertions(+), 1 deletions(-) > > > diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py > --- a/Lib/test/regrtest.py > +++ b/Lib/test/regrtest.py > @@ -172,6 +172,7 @@ > import platform > import random > import re > +import signal > import sys > import sysconfig > import tempfile > @@ -266,9 +267,18 @@ > on the command line. > """ > > - # Display the Python traceback fatal errors (e.g. segfault) > + # Display the Python traceback on fatal errors (e.g. segfault) > faulthandler.enable(all_threads=True) > > + # Display the Python traceback on SIGALRM or SIGUSR1 signal > + signals = [] > + if hasattr(signal, 'SIGALRM'): > + signals.append(signal.SIGALRM) > + if hasattr(signal, 'SIGUSR1'): > + signals.append(signal.SIGUSR1) > + for signum in signals: > + faulthandler.register(signum, chain=True) > + > replace_stdout() > > support.record_original_stdout(sys.stdout) > > > > > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins From ezio.melotti at gmail.com Thu Jul 14 14:57:48 2011 From: ezio.melotti at gmail.com (Ezio Melotti) Date: Thu, 14 Jul 2011 15:57:48 +0300 Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default): Merge with 3.2. In-Reply-To: References: Message-ID: <4E1EE7CC.7070906@gmail.com> An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Thu Jul 14 16:42:10 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 14 Jul 2011 16:42:10 +0200 Subject: [Python-Dev] PEP 3151 implementation Message-ID: <20110714164210.4b9fc93f@pitrou.net> Hello, Just a quick note that I have a working PEP 3151 implementation at http://hg.python.org/features/pep-3151/, in named branch "pep-3151". The implementation is (should be) complete on the feature side, so you can play with it right now. It still lacks a deprecation mechanism for old names. It is tracked in http://bugs.python.org/issue12555. Regards Antoine. From benjamin at python.org Thu Jul 14 19:55:08 2011 From: benjamin at python.org (Benjamin Peterson) Date: Thu, 14 Jul 2011 12:55:08 -0500 Subject: [Python-Dev] [Python-checkins] cpython (3.1): Issue #12502: asyncore: fix polling loop with AF_UNIX sockets. In-Reply-To: References: Message-ID: 2011/7/14 charles-francois.natali : > http://hg.python.org/cpython/rev/42ec507815d2 > changeset: ? 71334:42ec507815d2 > branch: ? ? ?3.1 > parent: ? ? ?71126:0d4ca1e77205 > user: ? ? ? ?Charles-Fran?ois Natali > date: ? ? ? ?Thu Jul 14 19:53:38 2011 +0200 > summary: > ?Issue #12502: asyncore: fix polling loop with AF_UNIX sockets. Please do not apply bugfixes to 3.1. -- Regards, Benjamin From cf.natali at gmail.com Thu Jul 14 20:05:51 2011 From: cf.natali at gmail.com (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=) Date: Thu, 14 Jul 2011 20:05:51 +0200 Subject: [Python-Dev] error upon Mercurial commit Message-ID: Hello, I just committed a patch, and got the following error: """ $ hg push pushing to ssh://hg at hg.python.org/cpython searching for changes remote: adding changesets remote: adding manifests remote: adding file changes remote: added 1 changesets with 2 changes to 2 files remote: change(s) NOT sent, something went wrong: remote: [Failure instance: Traceback from remote host -- Traceback unavailable remote: ] remote: sent email to roundup at report at bugs.python.org remote: notified python-checkins at python.org of incoming changeset ca077f2672e3 """ But the change seems to have been committed successfully. I get this on every branch. Am I the only one in this case? Any idea of what's going on here? From g.brandl at gmx.net Thu Jul 14 21:56:32 2011 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 14 Jul 2011 21:56:32 +0200 Subject: [Python-Dev] error upon Mercurial commit In-Reply-To: References: Message-ID: Am 14.07.2011 20:05, schrieb Charles-Fran?ois Natali: > Hello, > > I just committed a patch, and got the following error: > > """ > $ hg push > pushing to ssh://hg at hg.python.org/cpython > searching for changes > remote: adding changesets > remote: adding manifests > remote: adding file changes > remote: added 1 changesets with 2 changes to 2 files > remote: change(s) NOT sent, something went wrong: > remote: [Failure instance: Traceback from remote host -- Traceback unavailable > remote: ] > remote: sent email to roundup at report at bugs.python.org > remote: notified python-checkins at python.org of incoming changeset ca077f2672e3 > """ > > But the change seems to have been committed successfully. > I get this on every branch. > Am I the only one in this case? Any idea of what's going on here? That is just output from the buildbot hook. The changes have been pushed (not committed, that is done locally on your computer), just that hook failed to execute (I don't know why.) Georg From victor.stinner at haypocalc.com Thu Jul 14 22:03:25 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Thu, 14 Jul 2011 22:03:25 +0200 Subject: [Python-Dev] cpython: Issue #12550: regrtest displays the Python traceback on SIGALRM or SIGUSR1 In-Reply-To: References: Message-ID: <4E1F4B8D.8030508@haypocalc.com> Le 14/07/2011 13:33, Georg Brandl a ?crit : > Would it make sense to not propagate the signal in one case (e.g. SIGUSR1), > i.e. just display the traceback in this case? > I opened this issue for buildbots. If the test suite doesn't fail anymore because of a SIGALRM or SIGUSR1, we may miss a bug in a test, or a real bug in a module. I don't think that anyone will read the output of all builds. I patched faulthandler for this issue: I added a chain argument to faulthandler.register() to call also the previous signal handler. I prefer to call the original signal handler to not change the current behaviour of the test suite. Victor From victor.stinner at haypocalc.com Thu Jul 14 22:33:09 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Thu, 14 Jul 2011 22:33:09 +0200 Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default): Merge with 3.2. In-Reply-To: <4E1EE7CC.7070906@gmail.com> References: <4E1EE7CC.7070906@gmail.com> Message-ID: <4E1F5285.7020906@haypocalc.com> Done: changeset: 71337:66e519792e4c tag: tip user: Victor Stinner date: Thu Jul 14 22:28:36 2011 +0200 files: Lib/cgi.py Lib/test/test_cgi.py Misc/NEWS description: Add cgi.closelog() function to close the log file Le 14/07/2011 14:57, Ezio Melotti a ?crit : > >> diff --git a/Lib/test/test_cgi.py b/Lib/test/test_cgi.py >> --- a/Lib/test/test_cgi.py >> +++ b/Lib/test/test_cgi.py >> @@ -155,7 +155,13 @@ >> cgi.logfp = None >> cgi.logfile = "/dev/null" >> cgi.initlog("%s", "Testing log 3") >> - self.addCleanup(cgi.logfp.close) >> + def log_cleanup(): >> + """Restore the global state of the log vars.""" >> + cgi.logfile = '' >> + cgi.logfp.close() >> + cgi.logfp = None >> + cgi.log = cgi.initlog > > > It was suggested (on #python-dev) to move this function to the cgi > module itself, but since I'm not familiar with it I just added it here > in order to fix a failure in the test. > > The cgi module has two global vars (logfile and logfp) and a global > function (log) that is initialized to initlog and then reassigned to > either dolog or nolog (a dummy function that does nothing) in initlog > itself[0]. > > If someone thinks the log_cleanup function should be moved to the > cgi.py module and/or the code be refactored a bit, he can do it or > open an issue. > > [0]: http://hg.python.org/cpython/file/ac1c3291a689/Lib/cgi.py#l50 > > Best Regards, From status at bugs.python.org Fri Jul 15 18:07:26 2011 From: status at bugs.python.org (Python tracker) Date: Fri, 15 Jul 2011 18:07:26 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20110715160726.E2CC81CFD5@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2011-07-08 - 2011-07-15) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 2880 (+23) closed 21474 (+30) total 24354 (+53) Open issues with patches: 1254 Issues opened (36) ================== #2506: Add mechanism to disable optimizations http://bugs.python.org/issue2506 reopened by exarkun #6755: Patch: new method get_wch for ncurses bindings: accept wide ch http://bugs.python.org/issue6755 reopened by pitrou #10403: Use "member" consistently http://bugs.python.org/issue10403 reopened by eric.araujo #12417: Inappropriate copyright on profile files http://bugs.python.org/issue12417 reopened by eric.araujo #12520: spurious output in test_warnings http://bugs.python.org/issue12520 opened by pitrou #12521: IDLE 3.2 crashes on Mac OS 10.6 with ActiveState Tcl/Tk 8.5 http://bugs.python.org/issue12521 opened by almccann #12523: 'str' object has no attribute 'more' [/usr/lib/python3.2/async http://bugs.python.org/issue12523 opened by Gry #12524: change httplib docs POST example http://bugs.python.org/issue12524 opened by georg.brandl #12526: packaging.pypi.Crawler and resulting objects have a circular A http://bugs.python.org/issue12526 opened by michael.mulich #12527: assertRaisesRegex doc'd with 'msg' arg, but it's not implement http://bugs.python.org/issue12527 opened by Brian.Jones #12528: Implement configurable bitfield allocation strategy http://bugs.python.org/issue12528 opened by vladris #12529: cgi.parse_header fails on double quotes and semicolons http://bugs.python.org/issue12529 opened by Ben.Darnell #12531: documentation index entries for * and ** http://bugs.python.org/issue12531 opened by petere #12532: PyPI server index names with '/' in them http://bugs.python.org/issue12532 opened by michael.mulich #12533: python-celementtree prevents me from running python develop.py http://bugs.python.org/issue12533 opened by Kristoffer.Grundstr??m #12535: Chained tracebacks are confusing because the first traceback i http://bugs.python.org/issue12535 opened by r.david.murray #12537: mailbox's _become_message is very fragile http://bugs.python.org/issue12537 opened by r.david.murray #12540: "Restart Shell" command leaves pythonw.exe processes running http://bugs.python.org/issue12540 opened by Peter.Caven #12541: Accepting Badly formed headers in urllib HTTPBasicAuth http://bugs.python.org/issue12541 opened by Alex.Leon #12542: Remove duplicates of cmp_to_key used in tests http://bugs.python.org/issue12542 opened by eric.araujo #12545: Incorrect handling of return codes in the posix_lseek function http://bugs.python.org/issue12545 opened by Kuberan.Naganathan #12546: str.format cannot fill with \x00 char http://bugs.python.org/issue12546 opened by Gavin.Andresen #12551: Provide data for TLS channel binding http://bugs.python.org/issue12551 opened by Jajcus #12553: email should default to 8bit CTE unless policy.must_be_7bit is http://bugs.python.org/issue12553 opened by r.david.murray #12555: PEP 3151 implementation http://bugs.python.org/issue12555 opened by pitrou #12556: Disable size checks in mmap.mmap() http://bugs.python.org/issue12556 opened by superbobry #12558: Locale-dependent crash for float width argument to Tkinter wid http://bugs.python.org/issue12558 opened by hans.bering #12559: gzip.open() needs an optional encoding argument http://bugs.python.org/issue12559 opened by rhettinger #12560: libpython.so not built on OpenBSD http://bugs.python.org/issue12560 opened by stsp #12561: Compiler workaround for wide string constants in Modules/getpa http://bugs.python.org/issue12561 opened by jschneid #12562: calling mmap twice fails on Windows http://bugs.python.org/issue12562 opened by zolnie #12567: curses implementation of Unicode is wrong in Python 3 http://bugs.python.org/issue12567 opened by haypo #12568: Add functions to get the width in columns of a character http://bugs.python.org/issue12568 opened by haypo #12570: BaseHTTPServer.shutdown() locks if the last request 404'd http://bugs.python.org/issue12570 opened by Philip.Horger #12571: Python built on Linux 3.0 doesn't include DLFCN http://bugs.python.org/issue12571 opened by djc #12572: HP/UX compiler workarounds http://bugs.python.org/issue12572 opened by jschneid Most recent 15 issues with no replies (15) ========================================== #12568: Add functions to get the width in columns of a character http://bugs.python.org/issue12568 #12562: calling mmap twice fails on Windows http://bugs.python.org/issue12562 #12560: libpython.so not built on OpenBSD http://bugs.python.org/issue12560 #12556: Disable size checks in mmap.mmap() http://bugs.python.org/issue12556 #12553: email should default to 8bit CTE unless policy.must_be_7bit is http://bugs.python.org/issue12553 #12542: Remove duplicates of cmp_to_key used in tests http://bugs.python.org/issue12542 #12541: Accepting Badly formed headers in urllib HTTPBasicAuth http://bugs.python.org/issue12541 #12540: "Restart Shell" command leaves pythonw.exe processes running http://bugs.python.org/issue12540 #12532: PyPI server index names with '/' in them http://bugs.python.org/issue12532 #12531: documentation index entries for * and ** http://bugs.python.org/issue12531 #12526: packaging.pypi.Crawler and resulting objects have a circular A http://bugs.python.org/issue12526 #12520: spurious output in test_warnings http://bugs.python.org/issue12520 #12515: email modifies the message structure when the parsed email is http://bugs.python.org/issue12515 #12513: codec.StreamReaderWriter: issues with interlaced read-write http://bugs.python.org/issue12513 #12506: NIS module cant handle multiple NIS map entries for the same G http://bugs.python.org/issue12506 Most recent 15 issues waiting for review (15) ============================================= #12572: HP/UX compiler workarounds http://bugs.python.org/issue12572 #12567: curses implementation of Unicode is wrong in Python 3 http://bugs.python.org/issue12567 #12561: Compiler workaround for wide string constants in Modules/getpa http://bugs.python.org/issue12561 #12560: libpython.so not built on OpenBSD http://bugs.python.org/issue12560 #12559: gzip.open() needs an optional encoding argument http://bugs.python.org/issue12559 #12555: PEP 3151 implementation http://bugs.python.org/issue12555 #12551: Provide data for TLS channel binding http://bugs.python.org/issue12551 #12546: str.format cannot fill with \x00 char http://bugs.python.org/issue12546 #12542: Remove duplicates of cmp_to_key used in tests http://bugs.python.org/issue12542 #12531: documentation index entries for * and ** http://bugs.python.org/issue12531 #12528: Implement configurable bitfield allocation strategy http://bugs.python.org/issue12528 #12517: Large file support on Windows: sizeof(off_t) is 32 bits http://bugs.python.org/issue12517 #12516: imghdr.what should take one argument http://bugs.python.org/issue12516 #12514: timeit disables garbage collection if timed code raises an exc http://bugs.python.org/issue12514 #12509: test_gdb fails on debug build when builddir != srcdir http://bugs.python.org/issue12509 Top 10 most discussed issues (10) ================================= #8668: Packaging: add a 'develop' command http://bugs.python.org/issue8668 14 msgs #3177: Add shutil.open http://bugs.python.org/issue3177 9 msgs #6755: Patch: new method get_wch for ncurses bindings: accept wide ch http://bugs.python.org/issue6755 9 msgs #12279: Add build_distinfo command to packaging http://bugs.python.org/issue12279 8 msgs #12545: Incorrect handling of return codes in the posix_lseek function http://bugs.python.org/issue12545 8 msgs #12551: Provide data for TLS channel binding http://bugs.python.org/issue12551 8 msgs #5999: compile error on HP-UX 11.22 ia64 - 'mbstate_t' is used as a t http://bugs.python.org/issue5999 7 msgs #12326: Linux 3: tests should avoid using sys.platform == 'linux2' http://bugs.python.org/issue12326 7 msgs #12449: Add accelerator "F" to button "Finish" in all MSI installers m http://bugs.python.org/issue12449 6 msgs #12572: HP/UX compiler workarounds http://bugs.python.org/issue12572 6 msgs Issues closed (31) ================== #3565: array documentation, method names not 3.x-compliant http://bugs.python.org/issue3565 closed by pitrou #4376: Nested ctypes 'BigEndianStructure' fails http://bugs.python.org/issue4376 closed by python-dev #10647: scrollbar crash in non-US locale format settings http://bugs.python.org/issue10647 closed by hfischer #11863: Enforce PEP 11 - remove support for legacy systems http://bugs.python.org/issue11863 closed by pitrou #12149: Segfault in _PyObject_GenericGetAttrWithDict http://bugs.python.org/issue12149 closed by pitrou #12440: test_ssl.test_options() failure on Snow Leopard: can't clear o http://bugs.python.org/issue12440 closed by pitrou #12452: deprecation process in plistlib http://bugs.python.org/issue12452 closed by eric.araujo #12491: Update glossary documentation for the term 'attribute' http://bugs.python.org/issue12491 closed by rhettinger #12502: 100% cpu usage when using asyncore with UNIX socket http://bugs.python.org/issue12502 closed by neologix #12519: Call next version 3.3.0 http://bugs.python.org/issue12519 closed by eric.araujo #12522: Implement `os.startfile` under Linux and Mac http://bugs.python.org/issue12522 closed by rosslagerwall #12525: Unable to run a thread http://bugs.python.org/issue12525 closed by eric.smith #12530: cpython 3.3, __class__ missing. http://bugs.python.org/issue12530 closed by benjamin.peterson #12534: Tkinter doesn't support property attributes http://bugs.python.org/issue12534 closed by r.david.murray #12536: lib2to3 BaseFix class creates un-needed loggers leading to ref http://bugs.python.org/issue12536 closed by python-dev #12538: Extending int class http://bugs.python.org/issue12538 closed by r.david.murray #12539: multiprocessing.Event.wait(n) doesn't time out properly http://bugs.python.org/issue12539 closed by neologix #12543: `issubclass(collections.deque, collections.Sequence) == False` http://bugs.python.org/issue12543 closed by rhettinger #12544: Avoid using a pseudo-dict for _type_equality_funcs in unittest http://bugs.python.org/issue12544 closed by benjamin.peterson #12547: whatsnew/3.3: error in example about nntplib http://bugs.python.org/issue12547 closed by ezio.melotti #12548: Add suport native Functor http://bugs.python.org/issue12548 closed by ezio.melotti #12549: test_platform test_mac_ver fails on Darwin x86_64 http://bugs.python.org/issue12549 closed by ned.deily #12550: regrtest: register SIGALRM signal using faulthandler http://bugs.python.org/issue12550 closed by haypo #12552: email.MIMEText overide BASE64 for utf8 charset http://bugs.python.org/issue12552 closed by r.david.murray #12554: Failed imports clean up module, but not sub modules http://bugs.python.org/issue12554 closed by r.david.murray #12557: Crash idle on mac http://bugs.python.org/issue12557 closed by r.david.murray #12563: Check out my about.me profile! http://bugs.python.org/issue12563 closed by r.david.murray #12564: Check out my about.me profile! http://bugs.python.org/issue12564 closed by r.david.murray #12565: Check out my about.me profile! http://bugs.python.org/issue12565 closed by r.david.murray #12566: Check out my about.me profile! http://bugs.python.org/issue12566 closed by r.david.murray #12569: sqlite3 segfaults and bus errors when given certain unicode st http://bugs.python.org/issue12569 closed by ned.deily From lekmalek at gmail.com Sat Jul 16 10:33:22 2011 From: lekmalek at gmail.com (lekmalek) Date: Sat, 16 Jul 2011 10:33:22 +0200 Subject: [Python-Dev] Issue10271 - warnings.showwarning should allow any callable object - request commiter Message-ID: <20110716103322.330f805c@vimes> Hello all, Can any of you core devs have a look at http://bugs.python.org/issue10271. It seems Brett is really busy right now and this uncontroversial (AFAICT) one liner only needs someone to review it and commit it. The pb is, it's holding me back a little bit, and I really would like to have it in the next 3.2 release if possible. Thanks for your help, lekma From fijall at gmail.com Sat Jul 16 16:12:45 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Sat, 16 Jul 2011 16:12:45 +0200 Subject: [Python-Dev] making socket.getaddrinfo use cached dns In-Reply-To: <20110707175901.GA28561@iskra.aviel.ru> References: <20110707175901.GA28561@iskra.aviel.ru> Message-ID: On Thu, Jul 7, 2011 at 7:59 PM, Oleg Broytman wrote: > Hello. > > ? We are sorry but we cannot help you. This mailing list is to work on > developing Python (adding new features to Python itself and fixing bugs); Well, it seems this post is about adding a new feature isn't it? Cheers, fijal From phd at phdru.name Sat Jul 16 16:50:22 2011 From: phd at phdru.name (Oleg Broytman) Date: Sat, 16 Jul 2011 18:50:22 +0400 Subject: [Python-Dev] making socket.getaddrinfo use cached dns In-Reply-To: References: <20110707175901.GA28561@iskra.aviel.ru> Message-ID: <20110716145022.GD30480@iskra.aviel.ru> On Sat, Jul 16, 2011 at 04:12:45PM +0200, Maciej Fijalkowski wrote: > On Thu, Jul 7, 2011 at 7:59 PM, Oleg Broytman wrote: > > ? We are sorry but we cannot help you. This mailing list is to work on > > developing Python (adding new features to Python itself and fixing bugs); > > Well, it seems this post is about adding a new feature isn't it? I don't think so. The original post is about problems with pdnsd - could be just a local configuration problem, and has nothing with Python. The original post is about rolling back getaddrinfo and returning to gethostby* - certainly not a new feature. That's how I understand the original post. Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From ncoghlan at gmail.com Sat Jul 16 16:59:17 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 17 Jul 2011 00:59:17 +1000 Subject: [Python-Dev] making socket.getaddrinfo use cached dns In-Reply-To: References: <20110707175901.GA28561@iskra.aviel.ru> Message-ID: On Sun, Jul 17, 2011 at 12:12 AM, Maciej Fijalkowski wrote: > On Thu, Jul 7, 2011 at 7:59 PM, Oleg Broytman wrote: >> Hello. >> >> ? We are sorry but we cannot help you. This mailing list is to work on >> developing Python (adding new features to Python itself and fixing bugs); > > Well, it seems this post is about adding a new feature isn't it? Not really - the key question was "How can I get python to correctly use cached dns lookups and ipv4 only (at least in those cases where it is appropriate)." This isn't the place to ask that question (particularly since it's the wrong question - the real question is why the IPv6 lookups are taking so long. Since we just call into the C standard library for name resolution, whether it's slow or fast is an OS configuration problem). The latter part (very indirectly) made a feature suggestion via the reference off to the SO question. However, hardcoding the *app* to be IPv4 only really isn't a good workaround for IPv6 resolution taking a long time to fail at the OS level. Exposing those flags would encourage people to do exactly that, and that would be a *really* bad idea (it's unfortunate enough that PEP 3144 stalled, or we might have had better support for manipulating IPv6 addresses in the standard library by now. We really shouldn't make things even worse by making it easy for developers with broken IPv6 setups to switch off IPv6 support entirely). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From solipsis at pitrou.net Sat Jul 16 17:26:07 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 16 Jul 2011 17:26:07 +0200 Subject: [Python-Dev] making socket.getaddrinfo use cached dns References: <20110707175901.GA28561@iskra.aviel.ru> Message-ID: <20110716172607.5133294f@pitrou.net> On Sun, 17 Jul 2011 00:59:17 +1000 Nick Coghlan wrote: > > Exposing those flags would > encourage people to do exactly that, and that would be a *really* bad > idea Making DNS resolution configurable (for example by allowing the user to supply their own resolution function) in the stdlib's network APIs doesn't sound like a really bad idea to me. Of course, a "no_ipv6" flag would scale poorly and would therefore be a bad API. > We really shouldn't make things even worse by making > it easy for developers with broken IPv6 setups to switch off IPv6 > support entirely). I don't think moralist arguments should have a weight when deciding which features we add. If developers want to introduce bugs or limitations in their software they will always be able to do it. Regards Antoine. From guido at python.org Sat Jul 16 17:46:16 2011 From: guido at python.org (Guido van Rossum) Date: Sat, 16 Jul 2011 08:46:16 -0700 Subject: [Python-Dev] making socket.getaddrinfo use cached dns In-Reply-To: <20110716172607.5133294f@pitrou.net> References: <20110707175901.GA28561@iskra.aviel.ru> <20110716172607.5133294f@pitrou.net> Message-ID: On Sat, Jul 16, 2011 at 8:26 AM, Antoine Pitrou wrote: > I don't think moralist arguments should have a weight when deciding > which features we add. If developers want to introduce bugs or > limitations in their software they will always be able to do it. Actually when designing language features or APIs, what you call "moralist arguments" take place all the time. Personally I don't think there's anything "moral" about wanting to design an API that reduces common mistakes, and API design should always take expected behavior of programmers into account. Experienced developers have a huge store of information about that in their head. Anyway, even before the word "moralist" was used this thread would have been better on python-ideas. -- --Guido van Rossum (python.org/~guido) From brett at python.org Mon Jul 18 02:16:45 2011 From: brett at python.org (Brett Cannon) Date: Sun, 17 Jul 2011 17:16:45 -0700 Subject: [Python-Dev] Latest draft of PEP 399 (Pure Python/C Accelerator Module Compatibility Requirements) Message-ID: While at a mini-PyPy sprint w/ Alex Gaynor of PyPy and Phil Jenvey of Jython, I decided to finally put the time in to update this PEP yet again. The biggest changes is that the 100% branch coverage requirement has been replaced with "comprehensive" coverage from the tests. I think we are all enough grown-ups to not have to specify anything tighter than this. I also added a paragraph in the Details section about using the abstract C APIs (e.g., PyObject_GetItem) over type-specific ones (e.g., PyList_GetItem) in order to be more supportive of duck typing from the Python code. I figure the "be API compatible" assumes this, but mentioning it doesn't hurt (and should help make Raymond less angry =). PEP: 399 Title: Pure Python/C Accelerator Module Compatibility Requirements Version: $Revision: 88219 $ Last-Modified: $Date: 2011-01-27 13:47:00 -0800 (Thu, 27 Jan 2011) $ Author: Brett Cannon Status: Draft Type: Informational Content-Type: text/x-rst Created: 04-Apr-2011 Python-Version: 3.3 Post-History: 04-Apr-2011, 12-Apr-2011, 17-Jul-2011 Abstract ======== The Python standard library under CPython contains various instances of modules implemented in both pure Python and C (either entirely or partially). This PEP requires that in these instances that the C code *must* pass the test suite used for the pure Python code so as to act as much as a drop-in replacement as reasonably possible (C- and VM-specific tests are exempt). It is also required that new C-based modules lacking a pure Python equivalent implementation get special permission to be added to the standard library. Rationale ========= Python has grown beyond the CPython virtual machine (VM). IronPython_, Jython_, and PyPy_ are all currently viable alternatives to the CPython VM. The VM ecosystem that has sprung up around the Python programming language has led to Python being used in many different areas where CPython cannot be used, e.g., Jython allowing Python to be used in Java applications. A problem all of the VMs other than CPython face is handling modules from the standard library that are implemented (to some extent) in C. Since other VMs do not typically support the entire `C API of CPython`_ they are unable to use the code used to create the module. Often times this leads these other VMs to either re-implement the modules in pure Python or in the programming language used to implement the VM itself (e.g., in C# for IronPython). This duplication of effort between CPython, PyPy, Jython, and IronPython is extremely unfortunate as implementing a module *at least* in pure Python would help mitigate this duplicate effort. The purpose of this PEP is to minimize this duplicate effort by mandating that all new modules added to Python's standard library *must* have a pure Python implementation _unless_ special dispensation is given. This makes sure that a module in the stdlib is available to all VMs and not just to CPython (pre-existing modules that do not meet this requirement are exempt, although there is nothing preventing someone from adding in a pure Python implementation retroactively). Re-implementing parts (or all) of a module in C (in the case of CPython) is still allowed for performance reasons, but any such accelerated code must pass the same test suite (sans VM- or C-specific tests) to verify semantics and prevent divergence. To accomplish this, the test suite for the module must have comprehensive coverage of the pure Python implementation before the acceleration code may be added. Details ======= Starting in Python 3.3, any modules added to the standard library must have a pure Python implementation. This rule can only be ignored if the Python development team grants a special exemption for the module. Typically the exemption will be granted only when a module wraps a specific C-based library (e.g., sqlite3_). In granting an exemption it will be recognized that the module will be considered exclusive to CPython and not part of Python's standard library that other VMs are expected to support. Usage of ``ctypes`` to provide an API for a C library will continue to be frowned upon as ``ctypes`` lacks compiler guarantees that C code typically relies upon to prevent certain errors from occurring (e.g., API changes). Even though a pure Python implementation is mandated by this PEP, it does not preclude the use of a companion acceleration module. If an acceleration module is provided it is to be named the same as the module it is accelerating with an underscore attached as a prefix, e.g., ``_warnings`` for ``warnings``. The common pattern to access the accelerated code from the pure Python implementation is to import it with an ``import *``, e.g., ``from _warnings import *``. This is typically done at the end of the module to allow it to overwrite specific Python objects with their accelerated equivalents. This kind of import can also be done before the end of the module when needed, e.g., an accelerated base class is provided but is then subclassed by Python code. This PEP does not mandate that pre-existing modules in the stdlib that lack a pure Python equivalent gain such a module. But if people do volunteer to provide and maintain a pure Python equivalent (e.g., the PyPy team volunteering their pure Python implementation of the ``csv`` module and maintaining it) then such code will be accepted. In those instances the C version is considered the reference implementation in terms of expected semantics. Any new accelerated code must act as a drop-in replacement as close to the pure Python implementation as reasonable. Technical details of the VM providing the accelerated code are allowed to differ as necessary, e.g., a class being a ``type`` when implemented in C. To verify that the Python and equivalent C code operate as similarly as possible, both code bases must be tested using the same tests which apply to the pure Python code (tests specific to the C code or any VM do not follow under this requirement). The test suite is expected to be extensive in order to verify expected semantics. Acting as a drop-in replacement also dictates that no public API be provided in accelerated code that does not exist in the pure Python code. Without this requirement people could accidentally come to rely on a detail in the accelerated code which is not made available to other VMs that use the pure Python implementation. To help verify that the contract of semantic equivalence is being met, a module must be tested both with and without its accelerated code as thoroughly as possible. As an example, to write tests which exercise both the pure Python and C accelerated versions of a module, a basic idiom can be followed:: import collections.abc from test.support import import_fresh_module, run_unittest import unittest c_heapq = import_fresh_module('heapq', fresh=['_heapq']) py_heapq = import_fresh_module('heapq', blocked=['_heapq']) class ExampleTest(unittest.TestCase): def test_heappop_exc_for_non_MutableSequence(self): # Raise TypeError when heap is not a # collections.abc.MutableSequence. class Spam: """Test class lacking many ABC-required methods (e.g., pop()).""" def __len__(self): return 0 heap = Spam() self.assertFalse(isinstance(heap, collections.abc.MutableSequence)) with self.assertRaises(TypeError): self.heapq.heappop(heap) class AcceleratedExampleTest(ExampleTest): """Test using the accelerated code.""" heapq = c_heapq class PyExampleTest(ExampleTest): """Test with just the pure Python code.""" heapq = py_heapq def test_main(): run_unittest(AcceleratedExampleTest, PyExampleTest) if __name__ == '__main__': test_main() If this test were to provide extensive coverage for ``heapq.heappop()`` in the pure Python implementation then the accelerated C code would be allowed to be added to CPython's standard library. If it did not, then the test suite would need to be updated until proper coverage was provided before the accelerated C code could be added. To also help with compatibility, C code should use abstract APIs on objects to prevent accidental dependence on specific types. For instance, if a function accepts a sequence then the C code should default to using `PyObject_GetItem()` instead of something like `PyList_GetItem()`. C code is allowed to have a fast path if the proper `PyList_Check()` is used, but otherwise APIs should work with any object that duck types to the proper interface instead of a specific type. Copyright ========= This document has been placed in the public domain. .. _IronPython: http://ironpython.net/ .. _Jython: http://www.jython.org/ .. _PyPy: http://pypy.org/ .. _C API of CPython: http://docs.python.org/py3k/c-api/index.html .. _sqlite3: http://docs.python.org/py3k/library/sqlite3.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Mon Jul 18 04:19:59 2011 From: brett at python.org (Brett Cannon) Date: Sun, 17 Jul 2011 19:19:59 -0700 Subject: [Python-Dev] Issue10271 - warnings.showwarning should allow any callable object - request commiter In-Reply-To: <20110716103322.330f805c@vimes> References: <20110716103322.330f805c@vimes> Message-ID: Just so people know, I went ahead and fixed this for 3.3 (but not for 3.2 since it changes the API in a subtle way). On Sat, Jul 16, 2011 at 01:33, lekmalek wrote: > Hello all, > > Can any of you core devs have a look at > http://bugs.python.org/issue10271. It seems Brett is really busy right > now and this uncontroversial (AFAICT) one liner only needs someone to > review it and commit it. The pb is, it's holding me back a little bit, > and I really would like to have it in the next 3.2 release if possible. > > Thanks for your help, > > lekma > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/brett%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mail at kerrickstaley.com Mon Jul 18 08:43:46 2011 From: mail at kerrickstaley.com (Kerrick Staley) Date: Mon, 18 Jul 2011 01:43:46 -0500 Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the /usr/bin/python2 symlink upstream) Message-ID: Hi, These are two emails I sent a short while ago about finalizing PEP 394. There was no response, so in case the messages didn't go through, I'm resending them. Thanks, Kerrick Staley ---------- Forwarded message ---------- From: Kerrick Staley Date: Sat, Jul 9, 2011 at 7:45 PM Subject: Re: [Python-Dev] [PEPs] Support the /usr/bin/python2 symlink upstream To: python-dev at python.org Sorry that I dropped the ball on this. I'd still like to see this get implemented, but I got distracted with school and forgot about it. Updates I have made to the PEP will be sent as a patch immediately after this email. Here's a summary of what was happenening when we left off: * The draft SVN version from March 4 was pretty much complete. * Some were concerned about addressing Windows, but it was generally agreed to leave the Windows issue to another PEP. PEP 397 was started on March 15 to address the Windows side of the issue. PEP 397 recommends that the Windows Python launcher read the shebang and use it to determine which Python version to use; this allows one syntax for both operating systems that is compatible with the current PEP 394 recommendation. * There were concerns from Ned Deily about the naming of other binaries such as idle, pydoc, and python-config; these need to be created as idle2, pydoc2, and python2-config, with links created at the locations of the original binaries. * There were concerns from Glenn Linderman that the shebang line doesn't encode enough information to flexibly handle Windows launching (or even launching in general). ==== Here are my thoughts: * For Ned's comments, I agree. Although the issue isn't as large with these programs, there's no reason we can't handle them in the same way. I updated the PEP. * For Glenn's comments, I think the method you propose adds too much complexity. Regardless, if the #@ syntax is implemented, it can be described in PEP 397; it won't have an impact on the contents of this PEP. I think, though, that having multiple syntaxes may cause many scripts to be incompatible with multiple platforms when they don't have to be, since Unix coders will rarely add a #@ line, and Windows coders will likely forget the #! line. Also, #@ really ignores the purpose of a shebang: shebangs simply indicate an interpreter that works with the script; the shebang allows users to run arbitrary scripts without worrying about which interpreter they should specify. There's no reason that a script should use one interpreter on Unix, but be incompatible with that interpreter on Windows yet compatible with another. The name of the Unix binary is enough to determine the implementation and version of the interpreter to be used on Unix, and a Windows launcher should always invoke the same implementation/version on Windows (and this won't require hard-coding anything into the launcher if done right). If you want the script to run with a specific interpreter and version, possibly contingent on which operating system you're running the script under, then you can just invoke the interpreter by name with the script as an argument (e.g. python3 myprogram.py). TL;DR: shebangs encode a default implementation/version, and if you need something special, you can just manually run python3 myprogram.py or use a .bat file. ==== Also, I updated the PEP with the clarification that commands like python3 should be hard links (because they'll be invoked from code and are more efficient; also, hard links are just as flexible as symlinks here), while commands like python should be soft links (because this makes it clear to sysadmins that they can be "switched", and it's needed for flexibility if python3 changes). This really doesn't matter, but can we keep it this way unless there are serious objections? Regards, Kerrick Staley ---------- Forwarded message ---------- From: Kerrick Staley Date: Sat, Jul 9, 2011 at 7:46 PM Subject: Re: [Python-Dev] [PEPs] Support the /usr/bin/python2 symlink upstream To: python-dev at python.org $ svn diff Index: pep-0394.txt =================================================================== --- pep-0394.txt (revision 88860) +++ pep-0394.txt (working copy) @@ -1,5 +1,5 @@ PEP: 394 -Title: The "python" command on Unix-Like Systems +Title: The "python" Command on Unix-Like Systems Version: $Revision$ Last-Modified: $Date$ Author: Kerrick Staley , @@ -53,10 +53,14 @@ * When reinvoking the interpreter from a Python script, querying ``sys.executable`` to avoid hardcoded assumptions regarding the interpreter location remains the preferred approach. +* The ``idle``, ``pydoc``, and ``python-config`` binaries from Python 2.0 +should likewise be available as ``idle2``, ``pydoc2``, and ``python2-config``, +with the original commands invoking these binaries by default, but possibly +invoking the Python 3.0 versions instead. These recommendations are the outcome of the relevant python-dev discussion in -March 2011 [1] (NOTE: More accurately, they will be such once that "Draft" -status disappears from the PEP header, it has been moved into the "Other +March to July 2011 [1] (NOTE: More accurately, they will be such once that +"Draft" status disappears from the PEP header, it has been moved into the "Other Informational PEP" section in PEP 0 and this note has been deleted) @@ -144,11 +148,16 @@ While technically a new feature, the ``make install`` command and the Mac OS X installer in the 2.7 version of CPython will be adjusted to create the -new ``python2`` command in addition to the existing ``python`` and -``python2.7`` commands. This feature will first appear in CPython 2.7.2. +``python2.7``, ``idle2.7``, ``pydoc2.7``, and ``python2.7-config`` binaries, +with ``python2``, ``idle2``, ``pydoc2``, and ``python2-config`` as hard links +to the respective binaries, and ``python``, ``idle``, ``pydoc``, and +``python-config`` as symbolic links to the respective hard links. This feature +will first appear in CPython 2.7.2. -The ``make install`` command in the CPython 3.x series will continue to -install only the ``python3`` symlink for the foreseeable future. +The ``make install`` command in the CPython 3.x series will similarly install +the ``python3.x``, ``idle3.x``, ``pydoc3.x``, and ``python3.x-config`` binaries +(with appropriate ``x``), and ``python3``, ``idle3``, ``pydoc3``, and +``python3-config`` as hard links. Impact on PYTHON* Environment Variables @@ -166,27 +175,12 @@ Exclusions of MS Windows ======================== -This PEP deliberately excludes any proposals relating to Microsoft Windows. -The use of parallel installs on Windows suffers from numerous issues, -including the "last installed wins" behaviour for handling of file -associations, a lack of universal robust symlink support for easy aliasing of -commands, the fact that the Python executable is not available on ``PATH`` by -default, the fact that the ``python.exe`` and ``pythonw.exe`` names are -used for both Python 2 and Python 3 binaries and the lack of distinction -between the different Python versions when the Start menu shortcuts are -divorced from their containing folders (e.g. when they appear in the -"Recently Used" list. +This PEP deliberately excludes any proposals relating to Microsoft Windows: +devising an equivalent solution for Windows was deemed too complex to handle +here. PEP 397 and the related discussion on the python-dev mailing list address +this issue. -While these questions are well worth addressing, they do not have easy -answers. The authors of this particular PEP aren't inclined to even begin -trying to answer them, but anyone that wants to tackle them should feel free -to start working on their own PEP. -Note that, while the topic has been excluded from this PEP, there is plenty of -material in the linked python-dev discussion that may be useful in the design -and implementation of a Windows-specific solution. - - References ========== From nad at acm.org Mon Jul 18 10:03:50 2011 From: nad at acm.org (Ned Deily) Date: Mon, 18 Jul 2011 01:03:50 -0700 Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the /usr/bin/python2 symlink upstream) References: Message-ID: In article , Kerrick Staley wrote: > Here are my thoughts: > * For Ned's comments, I agree. Although the issue isn't as large with > these programs, there's no reason we can't handle them in the same > way. I updated the PEP. Thanks. > Also, I updated the PEP with the clarification that commands like > python3 should be hard links (because they'll be invoked from code and > are more efficient; also, hard links are just as flexible as symlinks > here), while commands like python should be soft links (because this > makes it clear to sysadmins that they can be "switched", and it's > needed for flexibility if python3 changes). This really doesn't > matter, but can we keep it this way unless there are serious > objections? I think adding the requirement to mandate hard link vs soft link usage is an unnecessary and unwarranted attempt at optimization. For instance, IIRC, the OS X installers don't use any hard links: that may complicate the install, plus hard links on OS X HFS* file systems are a bit of a kludge and not necessarily more efficient than symlinks. It's not a big deal but perhaps the wording should be changed to make a suggestion about hard links vs syminks rather than mandate which should be used. -- Ned Deily, nad at acm.org From durga.disc at gmail.com Mon Jul 18 17:32:38 2011 From: durga.disc at gmail.com (Durga D) Date: Mon, 18 Jul 2011 21:02:38 +0530 Subject: [Python-Dev] Python web adv. ? Message-ID: Hi All, My desktop application opens web page from tomcat/apache server in button click event (MySQL database by using JSP/Servlets on internet from user machine). Here, I found some delay in opening web pages. Is there any advantages with Python over JSP/Servlets? I am new to web programming. Please provide any inputs on Python over jsp/servlets. Thanks in advance. Regards, -------------- next part -------------- An HTML attachment was scrubbed... URL: From _ at lvh.cc Mon Jul 18 17:49:01 2011 From: _ at lvh.cc (Laurens Van Houtven) Date: Mon, 18 Jul 2011 17:49:01 +0200 Subject: [Python-Dev] Python web adv. ? In-Reply-To: References: Message-ID: Hi! This list is for developing Python itself, not about developing *in* Python. For more support try the comp.lang.python newsgroup or the equivalent http://mail.python.org/mailman/listinfo/python-list mailing list, or the #python IRC channel on Freenode. cheers lvh On Mon, Jul 18, 2011 at 5:32 PM, Durga D wrote: > Hi All, > > My desktop application opens web page from tomcat/apache server in > button click event (MySQL database by using JSP/Servlets on internet from > user machine). Here, I found some delay in opening web pages. Is there any > advantages with Python over JSP/Servlets? > > I am new to web programming. Please provide any inputs on Python over > jsp/servlets. > > Thanks in advance. > > Regards, > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/_%40lvh.cc > > -- cheers lvh -------------- next part -------------- An HTML attachment was scrubbed... URL: From v+python at g.nevcal.com Mon Jul 18 21:10:46 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Mon, 18 Jul 2011 12:10:46 -0700 Subject: [Python-Dev] 3.2.1 encoding surprise Message-ID: <4E248536.30006@g.nevcal.com> Attached reduced test case works fine with Python 3.1, fails with Python3.2: SyntaxError: Non-ASCII character '\xc3' in file D:\my\py\t32enc.py on line 1, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details I'm familiar with the PEP, but thought 3.x cured that. Python 3.1 produces no error; I'm not sure that 3.2 did or didn't report such problems, or if the programs I ran with 3.2 simply didn't contain non-ASCII characters. The file is UTF-8 encoded with no pseudo-BOM. Is this intentional, or a regression (in which case I will open a ticket)? If a regression, does that mean we have no tests for it, possibly because most of the tests contain the PEP 263 encoding directive, because most people using 3.x are still also using 2.x (I am not)? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- "L ? W" From tjreedy at udel.edu Mon Jul 18 21:40:39 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 18 Jul 2011 15:40:39 -0400 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: <4E248536.30006@g.nevcal.com> References: <4E248536.30006@g.nevcal.com> Message-ID: On 7/18/2011 3:10 PM, Glenn Linderman wrote: > Attached reduced test case works fine with Python 3.1, fails with Python3.2: > > SyntaxError: Non-ASCII character '\xc3' in file D:\my\py\t32enc.py on > line 1, but no encoding declared; see > http://www.python.org/peps/pep-0263.html for details It runs fine for me on winxp, 3.2.1, both command prompt and idle. When cut/paste worked, I downloaded file and ran that. > I'm familiar with the PEP, but thought 3.x cured that. Python 3.1 > produces no error; I'm not sure that 3.2 did or didn't report such > problems, or if the programs I ran with 3.2 simply didn't contain > non-ASCII characters. > > The file is UTF-8 encoded with no pseudo-BOM. > > Is this intentional, or a regression (in which case I will open a ticket)? > > If a regression, does that mean we have no tests for it, possibly > because most of the tests contain the PEP 263 encoding directive, > because most people using 3.x are still also using 2.x (I am not)? Or there could be a test (have not checked) which usually passes. -- Terry Jan Reedy From p.f.moore at gmail.com Mon Jul 18 22:01:06 2011 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 18 Jul 2011 21:01:06 +0100 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: <4E248536.30006@g.nevcal.com> References: <4E248536.30006@g.nevcal.com> Message-ID: 2011/7/18 Glenn Linderman : > Attached reduced test case works fine with Python 3.1, fails with Python3.2: PS D:\Data> py -3 .\t32enc.py PS D:\Data> py -2 .\t32enc.py File ".\t32enc.py", line 1 SyntaxError: Non-ASCII character '\xc3' in file .\t32enc.py on line 1, but no encoding declared; see http://www.python.o rg/peps/pep-0263.html for details PS D:\Data> py -3 --version Python 3.2.1 PS D:\Data> py -2 --version Python 2.7 Windows 7 32-bit, py is Vinay's implementation of the PEP 397 launcher for Windows. This looks like correct output to me. Could it be your environment somehow? Paul. From anthony.hw.kong at gmail.com Mon Jul 18 22:35:52 2011 From: anthony.hw.kong at gmail.com (Anthony Kong) Date: Tue, 19 Jul 2011 06:35:52 +1000 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: References: <4E248536.30006@g.nevcal.com> Message-ID: Similar outcome as Paul's. $ python3 t32enc.py $ python t32enc.py File "t32enc.py", line 1 SyntaxError: Non-ASCII character '\xc3' in file t32enc.py on line 1, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details $ python3 -V Python 3.2.1a0 $ python -V Python 2.6.1 Running the script on OSX 10.6.8 (And I just realised I am running 3.2.1a0...) Cheers On Tue, Jul 19, 2011 at 6:01 AM, Paul Moore wrote: > 2011/7/18 Glenn Linderman : > > Attached reduced test case works fine with Python 3.1, fails with > Python3.2: > > PS D:\Data> py -3 .\t32enc.py > PS D:\Data> py -2 .\t32enc.py > File ".\t32enc.py", line 1 > SyntaxError: Non-ASCII character '\xc3' in file .\t32enc.py on line 1, > but no encoding declared; see http://www.python.o > rg/peps/pep-0263.html for details > PS D:\Data> py -3 --version > Python 3.2.1 > PS D:\Data> py -2 --version > Python 2.7 > > Windows 7 32-bit, py is Vinay's implementation of the PEP 397 launcher > for Windows. > > This looks like correct output to me. > > Could it be your environment somehow? > Paul. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/anthony.hw.kong%40gmail.com > -- Tony Kong *blog:* www.ahwkong.com Don?t EVER make the mistake that you can design something better than what you get from ruthless massively parallel trial-and-error with a feedback cycle. That?s giving your intelligence *much* too much credit. - Linus Torvalds -------------- next part -------------- An HTML attachment was scrubbed... URL: From v+python at g.nevcal.com Mon Jul 18 22:36:48 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Mon, 18 Jul 2011 13:36:48 -0700 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: References: <4E248536.30006@g.nevcal.com> Message-ID: <4E249960.9000109@g.nevcal.com> On 7/18/2011 1:01 PM, Paul Moore wrote: > 2011/7/18 Glenn Linderman: >> Attached reduced test case works fine with Python 3.1, fails with Python3.2: > PS D:\Data> py -3 .\t32enc.py > PS D:\Data> py -2 .\t32enc.py > File ".\t32enc.py", line 1 > SyntaxError: Non-ASCII character '\xc3' in file .\t32enc.py on line 1, > but no encoding declared; see http://www.python.o > rg/peps/pep-0263.html for details > PS D:\Data> py -3 --version > Python 3.2.1 > PS D:\Data> py -2 --version > Python 2.7 > > Windows 7 32-bit, py is Vinay's implementation of the PEP 397 launcher > for Windows. > > This looks like correct output to me. > > Could it be your environment somehow? > Paul. Yes, a surprise. But it is my environment.... I'm not sure exactly how or why yet. I've continued investigating since posting that... but thanks for the confirmation that it isn't 3.2.1 ! Tweaked the program to show the version and it showed 2.6.4! Which is the only version of Python 2.x installed here. So my suspicion is that launchsys.adm64.msi, installed just after 3.2.1 (but which appeared to do nothing) has taken over ... something. Not sure what yet. The reason I thought it did nothing, is that I checked assoc ( =Python.File ) and ftype ( =c:\python32\python.exe "%1" %* ) both of which look familiar, and neither of which mention py.exe which is what I think the launcher is supposed to have been named; and running c:\python32\python.exe from the command line produces a Python that looks like 3.2.1 . But somehow, running t32enc.py from the command line fails, and loads 2.6.4 ! From what I've read about the launcher, I thought it was supposed to take over the assoc and ftype and point Python.File to c:\windows\system32\py.exe -- but that hasn't happened (and yes, I've rebooted since doing the installs, so it is not just a leftover CMD that didn't pick up new values). So my question now is: how does the launcher really get activated when invoking .py files from the command line, if the assoc and ftype indicate that it should run c:\python32\python.exe (which when I run by hand, seems to be what the name claims it to be)? We don't need to create mysteries! Glenn -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinay_sajip at yahoo.co.uk Mon Jul 18 23:13:27 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Mon, 18 Jul 2011 21:13:27 +0000 (UTC) Subject: [Python-Dev] 3.2.1 encoding surprise References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> Message-ID: > The reason I thought it did nothing, is that I checked assoc? ( > =Python.File ) and ftype ( =c:\python32\python.exe "%1" %* ) both of > which look familiar, and neither of which mention?? py.exe? which is > what I think the launcher is supposed to have been named; and > running? c:\python32\python.exe from the command line produces a > Python that looks like 3.2.1 . > But somehow, running??? t32enc.py? from the command line fails, and > loads 2.6.4 ! > From what I've read about the launcher, I thought it was supposed to > take over the assoc and ftype and point Python.File to? > c:\windows\system32\py.exe? -- but that hasn't happened (and yes, > I've rebooted since doing the installs, so it is not just a leftover > CMD that didn't pick up new values). > So my question now is: how does the launcher really get activated > when invoking .py files from the command line, if the assoc and > ftype indicate that it should run c:\python32\python.exe (which when > I run by hand, seems to be what the name claims it to be)? > We don't need to create mysteries! > Glenn The launcher's activation is done by the shell you're using, which does the lookups in the registry. Remember that there are two sets of locations - HKCU and HKLM - where the type associations are potentially held. Please do a registry search (with Administrator rights so you can search the whole registry) for "py.exe" or "pyw.exe" and see if they show up anywhere at all. The launcher code tries to add these keys in HKEY_CLASSES_ROOT, but I believe Windows can map this to HKCU rather than HKLM if you don't have administrator access, at least on XP. Regards, Vinay Sajip From v+python at g.nevcal.com Tue Jul 19 00:05:14 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Mon, 18 Jul 2011 15:05:14 -0700 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> Message-ID: <4E24AE1A.3000308@g.nevcal.com> On 7/18/2011 2:13 PM, Vinay Sajip wrote: > Remember that there are two sets of locations - HKCU and HKLM - where the type > associations are potentially held. Please do a registry search (with > Administrator rights so you can search the whole registry) for "py.exe" or > "pyw.exe" and see if they show up anywhere at all. The launcher code tries to > add these keys in HKEY_CLASSES_ROOT, but I believe Windows can map this to HKCU > rather than HKLM if you don't have administrator access, at least on XP. Oh, and I'm running Windows 7 64-bit, but most of my registry knowledge was learned on Win2K and XP, and I have no knowledge of what they did in Vista or 7, and haven't yet attempted to research such. I am running as an administrator, but a much more ignorant one on 7 than I was on XP. I have no idea why CMD would show an ftype Python.File as one thing, but then the execution would use something from the registry that is different. -------------- next part -------------- An HTML attachment was scrubbed... URL: From v+python at g.nevcal.com Tue Jul 19 00:06:30 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Mon, 18 Jul 2011 15:06:30 -0700 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> Message-ID: <4E24AE66.5030305@g.nevcal.com> On 7/18/2011 2:13 PM, Vinay Sajip wrote: >> The reason I thought it did nothing, is that I checked assoc ( >> =Python.File ) and ftype ( =c:\python32\python.exe "%1" %* ) both of >> which look familiar, and neither of which mention py.exe which is >> what I think the launcher is supposed to have been named; and >> running c:\python32\python.exe from the command line produces a >> Python that looks like 3.2.1 . >> But somehow, running t32enc.py from the command line fails, and >> loads 2.6.4 ! >> From what I've read about the launcher, I thought it was supposed to >> take over the assoc and ftype and point Python.File to >> c:\windows\system32\py.exe -- but that hasn't happened (and yes, >> I've rebooted since doing the installs, so it is not just a leftover >> CMD that didn't pick up new values). >> So my question now is: how does the launcher really get activated >> when invoking .py files from the command line, if the assoc and >> ftype indicate that it should run c:\python32\python.exe (which when >> I run by hand, seems to be what the name claims it to be)? >> We don't need to create mysteries! >> Glenn > The launcher's activation is done by the shell you're using, which does the > lookups in the registry. > > Remember that there are two sets of locations - HKCU and HKLM - where the type > associations are potentially held. Please do a registry search (with > Administrator rights so you can search the whole registry) for "py.exe" or > "pyw.exe" and see if they show up anywhere at all. The launcher code tries to > add these keys in HKEY_CLASSES_ROOT, but I believe Windows can map this to HKCU > rather than HKLM if you don't have administrator access, at least on XP. Thanks for the response, Vinay... you are the one that would know, now that I've figured out it is a launcher issue rather than a Python issue. Here is a list of py.exe references in my registry: HCR\Python.CompiledFile HCR\Python.File HCR\Python.NoConFile HCU\Software\Classes\Python.CompiledFile HCU\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts\.py\OpenWithListk HLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Installer\UserData\S-1-5-21.....\Componer HLM\SOFTWARE\Wow6432Note\Microsoft\Windows\CurrentVersion\SharedDLLs HKU\S-1-5-21-.....\Software\Classes\Python.CompiledFile (and similar corresponding to the above HCR ones) My (perhaps dated) understanding of the CMD commands attrib and ftype is that they reflect the contents of HCR. But here HCR\Python.File is different than the value displayed by the CMD ftype Python.File. Why? What am I missing? -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinay_sajip at yahoo.co.uk Tue Jul 19 01:55:11 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Mon, 18 Jul 2011 23:55:11 +0000 (UTC) Subject: [Python-Dev] 3.2.1 encoding surprise References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> Message-ID: Glenn Linderman g.nevcal.com> writes: > Here is a list of py.exe references in my registry: > HCR\Python.CompiledFile > HCR\Python.File > HCR\Python.NoConFile [snip] There shouldn't be so many references, so I suggest you may want to try the following (after backing up your registry first): 1. Uninstall all instances of the launcher. 2. Remove all registry keys for .py, .pyc, .pyo, .pyw, Python.File, Python.NoConFile, and Python.CompiledFile. 3. Re-install the launcher (64-bit) using "msiexec /i launchsys.amd64.msi ALLUSERS=1" (or launcher.amd64.msi if you prefer - the only difference is the installation directory). 4. Check that the removed associations have appeared under HKEY_LOCAL_MACHINE, and that they all point to the launcher installed in the previous step. These may also appear in HKEY_CLASSES_ROOT if it is aliased to the HKEY_LOCAL_MACHINE\Software\Classes tree. Regards, Vinay Sajip From v+python at g.nevcal.com Tue Jul 19 03:04:59 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Mon, 18 Jul 2011 18:04:59 -0700 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> Message-ID: <4E24D83B.8040100@g.nevcal.com> On 7/18/2011 4:55 PM, Vinay Sajip wrote: > Glenn Linderman g.nevcal.com> writes: > > >> Here is a list of py.exe references in my registry: >> HCR\Python.CompiledFile >> HCR\Python.File >> HCR\Python.NoConFile > [snip] > > > There shouldn't be so many references, so I suggest you may want to try the > following (after backing up your registry first): So, then, there must be 2 problems... one in the launcher installer, and the other being whatever happened to my machine. Can you list the ones there should be? > 1. Uninstall all instances of the launcher. Well, there is only one. But I can certainly remove it, and see what happens. > 2. Remove all registry keys for .py, .pyc, .pyo, .pyw, Python.File, > Python.NoConFile, and Python.CompiledFile. Yes, I can do this. > 3. Re-install the launcher (64-bit) using "msiexec /i launchsys.amd64.msi > ALLUSERS=1" (or launcher.amd64.msi if you prefer - the only difference is the > installation directory). So what I had done was just double click launchsys.msi. Why should there be command line parameters, rather than question/answer dialogs? This is Windows after all. What does the parameter do, that would be different than what double click did? > 4. Check that the removed associations have appeared under HKEY_LOCAL_MACHINE, > and that they all point to the launcher installed in the previous step. These > may also appear in HKEY_CLASSES_ROOT if it is aliased to the > HKEY_LOCAL_MACHINE\Software\Classes tree. I also still don't understand why CMD reports a different command line via its assoc/ftype commands for the Python.File than actually gets used by CMD to launch the .py file. Found the following at http://zetconsultants.com/blog/?p=36: > Second issue (not really an issue, however something that makes our > situation more complicated) is the fact that HKCR is in fact registry > symlink and this registry key doesn?t really exists on system. It is > merge of two different keys ? HKLM\Software\Classes and > HKCU\Software\Classes. You probably already guessed (correctly) that > this is the way how you can overwrite some settings on per-user basis > ? HKCU key always wins. which is useful, but still doesn't explain the variant results from ftype and launching behavior.} They further say: > > You need to create file type using FTYPE first and then associate this > TFYPE with executable using ASSOC command. Even though this process is > not very simple, you can use it. There is one HUGE disadvantage though > and I still don?t understand how is it possible ? both FTYPE and ASSOC > works only on per-machine level! Therefore you cannot configure > per-user associations using these tools! > > Currently, there is no way through the UI to change or edit the > user-specific file type associations stored in the > HKEY_CURRENT_USER\SOFTWARE\Classes registry key. If you want to do > this, you have to directly edit the registry or develop your own > UI to gain access to this information. > > Above you can see official Microsoft statement > about this issue. > So maybe the problem is that ftype and assoc report the HLM values, but CMD actually uses the HCU values. This actually would explain what I am seeing, and this is the piece of information I didn't understand that caused my confusion. So as long as everything is installed for "all users", then there is no confusion! I see I have less than 3 dozen extensions installed in HCU, and over 800 in HLM, so clearly I choose "all users" most of the time. It would be painstaking to see if the 3 dozen were all consistent or not, without writing a program to help. Clearly the launcher installer should offer the same choices as Python itself for installing for one or all users, rather than defaulting silently to one user. Better, the default should be inherited from the Python installation... if they are all consistent, and if permissions allow the modification of HLM settings. > Regards, > > Vinay Sajip So I can fix my machine, now that I understand what went wrong (delete py.exe entries from HCU, and put them in HLM instead). Then the other problem I have, is why py.exe launched py 2.6.4 instead of py 3.2.1 when 3.2.1 is newer, and I don't have a #! line. That is probably the defined behavior of the launcher, to prefer 2.x if 3 isn't specified. I'll have to reread the launcher PEP. In my environment, I would much prefer to use 3.x if 2.x isn't specified... so hopefully when I reread the launcher PEP, I'll discover a way to do that. (I recall one can put -3 on the command line, so I could do that in the py.exe ftype commands, but hopefully there is a way to tweak the py.ini to do that.) Thanks for your response, and help in creating the launcher. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinay_sajip at yahoo.co.uk Tue Jul 19 03:41:16 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Tue, 19 Jul 2011 01:41:16 +0000 (UTC) Subject: [Python-Dev] 3.2.1 encoding surprise References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> Message-ID: Glenn Linderman g.nevcal.com> writes: > So, then, there must be 2 problems... one in the launcher installer, > and the other being whatever happened to my machine. I think so. I know that Windows sometimes adds file type associations under HKCU which shadow the ones set up in HKLM, but I'm not sure why or how it happens. > Well, there is only one. But I can certainly remove it, and see > what happens. Okay, just making sure. > So what I had done was just double click launchsys.msi. Why should > there be command line parameters, rather than question/answer > dialogs? This is Windows after all. What does the parameter do, > that would be different than what double click did? Windows is unfortunately not consistent in its behaviour across different versions - if you Google for ALLUSERS you'll probably find the information that makes some people tear their hair out. So the ALLUSERS=1 forces Windows to install under HKLM ("for all users" as opposed to "just for me") and to fail if you don't have admin rights. Note that the ALLUSERS= is something introduced by Microsoft, not something launcher-specific. > I also still don't understand why CMD reports a different command > line via its assoc/ftype commands for the Python.File than actually > gets used by CMD to launch the .py file. I believe it's to do with multiple versions in the registry. > something that makes our situation more complicated) is the fact > that HKCR is in fact registry symlink and this registry key > doesn?t really exists on system. It is merge of two different keys > ? HKLM\Software\Classes and HKCU\Software\Classes. You probably > already guessed (correctly) that this is the way how you can > overwrite some settings on per-user basis ? HKCU key always wins. > > which is useful, but still doesn't explain the variant results from > ftype and launching behavior.} > They further say: > You need to create file type using FTYPE first and then > associate this TFYPE with executable using ASSOC command. Even > though this process is not very simple, you can use it. There is > one HUGE disadvantage though and I still don?t understand how is > it possible ? both FTYPE and ASSOC works only on per-machine > level! Therefore you cannot configure per-user associations > using these tools! I have certainly seen file type associations for .py in HKCU, whose presence there I can't explain, but it's probably wrongly created by some Windows software common in the ecosystem (if not Windows itself), which manifests in odd ways - search deadlybloodyserious.com for problems where command line arguments aren't passed to scripts because of bad ftype/assoc registry entries. I would advise not using ftype/assoc commands, as you certainly shouldn't need them for Python files if you install the launcher. > Currently, there is no way through the UI to change or edit > the user-specific file type associations stored in the > HKEY_CURRENT_USER\SOFTWARE\Classes registry key. If you want > to do this, you have to directly edit the registry or develop > your own UI to gain access to this information. > > Above you can see official Microsoft > statement about this issue. Yes, but some software in Windows (perhaps "select program from a list" or similar) definitely does set values in HKCU as a side-effect of doing something else, which comes back to bite you later. I have personally experienced the same problem as the blogger on deadlybloodyserious.com (which is how I came across the solution on that site - by Googling to see if anyone else had encountered it). > Clearly the launcher installer should offer the same choices as > Python itself for installing for one or all users, rather than > defaulting silently to one user. Better, the default should be > inherited from the Python installation... if they are all > consistent, and if permissions allow the modification of HLM > settings. Yes, but the launcher installer is beta software, and it's not yet clear exactly how the launcher will be packaged with Python 3.3. Clearly if packaged as an automatic installation with Python, it will use Python defaults. Note that ALLUSERS was introduced precisely because Microsoft didn't think things through initially with sufficient care, and there's no way to get consistent behaviour across XP and Vista/Windows 7. It's possible to force ALLUSERS=1 in the .msi, but this means that non-admin users can't try the launcher. Unfortunately AFAICT, there's no obvious solution which "just works", though if someone were to point out a solution it could certainly be tried. > So I can fix my machine, now that I understand what went wrong > (delete py.exe entries from HCU, and put them in HLM instead). > Then the other problem I have, is why py.exe launched py 2.6.4 > instead of py 3.2.1 when 3.2.1 is newer, and I don't have a #! > line. That is probably the defined behavior of the launcher, to > prefer 2.x if 3 isn't specified. I'll have to reread the launcher > PEP. In my environment, I would much prefer to use 3.x if 2.x isn't > specified... so hopefully when I reread the launcher PEP, I'll > discover a way to do that. (I recall one can put -3 on the command > line, so I could do that in the py.exe ftype commands, but hopefully > there is a way to tweak the py.ini to do that.) Yes, the PEP explains what you need to do to get Python 3 to be the default: use the py.ini file or use an environment variable. The use of py from the command line is merely a convenience for developers (as the PEP says) - it's better to rely on shebang lines together with settings in the .ini to get the behaviour you want. Regards, Vinay Sajip From v+python at g.nevcal.com Tue Jul 19 04:03:43 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Mon, 18 Jul 2011 19:03:43 -0700 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> Message-ID: <4E24E5FF.1010805@g.nevcal.com> On 7/18/2011 6:41 PM, Vinay Sajip wrote: > Yes, but the launcher installer is beta software, and it's not yet clear > exactly how the launcher will be packaged with Python 3.3. Clearly if > packaged as an automatic installation with Python, it will use Python > defaults. Sure, and that's why I'm trying it out now, to provide feedback. But because I couldn't tell from assoc and ftype commands that it had done anything, I was a bit surprised by the behaviors I did encounter when I encountered them. So now I've learned about the limits of ftype and assoc, sadly. And they only work for setting things in HLM if you "Run as administrator", by the way. Wonder if there is a third party command line tool which augments them for reading/setting HCU Classes? Will search. I know there is a command line registry tool somewhere, but specifying the paths correctly would be onerous. Would be nice if the default of Py 2.x or 3.x were an install question, perhaps. I know that only affects scripts without #! lines, but it would be nice to limit the required creation of those to the smaller class of preexisting scripts on a particular installation. > Note that ALLUSERS was introduced precisely because Microsoft didn't think > things through initially with sufficient care, and there's no way to get > consistent behaviour across XP and Vista/Windows 7. It's possible to force > ALLUSERS=1 in the .msi, but this means that non-admin users can't try the > launcher. Unfortunately AFAICT, there's no obvious solution which "just > works", though if someone were to point out a solution it could certainly be > tried. Could be, with .msi and/or .cab installers. I have no knowledge of them, except that when I first needed to use an installer to package things, I read a bit about them, and determined that they are too complex for my brain. I found NSIS, still complex, but it is much friendlier to my brain, but it seems to handle this case correctly, giving the installer creator the option of asking for or demanding elevation, and exposing the answer to the rest of the logic of the installer, so it can do different things for those cases if it desires. -------------- next part -------------- An HTML attachment was scrubbed... URL: From v+python at g.nevcal.com Tue Jul 19 05:27:15 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Mon, 18 Jul 2011 20:27:15 -0700 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: <4E24E5FF.1010805@g.nevcal.com> References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E24E5FF.1010805@g.nevcal.com> Message-ID: <4E24F993.6090406@g.nevcal.com> On 7/18/2011 7:03 PM, Glenn Linderman wrote: > Wonder if there is a third party command line tool which augments them > for reading/setting HCU Classes? Will search. I know there is a > command line registry tool somewhere, but specifying the paths > correctly would be onerous. XP+ has REG and REGINI commands which can manipulate the registry from the command line, without resorting to third party applications. So a batch file could wrap them, to avoid mistyping keys, and one could make an FTYPE.CMD and ASSOC.CMD (I haven't, at least not yet). Also although specific to Vim, is addressing the general problem, and comments by readers on that page seem relevant: > The following is a "for reader's information" comment; not a > suggestion that the tip be changed. In classic Microsoft style, using > HKEY_CLASSES_ROOT is ambiguous. I do not know the details, but on at > least some systems (after Windows 9x), I think the following is correct: > > HKEY_LOCAL_MACHINE\Software\Classes : for all users > HKEY_CURRENT_USER\Software\Classes : for interactive user > HKEY_CLASSES_ROOT : merged view of above (and is used by Win9x > apps ) > > I don't know what happens if you write to HKEY_CLASSES_ROOT (which > does not exist). Perhaps (like installing some apps), if you are in > the Administrators group, you will write to HKLM, otherwise you will > write to HKCU. JohnBeckett > 08:24, 2 July 2009 (UTC) > > ------------------------------------------------------------------------ > > This is correct as far as I know. I wanted to include some information > about how HKCR is a merged view of HKLM/Software/Classes and > HKCU/Software/Classes, but doing so would beg information such as > "what happens when I edit an entry that actually exists in HKCU?" etc. > From experimentation, it seems that creating /new/ keys in this area > will always create it in HKLM (system-wide), and ftype and assoc > certainly do that, but I don't really have any documentation of that > fact, and I don't know what it will do for limited-privilege accounts. > I also don't know what happens when you edit an existing key, but I > imagine it will "do the right thing" and keep the original where it > was. I don't know this for a fact though. For these reasons, I left > out that tidbit. But if we can answer some of these questions, it > would be a good thing to include. > So attempts by applications to use the HKEY_CLASSES_ROOT directly may result in variant behavior depending on the privilege of the application. My overall opinion? It is sad that M$ even invented the registry. -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Tue Jul 19 17:00:57 2011 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 19 Jul 2011 16:00:57 +0100 Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise) Message-ID: On 19 July 2011 02:41, Vinay Sajip wrote: > The use of py from the command line is merely a convenience for developers (as > the PEP says) - it's better to rely on shebang lines together with settings in > the .ini to get the behaviour you want. But it's a *huge* convenience for running multiple Python versions, particularly as no existing Python versions install executables with the version in the name (python3.exe, python3.2.exe, etc).And BAT files aren't a suitable option (I'll rant about the issues with BAT files if you want, but I recommend you don't ask :-)) Being able to say py -3, py -2.7, etc, rather than having to hack PATH, create renamed copies of exes, etc, is arguably more of a benefit to me than shebang support. This may explain why I'd like to see a command-line means of invoking custom commands. Something like py.exe looking at an initial argument, and if it's of the form "-cmd" for a command in py.ini, then run that command, passing remaining arguments just as for py -3. (Maybe --cmd to match standard long option usage would be better?) Presumably, if this idea is to go anywhere, it would need adding to the PEP. Mark, do you think it would be useful? It makes it possible to invoke IronPython/Jython in the same manner as CPython. (And, of course, to abuse things so that "py -perl" runs Perl, but no-one would do that :-)) Paul. From solipsis at pitrou.net Tue Jul 19 17:16:47 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 19 Jul 2011 17:16:47 +0200 Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise) References: Message-ID: <20110719171647.03f9f9c9@pitrou.net> On Tue, 19 Jul 2011 16:00:57 +0100 Paul Moore wrote: > On 19 July 2011 02:41, Vinay Sajip wrote: > > The use of py from the command line is merely a convenience for developers (as > > the PEP says) - it's better to rely on shebang lines together with settings in > > the .ini to get the behaviour you want. > > But it's a *huge* convenience for running multiple Python versions, > particularly as no existing Python versions install executables with > the version in the name (python3.exe, python3.2.exe, etc). Perhaps this could be changed? As far as I can see, python.exe is a small executable around ~25KB (all the code being in the DLL), so there doesn't seem to be any harm to make a copy of it named either pythonXY.exe or pythonX.Y.exe. Regards Antoine. From p.f.moore at gmail.com Tue Jul 19 18:21:30 2011 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 19 Jul 2011 17:21:30 +0100 Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise) In-Reply-To: <20110719171647.03f9f9c9@pitrou.net> References: <20110719171647.03f9f9c9@pitrou.net> Message-ID: On 19 July 2011 16:16, Antoine Pitrou wrote: > On Tue, 19 Jul 2011 16:00:57 +0100 > Paul Moore wrote: > >> On 19 July 2011 02:41, Vinay Sajip wrote: >> > The use of py from the command line is merely a convenience for developers (as >> > the PEP says) - it's better to rely on shebang lines together with settings in >> > the .ini to get the behaviour you want. >> >> But it's a *huge* convenience for running multiple Python versions, >> particularly as no existing Python versions install executables with >> the version in the name (python3.exe, python3.2.exe, etc). > > Perhaps this could be changed? As far as I can see, python.exe is > a small executable around ~25KB (all the code being in the DLL), so > there doesn't seem to be any harm to make a copy of it named either > pythonXY.exe or pythonX.Y.exe. I'm sure it could (and in fact, I thought that this had been discussed some time back and it may even be already happening in 3.3) but it doesn't help for existing versions, where the py.exe launcher does. So as a longer-term solution, supplying pythonXY.exe binaries may be useful (depending on how PEP 397 progresses), but the benefits won't be for quite some time. (And there's still the question of what gets put on PATH by default even if version-specific binaries exist). It's a topic worthy of discussion, but I suspect that in actual fact, PEP 397 may offer a more complete solution to the various Windows installation niggles. Two questions: 1. What level of support is there for PEP 397? If it's unlikely to get accepted, there's little point in basing a solution on it. 2. Would it be worth extending the goals of the PEP to make simplifying command line usage an explicit goal? Or is it better to keep PEP 397 focused on one thing and have a separate PEP covering such further extensions to the PEP 397 launcher? Paul. From 98k.master at gmail.com Tue Jul 19 20:16:46 2011 From: 98k.master at gmail.com (Timothy D. Kadich) Date: Tue, 19 Jul 2011 11:16:46 -0700 Subject: [Python-Dev] knee.py import hook in 2.6 Message-ID: Hi, I'm trying to use the import hook in Python2.6, but I'm having a problem. It doesn't work for numpy. My error is such: > >>> import knee > >>> import numpy > Traceback (most recent call last): > File "", line 1, in > File "knee.py", line 16, in import_hook > q, tail = find_head_package(parent, name) > File "knee.py", line 52, in find_head_package > q = import_module(head, qname, parent) > File "knee.py", line 101, in import_module > m = imp.load_module(fqname, fp, pathname, stuff) > File "/usr/apps/python2.6/lib/python2.6/site-packages/numpy/__init__.py", > line 130, in > import add_newdocs > File "knee.py", line 16, in import_hook > q, tail = find_head_package(parent, name) > File "knee.py", line 52, in find_head_package > q = import_module(head, qname, parent) > File "knee.py", line 101, in import_module > m = imp.load_module(fqname, fp, pathname, stuff) > File > "/usr/apps/python2.6/lib/python2.6/site-packages/numpy/add_newdocs.py", line > 9, in > from lib import add_newdoc > File "knee.py", line 16, in import_hook > q, tail = find_head_package(parent, name) > File "knee.py", line 52, in find_head_package > q = import_module(head, qname, parent) > File "knee.py", line 101, in import_module > m = imp.load_module(fqname, fp, pathname, stuff) > File > "/usr/apps/python2.6/lib/python2.6/site-packages/numpy/lib/__init__.py", > line 4, in > from type_check import * > File "knee.py", line 16, in import_hook > q, tail = find_head_package(parent, name) > File "knee.py", line 52, in find_head_package > q = import_module(head, qname, parent) > File "knee.py", line 101, in import_module > m = imp.load_module(fqname, fp, pathname, stuff) > File > "/usr/apps/python2.6/lib/python2.6/site-packages/numpy/lib/type_check.py", > line 8, in > import numpy.core.numeric as _nx > File "knee.py", line 17, in import_hook > m = load_tail(q, tail) > File "knee.py", line 68, in load_tail > m = import_module(head, mname, m) > File "knee.py", line 101, in import_module > m = imp.load_module(fqname, fp, pathname, stuff) > File > "/usr/apps/python2.6/lib/python2.6/site-packages/numpy/core/__init__.py", > line 6, in > import umath > File "knee.py", line 16, in import_hook > q, tail = find_head_package(parent, name) > File "knee.py", line 52, in find_head_package > q = import_module(head, qname, parent) > File "knee.py", line 101, in import_module > m = imp.load_module(fqname, fp, pathname, stuff) > TypeError: import_hook() takes at most 4 arguments (5 given) So I don't know what is going on, unless a "self" is being passed along the way. (which seems like it could happen when looking at __import__ in the source) Can any of you identify my problem or let me know of a fixed import hook? Thank you, Timothy D. Kadich -------------- next part -------------- An HTML attachment was scrubbed... URL: From phd at phdru.name Tue Jul 19 21:08:54 2011 From: phd at phdru.name (Oleg Broytman) Date: Tue, 19 Jul 2011 23:08:54 +0400 Subject: [Python-Dev] knee.py import hook in 2.6 In-Reply-To: References: Message-ID: <20110719190854.GA5691@iskra.aviel.ru> Hello. We are sorry but we cannot help you. This mailing list is to work on developing Python (adding new features to Python itself and fixing bugs); if you're having problems learning, understanding or using Python, please find another forum. Probably python-list/comp.lang.python mailing list/news group is the best place; there are Python developers who participate in it; you may get a faster, and probably more complete, answer there. See http://www.python.org/community/ for other lists/news groups/fora. Thank you for understanding. On Tue, Jul 19, 2011 at 11:16:46AM -0700, Timothy D. Kadich wrote: > I'm trying to use the import hook in Python2.6, but I'm having a problem. It > doesn't work for numpy. My error is such: [skip] > > TypeError: import_hook() takes at most 4 arguments (5 given) Seems like import_hook is from an older version of Python. Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From solipsis at pitrou.net Tue Jul 19 22:59:54 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 19 Jul 2011 22:59:54 +0200 Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise) In-Reply-To: References: <20110719171647.03f9f9c9@pitrou.net> Message-ID: <20110719225954.0def2f75@pitrou.net> On Tue, 19 Jul 2011 17:21:30 +0100 Paul Moore wrote: > > Two questions: > 1. What level of support is there for PEP 397? If it's unlikely to get > accepted, there's little point in basing a solution on it. It only needs support from our Windows users or developers. It is doubtful than any Linux or OS X user would oppose it on purely platonic grounds. I myself, as a casual Windows user, understand that the current situation is not optimal and believe that any improvement is welcome. Practically, if Mark Hammond is satisfied with his own proposal, if Martin doesn't oppose it, and if other users like you say it's a good step forward, then I don't see any reason for it *not* to be accepted. (if you want an explicit +1, here it is :-)) > 2. Would it be worth extending the goals of the PEP to make > simplifying command line usage an explicit goal? Or is it better to > keep PEP 397 focused on one thing and have a separate PEP covering > such further extensions to the PEP 397 launcher? I have no opinion about that. Regards Antoine. From rdmurray at bitdance.com Tue Jul 19 23:21:39 2011 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 19 Jul 2011 17:21:39 -0400 Subject: [Python-Dev] email-6.0.0.a1 Message-ID: <20110719212139.D5D732500D5@webabinitio.net> OK, so I've released the first iteration of the email6 package on pypi as email-6.0.0a1. After install you import it as email6. This will allow anyone curious and/or motivated to test it out under Python 3.2. I'm especially interested in anyone with a working program that uses email in 3.2: it should be completely backward compatible, and if it isn't I want to know ASAP.[*] I've also opened issue 12586 for review of the delta between default and the code that is in the release. I'd like to check the code in to default and continue to work on it from there. As I said in the issue comments: "When we originally planned out email6 we thought we'd be making a "compatibility break" with backward compatibility shims. As things have turned out the work is more a matter of incremental improvement of the API while maintaining the old API, and thus it seems reasonable to me to work on it directly in default rather than continue to work on it in a separate feature branch." Assuming, that is, that the general approach represented by *this* delta is accepted. What this delta adds to email is a conversion to handling all headers as full blown objects (as opposed to strings, tuples of strings, or Header objects, depending on context). The object type is a subclass of str, so the headers act like strings if you don't use their additional API. The basic additional API is that a 'source' attribute contains the text the generator read from the input source, and a 'value' attribute that contains the value with all the Content-Transfer-Encoding stuff undone so that you have a real unicode string. By changing a policy setting, you can have that value as the string value of the header. You can also assign a string with non-ASCII characters to a header, and the right thing will happen. (Well, eventually it will happen...right now it only works correctly for unstructured headers). Further, Date headers have a datetime attribute (and accept being set to a datetime), and address headers have attributes for accessing the individual addresses in the header. Other structured headers will eventually grow additional attributes as well. The general approach has been discussed with and approved by the email-sig, but all comments are welcome. I know there's room for bikeshedding on some aspects of the API; in some cases I've dome some "placeholder" stuff pending a more complete solution to certain design goals. I have a big project in the offing over the next couple months. QNX is still fully behind the funding for email6 development, but I probably won't be able to complete it until the fall. So I'd like to get this chunk (the biggest chunk of new code, considering the size of the parser) reviewed and checked in if possible. I'll keep working on the bits of functionality that aren't quite complete and the bugs that I know are there until my big project kicks off, but I wanted to release/post now so that there might be a chance of some review happening while I still have time to respond quickly to the feedback. -- R. David Murray http://www.bitdance.com [*] I believe that if you try to use an email6 Message object with the 3.2 mailbox module you will run in to some trouble, but I think it ought to be possible to make it work with the right magic :) PS: I don't have much experience writing parsers, so I'm expecting some critical comments about my parser design. It had to be a custom parser since otherwise I'd be blocked on waiting for some other software to get accepted into the stdlib, but it certainly wound up being a bigger chunk of code than I expected when I started writing it. From ncoghlan at gmail.com Wed Jul 20 01:08:20 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 20 Jul 2011 09:08:20 +1000 Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise) In-Reply-To: <20110719225954.0def2f75@pitrou.net> References: <20110719171647.03f9f9c9@pitrou.net> <20110719225954.0def2f75@pitrou.net> Message-ID: On Wed, Jul 20, 2011 at 6:59 AM, Antoine Pitrou wrote: > (if you want an explicit +1, here it is :-)) FWIW, +1 from me as well, but keep in mind that I actively avoid programming on Windows (although I'm happy enough using it as a gaming platform) Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From skippy.hammond at gmail.com Wed Jul 20 02:15:31 2011 From: skippy.hammond at gmail.com (Mark Hammond) Date: Wed, 20 Jul 2011 10:15:31 +1000 Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise) In-Reply-To: References: Message-ID: <4E261E23.3020005@gmail.com> On 20/07/2011 1:00 AM, Paul Moore wrote: > On 19 July 2011 02:41, Vinay Sajip wrote: >> The use of py from the command line is merely a convenience for developers (as >> the PEP says) - it's better to rely on shebang lines together with settings in >> the .ini to get the behaviour you want. > > But it's a *huge* convenience for running multiple Python versions, > particularly as no existing Python versions install executables with > the version in the name (python3.exe, python3.2.exe, etc).And BAT > files aren't a suitable option (I'll rant about the issues with BAT > files if you want, but I recommend you don't ask :-)) > > Being able to say py -3, py -2.7, etc, rather than having to hack > PATH, create renamed copies of exes, etc, is arguably more of a > benefit to me than shebang support. Ditto for me. > This may explain why I'd like to see a command-line means of invoking > custom commands. Something like py.exe looking at an initial argument, > and if it's of the form "-cmd" for a command in py.ini, then run that > command, passing remaining arguments just as for py -3. (Maybe --cmd > to match standard long option usage would be better?) > > Presumably, if this idea is to go anywhere, it would need adding to > the PEP. Mark, do you think it would be useful? I doubt I will find it useful - but I'm on record as saying I wont find the custom command support itself useful :) But similarly with that support, evidence that enough people *will* find it useful is enough for me to support the concept. My current thinking re the PEP is to make it much smaller - just describe the concepts and try to avoid as much implementation detail as possible - I see no reason the PEP needs to take a stance on issues like that - this feature really could be treated just like any other feature request in Python - a loose consensus and acceptable patch is all that is needed. Mark From tjreedy at udel.edu Wed Jul 20 04:21:31 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 19 Jul 2011 22:21:31 -0400 Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise) In-Reply-To: References: <20110719171647.03f9f9c9@pitrou.net> Message-ID: On 7/19/2011 12:21 PM, Paul Moore wrote: > On 19 July 2011 16:16, Antoine Pitrou wrote: >> On Tue, 19 Jul 2011 16:00:57 +0100 >> Perhaps this could be changed? As far as I can see, python.exe is >> a small executable around ~25KB (all the code being in the DLL), so >> there doesn't seem to be any harm to make a copy of it named either >> pythonXY.exe or pythonX.Y.exe. > > I'm sure it could (and in fact, I thought that this had been discussed > some time back and it may even be already happening in 3.3) but it > doesn't help for existing versions, where the py.exe launcher does. So > as a longer-term solution, supplying pythonXY.exe binaries may be > useful (depending on how PEP 397 progresses), but the benefits won't > be for quite some time. (And there's still the question of what gets > put on PATH by default even if version-specific binaries exist). Suppose for Windows there were one '.../python' directory wherever the user first asks it to be put and that all pythons, not just cpython, are installed in directories below that and that the small startup file is copied into or linked from the python directory. Then the one python directory could be put on the path and left there and never removed by any python de-installer (unless perhaps it check that there are no subdirs and *asks* the user. -- Terry Jan Reedy From pje at telecommunity.com Wed Jul 20 05:58:55 2011 From: pje at telecommunity.com (P.J. Eby) Date: Tue, 19 Jul 2011 23:58:55 -0400 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" Message-ID: <20110720040505.400E23A4116@sparrow.telecommunity.com> So, over on the Import-SIG, we were talking about the implementation and terminology for PEP 382, and it became increasingly obvious that things were, well, not entirely okay in the "implementation is easy to explain" department. Anyway, to make a long story short, we came up with an alternative implementation plan that actually solves some other problems besides the one that PEP 382 sets out to solve, and whose implementation a bit is easier to explain. (In fact, for users coming from various other languages, it hardly needs any explanation at all.) However, for long-time users of Python, the approach may require a bit more justification, which is why roughly 2/3rds of the PEP consists of a detailed rationale, specification overview, rejected alternatives, and backwards-compatibility discussion... which is still a lot less verbiage than reading through the lengthy Import-SIG threads that led up to the proposal. ;-) (The remaining 1/3rd of the PEP is the short, sweet, and easy-to-explain implementation detail.) Anyway, the PEP has already been discussed on the Import-SIG, and is proposed as an alternative to PEP 382 ("Namespace packages"). We expect, however, that many people will be interested in it for reasons having little to do with the namespace packaging use case. So, we would like to submit this for discussion, hole-finding, and eventual Pronouncement. As Barry put it, "I think it's certainly worthy of posting to python-dev to see if anybody else can shoot holes in it, or come up with useful solutions to open questions. I'll be very interested to see Guido's reaction to it. :)" So, without further ado, here it is: PEP: XXX Title: Simplified Package Layout and Partitioning Version: $Revision$ Last-Modified: $Date$ Author: P.J. Eby Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 12-Jul-2011 Python-Version: 3.3 Post-History: Replaces: 382 Abstract ======== This PEP proposes an enhancement to Python's package importing to: * Surprise users of other languages less, * Make it easier to convert a module into a package, and * Support dividing packages into separately installed components (ala "namespace packages", as described in PEP 382) The proposed enhancements do not change the semantics of any currently-importable directory layouts, but make it possible for packages to use a simplified directory layout (that is not importable currently). However, the proposed changes do NOT add any performance overhead to the importing of existing modules or packages, and performance for the new directory layout should be about the same as that of previous "namespace package" solutions (such as ``pkgutil.extend_path()``). The Problem =========== .. epigraph:: "Most packages are like modules. Their contents are highly interdependent and can't be pulled apart. [However,] some packages exist to provide a separate namespace. ... It should be possible to distribute sub-packages or submodules of these [namespace packages] independently." -- Jim Fulton, shortly before the release of Python 2.3 [1]_ When new users come to Python from other languages, they are often confused by Python's packaging semantics. At Google, for example, Guido received complaints from "a large crowd with pitchforks" [2]_ that the requirement for packages to contain an ``__init__`` module was a "misfeature", and should be dropped. In addition, users coming from languages like Java or Perl are sometimes confused by a difference in Python's import path searching. In most other languages that have a similar path mechanism to Python's ``sys.path``, a package is merely a namespace that contains modules or classes, and can thus be spread across multiple directories in the language's path. In Perl, for instance, a ``Foo::Bar`` module will be searched for in ``Foo/`` subdirectories all along the module include path, not just in the first such subdirectory found. Worse, this is not just a problem for new users: it prevents *anyone* from easily splitting a package into separately-installable components. In Perl terms, it would be as if every possible ``Net::`` module on CPAN had to be bundled up and shipped in a single tarball! For that reason, various workarounds for this latter limitation exist, circulated under the term "namespace packages". The Python standard library has provided one such workaround since Python 2.3 (via the ``pkgutil.extend_path()`` function), and the "setuptools" package provides another (via ``pkg_resources.declare_namespace()``). The workarounds themselves, however, fall prey to a *third* issue with Python's way of laying out packages in the filesystem. Because a package *must* contain an ``__init__`` module, any attempt to distribute modules for that package must necessarily include that ``__init__`` module, if those modules are to be importable. However, the very fact that each distribution of modules for a package must contain this (duplicated) ``__init__`` module, means that OS vendors who package up these module distributions must somehow handle the conflict caused by several module distributions installing that ``__init__`` module to the same location in the filesystem. This led to the proposing of PEP 382 ("Namespace Packages") - a way to signal to Python's import machinery that a directory was importable, using unique filenames per module distribution. However, there was more than one downside to this approach. Performance for all import operations would be affected, and the process of designating a package became even more complex. New terminology had to be invented to explain the solution, and so on. As terminology discussions continued on the Import-SIG, it soon became apparent that the main reason it was so difficult to explain the concepts related to "namespace packages" was because Python's current way of handling packages is somewhat underpowered, when compared to other languages. That is, in other popular languages with package systems, no special term is needed to describe "namespace packages", because *all* packages generally behave in the desired fashion. Rather than being an isolated single directory with a special marker module (as in Python), packages in other languages are typically just the *union* of appropriately-named directories across the *entire* import or inclusion path. In Perl, for example, the module ``Foo`` is always found in a ``Foo.pm`` file, and a module ``Foo::Bar`` is always found in a ``Foo/Bar.pm`` file. (In other words, there is One Obvious Way to find the location of a particular module.) This is because Perl considers a module to be *different* from a package: the package is purely a *namespace* in which other modules may reside, and is only *coincidentally* the name of a module as well. In current versions of Python, however, the module and the package are more tightly bound together. ``Foo`` is always a module -- whether it is found in ``Foo.py`` or ``Foo/__init__.py`` -- and it is tightly linked to its submodules (if any), which *must* reside in the exact same directory where the ``__init__.py`` was found. On the positive side, this design choice means that a package is quite self-contained, and can be installed, copied, etc. as a unit just by performing an operation on the package's root directory. On the negative side, however, it is non-intuitive for beginners, and requires a more complex step to turn a module into a package. If ``Foo`` begins its life as ``Foo.py``, then it must be moved and renamed to ``Foo/__init__.py``. Conversely, if you intend to create a ``Foo.Bar`` module from the start, but have no particular module contents to put in ``Foo`` itself, then you have to create an empty and seemingly-irrelevant ``Foo/__init__.py`` file, just so that ``Foo.Bar`` can be imported. (And these issues don't just confuse newcomers to the language, either: they annoy many experienced developers as well.) So, after some discussion on the Import-SIG, this PEP was created as an alternative to PEP \382, in an attempt to solve *all* of the above problems, not just the "namespace package" use cases. And, as a delightful side effect, the solution proposed in this PEP does not affect the import performance of ordinary modules or self-contained (i.e. ``__init__``-based) packages. The Solution ============ In the past, various proposals have been made to allow more intuitive approaches to package directory layout. However, most of them failed because of an apparent backward-compatibility problem. That is, if the requirement for an ``__init__`` module were simply dropped, it would open up the possibility for a directory named, say, ``string`` on ``sys.path``, to block importing of the standard library ``string`` module. Paradoxically, however, the failure of this approach does *not* arise from the elimination of the ``__init__`` requirement! Rather, the failure arises because the underlying approach takes for granted that a package is just ONE thing, instead of two. In truth, a package comprises two separate, but related entities: a module (with its own, optional contents), and a *namespace* where *other* modules or packages can be found. In current versions of Python, however, the module part (found in ``__init__``) and the namespace for submodule imports (represented by the ``__path__`` attribute) are both initialized at the same time, when the package is first imported. And, if you assume this is the *only* way to initialize these two things, then there is no way to drop the need for an ``__init__`` module, while still being backwards-compatible with existing directory layouts. After all, as soon as you encounter a directory on ``sys.path`` matching the desired name, that means you've "found" the package, and must stop searching, right? Well, not quite. A Thought Experiment -------------------- Let's hop into the time machine for a moment, and pretend we're back in the early 1990s, shortly before Python packages and ``__init__.py`` have been invented. But, imagine that we *are* familiar with Perl-like package imports, and we want to implement a similar system in Python. We'd still have Python's *module* imports to build on, so we could certainly conceive of having ``Foo.py`` as a parent ``Foo`` module for a ``Foo`` package. But how would we implement submodule and subpackage imports? Well, if we didn't have the idea of ``__path__`` attributes yet, we'd probably just search ``sys.path`` looking for ``Foo/Bar.py``. But we'd *only* do it when someone actually tried to *import* ``Foo.Bar``. NOT when they imported ``Foo``. And *that* lets us get rid of the backwards-compatibility problem of dropping the ``__init__`` requirement, back here in 2011. How? Well, when we ``import Foo``, we're not even *looking* for ``Foo/`` directories on ``sys.path``, because we don't *care* yet. The only point at which we care, is the point when somebody tries to actually import a submodule or subpackage of ``Foo``. That means that if ``Foo`` is a standard library module (for example), and I happen to have a ``Foo`` directory on ``sys.path`` (without an ``__init__.py``, of course), then *nothing breaks*. The ``Foo`` module is still just a module, and it's still imported normally. Self-Contained vs. "Virtual" Packages ------------------------------------- Of course, in today's Python, trying to ``import Foo.Bar`` will fail if ``Foo`` is just a ``Foo.py`` module (and thus lacks a ``__path__`` attribute). So, this PEP proposes to *dynamically* create a ``__path__``, in the case where one is missing. That is, if I try to ``import Foo.Bar`` the proposed change to the import machinery will notice that the ``Foo`` module lacks a ``__path__``, and will therefore try to *build* one before proceeding. And it will do this by making a list of all the existing ``Foo/`` subdirectories of the directories listed in ``sys.path``. If the list is empty, the import will fail with ``ImportError``, just like today. But if the list is *not* empty, then it is saved in a new ``Foo.__path__`` attribute, making the module a "virtual package". That is, because it now has a valid ``__path__``, we can proceed to import submodules or subpackages in the normal way. Now, notice that this change does not affect "classic", self-contained packages that have an ``__init__`` module in them. Such packages already *have* a ``__path__`` attribute (initialized at import time) so the import machinery won't try to create another one later. This means that (for example) the standard library ``email`` package will not be affected in any way by you having a bunch of unrelated directories named ``email`` on ``sys.path``. (Even if they contain ``*.py`` files.) But it *does* mean that if you want to turn your ``Foo`` module into a ``Foo`` package, all you have to do is add a ``Foo/`` directory somewhere on ``sys.path``, and start adding modules to it. But what if you only want a "namespace package"? That is, a package that is *only* a namespace for various separately-distributed submodules and subpackages? For example, if you're Zope Corporation, distributing dozens of separate tools like ``zc.buildout``, each in packages under the ``zc`` namespace, you don't want to have to make and include an empty ``zc.py`` in every tool you ship. (And, if you're a Linux or other OS vendor, you don't want to deal with the package installation conflicts created by trying to install ten copies of ``zc.py`` to the same location!) No problem. All we have to do is make one more minor tweak to the import process: if the "classic" import process fails to find a self-contained module or package (e.g., if ``import zc`` fails to find a ``zc.py`` or ``zc/__init__.py``), then we once more try to build a ``__path__`` by searching for all the ``zc/`` directories on ``sys.path``, and putting them in a list. If this list is empty, we raise ``ImportError``. But if it's non-empty, we create an empty ``zc`` module, and put the list in ``zc.__path__``. Congratulations: ``zc`` is now a namespace-only, "pure virtual" package! It has no module contents, but you can still import submodules and subpackages from it, regardless of where they're located on ``sys.path``. (By the way, both of these additions to the import protocol (i.e. the dynamically-added ``__path__``, and dynamically-created modules) apply recursively to child packages, using the parent package's ``__path__`` in place of ``sys.path`` as a basis for generating a child ``__path__``. This means that self-contained and virtual packages can contain each other without limitation, with the caveat that if you put a virtual package inside a self-contained one, it's gonna have a really short ``__path__``!) Backwards Compatibility and Performance --------------------------------------- Notice that these two changes *only* affect import operations that today would result in ``ImportError``. As a result, the performance of imports that do not involve virtual packages is unaffected, and potential backward compatibility issues are very restricted. Today, if you try to import submodules or subpackages from a module with no ``__path__``, it's an immediate error. And of course, if you don't have a ``zc.py`` or ``zc/__init__.py`` somewhere on ``sys.path`` today, ``import zc`` would likewise fail. Thus, the only potential backwards-compatibility issues are: 1. Tools that expect package directories to have an ``__init__`` module, that expect directories without an ``__init__`` module to be unimportable, or that expect ``__path__`` attributes to be static, will not recognize virtual packages as packages. (In practice, this just means that tools will need updating to support virtual packages, e.g. by using ``pkgutil.walk_modules()`` instead of using hardcoded filesystem searches.) 2. Code that *expects* certain imports to fail may now do something unexpected. This should be fairly rare in practice, as most sane, non-test code does not import things that are expected not to exist! The biggest likely exception to the above would be when a piece of code tries to check whether some package is installed by importing it. If this is done *only* by importing a top-level module (i.e., not checking for a ``__version__`` or some other attribute), *and* there is a directory of the same name as the sought-for package on ``sys.path`` somewhere, *and* the package is not actually installed, then such code could *perhaps* be fooled into thinking a package is installed that really isn't. However, even in the rare case where all these conditions line up to happen at once, the failure is more likely to be annoying than damaging. In most cases, after all, the code will simply fail a little later on, when it actually tries to DO something with the imported (but empty) module. (And code that checks ``__version__`` attributes or for the presence of some desired function, class, or module in the package will not see a false positive result in the first place.) Meanwhile, tools that expect to locate packages and modules by walking a directory tree can be updated to use the existing ``pkgutil.walk_modules()`` API, and tools that need to inspect packages in memory should use the other APIs described in the `Standard Library Changes/Additions`_ section below. Specification ============= Two changes are made to the existing import process. First, the built-in ``__import__`` function must not raise an ``ImportError`` when importing a submodule of a module with no ``__path__``. Instead, it must attempt to *create* a ``__path__`` attribute for the parent module first, as described in `__path__ creation`_, below. Second, if searching ``sys.meta_path`` and ``sys.path`` (or a parent package ``__path__``) fails to find a module being imported, the import process must attempt to create a ``__path__`` attribute for the missing module. If the attempt succeeds, an empty module is created and its ``__path__`` is set. Otherwise, importing fails. In both of the above cases, if a non-empty ``__path__`` is created, the name of the module whose ``__path__`` was created is added to ``sys.virtual_packages`` -- an initially-empty ``set()`` of package names. (This way, code that extends ``sys.path`` at runtime can find out what virtual packages are currently imported, and thereby add any new subdirectories to those packages' ``__path__`` attributes. See `Standard Library Changes/Additions`_ below for more details.) Conversely, if an empty ``__path__`` results, an ``ImportError`` is immediately raised, and the module is not created or changed, nor is its name added to ``sys.virtual_packages``. ``__path__`` Creation --------------------- A virtual ``__path__`` is created by obtaining a PEP 302 "importer" object for each of the path entries found in ``sys.path`` (for a top-level module) or the parent ``__path__`` (for a submodule). (Note: because ``sys.meta_path`` importers are not associated with ``sys.path`` or ``__path__`` entry strings, such importers do *not* participate in this process.) Each importer is checked for a ``get_subpath()`` method, and if present, the method is called with the full name of the module/package the ``__path__`` is being constructed for. The return value is either a string representing a subdirectory for the requested package, or ``None`` if no such subdirectory exists. The strings returned by the importers are added to the ``__path__`` being built, in the same order as they are found. (``None`` values and missing ``get_subpath()`` methods are simply skipped.) In Python code, the algorithm would look something like this:: def get_virtual_path(modulename, parent_path=None): if parent_path is None: parent_path = sys.path path = [] for entry in parent_path: # Obtain a PEP 302 importer object - see pkgutil module importer = pkgutil.get_importer(entry) if hasattr(importer, 'get_subpath'): subpath = importer.get_subpath(modulename) if subpath is not None: path.append(subpath) return path And a function like this one should be exposed in the standard library as e.g. ``imp.get_virtual_path()``, so that people creating ``__import__`` replacements or ``sys.meta_path`` hooks can reuse it. Standard Library Changes/Additions ---------------------------------- The ``pkgutil`` module should be updated to handle this specification appropriately, including any necessary changes to ``extend_path()``, ``iter_modules()``, etc. Specifically the proposed changes and additions to ``pkgutil`` are: * A new ``extend_virtual_paths(path_entry)`` function, to extend existing, already-imported virtual packages' ``__path__`` attributes to include any portions found in a new ``sys.path`` entry. This function should be called by applications extending ``sys.path`` at runtime, e.g. when adding a plugin directory or an egg to the path. The implementation of this function does a simple top-down traversal of ``sys.virtual_packages``, and performs any necessary ``get_subpath()`` calls to identify what path entries need to be added to each package's ``__path__``, given that `path_entry` has been added to ``sys.path``. (Or, in the case of sub-packages, adding a derived subpath entry, based on their parent namespace's ``__path__``.) * A new ``iter_virtual_packages(parent='')`` function to allow top-down traversal of virtual packages in ``sys.virtual_packages``, by yielding the child virtual packages of `parent`. For example, calling ``iter_virtual_packages("zope")`` might yield ``zope.app`` and ``zope.products`` (if they are imported virtual packages listed in ``sys.virtual_packages``), but **not** ``zope.foo.bar``. (This function is needed to implement ``extend_virtual_paths()``, but is also potentially useful for other code that needs to inspect imported virtual packages.) * ``ImpImporter.iter_modules()`` should be changed to also detect and yield the names of modules found in virtual packages. In addition to the above changes, the ``zipimport`` importer should have its ``iter_modules()`` implementation similarly changed. (Note: current versions of Python implement this via a shim in ``pkgutil``, so technically this is also a change to ``pkgutil``.) Last, but not least, the ``imp`` module (or ``importlib``, if appropriate) should expose the algorithm described in the `__path__ creation`_ section above, as a ``get_virtual_path(modulename, parent_path=None)`` function, so that creators of ``__import__`` replacements can use it. Implementation Notes -------------------- For users, developers, and distributors of virtual packages: * While virtual packages are easy to set up and use, there is still a time and place for using self-contained packages. While it's not strictly necessary, adding an ``__init__`` module to your self-contained packages lets users of the package (and Python itself) know that *all* of the package's code will be found in that single subdirectory. In addition, it lets you define ``__all__``, expose a public API, provide a package-level docstring, and do other things that make more sense for a self-contained project than for a mere "namespace" package. * ``sys.virtual_packages`` is allowed to contain non-existent or not-yet-imported package names; code that uses its contents should not assume that every name in this set is also present in ``sys.modules`` or that importing the name will necessarily succeed. * If you are changing a currently self-contained package into a virtual one, it's important to note that you can no longer use its ``__file__`` attribute to locate data files stored in a package directory. Instead, you must search ``__path__`` or use the ``__file__`` of a submodule adjacent to the desired files, or of a self-contained subpackage that contains the desired files. (Note: this caveat is already true for existing users of "namespace packages" today. That is, it is an inherent result of being able to partition a package, that you must know *which* partition the desired data file lives in. We mention it here simply so that *new* users converting from self-contained to virtual packages will also be aware of it.) * XXX what is the __file__ of a "pure virtual" package? ``None``? Some arbitrary string? The path of the first directory with a trailing separator? No matter what we put, *some* code is going to break, but the last choice might allow some code to accidentally work. Is that good or bad? For those implementing PEP \302 importer objects: * Importers that support the ``iter_modules()`` method (used by ``pkgutil`` to locate importable modules and packages) and want to add virtual package support should modify their ``iter_modules()`` method so that it discovers and lists virtual packages as well as standard modules and packages. To do this, the importer should simply list all immediate subdirectory names in its jurisdiction that are valid Python identifiers. XXX This might list a lot of not-really-packages. Should we require importable contents to exist? If so, how deep do we search, and how do we prevent e.g. link loops, or traversing onto different filesystems, etc.? Ick. * "Meta" importers (i.e., importers placed on ``sys.meta_path``) do not need to implement ``get_subpath()``, because the method is only called on importers corresponding to ``sys.path`` entries and ``__path__`` entries. If a meta importer wishes to support virtual packages, it must do so entirely within its own ``find_module()`` implementation. Unfortunately, it is unlikely that any such implementation will be able to merge its package subpaths with those of other meta importers or ``sys.path`` importers, so the meaning of "supporting virtual packages" for a meta importer is currently undefined! (However, since the intended use case for meta importers is to replace Python's normal import process entirely for some subset of modules, and the number of such importers currently implemented is quite small, this seems unlikely to be a big issue in practice.) References ========== .. [1] "namespace" vs "module" packages (mailing list thread) (http://mail.zope.org/pipermail/zope3-dev/2002-December/004251.html) .. [2] "Dropping __init__.py requirement for subpackages" (http://mail.python.org/pipermail/python-dev/2006-April/064400.html) Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: From mail at kerrickstaley.com Wed Jul 20 08:53:09 2011 From: mail at kerrickstaley.com (Kerrick Staley) Date: Wed, 20 Jul 2011 01:53:09 -0500 Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the /usr/bin/python2 symlink upstream) In-Reply-To: References: Message-ID: On Mon, Jul 18, 2011 at 3:03 AM, Ned Deily wrote: > I think adding the requirement to mandate hard link vs soft link usage > is an unnecessary and unwarranted attempt at optimization. For > instance, IIRC, the OS X installers don't use any hard links: that may > complicate the install, plus hard links on OS X HFS* file systems are a > bit of a kludge and not necessarily more efficient than symlinks. It's > not a big deal but perhaps the wording should be changed to make a > suggestion about hard links vs syminks rather than mandate which should > be used. Ah, OK. The wording's been changed so that symbolic links will be installed on Mac OS X and hard links elsewhere (although maybe symbolic links are also better on certain other platforms; I'm not sure). I do think that specific instructions must be given (rather than just a suggestion) because it's indicating what must be done to CPython. The instructions *should* be as close as possible to what the installer already does, but I'm not entirely sure what the installer does by default, and the hard-link recommendation was based off a cursory inspection of my own system, so further input from yourself and the rest of python-dev would be appreciated. Nick, can you please apply the patch (will be sent in the following email) to the PEP SVN as soon as we get the hard-link issue is figured out? Alternatively, could you provide me write access to just the pep-0394.txt file so I can update it myself? Thanks. -Kerrick Staley From mail at kerrickstaley.com Wed Jul 20 08:53:22 2011 From: mail at kerrickstaley.com (Kerrick Staley) Date: Wed, 20 Jul 2011 01:53:22 -0500 Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the /usr/bin/python2 symlink upstream) In-Reply-To: References: Message-ID: $ svn diff Index: pep-0394.txt =================================================================== --- pep-0394.txt (revision 88866) +++ pep-0394.txt (working copy) @@ -1,5 +1,5 @@ PEP: 394 -Title: The "python" command on Unix-Like Systems +Title: The "python" Command on Unix-Like Systems Version: $Revision$ Last-Modified: $Date$ Author: Kerrick Staley , @@ -53,10 +53,14 @@ * When reinvoking the interpreter from a Python script, querying ``sys.executable`` to avoid hardcoded assumptions regarding the interpreter location remains the preferred approach. +* The ``idle``, ``pydoc``, and ``python-config`` binaries from Python 2.0 +should likewise be available as ``idle2``, ``pydoc2``, and ``python2-config``, +with the original commands invoking these binaries by default, but possibly +invoking the Python 3.0 versions instead. These recommendations are the outcome of the relevant python-dev discussion in -March 2011 [1] (NOTE: More accurately, they will be such once that "Draft" -status disappears from the PEP header, it has been moved into the "Other +March to July 2011 [1] (NOTE: More accurately, they will be such once that +"Draft" status disappears from the PEP header, it has been moved into the "Other Informational PEP" section in PEP 0 and this note has been deleted) @@ -142,13 +146,21 @@ Application to the CPython Reference Interpreter ================================================ -While technically a new feature, the ``make install`` command and the Mac OS -X installer in the 2.7 version of CPython will be adjusted to create the -new ``python2`` command in addition to the existing ``python`` and -``python2.7`` commands. This feature will first appear in CPython 2.7.2. +While technically a new feature, the ``make install`` command and the Mac +OS X installer in the 2.7 version of CPython will be adjusted to create the +``python2.7``, ``idle2.7``, ``pydoc2.7``, and ``python2.7-config`` binaries, +with ``python2``, ``idle2``, ``pydoc2``, and ``python2-config`` as links to +the respective binaries. On Mac OS X, these will be symbolic links; on other +platforms, they will be hard links, because hard links improve performance +on filesystems other than Mac OS X's HFS. ``python``, ``idle``, ``pydoc``, +and ``python-config`` will be created as symbolic links to these links. +This feature will first appear in CPython 2.7.2. -The ``make install`` command in the CPython 3.x series will continue to -install only the ``python3`` symlink for the foreseeable future. +The ``make install`` command in the CPython 3.x series will similarly install +the ``python3.x``, ``idle3.x``, ``pydoc3.x``, and ``python3.x-config`` binaries +(with appropriate ``x``), and ``python3``, ``idle3``, ``pydoc3``, and +``python3-config`` as symbolic links on Mac OS X and as hard links on other +platforms. Impact on PYTHON* Environment Variables @@ -166,27 +178,12 @@ Exclusions of MS Windows ======================== -This PEP deliberately excludes any proposals relating to Microsoft Windows. -The use of parallel installs on Windows suffers from numerous issues, -including the "last installed wins" behaviour for handling of file -associations, a lack of universal robust symlink support for easy aliasing of -commands, the fact that the Python executable is not available on ``PATH`` by -default, the fact that the ``python.exe`` and ``pythonw.exe`` names are -used for both Python 2 and Python 3 binaries and the lack of distinction -between the different Python versions when the Start menu shortcuts are -divorced from their containing folders (e.g. when they appear in the -"Recently Used" list. +This PEP deliberately excludes any proposals relating to Microsoft Windows: +devising an equivalent solution for Windows was deemed too complex to handle +here. PEP 397 and the related discussion on the python-dev mailing list address +this issue. -While these questions are well worth addressing, they do not have easy -answers. The authors of this particular PEP aren't inclined to even begin -trying to answer them, but anyone that wants to tackle them should feel free -to start working on their own PEP. -Note that, while the topic has been excluded from this PEP, there is plenty of -material in the linked python-dev discussion that may be useful in the design -and implementation of a Windows-specific solution. - - References ========== From p.f.moore at gmail.com Wed Jul 20 09:22:02 2011 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 20 Jul 2011 08:22:02 +0100 Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise) In-Reply-To: References: <20110719171647.03f9f9c9@pitrou.net> Message-ID: On 20 July 2011 03:21, Terry Reedy wrote: > Suppose for Windows there were one '.../python' directory wherever the user > first asks it to be put and that all pythons, not just cpython, are > installed in directories below that and that the small startup file is > copied into or linked from the python directory. Then the one python > directory could be put on the path and left there and never removed by any > python de-installer (unless perhaps it check that there are no subdirs and > *asks* the user. Hmm. Suppose that directory was "C:\Program Files\Python Launcher" (or "C:\Windows\system32" if you don't want to add an extra directory to PATH). And suppose that instead of having a startup file per Python installation you have a single file called py.exe. Then you have the launcher! Plus, the launcher has its own uninstaller, making it a "normal" part of the Windows environment, rather than being a directory created by something which doesn't get uninstalled. Plus, the launcher has a means of dealing with the "generic" python, python2 and python3 commands, which your proposal doesn't. Plus, the launcher deals with existing versions of Python, which your proposal doesn't (except by manual intervention). But yes, the idea is sound, which is why it's so similar to what Vinay did with the launcher IMO. Paul. From nad at acm.org Wed Jul 20 09:56:29 2011 From: nad at acm.org (Ned Deily) Date: Wed, 20 Jul 2011 00:56:29 -0700 Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the /usr/bin/python2 symlink upstream) References: Message-ID: In article , Kerrick Staley wrote: > On Mon, Jul 18, 2011 at 3:03 AM, Ned Deily wrote: > > I think adding the requirement to mandate hard link vs soft link usage > > is an unnecessary and unwarranted attempt at optimization. For > > instance, IIRC, the OS X installers don't use any hard links: that may > > complicate the install, plus hard links on OS X HFS* file systems are a > > bit of a kludge and not necessarily more efficient than symlinks. It's > > not a big deal but perhaps the wording should be changed to make a > > suggestion about hard links vs syminks rather than mandate which should > > be used. > > Ah, OK. The wording's been changed so that symbolic links will be > installed on Mac OS X and hard links elsewhere (although maybe > symbolic links are also better on certain other platforms; I'm not > sure). > > I do think that specific instructions must be given (rather than just > a suggestion) because it's indicating what must be done to CPython. > The instructions *should* be as close as possible to what the > installer already does, but I'm not entirely sure what the installer > does by default, and the hard-link recommendation was based off a > cursory inspection of my own system, so further input from yourself > and the rest of python-dev would be appreciated. Thanks for addressing the comments. Unfortunately, I was relying too much on memory and I should have checked more carefully. I see now that, within the framework bin directory for Python 3, the python3 -> python3.x link is, in fact, a hard link because it is being produced by the standard Makefile target "bininstall". I was thinking more of the optional /usr/local/bin links which are all symlinks. So, if the Python 3 installer does is, there's no reason why Python 2 can't do it, I suppose. But if you look at the Python 3 "bininstall" target (Makefile.pre.in starting around line 870 or so), you'll see that, for Python 3, symlinks are made for all the versioned files except "python3". I'm not sure that there is a particular reason why a distinction was made; IIRC, the other versioned links were added later in the cycle. The other Python 3 versioned links could probably be changed to hard links as well. In the end, I don't think it makes a lot of difference. But it would be better if Python 2 and Python 3 were consistent here. My apologies for the confusion and extra work. -- Ned Deily, nad at acm.org From ncoghlan at gmail.com Wed Jul 20 10:28:16 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 20 Jul 2011 18:28:16 +1000 Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the /usr/bin/python2 symlink upstream) In-Reply-To: References: Message-ID: On Wed, Jul 20, 2011 at 4:53 PM, Kerrick Staley wrote: > Nick, can you please apply the patch (will be sent in the following > email) to the PEP SVN as soon as we get the hard-link issue is figured > out? Alternatively, could you provide me write access to just the > pep-0394.txt file so I can update it myself? Thanks. Done: http://www.python.org/dev/peps/pep-0394/ Main thing needed now is a tracker issue to reference (ideally with a first cut at a patch) and python-dev's collective blessing to take it out of Draft status. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Wed Jul 20 10:46:27 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 20 Jul 2011 18:46:27 +1000 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <20110720040505.400E23A4116@sparrow.telecommunity.com> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> Message-ID: On Wed, Jul 20, 2011 at 1:58 PM, P.J. Eby wrote: > So, without further ado, here it is: I pushed this version up to the PEPs repo, so it now has a number (402) and can be read in prettier HTML format: http://www.python.org/dev/peps/pep-0402/ Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From piotr at debian.org Wed Jul 20 10:58:33 2011 From: piotr at debian.org (Piotr =?utf-8?Q?O=C5=BCarowski?=) Date: Wed, 20 Jul 2011 10:58:33 +0200 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <20110720040505.400E23A4116@sparrow.telecommunity.com> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> Message-ID: <20110720085833.GE30625@piotro.eu> +1 (and yay!) -- Piotr O?arowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645 From v+python at g.nevcal.com Wed Jul 20 11:24:12 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Wed, 20 Jul 2011 02:24:12 -0700 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <20110720040505.400E23A4116@sparrow.telecommunity.com> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> Message-ID: <4E269EBC.90708@g.nevcal.com> On 7/19/2011 8:58 PM, P.J. Eby wrote: > Standard Library Changes/Additions > ---------------------------------- > > The ``pkgutil`` module should be updated to handle this > specification appropriately, including any necessary changes to > ``extend_path()``, ``iter_modules()``, etc. > > Specifically the proposed changes and additions to ``pkgutil`` are: > > * A new ``extend_virtual_paths(path_entry)`` function, to extend > existing, already-imported virtual packages' ``__path__`` attributes > to include any portions found in a new ``sys.path`` entry. This > function should be called by applications extending ``sys.path`` > at runtime, e.g. when adding a plugin directory or an egg to the > path. > > The implementation of this function does a simple top-down traversal > of ``sys.virtual_packages``, and performs any necessary > ``get_subpath()`` calls to identify what path entries need to > be added to each package's ``__path__``, given that `path_entry` > has been added to ``sys.path``. (Or, in the case of sub-packages, > adding a derived subpath entry, based on their parent namespace's > ``__path__``.) When I read about creating __path__ from sys.path, I immediately thought of the issue of programs that extend sys.path, and the above is the "workaround" for such programs. but it requires such programs to do work, and there are a lot of such programs (I, a relative newbie, have had to write some). As it turns out, I can't think of a situation where I have extended sys.path that would result in a problem for fancy namespace packages, because so far I've only written modules, not packages, and only modules are on the paths that I add to sys.path. But that does not make for a general solution. Is there some way to create a new __path__ that would reflect the fact that it has been dynamically created, rather than set from __init__.py, and then when it is referenced, calculate (and cache?) a new value of __path__ to actually search? -------------- next part -------------- An HTML attachment was scrubbed... URL: From v+python at g.nevcal.com Wed Jul 20 11:17:58 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Wed, 20 Jul 2011 02:17:58 -0700 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> Message-ID: <4E269D46.1010806@g.nevcal.com> On 7/18/2011 6:41 PM, Vinay Sajip wrote: >> > So I can fix my machine, now that I understand what went wrong >> > (delete py.exe entries from HCU, and put them in HLM instead). >> > Then the other problem I have, is why py.exe launched py 2.6.4 >> > instead of py 3.2.1 when 3.2.1 is newer, and I don't have a #! >> > line. That is probably the defined behavior of the launcher, to >> > prefer 2.x if 3 isn't specified. I'll have to reread the launcher >> > PEP. In my environment, I would much prefer to use 3.x if 2.x isn't >> > specified... so hopefully when I reread the launcher PEP, I'll >> > discover a way to do that. (I recall one can put -3 on the command >> > line, so I could do that in the py.exe ftype commands, but hopefully >> > there is a way to tweak the py.ini to do that.) > Yes, the PEP explains what you need to do to get Python 3 to be the default: > use the py.ini file or use an environment variable. > > The use of py from the command line is merely a convenience for developers (as > the PEP says) - it's better to rely on shebang lines together with settings in > the .ini to get the behaviour you want. OK, took a few minutes to play with the launcher again. I have removed the HCU classes that point at the launcher, leaving me with all my Python.File stuff pointing at some version of Python or another. I seem to have Python 2.6.4, Python 3.1, and Python 3.2.1 installed, all 64-bit. py.exe is located in c:\Windows\system32.exe, I modified the empty py.ini that was installed to contain Since I don't yet have associations set up that point at the launcher, I thought I'd play with saying "py" in front of the command. So I have a command foo.py using Python 3 syntax in a directory on my PATH. It works fine, since Python 3.2.1 is in Python.file. foo.py does not have a #! line. I can successfully execute: foo.py However, the following fails: py foo.py It fails, because foo.py is not found. Instead, I have to specify: py d:\path\to\foo.py This is annoying, py should walk the PATH for unqualified files (the Windows PATH implicitly includes the current directory, of course, but if it were in the current directory, it would just work). So when I: type c:\windows\system32\py.ini I get a set of h2 h3 v2 v3 commands in a section called [commands]. Not sure where my [defaults] went. Is there some more Windows magic to be explained? WOW6432-ness, perhaps? Probably my editor is running as a 32-bit process, so looked in the wrong C:\windows\system32 !! OK, with that mystery solved, and using Notepad running as administrator to actually, successfully edit the file, it still runs the wrong version of Python. Here is the content of the file, what is wrong? And why is the spacing around the = in the [commands] section so inconsistent? [defaults] python=3 [commands] h2=c:\Python26\python -h h3 = c:\Python32\python -h v2= c:\Python26\python -V v3 =c:\Python32\python -V OK, so when I specify: py d:\path\to\foo.py It fails, but this time it is because it launches Python 2.6.4. So the [defaults] section doesn't seem to have an effect. -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Wed Jul 20 13:28:23 2011 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 20 Jul 2011 12:28:23 +0100 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: <4E269D46.1010806@g.nevcal.com> References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> Message-ID: On 20 July 2011 10:17, Glenn Linderman wrote: > However, the following fails:? py foo.py > It fails, because foo.py is not found.? Instead, I have to specify: py > d:\path\to\foo.py > This is annoying, py should walk the PATH for unqualified files (the Windows > PATH implicitly includes the current directory, of course, but if it were in > the current directory, it would just work). Just as a reference point: PS 12:26 D:\Data >type .\tt.py import sys print sys.argv PS 12:26 D:\Data >py tt.py ['tt.py'] This is Windows XP 32-bit, using Powershell as my shell. Also works in cmd.exe. Paul. From pje at telecommunity.com Wed Jul 20 14:57:55 2011 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 20 Jul 2011 08:57:55 -0400 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: References: <20110720040505.400E23A4116@sparrow.telecommunity.com> Message-ID: <20110720131415.2644B3A409B@sparrow.telecommunity.com> At 06:46 PM 7/20/2011 +1000, Nick Coghlan wrote: >On Wed, Jul 20, 2011 at 1:58 PM, P.J. Eby wrote: > > So, without further ado, here it is: > >I pushed this version up to the PEPs repo, so it now has a number >(402) and can be read in prettier HTML format: >http://www.python.org/dev/peps/pep-0402/ Technically, shouldn't this be a 3XXX series PEP? Or are we not doing those any more now that all PEPs would be 3XXX? From pje at telecommunity.com Wed Jul 20 15:05:47 2011 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 20 Jul 2011 09:05:47 -0400 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <4E269EBC.90708@g.nevcal.com> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <4E269EBC.90708@g.nevcal.com> Message-ID: <20110720131415.A2DD43A4100@sparrow.telecommunity.com> At 02:24 AM 7/20/2011 -0700, Glenn Linderman wrote: >When I read about creating __path__ from sys.path, I immediately >thought of the issue of programs that extend sys.path, and the above >is the "workaround" for such programs.? but it requires such >programs to do work, and there are a lot of such programs (I, a >relative newbie, have had to write some).? As it turns out, I can't >think of a situation where I have extended sys.path that would >result in a problem for fancy namespace packages, because so far >I've only written modules, not packages, and only modules are on the >paths that I add to sys.path.? But that does not make for a general solution. Most programs extend sys.path in order to import things. If those things aren't yet imported, they don't have a __path__ yet, and so don't need to be fixed. Only programs that modify sys.path *after* importing something that has a dynamic __path__ would need to do anything about that. >Is there some way to create a new __path__ that would reflect the >fact that it has been dynamically created, rather than set from >__init__.py, and then when it is referenced, calculate (and cache?) >a new value of __path__ to actually search? That's what extend_virtual_paths() is for. It updates the __path__ of all currently-imported virtual packages. Where before you wrote: sys.path.append('foo') You would now write: sys.path.append('foo') pkgutil.extend_virtual_paths('foo') ...assuming you have virtual packages you've already imported. If you don't, there's no reason to call extend_virtual_paths(). But it doesn't hurt anything if you call it unnecessarily, because it uses sys.virtual_packages to find out what to update, and if you haven't imported any virtual packages, there's nothing to update and the call will be a quick no-op. From eric at trueblade.com Wed Jul 20 15:29:21 2011 From: eric at trueblade.com (Eric V. Smith) Date: Wed, 20 Jul 2011 09:29:21 -0400 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <20110720131415.2644B3A409B@sparrow.telecommunity.com> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <20110720131415.2644B3A409B@sparrow.telecommunity.com> Message-ID: <4E26D831.8000307@trueblade.com> On 07/20/2011 08:57 AM, P.J. Eby wrote: > At 06:46 PM 7/20/2011 +1000, Nick Coghlan wrote: >> On Wed, Jul 20, 2011 at 1:58 PM, P.J. Eby wrote: >> > So, without further ado, here it is: >> >> I pushed this version up to the PEPs repo, so it now has a number >> (402) and can be read in prettier HTML format: >> http://www.python.org/dev/peps/pep-0402/ > > Technically, shouldn't this be a 3XXX series PEP? Or are we not doing > those any more now that all PEPs would be 3XXX? I think we're back to normal PEP numbering. PEP 382 was also 3.x only. Eric. From rdmurray at bitdance.com Wed Jul 20 15:57:07 2011 From: rdmurray at bitdance.com (R. David Murray) Date: Wed, 20 Jul 2011 09:57:07 -0400 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <20110720040505.400E23A4116@sparrow.telecommunity.com> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> Message-ID: <20110720135708.657C92500D5@webabinitio.net> On Tue, 19 Jul 2011 23:58:55 -0400, "P.J. Eby" wrote: > Worse, this is not just a problem for new users: it prevents *anyone* > from easily splitting a package into separately-installable > components. In Perl terms, it would be as if every possible ``Net::`` > module on CPAN had to be bundled up and shipped in a single tarball! In general the simplicity of the proposed mechanism and implementation is attractive. However, this bit of discussion struck me as sending the wrong message. We don't *want* something like the CPAN module hierarchy. I prefer to keep things as flat as practical. Namespace packages clearly have utility, but please let's not descend into java-esq package hierarchies. -- R. David Murray http://www.bitdance.com From vinay_sajip at yahoo.co.uk Wed Jul 20 16:19:08 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Wed, 20 Jul 2011 14:19:08 +0000 (UTC) Subject: [Python-Dev] 3.2.1 encoding surprise References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> Message-ID: Glenn Linderman g.nevcal.com> writes: > Since I don't yet have associations set up that point at the > launcher, I thought I'd play with saying "py" in front of the > command. Why don't you have any associations pointing to the launcher? Did you delete them? If you uninstall and install the launcher, are they present? > So I have a command foo.py using Python 3 syntax in a directory on > my PATH.? It works fine, since Python 3.2.1 is in Python.file.? > foo.py does not have a #! line. > I can successfully execute:? foo.py > However, the following fails:? py foo.py > It fails, because foo.py is not found.? Instead, I have to specify: > py d:\path\to\foo.py > This is annoying, py should walk the PATH for unqualified files (the > Windows PATH implicitly includes the current directory, of course, It's not py's job to walk the path: the shell does that when you just type "foo". It locates foo.py, and then invokes py because of file association - py then checks the file for a shebang to decide which Python to dispatch it to. > OK, with that mystery solved, and using Notepad running as > administrator to actually, successfully edit the file, it still runs > the wrong version of Python.? Here is the content of the file, what I'm afraid I can't reproduce this. When I invoke a script with the default py.ini, py runs it with Python 2. When I add a [defaults]python=3 entry, py correctly runs it with Python 3. > is wrong?? And why is the spacing around the = in the [commands] > section so inconsistent? That's just test data, not a real "production" py.ini. I was testing out something, which is why the spaces around = are every which way, and I never got around to changing it. More importantly, those customised commands, while perhaps useful for testing, are useless in everyday Python usage: perhaps -O, -Werror, -E, -S etc. might be more useful. I'll take suggestions as to what might be useful customised commands to ship as a default. Regards, Vinay Sajip From merwok at netwok.org Wed Jul 20 16:21:28 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Wed, 20 Jul 2011 16:21:28 +0200 Subject: [Python-Dev] [Python-checkins] peps: Restore whitespace characters lost via email transmission. In-Reply-To: References: Message-ID: <4E26E468.20800@netwok.org> Hi, > changeset: 3903:728660b53208 > user: pje > date: Wed Jul 20 09:56:48 2011 -0400 > summary: > Restore whitespace characters lost via email transmission. > [...] > diff --git a/pep-0402.txt b/pep-0402.txt > --- a/pep-0402.txt > +++ b/pep-0402.txt > @@ -38,13 +38,13 @@ > > .. epigraph:: > > - "Most packages are like modules. Their contents are highly > [snip] > + "Most packages are like modules. Their contents are highly FYI, reST uses three-space indents, not four (so that blocks align nicely under the leading two dots + one space), so I think the change was intentional. The ?Documenting Python? guide tells this (in the standard docs), and I think it applies to PEPs too. Regards From ndbecker2 at gmail.com Wed Jul 20 16:40:18 2011 From: ndbecker2 at gmail.com (Neal Becker) Date: Wed, 20 Jul 2011 10:40:18 -0400 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" References: <20110720040505.400E23A4116@sparrow.telecommunity.com> Message-ID: I wonder if this fixes the long-standing issue in OS vendor's distributions. In Fedora, for example, there is both arch-specific and non-arch directories: /usr/lib/python2.7 + /usr/lib64/python2.7, for example. Pure python goes into /usr/lib/python2.7, and code including binaries goes into /usr/lib64/python2.7. But if a package has both, it all has to go into /usr/lib64/python2.7, because the current loader can't find pieces in 2 different directories. You can't have both /usr/lib/python2.7/site-packages/foo and /usr/lib64/python2.7/site-packages/foo. So if this PEP will allow pieces of foo to be found in 2 different places, that would be helpful, IMO. From pje at telecommunity.com Wed Jul 20 16:55:04 2011 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 20 Jul 2011 10:55:04 -0400 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: References: <20110720040505.400E23A4116@sparrow.telecommunity.com> Message-ID: <20110720145543.E834D3A409B@sparrow.telecommunity.com> At 10:40 AM 7/20/2011 -0400, Neal Becker wrote: >I wonder if this fixes the long-standing issue in OS vendor's >distributions. In >Fedora, for example, there is both arch-specific and non-arch directories: >/usr/lib/python2.7 + /usr/lib64/python2.7, for example. Pure python >goes into >/usr/lib/python2.7, and code including binaries goes into >/usr/lib64/python2.7. >But if a package has both, it all has to go into >/usr/lib64/python2.7, because >the current loader can't find pieces in 2 different directories. > >You can't have both /usr/lib/python2.7/site-packages/foo and >/usr/lib64/python2.7/site-packages/foo. > >So if this PEP will allow pieces of foo to be found in 2 different >places, that >would be helpful, IMO. It's more of a long-term solution than a short-term one. In order for it to work the way you want, 'foo' would need to have its main code in foo.py rather than foo/__init__.py. You could of course make that change on the author's behalf for your distro, or remove it altogether if it doesn't contain any actual code. However, if you're going to make changes, you could change its __init__.py right now to append extra directories to the module __path__... and that's something you can do right now. From pje at telecommunity.com Wed Jul 20 16:55:39 2011 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 20 Jul 2011 10:55:39 -0400 Subject: [Python-Dev] [Python-checkins] peps: Restore whitespace characters lost via email transmission. Message-ID: <20110720145618.7A3F33A409B@sparrow.telecommunity.com> At 04:21 PM 7/20/2011 +0200, ??ric Araujo wrote: >FYI, reST uses three-space indents, not four (so that blocks align >nicely under the leading two dots + one space), so I think the change >was intentional. The ???Documenting Python??? guide tells this (in the >standard docs), and I think it applies to PEPs too. PEP 12 prescribes four-space indents for PEPs, actually, wherever it mentions an specific indentation depth. Also, a formfeed character was lost, not just the leading spaces. Essentially, though, I was just merging my working copy, and those were the only differences that showed up (apart from the filled-in Post-History header), so I assumed it was just whitespace lost in transmission. (I'm a bit surprised that three-space indents are mandated for anything involving documenting Python in reST, though, since that would mean you'd also have to indent your code samples by three spaces, or else have an editor that supports two different tab widths.) From merwok at netwok.org Wed Jul 20 17:05:28 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Wed, 20 Jul 2011 17:05:28 +0200 Subject: [Python-Dev] [Python-checkins] peps: Restore whitespace characters lost via email transmission. In-Reply-To: <20110720144058.92DD33A409B@sparrow.telecommunity.com> References: <4E26E468.20800@netwok.org> <20110720144058.92DD33A409B@sparrow.telecommunity.com> Message-ID: <4E26EEB8.4050200@netwok.org> Le 20/07/2011 16:40, P.J. Eby a ?crit : > PEP 12 prescribes four-space indents for PEPs, actually, wherever it > mentions an specific indentation depth. Ah, thanks. I see in my docutils docs that David Goodger used three and four-space indents (three for indenting the body of a directive, four for other cases). > (I'm a bit surprised that three-space indents are mandated for > anything involving documenting Python in reST, though, since that > would mean you'd also have to indent your code samples by three > spaces, or else have an editor that supports two different tab widths.) I don?t remember this being a pain. Maybe it?s because the reST I tend to edit has much more reST indents than Python code example indents, so I don?t mind typing . Cheers From jdhardy at gmail.com Wed Jul 20 17:56:33 2011 From: jdhardy at gmail.com (Jeff Hardy) Date: Wed, 20 Jul 2011 08:56:33 -0700 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <20110720040505.400E23A4116@sparrow.telecommunity.com> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> Message-ID: On Tue, Jul 19, 2011 at 8:58 PM, P.J. Eby wrote: > The biggest likely exception to the above would be when a piece of > code tries to check whether some package is installed by importing > it. ?If this is done *only* by importing a top-level module (i.e., not > checking for a ``__version__`` or some other attribute), *and* there > is a directory of the same name as the sought-for package on > ``sys.path`` somewhere, *and* the package is not actually installed, > then such code could *perhaps* be fooled into thinking a package is > installed that really isn't. This part worries me slightly. Imagine a program as such: datagen.py json/foo.js json/bar.js datagen.py uses the files in json/ to generate sample data for a database. In datagen.py is the following code: try: import json except ImportError: import simplejson as json Currently, this works just fine, but if will break (as I understand it) under the PEP because the json directory will become a virtual package and no ImportError will be raised. Is there a mitigation for this in the PEP that I've missed? > However, even in the rare case where all these conditions line up to > happen at once, the failure is more likely to be annoying than > damaging. In most cases, after all, the code will simply fail a > little later on, when it actually tries to DO something with the > imported (but empty) module. (And code that checks ``__version__`` > attributes or for the presence of some desired function, class, or > module in the package will not see a false positive result in the > first place.) It may only be annoying, but it's still a breaking change, and a subtle one at that. Checking __version__ is of course possible, but it's never been necessary before, so it's unlikely there's much code that does it. It also makes the fallback code significantly less neat. - Jeff From merwok at netwok.org Wed Jul 20 17:58:17 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Wed, 20 Jul 2011 17:58:17 +0200 Subject: [Python-Dev] New tests in stable versions Message-ID: <4E26FB19.2070406@netwok.org> Hello everyone, I?ve seen recent commits in the default branch (3.3) that improve test coverage (for example logging) or add new test files (cgitb, committed by Brian). Do we have a policy of not adding new test files to stable branches? For existing test files that get more tests, is the commit to stable branches left to the decision of the committer, or should stable versions get the new tests too? Regards From hyugaricdeau at gmail.com Wed Jul 20 18:37:34 2011 From: hyugaricdeau at gmail.com (Erik) Date: Wed, 20 Jul 2011 12:37:34 -0400 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: References: <20110720040505.400E23A4116@sparrow.telecommunity.com> Message-ID: On Wed, Jul 20, 2011 at 11:56 AM, Jeff Hardy wrote: > On Tue, Jul 19, 2011 at 8:58 PM, P.J. Eby wrote: >> The biggest likely exception to the above would be when a piece of >> code tries to check whether some package is installed by importing >> it. ?If this is done *only* by importing a top-level module (i.e., not >> checking for a ``__version__`` or some other attribute), *and* there >> is a directory of the same name as the sought-for package on >> ``sys.path`` somewhere, *and* the package is not actually installed, >> then such code could *perhaps* be fooled into thinking a package is >> installed that really isn't. > > This part worries me slightly. Imagine a program as such: > > datagen.py > json/foo.js > json/bar.js > > datagen.py uses the files in json/ to generate sample data for a > database. In datagen.py is the following code: > > try: > ? ?import json > except ImportError: > ? ?import simplejson as json > > Currently, this works just fine, but if will break (as I understand > it) under the PEP because the json directory will become a virtual > package and no ImportError will be raised. Is there a mitigation for > this in the PEP that I've missed? This problem was brought up a few times on import-sig, but I don't think a solution was ever decided on. The best solution I can think of would be to have a way for a module to mark itself as "finalized" (I'm not sure if that's the best term--just the first that popped into my head). This would prevent its __path__ from being created or extended in any way. For example, if the json module contains `__finalized__ = True` or something of the like, any `import json.foo` would immediately fail. Of course, this would put all the onus on the json module to solve this problem, and other modules might actually wish to be extendable into packages, in which case you'd still have this problem. In that case there would need to be a way to mark a directory as not containing importable code. Not sure what the best approach to that would be, especially since one of the goals of this PEP seems to be to avoid marker files. Erik From victor.stinner at haypocalc.com Wed Jul 20 18:25:52 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Wed, 20 Jul 2011 18:25:52 +0200 Subject: [Python-Dev] New tests in stable versions In-Reply-To: <4E26FB19.2070406@netwok.org> References: <4E26FB19.2070406@netwok.org> Message-ID: <4E270190.1000209@haypocalc.com> Le 20/07/2011 17:58, ?ric Araujo a ?crit : > Do we have a policy of not adding new test files to stable branches? New logging tests failed during some weeks. If we add new tests, we may also break some stable buildbots. I don't think that we need to add these new tests to a stable version. Victor From pje at telecommunity.com Wed Jul 20 19:04:05 2011 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 20 Jul 2011 13:04:05 -0400 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: References: <20110720040505.400E23A4116@sparrow.telecommunity.com> Message-ID: <20110720170448.D7D793A409B@sparrow.telecommunity.com> At 08:56 AM 7/20/2011 -0700, Jeff Hardy wrote: >On Tue, Jul 19, 2011 at 8:58 PM, P.J. Eby wrote: > > The biggest likely exception to the above would be when a piece of > > code tries to check whether some package is installed by importing > > it. If this is done *only* by importing a top-level module (i.e., not > > checking for a ``__version__`` or some other attribute), *and* there > > is a directory of the same name as the sought-for package on > > ``sys.path`` somewhere, *and* the package is not actually installed, > > then such code could *perhaps* be fooled into thinking a package is > > installed that really isn't. > >This part worries me slightly. Imagine a program as such: > >datagen.py >json/foo.js >json/bar.js > >datagen.py uses the files in json/ to generate sample data for a >database. In datagen.py is the following code: > >try: > import json >except ImportError: > import simplejson as json > >Currently, this works just fine, but if will break (as I understand >it) under the PEP because the json directory will become a virtual >package and no ImportError will be raised. Well, it won't fail as long if there actually *is* a json module or package on the path. ;-) But I do see your point. >Is there a mitigation for this in the PEP that I've missed? A possible mitigation would be to require that get_subpath() only return a directory name if that directory in fact contains importable modules somewhere. This is actually discussed a bit later as an open issue under "Implementation Notes", indicating that iter_modules() has this issue as well. The main open questions in doing this kind of checking have to do with recursion: it's perfectly valid to have say, a 'zc/' directory whose only content is a 'buildout/' subdirectory. Of course, it still wouldn't help if the 'json/' subdirectory in your example did contain .py files. There is another possibility, though: What if we change the logic for pure-virtual package creation so that the parent module is created *if and only if* a child module is found? In that case, trying to import a pure virtual 'zc' package would fail, but importing 'zc.buildout' would succeed as long as there was a zc/buildout.py or a zc/buildout/__init__.py somewhere. And in your example, 'import json' would fail -- which is to say, succeed. ;-) This is a minor change to the spec, though perhaps a bit hairier to implement in practice. The current import.c loop over the module name parts (iterating over say, 'zc', then 'buildout', and importing them in turn) would need to be reworked so that it could either roll back the virtual package creation in the event of sub-import failure or conversely delay creation of the parent package(s) until a sub-import finds a module. I certainly think it's *doable*, mind you, but I'd hate to have to do it in C. ;-) Hm. Here's another variant that might be easier to implement (even in C), and could offer some other advantages as well. Suppose we replace the sys.virtual_packages set() with a sys.virtual_paths dict(): a dictionary that maps from module names to __path__ lists, and that's populated by the __path__ creation algorithm described in the PEP. (An empty list would mean that __path__ creation failed for that module/package name.) Now, if a module doesn't have a __path__ (or doesn't exist), we look in sys.virtual_paths for the module name. If the retrieved list is empty, we fail the import. If it's not, we proceed... but *don't* create a module or set the existing module's __path__. Then, at the point where an import succeeds, and we're going to set an attribute on the parent module, we recursively construct parent modules and set their __path__ attributes from sys.virtual_paths, if a module doesn't exist in sys.path, or its __path__ isn't set. Voila. Now there are fewer introspection problems as well: trying to 'import json.foo' when there's no 'foo.py' in any json/ directory will *not* create an empty 'json' package in sys.modules as a side-effect. And it won't add a __path__ to the 'json' module if there were a json.py found, either. What's more, since importing a pure virtual package now fails unless you've successfully imported something from it before, it makes more sense for it to not have a __file__, or a __file__ of None. Actually, it's too bad that we have to have parent packages in sys.modules, or I'd suggest we just make pure virtual packages unimportable, period. Technically, we *could* always create dummy parent modules for virtual packages and *not* put them in sys.modules, but I'm not sure if that's a good idea. It would be more consistent in some ways with the idea that virtual packages are not directly importable, but an interesting side effect would be that if module A does: import foo.bar and module B does: import foo.baz Then module A's version of 'foo' has *only* a 'bar' attribute and B's version has *only* a 'baz' attribute. This could be considered a good thing, a bad thing, or a weird thing, depending on how you look at it. ;-) Probably, we should stick with the current shared 'foo' instance, even for pure virtual packages. It's just that 'foo' should not exist in sys.packages until one of the above imports succeeds. Anyway, thanks for bringing this issue up, because now we can fix the hole *entirely*. If pure virtual packages can never be imported directly, then they can *never* create false positive imports -- and the "Backward Compatibility" part of the PEP gets shorter. ;-) Hurray! (I'm tempted to run off and tweak the PEP for this right now, but I want to see if any of the folks who'd be doing the actual 3.x implementation of this want to weigh in on the details first.) From pje at telecommunity.com Wed Jul 20 19:22:36 2011 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 20 Jul 2011 13:22:36 -0400 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: References: <20110720040505.400E23A4116@sparrow.telecommunity.com> Message-ID: <20110720172317.21DAA3A409B@sparrow.telecommunity.com> At 12:37 PM 7/20/2011 -0400, Erik wrote: >The best solution I can think of would be to have a way for a module >to mark itself as "finalized" (I'm not sure if that's the best >term--just the first that popped into my head). This would prevent >its __path__ from being created or extended in any way. For example, >if the json module contains `__finalized__ = True` or something of the >like, any `import json.foo` would immediately fail. That wouldn't actually fix the problem Jeff brought up, which was the case where there *wasn't* a json.py. In any case, we can fix this now by banning direct import of pure-virtual packages. >In that case there would need to be a way to mark a directory as not >containing importable code. Not sure what the best approach to that >would be, especially since one of the goals of this PEP seems to be to >avoid marker files. For this particular issue, we don't need it. For tools that process Python code, or use pkgutil.walk_modules(), there may still be use cases, so we'll keep an eye open for relevant input. Hopefully someone will say something that jars loose an idea or two, as happened with Jeff's issue above. (Btw, as we speak, I am swiping Jeff's example and adding it into the PEP. ;-) It makes a great motivating example for banning pure-virtual package imports.) From mail at kerrickstaley.com Wed Jul 20 20:03:26 2011 From: mail at kerrickstaley.com (Kerrick Staley) Date: Wed, 20 Jul 2011 13:03:26 -0500 Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the /usr/bin/python2 symlink upstream) In-Reply-To: References: Message-ID: On Wed, Jul 20, 2011 at 3:28 AM, Nick Coghlan wrote: > Done: http://www.python.org/dev/peps/pep-0394/ Quick question: When I do "svn up", it doesn't show any changes. Has it been switched over to Mercurial recently? Thanks, Kerrick Staley From ericsnowcurrently at gmail.com Wed Jul 20 20:14:17 2011 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Wed, 20 Jul 2011 12:14:17 -0600 Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the /usr/bin/python2 symlink upstream) In-Reply-To: References: Message-ID: On Wed, Jul 20, 2011 at 12:03 PM, Kerrick Staley wrote: > On Wed, Jul 20, 2011 at 3:28 AM, Nick Coghlan wrote: >> Done: http://www.python.org/dev/peps/pep-0394/ > > Quick question: When I do "svn up", it doesn't show any changes. Has > it been switched over to Mercurial recently? http://hg.python.org/peps/ -eric > > Thanks, > Kerrick Staley > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ericsnowcurrently%40gmail.com > From tjreedy at udel.edu Wed Jul 20 20:38:47 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 20 Jul 2011 14:38:47 -0400 Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise) In-Reply-To: References: <20110719171647.03f9f9c9@pitrou.net> Message-ID: On 7/20/2011 3:22 AM, Paul Moore wrote: > On 20 July 2011 03:21, Terry Reedy wrote: >> Suppose for Windows there were one '.../python' directory wherever the user >> first asks it to be put and that all pythons, not just cpython, are >> installed in directories below that and that the small startup file is >> copied into or linked from the python directory. Then the one python >> directory could be put on the path and left there and never removed by any >> python de-installer (unless perhaps it check that there are no subdirs and >> *asks* the user. > > Hmm. Suppose that directory was "C:\Program Files\Python Launcher" (or > "C:\Windows\system32" if you don't want to add an extra directory to > PATH). And suppose that instead of having a startup file per Python > installation you have a single file called py.exe. Then you have the > launcher! > > Plus, the launcher has its own uninstaller, making it a "normal" part > of the Windows environment, rather than being a directory created by > something which doesn't get uninstalled. > > Plus, the launcher has a means of dealing with the "generic" python, > python2 and python3 commands, which your proposal doesn't. > > Plus, the launcher deals with existing versions of Python, which your > proposal doesn't (except by manual intervention). > > But yes, the idea is sound, which is why it's so similar to what Vinay > did with the launcher IMO. Many installers first make an organization directory and then an app directory within that. This annoys me sometimes when they only have one app to ever install, but is useful when there might really be multiple directories, as in our case. (Ditto for start menu entries.) This is what python should have done a decade ago. Now is not too late. The launcher has to be in a directory somewhere on the path. That directory could just as well be 'our' directory. The two proposals overlap but are not mutually exclusive. For future pythons, 'python33' is easier to remember and type than 'py -v 3.3' or whatever the proposed encantation is. A python directory also gives a sensible (though optional) place to put other interpreters and even python-based apps. The launcher does not. -- Terry Jan Reedy From tjreedy at udel.edu Wed Jul 20 20:48:18 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 20 Jul 2011 14:48:18 -0400 Subject: [Python-Dev] New tests in stable versions In-Reply-To: <4E270190.1000209@haypocalc.com> References: <4E26FB19.2070406@netwok.org> <4E270190.1000209@haypocalc.com> Message-ID: On 7/20/2011 12:25 PM, Victor Stinner wrote: > Le 20/07/2011 17:58, ?ric Araujo a ?crit : >> Do we have a policy of not adding new test files to stable branches? > New logging tests failed during some weeks. If we add new tests, we may > also break some stable buildbots. I don't think that we need to add > these new tests to a stable version. When bugs are fixed in stable branches, they are usually accompanied by tests that fail without the bugfix. I have understood the policy to be that new tests go into stable branches. Failure indicates a bug in either the not-really-so-stable branch or the test. In the latter case, remove the test everywhere until fixed. In the former case, either fix the bug in the stable branch immediately or open an issue and attach the test code (skipping the test needed stage) or just disable it and note on the issue that a fix patch should re-enable. The logging tests may have been exceptional some way. -- Terry Jan Reedy From g.brandl at gmx.net Wed Jul 20 21:10:01 2011 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 20 Jul 2011 21:10:01 +0200 Subject: [Python-Dev] [Python-checkins] peps: Restore whitespace characters lost via email transmission. In-Reply-To: <4E26EEB8.4050200@netwok.org> References: <4E26E468.20800@netwok.org> <20110720144058.92DD33A409B@sparrow.telecommunity.com> <4E26EEB8.4050200@netwok.org> Message-ID: On 07/20/11 17:05, ?ric Araujo wrote: > Le 20/07/2011 16:40, P.J. Eby a ?crit : >> PEP 12 prescribes four-space indents for PEPs, actually, wherever it >> mentions an specific indentation depth. > > Ah, thanks. I see in my docutils docs that David Goodger used three and > four-space indents (three for indenting the body of a directive, four > for other cases). > >> (I'm a bit surprised that three-space indents are mandated for >> anything involving documenting Python in reST, though, since that >> would mean you'd also have to indent your code samples by three >> spaces, or else have an editor that supports two different tab widths.) > > I don?t remember this being a pain. Maybe it?s because the reST I tend > to edit has much more reST indents than Python code example indents, so > I don?t mind typing . Also, chances are that you've tried out your sample code (!) and thus copy it from a Python file anyway. Georg From tjreedy at udel.edu Wed Jul 20 20:12:42 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 20 Jul 2011 14:12:42 -0400 Subject: [Python-Dev] [Python-checkins] cpython: #665194: support roundtripping RFC2822 date stamps in the email.utils module In-Reply-To: References: Message-ID: <4E271A9A.1040703@udel.edu> On 7/20/2011 11:41 AM, r.david.murray wrote: > diff --git a/Lib/email/utils.py b/Lib/email/utils.py > # We need wormarounds for bugs in these methods in older Pythons (see below) Is 'wormaround' (variation of workaround) an intentional play on the fact that some worms prey on other 'bugs' ;-? (rather then eating plant matter?) Terry From ericsnowcurrently at gmail.com Wed Jul 20 21:35:46 2011 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Wed, 20 Jul 2011 13:35:46 -0600 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <20110720170448.D7D793A409B@sparrow.telecommunity.com> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <20110720170448.D7D793A409B@sparrow.telecommunity.com> Message-ID: On Wed, Jul 20, 2011 at 11:04 AM, P.J. Eby wrote: > Hm. ?Here's another variant that might be easier to implement (even in C), > and could offer some other advantages as well. > > Suppose we replace the sys.virtual_packages set() with a sys.virtual_paths > dict(): a dictionary that maps from module names to __path__ lists, and > that's populated by the __path__ creation algorithm described in the PEP. > ?(An empty list would mean that __path__ creation failed for that > module/package name.) > > Now, if a module doesn't have a __path__ (or doesn't exist), we look in > sys.virtual_paths for the module name. ?If the retrieved list is empty, we > fail the import. ?If it's not, we proceed... ?but *don't* create a module or > set the existing module's __path__. > > Then, at the point where an import succeeds, and we're going to set an > attribute on the parent module, we recursively construct parent modules and > set their __path__ attributes from sys.virtual_paths, if a module doesn't > exist in sys.path, or its __path__ isn't set. (I'm guessing you meant sys.modules in that last sentence.) This is a really nice solution. So a virtual package is not imported until a submodule of the virtual package is successfully imported (except for direct import of pure virtual packages). It seems like sys.virtual_packages should be populated even during a failed submodule import. Is that right? Also, it makes sense that the above applies to all virtual packages, not just pure ones. > > Voila. ?Now there are fewer introspection problems as well: trying to > 'import json.foo' when there's no 'foo.py' in any json/ directory will *not* > create an empty 'json' package in sys.modules as a side-effect. ?And it > won't add a __path__ to the 'json' module if there were a json.py found, > either. > > What's more, since importing a pure virtual package now fails unless you've > successfully imported something from it before, it makes more sense for it > to not have a __file__, or a __file__ of None. > > Actually, it's too bad that we have to have parent packages in sys.modules, > or I'd suggest we just make pure virtual packages unimportable, period. It wouldn't be that hard to disallow their direct import entirely, but still allow the indirect import when successfully importing a submodule. However, that would effectively imply that the import of submodules of the virtual package will also fail. In other words, it may be a source of confusion if a package can't be imported but its submodule can. There is one remaining difference between the two types of virtual packages that's derived from allowing direct import of pure virtual packages. When a pure virtual package is directly imported, a new [empty] module is created and its __path__ is set to the matching value in sys.virtual_packages. However, an "impure" virtual package is not created upon direct import, and its __path__ is not updated until a submodule import is attempted. Even the sys.virtual_packages entry is not generated until the submodule attempt, since the virtual package mechanism doesn't kick in until the point that an ImportError is currently raised. This isn't that big a deal, but it would be the one behavioral difference between the two kinds of virtual packages. So either leave that one difference, disallow direct import of pure virtual packages, or attempt to make virtual packages for all non-package imports. That last one would impose the virtual package overhead on many more imports so it is probably too impractical. I'm fine with leaving the one difference. > > Technically, we *could* always create dummy parent modules for virtual > packages and *not* put them in sys.modules, but I'm not sure if that's a > good idea. ?It would be more consistent in some ways with the idea that > virtual packages are not directly importable, but an interesting side effect > would be that if module A does: > > ?import foo.bar > > and module B does: > > ?import foo.baz > > Then module A's version of 'foo' has *only* a 'bar' attribute and B's > version has *only* a 'baz' attribute. ?This could be considered a good > thing, a bad thing, or a weird thing, depending on how you look at it. ?;-) > > Probably, we should stick with the current shared 'foo' instance, even for > pure virtual packages. ?It's just that 'foo' should not exist in > sys.packages until one of the above imports succeeds. (Guessing you meant sys.virtual_packages.) Agreed. FYI, last night I started on an importlib-based implementation for the PEP and the above solution would be really easy to incorporate. -eric > > Anyway, thanks for bringing this issue up, because now we can fix the hole > *entirely*. ?If pure virtual packages can never be imported directly, then > they can *never* create false positive imports -- and the "Backward > Compatibility" part of the PEP gets shorter. ?;-) > > Hurray! ?(I'm tempted to run off and tweak the PEP for this right now, but I > want to see if any of the folks who'd be doing the actual 3.x implementation > of this want to weigh in on the details first.) > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ericsnowcurrently%40gmail.com > From tjreedy at udel.edu Wed Jul 20 21:51:39 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 20 Jul 2011 15:51:39 -0400 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <20110720170448.D7D793A409B@sparrow.telecommunity.com> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <20110720170448.D7D793A409B@sparrow.telecommunity.com> Message-ID: On 7/20/2011 1:04 PM, P.J. Eby wrote: >> This part worries me slightly. Imagine a program as such: >> >> datagen.py >> json/foo.js >> json/bar.js >> >> datagen.py uses the files in json/ to generate sample data for a >> database. In datagen.py is the following code: >> >> try: >> import json >> except ImportError: >> import simplejson as json While reading the PEP, I worried about this standard usage too but missed the scenario you imagined. Good catch. > A possible mitigation would be to require that get_subpath() only return > a directory name if that directory in fact contains importable modules > somewhere. This is actually discussed a bit later as an open issue under > "Implementation Notes", indicating that iter_modules() has this issue as > well. If one actually wants to create a bare-as-possible empty module, one can do that now either with a directory containing an empty __init__.py or, even cleaner, imp.new_module. So there is no need for the new mechanism to ever duplicate either ;-). So +1 on improving back-compatibility. -- Terry Jan Reedy From solipsis at pitrou.net Wed Jul 20 22:40:17 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 20 Jul 2011 22:40:17 +0200 Subject: [Python-Dev] [Python-checkins] cpython: #665194: support roundtripping RFC2822 date stamps in the email.utils module References: <4E271A9A.1040703@udel.edu> Message-ID: <20110720224017.4a2ddf83@pitrou.net> On Wed, 20 Jul 2011 14:12:42 -0400 Terry Reedy wrote: > On 7/20/2011 11:41 AM, r.david.murray wrote: > > > diff --git a/Lib/email/utils.py b/Lib/email/utils.py > > > # We need wormarounds for bugs in these methods in older Pythons (see below) > > Is 'wormaround' (variation of workaround) an intentional play on the > fact that some worms prey on other 'bugs' ;-? (rather then eating plant > matter?) Or perhaps worms dig their way carefully around known bugs? From pje at telecommunity.com Wed Jul 20 22:44:15 2011 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 20 Jul 2011 16:44:15 -0400 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <20110720170448.D7D793A409B@sparrow.telecommunity.com> Message-ID: <20110720204456.5D1AB3A409B@sparrow.telecommunity.com> At 01:35 PM 7/20/2011 -0600, Eric Snow wrote: >This is a really nice solution. So a virtual package is not imported >until a submodule of the virtual package is successfully imported Correct... >(except for direct import of pure virtual packages). Not correct. ;-) What we do is avoid creating a parent module or altering its __path__ until a submodule/subpackage import is just about to be successfully completed. See the change I just pushed to the PEP: http://hg.python.org/peps/rev/a6f02035c66c Or read the revised Specification section here (which is a bit easier to read than the diff): http://www.python.org/dev/peps/pep-0402/#specification The change is basically that we wait until a successful find_module() happens before creating or tweaking any parent modules. This way, the load_module() part will still see an initialized parent package in sys.modules, and if it does any relative imports, they'll still work. (It *does* mean that if an error happens during load_module(), then future imports of the virtual package will succeed, but I'm okay with that corner case.) >It seems like >sys.virtual_packages should be populated even during a failed >submodule import. Is that right? Yes. In the actual draft, btw, I dubbed it ``sys.virtual_package_paths`` and made it a dictionary. This actually makes the pkgutil.extend_path() code more general: it'll be able to fix the paths of things you haven't actually imported yet. ;-) >Also, it makes sense that the above applies to all virtual packages, >not just pure ones. Well, if the package isn't "pure" then what you've imported is really just an ordinary module, not a package at all. ;-) >When a pure virtual package is directly imported, a new [empty] module >is created and its __path__ is set to the matching value in >sys.virtual_packages. However, an "impure" virtual package is not >created upon direct import, and its __path__ is not updated until a >submodule import is attempted. Even the sys.virtual_packages entry is >not generated until the submodule attempt, since the virtual package >mechanism doesn't kick in until the point that an ImportError is >currently raised. > >This isn't that big a deal, but it would be the one behavioral >difference between the two kinds of virtual packages. So either leave >that one difference, disallow direct import of pure virtual packages, >or attempt to make virtual packages for all non-package imports. That >last one would impose the virtual package overhead on many more >imports so it is probably too impractical. I'm fine with leaving the >one difference. At this point, I've updated the PEP to disallow direct imports of pure virtual packages. AFAICT it's the only approach that ensures you can't get false positive imports by having unrelated-but-similarly-named directories floating around. So, really, there's not a difference, except that you can't import a useless empty module that you have no real business importing in the first place... and I'm fine with that. ;-) >FYI, last night I started on an importlib-based implementation for the >PEP and the above solution would be really easy to incorporate. Well, you might want to double-check that now that I've updated the spec. ;-) In the new approach, you cannot rely on parent modules existing before proceeding to the submodule import. However, I've just glanced at the importlib trunk, and I think I see what you mean. It's already using a recursive approach, rather than an iterative one, so the change should be a lot simpler there than in import.c. There probably just needs to be a pair of functions like: def _get_parent_path(parent): pmod = sys.modules.get(parent) if pmod is None: try: pmod = _gcd_import(parent) except ImportError: # Can't import parent, is it a virtual package? path = imp.get_virtual_path(parent) if not path: # no, allow the parent's import error to propagate raise return path if hasattr(pmod, '__path__'): return pmod.__path__ else: return imp.get_virtual_path(parent) def _get_parent_module(parent): pmod = sys.modules.get(parent) if pmod is None: pmod = sys.modules[parent] = imp.new_module(parent) if '.' in parent: head, _, tail = parent.rpartition('.') setattr(_get_parent_module(head), tail, pmod) if not hasattr(pmod, '__path__'): pmod.__path__ = imp.get_virtual_path(parent) return pmod And then instead of hanging on to parent_module during the import process, you'd just grab a path from _get_parent_path(), and initialize parent_module a little later, i.e.: if parent: path = _get_parent_path(parent) if not path: msg = (_ERR_MSG + '; {} is not a package').format(name, parent) raise ImportError(msg) meta_path = sys.meta_path + _IMPLICIT_META_PATH for finder in meta_path: loader = finder.find_module(name, path) if loader is not None: # ensure parent module exists and is a package before loading parent_module = _get_parent_module(parent) loader.load_module(name) break else: raise ImportError(_ERR_MSG.format(name)) So, yeah, actually, that's looking pretty sweet. Basically, we just have to throw a virtual_package_paths dict into the sys module, and do the above along with the get_virtual_path() function and add get_subpath() to the importer objects, in order to get the PEP's core functionality working. From ericsnowcurrently at gmail.com Wed Jul 20 23:22:05 2011 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Wed, 20 Jul 2011 15:22:05 -0600 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <20110720204456.5D1AB3A409B@sparrow.telecommunity.com> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <20110720170448.D7D793A409B@sparrow.telecommunity.com> <20110720204456.5D1AB3A409B@sparrow.telecommunity.com> Message-ID: On Wed, Jul 20, 2011 at 2:44 PM, P.J. Eby wrote: > At 01:35 PM 7/20/2011 -0600, Eric Snow wrote: >> >> This is a really nice solution. ?So a virtual package is not imported >> until a submodule of the virtual package is successfully imported > > Correct... > >> (except for direct import of pure virtual packages). > > Not correct. ?;-) ?What we do is avoid creating a parent module or altering > its __path__ until a submodule/subpackage import is just about to be > successfully completed. Good point, though I was talking about direct imports of pure virtual packages (which you've indicated are disallowed by the current draft). >> Also, it makes sense that the above applies to all virtual packages, >> not just pure ones. > > Well, if the package isn't "pure" then what you've imported is really just > an ordinary module, not a package at all. ?;-) I meant that if the submodule import fails in the "impure" case, the existing module does not end up with a __path__. >> When a pure virtual package is directly imported, a new [empty] module >> is created and its __path__ is set to the matching value in >> sys.virtual_packages. ?However, an "impure" virtual package is not >> created upon direct import, and its __path__ is not updated until a >> submodule import is attempted. ?Even the sys.virtual_packages entry is >> not generated until the submodule attempt, since the virtual package >> mechanism doesn't kick in until the point that an ImportError is >> currently raised. >> >> This isn't that big a deal, but it would be the one behavioral >> difference between the two kinds of virtual packages. ?So either leave >> that one difference, disallow direct import of pure virtual packages, >> or attempt to make virtual packages for all non-package imports. ?That >> last one would impose the virtual package overhead on many more >> imports so it is probably too impractical. ?I'm fine with leaving the >> one difference. > > At this point, I've updated the PEP to disallow direct imports of pure > virtual packages. ?AFAICT it's the only approach that ensures you can't get > false positive imports by having unrelated-but-similarly-named directories > floating around. I see what you mean. That case is probably more important than the case of having a package that fails to import but submodules of the package that succeed. >> FYI, last night I started on an importlib-based implementation for the >> PEP and the above solution would be really easy to incorporate. > > Well, you might want to double-check that now that I've updated the spec. > ?;-) ?In the new approach, you cannot rely on parent modules existing before > proceeding to the submodule import. > > However, I've just glanced at the importlib trunk, and I think I see what > you mean. ?It's already using a recursive approach, rather than an iterative > one, so the change should be a lot simpler there than in import.c. > > > So, yeah, actually, that's looking pretty sweet. ?Basically, we just have to > throw a virtual_package_paths dict into the sys module, and do the above > along with the get_virtual_path() function and add get_subpath() to the > importer objects, in order to get the PEP's core functionality working. Exactly. That's part of why the importlib approach is so appealing to me. Brett really did a nice job. -eric From v+python at g.nevcal.com Wed Jul 20 23:43:43 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Wed, 20 Jul 2011 14:43:43 -0700 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> Message-ID: <4E274C0F.40400@g.nevcal.com> On 7/20/2011 7:19 AM, Vinay Sajip wrote: > Glenn Linderman g.nevcal.com> writes: > > >> Since I don't yet have associations set up that point at the >> launcher, I thought I'd play with saying "py" in front of the >> command. > Why don't you have any associations pointing to the launcher? Did you delete > them? If you uninstall and install the launcher, are they present? This is a followon from the other day... the launcher installer had placed launcher associations in HCU, which are not visible from ftype and assoc commands... so I deleted those, which returned my system to not having launcher associations, but Python 3.2.1 associations instead (they were, and remained, in HLM throughout the install, but HCU temporarily overrode them in some circumstances). So the launcher is installed in c:\windows\system32, but doesn't have associations. Thought I'd play with it from the command line, before reinstalling with ALLUSERS=1 (or establishing the associations by hand). It is still my opinion that the installer should ask whether it should be installed for all users or not, but from what you said the other day, this may be beyond the .msi installer. > >> So I have a command foo.py using Python 3 syntax in a directory on >> my PATH. It works fine, since Python 3.2.1 is in Python.file. >> foo.py does not have a #! line. >> I can successfully execute: foo.py >> However, the following fails: py foo.py >> It fails, because foo.py is not found. Instead, I have to specify: >> py d:\path\to\foo.py >> This is annoying, py should walk the PATH for unqualified files (the >> Windows PATH implicitly includes the current directory, of course, > It's not py's job to walk the path: the shell does that when you just type > "foo". It locates foo.py, and then invokes py because of file association - py > then checks the file for a shebang to decide which Python to dispatch it to. Certainly when the launcher is invoked via an association, this would be the case. However, when the launcher is invoked via the command line, then the unqualified name is passed through. To be useful from the command line, the launcher should walk the PATH to find the .py file. > >> OK, with that mystery solved, and using Notepad running as >> administrator to actually, successfully edit the file, it still runs >> the wrong version of Python. Here is the content of the file, what > I'm afraid I can't reproduce this. When I invoke a script with the default > py.ini, py runs it with Python 2. When I add a [defaults]python=3 entry, py > correctly runs it with Python 3. I still get the same behavior. Is there any debugging output produced by py.exe that would tell what py.ini it looks in, and what the content is? What diagnostic steps might I take to produce additional information that would help you (or me, along the way) analyze the issue? I do not presently have a C compiler installed on this machine, however, unless it came along for a ride with something else. > >> is wrong? And why is the spacing around the = in the [commands] >> section so inconsistent? > That's just test data, not a real "production" py.ini. I was testing out > something, which is why the spaces around = are every which way, and I never got > around to changing it. More importantly, those customised commands, while > perhaps useful for testing, are useless in everyday Python usage: perhaps -O, > -Werror, -E, -S etc. might be more useful. I'll take suggestions as to what > might be useful customised commands to ship as a default. Fine. I realize the launcher is still Beta. This was just a curiosity question. > Regards, > > Vinay Sajip -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdmurray at bitdance.com Wed Jul 20 23:51:03 2011 From: rdmurray at bitdance.com (R. David Murray) Date: Wed, 20 Jul 2011 17:51:03 -0400 Subject: [Python-Dev] [Python-checkins] cpython: #665194: support roundtripping RFC2822 date stamps in the email.utils module In-Reply-To: <20110720224017.4a2ddf83@pitrou.net> References: <4E271A9A.1040703@udel.edu> <20110720224017.4a2ddf83@pitrou.net> Message-ID: <20110720215103.DD85B25008E@webabinitio.net> On Wed, 20 Jul 2011 22:40:17 +0200, Antoine Pitrou wrote: > On Wed, 20 Jul 2011 14:12:42 -0400 > Terry Reedy wrote: > > On 7/20/2011 11:41 AM, r.david.murray wrote: > > > > > diff --git a/Lib/email/utils.py b/Lib/email/utils.py > > > > > # We need wormarounds for bugs in these methods in older Pythons (see below) > > > > Is 'wormaround' (variation of workaround) an intentional play on the > > fact that some worms prey on other 'bugs' ;-? (rather then eating plant > > matter?) > > Or perhaps worms dig their way carefully around known bugs? Hexlify, wormaround...our Barry is just full of interesting words :) -- R. David Murray http://www.bitdance.com From vinay_sajip at yahoo.co.uk Thu Jul 21 00:07:49 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Wed, 20 Jul 2011 22:07:49 +0000 (UTC) Subject: [Python-Dev] New tests in stable versions References: <4E26FB19.2070406@netwok.org> <4E270190.1000209@haypocalc.com> Message-ID: Victor Stinner haypocalc.com> writes: > New logging tests failed during some weeks. If we add new tests, we may > also break some stable buildbots. I don't think that we need to add > these new tests to a stable version. Just for my information, which logging test failures are you referring to? Regards, Vinay Sajip From v+python at g.nevcal.com Thu Jul 21 00:09:49 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Wed, 20 Jul 2011 15:09:49 -0700 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <20110720131415.A2DD43A4100@sparrow.telecommunity.com> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <4E269EBC.90708@g.nevcal.com> <20110720131415.A2DD43A4100@sparrow.telecommunity.com> Message-ID: <4E27522D.8010300@g.nevcal.com> On 7/20/2011 6:05 AM, P.J. Eby wrote: > At 02:24 AM 7/20/2011 -0700, Glenn Linderman wrote: >> When I read about creating __path__ from sys.path, I immediately >> thought of the issue of programs that extend sys.path, and the above >> is the "workaround" for such programs.? but it requires such >> programs to do work, and there are a lot of such programs (I, a >> relative newbie, have had to write some).? As it turns out, I can't >> think of a situation where I have extended sys.path that would result >> in a problem for fancy namespace packages, because so far I've only >> written modules, not packages, and only modules are on the paths that >> I add to sys.path.? But that does not make for a general solution. > > Most programs extend sys.path in order to import things. If those > things aren't yet imported, they don't have a __path__ yet, and so > don't need to be fixed. Only programs that modify sys.path *after* > importing something that has a dynamic __path__ would need to do > anything about that. Sure. But there are a lot of things already imported by Python itself, and if this mechanism gets used in the stdlib, a program wouldn't know whether it is safe or not, to not bother with the pkgutil.extend_virtual_paths() call or not. Plus, that requires importing pkgutil, which isn't necessarily done by every program that extends the sys.path ("import sys" is sufficient at present). Plus, if some 3rd party packages are imported before sys.path is extended, the knowledge of how they are implement is required to make a choice about whether it is needed to import pkgutil and call extend_virtual_paths or not. So I am still left with my original question: > >> Is there some way to create a new __path__ that would reflect the >> fact that it has been dynamically created, rather than set from >> __init__.py, and then when it is referenced, calculate (and cache?) a >> new value of __path__ to actually search? > > That's what extend_virtual_paths() is for. It updates the __path__ of > all currently-imported virtual packages. Where before you wrote: > > sys.path.append('foo') > > You would now write: > > sys.path.append('foo') > pkgutil.extend_virtual_paths('foo') > > ...assuming you have virtual packages you've already imported. If you > don't, there's no reason to call extend_virtual_paths(). But it > doesn't hurt anything if you call it unnecessarily, because it uses > sys.virtual_packages to find out what to update, and if you haven't > imported any virtual packages, there's nothing to update and the call > will be a quick no-op. I think I would have to write sys.path.append('foo') import pkgutil pkgutil.extend_virtual_paths('foo') or I'd get an error. And, in the absence of knowing (because I didn't write them) whether any of the packages I imported before extending sys.path are virtual packages or not, I would have to do this every time I extend sys.path. And so it becomes a burden on writing programs. If the code is so boilerplate as you describe, should sys.path become an object that acts like a list, instead of a list, and have its append method automatically do the pkgutil.extend_virtual_paths for me? Then I wouldn't have to worry about whether any of the packages I imported were virtual packages or not. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Thu Jul 21 00:16:12 2011 From: brett at python.org (Brett Cannon) Date: Wed, 20 Jul 2011 15:16:12 -0700 Subject: [Python-Dev] New tests in stable versions In-Reply-To: References: <4E26FB19.2070406@netwok.org> <4E270190.1000209@haypocalc.com> Message-ID: On Wed, Jul 20, 2011 at 11:48, Terry Reedy wrote: > On 7/20/2011 12:25 PM, Victor Stinner wrote: > >> Le 20/07/2011 17:58, ?ric Araujo a ?crit : >> >>> Do we have a policy of not adding new test files to stable branches? >>> >> New logging tests failed during some weeks. If we add new tests, we may >> also break some stable buildbots. I don't think that we need to add >> these new tests to a stable version. >> > > When bugs are fixed in stable branches, they are usually accompanied by > tests that fail without the bugfix. I have understood the policy to be that > new tests go into stable branches. Failure indicates a bug in either the > not-really-so-stable branch or the test. In the latter case, remove the test > everywhere until fixed. In the former case, either fix the bug in the stable > branch immediately or open an issue and attach the test code (skipping the > test needed stage) or just disable it and note on the issue that a fix patch > should re-enable. The logging tests may have been exceptional some way Right, but Eric is asking about new tests that do nothing more than improve test coverage, not exercise a fix for a bug. I say don't add new tests for the sake of coverage or adding new tests to stable branches. Tests for bugfixes are practically required. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Thu Jul 21 00:51:25 2011 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 20 Jul 2011 15:51:25 -0700 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: <4E274C0F.40400@g.nevcal.com> References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> <4E274C0F.40400@g.nevcal.com> Message-ID: <4E275BED.4050901@stoneleaf.us> Glenn Linderman wrote: > On 7/20/2011 7:19 AM, Vinay Sajip wrote: >> It's not py's job to walk the path: the shell does that when you just type >> "foo". It locates foo.py, and then invokes py because of file association - py >> then checks the file for a shebang to decide which Python to dispatch it to. > > Certainly when the launcher is invoked via an association, this would be > the case. However, when the launcher is invoked via the command line, > then the unqualified name is passed through. To be useful from the > command line, the launcher should walk the PATH to find the .py file. I would say that would be a cool enhancement, as it could save a bit of typing, but I think the launcher is quite useful even without path traversal. ~Ethan~ From pje at telecommunity.com Thu Jul 21 00:39:14 2011 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 20 Jul 2011 18:39:14 -0400 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <20110720170448.D7D793A409B@sparrow.telecommunity.com> <20110720204456.5D1AB3A409B@sparrow.telecommunity.com> Message-ID: <20110720223954.124953A409B@sparrow.telecommunity.com> At 03:22 PM 7/20/2011 -0600, Eric Snow wrote: >On Wed, Jul 20, 2011 at 2:44 PM, P.J. Eby wrote: > > > > So, yeah, actually, that's looking pretty sweet. Basically, we > just have to > > throw a virtual_package_paths dict into the sys module, and do the above > > along with the get_virtual_path() function and add get_subpath() to the > > importer objects, in order to get the PEP's core functionality working. > > >Exactly. That's part of why the importlib approach is so appealing to >me. Actually, it turns out I was a little too optimistic -- the sketch I gave doesn't work right for anything but top-level virtual packages, because I didn't take into account the part where get_virtual_path() needs a parent path. Fixing *that* error then leads to a really nasty bit of mutual recursion in which the parent module imports are attempted over and over again in something like O(N**2), I think. In order to get rid of that, _gcd_import would have to grow some internal memoization so it doesn't retry the same imports repeatedly. Ironically enough, this is because _gcd_import() is recursive, and thus attempts the imports in the opposite order (sort of) than import.c does, which means that you can't get hold of the parent's __path__ without recursing (again). :-( And trying to work around that with memoization, led me to the realization that you actually can't implement PEP 402 using that type of recursion. That is, to implement the spec correctly, _gcd_import is going to have to be refactored to iterate left-to-right over module name parts, rather than recursing right-to-left. That's because PEP 402 only allows for processing a virtual path if a module is not found, *not* if a module is found but can't be loaded. But, with importlib currently being recursive, it only knows that a parent import failed via ImportError, not whether that error arose from failing to find the module, or failing to load the module! So, the core part of the _gcd_import() function will need to be rewritten to iterate instead of recursing. (Still, it's probably not going to be *terribly* difficult. I'll take a look at doing a sketch of that next, but if I do one I'll send it to Import-SIG instead of here; it's not a detail that matters to the general PEP discussion.) From pje at telecommunity.com Thu Jul 21 01:03:58 2011 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 20 Jul 2011 19:03:58 -0400 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <4E27522D.8010300@g.nevcal.com> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <4E269EBC.90708@g.nevcal.com> <20110720131415.A2DD43A4100@sparrow.telecommunity.com> <4E27522D.8010300@g.nevcal.com> Message-ID: <20110720230439.6ADE53A409B@sparrow.telecommunity.com> At 03:09 PM 7/20/2011 -0700, Glenn Linderman wrote: >On 7/20/2011 6:05 AM, P.J. Eby wrote: >>At 02:24 AM 7/20/2011 -0700, Glenn Linderman wrote: >>>When I read about creating __path__ from sys.path, I immediately >>>thought of the issue of programs that extend sys.path, and the >>>above is the "workaround" for such programs.??? but it requires >>>such programs to do work, and there are a lot of such programs (I, >>>a relative newbie, have had to write some).??? As it turns out, I >>>can't think of a situation where I have extended sys.path that >>>would result in a problem for fancy namespace packages, because so >>>far I've only written modules, not packages, and only modules are >>>on the paths that I add to sys.path.??? But that does not make >>>for a general solution. >> >>Most programs extend sys.path in order to import things.? If those >>things aren't yet imported, they don't have a __path__ yet, and so >>don't need to be fixed.? Only programs that modify sys.path >>*after* importing something that has a dynamic __path__ would need >>to do anything about that. > >Sure. But there are a lot of things already imported by Python >itself, and if this mechanism gets used in the stdlib, a program >wouldn't know whether it is safe or not, to not bother with the >pkgutil.extend_virtual_paths() call or not. I'm not sure I see how the mechanism could meaningfully be used in the stdlib, since IIUC we're not going for Perl-style package naming. So, all stdlib packages would be self-contained. >Plus, that requires importing pkgutil, which isn't necessarily done >by every program that extends the sys.path ("import sys" is >sufficient at present). > >Plus, if some 3rd party packages are imported before sys.path is >extended, the knowledge of how they are implement is required to >make a choice about whether it is needed to import pkgutil and call >extend_virtual_paths or not. I'd recommend *always* using it, outside of simple startup code. >So I am still left with my original question: > >>>Is there some way to create a new __path__ that would reflect the >>>fact that it has been dynamically created, rather than set from >>>__init__.py, and then when it is referenced, calculate (and >>>cache?) a new value of __path__ to actually search? Hm. Yes, there is a way to do something like that, but it would complicate things a bit. We'd need to: 1. Leave __path__ off of the modules, and always pull them from sys.virtual_package_paths, and 2. Before using a value in sys.virtual_package_paths, we'd need to check whether sys.path had changed since we last cached anything, and if so, clear sys.virtual_package_paths first, to force a refresh. This doesn't sound particularly forbidding, but there are various unpleasant consequences, like being unable to tell whether a module is a package or not, and whether it's a virtual package or not. We'd have to invent new ways to denote these things. On the bright side, though, it *would* allow transparent live updates to virtual package paths, so it might be worth considering. By the way, the reason we have to get rid of __path__ is that if we kept it, then code could change it, and then we wouldn't know if it was actually safe to change it automatically... even if no code had actually changed it. In principle, we could keep __path__ attributes around, and automatically update them in the case where sys.path has changed, so long as user code hasn't directly altered or replaced the __path__. But it seems to me to be a dangerous corner case; I'd rather that code which touches __path__ be taking responsibility for that path's correctness from then on, rather than having it get updated (possibly incorrectly) behind its back. So, I'd say that for this approach, we'd have to actually leave __path__ off of virtual packages' parent modules. Anyway, it seems worth considering. We just need to sort out what the downsides are for any current tools thinking that such modules aren't packages. (But hey, at least it'll be consistent with what such tools would think of the on-disk representation! That is, a tool that thinks foo.py alongside a foo/ subdirectory is just a module with no package, will also think that 'foo', once imported, is a module with no package.) >And, in the absence of knowing (because I didn't write them) whether >any of the packages I imported before extending sys.path are virtual >packages or not, I would have to do this every time I extend >sys.path.? And so it becomes a burden on writing programs. > >If the code is so boilerplate as you describe, should sys.path >become an object that acts like a list, instead of a list, and have >its append method automatically do the pkgutil.extend_virtual_paths >for me?? Then I wouldn't have to worry about whether any of the >packages I imported were virtual packages or not. Well, then we'd have to worry about other mutation methods, and things like 'sys.path = [blah, blah]', as well. So if we're going to ditch the need for extend_virtual_paths(), we should probably do it via the absence of __path__ attributes. From v+python at g.nevcal.com Thu Jul 21 01:39:35 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Wed, 20 Jul 2011 16:39:35 -0700 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <20110720230439.6ADE53A409B@sparrow.telecommunity.com> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <4E269EBC.90708@g.nevcal.com> <20110720131415.A2DD43A4100@sparrow.telecommunity.com> <4E27522D.8010300@g.nevcal.com> <20110720230439.6ADE53A409B@sparrow.telecommunity.com> Message-ID: <4E276737.8080104@g.nevcal.com> On 7/20/2011 4:03 PM, P.J. Eby wrote: > I'd recommend *always* using it, outside of simple startup code. So that is a burden on every program. Documentation would help, but it certainly makes updating sys.path much more complex -- 3 lines (counting import of pkgutil) instead of one, and the complexity of understanding why there is a need for it, when in simple cases the single line works fine, but it would be bug prone to have both ways. > >> So I am still left with my original question: >> >>>> Is there some way to create a new __path__ that would reflect the >>>> fact that it has been dynamically created, rather than set from >>>> __init__.py, and then when it is referenced, calculate (and cache?) >>>> a new value of __path__ to actually search? > > Hm. Yes, there is a way to do something like that, but it would > complicate things a bit From what you said, it would complicate the solution for complex packaging tasks, but would return simple extensions of sys.path to being simple again. Sounds like a good tradeoff, but I'll leave that to you and other more knowledgeable people to figure out the details and implementation... I snipped the explanation, because it is beyond my present knowledge base. > Anyway, it seems worth considering. We just need to sort out what the > downsides are for any current tools thinking that such modules aren't > packages. (But hey, at least it'll be consistent with what such tools > would think of the on-disk representation! That is, a tool that > thinks foo.py alongside a foo/ subdirectory is just a module with no > package, will also think that 'foo', once imported, is a module with > no package.) Please consider it. I think your initial proposal solves some problems, but a version that doesn't complicate the normal, simple, extension of sys.path would be a much better solution, so I am happy to hear that you have ideas in that regard. Hopefully, they don't complicate things too much more. So far, I haven't gotten my head around packages as they presently exist (this __init__.py stuff seems much more complex than the simplicity of Perl imports that I was used to, although I certainly like many things about Python better than Perl, and have switched whole-heartedly, although I still have a fair bit of Perl code to port in the fullness of time). I think your proposal here, although maintaining some amount of backward-compatibility may require complexity of implementation, can simplify the requirements for creating new packages, to the extent I understand it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From skippy.hammond at gmail.com Thu Jul 21 01:41:09 2011 From: skippy.hammond at gmail.com (Mark Hammond) Date: Thu, 21 Jul 2011 09:41:09 +1000 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: <4E274C0F.40400@g.nevcal.com> References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> <4E274C0F.40400@g.nevcal.com> Message-ID: <4E276795.9050404@gmail.com> On 21/07/2011 7:43 AM, Glenn Linderman wrote: ... > > I still get the same behavior. Is there any debugging output produced > by py.exe that would tell what py.ini it looks in, and what the content > is? What diagnostic steps might I take to produce additional > information that would help you (or me, along the way) analyze the > issue? I do not presently have a C compiler installed on this machine, > however, unless it came along for a ride with something else. It doesn't do exactly what you ask, but setting an environment variable PYLAUNCH_DEBUG to any value will cause some debug output to be generated to stdout. Mark From ncoghlan at gmail.com Thu Jul 21 01:54:21 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 21 Jul 2011 09:54:21 +1000 Subject: [Python-Dev] New tests in stable versions In-Reply-To: References: <4E26FB19.2070406@netwok.org> <4E270190.1000209@haypocalc.com> Message-ID: On Thu, Jul 21, 2011 at 8:16 AM, Brett Cannon wrote: > I say don't add new tests for the sake of coverage or adding new tests to > stable branches. Tests for bugfixes are practically required. I don't *object* to enhanced tests going into maintenance branches, but the workflow of committing directly to trunk is so much simpler that I personally would only apply such changes if the new tests actually uncovered implementation bugs. Then backporting the tests would useful either as part of fixing the same bugs or else to demonstrate that the maintenance branch did not have the problem. So slightly more relaxed than Brett's view: - definitely apply bug fixes and their tests to affected maintenance branches - *optionally* apply coverage enhancements to maintenance branches, but don't feel obliged to do so I see it as a productivity trade-off - time spent improving coverage on multiple branches is time not spent on other things. I'm more than willing to sacrifice some test coverage on the maintenance branches to get even better test coverage and appropriate new features on trunk and more bug fixes on maintenance branches. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From skippy.hammond at gmail.com Thu Jul 21 01:55:28 2011 From: skippy.hammond at gmail.com (Mark Hammond) Date: Thu, 21 Jul 2011 09:55:28 +1000 Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise) In-Reply-To: References: <20110719171647.03f9f9c9@pitrou.net> Message-ID: <4E276AF0.5090409@gmail.com> On 21/07/2011 4:38 AM, Terry Reedy wrote: > Many installers first make an organization directory and then an app > directory within that. This annoys me sometimes when they only have one > app to ever install, but is useful when there might really be multiple > directories, as in our case. (Ditto for start menu entries.) This is > what python should have done a decade ago. I disagree. If we followed that advice we would also be in "\Program Files". I have no problem with multiple Python versions installed directly off the root, especially given most users probably have a very small number of such installations. I think Python being a developer tool rather than a user app is a reasonable justification for that (and the justification used when the existing scheme was decided) > The two proposals > overlap but are not mutually exclusive. For future pythons, 'python33' > is easier to remember and type than 'py -v 3.3' or whatever the proposed > encantation is. 'py -3.3' - less chars to type than 'python33' and with no need to have every Python directory on your PATH. IMO it is also simple enough that people will remember it fairly easily. Also, the launcher supports the ability to select either the 32 or 64bit implementation - so maybe 'python33.exe' isn't really good enough and should reflect the bittedness? > A python directory also gives a sensible (though optional) place to put > other interpreters and even python-based apps. The launcher does not. What other interpreters? IMO it doesn't make sense to have IronPython, jython etc be installed there. Ditto for apps - especially given most apps tend to be tied to a subset of all possible Python versions. Mark From v+python at g.nevcal.com Thu Jul 21 02:02:04 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Wed, 20 Jul 2011 17:02:04 -0700 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: <4E276795.9050404@gmail.com> References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> <4E274C0F.40400@g.nevcal.com> <4E276795.9050404@gmail.com> Message-ID: <4E276C7C.1020605@g.nevcal.com> On 7/20/2011 4:41 PM, Mark Hammond wrote: > On 21/07/2011 7:43 AM, Glenn Linderman wrote: > ... >> >> I still get the same behavior. Is there any debugging output produced >> by py.exe that would tell what py.ini it looks in, and what the content >> is? What diagnostic steps might I take to produce additional >> information that would help you (or me, along the way) analyze the >> issue? I do not presently have a C compiler installed on this machine, >> however, unless it came along for a ride with something else. > > It doesn't do exactly what you ask, but setting an environment > variable PYLAUNCH_DEBUG to any value will cause some debug output to > be generated to stdout. > > Mark > It produces: d:\>py d:\path\to\foo.py launcher build: 64bit launcher executable: Console File 'C:\Users\Glenn\AppData\Roaming\py.ini' non-existent maybe_handle_shebang: read 256 bytes maybe_handle_shebang: BOM not found, using UTF-8 locate_pythons_for_key: unable to open PythonCore key locate_pythons_for_key: unable to open PythonCore key run_child: about to run 'C:\Python26\python.exe d:\path\to\foo.py' File "d:\path\to\foo.py", line 469 child process exit code: 1 So this tells me that it didn't find a local py.ini (no surprise, I don't have one) but doesn't tell me that it did find or read c:\Windows\system32\py.ini much less whether I have the syntax correct for my [defaults] section. It doesn't tell me that it didn't find a #! line (but there isn't one). Perhaps the problem is that there isn't one? If it finds no virtual command, maybe it doesn't obey the [defaults] python=3 directive? The PEP says it should act like '!#python' (I think the PEP meant '#!python', though????). There is no " " after '!#python' in the PEP, so that disqualifies it from being a customized command, there is no '/usr/bin/' nor '/usr/bin/env ' in front of the 'python' so that means it is not a virtual command; and it is not a fully-qualified name, so it doesn't mean that case either.... looks like the PEP needs a bit of clarification here. I do have a python on my path, but it is 3.1.2, not 3.2.1 or 2.6.4, and it runs 2.6.4 as the output shows. And I would expect it to run 3.2.1 with the [defaults] python=3 directive, since that is newer than 3.1.2, which is on my PATH. So, still a mystery. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Thu Jul 21 02:07:31 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 21 Jul 2011 10:07:31 +1000 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: <4E275BED.4050901@stoneleaf.us> References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> <4E274C0F.40400@g.nevcal.com> <4E275BED.4050901@stoneleaf.us> Message-ID: On Thu, Jul 21, 2011 at 8:51 AM, Ethan Furman wrote: > I would say that would be a cool enhancement, as it could save a bit of > typing, but I think the launcher is quite useful even without path > traversal. Two related points: 1. Walking PATH isn't necessary, but the cwd of the py process should be inherited from the shell correctly. If it is, then 'py foo.py' shouldn't need path traversal, it should just look in the current directory. Using PATHEXT to turn 'foo.py' directly into an executable command on PATH from any directory is different and out of scope for the launcher. 2. The defined launched command line handling means that "py -m foo" should also work, so long as the current directory is inherited correctly. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From solipsis at pitrou.net Thu Jul 21 02:08:56 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 21 Jul 2011 02:08:56 +0200 Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise) References: <20110719171647.03f9f9c9@pitrou.net> <4E276AF0.5090409@gmail.com> Message-ID: <20110721020856.2b38528b@pitrou.net> On Thu, 21 Jul 2011 09:55:28 +1000 Mark Hammond wrote: > > > The two proposals > > overlap but are not mutually exclusive. For future pythons, 'python33' > > is easier to remember and type than 'py -v 3.3' or whatever the proposed > > encantation is. > > 'py -3.3' - less chars to type than 'python33' and with no need to have > every Python directory on your PATH. IMO it is also simple enough that > people will remember it fairly easily. Given that Python 2.x has a "-3" option, isn't "py -3.3" kind of confusing, at least to the eye? Regards Antoine. From skippy.hammond at gmail.com Thu Jul 21 02:11:43 2011 From: skippy.hammond at gmail.com (Mark Hammond) Date: Thu, 21 Jul 2011 10:11:43 +1000 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: <4E276C7C.1020605@g.nevcal.com> References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> <4E274C0F.40400@g.nevcal.com> <4E276795.9050404@gmail.com> <4E276C7C.1020605@g.nevcal.com> Message-ID: <4E276EBF.6000509@gmail.com> On 21/07/2011 10:02 AM, Glenn Linderman wrote: > So this tells me that it didn't find a local py.ini (no surprise, I > don't have one) but doesn't tell me that it did find or read > c:\Windows\system32\py.ini much less whether I have the syntax correct > for my [defaults] section. It doesn't tell me that it didn't find a #! > line (but there isn't one). I'll have a go at enhancing the debug output for most of the above (although note that if it *did* find a shebang line extra output would have been generated.) > Perhaps the problem is that there isn't one? If it finds no virtual > command, maybe it doesn't obey the [defaults] python=3 directive? The > PEP says it should act like '!#python' (I think the PEP meant > '#!python', though????). There is no " " after '!#python' in the PEP, > so that disqualifies it from being a customized command, there is no > '/usr/bin/' nor '/usr/bin/env ' in front of the 'python' so that means > it is not a virtual command; and it is not a fully-qualified name, so it > doesn't mean that case either.... looks like the PEP needs a bit of > clarification here. I'm not sure the PEP needs clarification - possibly just the implementation ;) But let me know if you think otherwise. > I do have a python on my path, but it is 3.1.2, not 3.2.1 or 2.6.4, and > it runs 2.6.4 as the output shows. FYI, what is on the path isn't relevant to the launcher. > And I would expect it to run 3.2.1 > with the [defaults] python=3 directive, since that is newer than 3.1.2, > which is on my PATH. It may be that your copy of the launcher is a little old - some changes were pushed just yesterday (but I'm not sure if Vinay made a new installer yet). It has slightly better debug output (although generally not what you are asking for yet) and better "cross-bittedness" support. Cheers, Mark From v+python at g.nevcal.com Thu Jul 21 02:13:57 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Wed, 20 Jul 2011 17:13:57 -0700 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: <4E274C0F.40400@g.nevcal.com> References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> <4E274C0F.40400@g.nevcal.com> Message-ID: <4E276F45.7070105@g.nevcal.com> On 7/20/2011 2:43 PM, Glenn Linderman wrote: >> It's not py's job to walk the path: the shell does that when you just type >> "foo". It locates foo.py, and then invokes py because of file association - py >> then checks the file for a shebang to decide which Python to dispatch it to. > > Certainly when the launcher is invoked via an association, this would > be the case. However, when the launcher is invoked via the command > line, then the unqualified name is passed through. To be useful from > the command line, the launcher should walk the PATH to find the .py file. The Windows SearchPath API makes it a pretty easy job to walk the PATH. Would even allow low cost additional feature of searching for both foo and foo.py at the same time. -------------- next part -------------- An HTML attachment was scrubbed... URL: From skippy.hammond at gmail.com Thu Jul 21 02:17:44 2011 From: skippy.hammond at gmail.com (Mark Hammond) Date: Thu, 21 Jul 2011 10:17:44 +1000 Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise) In-Reply-To: <20110721020856.2b38528b@pitrou.net> References: <20110719171647.03f9f9c9@pitrou.net> <4E276AF0.5090409@gmail.com> <20110721020856.2b38528b@pitrou.net> Message-ID: <4E277028.4040202@gmail.com> On 21/07/2011 10:08 AM, Antoine Pitrou wrote: > On Thu, 21 Jul 2011 09:55:28 +1000 > Mark Hammond wrote: >> >> > The two proposals >>> overlap but are not mutually exclusive. For future pythons, 'python33' >>> is easier to remember and type than 'py -v 3.3' or whatever the proposed >>> encantation is. >> >> 'py -3.3' - less chars to type than 'python33' and with no need to have >> every Python directory on your PATH. IMO it is also simple enough that >> people will remember it fairly easily. > > Given that Python 2.x has a "-3" option, isn't "py -3.3" kind of > confusing, at least to the eye? A little, yeah, but IMO practicality beats purity here. I'd probably feel different if I felt 'python -3' was regularly used and would continue to be so in the future. Also, I think most people who would potentially use 'python -3' will be aware that running 'py' is a totally different command and will adjust accordingly (either by continuing to use 'python -3' or adjusting to running 'py -2 -3'.) The PEP does make explicit mention of this... Cheers, Mark From v+python at g.nevcal.com Thu Jul 21 02:18:33 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Wed, 20 Jul 2011 17:18:33 -0700 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> <4E274C0F.40400@g.nevcal.com> <4E275BED.4050901@stoneleaf.us> Message-ID: <4E277059.3080301@g.nevcal.com> On 7/20/2011 5:07 PM, Nick Coghlan wrote: > On Thu, Jul 21, 2011 at 8:51 AM, Ethan Furman wrote: >> I would say that would be a cool enhancement, as it could save a bit of >> typing, but I think the launcher is quite useful even without path >> traversal. > Two related points: > > 1. Walking PATH isn't necessary, but the cwd of the py process should > be inherited from the shell correctly. If it is, then 'py foo.py' > shouldn't need path traversal, it should just look in the current > directory. Using PATHEXT to turn 'foo.py' directly into an executable > command on PATH from any directory is different and out of scope for > the launcher. Sorry, I disagree that it is out of scope. Looking in the current directory is fine, when the script is there, but my scripts are seldom in my data directories, and I want to run scripts (from the PATH) on data that is in the CWD. I consider this a _very common_ use case for using scripts/programs, but then if you want to use py from the command line to tweak which version of Python gets used to execute the script (if the default one didn't work, for example, and you want to try a different one), then suddenly, you have to find the path to the script, and specify it explicitly. > 2. The defined launched command line handling means that "py -m foo" > should also work, so long as the current directory is inherited > correctly. Even if CWD is a data directory, and foo.py is somewhere on the path? > Cheers, > Nick. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From v+python at g.nevcal.com Thu Jul 21 02:22:03 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Wed, 20 Jul 2011 17:22:03 -0700 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: <4E276EBF.6000509@gmail.com> References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> <4E274C0F.40400@g.nevcal.com> <4E276795.9050404@gmail.com> <4E276C7C.1020605@g.nevcal.com> <4E276EBF.6000509@gmail.com> Message-ID: <4E27712B.2030704@g.nevcal.com> On 7/20/2011 5:11 PM, Mark Hammond wrote: > On 21/07/2011 10:02 AM, Glenn Linderman wrote: >> So this tells me that it didn't find a local py.ini (no surprise, I >> don't have one) but doesn't tell me that it did find or read >> c:\Windows\system32\py.ini much less whether I have the syntax correct >> for my [defaults] section. It doesn't tell me that it didn't find a #! >> line (but there isn't one). > > I'll have a go at enhancing the debug output for most of the above > (although note that if it *did* find a shebang line extra output would > have been generated.) > >> Perhaps the problem is that there isn't one? If it finds no virtual >> command, maybe it doesn't obey the [defaults] python=3 directive? The >> PEP says it should act like '!#python' (I think the PEP meant >> '#!python', though????). There is no " " after '!#python' in the PEP, >> so that disqualifies it from being a customized command, there is no >> '/usr/bin/' nor '/usr/bin/env ' in front of the 'python' so that means >> it is not a virtual command; and it is not a fully-qualified name, so it >> doesn't mean that case either.... looks like the PEP needs a bit of >> clarification here. > > I'm not sure the PEP needs clarification - possibly just the > implementation ;) But let me know if you think otherwise. Well, at least !# should be changed to #! :) And then the example of "customized command" shows "#! vpython" with the space before the vpython, but the text describes the necessary space as being after the customized command, at least the way I read it. So I really don't know whether the example is showing an extraneous space after the ! (and not one after vpython) or exactly what is meant. > >> I do have a python on my path, but it is 3.1.2, not 3.2.1 or 2.6.4, and >> it runs 2.6.4 as the output shows. > > FYI, what is on the path isn't relevant to the launcher. I didn't expect it to be, I just mentioned it in passing, because the launcher doesn't seem to be doing what I expect it to do, but neither does it seem to be using the PATH to find a Python either. > >> And I would expect it to run 3.2.1 >> with the [defaults] python=3 directive, since that is newer than 3.1.2, >> which is on my PATH. > > It may be that your copy of the launcher is a little old - some > changes were pushed just yesterday (but I'm not sure if Vinay made a > new installer yet). It has slightly better debug output (although > generally not what you are asking for yet) and better > "cross-bittedness" support. > > Cheers, > > Mark > Mine's a week old, yes. -------------- next part -------------- An HTML attachment was scrubbed... URL: From skippy.hammond at gmail.com Thu Jul 21 02:34:34 2011 From: skippy.hammond at gmail.com (Mark Hammond) Date: Thu, 21 Jul 2011 10:34:34 +1000 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: <4E276F45.7070105@g.nevcal.com> References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> <4E274C0F.40400@g.nevcal.com> <4E276F45.7070105@g.nevcal.com> Message-ID: <4E27741A.1040202@gmail.com> On 21/07/2011 10:13 AM, Glenn Linderman wrote: > On 7/20/2011 2:43 PM, Glenn Linderman wrote: >>> It's not py's job to walk the path: the shell does that when you just type >>> "foo". It locates foo.py, and then invokes py because of file association - py >>> then checks the file for a shebang to decide which Python to dispatch it to. >> >> Certainly when the launcher is invoked via an association, this would >> be the case. However, when the launcher is invoked via the command >> line, then the unqualified name is passed through. To be useful from >> the command line, the launcher should walk the PATH to find the .py file. > > The Windows SearchPath API > > makes it a pretty easy job to walk the PATH. Would even allow low cost > additional feature of searching for both foo and foo.py at the same time. But that would also require us to parse the command-line, understand which options require 2 args and which just require 1 (making it fragile in the face of later versions introducing new options), then re-stitching a modified command-line back together for the child process. IMO that is all out of scope. Are you maybe forgetting about the PATHEXT functionality? If you include .py in your PATHEXT and file associations are set so .py files are handled by the launcher, you should be able to directly execute just 'foo' and have the command processor search the PATH for a foo.py and invoke it via the launcher. Mark From skippy.hammond at gmail.com Thu Jul 21 02:27:31 2011 From: skippy.hammond at gmail.com (Mark Hammond) Date: Thu, 21 Jul 2011 10:27:31 +1000 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: <4E277059.3080301@g.nevcal.com> References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> <4E274C0F.40400@g.nevcal.com> <4E275BED.4050901@stoneleaf.us> <4E277059.3080301@g.nevcal.com> Message-ID: <4E277273.5060508@gmail.com> On 21/07/2011 10:18 AM, Glenn Linderman wrote: > On 7/20/2011 5:07 PM, Nick Coghlan wrote: >> On Thu, Jul 21, 2011 at 8:51 AM, Ethan Furman wrote: >>> I would say that would be a cool enhancement, as it could save a bit of >>> typing, but I think the launcher is quite useful even without path >>> traversal. >> Two related points: >> >> 1. Walking PATH isn't necessary, but the cwd of the py process should >> be inherited from the shell correctly. If it is, then 'py foo.py' >> shouldn't need path traversal, it should just look in the current >> directory. Using PATHEXT to turn 'foo.py' directly into an executable >> command on PATH from any directory is different and out of scope for >> the launcher. > > Sorry, I disagree that it is out of scope. Looking in the current > directory is fine, when the script is there, but my scripts are seldom > in my data directories, and I want to run scripts (from the PATH) on > data that is in the CWD. I consider this a _very common_ use case for > using scripts/programs, but then if you want to use py from the command > line to tweak which version of Python gets used to execute the script > (if the default one didn't work, for example, and you want to try a > different one), then suddenly, you have to find the path to the script, > and specify it explicitly. The arguments above apply equally to python.exe. The launcher's job is to find an appropriate python.exe and launch it, not to locate the scripts and all the command-line parsing that would entail. If you want this feature you should advocate for it to be added to Python itself and it will then automatically work in the launcher too. Mark From ben+python at benfinney.id.au Thu Jul 21 03:31:21 2011 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 21 Jul 2011 11:31:21 +1000 Subject: [Python-Dev] Indentation in reStructuredText documents (was: [Python-checkins] peps: Restore whitespace characters lost via email transmission.) References: <4E26E468.20800@netwok.org> Message-ID: <87livss9li.fsf_-_@benfinney.id.au> ?ric Araujo writes: > FYI, reST uses three-space indents, not four (so that blocks align > nicely under the leading two dots + one space), so I think the change > was intentional. No, reST doesn't specify any particular level of indentation. Like most Python programmers I prefer four-space indentation, so I do the same in my reST documents:: .. epigraph:: ?Foo bar baz? ?Fnorble I strongly recommend the same for Python documentation. -- \ ?I was in the first submarine. Instead of a periscope, they had | `\ a kaleidoscope. ?We're surrounded.?? ?Steven Wright | _o__) | Ben Finney From ncoghlan at gmail.com Thu Jul 21 03:52:23 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 21 Jul 2011 11:52:23 +1000 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <20110720230439.6ADE53A409B@sparrow.telecommunity.com> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <4E269EBC.90708@g.nevcal.com> <20110720131415.A2DD43A4100@sparrow.telecommunity.com> <4E27522D.8010300@g.nevcal.com> <20110720230439.6ADE53A409B@sparrow.telecommunity.com> Message-ID: On Thu, Jul 21, 2011 at 9:03 AM, P.J. Eby wrote: > Hm. ?Yes, there is a way to do something like that, but it would complicate > things a bit. ?We'd need to: > > 1. Leave __path__ off of the modules, and always pull them from > sys.virtual_package_paths, and Setting __path__ to a sentinel value (imp.VirtualPath?) would break less code, as hasattr(mod, '__path__') checks would still work. Even better would be for these (and sys.path) to be list subclasses that did the right thing under the hood as Glenn suggested. Code that *replaces* rather than modifies these attributes would still potentially break virtual packages, but code that modifies them in place would do the right thing automatically. (Note that all code that manipulates sys.path and __path__ attributes requires explicit calls to correctly support current namespace package mechanisms, so this would actually be an improvement on the status quo rather than making anything worse). I'll note that this kind of thing is one of the key reasons the import state should some day move to a real class - state coherency is one of the major use cases for the descriptor protocol, which is unavailable when interdependent state is stored as module attributes. (Don't worry, that day is a very long way away, if it ever happens at all) > 2. Before using a value in sys.virtual_package_paths, we'd need to check > whether sys.path had changed since we last cached anything, and if so, clear > sys.virtual_package_paths first, to force a refresh. > > This doesn't sound particularly forbidding, but there are various unpleasant > consequences, like being unable to tell whether a module is a package or > not, and whether it's a virtual package or not. ?We'd have to invent new > ways to denote these things. Trying to change how packages are identified at the Python level makes PEP 382 sound positively appealing. __path__ needs to stay :) Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From brian.curtin at gmail.com Thu Jul 21 04:24:38 2011 From: brian.curtin at gmail.com (Brian Curtin) Date: Wed, 20 Jul 2011 21:24:38 -0500 Subject: [Python-Dev] Indentation in reStructuredText documents (was: [Python-checkins] peps: Restore whitespace characters lost via email transmission.) In-Reply-To: <87livss9li.fsf_-_@benfinney.id.au> References: <4E26E468.20800@netwok.org> <87livss9li.fsf_-_@benfinney.id.au> Message-ID: On Wed, Jul 20, 2011 at 20:31, Ben Finney wrote: > ?ric Araujo writes: > > > FYI, reST uses three-space indents, not four (so that blocks align > > nicely under the leading two dots + one space), so I think the change > > was intentional. > > No, reST doesn't specify any particular level of indentation. Like most > Python programmers I prefer four-space indentation, so I do the same in > my reST documents:: > > .. epigraph:: > ?Foo bar baz? ?Fnorble > > I strongly recommend the same for Python documentation. > We already use three and it seems to look and function properly. -1 to a mass re-spacing. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ericsnowcurrently at gmail.com Thu Jul 21 04:39:19 2011 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Wed, 20 Jul 2011 20:39:19 -0600 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <4E269EBC.90708@g.nevcal.com> <20110720131415.A2DD43A4100@sparrow.telecommunity.com> <4E27522D.8010300@g.nevcal.com> <20110720230439.6ADE53A409B@sparrow.telecommunity.com> Message-ID: On Wed, Jul 20, 2011 at 7:52 PM, Nick Coghlan wrote: > Even better would be for these (and sys.path) to be list subclasses > that did the right thing under the hood as Glenn suggested. Code that > *replaces* rather than modifies these attributes would still > potentially break virtual packages, but code that modifies them in > place would do the right thing automatically. (Note that all code that > manipulates sys.path and __path__ attributes requires explicit calls > to correctly support current namespace package mechanisms, so this > would actually be an improvement on the status quo rather than making > anything worse). +1 as a solution to the problem Glenn brought up. However, I'm still not clear on how much code out there changes sys.path in the offending way, forcing the need to provide a more implicit solution in this PEP than extend_virtual_paths(). And in cases where sys.path *is* changed, and it impacts some virtual package, how many places is that going to happen in one project? My guess is not many (and so not many "boilerplate" calls). Is it worth adding implicit __path__ updates for that use case, rather than just the extend_virtual_paths() function? As an aside, my first reaction to Glenn's suggestion was "that would be cool". Would it be a pursuable option? We can take this over to import-sig if it is. -eric From stephen at xemacs.org Thu Jul 21 04:45:10 2011 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 21 Jul 2011 11:45:10 +0900 Subject: [Python-Dev] [Python-checkins] cpython: #665194: support roundtripping RFC2822 date stamps in the email.utils module In-Reply-To: <20110720215103.DD85B25008E@webabinitio.net> References: <4E271A9A.1040703@udel.edu> <20110720224017.4a2ddf83@pitrou.net> <20110720215103.DD85B25008E@webabinitio.net> Message-ID: <87aac8s66h.fsf@uwakimon.sk.tsukuba.ac.jp> R. David Murray writes: > Hexlify, wormaround...our Barry is just full of interesting words :) Not "full" at all---there's no there there to put them in. He's a generator! The FLUFL-always-uses-efficient-idioms-ly y'rs, From rdmurray at bitdance.com Thu Jul 21 04:52:15 2011 From: rdmurray at bitdance.com (R. David Murray) Date: Wed, 20 Jul 2011 22:52:15 -0400 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: <4E277273.5060508@gmail.com> References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> <4E274C0F.40400@g.nevcal.com> <4E275BED.4050901@stoneleaf.us> <4E277059.3080301@g.nevcal.com> <4E277273.5060508@gmail.com> Message-ID: <20110721025216.63DCF2500D6@webabinitio.net> On Thu, 21 Jul 2011 10:27:31 +1000, Mark Hammond wrote: > On 21/07/2011 10:18 AM, Glenn Linderman wrote: > > On 7/20/2011 5:07 PM, Nick Coghlan wrote: > >> On Thu, Jul 21, 2011 at 8:51 AM, Ethan Furman wrote: > >>> I would say that would be a cool enhancement, as it could save a bit of > >>> typing, but I think the launcher is quite useful even without path > >>> traversal. > >> Two related points: > >> > >> 1. Walking PATH isn't necessary, but the cwd of the py process should > >> be inherited from the shell correctly. If it is, then 'py foo.py' > >> shouldn't need path traversal, it should just look in the current > >> directory. Using PATHEXT to turn 'foo.py' directly into an executable > >> command on PATH from any directory is different and out of scope for > >> the launcher. > > > > Sorry, I disagree that it is out of scope. Looking in the current > > directory is fine, when the script is there, but my scripts are seldom > > in my data directories, and I want to run scripts (from the PATH) on > > data that is in the CWD. I consider this a _very common_ use case for > > using scripts/programs, but then if you want to use py from the command > > line to tweak which version of Python gets used to execute the script > > (if the default one didn't work, for example, and you want to try a > > different one), then suddenly, you have to find the path to the script, > > and specify it explicitly. > > The arguments above apply equally to python.exe. The launcher's job is > to find an appropriate python.exe and launch it, not to locate the > scripts and all the command-line parsing that would entail. If you want > this feature you should advocate for it to be added to Python itself and > it will then automatically work in the launcher too. Indeed. If I want to run a script with a different python version on a unix-like system, I need to know the path to said script. We're trying to make python as easy to use on Windows as it is on Unix. If find-script-on-path is considered a worthwhile feature, then as Mark said it should be added to base Python (on all platforms), not special-cased in the Windows launcher. -- R. David Murray http://www.bitdance.com From ncoghlan at gmail.com Thu Jul 21 05:22:11 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 21 Jul 2011 13:22:11 +1000 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: <20110721025216.63DCF2500D6@webabinitio.net> References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> <4E274C0F.40400@g.nevcal.com> <4E275BED.4050901@stoneleaf.us> <4E277059.3080301@g.nevcal.com> <4E277273.5060508@gmail.com> <20110721025216.63DCF2500D6@webabinitio.net> Message-ID: On Thu, Jul 21, 2011 at 12:52 PM, R. David Murray wrote: > Indeed. ?If I want to run a script with a different python version > on a unix-like system, I need to know the path to said script. > We're trying to make python as easy to use on Windows as it is on Unix. > If find-script-on-path is considered a worthwhile feature, then as Mark > said it should be added to base Python (on all platforms), not special-cased > in the Windows launcher. And given the diverse range of what Python considers to be an executable script these days, -1000 to that particular feature. Use `which scriptname`, that's what it's for. The lack of such functionality in the underpowered cmd shell on Windows isn't Python's problem to solve - ask MS for a better shell and command line utilities (assuming Powershell doesn't already offer something comparable to 'which'). There are reasons I only code specifically for Windows if someone pays me to do so :P Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From lekmalek at gmail.com Thu Jul 21 08:05:52 2011 From: lekmalek at gmail.com (lekmalek) Date: Thu, 21 Jul 2011 08:05:52 +0200 Subject: [Python-Dev] Issue10271 - warnings.showwarning should allow any callable object - request commiter In-Reply-To: References: <20110716103322.330f805c@vimes> Message-ID: <20110721080552.40e767a1@vimes> On Sun, 17 Jul 2011 19:19:59 -0700 Brett Cannon wrote: > Just so people know, I went ahead and fixed this for 3.3 (but not for > 3.2 since it changes the API in a subtle way). Yeah, but that shouldn't break anything. Anyway, thanks a lot for your help, I'm happy it's in 3.3. lekma > > On Sat, Jul 16, 2011 at 01:33, lekmalek wrote: > > > Hello all, > > > > Can any of you core devs have a look at > > http://bugs.python.org/issue10271. It seems Brett is really busy > > right now and this uncontroversial (AFAICT) one liner only needs > > someone to review it and commit it. The pb is, it's holding me back > > a little bit, and I really would like to have it in the next 3.2 > > release if possible. > > > > Thanks for your help, > > > > lekma > > _______________________________________________ > > Python-Dev mailing list > > Python-Dev at python.org > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > > http://mail.python.org/mailman/options/python-dev/brett%40python.org > > From skippy.hammond at gmail.com Thu Jul 21 08:35:38 2011 From: skippy.hammond at gmail.com (Mark Hammond) Date: Thu, 21 Jul 2011 16:35:38 +1000 Subject: [Python-Dev] New update to PEP397 - Python launcher for Windows Message-ID: <4E27C8BA.8040000@gmail.com> I've updated PEP 397 - "Python launcher for Windows" based on recent discussions and Vinay's implementation. http://www.python.org/dev/peps/pep-0397/ and a copy is attached for your convenience. The main changes are: * All mentions of the Python reference implementation have been removed now that a C implementation exists and is a more accurate implementation of the PEP than the Python version. * Some "implementation details" have been removed and links added to the launcher docs. This was done mainly so the implementation is free to make changes based on user feedback and still stay true to the PEP. Note that the launcher doc link doesn't exist right now, but should magically appear over the next couple of days as Vinay pushes some changes I just sent him. * The recommendation to install into System32 has been changed to a recommendation to install directly into \Windows, as the System32 directory is not on the default path for 32bit processes on a 64bit Windows. Vinay even has a version of an MSI installer which installs into this directory. The PEP also gives installers more leeway on where to install the launcher if this directory is not writable. I think this PEP is getting close to being finalized - please suggest whatever changes you feel are necessary ASAP as soon I'll be asking for pronouncement. Thanks - especially to Vinay for taking on the C implementation! Mark -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: pep-0397.txt URL: From raymond.hettinger at gmail.com Thu Jul 21 08:58:14 2011 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Wed, 20 Jul 2011 23:58:14 -0700 Subject: [Python-Dev] New tests in stable versions In-Reply-To: References: <4E26FB19.2070406@netwok.org> <4E270190.1000209@haypocalc.com> Message-ID: On Jul 20, 2011, at 3:16 PM, Brett Cannon wrote: > > > On Wed, Jul 20, 2011 at 11:48, Terry Reedy wrote: > On 7/20/2011 12:25 PM, Victor Stinner wrote: > Le 20/07/2011 17:58, ?ric Araujo a ?crit : > Do we have a policy of not adding new test files to stable branches? > New logging tests failed during some weeks. If we add new tests, we may > also break some stable buildbots. I don't think that we need to add > these new tests to a stable version. > > When bugs are fixed in stable branches, they are usually accompanied by tests that fail without the bugfix. I have understood the policy to be that new tests go into stable branches. Failure indicates a bug in either the not-really-so-stable branch or the test. In the latter case, remove the test everywhere until fixed. In the former case, either fix the bug in the stable branch immediately or open an issue and attach the test code (skipping the test needed stage) or just disable it and note on the issue that a fix patch should re-enable. The logging tests may have been exceptional some way > > Right, but Eric is asking about new tests that do nothing more than improve test coverage, not exercise a fix for a bug. > > I say don't add new tests for the sake of coverage or adding new tests to stable branches. Tests for bugfixes are practically required. I concur with Brett. Nothing good will come from backporting tests that aren't aimed at a specific bugfix. Raymond -------------- next part -------------- An HTML attachment was scrubbed... URL: From v+python at g.nevcal.com Thu Jul 21 10:06:57 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Thu, 21 Jul 2011 01:06:57 -0700 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> <4E274C0F.40400@g.nevcal.com> <4E275BED.4050901@stoneleaf.us> <4E277059.3080301@g.nevcal.com> <4E277273.5060508@gmail.com> <20110721025216.63DCF2500D6@webabinitio.net> Message-ID: <4E27DE21.4050304@g.nevcal.com> On 7/20/2011 8:22 PM, Nick Coghlan wrote: > On Thu, Jul 21, 2011 at 12:52 PM, R. David Murray wrote: >> Indeed. If I want to run a script with a different python version >> on a unix-like system, I need to know the path to said script. >> We're trying to make python as easy to use on Windows as it is on Unix. >> If find-script-on-path is considered a worthwhile feature, then as Mark >> said it should be added to base Python (on all platforms), not special-cased >> in the Windows launcher. > And given the diverse range of what Python considers to be an > executable script these days, -1000 to that particular feature. Use > `which scriptname`, that's what it's for. The lack of such > functionality in the underpowered cmd shell on Windows isn't Python's > problem to solve - ask MS for a better shell and command line > utilities (assuming Powershell doesn't already offer something > comparable to 'which'). > > There are reasons I only code specifically for Windows if someone pays > me to do so :P Interesting feedback. Well, I have a "which" program on my machine, but as a 32-bit executable, it won't find py in the 64-bit c:\windows\system32 directory! Another good reason to demand pay for Windows programming. There are some interesting gotchas to the way 32- vs 64-bit "compatibility" is achieved in Windows (groan). I'll find or write a better one, in due time. Meantime, the launcher testing has been a good learning exercise for me. Interesting, David, that you feel it that Python usability on Windows should be limited to its usability on Unix, rather than to exceed it. I don't see that as a necessary or appropriate limit. Windows and Unix are different. Unix people are accustomed to using tools like which, and using command lines, and path manipulations; Windows people are not. So the use of the command line is already somewhat foreign to them, and limiting the launcher so that they have to use other command line tools to get the work done, would only serve to frustrate them. Now the argument can possibly be made that people that want to use launcher from the command line would be those that are already command line experts might be realistic, but I will note that Perl has a -S option to find its script on the PATH, not that that is a sufficient reason to add such to Python, or even to the launcher, but just to note that there are at least some people besides myself that might think that is a friendly idea. My goal in writing software is to make it easy to use, regardless of whether some other system should be the responsible party or not -- especially when I don't control the other system. But then, I haven't found time to write a competing launcher, either, with friendlier features. So, I'll just reiterate that I would find it friendly if the launcher searched the path to find the script, and agree that if Python had a feature to do so, that would also be friendly to the Windows platform, but less necessary on Unix where you can `which script.py` (although that would still require more typing than if the path searching were automatic. -------------- next part -------------- An HTML attachment was scrubbed... URL: From v+python at g.nevcal.com Thu Jul 21 10:13:56 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Thu, 21 Jul 2011 01:13:56 -0700 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: <4E27741A.1040202@gmail.com> References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> <4E274C0F.40400@g.nevcal.com> <4E276F45.7070105@g.nevcal.com> <4E27741A.1040202@gmail.com> Message-ID: <4E27DFC4.1000801@g.nevcal.com> On 7/20/2011 5:34 PM, Mark Hammond wrote: > On 21/07/2011 10:13 AM, Glenn Linderman wrote: >> On 7/20/2011 2:43 PM, Glenn Linderman wrote: >>>> It's not py's job to walk the path: the shell does that when you >>>> just type >>>> "foo". It locates foo.py, and then invokes py because of file >>>> association - py >>>> then checks the file for a shebang to decide which Python to >>>> dispatch it to. >>> >>> Certainly when the launcher is invoked via an association, this would >>> be the case. However, when the launcher is invoked via the command >>> line, then the unqualified name is passed through. To be useful from >>> the command line, the launcher should walk the PATH to find the .py >>> file. >> >> The Windows SearchPath API >> >> makes it a pretty easy job to walk the PATH. Would even allow low cost >> additional feature of searching for both foo and foo.py at the >> same time. > > But that would also require us to parse the command-line, understand > which options require 2 args and which just require 1 (making it > fragile in the face of later versions introducing new options), then > re-stitching a modified command-line back together for the child > process. IMO that is all out of scope. Yes it would be more work. > > Are you maybe forgetting about the PATHEXT functionality? If you > include .py in your PATHEXT and file associations are set so .py files > are handled by the launcher, you should be able to directly execute > just 'foo' and have the command processor search the PATH for a foo.py > and invoke it via the launcher. Not at all. I was just testing the use of the launcher from the command line, and seeing the cumbersomeness of using it as a prefix to a command that would work on its own, but fails when launched by the launcher from the command line. And the SearchPath API has a limited form of PATHEXT support built in, which is what I was trying to point out above. Yes, the use of the launcher via file associations can be friendly because it leverages Windows features, but its use from the command line presently seems rather unfriendly, because it leverages Windows misfeatures. > > > Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From v+python at g.nevcal.com Thu Jul 21 10:32:59 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Thu, 21 Jul 2011 01:32:59 -0700 Subject: [Python-Dev] New update to PEP397 - Python launcher for Windows In-Reply-To: <4E27C8BA.8040000@gmail.com> References: <4E27C8BA.8040000@gmail.com> Message-ID: <4E27E43B.2070906@g.nevcal.com> On 7/20/2011 11:35 PM, Mark Hammond wrote: > * If the command starts with the definition of a customized command > followed by a space character, the customized command will be used. > See below for a description of customized commands. > Then a shebang line of '#! vpython' in a script named 'doit.py' will > result in the launcher using the command-line 'c:\bin\vpython.exe -foo > doit.py' Shouldn't the second paragraph include a space before the 2nd ' ? In other words, the command as quoted does not show a "customized command followed by a space character" as the definition requires. I don't know why the space character is required, or maybe "white space" was meant, so that the line terminating newline character qualifies also to delimit the customized command? -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Thu Jul 21 10:47:57 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 21 Jul 2011 18:47:57 +1000 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: <4E27DE21.4050304@g.nevcal.com> References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> <4E274C0F.40400@g.nevcal.com> <4E275BED.4050901@stoneleaf.us> <4E277059.3080301@g.nevcal.com> <4E277273.5060508@gmail.com> <20110721025216.63DCF2500D6@webabinitio.net> <4E27DE21.4050304@g.nevcal.com> Message-ID: On Thu, Jul 21, 2011 at 6:06 PM, Glenn Linderman wrote: > Interesting, David, that you feel it that Python usability on Windows should > be limited to its usability on Unix, rather than to exceed it. I don't see > that as a necessary or appropriate limit.? Windows and Unix are different. > Unix people are accustomed to using tools like which, and using command > lines, and path manipulations; Windows people are not.? So the use of the > command line is already somewhat foreign to them, and limiting the launcher > so that they have to use other command line tools to get the work done, > would only serve to frustrate them.? Now the argument can possibly be made > that people that want to use launcher from the command line would be those > that are already command line experts might be realistic, but I will note > that Perl has a -S option to find its script on the PATH, not that that is a > sufficient reason to add such to Python, or even to the launcher, but just > to note that there are at least some people besides myself that might think > that is a friendly idea. If a PEP is put forward proposing such a feature, with a reference implementation that supports at least Windows, *nix and OS X and works for at least the 4 script types understood by the CPython executable without a command line option (source and bytecode files along with directories and zipfiles providing top level __main__ modules), then I'd be prepared to moderate my response all the way to a +0 (reserving the extreme negative reaction for proposals that are either platform specific or only handle some script types). I believe that is significantly easier said than done, though. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From skippy.hammond at gmail.com Thu Jul 21 10:51:15 2011 From: skippy.hammond at gmail.com (Mark Hammond) Date: Thu, 21 Jul 2011 18:51:15 +1000 Subject: [Python-Dev] New update to PEP397 - Python launcher for Windows In-Reply-To: <4E27E43B.2070906@g.nevcal.com> References: <4E27C8BA.8040000@gmail.com> <4E27E43B.2070906@g.nevcal.com> Message-ID: <4E27E883.2060404@gmail.com> On 21/07/2011 6:32 PM, Glenn Linderman wrote: > On 7/20/2011 11:35 PM, Mark Hammond wrote: >> * If the command starts with the definition of a customized command >> followed by a space character, the customized command will be used. >> See below for a description of customized commands. > >> Then a shebang line of '#! vpython' in a script named 'doit.py' will >> result in the launcher using the command-line 'c:\bin\vpython.exe -foo >> doit.py' > > Shouldn't the second paragraph include a space before the 2nd ' ? In > other words, the command as quoted does not show a "customized command > followed by a space character" as the definition requires. I don't know > why the space character is required, or maybe "white space" was meant, > so that the line terminating newline character qualifies also to delimit > the customized command? Yes, thanks for pointing that out - I've pushed a new version of the PEP with that paragraph reading as: """ If the command starts with the definition of a customized command followed by a whitespace character (including a newline), the customized command will be used. See below for a description of customized commands. """ Thanks, Mark > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/skippy.hammond%40gmail.com From victor.stinner at haypocalc.com Thu Jul 21 12:44:11 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Thu, 21 Jul 2011 12:44:11 +0200 Subject: [Python-Dev] New tests in stable versions In-Reply-To: References: <4E26FB19.2070406@netwok.org> <4E270190.1000209@haypocalc.com> Message-ID: <4E2802FB.9080009@haypocalc.com> On 21/07/2011 00:07, Vinay Sajip wrote: > Victor Stinner haypocalc.com> writes: > >> New logging tests failed during some weeks. If we add new tests, we may >> also break some stable buildbots. I don't think that we need to add >> these new tests to a stable version. > Just for my information, which logging test failures are you referring to? Sorry, I don't remember the details, I just remember that some new tests added to test_logging were failing during some days/weeks. Victor From rdmurray at bitdance.com Thu Jul 21 14:00:10 2011 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 21 Jul 2011 08:00:10 -0400 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: <4E27DE21.4050304@g.nevcal.com> References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> <4E274C0F.40400@g.nevcal.com> <4E275BED.4050901@stoneleaf.us> <4E277059.3080301@g.nevcal.com> <4E277273.5060508@gmail.com> <20110721025216.63DCF2500D6@webabinitio.net> <4E27DE21.4050304@g.nevcal.com> Message-ID: <20110721120011.2E460B14005@webabinitio.net> On Thu, 21 Jul 2011 01:06:57 -0700, Glenn Linderman wrote: > Interesting, David, that you feel it that Python usability on Windows > should be limited to its usability on Unix, rather than to exceed it. I > don't see that as a necessary or appropriate limit. Windows and Unix That wasn't how I intended my comment. My point was that the goal of the *PEP* is to make it "as usable" (and actually just in the specific case of #! lines), and that if the additional feature is desirable why not make it available on all platforms? I can see how you read what you did in what I wrote, though. -- R. David Murray http://www.bitdance.com From techtonik at gmail.com Thu Jul 21 14:13:16 2011 From: techtonik at gmail.com (anatoly techtonik) Date: Thu, 21 Jul 2011 15:13:16 +0300 Subject: [Python-Dev] New update to PEP397 - Python launcher for Windows In-Reply-To: <4E27C8BA.8040000@gmail.com> References: <4E27C8BA.8040000@gmail.com> Message-ID: If you're going to include this into standard Python distribution, it needs more attention from _users_. As a user, I can not find any references to any user stories in this PEP article. Abstract chapter is totally useless "This PEP (named 'Python launcher for Windows') describes a Python launcher for the Windows platform." - it is a waste of time to read such stuff. "A Python launcher is a single executable which uses a number of heuristics to locate a Python executable and launch it with a specified command line." - this doesn't answer main questions - Why Launcher is needed, i.e. is there a problem? Can it be solved? How this PEP helps to solve the problem? I see that this launcher is an .msi file. Shouldn't it be portable and don't require administrative privileges? How about KISS? -- anatoly t. On Thu, Jul 21, 2011 at 9:35 AM, Mark Hammond wrote: > I've updated PEP 397 - "Python launcher for Windows" based on recent > discussions and Vinay's implementation. > > http://www.python.org/dev/peps/pep-0397/ and a copy is attached for your > convenience. > > The main changes are: > > * All mentions of the Python reference implementation have been removed now > that a C implementation exists and is a more accurate implementation of the > PEP than the Python version. > > * Some "implementation details" have been removed and links added to the > launcher docs. ?This was done mainly so the implementation is free to make > changes based on user feedback and still stay true to the PEP. Note that the > launcher doc link doesn't exist right now, but should magically appear over > the next couple of days as Vinay pushes some changes I just sent him. > > * The recommendation to install into System32 has been changed to a > recommendation to install directly into \Windows, as the System32 directory > is not on the default path for 32bit processes on a 64bit Windows. ?Vinay > even has a version of an MSI installer which installs into this directory. > ?The PEP also gives installers more leeway on where to install the launcher > if this directory is not writable. > > I think this PEP is getting close to being finalized - please suggest > whatever changes you feel are necessary ASAP as soon I'll be asking for > pronouncement. > > Thanks - especially to Vinay for taking on the C implementation! > > Mark > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/techtonik%40gmail.com > > From brian.curtin at gmail.com Thu Jul 21 14:54:26 2011 From: brian.curtin at gmail.com (Brian Curtin) Date: Thu, 21 Jul 2011 07:54:26 -0500 Subject: [Python-Dev] New update to PEP397 - Python launcher for Windows In-Reply-To: References: <4E27C8BA.8040000@gmail.com> Message-ID: On Jul 21, 2011 7:15 AM, "anatoly techtonik" wrote: > > If you're going to include this into standard Python distribution, it > needs more attention from _users_. As a user, I can not find any > references to any user stories in this PEP article. Abstract chapter > is totally useless > > "This PEP (named 'Python launcher for Windows') describes a Python > launcher for the Windows platform." - it is a waste of time to read > such stuff. > > "A Python launcher is a single executable which uses a number of > heuristics to locate a Python executable and launch it with a > specified command line." - this doesn't answer main questions - Why > Launcher is needed, i.e. is there a problem? Can it be solved? How > this PEP helps to solve the problem? > > > I see that this launcher is an .msi file. Shouldn't it be portable and > don't require administrative privileges? How about KISS? No. As stated in the title, this is for Windows. It also installs to system directories which is likely the need for admin access. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mhammond at skippinet.com.au Thu Jul 21 15:07:12 2011 From: mhammond at skippinet.com.au (Mark Hammond) Date: Thu, 21 Jul 2011 23:07:12 +1000 Subject: [Python-Dev] New update to PEP397 - Python launcher for Windows In-Reply-To: References: <4E27C8BA.8040000@gmail.com> Message-ID: <4E282480.7000700@skippinet.com.au> On 21/07/2011 10:13 PM, anatoly techtonik wrote: > If you're going to include this into standard Python distribution, it > needs more attention from _users_. As a user, I can not find any > references to any user stories in this PEP article. Abstract chapter > is totally useless Could you please be a little more constructive and offer concrete improvements? Mark From pje at telecommunity.com Thu Jul 21 15:20:53 2011 From: pje at telecommunity.com (P.J. Eby) Date: Thu, 21 Jul 2011 09:20:53 -0400 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <4E269EBC.90708@g.nevcal.com> <20110720131415.A2DD43A4100@sparrow.telecommunity.com> <4E27522D.8010300@g.nevcal.com> <20110720230439.6ADE53A409B@sparrow.telecommunity.com> Message-ID: <20110721132153.D18B03A411E@sparrow.telecommunity.com> At 11:52 AM 7/21/2011 +1000, Nick Coghlan wrote: >Trying to change how packages are identified at the Python level makes >PEP 382 sound positively appealing. __path__ needs to stay :) In which case, it should be a list, not a sentinel. ;-) >Even better would be for these (and sys.path) to be list subclasses >that did the right thing under the hood as Glenn suggested. Code that >*replaces* rather than modifies these attributes would still >potentially break virtual packages, but code that modifies them in >place would do the right thing automatically. (Note that all code that >manipulates sys.path and __path__ attributes requires explicit calls >to correctly support current namespace package mechanisms, so this >would actually be an improvement on the status quo rather than making >anything worse). I think the simplest thing, if we're keeping __path__ (and on reflection, I think we should), would be to simply call extend_virtual_paths() automatically on new path entries found in sys.path when an import is performed, relative to the previous value of sys.path. That is, we save an "old" copy of sys.path somewhere, and whenever __import__() is called (well, once it gets past checking if the target is already in sys.modules, anyway), it checks the current sys.path against it, and calls extend_virtual_paths() on any sys.path entries that weren't in the "old" sys.path. This is not the most efficient thing in the world, as it will cause a bunch of stat calls to happen against the new directories, in the middle of a possibly-entirely-unrelated import operation, but it would certainly address the issue in the Simplest Way That Could Possibly Work. A stricter (safer) version of the same thing would be one where we only update __path__ values that are unchanged since we created them, and rather than only appending new entries, we replace the __path__ with a newly-computed one. This version is safer because it avoids corner cases like "I imported foo.bar while foo.baz 1.1 was on my path, then I prepended a directory to sys.path that has foo.baz 1.2, but I still get foo.baz 1.1 when I import." But it loses in cases where people do direct __path__ manipulation. On the other hand, it's a lot easier to say "you break it, you bought it" where __path__ manipulation is concerned, so I'm actually pretty inclined towards using the strict version. Hey... here's a crazy idea. Suppose that a virtual package __path__ is a *tuple* instead of a list? Now, in order to change it, you *have* to replace it. And we can cache the tuple we initially set it to in sys.virtual_package_paths, so we can do an 'is' check before replacing it. Voila: __path__ still exists and is still a sequence for a virtual path, but you have to explicitly replace it if you want to do anything funky -- at which point you're responsible for maintaining it. I'm tempted to say, "well, why not use a list-subclass proxy, then?", but that means more work for no real difference. I just went through dozens of examples of __path__ usage (found via Google), and I found exactly two examples of code that modifies a __path__ that is not: 1. In the __init__.py whose __path__ it is (i.e., code that'll still have a list), or 2. Modifying the __path__ of an explicitly-named self-contained package that's part of the same distribution. The two examples are from Twisted, and Google AppEngine. In the Twisted case, it's some sort of namespace package-like plugin chicanery, and in the AppEngine case, well, I'm not sure what the heck it's doing, but it seems to be making sure that you can still import stuff that has the same name as stdlib stuff, or something. The Twisted case (and an apparent copy of the same code in a project called "flumotion") uses ihooks, though, so I'm not sure it'll even get executed for virtual packages. The Google case loops over everything in sys.modules, in a function by the name of appengine.dist.fix_paths()... but I wasn't able to find out who calls this function, when and why. So, pretty much, except for these bits of "nosy" code, the vast majority of code out there seems to only mess with its own self-contained paths, making the use of tuples seem like a pretty safe choice. (Oh, and all the code I found that reads paths without modifying them only use tuple-safe operations.) So, if we implement automatic __path__ updates for virtual packages, I'm currently leaning towards the strict approach using tuples, but could possibly be persuaded towards read-only list-proxies instead. Side note: it looks like a *lot* of code out there abuses __path__[0] to find data files, so I probably need to add a note to the PEP about not doing that when you convert a self-contained package to a virtual one. Of course, I suppose using a sentinel could address *that* problem, or an iteration-only proxy. The main concern here is that using __path__[0] will *seem* to work when you first use it with a virtual package, because it'll be the right directory. But it'll be wrong long-term. This seems to lean in favor of making a simple reiterable wrapper type for the __path__, that only allows you to take the length and iterate over it. With an appropriate design, it could actually update itself automatically, given a subname and a parent __path__/sys.path. That is, it could keep a tuple copy of the last-seen parent path, and before iteration, compare tuple(self.parent_path) to self.last_seen_path. If they're different, it rebuilds the value to be iterated over. Voila: transparent updating of all virtual __path__ values from sys.path changes (or modifications to self-contained __path__ parents, btw), and trying to change it (or read an item from it positionally) will not create any silent failures. Alright... *if* we support automatic updates to virtual __paths__, this is probably how we should do it. (It will require, though, that imp.find_module be changed to use a different iteration method than PyList_GetItem, as it's quite possible a virtual __path__ will get passed into it.) Also, we *long* ago passed the point where any of this can be sanely backported to Python 2.x with a simple shim, alas. For my purposes at least, needing a full importlib for the implementation is a no-go. :-( Still, for the future of Python, this all makes good sense. I just wish we'd thought of all this in 2006 when the discussion came up before: we maybe could've had this in Python 2.6. Where's that damn time machine when you *really* need it? ;-) From p.f.moore at gmail.com Thu Jul 21 16:43:04 2011 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 21 Jul 2011 15:43:04 +0100 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: <4E27DFC4.1000801@g.nevcal.com> References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> <4E274C0F.40400@g.nevcal.com> <4E276F45.7070105@g.nevcal.com> <4E27741A.1040202@gmail.com> <4E27DFC4.1000801@g.nevcal.com> Message-ID: On 21 July 2011 09:13, Glenn Linderman wrote: > Certainly when the launcher is invoked via an association, this would > be the case.? However, when the launcher is invoked via the command > line, then the unqualified name is passed through.? To be useful from > the command line, the launcher should walk the PATH to find the .py file. It's equally as arguable (and would match my expectations much more closely) that "py a_file.py" should do whatever "python a_file.py" would do. So path search in that context would only be reasonable if it were a Python feature rather than a feature of the launcher. This is what the launcher currently does (so I guess it's not surprising that I'm happy with the current behaviour). I can see the benefits of path search, but I'd want it to be a Python feature (and hence inherited "for free" by the launcher) and not a launcher-only one. Paul. From fuzzyman at voidspace.org.uk Thu Jul 21 17:20:07 2011 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 21 Jul 2011 16:20:07 +0100 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> <4E274C0F.40400@g.nevcal.com> <4E276F45.7070105@g.nevcal.com> <4E27741A.1040202@gmail.com> <4E27DFC4.1000801@g.nevcal.com> Message-ID: <4E2843A7.4090502@voidspace.org.uk> On 21/07/2011 15:43, Paul Moore wrote: > On 21 July 2011 09:13, Glenn Linderman wrote: >> Certainly when the launcher is invoked via an association, this would >> be the case. However, when the launcher is invoked via the command >> line, then the unqualified name is passed through. To be useful from >> the command line, the launcher should walk the PATH to find the .py file. > It's equally as arguable (and would match my expectations much more > closely) that "py a_file.py" should do whatever "python a_file.py" > would do. So path search in that context would only be reasonable if > it were a Python feature rather than a feature of the launcher. > > This is what the launcher currently does (so I guess it's not > surprising that I'm happy with the current behaviour). > > I can see the benefits of path search, but I'd want it to be a Python > feature (and hence inherited "for free" by the launcher) and not a > launcher-only one. What he said ^^. (+1) py launcher and python binaries behaving differently in this regard would be a recipe for confusion and hard to debug problems. Michael > > Paul. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html From brett at python.org Thu Jul 21 20:15:49 2011 From: brett at python.org (Brett Cannon) Date: Thu, 21 Jul 2011 11:15:49 -0700 Subject: [Python-Dev] Issue10271 - warnings.showwarning should allow any callable object - request commiter In-Reply-To: <20110721080552.40e767a1@vimes> References: <20110716103322.330f805c@vimes> <20110721080552.40e767a1@vimes> Message-ID: On Wed, Jul 20, 2011 at 23:05, lekmalek wrote: > On Sun, 17 Jul 2011 19:19:59 -0700 > Brett Cannon wrote: > > > Just so people know, I went ahead and fixed this for 3.3 (but not for > > 3.2 since it changes the API in a subtle way). > Yeah, but that shouldn't break anything. > It won't break any _existing_ code, but it could cause compatibility for _future_ code. Imagine I wrote some code for 3.2.2 where this change was backported and worked *only* with this fix. That would mean my code would fail in any Python 3.2.1 or older interpreter. That's simply not acceptable as that means that code would be magically busted for some versions of Python 3.2 but not all of them. > > Anyway, thanks a lot for your help, I'm happy it's in 3.3. > I'm just sorry it took so long to resolve. Life has been crazy for me in 2011. -Brett > > lekma > > > > > On Sat, Jul 16, 2011 at 01:33, lekmalek wrote: > > > > > Hello all, > > > > > > Can any of you core devs have a look at > > > http://bugs.python.org/issue10271. It seems Brett is really busy > > > right now and this uncontroversial (AFAICT) one liner only needs > > > someone to review it and commit it. The pb is, it's holding me back > > > a little bit, and I really would like to have it in the next 3.2 > > > release if possible. > > > > > > Thanks for your help, > > > > > > lekma > > > _______________________________________________ > > > Python-Dev mailing list > > > Python-Dev at python.org > > > http://mail.python.org/mailman/listinfo/python-dev > > > Unsubscribe: > > > http://mail.python.org/mailman/options/python-dev/brett%40python.org > > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/brett%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Thu Jul 21 20:49:47 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 21 Jul 2011 14:49:47 -0400 Subject: [Python-Dev] Issue10271 - warnings.showwarning should allow any callable object - request commiter In-Reply-To: References: <20110716103322.330f805c@vimes> <20110721080552.40e767a1@vimes> Message-ID: On 7/21/2011 2:15 PM, Brett Cannon wrote: > > It won't break any _existing_ code, but it could cause compatibility for > _future_ code. Imagine I wrote some code for 3.2.2 where this change was > backported and worked *only* with this fix. That would mean my code > would fail in any Python 3.2.1 or older interpreter. That's simply not > acceptable as that means that code would be magically busted for some > versions of Python 3.2 but not all of them. In other words, Python 3.2 is a fixed language, cpython3.2.z's are intended to be increasingly better implementations of that language, although regressions can and do happen (as with issue 12540). -- Terry Jan Reedy From tjreedy at udel.edu Thu Jul 21 21:20:59 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 21 Jul 2011 15:20:59 -0400 Subject: [Python-Dev] New tests in stable versions In-Reply-To: References: <4E26FB19.2070406@netwok.org> <4E270190.1000209@haypocalc.com> Message-ID: On 7/21/2011 2:58 AM, Raymond Hettinger wrote: > I concur with Brett. Nothing good will come from backporting tests that > aren't aimed at a specific bugfix. They could catch reversions that otherwise would not be caught. This would mainly apply to 2.7. It would not be an issue for 3.2 if all fixes are forward ported to 3.3 and tested there (before pushing) where there are tests not in 3.2. If people fix in 3.2, test, commit, and push, and just assume OK in 3.3, the new test will not do any good until someone else runs them with the fix. -- Terry Jan Reedy From tjreedy at udel.edu Thu Jul 21 21:42:44 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 21 Jul 2011 15:42:44 -0400 Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise) In-Reply-To: <4E276AF0.5090409@gmail.com> References: <20110719171647.03f9f9c9@pitrou.net> <4E276AF0.5090409@gmail.com> Message-ID: On 7/20/2011 7:55 PM, Mark Hammond wrote: > On 21/07/2011 4:38 AM, Terry Reedy wrote: > >> Many installers first make an organization directory and then an app >> directory within that. This annoys me sometimes when they only have one >> app to ever install, but is useful when there might really be multiple >> directories, as in our case. (Ditto for start menu entries.) This is >> what python should have done a decade ago. > I disagree. If we followed that advice we would also be in "\Program > Files". That is not what I suggested. I said let the use pick. > I have no problem with multiple Python versions installed > directly off the root, especially given most users probably have a very > small number of such installations. I think Python being a developer > tool rather than a user app is a reasonable justification for that (and > the justification used when the existing scheme was decided) I put the multiple installations and several other directories into /programs. On my next machine, on order, I will use /devel. > > The two proposals >> overlap but are not mutually exclusive. For future pythons, 'python33' >> is easier to remember and type than 'py -v 3.3' or whatever the proposed >> encantation is. > > 'py -3.3' - less chars to type than 'python33' and with no need to have > every Python directory on your PATH. My proposal, as I clearly said, was for EXACTLY ONE directory to be added to PATH. In spite of Microsoft making is damned difficult for users to edit PATH, (and deleted programs not deleting their entries) I added 'C:/programs;'. I copied python32/python as py32 and python27/python as py27. Those are even fewer characters to type (4 versus 7). Now I can click a 'Command Prompt' icon and enter 'py32 -m test.regrtest' and it works without cd-ing to /programs/python32. Of course, I will have to re-copy with every install, which is why I would like something like this as part of installs. > IMO it is also simple enough that > people will remember it fairly easily. py32 is even easier to remember. > Also, the launcher supports the ability to select either the 32 or 64bit > implementation - so maybe 'python33.exe' isn't really good enough and > should reflect the bittedness? Like py32-6? If I install both Pythons on my new 64 bit machine, I will think about it, though I have no need for both now. >> A python directory also gives a sensible (though optional) place to put >> other interpreters and even python-based apps. The launcher does not. > > What other interpreters? IMO it doesn't make sense to have IronPython, > jython etc be installed there. Ditto for apps - especially given most > apps tend to be tied to a subset of all possible Python versions. If I install pypy, /programs is exactly where I would put it until I somehow discovered that to be a problem. Its startup could be copied as pp26 or something. My idea may be not so good for general use, even though is now solves my problems, but please criticize what I said, allowing for obvious modifications like py32 instead of python32, and not a strawman that is wildly different. -- Terry Jan Reedy From v+python at g.nevcal.com Thu Jul 21 22:05:45 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Thu, 21 Jul 2011 13:05:45 -0700 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: <4E2843A7.4090502@voidspace.org.uk> References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> <4E274C0F.40400@g.nevcal.com> <4E276F45.7070105@g.nevcal.com> <4E27741A.1040202@gmail.com> <4E27DFC4.1000801@g.nevcal.com> <4E2843A7.4090502@voidspace.org.uk> Message-ID: <4E288699.3060609@g.nevcal.com> On 7/21/2011 8:20 AM, Michael Foord wrote: > On 21/07/2011 15:43, Paul Moore wrote: >> On 21 July 2011 09:13, Glenn Linderman wrote: >>> Certainly when the launcher is invoked via an association, this would >>> be the case. However, when the launcher is invoked via the command >>> line, then the unqualified name is passed through. To be useful from >>> the command line, the launcher should walk the PATH to find the .py >>> file. >> It's equally as arguable (and would match my expectations much more >> closely) that "py a_file.py" should do whatever "python a_file.py" >> would do. So path search in that context would only be reasonable if >> it were a Python feature rather than a feature of the launcher. >> >> This is what the launcher currently does (so I guess it's not >> surprising that I'm happy with the current behaviour). >> >> I can see the benefits of path search, but I'd want it to be a Python >> feature (and hence inherited "for free" by the launcher) and not a >> launcher-only one. > > What he said ^^. (+1) > > py launcher and python binaries behaving differently in this regard > would be a recipe for confusion and hard to debug problems. I see the point. Although the incremental benefit is higher to Windows users, and although we are creating a Windows-only piece of code that could be the vehicle for adding the functionality, it would be beneficial for all platforms, and a common implementation would serve that need better. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pje at telecommunity.com Thu Jul 21 23:12:41 2011 From: pje at telecommunity.com (P.J. Eby) Date: Thu, 21 Jul 2011 17:12:41 -0400 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <4E288527.8060202@gmail.com> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <4E269EBC.90708@g.nevcal.com> <20110720131415.A2DD43A4100@sparrow.telecommunity.com> <4E27522D.8010300@g.nevcal.com> <20110720230439.6ADE53A409B@sparrow.telecommunity.com> <20110721132153.D18B03A411E@sparrow.telecommunity.com> <4E288527.8060202@gmail.com> Message-ID: <20110721211336.7C1393A40AA@sparrow.telecommunity.com> At 12:59 PM 7/21/2011 -0700, Reliable Domains wrote: >I assume that the implicit extend_virtual_paths() would be smart >enough to only do real work if there are virtual packages to do it >in, so much of the performance costs (bunch of stats) are bounded by >the existence of and number of virtual packages that have actually >been imported, correct? Yes - this is true even for an explicit call. It only does this for imported virtual packages, and child virtual packages are only checked for if the parent package exists. So, in the case of a directory being added that has no parent packages, then the cost in stats is equal to the number of top-level, *imported* virtual packages. The __path__ wrapper scheme can do this even better, and defer doing any of the stat calls until/unless another import occurs for one of those packages. So if you munge sys.path and then don't import anything from a virtual package, no extra stat calls would happen at all. From ncoghlan at gmail.com Fri Jul 22 00:06:30 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 22 Jul 2011 08:06:30 +1000 Subject: [Python-Dev] New tests in stable versions In-Reply-To: References: <4E26FB19.2070406@netwok.org> <4E270190.1000209@haypocalc.com> Message-ID: On Fri, Jul 22, 2011 at 5:20 AM, Terry Reedy wrote: > On 7/21/2011 2:58 AM, Raymond Hettinger wrote: > >> I concur with Brett. Nothing good will come from backporting tests that >> aren't aimed at a specific bugfix. > > They could catch reversions that otherwise would not be caught. This would > mainly apply to 2.7. It would not be an issue for 3.2 if all fixes are > forward ported to 3.3 and tested there (before pushing) where there are > tests not in 3.2. If people fix in 3.2, test, commit, and push, and just > assume OK in 3.3, the new test will not do any good until someone else runs > them with the fix. None of that contradicts what Raymond and Brett said. Backporting test improvements that aren't targeting specific known bugs does not make efficient use of limited development resources. Forward porting of any changes made to maintenance branches (or explicitly blocking same as being irrelevant), OTOH, is mandatory. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Fri Jul 22 00:17:24 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 22 Jul 2011 08:17:24 +1000 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: <4E288699.3060609@g.nevcal.com> References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> <4E274C0F.40400@g.nevcal.com> <4E276F45.7070105@g.nevcal.com> <4E27741A.1040202@gmail.com> <4E27DFC4.1000801@g.nevcal.com> <4E2843A7.4090502@voidspace.org.uk> <4E288699.3060609@g.nevcal.com> Message-ID: On Fri, Jul 22, 2011 at 6:05 AM, Glenn Linderman wrote: > On 7/21/2011 8:20 AM, Michael Foord wrote: >> py launcher and python binaries behaving differently in this regard would be >> a recipe for confusion and hard to debug problems. > > I see the point.? Although the incremental benefit is higher to Windows > users, and although we are creating a Windows-only piece of code that could > be the vehicle for adding the functionality, it would be beneficial for all > platforms, and a common implementation would serve that need better. Well that, and the desire to have the Windows launcher *just* find an interpreter to run, so that "py -3.2 " and "c:\python32\python " handle the arguments the same way. While further discussion of the PATH walking concept should be done in a new thread on python-ideas, I'll note that I'm no longer actively opposed to the idea. However, I still think it needs its own PEP to work out the details (and whether or not it happens at all). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Fri Jul 22 00:35:15 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 22 Jul 2011 08:35:15 +1000 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <20110721132153.D18B03A411E@sparrow.telecommunity.com> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <4E269EBC.90708@g.nevcal.com> <20110720131415.A2DD43A4100@sparrow.telecommunity.com> <4E27522D.8010300@g.nevcal.com> <20110720230439.6ADE53A409B@sparrow.telecommunity.com> <20110721132153.D18B03A411E@sparrow.telecommunity.com> Message-ID: On Thu, Jul 21, 2011 at 11:20 PM, P.J. Eby wrote: > This seems to lean in favor of making a simple reiterable wrapper type for > the __path__, that only allows you to take the length and iterate over it. > ?With an appropriate design, it could actually update itself automatically, > given a subname and a parent __path__/sys.path. ?That is, it could keep a > tuple copy of the last-seen parent path, and before iteration, compare > tuple(self.parent_path) to self.last_seen_path. ?If they're different, it > rebuilds the value to be iterated over. > > Voila: transparent updating of all virtual __path__ values from sys.path > changes (or modifications to self-contained __path__ parents, btw), and > trying to change it (or read an item from it positionally) will not create > any silent failures. > > Alright... ?*if* we support automatic updates to virtual __paths__, this is > probably how we should do it. ?(It will require, though, that > imp.find_module be changed to use a different iteration method than > PyList_GetItem, as it's quite possible a virtual __path__ will get passed > into it.) A no-indexing tuple wrapper for virtual package __path__ values that automatically updates itself in response to parent path modifications sounds good to me (errors shall not pass silently, etc). This also allows virtual packages to be indicated clearly just through the type of their __path__ attribute rather than having to look them up in the import state. I still like the idea of keeping sys.virtual_packages as a dict mapping to the path values, though - it makes it easier to debug erroneous __path__ replacement in virtual packages by checking "pkg.__path__ is sys.virtual_package_paths[pkg.__name__]" Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From raymond.hettinger at gmail.com Fri Jul 22 00:37:01 2011 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Thu, 21 Jul 2011 15:37:01 -0700 Subject: [Python-Dev] [Python-checkins] devguide: Add a communications section to the devguide FAQ (closes #11690) In-Reply-To: References: <4E27EE68.4000801@gmail.com> Message-ID: <44F06A8A-8153-443C-A4FB-0E57ABCBFDA6@gmail.com> Please don't add the IRC link to the devguide. Based on conversations with Guido, he is against it being part of the core development process. Raymond On Jul 21, 2011, at 4:08 AM, Nick Coghlan wrote: > On Thu, Jul 21, 2011 at 7:16 PM, Ezio Melotti wrote: >> >> FWIW you can make #python a link using either irc://chat.freenode.net/python >> (this will open the default IRC app, and I think Firefox even ask if you >> want to use a webchat) or http://webchat.freenode.net/?channels=python (for >> the freenode webchat). If you are going to do it, it might be worth >> mentioning that the channel requires registration. >> >> I agree with Victor that the network (Freenode or irc.freenode.net) should >> be specified. >> Also the #python-dev channel should be mentioned (it doesn't require >> registration, so links are fine here). > > If someone more knowledgeable on IRC matters than me could either > commit a fix directly to the devguide repo or else put a patch on the > tracker, that would be great. > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins From solipsis at pitrou.net Fri Jul 22 00:45:20 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 22 Jul 2011 00:45:20 +0200 Subject: [Python-Dev] devguide: Add a communications section to the devguide FAQ (closes #11690) References: <4E27EE68.4000801@gmail.com> <44F06A8A-8153-443C-A4FB-0E57ABCBFDA6@gmail.com> Message-ID: <20110722004520.1f430e6d@pitrou.net> On Thu, 21 Jul 2011 15:37:01 -0700 Raymond Hettinger wrote: > Please don't add the IRC link to the devguide. > > Based on conversations with Guido, he is against it being part of the core > development process. IRC is very much outside of the core development process, but it's still a reasonable place to ask help in. Regards Antoine. From riscutiavlad at gmail.com Fri Jul 22 00:54:30 2011 From: riscutiavlad at gmail.com (Vlad Riscutia) Date: Thu, 21 Jul 2011 15:54:30 -0700 Subject: [Python-Dev] ctypes: Configurable bitfield allocation strategy In-Reply-To: References: <4E1AAA75.7010903@v.loewis.de> Message-ID: Anyone care to take a look at this? Thank you, Vlad On Mon, Jul 11, 2011 at 8:59 AM, Vlad Riscutia wrote: > Actually I already attached patch implementing everything to the issue (not > too much time spent :)). I'm hoping someone can review. > > Thank you, > Vlad > > > On Mon, Jul 11, 2011 at 12:47 AM, "Martin v. L?wis" wrote: > >> Am 25.06.2011 18:33, schrieb Vlad Riscutia: >> > I would like to hear some opinions on this. >> >> Sounds fine to me in principle. Warnings can be added to the >> documentation at any time; please submit a patch to the tracker. >> As for the API change - make sure you post your proposed API >> to the list before spending time to implement it. >> >> Regards, >> Martin >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From v+python at g.nevcal.com Fri Jul 22 01:02:46 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Thu, 21 Jul 2011 16:02:46 -0700 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: <4E276EBF.6000509@gmail.com> References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> <4E274C0F.40400@g.nevcal.com> <4E276795.9050404@gmail.com> <4E276C7C.1020605@g.nevcal.com> <4E276EBF.6000509@gmail.com> Message-ID: <4E28B016.5050500@g.nevcal.com> On 7/20/2011 5:11 PM, Mark Hammond wrote: > It may be that your copy of the launcher is a little old - some > changes were pushed just yesterday (but I'm not sure if Vinay made a > new installer yet). It has slightly better debug output (although > generally not what you are asking for yet) and better > "cross-bittedness" support. Installed new version: msiexec /i launchwin.amd64.msi ALLUSERS=1 Expected behaviors of registry changes occurred. Still launches python 2, though, whereas c:\windows\py.ini contains: [defaults] python=3 [commands] /usr/bin/perl=C:\perl\bin\perl.exe Here is the debug output. Seems like it isn't recognizing the python=3, even the new version. d:\path\to\data>>set PYLAUNCH_DEBUG=1 set PYLAUNCH_DEBUG=1 d:\path\to\data>foo.py --pyver --clean foo.py --pyver --clean launcher build: 64bit launcher executable: Console File 'C:\Users\Glenn\AppData\Roaming\py.ini' non-existent Using global configuration file 'C:\Windows\py.ini' maybe_handle_shebang: read 256 bytes maybe_handle_shebang: BOM not found, using UTF-8 locating Pythons in 32bit registry locate_pythons_for_key: unable to open PythonCore key in HKCU locate_pythons_for_key: C:\Python26\python.exe is a 32bit executable locate_pythons_for_key: C:\Python26\PCBuild\python.exe: The system cannot find the path specified. locate_pythons_for_key: C:\Python26\PCBuild\amd64\python.exe: The system cannot find the path specified. locate_pythons_for_key: C:\Python31\python.exe is a 32bit executable locate_pythons_for_key: C:\Python31\PCBuild\python.exe: The system cannot find the path specified. locate_pythons_for_key: C:\Python31\PCBuild\amd64\python.exe: The system cannot find the path specified. locating Pythons in native registry locate_pythons_for_key: unable to open PythonCore key in HKCU locate_pythons_for_key: C:\Python32\python.exe is a 64bit executable locate_pythons_for_key: C:\Python32\PCBuild\python.exe: The system cannot find the path specified. locate_pythons_for_key: C:\Python32\PCBuild\amd64\python.exe: The system cannot find the path specified. found no configured value for 'python' search for default Python found version 2.6 at 'C:\Python26\python.exe' run_child: about to run 'C:\Python26\python.exe "D:\my\py\foo.py" --pyver --clean' File "D:\my\py\foo.py", line 469 SyntaxError: Non-ASCII character '\xc3' in file D:\my\py\foo.py on line 470, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details child process exit code: 1 d:\path\to\data> So, looking at the code, get_configured_value produces that message, but there are 3 places to look for "python". Setting the environment variable makes it work. Eliminating the environment variable, I then copied c:\Windows\py.ini to c:\users\glenn\appdata\roaming\py.ini. That worked. So I guess the syntax of my py.ini file is correct. But apparently it isn't properly using c:\windows\py.ini !! Yet curiously, it reports the correct name for the global configuration file. Aha! Bad logic is get_configured_value! get_configured_value only looks in the global configuration file if there is a local configuration file that doesn't have the setting. It should look in the global configuration file if there is no local configuration file _OR_ the is a local configuration file without the setting. I'll await a new launcher. -------------- next part -------------- An HTML attachment was scrubbed... URL: From v+python at g.nevcal.com Fri Jul 22 01:19:17 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Thu, 21 Jul 2011 16:19:17 -0700 Subject: [Python-Dev] New update to PEP397 - Python launcher for Windows In-Reply-To: <4E27C8BA.8040000@gmail.com> References: <4E27C8BA.8040000@gmail.com> Message-ID: <4E28B3F5.8080802@g.nevcal.com> On 7/20/2011 11:35 PM, Mark Hammond wrote: > Customized Commands: > > The launcher will support the ability to define "Customized Commands" in a > Windows .ini file (ie, a file which can be parsed by the Windows function > GetPrivateProfileString). A section called '[commands]' can be created > with key names defining the virtual command and the value specifying the > actual command-line to be used for this virtual command. > > For example, if an INI file has the contents: > > [commands] > vpython=c:\bin\vpython.exe -foo > > Then a shebang line of '#! vpython' in a script named 'doit.py' will > result in the launcher using the command-line 'c:\bin\vpython.exe -foo > doit.py' I experimented, and empirically determined, that a customized command can be of the form [commands] /usr/bin/perl=C:\perl\bin\perl.exe and that this works to launch Unix perl scripts on Windows (successfully, if the perl script is actually portable). This does not contradict the above description, but may be surprising to some. I think it is a good thing. Of course, the extra handling of versions, and locating of corresponding installed versions of things applies only to Python, but some may find uses for launching non-Python programs. This also would work for python programs using non-CPython implementations that may not set the registry in the same way. However, because virtual commands take precedence over Customized Commands, there is no way to override even a specific virtual command to point at a Python other than a CPython. (perhaps some serious registry hacking could make a non-CPython masquerade in the registry as a version of CPython.) -------------- next part -------------- An HTML attachment was scrubbed... URL: From v+python at g.nevcal.com Fri Jul 22 01:12:45 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Thu, 21 Jul 2011 16:12:45 -0700 Subject: [Python-Dev] New update to PEP397 - Python launcher for Windows In-Reply-To: <4E27C8BA.8040000@gmail.com> References: <4E27C8BA.8040000@gmail.com> Message-ID: <4E28B26D.7070902@g.nevcal.com> On 7/20/2011 11:35 PM, Mark Hammond wrote: > Virtual commands in shebang lines: > > Virtual Commands are shebang lines which start with strings which would > be expected to work on Unix platforms - examples include > '/usr/bin/python', '/usr/bin/env python' and 'python'. Optionally, the > virtual command may be suffixed with a version qualifier (see below), > such as '/usr/bin/python2' or '/usr/bin/python3.2'. The command executed > is based on the rules described in Python Version Qualifiers below. I note in passing that '/usr/bin/env python' is hard-coded in the launcher, which conforms to the above documentation. But there is no hard requirement in Unix, if I correctly understand it, that '/usr/bin/env' be separated from 'python' (or whatever) by exactly one space. While I doubt it is frequently used with other than a single space, I think it would be legal to have 2 or 3 or 10 spaces, and maybe even tabs or a mixture, and it would work on Unix... but not in the launcher. It would somewhat complicate the launcher code to have an additional case to check for /usr/bin/env, skip following white space, and then compare to python, but it would be more robust. If it is thought that hard-coding a single space covers most of the uses, it should at least be emphasized in the documentation that only commands of than nature containing a single space will work with the launcher. -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Fri Jul 22 01:35:31 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 22 Jul 2011 01:35:31 +0200 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" References: <20110720040505.400E23A4116@sparrow.telecommunity.com> Message-ID: <20110722013531.4fa9ac30@pitrou.net> On Tue, 19 Jul 2011 23:58:55 -0400 "P.J. Eby" wrote: > > Anyway, to make a long story short, we came up with an alternative > implementation plan that actually solves some other problems besides > the one that PEP 382 sets out to solve, and whose implementation a > bit is easier to explain. (In fact, for users coming from various > other languages, it hardly needs any explanation at all.) I have a question. If I have (on sys.path) a module "x.py" containing, say: y = 5 and (also on sys.path), a directory "x" containing a "y.py" module. What is "from x import y" supposed to do? (currently, it would bind "y" to its value in x.py) Regards Antoine. From merwok at netwok.org Fri Jul 22 01:37:05 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Fri, 22 Jul 2011 01:37:05 +0200 Subject: [Python-Dev] New tests in stable versions In-Reply-To: References: <4E26FB19.2070406@netwok.org> <4E270190.1000209@haypocalc.com> Message-ID: <4E28B821.3000800@netwok.org> Le 21/07/2011 01:54, Nick Coghlan a ?crit : > [...] > So slightly more relaxed than Brett's view: > - definitely apply bug fixes and their tests to affected maintenance branches > - *optionally* apply coverage enhancements to maintenance branches, > but don't feel obliged to do so > > I see it as a productivity trade-off - time spent improving coverage > on multiple branches is time not spent on other things. [...] Thanks all! Nick?s viewpoint and other people?s more-or-less according replies are clear and make sense. Cheers From merwok at netwok.org Fri Jul 22 01:42:55 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Fri, 22 Jul 2011 01:42:55 +0200 Subject: [Python-Dev] Indentation in reStructuredText documents In-Reply-To: <87livss9li.fsf_-_@benfinney.id.au> References: <4E26E468.20800@netwok.org> <87livss9li.fsf_-_@benfinney.id.au> Message-ID: <4E28B97F.7050106@netwok.org> Le 21/07/2011 03:31, Ben Finney a ?crit : > ?ric Araujo writes: >> FYI, reST uses three-space indents, not four (so that blocks align >> nicely under the leading two dots + one space), so I think the change >> was intentional. > > No, reST doesn't specify any particular level of indentation. My phrasing was a shortcut for ?the reST files in our documentation?, as was made clear by followups. Regards From ncoghlan at gmail.com Fri Jul 22 01:53:57 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 22 Jul 2011 09:53:57 +1000 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <20110722013531.4fa9ac30@pitrou.net> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <20110722013531.4fa9ac30@pitrou.net> Message-ID: On Fri, Jul 22, 2011 at 9:35 AM, Antoine Pitrou wrote: > On Tue, 19 Jul 2011 23:58:55 -0400 > "P.J. Eby" wrote: >> >> Anyway, to make a long story short, we came up with an alternative >> implementation plan that actually solves some other problems besides >> the one that PEP 382 sets out to solve, and whose implementation a >> bit is easier to explain. ?(In fact, for users coming from various >> other languages, it hardly needs any explanation at all.) > > I have a question. > > If I have (on sys.path) a module "x.py" containing, say: > > ? ?y = 5 > > and (also on sys.path), a directory "x" containing a "y.py" module. > > What is "from x import y" supposed to do? > > (currently, it would bind "y" to its value in x.py) It would behave the same as it does today: the imported value of 'y' would be 5. Virtual packages only kick in if an import would otherwise fail. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From solipsis at pitrou.net Fri Jul 22 02:00:07 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 22 Jul 2011 02:00:07 +0200 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <20110722013531.4fa9ac30@pitrou.net> Message-ID: <1311292807.3039.4.camel@localhost.localdomain> Le vendredi 22 juillet 2011 ? 09:53 +1000, Nick Coghlan a ?crit : > On Fri, Jul 22, 2011 at 9:35 AM, Antoine Pitrou wrote: > > On Tue, 19 Jul 2011 23:58:55 -0400 > > "P.J. Eby" wrote: > >> > >> Anyway, to make a long story short, we came up with an alternative > >> implementation plan that actually solves some other problems besides > >> the one that PEP 382 sets out to solve, and whose implementation a > >> bit is easier to explain. (In fact, for users coming from various > >> other languages, it hardly needs any explanation at all.) > > > > I have a question. > > > > If I have (on sys.path) a module "x.py" containing, say: > > > > y = 5 > > > > and (also on sys.path), a directory "x" containing a "y.py" module. > > > > What is "from x import y" supposed to do? > > > > (currently, it would bind "y" to its value in x.py) > > It would behave the same as it does today: the imported value of 'y' would be 5. > > Virtual packages only kick in if an import would otherwise fail. Wouldn't it produce confusing situations like the above example? Regards Antoine. From v+python at g.nevcal.com Fri Jul 22 02:31:04 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Thu, 21 Jul 2011 17:31:04 -0700 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <1311292807.3039.4.camel@localhost.localdomain> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <20110722013531.4fa9ac30@pitrou.net> <1311292807.3039.4.camel@localhost.localdomain> Message-ID: <4E28C4C8.5060104@g.nevcal.com> On 7/21/2011 5:00 PM, Antoine Pitrou wrote: > Le vendredi 22 juillet 2011 ? 09:53 +1000, Nick Coghlan a ?crit : >> On Fri, Jul 22, 2011 at 9:35 AM, Antoine Pitrou wrote: >>> On Tue, 19 Jul 2011 23:58:55 -0400 >>> "P.J. Eby" wrote: >>>> Anyway, to make a long story short, we came up with an alternative >>>> implementation plan that actually solves some other problems besides >>>> the one that PEP 382 sets out to solve, and whose implementation a >>>> bit is easier to explain. (In fact, for users coming from various >>>> other languages, it hardly needs any explanation at all.) >>> I have a question. >>> >>> If I have (on sys.path) a module "x.py" containing, say: >>> >>> y = 5 >>> >>> and (also on sys.path), a directory "x" containing a "y.py" module. >>> >>> What is "from x import y" supposed to do? >>> >>> (currently, it would bind "y" to its value in x.py) >> It would behave the same as it does today: the imported value of 'y' would be 5. >> >> Virtual packages only kick in if an import would otherwise fail. > Wouldn't it produce confusing situations like the above example? > > Regards > > Antoine. If I have (on sys.path), a directory "x" containing a "y.py" module, and later (on sys.path), another directory "x" containing a "y.py" module, what is "from x import y" supposed to do? OR If I have (on sys.path), a module "x.py" containing, say: y = 5 and later (on sys.path), another module "x.py" containing, say: y = 6 what is "from x import y" supposed to do? I guess I don't see how this new proposal makes anything more confusing than it already is? -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Fri Jul 22 02:38:05 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 22 Jul 2011 02:38:05 +0200 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <20110722013531.4fa9ac30@pitrou.net> <1311292807.3039.4.camel@localhost.localdomain> <4E28C4C8.5060104@g.nevcal.com> Message-ID: <20110722023805.623f4a7f@pitrou.net> On Thu, 21 Jul 2011 17:31:04 -0700 Glenn Linderman wrote: > > If I have (on sys.path), a directory "x" containing a "y.py" module, and > later (on sys.path), another directory "x" containing a "y.py" module, > what is "from x import y" supposed to do? > > OR > > If I have (on sys.path), a module "x.py" containing, say: > > y = 5 > > and later (on sys.path), another module "x.py" containing, say: > > y = 6 > > what is "from x import y" supposed to do? > > > I guess I don't see how this new proposal makes anything more confusing > than it already is? It does. In your two examples, the "x.py" files (or the "x" directories) live in two different base directories; imports are then resolved in sys.path order, which is expected and intuitive. However, you can have a "x.py" file and a "x" directory *in the same base directory which is present in sys.path*, meaning sys.path can't help disambiguate in this case. Regards Antoine. From mhammond at skippinet.com.au Fri Jul 22 02:44:01 2011 From: mhammond at skippinet.com.au (Mark Hammond) Date: Fri, 22 Jul 2011 10:44:01 +1000 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: <4E28B016.5050500@g.nevcal.com> References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> <4E274C0F.40400@g.nevcal.com> <4E276795.9050404@gmail.com> <4E276C7C.1020605@g.nevcal.com> <4E276EBF.6000509@gmail.com> <4E28B016.5050500@g.nevcal.com> Message-ID: <4E28C7D1.3010002@skippinet.com.au> On 22/07/2011 9:02 AM, Glenn Linderman wrote: > Bad logic is get_configured_value! get_configured_value only looks in > the global configuration file if there is a local configuration file > that doesn't have the setting. It should look in the global > configuration file if there is no local configuration file _OR_ the is a > local configuration file without the setting. > > I'll await a new launcher. I just pushed a fix and hopefully Vinay will push a new MSI soon. Thanks, Mark From v+python at g.nevcal.com Fri Jul 22 02:53:09 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Thu, 21 Jul 2011 17:53:09 -0700 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <20110722023805.623f4a7f@pitrou.net> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <20110722013531.4fa9ac30@pitrou.net> <1311292807.3039.4.camel@localhost.localdomain> <4E28C4C8.5060104@g.nevcal.com> <20110722023805.623f4a7f@pitrou.net> Message-ID: <4E28C9F5.3040904@g.nevcal.com> On 7/21/2011 5:38 PM, Antoine Pitrou wrote: > However, you can have a "x.py" file and a "x" directory *in the same > base directory which is present in sys.path*, meaning sys.path can't > help disambiguate in this case. Ah yes. It means there has to be one more rule for disambiguation, which Nick supplied. Your case wasn't clear to me from your first description, however. As long as there is an ordering, and it is documented, it is not particularly confusing, however. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Fri Jul 22 02:58:20 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 22 Jul 2011 10:58:20 +1000 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <1311292807.3039.4.camel@localhost.localdomain> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <20110722013531.4fa9ac30@pitrou.net> <1311292807.3039.4.camel@localhost.localdomain> Message-ID: On Fri, Jul 22, 2011 at 10:00 AM, Antoine Pitrou wrote: > Wouldn't it produce confusing situations like the above example? I don't see how it is any more confusing than any other form of module shadowing. For backwards compatibility reasons, the precedence model will be: 1. Modules and self-contained packages that can satisfy the import request are checked for first (along the whole length of sys.path). 2. If that fails, the virtual package mechanism is checked PEP 402 eliminates some cases of package shadowing by making __init__.py files optional, so your scenario will actually *work*, so long as the submodule name doesn't conflict with a module attribute. *Today* if you have: x.py x.pyd x.so x/__init__.py in the same sys.path directory, x.py wins (search order is controlled by the internal order of checks within the import system - and source files are first on that list). With PEP 302, x.py still wins, but the submodules within the x directory become accessible so long as they don't conflict with *actual* attributes set in the x module. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From riscutiavlad at gmail.com Fri Jul 22 03:03:22 2011 From: riscutiavlad at gmail.com (Vlad Riscutia) Date: Thu, 21 Jul 2011 18:03:22 -0700 Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise) In-Reply-To: References: <20110719171647.03f9f9c9@pitrou.net> <4E276AF0.5090409@gmail.com> Message-ID: I'm kind of -1 on changing Python executable name. It would make sense for different major versions, where there are known incompatibilities, so python2-python3 would make sense but python31 python32 not that much... If my team is using Python and it gets pre-installed with other dev-tools, do I need to let everyone know they must call python*31*? And if we upgrade, make sure everyone knows they should now call python*32*? What if we have scripts that call python? Make sure we update all of them whenever minor version is changed? The way I look at it, most people have only one version of Python installed at one time and it's just extra burden to make them remember major+minor version number they have. If you actually install multiple versions, you do that for a reason and, since you know what you're doing, you would rather remember to pass correct -v argument to py than users who *just want to use Python*. Thank you, Vlad On Thu, Jul 21, 2011 at 12:42 PM, Terry Reedy wrote: > On 7/20/2011 7:55 PM, Mark Hammond wrote: > >> On 21/07/2011 4:38 AM, Terry Reedy wrote: >> >> Many installers first make an organization directory and then an app >>> directory within that. This annoys me sometimes when they only have one >>> app to ever install, but is useful when there might really be multiple >>> directories, as in our case. (Ditto for start menu entries.) This is >>> what python should have done a decade ago. >>> >> > I disagree. If we followed that advice we would also be in "\Program >> Files". >> > > That is not what I suggested. I said let the use pick. > > > I have no problem with multiple Python versions installed >> directly off the root, especially given most users probably have a very >> small number of such installations. I think Python being a developer >> tool rather than a user app is a reasonable justification for that (and >> the justification used when the existing scheme was decided) >> > > I put the multiple installations and several other directories into > /programs. On my next machine, on order, I will use /devel. > > > > The two proposals >> >>> overlap but are not mutually exclusive. For future pythons, 'python33' >>> is easier to remember and type than 'py -v 3.3' or whatever the proposed >>> encantation is. >>> >> >> 'py -3.3' - less chars to type than 'python33' and with no need to have >> every Python directory on your PATH. >> > > My proposal, as I clearly said, was for EXACTLY ONE directory to be added > to PATH. In spite of Microsoft making is damned difficult for users to edit > PATH, (and deleted programs not deleting their entries) I added > 'C:/programs;'. I copied python32/python as py32 and python27/python as > py27. Those are even fewer characters to type (4 versus 7). Now I can click > a 'Command Prompt' icon and enter 'py32 -m test.regrtest' and it works > without cd-ing to /programs/python32. Of course, I will have to re-copy with > every install, which is why I would like something like this as part of > installs. > > > > IMO it is also simple enough that >> people will remember it fairly easily. >> > > py32 is even easier to remember. > > > Also, the launcher supports the ability to select either the 32 or 64bit >> implementation - so maybe 'python33.exe' isn't really good enough and >> should reflect the bittedness? >> > > Like py32-6? If I install both Pythons on my new 64 bit machine, I will > think about it, though I have no need for both now. > > > A python directory also gives a sensible (though optional) place to put >>> other interpreters and even python-based apps. The launcher does not. >>> >> >> What other interpreters? IMO it doesn't make sense to have IronPython, >> jython etc be installed there. Ditto for apps - especially given most >> apps tend to be tied to a subset of all possible Python versions. >> > > If I install pypy, /programs is exactly where I would put it until I > somehow discovered that to be a problem. Its startup could be copied as pp26 > or something. > > > My idea may be not so good for general use, even though is now solves my > problems, but please criticize what I said, allowing for obvious > modifications like py32 instead of python32, and not a strawman that is > wildly different. > > -- > Terry Jan Reedy > > > ______________________________**_________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/**mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/**mailman/options/python-dev/** > riscutiavlad%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Fri Jul 22 03:03:26 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 22 Jul 2011 11:03:26 +1000 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <4E28C9F5.3040904@g.nevcal.com> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <20110722013531.4fa9ac30@pitrou.net> <1311292807.3039.4.camel@localhost.localdomain> <4E28C4C8.5060104@g.nevcal.com> <20110722023805.623f4a7f@pitrou.net> <4E28C9F5.3040904@g.nevcal.com> Message-ID: On Fri, Jul 22, 2011 at 10:53 AM, Glenn Linderman wrote: > Ah yes.? It means there has to be one more rule for disambiguation, which > Nick supplied.? Your case wasn't clear to me from your first description, > however.? As long as there is an ordering, and it is documented, it is not > particularly confusing, however. The genuinely confusing part is that x.py still takes precedence, even if it appears on sys.path *after* x/y.py. However, we're forced into that behaviour by backwards compatibility requirements. The alternative of allowing x/y.py to take precedence has been rejected on those grounds more than once. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From solipsis at pitrou.net Fri Jul 22 03:04:49 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 22 Jul 2011 03:04:49 +0200 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <20110722013531.4fa9ac30@pitrou.net> <1311292807.3039.4.camel@localhost.localdomain> Message-ID: <1311296689.3039.7.camel@localhost.localdomain> Le vendredi 22 juillet 2011 ? 10:58 +1000, Nick Coghlan a ?crit : > On Fri, Jul 22, 2011 at 10:00 AM, Antoine Pitrou wrote: > > Wouldn't it produce confusing situations like the above example? > > I don't see how it is any more confusing than any other form of module > shadowing. The additional confusion lies in the fact that a module can be shadowed by something which is not a module (a mere global variable). I find it rather baffling. Regards Antoine. From merwok at netwok.org Fri Jul 22 03:07:31 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Fri, 22 Jul 2011 03:07:31 +0200 Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise) In-Reply-To: References: <20110719171647.03f9f9c9@pitrou.net> <4E276AF0.5090409@gmail.com> Message-ID: <4E28CD53.1090707@netwok.org> Hi, Le 22/07/2011 03:03, Vlad Riscutia a ?crit : > I'm kind of -1 on changing Python executable name. It would make sense for > different major versions, where there are known incompatibilities, so > python2-python3 would make sense but python31 python32 not that much... > > If my team is using Python and it gets pre-installed with other dev-tools, > do I need to let everyone know they must call python*31*? And if we upgrade, > make sure everyone knows they should now call python*32*? What if we have > scripts that call python? Make sure we update all of them whenever minor > version is changed? If I understand correctly, adding versioned filenames like python3.3.exe would happen in addition of python.exe, not in replacement. Regards From riscutiavlad at gmail.com Fri Jul 22 03:30:21 2011 From: riscutiavlad at gmail.com (Vlad Riscutia) Date: Thu, 21 Jul 2011 18:30:21 -0700 Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise) In-Reply-To: <4E28CD53.1090707@netwok.org> References: <20110719171647.03f9f9c9@pitrou.net> <4E276AF0.5090409@gmail.com> <4E28CD53.1090707@netwok.org> Message-ID: If versioned filenames are added in addition to python.exe, it still might look confusing for most users: Why do I have python and python3.2 executables? What's the difference? I'd rather go with -v argument either way, for people that *know* they want to call Python 3.2 instead of Python 3.1... Thank you, Vlad On Thu, Jul 21, 2011 at 6:07 PM, ?ric Araujo wrote: > Hi, > > Le 22/07/2011 03:03, Vlad Riscutia a ?crit : > > I'm kind of -1 on changing Python executable name. It would make sense > for > > different major versions, where there are known incompatibilities, so > > python2-python3 would make sense but python31 python32 not that much... > > > > If my team is using Python and it gets pre-installed with other > dev-tools, > > do I need to let everyone know they must call python*31*? And if we > upgrade, > > make sure everyone knows they should now call python*32*? What if we have > > scripts that call python? Make sure we update all of them whenever minor > > version is changed? > > If I understand correctly, adding versioned filenames like python3.3.exe > would happen in addition of python.exe, not in replacement. > > Regards > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pje at telecommunity.com Fri Jul 22 04:05:22 2011 From: pje at telecommunity.com (P.J. Eby) Date: Thu, 21 Jul 2011 22:05:22 -0400 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <1311296689.3039.7.camel@localhost.localdomain> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <20110722013531.4fa9ac30@pitrou.net> <1311292807.3039.4.camel@localhost.localdomain> <1311296689.3039.7.camel@localhost.localdomain> Message-ID: <20110722020615.71C913A40AA@sparrow.telecommunity.com> At 03:04 AM 7/22/2011 +0200, Antoine Pitrou wrote: >The additional confusion lies in the fact that a module can be >shadowed by something which is not a module (a mere global >variable). I find it rather baffling. If you move x.py to x/__init__.py, it does *exactly the same thing* in current versions of Python: Python 2.7.1 (r271:86832, Nov 27 2010, 18:30:46) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> from x import y >>> import x.y >>> x.y >>> y 5 The PEP does nothing new or different here. If something is baffling you, it's the behavior of "from ... import", not the actual importing process. "from x import y" means "import x; y = x.y". The PEP does not propose we change this. ;-) From ncoghlan at gmail.com Fri Jul 22 04:21:51 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 22 Jul 2011 12:21:51 +1000 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <1311296689.3039.7.camel@localhost.localdomain> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <20110722013531.4fa9ac30@pitrou.net> <1311292807.3039.4.camel@localhost.localdomain> <1311296689.3039.7.camel@localhost.localdomain> Message-ID: On Fri, Jul 22, 2011 at 11:04 AM, Antoine Pitrou wrote: > > Le vendredi 22 juillet 2011 ? 10:58 +1000, Nick Coghlan a ?crit : >> On Fri, Jul 22, 2011 at 10:00 AM, Antoine Pitrou wrote: >> > Wouldn't it produce confusing situations like the above example? >> >> I don't see how it is any more confusing than any other form of module >> shadowing. > > The additional confusion lies in the fact that a module can be shadowed > by something which is not a module (a mere global variable). I find it > rather baffling. It's still an improvement on current Python. There a submodule can be shadowed uselessly by something that doesn't even exist. For example: x.py <-- No 'y' attribute x/__init__.py <-- not needed in PEP 402 x/y.py from x import y <-- ImportError now, but would work in PEP 402 However, this does highlight an interesting corner case not yet covered by the PEP: when building a virtual path to add to an existing module, what do we do with directories that contain __init__.py[co] files? 1. Ignore the entire directory (i.e leave it out of the created path)? (always emit ImportWarning) 2. Ignore the file and add the directory to the created path anyway? (never emit ImportWarning) 3. Ignore the file and add the directory to the created path anyway? (emit ImportWarning if __init__.py is not empty) 4. Ignore the file only if it is empty, otherwise ignore the whole directory? (emit ImportWarning if __init__.py is not empty) 5. Execute the file in the namespace of the existing module? I suspect option 1 will lead to the fewest quirks, since it preserves current shadowing behaviour for modules and self-contained packages. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From stephen at xemacs.org Fri Jul 22 07:22:52 2011 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 22 Jul 2011 14:22:52 +0900 Subject: [Python-Dev] New update to PEP397 - Python launcher for Windows In-Reply-To: <4E28B26D.7070902@g.nevcal.com> References: <4E27C8BA.8040000@gmail.com> <4E28B26D.7070902@g.nevcal.com> Message-ID: <877h7asxcj.fsf@uwakimon.sk.tsukuba.ac.jp> Glenn Linderman writes: > I note in passing that '/usr/bin/env python' is hard-coded in the > launcher, which conforms to the above documentation. A single character (space or tab) is also hard-coded in Emacs's python-mode. > But there is no hard requirement in Unix, if I correctly understand > it, that '/usr/bin/env' be separated from 'python' (or whatever) by > exactly one space. No, there is no hard requirement; the shebang still is not standardized AFAIK. A single space is probably the most portable choice, however. It would not be hard to allow arbitrary whitespace, but I think documenting the restriction is sufficient. From greg.ewing at canterbury.ac.nz Fri Jul 22 09:37:40 2011 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 22 Jul 2011 19:37:40 +1200 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <1311296689.3039.7.camel@localhost.localdomain> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <20110722013531.4fa9ac30@pitrou.net> <1311292807.3039.4.camel@localhost.localdomain> <1311296689.3039.7.camel@localhost.localdomain> Message-ID: <4E2928C4.70506@canterbury.ac.nz> Antoine Pitrou wrote: > The additional confusion lies in the fact that a module can be shadowed > by something which is not a module (a mere global variable). I find it > rather baffling. I think we're stuck with that as long as we use the same syntax for importing a submodule and importing a non-module name from a module, i.e. 'from x import y'. -- Greg From greg.ewing at canterbury.ac.nz Fri Jul 22 09:37:45 2011 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 22 Jul 2011 19:37:45 +1200 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <20110722020615.71C913A40AA@sparrow.telecommunity.com> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <20110722013531.4fa9ac30@pitrou.net> <1311292807.3039.4.camel@localhost.localdomain> <1311296689.3039.7.camel@localhost.localdomain> <20110722020615.71C913A40AA@sparrow.telecommunity.com> Message-ID: <4E2928C9.2070903@canterbury.ac.nz> P.J. Eby wrote: > "from x import y" means "import x; y = x.y". It actually means slightly more that that if y is a submodule, in which case it means "import x.y; y = x.y". -- Greg From greg.ewing at canterbury.ac.nz Fri Jul 22 11:29:23 2011 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 22 Jul 2011 21:29:23 +1200 Subject: [Python-Dev] Import lock considered mysterious Message-ID: <4E2942F3.3090704@canterbury.ac.nz> I recently encountered a very mysterious bug in which a background thread would inexplicably hang while attempting to make a connection using httplib. Debugging as far as I could at the Python level led to the surprising conclusion that it was hanging while using the ascii codec to encode the host name. I couldn't imagine why, until I resorted to breaking into it with gdb and found that it was trying to import "codecs.ascii", and blocked on acquiring the import lock. The reason for *that* was that my main module was a stub that imported the real main module, which did all its work directly from the module code. So the whole program was effectively running inside an import statement and holding onto the import lock. Once I realised what was happening, it was easy to fix, but I'm a bit disturbed about how difficult it was to track down the cause. This whole episode seems to have resulted from a collision between two rather obscure things: that import statements involve locking things, and that some fairly innocuous looking calls, such as s.encode('ascii'), can trigger an import. I'm wondering whether anything can be done to make problems like this less likely to occur, or at least to make the cause more readily apparent. One shouldn't really have to use gdb to track down bugs in Python code. Is it really necessary to hold the import lock for so long? Presumably the import lock is there to protect access to things like sys.modules. Is that all? Could it be released after the module code is loaded and sys.modules has been updated, but before executing the module body? -- Greg From fuzzyman at voidspace.org.uk Fri Jul 22 11:49:04 2011 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 22 Jul 2011 10:49:04 +0100 Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise) In-Reply-To: References: <20110719171647.03f9f9c9@pitrou.net> <4E276AF0.5090409@gmail.com> <4E28CD53.1090707@netwok.org> Message-ID: <4E294790.9020409@voidspace.org.uk> On 22/07/2011 02:30, Vlad Riscutia wrote: > If versioned filenames are added in addition to python.exe, it still > might look confusing for most users: Why do I have python and > python3.2 executables? What's the difference? I'd rather go with -v > argument either way, for people that /know/ they want to call Python > 3.2 instead of Python 3.1... > It doesn't seem to be too confusing for Linux / Mac OS X users where you have both. What's more it's very useful. I still like being able to specify version from the launcher, it's probably what I will use it most for (on the rare occasions I still use Windows). Michael > Thank you, > Vlad > > On Thu, Jul 21, 2011 at 6:07 PM, ?ric Araujo > wrote: > > Hi, > > Le 22/07/2011 03:03, Vlad Riscutia a ?crit : > > I'm kind of -1 on changing Python executable name. It would make > sense for > > different major versions, where there are known > incompatibilities, so > > python2-python3 would make sense but python31 python32 not that > much... > > > > If my team is using Python and it gets pre-installed with other > dev-tools, > > do I need to let everyone know they must call python*31*? And if > we upgrade, > > make sure everyone knows they should now call python*32*? What > if we have > > scripts that call python? Make sure we update all of them > whenever minor > > version is changed? > > If I understand correctly, adding versioned filenames like > python3.3.exe > would happen in addition of python.exe, not in replacement. > > Regards > > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From cs at zip.com.au Fri Jul 22 11:58:46 2011 From: cs at zip.com.au (Cameron Simpson) Date: Fri, 22 Jul 2011 19:58:46 +1000 Subject: [Python-Dev] Import lock considered mysterious In-Reply-To: <4E2942F3.3090704@canterbury.ac.nz> References: <4E2942F3.3090704@canterbury.ac.nz> Message-ID: <20110722095846.GA32631@cskk.homeip.net> On 22Jul2011 21:29, Greg Ewing wrote: [...] | This whole episode seems to have resulted from a collision | between two rather obscure things: that import statements | involve locking things, Necessary to avoid performing the module definitons twice when a module is imported twice, surely? | and that some fairly innocuous | looking calls, such as s.encode('ascii'), can trigger an | import. There are plenty of occasions where an import might be deferred until a function call - it is a common workaround for otherwise circular imports. Personally I also sometimes defer an import if I'm importing something "large" that the module would only depend upon on in unusual (or anyway, often avoidable) circumstances; for example, a facility not in the stdlib, which a caller might avoid requiring by choosing a different aspect of the importing module such as choosing a CSV storage backend over and SQLalchemy backend. | Is it really necessary to hold the import lock for so long? | Presumably the import lock is there to protect access to | things like sys.modules. Is that all? Could it be released | after the module code is loaded and sys.modules has been | updated, but before executing the module body? Sometimes names are made by executing the module body. Since one expects to access the module's names after the import, how is this avoidable? Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ What's the point of having Nebraska if we can't put it to its highest and best use? - Patrick Bedard arguing for a 100 mph speed limit From p.f.moore at gmail.com Fri Jul 22 12:30:40 2011 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 22 Jul 2011 11:30:40 +0100 Subject: [Python-Dev] Import lock considered mysterious In-Reply-To: <4E2942F3.3090704@canterbury.ac.nz> References: <4E2942F3.3090704@canterbury.ac.nz> Message-ID: On 22 July 2011 10:29, Greg Ewing wrote: > The reason for *that* was that my main module was a stub > that imported the real main module, which did all its > work directly from the module code. So the whole program > was effectively running inside an import statement and > holding onto the import lock. I realise you probably know this, but I think this is covered by the standard advice to avoid having substantial code running at module level (i.e., on import). It may be useful to clarify or further emphasize that advice in the Python documentation. Paul. PS This is not to say that I oppose trying to reduce the chance of this type of thing happening... From solipsis at pitrou.net Fri Jul 22 14:48:58 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 22 Jul 2011 14:48:58 +0200 Subject: [Python-Dev] Import lock considered mysterious References: <4E2942F3.3090704@canterbury.ac.nz> Message-ID: <20110722144858.49cb7f2f@pitrou.net> On Fri, 22 Jul 2011 21:29:23 +1200 Greg Ewing wrote: > > This whole episode seems to have resulted from a collision > between two rather obscure things: that import statements > involve locking things, and that some fairly innocuous > looking calls, such as s.encode('ascii'), can trigger an > import. Indeed. > I'm wondering whether anything can be done to make problems > like this less likely to occur, or at least to make the > cause more readily apparent. One shouldn't really have to > use gdb to track down bugs in Python code. See http://bugs.python.org/issue9260 There's a patch there but it needs additional sophistication to remove deadlocks when doing concurrent circular imports. Regards Antoine. From brian.curtin at gmail.com Fri Jul 22 15:41:26 2011 From: brian.curtin at gmail.com (Brian Curtin) Date: Fri, 22 Jul 2011 08:41:26 -0500 Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise) In-Reply-To: References: <20110719171647.03f9f9c9@pitrou.net> <4E276AF0.5090409@gmail.com> <4E28CD53.1090707@netwok.org> Message-ID: On Thu, Jul 21, 2011 at 20:30, Vlad Riscutia wrote: > If versioned filenames are added in addition to python.exe, it still might > look confusing for most users: Why do I have python and python3.2 > executables? What's the difference? I'd rather go with -v argument either > way, for people that *know* they want to call Python 3.2 instead of Python > 3.1... > > Thank you, > Vlad > Honestly, would it really be that confusing? Seeing python32.exe inside C:\Python32 shouldn't be a huge surprise, and ActiveState has been doing something like this for years (forever?). Versioned executables in addition to the standard python.exe is something I've wanted for a while, but that's outside of this PEP. This way you could have C:\Python27 and C:\Python32 on your path and explicitly open up the right one. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Fri Jul 22 16:24:20 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 23 Jul 2011 00:24:20 +1000 Subject: [Python-Dev] Import lock considered mysterious In-Reply-To: <4E2942F3.3090704@canterbury.ac.nz> References: <4E2942F3.3090704@canterbury.ac.nz> Message-ID: On Fri, Jul 22, 2011 at 7:29 PM, Greg Ewing wrote: > Is it really necessary to hold the import lock for so long? > Presumably the import lock is there to protect access to > things like sys.modules. Is that all? Could it be released > after the module code is loaded and sys.modules has been > updated, but before executing the module body? No, because we don't know if we're keeping it until the module body is executed successfully and we don't want other threads to see the partially initialised module. There's a reason the docs say never to spawn and then wait for threads as a side effect of import (http://docs.python.org/library/threading#importing-in-threaded-code) If you want to delegate main module execution to another file, then runpy.run_module is available. Antoine's fine grained import locking patch (or something along those lines) would improve matters, but currently causes its own deadlock problems. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From pje at telecommunity.com Fri Jul 22 17:01:57 2011 From: pje at telecommunity.com (P.J. Eby) Date: Fri, 22 Jul 2011 11:01:57 -0400 Subject: [Python-Dev] Import lock considered mysterious In-Reply-To: <20110722144858.49cb7f2f@pitrou.net> References: <4E2942F3.3090704@canterbury.ac.nz> <20110722144858.49cb7f2f@pitrou.net> Message-ID: <20110722150247.6C7083A409B@sparrow.telecommunity.com> At 02:48 PM 7/22/2011 +0200, Antoine Pitrou wrote: >See http://bugs.python.org/issue9260 > >There's a patch there but it needs additional sophistication to remove >deadlocks when doing concurrent circular imports. I don't think that approach can work, as PEP 302 loaders can currently assume the global import lock is being held when they run... and in general, there are too many global data structures in sys that need to be protected by code that uses such things. A simpler solution to Greg's problem would be to have a timeout on attempts to acquire the import lock, and have it fail with a RuntimeError describing the problem. (*Not* an ImportError, mind you, which might get ignored and trigger a different code path.) The timeout would need to be on the order of seconds to prevent false positives, and there'd need to be a way to change or remove the timeout in the event somebody really needs to. But it would eliminate the mysteriousness. A unique and Google-able error message would let someone find a clear explanation of what's going on, as well. A second thing that *could* be helpful would be to issue a warning when a new thread is started (or waited on) while the import lock is held. This is already known to be a bad thing to do. The tricky part is issuing the warning for the right caller level, but I suppose you could walk back up the call stack until you found some module-level code, and then fingered that line of code as the culprit. Yes, that might do it: the code for starting or waiting on a thread, could check to see if the import lock is held by the current thread, and if so, walk up the stack to find a module frame (one where f_globals is f_locals and __name__ in f_locals and sys.modules[__name__].__dict__ is f_locals), and if one is found, issue a warning about not starting or waiting on threads in module-level code. Between that and the timeout, the mysteriousness could be completely done away with, without throwing a monkey wrench into the current import mechanisms. From solipsis at pitrou.net Fri Jul 22 17:07:07 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 22 Jul 2011 17:07:07 +0200 Subject: [Python-Dev] Import lock considered mysterious In-Reply-To: <20110722150247.6C7083A409B@sparrow.telecommunity.com> References: <4E2942F3.3090704@canterbury.ac.nz> <20110722144858.49cb7f2f@pitrou.net> <20110722150247.6C7083A409B@sparrow.telecommunity.com> Message-ID: <1311347227.3605.4.camel@localhost.localdomain> > >See http://bugs.python.org/issue9260 > > > >There's a patch there but it needs additional sophistication to remove > >deadlocks when doing concurrent circular imports. > > I don't think that approach can work, as PEP 302 loaders can > currently assume the global import lock is being held when they > run... and in general, there are too many global data structures in > sys that need to be protected by code that uses such things. Some of the comments in the issue are about that. The global import lock could *still* protect those structures when needed. It just doesn't have to be held when executing a module's body. > Between that and the timeout, the mysteriousness could be completely > done away with, without throwing a monkey wrench into the current > import mechanisms. Isn't the current import mechanism already a monkey wrench? Regards Antoine. From riscutiavlad at gmail.com Fri Jul 22 18:02:35 2011 From: riscutiavlad at gmail.com (Vlad Riscutia) Date: Fri, 22 Jul 2011 09:02:35 -0700 Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise) In-Reply-To: References: <20110719171647.03f9f9c9@pitrou.net> <4E276AF0.5090409@gmail.com> <4E28CD53.1090707@netwok.org> Message-ID: OK then. I don't have a *strong* opinion against it, just thought that most people have one version of Python, maybe 2 versions as in 2.x and 3.x, so I would understand python2.exe, python3.exe but yeah, it's not that big of a deal either way. Thank you, Vlad On Fri, Jul 22, 2011 at 6:41 AM, Brian Curtin wrote: > On Thu, Jul 21, 2011 at 20:30, Vlad Riscutia wrote: > >> If versioned filenames are added in addition to python.exe, it still might >> look confusing for most users: Why do I have python and python3.2 >> executables? What's the difference? I'd rather go with -v argument either >> way, for people that *know* they want to call Python 3.2 instead of >> Python 3.1... >> >> Thank you, >> Vlad >> > > Honestly, would it really be that confusing? Seeing python32.exe inside > C:\Python32 shouldn't be a huge surprise, and ActiveState has been doing > something like this for years (forever?). > > Versioned executables in addition to the standard python.exe is something > I've wanted for a while, but that's outside of this PEP. This way you could > have C:\Python27 and C:\Python32 on your path and explicitly open up the > right one. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From status at bugs.python.org Fri Jul 22 18:07:26 2011 From: status at bugs.python.org (Python tracker) Date: Fri, 22 Jul 2011 18:07:26 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20110722160726.B02D01DE28@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2011-07-15 - 2011-07-22) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 2886 ( +6) closed 21507 (+33) total 24393 (+39) Open issues with patches: 1253 Issues opened (29) ================== #10881: test_site and macframework builds fails http://bugs.python.org/issue10881 reopened by vinay.sajip #12266: str.capitalize contradicts oneself http://bugs.python.org/issue12266 reopened by ezio.melotti #12575: add a AST validator http://bugs.python.org/issue12575 opened by benjamin.peterson #12576: urlib.request fails to open some sites http://bugs.python.org/issue12576 opened by daniel.ugra #12578: Erratic socket.gaierror: [Errno 11004] when using smtplib http://bugs.python.org/issue12578 opened by David.Ward #12581: Increased test coverage of test_urlparse http://bugs.python.org/issue12581 opened by haggholm #12583: More detailed ImportError messages http://bugs.python.org/issue12583 opened by cool-RR #12585: distutils dereferences symlinks for zip but not for bztar/gzta http://bugs.python.org/issue12585 opened by fberger #12586: Enhanced email API: header objects http://bugs.python.org/issue12586 opened by r.david.murray #12588: test_capi.test_subinterps() failed on OpenBSD (powerpc) http://bugs.python.org/issue12588 opened by rpointel #12589: test_long.test_nan_inf() failed on OpenBSD (powerpc) http://bugs.python.org/issue12589 opened by rpointel #12590: First line and cursor not visible when opening files http://bugs.python.org/issue12590 opened by sforman #12591: text files returned by subprocess.Popen with universal_newline http://bugs.python.org/issue12591 opened by anacrolix #12592: modify configure.in to detect OpenBSD 5.x http://bugs.python.org/issue12592 opened by rpointel #12594: Docs for py3k still refer to "MacPython 2.5 folder" http://bugs.python.org/issue12594 opened by hop #12595: python.h redundant redeclarations http://bugs.python.org/issue12595 opened by verticalduck #12596: cPickle - stored data differ for same dictionary http://bugs.python.org/issue12596 opened by Philipp.M??lders #12598: Move sys variable initialization from import.c to sysmodule.c http://bugs.python.org/issue12598 opened by ericsnow #12599: Use 'is not None' where appropriate in importlib http://bugs.python.org/issue12599 opened by ncoghlan #12600: Support parameterized TestCases in unittest http://bugs.python.org/issue12600 opened by abingham #12602: Missing using docs cross-references http://bugs.python.org/issue12602 opened by ncoghlan #12603: pydoc.synopsis breaks if filesystem returns mtime of 0 (common http://bugs.python.org/issue12603 opened by joshtriplett #12604: VTRACE macro in _sre.c should use do {} while (0) http://bugs.python.org/issue12604 opened by joshtriplett #12605: Enhancements to gdb 7 debugging hooks http://bugs.python.org/issue12605 opened by dmalcolm #12606: Mutable Sequence Type works different for lists and bytearrays http://bugs.python.org/issue12606 opened by py.user #12607: subprocess(stdout=..., stderr=sys.stdout) breaks stderr for ch http://bugs.python.org/issue12607 opened by chn #12608: crash in PyAST_Compile when running Python code http://bugs.python.org/issue12608 opened by Albert.Zeyer #12610: Fatal Python error: non-string found in code slot http://bugs.python.org/issue12610 opened by Albert.Zeyer #12611: 2to3 crashes when converting doctest using reduce() http://bugs.python.org/issue12611 opened by VPeric Most recent 15 issues with no replies (15) ========================================== #12611: 2to3 crashes when converting doctest using reduce() http://bugs.python.org/issue12611 #12610: Fatal Python error: non-string found in code slot http://bugs.python.org/issue12610 #12604: VTRACE macro in _sre.c should use do {} while (0) http://bugs.python.org/issue12604 #12603: pydoc.synopsis breaks if filesystem returns mtime of 0 (common http://bugs.python.org/issue12603 #12599: Use 'is not None' where appropriate in importlib http://bugs.python.org/issue12599 #12598: Move sys variable initialization from import.c to sysmodule.c http://bugs.python.org/issue12598 #12595: python.h redundant redeclarations http://bugs.python.org/issue12595 #12592: modify configure.in to detect OpenBSD 5.x http://bugs.python.org/issue12592 #12590: First line and cursor not visible when opening files http://bugs.python.org/issue12590 #12586: Enhanced email API: header objects http://bugs.python.org/issue12586 #12562: calling mmap twice fails on Windows http://bugs.python.org/issue12562 #12553: email should default to 8bit CTE unless policy.must_be_7bit is http://bugs.python.org/issue12553 #12542: Remove duplicates of cmp_to_key used in tests http://bugs.python.org/issue12542 #12541: Accepting Badly formed headers in urllib HTTPBasicAuth http://bugs.python.org/issue12541 #12532: PyPI server index names with '/' in them http://bugs.python.org/issue12532 Most recent 15 issues waiting for review (15) ============================================= #12607: subprocess(stdout=..., stderr=sys.stdout) breaks stderr for ch http://bugs.python.org/issue12607 #12605: Enhancements to gdb 7 debugging hooks http://bugs.python.org/issue12605 #12598: Move sys variable initialization from import.c to sysmodule.c http://bugs.python.org/issue12598 #12586: Enhanced email API: header objects http://bugs.python.org/issue12586 #12581: Increased test coverage of test_urlparse http://bugs.python.org/issue12581 #12575: add a AST validator http://bugs.python.org/issue12575 #12572: HP/UX compiler workarounds http://bugs.python.org/issue12572 #12567: curses implementation of Unicode is wrong in Python 3 http://bugs.python.org/issue12567 #12561: Compiler workaround for wide string constants in Modules/getpa http://bugs.python.org/issue12561 #12560: libpython.so not built on OpenBSD http://bugs.python.org/issue12560 #12559: gzip.open() needs an optional encoding argument http://bugs.python.org/issue12559 #12555: PEP 3151 implementation http://bugs.python.org/issue12555 #12546: builtin __format__ methods cannot fill with \x00 char http://bugs.python.org/issue12546 #12542: Remove duplicates of cmp_to_key used in tests http://bugs.python.org/issue12542 #12531: documentation index entries for * and ** http://bugs.python.org/issue12531 Top 10 most discussed issues (10) ================================= #7897: Support parametrized tests in unittest http://bugs.python.org/issue7897 24 msgs #11343: Make errors due to full parser stack identifiable http://bugs.python.org/issue11343 18 msgs #12589: test_long.test_nan_inf() failed on OpenBSD (powerpc) http://bugs.python.org/issue12589 15 msgs #12572: HP/UX compiler workarounds http://bugs.python.org/issue12572 13 msgs #12583: More detailed ImportError messages http://bugs.python.org/issue12583 13 msgs #1626300: 'Installing Python Modules' does not work for Windows http://bugs.python.org/issue1626300 9 msgs #12326: Linux 3: tests should avoid using sys.platform == 'linux2' http://bugs.python.org/issue12326 8 msgs #12540: "Restart Shell" command leaves pythonw.exe processes running http://bugs.python.org/issue12540 8 msgs #1170: shlex have problems with parsing unicode http://bugs.python.org/issue1170 7 msgs #6721: Locks in python standard library should be sanitized on fork http://bugs.python.org/issue6721 7 msgs Issues closed (34) ================== #7484: smtplib: verify breaks with Postfix servers http://bugs.python.org/issue7484 closed by r.david.murray #10271: warnings.showwarning should allow any callable object http://bugs.python.org/issue10271 closed by brett.cannon #10309: dlmalloc.c needs _GNU_SOURCE for mremap() http://bugs.python.org/issue10309 closed by barry #11055: OS X IDLE 3.2 Save As menu accelerator opens two Save windows http://bugs.python.org/issue11055 closed by ned.deily #11321: 9th import of module _pickle always crashes http://bugs.python.org/issue11321 closed by pitrou #11436: Clarify struct doc for format 's', when it is mentioned withou http://bugs.python.org/issue11436 closed by python-dev #11603: Python crashes or hangs when rebinding __repr__ as __str__ http://bugs.python.org/issue11603 closed by pitrou #11627: segfault raising an arbitrary object as an exception http://bugs.python.org/issue11627 closed by python-dev #11629: Reference implementation for PEP 397 http://bugs.python.org/issue11629 closed by mhammond #12273: Change ast.__version__ calculation to provide consistent order http://bugs.python.org/issue12273 closed by python-dev #12279: Add build_distinfo command to packaging http://bugs.python.org/issue12279 closed by eric.araujo #12372: semaphore errors on AIX 7.1 http://bugs.python.org/issue12372 closed by neologix #12434: Strengthen 2.7 io types warning http://bugs.python.org/issue12434 closed by eli.bendersky #12478: Possible error in HTTPErrorProcessor documentation http://bugs.python.org/issue12478 closed by python-dev #12479: Add HTTPErrorProcessor class definition http://bugs.python.org/issue12479 closed by python-dev #12521: IDLE 3.2 crashes on Mac OS 10.6 with ActiveState Tcl/Tk 8.5 http://bugs.python.org/issue12521 closed by ned.deily #12524: change httplib docs POST example http://bugs.python.org/issue12524 closed by python-dev #12551: Provide data for TLS channel binding http://bugs.python.org/issue12551 closed by pitrou #12556: Disable size checks in mmap.mmap() http://bugs.python.org/issue12556 closed by superbobry #12570: BaseHTTPServer.shutdown() locks if the last request 404'd http://bugs.python.org/issue12570 closed by orsenthil #12571: Python built on Linux 3.0 doesn't include DLFCN http://bugs.python.org/issue12571 closed by pitrou #12573: add resource checks for dangling threads and processes http://bugs.python.org/issue12573 closed by pitrou #12574: Documentation for Queue in 2.x has an incorrect title http://bugs.python.org/issue12574 closed by eli.bendersky #12577: Misleading shutil.move docs regarding when os.rename is used http://bugs.python.org/issue12577 closed by python-dev #12579: str.format_map raises a SystemError for format strings with po http://bugs.python.org/issue12579 closed by python-dev #12580: Documentation error in Decimal module http://bugs.python.org/issue12580 closed by holdenweb #12582: lib-dynload missing in python install http://bugs.python.org/issue12582 closed by ned.deily #12584: win.protocol('WM_DELETE_WINDOW'...) still deletes window on OS http://bugs.python.org/issue12584 closed by ned.deily #12587: tokenize_tests-utf8-coding-cookie-and-no-utf8-bom-sig.txt has http://bugs.python.org/issue12587 closed by ned.deily #12593: define OpenBSD in configure.in if Py_ENABLE_SHARED == 1 http://bugs.python.org/issue12593 closed by skrah #12597: List created by multiplication behaves different http://bugs.python.org/issue12597 closed by sandro.tosi #12601: Spelling error in the comments in the webbrowser module. http://bugs.python.org/issue12601 closed by ezio.melotti #12609: SystemError: Objects/codeobject.c:64: bad argument to internal http://bugs.python.org/issue12609 closed by python-dev #665194: datetime-RFC2822 roundtripping http://bugs.python.org/issue665194 closed by r.david.murray From v+python at g.nevcal.com Sat Jul 23 05:01:23 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Fri, 22 Jul 2011 20:01:23 -0700 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: <4E28C7D1.3010002@skippinet.com.au> References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> <4E274C0F.40400@g.nevcal.com> <4E276795.9050404@gmail.com> <4E276C7C.1020605@g.nevcal.com> <4E276EBF.6000509@gmail.com> <4E28B016.5050500@g.nevcal.com> <4E28C7D1.3010002@skippinet.com.au> Message-ID: <4E2A3983.2060901@g.nevcal.com> On 7/21/2011 5:44 PM, Mark Hammond wrote: > On 22/07/2011 9:02 AM, Glenn Linderman wrote: >> Bad logic is get_configured_value! get_configured_value only looks in >> the global configuration file if there is a local configuration file >> that doesn't have the setting. It should look in the global >> configuration file if there is no local configuration file _OR_ the is a >> local configuration file without the setting. >> >> I'll await a new launcher. > > I just pushed a fix and hopefully Vinay will push a new MSI soon. > > Thanks, > > Mark > What (free?) toolset is needed for building the launcher? I don't even have a C compiler installed on this computer yet. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Sat Jul 23 08:55:36 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 23 Jul 2011 16:55:36 +1000 Subject: [Python-Dev] [Python-checkins] r88867 - in tracker/instances/python-dev: extensions/pydevutils.py html/file.item.html html/issue.item.html html/msg.item.html In-Reply-To: <3RLRcY6MHZzMml@mail.python.org> References: <3RLRcY6MHZzMml@mail.python.org> Message-ID: On Sat, Jul 23, 2011 at 8:34 AM, ezio.melotti wrote: > Author: ezio.melotti > Date: Sat Jul 23 00:34:53 2011 > New Revision: 88867 > > Log: > #267: remove the remove button from the issue page, move it to the msg/file page, and add a button to add back removed messages/files. Thank you! (I accidentally deleted one of RDM's comments just the other day) Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From g.brandl at gmx.net Sat Jul 23 09:44:50 2011 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 23 Jul 2011 09:44:50 +0200 Subject: [Python-Dev] cpython: Merge - Issue #12592: Make Python build on OpenBSD 5 (and future major In-Reply-To: References: Message-ID: Am 22.07.2011 23:51, schrieb charles-francois.natali: > http://hg.python.org/cpython/rev/63de97ae832e > changeset: 71462:63de97ae832e > parent: 71459:d3f0f72c31f8 > user: Charles-Fran?ois Natali > date: Fri Jul 22 23:52:02 2011 +0200 > summary: > Merge - Issue #12592: Make Python build on OpenBSD 5 (and future major > releases). > > files: > Misc/NEWS | 2 + > configure | 597 ++++++++++++++++++-------------------- > configure.in | 4 +- > 3 files changed, 291 insertions(+), 312 deletions(-) Note that this commit wasn't actually a merge -- you'll have to use the "hg merge" command for that. Georg From ezio.melotti at gmail.com Sat Jul 23 10:45:25 2011 From: ezio.melotti at gmail.com (Ezio Melotti) Date: Sat, 23 Jul 2011 11:45:25 +0300 Subject: [Python-Dev] [Python-checkins] r88867 - in tracker/instances/python-dev: extensions/pydevutils.py html/file.item.html html/issue.item.html html/msg.item.html In-Reply-To: References: <3RLRcY6MHZzMml@mail.python.org> Message-ID: <4E2A8A25.2010006@gmail.com> On 23/07/2011 9.55, Nick Coghlan wrote: > On Sat, Jul 23, 2011 at 8:34 AM, ezio.melotti > wrote: >> Author: ezio.melotti >> Date: Sat Jul 23 00:34:53 2011 >> New Revision: 88867 >> >> Log: >> #267: remove the remove button from the issue page, move it to the msg/file page, and add a button to add back removed messages/files. > Thank you! (I accidentally deleted one of RDM's comments just the other day) You are welcome! (That was actually one of the reasons that made me look at that issue again) Now restoring messages and files that got deleted should be trivial, and deleting them accidentally (almost) impossible. Messages and files can be deleted/restored from their pages. Links to these pages can be found in the history of the issue. For messages and files that are still linked to the issue, the "(view)" and "edit" links can also be used. Best Regards, Ezio Melotti > > Cheers, > Nick. > From cf.natali at gmail.com Sat Jul 23 11:46:05 2011 From: cf.natali at gmail.com (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=) Date: Sat, 23 Jul 2011 11:46:05 +0200 Subject: [Python-Dev] cpython: Merge - Issue #12592: Make Python build on OpenBSD 5 (and future major In-Reply-To: References: Message-ID: > Note that this commit wasn't actually a merge -- you'll have to use the > "hg merge" command for that. You're right. I guess that's what happens when I try to work past my usual bedtime ;-) By the way, I'm still getting errors upon push, and it looks like when I push a patch, this doesn't trigger any build on the buildbots. It used to work, any idea what's going on? From vinay_sajip at yahoo.co.uk Sat Jul 23 14:34:28 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Sat, 23 Jul 2011 12:34:28 +0000 (UTC) Subject: [Python-Dev] 3.2.1 encoding surprise References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> <4E274C0F.40400@g.nevcal.com> <4E276795.9050404@gmail.com> <4E276C7C.1020605@g.nevcal.com> <4E276EBF.6000509@gmail.com> <4E28B016.5050500@g.nevcal.com> <4E28C7D1.3010002@skippinet.com.au> <4E2A3983.2060901@g.nevcal.com> Message-ID: Glenn Linderman g.nevcal.com> writes: I aim to update the launcher downloads Real Soon Now. > What (free?) toolset is needed for building the launcher?? I don't > even have a C compiler installed on this computer yet. > The tools I use for building the launcher are: Windows SDK (for the 64-bit compilers), Visual Studio Express C++ (2008 Edition), WiX for the installers, and Python. All are free as in beer, and some are also free as in speech. Regards, Vinay Sajip From solipsis at pitrou.net Sat Jul 23 15:44:07 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 23 Jul 2011 15:44:07 +0200 Subject: [Python-Dev] cpython: Merge - Issue #12592: Make Python build on OpenBSD 5 (and future major References: Message-ID: <20110723154407.4b10ba34@pitrou.net> On Sat, 23 Jul 2011 11:46:05 +0200 Charles-Fran?ois Natali wrote: > > Note that this commit wasn't actually a merge -- you'll have to use the > > "hg merge" command for that. > > You're right. > I guess that's what happens when I try to work past my usual bedtime ;-) > > By the way, I'm still getting errors upon push, and it looks like when > I push a patch, this doesn't trigger any build on the buildbots. It > used to work, any idea what's going on? What is the error? Is a traceback printed? Regards Antoine. From cf.natali at gmail.com Sat Jul 23 16:10:38 2011 From: cf.natali at gmail.com (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=) Date: Sat, 23 Jul 2011 16:10:38 +0200 Subject: [Python-Dev] cpython: Merge - Issue #12592: Make Python build on OpenBSD 5 (and future major In-Reply-To: <20110723154407.4b10ba34@pitrou.net> References: <20110723154407.4b10ba34@pitrou.net> Message-ID: (See http://mail.python.org/pipermail/python-dev/2011-July/112309.html for the original mail): """ $ hg push pushing to ssh://hg at hg.python.org/cpython searching for changes remote: adding changesets remote: adding manifests remote: adding file changes remote: added 1 changesets with 2 changes to 2 files remote: change(s) NOT sent, something went wrong: remote: [Failure instance: Traceback from remote host -- Traceback unavailable remote: ] remote: sent email to roundup at report at bugs.python.org remote: notified python-checkins at python.org of incoming changeset ca077f2672e3 """ No traceback. Last time I did a push (yesterday), there was a slight difference, the lines reporting the failure were prefixed by "buildbot:" (IIRC). And indeed, none of my pushes triggers a build on the buildbots (and I'm sure it worked a couple weeks ago). Thanks, cf From solipsis at pitrou.net Sat Jul 23 16:34:20 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 23 Jul 2011 16:34:20 +0200 Subject: [Python-Dev] cpython: Merge - Issue #12592: Make Python build on OpenBSD 5 (and future major In-Reply-To: References: <20110723154407.4b10ba34@pitrou.net> Message-ID: <20110723163420.7047a6b9@pitrou.net> On Sat, 23 Jul 2011 16:10:38 +0200 Charles-Fran?ois Natali wrote: > > No traceback. > Last time I did a push (yesterday), there was a slight difference, the > lines reporting the failure were prefixed by "buildbot:" (IIRC). And > indeed, none of my pushes triggers a build on the buildbots (and I'm > sure it worked a couple weeks ago). Ok, it should be fixed now. Regards Antoine. From cf.natali at gmail.com Sat Jul 23 19:22:34 2011 From: cf.natali at gmail.com (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=) Date: Sat, 23 Jul 2011 19:22:34 +0200 Subject: [Python-Dev] cpython: Merge - Issue #12592: Make Python build on OpenBSD 5 (and future major In-Reply-To: <20110723163420.7047a6b9@pitrou.net> References: <20110723154407.4b10ba34@pitrou.net> <20110723163420.7047a6b9@pitrou.net> Message-ID: > Ok, it should be fixed now. > Indeed. Thanks! cf From andrew.svetlov at gmail.com Sun Jul 24 00:20:08 2011 From: andrew.svetlov at gmail.com (Andrew Svetlov) Date: Sun, 24 Jul 2011 01:20:08 +0300 Subject: [Python-Dev] tuple[int] optimization Message-ID: tuple[int] is 30% slower than list[int]. Please let me explain the problem. (1, 2, 3)[1] has compiled as BINARY_SUBSCR operation. ceval.c has optimization for list: case BINARY_SUBSCR: w = POP(); v = TOP(); if (PyList_CheckExact(v) && PyInt_CheckExact(w)) { /* INLINE: list[int] */ Py_ssize_t i = PyInt_AsSsize_t(w); if (i < 0) i += PyList_GET_SIZE(v); if (i >= 0 && i < PyList_GET_SIZE(v)) { x = PyList_GET_ITEM(v, i); Py_INCREF(x); } else goto slow_get; } else slow_get: x = PyObject_GetItem(v, w); Py_DECREF(v); Py_DECREF(w); SET_TOP(x); if (x != NULL) continue; break; For tuple code follows slow way via tp_as_mapping slot and calls tuplesubscript from Objects/tupleobject.c Looks like tuple is goot applicant to be optimized as list does. tuples is used in python programs very often and getting element from tuple by index as common operation as list[int]. Is there reasons to prevent special case for tuples in BINARY_SUBSCR? From ryan at rfk.id.au Sun Jul 24 01:13:07 2011 From: ryan at rfk.id.au (Ryan Kelly) Date: Sun, 24 Jul 2011 09:13:07 +1000 Subject: [Python-Dev] tuple[int] optimization In-Reply-To: References: Message-ID: <1311462787.2225.5.camel@durian> On Sun, 2011-07-24 at 01:20 +0300, Andrew Svetlov wrote: > tuple[int] is 30% slower than list[int]. > Please let me explain the problem. > > (1, 2, 3)[1] has compiled as BINARY_SUBSCR operation. > ceval.c has optimization for list: > > case BINARY_SUBSCR: > w = POP(); > v = TOP(); > if (PyList_CheckExact(v) && PyInt_CheckExact(w)) { > /* INLINE: list[int] */ > Py_ssize_t i = PyInt_AsSsize_t(w); > if (i < 0) > i += PyList_GET_SIZE(v); > if (i >= 0 && i < PyList_GET_SIZE(v)) { > x = PyList_GET_ITEM(v, i); > Py_INCREF(x); > } > else > goto slow_get; > } > else > slow_get: > x = PyObject_GetItem(v, w); > Py_DECREF(v); > Py_DECREF(w); > SET_TOP(x); > if (x != NULL) continue; > break; > > For tuple code follows slow way via tp_as_mapping slot and calls > tuplesubscript from Objects/tupleobject.c > > Looks like tuple is goot applicant to be optimized as list does. > tuples is used in python programs very often and getting element from > tuple by index as common operation as list[int]. > > Is there reasons to prevent special case for tuples in BINARY_SUBSCR? In latest trunk this optimisation seems to have gone away, the code is now: TARGET(BINARY_SUBSCR) w = POP(); v = TOP(); x = PyObject_GetItem(v, w); Py_DECREF(v); Py_DECREF(w); SET_TOP(x); if (x != NULL) DISPATCH(); break; The implementation of PyObject_GetItem doesn't appear to have changed though. Maybe this optimisation was no longer worth it in practice? Ryan -- Ryan Kelly http://www.rfk.id.au | This message is digitally signed. Please visit ryan at rfk.id.au | http://www.rfk.id.au/ramblings/gpg/ for details -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part URL: From solipsis at pitrou.net Sun Jul 24 01:27:55 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 24 Jul 2011 01:27:55 +0200 Subject: [Python-Dev] tuple[int] optimization References: <1311462787.2225.5.camel@durian> Message-ID: <20110724012755.32bf6a8f@pitrou.net> On Sun, 24 Jul 2011 09:13:07 +1000 Ryan Kelly wrote: > > In latest trunk this optimisation seems to have gone away, the code is > now: > > TARGET(BINARY_SUBSCR) > w = POP(); > v = TOP(); > x = PyObject_GetItem(v, w); > Py_DECREF(v); > Py_DECREF(w); > SET_TOP(x); > if (x != NULL) DISPATCH(); > break; > > The implementation of PyObject_GetItem doesn't appear to have changed > though. Maybe this optimisation was no longer worth it in practice? The optimization was probably removed because PyInt objects don't exist anymore. There's a related but more ambitious patch at http://bugs.python.org/issue10044. In practice however, such micro-optimizations usually have little or no effect. Regards Antoine. From andrew.svetlov at gmail.com Sun Jul 24 02:15:52 2011 From: andrew.svetlov at gmail.com (Andrew Svetlov) Date: Sun, 24 Jul 2011 03:15:52 +0300 Subject: [Python-Dev] tuple[int] optimization In-Reply-To: <20110724012755.32bf6a8f@pitrou.net> References: <1311462787.2225.5.camel@durian> <20110724012755.32bf6a8f@pitrou.net> Message-ID: You right. Sorry, I missed changes in ceval.c for py3k. Please note, simple test like: from timeit import timeit print('list', timeit("l[0]", "l = [1]")) print('tuple', timeit("l[0]", "l = (1,)")) Has results: andrew at ocean ~/p/cpython> python2.7 z.py ('list', 0.03479599952697754) ('tuple', 0.046610116958618164) andrew at ocean ~/p/cpython> python3.2 z.py list 0.04870104789733887 tuple 0.04825997352600098 For python2.7 list[int] microoptimization saves 25-30%, while 3.2 (and trunk) very close to "unoptimized" 2.7 version. On Sun, Jul 24, 2011 at 2:27 AM, Antoine Pitrou wrote: > On Sun, 24 Jul 2011 09:13:07 +1000 > Ryan Kelly wrote: >> >> In latest trunk this optimisation seems to have gone away, the code is >> now: >> >> ? ? ? ? TARGET(BINARY_SUBSCR) >> ? ? ? ? ? ? w = POP(); >> ? ? ? ? ? ? v = TOP(); >> ? ? ? ? ? ? x = PyObject_GetItem(v, w); >> ? ? ? ? ? ? Py_DECREF(v); >> ? ? ? ? ? ? Py_DECREF(w); >> ? ? ? ? ? ? SET_TOP(x); >> ? ? ? ? ? ? if (x != NULL) DISPATCH(); >> ? ? ? ? ? ? break; >> >> The implementation of PyObject_GetItem doesn't appear to have changed >> though. ?Maybe this optimisation was no longer worth it in practice? > > The optimization was probably removed because PyInt objects don't exist > anymore. There's a related but more ambitious patch at > http://bugs.python.org/issue10044. > > In practice however, such micro-optimizations usually have little or no > effect. > > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com > From solipsis at pitrou.net Sun Jul 24 02:18:23 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 24 Jul 2011 02:18:23 +0200 Subject: [Python-Dev] tuple[int] optimization In-Reply-To: References: <1311462787.2225.5.camel@durian> <20110724012755.32bf6a8f@pitrou.net> Message-ID: <1311466703.3001.12.camel@localhost.localdomain> Le dimanche 24 juillet 2011 ? 03:15 +0300, Andrew Svetlov a ?crit : > You right. Sorry, I missed changes in ceval.c for py3k. > Please note, simple test like: > > from timeit import timeit > > print('list', timeit("l[0]", "l = [1]")) > print('tuple', timeit("l[0]", "l = (1,)")) > > Has results: > > andrew at ocean ~/p/cpython> python2.7 z.py > ('list', 0.03479599952697754) > ('tuple', 0.046610116958618164) > > andrew at ocean ~/p/cpython> python3.2 z.py > list 0.04870104789733887 > tuple 0.04825997352600098 > > For python2.7 list[int] microoptimization saves 25-30%, while 3.2 (and > trunk) very close to "unoptimized" 2.7 version. My point is that on non-trivial benchmarks, the savings are almost zero. If you look at the (much more complex) patch I linked to, the savings are at most 10% on a couple of select benchmarks, other benchmarks showing almost no difference. Regards Antoine. From andrew.svetlov at gmail.com Sun Jul 24 02:26:26 2011 From: andrew.svetlov at gmail.com (Andrew Svetlov) Date: Sun, 24 Jul 2011 03:26:26 +0300 Subject: [Python-Dev] tuple[int] optimization In-Reply-To: <1311466703.3001.12.camel@localhost.localdomain> References: <1311462787.2225.5.camel@durian> <20110724012755.32bf6a8f@pitrou.net> <1311466703.3001.12.camel@localhost.localdomain> Message-ID: I undrestand your point. Thank you for explanation. On Sun, Jul 24, 2011 at 3:18 AM, Antoine Pitrou wrote: > Le dimanche 24 juillet 2011 ? 03:15 +0300, Andrew Svetlov a ?crit : >> You right. Sorry, I missed changes in ceval.c for py3k. >> Please note, simple test like: >> >> from timeit import timeit >> >> print('list', timeit("l[0]", "l = [1]")) >> print('tuple', timeit("l[0]", "l = (1,)")) >> >> Has results: >> >> andrew at ocean ~/p/cpython> python2.7 z.py >> ('list', 0.03479599952697754) >> ('tuple', 0.046610116958618164) >> >> andrew at ocean ~/p/cpython> python3.2 z.py >> list 0.04870104789733887 >> tuple 0.04825997352600098 >> >> For python2.7 list[int] microoptimization saves 25-30%, while 3.2 (and >> trunk) very close to "unoptimized" 2.7 version. > > My point is that on non-trivial benchmarks, the savings are almost zero. > If you look at the (much more complex) patch I linked to, the savings > are at most 10% on a couple of select benchmarks, other benchmarks > showing almost no difference. > > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com > From raymond.hettinger at gmail.com Sun Jul 24 03:34:16 2011 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Sat, 23 Jul 2011 18:34:16 -0700 Subject: [Python-Dev] tuple[int] optimization In-Reply-To: <1311466703.3001.12.camel@localhost.localdomain> References: <1311462787.2225.5.camel@durian> <20110724012755.32bf6a8f@pitrou.net> <1311466703.3001.12.camel@localhost.localdomain> Message-ID: <645E4C22-4532-42A0-A87C-B65D8EB49B7B@gmail.com> On Jul 23, 2011, at 5:18 PM, Antoine Pitrou wrote: > > My point is that on non-trivial benchmarks, the savings are almost zero. That could be said of any optimization in Python. Typical Python scripts exercise many features at time, so any one optimization by itself if almost useless. Collectively however, we've gotten nice speed-ups on real programs. Raymond From mail at kerrickstaley.com Sun Jul 24 04:48:43 2011 From: mail at kerrickstaley.com (Kerrick Staley) Date: Sat, 23 Jul 2011 21:48:43 -0500 Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the /usr/bin/python2 symlink upstream) In-Reply-To: References: Message-ID: I put up a tracker issue at http://bugs.python.org/issue12627 There are patches for 2.7 as well as tip, but they only fix the Makefiles; no changes are done to documentation. Also, Ned, it appears that Python 2.7 doesn't install the Idle or PyDoc binaries (grep the 2.7 Makefile to see what I mean), so I didn't make any changes related to idle2 or pydoc2. -Kerrick Staley On Wed, Jul 20, 2011 at 3:28 AM, Nick Coghlan wrote: > > On Wed, Jul 20, 2011 at 4:53 PM, Kerrick Staley wrote: > > Nick, can you please apply the patch (will be sent in the following > > email) to the PEP SVN as soon as we get the hard-link issue is figured > > out? Alternatively, could you provide me write access to just the > > pep-0394.txt file so I can update it myself? Thanks. > > Done: http://www.python.org/dev/peps/pep-0394/ > > Main thing needed now is a tracker issue to reference (ideally with a > first cut at a patch) and python-dev's collective blessing to take it > out of Draft status. > > Cheers, > Nick. > > -- > Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From eliben at gmail.com Sun Jul 24 05:35:56 2011 From: eliben at gmail.com (Eli Bendersky) Date: Sun, 24 Jul 2011 06:35:56 +0300 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions Message-ID: Some background: I'm working (on and off) on issue 11015 - documenting the public functions in test.support Some of the functions in test.support (for example unlink, rmtree) simply shadow existing & popular stdlib functions, with the aim of swallowing the exceptions these may throw. This is confusing, IMHO. For example, grepping 'unlink' on Lib/test/test_*.py files doesn't say much about which unlink is being used. A couple of options to handle this are: 1. Remove these functions altogether, trying to use existing constructs (such as the ignore_errors parameter in rmtree). 2. Adapt a naming convention for such functions, for instance rmtree_silent and unlink_silent (or a similar convention, if one exists) Opinions? Eli From ericsnowcurrently at gmail.com Sun Jul 24 06:53:38 2011 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Sat, 23 Jul 2011 22:53:38 -0600 Subject: [Python-Dev] is sys.modules not meant to be replaced? Message-ID: The documentation[1] doesn't say, but the implementation of the imp module makes me wonder if sys.modules was not meant to be replaceable. No doubt this has to do with its tie to the interpreter's modules dict. I ran into this doing "sys.modules = sys.modules.copy()" to protect the actual sys.modules dict during some import related test cases. If the modules I imported were extension modules it broke. So, is sys.modules not meant to be open to re-binding? -eric p.s. I tried opening a tracker ticket on this, but it wouldn't go through. I'll try again later. [1] http://docs.python.org/library/sys.html#sys.modules From benjamin at python.org Sun Jul 24 06:55:38 2011 From: benjamin at python.org (Benjamin Peterson) Date: Sat, 23 Jul 2011 23:55:38 -0500 Subject: [Python-Dev] is sys.modules not meant to be replaced? In-Reply-To: References: Message-ID: 2011/7/23 Eric Snow : > The documentation[1] doesn't say, but the implementation of the imp > module makes me wonder if sys.modules was not meant to be replaceable. > ?No doubt this has to do with its tie to the interpreter's modules > dict. ?I ran into this doing "sys.modules = sys.modules.copy()" to > protect the actual sys.modules dict during some import related test > cases. ?If the modules I imported were extension modules it broke. > > So, is sys.modules not meant to be open to re-binding? Not any more or less than other global mutable objects. You can expect other code to be holding on to old references. -- Regards, Benjamin From ericsnowcurrently at gmail.com Sun Jul 24 07:05:24 2011 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Sat, 23 Jul 2011 23:05:24 -0600 Subject: [Python-Dev] is sys.modules not meant to be replaced? In-Reply-To: References: Message-ID: On Sat, Jul 23, 2011 at 10:55 PM, Benjamin Peterson wrote: > 2011/7/23 Eric Snow : >> The documentation[1] doesn't say, but the implementation of the imp >> module makes me wonder if sys.modules was not meant to be replaceable. >> ?No doubt this has to do with its tie to the interpreter's modules >> dict. ?I ran into this doing "sys.modules = sys.modules.copy()" to >> protect the actual sys.modules dict during some import related test >> cases. ?If the modules I imported were extension modules it broke. >> >> So, is sys.modules not meant to be open to re-binding? > > Not any more or less than other global mutable objects. You can expect > other code to be holding on to old references. But, isn't sys.modules a little different because other modules (at least the imp module) don't use it. From what I understand the interpreter object's modules dict (to which sys.modules is initially bound) is used directly instead. So rebinding sys.modules causes a disconnect. That's why I am wondering if sys.modules is not meant to be rebound. Are there other objects in the interpreter state that are exposed in sys that would have the same problem? -eric > > > -- > Regards, > Benjamin > From benjamin at python.org Sun Jul 24 07:09:13 2011 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 24 Jul 2011 00:09:13 -0500 Subject: [Python-Dev] is sys.modules not meant to be replaced? In-Reply-To: References: Message-ID: 2011/7/24 Eric Snow : > On Sat, Jul 23, 2011 at 10:55 PM, Benjamin Peterson wrote: >> 2011/7/23 Eric Snow : >>> The documentation[1] doesn't say, but the implementation of the imp >>> module makes me wonder if sys.modules was not meant to be replaceable. >>> ?No doubt this has to do with its tie to the interpreter's modules >>> dict. ?I ran into this doing "sys.modules = sys.modules.copy()" to >>> protect the actual sys.modules dict during some import related test >>> cases. ?If the modules I imported were extension modules it broke. >>> >>> So, is sys.modules not meant to be open to re-binding? >> >> Not any more or less than other global mutable objects. You can expect >> other code to be holding on to old references. > > But, isn't sys.modules a little different because other modules (at > least the imp module) don't use it. ?From what I understand the > interpreter object's modules dict (to which sys.modules is initially > bound) is used directly instead. ?So rebinding sys.modules causes a > disconnect. ?That's why I am wondering if sys.modules is not meant to > be rebound. Sure. I'm not sure what point you're trying to make, though. > > Are there other objects in the interpreter state that are exposed in > sys that would have the same problem? Is it problematic? -- Regards, Benjamin From v+python at g.nevcal.com Sun Jul 24 08:29:55 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Sat, 23 Jul 2011 23:29:55 -0700 Subject: [Python-Dev] 3.2.1 encoding surprise In-Reply-To: References: <4E248536.30006@g.nevcal.com> <4E249960.9000109@g.nevcal.com> <4E24AE66.5030305@g.nevcal.com> <4E24D83B.8040100@g.nevcal.com> <4E269D46.1010806@g.nevcal.com> <4E274C0F.40400@g.nevcal.com> <4E276795.9050404@gmail.com> <4E276C7C.1020605@g.nevcal.com> <4E276EBF.6000509@gmail.com> <4E28B016.5050500@g.nevcal.com> <4E28C7D1.3010002@skippinet.com.au> <4E2A3983.2060901@g.nevcal.com> Message-ID: <4E2BBBE3.2050304@g.nevcal.com> On 7/23/2011 5:34 AM, Vinay Sajip wrote: > Glenn Linderman g.nevcal.com> writes: > > I aim to update the launcher downloads Real Soon Now. Has fixed my problem with not having a local py.ini file, and now is picking up python=3 from the py.ini coresident to the py.exe. Thanks, Mark & Vinay. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Sun Jul 24 08:54:33 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 24 Jul 2011 16:54:33 +1000 Subject: [Python-Dev] is sys.modules not meant to be replaced? In-Reply-To: References: Message-ID: On Sun, Jul 24, 2011 at 3:05 PM, Eric Snow wrote: > Are there other objects in the interpreter state that are exposed in > sys that would have the same problem? Rebinding (rather than mutating) any global state in any module is always dubious due to the potential for direct references to the previous value. (This is a large part of the reason why imp.reload() often doesn't work in practice) sys.modules, sys.path, sys.argv, sys.stdout, et al are all subject to that caveat. For mutable state, the onus is typically on the developer performing the substitution to decide whether or not those consequences are acceptable (usually, they're not, hence you get idioms like "sys.path[:] = saved_copy" to revert sys.path to a saved value). For immutable state (such as sys.stdout), where modification in place isn't possible, the obligation often goes the other way, with developers deliberately avoiding cached references in order to correctly react to runtime changes. sys.modules is by far the worst case though, since dropping modules out of that cache is only safe for modules that correctly support imp.reload(). This is not the case for any module that mutates other global state (or is otherwise referenced from other modules), nor does it work properly for extension modules. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From petri at digip.org Sun Jul 24 19:51:37 2011 From: petri at digip.org (Petri Lehtinen) Date: Sun, 24 Jul 2011 20:51:37 +0300 Subject: [Python-Dev] is sys.modules not meant to be replaced? In-Reply-To: References: Message-ID: <20110724175137.GA1624@ihaa> Eric Snow wrote: > p.s. I tried opening a tracker ticket on this, but it wouldn't go > through. I'll try again later. Adding the following line to /etc/hosts makes the bug tracker fast when python.org is down: 127.0.0.1 python.org This is because bugs.python.org works fine, but it links to CSS files and images that are on python.org. Forcing DNS lookups for python.org to return localhost makes the requests for these resources fail fast. A more long-term solution would be to duplicate all the resources to bugs.python.org... Petri From solipsis at pitrou.net Sun Jul 24 23:16:55 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 24 Jul 2011 23:16:55 +0200 Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the /usr/bin/python2 symlink upstream) References: Message-ID: <20110724231655.0b69d36a@pitrou.net> On Wed, 20 Jul 2011 01:53:09 -0500 Kerrick Staley wrote: > On Mon, Jul 18, 2011 at 3:03 AM, Ned Deily wrote: > > I think adding the requirement to mandate hard link vs soft link usage > > is an unnecessary and unwarranted attempt at optimization. For > > instance, IIRC, the OS X installers don't use any hard links: that may > > complicate the install, plus hard links on OS X HFS* file systems are a > > bit of a kludge and not necessarily more efficient than symlinks. It's > > not a big deal but perhaps the wording should be changed to make a > > suggestion about hard links vs syminks rather than mandate which should > > be used. > > Ah, OK. The wording's been changed so that symbolic links will be > installed on Mac OS X and hard links elsewhere (although maybe > symbolic links are also better on certain other platforms; I'm not > sure). > > I do think that specific instructions must be given (rather than just > a suggestion) because it's indicating what must be done to CPython. > The instructions *should* be as close as possible to what the > installer already does, but I'm not entirely sure what the installer > does by default, and the hard-link recommendation was based off a > cursory inspection of my own system, so further input from yourself > and the rest of python-dev would be appreciated. I think the recommendation should be symbolic links for all systems. Hard links are generally harder to discover, while it is trivial to find out that a given file is a symbolink link, and to which other file. The optimization is probably not useful in the real world (our executables are relatively small, and people worried about a couple of megabytes can always go for the shared library option). Regards Antoine. From solipsis at pitrou.net Sun Jul 24 23:21:06 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 24 Jul 2011 23:21:06 +0200 Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the /usr/bin/python2 symlink upstream) References: <20110724231655.0b69d36a@pitrou.net> Message-ID: <20110724232106.53161528@pitrou.net> On Sun, 24 Jul 2011 23:16:55 +0200 Antoine Pitrou wrote: > > I think the recommendation should be symbolic links for all systems. > Hard links are generally harder to discover, while it is trivial to > find out that a given file is a symbolink link, and to which other file. > The optimization is probably not useful in the real world (our > executables are relatively small, and people worried about a couple of > megabytes can always go for the shared library option). Oh, I don't even know why I thought hard links would be a size optimization anyway. I guess I thought "file copy" instead of "symbolic link". Regards Antoine. From victor.stinner at haypocalc.com Mon Jul 25 01:56:32 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Mon, 25 Jul 2011 01:56:32 +0200 Subject: [Python-Dev] Comments of the PEP 3151 Message-ID: <4E2CB130.7060701@haypocalc.com> Arguments in favor of specific errors ------------------------------------- Using specific errors avoids the need of "import errno". The import is sometimes done in the except block, which may raise a new error (and may be slow). I like specific exceptions because it avoids the need of re-raise unwanted exception. It makes the code more readable: we ignore explicitly specific errors instead of re-raised unwanted exceptions and implicitly ignore some errors. For example: try: os.mkdir(dir) except OSError as err: if err.errno != errno.EEXIST: raise becomes try: os.mkdir(dir) except FileExistsError: pass By the way, is it faster to not handle and than re-raise unwanted exceptions? (I don't like tests like "err.errno != errno.EEXIST", it's confusing to me. It uses errno identifier twice: once as an attribute, once as a module name.) Make specific errors builtins ----------------------------- I agree that it's better to make specific errors builtins. If we move exceptions to more specific modules, like io and socket, you have to load these modules to catch an exception and remember in which module the exception is. An advantage of builtin specific exceptions is to avoid an import (import errno). Making them builtin may also avoid a bootstrap issue (catch a specific error at startup). Choice of the specific errors ----------------------------- I don't understand how Antoine decided which errno should have an exception or not. For example, EINTR and EPIPE are handled in many modules of the standard library but don't have their exception, whereas EISDIR (IsADirectoryError) and ENOTDIR (NotADirectoryError) are very rare (EISDIR is never handled in the standard library) but have their exception. If we add EINTR, I don't know if it's better to add it to BlockingIOError or to create a new exception (InterruptError?). Another good candidate is EINVAL. Would it be possible to have an (exhaustive?) list of errno with their popularity? At least, their popularity in the Python standard library? Raise a specific error ---------------------- Does raising a specific error (e.g. raise FileNotFound()) set errno and message attributes (to something different than None)? If we provide an error message error: should it be localized? The description of FileDescriptorError tells about the "default error message". It we use a localized message, it's not possible to preallocate or cache instances. I don't see how we could choose a default errno value, because some exceptions handle multiple errno. For example, PermissionError.errno can be EACCES or EPERM. Do specific errors slows down raising exceptions? Do raising an IOError (or a "legacy" error) use a O(1) lookup to choose the exception class? PermissionError --------------- EACCES and EPERM have a different meaning. Even that they are very similar, we might provide two different exceptions. EPERM is only used once in the Python stdlib, so we might only provide AccesError. On Linux, EPERM usually indicates an operation requiring root priviledge. EACCES can be related to filesystem permissions (read-only, user is not allowed to write, etc.) or can be an mmap error. Deprecation ----------- Because IOError, OSError, select.error, etc. are well known exceptions, I'm not in favor of removing them from Python 3. It would make porting from Python 2 worse. If we don't remove them, they should not be deprecated. I'm in favor of adding a note in the documentation of all legacy exceptions to advice to use IOError or specific exceptions instead. I suppose that these notes will have to indicate a Python version. Test the implementation ----------------------- Did someone test Blender, Django or another major applications on the implementation of the PEP 3151? -1 on FileSystemError --------------------- I'm not sure that we need FileSystemError or ConnectionError. Careless code will use IOError, whereas careful code will use an explicit list like (ConnectionAbortedError, ConnectionRefusedError, ConnectionResetError). If we remove IsADirectoryError and NotADirectoryError, FileSystemError only contains FileExistsError and FileNotFoundError. I don't think that these errors can occur on a same function. For example, rmdir() only raises ENOTDIR. I only see one advantage of FileSystemError: it doesn't handle FileDescriptorError. Advice usage of FileSystemError would avoid to hide real bugs like FileDescriptorError. I don't really care of ConnectionError. Anyway, FileSystemError and ConnectionError can be added later if needed. WindowsError and winerror attribute ----------------------------------- I don't like the idea of having a winerror attribute platforms other than Windows. Why not using hasattr(err, "winerror") instead? WindowsError is already specific to Windows. I don't see any usage of the winerror attribute in the Python stdlib. shutil module uses: try: WindowsError except NameError: WindowsError = None (...) try: copystat(src, dst) except OSError as why: if WindowsError is not None and isinstance(why, WindowsError): # Copying file access times may fail on Windows pass else: errors.extend((src, dst, str(why))) Misc ---- Lock.acquire() doesn't raise an exception on timeout: just remove "(for example in Lock.acquire())". Should FileNotFound handle ENODEV? (see test_ossaudiodev) Victor From solipsis at pitrou.net Mon Jul 25 02:24:30 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 25 Jul 2011 02:24:30 +0200 Subject: [Python-Dev] Comments of the PEP 3151 References: <4E2CB130.7060701@haypocalc.com> Message-ID: <20110725022430.0ee6f5b4@pitrou.net> Hello, > By the way, is it faster to not handle and than re-raise unwanted > exceptions? You mean "is it faster to not handle than re-raise unwanted exceptions?". It probably is, yes. > I don't understand how Antoine decided which errno should have an > exception or not. Mostly experience with the stdlib combined with intuition. The list is not cast in stone. > If we add EINTR, I don't know if it's better to add it to > BlockingIOError or to create a new exception (InterruptError?). EINTR is not related to non-blocking I/O, so it shouldn't map to BlockingIOError. > Would it be possible to have an (exhaustive?) list of errno with their > popularity? At least, their popularity in the Python standard library? How do you suggest to do that? > Does raising a specific error (e.g. raise FileNotFound()) set errno and > message attributes (to something different than None)? No, it doesn't. "errno" should IMO only be set if it's truely returned by a system routine. As for the message, like with every other exception it should be supplied by the developer. > Do specific errors slows down raising exceptions? Do raising an IOError > (or a "legacy" error) use a O(1) lookup to choose the exception class? It uses a dict. You could run some benchmarks if you want, but I doubt it makes much sense to worry about that. > EACCES and EPERM have a different meaning. Even that they are very > similar, we might provide two different exceptions. EPERM is only used > once in the Python stdlib, so we might only provide AccesError. > > On Linux, EPERM usually indicates an operation requiring root > priviledge. EACCES can be related to filesystem permissions (read-only, > user is not allowed to write, etc.) or can be an mmap error. I'd have to research that a bit more. Perhaps EACCES can be mapped to AccessError and EPERM to PermissionError, but I think the proximity of the two concepts will make things confusing, for little benefit. > Because IOError, OSError, select.error, etc. are well known exceptions, > I'm not in favor of removing them from Python 3. It would make porting > from Python 2 worse. If we don't remove them, they should not be > deprecated. Ok. Does anyone else have an opinion on deprecations? (not deprecating them means less work for me, anyway) > Test the implementation > ----------------------- > > Did someone test Blender, Django or another major applications on the > implementation of the PEP 3151? Does Django have a working Python 3 port yet? > WindowsError and winerror attribute > ----------------------------------- > > I don't like the idea of having a winerror attribute platforms other > than Windows. Well, there isn't, as you can see in the tests (test_windows_error in Lib/test/test_pep3151.py). > shutil module uses: > > (...) > > try: > copystat(src, dst) > except OSError as why: > if WindowsError is not None and isinstance(why, WindowsError): > # Copying file access times may fail on Windows > pass That's the kind of code the PEP hopes to discourage :) > Lock.acquire() doesn't raise an exception on timeout: just remove "(for > example in Lock.acquire())". Oops, will fix. > Should FileNotFound handle ENODEV? (see test_ossaudiodev) That's not really the same thing. For example, mmap says: [ENODEV] The fildes argument refers to a file whose type is not supported by mmap(). (http://www.opengroup.org/sud/sud1/xsh/mmap.htm) Regards Antoine. From stephen at xemacs.org Mon Jul 25 05:25:21 2011 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 25 Jul 2011 12:25:21 +0900 Subject: [Python-Dev] Comments of the PEP 3151 In-Reply-To: <20110725022430.0ee6f5b4@pitrou.net> References: <4E2CB130.7060701@haypocalc.com> <20110725022430.0ee6f5b4@pitrou.net> Message-ID: <87wrf7qbxa.fsf@uwakimon.sk.tsukuba.ac.jp> Antoine Pitrou writes: > > Did someone test Blender, Django or another major applications on the > > implementation of the PEP 3151? > > Does Django have a working Python 3 port yet? Define "working". Martin ported Django to Python 3 years ago as proof of concept, but never claimed it was ready for production use or as the main line of development. From ericsnowcurrently at gmail.com Mon Jul 25 06:50:47 2011 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Sun, 24 Jul 2011 22:50:47 -0600 Subject: [Python-Dev] is sys.modules not meant to be replaced? In-Reply-To: References: Message-ID: On Sun, Jul 24, 2011 at 12:54 AM, Nick Coghlan wrote: > On Sun, Jul 24, 2011 at 3:05 PM, Eric Snow wrote: >> Are there other objects in the interpreter state that are exposed in >> sys that would have the same problem? > > Rebinding (rather than mutating) any global state in any module is > always dubious due to the potential for direct references to the > previous value. (This is a large part of the reason why imp.reload() > often doesn't work in practice) > > sys.modules, sys.path, sys.argv, sys.stdout, et al are all subject to > that caveat. For mutable state, the onus is typically on the developer > performing the substitution to decide whether or not those > consequences are acceptable (usually, they're not, hence you get > idioms like "sys.path[:] = saved_copy" to revert sys.path to a saved > value). For immutable state (such as sys.stdout), where modification > in place isn't possible, the obligation often goes the other way, with > developers deliberately avoiding cached references in order to > correctly react to runtime changes. I agree with what you are saying wholeheartedly, but still think there is something fishy with the way that sys.modules works. I'll take it from here to a tracker issue though. :) -eric > sys.modules is by far the worst case though, since dropping modules > out of that cache is only safe for modules that correctly support > imp.reload(). This is not the case for any module that mutates other > global state (or is otherwise referenced from other modules), nor does > it work properly for extension modules. > > Cheers, > Nick. > > -- > Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia > From ncoghlan at gmail.com Mon Jul 25 07:28:47 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 25 Jul 2011 15:28:47 +1000 Subject: [Python-Dev] Comments of the PEP 3151 In-Reply-To: <4E2CB130.7060701@haypocalc.com> References: <4E2CB130.7060701@haypocalc.com> Message-ID: On Mon, Jul 25, 2011 at 9:56 AM, Victor Stinner wrote: > By the way, is it faster to not handle and than re-raise unwanted > exceptions? Yes, but probably not that much faster given the overhead of instantiating the exception and unwinding the stack in the first place. > Choice of the specific errors > ----------------------------- > > I don't understand how Antoine decided which errno should have an > exception or not. > > For example, EINTR and EPIPE are handled in many modules of the standard > library but don't have their exception, whereas EISDIR > (IsADirectoryError) and ENOTDIR (NotADirectoryError) are very rare > (EISDIR is never handled in the standard library) but have their > exception. We do tend to call isdir() fairly often, though. Part of the appeal of isdir() is the ugliness of using EAFP when you have to do explicit errno checks. > If we add EINTR, I don't know if it's better to add it to > BlockingIOError or to create a new exception (InterruptError?). InterruptedError seems like a reasonable candidate for addition to me - catch and retry in that case is something developers are likely to want to do. Perhaps EPIPE should map to FileDescriptorError along with EBADF, with different messages based on the exact error code? Potentially renamed to DescriptorStateError? I'm not sure of the reason for singling out EBADF for special treatment in this case - there are several other errno values that indicate a descriptor is no longer in a usable state (ECONSHUTDOWN would be another one, in the same vein as EPIPE). To be honest though, what's the use case for *catching* FileDescriptorError without catching IOError in general? Perhaps this one should be dropped entirely, to be handled by broadly catching IOError? > Another good candidate is EINVAL. As with EBADF, I'm having trouble visualising a clear use case for handling things like EINVAL and EOPNOTSUP differently from other kinds of IO errors. > Would it be possible to have an (exhaustive?) list of errno with their > popularity? At least, their popularity in the Python standard library? Probably, but the stdlib is more a *generator* of low level exceptions, so our code likely isn't representative enough to get a decent set of numbers. > If we provide an error message error: should it be localized? The > description of FileDescriptorError tells about the "default error > message". It we use a localized message, it's not possible to > preallocate or cache instances. We don't localize anything else, so there's no reason to start here. > PermissionError > --------------- > > EACCES and EPERM have a different meaning. Even that they are very > similar, we might provide two different exceptions. EPERM is only used > once in the Python stdlib, so we might only provide AccesError. > > On Linux, EPERM usually indicates an operation requiring root > priviledge. ?EACCES can be related to filesystem permissions (read-only, > user is not allowed to write, etc.) or can be an mmap error. Code that cares can still fall back to exc.errno == EPERM. I don't think we'd be doing anyone any favours by exposing subtle distinctions like this at the Python level. > Deprecation > ----------- > > Because IOError, OSError, select.error, etc. are well known exceptions, > I'm not in favor of removing them from Python 3. It would make porting > from Python 2 worse. If we don't remove them, they should not be > deprecated. > > I'm in favor of adding a note in the documentation of all legacy > exceptions to advice to use IOError or specific exceptions instead. I > suppose that these notes will have to indicate a Python version. +1 for grandfathering in the old exception names, but documenting the recommended alternatives as of 3.3. > -1 on FileSystemError > --------------------- > > I'm not sure that we need FileSystemError or ConnectionError. Careless > code will use IOError, whereas careful code will use an explicit list > like (ConnectionAbortedError, ConnectionRefusedError, > ConnectionResetError). > > If we remove IsADirectoryError and NotADirectoryError, FileSystemError > only contains FileExistsError and FileNotFoundError. I don't think that > these errors can occur on a same function. For example, rmdir() only > raises ENOTDIR. > > I only see one advantage of FileSystemError: it doesn't handle > FileDescriptorError. Advice usage of FileSystemError would avoid to hide > real bugs like FileDescriptorError. And that's precisely why FileSystemError is worthwhile - to separate out when the FS is objecting, rather than there being something wrong with the internal application state or when a previously valid descriptor has become unusable for some reason. It may also be reasonable to return a new DeviceNotAvailableError for ENODEV and EBUSY (as a new FileSystemError subclass). > I don't really care of ConnectionError. Anyway, FileSystemError and > ConnectionError can be added later if needed. But the use case for grouping them is quite obvious - there's is plenty of application code that will want to handle them in a particular way, while allowing other kinds of IOError to propagate further up the stack. Baking this into the exception heirarchy is far more future proof than people making their own explicit lists. There may be some error codes that we choose to map to these generic errors, even if we don't give them their own exception types at this point (e.g. ECONSHUTDOWN could map directly to ConnectionError). > Should FileNotFound handle ENODEV? (see test_ossaudiodev) See above for my suggestion of a specific DeviceNotAvailable exception. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Mon Jul 25 07:33:09 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 25 Jul 2011 15:33:09 +1000 Subject: [Python-Dev] is sys.modules not meant to be replaced? In-Reply-To: References: Message-ID: On Mon, Jul 25, 2011 at 2:50 PM, Eric Snow wrote: > I agree with what you are saying wholeheartedly, but still think there > is something fishy with the way that sys.modules works. ?I'll take it > from here to a tracker issue though. :) Yup - the import system has a whole mess of interrelated global state that really needs to be on a class that can use descriptors to correctly invalidate caches when attributes are mutated or rebound. The ImportEngine GSoC project is the first step along the long, winding path towards doing something about that :) Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From martin at v.loewis.de Mon Jul 25 08:26:09 2011 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 25 Jul 2011 08:26:09 +0200 Subject: [Python-Dev] is sys.modules not meant to be replaced? In-Reply-To: References: Message-ID: <4E2D0C81.7070509@v.loewis.de> > I agree with what you are saying wholeheartedly, but still think there > is something fishy with the way that sys.modules works. I'll take it > from here to a tracker issue though. :) Well, there is a short answer to you question: sys.modules is *not* meant to be replaced. Doing so will result in undefined behavior. So there is nothing fishy about it. Regards, Martin From solipsis at pitrou.net Mon Jul 25 12:43:51 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 25 Jul 2011 12:43:51 +0200 Subject: [Python-Dev] Comments of the PEP 3151 References: <4E2CB130.7060701@haypocalc.com> Message-ID: <20110725124351.15529bc7@pitrou.net> On Mon, 25 Jul 2011 15:28:47 +1000 Nick Coghlan wrote: > > > If we add EINTR, I don't know if it's better to add it to > > BlockingIOError or to create a new exception (InterruptError?). > > InterruptedError seems like a reasonable candidate for addition to me > - catch and retry in that case is something developers are likely to > want to do. Ok, let's call it InterruptError then. InterruptedError sounds like the error was interrupted ;) > Perhaps EPIPE should map to FileDescriptorError along with EBADF, with > different messages based on the exact error code? Potentially renamed > to DescriptorStateError? I'd rather have a separate BrokenPipeError for EPIPE. EBADF and EPIPE are really quite different; EPIPE indicates that the other end of the pipe closed it, while EBADF generally points to a programming error (you are giving an invalid file descriptor to a system routine). > To be honest though, what's the use case for *catching* > FileDescriptorError without catching IOError in general? Perhaps this > one should be dropped entirely, to be handled by broadly catching > IOError? Good point indeed. > > I'm in favor of adding a note in the documentation of all legacy > > exceptions to advice to use IOError or specific exceptions instead. I > > suppose that these notes will have to indicate a Python version. > > +1 for grandfathering in the old exception names, but documenting the > recommended alternatives as of 3.3. Ok. > It may also be reasonable to return a new DeviceNotAvailableError for > ENODEV and EBUSY (as a new FileSystemError subclass). EBUSY is really quite different from these other errors. It is triggered by runtime protections in the OS (can't destroy some object that is in use, see for example in pthread_cond_destroy: http://pubs.opengroup.org/onlinepubs/009604499/functions/pthread_cond_destroy.html), rather than indication some misassumption about the filesystem layout. As for ENODEV, I'll have to take a look. > There may be some error codes that we choose to map to these generic > errors, even if we don't give them their own exception types at this > point (e.g. ECONSHUTDOWN could map directly to ConnectionError). Good point as well. Regards Antoine. From v+python at g.nevcal.com Mon Jul 25 18:17:36 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Mon, 25 Jul 2011 09:17:36 -0700 Subject: [Python-Dev] Comments of the PEP 3151 In-Reply-To: <20110725124351.15529bc7@pitrou.net> References: <4E2CB130.7060701@haypocalc.com> <20110725124351.15529bc7@pitrou.net> Message-ID: <4E2D9720.8040901@g.nevcal.com> On 7/25/2011 3:43 AM, Antoine Pitrou wrote: > On Mon, 25 Jul 2011 15:28:47 +1000 > Nick Coghlan wrote: >> > >>> > > If we add EINTR, I don't know if it's better to add it to >>> > > BlockingIOError or to create a new exception (InterruptError?). >> > >> > InterruptedError seems like a reasonable candidate for addition to me >> > - catch and retry in that case is something developers are likely to >> > want to do. > Ok, let's call it InterruptError then. InterruptedError sounds like the > error was interrupted;) > Sorry, no. "InterruptError" sounds too much like a CPU interrupt signal, which the error is not. "InterruptedError" is OK by me, I don't see the confusion you do. But maybe "InterruptedOperationError" would be the most clear. Way too long, of course, so maybe "InterruptedAPIError" or "InterruptedOpError" or "EINTRError" in my order of preference. -------------- next part -------------- An HTML attachment was scrubbed... URL: From victor.stinner at haypocalc.com Mon Jul 25 18:33:49 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Mon, 25 Jul 2011 18:33:49 +0200 Subject: [Python-Dev] Comments of the PEP 3151 In-Reply-To: <20110725022430.0ee6f5b4@pitrou.net> References: <4E2CB130.7060701@haypocalc.com> <20110725022430.0ee6f5b4@pitrou.net> Message-ID: <4E2D9AED.8090404@haypocalc.com> On 25/07/2011 02:24, Antoine Pitrou wrote: > > Hello, > >> By the way, is it faster to not handle and than re-raise unwanted >> exceptions? > > You mean "is it faster to not handle than re-raise unwanted > exceptions?". It probably is, yes. I asked if: try: ... except FileNotFound: pass is faster than: try: ... except IOError as err: if err.errno != errno.ENOENT: raise else: pass f an IOError with errno != ENOENT is raised. >> I don't understand how Antoine decided which errno should have an >> exception or not. > > Mostly experience with the stdlib combined with intuition. The list is > not cast in stone. "The list is not cast in stone." : ok :-) >> If we add EINTR, I don't know if it's better to add it to >> BlockingIOError or to create a new exception (InterruptError?). > > EINTR is not related to non-blocking I/O, so it shouldn't map to > BlockingIOError. Ok, so let add InterruptError. >> Would it be possible to have an (exhaustive?) list of errno with their >> popularity? At least, their popularity in the Python standard library? > > How do you suggest to do that? Use the list of all errno from the Appendix A, and then count the number of occurence in the Python source code (excluding the test suite). You can for example using a shell for that: $ grep -h -c EINVAL $(find -name "*.py"|grep -v Lib/test)|grep -v '^0$' 1 1 2 These numbers give an approximation of the popularity of the different errno. >> Does raising a specific error (e.g. raise FileNotFound()) set errno and >> message attributes (to something different than None)? > > No, ... As for the message, like with every other exception > it should be supplied by the developer. Ok, I agree. >> Do specific errors slows down raising exceptions? Do raising an IOError >> (or a "legacy" error) use a O(1) lookup to choose the exception class? > > It uses a dict. You could run some benchmarks if you want, but I doubt > it makes much sense to worry about that. Ok, a dict should be fast. >> EACCES and EPERM have a different meaning. Even that they are very >> similar, we might provide two different exceptions. EPERM is only used >> once in the Python stdlib, so we might only provide AccesError. >> >> On Linux, EPERM usually indicates an operation requiring root >> priviledge. EACCES can be related to filesystem permissions (read-only, >> user is not allowed to write, etc.) or can be an mmap error. > > I'd have to research that a bit more. Perhaps EACCES can be mapped to > AccessError and EPERM to PermissionError, but I think the proximity of > the two concepts will make things confusing, for little benefit. You are probable right. >> WindowsError and winerror attribute >> ----------------------------------- >> >> I don't like the idea of having a winerror attribute platforms other >> than Windows. > > Well, there isn't, as you can see in the tests Oh, it's not clear in the PEP. >> Should FileNotFound handle ENODEV? (see test_ossaudiodev) > > That's not really the same thing. Ok. Victor From ethan at stoneleaf.us Mon Jul 25 20:44:29 2011 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 25 Jul 2011 11:44:29 -0700 Subject: [Python-Dev] Comments of the PEP 3151 In-Reply-To: <4E2D9720.8040901@g.nevcal.com> References: <4E2CB130.7060701@haypocalc.com> <20110725124351.15529bc7@pitrou.net> <4E2D9720.8040901@g.nevcal.com> Message-ID: <4E2DB98D.20106@stoneleaf.us> Glenn Linderman wrote: > On 7/25/2011 3:43 AM, Antoine Pitrou wrote: >> On Mon, 25 Jul 2011 15:28:47 +1000 >> Nick Coghlan wrote: >>> > >>>> > > If we add EINTR, I don't know if it's better to add it to >>>> > > BlockingIOError or to create a new exception (InterruptError?). >>> > >>> > InterruptedError seems like a reasonable candidate for addition to me >>> > - catch and retry in that case is something developers are likely to >>> > want to do. >> Ok, let's call it InterruptError then. InterruptedError sounds like the >> error was interrupted ;) >> > > Sorry, no. "InterruptError" sounds too much like a CPU interrupt > signal, which the error is not. It does, a bit -- but is that something we need to worry about at the Python level? Seems to me we should have the names make sense for Python, and not worry about what assembly, C, Pascal, Perl, or names might mean for them. > "InterruptedError" is OK by me, I don't > see the confusion you do. But maybe "InterruptedOperationError" would > be the most clear. Way too long, of course, so maybe > "InterruptedAPIError" or "InterruptedOpError" or "EINTRError" in my > order of preference. Please not that last one! ;) I prefer InterruptedError or InterruptError. ~Ethan~ From brett at python.org Tue Jul 26 02:34:33 2011 From: brett at python.org (Brett Cannon) Date: Mon, 25 Jul 2011 17:34:33 -0700 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: References: Message-ID: On Sat, Jul 23, 2011 at 20:35, Eli Bendersky wrote: > Some background: I'm working (on and off) on issue 11015 - documenting > the public functions in test.support > > Some of the functions in test.support (for example unlink, rmtree) > simply shadow existing & popular stdlib functions, with the aim of > swallowing the exceptions these may throw. This is confusing, IMHO. > For example, grepping 'unlink' on Lib/test/test_*.py files doesn't say > much about which unlink is being used. > > A couple of options to handle this are: > > 1. Remove these functions altogether, trying to use existing > constructs (such as the ignore_errors parameter in rmtree). > 2. Adapt a naming convention for such functions, for instance > rmtree_silent and unlink_silent (or a similar convention, if one > exists) > > Opinions? > The mere fact that these functions exist in a different module suggests different semantics from those found in other places in the stdlib. I don't think they should be renamed simply because some code imports the functions directly instead of the module itself (suggesting they should be doing the latter over the former). Now if the functions are redundant that's something else entirely and removal should be fine. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Tue Jul 26 02:36:11 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 26 Jul 2011 10:36:11 +1000 Subject: [Python-Dev] Comments of the PEP 3151 In-Reply-To: <4E2DB98D.20106@stoneleaf.us> References: <4E2CB130.7060701@haypocalc.com> <20110725124351.15529bc7@pitrou.net> <4E2D9720.8040901@g.nevcal.com> <4E2DB98D.20106@stoneleaf.us> Message-ID: On Tue, Jul 26, 2011 at 4:44 AM, Ethan Furman wrote: > Glenn Linderman wrote: >> >> ?On 7/25/2011 3:43 AM, Antoine Pitrou wrote: >>> Ok, let's call it InterruptError then. InterruptedError sounds like the >>> error was interrupted ;) >>> >> >> Sorry, no. ?"InterruptError" sounds too much like a CPU interrupt signal, >> which the error is not. > > It does, a bit -- but is that something we need to worry about at the Python > level? ?Seems to me we should have the names make sense for Python, and not > worry about what assembly, C, Pascal, Perl, or > names might mean for them. Like Glenn, I believe "InterruptError" sounds wrong - the event being reported is that a system call was interrupted for some unknown reason, not necessarily that the process received an interrupt. 'Interrupt' in computer science requires context to distinguish between the verb and noun forms, and an exception name doesn't provide that context. 'Interrupted' forces interpretation as the past tense of the verb form, which is the meaning we want. If the subject of the verb feels too ambiguous then I'd prefer to switch to the explicit adjective form 'InterruptedCallError' rather than allowing the verb/noun ambiguity. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From andrew at bemusement.org Tue Jul 26 02:26:44 2011 From: andrew at bemusement.org (Andrew Bennetts) Date: Tue, 26 Jul 2011 10:26:44 +1000 Subject: [Python-Dev] Comments of the PEP 3151 In-Reply-To: <4E2DB98D.20106@stoneleaf.us> References: <4E2CB130.7060701@haypocalc.com> <20110725124351.15529bc7@pitrou.net> <4E2D9720.8040901@g.nevcal.com> <4E2DB98D.20106@stoneleaf.us> Message-ID: <20110726002644.GJ25354@aihal.home.puzzling.org> Ethan Furman wrote: > > [?] or "EINTRError" in my order of preference. > > Please not that last one! ;) Why not, exactly? When EINTR happens it's frequently a surprise, but programmers new to the concept can always search the web for advice on what causes it and how to deal with it (and after several attempts at dealing with it they may even get it right). Searching Google for ?InterruptedError? isn't going to find anything helpful today, and eventually what I expect it would find is a bunch of pages saying ?Look up EINTR.? How about we just cut out that middle step and call it what the rest of the world calls it? If ?InterruptedError? were going to be used for anything other than EINTR then I could see an argument for abstracting the concept behind a platform-independent name. But I think it would be a mistake to treat anything else as being the same as EINTR. -Andrew. From ncoghlan at gmail.com Tue Jul 26 03:45:33 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 26 Jul 2011 11:45:33 +1000 Subject: [Python-Dev] Comments of the PEP 3151 In-Reply-To: <20110726002644.GJ25354@aihal.home.puzzling.org> References: <4E2CB130.7060701@haypocalc.com> <20110725124351.15529bc7@pitrou.net> <4E2D9720.8040901@g.nevcal.com> <4E2DB98D.20106@stoneleaf.us> <20110726002644.GJ25354@aihal.home.puzzling.org> Message-ID: On Tue, Jul 26, 2011 at 10:26 AM, Andrew Bennetts wrote: > When EINTR happens it's frequently a surprise, but programmers new to > the concept can always search the web for advice on what causes it and > how to deal with it (and after several attempts at dealing with it they > may even get it right). ?Searching Google for ?InterruptedError? isn't > going to find anything helpful today, and eventually what I expect it > would find is a bunch of pages saying ?Look up EINTR.? ?How about we > just cut out that middle step and call it what the rest of the world > calls it? > > If ?InterruptedError? were going to be used for anything other than > EINTR then I could see an argument for abstracting the concept behind a > platform-independent name. ?But I think it would be a mistake to treat > anything else as being the same as EINTR. The whole point of PEP 3151 is so that Python programmers can perform key tasks without needing to worry about the existence of error numbers under the hood. Including the cryptic errno abbreviations in the interrupt names would completely miss the point. However, the docs will point to appropriate information in the errno module (and the exception details and docstrings may also mention the errno codes). Abstractions do leak, after all, but that doesn't mean we need to go punching holes in them with an icepick. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ethan at stoneleaf.us Tue Jul 26 04:31:48 2011 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 25 Jul 2011 19:31:48 -0700 Subject: [Python-Dev] Comments of the PEP 3151 In-Reply-To: <20110726002644.GJ25354@aihal.home.puzzling.org> References: <4E2CB130.7060701@haypocalc.com> <20110725124351.15529bc7@pitrou.net> <4E2D9720.8040901@g.nevcal.com> <4E2DB98D.20106@stoneleaf.us> <20110726002644.GJ25354@aihal.home.puzzling.org> Message-ID: <4E2E2714.3040809@stoneleaf.us> Andrew Bennetts wrote: > Ethan Furman wrote: >>> [?] or "EINTRError" in my order of preference. >> >> Please not that last one! ;) > > Why not, exactly? Because this is Python, and readability counts. Yes, it does take some getting used to (I finally stopped typing 'enum' a couple months ago ;) , but it is a worthwhile goal -- a goal we take a step back from with names like EINTRError instead of InterruptedError. ~Ethan~ From stephen at xemacs.org Tue Jul 26 05:47:01 2011 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 26 Jul 2011 12:47:01 +0900 Subject: [Python-Dev] Comments of the PEP 3151 In-Reply-To: <4E2D9720.8040901@g.nevcal.com> References: <4E2CB130.7060701@haypocalc.com> <20110725124351.15529bc7@pitrou.net> <4E2D9720.8040901@g.nevcal.com> Message-ID: <87oc0hr9e2.fsf@uwakimon.sk.tsukuba.ac.jp> Glenn Linderman writes: > Sorry, no. "InterruptError" sounds too much like a CPU interrupt > signal, which the error is not. "InterruptedError" is OK by me, I don't > see the confusion you do. But maybe "InterruptedOperationError" would > be the most clear. Way too long, of course, so maybe > "InterruptedAPIError" or "InterruptedOpError" or "EINTRError" in my > order of preference. Eh, doesn't it bother anybody that it's not an error, but a user action? If it needs a separate name, something like InterruptException seems most accurate to me. From ncoghlan at gmail.com Tue Jul 26 06:18:58 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 26 Jul 2011 14:18:58 +1000 Subject: [Python-Dev] Comments of the PEP 3151 In-Reply-To: <87oc0hr9e2.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4E2CB130.7060701@haypocalc.com> <20110725124351.15529bc7@pitrou.net> <4E2D9720.8040901@g.nevcal.com> <87oc0hr9e2.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Tue, Jul 26, 2011 at 1:47 PM, Stephen J. Turnbull wrote: > Eh, doesn't it bother anybody that it's not an error, but a user > action? Nope, doesn't bother me in the slightest. It's an error number code, just like all the others. Several other error numbers may or may not be errors too, depending on context. Rather than quibbling about that part of the naming scheme it's easier to say that the call failing to complete successfully is an error by default, but an application may choose to interpret some cases as not really being errors, since there's a defined response (most obvious case: a file or directory already existing or not existing often isn't an error from the application's point of view, it's just the application ensuring that a particular configuration exists on the file system in an idempotent fashion). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From merwok at netwok.org Tue Jul 26 15:17:53 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Tue, 26 Jul 2011 15:17:53 +0200 Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default): Issue #12102: Merge with 3.2. In-Reply-To: References: Message-ID: <4E2EBE81.8010701@netwok.org> Hi, > changeset: 71497:4898b14dcd69 > user: Ross Lagerwall > summary: > Issue #12102: Merge with 3.2. > > files: > Doc/ACKS.txt | 1 + > Doc/library/mmap.rst | 6 ++++++ > Misc/NEWS | 3 +++ > diff --git a/Misc/NEWS b/Misc/NEWS > --- a/Misc/NEWS > +++ b/Misc/NEWS > @@ -237,6 +237,9 @@ > Library > ------- > > +- Issue #12102: Document that buffered files must be flushed before being used > + with mmap. Patch by Steffen Daode Nurpmeso. We don?t add NEWS entries for each and every doc fix, otherwise it would be very huge :) We rather document large changes to the documentation, like adding links to the source files, using ?python3? instead of ?python? in all examples, etc. In addition, Library is the wrong section, it should be Documentation. I?m doing commits this afternoon, so I?ll take the occasion to remove this entry. Cheers From merwok at netwok.org Tue Jul 26 15:20:55 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Tue, 26 Jul 2011 15:20:55 +0200 Subject: [Python-Dev] [Python-checkins] cpython: Issue #11049: adding some tests to test.support In-Reply-To: References: Message-ID: <4E2EBF37.7020702@netwok.org> Hi, > changeset: 71465:be558ad15789 > user: Eli Bendersky > summary: > Issue #11049: adding some tests to test.support > Based on original patch by Giampaolo Rodola with contributions from R. David Murray > > files: > Lib/test/support.py | 21 +- > Lib/test/test_support.py | 178 +++++++++++++++++++++++++++ > 2 files changed, 189 insertions(+), 10 deletions(-) > > diff --git a/Lib/test/support.py b/Lib/test/support.py > --- a/Lib/test/support.py > +++ b/Lib/test/support.py > @@ -170,7 +170,7 @@ > attribute = getattr(obj, name) > except AttributeError: > raise unittest.SkipTest("module %s has no attribute %s" % ( > - obj.__name__, name)) > + repr(obj), name)) I would use %r instead of %s for both fields here. Non-ASCII characters and unseen whitespace are at least two reasons to overuse %r in debug/error messages instead of %s. Regards From solipsis at pitrou.net Tue Jul 26 15:30:25 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 26 Jul 2011 15:30:25 +0200 Subject: [Python-Dev] [Python-checkins] cpython: Issue #11049: adding some tests to test.support References: <4E2EBF37.7020702@netwok.org> Message-ID: <20110726153025.66b0de8b@pitrou.net> On Tue, 26 Jul 2011 15:20:55 +0200 ?ric Araujo wrote: > > > > diff --git a/Lib/test/support.py b/Lib/test/support.py > > --- a/Lib/test/support.py > > +++ b/Lib/test/support.py > > @@ -170,7 +170,7 @@ > > attribute = getattr(obj, name) > > except AttributeError: > > raise unittest.SkipTest("module %s has no attribute %s" % ( > > - obj.__name__, name)) > > + repr(obj), name)) > > I would use %r instead of %s for both fields here. Non-ASCII characters > and unseen whitespace are at least two reasons to overuse %r in > debug/error messages instead of %s. Actually, you want %a for non-ASCII messages to be escaped. (however, there's hardly any reason to worry about it when it comes to stdlib module names) Regards Antoine. From merwok at netwok.org Tue Jul 26 16:10:47 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Tue, 26 Jul 2011 16:10:47 +0200 Subject: [Python-Dev] [Python-checkins] cpython: Issue #11784: Improve multiprocessing.Process.join() documentation. Patch by In-Reply-To: References: Message-ID: <4E2ECAE7.503@netwok.org> > changeset: 71499:8d67fd820627 > parent: 71497:4898b14dcd69 > user: Charles-Fran?ois Natali > date: Mon Jul 25 18:35:49 2011 +0200 > summary: > Issue #11784: Improve multiprocessing.Process.join() documentation. Patch by > Patrick Sabin. > > files: > Doc/library/multiprocessing.rst | 7 +++---- > Misc/ACKS | 1 + There?s a dedicated file to thank doc contributors: Doc/ACKS.rst Regards From merwok at netwok.org Tue Jul 26 16:10:50 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Tue, 26 Jul 2011 16:10:50 +0200 Subject: [Python-Dev] [Python-checkins] cpython: Issue #11049: adding some tests to test.support In-Reply-To: <20110726153025.66b0de8b@pitrou.net> References: <4E2EBF37.7020702@netwok.org> <20110726153025.66b0de8b@pitrou.net> Message-ID: <4E2ECAEA.4090308@netwok.org> Le 26/07/2011 15:30, Antoine Pitrou a ?crit : > Actually, you want %a for non-ASCII messages to be escaped. Thanks for the reminder, I should use more %a instead of %r. In the packaging code however, we can?t, given that we want to backport. > (however, there's hardly any reason to worry about it when it comes to > stdlib module names) I lacked context to see that. If the code in question only ever handles stdlib modules, then okay. Cheers From merwok at netwok.org Tue Jul 26 16:39:31 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Tue, 26 Jul 2011 16:39:31 +0200 Subject: [Python-Dev] [Python-checkins] cpython (2.7): Issue #8746: Correct faulty configure checks so that os.chflags() and In-Reply-To: References: Message-ID: <4E2ED1A3.3050208@netwok.org> Hi, > changeset: 71030:abfe28e7e5cd > branch: 2.7 > user: Ned Deily > diff --git a/Lib/test/test_posix.py b/Lib/test/test_posix.py > --- a/Lib/test/test_posix.py > +++ b/Lib/test/test_posix.py > @@ -11,10 +11,12 @@ > import os > import pwd > import shutil > +import stat > import sys > import unittest > import warnings > > +_DUMMY_SYMLINK = '%s/dummy-symlink' % os.getenv('TMPDIR', '/tmp') Why not let the tempfile module do this for you? Regards From sdaoden at googlemail.com Tue Jul 26 16:42:18 2011 From: sdaoden at googlemail.com (Steffen Daode Nurpmeso) Date: Tue, 26 Jul 2011 16:42:18 +0200 Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default): Issue #12102: Merge with 3.2. In-Reply-To: <4E2EBE81.8010701@netwok.org> References: <4E2EBE81.8010701@netwok.org> Message-ID: <20110726144218.GA15388@sherwood.local> @ ?ric Araujo wrote (2011-07-26 15:17+0200): > > [.] > > Doc/library/mmap.rst | 6 ++++++ > > [.] > > + with mmap. Patch by Steffen Daode Nurpmeso. > > I?m doing commits this afternoon, so I?ll take the occasion to remove > this entry. Murmur... but it hits me little: where is the difference in between Doc/ACKS.txt and Misc/ACKS? I'm mentioned in the latter (on multiple branches), but not at all in the former. Sob. But this i tell you - if i would be an italian.. then i.. would.. SOOOOOOOOOB!! Viva la mamma - per que!!! --Steffen Ciao, sdaoden(*)(gmail.com) ASCII ribbon campaign ( ) More nuclear fission plants against HTML e-mail X can serve more coloured and proprietary attachments / \ and sounding animations From merwok at netwok.org Tue Jul 26 16:48:22 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Tue, 26 Jul 2011 16:48:22 +0200 Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default): Issue #12102: Merge with 3.2. In-Reply-To: <20110726144218.GA15388@sherwood.local> References: <4E2EBE81.8010701@netwok.org> <20110726144218.GA15388@sherwood.local> Message-ID: <4E2ED3B6.9090907@netwok.org> Hi, > Murmur... but it hits me little: where is the difference in > between Doc/ACKS.txt and Misc/ACKS? Doc/ACKS.txt is used for doc contributions, Misc/ACKS for other contributions (but sometimes doc too!). Doc/A is also displayed on docs.python.org whereas Misc/A is only readable in tarballs. We talked about merging them a little time ago; I have to check my archives, I don?t remember what issues there were. Regards From mail at kerrickstaley.com Tue Jul 26 17:56:56 2011 From: mail at kerrickstaley.com (Kerrick Staley) Date: Tue, 26 Jul 2011 10:56:56 -0500 Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the /usr/bin/python2 symlink upstream) In-Reply-To: <20110724232106.53161528@pitrou.net> References: <20110724231655.0b69d36a@pitrou.net> <20110724232106.53161528@pitrou.net> Message-ID: I'm indifferent either way. python3 is a hard link to python3.2, so I thought we'd make everything that way for consistency. Higher-level links (python/idle/pydoc/python-config) have to be soft links so that if, e.g., python points to python3, and python3 is then pointed to another location, python also gets changed. -Kerrick Staley From rosslagerwall at gmail.com Tue Jul 26 18:03:21 2011 From: rosslagerwall at gmail.com (Ross Lagerwall) Date: Tue, 26 Jul 2011 18:03:21 +0200 Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default): Issue #12102: Merge with 3.2. In-Reply-To: <4E2EBE81.8010701@netwok.org> References: <4E2EBE81.8010701@netwok.org> Message-ID: <1311696201.1508.0.camel@hobo> > We don?t add NEWS entries for each and every doc fix, otherwise it would > be very huge :) We rather document large changes to the documentation, > like adding links to the source files, using ?python3? instead of > ?python? in all examples, etc. In addition, Library is the wrong > section, it should be Documentation. > > I?m doing commits this afternoon, so I?ll take the occasion to remove > this entry. > Thanks for the heads up. I'll keep it in mind for next time. Cheers Ross From solipsis at pitrou.net Tue Jul 26 18:05:28 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 26 Jul 2011 18:05:28 +0200 Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the /usr/bin/python2 symlink upstream) In-Reply-To: References: <20110724231655.0b69d36a@pitrou.net> <20110724232106.53161528@pitrou.net> Message-ID: <1311696328.3648.0.camel@localhost.localdomain> Le mardi 26 juillet 2011 ? 10:56 -0500, Kerrick Staley a ?crit : > I'm indifferent either way. python3 is a hard link to python3.2, so I > thought we'd make everything that way for consistency. Is it? Yikes, I didn't know about that. Regards Antoine. From merwok at netwok.org Tue Jul 26 18:15:15 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Tue, 26 Jul 2011 18:15:15 +0200 Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the /usr/bin/python2 symlink upstream) In-Reply-To: <1311696328.3648.0.camel@localhost.localdomain> References: <20110724231655.0b69d36a@pitrou.net> <20110724232106.53161528@pitrou.net> <1311696328.3648.0.camel@localhost.localdomain> Message-ID: <4E2EE813.1080407@netwok.org> Le 26/07/2011 18:05, Antoine Pitrou a ?crit : > Le mardi 26 juillet 2011 ? 10:56 -0500, Kerrick Staley a ?crit : >> I'm indifferent either way. python3 is a hard link to python3.2, so I >> thought we'd make everything that way for consistency. > > Is it? Yikes, I didn't know about that. Yikes for me too. I?ve had a quick look at the Makefile (look for $(LN)) and found that all scripts use symbolic links, but the python3 to python3.y link is hard. I wonder why this is. FTR, downstream packagers may change this. Example on Debian: /usr/bin/python3: symbolic link to `python3.2' /usr/bin/python3.2: symbolic link to `python3.2mu' Cheers From nad at acm.org Tue Jul 26 20:50:47 2011 From: nad at acm.org (Ned Deily) Date: Tue, 26 Jul 2011 11:50:47 -0700 Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the /usr/bin/python2 symlink upstream) References: <20110724231655.0b69d36a@pitrou.net> <20110724232106.53161528@pitrou.net> <1311696328.3648.0.camel@localhost.localdomain> <4E2EE813.1080407@netwok.org> Message-ID: In article <4E2EE813.1080407 at netwok.org>, ?ric Araujo wrote: > Le 26/07/2011 18:05, Antoine Pitrou a ?crit : > > Le mardi 26 juillet 2011 ? 10:56 -0500, Kerrick Staley a ?crit : > >> I'm indifferent either way. python3 is a hard link to python3.2, so I > >> thought we'd make everything that way for consistency. > > Is it? Yikes, I didn't know about that. > Yikes for me too. I?ve had a quick look at the Makefile (look for > $(LN)) and found that all scripts use symbolic links, but the python3 to > python3.y link is hard. I wonder why this is. I pointed that out earlier in the thread: "But if you look at the Python 3 "bininstall" target (Makefile.pre.in starting around line 870 or so), you'll see that, for Python 3, symlinks are made for all the versioned files except "python3". I'm not sure that there is a particular reason why a distinction was made; IIRC, the other versioned links were added later in the cycle. The other Python 3 versioned links could probably be changed to hard links as well. In the end, I don't think it makes a lot of difference. But it would be better if Python 2 and Python 3 were consistent here." I don't think it makes all that much difference one way or the other. But it would be better for them all to be one kind or the other. -- Ned Deily, nad at acm.org From cf.natali at gmail.com Tue Jul 26 22:43:00 2011 From: cf.natali at gmail.com (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=) Date: Tue, 26 Jul 2011 22:43:00 +0200 Subject: [Python-Dev] [Python-checkins] cpython: Issue #11784: Improve multiprocessing.Process.join() documentation. Patch by In-Reply-To: <4E2ECAE7.503@netwok.org> References: <4E2ECAE7.503@netwok.org> Message-ID: > There?s a dedicated file to thank doc contributors: Doc/ACKS.rst I didn't know about this file, thanks. In my "defense", there's this comment at the top of Misc/ACKS: """ This list is not complete and not in any useful order, but I would like to thank everybody who contributed in any way, with code, hints, bug reports, ideas, moral support, endorsement, or even complaints.... Without you, I would've stopped working on Python long ago! --Guido """ What's the rationale for having a dedicated file? From nad at acm.org Tue Jul 26 23:17:24 2011 From: nad at acm.org (Ned Deily) Date: Tue, 26 Jul 2011 14:17:24 -0700 Subject: [Python-Dev] [Python-checkins] cpython (2.7): Issue #8746: Correct faulty configure checks so that os.chflags() and References: <4E2ED1A3.3050208@netwok.org> Message-ID: In article <4E2ED1A3.3050208 at netwok.org>, Eric Araujo wrote: > > changeset: 71030:abfe28e7e5cd > > branch: 2.7 > > user: Ned Deily > > > diff --git a/Lib/test/test_posix.py b/Lib/test/test_posix.py > > --- a/Lib/test/test_posix.py > > +++ b/Lib/test/test_posix.py > > @@ -11,10 +11,12 @@ > > import os > > import pwd > > import shutil > > +import stat > > import sys > > import unittest > > import warnings > > > > +_DUMMY_SYMLINK = '%s/dummy-symlink' % os.getenv('TMPDIR', '/tmp') > > Why not let the tempfile module do this for you? I didn't feel it was worthwhile modifying the OP's submitted working patch. But, since you've pointed it out, I've modified the test to use tempfile and randomize the symlink file name as well. -- Ned Deily, nad at acm.org From sdaoden at googlemail.com Tue Jul 26 22:18:12 2011 From: sdaoden at googlemail.com (Steffen Daode Nurpmeso) Date: Tue, 26 Jul 2011 22:18:12 +0200 Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default): Issue #12102: Merge with 3.2. In-Reply-To: <4E2EBE81.8010701@netwok.org> References: <4E2EBE81.8010701@netwok.org> Message-ID: <20110726201812.GA49358@sherwood.local> I was contacted off-list due to anxiety about cleanliness of python-dev in respect to the following lines: > But this i tell you - if i would be an italian.. then i.. would.. > Viva la mamma - per que!!! Sob. This is **NOT** offensive against just anybody. (Except maybe that i don't take *myself* too seriously, because doing so is a bit strange on a planet which disappears in some billion years, at which time i'll be gone and away. At latest.) But especially *NOT* against italians. I *love* Italy! I'm a german!! I'm actually talking about http://www.youtube.com/watch?v=czjGpFyS-5w here, though i feel more like http://www.youtube.com/watch?v=lblz8g5mQRk now. (Aah, Branduardi. He's a good one.) Hm. Maybe "that" Italy is gone now though, i haven't been there for some years ... But that is something completely different. (Dunno - is it still possible to do sightseeing? We here in Germany do have some ancient Rome stuff in good condition, just in case it would be needed? Hey - you know where you can find it ...) Yes, yes, i *love* France, too! My lovely woman and i came together during a short trip to France, 20 years ago. [Maybe we will spend our old days there, if we can afford it (and i'm able to learn the language). But *i* also can imagine some italian island south-of-Rome for that purpose.] But first i was there due to a city partnership, in 1985, football (aeh - this one you play with your feet) tournament. Small town. Played the german national anthem. Played the french national anthem. Sorry, maybe i misunderstood "Marchons, marchons". The french boys grinned anyway. I was a german goalkeeper, then. And my guest-family (little farmer) grilled their major (!!!!) rooster for me (and a second boy they hosted). I mean - i wouldn't have grilled a lion (heraldic animal) if they came to Hessen in turn! Isn't that a bit strange?? Luckily i already knew the meaning of "Je ne regrette rien". (Aah - Edith Piaf!!) But the farmers wife stated "NO RIEN!! NO RIEN!!!" in turn! Can you imagine that? Such an aggressive rooster!!! Even dead. Even *completely* eaten up! I can tell you. The worst thing was that i couldn't go to toilet for those three days, because - it was *soo* shitty! Sob. Just in case i won't get blacklisted: next time i'm contacted off-list i'll tell the story when our football (with the feet, with your feet!) team went to Poland due to city partnership in the year after tchernobyl. And guess who talked :) to the mayor of the city? Hah!! [Nose blow.] Oh, and before i forget it: my boys (Football - feet!) were mostly Turks, one Moroccan, one Italian (!), Andreas from Poland (fantastic defender on the left side!) and less than 50% germans. And we were a really good *playing* youth team. No, not always the winning team. Horsy horse says Nighty night. --Steffen Ciao, sdaoden(*)(gmail.com) ASCII ribbon campaign ( ) More nuclear fission plants against HTML e-mail X can serve more coloured and proprietary attachments / \ and sounding animations From solipsis at pitrou.net Wed Jul 27 00:19:12 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 27 Jul 2011 00:19:12 +0200 Subject: [Python-Dev] hard linking executables References: <20110724231655.0b69d36a@pitrou.net> <20110724232106.53161528@pitrou.net> <1311696328.3648.0.camel@localhost.localdomain> <4E2EE813.1080407@netwok.org> Message-ID: <20110727001912.4cf67481@pitrou.net> Ok, apparently the decision to make hard links for executables dates at least back to: changeset: 16221:588691f806f4 branch: legacy-trunk user: Neil Schemenauer date: Wed Jan 24 17:11:43 2001 +0000 files: Makefile.pre.in description: Flat makefile based on toplevel Makefile.in and makefiles in build subdirectories. Those other makefiles will go away eventually. [...] +# Install the interpreter (by creating a hard link to python$(VERSION)) +bininstall: altbininstall + -if test -f $(BINDIR)/$(PYTHON); \ + then rm -f $(BINDIR)/$(PYTHON); \ + else true; \ + fi + (cd $(BINDIR); $(LN) python$(VERSION)$(EXEEXT) python$(EXEEXT)) + Regards Antoine. From solipsis at pitrou.net Wed Jul 27 00:23:23 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 27 Jul 2011 00:23:23 +0200 Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default): Issue #12102: Merge with 3.2. References: <4E2EBE81.8010701@netwok.org> <20110726201812.GA49358@sherwood.local> Message-ID: <20110727002323.62cb398e@pitrou.net> Hi, > > But this i tell you - if i would be an italian.. then i.. would.. > > Viva la mamma - per que!!! > > Sob. > > This is **NOT** offensive against just anybody. > (Except maybe that i don't take *myself* too seriously, because > doing so is a bit strange on a planet which disappears in some > billion years, at which time i'll be gone and away. At latest.) > But especially *NOT* against italians. > I *love* Italy! I'm a german!! Well I don't think anyone would take it as offensive. However, obscure jokes without any context don't (IMHO) make your messages very readable either, and I think they would benefit from being concise and to the point ;) Hope that helps. Regards Antoine. From solipsis at pitrou.net Wed Jul 27 00:49:55 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 27 Jul 2011 00:49:55 +0200 Subject: [Python-Dev] Comments of the PEP 3151 References: <4E2CB130.7060701@haypocalc.com> Message-ID: <20110727004955.1642e815@pitrou.net> On Mon, 25 Jul 2011 15:28:47 +1000 Nick Coghlan wrote: > There may be some error codes that we choose to map to these generic > errors, even if we don't give them their own exception types at this > point (e.g. ECONSHUTDOWN could map directly to ConnectionError). Ok, I can find neither ECONSHUTDOWN nor ECONNSHUTDOWN on www.opengroup.org, and it's not mentioned in errnomodule.c. Is it some system-specific error code? Regards Antoine. From glyph at twistedmatrix.com Wed Jul 27 01:32:56 2011 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Tue, 26 Jul 2011 19:32:56 -0400 Subject: [Python-Dev] Comments of the PEP 3151 In-Reply-To: <20110727004955.1642e815@pitrou.net> References: <4E2CB130.7060701@haypocalc.com> <20110727004955.1642e815@pitrou.net> Message-ID: <51E7C85E-8AE2-4E7B-987C-EB6CF5BFDCDB@twistedmatrix.com> On Jul 26, 2011, at 6:49 PM, Antoine Pitrou wrote: > On Mon, 25 Jul 2011 15:28:47 +1000 > Nick Coghlan wrote: >> There may be some error codes that we choose to map to these generic >> errors, even if we don't give them their own exception types at this >> point (e.g. ECONSHUTDOWN could map directly to ConnectionError). > > Ok, I can find neither ECONSHUTDOWN nor ECONNSHUTDOWN on > www.opengroup.org, and it's not mentioned in errnomodule.c. Is it some > system-specific error code? I assume that ESHUTDOWN is the errno in question? (This is also already mentioned in the PEP.) From solipsis at pitrou.net Wed Jul 27 01:45:18 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 27 Jul 2011 01:45:18 +0200 Subject: [Python-Dev] Comments of the PEP 3151 In-Reply-To: <51E7C85E-8AE2-4E7B-987C-EB6CF5BFDCDB@twistedmatrix.com> References: <4E2CB130.7060701@haypocalc.com> <20110727004955.1642e815@pitrou.net> <51E7C85E-8AE2-4E7B-987C-EB6CF5BFDCDB@twistedmatrix.com> Message-ID: <20110727014518.6515fd5f@pitrou.net> On Tue, 26 Jul 2011 19:32:56 -0400 Glyph Lefkowitz wrote: > > On Jul 26, 2011, at 6:49 PM, Antoine Pitrou wrote: > > > On Mon, 25 Jul 2011 15:28:47 +1000 > > Nick Coghlan wrote: > >> There may be some error codes that we choose to map to these generic > >> errors, even if we don't give them their own exception types at this > >> point (e.g. ECONSHUTDOWN could map directly to ConnectionError). > > > > Ok, I can find neither ECONSHUTDOWN nor ECONNSHUTDOWN on > > www.opengroup.org, and it's not mentioned in errnomodule.c. Is it some > > system-specific error code? > > I assume that ESHUTDOWN is the errno in question? (This is also already mentioned in the PEP.) Indeed, I mentioned it in the PEP, as it appears in asyncore.py. But I can't find it on www.opengroup.org, and no man page on my Linux system (except the "errno" man page) seems to mention it. The description from errnomodule.c says "Cannot send after transport endpoint shutdown", but send() actually returns EPIPE, not ESHUTDOWN, when the socket has been shutdown: >>> conn = socket.create_connection(("www.python.org", 80)) >>> conn.shutdown(socket.SHUT_WR) >>> conn.send(b"xxx") Traceback (most recent call last): File "", line 1, in socket.error: [Errno 32] Broken pipe From the send() man page: EPIPE The local end has been shut down on a connection oriented socket. In this case the process will also receive a SIGPIPE unless MSG_NOSIGNAL is set. Regards Antoine. From solipsis at pitrou.net Wed Jul 27 02:03:19 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 27 Jul 2011 02:03:19 +0200 Subject: [Python-Dev] ESHUTDOWN References: <4E2CB130.7060701@haypocalc.com> <20110727004955.1642e815@pitrou.net> <51E7C85E-8AE2-4E7B-987C-EB6CF5BFDCDB@twistedmatrix.com> <20110727014518.6515fd5f@pitrou.net> Message-ID: <20110727020319.4e5ca9d3@pitrou.net> Some more digging indicates that ESHUTDOWN appears in asyncore with the following commit: changeset: 10934:c089020a7a1e branch: legacy-trunk user: Guido van Rossum date: Tue Jun 08 13:20:05 1999 +0000 files: Lib/asynchat.py Lib/asyncore.py description: Sam's latest versions while it appears in errnomodule.c with the following commit: changeset: 3804:48776bf4bd49 branch: legacy-trunk user: Guido van Rossum date: Wed Jul 24 00:51:51 1996 +0000 files: Modules/Setup.in Modules/errnomodule.c description: Added Sam Rushing's errno module It also seems that WSAESHUTDOWN can be returned under Windows by send(), rather than EPIPE: ?WSAESHUTDOWN The socket has been shut down; it is not possible to send on a socket after shutdown has been invoked with how set to SD_SEND or SD_BOTH.? Regards Antoine. From ncoghlan at gmail.com Wed Jul 27 02:41:34 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 27 Jul 2011 10:41:34 +1000 Subject: [Python-Dev] [Python-checkins] cpython: Issue #11049: adding some tests to test.support In-Reply-To: <4E2ECAEA.4090308@netwok.org> References: <4E2EBF37.7020702@netwok.org> <20110726153025.66b0de8b@pitrou.net> <4E2ECAEA.4090308@netwok.org> Message-ID: On Wed, Jul 27, 2011 at 12:10 AM, ?ric Araujo wrote: > Le 26/07/2011 15:30, Antoine Pitrou a ?crit : >> Actually, you want %a for non-ASCII messages to be escaped. > > Thanks for the reminder, I should use more %a instead of %r. ?In the > packaging code however, we can?t, given that we want to backport. > >> (however, there's hardly any reason to worry about it when it comes to >> stdlib module names) > > I lacked context to see that. ?If the code in question only ever handles > stdlib modules, then okay. The other reason to use %r is to get the enclosing quotes in the displayed message. That reason applies even in cases like this where the escaping aspect isn't a concern. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From brett at python.org Wed Jul 27 04:04:20 2011 From: brett at python.org (Brett Cannon) Date: Tue, 26 Jul 2011 19:04:20 -0700 Subject: [Python-Dev] [Python-checkins] cpython: Issue #11049: adding some tests to test.support In-Reply-To: References: <4E2EBF37.7020702@netwok.org> <20110726153025.66b0de8b@pitrou.net> <4E2ECAEA.4090308@netwok.org> Message-ID: On Tue, Jul 26, 2011 at 17:41, Nick Coghlan wrote: > On Wed, Jul 27, 2011 at 12:10 AM, ?ric Araujo wrote: > > Le 26/07/2011 15:30, Antoine Pitrou a ?crit : > >> Actually, you want %a for non-ASCII messages to be escaped. > > > > Thanks for the reminder, I should use more %a instead of %r. In the > > packaging code however, we can?t, given that we want to backport. > > > >> (however, there's hardly any reason to worry about it when it comes to > >> stdlib module names) > > > > I lacked context to see that. If the code in question only ever handles > > stdlib modules, then okay. > > The other reason to use %r is to get the enclosing quotes in the > displayed message. That reason applies even in cases like this where > the escaping aspect isn't a concern. > And then you make it {!r} so you can use str.format and you complete the tweak of the string formatting! =) Seriously, though, it wouldn't hurt to update it to use str.format(). -------------- next part -------------- An HTML attachment was scrubbed... URL: From neologix at free.fr Wed Jul 27 09:11:35 2011 From: neologix at free.fr (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=) Date: Wed, 27 Jul 2011 09:11:35 +0200 Subject: [Python-Dev] Comments of the PEP 3151 In-Reply-To: <20110727014518.6515fd5f@pitrou.net> References: <4E2CB130.7060701@haypocalc.com> <20110727004955.1642e815@pitrou.net> <51E7C85E-8AE2-4E7B-987C-EB6CF5BFDCDB@twistedmatrix.com> <20110727014518.6515fd5f@pitrou.net> Message-ID: >> I assume that ESHUTDOWN is the errno in question? ?(This is also already mentioned in the PEP.) > > Indeed, I mentioned it in the PEP, as it appears in asyncore.py. > But I can't find it on www.opengroup.org, and no man page on my Linux > system (except the "errno" man page) seems to mention it. It's not POSIX, but it's defined on Linux and FreeBSD (at least): http://lxr.free-electrons.com/source/include/asm-generic/errno.h#L81 http://fxr.watson.org/fxr/source/sys/errno.h?v=FREEBSD53#L122 > The description from errnomodule.c says "Cannot send after transport > endpoint shutdown", but send() actually returns EPIPE, not ESHUTDOWN, > when the socket has been shutdown: Indeed, as required by POSIX. But grepping through the Linux kernel source code, it seems to be used extensively for USB devices, see http://lxr.free-electrons.com/ident?i=ESHUTDOWN So the "transport endpoint" doesn't necessarily refer to a socket. It's also documented in http://lxr.free-electrons.com/source/Documentation/usb/error-codes.txt Finally, I found one place in the networking stack where ESHUTDOWN is used, in the SCTP code: http://lxr.free-electrons.com/source/net/sctp/outqueue.c#L329 From ncoghlan at gmail.com Wed Jul 27 09:53:30 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 27 Jul 2011 17:53:30 +1000 Subject: [Python-Dev] Comments of the PEP 3151 In-Reply-To: References: <4E2CB130.7060701@haypocalc.com> <20110727004955.1642e815@pitrou.net> <51E7C85E-8AE2-4E7B-987C-EB6CF5BFDCDB@twistedmatrix.com> <20110727014518.6515fd5f@pitrou.net> Message-ID: 2011/7/27 Charles-Fran?ois Natali : >>> I assume that ESHUTDOWN is the errno in question? ?(This is also already mentioned in the PEP.) >> >> Indeed, I mentioned it in the PEP, as it appears in asyncore.py. >> But I can't find it on www.opengroup.org, and no man page on my Linux >> system (except the "errno" man page) seems to mention it. > > It's not POSIX, but it's defined on Linux and FreeBSD (at least): > http://lxr.free-electrons.com/source/include/asm-generic/errno.h#L81 > http://fxr.watson.org/fxr/source/sys/errno.h?v=FREEBSD53#L122 > >> The description from errnomodule.c says "Cannot send after transport >> endpoint shutdown", but send() actually returns EPIPE, not ESHUTDOWN, >> when the socket has been shutdown: > > Indeed, as required by POSIX. > > But grepping through the Linux kernel source code, it seems to be used > extensively for USB devices, see > http://lxr.free-electrons.com/ident?i=ESHUTDOWN > So the "transport endpoint" doesn't necessarily refer to a socket. > It's also documented in > http://lxr.free-electrons.com/source/Documentation/usb/error-codes.txt > > Finally, I found one place in the networking stack where ESHUTDOWN is > used, in the SCTP code: > http://lxr.free-electrons.com/source/net/sctp/outqueue.c#L329 Perhaps the right thing to do is to just have a ConnectionBrokenError that covers EPIPE, ESHUTDOWN and ECONNRESET? The current version of the PEP has these as two separate exception types (BrokenPipeError for the first two and ConnectionResetError for the last), but I'm not seeing a strong reason to avoid combining them. (ECONNABORTED and ECONNREFUSED are different, since they relate to failures when *initiating* a connection) Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From fuzzyman at voidspace.org.uk Wed Jul 27 11:50:40 2011 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 27 Jul 2011 10:50:40 +0100 Subject: [Python-Dev] [Python-checkins] cpython: Issue #11049: adding some tests to test.support In-Reply-To: References: <4E2EBF37.7020702@netwok.org> <20110726153025.66b0de8b@pitrou.net> <4E2ECAEA.4090308@netwok.org> Message-ID: <4E2FDF70.3080803@voidspace.org.uk> On 27/07/2011 03:04, Brett Cannon wrote: > > > On Tue, Jul 26, 2011 at 17:41, Nick Coghlan > wrote: > > On Wed, Jul 27, 2011 at 12:10 AM, ?ric Araujo > wrote: > > Le 26/07/2011 15:30, Antoine Pitrou a ?crit : > >> Actually, you want %a for non-ASCII messages to be escaped. > > > > Thanks for the reminder, I should use more %a instead of %r. In the > > packaging code however, we can't, given that we want to backport. > > > >> (however, there's hardly any reason to worry about it when it > comes to > >> stdlib module names) > > > > I lacked context to see that. If the code in question only ever > handles > > stdlib modules, then okay. > > The other reason to use %r is to get the enclosing quotes in the > displayed message. That reason applies even in cases like this where > the escaping aspect isn't a concern. > > > And then you make it {!r} so you can use str.format and you complete > the tweak of the string formatting! =) Seriously, though, it wouldn't > hurt to update it to use str.format(). > Well, except that {!r} is twice as verbose as %r.... Michael > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Wed Jul 27 11:52:32 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 27 Jul 2011 11:52:32 +0200 Subject: [Python-Dev] cpython (3.2): Fix closes Issue12576 - fix urlopen behavior on sites which do not send (or References: Message-ID: <20110727115232.5bbf7aea@pitrou.net> On Wed, 27 Jul 2011 02:25:56 +0200 senthil.kumaran wrote: > > + def test_sites_no_connection_close(self): > + # Some sites do not send Connection: close header. > + # Verify that those work properly. (#issue12576) > + > + try: > + with urllib.request.urlopen('http://www.imdb.com') as res: > + pass Can you please at least use support.transient_internet() as in other tests in this file? > + except ValueError as e: > + self.fail("urlopen failed for sites not sending Connection:close") > + else: > + self.assertTrue(res) > + > + req = urllib.request.urlopen('http://www.imdb.com') Why a second time? > + res = req.read() > + self.assertTrue(res) Also, when does "req" get closed? Right now I get resource warnings: test_sites_no_connection_close (test.test_urllib2net.OtherNetworkTests) ... /home/antoine/cpython/default/Lib/socket.py:342: ResourceWarning: unclosed self._sock = None /home/antoine/cpython/default/Lib/socket.py:342: ResourceWarning: unclosed self._sock = None ok Regards Antoine. From senthil at uthcode.com Wed Jul 27 14:16:01 2011 From: senthil at uthcode.com (Senthil Kumaran) Date: Wed, 27 Jul 2011 20:16:01 +0800 Subject: [Python-Dev] cpython (3.2): Fix closes Issue12576 - fix urlopen behavior on sites which do not send (or In-Reply-To: <20110727115232.5bbf7aea@pitrou.net> References: <20110727115232.5bbf7aea@pitrou.net> Message-ID: <20110727121600.GA3140@mathmagic> On Wed, Jul 27, 2011 at 11:52:32AM +0200, Antoine Pitrou wrote: > > + > > + try: > > + with urllib.request.urlopen('http://www.imdb.com') as res: > > + pass > > Can you please at least use support.transient_internet() as in other > tests in this file? It was intentional because ValueError was raised from context manager use case for a bug where the request object was closed prematurely. support.transient_internet, I believe would not have covered that case (Usage scenario). > > + req = urllib.request.urlopen('http://www.imdb.com') > > Why a second time? When used outside of context manager, it gave empty response instead of ValueError and the test case was to check that. > > + res = req.read() > > + self.assertTrue(res) > > Also, when does "req" get closed? Right now I get resource warnings: I shall fix this one. I think, attempting to fix the Resource warning caused the regression wherein the request object closed prematurely. I shall look at this. Thanks, Senthil From solipsis at pitrou.net Wed Jul 27 14:21:59 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 27 Jul 2011 14:21:59 +0200 Subject: [Python-Dev] cpython (3.2): Fix closes Issue12576 - fix urlopen behavior on sites which do not send (or In-Reply-To: <20110727121600.GA3140@mathmagic> References: <20110727115232.5bbf7aea@pitrou.net> <20110727121600.GA3140@mathmagic> Message-ID: <20110727142159.65e25267@pitrou.net> On Wed, 27 Jul 2011 20:16:01 +0800 Senthil Kumaran wrote: > On Wed, Jul 27, 2011 at 11:52:32AM +0200, Antoine Pitrou wrote: > > > + > > > + try: > > > + with urllib.request.urlopen('http://www.imdb.com') as res: > > > + pass > > > > Can you please at least use support.transient_internet() as in other > > tests in this file? > > It was intentional because ValueError was raised from context manager > use case for a bug where the request object was closed prematurely. > support.transient_internet, I believe would not have covered that > case (Usage scenario). Unless I'm reading wrongly, transient_internet doesn't silence ValueError at all. > > > + res = req.read() > > > + self.assertTrue(res) > > > > Also, when does "req" get closed? Right now I get resource warnings: > > I shall fix this one. I think, attempting to fix the Resource warning > caused the regression wherein the request object closed prematurely. I > shall look at this. Well, the test should simply call close() as is done in other tests. Regards Antoine. From eliben at gmail.com Wed Jul 27 15:14:40 2011 From: eliben at gmail.com (Eli Bendersky) Date: Wed, 27 Jul 2011 16:14:40 +0300 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: References: Message-ID: > > > > The mere fact that these functions exist in a different module suggests > different semantics from those found in other places in the stdlib. I don't > think they should be renamed simply because some code imports the functions > directly instead of the module itself (suggesting they should be doing the > latter over the former). Now if the functions are redundant that's something > else entirely and removal should be fine. > I will try to clarify my concern. Documented functions from test.support currently appear in the global index of the Sphinx-generated documentation. Suppose we document test.support's "unlink". Now the index entry for "unlink" will contain two "unlink" references (*), with slightly different semantics - one in os and another in test.support Will it take long for newbie code to appear with the test.support version? Not to mention that grepping code that imports the "unlink" function directly doesn't reveal which one is being used. I think this is troublesome. I think at least two separate actions are required here: 1. In the documentation of test.support mention explicitly that it's code for CPython's internal use only, and these APIs aren't guaranteed to be stable. 2. Some functions like unlink and rmtree are obviously redundant, and shadow frequently used Python stdlib functions, so I would either kill them completely or at least rename them appropriately. Eli (*) Actually, three, since there's also xml.dom.minidom.Node, but that's obviously unrelated). -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Wed Jul 27 15:24:10 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 27 Jul 2011 15:24:10 +0200 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions References: Message-ID: <20110727152410.22871784@pitrou.net> On Wed, 27 Jul 2011 16:14:40 +0300 Eli Bendersky wrote: > > Will it take long for newbie code to appear with the test.support version? > Not to mention that grepping code that imports the "unlink" function > directly doesn't reveal which one is being used. > > I think this is troublesome. I think at least two separate actions are > required here: > > 1. In the documentation of test.support mention explicitly that it's code > for CPython's internal use only, and these APIs aren't guaranteed to be > stable. There is a top-level note at http://docs.python.org/dev/library/test.html, but it won't be visible by people who arrive at an anchor point. I think officially documenting test.support is a mistake. There is no guarantee that APIs are stable or will even continue to exist. Docstrings are sufficient for own our purposes. > 2. Some functions like unlink and rmtree are obviously redundant, and shadow > frequently used Python stdlib functions, so I would either kill them > completely or at least rename them appropriately. They are not redundant, since they provide slightly different semantics (for example, they silence errors that happen when the path doesn't exist). Regards Antoine. From eliben at gmail.com Wed Jul 27 15:31:54 2011 From: eliben at gmail.com (Eli Bendersky) Date: Wed, 27 Jul 2011 16:31:54 +0300 Subject: [Python-Dev] [Python-checkins] cpython: Issue #11049: adding some tests to test.support In-Reply-To: <20110726153025.66b0de8b@pitrou.net> References: <4E2EBF37.7020702@netwok.org> <20110726153025.66b0de8b@pitrou.net> Message-ID: > > I would use %r instead of %s for both fields here. Non-ASCII characters > > and unseen whitespace are at least two reasons to overuse %r in > > debug/error messages instead of %s. > > Actually, you want %a for non-ASCII messages to be escaped. > (however, there's hardly any reason to worry about it when it comes to > stdlib module names) > I wasn't aware of '%a' at all? It doesn't appear to be documented, and Python 2.6 doesn't support it: ValueError: unsupported format character 'a' (0x61) at index 1 If it's new, it should at least be documented in library/stdtypes with the other formatting operations. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From eliben at gmail.com Wed Jul 27 15:35:24 2011 From: eliben at gmail.com (Eli Bendersky) Date: Wed, 27 Jul 2011 16:35:24 +0300 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: <20110727152410.22871784@pitrou.net> References: <20110727152410.22871784@pitrou.net> Message-ID: > > 1. In the documentation of test.support mention explicitly that it's code > > for CPython's internal use only, and these APIs aren't guaranteed to be > > stable. > > There is a top-level note at > http://docs.python.org/dev/library/test.html, but it won't be visible > by people who arrive at an anchor point. > > I think officially documenting test.support is a mistake. There is no > guarantee that APIs are stable or will even continue to exist. > Docstrings are sufficient for own our purposes. > Initially I was *for* documenting, but this thing with showing up in the index is a compelling counter-point. > 2. Some functions like unlink and rmtree are obviously redundant, and shadow > frequently used Python stdlib functions, so I would either kill them > completely or at least rename them appropriately. They are not redundant, since they provide slightly different semantics > (for example, they silence errors that happen when the path doesn't > exist). > Sure, but I'm still leery of two functions with the same name doing acting slightly differently. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdmurray at bitdance.com Wed Jul 27 15:36:19 2011 From: rdmurray at bitdance.com (R. David Murray) Date: Wed, 27 Jul 2011 09:36:19 -0400 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: References: Message-ID: <20110727133619.F0B622506ED@webabinitio.net> On Wed, 27 Jul 2011 16:14:40 +0300, Eli Bendersky wrote: > 1. In the documentation of test.support mention explicitly that it's code > for CPython's internal use only, and these APIs aren't guaranteed to be > stable. This was already done. > 2. Some functions like unlink and rmtree are obviously redundant, and shadow > frequently used Python stdlib functions, so I would either kill them > completely or at least rename them appropriately. But they aren't redundant, since the test.support versions ignore errors. Perhaps what we could do is move the documentation for test.support to the devguide, and then vet the test suite so that unlink and friends are always called as 'support.unlink', etc. -- R. David Murray http://www.bitdance.com From solipsis at pitrou.net Wed Jul 27 15:44:07 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 27 Jul 2011 15:44:07 +0200 Subject: [Python-Dev] [Python-checkins] cpython: Issue #11049: adding some tests to test.support In-Reply-To: References: <4E2EBF37.7020702@netwok.org> <20110726153025.66b0de8b@pitrou.net> Message-ID: <20110727154407.4f3b2a69@pitrou.net> On Wed, 27 Jul 2011 16:31:54 +0300 Eli Bendersky wrote: > > I wasn't aware of '%a' at all? It doesn't appear to be documented, and > Python 2.6 doesn't support it: > > ValueError: unsupported format character 'a' (0x61) at index 1 > > If it's new, it should at least be documented in library/stdtypes with the > other formatting operations. It's new in 3.x and maps to the ascii() builtin: >>> ascii('h?') "'h\\xe9'" >>> '%a' % 'h?' "'h\\xe9'" The docstring for ascii(): ascii(object) -> string As repr(), return a string containing a printable representation of an object, but escape the non-ASCII characters in the string returned by repr() using \x, \u or \U escapes. This generates a string similar to that returned by repr() in Python 2. Regards Antoine. From eliben at gmail.com Wed Jul 27 15:54:34 2011 From: eliben at gmail.com (Eli Bendersky) Date: Wed, 27 Jul 2011 16:54:34 +0300 Subject: [Python-Dev] [Python-checkins] cpython: Issue #11049: adding some tests to test.support In-Reply-To: <20110727154407.4f3b2a69@pitrou.net> References: <4E2EBF37.7020702@netwok.org> <20110726153025.66b0de8b@pitrou.net> <20110727154407.4f3b2a69@pitrou.net> Message-ID: On Wed, Jul 27, 2011 at 16:44, Antoine Pitrou wrote: > On Wed, 27 Jul 2011 16:31:54 +0300 > Eli Bendersky wrote: > > > > I wasn't aware of '%a' at all? It doesn't appear to be documented, and > > Python 2.6 doesn't support it: > > > > ValueError: unsupported format character 'a' (0x61) at index 1 > > > > If it's new, it should at least be documented in library/stdtypes with > the > > other formatting operations. > > It's new in 3.x and maps to the ascii() builtin: > > >>> ascii('h?') > "'h\\xe9'" > >>> '%a' % 'h?' > "'h\\xe9'" > > The docstring for ascii(): > > ascii(object) -> string > > As repr(), return a string containing a printable representation of > an object, but escape the non-ASCII characters in the string > returned by repr() using \x, \u or \U escapes. This generates a > string similar to that returned by repr() in Python 2. > > Thanks. I also saw this documented in the {!a} formatting documentation. Was it left out of the library/stdtypes doc on purpose (to encourage the new str.format), or is this an omission? Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From eliben at gmail.com Wed Jul 27 15:58:53 2011 From: eliben at gmail.com (Eli Bendersky) Date: Wed, 27 Jul 2011 16:58:53 +0300 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: <20110727133619.F0B622506ED@webabinitio.net> References: <20110727133619.F0B622506ED@webabinitio.net> Message-ID: > > 2. Some functions like unlink and rmtree are obviously redundant, and > shadow > > frequently used Python stdlib functions, so I would either kill them > > completely or at least rename them appropriately. > > But they aren't redundant, since the test.support versions ignore > errors. > As I mentioned elsewhere, it's not good practice to have two functions with the same name doing something slightly different, in different modules in the code-base. > > Perhaps what we could do is move the documentation for test.support to > the devguide, and then vet the test suite so that unlink and friends > are always called as 'support.unlink', etc. > > Moving the documentation to the devguide is a good compromise between not documenting them at all and placing the documentation in a user-visible location. What do you mean by vetting the test suite so that unlink is always taken from test.support? I suppose some tests would specifically want the original unlink's functionality. In fact, at least a few tests use os.unlink exlicitly. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Wed Jul 27 15:59:11 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 27 Jul 2011 15:59:11 +0200 Subject: [Python-Dev] [Python-checkins] cpython: Issue #11049: adding some tests to test.support In-Reply-To: References: <4E2EBF37.7020702@netwok.org> <20110726153025.66b0de8b@pitrou.net> <20110727154407.4f3b2a69@pitrou.net> Message-ID: <1311775151.3583.0.camel@localhost.localdomain> Le mercredi 27 juillet 2011 ? 16:54 +0300, Eli Bendersky a ?crit : > > Thanks. I also saw this documented in the {!a} formatting > documentation. > > Was it left out of the library/stdtypes doc on purpose (to encourage > the new str.format), or is this an omission? Certainly an omission. Regards Antoine. From ezio.melotti at gmail.com Wed Jul 27 16:09:58 2011 From: ezio.melotti at gmail.com (Ezio Melotti) Date: Wed, 27 Jul 2011 17:09:58 +0300 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: References: <20110727152410.22871784@pitrou.net> Message-ID: <4E301C36.1040103@gmail.com> An HTML attachment was scrubbed... URL: From eliben at gmail.com Wed Jul 27 16:27:03 2011 From: eliben at gmail.com (Eli Bendersky) Date: Wed, 27 Jul 2011 17:27:03 +0300 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: <4E301C36.1040103@gmail.com> References: <20110727152410.22871784@pitrou.net> <4E301C36.1040103@gmail.com> Message-ID: > Initially I was *for* documenting, but this thing with showing up in the > index is a compelling counter-point. > > > "The basic version makes entries in the general index; if no index entry is > desired, you can give the directive option flag :noindex:." ( > http://docs.python.org/documenting/markup.html#information-units) > Ezio, this is also a good idea, but currently I really think placing this documentation in the devguide is probably the best approach. Now we have a very nice Devguide, and this documentation simply belongs there, and not in the user-visible portion of the official Python documentation. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From eliben at gmail.com Wed Jul 27 16:29:45 2011 From: eliben at gmail.com (Eli Bendersky) Date: Wed, 27 Jul 2011 17:29:45 +0300 Subject: [Python-Dev] [Python-checkins] cpython: Issue #11049: adding some tests to test.support In-Reply-To: <1311775151.3583.0.camel@localhost.localdomain> References: <4E2EBF37.7020702@netwok.org> <20110726153025.66b0de8b@pitrou.net> <20110727154407.4f3b2a69@pitrou.net> <1311775151.3583.0.camel@localhost.localdomain> Message-ID: > Thanks. I also saw this documented in the {!a} formatting > > documentation. > > > > Was it left out of the library/stdtypes doc on purpose (to encourage > > the new str.format), or is this an omission? > > Certainly an omission. > > Alright, I created issue 12644 as a reminder to add this. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdmurray at bitdance.com Wed Jul 27 17:11:10 2011 From: rdmurray at bitdance.com (R. David Murray) Date: Wed, 27 Jul 2011 11:11:10 -0400 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: References: <20110727133619.F0B622506ED@webabinitio.net> Message-ID: <20110727151111.4F3C82506ED@webabinitio.net> On Wed, 27 Jul 2011 16:58:53 +0300, Eli Bendersky wrote: > R. David Murray wrote: > > But they aren't redundant, since the test.support versions ignore > > errors. > > As I mentioned elsewhere, it's not good practice to have two functions with > the same name doing something slightly different, in different modules in > the code-base. Well, that would seem to be a matter of opinion. I see your point, but I'm not sure that I agree. But see below. > What do you mean by vetting the test suite so that unlink is always taken > from test.support? I suppose some tests would specifically want the original > unlink's functionality. In fact, at least a few tests use os.unlink > exlicitly. What I mean is that if the test code always did: import support [...] support.unlink('testtempfile') then there would be no confusion when someone grepped the code for 'unlink' or was reading the code without having noticed the import. That is, give the functions a unique name by using the 'support' name space explicitly, rather than by renaming them within the module. -- R. David Murray http://www.bitdance.com From senthil at uthcode.com Wed Jul 27 17:18:45 2011 From: senthil at uthcode.com (Senthil Kumaran) Date: Wed, 27 Jul 2011 23:18:45 +0800 Subject: [Python-Dev] cpython (3.2): Fix closes Issue12576 - fix urlopen behavior on sites which do not send (or In-Reply-To: <20110727142159.65e25267@pitrou.net> References: <20110727115232.5bbf7aea@pitrou.net> <20110727121600.GA3140@mathmagic> <20110727142159.65e25267@pitrou.net> Message-ID: <20110727151845.GA2206@mathmagic> On Wed, Jul 27, 2011 at 02:21:59PM +0200, Antoine Pitrou wrote: > transient_internet doesn't silence ValueError at all. Yes, that is correct. I missed recollecting it in the first place. I guess, I did not see using a content manager withing another context manager block as something nice.. (nothing wrong but just the indented code with additional wrap of exceptional handler). But yes, it would be better to wrap it with transient_internet call. Shall do. > Well, the test should simply call close() as is done in other tests. I tried that before responding, it does not silence the Resource Warning. The fix (at least for these cases where Connection:close header is not sent) lies in closing request in the urllib.request. It is tricky as the server is not sending a Connection:close header which httplib relies on to close the socket. -- Senthil From tjreedy at udel.edu Wed Jul 27 19:07:24 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 27 Jul 2011 13:07:24 -0400 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: <20110727152410.22871784@pitrou.net> References: <20110727152410.22871784@pitrou.net> Message-ID: On 7/27/2011 9:24 AM, Antoine Pitrou wrote: > On Wed, 27 Jul 2011 16:14:40 +0300 > Eli Bendersky wrote: >> Will it take long for newbie code to appear with the test.support version? >> Not to mention that grepping code that imports the "unlink" function >> directly doesn't reveal which one is being used. >> >> I think this is troublesome. I think at least two separate actions are >> required here: >> >> 1. In the documentation of test.support mention explicitly that it's code >> for CPython's internal use only, and these APIs aren't guaranteed to be >> stable. > > There is a top-level note at > http://docs.python.org/dev/library/test.html, but it won't be visible > by people who arrive at an anchor point. I think the 'Note' (gray box), not a 'Warning' (red box) should be repeated at the top of the test.support section. Or, or in addition, footnote each entry (with same number jumping to same footnote, if this can be done): "test.support.verbose(1). Seeing every entry decorated with the same footnote number should indicate there there is something unusual, and that we really mean it. > I think officially documenting test.support is a mistake. There is no > guarantee that APIs are stable or will even continue to exist. > Docstrings are sufficient for own our purposes. Maybe sufficient for you, but if I am to learn and use this stuff, I need a proper listing. Individual docstrings require that you know the object exists and its name. Any help(module) listing is harder for me to use than the doc chapter. >> 2. Some functions like unlink and rmtree are obviously redundant, and shadow >> frequently used Python stdlib functions, so I would either kill them >> completely or at least rename them appropriately. > > They are not redundant, since they provide slightly different semantics > (for example, they silence errors that happen when the path doesn't > exist). rmtree has an ignore_errors=False parameter "ignore_errors is true, errors resulting from failed removals will be ignored; ". If that does not ignore enough errors, then perhaps it should be changed. os.unlink is an alias for os.remove. The doc for the latter says error if path is a directory or a file in use on Windows (but not *nix). Neither should be case in test uses. Doc does not specify errors if file not exist or other conditions (inadequate permission. Being thin wrappers, a quiet param is not allowed in os.remove/unlink, though I can imagine others wanting a quieted version for removal of files whose creation might have failed. In anycase, naming the quieted version in test.support unlink_quiet or q_remove or something would make its reason-for-being and use clearer. --- Side note: test.support.import_fresh_module typo. /is/if/ in "This function will raise unittest.SkipTest is the named module cannot be imported." -- Terry Jan Reedy From tjreedy at udel.edu Wed Jul 27 19:17:59 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 27 Jul 2011 13:17:59 -0400 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: References: <20110727152410.22871784@pitrou.net> <4E301C36.1040103@gmail.com> Message-ID: On 7/27/2011 10:27 AM, Eli Bendersky wrote: > >> Initially I was *for* documenting, but this thing with showing up >> in the index is a compelling counter-point. > > "The basic version makes entries in the general index; if no index > entry is desired, you can give the directive option flag :noindex:." > (http://docs.python.org/documenting/markup.html#information-units) > > > Ezio, this is also a good idea, but currently I really think placing > this documentation in the devguide is probably the best approach. Now we > have a very nice Devguide, and this documentation simply belongs there, > and not in the user-visible portion of the official Python documentation. You mean the dev guide only accessible online? It could be included in HowTos (Help develop Python). If you are going to un-index the section, whereever it is, please put the entries in alpha order instead of the current jumble. Or is the current order somehow makes sense, an alphabetical index at the top. -- Terry Jan Reedy From brett at python.org Wed Jul 27 19:27:16 2011 From: brett at python.org (Brett Cannon) Date: Wed, 27 Jul 2011 10:27:16 -0700 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: <20110727133619.F0B622506ED@webabinitio.net> References: <20110727133619.F0B622506ED@webabinitio.net> Message-ID: On Wed, Jul 27, 2011 at 06:36, R. David Murray wrote: > On Wed, 27 Jul 2011 16:14:40 +0300, Eli Bendersky > wrote: > > 1. In the documentation of test.support mention explicitly that it's code > > for CPython's internal use only, and these APIs aren't guaranteed to be > > stable. > > This was already done. > > > 2. Some functions like unlink and rmtree are obviously redundant, and > shadow > > frequently used Python stdlib functions, so I would either kill them > > completely or at least rename them appropriately. > > But they aren't redundant, since the test.support versions ignore > errors. > > Perhaps what we could do is move the documentation for test.support to > the devguide, and then vet the test suite so that unlink and friends > are always called as 'support.unlink', etc. > I like this solution since this issue of documenting test.support keeps coming up. Otherwise we can not document test.support, but then we need to do a pass through the module to make sure that the docstrings are properly updated and we start deprecating some of the stuff in there that is just pure cruft. -------------- next part -------------- An HTML attachment was scrubbed... URL: From eliben at gmail.com Wed Jul 27 19:27:52 2011 From: eliben at gmail.com (Eli Bendersky) Date: Wed, 27 Jul 2011 20:27:52 +0300 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: References: <20110727152410.22871784@pitrou.net> <4E301C36.1040103@gmail.com> Message-ID: > Ezio, this is also a good idea, but currently I really think placing >> this documentation in the devguide is probably the best approach. Now we >> have a very nice Devguide, and this documentation simply belongs there, >> and not in the user-visible portion of the official Python documentation. >> > > You mean the dev guide only accessible online? Yes. You can also pull it from http://hg.python.org/devguide/ for a local copy (hg.python.org also allows to download a ZIP). My point being - isn't the official Python documentation targeted at *users* of Python, and wasn't the devguide specifically created for *developers* of Python? If so, then test.support clearly being the domain of developers rather than users, belongs in the devguide. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From eliben at gmail.com Wed Jul 27 19:31:58 2011 From: eliben at gmail.com (Eli Bendersky) Date: Wed, 27 Jul 2011 20:31:58 +0300 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: References: <20110727152410.22871784@pitrou.net> Message-ID: > --- > Side note: test.support.import_fresh_**module typo. /is/if/ in > "This function will raise unittest.SkipTest is the named module cannot be > imported." > Fixed in 8989aa5b357c Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Wed Jul 27 19:44:23 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 27 Jul 2011 13:44:23 -0400 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: <20110727152410.22871784@pitrou.net> References: <20110727152410.22871784@pitrou.net> Message-ID: On 7/27/2011 9:24 AM, Antoine Pitrou wrote: > Docstrings are sufficient for own our purposes. >>> import test.support as t >>> help(t.rmtree) Help on function rmtree in module test.support: rmtree(path) ;-) -- Terry Jan Reedy From tjreedy at udel.edu Wed Jul 27 19:47:13 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 27 Jul 2011 13:47:13 -0400 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: References: <20110727133619.F0B622506ED@webabinitio.net> Message-ID: On 7/27/2011 1:27 PM, Brett Cannon wrote: > Perhaps what we could do is move the documentation for test.support to > the devguide, and then vet the test suite so that unlink and friends > are always called as 'support.unlink', etc. > > > I like this solution since this issue of documenting test.support keeps > coming up. Otherwise we can not document test.support, We already do. 25.6. test.support ? Utility functions for tests is about half of the page that also contains 25.5. test ? Regression tests package for Python The latter contains 25.5.1. Writing Unit Tests for the test package which should also be moved to the dev guide if 25.6 is. That would leave 25.5 as a short page explaining what lib/test is and how to run the tests, which is something user sometimes need to do. > but then we need > to do a pass through the module to make sure that the docstrings are > properly updated and we start deprecating some of the stuff in there > that is just pure cruft. I believe that is what Eli is doing and hence the suggestion to dump .rmtree. Agreed that missing doc strings should be 'updated' from ''. -- Terry Jan Reedy From eliben at gmail.com Wed Jul 27 19:57:48 2011 From: eliben at gmail.com (Eli Bendersky) Date: Wed, 27 Jul 2011 20:57:48 +0300 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: References: <20110727133619.F0B622506ED@webabinitio.net> Message-ID: > I like this solution since this issue of documenting test.support keeps >> coming up. Otherwise we can not document test.support, >> > > We already do. > > 25.6. test.support ? Utility functions for tests > is about half of the page that also contains > 25.5. test ? Regression tests package for Python > The latter contains > 25.5.1. Writing Unit Tests for the test package > which should also be moved to the dev guide if 25.6 is. > > +1 > That would leave 25.5 as a short page explaining what lib/test is and how > to run the tests, which is something user sometimes need to do. > > Out of curiosity, why would a user need to run Python's tests? Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From ezio.melotti at gmail.com Wed Jul 27 19:46:36 2011 From: ezio.melotti at gmail.com (Ezio Melotti) Date: Wed, 27 Jul 2011 20:46:36 +0300 Subject: [Python-Dev] [Python-checkins] cpython: fix doc typo for library/test.rst In-Reply-To: References: Message-ID: <4E304EFC.6000309@gmail.com> Hi, On 27/07/2011 20.31, eli.bendersky wrote: > http://hg.python.org/cpython/rev/8989aa5b357c > changeset: 71532:8989aa5b357c > user: Eli Bendersky > date: Wed Jul 27 20:29:59 2011 +0300 > summary: > fix doc typo for library/test.rst > > files: > Doc/library/test.rst | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > > diff --git a/Doc/library/test.rst b/Doc/library/test.rst > --- a/Doc/library/test.rst > +++ b/Doc/library/test.rst > @@ -447,7 +447,7 @@ > Module and package deprecation messages are suppressed during this import > if *deprecated* is ``True``. > > - This function will raise :exc:`unittest.SkipTest` is the named module > + This function will raise :exc:`unittest.SkipTest` if the named module Actually I think this is no longer true. import_fresh_module raises an ImportError if *name* can't be imported, or returns None if the fresh module is not found. Its use case is to enable or block accelerations for modules that optionally provide one. All the modules that currently use import_fresh_module are (afaik) always available (json, warnings, heapq), so raising SkipTest when the module is missing is not useful now. It returns None in the case an acceleration is missing, so e.g. in "cjson = import_fresh_module('json', fresh=['_json'])" cjson will be None and it will be possible to do things like @skipUnless(cjson, 'requires _json'). Here raising an ImportError will defeat (part of) the purpose of the function, i.e. avoiding: try: import _json except ImportError: _json = None and raising a SkipTest when the accelerations are missing is not an option if there are other tests (e.g. the tests for the Python implementation). These changes come from http://hg.python.org/cpython/rev/c1a12a308c5b . Before the change import_fresh_module was still returning the module (e.g. json) even when the acceleration (fresh=['_json']) was missing, and the C tests were run twice using the same pure-python module used for the Py ones. The typo and the wrong doc is also on 2.7. Best Regards, Ezio Melotti > cannot be imported. > > Example use:: > From eliben at gmail.com Wed Jul 27 20:04:10 2011 From: eliben at gmail.com (Eli Bendersky) Date: Wed, 27 Jul 2011 21:04:10 +0300 Subject: [Python-Dev] [Python-checkins] cpython: fix doc typo for library/test.rst In-Reply-To: <4E304EFC.6000309@gmail.com> References: <4E304EFC.6000309@gmail.com> Message-ID: Actually I think this is no longer true. import_fresh_module raises an ImportError if *name* can't be imported, or returns None if the fresh module is not found. > > Its use case is to enable or block accelerations for modules that > optionally provide one. All the modules that currently use > import_fresh_module are (afaik) always available (json, warnings, heapq), so > raising SkipTest when the module is missing is not useful now. > It returns None in the case an acceleration is missing, so e.g. in "cjson = > import_fresh_module('json', fresh=['_json'])" cjson will be None and it will > be possible to do things like @skipUnless(cjson, 'requires _json'). Here > raising an ImportError will defeat (part of) the purpose of the function, > i.e. avoiding: > try: > import _json > except ImportError: > _json = None > > and raising a SkipTest when the accelerations are missing is not an option > if there are other tests (e.g. the tests for the Python implementation). > > These changes come from http://hg.python.org/cpython/**rev/c1a12a308c5b. Before the change import_fresh_module was still returning the module > (e.g. json) even when the acceleration (fresh=['_json']) was missing, and > the C tests were run twice using the same pure-python module used for the Py > ones. > > The typo and the wrong doc is also on 2.7. > > Ezio, thanks. I opened issue 12645 to track this. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From tseaver at palladion.com Wed Jul 27 20:08:30 2011 From: tseaver at palladion.com (Tres Seaver) Date: Wed, 27 Jul 2011 14:08:30 -0400 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: References: <20110727133619.F0B622506ED@webabinitio.net> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 07/27/2011 01:57 PM, Eli Bendersky wrote: > Out of curiosity, why would a user need to run Python's tests? A couple of cases occur to me: - - To verify that they got a corrrect build with all expected modules included. - - To test the build after updating an underlying system library. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk4wVB4ACgkQ+gerLs4ltQ5NSACfXg7QoZAFNHWehT81oPTUwzoo +m4AnRMQFcGod1/3XxEk/T6CDVpE7i7c =/VoI -----END PGP SIGNATURE----- From tjreedy at udel.edu Wed Jul 27 20:12:45 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 27 Jul 2011 14:12:45 -0400 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: References: <20110727133619.F0B622506ED@webabinitio.net> Message-ID: On 7/27/2011 1:57 PM, Eli Bendersky wrote: > Out of curiosity, why would a user need to run Python's tests? If one compiles Python, running the tests is essential. Some people like to run a test suite to verify an installation. Sometimes people have problems that might arise from an installation getting messed up. Whatever is left of 25.5 in the public doc, I think *all* the info should be in the dev guide. -- Terry Jan Reedy From ethan at stoneleaf.us Wed Jul 27 20:40:58 2011 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 27 Jul 2011 11:40:58 -0700 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: References: <20110727133619.F0B622506ED@webabinitio.net> Message-ID: <4E305BBA.5080804@stoneleaf.us> Eli Bendersky wrote: > > I like this solution since this issue of documenting > test.support keeps > coming up. Otherwise we can not document test.support, > > > We already do. > > 25.6. test.support ? Utility functions for tests > is about half of the page that also contains > 25.5. test ? Regression tests package for Python > The latter contains > 25.5.1. Writing Unit Tests for the test package > which should also be moved to the dev guide if 25.6 is. > > > +1 > > > That would leave 25.5 as a short page explaining what lib/test is > and how to run the tests, which is something user sometimes need to do. > > > Out of curiosity, why would a user need to run Python's tests? Curiosity. ;) ~Ethan~ From barry at python.org Wed Jul 27 00:32:50 2011 From: barry at python.org (Barry Warsaw) Date: Tue, 26 Jul 2011 18:32:50 -0400 Subject: [Python-Dev] hard linking executables In-Reply-To: <20110727001912.4cf67481@pitrou.net> References: <20110724231655.0b69d36a@pitrou.net> <20110724232106.53161528@pitrou.net> <1311696328.3648.0.camel@localhost.localdomain> <4E2EE813.1080407@netwok.org> <20110727001912.4cf67481@pitrou.net> Message-ID: <20110726183250.03932089@resist.wooz.org> On Jul 27, 2011, at 12:19 AM, Antoine Pitrou wrote: >Ok, apparently the decision to make hard links for executables dates at >least back to: That still doesn't explain *why* hardlinks were originally chosen instead of symlinks. In the absence of any other compelling argument against it, I think they should all consistently be symlinks. I don't see any Ubuntu or Debian (where /usr/bin/python3 -> python3.2) bug reports indicating any problems, and I haven't experienced any issues with it personally. >changeset: 16221:588691f806f4 >branch: legacy-trunk >user: Neil Schemenauer >date: Wed Jan 24 17:11:43 2001 +0000 >files: Makefile.pre.in >description: >Flat makefile based on toplevel Makefile.in and makefiles in build >subdirectories. Those other makefiles will go away eventually. > >[...] > >+# Install the interpreter (by creating a hard link to python$(VERSION)) >+bininstall: altbininstall >+ -if test -f $(BINDIR)/$(PYTHON); \ >+ then rm -f $(BINDIR)/$(PYTHON); \ >+ else true; \ >+ fi >+ (cd $(BINDIR); $(LN) python$(VERSION)$(EXEEXT) python$(EXEEXT)) >+ -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Wed Jul 27 00:32:50 2011 From: barry at python.org (Barry Warsaw) Date: Tue, 26 Jul 2011 18:32:50 -0400 Subject: [Python-Dev] hard linking executables In-Reply-To: <20110727001912.4cf67481@pitrou.net> References: <20110724231655.0b69d36a@pitrou.net> <20110724232106.53161528@pitrou.net> <1311696328.3648.0.camel@localhost.localdomain> <4E2EE813.1080407@netwok.org> <20110727001912.4cf67481@pitrou.net> Message-ID: <20110726183250.03932089@resist.wooz.org> On Jul 27, 2011, at 12:19 AM, Antoine Pitrou wrote: >Ok, apparently the decision to make hard links for executables dates at >least back to: That still doesn't explain *why* hardlinks were originally chosen instead of symlinks. In the absence of any other compelling argument against it, I think they should all consistently be symlinks. I don't see any Ubuntu or Debian (where /usr/bin/python3 -> python3.2) bug reports indicating any problems, and I haven't experienced any issues with it personally. >changeset: 16221:588691f806f4 >branch: legacy-trunk >user: Neil Schemenauer >date: Wed Jan 24 17:11:43 2001 +0000 >files: Makefile.pre.in >description: >Flat makefile based on toplevel Makefile.in and makefiles in build >subdirectories. Those other makefiles will go away eventually. > >[...] > >+# Install the interpreter (by creating a hard link to python$(VERSION)) >+bininstall: altbininstall >+ -if test -f $(BINDIR)/$(PYTHON); \ >+ then rm -f $(BINDIR)/$(PYTHON); \ >+ else true; \ >+ fi >+ (cd $(BINDIR); $(LN) python$(VERSION)$(EXEEXT) python$(EXEEXT)) >+ -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Wed Jul 20 15:41:02 2011 From: barry at python.org (Barry Warsaw) Date: Wed, 20 Jul 2011 09:41:02 -0400 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <20110720131415.2644B3A409B@sparrow.telecommunity.com> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <20110720131415.2644B3A409B@sparrow.telecommunity.com> Message-ID: <20110720094102.1f84a733@resist.wooz.org> First off, kudos to PJE for his work on this PEP. He really had the key insight for this new approach, and did a great job of explaining his vision in a clear way so that I think everybody over on import-sig "got it". On Jul 20, 2011, at 08:57 AM, P.J. Eby wrote: >At 06:46 PM 7/20/2011 +1000, Nick Coghlan wrote: >>On Wed, Jul 20, 2011 at 1:58 PM, P.J. Eby wrote: >> > So, without further ado, here it is: >> >>I pushed this version up to the PEPs repo, so it now has a number >>(402) and can be read in prettier HTML format: >>http://www.python.org/dev/peps/pep-0402/ > >Technically, shouldn't this be a 3XXX series PEP? Or are we not doing those >any more now that all PEPs would be 3XXX? Great question. I don't know if we want/need to make the distinction any more. It does feel a little odd putting Python 3 PEPs (the only kind of new Standards Track PEPs) in the 0XXX numbers, but now that we're all moving to Python 3 , it seems like segregating new PEPs to the 3XXX range is a bit contrived. I think filling up 0XXX is probably fine. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Wed Jul 20 00:45:46 2011 From: barry at python.org (Barry Warsaw) Date: Tue, 19 Jul 2011 18:45:46 -0400 Subject: [Python-Dev] [Email-SIG] email-6.0.0.a1 In-Reply-To: <20110719212139.D5D732500D5@webabinitio.net> References: <20110719212139.D5D732500D5@webabinitio.net> Message-ID: <20110719184546.4eb8f52a@resist.wooz.org> On Jul 19, 2011, at 05:21 PM, R. David Murray wrote: >OK, so I've released the first iteration of the email6 package on pypi >as email-6.0.0a1. After install you import it as email6. This will >allow anyone curious and/or motivated to test it out under Python 3.2. >I'm especially interested in anyone with a working program that uses >email in 3.2: it should be completely backward compatible, and if it >isn't I want to know ASAP.[*] It'll take some time to digest, but congratulations RDM! You've accomplished an impressive milestone. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From drsalists at gmail.com Wed Jul 27 22:00:05 2011 From: drsalists at gmail.com (Dan Stromberg) Date: Wed, 27 Jul 2011 13:00:05 -0700 Subject: [Python-Dev] hard linking executables In-Reply-To: <20110726183250.03932089@resist.wooz.org> References: <20110724231655.0b69d36a@pitrou.net> <20110724232106.53161528@pitrou.net> <1311696328.3648.0.camel@localhost.localdomain> <4E2EE813.1080407@netwok.org> <20110727001912.4cf67481@pitrou.net> <20110726183250.03932089@resist.wooz.org> Message-ID: It's been suggested that *ix has hardlinks because someone thought up hardlinks before someone thought up symlinks - IOW, there are those who suggest that if people had added symlinks first, no one would've bothered adding hardlinks. Symlinks are almost always more flexible, and almost always more clear. The main counterexample seems to be rsync-based backup systems, that will hardlink identical files of a given pathname. But that seems to be a bit of a mess when it comes time to transfer such a backup from one filesystem to another. On Tue, Jul 26, 2011 at 3:32 PM, Barry Warsaw wrote: > On Jul 27, 2011, at 12:19 AM, Antoine Pitrou wrote: > > >Ok, apparently the decision to make hard links for executables dates at > >least back to: > > That still doesn't explain *why* hardlinks were originally chosen instead > of > symlinks. In the absence of any other compelling argument against it, I > think > they should all consistently be symlinks. I don't see any Ubuntu or Debian > (where /usr/bin/python3 -> python3.2) bug reports indicating any problems, > and > I haven't experienced any issues with it personally. > > >changeset: 16221:588691f806f4 > >branch: legacy-trunk > >user: Neil Schemenauer > >date: Wed Jan 24 17:11:43 2001 +0000 > >files: Makefile.pre.in > >description: > >Flat makefile based on toplevel Makefile.in and makefiles in build > >subdirectories. Those other makefiles will go away eventually. > > > >[...] > > > >+# Install the interpreter (by creating a hard link to python$(VERSION)) > >+bininstall: altbininstall > >+ -if test -f $(BINDIR)/$(PYTHON); \ > >+ then rm -f $(BINDIR)/$(PYTHON); \ > >+ else true; \ > >+ fi > >+ (cd $(BINDIR); $(LN) python$(VERSION)$(EXEEXT) python$(EXEEXT)) > >+ > > -Barry > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/drsalists%40gmail.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Wed Jul 27 22:06:09 2011 From: guido at python.org (Guido van Rossum) Date: Wed, 27 Jul 2011 13:06:09 -0700 Subject: [Python-Dev] hard linking executables In-Reply-To: <20110726183250.03932089@resist.wooz.org> References: <20110724231655.0b69d36a@pitrou.net> <20110724232106.53161528@pitrou.net> <1311696328.3648.0.camel@localhost.localdomain> <4E2EE813.1080407@netwok.org> <20110727001912.4cf67481@pitrou.net> <20110726183250.03932089@resist.wooz.org> Message-ID: On Tue, Jul 26, 2011 at 3:32 PM, Barry Warsaw wrote: > On Jul 27, 2011, at 12:19 AM, Antoine Pitrou wrote: > >>Ok, apparently the decision to make hard links for executables dates at >>least back to: > > That still doesn't explain *why* hardlinks were originally chosen instead of > symlinks. ?In the absence of any other compelling argument against it, I think > they should all consistently be symlinks. ?I don't see any Ubuntu or Debian > (where /usr/bin/python3 -> python3.2) bug reports indicating any problems, and > I haven't experienced any issues with it personally. I ought to remember why because I remember I was involved. (And I have a feeling that the change Antoine dug up was just a refactoring, moving an existing hard-link target to a different file. Because in my memory the hard link was my idea and I implemented it. But that change is Neil Schemenauer's.) But I can't. :-( The best I can come up with is that when it's a hard link, you can replace either "python" or "python2.1" with another version without disturbing the other. And I think it has something to do with altinstall vs. install. So it would mean that after first installing 2.1 (so python and python2.1 are the same file), and then alt-installing 2.1.1, the original 2.1 binary is still accessible as "python". But I don't know why you'd want that. -- --Guido van Rossum (python.org/~guido) From solipsis at pitrou.net Wed Jul 27 22:34:46 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 27 Jul 2011 22:34:46 +0200 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions References: <20110727133619.F0B622506ED@webabinitio.net> Message-ID: <20110727223446.5b83a48f@pitrou.net> On Wed, 27 Jul 2011 10:27:16 -0700 Brett Cannon wrote: > > > > Perhaps what we could do is move the documentation for test.support to > > the devguide, and then vet the test suite so that unlink and friends > > are always called as 'support.unlink', etc. > > > > I like this solution since this issue of documenting test.support keeps > coming up. Otherwise we can not document test.support, but then we need to > do a pass through the module to make sure that the docstrings are properly > updated and we start deprecating some of the stuff in there that is just > pure cruft. We don't need to deprecate that cruft, we can just remove it (and all uses of it, of course). Regards Antoine. From solipsis at pitrou.net Wed Jul 27 22:36:38 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 27 Jul 2011 22:36:38 +0200 Subject: [Python-Dev] cpython (2.7): - Issue #12603: Fix pydoc.synopsis() on files with non-negative st_mtime. References: Message-ID: <20110727223638.1bafb8e8@pitrou.net> On Wed, 27 Jul 2011 19:36:36 +0200 charles-francois.natali wrote: > > +- Issue #12603: Fix pydoc.synopsis() on files with non-negative st_mtime. > + Surely you mean non-positive? Non-negative st_mtime being the common case. Regards Antoine. From cf.natali at gmail.com Wed Jul 27 22:55:14 2011 From: cf.natali at gmail.com (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=) Date: Wed, 27 Jul 2011 22:55:14 +0200 Subject: [Python-Dev] cpython (2.7): - Issue #12603: Fix pydoc.synopsis() on files with non-negative st_mtime. In-Reply-To: <20110727223638.1bafb8e8@pitrou.net> References: <20110727223638.1bafb8e8@pitrou.net> Message-ID: >> +- Issue #12603: Fix pydoc.synopsis() on files with non-negative >> st_mtime. >> + > > Surely you mean non-positive? Non-negative st_mtime being the common > case. Of course (st_mtime <= 0). From ben+python at benfinney.id.au Wed Jul 27 23:37:47 2011 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 28 Jul 2011 07:37:47 +1000 Subject: [Python-Dev] hard linking executables References: <20110724231655.0b69d36a@pitrou.net> <20110724232106.53161528@pitrou.net> <1311696328.3648.0.camel@localhost.localdomain> <4E2EE813.1080407@netwok.org> <20110727001912.4cf67481@pitrou.net> <20110726183250.03932089@resist.wooz.org> Message-ID: <87k4b3pfpw.fsf@benfinney.id.au> Dan Stromberg writes: > It's been suggested that *ix has hardlinks because someone thought up > hardlinks before someone thought up symlinks - IOW, there are those who > suggest that if people had added symlinks first, no one would've bothered > adding hardlinks. Well, that suggestion is faulty. It ignores the fact that *all* ordinary files on Unix are hard links. Any ordinary file entry in a directory is a hard link to the file's data. The ?hard links? capability, therefore, isn't something that was added; it's fundamental to Unix filesystems from their inception. The ?ln? command adds *additional* hard links to an existing file's data; but, once added, they're exactly the same as any other ordinary file entry. > Symlinks are almost always more flexible, and almost always more > clear. Yet many tools don't work as expected with symbolic links which will work with an ordinary file (a ?hard link?). One can argue that such tools are to that extent buggy, but symbolic links produce deliberately different behaviour which is sometimes not what one needs. -- \ ?Wrinkles should merely indicate where smiles have been.? ?Mark | `\ Twain, _Following the Equator_ | _o__) | Ben Finney From victor.stinner at haypocalc.com Wed Jul 27 23:53:20 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Wed, 27 Jul 2011 23:53:20 +0200 Subject: [Python-Dev] Status of the PEP 400? (deprecate codecs.StreamReader/StreamWriter) Message-ID: <4E3088D0.5040900@haypocalc.com> Hi, Three weeks ago, I posted a draft on my PEP on this mailing list. I tried to include all remarks you made, and the PEP is now online: http://www.python.org/dev/peps/pep-0400/ It's now unclear to me if the PEP will be accepted or rejected. I don't know what to do to move forward. I asked Guido in private, but I didn't get any answer. Victor From victor.stinner at haypocalc.com Thu Jul 28 00:12:51 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Thu, 28 Jul 2011 00:12:51 +0200 Subject: [Python-Dev] Status of the PEP 400? (deprecate codecs.StreamReader/StreamWriter) Message-ID: <4E308D63.9090901@haypocalc.com> Hi, Three weeks ago, I posted a draft on my PEP on this mailing list. I tried to include all remarks you made, and the PEP is now online: http://www.python.org/dev/peps/pep-0400/ It's now unclear to me if the PEP will be accepted or rejected. I don't know what to do to move forward. Victor From guido at python.org Thu Jul 28 00:36:52 2011 From: guido at python.org (Guido van Rossum) Date: Wed, 27 Jul 2011 15:36:52 -0700 Subject: [Python-Dev] Status of the PEP 400? (deprecate codecs.StreamReader/StreamWriter) In-Reply-To: <4E3088D0.5040900@haypocalc.com> References: <4E3088D0.5040900@haypocalc.com> Message-ID: On Wed, Jul 27, 2011 at 2:53 PM, Victor Stinner wrote: > Hi, > > Three weeks ago, I posted a draft on my PEP on this mailing list. I tried to > include all remarks you made, and the PEP is now online: > > ? http://www.python.org/dev/peps/pep-0400/ > > It's now unclear to me if the PEP will be accepted or rejected. I don't know > what to do to move forward. I asked Guido in private, but I didn't get any > answer. > > Victor Sorry Victor, I somehow didn't see that message even though I received it (I probably thought it was a continuation of the python-dev thread which I've been ignoring). Unfortunately I don't have time to go back and read the whole thread. I think I haven't used codecs.StreamReader/Writer myself, and I don't think I've seen much use of them either (which doesn't mean there isn't). My gut feeling is that yes, they should eventually go away, but no, there's no particular hurry, and no, I don't think you should change codecs.open() to call io.open(). I think the best thing is to campaign (e.g. in docs) for people to stop using codecs.open/StreamReader/Writer and start deprecating them formally once we feel that most users have switched. It's possible that that could happen before 3.3 is released, but I'm kind of doubtful about that myself. Sorry again for missing your private emails! -- --Guido van Rossum (python.org/~guido) From ncoghlan at gmail.com Thu Jul 28 01:05:11 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 28 Jul 2011 09:05:11 +1000 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: References: <20110727133619.F0B622506ED@webabinitio.net> Message-ID: On Thu, Jul 28, 2011 at 3:27 AM, Brett Cannon wrote: > On Wed, Jul 27, 2011 at 06:36, R. David Murray > wrote: >> Perhaps what we could do is move the documentation for test.support to >> the devguide, and then vet the test suite so that unlink and friends >> are always called as 'support.unlink', etc. > > I like this solution since this issue of documenting test.support keeps > coming up. Otherwise we can not document test.support, but? then we need to > do a pass through the module to make sure? that the docstrings are properly > updated and we start deprecating some of the stuff in there that is just > pure cruft. +1 for relocating this to the devguide. The only reason this is in the main docs now is that it used to be the only way to easily get pretty documentation for it. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From victor.stinner at haypocalc.com Thu Jul 28 01:11:33 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Thu, 28 Jul 2011 01:11:33 +0200 Subject: [Python-Dev] Status of the PEP 400? (deprecate codecs.StreamReader/StreamWriter) In-Reply-To: References: <4E3088D0.5040900@haypocalc.com> Message-ID: <4E309B25.7090304@haypocalc.com> Le 28/07/2011 00:36, Guido van Rossum a ?crit : > Sorry Victor, I somehow didn't see that message even though I received > it (I probably thought it was a continuation of the python-dev thread > which I've been ignoring). No problem. > no, there's no particular hurry That's why it's a deprecation process and the removal is schedule for later: "3.4 (or maybe later)". I added "or maybe later" before reopening a new thread on this list. > no, I don't think you should change codecs.open() to call io.open() The PEP is useless without this change. If we don't deprecate any class and don't change codecs.open(), it's better to just reject the PEP. > start deprecating them formally once we feel that most users have switched Users of codecs.open() or users of codecs.Stream* classes? Victor From guido at python.org Thu Jul 28 01:38:26 2011 From: guido at python.org (Guido van Rossum) Date: Wed, 27 Jul 2011 16:38:26 -0700 Subject: [Python-Dev] Status of the PEP 400? (deprecate codecs.StreamReader/StreamWriter) In-Reply-To: <4E309B25.7090304@haypocalc.com> References: <4E3088D0.5040900@haypocalc.com> <4E309B25.7090304@haypocalc.com> Message-ID: On Wed, Jul 27, 2011 at 4:11 PM, Victor Stinner wrote: > Le 28/07/2011 00:36, Guido van Rossum a ?crit : >> >> Sorry Victor, I somehow didn't see that message even though I received >> it (I probably thought it was a continuation of the python-dev thread >> which I've been ignoring). > > No problem. > >> no, there's no particular hurry > > That's why it's a deprecation process and the removal is schedule for later: > "3.4 (or maybe later)". I added "or maybe later" before reopening a new > thread on this list. That still sounds fairly aggressive. >> no, I don't think you should change codecs.open() to call io.open() > > The PEP is useless without this change. If we don't deprecate any class and > don't change codecs.open(), it's better to just reject the PEP. Why? (Not that I am against rejecting the PEP. I feel weakly opinioned in this case, about -0.) >> start deprecating them formally once we feel that most users have switched > > Users of codecs.open() or users of codecs.Stream* classes? I would think both. Is there any reason to continue using codecs.open()? -- --Guido van Rossum (python.org/~guido) From steve at pearwood.info Thu Jul 28 01:53:21 2011 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 28 Jul 2011 09:53:21 +1000 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: References: <20110727152410.22871784@pitrou.net> Message-ID: <4E30A4F1.6050305@pearwood.info> Eli Bendersky wrote: > Sure, but I'm still leery of two functions with the same name doing acting > slightly differently. and then in a later post: > As I mentioned elsewhere, it's not good practice to have two functions with > the same name doing something slightly different, in different modules in > the code-base. artist.draw() and gunslinger.draw() do not necessarily need to do the same thing, and I don't agree that a (futile) preference for globally unique names is good practice: it can lead to unnecessarily long names (artist.draw_with_pencil_on_paper) or redundant characters prefixing the name (itertools.imap -- what does the "i" in imap tell you that the itertools didn't already?). Solving this problem is what namespaces are for. Renaming test.support.ulink to something else doesn't fix the problem of unsupported support functions being used in third-party code. It may even *encourage* it, by making the extra functionality more explicit. However, is there any reason why test.support itself shouldn't be renamed test._support, or possibly _test.support, so that the *entire* suite is marked as a private implementation detail? -- Steven From ncoghlan at gmail.com Thu Jul 28 03:00:09 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 28 Jul 2011 11:00:09 +1000 Subject: [Python-Dev] Status of the PEP 400? (deprecate codecs.StreamReader/StreamWriter) In-Reply-To: References: <4E3088D0.5040900@haypocalc.com> <4E309B25.7090304@haypocalc.com> Message-ID: On Thu, Jul 28, 2011 at 9:38 AM, Guido van Rossum wrote: >> Users of codecs.open() or users of codecs.Stream* classes? > > I would think both. Is there any reason to continue using codecs.open()? It's the easiest way to write Unicode friendly code that spans both 2.x and 3.x. The problem is that naive 2.x code will migrate to the optimised IO stack automatically on the 2.x -> 3.x transition, while code that tried to do the right thing has to be changed manually (either in 3.x only, or by switching to the io module for 2.x as well) in order to adjust for the differences in argument order. The idea behind changing codecs.open to be a wrapper around io.open was to allow such code to switch to the new optimised IO stack as easily as code that just uses the open builtin. If it's acceptable for the builtin behaviour to change (far more substantially), why not change codecs.open as well? Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From guido at python.org Thu Jul 28 04:05:18 2011 From: guido at python.org (Guido van Rossum) Date: Wed, 27 Jul 2011 19:05:18 -0700 Subject: [Python-Dev] Status of the PEP 400? (deprecate codecs.StreamReader/StreamWriter) In-Reply-To: References: <4E3088D0.5040900@haypocalc.com> <4E309B25.7090304@haypocalc.com> Message-ID: On Wed, Jul 27, 2011 at 6:00 PM, Nick Coghlan wrote: > On Thu, Jul 28, 2011 at 9:38 AM, Guido van Rossum wrote: >>> Users of codecs.open() or users of codecs.Stream* classes? >> >> I would think both. Is there any reason to continue using codecs.open()? > > It's the easiest way to write Unicode friendly code that spans both 2.x and 3.x. Even on 2.6, where the io module exists? > The problem is that naive 2.x code will migrate to the optimised IO > stack automatically on the 2.x -> 3.x transition, while code that > tried to do the right thing has to be changed manually (either in 3.x > only, or by switching to the io module for 2.x as well) in order to > adjust for the differences in argument order. > > The idea behind changing codecs.open to be a wrapper around io.open > was to allow such code to switch to the new optimised IO stack as > easily as code that just uses the open builtin. If it's acceptable for > the builtin behaviour to change (far more substantially), why not > change codecs.open as well? Aren't the cases different? Using built-in open() just means you want to open a file in the default way. Using codecs.open() presumably means that you've thought about Unicode. TBH, I said I was only -0 on the PEP, and if the stream returned by codecs.open() in 3.3 is sufficiently compatible with the stream returned the same function returns in 3.2, I am okay with it. (Except I also want you to cut a trillion dollars from the non-military budget, without raising taxes. :-) -- --Guido van Rossum (python.org/~guido) From drsalists at gmail.com Thu Jul 28 04:34:36 2011 From: drsalists at gmail.com (Dan Stromberg) Date: Wed, 27 Jul 2011 19:34:36 -0700 Subject: [Python-Dev] hard linking executables In-Reply-To: <87k4b3pfpw.fsf@benfinney.id.au> References: <20110724231655.0b69d36a@pitrou.net> <20110724232106.53161528@pitrou.net> <1311696328.3648.0.camel@localhost.localdomain> <4E2EE813.1080407@netwok.org> <20110727001912.4cf67481@pitrou.net> <20110726183250.03932089@resist.wooz.org> <87k4b3pfpw.fsf@benfinney.id.au> Message-ID: On Wed, Jul 27, 2011 at 2:37 PM, Ben Finney wrote: > Dan Stromberg writes: > > > It's been suggested that *ix has hardlinks because someone thought up > > hardlinks before someone thought up symlinks - IOW, there are those who > > suggest that if people had added symlinks first, no one would've bothered > > adding hardlinks. > > Well, that suggestion is faulty. It ignores the fact that *all* ordinary > files on Unix are hard links. Any ordinary file entry in a directory is > a hard link to the file's data. > Not really. Whether hard links is supported is mostly a matter of what filesystem you are using - in modern times. It's true that filesystems with complete POSIX semantics probably all support hardlinks, but that's far from every file on any given *ix. And of course, POSIX doesn't appear to have been created until the late 1990's. > The ?hard links? capability, therefore, isn't something that was added; > it's fundamental to Unix filesystems from their inception. > Hard linking was reportedly in Unix Version 1, but I see nothing indicating it was in the original Unics of 1969. Then again, I don't see much of anything on the net about what was and wasn't in Unics. > The ?ln? command adds *additional* hard links to an existing file's > data; but, once added, they're exactly the same as any other ordinary > file entry. > Well, if you're in a filesystem that supports hardlinking anyway. Supporting that isn't inherent. It requires some sort of on-disk representation for persistence, and not all filesystems support that. > > Symlinks are almost always more flexible, and almost always more > > clear. > > Yet many tools don't work as expected with symbolic links which will > work with an ordinary file (a ?hard link?). One can argue that such > tools are to that extent buggy, but symbolic links produce deliberately > different behaviour which is sometimes not what one needs. > Please recall that I was paraphrasing someone saying that hardlinks were seldom better, not never better. I don't know that there's anything in your post that addresses that. It's much easier to imagine a system with no hardlinks, than to imagine a system with no symlinks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From eliben at gmail.com Thu Jul 28 05:43:12 2011 From: eliben at gmail.com (Eli Bendersky) Date: Thu, 28 Jul 2011 06:43:12 +0300 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: <4E30A4F1.6050305@pearwood.info> References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> Message-ID: On Thu, Jul 28, 2011 at 02:53, Steven D'Aprano wrote: > Eli Bendersky wrote: > > Sure, but I'm still leery of two functions with the same name doing acting >> slightly differently. >> > > > and then in a later post: > > > As I mentioned elsewhere, it's not good practice to have two functions >> with >> the same name doing something slightly different, in different modules in >> the code-base. >> > > artist.draw() and gunslinger.draw() do not necessarily need to do the same > thing, > Of course, but why do you ignore the "slightly different". Had support.rmtree been something completely different from shutil.rmtree, I wouldn't have a problem with it. But it just calls rmtree and ignores some errors. This is the part I have the problem with. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin at python.org Thu Jul 28 06:10:28 2011 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 27 Jul 2011 23:10:28 -0500 Subject: [Python-Dev] Status of the PEP 400? (deprecate codecs.StreamReader/StreamWriter) In-Reply-To: References: <4E3088D0.5040900@haypocalc.com> <4E309B25.7090304@haypocalc.com> Message-ID: 2011/7/27 Guido van Rossum : > On Wed, Jul 27, 2011 at 6:00 PM, Nick Coghlan wrote: >> On Thu, Jul 28, 2011 at 9:38 AM, Guido van Rossum wrote: >>>> Users of codecs.open() or users of codecs.Stream* classes? >>> >>> I would think both. Is there any reason to continue using codecs.open()? >> >> It's the easiest way to write Unicode friendly code that spans both 2.x and 3.x. > > Even on 2.6, where the io module exists? io on 2.6 is fairly broken and dead slow. The advantage of codecs.open is it hasn't changed in the very long time. It still has the same reliable buggy behavior no matter what version you're on. I don't see the problem with leaving codecs.open() to rot a few more releases before deprecating it while leaving messaging in the docs suggesting io.*. > >> The problem is that naive 2.x code will migrate to the optimised IO >> stack automatically on the 2.x -> 3.x transition, while code that >> tried to do the right thing has to be changed manually (either in 3.x >> only, or by switching to the io module for 2.x as well) in order to >> adjust for the differences in argument order. >> >> The idea behind changing codecs.open to be a wrapper around io.open >> was to allow such code to switch to the new optimised IO stack as >> easily as code that just uses the open builtin. If it's acceptable for >> the builtin behaviour to change (far more substantially), why not >> change codecs.open as well? > > Aren't the cases different? Using built-in open() just means you want > to open a file in the default way. Using codecs.open() presumably > means that you've thought about Unicode. > > TBH, I said I was only -0 on the PEP, and if the stream returned by > codecs.open() in 3.3 is sufficiently compatible with the stream > returned the same function returns in 3.2, I am okay with it. (Except > I also want you to cut a trillion dollars from the non-military > budget, without raising taxes. :-) May I suggest you include the military budget? :) -- Regards, Benjamin From ben+python at benfinney.id.au Thu Jul 28 06:35:59 2011 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 28 Jul 2011 14:35:59 +1000 Subject: [Python-Dev] hard linking executables References: <20110724231655.0b69d36a@pitrou.net> <20110724232106.53161528@pitrou.net> <1311696328.3648.0.camel@localhost.localdomain> <4E2EE813.1080407@netwok.org> <20110727001912.4cf67481@pitrou.net> <20110726183250.03932089@resist.wooz.org> <87k4b3pfpw.fsf@benfinney.id.au> Message-ID: <871uxbowcw.fsf@benfinney.id.au> Dan Stromberg writes: > On Wed, Jul 27, 2011 at 2:37 PM, Ben Finney wrote: > > > Dan Stromberg writes: > > > > > It's been suggested that [?] if people had added symlinks first, > > > no one would've bothered adding hardlinks. > > > > Well, that suggestion is faulty. It ignores the fact that *all* > > ordinary files on Unix are hard links. Any ordinary file entry in a > > directory is a hard link to the file's data. > > Not really. Whether hard links is supported is mostly a matter of what > filesystem you are using - in modern times. It's true that filesystems > with complete POSIX semantics probably all support hardlinks, but > that's far from every file on any given *ix. And of course, POSIX > doesn't appear to have been created until the late 1990's. POSIX didn't create those semantics, though; it formalised semantics that were already long-time convention. My only point was that, unlike symbolic links, hard linking isn't some added feature, it's a property of the way Unix filesystems represent normal files. And you're right that I mean ?on filesystems with POSIX semantics?. > It's much easier to imagine a system with no hardlinks, than to > imagine a system with no symlinks. Why imagine? There have been many of both. -- \ ?But it is permissible to make a judgment after you have | `\ examined the evidence. In some circles it is even encouraged.? | _o__) ?Carl Sagan, _The Burden of Skepticism_, 1987 | Ben Finney From nas at arctrix.com Thu Jul 28 07:00:51 2011 From: nas at arctrix.com (Neil Schemenauer) Date: Thu, 28 Jul 2011 05:00:51 +0000 (UTC) Subject: [Python-Dev] hard linking executables References: <20110724231655.0b69d36a@pitrou.net> <20110724232106.53161528@pitrou.net> <1311696328.3648.0.camel@localhost.localdomain> <4E2EE813.1080407@netwok.org> <20110727001912.4cf67481@pitrou.net> <20110726183250.03932089@resist.wooz.org> Message-ID: Guido van Rossum wrote: > I ought to remember why because I remember I was involved. (And I have > a feeling that the change Antoine dug up was just a refactoring, You are correct. I checked Python 1.5.2 and it also creates hard links (prior to Makefile overhaul). > The best I can come up with is that when it's a hard link, you can > replace either "python" or "python2.1" with another version without > disturbing the other. So, I don't think anyone has a good argument for the status quo. A symlink would make more sense to me. Neil From victor.stinner at haypocalc.com Thu Jul 28 10:49:33 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Thu, 28 Jul 2011 10:49:33 +0200 Subject: [Python-Dev] Status of the PEP 400? (deprecate codecs.StreamReader/StreamWriter) In-Reply-To: References: <4E3088D0.5040900@haypocalc.com> <4E309B25.7090304@haypocalc.com> Message-ID: <4E31229D.9010207@haypocalc.com> Le 28/07/2011 06:10, Benjamin Peterson a ?crit : >>>> there any reason to continue using codecs.open()? >>> >>> It's the easiest way to write Unicode friendly code that spans both 2.x and 3.x. >> >> Even on 2.6, where the io module exists? > > io on 2.6 is fairly broken and dead slow. The advantage of codecs.open > is it hasn't changed in the very long time. It still has the same > reliable buggy behavior no matter what version you're on. > > I don't see the problem with leaving codecs.open() to rot a few more > releases before deprecating it while leaving messaging in the docs > suggesting io.*. All these points were already discussed before. There is a subsection in "Backwards Compatibility" section in the PEP 400 explaining why codecs.open is NOT deprecated: http://www.python.org/dev/peps/pep-0400/#keep-the-public-api-codecs-open "codecs.open() can be replaced by the builtin open() function. open() has a similar API but has also more options. Both functions return file-like objects (same API). codecs.open() was the only way to open a text file in Unicode mode until Python 2.6. Many Python 2 programs uses this function. Removing codecs.open() implies more work to port programs from Python 2 to Python 3, especially projets using the same code base for the two Python versions (without using 2to3 program). codecs.open() is kept for backward compatibility with Python 2." Victor From victor.stinner at haypocalc.com Thu Jul 28 11:28:43 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Thu, 28 Jul 2011 11:28:43 +0200 Subject: [Python-Dev] Status of the PEP 400? (deprecate codecs.StreamReader/StreamWriter) In-Reply-To: <4E3125D7.2030103@egenix.com> References: <4E308D63.9090901@haypocalc.com> <4E3125D7.2030103@egenix.com> Message-ID: <4E312BCB.3080301@haypocalc.com> Le 28/07/2011 11:03, M.-A. Lemburg a ?crit : > Victor Stinner wrote: >> Hi, >> >> Three weeks ago, I posted a draft on my PEP on this mailing list. I >> tried to include all remarks you made, and the PEP is now online: >> >> http://www.python.org/dev/peps/pep-0400/ >> >> It's now unclear to me if the PEP will be accepted or rejected. I don't >> know what to do to move forward. > > The PEP still compares apples and oranges, issues and features, I don't know how to write a PEP and this is my first PEP. I think that it is possible to compare two classes using a list of issues and features. How should I change the PEP to compare comparable things? > and doesn't cover the fact that it is proposing to not just deprecate > a feature, but a part of a design concept which will then no longer > be available in Python. The "Usage of StreamReader and StreamWriter" section tries to list usages of these classes, and "Deprecate StreamReader and StreamWriter" section explains that these classes will be removed. I agree that these sections are short, but I don't know what to add. Could you please enhance these sections? > I'm still -1 on that part of the PEP. Ok. > As I mentioned before, having > codecs.open() changed to be a wrapper around io.open() in Python 3.3, > should be investigated. If it doesn't cause too much trouble, this > would be a good idea. I did already try on the full Python test suite, and all test pass. I don't know if it's representative. > Please do keep the original implementation > around (e.g. renamed to codecs.open_stream()), though, so that it's > still possible to get easy-to-use access to codec StreamReader/Writers. I will add your alternative to the PEP (except if you would like to do that yourself?). If I understood correctly, you propose to: * rename codecs.open() to codecs.open_stream() * change codecs.open() to reuse open() (and so io.TextIOWrapper) (and don't deprecate anything) Add a new function to Python 3.3 means that we will have to maintain it for later versions. It's just the opposite of my proposition :-) Victor From mal at egenix.com Thu Jul 28 11:03:19 2011 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 28 Jul 2011 11:03:19 +0200 Subject: [Python-Dev] Status of the PEP 400? (deprecate codecs.StreamReader/StreamWriter) In-Reply-To: <4E308D63.9090901@haypocalc.com> References: <4E308D63.9090901@haypocalc.com> Message-ID: <4E3125D7.2030103@egenix.com> Victor Stinner wrote: > Hi, > > Three weeks ago, I posted a draft on my PEP on this mailing list. I > tried to include all remarks you made, and the PEP is now online: > > http://www.python.org/dev/peps/pep-0400/ > > It's now unclear to me if the PEP will be accepted or rejected. I don't > know what to do to move forward. The PEP still compares apples and oranges, issues and features, and doesn't cover the fact that it is proposing to not just deprecate a feature, but a part of a design concept which will then no longer be available in Python. I'm still -1 on that part of the PEP. As I mentioned before, having codecs.open() changed to be a wrapper around io.open() in Python 3.3, should be investigated. If it doesn't cause too much trouble, this would be a good idea. Please do keep the original implementation around (e.g. renamed to codecs.open_stream()), though, so that it's still possible to get easy-to-use access to codec StreamReader/Writers. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 28 2011) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From mattbasta at gmail.com Thu Jul 28 20:25:15 2011 From: mattbasta at gmail.com (Matt) Date: Thu, 28 Jul 2011 11:25:15 -0700 Subject: [Python-Dev] HTMLParser and HTML5 Message-ID: Hello all, I wanted to ask a few questions and start a discussion about HTML5 support within the HTMLParser class(es). Over on issue?670664, an inconsistency with the way browsers and the HTMLParser parse script and style tags was discovered. Currently, HTMLParser adheres strictly to the HTML4 standard, which says that these tags should exit CDATA mode when the start of *any* closing tag is found. No browsers, to my knowledge, have ever supported this (at least in the 21st century). Instead, all browsers implement the behavior described in the HTML5 spec, which states that script tags should exit their "raw text mode" when the full closing tag for that element is encountered. The repercussions of adhering to the HTML4 standard in HTMLParser are somewhat serious: a good number of documents will either encounter exceptions for broken markup (which aren't actually broken). Libraries like Beautiful Soup (which depend on HTMLParser) are also affected, requiring the use of hacks just to get the document to parse at all. Rather than bore you all with another paragraph about how HTML4 is terrible, feel free to look at the issue (http://bugs.python.org/issue670664), which quite thoroughly outlines the pros and cons of this particular change. Any feedback/input on the proposed changes is welcome. So here are my questions: - What plans, if any, are there to support HTML5 parsing behaviors, since the HTML5 spec effectively describes current web browser behavior? - What policies are in place for keeping parity with other HTML parsers (such as those in web browsers)? Given the semi-backward-compatible nature of HTML5's syntax, this seems like a rather unique problem that could use some more discussion. Thanks Matt Basta From brett at python.org Thu Jul 28 23:33:39 2011 From: brett at python.org (Brett Cannon) Date: Thu, 28 Jul 2011 14:33:39 -0700 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: <4E30A4F1.6050305@pearwood.info> References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> Message-ID: On Wed, Jul 27, 2011 at 16:53, Steven D'Aprano wrote: > Eli Bendersky wrote: > > Sure, but I'm still leery of two functions with the same name doing acting >> slightly differently. >> > > > and then in a later post: > > > As I mentioned elsewhere, it's not good practice to have two functions >> with >> the same name doing something slightly different, in different modules in >> the code-base. >> > > artist.draw() and gunslinger.draw() do not necessarily need to do the same > thing, and I don't agree that a (futile) preference for globally unique > names is good practice: it can lead to unnecessarily long names > (artist.draw_with_pencil_on_**paper) or redundant characters prefixing the > name (itertools.imap -- what does the "i" in imap tell you that the > itertools didn't already?). Solving this problem is what namespaces are for. > > Renaming test.support.ulink to something else doesn't fix the problem of > unsupported support functions being used in third-party code. It may even > *encourage* it, by making the extra functionality more explicit. > > However, is there any reason why test.support itself shouldn't be renamed > test._support, or possibly _test.support, so that the *entire* suite is > marked as a private implementation detail? > Technically no for the _test idea, although it would promote the idea of not shipping Python with its tests, which would be a shame as it's hard enough to get people to run them. Renaming test.support is much more acceptable, just more work considering how many times that module is used in the test suite. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Thu Jul 28 23:49:14 2011 From: brett at python.org (Brett Cannon) Date: Thu, 28 Jul 2011 14:49:14 -0700 Subject: [Python-Dev] HTMLParser and HTML5 In-Reply-To: References: Message-ID: On Thu, Jul 28, 2011 at 11:25, Matt wrote: > Hello all, > > I wanted to ask a few questions and start a discussion about HTML5 > support within the HTMLParser class(es). Over on issue 670664, an > inconsistency with the way browsers and the HTMLParser parse script > and style tags was discovered. Currently, HTMLParser adheres strictly > to the HTML4 standard, which says that these tags should exit CDATA > mode when the start of *any* closing tag is found. No browsers, to my > knowledge, have ever supported this (at least in the 21st century). > Instead, all browsers implement the behavior described in the HTML5 > spec, which states that script tags should exit their "raw text mode" > when the full closing tag for that element is encountered. > > The repercussions of adhering to the HTML4 standard in HTMLParser are > somewhat serious: a good number of documents will either encounter > exceptions for broken markup (which aren't actually broken). Libraries > like Beautiful Soup (which depend on HTMLParser) are also affected, > requiring the use of hacks just to get the document to parse at all. > > Rather than bore you all with another paragraph about how HTML4 is > terrible, feel free to look at the issue > (http://bugs.python.org/issue670664), which quite thoroughly outlines > the pros and cons of this particular change. Any feedback/input on > the proposed changes is welcome. > > So here are my questions: > > - What plans, if any, are there to support HTML5 parsing behaviors, > since the HTML5 spec effectively describes current web browser > behavior? > There are not specific plans that have been publicly brought up (to my knowledge). > - What policies are in place for keeping parity with other HTML > parsers (such as those in web browsers)? > There aren't any beyond "it would be nice". > > Given the semi-backward-compatible nature of HTML5's syntax, this > seems like a rather unique problem that could use some more > discussion. > It's more of an issue of someone caring enough to do the coding work to bring the parser up to spec for HTML5 (or introduce new code to live beside the HTML4 parsing code). IOW there is no policies specifically about this topic beyond the general desire to stay up-to-date with stable specs. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Fri Jul 29 02:39:47 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 29 Jul 2011 10:39:47 +1000 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> Message-ID: On Fri, Jul 29, 2011 at 7:33 AM, Brett Cannon wrote: >> However, is there any reason why test.support itself shouldn't be renamed >> test._support, or possibly _test.support, so that the *entire* suite is >> marked as a private implementation detail? > > Technically no for the _test idea, although it would promote the idea of not > shipping Python with its tests, which would be a shame as it's hard enough > to get people to run them. Renaming test.support is much more acceptable, > just more work considering how many times that module is used in the test > suite. Moving the docs should deal with the module and name indexing issue, so -1 one to expending any developer effort on unnecessary name changes in the test suite. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From stefan_ml at behnel.de Fri Jul 29 06:37:17 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 29 Jul 2011 06:37:17 +0200 Subject: [Python-Dev] HTMLParser and HTML5 In-Reply-To: References: Message-ID: Brett Cannon, 28.07.2011 23:49: > On Thu, Jul 28, 2011 at 11:25, Matt wrote: >> - What policies are in place for keeping parity with other HTML >> parsers (such as those in web browsers)? > > There aren't any beyond "it would be nice". >[...] > It's more of an issue of someone caring enough to do the coding work to > bring the parser up to spec for HTML5 (or introduce new code to live beside > the HTML4 parsing code). Which, given that html5lib readily exists, would likely be a lot more work than anyone who is interested in HTML5 handling would want to invest. I don't think we need a new HTML5 parsing implementation only to have it in the stdlib. That's the old sunny Java way of doing it. Stefan From eliben at gmail.com Fri Jul 29 07:24:59 2011 From: eliben at gmail.com (Eli Bendersky) Date: Fri, 29 Jul 2011 08:24:59 +0300 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> Message-ID: On Fri, Jul 29, 2011 at 03:39, Nick Coghlan wrote: > On Fri, Jul 29, 2011 at 7:33 AM, Brett Cannon wrote: > >> However, is there any reason why test.support itself shouldn't be > renamed > >> test._support, or possibly _test.support, so that the *entire* suite is > >> marked as a private implementation detail? > > > > Technically no for the _test idea, although it would promote the idea of > not > > shipping Python with its tests, which would be a shame as it's hard > enough > > to get people to run them. Renaming test.support is much more acceptable, > > just more work considering how many times that module is used in the test > > suite. > > Moving the docs should deal with the module and name indexing issue, > so -1 one to expending any developer effort on unnecessary name > changes in the test suite. > > Alright, I think there's now a sufficiently wide consensus to move the documentation of Lib/test and Lib/test/support in particular to the devguide, which raises a question: Currently test.support is documented both for Python 3K and 2.7, while the devguide (AFAIK) comes in a single version. I propose to just move 3K's docs to the devguide, and make both doc pages (in 3K and 2.7) point to it. Is this acceptable? Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Fri Jul 29 07:48:07 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 29 Jul 2011 15:48:07 +1000 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> Message-ID: On Fri, Jul 29, 2011 at 3:24 PM, Eli Bendersky wrote: > I propose to just move 3K's docs to the devguide, and make both doc pages > (in 3K and 2.7) point to it. Is this acceptable? Yeah, just include a note in the devguide version saying that anything added in 3.2 or later may not be available when writing tests for the 2.7 maintenance branch. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From eliben at gmail.com Fri Jul 29 08:48:15 2011 From: eliben at gmail.com (Eli Bendersky) Date: Fri, 29 Jul 2011 09:48:15 +0300 Subject: [Python-Dev] [Python-checkins] cpython: Issue #10639: reindent.py tool now accepts a --newline option to specify the In-Reply-To: References: Message-ID: On Tue, Jul 26, 2011 at 18:59, jason.coombs wrote: > http://hg.python.org/cpython/rev/4feb889d3bed > changeset: 71506:4feb889d3bed > user: Jason R. Coombs > date: Tue Jul 26 11:38:04 2011 -0400 > summary: > Issue #10639: reindent.py tool now accepts a --newline option to specify > the newline to be used in the output of converted files. > Jason, this breaks "make patchcheck". Please see my comment on the issue. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From eliben at gmail.com Fri Jul 29 08:52:40 2011 From: eliben at gmail.com (Eli Bendersky) Date: Fri, 29 Jul 2011 09:52:40 +0300 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> Message-ID: On Fri, Jul 29, 2011 at 08:48, Nick Coghlan wrote: > On Fri, Jul 29, 2011 at 3:24 PM, Eli Bendersky wrote: > > I propose to just move 3K's docs to the devguide, and make both doc pages > > (in 3K and 2.7) point to it. Is this acceptable? > > Yeah, just include a note in the devguide version saying that anything > added in 3.2 or later may not be available when writing tests for the > 2.7 maintenance branch. > Alright, and what should stay in the docs then? I guess only 25.5.2 (usage of regrtest). The rest is either documentation of support, or advice for people writing new tests, both of which belong to the devguide. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Fri Jul 29 09:10:42 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 29 Jul 2011 17:10:42 +1000 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> Message-ID: On Fri, Jul 29, 2011 at 4:52 PM, Eli Bendersky wrote: > On Fri, Jul 29, 2011 at 08:48, Nick Coghlan wrote: >> >> On Fri, Jul 29, 2011 at 3:24 PM, Eli Bendersky wrote: >> > I propose to just move 3K's docs to the devguide, and make both doc >> > pages >> > (in 3K and 2.7) point to it. Is this acceptable? >> >> Yeah, just include a note in the devguide version saying that anything >> added in 3.2 or later may not be available when writing tests for the >> 2.7 maintenance branch. > > Alright, and what should stay in the docs then? I guess only 25.5.2 (usage > of regrtest). The rest is either documentation of support, or advice for > people writing new tests, both of which belong to the devguide. How to run the test suite should stay, along with a pointer to the additional information in the devguide. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From eliben at gmail.com Fri Jul 29 09:51:27 2011 From: eliben at gmail.com (Eli Bendersky) Date: Fri, 29 Jul 2011 10:51:27 +0300 Subject: [Python-Dev] Moving documentation of test.support into the devguide Message-ID: I've opened issue 12652 to track this task. Proposed workflow: 1. First, the devguide incorporates the documentation 2. Then, I'll clean up the official docs and add pointers to the devguide instead The transition of the documentation into the devguide also has a few steps: 1. Move the reference of the utilities test.support provides. This reference documentation isn't complete (not all functions are documented), but can be completed later in the devguide. 2. (Optionally) incorporate the test-writing advice given in 25.5.1 into the devguide (which currently has only a short couple of paragraphs 3. Move any other information from the official docs (chapter 25) into the devguide, leaving only a section on running the tests (which should be identical to the relevant section in the devguide) and a pointer to the devguide. Issue #12652 now has a patch for step (1) above. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Fri Jul 29 11:26:51 2011 From: barry at python.org (Barry Warsaw) Date: Fri, 29 Jul 2011 05:26:51 -0400 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> Message-ID: <20110729052651.311dbd53@resist.wooz.org> On Jul 29, 2011, at 08:24 AM, Eli Bendersky wrote: >Alright, I think there's now a sufficiently wide consensus to move the >documentation of Lib/test and Lib/test/support in particular to the >devguide, which raises a question: I haven't been following this thread, so I caught up on Gmane. I'm somewhat uncomfortable with this decision. I understand why you want to do this, but I still think it might not be the right thing. For better or worse, test.support is part of the standard library, so moving its documentation elsewhere sets a bad precedent to me. Many times I work on Python while completely off-line. If I've had the foresight to build the docs before I go off-line, all the better, but even if not, I have the .rst files to consult. Now I'll have to remember to *also* grab the devguide so that I have the complete documentation of the stdlib. I also think this underestimates how blurry the line is between users and developers. Think about yourself: when did you suddenly graduate from user of Python to developer of Python? I know that I often talk to people who would consider themselves "just" users of Python, but feel comfortable with their occasional spelunking into the Python tree looking for whatever nugget of code or documentation will help them understand their problem. The devguide, as useful and cool as it is, is still immature and hard to discover. I think more time will improve its organization, and it's not even linked to from docs.python.org. So I'm curious, why is this move better than adding noindexes, or just trusting users to understand the difference between test.support.unlink() and os.unlink()? If I currently search for 'unlink', os.unlink comes up first, which is good, and that should be preserved. The presence or not of some test.support.unlink documentation isn't going to make the search results that much better or worse (there's already 14 hits). Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From eliben at gmail.com Fri Jul 29 13:07:50 2011 From: eliben at gmail.com (Eli Bendersky) Date: Fri, 29 Jul 2011 14:07:50 +0300 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: <20110729052651.311dbd53@resist.wooz.org> References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> Message-ID: > >Alright, I think there's now a sufficiently wide consensus to move the > >documentation of Lib/test and Lib/test/support in particular to the > >devguide, which raises a question: > > I haven't been following this thread, so I caught up on Gmane. > > I'm somewhat uncomfortable with this decision. I understand why you want > to > do this, but I still think it might not be the right thing. > > For better or worse, test.support is part of the standard library, so > moving > Why is it part of stdlib though? Isn't the stdlib something that's exposed to all Python programmers? How should an ordinary programmer (not a core dev) know some parts of stdlib are out of limits, if they are even documented and appear in the index - should he read the note at the top of the documentation page? Imagine a desperate programmer behind schedule who is looking for some particular functionality in stdlib and finds it in test.support - will he not use it and later complain we don't keep backwards-compatibility. Sure, he's at fault, but I think we can make such cases much rarer by making the distinction between user-exposed modules and core-dev-exposed modules more explicit. > its documentation elsewhere sets a bad precedent to me. Many times I work > on > Python while completely off-line. If I've had the foresight to build the > docs > before I go off-line, all the better, but even if not, I have the .rst > files > to consult. Now I'll have to remember to *also* grab the devguide so that > I > have the complete documentation of the stdlib. > > I think this is because you're experienced and probably know all the contents of the devguide anyway. Junior core-devs need the devguide handy for other reasons as well. > I also think this underestimates how blurry the line is between users and > developers. Think about yourself: when did you suddenly graduate from user > of > Python to developer of Python? I know that I often talk to people who > would > consider themselves "just" users of Python, but feel comfortable with their > occasional spelunking into the Python tree looking for whatever nugget of > code > or documentation will help them understand their problem. > Isn't this what we're trying to prevent, though? One should never even have to look at test.support unless he's working *on Python*. > > The devguide, as useful and cool as it is, is still immature and hard to > discover. I think more time will improve its organization, and it's not > even > linked to from docs.python.org. > > Maybe this is the main point here - what makes it immature? What makes it hard to discover? Maybe these issues can be fixed. > So I'm curious, why is this move better than adding noindexes, or just > trusting users to understand the difference between test.support.unlink() > and > os.unlink()? If I currently search for 'unlink', os.unlink comes up first, > which is good, and that should be preserved. The presence or not of some > test.support.unlink documentation isn't going to make the search results > that > much better or worse (there's already 14 hits). > I think the unlink&rmtree functions are just a symptom. The real issue here is - what is the devguide for, and how is it different from Python's existing documentation? What should go into the official docs, and what should go into the devguide? Once we agree on these question, I think the test.support dilemma will solve itself. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From jsbueno at python.org.br Fri Jul 29 13:22:17 2011 From: jsbueno at python.org.br (Joao S. O. Bueno) Date: Fri, 29 Jul 2011 08:22:17 -0300 Subject: [Python-Dev] HTMLParser and HTML5 In-Reply-To: References: Message-ID: On Fri, Jul 29, 2011 at 1:37 AM, Stefan Behnel wrote: > Brett Cannon, 28.07.2011 23:49: >> >> On Thu, Jul 28, 2011 at 11:25, Matt wrote: >>> >>> - What policies are in place for keeping parity with other HTML >>> parsers (such as those in web browsers)? >> >> There aren't any beyond "it would be nice". >> [...] >> It's more of an issue of someone caring enough to do the coding work to >> bring the parser up to spec for HTML5 (or introduce new code to live >> beside >> the HTML4 parsing code). > > Which, given that html5lib readily exists, would likely be a lot more work > than anyone who is interested in HTML5 handling would want to invest. > > I don't think we need a new HTML5 parsing implementation only to have it in > the stdlib. That's the old sunny Java way of doing it. > I disaagree. Having proper html parsing out of the box is part of the "batteries included" thing. And it is not a matter of "having html 5" - as stated on this thread, fixing it for html5 will fix it for html that exists in the "real world". Python _has_ to work with quick 30-50 lines scripts deliverable everywhere, not just has proper 3rd party libraries that can work as part of a huge project using buildout. js -><- > Stefan > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/jsbueno%40python.org.br > From stefan_ml at behnel.de Fri Jul 29 13:46:54 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 29 Jul 2011 13:46:54 +0200 Subject: [Python-Dev] HTMLParser and HTML5 In-Reply-To: References: Message-ID: Joao S. O. Bueno, 29.07.2011 13:22: > On Fri, Jul 29, 2011 at 1:37 AM, Stefan Behnel wrote: >> Brett Cannon, 28.07.2011 23:49: >>> >>> On Thu, Jul 28, 2011 at 11:25, Matt wrote: >>>> >>>> - What policies are in place for keeping parity with other HTML >>>> parsers (such as those in web browsers)? >>> >>> There aren't any beyond "it would be nice". >>> [...] >>> It's more of an issue of someone caring enough to do the coding work to >>> bring the parser up to spec for HTML5 (or introduce new code to live >>> beside >>> the HTML4 parsing code). >> >> Which, given that html5lib readily exists, would likely be a lot more work >> than anyone who is interested in HTML5 handling would want to invest. >> >> I don't think we need a new HTML5 parsing implementation only to have it in >> the stdlib. That's the old sunny Java way of doing it. > > I disaagree. > Having proper html parsing out of the box is part of the "batteries > included" thing. Well, you can easily prove me wrong by implementing this. Stefan From victor.stinner at haypocalc.com Fri Jul 29 14:26:06 2011 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Fri, 29 Jul 2011 14:26:06 +0200 Subject: [Python-Dev] Status of the PEP 400? (deprecate codecs.StreamReader/StreamWriter) In-Reply-To: <4E312BCB.3080301@haypocalc.com> References: <4E308D63.9090901@haypocalc.com> <4E3125D7.2030103@egenix.com> <4E312BCB.3080301@haypocalc.com> Message-ID: <4E32A6DE.7060001@haypocalc.com> Le 28/07/2011 11:28, Victor Stinner a ?crit : >> Please do keep the original implementation >> around (e.g. renamed to codecs.open_stream()), though, so that it's >> still possible to get easy-to-use access to codec StreamReader/Writers. > > I will add your alternative to the PEP (except if you would like to do > that yourself?). If I understood correctly, you propose to: > > * rename codecs.open() to codecs.open_stream() > * change codecs.open() to reuse open() (and so io.TextIOWrapper) > > (and don't deprecate anything) I added your proposal to the PEP as an "Alternative Approache". Victor From solipsis at pitrou.net Fri Jul 29 14:46:10 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 29 Jul 2011 14:46:10 +0200 Subject: [Python-Dev] cpython (3.2): Fix sorting or wording of some NEWS entries. References: Message-ID: <20110729144610.5e7c7681@pitrou.net> On Fri, 29 Jul 2011 14:35:19 +0200 eric.araujo wrote: > http://hg.python.org/cpython/rev/97527f3f9829 > changeset: 71555:97527f3f9829 > branch: 3.2 > user: ?ric Araujo > date: Tue Jul 26 17:32:50 2011 +0200 > summary: > Fix sorting or wording of some NEWS entries. > > I would have put io and ctypes fixes into Extension Modules, but I > respected the choice of Antoine or Victor and left them in Library. There's no practical difference (from the user's point of view) between extension modules and the library, so I think the "Extension Modules" section should simply die. Regards Antoine. From solipsis at pitrou.net Fri Jul 29 14:48:43 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 29 Jul 2011 14:48:43 +0200 Subject: [Python-Dev] cpython: Remove indirection in threading (issue #10968). References: Message-ID: <20110729144843.218e9469@pitrou.net> On Fri, 29 Jul 2011 14:35:23 +0200 eric.araujo wrote: > > It is now possible to inherit from Thread and other classes, without > having to import the private underscored names like multiprocessing did. A correction: it was already possible to inherit from Thread (since it's quite useful to do so). Regards Antoine. From nadeem.vawda at gmail.com Fri Jul 29 14:59:32 2011 From: nadeem.vawda at gmail.com (Nadeem Vawda) Date: Fri, 29 Jul 2011 14:59:32 +0200 Subject: [Python-Dev] cpython (3.2): Fix sorting or wording of some NEWS entries. In-Reply-To: <20110729144610.5e7c7681@pitrou.net> References: <20110729144610.5e7c7681@pitrou.net> Message-ID: On Fri, Jul 29, 2011 at 2:46 PM, Antoine Pitrou wrote: > There's no practical difference (from the user's point of view) between > extension modules and the library, so I think the "Extension Modules" > section should simply die. +1. This has been bugging me for a while now. From solipsis at pitrou.net Fri Jul 29 14:50:45 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 29 Jul 2011 14:50:45 +0200 Subject: [Python-Dev] cpython (3.2): Stop ignoring Mercurial merge conflits files (#12255). References: Message-ID: <20110729145045.2c252ea0@pitrou.net> On Fri, 29 Jul 2011 14:35:24 +0200 eric.araujo wrote: > http://hg.python.org/cpython/rev/bdad5bc9a2ed > changeset: 71562:bdad5bc9a2ed > branch: 3.2 > user: ?ric Araujo > date: Thu Jul 28 22:45:46 2011 +0200 > summary: > Stop ignoring Mercurial merge conflits files (#12255). > > R. David Murray and I think that it?s more useful to have these files > show up in the output of ?hg status?, to let the user know that some > merged file have to be checked before commit. Mercurial will prevent you from committing before you have solved conflicts, so I'm not sure what this brings. "hg res -l" is the command to remember when you want to list files with conflicts. The fact that "make clean" doesn't wipe these files is an additional annoyance. Regards Antoine. From merwok at netwok.org Fri Jul 29 15:17:12 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Fri, 29 Jul 2011 15:17:12 +0200 Subject: [Python-Dev] cpython: Remove indirection in threading (issue #10968). In-Reply-To: <20110729144843.218e9469@pitrou.net> References: <20110729144843.218e9469@pitrou.net> Message-ID: <4E32B2D8.2040701@netwok.org> Le 29/07/2011 14:48, Antoine Pitrou a ?crit : >> It is now possible to inherit from Thread and other classes, without >> having to import the private underscored names like multiprocessing did. > A correction: it was already possible to inherit from Thread (since > it's quite useful to do so). Indeed, and Timer for example inherits from Thread. The other classes I changed however were not subclassable (the original bug report was about subclassing Timer, not Thread), I just picked the one wrong example when writing ?X and other classes? :) Regards From merwok at netwok.org Fri Jul 29 15:28:44 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Fri, 29 Jul 2011 15:28:44 +0200 Subject: [Python-Dev] cpython (3.2): Stop ignoring Mercurial merge conflits files (#12255). In-Reply-To: <20110729145045.2c252ea0@pitrou.net> References: <20110729145045.2c252ea0@pitrou.net> Message-ID: <4E32B58C.3070809@netwok.org> Le 29/07/2011 14:50, Antoine Pitrou a ?crit : >> changeset: 71562:bdad5bc9a2ed >> user: ?ric Araujo >> summary: >> Stop ignoring Mercurial merge conflits files (#12255). >> >> R. David Murray and I think that it?s more useful to have these files >> show up in the output of ?hg status?, to let the user know that some >> merged file have to be checked before commit. > > Mercurial will prevent you from committing before you have solved > conflicts, so I'm not sure what this brings. "hg res -l" is the command > to remember when you want to list files with conflicts. People confused by the merge/resolve system could exit their merge tool without actually merging the changes (I know it happened to me!), so these files act as a reminder that not everything is right. .orig is also created by hg revert; my usage is that I remove this file after I?ve checked that the revert is okay. > The fact that "make clean" doesn't wipe these files is an additional > annoyance. make clean removes generated files, but *.rej and *.orig are backups, which you may want to save or re-apply. I?m not sure it would be right to lose them. Maybe distclean? Regards From solipsis at pitrou.net Fri Jul 29 15:38:31 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 29 Jul 2011 15:38:31 +0200 Subject: [Python-Dev] cpython (3.2): Stop ignoring Mercurial merge conflits files (#12255). References: <20110729145045.2c252ea0@pitrou.net> <4E32B58C.3070809@netwok.org> Message-ID: <20110729153831.455b5d67@pitrou.net> On Fri, 29 Jul 2011 15:28:44 +0200 ?ric Araujo wrote: > Le 29/07/2011 14:50, Antoine Pitrou a ?crit : > >> changeset: 71562:bdad5bc9a2ed > >> user: ?ric Araujo > >> summary: > >> Stop ignoring Mercurial merge conflits files (#12255). > >> > >> R. David Murray and I think that it?s more useful to have these files > >> show up in the output of ?hg status?, to let the user know that some > >> merged file have to be checked before commit. > > > > Mercurial will prevent you from committing before you have solved > > conflicts, so I'm not sure what this brings. "hg res -l" is the command > > to remember when you want to list files with conflicts. > > People confused by the merge/resolve system could exit their merge tool > without actually merging the changes (I know it happened to me!), so > these files act as a reminder that not everything is right. I don't know, I don't use a merge tool. But presumably the merge tool would only call "hg resolve -m" on those files which have actually been resolved? (if it calls that command at all) > > The fact that "make clean" doesn't wipe these files is an additional > > annoyance. > > make clean removes generated files, but *.rej and *.orig are backups, > which you may want to save or re-apply. What use are these backups really? We are using a (D)VCS, you are not losing anything. Regards Antoine. From merwok at netwok.org Fri Jul 29 15:22:08 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Fri, 29 Jul 2011 15:22:08 +0200 Subject: [Python-Dev] cpython (3.2): Fix sorting or wording of some NEWS entries. In-Reply-To: <20110729144610.5e7c7681@pitrou.net> References: <20110729144610.5e7c7681@pitrou.net> Message-ID: <4E32B400.9060206@netwok.org> Le 29/07/2011 14:46, Antoine Pitrou a ?crit : >> changeset: 71555:97527f3f9829 >> branch: 3.2 >> user: ?ric Araujo >> summary: >> Fix sorting or wording of some NEWS entries. >> >> I would have put io and ctypes fixes into Extension Modules, but I >> respected the choice of Antoine or Victor and left them in Library. > > There's no practical difference (from the user's point of view) between > extension modules and the library, so I think the "Extension Modules" > section should simply die. +1 to that. Would built-in modules like imp belong to Library or Interpreter Core? (I?d say Core.) When merging 3.2 into 3.3, I started sorting the Misc/NEWS file there too but stopped after ~30 changes. On one hand, I know that developer time would be better spent on fixing bugs rather than reading a commit sorting NEWS entries, but on the other hand I thought it best for readers of this file to have something that makes sense. Build fixes were not in a dedicated section, Core and Library are happily mixed, so I thought that my sorting could help even the What?s New author. I wanted to ask python-dev first, though. Regards From rdmurray at bitdance.com Fri Jul 29 16:04:26 2011 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 29 Jul 2011 10:04:26 -0400 Subject: [Python-Dev] cpython (3.2): Stop ignoring Mercurial merge conflits files (#12255). In-Reply-To: <4E32B58C.3070809@netwok.org> References: <20110729145045.2c252ea0@pitrou.net> <4E32B58C.3070809@netwok.org> Message-ID: <20110729140426.A78DC2506F0@webabinitio.net> On Fri, 29 Jul 2011 15:28:44 +0200, =?UTF-8?B?w4lyaWMgQXJhdWpv?= wrote: > Le 29/07/2011 14:50, Antoine Pitrou a ??crit : > >> changeset: 71562:bdad5bc9a2ed > >> user: ??ric Araujo > >> summary: > >> Stop ignoring Mercurial merge conflits files (#12255). > >> > >> R. David Murray and I think that it???s more useful to have these files > >> show up in the output of ???hg status???, to let the user know that some > >> merged file have to be checked before commit. > > > > Mercurial will prevent you from committing before you have solved > > conflicts, so I'm not sure what this brings. "hg res -l" is the command > > to remember when you want to list files with conflicts. > > People confused by the merge/resolve system could exit their merge tool > without actually merging the changes (I know it happened to me!), so > these files act as a reminder that not everything is right. Or people who decide the particular merge tool isn't going to help in a particular case. I do that reasonably often. > .orig is also created by hg revert; my usage is that I remove this file > after I???ve checked that the revert is okay. And by patch. This is my major reason for disliking having these files ignored. I want to know about the .rej and .orig files patch generates that I may have forgotten about, and not just to clean them up. Their existence is a signal that there's something I haven't finished working through (or that I at least need to check that I worked through it before deleting them). > > The fact that "make clean" doesn't wipe these files is an additional > > annoyance. > > make clean removes generated files, but *.rej and *.orig are backups, > which you may want to save or re-apply. I???m not sure it would be right > to lose them. Maybe distclean? Exactly. Which is another reason these files should not be ignored: even distclean should probably not remove the backup files generated by patch, which means that hg status should show the files because they aren't part of the distribution. Which means (assuming there is agreement with my logic :) that make distclean probably *should* remove the stuff generated by coverage, given that we are ignoring that stuff now. --David From merwok at netwok.org Fri Jul 29 16:21:42 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Fri, 29 Jul 2011 16:21:42 +0200 Subject: [Python-Dev] cpython (3.2): Stop ignoring Mercurial merge conflits files (#12255). In-Reply-To: <20110729153831.455b5d67@pitrou.net> References: <20110729145045.2c252ea0@pitrou.net> <4E32B58C.3070809@netwok.org> <20110729153831.455b5d67@pitrou.net> Message-ID: <4E32C1F6.4050204@netwok.org> >> People confused by the merge/resolve system could exit their merge tool >> without actually merging the changes (I know it happened to me!), so >> these files act as a reminder that not everything is right. > > I don't know, I don't use a merge tool. But presumably the merge tool > would only call "hg resolve -m" on those files which have actually been > resolved? (if it calls that command at all) You have it backwards: it?s hg resolve that runs a merge tool. (BTW, how can you not use a merge tool? Even if you use internal hg tools like the one that adds merge conflict markers, that?s a merge tool.) > What use are these backups really? We are using a (D)VCS, you are not > losing anything. The .orig files after a revert could contain code that?s not committed anywhere. See also RDM?s reply to your message. Regards From rdmurray at bitdance.com Fri Jul 29 16:50:57 2011 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 29 Jul 2011 10:50:57 -0400 Subject: [Python-Dev] cpython (3.2): Stop ignoring Mercurial merge conflits files (#12255). In-Reply-To: <20110729153831.455b5d67@pitrou.net> References: <20110729145045.2c252ea0@pitrou.net> <4E32B58C.3070809@netwok.org> <20110729153831.455b5d67@pitrou.net> Message-ID: <20110729145058.22F732506F0@webabinitio.net> On Fri, 29 Jul 2011 15:38:31 +0200, Antoine Pitrou wrote: > On Fri, 29 Jul 2011 15:28:44 +0200 > ??ric Araujo wrote: > > make clean removes generated files, but *.rej and *.orig are backups, > > which you may want to save or re-apply. > > What use are these backups really? We are using a (D)VCS, you are not > losing anything. As ??ric said, the .orig files may contain uncommitted changes. The .rej files are information about what parts of the patch were *not* applied, which is information that is not retained anywhere else. --David From solipsis at pitrou.net Fri Jul 29 16:49:35 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 29 Jul 2011 16:49:35 +0200 Subject: [Python-Dev] cpython (3.2): Stop ignoring Mercurial merge conflits files (#12255). References: <20110729145045.2c252ea0@pitrou.net> <4E32B58C.3070809@netwok.org> <20110729153831.455b5d67@pitrou.net> <4E32C1F6.4050204@netwok.org> Message-ID: <20110729164935.2dde6219@pitrou.net> On Fri, 29 Jul 2011 16:21:42 +0200 ?ric Araujo wrote: > > What use are these backups really? We are using a (D)VCS, you are not > > losing anything. > > The .orig files after a revert could contain code that?s not committed > anywhere. See also RDM?s reply to your message. I would point out that by using "hg revert", you deliberately want to forget some local changes. Besides, "hg status" is meant to show untracked files which could *potentially* be tracked. It's not like anybody wants to track .orig and .rej files, so having them in the ignore list is still the right thing to do. Regards Antoine. From solipsis at pitrou.net Fri Jul 29 16:55:03 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 29 Jul 2011 16:55:03 +0200 Subject: [Python-Dev] cpython (3.2): Fix sorting or wording of some NEWS entries. References: <20110729144610.5e7c7681@pitrou.net> <4E32B400.9060206@netwok.org> Message-ID: <20110729165503.2f25d692@pitrou.net> On Fri, 29 Jul 2011 15:22:08 +0200 ?ric Araujo wrote: > > > > There's no practical difference (from the user's point of view) between > > extension modules and the library, so I think the "Extension Modules" > > section should simply die. > > +1 to that. Would built-in modules like imp belong to Library or > Interpreter Core? (I?d say Core.) As soon as you are fixing a library API, it should IMO go to Library. The fact that some modules are "built-in" is an implementation detail. For example, "_io" is compiled in the executable (for bootstrap reasons), yet it's still considered part of the library. > When merging 3.2 into 3.3, I started sorting the Misc/NEWS file there > too but stopped after ~30 changes. On one hand, I know that developer > time would be better spent on fixing bugs rather than reading a commit > sorting NEWS entries, but on the other hand I thought it best for > readers of this file to have something that makes sense. Build fixes > were not in a dedicated section, Core and Library are happily mixed, so > I thought that my sorting could help even the What?s New author. I > wanted to ask python-dev first, though. Well, sorting is fine, if you don't think your developer time could be used for more valuable tasks :) Regards Antoine. From ncoghlan at gmail.com Fri Jul 29 17:02:31 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 30 Jul 2011 01:02:31 +1000 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: <20110729052651.311dbd53@resist.wooz.org> References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> Message-ID: On Fri, Jul 29, 2011 at 7:26 PM, Barry Warsaw wrote: > The devguide, as useful and cool as it is, is still immature and hard to > discover. ?I think more time will improve its organization, and it's not even > linked to from docs.python.org. > > So I'm curious, why is this move better than adding noindexes, or just > trusting users to understand the difference between test.support.unlink() and > os.unlink()? ?If I currently search for 'unlink', os.unlink comes up first, > which is good, and that should be preserved. ?The presence or not of some > test.support.unlink documentation isn't going to make the search results that > much better or worse (there's already 14 hits). It's worthwhile because it is what the devguide is for: documenting how to *change* Python, rather than just using it as it is delivered to you. There's a clear transition from user of Python to developer of Python: you stop treating the standard library (and perhaps even the interpreter core) as sacrosanct, and will actually change the code in those files if its wrong (and is affecting your own application). Agreed that docs.python.org front page should have a link to docs.python.org/devguide, though. Cheers. Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From barry at python.org Fri Jul 29 17:18:37 2011 From: barry at python.org (Barry Warsaw) Date: Fri, 29 Jul 2011 11:18:37 -0400 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> Message-ID: <20110729111837.4f5b88cd@resist.wooz.org> On Jul 29, 2011, at 02:07 PM, Eli Bendersky wrote: >Why is it part of stdlib though? Isn't the stdlib something that's exposed >to all Python programmers? How should an ordinary programmer (not a core >dev) know some parts of stdlib are out of limits, if they are even >documented and appear in the index - should he read the note at the top of >the documentation page? Imagine a desperate programmer behind schedule who >is looking for some particular functionality in stdlib and finds it in >test.support - will he not use it and later complain we don't keep >backwards-compatibility. Sure, he's at fault, but I think we can make such >cases much rarer by making the distinction between user-exposed modules and >core-dev-exposed modules more explicit. How common are those mistakes right now? I get way more complaints about squishy APIs in third party libraries and actual *supported* parts of the stdlib than I do about test.support APIs. What's the real problem we're trying to solve anyway? Is it "protect the harried user from some future breakage" or "eliminate complaints we Python developers get from harried users"? I'd much rather solve this problem by adding markup to functions that explicitly disclaim our normal backward compatibility guarantees. Squirreling away documentation for some parts of the stdlib seems similar to security-by-obscurity arguments. test.support *is* part of the stdlib. If you want to propose moving it out of the stdlib, that's a different argument to make. Maybe Python's entire test suite should be in some Cheeseshop package instead of in the core source tree. (I wouldn't agree with that either, but I'm just saying.) So once again, is this a real actual problem you've witnessed in enough cases to go through all the trouble of moving the documentation, and making it more difficult to find? >> its documentation elsewhere sets a bad precedent to me. Many times I work >> on Python while completely off-line. If I've had the foresight to build >> the docs before I go off-line, all the better, but even if not, I have the >> .rst files to consult. Now I'll have to remember to *also* grab the >> devguide so that I have the complete documentation of the stdlib. >> >I think this is because you're experienced and probably know all the >contents of the devguide anyway. Junior core-devs need the devguide handy >for other reasons as well. I'm not saying the devguide isn't handy - in fact it's incredibly handy! I love that it explains the nuances of how we structure our Mercurial branches, and define our workflow for example. I readily admit those are not things I know by heart. (But actually, those instructions are not always easy to find, which is why I think the organization is immature. I'm not faulting the devguide authors, I just think it needs more time to bake.) >> I also think this underestimates how blurry the line is between users and >> developers. Think about yourself: when did you suddenly graduate from user >> of Python to developer of Python? I know that I often talk to people who >> would consider themselves "just" users of Python, but feel comfortable with >> their occasional spelunking into the Python tree looking for whatever >> nugget of code or documentation will help them understand their problem. > >Isn't this what we're trying to prevent, though? One should never even have >to look at test.support unless he's working *on Python*. Again, I think that line is blurred here. Let's say you're working on some off-beat or new hardware and you want to know if your basic tool chain works for it. You build Python, run the test suite, and something fails. You probably don't consider yourself a Python developer, but now you're digging through the test.support to figure out your problem. Yes, I've seen this happen. >> The devguide, as useful and cool as it is, is still immature and hard to >> discover. I think more time will improve its organization, and it's not >> even linked to from docs.python.org. >> >Maybe this is the main point here - what makes it immature? What makes it >hard to discover? Maybe these issues can be fixed. Sure, there are issues that can be fixed, but they're unrelated to documenting test.support. >> So I'm curious, why is this move better than adding noindexes, or just >> trusting users to understand the difference between test.support.unlink() >> and os.unlink()? If I currently search for 'unlink', os.unlink comes up >> first, which is good, and that should be preserved. The presence or not of >> some test.support.unlink documentation isn't going to make the search >> results that much better or worse (there's already 14 hits). >> > >I think the unlink&rmtree functions are just a symptom. The real issue here >is - what is the devguide for, and how is it different from Python's >existing documentation? What should go into the official docs, and what >should go into the devguide? Once we agree on these question, I think the >test.support dilemma will solve itself. Let's see! I think the devguide should document things like "how to submit bugs", "how to use Mercurial", "how to propose changes to Python", "how to ensure code works across all existing interpreter implementations", "where to find continuous integration results and how to interpret them", "what's the right forum to discuss Python", etc. I.e. processes, workflows, and conventions that are important cultural artifacts we've built up over 20 years. I don't think the devguide should document the actual code we ship. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From solipsis at pitrou.net Fri Jul 29 17:17:31 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 29 Jul 2011 17:17:31 +0200 Subject: [Python-Dev] Status of the PEP 400? (deprecate codecs.StreamReader/StreamWriter) References: <4E308D63.9090901@haypocalc.com> <4E3125D7.2030103@egenix.com> <4E312BCB.3080301@haypocalc.com> Message-ID: <20110729171731.2059cc3e@pitrou.net> On Thu, 28 Jul 2011 11:28:43 +0200 Victor Stinner wrote: > > I will add your alternative to the PEP (except if you would like to do > that yourself?). If I understood correctly, you propose to: > > * rename codecs.open() to codecs.open_stream() > * change codecs.open() to reuse open() (and so io.TextIOWrapper) > > (and don't deprecate anything) This may be an interesting approach. In a few years, we can evaluate whether users are calling open_stream(), and if there aren't any, we can deprecate the whole thing. Regards Antoine. From solipsis at pitrou.net Fri Jul 29 17:25:00 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 29 Jul 2011 17:25:00 +0200 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> <20110729111837.4f5b88cd@resist.wooz.org> Message-ID: <20110729172500.67844af4@pitrou.net> On Fri, 29 Jul 2011 11:18:37 -0400 Barry Warsaw wrote: > > I'd much rather solve this problem by adding markup to functions that > explicitly disclaim our normal backward compatibility guarantees. Squirreling > away documentation for some parts of the stdlib seems similar to > security-by-obscurity arguments. Well, here the whole module should not be called by the user. I'm not sure we want to flag each function individually. > test.support *is* part of the stdlib. We have lots of internal APIs which are not documented, though. And test.support *is* for internal use. Regards Antoine. From ethan at stoneleaf.us Fri Jul 29 17:42:45 2011 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 29 Jul 2011 08:42:45 -0700 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: <20110729111837.4f5b88cd@resist.wooz.org> References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> <20110729111837.4f5b88cd@resist.wooz.org> Message-ID: <4E32D4F5.505@stoneleaf.us> Barry Warsaw wrote: > On Jul 29, 2011, at 02:07 PM, Eli Bendersky wrote: >> I think the unlink&rmtree functions are just a symptom. The real issue here >> is - what is the devguide for, and how is it different from Python's >> existing documentation? What should go into the official docs, and what >> should go into the devguide? Once we agree on these question, I think the >> test.support dilemma will solve itself. > > Let's see! > > I think the devguide should document things like "how to submit bugs", "how to > use Mercurial", "how to propose changes to Python", "how to ensure code works > across all existing interpreter implementations", "where to find continuous > integration results and how to interpret them", "what's the right forum to > discuss Python", etc. I.e. processes, workflows, and conventions that are > important cultural artifacts we've built up over 20 years. > > I don't think the devguide should document the actual code we ship. +1 ~Ethan~ From ncoghlan at gmail.com Fri Jul 29 17:32:00 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 30 Jul 2011 01:32:00 +1000 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: <20110729111837.4f5b88cd@resist.wooz.org> References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> <20110729111837.4f5b88cd@resist.wooz.org> Message-ID: On Sat, Jul 30, 2011 at 1:18 AM, Barry Warsaw wrote: > On Jul 29, 2011, at 02:07 PM, Eli Bendersky wrote: >>I think the unlink&rmtree functions are just a symptom. The real issue here >>is - what is the devguide for, and how is it different from Python's >>existing documentation? What should go into the official docs, and what >>should go into the devguide? Once we agree on these question, I think the >>test.support dilemma will solve itself. > > Let's see! > > I think the devguide should document things like "how to submit bugs", "how to > use Mercurial", "how to propose changes to Python", "how to ensure code works > across all existing interpreter implementations", "where to find continuous > integration results and how to interpret them", "what's the right forum to > discuss Python", etc. ?I.e. processes, workflows, and conventions that are > important cultural artifacts we've built up over 20 years. > > I don't think the devguide should document the actual code we ship. I think everything related to *changing* Python should be in the devguide, not the standard library docs. So the documentation on how to *run* the test suite belongs in the devguide, but the details of how the test suite works internally, including the APIs that are used to write new tests, belong in the dev guide. Now, perhaps a copy of the dev guide should be bundled with all future releases rather than relying on the availability of a net connection to access wart, but finally getting rid of the dodgy test.support sort-of-docs out of the standard library docs is an excellent change. As far as improving the arrangement goes, the checkin privileges are the same as those for the main source tree - patches welcome :) Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Fri Jul 29 17:33:53 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 30 Jul 2011 01:33:53 +1000 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> <20110729111837.4f5b88cd@resist.wooz.org> Message-ID: On Sat, Jul 30, 2011 at 1:32 AM, Nick Coghlan wrote: > So the documentation on how > to *run* the test suite belongs in the devguide, but the details of > how the test suite works internally, including the APIs that are used > to write new tests, belong in the dev guide. Gah, that first instance of 'the devguide' should read 'the standard library docs'. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Fri Jul 29 17:37:46 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 30 Jul 2011 01:37:46 +1000 Subject: [Python-Dev] Status of the PEP 400? (deprecate codecs.StreamReader/StreamWriter) In-Reply-To: <20110729171731.2059cc3e@pitrou.net> References: <4E308D63.9090901@haypocalc.com> <4E3125D7.2030103@egenix.com> <4E312BCB.3080301@haypocalc.com> <20110729171731.2059cc3e@pitrou.net> Message-ID: On Sat, Jul 30, 2011 at 1:17 AM, Antoine Pitrou wrote: > On Thu, 28 Jul 2011 11:28:43 +0200 > Victor Stinner wrote: >> >> I will add your alternative to the PEP (except if you would like to do >> that yourself?). If I understood correctly, you propose to: >> >> ? * rename codecs.open() to codecs.open_stream() >> ? * change codecs.open() to reuse open() (and so io.TextIOWrapper) >> >> (and don't deprecate anything) > > This may be an interesting approach. In a few years, we can evaluate > whether users are calling open_stream(), and if there aren't any, we > can deprecate the whole thing. Indeed. I'm also heavily influenced by MAL's opinion on this particular topic, so the fact he's OK with this approach counts for a lot. It achieves the main benefit I'm interested in (transparently migrating users of the codecs.open API to the new IO stack), while paving the way for eliminating the redundancy at some point in the future. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From v+python at g.nevcal.com Fri Jul 29 17:37:51 2011 From: v+python at g.nevcal.com (Glenn Linderman) Date: Fri, 29 Jul 2011 08:37:51 -0700 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: <20110729111837.4f5b88cd@resist.wooz.org> References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> <20110729111837.4f5b88cd@resist.wooz.org> Message-ID: <4E32D3CF.40107@g.nevcal.com> On 7/29/2011 8:18 AM, Barry Warsaw wrote: > I think the devguide should document things like ... > "how to ensure code works > across all existing interpreter implementations", "where to find continuous > integration results and how to interpret them" ... > I don't think the devguide should document the actual code we ship. Seems to me that test.support documentation helps explain the above items. -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Fri Jul 29 17:49:01 2011 From: barry at python.org (Barry Warsaw) Date: Fri, 29 Jul 2011 11:49:01 -0400 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> Message-ID: <20110729114901.1fbcbd12@resist.wooz.org> On Jul 30, 2011, at 01:02 AM, Nick Coghlan wrote: >It's worthwhile because it is what the devguide is for: documenting >how to *change* Python, rather than just using it as it is delivered >to you. There's a clear transition from user of Python to developer of >Python: you stop treating the standard library (and perhaps even the >interpreter core) as sacrosanct, and will actually change the code in >those files if its wrong (and is affecting your own application). I'm not so sure the line's that bright. But anyway... Using the criteria in your first sentence, test.support is clearly "what's delivered to you" not "changing Python". IOW, we ship test.support in the tarball, so *if* it's to be documented, then the stdlib is the right place to document it, IMO. If test.support is truly and only an internal implementation detail, then it should adhere to Pythonic convention for such things, and be renamed test._support. Then you won't need to document it at all except in the module. Here's another problem with moving the documentation for test.support to the devguide. They will invariably get out of sync. It's hard enough to keep stdlib code and stdlib docs in sync, but I think it will be even harder to keep stdlib code in sync with devguide documentation. It will also be harder to know what version of the devguide corresponds to what version of the code repository. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Fri Jul 29 17:51:18 2011 From: barry at python.org (Barry Warsaw) Date: Fri, 29 Jul 2011 11:51:18 -0400 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: <20110729172500.67844af4@pitrou.net> References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> <20110729111837.4f5b88cd@resist.wooz.org> <20110729172500.67844af4@pitrou.net> Message-ID: <20110729115118.35a5c640@resist.wooz.org> On Jul 29, 2011, at 05:25 PM, Antoine Pitrou wrote: >> test.support *is* part of the stdlib. > >We have lots of internal APIs which are not documented, though. >And test.support *is* for internal use. The solution then is to rename test.support to test._support to make it clear it's an internal implementation detail. Then you can remove the entire section from the stdlib docs and just document it in the code. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From status at bugs.python.org Fri Jul 29 18:07:27 2011 From: status at bugs.python.org (Python tracker) Date: Fri, 29 Jul 2011 18:07:27 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20110729160727.747221CDCC@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2011-07-22 - 2011-07-29) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 2889 ( +3) closed 21547 (+40) total 24436 (+43) Open issues with patches: 1259 Issues opened (31) ================== #12612: Valgrind suppressions http://bugs.python.org/issue12612 opened by Paul.Price #12613: itertools fixer fails http://bugs.python.org/issue12613 opened by VPeric #12614: Allow to explicitly set the method of urllib.request.Request http://bugs.python.org/issue12614 opened by tebeka #12616: zip fixer fails on zip()[:-1] http://bugs.python.org/issue12616 opened by VPeric #12617: Mutable Sequence Type can work not only with iterable in slice http://bugs.python.org/issue12617 opened by py.user #12618: py_compile cannot create files in current directory http://bugs.python.org/issue12618 opened by sjdv1982 #12619: Automatically regenerate platform-specific modules http://bugs.python.org/issue12619 opened by Arfrever #12622: failfast argument to TextTestRunner not documented http://bugs.python.org/issue12622 opened by pitrou #12623: "universal newlines" subprocess support broken with select- an http://bugs.python.org/issue12623 opened by pitrou #12625: sporadic test_unittest failure http://bugs.python.org/issue12625 opened by pitrou #12626: run test cases based on a glob filter http://bugs.python.org/issue12626 opened by pitrou #12627: Implement PEP 394: The "python" Command on Unix-Like Systems http://bugs.python.org/issue12627 opened by Kerrick.Staley #12629: HTMLParser silently stops parsing with malformed attributes http://bugs.python.org/issue12629 opened by teoryn #12631: Mutable Sequence Type in .remove() is consistent only with lis http://bugs.python.org/issue12631 opened by py.user #12632: Windows GPF with Code Page 65001 http://bugs.python.org/issue12632 opened by bferris57 #12633: sys.modules doc entry should reflect restrictions http://bugs.python.org/issue12633 opened by ericsnow #12634: Random Remarks in class documentation http://bugs.python.org/issue12634 opened by ats.engg #12636: IDLE ignores -*- coding -*- with -r option http://bugs.python.org/issue12636 opened by ledave123 #12638: urllib.URLopener prematurely deletes files on cleanup http://bugs.python.org/issue12638 opened by carlbook #12639: msilib Directory.start_component() fails if keyfile is not Non http://bugs.python.org/issue12639 opened by john.keeping #12640: test_ctypes seg fault (test_callback_register_double); armv7; http://bugs.python.org/issue12640 opened by python272 #12641: Remove -mno-cygwin from distutils http://bugs.python.org/issue12641 opened by jonforums #12643: code.InteractiveConsole ignores sys.excepthook http://bugs.python.org/issue12643 opened by aliles #12645: test.support. import_fresh_module - incorrect doc http://bugs.python.org/issue12645 opened by eli.bendersky #12646: zlib.Decompress.decompress/flush do not raise any exceptions w http://bugs.python.org/issue12646 opened by chortos #12648: Wrong import module search order on Windows http://bugs.python.org/issue12648 opened by kota #12649: email.Header ignores maxlinelen when wrapping encoded words http://bugs.python.org/issue12649 opened by dandre #12650: Subprocess leaks fd upon kill() http://bugs.python.org/issue12650 opened by gabriele.trombetti #12652: Move documentation of test.support into the devguide http://bugs.python.org/issue12652 opened by eli.bendersky #12653: Provide accelerators for all buttons in Windows installers http://bugs.python.org/issue12653 opened by cool-RR #12654: sum() works with bytes objects http://bugs.python.org/issue12654 opened by pitrou Most recent 15 issues with no replies (15) ========================================== #12654: sum() works with bytes objects http://bugs.python.org/issue12654 #12653: Provide accelerators for all buttons in Windows installers http://bugs.python.org/issue12653 #12648: Wrong import module search order on Windows http://bugs.python.org/issue12648 #12645: test.support. import_fresh_module - incorrect doc http://bugs.python.org/issue12645 #12640: test_ctypes seg fault (test_callback_register_double); armv7; http://bugs.python.org/issue12640 #12639: msilib Directory.start_component() fails if keyfile is not Non http://bugs.python.org/issue12639 #12623: "universal newlines" subprocess support broken with select- an http://bugs.python.org/issue12623 #12622: failfast argument to TextTestRunner not documented http://bugs.python.org/issue12622 #12616: zip fixer fails on zip()[:-1] http://bugs.python.org/issue12616 #12613: itertools fixer fails http://bugs.python.org/issue12613 #12612: Valgrind suppressions http://bugs.python.org/issue12612 #12599: Use 'is not None' where appropriate in importlib http://bugs.python.org/issue12599 #12598: Move sys variable initialization from import.c to sysmodule.c http://bugs.python.org/issue12598 #12586: Enhanced email API: header objects http://bugs.python.org/issue12586 #12562: calling mmap twice fails on Windows http://bugs.python.org/issue12562 Most recent 15 issues waiting for review (15) ============================================= #12652: Move documentation of test.support into the devguide http://bugs.python.org/issue12652 #12650: Subprocess leaks fd upon kill() http://bugs.python.org/issue12650 #12646: zlib.Decompress.decompress/flush do not raise any exceptions w http://bugs.python.org/issue12646 #12639: msilib Directory.start_component() fails if keyfile is not Non http://bugs.python.org/issue12639 #12633: sys.modules doc entry should reflect restrictions http://bugs.python.org/issue12633 #12627: Implement PEP 394: The "python" Command on Unix-Like Systems http://bugs.python.org/issue12627 #12626: run test cases based on a glob filter http://bugs.python.org/issue12626 #12625: sporadic test_unittest failure http://bugs.python.org/issue12625 #12619: Automatically regenerate platform-specific modules http://bugs.python.org/issue12619 #12618: py_compile cannot create files in current directory http://bugs.python.org/issue12618 #12614: Allow to explicitly set the method of urllib.request.Request http://bugs.python.org/issue12614 #12605: Enhancements to gdb 7 debugging hooks http://bugs.python.org/issue12605 #12604: VTRACE macro in _sre.c should use do {} while (0) http://bugs.python.org/issue12604 #12598: Move sys variable initialization from import.c to sysmodule.c http://bugs.python.org/issue12598 #12586: Enhanced email API: header objects http://bugs.python.org/issue12586 Top 10 most discussed issues (10) ================================= #12540: "Restart Shell" command leaves pythonw.exe processes running http://bugs.python.org/issue12540 18 msgs #12463: Calling SocketServer.shutdown() when server_forever() was not http://bugs.python.org/issue12463 15 msgs #670664: HTMLParser.py - more robust SCRIPT tag parsing http://bugs.python.org/issue670664 13 msgs #12464: tempfile.TemporaryDirectory.cleanup follows symbolic links http://bugs.python.org/issue12464 11 msgs #12394: packaging: generate scripts from callable (dotted paths) http://bugs.python.org/issue12394 10 msgs #12632: Windows GPF with Code Page 65001 http://bugs.python.org/issue12632 10 msgs #11049: add tests for test.support http://bugs.python.org/issue11049 9 msgs #12618: py_compile cannot create files in current directory http://bugs.python.org/issue12618 8 msgs #12641: Remove -mno-cygwin from distutils http://bugs.python.org/issue12641 8 msgs #12649: email.Header ignores maxlinelen when wrapping encoded words http://bugs.python.org/issue12649 8 msgs Issues closed (38) ================== #1813: Codec lookup failing under turkish locale http://bugs.python.org/issue1813 closed by pitrou #4506: 3.0 make test failures on Solaris 10 http://bugs.python.org/issue4506 closed by terry.reedy #8887: "pydoc str" works but not "pydoc str.translate" http://bugs.python.org/issue8887 closed by eric.araujo #9723: Add shlex.quote http://bugs.python.org/issue9723 closed by eric.araujo #10883: urllib: socket is not closed explicitly http://bugs.python.org/issue10883 closed by nadeem.vawda #11409: pysetup --search should return non-zero when a dist is not ins http://bugs.python.org/issue11409 closed by eric.araujo #11439: subversion keyword breakage http://bugs.python.org/issue11439 closed by python-dev #11784: multiprocessing.Process.join: timeout argument doesn't specify http://bugs.python.org/issue11784 closed by neologix #11871: test_default_timeout() of test_threading.BarrierTests failure: http://bugs.python.org/issue11871 closed by neologix #12043: Update shutil documentation http://bugs.python.org/issue12043 closed by eric.araujo #12102: mmap requires file to be synced http://bugs.python.org/issue12102 closed by rosslagerwall #12255: A few changes to .*ignore http://bugs.python.org/issue12255 closed by eric.araujo #12381: refactor slice checks made by methods that take "slice like" a http://bugs.python.org/issue12381 closed by petri.lehtinen #12439: BaseHTTPServer's send_reponse adds extra "\r\n" when using HTT http://bugs.python.org/issue12439 closed by petri.lehtinen #12514: timeit disables garbage collection if timed code raises an exc http://bugs.python.org/issue12514 closed by rhettinger #12542: Remove duplicates of cmp_to_key used in tests http://bugs.python.org/issue12542 closed by eric.araujo #12560: libpython.so not built on OpenBSD http://bugs.python.org/issue12560 closed by neologix #12576: urlib.request fails to open some sites http://bugs.python.org/issue12576 closed by python-dev #12581: Increased test coverage of test_urlparse http://bugs.python.org/issue12581 closed by python-dev #12590: First line and cursor not visible when opening files http://bugs.python.org/issue12590 closed by ned.deily #12591: TextIOWrapper should fall back on read() if read1() doesn't ex http://bugs.python.org/issue12591 closed by pitrou #12592: modify configure.in to detect OpenBSD 5.x http://bugs.python.org/issue12592 closed by neologix #12595: python.h redundant redeclarations http://bugs.python.org/issue12595 closed by petri.lehtinen #12603: pydoc.synopsis breaks if filesystem returns mtime of 0 http://bugs.python.org/issue12603 closed by neologix #12607: subprocess(stdout=..., stderr=sys.stdout) breaks stderr for ch http://bugs.python.org/issue12607 closed by rosslagerwall #12610: Fatal Python error: non-string found in code slot http://bugs.python.org/issue12610 closed by benjamin.peterson #12615: add array.zeroes http://bugs.python.org/issue12615 closed by rhettinger #12620: pending calls: make `pendingbusy` flag static to Py_MakePendin http://bugs.python.org/issue12620 closed by neologix #12621: Errors in docstrings of find and rfind methods of bytes and by http://bugs.python.org/issue12621 closed by python-dev #12624: failfast support for regrtest http://bugs.python.org/issue12624 closed by pitrou #12628: urllib.request.urlopen gives empty response bodies for some si http://bugs.python.org/issue12628 closed by ezio.melotti #12630: pydoc does not remove '#'-s from doc comments http://bugs.python.org/issue12630 closed by ncoghlan #12635: use "as" for block scope support http://bugs.python.org/issue12635 closed by ezio.melotti #12637: logging lastResort handler not ignoring messages less than WAR http://bugs.python.org/issue12637 closed by vinay.sajip #12642: Python2 documentation of the open() built-in function http://bugs.python.org/issue12642 closed by ezio.melotti #12644: document the "%a" conversion in the old string formatting oper http://bugs.python.org/issue12644 closed by eli.bendersky #12647: Add __bool__ to None http://bugs.python.org/issue12647 closed by rhettinger #12651: -O2 or -O3? this brings excitement to computing! http://bugs.python.org/issue12651 closed by benjamin.peterson From rdmurray at bitdance.com Fri Jul 29 18:13:44 2011 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 29 Jul 2011 12:13:44 -0400 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: <20110729111837.4f5b88cd@resist.wooz.org> References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> <20110729111837.4f5b88cd@resist.wooz.org> Message-ID: <20110729161345.0EE592506F0@webabinitio.net> On Fri, 29 Jul 2011 11:18:37 -0400, Barry Warsaw wrote: > On Jul 29, 2011, at 02:07 PM, Eli Bendersky wrote: > >Isn't this what we're trying to prevent, though? One should never even have > >to look at test.support unless he's working *on Python*. > > Again, I think that line is blurred here. Let's say you're working on some > off-beat or new hardware and you want to know if your basic tool chain works > for it. You build Python, run the test suite, and something fails. You > probably don't consider yourself a Python developer, but now you're digging > through the test.support to figure out your problem. Yes, I've seen this > happen. In that case, you are working *on Python*. Not using Python. Personally, I'd expect such information to be in a devguide, although really I'd not expect it to be documented at all, for most projects. Making the devguide a more prominent part of the documentation would be good: it would invite more people to cross that line and help us improve Python. As someone else pointed out, there is lots of stuff in the stdlib that is not public API, and therefore not documented. The problem with test.support is that it is not a public API, but *is* documented. So logically either we should remove the documentation, or we should not only do as you suggested by marking each function as not-a-stable-API, but also document as many of the other internal APIs as we care to in the same way. That would doubtless help our users, but more care than just marking functions as unstable would be required, I think. Personally, I always thought the devguide should be part of Docs anyway. It isn't because people didn't want it versioned along side Python, but I still don't see why that would be a problem. -- R. David Murray http://www.bitdance.com From rdmurray at bitdance.com Fri Jul 29 18:19:45 2011 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 29 Jul 2011 12:19:45 -0400 Subject: [Python-Dev] cpython (3.2): Stop ignoring Mercurial merge conflits files (#12255). In-Reply-To: <20110729164935.2dde6219@pitrou.net> References: <20110729145045.2c252ea0@pitrou.net> <4E32B58C.3070809@netwok.org> <20110729153831.455b5d67@pitrou.net> <4E32C1F6.4050204@netwok.org> <20110729164935.2dde6219@pitrou.net> Message-ID: <20110729161946.559ED2506F0@webabinitio.net> On Fri, 29 Jul 2011 16:49:35 +0200, Antoine Pitrou wrote: > On Fri, 29 Jul 2011 16:21:42 +0200 wrote: > > > What use are these backups really? We are using a (D)VCS, you are not > > > losing anything. > > > > The .orig files after a revert could contain code that???s not committed > > anywhere. See also RDM???s reply to your message. > > I would point out that by using "hg revert", you deliberately want to > forget some local changes. But *I'm* talking about 'patch'. That has nothing to do with either merge or revert. > Besides, "hg status" is meant to show untracked files which could > *potentially* be tracked. It's not like anybody wants to track .orig > and .rej files, so having them in the ignore list is still the right > thing to do. That's one view. My view is that 'hg status' tells me all the files that have appeared in my tree that I'm either not currently tracking or explicitly ignoring (because the project's automated tools will deal with them). Nothing in there about limiting it to files I *might* want to track. That is how I've always used my version control systems. -- R. David Murray http://www.bitdance.com From guido at python.org Fri Jul 29 19:01:06 2011 From: guido at python.org (Guido van Rossum) Date: Fri, 29 Jul 2011 10:01:06 -0700 Subject: [Python-Dev] Status of the PEP 400? (deprecate codecs.StreamReader/StreamWriter) In-Reply-To: References: <4E308D63.9090901@haypocalc.com> <4E3125D7.2030103@egenix.com> <4E312BCB.3080301@haypocalc.com> <20110729171731.2059cc3e@pitrou.net> Message-ID: On Fri, Jul 29, 2011 at 8:37 AM, Nick Coghlan wrote: > On Sat, Jul 30, 2011 at 1:17 AM, Antoine Pitrou wrote: >> On Thu, 28 Jul 2011 11:28:43 +0200 >> Victor Stinner wrote: >>> >>> I will add your alternative to the PEP (except if you would like to do >>> that yourself?). If I understood correctly, you propose to: >>> >>> ? * rename codecs.open() to codecs.open_stream() >>> ? * change codecs.open() to reuse open() (and so io.TextIOWrapper) >>> >>> (and don't deprecate anything) >> >> This may be an interesting approach. In a few years, we can evaluate >> whether users are calling open_stream(), and if there aren't any, we >> can deprecate the whole thing. > > Indeed. I'm also heavily influenced by MAL's opinion on this > particular topic, so the fact he's OK with this approach counts for a > lot. It achieves the main benefit I'm interested in (transparently > migrating users of the codecs.open API to the new IO stack), while > paving the way for eliminating the redundancy at some point in the > future. +1 -- --Guido van Rossum (python.org/~guido) From mal at egenix.com Fri Jul 29 19:14:16 2011 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 29 Jul 2011 19:14:16 +0200 Subject: [Python-Dev] Status of the PEP 400? (deprecate codecs.StreamReader/StreamWriter) In-Reply-To: <4E32A6DE.7060001@haypocalc.com> References: <4E308D63.9090901@haypocalc.com> <4E3125D7.2030103@egenix.com> <4E312BCB.3080301@haypocalc.com> <4E32A6DE.7060001@haypocalc.com> Message-ID: <4E32EA68.3010603@egenix.com> Victor Stinner wrote: > Le 28/07/2011 11:28, Victor Stinner a ?crit : >>> Please do keep the original implementation >>> around (e.g. renamed to codecs.open_stream()), though, so that it's >>> still possible to get easy-to-use access to codec StreamReader/Writers. >> >> I will add your alternative to the PEP (except if you would like to do >> that yourself?). If I understood correctly, you propose to: >> >> * rename codecs.open() to codecs.open_stream() >> * change codecs.open() to reuse open() (and so io.TextIOWrapper) >> >> (and don't deprecate anything) > > I added your proposal to the PEP as an "Alternative Approache". Thanks. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 29 2011) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From barry at python.org Fri Jul 29 19:27:31 2011 From: barry at python.org (Barry Warsaw) Date: Fri, 29 Jul 2011 13:27:31 -0400 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: <20110729161345.0EE592506F0@webabinitio.net> References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> <20110729111837.4f5b88cd@resist.wooz.org> <20110729161345.0EE592506F0@webabinitio.net> Message-ID: <20110729132731.51b27bb0@resist.wooz.org> On Jul 29, 2011, at 12:13 PM, R. David Murray wrote: >In that case, you are working *on Python*. Not using Python. My point was, it's a fine line between the two. >Personally, I always thought the devguide should be part of Docs anyway. >It isn't because people didn't want it versioned along side Python, >but I still don't see why that would be a problem. +1 -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From glyph at twistedmatrix.com Fri Jul 29 20:03:02 2011 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Fri, 29 Jul 2011 14:03:02 -0400 Subject: [Python-Dev] HTMLParser and HTML5 In-Reply-To: References: Message-ID: On Jul 29, 2011, at 7:46 AM, Stefan Behnel wrote: > Joao S. O. Bueno, 29.07.2011 13:22: >> On Fri, Jul 29, 2011 at 1:37 AM, Stefan Behnel wrote: >>> Brett Cannon, 28.07.2011 23:49: >>>> >>>> On Thu, Jul 28, 2011 at 11:25, Matt wrote: >>>>> >>>>> - What policies are in place for keeping parity with other HTML >>>>> parsers (such as those in web browsers)? >>>> >>>> There aren't any beyond "it would be nice". >>>> [...] >>>> It's more of an issue of someone caring enough to do the coding work to >>>> bring the parser up to spec for HTML5 (or introduce new code to live >>>> beside >>>> the HTML4 parsing code). >>> >>> Which, given that html5lib readily exists, would likely be a lot more work >>> than anyone who is interested in HTML5 handling would want to invest. >>> >>> I don't think we need a new HTML5 parsing implementation only to have it in >>> the stdlib. That's the old sunny Java way of doing it. >> >> I disaagree. >> Having proper html parsing out of the box is part of the "batteries >> included" thing. > > Well, you can easily prove me wrong by implementing this. > > Stefan Please don't implement this just to profe Stefan wrong :). The thing to do, if you want html parsing in the stdlib, is to _incorporate_ html5lib, which is already a perfectly good, thoroughly tested HTML parser, and simply deprecate HTMLParser and friends. Implementing a new parser would serve no purpose I can see. -glyph From tseaver at palladion.com Fri Jul 29 20:31:50 2011 From: tseaver at palladion.com (Tres Seaver) Date: Fri, 29 Jul 2011 14:31:50 -0400 Subject: [Python-Dev] HTMLParser and HTML5 In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 07/29/2011 07:22 AM, Joao S. O. Bueno wrote: > I disaagree. Having proper html parsing out of the box is part of > the "batteries included" thing. And it is not a matter of "having > html 5" - as stated on this thread, fixing it for html5 will fix it > for html that exists in the "real world". > > Python _has_ to work with quick 30-50 lines scripts deliverable > everywhere, not just has proper 3rd party libraries that can work as > part of a huge project using buildout. Assuming it were merged today, that parser would only be available on Python 3.3 and later: how is that "everywhere"? Having scripts that work against html5lib (which *doesn't* need buildout to install, or even setuptools) makes them portable to any version of Python supported by the library (Python 2.3+, AFAICT). Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk4y/JYACgkQ+gerLs4ltQ4KKwCgkyOlmb8xxhxg1qWH9RRbEpEw ne0AoL6NgRElbY61QRqnXJjiKoHq0ToW =fk3k -----END PGP SIGNATURE----- From rdmurray at bitdance.com Fri Jul 29 20:38:34 2011 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 29 Jul 2011 14:38:34 -0400 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: <20110729114901.1fbcbd12@resist.wooz.org> References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> <20110729114901.1fbcbd12@resist.wooz.org> Message-ID: <20110729183835.8BD9E2506C2@webabinitio.net> On Fri, 29 Jul 2011 11:49:01 -0400, Barry Warsaw wrote: > If test.support is truly and only an internal implementation detail, then it > should adhere to Pythonic convention for such things, and be renamed > test._support. Then you won't need to document it at all except in the > module. I'd be fine with renaming it and not documenting it, myself. Other developers prefer to have as much as practical documented in html form. > Here's another problem with moving the documentation for test.support to the > devguide. They will invariably get out of sync. It's hard enough to keep > stdlib code and stdlib docs in sync, but I think it will be even harder to > keep stdlib code in sync with devguide documentation. It will also be harder > to know what version of the devguide corresponds to what version of the code > repository. Which wouldn't be any more of a problem than the regular python docs if the devguide were in the normal doc tree. Just saying :) -- R. David Murray http://www.bitdance.com From mattbasta at gmail.com Fri Jul 29 21:00:01 2011 From: mattbasta at gmail.com (Matt) Date: Fri, 29 Jul 2011 12:00:01 -0700 Subject: [Python-Dev] HTMLParser and HTML5 In-Reply-To: References: Message-ID: On Fri, Jul 29, 2011 at 11:03 AM, Glyph Lefkowitz wrote: > > On Jul 29, 2011, at 7:46 AM, Stefan Behnel wrote: > > > Joao S. O. Bueno, 29.07.2011 13:22: > >> On Fri, Jul 29, 2011 at 1:37 AM, Stefan Behnel wrote: > >>> Brett Cannon, 28.07.2011 23:49: > >>>> > >>>> On Thu, Jul 28, 2011 at 11:25, Matt wrote: > >>>>> > >>>>> - What policies are in place for keeping parity with other HTML > >>>>> parsers (such as those in web browsers)? > >>>> > >>>> There aren't any beyond "it would be nice". > >>>> [...] > >>>> It's more of an issue of someone caring enough to do the coding work > to > >>>> bring the parser up to spec for HTML5 (or introduce new code to live > >>>> beside > >>>> the HTML4 parsing code). > >>> > >>> Which, given that html5lib readily exists, would likely be a lot more > work > >>> than anyone who is interested in HTML5 handling would want to invest. > >>> > >>> I don't think we need a new HTML5 parsing implementation only to have > it in > >>> the stdlib. That's the old sunny Java way of doing it. > >> > >> I disaagree. > >> Having proper html parsing out of the box is part of the "batteries > >> included" thing. > > > > Well, you can easily prove me wrong by implementing this. > As far as the issue described in my initial message goes, there is a patch and tests for the patch. > > Please don't implement this just to profe Stefan wrong :). > > The thing to do, if you want html parsing in the stdlib, is to > _incorporate_ html5lib, which is already a perfectly good, thoroughly tested > HTML parser, and simply deprecate HTMLParser and friends. Implementing a > new parser would serve no purpose I can see. > I don't see any real reason to drop a decent piece of code (HTMLParser, that is) in favor of a third party library when only relatively minor updates are needed to bring it up to speed with the latest spec. As far as structure goes, HTML4 and HTML5 are practically identical. The differences between the two that are applicable to HTMLParser involve the way the specs deal with special element types and broken syntax. For what it's worth, the rules HTML4 does define are (in many cases) ignored in favor of more modern, Postel's Law-agreeable rules. HTML5 simply standardized what browsers actually do. Deprecating HTMLParser in favor of a newer/better/faster HTML library is a bad thing for everybody that's already using HTMLParser, whether directly or indirectly. html5lib does not have an interface compatible with HTMLParser, so code would largely need to be rewritten from scratch to gain the benefits of HTML5's support for broken code. Developers using HTMLParser would be permanently stuck using a library that throws exceptions for perfectly valid HTML. Keep in mind that these are solved problems: all of the thinking on how to handle broken code has been done for us by the folks at the WHATWG. It's simply a matter of updating our existing code with these new rules. While I agree that there are merits to dropping support for the old code, it does not solve the existing problems that folks are having right now (namely incorrect parser output or exceptions). It would be more ideal to perhaps patch the obvious issues stemming from HTML4 support for now, leaving anything that goes beyond parity with browsers for a later time or implementing as an opt-in feature (i.e.: enabled by a parameter). Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Fri Jul 29 22:08:30 2011 From: brett at python.org (Brett Cannon) Date: Fri, 29 Jul 2011 13:08:30 -0700 Subject: [Python-Dev] [Python-checkins] cpython: Issue 12647: Add __bool__() method to the None object. In-Reply-To: References: Message-ID: On Thu, Jul 28, 2011 at 09:55, raymond.hettinger wrote: > http://hg.python.org/cpython/rev/ccce01988603 > changeset: 71542:ccce01988603 > user: Raymond Hettinger > date: Thu Jul 28 09:55:13 2011 -0700 > summary: > Issue 12647: Add __bool__() method to the None object. > > files: > Lib/test/test_descr.py | 3 - > Misc/NEWS | 4 ++ > Objects/object.c | 46 ++++++++++++++++++++++++++++- > 3 files changed, 48 insertions(+), 5 deletions(-) > > > diff --git a/Lib/test/test_descr.py b/Lib/test/test_descr.py > --- a/Lib/test/test_descr.py > +++ b/Lib/test/test_descr.py > @@ -2068,9 +2068,6 @@ > # Two essentially featureless objects, just inheriting stuff from > # object. > self.assertEqual(dir(NotImplemented), dir(Ellipsis)) > - if support.check_impl_detail(): > - # None differs in PyPy: it has a __nonzero__ > - self.assertEqual(dir(None), dir(Ellipsis)) > > Wasn't this change only in 3.3 where __nonzero__ doesn't exist? So when PyPy eventually supports Python 3 they will have to update to support __bool__ on None but this test won't exercise that for them. IOW I think the guard is wrong and should go. -Brett > # Nasty test case for proxied objects > class Wrapper(object): > diff --git a/Misc/NEWS b/Misc/NEWS > --- a/Misc/NEWS > +++ b/Misc/NEWS > @@ -13,6 +13,10 @@ > - Verify the types of AST strings and identifiers provided by the user > before > compiling them. > > +- Issue #12647: The None object now has a __bool__() method that returns > False. > + Formerly, bool(None) returned False only because of special case logic > + in PyObject_IsTrue(). > + > - Issue #12579: str.format_map() now raises a ValueError if used on a > format string that contains positional fields. Initial patch by > Julian Berman. > diff --git a/Objects/object.c b/Objects/object.c > --- a/Objects/object.c > +++ b/Objects/object.c > @@ -1255,7 +1255,7 @@ > } > > /* > -None is as a non-NULL undefined value. > +None is a non-NULL undefined value. > There is (and should be!) no way to create other objects of this type, > so there is exactly one (which is indestructible, by the way). > */ > @@ -1277,6 +1277,48 @@ > Py_FatalError("deallocating None"); > } > > +static int > +none_bool(PyObject *v) > +{ > + return 0; > +} > + > +static PyNumberMethods none_as_number = { > + 0, /* nb_add */ > + 0, /* nb_subtract */ > + 0, /* nb_multiply */ > + 0, /* nb_remainder */ > + 0, /* nb_divmod */ > + 0, /* nb_power */ > + 0, /* nb_negative */ > + 0, /* nb_positive */ > + 0, /* nb_absolute */ > + (inquiry)none_bool, /* nb_bool */ > + 0, /* nb_invert */ > + 0, /* nb_lshift */ > + 0, /* nb_rshift */ > + 0, /* nb_and */ > + 0, /* nb_xor */ > + 0, /* nb_or */ > + 0, /* nb_int */ > + 0, /* nb_reserved */ > + 0, /* nb_float */ > + 0, /* nb_inplace_add */ > + 0, /* nb_inplace_subtract */ > + 0, /* nb_inplace_multiply */ > + 0, /* nb_inplace_remainder */ > + 0, /* nb_inplace_power */ > + 0, /* nb_inplace_lshift */ > + 0, /* nb_inplace_rshift */ > + 0, /* nb_inplace_and */ > + 0, /* nb_inplace_xor */ > + 0, /* nb_inplace_or */ > + 0, /* nb_floor_divide */ > + 0, /* nb_true_divide */ > + 0, /* nb_inplace_floor_divide */ > + 0, /* nb_inplace_true_divide */ > + 0, /* nb_index */ > +}; > > static PyTypeObject PyNone_Type = { > PyVarObject_HEAD_INIT(&PyType_Type, 0) > @@ -1289,7 +1331,7 @@ > 0, /*tp_setattr*/ > 0, /*tp_reserved*/ > none_repr, /*tp_repr*/ > - 0, /*tp_as_number*/ > + &none_as_number, /*tp_as_number*/ > 0, /*tp_as_sequence*/ > 0, /*tp_as_mapping*/ > 0, /*tp_hash */ > > -- > Repository URL: http://hg.python.org/cpython > > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From glyph at twistedmatrix.com Fri Jul 29 22:16:07 2011 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Fri, 29 Jul 2011 16:16:07 -0400 Subject: [Python-Dev] HTMLParser and HTML5 In-Reply-To: References: Message-ID: On Jul 29, 2011, at 3:00 PM, Matt wrote: > I don't see any real reason to drop a decent piece of code (HTMLParser, that is) in favor of a third party library when only relatively minor updates are needed to bring it up to speed with the latest spec. I am not really one to throw stones here, as Twisted contains a lenient pseudo-XML parser which I still maintain - one which decidedly does not agree with html5's requirements for dealing with invalid data, but just a bunch of ad-hoc guesses of my own. My impression of HTML5 is that HTMLParser would require significant modifications and possibly a drastic re-architecture in order to really do HTML5 "right"; especially the parts that the html5lib authors claim makes HTML5 streaming-unfriendly, i.e. subtree reordering when encountering certain types of invalid data. But if I'm wrong about that, and there are just a few spec updates and bugfixes that need to be applied, by all means, ignore my comment. -glyph -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Fri Jul 29 22:31:48 2011 From: brett at python.org (Brett Cannon) Date: Fri, 29 Jul 2011 13:31:48 -0700 Subject: [Python-Dev] HTMLParser and HTML5 In-Reply-To: References: Message-ID: On Fri, Jul 29, 2011 at 11:31, Tres Seaver wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 07/29/2011 07:22 AM, Joao S. O. Bueno wrote: > > > I disaagree. Having proper html parsing out of the box is part of > > the "batteries included" thing. And it is not a matter of "having > > html 5" - as stated on this thread, fixing it for html5 will fix it > > for html that exists in the "real world". > > > > Python _has_ to work with quick 30-50 lines scripts deliverable > > everywhere, not just has proper 3rd party libraries that can work as > > part of a huge project using buildout. > > Assuming it were merged today, that parser would only be available on > Python 3.3 and later: how is that "everywhere"? Well, "everywhere, eventually". This gets down to the usual philosophical debate of what should (not) be in the stdlib so that those who have strict third-party code get access to useful libraries while balancing the desire of those who want to keep the stdlib lean or prevent stagnating the API of a module. > Having scripts that > work against html5lib (which *doesn't* need buildout to install, or even > setuptools) makes them portable to any version of Python supported by > the library (Python 2.3+, AFAICT). > If the library was brought in they could probably continue to be portable with possibly just the addition of a try/finally on the import line. -Brett > > > Tres. > - -- > =================================================================== > Tres Seaver +1 540-429-0999 tseaver at palladion.com > Palladion Software "Excellence by Design" http://palladion.com > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iEYEARECAAYFAk4y/JYACgkQ+gerLs4ltQ4KKwCgkyOlmb8xxhxg1qWH9RRbEpEw > ne0AoL6NgRElbY61QRqnXJjiKoHq0ToW > =fk3k > -----END PGP SIGNATURE----- > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/brett%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Fri Jul 29 22:34:13 2011 From: brett at python.org (Brett Cannon) Date: Fri, 29 Jul 2011 13:34:13 -0700 Subject: [Python-Dev] HTMLParser and HTML5 In-Reply-To: References: Message-ID: On Fri, Jul 29, 2011 at 13:16, Glyph Lefkowitz wrote: > On Jul 29, 2011, at 3:00 PM, Matt wrote: > > I don't see any real reason to drop a decent piece of code (HTMLParser, > that is) in favor of a third party library when only relatively minor > updates are needed to bring it up to speed with the latest spec. > > > I am not really one to throw stones here, as Twisted contains a lenient > pseudo-XML parser which I still maintain - one which decidedly does *not* agree > with html5's requirements for dealing with invalid data, but just a bunch of > ad-hoc guesses of my own. > > My impression of HTML5 is that HTMLParser would require significant > modifications and possibly a drastic re-architecture in order to really do > HTML5 "right"; especially the parts that the html5lib authors claim makes > HTML5 streaming-unfriendly, i.e. subtree reordering when encountering > certain types of invalid data. > We could also have the code live side-by-side for a while (or indefinitely if that was really desired) by bringing html5lib in as either a separate module or having the relevant classes live in htmllib under different names. But all of this is just hypothetical until someone decides to do the legwork to actually make a proposal and get the coding done. -Brett > > But if I'm wrong about that, and there are just a few spec updates and > bugfixes that need to be applied, by all means, ignore my comment. > > -glyph > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/brett%40python.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Fri Jul 29 23:32:57 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 29 Jul 2011 23:32:57 +0200 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> <20110729111837.4f5b88cd@resist.wooz.org> <20110729172500.67844af4@pitrou.net> <20110729115118.35a5c640@resist.wooz.org> Message-ID: <20110729233257.47f03c0d@pitrou.net> On Fri, 29 Jul 2011 11:51:18 -0400 Barry Warsaw wrote: > On Jul 29, 2011, at 05:25 PM, Antoine Pitrou wrote: > > >> test.support *is* part of the stdlib. > > > >We have lots of internal APIs which are not documented, though. > >And test.support *is* for internal use. > > The solution then is to rename test.support to test._support to make it clear > it's an internal implementation detail. Then you can remove the entire > section from the stdlib docs and just document it in the code. Ideally so. Practically, it's a lot of churn and additional pain merging 3.2 bugfixes into default. The lack of an underscore doesn't always mean the API is public, because it hasn't always worked like this (we have many private APIs without an underscore). Regards Antoine. From solipsis at pitrou.net Fri Jul 29 23:36:03 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 29 Jul 2011 23:36:03 +0200 Subject: [Python-Dev] cpython (3.2): Stop ignoring Mercurial merge conflits files (#12255). In-Reply-To: <20110729161946.559ED2506F0@webabinitio.net> References: <20110729145045.2c252ea0@pitrou.net> <4E32B58C.3070809@netwok.org> <20110729153831.455b5d67@pitrou.net> <4E32C1F6.4050204@netwok.org> <20110729164935.2dde6219@pitrou.net> <20110729161946.559ED2506F0@webabinitio.net> Message-ID: <20110729233603.6a0adf1c@pitrou.net> On Fri, 29 Jul 2011 12:19:45 -0400 "R. David Murray" wrote: > > > Besides, "hg status" is meant to show untracked files which could > > *potentially* be tracked. It's not like anybody wants to track .orig > > and .rej files, so having them in the ignore list is still the right > > thing to do. > > That's one view. My view is that 'hg status' tells me all the files > that have appeared in my tree that I'm either not currently tracking or > explicitly ignoring (because the project's automated tools will deal > with them). Nothing in there about limiting it to files I *might* > want to track. That is how I've always used my version control > systems. Ok, I understand. However, it also makes things more tedious for other people who don't user their VCS in such a way, so it would be nice how other people feel about this. Regards Antoine. From solipsis at pitrou.net Fri Jul 29 23:35:07 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 29 Jul 2011 23:35:07 +0200 Subject: [Python-Dev] HTMLParser and HTML5 References: Message-ID: <20110729233507.6b863182@pitrou.net> On Fri, 29 Jul 2011 13:34:13 -0700 Brett Cannon wrote: > On Fri, Jul 29, 2011 at 13:16, Glyph Lefkowitz wrote: > > > On Jul 29, 2011, at 3:00 PM, Matt wrote: > > > > I don't see any real reason to drop a decent piece of code (HTMLParser, > > that is) in favor of a third party library when only relatively minor > > updates are needed to bring it up to speed with the latest spec. > > > > > > I am not really one to throw stones here, as Twisted contains a lenient > > pseudo-XML parser which I still maintain - one which decidedly does *not* agree > > with html5's requirements for dealing with invalid data, but just a bunch of > > ad-hoc guesses of my own. > > > > My impression of HTML5 is that HTMLParser would require significant > > modifications and possibly a drastic re-architecture in order to really do > > HTML5 "right"; especially the parts that the html5lib authors claim makes > > HTML5 streaming-unfriendly, i.e. subtree reordering when encountering > > certain types of invalid data. > > > > We could also have the code live side-by-side for a while (or indefinitely > if that was really desired) by bringing html5lib in as either a separate > module or having the relevant classes live in htmllib under different names. Unless html5lib is better in some fundamental ways which are difficult to fix in htmllib, I'm not sure there's any point in adding it to the stdlib. We don't really do users a service if we keep adding alternative APIs for common functionality. Regards Antoine. From tjreedy at udel.edu Sat Jul 30 00:42:46 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 29 Jul 2011 18:42:46 -0400 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: <20110729111837.4f5b88cd@resist.wooz.org> References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> <20110729111837.4f5b88cd@resist.wooz.org> Message-ID: On 7/29/2011 11:18 AM, Barry Warsaw wrote: > I'd much rather solve this problem by adding markup to functions that > explicitly disclaim our normal backward compatibility guarantees. I suggested adding a footnote marker (1) to each one. > test.support *is* part of the stdlib. > So once again, is this a real actual problem you've witnessed in enough cases > to go through all the trouble of moving the documentation, and making it more > difficult to find? I agree with this. When working on tests, I would prefer to have test.support doc in the same offine doc set as unittest docs. I only supported *moving* as an alternative to *deleting*. Terry J. Reedy From tjreedy at udel.edu Sat Jul 30 00:47:07 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 29 Jul 2011 18:47:07 -0400 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: <20110729172500.67844af4@pitrou.net> References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> <20110729111837.4f5b88cd@resist.wooz.org> <20110729172500.67844af4@pitrou.net> Message-ID: On 7/29/2011 11:25 AM, Antoine Pitrou wrote: t > We have lots of internal APIs which are not documented, though. They are generally used only within the module itself as helper functions. So one only needs to even know about them when looking at the module code. > And test.support *is* for internal use. No, the stuff in there is *not* for internal use within the module but for external use is possiby every test module. Terry J. Reedy From solipsis at pitrou.net Sat Jul 30 00:54:22 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 30 Jul 2011 00:54:22 +0200 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> <20110729111837.4f5b88cd@resist.wooz.org> <20110729172500.67844af4@pitrou.net> Message-ID: <20110730005422.05487503@pitrou.net> On Fri, 29 Jul 2011 18:47:07 -0400 Terry Reedy wrote: > > > And test.support *is* for internal use. > > No, the stuff in there is *not* for internal use within the module but > for external use is possiby every test module. I meant internal use for us. Really, whether or not it's used cross-module is irrelevant. Regards Antoine. From tjreedy at udel.edu Sat Jul 30 01:02:32 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 29 Jul 2011 19:02:32 -0400 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: <20110729233257.47f03c0d@pitrou.net> References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> <20110729111837.4f5b88cd@resist.wooz.org> <20110729172500.67844af4@pitrou.net> <20110729115118.35a5c640@resist.wooz.org> <20110729233257.47f03c0d@pitrou.net> Message-ID: On 7/29/2011 5:32 PM, Antoine Pitrou wrote: > On Fri, 29 Jul 2011 11:51:18 -0400 > Barry Warsaw wrote: >> On Jul 29, 2011, at 05:25 PM, Antoine Pitrou wrote: >> >>>> test.support *is* part of the stdlib. >>> >>> We have lots of internal APIs which are not documented, though. >>> And test.support *is* for internal use. >> >> The solution then is to rename test.support to test._support to make it clear >> it's an internal implementation detail. Then you can remove the entire >> section from the stdlib docs and just document it in the code. > > Ideally so. The effect of this will be to discourage new people (including myself in the category) from writing or reviewing patches. Terry Jan Reedy From solipsis at pitrou.net Sat Jul 30 01:27:05 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 30 Jul 2011 01:27:05 +0200 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> <20110729111837.4f5b88cd@resist.wooz.org> <20110729172500.67844af4@pitrou.net> <20110729115118.35a5c640@resist.wooz.org> <20110729233257.47f03c0d@pitrou.net> Message-ID: <20110730012705.647abc9f@pitrou.net> On Fri, 29 Jul 2011 19:02:32 -0400 Terry Reedy wrote: > On 7/29/2011 5:32 PM, Antoine Pitrou wrote: > > On Fri, 29 Jul 2011 11:51:18 -0400 > > Barry Warsaw wrote: > >> On Jul 29, 2011, at 05:25 PM, Antoine Pitrou wrote: > >> > >>>> test.support *is* part of the stdlib. > >>> > >>> We have lots of internal APIs which are not documented, though. > >>> And test.support *is* for internal use. > >> > >> The solution then is to rename test.support to test._support to make it clear > >> it's an internal implementation detail. Then you can remove the entire > >> section from the stdlib docs and just document it in the code. > > > > Ideally so. > > The effect of this will be to discourage new people (including myself in > the category) from writing or reviewing patches. I'm sorry, can you be more precise. The effect of what? And why would that be so? From ben+python at benfinney.id.au Sat Jul 30 04:00:53 2011 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 30 Jul 2011 12:00:53 +1000 Subject: [Python-Dev] cpython (3.2): Stop ignoring Mercurial merge conflits files (#12255). References: <20110729145045.2c252ea0@pitrou.net> <4E32B58C.3070809@netwok.org> <20110729153831.455b5d67@pitrou.net> <4E32C1F6.4050204@netwok.org> <20110729164935.2dde6219@pitrou.net> Message-ID: <87r558msru.fsf@benfinney.id.au> Antoine Pitrou writes: > Besides, "hg status" is meant to show untracked files which could > *potentially* be tracked. And if you don't want to track them, you need to deal with them somehow. > It's not like anybody wants to track .orig and .rej files, so having > them in the ignore list is still the right thing to do. No, because their appearance in the working tree means something needs to be dealt with by the programmer. That's unlike other files which are generated and *don't* need to be dealt with by the programmer (or can be deterministically regenerated at will), so can safely be ignored. -- \ ?When I was a kid I used to pray every night for a new bicycle. | `\ Then I realised that the Lord doesn't work that way so I stole | _o__) one and asked Him to forgive me.? ?Emo Philips | Ben Finney From rdmurray at bitdance.com Sat Jul 30 04:22:05 2011 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 29 Jul 2011 22:22:05 -0400 Subject: [Python-Dev] cpython (3.2): Stop ignoring Mercurial merge conflits files (#12255). In-Reply-To: <20110729233603.6a0adf1c@pitrou.net> References: <20110729145045.2c252ea0@pitrou.net> <4E32B58C.3070809@netwok.org> <20110729153831.455b5d67@pitrou.net> <4E32C1F6.4050204@netwok.org> <20110729164935.2dde6219@pitrou.net> <20110729161946.559ED2506F0@webabinitio.net> <20110729233603.6a0adf1c@pitrou.net> Message-ID: <20110730022206.B12412505A8@webabinitio.net> On Fri, 29 Jul 2011 23:36:03 +0200, Antoine Pitrou wrote: > On Fri, 29 Jul 2011 12:19:45 -0400 > "R. David Murray" wrote: > > > > > Besides, "hg status" is meant to show untracked files which could > > > *potentially* be tracked. It's not like anybody wants to track .orig > > > and .rej files, so having them in the ignore list is still the right > > > thing to do. > > > > That's one view. My view is that 'hg status' tells me all the files > > that have appeared in my tree that I'm either not currently tracking or > > explicitly ignoring (because the project's automated tools will deal > > with them). Nothing in there about limiting it to files I *might* > > want to track. That is how I've always used my version control > > systems. > > Ok, I understand. However, it also makes things more tedious for other > people who don't user their VCS in such a way, so it would be nice how > other people feel about this. They can add those files to their personal .hgrc. I can't *remove* those ignores via mine. -- R. David Murray http://www.bitdance.com From rdmurray at bitdance.com Sat Jul 30 04:26:28 2011 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 29 Jul 2011 22:26:28 -0400 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: <20110729233257.47f03c0d@pitrou.net> References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> <20110729111837.4f5b88cd@resist.wooz.org> <20110729172500.67844af4@pitrou.net> <20110729115118.35a5c640@resist.wooz.org> <20110729233257.47f03c0d@pitrou.net> Message-ID: <20110730022629.18BE82505A8@webabinitio.net> On Fri, 29 Jul 2011 23:32:57 +0200, Antoine Pitrou wrote: > On Fri, 29 Jul 2011 11:51:18 -0400 > Barry Warsaw wrote: > > On Jul 29, 2011, at 05:25 PM, Antoine Pitrou wrote: > > > > >> test.support *is* part of the stdlib. > > > > > >We have lots of internal APIs which are not documented, though. > > >And test.support *is* for internal use. > > > > The solution then is to rename test.support to test._support to make it clear > > it's an internal implementation detail. Then you can remove the entire > > section from the stdlib docs and just document it in the code. > > Ideally so. Practically, it's a lot of churn and additional pain > merging 3.2 bugfixes into default. The lack of an underscore doesn't > always mean the API is public, because it hasn't always worked like > this (we have many private APIs without an underscore). I'm not sure it makes merging more difficult. I haven't had any problems with email test merges even though I moved (i.e. renamed) the test directory. -- R. David Murray http://www.bitdance.com From guido at python.org Sat Jul 30 04:28:30 2011 From: guido at python.org (Guido van Rossum) Date: Fri, 29 Jul 2011 19:28:30 -0700 Subject: [Python-Dev] cpython (3.2): Stop ignoring Mercurial merge conflits files (#12255). In-Reply-To: <20110730022206.B12412505A8@webabinitio.net> References: <20110729145045.2c252ea0@pitrou.net> <4E32B58C.3070809@netwok.org> <20110729153831.455b5d67@pitrou.net> <4E32C1F6.4050204@netwok.org> <20110729164935.2dde6219@pitrou.net> <20110729161946.559ED2506F0@webabinitio.net> <20110729233603.6a0adf1c@pitrou.net> <20110730022206.B12412505A8@webabinitio.net> Message-ID: On Fri, Jul 29, 2011 at 7:22 PM, R. David Murray wrote: > On Fri, 29 Jul 2011 23:36:03 +0200, Antoine Pitrou wrote: >> On Fri, 29 Jul 2011 12:19:45 -0400 >> "R. David Murray" wrote: >> > >> > > Besides, "hg status" is meant to show untracked files which could >> > > *potentially* be tracked. It's not like anybody wants to track .orig >> > > and .rej files, so having them in the ignore list is still the right >> > > thing to do. >> > >> > That's one view. ?My view is that 'hg status' tells me all the files >> > that have appeared in my tree that I'm either not currently tracking or >> > explicitly ignoring (because the project's automated tools will deal >> > with them). ?Nothing in there about limiting it to files I *might* >> > want to track. ?That is how I've always used my version control >> > systems. >> >> Ok, I understand. However, it also makes things more tedious for other >> people who don't user their VCS in such a way, so it would be nice how >> other people feel about this. > > They can add those files to their personal .hgrc. ?I can't *remove* > those ignores via mine. Well, *I* don't ever want to check in .orig or .rej files (because various tools create them) so I want them in my .hgignore file. Here's a sample .hgignore file I carry around with me: http://code.google.com/p/appengine-ndb-experiment/source/browse/.hgignore -- --Guido van Rossum (python.org/~guido) From eliben at gmail.com Sat Jul 30 05:30:40 2011 From: eliben at gmail.com (Eli Bendersky) Date: Sat, 30 Jul 2011 06:30:40 +0300 Subject: [Python-Dev] the history of tests being inside Lib/ in Python Message-ID: The other thread had some claims (*) that made me wonder - why are the tests in Python kept in Lib/ at all? AFAIK, this is rather an unusual project structure. Tests usually have a top-level directory of their own, in parallel to Lib/, Doc/ and others. Some effects of this in other projects: * The tests usually aren't even installed. The user can run them during installation, but once it goes through, tests are not copied into /usr/whatever... * Tests naturally become "developer-domain", removed from the "user-domain". No sane user would even consider using code from inside the Tests/ directory and somehow expect it to keep working in later versions. In addition, tests are then usually documented in special "hacking guides" and "developer docs" instead of in the official documentation of the project. This mail can appear as if advocating the transfer of Lib/test into Tests/, but this is not my intention here. Honest :-) I'm just trying to understand the history and rationale behind this structure in the CPython project. Eli (*) I refer to this reasoning someone raised: "test.support is part of the tests" + "tests are part of stdlib" --> "test.support must be documented where the rest of stdlib is" -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin at python.org Sat Jul 30 05:36:21 2011 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 29 Jul 2011 22:36:21 -0500 Subject: [Python-Dev] the history of tests being inside Lib/ in Python In-Reply-To: References: Message-ID: 2011/7/29 Eli Bendersky : > The other thread had some claims (*) that made me wonder - why are the tests > in Python kept in Lib/ at all? > > AFAIK, this is rather an unusual project structure. Not really. It seems to be about half/half to me. -- Regards, Benjamin From eliben at gmail.com Sat Jul 30 05:44:07 2011 From: eliben at gmail.com (Eli Bendersky) Date: Sat, 30 Jul 2011 06:44:07 +0300 Subject: [Python-Dev] the history of tests being inside Lib/ in Python In-Reply-To: References: Message-ID: On Sat, Jul 30, 2011 at 06:36, Benjamin Peterson wrote: > 2011/7/29 Eli Bendersky : > > The other thread had some claims (*) that made me wonder - why are the > tests > > in Python kept in Lib/ at all? > > > > AFAIK, this is rather an unusual project structure. > > Not really. It seems to be about half/half to me. > I guess I'm mis-informed then :) Out of curiosity, could you point to a few projects that also place tests inside their code? Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From nad at acm.org Sat Jul 30 05:48:44 2011 From: nad at acm.org (Ned Deily) Date: Fri, 29 Jul 2011 20:48:44 -0700 Subject: [Python-Dev] the history of tests being inside Lib/ in Python References: Message-ID: In article , Eli Bendersky wrote: > * The tests usually aren't even installed. The user can run them during > installation, but once it goes through, tests are not copied into > /usr/whatever... That's not true across the board. For instance, the python.org Mac OS X installers do install the tests "permanently" and, with the addition of the "-m test" (or "-m test.regrtest" in 2.7) command line options, the tests can be easily run at any time on the end user's system. That's one of the reasons why it is important that additions and changes to the tests need to ensure that auxiliary directories and files are installed by the Makefile install targets and not just resident in the build directory. -- Ned Deily, nad at acm.org From guido at python.org Sat Jul 30 05:50:09 2011 From: guido at python.org (Guido van Rossum) Date: Fri, 29 Jul 2011 20:50:09 -0700 Subject: [Python-Dev] the history of tests being inside Lib/ in Python In-Reply-To: References: Message-ID: On Fri, Jul 29, 2011 at 8:44 PM, Eli Bendersky wrote: > > On Sat, Jul 30, 2011 at 06:36, Benjamin Peterson > wrote: >> >> 2011/7/29 Eli Bendersky : >> > The other thread had some claims (*) that made me wonder - why are the >> > tests >> > in Python kept in Lib/ at all? >> > >> > AFAIK, this is rather an unusual project structure. >> >> Not really. It seems to be about half/half to me. > > I guess I'm mis-informed then :) Out of curiosity, could you point to a few > projects that also place tests inside their code? You're asking too many questions. Please do some research of your own. Really, this isn't going to change, and asking why it is so or stating that it is rather unusual is just going to irritate the experienced developers. If you want a friendlier answer, please sign up for core-mentorship at python.org, where asking questions about why or why not are welcome. -- --Guido van Rossum (python.org/~guido) From benjamin at python.org Sat Jul 30 05:55:51 2011 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 29 Jul 2011 22:55:51 -0500 Subject: [Python-Dev] the history of tests being inside Lib/ in Python In-Reply-To: References: Message-ID: 2011/7/29 Eli Bendersky : > > On Sat, Jul 30, 2011 at 06:36, Benjamin Peterson > wrote: >> >> 2011/7/29 Eli Bendersky : >> > The other thread had some claims (*) that made me wonder - why are the >> > tests >> > in Python kept in Lib/ at all? >> > >> > AFAIK, this is rather an unusual project structure. >> >> Not really. It seems to be about half/half to me. > > I guess I'm mis-informed then :) Out of curiosity, could you point to a few > projects that also place tests inside their code? Twisted, PyPy, simplejson, Bazaar, buildbot, pyramid, SAGE, Jinja2, Portage, pylint, and Tahoe-LAFS for starters. -- Regards, Benjamin From g.brandl at gmx.net Sat Jul 30 06:35:35 2011 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 30 Jul 2011 06:35:35 +0200 Subject: [Python-Dev] cpython: make the types of None and Ellipsis callable In-Reply-To: References: Message-ID: Am 30.07.2011 01:20, schrieb benjamin.peterson: > http://hg.python.org/cpython/rev/84c3be27b4c7 > changeset: 71614:84c3be27b4c7 > parent: 71611:a6afd26caa8a > user: Benjamin Peterson > date: Fri Jul 29 18:19:43 2011 -0500 > summary: > make the types of None and Ellipsis callable > > files: > Lib/test/test_builtin.py | 7 +++++ > Misc/NEWS | 3 ++ > Objects/object.c | 34 ++++++++++++++++++++++++++++ > Objects/sliceobject.c | 29 +++++++++++++++++++++++ > 4 files changed, 73 insertions(+), 0 deletions(-) Shouldn't there be a matching Doc change somewhere? Georg From solipsis at pitrou.net Sat Jul 30 08:36:41 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 30 Jul 2011 08:36:41 +0200 Subject: [Python-Dev] cpython (3.2): Stop ignoring Mercurial merge conflits files (#12255). In-Reply-To: <20110730022206.B12412505A8@webabinitio.net> References: <20110729145045.2c252ea0@pitrou.net> <4E32B58C.3070809@netwok.org> <20110729153831.455b5d67@pitrou.net> <4E32C1F6.4050204@netwok.org> <20110729164935.2dde6219@pitrou.net> <20110729161946.559ED2506F0@webabinitio.net> <20110729233603.6a0adf1c@pitrou.net> <20110730022206.B12412505A8@webabinitio.net> Message-ID: <20110730083641.47bb3c73@pitrou.net> On Fri, 29 Jul 2011 22:22:05 -0400 "R. David Murray" wrote: > On Fri, 29 Jul 2011 23:36:03 +0200, Antoine Pitrou wrote: > > On Fri, 29 Jul 2011 12:19:45 -0400 > > "R. David Murray" wrote: > > > > > > > Besides, "hg status" is meant to show untracked files which could > > > > *potentially* be tracked. It's not like anybody wants to track .orig > > > > and .rej files, so having them in the ignore list is still the right > > > > thing to do. > > > > > > That's one view. My view is that 'hg status' tells me all the files > > > that have appeared in my tree that I'm either not currently tracking or > > > explicitly ignoring (because the project's automated tools will deal > > > with them). Nothing in there about limiting it to files I *might* > > > want to track. That is how I've always used my version control > > > systems. > > > > Ok, I understand. However, it also makes things more tedious for other > > people who don't user their VCS in such a way, so it would be nice how > > other people feel about this. > > They can add those files to their personal .hgrc. I can't *remove* > those ignores via mine. That's a good point. I hadn't thought about that. Regards Antoine. From solipsis at pitrou.net Sat Jul 30 08:46:00 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 30 Jul 2011 08:46:00 +0200 Subject: [Python-Dev] cpython: Fix closes Issue11281 - smtplib.STMP gets source_address parameter, which adds References: Message-ID: <20110730084600.33036cb3@pitrou.net> Hi Senthil, > + if source_address: self.source_address = source_address Could you try to follow PEP 8? (I know PEP 8 is not always followed in old code, but there's no reason not to follow it in code that we add to the stdlib) > + SMTP.__init__(self, host, port, local_hostname = local_hostname, > + source_address = source_address) Ditto here (and other similar occurrences). > + def testSourceAddress(self): > + # connect > + smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3, > + source_address=('127.0.0.1', 19876)) > + self.assertEqual(smtp.source_address, ('127.0.0.1', 19876)) > + self.assertEqual(smtp.local_hostname, 'localhost') Unless this test is also using some kind of mock socket (it doesn't seem to), this can break as soon as port 19876 is already in use. There are utilities in test.support to help with this, such as bind_port() or (less reliable) find_unused_port(). Actually, this can be triggered quite easily by running the test a couple of times in parallel: $ ./python -m test -m testSourceAddress -Fv -j3 test_smtplib [...] [ 14/4] test_smtplib testSourceAddress (test.test_smtplib.GeneralTests) ... ok testSourceAddress (test.test_smtplib.DebuggingServerTests) ... ERROR ====================================================================== ERROR: testSourceAddress (test.test_smtplib.DebuggingServerTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/antoine/cpython/default/Lib/test/test_smtplib.py", line 220, in testSourceAddress source_address=('127.0.0.1', 19876)) File "/home/antoine/cpython/default/Lib/smtplib.py", line 238, in __init__ (code, msg) = self.connect(host, port) File "/home/antoine/cpython/default/Lib/smtplib.py", line 313, in connect self.sock = self._get_socket(host, port, self.timeout) File "/home/antoine/cpython/default/Lib/smtplib.py", line 287, in _get_socket self.source_address) File "/home/antoine/cpython/default/Lib/socket.py", line 407, in create_connection raise err File "/home/antoine/cpython/default/Lib/socket.py", line 397, in create_connection sock.bind(source_address) socket.error: [Errno 98] Address already in use ---------------------------------------------------------------------- > + print(dir(smtp)) Usually, we avoid printing anything in tests, except when support.verbose is True. Regards Antoine. From g.brandl at gmx.net Sat Jul 30 08:55:09 2011 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 30 Jul 2011 08:55:09 +0200 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: <20110729052651.311dbd53@resist.wooz.org> References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> Message-ID: Am 29.07.2011 11:26, schrieb Barry Warsaw: > So I'm curious, why is this move better than adding noindexes, or just > trusting users to understand the difference between test.support.unlink() and > os.unlink()? If I currently search for 'unlink', os.unlink comes up first, > which is good, and that should be preserved. The presence or not of some > test.support.unlink documentation isn't going to make the search results that > much better or worse (there's already 14 hits). Also don't forget that this is open-source: we shouldn't do questionable decisions (-1 from me on moving test.support docs to the devguide) just because the tool is lacking. I can probably hack up a small addition to Doc/tools/sphinxext in five minutes that ignores the whole of test.* when building the index. Georg From g.brandl at gmx.net Sat Jul 30 08:55:59 2011 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 30 Jul 2011 08:55:59 +0200 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: References: <20110727152410.22871784@pitrou.net> Message-ID: Am 27.07.2011 19:44, schrieb Terry Reedy: > On 7/27/2011 9:24 AM, Antoine Pitrou wrote: > >> Docstrings are sufficient for own our purposes. > > >>> import test.support as t > >>> help(t.rmtree) > Help on function rmtree in module test.support: > > rmtree(path) Well, what are you waiting for... just add the docstring! Georg From g.brandl at gmx.net Sat Jul 30 09:02:12 2011 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 30 Jul 2011 09:02:12 +0200 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: References: <20110727133619.F0B622506ED@webabinitio.net> Message-ID: Am 27.07.2011 19:47, schrieb Terry Reedy: > On 7/27/2011 1:27 PM, Brett Cannon wrote: > >> Perhaps what we could do is move the documentation for test.support to >> the devguide, and then vet the test suite so that unlink and friends >> are always called as 'support.unlink', etc. >> >> >> I like this solution since this issue of documenting test.support keeps >> coming up. Otherwise we can not document test.support, > > We already do. > > 25.6. test.support ? Utility functions for tests FWIW, this heading could give wrong impressions. I just changed it to :mod:`test.support` --- Utilities for the Python test suite and also added another note under it that the API is subject to change at any time. > is about half of the page that also contains > 25.5. test ? Regression tests package for Python > The latter contains > 25.5.1. Writing Unit Tests for the test package > which should also be moved to the dev guide if 25.6 is. Georg From sandro.tosi at gmail.com Sat Jul 30 11:40:52 2011 From: sandro.tosi at gmail.com (Sandro Tosi) Date: Sat, 30 Jul 2011 11:40:52 +0200 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <20110720040505.400E23A4116@sparrow.telecommunity.com> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> Message-ID: Hi, sorry for nitpicking, but... On Wed, Jul 20, 2011 at 05:58, P.J. Eby wrote: ... > For those implementing PEP \302 importer objects: the '\' should be removed, right? Cheers, -- Sandro Tosi (aka morph, morpheus, matrixhasu) My website: http://matrixhasu.altervista.org/ Me at Debian: http://wiki.debian.org/SandroTosi From merwok at netwok.org Sat Jul 30 14:57:47 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Sat, 30 Jul 2011 14:57:47 +0200 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: References: <20110720040505.400E23A4116@sparrow.telecommunity.com> Message-ID: <4E33FFCB.6040507@netwok.org> Hi Sandro, > On Wed, Jul 20, 2011 at 05:58, P.J. Eby wrote: >> For those implementing PEP \302 importer objects: > > the '\' should be removed, right? No. Philip used backslashes to prevent the HTML conversion to transform each and every instance of ?PEP \d+? to a link, which gets annoying after the few first hundred times. (It was discussed a few months ago probably on web-sig or python-dev for PEP 333 or 3333, if memory serves.) Cheers From tjreedy at udel.edu Sat Jul 30 15:07:19 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 30 Jul 2011 09:07:19 -0400 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: <20110730005422.05487503@pitrou.net> References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> <20110729111837.4f5b88cd@resist.wooz.org> <20110729172500.67844af4@pitrou.net> <20110730005422.05487503@pitrou.net> Message-ID: On 7/29/2011 6:54 PM, Antoine Pitrou wrote: > On Fri, 29 Jul 2011 18:47:07 -0400 > Terry Reedy wrote: >> >>> And test.support *is* for internal use. >> >> No, the stuff in there is *not* for internal use within the module but >> for external use is possiby every test module. > > I meant internal use for us. Really, whether or not it's > used cross-module is irrelevant. It is not at all irrelevant to me as a newer developer. I see uses of test.support.x quite often in checkins and I apparently should know about the contents of test.support for writing tests so I can use things as appropriate. Ditto for everyone else. On the other hand, the private support functions in trace.py are irrelevant to everyone except for the occasional person doing occasion maintenance work on that module. Terry Jan Reedy From tjreedy at udel.edu Sat Jul 30 15:25:27 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 30 Jul 2011 09:25:27 -0400 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: <20110730012705.647abc9f@pitrou.net> References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> <20110729111837.4f5b88cd@resist.wooz.org> <20110729172500.67844af4@pitrou.net> <20110729115118.35a5c640@resist.wooz.org> <20110729233257.47f03c0d@pitrou.net> <20110730012705.647abc9f@pitrou.net> Message-ID: On 7/29/2011 7:27 PM, Antoine Pitrou wrote: > On Fri, 29 Jul 2011 19:02:32 -0400 > Terry Reedy wrote: >> On 7/29/2011 5:32 PM, Antoine Pitrou wrote: >>> On Fri, 29 Jul 2011 11:51:18 -0400 >>> Barry Warsaw wrote: >>>> The solution then is to rename test.support to test._support to make it clear >>>> it's an internal implementation detail. Then you can remove the entire >>>> section from the stdlib docs and just document it in the code. >>> >>> Ideally so. >> >> The effect of this will be to discourage new people (including myself in >> the category) from writing or reviewing patches. > > I'm sorry, can you be more precise. The effect of what? Your proposal to remove the current formatted documentation of test.support instead of completing it and force all developers to only have reference to the docstrings scattered through the code. > And why would that be so? Because not everyone is like you. Because the condensed formatted docs make learning and using a module significantly easier for some people. Terry Jan Reedy From rdmurray at bitdance.com Sat Jul 30 15:31:55 2011 From: rdmurray at bitdance.com (R. David Murray) Date: Sat, 30 Jul 2011 09:31:55 -0400 Subject: [Python-Dev] the history of tests being inside Lib/ in Python In-Reply-To: References: Message-ID: <20110730133155.DDF6E2506C2@webabinitio.net> On Sat, 30 Jul 2011 06:30:40 +0300, Eli Bendersky wrote: > This mail can appear as if advocating the transfer of Lib/test into Tests/, > but this is not my intention here. Honest :-) I'm just trying to understand > the history and rationale behind this structure in the CPython project. My understanding (I could well be wrong) is that one reason is that we prefer that the tests be installed. That is also the reason that a number of tests that use large data files fetch them from the network (if and only if the relevant resource is enabled). -- R. David Murray http://www.bitdance.com From solipsis at pitrou.net Sat Jul 30 15:38:00 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 30 Jul 2011 15:38:00 +0200 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> <20110729111837.4f5b88cd@resist.wooz.org> <20110729172500.67844af4@pitrou.net> <20110729115118.35a5c640@resist.wooz.org> <20110729233257.47f03c0d@pitrou.net> <20110730012705.647abc9f@pitrou.net> Message-ID: <20110730153800.4cb9a540@pitrou.net> On Sat, 30 Jul 2011 09:25:27 -0400 Terry Reedy wrote: > > > > I'm sorry, can you be more precise. The effect of what? > > Your proposal to remove the current formatted documentation of > test.support instead of completing it and force all developers to only > have reference to the docstrings scattered through the code. > > > And why would that be so? > > Because not everyone is like you. Because the condensed formatted docs > make learning and using a module significantly easier for some people. Ok, I understand. I just hope the notice at the top of the module doc is prominent enough that nobody will think there's any API guarantee there. Regards Antoine. From senthil at uthcode.com Sat Jul 30 16:07:58 2011 From: senthil at uthcode.com (Senthil Kumaran) Date: Sat, 30 Jul 2011 22:07:58 +0800 Subject: [Python-Dev] cpython: Fix closes Issue11281 - smtplib.STMP gets source_address parameter, which adds In-Reply-To: <20110730084600.33036cb3@pitrou.net> References: <20110730084600.33036cb3@pitrou.net> Message-ID: <20110730140758.GA2262@mathmagic> Hello Antoine, On Sat, Jul 30, 2011 at 08:46:00AM +0200, Antoine Pitrou wrote: > (I know PEP 8 is not always followed in old code, but there's no reason > not to follow it in code that we add to the stdlib) > Thanks for pointing out. I somehow overlooked it. I shall refactor that lib. > Unless this test is also using some kind of mock socket (it doesn't > seem to), this can break as soon as port 19876 is already in use. Yes, there is one test which does not follow the mock socket and had not realized this it may break when run in parallel (and 19876 is use). Shall use the test facilities which are provided to resolve (/synchronize) that condition. > > + print(dir(smtp)) :-) That's was definitely my unintentional mistake. Funny that when I ran the individual tests a couple of times, I did not see that remaining and hard to see it when run the entire suite. Should be removed. -- Senthil From benjamin at python.org Sat Jul 30 16:12:19 2011 From: benjamin at python.org (Benjamin Peterson) Date: Sat, 30 Jul 2011 09:12:19 -0500 Subject: [Python-Dev] cpython: make the types of None and Ellipsis callable In-Reply-To: References: Message-ID: 2011/7/29 Georg Brandl : > Am 30.07.2011 01:20, schrieb benjamin.peterson: >> http://hg.python.org/cpython/rev/84c3be27b4c7 >> changeset: ? 71614:84c3be27b4c7 >> parent: ? ? ?71611:a6afd26caa8a >> user: ? ? ? ?Benjamin Peterson >> date: ? ? ? ?Fri Jul 29 18:19:43 2011 -0500 >> summary: >> ? make the types of None and Ellipsis callable >> >> files: >> ? Lib/test/test_builtin.py | ? 7 +++++ >> ? Misc/NEWS ? ? ? ? ? ? ? ?| ? 3 ++ >> ? Objects/object.c ? ? ? ? | ?34 ++++++++++++++++++++++++++++ >> ? Objects/sliceobject.c ? ?| ?29 +++++++++++++++++++++++ >> ? 4 files changed, 73 insertions(+), 0 deletions(-) > > Shouldn't there be a matching Doc change somewhere? Somewhere is the question! -- Regards, Benjamin From sandro.tosi at gmail.com Sat Jul 30 16:43:50 2011 From: sandro.tosi at gmail.com (Sandro Tosi) Date: Sat, 30 Jul 2011 16:43:50 +0200 Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning" In-Reply-To: <4E33FFCB.6040507@netwok.org> References: <20110720040505.400E23A4116@sparrow.telecommunity.com> <4E33FFCB.6040507@netwok.org> Message-ID: On Sat, Jul 30, 2011 at 14:57, ?ric Araujo wrote: >> On Wed, Jul 20, 2011 at 05:58, P.J. Eby wrote: >>> For those implementing PEP \302 importer objects: >> >> the '\' should be removed, right? > > No. ?Philip used backslashes to prevent the HTML conversion to transform > each and every instance of ?PEP \d+? to a link, which gets annoying > after the few first hundred times. ?(It was discussed a few months ago > probably on web-sig or python-dev for PEP 333 or 3333, if memory serves.) Gaah, sorry for the noise then! (but at least I learnt a new thing!) Cheers, -- Sandro Tosi (aka morph, morpheus, matrixhasu) My website: http://matrixhasu.altervista.org/ Me at Debian: http://wiki.debian.org/SandroTosi From ncoghlan at gmail.com Sat Jul 30 17:23:40 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 31 Jul 2011 01:23:40 +1000 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: <20110729114901.1fbcbd12@resist.wooz.org> References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> <20110729114901.1fbcbd12@resist.wooz.org> Message-ID: On Sat, Jul 30, 2011 at 1:49 AM, Barry Warsaw wrote: > On Jul 30, 2011, at 01:02 AM, Nick Coghlan wrote: > If test.support is truly and only an internal implementation detail, then it > should adhere to Pythonic convention for such things, and be renamed > test._support. ?Then you won't need to document it at all except in the > module. Aside from test.regrtest and test.support, the entirety of the rest of the test package is completely undocumented. test.support is unique, as it is the only component other than the main executable that is documented *at all*, solely for the benefits of people writing new tests. We don't even document the resources that can be enabled from the command line, instead telling people to look at the command line help output. The entire test package is an implementation detail, underscore or no underscore - we will never, ever, offer any kind of backwards compatibility guarantee for code in that part of the package hierarchy. > Here's another problem with moving the documentation for test.support to the > devguide. ?They will invariably get out of sync. ?It's hard enough to keep > stdlib code and stdlib docs in sync, but I think it will be even harder to > keep stdlib code in sync with devguide documentation. ?It will also be harder > to know what version of the devguide corresponds to what version of the code > repository. By that reasoning, we should just give up and delete the test.support docs entirely (since they're *already* treated with disrespect by contrast to the real stdlib docs). It sounds to me like you're really objecting to the devguide living in a separate clone. This doesn't bode well for the prospects of ever splitting the stdlib out from the CPython interpreter core... Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Sat Jul 30 17:30:41 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 31 Jul 2011 01:30:41 +1000 Subject: [Python-Dev] cpython: make the types of None and Ellipsis callable In-Reply-To: References: Message-ID: On Sun, Jul 31, 2011 at 12:12 AM, Benjamin Peterson wrote: > 2011/7/29 Georg Brandl : >> Shouldn't there be a matching Doc change somewhere? > > Somewhere is the question! While None/NotImplemented/Ellipsis are all documented as singletons, the behaviour of calling their types is not specified anywhere. If this was to be detailed anywhere, then the sections on None/NotImplemented/Ellipsis in section 3.2 of the language reference would be the place. And on that note... was there any particular reason for leaving NotImplemented out for this change? Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From barry at python.org Sat Jul 30 17:38:22 2011 From: barry at python.org (Barry Warsaw) Date: Sat, 30 Jul 2011 11:38:22 -0400 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> <20110729114901.1fbcbd12@resist.wooz.org> Message-ID: <20110730113822.469a0d6b@resist.wooz.org> On Jul 31, 2011, at 01:23 AM, Nick Coghlan wrote: >It sounds to me like you're really objecting to the devguide living in >a separate clone. This doesn't bode well for the prospects of ever >splitting the stdlib out from the CPython interpreter core... Actually, no. I'm objecting to moving documentation for code to a different repo than the code. If/when the stdlib is split out (which I support), then the documentation should go with it. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From ncoghlan at gmail.com Sat Jul 30 18:00:45 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 31 Jul 2011 02:00:45 +1000 Subject: [Python-Dev] Convention on functions that shadow existing stdlib functions In-Reply-To: <20110730113822.469a0d6b@resist.wooz.org> References: <20110727152410.22871784@pitrou.net> <4E30A4F1.6050305@pearwood.info> <20110729052651.311dbd53@resist.wooz.org> <20110729114901.1fbcbd12@resist.wooz.org> <20110730113822.469a0d6b@resist.wooz.org> Message-ID: On Sun, Jul 31, 2011 at 1:38 AM, Barry Warsaw wrote: > On Jul 31, 2011, at 01:23 AM, Nick Coghlan wrote: > >>It sounds to me like you're really objecting to the devguide living in >>a separate clone. This doesn't bode well for the prospects of ever >>splitting the stdlib out from the CPython interpreter core... > > Actually, no. ?I'm objecting to moving documentation for code to a different > repo than the code. ?If/when the stdlib is split out (which I support), then > the documentation should go with it. That's a rather valid point that I can definitely agree with. OK, you've convinced me that the right thing to do is leave the test.support docs where they are and just apply the appropriate markup to keep them out of the various indices. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Sat Jul 30 18:18:15 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 31 Jul 2011 02:18:15 +1000 Subject: [Python-Dev] [Python-checkins] cpython: Issue 12647: Add __bool__() method to the None object. In-Reply-To: References: Message-ID: On Sat, Jul 30, 2011 at 6:08 AM, Brett Cannon wrote: > Wasn't this change only in 3.3 where __nonzero__ doesn't exist? So when PyPy > eventually supports Python 3 they will have to update to support __bool__ on > None but this test won't exercise that for them. IOW I think the guard is > wrong and should go. The entire assertion was removed by Raymond's checkin, as the addition of None.__bool__ made it false on CPython as well. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Sat Jul 30 18:19:23 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 31 Jul 2011 02:19:23 +1000 Subject: [Python-Dev] cpython: make the types of None and Ellipsis callable In-Reply-To: References: Message-ID: On Sun, Jul 31, 2011 at 1:30 AM, Nick Coghlan wrote: > And on that note... was there any particular reason for leaving > NotImplemented out for this change? Never mind, just noticed the subsequest checkin. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From g.brandl at gmx.net Sat Jul 30 19:12:02 2011 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 30 Jul 2011 19:12:02 +0200 Subject: [Python-Dev] cpython (2.7): note Ellipsis syntax In-Reply-To: References: Message-ID: On 07/30/11 17:00, benjamin.peterson wrote: > http://hg.python.org/cpython/rev/402f94edf11b > changeset: 71637:402f94edf11b > branch: 2.7 > user: Benjamin Peterson > date: Sat Jul 30 09:59:12 2011 -0500 > summary: > note Ellipsis syntax > > files: > Doc/library/stdtypes.rst | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > > diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst > --- a/Doc/library/stdtypes.rst > +++ b/Doc/library/stdtypes.rst > @@ -2930,7 +2930,7 @@ > supports no special operations. There is exactly one ellipsis object, named > :const:`Ellipsis` (a built-in name). > > -It is written as ``Ellipsis``. > +It is written as ``Ellipsis`` or ``...``. In 2.7, this is not true; ``...`` only works in slices there. Georg From g.brandl at gmx.net Sat Jul 30 19:13:20 2011 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 30 Jul 2011 19:13:20 +0200 Subject: [Python-Dev] cpython: we can call singleton types now In-Reply-To: References: Message-ID: On 07/30/11 17:03, benjamin.peterson wrote: > http://hg.python.org/cpython/rev/4a07b772f0e0 > changeset: 71639:4a07b772f0e0 > user: Benjamin Peterson > date: Sat Jul 30 10:03:09 2011 -0500 > summary: > we can call singleton types now > > files: > Doc/library/stdtypes.rst | 8 +++++--- > 1 files changed, 5 insertions(+), 3 deletions(-) > > > diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst > --- a/Doc/library/stdtypes.rst > +++ b/Doc/library/stdtypes.rst > @@ -2706,7 +2706,7 @@ > > This object is returned by functions that don't explicitly return a value. It > supports no special operations. There is exactly one null object, named > -``None`` (a built-in name). > +``None`` (a built-in name). Calling ``type(None)`` produces the same singleton. I know this is technically correct, but it will look like you mean "calling type with None as argument" (same for the other singletons). Probably better to say something like "``type(None)()`` produces ...". Georg From benjamin at python.org Sat Jul 30 19:25:23 2011 From: benjamin at python.org (Benjamin Peterson) Date: Sat, 30 Jul 2011 12:25:23 -0500 Subject: [Python-Dev] cpython (2.7): note Ellipsis syntax In-Reply-To: References: Message-ID: 2011/7/30 Georg Brandl : > On 07/30/11 17:00, benjamin.peterson wrote: >> http://hg.python.org/cpython/rev/402f94edf11b >> changeset: ? 71637:402f94edf11b >> branch: ? ? ?2.7 >> user: ? ? ? ?Benjamin Peterson >> date: ? ? ? ?Sat Jul 30 09:59:12 2011 -0500 >> summary: >> ? note Ellipsis syntax >> >> files: >> ? Doc/library/stdtypes.rst | ?2 +- >> ? 1 files changed, 1 insertions(+), 1 deletions(-) >> >> >> diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst >> --- a/Doc/library/stdtypes.rst >> +++ b/Doc/library/stdtypes.rst >> @@ -2930,7 +2930,7 @@ >> ?supports no special operations. ?There is exactly one ellipsis object, named >> ?:const:`Ellipsis` (a built-in name). >> >> -It is written as ``Ellipsis``. >> +It is written as ``Ellipsis`` or ``...``. > > In 2.7, this is not true; ``...`` only works in slices there. I know, but why would you use Ellipsis outside of slices? -- Regards, Benjamin From ezio.melotti at gmail.com Sat Jul 30 21:05:20 2011 From: ezio.melotti at gmail.com (Ezio Melotti) Date: Sat, 30 Jul 2011 22:05:20 +0300 Subject: [Python-Dev] [Python-checkins] cpython: Modernize modulefinder module and tests a bit. In-Reply-To: References: Message-ID: <4E3455F0.5010507@gmail.com> Hi, On 29/07/2011 15.35, eric.araujo wrote: > http://hg.python.org/cpython/rev/1521d9837d16 > changeset: 71569:1521d9837d16 > user: ?ric Araujo > date: Thu Jul 28 23:35:29 2011 +0200 > summary: > Modernize modulefinder module and tests a bit. > > The tests don?t use an internal distutils function anymore, and use > regular assertEqual with sorted lists instead of a convoluted manual > diff. > > files: > Lib/modulefinder.py | 15 ++---- > Lib/test/test_modulefinder.py | 48 +++++++++++----------- > 2 files changed, 31 insertions(+), 32 deletions(-) > > > diff --git a/Lib/modulefinder.py b/Lib/modulefinder.py > --- a/Lib/modulefinder.py > +++ b/Lib/modulefinder.py > @@ -1,6 +1,5 @@ > """Find modules used by a script, using introspection.""" > > -from __future__ import generators > import dis > import imp > import marshal > @@ -9,8 +8,6 @@ > import types > import struct > > -READ_MODE = "rU" > - > # XXX Clean up once str8's cstor matches bytes. > LOAD_CONST = bytes([dis.opname.index('LOAD_CONST')]) > IMPORT_NAME = bytes([dis.opname.index('IMPORT_NAME')]) > @@ -29,8 +26,7 @@ > > # A Public interface > def AddPackagePath(packagename, path): > - paths = packagePathMap.get(packagename, []) > - paths.append(path) > + paths = packagePathMap.setdefault(packagename, []).append(path) I'm assuming that packagePathMap is a dict that might contain or not a *packagename* key that maps to a list of paths. Now, unless I'm missing something, the old code assigned to *paths* the list of paths or [] if it wasn't there, and then appended *path* to it. AFAICS, the new code introduced two changes: 1) the packagename key is added to the dict if it was missing -- and this seems reasonable; 2) append is now on the same line, it returns None, and None is assigned to *paths* -- and this seems wrong; > packagePathMap[packagename] = paths Also this is not necessary anymore if you use setdefault. > replacePackageMap = {} > @@ -106,14 +102,14 @@ > > [...] Best Regards, Ezio Melotti From ezio.melotti at gmail.com Sat Jul 30 22:11:08 2011 From: ezio.melotti at gmail.com (Ezio Melotti) Date: Sat, 30 Jul 2011 23:11:08 +0300 Subject: [Python-Dev] [Python-checkins] cpython: Fix closes Issue11281 - smtplib.STMP gets source_address parameter, which adds In-Reply-To: References: Message-ID: <4E34655C.1030201@gmail.com> Hi, On 30/07/2011 5.58, senthil.kumaran wrote: > http://hg.python.org/cpython/rev/26839edf3cc1 > changeset: 71617:26839edf3cc1 > parent: 71613:018e14a46454 > user: Senthil Kumaran > date: Sat Jul 30 10:56:50 2011 +0800 > summary: > Fix closes Issue11281 - smtplib.STMP gets source_address parameter, which adds the ability to bind to specific source address on a machine with multiple interfaces. Patch by Paulo Scardine. > > files: > Doc/library/smtplib.rst | 33 +++++++++++++++++----- > Lib/smtplib.py | 40 ++++++++++++++++++--------- > Lib/test/mock_socket.py | 3 +- > Lib/test/test_smtplib.py | 17 +++++++++++ > Misc/NEWS | 4 ++ > 5 files changed, 75 insertions(+), 22 deletions(-) > > > diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst > --- a/Doc/library/smtplib.rst > +++ b/Doc/library/smtplib.rst > @@ -20,7 +20,7 @@ > Protocol) and :rfc:`1869` (SMTP Service Extensions). > > > -.. class:: SMTP(host='', port=0, local_hostname=None[, timeout]) > +.. class:: SMTP(host='', port=0, local_hostname=None[, timeout], source_address=None) The "[, timeout]" now looks weird there, and it would be better to convert it to ", timeout=..." to match the other args. However I don't know what the value should be, since the real value is socket._GLOBAL_DEFAULT_TIMEOUT (i.e. object()) and I don't think it's a good idea to expose that. Maybe "None" can be used instead? > > A :class:`SMTP` instance encapsulates an SMTP connection. It has methods > that support a full repertoire of SMTP and ESMTP operations. If the optional > @@ -29,7 +29,12 @@ > raised if the specified host doesn't respond correctly. The optional > *timeout* parameter specifies a timeout in seconds for blocking operations > like the connection attempt (if not specified, the global default timeout > - setting will be used). > + setting will be used). The optional source_address parameter allows to bind to some > + specific source address in a machine with multiple network interfaces, > + and/or to some specific source tcp port. It takes a 2-tuple (host, port), I think TCP should be uppercase. > + for the socket to bind to as its source address before connecting. If > + ommited (or if host or port are '' and/or 0 respectively) the OS default s/ommited/omitted/ and s/''/``''``/ > + behavior will be used. > > For normal use, you should only require the initialization/connect, > :meth:`sendmail`, and :meth:`quit` methods. An example is included below. > @@ -48,8 +53,10 @@ > .. versionchanged:: 3.3 > Support for the :keyword:`with` statement was added. > > + .. versionadded:: 3.3 > + source_address parameter. I think the convention is to use "versionadded" when the function/method/class/etc has been added, and "versionchanged" for all the changes, including new arguments. > -.. class:: SMTP_SSL(host='', port=0, local_hostname=None, keyfile=None, certfile=None[, timeout], context=None) > +.. class:: SMTP_SSL(host='', port=0, local_hostname=None, keyfile=None, certfile=None[, timeout], context=None, source_address=None) Ditto for "[, timeout]" and the typos/markup below. > A :class:`SMTP_SSL` instance behaves exactly the same as instances of > :class:`SMTP`. :class:`SMTP_SSL` should be used for situations where SSL is > @@ -62,18 +69,28 @@ > keyfile and certfile must be None. The optional *timeout* > parameter specifies a timeout in seconds for blocking operations like the > connection attempt (if not specified, the global default timeout setting > - will be used). > + will be used). The optional source_address parameter allows to bind to some > + specific source address in a machine with multiple network interfaces, > + and/or to some specific source tcp port. It takes a 2-tuple (host, port), > + for the socket to bind to as its source address before connecting. If > + ommited (or if host or port are '' and/or 0 respectively) the OS default > + behavior will be used. > > .. versionchanged:: 3.3 > *context* was added. > > + .. versionadded:: 3.3 > + source_address parameter. > > -.. class:: LMTP(host='', port=LMTP_PORT, local_hostname=None) > + > +.. class:: LMTP(host='', port=LMTP_PORT, local_hostname=None, source_address=None) > > The LMTP protocol, which is very similar to ESMTP, is heavily based on the > - standard SMTP client. It's common to use Unix sockets for LMTP, so our :meth:`connect` > - method must support that as well as a regular host:port server. To specify a > - Unix socket, you must use an absolute path for *host*, starting with a '/'. > + standard SMTP client. It's common to use Unix sockets for LMTP, so our > + :meth:`connect` method must support that as well as a regular host:port > + server. The optional parameters local_hostname and source_address has the s/has/have/? Also I prefer 'arguments' rather than 'parameters', the smtplib doc uses both, but 'arguments' seems to be used more. > + same meaning as that of SMTP client.To specify a Unix socket, you must use Missing space after the '.' (there should be two spaces, but here a single space is used consistently so it's fine). > + an absolute path for *host*, starting with a '/'. > > Authentication is supported, using the regular SMTP mechanism. When using a Unix > socket, LMTP generally don't support or require any authentication, but your > diff --git a/Lib/smtplib.py b/Lib/smtplib.py > --- a/Lib/smtplib.py > +++ b/Lib/smtplib.py > @@ -215,7 +215,8 @@ > [...] Best Regards, Ezio Melotti From greg.ewing at canterbury.ac.nz Sun Jul 31 01:44:30 2011 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 31 Jul 2011 11:44:30 +1200 Subject: [Python-Dev] cpython (2.7): note Ellipsis syntax In-Reply-To: References: Message-ID: <4E34975E.9060004@canterbury.ac.nz> Benjamin Peterson wrote: > why would you use Ellipsis outside of slices? I could imagine someone wanting to use it as part of a function API. For example, print(a, b, c, ...) would have been a nice way to tell print() not to put a newline on the end. -- Greg From senthil at uthcode.com Sun Jul 31 03:26:33 2011 From: senthil at uthcode.com (Senthil Kumaran) Date: Sun, 31 Jul 2011 09:26:33 +0800 Subject: [Python-Dev] [Python-checkins] cpython: Fix closes Issue11281 - smtplib.STMP gets source_address parameter, which adds In-Reply-To: <4E34655C.1030201@gmail.com> References: <4E34655C.1030201@gmail.com> Message-ID: <20110731012633.GA2394@mathmagic> On Sat, Jul 30, 2011 at 11:11:08PM +0300, Ezio Melotti wrote: > >-.. class:: SMTP(host='', port=0, local_hostname=None[, timeout]) > >+.. class:: SMTP(host='', port=0, local_hostname=None[, timeout], source_address=None) > > The "[, timeout]" now looks weird there, and it would be better to > convert it to ", timeout=..." to match the other args. > However I don't know what the value should be, since the real value > is socket._GLOBAL_DEFAULT_TIMEOUT (i.e. object()) and I don't think > it's a good idea to expose that. Maybe "None" can be used instead? I found that [, timeout] bit odd too. But is not mentioning that as timeout=None when it is timeout=socket._GLOBAL_DEFAULT_TIME technically inaccurate? FWIW, I see similar style (...,[,timeout], kw=val) adopted elsewhere in the library docs too. urllib, httplib, nntplib etc. Though it does not look okay, it is better than giving inaccurate information. While ftplib and poplib, has them as timeout=None, while they default to socket._GLOBAL_DEFAULT_TIMEOUT object. If we decide upon something, it can be made consistent. So, the question is, when the timeout argument refers to socket._GLOBAL_DEFAULT_TIME, how should we write the docs. 1. def somesocketmethod(arg1,arg2, timeout=socket._GLOBAL_DEFAULT_TIMEOUT, ...) - I don't see anything is wrong with this. 2. def somesocketmethod(arg1,arg2, timeout=None, ...) - And explain that it actually points to a socket default timeout object, which is odd and can lead to user errors. 3. def somesocketmethod(arg1,arg2[,timeout], kwarg=value) - That's how it is currently explained at some places. If this style is okay, we can change the places were it refers to None to be above. Thanks for your review comments, I have address the remaining ones. -- Senthil From g.brandl at gmx.net Sun Jul 31 08:28:24 2011 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 31 Jul 2011 08:28:24 +0200 Subject: [Python-Dev] cpython (2.7): note Ellipsis syntax In-Reply-To: References: Message-ID: On 07/30/11 19:25, Benjamin Peterson wrote: > 2011/7/30 Georg Brandl : >> On 07/30/11 17:00, benjamin.peterson wrote: >>> http://hg.python.org/cpython/rev/402f94edf11b >>> changeset: 71637:402f94edf11b >>> branch: 2.7 >>> user: Benjamin Peterson >>> date: Sat Jul 30 09:59:12 2011 -0500 >>> summary: >>> note Ellipsis syntax >>> >>> files: >>> Doc/library/stdtypes.rst | 2 +- >>> 1 files changed, 1 insertions(+), 1 deletions(-) >>> >>> >>> diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst >>> --- a/Doc/library/stdtypes.rst >>> +++ b/Doc/library/stdtypes.rst >>> @@ -2930,7 +2930,7 @@ >>> supports no special operations. There is exactly one ellipsis object, named >>> :const:`Ellipsis` (a built-in name). >>> >>> -It is written as ``Ellipsis``. >>> +It is written as ``Ellipsis`` or ``...``. >> >> In 2.7, this is not true; ``...`` only works in slices there. > > I know, but why would you use Ellipsis outside of slices? I wouldn't, but that's not the point: the wording as it is now will lead readers to think that they can use the Ellipsis singleton as in Python 3, and they will complain and report bugs about this. (Also, there must have been some reason to make "..." available everywhere for Python 3.) Georg From raymond.hettinger at gmail.com Sun Jul 31 08:47:36 2011 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Sat, 30 Jul 2011 23:47:36 -0700 Subject: [Python-Dev] cpython (2.7): note Ellipsis syntax In-Reply-To: References: Message-ID: On Jul 30, 2011, at 11:28 PM, Georg Brandl wrote: > > (Also, there must have been some reason to make "..." available everywhere > for Python 3.) > It's really nice for stub functions: def foo(x): ... Raymond From ncoghlan at gmail.com Sun Jul 31 10:15:25 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 31 Jul 2011 18:15:25 +1000 Subject: [Python-Dev] cpython (2.7): note Ellipsis syntax In-Reply-To: References: Message-ID: On Sun, Jul 31, 2011 at 4:28 PM, Georg Brandl wrote: > (Also, there must have been some reason to make "..." available everywhere > for Python 3.) Not really - it just let us ditch some special casing in the compilation toolchain that *restricted* it to being used in subscripts (i.e. we were looking at the question from the "is there a good rationale for keeping this arbitrary restriction?" angle). Functionality wise, you could already write 'Ellipsis' everywhere you would otherwise have written '...' and you still have to write ':' as 'slice(None)' outside the context of a subscript operation. Although, as Raymond notes, it can make a nice substitute for 'pass' as a placeholder statement, and can also be used as a placeholder expression. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Sun Jul 31 10:26:42 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 31 Jul 2011 18:26:42 +1000 Subject: [Python-Dev] [Python-checkins] cpython: Issue #11651: Improve Makefile test targets. In-Reply-To: References: Message-ID: On Sun, Jul 31, 2011 at 9:09 AM, nadeem.vawda wrote: > http://hg.python.org/cpython/rev/b76684d62a8d > changeset: ? 71645:b76684d62a8d > user: ? ? ? ?Nadeem Vawda > date: ? ? ? ?Sun Jul 31 01:09:04 2011 +0200 > summary: > ?Issue #11651: Improve Makefile test targets. > > - Use -j0 option by default > - Remove duplicate test run in "make test" and "make testuniversal" That seems very questionable - the rationale for running the test suite twice by default to ensure PYC generation is working correctly still holds. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From nadeem.vawda at gmail.com Sun Jul 31 11:52:32 2011 From: nadeem.vawda at gmail.com (Nadeem Vawda) Date: Sun, 31 Jul 2011 11:52:32 +0200 Subject: [Python-Dev] [Python-checkins] cpython: Issue #11651: Improve Makefile test targets. In-Reply-To: References: Message-ID: On Sun, Jul 31, 2011 at 10:26 AM, Nick Coghlan wrote: > On Sun, Jul 31, 2011 at 9:09 AM, nadeem.vawda > wrote: >> - Remove duplicate test run in "make test" and "make testuniversal" > > That seems very questionable - the rationale for running the test > suite twice by default to ensure PYC generation is working correctly > still holds. The consensus on the tracker was that it isn't worth doubling the time taken to run "make test" to check for a class of bug that seems to be relatively rare. For changes that touch anything as far-reaching as .pyc generation, you should be using "make testall" anyway (and that does still use two passes). Cheers, Nadeem From barry at python.org Sun Jul 31 12:55:30 2011 From: barry at python.org (Barry Warsaw) Date: Sun, 31 Jul 2011 06:55:30 -0400 Subject: [Python-Dev] [Python-checkins] cpython: Issue #11651: Improve Makefile test targets. In-Reply-To: References: Message-ID: <20110731065530.71962597@resist.wooz.org> On Jul 31, 2011, at 06:26 PM, Nick Coghlan wrote: >That seems very questionable - the rationale for running the test >suite twice by default to ensure PYC generation is working correctly >still holds. Agreed. I'd at least like to have seen discussion on python-dev instead of just in the tracker. FWIW, I wasn't even aware of this issue. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Sun Jul 31 12:58:19 2011 From: barry at python.org (Barry Warsaw) Date: Sun, 31 Jul 2011 06:58:19 -0400 Subject: [Python-Dev] [Python-checkins] cpython: Issue #11651: Improve Makefile test targets. In-Reply-To: <20110731065530.71962597@resist.wooz.org> References: <20110731065530.71962597@resist.wooz.org> Message-ID: <20110731065819.1fd0cb2e@resist.wooz.org> On Jul 31, 2011, at 06:55 AM, Barry Warsaw wrote: >On Jul 31, 2011, at 06:26 PM, Nick Coghlan wrote: > >>That seems very questionable - the rationale for running the test >>suite twice by default to ensure PYC generation is working correctly >>still holds. > >Agreed. I'd at least like to have seen discussion on python-dev instead of >just in the tracker. FWIW, I wasn't even aware of this issue. Er, nm. I need coffee. -B -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From solipsis at pitrou.net Sun Jul 31 15:07:06 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 31 Jul 2011 15:07:06 +0200 Subject: [Python-Dev] cpython (2.7): note Ellipsis syntax References: Message-ID: <20110731150706.29ffd099@pitrou.net> On Sat, 30 Jul 2011 23:47:36 -0700 Raymond Hettinger wrote: > > > > (Also, there must have been some reason to make "..." available everywhere > > for Python 3.) > > > > It's really nice for stub functions: > > def foo(x): > ... Using a docstring looks a lot less hackish (and it encourages you to write a doc!): def foo(x): """Some stub function.""" From solipsis at pitrou.net Sun Jul 31 15:10:43 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 31 Jul 2011 15:10:43 +0200 Subject: [Python-Dev] cpython: Issue #11651: Improve Makefile test targets. References: Message-ID: <20110731151043.042638e6@pitrou.net> On Sun, 31 Jul 2011 18:26:42 +1000 Nick Coghlan wrote: > On Sun, Jul 31, 2011 at 9:09 AM, nadeem.vawda > wrote: > > http://hg.python.org/cpython/rev/b76684d62a8d > > changeset: ? 71645:b76684d62a8d > > user: ? ? ? ?Nadeem Vawda > > date: ? ? ? ?Sun Jul 31 01:09:04 2011 +0200 > > summary: > > ?Issue #11651: Improve Makefile test targets. > > > > - Use -j0 option by default > > - Remove duplicate test run in "make test" and "make testuniversal" > > That seems very questionable - the rationale for running the test > suite twice by default to ensure PYC generation is working correctly > still holds. But nobody did it anyway, even the buildbots. Regards Antoine. From fuzzyman at voidspace.org.uk Sun Jul 31 18:17:00 2011 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sun, 31 Jul 2011 17:17:00 +0100 Subject: [Python-Dev] [Python-checkins] cpython: Fix closes Issue11281 - smtplib.STMP gets source_address parameter, which adds In-Reply-To: <20110731012633.GA2394@mathmagic> References: <4E34655C.1030201@gmail.com> <20110731012633.GA2394@mathmagic> Message-ID: On 31 Jul 2011, at 02:26, Senthil Kumaran wrote: > On Sat, Jul 30, 2011 at 11:11:08PM +0300, Ezio Melotti wrote: >>> -.. class:: SMTP(host='', port=0, local_hostname=None[, timeout]) >>> +.. class:: SMTP(host='', port=0, local_hostname=None[, timeout], source_address=None) >> >> The "[, timeout]" now looks weird there, and it would be better to >> convert it to ", timeout=..." to match the other args. >> However I don't know what the value should be, since the real value >> is socket._GLOBAL_DEFAULT_TIMEOUT (i.e. object()) and I don't think >> it's a good idea to expose that. Maybe "None" can be used instead? > > I found that [, timeout] bit odd too. But is not mentioning that as > timeout=None when it is timeout=socket._GLOBAL_DEFAULT_TIME > technically inaccurate? > It does mean that users will expect to be able to call with an explicit timeout=None and get the default behaviour. Just use the numeric value of the global timeout perhaps? MIchael Foord > FWIW, I see similar style (...,[,timeout], kw=val) adopted elsewhere > in the library docs too. urllib, httplib, nntplib etc. Though it does > not look okay, it is better than giving inaccurate information. > > While ftplib and poplib, has them as timeout=None, while they default > to socket._GLOBAL_DEFAULT_TIMEOUT object. > > If we decide upon something, it can be made consistent. So, the > question is, when the timeout argument refers to > socket._GLOBAL_DEFAULT_TIME, how should we write the docs. > > 1. def somesocketmethod(arg1,arg2, timeout=socket._GLOBAL_DEFAULT_TIMEOUT, ...) > > - I don't see anything is wrong with this. > > 2. def somesocketmethod(arg1,arg2, timeout=None, ...) > > - And explain that it actually points to a socket default timeout > object, which is odd and can lead to user errors. > > 3. def somesocketmethod(arg1,arg2[,timeout], kwarg=value) > > - That's how it is currently explained at some places. If this style > is okay, we can change the places were it refers to None to be > above. > > Thanks for your review comments, I have address the remaining ones. -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html From solipsis at pitrou.net Sun Jul 31 18:41:49 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 31 Jul 2011 18:41:49 +0200 Subject: [Python-Dev] [Python-checkins] cpython: Fix closes Issue11281 - smtplib.STMP gets source_address parameter, which adds References: <4E34655C.1030201@gmail.com> <20110731012633.GA2394@mathmagic> Message-ID: <20110731184149.0a3c7266@pitrou.net> On Sun, 31 Jul 2011 17:17:00 +0100 Michael Foord wrote: > > I found that [, timeout] bit odd too. But is not mentioning that as > > timeout=None when it is timeout=socket._GLOBAL_DEFAULT_TIME > > technically inaccurate? > > > > It does mean that users will expect to be able to call with an explicit timeout=None and get the default behaviour. Just use the numeric value of the global timeout perhaps? The global timeout is controllable at runtime through socket.setdefaulttimeout(). That's the whole point of using an opaque sentinel. Regards Antoine.